2024-06-02T22:49:11.532 INFO:root:teuthology version: 0.0.1.dev303+g861a8dc 2024-06-02T22:49:11.532 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi --archive /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637 --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/orig.config.yaml 2024-06-02T22:49:11.583 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-02T22:49:11.669 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7738637' kernel: kdb: 1 sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba subvols: create: 2 subvol_options: --namespace-isolated ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported install: ceph: flavor: default sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba kclient: client.0: mount_subvol_num: 1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 2064 sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba sleep_before_teardown: 0 subset: 125/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi052.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGSrtxy15/UpqwMTEzpkLZO10xV3f0gU970HUyaiPEdU4ygI8XNYB0vh3pfwZ4hrZKuXcSGW93DSFaoxgw/8hnY= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3q1FWB3w6MkfpwIINRXrs4rD8Fb4T14AdUV7+kjkoOdgJu8AwGZ78qDTndcnsCkOuQLgZYT4Uf44q4dxbG7ok= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKzN0q3BylydWk6F5CdE7bHOc+T/vg/3nJVit2MeJzjlpVDcehvrpG2/vBZO4FuScWOxICxjvQYWE0hnSMoiaVU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - ceph-fuse: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 861a8dcf7aa816a26e13f039336f7ed0a9aec0fa timestamp: 2024-06-02_21:24:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2698841 2024-06-02T22:49:11.670 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa; will attempt to use it 2024-06-02T22:49:11.670 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks 2024-06-02T22:49:11.670 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-06-02T22:49:11.672 INFO:teuthology.task.internal:Checking packages... 2024-06-02T22:49:11.692 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '3d89accfe3201286f000e065a1faea2a88c7a6ba' 2024-06-02T22:49:11.692 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-06-02T22:49:11.692 INFO:teuthology.packaging:ref: None 2024-06-02T22:49:11.692 INFO:teuthology.packaging:tag: None 2024-06-02T22:49:11.692 INFO:teuthology.packaging:branch: squid 2024-06-02T22:49:11.692 INFO:teuthology.packaging:sha1: 3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:49:11.692 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-06-02T22:49:11.883 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2952.gdfcc4b3c 2024-06-02T22:49:11.884 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-06-02T22:49:11.891 INFO:teuthology.task.internal:no buildpackages task found 2024-06-02T22:49:11.891 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-06-02T22:49:11.930 INFO:teuthology.task.internal:Saving configuration 2024-06-02T22:49:11.949 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-06-02T22:49:11.956 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-06-02T22:49:11.980 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi052.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-02 22:44:10.682517', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGSrtxy15/UpqwMTEzpkLZO10xV3f0gU970HUyaiPEdU4ygI8XNYB0vh3pfwZ4hrZKuXcSGW93DSFaoxgw/8hnY='} 2024-06-02T22:49:12.000 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-02 22:44:10.683922', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3q1FWB3w6MkfpwIINRXrs4rD8Fb4T14AdUV7+kjkoOdgJu8AwGZ78qDTndcnsCkOuQLgZYT4Uf44q4dxbG7ok='} 2024-06-02T22:49:12.021 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-06-02 22:44:10.680743', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKzN0q3BylydWk6F5CdE7bHOc+T/vg/3nJVit2MeJzjlpVDcehvrpG2/vBZO4FuScWOxICxjvQYWE0hnSMoiaVU='} 2024-06-02T22:49:12.021 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-06-02T22:49:12.028 INFO:teuthology.task.internal:roles: ubuntu@smithi052.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-06-02T22:49:12.028 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-06-02T22:49:12.028 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-06-02T22:49:12.028 INFO:teuthology.run_tasks:Running task console_log... 2024-06-02T22:49:12.120 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f121b2afbe0>, signals=[15]) 2024-06-02T22:49:12.120 INFO:teuthology.run_tasks:Running task internal.connect... 2024-06-02T22:49:12.126 INFO:teuthology.task.internal:Opening connections... 2024-06-02T22:49:12.127 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T22:49:12.128 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:49:12.207 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-06-02T22:49:12.208 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:49:12.281 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-06-02T22:49:12.282 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:49:12.359 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-06-02T22:49:12.365 DEBUG:teuthology.orchestra.run.smithi052:> uname -m 2024-06-02T22:49:12.384 INFO:teuthology.orchestra.run.smithi052.stdout:x86_64 2024-06-02T22:49:12.384 DEBUG:teuthology.orchestra.run.smithi052:> cat /etc/os-release 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:NAME="CentOS Stream" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION="9" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:ID="centos" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:ID_LIKE="rhel fedora" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:VERSION_ID="9" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:PLATFORM_ID="platform:el9" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:ANSI_COLOR="0;31" 2024-06-02T22:49:12.441 INFO:teuthology.orchestra.run.smithi052.stdout:LOGO="fedora-logo-icon" 2024-06-02T22:49:12.442 INFO:teuthology.orchestra.run.smithi052.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-02T22:49:12.442 INFO:teuthology.orchestra.run.smithi052.stdout:HOME_URL="https://centos.org/" 2024-06-02T22:49:12.442 INFO:teuthology.orchestra.run.smithi052.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-02T22:49:12.442 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-02T22:49:12.442 INFO:teuthology.orchestra.run.smithi052.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-02T22:49:12.442 INFO:teuthology.lock.ops:Updating smithi052.front.sepia.ceph.com on lock server 2024-06-02T22:49:12.465 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-06-02T22:49:12.482 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-06-02T22:49:12.483 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-06-02T22:49:12.540 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-02T22:49:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-02T22:49:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-02T22:49:12.541 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-06-02T22:49:12.565 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-06-02T22:49:12.582 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-06-02T22:49:12.583 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-06-02T22:49:12.639 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-06-02T22:49:12.640 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-06-02T22:49:12.640 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-06-02T22:49:12.640 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-06-02T22:49:12.640 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-06-02T22:49:12.640 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-06-02T22:49:12.664 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-06-02T22:49:12.674 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-06-02T22:49:12.705 INFO:teuthology.task.internal:Checking for old test directory... 2024-06-02T22:49:12.705 DEBUG:teuthology.orchestra.run.smithi052:> test '!' -e /home/ubuntu/cephtest 2024-06-02T22:49:12.707 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-06-02T22:49:12.710 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-06-02T22:49:12.725 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-06-02T22:49:12.738 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-06-02T22:49:12.738 DEBUG:teuthology.orchestra.run.smithi052:> test -z $(ls -A /var/lib/ceph) 2024-06-02T22:49:12.765 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-06-02T22:49:12.767 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-06-02T22:49:12.792 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-06-02T22:49:12.866 INFO:teuthology.run_tasks:Running task kernel... 2024-06-02T22:49:12.880 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-06-02T22:49:12.880 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-06-02T22:49:12.881 DEBUG:teuthology.orchestra.run.smithi052:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-02T22:49:12.882 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-02T22:49:12.882 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-06-02T22:49:12.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T22:49:12.898 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-06-02T22:49:12.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T22:49:12.899 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-06-02T22:49:12.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T22:49:12.900 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-06-02T22:49:12.954 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-437.el9.x86_64 2024-06-02T22:49:12.954 INFO:teuthology.task.kernel:Running kernel on smithi174: 5.14.0-437.el9.x86_64 2024-06-02T22:49:12.955 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum install -y kernel 2024-06-02T22:49:12.956 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-437.el9.x86_64 2024-06-02T22:49:12.957 INFO:teuthology.task.kernel:Running kernel on smithi123: 5.14.0-437.el9.x86_64 2024-06-02T22:49:12.957 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2024-06-02T22:49:12.959 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-437.el9.x86_64 2024-06-02T22:49:12.959 INFO:teuthology.task.kernel:Running kernel on smithi052: 5.14.0-437.el9.x86_64 2024-06-02T22:49:12.959 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum install -y kernel 2024-06-02T22:49:15.248 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-06-02T22:49:15.324 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.1 MB 00:00 2024-06-02T22:49:17.033 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 32 MB/s | 19 MB 00:00 2024-06-02T22:49:17.057 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 27 MB/s | 19 MB 00:00 2024-06-02T22:49:19.146 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.1 MB 00:04 2024-06-02T22:49:20.880 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 33 MB/s | 19 MB 00:00 2024-06-02T22:49:21.283 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.2 MB 00:00 2024-06-02T22:49:21.800 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 5.2 MB/s | 6.2 MB 00:01 2024-06-02T22:49:22.751 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 17 kB 00:00 2024-06-02T22:49:23.333 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 83 kB/s | 17 kB 00:00 2024-06-02T22:49:24.088 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-06-02T22:49:24.779 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-02T22:49:25.048 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-06-02T22:49:26.540 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 17 kB 00:00 2024-06-02T22:49:27.892 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 18 MB/s | 21 MB 00:01 2024-06-02T22:49:29.335 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-06-02T22:49:30.043 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-06-02T22:49:31.308 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-02T22:49:31.308 INFO:teuthology.orchestra.run.smithi174.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-02T22:49:31.398 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-06-02T22:49:31.405 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-06-02T22:49:31.405 INFO:teuthology.orchestra.run.smithi174.stdout: Package Architecture Version Repository Size 2024-06-02T22:49:31.405 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-06-02T22:49:31.405 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-06-02T22:49:31.405 INFO:teuthology.orchestra.run.smithi174.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout:================================================================================ 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout:Install 4 Packages 2024-06-02T22:49:31.406 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:49:31.407 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 80 M 2024-06-02T22:49:31.407 INFO:teuthology.orchestra.run.smithi174.stdout:Installed size: 124 M 2024-06-02T22:49:31.407 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-06-02T22:49:31.845 INFO:teuthology.orchestra.run.smithi174.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 2.9 MB/s | 893 kB 00:00 2024-06-02T22:49:32.057 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-02T22:49:32.057 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-02T22:49:32.155 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:49:32.164 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-06-02T22:49:32.165 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2024-06-02T22:49:32.165 INFO:teuthology.orchestra.run.smithi123.stdout:Install 4 Packages 2024-06-02T22:49:32.165 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:49:32.165 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 80 M 2024-06-02T22:49:32.165 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 124 M 2024-06-02T22:49:32.165 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-06-02T22:49:32.344 INFO:teuthology.orchestra.run.smithi123.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 6.2 MB/s | 893 kB 00:00 2024-06-02T22:49:32.619 INFO:teuthology.orchestra.run.smithi123.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 38 MB/s | 16 MB 00:00 2024-06-02T22:49:32.970 INFO:teuthology.orchestra.run.smithi123.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 46 MB/s | 29 MB 00:00 2024-06-02T22:49:33.192 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-06-02T22:49:33.270 INFO:teuthology.orchestra.run.smithi123.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 32 MB/s | 35 MB 00:01 2024-06-02T22:49:33.270 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:49:33.270 INFO:teuthology.orchestra.run.smithi123.stdout:Total 73 MB/s | 80 MB 00:01 2024-06-02T22:49:33.570 INFO:teuthology.orchestra.run.smithi174.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 7.9 MB/s | 16 MB 00:02 2024-06-02T22:49:33.966 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-06-02T22:49:34.292 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-06-02T22:49:34.292 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-06-02T22:49:35.240 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-06-02T22:49:35.240 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-06-02T22:49:35.290 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-06-02T22:49:35.290 INFO:teuthology.orchestra.run.smithi052.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-06-02T22:49:35.396 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout: Package Architecture Version Repository Size 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout: kernel x86_64 5.14.0-452.el9 baseos 893 k 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core x86_64 5.14.0-452.el9 baseos 16 M 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules x86_64 5.14.0-452.el9 baseos 35 M 2024-06-02T22:49:35.407 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core x86_64 5.14.0-452.el9 baseos 29 M 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout:================================================================================ 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout:Install 4 Packages 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 80 M 2024-06-02T22:49:35.408 INFO:teuthology.orchestra.run.smithi052.stdout:Installed size: 124 M 2024-06-02T22:49:35.409 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-06-02T22:49:35.520 INFO:teuthology.orchestra.run.smithi174.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 7.9 MB/s | 29 MB 00:03 2024-06-02T22:49:35.718 INFO:teuthology.orchestra.run.smithi052.stdout:(1/4): kernel-5.14.0-452.el9.x86_64.rpm 3.2 MB/s | 893 kB 00:00 2024-06-02T22:49:35.820 INFO:teuthology.orchestra.run.smithi174.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 8.1 MB/s | 35 MB 00:04 2024-06-02T22:49:35.821 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:49:35.821 INFO:teuthology.orchestra.run.smithi174.stdout:Total 18 MB/s | 80 MB 00:04 2024-06-02T22:49:36.227 INFO:teuthology.orchestra.run.smithi052.stdout:(2/4): kernel-core-5.14.0-452.el9.x86_64.rpm 20 MB/s | 16 MB 00:00 2024-06-02T22:49:36.514 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-06-02T22:49:36.849 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-06-02T22:49:36.849 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-06-02T22:49:37.035 INFO:teuthology.orchestra.run.smithi052.stdout:(3/4): kernel-modules-core-5.14.0-452.el9.x86_6 22 MB/s | 29 MB 00:01 2024-06-02T22:49:37.344 INFO:teuthology.orchestra.run.smithi052.stdout:(4/4): kernel-modules-5.14.0-452.el9.x86_64.rpm 18 MB/s | 35 MB 00:01 2024-06-02T22:49:37.344 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:49:37.345 INFO:teuthology.orchestra.run.smithi052.stdout:Total 41 MB/s | 80 MB 00:01 2024-06-02T22:49:37.629 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-06-02T22:49:37.782 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-06-02T22:49:37.783 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-06-02T22:49:38.073 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-06-02T22:49:38.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-02T22:49:38.420 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:49:38.497 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-06-02T22:49:38.497 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-06-02T22:49:39.476 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-06-02T22:49:39.477 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-06-02T22:49:40.167 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-06-02T22:49:40.792 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-02T22:49:40.878 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:49:41.084 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:49:41.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:49:42.001 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-06-02T22:49:42.666 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-core-5.14.0-452.el9.x86_64 1/4 2024-06-02T22:49:42.761 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:49:43.529 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:49:43.670 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:49:45.530 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:49:45.665 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:49:46.111 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:49:46.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:49:48.368 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:49:48.425 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:49:50.756 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:49:50.813 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : kernel-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:49:50.985 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:49:53.207 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:49:55.932 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:20.296 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:20.336 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:20.701 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:20.701 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-02T22:50:20.701 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:50:20.701 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-02T22:50:21.484 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-02T22:50:21.485 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:50:21.485 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-06-02T22:50:21.879 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-06-02T22:50:21.952 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:21.952 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-02T22:50:21.952 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-02T22:50:21.952 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-02T22:50:21.952 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-02T22:50:21.953 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-06-02T22:50:21.953 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2024-06-02T22:50:21.953 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2024-06-02T22:50:22.004 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:22.004 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-02T22:50:22.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2024-06-02T22:50:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2024-06-02T22:50:22.047 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.06 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 68.el9 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Group : Unspecified 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 8264254 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : builder@centos.org 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2024-06-02T22:50:22.048 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:50:22.049 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2024-06-02T22:50:22.049 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-02T22:50:22.049 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 5.14.0-452.el9.x86_64 2024-06-02T22:50:22.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-02T22:50:22.633 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2024-06-02T22:50:22.690 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:22.731 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:22.993 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:22.993 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-02T22:50:22.993 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:50:22.993 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:50:23.786 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:50:23.787 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-06-02T22:50:24.013 INFO:teuthology.orchestra.run.smithi123.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-02T22:50:24.043 INFO:teuthology.orchestra.run.smithi123.stderr:done 2024-06-02T22:50:24.045 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2024-06-02T22:50:24.059 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.HoTo79iVr0 2024-06-02T22:50:24.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.HoTo79iVr0 2024-06-02T22:50:24.124 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.HoTo79iVr0 2024-06-02T22:50:24.131 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-06-02T22:50:24.220 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:24.221 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-02T22:50:24.221 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-02T22:50:24.221 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-02T22:50:24.221 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-02T22:50:24.221 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-06-02T22:50:24.221 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi174.front.sepia.ceph.com, path=None, version=distro) 2024-06-02T22:50:24.221 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q kernel | sort -rV | head -n 1 2024-06-02T22:50:24.257 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.HoTo79iVr0 is 6KB 2024-06-02T22:50:24.272 INFO:teuthology.orchestra.run.smithi174.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:24.273 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-02T22:50:24.274 DEBUG:teuthology.orchestra.run.smithi174:> sudo rpm -qi grub2-tools 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Name : grub2-tools 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Epoch : 1 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Version : 2.06 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Release : 68.el9 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Architecture: x86_64 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Group : Unspecified 2024-06-02T22:50:24.304 INFO:teuthology.orchestra.run.smithi174.stdout:Size : 8264254 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:License : GPLv3+ 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Packager : builder@centos.org 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Vendor : CentOS 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Summary : Support tools for GRUB. 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:Description : 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-02T22:50:24.305 INFO:teuthology.orchestra.run.smithi174.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-02T22:50:24.306 INFO:teuthology.orchestra.run.smithi174.stdout:hardware devices. 2024-06-02T22:50:24.306 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:50:24.306 INFO:teuthology.orchestra.run.smithi174.stdout:This subpackage provides tools for support of all platforms. 2024-06-02T22:50:24.307 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-02T22:50:24.307 INFO:teuthology.task.kernel:Updating grub on smithi174 to boot 5.14.0-452.el9.x86_64 2024-06-02T22:50:24.307 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-02T22:50:24.308 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.HoTo79iVr0 2024-06-02T22:50:24.322 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2024-06-02T22:50:24.383 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-02T22:50:24.383 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-02T22:50:24.383 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-02T22:50:24.383 INFO:teuthology.orchestra.run.smithi123.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-02T22:50:24.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-02T22:50:24.543 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2024-06-02T22:50:24.911 INFO:teuthology.orchestra.run.smithi174.stderr:Generating grub configuration file ... 2024-06-02T22:50:26.301 INFO:teuthology.orchestra.run.smithi174.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-02T22:50:26.348 INFO:teuthology.orchestra.run.smithi174.stderr:done 2024-06-02T22:50:26.354 DEBUG:teuthology.orchestra.run.smithi174:> mktemp 2024-06-02T22:50:26.369 INFO:teuthology.orchestra.run.smithi174.stdout:/tmp/tmp.Uj4NeeCclq 2024-06-02T22:50:26.369 DEBUG:teuthology.orchestra.run.smithi174:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Uj4NeeCclq 2024-06-02T22:50:26.434 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 0666 /tmp/tmp.Uj4NeeCclq 2024-06-02T22:50:26.568 DEBUG:teuthology.orchestra.remote:smithi174:/tmp/tmp.Uj4NeeCclq is 6KB 2024-06-02T22:50:26.616 DEBUG:teuthology.orchestra.run.smithi174:> rm -fr /tmp/tmp.Uj4NeeCclq 2024-06-02T22:50:26.631 DEBUG:teuthology.orchestra.run.smithi174:> sudo /bin/ls /boot/loader/entries || true 2024-06-02T22:50:26.695 INFO:teuthology.orchestra.run.smithi174.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-02T22:50:26.696 INFO:teuthology.orchestra.run.smithi174.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-02T22:50:26.696 INFO:teuthology.orchestra.run.smithi174.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-02T22:50:26.696 INFO:teuthology.orchestra.run.smithi174.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-02T22:50:26.697 DEBUG:teuthology.orchestra.run.smithi174:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-02T22:50:26.875 DEBUG:teuthology.orchestra.run.smithi174:> sudo shutdown -r now 2024-06-02T22:50:27.007 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:27.049 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-modules-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:27.416 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: kernel-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:27.416 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-5.14.0-452.el9.x86_64 1/4 2024-06-02T22:50:27.416 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-core-5.14.0-452.el9.x86_64 2/4 2024-06-02T22:50:27.416 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-5.14.0-452.el9.x86_64 3/4 2024-06-02T22:50:28.281 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : kernel-modules-core-5.14.0-452.el9.x86_64 4/4 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-core-5.14.0-452.el9.x86_64 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-5.14.0-452.el9.x86_64 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout: kernel-modules-core-5.14.0-452.el9.x86_64 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:50:28.282 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-06-02T22:50:28.620 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-06-02T22:50:28.737 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:28.737 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-02T22:50:28.737 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-452.el9.x86_64 2024-06-02T22:50:28.737 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-06-02T22:50:28.737 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-06-02T22:50:28.737 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-06-02T22:50:28.737 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi052.front.sepia.ceph.com, path=None, version=distro) 2024-06-02T22:50:28.737 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q kernel | sort -rV | head -n 1 2024-06-02T22:50:28.789 INFO:teuthology.orchestra.run.smithi052.stdout:kernel-5.14.0-452.el9.x86_64 2024-06-02T22:50:28.790 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-452.el9.x86_64 2024-06-02T22:50:28.790 DEBUG:teuthology.orchestra.run.smithi052:> sudo rpm -qi grub2-tools 2024-06-02T22:50:28.830 INFO:teuthology.orchestra.run.smithi052.stdout:Name : grub2-tools 2024-06-02T22:50:28.830 INFO:teuthology.orchestra.run.smithi052.stdout:Epoch : 1 2024-06-02T22:50:28.830 INFO:teuthology.orchestra.run.smithi052.stdout:Version : 2.06 2024-06-02T22:50:28.830 INFO:teuthology.orchestra.run.smithi052.stdout:Release : 68.el9 2024-06-02T22:50:28.830 INFO:teuthology.orchestra.run.smithi052.stdout:Architecture: x86_64 2024-06-02T22:50:28.830 INFO:teuthology.orchestra.run.smithi052.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Group : Unspecified 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Size : 8264254 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:License : GPLv3+ 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Packager : builder@centos.org 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Vendor : CentOS 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:URL : http://www.gnu.org/software/grub/ 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Summary : Support tools for GRUB. 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:Description : 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:50:28.831 INFO:teuthology.orchestra.run.smithi052.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-06-02T22:50:28.832 INFO:teuthology.orchestra.run.smithi052.stdout:customizable bootloader with modular architecture. It supports a rich 2024-06-02T22:50:28.832 INFO:teuthology.orchestra.run.smithi052.stdout:variety of kernel formats, file systems, computer architectures and 2024-06-02T22:50:28.832 INFO:teuthology.orchestra.run.smithi052.stdout:hardware devices. 2024-06-02T22:50:28.832 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:50:28.832 INFO:teuthology.orchestra.run.smithi052.stdout:This subpackage provides tools for support of all platforms. 2024-06-02T22:50:28.833 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-06-02T22:50:28.833 INFO:teuthology.task.kernel:Updating grub on smithi052 to boot 5.14.0-452.el9.x86_64 2024-06-02T22:50:28.833 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-06-02T22:50:29.481 INFO:teuthology.orchestra.run.smithi052.stderr:Generating grub configuration file ... 2024-06-02T22:50:31.117 INFO:teuthology.orchestra.run.smithi052.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-06-02T22:50:31.162 INFO:teuthology.orchestra.run.smithi052.stderr:done 2024-06-02T22:50:31.164 DEBUG:teuthology.orchestra.run.smithi052:> mktemp 2024-06-02T22:50:31.179 INFO:teuthology.orchestra.run.smithi052.stdout:/tmp/tmp.mbED7xinH6 2024-06-02T22:50:31.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.mbED7xinH6 2024-06-02T22:50:31.267 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0666 /tmp/tmp.mbED7xinH6 2024-06-02T22:50:31.365 DEBUG:teuthology.orchestra.remote:smithi052:/tmp/tmp.mbED7xinH6 is 6KB 2024-06-02T22:50:31.414 DEBUG:teuthology.orchestra.run.smithi052:> rm -fr /tmp/tmp.mbED7xinH6 2024-06-02T22:50:31.428 DEBUG:teuthology.orchestra.run.smithi052:> sudo /bin/ls /boot/loader/entries || true 2024-06-02T22:50:31.501 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-06-02T22:50:31.501 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-06-02T22:50:31.501 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-06-02T22:50:31.501 INFO:teuthology.orchestra.run.smithi052.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64.conf 2024-06-02T22:50:31.502 DEBUG:teuthology.orchestra.run.smithi052:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-452.el9.x86_64 2024-06-02T22:50:31.647 DEBUG:teuthology.orchestra.run.smithi052:> sudo shutdown -r now 2024-06-02T22:50:54.574 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-06-02T22:50:54.575 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-02T22:50:54.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:50:56.880 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-06-02T22:50:56.881 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-06-02T22:50:56.881 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:51:01.655 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-06-02T22:51:01.656 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-02T22:51:01.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:51:54.606 DEBUG:teuthology.orchestra.remote:timed out 2024-06-02T22:51:56.886 DEBUG:teuthology.orchestra.remote:timed out 2024-06-02T22:52:01.662 DEBUG:teuthology.orchestra.remote:timed out 2024-06-02T22:52:03.610 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-02T22:52:03.611 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:52:05.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-06-02T22:52:05.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:52:07.173 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-06-02T22:52:07.368 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-06-02T22:52:07.632 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-02T22:52:07.632 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-452.el9.x86_64"... 2024-06-02T22:52:07.632 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2024-06-02T22:52:07.647 INFO:teuthology.orchestra.run.smithi123.stdout:5.14.0-452.el9.x86_64 2024-06-02T22:52:07.647 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-02T22:52:07.647 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-02T22:52:07.647 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-02T22:52:07.840 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-06-02T22:52:07.840 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-452.el9.x86_64"... 2024-06-02T22:52:07.840 DEBUG:teuthology.orchestra.run.smithi174:> uname -r 2024-06-02T22:52:07.855 INFO:teuthology.orchestra.run.smithi174.stdout:5.14.0-452.el9.x86_64 2024-06-02T22:52:07.855 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-02T22:52:07.856 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-02T22:52:07.856 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-02T22:52:08.649 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-06-02T22:52:08.649 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-02T22:52:08.725 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2024-06-02T22:52:08.767 DEBUG:teuthology.parallel:result is None 2024-06-02T22:52:08.856 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-06-02T22:52:08.856 DEBUG:teuthology.orchestra.run.smithi174:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-02T22:52:08.918 INFO:teuthology.orchestra.run.smithi174.stdout:ttyS1 2024-06-02T22:52:08.959 DEBUG:teuthology.parallel:result is None 2024-06-02T22:52:10.663 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-02T22:52:10.664 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:52:11.115 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-06-02T22:52:11.499 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-02T22:52:11.499 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-452.el9.x86_64"... 2024-06-02T22:52:11.500 DEBUG:teuthology.orchestra.run.smithi052:> uname -r 2024-06-02T22:52:11.515 INFO:teuthology.orchestra.run.smithi052.stdout:5.14.0-452.el9.x86_64 2024-06-02T22:52:11.515 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-452.el9.x86_64 vs 5.14.0-452.el9.x86_64 2024-06-02T22:52:11.515 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-06-02T22:52:11.515 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-06-02T22:52:12.518 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-06-02T22:52:12.518 DEBUG:teuthology.orchestra.run.smithi052:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-06-02T22:52:12.599 INFO:teuthology.orchestra.run.smithi052.stdout:ttyS1 2024-06-02T22:52:12.643 DEBUG:teuthology.parallel:result is None 2024-06-02T22:52:12.643 INFO:teuthology.run_tasks:Running task internal.base... 2024-06-02T22:52:12.668 INFO:teuthology.task.internal:Creating test directory... 2024-06-02T22:52:12.669 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-02T22:52:12.671 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-02T22:52:12.673 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-06-02T22:52:12.689 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-06-02T22:52:12.706 INFO:teuthology.run_tasks:Running task internal.archive... 2024-06-02T22:52:12.741 INFO:teuthology.task.internal:Creating archive directory... 2024-06-02T22:52:12.742 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-02T22:52:12.744 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-02T22:52:12.746 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-06-02T22:52:12.803 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-06-02T22:52:12.810 INFO:teuthology.task.internal:Enabling coredump saving... 2024-06-02T22:52:12.810 DEBUG:teuthology.orchestra.run.smithi052:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-02T22:52:12.814 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-02T22:52:12.816 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-06-02T22:52:12.845 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T22:52:12.853 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T22:52:12.855 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T22:52:12.864 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T22:52:13.001 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T22:52:13.012 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-06-02T22:52:13.014 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-06-02T22:52:13.022 INFO:teuthology.task.internal:Configuring sudo... 2024-06-02T22:52:13.022 DEBUG:teuthology.orchestra.run.smithi052:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-02T22:52:13.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-02T22:52:13.026 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-06-02T22:52:13.051 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-06-02T22:52:13.086 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-06-02T22:52:13.086 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-02T22:52:13.097 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-02T22:52:13.099 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-06-02T22:52:13.114 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T22:52:13.199 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T22:52:13.250 DEBUG:teuthology.orchestra.run.smithi052:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T22:52:13.331 DEBUG:teuthology.orchestra.run.smithi052:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T22:52:13.381 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:52:13.381 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-02T22:52:13.450 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T22:52:13.516 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T22:52:13.587 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T22:52:13.667 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T22:52:13.724 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T22:52:13.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-02T22:52:13.783 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T22:52:13.825 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-06-02T22:52:13.959 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T22:52:14.052 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-06-02T22:52:14.117 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T22:52:14.117 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-06-02T22:52:14.170 DEBUG:teuthology.orchestra.run.smithi052:> sudo service rsyslog restart 2024-06-02T22:52:14.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-06-02T22:52:14.174 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-06-02T22:52:14.220 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T22:52:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T22:52:14.397 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T22:52:14.809 INFO:teuthology.run_tasks:Running task internal.timer... 2024-06-02T22:52:14.816 INFO:teuthology.task.internal:Starting timer... 2024-06-02T22:52:14.816 INFO:teuthology.run_tasks:Running task pcp... 2024-06-02T22:52:14.853 INFO:teuthology.run_tasks:Running task selinux... 2024-06-02T22:52:14.885 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-06-02T22:52:14.885 DEBUG:teuthology.orchestra.run.smithi052:> sudo service auditd rotate 2024-06-02T22:52:14.931 INFO:teuthology.orchestra.run.smithi052.stdout:Rotating logs: 2024-06-02T22:52:14.933 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-06-02T22:52:14.994 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-06-02T22:52:14.996 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-06-02T22:52:15.060 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-06-02T22:52:15.063 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-06-02T22:52:15.063 DEBUG:teuthology.orchestra.run.smithi052:> /usr/sbin/getenforce 2024-06-02T22:52:15.083 INFO:teuthology.orchestra.run.smithi052.stdout:Permissive 2024-06-02T22:52:15.083 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-06-02T22:52:15.102 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-06-02T22:52:15.102 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-06-02T22:52:15.124 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-06-02T22:52:15.124 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi052.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-06-02T22:52:15.124 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T22:52:15.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T22:52:15.153 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T22:52:15.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T22:52:15.178 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T22:52:15.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T22:52:15.203 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-06-02T22:52:15.204 DEBUG:teuthology.orchestra.run.smithi052:> sudo /usr/sbin/setenforce permissive 2024-06-02T22:52:15.469 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-06-02T22:52:15.495 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-06-02T22:52:15.626 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-06-02T22:52:15.636 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-06-02T22:52:15.639 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-06-02T22:52:15.787 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-06-02T22:52:15.798 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-06-02T22:52:15.799 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi052.front.sepia.ceph.com,smithi123.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-06-02T22:56:40.078 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi052.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-06-02T22:56:40.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-02T22:56:40.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi052.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:56:40.165 DEBUG:teuthology.orchestra.run.smithi052:> true 2024-06-02T22:56:40.245 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi052.front.sepia.ceph.com' 2024-06-02T22:56:40.245 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-02T22:56:40.246 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:56:40.319 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-06-02T22:56:40.399 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-06-02T22:56:40.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-06-02T22:56:40.401 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-06-02T22:56:40.478 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-06-02T22:56:40.520 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-06-02T22:56:40.521 INFO:teuthology.run_tasks:Running task clock... 2024-06-02T22:56:40.532 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-06-02T22:56:40.533 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-02T22:56:40.533 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T22:56:40.536 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-02T22:56:40.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T22:56:40.539 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-06-02T22:56:40.539 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T22:56:40.569 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-02T22:56:40.570 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-02T22:56:40.585 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-02T22:56:40.586 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-02T22:56:40.590 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-06-02T22:56:40.604 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-06-02T22:56:40.611 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-06-02T22:56:40.614 INFO:teuthology.orchestra.run.smithi052.stderr:sudo: ntpd: command not found 2024-06-02T22:56:40.622 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-06-02T22:56:40.626 INFO:teuthology.orchestra.run.smithi052.stdout:506 Cannot talk to daemon 2024-06-02T22:56:40.629 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-06-02T22:56:40.638 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-02T22:56:40.639 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-06-02T22:56:40.642 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-02T22:56:40.652 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-02T22:56:40.653 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-06-02T22:56:40.657 INFO:teuthology.orchestra.run.smithi052.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-02T22:56:40.668 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-06-02T22:56:40.720 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-06-02T22:56:40.720 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-06-02T22:56:40.721 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-06-02T22:56:40.723 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T22:56:40.724 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-06-02T22:56:40.724 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.724 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.724 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.725 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.727 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T22:56:40.728 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-06-02T22:56:40.728 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.728 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.728 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.728 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.728 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T22:56:40.729 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-06-02T22:56:40.729 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.729 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.729 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.729 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T22:56:40.729 INFO:teuthology.run_tasks:Running task install... 2024-06-02T22:56:40.737 DEBUG:teuthology.task.install:project ceph 2024-06-02T22:56:40.738 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'}} 2024-06-02T22:56:40.738 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba'} 2024-06-02T22:56:40.738 INFO:teuthology.task.install:Using flavor: default 2024-06-02T22:56:40.745 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-06-02T22:56:40.745 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-06-02T22:56:40.745 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-02T22:56:40.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:56:40.748 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-02T22:56:40.748 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:56:40.749 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'tag': None, 'wait_for_package': False} 2024-06-02T22:56:40.749 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:56:40.895 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-02T22:56:40.895 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-02T22:56:40.963 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-02T22:56:40.963 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-02T22:56:40.977 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/ 2024-06-02T22:56:40.977 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2946.g3d89accf 2024-06-02T22:56:41.075 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-02T22:56:41.075 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T22:56:41.076 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-02T22:56:41.078 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-02T22:56:41.078 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:56:41.078 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-02T22:56:41.104 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-02T22:56:41.104 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-02T22:56:41.106 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-02T22:56:41.106 DEBUG:teuthology.orchestra.run.smithi052:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-02T22:56:41.145 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-06-02T22:56:41.145 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T22:56:41.145 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-06-02T22:56:41.173 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-02T22:56:41.174 DEBUG:teuthology.orchestra.run.smithi052:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-02T22:56:41.175 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-06-02T22:56:41.175 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/3d89accfe3201286f000e065a1faea2a88c7a6ba/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-06-02T22:56:41.239 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-06-02T22:56:41.245 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-02T22:56:41.252 DEBUG:teuthology.orchestra.run.smithi052:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-02T22:56:41.309 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-06-02T22:56:41.311 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-06-02T22:56:41.314 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-06-02T22:56:41.319 INFO:teuthology.orchestra.run.smithi052.stdout:check_obsoletes = 1 2024-06-02T22:56:41.320 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum clean all 2024-06-02T22:56:41.378 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-06-02T22:56:41.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-06-02T22:56:41.552 INFO:teuthology.orchestra.run.smithi174.stdout:45 files removed 2024-06-02T22:56:41.561 INFO:teuthology.orchestra.run.smithi052.stdout:45 files removed 2024-06-02T22:56:41.579 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-02T22:56:41.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-02T22:56:41.606 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-06-02T22:56:41.633 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-06-02T22:56:42.237 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 253 kB/s | 83 kB 00:00 2024-06-02T22:56:42.321 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 178 kB/s | 83 kB 00:00 2024-06-02T22:56:42.369 INFO:teuthology.orchestra.run.smithi052.stdout:ceph packages for x86_64 170 kB/s | 83 kB 00:00 2024-06-02T22:56:42.596 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 38 kB/s | 11 kB 00:00 2024-06-02T22:56:42.746 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 31 kB/s | 11 kB 00:00 2024-06-02T22:56:42.820 INFO:teuthology.orchestra.run.smithi052.stdout:ceph noarch packages 29 kB/s | 11 kB 00:00 2024-06-02T22:56:42.947 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 6.1 kB/s | 1.9 kB 00:00 2024-06-02T22:56:43.147 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-06-02T22:56:43.196 INFO:teuthology.orchestra.run.smithi052.stdout:ceph source packages 5.7 kB/s | 1.9 kB 00:00 2024-06-02T22:56:43.539 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 23 MB/s | 8.1 MB 00:00 2024-06-02T22:56:43.655 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.1 MB 00:00 2024-06-02T22:56:44.479 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.1 MB 00:01 2024-06-02T22:56:45.456 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-02T22:56:46.322 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - AppStream 28 MB/s | 19 MB 00:00 2024-06-02T22:56:49.815 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-06-02T22:56:50.514 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 19 MB 00:05 2024-06-02T22:56:50.923 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - CRB 7.2 MB/s | 6.2 MB 00:00 2024-06-02T22:56:51.566 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 17 kB 00:00 2024-06-02T22:56:52.682 INFO:teuthology.orchestra.run.smithi052.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 17 kB 00:00 2024-06-02T22:56:53.212 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-06-02T22:56:54.128 INFO:teuthology.orchestra.run.smithi052.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-06-02T22:56:54.523 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-06-02T22:56:55.941 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 67 kB/s | 17 kB 00:00 2024-06-02T22:56:57.009 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 23 MB/s | 21 MB 00:00 2024-06-02T22:56:58.351 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-06-02T22:56:59.542 INFO:teuthology.orchestra.run.smithi052.stdout:lab-extras 37 kB/s | 1.7 kB 00:00 2024-06-02T22:57:00.702 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T22:57:00.703 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T22:57:00.714 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-02T22:57:00.715 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-02T22:57:00.715 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-02T22:57:00.716 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-02T22:57:00.718 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-02T22:57:00.720 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-02T22:57:00.721 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-02T22:57:00.785 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-06-02T22:57:00.803 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-06-02T22:57:00.803 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-06-02T22:57:00.803 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-02T22:57:00.804 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-02T22:57:00.805 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-02T22:57:00.806 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-02T22:57:00.807 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-02T22:57:00.808 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-02T22:57:00.809 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-02T22:57:00.810 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-02T22:57:00.811 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-02T22:57:00.812 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-02T22:57:00.813 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-02T22:57:00.814 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-02T22:57:00.815 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-02T22:57:00.816 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-02T22:57:00.817 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:00.818 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-06-02T22:57:00.819 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-06-02T22:57:00.819 INFO:teuthology.orchestra.run.smithi123.stdout:Install 148 Packages 2024-06-02T22:57:00.819 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 11 Packages 2024-06-02T22:57:00.819 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:00.819 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 225 M 2024-06-02T22:57:00.819 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-06-02T22:57:01.594 INFO:teuthology.orchestra.run.smithi123.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-06-02T22:57:01.936 INFO:teuthology.orchestra.run.smithi123.stdout:(2/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 9.2 MB/s | 5.4 MB 00:00 2024-06-02T22:57:01.970 INFO:teuthology.orchestra.run.smithi052.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T22:57:01.970 INFO:teuthology.orchestra.run.smithi052.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T22:57:01.982 INFO:teuthology.orchestra.run.smithi052.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-02T22:57:01.982 INFO:teuthology.orchestra.run.smithi052.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-02T22:57:01.983 INFO:teuthology.orchestra.run.smithi052.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-02T22:57:01.984 INFO:teuthology.orchestra.run.smithi052.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-02T22:57:01.985 INFO:teuthology.orchestra.run.smithi052.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-02T22:57:01.988 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-02T22:57:01.989 INFO:teuthology.orchestra.run.smithi052.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-02T22:57:02.003 INFO:teuthology.orchestra.run.smithi123.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 2.1 MB/s | 145 kB 00:00 2024-06-02T22:57:02.056 INFO:teuthology.orchestra.run.smithi052.stdout:Dependencies resolved. 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: Package Arch Version Repository Size 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout:Installing: 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-02T22:57:02.067 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-02T22:57:02.068 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-02T22:57:02.069 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrading: 2024-06-02T22:57:02.070 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-02T22:57:02.071 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout:Installing dependencies: 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-02T22:57:02.072 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-02T22:57:02.073 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-02T22:57:02.074 INFO:teuthology.orchestra.run.smithi052.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-02T22:57:02.075 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-02T22:57:02.076 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-02T22:57:02.077 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-02T22:57:02.078 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-02T22:57:02.079 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-02T22:57:02.080 INFO:teuthology.orchestra.run.smithi052.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout:Installing weak dependencies: 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction Summary 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout:======================================================================================= 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout:Install 148 Packages 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout:Upgrade 11 Packages 2024-06-02T22:57:02.081 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:02.082 INFO:teuthology.orchestra.run.smithi052.stdout:Total download size: 225 M 2024-06-02T22:57:02.082 INFO:teuthology.orchestra.run.smithi052.stdout:Downloading Packages: 2024-06-02T22:57:02.086 INFO:teuthology.orchestra.run.smithi123.stdout:(4/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.8 MB/s | 890 kB 00:00 2024-06-02T22:57:02.167 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-06-02T22:57:02.220 INFO:teuthology.orchestra.run.smithi123.stdout:(5/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 2.4 MB 00:00 2024-06-02T22:57:02.629 INFO:teuthology.orchestra.run.smithi123.stdout:(6/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 11 MB/s | 4.7 MB 00:00 2024-06-02T22:57:02.679 INFO:teuthology.orchestra.run.smithi123.stdout:(7/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 2.5 MB/s | 1.5 MB 00:00 2024-06-02T22:57:02.977 INFO:teuthology.orchestra.run.smithi052.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 20 kB/s | 6.6 kB 00:00 2024-06-02T22:57:03.511 INFO:teuthology.orchestra.run.smithi052.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 1.6 MB/s | 890 kB 00:00 2024-06-02T22:57:03.594 INFO:teuthology.orchestra.run.smithi052.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 1.7 MB/s | 145 kB 00:00 2024-06-02T22:57:03.612 INFO:teuthology.orchestra.run.smithi123.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 9.3 MB/s | 21 MB 00:02 2024-06-02T22:57:03.696 INFO:teuthology.orchestra.run.smithi123.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 303 kB/s | 25 kB 00:00 2024-06-02T22:57:04.046 INFO:teuthology.orchestra.run.smithi123.stdout:(10/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 12 MB/s | 17 MB 00:01 2024-06-02T22:57:04.113 INFO:teuthology.orchestra.run.smithi123.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 3.3 MB/s | 221 kB 00:00 2024-06-02T22:57:04.163 INFO:teuthology.orchestra.run.smithi123.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 543 kB/s | 27 kB 00:00 2024-06-02T22:57:04.194 INFO:teuthology.orchestra.run.smithi052.stdout:(4/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 3.9 MB/s | 2.4 MB 00:00 2024-06-02T22:57:04.205 INFO:teuthology.orchestra.run.smithi123.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 760 kB/s | 32 kB 00:00 2024-06-02T22:57:04.297 INFO:teuthology.orchestra.run.smithi123.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 8.0 MB/s | 747 kB 00:00 2024-06-02T22:57:04.303 INFO:teuthology.orchestra.run.smithi052.stdout:(5/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 3.2 MB/s | 5.4 MB 00:01 2024-06-02T22:57:04.347 INFO:teuthology.orchestra.run.smithi123.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 3.2 MB/s | 163 kB 00:00 2024-06-02T22:57:04.406 INFO:teuthology.orchestra.run.smithi123.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.1 MB/s | 127 kB 00:00 2024-06-02T22:57:04.498 INFO:teuthology.orchestra.run.smithi123.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 5.3 MB/s | 501 kB 00:00 2024-06-02T22:57:04.527 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T22:57:04.528 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-06-02T22:57:04.539 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-06-02T22:57:04.540 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-06-02T22:57:04.541 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-06-02T22:57:04.541 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-06-02T22:57:04.543 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-06-02T22:57:04.545 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-06-02T22:57:04.546 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-06-02T22:57:04.609 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-06-02T22:57:04.630 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-06-02T22:57:04.630 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-06-02T22:57:04.630 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 6.6 k 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.4 M 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 890 k 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 145 k 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 1.5 M 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 142 k 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 3.5 M 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 7.4 M 2024-06-02T22:57:04.631 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 49 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 10 M 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 48 M 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 264 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 763 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 221 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 27 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 18 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-06-02T22:57:04.632 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 32 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 747 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 127 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 162 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 321 k 2024-06-02T22:57:04.633 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 300 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 99 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 85 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 171 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager x86_64 1:1.47.91-1.el9 baseos 2.3 M 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm x86_64 1:1.47.91-1.el9 baseos 1.8 M 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team x86_64 1:1.47.91-1.el9 baseos 41 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui x86_64 1:1.47.91-1.el9 baseos 248 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-debuginfod-client x86_64 0.191-4.el9 baseos 37 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf x86_64 0.191-4.el9 baseos 207 k 2024-06-02T22:57:04.634 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libs x86_64 0.191-4.el9 baseos 260 k 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.4 M 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 3.1 M 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 21 M 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 32 k 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 2.4 M 2024-06-02T22:57:04.635 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 249 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 4.7 M 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 17 M 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2946.g3d89accf.el9 ceph-noarch 16 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 25 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-06-02T22:57:04.636 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 163 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 501 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 5.3 M 2024-06-02T22:57:04.637 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-06-02T22:57:04.638 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 45 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.0.0-2946.g3d89accf.el9 ceph 135 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-06-02T22:57:04.639 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-06-02T22:57:04.640 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-06-02T22:57:04.641 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-06-02T22:57:04.642 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-06-02T22:57:04.643 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-06-02T22:57:04.644 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 11 Packages 2024-06-02T22:57:04.645 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:57:04.645 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 225 M 2024-06-02T22:57:04.645 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-06-02T22:57:04.670 INFO:teuthology.orchestra.run.smithi052.stdout:(6/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-06-02T22:57:05.012 INFO:teuthology.orchestra.run.smithi052.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 6.6 MB/s | 4.7 MB 00:00 2024-06-02T22:57:05.082 INFO:teuthology.orchestra.run.smithi123.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 9.1 MB/s | 5.3 MB 00:00 2024-06-02T22:57:05.140 INFO:teuthology.orchestra.run.smithi123.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 776 kB/s | 45 kB 00:00 2024-06-02T22:57:05.224 INFO:teuthology.orchestra.run.smithi123.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 1.6 MB/s | 135 kB 00:00 2024-06-02T22:57:05.316 INFO:teuthology.orchestra.run.smithi123.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.7 MB/s | 162 kB 00:00 2024-06-02T22:57:05.408 INFO:teuthology.orchestra.run.smithi123.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 3.4 MB/s | 321 kB 00:00 2024-06-02T22:57:05.508 INFO:teuthology.orchestra.run.smithi123.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 300 kB 00:00 2024-06-02T22:57:05.585 INFO:teuthology.orchestra.run.smithi174.stdout:(1/159): ceph-19.0.0-2946.g3d89accf.el9.x86_64. 21 kB/s | 6.6 kB 00:00 2024-06-02T22:57:05.600 INFO:teuthology.orchestra.run.smithi123.stdout:(24/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.1 MB/s | 99 kB 00:00 2024-06-02T22:57:05.684 INFO:teuthology.orchestra.run.smithi123.stdout:(25/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.0 MB/s | 85 kB 00:00 2024-06-02T22:57:06.071 INFO:teuthology.orchestra.run.smithi052.stdout:(8/159): ceph-radosgw-19.0.0-2946.g3d89accf.el9 9.7 MB/s | 10 MB 00:01 2024-06-02T22:57:06.154 INFO:teuthology.orchestra.run.smithi052.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 303 kB/s | 25 kB 00:00 2024-06-02T22:57:06.184 INFO:teuthology.orchestra.run.smithi123.stdout:(26/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 6.2 MB/s | 3.1 MB 00:00 2024-06-02T22:57:06.259 INFO:teuthology.orchestra.run.smithi123.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.2 MB/s | 171 kB 00:00 2024-06-02T22:57:06.334 INFO:teuthology.orchestra.run.smithi123.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 423 kB/s | 32 kB 00:00 2024-06-02T22:57:06.418 INFO:teuthology.orchestra.run.smithi123.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 1.7 MB/s | 142 kB 00:00 2024-06-02T22:57:06.669 INFO:teuthology.orchestra.run.smithi174.stdout:(2/159): ceph-fuse-19.0.0-2946.g3d89accf.el9.x8 821 kB/s | 890 kB 00:01 2024-06-02T22:57:06.919 INFO:teuthology.orchestra.run.smithi174.stdout:(3/159): ceph-immutable-object-cache-19.0.0-294 579 kB/s | 145 kB 00:00 2024-06-02T22:57:07.118 INFO:teuthology.orchestra.run.smithi123.stdout:(30/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.0 MB/s | 3.5 MB 00:00 2024-06-02T22:57:08.079 INFO:teuthology.orchestra.run.smithi052.stdout:(10/159): ceph-common-19.0.0-2946.g3d89accf.el9 3.9 MB/s | 21 MB 00:05 2024-06-02T22:57:08.230 INFO:teuthology.orchestra.run.smithi052.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 1.4 MB/s | 221 kB 00:00 2024-06-02T22:57:08.313 INFO:teuthology.orchestra.run.smithi052.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 325 kB/s | 27 kB 00:00 2024-06-02T22:57:08.377 INFO:teuthology.orchestra.run.smithi174.stdout:(4/159): ceph-base-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 5.4 MB 00:03 2024-06-02T22:57:08.388 INFO:teuthology.orchestra.run.smithi052.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 423 kB/s | 32 kB 00:00 2024-06-02T22:57:08.518 INFO:teuthology.orchestra.run.smithi123.stdout:(31/159): ceph-mgr-diskprediction-local-19.0.0- 5.3 MB/s | 7.4 MB 00:01 2024-06-02T22:57:08.602 INFO:teuthology.orchestra.run.smithi123.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.9 MB/s | 249 kB 00:00 2024-06-02T22:57:08.639 INFO:teuthology.orchestra.run.smithi052.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 2.9 MB/s | 747 kB 00:00 2024-06-02T22:57:08.711 INFO:teuthology.orchestra.run.smithi123.stdout:(33/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 1.7 MB/s | 10 MB 00:06 2024-06-02T22:57:08.714 INFO:teuthology.orchestra.run.smithi052.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 2.1 MB/s | 163 kB 00:00 2024-06-02T22:57:08.752 INFO:teuthology.orchestra.run.smithi123.stdout:(34/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 328 kB/s | 49 kB 00:00 2024-06-02T22:57:08.786 INFO:teuthology.orchestra.run.smithi123.stdout:(35/159): ceph-prometheus-alerts-19.0.0-2946.g3 219 kB/s | 16 kB 00:00 2024-06-02T22:57:08.789 INFO:teuthology.orchestra.run.smithi052.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 1.6 MB/s | 127 kB 00:00 2024-06-02T22:57:08.836 INFO:teuthology.orchestra.run.smithi123.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 3.1 MB/s | 264 kB 00:00 2024-06-02T22:57:08.920 INFO:teuthology.orchestra.run.smithi123.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 219 kB/s | 18 kB 00:00 2024-06-02T22:57:08.953 INFO:teuthology.orchestra.run.smithi174.stdout:(5/159): ceph-mgr-19.0.0-2946.g3d89accf.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-06-02T22:57:08.973 INFO:teuthology.orchestra.run.smithi052.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 2.7 MB/s | 501 kB 00:00 2024-06-02T22:57:08.995 INFO:teuthology.orchestra.run.smithi123.stdout:(38/159): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 80 kB 00:00 2024-06-02T22:57:09.003 INFO:teuthology.orchestra.run.smithi174.stdout:(6/159): ceph-mds-19.0.0-2946.g3d89accf.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-06-02T22:57:09.079 INFO:teuthology.orchestra.run.smithi123.stdout:(39/159): libgfortran-11.4.1-3.el9.x86_64.rpm 9.4 MB/s | 803 kB 00:00 2024-06-02T22:57:09.121 INFO:teuthology.orchestra.run.smithi123.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-06-02T22:57:09.163 INFO:teuthology.orchestra.run.smithi123.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-06-02T22:57:09.230 INFO:teuthology.orchestra.run.smithi123.stdout:(42/159): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-06-02T22:57:09.255 INFO:teuthology.orchestra.run.smithi123.stdout:(43/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 1.6 MB/s | 763 kB 00:00 2024-06-02T22:57:09.280 INFO:teuthology.orchestra.run.smithi123.stdout:(44/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-06-02T22:57:09.314 INFO:teuthology.orchestra.run.smithi123.stdout:(45/159): python3-requests-2.25.1-8.el9.noarch. 3.7 MB/s | 125 kB 00:00 2024-06-02T22:57:09.356 INFO:teuthology.orchestra.run.smithi123.stdout:(46/159): python3-urllib3-1.26.5-5.el9.noarch.r 5.1 MB/s | 215 kB 00:00 2024-06-02T22:57:09.381 INFO:teuthology.orchestra.run.smithi123.stdout:(47/159): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-06-02T22:57:09.448 INFO:teuthology.orchestra.run.smithi123.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 7.3 MB/s | 689 kB 00:00 2024-06-02T22:57:09.490 INFO:teuthology.orchestra.run.smithi123.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-06-02T22:57:09.515 INFO:teuthology.orchestra.run.smithi123.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-06-02T22:57:09.541 INFO:teuthology.orchestra.run.smithi123.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 432 kB/s | 22 kB 00:00 2024-06-02T22:57:09.574 INFO:teuthology.orchestra.run.smithi123.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-06-02T22:57:09.600 INFO:teuthology.orchestra.run.smithi123.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 558 kB/s | 33 kB 00:00 2024-06-02T22:57:09.650 INFO:teuthology.orchestra.run.smithi123.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-06-02T22:57:09.717 INFO:teuthology.orchestra.run.smithi123.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-06-02T22:57:09.742 INFO:teuthology.orchestra.run.smithi123.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-06-02T22:57:09.767 INFO:teuthology.orchestra.run.smithi123.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-06-02T22:57:09.793 INFO:teuthology.orchestra.run.smithi123.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 902 kB/s | 45 kB 00:00 2024-06-02T22:57:09.827 INFO:teuthology.orchestra.run.smithi123.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-06-02T22:57:09.852 INFO:teuthology.orchestra.run.smithi123.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 9.7 MB/s | 582 kB 00:00 2024-06-02T22:57:09.877 INFO:teuthology.orchestra.run.smithi123.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 504 kB/s | 25 kB 00:00 2024-06-02T22:57:09.902 INFO:teuthology.orchestra.run.smithi123.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-02T22:57:09.928 INFO:teuthology.orchestra.run.smithi123.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-02T22:57:09.953 INFO:teuthology.orchestra.run.smithi123.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-02T22:57:09.978 INFO:teuthology.orchestra.run.smithi123.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-06-02T22:57:10.062 INFO:teuthology.orchestra.run.smithi123.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 44 MB/s | 4.7 MB 00:00 2024-06-02T22:57:10.137 INFO:teuthology.orchestra.run.smithi123.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 28 MB/s | 4.4 MB 00:00 2024-06-02T22:57:10.179 INFO:teuthology.orchestra.run.smithi123.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-06-02T22:57:10.213 INFO:teuthology.orchestra.run.smithi123.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-02T22:57:10.238 INFO:teuthology.orchestra.run.smithi123.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-02T22:57:10.297 INFO:teuthology.orchestra.run.smithi123.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-06-02T22:57:10.298 INFO:teuthology.orchestra.run.smithi052.stdout:(18/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 4.0 MB/s | 5.3 MB 00:01 2024-06-02T22:57:10.365 INFO:teuthology.orchestra.run.smithi052.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 681 kB/s | 45 kB 00:00 2024-06-02T22:57:10.372 INFO:teuthology.orchestra.run.smithi123.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-06-02T22:57:10.397 INFO:teuthology.orchestra.run.smithi123.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 874 kB/s | 87 kB 00:00 2024-06-02T22:57:10.432 INFO:teuthology.orchestra.run.smithi052.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.0 MB/s | 135 kB 00:00 2024-06-02T22:57:10.439 INFO:teuthology.orchestra.run.smithi123.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-02T22:57:10.507 INFO:teuthology.orchestra.run.smithi052.stdout:(21/159): python3-cephfs-19.0.0-2946.g3d89accf. 2.1 MB/s | 162 kB 00:00 2024-06-02T22:57:10.523 INFO:teuthology.orchestra.run.smithi123.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-06-02T22:57:10.548 INFO:teuthology.orchestra.run.smithi123.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.2 MB/s | 244 kB 00:00 2024-06-02T22:57:10.573 INFO:teuthology.orchestra.run.smithi123.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-02T22:57:10.599 INFO:teuthology.orchestra.run.smithi123.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-06-02T22:57:10.616 INFO:teuthology.orchestra.run.smithi123.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 457 kB/s | 19 kB 00:00 2024-06-02T22:57:10.641 INFO:teuthology.orchestra.run.smithi052.stdout:(22/159): python3-rados-19.0.0-2946.g3d89accf.e 2.4 MB/s | 321 kB 00:00 2024-06-02T22:57:10.641 INFO:teuthology.orchestra.run.smithi123.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 4.0 MB/s | 172 kB 00:00 2024-06-02T22:57:10.645 INFO:teuthology.orchestra.run.smithi174.stdout:(7/159): ceph-mon-19.0.0-2946.g3d89accf.el9.x86 2.8 MB/s | 4.7 MB 00:01 2024-06-02T22:57:10.658 INFO:teuthology.orchestra.run.smithi123.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 832 kB/s | 35 kB 00:00 2024-06-02T22:57:10.700 INFO:teuthology.orchestra.run.smithi123.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-02T22:57:10.758 INFO:teuthology.orchestra.run.smithi052.stdout:(23/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.5 MB/s | 300 kB 00:00 2024-06-02T22:57:10.775 INFO:teuthology.orchestra.run.smithi123.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 38 MB/s | 5.1 MB 00:00 2024-06-02T22:57:10.801 INFO:teuthology.orchestra.run.smithi123.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-06-02T22:57:10.826 INFO:teuthology.orchestra.run.smithi123.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-02T22:57:10.851 INFO:teuthology.orchestra.run.smithi123.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 844 kB/s | 42 kB 00:00 2024-06-02T22:57:10.885 INFO:teuthology.orchestra.run.smithi123.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-06-02T22:57:10.910 INFO:teuthology.orchestra.run.smithi123.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.7 MB/s | 159 kB 00:00 2024-06-02T22:57:10.933 INFO:teuthology.orchestra.run.smithi052.stdout:(24/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 2.7 MB/s | 17 MB 00:06 2024-06-02T22:57:10.936 INFO:teuthology.orchestra.run.smithi123.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-02T22:57:10.958 INFO:teuthology.orchestra.run.smithi052.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 496 kB/s | 99 kB 00:00 2024-06-02T22:57:10.961 INFO:teuthology.orchestra.run.smithi123.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-06-02T22:57:10.986 INFO:teuthology.orchestra.run.smithi123.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-02T22:57:11.000 INFO:teuthology.orchestra.run.smithi052.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.3 MB/s | 85 kB 00:00 2024-06-02T22:57:11.028 INFO:teuthology.orchestra.run.smithi123.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-06-02T22:57:11.070 INFO:teuthology.orchestra.run.smithi123.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-06-02T22:57:11.092 INFO:teuthology.orchestra.run.smithi052.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 1.8 MB/s | 171 kB 00:00 2024-06-02T22:57:11.112 INFO:teuthology.orchestra.run.smithi123.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-06-02T22:57:11.151 INFO:teuthology.orchestra.run.smithi052.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 546 kB/s | 32 kB 00:00 2024-06-02T22:57:11.154 INFO:teuthology.orchestra.run.smithi123.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-06-02T22:57:11.329 INFO:teuthology.orchestra.run.smithi123.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-06-02T22:57:11.355 INFO:teuthology.orchestra.run.smithi123.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 237 kB/s | 47 kB 00:00 2024-06-02T22:57:11.380 INFO:teuthology.orchestra.run.smithi123.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 6.0 MB/s | 305 kB 00:00 2024-06-02T22:57:11.405 INFO:teuthology.orchestra.run.smithi123.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-06-02T22:57:11.430 INFO:teuthology.orchestra.run.smithi123.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-02T22:57:11.456 INFO:teuthology.orchestra.run.smithi123.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-02T22:57:11.481 INFO:teuthology.orchestra.run.smithi123.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-06-02T22:57:11.506 INFO:teuthology.orchestra.run.smithi123.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-06-02T22:57:11.548 INFO:teuthology.orchestra.run.smithi123.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-06-02T22:57:11.652 INFO:teuthology.orchestra.run.smithi052.stdout:(29/159): ceph-test-19.0.0-2946.g3d89accf.el9.x 8.8 MB/s | 48 MB 00:05 2024-06-02T22:57:11.677 INFO:teuthology.orchestra.run.smithi052.stdout:(30/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 270 kB/s | 142 kB 00:00 2024-06-02T22:57:11.744 INFO:teuthology.orchestra.run.smithi052.stdout:(31/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 3.9 MB/s | 3.1 MB 00:00 2024-06-02T22:57:11.886 INFO:teuthology.orchestra.run.smithi052.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 1.7 MB/s | 249 kB 00:00 2024-06-02T22:57:11.961 INFO:teuthology.orchestra.run.smithi052.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 658 kB/s | 49 kB 00:00 2024-06-02T22:57:12.028 INFO:teuthology.orchestra.run.smithi052.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 246 kB/s | 16 kB 00:00 2024-06-02T22:57:12.065 INFO:teuthology.orchestra.run.smithi123.stdout:(105/159): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-06-02T22:57:12.099 INFO:teuthology.orchestra.run.smithi123.stdout:(106/159): abseil-cpp-20211102.0-3.el9.x86_64.r 926 kB/s | 548 kB 00:00 2024-06-02T22:57:12.111 INFO:teuthology.orchestra.run.smithi174.stdout:(8/159): ceph-common-19.0.0-2946.g3d89accf.el9. 3.1 MB/s | 21 MB 00:06 2024-06-02T22:57:12.112 INFO:teuthology.orchestra.run.smithi052.stdout:(35/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 7.6 MB/s | 3.5 MB 00:00 2024-06-02T22:57:12.137 INFO:teuthology.orchestra.run.smithi052.stdout:(36/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.4 MB/s | 264 kB 00:00 2024-06-02T22:57:12.157 INFO:teuthology.orchestra.run.smithi123.stdout:(107/159): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-06-02T22:57:12.161 INFO:teuthology.orchestra.run.smithi174.stdout:(9/159): ceph-selinux-19.0.0-2946.g3d89accf.el9 505 kB/s | 25 kB 00:00 2024-06-02T22:57:12.204 INFO:teuthology.orchestra.run.smithi052.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 274 kB/s | 18 kB 00:00 2024-06-02T22:57:12.238 INFO:teuthology.orchestra.run.smithi052.stdout:(38/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 5.9 MB/s | 763 kB 00:00 2024-06-02T22:57:12.258 INFO:teuthology.orchestra.run.smithi123.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-06-02T22:57:12.330 INFO:teuthology.orchestra.run.smithi052.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 641 kB/s | 80 kB 00:00 2024-06-02T22:57:12.366 INFO:teuthology.orchestra.run.smithi123.stdout:(109/159): liboath-2.6.7-2.el9.x86_64.rpm 449 kB/s | 49 kB 00:00 2024-06-02T22:57:12.371 INFO:teuthology.orchestra.run.smithi052.stdout:(40/159): libgfortran-11.4.1-3.el9.x86_64.rpm 5.9 MB/s | 803 kB 00:00 2024-06-02T22:57:12.397 INFO:teuthology.orchestra.run.smithi052.stdout:(41/159): libquadmath-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 195 kB 00:00 2024-06-02T22:57:12.422 INFO:teuthology.orchestra.run.smithi052.stdout:(42/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-06-02T22:57:12.456 INFO:teuthology.orchestra.run.smithi052.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 3.1 MB/s | 106 kB 00:00 2024-06-02T22:57:12.466 INFO:teuthology.orchestra.run.smithi123.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 675 kB/s | 67 kB 00:00 2024-06-02T22:57:12.489 INFO:teuthology.orchestra.run.smithi052.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 13 MB/s | 1.2 MB 00:00 2024-06-02T22:57:12.514 INFO:teuthology.orchestra.run.smithi052.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-02T22:57:12.540 INFO:teuthology.orchestra.run.smithi052.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-02T22:57:12.541 INFO:teuthology.orchestra.run.smithi123.stdout:(111/159): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-06-02T22:57:12.565 INFO:teuthology.orchestra.run.smithi052.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-06-02T22:57:12.584 INFO:teuthology.orchestra.run.smithi123.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 1.3 MB/s | 151 kB 00:00 2024-06-02T22:57:12.599 INFO:teuthology.orchestra.run.smithi052.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 11 MB/s | 689 kB 00:00 2024-06-02T22:57:12.641 INFO:teuthology.orchestra.run.smithi052.stdout:(49/159): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-06-02T22:57:12.659 INFO:teuthology.orchestra.run.smithi123.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-06-02T22:57:12.666 INFO:teuthology.orchestra.run.smithi052.stdout:(50/159): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-06-02T22:57:12.683 INFO:teuthology.orchestra.run.smithi052.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 516 kB/s | 22 kB 00:00 2024-06-02T22:57:12.709 INFO:teuthology.orchestra.run.smithi052.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-06-02T22:57:12.742 INFO:teuthology.orchestra.run.smithi052.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-06-02T22:57:12.745 INFO:teuthology.orchestra.run.smithi174.stdout:(10/159): ceph-radosgw-19.0.0-2946.g3d89accf.el 4.9 MB/s | 10 MB 00:02 2024-06-02T22:57:12.759 INFO:teuthology.orchestra.run.smithi123.stdout:(114/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-06-02T22:57:12.784 INFO:teuthology.orchestra.run.smithi052.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-06-02T22:57:12.785 INFO:teuthology.orchestra.run.smithi123.stdout:(115/159): python3-asyncssh-2.13.2-5.el9.noarch 2.7 MB/s | 548 kB 00:00 2024-06-02T22:57:12.820 INFO:teuthology.orchestra.run.smithi174.stdout:(11/159): cephfs-mirror-19.0.0-2946.g3d89accf.e 2.9 MB/s | 221 kB 00:00 2024-06-02T22:57:12.851 INFO:teuthology.orchestra.run.smithi052.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 21 MB/s | 3.0 MB 00:00 2024-06-02T22:57:12.851 INFO:teuthology.orchestra.run.smithi123.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 350 kB/s | 32 kB 00:00 2024-06-02T22:57:12.863 INFO:teuthology.orchestra.run.smithi174.stdout:(12/159): cephfs-shell-19.0.0-2946.g3d89accf.el 648 kB/s | 27 kB 00:00 2024-06-02T22:57:12.868 INFO:teuthology.orchestra.run.smithi052.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 133 kB/s | 11 kB 00:00 2024-06-02T22:57:12.885 INFO:teuthology.orchestra.run.smithi123.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 149 kB/s | 15 kB 00:00 2024-06-02T22:57:12.893 INFO:teuthology.orchestra.run.smithi052.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.7 MB/s | 160 kB 00:00 2024-06-02T22:57:12.910 INFO:teuthology.orchestra.run.smithi052.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-02T22:57:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-06-02T22:57:12.959 INFO:teuthology.orchestra.run.smithi123.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2024-06-02T22:57:12.977 INFO:teuthology.orchestra.run.smithi052.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-06-02T22:57:13.011 INFO:teuthology.orchestra.run.smithi052.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-06-02T22:57:13.012 INFO:teuthology.orchestra.run.smithi174.stdout:(13/159): libcephfs-devel-19.0.0-2946.g3d89accf 211 kB/s | 32 kB 00:00 2024-06-02T22:57:13.037 INFO:teuthology.orchestra.run.smithi052.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 806 kB/s | 47 kB 00:00 2024-06-02T22:57:13.062 INFO:teuthology.orchestra.run.smithi052.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-02T22:57:13.087 INFO:teuthology.orchestra.run.smithi052.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-02T22:57:13.113 INFO:teuthology.orchestra.run.smithi052.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-06-02T22:57:13.119 INFO:teuthology.orchestra.run.smithi123.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 1.5 MB/s | 358 kB 00:00 2024-06-02T22:57:13.154 INFO:teuthology.orchestra.run.smithi174.stdout:(14/159): libcephfs2-19.0.0-2946.g3d89accf.el9. 5.1 MB/s | 747 kB 00:00 2024-06-02T22:57:13.169 INFO:teuthology.orchestra.run.smithi123.stdout:(120/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.4 MB/s | 303 kB 00:00 2024-06-02T22:57:13.205 INFO:teuthology.orchestra.run.smithi174.stdout:(15/159): libcephsqlite-19.0.0-2946.g3d89accf.e 3.2 MB/s | 163 kB 00:00 2024-06-02T22:57:13.221 INFO:teuthology.orchestra.run.smithi052.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 35 MB/s | 4.7 MB 00:00 2024-06-02T22:57:13.228 INFO:teuthology.orchestra.run.smithi123.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-06-02T22:57:13.255 INFO:teuthology.orchestra.run.smithi174.stdout:(16/159): librados-devel-19.0.0-2946.g3d89accf. 2.5 MB/s | 127 kB 00:00 2024-06-02T22:57:13.288 INFO:teuthology.orchestra.run.smithi052.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-06-02T22:57:13.312 INFO:teuthology.orchestra.run.smithi123.stdout:(122/159): python3-grpcio-1.46.7-10.el9.x86_64. 14 MB/s | 2.0 MB 00:00 2024-06-02T22:57:13.337 INFO:teuthology.orchestra.run.smithi123.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-06-02T22:57:13.364 INFO:teuthology.orchestra.run.smithi174.stdout:(17/159): libradosstriper1-19.0.0-2946.g3d89acc 4.5 MB/s | 501 kB 00:00 2024-06-02T22:57:13.380 INFO:teuthology.orchestra.run.smithi052.stdout:(68/159): ceph-mgr-diskprediction-local-19.0.0- 4.3 MB/s | 7.4 MB 00:01 2024-06-02T22:57:13.396 INFO:teuthology.orchestra.run.smithi123.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 128 kB/s | 11 kB 00:00 2024-06-02T22:57:13.414 INFO:teuthology.orchestra.run.smithi052.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 7.1 MB/s | 911 kB 00:00 2024-06-02T22:57:13.429 INFO:teuthology.orchestra.run.smithi123.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-06-02T22:57:13.472 INFO:teuthology.orchestra.run.smithi052.stdout:(70/159): postgresql-13.14-1.el9.x86_64.rpm 6.3 MB/s | 1.6 MB 00:00 2024-06-02T22:57:13.488 INFO:teuthology.orchestra.run.smithi123.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 252 kB/s | 23 kB 00:00 2024-06-02T22:57:13.521 INFO:teuthology.orchestra.run.smithi123.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 212 kB/s | 19 kB 00:00 2024-06-02T22:57:13.523 INFO:teuthology.orchestra.run.smithi052.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-06-02T22:57:13.548 INFO:teuthology.orchestra.run.smithi052.stdout:(72/159): postgresql-private-libs-13.14-1.el9.x 797 kB/s | 133 kB 00:00 2024-06-02T22:57:13.573 INFO:teuthology.orchestra.run.smithi052.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-06-02T22:57:13.580 INFO:teuthology.orchestra.run.smithi123.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-06-02T22:57:13.657 INFO:teuthology.orchestra.run.smithi052.stdout:(74/159): postgresql-server-13.14-1.el9.x86_64. 24 MB/s | 5.8 MB 00:00 2024-06-02T22:57:13.673 INFO:teuthology.orchestra.run.smithi123.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 506 kB/s | 46 kB 00:00 2024-06-02T22:57:13.682 INFO:teuthology.orchestra.run.smithi052.stdout:(75/159): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-06-02T22:57:13.706 INFO:teuthology.orchestra.run.smithi123.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-06-02T22:57:13.707 INFO:teuthology.orchestra.run.smithi052.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 4.7 MB/s | 244 kB 00:00 2024-06-02T22:57:13.733 INFO:teuthology.orchestra.run.smithi052.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-02T22:57:13.758 INFO:teuthology.orchestra.run.smithi052.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-06-02T22:57:13.764 INFO:teuthology.orchestra.run.smithi123.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 860 kB/s | 79 kB 00:00 2024-06-02T22:57:13.783 INFO:teuthology.orchestra.run.smithi052.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-06-02T22:57:13.798 INFO:teuthology.orchestra.run.smithi123.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 628 kB/s | 58 kB 00:00 2024-06-02T22:57:13.817 INFO:teuthology.orchestra.run.smithi052.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-06-02T22:57:13.842 INFO:teuthology.orchestra.run.smithi052.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-06-02T22:57:13.865 INFO:teuthology.orchestra.run.smithi123.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 2.7 MB/s | 272 kB 00:00 2024-06-02T22:57:13.884 INFO:teuthology.orchestra.run.smithi052.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-06-02T22:57:13.890 INFO:teuthology.orchestra.run.smithi123.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-06-02T22:57:13.957 INFO:teuthology.orchestra.run.smithi123.stdout:(135/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 979 kB/s | 90 kB 00:00 2024-06-02T22:57:13.968 INFO:teuthology.orchestra.run.smithi052.stdout:(83/159): python3-babel-2.9.1-2.el9.noarch.rpm 14 MB/s | 6.0 MB 00:00 2024-06-02T22:57:13.982 INFO:teuthology.orchestra.run.smithi123.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-06-02T22:57:13.993 INFO:teuthology.orchestra.run.smithi052.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 2.0 MB/s | 222 kB 00:00 2024-06-02T22:57:14.018 INFO:teuthology.orchestra.run.smithi052.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-06-02T22:57:14.022 INFO:teuthology.orchestra.run.smithi174.stdout:(18/159): ceph-osd-19.0.0-2946.g3d89accf.el9.x8 3.3 MB/s | 17 MB 00:05 2024-06-02T22:57:14.044 INFO:teuthology.orchestra.run.smithi052.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 836 kB/s | 42 kB 00:00 2024-06-02T22:57:14.049 INFO:teuthology.orchestra.run.smithi123.stdout:(137/159): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-06-02T22:57:14.072 INFO:teuthology.orchestra.run.smithi174.stdout:(19/159): python3-ceph-argparse-19.0.0-2946.g3d 906 kB/s | 45 kB 00:00 2024-06-02T22:57:14.083 INFO:teuthology.orchestra.run.smithi123.stdout:(138/159): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-06-02T22:57:14.131 INFO:teuthology.orchestra.run.smithi174.stdout:(20/159): python3-ceph-common-19.0.0-2946.g3d89 2.3 MB/s | 135 kB 00:00 2024-06-02T22:57:14.141 INFO:teuthology.orchestra.run.smithi123.stdout:(139/159): python3-rsa-4.9-2.el9.noarch.rpm 644 kB/s | 59 kB 00:00 2024-06-02T22:57:14.175 INFO:teuthology.orchestra.run.smithi123.stdout:(140/159): python3-tempora-5.0.0-2.el9.noarch.r 388 kB/s | 36 kB 00:00 2024-06-02T22:57:14.186 INFO:teuthology.orchestra.run.smithi052.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 1.6 MB/s | 269 kB 00:00 2024-06-02T22:57:14.211 INFO:teuthology.orchestra.run.smithi052.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 953 kB/s | 159 kB 00:00 2024-06-02T22:57:14.231 INFO:teuthology.orchestra.run.smithi174.stdout:(21/159): librgw2-19.0.0-2946.g3d89accf.el9.x86 6.1 MB/s | 5.3 MB 00:00 2024-06-02T22:57:14.234 INFO:teuthology.orchestra.run.smithi123.stdout:(141/159): python3-typing-extensions-4.4.0-2.el 555 kB/s | 51 kB 00:00 2024-06-02T22:57:14.236 INFO:teuthology.orchestra.run.smithi052.stdout:(89/159): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-06-02T22:57:14.253 INFO:teuthology.orchestra.run.smithi052.stdout:(90/159): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 51 kB 00:00 2024-06-02T22:57:14.256 INFO:teuthology.orchestra.run.smithi174.stdout:(22/159): python3-cephfs-19.0.0-2946.g3d89accf. 1.3 MB/s | 162 kB 00:00 2024-06-02T22:57:14.270 INFO:teuthology.orchestra.run.smithi052.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.5 MB/s | 54 kB 00:00 2024-06-02T22:57:14.276 INFO:teuthology.orchestra.run.smithi123.stdout:(142/159): python3-webob-1.8.7-6.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-06-02T22:57:14.307 INFO:teuthology.orchestra.run.smithi174.stdout:(23/159): python3-rados-19.0.0-2946.g3d89accf.e 4.2 MB/s | 321 kB 00:00 2024-06-02T22:57:14.326 INFO:teuthology.orchestra.run.smithi123.stdout:(143/159): python3-websocket-client-1.2.3-2.el9 976 kB/s | 90 kB 00:00 2024-06-02T22:57:14.337 INFO:teuthology.orchestra.run.smithi052.stdout:(92/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.8 MB/s | 5.1 MB 00:00 2024-06-02T22:57:14.354 INFO:teuthology.orchestra.run.smithi052.stdout:(93/159): python3-toml-0.10.2-6.el9.noarch.rpm 499 kB/s | 42 kB 00:00 2024-06-02T22:57:14.357 INFO:teuthology.orchestra.run.smithi174.stdout:(24/159): python3-rbd-19.0.0-2946.g3d89accf.el9 2.9 MB/s | 300 kB 00:00 2024-06-02T22:57:14.371 INFO:teuthology.orchestra.run.smithi052.stdout:(94/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-06-02T22:57:14.382 INFO:teuthology.orchestra.run.smithi174.stdout:(25/159): python3-rgw-19.0.0-2946.g3d89accf.el9 1.3 MB/s | 99 kB 00:00 2024-06-02T22:57:14.396 INFO:teuthology.orchestra.run.smithi052.stdout:(95/159): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-06-02T22:57:14.408 INFO:teuthology.orchestra.run.smithi174.stdout:(26/159): rbd-fuse-19.0.0-2946.g3d89accf.el9.x8 1.7 MB/s | 85 kB 00:00 2024-06-02T22:57:14.414 INFO:teuthology.orchestra.run.smithi052.stdout:(96/159): qatlib-service-24.02.0-1.el9.x86_64.r 855 kB/s | 36 kB 00:00 2024-06-02T22:57:14.439 INFO:teuthology.orchestra.run.smithi052.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-06-02T22:57:14.467 INFO:teuthology.orchestra.run.smithi174.stdout:(27/159): rbd-nbd-19.0.0-2946.g3d89accf.el9.x86 2.9 MB/s | 171 kB 00:00 2024-06-02T22:57:14.481 INFO:teuthology.orchestra.run.smithi052.stdout:(98/159): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-06-02T22:57:14.506 INFO:teuthology.orchestra.run.smithi052.stdout:(99/159): uuid-1.6.2-55.el9.x86_64.rpm 864 kB/s | 58 kB 00:00 2024-06-02T22:57:14.526 INFO:teuthology.orchestra.run.smithi174.stdout:(28/159): ceph-grafana-dashboards-19.0.0-2946.g 545 kB/s | 32 kB 00:00 2024-06-02T22:57:14.531 INFO:teuthology.orchestra.run.smithi052.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-06-02T22:57:14.557 INFO:teuthology.orchestra.run.smithi052.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-06-02T22:57:14.582 INFO:teuthology.orchestra.run.smithi052.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 895 kB/s | 45 kB 00:00 2024-06-02T22:57:14.584 INFO:teuthology.orchestra.run.smithi174.stdout:(29/159): ceph-mgr-cephadm-19.0.0-2946.g3d89acc 2.4 MB/s | 142 kB 00:00 2024-06-02T22:57:14.760 INFO:teuthology.orchestra.run.smithi123.stdout:(144/159): ceph-test-19.0.0-2946.g3d89accf.el9. 4.4 MB/s | 48 MB 00:11 2024-06-02T22:57:14.785 INFO:teuthology.orchestra.run.smithi123.stdout:(145/159): python3-zc-lockfile-2.0-10.el9.noarc 43 kB/s | 20 kB 00:00 2024-06-02T22:57:14.800 INFO:teuthology.orchestra.run.smithi174.stdout:(30/159): rbd-mirror-19.0.0-2946.g3d89accf.el9. 7.4 MB/s | 3.1 MB 00:00 2024-06-02T22:57:14.810 INFO:teuthology.orchestra.run.smithi123.stdout:(146/159): python3-werkzeug-2.0.3-3.el9.1.noarc 799 kB/s | 427 kB 00:00 2024-06-02T22:57:14.841 INFO:teuthology.orchestra.run.smithi052.stdout:(103/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-06-02T22:57:14.866 INFO:teuthology.orchestra.run.smithi052.stdout:(104/159): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-06-02T22:57:14.919 INFO:teuthology.orchestra.run.smithi123.stdout:(147/159): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-06-02T22:57:15.058 INFO:teuthology.orchestra.run.smithi052.stdout:(105/159): protobuf-compiler-3.14.0-13.el9.x86_ 1.8 MB/s | 863 kB 00:00 2024-06-02T22:57:15.116 INFO:teuthology.orchestra.run.smithi052.stdout:(106/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-06-02T22:57:15.142 INFO:teuthology.orchestra.run.smithi052.stdout:(107/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.8 MB/s | 548 kB 00:00 2024-06-02T22:57:15.167 INFO:teuthology.orchestra.run.smithi052.stdout:(108/159): grpc-data-1.46.7-10.el9.noarch.rpm 178 kB/s | 19 kB 00:00 2024-06-02T22:57:15.185 INFO:teuthology.orchestra.run.smithi174.stdout:(31/159): ceph-mgr-dashboard-19.0.0-2946.g3d89a 5.9 MB/s | 3.5 MB 00:00 2024-06-02T22:57:15.192 INFO:teuthology.orchestra.run.smithi052.stdout:(109/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 516 kB/s | 26 kB 00:00 2024-06-02T22:57:15.228 INFO:teuthology.orchestra.run.smithi123.stdout:(148/159): re2-20211101-3.el9.x86_64.rpm 410 kB/s | 192 kB 00:00 2024-06-02T22:57:15.242 INFO:teuthology.orchestra.run.smithi052.stdout:(110/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-06-02T22:57:15.268 INFO:teuthology.orchestra.run.smithi052.stdout:(111/159): liboath-2.6.7-2.el9.x86_64.rpm 484 kB/s | 49 kB 00:00 2024-06-02T22:57:15.285 INFO:teuthology.orchestra.run.smithi174.stdout:(32/159): ceph-mgr-modules-core-19.0.0-2946.g3d 2.4 MB/s | 249 kB 00:00 2024-06-02T22:57:15.301 INFO:teuthology.orchestra.run.smithi052.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-06-02T22:57:15.343 INFO:teuthology.orchestra.run.smithi174.stdout:(33/159): ceph-mgr-rook-19.0.0-2946.g3d89accf.e 844 kB/s | 49 kB 00:00 2024-06-02T22:57:15.368 INFO:teuthology.orchestra.run.smithi052.stdout:(113/159): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-06-02T22:57:15.386 INFO:teuthology.orchestra.run.smithi123.stdout:(149/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 6.7 MB/s | 3.1 MB 00:00 2024-06-02T22:57:15.394 INFO:teuthology.orchestra.run.smithi174.stdout:(34/159): ceph-prometheus-alerts-19.0.0-2946.g3 329 kB/s | 16 kB 00:00 2024-06-02T22:57:15.402 INFO:teuthology.orchestra.run.smithi052.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-06-02T22:57:15.427 INFO:teuthology.orchestra.run.smithi052.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 742 kB/s | 43 kB 00:00 2024-06-02T22:57:15.436 INFO:teuthology.orchestra.run.smithi123.stdout:(150/159): NetworkManager-1.47.91-1.el9.x86_64. 11 MB/s | 2.3 MB 00:00 2024-06-02T22:57:15.452 INFO:teuthology.orchestra.run.smithi052.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-06-02T22:57:15.470 INFO:teuthology.orchestra.run.smithi123.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-02T22:57:15.478 INFO:teuthology.orchestra.run.smithi052.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 299 kB/s | 15 kB 00:00 2024-06-02T22:57:15.485 INFO:teuthology.orchestra.run.smithi174.stdout:(35/159): ceph-volume-19.0.0-2946.g3d89accf.el9 2.8 MB/s | 264 kB 00:00 2024-06-02T22:57:15.512 INFO:teuthology.orchestra.run.smithi123.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.8 MB/s | 248 kB 00:00 2024-06-02T22:57:15.520 INFO:teuthology.orchestra.run.smithi052.stdout:(118/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.3 MB/s | 839 kB 00:00 2024-06-02T22:57:15.545 INFO:teuthology.orchestra.run.smithi052.stdout:(119/159): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-06-02T22:57:15.554 INFO:teuthology.orchestra.run.smithi123.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 884 kB/s | 37 kB 00:00 2024-06-02T22:57:15.579 INFO:teuthology.orchestra.run.smithi052.stdout:(120/159): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-06-02T22:57:15.596 INFO:teuthology.orchestra.run.smithi123.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-02T22:57:15.604 INFO:teuthology.orchestra.run.smithi052.stdout:(121/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.5 MB/s | 303 kB 00:00 2024-06-02T22:57:15.629 INFO:teuthology.orchestra.run.smithi123.stdout:(155/159): NetworkManager-libnm-1.47.91-1.el9.x 7.4 MB/s | 1.8 MB 00:00 2024-06-02T22:57:15.630 INFO:teuthology.orchestra.run.smithi052.stdout:(122/159): python3-google-auth-2.29.0-1.el9.noa 2.4 MB/s | 206 kB 00:00 2024-06-02T22:57:15.644 INFO:teuthology.orchestra.run.smithi174.stdout:(36/159): cephadm-19.0.0-2946.g3d89accf.el9.noa 4.7 MB/s | 763 kB 00:00 2024-06-02T22:57:15.654 INFO:teuthology.orchestra.run.smithi123.stdout:(156/159): elfutils-libs-0.191-4.el9.x86_64.rpm 4.3 MB/s | 260 kB 00:00 2024-06-02T22:57:15.694 INFO:teuthology.orchestra.run.smithi174.stdout:(37/159): cephfs-top-19.0.0-2946.g3d89accf.el9. 366 kB/s | 18 kB 00:00 2024-06-02T22:57:15.696 INFO:teuthology.orchestra.run.smithi123.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 20 MB/s | 1.3 MB 00:00 2024-06-02T22:57:15.747 INFO:teuthology.orchestra.run.smithi123.stdout:(158/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 26 MB/s | 2.4 MB 00:00 2024-06-02T22:57:15.771 INFO:teuthology.orchestra.run.smithi052.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 865 kB/s | 144 kB 00:00 2024-06-02T22:57:15.797 INFO:teuthology.orchestra.run.smithi052.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-06-02T22:57:15.828 INFO:teuthology.orchestra.run.smithi174.stdout:(38/159): ceph-mgr-diskprediction-local-19.0.0- 7.2 MB/s | 7.4 MB 00:01 2024-06-02T22:57:15.838 INFO:teuthology.orchestra.run.smithi052.stdout:(125/159): python3-grpcio-1.46.7-10.el9.x86_64. 7.9 MB/s | 2.0 MB 00:00 2024-06-02T22:57:15.855 INFO:teuthology.orchestra.run.smithi052.stdout:(126/159): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-06-02T22:57:15.861 INFO:teuthology.orchestra.run.smithi174.stdout:(39/159): fuse-2.9.9-15.el9.x86_64.rpm 482 kB/s | 80 kB 00:00 2024-06-02T22:57:15.872 INFO:teuthology.orchestra.run.smithi052.stdout:(127/159): python3-jaraco-collections-3.0.0-8.e 308 kB/s | 23 kB 00:00 2024-06-02T22:57:15.898 INFO:teuthology.orchestra.run.smithi052.stdout:(128/159): python3-jaraco-functools-3.5.0-2.el9 331 kB/s | 19 kB 00:00 2024-06-02T22:57:15.923 INFO:teuthology.orchestra.run.smithi052.stdout:(129/159): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-06-02T22:57:15.949 INFO:teuthology.orchestra.run.smithi052.stdout:(130/159): python3-logutils-0.3.5-21.el9.noarch 922 kB/s | 46 kB 00:00 2024-06-02T22:57:15.978 INFO:teuthology.orchestra.run.smithi174.stdout:(40/159): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-06-02T22:57:15.982 INFO:teuthology.orchestra.run.smithi052.stdout:(131/159): python3-kubernetes-26.1.0-2.el9.noar 9.4 MB/s | 1.0 MB 00:00 2024-06-02T22:57:15.999 INFO:teuthology.orchestra.run.smithi052.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-02T22:57:16.024 INFO:teuthology.orchestra.run.smithi052.stdout:(133/159): python3-more-itertools-8.12.0-2.el9. 784 kB/s | 79 kB 00:00 2024-06-02T22:57:16.050 INFO:teuthology.orchestra.run.smithi052.stdout:(134/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-06-02T22:57:16.070 INFO:teuthology.orchestra.run.smithi174.stdout:(41/159): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-06-02T22:57:16.074 INFO:teuthology.orchestra.run.smithi052.stdout:(135/159): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-06-02T22:57:16.096 INFO:teuthology.orchestra.run.smithi174.stdout:(42/159): libgfortran-11.4.1-3.el9.x86_64.rpm 2.9 MB/s | 803 kB 00:00 2024-06-02T22:57:16.097 INFO:teuthology.orchestra.run.smithi123.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 2.6 MB/s | 3.4 MB 00:01 2024-06-02T22:57:16.100 INFO:teuthology.orchestra.run.smithi052.stdout:(136/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-06-02T22:57:16.100 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:57:16.100 INFO:teuthology.orchestra.run.smithi123.stdout:Total 15 MB/s | 225 MB 00:15 2024-06-02T22:57:16.117 INFO:teuthology.orchestra.run.smithi052.stdout:(137/159): python3-pyperclip-1.8.2-2.el9.noarch 421 kB/s | 28 kB 00:00 2024-06-02T22:57:16.142 INFO:teuthology.orchestra.run.smithi052.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 458 kB/s | 31 kB 00:00 2024-06-02T22:57:16.146 INFO:teuthology.orchestra.run.smithi174.stdout:(43/159): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-06-02T22:57:16.167 INFO:teuthology.orchestra.run.smithi052.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.7 MB/s | 188 kB 00:00 2024-06-02T22:57:16.188 INFO:teuthology.orchestra.run.smithi174.stdout:(44/159): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-06-02T22:57:16.193 INFO:teuthology.orchestra.run.smithi052.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-06-02T22:57:16.205 INFO:teuthology.orchestra.run.smithi174.stdout:(45/159): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-06-02T22:57:16.218 INFO:teuthology.orchestra.run.smithi052.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-06-02T22:57:16.239 INFO:teuthology.orchestra.run.smithi174.stdout:(46/159): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-06-02T22:57:16.243 INFO:teuthology.orchestra.run.smithi052.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-06-02T22:57:16.269 INFO:teuthology.orchestra.run.smithi052.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-02T22:57:16.272 INFO:teuthology.orchestra.run.smithi174.stdout:(47/159): python3-urllib3-1.26.5-5.el9.noarch.r 3.1 MB/s | 215 kB 00:00 2024-06-02T22:57:16.294 INFO:teuthology.orchestra.run.smithi052.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-06-02T22:57:16.314 INFO:teuthology.orchestra.run.smithi174.stdout:(48/159): automake-1.16.2-8.el9.noarch.rpm 9.0 MB/s | 689 kB 00:00 2024-06-02T22:57:16.319 INFO:teuthology.orchestra.run.smithi052.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-06-02T22:57:16.344 INFO:teuthology.orchestra.run.smithi052.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 265 kB/s | 20 kB 00:00 2024-06-02T22:57:16.356 INFO:teuthology.orchestra.run.smithi174.stdout:(49/159): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-06-02T22:57:16.370 INFO:teuthology.orchestra.run.smithi052.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-06-02T22:57:16.381 INFO:teuthology.orchestra.run.smithi174.stdout:(50/159): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-06-02T22:57:16.412 INFO:teuthology.orchestra.run.smithi052.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-06-02T22:57:16.507 INFO:teuthology.orchestra.run.smithi174.stdout:(51/159): elfutils-libelf-devel-0.191-4.el9.x86 144 kB/s | 22 kB 00:00 2024-06-02T22:57:16.532 INFO:teuthology.orchestra.run.smithi174.stdout:(52/159): flex-2.6.4-9.el9.x86_64.rpm 2.0 MB/s | 313 kB 00:00 2024-06-02T22:57:16.557 INFO:teuthology.orchestra.run.smithi174.stdout:(53/159): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-06-02T22:57:16.562 INFO:teuthology.orchestra.run.smithi052.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-06-02T22:57:16.599 INFO:teuthology.orchestra.run.smithi174.stdout:(54/159): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-06-02T22:57:16.629 INFO:teuthology.orchestra.run.smithi052.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-06-02T22:57:16.649 INFO:teuthology.orchestra.run.smithi174.stdout:(55/159): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-06-02T22:57:16.663 INFO:teuthology.orchestra.run.smithi052.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 1.2 MB/s | 41 kB 00:00 2024-06-02T22:57:16.666 INFO:teuthology.orchestra.run.smithi174.stdout:(56/159): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-06-02T22:57:16.691 INFO:teuthology.orchestra.run.smithi174.stdout:(57/159): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.8 MB/s | 160 kB 00:00 2024-06-02T22:57:16.704 INFO:teuthology.orchestra.run.smithi052.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 5.9 MB/s | 248 kB 00:00 2024-06-02T22:57:16.708 INFO:teuthology.orchestra.run.smithi174.stdout:(58/159): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-06-02T22:57:16.738 INFO:teuthology.orchestra.run.smithi052.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 1.1 MB/s | 37 kB 00:00 2024-06-02T22:57:16.742 INFO:teuthology.orchestra.run.smithi174.stdout:(59/159): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 662 kB 00:00 2024-06-02T22:57:16.776 INFO:teuthology.orchestra.run.smithi174.stdout:(60/159): libtool-2.4.6-45.el9.x86_64.rpm 8.4 MB/s | 582 kB 00:00 2024-06-02T22:57:16.780 INFO:teuthology.orchestra.run.smithi052.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.9 MB/s | 207 kB 00:00 2024-06-02T22:57:16.801 INFO:teuthology.orchestra.run.smithi174.stdout:(61/159): libuuid-devel-2.37.4-18.el9.x86_64.rp 432 kB/s | 25 kB 00:00 2024-06-02T22:57:16.814 INFO:teuthology.orchestra.run.smithi052.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 7.6 MB/s | 260 kB 00:00 2024-06-02T22:57:16.827 INFO:teuthology.orchestra.run.smithi174.stdout:(62/159): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-06-02T22:57:16.852 INFO:teuthology.orchestra.run.smithi174.stdout:(63/159): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-06-02T22:57:16.873 INFO:teuthology.orchestra.run.smithi052.stdout:(156/159): openssl-3.2.1-1.el9.x86_64.rpm 23 MB/s | 1.3 MB 00:00 2024-06-02T22:57:16.877 INFO:teuthology.orchestra.run.smithi174.stdout:(64/159): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-06-02T22:57:16.903 INFO:teuthology.orchestra.run.smithi174.stdout:(65/159): openblas-0.3.21-2.el9.x86_64.rpm 702 kB/s | 35 kB 00:00 2024-06-02T22:57:16.939 INFO:teuthology.orchestra.run.smithi052.stdout:(157/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 36 MB/s | 2.4 MB 00:00 2024-06-02T22:57:17.070 INFO:teuthology.orchestra.run.smithi174.stdout:(66/159): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-06-02T22:57:17.137 INFO:teuthology.orchestra.run.smithi174.stdout:(67/159): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-06-02T22:57:17.179 INFO:teuthology.orchestra.run.smithi174.stdout:(68/159): postgresql-13.14-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-06-02T22:57:17.212 INFO:teuthology.orchestra.run.smithi174.stdout:(69/159): postgresql-contrib-13.14-1.el9.x86_64 12 MB/s | 911 kB 00:00 2024-06-02T22:57:17.238 INFO:teuthology.orchestra.run.smithi174.stdout:(70/159): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-06-02T22:57:17.305 INFO:teuthology.orchestra.run.smithi174.stdout:(71/159): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-06-02T22:57:17.312 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-06-02T22:57:17.380 INFO:teuthology.orchestra.run.smithi174.stdout:(72/159): postgresql-server-13.14-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-06-02T22:57:17.398 INFO:teuthology.orchestra.run.smithi052.stdout:(158/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 3.0 MB/s | 3.1 MB 00:01 2024-06-02T22:57:17.405 INFO:teuthology.orchestra.run.smithi174.stdout:(73/159): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-06-02T22:57:17.447 INFO:teuthology.orchestra.run.smithi174.stdout:(74/159): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-06-02T22:57:17.465 INFO:teuthology.orchestra.run.smithi052.stdout:(159/159): librados2-19.0.0-2946.g3d89accf.el9. 3.0 MB/s | 3.4 MB 00:01 2024-06-02T22:57:17.468 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:57:17.468 INFO:teuthology.orchestra.run.smithi052.stdout:Total 15 MB/s | 225 MB 00:15 2024-06-02T22:57:17.468 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-06-02T22:57:17.469 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-06-02T22:57:17.522 INFO:teuthology.orchestra.run.smithi174.stdout:(75/159): python3-babel-2.9.1-2.el9.noarch.rpm 42 MB/s | 6.0 MB 00:00 2024-06-02T22:57:17.547 INFO:teuthology.orchestra.run.smithi174.stdout:(76/159): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-06-02T22:57:17.573 INFO:teuthology.orchestra.run.smithi174.stdout:(77/159): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-06-02T22:57:17.598 INFO:teuthology.orchestra.run.smithi174.stdout:(78/159): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-06-02T22:57:17.623 INFO:teuthology.orchestra.run.smithi174.stdout:(79/159): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-06-02T22:57:17.649 INFO:teuthology.orchestra.run.smithi174.stdout:(80/159): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-06-02T22:57:17.674 INFO:teuthology.orchestra.run.smithi174.stdout:(81/159): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-06-02T22:57:17.824 INFO:teuthology.orchestra.run.smithi174.stdout:(82/159): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-06-02T22:57:17.950 INFO:teuthology.orchestra.run.smithi174.stdout:(83/159): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-06-02T22:57:17.975 INFO:teuthology.orchestra.run.smithi174.stdout:(84/159): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-06-02T22:57:18.016 INFO:teuthology.orchestra.run.smithi174.stdout:(85/159): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-06-02T22:57:18.042 INFO:teuthology.orchestra.run.smithi174.stdout:(86/159): python3-prettytable-0.7.2-27.el9.noar 634 kB/s | 42 kB 00:00 2024-06-02T22:57:18.092 INFO:teuthology.orchestra.run.smithi174.stdout:(87/159): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-06-02T22:57:18.118 INFO:teuthology.orchestra.run.smithi174.stdout:(88/159): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 159 kB 00:00 2024-06-02T22:57:18.193 INFO:teuthology.orchestra.run.smithi174.stdout:(89/159): python3-pytz-2021.1-5.el9.noarch.rpm 680 kB/s | 51 kB 00:00 2024-06-02T22:57:18.218 INFO:teuthology.orchestra.run.smithi174.stdout:(90/159): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-06-02T22:57:18.244 INFO:teuthology.orchestra.run.smithi174.stdout:(91/159): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-06-02T22:57:18.302 INFO:teuthology.orchestra.run.smithi174.stdout:(92/159): python3-toml-0.10.2-6.el9.noarch.rpm 716 kB/s | 42 kB 00:00 2024-06-02T22:57:18.369 INFO:teuthology.orchestra.run.smithi174.stdout:(93/159): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-06-02T22:57:18.486 INFO:teuthology.orchestra.run.smithi174.stdout:(94/159): qatlib-24.02.0-1.el9.x86_64.rpm 1.9 MB/s | 221 kB 00:00 2024-06-02T22:57:18.553 INFO:teuthology.orchestra.run.smithi174.stdout:(95/159): qatlib-service-24.02.0-1.el9.x86_64.r 541 kB/s | 36 kB 00:00 2024-06-02T22:57:18.697 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction check 2024-06-02T22:57:18.704 INFO:teuthology.orchestra.run.smithi174.stdout:(96/159): python3-scipy-1.6.2-8.el9.x86_64.rpm 34 MB/s | 16 MB 00:00 2024-06-02T22:57:18.728 INFO:teuthology.orchestra.run.smithi174.stdout:(97/159): qatzip-libs-1.2.0-1.el9.x86_64.rpm 271 kB/s | 47 kB 00:00 2024-06-02T22:57:18.770 INFO:teuthology.orchestra.run.smithi174.stdout:(98/159): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-06-02T22:57:18.796 INFO:teuthology.orchestra.run.smithi174.stdout:(99/159): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-06-02T22:57:18.838 INFO:teuthology.orchestra.run.smithi174.stdout:(100/159): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.4 MB/s | 94 kB 00:00 2024-06-02T22:57:18.863 INFO:teuthology.orchestra.run.smithi174.stdout:(101/159): xmlstarlet-1.6.1-20.el9.x86_64.rpm 951 kB/s | 64 kB 00:00 2024-06-02T22:57:18.871 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction check succeeded. 2024-06-02T22:57:18.871 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction test 2024-06-02T22:57:18.888 INFO:teuthology.orchestra.run.smithi174.stdout:(102/159): zlib-devel-1.2.11-41.el9.x86_64.rpm 890 kB/s | 45 kB 00:00 2024-06-02T22:57:19.030 INFO:teuthology.orchestra.run.smithi174.stdout:(103/159): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-06-02T22:57:19.205 INFO:teuthology.orchestra.run.smithi174.stdout:(104/159): protobuf-compiler-3.14.0-13.el9.x86_ 2.7 MB/s | 863 kB 00:00 2024-06-02T22:57:19.314 INFO:teuthology.orchestra.run.smithi174.stdout:(105/159): abseil-cpp-20211102.0-3.el9.x86_64.r 1.9 MB/s | 548 kB 00:00 2024-06-02T22:57:19.364 INFO:teuthology.orchestra.run.smithi174.stdout:(106/159): grpc-data-1.46.7-10.el9.noarch.rpm 391 kB/s | 19 kB 00:00 2024-06-02T22:57:19.440 INFO:teuthology.orchestra.run.smithi174.stdout:(107/159): gperftools-libs-2.9.1-2.el9.x86_64.r 1.3 MB/s | 309 kB 00:00 2024-06-02T22:57:19.490 INFO:teuthology.orchestra.run.smithi174.stdout:(108/159): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-06-02T22:57:19.565 INFO:teuthology.orchestra.run.smithi174.stdout:(109/159): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-06-02T22:57:19.575 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-06-02T22:57:19.575 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-06-02T22:57:19.582 INFO:teuthology.orchestra.run.smithi174.stdout:(110/159): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-06-02T22:57:19.624 INFO:teuthology.orchestra.run.smithi174.stdout:(111/159): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-06-02T22:57:19.649 INFO:teuthology.orchestra.run.smithi174.stdout:(112/159): luarocks-3.9.2-5.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-06-02T22:57:19.691 INFO:teuthology.orchestra.run.smithi174.stdout:(113/159): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-06-02T22:57:19.741 INFO:teuthology.orchestra.run.smithi174.stdout:(114/159): python3-asyncssh-2.13.2-5.el9.noarch 5.8 MB/s | 548 kB 00:00 2024-06-02T22:57:19.767 INFO:teuthology.orchestra.run.smithi174.stdout:(115/159): python3-bcrypt-3.2.2-1.el9.x86_64.rp 579 kB/s | 43 kB 00:00 2024-06-02T22:57:19.793 INFO:teuthology.orchestra.run.smithi174.stdout:(116/159): python3-cachetools-4.2.4-1.el9.noarc 641 kB/s | 32 kB 00:00 2024-06-02T22:57:19.817 INFO:teuthology.orchestra.run.smithi174.stdout:(117/159): python3-certifi-2021.10.8-2.el9.noar 297 kB/s | 15 kB 00:00 2024-06-02T22:57:19.859 INFO:teuthology.orchestra.run.smithi174.stdout:(118/159): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-06-02T22:57:19.885 INFO:teuthology.orchestra.run.smithi174.stdout:(119/159): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-06-02T22:57:20.277 INFO:teuthology.orchestra.run.smithi174.stdout:(120/159): ceph-test-19.0.0-2946.g3d89accf.el9. 6.0 MB/s | 48 MB 00:08 2024-06-02T22:57:20.302 INFO:teuthology.orchestra.run.smithi174.stdout:(121/159): python3-google-auth-2.29.0-1.el9.noa 493 kB/s | 206 kB 00:00 2024-06-02T22:57:20.327 INFO:teuthology.orchestra.run.smithi174.stdout:(122/159): python3-cmd2-2.4.2-2.el9.noarch.rpm 648 kB/s | 303 kB 00:00 2024-06-02T22:57:20.361 INFO:teuthology.orchestra.run.smithi174.stdout:(123/159): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-06-02T22:57:20.386 INFO:teuthology.orchestra.run.smithi174.stdout:(124/159): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-06-02T22:57:20.411 INFO:teuthology.orchestra.run.smithi174.stdout:(125/159): python3-jaraco-classes-3.2.1-5.el9.n 353 kB/s | 18 kB 00:00 2024-06-02T22:57:20.437 INFO:teuthology.orchestra.run.smithi174.stdout:(126/159): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-06-02T22:57:20.462 INFO:teuthology.orchestra.run.smithi174.stdout:(127/159): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-06-02T22:57:20.496 INFO:teuthology.orchestra.run.smithi174.stdout:(128/159): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-06-02T22:57:20.546 INFO:teuthology.orchestra.run.smithi174.stdout:(129/159): python3-logutils-0.3.5-21.el9.noarch 929 kB/s | 46 kB 00:00 2024-06-02T22:57:20.579 INFO:teuthology.orchestra.run.smithi174.stdout:(130/159): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-06-02T22:57:20.605 INFO:teuthology.orchestra.run.smithi174.stdout:(131/159): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-06-02T22:57:20.630 INFO:teuthology.orchestra.run.smithi174.stdout:(132/159): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-06-02T22:57:20.664 INFO:teuthology.orchestra.run.smithi174.stdout:(133/159): python3-pecan-1.4.2-3.el9.noarch.rpm 4.5 MB/s | 272 kB 00:00 2024-06-02T22:57:20.689 INFO:teuthology.orchestra.run.smithi174.stdout:(134/159): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-06-02T22:57:20.731 INFO:teuthology.orchestra.run.smithi174.stdout:(135/159): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-06-02T22:57:20.756 INFO:teuthology.orchestra.run.smithi174.stdout:(136/159): python3-pyperclip-1.8.2-2.el9.noarch 422 kB/s | 28 kB 00:00 2024-06-02T22:57:20.781 INFO:teuthology.orchestra.run.smithi174.stdout:(137/159): python3-pyOpenSSL-21.0.0-1.el9.noarc 766 kB/s | 90 kB 00:00 2024-06-02T22:57:20.807 INFO:teuthology.orchestra.run.smithi174.stdout:(138/159): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-06-02T22:57:20.832 INFO:teuthology.orchestra.run.smithi174.stdout:(139/159): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-06-02T22:57:20.857 INFO:teuthology.orchestra.run.smithi174.stdout:(140/159): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-06-02T22:57:20.882 INFO:teuthology.orchestra.run.smithi174.stdout:(141/159): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-06-02T22:57:20.908 INFO:teuthology.orchestra.run.smithi174.stdout:(142/159): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-06-02T22:57:20.933 INFO:teuthology.orchestra.run.smithi174.stdout:(143/159): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-06-02T22:57:20.950 INFO:teuthology.orchestra.run.smithi174.stdout:(144/159): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-06-02T22:57:20.975 INFO:teuthology.orchestra.run.smithi174.stdout:(145/159): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-06-02T22:57:21.000 INFO:teuthology.orchestra.run.smithi174.stdout:(146/159): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-06-02T22:57:21.026 INFO:teuthology.orchestra.run.smithi174.stdout:(147/159): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-06-02T22:57:21.076 INFO:teuthology.orchestra.run.smithi174.stdout:(148/159): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-06-02T22:57:21.093 INFO:teuthology.orchestra.run.smithi052.stdout:Transaction test succeeded. 2024-06-02T22:57:21.094 INFO:teuthology.orchestra.run.smithi052.stdout:Running transaction 2024-06-02T22:57:21.277 INFO:teuthology.orchestra.run.smithi174.stdout:(149/159): NetworkManager-1.47.91-1.el9.x86_64. 12 MB/s | 2.3 MB 00:00 2024-06-02T22:57:21.368 INFO:teuthology.orchestra.run.smithi174.stdout:(150/159): NetworkManager-libnm-1.47.91-1.el9.x 20 MB/s | 1.8 MB 00:00 2024-06-02T22:57:21.410 INFO:teuthology.orchestra.run.smithi174.stdout:(151/159): NetworkManager-team-1.47.91-1.el9.x8 997 kB/s | 41 kB 00:00 2024-06-02T22:57:21.461 INFO:teuthology.orchestra.run.smithi174.stdout:(152/159): NetworkManager-tui-1.47.91-1.el9.x86 4.8 MB/s | 248 kB 00:00 2024-06-02T22:57:21.503 INFO:teuthology.orchestra.run.smithi174.stdout:(153/159): elfutils-debuginfod-client-0.191-4.e 881 kB/s | 37 kB 00:00 2024-06-02T22:57:21.553 INFO:teuthology.orchestra.run.smithi174.stdout:(154/159): elfutils-libelf-0.191-4.el9.x86_64.r 4.1 MB/s | 207 kB 00:00 2024-06-02T22:57:21.645 INFO:teuthology.orchestra.run.smithi174.stdout:(155/159): elfutils-libs-0.191-4.el9.x86_64.rpm 2.8 MB/s | 260 kB 00:00 2024-06-02T22:57:22.395 INFO:teuthology.orchestra.run.smithi174.stdout:(156/159): librados2-19.0.0-2946.g3d89accf.el9. 2.4 MB/s | 3.4 MB 00:01 2024-06-02T22:57:22.429 INFO:teuthology.orchestra.run.smithi174.stdout:(157/159): openssl-3.2.1-1.el9.x86_64.rpm 1.7 MB/s | 1.3 MB 00:00 2024-06-02T22:57:22.487 INFO:teuthology.orchestra.run.smithi174.stdout:(158/159): librbd1-19.0.0-2946.g3d89accf.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-06-02T22:57:22.662 INFO:teuthology.orchestra.run.smithi174.stdout:(159/159): openssl-libs-3.2.1-1.el9.x86_64.rpm 9.0 MB/s | 2.4 MB 00:00 2024-06-02T22:57:22.666 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:57:22.667 INFO:teuthology.orchestra.run.smithi174.stdout:Total 12 MB/s | 225 MB 00:18 2024-06-02T22:57:23.911 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-06-02T22:57:24.069 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-06-02T22:57:24.070 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-06-02T22:57:25.937 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-06-02T22:57:26.035 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-02T22:57:26.077 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-02T22:57:26.139 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-02T22:57:26.177 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-02T22:57:26.202 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-02T22:57:26.229 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-06-02T22:57:26.229 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-06-02T22:57:26.252 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-02T22:57:26.593 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-02T22:57:26.703 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:57:26.833 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:57:26.863 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:57:26.957 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:57:26.984 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-02T22:57:27.026 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-02T22:57:27.205 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-02T22:57:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-02T22:57:27.302 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:57:27.405 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:57:27.427 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-02T22:57:27.490 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-02T22:57:27.512 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T22:57:27.532 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T22:57:27.653 INFO:teuthology.orchestra.run.smithi052.stdout: Preparing : 1/1 2024-06-02T22:57:27.729 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T22:57:27.753 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-02T22:57:27.792 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-02T22:57:27.866 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-02T22:57:27.904 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-02T22:57:27.924 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T22:57:27.929 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-02T22:57:27.995 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-02T22:57:28.096 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T22:57:28.153 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T22:57:28.261 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:28.338 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:28.349 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-02T22:57:28.464 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:57:28.538 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:28.571 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:57:28.598 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:57:28.666 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-02T22:57:28.693 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:57:28.719 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-02T22:57:28.763 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-02T22:57:28.764 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-02T22:57:28.802 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-02T22:57:28.836 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-02T22:57:28.909 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-02T22:57:28.944 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-02T22:57:28.969 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-02T22:57:28.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-02T22:57:29.018 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-02T22:57:29.020 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-02T22:57:29.048 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-02T22:57:29.067 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:57:29.116 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-02T22:57:29.170 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-02T22:57:29.176 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:57:29.201 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-02T22:57:29.204 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-02T22:57:29.244 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-02T22:57:29.267 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-02T22:57:29.289 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T22:57:29.315 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T22:57:29.320 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-02T22:57:29.355 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-02T22:57:29.399 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-02T22:57:29.427 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-02T22:57:29.459 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T22:57:29.495 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-02T22:57:29.550 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-02T22:57:29.624 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-02T22:57:29.644 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T22:57:29.758 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-02T22:57:29.778 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T22:57:29.811 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-02T22:57:29.822 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T22:57:29.928 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:29.998 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:30.227 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:30.333 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-02T22:57:30.344 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-02T22:57:30.415 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-02T22:57:30.444 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-02T22:57:30.446 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-02T22:57:30.479 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-02T22:57:30.481 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-02T22:57:30.529 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-02T22:57:30.531 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-02T22:57:30.567 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-02T22:57:30.607 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-02T22:57:30.608 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-02T22:57:30.656 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-02T22:57:30.699 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-02T22:57:30.731 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-02T22:57:30.759 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-02T22:57:30.766 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-02T22:57:30.828 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-02T22:57:30.831 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-02T22:57:30.882 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-02T22:57:30.883 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-02T22:57:30.920 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-02T22:57:30.955 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-02T22:57:30.964 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-02T22:57:30.987 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-02T22:57:31.024 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-02T22:57:31.043 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-02T22:57:31.074 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-02T22:57:31.078 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-02T22:57:31.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-02T22:57:31.118 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-02T22:57:31.146 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-02T22:57:31.157 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-02T22:57:31.199 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-02T22:57:31.221 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-02T22:57:31.278 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-02T22:57:31.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-02T22:57:31.357 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-02T22:57:31.460 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-02T22:57:31.511 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-02T22:57:31.574 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-02T22:57:31.670 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-02T22:57:31.766 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-02T22:57:31.803 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-02T22:57:31.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-02T22:57:31.875 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-02T22:57:31.900 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-02T22:57:31.991 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-02T22:57:32.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-02T22:57:32.138 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-02T22:57:32.151 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-02T22:57:32.179 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-02T22:57:32.213 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-02T22:57:32.238 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-02T22:57:32.252 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-02T22:57:32.280 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-02T22:57:32.283 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-02T22:57:32.313 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-02T22:57:32.325 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-02T22:57:32.348 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-02T22:57:32.381 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-02T22:57:32.382 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-02T22:57:32.420 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-02T22:57:32.423 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-02T22:57:32.457 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-02T22:57:32.463 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-02T22:57:32.544 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-02T22:57:32.582 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-06-02T22:57:32.594 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-02T22:57:32.618 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-02T22:57:32.636 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-02T22:57:32.673 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-02T22:57:32.675 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/170 2024-06-02T22:57:32.682 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-02T22:57:32.715 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/170 2024-06-02T22:57:32.730 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-02T22:57:32.780 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-02T22:57:32.791 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/170 2024-06-02T22:57:32.805 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-02T22:57:32.825 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/170 2024-06-02T22:57:32.829 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-02T22:57:32.849 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-02T22:57:32.886 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-02T22:57:32.892 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/170 2024-06-02T22:57:32.934 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-02T22:57:32.985 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-02T22:57:33.028 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-02T22:57:33.070 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-02T22:57:33.143 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/170 2024-06-02T22:57:33.215 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-02T22:57:33.255 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-02T22:57:33.297 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-02T22:57:33.396 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-02T22:57:33.438 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-02T22:57:33.479 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-02T22:57:33.488 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/170 2024-06-02T22:57:33.511 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-02T22:57:33.536 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-02T22:57:33.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-02T22:57:33.592 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-02T22:57:33.593 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:57:33.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-02T22:57:33.627 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-02T22:57:33.716 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:57:33.724 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-02T22:57:33.744 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:57:33.773 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-02T22:57:33.814 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-02T22:57:33.818 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-02T22:57:33.845 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:57:33.854 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-02T22:57:33.874 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-02T22:57:33.882 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-02T22:57:33.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-02T22:57:33.917 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/170 2024-06-02T22:57:33.925 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-02T22:57:33.937 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-02T22:57:33.965 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-02T22:57:34.005 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-02T22:57:34.034 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-02T22:57:34.068 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-02T22:57:34.102 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-02T22:57:34.104 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : elfutils-libelf-0.191-4.el9.x86_64 12/170 2024-06-02T22:57:34.114 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-02T22:57:34.151 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-02T22:57:34.162 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.14-1.el9.x86_64 13/170 2024-06-02T22:57:34.200 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:57:34.230 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-02T22:57:34.277 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-02T22:57:34.304 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:57:34.334 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-02T22:57:34.397 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 15/170 2024-06-02T22:57:34.418 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T22:57:34.443 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/170 2024-06-02T22:57:34.459 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-02T22:57:34.614 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-02T22:57:34.629 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T22:57:34.660 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-02T22:57:34.698 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-02T22:57:34.747 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-02T22:57:34.805 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/170 2024-06-02T22:57:34.858 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-02T22:57:34.900 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-02T22:57:34.909 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-02T22:57:34.965 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T22:57:35.009 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 18/170 2024-06-02T22:57:35.015 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-02T22:57:35.119 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:35.194 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:35.403 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.47.91-1.el9.x86_64 19/170 2024-06-02T22:57:35.512 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 20/170 2024-06-02T22:57:35.603 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/170 2024-06-02T22:57:35.650 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 22/170 2024-06-02T22:57:35.690 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-02T22:57:35.692 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 23/170 2024-06-02T22:57:35.738 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-02T22:57:35.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 24/170 2024-06-02T22:57:35.772 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-02T22:57:35.801 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 25/170 2024-06-02T22:57:35.844 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/170 2024-06-02T22:57:35.878 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-02T22:57:35.878 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/170 2024-06-02T22:57:35.905 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/170 2024-06-02T22:57:35.929 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-02T22:57:35.974 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-02T22:57:35.974 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/170 2024-06-02T22:57:36.028 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 30/170 2024-06-02T22:57:36.059 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/170 2024-06-02T22:57:36.065 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-02T22:57:36.100 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-02T22:57:36.111 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 32/170 2024-06-02T22:57:36.187 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 33/170 2024-06-02T22:57:36.221 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 34/170 2024-06-02T22:57:36.265 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 35/170 2024-06-02T22:57:36.265 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-02T22:57:36.278 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-02T22:57:36.293 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 36/170 2024-06-02T22:57:36.314 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-02T22:57:36.355 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-02T22:57:36.363 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/170 2024-06-02T22:57:36.382 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-02T22:57:36.395 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-02T22:57:36.408 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/170 2024-06-02T22:57:36.415 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-02T22:57:36.483 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/170 2024-06-02T22:57:36.615 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/170 2024-06-02T22:57:36.669 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/170 2024-06-02T22:57:36.742 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-02T22:57:36.779 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-02T22:57:36.878 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-02T22:57:36.908 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-02T22:57:36.945 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-02T22:57:37.101 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-02T22:57:37.183 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/170 2024-06-02T22:57:37.225 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-02T22:57:37.264 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/170 2024-06-02T22:57:37.293 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/170 2024-06-02T22:57:37.331 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/170 2024-06-02T22:57:37.371 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : elfutils-libs-0.191-4.el9.x86_64 46/170 2024-06-02T22:57:37.400 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 47/170 2024-06-02T22:57:37.417 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-02T22:57:37.440 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-02T22:57:37.441 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/170 2024-06-02T22:57:37.511 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-02T22:57:37.550 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-02T22:57:37.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 49/170 2024-06-02T22:57:37.680 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 50/170 2024-06-02T22:57:37.734 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 51/170 2024-06-02T22:57:37.807 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/170 2024-06-02T22:57:37.838 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 53/170 2024-06-02T22:57:37.875 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 54/170 2024-06-02T22:57:37.921 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/170 2024-06-02T22:57:37.957 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/170 2024-06-02T22:57:38.000 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/170 2024-06-02T22:57:38.042 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/170 2024-06-02T22:57:38.187 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/170 2024-06-02T22:57:38.236 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/170 2024-06-02T22:57:38.277 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 61/170 2024-06-02T22:57:38.372 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/170 2024-06-02T22:57:38.403 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 63/170 2024-06-02T22:57:38.444 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/170 2024-06-02T22:57:38.505 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/170 2024-06-02T22:57:38.534 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/170 2024-06-02T22:57:38.565 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-02T22:57:38.619 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/170 2024-06-02T22:57:38.652 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-02T22:57:38.680 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-02T22:57:38.713 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-02T22:57:38.744 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/170 2024-06-02T22:57:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/170 2024-06-02T22:57:38.802 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/170 2024-06-02T22:57:38.837 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/170 2024-06-02T22:57:38.876 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/170 2024-06-02T22:57:38.907 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/170 2024-06-02T22:57:38.937 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/170 2024-06-02T22:57:38.972 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/170 2024-06-02T22:57:39.013 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/170 2024-06-02T22:57:39.041 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-02T22:57:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/170 2024-06-02T22:57:39.084 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-02T22:57:39.089 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/170 2024-06-02T22:57:39.167 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-02T22:57:39.176 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/170 2024-06-02T22:57:39.197 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-02T22:57:39.222 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/170 2024-06-02T22:57:39.234 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-02T22:57:39.259 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/170 2024-06-02T22:57:39.307 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/170 2024-06-02T22:57:39.405 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 83/170 2024-06-02T22:57:39.453 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 84/170 2024-06-02T22:57:39.734 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-02T22:57:39.762 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-02T22:57:39.833 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-02T22:57:39.863 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-02T22:57:39.867 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:39.905 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:40.176 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 85/170 2024-06-02T22:57:40.224 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 86/170 2024-06-02T22:57:40.249 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 87/170 2024-06-02T22:57:40.348 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 88/170 2024-06-02T22:57:40.405 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 89/170 2024-06-02T22:57:40.450 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 90/170 2024-06-02T22:57:40.461 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:40.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-02T22:57:40.537 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/170 2024-06-02T22:57:40.568 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/170 2024-06-02T22:57:40.593 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-02T22:57:40.593 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-02T22:57:40.593 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-02T22:57:40.593 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:40.626 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-02T22:57:40.747 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/170 2024-06-02T22:57:40.775 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/170 2024-06-02T22:57:40.858 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/170 2024-06-02T22:57:41.508 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/170 2024-06-02T22:57:41.623 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/170 2024-06-02T22:57:42.300 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:42.335 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:42.899 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/170 2024-06-02T22:57:42.906 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:42.947 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-02T22:57:42.969 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/170 2024-06-02T22:57:42.997 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 100/170 2024-06-02T22:57:43.030 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 101/170 2024-06-02T22:57:43.036 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-02T22:57:43.036 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-02T22:57:43.036 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-02T22:57:43.037 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:43.065 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-02T22:57:43.346 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 102/170 2024-06-02T22:57:43.393 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-02T22:57:43.492 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 103/170 2024-06-02T22:57:43.522 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 104/170 2024-06-02T22:57:43.559 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 105/170 2024-06-02T22:57:44.046 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 106/170 2024-06-02T22:57:44.070 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-02T22:57:44.142 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 107/170 2024-06-02T22:57:44.172 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 108/170 2024-06-02T22:57:46.408 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:46.435 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:46.999 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 109/170 2024-06-02T22:57:47.030 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-02T22:57:47.118 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 110/170 2024-06-02T22:57:47.118 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-06-02T22:57:47.118 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-06-02T22:57:47.118 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:57:47.147 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-06-02T22:57:52.401 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:52.564 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-02T22:57:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-02T22:57:52.579 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:52.579 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-02T22:57:52.579 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:54.066 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-02T22:57:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-02T22:57:54.092 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:54.092 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-02T22:57:54.092 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T22:57:54.092 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T22:57:54.092 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:54.326 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-02T22:57:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-02T22:57:54.349 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:54.349 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-02T22:57:54.349 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T22:57:54.349 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T22:57:54.349 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:54.803 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-02T22:57:54.826 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-02T22:57:54.826 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:54.826 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-02T22:57:54.827 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T22:57:54.827 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T22:57:54.827 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:54.924 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-02T22:57:54.973 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-02T22:57:55.037 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-02T22:57:55.104 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-02T22:57:55.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /sys 2024-06-02T22:57:55.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /proc 2024-06-02T22:57:55.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /mnt 2024-06-02T22:57:55.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /var/tmp 2024-06-02T22:57:55.104 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /home 2024-06-02T22:57:55.105 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /root 2024-06-02T22:57:55.105 INFO:teuthology.orchestra.run.smithi052.stdout:skipping the directory /tmp 2024-06-02T22:57:55.105 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:55.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-02T22:57:55.270 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-02T22:57:55.271 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-02T22:57:55.287 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-02T22:57:55.288 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:55.288 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-02T22:57:55.288 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:55.347 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-02T22:57:55.435 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-02T22:57:55.590 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-02T22:57:55.628 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-02T22:57:55.669 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-02T22:57:55.696 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-02T22:57:56.151 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-02T22:57:56.178 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-02T22:57:56.209 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-02T22:57:56.272 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-02T22:57:56.301 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-02T22:57:56.834 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-02T22:57:56.862 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-02T22:57:56.862 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:56.862 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-02T22:57:56.863 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T22:57:56.863 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T22:57:56.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:57.101 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-02T22:57:57.125 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-02T22:57:57.125 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:57.125 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-02T22:57:57.125 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T22:57:57.126 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T22:57:57.126 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:57.339 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-02T22:57:57.369 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-02T22:57:57.602 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-02T22:57:57.626 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-02T22:57:57.626 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:57.626 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-02T22:57:57.626 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T22:57:57.626 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T22:57:57.627 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:57:57.733 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-02T22:57:57.783 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-02T22:57:57.847 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-02T22:57:58.043 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-02T22:57:58.096 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-02T22:57:58.168 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-02T22:57:58.266 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-02T22:57:58.331 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-02T22:57:58.353 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-02T22:57:58.410 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-02T22:57:58.455 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-02T22:57:58.462 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-02T22:57:58.494 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 111/170 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-06-02T22:57:58.497 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:57:58.521 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-02T22:57:58.619 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-02T22:57:58.650 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-02T22:57:58.668 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-02T22:57:58.674 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-02T22:57:58.674 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:58.674 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-02T22:57:58.674 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T22:57:58.674 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T22:57:58.674 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:58.684 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 112/170 2024-06-02T22:57:58.684 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:58.684 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-06-02T22:57:58.684 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:57:58.718 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-02T22:57:58.729 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-02T22:57:58.826 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-02T22:57:58.884 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-02T22:57:58.977 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-02T22:57:59.003 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-02T22:57:59.035 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-02T22:57:59.100 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-02T22:57:59.126 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-02T22:57:59.780 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-02T22:57:59.812 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-02T22:57:59.835 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-02T22:57:59.836 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:59.836 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-02T22:57:59.836 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T22:57:59.836 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T22:57:59.836 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:59.879 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-02T22:57:59.902 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-02T22:57:59.902 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:59.902 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-02T22:57:59.902 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:57:59.947 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-02T22:57:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-02T22:57:59.970 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:57:59.970 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-02T22:57:59.970 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T22:57:59.970 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T22:57:59.970 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:58:00.154 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-02T22:58:00.181 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 113/170 2024-06-02T22:58:00.181 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:00.181 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-06-02T22:58:00.181 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T22:58:00.181 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-06-02T22:58:00.181 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:00.223 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-02T22:58:00.255 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-02T22:58:00.272 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-02T22:58:00.295 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-02T22:58:00.295 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:00.295 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-02T22:58:00.295 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T22:58:00.295 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T22:58:00.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:58:00.404 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-02T22:58:00.427 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 114/170 2024-06-02T22:58:00.427 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:00.427 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-06-02T22:58:00.428 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T22:58:00.428 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-06-02T22:58:00.428 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:00.872 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-02T22:58:00.895 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 115/170 2024-06-02T22:58:00.895 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:00.895 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-06-02T22:58:00.895 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T22:58:00.895 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-06-02T22:58:00.895 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:00.993 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 116/170 2024-06-02T22:58:01.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 117/170 2024-06-02T22:58:01.122 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 118/170 2024-06-02T22:58:01.273 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-02T22:58:01.297 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-02T22:58:01.306 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 119/170 2024-06-02T22:58:01.347 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 120/170 2024-06-02T22:58:01.408 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-02T22:58:01.424 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/170 2024-06-02T22:58:01.521 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 122/170 2024-06-02T22:58:01.607 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-02T22:58:01.639 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-02T22:58:01.661 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 123/170 2024-06-02T22:58:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-02T22:58:01.664 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:01.664 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-02T22:58:01.664 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T22:58:01.664 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T22:58:01.664 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:01.697 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 124/170 2024-06-02T22:58:01.715 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-02T22:58:01.726 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-02T22:58:01.729 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 125/170 2024-06-02T22:58:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 126/170 2024-06-02T22:58:01.827 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-02T22:58:01.888 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-02T22:58:02.226 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-02T22:58:02.255 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 127/170 2024-06-02T22:58:02.286 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 128/170 2024-06-02T22:58:02.349 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 129/170 2024-06-02T22:58:02.378 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-02T22:58:02.839 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-02T22:58:02.869 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-02T22:58:02.891 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-02T22:58:02.891 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:02.891 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-02T22:58:02.891 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T22:58:02.891 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T22:58:02.891 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:02.942 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-02T22:58:02.966 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-02T22:58:02.966 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:02.966 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-02T22:58:02.966 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:03.010 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-02T22:58:03.034 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-02T22:58:03.034 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:03.034 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-02T22:58:03.034 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T22:58:03.034 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T22:58:03.034 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:03.352 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-02T22:58:03.376 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-02T22:58:03.377 INFO:teuthology.orchestra.run.smithi052.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:03.377 INFO:teuthology.orchestra.run.smithi052.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-02T22:58:03.377 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T22:58:03.377 INFO:teuthology.orchestra.run.smithi052.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T22:58:03.377 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:03.404 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 130/170 2024-06-02T22:58:03.429 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-02T22:58:04.389 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 131/170 2024-06-02T22:58:04.438 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-02T22:58:04.540 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 132/170 2024-06-02T22:58:04.709 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 133/170 2024-06-02T22:58:04.752 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-02T22:58:04.775 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-02T22:58:04.776 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 134/170 2024-06-02T22:58:04.776 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:04.776 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-06-02T22:58:04.776 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T22:58:04.776 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-06-02T22:58:04.776 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:04.817 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-02T22:58:04.828 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-02T22:58:04.840 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 135/170 2024-06-02T22:58:04.857 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-02T22:58:04.889 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-02T22:58:04.916 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-02T22:58:04.936 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 136/170 2024-06-02T22:58:05.001 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 137/170 2024-06-02T22:58:05.096 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-02T22:58:05.137 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-02T22:58:05.171 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-02T22:58:05.202 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-02T22:58:05.231 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-02T22:58:05.278 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T22:58:05.888 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-45.el9.x86_64 138/170 2024-06-02T22:58:05.923 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-02T22:58:05.927 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:05.946 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 139/170 2024-06-02T22:58:05.947 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:05.947 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-06-02T22:58:05.947 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T22:58:05.947 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-06-02T22:58:05.947 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:05.975 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:05.996 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:05.999 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-02T22:58:06.021 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 140/170 2024-06-02T22:58:06.021 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:06.022 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-06-02T22:58:06.022 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:06.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-02T22:58:06.089 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 141/170 2024-06-02T22:58:06.089 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:06.089 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-06-02T22:58:06.089 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T22:58:06.089 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-06-02T22:58:06.089 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:06.392 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-02T22:58:06.415 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 142/170 2024-06-02T22:58:06.415 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-06-02T22:58:06.415 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-06-02T22:58:06.415 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T22:58:06.415 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-06-02T22:58:06.415 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:06.841 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-02T22:58:06.938 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-02T22:58:06.979 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-02T22:58:07.068 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-02T22:58:07.100 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-02T22:58:07.124 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T22:58:07.150 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-02T22:58:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-02T22:58:07.176 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T22:58:07.252 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T22:58:07.285 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-02T22:58:07.285 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T22:58:07.369 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T22:58:07.394 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-02T22:58:07.400 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:07.400 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:07.588 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:07.589 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T22:58:07.673 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T22:58:07.697 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-02T22:58:07.762 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T22:58:08.180 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-02T22:58:08.231 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-02T22:58:08.272 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-02T22:58:08.303 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-02T22:58:08.330 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-02T22:58:08.540 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-02T22:58:08.584 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-02T22:58:08.617 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-02T22:58:08.657 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-02T22:58:08.687 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-02T22:58:08.733 INFO:teuthology.orchestra.run.smithi052.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T22:58:09.379 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:09.422 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:09.442 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:10.342 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-02T22:58:10.434 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-02T22:58:10.477 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-02T22:58:10.581 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-02T22:58:10.612 INFO:teuthology.orchestra.run.smithi052.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-02T22:58:10.637 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T22:58:10.662 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-02T22:58:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-02T22:58:10.689 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T22:58:10.773 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T22:58:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-02T22:58:10.806 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T22:58:10.899 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T22:58:10.924 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-02T22:58:10.928 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 143/170 2024-06-02T22:58:10.929 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:10.930 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:10.970 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 144/170 2024-06-02T22:58:11.010 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 145/170 2024-06-02T22:58:11.042 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 146/170 2024-06-02T22:58:11.069 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 147/170 2024-06-02T22:58:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:11.118 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T22:58:11.202 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T22:58:11.226 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-02T22:58:11.268 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 148/170 2024-06-02T22:58:11.296 INFO:teuthology.orchestra.run.smithi052.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T22:58:11.307 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 149/170 2024-06-02T22:58:11.340 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 150/170 2024-06-02T22:58:11.380 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : elfutils-debuginfod-client-0.191-4.el9.x86_64 151/170 2024-06-02T22:58:11.409 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-team-1:1.47.91-1.el9.x86_64 152/170 2024-06-02T22:58:11.456 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : NetworkManager-tui-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T22:58:12.065 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:12.111 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:12.132 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 154/170 2024-06-02T22:58:12.970 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 155/170 2024-06-02T22:58:13.049 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 156/170 2024-06-02T22:58:13.090 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 157/170 2024-06-02T22:58:13.187 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 158/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-02T22:58:13.197 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-02T22:58:13.198 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-02T22:58:13.199 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-02T22:58:13.200 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-02T22:58:13.201 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-02T22:58:13.202 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-02T22:58:13.203 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-02T22:58:13.204 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-02T22:58:13.205 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-02T22:58:13.206 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-02T22:58:13.207 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-02T22:58:13.208 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-02T22:58:13.209 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-02T22:58:13.210 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-02T22:58:13.227 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 159/170 2024-06-02T22:58:13.251 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T22:58:13.277 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : elfutils-debuginfod-client-0.190-2.el9.x86_64 161/170 2024-06-02T22:58:13.312 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : elfutils-libs-0.190-2.el9.x86_64 162/170 2024-06-02T22:58:13.312 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T22:58:13.379 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 163/170 2024-06-02T22:58:13.412 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 164/170 2024-06-02T22:58:13.412 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T22:58:13.497 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 165/170 2024-06-02T22:58:13.522 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 166/170 2024-06-02T22:58:13.533 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:13.533 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:13.715 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 167/170 2024-06-02T22:58:13.715 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T22:58:13.800 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 168/170 2024-06-02T22:58:13.824 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 169/170 2024-06-02T22:58:13.889 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T22:58:15.841 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-02T22:58:15.841 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:58:15.841 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-06-02T22:58:15.841 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-02T22:58:15.842 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.843 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.844 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.845 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-02T22:58:15.846 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.14-1.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-02T22:58:15.847 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-02T22:58:15.848 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-02T22:58:15.849 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-02T22:58:15.850 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-02T22:58:15.851 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-02T22:58:15.852 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-3.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-02T22:58:15.853 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-02T22:58:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-02T22:58:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-02T22:58:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-02T22:58:15.854 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-06-02T22:58:15.854 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-06-02T22:58:16.276 DEBUG:teuthology.parallel:result is None 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-02T22:58:16.781 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-02T22:58:16.782 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-02T22:58:16.783 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-02T22:58:16.784 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-02T22:58:16.785 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-02T22:58:16.786 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-02T22:58:16.787 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-02T22:58:16.788 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-02T22:58:16.789 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-02T22:58:16.790 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-02T22:58:16.791 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-02T22:58:16.792 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-02T22:58:16.793 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: elfutils-libelf-0.190-2.el9.x86_64 170/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 1/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 3/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 4/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2946.g3d89a 5/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 6/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 7/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 8/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 9/170 2024-06-02T22:58:19.354 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 10/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 11/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 12/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 13/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 14/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_ 15/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 16/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 17/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_6 18/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86 19/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 20/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el 21/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.0.0-2946.g3d89accf.el9. 22/170 2024-06-02T22:58:19.355 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_6 23/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 24/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 25/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 26/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 27/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 28/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 29/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf. 30/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noa 31/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.n 32/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d8 33/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el 34/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 35/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.e 36/170 2024-06-02T22:58:19.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 37/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 38/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 39/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/170 2024-06-02T22:58:19.357 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/170 2024-06-02T22:58:19.358 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 77/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/170 2024-06-02T22:58:19.359 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/170 2024-06-02T22:58:19.360 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 99/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 101/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/170 2024-06-02T22:58:19.361 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 107/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 118/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/170 2024-06-02T22:58:19.362 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/170 2024-06-02T22:58:19.363 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/170 2024-06-02T22:58:19.364 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 149/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 151/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.47.91-1.el9.x86_64 153/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.47.91-1.el9.x86_64 155/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.47.91-1.el9.x86_64 157/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.47.91-1.el9.x86_64 159/170 2024-06-02T22:58:19.365 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-debuginfod-client-0.191-4.el9.x86_64 161/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 162/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-0.191-4.el9.x86_64 163/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 164/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libs-0.191-4.el9.x86_64 165/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 166/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 167/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 168/170 2024-06-02T22:58:19.366 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 169/170 2024-06-02T22:58:19.451 INFO:teuthology.orchestra.run.smithi052.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-02T22:58:19.451 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:19.451 INFO:teuthology.orchestra.run.smithi052.stdout:Upgraded: 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout:Installed: 2024-06-02T22:58:19.452 INFO:teuthology.orchestra.run.smithi052.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: automake-1.16.2-8.el9.noarch 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: bison-3.7.4-5.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.453 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.454 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: flex-2.6.4-9.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-02T22:58:19.455 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-02T22:58:19.456 INFO:teuthology.orchestra.run.smithi052.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-02T22:58:19.457 INFO:teuthology.orchestra.run.smithi052.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: lua-5.4.4-4.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-13.14-1.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-02T22:58:19.458 INFO:teuthology.orchestra.run.smithi052.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-02T22:58:19.459 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-02T22:58:19.460 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-02T22:58:19.461 INFO:teuthology.orchestra.run.smithi052.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-ply-3.11-14.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-02T22:58:19.462 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.463 INFO:teuthology.orchestra.run.smithi052.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-02T22:58:19.464 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: re2-1:20211101-3.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-02T22:58:19.465 INFO:teuthology.orchestra.run.smithi052.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-02T22:58:19.466 INFO:teuthology.orchestra.run.smithi052.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-02T22:58:19.466 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:58:19.466 INFO:teuthology.orchestra.run.smithi052.stdout:Complete! 2024-06-02T22:58:19.879 DEBUG:teuthology.parallel:result is None 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 170/170 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-libnm-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-team-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:22.356 INFO:teuthology.orchestra.run.smithi174.stdout: NetworkManager-tui-1:1.47.91-1.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-debuginfod-client-0.191-4.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-0.191-4.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libs-0.191-4.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-06-02T22:58:22.357 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.358 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.359 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.0.0-2946.g3d89accf.el9.noarch 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-15.el9.x86_64 2024-06-02T22:58:22.360 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-06-02T22:58:22.361 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-45.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-06-02T22:58:22.362 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-5.el9.noarch 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.14-1.el9.x86_64 2024-06-02T22:58:22.363 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.14-1.el9.x86_64 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-06-02T22:58:22.364 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-06-02T22:58:22.365 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-06-02T22:58:22.366 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-06-02T22:58:22.366 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-06-02T22:58:22.366 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-06-02T22:58:22.366 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-06-02T22:58:22.366 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-06-02T22:58:22.391 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-06-02T22:58:22.392 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-06-02T22:58:22.393 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-06-02T22:58:22.394 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.0.0-2946.g3d89accf.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-06-02T22:58:22.395 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-06-02T22:58:22.396 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-06-02T22:58:22.396 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T22:58:22.396 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-06-02T22:58:22.774 DEBUG:teuthology.parallel:result is None 2024-06-02T22:58:22.775 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:58:22.919 DEBUG:teuthology.orchestra.run.smithi052:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-02T22:58:22.946 INFO:teuthology.orchestra.run.smithi052.stdout:19.0.0-2946.g3d89accf.el9 2024-06-02T22:58:22.946 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-02T22:58:22.946 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-02T22:58:22.948 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:58:23.092 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-02T22:58:23.118 INFO:teuthology.orchestra.run.smithi123.stdout:19.0.0-2946.g3d89accf.el9 2024-06-02T22:58:23.119 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-02T22:58:23.119 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-02T22:58:23.121 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:58:23.252 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-06-02T22:58:23.275 INFO:teuthology.orchestra.run.smithi174.stdout:19.0.0-2946.g3d89accf.el9 2024-06-02T22:58:23.276 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2946.g3d89accf.el9 2024-06-02T22:58:23.276 INFO:teuthology.task.install:The correct ceph version 19.0.0-2946.g3d89accf is installed. 2024-06-02T22:58:23.277 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-06-02T22:58:23.278 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:58:23.278 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-02T22:58:23.311 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T22:58:23.311 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-02T22:58:23.343 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T22:58:23.343 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-06-02T22:58:23.371 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-06-02T22:58:23.371 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:58:23.371 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/daemon-helper 2024-06-02T22:58:23.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-02T22:58:23.471 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T22:58:23.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-06-02T22:58:23.498 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-02T22:58:23.565 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T22:58:23.565 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-06-02T22:58:23.592 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-06-02T22:58:23.659 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-06-02T22:58:23.659 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:58:23.659 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-02T22:58:23.690 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-02T22:58:23.760 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T22:58:23.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-02T22:58:23.787 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-02T22:58:23.854 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T22:58:23.854 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-06-02T22:58:23.881 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-06-02T22:58:23.948 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-06-02T22:58:23.948 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:58:23.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/usr/bin/stdin-killer 2024-06-02T22:58:23.978 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-02T22:58:24.047 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T22:58:24.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-06-02T22:58:24.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-02T22:58:24.139 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T22:58:24.140 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-06-02T22:58:24.168 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-06-02T22:58:24.236 INFO:teuthology.run_tasks:Running task cephadm... 2024-06-02T22:58:24.331 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-02T22:58:24.332 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:58:24.332 INFO:tasks.cephadm:Cluster fsid is 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:58:24.332 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-06-02T22:58:24.332 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.52', 'mon.b': '172.21.15.123', 'mon.c': '172.21.15.174'} 2024-06-02T22:58:24.333 INFO:tasks.cephadm:First mon is mon.a on smithi052 2024-06-02T22:58:24.333 INFO:tasks.cephadm:First mgr is x 2024-06-02T22:58:24.333 INFO:tasks.cephadm:Normalizing hostnames... 2024-06-02T22:58:24.333 DEBUG:teuthology.orchestra.run.smithi052:> sudo hostname $(hostname -s) 2024-06-02T22:58:24.364 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-06-02T22:58:24.391 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-06-02T22:58:24.420 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-06-02T22:58:24.420 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:58:24.638 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '3d89accfe3201286f000e065a1faea2a88c7a6ba', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/55273/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2946-g3d89accf', 'node_name': '172.21.2.14+braggi14', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2946.g3d89accf'}, 'url': 'https://3.chacra.ceph.com/r/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-30 17:04:17.570799', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-06-02T22:58:24.860 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref squid, sha1 3d89accfe3201286f000e065a1faea2a88c7a6ba from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:58:24.861 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm 2024-06-02T22:58:24.862 DEBUG:teuthology.orchestra.run.smithi052:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:25.272 INFO:teuthology.orchestra.run.smithi052.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 2 22:58 /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:25.273 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:25.858 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 2 22:58 /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:25.858 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/squid/3d89accfe3201286f000e065a1faea2a88c7a6ba/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:26.266 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 781907 Jun 2 22:58 /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:26.266 DEBUG:teuthology.orchestra.run.smithi052:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:26.287 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:26.306 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-06-02T22:58:26.344 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba on all hosts... 2024-06-02T22:58:26.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-02T22:58:26.346 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-02T22:58:26.351 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba pull 2024-06-02T22:58:27.210 INFO:teuthology.orchestra.run.smithi052.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-02T22:58:27.231 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-02T22:58:27.233 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-02T22:58:57.199 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-02T22:58:57.200 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-02T22:58:57.200 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-02T22:58:57.200 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-06-02T22:58:57.200 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-02T22:58:57.200 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-06-02T22:58:57.200 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-06-02T22:58:58.313 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout: "image_id": "b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1", 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout: "repo_digests": [ 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9" 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T22:59:02.969 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T22:59:02.995 DEBUG:teuthology.orchestra.run.smithi052:> sudo mkdir -p /etc/ceph 2024-06-02T22:59:03.035 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-06-02T22:59:03.068 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-06-02T22:59:03.102 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 777 /etc/ceph 2024-06-02T22:59:03.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-06-02T22:59:03.156 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-06-02T22:59:03.185 INFO:tasks.cephadm:Writing seed config... 2024-06-02T22:59:03.186 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [client] debug client = 20 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-06-02T22:59:03.187 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-06-02T22:59:03.188 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-06-02T22:59:03.189 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-06-02T22:59:03.190 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-06-02T22:59:03.191 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-06-02T22:59:03.191 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-06-02T22:59:03.191 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-06-02T22:59:03.191 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-06-02T22:59:03.191 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-06-02T22:59:03.192 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T22:59:03.192 DEBUG:teuthology.orchestra.run.smithi052:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-06-02T22:59:03.208 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9d9fe36c-2133-11ef-bc9b-c7b262605968 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-06-02T22:59:03.208 DEBUG:teuthology.orchestra.run.smithi052:mon.a> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service 2024-06-02T22:59:03.250 DEBUG:teuthology.orchestra.run.smithi052:mgr.x> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x.service 2024-06-02T22:59:03.293 INFO:tasks.cephadm:Bootstrapping... 2024-06-02T22:59:03.293 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba -v bootstrap --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.52 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-06-02T22:59:03.490 INFO:teuthology.orchestra.run.smithi052.stdout:-------------------------------------------------------------------------------- 2024-06-02T22:59:03.490 INFO:teuthology.orchestra.run.smithi052.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba', '-v', 'bootstrap', '--fsid', '9d9fe36c-2133-11ef-bc9b-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.52', '--skip-admin-label'] 2024-06-02T22:59:03.491 INFO:teuthology.orchestra.run.smithi052.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-06-02T22:59:03.491 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying podman|docker is present... 2024-06-02T22:59:03.516 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 5.0.2 2024-06-02T22:59:03.516 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying lvm2 is present... 2024-06-02T22:59:03.516 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying time synchronization is in place... 2024-06-02T22:59:03.524 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-02T22:59:03.524 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-02T22:59:03.530 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-02T22:59:03.530 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-02T22:59:03.537 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-06-02T22:59:03.544 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-06-02T22:59:03.544 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-06-02T22:59:03.544 INFO:teuthology.orchestra.run.smithi052.stdout:Repeating the final host check... 2024-06-02T22:59:03.567 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout 5.0.2 2024-06-02T22:59:03.568 INFO:teuthology.orchestra.run.smithi052.stdout:podman (/bin/podman) version 5.0.2 is present 2024-06-02T22:59:03.568 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl is present 2024-06-02T22:59:03.568 INFO:teuthology.orchestra.run.smithi052.stdout:lvcreate is present 2024-06-02T22:59:03.574 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-06-02T22:59:03.574 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-06-02T22:59:03.581 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-06-02T22:59:03.581 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-02T22:59:03.587 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout enabled 2024-06-02T22:59:03.594 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout active 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Unit chronyd.service is enabled and running 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Host looks OK 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Cluster fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Acquiring lock 139787683181952 on /run/cephadm/9d9fe36c-2133-11ef-bc9b-c7b262605968.lock 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Lock 139787683181952 acquired on /run/cephadm/9d9fe36c-2133-11ef-bc9b-c7b262605968.lock 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 3300 ... 2024-06-02T22:59:03.595 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying IP 172.21.15.52 port 6789 ... 2024-06-02T22:59:03.596 INFO:teuthology.orchestra.run.smithi052.stdout:Base mon IP(s) is [172.21.15.52:3300, 172.21.15.52:6789], mon addrv is [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-06-02T22:59:03.599 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.52 metric 100 2024-06-02T22:59:03.599 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.52 metric 100 2024-06-02T22:59:03.601 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-06-02T22:59:03.601 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-06-02T22:59:03.602 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:Mon IP `172.21.15.52` is in CIDR network `172.21.0.0/20` 2024-06-02T22:59:03.605 INFO:teuthology.orchestra.run.smithi052.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-06-02T22:59:03.606 INFO:teuthology.orchestra.run.smithi052.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-06-02T22:59:03.606 INFO:teuthology.orchestra.run.smithi052.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-02T22:59:04.047 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stdout b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-02T22:59:04.047 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba... 2024-06-02T22:59:04.047 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Getting image source signatures 2024-06-02T22:59:04.047 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:53b2d4191269e9d20972bfa652565296fad6514cf6a3d99f19db644875f67876 2024-06-02T22:59:04.048 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying blob sha256:3ca969c7b6d79946c6f10490fffc00028babe7df67f50ed53c7bc209325e0612 2024-06-02T22:59:04.048 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Copying config sha256:b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 2024-06-02T22:59:04.048 INFO:teuthology.orchestra.run.smithi052.stdout:/bin/podman: stderr Writing manifest to image destination 2024-06-02T22:59:05.001 INFO:teuthology.orchestra.run.smithi052.stdout:ceph: stdout ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-02T22:59:05.001 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph version: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc) 2024-06-02T22:59:05.001 INFO:teuthology.orchestra.run.smithi052.stdout:Extracting ceph user uid/gid from container image... 2024-06-02T22:59:05.740 INFO:teuthology.orchestra.run.smithi052.stdout:stat: stdout 167 167 2024-06-02T22:59:05.740 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial keys... 2024-06-02T22:59:06.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQA6+VxmOEMKAxAAQMXVEH/qtOKsrZzHWC2uxw== 2024-06-02T22:59:07.517 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQA6+VxmhayIOBAAQM9Ci/fE2Jt/Rb/gzgSy1g== 2024-06-02T22:59:08.327 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph-authtool: stdout AQA7+VxmOEZtMRAAE62igI8fDXgEOZl0YFscgw== 2024-06-02T22:59:08.328 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial monmap... 2024-06-02T22:59:09.228 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:monmaptool for a [v2:172.21.15.52:3300,v1:172.21.15.52:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:setting min_mon_release = quincy 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: set fsid to 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T22:59:09.229 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mon... 2024-06-02T22:59:10.267 INFO:teuthology.orchestra.run.smithi052.stdout:create mon.a on 2024-06-02T22:59:10.487 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-06-02T22:59:10.633 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-06-02T22:59:10.821 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968.target. 2024-06-02T22:59:10.821 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968.target → /etc/systemd/system/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968.target. 2024-06-02T22:59:11.067 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a 2024-06-02T22:59:11.067 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service: Unit ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service not loaded. 2024-06-02T22:59:11.232 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968.target.wants/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service → /etc/systemd/system/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@.service. 2024-06-02T22:59:11.733 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:11 smithi052 ceph-mon[31968]: mkfs 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:11.734 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:11 smithi052 ceph-mon[31968]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-02T22:59:11.774 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T22:59:11.775 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-02T22:59:11.782 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T22:59:11.782 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-02T22:59:11.782 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-02T22:59:11.782 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-06-02T22:59:11.783 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon to start... 2024-06-02T22:59:11.783 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mon... 2024-06-02T22:59:11.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:11 smithi052 bash[31949]: 6ead6663ea41750f01c9b0d1e9a6191d91209d90fe3f09a82e2903555b61b92f 2024-06-02T22:59:11.985 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:11 smithi052 systemd[1]: Started Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T22:59:12.753 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout cluster: 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout id: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout services: 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.552604s) 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout data: 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-06-02T22:59:12.759 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout pgs: 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf72bf700 1 Processor -- start 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf72bf700 1 -- start start 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf72bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf01080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf72bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf0108680 con 0x7fbdf0107c90 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf505b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf01080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf505b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf01080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48460/0 (socket says 172.21.15.52:48460) 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.200+0000 7fbdf505b700 1 -- 172.21.15.52:0/1636662965 learned_addr learned my addr 172.21.15.52:0/1636662965 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.201+0000 7fbdf505b700 1 -- 172.21.15.52:0/1636662965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf0108e90 con 0x7fbdf0107c90 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.201+0000 7fbdf505b700 1 --2- 172.21.15.52:0/1636662965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf01080b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0098a0 tx=0x7fbdec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6aacb4f93a5e52 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.201+0000 7fbde7fff700 1 -- 172.21.15.52:0/1636662965 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdec004160 con 0x7fbdf0107c90 2024-06-02T22:59:12.760 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.201+0000 7fbde7fff700 1 -- 172.21.15.52:0/1636662965 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdec004300 con 0x7fbdf0107c90 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.201+0000 7fbde7fff700 1 -- 172.21.15.52:0/1636662965 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdec0036a0 con 0x7fbdf0107c90 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 -- 172.21.15.52:0/1636662965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 msgr2=0x7fbdf01080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 --2- 172.21.15.52:0/1636662965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf01080b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0098a0 tx=0x7fbdec009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 -- 172.21.15.52:0/1636662965 shutdown_connections 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 --2- 172.21.15.52:0/1636662965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf01080b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 -- 172.21.15.52:0/1636662965 >> 172.21.15.52:0/1636662965 conn(0x7fbdf0103290 msgr2=0x7fbdf01056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 -- 172.21.15.52:0/1636662965 shutdown_connections 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.202+0000 7fbdf72bf700 1 -- 172.21.15.52:0/1636662965 wait complete. 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf72bf700 1 Processor -- start 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf72bf700 1 -- start start 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf72bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf72bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbdf01093c0 con 0x7fbdf0107c90 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf505b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:12.761 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf505b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf019b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48466/0 (socket says 172.21.15.52:48466) 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf505b700 1 -- 172.21.15.52:0/3135050087 learned_addr learned my addr 172.21.15.52:0/3135050087 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf505b700 1 -- 172.21.15.52:0/3135050087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdf019b980 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.203+0000 7fbdf505b700 1 --2- 172.21.15.52:0/3135050087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf019b440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000c00 tx=0x7fbdec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdec003db0 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbdec011630 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdf019bc10 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbdec011950 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdf019e900 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fbdec01a460 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.204+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fbdec01b780 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.205+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbdf0063af0 con 0x7fbdf0107c90 2024-06-02T22:59:12.762 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.207+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fbdec032080 con 0x7fbdf0107c90 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.281+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fbdf002def0 con 0x7fbdf0107c90 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.281+0000 7fbde67fc700 1 -- 172.21.15.52:0/3135050087 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fbdec01b9d0 con 0x7fbdf0107c90 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 msgr2=0x7fbdf019b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 --2- 172.21.15.52:0/3135050087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf019b440 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbdec000c00 tx=0x7fbdec004750 comp rx=0 tx=0).stop 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 shutdown_connections 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 --2- 172.21.15.52:0/3135050087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbdf0107c90 0x7fbdf019b440 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 >> 172.21.15.52:0/3135050087 conn(0x7fbdf0103290 msgr2=0x7fbdf0191fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 shutdown_connections 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:12.282+0000 7fbdf72bf700 1 -- 172.21.15.52:0/3135050087 wait complete. 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:mon is available 2024-06-02T22:59:12.763 INFO:teuthology.orchestra.run.smithi052.stdout:Assimilating anything we can from ceph.conf... 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: monmap epoch 1 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: last_changed 2024-06-02T22:59:08.623389+0000 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: min_mon_release 19 (squid) 2024-06-02T22:59:12.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: election_strategy: 1 2024-06-02T22:59:12.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T22:59:12.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: fsmap 2024-06-02T22:59:12.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: osdmap e1: 0 total, 0 up, 0 in 2024-06-02T22:59:12.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: mgrmap e1: no daemons active 2024-06-02T22:59:12.882 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:12 smithi052 ceph-mon[31968]: from='client.? 172.21.15.52:0/3135050087' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-06-02T22:59:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:13 smithi052 ceph-mon[31968]: from='client.? 172.21.15.52:0/2959535748' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-02T22:59:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:13 smithi052 ceph-mon[31968]: from='client.? 172.21.15.52:0/2959535748' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.52:3300,v1:172.21.15.52:6789] 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-02T22:59:13.915 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ef4f8700 1 Processor -- start 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ef4f8700 1 -- start start 2024-06-02T22:59:13.916 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ef4f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e8106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ef4f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f04e807beb0 con 0x7f04e8106510 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ed294700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e8106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ed294700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e8106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48474/0 (socket says 172.21.15.52:48474) 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ed294700 1 -- 172.21.15.52:0/2674092571 learned_addr learned my addr 172.21.15.52:0/2674092571 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.277+0000 7f04ed294700 1 -- 172.21.15.52:0/2674092571 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e8106e70 con 0x7f04e8106510 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04ed294700 1 --2- 172.21.15.52:0/2674092571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e8106930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f04e40098a0 tx=0x7f04e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59137e7239588691 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04dffff700 1 -- 172.21.15.52:0/2674092571 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e4004160 con 0x7f04e8106510 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04dffff700 1 -- 172.21.15.52:0/2674092571 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e4004300 con 0x7f04e8106510 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04dffff700 1 -- 172.21.15.52:0/2674092571 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e40036a0 con 0x7f04e8106510 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2674092571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 msgr2=0x7f04e8106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04ef4f8700 1 --2- 172.21.15.52:0/2674092571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e8106930 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f04e40098a0 tx=0x7f04e4009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2674092571 shutdown_connections 2024-06-02T22:59:13.917 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04ef4f8700 1 --2- 172.21.15.52:0/2674092571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e8106930 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.278+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2674092571 >> 172.21.15.52:0/2674092571 conn(0x7f04e8102170 msgr2=0x7f04e8104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2674092571 shutdown_connections 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2674092571 wait complete. 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ef4f8700 1 Processor -- start 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ef4f8700 1 -- start start 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ef4f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ef4f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f04e81073a0 con 0x7f04e8106510 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ed294700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ed294700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e819d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48476/0 (socket says 172.21.15.52:48476) 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.279+0000 7f04ed294700 1 -- 172.21.15.52:0/2959535748 learned_addr learned my addr 172.21.15.52:0/2959535748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04ed294700 1 -- 172.21.15.52:0/2959535748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04e819daf0 con 0x7f04e8106510 2024-06-02T22:59:13.918 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04ed294700 1 --2- 172.21.15.52:0/2959535748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e819d5b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f04e4000c00 tx=0x7f04e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e4003db0 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04e4011630 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04e4011950 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04e819dd80 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04e81a0a70 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f04e401a460 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.280+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f04e401b780 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.281+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04e8063af0 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.283+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f04e401ba20 con 0x7f04e8106510 2024-06-02T22:59:13.919 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.354+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f04e81a0c40 con 0x7f04e8106510 2024-06-02T22:59:13.920 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.357+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f04e401bc00 con 0x7f04e8106510 2024-06-02T22:59:13.920 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.357+0000 7f04de7fc700 1 -- 172.21.15.52:0/2959535748 <== mon.0 v2:172.21.15.52:3300/0 8 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f04e401bde0 con 0x7f04e8106510 2024-06-02T22:59:13.920 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 msgr2=0x7f04e819d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:13.920 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 --2- 172.21.15.52:0/2959535748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e819d5b0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f04e4000c00 tx=0x7f04e4004750 comp rx=0 tx=0).stop 2024-06-02T22:59:13.921 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 shutdown_connections 2024-06-02T22:59:13.921 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 --2- 172.21.15.52:0/2959535748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f04e8106510 0x7f04e819d5b0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:13.921 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 >> 172.21.15.52:0/2959535748 conn(0x7f04e8102170 msgr2=0x7f04e8193bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:13.921 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 shutdown_connections 2024-06-02T22:59:13.921 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:13.358+0000 7f04ef4f8700 1 -- 172.21.15.52:0/2959535748 wait complete. 2024-06-02T22:59:13.921 INFO:teuthology.orchestra.run.smithi052.stdout:Generating new minimal ceph.conf... 2024-06-02T22:59:15.029 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:14 smithi052 ceph-mon[31968]: from='client.? 172.21.15.52:0/4292551894' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T22:59:15.036 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.357+0000 7fb694c9e700 1 Processor -- start 2024-06-02T22:59:15.036 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.357+0000 7fb694c9e700 1 -- start start 2024-06-02T22:59:15.036 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.357+0000 7fb694c9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69010aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb694c9e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb69007beb0 con 0x7fb690108700 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69010aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69010aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58120/0 (socket says 172.21.15.52:58120) 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68e59c700 1 -- 172.21.15.52:0/1956635578 learned_addr learned my addr 172.21.15.52:0/1956635578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68e59c700 1 -- 172.21.15.52:0/1956635578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69010b030 con 0x7fb690108700 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68e59c700 1 --2- 172.21.15.52:0/1956635578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69010aaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e80 tx=0x7fb68000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24d787b870c14dec server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68d59a700 1 -- 172.21.15.52:0/1956635578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680017070 con 0x7fb690108700 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68d59a700 1 -- 172.21.15.52:0/1956635578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb68000b8c0 con 0x7fb690108700 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.358+0000 7fb68d59a700 1 -- 172.21.15.52:0/1956635578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680004520 con 0x7fb690108700 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 -- 172.21.15.52:0/1956635578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 msgr2=0x7fb69010aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 --2- 172.21.15.52:0/1956635578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69010aaf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb680009e80 tx=0x7fb68000b330 comp rx=0 tx=0).stop 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 -- 172.21.15.52:0/1956635578 shutdown_connections 2024-06-02T22:59:15.037 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 --2- 172.21.15.52:0/1956635578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69010aaf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 -- 172.21.15.52:0/1956635578 >> 172.21.15.52:0/1956635578 conn(0x7fb690102150 msgr2=0x7fb690104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 -- 172.21.15.52:0/1956635578 shutdown_connections 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.359+0000 7fb694c9e700 1 -- 172.21.15.52:0/1956635578 wait complete. 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb694c9e700 1 Processor -- start 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb694c9e700 1 -- start start 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb694c9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69019bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb694c9e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb69010c000 con 0x7fb690108700 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb68e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69019bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb68e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69019bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58126/0 (socket says 172.21.15.52:58126) 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb68e59c700 1 -- 172.21.15.52:0/4292551894 learned_addr learned my addr 172.21.15.52:0/4292551894 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.360+0000 7fb68e59c700 1 -- 172.21.15.52:0/4292551894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb69019c0b0 con 0x7fb690108700 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb68e59c700 1 --2- 172.21.15.52:0/4292551894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69019bb70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb68000b7b0 tx=0x7fb680004670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:15.038 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680020020 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb68000bcb0 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb69019c340 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb680017040 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb69019f030 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb680023020 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.361+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb68000f050 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.362+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb690063af0 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.364+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fb68002a020 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.435+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fb69019f200 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.435+0000 7fb67f7fe700 1 -- 172.21.15.52:0/4292551894 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7fb68000b860 con 0x7fb690108700 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 msgr2=0x7fb69019bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:15.039 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 --2- 172.21.15.52:0/4292551894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69019bb70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb68000b7b0 tx=0x7fb680004670 comp rx=0 tx=0).stop 2024-06-02T22:59:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 shutdown_connections 2024-06-02T22:59:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 --2- 172.21.15.52:0/4292551894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb690108700 0x7fb69019bb70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 >> 172.21.15.52:0/4292551894 conn(0x7fb690102150 msgr2=0x7fb690102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 shutdown_connections 2024-06-02T22:59:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:14.436+0000 7fb694c9e700 1 -- 172.21.15.52:0/4292551894 wait complete. 2024-06-02T22:59:15.040 INFO:teuthology.orchestra.run.smithi052.stdout:Restarting the monitor... 2024-06-02T22:59:15.300 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:15 smithi052 systemd[1]: Stopping Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T22:59:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a[31964]: 2024-06-02T22:59:15.299+0000 7f9155e4c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T22:59:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a[31964]: 2024-06-02T22:59:15.299+0000 7f9155e4c700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-06-02T22:59:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:15 smithi052 podman[32213]: 2024-06-02 22:59:15.515540602 +0000 UTC m=+0.307679472 container died 6ead6663ea41750f01c9b0d1e9a6191d91209d90fe3f09a82e2903555b61b92f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc) 2024-06-02T22:59:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:15 smithi052 podman[32213]: 2024-06-02 22:59:15.832794237 +0000 UTC m=+0.624933097 container cleanup 6ead6663ea41750f01c9b0d1e9a6191d91209d90fe3f09a82e2903555b61b92f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T22:59:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:15 smithi052 bash[32213]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a 2024-06-02T22:59:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 podman[32227]: 2024-06-02 22:59:16.008497152 +0000 UTC m=+0.490626470 container remove 6ead6663ea41750f01c9b0d1e9a6191d91209d90fe3f09a82e2903555b61b92f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T22:59:16.611 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-02T22:59:16.611 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 systemd[1]: Stopped Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T22:59:16.611 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 systemd[1]: Starting Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T22:59:16.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 podman[32321]: 2024-06-02 22:59:16.398173963 +0000 UTC m=+0.017279706 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:59:16.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 podman[32321]: 2024-06-02 22:59:16.500925558 +0000 UTC m=+0.120031323 container create a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-06-02T22:59:16.790 INFO:teuthology.orchestra.run.smithi052.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-06-02T22:59:17.001 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 podman[32321]: 2024-06-02 22:59:16.711644744 +0000 UTC m=+0.330750478 container init a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 podman[32321]: 2024-06-02 22:59:16.714864007 +0000 UTC m=+0.333969740 container start a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: set uid:gid to 167:167 (ceph:ceph) 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc), process ceph-mon, pid 2 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: pidfile_write: ignore empty --pid-file 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: load: jerasure load: lrc 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: RocksDB version: 7.9.2 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Git sha 0 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Compile date 2024-05-30 15:42:47 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: DB SUMMARY 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: DB Session ID: XT7QXGPOQJA2T1CU6YE2 2024-06-02T22:59:17.002 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: CURRENT file: CURRENT 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: IDENTITY file: IDENTITY 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 103762 ; 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.error_if_exists: 0 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.create_if_missing: 0 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.paranoid_checks: 1 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.flush_verify_memtable_count: 1 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.env: 0x556c858e4c60 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.fs: PosixFileSystem 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.info_log: 0x556c878d7b20 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_file_opening_threads: 16 2024-06-02T22:59:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.statistics: (nil) 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.use_fsync: 0 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_log_file_size: 0 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.log_file_time_to_roll: 0 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.keep_log_file_num: 1000 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.recycle_log_file_num: 0 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.allow_fallocate: 1 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.allow_mmap_reads: 0 2024-06-02T22:59:17.005 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.allow_mmap_writes: 0 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.use_direct_reads: 0 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.create_missing_column_families: 0 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.db_log_dir: 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.wal_dir: 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.table_cache_numshardbits: 6 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.WAL_ttl_seconds: 0 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.WAL_size_limit_MB: 0 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.is_fd_close_on_exec: 1 2024-06-02T22:59:17.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.advise_random_on_open: 1 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.db_write_buffer_size: 0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.write_buffer_manager: 0x556c878a81e0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.use_adaptive_mutex: 0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.rate_limiter: (nil) 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.wal_recovery_mode: 2 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.enable_thread_tracking: 0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.enable_pipelined_write: 0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.unordered_write: 0 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-06-02T22:59:17.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.row_cache: None 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.wal_filter: None 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.allow_ingest_behind: 0 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.two_write_queues: 0 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.manual_wal_flush: 0 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.wal_compression: 0 2024-06-02T22:59:17.008 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.atomic_flush: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.persist_stats_to_disk: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.write_dbid_to_manifest: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.log_readahead_size: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.best_efforts_recovery: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.allow_data_in_errors: 0 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.db_host_id: __hostname__ 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.enforce_single_del_contracts: true 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_background_jobs: 2 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_background_compactions: -1 2024-06-02T22:59:17.009 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_subcompactions: 1 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.delayed_write_rate : 16777216 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_total_wal_size: 0 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.stats_dump_period_sec: 600 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.stats_persist_period_sec: 600 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_open_files: -1 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bytes_per_sync: 0 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.wal_bytes_per_sync: 0 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.strict_bytes_per_sync: 0 2024-06-02T22:59:17.010 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_readahead_size: 0 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_background_flushes: -1 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Compression algorithms supported: 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kZSTD supported: 0 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kXpressCompression supported: 0 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kLZ4HCCompression supported: 1 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kZlibCompression supported: 1 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kSnappyCompression supported: 1 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kLZ4Compression supported: 1 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: kBZip2Compression supported: 0 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Fast CRC32 supported: Supported on x86 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: DMutex implementation: pthread_mutex_t 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-06-02T22:59:17.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.merge_operator: 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_filter: None 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_filter_factory: None 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.sst_partitioner_factory: None 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.memtable_factory: SkipListFactory 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.table_factory: BlockBasedTable 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556c878d7c40) 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks: 1 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout: pin_top_level_index_and_filter: 1 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_type: 0 2024-06-02T22:59:17.012 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_index_type: 0 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_shortening: 1 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: data_block_hash_table_util_ratio: 0.750000 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: checksum: 4 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: no_block_cache: 0 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache: 0x556c878d9090 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_name: BinnedLRUCache 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_options: 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: capacity : 536870912 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_shard_bits : 4 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: strict_capacity_limit : 0 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: high_pri_pool_ratio: 0.000 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_cache_compressed: (nil) 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: persistent_cache: (nil) 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size: 4096 2024-06-02T22:59:17.013 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_size_deviation: 10 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_restart_interval: 16 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: index_block_restart_interval: 1 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: metadata_block_size: 4096 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: partition_filters: 0 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: use_delta_encoding: 1 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: filter_policy: bloomfilter 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: whole_key_filtering: 1 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: verify_compression: 0 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: read_amp_bytes_per_bit: 0 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: format_version: 5 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_index_compression: 1 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: block_align: 0 2024-06-02T22:59:17.014 INFO:journalctl@ceph.mon.a.smithi052.stdout: max_auto_readahead_size: 262144 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout: prepopulate_block_cache: 0 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout: initial_auto_readahead_size: 8192 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout: num_file_reads_for_auto_readahead: 2 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.write_buffer_size: 33554432 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_write_buffer_number: 2 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression: NoCompression 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression: Disabled 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.prefix_extractor: nullptr 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.num_levels: 7 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-06-02T22:59:17.015 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.window_bits: -14 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.level: 32767 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.strategy: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-06-02T22:59:17.016 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.enabled: false 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.target_file_size_base: 67108864 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.target_file_size_multiplier: 1 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-06-02T22:59:17.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.arena_block_size: 1048576 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.disable_auto_compactions: 0 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-06-02T22:59:17.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.table_properties_collectors: 2024-06-02T22:59:17.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.inplace_update_support: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.inplace_update_num_locks: 10000 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.memtable_huge_page_size: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.bloom_locality: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.max_successive_merges: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.optimize_filters_for_hits: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.paranoid_file_checks: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.force_consistency_checks: 1 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.report_bg_io_stats: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.ttl: 2592000 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.periodic_compaction_seconds: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-06-02T22:59:17.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.enable_blob_files: false 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.min_blob_size: 0 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.blob_file_size: 268435456 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.blob_compression_type: NoCompression 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.enable_blob_garbage_collection: false 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.blob_file_starting_level: 0 2024-06-02T22:59:17.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 60ca5255-7854-430e-b5f9-b40bcfb8b716 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717369156745522, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717369156747086, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 98935, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 322, "table_properties": {"data_size": 96932, "index_size": 285, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13919, "raw_average_key_size": 49, "raw_value_size": 89328, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 281, "num_filter_entries": 281, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1717369156, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "60ca5255-7854-430e-b5f9-b40bcfb8b716", "db_session_id": "XT7QXGPOQJA2T1CU6YE2", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: EVENT_LOG_v1 {"time_micros": 1717369156747200, "job": 1, "event": "recovery_finished"} 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556c879c8000 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: DB pointer 0x556c879b2000 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-06-02T22:59:17.022 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** DB Stats ** 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-06-02T22:59:17.023 INFO:journalctl@ceph.mon.a.smithi052.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: L0 2/0 98.45 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: Sum 2/0 98.45 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 105.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** Compaction Stats [default] ** 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 105.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-06-02T22:59:17.024 INFO:journalctl@ceph.mon.a.smithi052.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: AddFile(Keys): cumulative 0, interval 0 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: Cumulative compaction: 0.00 GB write, 20.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: Interval compaction: 0.00 GB write, 20.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache BinnedLRUCache@0x556c878d9090#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout: ** File Read Latency Histogram By Level [default] ** 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: starting mon.a rank 0 at public addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] at bind addrs [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???) e1 preinit fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:17.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).mds e1 new map 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).mds e1 print_map 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout: e1 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout: legacy client fscid: -1 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout: 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout: No filesystems configured 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-06-02T22:59:17.026 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 bash[32321]: a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 systemd[1]: Started Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mon.a is new leader, mons a in quorum (ranks 0) 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: monmap epoch 1 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: last_changed 2024-06-02T22:59:08.623389+0000 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: min_mon_release 19 (squid) 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: election_strategy: 1 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: fsmap 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: osdmap e1: 0 total, 0 up, 0 in 2024-06-02T22:59:17.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:16 smithi052 ceph-mon[32340]: mgrmap e1: no daemons active 2024-06-02T22:59:17.806 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.246+0000 7f83f290b700 1 Processor -- start 2024-06-02T22:59:17.806 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.246+0000 7f83f290b700 1 -- start start 2024-06-02T22:59:17.806 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.246+0000 7f83f290b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec109850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:17.806 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.246+0000 7f83f290b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec109e20 con 0x7f83ec109430 2024-06-02T22:59:17.806 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec109850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:17.806 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec109850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58128/0 (socket says 172.21.15.52:58128) 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83ebfff700 1 -- 172.21.15.52:0/3032362906 learned_addr learned my addr 172.21.15.52:0/3032362906 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83ebfff700 1 -- 172.21.15.52:0/3032362906 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec105cf0 con 0x7f83ec109430 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83ebfff700 1 --2- 172.21.15.52:0/3032362906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec109850 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0098a0 tx=0x7f83dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d4598ca1377318 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83eb7fe700 1 -- 172.21.15.52:0/3032362906 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83dc004160 con 0x7f83ec109430 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83eb7fe700 1 -- 172.21.15.52:0/3032362906 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f83dc004300 con 0x7f83ec109430 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.247+0000 7f83eb7fe700 1 -- 172.21.15.52:0/3032362906 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83dc011a80 con 0x7f83ec109430 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 -- 172.21.15.52:0/3032362906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 msgr2=0x7f83ec109850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 --2- 172.21.15.52:0/3032362906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec109850 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0098a0 tx=0x7f83dc009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 -- 172.21.15.52:0/3032362906 shutdown_connections 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 --2- 172.21.15.52:0/3032362906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec109850 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 -- 172.21.15.52:0/3032362906 >> 172.21.15.52:0/3032362906 conn(0x7f83ec101930 msgr2=0x7f83ec103d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 -- 172.21.15.52:0/3032362906 shutdown_connections 2024-06-02T22:59:17.807 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.248+0000 7f83f290b700 1 -- 172.21.15.52:0/3032362906 wait complete. 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83f290b700 1 Processor -- start 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83f290b700 1 -- start start 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83f290b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec19bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83f290b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83ec10c3c0 con 0x7f83ec109430 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec19bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec19bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58142/0 (socket says 172.21.15.52:58142) 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83ebfff700 1 -- 172.21.15.52:0/3292378897 learned_addr learned my addr 172.21.15.52:0/3292378897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.249+0000 7f83ebfff700 1 -- 172.21.15.52:0/3292378897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83ec19c2a0 con 0x7f83ec109430 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83ebfff700 1 --2- 172.21.15.52:0/3292378897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec19bd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f83dc011740 tx=0x7f83dc011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83dc003b70 con 0x7f83ec109430 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f83dc01a8c0 con 0x7f83ec109430 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83ec19c530 con 0x7f83ec109430 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83dc024890 con 0x7f83ec109430 2024-06-02T22:59:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83ec19f220 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f83dc022020 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.250+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f83dc02ddd0 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.251+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83ec063af0 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.254+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f83dc004300 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.323+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f83ec19f6f0 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.325+0000 7f83e9ffb700 1 -- 172.21.15.52:0/3292378897 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f83dc01aa70 con 0x7f83ec109430 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.326+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 msgr2=0x7f83ec19bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.326+0000 7f83f290b700 1 --2- 172.21.15.52:0/3292378897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec19bd60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f83dc011740 tx=0x7f83dc011b00 comp rx=0 tx=0).stop 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.326+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 shutdown_connections 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.326+0000 7f83f290b700 1 --2- 172.21.15.52:0/3292378897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83ec109430 0x7f83ec19bd60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.326+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 >> 172.21.15.52:0/3292378897 conn(0x7f83ec101930 msgr2=0x7f83ec102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.327+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 shutdown_connections 2024-06-02T22:59:17.809 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:17.327+0000 7f83f290b700 1 -- 172.21.15.52:0/3292378897 wait complete. 2024-06-02T22:59:17.810 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote config to /etc/ceph/ceph.conf 2024-06-02T22:59:17.824 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-06-02T22:59:17.824 INFO:teuthology.orchestra.run.smithi052.stdout:Creating mgr... 2024-06-02T22:59:17.825 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:9283 ... 2024-06-02T22:59:17.825 INFO:teuthology.orchestra.run.smithi052.stdout:Verifying port 0.0.0.0:8765 ... 2024-06-02T22:59:18.076 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x 2024-06-02T22:59:18.077 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x.service: Unit ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x.service not loaded. 2024-06-02T22:59:18.231 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968.target.wants/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@.service. 2024-06-02T22:59:18.248 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 systemd[1]: Starting Ceph mgr.x for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T22:59:18.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:18 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3292378897' entity='client.admin' 2024-06-02T22:59:18.609 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 podman[32554]: 2024-06-02 22:59:18.37204486 +0000 UTC m=+0.017476394 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba 2024-06-02T22:59:18.609 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 podman[32554]: 2024-06-02 22:59:18.469540912 +0000 UTC m=+0.114972447 container create b64f2c3c3485e7f08c2e339b2abd54ec17e5edc1c408e73005533f62d5b1100c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-02T22:59:18.811 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T22:59:18.811 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-02T22:59:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T22:59:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-02T22:59:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-02T22:59:18.819 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to enable service . firewalld.service is not available 2024-06-02T22:59:18.828 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T22:59:18.829 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-02T22:59:18.837 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T22:59:18.837 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-02T22:59:18.837 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-02T22:59:18.837 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-06-02T22:59:18.837 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr to start... 2024-06-02T22:59:18.837 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr... 2024-06-02T22:59:18.879 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 podman[32554]: 2024-06-02 22:59:18.713796774 +0000 UTC m=+0.359228299 container init b64f2c3c3485e7f08c2e339b2abd54ec17e5edc1c408e73005533f62d5b1100c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-06-02T22:59:18.880 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 podman[32554]: 2024-06-02 22:59:18.716722081 +0000 UTC m=+0.362153606 container start b64f2c3c3485e7f08c2e339b2abd54ec17e5edc1c408e73005533f62d5b1100c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc) 2024-06-02T22:59:18.880 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 bash[32554]: b64f2c3c3485e7f08c2e339b2abd54ec17e5edc1c408e73005533f62d5b1100c 2024-06-02T22:59:18.880 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 systemd[1]: Started Ceph mgr.x for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T22:59:19.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:18.878+0000 7f224963a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T22:59:19.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:19.074+0000 7f224963a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T22:59:19.435 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:19.183+0000 7f224963a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T22:59:19.735 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:19 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3144326170' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T22:59:19.736 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:19.516+0000 7f224963a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T22:59:19.736 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:19.605+0000 7f224963a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T22:59:19.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:19.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T22:59:19.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T22:59:19.982 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T22:59:19.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T22:59:19.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T22:59:19.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T22:59:19.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T22:59:19.983 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T22:59:19.984 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T22:59:19.985 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T22:59:19.986 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T22:59:19.987 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T22:59:11.731067+0000", 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.302+0000 7f3a17a3b700 1 Processor -- start 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.302+0000 7f3a17a3b700 1 -- start start 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a17a3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a10106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:19.988 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a17a3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10107580 con 0x7f3a10106b90 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a157d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a10106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a157d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a10106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58166/0 (socket says 172.21.15.52:58166) 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a157d7700 1 -- 172.21.15.52:0/4151929893 learned_addr learned my addr 172.21.15.52:0/4151929893 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a157d7700 1 -- 172.21.15.52:0/4151929893 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10107d90 con 0x7f3a10106b90 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.303+0000 7f3a157d7700 1 --2- 172.21.15.52:0/4151929893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a10106fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0098a0 tx=0x7f3a0c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d53cfb7bdbb9e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a07fff700 1 -- 172.21.15.52:0/4151929893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c004160 con 0x7f3a10106b90 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a07fff700 1 -- 172.21.15.52:0/4151929893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3a0c004300 con 0x7f3a10106b90 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a07fff700 1 -- 172.21.15.52:0/4151929893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c011a80 con 0x7f3a10106b90 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a17a3b700 1 -- 172.21.15.52:0/4151929893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 msgr2=0x7f3a10106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a17a3b700 1 --2- 172.21.15.52:0/4151929893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a10106fb0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c0098a0 tx=0x7f3a0c009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a17a3b700 1 -- 172.21.15.52:0/4151929893 shutdown_connections 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a17a3b700 1 --2- 172.21.15.52:0/4151929893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a10106fb0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:19.989 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.304+0000 7f3a17a3b700 1 -- 172.21.15.52:0/4151929893 >> 172.21.15.52:0/4151929893 conn(0x7f3a10102150 msgr2=0x7f3a10104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.305+0000 7f3a17a3b700 1 -- 172.21.15.52:0/4151929893 shutdown_connections 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.305+0000 7f3a17a3b700 1 -- 172.21.15.52:0/4151929893 wait complete. 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.305+0000 7f3a17a3b700 1 Processor -- start 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.305+0000 7f3a17a3b700 1 -- start start 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a17a3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a1019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a17a3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010b140 con 0x7f3a10106b90 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a157d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a1019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a157d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a1019ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58178/0 (socket says 172.21.15.52:58178) 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a157d7700 1 -- 172.21.15.52:0/3144326170 learned_addr learned my addr 172.21.15.52:0/3144326170 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a157d7700 1 -- 172.21.15.52:0/3144326170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019b060 con 0x7f3a10106b90 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a157d7700 1 --2- 172.21.15.52:0/3144326170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a1019ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006af0 tx=0x7f3a0c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:19.990 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c0039e0 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f3a0c003b80 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.306+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019b2f0 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.307+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1019dfe0 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.307+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a0c0196a0 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.307+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3a0c02d440 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.307+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3a0c024c00 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.308+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10063af0 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.310+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f3a0c043dd0 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.383+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3a1019e520 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.384+0000 7f3a067fc700 1 -- 172.21.15.52:0/3144326170 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3a0c032020 con 0x7f3a10106b90 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 msgr2=0x7f3a1019ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 --2- 172.21.15.52:0/3144326170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a1019ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3a0c006af0 tx=0x7f3a0c011c40 comp rx=0 tx=0).stop 2024-06-02T22:59:19.991 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 shutdown_connections 2024-06-02T22:59:19.992 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 --2- 172.21.15.52:0/3144326170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a10106b90 0x7f3a1019ab20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:19.992 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 >> 172.21.15.52:0/3144326170 conn(0x7f3a10102150 msgr2=0x7f3a10102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:19.992 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 shutdown_connections 2024-06-02T22:59:19.992 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:19.385+0000 7f3a17a3b700 1 -- 172.21.15.52:0/3144326170 wait complete. 2024-06-02T22:59:19.992 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (1/15)... 2024-06-02T22:59:21.072 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:20.895+0000 7f224963a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T22:59:21.072 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:21.071+0000 7f224963a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T22:59:21.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:21.247+0000 7f224963a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T22:59:21.853 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:21.546+0000 7f224963a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T22:59:21.853 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:21.642+0000 7f224963a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T22:59:21.853 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:21.852+0000 7f224963a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T22:59:22.150 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:22.056+0000 7f224963a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T22:59:22.699 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:22 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/825859565' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T22:59:23.007 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:22.699+0000 7f224963a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T22:59:23.012 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:23.012 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T22:59:23.012 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T22:59:23.012 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T22:59:23.012 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:23.013 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T22:59:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T22:59:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T22:59:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T22:59:23.014 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T22:59:23.023 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T22:59:23.023 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T22:59:23.023 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T22:59:23.023 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T22:59:23.023 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T22:59:23.024 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T22:59:23.025 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": false, 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T22:59:23.026 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T22:59:11.731067+0000", 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:23.027 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e5740f700 1 Processor -- start 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e5740f700 1 -- start start 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e5740f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e501080f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e5740f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501086c0 con 0x7f6e50107cd0 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e551ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e501080f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e551ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e501080f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58188/0 (socket says 172.21.15.52:58188) 2024-06-02T22:59:23.028 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.341+0000 7f6e551ab700 1 -- 172.21.15.52:0/2952185733 learned_addr learned my addr 172.21.15.52:0/2952185733 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.342+0000 7f6e551ab700 1 -- 172.21.15.52:0/2952185733 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e50108ed0 con 0x7f6e50107cd0 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.342+0000 7f6e551ab700 1 --2- 172.21.15.52:0/2952185733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e501080f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0098a0 tx=0x7f6e4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=600388db900f8cae server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.342+0000 7f6e47fff700 1 -- 172.21.15.52:0/2952185733 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e4c004160 con 0x7f6e50107cd0 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.342+0000 7f6e47fff700 1 -- 172.21.15.52:0/2952185733 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6e4c004300 con 0x7f6e50107cd0 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.342+0000 7f6e47fff700 1 -- 172.21.15.52:0/2952185733 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e4c011a80 con 0x7f6e50107cd0 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 -- 172.21.15.52:0/2952185733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 msgr2=0x7f6e501080f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:23.029 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 --2- 172.21.15.52:0/2952185733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e501080f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0098a0 tx=0x7f6e4c009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 -- 172.21.15.52:0/2952185733 shutdown_connections 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 --2- 172.21.15.52:0/2952185733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e501080f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 -- 172.21.15.52:0/2952185733 >> 172.21.15.52:0/2952185733 conn(0x7f6e50103290 msgr2=0x7f6e501056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 -- 172.21.15.52:0/2952185733 shutdown_connections 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 -- 172.21.15.52:0/2952185733 wait complete. 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.343+0000 7f6e5740f700 1 Processor -- start 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e5740f700 1 -- start start 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e5740f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e5019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:23.030 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e5740f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010c280 con 0x7f6e50107cd0 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e551ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e5019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e551ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e5019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58192/0 (socket says 172.21.15.52:58192) 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e551ab700 1 -- 172.21.15.52:0/825859565 learned_addr learned my addr 172.21.15.52:0/825859565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e551ab700 1 -- 172.21.15.52:0/825859565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e5019c1f0 con 0x7f6e50107cd0 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e551ab700 1 --2- 172.21.15.52:0/825859565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e5019bcb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006af0 tx=0x7f6e4c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e4c003930 con 0x7f6e50107cd0 2024-06-02T22:59:23.031 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e5019c480 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.344+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f6e4c003ad0 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.345+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e4c0195e0 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.345+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e5019f170 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.345+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6e4c019c90 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.345+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6e4c024730 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.346+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e50063af0 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.348+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6e4c043dd0 con 0x7f6e50107cd0 2024-06-02T22:59:23.032 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.422+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f6e5019f340 con 0x7f6e50107cd0 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.423+0000 7f6e467fc700 1 -- 172.21.15.52:0/825859565 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6e4c032020 con 0x7f6e50107cd0 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.424+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 msgr2=0x7f6e5019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.424+0000 7f6e5740f700 1 --2- 172.21.15.52:0/825859565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e5019bcb0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006af0 tx=0x7f6e4c011b70 comp rx=0 tx=0).stop 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.424+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 shutdown_connections 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.424+0000 7f6e5740f700 1 --2- 172.21.15.52:0/825859565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50107cd0 0x7f6e5019bcb0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.424+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 >> 172.21.15.52:0/825859565 conn(0x7f6e50103290 msgr2=0x7f6e50103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.425+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 shutdown_connections 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:22.425+0000 7f6e5740f700 1 -- 172.21.15.52:0/825859565 wait complete. 2024-06-02T22:59:23.033 INFO:teuthology.orchestra.run.smithi052.stdout:mgr not available, waiting (2/15)... 2024-06-02T22:59:23.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:23.139+0000 7f224963a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T22:59:23.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:23.240+0000 7f224963a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T22:59:23.899 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:23.449+0000 7f224963a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T22:59:24.343 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:23.995+0000 7f224963a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T22:59:24.343 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:24.090+0000 7f224963a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T22:59:24.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:24.342+0000 7f224963a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T22:59:24.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:24.482+0000 7f224963a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T22:59:24.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:24.580+0000 7f224963a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T22:59:25.013 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:24.701+0000 7f224963a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T22:59:25.013 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:25.012+0000 7f224963a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T22:59:26.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:25.649+0000 7f224963a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:25.874+0000 7f224963a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: Activating manager daemon x 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: mgrmap e2: x(active, starting, since 0.00324005s) 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: Manager daemon x is now available 2024-06-02T22:59:26.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-02T22:59:26.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-02T22:59:26.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' 2024-06-02T22:59:26.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' 2024-06-02T22:59:26.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:25 smithi052 ceph-mon[32340]: from='mgr.14100 172.21.15.52:0/3930456101' entity='mgr.x' 2024-06-02T22:59:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:27 smithi052 ceph-mon[32340]: mgrmap e3: x(active, since 1.00752s) 2024-06-02T22:59:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:28 smithi052 ceph-mon[32340]: mgrmap e4: x(active, since 2s) 2024-06-02T22:59:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:28 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3639604883' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-06-02T22:59:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T22:59:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T22:59:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "health": { 2024-06-02T22:59:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-06-02T22:59:33.838 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 0 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "a" 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "monmap": { 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-06-02T22:59:33.839 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-06-02T22:59:33.840 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-06-02T22:59:33.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-06-02T22:59:33.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-06-02T22:59:33.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-06-02T22:59:33.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-06-02T22:59:33.848 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-06-02T22:59:33.849 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modules": [ 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "iostat", 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "nfs", 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "restful" 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ], 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "modified": "2024-06-02T22:59:11.731067+0000", 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "services": {} 2024-06-02T22:59:33.850 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout }, 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.053+0000 7f8b06f09700 1 Processor -- start 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.053+0000 7f8b06f09700 1 -- start start 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.053+0000 7f8b06f09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0010ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8b06f09700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0007be70 con 0x7f8b00108740 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8b04ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0010ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8b04ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0010ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36462/0 (socket says 172.21.15.52:36462) 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8b04ca5700 1 -- 172.21.15.52:0/2911450709 learned_addr learned my addr 172.21.15.52:0/2911450709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8b04ca5700 1 -- 172.21.15.52:0/2911450709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0010b070 con 0x7f8b00108740 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8b04ca5700 1 --2- 172.21.15.52:0/2911450709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0010ab30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80098a0 tx=0x7f8ae8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24cf0da497fe0af server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8aff7fe700 1 -- 172.21.15.52:0/2911450709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8004160 con 0x7f8b00108740 2024-06-02T22:59:33.851 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8aff7fe700 1 -- 172.21.15.52:0/2911450709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ae8004300 con 0x7f8b00108740 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.054+0000 7f8aff7fe700 1 -- 172.21.15.52:0/2911450709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae8011a80 con 0x7f8b00108740 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 -- 172.21.15.52:0/2911450709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 msgr2=0x7f8b0010ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 --2- 172.21.15.52:0/2911450709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0010ab30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8ae80098a0 tx=0x7f8ae8009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 -- 172.21.15.52:0/2911450709 shutdown_connections 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 --2- 172.21.15.52:0/2911450709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0010ab30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 -- 172.21.15.52:0/2911450709 >> 172.21.15.52:0/2911450709 conn(0x7f8b001020f0 msgr2=0x7f8b00104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 -- 172.21.15.52:0/2911450709 shutdown_connections 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.055+0000 7f8b06f09700 1 -- 172.21.15.52:0/2911450709 wait complete. 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b06f09700 1 Processor -- start 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b06f09700 1 -- start start 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b06f09700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b06f09700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8b0010c030 con 0x7f8b00108740 2024-06-02T22:59:33.852 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b04ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b04ca5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36466/0 (socket says 172.21.15.52:36466) 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b04ca5700 1 -- 172.21.15.52:0/3639604883 learned_addr learned my addr 172.21.15.52:0/3639604883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b04ca5700 1 -- 172.21.15.52:0/3639604883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b0019c110 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.056+0000 7f8b04ca5700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0019bbd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006af0 tx=0x7f8ae8011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.057+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae80039e0 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.057+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8ae8003b80 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.057+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b0019c3a0 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.057+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ae80196a0 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.057+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b0019f090 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.058+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f8ae801ab50 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.058+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b00063af0 con 0x7f8b00108740 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.058+0000 7f8afdffb700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f8aec039000 0x7f8aec03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:33.853 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.058+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8ae804d510 con 0x7f8b00108740 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.058+0000 7f8afffff700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f8aec039000 0x7f8aec03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.058+0000 7f8afffff700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f8aec039000 0x7f8aec03b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8af0006fd0 tx=0x7f8af0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.064+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ae80172b0 con 0x7f8b00108740 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.361+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f8b0019f260 con 0x7f8b00108740 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.362+0000 7f8afdffb700 1 -- 172.21.15.52:0/3639604883 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f8b0019f260 con 0x7f8b00108740 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f8aec039000 msgr2=0x7f8aec03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f8aec039000 0x7f8aec03b4c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8af0006fd0 tx=0x7f8af0006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 msgr2=0x7f8b0019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0019bbd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8ae8006af0 tx=0x7f8ae8011c40 comp rx=0 tx=0).stop 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 shutdown_connections 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f8aec039000 0x7f8aec03b4c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 --2- 172.21.15.52:0/3639604883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8b00108740 0x7f8b0019bbd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 >> 172.21.15.52:0/3639604883 conn(0x7f8b001020f0 msgr2=0x7f8b00102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:33.854 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 shutdown_connections 2024-06-02T22:59:33.855 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:28.365+0000 7f8b06f09700 1 -- 172.21.15.52:0/3639604883 wait complete. 2024-06-02T22:59:33.855 INFO:teuthology.orchestra.run.smithi052.stdout:mgr is available 2024-06-02T22:59:34.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:34 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4174033564' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [global] 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout fsid = 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [mgr] 2024-06-02T22:59:35.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout [osd] 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.333+0000 7f7962c2f700 1 Processor -- start 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.333+0000 7f7962c2f700 1 -- start start 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.333+0000 7f7962c2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c10abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.333+0000 7f7962c2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f795c07bdd0 con 0x7f795c108800 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f79609cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c10abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:35.106 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f79609cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c10abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38198/0 (socket says 172.21.15.52:38198) 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f79609cb700 1 -- 172.21.15.52:0/3753775883 learned_addr learned my addr 172.21.15.52:0/3753775883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f79609cb700 1 -- 172.21.15.52:0/3753775883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c10b130 con 0x7f795c108800 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f79609cb700 1 --2- 172.21.15.52:0/3753775883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c10abf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098a0 tx=0x7f794c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01535b760a1816a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f795b7fe700 1 -- 172.21.15.52:0/3753775883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c004160 con 0x7f795c108800 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f795b7fe700 1 -- 172.21.15.52:0/3753775883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f794c004300 con 0x7f795c108800 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.334+0000 7f795b7fe700 1 -- 172.21.15.52:0/3753775883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c011a80 con 0x7f795c108800 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 -- 172.21.15.52:0/3753775883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 msgr2=0x7f795c10abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 --2- 172.21.15.52:0/3753775883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c10abf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f794c0098a0 tx=0x7f794c009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 -- 172.21.15.52:0/3753775883 shutdown_connections 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 --2- 172.21.15.52:0/3753775883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c10abf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 -- 172.21.15.52:0/3753775883 >> 172.21.15.52:0/3753775883 conn(0x7f795c102170 msgr2=0x7f795c104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 -- 172.21.15.52:0/3753775883 shutdown_connections 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 -- 172.21.15.52:0/3753775883 wait complete. 2024-06-02T22:59:35.107 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.335+0000 7f7962c2f700 1 Processor -- start 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f7962c2f700 1 -- start start 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f7962c2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f7962c2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f795c10c030 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f79609cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f79609cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38210/0 (socket says 172.21.15.52:38210) 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f79609cb700 1 -- 172.21.15.52:0/4174033564 learned_addr learned my addr 172.21.15.52:0/4174033564 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f79609cb700 1 -- 172.21.15.52:0/4174033564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795c19c170 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f79609cb700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c19bc30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f794c011a50 tx=0x7f794c004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c0039f0 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f794c003b90 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.336+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795c19c400 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.337+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f794c01a600 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.337+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795c19f0f0 con 0x7f795c108800 2024-06-02T22:59:35.108 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.338+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f794c024a40 con 0x7f795c108800 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.338+0000 7f7959ffb700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f7944039050 0x7f794403b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.338+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f794c04d400 con 0x7f795c108800 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.338+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f795c063af0 con 0x7f795c108800 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.338+0000 7f795bfff700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f7944039050 0x7f794403b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.338+0000 7f795bfff700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f7944039050 0x7f794403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7950006fd0 tx=0x7f7950006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.343+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f794c0172b0 con 0x7f795c108800 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.553+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f795c1a07e0 con 0x7f795c108800 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.554+0000 7f7959ffb700 1 -- 172.21.15.52:0/4174033564 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+373 (secure 0 0 0) 0x7f794c04ce20 con 0x7f795c108800 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f7944039050 msgr2=0x7f794403b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f7944039050 0x7f794403b510 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7950006fd0 tx=0x7f7950006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 msgr2=0x7f795c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c19bc30 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f794c011a50 tx=0x7f794c004290 comp rx=0 tx=0).stop 2024-06-02T22:59:35.109 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 shutdown_connections 2024-06-02T22:59:35.110 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f7944039050 0x7f794403b510 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:35.110 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 --2- 172.21.15.52:0/4174033564 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f795c108800 0x7f795c19bc30 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:35.110 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 >> 172.21.15.52:0/4174033564 conn(0x7f795c102170 msgr2=0x7f795c102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:35.110 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 shutdown_connections 2024-06-02T22:59:35.110 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:34.557+0000 7f7962c2f700 1 -- 172.21.15.52:0/4174033564 wait complete. 2024-06-02T22:59:35.110 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling cephadm module... 2024-06-02T22:59:36.093 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:35 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/648502533' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-06-02T22:59:36.093 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: ignoring --setuser ceph since I am not root 2024-06-02T22:59:36.094 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: ignoring --setgroup ceph since I am not root 2024-06-02T22:59:36.094 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:36.001+0000 7f6170d41200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T22:59:36.388 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:36.196+0000 7f6170d41200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T22:59:36.388 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:36.305+0000 7f6170d41200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T22:59:36.398 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.500+0000 7f1d3e861700 1 Processor -- start 2024-06-02T22:59:36.398 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.500+0000 7f1d3e861700 1 -- start start 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d3e861700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d38106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d3e861700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d38107580 con 0x7f1d38106b90 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d38106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d38106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38216/0 (socket says 172.21.15.52:38216) 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d37fff700 1 -- 172.21.15.52:0/985753430 learned_addr learned my addr 172.21.15.52:0/985753430 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d37fff700 1 -- 172.21.15.52:0/985753430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d38107d90 con 0x7f1d38106b90 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.501+0000 7f1d37fff700 1 --2- 172.21.15.52:0/985753430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d38106fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d280098a0 tx=0x7f1d28009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e4a6aa2d638c55 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d377fe700 1 -- 172.21.15.52:0/985753430 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d28004160 con 0x7f1d38106b90 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d377fe700 1 -- 172.21.15.52:0/985753430 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1d28004300 con 0x7f1d38106b90 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d377fe700 1 -- 172.21.15.52:0/985753430 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d28011a80 con 0x7f1d38106b90 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 -- 172.21.15.52:0/985753430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 msgr2=0x7f1d38106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:36.399 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 --2- 172.21.15.52:0/985753430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d38106fb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d280098a0 tx=0x7f1d28009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 -- 172.21.15.52:0/985753430 shutdown_connections 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 --2- 172.21.15.52:0/985753430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d38106fb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 -- 172.21.15.52:0/985753430 >> 172.21.15.52:0/985753430 conn(0x7f1d38102150 msgr2=0x7f1d38104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 -- 172.21.15.52:0/985753430 shutdown_connections 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.502+0000 7f1d3e861700 1 -- 172.21.15.52:0/985753430 wait complete. 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d3e861700 1 Processor -- start 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d3e861700 1 -- start start 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d3e861700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d3819bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:36.400 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d3e861700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d3810b140 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d3819bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d3819bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38230/0 (socket says 172.21.15.52:38230) 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.503+0000 7f1d37fff700 1 -- 172.21.15.52:0/648502533 learned_addr learned my addr 172.21.15.52:0/648502533 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d37fff700 1 -- 172.21.15.52:0/648502533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d3819c170 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d37fff700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d3819bc30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d28006af0 tx=0x7f1d28011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d280039e0 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f1d28003b80 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d3819c400 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d280196a0 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.504+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d3819f0f0 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.505+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 4) ==== 46276+0+0 (secure 0 0 0) 0x7f1d280199a0 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.505+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d38063af0 con 0x7f1d38106b90 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.505+0000 7f1d35ffb700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f1d24038fb0 0x7f1d2403b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:36.401 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.505+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1d2804d5c0 con 0x7f1d38106b90 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.505+0000 7f1d2ffff700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f1d24038fb0 0x7f1d2403b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.506+0000 7f1d2ffff700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f1d24038fb0 0x7f1d2403b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006fd0 tx=0x7f1d20006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.510+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d2804cfe0 con 0x7f1d38106b90 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.775+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f1d380008d0 con 0x7f1d38106b90 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.827+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f1d28019d40 con 0x7f1d38106b90 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.828+0000 7f1d35ffb700 1 -- 172.21.15.52:0/648502533 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f1d2802d960 con 0x7f1d38106b90 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.829+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f1d24038fb0 msgr2=0x7f1d2403b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.829+0000 7f1d3e861700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f1d24038fb0 0x7f1d2403b470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1d20006fd0 tx=0x7f1d20006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.829+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 msgr2=0x7f1d3819bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.829+0000 7f1d3e861700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d3819bc30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1d28006af0 tx=0x7f1d28011c40 comp rx=0 tx=0).stop 2024-06-02T22:59:36.402 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.830+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 shutdown_connections 2024-06-02T22:59:36.403 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.830+0000 7f1d3e861700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f1d24038fb0 0x7f1d2403b470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:36.403 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.830+0000 7f1d3e861700 1 --2- 172.21.15.52:0/648502533 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d38106b90 0x7f1d3819bc30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:36.403 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.830+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 >> 172.21.15.52:0/648502533 conn(0x7f1d38102150 msgr2=0x7f1d38103020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:36.403 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.830+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 shutdown_connections 2024-06-02T22:59:36.403 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:35.830+0000 7f1d3e861700 1 -- 172.21.15.52:0/648502533 wait complete. 2024-06-02T22:59:36.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/648502533' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-06-02T22:59:36.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-mon[32340]: mgrmap e5: x(active, since 9s) 2024-06-02T22:59:36.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:36.640+0000 7f6170d41200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T22:59:36.900 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:36.732+0000 7f6170d41200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.877+0000 7fd491373700 1 Processor -- start 2024-06-02T22:59:37.735 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.877+0000 7fd491373700 1 -- start start 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd491373700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c1080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd491373700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c108680 con 0x7fd48c107c90 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd48affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c1080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd48affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c1080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38256/0 (socket says 172.21.15.52:38256) 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd48affd700 1 -- 172.21.15.52:0/1644042067 learned_addr learned my addr 172.21.15.52:0/1644042067 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd48affd700 1 -- 172.21.15.52:0/1644042067 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c108e90 con 0x7fd48c107c90 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.878+0000 7fd48affd700 1 --2- 172.21.15.52:0/1644042067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c1080b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0098a0 tx=0x7fd47c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec4baad5068961e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd489ffb700 1 -- 172.21.15.52:0/1644042067 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd47c004160 con 0x7fd48c107c90 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd489ffb700 1 -- 172.21.15.52:0/1644042067 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd47c004300 con 0x7fd48c107c90 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd489ffb700 1 -- 172.21.15.52:0/1644042067 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd47c011a80 con 0x7fd48c107c90 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 -- 172.21.15.52:0/1644042067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 msgr2=0x7fd48c1080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 --2- 172.21.15.52:0/1644042067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c1080b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0098a0 tx=0x7fd47c009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 -- 172.21.15.52:0/1644042067 shutdown_connections 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 --2- 172.21.15.52:0/1644042067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c1080b0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:37.736 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 -- 172.21.15.52:0/1644042067 >> 172.21.15.52:0/1644042067 conn(0x7fd48c103290 msgr2=0x7fd48c1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 -- 172.21.15.52:0/1644042067 shutdown_connections 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.879+0000 7fd491373700 1 -- 172.21.15.52:0/1644042067 wait complete. 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.880+0000 7fd491373700 1 Processor -- start 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.880+0000 7fd491373700 1 -- start start 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.880+0000 7fd491373700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c19bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.880+0000 7fd491373700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c10c240 con 0x7fd48c107c90 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd48affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c19bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd48affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c19bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38270/0 (socket says 172.21.15.52:38270) 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd48affd700 1 -- 172.21.15.52:0/4160267480 learned_addr learned my addr 172.21.15.52:0/4160267480 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd48affd700 1 -- 172.21.15.52:0/4160267480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19c1b0 con 0x7fd48c107c90 2024-06-02T22:59:37.737 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd48affd700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c19bc70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd47c006af0 tx=0x7fd47c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd47c003930 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fd47c003ad0 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19c440 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd47c0195e0 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.881+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c19f130 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.882+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7fd47c019890 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.882+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c063af0 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.883+0000 7fd483fff700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 0x7fd47803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.883+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd47c04d7a0 con 0x7fd48c107c90 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.883+0000 7fd48a7fc700 1 -- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 msgr2=0x7fd47803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.883+0000 7fd48a7fc700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 0x7fd47803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T22:59:37.738 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:36.888+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd47c04d1c0 con 0x7fd48c107c90 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.084+0000 7fd48a7fc700 1 -- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 msgr2=0x7fd47803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.084+0000 7fd48a7fc700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 0x7fd47803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.200+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd48c19f300 con 0x7fd48c107c90 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.200+0000 7fd483fff700 1 -- 172.21.15.52:0/4160267480 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fd48c19f300 con 0x7fd48c107c90 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 msgr2=0x7fd47803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 0x7fd47803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 msgr2=0x7fd48c19bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c19bc70 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd47c006af0 tx=0x7fd47c011b70 comp rx=0 tx=0).stop 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 shutdown_connections 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7fd478039050 0x7fd47803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:37.739 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 --2- 172.21.15.52:0/4160267480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd48c107c90 0x7fd48c19bc70 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:37.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 >> 172.21.15.52:0/4160267480 conn(0x7fd48c103290 msgr2=0x7fd48c103dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:37.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 shutdown_connections 2024-06-02T22:59:37.740 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:37.202+0000 7fd491373700 1 -- 172.21.15.52:0/4160267480 wait complete. 2024-06-02T22:59:37.740 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-06-02T22:59:37.740 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 5... 2024-06-02T22:59:38.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:37 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4160267480' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-02T22:59:38.113 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:37 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:37.996+0000 7f6170d41200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T22:59:38.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:38.175+0000 7f6170d41200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T22:59:38.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:38.354+0000 7f6170d41200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T22:59:38.754 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:38.659+0000 7f6170d41200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T22:59:38.754 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:38.755+0000 7f6170d41200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T22:59:39.127 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:38.966+0000 7f6170d41200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T22:59:39.391 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:39.171+0000 7f6170d41200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T22:59:40.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:39.817+0000 7f6170d41200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T22:59:40.575 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:40.262+0000 7f6170d41200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T22:59:40.575 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:40.365+0000 7f6170d41200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T22:59:40.575 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:40.576+0000 7f6170d41200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T22:59:41.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:41.140+0000 7f6170d41200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T22:59:41.399 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:41.237+0000 7f6170d41200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T22:59:41.856 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:41.495+0000 7f6170d41200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T22:59:41.856 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:41.636+0000 7f6170d41200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T22:59:41.857 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:41.735+0000 7f6170d41200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T22:59:41.857 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:41.857+0000 7f6170d41200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T22:59:42.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:42.167+0000 7f6170d41200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T22:59:43.074 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:42.795+0000 7f6170d41200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T22:59:43.075 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T22:59:43.022+0000 7f6170d41200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T22:59:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: Active manager daemon x restarted 2024-06-02T22:59:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: Activating manager daemon x 2024-06-02T22:59:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: osdmap e2: 0 total, 0 up, 0 in 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: mgrmap e6: x(active, starting, since 0.0032683s) 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: Manager daemon x is now available 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T22:59:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:43 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-02T22:59:44.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: Found migration_current of "None". Setting to last migration. 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: [02/Jun/2024:22:59:43] ENGINE Bus STARTING 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: [02/Jun/2024:22:59:43] ENGINE Serving on http://172.21.15.52:8765 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T22:59:44.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:44 smithi052 ceph-mon[32340]: mgrmap e7: x(active, since 1.00611s) 2024-06-02T22:59:44.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.220+0000 7f525873b700 1 Processor -- start 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.220+0000 7f525873b700 1 -- start start 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.220+0000 7f525873b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f52501074a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.220+0000 7f525873b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5250107a70 con 0x7f5250107080 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52564d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f52501074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52564d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f52501074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38286/0 (socket says 172.21.15.52:38286) 2024-06-02T22:59:44.575 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52564d7700 1 -- 172.21.15.52:0/3604532023 learned_addr learned my addr 172.21.15.52:0/3604532023 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52564d7700 1 -- 172.21.15.52:0/3604532023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5250107da0 con 0x7f5250107080 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52564d7700 1 --2- 172.21.15.52:0/3604532023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f52501074a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52440098a0 tx=0x7f5244009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a6089b0e677d4a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52554d5700 1 -- 172.21.15.52:0/3604532023 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5244004160 con 0x7f5250107080 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52554d5700 1 -- 172.21.15.52:0/3604532023 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5244004300 con 0x7f5250107080 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.221+0000 7f52554d5700 1 -- 172.21.15.52:0/3604532023 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5244011a80 con 0x7f5250107080 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 -- 172.21.15.52:0/3604532023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 msgr2=0x7f52501074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 --2- 172.21.15.52:0/3604532023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f52501074a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52440098a0 tx=0x7f5244009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 -- 172.21.15.52:0/3604532023 shutdown_connections 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 --2- 172.21.15.52:0/3604532023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f52501074a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 -- 172.21.15.52:0/3604532023 >> 172.21.15.52:0/3604532023 conn(0x7f52501018a0 msgr2=0x7f5250103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 -- 172.21.15.52:0/3604532023 shutdown_connections 2024-06-02T22:59:44.576 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.222+0000 7f525873b700 1 -- 172.21.15.52:0/3604532023 wait complete. 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f525873b700 1 Processor -- start 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f525873b700 1 -- start start 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f525873b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f525010ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f525873b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5250105be0 con 0x7f5250107080 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f52564d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f525010ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f52564d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f525010ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38296/0 (socket says 172.21.15.52:38296) 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.223+0000 7f52564d7700 1 -- 172.21.15.52:0/1146683380 learned_addr learned my addr 172.21.15.52:0/1146683380 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f52564d7700 1 -- 172.21.15.52:0/1146683380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f525010cf50 con 0x7f5250107080 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f52564d7700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f525010ca10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5244011a50 tx=0x7f5244011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:44.577 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5244003a80 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f525010b190 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f525010b670 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f5244003c20 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.224+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524401a650 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.225+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 5) ==== 46287+0+0 (secure 0 0 0) 0x7f524401a9f0 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.225+0000 7f52437fe700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.225+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 --> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f523c03bc20 con 0x7f523c039050 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.225+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f524404d5b0 con 0x7f5250107080 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.225+0000 7f5255cd6700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 msgr2=0x7f523c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.226+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.427+0000 7f5255cd6700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 msgr2=0x7f523c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:44.578 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.427+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.828+0000 7f5255cd6700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 msgr2=0x7f523c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:38.828+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:39.629+0000 7f5255cd6700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 msgr2=0x7f523c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:39.629+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:41.230+0000 7f5255cd6700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 msgr2=0x7f523c03b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/2620214668 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:41.231+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:43.025+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 6) ==== 46054+0+0 (secure 0 0 0) 0x7f5244024ad0 con 0x7f5250107080 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:43.026+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 msgr2=0x7f523c03b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:43.026+0000 7f52437fe700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.027+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f524404f600 con 0x7f5250107080 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.027+0000 7f52437fe700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f523c03c980 0x7f523c03ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.028+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f523c03bc20 con 0x7f523c03c980 2024-06-02T22:59:44.579 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.028+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f523c03c980 0x7f523c03ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.028+0000 7f5255cd6700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f523c03c980 0x7f523c03ed70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f524c003a10 tx=0x7f524c009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.029+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f523c03bc20 con 0x7f523c03c980 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.036+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f5234002630 con 0x7f523c03c980 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.036+0000 7f52437fe700 1 -- 172.21.15.52:0/1146683380 <== mgr.14118 v2:172.21.15.52:6800/540942425 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f5234002630 con 0x7f523c03c980 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f523c03c980 msgr2=0x7f523c03ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f523c03c980 0x7f523c03ed70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f524c003a10 tx=0x7f524c009290 comp rx=0 tx=0).stop 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 msgr2=0x7f525010ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f525010ca10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5244011a50 tx=0x7f5244011b30 comp rx=0 tx=0).stop 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 shutdown_connections 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5250107080 0x7f525010ca10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:44.580 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f523c03c980 0x7f523c03ed70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 --2- 172.21.15.52:0/1146683380 >> [v2:172.21.15.52:6800/2620214668,v1:172.21.15.52:6801/2620214668] conn(0x7f523c039050 0x7f523c03b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 >> 172.21.15.52:0/1146683380 conn(0x7f52501018a0 msgr2=0x7f5250102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 shutdown_connections 2024-06-02T22:59:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.037+0000 7f525873b700 1 -- 172.21.15.52:0/1146683380 wait complete. 2024-06-02T22:59:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 5 is available 2024-06-02T22:59:44.581 INFO:teuthology.orchestra.run.smithi052.stdout:Setting orchestrator backend to cephadm... 2024-06-02T22:59:45.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:45 smithi052 ceph-mon[32340]: [02/Jun/2024:22:59:43] ENGINE Serving on https://172.21.15.52:7150 2024-06-02T22:59:45.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:45 smithi052 ceph-mon[32340]: [02/Jun/2024:22:59:43] ENGINE Bus STARTED 2024-06-02T22:59:45.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:45 smithi052 ceph-mon[32340]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-02T22:59:45.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:45 smithi052 ceph-mon[32340]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-02T22:59:45.786 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.998+0000 7f20749f4700 1 Processor -- start 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.998+0000 7f20749f4700 1 -- start start 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f20749f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207010ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f20749f4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f207007be70 con 0x7f2070108750 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207010ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207010ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34428/0 (socket says 172.21.15.52:34428) 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206e59c700 1 -- 172.21.15.52:0/3437739590 learned_addr learned my addr 172.21.15.52:0/3437739590 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206e59c700 1 -- 172.21.15.52:0/3437739590 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207010b080 con 0x7f2070108750 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206e59c700 1 --2- 172.21.15.52:0/3437739590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207010ab40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20600098a0 tx=0x7f2060009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977504422903a9a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206dd9b700 1 -- 172.21.15.52:0/3437739590 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2060004160 con 0x7f2070108750 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206dd9b700 1 -- 172.21.15.52:0/3437739590 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2060004300 con 0x7f2070108750 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:44.999+0000 7f206dd9b700 1 -- 172.21.15.52:0/3437739590 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2060011a80 con 0x7f2070108750 2024-06-02T22:59:45.787 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 -- 172.21.15.52:0/3437739590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 msgr2=0x7f207010ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 --2- 172.21.15.52:0/3437739590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207010ab40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f20600098a0 tx=0x7f2060009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 -- 172.21.15.52:0/3437739590 shutdown_connections 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 --2- 172.21.15.52:0/3437739590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207010ab40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 -- 172.21.15.52:0/3437739590 >> 172.21.15.52:0/3437739590 conn(0x7f2070102160 msgr2=0x7f2070104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 -- 172.21.15.52:0/3437739590 shutdown_connections 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.000+0000 7f20749f4700 1 -- 172.21.15.52:0/3437739590 wait complete. 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f20749f4700 1 Processor -- start 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f20749f4700 1 -- start start 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f20749f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207019bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f20749f4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f207010c040 con 0x7f2070108750 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f206e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207019bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f206e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207019bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34438/0 (socket says 172.21.15.52:34438) 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.001+0000 7f206e59c700 1 -- 172.21.15.52:0/2701009293 learned_addr learned my addr 172.21.15.52:0/2701009293 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:45.788 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f206e59c700 1 -- 172.21.15.52:0/2701009293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207019c0f0 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f206e59c700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207019bbb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2060011a50 tx=0x7f2060011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2060003a80 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f2060003c20 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207019c380 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f206001a650 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.002+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f207019f070 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.003+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 7) ==== 46181+0+0 (secure 0 0 0) 0x7f2060024440 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.003+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2070063af0 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.003+0000 7f20677fe700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f205c038f30 0x7f205c03b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.003+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f206004d360 con 0x7f2070108750 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.003+0000 7f2067fff700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f205c038f30 0x7f205c03b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.004+0000 7f2067fff700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f205c038f30 0x7f205c03b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2058006fd0 tx=0x7f2058006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:45.789 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.009+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f206000e260 con 0x7f2070108750 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.080+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f2060024ad0 con 0x7f2070108750 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.237+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2070103340 con 0x7f205c038f30 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.242+0000 7f20677fe700 1 -- 172.21.15.52:0/2701009293 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2070103340 con 0x7f205c038f30 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f205c038f30 msgr2=0x7f205c03b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f205c038f30 0x7f205c03b3f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2058006fd0 tx=0x7f2058006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 msgr2=0x7f207019bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207019bbb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2060011a50 tx=0x7f2060011b30 comp rx=0 tx=0).stop 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 shutdown_connections 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f205c038f30 0x7f205c03b3f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 --2- 172.21.15.52:0/2701009293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2070108750 0x7f207019bbb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 >> 172.21.15.52:0/2701009293 conn(0x7f2070102160 msgr2=0x7f2070102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 shutdown_connections 2024-06-02T22:59:45.790 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:45.244+0000 7f20749f4700 1 -- 172.21.15.52:0/2701009293 wait complete. 2024-06-02T22:59:46.181 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:46 smithi052 ceph-mon[32340]: mgrmap e8: x(active, since 2s) 2024-06-02T22:59:46.182 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:46 smithi052 ceph-mon[32340]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:46.182 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:46 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:46.182 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:46 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T22:59:47.143 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:47 smithi052 ceph-mon[32340]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout value unchanged 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.219+0000 7fb1c7340700 1 Processor -- start 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.219+0000 7fb1c7340700 1 -- start start 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c7340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c010ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c7340700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c007be50 con 0x7fb1c0108760 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c50dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c010ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c50dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c010ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34444/0 (socket says 172.21.15.52:34444) 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c50dc700 1 -- 172.21.15.52:0/2902131280 learned_addr learned my addr 172.21.15.52:0/2902131280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c50dc700 1 -- 172.21.15.52:0/2902131280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c010b090 con 0x7fb1c0108760 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.220+0000 7fb1c50dc700 1 --2- 172.21.15.52:0/2902131280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c010ab50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0098a0 tx=0x7fb1bc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7abf5161e1f4727b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1b7fff700 1 -- 172.21.15.52:0/2902131280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1bc004160 con 0x7fb1c0108760 2024-06-02T22:59:47.152 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1b7fff700 1 -- 172.21.15.52:0/2902131280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb1bc004300 con 0x7fb1c0108760 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1b7fff700 1 -- 172.21.15.52:0/2902131280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1bc011a80 con 0x7fb1c0108760 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 -- 172.21.15.52:0/2902131280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 msgr2=0x7fb1c010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 --2- 172.21.15.52:0/2902131280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c010ab50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc0098a0 tx=0x7fb1bc009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 -- 172.21.15.52:0/2902131280 shutdown_connections 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 --2- 172.21.15.52:0/2902131280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c010ab50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 -- 172.21.15.52:0/2902131280 >> 172.21.15.52:0/2902131280 conn(0x7fb1c0102150 msgr2=0x7fb1c0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 -- 172.21.15.52:0/2902131280 shutdown_connections 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.221+0000 7fb1c7340700 1 -- 172.21.15.52:0/2902131280 wait complete. 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c7340700 1 Processor -- start 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c7340700 1 -- start start 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c7340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c7340700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1c010c000 con 0x7fb1c0108760 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c50dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:47.153 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c50dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c019bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34460/0 (socket says 172.21.15.52:34460) 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.222+0000 7fb1c50dc700 1 -- 172.21.15.52:0/2836033408 learned_addr learned my addr 172.21.15.52:0/2836033408 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1c50dc700 1 -- 172.21.15.52:0/2836033408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1c019c100 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1c50dc700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c019bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006af0 tx=0x7fb1bc011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1bc0039e0 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fb1bc003b80 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1c019c390 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1bc0196a0 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.223+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1c019f080 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.224+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fb1bc019950 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.224+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1c0063af0 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.224+0000 7fb1b67fc700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fb1ac039050 0x7fb1ac03b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.224+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb1bc04d620 con 0x7fb1c0108760 2024-06-02T22:59:47.154 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.224+0000 7fb1c48db700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fb1ac039050 0x7fb1ac03b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.225+0000 7fb1c48db700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fb1ac039050 0x7fb1ac03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0006fd0 tx=0x7fb1b0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.229+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb1bc023e20 con 0x7fb1c0108760 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.436+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fb1c0103390 con 0x7fb1ac039050 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.437+0000 7fb1b67fc700 1 -- 172.21.15.52:0/2836033408 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fb1c0103390 con 0x7fb1ac039050 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fb1ac039050 msgr2=0x7fb1ac03b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fb1ac039050 0x7fb1ac03b510 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0006fd0 tx=0x7fb1b0006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 msgr2=0x7fb1c019bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c019bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006af0 tx=0x7fb1bc011c40 comp rx=0 tx=0).stop 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 shutdown_connections 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fb1ac039050 0x7fb1ac03b510 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 --2- 172.21.15.52:0/2836033408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1c0108760 0x7fb1c019bbc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 >> 172.21.15.52:0/2836033408 conn(0x7fb1c0102150 msgr2=0x7fb1c0102c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:47.155 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 shutdown_connections 2024-06-02T22:59:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:46.439+0000 7fb1c7340700 1 -- 172.21.15.52:0/2836033408 wait complete. 2024-06-02T22:59:47.156 INFO:teuthology.orchestra.run.smithi052.stdout:Generating ssh key... 2024-06-02T22:59:48.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-mon[32340]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:48.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-mon[32340]: Generating ssh key... 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: Generating public/private rsa key pair. 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: Your identification has been saved in /tmp/tmp1_d8rjn5/key. 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: Your public key has been saved in /tmp/tmp1_d8rjn5/key.pub. 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: The key fingerprint is: 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: SHA256:4dLqnKs7SMIgcie30k7v/bdRXjfScDz1feo1E68YWlk ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: The key's randomart image is: 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: +---[RSA 3072]----+ 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: | .| 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: | . +| 2024-06-02T22:59:48.392 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: | . .E=+| 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: |+ o o o . o+.=| 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: |=. = .. S +.o==| 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: |..o + o o *.+=| 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: | o = .. . o + | 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: | . oo.o .. | 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: | o=*....... | 2024-06-02T22:59:48.393 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 22:59:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: +----[SHA256]-----+ 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a6eac700 1 Processor -- start 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a6eac700 1 -- start start 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a6eac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a007c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a6eac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a007c8b0 con 0x7fa8a007de80 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a4c48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a007c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a4c48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a007c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34472/0 (socket says 172.21.15.52:34472) 2024-06-02T22:59:48.660 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.583+0000 7fa8a4c48700 1 -- 172.21.15.52:0/2367841360 learned_addr learned my addr 172.21.15.52:0/2367841360 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.584+0000 7fa8a4c48700 1 -- 172.21.15.52:0/2367841360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a007ca30 con 0x7fa8a007de80 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.584+0000 7fa8a4c48700 1 --2- 172.21.15.52:0/2367841360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a007c2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098a0 tx=0x7fa894009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bd54c6551e28fc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.584+0000 7fa89f7fe700 1 -- 172.21.15.52:0/2367841360 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa894004160 con 0x7fa8a007de80 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.584+0000 7fa89f7fe700 1 -- 172.21.15.52:0/2367841360 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa894004300 con 0x7fa8a007de80 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.584+0000 7fa89f7fe700 1 -- 172.21.15.52:0/2367841360 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa894011a80 con 0x7fa8a007de80 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 -- 172.21.15.52:0/2367841360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 msgr2=0x7fa8a007c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 --2- 172.21.15.52:0/2367841360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a007c2e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa8940098a0 tx=0x7fa894009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 -- 172.21.15.52:0/2367841360 shutdown_connections 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 --2- 172.21.15.52:0/2367841360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a007c2e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:48.661 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 -- 172.21.15.52:0/2367841360 >> 172.21.15.52:0/2367841360 conn(0x7fa8a0103290 msgr2=0x7fa8a01056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 -- 172.21.15.52:0/2367841360 shutdown_connections 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 -- 172.21.15.52:0/2367841360 wait complete. 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 Processor -- start 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.585+0000 7fa8a6eac700 1 -- start start 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a6eac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a6eac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a010e260 con 0x7fa8a007de80 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a4c48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a4c48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a019ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34486/0 (socket says 172.21.15.52:34486) 2024-06-02T22:59:48.662 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a4c48700 1 -- 172.21.15.52:0/372529241 learned_addr learned my addr 172.21.15.52:0/372529241 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a4c48700 1 -- 172.21.15.52:0/372529241 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a019e330 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a4c48700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a019ddf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa894006af0 tx=0x7fa894011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa894003930 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7fa894003ad0 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a019e5c0 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.586+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8940195e0 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.587+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a01a12b0 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.587+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7fa8940198e0 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.587+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8a0063af0 con 0x7fa8a007de80 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.588+0000 7fa89dffb700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fa890039000 0x7fa89003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:48.663 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.588+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa89404d710 con 0x7fa8a007de80 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.588+0000 7fa89ffff700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fa890039000 0x7fa89003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.588+0000 7fa89ffff700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fa890039000 0x7fa89003b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006fd0 tx=0x7fa88c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.593+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa894019cd0 con 0x7fa8a007de80 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:47.799+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7fa8a0104560 con 0x7fa890039000 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.142+0000 7fa89dffb700 1 -- 172.21.15.52:0/372529241 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa8a0104560 con 0x7fa890039000 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fa890039000 msgr2=0x7fa89003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fa890039000 0x7fa89003b4c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa88c006fd0 tx=0x7fa88c006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 msgr2=0x7fa8a019ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a019ddf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa894006af0 tx=0x7fa894011b70 comp rx=0 tx=0).stop 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 shutdown_connections 2024-06-02T22:59:48.664 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7fa890039000 0x7fa89003b4c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:48.665 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 --2- 172.21.15.52:0/372529241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8a007de80 0x7fa8a019ddf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:48.665 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 >> 172.21.15.52:0/372529241 conn(0x7fa8a0103290 msgr2=0x7fa8a0103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:48.665 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 shutdown_connections 2024-06-02T22:59:48.665 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:48.145+0000 7fa8a6eac700 1 -- 172.21.15.52:0/372529241 wait complete. 2024-06-02T22:59:49.289 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:49 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:49.289 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:49 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.071+0000 7f7108ecc700 1 Processor -- start 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.071+0000 7f7108ecc700 1 -- start start 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f7108ecc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f7108ecc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f710407c8d0 con 0x7f710407dea0 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f710259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:49.889 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f710259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34500/0 (socket says 172.21.15.52:34500) 2024-06-02T22:59:49.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f710259c700 1 -- 172.21.15.52:0/646363512 learned_addr learned my addr 172.21.15.52:0/646363512 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:49.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f710259c700 1 -- 172.21.15.52:0/646363512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710407ca50 con 0x7f710407dea0 2024-06-02T22:59:49.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f710259c700 1 --2- 172.21.15.52:0/646363512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710407c300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70f40098a0 tx=0x7f70f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c594855e1d310254 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:49.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f7101d9b700 1 -- 172.21.15.52:0/646363512 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f4004160 con 0x7f710407dea0 2024-06-02T22:59:49.890 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f7101d9b700 1 -- 172.21.15.52:0/646363512 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f70f4004300 con 0x7f710407dea0 2024-06-02T22:59:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.072+0000 7f7101d9b700 1 -- 172.21.15.52:0/646363512 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f4011a80 con 0x7f710407dea0 2024-06-02T22:59:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 -- 172.21.15.52:0/646363512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 msgr2=0x7f710407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 --2- 172.21.15.52:0/646363512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710407c300 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70f40098a0 tx=0x7f70f4009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 -- 172.21.15.52:0/646363512 shutdown_connections 2024-06-02T22:59:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 --2- 172.21.15.52:0/646363512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710407c300 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:49.891 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 -- 172.21.15.52:0/646363512 >> 172.21.15.52:0/646363512 conn(0x7f7104103290 msgr2=0x7f71041056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 -- 172.21.15.52:0/646363512 shutdown_connections 2024-06-02T22:59:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.073+0000 7f7108ecc700 1 -- 172.21.15.52:0/646363512 wait complete. 2024-06-02T22:59:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f7108ecc700 1 Processor -- start 2024-06-02T22:59:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f7108ecc700 1 -- start start 2024-06-02T22:59:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f7108ecc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710419de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:49.893 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f7108ecc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f710410e240 con 0x7f710407dea0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f710259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710419de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f710259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710419de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34506/0 (socket says 172.21.15.52:34506) 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.074+0000 7f710259c700 1 -- 172.21.15.52:0/2494084755 learned_addr learned my addr 172.21.15.52:0/2494084755 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f710259c700 1 -- 172.21.15.52:0/2494084755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f710419e3a0 con 0x7f710407dea0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f710259c700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710419de60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006af0 tx=0x7f70f4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f4003930 con 0x7f710407dea0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f70f4003ad0 con 0x7f710407dea0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f710419e630 con 0x7f710407dea0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70f40195e0 con 0x7f710407dea0 2024-06-02T22:59:49.894 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.075+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71041a1320 con 0x7f710407dea0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.076+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f70f4019940 con 0x7f710407dea0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.076+0000 7f70fbfff700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f70f0039000 0x7f70f003b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.076+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7104063af0 con 0x7f710407dea0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.076+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f70f404d590 con 0x7f710407dea0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.076+0000 7f70f9d9b700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f70f0039000 0x7f70f003b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.077+0000 7f70f9d9b700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f70f0039000 0x7f70f003b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70ec006fd0 tx=0x7f70ec006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.082+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f70f400fda0 con 0x7f710407dea0 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.290+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f7104104530 con 0x7f70f0039000 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.290+0000 7f70fbfff700 1 -- 172.21.15.52:0/2494084755 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f7104104530 con 0x7f70f0039000 2024-06-02T22:59:49.895 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f70f0039000 msgr2=0x7f70f003b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f70f0039000 0x7f70f003b4c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f70ec006fd0 tx=0x7f70ec006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 msgr2=0x7f710419de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710419de60 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f70f4006af0 tx=0x7f70f4011b70 comp rx=0 tx=0).stop 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 shutdown_connections 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f70f0039000 0x7f70f003b4c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 --2- 172.21.15.52:0/2494084755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f710407dea0 0x7f710419de60 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.293+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 >> 172.21.15.52:0/2494084755 conn(0x7f7104103290 msgr2=0x7f7104103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.294+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 shutdown_connections 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:49.294+0000 7f7108ecc700 1 -- 172.21.15.52:0/2494084755 wait complete. 2024-06-02T22:59:49.896 INFO:teuthology.orchestra.run.smithi052.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-06-02T22:59:49.897 INFO:teuthology.orchestra.run.smithi052.stdout:Adding key to root@localhost authorized_keys... 2024-06-02T22:59:49.897 INFO:teuthology.orchestra.run.smithi052.stdout:Adding host smithi052... 2024-06-02T22:59:50.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:50 smithi052 ceph-mon[32340]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:51.277 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:51 smithi052 ceph-mon[32340]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:52.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:52 smithi052 ceph-mon[32340]: Deploying cephadm binary to smithi052 2024-06-02T22:59:55.363 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Added host 'smithi052' with addr '172.21.15.52' 2024-06-02T22:59:55.363 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.286+0000 7f8fccccc700 1 Processor -- start 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.286+0000 7f8fccccc700 1 -- start start 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fccccc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc8106950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fccccc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc807be70 con 0x7f8fc8106530 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc8106950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc8106950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34514/0 (socket says 172.21.15.52:34514) 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc659c700 1 -- 172.21.15.52:0/1441511124 learned_addr learned my addr 172.21.15.52:0/1441511124 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc659c700 1 -- 172.21.15.52:0/1441511124 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc8106e90 con 0x7f8fc8106530 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc659c700 1 --2- 172.21.15.52:0/1441511124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc8106950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80098a0 tx=0x7f8fb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952c4a40c8d0e9f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc559a700 1 -- 172.21.15.52:0/1441511124 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb8004160 con 0x7f8fc8106530 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc559a700 1 -- 172.21.15.52:0/1441511124 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8fb8004300 con 0x7f8fc8106530 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.287+0000 7f8fc559a700 1 -- 172.21.15.52:0/1441511124 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb8011a80 con 0x7f8fc8106530 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 -- 172.21.15.52:0/1441511124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 msgr2=0x7f8fc8106950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:55.364 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 --2- 172.21.15.52:0/1441511124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc8106950 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8fb80098a0 tx=0x7f8fb8009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 -- 172.21.15.52:0/1441511124 shutdown_connections 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 --2- 172.21.15.52:0/1441511124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc8106950 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 -- 172.21.15.52:0/1441511124 >> 172.21.15.52:0/1441511124 conn(0x7f8fc8102150 msgr2=0x7f8fc8104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 -- 172.21.15.52:0/1441511124 shutdown_connections 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.288+0000 7f8fccccc700 1 -- 172.21.15.52:0/1441511124 wait complete. 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fccccc700 1 Processor -- start 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fccccc700 1 -- start start 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fccccc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc819ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fccccc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc8107ec0 con 0x7f8fc8106530 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc819ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc819ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34516/0 (socket says 172.21.15.52:34516) 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.289+0000 7f8fc659c700 1 -- 172.21.15.52:0/3167270616 learned_addr learned my addr 172.21.15.52:0/3167270616 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fc659c700 1 -- 172.21.15.52:0/3167270616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc819e310 con 0x7f8fc8106530 2024-06-02T22:59:55.365 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fc659c700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc819ddd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8011a50 tx=0x7f8fb8011b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb8003a80 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7f8fb8003c20 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc819e5a0 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8fb801a650 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.290+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc81a1290 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.291+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f8fb801a900 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.291+0000 7f8fb77fe700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8fb0039100 0x7f8fb003b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.291+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f8fb804d5a0 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.291+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc8063af0 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.291+0000 7f8fc5d9b700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8fb0039100 0x7f8fb003b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.292+0000 7f8fc5d9b700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8fb0039100 0x7f8fb003b5c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc006fd0 tx=0x7f8fbc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.296+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8fb8019c70 con 0x7f8fc8106530 2024-06-02T22:59:55.366 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:50.509+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi052", "addr": "172.21.15.52", "target": ["mon-mgr", ""]}) -- 0x7f8fc802df80 con 0x7f8fb0039100 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.788+0000 7f8fb77fe700 1 -- 172.21.15.52:0/3167270616 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f8fc802df80 con 0x7f8fb0039100 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8fb0039100 msgr2=0x7f8fb003b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8fb0039100 0x7f8fb003b5c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc006fd0 tx=0x7f8fbc006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 msgr2=0x7f8fc819ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc819ddd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8011a50 tx=0x7f8fb8011b30 comp rx=0 tx=0).stop 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 shutdown_connections 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8fb0039100 0x7f8fb003b5c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 --2- 172.21.15.52:0/3167270616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8fc8106530 0x7f8fc819ddd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 >> 172.21.15.52:0/3167270616 conn(0x7f8fc8102150 msgr2=0x7f8fc81949f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 shutdown_connections 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:54.791+0000 7f8fccccc700 1 -- 172.21.15.52:0/3167270616 wait complete. 2024-06-02T22:59:55.367 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mon service... 2024-06-02T22:59:56.014 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:55 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:56.014 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:55 smithi052 ceph-mon[32340]: Added host smithi052 2024-06-02T22:59:56.014 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:55 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T22:59:56.689 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-06-02T22:59:56.689 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.757+0000 7faa2b7f2700 1 Processor -- start 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.757+0000 7faa2b7f2700 1 -- start start 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2b7f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa2410ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2b7f2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa2407be30 con 0x7faa24108790 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2958e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa2410ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2958e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa2410ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42568/0 (socket says 172.21.15.52:42568) 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2958e700 1 -- 172.21.15.52:0/731859996 learned_addr learned my addr 172.21.15.52:0/731859996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2958e700 1 -- 172.21.15.52:0/731859996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2410b0c0 con 0x7faa24108790 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.758+0000 7faa2958e700 1 --2- 172.21.15.52:0/731859996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa2410ab80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa200098a0 tx=0x7faa20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec3060fe87a20979 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa1bfff700 1 -- 172.21.15.52:0/731859996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa20004160 con 0x7faa24108790 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa1bfff700 1 -- 172.21.15.52:0/731859996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faa20004300 con 0x7faa24108790 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa1bfff700 1 -- 172.21.15.52:0/731859996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa20011a80 con 0x7faa24108790 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa2b7f2700 1 -- 172.21.15.52:0/731859996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 msgr2=0x7faa2410ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:56.690 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa2b7f2700 1 --2- 172.21.15.52:0/731859996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa2410ab80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa200098a0 tx=0x7faa20009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa2b7f2700 1 -- 172.21.15.52:0/731859996 shutdown_connections 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa2b7f2700 1 --2- 172.21.15.52:0/731859996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa2410ab80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.759+0000 7faa2b7f2700 1 -- 172.21.15.52:0/731859996 >> 172.21.15.52:0/731859996 conn(0x7faa24102160 msgr2=0x7faa24104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.760+0000 7faa2b7f2700 1 -- 172.21.15.52:0/731859996 shutdown_connections 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.760+0000 7faa2b7f2700 1 -- 172.21.15.52:0/731859996 wait complete. 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.760+0000 7faa2b7f2700 1 Processor -- start 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.760+0000 7faa2b7f2700 1 -- start start 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.760+0000 7faa2b7f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa24108010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.760+0000 7faa2b7f2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa2410c020 con 0x7faa24108790 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2958e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa24108010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2958e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa24108010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42580/0 (socket says 172.21.15.52:42580) 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2958e700 1 -- 172.21.15.52:0/1696508813 learned_addr learned my addr 172.21.15.52:0/1696508813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2958e700 1 -- 172.21.15.52:0/1696508813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa24108550 con 0x7faa24108790 2024-06-02T22:59:56.691 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2958e700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa24108010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faa20006af0 tx=0x7faa20011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa200039e0 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa24106740 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7faa20003b80 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faa200196a0 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.761+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa24106c80 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.762+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7faa20019970 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.762+0000 7faa1a7fc700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7faa10039050 0x7faa1003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.762+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7faa2004d630 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.763+0000 7faa28d8d700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7faa10039050 0x7faa1003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.763+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa24063af0 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.763+0000 7faa28d8d700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7faa10039050 0x7faa1003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa14006fd0 tx=0x7faa14006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.768+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faa2000bc10 con 0x7faa24108790 2024-06-02T22:59:56.692 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:55.995+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7faa241075b0 con 0x7faa10039050 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.000+0000 7faa1a7fc700 1 -- 172.21.15.52:0/1696508813 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faa241075b0 con 0x7faa10039050 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7faa10039050 msgr2=0x7faa1003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7faa10039050 0x7faa1003b510 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7faa14006fd0 tx=0x7faa14006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 msgr2=0x7faa24108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa24108010 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faa20006af0 tx=0x7faa20011c40 comp rx=0 tx=0).stop 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 shutdown_connections 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7faa10039050 0x7faa1003b510 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.002+0000 7faa2b7f2700 1 --2- 172.21.15.52:0/1696508813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa24108790 0x7faa24108010 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.003+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 >> 172.21.15.52:0/1696508813 conn(0x7faa24102160 msgr2=0x7faa24102c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.003+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 shutdown_connections 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:56.003+0000 7faa2b7f2700 1 -- 172.21.15.52:0/1696508813 wait complete. 2024-06-02T22:59:56.693 INFO:teuthology.orchestra.run.smithi052.stdout:Deploying unmanaged mgr service... 2024-06-02T22:59:57.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:57 smithi052 ceph-mon[32340]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:57.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:57 smithi052 ceph-mon[32340]: Saving service mon spec with placement count:5 2024-06-02T22:59:57.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:57 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-06-02T22:59:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.139+0000 7ff0d9b92700 1 Processor -- start 2024-06-02T22:59:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d9b92700 1 -- start start 2024-06-02T22:59:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d9b92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d4108090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d9b92700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4108660 con 0x7ff0d4107c70 2024-06-02T22:59:58.593 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d4108090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d4108090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42596/0 (socket says 172.21.15.52:42596) 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d37fe700 1 -- 172.21.15.52:0/3566054431 learned_addr learned my addr 172.21.15.52:0/3566054431 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d37fe700 1 -- 172.21.15.52:0/3566054431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d4108e70 con 0x7ff0d4107c70 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.140+0000 7ff0d37fe700 1 --2- 172.21.15.52:0/3566054431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d4108090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40098a0 tx=0x7ff0c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a693a191b264507 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d2ffd700 1 -- 172.21.15.52:0/3566054431 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c4004160 con 0x7ff0d4107c70 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d2ffd700 1 -- 172.21.15.52:0/3566054431 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff0c4004300 con 0x7ff0d4107c70 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d2ffd700 1 -- 172.21.15.52:0/3566054431 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c4011a80 con 0x7ff0d4107c70 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 -- 172.21.15.52:0/3566054431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 msgr2=0x7ff0d4108090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 --2- 172.21.15.52:0/3566054431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d4108090 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40098a0 tx=0x7ff0c4009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 -- 172.21.15.52:0/3566054431 shutdown_connections 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 --2- 172.21.15.52:0/3566054431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d4108090 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 -- 172.21.15.52:0/3566054431 >> 172.21.15.52:0/3566054431 conn(0x7ff0d4103290 msgr2=0x7ff0d41056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:58.594 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 -- 172.21.15.52:0/3566054431 shutdown_connections 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.141+0000 7ff0d9b92700 1 -- 172.21.15.52:0/3566054431 wait complete. 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d9b92700 1 Processor -- start 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d9b92700 1 -- start start 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d9b92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d419bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d9b92700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d410c220 con 0x7ff0d4107c70 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d419bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d419bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42598/0 (socket says 172.21.15.52:42598) 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.142+0000 7ff0d37fe700 1 -- 172.21.15.52:0/4223458358 learned_addr learned my addr 172.21.15.52:0/4223458358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d37fe700 1 -- 172.21.15.52:0/4223458358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d419c160 con 0x7ff0d4107c70 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d37fe700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d419bc20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c4004290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c40039c0 con 0x7ff0d4107c70 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d419c3f0 con 0x7ff0d4107c70 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1071+0+0 (secure 0 0 0) 0x7ff0c4003b60 con 0x7ff0d4107c70 2024-06-02T22:59:58.595 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff0c401a6f0 con 0x7ff0d4107c70 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.143+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d419f0e0 con 0x7ff0d4107c70 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.144+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7ff0c401a9a0 con 0x7ff0d4107c70 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.144+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d4063af0 con 0x7ff0d4107c70 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.144+0000 7ff0d17fa700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7ff0c0039050 0x7ff0c003b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.144+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff0c404d3c0 con 0x7ff0d4107c70 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.145+0000 7ff0caffd700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7ff0c0039050 0x7ff0c003b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.145+0000 7ff0caffd700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7ff0c0039050 0x7ff0c003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc006fd0 tx=0x7ff0bc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.150+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff0c4020240 con 0x7ff0d4107c70 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.375+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ff0d4104510 con 0x7ff0c0039050 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.379+0000 7ff0d17fa700 1 -- 172.21.15.52:0/4223458358 <== mgr.14118 v2:172.21.15.52:6800/540942425 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ff0d4104510 con 0x7ff0c0039050 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7ff0c0039050 msgr2=0x7ff0c003b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7ff0c0039050 0x7ff0c003b510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc006fd0 tx=0x7ff0bc006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 msgr2=0x7ff0d419bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:58.596 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d419bc20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c4004290 comp rx=0 tx=0).stop 2024-06-02T22:59:58.597 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 shutdown_connections 2024-06-02T22:59:58.597 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7ff0c0039050 0x7ff0c003b510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:58.597 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 --2- 172.21.15.52:0/4223458358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0d4107c70 0x7ff0d419bc20 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:58.597 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 >> 172.21.15.52:0/4223458358 conn(0x7ff0d4103290 msgr2=0x7ff0d4103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:58.597 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.381+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 shutdown_connections 2024-06-02T22:59:58.597 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:57.382+0000 7ff0d9b92700 1 -- 172.21.15.52:0/4223458358 wait complete. 2024-06-02T22:59:58.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:58 smithi052 ceph-mon[32340]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T22:59:58.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:58 smithi052 ceph-mon[32340]: Saving service mgr spec with placement count:2 2024-06-02T22:59:58.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:58 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:58.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:58 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T22:59:59.817 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469aac0700 1 Processor -- start 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469aac0700 1 -- start start 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469aac0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f4694106930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469aac0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f469407be50 con 0x7f4694106510 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469885c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f4694106930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469885c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f4694106930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42606/0 (socket says 172.21.15.52:42606) 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469885c700 1 -- 172.21.15.52:0/1394047838 learned_addr learned my addr 172.21.15.52:0/1394047838 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.041+0000 7f469885c700 1 -- 172.21.15.52:0/1394047838 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4694106e70 con 0x7f4694106510 2024-06-02T22:59:59.818 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.042+0000 7f469885c700 1 --2- 172.21.15.52:0/1394047838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f4694106930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46880098a0 tx=0x7f4688009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7afb9a77ccc7d515 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.042+0000 7f46937fe700 1 -- 172.21.15.52:0/1394047838 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4688004160 con 0x7f4694106510 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.042+0000 7f46937fe700 1 -- 172.21.15.52:0/1394047838 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4688004300 con 0x7f4694106510 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.042+0000 7f46937fe700 1 -- 172.21.15.52:0/1394047838 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4688011a50 con 0x7f4694106510 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 -- 172.21.15.52:0/1394047838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 msgr2=0x7f4694106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 --2- 172.21.15.52:0/1394047838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f4694106930 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f46880098a0 tx=0x7f4688009bb0 comp rx=0 tx=0).stop 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 -- 172.21.15.52:0/1394047838 shutdown_connections 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 --2- 172.21.15.52:0/1394047838 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f4694106930 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 -- 172.21.15.52:0/1394047838 >> 172.21.15.52:0/1394047838 conn(0x7f4694102130 msgr2=0x7f4694104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 -- 172.21.15.52:0/1394047838 shutdown_connections 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 -- 172.21.15.52:0/1394047838 wait complete. 2024-06-02T22:59:59.819 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 Processor -- start 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.043+0000 7f469aac0700 1 -- start start 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469aac0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f469419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469aac0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4694107ee0 con 0x7f4694106510 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469885c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f469419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469885c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f469419dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42616/0 (socket says 172.21.15.52:42616) 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469885c700 1 -- 172.21.15.52:0/1594069723 learned_addr learned my addr 172.21.15.52:0/1594069723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469885c700 1 -- 172.21.15.52:0/1594069723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469419e250 con 0x7f4694106510 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469885c700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f469419dd10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4688011a20 tx=0x7f4688011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4688003b70 con 0x7f4694106510 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f468801a8e0 con 0x7f4694106510 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469419e4e0 con 0x7f4694106510 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.044+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4688024990 con 0x7f4694106510 2024-06-02T22:59:59.820 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.045+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a11c0 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.045+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f4688022020 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.046+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4694063af0 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.046+0000 7f4691ffb700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f4684039000 0x7f468403b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.046+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f468804d5d0 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.046+0000 7f4693fff700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f4684039000 0x7f468403b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.046+0000 7f4693fff700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f4684039000 0x7f468403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4680006fd0 tx=0x7f4680006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.055+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f468800fdd0 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.263+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f469402df30 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.266+0000 7f4691ffb700 1 -- 172.21.15.52:0/1594069723 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f468802d910 con 0x7f4694106510 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f4684039000 msgr2=0x7f468403b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f4684039000 0x7f468403b4c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4680006fd0 tx=0x7f4680006e40 comp rx=0 tx=0).stop 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 msgr2=0x7f469419dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T22:59:59.821 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f469419dd10 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4688011a20 tx=0x7f4688011b00 comp rx=0 tx=0).stop 2024-06-02T22:59:59.822 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 shutdown_connections 2024-06-02T22:59:59.822 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f4684039000 0x7f468403b4c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:59.822 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 --2- 172.21.15.52:0/1594069723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4694106510 0x7f469419dd10 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T22:59:59.822 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.268+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 >> 172.21.15.52:0/1594069723 conn(0x7f4694102130 msgr2=0x7f4694194930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T22:59:59.822 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.269+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 shutdown_connections 2024-06-02T22:59:59.822 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T22:59:59.269+0000 7f469aac0700 1 -- 172.21.15.52:0/1594069723 wait complete. 2024-06-02T23:00:00.204 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:59 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T23:00:00.205 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 22:59:59 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1594069723' entity='client.admin' 2024-06-02T23:00:01.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f69652a4700 1 Processor -- start 2024-06-02T23:00:01.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f69652a4700 1 -- start start 2024-06-02T23:00:01.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f69652a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:01.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f69652a4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69601075a0 con 0x7f6960106bb0 2024-06-02T23:00:01.101 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f695effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f695effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42632/0 (socket says 172.21.15.52:42632) 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.226+0000 7f695effd700 1 -- 172.21.15.52:0/1145428330 learned_addr learned my addr 172.21.15.52:0/1145428330 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.227+0000 7f695effd700 1 -- 172.21.15.52:0/1145428330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6960107db0 con 0x7f6960106bb0 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.227+0000 7f695effd700 1 --2- 172.21.15.52:0/1145428330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960106fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69500098a0 tx=0x7f6950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113aa6767e231514 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.227+0000 7f695dffb700 1 -- 172.21.15.52:0/1145428330 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6950004160 con 0x7f6960106bb0 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.227+0000 7f695dffb700 1 -- 172.21.15.52:0/1145428330 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950004300 con 0x7f6960106bb0 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.227+0000 7f695dffb700 1 -- 172.21.15.52:0/1145428330 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6950011a50 con 0x7f6960106bb0 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 -- 172.21.15.52:0/1145428330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 msgr2=0x7f6960106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 --2- 172.21.15.52:0/1145428330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960106fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f69500098a0 tx=0x7f6950009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 -- 172.21.15.52:0/1145428330 shutdown_connections 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 --2- 172.21.15.52:0/1145428330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960106fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 -- 172.21.15.52:0/1145428330 >> 172.21.15.52:0/1145428330 conn(0x7f6960102170 msgr2=0x7f6960104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:01.102 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 -- 172.21.15.52:0/1145428330 shutdown_connections 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 -- 172.21.15.52:0/1145428330 wait complete. 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.228+0000 7f69652a4700 1 Processor -- start 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f69652a4700 1 -- start start 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f69652a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960191d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f69652a4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f696010b160 con 0x7f6960106bb0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f695effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960191d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f695effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960191d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42642/0 (socket says 172.21.15.52:42642) 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f695effd700 1 -- 172.21.15.52:0/2366012508 learned_addr learned my addr 172.21.15.52:0/2366012508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f695effd700 1 -- 172.21.15.52:0/2366012508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6960192260 con 0x7f6960106bb0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f695effd700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960191d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6950011710 tx=0x7f6950011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69500039c0 con 0x7f6960106bb0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6950003b60 con 0x7f6960106bb0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6960190450 con 0x7f6960106bb0 2024-06-02T23:00:01.103 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.229+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f695001a690 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.230+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6960190990 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.231+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f695001a940 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.231+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6960063af0 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.231+0000 7f6947fff700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f6948039050 0x7f694803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.231+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f695004d7c0 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.231+0000 7f695e7fc700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f6948039050 0x7f694803b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.231+0000 7f695e7fc700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f6948039050 0x7f694803b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6954006fd0 tx=0x7f6954006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.236+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6950019c70 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.449+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f6960195a80 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.452+0000 7f6947fff700 1 -- 172.21.15.52:0/2366012508 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f695002cde0 con 0x7f6960106bb0 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f6948039050 msgr2=0x7f694803b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:01.104 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f6948039050 0x7f694803b510 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f6954006fd0 tx=0x7f6954006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 msgr2=0x7f6960191d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960191d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6950011710 tx=0x7f6950011ad0 comp rx=0 tx=0).stop 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 shutdown_connections 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f6948039050 0x7f694803b510 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 --2- 172.21.15.52:0/2366012508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6960106bb0 0x7f6960191d20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.454+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 >> 172.21.15.52:0/2366012508 conn(0x7f6960102170 msgr2=0x7f6960102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.455+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 shutdown_connections 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:00.455+0000 7f69652a4700 1 -- 172.21.15.52:0/2366012508 wait complete. 2024-06-02T23:00:01.105 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling the dashboard module... 2024-06-02T23:00:01.772 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:01 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2366012508' entity='client.admin' 2024-06-02T23:00:02.707 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2075831505' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-06-02T23:00:02.707 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-mon[32340]: from='mgr.14118 172.21.15.52:0/2188194298' entity='mgr.x' 2024-06-02T23:00:02.708 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: ignoring --setuser ceph since I am not root 2024-06-02T23:00:02.708 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: ignoring --setgroup ceph since I am not root 2024-06-02T23:00:02.708 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:02.643+0000 7f28c4d91200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T23:00:03.060 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:02.844+0000 7f28c4d91200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T23:00:03.060 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:02.952+0000 7f28c4d91200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T23:00:03.067 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.637+0000 7f6969605700 1 Processor -- start 2024-06-02T23:00:03.067 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.637+0000 7f6969605700 1 -- start start 2024-06-02T23:00:03.067 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.637+0000 7f6969605700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f696410ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:03.068 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.637+0000 7f6969605700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f696407be70 con 0x7f6964108740 2024-06-02T23:00:03.068 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f696410ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:03.068 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f696410ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42654/0 (socket says 172.21.15.52:42654) 2024-06-02T23:00:03.068 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6962ffd700 1 -- 172.21.15.52:0/1659969431 learned_addr learned my addr 172.21.15.52:0/1659969431 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6962ffd700 1 -- 172.21.15.52:0/1659969431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f696410b070 con 0x7f6964108740 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6962ffd700 1 --2- 172.21.15.52:0/1659969431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f696410ab30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69540098a0 tx=0x7f6954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=842471d6d8cbfe97 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6961ffb700 1 -- 172.21.15.52:0/1659969431 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6954004160 con 0x7f6964108740 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6961ffb700 1 -- 172.21.15.52:0/1659969431 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6954004300 con 0x7f6964108740 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.638+0000 7f6961ffb700 1 -- 172.21.15.52:0/1659969431 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6954011a50 con 0x7f6964108740 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 -- 172.21.15.52:0/1659969431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 msgr2=0x7f696410ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:03.069 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 --2- 172.21.15.52:0/1659969431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f696410ab30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69540098a0 tx=0x7f6954009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 -- 172.21.15.52:0/1659969431 shutdown_connections 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 --2- 172.21.15.52:0/1659969431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f696410ab30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 -- 172.21.15.52:0/1659969431 >> 172.21.15.52:0/1659969431 conn(0x7f6964102150 msgr2=0x7f6964104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 -- 172.21.15.52:0/1659969431 shutdown_connections 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 -- 172.21.15.52:0/1659969431 wait complete. 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.639+0000 7f6969605700 1 Processor -- start 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6969605700 1 -- start start 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6969605700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f6964107fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6969605700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f696410c070 con 0x7f6964108740 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f6964107fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6962ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f6964107fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42660/0 (socket says 172.21.15.52:42660) 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6962ffd700 1 -- 172.21.15.52:0/2075831505 learned_addr learned my addr 172.21.15.52:0/2075831505 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:03.070 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6962ffd700 1 -- 172.21.15.52:0/2075831505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6964108500 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6962ffd700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f6964107fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6954011a20 tx=0x7f6954011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6954003b70 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f695401a8e0 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69641066f0 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.640+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6954024990 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.641+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6964106c30 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.641+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 8) ==== 46287+0+0 (secure 0 0 0) 0x7f6954022020 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.641+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6964063af0 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.642+0000 7f694bfff700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f694c039000 0x7f694c03b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.642+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f695404d4b0 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.642+0000 7f69627fc700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f694c039000 0x7f694c03b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.642+0000 7f69627fc700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f694c039000 0x7f694c03b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6958006fd0 tx=0x7f6958006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.647+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f695400be30 con 0x7f6964108740 2024-06-02T23:00:03.071 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:01.911+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f6964106eb0 con 0x7f6964108740 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.454+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f695402c510 con 0x7f6964108740 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.456+0000 7f694bfff700 1 -- 172.21.15.52:0/2075831505 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f695402dc90 con 0x7f6964108740 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f694c039000 msgr2=0x7f694c03b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f694c039000 0x7f694c03b4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f6958006fd0 tx=0x7f6958006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 msgr2=0x7f6964107fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f6964107fc0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6954011a20 tx=0x7f6954011b00 comp rx=0 tx=0).stop 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 shutdown_connections 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f694c039000 0x7f694c03b4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 --2- 172.21.15.52:0/2075831505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6964108740 0x7f6964107fc0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 >> 172.21.15.52:0/2075831505 conn(0x7f6964102150 msgr2=0x7f6964102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 shutdown_connections 2024-06-02T23:00:03.072 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:02.458+0000 7f6969605700 1 -- 172.21.15.52:0/2075831505 wait complete. 2024-06-02T23:00:03.311 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:03 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:03.283+0000 7f28c4d91200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T23:00:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:03 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2075831505' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-06-02T23:00:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:03 smithi052 ceph-mon[32340]: mgrmap e9: x(active, since 19s) 2024-06-02T23:00:03.650 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:03 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:03.374+0000 7f28c4d91200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "available": true, 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.485+0000 7f88b341b700 1 Processor -- start 2024-06-02T23:00:04.358 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.485+0000 7f88b341b700 1 -- start start 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b341b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b341b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac107580 con 0x7f88ac106b90 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b11b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b11b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42692/0 (socket says 172.21.15.52:42692) 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b11b7700 1 -- 172.21.15.52:0/1778784407 learned_addr learned my addr 172.21.15.52:0/1778784407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b11b7700 1 -- 172.21.15.52:0/1778784407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac107d90 con 0x7f88ac106b90 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.486+0000 7f88b11b7700 1 --2- 172.21.15.52:0/1778784407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac106fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88a80098a0 tx=0x7f88a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d621684d009862 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.487+0000 7f88a3fff700 1 -- 172.21.15.52:0/1778784407 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a8004160 con 0x7f88ac106b90 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.487+0000 7f88a3fff700 1 -- 172.21.15.52:0/1778784407 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a8004300 con 0x7f88ac106b90 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.487+0000 7f88a3fff700 1 -- 172.21.15.52:0/1778784407 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a8011a50 con 0x7f88ac106b90 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 -- 172.21.15.52:0/1778784407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 msgr2=0x7f88ac106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 --2- 172.21.15.52:0/1778784407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac106fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88a80098a0 tx=0x7f88a8009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:04.359 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 -- 172.21.15.52:0/1778784407 shutdown_connections 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 --2- 172.21.15.52:0/1778784407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac106fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 -- 172.21.15.52:0/1778784407 >> 172.21.15.52:0/1778784407 conn(0x7f88ac102150 msgr2=0x7f88ac104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 -- 172.21.15.52:0/1778784407 shutdown_connections 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 -- 172.21.15.52:0/1778784407 wait complete. 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 Processor -- start 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.488+0000 7f88b341b700 1 -- start start 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b341b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b341b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f88ac10b140 con 0x7f88ac106b90 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b11b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b11b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42704/0 (socket says 172.21.15.52:42704) 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b11b7700 1 -- 172.21.15.52:0/881934811 learned_addr learned my addr 172.21.15.52:0/881934811 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b11b7700 1 -- 172.21.15.52:0/881934811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88ac19c100 con 0x7f88ac106b90 2024-06-02T23:00:04.360 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b11b7700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac19bbc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006ba0 tx=0x7f88a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a8003af0 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f88a801a8e0 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.489+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88ac19c390 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.490+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f88a8024900 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.490+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88ac19f080 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.490+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f88a8022020 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.491+0000 7f88a27fc700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 0x7f889803b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.491+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f88a804d6f0 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.491+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88ac063af0 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.491+0000 7f88b09b6700 1 -- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 msgr2=0x7f889803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.491+0000 7f88b09b6700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 0x7f889803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.496+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f88a800b3e0 con 0x7f88ac106b90 2024-06-02T23:00:04.361 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.692+0000 7f88b09b6700 1 -- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 msgr2=0x7f889803b510 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.692+0000 7f88b09b6700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 0x7f889803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.760+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f88ac02de30 con 0x7f88ac106b90 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.760+0000 7f88a27fc700 1 -- 172.21.15.52:0/881934811 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f88a801ab00 con 0x7f88ac106b90 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.762+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 msgr2=0x7f889803b510 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.762+0000 7f88b341b700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 0x7f889803b510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.762+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 msgr2=0x7f88ac19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.762+0000 7f88b341b700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac19bbc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88a8006ba0 tx=0x7f88a8011ad0 comp rx=0 tx=0).stop 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.763+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 shutdown_connections 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.763+0000 7f88b341b700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f8898039050 0x7f889803b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.763+0000 7f88b341b700 1 --2- 172.21.15.52:0/881934811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f88ac106b90 0x7f88ac19bbc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.763+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 >> 172.21.15.52:0/881934811 conn(0x7f88ac102150 msgr2=0x7f88ac102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.763+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 shutdown_connections 2024-06-02T23:00:04.362 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:03.763+0000 7f88b341b700 1 -- 172.21.15.52:0/881934811 wait complete. 2024-06-02T23:00:04.363 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for the mgr to restart... 2024-06-02T23:00:04.363 INFO:teuthology.orchestra.run.smithi052.stdout:Waiting for mgr epoch 9... 2024-06-02T23:00:04.786 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:04 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/881934811' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-06-02T23:00:04.787 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:04 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:04.645+0000 7f28c4d91200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T23:00:05.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:04 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:04.827+0000 7f28c4d91200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T23:00:05.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:05.004+0000 7f28c4d91200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T23:00:05.613 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:05.305+0000 7f28c4d91200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T23:00:05.613 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:05.402+0000 7f28c4d91200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T23:00:05.613 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:05.613+0000 7f28c4d91200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T23:00:05.899 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:05.814+0000 7f28c4d91200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T23:00:06.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:06 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:06.445+0000 7f28c4d91200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T23:00:07.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:06 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:06.886+0000 7f28c4d91200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T23:00:07.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:06 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:06.988+0000 7f28c4d91200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T23:00:07.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:07.201+0000 7f28c4d91200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T23:00:08.118 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:07.767+0000 7f28c4d91200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T23:00:08.119 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:07.863+0000 7f28c4d91200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T23:00:08.396 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:08.118+0000 7f28c4d91200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T23:00:08.396 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:08.262+0000 7f28c4d91200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T23:00:08.396 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:08.364+0000 7f28c4d91200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T23:00:08.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:08.490+0000 7f28c4d91200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T23:00:09.149 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:08.806+0000 7f28c4d91200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T23:00:09.649 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:09.442+0000 7f28c4d91200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:00:09.669+0000 7f28c4d91200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: Active manager daemon x restarted 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: Activating manager daemon x 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: osdmap e3: 0 total, 0 up, 0 in 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: mgrmap e10: x(active, starting, since 0.00358836s) 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-06-02T23:00:10.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-06-02T23:00:10.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-06-02T23:00:10.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: Manager daemon x is now available 2024-06-02T23:00:10.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:10.880 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-06-02T23:00:10.880 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-06-02T23:00:10.880 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:10 smithi052 ceph-mon[32340]: mgrmap e11: x(active, since 1.00604s) 2024-06-02T23:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout { 2024-06-02T23:00:11.478 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout "initialized": true 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout } 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.799+0000 7f3e29299700 1 Processor -- start 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.799+0000 7f3e29299700 1 -- start start 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.799+0000 7f3e29299700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e24106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.799+0000 7f3e29299700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e241075b0 con 0x7f3e24106bc0 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.799+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e24106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.799+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e24106fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43312/0 (socket says 172.21.15.52:43312) 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.800+0000 7f3e22ffd700 1 -- 172.21.15.52:0/2950058199 learned_addr learned my addr 172.21.15.52:0/2950058199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.800+0000 7f3e22ffd700 1 -- 172.21.15.52:0/2950058199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e24107dc0 con 0x7f3e24106bc0 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.800+0000 7f3e22ffd700 1 --2- 172.21.15.52:0/2950058199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e24106fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e140098a0 tx=0x7f3e14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=479d23241c68e476 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:11.479 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.800+0000 7f3e21ffb700 1 -- 172.21.15.52:0/2950058199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14004160 con 0x7f3e24106bc0 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.800+0000 7f3e21ffb700 1 -- 172.21.15.52:0/2950058199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e14004300 con 0x7f3e24106bc0 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.800+0000 7f3e21ffb700 1 -- 172.21.15.52:0/2950058199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14011a50 con 0x7f3e24106bc0 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 -- 172.21.15.52:0/2950058199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 msgr2=0x7f3e24106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 --2- 172.21.15.52:0/2950058199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e24106fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e140098a0 tx=0x7f3e14009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 -- 172.21.15.52:0/2950058199 shutdown_connections 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 --2- 172.21.15.52:0/2950058199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e24106fe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 -- 172.21.15.52:0/2950058199 >> 172.21.15.52:0/2950058199 conn(0x7f3e24102180 msgr2=0x7f3e241045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 -- 172.21.15.52:0/2950058199 shutdown_connections 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.801+0000 7f3e29299700 1 -- 172.21.15.52:0/2950058199 wait complete. 2024-06-02T23:00:11.480 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e29299700 1 Processor -- start 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e29299700 1 -- start start 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e29299700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e2419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e29299700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3e2410b170 con 0x7f3e24106bc0 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e2419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e22ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e2419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43316/0 (socket says 172.21.15.52:43316) 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.802+0000 7f3e22ffd700 1 -- 172.21.15.52:0/3469076482 learned_addr learned my addr 172.21.15.52:0/3469076482 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e22ffd700 1 -- 172.21.15.52:0/3469076482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e2419b030 con 0x7f3e24106bc0 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e22ffd700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e2419aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e14011a20 tx=0x7f3e14011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:11.481 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14003af0 con 0x7f3e24106bc0 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e1401a8e0 con 0x7f3e24106bc0 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e14024860 con 0x7f3e24106bc0 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e2419b2c0 con 0x7f3e24106bc0 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.803+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e2419dfb0 con 0x7f3e24106bc0 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.804+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 9) ==== 46300+0+0 (secure 0 0 0) 0x7f3e14022020 con 0x7f3e24106bc0 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.804+0000 7f3e0bfff700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:11.482 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.804+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 --> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e0c03bcd0 con 0x7f3e0c039100 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.804+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e1404d390 con 0x7f3e24106bc0 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.804+0000 7f3e227fc700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:04.804+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:05.005+0000 7f3e227fc700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:05.005+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:05.406+0000 7f3e227fc700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:05.406+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:06.207+0000 7f3e227fc700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:11.483 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:06.207+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:07.808+0000 7f3e227fc700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.52:6800/540942425 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:07.808+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:09.672+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mgrmap(e 10) ==== 46067+0+0 (secure 0 0 0) 0x7f3e1402daa0 con 0x7f3e24106bc0 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:09.672+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 msgr2=0x7f3e0c03b5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:09.672+0000 7f3e0bfff700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.674+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f3e1404f1f0 con 0x7f3e24106bc0 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.674+0000 7f3e0bfff700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e0c03cbc0 0x7f3e0c03efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:11.484 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.674+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e14007da0 con 0x7f3e0c03cbc0 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.675+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e0c03cbc0 0x7f3e0c03efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.675+0000 7f3e227fc700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e0c03cbc0 0x7f3e0c03efb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3e18003a10 tx=0x7f3e18009270 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.676+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f3e14007da0 con 0x7f3e0c03cbc0 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.682+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3e24063af0 con 0x7f3e0c03cbc0 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.682+0000 7f3e0bfff700 1 -- 172.21.15.52:0/3469076482 <== mgr.14150 v2:172.21.15.52:6800/2596667410 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3e24063af0 con 0x7f3e0c03cbc0 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e0c03cbc0 msgr2=0x7f3e0c03efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e0c03cbc0 0x7f3e0c03efb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f3e18003a10 tx=0x7f3e18009270 comp rx=0 tx=0).stop 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 msgr2=0x7f3e2419aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:11.485 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e2419aaf0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e14011a20 tx=0x7f3e14011b00 comp rx=0 tx=0).stop 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 shutdown_connections 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e0c03cbc0 0x7f3e0c03efb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3e24106bc0 0x7f3e2419aaf0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 --2- 172.21.15.52:0/3469076482 >> [v2:172.21.15.52:6800/540942425,v1:172.21.15.52:6801/540942425] conn(0x7f3e0c039100 0x7f3e0c03b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 >> 172.21.15.52:0/3469076482 conn(0x7f3e24102180 msgr2=0x7f3e24102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 shutdown_connections 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:10.683+0000 7f3e29299700 1 -- 172.21.15.52:0/3469076482 wait complete. 2024-06-02T23:00:11.486 INFO:teuthology.orchestra.run.smithi052.stdout:mgr epoch 9 is available 2024-06-02T23:00:11.487 INFO:teuthology.orchestra.run.smithi052.stdout:Generating a dashboard self-signed certificate... 2024-06-02T23:00:11.880 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: [02/Jun/2024:23:00:10] ENGINE Bus STARTING 2024-06-02T23:00:11.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-06-02T23:00:11.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-06-02T23:00:11.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: [02/Jun/2024:23:00:10] ENGINE Serving on https://172.21.15.52:7150 2024-06-02T23:00:11.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: [02/Jun/2024:23:00:10] ENGINE Serving on http://172.21.15.52:8765 2024-06-02T23:00:11.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: [02/Jun/2024:23:00:10] ENGINE Bus STARTED 2024-06-02T23:00:11.881 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:12.940 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-06-02T23:00:12.940 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f451a191700 1 Processor -- start 2024-06-02T23:00:12.940 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f451a191700 1 -- start start 2024-06-02T23:00:12.940 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f451a191700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f4514106970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:12.940 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f451a191700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f451407be10 con 0x7f4514106550 2024-06-02T23:00:12.940 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f45137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f4514106970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f45137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f4514106970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43350/0 (socket says 172.21.15.52:43350) 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.985+0000 7f45137fe700 1 -- 172.21.15.52:0/417372955 learned_addr learned my addr 172.21.15.52:0/417372955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.986+0000 7f45137fe700 1 -- 172.21.15.52:0/417372955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4514106eb0 con 0x7f4514106550 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.986+0000 7f45137fe700 1 --2- 172.21.15.52:0/417372955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f4514106970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45040098a0 tx=0x7f4504009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4afce25694ed0405 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.986+0000 7f45127fc700 1 -- 172.21.15.52:0/417372955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4504004160 con 0x7f4514106550 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.986+0000 7f45127fc700 1 -- 172.21.15.52:0/417372955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4504004300 con 0x7f4514106550 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.986+0000 7f45127fc700 1 -- 172.21.15.52:0/417372955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4504011a50 con 0x7f4514106550 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 -- 172.21.15.52:0/417372955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 msgr2=0x7f4514106970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 --2- 172.21.15.52:0/417372955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f4514106970 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45040098a0 tx=0x7f4504009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 -- 172.21.15.52:0/417372955 shutdown_connections 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 --2- 172.21.15.52:0/417372955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f4514106970 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 -- 172.21.15.52:0/417372955 >> 172.21.15.52:0/417372955 conn(0x7f4514102130 msgr2=0x7f4514104550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:12.941 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 -- 172.21.15.52:0/417372955 shutdown_connections 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 -- 172.21.15.52:0/417372955 wait complete. 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.987+0000 7f451a191700 1 Processor -- start 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f451a191700 1 -- start start 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f451a191700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f451419dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f451a191700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4514107f20 con 0x7f4514106550 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f45137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f451419dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f45137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f451419dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43366/0 (socket says 172.21.15.52:43366) 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f45137fe700 1 -- 172.21.15.52:0/773612718 learned_addr learned my addr 172.21.15.52:0/773612718 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f45137fe700 1 -- 172.21.15.52:0/773612718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f451419e270 con 0x7f4514106550 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.988+0000 7f45137fe700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f451419dd30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4504011a20 tx=0x7f4504011b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.989+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4504003b70 con 0x7f4514106550 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.989+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f451419e500 con 0x7f4514106550 2024-06-02T23:00:12.942 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.989+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f450401a8e0 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.989+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4504024990 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.989+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45141a11f0 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.990+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 11) ==== 46194+0+0 (secure 0 0 0) 0x7f4504022020 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.990+0000 7f4510ff9700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f44fc038f40 0x7f44fc03b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.990+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f450404d6c0 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.990+0000 7f4512ffd700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f44fc038f40 0x7f44fc03b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.990+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4514063af0 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.991+0000 7f4512ffd700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f44fc038f40 0x7f44fc03b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4508006fd0 tx=0x7f4508006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:11.996+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f450400b3e0 con 0x7f4514106550 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.216+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f451402de20 con 0x7f44fc038f40 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.402+0000 7f4510ff9700 1 -- 172.21.15.52:0/773612718 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f451402de20 con 0x7f44fc038f40 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.404+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f44fc038f40 msgr2=0x7f44fc03b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:12.943 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.404+0000 7f451a191700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f44fc038f40 0x7f44fc03b400 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4508006fd0 tx=0x7f4508006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.404+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 msgr2=0x7f451419dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.404+0000 7f451a191700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f451419dd30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f4504011a20 tx=0x7f4504011b00 comp rx=0 tx=0).stop 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.405+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 shutdown_connections 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.405+0000 7f451a191700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f44fc038f40 0x7f44fc03b400 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.405+0000 7f451a191700 1 --2- 172.21.15.52:0/773612718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4514106550 0x7f451419dd30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.405+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 >> 172.21.15.52:0/773612718 conn(0x7f4514102130 msgr2=0x7f4514194970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.405+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 shutdown_connections 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:12.405+0000 7f451a191700 1 -- 172.21.15.52:0/773612718 wait complete. 2024-06-02T23:00:12.944 INFO:teuthology.orchestra.run.smithi052.stdout:Creating initial admin user... 2024-06-02T23:00:13.646 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:13 smithi052 ceph-mon[32340]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:00:13.646 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:13.646 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:13.647 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:13 smithi052 ceph-mon[32340]: mgrmap e12: x(active, since 2s) 2024-06-02T23:00:13.647 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:14.568 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$HGDgtoRtaa2tPrz.qtnzX.y5WmpXQPH1HdDXN.40.HZZuk7LIN8fu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1717369213, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.413+0000 7f1592e43700 1 Processor -- start 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1592e43700 1 -- start start 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1592e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1592e43700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f158c107580 con 0x7f158c106b90 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1590bdf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1590bdf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43380/0 (socket says 172.21.15.52:43380) 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1590bdf700 1 -- 172.21.15.52:0/3590626310 learned_addr learned my addr 172.21.15.52:0/3590626310 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1590bdf700 1 -- 172.21.15.52:0/3590626310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c107d90 con 0x7f158c106b90 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.414+0000 7f1590bdf700 1 --2- 172.21.15.52:0/3590626310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c106fb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15800098a0 tx=0x7f1580009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dbf7279b0ebf5d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:14.569 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f158b7fe700 1 -- 172.21.15.52:0/3590626310 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1580004160 con 0x7f158c106b90 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f158b7fe700 1 -- 172.21.15.52:0/3590626310 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1580004300 con 0x7f158c106b90 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f158b7fe700 1 -- 172.21.15.52:0/3590626310 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1580011a50 con 0x7f158c106b90 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f1592e43700 1 -- 172.21.15.52:0/3590626310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 msgr2=0x7f158c106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f1592e43700 1 --2- 172.21.15.52:0/3590626310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c106fb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f15800098a0 tx=0x7f1580009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f1592e43700 1 -- 172.21.15.52:0/3590626310 shutdown_connections 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f1592e43700 1 --2- 172.21.15.52:0/3590626310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c106fb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f1592e43700 1 -- 172.21.15.52:0/3590626310 >> 172.21.15.52:0/3590626310 conn(0x7f158c102150 msgr2=0x7f158c104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:14.570 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.415+0000 7f1592e43700 1 -- 172.21.15.52:0/3590626310 shutdown_connections 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1592e43700 1 -- 172.21.15.52:0/3590626310 wait complete. 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1592e43700 1 Processor -- start 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1592e43700 1 -- start start 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1592e43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c19dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1592e43700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f158c10b140 con 0x7f158c106b90 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1590bdf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c19dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1590bdf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c19dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43394/0 (socket says 172.21.15.52:43394) 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.416+0000 7f1590bdf700 1 -- 172.21.15.52:0/1251332912 learned_addr learned my addr 172.21.15.52:0/1251332912 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1590bdf700 1 -- 172.21.15.52:0/1251332912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f158c19e320 con 0x7f158c106b90 2024-06-02T23:00:14.571 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1590bdf700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c19dde0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1580006ba0 tx=0x7f1580011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1580003af0 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f158001a8e0 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1580024900 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f158c19e5b0 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.417+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f158c1a12a0 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.418+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7f1580022020 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.418+0000 7f1589ffb700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f157c0390b0 0x7f157c03b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.419+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f158004d510 con 0x7f158c106b90 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.419+0000 7f158bfff700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f157c0390b0 0x7f157c03b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:14.572 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.419+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f158c063af0 con 0x7f158c106b90 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.419+0000 7f158bfff700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f157c0390b0 0x7f157c03b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1578006fd0 tx=0x7f1578006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.424+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f158000f2e0 con 0x7f158c106b90 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.646+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f158c02ddb0 con 0x7f157c0390b0 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.895+0000 7f1589ffb700 1 -- 172.21.15.52:0/1251332912 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f158c02ddb0 con 0x7f157c0390b0 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f157c0390b0 msgr2=0x7f157c03b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f157c0390b0 0x7f157c03b570 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1578006fd0 tx=0x7f1578006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 msgr2=0x7f158c19dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c19dde0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1580006ba0 tx=0x7f1580011ad0 comp rx=0 tx=0).stop 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 shutdown_connections 2024-06-02T23:00:14.573 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f157c0390b0 0x7f157c03b570 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:14.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 --2- 172.21.15.52:0/1251332912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f158c106b90 0x7f158c19dde0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:14.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.897+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 >> 172.21.15.52:0/1251332912 conn(0x7f158c102150 msgr2=0x7f158c102f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:14.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.898+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 shutdown_connections 2024-06-02T23:00:14.574 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:13.898+0000 7f1592e43700 1 -- 172.21.15.52:0/1251332912 wait complete. 2024-06-02T23:00:14.574 INFO:teuthology.orchestra.run.smithi052.stdout:Fetching dashboard port number... 2024-06-02T23:00:15.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:14 smithi052 ceph-mon[32340]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:00:15.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:16.111 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:15 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/449709630' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-06-02T23:00:16.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stdout 8443 2024-06-02T23:00:16.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.614+0000 7ffac2664700 1 Processor -- start 2024-06-02T23:00:16.335 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.614+0000 7ffac2664700 1 -- start start 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffac2664700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc07c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffac2664700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc07c910 con 0x7ffabc07dee0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc07c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc07c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53398/0 (socket says 172.21.15.52:53398) 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabbfff700 1 -- 172.21.15.52:0/3766721611 learned_addr learned my addr 172.21.15.52:0/3766721611 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabbfff700 1 -- 172.21.15.52:0/3766721611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc07ca90 con 0x7ffabc07dee0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabbfff700 1 --2- 172.21.15.52:0/3766721611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc07c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0098a0 tx=0x7ffaac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5809275221eebc6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabb7fe700 1 -- 172.21.15.52:0/3766721611 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaac004160 con 0x7ffabc07dee0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.615+0000 7ffabb7fe700 1 -- 172.21.15.52:0/3766721611 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaac004300 con 0x7ffabc07dee0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffabb7fe700 1 -- 172.21.15.52:0/3766721611 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaac011a50 con 0x7ffabc07dee0 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 -- 172.21.15.52:0/3766721611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 msgr2=0x7ffabc07c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 --2- 172.21.15.52:0/3766721611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc07c340 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0098a0 tx=0x7ffaac009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 -- 172.21.15.52:0/3766721611 shutdown_connections 2024-06-02T23:00:16.336 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 --2- 172.21.15.52:0/3766721611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc07c340 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 -- 172.21.15.52:0/3766721611 >> 172.21.15.52:0/3766721611 conn(0x7ffabc103290 msgr2=0x7ffabc1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 -- 172.21.15.52:0/3766721611 shutdown_connections 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.616+0000 7ffac2664700 1 -- 172.21.15.52:0/3766721611 wait complete. 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffac2664700 1 Processor -- start 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffac2664700 1 -- start start 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffac2664700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc1a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffac2664700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffabc109ed0 con 0x7ffabc07dee0 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc1a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffabbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc1a2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53400/0 (socket says 172.21.15.52:53400) 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffabbfff700 1 -- 172.21.15.52:0/449709630 learned_addr learned my addr 172.21.15.52:0/449709630 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:16.337 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.617+0000 7ffabbfff700 1 -- 172.21.15.52:0/449709630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffabc1a2790 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.618+0000 7ffabbfff700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc1a2250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffaac000c00 tx=0x7ffaac011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.618+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaac003aa0 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.618+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffabc1a2a20 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.618+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffaac01a8e0 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.618+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffaac0248e0 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.618+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffabc1a5710 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.619+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 12) ==== 46300+0+0 (secure 0 0 0) 0x7ffaac022020 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.619+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffabc063af0 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.619+0000 7ffab9ffb700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffaa80390a0 0x7ffaa803b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.619+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ffaac04d370 con 0x7ffabc07dee0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.619+0000 7ffab3fff700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffaa80390a0 0x7ffaa803b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.619+0000 7ffab3fff700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffaa80390a0 0x7ffaa803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4006fd0 tx=0x7ffaa4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:16.338 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.624+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ffaac02c3a0 con 0x7ffabc07dee0 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.834+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7ffabc19b7b0 con 0x7ffabc07dee0 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.834+0000 7ffab9ffb700 1 -- 172.21.15.52:0/449709630 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7ffabc19b7b0 con 0x7ffabc07dee0 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.836+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffaa80390a0 msgr2=0x7ffaa803b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.836+0000 7ffac2664700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffaa80390a0 0x7ffaa803b560 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4006fd0 tx=0x7ffaa4006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.836+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 msgr2=0x7ffabc1a2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.836+0000 7ffac2664700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc1a2250 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffaac000c00 tx=0x7ffaac011ad0 comp rx=0 tx=0).stop 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.837+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 shutdown_connections 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.837+0000 7ffac2664700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffaa80390a0 0x7ffaa803b560 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.837+0000 7ffac2664700 1 --2- 172.21.15.52:0/449709630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffabc07dee0 0x7ffabc1a2250 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.837+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 >> 172.21.15.52:0/449709630 conn(0x7ffabc103290 msgr2=0x7ffabc105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.837+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 shutdown_connections 2024-06-02T23:00:16.339 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:15.837+0000 7ffac2664700 1 -- 172.21.15.52:0/449709630 wait complete. 2024-06-02T23:00:16.344 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-06-02T23:00:16.344 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout disabled 2024-06-02T23:00:16.353 INFO:teuthology.orchestra.run.smithi052.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-06-02T23:00:16.353 INFO:teuthology.orchestra.run.smithi052.stdout:systemctl: stdout inactive 2024-06-02T23:00:16.353 INFO:teuthology.orchestra.run.smithi052.stdout:firewalld.service is not enabled 2024-06-02T23:00:16.353 INFO:teuthology.orchestra.run.smithi052.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-06-02T23:00:16.355 INFO:teuthology.orchestra.run.smithi052.stdout:Ceph Dashboard is now available at: 2024-06-02T23:00:16.355 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:16.355 INFO:teuthology.orchestra.run.smithi052.stdout: URL: https://smithi052.front.sepia.ceph.com:8443/ 2024-06-02T23:00:16.355 INFO:teuthology.orchestra.run.smithi052.stdout: User: admin 2024-06-02T23:00:16.355 INFO:teuthology.orchestra.run.smithi052.stdout: Password: 873qmdfxvn 2024-06-02T23:00:16.355 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:16.356 INFO:teuthology.orchestra.run.smithi052.stdout:Saving cluster configuration to /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config directory 2024-06-02T23:00:16.356 INFO:teuthology.orchestra.run.smithi052.stdout:Enabling autotune for osd_memory_target 2024-06-02T23:00:17.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:16 smithi052 ceph-mon[32340]: mgrmap e13: x(active, since 6s) 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.787+0000 7fa558407700 1 Processor -- start 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.787+0000 7fa558407700 1 -- start start 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa558407700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55010ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa558407700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa55007be50 con 0x7fa550108760 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5561a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55010ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5561a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55010ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53414/0 (socket says 172.21.15.52:53414) 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5561a3700 1 -- 172.21.15.52:0/1479511783 learned_addr learned my addr 172.21.15.52:0/1479511783 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5561a3700 1 -- 172.21.15.52:0/1479511783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55010b090 con 0x7fa550108760 2024-06-02T23:00:17.617 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5561a3700 1 --2- 172.21.15.52:0/1479511783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55010ab50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa5400098a0 tx=0x7fa540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbda915d25cf6b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5551a1700 1 -- 172.21.15.52:0/1479511783 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540004160 con 0x7fa550108760 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5551a1700 1 -- 172.21.15.52:0/1479511783 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa540004300 con 0x7fa550108760 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.788+0000 7fa5551a1700 1 -- 172.21.15.52:0/1479511783 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540011a50 con 0x7fa550108760 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 -- 172.21.15.52:0/1479511783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 msgr2=0x7fa55010ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 --2- 172.21.15.52:0/1479511783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55010ab50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa5400098a0 tx=0x7fa540009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 -- 172.21.15.52:0/1479511783 shutdown_connections 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 --2- 172.21.15.52:0/1479511783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55010ab50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 -- 172.21.15.52:0/1479511783 >> 172.21.15.52:0/1479511783 conn(0x7fa5501020f0 msgr2=0x7fa550104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 -- 172.21.15.52:0/1479511783 shutdown_connections 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.789+0000 7fa558407700 1 -- 172.21.15.52:0/1479511783 wait complete. 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa558407700 1 Processor -- start 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa558407700 1 -- start start 2024-06-02T23:00:17.618 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa558407700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55019bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa558407700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa55010c040 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa5561a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55019bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa5561a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55019bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53416/0 (socket says 172.21.15.52:53416) 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa5561a3700 1 -- 172.21.15.52:0/1995033496 learned_addr learned my addr 172.21.15.52:0/1995033496 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.790+0000 7fa5561a3700 1 -- 172.21.15.52:0/1995033496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa55019c0a0 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.791+0000 7fa5561a3700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55019bb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa540006b20 tx=0x7fa540011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.791+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa540003b70 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.791+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa54001a8e0 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.791+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa55019c330 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.791+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa5400249b0 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.791+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa55019f020 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.792+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa540022020 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.792+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa550063af0 con 0x7fa550108760 2024-06-02T23:00:17.619 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.792+0000 7fa54f7fe700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5380391a0 0x7fa53803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.792+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa54004d9b0 con 0x7fa550108760 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.792+0000 7fa5559a2700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5380391a0 0x7fa53803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.792+0000 7fa5559a2700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5380391a0 0x7fa53803b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa544006fd0 tx=0x7fa544006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:16.797+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa54002c3a0 con 0x7fa550108760 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.002+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fa55019f640 con 0x7fa550108760 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.003+0000 7fa54f7fe700 1 -- 172.21.15.52:0/1995033496 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fa540016260 con 0x7fa550108760 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5380391a0 msgr2=0x7fa53803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5380391a0 0x7fa53803b660 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa544006fd0 tx=0x7fa544006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 msgr2=0x7fa55019bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55019bb60 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa540006b20 tx=0x7fa540011ad0 comp rx=0 tx=0).stop 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 shutdown_connections 2024-06-02T23:00:17.620 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5380391a0 0x7fa53803b660 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:17.621 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 --2- 172.21.15.52:0/1995033496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa550108760 0x7fa55019bb60 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:17.621 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 >> 172.21.15.52:0/1995033496 conn(0x7fa5501020f0 msgr2=0x7fa550102c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:17.621 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 shutdown_connections 2024-06-02T23:00:17.621 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:17.005+0000 7fa558407700 1 -- 172.21.15.52:0/1995033496 wait complete. 2024-06-02T23:00:18.907 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.008+0000 7f8dc5ef5700 1 Processor -- start 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.008+0000 7f8dc5ef5700 1 -- start start 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dc5ef5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc01069b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dc5ef5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc007be10 con 0x7f8dc0106590 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc01069b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc01069b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53428/0 (socket says 172.21.15.52:53428) 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbf7fe700 1 -- 172.21.15.52:0/1688676827 learned_addr learned my addr 172.21.15.52:0/1688676827 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbf7fe700 1 -- 172.21.15.52:0/1688676827 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc0106ef0 con 0x7f8dc0106590 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbf7fe700 1 --2- 172.21.15.52:0/1688676827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc01069b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e4b8cfd2b1f16a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbe7fc700 1 -- 172.21.15.52:0/1688676827 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db0004160 con 0x7f8dc0106590 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbe7fc700 1 -- 172.21.15.52:0/1688676827 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8db0004300 con 0x7f8dc0106590 2024-06-02T23:00:18.908 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.009+0000 7f8dbe7fc700 1 -- 172.21.15.52:0/1688676827 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db0011a50 con 0x7f8dc0106590 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1688676827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 msgr2=0x7f8dc01069b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 --2- 172.21.15.52:0/1688676827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc01069b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8db00098a0 tx=0x7f8db0009bb0 comp rx=0 tx=0).stop 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1688676827 shutdown_connections 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 --2- 172.21.15.52:0/1688676827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc01069b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1688676827 >> 172.21.15.52:0/1688676827 conn(0x7f8dc0102150 msgr2=0x7f8dc0104570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1688676827 shutdown_connections 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.010+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1688676827 wait complete. 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dc5ef5700 1 Processor -- start 2024-06-02T23:00:18.909 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dc5ef5700 1 -- start start 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dc5ef5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc019dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dc5ef5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc0107f60 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc019dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dbf7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc019dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53434/0 (socket says 172.21.15.52:53434) 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.011+0000 7f8dbf7fe700 1 -- 172.21.15.52:0/1416016642 learned_addr learned my addr 172.21.15.52:0/1416016642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dbf7fe700 1 -- 172.21.15.52:0/1416016642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc019e2a0 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dbf7fe700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc019dd60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006b20 tx=0x7f8db0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db0003b70 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8db001a8e0 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc019e530 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8db00249b0 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.012+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc01a1220 con 0x7f8dc0106590 2024-06-02T23:00:18.910 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.013+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f8db0022020 con 0x7f8dc0106590 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.013+0000 7f8dbcff9700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8da80391a0 0x7f8da803b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.013+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8db004d9d0 con 0x7f8dc0106590 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.013+0000 7f8dbeffd700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8da80391a0 0x7f8da803b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.013+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc0063af0 con 0x7f8dc0106590 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.014+0000 7f8dbeffd700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8da80391a0 0x7f8da803b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006fd0 tx=0x7f8db4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.019+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8db000b3e0 con 0x7f8dc0106590 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.337+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f8dc002ddb0 con 0x7f8dc0106590 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.339+0000 7f8dbcff9700 1 -- 172.21.15.52:0/1416016642 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f8db00172b0 con 0x7f8dc0106590 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8da80391a0 msgr2=0x7f8da803b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8da80391a0 0x7f8da803b660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8db4006fd0 tx=0x7f8db4006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 msgr2=0x7f8dc019dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc019dd60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006b20 tx=0x7f8db0011ad0 comp rx=0 tx=0).stop 2024-06-02T23:00:18.911 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 shutdown_connections 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8da80391a0 0x7f8da803b660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 --2- 172.21.15.52:0/1416016642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8dc0106590 0x7f8dc019dd60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 >> 172.21.15.52:0/1416016642 conn(0x7f8dc0102150 msgr2=0x7f8dc0194a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 shutdown_connections 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr 2024-06-02T23:00:18.342+0000 7f8dc5ef5700 1 -- 172.21.15.52:0/1416016642 wait complete. 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout:Or, if you are only running a single cluster on this host: 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-06-02T23:00:18.912 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout:Please consider enabling telemetry to help improve Ceph: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout: ceph telemetry on 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout:For more information see: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:18.913 INFO:teuthology.orchestra.run.smithi052.stdout:Bootstrap complete. 2024-06-02T23:00:18.944 INFO:tasks.cephadm:Fetching config... 2024-06-02T23:00:18.944 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:00:18.944 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-06-02T23:00:18.961 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-06-02T23:00:18.961 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:00:18.961 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-06-02T23:00:19.017 INFO:tasks.cephadm:Fetching mon keyring... 2024-06-02T23:00:19.018 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:00:19.018 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/keyring of=/dev/stdout 2024-06-02T23:00:19.084 INFO:tasks.cephadm:Fetching pub ssh key... 2024-06-02T23:00:19.084 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:00:19.085 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-06-02T23:00:19.141 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-06-02T23:00:19.141 DEBUG:teuthology.orchestra.run.smithi052:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-02T23:00:19.216 INFO:teuthology.orchestra.run.smithi052.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:00:19.228 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-02T23:00:19.272 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:00:19.285 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-06-02T23:00:19.329 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDKa/myoIJdy8Yy6Mqo0N6eLJYop/XyEsGN6CkiZpWKJf4DQG6k+xHlunBUY6P8Fh74XxphuI3VZ53LrGt+aSoJV8wD0baGyV2RHIOSo8HpOAinEq2M+7uAd9niGpMpO40xYMHn1hXiS7jzrMF53CMmpFvnhFHTWsAwlyhE3N5O6gQVb5r2W9qctak7se2h5RoLtfs9nNfsfJJDoZTEizXNG25ZDqr6b9oF25MPSmQWn7bCsGL7+k7mqa6uFsIuL35Ac7CLknoYvIFs4bXG0YHihjkSFRVX96k6w2bz+PZNYU8Q6d4BIDrOiSwfV6mozkHkJgAdWskErQPbMHG6XLbfdXFbhz7zJ8F7+v2eGhzNaTe6+DNBQFrnt3O7L35ZjLKEXYM9cT6Af0Xu9zaAor/x4W5soaZLY/I9mE0atKLDrboke8dztePoXFjdGbsG6+G7haZiIcRxfyous5EbKG3KF7Ove9o4EoaTvrun6ciV9efi6SU2PuUWqOzUsww5on0= ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:00:19.342 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-06-02T23:00:19.467 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:19 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1416016642' entity='client.admin' 2024-06-02T23:00:19.609 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.809+0000 7fa6690ba700 1 -- 172.21.15.52:0/3782395886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 msgr2=0x7fa664103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.809+0000 7fa6690ba700 1 --2- 172.21.15.52:0/3782395886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664103980 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa654009910 tx=0x7fa654009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.809+0000 7fa6690ba700 1 -- 172.21.15.52:0/3782395886 shutdown_connections 2024-06-02T23:00:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.810+0000 7fa6690ba700 1 --2- 172.21.15.52:0/3782395886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664103980 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.810+0000 7fa6690ba700 1 -- 172.21.15.52:0/3782395886 >> 172.21.15.52:0/3782395886 conn(0x7fa6640fd4e0 msgr2=0x7fa6640ff900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.810+0000 7fa6690ba700 1 -- 172.21.15.52:0/3782395886 shutdown_connections 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.810+0000 7fa6690ba700 1 -- 172.21.15.52:0/3782395886 wait complete. 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.810+0000 7fa6690ba700 1 Processor -- start 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa6690ba700 1 -- start start 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa6690ba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664076150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa6690ba700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa664107ee0 con 0x7fa664103560 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa662d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664076150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa662d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664076150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53446/0 (socket says 172.21.15.52:53446) 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa662d9d700 1 -- 172.21.15.52:0/1763000146 learned_addr learned my addr 172.21.15.52:0/1763000146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa662d9d700 1 -- 172.21.15.52:0/1763000146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa664076690 con 0x7fa664103560 2024-06-02T23:00:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa662d9d700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664076150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa654005190 tx=0x7fa654004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa654027070 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.811+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa664074880 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.812+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa654022470 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.812+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa65400f5f0 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.812+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa664074dc0 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.813+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fa65402a070 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.813+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa664108660 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.813+0000 7fa64bfff700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa64c0390f0 0x7fa64c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.813+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa65401e030 con 0x7fa664103560 2024-06-02T23:00:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.813+0000 7fa66259c700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa64c0390f0 0x7fa64c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:21.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.813+0000 7fa66259c700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa64c0390f0 0x7fa64c03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa658006fd0 tx=0x7fa658006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:21.818+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa654013070 con 0x7fa664103560 2024-06-02T23:00:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.028+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fa664190850 con 0x7fa664103560 2024-06-02T23:00:22.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.030+0000 7fa64bfff700 1 -- 172.21.15.52:0/1763000146 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fa6540310c0 con 0x7fa664103560 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa64c0390f0 msgr2=0x7fa64c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa64c0390f0 0x7fa64c03b5b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa658006fd0 tx=0x7fa658006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 msgr2=0x7fa664076150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664076150 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa654005190 tx=0x7fa654004f80 comp rx=0 tx=0).stop 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 shutdown_connections 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa64c0390f0 0x7fa64c03b5b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 --2- 172.21.15.52:0/1763000146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa664103560 0x7fa664076150 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.032+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 >> 172.21.15.52:0/1763000146 conn(0x7fa6640fd4e0 msgr2=0x7fa6640fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.033+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 shutdown_connections 2024-06-02T23:00:22.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:22.033+0000 7fa6690ba700 1 -- 172.21.15.52:0/1763000146 wait complete. 2024-06-02T23:00:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi052", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:00:22.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:22.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:22.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:22.760 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-06-02T23:00:22.760 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-06-02T23:00:23.006 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:23.356 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:23 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1763000146' entity='client.admin' 2024-06-02T23:00:23.357 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:23 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:23.357 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:23 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:24.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.467+0000 7f3a4b397700 1 -- 172.21.15.52:0/1364474707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 msgr2=0x7f3a44101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.467+0000 7f3a4b397700 1 --2- 172.21.15.52:0/1364474707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a44101230 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a40009910 tx=0x7f3a40009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.467+0000 7f3a4b397700 1 -- 172.21.15.52:0/1364474707 shutdown_connections 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.467+0000 7f3a4b397700 1 --2- 172.21.15.52:0/1364474707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a44101230 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.467+0000 7f3a4b397700 1 -- 172.21.15.52:0/1364474707 >> 172.21.15.52:0/1364474707 conn(0x7f3a440fc3d0 msgr2=0x7f3a440fe7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.467+0000 7f3a4b397700 1 -- 172.21.15.52:0/1364474707 shutdown_connections 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a4b397700 1 -- 172.21.15.52:0/1364474707 wait complete. 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a4b397700 1 Processor -- start 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a4b397700 1 -- start start 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a4b397700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a441975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a4b397700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a44106cd0 con 0x7f3a44100e10 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a49133700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a441975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:24.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a49133700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a441975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53542/0 (socket says 172.21.15.52:53542) 2024-06-02T23:00:24.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.468+0000 7f3a49133700 1 -- 172.21.15.52:0/2835690534 learned_addr learned my addr 172.21.15.52:0/2835690534 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:24.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a49133700 1 -- 172.21.15.52:0/2835690534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a44197b10 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a49133700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a441975d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005190 tx=0x7f3a40004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a40026070 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a4000bc00 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a40003960 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a44197da0 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.469+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4419aa90 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.470+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f3a40029070 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.470+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a44107450 con 0x7f3a44100e10 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.470+0000 7f3a3a7fc700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a30039150 0x7f3a3003b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:24.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.470+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a4001e070 con 0x7f3a44100e10 2024-06-02T23:00:24.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.470+0000 7f3a48932700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a30039150 0x7f3a3003b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:24.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.471+0000 7f3a48932700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a30039150 0x7f3a3003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a34006fd0 tx=0x7f3a34006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:24.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.475+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a40013070 con 0x7f3a44100e10 2024-06-02T23:00:24.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.686+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f3a4402ddb0 con 0x7f3a30039150 2024-06-02T23:00:24.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.689+0000 7f3a3a7fc700 1 -- 172.21.15.52:0/2835690534 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3a4402ddb0 con 0x7f3a30039150 2024-06-02T23:00:24.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a30039150 msgr2=0x7f3a3003b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a30039150 0x7f3a3003b610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a34006fd0 tx=0x7f3a34006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 msgr2=0x7f3a441975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a441975d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a40005190 tx=0x7f3a40004f80 comp rx=0 tx=0).stop 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 shutdown_connections 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a30039150 0x7f3a3003b610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 --2- 172.21.15.52:0/2835690534 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a44100e10 0x7f3a441975d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 >> 172.21.15.52:0/2835690534 conn(0x7f3a440fc3d0 msgr2=0x7f3a4406a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 shutdown_connections 2024-06-02T23:00:24.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:24.691+0000 7f3a4b397700 1 -- 172.21.15.52:0/2835690534 wait complete. 2024-06-02T23:00:25.418 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-06-02T23:00:25.419 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T23:00:25.419 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-06-02T23:00:25.437 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T23:00:25.438 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:00:25.496 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-06-02T23:00:25.496 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch host add smithi123 2024-06-02T23:00:25.747 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:25.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:00:25.989 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: Updating smithi052:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: Updating smithi052:/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:00:25.990 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:25 smithi052 ceph-mon[32340]: Updating smithi052:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T23:00:26.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:26 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:26.979 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:26 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:26.979 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:26 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:27.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 -- 172.21.15.52:0/3561964675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 msgr2=0x7f5100101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:27.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 --2- 172.21.15.52:0/3561964675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f5100101420 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f50f0009910 tx=0x7f50f0009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:27.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 -- 172.21.15.52:0/3561964675 shutdown_connections 2024-06-02T23:00:27.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 --2- 172.21.15.52:0/3561964675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f5100101420 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:27.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 -- 172.21.15.52:0/3561964675 >> 172.21.15.52:0/3561964675 conn(0x7f51000768f0 msgr2=0x7f5100076d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:27.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 -- 172.21.15.52:0/3561964675 shutdown_connections 2024-06-02T23:00:27.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.205+0000 7f5107165700 1 -- 172.21.15.52:0/3561964675 wait complete. 2024-06-02T23:00:27.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.206+0000 7f5107165700 1 Processor -- start 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.206+0000 7f5107165700 1 -- start start 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.206+0000 7f5107165700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f510019c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.206+0000 7f5107165700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f510010b270 con 0x7f5100101040 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5104f01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f510019c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5104f01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f510019c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53552/0 (socket says 172.21.15.52:53552) 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5104f01700 1 -- 172.21.15.52:0/1216811456 learned_addr learned my addr 172.21.15.52:0/1216811456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5104f01700 1 -- 172.21.15.52:0/1216811456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510019c980 con 0x7f5100101040 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5104f01700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f510019c440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50f00056d0 tx=0x7f50f00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50f0026070 con 0x7f5100101040 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f000bba0 con 0x7f5100101040 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510019cc70 con 0x7f5100101040 2024-06-02T23:00:27.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f50f00037f0 con 0x7f5100101040 2024-06-02T23:00:27.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.207+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5100196600 con 0x7f5100101040 2024-06-02T23:00:27.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.208+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f50f0029070 con 0x7f5100101040 2024-06-02T23:00:27.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.208+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51001024d0 con 0x7f5100101040 2024-06-02T23:00:27.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.208+0000 7f50fdffb700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e8039150 0x7f50e803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:27.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.208+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f50f001e070 con 0x7f5100101040 2024-06-02T23:00:27.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.209+0000 7f50fffff700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e8039150 0x7f50e803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:27.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.209+0000 7f50fffff700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e8039150 0x7f50e803b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:27.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.214+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f50f0013070 con 0x7f5100101040 2024-06-02T23:00:27.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:27.422+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}) -- 0x7f5100197b30 con 0x7f50e8039150 2024-06-02T23:00:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:27 smithi052 ceph-mon[32340]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:00:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:28 smithi052 ceph-mon[32340]: Deploying cephadm binary to smithi123 2024-06-02T23:00:31.021 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:30 smithi052 ceph-mon[32340]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.646+0000 7f50fdffb700 1 -- 172.21.15.52:0/1216811456 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f5100197b30 con 0x7f50e8039150 2024-06-02T23:00:31.646 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-06-02T23:00:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e8039150 msgr2=0x7f50e803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e8039150 0x7f50e803b610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f4006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 msgr2=0x7f510019c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f510019c440 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50f00056d0 tx=0x7f50f00057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 shutdown_connections 2024-06-02T23:00:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e8039150 0x7f50e803b610 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 --2- 172.21.15.52:0/1216811456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5100101040 0x7f510019c440 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 >> 172.21.15.52:0/1216811456 conn(0x7f51000768f0 msgr2=0x7f51001029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 shutdown_connections 2024-06-02T23:00:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:31.649+0000 7f5107165700 1 -- 172.21.15.52:0/1216811456 wait complete. 2024-06-02T23:00:32.248 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-02T23:00:32.497 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:32.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:32.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:32 smithi052 ceph-mon[32340]: Added host smithi123 2024-06-02T23:00:32.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:32 smithi052 ceph-mon[32340]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 -- 172.21.15.52:0/1635889829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 msgr2=0x7f558c1024e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 --2- 172.21.15.52:0/1635889829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c1024e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f557c009910 tx=0x7f557c009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 -- 172.21.15.52:0/1635889829 shutdown_connections 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 --2- 172.21.15.52:0/1635889829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c1024e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 -- 172.21.15.52:0/1635889829 >> 172.21.15.52:0/1635889829 conn(0x7f558c0fd9d0 msgr2=0x7f558c0ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 -- 172.21.15.52:0/1635889829 shutdown_connections 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.874+0000 7f5591a8e700 1 -- 172.21.15.52:0/1635889829 wait complete. 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f5591a8e700 1 Processor -- start 2024-06-02T23:00:33.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f5591a8e700 1 -- start start 2024-06-02T23:00:33.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f5591a8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:33.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f5591a8e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f558c10c380 con 0x7f558c102100 2024-06-02T23:00:33.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f558b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:33.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f558b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54132/0 (socket says 172.21.15.52:54132) 2024-06-02T23:00:33.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f558b7fe700 1 -- 172.21.15.52:0/2111068725 learned_addr learned my addr 172.21.15.52:0/2111068725 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f558b7fe700 1 -- 172.21.15.52:0/2111068725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c076680 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.875+0000 7f558b7fe700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c074790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f557c0056d0 tx=0x7f557c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.876+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557c026070 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.876+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f557c00bc00 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.876+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c074d30 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.876+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f557c0039b0 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.876+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c0751f0 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.877+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f557c029070 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.877+0000 7f55897fa700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5578039150 0x7f557803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.877+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c1035e0 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.877+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f557c01e070 con 0x7f558c102100 2024-06-02T23:00:33.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.877+0000 7f5582ffd700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5578039150 0x7f557803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:33.879 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:33.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.877+0000 7f5582ffd700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5578039150 0x7f557803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5574006fd0 tx=0x7f5574006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:33.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:33.882+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f557c030070 con 0x7f558c102100 2024-06-02T23:00:34.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.094+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f558c075d30 con 0x7f5578039150 2024-06-02T23:00:34.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.095+0000 7f55897fa700 1 -- 172.21.15.52:0/2111068725 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f558c075d30 con 0x7f5578039150 2024-06-02T23:00:34.096 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:34.096 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-06-02T23:00:34.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5578039150 msgr2=0x7f557803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:34.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5578039150 0x7f557803b610 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5574006fd0 tx=0x7f5574006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:34.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 msgr2=0x7f558c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:34.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c074790 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f557c0056d0 tx=0x7f557c0057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:34.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 shutdown_connections 2024-06-02T23:00:34.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5578039150 0x7f557803b610 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:34.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 --2- 172.21.15.52:0/2111068725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f558c102100 0x7f558c074790 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:34.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 >> 172.21.15.52:0/2111068725 conn(0x7f558c0fd9d0 msgr2=0x7f558c103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:34.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 shutdown_connections 2024-06-02T23:00:34.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:34.098+0000 7f5591a8e700 1 -- 172.21.15.52:0/2111068725 wait complete. 2024-06-02T23:00:34.658 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-06-02T23:00:34.659 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T23:00:34.659 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-06-02T23:00:34.678 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T23:00:34.678 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:00:34.736 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-06-02T23:00:34.737 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch host add smithi174 2024-06-02T23:00:34.984 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:35 smithi052 ceph-mon[32340]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:35 smithi052 ceph-mon[32340]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:00:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 -- 172.21.15.52:0/2197980847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 msgr2=0x7f10c01032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 --2- 172.21.15.52:0/2197980847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c01032b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f10b0009910 tx=0x7f10b0009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 -- 172.21.15.52:0/2197980847 shutdown_connections 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 --2- 172.21.15.52:0/2197980847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c01032b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 -- 172.21.15.52:0/2197980847 >> 172.21.15.52:0/2197980847 conn(0x7f10c00fc000 msgr2=0x7f10c00fe420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 -- 172.21.15.52:0/2197980847 shutdown_connections 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 -- 172.21.15.52:0/2197980847 wait complete. 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.463+0000 7f10c707c700 1 Processor -- start 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c707c700 1 -- start start 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c707c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c019d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:36.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c707c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10c010b540 con 0x7f10c0102ed0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c4e18700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c019d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c4e18700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c019d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54158/0 (socket says 172.21.15.52:54158) 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c4e18700 1 -- 172.21.15.52:0/1212762170 learned_addr learned my addr 172.21.15.52:0/1212762170 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c4e18700 1 -- 172.21.15.52:0/1212762170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10c019dcc0 con 0x7f10c0102ed0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c4e18700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c019d780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10b00056d0 tx=0x7f10b00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b0026070 con 0x7f10c0102ed0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10b000bc00 con 0x7f10c0102ed0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10c019dfb0 con 0x7f10c0102ed0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.464+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10b00039b0 con 0x7f10c0102ed0 2024-06-02T23:00:36.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.465+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10c0197940 con 0x7f10c0102ed0 2024-06-02T23:00:36.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.465+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f10b0029070 con 0x7f10c0102ed0 2024-06-02T23:00:36.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.466+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10c01027a0 con 0x7f10c0102ed0 2024-06-02T23:00:36.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.466+0000 7f10bdffb700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10a8039150 0x7f10a803b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:36.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.466+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f10b001e070 con 0x7f10c0102ed0 2024-06-02T23:00:36.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.466+0000 7f10bffff700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10a8039150 0x7f10a803b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:36.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.466+0000 7f10bffff700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10a8039150 0x7f10a803b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006fd0 tx=0x7f10b4006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:36.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.471+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f10b0013070 con 0x7f10c0102ed0 2024-06-02T23:00:36.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:36.686+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7f10c006a160 con 0x7f10a8039150 2024-06-02T23:00:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:00:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:00:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:38 smithi052 ceph-mon[32340]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:00:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:38 smithi052 ceph-mon[32340]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-02T23:00:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:38 smithi052 ceph-mon[32340]: Deploying cephadm binary to smithi174 2024-06-02T23:00:39.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:39 smithi052 ceph-mon[32340]: Updating smithi123:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:00:39.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:39 smithi052 ceph-mon[32340]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:39.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:39 smithi052 ceph-mon[32340]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:00:41.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.010+0000 7f10bdffb700 1 -- 172.21.15.52:0/1212762170 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f10c006a160 con 0x7f10a8039150 2024-06-02T23:00:41.012 INFO:teuthology.orchestra.run.smithi052.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10a8039150 msgr2=0x7f10a803b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10a8039150 0x7f10a803b610 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f10b4006fd0 tx=0x7f10b4006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 msgr2=0x7f10c019d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c019d780 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f10b00056d0 tx=0x7f10b00057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 shutdown_connections 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10a8039150 0x7f10a803b610 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 --2- 172.21.15.52:0/1212762170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10c0102ed0 0x7f10c019d780 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.012+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 >> 172.21.15.52:0/1212762170 conn(0x7f10c00fc000 msgr2=0x7f10c01003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:41.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.013+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 shutdown_connections 2024-06-02T23:00:41.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:41.013+0000 7f10c707c700 1 -- 172.21.15.52:0/1212762170 wait complete. 2024-06-02T23:00:41.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:41.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: Updating smithi123:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T23:00:41.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:41.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:41.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:41.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:41.558 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:41.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch host ls --format=json 2024-06-02T23:00:41.836 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:42.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:42 smithi052 ceph-mon[32340]: Added host smithi174 2024-06-02T23:00:42.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:42.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:43.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 -- 172.21.15.52:0/3043119469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 msgr2=0x7f0b581013e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:43.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 --2- 172.21.15.52:0/3043119469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b581013e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009910 tx=0x7f0b48009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:43.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 -- 172.21.15.52:0/3043119469 shutdown_connections 2024-06-02T23:00:43.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 --2- 172.21.15.52:0/3043119469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b581013e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:43.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 -- 172.21.15.52:0/3043119469 >> 172.21.15.52:0/3043119469 conn(0x7f0b580fc890 msgr2=0x7f0b580fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:43.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 -- 172.21.15.52:0/3043119469 shutdown_connections 2024-06-02T23:00:43.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.197+0000 7f0b5cc97700 1 -- 172.21.15.52:0/3043119469 wait complete. 2024-06-02T23:00:43.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5cc97700 1 Processor -- start 2024-06-02T23:00:43.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5cc97700 1 -- start start 2024-06-02T23:00:43.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5cc97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b58102aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:43.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5cc97700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5810b280 con 0x7f0b58101000 2024-06-02T23:00:43.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b58102aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b58102aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54178/0 (socket says 172.21.15.52:54178) 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5659c700 1 -- 172.21.15.52:0/1111631166 learned_addr learned my addr 172.21.15.52:0/1111631166 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5659c700 1 -- 172.21.15.52:0/1111631166 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b58104990 con 0x7f0b58101000 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.198+0000 7f0b5659c700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b58102aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b480056d0 tx=0x7f0b480057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.199+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b48026070 con 0x7f0b58101000 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.199+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b4800bc00 con 0x7f0b58101000 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.199+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b58103040 con 0x7f0b58101000 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.199+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b480039b0 con 0x7f0b58101000 2024-06-02T23:00:43.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.199+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b58103500 con 0x7f0b58101000 2024-06-02T23:00:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.200+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f0b48029070 con 0x7f0b58101000 2024-06-02T23:00:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.200+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b581024e0 con 0x7f0b58101000 2024-06-02T23:00:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.200+0000 7f0b477fe700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b400391a0 0x7f0b4003b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.200+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f0b4801e070 con 0x7f0b58101000 2024-06-02T23:00:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.200+0000 7f0b55d9b700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b400391a0 0x7f0b4003b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:43.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.201+0000 7f0b55d9b700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b400391a0 0x7f0b4003b660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c006fd0 tx=0x7f0b4c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:43.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.205+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b48013070 con 0x7f0b58101000 2024-06-02T23:00:43.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.417+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0b58103fd0 con 0x7f0b400391a0 2024-06-02T23:00:43.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.418+0000 7f0b477fe700 1 -- 172.21.15.52:0/1111631166 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7f0b58103fd0 con 0x7f0b400391a0 2024-06-02T23:00:43.418 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:00:43.419 INFO:teuthology.orchestra.run.smithi052.stdout:[{"addr": "172.21.15.52", "hostname": "smithi052", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b400391a0 msgr2=0x7f0b4003b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b400391a0 0x7f0b4003b660 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c006fd0 tx=0x7f0b4c006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 msgr2=0x7f0b58102aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b58102aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0b480056d0 tx=0x7f0b480057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 shutdown_connections 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b400391a0 0x7f0b4003b660 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:43.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 --2- 172.21.15.52:0/1111631166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b58101000 0x7f0b58102aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:43.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.420+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 >> 172.21.15.52:0/1111631166 conn(0x7f0b580fc890 msgr2=0x7f0b5806a300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:43.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.421+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 shutdown_connections 2024-06-02T23:00:43.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:43.421+0000 7f0b5cc97700 1 -- 172.21.15.52:0/1111631166 wait complete. 2024-06-02T23:00:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:43 smithi052 ceph-mon[32340]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:43.991 INFO:tasks.cephadm:Setting crush tunables to default 2024-06-02T23:00:43.992 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd crush tunables default 2024-06-02T23:00:44.237 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:00:44.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:44 smithi052 ceph-mon[32340]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:00:44.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 -- 172.21.15.52:0/726858781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 msgr2=0x7f14c4103480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 --2- 172.21.15.52:0/726858781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c4103480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009910 tx=0x7f14b4009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 -- 172.21.15.52:0/726858781 shutdown_connections 2024-06-02T23:00:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 --2- 172.21.15.52:0/726858781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c4103480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 -- 172.21.15.52:0/726858781 >> 172.21.15.52:0/726858781 conn(0x7f14c40fc8f0 msgr2=0x7f14c40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:45.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 -- 172.21.15.52:0/726858781 shutdown_connections 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.595+0000 7f14ca362700 1 -- 172.21.15.52:0/726858781 wait complete. 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14ca362700 1 Processor -- start 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14ca362700 1 -- start start 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14ca362700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14ca362700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14c410c420 con 0x7f14c41030a0 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14c3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14c3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44126/0 (socket says 172.21.15.52:44126) 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14c3fff700 1 -- 172.21.15.52:0/1780950253 learned_addr learned my addr 172.21.15.52:0/1780950253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.596+0000 7f14c3fff700 1 -- 172.21.15.52:0/1780950253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c419dbb0 con 0x7f14c41030a0 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.597+0000 7f14c3fff700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c419d670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14b40056d0 tx=0x7f14b40057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:45.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.597+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b4026070 con 0x7f14c41030a0 2024-06-02T23:00:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.597+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14b4004370 con 0x7f14c41030a0 2024-06-02T23:00:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.597+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c419de40 con 0x7f14c41030a0 2024-06-02T23:00:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.597+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f14b40039b0 con 0x7f14c41030a0 2024-06-02T23:00:45.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.597+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c41977f0 con 0x7f14c41030a0 2024-06-02T23:00:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.598+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f14b4029070 con 0x7f14c41030a0 2024-06-02T23:00:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.598+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c406abe0 con 0x7f14c41030a0 2024-06-02T23:00:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.598+0000 7f14c1ffb700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14b00390f0 0x7f14b003b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.598+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f14b401e070 con 0x7f14c41030a0 2024-06-02T23:00:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.598+0000 7f14bbfff700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14b00390f0 0x7f14b003b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:45.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.598+0000 7f14bbfff700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14b00390f0 0x7f14b003b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006fd0 tx=0x7f14ac006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:45.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.603+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f14b4013070 con 0x7f14c41030a0 2024-06-02T23:00:45.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:45 smithi052 ceph-mon[32340]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:45.816+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f14c4063af0 con 0x7f14c41030a0 2024-06-02T23:00:46.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.379+0000 7f14c1ffb700 1 -- 172.21.15.52:0/1780950253 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f14b4030090 con 0x7f14c41030a0 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14b00390f0 msgr2=0x7f14b003b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14b00390f0 0x7f14b003b5b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f14ac006fd0 tx=0x7f14ac006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 msgr2=0x7f14c419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c419d670 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f14b40056d0 tx=0x7f14b40057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 shutdown_connections 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14b00390f0 0x7f14b003b5b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 --2- 172.21.15.52:0/1780950253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14c41030a0 0x7f14c419d670 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:46.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 >> 172.21.15.52:0/1780950253 conn(0x7f14c40fc8f0 msgr2=0x7f14c4103e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:46.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 shutdown_connections 2024-06-02T23:00:46.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:00:46.382+0000 7f14ca362700 1 -- 172.21.15.52:0/1780950253 wait complete. 2024-06-02T23:00:46.383 INFO:teuthology.orchestra.run.smithi052.stderr:adjusted tunables profile to default 2024-06-02T23:00:46.638 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:46 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1780950253' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-06-02T23:00:46.966 INFO:tasks.cephadm:Adding mon.a on smithi052 2024-06-02T23:00:46.966 INFO:tasks.cephadm:Adding mon.b on smithi123 2024-06-02T23:00:46.966 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-06-02T23:00:46.967 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch apply mon '3;smithi052:172.21.15.52=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c' 2024-06-02T23:00:47.199 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:00:47.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1780950253' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:47.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:00:48.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 -- 172.21.15.174:0/3088973879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 msgr2=0x7f22581013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:48.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 --2- 172.21.15.174:0/3088973879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f22581013c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2240009910 tx=0x7f2240009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:48.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 -- 172.21.15.174:0/3088973879 shutdown_connections 2024-06-02T23:00:48.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 --2- 172.21.15.174:0/3088973879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f22581013c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:48.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 -- 172.21.15.174:0/3088973879 >> 172.21.15.174:0/3088973879 conn(0x7f22580fc890 msgr2=0x7f22580fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:48.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 -- 172.21.15.174:0/3088973879 shutdown_connections 2024-06-02T23:00:48.624 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.622+0000 7f225cae0700 1 -- 172.21.15.174:0/3088973879 wait complete. 2024-06-02T23:00:48.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225cae0700 1 Processor -- start 2024-06-02T23:00:48.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225cae0700 1 -- start start 2024-06-02T23:00:48.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225cae0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f225819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:48.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225cae0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f225810b260 con 0x7f2258100fe0 2024-06-02T23:00:48.625 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f225819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f225819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.174:35356/0 (socket says 172.21.15.174:35356) 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225659c700 1 -- 172.21.15.174:0/1251179679 learned_addr learned my addr 172.21.15.174:0/1251179679 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.623+0000 7f225659c700 1 -- 172.21.15.174:0/1251179679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225819db60 con 0x7f2258100fe0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.624+0000 7f225659c700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f225819d620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22400056d0 tx=0x7f22400057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.624+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2240026070 con 0x7f2258100fe0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.624+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225819de50 con 0x7f2258100fe0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.624+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22581978c0 con 0x7f2258100fe0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.624+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f224000b6f0 con 0x7f2258100fe0 2024-06-02T23:00:48.626 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.624+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2240030060 con 0x7f2258100fe0 2024-06-02T23:00:48.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.625+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2238005330 con 0x7f2258100fe0 2024-06-02T23:00:48.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.625+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2240029070 con 0x7f2258100fe0 2024-06-02T23:00:48.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.625+0000 7f224f7fe700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f22440390d0 0x7f224403b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:48.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.625+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f224001e070 con 0x7f2258100fe0 2024-06-02T23:00:48.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.626+0000 7f2255d9b700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f22440390d0 0x7f224403b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:48.628 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.626+0000 7f2255d9b700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f22440390d0 0x7f224403b590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2248006fd0 tx=0x7f2248006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:48.631 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.631+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2240013070 con 0x7f2258100fe0 2024-06-02T23:00:48.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: Updating smithi174:/etc/ceph/ceph.conf 2024-06-02T23:00:48.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: Updating smithi174:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:00:48.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:00:48.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: Updating smithi174:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.client.admin.keyring 2024-06-02T23:00:48.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:48.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:48.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:48.837 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.837+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) -- 0x7f2238000c80 con 0x7f22440390d0 2024-06-02T23:00:48.844 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.843+0000 7f224f7fe700 1 -- 172.21.15.174:0/1251179679 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2238000c80 con 0x7f22440390d0 2024-06-02T23:00:48.844 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f22440390d0 msgr2=0x7f224403b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f22440390d0 0x7f224403b590 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2248006fd0 tx=0x7f2248006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 msgr2=0x7f225819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f225819d620 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f22400056d0 tx=0x7f22400057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 shutdown_connections 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f22440390d0 0x7f224403b590 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 --2- 172.21.15.174:0/1251179679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2258100fe0 0x7f225819d620 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 >> 172.21.15.174:0/1251179679 conn(0x7f22580fc890 msgr2=0x7f225806a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 shutdown_connections 2024-06-02T23:00:48.847 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:48.846+0000 7f225cae0700 1 -- 172.21.15.174:0/1251179679 wait complete. 2024-06-02T23:00:49.455 DEBUG:teuthology.orchestra.run.smithi123:mon.b> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.b.service 2024-06-02T23:00:49.457 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.c.service 2024-06-02T23:00:49.460 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-02T23:00:49.460 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-02T23:00:49.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:49.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:49.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:00:49.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:49.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:00:49.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:49.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:00:49.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:50.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:50 smithi052 ceph-mon[32340]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi052:172.21.15.52=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:00:50.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:50 smithi052 ceph-mon[32340]: Saving service mon spec with placement smithi052:172.21.15.52=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c;count:3 2024-06-02T23:00:50.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:50 smithi052 ceph-mon[32340]: Deploying daemon mon.c on smithi174 2024-06-02T23:00:51.404 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:00:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:51 smithi052 ceph-mon[32340]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:53.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.293+0000 7fe3d2b97700 1 -- 172.21.15.174:0/4002381237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 msgr2=0x7fe3cc1014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:53.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.293+0000 7fe3d2b97700 1 --2- 172.21.15.174:0/4002381237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc1014c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0009910 tx=0x7fe3c0009c20 comp rx=0 tx=0).stop 2024-06-02T23:00:53.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.293+0000 7fe3d2b97700 1 -- 172.21.15.174:0/4002381237 shutdown_connections 2024-06-02T23:00:53.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.293+0000 7fe3d2b97700 1 --2- 172.21.15.174:0/4002381237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc1014c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:53.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.293+0000 7fe3d2b97700 1 -- 172.21.15.174:0/4002381237 >> 172.21.15.174:0/4002381237 conn(0x7fe3cc0fc970 msgr2=0x7fe3cc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:53.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.294+0000 7fe3d2b97700 1 -- 172.21.15.174:0/4002381237 shutdown_connections 2024-06-02T23:00:53.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.294+0000 7fe3d2b97700 1 -- 172.21.15.174:0/4002381237 wait complete. 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.294+0000 7fe3d2b97700 1 Processor -- start 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.294+0000 7fe3d2b97700 1 -- start start 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.294+0000 7fe3d2b97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.294+0000 7fe3d2b97700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe3cc10b360 con 0x7fe3cc1010e0 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3d0933700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3d0933700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc19c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.174:37468/0 (socket says 172.21.15.174:37468) 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3d0933700 1 -- 172.21.15.174:0/2283799001 learned_addr learned my addr 172.21.15.174:0/2283799001 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:00:53.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3d0933700 1 -- 172.21.15.174:0/2283799001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3cc19cb70 con 0x7fe3cc1010e0 2024-06-02T23:00:53.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3d0933700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc19c630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00056d0 tx=0x7fe3c00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:53.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0026070 con 0x7fe3cc1010e0 2024-06-02T23:00:53.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3cc19ce60 con 0x7fe3cc1010e0 2024-06-02T23:00:53.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.295+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3c000bba0 con 0x7fe3cc1010e0 2024-06-02T23:00:53.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.296+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3cc196800 con 0x7fe3cc1010e0 2024-06-02T23:00:53.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.296+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3c0003860 con 0x7fe3cc1010e0 2024-06-02T23:00:53.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.296+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3cc1025c0 con 0x7fe3cc1010e0 2024-06-02T23:00:53.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.297+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fe3c0029070 con 0x7fe3cc1010e0 2024-06-02T23:00:53.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.297+0000 7fe3c9ffb700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3bc0390f0 0x7fe3bc03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:00:53.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.297+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe3c001e070 con 0x7fe3cc1010e0 2024-06-02T23:00:53.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.297+0000 7fe3cbfff700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3bc0390f0 0x7fe3bc03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:00:53.299 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.297+0000 7fe3cbfff700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3bc0390f0 0x7fe3bc03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8006fd0 tx=0x7fe3b8006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:00:53.302 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.302+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe3c0013070 con 0x7fe3cc1010e0 2024-06-02T23:00:53.393 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.392+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe3c0029bf0 con 0x7fe3cc1010e0 2024-06-02T23:00:53.600 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:53.600+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe3cc197ce0 con 0x7fe3cc1010e0 2024-06-02T23:00:53.601 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:53 smithi174 systemd[1]: Started Ceph mon.c for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:00:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:53 smithi052 ceph-mon[32340]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:58.403 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.403+0000 7fe3c9ffb700 1 -- 172.21.15.174:0/2283799001 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe3c002a3b0 con 0x7fe3cc1010e0 2024-06-02T23:00:58.403 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T23:00:58.403 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","modified":"2024-06-02T23:00:53.392910Z","created":"2024-06-02T22:59:08.623389Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-02T23:00:58.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.406+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3bc0390f0 msgr2=0x7fe3bc03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:58.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.406+0000 7fe3d2b97700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3bc0390f0 0x7fe3bc03b5b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe3b8006fd0 tx=0x7fe3b8006e40 comp rx=0 tx=0).stop 2024-06-02T23:00:58.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.406+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 msgr2=0x7fe3cc19c630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.406+0000 7fe3d2b97700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc19c630 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe3c00056d0 tx=0x7fe3c00057b0 comp rx=0 tx=0).stop 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.407+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 shutdown_connections 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.407+0000 7fe3d2b97700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3bc0390f0 0x7fe3bc03b5b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.407+0000 7fe3d2b97700 1 --2- 172.21.15.174:0/2283799001 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe3cc1010e0 0x7fe3cc19c630 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.407+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 >> 172.21.15.174:0/2283799001 conn(0x7fe3cc0fc970 msgr2=0x7fe3cc102ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.407+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 shutdown_connections 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:00:58.407+0000 7fe3d2b97700 1 -- 172.21.15.174:0/2283799001 wait complete. 2024-06-02T23:00:58.407 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: mon.a calling monitor election 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/2283799001' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: mon.c calling monitor election 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: monmap epoch 2 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: last_changed 2024-06-02T23:00:53.392910+0000 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:00:58.714 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: min_mon_release 19 (squid) 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: election_strategy: 1 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: fsmap 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: mgrmap e13: x(active, since 48s) 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: overall HEALTH_OK 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:00:58.715 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: mon.a calling monitor election 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/2283799001' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: mon.c calling monitor election 2024-06-02T23:00:58.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: monmap epoch 2 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: last_changed 2024-06-02T23:00:53.392910+0000 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: min_mon_release 19 (squid) 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: election_strategy: 1 2024-06-02T23:00:58.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: fsmap 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: mgrmap e13: x(active, since 48s) 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: overall HEALTH_OK 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:00:58.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:00:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:59 smithi174 ceph-mon[35994]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:59 smithi174 ceph-mon[35994]: Deploying daemon mon.b on smithi123 2024-06-02T23:00:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:00:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:00:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:59 smithi052 ceph-mon[32340]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:00:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:59 smithi052 ceph-mon[32340]: Deploying daemon mon.b on smithi123 2024-06-02T23:00:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:00:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:01:00.029 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-02T23:01:00.030 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-02T23:01:00.280 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:01:01.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:01 smithi174 ceph-mon[35994]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:01.779 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 -- 172.21.15.174:0/1349798032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 msgr2=0x7f2ff4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:01.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/1349798032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff4101490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4009910 tx=0x7f2fe4009c20 comp rx=0 tx=0).stop 2024-06-02T23:01:01.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 -- 172.21.15.174:0/1349798032 shutdown_connections 2024-06-02T23:01:01.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/1349798032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff4101490 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:01.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 -- 172.21.15.174:0/1349798032 >> 172.21.15.174:0/1349798032 conn(0x7f2ff40fc940 msgr2=0x7f2ff40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:01.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 -- 172.21.15.174:0/1349798032 shutdown_connections 2024-06-02T23:01:01.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 -- 172.21.15.174:0/1349798032 wait complete. 2024-06-02T23:01:01.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 Processor -- start 2024-06-02T23:01:01.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 -- start start 2024-06-02T23:01:01.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.780+0000 7f2ff9f03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff419d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:01.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff9f03700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:01.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff9f03700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff410b330 con 0x7f2ff41010b0 2024-06-02T23:01:01.783 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff9f03700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff410b1b0 con 0x7f2ff419dcf0 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:41604/0 (socket says 172.21.15.174:41604) 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 -- 172.21.15.174:0/2864635468 learned_addr learned my addr 172.21.15.174:0/2864635468 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff419d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 -- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 msgr2=0x7f2ff41979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 -- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 msgr2=0x7f2ff41979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2febfff700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff37fe700 1 -- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 msgr2=0x7f2ff41979c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff37fe700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff37fe700 1 -- 172.21.15.174:0/2864635468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff419e330 con 0x7f2ff41010b0 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff37fe700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff419d7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4014000 tx=0x7f2fe4005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:01.784 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2fe4005dc0 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff4197f00 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.781+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2fe400b750 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.782+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f2fe4003940 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.782+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff4198490 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.782+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f2fe4029070 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.782+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff4102590 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.783+0000 7f2ff17fa700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2fe0038e90 0x7f2fe003b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.783+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2fe401e070 con 0x7f2ff41010b0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.783+0000 7f2febfff700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2fe0038e90 0x7f2fe003b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:01.785 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.783+0000 7f2febfff700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2fe0038e90 0x7f2fe003b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009180 tx=0x7f2fdc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:01.788 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:01.788+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2fe4051070 con 0x7f2ff41010b0 2024-06-02T23:01:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:01 smithi052 ceph-mon[32340]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:02.066 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.067+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f2ff4198f40 con 0x7f2ff41010b0 2024-06-02T23:01:02.067 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.067+0000 7f2ff17fa700 1 -- 172.21.15.174:0/2864635468 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f2fe4013070 con 0x7f2ff41010b0 2024-06-02T23:01:02.067 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T23:01:02.067 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","modified":"2024-06-02T23:00:53.392910Z","created":"2024-06-02T22:59:08.623389Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-02T23:01:02.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2fe0038e90 msgr2=0x7f2fe003b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:02.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2fe0038e90 0x7f2fe003b350 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc009180 tx=0x7f2fdc008040 comp rx=0 tx=0).stop 2024-06-02T23:01:02.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 msgr2=0x7f2ff419d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:02.069 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff419d7b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2fe4014000 tx=0x7f2fe4005030 comp rx=0 tx=0).stop 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 shutdown_connections 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2fe0038e90 0x7f2fe003b350 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ff41010b0 0x7f2ff419d7b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 --2- 172.21.15.174:0/2864635468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ff419dcf0 0x7f2ff41979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 >> 172.21.15.174:0/2864635468 conn(0x7f2ff40fc940 msgr2=0x7f2ff4069e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 shutdown_connections 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:02.069+0000 7f2ff9f03700 1 -- 172.21.15.174:0/2864635468 wait complete. 2024-06-02T23:01:02.070 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-06-02T23:01:02.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:02 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/2864635468' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T23:01:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:02 smithi123 systemd[1]: Started Ceph mon.b for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:01:02.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:02 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/2864635468' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-06-02T23:01:03.609 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-06-02T23:01:03.609 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph mon dump -f json 2024-06-02T23:01:03.854 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: mon.c calling monitor election 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: mon.a calling monitor election 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: monmap epoch 3 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: last_changed 2024-06-02T23:01:02.480893+0000 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: min_mon_release 19 (squid) 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: election_strategy: 1 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: fsmap 2024-06-02T23:01:07.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:01:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: mgrmap e13: x(active, since 57s) 2024-06-02T23:01:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: overall HEALTH_OK 2024-06-02T23:01:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: mon.c calling monitor election 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: mon.a calling monitor election 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: monmap epoch 3 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: last_changed 2024-06-02T23:01:02.480893+0000 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: min_mon_release 19 (squid) 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: election_strategy: 1 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: fsmap 2024-06-02T23:01:07.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:01:07.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: mgrmap e13: x(active, since 57s) 2024-06-02T23:01:07.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: overall HEALTH_OK 2024-06-02T23:01:07.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:07.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:08.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 -- 172.21.15.174:0/1744020760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f0101100 msgr2=0x7ff7d8005610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:08.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/1744020760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f0101100 0x7ff7d8005610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0005fa0 tx=0x7ff7e000ffc0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 -- 172.21.15.174:0/1744020760 shutdown_connections 2024-06-02T23:01:08.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/1744020760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f0101100 0x7ff7d8005610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 -- 172.21.15.174:0/1744020760 >> 172.21.15.174:0/1744020760 conn(0x7ff7f00fc990 msgr2=0x7ff7f00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:08.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 -- 172.21.15.174:0/1744020760 shutdown_connections 2024-06-02T23:01:08.293 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.292+0000 7ff7f67e2700 1 -- 172.21.15.174:0/1744020760 wait complete. 2024-06-02T23:01:08.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 Processor -- start 2024-06-02T23:01:08.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 -- start start 2024-06-02T23:01:08.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 0x7ff7f019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:08.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 0x7ff7f01978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:08.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 0x7ff7f01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0108d90 con 0x7ff7f019dc50 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0108c10 con 0x7ff7f0101100 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.293+0000 7ff7f67e2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0108f10 con 0x7ff7f0197e10 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 0x7ff7f01978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 0x7ff7f01978d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.174:59950/0 (socket says 172.21.15.174:59950) 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 -- 172.21.15.174:0/2116107765 learned_addr learned my addr 172.21.15.174:0/2116107765 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7effff700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 0x7ff7f019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f4d7f700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 0x7ff7f01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7effff700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 msgr2=0x7ff7f019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7effff700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 msgr2=0x7ff7f019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7effff700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 0x7ff7f019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7effff700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 0x7ff7f019d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T23:01:08.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f4d7f700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 msgr2=0x7ff7f01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f4d7f700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 msgr2=0x7ff7f01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f4d7f700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 0x7ff7f01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f4d7f700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 0x7ff7f01982c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 msgr2=0x7ff7f01982c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 0x7ff7f01982c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 msgr2=0x7ff7f019d710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 0x7ff7f019d710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 -- 172.21.15.174:0/2116107765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f01a45e0 con 0x7ff7f019dc50 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ef7fe700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 0x7ff7f01978d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400b780 tx=0x7ff7e400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4004230 con 0x7ff7f019dc50 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7e40043d0 con 0x7ff7f019dc50 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f01a4810 con 0x7ff7f019dc50 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4011b80 con 0x7ff7f019dc50 2024-06-02T23:01:08.296 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.294+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f01a4d50 con 0x7ff7f019dc50 2024-06-02T23:01:08.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.295+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7ff7e401a400 con 0x7ff7f019dc50 2024-06-02T23:01:08.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.295+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7f0063af0 con 0x7ff7f019dc50 2024-06-02T23:01:08.297 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.296+0000 7ff7ed7fa700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7dc039580 0x7ff7dc03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:08.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.296+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff7e404be70 con 0x7ff7f019dc50 2024-06-02T23:01:08.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.296+0000 7ff7effff700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7dc039580 0x7ff7dc03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:08.298 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.296+0000 7ff7effff700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7dc039580 0x7ff7dc03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000f470 tx=0x7ff7e0005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:08.301 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.301+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff7e401a840 con 0x7ff7f019dc50 2024-06-02T23:01:08.585 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.585+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7ff7f002de60 con 0x7ff7f019dc50 2024-06-02T23:01:08.586 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.586+0000 7ff7ed7fa700 1 -- 172.21.15.174:0/2116107765 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7ff7e4018750 con 0x7ff7f019dc50 2024-06-02T23:01:08.586 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-06-02T23:01:08.586 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","modified":"2024-06-02T23:01:02.480893Z","created":"2024-06-02T22:59:08.623389Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:3300","nonce":0},{"type":"v1","addr":"172.21.15.52:6789","nonce":0}]},"addr":"172.21.15.52:6789/0","public_addr":"172.21.15.52:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-06-02T23:01:08.588 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.588+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7dc039580 msgr2=0x7ff7dc03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.588+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7dc039580 0x7ff7dc03ba40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000f470 tx=0x7ff7e0005c00 comp rx=0 tx=0).stop 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.588+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 msgr2=0x7ff7f01978d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.588+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 0x7ff7f01978d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400b780 tx=0x7ff7e400bb40 comp rx=0 tx=0).stop 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 shutdown_connections 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7dc039580 0x7ff7dc03ba40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7f0197e10 0x7ff7f01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7f0101100 0x7ff7f019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 --2- 172.21.15.174:0/2116107765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7f019dc50 0x7ff7f01978d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 >> 172.21.15.174:0/2116107765 conn(0x7ff7f00fc990 msgr2=0x7ff7f00fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 shutdown_connections 2024-06-02T23:01:08.589 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:08.589+0000 7ff7f67e2700 1 -- 172.21.15.174:0/2116107765 wait complete. 2024-06-02T23:01:08.590 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-06-02T23:01:08.802 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:08 smithi052 ceph-mon[32340]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:08.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:08.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:08.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:08 smithi174 ceph-mon[35994]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:08.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:08.872 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.560 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-06-02T23:01:09.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph config generate-minimal-conf 2024-06-02T23:01:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: mon.c calling monitor election 2024-06-02T23:01:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:01:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: mon.a calling monitor election 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: monmap epoch 3 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:09.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: last_changed 2024-06-02T23:01:02.480893+0000 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: min_mon_release 19 (squid) 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: election_strategy: 1 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: fsmap 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: mgrmap e13: x(active, since 57s) 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: overall HEALTH_OK 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:09.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:09.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:09.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:01:10.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: mon.b calling monitor election 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: Updating smithi174:/etc/ceph/ceph.conf 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: Updating smithi052:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: Updating smithi123:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: Updating smithi174:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: mon.b calling monitor election 2024-06-02T23:01:10.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: mon.c calling monitor election 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: mon.a calling monitor election 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: monmap epoch 3 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: last_changed 2024-06-02T23:01:02.480893+0000 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: min_mon_release 19 (squid) 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: election_strategy: 1 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:01:10.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: fsmap 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: mgrmap e13: x(active, since 59s) 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: overall HEALTH_OK 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:10.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:10.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:10.653 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: mon.b calling monitor election 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: Updating smithi174:/etc/ceph/ceph.conf 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: Updating smithi052:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: Updating smithi123:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: Updating smithi174:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: mon.b calling monitor election 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: mon.c calling monitor election 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: mon.a calling monitor election 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: monmap epoch 3 2024-06-02T23:01:10.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: last_changed 2024-06-02T23:01:02.480893+0000 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: min_mon_release 19 (squid) 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: election_strategy: 1 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: fsmap 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:01:10.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: mgrmap e13: x(active, since 59s) 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: overall HEALTH_OK 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:10.760 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: mon.b calling monitor election 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: Updating smithi052:/etc/ceph/ceph.conf 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: Updating smithi123:/etc/ceph/ceph.conf 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: Updating smithi174:/etc/ceph/ceph.conf 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: Updating smithi052:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: Updating smithi123:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: Updating smithi174:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/config/ceph.conf 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: mon.b calling monitor election 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: mon.c calling monitor election 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: mon.a calling monitor election 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: monmap epoch 3 2024-06-02T23:01:10.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: last_changed 2024-06-02T23:01:02.480893+0000 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: created 2024-06-02T22:59:08.623389+0000 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: min_mon_release 19 (squid) 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: election_strategy: 1 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: 0: [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] mon.a 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: 2: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.b 2024-06-02T23:01:10.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: fsmap 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: osdmap e4: 0 total, 0 up, 0 in 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: mgrmap e13: x(active, since 59s) 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: overall HEALTH_OK 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:10.896 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:11 smithi052 ceph-mon[32340]: Reconfiguring mon.a (unknown last config time)... 2024-06-02T23:01:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:11 smithi052 ceph-mon[32340]: Reconfiguring daemon mon.a on smithi052 2024-06-02T23:01:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:11 smithi052 ceph-mon[32340]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:11.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:11 smithi123 ceph-mon[35977]: Reconfiguring mon.a (unknown last config time)... 2024-06-02T23:01:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:11 smithi123 ceph-mon[35977]: Reconfiguring daemon mon.a on smithi052 2024-06-02T23:01:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:11 smithi123 ceph-mon[35977]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:11 smithi174 ceph-mon[35994]: Reconfiguring mon.a (unknown last config time)... 2024-06-02T23:01:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:11 smithi174 ceph-mon[35994]: Reconfiguring daemon mon.a on smithi052 2024-06-02T23:01:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:11 smithi174 ceph-mon[35994]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:01:12.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 -- 172.21.15.52:0/3170718615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef181030f0 msgr2=0x7fef18107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 --2- 172.21.15.52:0/3170718615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef181030f0 0x7fef18107110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fef0c0099c0 tx=0x7fef0c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 -- 172.21.15.52:0/3170718615 shutdown_connections 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 --2- 172.21.15.52:0/3170718615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef18107840 0x7fef1810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 --2- 172.21.15.52:0/3170718615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef18102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 --2- 172.21.15.52:0/3170718615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef181030f0 0x7fef18107110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 -- 172.21.15.52:0/3170718615 >> 172.21.15.52:0/3170718615 conn(0x7fef180795f0 msgr2=0x7fef18079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 -- 172.21.15.52:0/3170718615 shutdown_connections 2024-06-02T23:01:12.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.540+0000 7fef1d991700 1 -- 172.21.15.52:0/3170718615 wait complete. 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 Processor -- start 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 -- start start 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef1819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef181030f0 0x7fef1819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 0x7fef18198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fef18112670 con 0x7fef18107840 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef181124f0 con 0x7fef18102740 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef1d991700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef181127f0 con 0x7fef181030f0 2024-06-02T23:01:12.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef177fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 0x7fef18198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef16ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef1819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef16ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef1819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59302/0 (socket says 172.21.15.52:59302) 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef177fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 0x7fef18198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40934/0 (socket says 172.21.15.52:40934) 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef16ffd700 1 -- 172.21.15.52:0/3504740436 learned_addr learned my addr 172.21.15.52:0/3504740436 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.541+0000 7fef167fc700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef181030f0 0x7fef1819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef16ffd700 1 -- 172.21.15.52:0/3504740436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef181030f0 msgr2=0x7fef1819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef16ffd700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef181030f0 0x7fef1819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef16ffd700 1 -- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 msgr2=0x7fef18198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef16ffd700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 0x7fef18198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef16ffd700 1 -- 172.21.15.52:0/3504740436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef18199070 con 0x7fef18102740 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef177fe700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 0x7fef18198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef16ffd700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef1819e6b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fef08007e20 tx=0x7fef0800a340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:12.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08016020 con 0x7fef18102740 2024-06-02T23:01:12.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef18199360 con 0x7fef18102740 2024-06-02T23:01:12.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef0800ac90 con 0x7fef18102740 2024-06-02T23:01:12.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef181a5580 con 0x7fef18102740 2024-06-02T23:01:12.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.542+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef08004db0 con 0x7fef18102740 2024-06-02T23:01:12.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.543+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7fef08019020 con 0x7fef18102740 2024-06-02T23:01:12.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.543+0000 7fef1c98f700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef00039330 0x7fef0003b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:12.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.543+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fef0804c170 con 0x7fef18102740 2024-06-02T23:01:12.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.543+0000 7fef167fc700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef00039330 0x7fef0003b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:12.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.543+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef18069180 con 0x7fef18102740 2024-06-02T23:01:12.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.544+0000 7fef167fc700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef00039330 0x7fef0003b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef0c000c00 tx=0x7fef0c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:12.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.549+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fef0800e1c0 con 0x7fef18102740 2024-06-02T23:01:12.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.754+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fef1802dca0 con 0x7fef18102740 2024-06-02T23:01:12.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.755+0000 7fef1c98f700 1 -- 172.21.15.52:0/3504740436 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fef0801b410 con 0x7fef18102740 2024-06-02T23:01:12.754 INFO:teuthology.orchestra.run.smithi052.stdout:# minimal ceph.conf for 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:12.754 INFO:teuthology.orchestra.run.smithi052.stdout:[global] 2024-06-02T23:01:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: fsid = 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:01:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: mon_host = [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-06-02T23:01:12.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef00039330 msgr2=0x7fef0003b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef00039330 0x7fef0003b7f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fef0c000c00 tx=0x7fef0c005fb0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 msgr2=0x7fef1819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef1819e6b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fef08007e20 tx=0x7fef0800a340 comp rx=0 tx=0).stop 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 shutdown_connections 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef00039330 0x7fef0003b7f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef18107840 0x7fef18198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef18102740 0x7fef1819e6b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 --2- 172.21.15.52:0/3504740436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef181030f0 0x7fef1819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 >> 172.21.15.52:0/3504740436 conn(0x7fef180795f0 msgr2=0x7fef181006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 shutdown_connections 2024-06-02T23:01:12.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:12.757+0000 7fef1d991700 1 -- 172.21.15.52:0/3504740436 wait complete. 2024-06-02T23:01:12.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:12.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:12.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:12.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: Reconfiguring mon.b (monmap changed)... 2024-06-02T23:01:12.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:12.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:12.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:12.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: Reconfiguring daemon mon.b on smithi123 2024-06-02T23:01:12.972 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3504740436' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:13.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:13.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:13.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:13.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: Reconfiguring mon.b (monmap changed)... 2024-06-02T23:01:13.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:13.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:13.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:13.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: Reconfiguring daemon mon.b on smithi123 2024-06-02T23:01:13.102 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3504740436' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:13.389 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-06-02T23:01:13.390 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:01:13.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.conf 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: Reconfiguring mon.b (monmap changed)... 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: Reconfiguring daemon mon.b on smithi123 2024-06-02T23:01:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3504740436' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:13.420 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:01:13.420 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:01:13.487 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T23:01:13.487 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-06-02T23:01:13.517 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T23:01:13.517 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:01:13.581 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T23:01:13.582 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-06-02T23:01:13.615 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T23:01:13.616 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-06-02T23:01:13.683 INFO:tasks.cephadm:Adding mgr.x on smithi052 2024-06-02T23:01:13.683 INFO:tasks.cephadm:Adding mgr.y on smithi123 2024-06-02T23:01:13.683 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-06-02T23:01:13.683 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch apply mgr '3;smithi052=x;smithi123=y;smithi174=z' 2024-06-02T23:01:13.960 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:01:14.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:14.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: Reconfiguring mon.c (monmap changed)... 2024-06-02T23:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: Reconfiguring daemon mon.c on smithi174 2024-06-02T23:01:14.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:14 smithi174 ceph-mon[35994]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: Reconfiguring mon.c (monmap changed)... 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: Reconfiguring daemon mon.c on smithi174 2024-06-02T23:01:14.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:14 smithi052 ceph-mon[32340]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: Reconfiguring mon.c (monmap changed)... 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: Reconfiguring daemon mon.c on smithi174 2024-06-02T23:01:15.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:14 smithi123 ceph-mon[35977]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 -- 172.21.15.174:0/2184457631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954100f10 msgr2=0x7f3954101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 --2- 172.21.15.174:0/2184457631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954100f10 0x7f3954101330 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3944009910 tx=0x7f3944009c20 comp rx=0 tx=0).stop 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 -- 172.21.15.174:0/2184457631 shutdown_connections 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 --2- 172.21.15.174:0/2184457631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954100f10 0x7f3954101330 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 -- 172.21.15.174:0/2184457631 >> 172.21.15.174:0/2184457631 conn(0x7f39540fc4d0 msgr2=0x7f39540fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 -- 172.21.15.174:0/2184457631 shutdown_connections 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 -- 172.21.15.174:0/2184457631 wait complete. 2024-06-02T23:01:16.267 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.267+0000 7f39599f9700 1 Processor -- start 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 -- start start 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3954100f10 0x7f3954197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954197cc0 0x7f395419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 0x7f395419f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3954106dd0 con 0x7f3954197cc0 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3954106c50 con 0x7f3954100f10 2024-06-02T23:01:16.270 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39599f9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3954106f50 con 0x7f3954198140 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 0x7f395419f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39527fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954197cc0 0x7f395419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 0x7f395419f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:42604/0 (socket says 172.21.15.174:42604) 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 -- 172.21.15.174:0/1678248725 learned_addr learned my addr 172.21.15.174:0/1678248725 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f3952ffd700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3954100f10 0x7f3954197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 -- 172.21.15.174:0/1678248725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3954100f10 msgr2=0x7f3954197780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3954100f10 0x7f3954197780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 -- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954197cc0 msgr2=0x7f395419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954197cc0 0x7f395419cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 -- 172.21.15.174:0/1678248725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395419fb90 con 0x7f3954198140 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39527fc700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954197cc0 0x7f395419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.268+0000 7f39537fe700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 0x7f395419f2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f394c00ba50 tx=0x7f394c00bd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:16.271 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.269+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c019070 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.269+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f394c014440 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.269+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395419fe80 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.269+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394c00ce60 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.269+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39541a0360 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.270+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f394c01c070 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.270+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3954063af0 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.270+0000 7f39589f7700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f393c039580 0x7f393c03ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.270+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f394c04b820 con 0x7f3954198140 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.270+0000 7f3952ffd700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f393c039580 0x7f393c03ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:16.272 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.270+0000 7f3952ffd700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f393c039580 0x7f393c03ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3944005190 tx=0x7f394401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:16.274 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.275+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f394c012500 con 0x7f3954198140 2024-06-02T23:01:16.484 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.485+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}) -- 0x7f39540628d0 con 0x7f393c039580 2024-06-02T23:01:16.490 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.490+0000 7f39589f7700 1 -- 172.21.15.174:0/1678248725 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f39540628d0 con 0x7f393c039580 2024-06-02T23:01:16.490 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.492+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f393c039580 msgr2=0x7f393c03ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.492+0000 7f39599f9700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f393c039580 0x7f393c03ba40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3944005190 tx=0x7f394401a040 comp rx=0 tx=0).stop 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.492+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 msgr2=0x7f395419f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.492+0000 7f39599f9700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 0x7f395419f2c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f394c00ba50 tx=0x7f394c00bd60 comp rx=0 tx=0).stop 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 shutdown_connections 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f393c039580 0x7f393c03ba40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3954198140 0x7f395419f2c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3954100f10 0x7f3954197780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 --2- 172.21.15.174:0/1678248725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3954197cc0 0x7f395419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 >> 172.21.15.174:0/1678248725 conn(0x7f39540fc4d0 msgr2=0x7f395418e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:16.493 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 shutdown_connections 2024-06-02T23:01:16.494 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:01:16.493+0000 7f39599f9700 1 -- 172.21.15.174:0/1678248725 wait complete. 2024-06-02T23:01:17.079 DEBUG:teuthology.orchestra.run.smithi123:mgr.y> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.y.service 2024-06-02T23:01:17.081 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.z.service 2024-06-02T23:01:17.084 INFO:tasks.cephadm:Deploying OSDs... 2024-06-02T23:01:17.084 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:01:17.084 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/scratch_devs of=/dev/stdout 2024-06-02T23:01:17.101 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-02T23:01:17.101 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_1 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-02 23:00:20.007688784 +0000 2024-06-02T23:01:17.159 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-02 22:56:00.375291189 +0000 2024-06-02T23:01:17.160 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-02 22:56:00.375291189 +0000 2024-06-02T23:01:17.160 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-02 22:56:00.375291189 +0000 2024-06-02T23:01:17.160 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-02T23:01:17.225 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-02T23:01:17.225 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-02T23:01:17.225 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 9.808e-05 s, 5.2 MB/s 2024-06-02T23:01:17.226 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-02T23:01:17.284 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_2 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-02 23:00:20.044688333 +0000 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-02 22:56:00.702287103 +0000 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-02 22:56:00.702287103 +0000 2024-06-02T23:01:17.343 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-02 22:56:00.702287103 +0000 2024-06-02T23:01:17.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-02T23:01:17.351 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:17.351 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.351 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.351 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: Saving service mgr spec with placement smithi052=x;smithi123=y;smithi174=z;count:3 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:17.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: Saving service mgr spec with placement smithi052=x;smithi123=y;smithi174=z;count:3 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:17.408 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-02T23:01:17.409 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-02T23:01:17.409 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000107748 s, 4.8 MB/s 2024-06-02T23:01:17.410 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-02T23:01:17.467 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_3 2024-06-02T23:01:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi052=x;smithi123=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: Saving service mgr spec with placement smithi052=x;smithi123=y;smithi174=z;count:3 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:17.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:17.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T23:01:17.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:17.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:17.524 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-02 23:00:20.080687895 +0000 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-02 22:56:01.042282855 +0000 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-02 22:56:01.042282855 +0000 2024-06-02T23:01:17.525 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-02 22:56:01.042282855 +0000 2024-06-02T23:01:17.526 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-02T23:01:17.589 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-02T23:01:17.589 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-02T23:01:17.589 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.000106048 s, 4.8 MB/s 2024-06-02T23:01:17.590 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-02T23:01:17.647 DEBUG:teuthology.orchestra.run.smithi052:> stat /dev/vg_nvme/lv_4 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-02 23:00:20.116687457 +0000 2024-06-02T23:01:17.704 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-02 22:56:01.390278507 +0000 2024-06-02T23:01:17.705 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-02 22:56:01.390278507 +0000 2024-06-02T23:01:17.705 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: 2024-06-02 22:56:01.390278507 +0000 2024-06-02T23:01:17.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-02T23:01:17.768 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records in 2024-06-02T23:01:17.769 INFO:teuthology.orchestra.run.smithi052.stderr:1+0 records out 2024-06-02T23:01:17.769 INFO:teuthology.orchestra.run.smithi052.stderr:512 bytes copied, 0.0001029 s, 5.0 MB/s 2024-06-02T23:01:17.770 DEBUG:teuthology.orchestra.run.smithi052:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-02T23:01:17.827 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-06-02T23:01:17.828 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-06-02T23:01:17.847 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-02T23:01:17.847 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-06-02T23:01:17.905 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-02T23:01:17.906 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:17.906 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-06-02T23:01:17.906 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:17.906 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:17.906 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-02 23:01:01.855114371 +0000 2024-06-02T23:01:17.906 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-02 22:56:01.650716684 +0000 2024-06-02T23:01:17.907 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-02 22:56:01.650716684 +0000 2024-06-02T23:01:17.907 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-02 22:56:01.650716684 +0000 2024-06-02T23:01:17.907 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-02T23:01:17.972 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-02T23:01:17.972 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-02T23:01:17.972 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000284711 s, 1.8 MB/s 2024-06-02T23:01:17.974 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-02T23:01:18.034 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-06-02T23:01:18.094 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-02 23:01:01.856114349 +0000 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-02 22:56:01.989708712 +0000 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-02 22:56:01.989708712 +0000 2024-06-02T23:01:18.095 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-02 22:56:01.989708712 +0000 2024-06-02T23:01:18.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-02T23:01:18.170 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-02T23:01:18.170 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-02T23:01:18.170 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000203665 s, 2.5 MB/s 2024-06-02T23:01:18.171 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-02T23:01:18.230 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-06-02T23:01:18.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:18 smithi123 ceph-mon[35977]: Deploying daemon mgr.z on smithi174 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-02 23:01:01.856114349 +0000 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-02 22:56:02.330700693 +0000 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-02 22:56:02.330700693 +0000 2024-06-02T23:01:18.275 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-02 22:56:02.330700693 +0000 2024-06-02T23:01:18.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-02T23:01:18.339 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:18 smithi174 ceph-mon[35994]: Deploying daemon mgr.z on smithi174 2024-06-02T23:01:18.339 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-02T23:01:18.339 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-02T23:01:18.339 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00020437 s, 2.5 MB/s 2024-06-02T23:01:18.340 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-02T23:01:18.398 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-06-02T23:01:18.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:18 smithi052 ceph-mon[32340]: Deploying daemon mgr.z on smithi174 2024-06-02T23:01:18.454 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-02T23:01:18.454 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:18.454 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-02T23:01:18.454 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:18.454 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:18.455 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-06-02 23:01:01.857114326 +0000 2024-06-02T23:01:18.455 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-06-02 22:56:02.666692792 +0000 2024-06-02T23:01:18.455 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-06-02 22:56:02.666692792 +0000 2024-06-02T23:01:18.455 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-06-02 22:56:02.666692792 +0000 2024-06-02T23:01:18.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-02T23:01:18.518 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-06-02T23:01:18.518 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-06-02T23:01:18.518 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000215077 s, 2.4 MB/s 2024-06-02T23:01:18.519 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-02T23:01:18.576 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-06-02T23:01:18.576 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-06-02T23:01:18.594 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-06-02T23:01:18.594 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-06-02T23:01:18.643 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:18 smithi174 systemd[1]: Starting Ceph mgr.z for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:01:18.651 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-06-02T23:01:18.651 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:18.651 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-06-02T23:01:18.651 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:18.651 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:18.652 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-06-02 23:01:18.486415421 +0000 2024-06-02T23:01:18.652 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-06-02 22:55:53.580267350 +0000 2024-06-02T23:01:18.652 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-06-02 22:55:53.580267350 +0000 2024-06-02T23:01:18.652 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-06-02 22:55:53.580267350 +0000 2024-06-02T23:01:18.652 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-06-02T23:01:18.718 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-06-02T23:01:18.718 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-06-02T23:01:18.718 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000208472 s, 2.5 MB/s 2024-06-02T23:01:18.719 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-06-02T23:01:18.776 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-06-02T23:01:18.831 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-06-02T23:01:18.831 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:18.831 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-06-02T23:01:18.831 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:18.831 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:18.832 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-06-02 23:01:18.486415421 +0000 2024-06-02T23:01:18.832 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-06-02 22:55:53.914262302 +0000 2024-06-02T23:01:18.832 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-06-02 22:55:53.915262287 +0000 2024-06-02T23:01:18.832 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-06-02 22:55:53.914262302 +0000 2024-06-02T23:01:18.832 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-06-02T23:01:18.909 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-06-02T23:01:18.909 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-06-02T23:01:18.909 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000221599 s, 2.3 MB/s 2024-06-02T23:01:18.910 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-06-02T23:01:18.966 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-06-02 23:01:18.487415407 +0000 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-06-02 22:55:54.236257436 +0000 2024-06-02T23:01:19.022 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-06-02 22:55:54.236257436 +0000 2024-06-02T23:01:19.023 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-06-02 22:55:54.236257436 +0000 2024-06-02T23:01:19.023 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-06-02T23:01:19.027 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:18 smithi174 podman[37649]: 2024-06-02 23:01:18.666952396 +0000 UTC m=+0.017204979 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:01:19.027 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:18 smithi174 podman[37649]: 2024-06-02 23:01:18.78433498 +0000 UTC m=+0.134587554 container create 3a242d6759ea6c0145f3d66872fc346eecb431185a75d8a73e113c8706d0fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-06-02T23:01:19.027 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 podman[37649]: 2024-06-02 23:01:19.028432832 +0000 UTC m=+0.378685401 container init 3a242d6759ea6c0145f3d66872fc346eecb431185a75d8a73e113c8706d0fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-02T23:01:19.050 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-06-02T23:01:19.050 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-06-02T23:01:19.050 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000213409 s, 2.4 MB/s 2024-06-02T23:01:19.051 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-06-02T23:01:19.108 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-06-02T23:01:19.163 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-06-02 23:01:18.487415407 +0000 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-06-02 22:55:54.555252615 +0000 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-06-02 22:55:54.555252615 +0000 2024-06-02T23:01:19.164 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-06-02 22:55:54.555252615 +0000 2024-06-02T23:01:19.164 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-06-02T23:01:19.228 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-06-02T23:01:19.228 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-06-02T23:01:19.228 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000193928 s, 2.6 MB/s 2024-06-02T23:01:19.230 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-06-02T23:01:19.286 INFO:tasks.cephadm:Deploying osd.0 on smithi052 with /dev/vg_nvme/lv_4... 2024-06-02T23:01:19.286 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-02T23:01:19.322 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-mon[35994]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:19.323 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 podman[37649]: 2024-06-02 23:01:19.031758221 +0000 UTC m=+0.382010810 container start 3a242d6759ea6c0145f3d66872fc346eecb431185a75d8a73e113c8706d0fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-02T23:01:19.323 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-mgr[37785]: -- 172.21.15.174:0/1116620259 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556043d1a5a0 con 0x556043cef000 2024-06-02T23:01:19.323 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 bash[37649]: 3a242d6759ea6c0145f3d66872fc346eecb431185a75d8a73e113c8706d0fa8b 2024-06-02T23:01:19.323 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 systemd[1]: Started Ceph mgr.z for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:01:19.323 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:19.206+0000 7fc471afe200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-06-02T23:01:19.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:19 smithi052 ceph-mon[32340]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:19.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:19 smithi123 ceph-mon[35977]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:19.534 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:01:19.643 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:19.402+0000 7fc471afe200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T23:01:19.643 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:19.504+0000 7fc471afe200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T23:01:20.143 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:19.815+0000 7fc471afe200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T23:01:20.143 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:19.904+0000 7fc471afe200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T23:01:20.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:20.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T23:01:20.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:20.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:20.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:20 smithi052 ceph-mon[32340]: Deploying daemon mgr.y on smithi123 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:20 smithi123 ceph-mon[35977]: Deploying daemon mgr.y on smithi123 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:20.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:20 smithi174 ceph-mon[35994]: Deploying daemon mgr.y on smithi123 2024-06-02T23:01:21.194 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:21.112+0000 7fc471afe200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T23:01:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-02T23:01:21.220 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-02T23:01:21.221 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-02T23:01:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-02T23:01:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00866766 s, 1.2 GB/s 2024-06-02T23:01:21.221 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-02T23:01:21.471 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:21 smithi174 ceph-mon[35994]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:21.471 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:21.290+0000 7fc471afe200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T23:01:21.471 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:21.470+0000 7fc471afe200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T23:01:21.487 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:21 smithi123 ceph-mon[35977]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:21.532 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:21 smithi052 ceph-mon[32340]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:21.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_4 2024-06-02T23:01:21.942 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:21 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:21.941+0000 7fb2f76c8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-06-02T23:01:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:01:22.086 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:21.777+0000 7fc471afe200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T23:01:22.086 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:21.871+0000 7fc471afe200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T23:01:22.086 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:22.086+0000 7fc471afe200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T23:01:22.257 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:22.049+0000 7fb2f76c8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-06-02T23:01:22.393 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:22.290+0000 7fc471afe200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T23:01:22.738 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:22.374+0000 7fb2f76c8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-06-02T23:01:22.739 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:22.467+0000 7fb2f76c8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-06-02T23:01:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-mon[35977]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:22 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:23.143 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:22.938+0000 7fc471afe200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T23:01:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-mon[35994]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:22 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:23.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:22 smithi052 ceph-mon[32340]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:23.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:22 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:22 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:22 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:22 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:23.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:22 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:23.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 -- 172.21.15.52:0/3073659806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281013f0 msgr2=0x7f7128101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:23.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3073659806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281013f0 0x7f7128101810 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f71180099c0 tx=0x7f7118009cd0 comp rx=0 tx=0).stop 2024-06-02T23:01:23.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 -- 172.21.15.52:0/3073659806 shutdown_connections 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3073659806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7128102f90 0x7f7128109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3073659806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281013f0 0x7f7128101810 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3073659806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f71281025f0 0x7f7128102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 -- 172.21.15.52:0/3073659806 >> 172.21.15.52:0/3073659806 conn(0x7f71280fc990 msgr2=0x7f71280fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 -- 172.21.15.52:0/3073659806 shutdown_connections 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.573+0000 7f712d4a0700 1 -- 172.21.15.52:0/3073659806 wait complete. 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 Processor -- start 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 -- start start 2024-06-02T23:01:23.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71281013f0 0x7f71281986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 0x7f7128198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128102f90 0x7f712819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f712810ce60 con 0x7f71281025f0 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f712810cce0 con 0x7f71281013f0 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f712d4a0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f712810cfe0 con 0x7f7128102f90 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f711ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 0x7f7128198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f711ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 0x7f7128198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38358/0 (socket says 172.21.15.52:38358) 2024-06-02T23:01:23.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f711ffff700 1 -- 172.21.15.52:0/3691298444 learned_addr learned my addr 172.21.15.52:0/3691298444 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.574+0000 7f7126ffd700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71281013f0 0x7f71281986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f71277fe700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128102f90 0x7f712819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f711ffff700 1 -- 172.21.15.52:0/3691298444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128102f90 msgr2=0x7f712819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f711ffff700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128102f90 0x7f712819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f711ffff700 1 -- 172.21.15.52:0/3691298444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71281013f0 msgr2=0x7f71281986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f711ffff700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71281013f0 0x7f71281986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f711ffff700 1 -- 172.21.15.52:0/3691298444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71281a0330 con 0x7f71281025f0 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f7126ffd700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71281013f0 0x7f71281986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:01:23.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f711ffff700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 0x7f7128198c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f711000eb60 tx=0x7f711000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711000cbd0 con 0x7f71281025f0 2024-06-02T23:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71281a0620 con 0x7f71281025f0 2024-06-02T23:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f71100044d0 con 0x7f71281025f0 2024-06-02T23:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110010430 con 0x7f71281025f0 2024-06-02T23:01:23.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.575+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71281a0b60 con 0x7f71281025f0 2024-06-02T23:01:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.576+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 13) ==== 46343+0+0 (secure 0 0 0) 0x7f7110004830 con 0x7f71281025f0 2024-06-02T23:01:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.576+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7128069180 con 0x7f71281025f0 2024-06-02T23:01:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.576+0000 7f7124ff9700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7114039330 0x7f711403b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.576+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f711004c440 con 0x7f71281025f0 2024-06-02T23:01:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.577+0000 7f7126ffd700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7114039330 0x7f711403b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.577+0000 7f7126ffd700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7114039330 0x7f711403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7118000c00 tx=0x7f7118005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:23.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.586+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7110020390 con 0x7f71281025f0 2024-06-02T23:01:23.643 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:23.377+0000 7fc471afe200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T23:01:23.644 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:23.475+0000 7fc471afe200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T23:01:23.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:23.813+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f712806a8d0 con 0x7f7114039330 2024-06-02T23:01:24.007 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:23.729+0000 7fb2f76c8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-06-02T23:01:24.007 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:23.897+0000 7fb2f76c8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-06-02T23:01:24.105 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:23.677+0000 7fc471afe200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T23:01:24.355 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:24.067+0000 7fb2f76c8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:24.355+0000 7fb2f76c8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: Reconfiguring mgr.x (unknown last config time)... 2024-06-02T23:01:24.356 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:24.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:24.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: Reconfiguring daemon mgr.x on smithi052 2024-06-02T23:01:24.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:01:24.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:01:24.357 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.393 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:24.199+0000 7fc471afe200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T23:01:24.393 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:24.292+0000 7fc471afe200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T23:01:24.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: Reconfiguring mgr.x (unknown last config time)... 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: Reconfiguring daemon mgr.x on smithi052 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:01:24.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:01:24.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: Reconfiguring mgr.x (unknown last config time)... 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: Reconfiguring daemon mgr.x on smithi052 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:01:24.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:24.649 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:24.447+0000 7fb2f76c8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-06-02T23:01:24.649 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:24.649+0000 7fb2f76c8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-06-02T23:01:24.880 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:24.536+0000 7fc471afe200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T23:01:24.880 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:24.670+0000 7fc471afe200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T23:01:24.880 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:24.765+0000 7fc471afe200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T23:01:24.880 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:24.880+0000 7fc471afe200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T23:01:25.006 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:24.841+0000 7fb2f76c8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:25.445+0000 7fb2f76c8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.562 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:25.562 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:25.562 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.562 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.562 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.562 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:25.563 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:25.563 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:25.563 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.643 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:25.184+0000 7fc471afe200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T23:01:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:26.143 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:25.789+0000 7fc471afe200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T23:01:26.143 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:01:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z[37744]: 2024-06-02T23:01:26.006+0000 7fc471afe200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T23:01:26.165 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:25.868+0000 7fb2f76c8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-06-02T23:01:26.166 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:25.965+0000 7fb2f76c8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-06-02T23:01:26.166 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:26.165+0000 7fb2f76c8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-06-02T23:01:26.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:26.224+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 14) ==== 92060+0+0 (secure 0 0 0) 0x7f7110017b20 con 0x7f71281025f0 2024-06-02T23:01:26.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:26 smithi052 ceph-mon[32340]: Standby manager daemon z started 2024-06-02T23:01:26.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:26 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-02T23:01:26.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:26 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T23:01:26.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:26 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-02T23:01:26.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:26 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T23:01:26.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-mon[35977]: Standby manager daemon z started 2024-06-02T23:01:26.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-02T23:01:26.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T23:01:26.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-02T23:01:26.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T23:01:26.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:26 smithi174 ceph-mon[35994]: Standby manager daemon z started 2024-06-02T23:01:26.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:26 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-06-02T23:01:26.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:26 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T23:01:26.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:26 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-06-02T23:01:26.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:26 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.174:0/2276302187' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T23:01:27.007 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:26.699+0000 7fb2f76c8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-06-02T23:01:27.007 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:26.791+0000 7fb2f76c8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-06-02T23:01:27.377 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/441446936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01ccc644-7329-48bb-9a2c-f48aafdfad4f"}]: dispatch 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/441446936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01ccc644-7329-48bb-9a2c-f48aafdfad4f"}]': finished 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: osdmap e5: 1 total, 0 up, 1 in 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:27.033+0000 7fb2f76c8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:27.167+0000 7fb2f76c8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:27.262+0000 7fb2f76c8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-06-02T23:01:27.378 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:27.377+0000 7fb2f76c8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-06-02T23:01:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-02T23:01:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-02T23:01:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/441446936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01ccc644-7329-48bb-9a2c-f48aafdfad4f"}]: dispatch 2024-06-02T23:01:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/441446936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01ccc644-7329-48bb-9a2c-f48aafdfad4f"}]': finished 2024-06-02T23:01:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: osdmap e5: 1 total, 0 up, 1 in 2024-06-02T23:01:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: mgrmap e14: x(active, since 76s), standbys: z 2024-06-02T23:01:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-06-02T23:01:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/441446936' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01ccc644-7329-48bb-9a2c-f48aafdfad4f"}]: dispatch 2024-06-02T23:01:27.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/441446936' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01ccc644-7329-48bb-9a2c-f48aafdfad4f"}]': finished 2024-06-02T23:01:27.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: osdmap e5: 1 total, 0 up, 1 in 2024-06-02T23:01:27.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:28.007 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:27.672+0000 7fb2f76c8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-06-02T23:01:28.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:28 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2531324606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:01:28.507 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:28.260+0000 7fb2f76c8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-06-02T23:01:28.507 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:01:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y[37349]: 2024-06-02T23:01:28.470+0000 7fb2f76c8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-06-02T23:01:28.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:28 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2531324606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:01:28.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:28 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2531324606' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:01:29.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:29.238+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mon.0 v2:172.21.15.52:3300/0 8 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f7110005000 con 0x7f71281025f0 2024-06-02T23:01:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:29 smithi123 ceph-mon[35977]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:29 smithi123 ceph-mon[35977]: Standby manager daemon y started 2024-06-02T23:01:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:29 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-02T23:01:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:29 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T23:01:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:29 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-02T23:01:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:29 smithi123 ceph-mon[35977]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T23:01:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:29 smithi174 ceph-mon[35994]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:29 smithi174 ceph-mon[35994]: Standby manager daemon y started 2024-06-02T23:01:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:29 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-02T23:01:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:29 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T23:01:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:29 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-02T23:01:29.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:29 smithi174 ceph-mon[35994]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T23:01:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:29 smithi052 ceph-mon[32340]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:29 smithi052 ceph-mon[32340]: Standby manager daemon y started 2024-06-02T23:01:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:29 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-06-02T23:01:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:29 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-06-02T23:01:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:29 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-06-02T23:01:29.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:29 smithi052 ceph-mon[32340]: from='mgr.? 172.21.15.123:0/2511133250' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-06-02T23:01:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:30 smithi123 ceph-mon[35977]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-02T23:01:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-02T23:01:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:30 smithi174 ceph-mon[35994]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-02T23:01:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-02T23:01:30.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:30 smithi052 ceph-mon[32340]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-06-02T23:01:30.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-06-02T23:01:31.240 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:31 smithi123 ceph-mon[35977]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:31.639 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:31 smithi174 ceph-mon[35994]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:31 smithi052 ceph-mon[32340]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:33.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:33 smithi052 ceph-mon[32340]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:33.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:33 smithi123 ceph-mon[35977]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:33 smithi174 ceph-mon[35994]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:34.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-02T23:01:34.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-02T23:01:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:34.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-06-02T23:01:34.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:35 smithi123 ceph-mon[35977]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:35 smithi123 ceph-mon[35977]: Deploying daemon osd.0 on smithi052 2024-06-02T23:01:35.640 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:35 smithi052 ceph-mon[32340]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:35.640 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:35 smithi052 ceph-mon[32340]: Deploying daemon osd.0 on smithi052 2024-06-02T23:01:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:35 smithi174 ceph-mon[35994]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:35 smithi174 ceph-mon[35994]: Deploying daemon osd.0 on smithi052 2024-06-02T23:01:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:37 smithi123 ceph-mon[35977]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:37.554 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:37 smithi052 ceph-mon[32340]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:37 smithi174 ceph-mon[35994]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:39.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:39 smithi052 ceph-mon[32340]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:39 smithi123 ceph-mon[35977]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:39 smithi174 ceph-mon[35994]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:40.629 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:40.629 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:40.630 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:40.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:40.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:40.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:41.517 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:41 smithi052 ceph-mon[32340]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:41 smithi123 ceph-mon[35977]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:41 smithi174 ceph-mon[35994]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:42.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.967+0000 7f7124ff9700 1 -- 172.21.15.52:0/3691298444 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f712806a8d0 con 0x7f7114039330 2024-06-02T23:01:42.968 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 0 on host 'smithi052' 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7114039330 msgr2=0x7f711403b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7114039330 0x7f711403b7f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7118000c00 tx=0x7f7118005fb0 comp rx=0 tx=0).stop 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 msgr2=0x7f7128198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 0x7f7128198c10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f711000eb60 tx=0x7f711000ee70 comp rx=0 tx=0).stop 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 shutdown_connections 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7114039330 0x7f711403b7f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7128102f90 0x7f712819fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71281013f0 0x7f71281986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 --2- 172.21.15.52:0/3691298444 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f71281025f0 0x7f7128198c10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:42.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.969+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 >> 172.21.15.52:0/3691298444 conn(0x7f71280fc990 msgr2=0x7f712806a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:42.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.970+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 shutdown_connections 2024-06-02T23:01:42.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:01:42.970+0000 7f712d4a0700 1 -- 172.21.15.52:0/3691298444 wait complete. 2024-06-02T23:01:43.269 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:43.269 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.269 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:43.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:43.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.270 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:01:43.552 DEBUG:teuthology.orchestra.run.smithi052:osd.0> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.0.service 2024-06-02T23:01:43.554 INFO:tasks.cephadm:Deploying osd.1 on smithi123 with /dev/vg_nvme/lv_4... 2024-06-02T23:01:43.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-02T23:01:43.794 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:01:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:44 smithi052 ceph-mon[32340]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-02T23:01:44.188 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:44 smithi123 ceph-mon[35977]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-02T23:01:44.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:44 smithi174 ceph-mon[35994]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-06-02T23:01:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:45 smithi174 ceph-mon[35994]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:45 smithi174 ceph-mon[35994]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-02T23:01:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:45 smithi174 ceph-mon[35994]: osdmap e6: 1 total, 0 up, 1 in 2024-06-02T23:01:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:45 smithi174 ceph-mon[35994]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:01:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:45.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-mon[32340]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:45.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-mon[32340]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-02T23:01:45.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-mon[32340]: osdmap e6: 1 total, 0 up, 1 in 2024-06-02T23:01:45.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-mon[32340]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:01:45.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:45 smithi123 ceph-mon[35977]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:45 smithi123 ceph-mon[35977]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-06-02T23:01:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:45 smithi123 ceph-mon[35977]: osdmap e6: 1 total, 0 up, 1 in 2024-06-02T23:01:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:45 smithi123 ceph-mon[35977]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:01:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:45.666 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-02T23:01:45.666 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-02T23:01:45.666 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-02T23:01:45.666 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-02T23:01:45.666 INFO:teuthology.orchestra.run.smithi123.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0132198 s, 793 MB/s 2024-06-02T23:01:45.667 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-02T23:01:45.886 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0[44718]: 2024-06-02T23:01:45.552+0000 7f4c418b7700 -1 osd.0 0 waiting for initial osdmap 2024-06-02T23:01:45.886 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:01:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0[44718]: 2024-06-02T23:01:45.555+0000 7f4c396a2700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:01:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:46 smithi052 ceph-mon[32340]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:01:46.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:46 smithi052 ceph-mon[32340]: osdmap e7: 1 total, 0 up, 1 in 2024-06-02T23:01:46.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:46.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:46.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:46 smithi052 ceph-mon[32340]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' 2024-06-02T23:01:46.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:46 smithi123 ceph-mon[35977]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:01:46.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:46 smithi123 ceph-mon[35977]: osdmap e7: 1 total, 0 up, 1 in 2024-06-02T23:01:46.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:46.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:46.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:46 smithi123 ceph-mon[35977]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' 2024-06-02T23:01:46.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-06-02T23:01:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:46 smithi174 ceph-mon[35994]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:01:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:46 smithi174 ceph-mon[35994]: osdmap e7: 1 total, 0 up, 1 in 2024-06-02T23:01:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:46 smithi174 ceph-mon[35994]: from='osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430]' entity='osd.0' 2024-06-02T23:01:46.535 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:01:47.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:01:47.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:01:47.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:47.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:47.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi052 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 17092M 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] boot 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: osdmap e8: 1 total, 1 up, 1 in 2024-06-02T23:01:47.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:01:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:01:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi052 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 17092M 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] boot 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: osdmap e8: 1 total, 1 up, 1 in 2024-06-02T23:01:47.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:01:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:01:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-06-02T23:01:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi052 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 17092M 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: osd.0 [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] boot 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: osdmap e8: 1 total, 1 up, 1 in 2024-06-02T23:01:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-06-02T23:01:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 -- 172.21.15.123:0/2945671247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4101410 msgr2=0x7fa3b4101830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 --2- 172.21.15.123:0/2945671247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4101410 0x7fa3b4101830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40099c0 tx=0x7fa3a4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:01:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 -- 172.21.15.123:0/2945671247 shutdown_connections 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 --2- 172.21.15.123:0/2945671247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b4102fb0 0x7fa3b4109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 --2- 172.21.15.123:0/2945671247 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4101410 0x7fa3b4101830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 --2- 172.21.15.123:0/2945671247 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102610 0x7fa3b4102a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.101+0000 7fa3ba166700 1 -- 172.21.15.123:0/2945671247 >> 172.21.15.123:0/2945671247 conn(0x7fa3b40fc990 msgr2=0x7fa3b40fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 -- 172.21.15.123:0/2945671247 shutdown_connections 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 -- 172.21.15.123:0/2945671247 wait complete. 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 Processor -- start 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 -- start start 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b4101410 0x7fa3b4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 0x7fa3b4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102fb0 0x7fa3b419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b410cf60 con 0x7fa3b4102fb0 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b410cde0 con 0x7fa3b4102610 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3ba166700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3b410d0e0 con 0x7fa3b4101410 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.102+0000 7fa3abfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 0x7fa3b4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 0x7fa3b4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:54492/0 (socket says 172.21.15.123:54492) 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 -- 172.21.15.123:0/278975825 learned_addr learned my addr 172.21.15.123:0/278975825 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3b37fe700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b4101410 0x7fa3b4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3b3fff700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102fb0 0x7fa3b419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 -- 172.21.15.123:0/278975825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b4101410 msgr2=0x7fa3b4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b4101410 0x7fa3b4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 -- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102fb0 msgr2=0x7fa3b419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102fb0 0x7fa3b419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 -- 172.21.15.123:0/278975825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3b41a0290 con 0x7fa3b4102610 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3b3fff700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102fb0 0x7fa3b419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3abfff700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 0x7fa3b4198b70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c9b0 tx=0x7fa39c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c007b00 con 0x7fa3b4102610 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa39c007ca0 con 0x7fa3b4102610 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c0186f0 con 0x7fa3b4102610 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3b41a0580 con 0x7fa3b4102610 2024-06-02T23:01:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.103+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3b41a0b90 con 0x7fa3b4102610 2024-06-02T23:01:48.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.104+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3b4069180 con 0x7fa3b4102610 2024-06-02T23:01:48.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.105+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7fa39c00f450 con 0x7fa3b4102610 2024-06-02T23:01:48.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.106+0000 7fa3b17fa700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3a00a2c10 0x7fa3a00a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:01:48.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.106+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa39c0cd550 con 0x7fa3b4102610 2024-06-02T23:01:48.107 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.107+0000 7fa3b37fe700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3a00a2c10 0x7fa3a00a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:01:48.108 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.107+0000 7fa3b37fe700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3a00a2c10 0x7fa3a00a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000c00 tx=0x7fa3a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:01:48.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.111+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa39c014030 con 0x7fa3b4102610 2024-06-02T23:01:48.316 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:01:48.316+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa3b406b2d0 con 0x7fa3a00a2c10 2024-06-02T23:01:49.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:49 smithi123 ceph-mon[35977]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:49.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:49 smithi123 ceph-mon[35977]: osdmap e9: 1 total, 1 up, 1 in 2024-06-02T23:01:49.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:49 smithi123 ceph-mon[35977]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:49.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:01:49.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:01:49.209 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:49 smithi174 ceph-mon[35994]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:49 smithi174 ceph-mon[35994]: osdmap e9: 1 total, 1 up, 1 in 2024-06-02T23:01:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:49 smithi174 ceph-mon[35994]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:01:49.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:01:49.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:49 smithi052 ceph-mon[32340]: pgmap v47: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:49 smithi052 ceph-mon[32340]: osdmap e9: 1 total, 1 up, 1 in 2024-06-02T23:01:49.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:49 smithi052 ceph-mon[32340]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:01:49.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:01:49.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:01:49.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:51 smithi174 ceph-mon[35994]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:51 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/424178251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]: dispatch 2024-06-02T23:01:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:51 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]: dispatch 2024-06-02T23:01:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:51 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]': finished 2024-06-02T23:01:51.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:51 smithi174 ceph-mon[35994]: osdmap e10: 2 total, 1 up, 2 in 2024-06-02T23:01:51.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:01:51.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:51 smithi052 ceph-mon[32340]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:51.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:51 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/424178251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]: dispatch 2024-06-02T23:01:51.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:51 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]: dispatch 2024-06-02T23:01:51.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:51 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]': finished 2024-06-02T23:01:51.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:51 smithi052 ceph-mon[32340]: osdmap e10: 2 total, 1 up, 2 in 2024-06-02T23:01:51.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:01:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:51 smithi123 ceph-mon[35977]: pgmap v49: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:51 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/424178251' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]: dispatch 2024-06-02T23:01:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:51 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]: dispatch 2024-06-02T23:01:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:51 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "08319da8-7871-414b-ad99-e708bcd4bed9"}]': finished 2024-06-02T23:01:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:51 smithi123 ceph-mon[35977]: osdmap e10: 2 total, 1 up, 2 in 2024-06-02T23:01:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:01:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:52 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/3182838777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:01:52.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:52 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/3182838777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:01:52.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:52 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/3182838777' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:01:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:53 smithi174 ceph-mon[35994]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:53.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:53 smithi052 ceph-mon[32340]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:53 smithi123 ceph-mon[35977]: pgmap v51: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:55 smithi123 ceph-mon[35977]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:55 smithi174 ceph-mon[35994]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:55.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:55 smithi052 ceph-mon[32340]: pgmap v52: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:57.231 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:57 smithi123 ceph-mon[35977]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:57 smithi174 ceph-mon[35994]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:57 smithi052 ceph-mon[32340]: pgmap v53: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:58.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-02T23:01:58.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-02T23:01:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:58.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-06-02T23:01:58.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:01:59.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:59 smithi123 ceph-mon[35977]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:59.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:01:59 smithi123 ceph-mon[35977]: Deploying daemon osd.1 on smithi123 2024-06-02T23:01:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:59 smithi174 ceph-mon[35994]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:01:59 smithi174 ceph-mon[35994]: Deploying daemon osd.1 on smithi123 2024-06-02T23:01:59.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:59 smithi052 ceph-mon[32340]: pgmap v54: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:01:59.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:01:59 smithi052 ceph-mon[32340]: Deploying daemon osd.1 on smithi123 2024-06-02T23:02:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:01 smithi174 ceph-mon[35994]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:01.395 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:01 smithi123 ceph-mon[35977]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:01.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:01 smithi052 ceph-mon[32340]: pgmap v55: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:03.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:03 smithi123 ceph-mon[35977]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:03 smithi174 ceph-mon[35994]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:03.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:03 smithi052 ceph-mon[32340]: pgmap v56: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:04.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:04.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:04.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:04.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:04.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:04.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:04.706 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:04.706 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:04.706 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:05 smithi123 ceph-mon[35977]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:05 smithi174 ceph-mon[35994]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:05.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:05 smithi052 ceph-mon[32340]: pgmap v57: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:06.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:06 smithi123 ceph-mon[35977]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:06.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:06.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:06.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:06.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:06.900 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:06.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.982+0000 7fa3b17fa700 1 -- 172.21.15.123:0/278975825 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa3b406b2d0 con 0x7fa3a00a2c10 2024-06-02T23:02:06.983 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 1 on host 'smithi123' 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.984+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3a00a2c10 msgr2=0x7fa3a00a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.984+0000 7fa3ba166700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3a00a2c10 0x7fa3a00a50d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4000c00 tx=0x7fa3a4005fb0 comp rx=0 tx=0).stop 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.984+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 msgr2=0x7fa3b4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.984+0000 7fa3ba166700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 0x7fa3b4198b70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c9b0 tx=0x7fa39c00cd70 comp rx=0 tx=0).stop 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 shutdown_connections 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3a00a2c10 0x7fa3a00a50d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3b4102fb0 0x7fa3b419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3b4101410 0x7fa3b4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 --2- 172.21.15.123:0/278975825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3b4102610 0x7fa3b4198b70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:06.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 >> 172.21.15.123:0/278975825 conn(0x7fa3b40fc990 msgr2=0x7fa3b406abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:02:06.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 shutdown_connections 2024-06-02T23:02:06.986 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:02:06.985+0000 7fa3ba166700 1 -- 172.21.15.123:0/278975825 wait complete. 2024-06-02T23:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:06 smithi174 ceph-mon[35994]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:06 smithi052 ceph-mon[32340]: pgmap v58: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:07.595 DEBUG:teuthology.orchestra.run.smithi123:osd.1> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.1.service 2024-06-02T23:02:07.598 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-06-02T23:02:07.598 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-06-02T23:02:07.840 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:02:08.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:08.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:08.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:08.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:07 smithi123 ceph-mon[35977]: from='osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T23:02:08.123 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:07 smithi123 ceph-mon[35977]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T23:02:08.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:08.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:08.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:08.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:07 smithi174 ceph-mon[35994]: from='osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T23:02:08.263 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:07 smithi174 ceph-mon[35994]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T23:02:08.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:08.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:08.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:08.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:07 smithi052 ceph-mon[32340]: from='osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T23:02:08.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:07 smithi052 ceph-mon[32340]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-06-02T23:02:08.974 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:08 smithi174 ceph-mon[35994]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:08.974 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:08 smithi174 ceph-mon[35994]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-02T23:02:08.974 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:08 smithi174 ceph-mon[35994]: from='osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:02:08.974 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:08 smithi174 ceph-mon[35994]: osdmap e11: 2 total, 1 up, 2 in 2024-06-02T23:02:08.974 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:08.974 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:08 smithi174 ceph-mon[35994]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:02:09.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-mon[35977]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:09.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-mon[35977]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-02T23:02:09.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-mon[35977]: from='osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:02:09.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-mon[35977]: osdmap e11: 2 total, 1 up, 2 in 2024-06-02T23:02:09.048 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:09.049 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-mon[35977]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:02:09.049 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1[40532]: 2024-06-02T23:02:08.945+0000 7ff341415700 -1 osd.1 0 waiting for initial osdmap 2024-06-02T23:02:09.049 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:08 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1[40532]: 2024-06-02T23:02:08.949+0000 7ff33d208700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:02:09.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:08 smithi052 ceph-mon[32340]: pgmap v59: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:09.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:08 smithi052 ceph-mon[32340]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-06-02T23:02:09.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:08 smithi052 ceph-mon[32340]: from='osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:02:09.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:08 smithi052 ceph-mon[32340]: osdmap e11: 2 total, 1 up, 2 in 2024-06-02T23:02:09.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:09.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:08 smithi052 ceph-mon[32340]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:02:09.527 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-06-02T23:02:09.528 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-06-02T23:02:09.528 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-06-02T23:02:09.528 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-06-02T23:02:09.528 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114486 s, 916 MB/s 2024-06-02T23:02:09.528 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-06-02T23:02:10.116 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-06-02T23:02:10.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:09 smithi123 ceph-mon[35977]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:02:10.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:09 smithi123 ceph-mon[35977]: osdmap e12: 2 total, 1 up, 2 in 2024-06-02T23:02:10.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:10.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:10.315 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:09 smithi052 ceph-mon[32340]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:02:10.316 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:09 smithi052 ceph-mon[32340]: osdmap e12: 2 total, 1 up, 2 in 2024-06-02T23:02:10.316 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:10.316 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:10.360 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:02:10.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:09 smithi174 ceph-mon[35994]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:02:10.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:09 smithi174 ceph-mon[35994]: osdmap e12: 2 total, 1 up, 2 in 2024-06-02T23:02:10.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:10.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] boot 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: osdmap e13: 2 total, 2 up, 2 in 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:11.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi123 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 17088M 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:11.246 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] boot 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: osdmap e13: 2 total, 2 up, 2 in 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi123 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 17088M 2024-06-02T23:02:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:11.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:11.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: pgmap v62: 0 pgs: ; 0 B data, 35 MiB used, 89 GiB / 89 GiB avail 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: osd.1 [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] boot 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: osdmap e13: 2 total, 2 up, 2 in 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi123 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 17088M 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:11.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 -- 172.21.15.174:0/309892333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38101d80 msgr2=0x7f5d381021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 --2- 172.21.15.174:0/309892333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38101d80 0x7f5d381021e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009990 tx=0x7f5d2c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 -- 172.21.15.174:0/309892333 shutdown_connections 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 --2- 172.21.15.174:0/309892333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d38102720 0x7f5d38108fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 --2- 172.21.15.174:0/309892333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38100b80 0x7f5d38100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 --2- 172.21.15.174:0/309892333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38101d80 0x7f5d381021e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 -- 172.21.15.174:0/309892333 >> 172.21.15.174:0/309892333 conn(0x7f5d380fc100 msgr2=0x7f5d380fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 -- 172.21.15.174:0/309892333 shutdown_connections 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.743+0000 7f5d3e275700 1 -- 172.21.15.174:0/309892333 wait complete. 2024-06-02T23:02:11.744 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.744+0000 7f5d3e275700 1 Processor -- start 2024-06-02T23:02:11.745 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.744+0000 7f5d3e275700 1 -- start start 2024-06-02T23:02:11.746 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 0x7f5d3810a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101d80 0x7f5d3819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d38102720 0x7f5d3819fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810cf60 con 0x7f5d38102720 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810cde0 con 0x7f5d38101d80 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d3810d0e0 con 0x7f5d38100b80 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 0x7f5d3810a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 0x7f5d3810a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:52136/0 (socket says 172.21.15.174:52136) 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 -- 172.21.15.174:0/2790367980 learned_addr learned my addr 172.21.15.174:0/2790367980 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d377fe700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101d80 0x7f5d3819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3c812700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d38102720 0x7f5d3819fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 -- 172.21.15.174:0/2790367980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101d80 msgr2=0x7f5d3819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:11.747 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101d80 0x7f5d3819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 -- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d38102720 msgr2=0x7f5d3819fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d38102720 0x7f5d3819fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 -- 172.21.15.174:0/2790367980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d381a0350 con 0x7f5d38100b80 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d377fe700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101d80 0x7f5d3819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d37fff700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 0x7f5d3810a560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b500 tx=0x7f5d2800b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28015020 con 0x7f5d38100b80 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d381a0640 con 0x7f5d38100b80 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d28011ce0 con 0x7f5d38100b80 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d28010d60 con 0x7f5d38100b80 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.745+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d381a0bb0 con 0x7f5d38100b80 2024-06-02T23:02:11.748 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.746+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d38069180 con 0x7f5d38100b80 2024-06-02T23:02:11.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.748+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f5d28018020 con 0x7f5d38100b80 2024-06-02T23:02:11.749 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.748+0000 7f5d357fa700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d200a2b90 0x7f5d200a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:11.750 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.748+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f5d280cd5a0 con 0x7f5d38100b80 2024-06-02T23:02:11.750 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.748+0000 7f5d377fe700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d200a2b90 0x7f5d200a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:11.750 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.749+0000 7f5d377fe700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d200a2b90 0x7f5d200a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00b5c0 tx=0x7f5d2c005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:02:11.751 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.752+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5d2801f020 con 0x7f5d38100b80 2024-06-02T23:02:11.953 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:11.954+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5d380628d0 con 0x7f5d200a2b90 2024-06-02T23:02:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:11 smithi123 ceph-mon[35977]: osdmap e14: 2 total, 2 up, 2 in 2024-06-02T23:02:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:02:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:11 smithi174 ceph-mon[35994]: osdmap e14: 2 total, 2 up, 2 in 2024-06-02T23:02:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:02:12.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:11 smithi052 ceph-mon[32340]: osdmap e14: 2 total, 2 up, 2 in 2024-06-02T23:02:12.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:02:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:12 smithi174 ceph-mon[35994]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:12 smithi174 ceph-mon[35994]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:02:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:02:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:12 smithi123 ceph-mon[35977]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:12 smithi123 ceph-mon[35977]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:02:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:02:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:12 smithi052 ceph-mon[32340]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:12 smithi052 ceph-mon[32340]: from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:02:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:02:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:15.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:14 smithi123 ceph-mon[35977]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:15.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:14 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/3676524943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]: dispatch 2024-06-02T23:02:15.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:14 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]: dispatch 2024-06-02T23:02:15.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:14 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]': finished 2024-06-02T23:02:15.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:14 smithi123 ceph-mon[35977]: osdmap e15: 3 total, 2 up, 3 in 2024-06-02T23:02:15.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:14 smithi174 ceph-mon[35994]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:14 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/3676524943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]: dispatch 2024-06-02T23:02:15.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:14 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]: dispatch 2024-06-02T23:02:15.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:14 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]': finished 2024-06-02T23:02:15.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:14 smithi174 ceph-mon[35994]: osdmap e15: 3 total, 2 up, 3 in 2024-06-02T23:02:15.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:14 smithi052 ceph-mon[32340]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:14 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/3676524943' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]: dispatch 2024-06-02T23:02:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:14 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]: dispatch 2024-06-02T23:02:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:14 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4248f78e-ee22-4cd7-92e9-e449a51cb4b1"}]': finished 2024-06-02T23:02:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:14 smithi052 ceph-mon[32340]: osdmap e15: 3 total, 2 up, 3 in 2024-06-02T23:02:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:15 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/4114969086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:02:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:15 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/4114969086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:02:16.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:15 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/4114969086' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:02:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:16 smithi123 ceph-mon[35977]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:16 smithi174 ceph-mon[35994]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:16 smithi052 ceph-mon[32340]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:19.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:18 smithi174 ceph-mon[35994]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:19.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:18 smithi123 ceph-mon[35977]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:19.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:18 smithi052 ceph-mon[32340]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:21.141 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:20 smithi174 ceph-mon[35994]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:20 smithi123 ceph-mon[35977]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:21.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:20 smithi052 ceph-mon[32340]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-02T23:02:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:21 smithi123 ceph-mon[35977]: Deploying daemon osd.2 on smithi174 2024-06-02T23:02:22.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-02T23:02:22.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:22.277 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:21 smithi174 ceph-mon[35994]: Deploying daemon osd.2 on smithi174 2024-06-02T23:02:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-06-02T23:02:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:21 smithi052 ceph-mon[32340]: Deploying daemon osd.2 on smithi174 2024-06-02T23:02:23.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:22 smithi123 ceph-mon[35977]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:23.306 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:22 smithi174 ceph-mon[35994]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:23.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:22 smithi052 ceph-mon[32340]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:25.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:24 smithi123 ceph-mon[35977]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:25.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:24 smithi174 ceph-mon[35994]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:25.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:24 smithi052 ceph-mon[32340]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:26 smithi123 ceph-mon[35977]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:26 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:26 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:26 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:27.288 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:26 smithi174 ceph-mon[35994]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:27.288 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:26 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:27.288 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:26 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:27.288 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:26 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:26 smithi052 ceph-mon[32340]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:26 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:26 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:26 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:28 smithi123 ceph-mon[35977]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:29.259 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:28 smithi174 ceph-mon[35994]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:29.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:28 smithi052 ceph-mon[32340]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:30.276 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.276+0000 7f5d357fa700 1 -- 172.21.15.174:0/2790367980 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5d380628d0 con 0x7f5d200a2b90 2024-06-02T23:02:30.277 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d200a2b90 msgr2=0x7f5d200a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d200a2b90 0x7f5d200a5050 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c00b5c0 tx=0x7f5d2c005fd0 comp rx=0 tx=0).stop 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 msgr2=0x7f5d3810a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 0x7f5d3810a560 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5d2800b500 tx=0x7f5d2800b8c0 comp rx=0 tx=0).stop 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 shutdown_connections 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d200a2b90 0x7f5d200a5050 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d38102720 0x7f5d3819fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d38100b80 0x7f5d3810a560 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 --2- 172.21.15.174:0/2790367980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d38101d80 0x7f5d3819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 >> 172.21.15.174:0/2790367980 conn(0x7f5d380fc100 msgr2=0x7f5d38105150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 shutdown_connections 2024-06-02T23:02:30.279 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:02:30.279+0000 7f5d3e275700 1 -- 172.21.15.174:0/2790367980 wait complete. 2024-06-02T23:02:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:30.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:30.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:30.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:30.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:30.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.017 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.2.service 2024-06-02T23:02:31.019 INFO:tasks.cephadm:Deploying osd.3 on smithi052 with /dev/vg_nvme/lv_3... 2024-06-02T23:02:31.019 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-02T23:02:31.277 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:02:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:31 smithi052 ceph-mon[32340]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:31 smithi052 ceph-mon[32340]: from='osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T23:02:31.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:31 smithi052 ceph-mon[32340]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T23:02:31.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:31 smithi174 ceph-mon[35994]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:31.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:31.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:31 smithi174 ceph-mon[35994]: from='osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T23:02:31.578 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:31 smithi174 ceph-mon[35994]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T23:02:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:31 smithi123 ceph-mon[35977]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:31 smithi123 ceph-mon[35977]: from='osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T23:02:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:31 smithi123 ceph-mon[35977]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-06-02T23:02:32.466 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-mon[35994]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-02T23:02:32.466 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-mon[35994]: osdmap e16: 3 total, 2 up, 3 in 2024-06-02T23:02:32.466 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:32.466 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-mon[35994]: from='osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:02:32.467 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-mon[35994]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:02:32.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:32 smithi052 ceph-mon[32340]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-02T23:02:32.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:32 smithi052 ceph-mon[32340]: osdmap e16: 3 total, 2 up, 3 in 2024-06-02T23:02:32.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:32.586 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:32 smithi052 ceph-mon[32340]: from='osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:02:32.587 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:32 smithi052 ceph-mon[32340]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:02:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:32 smithi123 ceph-mon[35977]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-06-02T23:02:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:32 smithi123 ceph-mon[35977]: osdmap e16: 3 total, 2 up, 3 in 2024-06-02T23:02:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:32 smithi123 ceph-mon[35977]: from='osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:02:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:32 smithi123 ceph-mon[35977]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:02:33.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2[41129]: 2024-06-02T23:02:32.721+0000 7f325bc25700 -1 osd.2 0 waiting for initial osdmap 2024-06-02T23:02:33.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2[41129]: 2024-06-02T23:02:32.727+0000 7f3253a10700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:02:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-02T23:02:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-02T23:02:33.187 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-02T23:02:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-02T23:02:33.187 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00843469 s, 1.2 GB/s 2024-06-02T23:02:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-02T23:02:33.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:33 smithi052 ceph-mon[32340]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:33.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:33 smithi052 ceph-mon[32340]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:02:33.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:33 smithi052 ceph-mon[32340]: osdmap e17: 3 total, 2 up, 3 in 2024-06-02T23:02:33.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:33.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:33.495 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:33 smithi052 ceph-mon[32340]: from='osd.2 ' entity='osd.2' 2024-06-02T23:02:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:33 smithi174 ceph-mon[35994]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:33 smithi174 ceph-mon[35994]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:02:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:33 smithi174 ceph-mon[35994]: osdmap e17: 3 total, 2 up, 3 in 2024-06-02T23:02:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:33 smithi174 ceph-mon[35994]: from='osd.2 ' entity='osd.2' 2024-06-02T23:02:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:33 smithi123 ceph-mon[35977]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:33 smithi123 ceph-mon[35977]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:02:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:33 smithi123 ceph-mon[35977]: osdmap e17: 3 total, 2 up, 3 in 2024-06-02T23:02:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:33 smithi123 ceph-mon[35977]: from='osd.2 ' entity='osd.2' 2024-06-02T23:02:33.809 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_3 2024-06-02T23:02:34.070 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:02:34.456 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:02:34.456 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:02:34.456 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:34.456 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] boot 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: osdmap e18: 3 total, 3 up, 3 in 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:34.457 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:02:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:02:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] boot 2024-06-02T23:02:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: osdmap e18: 3 total, 3 up, 3 in 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:34.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: osd.2 [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] boot 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: osdmap e18: 3 total, 3 up, 3 in 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-06-02T23:02:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:34.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:35.386 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:35 smithi052 ceph-mon[32340]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:35.386 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:35 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi174 2024-06-02T23:02:35.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:35 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 17088M 2024-06-02T23:02:35.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-02T23:02:35.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:35 smithi052 ceph-mon[32340]: osdmap e19: 3 total, 3 up, 3 in 2024-06-02T23:02:35.387 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 -- 172.21.15.52:0/2140950521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0130102f80 msgr2=0x7f0130105370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 --2- 172.21.15.52:0/2140950521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0130102f80 0x7f0130105370 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f012000b0a0 tx=0x7f012000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 -- 172.21.15.52:0/2140950521 shutdown_connections 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 --2- 172.21.15.52:0/2140950521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 0x7f013010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 --2- 172.21.15.52:0/2140950521 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0130102f80 0x7f0130105370 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 --2- 172.21.15.52:0/2140950521 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01301058b0 0x7f0130107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 -- 172.21.15.52:0/2140950521 >> 172.21.15.52:0/2140950521 conn(0x7f01300fc9b0 msgr2=0x7f01300fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 -- 172.21.15.52:0/2140950521 shutdown_connections 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.617+0000 7f013568c700 1 -- 172.21.15.52:0/2140950521 wait complete. 2024-06-02T23:02:35.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 Processor -- start 2024-06-02T23:02:35.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 -- start start 2024-06-02T23:02:35.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0130102f80 0x7f01301985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 0x7f0130198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 0x7f013019fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f013010dfb0 con 0x7f01301058b0 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f013010de30 con 0x7f0130102f80 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f013568c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f013010e130 con 0x7f01301081e0 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.618+0000 7f012e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 0x7f0130198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 0x7f0130198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57782/0 (socket says 172.21.15.52:57782) 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 -- 172.21.15.52:0/2055170381 learned_addr learned my addr 172.21.15.52:0/2055170381 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012effd700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0130102f80 0x7f01301985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012f7fe700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 0x7f013019fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 -- 172.21.15.52:0/2055170381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 msgr2=0x7f013019fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 0x7f013019fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 -- 172.21.15.52:0/2055170381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0130102f80 msgr2=0x7f01301985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:35.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0130102f80 0x7f01301985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 -- 172.21.15.52:0/2055170381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01301a0240 con 0x7f01301058b0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012f7fe700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 0x7f013019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f012e7fc700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 0x7f0130198b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f011800e9d0 tx=0x7f011800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f011800cbd0 con 0x7f01301058b0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f011800cd70 con 0x7f01301058b0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01301a0530 con 0x7f01301058b0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0118018680 con 0x7f01301058b0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.619+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01301a0a70 con 0x7f01301058b0 2024-06-02T23:02:35.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.620+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0130069180 con 0x7f01301058b0 2024-06-02T23:02:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.622+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 15) ==== 137777+0+0 (secure 0 0 0) 0x7f0118010430 con 0x7f01301058b0 2024-06-02T23:02:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.622+0000 7f0127fff700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f011c0a2b90 0x7f011c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:02:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.622+0000 7f012effd700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f011c0a2b90 0x7f011c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:02:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.622+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7f01180cde30 con 0x7f01301058b0 2024-06-02T23:02:35.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.623+0000 7f012effd700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f011c0a2b90 0x7f011c0a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f012000b780 tx=0x7f01200060c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:02:35.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.627+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f011809c250 con 0x7f01301058b0 2024-06-02T23:02:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:35 smithi174 ceph-mon[35994]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:35 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi174 2024-06-02T23:02:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:35 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 17088M 2024-06-02T23:02:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-02T23:02:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:35 smithi174 ceph-mon[35994]: osdmap e19: 3 total, 3 up, 3 in 2024-06-02T23:02:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:02:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:35 smithi123 ceph-mon[35977]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-06-02T23:02:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:35 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi174 2024-06-02T23:02:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:35 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 17088M 2024-06-02T23:02:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-06-02T23:02:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:35 smithi123 ceph-mon[35977]: osdmap e19: 3 total, 3 up, 3 in 2024-06-02T23:02:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:02:35.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:35.861+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f01300628d0 con 0x7f011c0a2b90 2024-06-02T23:02:36.609 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49454]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T23:02:36.609 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49454]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:36.609 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49454]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:36.609 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49454]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:36.899 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49519]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T23:02:36.899 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49519]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:36.900 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49519]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:36.900 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:02:36 smithi052 sudo[49519]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: osdmap e20: 3 total, 3 up, 3 in 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:02:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:36.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:36 smithi052 ceph-mon[32340]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T23:02:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-02T23:02:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: osdmap e20: 3 total, 3 up, 3 in 2024-06-02T23:02:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:02:37.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:02:37.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:02:37.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:37.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:36 smithi123 ceph-mon[35977]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T23:02:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-06-02T23:02:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: osdmap e20: 3 total, 3 up, 3 in 2024-06-02T23:02:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: from='client.14277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:02:37.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:02:37.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:02:37.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:37.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:36 smithi174 ceph-mon[35994]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T23:02:37.276 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T23:02:37.276 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:37.276 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:37.276 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44311]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:37.738 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T23:02:37.738 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:37.738 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:37.738 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:02:37 smithi123 sudo[44314]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:37.798 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45121]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T23:02:37.798 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45121]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:37.798 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45121]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:37.798 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45121]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:37.799 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:37 smithi174 ceph-mon[35994]: osdmap e21: 3 total, 3 up, 3 in 2024-06-02T23:02:37.799 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:37.799 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:37 smithi123 ceph-mon[35977]: osdmap e21: 3 total, 3 up, 3 in 2024-06-02T23:02:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:38.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45124]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T23:02:38.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45124]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:38.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45124]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:38.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:02:37 smithi174 sudo[45124]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:37 smithi052 ceph-mon[32340]: osdmap e21: 3 total, 3 up, 3 in 2024-06-02T23:02:38.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:38.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T23:02:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49587]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:38.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49587]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:38.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:38.762+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f01180981c0 con 0x7f01301058b0 2024-06-02T23:02:39.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49607]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T23:02:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49607]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49607]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:38 smithi052 sudo[49607]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:39.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:38 smithi174 sudo[45127]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T23:02:39.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:38 smithi174 sudo[45127]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:39.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:38 smithi174 sudo[45127]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:39.245 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 sudo[45127]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:39.503 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 sudo[45130]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 sudo[45130]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 sudo[45130]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 sudo[45130]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: Cluster is now healthy 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:39.504 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:39.505 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-02T23:02:39.505 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:39.505 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:39.505 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: Cluster is now healthy 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44317]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44317]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: pgmap v85: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: Cluster is now healthy 2024-06-02T23:02:39.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-06-02T23:02:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44317]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-06-02T23:02:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-06-02T23:02:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44320]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-06-02T23:02:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:39 smithi123 sudo[44320]: pam_unix(sudo:session): session closed for user root 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4277486781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e32ed956-2af4-430f-ac3c-e349190ebfa3"}]: dispatch 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4277486781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e32ed956-2af4-430f-ac3c-e349190ebfa3"}]': finished 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: osdmap e22: 4 total, 3 up, 4 in 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:40.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:40.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:40.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:40.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:40 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2591312788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:02:40.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:40.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4277486781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e32ed956-2af4-430f-ac3c-e349190ebfa3"}]: dispatch 2024-06-02T23:02:40.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:40.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:40.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4277486781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e32ed956-2af4-430f-ac3c-e349190ebfa3"}]': finished 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: osdmap e22: 4 total, 3 up, 4 in 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:40.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:40 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2591312788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:02:40.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:40.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4277486781' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e32ed956-2af4-430f-ac3c-e349190ebfa3"}]: dispatch 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4277486781' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e32ed956-2af4-430f-ac3c-e349190ebfa3"}]': finished 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: osdmap e22: 4 total, 3 up, 4 in 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-06-02T23:02:40.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:40.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-06-02T23:02:40.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:40 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2591312788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:02:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:41 smithi174 ceph-mon[35994]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:41.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:41 smithi052 ceph-mon[32340]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:41.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:41 smithi123 ceph-mon[35977]: pgmap v87: 1 pgs: 1 active+clean; 449 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:43.570 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:43 smithi052 ceph-mon[32340]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 368 KiB/s wr, 24 op/s 2024-06-02T23:02:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:43 smithi174 ceph-mon[35994]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 368 KiB/s wr, 24 op/s 2024-06-02T23:02:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:43 smithi123 ceph-mon[35977]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 7.2 KiB/s rd, 368 KiB/s wr, 24 op/s 2024-06-02T23:02:45.497 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:45 smithi052 ceph-mon[32340]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 262 KiB/s wr, 17 op/s 2024-06-02T23:02:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:45 smithi174 ceph-mon[35994]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 262 KiB/s wr, 17 op/s 2024-06-02T23:02:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:45 smithi123 ceph-mon[35977]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 5.2 KiB/s rd, 262 KiB/s wr, 17 op/s 2024-06-02T23:02:47.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:47 smithi052 ceph-mon[32340]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 204 KiB/s wr, 13 op/s 2024-06-02T23:02:47.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-02T23:02:47.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:47 smithi174 ceph-mon[35994]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 204 KiB/s wr, 13 op/s 2024-06-02T23:02:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-02T23:02:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:47 smithi123 ceph-mon[35977]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 4.0 KiB/s rd, 204 KiB/s wr, 13 op/s 2024-06-02T23:02:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-06-02T23:02:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:48.561 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:48 smithi052 ceph-mon[32340]: Deploying daemon osd.3 on smithi052 2024-06-02T23:02:48.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:48 smithi174 ceph-mon[35994]: Deploying daemon osd.3 on smithi052 2024-06-02T23:02:48.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:48 smithi123 ceph-mon[35977]: Deploying daemon osd.3 on smithi052 2024-06-02T23:02:49.611 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:49 smithi052 ceph-mon[32340]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-06-02T23:02:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:49 smithi174 ceph-mon[35994]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-06-02T23:02:49.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:49 smithi123 ceph-mon[35977]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 183 KiB/s wr, 12 op/s 2024-06-02T23:02:51.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:51 smithi052 ceph-mon[32340]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 175 KiB/s wr, 11 op/s 2024-06-02T23:02:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:51 smithi174 ceph-mon[35994]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 175 KiB/s wr, 11 op/s 2024-06-02T23:02:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:51 smithi123 ceph-mon[35977]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.4 KiB/s rd, 175 KiB/s wr, 11 op/s 2024-06-02T23:02:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:53 smithi052 ceph-mon[32340]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-06-02T23:02:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:53.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:53 smithi174 ceph-mon[35994]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-06-02T23:02:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:53 smithi123 ceph-mon[35977]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 152 KiB/s wr, 10 op/s 2024-06-02T23:02:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:55 smithi052 ceph-mon[32340]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:55.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:55.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:55.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:55 smithi174 ceph-mon[35994]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:55 smithi123 ceph-mon[35977]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:02:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:02:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:55.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.970+0000 7f0127fff700 1 -- 172.21.15.52:0/2055170381 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f01300628d0 con 0x7f011c0a2b90 2024-06-02T23:02:55.972 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 3 on host 'smithi052' 2024-06-02T23:02:55.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f011c0a2b90 msgr2=0x7f011c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:55.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f011c0a2b90 0x7f011c0a5050 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f012000b780 tx=0x7f01200060c0 comp rx=0 tx=0).stop 2024-06-02T23:02:55.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 msgr2=0x7f0130198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:02:55.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 0x7f0130198b20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f011800e9d0 tx=0x7f011800ece0 comp rx=0 tx=0).stop 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 shutdown_connections 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f011c0a2b90 0x7f011c0a5050 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01301081e0 0x7f013019fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0130102f80 0x7f01301985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 --2- 172.21.15.52:0/2055170381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01301058b0 0x7f0130198b20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 >> 172.21.15.52:0/2055170381 conn(0x7f01300fc9b0 msgr2=0x7f0130104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 shutdown_connections 2024-06-02T23:02:55.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:02:55.973+0000 7f013568c700 1 -- 172.21.15.52:0/2055170381 wait complete. 2024-06-02T23:02:56.741 DEBUG:teuthology.orchestra.run.smithi052:osd.3> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.3.service 2024-06-02T23:02:56.743 INFO:tasks.cephadm:Deploying osd.4 on smithi123 with /dev/vg_nvme/lv_3... 2024-06-02T23:02:56.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-02T23:02:56.988 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:02:57.078 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:56 smithi052 ceph-mon[32340]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:57.078 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:56 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:57.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:56 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:57.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:56 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:57.079 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:56 smithi052 ceph-mon[32340]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T23:02:57.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:56 smithi123 ceph-mon[35977]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:57.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:56 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:57.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:56 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:57.110 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:56 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:57.111 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:56 smithi123 ceph-mon[35977]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T23:02:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:56 smithi174 ceph-mon[35994]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:56 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:56 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:02:57.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:56 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:02:57.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:56 smithi174 ceph-mon[35994]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-06-02T23:02:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:57 smithi123 ceph-mon[35977]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-02T23:02:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:57 smithi123 ceph-mon[35977]: osdmap e23: 4 total, 3 up, 4 in 2024-06-02T23:02:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:57 smithi123 ceph-mon[35977]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:02:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:58.052 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:57 smithi052 ceph-mon[32340]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-02T23:02:58.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:57 smithi052 ceph-mon[32340]: osdmap e23: 4 total, 3 up, 4 in 2024-06-02T23:02:58.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:57 smithi052 ceph-mon[32340]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:02:58.053 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:57 smithi174 ceph-mon[35994]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-06-02T23:02:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:57 smithi174 ceph-mon[35994]: osdmap e23: 4 total, 3 up, 4 in 2024-06-02T23:02:58.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:57 smithi174 ceph-mon[35994]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:02:58.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:58.650 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:02:58.460+0000 7fdb12dc3700 -1 osd.3 0 waiting for initial osdmap 2024-06-02T23:02:58.650 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:02:58.467+0000 7fdb0a3ad700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:02:59.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-mon[32340]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:59.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-mon[32340]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:02:59.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-mon[32340]: osdmap e24: 4 total, 3 up, 4 in 2024-06-02T23:02:59.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:59.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:59.239 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:58 smithi052 ceph-mon[32340]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' 2024-06-02T23:02:59.243 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-02T23:02:59.244 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-02T23:02:59.244 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-02T23:02:59.244 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-02T23:02:59.244 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119644 s, 876 MB/s 2024-06-02T23:02:59.244 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-02T23:02:59.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:58 smithi123 ceph-mon[35977]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:59.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:58 smithi123 ceph-mon[35977]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:02:59.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:58 smithi123 ceph-mon[35977]: osdmap e24: 4 total, 3 up, 4 in 2024-06-02T23:02:59.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:59.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:59.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:58 smithi123 ceph-mon[35977]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' 2024-06-02T23:02:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:58 smithi174 ceph-mon[35994]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:02:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:58 smithi174 ceph-mon[35994]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:02:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:58 smithi174 ceph-mon[35994]: osdmap e24: 4 total, 3 up, 4 in 2024-06-02T23:02:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:59.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:02:59.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:58 smithi174 ceph-mon[35994]: from='osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243]' entity='osd.3' 2024-06-02T23:02:59.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-06-02T23:03:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] boot 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: osdmap e25: 4 total, 4 up, 4 in 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:00.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:00.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:02:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.157 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] boot 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: osdmap e25: 4 total, 4 up, 4 in 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:00.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:00.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:00.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:00.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:02:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:03:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:03:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: osd.3 [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] boot 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: osdmap e25: 4 total, 4 up, 4 in 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:00.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:02:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:01.251 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:00 smithi123 ceph-mon[35977]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:03:01.252 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:00 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi052 2024-06-02T23:03:01.252 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:00 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 8546M 2024-06-02T23:03:01.252 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:00 smithi123 ceph-mon[35977]: osdmap e26: 4 total, 4 up, 4 in 2024-06-02T23:03:01.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:00 smithi174 ceph-mon[35994]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:03:01.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:00 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi052 2024-06-02T23:03:01.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:00 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 8546M 2024-06-02T23:03:01.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:00 smithi174 ceph-mon[35994]: osdmap e26: 4 total, 4 up, 4 in 2024-06-02T23:03:01.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:00 smithi052 ceph-mon[32340]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 92 MiB used, 268 GiB / 268 GiB avail 2024-06-02T23:03:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:00 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi052 2024-06-02T23:03:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:00 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 8546M 2024-06-02T23:03:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:00 smithi052 ceph-mon[32340]: osdmap e26: 4 total, 4 up, 4 in 2024-06-02T23:03:01.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 -- 172.21.15.123:0/5455552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05281024e0 msgr2=0x7f0528102900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:01.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 --2- 172.21.15.123:0/5455552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05281024e0 0x7f0528102900 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f05100099c0 tx=0x7f0510009cd0 comp rx=0 tx=0).stop 2024-06-02T23:03:01.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 -- 172.21.15.123:0/5455552 shutdown_connections 2024-06-02T23:03:01.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 --2- 172.21.15.123:0/5455552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0528104080 0x7f052810a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:01.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 --2- 172.21.15.123:0/5455552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05281024e0 0x7f0528102900 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:01.760 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 --2- 172.21.15.123:0/5455552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05281036e0 0x7f0528103b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 -- 172.21.15.123:0/5455552 >> 172.21.15.123:0/5455552 conn(0x7f05280fda80 msgr2=0x7f05280ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 -- 172.21.15.123:0/5455552 shutdown_connections 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.758+0000 7f052da13700 1 -- 172.21.15.123:0/5455552 wait complete. 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 Processor -- start 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 -- start start 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05281024e0 0x7f05281986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05281036e0 0x7f0528198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 0x7f052819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f052810dfe0 con 0x7f05281024e0 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f052810de60 con 0x7f0528104080 2024-06-02T23:03:01.761 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f052da13700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f052810e160 con 0x7f05281036e0 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f05277fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 0x7f052819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f05277fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 0x7f052819fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:49342/0 (socket says 172.21.15.123:49342) 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f05277fe700 1 -- 172.21.15.123:0/1225479880 learned_addr learned my addr 172.21.15.123:0/1225479880 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.759+0000 7f0526ffd700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05281024e0 0x7f05281986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05267fc700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05281036e0 0x7f0528198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05277fe700 1 -- 172.21.15.123:0/1225479880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05281036e0 msgr2=0x7f0528198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05277fe700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05281036e0 0x7f0528198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05277fe700 1 -- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05281024e0 msgr2=0x7f05281986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05277fe700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05281024e0 0x7f05281986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05277fe700 1 -- 172.21.15.123:0/1225479880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05281a0310 con 0x7f0528104080 2024-06-02T23:03:01.762 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05267fc700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05281036e0 0x7f0528198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:03:01.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f05277fe700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 0x7f052819fc70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f052000b970 tx=0x7f052000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:03:01.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520019070 con 0x7f0528104080 2024-06-02T23:03:01.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f052000cde0 con 0x7f0528104080 2024-06-02T23:03:01.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05281a0600 con 0x7f0528104080 2024-06-02T23:03:01.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520007370 con 0x7f0528104080 2024-06-02T23:03:01.763 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.760+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a0bc0 con 0x7f0528104080 2024-06-02T23:03:01.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.761+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0528069180 con 0x7f0528104080 2024-06-02T23:03:01.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.762+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f052001c070 con 0x7f0528104080 2024-06-02T23:03:01.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.763+0000 7f052ca11700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05140a2c20 0x7f05140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:01.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.763+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3698+0+0 (secure 0 0 0) 0x7f05200cd2c0 con 0x7f0528104080 2024-06-02T23:03:01.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.763+0000 7f0526ffd700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05140a2c20 0x7f05140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:01.764 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.763+0000 7f0526ffd700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05140a2c20 0x7f05140a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0510000c00 tx=0x7f0510005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:03:01.767 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.767+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f052009b420 con 0x7f0528104080 2024-06-02T23:03:01.974 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:01.974+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f05280628d0 con 0x7f05140a2c20 2024-06-02T23:03:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:02 smithi123 ceph-mon[35977]: osdmap e27: 4 total, 4 up, 4 in 2024-06-02T23:03:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:02 smithi174 ceph-mon[35994]: osdmap e27: 4 total, 4 up, 4 in 2024-06-02T23:03:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:01 smithi052 ceph-mon[32340]: osdmap e27: 4 total, 4 up, 4 in 2024-06-02T23:03:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:03.228 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:03 smithi123 ceph-mon[35977]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:03.229 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:03 smithi123 ceph-mon[35977]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:03.229 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:03 smithi123 ceph-mon[35977]: osdmap e28: 4 total, 4 up, 4 in 2024-06-02T23:03:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:03 smithi174 ceph-mon[35994]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:03 smithi174 ceph-mon[35994]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:03 smithi174 ceph-mon[35994]: osdmap e28: 4 total, 4 up, 4 in 2024-06-02T23:03:03.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:03 smithi052 ceph-mon[32340]: pgmap v103: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:03.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:03 smithi052 ceph-mon[32340]: from='client.24215 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:03.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:03 smithi052 ceph-mon[32340]: osdmap e28: 4 total, 4 up, 4 in 2024-06-02T23:03:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:04 smithi123 ceph-mon[35977]: osdmap e29: 4 total, 4 up, 4 in 2024-06-02T23:03:04.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:04 smithi174 ceph-mon[35994]: osdmap e29: 4 total, 4 up, 4 in 2024-06-02T23:03:04.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:04 smithi052 ceph-mon[32340]: osdmap e29: 4 total, 4 up, 4 in 2024-06-02T23:03:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:05 smithi123 ceph-mon[35977]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:05 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/2519695597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]: dispatch 2024-06-02T23:03:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:05 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]: dispatch 2024-06-02T23:03:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:05 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]': finished 2024-06-02T23:03:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:05 smithi123 ceph-mon[35977]: osdmap e30: 5 total, 4 up, 5 in 2024-06-02T23:03:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:05 smithi174 ceph-mon[35994]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:05 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/2519695597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]: dispatch 2024-06-02T23:03:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:05 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]: dispatch 2024-06-02T23:03:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:05 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]': finished 2024-06-02T23:03:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:05 smithi174 ceph-mon[35994]: osdmap e30: 5 total, 4 up, 5 in 2024-06-02T23:03:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:05.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:05 smithi052 ceph-mon[32340]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:05.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:05 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/2519695597' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]: dispatch 2024-06-02T23:03:05.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:05 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]: dispatch 2024-06-02T23:03:05.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:05 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9734ce9-e810-4891-a831-1afaef0f45b6"}]': finished 2024-06-02T23:03:05.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:05 smithi052 ceph-mon[32340]: osdmap e30: 5 total, 4 up, 5 in 2024-06-02T23:03:05.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:06 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/2430447217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:06.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:06 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/2430447217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:06.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:06 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/2430447217' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:07 smithi174 ceph-mon[35994]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:07.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:07 smithi052 ceph-mon[32340]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:07.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:07 smithi123 ceph-mon[35977]: pgmap v108: 1 pgs: 1 remapped+peering; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:09 smithi123 ceph-mon[35977]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:03:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:09 smithi174 ceph-mon[35994]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:03:09.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:09 smithi052 ceph-mon[32340]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:03:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:11 smithi123 ceph-mon[35977]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-02T23:03:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:11 smithi174 ceph-mon[35994]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-02T23:03:11.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:11 smithi052 ceph-mon[32340]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-06-02T23:03:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-02T23:03:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-02T23:03:12.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:12.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-06-02T23:03:12.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:13 smithi123 ceph-mon[35977]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-02T23:03:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:13 smithi123 ceph-mon[35977]: Deploying daemon osd.4 on smithi123 2024-06-02T23:03:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:13 smithi174 ceph-mon[35994]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-02T23:03:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:13 smithi174 ceph-mon[35994]: Deploying daemon osd.4 on smithi123 2024-06-02T23:03:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:13 smithi052 ceph-mon[32340]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-06-02T23:03:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:13 smithi052 ceph-mon[32340]: Deploying daemon osd.4 on smithi123 2024-06-02T23:03:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:15 smithi174 ceph-mon[35994]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:03:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:15 smithi052 ceph-mon[32340]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:03:15.403 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:15 smithi123 ceph-mon[35977]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:03:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:17 smithi123 ceph-mon[35977]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-02T23:03:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:17 smithi174 ceph-mon[35994]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-02T23:03:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:17 smithi052 ceph-mon[32340]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-06-02T23:03:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:18 smithi174 ceph-mon[35994]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-02T23:03:18.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:18.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:18.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:18.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:18 smithi052 ceph-mon[32340]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-02T23:03:19.001 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:19.001 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:19.001 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:19.001 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:18 smithi123 ceph-mon[35977]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-02T23:03:21.448 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 4 on host 'smithi123' 2024-06-02T23:03:21.448 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.447+0000 7f052ca11700 1 -- 172.21.15.123:0/1225479880 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05280628d0 con 0x7f05140a2c20 2024-06-02T23:03:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05140a2c20 msgr2=0x7f05140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05140a2c20 0x7f05140a50e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0510000c00 tx=0x7f0510005fb0 comp rx=0 tx=0).stop 2024-06-02T23:03:21.450 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 msgr2=0x7f052819fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 0x7f052819fc70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f052000b970 tx=0x7f052000bc80 comp rx=0 tx=0).stop 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 shutdown_connections 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05140a2c20 0x7f05140a50e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0528104080 0x7f052819fc70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05281024e0 0x7f05281986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 --2- 172.21.15.123:0/1225479880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05281036e0 0x7f0528198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 >> 172.21.15.123:0/1225479880 conn(0x7f05280fda80 msgr2=0x7f0528106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 shutdown_connections 2024-06-02T23:03:21.451 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:03:21.450+0000 7f052da13700 1 -- 172.21.15.123:0/1225479880 wait complete. 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T23:03:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:21 smithi123 ceph-mon[35977]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T23:03:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T23:03:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:21 smithi174 ceph-mon[35994]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T23:03:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T23:03:21.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:21 smithi052 ceph-mon[32340]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-06-02T23:03:22.082 DEBUG:teuthology.orchestra.run.smithi123:osd.4> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.4.service 2024-06-02T23:03:22.084 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-06-02T23:03:22.084 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-06-02T23:03:22.329 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:03:23.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-mon[35977]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:23.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-mon[35977]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-02T23:03:23.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-mon[35977]: osdmap e31: 5 total, 4 up, 5 in 2024-06-02T23:03:23.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:23.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-mon[35977]: from='osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:03:23.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-mon[35977]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:03:23.559 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:03:23.410+0000 7fd6b4928700 -1 osd.4 0 waiting for initial osdmap 2024-06-02T23:03:23.560 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:03:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:03:23.417+0000 7fd6b071b700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:03:23.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:23 smithi174 ceph-mon[35994]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:23.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:23 smithi174 ceph-mon[35994]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-02T23:03:23.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:23 smithi174 ceph-mon[35994]: osdmap e31: 5 total, 4 up, 5 in 2024-06-02T23:03:23.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:23 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:23.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:23 smithi174 ceph-mon[35994]: from='osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:03:23.685 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:23 smithi174 ceph-mon[35994]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:03:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:23 smithi052 ceph-mon[32340]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:23 smithi052 ceph-mon[32340]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-06-02T23:03:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:23 smithi052 ceph-mon[32340]: osdmap e31: 5 total, 4 up, 5 in 2024-06-02T23:03:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:23 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:23 smithi052 ceph-mon[32340]: from='osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:03:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:23 smithi052 ceph-mon[32340]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:03:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-06-02T23:03:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-06-02T23:03:24.303 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-06-02T23:03:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-06-02T23:03:24.303 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114774 s, 914 MB/s 2024-06-02T23:03:24.303 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-06-02T23:03:24.705 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:24 smithi123 ceph-mon[35977]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:03:24.705 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:24 smithi123 ceph-mon[35977]: osdmap e32: 5 total, 4 up, 5 in 2024-06-02T23:03:24.705 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:24.706 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:24 smithi174 ceph-mon[35994]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:03:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:24 smithi174 ceph-mon[35994]: osdmap e32: 5 total, 4 up, 5 in 2024-06-02T23:03:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:24 smithi052 ceph-mon[32340]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:03:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:24 smithi052 ceph-mon[32340]: osdmap e32: 5 total, 4 up, 5 in 2024-06-02T23:03:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:24.934 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-06-02T23:03:25.175 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] boot 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: osdmap e33: 5 total, 5 up, 5 in 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:25.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:25.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:25.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:25.561 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] boot 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: osdmap e33: 5 total, 5 up, 5 in 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 127 MiB used, 357 GiB / 358 GiB avail 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: osd.4 [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] boot 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: osdmap e33: 5 total, 5 up, 5 in 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:25.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:26.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:26 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi123 2024-06-02T23:03:26.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:26 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 8544M 2024-06-02T23:03:26.670 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:26 smithi174 ceph-mon[35994]: osdmap e34: 5 total, 5 up, 5 in 2024-06-02T23:03:26.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:26 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi123 2024-06-02T23:03:26.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:26 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 8544M 2024-06-02T23:03:26.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:26 smithi123 ceph-mon[35977]: osdmap e34: 5 total, 5 up, 5 in 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 -- 172.21.15.174:0/3197644497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed38103700 msgr2=0x7fed38105b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/3197644497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed38103700 0x7fed38105b90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fed20009990 tx=0x7fed20009ca0 comp rx=0 tx=0).stop 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 -- 172.21.15.174:0/3197644497 shutdown_connections 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/3197644497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed381060d0 0x7fed38106550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/3197644497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38100d90 0x7fed381031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/3197644497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed38103700 0x7fed38105b90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 -- 172.21.15.174:0/3197644497 >> 172.21.15.174:0/3197644497 conn(0x7fed380fc9f0 msgr2=0x7fed380fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 -- 172.21.15.174:0/3197644497 shutdown_connections 2024-06-02T23:03:26.772 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 -- 172.21.15.174:0/3197644497 wait complete. 2024-06-02T23:03:26.774 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.772+0000 7fed3e3fb700 1 Processor -- start 2024-06-02T23:03:26.774 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 -- start start 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed38100d90 0x7fed3819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38103700 0x7fed3819f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 0x7fed381a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fed38114500 con 0x7fed38100d90 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fed38114380 con 0x7fed38103700 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed38114680 con 0x7fed381060d0 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 0x7fed381a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 0x7fed381a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48576/0 (socket says 172.21.15.174:48576) 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 -- 172.21.15.174:0/2858676131 learned_addr learned my addr 172.21.15.174:0/2858676131 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed2ffff700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38103700 0x7fed3819f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed37fff700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed38100d90 0x7fed3819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 -- 172.21.15.174:0/2858676131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38103700 msgr2=0x7fed3819f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:26.775 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38103700 0x7fed3819f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 -- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed38100d90 msgr2=0x7fed3819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed38100d90 0x7fed3819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 -- 172.21.15.174:0/2858676131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed381a6810 con 0x7fed381060d0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed37fff700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed38100d90 0x7fed3819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3c998700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 0x7fed381a6170 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed30007f10 tx=0x7fed3000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30004bb0 con 0x7fed381060d0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fed30004d50 con 0x7fed381060d0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.773+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed381a6b00 con 0x7fed381060d0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.774+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30005990 con 0x7fed381060d0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.774+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed381a7040 con 0x7fed381060d0 2024-06-02T23:03:26.776 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.775+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed38069180 con 0x7fed381060d0 2024-06-02T23:03:26.777 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.776+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fed3001a020 con 0x7fed381060d0 2024-06-02T23:03:26.777 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.776+0000 7fed35ffb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fed240a2ca0 0x7fed240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:26.777 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.776+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fed300cd720 con 0x7fed381060d0 2024-06-02T23:03:26.777 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.777+0000 7fed37fff700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fed240a2ca0 0x7fed240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:26.777 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.777+0000 7fed37fff700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fed240a2ca0 0x7fed240a5160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed28005950 tx=0x7fed280058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:03:26.780 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.781+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fed30010070 con 0x7fed381060d0 2024-06-02T23:03:26.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:26 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi123 2024-06-02T23:03:26.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:26 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 8544M 2024-06-02T23:03:26.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:26 smithi052 ceph-mon[32340]: osdmap e34: 5 total, 5 up, 5 in 2024-06-02T23:03:26.982 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:26.982+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fed380628d0 con 0x7fed240a2ca0 2024-06-02T23:03:27.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:27 smithi174 ceph-mon[35994]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:27.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:27.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:27.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:27 smithi123 ceph-mon[35977]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:27 smithi052 ceph-mon[32340]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:27.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:27.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:27.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:28.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:28 smithi174 ceph-mon[35994]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:28.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:28 smithi123 ceph-mon[35977]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:28.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:28 smithi052 ceph-mon[32340]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:29 smithi123 ceph-mon[35977]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:29.799 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:29 smithi174 ceph-mon[35994]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:29.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:29 smithi052 ceph-mon[32340]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:30 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/2728748834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]: dispatch 2024-06-02T23:03:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:30 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]: dispatch 2024-06-02T23:03:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:30 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]': finished 2024-06-02T23:03:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:30 smithi174 ceph-mon[35994]: osdmap e35: 6 total, 5 up, 6 in 2024-06-02T23:03:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:30.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:30 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/4011281021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:30.736 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:30 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/2728748834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]: dispatch 2024-06-02T23:03:30.736 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:30 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]: dispatch 2024-06-02T23:03:30.736 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:30 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]': finished 2024-06-02T23:03:30.736 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:30 smithi123 ceph-mon[35977]: osdmap e35: 6 total, 5 up, 6 in 2024-06-02T23:03:30.737 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:30.737 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:30 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/4011281021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:30.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:30 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/2728748834' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]: dispatch 2024-06-02T23:03:30.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:30 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]: dispatch 2024-06-02T23:03:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:30 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd"}]': finished 2024-06-02T23:03:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:30 smithi052 ceph-mon[32340]: osdmap e35: 6 total, 5 up, 6 in 2024-06-02T23:03:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:30 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/4011281021' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:31 smithi123 ceph-mon[35977]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:31 smithi174 ceph-mon[35994]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:31 smithi052 ceph-mon[32340]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:33 smithi123 ceph-mon[35977]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:33 smithi174 ceph-mon[35994]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:33 smithi052 ceph-mon[32340]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:35.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:35 smithi174 ceph-mon[35994]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:35 smithi123 ceph-mon[35977]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:35 smithi052 ceph-mon[32340]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:37.522 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:37 smithi174 ceph-mon[35994]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:37.522 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-02T23:03:37.522 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:37 smithi123 ceph-mon[35977]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-02T23:03:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:37 smithi052 ceph-mon[32340]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-06-02T23:03:37.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:38.551 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:38 smithi174 ceph-mon[35994]: Deploying daemon osd.5 on smithi174 2024-06-02T23:03:38.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:38 smithi123 ceph-mon[35977]: Deploying daemon osd.5 on smithi174 2024-06-02T23:03:38.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:38 smithi052 ceph-mon[32340]: Deploying daemon osd.5 on smithi174 2024-06-02T23:03:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:39 smithi123 ceph-mon[35977]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:39.791 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:39 smithi174 ceph-mon[35994]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:39.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:39 smithi052 ceph-mon[32340]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:41.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:41 smithi174 ceph-mon[35994]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:41 smithi123 ceph-mon[35977]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:41.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:41 smithi052 ceph-mon[32340]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:43 smithi174 ceph-mon[35994]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:43 smithi052 ceph-mon[32340]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:43 smithi123 ceph-mon[35977]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:45 smithi174 ceph-mon[35994]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:45.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:45 smithi052 ceph-mon[32340]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:45.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:45.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:45.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:45 smithi123 ceph-mon[35977]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:45.999 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:45.999+0000 7fed35ffb700 1 -- 172.21.15.174:0/2858676131 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fed380628d0 con 0x7fed240a2ca0 2024-06-02T23:03:45.999 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-06-02T23:03:46.001 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fed240a2ca0 msgr2=0x7fed240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fed240a2ca0 0x7fed240a5160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fed28005950 tx=0x7fed280058e0 comp rx=0 tx=0).stop 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 msgr2=0x7fed381a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 0x7fed381a6170 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fed30007f10 tx=0x7fed3000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 shutdown_connections 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fed240a2ca0 0x7fed240a5160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed381060d0 0x7fed381a6170 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fed38100d90 0x7fed3819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 --2- 172.21.15.174:0/2858676131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fed38103700 0x7fed3819f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 >> 172.21.15.174:0/2858676131 conn(0x7fed380fc9f0 msgr2=0x7fed38101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 shutdown_connections 2024-06-02T23:03:46.002 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:03:46.002+0000 7fed3e3fb700 1 -- 172.21.15.174:0/2858676131 wait complete. 2024-06-02T23:03:46.324 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:46.324 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.325 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.325 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.325 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:46.325 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T23:03:46.325 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:46 smithi174 ceph-mon[35994]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T23:03:46.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:46.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:46.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T23:03:46.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:46 smithi052 ceph-mon[32340]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T23:03:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:46 smithi123 ceph-mon[35977]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-06-02T23:03:46.659 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.5.service 2024-06-02T23:03:46.661 INFO:tasks.cephadm:Deploying osd.6 on smithi052 with /dev/vg_nvme/lv_2... 2024-06-02T23:03:46.661 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-02T23:03:46.919 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:03:47.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:47 smithi174 ceph-mon[35994]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:47.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:47 smithi174 ceph-mon[35994]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-02T23:03:47.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:47 smithi174 ceph-mon[35994]: osdmap e36: 6 total, 5 up, 6 in 2024-06-02T23:03:47.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:47.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:47 smithi174 ceph-mon[35994]: from='osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:03:47.160 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:47 smithi174 ceph-mon[35994]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:03:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:47 smithi052 ceph-mon[32340]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:47 smithi052 ceph-mon[32340]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-02T23:03:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:47 smithi052 ceph-mon[32340]: osdmap e36: 6 total, 5 up, 6 in 2024-06-02T23:03:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:47 smithi052 ceph-mon[32340]: from='osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:03:47.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:47 smithi052 ceph-mon[32340]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:03:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:47 smithi123 ceph-mon[35977]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:47 smithi123 ceph-mon[35977]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-06-02T23:03:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:47 smithi123 ceph-mon[35977]: osdmap e36: 6 total, 5 up, 6 in 2024-06-02T23:03:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:47 smithi123 ceph-mon[35977]: from='osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:03:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:47 smithi123 ceph-mon[35977]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:03:48.644 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:03:48.412+0000 7fef029d5700 -1 osd.5 0 waiting for initial osdmap 2024-06-02T23:03:48.644 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:03:48.419+0000 7feefa7c0700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:03:49.008 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-02T23:03:49.008 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-02T23:03:49.009 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-02T23:03:49.009 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-02T23:03:49.009 INFO:teuthology.orchestra.run.smithi052.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00838467 s, 1.3 GB/s 2024-06-02T23:03:49.009 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-02T23:03:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:48 smithi123 ceph-mon[35977]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:48 smithi123 ceph-mon[35977]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:03:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:48 smithi123 ceph-mon[35977]: osdmap e37: 6 total, 5 up, 6 in 2024-06-02T23:03:49.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:49.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:49.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:48 smithi123 ceph-mon[35977]: from='osd.5 ' entity='osd.5' 2024-06-02T23:03:49.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:48 smithi052 ceph-mon[32340]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:48 smithi052 ceph-mon[32340]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:03:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:48 smithi052 ceph-mon[32340]: osdmap e37: 6 total, 5 up, 6 in 2024-06-02T23:03:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:49.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:48 smithi052 ceph-mon[32340]: from='osd.5 ' entity='osd.5' 2024-06-02T23:03:49.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-mon[35994]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 154 MiB used, 447 GiB / 447 GiB avail 2024-06-02T23:03:49.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-mon[35994]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:03:49.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-mon[35994]: osdmap e37: 6 total, 5 up, 6 in 2024-06-02T23:03:49.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:49.346 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:49.347 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:48 smithi174 ceph-mon[35994]: from='osd.5 ' entity='osd.5' 2024-06-02T23:03:49.615 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_2 2024-06-02T23:03:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] boot 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: osdmap e38: 6 total, 6 up, 6 in 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:50.213 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.214 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:50.214 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:50.214 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] boot 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: osdmap e38: 6 total, 6 up, 6 in 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:03:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:03:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: osd.5 [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] boot 2024-06-02T23:03:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: osdmap e38: 6 total, 6 up, 6 in 2024-06-02T23:03:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:03:50.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:03:51.236 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:50 smithi052 ceph-mon[32340]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:51.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:50 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi174 2024-06-02T23:03:51.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:50 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 8544M 2024-06-02T23:03:51.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:50 smithi052 ceph-mon[32340]: osdmap e39: 6 total, 6 up, 6 in 2024-06-02T23:03:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:50 smithi123 ceph-mon[35977]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:50 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi174 2024-06-02T23:03:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:50 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 8544M 2024-06-02T23:03:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:50 smithi123 ceph-mon[35977]: osdmap e39: 6 total, 6 up, 6 in 2024-06-02T23:03:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:50 smithi174 ceph-mon[35994]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:50 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi174 2024-06-02T23:03:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:50 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 8544M 2024-06-02T23:03:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:50 smithi174 ceph-mon[35994]: osdmap e39: 6 total, 6 up, 6 in 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 -- 172.21.15.52:0/868672066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c8102fa0 msgr2=0x7fc4c8105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 --2- 172.21.15.52:0/868672066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8105390 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc4c4009990 tx=0x7fc4c4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 -- 172.21.15.52:0/868672066 shutdown_connections 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 --2- 172.21.15.52:0/868672066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8108200 0x7fc4c810a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 --2- 172.21.15.52:0/868672066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8105390 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 --2- 172.21.15.52:0/868672066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c81058d0 0x7fc4c8107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 -- 172.21.15.52:0/868672066 >> 172.21.15.52:0/868672066 conn(0x7fc4c80fc9b0 msgr2=0x7fc4c80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 -- 172.21.15.52:0/868672066 shutdown_connections 2024-06-02T23:03:51.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.478+0000 7fc4d0467700 1 -- 172.21.15.52:0/868672066 wait complete. 2024-06-02T23:03:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 Processor -- start 2024-06-02T23:03:51.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 -- start start 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8102900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 0x7fc4c8100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8108200 0x7fc4c8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c810e020 con 0x7fc4c81058d0 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c810dea0 con 0x7fc4c8108200 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4d0467700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c810e1a0 con 0x7fc4c8102fa0 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4cda02700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 0x7fc4c8100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.479+0000 7fc4cda02700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 0x7fc4c8100f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37106/0 (socket says 172.21.15.52:37106) 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 -- 172.21.15.52:0/3341806354 learned_addr learned my addr 172.21.15.52:0/3341806354 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4ce203700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8102900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cea04700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8108200 0x7fc4c8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:51.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 -- 172.21.15.52:0/3341806354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8102fa0 msgr2=0x7fc4c8102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8102900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 -- 172.21.15.52:0/3341806354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8108200 msgr2=0x7fc4c8101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8108200 0x7fc4c8101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 -- 172.21.15.52:0/3341806354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c8101d40 con 0x7fc4c81058d0 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4ce203700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4cda02700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 0x7fc4c8100f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9a0 tx=0x7fc4b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:03:51.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800ca10 con 0x7fc4c81058d0 2024-06-02T23:03:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4b800cbb0 con 0x7fc4c81058d0 2024-06-02T23:03:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c81a27d0 con 0x7fc4c81058d0 2024-06-02T23:03:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8018680 con 0x7fc4c81058d0 2024-06-02T23:03:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.480+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c81a2ce0 con 0x7fc4c81058d0 2024-06-02T23:03:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.481+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4c8069180 con 0x7fc4c81058d0 2024-06-02T23:03:51.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.483+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc4b8010430 con 0x7fc4c81058d0 2024-06-02T23:03:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.483+0000 7fc4bf7fe700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4b40a2c20 0x7fc4b40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:03:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.483+0000 7fc4ce203700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4b40a2c20 0x7fc4b40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:03:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.483+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7fc4b80ce540 con 0x7fc4c81058d0 2024-06-02T23:03:51.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.484+0000 7fc4ce203700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4b40a2c20 0x7fc4b40a50e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400b670 tx=0x7fc4c401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:03:51.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.487+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc4b809c340 con 0x7fc4c81058d0 2024-06-02T23:03:51.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:03:51.708+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc4c80628d0 con 0x7fc4b40a2c20 2024-06-02T23:03:52.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:51 smithi123 ceph-mon[35977]: osdmap e40: 6 total, 6 up, 6 in 2024-06-02T23:03:52.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:52.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:52.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:52.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:51 smithi123 ceph-mon[35977]: osdmap e41: 6 total, 6 up, 6 in 2024-06-02T23:03:52.342 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:51 smithi052 ceph-mon[32340]: osdmap e40: 6 total, 6 up, 6 in 2024-06-02T23:03:52.342 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:52.342 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:52.342 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:52.342 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:51 smithi052 ceph-mon[32340]: osdmap e41: 6 total, 6 up, 6 in 2024-06-02T23:03:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:51 smithi174 ceph-mon[35994]: osdmap e40: 6 total, 6 up, 6 in 2024-06-02T23:03:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:03:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:03:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:03:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:51 smithi174 ceph-mon[35994]: osdmap e41: 6 total, 6 up, 6 in 2024-06-02T23:03:53.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:52 smithi123 ceph-mon[35977]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:53.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:52 smithi123 ceph-mon[35977]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:53.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:52 smithi123 ceph-mon[35977]: osdmap e42: 6 total, 6 up, 6 in 2024-06-02T23:03:53.307 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:52 smithi052 ceph-mon[32340]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:53.307 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:52 smithi052 ceph-mon[32340]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:53.308 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:52 smithi052 ceph-mon[32340]: osdmap e42: 6 total, 6 up, 6 in 2024-06-02T23:03:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:52 smithi174 ceph-mon[35994]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:03:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:52 smithi174 ceph-mon[35994]: pgmap v141: 1 pgs: 1 remapped+peering; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:53.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:52 smithi174 ceph-mon[35994]: osdmap e42: 6 total, 6 up, 6 in 2024-06-02T23:03:55.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:54 smithi052 ceph-mon[32340]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:55.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:54 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2646725267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3667d476-c6ce-4999-8954-4ade4fbe51a1"}]: dispatch 2024-06-02T23:03:55.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:54 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2646725267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3667d476-c6ce-4999-8954-4ade4fbe51a1"}]': finished 2024-06-02T23:03:55.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:54 smithi052 ceph-mon[32340]: osdmap e43: 7 total, 6 up, 7 in 2024-06-02T23:03:55.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:03:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:54 smithi123 ceph-mon[35977]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:54 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2646725267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3667d476-c6ce-4999-8954-4ade4fbe51a1"}]: dispatch 2024-06-02T23:03:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:54 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2646725267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3667d476-c6ce-4999-8954-4ade4fbe51a1"}]': finished 2024-06-02T23:03:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:54 smithi123 ceph-mon[35977]: osdmap e43: 7 total, 6 up, 7 in 2024-06-02T23:03:55.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:03:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:54 smithi174 ceph-mon[35994]: pgmap v144: 1 pgs: 1 remapped+peering; 577 KiB data, 202 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:54 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2646725267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3667d476-c6ce-4999-8954-4ade4fbe51a1"}]: dispatch 2024-06-02T23:03:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:54 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2646725267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3667d476-c6ce-4999-8954-4ade4fbe51a1"}]': finished 2024-06-02T23:03:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:54 smithi174 ceph-mon[35994]: osdmap e43: 7 total, 6 up, 7 in 2024-06-02T23:03:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:03:56.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:55 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1664656373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:56.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:55 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1664656373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:56.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:55 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1664656373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:03:57.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:56 smithi123 ceph-mon[35977]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:56 smithi174 ceph-mon[35994]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:56 smithi052 ceph-mon[32340]: pgmap v146: 1 pgs: 1 remapped+peering; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:03:59.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:03:58 smithi052 ceph-mon[32340]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:03:59.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:03:58 smithi123 ceph-mon[35977]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:03:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:03:58 smithi174 ceph-mon[35994]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:04:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:00 smithi123 ceph-mon[35977]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-02T23:04:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:00 smithi174 ceph-mon[35994]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-02T23:04:01.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:00 smithi052 ceph-mon[32340]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-02T23:04:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:01 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-02T23:04:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:01 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:01 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-02T23:04:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:01 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-06-02T23:04:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:02 smithi123 ceph-mon[35977]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-02T23:04:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:02 smithi123 ceph-mon[35977]: Deploying daemon osd.6 on smithi052 2024-06-02T23:04:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:02 smithi174 ceph-mon[35994]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-02T23:04:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:02 smithi174 ceph-mon[35994]: Deploying daemon osd.6 on smithi052 2024-06-02T23:04:03.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:02 smithi052 ceph-mon[32340]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-06-02T23:04:03.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:02 smithi052 ceph-mon[32340]: Deploying daemon osd.6 on smithi052 2024-06-02T23:04:05.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:04 smithi052 ceph-mon[32340]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:04:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:04 smithi123 ceph-mon[35977]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:04:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:04 smithi174 ceph-mon[35994]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:04:07.250 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:06 smithi052 ceph-mon[32340]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-06-02T23:04:07.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:06 smithi123 ceph-mon[35977]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-06-02T23:04:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:06 smithi174 ceph-mon[35994]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-06-02T23:04:08.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:08.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:08.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:08.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:08 smithi052 ceph-mon[32340]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-02T23:04:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:08 smithi123 ceph-mon[35977]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-02T23:04:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:08 smithi174 ceph-mon[35994]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-06-02T23:04:10.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:10 smithi052 ceph-mon[32340]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:10.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:10.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:10.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:10.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:10.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:10 smithi174 ceph-mon[35994]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:10 smithi123 ceph-mon[35977]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 6 on host 'smithi052' 2024-06-02T23:04:11.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.534+0000 7fc4bf7fe700 1 -- 172.21.15.52:0/3341806354 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc4c80628d0 con 0x7fc4b40a2c20 2024-06-02T23:04:11.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4b40a2c20 msgr2=0x7fc4b40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:11.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4b40a2c20 0x7fc4b40a50e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc4c400b670 tx=0x7fc4c401a040 comp rx=0 tx=0).stop 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 msgr2=0x7fc4c8100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 0x7fc4c8100f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800e9a0 tx=0x7fc4b800ecb0 comp rx=0 tx=0).stop 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 shutdown_connections 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4b40a2c20 0x7fc4b40a50e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4c8108200 0x7fc4c8101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4c8102fa0 0x7fc4c8102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 --2- 172.21.15.52:0/3341806354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4c81058d0 0x7fc4c8100f50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 >> 172.21.15.52:0/3341806354 conn(0x7fc4c80fc9b0 msgr2=0x7fc4c8103a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 shutdown_connections 2024-06-02T23:04:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:04:11.537+0000 7fc4d0467700 1 -- 172.21.15.52:0/3341806354 wait complete. 2024-06-02T23:04:12.202 DEBUG:teuthology.orchestra.run.smithi052:osd.6> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.6.service 2024-06-02T23:04:12.204 INFO:tasks.cephadm:Deploying osd.7 on smithi123 with /dev/vg_nvme/lv_2... 2024-06-02T23:04:12.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-02T23:04:12.456 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:04:12.782 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:12.782 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:12.782 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:12 smithi123 ceph-mon[35977]: from='osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T23:04:12.783 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:12 smithi123 ceph-mon[35977]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T23:04:12.783 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:12 smithi174 ceph-mon[35994]: from='osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T23:04:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:12 smithi174 ceph-mon[35994]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T23:04:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:12 smithi052 ceph-mon[32340]: from='osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T23:04:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:12 smithi052 ceph-mon[32340]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-06-02T23:04:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-mon[32340]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:13.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-mon[32340]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-02T23:04:13.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-mon[32340]: osdmap e44: 7 total, 6 up, 7 in 2024-06-02T23:04:13.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-mon[32340]: from='osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:04:13.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:13.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-mon[32340]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:04:13.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:04:13.540+0000 7f068d773700 -1 osd.6 0 waiting for initial osdmap 2024-06-02T23:04:13.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:04:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:04:13.552+0000 7f0689566700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:04:13.794 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:13 smithi123 ceph-mon[35977]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:13.795 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:13 smithi123 ceph-mon[35977]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-02T23:04:13.795 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:13 smithi123 ceph-mon[35977]: osdmap e44: 7 total, 6 up, 7 in 2024-06-02T23:04:13.795 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:13 smithi123 ceph-mon[35977]: from='osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:04:13.795 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:13.795 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:13 smithi123 ceph-mon[35977]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:04:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:13 smithi174 ceph-mon[35994]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:13 smithi174 ceph-mon[35994]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-06-02T23:04:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:13 smithi174 ceph-mon[35994]: osdmap e44: 7 total, 6 up, 7 in 2024-06-02T23:04:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:13 smithi174 ceph-mon[35994]: from='osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:04:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:13 smithi174 ceph-mon[35994]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:04:14.690 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-02T23:04:14.691 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-02T23:04:14.691 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-02T23:04:14.691 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-02T23:04:14.691 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012231 s, 857 MB/s 2024-06-02T23:04:14.691 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-02T23:04:14.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:14 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:04:14.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:14 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:04:14.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:14 smithi174 ceph-mon[35994]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:04:14.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:14 smithi174 ceph-mon[35994]: osdmap e45: 7 total, 6 up, 7 in 2024-06-02T23:04:14.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:14.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:14 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:04:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:14 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:04:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:14 smithi052 ceph-mon[32340]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:04:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:14 smithi052 ceph-mon[32340]: osdmap e45: 7 total, 6 up, 7 in 2024-06-02T23:04:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:14.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:14.992 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:14 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:04:14.992 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:14 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:04:14.992 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:14 smithi123 ceph-mon[35977]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:04:14.993 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:14 smithi123 ceph-mon[35977]: osdmap e45: 7 total, 6 up, 7 in 2024-06-02T23:04:14.993 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:14.993 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:15.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-06-02T23:04:15.590 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:04:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] boot 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: osdmap e46: 7 total, 7 up, 7 in 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:15.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:15.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.712 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] boot 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: osdmap e46: 7 total, 7 up, 7 in 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:15.713 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 185 MiB used, 536 GiB / 536 GiB avail 2024-06-02T23:04:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: osd.6 [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] boot 2024-06-02T23:04:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: osdmap e46: 7 total, 7 up, 7 in 2024-06-02T23:04:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:15.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:16.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:16 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi052 2024-06-02T23:04:16.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:16 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 5697M 2024-06-02T23:04:16.726 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:16 smithi123 ceph-mon[35977]: osdmap e47: 7 total, 7 up, 7 in 2024-06-02T23:04:16.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:16 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi052 2024-06-02T23:04:16.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:16 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 5697M 2024-06-02T23:04:16.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:16 smithi174 ceph-mon[35994]: osdmap e47: 7 total, 7 up, 7 in 2024-06-02T23:04:16.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:16 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi052 2024-06-02T23:04:16.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:16 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 5697M 2024-06-02T23:04:16.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:16 smithi052 ceph-mon[32340]: osdmap e47: 7 total, 7 up, 7 in 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 -- 172.21.15.123:0/3841709222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac1036e0 msgr2=0x7f05ac103b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 --2- 172.21.15.123:0/3841709222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac1036e0 0x7f05ac103b40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f05940099c0 tx=0x7f0594009cd0 comp rx=0 tx=0).stop 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 -- 172.21.15.123:0/3841709222 shutdown_connections 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 --2- 172.21.15.123:0/3841709222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac104080 0x7f05ac10a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 --2- 172.21.15.123:0/3841709222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05ac1024e0 0x7f05ac102900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 --2- 172.21.15.123:0/3841709222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac1036e0 0x7f05ac103b40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 -- 172.21.15.123:0/3841709222 >> 172.21.15.123:0/3841709222 conn(0x7f05ac0fda80 msgr2=0x7f05ac0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:04:17.243 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 -- 172.21.15.123:0/3841709222 shutdown_connections 2024-06-02T23:04:17.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.242+0000 7f05b11d7700 1 -- 172.21.15.123:0/3841709222 wait complete. 2024-06-02T23:04:17.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.243+0000 7f05b11d7700 1 Processor -- start 2024-06-02T23:04:17.244 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.243+0000 7f05b11d7700 1 -- start start 2024-06-02T23:04:17.245 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.243+0000 7f05b11d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 0x7f05ac076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05b11d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05ac1036e0 0x7f05ac074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05b11d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac104080 0x7f05ac074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05b11d7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10e000 con 0x7f05ac1024e0 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05b11d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10de80 con 0x7f05ac104080 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05b11d7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10e180 con 0x7f05ac1036e0 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 0x7f05ac076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 0x7f05ac076230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.123:37212/0 (socket says 172.21.15.123:37212) 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 -- 172.21.15.123:0/2485243935 learned_addr learned my addr 172.21.15.123:0/2485243935 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05ab59e700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac104080 0x7f05ac074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aa59c700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05ac1036e0 0x7f05ac074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 -- 172.21.15.123:0/2485243935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05ac1036e0 msgr2=0x7f05ac074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05ac1036e0 0x7f05ac074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:17.246 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 -- 172.21.15.123:0/2485243935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac104080 msgr2=0x7f05ac074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac104080 0x7f05ac074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 -- 172.21.15.123:0/2485243935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac075670 con 0x7f05ac1024e0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05ab59e700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac104080 0x7f05ac074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05aad9d700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 0x7f05ac076230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b500 tx=0x7f059c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.244+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c015020 con 0x7f05ac1024e0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.245+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac1a2860 con 0x7f05ac1024e0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.245+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac1a2df0 con 0x7f05ac1024e0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.245+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f059c011ce0 con 0x7f05ac1024e0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.245+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059c01ac30 con 0x7f05ac1024e0 2024-06-02T23:04:17.247 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.245+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac069180 con 0x7f05ac1024e0 2024-06-02T23:04:17.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.247+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f059c018020 con 0x7f05ac1024e0 2024-06-02T23:04:17.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.248+0000 7f05a3fff700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05980a2c20 0x7f05980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:17.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.248+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f059c0ce020 con 0x7f05ac1024e0 2024-06-02T23:04:17.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.248+0000 7f05aa59c700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05980a2c20 0x7f05980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:17.248 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.248+0000 7f05aa59c700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05980a2c20 0x7f05980a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0594000c00 tx=0x7f0594005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:04:17.251 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.252+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f059c01f020 con 0x7f05ac1024e0 2024-06-02T23:04:17.454 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:17.454+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f05ac0628d0 con 0x7f05980a2c20 2024-06-02T23:04:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:17 smithi123 ceph-mon[35977]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:04:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:04:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:17 smithi174 ceph-mon[35994]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:04:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:04:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:17 smithi052 ceph-mon[32340]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:04:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:04:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:18.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:18 smithi123 ceph-mon[35977]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:04:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:18 smithi174 ceph-mon[35994]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:04:18.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:18 smithi052 ceph-mon[32340]: from='client.14385 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:04:19.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:19 smithi123 ceph-mon[35977]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:19.850 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:19 smithi174 ceph-mon[35994]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:19 smithi052 ceph-mon[32340]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:20.662 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:20 smithi123 ceph-mon[35977]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:20.663 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:20 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/4076096303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]: dispatch 2024-06-02T23:04:20.663 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:20 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]: dispatch 2024-06-02T23:04:20.663 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:20 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]': finished 2024-06-02T23:04:20.663 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:20 smithi123 ceph-mon[35977]: osdmap e48: 8 total, 7 up, 8 in 2024-06-02T23:04:20.663 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:20 smithi174 ceph-mon[35994]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:20 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/4076096303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]: dispatch 2024-06-02T23:04:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:20 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]: dispatch 2024-06-02T23:04:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:20 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]': finished 2024-06-02T23:04:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:20 smithi174 ceph-mon[35994]: osdmap e48: 8 total, 7 up, 8 in 2024-06-02T23:04:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:20 smithi052 ceph-mon[32340]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:20 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/4076096303' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]: dispatch 2024-06-02T23:04:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:20 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]: dispatch 2024-06-02T23:04:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:20 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0ef49c6a-2339-4f91-9474-0c747c53a20d"}]': finished 2024-06-02T23:04:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:20 smithi052 ceph-mon[32340]: osdmap e48: 8 total, 7 up, 8 in 2024-06-02T23:04:20.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:21 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/1336457900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:04:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:21 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/1336457900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:04:22.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:21 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/1336457900' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:04:22.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:22 smithi174 ceph-mon[35994]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:22.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:22 smithi052 ceph-mon[32340]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:22 smithi123 ceph-mon[35977]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:25.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:24 smithi174 ceph-mon[35994]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:25.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:24 smithi052 ceph-mon[32340]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:25.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:24 smithi123 ceph-mon[35977]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:26 smithi123 ceph-mon[35977]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:27.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:26 smithi174 ceph-mon[35994]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:27.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:26 smithi052 ceph-mon[32340]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:28 smithi174 ceph-mon[35994]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-02T23:04:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:28 smithi174 ceph-mon[35994]: Deploying daemon osd.7 on smithi123 2024-06-02T23:04:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:28 smithi052 ceph-mon[32340]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-02T23:04:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:29.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:28 smithi052 ceph-mon[32340]: Deploying daemon osd.7 on smithi123 2024-06-02T23:04:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:28 smithi123 ceph-mon[35977]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-06-02T23:04:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:28 smithi123 ceph-mon[35977]: Deploying daemon osd.7 on smithi123 2024-06-02T23:04:31.005 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:30 smithi123 ceph-mon[35977]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:30 smithi174 ceph-mon[35994]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:31.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:30 smithi052 ceph-mon[32340]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:33.072 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:32 smithi123 ceph-mon[35977]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:32 smithi174 ceph-mon[35994]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:33.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:32 smithi052 ceph-mon[32340]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:34 smithi123 ceph-mon[35977]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:34 smithi174 ceph-mon[35994]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:34 smithi052 ceph-mon[32340]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:36 smithi174 ceph-mon[35994]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:37.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:36 smithi052 ceph-mon[32340]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:37.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:37.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:36 smithi123 ceph-mon[35977]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:37.682 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 7 on host 'smithi123' 2024-06-02T23:04:37.683 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.682+0000 7f05a3fff700 1 -- 172.21.15.123:0/2485243935 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f05ac0628d0 con 0x7f05980a2c20 2024-06-02T23:04:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05980a2c20 msgr2=0x7f05980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05980a2c20 0x7f05980a50e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0594000c00 tx=0x7f0594005fb0 comp rx=0 tx=0).stop 2024-06-02T23:04:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 msgr2=0x7f05ac076230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 0x7f05ac076230 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f059c00b500 tx=0x7f059c00b810 comp rx=0 tx=0).stop 2024-06-02T23:04:37.684 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 shutdown_connections 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f05980a2c20 0x7f05980a50e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05ac104080 0x7f05ac074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f05ac1024e0 0x7f05ac076230 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 --2- 172.21.15.123:0/2485243935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f05ac1036e0 0x7f05ac074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 >> 172.21.15.123:0/2485243935 conn(0x7f05ac0fda80 msgr2=0x7f05ac106a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 shutdown_connections 2024-06-02T23:04:37.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:04:37.684+0000 7f05b11d7700 1 -- 172.21.15.123:0/2485243935 wait complete. 2024-06-02T23:04:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:37 smithi123 ceph-mon[35977]: from='osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T23:04:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:37 smithi123 ceph-mon[35977]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T23:04:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:37 smithi174 ceph-mon[35994]: from='osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T23:04:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:37 smithi174 ceph-mon[35994]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T23:04:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:37 smithi052 ceph-mon[32340]: from='osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T23:04:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:37 smithi052 ceph-mon[32340]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-06-02T23:04:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:38.312 DEBUG:teuthology.orchestra.run.smithi123:osd.7> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.7.service 2024-06-02T23:04:38.314 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-06-02T23:04:38.315 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-06-02T23:04:38.560 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:04:38.964 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:38 smithi174 ceph-mon[35994]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:38.964 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:38 smithi174 ceph-mon[35994]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-02T23:04:38.964 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:38 smithi174 ceph-mon[35994]: osdmap e49: 8 total, 7 up, 8 in 2024-06-02T23:04:38.964 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:38 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:38.965 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:38 smithi174 ceph-mon[35994]: from='osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:04:38.965 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:38 smithi174 ceph-mon[35994]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:04:39.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:38 smithi123 ceph-mon[35977]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:39.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:38 smithi123 ceph-mon[35977]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-02T23:04:39.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:38 smithi123 ceph-mon[35977]: osdmap e49: 8 total, 7 up, 8 in 2024-06-02T23:04:39.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:38 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:39.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:38 smithi123 ceph-mon[35977]: from='osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:04:39.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:38 smithi123 ceph-mon[35977]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:04:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:38 smithi052 ceph-mon[32340]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:38 smithi052 ceph-mon[32340]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-06-02T23:04:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:38 smithi052 ceph-mon[32340]: osdmap e49: 8 total, 7 up, 8 in 2024-06-02T23:04:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:38 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:38 smithi052 ceph-mon[32340]: from='osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:04:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:38 smithi052 ceph-mon[32340]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:04:39.438 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:04:39.214+0000 7ffb00db4700 -1 osd.7 0 waiting for initial osdmap 2024-06-02T23:04:39.438 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:04:39.224+0000 7ffaf8b9f700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:04:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-mon[35977]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:04:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-mon[35977]: osdmap e50: 8 total, 7 up, 8 in 2024-06-02T23:04:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:39 smithi123 ceph-mon[35977]: from='osd.7 ' entity='osd.7' 2024-06-02T23:04:40.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:39 smithi174 ceph-mon[35994]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:04:40.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:39 smithi174 ceph-mon[35994]: osdmap e50: 8 total, 7 up, 8 in 2024-06-02T23:04:40.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.021 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:39 smithi174 ceph-mon[35994]: from='osd.7 ' entity='osd.7' 2024-06-02T23:04:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:39 smithi052 ceph-mon[32340]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:04:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:39 smithi052 ceph-mon[32340]: osdmap e50: 8 total, 7 up, 8 in 2024-06-02T23:04:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:39 smithi052 ceph-mon[32340]: from='osd.7 ' entity='osd.7' 2024-06-02T23:04:40.642 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-06-02T23:04:40.642 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-06-02T23:04:40.642 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-06-02T23:04:40.642 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-06-02T23:04:40.642 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.015104 s, 694 MB/s 2024-06-02T23:04:40.643 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] boot 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: osdmap e51: 8 total, 8 up, 8 in 2024-06-02T23:04:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:40 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] boot 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: osdmap e51: 8 total, 8 up, 8 in 2024-06-02T23:04:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:40 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:04:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:04:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 212 MiB used, 626 GiB / 626 GiB avail 2024-06-02T23:04:41.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:41.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: osd.7 [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] boot 2024-06-02T23:04:41.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: osdmap e51: 8 total, 8 up, 8 in 2024-06-02T23:04:41.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:40 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-06-02T23:04:41.360 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-06-02T23:04:41.597 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:04:42.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: osdmap e52: 8 total, 8 up, 8 in 2024-06-02T23:04:42.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi123 2024-06-02T23:04:42.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.037 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 5696M 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:42.038 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:41 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: osdmap e52: 8 total, 8 up, 8 in 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi123 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 5696M 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:41 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: osdmap e52: 8 total, 8 up, 8 in 2024-06-02T23:04:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi123 2024-06-02T23:04:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:04:42.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 5696M 2024-06-02T23:04:42.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:42.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:04:42.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:41 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:42.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:42 smithi174 ceph-mon[35994]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:42.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:42 smithi174 ceph-mon[35994]: osdmap e53: 8 total, 8 up, 8 in 2024-06-02T23:04:43.149 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/90732162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 msgr2=0x7fd3f4100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/90732162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f4100be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0009990 tx=0x7fd3f0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/90732162 shutdown_connections 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/90732162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4104140 0x7fd3f410a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/90732162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f4100be0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/90732162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 0x7fd3f4103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/90732162 >> 172.21.15.174:0/90732162 conn(0x7fd3f40fc120 msgr2=0x7fd3f40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/90732162 shutdown_connections 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.147+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/90732162 wait complete. 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 Processor -- start 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 -- start start 2024-06-02T23:04:43.150 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 0x7fd3f419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4104140 0x7fd3f41a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410dfc0 con 0x7fd3f4104140 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410de40 con 0x7fd3f41037a0 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3fb4a8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3f410e140 con 0x7fd3f41007c0 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3f9244700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3f9244700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f419a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:53052/0 (socket says 172.21.15.174:53052) 2024-06-02T23:04:43.151 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3f9244700 1 -- 172.21.15.174:0/4177733442 learned_addr learned my addr 172.21.15.174:0/4177733442 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.148+0000 7fd3f8a43700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 0x7fd3f419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9a45700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4104140 0x7fd3f41a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9244700 1 -- 172.21.15.174:0/4177733442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 msgr2=0x7fd3f419ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9244700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 0x7fd3f419ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9244700 1 -- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4104140 msgr2=0x7fd3f41a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9244700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4104140 0x7fd3f41a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9244700 1 -- 172.21.15.174:0/4177733442 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3f41a24d0 con 0x7fd3f41007c0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f8a43700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 0x7fd3f419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3f9244700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f419a910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00057c0 tx=0x7fd3f00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f0027070 con 0x7fd3f41007c0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3f0022470 con 0x7fd3f41007c0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3f41a2760 con 0x7fd3f41007c0 2024-06-02T23:04:43.152 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f000f6c0 con 0x7fd3f41007c0 2024-06-02T23:04:43.153 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.149+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3f41a2c10 con 0x7fd3f41007c0 2024-06-02T23:04:43.153 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:42 smithi052 ceph-mon[32340]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:43.153 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:42 smithi052 ceph-mon[32340]: osdmap e53: 8 total, 8 up, 8 in 2024-06-02T23:04:43.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.150+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3f4069180 con 0x7fd3f41007c0 2024-06-02T23:04:43.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.151+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd3f002a070 con 0x7fd3f41007c0 2024-06-02T23:04:43.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.152+0000 7fd3e67fc700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3e00a2ca0 0x7fd3e00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:04:43.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.152+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5426+0+0 (secure 0 0 0) 0x7fd3f00cf2f0 con 0x7fd3f41007c0 2024-06-02T23:04:43.154 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.152+0000 7fd3f8a43700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3e00a2ca0 0x7fd3e00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:04:43.155 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.153+0000 7fd3f8a43700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3e00a2ca0 0x7fd3e00a5160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005950 tx=0x7fd3e80058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:04:43.156 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.156+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3f005e850 con 0x7fd3f41007c0 2024-06-02T23:04:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:42 smithi123 ceph-mon[35977]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:42 smithi123 ceph-mon[35977]: osdmap e53: 8 total, 8 up, 8 in 2024-06-02T23:04:43.358 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:04:43.358+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fd3f410a660 con 0x7fd3e00a2ca0 2024-06-02T23:04:44.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:43 smithi174 ceph-mon[35994]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:04:44.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:04:44.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:04:44.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:44.098 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:43 smithi174 ceph-mon[35994]: osdmap e54: 8 total, 8 up, 8 in 2024-06-02T23:04:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:43 smithi052 ceph-mon[32340]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:04:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:04:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:04:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:44.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:43 smithi052 ceph-mon[32340]: osdmap e54: 8 total, 8 up, 8 in 2024-06-02T23:04:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:43 smithi123 ceph-mon[35977]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:04:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:04:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:04:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:43 smithi123 ceph-mon[35977]: osdmap e54: 8 total, 8 up, 8 in 2024-06-02T23:04:45.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:45 smithi123 ceph-mon[35977]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:45.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:45 smithi123 ceph-mon[35977]: osdmap e55: 8 total, 8 up, 8 in 2024-06-02T23:04:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:45 smithi174 ceph-mon[35994]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:45 smithi174 ceph-mon[35994]: osdmap e55: 8 total, 8 up, 8 in 2024-06-02T23:04:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:45 smithi052 ceph-mon[32340]: pgmap v181: 1 pgs: 1 unknown; 577 KiB data, 243 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:45 smithi052 ceph-mon[32340]: osdmap e55: 8 total, 8 up, 8 in 2024-06-02T23:04:46.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:46 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/2838241586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]: dispatch 2024-06-02T23:04:46.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:46 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]: dispatch 2024-06-02T23:04:46.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:46 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]': finished 2024-06-02T23:04:46.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:46 smithi174 ceph-mon[35994]: osdmap e56: 9 total, 8 up, 9 in 2024-06-02T23:04:46.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:04:46.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:46 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/2838241586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]: dispatch 2024-06-02T23:04:46.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:46 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]: dispatch 2024-06-02T23:04:46.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:46 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]': finished 2024-06-02T23:04:46.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:46 smithi123 ceph-mon[35977]: osdmap e56: 9 total, 8 up, 9 in 2024-06-02T23:04:46.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:04:46.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:46 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/2838241586' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]: dispatch 2024-06-02T23:04:46.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:46 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]: dispatch 2024-06-02T23:04:46.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:46 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "27a74c0f-68e7-4c68-aaf3-69859a1b47e3"}]': finished 2024-06-02T23:04:46.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:46 smithi052 ceph-mon[32340]: osdmap e56: 9 total, 8 up, 9 in 2024-06-02T23:04:46.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:04:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:47 smithi123 ceph-mon[35977]: pgmap v183: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:47 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/3416659033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:04:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:47 smithi174 ceph-mon[35994]: pgmap v183: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:47 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/3416659033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:04:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:47 smithi052 ceph-mon[32340]: pgmap v183: 1 pgs: 1 unknown; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:47 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/3416659033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:04:49.686 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:49 smithi174 ceph-mon[35994]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:49 smithi123 ceph-mon[35977]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:49.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:49 smithi052 ceph-mon[32340]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:51 smithi174 ceph-mon[35994]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:51 smithi123 ceph-mon[35977]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:51 smithi052 ceph-mon[32340]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:53 smithi174 ceph-mon[35994]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-02T23:04:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:53 smithi123 ceph-mon[35977]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-02T23:04:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:53 smithi052 ceph-mon[32340]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-06-02T23:04:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:04:54.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:54 smithi174 ceph-mon[35994]: Deploying daemon osd.8 on smithi174 2024-06-02T23:04:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:54 smithi123 ceph-mon[35977]: Deploying daemon osd.8 on smithi174 2024-06-02T23:04:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:54 smithi052 ceph-mon[32340]: Deploying daemon osd.8 on smithi174 2024-06-02T23:04:55.590 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:55 smithi174 ceph-mon[35994]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:55.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:55 smithi123 ceph-mon[35977]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:55 smithi052 ceph-mon[32340]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:57 smithi174 ceph-mon[35994]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:57 smithi123 ceph-mon[35977]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:57 smithi052 ceph-mon[32340]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:59.477 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:59 smithi174 ceph-mon[35994]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:59.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:59.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:59.478 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:04:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:59 smithi123 ceph-mon[35977]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:04:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:04:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:59 smithi052 ceph-mon[32340]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:04:59.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:59.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:04:59.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:04:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:01 smithi123 ceph-mon[35977]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:01 smithi174 ceph-mon[35994]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:01 smithi052 ceph-mon[32340]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:02.627 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.628+0000 7fd3e67fc700 1 -- 172.21.15.174:0/4177733442 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd3f410a660 con 0x7fd3e00a2ca0 2024-06-02T23:05:02.628 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3e00a2ca0 msgr2=0x7fd3e00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3e00a2ca0 0x7fd3e00a5160 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd3e8005950 tx=0x7fd3e80058e0 comp rx=0 tx=0).stop 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 msgr2=0x7fd3f419a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f419a910 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00057c0 tx=0x7fd3f00057f0 comp rx=0 tx=0).stop 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 shutdown_connections 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3e00a2ca0 0x7fd3e00a5160 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:02.630 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3f4104140 0x7fd3f41a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:02.631 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3f41007c0 0x7fd3f419a910 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:02.631 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 --2- 172.21.15.174:0/4177733442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3f41037a0 0x7fd3f419ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:02.631 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 >> 172.21.15.174:0/4177733442 conn(0x7fd3f40fc120 msgr2=0x7fd3f4108f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:05:02.631 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 shutdown_connections 2024-06-02T23:05:02.631 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:05:02.630+0000 7fd3fb4a8700 1 -- 172.21.15.174:0/4177733442 wait complete. 2024-06-02T23:05:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T23:05:03.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:02 smithi174 ceph-mon[35994]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T23:05:03.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:03.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T23:05:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:02 smithi052 ceph-mon[32340]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:03.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:03.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T23:05:03.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:02 smithi123 ceph-mon[35977]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-06-02T23:05:03.299 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.8.service 2024-06-02T23:05:03.301 INFO:tasks.cephadm:Deploying osd.9 on smithi052 with /dev/vg_nvme/lv_1... 2024-06-02T23:05:03.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-02T23:05:03.561 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:05:04.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-mon[35994]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-02T23:05:04.751 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-mon[35994]: osdmap e57: 9 total, 8 up, 9 in 2024-06-02T23:05:04.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:04.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-mon[35994]: from='osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:05:04.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-mon[35994]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:05:04.752 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-mon[35994]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:04.752 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:05:04.638+0000 7fe13cd12700 -1 osd.8 0 waiting for initial osdmap 2024-06-02T23:05:04.752 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:05:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:05:04.651+0000 7fe138b05700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:05:04.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:04 smithi052 ceph-mon[32340]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-02T23:05:04.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:04 smithi052 ceph-mon[32340]: osdmap e57: 9 total, 8 up, 9 in 2024-06-02T23:05:04.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:04.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:04 smithi052 ceph-mon[32340]: from='osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:05:04.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:04 smithi052 ceph-mon[32340]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:05:04.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:04 smithi052 ceph-mon[32340]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:05.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:04 smithi123 ceph-mon[35977]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-06-02T23:05:05.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:04 smithi123 ceph-mon[35977]: osdmap e57: 9 total, 8 up, 9 in 2024-06-02T23:05:05.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:05.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:04 smithi123 ceph-mon[35977]: from='osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:05:05.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:04 smithi123 ceph-mon[35977]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:05:05.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:04 smithi123 ceph-mon[35977]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 244 MiB used, 715 GiB / 715 GiB avail 2024-06-02T23:05:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-02T23:05:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-02T23:05:05.632 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10+0 records in 2024-06-02T23:05:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:10+0 records out 2024-06-02T23:05:05.632 INFO:teuthology.orchestra.run.smithi052.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00884866 s, 1.2 GB/s 2024-06-02T23:05:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:--> Zapping successful for: 2024-06-02T23:05:05.891 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:05 smithi174 ceph-mon[35994]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:05:05.891 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:05 smithi174 ceph-mon[35994]: osdmap e58: 9 total, 8 up, 9 in 2024-06-02T23:05:05.891 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:05.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:05 smithi052 ceph-mon[32340]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:05:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:05 smithi052 ceph-mon[32340]: osdmap e58: 9 total, 8 up, 9 in 2024-06-02T23:05:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:06.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:05 smithi123 ceph-mon[35977]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:05:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:05 smithi123 ceph-mon[35977]: osdmap e58: 9 total, 8 up, 9 in 2024-06-02T23:05:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:06.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi052:vg_nvme/lv_1 2024-06-02T23:05:06.715 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:05:06.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:05:06.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:05:06.837 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] boot 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: osdmap e59: 9 total, 9 up, 9 in 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi174 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 5696M 2024-06-02T23:05:06.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:06.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:06.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:06.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:06.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:06 smithi052 ceph-mon[32340]: osdmap e60: 9 total, 9 up, 9 in 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] boot 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: osdmap e59: 9 total, 9 up, 9 in 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi174 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 5696M 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:06 smithi123 ceph-mon[35977]: osdmap e60: 9 total, 9 up, 9 in 2024-06-02T23:05:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:05:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:05:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: osd.8 [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] boot 2024-06-02T23:05:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: osdmap e59: 9 total, 9 up, 9 in 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi174 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 5696M 2024-06-02T23:05:07.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:07.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:07.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:07.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:06 smithi174 ceph-mon[35994]: osdmap e60: 9 total, 9 up, 9 in 2024-06-02T23:05:08.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 -- 172.21.15.52:0/1757978201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec102500 msgr2=0x7fceec102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:08.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 --2- 172.21.15.52:0/1757978201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec102500 0x7fceec102920 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fcedc0099c0 tx=0x7fcedc009cd0 comp rx=0 tx=0).stop 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 -- 172.21.15.52:0/1757978201 shutdown_connections 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 --2- 172.21.15.52:0/1757978201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fceec1040a0 0x7fceec10a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 --2- 172.21.15.52:0/1757978201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec102500 0x7fceec102920 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 --2- 172.21.15.52:0/1757978201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceec103700 0x7fceec103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 -- 172.21.15.52:0/1757978201 >> 172.21.15.52:0/1757978201 conn(0x7fceec0fdaa0 msgr2=0x7fceec0ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 -- 172.21.15.52:0/1757978201 shutdown_connections 2024-06-02T23:05:08.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.258+0000 7fcef1e94700 1 -- 172.21.15.52:0/1757978201 wait complete. 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 Processor -- start 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 -- start start 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fceec102500 0x7fceec076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceec103700 0x7fceec074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 0x7fceec074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fceec10dfd0 con 0x7fceec1040a0 2024-06-02T23:05:08.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fceec10de50 con 0x7fceec102500 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fcef1e94700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fceec10e150 con 0x7fceec103700 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fceebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 0x7fceec074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fceebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 0x7fceec074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33254/0 (socket says 172.21.15.52:33254) 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fceebfff700 1 -- 172.21.15.52:0/3150872283 learned_addr learned my addr 172.21.15.52:0/3150872283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fceeaffd700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceec103700 0x7fceec074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.259+0000 7fceeb7fe700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fceec102500 0x7fceec076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fceebfff700 1 -- 172.21.15.52:0/3150872283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceec103700 msgr2=0x7fceec074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fceebfff700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceec103700 0x7fceec074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fceebfff700 1 -- 172.21.15.52:0/3150872283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fceec102500 msgr2=0x7fceec076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fceebfff700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fceec102500 0x7fceec076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fceebfff700 1 -- 172.21.15.52:0/3150872283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fceec075670 con 0x7fceec1040a0 2024-06-02T23:05:08.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fceebfff700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 0x7fceec074dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcee4007d70 tx=0x7fcee400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:05:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee4016020 con 0x7fceec1040a0 2024-06-02T23:05:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fceec1a2770 con 0x7fceec1040a0 2024-06-02T23:05:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee400ac90 con 0x7fceec1040a0 2024-06-02T23:05:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee40115e0 con 0x7fceec1040a0 2024-06-02T23:05:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.260+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fceec1a2c60 con 0x7fceec1040a0 2024-06-02T23:05:08.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.261+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fceec069180 con 0x7fceec1040a0 2024-06-02T23:05:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.262+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcee4019020 con 0x7fceec1040a0 2024-06-02T23:05:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.263+0000 7fcee8ff9700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fced40a2c20 0x7fced40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.263+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5858+0+0 (secure 0 0 0) 0x7fcee40cea70 con 0x7fceec1040a0 2024-06-02T23:05:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.263+0000 7fceeb7fe700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fced40a2c20 0x7fced40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:08.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.264+0000 7fceeb7fe700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fced40a2c20 0x7fced40a50e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcedc000c00 tx=0x7fcedc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:05:08.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.267+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcee4020020 con 0x7fceec1040a0 2024-06-02T23:05:08.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:08.483+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fceec0628d0 con 0x7fced40a2c20 2024-06-02T23:05:08.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:08 smithi052 ceph-mon[32340]: osdmap e61: 9 total, 9 up, 9 in 2024-06-02T23:05:08.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:08 smithi052 ceph-mon[32340]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:08.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:08 smithi052 ceph-mon[32340]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:05:08.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:05:08.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:05:08.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:08 smithi123 ceph-mon[35977]: osdmap e61: 9 total, 9 up, 9 in 2024-06-02T23:05:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:08 smithi123 ceph-mon[35977]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:08 smithi123 ceph-mon[35977]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:05:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:05:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:05:09.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:08 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:08 smithi174 ceph-mon[35994]: osdmap e61: 9 total, 9 up, 9 in 2024-06-02T23:05:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:08 smithi174 ceph-mon[35994]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:08 smithi174 ceph-mon[35994]: from='client.14421 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi052:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:05:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:05:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:05:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:09 smithi123 ceph-mon[35977]: osdmap e62: 9 total, 9 up, 9 in 2024-06-02T23:05:10.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:09 smithi174 ceph-mon[35994]: osdmap e62: 9 total, 9 up, 9 in 2024-06-02T23:05:10.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:09 smithi052 ceph-mon[32340]: osdmap e62: 9 total, 9 up, 9 in 2024-06-02T23:05:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:10 smithi123 ceph-mon[35977]: osdmap e63: 9 total, 9 up, 9 in 2024-06-02T23:05:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:10 smithi123 ceph-mon[35977]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:10 smithi174 ceph-mon[35994]: osdmap e63: 9 total, 9 up, 9 in 2024-06-02T23:05:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:10 smithi174 ceph-mon[35994]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:10 smithi052 ceph-mon[32340]: osdmap e63: 9 total, 9 up, 9 in 2024-06-02T23:05:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:10 smithi052 ceph-mon[32340]: pgmap v203: 1 pgs: 1 remapped+peering; 577 KiB data, 271 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2622464420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "482dc129-e665-4a35-add6-b40d6359d13a"}]: dispatch 2024-06-02T23:05:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2622464420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "482dc129-e665-4a35-add6-b40d6359d13a"}]': finished 2024-06-02T23:05:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:11 smithi052 ceph-mon[32340]: osdmap e64: 10 total, 9 up, 10 in 2024-06-02T23:05:11.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:12.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2622464420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "482dc129-e665-4a35-add6-b40d6359d13a"}]: dispatch 2024-06-02T23:05:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2622464420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "482dc129-e665-4a35-add6-b40d6359d13a"}]': finished 2024-06-02T23:05:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:11 smithi123 ceph-mon[35977]: osdmap e64: 10 total, 9 up, 10 in 2024-06-02T23:05:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2622464420' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "482dc129-e665-4a35-add6-b40d6359d13a"}]: dispatch 2024-06-02T23:05:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2622464420' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "482dc129-e665-4a35-add6-b40d6359d13a"}]': finished 2024-06-02T23:05:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:11 smithi174 ceph-mon[35994]: osdmap e64: 10 total, 9 up, 10 in 2024-06-02T23:05:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:12 smithi123 ceph-mon[35977]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-02T23:05:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3470941788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:05:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:12 smithi174 ceph-mon[35994]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-02T23:05:13.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3470941788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:05:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:12 smithi052 ceph-mon[32340]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 141 KiB/s, 0 objects/s recovering 2024-06-02T23:05:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3470941788' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:05:15.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:14 smithi174 ceph-mon[35994]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:05:15.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:14 smithi052 ceph-mon[32340]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:05:15.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:14 smithi123 ceph-mon[35977]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:05:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:16 smithi174 ceph-mon[35994]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-02T23:05:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:16 smithi052 ceph-mon[32340]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-02T23:05:17.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:16 smithi123 ceph-mon[35977]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-06-02T23:05:18.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:18 smithi052 ceph-mon[32340]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-02T23:05:19.059 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:18 smithi123 ceph-mon[35977]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-02T23:05:19.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:18 smithi174 ceph-mon[35994]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-06-02T23:05:19.993 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:19 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-02T23:05:19.993 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:19 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:19.993 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:19 smithi052 ceph-mon[32340]: Deploying daemon osd.9 on smithi052 2024-06-02T23:05:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:19 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-02T23:05:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:19 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:19 smithi174 ceph-mon[35994]: Deploying daemon osd.9 on smithi052 2024-06-02T23:05:20.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:19 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-06-02T23:05:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:19 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:19 smithi123 ceph-mon[35977]: Deploying daemon osd.9 on smithi052 2024-06-02T23:05:20.939 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:20 smithi052 ceph-mon[32340]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:05:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:20 smithi174 ceph-mon[35994]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:05:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:20 smithi123 ceph-mon[35977]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-06-02T23:05:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:22 smithi174 ceph-mon[35994]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:23.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:22 smithi052 ceph-mon[32340]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:23.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:22 smithi123 ceph-mon[35977]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:25.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:24 smithi174 ceph-mon[35994]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:25.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:24 smithi052 ceph-mon[32340]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:25.261 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:24 smithi123 ceph-mon[35977]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:26.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:26.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:26.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:27.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:26 smithi052 ceph-mon[32340]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:26 smithi123 ceph-mon[35977]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:27.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:26 smithi174 ceph-mon[35994]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:28.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.815+0000 7fcee8ff9700 1 -- 172.21.15.52:0/3150872283 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fceec0628d0 con 0x7fced40a2c20 2024-06-02T23:05:28.816 INFO:teuthology.orchestra.run.smithi052.stdout:Created osd(s) 9 on host 'smithi052' 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fced40a2c20 msgr2=0x7fced40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fced40a2c20 0x7fced40a50e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fcedc000c00 tx=0x7fcedc005fb0 comp rx=0 tx=0).stop 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 msgr2=0x7fceec074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 0x7fceec074dc0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fcee4007d70 tx=0x7fcee400a340 comp rx=0 tx=0).stop 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 shutdown_connections 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fced40a2c20 0x7fced40a50e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fceec1040a0 0x7fceec074dc0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fceec102500 0x7fceec076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 --2- 172.21.15.52:0/3150872283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fceec103700 0x7fceec074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 >> 172.21.15.52:0/3150872283 conn(0x7fceec0fdaa0 msgr2=0x7fceec106a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:05:28.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 shutdown_connections 2024-06-02T23:05:28.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:05:28.817+0000 7fcef1e94700 1 -- 172.21.15.52:0/3150872283 wait complete. 2024-06-02T23:05:29.139 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:29.139 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.139 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.139 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:29.139 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:29.140 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.140 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.140 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.140 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T23:05:29.140 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T23:05:29.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:29.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-06-02T23:05:29.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:29.492 DEBUG:teuthology.orchestra.run.smithi052:osd.9> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.9.service 2024-06-02T23:05:29.494 INFO:tasks.cephadm:Deploying osd.10 on smithi123 with /dev/vg_nvme/lv_1... 2024-06-02T23:05:29.494 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-02T23:05:29.750 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:05:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-mon[32340]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-mon[32340]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-02T23:05:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-mon[32340]: osdmap e65: 10 total, 9 up, 10 in 2024-06-02T23:05:30.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-mon[32340]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:05:30.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:30.901 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:05:30.781+0000 7fa04c6c4700 -1 osd.9 0 waiting for initial osdmap 2024-06-02T23:05:30.901 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:05:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:05:30.797+0000 7fa0484b7700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:05:31.063 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:30 smithi123 ceph-mon[35977]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:31.063 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:30 smithi123 ceph-mon[35977]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-02T23:05:31.063 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:30 smithi123 ceph-mon[35977]: osdmap e65: 10 total, 9 up, 10 in 2024-06-02T23:05:31.063 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:30 smithi123 ceph-mon[35977]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:05:31.063 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:30 smithi174 ceph-mon[35994]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:30 smithi174 ceph-mon[35994]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-06-02T23:05:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:30 smithi174 ceph-mon[35994]: osdmap e65: 10 total, 9 up, 10 in 2024-06-02T23:05:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:30 smithi174 ceph-mon[35994]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]: dispatch 2024-06-02T23:05:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:31.801 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-02T23:05:31.801 INFO:teuthology.orchestra.run.smithi123.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-02T23:05:31.801 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10+0 records in 2024-06-02T23:05:31.801 INFO:teuthology.orchestra.run.smithi123.stderr:10+0 records out 2024-06-02T23:05:31.801 INFO:teuthology.orchestra.run.smithi123.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0143662 s, 730 MB/s 2024-06-02T23:05:31.802 INFO:teuthology.orchestra.run.smithi123.stderr:--> Zapping successful for: 2024-06-02T23:05:32.117 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:31 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:05:32.117 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:31 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:05:32.117 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:31 smithi123 ceph-mon[35977]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:05:32.117 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:31 smithi123 ceph-mon[35977]: osdmap e66: 10 total, 9 up, 10 in 2024-06-02T23:05:32.117 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.117 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:31 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:05:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:31 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:05:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:31 smithi174 ceph-mon[35994]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:05:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:31 smithi174 ceph-mon[35994]: osdmap e66: 10 total, 9 up, 10 in 2024-06-02T23:05:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:31 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:05:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:31 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:05:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:31 smithi052 ceph-mon[32340]: from='osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi052", "root=default"]}]': finished 2024-06-02T23:05:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:31 smithi052 ceph-mon[32340]: osdmap e66: 10 total, 9 up, 10 in 2024-06-02T23:05:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.438 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-06-02T23:05:32.683 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:05:32.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] boot 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: osdmap e67: 10 total, 10 up, 10 in 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:32.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.965 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] boot 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: osdmap e67: 10 total, 10 up, 10 in 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:32.966 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:32.967 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 272 MiB used, 804 GiB / 805 GiB avail 2024-06-02T23:05:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: osd.9 [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] boot 2024-06-02T23:05:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: osdmap e67: 10 total, 10 up, 10 in 2024-06-02T23:05:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-06-02T23:05:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:33.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:33.843 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:33 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi052 2024-06-02T23:05:33.843 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:33 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 4273M 2024-06-02T23:05:33.843 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:33 smithi123 ceph-mon[35977]: osdmap e68: 10 total, 10 up, 10 in 2024-06-02T23:05:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:33 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi052 2024-06-02T23:05:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:33 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 4273M 2024-06-02T23:05:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:33 smithi174 ceph-mon[35994]: osdmap e68: 10 total, 10 up, 10 in 2024-06-02T23:05:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:33 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi052 2024-06-02T23:05:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:33 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 4273M 2024-06-02T23:05:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:33 smithi052 ceph-mon[32340]: osdmap e68: 10 total, 10 up, 10 in 2024-06-02T23:05:34.367 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 -- 172.21.15.123:0/2692809915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc00102f90 msgr2=0x7fdc00109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:34.390 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2692809915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc00102f90 0x7fdc00109840 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800b0d0 tx=0x7fdbf800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:05:34.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 -- 172.21.15.123:0/2692809915 shutdown_connections 2024-06-02T23:05:34.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2692809915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc00102f90 0x7fdc00109840 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:34.391 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2692809915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc001013f0 0x7fdc00101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:34.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2692809915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc001025f0 0x7fdc00102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:34.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 -- 172.21.15.123:0/2692809915 >> 172.21.15.123:0/2692809915 conn(0x7fdc000fc990 msgr2=0x7fdc000fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:05:34.392 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 -- 172.21.15.123:0/2692809915 shutdown_connections 2024-06-02T23:05:34.393 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.367+0000 7fdc066ef700 1 -- 172.21.15.123:0/2692809915 wait complete. 2024-06-02T23:05:34.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 Processor -- start 2024-06-02T23:05:34.394 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 -- start start 2024-06-02T23:05:34.399 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc001013f0 0x7fdc00198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 0x7fdc00198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc00102f90 0x7fdc0019fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0010ce80 con 0x7fdc001013f0 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0010cd00 con 0x7fdc001025f0 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdc066ef700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0010d000 con 0x7fdc00102f90 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 0x7fdc00198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 0x7fdc00198b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:60512/0 (socket says 172.21.15.123:60512) 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 -- 172.21.15.123:0/2409254385 learned_addr learned my addr 172.21.15.123:0/2409254385 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-06-02T23:05:34.400 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbfffff700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc001013f0 0x7fdc00198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 -- 172.21.15.123:0/2409254385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc00102f90 msgr2=0x7fdc0019fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc00102f90 0x7fdc0019fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 -- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc001013f0 msgr2=0x7fdc00198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc001013f0 0x7fdc00198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.368+0000 7fdbff7fe700 1 -- 172.21.15.123:0/2409254385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc001a0260 con 0x7fdc001025f0 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.369+0000 7fdbff7fe700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 0x7fdc00198b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400d980 tx=0x7fdbf400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.369+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4017070 con 0x7fdc001025f0 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.369+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdbf4004830 con 0x7fdc001025f0 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.369+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc001a0550 con 0x7fdc001025f0 2024-06-02T23:05:34.401 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.369+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf4005440 con 0x7fdc001025f0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.369+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc001a0a40 con 0x7fdc001025f0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.370+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc00069180 con 0x7fdc001025f0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.371+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fdbf40055e0 con 0x7fdc001025f0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.372+0000 7fdbfd7fa700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdbe80a2c20 0x7fdbe80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.372+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7fdbf40ce7f0 con 0x7fdc001025f0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.372+0000 7fdbfffff700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdbe80a2c20 0x7fdbe80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.372+0000 7fdbfffff700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdbe80a2c20 0x7fdbe80a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc00102450 tx=0x7fdbf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:05:34.402 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.376+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdbf4020020 con 0x7fdc001025f0 2024-06-02T23:05:34.588 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:34.585+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fdc000febc0 con 0x7fdbe80a2c20 2024-06-02T23:05:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:34 smithi123 ceph-mon[35977]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:05:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:05:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:34 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:34 smithi174 ceph-mon[35994]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:05:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:05:35.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:34 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:34 smithi052 ceph-mon[32340]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:05:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:05:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:34 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:35 smithi174 ceph-mon[35994]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:05:36.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:35 smithi052 ceph-mon[32340]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:05:36.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:35 smithi123 ceph-mon[35977]: from='client.24335 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:05:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:36 smithi123 ceph-mon[35977]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:36 smithi174 ceph-mon[35994]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:36 smithi052 ceph-mon[32340]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:37 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/4046755335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]: dispatch 2024-06-02T23:05:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:37 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]: dispatch 2024-06-02T23:05:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:37 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]': finished 2024-06-02T23:05:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:37 smithi174 ceph-mon[35994]: osdmap e69: 11 total, 10 up, 11 in 2024-06-02T23:05:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:37 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/4046755335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]: dispatch 2024-06-02T23:05:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:37 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]: dispatch 2024-06-02T23:05:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:37 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]': finished 2024-06-02T23:05:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:37 smithi052 ceph-mon[32340]: osdmap e69: 11 total, 10 up, 11 in 2024-06-02T23:05:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:37 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/4046755335' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]: dispatch 2024-06-02T23:05:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:37 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]: dispatch 2024-06-02T23:05:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:37 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bea87b2-167c-4b06-939d-59e6a5c69b07"}]': finished 2024-06-02T23:05:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:37 smithi123 ceph-mon[35977]: osdmap e69: 11 total, 10 up, 11 in 2024-06-02T23:05:38.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:39.050 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:38 smithi052 ceph-mon[32340]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:39.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:38 smithi052 ceph-mon[32340]: from='client.? 172.21.15.123:0/2748744819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:05:39.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:38 smithi174 ceph-mon[35994]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:39.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:38 smithi174 ceph-mon[35994]: from='client.? 172.21.15.123:0/2748744819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:05:39.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:38 smithi123 ceph-mon[35977]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:39.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:38 smithi123 ceph-mon[35977]: from='client.? 172.21.15.123:0/2748744819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:05:41.096 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:40 smithi123 ceph-mon[35977]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:41.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:40 smithi174 ceph-mon[35994]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:40 smithi052 ceph-mon[32340]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:43.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:42 smithi123 ceph-mon[35977]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:43.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:42 smithi174 ceph-mon[35994]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:43.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:42 smithi052 ceph-mon[32340]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:45.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:44 smithi174 ceph-mon[35994]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:45.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:44 smithi052 ceph-mon[32340]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:45.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:44 smithi123 ceph-mon[35977]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-02T23:05:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:45 smithi174 ceph-mon[35994]: Deploying daemon osd.10 on smithi123 2024-06-02T23:05:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-02T23:05:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:45 smithi052 ceph-mon[32340]: Deploying daemon osd.10 on smithi123 2024-06-02T23:05:46.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-06-02T23:05:46.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:46.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:45 smithi123 ceph-mon[35977]: Deploying daemon osd.10 on smithi123 2024-06-02T23:05:47.094 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:46 smithi123 ceph-mon[35977]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:46 smithi174 ceph-mon[35994]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:47.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:46 smithi052 ceph-mon[32340]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:49.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:48 smithi174 ceph-mon[35994]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:48 smithi052 ceph-mon[32340]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:48 smithi123 ceph-mon[35977]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:51 smithi123 ceph-mon[35977]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:51 smithi174 ceph-mon[35994]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:51 smithi052 ceph-mon[32340]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:53 smithi174 ceph-mon[35994]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:53 smithi052 ceph-mon[32340]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:53 smithi123 ceph-mon[35977]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:53.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:53.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:53.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:53.971 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:54.354 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.354+0000 7fdbfd7fa700 1 -- 172.21.15.123:0/2409254385 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fdc000febc0 con 0x7fdbe80a2c20 2024-06-02T23:05:54.354 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 10 on host 'smithi123' 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdbe80a2c20 msgr2=0x7fdbe80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdbe80a2c20 0x7fdbe80a50e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdc00102450 tx=0x7fdbf0009260 comp rx=0 tx=0).stop 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 msgr2=0x7fdc00198b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 0x7fdc00198b90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdbf400d980 tx=0x7fdbf400dd40 comp rx=0 tx=0).stop 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 shutdown_connections 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdbe80a2c20 0x7fdbe80a50e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:54.356 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc00102f90 0x7fdc0019fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc001013f0 0x7fdc00198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 --2- 172.21.15.123:0/2409254385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc001025f0 0x7fdc00198b90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:05:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 >> 172.21.15.123:0/2409254385 conn(0x7fdc000fc990 msgr2=0x7fdc000fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:05:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 shutdown_connections 2024-06-02T23:05:54.357 INFO:teuthology.orchestra.run.smithi123.stderr:2024-06-02T23:05:54.356+0000 7fdc066ef700 1 -- 172.21.15.123:0/2409254385 wait complete. 2024-06-02T23:05:54.983 DEBUG:teuthology.orchestra.run.smithi123:osd.10> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.10.service 2024-06-02T23:05:54.985 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-06-02T23:05:54.985 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-06-02T23:05:55.234 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:05:55.559 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:55 smithi174 ceph-mon[35994]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:55.559 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:55.559 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:55.559 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:55.559 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:55 smithi174 ceph-mon[35994]: from='osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T23:05:55.559 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:55 smithi174 ceph-mon[35994]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T23:05:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:55 smithi052 ceph-mon[32340]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:55.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:55.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:55.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:55 smithi052 ceph-mon[32340]: from='osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T23:05:55.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:55 smithi052 ceph-mon[32340]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T23:05:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:55 smithi123 ceph-mon[35977]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:05:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:55 smithi123 ceph-mon[35977]: from='osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T23:05:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:55 smithi123 ceph-mon[35977]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-06-02T23:05:56.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-mon[35977]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-02T23:05:56.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-mon[35977]: osdmap e70: 11 total, 10 up, 11 in 2024-06-02T23:05:56.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:56.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-mon[35977]: from='osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:05:56.426 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-mon[35977]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:05:56.427 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:05:56.318+0000 7f9bde941700 -1 osd.10 0 waiting for initial osdmap 2024-06-02T23:05:56.427 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:05:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:05:56.335+0000 7f9bda734700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:05:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:56 smithi174 ceph-mon[35994]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-02T23:05:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:56 smithi174 ceph-mon[35994]: osdmap e70: 11 total, 10 up, 11 in 2024-06-02T23:05:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:56 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:56 smithi174 ceph-mon[35994]: from='osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:05:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:56 smithi174 ceph-mon[35994]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:05:56.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:56 smithi052 ceph-mon[32340]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-06-02T23:05:56.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:56 smithi052 ceph-mon[32340]: osdmap e70: 11 total, 10 up, 11 in 2024-06-02T23:05:56.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:56 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:56.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:56 smithi052 ceph-mon[32340]: from='osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:05:56.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:56 smithi052 ceph-mon[32340]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]: dispatch 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: osdmap e71: 11 total, 10 up, 11 in 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] boot 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: osdmap e72: 11 total, 11 up, 11 in 2024-06-02T23:05:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0115866 s, 905 MB/s 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 2024-06-02T23:05:57.550 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-06-02T23:05:57.576 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:57.576 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:05:57.576 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: osdmap e71: 11 total, 10 up, 11 in 2024-06-02T23:05:57.576 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] boot 2024-06-02T23:05:57.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: osdmap e72: 11 total, 11 up, 11 in 2024-06-02T23:05:57.577 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 299 MiB used, 894 GiB / 894 GiB avail 2024-06-02T23:05:57.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-06-02T23:05:57.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: osdmap e71: 11 total, 10 up, 11 in 2024-06-02T23:05:57.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:57.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: osd.10 [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] boot 2024-06-02T23:05:57.715 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: osdmap e72: 11 total, 11 up, 11 in 2024-06-02T23:05:57.716 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-06-02T23:05:58.209 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-06-02T23:05:58.450 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:05:58.540 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:05:58.540 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:05:58.540 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: osdmap e73: 11 total, 11 up, 11 in 2024-06-02T23:05:58.540 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:58.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: osdmap e73: 11 total, 11 up, 11 in 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:58.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:58.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: osdmap e73: 11 total, 11 up, 11 in 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:05:58.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:58.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:05:58.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:05:58.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:05:59.620 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:59 smithi174 ceph-mon[35994]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:05:59.621 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:59 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi123 2024-06-02T23:05:59.621 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:59 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 4272M 2024-06-02T23:05:59.622 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:05:59 smithi174 ceph-mon[35994]: osdmap e74: 11 total, 11 up, 11 in 2024-06-02T23:05:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:59 smithi052 ceph-mon[32340]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:05:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:59 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi123 2024-06-02T23:05:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:59 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 4272M 2024-06-02T23:05:59.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:05:59 smithi052 ceph-mon[32340]: osdmap e74: 11 total, 11 up, 11 in 2024-06-02T23:05:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:59 smithi123 ceph-mon[35977]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:05:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:59 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi123 2024-06-02T23:05:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:59 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 4272M 2024-06-02T23:05:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:05:59 smithi123 ceph-mon[35977]: osdmap e74: 11 total, 11 up, 11 in 2024-06-02T23:06:00.200 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 -- 172.21.15.174:0/736524068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1036f0 msgr2=0x7f3a7c103b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 --2- 172.21.15.174:0/736524068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c103b50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009990 tx=0x7f3a6c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 -- 172.21.15.174:0/736524068 shutdown_connections 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 --2- 172.21.15.174:0/736524068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c104090 0x7f3a7c10a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 --2- 172.21.15.174:0/736524068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 --2- 172.21.15.174:0/736524068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c103b50 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 -- 172.21.15.174:0/736524068 >> 172.21.15.174:0/736524068 conn(0x7f3a7c0fda90 msgr2=0x7f3a7c0ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 -- 172.21.15.174:0/736524068 shutdown_connections 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 -- 172.21.15.174:0/736524068 wait complete. 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.199+0000 7f3a83c48700 1 Processor -- start 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 -- start start 2024-06-02T23:06:00.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c19ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c19f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c104090 0x7f3a7c1a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c114640 con 0x7f3a7c1036f0 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1144c0 con 0x7f3a7c104090 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a7c1147c0 con 0x7f3a7c1024f0 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c19ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c19ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:38100/0 (socket says 172.21.15.174:38100) 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 -- 172.21.15.174:0/2468496678 learned_addr learned my addr 172.21.15.174:0/2468496678 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a821e5700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c104090 0x7f3a7c1a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a811e3700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c19f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 -- 172.21.15.174:0/2468496678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c104090 msgr2=0x7f3a7c1a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:00.202 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c104090 0x7f3a7c1a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 -- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c1036f0 msgr2=0x7f3a7c19f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c19f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 -- 172.21.15.174:0/2468496678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7c1a6930 con 0x7f3a7c1024f0 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a811e3700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a819e4700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c19ecb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b500 tx=0x7f3a7800b810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78015020 con 0x7f3a7c1024f0 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a78011ce0 con 0x7f3a7c1024f0 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.200+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a7c1a6c20 con 0x7f3a7c1024f0 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.201+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a78010d60 con 0x7f3a7c1024f0 2024-06-02T23:06:00.203 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.201+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a7c1a7110 con 0x7f3a7c1024f0 2024-06-02T23:06:00.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.201+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a7c069180 con 0x7f3a7c1024f0 2024-06-02T23:06:00.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.203+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3a78018020 con 0x7f3a7c1024f0 2024-06-02T23:06:00.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.204+0000 7f3a72ffd700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a680a2c10 0x7f3a680a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:00.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.204+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6694+0+0 (secure 0 0 0) 0x7f3a780ce710 con 0x7f3a7c1024f0 2024-06-02T23:06:00.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.204+0000 7f3a811e3700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a680a2c10 0x7f3a680a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:00.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.205+0000 7f3a811e3700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a680a2c10 0x7f3a680a50d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c003920 tx=0x7f3a6c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:00.207 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.208+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3a7801f020 con 0x7f3a7c1024f0 2024-06-02T23:06:00.415 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:00.415+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3a7c0628d0 con 0x7f3a680a2c10 2024-06-02T23:06:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:00 smithi174 ceph-mon[35994]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:00 smithi174 ceph-mon[35994]: osdmap e75: 11 total, 11 up, 11 in 2024-06-02T23:06:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:00 smithi174 ceph-mon[35994]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:06:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:00 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:06:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:00 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:06:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:00 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:00 smithi052 ceph-mon[32340]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:00 smithi052 ceph-mon[32340]: osdmap e75: 11 total, 11 up, 11 in 2024-06-02T23:06:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:00 smithi052 ceph-mon[32340]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:06:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:00 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:06:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:00 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:06:01.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:00 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:00 smithi123 ceph-mon[35977]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:00 smithi123 ceph-mon[35977]: osdmap e75: 11 total, 11 up, 11 in 2024-06-02T23:06:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:00 smithi123 ceph-mon[35977]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:06:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:00 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-06-02T23:06:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:00 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-06-02T23:06:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:00 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:01 smithi123 ceph-mon[35977]: osdmap e76: 11 total, 11 up, 11 in 2024-06-02T23:06:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:01 smithi174 ceph-mon[35994]: osdmap e76: 11 total, 11 up, 11 in 2024-06-02T23:06:02.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:01 smithi052 ceph-mon[32340]: osdmap e76: 11 total, 11 up, 11 in 2024-06-02T23:06:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:02 smithi174 ceph-mon[35994]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-02T23:06:03.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:02 smithi052 ceph-mon[32340]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-02T23:06:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:02 smithi123 ceph-mon[35977]: pgmap v242: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-06-02T23:06:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:03 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/3185811652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]: dispatch 2024-06-02T23:06:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:03 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]: dispatch 2024-06-02T23:06:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:03 smithi174 ceph-mon[35994]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]': finished 2024-06-02T23:06:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:03 smithi174 ceph-mon[35994]: osdmap e77: 12 total, 11 up, 12 in 2024-06-02T23:06:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:03 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:03 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/3185811652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]: dispatch 2024-06-02T23:06:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:03 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]: dispatch 2024-06-02T23:06:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:03 smithi052 ceph-mon[32340]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]': finished 2024-06-02T23:06:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:03 smithi052 ceph-mon[32340]: osdmap e77: 12 total, 11 up, 12 in 2024-06-02T23:06:04.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:03 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:03 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/3185811652' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]: dispatch 2024-06-02T23:06:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:03 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]: dispatch 2024-06-02T23:06:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:03 smithi123 ceph-mon[35977]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d9af8f08-98ca-4c39-8571-2a6fa0c08075"}]': finished 2024-06-02T23:06:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:03 smithi123 ceph-mon[35977]: osdmap e77: 12 total, 11 up, 12 in 2024-06-02T23:06:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:03 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:05.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:04 smithi174 ceph-mon[35994]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-02T23:06:05.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:04 smithi174 ceph-mon[35994]: from='client.? 172.21.15.174:0/998466884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:06:05.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:04 smithi052 ceph-mon[32340]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-02T23:06:05.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:04 smithi052 ceph-mon[32340]: from='client.? 172.21.15.174:0/998466884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:06:05.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:04 smithi123 ceph-mon[35977]: pgmap v244: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-06-02T23:06:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:04 smithi123 ceph-mon[35977]: from='client.? 172.21.15.174:0/998466884' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-06-02T23:06:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:06 smithi174 ceph-mon[35994]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:06:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:06 smithi052 ceph-mon[32340]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:06:07.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:06 smithi123 ceph-mon[35977]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-06-02T23:06:08.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:08 smithi174 ceph-mon[35994]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-02T23:06:09.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:08 smithi052 ceph-mon[32340]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-02T23:06:09.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:08 smithi123 ceph-mon[35977]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-06-02T23:06:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:10 smithi174 ceph-mon[35994]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-02T23:06:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:10 smithi052 ceph-mon[32340]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-02T23:06:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:10 smithi123 ceph-mon[35977]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-06-02T23:06:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:11 smithi174 ceph-mon[35994]: Deploying daemon osd.11 on smithi174 2024-06-02T23:06:12.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:11 smithi052 ceph-mon[32340]: Deploying daemon osd.11 on smithi174 2024-06-02T23:06:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:11 smithi123 ceph-mon[35977]: Deploying daemon osd.11 on smithi174 2024-06-02T23:06:12.963 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:12 smithi174 ceph-mon[35994]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:12 smithi052 ceph-mon[32340]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:12 smithi123 ceph-mon[35977]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:15.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:14 smithi174 ceph-mon[35994]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:15.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:14 smithi052 ceph-mon[32340]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:15.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:14 smithi123 ceph-mon[35977]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:17.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:17 smithi174 ceph-mon[35994]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:17.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:17.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:17.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:17 smithi052 ceph-mon[32340]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:17 smithi123 ceph-mon[35977]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:19 smithi174 ceph-mon[35994]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:19.809 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:19 smithi123 ceph-mon[35977]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:19 smithi052 ceph-mon[32340]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:20.234 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-06-02T23:06:20.234 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.234+0000 7f3a72ffd700 1 -- 172.21.15.174:0/2468496678 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3a7c0628d0 con 0x7f3a680a2c10 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a680a2c10 msgr2=0x7f3a680a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a680a2c10 0x7f3a680a50d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c003920 tx=0x7f3a6c005fb0 comp rx=0 tx=0).stop 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 msgr2=0x7f3a7c19ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c19ecb0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a7800b500 tx=0x7f3a7800b810 comp rx=0 tx=0).stop 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 shutdown_connections 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a680a2c10 0x7f3a680a50d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7c104090 0x7f3a7c1a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7c1024f0 0x7f3a7c19ecb0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:20.236 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 --2- 172.21.15.174:0/2468496678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7c1036f0 0x7f3a7c19f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:20.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 >> 172.21.15.174:0/2468496678 conn(0x7f3a7c0fda90 msgr2=0x7f3a7c106b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:20.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 shutdown_connections 2024-06-02T23:06:20.237 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:20.236+0000 7f3a83c48700 1 -- 172.21.15.174:0/2468496678 wait complete. 2024-06-02T23:06:20.864 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.11.service 2024-06-02T23:06:20.867 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-06-02T23:06:20.867 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-02T23:06:21.133 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:21.470 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:21.470 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.470 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.470 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: from='osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T23:06:21.471 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T23:06:21.471 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.471 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:21 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: from='osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T23:06:21.473 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T23:06:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.474 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:21 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: from='osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:21 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:22 smithi123 ceph-mon[35977]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-02T23:06:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:22 smithi123 ceph-mon[35977]: osdmap e78: 12 total, 11 up, 12 in 2024-06-02T23:06:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:22 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:22 smithi123 ceph-mon[35977]: from='osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:06:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:22 smithi123 ceph-mon[35977]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:06:22.554 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-mon[35994]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-02T23:06:22.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-mon[35994]: osdmap e78: 12 total, 11 up, 12 in 2024-06-02T23:06:22.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:22.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-mon[35994]: from='osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:06:22.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-mon[35994]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:06:22.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:22 smithi052 ceph-mon[32340]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-06-02T23:06:22.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:22 smithi052 ceph-mon[32340]: osdmap e78: 12 total, 11 up, 12 in 2024-06-02T23:06:22.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:22 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:22.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:22 smithi052 ceph-mon[32340]: from='osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:06:22.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:22 smithi052 ceph-mon[32340]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-06-02T23:06:22.857 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:06:22.654+0000 7ff2cc921700 -1 osd.11 0 waiting for initial osdmap 2024-06-02T23:06:22.857 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:06:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:06:22.666+0000 7ff2c470c700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-06-02T23:06:22.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 -- 172.21.15.52:0/1505796308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520102510 msgr2=0x7fa520102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:22.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 --2- 172.21.15.52:0/1505796308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520102510 0x7fa520102930 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa5100099c0 tx=0x7fa510009cd0 comp rx=0 tx=0).stop 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 -- 172.21.15.52:0/1505796308 shutdown_connections 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 --2- 172.21.15.52:0/1505796308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5201040b0 0x7fa52010a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 --2- 172.21.15.52:0/1505796308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520102510 0x7fa520102930 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 --2- 172.21.15.52:0/1505796308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa520103710 0x7fa520103b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 -- 172.21.15.52:0/1505796308 >> 172.21.15.52:0/1505796308 conn(0x7fa5200fdab0 msgr2=0x7fa5200ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 -- 172.21.15.52:0/1505796308 shutdown_connections 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.986+0000 7fa526538700 1 -- 172.21.15.52:0/1505796308 wait complete. 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 Processor -- start 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 -- start start 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520102510 0x7fa520076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 0x7fa520074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5201040b0 0x7fa520074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:22.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010e010 con 0x7fa520103710 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010de90 con 0x7fa520102510 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa526538700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa52010e190 con 0x7fa5201040b0 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa517fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 0x7fa520074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa517fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 0x7fa520074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43890/0 (socket says 172.21.15.52:43890) 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa517fff700 1 -- 172.21.15.52:0/1906873097 learned_addr learned my addr 172.21.15.52:0/1906873097 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa524ad5700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5201040b0 0x7fa520074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.987+0000 7fa51ffff700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520102510 0x7fa520076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa517fff700 1 -- 172.21.15.52:0/1906873097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5201040b0 msgr2=0x7fa520074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa517fff700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5201040b0 0x7fa520074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa517fff700 1 -- 172.21.15.52:0/1906873097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520102510 msgr2=0x7fa520076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa517fff700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520102510 0x7fa520076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa517fff700 1 -- 172.21.15.52:0/1906873097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa520075670 con 0x7fa520103710 2024-06-02T23:06:22.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa517fff700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 0x7fa520074880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa50800b530 tx=0x7fa50800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508015020 con 0x7fa520103710 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5201a2830 con 0x7fa520103710 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa508011ce0 con 0x7fa520103710 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa508010dd0 con 0x7fa520103710 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.988+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5201a2df0 con 0x7fa520103710 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.989+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa520069180 con 0x7fa520103710 2024-06-02T23:06:22.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.991+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa508018020 con 0x7fa520103710 2024-06-02T23:06:22.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.992+0000 7fa51dffb700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa50c0a2c20 0x7fa50c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:22.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.992+0000 7fa51ffff700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa50c0a2c20 0x7fa50c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:22.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.992+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 6847+0+0 (secure 0 0 0) 0x7fa5080ce510 con 0x7fa520103710 2024-06-02T23:06:22.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.992+0000 7fa51ffff700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa50c0a2c20 0x7fa50c0a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa510000c00 tx=0x7fa510005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:22.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:22.996+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa50801f030 con 0x7fa520103710 2024-06-02T23:06:23.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.204+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fa52002dcd0 con 0x7fa520103710 2024-06-02T23:06:23.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.206+0000 7fa51dffb700 1 -- 172.21.15.52:0/1906873097 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) ==== 74+0+133 (secure 0 0 0) 0x7fa50809ba30 con 0x7fa520103710 2024-06-02T23:06:23.205 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:23.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa50c0a2c20 msgr2=0x7fa50c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:23.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa50c0a2c20 0x7fa50c0a50e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa510000c00 tx=0x7fa510005fb0 comp rx=0 tx=0).stop 2024-06-02T23:06:23.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 msgr2=0x7fa520074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 0x7fa520074880 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa50800b530 tx=0x7fa50800b840 comp rx=0 tx=0).stop 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 shutdown_connections 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa50c0a2c20 0x7fa50c0a50e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5201040b0 0x7fa520074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa520102510 0x7fa520076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 --2- 172.21.15.52:0/1906873097 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa520103710 0x7fa520074880 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 >> 172.21.15.52:0/1906873097 conn(0x7fa5200fdab0 msgr2=0x7fa520106ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 shutdown_connections 2024-06-02T23:06:23.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:23.208+0000 7fa526538700 1 -- 172.21.15.52:0/1906873097 wait complete. 2024-06-02T23:06:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:23 smithi123 ceph-mon[35977]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:23 smithi123 ceph-mon[35977]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:06:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:23 smithi123 ceph-mon[35977]: osdmap e79: 12 total, 11 up, 12 in 2024-06-02T23:06:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:23 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:23 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:23 smithi123 ceph-mon[35977]: from='osd.11 ' entity='osd.11' 2024-06-02T23:06:23.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:23 smithi052 ceph-mon[32340]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:23.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:23 smithi052 ceph-mon[32340]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:06:23.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:23 smithi052 ceph-mon[32340]: osdmap e79: 12 total, 11 up, 12 in 2024-06-02T23:06:23.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:23 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:23.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:23 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:23.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:23 smithi052 ceph-mon[32340]: from='osd.11 ' entity='osd.11' 2024-06-02T23:06:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:23 smithi174 ceph-mon[35994]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:23 smithi174 ceph-mon[35994]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-06-02T23:06:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:23 smithi174 ceph-mon[35994]: osdmap e79: 12 total, 11 up, 12 in 2024-06-02T23:06:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:23 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:23 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:23 smithi174 ceph-mon[35994]: from='osd.11 ' entity='osd.11' 2024-06-02T23:06:23.837 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1717369556,"num_in_osds":12,"osd_in_since":1717369563,"num_remapped_pgs":0} 2024-06-02T23:06:24.313 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: purged_snaps scrub starts 2024-06-02T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: purged_snaps scrub ok 2024-06-02T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1906873097' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] boot 2024-06-02T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: osdmap e80: 12 total, 12 up, 12 in 2024-06-02T23:06:24.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: purged_snaps scrub starts 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: purged_snaps scrub ok 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1906873097' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] boot 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: osdmap e80: 12 total, 12 up, 12 in 2024-06-02T23:06:24.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: purged_snaps scrub starts 2024-06-02T23:06:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: purged_snaps scrub ok 2024-06-02T23:06:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1906873097' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T23:06:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: osd.11 [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] boot 2024-06-02T23:06:24.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: osdmap e80: 12 total, 12 up, 12 in 2024-06-02T23:06:24.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-06-02T23:06:24.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd stat -f json 2024-06-02T23:06:25.087 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: Detected new or changed devices on smithi174 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 4272M 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:25.514 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.515 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:25 smithi052 ceph-mon[32340]: osdmap e81: 12 total, 12 up, 12 in 2024-06-02T23:06:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: Detected new or changed devices on smithi174 2024-06-02T23:06:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 4272M 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:25 smithi174 ceph-mon[35994]: osdmap e81: 12 total, 12 up, 12 in 2024-06-02T23:06:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-06-02T23:06:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: Detected new or changed devices on smithi174 2024-06-02T23:06:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 4272M 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:25.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:25 smithi123 ceph-mon[35977]: osdmap e81: 12 total, 12 up, 12 in 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 -- 172.21.15.52:0/1019761645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1013f0 msgr2=0x7ff1fc101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 --2- 172.21.15.52:0/1019761645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1013f0 0x7ff1fc101810 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0099c0 tx=0x7ff1ec009cd0 comp rx=0 tx=0).stop 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 -- 172.21.15.52:0/1019761645 shutdown_connections 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 --2- 172.21.15.52:0/1019761645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1fc102f90 0x7ff1fc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 --2- 172.21.15.52:0/1019761645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1013f0 0x7ff1fc101810 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 --2- 172.21.15.52:0/1019761645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 -- 172.21.15.52:0/1019761645 >> 172.21.15.52:0/1019761645 conn(0x7ff1fc0fc990 msgr2=0x7ff1fc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:26.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 -- 172.21.15.52:0/1019761645 shutdown_connections 2024-06-02T23:06:26.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.904+0000 7ff2027f8700 1 -- 172.21.15.52:0/1019761645 wait complete. 2024-06-02T23:06:26.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 Processor -- start 2024-06-02T23:06:26.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 -- start start 2024-06-02T23:06:26.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1fc1013f0 0x7ff1fc105380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1fc102f90 0x7ff1fc103f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10cf60 con 0x7ff1fc1025f0 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10cde0 con 0x7ff1fc102f90 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff2027f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1fc10d0e0 con 0x7ff1fc1013f0 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.905+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc1039d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49368/0 (socket says 172.21.15.52:49368) 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 -- 172.21.15.52:0/2143794835 learned_addr learned my addr 172.21.15.52:0/2143794835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fbfff700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1fc1013f0 0x7ff1fc105380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff200d95700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1fc102f90 0x7ff1fc103f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 -- 172.21.15.52:0/2143794835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1fc1013f0 msgr2=0x7ff1fc105380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1fc1013f0 0x7ff1fc105380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:26.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 -- 172.21.15.52:0/2143794835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1fc102f90 msgr2=0x7ff1fc103f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1fc102f90 0x7ff1fc103f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 -- 172.21.15.52:0/2143794835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1fc1047c0 con 0x7ff1fc1025f0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1fb7fe700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc1039d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b530 tx=0x7ff1f000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0015020 con 0x7ff1fc1025f0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1fc1a2970 con 0x7ff1fc1025f0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1f0011ce0 con 0x7ff1fc1025f0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1f0010dd0 con 0x7ff1fc1025f0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.906+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1fc1a2f30 con 0x7ff1fc1025f0 2024-06-02T23:06:26.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.907+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1fc069180 con 0x7ff1fc1025f0 2024-06-02T23:06:26.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.909+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7ff1f0018020 con 0x7ff1fc1025f0 2024-06-02T23:06:26.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.909+0000 7ff1f97fa700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1e40a2c20 0x7ff1e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:26.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.909+0000 7ff1fbfff700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1e40a2c20 0x7ff1e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:26.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.909+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff1f00ce770 con 0x7ff1fc1025f0 2024-06-02T23:06:26.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.910+0000 7ff1fbfff700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1e40a2c20 0x7ff1e40a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:26.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:26.913+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff1f001f020 con 0x7ff1fc1025f0 2024-06-02T23:06:27.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.121+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7ff1fc1a31c0 con 0x7ff1fc1025f0 2024-06-02T23:06:27.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.122+0000 7ff1f97fa700 1 -- 172.21.15.52:0/2143794835 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7ff1fc1a31c0 con 0x7ff1fc1025f0 2024-06-02T23:06:27.121 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:27.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1e40a2c20 msgr2=0x7ff1e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1e40a2c20 0x7ff1e40a50e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec000c00 tx=0x7ff1ec005fb0 comp rx=0 tx=0).stop 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 msgr2=0x7ff1fc1039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc1039d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff1f000b530 tx=0x7ff1f000b840 comp rx=0 tx=0).stop 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 shutdown_connections 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1e40a2c20 0x7ff1e40a50e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.124+0000 7ff2027f8700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1fc102f90 0x7ff1fc103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.125+0000 7ff2027f8700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1fc1013f0 0x7ff1fc105380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.125+0000 7ff2027f8700 1 --2- 172.21.15.52:0/2143794835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1fc1025f0 0x7ff1fc1039d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.125+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 >> 172.21.15.52:0/2143794835 conn(0x7ff1fc0fc990 msgr2=0x7ff1fc06a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.125+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 shutdown_connections 2024-06-02T23:06:27.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:27.125+0000 7ff2027f8700 1 -- 172.21.15.52:0/2143794835 wait complete. 2024-06-02T23:06:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:27 smithi174 ceph-mon[35994]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2143794835' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T23:06:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:27 smithi052 ceph-mon[32340]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2143794835' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T23:06:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:27 smithi123 ceph-mon[35977]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2143794835' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-06-02T23:06:27.814 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1717369583,"num_in_osds":12,"osd_in_since":1717369563,"num_remapped_pgs":0} 2024-06-02T23:06:27.814 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:06:28.064 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 -- 172.21.15.52:0/2356552179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101fc0 msgr2=0x7f30f0106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 --2- 172.21.15.52:0/2356552179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101fc0 0x7f30f0106010 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f30e40099c0 tx=0x7f30e4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 -- 172.21.15.52:0/2356552179 shutdown_connections 2024-06-02T23:06:29.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 --2- 172.21.15.52:0/2356552179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 0x7f30f010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 --2- 172.21.15.52:0/2356552179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30f0101610 0x7f30f01019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 --2- 172.21.15.52:0/2356552179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101fc0 0x7f30f0106010 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 -- 172.21.15.52:0/2356552179 >> 172.21.15.52:0/2356552179 conn(0x7f30f0078430 msgr2=0x7f30f0078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.639+0000 7f30f5415700 1 -- 172.21.15.52:0/2356552179 shutdown_connections 2024-06-02T23:06:29.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.640+0000 7f30f5415700 1 -- 172.21.15.52:0/2356552179 wait complete. 2024-06-02T23:06:29.640 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:29 smithi052 ceph-mon[32340]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.640+0000 7f30f5415700 1 Processor -- start 2024-06-02T23:06:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 -- start start 2024-06-02T23:06:29.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 0x7f30f0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30f0101fc0 0x7f30f0199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 0x7f30f01a4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0111580 con 0x7f30f0101610 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0111400 con 0x7f30f0101fc0 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30f5415700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30f0111700 con 0x7f30f01066f0 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 0x7f30f0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 0x7f30f0199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49388/0 (socket says 172.21.15.52:49388) 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 -- 172.21.15.52:0/2205152774 learned_addr learned my addr 172.21.15.52:0/2205152774 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30ee7fc700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30f0101fc0 0x7f30f0199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30ef7fe700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 0x7f30f01a4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:29.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 -- 172.21.15.52:0/2205152774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 msgr2=0x7f30f01a4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 0x7f30f01a4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 -- 172.21.15.52:0/2205152774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30f0101fc0 msgr2=0x7f30f0199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30f0101fc0 0x7f30f0199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30eeffd700 1 -- 172.21.15.52:0/2205152774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30f01a5000 con 0x7f30f0101610 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.641+0000 7f30ef7fe700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 0x7f30f01a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.642+0000 7f30eeffd700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 0x7f30f0199a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30e000ada0 tx=0x7f30e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.642+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0017020 con 0x7f30f0101610 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.642+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30f0197890 con 0x7f30f0101610 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.642+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f30e000ce30 con 0x7f30f0101610 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.642+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e0005390 con 0x7f30f0101610 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.642+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30f0197da0 con 0x7f30f0101610 2024-06-02T23:06:29.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.643+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30f0069180 con 0x7f30f0101610 2024-06-02T23:06:29.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:29 smithi174 ceph-mon[35994]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.644+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f30e000c980 con 0x7f30f0101610 2024-06-02T23:06:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.645+0000 7f30d7fff700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30d80a2c20 0x7f30d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.645+0000 7f30ee7fc700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30d80a2c20 0x7f30d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.645+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f30e00ce930 con 0x7f30f0101610 2024-06-02T23:06:29.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.645+0000 7f30ee7fc700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30d80a2c20 0x7f30d80a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000c00 tx=0x7f30e4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:29.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.648+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f30e0021020 con 0x7f30f0101610 2024-06-02T23:06:29.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:29 smithi123 ceph-mon[35977]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:29.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.855+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30f002dca0 con 0x7f30f0101610 2024-06-02T23:06:29.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.856+0000 7f30d7fff700 1 -- 172.21.15.52:0/2205152774 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18442 (secure 0 0 0) 0x7f30e009bdc0 con 0x7f30f0101610 2024-06-02T23:06:29.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:29.856 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":81,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:06:25.204024+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30d80a2c20 msgr2=0x7f30d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30d80a2c20 0x7f30d80a50e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f30e4000c00 tx=0x7f30e4005fb0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 msgr2=0x7f30f0199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 0x7f30f0199a40 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f30e000ada0 tx=0x7f30e000c330 comp rx=0 tx=0).stop 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 shutdown_connections 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30d80a2c20 0x7f30d80a50e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30f01066f0 0x7f30f01a4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30f0101610 0x7f30f0199a40 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 --2- 172.21.15.52:0/2205152774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30f0101fc0 0x7f30f0199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:29.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.858+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 >> 172.21.15.52:0/2205152774 conn(0x7f30f0078430 msgr2=0x7f30f01078c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.859+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 shutdown_connections 2024-06-02T23:06:29.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:29.859+0000 7f30f5415700 1 -- 172.21.15.52:0/2205152774 wait complete. 2024-06-02T23:06:30.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:30 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2205152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:06:30.624 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-02T23:02:34.195730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-02T23:06:30.625 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-06-02T23:06:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:30 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2205152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:06:30.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:30 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2205152774' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:06:30.876 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:31 smithi052 ceph-mon[32340]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:31 smithi123 ceph-mon[35977]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:31 smithi174 ceph-mon[35994]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 -- 172.21.15.52:0/698056608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102ed0 msgr2=0x7efdf010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 --2- 172.21.15.52:0/698056608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102ed0 0x7efdf010f360 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efde800b0a0 tx=0x7efde800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 -- 172.21.15.52:0/698056608 shutdown_connections 2024-06-02T23:06:32.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 --2- 172.21.15.52:0/698056608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102ed0 0x7efdf010f360 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 --2- 172.21.15.52:0/698056608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf0108520 0x7efdf0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 --2- 172.21.15.52:0/698056608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0102510 0x7efdf0102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 -- 172.21.15.52:0/698056608 >> 172.21.15.52:0/698056608 conn(0x7efdf0079640 msgr2=0x7efdf0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.521+0000 7efdf485c700 1 -- 172.21.15.52:0/698056608 shutdown_connections 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.522+0000 7efdf485c700 1 -- 172.21.15.52:0/698056608 wait complete. 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.522+0000 7efdf485c700 1 Processor -- start 2024-06-02T23:06:32.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.522+0000 7efdf485c700 1 -- start start 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 0x7efdf019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf0102ed0 0x7efdf019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0108520 0x7efdf01988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efdf0112660 con 0x7efdf0102510 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01124e0 con 0x7efdf0102ed0 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efdf01127e0 con 0x7efdf0108520 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 0x7efdf019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 0x7efdf019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49408/0 (socket says 172.21.15.52:49408) 2024-06-02T23:06:32.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 -- 172.21.15.52:0/1330876148 learned_addr learned my addr 172.21.15.52:0/1330876148 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdedd9b700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf0102ed0 0x7efdf019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdeed9d700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0108520 0x7efdf01988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 -- 172.21.15.52:0/1330876148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0108520 msgr2=0x7efdf01988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0108520 0x7efdf01988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 -- 172.21.15.52:0/1330876148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf0102ed0 msgr2=0x7efdf019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf0102ed0 0x7efdf019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 -- 172.21.15.52:0/1330876148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf0199040 con 0x7efdf0102510 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdee59c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 0x7efdf019e6d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efde000d950 tx=0x7efde000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde0016070 con 0x7efdf0102510 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efde0009df0 con 0x7efdf0102510 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf0199330 con 0x7efdf0102510 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.523+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde000f3c0 con 0x7efdf0102510 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.524+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf01a55a0 con 0x7efdf0102510 2024-06-02T23:06:32.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.525+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efdf0069180 con 0x7efdf0102510 2024-06-02T23:06:32.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.526+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efde0019070 con 0x7efdf0102510 2024-06-02T23:06:32.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.527+0000 7efddf7fe700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdd80a2c20 0x7efdd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:32.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.527+0000 7efdedd9b700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdd80a2c20 0x7efdd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:32.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.527+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7efde00ce7b0 con 0x7efdf0102510 2024-06-02T23:06:32.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.527+0000 7efdedd9b700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdd80a2c20 0x7efdd80a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdf0079c40 tx=0x7efde400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:32.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.531+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efde009bbb0 con 0x7efdf0102510 2024-06-02T23:06:32.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.735+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efdf002dca0 con 0x7efdf0102510 2024-06-02T23:06:32.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.736+0000 7efddf7fe700 1 -- 172.21.15.52:0/1330876148 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7efde009b4b0 con 0x7efdf0102510 2024-06-02T23:06:32.735 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdd80a2c20 msgr2=0x7efdd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdd80a2c20 0x7efdd80a50e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7efdf0079c40 tx=0x7efde400b3c0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 msgr2=0x7efdf019e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 0x7efdf019e6d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efde000d950 tx=0x7efde000dc60 comp rx=0 tx=0).stop 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 shutdown_connections 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdd80a2c20 0x7efdd80a50e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efdf0108520 0x7efdf01988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efdf0102510 0x7efdf019e6d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 --2- 172.21.15.52:0/1330876148 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efdf0102ed0 0x7efdf019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 >> 172.21.15.52:0/1330876148 conn(0x7efdf0079640 msgr2=0x7efdf0101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:32.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 shutdown_connections 2024-06-02T23:06:32.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:32.738+0000 7efdf485c700 1 -- 172.21.15.52:0/1330876148 wait complete. 2024-06-02T23:06:33.507 INFO:tasks.cephadm:Adding mds.a on smithi052 2024-06-02T23:06:33.507 INFO:tasks.cephadm:Adding mds.d on smithi052 2024-06-02T23:06:33.507 INFO:tasks.cephadm:Adding mds.g on smithi052 2024-06-02T23:06:33.507 INFO:tasks.cephadm:Adding mds.j on smithi052 2024-06-02T23:06:33.507 INFO:tasks.cephadm:Adding mds.b on smithi123 2024-06-02T23:06:33.507 INFO:tasks.cephadm:Adding mds.e on smithi123 2024-06-02T23:06:33.508 INFO:tasks.cephadm:Adding mds.h on smithi123 2024-06-02T23:06:33.508 INFO:tasks.cephadm:Adding mds.k on smithi123 2024-06-02T23:06:33.508 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-06-02T23:06:33.508 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-06-02T23:06:33.508 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-06-02T23:06:33.508 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-06-02T23:06:33.508 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph orch apply mds all '12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-06-02T23:06:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:33 smithi052 ceph-mon[32340]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:33.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:33 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1330876148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:06:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:33 smithi123 ceph-mon[35977]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:33 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1330876148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:06:33.758 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:06:33.880 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:33 smithi174 ceph-mon[35994]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:33.880 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:33 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1330876148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:06:35.304 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.301+0000 7f7bfa583700 1 -- 172.21.15.174:0/1602420896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf41024d0 msgr2=0x7f7bf41028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.301+0000 7f7bfa583700 1 --2- 172.21.15.174:0/1602420896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf41024d0 0x7f7bf41028f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7be4009990 tx=0x7f7be4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.301+0000 7f7bfa583700 1 -- 172.21.15.174:0/1602420896 shutdown_connections 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.301+0000 7f7bfa583700 1 --2- 172.21.15.174:0/1602420896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf4104070 0x7f7bf410a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.301+0000 7f7bfa583700 1 --2- 172.21.15.174:0/1602420896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf41024d0 0x7f7bf41028f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.301+0000 7f7bfa583700 1 --2- 172.21.15.174:0/1602420896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf41036d0 0x7f7bf4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.302+0000 7f7bfa583700 1 -- 172.21.15.174:0/1602420896 >> 172.21.15.174:0/1602420896 conn(0x7f7bf40fda30 msgr2=0x7f7bf40ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.302+0000 7f7bfa583700 1 -- 172.21.15.174:0/1602420896 shutdown_connections 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.302+0000 7f7bfa583700 1 -- 172.21.15.174:0/1602420896 wait complete. 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.302+0000 7f7bfa583700 1 Processor -- start 2024-06-02T23:06:35.305 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.302+0000 7f7bfa583700 1 -- start start 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf41024d0 0x7f7bf4076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf41036d0 0x7f7bf4074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 0x7f7bf4074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf410dfd0 con 0x7f7bf41036d0 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf410de50 con 0x7f7bf41024d0 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf410e150 con 0x7f7bf4104070 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 0x7f7bf4074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 0x7f7bf4074dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:40118/0 (socket says 172.21.15.174:40118) 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 -- 172.21.15.174:0/2463846358 learned_addr learned my addr 172.21.15.174:0/2463846358 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf3fff700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf41024d0 0x7f7bf4076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf37fe700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf41036d0 0x7f7bf4074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:35.306 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 -- 172.21.15.174:0/2463846358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf41024d0 msgr2=0x7f7bf4076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf41024d0 0x7f7bf4076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 -- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf41036d0 msgr2=0x7f7bf4074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf41036d0 0x7f7bf4074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 -- 172.21.15.174:0/2463846358 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf4075670 con 0x7f7bf4104070 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf8b20700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 0x7f7bf4074dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007ee0 tx=0x7f7bec00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00dbd0 con 0x7f7bf4104070 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7bec00dd70 con 0x7f7bf4104070 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf41a2840 con 0x7f7bf4104070 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bec005990 con 0x7f7bf4104070 2024-06-02T23:06:35.307 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.303+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf41a2d00 con 0x7f7bf4104070 2024-06-02T23:06:35.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.305+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bf4069180 con 0x7f7bf4104070 2024-06-02T23:06:35.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.306+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f7bec01a020 con 0x7f7bf4104070 2024-06-02T23:06:35.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.306+0000 7f7bf17fa700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7bdc0a2c20 0x7f7bdc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:35.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.307+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7bec0ce300 con 0x7f7bf4104070 2024-06-02T23:06:35.308 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.307+0000 7f7bf3fff700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7bdc0a2c20 0x7f7bdc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:35.309 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.307+0000 7f7bf3fff700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7bdc0a2c20 0x7f7bdc0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:35.312 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.312+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7bec010070 con 0x7f7bf4104070 2024-06-02T23:06:35.520 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.519+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7f7bf4062a90 con 0x7f7bdc0a2c20 2024-06-02T23:06:35.528 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.528+0000 7f7bf17fa700 1 -- 172.21.15.174:0/2463846358 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f7bf4062a90 con 0x7f7bdc0a2c20 2024-06-02T23:06:35.528 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-06-02T23:06:35.530 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7bdc0a2c20 msgr2=0x7f7bdc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7bdc0a2c20 0x7f7bdc0a50e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7be4000c00 tx=0x7f7be4005fb0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 msgr2=0x7f7bf4074dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 0x7f7bf4074dc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7bec007ee0 tx=0x7f7bec00d3b0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 shutdown_connections 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7bdc0a2c20 0x7f7bdc0a50e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7bf4104070 0x7f7bf4074dc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7bf41024d0 0x7f7bf4076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 --2- 172.21.15.174:0/2463846358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7bf41036d0 0x7f7bf4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 >> 172.21.15.174:0/2463846358 conn(0x7f7bf40fda30 msgr2=0x7f7bf4106b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 shutdown_connections 2024-06-02T23:06:35.531 INFO:teuthology.orchestra.run.smithi174.stderr:2024-06-02T23:06:35.530+0000 7f7bfa583700 1 -- 172.21.15.174:0/2463846358 wait complete. 2024-06-02T23:06:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:35 smithi052 ceph-mon[32340]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:35.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:35 smithi123 ceph-mon[35977]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:35.883 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:35 smithi174 ceph-mon[35994]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:36.150 DEBUG:teuthology.orchestra.run.smithi052:mds.a> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.a.service 2024-06-02T23:06:36.153 DEBUG:teuthology.orchestra.run.smithi052:mds.d> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.d.service 2024-06-02T23:06:36.196 DEBUG:teuthology.orchestra.run.smithi052:mds.g> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.g.service 2024-06-02T23:06:36.239 DEBUG:teuthology.orchestra.run.smithi052:mds.j> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.j.service 2024-06-02T23:06:36.282 DEBUG:teuthology.orchestra.run.smithi123:mds.b> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.b.service 2024-06-02T23:06:36.284 DEBUG:teuthology.orchestra.run.smithi123:mds.e> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.e.service 2024-06-02T23:06:36.327 DEBUG:teuthology.orchestra.run.smithi123:mds.h> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.h.service 2024-06-02T23:06:36.371 DEBUG:teuthology.orchestra.run.smithi123:mds.k> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.k.service 2024-06-02T23:06:36.414 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.c.service 2024-06-02T23:06:36.416 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.f.service 2024-06-02T23:06:36.459 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.i.service 2024-06-02T23:06:36.503 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.l.service 2024-06-02T23:06:36.546 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-06-02T23:06:36.547 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:36.802 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:36.822 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:06:36.822 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-06-02T23:06:36.822 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:36.822 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:36.822 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:36.822 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:36.823 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:36.823 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:36.823 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:36.823 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:36.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: Saving service mds.all spec with placement smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:36.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:37 smithi174 ceph-mon[35994]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:37 smithi174 ceph-mon[35994]: Deploying daemon mds.l on smithi174 2024-06-02T23:06:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:37 smithi052 ceph-mon[32340]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:37 smithi052 ceph-mon[32340]: Deploying daemon mds.l on smithi174 2024-06-02T23:06:38.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:37 smithi123 ceph-mon[35977]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:37 smithi123 ceph-mon[35977]: Deploying daemon mds.l on smithi174 2024-06-02T23:06:38.393 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 systemd[1]: Starting Ceph mds.l for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:06:38.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 -- 172.21.15.52:0/1774714667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 msgr2=0x7fa5d410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:38.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 --2- 172.21.15.52:0/1774714667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d410f330 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa5cc00b0a0 tx=0x7fa5cc00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 -- 172.21.15.52:0/1774714667 shutdown_connections 2024-06-02T23:06:38.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 --2- 172.21.15.52:0/1774714667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d410f330 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 --2- 172.21.15.52:0/1774714667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5d41084f0 0x7fa5d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 --2- 172.21.15.52:0/1774714667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5d41024e0 0x7fa5d4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 -- 172.21.15.52:0/1774714667 >> 172.21.15.52:0/1774714667 conn(0x7fa5d4079530 msgr2=0x7fa5d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:38.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 -- 172.21.15.52:0/1774714667 shutdown_connections 2024-06-02T23:06:38.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.517+0000 7fa5da25d700 1 -- 172.21.15.52:0/1774714667 wait complete. 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 Processor -- start 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 -- start start 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5d41024e0 0x7fa5d4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5d41084f0 0x7fa5d4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d4112620 con 0x7fa5d4102e80 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41124a0 con 0x7fa5d41024e0 2024-06-02T23:06:38.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5da25d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5d41127a0 con 0x7fa5d41084f0 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5d2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5d2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54892/0 (socket says 172.21.15.52:54892) 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5d2ffd700 1 -- 172.21.15.52:0/2879645176 learned_addr learned my addr 172.21.15.52:0/2879645176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5d37fe700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5d41024e0 0x7fa5d4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.518+0000 7fa5d3fff700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5d41084f0 0x7fa5d4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d2ffd700 1 -- 172.21.15.52:0/2879645176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5d41084f0 msgr2=0x7fa5d4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d2ffd700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5d41084f0 0x7fa5d4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d2ffd700 1 -- 172.21.15.52:0/2879645176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5d41024e0 msgr2=0x7fa5d4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d2ffd700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5d41024e0 0x7fa5d4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d2ffd700 1 -- 172.21.15.52:0/2879645176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5d41a7020 con 0x7fa5d4102e80 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d2ffd700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d4074cd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800e9c0 tx=0x7fa5c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c800cbd0 con 0x7fa5d4102e80 2024-06-02T23:06:38.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5d41a7310 con 0x7fa5d4102e80 2024-06-02T23:06:38.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5c800ee60 con 0x7fa5d4102e80 2024-06-02T23:06:38.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5c80055b0 con 0x7fa5d4102e80 2024-06-02T23:06:38.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.519+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5d41a7850 con 0x7fa5d4102e80 2024-06-02T23:06:38.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.520+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5d4069180 con 0x7fa5d4102e80 2024-06-02T23:06:38.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.521+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fa5c8004750 con 0x7fa5d4102e80 2024-06-02T23:06:38.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.522+0000 7fa5d0ff9700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5bc0a2c20 0x7fa5bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:38.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.522+0000 7fa5d37fe700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5bc0a2c20 0x7fa5bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:38.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.522+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa5c80ceec0 con 0x7fa5d4102e80 2024-06-02T23:06:38.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.522+0000 7fa5d37fe700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5bc0a2c20 0x7fa5bc0a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007b60 tx=0x7fa5c40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:38.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.527+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa5c809c220 con 0x7fa5d4102e80 2024-06-02T23:06:38.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.780+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa5d4075cc0 con 0x7fa5d4102e80 2024-06-02T23:06:38.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.780+0000 7fa5d0ff9700 1 -- 172.21.15.52:0/2879645176 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fa5c809bb20 con 0x7fa5d4102e80 2024-06-02T23:06:38.780 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:38.780 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5bc0a2c20 msgr2=0x7fa5bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5bc0a2c20 0x7fa5bc0a50e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5c4007b60 tx=0x7fa5c40073d0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 msgr2=0x7fa5d4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d4074cd0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fa5c800e9c0 tx=0x7fa5c800ecd0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 shutdown_connections 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5bc0a2c20 0x7fa5bc0a50e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5d41084f0 0x7fa5d4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa5d41024e0 0x7fa5d4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 --2- 172.21.15.52:0/2879645176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5d4102e80 0x7fa5d4074cd0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 >> 172.21.15.52:0/2879645176 conn(0x7fa5d4079530 msgr2=0x7fa5d4100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 shutdown_connections 2024-06-02T23:06:38.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:38.783+0000 7fa5da25d700 1 -- 172.21.15.52:0/2879645176 wait complete. 2024-06-02T23:06:38.783 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 podman[65461]: 2024-06-02 23:06:38.401894124 +0000 UTC m=+0.017267670 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:06:38.783 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 podman[65461]: 2024-06-02 23:06:38.523015764 +0000 UTC m=+0.138389282 container create d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:06:38.783 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 podman[65461]: 2024-06-02 23:06:38.784465083 +0000 UTC m=+0.399838608 container init d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-02T23:06:38.784 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 1 2024-06-02T23:06:39.143 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 podman[65461]: 2024-06-02 23:06:38.787257179 +0000 UTC m=+0.402630704 container start d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-02T23:06:39.143 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 ceph-mds[65480]: -- 172.21.15.174:0/2751881716 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559985e6da40 con 0x559985e40c00 2024-06-02T23:06:39.143 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l[65476]: starting mds.l at 2024-06-02T23:06:39.143 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 bash[65461]: d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 2024-06-02T23:06:39.143 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:06:38 smithi174 systemd[1]: Started Ceph mds.l for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:39.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:39.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2879645176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:39.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6832/224943979,v1:172.21.15.174:6833/224943979] up:boot 2024-06-02T23:06:39.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: fsmap 1 up:standby 2024-06-02T23:06:39.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T23:06:39.855 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.856 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.856 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.856 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:39.856 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:39.856 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2879645176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6832/224943979,v1:172.21.15.174:6833/224943979] up:boot 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: fsmap 1 up:standby 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:39.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2879645176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6832/224943979,v1:172.21.15.174:6833/224943979] up:boot 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: fsmap 1 up:standby 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:40.795 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:40 smithi174 ceph-mon[35994]: Deploying daemon mds.k on smithi123 2024-06-02T23:06:40.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:40 smithi052 ceph-mon[32340]: Deploying daemon mds.k on smithi123 2024-06-02T23:06:40.875 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:40 smithi123 ceph-mon[35977]: Deploying daemon mds.k on smithi123 2024-06-02T23:06:41.257 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 systemd[1]: Starting Ceph mds.k for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:06:41.516 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 podman[64985]: 2024-06-02 23:06:41.166565075 +0000 UTC m=+0.017645139 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:06:41.516 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 podman[64985]: 2024-06-02 23:06:41.351617299 +0000 UTC m=+0.202697383 container create 43be9a76630d842e3dff11148705e5a878cd27e456d2baf71082435fa89c3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-06-02T23:06:41.523 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:06:41.776 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:41.806 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:41 smithi174 ceph-mon[35994]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:41.888 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:41 smithi052 ceph-mon[32340]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:41 smithi123 ceph-mon[35977]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:42.007 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 podman[64985]: 2024-06-02 23:06:41.624965775 +0000 UTC m=+0.476045839 container init 43be9a76630d842e3dff11148705e5a878cd27e456d2baf71082435fa89c3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:06:42.007 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 podman[64985]: 2024-06-02 23:06:41.627837481 +0000 UTC m=+0.478917523 container start 43be9a76630d842e3dff11148705e5a878cd27e456d2baf71082435fa89c3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc) 2024-06-02T23:06:42.007 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 ceph-mds[65004]: -- 172.21.15.123:0/4230245923 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557250f2da40 con 0x557250f00c00 2024-06-02T23:06:42.007 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k[65000]: starting mds.k at 2024-06-02T23:06:42.007 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 bash[64985]: 43be9a76630d842e3dff11148705e5a878cd27e456d2baf71082435fa89c3741 2024-06-02T23:06:42.008 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:06:41 smithi123 systemd[1]: Started Ceph mds.k for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:42.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:42.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:42.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:42.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:42.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:42.892 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:42.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:42.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: Deploying daemon mds.j on smithi052 2024-06-02T23:06:42.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6832/1267486579,v1:172.21.15.123:6833/1267486579] up:boot 2024-06-02T23:06:42.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: fsmap 2 up:standby 2024-06-02T23:06:42.893 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T23:06:43.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:43.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:43.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:43.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: Deploying daemon mds.j on smithi052 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6832/1267486579,v1:172.21.15.123:6833/1267486579] up:boot 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: fsmap 2 up:standby 2024-06-02T23:06:43.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: Deploying daemon mds.j on smithi052 2024-06-02T23:06:43.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6832/1267486579,v1:172.21.15.123:6833/1267486579] up:boot 2024-06-02T23:06:43.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: fsmap 2 up:standby 2024-06-02T23:06:43.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T23:06:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 -- 172.21.15.52:0/3978158113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 msgr2=0x7f8cb81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/3978158113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb81088e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8ca8009990 tx=0x7f8ca8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:06:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 -- 172.21.15.52:0/3978158113 shutdown_connections 2024-06-02T23:06:44.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/3978158113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb806ab30 0x7f8cb810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/3978158113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb81088e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/3978158113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 -- 172.21.15.52:0/3978158113 >> 172.21.15.52:0/3978158113 conn(0x7f8cb8078340 msgr2=0x7f8cb8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 -- 172.21.15.52:0/3978158113 shutdown_connections 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.811+0000 7f8cbca1a700 1 -- 172.21.15.52:0/3978158113 wait complete. 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.812+0000 7f8cbca1a700 1 Processor -- start 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.812+0000 7f8cbca1a700 1 -- start start 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cbca1a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cbca1a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb806ab30 0x7f8cb81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cbca1a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cbca1a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8112690 con 0x7f8cb8108500 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cbca1a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8112510 con 0x7f8cb806a170 2024-06-02T23:06:44.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cbca1a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8cb8112810 con 0x7f8cb806ab30 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb8101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59714/0 (socket says 172.21.15.52:59714) 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb6d9d700 1 -- 172.21.15.52:0/1489771927 learned_addr learned my addr 172.21.15.52:0/1489771927 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45796/0 (socket says 172.21.15.52:45796) 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cadd9b700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb806ab30 0x7f8cb81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 -- 172.21.15.52:0/1489771927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb806ab30 msgr2=0x7f8cb81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb806ab30 0x7f8cb81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 -- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 msgr2=0x7f8cb8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 -- 172.21.15.52:0/1489771927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cb81a70d0 con 0x7f8cb806a170 2024-06-02T23:06:44.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb6d9d700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:06:44.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.813+0000 7f8cb659c700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb8101460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80059d0 tx=0x7f8ca80047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:44.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.814+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8027070 con 0x7f8cb806a170 2024-06-02T23:06:44.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.814+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cb81a7360 con 0x7f8cb806a170 2024-06-02T23:06:44.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.814+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ca8003f10 con 0x7f8cb806a170 2024-06-02T23:06:44.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.814+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8022ba0 con 0x7f8cb806a170 2024-06-02T23:06:44.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.814+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cb81a77b0 con 0x7f8cb806a170 2024-06-02T23:06:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.815+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cb810a9e0 con 0x7f8cb806a170 2024-06-02T23:06:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.816+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8ca802a070 con 0x7f8cb806a170 2024-06-02T23:06:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.816+0000 7f8caffff700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ca40a2c20 0x7f8ca40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.817+0000 7f8cadd9b700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ca40a2c20 0x7f8ca40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.817+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8ca80d0100 con 0x7f8cb806a170 2024-06-02T23:06:44.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.817+0000 7f8cadd9b700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ca40a2c20 0x7f8ca40a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8078a20 tx=0x7f8ca00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:44.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:44.820+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ca8013070 con 0x7f8cb806a170 2024-06-02T23:06:45.051 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:44 smithi052 ceph-mon[32340]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:45.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.084+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cb8069180 con 0x7f8cb806a170 2024-06-02T23:06:45.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.085+0000 7f8caffff700 1 -- 172.21.15.52:0/1489771927 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f8ca809d4f0 con 0x7f8cb806a170 2024-06-02T23:06:45.085 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:45.085 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ca40a2c20 msgr2=0x7f8ca40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ca40a2c20 0x7f8ca40a50e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8078a20 tx=0x7f8ca00073d0 comp rx=0 tx=0).stop 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 msgr2=0x7f8cb8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb8101460 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8ca80059d0 tx=0x7f8ca80047b0 comp rx=0 tx=0).stop 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 shutdown_connections 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ca40a2c20 0x7f8ca40a50e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8cb8108500 0x7f8cb8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cb806a170 0x7f8cb8101460 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 --2- 172.21.15.52:0/1489771927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8cb806ab30 0x7f8cb81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.087+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 >> 172.21.15.52:0/1489771927 conn(0x7f8cb8078340 msgr2=0x7f8cb8100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.088+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 shutdown_connections 2024-06-02T23:06:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:45.088+0000 7f8cbca1a700 1 -- 172.21.15.52:0/1489771927 wait complete. 2024-06-02T23:06:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 3 2024-06-02T23:06:45.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:44 smithi174 ceph-mon[35994]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:45.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:44 smithi123 ceph-mon[35977]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:46.142 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:45 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1489771927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:45 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1489771927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:46.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:45 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1489771927' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:46.399 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:06:46 smithi052 systemd[1]: Started Ceph mds.j for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:47.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:47.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: Deploying daemon mds.d on smithi052 2024-06-02T23:06:47.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.52:6834/3011548911,v1:172.21.15.52:6835/3011548911] up:boot 2024-06-02T23:06:47.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: fsmap 3 up:standby 2024-06-02T23:06:47.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-02T23:06:47.856 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:06:47.886 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:47.886 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: Deploying daemon mds.d on smithi052 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.52:6834/3011548911,v1:172.21.15.52:6835/3011548911] up:boot 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: fsmap 3 up:standby 2024-06-02T23:06:47.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: Deploying daemon mds.d on smithi052 2024-06-02T23:06:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.52:6834/3011548911,v1:172.21.15.52:6835/3011548911] up:boot 2024-06-02T23:06:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: fsmap 3 up:standby 2024-06-02T23:06:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-06-02T23:06:48.118 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:49 smithi123 ceph-mon[35977]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:49.769 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:49 smithi052 ceph-mon[32340]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:49.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 -- 172.21.15.52:0/340562333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f681406a170 msgr2=0x7f681406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 --2- 172.21.15.52:0/340562333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f681406a170 0x7f681406a5d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f68080099c0 tx=0x7f6808009cd0 comp rx=0 tx=0).stop 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 -- 172.21.15.52:0/340562333 shutdown_connections 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 --2- 172.21.15.52:0/340562333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f681406ab10 0x7f681410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 --2- 172.21.15.52:0/340562333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68141084e0 0x7f68141088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 --2- 172.21.15.52:0/340562333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f681406a170 0x7f681406a5d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 -- 172.21.15.52:0/340562333 >> 172.21.15.52:0/340562333 conn(0x7f68140782c0 msgr2=0x7f68140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 -- 172.21.15.52:0/340562333 shutdown_connections 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.875+0000 7f681927d700 1 -- 172.21.15.52:0/340562333 wait complete. 2024-06-02T23:06:49.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 Processor -- start 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 -- start start 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f681406a170 0x7f681419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f681406ab10 0x7f681419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 0x7f6814198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68141125f0 con 0x7f68141084e0 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6814112470 con 0x7f681406a170 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f681927d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6814112770 con 0x7f681406ab10 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f68137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 0x7f6814198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f68137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 0x7f6814198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59758/0 (socket says 172.21.15.52:59758) 2024-06-02T23:06:49.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f68137fe700 1 -- 172.21.15.52:0/2410442622 learned_addr learned my addr 172.21.15.52:0/2410442622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f68127fc700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f681406ab10 0x7f681419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.876+0000 7f6812ffd700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f681406a170 0x7f681419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f68137fe700 1 -- 172.21.15.52:0/2410442622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f681406ab10 msgr2=0x7f681419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f68137fe700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f681406ab10 0x7f681419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f68137fe700 1 -- 172.21.15.52:0/2410442622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f681406a170 msgr2=0x7f681419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f68137fe700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f681406a170 0x7f681419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f68137fe700 1 -- 172.21.15.52:0/2410442622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68141990d0 con 0x7f68141084e0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f68137fe700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 0x7f6814198970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d980 tx=0x7f680c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c016070 con 0x7f68141084e0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68141993c0 con 0x7f68141084e0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f680c004830 con 0x7f68141084e0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f680c0054e0 con 0x7f68141084e0 2024-06-02T23:06:49.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.877+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68141a5680 con 0x7f68141084e0 2024-06-02T23:06:49.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.878+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6814069180 con 0x7f68141084e0 2024-06-02T23:06:49.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.879+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f680c0049e0 con 0x7f68141084e0 2024-06-02T23:06:49.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.880+0000 7f67fbfff700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f67fc0a2c20 0x7f67fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:49.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.880+0000 7f6812ffd700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f67fc0a2c20 0x7f67fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:49.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.880+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f680c0ced80 con 0x7f68141084e0 2024-06-02T23:06:49.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.881+0000 7f6812ffd700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f67fc0a2c20 0x7f67fc0a50e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6814199ab0 tx=0x7f6804009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:49.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:49.884+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f680c01f020 con 0x7f68141084e0 2024-06-02T23:06:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:49 smithi174 ceph-mon[35994]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:50.150 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:50.150 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-06-02T23:06:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.149+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6814199ea0 con 0x7f68141084e0 2024-06-02T23:06:50.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.149+0000 7f67fbfff700 1 -- 172.21.15.52:0/2410442622 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f680c09c180 con 0x7f68141084e0 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f67fc0a2c20 msgr2=0x7f67fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f67fc0a2c20 0x7f67fc0a50e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6814199ab0 tx=0x7f6804009180 comp rx=0 tx=0).stop 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 msgr2=0x7f6814198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 0x7f6814198970 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f680c00d980 tx=0x7f680c00dc90 comp rx=0 tx=0).stop 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 shutdown_connections 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f67fc0a2c20 0x7f67fc0a50e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68141084e0 0x7f6814198970 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f681406a170 0x7f681419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 --2- 172.21.15.52:0/2410442622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f681406ab10 0x7f681419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:50.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.151+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 >> 172.21.15.52:0/2410442622 conn(0x7f68140782c0 msgr2=0x7f68140ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:50.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.152+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 shutdown_connections 2024-06-02T23:06:50.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:50.152+0000 7f681927d700 1 -- 172.21.15.52:0/2410442622 wait complete. 2024-06-02T23:06:50.153 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 4 2024-06-02T23:06:50.736 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:50 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2410442622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:50.736 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:06:50 smithi052 systemd[1]: Started Ceph mds.d for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:50.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:50 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2410442622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:50 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2410442622' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:51.583 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:51.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:51.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:51.584 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:51.847 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:52.642 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:52 smithi123 ceph-mon[35977]: Deploying daemon mds.e on smithi123 2024-06-02T23:06:52.642 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:52 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.52:6836/4013141842,v1:172.21.15.52:6837/4013141842] up:boot 2024-06-02T23:06:52.643 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:52 smithi123 ceph-mon[35977]: fsmap 4 up:standby 2024-06-02T23:06:52.643 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:52 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-02T23:06:52.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:52 smithi052 ceph-mon[32340]: Deploying daemon mds.e on smithi123 2024-06-02T23:06:52.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:52 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.52:6836/4013141842,v1:172.21.15.52:6837/4013141842] up:boot 2024-06-02T23:06:52.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:52 smithi052 ceph-mon[32340]: fsmap 4 up:standby 2024-06-02T23:06:52.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:52 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-02T23:06:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:52 smithi174 ceph-mon[35994]: Deploying daemon mds.e on smithi123 2024-06-02T23:06:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:52 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.52:6836/4013141842,v1:172.21.15.52:6837/4013141842] up:boot 2024-06-02T23:06:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:52 smithi174 ceph-mon[35994]: fsmap 4 up:standby 2024-06-02T23:06:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:52 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-06-02T23:06:53.076 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:06:53.344 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:53.685 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:53 smithi052 ceph-mon[32340]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:53.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:53 smithi123 ceph-mon[35977]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:53.757 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:06:53 smithi123 systemd[1]: Started Ceph mds.e for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:53 smithi174 ceph-mon[35994]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:54.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: Deploying daemon mds.f on smithi174 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] up:boot 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: fsmap 5 up:standby 2024-06-02T23:06:54.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T23:06:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:54.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:54.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: Deploying daemon mds.f on smithi174 2024-06-02T23:06:54.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:54.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] up:boot 2024-06-02T23:06:54.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: fsmap 5 up:standby 2024-06-02T23:06:54.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 -- 172.21.15.52:0/188634472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0100ec0 msgr2=0x7f9fa0101340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/188634472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0100ec0 0x7f9fa0101340 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9f90009990 tx=0x7f9f90009ca0 comp rx=0 tx=0).stop 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 -- 172.21.15.52:0/188634472 shutdown_connections 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/188634472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0101880 0x7f9fa010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/188634472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fa0108670 0x7f9fa0108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/188634472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0100ec0 0x7f9fa0101340 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.926+0000 7f9fa79ce700 1 -- 172.21.15.52:0/188634472 >> 172.21.15.52:0/188634472 conn(0x7f9fa0076600 msgr2=0x7f9fa0076a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.927+0000 7f9fa79ce700 1 -- 172.21.15.52:0/188634472 shutdown_connections 2024-06-02T23:06:54.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.927+0000 7f9fa79ce700 1 -- 172.21.15.52:0/188634472 wait complete. 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.927+0000 7f9fa79ce700 1 Processor -- start 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.927+0000 7f9fa79ce700 1 -- start start 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa79ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0100ec0 0x7f9fa019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa79ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 0x7f9fa019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa79ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fa0108670 0x7f9fa0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa79ce700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0112710 con 0x7f9fa0101880 2024-06-02T23:06:54.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa79ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0112590 con 0x7f9fa0108670 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa79ce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa0112890 con 0x7f9fa0100ec0 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 0x7f9fa019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 0x7f9fa019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53628/0 (socket says 172.21.15.52:53628) 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 -- 172.21.15.52:0/509049515 learned_addr learned my addr 172.21.15.52:0/509049515 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa576a700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0100ec0 0x7f9fa019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 -- 172.21.15.52:0/509049515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0100ec0 msgr2=0x7f9fa019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0100ec0 0x7f9fa019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 -- 172.21.15.52:0/509049515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fa0108670 msgr2=0x7f9fa0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fa0108670 0x7f9fa0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 -- 172.21.15.52:0/509049515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa0199170 con 0x7f9fa0101880 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa576a700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0100ec0 0x7f9fa019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9fa4f69700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 0x7f9fa019ed40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f900047b0 tx=0x7f9f90009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:54.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.928+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90027070 con 0x7f9fa0101880 2024-06-02T23:06:54.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.929+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9f90003f80 con 0x7f9fa0101880 2024-06-02T23:06:54.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.929+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa0199400 con 0x7f9fa0101880 2024-06-02T23:06:54.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.929+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90022ca0 con 0x7f9fa0101880 2024-06-02T23:06:54.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.929+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa01a56d0 con 0x7f9fa0101880 2024-06-02T23:06:54.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.930+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa0069180 con 0x7f9fa0101880 2024-06-02T23:06:54.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.931+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f9f9002a070 con 0x7f9fa0101880 2024-06-02T23:06:54.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.931+0000 7f9f967fc700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9f8c0a2ca0 0x7f9f8c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:54.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.932+0000 7f9fa576a700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9f8c0a2ca0 0x7f9f8c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:54.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.932+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f9f900cfa50 con 0x7f9fa0101880 2024-06-02T23:06:54.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.932+0000 7f9fa576a700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9f8c0a2ca0 0x7f9f8c0a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0108180 tx=0x7f9f9c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:54.936+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9f9005e850 con 0x7f9fa0101880 2024-06-02T23:06:55.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:55.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:55.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:55.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:55.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:55.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:55.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: Deploying daemon mds.f on smithi174 2024-06-02T23:06:55.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:55.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] up:boot 2024-06-02T23:06:55.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: fsmap 5 up:standby 2024-06-02T23:06:55.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T23:06:55.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.190+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9fa002dca0 con 0x7f9fa0101880 2024-06-02T23:06:55.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.191+0000 7f9f967fc700 1 -- 172.21.15.52:0/509049515 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f9f90013070 con 0x7f9fa0101880 2024-06-02T23:06:55.191 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:06:55.191 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9f8c0a2ca0 msgr2=0x7f9f8c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9f8c0a2ca0 0x7f9f8c0a5160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9fa0108180 tx=0x7f9f9c008040 comp rx=0 tx=0).stop 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 msgr2=0x7f9fa019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 0x7f9fa019ed40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9f900047b0 tx=0x7f9f90009550 comp rx=0 tx=0).stop 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 shutdown_connections 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9f8c0a2ca0 0x7f9f8c0a5160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fa0108670 0x7f9fa0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fa0100ec0 0x7f9fa019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:55.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 --2- 172.21.15.52:0/509049515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fa0101880 0x7f9fa019ed40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 >> 172.21.15.52:0/509049515 conn(0x7f9fa0076600 msgr2=0x7f9fa00ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.193+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 shutdown_connections 2024-06-02T23:06:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:55.194+0000 7f9fa79ce700 1 -- 172.21.15.52:0/509049515 wait complete. 2024-06-02T23:06:55.195 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 6 2024-06-02T23:06:55.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:55 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/509049515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:55 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/509049515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:55.893 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:55 smithi174 systemd[1]: Starting Ceph mds.f for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:06:56.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:55 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/509049515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:06:56.255 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:55 smithi174 podman[65763]: 2024-06-02 23:06:55.887216945 +0000 UTC m=+0.016670030 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:06:56.255 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 podman[65763]: 2024-06-02 23:06:56.00230262 +0000 UTC m=+0.131755696 container create 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:06:56.255 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 podman[65763]: 2024-06-02 23:06:56.256278118 +0000 UTC m=+0.385731191 container init 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-06-02T23:06:56.643 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 podman[65763]: 2024-06-02 23:06:56.259099101 +0000 UTC m=+0.388552176 container start 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5) 2024-06-02T23:06:56.643 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 ceph-mds[65782]: -- 172.21.15.174:0/2531447288 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c693069a40 con 0x55c69303cc00 2024-06-02T23:06:56.643 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f[65778]: starting mds.f at 2024-06-02T23:06:56.644 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 bash[65763]: 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 2024-06-02T23:06:56.644 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:06:56 smithi174 systemd[1]: Started Ceph mds.f for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: Deploying daemon mds.c on smithi174 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6834/2486297365,v1:172.21.15.174:6835/2486297365] up:boot 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: fsmap 6 up:standby 2024-06-02T23:06:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T23:06:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: Deploying daemon mds.c on smithi174 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6834/2486297365,v1:172.21.15.174:6835/2486297365] up:boot 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: fsmap 6 up:standby 2024-06-02T23:06:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: Deploying daemon mds.c on smithi174 2024-06-02T23:06:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6834/2486297365,v1:172.21.15.174:6835/2486297365] up:boot 2024-06-02T23:06:57.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: fsmap 6 up:standby 2024-06-02T23:06:57.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T23:06:57.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:06:58.135 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:06:58.643 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:58 smithi174 systemd[1]: Starting Ceph mds.c for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:06:58.995 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:58 smithi174 podman[66068]: 2024-06-02 23:06:58.642714969 +0000 UTC m=+0.016757304 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:06:58.995 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:58 smithi174 podman[66068]: 2024-06-02 23:06:58.767957895 +0000 UTC m=+0.142000218 container create 2ed17d9c38427dc9c777c83636019e611c3a8936cbecdfb48511bb3f4fd2718b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-06-02T23:06:58.995 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:58 smithi174 podman[66068]: 2024-06-02 23:06:58.996134467 +0000 UTC m=+0.370176795 container init 2ed17d9c38427dc9c777c83636019e611c3a8936cbecdfb48511bb3f4fd2718b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:06:59.393 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:58 smithi174 podman[66068]: 2024-06-02 23:06:58.998946238 +0000 UTC m=+0.372988567 container start 2ed17d9c38427dc9c777c83636019e611c3a8936cbecdfb48511bb3f4fd2718b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:06:59.394 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mds[66087]: -- 172.21.15.174:0/3322304827 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2dcc2ba40 con 0x55c2dcbfec00 2024-06-02T23:06:59.394 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c[66083]: starting mds.c at 2024-06-02T23:06:59.394 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:59 smithi174 bash[66068]: 2ed17d9c38427dc9c777c83636019e611c3a8936cbecdfb48511bb3f4fd2718b 2024-06-02T23:06:59.394 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:06:59 smithi174 systemd[1]: Started Ceph mds.c for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:06:59.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:59.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:59.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:59.599 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:06:59 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:06:59 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 -- 172.21.15.52:0/2047010818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da406ab30 msgr2=0x7f8da410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/2047010818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da406ab30 0x7f8da410f320 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009990 tx=0x7f8d9c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 -- 172.21.15.52:0/2047010818 shutdown_connections 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/2047010818 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da406ab30 0x7f8da410f320 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/2047010818 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8da4108500 0x7f8da41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/2047010818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da406a170 0x7f8da406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:59.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 -- 172.21.15.52:0/2047010818 >> 172.21.15.52:0/2047010818 conn(0x7f8da4078340 msgr2=0x7f8da4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 -- 172.21.15.52:0/2047010818 shutdown_connections 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.867+0000 7f8da8cdd700 1 -- 172.21.15.52:0/2047010818 wait complete. 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 Processor -- start 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 -- start start 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da406a170 0x7f8da419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8da406ab30 0x7f8da419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 0x7f8da41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4112610 con 0x7f8da4108500 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4112490 con 0x7f8da406ab30 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.868+0000 7f8da8cdd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8da4112790 con 0x7f8da406a170 2024-06-02T23:06:59.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 0x7f8da41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 0x7f8da41988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53636/0 (socket says 172.21.15.52:53636) 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 -- 172.21.15.52:0/248153127 learned_addr learned my addr 172.21.15.52:0/248153127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da1d9b700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8da406ab30 0x7f8da419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da259c700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da406a170 0x7f8da419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 -- 172.21.15.52:0/248153127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da406a170 msgr2=0x7f8da419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da406a170 0x7f8da419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 -- 172.21.15.52:0/248153127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8da406ab30 msgr2=0x7f8da419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8da406ab30 0x7f8da419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 -- 172.21.15.52:0/248153127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8da4199050 con 0x7f8da4108500 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da2d9d700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 0x7f8da41988f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0047b0 tx=0x7f8d9c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c027070 con 0x7f8da4108500 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d9c022470 con 0x7f8da4108500 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8da41992e0 con 0x7f8da4108500 2024-06-02T23:06:59.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c00b610 con 0x7f8da4108500 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.869+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8da41a5600 con 0x7f8da4108500 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.870+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8da4103bc0 con 0x7f8da4108500 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.871+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f8d9c02a070 con 0x7f8da4108500 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.872+0000 7f8d937fe700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d8c0a2c20 0x7f8d8c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.872+0000 7f8da259c700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d8c0a2c20 0x7f8d8c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.872+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8d9c0cf920 con 0x7f8da4108500 2024-06-02T23:06:59.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.873+0000 7f8da259c700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d8c0a2c20 0x7f8d8c0a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8da4199a30 tx=0x7f8d9400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:06:59.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:06:59.877+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8d9c013070 con 0x7f8da4108500 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:06:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:06:59 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:00.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.129+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8da402dca0 con 0x7f8da4108500 2024-06-02T23:07:00.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.130+0000 7f8d937fe700 1 -- 172.21.15.52:0/248153127 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6352 (secure 0 0 0) 0x7f8d9c09cd10 con 0x7f8da4108500 2024-06-02T23:07:00.130 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:00.130 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-06-02T23:07:00.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d8c0a2c20 msgr2=0x7f8d8c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d8c0a2c20 0x7f8d8c0a50e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f8da4199a30 tx=0x7f8d9400b040 comp rx=0 tx=0).stop 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 msgr2=0x7f8da41988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 0x7f8da41988f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0047b0 tx=0x7f8d9c005e80 comp rx=0 tx=0).stop 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 shutdown_connections 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d8c0a2c20 0x7f8d8c0a50e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8da4108500 0x7f8da41988f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da406a170 0x7f8da419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 --2- 172.21.15.52:0/248153127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8da406ab30 0x7f8da419ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 >> 172.21.15.52:0/248153127 conn(0x7f8da4078340 msgr2=0x7f8da4100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 shutdown_connections 2024-06-02T23:07:00.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:00.132+0000 7f8da8cdd700 1 -- 172.21.15.52:0/248153127 wait complete. 2024-06-02T23:07:00.134 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 8 2024-06-02T23:07:00.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:00 smithi052 ceph-mon[32340]: Deploying daemon mds.h on smithi123 2024-06-02T23:07:00.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:00 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] up:boot 2024-06-02T23:07:00.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:00 smithi052 ceph-mon[32340]: fsmap 7 up:standby 2024-06-02T23:07:00.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:00 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T23:07:00.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:00 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/248153127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:00.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:00 smithi123 ceph-mon[35977]: Deploying daemon mds.h on smithi123 2024-06-02T23:07:00.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:00 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] up:boot 2024-06-02T23:07:00.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:00 smithi123 ceph-mon[35977]: fsmap 7 up:standby 2024-06-02T23:07:00.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:00 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T23:07:00.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:00 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/248153127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:00.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:00 smithi174 ceph-mon[35994]: Deploying daemon mds.h on smithi123 2024-06-02T23:07:00.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:00 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] up:boot 2024-06-02T23:07:00.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:00 smithi174 ceph-mon[35994]: fsmap 7 up:standby 2024-06-02T23:07:00.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:00 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T23:07:00.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:00 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/248153127' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:01.411 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 systemd[1]: Starting Ceph mds.h for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:07:01.707 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:01 smithi123 ceph-mon[35977]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:01.708 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 podman[65605]: 2024-06-02 23:07:01.446646966 +0000 UTC m=+0.017009124 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:07:01.708 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 podman[65605]: 2024-06-02 23:07:01.583124897 +0000 UTC m=+0.153487031 container create da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-02T23:07:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:01 smithi174 ceph-mon[35994]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:01 smithi052 ceph-mon[32340]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:02.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 podman[65605]: 2024-06-02 23:07:01.835905856 +0000 UTC m=+0.406267994 container init da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-02T23:07:02.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 podman[65605]: 2024-06-02 23:07:01.838929858 +0000 UTC m=+0.409291989 container start da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:07:02.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 ceph-mds[65625]: -- 172.21.15.123:0/1703580965 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564931549a40 con 0x56493151d000 2024-06-02T23:07:02.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h[65621]: starting mds.h at 2024-06-02T23:07:02.258 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 bash[65605]: da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b 2024-06-02T23:07:02.258 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:07:01 smithi123 systemd[1]: Started Ceph mds.h for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:07:02.790 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:07:03.052 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:03.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:03.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: Deploying daemon mds.i on smithi174 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] up:boot 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: fsmap 8 up:standby 2024-06-02T23:07:03.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: Deploying daemon mds.i on smithi174 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] up:boot 2024-06-02T23:07:03.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: fsmap 8 up:standby 2024-06-02T23:07:03.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T23:07:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:03.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:03.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:03.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: Deploying daemon mds.i on smithi174 2024-06-02T23:07:03.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] up:boot 2024-06-02T23:07:03.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: fsmap 8 up:standby 2024-06-02T23:07:03.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T23:07:04.388 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 systemd[1]: Starting Ceph mds.i for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:07:04.643 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 podman[66371]: 2024-06-02 23:07:04.291394926 +0000 UTC m=+0.017659877 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:07:04.643 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 podman[66371]: 2024-06-02 23:07:04.407736015 +0000 UTC m=+0.134000998 container create a7cd1c89f433968408ac27630ac56dc5e44cfbcacac7becf05e00e9e18198417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-06-02T23:07:04.643 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 podman[66371]: 2024-06-02 23:07:04.644216182 +0000 UTC m=+0.370481111 container init a7cd1c89f433968408ac27630ac56dc5e44cfbcacac7becf05e00e9e18198417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-06-02T23:07:04.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 -- 172.21.15.52:0/1362152094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 msgr2=0x7fc9dc105360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1362152094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc105360 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400b0a0 tx=0x7fc9d400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 -- 172.21.15.52:0/1362152094 shutdown_connections 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1362152094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc105360 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1362152094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 0x7fc9dc06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1362152094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc06b180 0x7fc9dc06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 -- 172.21.15.52:0/1362152094 >> 172.21.15.52:0/1362152094 conn(0x7fc9dc078430 msgr2=0x7fc9dc078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 -- 172.21.15.52:0/1362152094 shutdown_connections 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.749+0000 7fc9e3067700 1 -- 172.21.15.52:0/1362152094 wait complete. 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 Processor -- start 2024-06-02T23:07:04.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 -- start start 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc06b180 0x7fc9dc19f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc19f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 0x7fc9dc1a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc118c90 con 0x7fc9dc104f00 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc118b10 con 0x7fc9dc06b180 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e3067700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9dc118e10 con 0x7fc9dc1073f0 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc19f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc19f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40624/0 (socket says 172.21.15.52:40624) 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9dbfff700 1 -- 172.21.15.52:0/1285683915 learned_addr learned my addr 172.21.15.52:0/1285683915 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e1604700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 0x7fc9dc1a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e0e03700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc06b180 0x7fc9dc19f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e1604700 1 -- 172.21.15.52:0/1285683915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc06b180 msgr2=0x7fc9dc19f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e1604700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc06b180 0x7fc9dc19f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:04.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e1604700 1 -- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 msgr2=0x7fc9dc19f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e1604700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc19f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9e1604700 1 -- 172.21.15.52:0/1285683915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9dc1a40f0 con 0x7fc9dc1073f0 2024-06-02T23:07:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.750+0000 7fc9dbfff700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:07:04.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.751+0000 7fc9e1604700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 0x7fc9dc1a39b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400b780 tx=0x7fc9d40079a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.751+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d4021020 con 0x7fc9dc1073f0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.751+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9dc1a4380 con 0x7fc9dc1073f0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.751+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9d401d470 con 0x7fc9dc1073f0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.751+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d40052b0 con 0x7fc9dc1073f0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.751+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9dc1abc60 con 0x7fc9dc1073f0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.752+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9dc1019e0 con 0x7fc9dc1073f0 2024-06-02T23:07:04.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.753+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fc9d4024020 con 0x7fc9dc1073f0 2024-06-02T23:07:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.754+0000 7fc9d9ffb700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9cc0a2c20 0x7fc9cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.754+0000 7fc9e0e03700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9cc0a2c20 0x7fc9cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.754+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fc9d404fe00 con 0x7fc9dc1073f0 2024-06-02T23:07:04.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.754+0000 7fc9e0e03700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9cc0a2c20 0x7fc9cc0a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009be0 tx=0x7fc9d0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:04.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:04.757+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc9d409d320 con 0x7fc9dc1073f0 2024-06-02T23:07:04.979 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 podman[66371]: 2024-06-02 23:07:04.647327658 +0000 UTC m=+0.373592598 container start a7cd1c89f433968408ac27630ac56dc5e44cfbcacac7becf05e00e9e18198417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-02T23:07:04.980 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mds[66390]: -- 172.21.15.174:0/3439660452 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563575439a40 con 0x56357540cc00 2024-06-02T23:07:04.980 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i[66386]: starting mds.i at 2024-06-02T23:07:04.980 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 bash[66371]: a7cd1c89f433968408ac27630ac56dc5e44cfbcacac7becf05e00e9e18198417 2024-06-02T23:07:04.980 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:07:04 smithi174 systemd[1]: Started Ceph mds.i for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:07:05.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.019+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9dc069180 con 0x7fc9dc1073f0 2024-06-02T23:07:05.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:05.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.019 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:05.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:05.020 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:04 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:05.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.020+0000 7fc9d9ffb700 1 -- 172.21.15.52:0/1285683915 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8007 (secure 0 0 0) 0x7fc9dc069180 con 0x7fc9dc1073f0 2024-06-02T23:07:05.020 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:05.021 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9cc0a2c20 msgr2=0x7fc9cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9cc0a2c20 0x7fc9cc0a50e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc9d0009be0 tx=0x7fc9d0009260 comp rx=0 tx=0).stop 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 msgr2=0x7fc9dc1a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 0x7fc9dc1a39b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9d400b780 tx=0x7fc9d40079a0 comp rx=0 tx=0).stop 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 shutdown_connections 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9cc0a2c20 0x7fc9cc0a50e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9dc1073f0 0x7fc9dc1a39b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc06b180 0x7fc9dc19f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:05.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 --2- 172.21.15.52:0/1285683915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9dc104f00 0x7fc9dc19f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:05.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 >> 172.21.15.52:0/1285683915 conn(0x7fc9dc078430 msgr2=0x7fc9dc103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:05.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 shutdown_connections 2024-06-02T23:07:05.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:05.023+0000 7fc9e3067700 1 -- 172.21.15.52:0/1285683915 wait complete. 2024-06-02T23:07:05.024 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 10 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:04 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:04 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:06.207 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:05 smithi052 ceph-mon[32340]: Deploying daemon mds.g on smithi052 2024-06-02T23:07:06.207 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:05 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] up:boot 2024-06-02T23:07:06.207 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:05 smithi052 ceph-mon[32340]: fsmap 9 up:standby 2024-06-02T23:07:06.207 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T23:07:06.207 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:05 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1285683915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:06.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:05 smithi123 ceph-mon[35977]: Deploying daemon mds.g on smithi052 2024-06-02T23:07:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:05 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] up:boot 2024-06-02T23:07:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:05 smithi123 ceph-mon[35977]: fsmap 9 up:standby 2024-06-02T23:07:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T23:07:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:05 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1285683915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:05 smithi174 ceph-mon[35994]: Deploying daemon mds.g on smithi052 2024-06-02T23:07:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:05 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] up:boot 2024-06-02T23:07:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:05 smithi174 ceph-mon[35994]: fsmap 9 up:standby 2024-06-02T23:07:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T23:07:06.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:05 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1285683915' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:06 smithi052 ceph-mon[32340]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:07.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:06 smithi123 ceph-mon[35977]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:06 smithi174 ceph-mon[35994]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:07.621 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 systemd[1]: Starting Ceph mds.g for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:07:07.682 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:07:07.900 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 podman[72067]: 2024-06-02 23:07:07.524243832 +0000 UTC m=+0.017911252 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:07:07.900 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 podman[72067]: 2024-06-02 23:07:07.647409428 +0000 UTC m=+0.141076872 container create ce1e151382b0566ee8c6903aff2207e68ace95bb591e926d53d61d6f77d99e3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-02T23:07:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:08.247 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 podman[72067]: 2024-06-02 23:07:07.917955061 +0000 UTC m=+0.411622462 container init ce1e151382b0566ee8c6903aff2207e68ace95bb591e926d53d61d6f77d99e3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-06-02T23:07:08.248 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 podman[72067]: 2024-06-02 23:07:07.921137228 +0000 UTC m=+0.414804632 container start ce1e151382b0566ee8c6903aff2207e68ace95bb591e926d53d61d6f77d99e3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-02T23:07:08.248 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 ceph-mds[72116]: -- 172.21.15.52:0/3901682236 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ffc6a95a40 con 0x55ffc6a68800 2024-06-02T23:07:08.248 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g[72100]: starting mds.g at 2024-06-02T23:07:08.248 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:08 smithi052 bash[72067]: ce1e151382b0566ee8c6903aff2207e68ace95bb591e926d53d61d6f77d99e3c 2024-06-02T23:07:08.248 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:07:08 smithi052 systemd[1]: Started Ceph mds.g for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.52:6838/2175393174,v1:172.21.15.52:6839/2175393174] up:boot 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: fsmap 10 up:standby 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:09 smithi123 ceph-mon[35977]: Deploying daemon mds.a on smithi052 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.52:6838/2175393174,v1:172.21.15.52:6839/2175393174] up:boot 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: fsmap 10 up:standby 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.259 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:09.259 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:09.259 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:09.259 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:08 smithi052 ceph-mon[32340]: Deploying daemon mds.a on smithi052 2024-06-02T23:07:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.52:6838/2175393174,v1:172.21.15.52:6839/2175393174] up:boot 2024-06-02T23:07:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: fsmap 10 up:standby 2024-06-02T23:07:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:09.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:08 smithi174 ceph-mon[35994]: Deploying daemon mds.a on smithi052 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 -- 172.21.15.52:0/3509958196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4107550 msgr2=0x7f4fa4107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 --2- 172.21.15.52:0/3509958196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4107550 0x7f4fa4107930 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009990 tx=0x7f4fa0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 -- 172.21.15.52:0/3509958196 shutdown_connections 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 --2- 172.21.15.52:0/3509958196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa41017f0 0x7f4fa410f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 --2- 172.21.15.52:0/3509958196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4107550 0x7f4fa4107930 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 --2- 172.21.15.52:0/3509958196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa4100e30 0x7f4fa41012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 -- 172.21.15.52:0/3509958196 >> 172.21.15.52:0/3509958196 conn(0x7f4fa40766b0 msgr2=0x7f4fa4076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:09.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 -- 172.21.15.52:0/3509958196 shutdown_connections 2024-06-02T23:07:09.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 -- 172.21.15.52:0/3509958196 wait complete. 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 Processor -- start 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.803+0000 7f4fabe20700 1 -- start start 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 0x7f4fa4102010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa41017f0 0x7f4fa4102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa4107550 0x7f4fa4102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa41126c0 con 0x7f4fa4100e30 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4112540 con 0x7f4fa4107550 2024-06-02T23:07:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fa4112840 con 0x7f4fa41017f0 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 0x7f4fa4102010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 0x7f4fa4102010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40654/0 (socket says 172.21.15.52:40654) 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 -- 172.21.15.52:0/2294515288 learned_addr learned my addr 172.21.15.52:0/2294515288 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa93bb700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa41017f0 0x7f4fa4102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 -- 172.21.15.52:0/2294515288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa41017f0 msgr2=0x7f4fa4102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa41017f0 0x7f4fa4102550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 -- 172.21.15.52:0/2294515288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa4107550 msgr2=0x7f4fa4102a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa4107550 0x7f4fa4102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 -- 172.21.15.52:0/2294515288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fa41a7140 con 0x7f4fa4100e30 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fa9bbc700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 0x7f4fa4102010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0000c00 tx=0x7f4fa0003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0027070 con 0x7f4fa4100e30 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fa000bdc0 con 0x7f4fa4100e30 2024-06-02T23:07:09.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fa41a73d0 con 0x7f4fa4100e30 2024-06-02T23:07:09.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.804+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000f460 con 0x7f4fa4100e30 2024-06-02T23:07:09.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.805+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fa41a7880 con 0x7f4fa4100e30 2024-06-02T23:07:09.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.806+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fa4069180 con 0x7f4fa4100e30 2024-06-02T23:07:09.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.807+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4fa002a070 con 0x7f4fa4100e30 2024-06-02T23:07:09.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.807+0000 7f4f9affd700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f900a2ca0 0x7f4f900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:09.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.807+0000 7f4fa93bb700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f900a2ca0 0x7f4f900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:09.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.807+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f4fa00cf970 con 0x7f4fa4100e30 2024-06-02T23:07:09.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.808+0000 7f4fa93bb700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f900a2ca0 0x7f4f900a5160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f94006fd0 tx=0x7f4f94008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:09.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:09.813+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4fa0013070 con 0x7f4fa4100e30 2024-06-02T23:07:10.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.079+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fa402dca0 con 0x7f4fa4100e30 2024-06-02T23:07:10.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.080+0000 7f4f9affd700 1 -- 172.21.15.52:0/2294515288 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8833 (secure 0 0 0) 0x7f4fa009ce00 con 0x7f4fa4100e30 2024-06-02T23:07:10.080 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:10.080 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.083+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f900a2ca0 msgr2=0x7f4f900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.083+0000 7f4fabe20700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f900a2ca0 0x7f4f900a5160 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4f94006fd0 tx=0x7f4f94008040 comp rx=0 tx=0).stop 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.083+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 msgr2=0x7f4fa4102010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.083+0000 7f4fabe20700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 0x7f4fa4102010 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0000c00 tx=0x7f4fa0003b30 comp rx=0 tx=0).stop 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 shutdown_connections 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f900a2ca0 0x7f4f900a5160 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:10.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fa4107550 0x7f4fa4102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:10.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fa4100e30 0x7f4fa4102010 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:10.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 --2- 172.21.15.52:0/2294515288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fa41017f0 0x7f4fa4102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:10.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 >> 172.21.15.52:0/2294515288 conn(0x7f4fa40766b0 msgr2=0x7f4fa40ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:10.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 shutdown_connections 2024-06-02T23:07:10.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:10.084+0000 7f4fabe20700 1 -- 172.21.15.52:0/2294515288 wait complete. 2024-06-02T23:07:10.086 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 11 2024-06-02T23:07:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:11 smithi052 ceph-mon[32340]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2294515288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:11 smithi123 ceph-mon[35977]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2294515288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:11 smithi174 ceph-mon[35994]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2294515288' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:12.340 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 systemd[1]: Starting Ceph mds.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:07:12.618 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 podman[72537]: 2024-06-02 23:07:12.244051192 +0000 UTC m=+0.018193751 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:07:12.618 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 podman[72537]: 2024-06-02 23:07:12.349245602 +0000 UTC m=+0.123388183 container create ff3a6c7f7e1e0adb1e3230636cd627de968ae4920edd408863f78e5f0c77db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-02T23:07:12.618 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 podman[72537]: 2024-06-02 23:07:12.618859944 +0000 UTC m=+0.393002484 container init ff3a6c7f7e1e0adb1e3230636cd627de968ae4920edd408863f78e5f0c77db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-06-02T23:07:12.899 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 podman[72537]: 2024-06-02 23:07:12.621809102 +0000 UTC m=+0.395951649 container start ff3a6c7f7e1e0adb1e3230636cd627de968ae4920edd408863f78e5f0c77db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528) 2024-06-02T23:07:12.899 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 ceph-mds[72556]: -- 172.21.15.52:0/1077403946 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c637787a40 con 0x55c63775a800 2024-06-02T23:07:12.900 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a[72552]: starting mds.a at 2024-06-02T23:07:12.900 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 bash[72537]: ff3a6c7f7e1e0adb1e3230636cd627de968ae4920edd408863f78e5f0c77db2f 2024-06-02T23:07:12.900 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:07:12 smithi052 systemd[1]: Started Ceph mds.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:07:13.224 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:13.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-06-02T23:07:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:14.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:14 smithi174 ceph-mon[35994]: Deploying daemon mds.b on smithi123 2024-06-02T23:07:14.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:14 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.52:6840/4238066654,v1:172.21.15.52:6841/4238066654] up:boot 2024-06-02T23:07:14.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:14 smithi174 ceph-mon[35994]: fsmap 11 up:standby 2024-06-02T23:07:14.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T23:07:14.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:14 smithi052 ceph-mon[32340]: Deploying daemon mds.b on smithi123 2024-06-02T23:07:14.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:14 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.52:6840/4238066654,v1:172.21.15.52:6841/4238066654] up:boot 2024-06-02T23:07:14.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:14 smithi052 ceph-mon[32340]: fsmap 11 up:standby 2024-06-02T23:07:14.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T23:07:14.455 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:14 smithi123 ceph-mon[35977]: Deploying daemon mds.b on smithi123 2024-06-02T23:07:14.455 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:14 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.52:6840/4238066654,v1:172.21.15.52:6841/4238066654] up:boot 2024-06-02T23:07:14.455 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:14 smithi123 ceph-mon[35977]: fsmap 11 up:standby 2024-06-02T23:07:14.455 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T23:07:15.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 -- 172.21.15.52:0/977959144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43881030f0 msgr2=0x7f4388107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:15.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 --2- 172.21.15.52:0/977959144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43881030f0 0x7f4388107110 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f43700099c0 tx=0x7f4370009cd0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.222 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:15 smithi052 ceph-mon[32340]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 -- 172.21.15.52:0/977959144 shutdown_connections 2024-06-02T23:07:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 --2- 172.21.15.52:0/977959144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 0x7f438810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 --2- 172.21.15.52:0/977959144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4388102740 0x7f4388102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 --2- 172.21.15.52:0/977959144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f43881030f0 0x7f4388107110 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 -- 172.21.15.52:0/977959144 >> 172.21.15.52:0/977959144 conn(0x7f43880795f0 msgr2=0x7f4388079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:15.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 -- 172.21.15.52:0/977959144 shutdown_connections 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.221+0000 7f438ea5d700 1 -- 172.21.15.52:0/977959144 wait complete. 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 Processor -- start 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 -- start start 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 0x7f438819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43881030f0 0x7f438819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 0x7f4388198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4388112600 con 0x7f4388102740 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.222+0000 7f438ea5d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4388112480 con 0x7f4388107840 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f438ea5d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4388112780 con 0x7f43881030f0 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 0x7f438819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 0x7f438819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59098/0 (socket says 172.21.15.52:59098) 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 -- 172.21.15.52:0/3831197492 learned_addr learned my addr 172.21.15.52:0/3831197492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:15.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f438cffa700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 0x7f4388198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f437ffff700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43881030f0 0x7f438819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 -- 172.21.15.52:0/3831197492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43881030f0 msgr2=0x7f438819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43881030f0 0x7f438819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 -- 172.21.15.52:0/3831197492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 msgr2=0x7f4388198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 0x7f4388198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 -- 172.21.15.52:0/3831197492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43881990c0 con 0x7f4388102740 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f438cffa700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 0x7f4388198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4387fff700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 0x7f438819e7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f437800d950 tx=0x7f437800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4378016070 con 0x7f4388102740 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43881993b0 con 0x7f4388102740 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f437800fd40 con 0x7f4388102740 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.223+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437800e3b0 con 0x7f4388102740 2024-06-02T23:07:15.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.224+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43880746c0 con 0x7f4388102740 2024-06-02T23:07:15.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.224+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4388069180 con 0x7f4388102740 2024-06-02T23:07:15.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.226+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f4378019070 con 0x7f4388102740 2024-06-02T23:07:15.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.226+0000 7f4385ffb700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43740a2ca0 0x7f43740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:15.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.227+0000 7f437ffff700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43740a2ca0 0x7f43740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:15.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.227+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f43780ce580 con 0x7f4388102740 2024-06-02T23:07:15.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.227+0000 7f437ffff700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43740a2ca0 0x7f43740a5160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4388102110 tx=0x7f4370005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:15.228 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:15 smithi123 ceph-mon[35977]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:15.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.230+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f437809b980 con 0x7f4388102740 2024-06-02T23:07:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:15 smithi174 ceph-mon[35994]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:15.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.480+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f438802dca0 con 0x7f4388102740 2024-06-02T23:07:15.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.480+0000 7f4385ffb700 1 -- 172.21.15.52:0/3831197492 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9659 (secure 0 0 0) 0x7f437809b280 con 0x7f4388102740 2024-06-02T23:07:15.481 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:15.481 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.482+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43740a2ca0 msgr2=0x7f43740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.482+0000 7f438ea5d700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43740a2ca0 0x7f43740a5160 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4388102110 tx=0x7f4370005fb0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.482+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 msgr2=0x7f438819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.482+0000 7f438ea5d700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 0x7f438819e7a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f437800d950 tx=0x7f437800dc60 comp rx=0 tx=0).stop 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 shutdown_connections 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43740a2ca0 0x7f43740a5160 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4388107840 0x7f4388198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4388102740 0x7f438819e7a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 --2- 172.21.15.52:0/3831197492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f43881030f0 0x7f438819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 >> 172.21.15.52:0/3831197492 conn(0x7f43880795f0 msgr2=0x7f4388100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 shutdown_connections 2024-06-02T23:07:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:15.483+0000 7f438ea5d700 1 -- 172.21.15.52:0/3831197492 wait complete. 2024-06-02T23:07:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 12 2024-06-02T23:07:15.500 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:15 smithi123 podman[65907]: 2024-06-02 23:07:15.477656424 +0000 UTC m=+0.364648661 container start e52435dd9cf18044936a6d15ad75b6a4d5170d0d5777757c249c71a56aa4e910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-06-02T23:07:15.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:15 smithi123 ceph-mds[65926]: -- 172.21.15.123:0/2081347798 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edb789da40 con 0x55edb7870800 2024-06-02T23:07:15.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:15 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: starting mds.b at 2024-06-02T23:07:15.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:15 smithi123 bash[65907]: e52435dd9cf18044936a6d15ad75b6a4d5170d0d5777757c249c71a56aa4e910 2024-06-02T23:07:15.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:15 smithi123 systemd[1]: Started Ceph mds.b for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:07:16.091 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3831197492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:16.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:16 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3831197492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:16 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:07:16.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3831197492' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:16.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:16.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:16 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:07:17.233 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:17 smithi123 ceph-mon[35977]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:17.233 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:17 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] up:boot 2024-06-02T23:07:17.233 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:17 smithi123 ceph-mon[35977]: fsmap 12 up:standby 2024-06-02T23:07:17.233 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T23:07:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:17 smithi174 ceph-mon[35994]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:17 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] up:boot 2024-06-02T23:07:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:17 smithi174 ceph-mon[35994]: fsmap 12 up:standby 2024-06-02T23:07:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T23:07:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:17 smithi052 ceph-mon[32340]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:17 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] up:boot 2024-06-02T23:07:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:17 smithi052 ceph-mon[32340]: fsmap 12 up:standby 2024-06-02T23:07:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T23:07:18.121 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:07:18.373 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.746 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:18 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:07:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:07:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:18.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:18 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:19.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:18 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:07:19.520 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:19 smithi174 ceph-mon[35994]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:19 smithi123 ceph-mon[35977]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:19.809 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:19 smithi052 ceph-mon[32340]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.038+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/3239186488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a4107400 msgr2=0x7fe7a41077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.038+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/3239186488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a4107400 0x7fe7a41077e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe794009990 tx=0x7fe794009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/3239186488 shutdown_connections 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/3239186488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7a4101d60 0x7fe7a410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/3239186488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a4107400 0x7fe7a41077e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/3239186488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a41013c0 0x7fe7a4101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/3239186488 >> 172.21.15.52:0/3239186488 conn(0x7fe7a4078330 msgr2=0x7fe7a4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/3239186488 shutdown_connections 2024-06-02T23:07:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/3239186488 wait complete. 2024-06-02T23:07:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.039+0000 7fe7aa8e3700 1 Processor -- start 2024-06-02T23:07:20.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 -- start start 2024-06-02T23:07:20.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 0x7fe7a419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:20.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7a4101d60 0x7fe7a419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a4107400 0x7fe7a41989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4111510 con 0x7fe7a41013c0 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4111390 con 0x7fe7a4101d60 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7aa8e3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7a4111690 con 0x7fe7a4107400 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7a3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 0x7fe7a419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7a3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 0x7fe7a419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59116/0 (socket says 172.21.15.52:59116) 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7a3fff700 1 -- 172.21.15.52:0/1343897147 learned_addr learned my addr 172.21.15.52:0/1343897147 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7a37fe700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7a4101d60 0x7fe7a419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.040+0000 7fe7a8e80700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a4107400 0x7fe7a41989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a3fff700 1 -- 172.21.15.52:0/1343897147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a4107400 msgr2=0x7fe7a41989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:20.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a3fff700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a4107400 0x7fe7a41989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a3fff700 1 -- 172.21.15.52:0/1343897147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7a4101d60 msgr2=0x7fe7a419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a3fff700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7a4101d60 0x7fe7a419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a3fff700 1 -- 172.21.15.52:0/1343897147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7a4199140 con 0x7fe7a41013c0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a3fff700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 0x7fe7a419e820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c00 tx=0x7fe7940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794027070 con 0x7fe7a41013c0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7a41993d0 con 0x7fe7a41013c0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe794022470 con 0x7fe7a41013c0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7a41a56f0 con 0x7fe7a41013c0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.041+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe794017650 con 0x7fe7a41013c0 2024-06-02T23:07:20.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.042+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7a4102ab0 con 0x7fe7a41013c0 2024-06-02T23:07:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.044+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe79402a070 con 0x7fe7a41013c0 2024-06-02T23:07:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.044+0000 7fe7a17fa700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7900a2c20 0x7fe7900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.045+0000 7fe7a37fe700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7900a2c20 0x7fe7900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:20.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.045+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7940cfe20 con 0x7fe7a41013c0 2024-06-02T23:07:20.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.045+0000 7fe7a37fe700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7900a2c20 0x7fe7900a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4106220 tx=0x7fe78c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:20.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.050+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe794013070 con 0x7fe7a41013c0 2024-06-02T23:07:20.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.330+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe7a4069180 con 0x7fe7a41013c0 2024-06-02T23:07:20.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.331+0000 7fe7a17fa700 1 -- 172.21.15.52:0/1343897147 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10488 (secure 0 0 0) 0x7fe79409d210 con 0x7fe7a41013c0 2024-06-02T23:07:20.331 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:20.331 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24407,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24434,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24451,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24470,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-06-02T23:07:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7900a2c20 msgr2=0x7fe7900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:20.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7900a2c20 0x7fe7900a50e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4106220 tx=0x7fe78c00b3c0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 msgr2=0x7fe7a419e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 0x7fe7a419e820 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fe794000c00 tx=0x7fe7940047b0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 shutdown_connections 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7900a2c20 0x7fe7900a50e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe7a4107400 0x7fe7a41989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe7a41013c0 0x7fe7a419e820 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 --2- 172.21.15.52:0/1343897147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7a4101d60 0x7fe7a419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 >> 172.21.15.52:0/1343897147 conn(0x7fe7a4078330 msgr2=0x7fe7a4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 shutdown_connections 2024-06-02T23:07:20.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:20.333+0000 7fe7aa8e3700 1 -- 172.21.15.52:0/1343897147 wait complete. 2024-06-02T23:07:20.335 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 13 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:20 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1343897147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:21.133 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:20 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1343897147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:20 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1343897147' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:07:21.523 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:22.986 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:22 smithi052 ceph-mon[32340]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:22 smithi174 ceph-mon[35994]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:23.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 -- 172.21.15.52:0/2949865366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41024e0 msgr2=0x7fd5d4102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/2949865366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41024e0 0x7fd5d4102940 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009990 tx=0x7fd5bc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 -- 172.21.15.52:0/2949865366 shutdown_connections 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/2949865366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5d4102e80 0x7fd5d410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/2949865366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41084f0 0x7fd5d41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/2949865366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41024e0 0x7fd5d4102940 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 -- 172.21.15.52:0/2949865366 >> 172.21.15.52:0/2949865366 conn(0x7fd5d4079530 msgr2=0x7fd5d4079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 -- 172.21.15.52:0/2949865366 shutdown_connections 2024-06-02T23:07:23.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.232+0000 7fd5d9f32700 1 -- 172.21.15.52:0/2949865366 wait complete. 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 Processor -- start 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 -- start start 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41024e0 0x7fd5d419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5d4102e80 0x7fd5d419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 0x7fd5d419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4112600 con 0x7fd5d41084f0 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4112480 con 0x7fd5d41024e0 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d9f32700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5d4112780 con 0x7fd5d4102e80 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 0x7fd5d419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 0x7fd5d419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59140/0 (socket says 172.21.15.52:59140) 2024-06-02T23:07:23.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41024e0 0x7fd5d419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5d3fff700 1 -- 172.21.15.52:0/41150862 learned_addr learned my addr 172.21.15.52:0/41150862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.233+0000 7fd5cbfff700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5d4102e80 0x7fd5d419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d37fe700 1 -- 172.21.15.52:0/41150862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5d4102e80 msgr2=0x7fd5d419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d37fe700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5d4102e80 0x7fd5d419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d37fe700 1 -- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 msgr2=0x7fd5d419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d37fe700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 0x7fd5d419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d37fe700 1 -- 172.21.15.52:0/41150862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5d419fda0 con 0x7fd5d41024e0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d3fff700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 0x7fd5d419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d37fe700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41024e0 0x7fd5d419ad90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d750 tx=0x7fd5c400da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4014030 con 0x7fd5d41024e0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5d41a0090 con 0x7fd5d41024e0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd5c40044d0 con 0x7fd5d41024e0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c4005420 con 0x7fd5d41024e0 2024-06-02T23:07:23.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.234+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5d41a7910 con 0x7fd5d41024e0 2024-06-02T23:07:23.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.235+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5d4069180 con 0x7fd5d41024e0 2024-06-02T23:07:23.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.237+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd5c4004750 con 0x7fd5d41024e0 2024-06-02T23:07:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.237+0000 7fd5d17fa700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5c00a2c20 0x7fd5c00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.238+0000 7fd5cbfff700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5c00a2c20 0x7fd5c00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.238+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd5c40ce800 con 0x7fd5d41024e0 2024-06-02T23:07:23.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.238+0000 7fd5cbfff700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5c00a2c20 0x7fd5c00a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00b5c0 tx=0x7fd5bc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:23.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.241+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd5c409bb60 con 0x7fd5d41024e0 2024-06-02T23:07:23.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:22 smithi123 ceph-mon[35977]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:23.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.473+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fd5d41a7b40 con 0x7fd5d41024e0 2024-06-02T23:07:23.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.474+0000 7fd5d17fa700 1 -- 172.21.15.52:0/41150862 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7fd5c409b460 con 0x7fd5d41024e0 2024-06-02T23:07:23.474 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:23.474 INFO:teuthology.orchestra.run.smithi052.stdout:[ 2024-06-02T23:07:23.475 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:07:23.475 INFO:teuthology.orchestra.run.smithi052.stdout: "poolnum": 1, 2024-06-02T23:07:23.475 INFO:teuthology.orchestra.run.smithi052.stdout: "poolname": ".mgr" 2024-06-02T23:07:23.475 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:07:23.475 INFO:teuthology.orchestra.run.smithi052.stdout:] 2024-06-02T23:07:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5c00a2c20 msgr2=0x7fd5c00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5c00a2c20 0x7fd5c00a50e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00b5c0 tx=0x7fd5bc005960 comp rx=0 tx=0).stop 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41024e0 msgr2=0x7fd5d419ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41024e0 0x7fd5d419ad90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5c400d750 tx=0x7fd5c400da60 comp rx=0 tx=0).stop 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 shutdown_connections 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5c00a2c20 0x7fd5c00a50e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd5d41084f0 0x7fd5d419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd5d41024e0 0x7fd5d419ad90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 --2- 172.21.15.52:0/41150862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd5d4102e80 0x7fd5d419b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 >> 172.21.15.52:0/41150862 conn(0x7fd5d4079530 msgr2=0x7fd5d4100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 shutdown_connections 2024-06-02T23:07:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:23.476+0000 7fd5d9f32700 1 -- 172.21.15.52:0/41150862 wait complete. 2024-06-02T23:07:24.216 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-06-02T23:07:24.216 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-06-02T23:07:24.467 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:24.703 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.703 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.703 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:24.703 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:07:24.704 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.704 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:24 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/41150862' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-02T23:07:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:07:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:24 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/41150862' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-02T23:07:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:07:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:07:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:07:24.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:24 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/41150862' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-06-02T23:07:25.602 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:25 smithi052 ceph-mon[32340]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:25.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:25 smithi123 ceph-mon[35977]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:25 smithi174 ceph-mon[35994]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 -- 172.21.15.52:0/3040836697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 msgr2=0x7f62f8105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 --2- 172.21.15.52:0/3040836697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f8105fd0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f62e00099c0 tx=0x7f62e0009cd0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 -- 172.21.15.52:0/3040836697 shutdown_connections 2024-06-02T23:07:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 --2- 172.21.15.52:0/3040836697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f81066b0 0x7f62f810e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 --2- 172.21.15.52:0/3040836697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f8101620 0x7f62f8101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 --2- 172.21.15.52:0/3040836697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f8105fd0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.139+0000 7f62fcd10700 1 -- 172.21.15.52:0/3040836697 >> 172.21.15.52:0/3040836697 conn(0x7f62f8078340 msgr2=0x7f62f8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:26.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.140+0000 7f62fcd10700 1 -- 172.21.15.52:0/3040836697 shutdown_connections 2024-06-02T23:07:26.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.140+0000 7f62fcd10700 1 -- 172.21.15.52:0/3040836697 wait complete. 2024-06-02T23:07:26.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.140+0000 7f62fcd10700 1 Processor -- start 2024-06-02T23:07:26.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.140+0000 7f62fcd10700 1 -- start start 2024-06-02T23:07:26.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62fcd10700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f8101620 0x7f62f819e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:26.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62fcd10700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f819ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62fcd10700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f81066b0 0x7f62f8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62fcd10700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62f8112650 con 0x7f62f8101fd0 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62fcd10700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81124d0 con 0x7f62f81066b0 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62fcd10700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62f81127d0 con 0x7f62f8101620 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f819ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f819ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33956/0 (socket says 172.21.15.52:33956) 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 -- 172.21.15.52:0/4045716966 learned_addr learned my addr 172.21.15.52:0/4045716966 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62f6d9d700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f81066b0 0x7f62f8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62f659c700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f8101620 0x7f62f819e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 -- 172.21.15.52:0/4045716966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f8101620 msgr2=0x7f62f819e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f8101620 0x7f62f819e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 -- 172.21.15.52:0/4045716966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f81066b0 msgr2=0x7f62f8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:26.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f81066b0 0x7f62f8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 -- 172.21.15.52:0/4045716966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62f8199170 con 0x7f62f8101fd0 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62f6d9d700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f81066b0 0x7f62f8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.141+0000 7f62edd9b700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f819ed90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62e00045e0 tx=0x7f62e0004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.142+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0026070 con 0x7f62f8101fd0 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.142+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f62e000f460 con 0x7f62f8101fd0 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.142+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62f8199400 con 0x7f62f8101fd0 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.142+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0009e60 con 0x7f62f8101fd0 2024-06-02T23:07:26.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.142+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62f81a5720 con 0x7f62f8101fd0 2024-06-02T23:07:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.143+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62f8102ab0 con 0x7f62f8101fd0 2024-06-02T23:07:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.144+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f62e0029070 con 0x7f62f8101fd0 2024-06-02T23:07:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.145+0000 7f62effff700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62e40a2c20 0x7f62e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.145+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f62e00cf2c0 con 0x7f62f8101fd0 2024-06-02T23:07:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.145+0000 7f62f659c700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62e40a2c20 0x7f62e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:26.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.145+0000 7f62f659c700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62e40a2c20 0x7f62e40a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005950 tx=0x7f62e800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:26.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.149+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f62e0013070 con 0x7f62f8101fd0 2024-06-02T23:07:26.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.356+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f62f802dca0 con 0x7f62f8101fd0 2024-06-02T23:07:26.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.844+0000 7f62effff700 1 -- 172.21.15.52:0/4045716966 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7f62e009c750 con 0x7f62f8101fd0 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62e40a2c20 msgr2=0x7f62e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62e40a2c20 0x7f62e40a50e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f62e8005950 tx=0x7f62e800a300 comp rx=0 tx=0).stop 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 msgr2=0x7f62f819ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f819ed90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f62e00045e0 tx=0x7f62e0004610 comp rx=0 tx=0).stop 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 shutdown_connections 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62e40a2c20 0x7f62e40a50e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62f81066b0 0x7f62f8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62f8101620 0x7f62f819e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 --2- 172.21.15.52:0/4045716966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f62f8101fd0 0x7f62f819ed90 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 >> 172.21.15.52:0/4045716966 conn(0x7f62f8078340 msgr2=0x7f62f80ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 shutdown_connections 2024-06-02T23:07:26.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:26.847+0000 7f62fcd10700 1 -- 172.21.15.52:0/4045716966 wait complete. 2024-06-02T23:07:26.849 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' created 2024-06-02T23:07:26.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:26 smithi052 ceph-mon[32340]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:26.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:26 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4045716966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-02T23:07:27.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:26 smithi174 ceph-mon[35994]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:27.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:26 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4045716966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-02T23:07:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:26 smithi123 ceph-mon[35977]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:26 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4045716966' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-06-02T23:07:27.536 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-06-02T23:07:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4045716966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-02T23:07:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:27 smithi174 ceph-mon[35994]: osdmap e82: 12 total, 12 up, 12 in 2024-06-02T23:07:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:27 smithi174 ceph-mon[35994]: osdmap e83: 12 total, 12 up, 12 in 2024-06-02T23:07:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4045716966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-02T23:07:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:27 smithi052 ceph-mon[32340]: osdmap e82: 12 total, 12 up, 12 in 2024-06-02T23:07:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:27 smithi052 ceph-mon[32340]: osdmap e83: 12 total, 12 up, 12 in 2024-06-02T23:07:28.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4045716966' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-06-02T23:07:28.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:27 smithi123 ceph-mon[35977]: osdmap e82: 12 total, 12 up, 12 in 2024-06-02T23:07:28.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:27 smithi123 ceph-mon[35977]: osdmap e83: 12 total, 12 up, 12 in 2024-06-02T23:07:28.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:28 smithi052 ceph-mon[32340]: pgmap v291: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:28.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:28 smithi052 ceph-mon[32340]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T23:07:28.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:28 smithi052 ceph-mon[32340]: osdmap e84: 12 total, 12 up, 12 in 2024-06-02T23:07:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:28 smithi174 ceph-mon[35994]: pgmap v291: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:28 smithi174 ceph-mon[35994]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T23:07:29.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:28 smithi174 ceph-mon[35994]: osdmap e84: 12 total, 12 up, 12 in 2024-06-02T23:07:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:28 smithi123 ceph-mon[35977]: pgmap v291: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:28 smithi123 ceph-mon[35977]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-06-02T23:07:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:28 smithi123 ceph-mon[35977]: osdmap e84: 12 total, 12 up, 12 in 2024-06-02T23:07:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 -- 172.21.15.52:0/659877378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94101370 msgr2=0x7fcf941017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 --2- 172.21.15.52:0/659877378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94101370 0x7fcf941017f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c009990 tx=0x7fcf7c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 -- 172.21.15.52:0/659877378 shutdown_connections 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 --2- 172.21.15.52:0/659877378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 0x7fcf9410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 --2- 172.21.15.52:0/659877378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf94107420 0x7fcf94107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 --2- 172.21.15.52:0/659877378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94101370 0x7fcf941017f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 -- 172.21.15.52:0/659877378 >> 172.21.15.52:0/659877378 conn(0x7fcf94078340 msgr2=0x7fcf94078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 -- 172.21.15.52:0/659877378 shutdown_connections 2024-06-02T23:07:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.484+0000 7fcf99c33700 1 -- 172.21.15.52:0/659877378 wait complete. 2024-06-02T23:07:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 Processor -- start 2024-06-02T23:07:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 -- start start 2024-06-02T23:07:29.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf94101370 0x7fcf94197880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:29.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 0x7fcf94197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:29.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 0x7fcf94198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf941115b0 con 0x7fcf94107420 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94111430 con 0x7fcf94101d30 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf99c33700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf94111730 con 0x7fcf94101370 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 0x7fcf94198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 0x7fcf94198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33976/0 (socket says 172.21.15.52:33976) 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 -- 172.21.15.52:0/2650916570 learned_addr learned my addr 172.21.15.52:0/2650916570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf8affd700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 0x7fcf94197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf937fe700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf94101370 0x7fcf94197880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 -- 172.21.15.52:0/2650916570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf94101370 msgr2=0x7fcf94197880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf94101370 0x7fcf94197880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 -- 172.21.15.52:0/2650916570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 msgr2=0x7fcf94197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 0x7fcf94197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf93fff700 1 -- 172.21.15.52:0/2650916570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf941a70f0 con 0x7fcf94107420 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.485+0000 7fcf8affd700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 0x7fcf94197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.486+0000 7fcf93fff700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 0x7fcf94198300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00ef30 tx=0x7fcf8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.486+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c0092e0 con 0x7fcf94107420 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.486+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf8c004bb0 con 0x7fcf94107420 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.486+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf8c004060 con 0x7fcf94107420 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.486+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf941a7380 con 0x7fcf94107420 2024-06-02T23:07:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.486+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf941a7870 con 0x7fcf94107420 2024-06-02T23:07:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.487+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf94069180 con 0x7fcf94107420 2024-06-02T23:07:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.488+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fcf8c01b030 con 0x7fcf94107420 2024-06-02T23:07:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.489+0000 7fcf917fa700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf800a2c20 0x7fcf800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.489+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7497+0+0 (secure 0 0 0) 0x7fcf8c0cf280 con 0x7fcf94107420 2024-06-02T23:07:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.489+0000 7fcf937fe700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf800a2c20 0x7fcf800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.489+0000 7fcf937fe700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf800a2c20 0x7fcf800a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009a10 tx=0x7fcf84009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:29.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.493+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcf8c022020 con 0x7fcf94107420 2024-06-02T23:07:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:29.700+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fcf9402dca0 con 0x7fcf94107420 2024-06-02T23:07:30.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.189+0000 7fcf917fa700 1 -- 172.21.15.52:0/2650916570 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v85) ==== 170+0+0 (secure 0 0 0) 0x7fcf8c010040 con 0x7fcf94107420 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf800a2c20 msgr2=0x7fcf800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf800a2c20 0x7fcf800a50e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009a10 tx=0x7fcf84009190 comp rx=0 tx=0).stop 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 msgr2=0x7fcf94198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 0x7fcf94198300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c00ef30 tx=0x7fcf8c00c3a0 comp rx=0 tx=0).stop 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 shutdown_connections 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf800a2c20 0x7fcf800a50e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf94107420 0x7fcf94198300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf94101370 0x7fcf94197880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:30.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 --2- 172.21.15.52:0/2650916570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf94101d30 0x7fcf94197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:30.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 >> 172.21.15.52:0/2650916570 conn(0x7fcf94078340 msgr2=0x7fcf94100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:30.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 shutdown_connections 2024-06-02T23:07:30.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:30.192+0000 7fcf99c33700 1 -- 172.21.15.52:0/2650916570 wait complete. 2024-06-02T23:07:30.194 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' created 2024-06-02T23:07:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:30 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2650916570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T23:07:30.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:30 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2650916570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T23:07:30.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:30 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2650916570' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-06-02T23:07:30.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-06-02T23:07:31.156 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:31 smithi123 ceph-mon[35977]: pgmap v294: 33 pgs: 3 creating+peering, 12 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:31 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2650916570' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-02T23:07:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:31 smithi123 ceph-mon[35977]: osdmap e85: 12 total, 12 up, 12 in 2024-06-02T23:07:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:31 smithi123 ceph-mon[35977]: osdmap e86: 12 total, 12 up, 12 in 2024-06-02T23:07:31.544 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:31 smithi052 ceph-mon[32340]: pgmap v294: 33 pgs: 3 creating+peering, 12 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:31.544 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:31 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2650916570' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-02T23:07:31.544 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:31 smithi052 ceph-mon[32340]: osdmap e85: 12 total, 12 up, 12 in 2024-06-02T23:07:31.544 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:31 smithi052 ceph-mon[32340]: osdmap e86: 12 total, 12 up, 12 in 2024-06-02T23:07:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:31 smithi174 ceph-mon[35994]: pgmap v294: 33 pgs: 3 creating+peering, 12 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:31 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2650916570' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-06-02T23:07:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:31 smithi174 ceph-mon[35994]: osdmap e85: 12 total, 12 up, 12 in 2024-06-02T23:07:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:31 smithi174 ceph-mon[35994]: osdmap e86: 12 total, 12 up, 12 in 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 -- 172.21.15.52:0/3922810987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 msgr2=0x7f9a0c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 --2- 172.21.15.52:0/3922810987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c10f280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009990 tx=0x7f9a04009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 -- 172.21.15.52:0/3922810987 shutdown_connections 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 --2- 172.21.15.52:0/3922810987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c10f280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 --2- 172.21.15.52:0/3922810987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c107570 0x7f9a0c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 --2- 172.21.15.52:0/3922810987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c100e50 0x7f9a0c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:33.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 -- 172.21.15.52:0/3922810987 >> 172.21.15.52:0/3922810987 conn(0x7f9a0c0766b0 msgr2=0x7f9a0c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 -- 172.21.15.52:0/3922810987 shutdown_connections 2024-06-02T23:07:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.032+0000 7f9a12261700 1 -- 172.21.15.52:0/3922810987 wait complete. 2024-06-02T23:07:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 Processor -- start 2024-06-02T23:07:33.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 -- start start 2024-06-02T23:07:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c100e50 0x7f9a0c1a08a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:33.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c1a0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c107570 0x7f9a0c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c112660 con 0x7f9a0c101810 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1124e0 con 0x7f9a0c100e50 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a12261700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0c1127e0 con 0x7f9a0c107570 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a02ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c1a0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a02ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c1a0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33990/0 (socket says 172.21.15.52:33990) 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.033+0000 7f9a02ffd700 1 -- 172.21.15.52:0/1401589884 learned_addr learned my addr 172.21.15.52:0/1401589884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a0b7fe700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c100e50 0x7f9a0c1a08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a0bfff700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c107570 0x7f9a0c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a02ffd700 1 -- 172.21.15.52:0/1401589884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c107570 msgr2=0x7f9a0c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a02ffd700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c107570 0x7f9a0c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:33.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a02ffd700 1 -- 172.21.15.52:0/1401589884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c100e50 msgr2=0x7f9a0c1a08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a02ffd700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c100e50 0x7f9a0c1a08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a02ffd700 1 -- 172.21.15.52:0/1401589884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0c19b1c0 con 0x7f9a0c101810 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a02ffd700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c1a0de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c920 tx=0x7f99f400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4004a30 con 0x7f9a0c101810 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f99f40054d0 con 0x7f9a0c101810 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a0c19b4b0 con 0x7f9a0c101810 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99f4007c30 con 0x7f9a0c101810 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.034+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a0c1a7770 con 0x7f9a0c101810 2024-06-02T23:07:33.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.035+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a0c069180 con 0x7f9a0c101810 2024-06-02T23:07:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.037+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f99f4004bd0 con 0x7f9a0c101810 2024-06-02T23:07:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.038+0000 7f9a097fa700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f99f80a2c20 0x7f99f80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.038+0000 7f9a0b7fe700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f99f80a2c20 0x7f99f80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.038+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 7879+0+0 (secure 0 0 0) 0x7f99f40cef10 con 0x7f9a0c101810 2024-06-02T23:07:33.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.038+0000 7f9a0b7fe700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f99f80a2c20 0x7f99f80a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c19bba0 tx=0x7f99fc007a30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:33.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.042+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f99f4014030 con 0x7f9a0c101810 2024-06-02T23:07:33.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:33.294+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f9a0c19bf90 con 0x7f9a0c101810 2024-06-02T23:07:33.294 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:33 smithi052 ceph-mon[32340]: pgmap v297: 97 pgs: 23 creating+peering, 27 active+clean, 47 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:33.295 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:33 smithi052 ceph-mon[32340]: osdmap e87: 12 total, 12 up, 12 in 2024-06-02T23:07:33.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:33 smithi123 ceph-mon[35977]: pgmap v297: 97 pgs: 23 creating+peering, 27 active+clean, 47 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:33.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:33 smithi123 ceph-mon[35977]: osdmap e87: 12 total, 12 up, 12 in 2024-06-02T23:07:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:33 smithi174 ceph-mon[35994]: pgmap v297: 97 pgs: 23 creating+peering, 27 active+clean, 47 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:33 smithi174 ceph-mon[35994]: osdmap e87: 12 total, 12 up, 12 in 2024-06-02T23:07:34.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.200+0000 7f9a097fa700 1 -- 172.21.15.52:0/1401589884 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f99f409bf80 con 0x7f9a0c101810 2024-06-02T23:07:34.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.203+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f99f80a2c20 msgr2=0x7f99f80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:34.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.203+0000 7f9a12261700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f99f80a2c20 0x7f99f80a50e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9a0c19bba0 tx=0x7f99fc007a30 comp rx=0 tx=0).stop 2024-06-02T23:07:34.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.203+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 msgr2=0x7f9a0c1a0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:34.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.203+0000 7f9a12261700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c1a0de0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99f400c920 tx=0x7f99f400cc30 comp rx=0 tx=0).stop 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 shutdown_connections 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f99f80a2c20 0x7f99f80a50e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a0c107570 0x7f9a0c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a0c100e50 0x7f9a0c1a08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 --2- 172.21.15.52:0/1401589884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a0c101810 0x7f9a0c1a0de0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 >> 172.21.15.52:0/1401589884 conn(0x7f9a0c0766b0 msgr2=0x7f9a0c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 shutdown_connections 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:34.204+0000 7f9a12261700 1 -- 172.21.15.52:0/1401589884 wait complete. 2024-06-02T23:07:34.205 INFO:teuthology.orchestra.run.smithi052.stderr:new fs with metadata pool 2 and data pool 3 2024-06-02T23:07:34.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:34 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1401589884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-02T23:07:34.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:34 smithi123 ceph-mon[35977]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-02T23:07:34.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:34 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1401589884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-02T23:07:34.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:34 smithi052 ceph-mon[32340]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-02T23:07:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:34 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1401589884' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-06-02T23:07:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:34 smithi174 ceph-mon[35994]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-06-02T23:07:35.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:07:35.292 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:35 smithi052 ceph-mon[32340]: pgmap v299: 97 pgs: 20 creating+peering, 47 active+clean, 30 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:35 smithi052 ceph-mon[32340]: osdmap e88: 12 total, 12 up, 12 in 2024-06-02T23:07:35.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:35 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1401589884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-02T23:07:35.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:35 smithi052 ceph-mon[32340]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-02T23:07:35.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:35 smithi052 ceph-mon[32340]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-02T23:07:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:35 smithi123 ceph-mon[35977]: pgmap v299: 97 pgs: 20 creating+peering, 47 active+clean, 30 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:35 smithi123 ceph-mon[35977]: osdmap e88: 12 total, 12 up, 12 in 2024-06-02T23:07:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:35 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1401589884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-02T23:07:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:35 smithi123 ceph-mon[35977]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-02T23:07:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:35 smithi123 ceph-mon[35977]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-02T23:07:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:35 smithi174 ceph-mon[35994]: pgmap v299: 97 pgs: 20 creating+peering, 47 active+clean, 30 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:07:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:35 smithi174 ceph-mon[35994]: osdmap e88: 12 total, 12 up, 12 in 2024-06-02T23:07:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:35 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1401589884' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-06-02T23:07:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:35 smithi174 ceph-mon[35994]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-06-02T23:07:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:35 smithi174 ceph-mon[35994]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-06-02T23:07:36.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:36 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] up:active 2024-06-02T23:07:36.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:36 smithi123 ceph-mon[35977]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:36.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:36 smithi123 ceph-mon[35977]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:36.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:36 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:07:36.023+0000 7f7cb53cf700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-06-02T23:07:36.530 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:36 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] up:active 2024-06-02T23:07:36.530 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:36 smithi052 ceph-mon[32340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:36.530 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:36 smithi052 ceph-mon[32340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:36.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:36 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] up:active 2024-06-02T23:07:36.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:36 smithi174 ceph-mon[35994]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:36.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:36 smithi174 ceph-mon[35994]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:37.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 -- 172.21.15.52:0/525011021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad01085f0 msgr2=0x7f3ad01089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 --2- 172.21.15.52:0/525011021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad01085f0 0x7f3ad01089d0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009990 tx=0x7f3ac0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 -- 172.21.15.52:0/525011021 shutdown_connections 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 --2- 172.21.15.52:0/525011021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad0102ea0 0x7f3ad010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 --2- 172.21.15.52:0/525011021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad01085f0 0x7f3ad01089d0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 --2- 172.21.15.52:0/525011021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01024e0 0x7f3ad0102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 -- 172.21.15.52:0/525011021 >> 172.21.15.52:0/525011021 conn(0x7f3ad0079640 msgr2=0x7f3ad0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 -- 172.21.15.52:0/525011021 shutdown_connections 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.104+0000 7f3ad543b700 1 -- 172.21.15.52:0/525011021 wait complete. 2024-06-02T23:07:37.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 Processor -- start 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 -- start start 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad01024e0 0x7f3ad0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 0x7f3ad0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01085f0 0x7f3ad0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01126c0 con 0x7f3ad0102ea0 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112540 con 0x7f3ad01024e0 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ad543b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112840 con 0x7f3ad01085f0 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ace7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 0x7f3ad0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ace7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 0x7f3ad0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58398/0 (socket says 172.21.15.52:58398) 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3ace7fc700 1 -- 172.21.15.52:0/4129703475 learned_addr learned my addr 172.21.15.52:0/4129703475 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3aceffd700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad01024e0 0x7f3ad0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:37.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.105+0000 7f3acf7fe700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01085f0 0x7f3ad0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ace7fc700 1 -- 172.21.15.52:0/4129703475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01085f0 msgr2=0x7f3ad0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ace7fc700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01085f0 0x7f3ad0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ace7fc700 1 -- 172.21.15.52:0/4129703475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad01024e0 msgr2=0x7f3ad0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ace7fc700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad01024e0 0x7f3ad0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ace7fc700 1 -- 172.21.15.52:0/4129703475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad01a70a0 con 0x7f3ad0102ea0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3acf7fe700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01085f0 0x7f3ad0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3aceffd700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad01024e0 0x7f3ad0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ace7fc700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 0x7f3ad0074cd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400b530 tx=0x7f3ac400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4015020 con 0x7f3ad0102ea0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a7390 con 0x7f3ad0102ea0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ac4011ce0 con 0x7f3ad0102ea0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac4010e40 con 0x7f3ad0102ea0 2024-06-02T23:07:37.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.106+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a7920 con 0x7f3ad0102ea0 2024-06-02T23:07:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.107+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad010aa40 con 0x7f3ad0102ea0 2024-06-02T23:07:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.109+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f3ac4018020 con 0x7f3ad0102ea0 2024-06-02T23:07:37.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.109+0000 7f3ab7fff700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3ab80a2c20 0x7f3ab80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.110+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f3ac40ced70 con 0x7f3ad0102ea0 2024-06-02T23:07:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.110+0000 7f3aceffd700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3ab80a2c20 0x7f3ab80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:37.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.110+0000 7f3aceffd700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3ab80a2c20 0x7f3ab80a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000b5c0 tx=0x7f3ac0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:37.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.113+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3ac401f020 con 0x7f3ad0102ea0 2024-06-02T23:07:37.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.322+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3ad0075cc0 con 0x7f3ad0102ea0 2024-06-02T23:07:37.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.323+0000 7f3ab7fff700 1 -- 172.21.15.52:0/4129703475 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22205 (secure 0 0 0) 0x7f3ac4010380 con 0x7f3ad0102ea0 2024-06-02T23:07:37.324 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:37.324 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":88,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:07:33.296112+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:07:37.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3ab80a2c20 msgr2=0x7f3ab80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:37.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3ab80a2c20 0x7f3ab80a50e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ac000b5c0 tx=0x7f3ac0005960 comp rx=0 tx=0).stop 2024-06-02T23:07:37.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 msgr2=0x7f3ad0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 0x7f3ad0074cd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3ac400b530 tx=0x7f3ac400b840 comp rx=0 tx=0).stop 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 shutdown_connections 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3ab80a2c20 0x7f3ab80a50e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad01085f0 0x7f3ad0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad01024e0 0x7f3ad0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 --2- 172.21.15.52:0/4129703475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0102ea0 0x7f3ad0074cd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 >> 172.21.15.52:0/4129703475 conn(0x7f3ad0079640 msgr2=0x7f3ad0101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 shutdown_connections 2024-06-02T23:07:37.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:37.326+0000 7f3ad543b700 1 -- 172.21.15.52:0/4129703475 wait complete. 2024-06-02T23:07:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:37 smithi052 ceph-mon[32340]: pgmap v301: 97 pgs: 14 creating+peering, 83 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-06-02T23:07:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:37 smithi052 ceph-mon[32340]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-02T23:07:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:37 smithi052 ceph-mon[32340]: Cluster is now healthy 2024-06-02T23:07:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:37 smithi123 ceph-mon[35977]: pgmap v301: 97 pgs: 14 creating+peering, 83 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-06-02T23:07:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:37 smithi123 ceph-mon[35977]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-02T23:07:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:37 smithi123 ceph-mon[35977]: Cluster is now healthy 2024-06-02T23:07:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:37 smithi174 ceph-mon[35994]: pgmap v301: 97 pgs: 14 creating+peering, 83 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 5 op/s 2024-06-02T23:07:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:37 smithi174 ceph-mon[35994]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-06-02T23:07:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:37 smithi174 ceph-mon[35994]: Cluster is now healthy 2024-06-02T23:07:38.109 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:07:38.363 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:38.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:38 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4129703475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:07:38.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:38 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4129703475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:07:38.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:38 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4129703475' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:07:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:39 smithi123 ceph-mon[35977]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-06-02T23:07:39.530 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:39 smithi052 ceph-mon[32340]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-06-02T23:07:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:39 smithi174 ceph-mon[35994]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-06-02T23:07:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 -- 172.21.15.52:0/827839835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8241030f0 msgr2=0x7fd824107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:40.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 --2- 172.21.15.52:0/827839835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8241030f0 0x7fd824107110 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd8180099c0 tx=0x7fd818009cd0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 -- 172.21.15.52:0/827839835 shutdown_connections 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 --2- 172.21.15.52:0/827839835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd824107840 0x7fd82410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 --2- 172.21.15.52:0/827839835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd824102740 0x7fd824102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 --2- 172.21.15.52:0/827839835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8241030f0 0x7fd824107110 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 -- 172.21.15.52:0/827839835 >> 172.21.15.52:0/827839835 conn(0x7fd8240795f0 msgr2=0x7fd824079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 -- 172.21.15.52:0/827839835 shutdown_connections 2024-06-02T23:07:40.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.066+0000 7fd828ad4700 1 -- 172.21.15.52:0/827839835 wait complete. 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 Processor -- start 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 -- start start 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd824102740 0x7fd82419e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8241030f0 0x7fd82419eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 0x7fd824198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd824112620 con 0x7fd824107840 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8241124a0 con 0x7fd824102740 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd828ad4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8241127a0 con 0x7fd8241030f0 2024-06-02T23:07:40.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 0x7fd824198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 0x7fd824198920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58414/0 (socket says 172.21.15.52:58414) 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 -- 172.21.15.52:0/1444362361 learned_addr learned my addr 172.21.15.52:0/1444362361 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd82259c700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd824102740 0x7fd82419e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd821d9b700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8241030f0 0x7fd82419eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 -- 172.21.15.52:0/1444362361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8241030f0 msgr2=0x7fd82419eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8241030f0 0x7fd82419eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 -- 172.21.15.52:0/1444362361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd824102740 msgr2=0x7fd82419e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd824102740 0x7fd82419e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.067+0000 7fd822d9d700 1 -- 172.21.15.52:0/1444362361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd824199080 con 0x7fd824107840 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.068+0000 7fd822d9d700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 0x7fd824198920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd81c007ee0 tx=0x7fd81c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.068+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c004bb0 con 0x7fd824107840 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.068+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd81c004d50 con 0x7fd824107840 2024-06-02T23:07:40.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.068+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0058e0 con 0x7fd824107840 2024-06-02T23:07:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.068+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd824199370 con 0x7fd824107840 2024-06-02T23:07:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.068+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8241a5630 con 0x7fd824107840 2024-06-02T23:07:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.069+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82410aa10 con 0x7fd824107840 2024-06-02T23:07:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.070+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd81c01a020 con 0x7fd824107840 2024-06-02T23:07:40.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.071+0000 7fd8137fe700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd80c0a2ca0 0x7fd80c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.071+0000 7fd82259c700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd80c0a2ca0 0x7fd80c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.071+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd81c0ce5d0 con 0x7fd824107840 2024-06-02T23:07:40.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.071+0000 7fd82259c700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd80c0a2ca0 0x7fd80c0a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd824199a60 tx=0x7fd81400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:40.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.076+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd81c010070 con 0x7fd824107840 2024-06-02T23:07:40.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.283+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd82410a3f0 con 0x7fd824107840 2024-06-02T23:07:40.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.285+0000 7fd8137fe700 1 -- 172.21.15.52:0/1444362361 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+22205 (secure 0 0 0) 0x7fd81c09b680 con 0x7fd824107840 2024-06-02T23:07:40.284 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:07:40.285 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":88,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:07:33.296112+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:07:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd80c0a2ca0 msgr2=0x7fd80c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:40.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd80c0a2ca0 0x7fd80c0a5160 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd824199a60 tx=0x7fd81400b3c0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 msgr2=0x7fd824198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 0x7fd824198920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd81c007ee0 tx=0x7fd81c00d3b0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 shutdown_connections 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd80c0a2ca0 0x7fd80c0a5160 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd824107840 0x7fd824198920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd824102740 0x7fd82419e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 --2- 172.21.15.52:0/1444362361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8241030f0 0x7fd82419eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 >> 172.21.15.52:0/1444362361 conn(0x7fd8240795f0 msgr2=0x7fd824100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 shutdown_connections 2024-06-02T23:07:40.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:40.288+0000 7fd828ad4700 1 -- 172.21.15.52:0/1444362361 wait complete. 2024-06-02T23:07:40.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-06-02T23:07:41.240 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:41.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:41 smithi052 ceph-mon[32340]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T23:07:41.471 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:41 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1444362361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:07:41.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:41 smithi123 ceph-mon[35977]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T23:07:41.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:41 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1444362361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:07:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:41 smithi174 ceph-mon[35994]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-06-02T23:07:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:41 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1444362361' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/1805468886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb81085d0 msgr2=0x7f5bb81089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/1805468886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb81085d0 0x7f5bb81089b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ba80099c0 tx=0x7f5ba8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/1805468886 shutdown_connections 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/1805468886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb8102ef0 0x7f5bb810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/1805468886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb81085d0 0x7f5bb81089b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/1805468886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb8102530 0x7f5bb81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/1805468886 >> 172.21.15.52:0/1805468886 conn(0x7f5bb8079640 msgr2=0x7f5bb8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/1805468886 shutdown_connections 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/1805468886 wait complete. 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 Processor -- start 2024-06-02T23:07:43.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.004+0000 7f5bbe5b8700 1 -- start start 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbe5b8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb8102530 0x7f5bb8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbe5b8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 0x7f5bb8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbe5b8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb81085d0 0x7f5bb8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbe5b8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb81126c0 con 0x7f5bb8102ef0 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbe5b8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8112540 con 0x7f5bb81085d0 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbe5b8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb8112840 con 0x7f5bb8102530 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 0x7f5bb8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 0x7f5bb8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58440/0 (socket says 172.21.15.52:58440) 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 -- 172.21.15.52:0/2163545051 learned_addr learned my addr 172.21.15.52:0/2163545051 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bb7fff700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb8102530 0x7f5bb8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bbcb55700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb81085d0 0x7f5bb8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 -- 172.21.15.52:0/2163545051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb8102530 msgr2=0x7f5bb8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:43.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb8102530 0x7f5bb8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 -- 172.21.15.52:0/2163545051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb81085d0 msgr2=0x7f5bb8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb81085d0 0x7f5bb8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 -- 172.21.15.52:0/2163545051 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb81a70a0 con 0x7f5bb8102ef0 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5baffff700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 0x7f5bb8074cd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000d950 tx=0x7f5ba000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba0016070 con 0x7f5bb8102ef0 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.005+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba0009df0 con 0x7f5bb8102ef0 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.006+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb81a7390 con 0x7f5bb8102ef0 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.006+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba000f3c0 con 0x7f5bb8102ef0 2024-06-02T23:07:43.007 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.006+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb81a78d0 con 0x7f5bb8102ef0 2024-06-02T23:07:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.006+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb810aa60 con 0x7f5bb8102ef0 2024-06-02T23:07:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.008+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f5ba0019070 con 0x7f5bb8102ef0 2024-06-02T23:07:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.009+0000 7f5bb5ffb700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba40a2c20 0x7f5ba40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.009+0000 7f5bb7fff700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba40a2c20 0x7f5ba40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.009+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f5ba00ceb00 con 0x7f5bb8102ef0 2024-06-02T23:07:43.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.009+0000 7f5bb7fff700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba40a2c20 0x7f5ba40a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8000c00 tx=0x7f5ba8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:43.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.014+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5ba009bbb0 con 0x7f5bb8102ef0 2024-06-02T23:07:43.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:43.267+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5bb8069180 con 0x7f5bb8102ef0 2024-06-02T23:07:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:43 smithi123 ceph-mon[35977]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-02T23:07:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:43 smithi174 ceph-mon[35994]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-02T23:07:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:43 smithi052 ceph-mon[32340]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-06-02T23:07:44.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.228+0000 7f5bb5ffb700 1 -- 172.21.15.52:0/2163545051 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f5ba009b4b0 con 0x7f5bb8102ef0 2024-06-02T23:07:44.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba40a2c20 msgr2=0x7f5ba40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:44.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba40a2c20 0x7f5ba40a50e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8000c00 tx=0x7f5ba8005fb0 comp rx=0 tx=0).stop 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 msgr2=0x7f5bb8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 0x7f5bb8074cd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5ba000d950 tx=0x7f5ba000dc60 comp rx=0 tx=0).stop 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 shutdown_connections 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba40a2c20 0x7f5ba40a50e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb81085d0 0x7f5bb8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb8102530 0x7f5bb8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 --2- 172.21.15.52:0/2163545051 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb8102ef0 0x7f5bb8074cd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 >> 172.21.15.52:0/2163545051 conn(0x7f5bb8079640 msgr2=0x7f5bb8101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 shutdown_connections 2024-06-02T23:07:44.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:44.231+0000 7f5bbe5b8700 1 -- 172.21.15.52:0/2163545051 wait complete. 2024-06-02T23:07:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:44 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2163545051' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T23:07:44.546 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:44 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2163545051' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T23:07:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:44 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2163545051' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-06-02T23:07:44.983 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated'}} 2024-06-02T23:07:44.984 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-06-02T23:07:45.244 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:45 smithi123 ceph-mon[35977]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-02T23:07:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:45 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2163545051' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-02T23:07:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:45 smithi123 ceph-mon[35977]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:45.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:45 smithi052 ceph-mon[32340]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-02T23:07:45.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:45 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2163545051' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-02T23:07:45.536 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:45 smithi052 ceph-mon[32340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:45 smithi174 ceph-mon[35994]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-06-02T23:07:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:45 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2163545051' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-06-02T23:07:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:45 smithi174 ceph-mon[35994]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:47.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 -- 172.21.15.52:0/1388225293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81241066b0 msgr2=0x7f812410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:47.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 --2- 172.21.15.52:0/1388225293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81241066b0 0x7f812410e280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f811c00b0d0 tx=0x7f811c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:07:47.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 -- 172.21.15.52:0/1388225293 shutdown_connections 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 --2- 172.21.15.52:0/1388225293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81241066b0 0x7f812410e280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 --2- 172.21.15.52:0/1388225293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8124101620 0x7f8124101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 --2- 172.21.15.52:0/1388225293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8124101fd0 0x7f8124105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 -- 172.21.15.52:0/1388225293 >> 172.21.15.52:0/1388225293 conn(0x7f8124078340 msgr2=0x7f8124078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 -- 172.21.15.52:0/1388225293 shutdown_connections 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.078+0000 7f812a835700 1 -- 172.21.15.52:0/1388225293 wait complete. 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.079+0000 7f812a835700 1 Processor -- start 2024-06-02T23:07:47.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.079+0000 7f812a835700 1 -- start start 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f812a835700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8124101620 0x7f812419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f812a835700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 0x7f812419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f812a835700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81241066b0 0x7f812419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f812a835700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81241115c0 con 0x7f8124101fd0 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f812a835700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8124111440 con 0x7f81241066b0 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f812a835700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8124111740 con 0x7f8124101620 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 0x7f812419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:47.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 0x7f812419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59450/0 (socket says 172.21.15.52:59450) 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 -- 172.21.15.52:0/612781798 learned_addr learned my addr 172.21.15.52:0/612781798 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f8128dd2700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81241066b0 0x7f812419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f8123fff700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8124101620 0x7f812419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 -- 172.21.15.52:0/612781798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8124101620 msgr2=0x7f812419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8124101620 0x7f812419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 -- 172.21.15.52:0/612781798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81241066b0 msgr2=0x7f812419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81241066b0 0x7f812419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 -- 172.21.15.52:0/612781798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812419fd40 con 0x7f8124101fd0 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f8128dd2700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81241066b0 0x7f812419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.080+0000 7f81237fe700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 0x7f812419b270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f811800c9e0 tx=0x7f811800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.081+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118007aa0 con 0x7f8124101fd0 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.081+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8118007c40 con 0x7f8124101fd0 2024-06-02T23:07:47.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.081+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241a0030 con 0x7f8124101fd0 2024-06-02T23:07:47.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.081+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8118018660 con 0x7f8124101fd0 2024-06-02T23:07:47.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.081+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81241a78b0 con 0x7f8124101fd0 2024-06-02T23:07:47.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.082+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8124069180 con 0x7f8124101fd0 2024-06-02T23:07:47.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.083+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7f811800f450 con 0x7f8124101fd0 2024-06-02T23:07:47.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.084+0000 7f81217fa700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f810c0a2c20 0x7f810c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:47.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.084+0000 7f8123fff700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f810c0a2c20 0x7f810c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:47.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.084+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f81180ceec0 con 0x7f8124101fd0 2024-06-02T23:07:47.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.084+0000 7f8123fff700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f810c0a2c20 0x7f810c0a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8114009a10 tx=0x7f8114009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:47.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.089+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8118014030 con 0x7f8124101fd0 2024-06-02T23:07:47.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:47.346+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f81241a7b40 con 0x7f8124101fd0 2024-06-02T23:07:47.346 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:47 smithi052 ceph-mon[32340]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-02T23:07:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:47 smithi123 ceph-mon[35977]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-02T23:07:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:47 smithi174 ceph-mon[35994]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-06-02T23:07:48.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.238+0000 7f81217fa700 1 -- 172.21.15.52:0/612781798 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f811809bf70 con 0x7f8124101fd0 2024-06-02T23:07:48.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f810c0a2c20 msgr2=0x7f810c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f810c0a2c20 0x7f810c0a50e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8114009a10 tx=0x7f8114009190 comp rx=0 tx=0).stop 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 msgr2=0x7f812419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 0x7f812419b270 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f811800c9e0 tx=0x7f811800ccf0 comp rx=0 tx=0).stop 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 shutdown_connections 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f810c0a2c20 0x7f810c0a50e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81241066b0 0x7f812419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8124101620 0x7f812419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 --2- 172.21.15.52:0/612781798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8124101fd0 0x7f812419b270 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:48.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.242+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 >> 172.21.15.52:0/612781798 conn(0x7f8124078340 msgr2=0x7f812406ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.243+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 shutdown_connections 2024-06-02T23:07:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:48.243+0000 7f812a835700 1 -- 172.21.15.52:0/612781798 wait complete. 2024-06-02T23:07:48.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:48 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/612781798' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-02T23:07:48.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:48 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/612781798' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-02T23:07:48.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:48 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/612781798' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-06-02T23:07:48.949 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-06-02T23:07:49.210 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:49.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-06-02T23:07:49.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-02T23:07:49.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/612781798' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-02T23:07:49.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-02T23:07:49.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:49.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-02T23:07:49.488 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:49 smithi052 ceph-mon[32340]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-02T23:07:49.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-06-02T23:07:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-02T23:07:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/612781798' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-02T23:07:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-02T23:07:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-02T23:07:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:49 smithi123 ceph-mon[35977]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-02T23:07:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: pgmap v307: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-06-02T23:07:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-06-02T23:07:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/612781798' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-06-02T23:07:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-06-02T23:07:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-06-02T23:07:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-06-02T23:07:49.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:49 smithi174 ceph-mon[35994]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-06-02T23:07:50.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:50 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] up:active 2024-06-02T23:07:50.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:50 smithi052 ceph-mon[32340]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-02T23:07:50.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:50 smithi052 ceph-mon[32340]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-02T23:07:50.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:50 smithi052 ceph-mon[32340]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-02T23:07:50.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:50 smithi052 ceph-mon[32340]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-02T23:07:50.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:50 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] up:active 2024-06-02T23:07:50.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:50 smithi123 ceph-mon[35977]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-02T23:07:50.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:50 smithi123 ceph-mon[35977]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-02T23:07:50.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:50 smithi123 ceph-mon[35977]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-02T23:07:50.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:50 smithi123 ceph-mon[35977]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-02T23:07:50.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:50 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] up:active 2024-06-02T23:07:50.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:50 smithi174 ceph-mon[35994]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-06-02T23:07:50.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:50 smithi174 ceph-mon[35994]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-06-02T23:07:50.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:50 smithi174 ceph-mon[35994]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-06-02T23:07:50.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:50 smithi174 ceph-mon[35994]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1761553383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 msgr2=0x7fd9cc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1761553383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc10f3c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd9c400b0d0 tx=0x7fd9c400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1761553383 shutdown_connections 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1761553383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc10f3c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1761553383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 0x7fd9cc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1761553383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9cc1030f0 0x7fd9cc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1761553383 >> 172.21.15.52:0/1761553383 conn(0x7fd9cc0795f0 msgr2=0x7fd9cc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1761553383 shutdown_connections 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.053+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1761553383 wait complete. 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.054+0000 7fd9d13ae700 1 Processor -- start 2024-06-02T23:07:51.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.054+0000 7fd9d13ae700 1 -- start start 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.054+0000 7fd9d13ae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 0x7fd9cc19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.054+0000 7fd9d13ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9cc1030f0 0x7fd9cc19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9d13ae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9d13ae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc112690 con 0x7fd9cc107840 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9d13ae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc112510 con 0x7fd9cc1030f0 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9d13ae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9cc112810 con 0x7fd9cc102740 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59466/0 (socket says 172.21.15.52:59466) 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/1605113372 learned_addr learned my addr 172.21.15.52:0/1605113372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9c27fc700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9cc1030f0 0x7fd9cc19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9caffd700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 0x7fd9cc19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/1605113372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 msgr2=0x7fd9cc19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 0x7fd9cc19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/1605113372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9cc1030f0 msgr2=0x7fd9cc19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9cc1030f0 0x7fd9cc19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/1605113372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9cc1990b0 con 0x7fd9cc107840 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9caffd700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 0x7fd9cc19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9cb7fe700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc198950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c401f070 con 0x7fd9cc107840 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9cc199340 con 0x7fd9cc107840 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9c4003c00 con 0x7fd9cc107840 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c401b770 con 0x7fd9cc107840 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.055+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9cc1a5610 con 0x7fd9cc107840 2024-06-02T23:07:51.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.056+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9cc069180 con 0x7fd9cc107840 2024-06-02T23:07:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.058+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fd9c4022070 con 0x7fd9cc107840 2024-06-02T23:07:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.059+0000 7fd9c8ff9700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9b80a2ca0 0x7fd9b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.059+0000 7fd9caffd700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9b80a2ca0 0x7fd9b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:51.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.059+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fd9c400f020 con 0x7fd9cc107840 2024-06-02T23:07:51.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.060+0000 7fd9caffd700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9b80a2ca0 0x7fd9b80a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc107340 tx=0x7fd9bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:51.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.063+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd9c4029020 con 0x7fd9cc107840 2024-06-02T23:07:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.319+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fd9cc10aa80 con 0x7fd9cc107840 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:07:51.023+0000 7f7cb53cf700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-mon[35977]: pgmap v308: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] up:active 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-mon[35977]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-mon[35977]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-mon[35977]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-06-02T23:07:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:51 smithi123 ceph-mon[35977]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-02T23:07:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:51 smithi174 ceph-mon[35994]: pgmap v308: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-06-02T23:07:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:51 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] up:active 2024-06-02T23:07:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:51 smithi174 ceph-mon[35994]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-02T23:07:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:51 smithi174 ceph-mon[35994]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-02T23:07:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:51 smithi174 ceph-mon[35994]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-06-02T23:07:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:51 smithi174 ceph-mon[35994]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-02T23:07:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:51 smithi052 ceph-mon[32340]: pgmap v308: 97 pgs: 97 active+clean; 581 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s wr, 2 op/s 2024-06-02T23:07:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:51 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] up:active 2024-06-02T23:07:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:51 smithi052 ceph-mon[32340]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-06-02T23:07:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:51 smithi052 ceph-mon[32340]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-06-02T23:07:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:51 smithi052 ceph-mon[32340]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-06-02T23:07:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:51 smithi052 ceph-mon[32340]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-06-02T23:07:51.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.852+0000 7fd9c8ff9700 1 -- 172.21.15.52:0/1605113372 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v26) ==== 121+0+0 (secure 0 0 0) 0x7fd9c409d680 con 0x7fd9cc107840 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9b80a2ca0 msgr2=0x7fd9b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9b80a2ca0 0x7fd9b80a5160 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc107340 tx=0x7fd9bc008040 comp rx=0 tx=0).stop 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 msgr2=0x7fd9cc198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc198950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4009fd0 tx=0x7fd9c4007a20 comp rx=0 tx=0).stop 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 shutdown_connections 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9b80a2ca0 0x7fd9b80a5160 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9cc107840 0x7fd9cc198950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9cc102740 0x7fd9cc19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 --2- 172.21.15.52:0/1605113372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9cc1030f0 0x7fd9cc19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:51.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.855+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 >> 172.21.15.52:0/1605113372 conn(0x7fd9cc0795f0 msgr2=0x7fd9cc100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:51.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.856+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 shutdown_connections 2024-06-02T23:07:51.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:51.856+0000 7fd9d13ae700 1 -- 172.21.15.52:0/1605113372 wait complete. 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] up:active 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: Cluster is now healthy 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1605113372' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T23:07:52.548 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1605113372' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-02T23:07:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] up:active 2024-06-02T23:07:52.549 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:52 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:07:52.583 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-06-02T23:07:52.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] up:active 2024-06-02T23:07:52.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: Cluster is now healthy 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1605113372' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1605113372' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] up:active 2024-06-02T23:07:52.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:52 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] up:active 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: Cluster is now healthy 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1605113372' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1605113372' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-06-02T23:07:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] up:active 2024-06-02T23:07:52.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:52 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:07:52.841 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:53 smithi174 ceph-mon[35994]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-06-02T23:07:53.646 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:53 smithi052 ceph-mon[32340]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-06-02T23:07:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:53 smithi123 ceph-mon[35977]: pgmap v309: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 -- 172.21.15.52:0/276779662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac107400 msgr2=0x7fe6ac1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 --2- 172.21.15.52:0/276779662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac107400 0x7fe6ac1077e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe69c009990 tx=0x7fe69c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 -- 172.21.15.52:0/276779662 shutdown_connections 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 --2- 172.21.15.52:0/276779662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6ac101d50 0x7fe6ac10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 --2- 172.21.15.52:0/276779662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac107400 0x7fe6ac1077e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 --2- 172.21.15.52:0/276779662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac101390 0x7fe6ac101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 -- 172.21.15.52:0/276779662 >> 172.21.15.52:0/276779662 conn(0x7fe6ac078340 msgr2=0x7fe6ac078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 -- 172.21.15.52:0/276779662 shutdown_connections 2024-06-02T23:07:54.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.546+0000 7fe6b117e700 1 -- 172.21.15.52:0/276779662 wait complete. 2024-06-02T23:07:54.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 Processor -- start 2024-06-02T23:07:54.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 -- start start 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6ac101390 0x7fe6ac103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 0x7fe6ac1035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac107400 0x7fe6ac103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac1115a0 con 0x7fe6ac101d50 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac111420 con 0x7fe6ac107400 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6b117e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe6ac111720 con 0x7fe6ac101390 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.547+0000 7fe6a259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 0x7fe6ac1035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 0x7fe6ac1035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38342/0 (socket says 172.21.15.52:38342) 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 -- 172.21.15.52:0/4089916504 learned_addr learned my addr 172.21.15.52:0/4089916504 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6aad9d700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6ac101390 0x7fe6ac103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6ab59e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac107400 0x7fe6ac103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:54.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 -- 172.21.15.52:0/4089916504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6ac101390 msgr2=0x7fe6ac103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6ac101390 0x7fe6ac103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 -- 172.21.15.52:0/4089916504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac107400 msgr2=0x7fe6ac103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac107400 0x7fe6ac103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 -- 172.21.15.52:0/4089916504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6ac1a70e0 con 0x7fe6ac101d50 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6ab59e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac107400 0x7fe6ac103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a259c700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 0x7fe6ac1035b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe69400e9a0 tx=0x7fe69400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69400cbd0 con 0x7fe6ac101d50 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe69400cd70 con 0x7fe6ac101d50 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6ac1a73d0 con 0x7fe6ac101d50 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694018680 con 0x7fe6ac101d50 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.548+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6ac1a7910 con 0x7fe6ac101d50 2024-06-02T23:07:54.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.549+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6ac069180 con 0x7fe6ac101d50 2024-06-02T23:07:54.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.551+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7fe694010430 con 0x7fe6ac101d50 2024-06-02T23:07:54.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.551+0000 7fe6a8d99700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe6980a2c20 0x7fe6980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:54.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.552+0000 7fe6aad9d700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe6980a2c20 0x7fe6980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:54.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.552+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fe6940cf5d0 con 0x7fe6ac101d50 2024-06-02T23:07:54.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.552+0000 7fe6aad9d700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe6980a2c20 0x7fe6980a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b2e0 tx=0x7fe69c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:54.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.555+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe69409c680 con 0x7fe6ac101d50 2024-06-02T23:07:54.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:54.809+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe6ac1a7bc0 con 0x7fe6ac101d50 2024-06-02T23:07:55.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.279+0000 7fe6a8d99700 1 -- 172.21.15.52:0/4089916504 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v27) ==== 115+0+0 (secure 0 0 0) 0x7fe69409bf80 con 0x7fe6ac101d50 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.281+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe6980a2c20 msgr2=0x7fe6980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.281+0000 7fe6b117e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe6980a2c20 0x7fe6980a50e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe69c00b2e0 tx=0x7fe69c005960 comp rx=0 tx=0).stop 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.281+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 msgr2=0x7fe6ac1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.281+0000 7fe6b117e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 0x7fe6ac1035b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe69400e9a0 tx=0x7fe69400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.281+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 shutdown_connections 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe6980a2c20 0x7fe6980a50e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6ac107400 0x7fe6ac103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6ac101390 0x7fe6ac103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:55.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 --2- 172.21.15.52:0/4089916504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe6ac101d50 0x7fe6ac1035b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 >> 172.21.15.52:0/4089916504 conn(0x7fe6ac078340 msgr2=0x7fe6ac100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 shutdown_connections 2024-06-02T23:07:55.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:55.282+0000 7fe6b117e700 1 -- 172.21.15.52:0/4089916504 wait complete. 2024-06-02T23:07:55.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:55 smithi052 ceph-mon[32340]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-02T23:07:55.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:55 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4089916504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-02T23:07:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:55 smithi174 ceph-mon[35994]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-02T23:07:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:55 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4089916504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-02T23:07:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:55 smithi123 ceph-mon[35977]: pgmap v310: 97 pgs: 97 active+clean; 582 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s rd, 2.2 KiB/s wr, 9 op/s 2024-06-02T23:07:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:55 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4089916504' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-06-02T23:07:55.999 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated'} subvols for filesystem 'cephfs' 2024-06-02T23:07:55.999 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-06-02T23:07:56.256 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:56.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:56 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4089916504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-02T23:07:56.609 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:56 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:07:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:56 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4089916504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-02T23:07:56.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:56 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:07:56.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:56 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4089916504' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-06-02T23:07:56.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:56 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:07:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:57 smithi052 ceph-mon[32340]: pgmap v311: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-06-02T23:07:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:57 smithi174 ceph-mon[35994]: pgmap v311: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-06-02T23:07:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:57 smithi123 ceph-mon[35977]: pgmap v311: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-06-02T23:07:58.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 -- 172.21.15.52:0/3789260459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101620 msgr2=0x7efbfc101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 --2- 172.21.15.52:0/3789260459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101620 0x7efbfc101a00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efbf80099c0 tx=0x7efbf8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 -- 172.21.15.52:0/3789260459 shutdown_connections 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 --2- 172.21.15.52:0/3789260459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbfc1066b0 0x7efbfc10e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 --2- 172.21.15.52:0/3789260459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101620 0x7efbfc101a00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 --2- 172.21.15.52:0/3789260459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbfc101fd0 0x7efbfc105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 -- 172.21.15.52:0/3789260459 >> 172.21.15.52:0/3789260459 conn(0x7efbfc078340 msgr2=0x7efbfc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 -- 172.21.15.52:0/3789260459 shutdown_connections 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.071+0000 7efc03918700 1 -- 172.21.15.52:0/3789260459 wait complete. 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 Processor -- start 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 -- start start 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbfc101620 0x7efbfc19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 0x7efbfc19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbfc1066b0 0x7efbfc198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:58.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc111570 con 0x7efbfc101fd0 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1113f0 con 0x7efbfc1066b0 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc03918700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efbfc1116f0 con 0x7efbfc101620 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 0x7efbfc19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 0x7efbfc19ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38366/0 (socket says 172.21.15.52:38366) 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 -- 172.21.15.52:0/961087577 learned_addr learned my addr 172.21.15.52:0/961087577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc016b4700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbfc101620 0x7efbfc19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc01eb5700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbfc1066b0 0x7efbfc198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 -- 172.21.15.52:0/961087577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbfc101620 msgr2=0x7efbfc19e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbfc101620 0x7efbfc19e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 -- 172.21.15.52:0/961087577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbfc1066b0 msgr2=0x7efbfc198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:58.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbfc1066b0 0x7efbfc198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.072+0000 7efc00eb3700 1 -- 172.21.15.52:0/961087577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbfc1991c0 con 0x7efbfc101fd0 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.073+0000 7efc00eb3700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 0x7efbfc19ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efbec00d980 tx=0x7efbec00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.073+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec017070 con 0x7efbfc101fd0 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.073+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efbec009de0 con 0x7efbfc101fd0 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.073+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbfc1994b0 con 0x7efbfc101fd0 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.073+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbec00f3c0 con 0x7efbfc101fd0 2024-06-02T23:07:58.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.073+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbfc1a5770 con 0x7efbfc101fd0 2024-06-02T23:07:58.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.074+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbfc069180 con 0x7efbfc101fd0 2024-06-02T23:07:58.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.076+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 16) ==== 137833+0+0 (secure 0 0 0) 0x7efbec009930 con 0x7efbfc101fd0 2024-06-02T23:07:58.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.076+0000 7efbf27fc700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efbe80a2ca0 0x7efbe80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:07:58.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.077+0000 7efc016b4700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efbe80a2ca0 0x7efbe80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:07:58.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.077+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7efbec0cea70 con 0x7efbfc101fd0 2024-06-02T23:07:58.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.077+0000 7efc016b4700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efbe80a2ca0 0x7efbe80a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000c00 tx=0x7efbf8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:07:58.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.080+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efbec09bb00 con 0x7efbfc101fd0 2024-06-02T23:07:58.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.321+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7efbfc0628d0 con 0x7efbe80a2ca0 2024-06-02T23:07:58.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.349+0000 7efbf27fc700 1 -- 172.21.15.52:0/961087577 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7efbfc0628d0 con 0x7efbe80a2ca0 2024-06-02T23:07:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efbe80a2ca0 msgr2=0x7efbe80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efbe80a2ca0 0x7efbe80a5160 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7efbf8000c00 tx=0x7efbf8005fb0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 msgr2=0x7efbfc19ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:07:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 0x7efbfc19ede0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efbec00d980 tx=0x7efbec00dc90 comp rx=0 tx=0).stop 2024-06-02T23:07:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 shutdown_connections 2024-06-02T23:07:58.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efbe80a2ca0 0x7efbe80a5160 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efbfc1066b0 0x7efbfc198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efbfc101620 0x7efbfc19e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 --2- 172.21.15.52:0/961087577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efbfc101fd0 0x7efbfc19ede0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:07:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 >> 172.21.15.52:0/961087577 conn(0x7efbfc078340 msgr2=0x7efbfc06b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:07:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 shutdown_connections 2024-06-02T23:07:58.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:07:58.352+0000 7efc03918700 1 -- 172.21.15.52:0/961087577 wait complete. 2024-06-02T23:07:58.431 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:07:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:07:58.326+0000 7f2827110700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T23:07:58.431 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:07:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:07:58.326+0000 7f2827110700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T23:07:58.432 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:07:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:07:58.326+0000 7f2827110700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T23:07:58.432 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:07:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:07:58.326+0000 7f2827110700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T23:07:58.432 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:07:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:07:58.326+0000 7f2827110700 -1 client.0 error registering admin socket command: (17) File exists 2024-06-02T23:07:59.001 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-06-02T23:07:59.257 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:07:59.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:59 smithi052 ceph-mon[32340]: pgmap v312: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-02T23:07:59.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:07:59 smithi052 ceph-mon[32340]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:07:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:59 smithi174 ceph-mon[35994]: pgmap v312: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-02T23:07:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:07:59 smithi174 ceph-mon[35994]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:07:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:59 smithi123 ceph-mon[35977]: pgmap v312: 97 pgs: 97 active+clean; 584 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-06-02T23:07:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:07:59 smithi123 ceph-mon[35977]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 -- 172.21.15.52:0/3405018766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 msgr2=0x7fb828102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/3405018766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828102b50 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb8240099c0 tx=0x7fb824009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 -- 172.21.15.52:0/3405018766 shutdown_connections 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/3405018766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb828107860 0x7fb82810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/3405018766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828102b50 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/3405018766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb828103120 0x7fb828107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 -- 172.21.15.52:0/3405018766 >> 172.21.15.52:0/3405018766 conn(0x7fb828079600 msgr2=0x7fb828079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 -- 172.21.15.52:0/3405018766 shutdown_connections 2024-06-02T23:08:00.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.988+0000 7fb82f2aa700 1 -- 172.21.15.52:0/3405018766 wait complete. 2024-06-02T23:08:00.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 Processor -- start 2024-06-02T23:08:00.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 -- start start 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb828103120 0x7fb828074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb828107860 0x7fb828075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb828112650 con 0x7fb828102770 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb8281124d0 con 0x7fb828107860 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82f2aa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb8281127d0 con 0x7fb828103120 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82d046700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82d046700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38394/0 (socket says 172.21.15.52:38394) 2024-06-02T23:08:00.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82d046700 1 -- 172.21.15.52:0/1066759252 learned_addr learned my addr 172.21.15.52:0/1066759252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.989+0000 7fb82d847700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb828107860 0x7fb828075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82c845700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb828103120 0x7fb828074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82d046700 1 -- 172.21.15.52:0/1066759252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb828103120 msgr2=0x7fb828074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82d046700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb828103120 0x7fb828074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82d046700 1 -- 172.21.15.52:0/1066759252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb828107860 msgr2=0x7fb828075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82d046700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb828107860 0x7fb828075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82d046700 1 -- 172.21.15.52:0/1066759252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8281a7050 con 0x7fb828102770 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82d046700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828074790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb824006010 tx=0x7fb824009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824027070 con 0x7fb828102770 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8281a72e0 con 0x7fb828102770 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb824003f50 con 0x7fb828102770 2024-06-02T23:08:00.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb824022cd0 con 0x7fb828102770 2024-06-02T23:08:00.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.990+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8281a7740 con 0x7fb828102770 2024-06-02T23:08:00.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.991+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb82810aaa0 con 0x7fb828102770 2024-06-02T23:08:00.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.992+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb82402a070 con 0x7fb828102770 2024-06-02T23:08:00.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.993+0000 7fb81e7fc700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8140a2cb0 0x7fb8140a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:00.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.993+0000 7fb82c845700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8140a2cb0 0x7fb8140a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:00.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.993+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fb8240cfe30 con 0x7fb828102770 2024-06-02T23:08:00.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.993+0000 7fb82c845700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8140a2cb0 0x7fb8140a5170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb818005950 tx=0x7fb8180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:00.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:00.998+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb82405e850 con 0x7fb828102770 2024-06-02T23:08:01.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.237+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb8280628d0 con 0x7fb8140a2cb0 2024-06-02T23:08:01.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.268+0000 7fb81e7fc700 1 -- 172.21.15.52:0/1066759252 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb8280628d0 con 0x7fb8140a2cb0 2024-06-02T23:08:01.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.270+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8140a2cb0 msgr2=0x7fb8140a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:01.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.270+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8140a2cb0 0x7fb8140a5170 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb818005950 tx=0x7fb8180058e0 comp rx=0 tx=0).stop 2024-06-02T23:08:01.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.270+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 msgr2=0x7fb828074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:01.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.270+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828074790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb824006010 tx=0x7fb824009f80 comp rx=0 tx=0).stop 2024-06-02T23:08:01.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 shutdown_connections 2024-06-02T23:08:01.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8140a2cb0 0x7fb8140a5170 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:01.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb828107860 0x7fb828075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:01.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb828102770 0x7fb828074790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:01.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 --2- 172.21.15.52:0/1066759252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb828103120 0x7fb828074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:01.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 >> 172.21.15.52:0/1066759252 conn(0x7fb828079600 msgr2=0x7fb828100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:01.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 shutdown_connections 2024-06-02T23:08:01.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:01.271+0000 7fb82f2aa700 1 -- 172.21.15.52:0/1066759252 wait complete. 2024-06-02T23:08:01.589 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:01 smithi052 ceph-mon[32340]: pgmap v313: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 21 op/s 2024-06-02T23:08:01.590 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:01 smithi052 ceph-mon[32340]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-06-02T23:08:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:01 smithi174 ceph-mon[35994]: pgmap v313: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 21 op/s 2024-06-02T23:08:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:01 smithi174 ceph-mon[35994]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-06-02T23:08:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:01 smithi123 ceph-mon[35977]: pgmap v313: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.1 KiB/s wr, 21 op/s 2024-06-02T23:08:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:01 smithi123 ceph-mon[35977]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-06-02T23:08:01.956 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-06-02T23:08:02.213 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:02.558 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:02 smithi052 ceph-mon[32340]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:02 smithi174 ceph-mon[35994]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:02.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:02 smithi123 ceph-mon[35977]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:03.621 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:03 smithi052 ceph-mon[32340]: pgmap v314: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 20 op/s 2024-06-02T23:08:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:03 smithi174 ceph-mon[35994]: pgmap v314: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 20 op/s 2024-06-02T23:08:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:03 smithi123 ceph-mon[35977]: pgmap v314: 97 pgs: 97 active+clean; 585 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 3.7 KiB/s wr, 20 op/s 2024-06-02T23:08:03.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 -- 172.21.15.52:0/275579960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c06ac60 msgr2=0x7f6c4c06b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:03.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 --2- 172.21.15.52:0/275579960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c06ac60 0x7f6c4c06b0e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6c380099c0 tx=0x7f6c38009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 -- 172.21.15.52:0/275579960 shutdown_connections 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 --2- 172.21.15.52:0/275579960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4c06b620 0x7f6c4c10f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 --2- 172.21.15.52:0/275579960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4c108580 0x7f6c4c108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 --2- 172.21.15.52:0/275579960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c06ac60 0x7f6c4c06b0e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 -- 172.21.15.52:0/275579960 >> 172.21.15.52:0/275579960 conn(0x7f6c4c078430 msgr2=0x7f6c4c078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 -- 172.21.15.52:0/275579960 shutdown_connections 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.872+0000 7f6c53125700 1 -- 172.21.15.52:0/275579960 wait complete. 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.873+0000 7f6c53125700 1 Processor -- start 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.873+0000 7f6c53125700 1 -- start start 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.873+0000 7f6c53125700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4c06ac60 0x7f6c4c19e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:03.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.873+0000 7f6c53125700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4c06b620 0x7f6c4c19ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:03.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.873+0000 7f6c53125700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 0x7f6c4c1989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:03.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.873+0000 7f6c53125700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112680 con 0x7f6c4c108580 2024-06-02T23:08:03.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c53125700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112500 con 0x7f6c4c06ac60 2024-06-02T23:08:03.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c53125700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112800 con 0x7f6c4c06b620 2024-06-02T23:08:03.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 0x7f6c4c1989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:03.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 0x7f6c4c1989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41014/0 (socket says 172.21.15.52:41014) 2024-06-02T23:08:03.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 -- 172.21.15.52:0/3577378059 learned_addr learned my addr 172.21.15.52:0/3577378059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:03.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c50ec1700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4c06ac60 0x7f6c4c19e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:03.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c4bfff700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4c06b620 0x7f6c4c19ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:03.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 -- 172.21.15.52:0/3577378059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4c06b620 msgr2=0x7f6c4c19ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4c06b620 0x7f6c4c19ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 -- 172.21.15.52:0/3577378059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4c06ac60 msgr2=0x7f6c4c19e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4c06ac60 0x7f6c4c19e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 -- 172.21.15.52:0/3577378059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c199100 con 0x7f6c4c108580 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c516c2700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 0x7f6c4c1989a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c44007ee0 tx=0x7f6c4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c4400dbd0 con 0x7f6c4c108580 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c1993f0 con 0x7f6c4c108580 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6c4400dd70 con 0x7f6c4c108580 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.874+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44005a20 con 0x7f6c4c108580 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.875+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c101330 con 0x7f6c4c108580 2024-06-02T23:08:03.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.876+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c4c069180 con 0x7f6c4c108580 2024-06-02T23:08:03.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.877+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6c4401a020 con 0x7f6c4c108580 2024-06-02T23:08:03.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.878+0000 7f6c49ffb700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6c3c0a70c0 0x7f6c3c0a9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:03.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.878+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f6c440ce7e0 con 0x7f6c4c108580 2024-06-02T23:08:03.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.878+0000 7f6c50ec1700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6c3c0a70c0 0x7f6c3c0a9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:03.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.878+0000 7f6c50ec1700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6c3c0a70c0 0x7f6c3c0a9580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c199ae0 tx=0x7f6c4000b4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:03.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:03.882+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6c44010070 con 0x7f6c4c108580 2024-06-02T23:08:04.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.125+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f6c4c0628d0 con 0x7f6c3c0a70c0 2024-06-02T23:08:04.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.132+0000 7f6c49ffb700 1 -- 172.21.15.52:0/3577378059 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f6c4c0628d0 con 0x7f6c3c0a70c0 2024-06-02T23:08:04.132 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.134+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6c3c0a70c0 msgr2=0x7f6c3c0a9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.134+0000 7f6c53125700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6c3c0a70c0 0x7f6c3c0a9580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c199ae0 tx=0x7f6c4000b4d0 comp rx=0 tx=0).stop 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.134+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 msgr2=0x7f6c4c1989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.134+0000 7f6c53125700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 0x7f6c4c1989a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6c44007ee0 tx=0x7f6c4400d3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 shutdown_connections 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6c3c0a70c0 0x7f6c3c0a9580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6c4c108580 0x7f6c4c1989a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c4c06ac60 0x7f6c4c19e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:04.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 --2- 172.21.15.52:0/3577378059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6c4c06b620 0x7f6c4c19ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:04.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 >> 172.21.15.52:0/3577378059 conn(0x7f6c4c078430 msgr2=0x7f6c4c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:04.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 shutdown_connections 2024-06-02T23:08:04.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:04.135+0000 7f6c53125700 1 -- 172.21.15.52:0/3577378059 wait complete. 2024-06-02T23:08:04.999 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-06-02T23:08:05.255 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:05.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:05 smithi052 ceph-mon[32340]: pgmap v315: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-06-02T23:08:05.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:05 smithi052 ceph-mon[32340]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:05 smithi174 ceph-mon[35994]: pgmap v315: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-06-02T23:08:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:05 smithi174 ceph-mon[35994]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:05.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:05 smithi123 ceph-mon[35977]: pgmap v315: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 4.2 KiB/s wr, 18 op/s 2024-06-02T23:08:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:05 smithi123 ceph-mon[35977]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:07.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 -- 172.21.15.52:0/4202137111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 msgr2=0x7f65dc06b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:07.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 --2- 172.21.15.52:0/4202137111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc06b040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f65c80099c0 tx=0x7f65c8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 -- 172.21.15.52:0/4202137111 shutdown_connections 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 --2- 172.21.15.52:0/4202137111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65dc06b580 0x7f65dc10f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 --2- 172.21.15.52:0/4202137111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65dc1085a0 0x7f65dc108980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 --2- 172.21.15.52:0/4202137111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc06b040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 -- 172.21.15.52:0/4202137111 >> 172.21.15.52:0/4202137111 conn(0x7f65dc077200 msgr2=0x7f65dc077610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 -- 172.21.15.52:0/4202137111 shutdown_connections 2024-06-02T23:08:07.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.074+0000 7f65e3120700 1 -- 172.21.15.52:0/4202137111 wait complete. 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 Processor -- start 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 -- start start 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65dc06b580 0x7f65dc19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65dc1085a0 0x7f65dc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112690 con 0x7f65dc06abc0 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112510 con 0x7f65dc1085a0 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e3120700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc112810 con 0x7f65dc06b580 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc19e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41038/0 (socket says 172.21.15.52:41038) 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 -- 172.21.15.52:0/990885553 learned_addr learned my addr 172.21.15.52:0/990885553 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65dbfff700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65dc06b580 0x7f65dc19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e16bd700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65dc1085a0 0x7f65dc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 -- 172.21.15.52:0/990885553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65dc06b580 msgr2=0x7f65dc19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65dc06b580 0x7f65dc19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 -- 172.21.15.52:0/990885553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65dc1085a0 msgr2=0x7f65dc1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.075+0000 7f65e0ebc700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65dc1085a0 0x7f65dc1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65e0ebc700 1 -- 172.21.15.52:0/990885553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc199010 con 0x7f65dc06abc0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65e0ebc700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc19e6f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d000c920 tx=0x7f65d000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0007b00 con 0x7f65dc06abc0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f65d0004ce0 con 0x7f65dc06abc0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc199300 con 0x7f65dc06abc0 2024-06-02T23:08:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d0005600 con 0x7f65dc06abc0 2024-06-02T23:08:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.076+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a55c0 con 0x7f65dc06abc0 2024-06-02T23:08:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.077+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc103bf0 con 0x7f65dc06abc0 2024-06-02T23:08:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.078+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f65d0007ca0 con 0x7f65dc06abc0 2024-06-02T23:08:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.079+0000 7f65d9ffb700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65cc0a2cb0 0x7f65cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.079+0000 7f65dbfff700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65cc0a2cb0 0x7f65cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.079+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7f65d00cecd0 con 0x7f65dc06abc0 2024-06-02T23:08:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.079+0000 7f65dbfff700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65cc0a2cb0 0x7f65cc0a5170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f65dc077770 tx=0x7f65c8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.084+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f65d0014030 con 0x7f65dc06abc0 2024-06-02T23:08:07.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.324+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f65dc0628d0 con 0x7f65cc0a2cb0 2024-06-02T23:08:07.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.355+0000 7f65d9ffb700 1 -- 172.21.15.52:0/990885553 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f65dc0628d0 con 0x7f65cc0a2cb0 2024-06-02T23:08:07.355 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:07 smithi052 ceph-mon[32340]: pgmap v316: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-06-02T23:08:07.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65cc0a2cb0 msgr2=0x7f65cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:07.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65cc0a2cb0 0x7f65cc0a5170 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f65dc077770 tx=0x7f65c8005fd0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 msgr2=0x7f65dc19e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc19e6f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f65d000c920 tx=0x7f65d000cce0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 shutdown_connections 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65cc0a2cb0 0x7f65cc0a5170 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65dc1085a0 0x7f65dc1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f65dc06abc0 0x7f65dc19e6f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 --2- 172.21.15.52:0/990885553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65dc06b580 0x7f65dc19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 >> 172.21.15.52:0/990885553 conn(0x7f65dc077200 msgr2=0x7f65dc100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 shutdown_connections 2024-06-02T23:08:07.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:07.357+0000 7f65e3120700 1 -- 172.21.15.52:0/990885553 wait complete. 2024-06-02T23:08:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:07 smithi174 ceph-mon[35994]: pgmap v316: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-06-02T23:08:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:07 smithi123 ceph-mon[35977]: pgmap v316: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-06-02T23:08:08.111 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-06-02T23:08:08.364 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:08.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:08 smithi052 ceph-mon[32340]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:08.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:08 smithi174 ceph-mon[35994]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:08.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:08 smithi123 ceph-mon[35977]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:09 smithi174 ceph-mon[35994]: pgmap v317: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-06-02T23:08:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:09 smithi052 ceph-mon[32340]: pgmap v317: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-06-02T23:08:09.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:09 smithi123 ceph-mon[35977]: pgmap v317: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 3.0 KiB/s wr, 13 op/s 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 -- 172.21.15.52:0/2253795032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c102770 msgr2=0x7fcc2c102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/2253795032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c102770 0x7fcc2c102b50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c0099c0 tx=0x7fcc1c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 -- 172.21.15.52:0/2253795032 shutdown_connections 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/2253795032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c107860 0x7fcc2c10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/2253795032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c102770 0x7fcc2c102b50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/2253795032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc2c103120 0x7fcc2c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 -- 172.21.15.52:0/2253795032 >> 172.21.15.52:0/2253795032 conn(0x7fcc2c079600 msgr2=0x7fcc2c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 -- 172.21.15.52:0/2253795032 shutdown_connections 2024-06-02T23:08:10.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.061+0000 7fcc30ed6700 1 -- 172.21.15.52:0/2253795032 wait complete. 2024-06-02T23:08:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.062+0000 7fcc30ed6700 1 Processor -- start 2024-06-02T23:08:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.062+0000 7fcc30ed6700 1 -- start start 2024-06-02T23:08:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc30ed6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c102770 0x7fcc2c198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:10.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc30ed6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc2c103120 0x7fcc2c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc30ed6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 0x7fcc2c19d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc30ed6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c112630 con 0x7fcc2c107860 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc30ed6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1124b0 con 0x7fcc2c103120 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc30ed6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcc2c1127b0 con 0x7fcc2c102770 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 0x7fcc2c19d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 0x7fcc2c19d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41050/0 (socket says 172.21.15.52:41050) 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 -- 172.21.15.52:0/3779296350 learned_addr learned my addr 172.21.15.52:0/3779296350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc21d9b700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc2c103120 0x7fcc2c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2a59c700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c102770 0x7fcc2c198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 -- 172.21.15.52:0/3779296350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c102770 msgr2=0x7fcc2c198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c102770 0x7fcc2c198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 -- 172.21.15.52:0/3779296350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc2c103120 msgr2=0x7fcc2c199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:10.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc2c103120 0x7fcc2c199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 -- 172.21.15.52:0/3779296350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc2c19dab0 con 0x7fcc2c107860 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.063+0000 7fcc2ad9d700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 0x7fcc2c19d3b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007d70 tx=0x7fcc2400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.064+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc24016020 con 0x7fcc2c107860 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.064+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcc2400ac90 con 0x7fcc2c107860 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.064+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc2c19dda0 con 0x7fcc2c107860 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.064+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc240115e0 con 0x7fcc2c107860 2024-06-02T23:08:10.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.064+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc2c1a5660 con 0x7fcc2c107860 2024-06-02T23:08:10.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.065+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc2c10aaa0 con 0x7fcc2c107860 2024-06-02T23:08:10.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.066+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcc24019020 con 0x7fcc2c107860 2024-06-02T23:08:10.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.067+0000 7fcc23fff700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcc180a2cb0 0x7fcc180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:10.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.067+0000 7fcc2a59c700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcc180a2cb0 0x7fcc180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:10.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.067+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7979+0+0 (secure 0 0 0) 0x7fcc240cef30 con 0x7fcc2c107860 2024-06-02T23:08:10.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.067+0000 7fcc2a59c700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcc180a2cb0 0x7fcc180a5170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:10.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.071+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcc24020020 con 0x7fcc2c107860 2024-06-02T23:08:10.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.313+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fcc2c0628d0 con 0x7fcc180a2cb0 2024-06-02T23:08:10.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.321+0000 7fcc23fff700 1 -- 172.21.15.52:0/3779296350 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fcc2c0628d0 con 0x7fcc180a2cb0 2024-06-02T23:08:10.321 INFO:teuthology.orchestra.run.smithi052.stdout:/volumes/qa/sv_1/34bb3fdb-34a3-49e5-b2f5-1465b098eb70 2024-06-02T23:08:10.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.324+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcc180a2cb0 msgr2=0x7fcc180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.324+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcc180a2cb0 0x7fcc180a5170 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c009fd0 tx=0x7fcc1c005fb0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.324+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 msgr2=0x7fcc2c19d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:10.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.324+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 0x7fcc2c19d3b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcc24007d70 tx=0x7fcc2400a340 comp rx=0 tx=0).stop 2024-06-02T23:08:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.325+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 shutdown_connections 2024-06-02T23:08:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.325+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcc180a2cb0 0x7fcc180a5170 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.325+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcc2c107860 0x7fcc2c19d3b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.325+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcc2c102770 0x7fcc2c198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.325+0000 7fcc30ed6700 1 --2- 172.21.15.52:0/3779296350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc2c103120 0x7fcc2c199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:10.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.325+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 >> 172.21.15.52:0/3779296350 conn(0x7fcc2c079600 msgr2=0x7fcc2c100560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.328+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 shutdown_connections 2024-06-02T23:08:10.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:10.328+0000 7fcc30ed6700 1 -- 172.21.15.52:0/3779296350 wait complete. 2024-06-02T23:08:10.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-02T23:08:10.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-02T23:08:10.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-06-02T23:08:10.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-06-02T23:08:10.597 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-02T23:08:10.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-02T23:08:10.598 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-06-02T23:08:10.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-02T23:08:10.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-02T23:08:10.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-06-02T23:08:10.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-06-02T23:08:10.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-02T23:08:10.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-02T23:08:10.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]: dispatch 2024-06-02T23:08:10.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-06-02T23:08:11.049 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:08:11.305 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:11.600 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: pgmap v318: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-06-02T23:08:11.600 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-06-02T23:08:11.601 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:11 smithi052 ceph-mon[32340]: osdmap e89: 12 total, 12 up, 12 in 2024-06-02T23:08:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: pgmap v318: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-06-02T23:08:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-02T23:08:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-02T23:08:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-06-02T23:08:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-06-02T23:08:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-02T23:08:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-02T23:08:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-06-02T23:08:11.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:11 smithi174 ceph-mon[35994]: osdmap e89: 12 total, 12 up, 12 in 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: pgmap v318: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.9 KiB/s wr, 11 op/s 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [8, 11]}]': finished 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11, 2, 1]}]': finished 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-06-02T23:08:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.25", "id": [5, 9]}]': finished 2024-06-02T23:08:11.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-06-02T23:08:11.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:11 smithi123 ceph-mon[35977]: osdmap e89: 12 total, 12 up, 12 in 2024-06-02T23:08:12.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:12 smithi052 ceph-mon[32340]: osdmap e90: 12 total, 12 up, 12 in 2024-06-02T23:08:12.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:12 smithi174 ceph-mon[35994]: osdmap e90: 12 total, 12 up, 12 in 2024-06-02T23:08:12.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:12 smithi123 ceph-mon[35977]: osdmap e90: 12 total, 12 up, 12 in 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 -- 172.21.15.52:0/33642199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784101d50 msgr2=0x7f078410e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 --2- 172.21.15.52:0/33642199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784101d50 0x7f078410e240 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f077c009990 tx=0x7f077c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 -- 172.21.15.52:0/33642199 shutdown_connections 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 --2- 172.21.15.52:0/33642199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784101d50 0x7f078410e240 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 --2- 172.21.15.52:0/33642199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0784107400 0x7f07841077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 --2- 172.21.15.52:0/33642199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0784101390 0x7f0784101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 -- 172.21.15.52:0/33642199 >> 172.21.15.52:0/33642199 conn(0x7f0784078340 msgr2=0x7f0784078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 -- 172.21.15.52:0/33642199 shutdown_connections 2024-06-02T23:08:13.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.008+0000 7f07898b0700 1 -- 172.21.15.52:0/33642199 wait complete. 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.009+0000 7f07898b0700 1 Processor -- start 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 -- start start 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0784101390 0x7f078419d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0784101d50 0x7f078419dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 0x7f07841978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0784111570 con 0x7f0784107400 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07841113f0 con 0x7f0784101d50 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07898b0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07841116f0 con 0x7f0784101390 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 0x7f07841978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 0x7f07841978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41068/0 (socket says 172.21.15.52:41068) 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 -- 172.21.15.52:0/3896172121 learned_addr learned my addr 172.21.15.52:0/3896172121 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:13.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07827fc700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0784101d50 0x7f078419dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f0782ffd700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0784101390 0x7f078419d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 -- 172.21.15.52:0/3896172121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0784101390 msgr2=0x7f078419d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0784101390 0x7f078419d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 -- 172.21.15.52:0/3896172121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0784101d50 msgr2=0x7f078419dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0784101d50 0x7f078419dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.010+0000 7f07837fe700 1 -- 172.21.15.52:0/3896172121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0784198010 con 0x7f0784107400 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.011+0000 7f07837fe700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 0x7f07841978b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f077c006010 tx=0x7f077c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.011+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c026070 con 0x7f0784107400 2024-06-02T23:08:13.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.011+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07841982a0 con 0x7f0784107400 2024-06-02T23:08:13.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.011+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f077c003f80 con 0x7f0784107400 2024-06-02T23:08:13.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.011+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f077c00fb70 con 0x7f0784107400 2024-06-02T23:08:13.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.011+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07841a45c0 con 0x7f0784107400 2024-06-02T23:08:13.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.012+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0784069180 con 0x7f0784107400 2024-06-02T23:08:13.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.013+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f077c029070 con 0x7f0784107400 2024-06-02T23:08:13.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.014+0000 7f07888ae700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07700a2cb0 0x7f07700a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:13.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.014+0000 7f0782ffd700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07700a2cb0 0x7f07700a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:13.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.014+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f077c0cf930 con 0x7f0784107400 2024-06-02T23:08:13.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.014+0000 7f0782ffd700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07700a2cb0 0x7f07700a5170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0774009550 tx=0x7f07740091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:13.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.018+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f077c013070 con 0x7f0784107400 2024-06-02T23:08:13.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.272+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f078402dca0 con 0x7f0784107400 2024-06-02T23:08:13.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.273+0000 7f07888ae700 1 -- 172.21.15.52:0/3896172121 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f077c09c7f0 con 0x7f0784107400 2024-06-02T23:08:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:13.274 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:07:54.810487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.275+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07700a2cb0 msgr2=0x7f07700a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.275+0000 7f07898b0700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07700a2cb0 0x7f07700a5170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0774009550 tx=0x7f07740091f0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.275+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 msgr2=0x7f07841978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.275+0000 7f07898b0700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 0x7f07841978b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f077c006010 tx=0x7f077c0047b0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 shutdown_connections 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07700a2cb0 0x7f07700a5170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0784107400 0x7f07841978b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0784101390 0x7f078419d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 --2- 172.21.15.52:0/3896172121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0784101d50 0x7f078419dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 >> 172.21.15.52:0/3896172121 conn(0x7f0784078340 msgr2=0x7f07840fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:13.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 shutdown_connections 2024-06-02T23:08:13.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:13.276+0000 7f07898b0700 1 -- 172.21.15.52:0/3896172121 wait complete. 2024-06-02T23:08:13.277 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:08:13.629 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:13 smithi052 ceph-mon[32340]: pgmap v321: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-02T23:08:13.629 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:13 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3896172121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:08:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:13 smithi174 ceph-mon[35994]: pgmap v321: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-02T23:08:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:13 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3896172121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:08:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:13 smithi123 ceph-mon[35977]: pgmap v321: 97 pgs: 97 active+clean; 611 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-06-02T23:08:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:13 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3896172121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:08:13.974 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:08:14.235 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:15 smithi174 ceph-mon[35994]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-06-02T23:08:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:15 smithi052 ceph-mon[32340]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-06-02T23:08:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:15 smithi123 ceph-mon[35977]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-06-02T23:08:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.926+0000 7fa8412d4700 1 -- 172.21.15.52:0/709099431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c1027a0 msgr2=0x7fa83c102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.926+0000 7fa8412d4700 1 --2- 172.21.15.52:0/709099431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c1027a0 0x7fa83c102b80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa82c0099c0 tx=0x7fa82c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 -- 172.21.15.52:0/709099431 shutdown_connections 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 --2- 172.21.15.52:0/709099431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c107870 0x7fa83c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 --2- 172.21.15.52:0/709099431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c1027a0 0x7fa83c102b80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 --2- 172.21.15.52:0/709099431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa83c103150 0x7fa83c107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 -- 172.21.15.52:0/709099431 >> 172.21.15.52:0/709099431 conn(0x7fa83c079570 msgr2=0x7fa83c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 -- 172.21.15.52:0/709099431 shutdown_connections 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 -- 172.21.15.52:0/709099431 wait complete. 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.927+0000 7fa8412d4700 1 Processor -- start 2024-06-02T23:08:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 -- start start 2024-06-02T23:08:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa83c1027a0 0x7fa83c19e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c103150 0x7fa83c19ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 0x7fa83c198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c1126b0 con 0x7fa83c107870 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c112530 con 0x7fa83c103150 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa8412d4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c112830 con 0x7fa83c1027a0 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 0x7fa83c198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 0x7fa83c198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39102/0 (socket says 172.21.15.52:39102) 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 -- 172.21.15.52:0/1015795226 learned_addr learned my addr 172.21.15.52:0/1015795226 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83affd700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa83c1027a0 0x7fa83c19e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83a7fc700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c103150 0x7fa83c19ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 -- 172.21.15.52:0/1015795226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa83c1027a0 msgr2=0x7fa83c19e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa83c1027a0 0x7fa83c19e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 -- 172.21.15.52:0/1015795226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c103150 msgr2=0x7fa83c19ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c103150 0x7fa83c19ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83b7fe700 1 -- 172.21.15.52:0/1015795226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c199210 con 0x7fa83c107870 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.928+0000 7fa83a7fc700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c103150 0x7fa83c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.929+0000 7fa83b7fe700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 0x7fa83c198ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa834007d70 tx=0x7fa83400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.929+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa834016020 con 0x7fa83c107870 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.929+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83c199500 con 0x7fa83c107870 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.929+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa83400ad40 con 0x7fa83c107870 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.929+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8340115e0 con 0x7fa83c107870 2024-06-02T23:08:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.929+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83c1a57c0 con 0x7fa83c107870 2024-06-02T23:08:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.930+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83c10aab0 con 0x7fa83c107870 2024-06-02T23:08:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.931+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa834019020 con 0x7fa83c107870 2024-06-02T23:08:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.932+0000 7fa823fff700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8240a2cb0 0x7fa8240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.932+0000 7fa83affd700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8240a2cb0 0x7fa8240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.932+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8340cf0b0 con 0x7fa83c107870 2024-06-02T23:08:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.932+0000 7fa83affd700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8240a2cb0 0x7fa8240a5170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009990 tx=0x7fa82c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:15.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:15.936+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa834020020 con 0x7fa83c107870 2024-06-02T23:08:16.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.143+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa83c199fe0 con 0x7fa83c107870 2024-06-02T23:08:16.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.145+0000 7fa823fff700 1 -- 172.21.15.52:0/1015795226 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7fa83409bff0 con 0x7fa83c107870 2024-06-02T23:08:16.146 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:16.146 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8240a2cb0 msgr2=0x7fa8240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8240a2cb0 0x7fa8240a5170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009990 tx=0x7fa82c005fb0 comp rx=0 tx=0).stop 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 msgr2=0x7fa83c198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 0x7fa83c198ab0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa834007d70 tx=0x7fa83400a340 comp rx=0 tx=0).stop 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 shutdown_connections 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8240a2cb0 0x7fa8240a5170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:16.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa83c107870 0x7fa83c198ab0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa83c1027a0 0x7fa83c19e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 --2- 172.21.15.52:0/1015795226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83c103150 0x7fa83c19ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 >> 172.21.15.52:0/1015795226 conn(0x7fa83c079570 msgr2=0x7fa83c101cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 shutdown_connections 2024-06-02T23:08:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:16.148+0000 7fa8412d4700 1 -- 172.21.15.52:0/1015795226 wait complete. 2024-06-02T23:08:16.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:16 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1015795226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:16.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:16 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1015795226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:16.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:16 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1015795226' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:16.881 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-02T23:08:16.882 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:08:17.138 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:17.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:17 smithi052 ceph-mon[32340]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 9 op/s; 91 B/s, 1 keys/s, 2 objects/s recovering 2024-06-02T23:08:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:17 smithi174 ceph-mon[35994]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 9 op/s; 91 B/s, 1 keys/s, 2 objects/s recovering 2024-06-02T23:08:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:17 smithi123 ceph-mon[35977]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 3.7 KiB/s wr, 9 op/s; 91 B/s, 1 keys/s, 2 objects/s recovering 2024-06-02T23:08:18.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 -- 172.21.15.52:0/2778181221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 msgr2=0x7f0174107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 --2- 172.21.15.52:0/2778181221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f0174107110 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f01680099c0 tx=0x7f0168009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 -- 172.21.15.52:0/2778181221 shutdown_connections 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 --2- 172.21.15.52:0/2778181221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0174107840 0x7f017410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 --2- 172.21.15.52:0/2778181221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0174101020 0x7f0174101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 --2- 172.21.15.52:0/2778181221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f0174107110 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 -- 172.21.15.52:0/2778181221 >> 172.21.15.52:0/2778181221 conn(0x7f01740766c0 msgr2=0x7f0174076ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 -- 172.21.15.52:0/2778181221 shutdown_connections 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.808+0000 7f017ac31700 1 -- 172.21.15.52:0/2778181221 wait complete. 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 Processor -- start 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 -- start start 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0174101020 0x7f017419e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:18.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f017419ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0174107840 0x7f01741989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0174112660 con 0x7f01741019d0 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01741124e0 con 0x7f0174107840 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f017ac31700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01741127e0 con 0x7f0174101020 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f0173fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f017419ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f0173fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f017419ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39116/0 (socket says 172.21.15.52:39116) 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.809+0000 7f0173fff700 1 -- 172.21.15.52:0/2884242014 learned_addr learned my addr 172.21.15.52:0/2884242014 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f01789cd700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0174101020 0x7f017419e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f01791ce700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0174107840 0x7f01741989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0173fff700 1 -- 172.21.15.52:0/2884242014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0174101020 msgr2=0x7f017419e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0173fff700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0174101020 0x7f017419e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0173fff700 1 -- 172.21.15.52:0/2884242014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0174107840 msgr2=0x7f01741989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0173fff700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0174107840 0x7f01741989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:18.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0173fff700 1 -- 172.21.15.52:0/2884242014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0174199100 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0173fff700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f017419ed20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01680098e0 tx=0x7f0168004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0168026070 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0168003c70 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f016800fad0 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.810+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0174199390 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.811+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01741a56b0 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.812+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f017410aa80 con 0x7f01741019d0 2024-06-02T23:08:18.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.813+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f0168029070 con 0x7f01741019d0 2024-06-02T23:08:18.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.814+0000 7f0171ffb700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f015c0a2d30 0x7f015c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:18.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.814+0000 7f01789cd700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f015c0a2d30 0x7f015c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:18.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.814+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f01680cffa0 con 0x7f01741019d0 2024-06-02T23:08:18.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.814+0000 7f01789cd700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f015c0a2d30 0x7f015c0a51f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0174199ae0 tx=0x7f0164009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:18.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:18.817+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0168030020 con 0x7f01741019d0 2024-06-02T23:08:19.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.019+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0174199ed0 con 0x7f015c0a2d30 2024-06-02T23:08:19.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.026+0000 7f0171ffb700 1 -- 172.21.15.52:0/2884242014 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258967 (secure 0 0 0) 0x7f0174199ed0 con 0x7f015c0a2d30 2024-06-02T23:08:19.031 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f015c0a2d30 msgr2=0x7f015c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f015c0a2d30 0x7f015c0a51f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0174199ae0 tx=0x7f0164009260 comp rx=0 tx=0).stop 2024-06-02T23:08:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 msgr2=0x7f017419ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f017419ed20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f01680098e0 tx=0x7f0168004270 comp rx=0 tx=0).stop 2024-06-02T23:08:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 shutdown_connections 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f015c0a2d30 0x7f015c0a51f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0174107840 0x7f01741989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0174101020 0x7f017419e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 --2- 172.21.15.52:0/2884242014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f01741019d0 0x7f017419ed20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 >> 172.21.15.52:0/2884242014 conn(0x7f01740766c0 msgr2=0x7f01740ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 shutdown_connections 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:19.030+0000 7f017ac31700 1 -- 172.21.15.52:0/2884242014 wait complete. 2024-06-02T23:08:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:08:19.611 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:19 smithi052 ceph-mon[32340]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 12 op/s; 91 B/s, 1 keys/s, 2 objects/s recovering 2024-06-02T23:08:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:19 smithi174 ceph-mon[35994]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 12 op/s; 91 B/s, 1 keys/s, 2 objects/s recovering 2024-06-02T23:08:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:19 smithi123 ceph-mon[35977]: pgmap v324: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.7 KiB/s wr, 12 op/s; 91 B/s, 1 keys/s, 2 objects/s recovering 2024-06-02T23:08:19.955 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-06-02T23:08:17.771035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657251,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":403,"num_read_kb":324,"num_write":427,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":290,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369660,"kb_used_data":12140,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572164,"statfs":{"total":1151940427776,"available":1151561895936,"internally_reserved":0,"allocated":12431360,"data_stored":7717094,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31872,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":80,"num_read_kb":74,"num_write":23,"num_write_kb":38,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001886"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331749+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:08:11.331749+0000","last_peered":"2024-06-02T23:08:11.331749+0000","last_clean":"2024-06-02T23:08:11.331749+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:08:11.331749+0000","last_undegraded":"2024-06-02T23:08:11.331749+0000","last_fullsized":"2024-06-02T23:08:11.331749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331436+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:08:11.331436+0000","last_peered":"2024-06-02T23:08:11.331436+0000","last_clean":"2024-06-02T23:08:11.331436+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:08:11.331436+0000","last_undegraded":"2024-06-02T23:08:11.331436+0000","last_fullsized":"2024-06-02T23:08:11.331436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331379+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:08:11.331379+0000","last_peered":"2024-06-02T23:08:11.331379+0000","last_clean":"2024-06-02T23:08:11.331379+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:08:11.331379+0000","last_undegraded":"2024-06-02T23:08:11.331379+0000","last_fullsized":"2024-06-02T23:08:11.331379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450456+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:08:11.450456+0000","last_peered":"2024-06-02T23:08:11.450456+0000","last_clean":"2024-06-02T23:08:11.450456+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:08:11.450456+0000","last_undegraded":"2024-06-02T23:08:11.450456+0000","last_fullsized":"2024-06-02T23:08:11.450456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334614+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:08:11.334614+0000","last_peered":"2024-06-02T23:08:11.334614+0000","last_clean":"2024-06-02T23:08:11.334614+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:08:11.334614+0000","last_undegraded":"2024-06-02T23:08:11.334614+0000","last_fullsized":"2024-06-02T23:08:11.334614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449583+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:08:11.449583+0000","last_peered":"2024-06-02T23:08:11.449583+0000","last_clean":"2024-06-02T23:08:11.449583+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:08:11.449583+0000","last_undegraded":"2024-06-02T23:08:11.449583+0000","last_fullsized":"2024-06-02T23:08:11.449583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332875+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:08:11.332875+0000","last_peered":"2024-06-02T23:08:11.332875+0000","last_clean":"2024-06-02T23:08:11.332875+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:08:11.332875+0000","last_undegraded":"2024-06-02T23:08:11.332875+0000","last_fullsized":"2024-06-02T23:08:11.332875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748655+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:08:11.748655+0000","last_peered":"2024-06-02T23:08:11.748655+0000","last_clean":"2024-06-02T23:08:11.748655+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:08:11.748655+0000","last_undegraded":"2024-06-02T23:08:11.748655+0000","last_fullsized":"2024-06-02T23:08:11.748655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334826+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:08:11.334826+0000","last_peered":"2024-06-02T23:08:11.334826+0000","last_clean":"2024-06-02T23:08:11.334826+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:08:11.334826+0000","last_undegraded":"2024-06-02T23:08:11.334826+0000","last_fullsized":"2024-06-02T23:08:11.334826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449830+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:08:11.449830+0000","last_peered":"2024-06-02T23:08:11.449830+0000","last_clean":"2024-06-02T23:08:11.449830+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:08:11.449830+0000","last_undegraded":"2024-06-02T23:08:11.449830+0000","last_fullsized":"2024-06-02T23:08:11.449830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332762+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:08:11.332762+0000","last_peered":"2024-06-02T23:08:11.332762+0000","last_clean":"2024-06-02T23:08:11.332762+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:08:11.332762+0000","last_undegraded":"2024-06-02T23:08:11.332762+0000","last_fullsized":"2024-06-02T23:08:11.332762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450303+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:08:11.450303+0000","last_peered":"2024-06-02T23:08:11.450303+0000","last_clean":"2024-06-02T23:08:11.450303+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:08:11.450303+0000","last_undegraded":"2024-06-02T23:08:11.450303+0000","last_fullsized":"2024-06-02T23:08:11.450303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333334+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:08:11.333334+0000","last_peered":"2024-06-02T23:08:11.333334+0000","last_clean":"2024-06-02T23:08:11.333334+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:08:11.333334+0000","last_undegraded":"2024-06-02T23:08:11.333334+0000","last_fullsized":"2024-06-02T23:08:11.333334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330935+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:08:11.330935+0000","last_peered":"2024-06-02T23:08:11.330935+0000","last_clean":"2024-06-02T23:08:11.330935+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:08:11.330935+0000","last_undegraded":"2024-06-02T23:08:11.330935+0000","last_fullsized":"2024-06-02T23:08:11.330935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330848+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:08:11.330848+0000","last_peered":"2024-06-02T23:08:11.330848+0000","last_clean":"2024-06-02T23:08:11.330848+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:08:11.330848+0000","last_undegraded":"2024-06-02T23:08:11.330848+0000","last_fullsized":"2024-06-02T23:08:11.330848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331976+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:08:11.331976+0000","last_peered":"2024-06-02T23:08:11.331976+0000","last_clean":"2024-06-02T23:08:11.331976+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:08:11.331976+0000","last_undegraded":"2024-06-02T23:08:11.331976+0000","last_fullsized":"2024-06-02T23:08:11.331976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328445+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:08:11.328445+0000","last_peered":"2024-06-02T23:08:11.328445+0000","last_clean":"2024-06-02T23:08:11.328445+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:08:11.328445+0000","last_undegraded":"2024-06-02T23:08:11.328445+0000","last_fullsized":"2024-06-02T23:08:11.328445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333538+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:08:11.333538+0000","last_peered":"2024-06-02T23:08:11.333538+0000","last_clean":"2024-06-02T23:08:11.333538+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:08:11.333538+0000","last_undegraded":"2024-06-02T23:08:11.333538+0000","last_fullsized":"2024-06-02T23:08:11.333538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.893590+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:08:14.893590+0000","last_peered":"2024-06-02T23:08:14.893590+0000","last_clean":"2024-06-02T23:08:14.893590+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:08:14.893590+0000","last_undegraded":"2024-06-02T23:08:14.893590+0000","last_fullsized":"2024-06-02T23:08:14.893590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748992+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:08:11.748992+0000","last_peered":"2024-06-02T23:08:11.748992+0000","last_clean":"2024-06-02T23:08:11.748992+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:08:11.748992+0000","last_undegraded":"2024-06-02T23:08:11.748992+0000","last_fullsized":"2024-06-02T23:08:11.748992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749145+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749145+0000","last_peered":"2024-06-02T23:08:11.749145+0000","last_clean":"2024-06-02T23:08:11.749145+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749145+0000","last_undegraded":"2024-06-02T23:08:11.749145+0000","last_fullsized":"2024-06-02T23:08:11.749145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331872+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:08:11.331872+0000","last_peered":"2024-06-02T23:08:11.331872+0000","last_clean":"2024-06-02T23:08:11.331872+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:08:11.331872+0000","last_undegraded":"2024-06-02T23:08:11.331872+0000","last_fullsized":"2024-06-02T23:08:11.331872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.365621+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:11.365621+0000","last_peered":"2024-06-02T23:08:11.365621+0000","last_clean":"2024-06-02T23:08:11.365621+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:11.365621+0000","last_undegraded":"2024-06-02T23:08:11.365621+0000","last_fullsized":"2024-06-02T23:08:11.365621+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334580+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:08:11.334580+0000","last_peered":"2024-06-02T23:08:11.334580+0000","last_clean":"2024-06-02T23:08:11.334580+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:08:11.334580+0000","last_undegraded":"2024-06-02T23:08:11.334580+0000","last_fullsized":"2024-06-02T23:08:11.334580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335715+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335715+0000","last_peered":"2024-06-02T23:08:11.335715+0000","last_clean":"2024-06-02T23:08:11.335715+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335715+0000","last_undegraded":"2024-06-02T23:08:11.335715+0000","last_fullsized":"2024-06-02T23:08:11.335715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329756+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:08:11.329756+0000","last_peered":"2024-06-02T23:08:11.329756+0000","last_clean":"2024-06-02T23:08:11.329756+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:08:11.329756+0000","last_undegraded":"2024-06-02T23:08:11.329756+0000","last_fullsized":"2024-06-02T23:08:11.329756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":67,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:12.454562+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:08:12.454562+0000","last_peered":"2024-06-02T23:08:12.454562+0000","last_clean":"2024-06-02T23:08:12.454562+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:08:12.454562+0000","last_undegraded":"2024-06-02T23:08:12.454562+0000","last_fullsized":"2024-06-02T23:08:12.454562+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":21,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334793+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:08:11.334793+0000","last_peered":"2024-06-02T23:08:11.334793+0000","last_clean":"2024-06-02T23:08:11.334793+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:08:11.334793+0000","last_undegraded":"2024-06-02T23:08:11.334793+0000","last_fullsized":"2024-06-02T23:08:11.334793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449532+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449532+0000","last_peered":"2024-06-02T23:08:11.449532+0000","last_clean":"2024-06-02T23:08:11.449532+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449532+0000","last_undegraded":"2024-06-02T23:08:11.449532+0000","last_fullsized":"2024-06-02T23:08:11.449532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329554+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:08:11.329554+0000","last_peered":"2024-06-02T23:08:11.329554+0000","last_clean":"2024-06-02T23:08:11.329554+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:08:11.329554+0000","last_undegraded":"2024-06-02T23:08:11.329554+0000","last_fullsized":"2024-06-02T23:08:11.329554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332891+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332891+0000","last_peered":"2024-06-02T23:08:11.332891+0000","last_clean":"2024-06-02T23:08:11.332891+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332891+0000","last_undegraded":"2024-06-02T23:08:11.332891+0000","last_fullsized":"2024-06-02T23:08:11.332891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748738+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:08:11.748738+0000","last_peered":"2024-06-02T23:08:11.748738+0000","last_clean":"2024-06-02T23:08:11.748738+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:08:11.748738+0000","last_undegraded":"2024-06-02T23:08:11.748738+0000","last_fullsized":"2024-06-02T23:08:11.748738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329228+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329228+0000","last_peered":"2024-06-02T23:08:11.329228+0000","last_clean":"2024-06-02T23:08:11.329228+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329228+0000","last_undegraded":"2024-06-02T23:08:11.329228+0000","last_fullsized":"2024-06-02T23:08:11.329228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328243+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:08:11.328243+0000","last_peered":"2024-06-02T23:08:11.328243+0000","last_clean":"2024-06-02T23:08:11.328243+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:08:11.328243+0000","last_undegraded":"2024-06-02T23:08:11.328243+0000","last_fullsized":"2024-06-02T23:08:11.328243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:16.459601+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:08:16.459601+0000","last_peered":"2024-06-02T23:08:16.459601+0000","last_clean":"2024-06-02T23:08:16.459601+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:08:16.459601+0000","last_undegraded":"2024-06-02T23:08:16.459601+0000","last_fullsized":"2024-06-02T23:08:16.459601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334817+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:08:11.334817+0000","last_peered":"2024-06-02T23:08:11.334817+0000","last_clean":"2024-06-02T23:08:11.334817+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:08:11.334817+0000","last_undegraded":"2024-06-02T23:08:11.334817+0000","last_fullsized":"2024-06-02T23:08:11.334817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332982+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.332982+0000","last_peered":"2024-06-02T23:08:11.332982+0000","last_clean":"2024-06-02T23:08:11.332982+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.332982+0000","last_undegraded":"2024-06-02T23:08:11.332982+0000","last_fullsized":"2024-06-02T23:08:11.332982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331775+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:08:11.331775+0000","last_peered":"2024-06-02T23:08:11.331775+0000","last_clean":"2024-06-02T23:08:11.331775+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:08:11.331775+0000","last_undegraded":"2024-06-02T23:08:11.331775+0000","last_fullsized":"2024-06-02T23:08:11.331775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329928+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.329928+0000","last_peered":"2024-06-02T23:08:11.329928+0000","last_clean":"2024-06-02T23:08:11.329928+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.329928+0000","last_undegraded":"2024-06-02T23:08:11.329928+0000","last_fullsized":"2024-06-02T23:08:11.329928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334827+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:08:11.334827+0000","last_peered":"2024-06-02T23:08:11.334827+0000","last_clean":"2024-06-02T23:08:11.334827+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:08:11.334827+0000","last_undegraded":"2024-06-02T23:08:11.334827+0000","last_fullsized":"2024-06-02T23:08:11.334827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":114,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:16.460276+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:08:16.460276+0000","last_peered":"2024-06-02T23:08:16.460276+0000","last_clean":"2024-06-02T23:08:16.460276+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:08:16.460276+0000","last_undegraded":"2024-06-02T23:08:16.460276+0000","last_fullsized":"2024-06-02T23:08:16.460276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59837,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":86,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335764+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:08:11.335764+0000","last_peered":"2024-06-02T23:08:11.335764+0000","last_clean":"2024-06-02T23:08:11.335764+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:08:11.335764+0000","last_undegraded":"2024-06-02T23:08:11.335764+0000","last_fullsized":"2024-06-02T23:08:11.335764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332546+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332546+0000","last_peered":"2024-06-02T23:08:11.332546+0000","last_clean":"2024-06-02T23:08:11.332546+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332546+0000","last_undegraded":"2024-06-02T23:08:11.332546+0000","last_fullsized":"2024-06-02T23:08:11.332546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333123+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:08:11.333123+0000","last_peered":"2024-06-02T23:08:11.333123+0000","last_clean":"2024-06-02T23:08:11.333123+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:08:11.333123+0000","last_undegraded":"2024-06-02T23:08:11.333123+0000","last_fullsized":"2024-06-02T23:08:11.333123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329400+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329400+0000","last_peered":"2024-06-02T23:08:11.329400+0000","last_clean":"2024-06-02T23:08:11.329400+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329400+0000","last_undegraded":"2024-06-02T23:08:11.329400+0000","last_fullsized":"2024-06-02T23:08:11.329400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332914+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:08:11.332914+0000","last_peered":"2024-06-02T23:08:11.332914+0000","last_clean":"2024-06-02T23:08:11.332914+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:08:11.332914+0000","last_undegraded":"2024-06-02T23:08:11.332914+0000","last_fullsized":"2024-06-02T23:08:11.332914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:13.893541+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:08:13.893541+0000","last_peered":"2024-06-02T23:08:13.893541+0000","last_clean":"2024-06-02T23:08:13.893541+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:08:13.893541+0000","last_undegraded":"2024-06-02T23:08:13.893541+0000","last_fullsized":"2024-06-02T23:08:13.893541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332579+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:08:11.332579+0000","last_peered":"2024-06-02T23:08:11.332579+0000","last_clean":"2024-06-02T23:08:11.332579+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:08:11.332579+0000","last_undegraded":"2024-06-02T23:08:11.332579+0000","last_fullsized":"2024-06-02T23:08:11.332579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.348940+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:11.348940+0000","last_peered":"2024-06-02T23:08:11.348940+0000","last_clean":"2024-06-02T23:08:11.348940+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:11.348940+0000","last_undegraded":"2024-06-02T23:08:11.348940+0000","last_fullsized":"2024-06-02T23:08:11.348940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329323+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:08:11.329323+0000","last_peered":"2024-06-02T23:08:11.329323+0000","last_clean":"2024-06-02T23:08:11.329323+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:08:11.329323+0000","last_undegraded":"2024-06-02T23:08:11.329323+0000","last_fullsized":"2024-06-02T23:08:11.329323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335313+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:08:11.335313+0000","last_peered":"2024-06-02T23:08:11.335313+0000","last_clean":"2024-06-02T23:08:11.335313+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:08:11.335313+0000","last_undegraded":"2024-06-02T23:08:11.335313+0000","last_fullsized":"2024-06-02T23:08:11.335313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331466+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:08:11.331466+0000","last_peered":"2024-06-02T23:08:11.331466+0000","last_clean":"2024-06-02T23:08:11.331466+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:08:11.331466+0000","last_undegraded":"2024-06-02T23:08:11.331466+0000","last_fullsized":"2024-06-02T23:08:11.331466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328487+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328487+0000","last_peered":"2024-06-02T23:08:11.328487+0000","last_clean":"2024-06-02T23:08:11.328487+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328487+0000","last_undegraded":"2024-06-02T23:08:11.328487+0000","last_fullsized":"2024-06-02T23:08:11.328487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449500+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:08:11.449500+0000","last_peered":"2024-06-02T23:08:11.449500+0000","last_clean":"2024-06-02T23:08:11.449500+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:08:11.449500+0000","last_undegraded":"2024-06-02T23:08:11.449500+0000","last_fullsized":"2024-06-02T23:08:11.449500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:13.892744+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:08:13.892744+0000","last_peered":"2024-06-02T23:08:13.892744+0000","last_clean":"2024-06-02T23:08:13.892744+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:08:13.892744+0000","last_undegraded":"2024-06-02T23:08:13.892744+0000","last_fullsized":"2024-06-02T23:08:13.892744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449616+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:08:11.449616+0000","last_peered":"2024-06-02T23:08:11.449616+0000","last_clean":"2024-06-02T23:08:11.449616+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:08:11.449616+0000","last_undegraded":"2024-06-02T23:08:11.449616+0000","last_fullsized":"2024-06-02T23:08:11.449616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329755+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:08:11.329755+0000","last_peered":"2024-06-02T23:08:11.329755+0000","last_clean":"2024-06-02T23:08:11.329755+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:08:11.329755+0000","last_undegraded":"2024-06-02T23:08:11.329755+0000","last_fullsized":"2024-06-02T23:08:11.329755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:12.891363+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:08:12.891363+0000","last_peered":"2024-06-02T23:08:12.891363+0000","last_clean":"2024-06-02T23:08:12.891363+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:08:12.891363+0000","last_undegraded":"2024-06-02T23:08:12.891363+0000","last_fullsized":"2024-06-02T23:08:12.891363+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748857+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:08:11.748857+0000","last_peered":"2024-06-02T23:08:11.748857+0000","last_clean":"2024-06-02T23:08:11.748857+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:08:11.748857+0000","last_undegraded":"2024-06-02T23:08:11.748857+0000","last_fullsized":"2024-06-02T23:08:11.748857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450367+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450367+0000","last_peered":"2024-06-02T23:08:11.450367+0000","last_clean":"2024-06-02T23:08:11.450367+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450367+0000","last_undegraded":"2024-06-02T23:08:11.450367+0000","last_fullsized":"2024-06-02T23:08:11.450367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332820+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:11.332820+0000","last_peered":"2024-06-02T23:08:11.332820+0000","last_clean":"2024-06-02T23:08:11.332820+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:11.332820+0000","last_undegraded":"2024-06-02T23:08:11.332820+0000","last_fullsized":"2024-06-02T23:08:11.332820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332742+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:08:11.332742+0000","last_peered":"2024-06-02T23:08:11.332742+0000","last_clean":"2024-06-02T23:08:11.332742+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:08:11.332742+0000","last_undegraded":"2024-06-02T23:08:11.332742+0000","last_fullsized":"2024-06-02T23:08:11.332742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333015+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333015+0000","last_peered":"2024-06-02T23:08:11.333015+0000","last_clean":"2024-06-02T23:08:11.333015+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333015+0000","last_undegraded":"2024-06-02T23:08:11.333015+0000","last_fullsized":"2024-06-02T23:08:11.333015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335862+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335862+0000","last_peered":"2024-06-02T23:08:11.335862+0000","last_clean":"2024-06-02T23:08:11.335862+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335862+0000","last_undegraded":"2024-06-02T23:08:11.335862+0000","last_fullsized":"2024-06-02T23:08:11.335862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333307+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:08:11.333307+0000","last_peered":"2024-06-02T23:08:11.333307+0000","last_clean":"2024-06-02T23:08:11.333307+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:08:11.333307+0000","last_undegraded":"2024-06-02T23:08:11.333307+0000","last_fullsized":"2024-06-02T23:08:11.333307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.213929+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:08:14.213929+0000","last_peered":"2024-06-02T23:08:14.213929+0000","last_clean":"2024-06-02T23:08:14.213929+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:08:14.213929+0000","last_undegraded":"2024-06-02T23:08:14.213929+0000","last_fullsized":"2024-06-02T23:08:14.213929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749041+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:08:11.749041+0000","last_peered":"2024-06-02T23:08:11.749041+0000","last_clean":"2024-06-02T23:08:11.749041+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:08:11.749041+0000","last_undegraded":"2024-06-02T23:08:11.749041+0000","last_fullsized":"2024-06-02T23:08:11.749041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450403+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450403+0000","last_peered":"2024-06-02T23:08:11.450403+0000","last_clean":"2024-06-02T23:08:11.450403+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450403+0000","last_undegraded":"2024-06-02T23:08:11.450403+0000","last_fullsized":"2024-06-02T23:08:11.450403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.750091+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:08:11.750091+0000","last_peered":"2024-06-02T23:08:11.750091+0000","last_clean":"2024-06-02T23:08:11.750091+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:08:11.750091+0000","last_undegraded":"2024-06-02T23:08:11.750091+0000","last_fullsized":"2024-06-02T23:08:11.750091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334625+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:08:11.334625+0000","last_peered":"2024-06-02T23:08:11.334625+0000","last_clean":"2024-06-02T23:08:11.334625+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:08:11.334625+0000","last_undegraded":"2024-06-02T23:08:11.334625+0000","last_fullsized":"2024-06-02T23:08:11.334625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748865+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748865+0000","last_peered":"2024-06-02T23:08:11.748865+0000","last_clean":"2024-06-02T23:08:11.748865+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748865+0000","last_undegraded":"2024-06-02T23:08:11.748865+0000","last_fullsized":"2024-06-02T23:08:11.748865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332297+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:08:11.332297+0000","last_peered":"2024-06-02T23:08:11.332297+0000","last_clean":"2024-06-02T23:08:11.332297+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:08:11.332297+0000","last_undegraded":"2024-06-02T23:08:11.332297+0000","last_fullsized":"2024-06-02T23:08:11.332297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332588+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:08:11.332588+0000","last_peered":"2024-06-02T23:08:11.332588+0000","last_clean":"2024-06-02T23:08:11.332588+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:08:11.332588+0000","last_undegraded":"2024-06-02T23:08:11.332588+0000","last_fullsized":"2024-06-02T23:08:11.332588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748997+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:08:11.748997+0000","last_peered":"2024-06-02T23:08:11.748997+0000","last_clean":"2024-06-02T23:08:11.748997+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:08:11.748997+0000","last_undegraded":"2024-06-02T23:08:11.748997+0000","last_fullsized":"2024-06-02T23:08:11.748997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334746+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334746+0000","last_peered":"2024-06-02T23:08:11.334746+0000","last_clean":"2024-06-02T23:08:11.334746+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334746+0000","last_undegraded":"2024-06-02T23:08:11.334746+0000","last_fullsized":"2024-06-02T23:08:11.334746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333187+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:08:11.333187+0000","last_peered":"2024-06-02T23:08:11.333187+0000","last_clean":"2024-06-02T23:08:11.333187+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:08:11.333187+0000","last_undegraded":"2024-06-02T23:08:11.333187+0000","last_fullsized":"2024-06-02T23:08:11.333187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:12.931506+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:08:12.931506+0000","last_peered":"2024-06-02T23:08:12.931506+0000","last_clean":"2024-06-02T23:08:12.931506+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:08:12.931506+0000","last_undegraded":"2024-06-02T23:08:12.931506+0000","last_fullsized":"2024-06-02T23:08:12.931506+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332904+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:08:11.332904+0000","last_peered":"2024-06-02T23:08:11.332904+0000","last_clean":"2024-06-02T23:08:11.332904+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:08:11.332904+0000","last_undegraded":"2024-06-02T23:08:11.332904+0000","last_fullsized":"2024-06-02T23:08:11.332904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330120+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330120+0000","last_peered":"2024-06-02T23:08:11.330120+0000","last_clean":"2024-06-02T23:08:11.330120+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330120+0000","last_undegraded":"2024-06-02T23:08:11.330120+0000","last_fullsized":"2024-06-02T23:08:11.330120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332737+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:08:11.332737+0000","last_peered":"2024-06-02T23:08:11.332737+0000","last_clean":"2024-06-02T23:08:11.332737+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:08:11.332737+0000","last_undegraded":"2024-06-02T23:08:11.332737+0000","last_fullsized":"2024-06-02T23:08:11.332737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333475+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333475+0000","last_peered":"2024-06-02T23:08:11.333475+0000","last_clean":"2024-06-02T23:08:11.333475+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333475+0000","last_undegraded":"2024-06-02T23:08:11.333475+0000","last_fullsized":"2024-06-02T23:08:11.333475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332416+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:08:11.332416+0000","last_peered":"2024-06-02T23:08:11.332416+0000","last_clean":"2024-06-02T23:08:11.332416+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:08:11.332416+0000","last_undegraded":"2024-06-02T23:08:11.332416+0000","last_fullsized":"2024-06-02T23:08:11.332416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332160+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:08:11.332160+0000","last_peered":"2024-06-02T23:08:11.332160+0000","last_clean":"2024-06-02T23:08:11.332160+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:08:11.332160+0000","last_undegraded":"2024-06-02T23:08:11.332160+0000","last_fullsized":"2024-06-02T23:08:11.332160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332456+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:08:11.332456+0000","last_peered":"2024-06-02T23:08:11.332456+0000","last_clean":"2024-06-02T23:08:11.332456+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:08:11.332456+0000","last_undegraded":"2024-06-02T23:08:11.332456+0000","last_fullsized":"2024-06-02T23:08:11.332456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331147+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:08:11.331147+0000","last_peered":"2024-06-02T23:08:11.331147+0000","last_clean":"2024-06-02T23:08:11.331147+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:08:11.331147+0000","last_undegraded":"2024-06-02T23:08:11.331147+0000","last_fullsized":"2024-06-02T23:08:11.331147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335176+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:08:11.335176+0000","last_peered":"2024-06-02T23:08:11.335176+0000","last_clean":"2024-06-02T23:08:11.335176+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:08:11.335176+0000","last_undegraded":"2024-06-02T23:08:11.335176+0000","last_fullsized":"2024-06-02T23:08:11.335176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337292+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:08:11.337292+0000","last_peered":"2024-06-02T23:08:11.337292+0000","last_clean":"2024-06-02T23:08:11.337292+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:08:11.337292+0000","last_undegraded":"2024-06-02T23:08:11.337292+0000","last_fullsized":"2024-06-02T23:08:11.337292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333612+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:08:11.333612+0000","last_peered":"2024-06-02T23:08:11.333612+0000","last_clean":"2024-06-02T23:08:11.333612+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:08:11.333612+0000","last_undegraded":"2024-06-02T23:08:11.333612+0000","last_fullsized":"2024-06-02T23:08:11.333612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749044+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:08:11.749044+0000","last_peered":"2024-06-02T23:08:11.749044+0000","last_clean":"2024-06-02T23:08:11.749044+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:08:11.749044+0000","last_undegraded":"2024-06-02T23:08:11.749044+0000","last_fullsized":"2024-06-02T23:08:11.749044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329174+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:08:11.329174+0000","last_peered":"2024-06-02T23:08:11.329174+0000","last_clean":"2024-06-02T23:08:11.329174+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:08:11.329174+0000","last_undegraded":"2024-06-02T23:08:11.329174+0000","last_fullsized":"2024-06-02T23:08:11.329174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748682+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:08:11.748682+0000","last_peered":"2024-06-02T23:08:11.748682+0000","last_clean":"2024-06-02T23:08:11.748682+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:08:11.748682+0000","last_undegraded":"2024-06-02T23:08:11.748682+0000","last_fullsized":"2024-06-02T23:08:11.748682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334859+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:08:11.334859+0000","last_peered":"2024-06-02T23:08:11.334859+0000","last_clean":"2024-06-02T23:08:11.334859+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:08:11.334859+0000","last_undegraded":"2024-06-02T23:08:11.334859+0000","last_fullsized":"2024-06-02T23:08:11.334859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333208+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:08:11.333208+0000","last_peered":"2024-06-02T23:08:11.333208+0000","last_clean":"2024-06-02T23:08:11.333208+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:08:11.333208+0000","last_undegraded":"2024-06-02T23:08:11.333208+0000","last_fullsized":"2024-06-02T23:08:11.333208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337520+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:08:11.337520+0000","last_peered":"2024-06-02T23:08:11.337520+0000","last_clean":"2024-06-02T23:08:11.337520+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:08:11.337520+0000","last_undegraded":"2024-06-02T23:08:11.337520+0000","last_fullsized":"2024-06-02T23:08:11.337520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66649,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":277,"num_read_kb":215,"num_write":244,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":561152,"data_stored":194552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383705,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1131286,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332142,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1072855,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":540817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":481796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1064960,"data_stored":535410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36332,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708820,"statfs":{"total":95995035648,"available":95957831680,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":480550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072533,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:08:19.960 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:08:20.220 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:20 smithi174 ceph-mon[35994]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:08:20.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:20 smithi052 ceph-mon[32340]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:08:20.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:20 smithi123 ceph-mon[35977]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:08:21.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:21 smithi052 ceph-mon[32340]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 12 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-06-02T23:08:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:21 smithi174 ceph-mon[35994]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 12 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-06-02T23:08:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:21 smithi123 ceph-mon[35977]: pgmap v325: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.0 KiB/s wr, 12 op/s; 77 B/s, 1 keys/s, 1 objects/s recovering 2024-06-02T23:08:21.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 -- 172.21.15.52:0/1334024179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 msgr2=0x7f335010e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:21.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 --2- 172.21.15.52:0/1334024179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010e230 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f334800b0a0 tx=0x7f334800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:21.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 -- 172.21.15.52:0/1334024179 shutdown_connections 2024-06-02T23:08:21.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 --2- 172.21.15.52:0/1334024179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010e230 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:21.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 --2- 172.21.15.52:0/1334024179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33501073f0 0x7f33501077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:21.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 --2- 172.21.15.52:0/1334024179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f3350101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:21.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 -- 172.21.15.52:0/1334024179 >> 172.21.15.52:0/1334024179 conn(0x7f3350078300 msgr2=0x7f3350078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:21.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 -- 172.21.15.52:0/1334024179 shutdown_connections 2024-06-02T23:08:21.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.987+0000 7f3355de8700 1 -- 172.21.15.52:0/1334024179 wait complete. 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.988+0000 7f3355de8700 1 Processor -- start 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.988+0000 7f3355de8700 1 -- start start 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f3355de8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f335010eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f3355de8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f3355de8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33501073f0 0x7f335010f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f3355de8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3350112690 con 0x7f3350101d40 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f3355de8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3350112510 con 0x7f3350101380 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f3355de8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3350112810 con 0x7f33501073f0 2024-06-02T23:08:21.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f335010eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f335010eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45550/0 (socket says 172.21.15.52:45550) 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 -- 172.21.15.52:0/2156952269 learned_addr learned my addr 172.21.15.52:0/2156952269 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334effd700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334ffff700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33501073f0 0x7f335010f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 -- 172.21.15.52:0/2156952269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33501073f0 msgr2=0x7f335010f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33501073f0 0x7f335010f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 -- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 msgr2=0x7f335010f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 -- 172.21.15.52:0/2156952269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33501a70d0 con 0x7f3350101380 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334effd700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:08:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f335010eb40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f334000d950 tx=0x7f334000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.989+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3340016070 con 0x7f3350101380 2024-06-02T23:08:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.990+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f334000fd40 con 0x7f3350101380 2024-06-02T23:08:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.990+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33501a73c0 con 0x7f3350101380 2024-06-02T23:08:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.990+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334000e3b0 con 0x7f3350101380 2024-06-02T23:08:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.990+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33501a7900 con 0x7f3350101380 2024-06-02T23:08:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.990+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3350069180 con 0x7f3350101380 2024-06-02T23:08:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.992+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f3340019070 con 0x7f3350101380 2024-06-02T23:08:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.993+0000 7f334cff9700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cb0 0x7f333c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.993+0000 7f334effd700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cb0 0x7f333c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.993+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33400cec00 con 0x7f3350101380 2024-06-02T23:08:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.993+0000 7f334effd700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cb0 0x7f333c0a5170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3350078a20 tx=0x7f333800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:21.997+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f334009b9d0 con 0x7f3350101380 2024-06-02T23:08:22.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.198+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f33500628d0 con 0x7f333c0a2cb0 2024-06-02T23:08:22.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.205+0000 7f334cff9700 1 -- 172.21.15.52:0/2156952269 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258944 (secure 0 0 0) 0x7f33500628d0 con 0x7f333c0a2cb0 2024-06-02T23:08:22.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:22.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cb0 msgr2=0x7f333c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cb0 0x7f333c0a5170 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3350078a20 tx=0x7f333800a300 comp rx=0 tx=0).stop 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 msgr2=0x7f335010eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f335010eb40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f334000d950 tx=0x7f334000dc60 comp rx=0 tx=0).stop 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 shutdown_connections 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cb0 0x7f333c0a5170 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33501073f0 0x7f335010f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3350101380 0x7f335010eb40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:22.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 --2- 172.21.15.52:0/2156952269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3350101d40 0x7f335010f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:22.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 >> 172.21.15.52:0/2156952269 conn(0x7f3350078300 msgr2=0x7f3350100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:22.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 shutdown_connections 2024-06-02T23:08:22.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:22.210+0000 7f3355de8700 1 -- 172.21.15.52:0/2156952269 wait complete. 2024-06-02T23:08:22.212 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:08:22.946 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-06-02T23:08:21.772113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":657251,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":428,"num_read_kb":329,"num_write":427,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":348,"ondisk_log_size":348,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369656,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572168,"statfs":{"total":1151940427776,"available":1151561900032,"internally_reserved":0,"allocated":12435456,"data_stored":7722489,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31755,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":95,"num_read_kb":79,"num_write":22,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.444944"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331749+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:08:11.331749+0000","last_peered":"2024-06-02T23:08:11.331749+0000","last_clean":"2024-06-02T23:08:11.331749+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:08:11.331749+0000","last_undegraded":"2024-06-02T23:08:11.331749+0000","last_fullsized":"2024-06-02T23:08:11.331749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331436+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:08:11.331436+0000","last_peered":"2024-06-02T23:08:11.331436+0000","last_clean":"2024-06-02T23:08:11.331436+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:08:11.331436+0000","last_undegraded":"2024-06-02T23:08:11.331436+0000","last_fullsized":"2024-06-02T23:08:11.331436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331379+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:08:11.331379+0000","last_peered":"2024-06-02T23:08:11.331379+0000","last_clean":"2024-06-02T23:08:11.331379+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:08:11.331379+0000","last_undegraded":"2024-06-02T23:08:11.331379+0000","last_fullsized":"2024-06-02T23:08:11.331379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450456+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:08:11.450456+0000","last_peered":"2024-06-02T23:08:11.450456+0000","last_clean":"2024-06-02T23:08:11.450456+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:08:11.450456+0000","last_undegraded":"2024-06-02T23:08:11.450456+0000","last_fullsized":"2024-06-02T23:08:11.450456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334614+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:08:11.334614+0000","last_peered":"2024-06-02T23:08:11.334614+0000","last_clean":"2024-06-02T23:08:11.334614+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:08:11.334614+0000","last_undegraded":"2024-06-02T23:08:11.334614+0000","last_fullsized":"2024-06-02T23:08:11.334614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449583+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:08:11.449583+0000","last_peered":"2024-06-02T23:08:11.449583+0000","last_clean":"2024-06-02T23:08:11.449583+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:08:11.449583+0000","last_undegraded":"2024-06-02T23:08:11.449583+0000","last_fullsized":"2024-06-02T23:08:11.449583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332875+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:08:11.332875+0000","last_peered":"2024-06-02T23:08:11.332875+0000","last_clean":"2024-06-02T23:08:11.332875+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:08:11.332875+0000","last_undegraded":"2024-06-02T23:08:11.332875+0000","last_fullsized":"2024-06-02T23:08:11.332875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748655+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:08:11.748655+0000","last_peered":"2024-06-02T23:08:11.748655+0000","last_clean":"2024-06-02T23:08:11.748655+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:08:11.748655+0000","last_undegraded":"2024-06-02T23:08:11.748655+0000","last_fullsized":"2024-06-02T23:08:11.748655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334826+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:08:11.334826+0000","last_peered":"2024-06-02T23:08:11.334826+0000","last_clean":"2024-06-02T23:08:11.334826+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:08:11.334826+0000","last_undegraded":"2024-06-02T23:08:11.334826+0000","last_fullsized":"2024-06-02T23:08:11.334826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449830+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:08:11.449830+0000","last_peered":"2024-06-02T23:08:11.449830+0000","last_clean":"2024-06-02T23:08:11.449830+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:08:11.449830+0000","last_undegraded":"2024-06-02T23:08:11.449830+0000","last_fullsized":"2024-06-02T23:08:11.449830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332762+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:08:11.332762+0000","last_peered":"2024-06-02T23:08:11.332762+0000","last_clean":"2024-06-02T23:08:11.332762+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:08:11.332762+0000","last_undegraded":"2024-06-02T23:08:11.332762+0000","last_fullsized":"2024-06-02T23:08:11.332762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450303+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:08:11.450303+0000","last_peered":"2024-06-02T23:08:11.450303+0000","last_clean":"2024-06-02T23:08:11.450303+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:08:11.450303+0000","last_undegraded":"2024-06-02T23:08:11.450303+0000","last_fullsized":"2024-06-02T23:08:11.450303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333334+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:08:11.333334+0000","last_peered":"2024-06-02T23:08:11.333334+0000","last_clean":"2024-06-02T23:08:11.333334+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:08:11.333334+0000","last_undegraded":"2024-06-02T23:08:11.333334+0000","last_fullsized":"2024-06-02T23:08:11.333334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330935+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:08:11.330935+0000","last_peered":"2024-06-02T23:08:11.330935+0000","last_clean":"2024-06-02T23:08:11.330935+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:08:11.330935+0000","last_undegraded":"2024-06-02T23:08:11.330935+0000","last_fullsized":"2024-06-02T23:08:11.330935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330848+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:08:11.330848+0000","last_peered":"2024-06-02T23:08:11.330848+0000","last_clean":"2024-06-02T23:08:11.330848+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:08:11.330848+0000","last_undegraded":"2024-06-02T23:08:11.330848+0000","last_fullsized":"2024-06-02T23:08:11.330848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331976+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:08:11.331976+0000","last_peered":"2024-06-02T23:08:11.331976+0000","last_clean":"2024-06-02T23:08:11.331976+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:08:11.331976+0000","last_undegraded":"2024-06-02T23:08:11.331976+0000","last_fullsized":"2024-06-02T23:08:11.331976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328445+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:08:11.328445+0000","last_peered":"2024-06-02T23:08:11.328445+0000","last_clean":"2024-06-02T23:08:11.328445+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:08:11.328445+0000","last_undegraded":"2024-06-02T23:08:11.328445+0000","last_fullsized":"2024-06-02T23:08:11.328445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333538+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:08:11.333538+0000","last_peered":"2024-06-02T23:08:11.333538+0000","last_clean":"2024-06-02T23:08:11.333538+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:08:11.333538+0000","last_undegraded":"2024-06-02T23:08:11.333538+0000","last_fullsized":"2024-06-02T23:08:11.333538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":65,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.893590+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:08:14.893590+0000","last_peered":"2024-06-02T23:08:14.893590+0000","last_clean":"2024-06-02T23:08:14.893590+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:08:14.893590+0000","last_undegraded":"2024-06-02T23:08:14.893590+0000","last_fullsized":"2024-06-02T23:08:14.893590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748992+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:08:11.748992+0000","last_peered":"2024-06-02T23:08:11.748992+0000","last_clean":"2024-06-02T23:08:11.748992+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:08:11.748992+0000","last_undegraded":"2024-06-02T23:08:11.748992+0000","last_fullsized":"2024-06-02T23:08:11.748992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749145+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749145+0000","last_peered":"2024-06-02T23:08:11.749145+0000","last_clean":"2024-06-02T23:08:11.749145+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749145+0000","last_undegraded":"2024-06-02T23:08:11.749145+0000","last_fullsized":"2024-06-02T23:08:11.749145+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331872+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:08:11.331872+0000","last_peered":"2024-06-02T23:08:11.331872+0000","last_clean":"2024-06-02T23:08:11.331872+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:08:11.331872+0000","last_undegraded":"2024-06-02T23:08:11.331872+0000","last_fullsized":"2024-06-02T23:08:11.331872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.365621+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:11.365621+0000","last_peered":"2024-06-02T23:08:11.365621+0000","last_clean":"2024-06-02T23:08:11.365621+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:11.365621+0000","last_undegraded":"2024-06-02T23:08:11.365621+0000","last_fullsized":"2024-06-02T23:08:11.365621+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334580+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:08:11.334580+0000","last_peered":"2024-06-02T23:08:11.334580+0000","last_clean":"2024-06-02T23:08:11.334580+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:08:11.334580+0000","last_undegraded":"2024-06-02T23:08:11.334580+0000","last_fullsized":"2024-06-02T23:08:11.334580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335715+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335715+0000","last_peered":"2024-06-02T23:08:11.335715+0000","last_clean":"2024-06-02T23:08:11.335715+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335715+0000","last_undegraded":"2024-06-02T23:08:11.335715+0000","last_fullsized":"2024-06-02T23:08:11.335715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329756+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:08:11.329756+0000","last_peered":"2024-06-02T23:08:11.329756+0000","last_clean":"2024-06-02T23:08:11.329756+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:08:11.329756+0000","last_undegraded":"2024-06-02T23:08:11.329756+0000","last_fullsized":"2024-06-02T23:08:11.329756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":72,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:17.461601+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:08:17.461601+0000","last_peered":"2024-06-02T23:08:17.461601+0000","last_clean":"2024-06-02T23:08:17.461601+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:08:17.461601+0000","last_undegraded":"2024-06-02T23:08:17.461601+0000","last_fullsized":"2024-06-02T23:08:17.461601+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334793+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:08:11.334793+0000","last_peered":"2024-06-02T23:08:11.334793+0000","last_clean":"2024-06-02T23:08:11.334793+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:08:11.334793+0000","last_undegraded":"2024-06-02T23:08:11.334793+0000","last_fullsized":"2024-06-02T23:08:11.334793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449532+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449532+0000","last_peered":"2024-06-02T23:08:11.449532+0000","last_clean":"2024-06-02T23:08:11.449532+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449532+0000","last_undegraded":"2024-06-02T23:08:11.449532+0000","last_fullsized":"2024-06-02T23:08:11.449532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329554+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:08:11.329554+0000","last_peered":"2024-06-02T23:08:11.329554+0000","last_clean":"2024-06-02T23:08:11.329554+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:08:11.329554+0000","last_undegraded":"2024-06-02T23:08:11.329554+0000","last_fullsized":"2024-06-02T23:08:11.329554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332891+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332891+0000","last_peered":"2024-06-02T23:08:11.332891+0000","last_clean":"2024-06-02T23:08:11.332891+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332891+0000","last_undegraded":"2024-06-02T23:08:11.332891+0000","last_fullsized":"2024-06-02T23:08:11.332891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748738+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:08:11.748738+0000","last_peered":"2024-06-02T23:08:11.748738+0000","last_clean":"2024-06-02T23:08:11.748738+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:08:11.748738+0000","last_undegraded":"2024-06-02T23:08:11.748738+0000","last_fullsized":"2024-06-02T23:08:11.748738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329228+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329228+0000","last_peered":"2024-06-02T23:08:11.329228+0000","last_clean":"2024-06-02T23:08:11.329228+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329228+0000","last_undegraded":"2024-06-02T23:08:11.329228+0000","last_fullsized":"2024-06-02T23:08:11.329228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328243+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:08:11.328243+0000","last_peered":"2024-06-02T23:08:11.328243+0000","last_clean":"2024-06-02T23:08:11.328243+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:08:11.328243+0000","last_undegraded":"2024-06-02T23:08:11.328243+0000","last_fullsized":"2024-06-02T23:08:11.328243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":81,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:16.459601+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:08:16.459601+0000","last_peered":"2024-06-02T23:08:16.459601+0000","last_clean":"2024-06-02T23:08:16.459601+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:08:16.459601+0000","last_undegraded":"2024-06-02T23:08:16.459601+0000","last_fullsized":"2024-06-02T23:08:16.459601+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334817+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:08:11.334817+0000","last_peered":"2024-06-02T23:08:11.334817+0000","last_clean":"2024-06-02T23:08:11.334817+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:08:11.334817+0000","last_undegraded":"2024-06-02T23:08:11.334817+0000","last_fullsized":"2024-06-02T23:08:11.334817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332982+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.332982+0000","last_peered":"2024-06-02T23:08:11.332982+0000","last_clean":"2024-06-02T23:08:11.332982+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.332982+0000","last_undegraded":"2024-06-02T23:08:11.332982+0000","last_fullsized":"2024-06-02T23:08:11.332982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331775+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:08:11.331775+0000","last_peered":"2024-06-02T23:08:11.331775+0000","last_clean":"2024-06-02T23:08:11.331775+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:08:11.331775+0000","last_undegraded":"2024-06-02T23:08:11.331775+0000","last_fullsized":"2024-06-02T23:08:11.331775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329928+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.329928+0000","last_peered":"2024-06-02T23:08:11.329928+0000","last_clean":"2024-06-02T23:08:11.329928+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.329928+0000","last_undegraded":"2024-06-02T23:08:11.329928+0000","last_fullsized":"2024-06-02T23:08:11.329928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334827+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:08:11.334827+0000","last_peered":"2024-06-02T23:08:11.334827+0000","last_clean":"2024-06-02T23:08:11.334827+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:08:11.334827+0000","last_undegraded":"2024-06-02T23:08:11.334827+0000","last_fullsized":"2024-06-02T23:08:11.334827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'30","reported_seq":114,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:16.460276+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:08:16.460276+0000","last_peered":"2024-06-02T23:08:16.460276+0000","last_clean":"2024-06-02T23:08:16.460276+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:08:16.460276+0000","last_undegraded":"2024-06-02T23:08:16.460276+0000","last_fullsized":"2024-06-02T23:08:16.460276+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59837,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":86,"num_write":32,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335764+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:08:11.335764+0000","last_peered":"2024-06-02T23:08:11.335764+0000","last_clean":"2024-06-02T23:08:11.335764+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:08:11.335764+0000","last_undegraded":"2024-06-02T23:08:11.335764+0000","last_fullsized":"2024-06-02T23:08:11.335764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332546+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332546+0000","last_peered":"2024-06-02T23:08:11.332546+0000","last_clean":"2024-06-02T23:08:11.332546+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332546+0000","last_undegraded":"2024-06-02T23:08:11.332546+0000","last_fullsized":"2024-06-02T23:08:11.332546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333123+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:08:11.333123+0000","last_peered":"2024-06-02T23:08:11.333123+0000","last_clean":"2024-06-02T23:08:11.333123+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:08:11.333123+0000","last_undegraded":"2024-06-02T23:08:11.333123+0000","last_fullsized":"2024-06-02T23:08:11.333123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329400+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329400+0000","last_peered":"2024-06-02T23:08:11.329400+0000","last_clean":"2024-06-02T23:08:11.329400+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329400+0000","last_undegraded":"2024-06-02T23:08:11.329400+0000","last_fullsized":"2024-06-02T23:08:11.329400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332914+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:08:11.332914+0000","last_peered":"2024-06-02T23:08:11.332914+0000","last_clean":"2024-06-02T23:08:11.332914+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:08:11.332914+0000","last_undegraded":"2024-06-02T23:08:11.332914+0000","last_fullsized":"2024-06-02T23:08:11.332914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:18.900711+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:08:18.900711+0000","last_peered":"2024-06-02T23:08:18.900711+0000","last_clean":"2024-06-02T23:08:18.900711+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:08:18.900711+0000","last_undegraded":"2024-06-02T23:08:18.900711+0000","last_fullsized":"2024-06-02T23:08:18.900711+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332579+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:08:11.332579+0000","last_peered":"2024-06-02T23:08:11.332579+0000","last_clean":"2024-06-02T23:08:11.332579+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:08:11.332579+0000","last_undegraded":"2024-06-02T23:08:11.332579+0000","last_fullsized":"2024-06-02T23:08:11.332579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.348940+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:11.348940+0000","last_peered":"2024-06-02T23:08:11.348940+0000","last_clean":"2024-06-02T23:08:11.348940+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:11.348940+0000","last_undegraded":"2024-06-02T23:08:11.348940+0000","last_fullsized":"2024-06-02T23:08:11.348940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329323+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:08:11.329323+0000","last_peered":"2024-06-02T23:08:11.329323+0000","last_clean":"2024-06-02T23:08:11.329323+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:08:11.329323+0000","last_undegraded":"2024-06-02T23:08:11.329323+0000","last_fullsized":"2024-06-02T23:08:11.329323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335313+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:08:11.335313+0000","last_peered":"2024-06-02T23:08:11.335313+0000","last_clean":"2024-06-02T23:08:11.335313+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:08:11.335313+0000","last_undegraded":"2024-06-02T23:08:11.335313+0000","last_fullsized":"2024-06-02T23:08:11.335313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331466+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:08:11.331466+0000","last_peered":"2024-06-02T23:08:11.331466+0000","last_clean":"2024-06-02T23:08:11.331466+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:08:11.331466+0000","last_undegraded":"2024-06-02T23:08:11.331466+0000","last_fullsized":"2024-06-02T23:08:11.331466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328487+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328487+0000","last_peered":"2024-06-02T23:08:11.328487+0000","last_clean":"2024-06-02T23:08:11.328487+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328487+0000","last_undegraded":"2024-06-02T23:08:11.328487+0000","last_fullsized":"2024-06-02T23:08:11.328487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449500+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:08:11.449500+0000","last_peered":"2024-06-02T23:08:11.449500+0000","last_clean":"2024-06-02T23:08:11.449500+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:08:11.449500+0000","last_undegraded":"2024-06-02T23:08:11.449500+0000","last_fullsized":"2024-06-02T23:08:11.449500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":61,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:18.899631+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:08:18.899631+0000","last_peered":"2024-06-02T23:08:18.899631+0000","last_clean":"2024-06-02T23:08:18.899631+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:08:18.899631+0000","last_undegraded":"2024-06-02T23:08:18.899631+0000","last_fullsized":"2024-06-02T23:08:18.899631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449616+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:08:11.449616+0000","last_peered":"2024-06-02T23:08:11.449616+0000","last_clean":"2024-06-02T23:08:11.449616+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:08:11.449616+0000","last_undegraded":"2024-06-02T23:08:11.449616+0000","last_fullsized":"2024-06-02T23:08:11.449616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329755+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:08:11.329755+0000","last_peered":"2024-06-02T23:08:11.329755+0000","last_clean":"2024-06-02T23:08:11.329755+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:08:11.329755+0000","last_undegraded":"2024-06-02T23:08:11.329755+0000","last_fullsized":"2024-06-02T23:08:11.329755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:17.898609+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:08:17.898609+0000","last_peered":"2024-06-02T23:08:17.898609+0000","last_clean":"2024-06-02T23:08:17.898609+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:08:17.898609+0000","last_undegraded":"2024-06-02T23:08:17.898609+0000","last_fullsized":"2024-06-02T23:08:17.898609+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":29,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748857+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:08:11.748857+0000","last_peered":"2024-06-02T23:08:11.748857+0000","last_clean":"2024-06-02T23:08:11.748857+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:08:11.748857+0000","last_undegraded":"2024-06-02T23:08:11.748857+0000","last_fullsized":"2024-06-02T23:08:11.748857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450367+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450367+0000","last_peered":"2024-06-02T23:08:11.450367+0000","last_clean":"2024-06-02T23:08:11.450367+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450367+0000","last_undegraded":"2024-06-02T23:08:11.450367+0000","last_fullsized":"2024-06-02T23:08:11.450367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332820+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:11.332820+0000","last_peered":"2024-06-02T23:08:11.332820+0000","last_clean":"2024-06-02T23:08:11.332820+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:11.332820+0000","last_undegraded":"2024-06-02T23:08:11.332820+0000","last_fullsized":"2024-06-02T23:08:11.332820+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332742+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:08:11.332742+0000","last_peered":"2024-06-02T23:08:11.332742+0000","last_clean":"2024-06-02T23:08:11.332742+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:08:11.332742+0000","last_undegraded":"2024-06-02T23:08:11.332742+0000","last_fullsized":"2024-06-02T23:08:11.332742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333015+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333015+0000","last_peered":"2024-06-02T23:08:11.333015+0000","last_clean":"2024-06-02T23:08:11.333015+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333015+0000","last_undegraded":"2024-06-02T23:08:11.333015+0000","last_fullsized":"2024-06-02T23:08:11.333015+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335862+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335862+0000","last_peered":"2024-06-02T23:08:11.335862+0000","last_clean":"2024-06-02T23:08:11.335862+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335862+0000","last_undegraded":"2024-06-02T23:08:11.335862+0000","last_fullsized":"2024-06-02T23:08:11.335862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333307+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:08:11.333307+0000","last_peered":"2024-06-02T23:08:11.333307+0000","last_clean":"2024-06-02T23:08:11.333307+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:08:11.333307+0000","last_undegraded":"2024-06-02T23:08:11.333307+0000","last_fullsized":"2024-06-02T23:08:11.333307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.213929+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:08:14.213929+0000","last_peered":"2024-06-02T23:08:14.213929+0000","last_clean":"2024-06-02T23:08:14.213929+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:08:14.213929+0000","last_undegraded":"2024-06-02T23:08:14.213929+0000","last_fullsized":"2024-06-02T23:08:14.213929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749041+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:08:11.749041+0000","last_peered":"2024-06-02T23:08:11.749041+0000","last_clean":"2024-06-02T23:08:11.749041+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:08:11.749041+0000","last_undegraded":"2024-06-02T23:08:11.749041+0000","last_fullsized":"2024-06-02T23:08:11.749041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450403+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450403+0000","last_peered":"2024-06-02T23:08:11.450403+0000","last_clean":"2024-06-02T23:08:11.450403+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450403+0000","last_undegraded":"2024-06-02T23:08:11.450403+0000","last_fullsized":"2024-06-02T23:08:11.450403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.750091+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:08:11.750091+0000","last_peered":"2024-06-02T23:08:11.750091+0000","last_clean":"2024-06-02T23:08:11.750091+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:08:11.750091+0000","last_undegraded":"2024-06-02T23:08:11.750091+0000","last_fullsized":"2024-06-02T23:08:11.750091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334625+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:08:11.334625+0000","last_peered":"2024-06-02T23:08:11.334625+0000","last_clean":"2024-06-02T23:08:11.334625+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:08:11.334625+0000","last_undegraded":"2024-06-02T23:08:11.334625+0000","last_fullsized":"2024-06-02T23:08:11.334625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748865+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748865+0000","last_peered":"2024-06-02T23:08:11.748865+0000","last_clean":"2024-06-02T23:08:11.748865+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748865+0000","last_undegraded":"2024-06-02T23:08:11.748865+0000","last_fullsized":"2024-06-02T23:08:11.748865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332297+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:08:11.332297+0000","last_peered":"2024-06-02T23:08:11.332297+0000","last_clean":"2024-06-02T23:08:11.332297+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:08:11.332297+0000","last_undegraded":"2024-06-02T23:08:11.332297+0000","last_fullsized":"2024-06-02T23:08:11.332297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332588+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:08:11.332588+0000","last_peered":"2024-06-02T23:08:11.332588+0000","last_clean":"2024-06-02T23:08:11.332588+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:08:11.332588+0000","last_undegraded":"2024-06-02T23:08:11.332588+0000","last_fullsized":"2024-06-02T23:08:11.332588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748997+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:08:11.748997+0000","last_peered":"2024-06-02T23:08:11.748997+0000","last_clean":"2024-06-02T23:08:11.748997+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:08:11.748997+0000","last_undegraded":"2024-06-02T23:08:11.748997+0000","last_fullsized":"2024-06-02T23:08:11.748997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334746+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334746+0000","last_peered":"2024-06-02T23:08:11.334746+0000","last_clean":"2024-06-02T23:08:11.334746+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334746+0000","last_undegraded":"2024-06-02T23:08:11.334746+0000","last_fullsized":"2024-06-02T23:08:11.334746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333187+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:08:11.333187+0000","last_peered":"2024-06-02T23:08:11.333187+0000","last_clean":"2024-06-02T23:08:11.333187+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:08:11.333187+0000","last_undegraded":"2024-06-02T23:08:11.333187+0000","last_fullsized":"2024-06-02T23:08:11.333187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":63,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:17.939632+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:08:17.939632+0000","last_peered":"2024-06-02T23:08:17.939632+0000","last_clean":"2024-06-02T23:08:17.939632+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:08:17.939632+0000","last_undegraded":"2024-06-02T23:08:17.939632+0000","last_fullsized":"2024-06-02T23:08:17.939632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332904+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:08:11.332904+0000","last_peered":"2024-06-02T23:08:11.332904+0000","last_clean":"2024-06-02T23:08:11.332904+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:08:11.332904+0000","last_undegraded":"2024-06-02T23:08:11.332904+0000","last_fullsized":"2024-06-02T23:08:11.332904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330120+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330120+0000","last_peered":"2024-06-02T23:08:11.330120+0000","last_clean":"2024-06-02T23:08:11.330120+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330120+0000","last_undegraded":"2024-06-02T23:08:11.330120+0000","last_fullsized":"2024-06-02T23:08:11.330120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332737+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:08:11.332737+0000","last_peered":"2024-06-02T23:08:11.332737+0000","last_clean":"2024-06-02T23:08:11.332737+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:08:11.332737+0000","last_undegraded":"2024-06-02T23:08:11.332737+0000","last_fullsized":"2024-06-02T23:08:11.332737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333475+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333475+0000","last_peered":"2024-06-02T23:08:11.333475+0000","last_clean":"2024-06-02T23:08:11.333475+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333475+0000","last_undegraded":"2024-06-02T23:08:11.333475+0000","last_fullsized":"2024-06-02T23:08:11.333475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332416+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:08:11.332416+0000","last_peered":"2024-06-02T23:08:11.332416+0000","last_clean":"2024-06-02T23:08:11.332416+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:08:11.332416+0000","last_undegraded":"2024-06-02T23:08:11.332416+0000","last_fullsized":"2024-06-02T23:08:11.332416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332160+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:08:11.332160+0000","last_peered":"2024-06-02T23:08:11.332160+0000","last_clean":"2024-06-02T23:08:11.332160+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:08:11.332160+0000","last_undegraded":"2024-06-02T23:08:11.332160+0000","last_fullsized":"2024-06-02T23:08:11.332160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332456+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:08:11.332456+0000","last_peered":"2024-06-02T23:08:11.332456+0000","last_clean":"2024-06-02T23:08:11.332456+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:08:11.332456+0000","last_undegraded":"2024-06-02T23:08:11.332456+0000","last_fullsized":"2024-06-02T23:08:11.332456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331147+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:08:11.331147+0000","last_peered":"2024-06-02T23:08:11.331147+0000","last_clean":"2024-06-02T23:08:11.331147+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:08:11.331147+0000","last_undegraded":"2024-06-02T23:08:11.331147+0000","last_fullsized":"2024-06-02T23:08:11.331147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335176+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:08:11.335176+0000","last_peered":"2024-06-02T23:08:11.335176+0000","last_clean":"2024-06-02T23:08:11.335176+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:08:11.335176+0000","last_undegraded":"2024-06-02T23:08:11.335176+0000","last_fullsized":"2024-06-02T23:08:11.335176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337292+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:08:11.337292+0000","last_peered":"2024-06-02T23:08:11.337292+0000","last_clean":"2024-06-02T23:08:11.337292+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:08:11.337292+0000","last_undegraded":"2024-06-02T23:08:11.337292+0000","last_fullsized":"2024-06-02T23:08:11.337292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333612+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:08:11.333612+0000","last_peered":"2024-06-02T23:08:11.333612+0000","last_clean":"2024-06-02T23:08:11.333612+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:08:11.333612+0000","last_undegraded":"2024-06-02T23:08:11.333612+0000","last_fullsized":"2024-06-02T23:08:11.333612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749044+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:08:11.749044+0000","last_peered":"2024-06-02T23:08:11.749044+0000","last_clean":"2024-06-02T23:08:11.749044+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:08:11.749044+0000","last_undegraded":"2024-06-02T23:08:11.749044+0000","last_fullsized":"2024-06-02T23:08:11.749044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329174+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:08:11.329174+0000","last_peered":"2024-06-02T23:08:11.329174+0000","last_clean":"2024-06-02T23:08:11.329174+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:08:11.329174+0000","last_undegraded":"2024-06-02T23:08:11.329174+0000","last_fullsized":"2024-06-02T23:08:11.329174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748682+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:08:11.748682+0000","last_peered":"2024-06-02T23:08:11.748682+0000","last_clean":"2024-06-02T23:08:11.748682+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:08:11.748682+0000","last_undegraded":"2024-06-02T23:08:11.748682+0000","last_fullsized":"2024-06-02T23:08:11.748682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334859+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:08:11.334859+0000","last_peered":"2024-06-02T23:08:11.334859+0000","last_clean":"2024-06-02T23:08:11.334859+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:08:11.334859+0000","last_undegraded":"2024-06-02T23:08:11.334859+0000","last_fullsized":"2024-06-02T23:08:11.334859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333208+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:08:11.333208+0000","last_peered":"2024-06-02T23:08:11.333208+0000","last_clean":"2024-06-02T23:08:11.333208+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:08:11.333208+0000","last_undegraded":"2024-06-02T23:08:11.333208+0000","last_fullsized":"2024-06-02T23:08:11.333208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337520+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:08:11.337520+0000","last_peered":"2024-06-02T23:08:11.337520+0000","last_clean":"2024-06-02T23:08:11.337520+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:08:11.337520+0000","last_undegraded":"2024-06-02T23:08:11.337520+0000","last_fullsized":"2024-06-02T23:08:11.337520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66649,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":302,"num_read_kb":220,"num_write":244,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":201935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":209,"ondisk_log_size":209,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383706,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1131286,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332143,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1072855,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495667,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":540817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":481796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":540805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":480550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072533,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":61573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:08:22.950 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-02T23:08:22.951 INFO:tasks.cephadm:Setting up client nodes... 2024-06-02T23:08:22.951 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-06-02T23:08:23.248 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:23 smithi174 ceph-mon[35994]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s; 70 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:23 smithi174 ceph-mon[35994]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:08:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:23 smithi052 ceph-mon[32340]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s; 70 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:23 smithi052 ceph-mon[32340]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:08:23.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:23 smithi123 ceph-mon[35977]: pgmap v326: 97 pgs: 97 active+clean; 642 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.5 KiB/s wr, 11 op/s; 70 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:23 smithi123 ceph-mon[35977]: from='client.24563 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:08:24.609 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:24 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:08:24.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:24 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:08:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:24 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:08:24.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/474824008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec1040e0 msgr2=0x7fcdec10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/474824008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec1040e0 0x7fcdec10a990 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fcde400b0d0 tx=0x7fcde400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/474824008 shutdown_connections 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/474824008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec1040e0 0x7fcdec10a990 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/474824008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdec102540 0x7fcdec102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/474824008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdec103740 0x7fcdec103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/474824008 >> 172.21.15.52:0/474824008 conn(0x7fcdec0fdae0 msgr2=0x7fcdec0fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/474824008 shutdown_connections 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/474824008 wait complete. 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 Processor -- start 2024-06-02T23:08:24.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.887+0000 7fcdf1bb9700 1 -- start start 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdec102540 0x7fcdec076230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 0x7fcdec074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdec1040e0 0x7fcdec074dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec10dfc0 con 0x7fcdec103740 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec10de40 con 0x7fcdec1040e0 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdec10e140 con 0x7fcdec102540 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 0x7fcdec074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 0x7fcdec074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53072/0 (socket says 172.21.15.52:53072) 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 -- 172.21.15.52:0/4283636772 learned_addr learned my addr 172.21.15.52:0/4283636772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdebfff700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdec1040e0 0x7fcdec074dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdeb7fe700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdec102540 0x7fcdec076230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 -- 172.21.15.52:0/4283636772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdec102540 msgr2=0x7fcdec076230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:24.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdec102540 0x7fcdec076230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 -- 172.21.15.52:0/4283636772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdec1040e0 msgr2=0x7fcdec074dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdec1040e0 0x7fcdec074dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 -- 172.21.15.52:0/4283636772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdec075670 con 0x7fcdec103740 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde3fff700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 0x7fcdec074880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400d950 tx=0x7fcdd400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4016070 con 0x7fcdec103740 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdec1a26a0 con 0x7fcdec103740 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.888+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcdd400fd40 con 0x7fcdec103740 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.889+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdec1a2be0 con 0x7fcdec103740 2024-06-02T23:08:24.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.889+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdd400e3b0 con 0x7fcdec103740 2024-06-02T23:08:24.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.890+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdec050a00 con 0x7fcdec103740 2024-06-02T23:08:24.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.891+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fcdd4019070 con 0x7fcdec103740 2024-06-02T23:08:24.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.892+0000 7fcde97fa700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcdd80a2d60 0x7fcdd80a5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:24.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.892+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcdd40cef70 con 0x7fcdec103740 2024-06-02T23:08:24.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.892+0000 7fcdeb7fe700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcdd80a2d60 0x7fcdd80a5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:24.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.893+0000 7fcdeb7fe700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcdd80a2d60 0x7fcdd80a5220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdec1035a0 tx=0x7fcddc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:24.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:24.897+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcdd409be40 con 0x7fcdec103740 2024-06-02T23:08:25.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.190+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fcdec02dd40 con 0x7fcdec103740 2024-06-02T23:08:25.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.192+0000 7fcde97fa700 1 -- 172.21.15.52:0/4283636772 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fcdec02dd40 con 0x7fcdec103740 2024-06-02T23:08:25.192 INFO:teuthology.orchestra.run.smithi052.stdout:[client.0] 2024-06-02T23:08:25.192 INFO:teuthology.orchestra.run.smithi052.stdout: key = AQBp+1xmGypyCxAAchLbxJkhSVGzO7RvlTJcjg== 2024-06-02T23:08:25.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcdd80a2d60 msgr2=0x7fcdd80a5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcdd80a2d60 0x7fcdd80a5220 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcdec1035a0 tx=0x7fcddc00b410 comp rx=0 tx=0).stop 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 msgr2=0x7fcdec074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 0x7fcdec074880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400d950 tx=0x7fcdd400dc60 comp rx=0 tx=0).stop 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 shutdown_connections 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcdd80a2d60 0x7fcdd80a5220 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcdec1040e0 0x7fcdec074dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdec102540 0x7fcdec076230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 --2- 172.21.15.52:0/4283636772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcdec103740 0x7fcdec074880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 >> 172.21.15.52:0/4283636772 conn(0x7fcdec0fdae0 msgr2=0x7fcdec106970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 shutdown_connections 2024-06-02T23:08:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:25.195+0000 7fcdf1bb9700 1 -- 172.21.15.52:0/4283636772 wait complete. 2024-06-02T23:08:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4283636772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-02T23:08:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4283636772' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-02T23:08:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:25.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4283636772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-02T23:08:25.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4283636772' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-02T23:08:25.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: pgmap v327: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.2 KiB/s wr, 11 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4283636772' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4283636772' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:26.168 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:08:26.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-06-02T23:08:26.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-06-02T23:08:26.208 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-06-02T23:08:26.208 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-06-02T23:08:26.208 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph mgr dump --format=json 2024-06-02T23:08:26.518 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:27 smithi174 ceph-mon[35994]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:27 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:27 smithi052 ceph-mon[32340]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:27 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:27 smithi123 ceph-mon[35977]: pgmap v328: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s; 61 B/s, 0 keys/s, 1 objects/s recovering 2024-06-02T23:08:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:27 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:28.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1452605507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7381084f0 msgr2=0x7fa7381088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:28.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1452605507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7381084f0 0x7fa7381088d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b0a0 tx=0x7fa72800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1452605507 shutdown_connections 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1452605507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa73806b3c0 0x7fa73810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1452605507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7381084f0 0x7fa7381088d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1452605507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806aa00 0x7fa73806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1452605507 >> 172.21.15.52:0/1452605507 conn(0x7fa738078300 msgr2=0x7fa738078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1452605507 shutdown_connections 2024-06-02T23:08:28.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.272+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1452605507 wait complete. 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 Processor -- start 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 -- start start 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 0x7fa73819e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806b3c0 0x7fa73819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7381084f0 0x7fa7381989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa738112680 con 0x7fa73806aa00 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa738112500 con 0x7fa7381084f0 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.273+0000 7fa73cb6e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa738112800 con 0x7fa73806b3c0 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 0x7fa73819e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:28.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 0x7fa73819e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53094/0 (socket says 172.21.15.52:53094) 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 -- 172.21.15.52:0/1300049367 learned_addr learned my addr 172.21.15.52:0/1300049367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa736d9d700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7381084f0 0x7fa7381989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa735d9b700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806b3c0 0x7fa73819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 -- 172.21.15.52:0/1300049367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806b3c0 msgr2=0x7fa73819ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806b3c0 0x7fa73819ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 -- 172.21.15.52:0/1300049367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7381084f0 msgr2=0x7fa7381989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7381084f0 0x7fa7381989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 -- 172.21.15.52:0/1300049367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738199120 con 0x7fa73806aa00 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa735d9b700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806b3c0 0x7fa73819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73659c700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 0x7fa73819e800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b070 tx=0x7fa728004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa728021070 con 0x7fa73806aa00 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7381993b0 con 0x7fa73806aa00 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa728007c80 con 0x7fa73806aa00 2024-06-02T23:08:28.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa72801d450 con 0x7fa73806aa00 2024-06-02T23:08:28.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.274+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7381a56d0 con 0x7fa73806aa00 2024-06-02T23:08:28.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.275+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa73810a9d0 con 0x7fa73806aa00 2024-06-02T23:08:28.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.276+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa728003a00 con 0x7fa73806aa00 2024-06-02T23:08:28.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.277+0000 7fa72f7fe700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7240a2cb0 0x7fa7240a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:28.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.277+0000 7fa735d9b700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7240a2cb0 0x7fa7240a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:28.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.277+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa72800f020 con 0x7fa73806aa00 2024-06-02T23:08:28.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.278+0000 7fa735d9b700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7240a2cb0 0x7fa7240a5170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7380789e0 tx=0x7fa720008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:28.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.281+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa72802a020 con 0x7fa73806aa00 2024-06-02T23:08:28.556 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:08:28.556 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:08:28.556 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:28.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.555+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fa738199ef0 con 0x7fa73806aa00 2024-06-02T23:08:28.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.560+0000 7fa72f7fe700 1 -- 172.21.15.52:0/1300049367 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+263199 (secure 0 0 0) 0x7fa72809dad0 con 0x7fa73806aa00 2024-06-02T23:08:28.564 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.563+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7240a2cb0 msgr2=0x7fa7240a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.563+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7240a2cb0 0x7fa7240a5170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa7380789e0 tx=0x7fa720008040 comp rx=0 tx=0).stop 2024-06-02T23:08:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.563+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 msgr2=0x7fa73819e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.563+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 0x7fa73819e800 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa72800b070 tx=0x7fa728004db0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 shutdown_connections 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7240a2cb0 0x7fa7240a5170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7381084f0 0x7fa7381989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa73806aa00 0x7fa73819e800 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 --2- 172.21.15.52:0/1300049367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa73806b3c0 0x7fa73819ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 >> 172.21.15.52:0/1300049367 conn(0x7fa738078300 msgr2=0x7fa7381001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 shutdown_connections 2024-06-02T23:08:28.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:28.564+0000 7fa73cb6e700 1 -- 172.21.15.52:0/1300049367 wait complete. 2024-06-02T23:08:28.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:08:28.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:08:28.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:28.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:08:28.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:08:28.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:08:29.269 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6800","nonce":2596667410},{"type":"v1","addr":"172.21.15.52:6801","nonce":2596667410}]},"active_addr":"172.21.15.52:6801/2596667410","active_change":"2024-06-02T23:00:09.671774+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.52:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":3561073402}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":1949636686}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":392979611}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":4213821482}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.52:0","nonce":814665203}]}]} 2024-06-02T23:08:29.276 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-06-02T23:08:29.277 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-06-02T23:08:29.277 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:08:29.534 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:29.552 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:29 smithi052 ceph-mon[32340]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-06-02T23:08:29.552 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:29 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1300049367' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-02T23:08:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:29 smithi174 ceph-mon[35994]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-06-02T23:08:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:29 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1300049367' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-02T23:08:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:29 smithi123 ceph-mon[35977]: pgmap v329: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 853 B/s wr, 11 op/s 2024-06-02T23:08:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:29 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1300049367' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 -- 172.21.15.52:0/3808843227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c102740 msgr2=0x7fc21c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3808843227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c102740 0x7fc21c102b20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fc20c0099c0 tx=0x7fc20c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 -- 172.21.15.52:0/3808843227 shutdown_connections 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3808843227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c107840 0x7fc21c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3808843227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c102740 0x7fc21c102b20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3808843227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21c1030f0 0x7fc21c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 -- 172.21.15.52:0/3808843227 >> 172.21.15.52:0/3808843227 conn(0x7fc21c0795f0 msgr2=0x7fc21c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 -- 172.21.15.52:0/3808843227 shutdown_connections 2024-06-02T23:08:31.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.234+0000 7fc220e3d700 1 -- 172.21.15.52:0/3808843227 wait complete. 2024-06-02T23:08:31.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 Processor -- start 2024-06-02T23:08:31.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 -- start start 2024-06-02T23:08:31.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c102740 0x7fc21c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:31.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 0x7fc21c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:31.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21c107840 0x7fc21c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c112680 con 0x7fc21c1030f0 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c112500 con 0x7fc21c102740 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc220e3d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc21c112800 con 0x7fc21c107840 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.235+0000 7fc219d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 0x7fc21c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 0x7fc21c19ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53104/0 (socket says 172.21.15.52:53104) 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 -- 172.21.15.52:0/3040936782 learned_addr learned my addr 172.21.15.52:0/3040936782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc21a59c700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c102740 0x7fc21c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc21ad9d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21c107840 0x7fc21c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 -- 172.21.15.52:0/3040936782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21c107840 msgr2=0x7fc21c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:31.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21c107840 0x7fc21c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 -- 172.21.15.52:0/3040936782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c102740 msgr2=0x7fc21c19e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c102740 0x7fc21c19e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 -- 172.21.15.52:0/3040936782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc21c1990a0 con 0x7fc21c1030f0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc219d9b700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 0x7fc21c19ecc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc21000b530 tx=0x7fc21000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210015020 con 0x7fc21c1030f0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21c199390 con 0x7fc21c1030f0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc210011ce0 con 0x7fc21c1030f0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc210010dd0 con 0x7fc21c1030f0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.236+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21c1a5650 con 0x7fc21c1030f0 2024-06-02T23:08:31.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.237+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc21c10aa80 con 0x7fc21c1030f0 2024-06-02T23:08:31.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.239+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fc210018020 con 0x7fc21c1030f0 2024-06-02T23:08:31.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.239+0000 7fc20b7fe700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2040a2cb0 0x7fc2040a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:31.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.239+0000 7fc21a59c700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2040a2cb0 0x7fc2040a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:31.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.239+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc2100ced60 con 0x7fc21c1030f0 2024-06-02T23:08:31.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.240+0000 7fc21a59c700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2040a2cb0 0x7fc2040a5170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009990 tx=0x7fc20c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:31.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.243+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc21001f020 con 0x7fc21c1030f0 2024-06-02T23:08:31.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.448+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc21c10a460 con 0x7fc21c1030f0 2024-06-02T23:08:31.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.450+0000 7fc20b7fe700 1 -- 172.21.15.52:0/3040936782 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7fc21009bca0 con 0x7fc21c1030f0 2024-06-02T23:08:31.450 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:31.451 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2040a2cb0 msgr2=0x7fc2040a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2040a2cb0 0x7fc2040a5170 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc20c009990 tx=0x7fc20c005fb0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 msgr2=0x7fc21c19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 0x7fc21c19ecc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc21000b530 tx=0x7fc21000b840 comp rx=0 tx=0).stop 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 shutdown_connections 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2040a2cb0 0x7fc2040a5170 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc21c107840 0x7fc21c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc21c102740 0x7fc21c19e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 --2- 172.21.15.52:0/3040936782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc21c1030f0 0x7fc21c19ecc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 >> 172.21.15.52:0/3040936782 conn(0x7fc21c0795f0 msgr2=0x7fc21c108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 shutdown_connections 2024-06-02T23:08:31.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:31.452+0000 7fc220e3d700 1 -- 172.21.15.52:0/3040936782 wait complete. 2024-06-02T23:08:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:31 smithi174 ceph-mon[35994]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T23:08:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:31 smithi052 ceph-mon[32340]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T23:08:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:31 smithi123 ceph-mon[35977]: pgmap v330: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T23:08:32.175 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-06-02T23:08:32.175 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:08:32.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:32 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3040936782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:32.432 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:32.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:32 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3040936782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:32 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3040936782' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:33.582 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:33 smithi052 ceph-mon[32340]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T23:08:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:33 smithi174 ceph-mon[35994]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T23:08:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:33 smithi123 ceph-mon[35977]: pgmap v331: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 9 op/s 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 -- 172.21.15.52:0/2570008466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8102510 msgr2=0x7f56a8102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 --2- 172.21.15.52:0/2570008466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8102510 0x7f56a8102990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5698009990 tx=0x7f5698009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 -- 172.21.15.52:0/2570008466 shutdown_connections 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 --2- 172.21.15.52:0/2570008466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56a8102ed0 0x7f56a810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 --2- 172.21.15.52:0/2570008466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56a8108520 0x7f56a8108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 --2- 172.21.15.52:0/2570008466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8102510 0x7f56a8102990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 -- 172.21.15.52:0/2570008466 >> 172.21.15.52:0/2570008466 conn(0x7f56a8079640 msgr2=0x7f56a8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 -- 172.21.15.52:0/2570008466 shutdown_connections 2024-06-02T23:08:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.143+0000 7f56b0616700 1 -- 172.21.15.52:0/2570008466 wait complete. 2024-06-02T23:08:34.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 Processor -- start 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 -- start start 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56a8102510 0x7f56a819e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56a8102ed0 0x7f56a819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 0x7f56a81988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56a81126d0 con 0x7f56a8108520 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8112550 con 0x7f56a8102ed0 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.144+0000 7f56b0616700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56a8112850 con 0x7f56a8102510 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 0x7f56a81988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 0x7f56a81988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59328/0 (socket says 172.21.15.52:59328) 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 -- 172.21.15.52:0/1469382527 learned_addr learned my addr 172.21.15.52:0/1469382527 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56ae3b2700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56a8102510 0x7f56a819e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:34.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56adbb1700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56a8102ed0 0x7f56a819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 -- 172.21.15.52:0/1469382527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56a8102510 msgr2=0x7f56a819e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56a8102510 0x7f56a819e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 -- 172.21.15.52:0/1469382527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56a8102ed0 msgr2=0x7f56a819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56a8102ed0 0x7f56a819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 -- 172.21.15.52:0/1469382527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56a8199030 con 0x7f56a8108520 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56aebb3700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 0x7f56a81988d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c970 tx=0x7f56a000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a00078f0 con 0x7f56a8108520 2024-06-02T23:08:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f56a000f450 con 0x7f56a8108520 2024-06-02T23:08:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56a8199320 con 0x7f56a8108520 2024-06-02T23:08:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a0018640 con 0x7f56a8108520 2024-06-02T23:08:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.145+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56a81a55e0 con 0x7f56a8108520 2024-06-02T23:08:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.146+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56a8069180 con 0x7f56a8108520 2024-06-02T23:08:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.148+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f56a000f600 con 0x7f56a8108520 2024-06-02T23:08:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.148+0000 7f569f7fe700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56940a2d30 0x7f56940a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.149+0000 7f56ae3b2700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56940a2d30 0x7f56940a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.149+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f56a00cefc0 con 0x7f56a8108520 2024-06-02T23:08:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.149+0000 7f56ae3b2700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56940a2d30 0x7f56940a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56a8199a10 tx=0x7f56a400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:34.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.152+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f56a0014030 con 0x7f56a8108520 2024-06-02T23:08:34.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.360+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56a802dca0 con 0x7f56a8108520 2024-06-02T23:08:34.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.362+0000 7f569f7fe700 1 -- 172.21.15.52:0/1469382527 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7f56a009bf10 con 0x7f56a8108520 2024-06-02T23:08:34.362 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:08:34.362 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56940a2d30 msgr2=0x7f56940a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56940a2d30 0x7f56940a51f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f56a8199a10 tx=0x7f56a400d040 comp rx=0 tx=0).stop 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 msgr2=0x7f56a81988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 0x7f56a81988d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f56a000c970 tx=0x7f56a000cc80 comp rx=0 tx=0).stop 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 shutdown_connections 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56940a2d30 0x7f56940a51f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56a8108520 0x7f56a81988d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56a8102510 0x7f56a819e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 --2- 172.21.15.52:0/1469382527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f56a8102ed0 0x7f56a819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:34.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 >> 172.21.15.52:0/1469382527 conn(0x7f56a8079640 msgr2=0x7f56a81010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:34.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 shutdown_connections 2024-06-02T23:08:34.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:34.364+0000 7f56b0616700 1 -- 172.21.15.52:0/1469382527 wait complete. 2024-06-02T23:08:34.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:34 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1469382527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:34 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1469382527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:34 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1469382527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:08:35.057 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-06-02T23:08:35.057 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-06-02T23:08:35.058 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-06-02T23:08:35.058 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-06-02T23:08:35.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-06-02T23:08:35.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-06-02T23:08:35.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-06-02T23:08:35.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-06-02T23:08:35.061 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-06-02T23:08:35.061 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-06-02T23:08:35.061 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-06-02T23:08:35.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-06-02T23:08:35.385 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:35 smithi052 ceph-mon[32340]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-02T23:08:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:35 smithi174 ceph-mon[35994]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-02T23:08:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:35 smithi123 ceph-mon[35977]: pgmap v332: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-06-02T23:08:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:35.814 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:35.817 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:35.822 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:36.116 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:36.117 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:36.121 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:36.122 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:37.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:37 smithi052 ceph-mon[32340]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:37 smithi174 ceph-mon[35994]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:37 smithi123 ceph-mon[35977]: pgmap v333: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:39.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 -- 172.21.15.52:0/1912153542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0107400 msgr2=0x7fa0c01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:39.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1912153542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0107400 0x7fa0c01077e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000b0a0 tx=0x7fa0b000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 -- 172.21.15.52:0/1912153542 shutdown_connections 2024-06-02T23:08:39.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1912153542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0c0101d50 0x7fa0c010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1912153542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0107400 0x7fa0c01077e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1912153542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0c0101390 0x7fa0c0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 -- 172.21.15.52:0/1912153542 >> 172.21.15.52:0/1912153542 conn(0x7fa0c0078340 msgr2=0x7fa0c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:39.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 -- 172.21.15.52:0/1912153542 shutdown_connections 2024-06-02T23:08:39.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.185+0000 7fa0c5963700 1 -- 172.21.15.52:0/1912153542 wait complete. 2024-06-02T23:08:39.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 Processor -- start 2024-06-02T23:08:39.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 -- start start 2024-06-02T23:08:39.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0c0101390 0x7fa0c019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:39.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0101d50 0x7fa0c019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:39.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0c0107400 0x7fa0c0198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:39.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c0111520 con 0x7fa0c0101d50 2024-06-02T23:08:39.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01113a0 con 0x7fa0c0107400 2024-06-02T23:08:39.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.186+0000 7fa0c5963700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c01116a0 con 0x7fa0c0101390 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0101d50 0x7fa0c019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0101d50 0x7fa0c019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59344/0 (socket says 172.21.15.52:59344) 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 -- 172.21.15.52:0/1456151453 learned_addr learned my addr 172.21.15.52:0/1456151453 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0bf7fe700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0c0107400 0x7fa0c0198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0beffd700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0c0101390 0x7fa0c019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 -- 172.21.15.52:0/1456151453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0c0101390 msgr2=0x7fa0c019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0c0101390 0x7fa0c019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 -- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0c0107400 msgr2=0x7fa0c0198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0c0107400 0x7fa0c0198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 -- 172.21.15.52:0/1456151453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c0199080 con 0x7fa0c0101d50 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0be7fc700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0101d50 0x7fa0c019ec50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c9b0 tx=0x7fa0a800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8007b00 con 0x7fa0c0101d50 2024-06-02T23:08:39.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa0a8007ca0 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c0199370 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8018660 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.187+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c01a55e0 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.188+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa0c0105500 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.190+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa0a800f450 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.190+0000 7fa0c4961700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa0ac0a2cb0 0x7fa0ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.190+0000 7fa0beffd700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa0ac0a2cb0 0x7fa0ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.190+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa0a80cf5a0 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.191+0000 7fa0c4961700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] conn(0x7fa0ac0ac770 0x7fa0ac0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.191+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 --> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa0ac0af2e0 con 0x7fa0ac0ac770 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.191+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa0a80cf9f0 con 0x7fa0c0101d50 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.191+0000 7fa0beffd700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa0ac0a2cb0 0x7fa0ac0a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000b830 tx=0x7fa0b0006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:39.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.191+0000 7fa0bf7fe700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] conn(0x7fa0ac0ac770 0x7fa0ac0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:39.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.191+0000 7fa0bf7fe700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] conn(0x7fa0ac0ac770 0x7fa0ac0aec10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:39.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.192+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== osd.7 v2:172.21.15.123:6816/550996005 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa0ac0af2e0 con 0x7fa0ac0ac770 2024-06-02T23:08:39.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.209+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 --> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa0c0102ab0 con 0x7fa0ac0ac770 2024-06-02T23:08:39.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.209+0000 7fa0c4961700 1 -- 172.21.15.52:0/1456151453 <== osd.7 v2:172.21.15.123:6816/550996005 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa0c0102ab0 con 0x7fa0ac0ac770 2024-06-02T23:08:39.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] conn(0x7fa0ac0ac770 msgr2=0x7fa0ac0aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:39.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] conn(0x7fa0ac0ac770 0x7fa0ac0aec10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa0ac0a2cb0 msgr2=0x7fa0ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:39.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa0ac0a2cb0 0x7fa0ac0a5170 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000b830 tx=0x7fa0b0006210 comp rx=0 tx=0).stop 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0101d50 msgr2=0x7fa0c019ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 --2- 172.21.15.52:0/1456151453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa0c0101d50 0x7fa0c019ec50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800c9b0 tx=0x7fa0a800ccc0 comp rx=0 tx=0).stop 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0bf7fe700 1 -- 172.21.15.52:0/1456151453 reap_dead start 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 shutdown_connections 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 >> 172.21.15.52:0/1456151453 conn(0x7fa0c0078340 msgr2=0x7fa0c0100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 shutdown_connections 2024-06-02T23:08:39.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:39.210+0000 7fa0c5963700 1 -- 172.21.15.52:0/1456151453 wait complete. 2024-06-02T23:08:39.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:39 smithi052 ceph-mon[32340]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:39 smithi174 ceph-mon[35994]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:39 smithi123 ceph-mon[35977]: pgmap v334: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:40.181 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:08:41.593 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:41 smithi052 ceph-mon[32340]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:41 smithi123 ceph-mon[35977]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:41 smithi174 ceph-mon[35994]: pgmap v335: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:43 smithi052 ceph-mon[32340]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:08:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:43 smithi123 ceph-mon[35977]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:08:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:43 smithi174 ceph-mon[35994]: pgmap v336: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 -- 172.21.15.52:0/3222771189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 msgr2=0x7f6d8806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 --2- 172.21.15.52:0/3222771189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8806a5f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6d84009990 tx=0x7f6d84009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 -- 172.21.15.52:0/3222771189 shutdown_connections 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 --2- 172.21.15.52:0/3222771189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d8806ab30 0x7f6d8810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 --2- 172.21.15.52:0/3222771189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d88108500 0x7f6d881088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 --2- 172.21.15.52:0/3222771189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8806a5f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 -- 172.21.15.52:0/3222771189 >> 172.21.15.52:0/3222771189 conn(0x7f6d88078340 msgr2=0x7f6d88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:44.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 -- 172.21.15.52:0/3222771189 shutdown_connections 2024-06-02T23:08:44.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.908+0000 7f6d90031700 1 -- 172.21.15.52:0/3222771189 wait complete. 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 Processor -- start 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 -- start start 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d8806ab30 0x7f6d8819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d88108500 0x7f6d88198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d881126e0 con 0x7f6d8806a170 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88112560 con 0x7f6d8806ab30 2024-06-02T23:08:44.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.909+0000 7f6d90031700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d88112860 con 0x7f6d88108500 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8d5cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d8806ab30 0x7f6d8819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40692/0 (socket says 172.21.15.52:40692) 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 -- 172.21.15.52:0/2101223552 learned_addr learned my addr 172.21.15.52:0/2101223552 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8e5ce700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d88108500 0x7f6d88198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 -- 172.21.15.52:0/2101223552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d88108500 msgr2=0x7f6d88198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d88108500 0x7f6d88198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 -- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d8806ab30 msgr2=0x7f6d8819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d8806ab30 0x7f6d8819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 -- 172.21.15.52:0/2101223552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d881991a0 con 0x7f6d8806a170 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d8ddcd700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8819e880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c016070 con 0x7f6d8806a170 2024-06-02T23:08:44.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d7c009df0 con 0x7f6d8806a170 2024-06-02T23:08:44.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d88199490 con 0x7f6d8806a170 2024-06-02T23:08:44.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7c00f3c0 con 0x7f6d8806a170 2024-06-02T23:08:44.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.910+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d881a5750 con 0x7f6d8806a170 2024-06-02T23:08:44.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.911+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d88062c20 con 0x7f6d8806a170 2024-06-02T23:08:44.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.913+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f6d7c019070 con 0x7f6d8806a170 2024-06-02T23:08:44.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.913+0000 7f6d7affd700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d740a2cb0 0x7f6d740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:44.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d8d5cc700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d740a2cb0 0x7f6d740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:44.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6d7c0ced50 con 0x7f6d8806a170 2024-06-02T23:08:44.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d7affd700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] conn(0x7f6d740ac770 0x7f6d740aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:44.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 --> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d740af2e0 con 0x7f6d740ac770 2024-06-02T23:08:44.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6d7c0d2020 con 0x7f6d8806a170 2024-06-02T23:08:44.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d8d5cc700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d740a2cb0 0x7f6d740a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d88078a20 tx=0x7f6d84005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:44.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.914+0000 7f6d8e5ce700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] conn(0x7f6d740ac770 0x7f6d740aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:44.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.915+0000 7f6d8e5ce700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] conn(0x7f6d740ac770 0x7f6d740aec10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:44.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.916+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== osd.10 v2:172.21.15.123:6824/4143797452 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d740af2e0 con 0x7f6d740ac770 2024-06-02T23:08:44.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.934+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 --> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d88063020 con 0x7f6d740ac770 2024-06-02T23:08:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.934+0000 7f6d7affd700 1 -- 172.21.15.52:0/2101223552 <== osd.10 v2:172.21.15.123:6824/4143797452 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6d88063020 con 0x7f6d740ac770 2024-06-02T23:08:44.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] conn(0x7f6d740ac770 msgr2=0x7f6d740aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] conn(0x7f6d740ac770 0x7f6d740aec10 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d740a2cb0 msgr2=0x7f6d740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d740a2cb0 0x7f6d740a5170 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6d88078a20 tx=0x7f6d84005fd0 comp rx=0 tx=0).stop 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 msgr2=0x7f6d8819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 --2- 172.21.15.52:0/2101223552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d8806a170 0x7f6d8819e880 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c00d950 tx=0x7f6d7c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d8e5ce700 1 -- 172.21.15.52:0/2101223552 reap_dead start 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 shutdown_connections 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 >> 172.21.15.52:0/2101223552 conn(0x7f6d88078340 msgr2=0x7f6d88100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 shutdown_connections 2024-06-02T23:08:44.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:44.935+0000 7f6d90031700 1 -- 172.21.15.52:0/2101223552 wait complete. 2024-06-02T23:08:45.550 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:45 smithi052 ceph-mon[32340]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:45 smithi123 ceph-mon[35977]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:45 smithi174 ceph-mon[35994]: pgmap v337: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:46.401 INFO:teuthology.orchestra.run.smithi052.stdout:219043332147 2024-06-02T23:08:46.402 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-06-02T23:08:47.740 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:47 smithi052 ceph-mon[32340]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-02T23:08:47.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:47 smithi123 ceph-mon[35977]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-02T23:08:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:47 smithi174 ceph-mon[35994]: pgmap v338: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 -- 172.21.15.52:0/1363264172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741024e0 msgr2=0x7fb774102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/1363264172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741024e0 0x7fb774102940 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb768009990 tx=0x7fb768009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 -- 172.21.15.52:0/1363264172 shutdown_connections 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/1363264172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb774102e80 0x7fb77410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/1363264172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7741084f0 0x7fb7741088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/1363264172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741024e0 0x7fb774102940 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 -- 172.21.15.52:0/1363264172 >> 172.21.15.52:0/1363264172 conn(0x7fb774079530 msgr2=0x7fb774079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 -- 172.21.15.52:0/1363264172 shutdown_connections 2024-06-02T23:08:48.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.947+0000 7fb77b2a6700 1 -- 172.21.15.52:0/1363264172 wait complete. 2024-06-02T23:08:48.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 Processor -- start 2024-06-02T23:08:48.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 -- start start 2024-06-02T23:08:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7741024e0 0x7fb77419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:48.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb774102e80 0x7fb77419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741084f0 0x7fb7741988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb774112600 con 0x7fb7741084f0 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb774112480 con 0x7fb774102e80 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb77b2a6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb774112780 con 0x7fb7741024e0 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb779843700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741084f0 0x7fb7741988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb779843700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741084f0 0x7fb7741988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40700/0 (socket says 172.21.15.52:40700) 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb779843700 1 -- 172.21.15.52:0/3027510759 learned_addr learned my addr 172.21.15.52:0/3027510759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb778841700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb774102e80 0x7fb77419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb779042700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7741024e0 0x7fb77419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.948+0000 7fb779843700 1 -- 172.21.15.52:0/3027510759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7741024e0 msgr2=0x7fb77419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb779843700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7741024e0 0x7fb77419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb779843700 1 -- 172.21.15.52:0/3027510759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb774102e80 msgr2=0x7fb77419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb779843700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb774102e80 0x7fb77419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb779843700 1 -- 172.21.15.52:0/3027510759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb774199040 con 0x7fb7741084f0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb779843700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741084f0 0x7fb7741988e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00d980 tx=0x7fb76c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c016070 con 0x7fb7741084f0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb76c004830 con 0x7fb7741084f0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb774199330 con 0x7fb7741084f0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c0054e0 con 0x7fb7741084f0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.949+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7741a55f0 con 0x7fb7741084f0 2024-06-02T23:08:48.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.950+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb774062c20 con 0x7fb7741084f0 2024-06-02T23:08:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.951+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb76c0049e0 con 0x7fb7741084f0 2024-06-02T23:08:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb7727fc700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb75c0a2d30 0x7fb75c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb779042700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb75c0a2d30 0x7fb75c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb76c0cf510 con 0x7fb7741084f0 2024-06-02T23:08:48.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb7727fc700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] conn(0x7fb75c0ac7f0 0x7fb75c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 --> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb76c015ec0 con 0x7fb75c0ac7f0 2024-06-02T23:08:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb76c0cf8c0 con 0x7fb7741084f0 2024-06-02T23:08:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb778841700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] conn(0x7fb75c0ac7f0 0x7fb75c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.952+0000 7fb779042700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb75c0a2d30 0x7fb75c0a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb774199a20 tx=0x7fb7640079e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.953+0000 7fb778841700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] conn(0x7fb75c0ac7f0 0x7fb75c0aec90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:48.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.953+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== osd.6 v2:172.21.15.52:6818/2952400783 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb76c015ec0 con 0x7fb75c0ac7f0 2024-06-02T23:08:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 --> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb7740630a0 con 0x7fb75c0ac7f0 2024-06-02T23:08:48.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb7727fc700 1 -- 172.21.15.52:0/3027510759 <== osd.6 v2:172.21.15.52:6818/2952400783 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb7740630a0 con 0x7fb75c0ac7f0 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] conn(0x7fb75c0ac7f0 msgr2=0x7fb75c0aec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] conn(0x7fb75c0ac7f0 0x7fb75c0aec90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb75c0a2d30 msgr2=0x7fb75c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb75c0a2d30 0x7fb75c0a51f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb774199a20 tx=0x7fb7640079e0 comp rx=0 tx=0).stop 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741084f0 msgr2=0x7fb7741988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 --2- 172.21.15.52:0/3027510759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7741084f0 0x7fb7741988e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00d980 tx=0x7fb76c00dc90 comp rx=0 tx=0).stop 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb779843700 1 -- 172.21.15.52:0/3027510759 reap_dead start 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.970+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 shutdown_connections 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.971+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 >> 172.21.15.52:0/3027510759 conn(0x7fb774079530 msgr2=0x7fb774100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:48.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.971+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 shutdown_connections 2024-06-02T23:08:48.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:48.971+0000 7fb77b2a6700 1 -- 172.21.15.52:0/3027510759 wait complete. 2024-06-02T23:08:49.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 -- 172.21.15.52:0/117097250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406a170 msgr2=0x7fb2c406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:49.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 --2- 172.21.15.52:0/117097250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406a170 0x7fb2c406a5f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb2b40098f0 tx=0x7fb2b4009c00 comp rx=0 tx=0).stop 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 -- 172.21.15.52:0/117097250 shutdown_connections 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 --2- 172.21.15.52:0/117097250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c406ab30 0x7fb2c410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 --2- 172.21.15.52:0/117097250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c4108500 0x7fb2c41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 --2- 172.21.15.52:0/117097250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406a170 0x7fb2c406a5f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 -- 172.21.15.52:0/117097250 >> 172.21.15.52:0/117097250 conn(0x7fb2c4078340 msgr2=0x7fb2c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 -- 172.21.15.52:0/117097250 shutdown_connections 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.140+0000 7fb2cc002700 1 -- 172.21.15.52:0/117097250 wait complete. 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 Processor -- start 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 -- start start 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c406a170 0x7fb2c419e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406ab30 0x7fb2c419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:49.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c4108500 0x7fb2c41989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112610 con 0x7fb2c406a170 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112490 con 0x7fb2c406ab30 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2cc002700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c4112790 con 0x7fb2c4108500 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c406a170 0x7fb2c419e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c406a170 0x7fb2c419e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40720/0 (socket says 172.21.15.52:40720) 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 -- 172.21.15.52:0/3347491151 learned_addr learned my addr 172.21.15.52:0/3347491151 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c959d700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406ab30 0x7fb2c419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 -- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c4108500 msgr2=0x7fb2c41989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:49.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2ca59f700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c4108500 0x7fb2c41989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2c4108500 0x7fb2c41989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 -- 172.21.15.52:0/3347491151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406ab30 msgr2=0x7fb2c419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2c406ab30 0x7fb2c419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.141+0000 7fb2c9d9e700 1 -- 172.21.15.52:0/3347491151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c4199110 con 0x7fb2c406a170 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.142+0000 7fb2c9d9e700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c406a170 0x7fb2c419e7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e9c0 tx=0x7fb2c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.142+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c000cbd0 con 0x7fb2c406a170 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.142+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb2c000cd70 con 0x7fb2c406a170 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.142+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c4199400 con 0x7fb2c406a170 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.142+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c00055e0 con 0x7fb2c406a170 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.142+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c41a56c0 con 0x7fb2c406a170 2024-06-02T23:08:49.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.143+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb2c4062c20 con 0x7fb2c406a170 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.144+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fb2c0004750 con 0x7fb2c406a170 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.145+0000 7fb2baffd700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2b00a2cb0 0x7fb2b00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.145+0000 7fb2c959d700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2b00a2cb0 0x7fb2b00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.145+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb2c00cfab0 con 0x7fb2c406a170 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.145+0000 7fb2baffd700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] conn(0x7fb2b00ac770 0x7fb2b00aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.145+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 --> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2b00af2e0 con 0x7fb2b00ac770 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.145+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fb2c00cfe60 con 0x7fb2c406a170 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.146+0000 7fb2c959d700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2b00a2cb0 0x7fb2b00a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.146+0000 7fb2ca59f700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] conn(0x7fb2b00ac770 0x7fb2b00aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.146+0000 7fb2ca59f700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] conn(0x7fb2b00ac770 0x7fb2b00aec10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.147+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== osd.11 v2:172.21.15.174:6824/190382346 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2b00af2e0 con 0x7fb2b00ac770 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.164+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 --> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb2c40630a0 con 0x7fb2b00ac770 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2baffd700 1 -- 172.21.15.52:0/3347491151 <== osd.11 v2:172.21.15.174:6824/190382346 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb2c40630a0 con 0x7fb2b00ac770 2024-06-02T23:08:49.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] conn(0x7fb2b00ac770 msgr2=0x7fb2b00aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] conn(0x7fb2b00ac770 0x7fb2b00aec10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2b00a2cb0 msgr2=0x7fb2b00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2b00a2cb0 0x7fb2b00a5170 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b4019040 comp rx=0 tx=0).stop 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c406a170 msgr2=0x7fb2c419e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 --2- 172.21.15.52:0/3347491151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2c406a170 0x7fb2c419e7f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2c000e9c0 tx=0x7fb2c000ecd0 comp rx=0 tx=0).stop 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2ca59f700 1 -- 172.21.15.52:0/3347491151 reap_dead start 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 shutdown_connections 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.165+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 >> 172.21.15.52:0/3347491151 conn(0x7fb2c4078340 msgr2=0x7fb2c4100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.166+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 shutdown_connections 2024-06-02T23:08:49.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:49.166+0000 7fb2cc002700 1 -- 172.21.15.52:0/3347491151 wait complete. 2024-06-02T23:08:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:49 smithi123 ceph-mon[35977]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:49 smithi174 ceph-mon[35994]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:49 smithi052 ceph-mon[32340]: pgmap v339: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:08:51.599 INFO:teuthology.orchestra.run.smithi052.stdout:197568495673 2024-06-02T23:08:51.599 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-06-02T23:08:51.691 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:51 smithi052 ceph-mon[32340]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:51.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:51 smithi123 ceph-mon[35977]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:51 smithi174 ceph-mon[35994]: pgmap v340: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:08:52.095 INFO:teuthology.orchestra.run.smithi052.stdout:343597383712 2024-06-02T23:08:52.095 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-06-02T23:08:52.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.322+0000 7f195888c700 1 -- 172.21.15.52:0/843010919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101d30 msgr2=0x7f195410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:52.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.322+0000 7f195888c700 1 --2- 172.21.15.52:0/843010919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101d30 0x7f195410e210 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f194c00b0a0 tx=0x7f194c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 -- 172.21.15.52:0/843010919 shutdown_connections 2024-06-02T23:08:52.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 --2- 172.21.15.52:0/843010919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101d30 0x7f195410e210 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 --2- 172.21.15.52:0/843010919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19541073d0 0x7f19541077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 --2- 172.21.15.52:0/843010919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954101390 0x7f19541017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 -- 172.21.15.52:0/843010919 >> 172.21.15.52:0/843010919 conn(0x7f19540782c0 msgr2=0x7f19540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:52.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 -- 172.21.15.52:0/843010919 shutdown_connections 2024-06-02T23:08:52.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 -- 172.21.15.52:0/843010919 wait complete. 2024-06-02T23:08:52.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 Processor -- start 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.323+0000 7f195888c700 1 -- start start 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195888c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101390 0x7f19541033b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195888c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954101d30 0x7f19541038f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195888c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19541073d0 0x7f1954103e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195888c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1954111500 con 0x7f1954101390 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195888c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1954111380 con 0x7f1954101d30 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195888c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1954111680 con 0x7f19541073d0 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101390 0x7f19541033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101390 0x7f19541033b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40732/0 (socket says 172.21.15.52:40732) 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 -- 172.21.15.52:0/3240279757 learned_addr learned my addr 172.21.15.52:0/3240279757 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f1951d9b700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954101d30 0x7f19541038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:52.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f1952d9d700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19541073d0 0x7f1954103e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 -- 172.21.15.52:0/3240279757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19541073d0 msgr2=0x7f1954103e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19541073d0 0x7f1954103e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 -- 172.21.15.52:0/3240279757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954101d30 msgr2=0x7f19541038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1954101d30 0x7f19541038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 -- 172.21.15.52:0/3240279757 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19541a7040 con 0x7f1954101390 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.324+0000 7f195259c700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101390 0x7f19541033b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f194400d980 tx=0x7f194400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.325+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944017070 con 0x7f1954101390 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.325+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1944005020 con 0x7f1954101390 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.325+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19541a7330 con 0x7f1954101390 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.325+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944009b50 con 0x7f1954101390 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.325+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19541a7870 con 0x7f1954101390 2024-06-02T23:08:52.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.325+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f195406a440 con 0x7f1954101390 2024-06-02T23:08:52.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.327+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1944009cf0 con 0x7f1954101390 2024-06-02T23:08:52.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.327+0000 7f19437fe700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f193c0a2d30 0x7f193c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:52.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f1951d9b700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f193c0a2d30 0x7f193c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19440cf580 con 0x7f1954101390 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f19437fe700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] conn(0x7f193c0ac7f0 0x7f193c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 --> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f194401de70 con 0x7f193c0ac7f0 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f19440cf930 con 0x7f1954101390 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f1952d9d700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] conn(0x7f193c0ac7f0 0x7f193c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f1951d9b700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f193c0a2d30 0x7f193c0a51f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f195406b250 tx=0x7f194800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.328+0000 7f1952d9d700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] conn(0x7f193c0ac7f0 0x7f193c0aec90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:52.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.329+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== osd.0 v2:172.21.15.52:6802/1323575430 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f194401de70 con 0x7f193c0ac7f0 2024-06-02T23:08:52.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 --> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1954102a80 con 0x7f193c0ac7f0 2024-06-02T23:08:52.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f19437fe700 1 -- 172.21.15.52:0/3240279757 <== osd.0 v2:172.21.15.52:6802/1323575430 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1954102a80 con 0x7f193c0ac7f0 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] conn(0x7f193c0ac7f0 msgr2=0x7f193c0aec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] conn(0x7f193c0ac7f0 0x7f193c0aec90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f193c0a2d30 msgr2=0x7f193c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f193c0a2d30 0x7f193c0a51f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f195406b250 tx=0x7f194800b410 comp rx=0 tx=0).stop 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101390 msgr2=0x7f19541033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 --2- 172.21.15.52:0/3240279757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1954101390 0x7f19541033b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f194400d980 tx=0x7f194400dc90 comp rx=0 tx=0).stop 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f1952d9d700 1 -- 172.21.15.52:0/3240279757 reap_dead start 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 shutdown_connections 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.346+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 >> 172.21.15.52:0/3240279757 conn(0x7f19540782c0 msgr2=0x7f1954105650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:52.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.347+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 shutdown_connections 2024-06-02T23:08:52.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.347+0000 7f195888c700 1 -- 172.21.15.52:0/3240279757 wait complete. 2024-06-02T23:08:53.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 -- 172.21.15.52:0/3977133980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118107400 msgr2=0x7f41181077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 --2- 172.21.15.52:0/3977133980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118107400 0x7f41181077e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4108009990 tx=0x7f4108009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 -- 172.21.15.52:0/3977133980 shutdown_connections 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 --2- 172.21.15.52:0/3977133980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4118101d50 0x7f411810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 --2- 172.21.15.52:0/3977133980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118107400 0x7f41181077e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 --2- 172.21.15.52:0/3977133980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4118101390 0x7f4118101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 -- 172.21.15.52:0/3977133980 >> 172.21.15.52:0/3977133980 conn(0x7f4118078340 msgr2=0x7f4118078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 -- 172.21.15.52:0/3977133980 shutdown_connections 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.444+0000 7f41200e1700 1 -- 172.21.15.52:0/3977133980 wait complete. 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 Processor -- start 2024-06-02T23:08:53.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 -- start start 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4118101390 0x7f4118197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118101d50 0x7f4118197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4118107400 0x7f41181982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4118111550 con 0x7f4118101d50 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41181113d0 con 0x7f4118107400 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f41200e1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41181116d0 con 0x7f4118101390 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118101d50 0x7f4118197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118101d50 0x7f4118197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40792/0 (socket says 172.21.15.52:40792) 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 -- 172.21.15.52:0/2322455428 learned_addr learned my addr 172.21.15.52:0/2322455428 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411de7d700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4118101390 0x7f4118197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411e67e700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4118107400 0x7f41181982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 -- 172.21.15.52:0/2322455428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4118101390 msgr2=0x7f4118197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4118101390 0x7f4118197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 -- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4118107400 msgr2=0x7f41181982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4118107400 0x7f41181982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.445+0000 7f411d67c700 1 -- 172.21.15.52:0/2322455428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41181a70d0 con 0x7f4118101d50 2024-06-02T23:08:53.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.446+0000 7f411d67c700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118101d50 0x7f4118197da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c920 tx=0x7f410c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.446+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c007b00 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.446+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f410c004ce0 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.446+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41181a73c0 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.446+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f410c005550 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.446+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41181a7950 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.447+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4118062b60 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.448+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f410c004750 con 0x7f4118101d50 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f4116ffd700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41000a2d30 0x7f41000a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f411de7d700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41000a2d30 0x7f41000a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f410c0cf1d0 con 0x7f4118101d50 2024-06-02T23:08:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f4116ffd700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] conn(0x7f41000ac7f0 0x7f41000aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 --> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f410c012ec0 con 0x7f41000ac7f0 2024-06-02T23:08:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f410c0cf620 con 0x7f4118101d50 2024-06-02T23:08:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f411de7d700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41000a2d30 0x7f41000a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41080098e0 tx=0x7f4108005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:53.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.449+0000 7f411e67e700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] conn(0x7f41000ac7f0 0x7f41000aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.450+0000 7f411e67e700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] conn(0x7f41000ac7f0 0x7f41000aec90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:53.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.451+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== osd.4 v2:172.21.15.123:6808/3937434833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f410c012ec0 con 0x7f41000ac7f0 2024-06-02T23:08:53.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.467+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 --> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4118062f60 con 0x7f41000ac7f0 2024-06-02T23:08:53.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.467+0000 7f4116ffd700 1 -- 172.21.15.52:0/2322455428 <== osd.4 v2:172.21.15.123:6808/3937434833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4118062f60 con 0x7f41000ac7f0 2024-06-02T23:08:53.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] conn(0x7f41000ac7f0 msgr2=0x7f41000aec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] conn(0x7f41000ac7f0 0x7f41000aec90 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41000a2d30 msgr2=0x7f41000a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41000a2d30 0x7f41000a51f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f41080098e0 tx=0x7f4108005960 comp rx=0 tx=0).stop 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118101d50 msgr2=0x7f4118197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 --2- 172.21.15.52:0/2322455428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4118101d50 0x7f4118197da0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f410c00c920 tx=0x7f410c00cc30 comp rx=0 tx=0).stop 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f411e67e700 1 -- 172.21.15.52:0/2322455428 reap_dead start 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 shutdown_connections 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 >> 172.21.15.52:0/2322455428 conn(0x7f4118078340 msgr2=0x7f4118100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 shutdown_connections 2024-06-02T23:08:53.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.468+0000 7f41200e1700 1 -- 172.21.15.52:0/2322455428 wait complete. 2024-06-02T23:08:53.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 -- 172.21.15.52:0/2252542039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4101440 msgr2=0x7f24a410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 --2- 172.21.15.52:0/2252542039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4101440 0x7f24a410f330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f249c00b0a0 tx=0x7f249c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 -- 172.21.15.52:0/2252542039 shutdown_connections 2024-06-02T23:08:53.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 --2- 172.21.15.52:0/2252542039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4101440 0x7f24a410f330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 --2- 172.21.15.52:0/2252542039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4108510 0x7f24a41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 --2- 172.21.15.52:0/2252542039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24a4100a80 0x7f24a4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 -- 172.21.15.52:0/2252542039 >> 172.21.15.52:0/2252542039 conn(0x7f24a4077a00 msgr2=0x7f24a40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 -- 172.21.15.52:0/2252542039 shutdown_connections 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.356+0000 7f24a8dca700 1 -- 172.21.15.52:0/2252542039 wait complete. 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.357+0000 7f24a8dca700 1 Processor -- start 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.357+0000 7f24a8dca700 1 -- start start 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a8dca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24a4100a80 0x7f24a419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a8dca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4101440 0x7f24a419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a8dca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4108510 0x7f24a4198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a8dca700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4112640 con 0x7f24a4108510 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a8dca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41124c0 con 0x7f24a4101440 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a8dca700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24a41127c0 con 0x7f24a4100a80 2024-06-02T23:08:53.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4108510 0x7f24a4198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4108510 0x7f24a4198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40786/0 (socket says 172.21.15.52:40786) 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 -- 172.21.15.52:0/3442030112 learned_addr learned my addr 172.21.15.52:0/3442030112 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f2499d9b700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4101440 0x7f24a419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a259c700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24a4100a80 0x7f24a419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 -- 172.21.15.52:0/3442030112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24a4100a80 msgr2=0x7f24a419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24a4100a80 0x7f24a419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 -- 172.21.15.52:0/3442030112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4101440 msgr2=0x7f24a419ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24a4101440 0x7f24a419ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 -- 172.21.15.52:0/3442030112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a41990a0 con 0x7f24a4108510 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f24a2d9d700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4108510 0x7f24a4198940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f249c0062a0 tx=0x7f249c004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.358+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c021070 con 0x7f24a4108510 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.359+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a4199330 con 0x7f24a4108510 2024-06-02T23:08:53.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.359+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f249c007e30 con 0x7f24a4108510 2024-06-02T23:08:53.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.359+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f249c004780 con 0x7f24a4108510 2024-06-02T23:08:53.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.359+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a41a5600 con 0x7f24a4108510 2024-06-02T23:08:53.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.359+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f24a4062c20 con 0x7f24a4108510 2024-06-02T23:08:53.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.361+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f249c007980 con 0x7f24a4108510 2024-06-02T23:08:53.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f249bfff700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24900a2cb0 0x7f24900a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f24a259c700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24900a2cb0 0x7f24900a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f249c00f020 con 0x7f24a4108510 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f249bfff700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] conn(0x7f24900ac770 0x7f24900aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 --> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f249c026e50 con 0x7f24900ac770 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f249c0d0ec0 con 0x7f24a4108510 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f2499d9b700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] conn(0x7f24900ac770 0x7f24900aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.362+0000 7f24a259c700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24900a2cb0 0x7f24900a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a4107610 tx=0x7f2494009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.363+0000 7f2499d9b700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] conn(0x7f24900ac770 0x7f24900aec10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.363+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== osd.9 v2:172.21.15.52:6826/3771079645 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f249c026e50 con 0x7f24900ac770 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.379+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 --> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f24a40630a0 con 0x7f24900ac770 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f249bfff700 1 -- 172.21.15.52:0/3442030112 <== osd.9 v2:172.21.15.52:6826/3771079645 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f24a40630a0 con 0x7f24900ac770 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] conn(0x7f24900ac770 msgr2=0x7f24900aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] conn(0x7f24900ac770 0x7f24900aec10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24900a2cb0 msgr2=0x7f24900a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24900a2cb0 0x7f24900a5170 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f24a4107610 tx=0x7f2494009260 comp rx=0 tx=0).stop 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4108510 msgr2=0x7f24a4198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 --2- 172.21.15.52:0/3442030112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24a4108510 0x7f24a4198940 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f249c0062a0 tx=0x7f249c004c80 comp rx=0 tx=0).stop 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a2d9d700 1 -- 172.21.15.52:0/3442030112 reap_dead start 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 shutdown_connections 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.380+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 >> 172.21.15.52:0/3442030112 conn(0x7f24a4077a00 msgr2=0x7f24a40ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.381+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 shutdown_connections 2024-06-02T23:08:53.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:53.381+0000 7f24a8dca700 1 -- 172.21.15.52:0/3442030112 wait complete. 2024-06-02T23:08:53.531 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:53 smithi052 ceph-mon[32340]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-02T23:08:53.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:53 smithi123 ceph-mon[35977]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-02T23:08:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:53 smithi174 ceph-mon[35994]: pgmap v341: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 -- 172.21.15.52:0/1166726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab34101390 msgr2=0x7fab341017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 --2- 172.21.15.52:0/1166726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab34101390 0x7fab341017f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fab28009990 tx=0x7fab28009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 -- 172.21.15.52:0/1166726373 shutdown_connections 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 --2- 172.21.15.52:0/1166726373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab34101d30 0x7fab3410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 --2- 172.21.15.52:0/1166726373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab341073d0 0x7fab341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 --2- 172.21.15.52:0/1166726373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab34101390 0x7fab341017f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 -- 172.21.15.52:0/1166726373 >> 172.21.15.52:0/1166726373 conn(0x7fab340782c0 msgr2=0x7fab340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 -- 172.21.15.52:0/1166726373 shutdown_connections 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.567+0000 7fab398c2700 1 -- 172.21.15.52:0/1166726373 wait complete. 2024-06-02T23:08:54.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 Processor -- start 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 -- start start 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab34101390 0x7fab3419e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34101d30 0x7fab3419ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab341073d0 0x7fab34198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab341114e0 con 0x7fab341073d0 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111360 con 0x7fab34101d30 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab398c2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111660 con 0x7fab34101390 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab341073d0 0x7fab34198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab341073d0 0x7fab34198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40754/0 (socket says 172.21.15.52:40754) 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 -- 172.21.15.52:0/1015538312 learned_addr learned my addr 172.21.15.52:0/1015538312 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab32ffd700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab34101390 0x7fab3419e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab327fc700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34101d30 0x7fab3419ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 -- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab34101390 msgr2=0x7fab3419e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:54.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab34101390 0x7fab3419e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 -- 172.21.15.52:0/1015538312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34101d30 msgr2=0x7fab3419ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.568+0000 7fab337fe700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab34101d30 0x7fab3419ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab337fe700 1 -- 172.21.15.52:0/1015538312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab34199090 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab32ffd700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab34101390 0x7fab3419e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab337fe700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab341073d0 0x7fab34198930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00d980 tx=0x7fab2c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c017070 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab2c005020 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab34199380 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c0099b0 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.569+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a55f0 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.570+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab341054d0 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.571+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fab2c0051d0 con 0x7fab341073d0 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab388c0700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab1c0a2cb0 0x7fab1c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:54.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab32ffd700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab1c0a2cb0 0x7fab1c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fab2c0cf580 con 0x7fab341073d0 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab388c0700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] conn(0x7fab1c0ac770 0x7fab1c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 --> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fab1c0af2e0 con 0x7fab1c0ac770 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fab2c0cf930 con 0x7fab341073d0 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab32ffd700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab1c0a2cb0 0x7fab1c0a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fab34106f50 tx=0x7fab24009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.572+0000 7fab327fc700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] conn(0x7fab1c0ac770 0x7fab1c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.573+0000 7fab327fc700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] conn(0x7fab1c0ac770 0x7fab1c0aec10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.574+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== osd.2 v2:172.21.15.174:6800/2598349360 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fab1c0af2e0 con 0x7fab1c0ac770 2024-06-02T23:08:54.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.590+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 --> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab34102a80 con 0x7fab1c0ac770 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab388c0700 1 -- 172.21.15.52:0/1015538312 <== osd.2 v2:172.21.15.174:6800/2598349360 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fab34102a80 con 0x7fab1c0ac770 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] conn(0x7fab1c0ac770 msgr2=0x7fab1c0aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] conn(0x7fab1c0ac770 0x7fab1c0aec10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab1c0a2cb0 msgr2=0x7fab1c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab1c0a2cb0 0x7fab1c0a5170 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fab34106f50 tx=0x7fab24009040 comp rx=0 tx=0).stop 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab341073d0 msgr2=0x7fab34198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 --2- 172.21.15.52:0/1015538312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab341073d0 0x7fab34198930 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fab2c00d980 tx=0x7fab2c00dc90 comp rx=0 tx=0).stop 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab337fe700 1 -- 172.21.15.52:0/1015538312 reap_dead start 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 shutdown_connections 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.591+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 >> 172.21.15.52:0/1015538312 conn(0x7fab340782c0 msgr2=0x7fab341007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.592+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 shutdown_connections 2024-06-02T23:08:54.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:52.592+0000 7fab398c2700 1 -- 172.21.15.52:0/1015538312 wait complete. 2024-06-02T23:08:54.995 INFO:teuthology.orchestra.run.smithi052.stdout:77309411407 2024-06-02T23:08:54.996 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-06-02T23:08:55.623 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:55 smithi052 ceph-mon[32340]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-02T23:08:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:55 smithi123 ceph-mon[35977]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-02T23:08:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:55 smithi174 ceph-mon[35994]: pgmap v342: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 11 op/s 2024-06-02T23:08:56.983 INFO:teuthology.orchestra.run.smithi052.stdout:309237645348 2024-06-02T23:08:56.983 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-06-02T23:08:57.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 -- 172.21.15.52:0/470112715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341024e0 msgr2=0x7fa634102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 --2- 172.21.15.52:0/470112715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341024e0 0x7fa634102940 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fa628009990 tx=0x7fa628009ca0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 -- 172.21.15.52:0/470112715 shutdown_connections 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 --2- 172.21.15.52:0/470112715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa634102e80 0x7fa63410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 --2- 172.21.15.52:0/470112715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6341084f0 0x7fa6341088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 --2- 172.21.15.52:0/470112715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341024e0 0x7fa634102940 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 -- 172.21.15.52:0/470112715 >> 172.21.15.52:0/470112715 conn(0x7fa634079530 msgr2=0x7fa634079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 -- 172.21.15.52:0/470112715 shutdown_connections 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.646+0000 7fa638f25700 1 -- 172.21.15.52:0/470112715 wait complete. 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 Processor -- start 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 -- start start 2024-06-02T23:08:57.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6341024e0 0x7fa634074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634102e80 0x7fa634074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341084f0 0x7fa634075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa634112600 con 0x7fa6341084f0 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa634112480 con 0x7fa6341024e0 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa638f25700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa634112780 con 0x7fa634102e80 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.647+0000 7fa632d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341084f0 0x7fa634075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341084f0 0x7fa634075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55278/0 (socket says 172.21.15.52:55278) 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 -- 172.21.15.52:0/4143577453 learned_addr learned my addr 172.21.15.52:0/4143577453 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa631d9b700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634102e80 0x7fa634074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa63259c700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6341024e0 0x7fa634074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 -- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634102e80 msgr2=0x7fa634074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa634102e80 0x7fa634074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 -- 172.21.15.52:0/4143577453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6341024e0 msgr2=0x7fa634074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6341024e0 0x7fa634074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 -- 172.21.15.52:0/4143577453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6341a6fe0 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa632d9d700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341084f0 0x7fa634075210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00ef30 tx=0x7fa62c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0092e0 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa62c004bb0 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa62c004020 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a7270 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.648+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a7730 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.649+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa634062ba0 con 0x7fa6341084f0 2024-06-02T23:08:57.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.650+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa62c01b030 con 0x7fa6341084f0 2024-06-02T23:08:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.651+0000 7fa6237fe700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa61c0a2d30 0x7fa61c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.651+0000 7fa63259c700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa61c0a2d30 0x7fa61c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.651+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa62c0cf590 con 0x7fa6341084f0 2024-06-02T23:08:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.651+0000 7fa6237fe700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] conn(0x7fa61c0ac7f0 0x7fa61c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.651+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 --> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa62c01fe50 con 0x7fa61c0ac7f0 2024-06-02T23:08:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.651+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa62c0cf940 con 0x7fa6341084f0 2024-06-02T23:08:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.652+0000 7fa63259c700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa61c0a2d30 0x7fa61c0a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa62400abd0 tx=0x7fa62400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.652+0000 7fa631d9b700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] conn(0x7fa61c0ac7f0 0x7fa61c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.652+0000 7fa631d9b700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] conn(0x7fa61c0ac7f0 0x7fa61c0aec90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:57.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.653+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== osd.5 v2:172.21.15.174:6808/1358350436 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa62c01fe50 con 0x7fa61c0ac7f0 2024-06-02T23:08:57.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.669+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 --> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa634063020 con 0x7fa61c0ac7f0 2024-06-02T23:08:57.670 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:57 smithi052 ceph-mon[32340]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-02T23:08:57.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa6237fe700 1 -- 172.21.15.52:0/4143577453 <== osd.5 v2:172.21.15.174:6808/1358350436 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa634063020 con 0x7fa61c0ac7f0 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] conn(0x7fa61c0ac7f0 msgr2=0x7fa61c0aec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa638f25700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] conn(0x7fa61c0ac7f0 0x7fa61c0aec90 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa61c0a2d30 msgr2=0x7fa61c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa638f25700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa61c0a2d30 0x7fa61c0a51f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa62400abd0 tx=0x7fa62400a380 comp rx=0 tx=0).stop 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341084f0 msgr2=0x7fa634075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.670+0000 7fa638f25700 1 --2- 172.21.15.52:0/4143577453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6341084f0 0x7fa634075210 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa62c00ef30 tx=0x7fa62c00c3a0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.671+0000 7fa632d9d700 1 -- 172.21.15.52:0/4143577453 reap_dead start 2024-06-02T23:08:57.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.671+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 shutdown_connections 2024-06-02T23:08:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.671+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 >> 172.21.15.52:0/4143577453 conn(0x7fa634079530 msgr2=0x7fa634100590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.671+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 shutdown_connections 2024-06-02T23:08:57.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.671+0000 7fa638f25700 1 -- 172.21.15.52:0/4143577453 wait complete. 2024-06-02T23:08:57.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:57 smithi123 ceph-mon[35977]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-02T23:08:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 -- 172.21.15.52:0/966808022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 msgr2=0x7faec410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 --2- 172.21.15.52:0/966808022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 0x7faec410f330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faebc00b0a0 tx=0x7faebc00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 -- 172.21.15.52:0/966808022 shutdown_connections 2024-06-02T23:08:57.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 --2- 172.21.15.52:0/966808022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 0x7faec410f330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 --2- 172.21.15.52:0/966808022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4108510 0x7faec41088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 --2- 172.21.15.52:0/966808022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec4100a80 0x7faec4100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 -- 172.21.15.52:0/966808022 >> 172.21.15.52:0/966808022 conn(0x7faec4077a00 msgr2=0x7faec40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 -- 172.21.15.52:0/966808022 shutdown_connections 2024-06-02T23:08:57.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.880+0000 7faec9146700 1 -- 172.21.15.52:0/966808022 wait complete. 2024-06-02T23:08:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 Processor -- start 2024-06-02T23:08:57.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 -- start start 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec4100a80 0x7faec419e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 0x7faec419ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4108510 0x7faec4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faec4112620 con 0x7faec4101440 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faec41124a0 con 0x7faec4108510 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec9146700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faec41127a0 con 0x7faec4100a80 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 0x7faec419ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 0x7faec419ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55298/0 (socket says 172.21.15.52:55298) 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec259c700 1 -- 172.21.15.52:0/3071805373 learned_addr learned my addr 172.21.15.52:0/3071805373 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec359e700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4108510 0x7faec4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.881+0000 7faec2d9d700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec4100a80 0x7faec419e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec359e700 1 -- 172.21.15.52:0/3071805373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec4100a80 msgr2=0x7faec419e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec359e700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec4100a80 0x7faec419e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec359e700 1 -- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 msgr2=0x7faec419ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec359e700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faec4101440 0x7faec419ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec359e700 1 -- 172.21.15.52:0/3071805373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec4199080 con 0x7faec4108510 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec359e700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4108510 0x7faec4198920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faebc00d010 tx=0x7faebc004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc021070 con 0x7faec4108510 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec4199310 con 0x7faec4108510 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faebc007e30 con 0x7faec4108510 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc004590 con 0x7faec4108510 2024-06-02T23:08:57.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.882+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec4101b90 con 0x7faec4108510 2024-06-02T23:08:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.883+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faec4062c20 con 0x7faec4108510 2024-06-02T23:08:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.885+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7faebc004730 con 0x7faec4108510 2024-06-02T23:08:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.885+0000 7faeabfff700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faeac0a2cb0 0x7faeac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.885+0000 7faec2d9d700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faeac0a2cb0 0x7faeac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.885+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7faebc0cffe0 con 0x7faec4108510 2024-06-02T23:08:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.886+0000 7faeabfff700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] conn(0x7faeac0ac770 0x7faeac0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:08:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.886+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 --> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faeac0af2e0 con 0x7faeac0ac770 2024-06-02T23:08:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.886+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7faebc0d0470 con 0x7faec4108510 2024-06-02T23:08:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.886+0000 7faec2d9d700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faeac0a2cb0 0x7faeac0a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faec40fe490 tx=0x7faeb4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:57.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.886+0000 7faec259c700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] conn(0x7faeac0ac770 0x7faeac0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:08:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.886+0000 7faec259c700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] conn(0x7faeac0ac770 0x7faeac0aec10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:08:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.887+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== osd.1 v2:172.21.15.123:6800/338548510 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faeac0af2e0 con 0x7faeac0ac770 2024-06-02T23:08:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:57 smithi174 ceph-mon[35994]: pgmap v343: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-02T23:08:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.904+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 --> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faec40630a0 con 0x7faeac0ac770 2024-06-02T23:08:57.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.904+0000 7faeabfff700 1 -- 172.21.15.52:0/3071805373 <== osd.1 v2:172.21.15.123:6800/338548510 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7faec40630a0 con 0x7faeac0ac770 2024-06-02T23:08:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] conn(0x7faeac0ac770 msgr2=0x7faeac0aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] conn(0x7faeac0ac770 0x7faeac0aec10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:08:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faeac0a2cb0 msgr2=0x7faeac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faeac0a2cb0 0x7faeac0a5170 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7faec40fe490 tx=0x7faeb4009260 comp rx=0 tx=0).stop 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4108510 msgr2=0x7faec4198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 --2- 172.21.15.52:0/3071805373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faec4108510 0x7faec4198920 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faebc00d010 tx=0x7faebc004d00 comp rx=0 tx=0).stop 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec359e700 1 -- 172.21.15.52:0/3071805373 reap_dead start 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 shutdown_connections 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 >> 172.21.15.52:0/3071805373 conn(0x7faec4077a00 msgr2=0x7faec40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 shutdown_connections 2024-06-02T23:08:57.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:08:57.905+0000 7faec9146700 1 -- 172.21.15.52:0/3071805373 wait complete. 2024-06-02T23:08:58.660 INFO:teuthology.orchestra.run.smithi052.stdout:163208757312 2024-06-02T23:08:58.660 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-06-02T23:08:59.050 INFO:teuthology.orchestra.run.smithi052.stdout:34359738456 2024-06-02T23:08:59.051 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-06-02T23:08:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:08:59 smithi123 ceph-mon[35977]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-02T23:08:59.854 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:08:59 smithi052 ceph-mon[32340]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-02T23:08:59.882 INFO:teuthology.orchestra.run.smithi052.stdout:287762808875 2024-06-02T23:08:59.882 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-06-02T23:08:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:08:59 smithi174 ceph-mon[35994]: pgmap v344: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 10 op/s 2024-06-02T23:09:00.562 INFO:teuthology.orchestra.run.smithi052.stdout:55834574932 2024-06-02T23:09:00.563 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-06-02T23:09:00.603 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 -- 172.21.15.52:0/2524550325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006aa00 msgr2=0x7f35c006ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 --2- 172.21.15.52:0/2524550325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006aa00 0x7f35c006ae80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009990 tx=0x7f35b0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 -- 172.21.15.52:0/2524550325 shutdown_connections 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 --2- 172.21.15.52:0/2524550325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c006b3c0 0x7f35c010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 --2- 172.21.15.52:0/2524550325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c01084f0 0x7f35c01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 --2- 172.21.15.52:0/2524550325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006aa00 0x7f35c006ae80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 -- 172.21.15.52:0/2524550325 >> 172.21.15.52:0/2524550325 conn(0x7f35c0078300 msgr2=0x7f35c0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 -- 172.21.15.52:0/2524550325 shutdown_connections 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.190+0000 7f35c5169700 1 -- 172.21.15.52:0/2524550325 wait complete. 2024-06-02T23:09:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 Processor -- start 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 -- start start 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c006aa00 0x7f35c019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006b3c0 0x7f35c019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c01084f0 0x7f35c01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0112680 con 0x7f35c006b3c0 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0112500 con 0x7f35c01084f0 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35c5169700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0112800 con 0x7f35c006aa00 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.191+0000 7f35be59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006b3c0 0x7f35c019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006b3c0 0x7f35c019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55314/0 (socket says 172.21.15.52:55314) 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 -- 172.21.15.52:0/2776072494 learned_addr learned my addr 172.21.15.52:0/2776072494 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35bed9d700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c006aa00 0x7f35c019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35bf59e700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c01084f0 0x7f35c01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 -- 172.21.15.52:0/2776072494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c006aa00 msgr2=0x7f35c019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c006aa00 0x7f35c019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 -- 172.21.15.52:0/2776072494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c01084f0 msgr2=0x7f35c01989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c01084f0 0x7f35c01989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 -- 172.21.15.52:0/2776072494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c0199140 con 0x7f35c006b3c0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35be59c700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006b3c0 0x7f35c019ed60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000c00 tx=0x7f35b0003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0027070 con 0x7f35c006b3c0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c01993d0 con 0x7f35c006b3c0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b000bdc0 con 0x7f35c006b3c0 2024-06-02T23:09:01.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.192+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b000f460 con 0x7f35c006b3c0 2024-06-02T23:09:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.193+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c0101380 con 0x7f35c006b3c0 2024-06-02T23:09:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.193+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f35c0062c20 con 0x7f35c006b3c0 2024-06-02T23:09:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.195+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f35b002a070 con 0x7f35c006b3c0 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.195+0000 7f35b7fff700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2cb0 0x7f35ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35bed9d700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2cb0 0x7f35ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f35b00d0e90 con 0x7f35c006b3c0 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35b7fff700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] conn(0x7f35ac0ac770 0x7f35ac0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 --> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f35ac0af2e0 con 0x7f35ac0ac770 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f35b00d12e0 con 0x7f35c006b3c0 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35bed9d700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2cb0 0x7f35ac0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f35a8005950 tx=0x7f35a800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35bf59e700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] conn(0x7f35ac0ac770 0x7f35ac0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:01.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.196+0000 7f35bf59e700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] conn(0x7f35ac0ac770 0x7f35ac0aec10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:01.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.197+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== osd.3 v2:172.21.15.52:6810/2267151243 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f35ac0af2e0 con 0x7f35ac0ac770 2024-06-02T23:09:01.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.219+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 --> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f35c0063020 con 0x7f35ac0ac770 2024-06-02T23:09:01.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.219+0000 7f35b7fff700 1 -- 172.21.15.52:0/2776072494 <== osd.3 v2:172.21.15.52:6810/2267151243 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f35c0063020 con 0x7f35ac0ac770 2024-06-02T23:09:01.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.219+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] conn(0x7f35ac0ac770 msgr2=0x7f35ac0aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:01.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.219+0000 7f35c5169700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] conn(0x7f35ac0ac770 0x7f35ac0aec10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:01.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2cb0 msgr2=0x7f35ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:01.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2cb0 0x7f35ac0a5170 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f35a8005950 tx=0x7f35a800a300 comp rx=0 tx=0).stop 2024-06-02T23:09:01.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006b3c0 msgr2=0x7f35c019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:01.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 --2- 172.21.15.52:0/2776072494 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c006b3c0 0x7f35c019ed60 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000c00 tx=0x7f35b0003a80 comp rx=0 tx=0).stop 2024-06-02T23:09:01.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35bf59e700 1 -- 172.21.15.52:0/2776072494 reap_dead start 2024-06-02T23:09:01.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 shutdown_connections 2024-06-02T23:09:01.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 >> 172.21.15.52:0/2776072494 conn(0x7f35c0078300 msgr2=0x7f35c01000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:01.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 shutdown_connections 2024-06-02T23:09:01.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:01.220+0000 7f35c5169700 1 -- 172.21.15.52:0/2776072494 wait complete. 2024-06-02T23:09:01.554 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:01 smithi052 ceph-mon[32340]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:01.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:01 smithi123 ceph-mon[35977]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:01 smithi174 ceph-mon[35994]: pgmap v345: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:02.872 INFO:teuthology.orchestra.run.smithi052.stdout:107374182476 2024-06-02T23:09:02.872 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-06-02T23:09:03.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 -- 172.21.15.52:0/1276536101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1290101fc0 msgr2=0x7f1290106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:03.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 --2- 172.21.15.52:0/1276536101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1290101fc0 0x7f1290106010 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f127c0099c0 tx=0x7f127c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 -- 172.21.15.52:0/1276536101 shutdown_connections 2024-06-02T23:09:03.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 --2- 172.21.15.52:0/1276536101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12901066f0 0x7f129010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 --2- 172.21.15.52:0/1276536101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1290101610 0x7f12901019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 --2- 172.21.15.52:0/1276536101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1290101fc0 0x7f1290106010 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 -- 172.21.15.52:0/1276536101 >> 172.21.15.52:0/1276536101 conn(0x7f1290078430 msgr2=0x7f1290078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 -- 172.21.15.52:0/1276536101 shutdown_connections 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.525+0000 7f1296e28700 1 -- 172.21.15.52:0/1276536101 wait complete. 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 Processor -- start 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 -- start start 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1290101610 0x7f129010eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1290101fc0 0x7f129010f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12901066f0 0x7f129010f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f12901126f0 con 0x7f12901066f0 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1290112570 con 0x7f1290101fc0 2024-06-02T23:09:03.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f1296e28700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1290112870 con 0x7f1290101610 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f12953c5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12901066f0 0x7f129010f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.526+0000 7f12953c5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12901066f0 0x7f129010f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55330/0 (socket says 172.21.15.52:55330) 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f128ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1290101fc0 0x7f129010f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 -- 172.21.15.52:0/716493449 learned_addr learned my addr 172.21.15.52:0/716493449 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f1294bc4700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1290101610 0x7f129010eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 -- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1290101610 msgr2=0x7f129010eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1290101610 0x7f129010eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 -- 172.21.15.52:0/716493449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1290101fc0 msgr2=0x7f129010f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1290101fc0 0x7f129010f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 -- 172.21.15.52:0/716493449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12901a7120 con 0x7f12901066f0 2024-06-02T23:09:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f12953c5700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12901066f0 0x7f129010f720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1288007ee0 tx=0x7f128800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288004bb0 con 0x7f12901066f0 2024-06-02T23:09:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1288004d50 con 0x7f12901066f0 2024-06-02T23:09:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12901a7410 con 0x7f12901066f0 2024-06-02T23:09:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12880058e0 con 0x7f12901066f0 2024-06-02T23:09:03.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.527+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12901a78d0 con 0x7f12901066f0 2024-06-02T23:09:03.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.528+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1290062ba0 con 0x7f12901066f0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f128801a020 con 0x7f12901066f0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f128dffb700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12800a2cb0 0x7f12800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f1294bc4700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12800a2cb0 0x7f12800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f12880cefd0 con 0x7f12901066f0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f128dffb700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] conn(0x7f12800ac770 0x7f12800aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 --> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1288021e70 con 0x7f12800ac770 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.530+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f12880cf380 con 0x7f12901066f0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.531+0000 7f1294bc4700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12800a2cb0 0x7f12800a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1284005950 tx=0x7f128400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.531+0000 7f128ffff700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] conn(0x7f12800ac770 0x7f12800aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.531+0000 7f128ffff700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] conn(0x7f12800ac770 0x7f12800aec10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:03.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.532+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== osd.8 v2:172.21.15.174:6816/186727045 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1288021e70 con 0x7f12800ac770 2024-06-02T23:09:03.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.548+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 --> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1290063020 con 0x7f12800ac770 2024-06-02T23:09:03.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.549+0000 7f128dffb700 1 -- 172.21.15.52:0/716493449 <== osd.8 v2:172.21.15.174:6816/186727045 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1290063020 con 0x7f12800ac770 2024-06-02T23:09:03.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.549+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] conn(0x7f12800ac770 msgr2=0x7f12800aec10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:03.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.549+0000 7f1296e28700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] conn(0x7f12800ac770 0x7f12800aec10 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12800a2cb0 msgr2=0x7f12800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:03.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12800a2cb0 0x7f12800a5170 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1284005950 tx=0x7f128400a300 comp rx=0 tx=0).stop 2024-06-02T23:09:03.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12901066f0 msgr2=0x7f129010f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:03.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 --2- 172.21.15.52:0/716493449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12901066f0 0x7f129010f720 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1288007ee0 tx=0x7f128800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:09:03.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f12953c5700 1 -- 172.21.15.52:0/716493449 reap_dead start 2024-06-02T23:09:03.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 shutdown_connections 2024-06-02T23:09:03.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 >> 172.21.15.52:0/716493449 conn(0x7f1290078430 msgr2=0x7f12900ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:03.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 shutdown_connections 2024-06-02T23:09:03.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:03.550+0000 7f1296e28700 1 -- 172.21.15.52:0/716493449 wait complete. 2024-06-02T23:09:03.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:03 smithi123 ceph-mon[35977]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:03 smithi174 ceph-mon[35994]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:03 smithi052 ceph-mon[32340]: pgmap v346: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:03.991 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:04.021 INFO:teuthology.orchestra.run.smithi052.stdout:141733920836 2024-06-02T23:09:04.021 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-06-02T23:09:04.204 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:04.780 INFO:teuthology.orchestra.run.smithi052.stdout:253403070513 2024-06-02T23:09:04.780 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-06-02T23:09:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:05.177 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:05.178 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:05 smithi123 ceph-mon[35977]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:05.826 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:05 smithi052 ceph-mon[32340]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:05 smithi174 ceph-mon[35994]: pgmap v347: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:07.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:07 smithi052 ceph-mon[32340]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-02T23:09:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:07 smithi123 ceph-mon[35977]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-02T23:09:07.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:07 smithi174 ceph-mon[35994]: pgmap v348: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-06-02T23:09:09.493 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:09 smithi052 ceph-mon[32340]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-02T23:09:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:09 smithi123 ceph-mon[35977]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-02T23:09:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:09 smithi174 ceph-mon[35994]: pgmap v349: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-06-02T23:09:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.422+0000 7f1bbd757700 1 -- 172.21.15.52:0/4044111619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb806abc0 msgr2=0x7f1bb806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.422+0000 7f1bbd757700 1 --2- 172.21.15.52:0/4044111619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb806abc0 0x7f1bb806b040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1bac009990 tx=0x7f1bac009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 -- 172.21.15.52:0/4044111619 shutdown_connections 2024-06-02T23:09:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 --2- 172.21.15.52:0/4044111619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb806b580 0x7f1bb810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 --2- 172.21.15.52:0/4044111619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb8108500 0x7f1bb81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 --2- 172.21.15.52:0/4044111619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb806abc0 0x7f1bb806b040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 -- 172.21.15.52:0/4044111619 >> 172.21.15.52:0/4044111619 conn(0x7f1bb8078340 msgr2=0x7f1bb8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 -- 172.21.15.52:0/4044111619 shutdown_connections 2024-06-02T23:09:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 -- 172.21.15.52:0/4044111619 wait complete. 2024-06-02T23:09:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.423+0000 7f1bbd757700 1 Processor -- start 2024-06-02T23:09:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 -- start start 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb806abc0 0x7f1bb819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb806b580 0x7f1bb819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 0x7f1bb819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8112610 con 0x7f1bb8108500 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8112490 con 0x7f1bb806abc0 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bbd757700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1bb8112790 con 0x7f1bb806b580 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 0x7f1bb819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 0x7f1bb819f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41292/0 (socket says 172.21.15.52:41292) 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 -- 172.21.15.52:0/3388590055 learned_addr learned my addr 172.21.15.52:0/3388590055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb6ffd700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb806abc0 0x7f1bb819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb67fc700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb806b580 0x7f1bb819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 -- 172.21.15.52:0/3388590055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb806b580 msgr2=0x7f1bb819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb806b580 0x7f1bb819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 -- 172.21.15.52:0/3388590055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb806abc0 msgr2=0x7f1bb819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb806abc0 0x7f1bb819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.424+0000 7f1bb77fe700 1 -- 172.21.15.52:0/3388590055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bb819fd90 con 0x7f1bb8108500 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.425+0000 7f1bb77fe700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 0x7f1bb819f650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0007ee0 tx=0x7f1bb000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.425+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0004bb0 con 0x7f1bb8108500 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.425+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bb81a0080 con 0x7f1bb8108500 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.425+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1bb0004d50 con 0x7f1bb8108500 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.425+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bb00058e0 con 0x7f1bb8108500 2024-06-02T23:09:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.425+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bb81a7900 con 0x7f1bb8108500 2024-06-02T23:09:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.426+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1bb8069180 con 0x7f1bb8108500 2024-06-02T23:09:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.427+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1bb001a020 con 0x7f1bb8108500 2024-06-02T23:09:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.428+0000 7f1b9ffff700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ba00a2cb0 0x7f1ba00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.428+0000 7f1bb6ffd700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ba00a2cb0 0x7f1ba00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.428+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1bb00cefc0 con 0x7f1bb8108500 2024-06-02T23:09:10.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.428+0000 7f1bb6ffd700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ba00a2cb0 0x7f1ba00a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005950 tx=0x7f1ba800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:10.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.432+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1bb0010070 con 0x7f1bb8108500 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 -- 172.21.15.52:0/1830299499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c1084f0 msgr2=0x7f919c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 --2- 172.21.15.52:0/1830299499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c1084f0 0x7f919c1088d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f9198009990 tx=0x7f9198009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 -- 172.21.15.52:0/1830299499 shutdown_connections 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 --2- 172.21.15.52:0/1830299499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 0x7f919c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 --2- 172.21.15.52:0/1830299499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c1084f0 0x7f919c1088d0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 --2- 172.21.15.52:0/1830299499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c06aa00 0x7f919c06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 -- 172.21.15.52:0/1830299499 >> 172.21.15.52:0/1830299499 conn(0x7f919c078300 msgr2=0x7f919c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 -- 172.21.15.52:0/1830299499 shutdown_connections 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.593+0000 7f91a2af7700 1 -- 172.21.15.52:0/1830299499 wait complete. 2024-06-02T23:09:10.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 Processor -- start 2024-06-02T23:09:10.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 -- start start 2024-06-02T23:09:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 0x7f919c19adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 0x7f919c19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c1084f0 0x7f919c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f919c112600 con 0x7f919c06aa00 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f919c112480 con 0x7f919c06b3c0 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a2af7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f919c112780 con 0x7f919c1084f0 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a0893700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 0x7f919c19adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a0893700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 0x7f919c19adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41310/0 (socket says 172.21.15.52:41310) 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a0893700 1 -- 172.21.15.52:0/559131852 learned_addr learned my addr 172.21.15.52:0/559131852 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f9193fff700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 0x7f919c19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.594+0000 7f91a1094700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c1084f0 0x7f919c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a0893700 1 -- 172.21.15.52:0/559131852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c1084f0 msgr2=0x7f919c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a0893700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c1084f0 0x7f919c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a0893700 1 -- 172.21.15.52:0/559131852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 msgr2=0x7f919c19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a0893700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 0x7f919c19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a0893700 1 -- 172.21.15.52:0/559131852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919c19fe00 con 0x7f919c06aa00 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a1094700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c1084f0 0x7f919c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f9193fff700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 0x7f919c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a0893700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 0x7f919c19adf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9198000c00 tx=0x7f9198003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198027070 con 0x7f919c06aa00 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919c1a0090 con 0x7f919c06aa00 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f919800bdc0 con 0x7f919c06aa00 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9198022d30 con 0x7f919c06aa00 2024-06-02T23:09:10.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.595+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f919c1a7970 con 0x7f919c06aa00 2024-06-02T23:09:10.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.597+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f919c069180 con 0x7f919c06aa00 2024-06-02T23:09:10.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.598+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f919802a070 con 0x7f919c06aa00 2024-06-02T23:09:10.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.599+0000 7f9191ffb700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f91840a2cb0 0x7f91840a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:10.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.599+0000 7f9193fff700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f91840a2cb0 0x7f91840a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:10.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.599+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f91980cfef0 con 0x7f919c06aa00 2024-06-02T23:09:10.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.599+0000 7f9193fff700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f91840a2cb0 0x7f91840a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f919c103440 tx=0x7f918c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:10.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.603+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9198013070 con 0x7f919c06aa00 2024-06-02T23:09:10.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.663+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f1bb81a7ad0 con 0x7f1bb8108500 2024-06-02T23:09:10.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.664+0000 7f1b9ffff700 1 -- 172.21.15.52:0/3388590055 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f1bb009bd90 con 0x7f1bb8108500 2024-06-02T23:09:10.663 INFO:teuthology.orchestra.run.smithi052.stdout:219043332153 2024-06-02T23:09:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ba00a2cb0 msgr2=0x7f1ba00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ba00a2cb0 0x7f1ba00a5170 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8005950 tx=0x7f1ba800b410 comp rx=0 tx=0).stop 2024-06-02T23:09:10.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 msgr2=0x7f1bb819f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 0x7f1bb819f650 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0007ee0 tx=0x7f1bb000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 shutdown_connections 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ba00a2cb0 0x7f1ba00a5170 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1bb8108500 0x7f1bb819f650 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1bb806abc0 0x7f1bb819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 --2- 172.21.15.52:0/3388590055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1bb806b580 0x7f1bb819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 >> 172.21.15.52:0/3388590055 conn(0x7f1bb8078340 msgr2=0x7f1bb810d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 shutdown_connections 2024-06-02T23:09:10.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.666+0000 7f1bbd757700 1 -- 172.21.15.52:0/3388590055 wait complete. 2024-06-02T23:09:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.838+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f919c1a7b40 con 0x7f919c06aa00 2024-06-02T23:09:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.838+0000 7f9191ffb700 1 -- 172.21.15.52:0/559131852 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f919809cdb0 con 0x7f919c06aa00 2024-06-02T23:09:10.837 INFO:teuthology.orchestra.run.smithi052.stdout:309237645353 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f91840a2cb0 msgr2=0x7f91840a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f91840a2cb0 0x7f91840a5170 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f919c103440 tx=0x7f918c00b410 comp rx=0 tx=0).stop 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 msgr2=0x7f919c19adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 0x7f919c19adf0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9198000c00 tx=0x7f9198003a80 comp rx=0 tx=0).stop 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 shutdown_connections 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f91840a2cb0 0x7f91840a5170 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f919c1084f0 0x7f919c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f919c06aa00 0x7f919c19adf0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 --2- 172.21.15.52:0/559131852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f919c06b3c0 0x7f919c19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.842+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 >> 172.21.15.52:0/559131852 conn(0x7f919c078300 msgr2=0x7f919c10d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:10.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.843+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 shutdown_connections 2024-06-02T23:09:10.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:10.843+0000 7f91a2af7700 1 -- 172.21.15.52:0/559131852 wait complete. 2024-06-02T23:09:11.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:11 smithi174 ceph-mon[35994]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:11.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3388590055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-02T23:09:11.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/559131852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-02T23:09:11.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:11 smithi123 ceph-mon[35977]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3388590055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-02T23:09:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/559131852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-02T23:09:11.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:11 smithi052 ceph-mon[32340]: pgmap v350: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:11.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3388590055' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-06-02T23:09:11.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/559131852' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-06-02T23:09:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 -- 172.21.15.52:0/3554382729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 msgr2=0x7f28c806a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/3554382729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c806a5f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f28b8009990 tx=0x7f28b8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 -- 172.21.15.52:0/3554382729 shutdown_connections 2024-06-02T23:09:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/3554382729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c806ab30 0x7f28c810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/3554382729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c8108500 0x7f28c81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/3554382729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c806a5f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 -- 172.21.15.52:0/3554382729 >> 172.21.15.52:0/3554382729 conn(0x7f28c8078340 msgr2=0x7f28c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 -- 172.21.15.52:0/3554382729 shutdown_connections 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.747+0000 7f28cf6e9700 1 -- 172.21.15.52:0/3554382729 wait complete. 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 Processor -- start 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 -- start start 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c806ab30 0x7f28c819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c8108500 0x7f28c8198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8112610 con 0x7f28c806a170 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8112490 con 0x7f28c8108500 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cf6e9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28c8112790 con 0x7f28c806ab30 2024-06-02T23:09:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41314/0 (socket says 172.21.15.52:41314) 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 -- 172.21.15.52:0/4045194565 learned_addr learned my addr 172.21.15.52:0/4045194565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cdc86700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c8108500 0x7f28c8198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28ccc84700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c806ab30 0x7f28c819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 -- 172.21.15.52:0/4045194565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c806ab30 msgr2=0x7f28c819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c806ab30 0x7f28c819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 -- 172.21.15.52:0/4045194565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c8108500 msgr2=0x7f28c8198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c8108500 0x7f28c8198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.748+0000 7f28cd485700 1 -- 172.21.15.52:0/4045194565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28c81990d0 con 0x7f28c806a170 2024-06-02T23:09:13.371 INFO:teuthology.orchestra.run.smithi052.stdout:343597383717 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.749+0000 7f28cd485700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c819e7b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28c400d980 tx=0x7f28c400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.749+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4016070 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.749+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28c81993c0 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.749+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f28c4004830 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.749+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4005530 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.749+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28c81a5680 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.750+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28c8069180 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.751+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f28c40049e0 con 0x7f28c806a170 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.751+0000 7f28be7fc700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f28b40a2cb0 0x7f28b40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.752+0000 7f28ccc84700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f28b40a2cb0 0x7f28b40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.752+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f28c40cf390 con 0x7f28c806a170 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.752+0000 7f28ccc84700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f28b40a2cb0 0x7f28b40a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b2e0 tx=0x7f28b8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.755+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f28c401f020 con 0x7f28c806a170 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.975+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f28c802dca0 con 0x7f28c806a170 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.976+0000 7f28be7fc700 1 -- 172.21.15.52:0/4045194565 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f28c409c1c0 con 0x7f28c806a170 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.978+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f28b40a2cb0 msgr2=0x7f28b40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.978+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f28b40a2cb0 0x7f28b40a5170 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f28b800b2e0 tx=0x7f28b8005960 comp rx=0 tx=0).stop 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.978+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 msgr2=0x7f28c819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.978+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c819e7b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f28c400d980 tx=0x7f28c400dc90 comp rx=0 tx=0).stop 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.978+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 shutdown_connections 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f28b40a2cb0 0x7f28b40a5170 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28c8108500 0x7f28c8198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f28c806a170 0x7f28c819e7b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 --2- 172.21.15.52:0/4045194565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28c806ab30 0x7f28c819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 >> 172.21.15.52:0/4045194565 conn(0x7f28c8078340 msgr2=0x7f28c8100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 shutdown_connections 2024-06-02T23:09:13.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:12.979+0000 7f28cf6e9700 1 -- 172.21.15.52:0/4045194565 wait complete. 2024-06-02T23:09:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:13 smithi052 ceph-mon[32340]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:13 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4045194565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-02T23:09:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:13 smithi123 ceph-mon[35977]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:13 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4045194565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-02T23:09:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:13 smithi174 ceph-mon[35994]: pgmap v351: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:13 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4045194565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-06-02T23:09:14.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 -- 172.21.15.52:0/416973786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d801030f0 msgr2=0x7f2d80107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 --2- 172.21.15.52:0/416973786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d801030f0 0x7f2d80107110 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d700099c0 tx=0x7f2d70009cd0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 -- 172.21.15.52:0/416973786 shutdown_connections 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 --2- 172.21.15.52:0/416973786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80107840 0x7f2d8010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 --2- 172.21.15.52:0/416973786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d80102740 0x7f2d80102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 --2- 172.21.15.52:0/416973786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d801030f0 0x7f2d80107110 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 -- 172.21.15.52:0/416973786 >> 172.21.15.52:0/416973786 conn(0x7f2d800795f0 msgr2=0x7f2d80079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 -- 172.21.15.52:0/416973786 shutdown_connections 2024-06-02T23:09:14.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.656+0000 7f2d8746e700 1 -- 172.21.15.52:0/416973786 wait complete. 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.657+0000 7f2d8746e700 1 Processor -- start 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.657+0000 7f2d8746e700 1 -- start start 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.657+0000 7f2d8746e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 0x7f2d8019e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.657+0000 7f2d8746e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d801030f0 0x7f2d8019ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.657+0000 7f2d8746e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d80107840 0x7f2d80198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.657+0000 7f2d8746e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d80112620 con 0x7f2d80107840 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8746e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801124a0 con 0x7f2d80102740 2024-06-02T23:09:14.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8746e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d801127a0 con 0x7f2d801030f0 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8520a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 0x7f2d8019e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8520a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 0x7f2d8019e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35856/0 (socket says 172.21.15.52:35856) 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8520a700 1 -- 172.21.15.52:0/203030936 learned_addr learned my addr 172.21.15.52:0/203030936 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d80107840 0x7f2d80198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d84a09700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d801030f0 0x7f2d8019ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 -- 172.21.15.52:0/203030936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d801030f0 msgr2=0x7f2d8019ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d801030f0 0x7f2d8019ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 -- 172.21.15.52:0/203030936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 msgr2=0x7f2d8019e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 0x7f2d8019e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 -- 172.21.15.52:0/203030936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d801990c0 con 0x7f2d80107840 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8520a700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 0x7f2d8019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:14.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d85a0b700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d80107840 0x7f2d80198960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800cc50 tx=0x7f2d7800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:14.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d78016020 con 0x7f2d80107840 2024-06-02T23:09:14.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d801993b0 con 0x7f2d80107840 2024-06-02T23:09:14.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d78012420 con 0x7f2d80107840 2024-06-02T23:09:14.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.658+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d780115a0 con 0x7f2d80107840 2024-06-02T23:09:14.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.659+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d801a5620 con 0x7f2d80107840 2024-06-02T23:09:14.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.660+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d8010aa10 con 0x7f2d80107840 2024-06-02T23:09:14.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.661+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f2d78019020 con 0x7f2d80107840 2024-06-02T23:09:14.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.661+0000 7f2d767fc700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d6c0a2cb0 0x7f2d6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:14.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.662+0000 7f2d8520a700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d6c0a2cb0 0x7f2d6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:14.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.662+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2d780cf2c0 con 0x7f2d80107840 2024-06-02T23:09:14.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.662+0000 7f2d8520a700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d6c0a2cb0 0x7f2d6c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d801072c0 tx=0x7f2d7c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:14.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.665+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2d78020020 con 0x7f2d80107840 2024-06-02T23:09:14.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.879+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2d80199e70 con 0x7f2d80107840 2024-06-02T23:09:14.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.879+0000 7f2d767fc700 1 -- 172.21.15.52:0/203030936 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f2d80199e70 con 0x7f2d80107840 2024-06-02T23:09:14.879 INFO:teuthology.orchestra.run.smithi052.stdout:77309411411 2024-06-02T23:09:14.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d6c0a2cb0 msgr2=0x7f2d6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:14.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d6c0a2cb0 0x7f2d6c0a5170 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d801072c0 tx=0x7f2d7c00b480 comp rx=0 tx=0).stop 2024-06-02T23:09:14.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d80107840 msgr2=0x7f2d80198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:14.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d80107840 0x7f2d80198960 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800cc50 tx=0x7f2d7800cf60 comp rx=0 tx=0).stop 2024-06-02T23:09:14.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 shutdown_connections 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d6c0a2cb0 0x7f2d6c0a5170 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d80107840 0x7f2d80198960 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d80102740 0x7f2d8019e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 --2- 172.21.15.52:0/203030936 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d801030f0 0x7f2d8019ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 >> 172.21.15.52:0/203030936 conn(0x7f2d800795f0 msgr2=0x7f2d80108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 shutdown_connections 2024-06-02T23:09:14.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:14.882+0000 7f2d8746e700 1 -- 172.21.15.52:0/203030936 wait complete. 2024-06-02T23:09:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 -- 172.21.15.52:0/992681429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101380 msgr2=0x7fac80101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 --2- 172.21.15.52:0/992681429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101380 0x7fac80101800 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fac70009990 tx=0x7fac70009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 -- 172.21.15.52:0/992681429 shutdown_connections 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 --2- 172.21.15.52:0/992681429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac80101d40 0x7fac8010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 --2- 172.21.15.52:0/992681429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac801073f0 0x7fac801077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 --2- 172.21.15.52:0/992681429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101380 0x7fac80101800 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 -- 172.21.15.52:0/992681429 >> 172.21.15.52:0/992681429 conn(0x7fac80078300 msgr2=0x7fac80078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 -- 172.21.15.52:0/992681429 shutdown_connections 2024-06-02T23:09:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.432+0000 7fac87bf4700 1 -- 172.21.15.52:0/992681429 wait complete. 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 Processor -- start 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 -- start start 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac80101380 0x7fac8019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 0x7fac8019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac801073f0 0x7fac80198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fac80111540 con 0x7fac80101d40 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac801113c0 con 0x7fac801073f0 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac87bf4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac801116c0 con 0x7fac80101380 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.433+0000 7fac8518f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 0x7fac8019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 0x7fac8019ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39956/0 (socket says 172.21.15.52:39956) 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 -- 172.21.15.52:0/2616804164 learned_addr learned my addr 172.21.15.52:0/2616804164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac86191700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac801073f0 0x7fac80198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac85990700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac80101380 0x7fac8019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 -- 172.21.15.52:0/2616804164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac80101380 msgr2=0x7fac8019e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac80101380 0x7fac8019e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 -- 172.21.15.52:0/2616804164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac801073f0 msgr2=0x7fac80198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac801073f0 0x7fac80198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 -- 172.21.15.52:0/2616804164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac801990c0 con 0x7fac80101d40 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac8518f700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 0x7fac8019ece0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fac70000c00 tx=0x7fac70003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70027070 con 0x7fac80101d40 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac80199350 con 0x7fac80101d40 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac7000bdc0 con 0x7fac80101d40 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac70022d30 con 0x7fac80101d40 2024-06-02T23:09:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.434+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac801a5670 con 0x7fac80101d40 2024-06-02T23:09:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.435+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac80069180 con 0x7fac80101d40 2024-06-02T23:09:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.436+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fac7002a070 con 0x7fac80101d40 2024-06-02T23:09:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.437+0000 7fac76ffd700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac6c0a2cb0 0x7fac6c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.437+0000 7fac85990700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac6c0a2cb0 0x7fac6c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.437+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fac700d02a0 con 0x7fac80101d40 2024-06-02T23:09:15.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.438+0000 7fac85990700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac6c0a2cb0 0x7fac6c0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fac80199aa0 tx=0x7fac7c00b4d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:15.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.441+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac70013070 con 0x7fac80101d40 2024-06-02T23:09:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:15 smithi052 ceph-mon[32340]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:15 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/203030936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-02T23:09:15.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.651+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fac801a5840 con 0x7fac80101d40 2024-06-02T23:09:15.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.652+0000 7fac76ffd700 1 -- 172.21.15.52:0/2616804164 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fac7009d160 con 0x7fac80101d40 2024-06-02T23:09:15.651 INFO:teuthology.orchestra.run.smithi052.stdout:287762808879 2024-06-02T23:09:15.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac6c0a2cb0 msgr2=0x7fac6c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:15.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac6c0a2cb0 0x7fac6c0a5170 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fac80199aa0 tx=0x7fac7c00b4d0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 msgr2=0x7fac8019ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 0x7fac8019ece0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fac70000c00 tx=0x7fac70003a80 comp rx=0 tx=0).stop 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 shutdown_connections 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac6c0a2cb0 0x7fac6c0a5170 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac801073f0 0x7fac80198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac80101380 0x7fac8019e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 --2- 172.21.15.52:0/2616804164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac80101d40 0x7fac8019ece0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 >> 172.21.15.52:0/2616804164 conn(0x7fac80078300 msgr2=0x7fac80100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 shutdown_connections 2024-06-02T23:09:15.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:15.654+0000 7fac87bf4700 1 -- 172.21.15.52:0/2616804164 wait complete. 2024-06-02T23:09:15.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:15 smithi123 ceph-mon[35977]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:15 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/203030936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-02T23:09:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:15 smithi174 ceph-mon[35994]: pgmap v352: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:15 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/203030936' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-06-02T23:09:16.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:16 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2616804164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-02T23:09:16.775 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:16 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2616804164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-02T23:09:16.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:16 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2616804164' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-06-02T23:09:17.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383712 got 343597383717 for osd.11 2024-06-02T23:09:17.014 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:17.446 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808875 got 287762808879 for osd.9 2024-06-02T23:09:17.446 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:17.452 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:17.711 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:17 smithi052 ceph-mon[32340]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:17.711 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:17 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/419348191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-02T23:09:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:17 smithi123 ceph-mon[35977]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:17 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/419348191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-02T23:09:17.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 -- 172.21.15.52:0/2517998161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 msgr2=0x7f1d5010e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:17.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 --2- 172.21.15.52:0/2517998161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d5010e2c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d4800b0d0 tx=0x7f1d4800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 -- 172.21.15.52:0/2517998161 shutdown_connections 2024-06-02T23:09:17.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 --2- 172.21.15.52:0/2517998161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d5010e2c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 --2- 172.21.15.52:0/2517998161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d50101610 0x7f1d501019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 --2- 172.21.15.52:0/2517998161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d50101fc0 0x7f1d50106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 -- 172.21.15.52:0/2517998161 >> 172.21.15.52:0/2517998161 conn(0x7f1d50078430 msgr2=0x7f1d50078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 -- 172.21.15.52:0/2517998161 shutdown_connections 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.776+0000 7f1d55a07700 1 -- 172.21.15.52:0/2517998161 wait complete. 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 Processor -- start 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 -- start start 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d50101610 0x7f1d501999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d50101fc0 0x7f1d50199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:17.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d501a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d501115b0 con 0x7f1d501066f0 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50111430 con 0x7f1d50101fc0 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d55a07700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d50111730 con 0x7f1d50101610 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d501a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d501a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39982/0 (socket says 172.21.15.52:39982) 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d4f7fe700 1 -- 172.21.15.52:0/419348191 learned_addr learned my addr 172.21.15.52:0/419348191 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d4e7fc700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d50101fc0 0x7f1d50199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.777+0000 7f1d4effd700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d50101610 0x7f1d501999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d4f7fe700 1 -- 172.21.15.52:0/419348191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d50101610 msgr2=0x7f1d501999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d4f7fe700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d50101610 0x7f1d501999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d4f7fe700 1 -- 172.21.15.52:0/419348191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d50101fc0 msgr2=0x7f1d50199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d4f7fe700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d50101fc0 0x7f1d50199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d4f7fe700 1 -- 172.21.15.52:0/419348191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d501a4f90 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d4f7fe700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d501a4a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009850 tx=0x7f1d48007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4801f070 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d501a5160 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d48004c50 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4801b670 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.778+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d501a5560 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.779+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d50069180 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.780+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1d48022070 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.781+0000 7f1d54a05700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1d380a2cb0 0x7f1d380a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.781+0000 7f1d4effd700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1d380a2cb0 0x7f1d380a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.781+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1d4800f020 con 0x7f1d501066f0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.781+0000 7f1d4effd700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1d380a2cb0 0x7f1d380a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d40004630 tx=0x7f1d4000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:17.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.785+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d48029020 con 0x7f1d501066f0 2024-06-02T23:09:17.808 INFO:teuthology.orchestra.run.smithi052.stdout:34359738460 2024-06-02T23:09:17.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.994+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1d5002dca0 con 0x7f1d501066f0 2024-06-02T23:09:17.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.995+0000 7f1d54a05700 1 -- 172.21.15.52:0/419348191 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1d4809d0a0 con 0x7f1d501066f0 2024-06-02T23:09:17.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1d380a2cb0 msgr2=0x7f1d380a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:17.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1d380a2cb0 0x7f1d380a5170 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1d40004630 tx=0x7f1d4000b3c0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 msgr2=0x7f1d501a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d501a4a50 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d48009850 tx=0x7f1d48007a20 comp rx=0 tx=0).stop 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 shutdown_connections 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1d380a2cb0 0x7f1d380a5170 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d501066f0 0x7f1d501a4a50 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d50101610 0x7f1d501999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 --2- 172.21.15.52:0/419348191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d50101fc0 0x7f1d50199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 >> 172.21.15.52:0/419348191 conn(0x7f1d50078430 msgr2=0x7f1d500ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 shutdown_connections 2024-06-02T23:09:17.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:16.997+0000 7f1d55a07700 1 -- 172.21.15.52:0/419348191 wait complete. 2024-06-02T23:09:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:17 smithi174 ceph-mon[35994]: pgmap v353: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:17 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/419348191' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 -- 172.21.15.52:0/2771483673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128101390 msgr2=0x7f4128101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 --2- 172.21.15.52:0/2771483673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128101390 0x7f4128101810 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4114009990 tx=0x7f4114009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 -- 172.21.15.52:0/2771483673 shutdown_connections 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 --2- 172.21.15.52:0/2771483673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4128101d50 0x7f412810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 --2- 172.21.15.52:0/2771483673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128107400 0x7f41281077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 --2- 172.21.15.52:0/2771483673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128101390 0x7f4128101810 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 -- 172.21.15.52:0/2771483673 >> 172.21.15.52:0/2771483673 conn(0x7f4128078340 msgr2=0x7f4128078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 -- 172.21.15.52:0/2771483673 shutdown_connections 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.288+0000 7f412eb9a700 1 -- 172.21.15.52:0/2771483673 wait complete. 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 Processor -- start 2024-06-02T23:09:18.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 -- start start 2024-06-02T23:09:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128101390 0x7f412819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4128101d50 0x7f412819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 0x7f41281989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4128111550 con 0x7f4128107400 2024-06-02T23:09:18.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41281113d0 con 0x7f4128101390 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412eb9a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41281116d0 con 0x7f4128101d50 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 0x7f41281989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 0x7f41281989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40000/0 (socket says 172.21.15.52:40000) 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 -- 172.21.15.52:0/12097230 learned_addr learned my addr 172.21.15.52:0/12097230 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f4127fff700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4128101d50 0x7f412819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412c936700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128101390 0x7f412819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 -- 172.21.15.52:0/12097230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4128101d50 msgr2=0x7f412819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4128101d50 0x7f412819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 -- 172.21.15.52:0/12097230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128101390 msgr2=0x7f412819e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128101390 0x7f412819e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412d137700 1 -- 172.21.15.52:0/12097230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4128199120 con 0x7f4128107400 2024-06-02T23:09:18.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.289+0000 7f412c936700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128101390 0x7f412819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.290+0000 7f412d137700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 0x7f41281989c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4120007ee0 tx=0x7f412000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.290+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412000dbd0 con 0x7f4128107400 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.290+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f412000dd70 con 0x7f4128107400 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.290+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4128199410 con 0x7f4128107400 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.290+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4120005990 con 0x7f4128107400 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.290+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41281a5680 con 0x7f4128107400 2024-06-02T23:09:18.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.291+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4128069180 con 0x7f4128107400 2024-06-02T23:09:18.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.292+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f412001a020 con 0x7f4128107400 2024-06-02T23:09:18.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.293+0000 7f4125ffb700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41180a2d30 0x7f41180a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:18.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.293+0000 7f412c936700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41180a2d30 0x7f41180a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:18.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.293+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f41200ce820 con 0x7f4128107400 2024-06-02T23:09:18.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.293+0000 7f412c936700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41180a2d30 0x7f41180a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f411c005e10 tx=0x7f411c005d20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:18.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.298+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4120010070 con 0x7f4128107400 2024-06-02T23:09:18.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.512+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f41281a58b0 con 0x7f4128107400 2024-06-02T23:09:18.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.512+0000 7f4125ffb700 1 -- 172.21.15.52:0/12097230 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f412009b650 con 0x7f4128107400 2024-06-02T23:09:18.512 INFO:teuthology.orchestra.run.smithi052.stdout:163208757316 2024-06-02T23:09:18.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.514+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41180a2d30 msgr2=0x7f41180a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:18.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.514+0000 7f412eb9a700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41180a2d30 0x7f41180a51f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f411c005e10 tx=0x7f411c005d20 comp rx=0 tx=0).stop 2024-06-02T23:09:18.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.514+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 msgr2=0x7f41281989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:18.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.514+0000 7f412eb9a700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 0x7f41281989c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4120007ee0 tx=0x7f412000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 shutdown_connections 2024-06-02T23:09:18.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41180a2d30 0x7f41180a51f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4128107400 0x7f41281989c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4128101390 0x7f412819e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 --2- 172.21.15.52:0/12097230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4128101d50 0x7f412819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 >> 172.21.15.52:0/12097230 conn(0x7f4128078340 msgr2=0x7f41281006e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 shutdown_connections 2024-06-02T23:09:18.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:18.515+0000 7f412eb9a700 1 -- 172.21.15.52:0/12097230 wait complete. 2024-06-02T23:09:18.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332147 got 219043332153 for osd.7 2024-06-02T23:09:18.532 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:18.938 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:18.939 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:18.969 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645348 got 309237645353 for osd.10 2024-06-02T23:09:18.969 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:19.361 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738456 got 34359738460 for osd.0 2024-06-02T23:09:19.361 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:19.450 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:19 smithi052 ceph-mon[32340]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:19.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 -- 172.21.15.52:0/253224765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee101030f0 msgr2=0x7fee10107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:19.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 --2- 172.21.15.52:0/253224765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee101030f0 0x7fee10107110 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fee000099c0 tx=0x7fee00009cd0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 -- 172.21.15.52:0/253224765 shutdown_connections 2024-06-02T23:09:19.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 --2- 172.21.15.52:0/253224765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10107840 0x7fee1010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 --2- 172.21.15.52:0/253224765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee10102740 0x7fee10102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 --2- 172.21.15.52:0/253224765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee101030f0 0x7fee10107110 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 -- 172.21.15.52:0/253224765 >> 172.21.15.52:0/253224765 conn(0x7fee100795f0 msgr2=0x7fee10079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:19.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 -- 172.21.15.52:0/253224765 shutdown_connections 2024-06-02T23:09:19.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 -- 172.21.15.52:0/253224765 wait complete. 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 Processor -- start 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.566+0000 7fee17283700 1 -- start start 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee17283700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102740 0x7fee1019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee17283700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101030f0 0x7fee1019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee17283700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 0x7fee101988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee17283700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee10112600 con 0x7fee10107840 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee17283700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee10112480 con 0x7fee101030f0 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee17283700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee10112780 con 0x7fee10102740 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 0x7fee101988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 0x7fee101988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40022/0 (socket says 172.21.15.52:40022) 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 -- 172.21.15.52:0/3365347541 learned_addr learned my addr 172.21.15.52:0/3365347541 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:19.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee1481e700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101030f0 0x7fee1019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee1501f700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102740 0x7fee1019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 -- 172.21.15.52:0/3365347541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102740 msgr2=0x7fee1019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102740 0x7fee1019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 -- 172.21.15.52:0/3365347541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101030f0 msgr2=0x7fee1019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101030f0 0x7fee1019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 -- 172.21.15.52:0/3365347541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee10199040 con 0x7fee10107840 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee1481e700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101030f0 0x7fee1019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee1501f700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102740 0x7fee1019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee15820700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 0x7fee101988e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fee08007d70 tx=0x7fee0800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.567+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08016020 con 0x7fee10107840 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.568+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee0800ad40 con 0x7fee10107840 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.568+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee10199330 con 0x7fee10107840 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.568+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08011620 con 0x7fee10107840 2024-06-02T23:09:19.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.568+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee101a55f0 con 0x7fee10107840 2024-06-02T23:09:19.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.569+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee10069180 con 0x7fee10107840 2024-06-02T23:09:19.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.570+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fee08019020 con 0x7fee10107840 2024-06-02T23:09:19.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.570+0000 7fee067fc700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedfc0a2d30 0x7fedfc0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:19.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.570+0000 7fee1501f700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedfc0a2d30 0x7fedfc0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:19.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.571+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee080ceff0 con 0x7fee10107840 2024-06-02T23:09:19.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.571+0000 7fee1501f700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedfc0a2d30 0x7fedfc0a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005fd0 tx=0x7fee0c005f00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:19.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.575+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee08020020 con 0x7fee10107840 2024-06-02T23:09:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:19 smithi123 ceph-mon[35977]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:19 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/12097230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-02T23:09:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.780+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fee1002dca0 con 0x7fee10107840 2024-06-02T23:09:19.780 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:19 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/12097230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-02T23:09:19.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.781+0000 7fee067fc700 1 -- 172.21.15.52:0/3365347541 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fee0809bf30 con 0x7fee10107840 2024-06-02T23:09:19.781 INFO:teuthology.orchestra.run.smithi052.stdout:197568495679 2024-06-02T23:09:19.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedfc0a2d30 msgr2=0x7fedfc0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedfc0a2d30 0x7fedfc0a51f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee0c005fd0 tx=0x7fee0c005f00 comp rx=0 tx=0).stop 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 msgr2=0x7fee101988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 0x7fee101988e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fee08007d70 tx=0x7fee0800a340 comp rx=0 tx=0).stop 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 shutdown_connections 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedfc0a2d30 0x7fedfc0a51f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10107840 0x7fee101988e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102740 0x7fee1019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 --2- 172.21.15.52:0/3365347541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101030f0 0x7fee1019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 >> 172.21.15.52:0/3365347541 conn(0x7fee100795f0 msgr2=0x7fee101005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 shutdown_connections 2024-06-02T23:09:19.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:19.783+0000 7fee17283700 1 -- 172.21.15.52:0/3365347541 wait complete. 2024-06-02T23:09:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:19 smithi174 ceph-mon[35994]: pgmap v354: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:19 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/12097230' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-06-02T23:09:20.033 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411407 got 77309411411 for osd.2 2024-06-02T23:09:20.034 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:20.635 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:20 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3365347541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-02T23:09:20.665 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495673 got 197568495679 for osd.6 2024-06-02T23:09:20.666 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:20.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:20 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3365347541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-02T23:09:20.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:20 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3365347541' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-06-02T23:09:21.207 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:21.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:21 smithi052 ceph-mon[32340]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:21.656 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757312 got 163208757316 for osd.5 2024-06-02T23:09:21.656 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:21 smithi123 ceph-mon[35977]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:21 smithi174 ceph-mon[35994]: pgmap v355: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:23.534 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:23 smithi052 ceph-mon[32340]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:23 smithi123 ceph-mon[35977]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:23 smithi174 ceph-mon[35994]: pgmap v356: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:25.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 -- 172.21.15.52:0/263079457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 msgr2=0x7f443410e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:25.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 --2- 172.21.15.52:0/263079457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f443410e280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f442c00b0d0 tx=0x7f442c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 -- 172.21.15.52:0/263079457 shutdown_connections 2024-06-02T23:09:25.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 --2- 172.21.15.52:0/263079457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f443410e280 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 --2- 172.21.15.52:0/263079457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44341015d0 0x7f44341019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 --2- 172.21.15.52:0/263079457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4434101f80 0x7f4434105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 -- 172.21.15.52:0/263079457 >> 172.21.15.52:0/263079457 conn(0x7f4434078400 msgr2=0x7f4434078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:25.733 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:25 smithi052 ceph-mon[32340]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 -- 172.21.15.52:0/263079457 shutdown_connections 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.732+0000 7f4439f0d700 1 -- 172.21.15.52:0/263079457 wait complete. 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 Processor -- start 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 -- start start 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44341015d0 0x7f443419e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434101f80 0x7f443419ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:25.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f4434198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4434111560 con 0x7f44341066b0 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44341113e0 con 0x7f4434101f80 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4439f0d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44341116e0 con 0x7f44341015d0 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4433fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f4434198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4433fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f4434198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35978/0 (socket says 172.21.15.52:35978) 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4433fff700 1 -- 172.21.15.52:0/721012181 learned_addr learned my addr 172.21.15.52:0/721012181 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f44337fe700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44341015d0 0x7f443419e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.733+0000 7f4432ffd700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434101f80 0x7f443419ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4433fff700 1 -- 172.21.15.52:0/721012181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44341015d0 msgr2=0x7f443419e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4433fff700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44341015d0 0x7f443419e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4433fff700 1 -- 172.21.15.52:0/721012181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434101f80 msgr2=0x7f443419ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4433fff700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434101f80 0x7f443419ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4433fff700 1 -- 172.21.15.52:0/721012181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4434199070 con 0x7f44341066b0 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4432ffd700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434101f80 0x7f443419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:25.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4433fff700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f4434198910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f442c0095c0 tx=0x7f442c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c01f070 con 0x7f44341066b0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f442c003c00 con 0x7f44341066b0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4434199300 con 0x7f44341066b0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f442c01b6a0 con 0x7f44341066b0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.734+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44341a5620 con 0x7f44341066b0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.735+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4434102ab0 con 0x7f44341066b0 2024-06-02T23:09:25.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.736+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f442c022070 con 0x7f44341066b0 2024-06-02T23:09:25.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.737+0000 7f4430ff9700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f441c0a2cb0 0x7f441c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:25.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.737+0000 7f44337fe700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f441c0a2cb0 0x7f441c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:25.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.737+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f442c00f020 con 0x7f44341066b0 2024-06-02T23:09:25.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.738+0000 7f44337fe700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f441c0a2cb0 0x7f441c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4434199a50 tx=0x7f442400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:25.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.742+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f442c029020 con 0x7f44341066b0 2024-06-02T23:09:25.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:25 smithi123 ceph-mon[35977]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:25 smithi174 ceph-mon[35994]: pgmap v357: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.949+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f443402dca0 con 0x7f44341066b0 2024-06-02T23:09:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.949+0000 7f4430ff9700 1 -- 172.21.15.52:0/721012181 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f442c09d2a0 con 0x7f44341066b0 2024-06-02T23:09:25.949 INFO:teuthology.orchestra.run.smithi052.stdout:107374182480 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f441c0a2cb0 msgr2=0x7f441c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f441c0a2cb0 0x7f441c0a5170 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4434199a50 tx=0x7f442400b410 comp rx=0 tx=0).stop 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 msgr2=0x7f4434198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f4434198910 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f442c0095c0 tx=0x7f442c007a20 comp rx=0 tx=0).stop 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 shutdown_connections 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f441c0a2cb0 0x7f441c0a5170 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44341066b0 0x7f4434198910 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44341015d0 0x7f443419e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 --2- 172.21.15.52:0/721012181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4434101f80 0x7f443419ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.951+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 >> 172.21.15.52:0/721012181 conn(0x7f4434078400 msgr2=0x7f443406ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:25.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.952+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 shutdown_connections 2024-06-02T23:09:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:25.952+0000 7f4439f0d700 1 -- 172.21.15.52:0/721012181 wait complete. 2024-06-02T23:09:26.634 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182476 got 107374182480 for osd.3 2024-06-02T23:09:26.635 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:26.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:26 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/721012181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-02T23:09:26.772 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:26 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/721012181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-02T23:09:26.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 -- 172.21.15.52:0/1357743687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8441085d0 msgr2=0x7fa8441089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:26.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 --2- 172.21.15.52:0/1357743687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8441085d0 0x7fa8441089b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa8340099c0 tx=0x7fa834009cd0 comp rx=0 tx=0).stop 2024-06-02T23:09:26.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 -- 172.21.15.52:0/1357743687 shutdown_connections 2024-06-02T23:09:26.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 --2- 172.21.15.52:0/1357743687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa844102ef0 0x7fa84410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 --2- 172.21.15.52:0/1357743687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8441085d0 0x7fa8441089b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 --2- 172.21.15.52:0/1357743687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844102530 0x7fa8441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 -- 172.21.15.52:0/1357743687 >> 172.21.15.52:0/1357743687 conn(0x7fa844079640 msgr2=0x7fa844079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 -- 172.21.15.52:0/1357743687 shutdown_connections 2024-06-02T23:09:26.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.882+0000 7fa849256700 1 -- 172.21.15.52:0/1357743687 wait complete. 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 Processor -- start 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 -- start start 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844102530 0x7fa84419ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 0x7fa84419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8441085d0 0x7fa84419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa844112740 con 0x7fa844102ef0 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441125c0 con 0x7fa8441085d0 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa849256700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8441128c0 con 0x7fa844102530 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa84259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 0x7fa84419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa84259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 0x7fa84419b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35992/0 (socket says 172.21.15.52:35992) 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa84259c700 1 -- 172.21.15.52:0/1160232619 learned_addr learned my addr 172.21.15.52:0/1160232619 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:26.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa842d9d700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844102530 0x7fa84419ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.883+0000 7fa84359e700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8441085d0 0x7fa84419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa84259c700 1 -- 172.21.15.52:0/1160232619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844102530 msgr2=0x7fa84419ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa84259c700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844102530 0x7fa84419ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa84259c700 1 -- 172.21.15.52:0/1160232619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8441085d0 msgr2=0x7fa84419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa84259c700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8441085d0 0x7fa84419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa84259c700 1 -- 172.21.15.52:0/1160232619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa84419fd70 con 0x7fa844102ef0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa84259c700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 0x7fa84419b2a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa83800b530 tx=0x7fa83800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838015020 con 0x7fa844102ef0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8441a0060 con 0x7fa844102ef0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa838011ce0 con 0x7fa844102ef0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa838010dd0 con 0x7fa844102ef0 2024-06-02T23:09:26.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.884+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8441a78e0 con 0x7fa844102ef0 2024-06-02T23:09:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.885+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa844069180 con 0x7fa844102ef0 2024-06-02T23:09:26.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.886+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7fa838018020 con 0x7fa844102ef0 2024-06-02T23:09:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.887+0000 7fa82bfff700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa82c0a2cb0 0x7fa82c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.887+0000 7fa842d9d700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa82c0a2cb0 0x7fa82c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.887+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8380ced50 con 0x7fa844102ef0 2024-06-02T23:09:26.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.888+0000 7fa842d9d700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa82c0a2cb0 0x7fa82c0a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa834005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:26.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:26.892+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa83801f020 con 0x7fa844102ef0 2024-06-02T23:09:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:26 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/721012181' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-06-02T23:09:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.099+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa8441a7b10 con 0x7fa844102ef0 2024-06-02T23:09:27.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.100+0000 7fa82bfff700 1 -- 172.21.15.52:0/1160232619 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa83809bc30 con 0x7fa844102ef0 2024-06-02T23:09:27.100 INFO:teuthology.orchestra.run.smithi052.stdout:55834574937 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa82c0a2cb0 msgr2=0x7fa82c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa82c0a2cb0 0x7fa82c0a5170 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa834000c00 tx=0x7fa834005fb0 comp rx=0 tx=0).stop 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 msgr2=0x7fa84419b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 0x7fa84419b2a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa83800b530 tx=0x7fa83800b840 comp rx=0 tx=0).stop 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 shutdown_connections 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa82c0a2cb0 0x7fa82c0a5170 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8441085d0 0x7fa84419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa844102530 0x7fa84419ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 --2- 172.21.15.52:0/1160232619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa844102ef0 0x7fa84419b2a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 >> 172.21.15.52:0/1160232619 conn(0x7fa844079640 msgr2=0x7fa84410d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:27.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 shutdown_connections 2024-06-02T23:09:27.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:27.102+0000 7fa849256700 1 -- 172.21.15.52:0/1160232619 wait complete. 2024-06-02T23:09:27.644 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:27 smithi052 ceph-mon[32340]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:27.644 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1160232619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-02T23:09:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:27 smithi123 ceph-mon[35977]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1160232619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-02T23:09:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:27 smithi174 ceph-mon[35994]: pgmap v358: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1160232619' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-06-02T23:09:28.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574937 for osd.1 2024-06-02T23:09:28.331 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:28.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 -- 172.21.15.52:0/3889931137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7248108500 msgr2=0x7f72481088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:28.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 --2- 172.21.15.52:0/3889931137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7248108500 0x7f72481088e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7238009990 tx=0x7f7238009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 -- 172.21.15.52:0/3889931137 shutdown_connections 2024-06-02T23:09:28.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 --2- 172.21.15.52:0/3889931137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806ab30 0x7f724810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 --2- 172.21.15.52:0/3889931137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7248108500 0x7f72481088e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 --2- 172.21.15.52:0/3889931137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f724806a170 0x7f724806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 -- 172.21.15.52:0/3889931137 >> 172.21.15.52:0/3889931137 conn(0x7f7248078340 msgr2=0x7f7248078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:28.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 -- 172.21.15.52:0/3889931137 shutdown_connections 2024-06-02T23:09:28.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 -- 172.21.15.52:0/3889931137 wait complete. 2024-06-02T23:09:28.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 Processor -- start 2024-06-02T23:09:28.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.521+0000 7f724e8ce700 1 -- start start 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724e8ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 0x7f724819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724e8ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 0x7f724819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724e8ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7248108500 0x7f7248198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724e8ce700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7248112630 con 0x7f724806ab30 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724e8ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72481124b0 con 0x7f724806a170 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724e8ce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72481127b0 con 0x7f7248108500 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 0x7f724819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f72477fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 0x7f724819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 0x7f724819e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57806/0 (socket says 172.21.15.52:57806) 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 -- 172.21.15.52:0/2146821765 learned_addr learned my addr 172.21.15.52:0/2146821765 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:28.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f72477fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 0x7f724819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36016/0 (socket says 172.21.15.52:36016) 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f724ce6b700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7248108500 0x7f7248198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 -- 172.21.15.52:0/2146821765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7248108500 msgr2=0x7f7248198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7248108500 0x7f7248198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 -- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 msgr2=0x7f724819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 0x7f724819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f7247fff700 1 -- 172.21.15.52:0/2146821765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72481990f0 con 0x7f724806a170 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.522+0000 7f72477fe700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 0x7f724819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f724ce6b700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7248108500 0x7f7248198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f7247fff700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 0x7f724819e7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7238000c00 tx=0x7f7238004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7238027070 con 0x7f724806a170 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7238022470 con 0x7f724806a170 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f723800b610 con 0x7f724806a170 2024-06-02T23:09:28.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7248199380 con 0x7f724806a170 2024-06-02T23:09:28.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.523+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72481a56a0 con 0x7f724806a170 2024-06-02T23:09:28.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.524+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f724810a9e0 con 0x7f724806a170 2024-06-02T23:09:28.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.525+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f723802a070 con 0x7f724806a170 2024-06-02T23:09:28.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.526+0000 7f72457fa700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72300a2d30 0x7f72300a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:28.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.526+0000 7f72477fe700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72300a2d30 0x7f72300a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:28.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.526+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f72380d00f0 con 0x7f724806a170 2024-06-02T23:09:28.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.526+0000 7f72477fe700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72300a2d30 0x7f72300a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7248078a60 tx=0x7f723c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:28.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.529+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7238013070 con 0x7f724806a170 2024-06-02T23:09:28.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:28 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:09:28.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.742+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f7248199ec0 con 0x7f724806a170 2024-06-02T23:09:28.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.742+0000 7f72457fa700 1 -- 172.21.15.52:0/2146821765 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f723809cfb0 con 0x7f724806a170 2024-06-02T23:09:28.743 INFO:teuthology.orchestra.run.smithi052.stdout:141733920843 2024-06-02T23:09:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72300a2d30 msgr2=0x7f72300a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72300a2d30 0x7f72300a51f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7248078a60 tx=0x7f723c009260 comp rx=0 tx=0).stop 2024-06-02T23:09:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 msgr2=0x7f724819e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 0x7f724819e7d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7238000c00 tx=0x7f7238004790 comp rx=0 tx=0).stop 2024-06-02T23:09:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 shutdown_connections 2024-06-02T23:09:28.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72300a2d30 0x7f72300a51f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7248108500 0x7f7248198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f724806a170 0x7f724819e7d0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 --2- 172.21.15.52:0/2146821765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f724806ab30 0x7f724819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 >> 172.21.15.52:0/2146821765 conn(0x7f7248078340 msgr2=0x7f7248100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 shutdown_connections 2024-06-02T23:09:28.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:28.745+0000 7f724e8ce700 1 -- 172.21.15.52:0/2146821765 wait complete. 2024-06-02T23:09:28.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:28 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:09:28.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:28 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:09:29.454 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920836 got 141733920843 for osd.4 2024-06-02T23:09:29.454 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 -- 172.21.15.52:0/238072871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 msgr2=0x7f8f8c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 --2- 172.21.15.52:0/238072871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 0x7f8f8c107140 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0099c0 tx=0x7f8f7c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 -- 172.21.15.52:0/238072871 shutdown_connections 2024-06-02T23:09:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 --2- 172.21.15.52:0/238072871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c107870 0x7f8f8c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 --2- 172.21.15.52:0/238072871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 --2- 172.21.15.52:0/238072871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 0x7f8f8c107140 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 -- 172.21.15.52:0/238072871 >> 172.21.15.52:0/238072871 conn(0x7f8f8c079520 msgr2=0x7f8f8c079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:29.693 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:29 smithi052 ceph-mon[32340]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:29.693 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:29 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2146821765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 -- 172.21.15.52:0/238072871 shutdown_connections 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.692+0000 7f8f935bc700 1 -- 172.21.15.52:0/238072871 wait complete. 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 Processor -- start 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 -- start start 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 0x7f8f8c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 0x7f8f8c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c112650 con 0x7f8f8c1027a0 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1124d0 con 0x7f8f8c103150 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f935bc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f8c1127d0 con 0x7f8f8c107870 2024-06-02T23:09:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.693+0000 7f8f91358700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 0x7f8f8c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91358700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36028/0 (socket says 172.21.15.52:36028) 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 0x7f8f8c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:50006/0 (socket says 172.21.15.52:50006) 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91358700 1 -- 172.21.15.52:0/1546014857 learned_addr learned my addr 172.21.15.52:0/1546014857 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 -- 172.21.15.52:0/1546014857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 msgr2=0x7f8f8c19b300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 0x7f8f8c19b300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 -- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 msgr2=0x7f8f8c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 -- 172.21.15.52:0/1546014857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f8c19fdd0 con 0x7f8f8c107870 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91358700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f91b59700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 0x7f8f8c19f690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400efc0 tx=0x7f8f8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f840092e0 con 0x7f8f8c107870 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8f84004bb0 con 0x7f8f8c107870 2024-06-02T23:09:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.694+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f84004280 con 0x7f8f8c107870 2024-06-02T23:09:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.695+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f8c1a0060 con 0x7f8f8c107870 2024-06-02T23:09:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.695+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f8c1a7940 con 0x7f8f8c107870 2024-06-02T23:09:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.696+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f8c069180 con 0x7f8f8c107870 2024-06-02T23:09:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.697+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f8f8401b030 con 0x7f8f8c107870 2024-06-02T23:09:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.697+0000 7f8f827fc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8f780a2d30 0x7f8f780a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.698+0000 7f8f91358700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8f780a2d30 0x7f8f780a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.698+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8f840cf110 con 0x7f8f8c107870 2024-06-02T23:09:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.698+0000 7f8f91358700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8f780a2d30 0x7f8f780a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8f88011590 tx=0x7f8f88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.702+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8f84022020 con 0x7f8f8c107870 2024-06-02T23:09:29.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:29 smithi123 ceph-mon[35977]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:29 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2146821765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-02T23:09:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:29 smithi174 ceph-mon[35994]: pgmap v359: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:29 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2146821765' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-06-02T23:09:29.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.929+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f8f8c1a7b10 con 0x7f8f8c107870 2024-06-02T23:09:29.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.929+0000 7f8f827fc700 1 -- 172.21.15.52:0/1546014857 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f8f84010040 con 0x7f8f8c107870 2024-06-02T23:09:29.930 INFO:teuthology.orchestra.run.smithi052.stdout:253403070518 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8f780a2d30 msgr2=0x7f8f780a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8f780a2d30 0x7f8f780a51f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8f88011590 tx=0x7f8f88009260 comp rx=0 tx=0).stop 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 msgr2=0x7f8f8c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 0x7f8f8c19f690 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8f8400efc0 tx=0x7f8f8400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 shutdown_connections 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8f780a2d30 0x7f8f780a51f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f8c107870 0x7f8f8c19f690 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8f8c1027a0 0x7f8f8c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 --2- 172.21.15.52:0/1546014857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f8c103150 0x7f8f8c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:29.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 >> 172.21.15.52:0/1546014857 conn(0x7f8f8c079520 msgr2=0x7f8f8c1089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:29.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 shutdown_connections 2024-06-02T23:09:29.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:29.932+0000 7f8f935bc700 1 -- 172.21.15.52:0/1546014857 wait complete. 2024-06-02T23:09:31.111 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:30 smithi052 ceph-mon[32340]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:31.111 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.111 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:30 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.112 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:30 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1546014857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-02T23:09:31.138 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070513 got 253403070518 for osd.8 2024-06-02T23:09:31.138 DEBUG:teuthology.parallel:result is None 2024-06-02T23:09:31.138 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-02T23:09:31.139 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:09:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:30 smithi174 ceph-mon[35994]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:30 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:30 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1546014857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-02T23:09:31.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:30 smithi123 ceph-mon[35977]: pgmap v360: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:30 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:30 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1546014857' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-06-02T23:09:31.412 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.074+0000 7f3623681700 1 -- 172.21.15.52:0/572528915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c06b520 msgr2=0x7f361c10f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.074+0000 7f3623681700 1 --2- 172.21.15.52:0/572528915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c06b520 0x7f361c10f370 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f361400b0a0 tx=0x7f361400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 -- 172.21.15.52:0/572528915 shutdown_connections 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 --2- 172.21.15.52:0/572528915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c06b520 0x7f361c10f370 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 --2- 172.21.15.52:0/572528915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c108670 0x7f361c108a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 --2- 172.21.15.52:0/572528915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361c06ab60 0x7f361c06afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 -- 172.21.15.52:0/572528915 >> 172.21.15.52:0/572528915 conn(0x7f361c078340 msgr2=0x7f361c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 -- 172.21.15.52:0/572528915 shutdown_connections 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 -- 172.21.15.52:0/572528915 wait complete. 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 Processor -- start 2024-06-02T23:09:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.075+0000 7f3623681700 1 -- start start 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3623681700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c06ab60 0x7f361c101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3623681700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361c06b520 0x7f361c101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3623681700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 0x7f361c1022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3623681700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f361c112630 con 0x7f361c108670 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3623681700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1124b0 con 0x7f361c06b520 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3623681700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f361c1127b0 con 0x7f361c06ab60 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3621c1e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 0x7f361c1022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3621c1e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 0x7f361c1022c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36050/0 (socket says 172.21.15.52:36050) 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3621c1e700 1 -- 172.21.15.52:0/1135895315 learned_addr learned my addr 172.21.15.52:0/1135895315 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f362141d700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c06ab60 0x7f361c101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f3620c1c700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361c06b520 0x7f361c101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f362141d700 1 -- 172.21.15.52:0/1135895315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361c06b520 msgr2=0x7f361c101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.076+0000 7f362141d700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361c06b520 0x7f361c101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f362141d700 1 -- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 msgr2=0x7f361c1022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f362141d700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 0x7f361c1022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f362141d700 1 -- 172.21.15.52:0/1135895315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f361c1a6fd0 con 0x7f361c06ab60 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f3621c1e700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 0x7f361c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f362141d700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c06ab60 0x7f361c101840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f361800d750 tx=0x7f361800da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3618016070 con 0x7f361c06ab60 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3618005020 con 0x7f361c06ab60 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f361c1a72c0 con 0x7f361c06ab60 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f361c1a77b0 con 0x7f361c06ab60 2024-06-02T23:09:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.077+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f361800f9f0 con 0x7f361c06ab60 2024-06-02T23:09:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.078+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f361c103be0 con 0x7f361c06ab60 2024-06-02T23:09:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.080+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f36180051d0 con 0x7f361c06ab60 2024-06-02T23:09:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.081+0000 7f36127fc700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36080a2cb0 0x7f36080a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.081+0000 7f3620c1c700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36080a2cb0 0x7f36080a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.081+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36180ceff0 con 0x7f361c06ab60 2024-06-02T23:09:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.082+0000 7f3620c1c700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36080a2cb0 0x7f36080a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f361c078860 tx=0x7f360c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:33.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.084+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f361809be20 con 0x7f361c06ab60 2024-06-02T23:09:33.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.333+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f361c0628d0 con 0x7f36080a2cb0 2024-06-02T23:09:33.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.341+0000 7f36127fc700 1 -- 172.21.15.52:0/1135895315 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258951 (secure 0 0 0) 0x7f361c0628d0 con 0x7f36080a2cb0 2024-06-02T23:09:33.344 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:33.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36080a2cb0 msgr2=0x7f36080a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36080a2cb0 0x7f36080a5170 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f361c078860 tx=0x7f360c00a300 comp rx=0 tx=0).stop 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c06ab60 msgr2=0x7f361c101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c06ab60 0x7f361c101840 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f361800d750 tx=0x7f361800da60 comp rx=0 tx=0).stop 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 shutdown_connections 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36080a2cb0 0x7f36080a5170 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f361c108670 0x7f361c1022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f361c06ab60 0x7f361c101840 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 --2- 172.21.15.52:0/1135895315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361c06b520 0x7f361c101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 >> 172.21.15.52:0/1135895315 conn(0x7f361c078340 msgr2=0x7f361c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 shutdown_connections 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:33.344+0000 7f3623681700 1 -- 172.21.15.52:0/1135895315 wait complete. 2024-06-02T23:09:33.347 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:09:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:33 smithi174 ceph-mon[35994]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:09:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:09:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:33 smithi052 ceph-mon[32340]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:09:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:09:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:34.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:33 smithi123 ceph-mon[35977]: pgmap v361: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:09:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:09:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:09:34.038 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-06-02T23:09:31.790224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659239,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1129,"num_read_kb":681,"num_write":428,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369652,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572172,"statfs":{"total":1151940427776,"available":1151561904128,"internally_reserved":0,"allocated":12435456,"data_stored":7728453,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":110,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002838"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331749+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:08:11.331749+0000","last_peered":"2024-06-02T23:08:11.331749+0000","last_clean":"2024-06-02T23:08:11.331749+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:08:11.331749+0000","last_undegraded":"2024-06-02T23:08:11.331749+0000","last_fullsized":"2024-06-02T23:08:11.331749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331436+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:08:11.331436+0000","last_peered":"2024-06-02T23:08:11.331436+0000","last_clean":"2024-06-02T23:08:11.331436+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:08:11.331436+0000","last_undegraded":"2024-06-02T23:08:11.331436+0000","last_fullsized":"2024-06-02T23:08:11.331436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331379+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:08:11.331379+0000","last_peered":"2024-06-02T23:08:11.331379+0000","last_clean":"2024-06-02T23:08:11.331379+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:08:11.331379+0000","last_undegraded":"2024-06-02T23:08:11.331379+0000","last_fullsized":"2024-06-02T23:08:11.331379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450534+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:08:11.450534+0000","last_peered":"2024-06-02T23:08:11.450534+0000","last_clean":"2024-06-02T23:08:11.450534+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:08:11.450534+0000","last_undegraded":"2024-06-02T23:08:11.450534+0000","last_fullsized":"2024-06-02T23:08:11.450534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334614+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:08:11.334614+0000","last_peered":"2024-06-02T23:08:11.334614+0000","last_clean":"2024-06-02T23:08:11.334614+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:08:11.334614+0000","last_undegraded":"2024-06-02T23:08:11.334614+0000","last_fullsized":"2024-06-02T23:08:11.334614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449583+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:08:11.449583+0000","last_peered":"2024-06-02T23:08:11.449583+0000","last_clean":"2024-06-02T23:08:11.449583+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:08:11.449583+0000","last_undegraded":"2024-06-02T23:08:11.449583+0000","last_fullsized":"2024-06-02T23:08:11.449583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332875+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:08:11.332875+0000","last_peered":"2024-06-02T23:08:11.332875+0000","last_clean":"2024-06-02T23:08:11.332875+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:08:11.332875+0000","last_undegraded":"2024-06-02T23:08:11.332875+0000","last_fullsized":"2024-06-02T23:08:11.332875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748655+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:08:11.748655+0000","last_peered":"2024-06-02T23:08:11.748655+0000","last_clean":"2024-06-02T23:08:11.748655+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:08:11.748655+0000","last_undegraded":"2024-06-02T23:08:11.748655+0000","last_fullsized":"2024-06-02T23:08:11.748655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334826+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:08:11.334826+0000","last_peered":"2024-06-02T23:08:11.334826+0000","last_clean":"2024-06-02T23:08:11.334826+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:08:11.334826+0000","last_undegraded":"2024-06-02T23:08:11.334826+0000","last_fullsized":"2024-06-02T23:08:11.334826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449830+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:08:11.449830+0000","last_peered":"2024-06-02T23:08:11.449830+0000","last_clean":"2024-06-02T23:08:11.449830+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:08:11.449830+0000","last_undegraded":"2024-06-02T23:08:11.449830+0000","last_fullsized":"2024-06-02T23:08:11.449830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332762+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:08:11.332762+0000","last_peered":"2024-06-02T23:08:11.332762+0000","last_clean":"2024-06-02T23:08:11.332762+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:08:11.332762+0000","last_undegraded":"2024-06-02T23:08:11.332762+0000","last_fullsized":"2024-06-02T23:08:11.332762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450303+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:08:11.450303+0000","last_peered":"2024-06-02T23:08:11.450303+0000","last_clean":"2024-06-02T23:08:11.450303+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:08:11.450303+0000","last_undegraded":"2024-06-02T23:08:11.450303+0000","last_fullsized":"2024-06-02T23:08:11.450303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333334+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:08:11.333334+0000","last_peered":"2024-06-02T23:08:11.333334+0000","last_clean":"2024-06-02T23:08:11.333334+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:08:11.333334+0000","last_undegraded":"2024-06-02T23:08:11.333334+0000","last_fullsized":"2024-06-02T23:08:11.333334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330935+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:08:11.330935+0000","last_peered":"2024-06-02T23:08:11.330935+0000","last_clean":"2024-06-02T23:08:11.330935+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:08:11.330935+0000","last_undegraded":"2024-06-02T23:08:11.330935+0000","last_fullsized":"2024-06-02T23:08:11.330935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330848+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:08:11.330848+0000","last_peered":"2024-06-02T23:08:11.330848+0000","last_clean":"2024-06-02T23:08:11.330848+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:08:11.330848+0000","last_undegraded":"2024-06-02T23:08:11.330848+0000","last_fullsized":"2024-06-02T23:08:11.330848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331976+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:08:11.331976+0000","last_peered":"2024-06-02T23:08:11.331976+0000","last_clean":"2024-06-02T23:08:11.331976+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:08:11.331976+0000","last_undegraded":"2024-06-02T23:08:11.331976+0000","last_fullsized":"2024-06-02T23:08:11.331976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328445+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:08:11.328445+0000","last_peered":"2024-06-02T23:08:11.328445+0000","last_clean":"2024-06-02T23:08:11.328445+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:08:11.328445+0000","last_undegraded":"2024-06-02T23:08:11.328445+0000","last_fullsized":"2024-06-02T23:08:11.328445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333538+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:08:11.333538+0000","last_peered":"2024-06-02T23:08:11.333538+0000","last_clean":"2024-06-02T23:08:11.333538+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:08:11.333538+0000","last_undegraded":"2024-06-02T23:08:11.333538+0000","last_fullsized":"2024-06-02T23:08:11.333538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":135,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:24.994004+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:09:24.994004+0000","last_peered":"2024-06-02T23:09:24.994004+0000","last_clean":"2024-06-02T23:09:24.994004+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:09:24.994004+0000","last_undegraded":"2024-06-02T23:09:24.994004+0000","last_fullsized":"2024-06-02T23:09:24.994004+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748992+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:08:11.748992+0000","last_peered":"2024-06-02T23:08:11.748992+0000","last_clean":"2024-06-02T23:08:11.748992+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:08:11.748992+0000","last_undegraded":"2024-06-02T23:08:11.748992+0000","last_fullsized":"2024-06-02T23:08:11.748992+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331872+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:08:11.331872+0000","last_peered":"2024-06-02T23:08:11.331872+0000","last_clean":"2024-06-02T23:08:11.331872+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:08:11.331872+0000","last_undegraded":"2024-06-02T23:08:11.331872+0000","last_fullsized":"2024-06-02T23:08:11.331872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.365621+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:11.365621+0000","last_peered":"2024-06-02T23:08:11.365621+0000","last_clean":"2024-06-02T23:08:11.365621+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:11.365621+0000","last_undegraded":"2024-06-02T23:08:11.365621+0000","last_fullsized":"2024-06-02T23:08:11.365621+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334580+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:08:11.334580+0000","last_peered":"2024-06-02T23:08:11.334580+0000","last_clean":"2024-06-02T23:08:11.334580+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:08:11.334580+0000","last_undegraded":"2024-06-02T23:08:11.334580+0000","last_fullsized":"2024-06-02T23:08:11.334580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335715+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335715+0000","last_peered":"2024-06-02T23:08:11.335715+0000","last_clean":"2024-06-02T23:08:11.335715+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335715+0000","last_undegraded":"2024-06-02T23:08:11.335715+0000","last_fullsized":"2024-06-02T23:08:11.335715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329756+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:08:11.329756+0000","last_peered":"2024-06-02T23:08:11.329756+0000","last_clean":"2024-06-02T23:08:11.329756+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:08:11.329756+0000","last_undegraded":"2024-06-02T23:08:11.329756+0000","last_fullsized":"2024-06-02T23:08:11.329756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":142,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:27.562889+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:09:27.562889+0000","last_peered":"2024-06-02T23:09:27.562889+0000","last_clean":"2024-06-02T23:09:27.562889+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:09:27.562889+0000","last_undegraded":"2024-06-02T23:09:27.562889+0000","last_fullsized":"2024-06-02T23:09:27.562889+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334793+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:08:11.334793+0000","last_peered":"2024-06-02T23:08:11.334793+0000","last_clean":"2024-06-02T23:08:11.334793+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:08:11.334793+0000","last_undegraded":"2024-06-02T23:08:11.334793+0000","last_fullsized":"2024-06-02T23:08:11.334793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449532+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449532+0000","last_peered":"2024-06-02T23:08:11.449532+0000","last_clean":"2024-06-02T23:08:11.449532+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449532+0000","last_undegraded":"2024-06-02T23:08:11.449532+0000","last_fullsized":"2024-06-02T23:08:11.449532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329554+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:08:11.329554+0000","last_peered":"2024-06-02T23:08:11.329554+0000","last_clean":"2024-06-02T23:08:11.329554+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:08:11.329554+0000","last_undegraded":"2024-06-02T23:08:11.329554+0000","last_fullsized":"2024-06-02T23:08:11.329554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332891+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332891+0000","last_peered":"2024-06-02T23:08:11.332891+0000","last_clean":"2024-06-02T23:08:11.332891+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332891+0000","last_undegraded":"2024-06-02T23:08:11.332891+0000","last_fullsized":"2024-06-02T23:08:11.332891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748738+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:08:11.748738+0000","last_peered":"2024-06-02T23:08:11.748738+0000","last_clean":"2024-06-02T23:08:11.748738+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:08:11.748738+0000","last_undegraded":"2024-06-02T23:08:11.748738+0000","last_fullsized":"2024-06-02T23:08:11.748738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329228+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329228+0000","last_peered":"2024-06-02T23:08:11.329228+0000","last_clean":"2024-06-02T23:08:11.329228+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329228+0000","last_undegraded":"2024-06-02T23:08:11.329228+0000","last_fullsized":"2024-06-02T23:08:11.329228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328243+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:08:11.328243+0000","last_peered":"2024-06-02T23:08:11.328243+0000","last_clean":"2024-06-02T23:08:11.328243+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:08:11.328243+0000","last_undegraded":"2024-06-02T23:08:11.328243+0000","last_fullsized":"2024-06-02T23:08:11.328243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":221,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:26.560796+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:09:26.560796+0000","last_peered":"2024-06-02T23:09:26.560796+0000","last_clean":"2024-06-02T23:09:26.560796+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:09:26.560796+0000","last_undegraded":"2024-06-02T23:09:26.560796+0000","last_fullsized":"2024-06-02T23:09:26.560796+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":191,"num_read_kb":191,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334817+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:08:11.334817+0000","last_peered":"2024-06-02T23:08:11.334817+0000","last_clean":"2024-06-02T23:08:11.334817+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:08:11.334817+0000","last_undegraded":"2024-06-02T23:08:11.334817+0000","last_fullsized":"2024-06-02T23:08:11.334817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332982+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.332982+0000","last_peered":"2024-06-02T23:08:11.332982+0000","last_clean":"2024-06-02T23:08:11.332982+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.332982+0000","last_undegraded":"2024-06-02T23:08:11.332982+0000","last_fullsized":"2024-06-02T23:08:11.332982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331775+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:08:11.331775+0000","last_peered":"2024-06-02T23:08:11.331775+0000","last_clean":"2024-06-02T23:08:11.331775+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:08:11.331775+0000","last_undegraded":"2024-06-02T23:08:11.331775+0000","last_fullsized":"2024-06-02T23:08:11.331775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329928+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.329928+0000","last_peered":"2024-06-02T23:08:11.329928+0000","last_clean":"2024-06-02T23:08:11.329928+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.329928+0000","last_undegraded":"2024-06-02T23:08:11.329928+0000","last_fullsized":"2024-06-02T23:08:11.329928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334827+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:08:11.334827+0000","last_peered":"2024-06-02T23:08:11.334827+0000","last_clean":"2024-06-02T23:08:11.334827+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:08:11.334827+0000","last_undegraded":"2024-06-02T23:08:11.334827+0000","last_fullsized":"2024-06-02T23:08:11.334827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":256,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:26.560804+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:09:26.560804+0000","last_peered":"2024-06-02T23:09:26.560804+0000","last_clean":"2024-06-02T23:09:26.560804+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:09:26.560804+0000","last_undegraded":"2024-06-02T23:09:26.560804+0000","last_fullsized":"2024-06-02T23:09:26.560804+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61825,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":158,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335764+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:08:11.335764+0000","last_peered":"2024-06-02T23:08:11.335764+0000","last_clean":"2024-06-02T23:08:11.335764+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:08:11.335764+0000","last_undegraded":"2024-06-02T23:08:11.335764+0000","last_fullsized":"2024-06-02T23:08:11.335764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332546+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332546+0000","last_peered":"2024-06-02T23:08:11.332546+0000","last_clean":"2024-06-02T23:08:11.332546+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332546+0000","last_undegraded":"2024-06-02T23:08:11.332546+0000","last_fullsized":"2024-06-02T23:08:11.332546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333123+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:08:11.333123+0000","last_peered":"2024-06-02T23:08:11.333123+0000","last_clean":"2024-06-02T23:08:11.333123+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:08:11.333123+0000","last_undegraded":"2024-06-02T23:08:11.333123+0000","last_fullsized":"2024-06-02T23:08:11.333123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329400+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329400+0000","last_peered":"2024-06-02T23:08:11.329400+0000","last_clean":"2024-06-02T23:08:11.329400+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329400+0000","last_undegraded":"2024-06-02T23:08:11.329400+0000","last_fullsized":"2024-06-02T23:08:11.329400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332914+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:08:11.332914+0000","last_peered":"2024-06-02T23:08:11.332914+0000","last_clean":"2024-06-02T23:08:11.332914+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:08:11.332914+0000","last_undegraded":"2024-06-02T23:08:11.332914+0000","last_fullsized":"2024-06-02T23:08:11.332914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":132,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:29.020121+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:09:29.020121+0000","last_peered":"2024-06-02T23:09:29.020121+0000","last_clean":"2024-06-02T23:09:29.020121+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:09:29.020121+0000","last_undegraded":"2024-06-02T23:09:29.020121+0000","last_fullsized":"2024-06-02T23:09:29.020121+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332579+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:08:11.332579+0000","last_peered":"2024-06-02T23:08:11.332579+0000","last_clean":"2024-06-02T23:08:11.332579+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:08:11.332579+0000","last_undegraded":"2024-06-02T23:08:11.332579+0000","last_fullsized":"2024-06-02T23:08:11.332579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.348940+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:11.348940+0000","last_peered":"2024-06-02T23:08:11.348940+0000","last_clean":"2024-06-02T23:08:11.348940+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:11.348940+0000","last_undegraded":"2024-06-02T23:08:11.348940+0000","last_fullsized":"2024-06-02T23:08:11.348940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329323+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:08:11.329323+0000","last_peered":"2024-06-02T23:08:11.329323+0000","last_clean":"2024-06-02T23:08:11.329323+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:08:11.329323+0000","last_undegraded":"2024-06-02T23:08:11.329323+0000","last_fullsized":"2024-06-02T23:08:11.329323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335313+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:08:11.335313+0000","last_peered":"2024-06-02T23:08:11.335313+0000","last_clean":"2024-06-02T23:08:11.335313+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:08:11.335313+0000","last_undegraded":"2024-06-02T23:08:11.335313+0000","last_fullsized":"2024-06-02T23:08:11.335313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331466+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:08:11.331466+0000","last_peered":"2024-06-02T23:08:11.331466+0000","last_clean":"2024-06-02T23:08:11.331466+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:08:11.331466+0000","last_undegraded":"2024-06-02T23:08:11.331466+0000","last_fullsized":"2024-06-02T23:08:11.331466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328487+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328487+0000","last_peered":"2024-06-02T23:08:11.328487+0000","last_clean":"2024-06-02T23:08:11.328487+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328487+0000","last_undegraded":"2024-06-02T23:08:11.328487+0000","last_fullsized":"2024-06-02T23:08:11.328487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449500+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:08:11.449500+0000","last_peered":"2024-06-02T23:08:11.449500+0000","last_clean":"2024-06-02T23:08:11.449500+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:08:11.449500+0000","last_undegraded":"2024-06-02T23:08:11.449500+0000","last_fullsized":"2024-06-02T23:08:11.449500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":131,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:28.999956+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:09:28.999956+0000","last_peered":"2024-06-02T23:09:28.999956+0000","last_clean":"2024-06-02T23:09:28.999956+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:09:28.999956+0000","last_undegraded":"2024-06-02T23:09:28.999956+0000","last_fullsized":"2024-06-02T23:09:28.999956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449616+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:08:11.449616+0000","last_peered":"2024-06-02T23:08:11.449616+0000","last_clean":"2024-06-02T23:08:11.449616+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:08:11.449616+0000","last_undegraded":"2024-06-02T23:08:11.449616+0000","last_fullsized":"2024-06-02T23:08:11.449616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329755+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:08:11.329755+0000","last_peered":"2024-06-02T23:08:11.329755+0000","last_clean":"2024-06-02T23:08:11.329755+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:08:11.329755+0000","last_undegraded":"2024-06-02T23:08:11.329755+0000","last_fullsized":"2024-06-02T23:08:11.329755+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":130,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:28.018120+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:09:28.018120+0000","last_peered":"2024-06-02T23:09:28.018120+0000","last_clean":"2024-06-02T23:09:28.018120+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:09:28.018120+0000","last_undegraded":"2024-06-02T23:09:28.018120+0000","last_fullsized":"2024-06-02T23:09:28.018120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":99,"num_read_kb":99,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748857+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:08:11.748857+0000","last_peered":"2024-06-02T23:08:11.748857+0000","last_clean":"2024-06-02T23:08:11.748857+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:08:11.748857+0000","last_undegraded":"2024-06-02T23:08:11.748857+0000","last_fullsized":"2024-06-02T23:08:11.748857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450367+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450367+0000","last_peered":"2024-06-02T23:08:11.450367+0000","last_clean":"2024-06-02T23:08:11.450367+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450367+0000","last_undegraded":"2024-06-02T23:08:11.450367+0000","last_fullsized":"2024-06-02T23:08:11.450367+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333070+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:11.333070+0000","last_peered":"2024-06-02T23:08:11.333070+0000","last_clean":"2024-06-02T23:08:11.333070+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:11.333070+0000","last_undegraded":"2024-06-02T23:08:11.333070+0000","last_fullsized":"2024-06-02T23:08:11.333070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332742+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:08:11.332742+0000","last_peered":"2024-06-02T23:08:11.332742+0000","last_clean":"2024-06-02T23:08:11.332742+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:08:11.332742+0000","last_undegraded":"2024-06-02T23:08:11.332742+0000","last_fullsized":"2024-06-02T23:08:11.332742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335862+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335862+0000","last_peered":"2024-06-02T23:08:11.335862+0000","last_clean":"2024-06-02T23:08:11.335862+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335862+0000","last_undegraded":"2024-06-02T23:08:11.335862+0000","last_fullsized":"2024-06-02T23:08:11.335862+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333307+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:08:11.333307+0000","last_peered":"2024-06-02T23:08:11.333307+0000","last_clean":"2024-06-02T23:08:11.333307+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:08:11.333307+0000","last_undegraded":"2024-06-02T23:08:11.333307+0000","last_fullsized":"2024-06-02T23:08:11.333307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.213929+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:08:14.213929+0000","last_peered":"2024-06-02T23:08:14.213929+0000","last_clean":"2024-06-02T23:08:14.213929+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:08:14.213929+0000","last_undegraded":"2024-06-02T23:08:14.213929+0000","last_fullsized":"2024-06-02T23:08:14.213929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749041+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:08:11.749041+0000","last_peered":"2024-06-02T23:08:11.749041+0000","last_clean":"2024-06-02T23:08:11.749041+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:08:11.749041+0000","last_undegraded":"2024-06-02T23:08:11.749041+0000","last_fullsized":"2024-06-02T23:08:11.749041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450403+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450403+0000","last_peered":"2024-06-02T23:08:11.450403+0000","last_clean":"2024-06-02T23:08:11.450403+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450403+0000","last_undegraded":"2024-06-02T23:08:11.450403+0000","last_fullsized":"2024-06-02T23:08:11.450403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.750091+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:08:11.750091+0000","last_peered":"2024-06-02T23:08:11.750091+0000","last_clean":"2024-06-02T23:08:11.750091+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:08:11.750091+0000","last_undegraded":"2024-06-02T23:08:11.750091+0000","last_fullsized":"2024-06-02T23:08:11.750091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334625+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:08:11.334625+0000","last_peered":"2024-06-02T23:08:11.334625+0000","last_clean":"2024-06-02T23:08:11.334625+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:08:11.334625+0000","last_undegraded":"2024-06-02T23:08:11.334625+0000","last_fullsized":"2024-06-02T23:08:11.334625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748865+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748865+0000","last_peered":"2024-06-02T23:08:11.748865+0000","last_clean":"2024-06-02T23:08:11.748865+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748865+0000","last_undegraded":"2024-06-02T23:08:11.748865+0000","last_fullsized":"2024-06-02T23:08:11.748865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332297+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:08:11.332297+0000","last_peered":"2024-06-02T23:08:11.332297+0000","last_clean":"2024-06-02T23:08:11.332297+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:08:11.332297+0000","last_undegraded":"2024-06-02T23:08:11.332297+0000","last_fullsized":"2024-06-02T23:08:11.332297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332588+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:08:11.332588+0000","last_peered":"2024-06-02T23:08:11.332588+0000","last_clean":"2024-06-02T23:08:11.332588+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:08:11.332588+0000","last_undegraded":"2024-06-02T23:08:11.332588+0000","last_fullsized":"2024-06-02T23:08:11.332588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748997+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:08:11.748997+0000","last_peered":"2024-06-02T23:08:11.748997+0000","last_clean":"2024-06-02T23:08:11.748997+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:08:11.748997+0000","last_undegraded":"2024-06-02T23:08:11.748997+0000","last_fullsized":"2024-06-02T23:08:11.748997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334746+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334746+0000","last_peered":"2024-06-02T23:08:11.334746+0000","last_clean":"2024-06-02T23:08:11.334746+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334746+0000","last_undegraded":"2024-06-02T23:08:11.334746+0000","last_fullsized":"2024-06-02T23:08:11.334746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333187+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:08:11.333187+0000","last_peered":"2024-06-02T23:08:11.333187+0000","last_clean":"2024-06-02T23:08:11.333187+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:08:11.333187+0000","last_undegraded":"2024-06-02T23:08:11.333187+0000","last_fullsized":"2024-06-02T23:08:11.333187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":133,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:28.054522+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:09:28.054522+0000","last_peered":"2024-06-02T23:09:28.054522+0000","last_clean":"2024-06-02T23:09:28.054522+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:09:28.054522+0000","last_undegraded":"2024-06-02T23:09:28.054522+0000","last_fullsized":"2024-06-02T23:09:28.054522+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332904+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:08:11.332904+0000","last_peered":"2024-06-02T23:08:11.332904+0000","last_clean":"2024-06-02T23:08:11.332904+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:08:11.332904+0000","last_undegraded":"2024-06-02T23:08:11.332904+0000","last_fullsized":"2024-06-02T23:08:11.332904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330120+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330120+0000","last_peered":"2024-06-02T23:08:11.330120+0000","last_clean":"2024-06-02T23:08:11.330120+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330120+0000","last_undegraded":"2024-06-02T23:08:11.330120+0000","last_fullsized":"2024-06-02T23:08:11.330120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332737+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:08:11.332737+0000","last_peered":"2024-06-02T23:08:11.332737+0000","last_clean":"2024-06-02T23:08:11.332737+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:08:11.332737+0000","last_undegraded":"2024-06-02T23:08:11.332737+0000","last_fullsized":"2024-06-02T23:08:11.332737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333475+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333475+0000","last_peered":"2024-06-02T23:08:11.333475+0000","last_clean":"2024-06-02T23:08:11.333475+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333475+0000","last_undegraded":"2024-06-02T23:08:11.333475+0000","last_fullsized":"2024-06-02T23:08:11.333475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332416+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:08:11.332416+0000","last_peered":"2024-06-02T23:08:11.332416+0000","last_clean":"2024-06-02T23:08:11.332416+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:08:11.332416+0000","last_undegraded":"2024-06-02T23:08:11.332416+0000","last_fullsized":"2024-06-02T23:08:11.332416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332160+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:08:11.332160+0000","last_peered":"2024-06-02T23:08:11.332160+0000","last_clean":"2024-06-02T23:08:11.332160+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:08:11.332160+0000","last_undegraded":"2024-06-02T23:08:11.332160+0000","last_fullsized":"2024-06-02T23:08:11.332160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332456+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:08:11.332456+0000","last_peered":"2024-06-02T23:08:11.332456+0000","last_clean":"2024-06-02T23:08:11.332456+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:08:11.332456+0000","last_undegraded":"2024-06-02T23:08:11.332456+0000","last_fullsized":"2024-06-02T23:08:11.332456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331147+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:08:11.331147+0000","last_peered":"2024-06-02T23:08:11.331147+0000","last_clean":"2024-06-02T23:08:11.331147+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:08:11.331147+0000","last_undegraded":"2024-06-02T23:08:11.331147+0000","last_fullsized":"2024-06-02T23:08:11.331147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335176+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:08:11.335176+0000","last_peered":"2024-06-02T23:08:11.335176+0000","last_clean":"2024-06-02T23:08:11.335176+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:08:11.335176+0000","last_undegraded":"2024-06-02T23:08:11.335176+0000","last_fullsized":"2024-06-02T23:08:11.335176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337292+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:08:11.337292+0000","last_peered":"2024-06-02T23:08:11.337292+0000","last_clean":"2024-06-02T23:08:11.337292+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:08:11.337292+0000","last_undegraded":"2024-06-02T23:08:11.337292+0000","last_fullsized":"2024-06-02T23:08:11.337292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333612+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:08:11.333612+0000","last_peered":"2024-06-02T23:08:11.333612+0000","last_clean":"2024-06-02T23:08:11.333612+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:08:11.333612+0000","last_undegraded":"2024-06-02T23:08:11.333612+0000","last_fullsized":"2024-06-02T23:08:11.333612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749044+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:08:11.749044+0000","last_peered":"2024-06-02T23:08:11.749044+0000","last_clean":"2024-06-02T23:08:11.749044+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:08:11.749044+0000","last_undegraded":"2024-06-02T23:08:11.749044+0000","last_fullsized":"2024-06-02T23:08:11.749044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329174+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:08:11.329174+0000","last_peered":"2024-06-02T23:08:11.329174+0000","last_clean":"2024-06-02T23:08:11.329174+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:08:11.329174+0000","last_undegraded":"2024-06-02T23:08:11.329174+0000","last_fullsized":"2024-06-02T23:08:11.329174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748682+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:08:11.748682+0000","last_peered":"2024-06-02T23:08:11.748682+0000","last_clean":"2024-06-02T23:08:11.748682+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:08:11.748682+0000","last_undegraded":"2024-06-02T23:08:11.748682+0000","last_fullsized":"2024-06-02T23:08:11.748682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334859+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:08:11.334859+0000","last_peered":"2024-06-02T23:08:11.334859+0000","last_clean":"2024-06-02T23:08:11.334859+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:08:11.334859+0000","last_undegraded":"2024-06-02T23:08:11.334859+0000","last_fullsized":"2024-06-02T23:08:11.334859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333208+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:08:11.333208+0000","last_peered":"2024-06-02T23:08:11.333208+0000","last_clean":"2024-06-02T23:08:11.333208+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:08:11.333208+0000","last_undegraded":"2024-06-02T23:08:11.333208+0000","last_fullsized":"2024-06-02T23:08:11.333208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337520+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:08:11.337520+0000","last_peered":"2024-06-02T23:08:11.337520+0000","last_clean":"2024-06-02T23:08:11.337520+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:08:11.337520+0000","last_undegraded":"2024-06-02T23:08:11.337520+0000","last_fullsized":"2024-06-02T23:08:11.337520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68637,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1003,"num_read_kb":572,"num_write":245,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383721,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645357,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1133274,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808882,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1072855,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495682,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":542805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":481796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":542793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":480550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757319,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072533,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:09:34.043 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:09:34.301 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:34.698 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:34 smithi052 ceph-mon[32340]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:09:34.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:34 smithi174 ceph-mon[35994]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:09:35.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:34 smithi123 ceph-mon[35977]: from='client.24691 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:09:35.815 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:35 smithi052 ceph-mon[32340]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:35 smithi174 ceph-mon[35994]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:35 smithi123 ceph-mon[35977]: pgmap v362: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 -- 172.21.15.52:0/3032160886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c107840 msgr2=0x7f568c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 --2- 172.21.15.52:0/3032160886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c107840 0x7f568c10f3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f568400b0d0 tx=0x7f568400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 -- 172.21.15.52:0/3032160886 shutdown_connections 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 --2- 172.21.15.52:0/3032160886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c107840 0x7f568c10f3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 --2- 172.21.15.52:0/3032160886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c102740 0x7f568c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 --2- 172.21.15.52:0/3032160886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f568c1030f0 0x7f568c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 -- 172.21.15.52:0/3032160886 >> 172.21.15.52:0/3032160886 conn(0x7f568c0795f0 msgr2=0x7f568c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:36.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 -- 172.21.15.52:0/3032160886 shutdown_connections 2024-06-02T23:09:36.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.056+0000 7f5692a2e700 1 -- 172.21.15.52:0/3032160886 wait complete. 2024-06-02T23:09:36.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 Processor -- start 2024-06-02T23:09:36.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 -- start start 2024-06-02T23:09:36.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 0x7f568c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:36.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f568c1030f0 0x7f568c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:36.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c107840 0x7f568c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f568c112610 con 0x7f568c102740 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f568c112490 con 0x7f568c107840 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f5692a2e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f568c112790 con 0x7f568c1030f0 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f568bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 0x7f568c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.057+0000 7f568bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 0x7f568c19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49058/0 (socket says 172.21.15.52:49058) 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 -- 172.21.15.52:0/4083545116 learned_addr learned my addr 172.21.15.52:0/4083545116 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568b7fe700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f568c1030f0 0x7f568c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f5690fcb700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c107840 0x7f568c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 -- 172.21.15.52:0/4083545116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f568c1030f0 msgr2=0x7f568c19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f568c1030f0 0x7f568c19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 -- 172.21.15.52:0/4083545116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c107840 msgr2=0x7f568c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c107840 0x7f568c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 -- 172.21.15.52:0/4083545116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c199030 con 0x7f568c102740 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f5690fcb700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c107840 0x7f568c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f568bfff700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 0x7f568c19e710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d950 tx=0x7f567c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c016070 con 0x7f568c102740 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f567c009df0 con 0x7f568c102740 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c199320 con 0x7f568c102740 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f567c00f3c0 con 0x7f568c102740 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.058+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1a55e0 con 0x7f568c102740 2024-06-02T23:09:36.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.059+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c069180 con 0x7f568c102740 2024-06-02T23:09:36.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.060+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f567c019070 con 0x7f568c102740 2024-06-02T23:09:36.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.061+0000 7f56897fa700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56740a2cb0 0x7f56740a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:36.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.061+0000 7f568b7fe700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56740a2cb0 0x7f56740a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:36.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.061+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f567c0cedb0 con 0x7f568c102740 2024-06-02T23:09:36.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.062+0000 7f568b7fe700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56740a2cb0 0x7f56740a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f568c1070c0 tx=0x7f56800073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:36.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.066+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f567c09bbe0 con 0x7f568c102740 2024-06-02T23:09:36.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.268+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f568c0628d0 con 0x7f56740a2cb0 2024-06-02T23:09:36.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.276+0000 7f56897fa700 1 -- 172.21.15.52:0/4083545116 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258956 (secure 0 0 0) 0x7f568c0628d0 con 0x7f56740a2cb0 2024-06-02T23:09:36.279 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56740a2cb0 msgr2=0x7f56740a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56740a2cb0 0x7f56740a5170 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f568c1070c0 tx=0x7f56800073d0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 msgr2=0x7f568c19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 0x7f568c19e710 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f567c00d950 tx=0x7f567c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 shutdown_connections 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f56740a2cb0 0x7f56740a5170 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f568c107840 0x7f568c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f568c102740 0x7f568c19e710 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 --2- 172.21.15.52:0/4083545116 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f568c1030f0 0x7f568c19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.278+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 >> 172.21.15.52:0/4083545116 conn(0x7f568c0795f0 msgr2=0x7f568c100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.279+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 shutdown_connections 2024-06-02T23:09:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:36.279+0000 7f5692a2e700 1 -- 172.21.15.52:0/4083545116 wait complete. 2024-06-02T23:09:36.282 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:09:36.982 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":363,"stamp":"2024-06-02T23:09:35.791086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":659239,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1174,"num_read_kb":706,"num_write":428,"num_write_kb":3075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369652,"kb_used_data":12144,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572172,"statfs":{"total":1151940427776,"available":1151561904128,"internally_reserved":0,"allocated":12435456,"data_stored":7728453,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002773"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331749+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:08:11.331749+0000","last_peered":"2024-06-02T23:08:11.331749+0000","last_clean":"2024-06-02T23:08:11.331749+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:08:11.331749+0000","last_undegraded":"2024-06-02T23:08:11.331749+0000","last_fullsized":"2024-06-02T23:08:11.331749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331436+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:08:11.331436+0000","last_peered":"2024-06-02T23:08:11.331436+0000","last_clean":"2024-06-02T23:08:11.331436+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:08:11.331436+0000","last_undegraded":"2024-06-02T23:08:11.331436+0000","last_fullsized":"2024-06-02T23:08:11.331436+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331379+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:08:11.331379+0000","last_peered":"2024-06-02T23:08:11.331379+0000","last_clean":"2024-06-02T23:08:11.331379+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:08:11.331379+0000","last_undegraded":"2024-06-02T23:08:11.331379+0000","last_fullsized":"2024-06-02T23:08:11.331379+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450534+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:08:11.450534+0000","last_peered":"2024-06-02T23:08:11.450534+0000","last_clean":"2024-06-02T23:08:11.450534+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:08:11.450534+0000","last_undegraded":"2024-06-02T23:08:11.450534+0000","last_fullsized":"2024-06-02T23:08:11.450534+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334614+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:08:11.334614+0000","last_peered":"2024-06-02T23:08:11.334614+0000","last_clean":"2024-06-02T23:08:11.334614+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:08:11.334614+0000","last_undegraded":"2024-06-02T23:08:11.334614+0000","last_fullsized":"2024-06-02T23:08:11.334614+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449583+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:08:11.449583+0000","last_peered":"2024-06-02T23:08:11.449583+0000","last_clean":"2024-06-02T23:08:11.449583+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:08:11.449583+0000","last_undegraded":"2024-06-02T23:08:11.449583+0000","last_fullsized":"2024-06-02T23:08:11.449583+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332875+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:08:11.332875+0000","last_peered":"2024-06-02T23:08:11.332875+0000","last_clean":"2024-06-02T23:08:11.332875+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:08:11.332875+0000","last_undegraded":"2024-06-02T23:08:11.332875+0000","last_fullsized":"2024-06-02T23:08:11.332875+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748655+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:08:11.748655+0000","last_peered":"2024-06-02T23:08:11.748655+0000","last_clean":"2024-06-02T23:08:11.748655+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:08:11.748655+0000","last_undegraded":"2024-06-02T23:08:11.748655+0000","last_fullsized":"2024-06-02T23:08:11.748655+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334826+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:08:11.334826+0000","last_peered":"2024-06-02T23:08:11.334826+0000","last_clean":"2024-06-02T23:08:11.334826+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:08:11.334826+0000","last_undegraded":"2024-06-02T23:08:11.334826+0000","last_fullsized":"2024-06-02T23:08:11.334826+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449830+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:08:11.449830+0000","last_peered":"2024-06-02T23:08:11.449830+0000","last_clean":"2024-06-02T23:08:11.449830+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:08:11.449830+0000","last_undegraded":"2024-06-02T23:08:11.449830+0000","last_fullsized":"2024-06-02T23:08:11.449830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332762+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:08:11.332762+0000","last_peered":"2024-06-02T23:08:11.332762+0000","last_clean":"2024-06-02T23:08:11.332762+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:08:11.332762+0000","last_undegraded":"2024-06-02T23:08:11.332762+0000","last_fullsized":"2024-06-02T23:08:11.332762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450303+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:08:11.450303+0000","last_peered":"2024-06-02T23:08:11.450303+0000","last_clean":"2024-06-02T23:08:11.450303+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:08:11.450303+0000","last_undegraded":"2024-06-02T23:08:11.450303+0000","last_fullsized":"2024-06-02T23:08:11.450303+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333334+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:08:11.333334+0000","last_peered":"2024-06-02T23:08:11.333334+0000","last_clean":"2024-06-02T23:08:11.333334+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:08:11.333334+0000","last_undegraded":"2024-06-02T23:08:11.333334+0000","last_fullsized":"2024-06-02T23:08:11.333334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330935+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:08:11.330935+0000","last_peered":"2024-06-02T23:08:11.330935+0000","last_clean":"2024-06-02T23:08:11.330935+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:08:11.330935+0000","last_undegraded":"2024-06-02T23:08:11.330935+0000","last_fullsized":"2024-06-02T23:08:11.330935+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330848+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:08:11.330848+0000","last_peered":"2024-06-02T23:08:11.330848+0000","last_clean":"2024-06-02T23:08:11.330848+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:08:11.330848+0000","last_undegraded":"2024-06-02T23:08:11.330848+0000","last_fullsized":"2024-06-02T23:08:11.330848+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331976+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:08:11.331976+0000","last_peered":"2024-06-02T23:08:11.331976+0000","last_clean":"2024-06-02T23:08:11.331976+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:08:11.331976+0000","last_undegraded":"2024-06-02T23:08:11.331976+0000","last_fullsized":"2024-06-02T23:08:11.331976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328445+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:08:11.328445+0000","last_peered":"2024-06-02T23:08:11.328445+0000","last_clean":"2024-06-02T23:08:11.328445+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:08:11.328445+0000","last_undegraded":"2024-06-02T23:08:11.328445+0000","last_fullsized":"2024-06-02T23:08:11.328445+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333538+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:08:11.333538+0000","last_peered":"2024-06-02T23:08:11.333538+0000","last_clean":"2024-06-02T23:08:11.333538+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:08:11.333538+0000","last_undegraded":"2024-06-02T23:08:11.333538+0000","last_fullsized":"2024-06-02T23:08:11.333538+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":140,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:30.000627+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:09:30.000627+0000","last_peered":"2024-06-02T23:09:30.000627+0000","last_clean":"2024-06-02T23:09:30.000627+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:09:30.000627+0000","last_undegraded":"2024-06-02T23:09:30.000627+0000","last_fullsized":"2024-06-02T23:09:30.000627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749070+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:08:11.749070+0000","last_peered":"2024-06-02T23:08:11.749070+0000","last_clean":"2024-06-02T23:08:11.749070+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:08:11.749070+0000","last_undegraded":"2024-06-02T23:08:11.749070+0000","last_fullsized":"2024-06-02T23:08:11.749070+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331872+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:08:11.331872+0000","last_peered":"2024-06-02T23:08:11.331872+0000","last_clean":"2024-06-02T23:08:11.331872+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:08:11.331872+0000","last_undegraded":"2024-06-02T23:08:11.331872+0000","last_fullsized":"2024-06-02T23:08:11.331872+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.365621+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:11.365621+0000","last_peered":"2024-06-02T23:08:11.365621+0000","last_clean":"2024-06-02T23:08:11.365621+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:11.365621+0000","last_undegraded":"2024-06-02T23:08:11.365621+0000","last_fullsized":"2024-06-02T23:08:11.365621+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334580+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:08:11.334580+0000","last_peered":"2024-06-02T23:08:11.334580+0000","last_clean":"2024-06-02T23:08:11.334580+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:08:11.334580+0000","last_undegraded":"2024-06-02T23:08:11.334580+0000","last_fullsized":"2024-06-02T23:08:11.334580+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335715+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335715+0000","last_peered":"2024-06-02T23:08:11.335715+0000","last_clean":"2024-06-02T23:08:11.335715+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335715+0000","last_undegraded":"2024-06-02T23:08:11.335715+0000","last_fullsized":"2024-06-02T23:08:11.335715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329756+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:08:11.329756+0000","last_peered":"2024-06-02T23:08:11.329756+0000","last_clean":"2024-06-02T23:08:11.329756+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:08:11.329756+0000","last_undegraded":"2024-06-02T23:08:11.329756+0000","last_fullsized":"2024-06-02T23:08:11.329756+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"88'5","reported_seq":147,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:32.569216+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:09:32.569216+0000","last_peered":"2024-06-02T23:09:32.569216+0000","last_clean":"2024-06-02T23:09:32.569216+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:09:32.569216+0000","last_undegraded":"2024-06-02T23:09:32.569216+0000","last_fullsized":"2024-06-02T23:09:32.569216+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334793+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:08:11.334793+0000","last_peered":"2024-06-02T23:08:11.334793+0000","last_clean":"2024-06-02T23:08:11.334793+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:08:11.334793+0000","last_undegraded":"2024-06-02T23:08:11.334793+0000","last_fullsized":"2024-06-02T23:08:11.334793+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449532+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449532+0000","last_peered":"2024-06-02T23:08:11.449532+0000","last_clean":"2024-06-02T23:08:11.449532+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449532+0000","last_undegraded":"2024-06-02T23:08:11.449532+0000","last_fullsized":"2024-06-02T23:08:11.449532+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329554+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:08:11.329554+0000","last_peered":"2024-06-02T23:08:11.329554+0000","last_clean":"2024-06-02T23:08:11.329554+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:08:11.329554+0000","last_undegraded":"2024-06-02T23:08:11.329554+0000","last_fullsized":"2024-06-02T23:08:11.329554+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332891+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332891+0000","last_peered":"2024-06-02T23:08:11.332891+0000","last_clean":"2024-06-02T23:08:11.332891+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332891+0000","last_undegraded":"2024-06-02T23:08:11.332891+0000","last_fullsized":"2024-06-02T23:08:11.332891+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748738+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:08:11.748738+0000","last_peered":"2024-06-02T23:08:11.748738+0000","last_clean":"2024-06-02T23:08:11.748738+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:08:11.748738+0000","last_undegraded":"2024-06-02T23:08:11.748738+0000","last_fullsized":"2024-06-02T23:08:11.748738+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329228+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329228+0000","last_peered":"2024-06-02T23:08:11.329228+0000","last_clean":"2024-06-02T23:08:11.329228+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329228+0000","last_undegraded":"2024-06-02T23:08:11.329228+0000","last_fullsized":"2024-06-02T23:08:11.329228+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328243+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:08:11.328243+0000","last_peered":"2024-06-02T23:08:11.328243+0000","last_clean":"2024-06-02T23:08:11.328243+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:08:11.328243+0000","last_undegraded":"2024-06-02T23:08:11.328243+0000","last_fullsized":"2024-06-02T23:08:11.328243+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":231,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:31.567307+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:09:31.567307+0000","last_peered":"2024-06-02T23:09:31.567307+0000","last_clean":"2024-06-02T23:09:31.567307+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:09:31.567307+0000","last_undegraded":"2024-06-02T23:09:31.567307+0000","last_fullsized":"2024-06-02T23:09:31.567307+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":201,"num_read_kb":201,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334817+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:08:11.334817+0000","last_peered":"2024-06-02T23:08:11.334817+0000","last_clean":"2024-06-02T23:08:11.334817+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:08:11.334817+0000","last_undegraded":"2024-06-02T23:08:11.334817+0000","last_fullsized":"2024-06-02T23:08:11.334817+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332982+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.332982+0000","last_peered":"2024-06-02T23:08:11.332982+0000","last_clean":"2024-06-02T23:08:11.332982+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.332982+0000","last_undegraded":"2024-06-02T23:08:11.332982+0000","last_fullsized":"2024-06-02T23:08:11.332982+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331775+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:08:11.331775+0000","last_peered":"2024-06-02T23:08:11.331775+0000","last_clean":"2024-06-02T23:08:11.331775+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:08:11.331775+0000","last_undegraded":"2024-06-02T23:08:11.331775+0000","last_fullsized":"2024-06-02T23:08:11.331775+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329928+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.329928+0000","last_peered":"2024-06-02T23:08:11.329928+0000","last_clean":"2024-06-02T23:08:11.329928+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.329928+0000","last_undegraded":"2024-06-02T23:08:11.329928+0000","last_fullsized":"2024-06-02T23:08:11.329928+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334827+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:08:11.334827+0000","last_peered":"2024-06-02T23:08:11.334827+0000","last_clean":"2024-06-02T23:08:11.334827+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:08:11.334827+0000","last_undegraded":"2024-06-02T23:08:11.334827+0000","last_fullsized":"2024-06-02T23:08:11.334827+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'31","reported_seq":266,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:31.567295+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:09:31.567295+0000","last_peered":"2024-06-02T23:09:31.567295+0000","last_clean":"2024-06-02T23:09:31.567295+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:09:31.567295+0000","last_undegraded":"2024-06-02T23:09:31.567295+0000","last_fullsized":"2024-06-02T23:09:31.567295+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61825,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":209,"num_read_kb":163,"num_write":33,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335764+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:08:11.335764+0000","last_peered":"2024-06-02T23:08:11.335764+0000","last_clean":"2024-06-02T23:08:11.335764+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:08:11.335764+0000","last_undegraded":"2024-06-02T23:08:11.335764+0000","last_fullsized":"2024-06-02T23:08:11.335764+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332546+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332546+0000","last_peered":"2024-06-02T23:08:11.332546+0000","last_clean":"2024-06-02T23:08:11.332546+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332546+0000","last_undegraded":"2024-06-02T23:08:11.332546+0000","last_fullsized":"2024-06-02T23:08:11.332546+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333123+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:08:11.333123+0000","last_peered":"2024-06-02T23:08:11.333123+0000","last_clean":"2024-06-02T23:08:11.333123+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:08:11.333123+0000","last_undegraded":"2024-06-02T23:08:11.333123+0000","last_fullsized":"2024-06-02T23:08:11.333123+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329400+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329400+0000","last_peered":"2024-06-02T23:08:11.329400+0000","last_clean":"2024-06-02T23:08:11.329400+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329400+0000","last_undegraded":"2024-06-02T23:08:11.329400+0000","last_fullsized":"2024-06-02T23:08:11.329400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332914+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:08:11.332914+0000","last_peered":"2024-06-02T23:08:11.332914+0000","last_clean":"2024-06-02T23:08:11.332914+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:08:11.332914+0000","last_undegraded":"2024-06-02T23:08:11.332914+0000","last_fullsized":"2024-06-02T23:08:11.332914+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'5","reported_seq":137,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:34.028627+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:09:34.028627+0000","last_peered":"2024-06-02T23:09:34.028627+0000","last_clean":"2024-06-02T23:09:34.028627+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:09:34.028627+0000","last_undegraded":"2024-06-02T23:09:34.028627+0000","last_fullsized":"2024-06-02T23:09:34.028627+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":107,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332579+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:08:11.332579+0000","last_peered":"2024-06-02T23:08:11.332579+0000","last_clean":"2024-06-02T23:08:11.332579+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:08:11.332579+0000","last_undegraded":"2024-06-02T23:08:11.332579+0000","last_fullsized":"2024-06-02T23:08:11.332579+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.348940+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:11.348940+0000","last_peered":"2024-06-02T23:08:11.348940+0000","last_clean":"2024-06-02T23:08:11.348940+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:11.348940+0000","last_undegraded":"2024-06-02T23:08:11.348940+0000","last_fullsized":"2024-06-02T23:08:11.348940+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329323+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:08:11.329323+0000","last_peered":"2024-06-02T23:08:11.329323+0000","last_clean":"2024-06-02T23:08:11.329323+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:08:11.329323+0000","last_undegraded":"2024-06-02T23:08:11.329323+0000","last_fullsized":"2024-06-02T23:08:11.329323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335313+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:08:11.335313+0000","last_peered":"2024-06-02T23:08:11.335313+0000","last_clean":"2024-06-02T23:08:11.335313+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:08:11.335313+0000","last_undegraded":"2024-06-02T23:08:11.335313+0000","last_fullsized":"2024-06-02T23:08:11.335313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331466+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:08:11.331466+0000","last_peered":"2024-06-02T23:08:11.331466+0000","last_clean":"2024-06-02T23:08:11.331466+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:08:11.331466+0000","last_undegraded":"2024-06-02T23:08:11.331466+0000","last_fullsized":"2024-06-02T23:08:11.331466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328487+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328487+0000","last_peered":"2024-06-02T23:08:11.328487+0000","last_clean":"2024-06-02T23:08:11.328487+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328487+0000","last_undegraded":"2024-06-02T23:08:11.328487+0000","last_fullsized":"2024-06-02T23:08:11.328487+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449500+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:08:11.449500+0000","last_peered":"2024-06-02T23:08:11.449500+0000","last_clean":"2024-06-02T23:08:11.449500+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:08:11.449500+0000","last_undegraded":"2024-06-02T23:08:11.449500+0000","last_fullsized":"2024-06-02T23:08:11.449500+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'5","reported_seq":131,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:28.999956+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:09:28.999956+0000","last_peered":"2024-06-02T23:09:28.999956+0000","last_clean":"2024-06-02T23:09:28.999956+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:09:28.999956+0000","last_undegraded":"2024-06-02T23:09:28.999956+0000","last_fullsized":"2024-06-02T23:09:28.999956+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":100,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449616+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:08:11.449616+0000","last_peered":"2024-06-02T23:08:11.449616+0000","last_clean":"2024-06-02T23:08:11.449616+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:08:11.449616+0000","last_undegraded":"2024-06-02T23:08:11.449616+0000","last_fullsized":"2024-06-02T23:08:11.449616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329818+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:08:11.329818+0000","last_peered":"2024-06-02T23:08:11.329818+0000","last_clean":"2024-06-02T23:08:11.329818+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:08:11.329818+0000","last_undegraded":"2024-06-02T23:08:11.329818+0000","last_fullsized":"2024-06-02T23:08:11.329818+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":136,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:34.027730+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:09:34.027730+0000","last_peered":"2024-06-02T23:09:34.027730+0000","last_clean":"2024-06-02T23:09:34.027730+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:09:34.027730+0000","last_undegraded":"2024-06-02T23:09:34.027730+0000","last_fullsized":"2024-06-02T23:09:34.027730+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":104,"num_read_kb":104,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748857+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:08:11.748857+0000","last_peered":"2024-06-02T23:08:11.748857+0000","last_clean":"2024-06-02T23:08:11.748857+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:08:11.748857+0000","last_undegraded":"2024-06-02T23:08:11.748857+0000","last_fullsized":"2024-06-02T23:08:11.748857+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333070+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:11.333070+0000","last_peered":"2024-06-02T23:08:11.333070+0000","last_clean":"2024-06-02T23:08:11.333070+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:11.333070+0000","last_undegraded":"2024-06-02T23:08:11.333070+0000","last_fullsized":"2024-06-02T23:08:11.333070+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332742+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:08:11.332742+0000","last_peered":"2024-06-02T23:08:11.332742+0000","last_clean":"2024-06-02T23:08:11.332742+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:08:11.332742+0000","last_undegraded":"2024-06-02T23:08:11.332742+0000","last_fullsized":"2024-06-02T23:08:11.332742+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333307+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:08:11.333307+0000","last_peered":"2024-06-02T23:08:11.333307+0000","last_clean":"2024-06-02T23:08:11.333307+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:08:11.333307+0000","last_undegraded":"2024-06-02T23:08:11.333307+0000","last_fullsized":"2024-06-02T23:08:11.333307+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.213929+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:08:14.213929+0000","last_peered":"2024-06-02T23:08:14.213929+0000","last_clean":"2024-06-02T23:08:14.213929+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:08:14.213929+0000","last_undegraded":"2024-06-02T23:08:14.213929+0000","last_fullsized":"2024-06-02T23:08:14.213929+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749041+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:08:11.749041+0000","last_peered":"2024-06-02T23:08:11.749041+0000","last_clean":"2024-06-02T23:08:11.749041+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:08:11.749041+0000","last_undegraded":"2024-06-02T23:08:11.749041+0000","last_fullsized":"2024-06-02T23:08:11.749041+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450403+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450403+0000","last_peered":"2024-06-02T23:08:11.450403+0000","last_clean":"2024-06-02T23:08:11.450403+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450403+0000","last_undegraded":"2024-06-02T23:08:11.450403+0000","last_fullsized":"2024-06-02T23:08:11.450403+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.750091+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:08:11.750091+0000","last_peered":"2024-06-02T23:08:11.750091+0000","last_clean":"2024-06-02T23:08:11.750091+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:08:11.750091+0000","last_undegraded":"2024-06-02T23:08:11.750091+0000","last_fullsized":"2024-06-02T23:08:11.750091+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334410+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334410+0000","last_peered":"2024-06-02T23:08:11.334410+0000","last_clean":"2024-06-02T23:08:11.334410+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334410+0000","last_undegraded":"2024-06-02T23:08:11.334410+0000","last_fullsized":"2024-06-02T23:08:11.334410+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334625+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:08:11.334625+0000","last_peered":"2024-06-02T23:08:11.334625+0000","last_clean":"2024-06-02T23:08:11.334625+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:08:11.334625+0000","last_undegraded":"2024-06-02T23:08:11.334625+0000","last_fullsized":"2024-06-02T23:08:11.334625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748865+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748865+0000","last_peered":"2024-06-02T23:08:11.748865+0000","last_clean":"2024-06-02T23:08:11.748865+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748865+0000","last_undegraded":"2024-06-02T23:08:11.748865+0000","last_fullsized":"2024-06-02T23:08:11.748865+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332297+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:08:11.332297+0000","last_peered":"2024-06-02T23:08:11.332297+0000","last_clean":"2024-06-02T23:08:11.332297+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:08:11.332297+0000","last_undegraded":"2024-06-02T23:08:11.332297+0000","last_fullsized":"2024-06-02T23:08:11.332297+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332588+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:08:11.332588+0000","last_peered":"2024-06-02T23:08:11.332588+0000","last_clean":"2024-06-02T23:08:11.332588+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:08:11.332588+0000","last_undegraded":"2024-06-02T23:08:11.332588+0000","last_fullsized":"2024-06-02T23:08:11.332588+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748997+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:08:11.748997+0000","last_peered":"2024-06-02T23:08:11.748997+0000","last_clean":"2024-06-02T23:08:11.748997+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:08:11.748997+0000","last_undegraded":"2024-06-02T23:08:11.748997+0000","last_fullsized":"2024-06-02T23:08:11.748997+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334746+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334746+0000","last_peered":"2024-06-02T23:08:11.334746+0000","last_clean":"2024-06-02T23:08:11.334746+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334746+0000","last_undegraded":"2024-06-02T23:08:11.334746+0000","last_fullsized":"2024-06-02T23:08:11.334746+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333187+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:08:11.333187+0000","last_peered":"2024-06-02T23:08:11.333187+0000","last_clean":"2024-06-02T23:08:11.333187+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:08:11.333187+0000","last_undegraded":"2024-06-02T23:08:11.333187+0000","last_fullsized":"2024-06-02T23:08:11.333187+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":138,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:33.062167+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:09:33.062167+0000","last_peered":"2024-06-02T23:09:33.062167+0000","last_clean":"2024-06-02T23:09:33.062167+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:09:33.062167+0000","last_undegraded":"2024-06-02T23:09:33.062167+0000","last_fullsized":"2024-06-02T23:09:33.062167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1560,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":106,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332904+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:08:11.332904+0000","last_peered":"2024-06-02T23:08:11.332904+0000","last_clean":"2024-06-02T23:08:11.332904+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:08:11.332904+0000","last_undegraded":"2024-06-02T23:08:11.332904+0000","last_fullsized":"2024-06-02T23:08:11.332904+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330120+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330120+0000","last_peered":"2024-06-02T23:08:11.330120+0000","last_clean":"2024-06-02T23:08:11.330120+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330120+0000","last_undegraded":"2024-06-02T23:08:11.330120+0000","last_fullsized":"2024-06-02T23:08:11.330120+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"88'1","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332737+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:08:11.332737+0000","last_peered":"2024-06-02T23:08:11.332737+0000","last_clean":"2024-06-02T23:08:11.332737+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:08:11.332737+0000","last_undegraded":"2024-06-02T23:08:11.332737+0000","last_fullsized":"2024-06-02T23:08:11.332737+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333475+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333475+0000","last_peered":"2024-06-02T23:08:11.333475+0000","last_clean":"2024-06-02T23:08:11.333475+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333475+0000","last_undegraded":"2024-06-02T23:08:11.333475+0000","last_fullsized":"2024-06-02T23:08:11.333475+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332416+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:08:11.332416+0000","last_peered":"2024-06-02T23:08:11.332416+0000","last_clean":"2024-06-02T23:08:11.332416+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:08:11.332416+0000","last_undegraded":"2024-06-02T23:08:11.332416+0000","last_fullsized":"2024-06-02T23:08:11.332416+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332160+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:08:11.332160+0000","last_peered":"2024-06-02T23:08:11.332160+0000","last_clean":"2024-06-02T23:08:11.332160+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:08:11.332160+0000","last_undegraded":"2024-06-02T23:08:11.332160+0000","last_fullsized":"2024-06-02T23:08:11.332160+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332456+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:08:11.332456+0000","last_peered":"2024-06-02T23:08:11.332456+0000","last_clean":"2024-06-02T23:08:11.332456+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:08:11.332456+0000","last_undegraded":"2024-06-02T23:08:11.332456+0000","last_fullsized":"2024-06-02T23:08:11.332456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.331147+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:08:11.331147+0000","last_peered":"2024-06-02T23:08:11.331147+0000","last_clean":"2024-06-02T23:08:11.331147+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:08:11.331147+0000","last_undegraded":"2024-06-02T23:08:11.331147+0000","last_fullsized":"2024-06-02T23:08:11.331147+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335176+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:08:11.335176+0000","last_peered":"2024-06-02T23:08:11.335176+0000","last_clean":"2024-06-02T23:08:11.335176+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:08:11.335176+0000","last_undegraded":"2024-06-02T23:08:11.335176+0000","last_fullsized":"2024-06-02T23:08:11.335176+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337292+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:08:11.337292+0000","last_peered":"2024-06-02T23:08:11.337292+0000","last_clean":"2024-06-02T23:08:11.337292+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:08:11.337292+0000","last_undegraded":"2024-06-02T23:08:11.337292+0000","last_fullsized":"2024-06-02T23:08:11.337292+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333612+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:08:11.333612+0000","last_peered":"2024-06-02T23:08:11.333612+0000","last_clean":"2024-06-02T23:08:11.333612+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:08:11.333612+0000","last_undegraded":"2024-06-02T23:08:11.333612+0000","last_fullsized":"2024-06-02T23:08:11.333612+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749044+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:08:11.749044+0000","last_peered":"2024-06-02T23:08:11.749044+0000","last_clean":"2024-06-02T23:08:11.749044+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:08:11.749044+0000","last_undegraded":"2024-06-02T23:08:11.749044+0000","last_fullsized":"2024-06-02T23:08:11.749044+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329174+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:08:11.329174+0000","last_peered":"2024-06-02T23:08:11.329174+0000","last_clean":"2024-06-02T23:08:11.329174+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:08:11.329174+0000","last_undegraded":"2024-06-02T23:08:11.329174+0000","last_fullsized":"2024-06-02T23:08:11.329174+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748682+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:08:11.748682+0000","last_peered":"2024-06-02T23:08:11.748682+0000","last_clean":"2024-06-02T23:08:11.748682+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:08:11.748682+0000","last_undegraded":"2024-06-02T23:08:11.748682+0000","last_fullsized":"2024-06-02T23:08:11.748682+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334859+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:08:11.334859+0000","last_peered":"2024-06-02T23:08:11.334859+0000","last_clean":"2024-06-02T23:08:11.334859+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:08:11.334859+0000","last_undegraded":"2024-06-02T23:08:11.334859+0000","last_fullsized":"2024-06-02T23:08:11.334859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333208+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:08:11.333208+0000","last_peered":"2024-06-02T23:08:11.333208+0000","last_clean":"2024-06-02T23:08:11.333208+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:08:11.333208+0000","last_undegraded":"2024-06-02T23:08:11.333208+0000","last_fullsized":"2024-06-02T23:08:11.333208+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.337520+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:08:11.337520+0000","last_peered":"2024-06-02T23:08:11.337520+0000","last_clean":"2024-06-02T23:08:11.337520+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:08:11.337520+0000","last_undegraded":"2024-06-02T23:08:11.337520+0000","last_fullsized":"2024-06-02T23:08:11.337520+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68637,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1048,"num_read_kb":597,"num_write":245,"num_write_kb":135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":995328,"data_stored":480922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28132,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717020,"statfs":{"total":95995035648,"available":95966228480,"internally_reserved":0,"allocated":1142784,"data_stored":1133274,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808883,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332158,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":1072855,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495683,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":995328,"data_stored":480067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574939,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":542805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":481796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1069056,"data_stored":542793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":480550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1073152,"data_stored":1072533,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":63577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":63561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:09:36.986 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-02T23:09:36.986 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-06-02T23:09:36.986 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-06-02T23:09:36.986 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph health --format=json 2024-06-02T23:09:37.240 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:37 smithi174 ceph-mon[35994]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:37 smithi174 ceph-mon[35994]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:09:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:37 smithi052 ceph-mon[32340]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:37 smithi052 ceph-mon[32340]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:09:38.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:37 smithi123 ceph-mon[35977]: pgmap v363: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:37 smithi123 ceph-mon[35977]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:09:38.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 -- 172.21.15.52:0/729258992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810101390 msgr2=0x7f1810101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:38.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 --2- 172.21.15.52:0/729258992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810101390 0x7f1810101810 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f1804009990 tx=0x7f1804009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 -- 172.21.15.52:0/729258992 shutdown_connections 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 --2- 172.21.15.52:0/729258992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101d50 0x7f181010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 --2- 172.21.15.52:0/729258992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810107400 0x7f18101077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 --2- 172.21.15.52:0/729258992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810101390 0x7f1810101810 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 -- 172.21.15.52:0/729258992 >> 172.21.15.52:0/729258992 conn(0x7f1810078340 msgr2=0x7f1810078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 -- 172.21.15.52:0/729258992 shutdown_connections 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.863+0000 7f181522f700 1 -- 172.21.15.52:0/729258992 wait complete. 2024-06-02T23:09:38.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 Processor -- start 2024-06-02T23:09:38.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 -- start start 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 0x7f1810103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810101d50 0x7f18101035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810107400 0x7f1810103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1810111520 con 0x7f1810107400 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18101113a0 con 0x7f1810101d50 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f181522f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18101116a0 con 0x7f1810101390 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f180e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810101d50 0x7f18101035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f180ed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 0x7f1810103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f180f59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810107400 0x7f1810103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f180e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810101d50 0x7f18101035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42300/0 (socket says 172.21.15.52:42300) 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f180ed9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 0x7f1810103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:58830/0 (socket says 172.21.15.52:58830) 2024-06-02T23:09:38.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.864+0000 7f180e59c700 1 -- 172.21.15.52:0/600662626 learned_addr learned my addr 172.21.15.52:0/600662626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f180ed9d700 1 -- 172.21.15.52:0/600662626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810101d50 msgr2=0x7f18101035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f180ed9d700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810101d50 0x7f18101035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f180ed9d700 1 -- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810107400 msgr2=0x7f1810103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f180ed9d700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810107400 0x7f1810103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f180ed9d700 1 -- 172.21.15.52:0/600662626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18101a7030 con 0x7f1810101390 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f180ed9d700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 0x7f1810103070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f180000d950 tx=0x7f180000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1800016070 con 0x7f1810101390 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18101a7320 con 0x7f1810101390 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f180000fd40 con 0x7f1810101390 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180000e3b0 con 0x7f1810101390 2024-06-02T23:09:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.865+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18101a78b0 con 0x7f1810101390 2024-06-02T23:09:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.866+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1810069180 con 0x7f1810101390 2024-06-02T23:09:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.868+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 137884+0+0 (secure 0 0 0) 0x7f1800019070 con 0x7f1810101390 2024-06-02T23:09:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.869+0000 7f17f7fff700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17f80a2cb0 0x7f17f80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.869+0000 7f180e59c700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17f80a2cb0 0x7f17f80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.869+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f18000cf010 con 0x7f1810101390 2024-06-02T23:09:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.870+0000 7f180e59c700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17f80a2cb0 0x7f17f80a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18100788b0 tx=0x7f1804005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:38.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:38.872+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f180009be40 con 0x7f1810101390 2024-06-02T23:09:39.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.124+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f18101a7b70 con 0x7f1810101390 2024-06-02T23:09:39.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.124+0000 7f17f7fff700 1 -- 172.21.15.52:0/600662626 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f180009b740 con 0x7f1810101390 2024-06-02T23:09:39.125 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:39.125 INFO:teuthology.orchestra.run.smithi052.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.126+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17f80a2cb0 msgr2=0x7f17f80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.126+0000 7f181522f700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17f80a2cb0 0x7f17f80a5170 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f18100788b0 tx=0x7f1804005bc0 comp rx=0 tx=0).stop 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.126+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 msgr2=0x7f1810103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.126+0000 7f181522f700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 0x7f1810103070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f180000d950 tx=0x7f180000dc60 comp rx=0 tx=0).stop 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 shutdown_connections 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17f80a2cb0 0x7f17f80a5170 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:39.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1810107400 0x7f1810103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:39.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1810101390 0x7f1810103070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:39.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 --2- 172.21.15.52:0/600662626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1810101d50 0x7f18101035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:39.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 >> 172.21.15.52:0/600662626 conn(0x7f1810078340 msgr2=0x7f1810100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:39.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 shutdown_connections 2024-06-02T23:09:39.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:39.127+0000 7f181522f700 1 -- 172.21.15.52:0/600662626 wait complete. 2024-06-02T23:09:39.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:39 smithi052 ceph-mon[32340]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:39.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:39 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/600662626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-02T23:09:39.846 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-06-02T23:09:39.847 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:09:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:39 smithi174 ceph-mon[35994]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:39 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/600662626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-02T23:09:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:39 smithi123 ceph-mon[35977]: pgmap v364: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:39 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/600662626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-06-02T23:09:40.097 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:41.747 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:41 smithi052 ceph-mon[32340]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:41.748 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:41 smithi052 ceph-mon[32340]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-06-02T23:09:41.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.746+0000 7fec08e83700 1 -- 172.21.15.52:0/2343793959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 msgr2=0x7fec0410f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.746+0000 7fec08e83700 1 --2- 172.21.15.52:0/2343793959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0410f280 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b0a0 tx=0x7febfc00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.746+0000 7fec08e83700 1 -- 172.21.15.52:0/2343793959 shutdown_connections 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.746+0000 7fec08e83700 1 --2- 172.21.15.52:0/2343793959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0410f280 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.746+0000 7fec08e83700 1 --2- 172.21.15.52:0/2343793959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04107570 0x7fec04107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.746+0000 7fec08e83700 1 --2- 172.21.15.52:0/2343793959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04100e50 0x7fec041012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.747+0000 7fec08e83700 1 -- 172.21.15.52:0/2343793959 >> 172.21.15.52:0/2343793959 conn(0x7fec040766b0 msgr2=0x7fec04076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.747+0000 7fec08e83700 1 -- 172.21.15.52:0/2343793959 shutdown_connections 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.747+0000 7fec08e83700 1 -- 172.21.15.52:0/2343793959 wait complete. 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.747+0000 7fec08e83700 1 Processor -- start 2024-06-02T23:09:41.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 -- start start 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100e50 0x7fec0419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04107570 0x7fec04198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec04112660 con 0x7fec04101810 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec041124e0 con 0x7fec04100e50 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec08e83700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec041127e0 con 0x7fec04107570 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7febf9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7febf9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49082/0 (socket says 172.21.15.52:49082) 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7febf9d9b700 1 -- 172.21.15.52:0/2753399453 learned_addr learned my addr 172.21.15.52:0/2753399453 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec02d9d700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04107570 0x7fec04198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.748+0000 7fec0259c700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100e50 0x7fec0419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:41.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec02d9d700 1 -- 172.21.15.52:0/2753399453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100e50 msgr2=0x7fec0419e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec02d9d700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100e50 0x7fec0419e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec02d9d700 1 -- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 msgr2=0x7fec0419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec02d9d700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec02d9d700 1 -- 172.21.15.52:0/2753399453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec04198fe0 con 0x7fec04107570 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7febf9d9b700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec02d9d700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04107570 0x7fec04198880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b070 tx=0x7febfc004ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc021070 con 0x7fec04107570 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec04199270 con 0x7fec04107570 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7febfc007b80 con 0x7fec04107570 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc005040 con 0x7fec04107570 2024-06-02T23:09:41.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.749+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec041a5590 con 0x7fec04107570 2024-06-02T23:09:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.750+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec04069180 con 0x7fec04107570 2024-06-02T23:09:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.751+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7febfc024070 con 0x7fec04107570 2024-06-02T23:09:41.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.752+0000 7febfbfff700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7febf00a2c20 0x7febf00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.752+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7febfc00f020 con 0x7fec04107570 2024-06-02T23:09:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.752+0000 7fec0259c700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7febf00a2c20 0x7febf00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:41.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.753+0000 7fec0259c700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7febf00a2c20 0x7febf00a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fec041999c0 tx=0x7febf400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:41.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:41.755+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7febfc02b020 con 0x7fec04107570 2024-06-02T23:09:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:41 smithi174 ceph-mon[35994]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:41 smithi174 ceph-mon[35994]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-06-02T23:09:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:41 smithi123 ceph-mon[35977]: pgmap v365: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:09:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:41 smithi123 ceph-mon[35977]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-06-02T23:09:42.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.009+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec04199db0 con 0x7fec04107570 2024-06-02T23:09:42.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.010+0000 7febfbfff700 1 -- 172.21.15.52:0/2753399453 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7febfc09d520 con 0x7fec04107570 2024-06-02T23:09:42.010 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:42.010 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:07:54.810487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:09:42.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7febf00a2c20 msgr2=0x7febf00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:42.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7febf00a2c20 0x7febf00a50e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fec041999c0 tx=0x7febf400b410 comp rx=0 tx=0).stop 2024-06-02T23:09:42.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04107570 msgr2=0x7fec04198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:42.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04107570 0x7fec04198880 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7febfc00b070 tx=0x7febfc004ce0 comp rx=0 tx=0).stop 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 shutdown_connections 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7febf00a2c20 0x7febf00a50e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec04107570 0x7fec04198880 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec04100e50 0x7fec0419e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 --2- 172.21.15.52:0/2753399453 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec04101810 0x7fec0419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 >> 172.21.15.52:0/2753399453 conn(0x7fec040766b0 msgr2=0x7fec040ff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 shutdown_connections 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:42.012+0000 7fec08e83700 1 -- 172.21.15.52:0/2753399453 wait complete. 2024-06-02T23:09:42.013 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:09:42.685 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:09:42.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:42 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2753399453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:42.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:42 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2753399453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:43.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:42 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2753399453' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:43 smithi174 ceph-mon[35994]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:43 smithi052 ceph-mon[32340]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:44.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:43 smithi123 ceph-mon[35977]: pgmap v366: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:09:44.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 -- 172.21.15.52:0/3949803193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006b620 msgr2=0x7f356010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 --2- 172.21.15.52:0/3949803193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006b620 0x7f356010f350 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f355800b0d0 tx=0x7f355800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 -- 172.21.15.52:0/3949803193 shutdown_connections 2024-06-02T23:09:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 --2- 172.21.15.52:0/3949803193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006b620 0x7f356010f350 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 --2- 172.21.15.52:0/3949803193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3560108580 0x7f3560108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 --2- 172.21.15.52:0/3949803193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356006ac60 0x7f356006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 -- 172.21.15.52:0/3949803193 >> 172.21.15.52:0/3949803193 conn(0x7f3560078430 msgr2=0x7f3560078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:44.667 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:44 smithi052 ceph-mon[32340]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 -- 172.21.15.52:0/3949803193 shutdown_connections 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.665+0000 7f3568158700 1 -- 172.21.15.52:0/3949803193 wait complete. 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 Processor -- start 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 -- start start 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 0x7f3560101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356006b620 0x7f3560101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560108580 0x7f3560101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35601126f0 con 0x7f356006ac60 2024-06-02T23:09:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3560112570 con 0x7f356006b620 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3568158700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3560112870 con 0x7f3560108580 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3565ef4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 0x7f3560101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3565ef4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 0x7f3560101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56568/0 (socket says 172.21.15.52:56568) 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.666+0000 7f3565ef4700 1 -- 172.21.15.52:0/352296529 learned_addr learned my addr 172.21.15.52:0/352296529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f35666f5700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560108580 0x7f3560101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f35656f3700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356006b620 0x7f3560101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3565ef4700 1 -- 172.21.15.52:0/352296529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560108580 msgr2=0x7f3560101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3565ef4700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560108580 0x7f3560101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3565ef4700 1 -- 172.21.15.52:0/352296529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356006b620 msgr2=0x7f3560101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3565ef4700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356006b620 0x7f3560101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3565ef4700 1 -- 172.21.15.52:0/352296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35601a70e0 con 0x7f356006ac60 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3565ef4700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 0x7f3560101400 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c920 tx=0x7f355c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:44.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c007b00 con 0x7f356006ac60 2024-06-02T23:09:44.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f355c004ce0 con 0x7f356006ac60 2024-06-02T23:09:44.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35601a73d0 con 0x7f356006ac60 2024-06-02T23:09:44.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355c005550 con 0x7f356006ac60 2024-06-02T23:09:44.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.667+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35601a7960 con 0x7f356006ac60 2024-06-02T23:09:44.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.668+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3560069180 con 0x7f356006ac60 2024-06-02T23:09:44.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.669+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f355c004750 con 0x7f356006ac60 2024-06-02T23:09:44.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.670+0000 7f3556ffd700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f354c0a2ca0 0x7f354c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:44.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.670+0000 7f35656f3700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f354c0a2ca0 0x7f354c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:44.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.670+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f355c0ceef0 con 0x7f356006ac60 2024-06-02T23:09:44.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.671+0000 7f35656f3700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f354c0a2ca0 0x7f354c0a5160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3550006fd0 tx=0x7f3550008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:44.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.675+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f355c014030 con 0x7f356006ac60 2024-06-02T23:09:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:44 smithi174 ceph-mon[35994]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-02T23:09:44.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.928+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f356002dca0 con 0x7f356006ac60 2024-06-02T23:09:44.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.929+0000 7f3556ffd700 1 -- 172.21.15.52:0/352296529 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f355c09be20 con 0x7f356006ac60 2024-06-02T23:09:44.929 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:44.929 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:07:54.810487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:09:44.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f354c0a2ca0 msgr2=0x7f354c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:44.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f354c0a2ca0 0x7f354c0a5160 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3550006fd0 tx=0x7f3550008040 comp rx=0 tx=0).stop 2024-06-02T23:09:44.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 msgr2=0x7f3560101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:44.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 0x7f3560101400 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f355c00c920 tx=0x7f355c00cc30 comp rx=0 tx=0).stop 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 shutdown_connections 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f354c0a2ca0 0x7f354c0a5160 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3560108580 0x7f3560101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f356006ac60 0x7f3560101400 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 --2- 172.21.15.52:0/352296529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f356006b620 0x7f3560101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 >> 172.21.15.52:0/352296529 conn(0x7f3560078430 msgr2=0x7f35600fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 shutdown_connections 2024-06-02T23:09:44.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:44.931+0000 7f3568158700 1 -- 172.21.15.52:0/352296529 wait complete. 2024-06-02T23:09:44.933 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:09:45.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:44 smithi123 ceph-mon[35977]: pgmap v367: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-02T23:09:45.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:09:45.879 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:45 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/352296529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:45 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/352296529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:45 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/352296529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:46.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:46 smithi052 ceph-mon[32340]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-02T23:09:47.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:46 smithi123 ceph-mon[35977]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-02T23:09:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:46 smithi174 ceph-mon[35994]: pgmap v368: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-02T23:09:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 -- 172.21.15.52:0/3339125172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 msgr2=0x7fd61410f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 --2- 172.21.15.52:0/3339125172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd61410f3d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00b0d0 tx=0x7fd60c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 -- 172.21.15.52:0/3339125172 shutdown_connections 2024-06-02T23:09:47.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 --2- 172.21.15.52:0/3339125172 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd61410f3d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 --2- 172.21.15.52:0/3339125172 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61406ae80 0x7fd61406b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 --2- 172.21.15.52:0/3339125172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61406b830 0x7fd614107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 -- 172.21.15.52:0/3339125172 >> 172.21.15.52:0/3339125172 conn(0x7fd614078400 msgr2=0x7fd614078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 -- 172.21.15.52:0/3339125172 shutdown_connections 2024-06-02T23:09:47.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.495+0000 7fd61a139700 1 -- 172.21.15.52:0/3339125172 wait complete. 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 Processor -- start 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 -- start start 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61406ae80 0x7fd61419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61406b830 0x7fd61419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd614198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd614112620 con 0x7fd614107850 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141124a0 con 0x7fd61406ae80 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd61a139700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6141127a0 con 0x7fd61406b830 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd613fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd614198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd613fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd614198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56574/0 (socket says 172.21.15.52:56574) 2024-06-02T23:09:47.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd613fff700 1 -- 172.21.15.52:0/3041232966 learned_addr learned my addr 172.21.15.52:0/3041232966 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd6137fe700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61406ae80 0x7fd61419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.496+0000 7fd60affd700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61406b830 0x7fd61419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd613fff700 1 -- 172.21.15.52:0/3041232966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61406b830 msgr2=0x7fd61419ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd613fff700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61406b830 0x7fd61419ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd613fff700 1 -- 172.21.15.52:0/3041232966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61406ae80 msgr2=0x7fd61419e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd613fff700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61406ae80 0x7fd61419e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd613fff700 1 -- 172.21.15.52:0/3041232966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd614199060 con 0x7fd614107850 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd613fff700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd614198880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0095c0 tx=0x7fd60c007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01f070 con 0x7fd614107850 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6141992f0 con 0x7fd614107850 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd60c003c00 con 0x7fd614107850 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01b6a0 con 0x7fd614107850 2024-06-02T23:09:47.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.497+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6141a5510 con 0x7fd614107850 2024-06-02T23:09:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.498+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd614103be0 con 0x7fd614107850 2024-06-02T23:09:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.499+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd60c022070 con 0x7fd614107850 2024-06-02T23:09:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.500+0000 7fd6117fa700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6000a2c20 0x7fd6000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.500+0000 7fd6137fe700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6000a2c20 0x7fd6000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:47.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.500+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd60c00f020 con 0x7fd614107850 2024-06-02T23:09:47.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.501+0000 7fd6137fe700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6000a2c20 0x7fd6000a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd604005950 tx=0x7fd6040058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:47.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.504+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd60c029020 con 0x7fd614107850 2024-06-02T23:09:47.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.712+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd614199d60 con 0x7fd614107850 2024-06-02T23:09:47.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.713+0000 7fd6117fa700 1 -- 172.21.15.52:0/3041232966 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7fd60c09d770 con 0x7fd614107850 2024-06-02T23:09:47.714 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:47.714 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6000a2c20 msgr2=0x7fd6000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6000a2c20 0x7fd6000a50e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd604005950 tx=0x7fd6040058e0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 msgr2=0x7fd614198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd614198880 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd60c0095c0 tx=0x7fd60c007ad0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 shutdown_connections 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6000a2c20 0x7fd6000a50e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd614107850 0x7fd614198880 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd61406ae80 0x7fd61419e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 --2- 172.21.15.52:0/3041232966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61406b830 0x7fd61419ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:47.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 >> 172.21.15.52:0/3041232966 conn(0x7fd614078400 msgr2=0x7fd6140ff470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:47.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 shutdown_connections 2024-06-02T23:09:47.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:47.716+0000 7fd61a139700 1 -- 172.21.15.52:0/3041232966 wait complete. 2024-06-02T23:09:48.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:47 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3041232966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:09:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:47 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3041232966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:09:48.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:47 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3041232966' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:09:48.469 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:09:48.723 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:49.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:48 smithi052 ceph-mon[32340]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-02T23:09:49.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:48 smithi174 ceph-mon[35994]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-02T23:09:49.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:48 smithi123 ceph-mon[35977]: pgmap v369: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-02T23:09:50.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 -- 172.21.15.52:0/1675323344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 msgr2=0x7fecbc10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:50.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1675323344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc10ea00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009990 tx=0x7fecb0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 -- 172.21.15.52:0/1675323344 shutdown_connections 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1675323344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc10ef40 0x7fecbc111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1675323344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecbc06a1a0 0x7fecbc06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1675323344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc10ea00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 -- 172.21.15.52:0/1675323344 >> 172.21.15.52:0/1675323344 conn(0x7fecbc078340 msgr2=0x7fecbc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 -- 172.21.15.52:0/1675323344 shutdown_connections 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.403+0000 7fecc14bd700 1 -- 172.21.15.52:0/1675323344 wait complete. 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 Processor -- start 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 -- start start 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc06a1a0 0x7fecbc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:50.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecbc10ef40 0x7fecbc19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc114850 con 0x7fecbc06aac0 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1146d0 con 0x7fecbc10ef40 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecc14bd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1149d0 con 0x7fecbc06a1a0 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecba7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecba7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc19b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56588/0 (socket says 172.21.15.52:56588) 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecba7fc700 1 -- 172.21.15.52:0/1171465169 learned_addr learned my addr 172.21.15.52:0/1171465169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecbb7fe700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecbc10ef40 0x7fecbc19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.404+0000 7fecbaffd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc06a1a0 0x7fecbc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecba7fc700 1 -- 172.21.15.52:0/1171465169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc06a1a0 msgr2=0x7fecbc19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecba7fc700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc06a1a0 0x7fecbc19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecba7fc700 1 -- 172.21.15.52:0/1171465169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecbc10ef40 msgr2=0x7fecbc19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecba7fc700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecbc10ef40 0x7fecbc19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecba7fc700 1 -- 172.21.15.52:0/1171465169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc19fdb0 con 0x7fecbc06aac0 2024-06-02T23:09:50.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecba7fc700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc19b2e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb0003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:50.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb0027070 con 0x7fecbc06aac0 2024-06-02T23:09:50.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecb000bdc0 con 0x7fecbc06aac0 2024-06-02T23:09:50.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc1a0040 con 0x7fecbc06aac0 2024-06-02T23:09:50.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb000f6c0 con 0x7fecbc06aac0 2024-06-02T23:09:50.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.405+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc1a7920 con 0x7fecbc06aac0 2024-06-02T23:09:50.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.406+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecbc1019b0 con 0x7fecbc06aac0 2024-06-02T23:09:50.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.407+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fecb002a070 con 0x7fecbc06aac0 2024-06-02T23:09:50.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.408+0000 7feca3fff700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feca40a2c20 0x7feca40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:50.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.408+0000 7fecbaffd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feca40a2c20 0x7feca40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:50.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.408+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fecb00d0210 con 0x7fecbc06aac0 2024-06-02T23:09:50.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.408+0000 7fecbaffd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feca40a2c20 0x7feca40a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fecac0060b0 tx=0x7fecac007b80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:50.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.413+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fecb005e850 con 0x7fecbc06aac0 2024-06-02T23:09:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.664+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fecbc19c100 con 0x7fecbc06aac0 2024-06-02T23:09:50.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.665+0000 7feca3fff700 1 -- 172.21.15.52:0/1171465169 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7fecb0013070 con 0x7fecbc06aac0 2024-06-02T23:09:50.665 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:09:50.665 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:07:54.810487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:09:50.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feca40a2c20 msgr2=0x7feca40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:50.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feca40a2c20 0x7feca40a50e0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fecac0060b0 tx=0x7fecac007b80 comp rx=0 tx=0).stop 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 msgr2=0x7fecbc19b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc19b2e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecb0000c00 tx=0x7fecb0003b30 comp rx=0 tx=0).stop 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 shutdown_connections 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feca40a2c20 0x7feca40a50e0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fecbc10ef40 0x7fecbc19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fecbc06a1a0 0x7fecbc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 --2- 172.21.15.52:0/1171465169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fecbc06aac0 0x7fecbc19b2e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 >> 172.21.15.52:0/1171465169 conn(0x7fecbc078340 msgr2=0x7fecbc1104b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 shutdown_connections 2024-06-02T23:09:50.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:50.667+0000 7fecc14bd700 1 -- 172.21.15.52:0/1171465169 wait complete. 2024-06-02T23:09:50.669 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:09:51.047 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:50 smithi052 ceph-mon[32340]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-02T23:09:51.047 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:50 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1171465169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:51.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:50 smithi174 ceph-mon[35994]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-02T23:09:51.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:50 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1171465169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:51.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:50 smithi123 ceph-mon[35977]: pgmap v370: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-06-02T23:09:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:50 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1171465169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:09:51.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-02T23:07:33.296248+0000', 'modified': '2024-06-02T23:07:54.810487+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24451, 'mds_2': 24449, 'mds_3': 24436, 'mds_4': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/2175393174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2175393174}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2175393174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/4238066654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 4238066654}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 4238066654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/224943979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 224943979}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 224943979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/1267486579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1267486579}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1267486579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/2486297365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 2486297365}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 2486297365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.123:6835/3092837185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3092837185}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3092837185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.174:6837/457473811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 457473811}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 457473811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6837/785283486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 785283486}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 785283486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.174:6839/2115890873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2115890873}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2115890873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/2043673684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2043673684}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2043673684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24470, 24449, 24451, 24436, 24434]} 2024-06-02T23:09:51.332 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-02T23:09:51.333 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-02T23:09:51.609 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:53.066 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:52 smithi052 ceph-mon[32340]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-02T23:09:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:52 smithi174 ceph-mon[35994]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-02T23:09:53.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:52 smithi123 ceph-mon[35977]: pgmap v371: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 -- 172.21.15.52:0/3513807815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8108950 msgr2=0x7f57c810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 --2- 172.21.15.52:0/3513807815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8108950 0x7f57c810f380 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f57c000b0d0 tx=0x7f57c000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 -- 172.21.15.52:0/3513807815 shutdown_connections 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 --2- 172.21.15.52:0/3513807815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8108950 0x7f57c810f380 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 --2- 172.21.15.52:0/3513807815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8100870 0x7f57c8100c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 --2- 172.21.15.52:0/3513807815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57c8101220 0x7f57c8108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.311+0000 7f57ce700700 1 -- 172.21.15.52:0/3513807815 >> 172.21.15.52:0/3513807815 conn(0x7f57c8077590 msgr2=0x7f57c80fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.312+0000 7f57ce700700 1 -- 172.21.15.52:0/3513807815 shutdown_connections 2024-06-02T23:09:53.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.312+0000 7f57ce700700 1 -- 172.21.15.52:0/3513807815 wait complete. 2024-06-02T23:09:53.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.312+0000 7f57ce700700 1 Processor -- start 2024-06-02T23:09:53.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.312+0000 7f57ce700700 1 -- start start 2024-06-02T23:09:53.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ce700700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57c8100870 0x7f57c8102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ce700700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 0x7f57c81025c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ce700700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8108950 0x7f57c8102b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ce700700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8112660 con 0x7f57c8101220 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ce700700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81124e0 con 0x7f57c8108950 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ce700700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81127e0 con 0x7f57c8100870 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 0x7f57c81025c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 0x7f57c81025c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56608/0 (socket says 172.21.15.52:56608) 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 -- 172.21.15.52:0/348062911 learned_addr learned my addr 172.21.15.52:0/348062911 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c7fff700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57c8100870 0x7f57c8102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57ccc9d700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8108950 0x7f57c8102b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 -- 172.21.15.52:0/348062911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57c8100870 msgr2=0x7f57c8102080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57c8100870 0x7f57c8102080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 -- 172.21.15.52:0/348062911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8108950 msgr2=0x7f57c8102b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8108950 0x7f57c8102b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 -- 172.21.15.52:0/348062911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c81a6ff0 con 0x7f57c8101220 2024-06-02T23:09:53.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c77fe700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 0x7f57c81025c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00d980 tx=0x7f57bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:53.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.313+0000 7f57c57fa700 1 -- 172.21.15.52:0/348062911 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc017070 con 0x7f57c8101220 2024-06-02T23:09:53.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.314+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c81a72e0 con 0x7f57c8101220 2024-06-02T23:09:53.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.314+0000 7f57c57fa700 1 -- 172.21.15.52:0/348062911 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57bc009de0 con 0x7f57c8101220 2024-06-02T23:09:53.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.314+0000 7f57c57fa700 1 -- 172.21.15.52:0/348062911 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00f3c0 con 0x7f57c8101220 2024-06-02T23:09:53.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.314+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c81a7870 con 0x7f57c8101220 2024-06-02T23:09:53.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.316+0000 7f57c57fa700 1 -- 172.21.15.52:0/348062911 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f57bc009930 con 0x7f57c8101220 2024-06-02T23:09:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.317+0000 7f57c57fa700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57b00a2c20 0x7f57b00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.317+0000 7f57c7fff700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57b00a2c20 0x7f57b00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.317+0000 7f57c57fa700 1 -- 172.21.15.52:0/348062911 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f57bc0cee70 con 0x7f57c8101220 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.317+0000 7f57c7fff700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57b00a2c20 0x7f57b00a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57b8005950 tx=0x7f57b800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 Processor -- start 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 -- start start 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b50f10 0x55abc4b512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b4b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b4b750 0x55abc4b4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a3df10 con 0x55abc4b51830 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a9c800 con 0x55abc4b4b750 2024-06-02T23:09:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a04410 con 0x55abc4b50f10 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b4b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b4b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56620/0 (socket says 172.21.15.52:56620) 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57c77fe700 1 -- 172.21.15.52:0/1060731433 learned_addr learned my addr 172.21.15.52:0/1060731433 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57c7fff700 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b50f10 0x55abc4b512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57ccc9d700 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b4b750 0x55abc4b4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57c77fe700 1 -- 172.21.15.52:0/1060731433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b50f10 msgr2=0x55abc4b512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.318+0000 7f57c77fe700 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b50f10 0x55abc4b512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57c77fe700 1 -- 172.21.15.52:0/1060731433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b4b750 msgr2=0x55abc4b4bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57c77fe700 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b4b750 0x55abc4b4bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57c77fe700 1 -- 172.21.15.52:0/1060731433 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc49eb880 con 0x55abc4b51830 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57ccc9d700 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b4b750 0x55abc4b4bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57c77fe700 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b4b210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0a3de0 tx=0x7f57bc010710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ef26960fe008107 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57ae7fc700 1 -- 172.21.15.52:0/1060731433 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc020070 con 0x55abc4b51830 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57ae7fc700 1 -- 172.21.15.52:0/1060731433 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57bc0860b0 con 0x55abc4b51830 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.319+0000 7f57ae7fc700 1 -- 172.21.15.52:0/1060731433 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc010460 con 0x55abc4b51830 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 -- 172.21.15.52:0/1060731433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 msgr2=0x55abc4b4b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b4b210 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f57bc0a3de0 tx=0x7f57bc010710 comp rx=0 tx=0).stop 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 -- 172.21.15.52:0/1060731433 shutdown_connections 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b4b750 0x55abc4b4bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b50f10 0x55abc4b512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/1060731433 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b4b210 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 -- 172.21.15.52:0/1060731433 >> 172.21.15.52:0/1060731433 conn(0x55abc4a78030 msgr2=0x55abc4b461d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 -- 172.21.15.52:0/1060731433 shutdown_connections 2024-06-02T23:09:53.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 -- 172.21.15.52:0/1060731433 wait complete. 2024-06-02T23:09:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.320+0000 7f57d8f83b80 1 -- start start 2024-06-02T23:09:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 Processor -- start 2024-06-02T23:09:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b575e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b548e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b54e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a3cf70 con 0x55abc4b51830 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abc4a987c0 con 0x55abc4b50f10 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b58150 con 0x55abc4b4b750 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57d8f83b80 10 client.0 _ll_register_callbacks cb 0x55abc4a32560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57ccc9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b54e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b575e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b575e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41144/0 (socket says 172.21.15.52:41144) 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 -- 172.21.15.52:0/3518246261 learned_addr learned my addr 172.21.15.52:0/3518246261 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57ccc9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b54e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56630/0 (socket says 172.21.15.52:56630) 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c77fe700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 -- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 msgr2=0x55abc4b548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 -- 172.21.15.52:0/3518246261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 msgr2=0x55abc4b54e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.321+0000 7f57c7fff700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b54e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57c7fff700 1 -- 172.21.15.52:0/3518246261 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc4a62700 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57ccc9d700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b54e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57c7fff700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b575e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f57b801da70 tx=0x7f57b80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76a3f7683304503 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57a77fe700 1 -- 172.21.15.52:0/3518246261 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b802a060 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57a77fe700 1 -- 172.21.15.52:0/3518246261 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f57b801b910 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57bc0860b0 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57a77fe700 1 -- 172.21.15.52:0/3518246261 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57b8016070 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57d8f83b80 10 client.24733 fetch_fsmap learned FSMap version 27 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.322+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55abc4b62980 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57a77fe700 1 -- 172.21.15.52:0/3518246261 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57bc0860b0 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57a77fe700 1 -- 172.21.15.52:0/3518246261 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f57b8024030 con 0x55abc4b4b750 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57d8f83b80 10 client.24733 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:09:53.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57d8f83b80 10 client.24733 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57d8f83b80 20 client.24733 populate_metadata read hostname 'smithi052' 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4b64f70 0x55abc4b663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57d8f83b80 4 client.24733 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abc4aa9d00 con 0x55abc4b64f70 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.323+0000 7f57c77fe700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4b64f70 0x55abc4b663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.324+0000 7f57c77fe700 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4b64f70 0x55abc4b663e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=fc3cc0047fd5a3ab server_cookie=6c0cfe47e1e20fe4 in_seq=0 out_seq=0 2024-06-02T23:09:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.324+0000 7f57a77fe700 10 client.24733 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57a77fe700 1 -- 172.21.15.52:0/3518246261 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abc4aa9d00 con 0x55abc4b64f70 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57a77fe700 10 client.24733 handle_command_reply: tid=0 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 client.24733 shutdown 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 msgr2=0x55abc4b575e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b575e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f57b801da70 tx=0x7f57b80047c0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 shutdown_connections 2024-06-02T23:09:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b51830 0x55abc4b54e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b575e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4b64f70 0x55abc4b663e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/3518246261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57a77fe700 0 client.24733 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 >> 172.21.15.52:0/3518246261 conn(0x55abc4a78030 msgr2=0x55abc4b456b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 shutdown_connections 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 1 -- 172.21.15.52:0/3518246261 wait complete. 2024-06-02T23:09:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.325+0000 7f57d8f83b80 20 client.24733 trim_cache size 0 max 16384 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 Processor -- start 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 -- start start 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 0x55abc4b54e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b52360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b51830 0x55abc4b45e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b64720 con 0x55abc4b4b750 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b64df0 con 0x55abc4b50f10 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b654c0 con 0x55abc4b51830 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 0x55abc4b54e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 0x55abc4b54e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56644/0 (socket says 172.21.15.52:56644) 2024-06-02T23:09:53.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 -- 172.21.15.52:0/2776089522 learned_addr learned my addr 172.21.15.52:0/2776089522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c77fe700 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b52360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 -- 172.21.15.52:0/2776089522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b51830 msgr2=0x55abc4b45e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b51830 0x55abc4b45e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 -- 172.21.15.52:0/2776089522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 msgr2=0x55abc4b52360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b52360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.347+0000 7f57c7fff700 1 -- 172.21.15.52:0/2776089522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abc4b4c140 con 0x55abc4b4b750 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.348+0000 7f57c7fff700 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 0x55abc4b54e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57b8005ea0 tx=0x7f57b800f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41d3a25e537bb76 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.348+0000 7f57adffb700 1 -- 172.21.15.52:0/2776089522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8035030 con 0x55abc4b4b750 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.348+0000 7f57adffb700 1 -- 172.21.15.52:0/2776089522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b800f940 con 0x55abc4b4b750 2024-06-02T23:09:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.348+0000 7f57adffb700 1 -- 172.21.15.52:0/2776089522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b801be00 con 0x55abc4b4b750 2024-06-02T23:09:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.348+0000 7f57d8f83b80 1 -- 172.21.15.52:0/2776089522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 msgr2=0x55abc4b54e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.348+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 0x55abc4b54e90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f57b8005ea0 tx=0x7f57b800f7d0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- 172.21.15.52:0/2776089522 shutdown_connections 2024-06-02T23:09:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b51830 0x55abc4b45e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b4b750 0x55abc4b54e90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/2776089522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b50f10 0x55abc4b52360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- 172.21.15.52:0/2776089522 >> 172.21.15.52:0/2776089522 conn(0x55abc4a78030 msgr2=0x55abc4b51c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:53.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- 172.21.15.52:0/2776089522 shutdown_connections 2024-06-02T23:09:53.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- 172.21.15.52:0/2776089522 wait complete. 2024-06-02T23:09:53.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- start start 2024-06-02T23:09:53.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 Processor -- start 2024-06-02T23:09:53.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b6f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 0x55abc4b6fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b51830 0x55abc4b6a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b66260 con 0x55abc4b50f10 2024-06-02T23:09:53.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b65b90 con 0x55abc4b51830 2024-06-02T23:09:53.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abc4b66930 con 0x55abc4b4b750 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.349+0000 7f57d8f83b80 10 client.0 _ll_register_callbacks cb 0x55abc4a32560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 0x55abc4b6fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 0x55abc4b6fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56656/0 (socket says 172.21.15.52:56656) 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 -- 172.21.15.52:0/433480609 learned_addr learned my addr 172.21.15.52:0/433480609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c7fff700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b6f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57ccc9d700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b51830 0x55abc4b6a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 -- 172.21.15.52:0/433480609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 msgr2=0x55abc4b6f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b6f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 -- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b51830 msgr2=0x55abc4b6a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b51830 0x55abc4b6a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 -- 172.21.15.52:0/433480609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abc4b46600 con 0x55abc4b50f10 2024-06-02T23:09:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57ccc9d700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b51830 0x55abc4b6a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57c77fe700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 0x55abc4b6fda0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f57bc010fd0 tx=0x7f57bc009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77350d511b280eef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57a77fe700 1 -- 172.21.15.52:0/433480609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57bc020040 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57a77fe700 1 -- 172.21.15.52:0/433480609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f57bc0999a0 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57b800f940 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57a77fe700 1 -- 172.21.15.52:0/433480609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f57bc0860b0 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57d8f83b80 10 client.14982 fetch_fsmap learned FSMap version 27 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.350+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55abc4bd0d40 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57a77fe700 1 -- 172.21.15.52:0/433480609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57b800f940 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57a77fe700 1 -- 172.21.15.52:0/433480609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f57bc09b970 con 0x55abc4b50f10 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57d8f83b80 10 client.14982 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57d8f83b80 10 client.14982 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57d8f83b80 20 client.14982 populate_metadata read hostname 'smithi052' 2024-06-02T23:09:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4bd5440 0x55abc4bd4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57d8f83b80 4 client.14982 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "status"}) -- 0x55abc4aa4280 con 0x55abc4bd5440 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57c7fff700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4bd5440 0x55abc4bd4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57a77fe700 10 client.14982 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.351+0000 7f57c7fff700 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4bd5440 0x55abc4bd4150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d52ab95c7b3bb17 server_cookie=dac3e6547b8142d0 in_seq=0 out_seq=0 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57a77fe700 1 -- 172.21.15.52:0/433480609 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55abc4aa4280 con 0x55abc4bd5440 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57a77fe700 10 client.14982 handle_command_reply: tid=0 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 client.14982 shutdown 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 msgr2=0x55abc4b6fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 0x55abc4b6fda0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f57bc010fd0 tx=0x7f57bc009d60 comp rx=0 tx=0).stop 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 shutdown_connections 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55abc4bd5440 0x55abc4bd4150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abc4b51830 0x55abc4b6a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abc4b4b750 0x55abc4b6f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 --2- 172.21.15.52:0/433480609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abc4b50f10 0x55abc4b6fda0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57a77fe700 0 client.14982 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 >> 172.21.15.52:0/433480609 conn(0x55abc4a78030 msgr2=0x55abc4b50310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 shutdown_connections 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 1 -- 172.21.15.52:0/433480609 wait complete. 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.352+0000 7f57d8f83b80 20 client.14982 trim_cache size 0 max 16384 2024-06-02T23:09:53.365 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24434, 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 122.082808411, 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 179.83558285699999 2024-06-02T23:09:53.366 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:09:53.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57b00a2c20 msgr2=0x7f57b00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57b00a2c20 0x7f57b00a50e0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f57b8005950 tx=0x7f57b800a300 comp rx=0 tx=0).stop 2024-06-02T23:09:53.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 msgr2=0x7f57c81025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:53.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 0x7f57c81025c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f57bc00d980 tx=0x7f57bc00dc90 comp rx=0 tx=0).stop 2024-06-02T23:09:53.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 shutdown_connections 2024-06-02T23:09:53.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57b00a2c20 0x7f57b00a50e0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57c8108950 0x7f57c8102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57c8100870 0x7f57c8102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 --2- 172.21.15.52:0/348062911 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57c8101220 0x7f57c81025c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:53.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 >> 172.21.15.52:0/348062911 conn(0x7f57c8077590 msgr2=0x7f57c80ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:53.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 shutdown_connections 2024-06-02T23:09:53.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:53.353+0000 7f57ce700700 1 -- 172.21.15.52:0/348062911 wait complete. 2024-06-02T23:09:54.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-02T23:09:54.424 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:55.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:54 smithi174 ceph-mon[35994]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-02T23:09:55.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:54 smithi052 ceph-mon[32340]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-02T23:09:55.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:54 smithi123 ceph-mon[35977]: pgmap v372: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-06-02T23:09:56.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.124+0000 7f3f96640700 1 -- 172.21.15.52:0/1748875699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90107400 msgr2=0x7f3f901077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.124+0000 7f3f96640700 1 --2- 172.21.15.52:0/1748875699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90107400 0x7f3f901077e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009990 tx=0x7f3f80009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.124+0000 7f3f96640700 1 -- 172.21.15.52:0/1748875699 shutdown_connections 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.124+0000 7f3f96640700 1 --2- 172.21.15.52:0/1748875699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101d50 0x7f3f9010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 --2- 172.21.15.52:0/1748875699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90107400 0x7f3f901077e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 --2- 172.21.15.52:0/1748875699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f90101390 0x7f3f90101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 -- 172.21.15.52:0/1748875699 >> 172.21.15.52:0/1748875699 conn(0x7f3f90078340 msgr2=0x7f3f90078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 -- 172.21.15.52:0/1748875699 shutdown_connections 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 -- 172.21.15.52:0/1748875699 wait complete. 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 Processor -- start 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 -- start start 2024-06-02T23:09:56.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101390 0x7f3f90103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 0x7f3f901035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.125+0000 7f3f96640700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f90107400 0x7f3f90103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f96640700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f901115a0 con 0x7f3f90101d50 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f96640700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90111420 con 0x7f3f90107400 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f96640700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f90111720 con 0x7f3f90101390 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 0x7f3f901035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 0x7f3f901035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35248/0 (socket says 172.21.15.52:35248) 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 -- 172.21.15.52:0/1048674296 learned_addr learned my addr 172.21.15.52:0/1048674296 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101390 0x7f3f90103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f90107400 0x7f3f90103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 -- 172.21.15.52:0/1048674296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101390 msgr2=0x7f3f90103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101390 0x7f3f90103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 -- 172.21.15.52:0/1048674296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f90107400 msgr2=0x7f3f90103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f90107400 0x7f3f90103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 -- 172.21.15.52:0/1048674296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f901a7010 con 0x7f3f90101d50 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101390 0x7f3f90103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f87fff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 0x7f3f901035b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c9b0 tx=0x7f3f7800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f8dffb700 1 -- 172.21.15.52:0/1048674296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78007b00 con 0x7f3f90101d50 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f901a7300 con 0x7f3f90101d50 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f8dffb700 1 -- 172.21.15.52:0/1048674296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f78007ca0 con 0x7f3f90101d50 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.126+0000 7f3f8dffb700 1 -- 172.21.15.52:0/1048674296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78018660 con 0x7f3f90101d50 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.127+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f901a7890 con 0x7f3f90101d50 2024-06-02T23:09:56.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.129+0000 7f3f8dffb700 1 -- 172.21.15.52:0/1048674296 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f7800f450 con 0x7f3f90101d50 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.130+0000 7f3f8dffb700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f7c0a2c20 0x7f3f7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.130+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f7c0a2c20 0x7f3f7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.130+0000 7f3f8dffb700 1 -- 172.21.15.52:0/1048674296 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3f780cf020 con 0x7f3f90101d50 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.130+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f7c0a2c20 0x7f3f7c0a50e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006010 tx=0x7f3f8000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 Processor -- start 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 -- start start 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 0x564aa6ed42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed4830 0x564aa6ece210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564aa6dc0f10 con 0x564aa6ed3f10 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564aa6e1f800 con 0x564aa6ed4830 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564aa6d87410 con 0x564aa6ece750 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 0x564aa6ed42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 0x564aa6ed42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35250/0 (socket says 172.21.15.52:35250) 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 -- 172.21.15.52:0/1773913623 learned_addr learned my addr 172.21.15.52:0/1773913623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f87fff700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed4830 0x564aa6ece210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 -- 172.21.15.52:0/1773913623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 msgr2=0x564aa6eceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eceb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 -- 172.21.15.52:0/1773913623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed4830 msgr2=0x564aa6ece210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed4830 0x564aa6ece210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f8ffff700 1 -- 172.21.15.52:0/1773913623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa6d6e880 con 0x564aa6ed3f10 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.131+0000 7f3f87fff700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed4830 0x564aa6ece210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.132+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 0x564aa6ed42f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009990 tx=0x7f3f8001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eaf7e176c42437c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.132+0000 7f3f867fc700 1 -- 172.21.15.52:0/1773913623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000bbd0 con 0x564aa6ed3f10 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.132+0000 7f3f867fc700 1 -- 172.21.15.52:0/1773913623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8000bd70 con 0x564aa6ed3f10 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.132+0000 7f3f867fc700 1 -- 172.21.15.52:0/1773913623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000fb30 con 0x564aa6ed3f10 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/1773913623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 msgr2=0x564aa6ed42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 0x564aa6ed42f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f80009990 tx=0x7f3f8001f000 comp rx=0 tx=0).stop 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/1773913623 shutdown_connections 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed3f10 0x564aa6ed42f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/1773913623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed4830 0x564aa6ece210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/1773913623 >> 172.21.15.52:0/1773913623 conn(0x564aa6dfb030 msgr2=0x564aa6ec91d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:56.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/1773913623 shutdown_connections 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/1773913623 wait complete. 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 -- start start 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.133+0000 7f3fa0ec3b80 1 Processor -- start 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eda550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ed7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564aa6dbff70 con 0x564aa6ed4830 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564aa6e1b7c0 con 0x564aa6ed3f10 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564aa6edb1b0 con 0x564aa6ece750 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3fa0ec3b80 10 client.0 _ll_register_callbacks cb 0x564aa6db5560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35266/0 (socket says 172.21.15.52:35266) 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2403982909 learned_addr learned my addr 172.21.15.52:0/2403982909 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:56.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f87fff700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ed7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eda550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 msgr2=0x564aa6eda550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eda550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2403982909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 msgr2=0x564aa6ed7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ed7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.134+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2403982909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa6de5700 con 0x564aa6ed4830 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed7d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800aef0 tx=0x7f3f8800af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62ac94b6d6289f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3f6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2403982909 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f880079c0 con 0x564aa6ed4830 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2403982909 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3f88007b60 con 0x564aa6ed4830 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f8000bd70 con 0x564aa6ed4830 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2403982909 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f88007d60 con 0x564aa6ed4830 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3fa0ec3b80 10 client.14994 fetch_fsmap learned FSMap version 27 2024-06-02T23:09:56.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564aa6ee5970 con 0x564aa6ed4830 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.135+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2403982909 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8000bd70 con 0x564aa6ed4830 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2403982909 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3f8801f400 con 0x564aa6ed4830 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3fa0ec3b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3fa0ec3b80 10 client.14994 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3fa0ec3b80 20 client.14994 populate_metadata read hostname 'smithi052' 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6ee7fb0 0x564aa6ee9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3fa0ec3b80 4 client.14994 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564aa6e2cd00 con 0x564aa6ee7fb0 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.136+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6ee7fb0 0x564aa6ee9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.137+0000 7f3f6f7fe700 10 client.14994 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:09:56.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.137+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6ee7fb0 0x564aa6ee9420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52d5d4a5dda1c58d server_cookie=b1400363fe3fe9e9 in_seq=0 out_seq=0 2024-06-02T23:09:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.137+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2403982909 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564aa6e2cd00 con 0x564aa6ee7fb0 2024-06-02T23:09:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.137+0000 7f3f6f7fe700 10 client.14994 handle_command_reply: tid=0 2024-06-02T23:09:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.137+0000 7f3fa0ec3b80 1 client.14994 shutdown 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 msgr2=0x564aa6ed7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed7d90 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3f8800aef0 tx=0x7f3f8800af20 comp rx=0 tx=0).stop 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 shutdown_connections 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed7d90 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6ee7fb0 0x564aa6ee9420 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6eda550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2403982909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ed7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 >> 172.21.15.52:0/2403982909 conn(0x564aa6dfb030 msgr2=0x564aa6ec86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3f6f7fe700 0 client.14994 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 shutdown_connections 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2403982909 wait complete. 2024-06-02T23:09:56.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.138+0000 7f3fa0ec3b80 20 client.14994 trim_cache size 0 max 16384 2024-06-02T23:09:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 Processor -- start 2024-06-02T23:09:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 -- start start 2024-06-02T23:09:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ece750 0x564aa6ec8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ed3f10 0x564aa6ec9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed6160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564aa6ee7760 con 0x564aa6ed4830 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564aa6ee7e30 con 0x564aa6ece750 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.159+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564aa6ee8500 con 0x564aa6ed3f10 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35282/0 (socket says 172.21.15.52:35282) 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2906749234 learned_addr learned my addr 172.21.15.52:0/2906749234 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ece750 0x564aa6ec8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2906749234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ed3f10 msgr2=0x564aa6ec9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ed3f10 0x564aa6ec9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2906749234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ece750 msgr2=0x564aa6ec8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ece750 0x564aa6ec8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 -- 172.21.15.52:0/2906749234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564aa6ee7c50 con 0x564aa6ed4830 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed6160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3f8801f7d0 tx=0x7f3f88007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19731d274331ffad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f85ffb700 1 -- 172.21.15.52:0/2906749234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8802f070 con 0x564aa6ed4830 2024-06-02T23:09:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f85ffb700 1 -- 172.21.15.52:0/2906749234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8801b070 con 0x564aa6ed4830 2024-06-02T23:09:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.160+0000 7f3f85ffb700 1 -- 172.21.15.52:0/2906749234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88022750 con 0x564aa6ed4830 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2906749234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 msgr2=0x564aa6ed6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed6160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3f8801f7d0 tx=0x7f3f88007f00 comp rx=0 tx=0).stop 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2906749234 shutdown_connections 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ed6160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ece750 0x564aa6ec8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/2906749234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ed3f10 0x564aa6ec9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2906749234 >> 172.21.15.52:0/2906749234 conn(0x564aa6dfb030 msgr2=0x564aa6ed1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:56.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2906749234 shutdown_connections 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/2906749234 wait complete. 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 -- start start 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.161+0000 7f3fa0ec3b80 1 Processor -- start 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6ef2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ef3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ec8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564aa6ee92a0 con 0x564aa6ed4830 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564aa6ee8bd0 con 0x564aa6ed3f10 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564aa6ee9970 con 0x564aa6ece750 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3fa0ec3b80 10 client.0 _ll_register_callbacks cb 0x564aa6db5560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ec8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ec8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35292/0 (socket says 172.21.15.52:35292) 2024-06-02T23:09:56.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 -- 172.21.15.52:0/3021618759 learned_addr learned my addr 172.21.15.52:0/3021618759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f87fff700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ef3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 -- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 msgr2=0x564aa6ef2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6ef2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6ef2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 -- 172.21.15.52:0/3021618759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 msgr2=0x564aa6ef3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ef3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 -- 172.21.15.52:0/3021618759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564aa6ec8910 con 0x564aa6ed4830 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f87fff700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ef3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f94bdd700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ec8140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f8802d040 tx=0x7f3f880070b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=624c3732ea48792b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.162+0000 7f3f6f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/3021618759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8802f040 con 0x564aa6ed4830 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f8801b070 con 0x564aa6ed4830 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/3021618759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3f88007a80 con 0x564aa6ed4830 2024-06-02T23:09:56.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/3021618759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f8803a070 con 0x564aa6ed4830 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 10 client.15000 fetch_fsmap learned FSMap version 27 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x564aa6ef2490 con 0x564aa6ed4830 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/3021618759 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f8801b070 con 0x564aa6ed4830 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/3021618759 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f3f8802a4e0 con 0x564aa6ed4830 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 10 client.15000 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 10 client.15000 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 20 client.15000 populate_metadata read hostname 'smithi052' 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6f60440 0x564aa6f5f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 4 client.15000 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.163+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "status"}) -- 0x564aa6e27280 con 0x564aa6f60440 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.164+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6f60440 0x564aa6f5f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.164+0000 7f3f8ffff700 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6f60440 0x564aa6f5f150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ebe3dadfc8e16734 server_cookie=76d31e92e1add19f in_seq=0 out_seq=0 2024-06-02T23:09:56.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.164+0000 7f3f6f7fe700 10 client.15000 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.164+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/3021618759 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564aa6e27280 con 0x564aa6f60440 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.164+0000 7f3f6f7fe700 10 client.15000 handle_command_reply: tid=0 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.164+0000 7f3fa0ec3b80 1 client.15000 shutdown 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 msgr2=0x564aa6ec8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ec8140 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3f8802d040 tx=0x7f3f880070b0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 shutdown_connections 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564aa6ed4830 0x564aa6ec8140 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x564aa6f60440 0x564aa6f5f150 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564aa6ece750 0x564aa6ef2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3f6f7fe700 0 client.15000 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 --2- 172.21.15.52:0/3021618759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564aa6ed3f10 0x564aa6ef3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 >> 172.21.15.52:0/3021618759 conn(0x564aa6dfb030 msgr2=0x564aa6ed4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 shutdown_connections 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 1 -- 172.21.15.52:0/3021618759 wait complete. 2024-06-02T23:09:56.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3fa0ec3b80 20 client.15000 trim_cache size 0 max 16384 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24436, 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 125.90351035899999, 2024-06-02T23:09:56.170 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 177.14086141999999 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f7c0a2c20 msgr2=0x7f3f7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3f96640700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f7c0a2c20 0x7f3f7c0a50e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f80006010 tx=0x7f3f8000b540 comp rx=0 tx=0).stop 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 msgr2=0x7f3f901035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.165+0000 7f3f96640700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 0x7f3f901035b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800c9b0 tx=0x7f3f7800ccc0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 shutdown_connections 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f7c0a2c20 0x7f3f7c0a50e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f90107400 0x7f3f90103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f90101390 0x7f3f90103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 --2- 172.21.15.52:0/1048674296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f90101d50 0x7f3f901035b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 >> 172.21.15.52:0/1048674296 conn(0x7f3f90078340 msgr2=0x7f3f90100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 shutdown_connections 2024-06-02T23:09:56.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:56.166+0000 7f3f96640700 1 -- 172.21.15.52:0/1048674296 wait complete. 2024-06-02T23:09:56.893 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-02T23:09:57.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:56 smithi174 ceph-mon[35994]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-02T23:09:57.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:56 smithi052 ceph-mon[32340]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-02T23:09:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:09:57.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:56 smithi123 ceph-mon[35977]: pgmap v373: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 -- 172.21.15.52:0/1846643885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 msgr2=0x7f756c10f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 --2- 172.21.15.52:0/1846643885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c10f280 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7564009990 tx=0x7f7564009ca0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 -- 172.21.15.52:0/1846643885 shutdown_connections 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 --2- 172.21.15.52:0/1846643885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c10f280 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 --2- 172.21.15.52:0/1846643885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f756c107570 0x7f756c107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 --2- 172.21.15.52:0/1846643885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f756c100e50 0x7f756c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 -- 172.21.15.52:0/1846643885 >> 172.21.15.52:0/1846643885 conn(0x7f756c0766b0 msgr2=0x7f756c076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 -- 172.21.15.52:0/1846643885 shutdown_connections 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.839+0000 7f7573608700 1 -- 172.21.15.52:0/1846643885 wait complete. 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 Processor -- start 2024-06-02T23:09:58.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 -- start start 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f756c100e50 0x7f756c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f756c107570 0x7f756c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f756c112700 con 0x7f756c101810 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f756c112580 con 0x7f756c107570 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7573608700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f756c112880 con 0x7f756c100e50 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7570ba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7570ba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35312/0 (socket says 172.21.15.52:35312) 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.840+0000 7f7570ba3700 1 -- 172.21.15.52:0/2419007998 learned_addr learned my addr 172.21.15.52:0/2419007998 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7571ba5700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f756c107570 0x7f756c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f75713a4700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f756c100e50 0x7f756c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7570ba3700 1 -- 172.21.15.52:0/2419007998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f756c100e50 msgr2=0x7f756c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7570ba3700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f756c100e50 0x7f756c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7570ba3700 1 -- 172.21.15.52:0/2419007998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f756c107570 msgr2=0x7f756c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7570ba3700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f756c107570 0x7f756c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7570ba3700 1 -- 172.21.15.52:0/2419007998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c1990c0 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f7570ba3700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c19ec90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b530 tx=0x7f755c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.841+0000 7f75627fc700 1 -- 172.21.15.52:0/2419007998 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c015020 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.842+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c1993b0 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.842+0000 7f75627fc700 1 -- 172.21.15.52:0/2419007998 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f755c011ce0 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.842+0000 7f75627fc700 1 -- 172.21.15.52:0/2419007998 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c010dd0 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.842+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c1a5620 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.844+0000 7f75627fc700 1 -- 172.21.15.52:0/2419007998 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f755c018020 con 0x7f756c101810 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.845+0000 7f75627fc700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f75580a2c20 0x7f75580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.845+0000 7f75713a4700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f75580a2c20 0x7f75580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.845+0000 7f75627fc700 1 -- 172.21.15.52:0/2419007998 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f755c0ce790 con 0x7f756c101810 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.846+0000 7f75713a4700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f75580a2c20 0x7f75580a50e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f756c1072b0 tx=0x7f756800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.846+0000 7f757de8bb80 1 Processor -- start 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.846+0000 7f757de8bb80 1 -- start start 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.846+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c466f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.846+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c4669990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.846+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c466a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f757de8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592c45bb7c0 con 0x5592c466ff10 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f757de8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592c45636d0 con 0x5592c4669ed0 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f757de8bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592c45b7780 con 0x5592c466f5f0 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c4669990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c4669990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35318/0 (socket says 172.21.15.52:35318) 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 -- 172.21.15.52:0/2796135625 learned_addr learned my addr 172.21.15.52:0/2796135625 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7571ba5700 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c466a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f75713a4700 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c466f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 -- 172.21.15.52:0/2796135625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 msgr2=0x5592c466f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c466f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 -- 172.21.15.52:0/2796135625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 msgr2=0x5592c466a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c466a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 -- 172.21.15.52:0/2796135625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c4559f00 con 0x5592c466ff10 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7571ba5700 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c466a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f7570ba3700 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c4669990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f755c0a3700 tx=0x7f755c096c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8bc09d63fc59b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f75577fe700 1 -- 172.21.15.52:0/2796135625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c010510 con 0x5592c466ff10 2024-06-02T23:09:58.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f75577fe700 1 -- 172.21.15.52:0/2796135625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f755c09b6c0 con 0x5592c466ff10 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.847+0000 7f75577fe700 1 -- 172.21.15.52:0/2796135625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c0a3a00 con 0x5592c466ff10 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 -- 172.21.15.52:0/2796135625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 msgr2=0x5592c4669990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c4669990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f755c0a3700 tx=0x7f755c096c60 comp rx=0 tx=0).stop 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 -- 172.21.15.52:0/2796135625 shutdown_connections 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c466a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c466f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 --2- 172.21.15.52:0/2796135625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c4669990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 -- 172.21.15.52:0/2796135625 >> 172.21.15.52:0/2796135625 conn(0x5592c4594c70 msgr2=0x5592c4664800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 -- 172.21.15.52:0/2796135625 shutdown_connections 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 -- 172.21.15.52:0/2796135625 wait complete. 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 -- start start 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.848+0000 7f757de8bb80 1 Processor -- start 2024-06-02T23:09:58.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c4675c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592c4676980 con 0x5592c466ff10 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592c455bba0 con 0x5592c4669ed0 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592c4676b00 con 0x5592c466f5f0 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f757de8bb80 10 client.0 _ll_register_callbacks cb 0x5592c45558d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46734b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35334/0 (socket says 172.21.15.52:35334) 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 -- 172.21.15.52:0/3355055877 learned_addr learned my addr 172.21.15.52:0/3355055877 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f75713a4700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c4675c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7570ba3700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 -- 172.21.15.52:0/3355055877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 msgr2=0x5592c4672f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 -- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 msgr2=0x5592c4675c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c4675c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 -- 172.21.15.52:0/3355055877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c450a880 con 0x5592c466ff10 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7571ba5700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46734b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7564015000 tx=0x7f7564009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7b8cfc9cdc38a19 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.849+0000 7f7554ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f7554ff9700 1 -- 172.21.15.52:0/3355055877 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75640054e0 con 0x5592c466ff10 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f7554ff9700 1 -- 172.21.15.52:0/3355055877 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7564005680 con 0x5592c466ff10 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755c09b6c0 con 0x5592c466ff10 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f7554ff9700 1 -- 172.21.15.52:0/3355055877 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7564003850 con 0x5592c466ff10 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 10 client.15012 fetch_fsmap learned FSMap version 27 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592c4681280 con 0x5592c466ff10 2024-06-02T23:09:58.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f7554ff9700 1 -- 172.21.15.52:0/3355055877 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f755c09b6c0 con 0x5592c466ff10 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f7554ff9700 1 -- 172.21.15.52:0/3355055877 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7564023a40 con 0x5592c466ff10 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 10 client.15012 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 10 client.15012 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 20 client.15012 populate_metadata read hostname 'smithi052' 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46838e0 0x5592c4684d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 4 client.15012 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.850+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592c45c8cc0 con 0x5592c46838e0 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.851+0000 7f75713a4700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46838e0 0x5592c4684d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.851+0000 7f75713a4700 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46838e0 0x5592c4684d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f93d6428760fce2 server_cookie=d1040598fa0a501d in_seq=0 out_seq=0 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.851+0000 7f7554ff9700 10 client.15012 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f7554ff9700 1 -- 172.21.15.52:0/3355055877 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5592c45c8cc0 con 0x5592c46838e0 2024-06-02T23:09:58.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f7554ff9700 10 client.15012 handle_command_reply: tid=0 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 client.15012 shutdown 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 msgr2=0x5592c46734b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46734b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7564015000 tx=0x7f7564009ee0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 shutdown_connections 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46838e0 0x5592c4684d50 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46734b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c4669ed0 0x5592c4675c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3355055877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f7554ff9700 0 client.15012 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 >> 172.21.15.52:0/3355055877 conn(0x5592c4594c70 msgr2=0x5592c4668980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:58.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 shutdown_connections 2024-06-02T23:09:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 1 -- 172.21.15.52:0/3355055877 wait complete. 2024-06-02T23:09:58.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.852+0000 7f757de8bb80 20 client.15012 trim_cache size 0 max 16384 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.874+0000 7f757de8bb80 1 Processor -- start 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.874+0000 7f757de8bb80 1 -- start start 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 0x5592c4669990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466ff10 0x5592c46703b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f757de8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592c4683090 con 0x5592c4669ed0 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f757de8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592c4683760 con 0x5592c466ff10 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f757de8bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592c4683e30 con 0x5592c466f5f0 2024-06-02T23:09:58.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 0x5592c4669990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 0x5592c4669990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35338/0 (socket says 172.21.15.52:35338) 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 -- 172.21.15.52:0/3556466578 learned_addr learned my addr 172.21.15.52:0/3556466578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f7571ba5700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466ff10 0x5592c46703b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 -- 172.21.15.52:0/3556466578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 msgr2=0x5592c4672870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f7570ba3700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 -- 172.21.15.52:0/3556466578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466ff10 msgr2=0x5592c46703b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466ff10 0x5592c46703b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 -- 172.21.15.52:0/3556466578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592c465eec0 con 0x5592c4669ed0 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f7570ba3700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:58.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f7571ba5700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466ff10 0x5592c46703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.875+0000 7f75713a4700 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 0x5592c4669990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5592c4685830 tx=0x7f7568012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f53412da86b71050 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f7556ffd700 1 -- 172.21.15.52:0/3556466578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568031070 con 0x5592c4669ed0 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f7556ffd700 1 -- 172.21.15.52:0/3556466578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568023b70 con 0x5592c4669ed0 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f7556ffd700 1 -- 172.21.15.52:0/3556466578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756802cb50 con 0x5592c4669ed0 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 -- 172.21.15.52:0/3556466578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 msgr2=0x5592c4669990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 0x5592c4669990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x5592c4685830 tx=0x7f7568012000 comp rx=0 tx=0).stop 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 -- 172.21.15.52:0/3556466578 shutdown_connections 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466ff10 0x5592c46703b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c4669ed0 0x5592c4669990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 --2- 172.21.15.52:0/3556466578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c466f5f0 0x5592c4672870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.876+0000 7f757de8bb80 1 -- 172.21.15.52:0/3556466578 >> 172.21.15.52:0/3556466578 conn(0x5592c4594c70 msgr2=0x5592c4664330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.877+0000 7f757de8bb80 1 -- 172.21.15.52:0/3556466578 shutdown_connections 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.877+0000 7f757de8bb80 1 -- 172.21.15.52:0/3556466578 wait complete. 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.877+0000 7f757de8bb80 1 -- start start 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.877+0000 7f757de8bb80 1 Processor -- start 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c4669ed0 0x5592c468e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466f5f0 0x5592c4689180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592c4684bd0 con 0x5592c466ff10 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592c4684500 con 0x5592c466f5f0 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592c46852a0 con 0x5592c4669ed0 2024-06-02T23:09:58.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 10 client.0 _ll_register_callbacks cb 0x5592c45558d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46896c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35348/0 (socket says 172.21.15.52:35348) 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 -- 172.21.15.52:0/550443054 learned_addr learned my addr 172.21.15.52:0/550443054 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f75713a4700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c4669ed0 0x5592c468e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7570ba3700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466f5f0 0x5592c4689180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 -- 172.21.15.52:0/550443054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c4669ed0 msgr2=0x5592c468e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c4669ed0 0x5592c468e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 -- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466f5f0 msgr2=0x5592c4689180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:09:58.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466f5f0 0x5592c4689180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 -- 172.21.15.52:0/550443054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592c4689c80 con 0x5592c466ff10 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7570ba3700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466f5f0 0x5592c4689180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7571ba5700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46896c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7564023e60 tx=0x7f7564005a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ea51cfc59a26e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7554ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7554ff9700 1 -- 172.21.15.52:0/550443054 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564003bf0 con 0x5592c466ff10 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f7554ff9700 1 -- 172.21.15.52:0/550443054 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7564003d90 con 0x5592c466ff10 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.878+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7568023b70 con 0x5592c466ff10 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f7554ff9700 1 -- 172.21.15.52:0/550443054 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7564005730 con 0x5592c466ff10 2024-06-02T23:09:58.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 10 client.15018 fetch_fsmap learned FSMap version 27 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5592c46efe20 con 0x5592c466ff10 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f7554ff9700 1 -- 172.21.15.52:0/550443054 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7568023b70 con 0x5592c466ff10 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f7554ff9700 1 -- 172.21.15.52:0/550443054 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f7564023930 con 0x5592c466ff10 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 10 client.15018 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 10 client.15018 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 20 client.15018 populate_metadata read hostname 'smithi052' 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46f4a90 0x5592c46f37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 4 client.15018 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.879+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "status"}) -- 0x5592c45c3240 con 0x5592c46f4a90 2024-06-02T23:09:58.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f75713a4700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46f4a90 0x5592c46f37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f75713a4700 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46f4a90 0x5592c46f37a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d396999b3a26216 server_cookie=820fdd2136c01f7e in_seq=0 out_seq=0 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f7554ff9700 10 client.15018 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f7554ff9700 1 -- 172.21.15.52:0/550443054 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5592c45c3240 con 0x5592c46f4a90 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f7554ff9700 10 client.15018 handle_command_reply: tid=0 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f757de8bb80 1 client.15018 shutdown 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 msgr2=0x5592c46896c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.880+0000 7f757de8bb80 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46896c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7564023e60 tx=0x7f7564005a10 comp rx=0 tx=0).stop 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 shutdown_connections 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5592c46f4a90 0x5592c46f37a0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592c466ff10 0x5592c46896c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f7554ff9700 0 client.15018 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592c4669ed0 0x5592c468e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 --2- 172.21.15.52:0/550443054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592c466f5f0 0x5592c4689180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 >> 172.21.15.52:0/550443054 conn(0x5592c4594c70 msgr2=0x5592c4670390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 shutdown_connections 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 1 -- 172.21.15.52:0/550443054 wait complete. 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f757de8bb80 20 client.15018 trim_cache size 0 max 16384 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24449, 2024-06-02T23:09:58.895 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 129.62793644199999, 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 177.017200718 2024-06-02T23:09:58.896 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f75580a2c20 msgr2=0x7f75580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f7573608700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f75580a2c20 0x7f75580a50e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f756c1072b0 tx=0x7f756800c040 comp rx=0 tx=0).stop 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 msgr2=0x7f756c19ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.881+0000 7f7573608700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c19ec90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f755c00b530 tx=0x7f755c00b840 comp rx=0 tx=0).stop 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 shutdown_connections 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f75580a2c20 0x7f75580a50e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f756c107570 0x7f756c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f756c100e50 0x7f756c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 --2- 172.21.15.52:0/2419007998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f756c101810 0x7f756c19ec90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 >> 172.21.15.52:0/2419007998 conn(0x7f756c0766b0 msgr2=0x7f756c0ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 shutdown_connections 2024-06-02T23:09:58.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:09:58.882+0000 7f7573608700 1 -- 172.21.15.52:0/2419007998 wait complete. 2024-06-02T23:09:59.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:09:58 smithi174 ceph-mon[35994]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:59.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:09:58 smithi052 ceph-mon[32340]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:59.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:09:58 smithi123 ceph-mon[35977]: pgmap v374: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:09:59.567 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-02T23:09:59.824 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:00 smithi174 ceph-mon[35994]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:01.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:00 smithi174 ceph-mon[35994]: overall HEALTH_OK 2024-06-02T23:10:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:00 smithi052 ceph-mon[32340]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:00 smithi052 ceph-mon[32340]: overall HEALTH_OK 2024-06-02T23:10:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:00 smithi123 ceph-mon[35977]: pgmap v375: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:01.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:00 smithi123 ceph-mon[35977]: overall HEALTH_OK 2024-06-02T23:10:01.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 -- 172.21.15.52:0/1546840017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71410ee90 msgr2=0x7fa714111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 --2- 172.21.15.52:0/1546840017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71410ee90 0x7fa714111280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00b0a0 tx=0x7fa70c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 -- 172.21.15.52:0/1546840017 shutdown_connections 2024-06-02T23:10:01.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 --2- 172.21.15.52:0/1546840017 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa71410ee90 0x7fa714111280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 --2- 172.21.15.52:0/1546840017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7140746c0 0x7fa714074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 --2- 172.21.15.52:0/1546840017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714074fe0 0x7fa71410e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 -- 172.21.15.52:0/1546840017 >> 172.21.15.52:0/1546840017 conn(0x7fa714079530 msgr2=0x7fa714079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 -- 172.21.15.52:0/1546840017 shutdown_connections 2024-06-02T23:10:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.521+0000 7fa71908e700 1 -- 172.21.15.52:0/1546840017 wait complete. 2024-06-02T23:10:01.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.522+0000 7fa71908e700 1 Processor -- start 2024-06-02T23:10:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.522+0000 7fa71908e700 1 -- start start 2024-06-02T23:10:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71908e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 0x7fa7141a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71908e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714074fe0 0x7fa7141a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71908e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa71410ee90 0x7fa71419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71908e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa714114820 con 0x7fa7140746c0 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71908e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141146a0 con 0x7fa714074fe0 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71908e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141149a0 con 0x7fa71410ee90 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 0x7fa7141a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 0x7fa7141a0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35374/0 (socket says 172.21.15.52:35374) 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 -- 172.21.15.52:0/349266126 learned_addr learned my addr 172.21.15.52:0/349266126 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71259c700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714074fe0 0x7fa7141a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71359e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa71410ee90 0x7fa71419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 -- 172.21.15.52:0/349266126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa71410ee90 msgr2=0x7fa71419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa71410ee90 0x7fa71419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 -- 172.21.15.52:0/349266126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714074fe0 msgr2=0x7fa7141a0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714074fe0 0x7fa7141a0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 -- 172.21.15.52:0/349266126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa71419b260 con 0x7fa7140746c0 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa71359e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa71410ee90 0x7fa71419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.523+0000 7fa712d9d700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 0x7fa7141a0940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa70400c970 tx=0x7fa70400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.524+0000 7fa70bfff700 1 -- 172.21.15.52:0/349266126 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704007aa0 con 0x7fa7140746c0 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.524+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa71419b550 con 0x7fa7140746c0 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.524+0000 7fa70bfff700 1 -- 172.21.15.52:0/349266126 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa704004ce0 con 0x7fa7140746c0 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.524+0000 7fa70bfff700 1 -- 172.21.15.52:0/349266126 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa704005620 con 0x7fa7140746c0 2024-06-02T23:10:01.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.524+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7141a7810 con 0x7fa7140746c0 2024-06-02T23:10:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.526+0000 7fa70bfff700 1 -- 172.21.15.52:0/349266126 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa704004750 con 0x7fa7140746c0 2024-06-02T23:10:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.527+0000 7fa70bfff700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7000a2c20 0x7fa7000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.527+0000 7fa71259c700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7000a2c20 0x7fa7000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.527+0000 7fa70bfff700 1 -- 172.21.15.52:0/349266126 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa7040cee80 con 0x7fa7140746c0 2024-06-02T23:10:01.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.527+0000 7fa71259c700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7000a2c20 0x7fa7000a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc005fd0 tx=0x7fa6fc005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.527+0000 7fa723911b80 1 Processor -- start 2024-06-02T23:10:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.527+0000 7fa723911b80 1 -- start start 2024-06-02T23:10:01.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa723911b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e183f10 0x55f27e1842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa723911b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e17e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa723911b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 0x55f27e17eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa723911b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f27e070f10 con 0x55f27e17e750 2024-06-02T23:10:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa723911b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f27e0cf800 con 0x55f27e184830 2024-06-02T23:10:01.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa723911b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f27e037410 con 0x55f27e183f10 2024-06-02T23:10:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 0x55f27e17eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 0x55f27e17eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35376/0 (socket says 172.21.15.52:35376) 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 -- 172.21.15.52:0/1982465741 learned_addr learned my addr 172.21.15.52:0/1982465741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa712d9d700 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e183f10 0x55f27e1842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71259c700 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e17e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 -- 172.21.15.52:0/1982465741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e183f10 msgr2=0x55f27e1842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e183f10 0x55f27e1842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 -- 172.21.15.52:0/1982465741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 msgr2=0x55f27e17e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e17e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 -- 172.21.15.52:0/1982465741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27e01e880 con 0x55f27e17e750 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71259c700 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e17e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.528+0000 7fa71359e700 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 0x55f27e17eb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00b070 tx=0x7fa70c009150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4caa1a31b4943c0b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:01.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa7097fa700 1 -- 172.21.15.52:0/1982465741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c020030 con 0x55f27e17e750 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa7097fa700 1 -- 172.21.15.52:0/1982465741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa70c009420 con 0x55f27e17e750 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa7097fa700 1 -- 172.21.15.52:0/1982465741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c01c440 con 0x55f27e17e750 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 -- 172.21.15.52:0/1982465741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 msgr2=0x55f27e17eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 0x55f27e17eb70 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00b070 tx=0x7fa70c009150 comp rx=0 tx=0).stop 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 -- 172.21.15.52:0/1982465741 shutdown_connections 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e17e750 0x55f27e17eb70 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e183f10 0x55f27e1842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 --2- 172.21.15.52:0/1982465741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e17e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.529+0000 7fa723911b80 1 -- 172.21.15.52:0/1982465741 >> 172.21.15.52:0/1982465741 conn(0x55f27e0ab030 msgr2=0x55f27e1791d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 -- 172.21.15.52:0/1982465741 shutdown_connections 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 -- 172.21.15.52:0/1982465741 wait complete. 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 -- start start 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 Processor -- start 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e18a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e187840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e187d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f27e06ff70 con 0x55f27e183f10 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.530+0000 7fa723911b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f27e0cb7c0 con 0x55f27e17e750 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa723911b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f27e18b1b0 con 0x55f27e184830 2024-06-02T23:10:01.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa723911b80 10 client.0 _ll_register_callbacks cb 0x55f27e065560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e187840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e187840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35380/0 (socket says 172.21.15.52:35380) 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 -- 172.21.15.52:0/1718934480 learned_addr learned my addr 172.21.15.52:0/1718934480 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa712d9d700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e18a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71359e700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e187d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 -- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 msgr2=0x55f27e187d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e187d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 -- 172.21.15.52:0/1718934480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 msgr2=0x55f27e18a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e18a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 -- 172.21.15.52:0/1718934480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27e095700 con 0x55f27e183f10 2024-06-02T23:10:01.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa71259c700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e187840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc010780 tx=0x7fa6fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d97aaa4aa2c2fe4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa6fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/1718934480 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc004a30 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/1718934480 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa6fc004bd0 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.531+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa70c009420 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/1718934480 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6fc004dd0 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 10 client.15030 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f27e1958a0 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/1718934480 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa70c009420 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/1718934480 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6fc01d980 con 0x55f27e183f10 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 10 client.15030 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:01.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 10 client.15030 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 20 client.15030 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e197f60 0x55f27e1993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 4 client.15030 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f27e0dcd00 con 0x55f27e197f60 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.532+0000 7fa712d9d700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e197f60 0x55f27e1993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.533+0000 7fa712d9d700 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e197f60 0x55f27e1993d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e4dca0cd0e2d88f server_cookie=d22f94521aaa3dca in_seq=0 out_seq=0 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.533+0000 7fa6fa7fc700 10 client.15030 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/1718934480 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f27e0dcd00 con 0x55f27e197f60 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa6fa7fc700 10 client.15030 handle_command_reply: tid=0 2024-06-02T23:10:01.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 client.15030 shutdown 2024-06-02T23:10:01.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 msgr2=0x55f27e187840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e187840 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc010780 tx=0x7fa6fc004750 comp rx=0 tx=0).stop 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 shutdown_connections 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e197f60 0x55f27e1993d0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e187d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e18a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa6fa7fc700 0 client.15030 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:01.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 --2- 172.21.15.52:0/1718934480 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e187840 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 >> 172.21.15.52:0/1718934480 conn(0x55f27e0ab030 msgr2=0x55f27e1786b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 shutdown_connections 2024-06-02T23:10:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 1 -- 172.21.15.52:0/1718934480 wait complete. 2024-06-02T23:10:01.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.534+0000 7fa723911b80 20 client.15030 trim_cache size 0 max 16384 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.555+0000 7fa723911b80 1 Processor -- start 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.555+0000 7fa723911b80 1 -- start start 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa723911b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e17e750 0x55f27e17e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa723911b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e182000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa723911b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e1853e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa723911b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f27e197710 con 0x55f27e183f10 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa723911b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f27e197de0 con 0x55f27e184830 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa723911b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f27e1984b0 con 0x55f27e17e750 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e182000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e182000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35390/0 (socket says 172.21.15.52:35390) 2024-06-02T23:10:01.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 -- 172.21.15.52:0/2798846890 learned_addr learned my addr 172.21.15.52:0/2798846890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71359e700 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e1853e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa712d9d700 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e17e750 0x55f27e17e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 -- 172.21.15.52:0/2798846890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e17e750 msgr2=0x55f27e17e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e17e750 0x55f27e17e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 -- 172.21.15.52:0/2798846890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 msgr2=0x55f27e1853e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e1853e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 -- 172.21.15.52:0/2798846890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f27e18ac80 con 0x55f27e183f10 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa712d9d700 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e17e750 0x55f27e17e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.556+0000 7fa71259c700 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e182000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc013040 tx=0x7fa6fc00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e031a3f6b54440db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa708ff9700 1 -- 172.21.15.52:0/2798846890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc03f070 con 0x55f27e183f10 2024-06-02T23:10:01.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa708ff9700 1 -- 172.21.15.52:0/2798846890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6fc01d630 con 0x55f27e183f10 2024-06-02T23:10:01.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa708ff9700 1 -- 172.21.15.52:0/2798846890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc005240 con 0x55f27e183f10 2024-06-02T23:10:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 -- 172.21.15.52:0/2798846890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 msgr2=0x55f27e182000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e182000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc013040 tx=0x7fa6fc00efd0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 -- 172.21.15.52:0/2798846890 shutdown_connections 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e184830 0x55f27e1853e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e17e750 0x55f27e17e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 --2- 172.21.15.52:0/2798846890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e182000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 -- 172.21.15.52:0/2798846890 >> 172.21.15.52:0/2798846890 conn(0x55f27e0ab030 msgr2=0x55f27e184cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 -- 172.21.15.52:0/2798846890 shutdown_connections 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.557+0000 7fa723911b80 1 -- 172.21.15.52:0/2798846890 wait complete. 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 -- start start 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 Processor -- start 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e178840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e178d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e1792c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f27e199250 con 0x55f27e183f10 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f27e198b80 con 0x55f27e17e750 2024-06-02T23:10:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f27e199920 con 0x55f27e184830 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa723911b80 10 client.0 _ll_register_callbacks cb 0x55f27e065560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa71259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e178d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa71259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e178d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35404/0 (socket says 172.21.15.52:35404) 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa71259c700 1 -- 172.21.15.52:0/3414541292 learned_addr learned my addr 172.21.15.52:0/3414541292 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa712d9d700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e178840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.558+0000 7fa71359e700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e1792c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa71259c700 1 -- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 msgr2=0x55f27e1792c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa71259c700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e1792c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa71259c700 1 -- 172.21.15.52:0/3414541292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 msgr2=0x55f27e178840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa71259c700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e178840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa71259c700 1 -- 172.21.15.52:0/3414541292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f27e1a0fa0 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa71259c700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e178d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00e8e0 tx=0x7fa6fc01da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=856b92419cf21fd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa6fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/3414541292 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6fc03f030 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/3414541292 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa6fc036050 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6fc01d630 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/3414541292 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fa6fc010d70 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa723911b80 10 client.15036 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f27e20c420 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/3414541292 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6fc01d630 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.559+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/3414541292 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fa6fc0257f0 con 0x55f27e183f10 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 10 client.15036 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 10 client.15036 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 20 client.15036 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e210c20 0x55f27e20f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 4 client.15036 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "status"}) -- 0x55f27e0d7280 con 0x55f27e210c20 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa712d9d700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e210c20 0x55f27e20f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa712d9d700 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e210c20 0x55f27e20f930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6f7d196f3e854e server_cookie=eb017cd4b152961e in_seq=0 out_seq=0 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa6fa7fc700 10 client.15036 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa6fa7fc700 1 -- 172.21.15.52:0/3414541292 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f27e0d7280 con 0x55f27e210c20 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa6fa7fc700 10 client.15036 handle_command_reply: tid=0 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.560+0000 7fa723911b80 1 client.15036 shutdown 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 msgr2=0x55f27e178d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e178d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc00e8e0 tx=0x7fa6fc01da90 comp rx=0 tx=0).stop 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 shutdown_connections 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f27e210c20 0x55f27e20f930 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f27e184830 0x55f27e1792c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f27e17e750 0x55f27e178840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 --2- 172.21.15.52:0/3414541292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f27e183f10 0x55f27e178d80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 >> 172.21.15.52:0/3414541292 conn(0x55f27e0ab030 msgr2=0x55f27e189070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa6fa7fc700 0 client.15036 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 shutdown_connections 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 1 -- 172.21.15.52:0/3414541292 wait complete. 2024-06-02T23:10:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa723911b80 20 client.15036 trim_cache size 0 max 16384 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24451, 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 133.315758765, 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:01.568 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:01.569 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 176.88831237299999 2024-06-02T23:10:01.569 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7000a2c20 msgr2=0x7fa7000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa71908e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7000a2c20 0x7fa7000a50e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc005fd0 tx=0x7fa6fc005db0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 msgr2=0x7fa7141a0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.561+0000 7fa71908e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 0x7fa7141a0940 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa70400c970 tx=0x7fa70400cd30 comp rx=0 tx=0).stop 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 shutdown_connections 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7000a2c20 0x7fa7000a50e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa71410ee90 0x7fa71419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7140746c0 0x7fa7141a0940 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 --2- 172.21.15.52:0/349266126 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714074fe0 0x7fa7141a0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 >> 172.21.15.52:0/349266126 conn(0x7fa714079530 msgr2=0x7fa7140ff700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 shutdown_connections 2024-06-02T23:10:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:01.562+0000 7fa71908e700 1 -- 172.21.15.52:0/349266126 wait complete. 2024-06-02T23:10:02.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-02T23:10:02.608 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:03.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:02 smithi052 ceph-mon[32340]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:03.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:02 smithi174 ceph-mon[35994]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:03.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:02 smithi123 ceph-mon[35977]: pgmap v376: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 -- 172.21.15.52:0/120095957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 msgr2=0x7f5b9c102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/120095957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c102b20 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0099c0 tx=0x7f5b8c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 -- 172.21.15.52:0/120095957 shutdown_connections 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/120095957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c107840 0x7f5b9c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/120095957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c102b20 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/120095957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c1030f0 0x7f5b9c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 -- 172.21.15.52:0/120095957 >> 172.21.15.52:0/120095957 conn(0x7f5b9c0795f0 msgr2=0x7f5b9c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 -- 172.21.15.52:0/120095957 shutdown_connections 2024-06-02T23:10:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.320+0000 7f5ba09d5700 1 -- 172.21.15.52:0/120095957 wait complete. 2024-06-02T23:10:04.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.321+0000 7f5ba09d5700 1 Processor -- start 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 -- start start 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c19e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c1030f0 0x7f5b9c19ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c107840 0x7f5b9c198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c112610 con 0x7f5b9c102740 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c112490 con 0x7f5b9c107840 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5ba09d5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b9c112790 con 0x7f5b9c1030f0 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c19e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c19e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35008/0 (socket says 172.21.15.52:35008) 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 -- 172.21.15.52:0/945656300 learned_addr learned my addr 172.21.15.52:0/945656300 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c1030f0 0x7f5b9c19ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c107840 0x7f5b9c198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 -- 172.21.15.52:0/945656300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c1030f0 msgr2=0x7f5b9c19ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c1030f0 0x7f5b9c19ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 -- 172.21.15.52:0/945656300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c107840 msgr2=0x7f5b9c198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c107840 0x7f5b9c198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.322+0000 7f5b9a59c700 1 -- 172.21.15.52:0/945656300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b9c198ff0 con 0x7f5b9c102740 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.323+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c19e690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009fd0 tx=0x7f5b8c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.323+0000 7f5b93fff700 1 -- 172.21.15.52:0/945656300 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c027070 con 0x7f5b9c102740 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.323+0000 7f5b93fff700 1 -- 172.21.15.52:0/945656300 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b8c003fd0 con 0x7f5b9c102740 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.323+0000 7f5b93fff700 1 -- 172.21.15.52:0/945656300 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c022d00 con 0x7f5b9c102740 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.323+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b9c199280 con 0x7f5b9c102740 2024-06-02T23:10:04.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.323+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b9c0746c0 con 0x7f5b9c102740 2024-06-02T23:10:04.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.326+0000 7f5b93fff700 1 -- 172.21.15.52:0/945656300 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b8c02a070 con 0x7f5b9c102740 2024-06-02T23:10:04.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b93fff700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b880a2c20 0x7f5b880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 Processor -- start 2024-06-02T23:10:04.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b880a2c20 0x7f5b880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b93fff700 1 -- 172.21.15.52:0/945656300 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b8c0cfed0 con 0x7f5b9c102740 2024-06-02T23:10:04.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 -- start start 2024-06-02T23:10:04.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32338f10 0x562d323392f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d32333210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32333750 0x562d32333b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b880a2c20 0x7f5b880a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c108b90 tx=0x7f5b8400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562d32225f10 con 0x562d32339830 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d32284800 con 0x562d32333750 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5bab258b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d321ec410 con 0x562d32338f10 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b91d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d32333210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b91d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d32333210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35022/0 (socket says 172.21.15.52:35022) 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b91d9b700 1 -- 172.21.15.52:0/957492568 learned_addr learned my addr 172.21.15.52:0/957492568 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.327+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32338f10 0x562d323392f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32333750 0x562d32333b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b91d9b700 1 -- 172.21.15.52:0/957492568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32338f10 msgr2=0x562d323392f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32338f10 0x562d323392f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b91d9b700 1 -- 172.21.15.52:0/957492568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32333750 msgr2=0x562d32333b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32333750 0x562d32333b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b91d9b700 1 -- 172.21.15.52:0/957492568 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d321d3880 con 0x562d32339830 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32333750 0x562d32333b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d32333210 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400fea0 tx=0x7f5b84012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41668d7a620daa3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b90d99700 1 -- 172.21.15.52:0/957492568 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84016070 con 0x562d32339830 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b90d99700 1 -- 172.21.15.52:0/957492568 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b84014070 con 0x562d32339830 2024-06-02T23:10:04.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.328+0000 7f5b90d99700 1 -- 172.21.15.52:0/957492568 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b84010d90 con 0x562d32339830 2024-06-02T23:10:04.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 -- 172.21.15.52:0/957492568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 msgr2=0x562d32333210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d32333210 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5b8400fea0 tx=0x7f5b84012000 comp rx=0 tx=0).stop 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 -- 172.21.15.52:0/957492568 shutdown_connections 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32333750 0x562d32333b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32338f10 0x562d323392f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 --2- 172.21.15.52:0/957492568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d32333210 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 -- 172.21.15.52:0/957492568 >> 172.21.15.52:0/957492568 conn(0x562d32260030 msgr2=0x562d3232e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 -- 172.21.15.52:0/957492568 shutdown_connections 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 -- 172.21.15.52:0/957492568 wait complete. 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.329+0000 7f5bab258b80 1 -- start start 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 Processor -- start 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3233c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3233cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562d32224f70 con 0x562d32333750 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d322807c0 con 0x562d32338f10 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d32340210 con 0x562d32339830 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5bab258b80 10 client.0 _ll_register_callbacks cb 0x562d3221a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35038/0 (socket says 172.21.15.52:35038) 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 -- 172.21.15.52:0/1261398488 learned_addr learned my addr 172.21.15.52:0/1261398488 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3233c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3233cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 -- 172.21.15.52:0/1261398488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 msgr2=0x562d3233cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3233cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 -- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 msgr2=0x562d3233c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3233c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 -- 172.21.15.52:0/1261398488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d3224a700 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233f570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0cd9a0 tx=0x7f5b8c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328fb42c26bd4fe1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.330+0000 7f5b81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5b81ffb700 1 -- 172.21.15.52:0/1261398488 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0cde60 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5b81ffb700 1 -- 172.21.15.52:0/1261398488 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b8c09aa90 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b84014070 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5b81ffb700 1 -- 172.21.15.52:0/1261398488 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b8c09c790 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 10 client.15048 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d3234a970 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5b81ffb700 1 -- 172.21.15.52:0/1261398488 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b84014070 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5b81ffb700 1 -- 172.21.15.52:0/1261398488 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b8c0a4ee0 con 0x562d32333750 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 10 client.15048 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 20 client.15048 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d3234cf80 0x562d3234e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 4 client.15048 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.331+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562d32291d00 con 0x562d3234cf80 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.332+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d3234cf80 0x562d3234e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.332+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d3234cf80 0x562d3234e3f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7aeeab452aa3d8ef server_cookie=449982f9e266d595 in_seq=0 out_seq=0 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.332+0000 7f5b81ffb700 10 client.15048 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5b81ffb700 1 -- 172.21.15.52:0/1261398488 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562d32291d00 con 0x562d3234cf80 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5b81ffb700 10 client.15048 handle_command_reply: tid=0 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 client.15048 shutdown 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 msgr2=0x562d3233f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233f570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0cd9a0 tx=0x7f5b8c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 shutdown_connections 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d3234cf80 0x562d3234e3f0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3233cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233f570 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 --2- 172.21.15.52:0/1261398488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3233c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 >> 172.21.15.52:0/1261398488 conn(0x562d32260030 msgr2=0x562d3232d6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5b81ffb700 0 client.15048 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 shutdown_connections 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 1 -- 172.21.15.52:0/1261398488 wait complete. 2024-06-02T23:10:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.333+0000 7f5bab258b80 20 client.15048 trim_cache size 0 max 16384 2024-06-02T23:10:04.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.354+0000 7f5bab258b80 1 Processor -- start 2024-06-02T23:10:04.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.354+0000 7f5bab258b80 1 -- start start 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3232e370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5bab258b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562d3234c730 con 0x562d32333750 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5bab258b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d3234ce00 con 0x562d32338f10 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5bab258b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d3234d4d0 con 0x562d32339830 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35040/0 (socket says 172.21.15.52:35040) 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 -- 172.21.15.52:0/2975369812 learned_addr learned my addr 172.21.15.52:0/2975369812 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 -- 172.21.15.52:0/2975369812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 msgr2=0x562d3232e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3232e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 -- 172.21.15.52:0/2975369812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 msgr2=0x562d3232de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 -- 172.21.15.52:0/2975369812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562d3233fcb0 con 0x562d32333750 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233d500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c099160 tx=0x7f5b8c098b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd339b3a998ee0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b83fff700 1 -- 172.21.15.52:0/2975369812 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c0cde60 con 0x562d32333750 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b83fff700 1 -- 172.21.15.52:0/2975369812 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b8c09c9f0 con 0x562d32333750 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.355+0000 7f5b83fff700 1 -- 172.21.15.52:0/2975369812 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c09bd20 con 0x562d32333750 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 -- 172.21.15.52:0/2975369812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 msgr2=0x562d3233d500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233d500 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c099160 tx=0x7f5b8c098b60 comp rx=0 tx=0).stop 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 -- 172.21.15.52:0/2975369812 shutdown_connections 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32339830 0x562d3232e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32333750 0x562d3233d500 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 --2- 172.21.15.52:0/2975369812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 -- 172.21.15.52:0/2975369812 >> 172.21.15.52:0/2975369812 conn(0x562d32260030 msgr2=0x562d3233b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 -- 172.21.15.52:0/2975369812 shutdown_connections 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 -- 172.21.15.52:0/2975369812 wait complete. 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 -- start start 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.356+0000 7f5bab258b80 1 Processor -- start 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32333750 0x562d32357cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d3232d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562d3234e270 con 0x562d32339830 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562d3234dba0 con 0x562d32338f10 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562d3234e940 con 0x562d32333750 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 10 client.0 _ll_register_callbacks cb 0x562d3221a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d3232d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d3232d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35052/0 (socket says 172.21.15.52:35052) 2024-06-02T23:10:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 -- 172.21.15.52:0/3490946107 learned_addr learned my addr 172.21.15.52:0/3490946107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b91d9b700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32333750 0x562d32357cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 -- 172.21.15.52:0/3490946107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32333750 msgr2=0x562d32357cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32333750 0x562d32357cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 -- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 msgr2=0x562d3232d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 -- 172.21.15.52:0/3490946107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562d3232da80 con 0x562d32339830 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32333750 0x562d32357cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b9ad9d700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d3232d540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5b94013db0 tx=0x7f5b94011530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd3b84b468202d18 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b81ffb700 1 -- 172.21.15.52:0/3490946107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b94013ea0 con 0x562d32339830 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5b81ffb700 1 -- 172.21.15.52:0/3490946107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b94011cd0 con 0x562d32339830 2024-06-02T23:10:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.357+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b8c09c9f0 con 0x562d32339830 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5b81ffb700 1 -- 172.21.15.52:0/3490946107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5b94027050 con 0x562d32339830 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 10 client.15054 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x562d32357550 con 0x562d32339830 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5b81ffb700 1 -- 172.21.15.52:0/3490946107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b8c09c9f0 con 0x562d32339830 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5b81ffb700 1 -- 172.21.15.52:0/3490946107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5b94004690 con 0x562d32339830 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 10 client.15054 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 10 client.15054 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 20 client.15054 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d323bd160 0x562d323bbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 4 client.15054 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "status"}) -- 0x562d3228c280 con 0x562d323bd160 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.358+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d323bd160 0x562d323bbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5b9a59c700 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d323bd160 0x562d323bbe70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4c0074be3407481 server_cookie=afbf1383143a6e54 in_seq=0 out_seq=0 2024-06-02T23:10:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5b81ffb700 10 client.15054 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5b81ffb700 1 -- 172.21.15.52:0/3490946107 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x562d3228c280 con 0x562d323bd160 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5b81ffb700 10 client.15054 handle_command_reply: tid=0 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 client.15054 shutdown 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 msgr2=0x562d3232d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d3232d540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5b94013db0 tx=0x7f5b94011530 comp rx=0 tx=0).stop 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 shutdown_connections 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562d323bd160 0x562d323bbe70 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562d32339830 0x562d3232d540 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562d32333750 0x562d32357cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5b81ffb700 0 client.15054 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 --2- 172.21.15.52:0/3490946107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562d32338f10 0x562d3232d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 >> 172.21.15.52:0/3490946107 conn(0x562d32260030 msgr2=0x562d32339c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 shutdown_connections 2024-06-02T23:10:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 1 -- 172.21.15.52:0/3490946107 wait complete. 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.359+0000 7f5bab258b80 20 client.15054 trim_cache size 0 max 16384 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24470, 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 150.15742411900001, 2024-06-02T23:10:04.366 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:04.367 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:04.367 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:04.367 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 168.85642143000001 2024-06-02T23:10:04.367 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:04.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b880a2c20 msgr2=0x7f5b880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b880a2c20 0x7f5b880a50e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c108b90 tx=0x7f5b8400a380 comp rx=0 tx=0).stop 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 msgr2=0x7f5b9c19e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c19e690 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009fd0 tx=0x7f5b8c0047b0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 shutdown_connections 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b880a2c20 0x7f5b880a50e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b9c107840 0x7f5b9c198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b9c102740 0x7f5b9c19e690 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 --2- 172.21.15.52:0/945656300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b9c1030f0 0x7f5b9c19ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 >> 172.21.15.52:0/945656300 conn(0x7f5b9c0795f0 msgr2=0x7f5b9c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 shutdown_connections 2024-06-02T23:10:04.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:04.360+0000 7f5ba09d5700 1 -- 172.21.15.52:0/945656300 wait complete. 2024-06-02T23:10:05.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:04 smithi174 ceph-mon[35994]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:05.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:04 smithi052 ceph-mon[32340]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:05.252 INFO:tasks.cephadm:Setup complete, yielding 2024-06-02T23:10:05.252 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-02T23:10:05.264 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:10:05.264 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch status' 2024-06-02T23:10:05.265 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:04 smithi123 ceph-mon[35977]: pgmap v377: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:05.528 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:07.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:06 smithi052 ceph-mon[32340]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:06 smithi174 ceph-mon[35994]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:07.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:06 smithi123 ceph-mon[35977]: pgmap v378: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/376415460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4102540 msgr2=0x7fe4a4102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/376415460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4102540 0x7fe4a4102960 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe49400b0d0 tx=0x7fe49400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/376415460 shutdown_connections 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/376415460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a41040e0 0x7fe4a410a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/376415460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4102540 0x7fe4a4102960 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/376415460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4103740 0x7fe4a4103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/376415460 >> 172.21.15.52:0/376415460 conn(0x7fe4a40fdae0 msgr2=0x7fe4a40fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/376415460 shutdown_connections 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.396+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/376415460 wait complete. 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 Processor -- start 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 -- start start 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4102540 0x7fe4a4198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 0x7fe4a4198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:07.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a41040e0 0x7fe4a419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a410e030 con 0x7fe4a4103740 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a410deb0 con 0x7fe4a41040e0 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a8ba2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a410e1b0 con 0x7fe4a4102540 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 0x7fe4a4198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 0x7fe4a4198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35072/0 (socket says 172.21.15.52:35072) 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a1d9b700 1 -- 172.21.15.52:0/3052177711 learned_addr learned my addr 172.21.15.52:0/3052177711 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.397+0000 7fe4a2d9d700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a41040e0 0x7fe4a419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a259c700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4102540 0x7fe4a4198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a1d9b700 1 -- 172.21.15.52:0/3052177711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4102540 msgr2=0x7fe4a4198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a1d9b700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4102540 0x7fe4a4198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a1d9b700 1 -- 172.21.15.52:0/3052177711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a41040e0 msgr2=0x7fe4a419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a1d9b700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a41040e0 0x7fe4a419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a1d9b700 1 -- 172.21.15.52:0/3052177711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a41a02d0 con 0x7fe4a4103740 2024-06-02T23:10:07.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a1d9b700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 0x7fe4a4198bb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b560 tx=0x7fe48c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c015020 con 0x7fe4a4103740 2024-06-02T23:10:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a41a05c0 con 0x7fe4a4103740 2024-06-02T23:10:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe48c011ce0 con 0x7fe4a4103740 2024-06-02T23:10:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe48c010cb0 con 0x7fe4a4103740 2024-06-02T23:10:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.398+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a41a0b80 con 0x7fe4a4103740 2024-06-02T23:10:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.399+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4a4069180 con 0x7fe4a4103740 2024-06-02T23:10:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.400+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe48c018020 con 0x7fe4a4103740 2024-06-02T23:10:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.401+0000 7fe49b7fe700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4900a2c20 0x7fe4900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.401+0000 7fe4a259c700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4900a2c20 0x7fe4900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.401+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe48c0ce780 con 0x7fe4a4103740 2024-06-02T23:10:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.401+0000 7fe4a259c700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4900a2c20 0x7fe4900a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe494009040 tx=0x7fe49400b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.406+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe48c01f020 con 0x7fe4a4103740 2024-06-02T23:10:07.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.630+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe4a40628d0 con 0x7fe4900a2c20 2024-06-02T23:10:07.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.631+0000 7fe49b7fe700 1 -- 172.21.15.52:0/3052177711 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe4a40628d0 con 0x7fe4900a2c20 2024-06-02T23:10:07.631 INFO:teuthology.orchestra.run.smithi052.stdout:Backend: cephadm 2024-06-02T23:10:07.632 INFO:teuthology.orchestra.run.smithi052.stdout:Available: Yes 2024-06-02T23:10:07.632 INFO:teuthology.orchestra.run.smithi052.stdout:Paused: No 2024-06-02T23:10:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4900a2c20 msgr2=0x7fe4900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4900a2c20 0x7fe4900a50e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe494009040 tx=0x7fe49400b040 comp rx=0 tx=0).stop 2024-06-02T23:10:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 msgr2=0x7fe4a4198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 0x7fe4a4198bb0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe48c00b560 tx=0x7fe48c00b870 comp rx=0 tx=0).stop 2024-06-02T23:10:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 shutdown_connections 2024-06-02T23:10:07.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4900a2c20 0x7fe4900a50e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4a41040e0 0x7fe4a419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4a4102540 0x7fe4a4198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 --2- 172.21.15.52:0/3052177711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4a4103740 0x7fe4a4198bb0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:07.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 >> 172.21.15.52:0/3052177711 conn(0x7fe4a40fdae0 msgr2=0x7fe4a4106a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:07.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 shutdown_connections 2024-06-02T23:10:07.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:07.634+0000 7fe4a8ba2700 1 -- 172.21.15.52:0/3052177711 wait complete. 2024-06-02T23:10:08.419 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ps' 2024-06-02T23:10:08.674 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:09.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:08 smithi052 ceph-mon[32340]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:09.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:08 smithi052 ceph-mon[32340]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:08 smithi174 ceph-mon[35994]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:09.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:08 smithi174 ceph-mon[35994]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:08 smithi123 ceph-mon[35977]: from='client.15060 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:09.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:08 smithi123 ceph-mon[35977]: pgmap v379: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 -- 172.21.15.52:0/994801302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81013e0 msgr2=0x7fd0e8101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 --2- 172.21.15.52:0/994801302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81013e0 0x7fd0e8101800 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80099c0 tx=0x7fd0d8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 -- 172.21.15.52:0/994801302 shutdown_connections 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 --2- 172.21.15.52:0/994801302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8102f80 0x7fd0e8109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 --2- 172.21.15.52:0/994801302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81013e0 0x7fd0e8101800 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 --2- 172.21.15.52:0/994801302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e81025e0 0x7fd0e8102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 -- 172.21.15.52:0/994801302 >> 172.21.15.52:0/994801302 conn(0x7fd0e80fc980 msgr2=0x7fd0e80fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 -- 172.21.15.52:0/994801302 shutdown_connections 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.497+0000 7fd0edc89700 1 -- 172.21.15.52:0/994801302 wait complete. 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.498+0000 7fd0edc89700 1 Processor -- start 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.498+0000 7fd0edc89700 1 -- start start 2024-06-02T23:10:10.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.498+0000 7fd0edc89700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e81013e0 0x7fd0e806b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0edc89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e81025e0 0x7fd0e819d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0edc89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 0x7fd0e819f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0edc89700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810ce50 con 0x7fd0e8102f80 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0edc89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810ccd0 con 0x7fd0e81025e0 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0edc89700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810cfd0 con 0x7fd0e81013e0 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 0x7fd0e819f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 0x7fd0e819f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35090/0 (socket says 172.21.15.52:35090) 2024-06-02T23:10:10.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 -- 172.21.15.52:0/3210228990 learned_addr learned my addr 172.21.15.52:0/3210228990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e77fe700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e81013e0 0x7fd0e806b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e6ffd700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e81025e0 0x7fd0e819d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 -- 172.21.15.52:0/3210228990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e81013e0 msgr2=0x7fd0e806b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e81013e0 0x7fd0e806b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 -- 172.21.15.52:0/3210228990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e81025e0 msgr2=0x7fd0e819d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e81025e0 0x7fd0e819d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 -- 172.21.15.52:0/3210228990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e81a0220 con 0x7fd0e8102f80 2024-06-02T23:10:10.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e7fff700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 0x7fd0e819f950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000ada0 tx=0x7fd0e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0017020 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e000ce30 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e81a0510 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.499+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0005350 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.500+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e81a0a00 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.500+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0e8069180 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.502+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0e000c980 con 0x7fd0e8102f80 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.502+0000 7fd0e4ff9700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c20 0x7fd0d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.502+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd0e00ce760 con 0x7fd0e8102f80 2024-06-02T23:10:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.502+0000 7fd0e77fe700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c20 0x7fd0d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.503+0000 7fd0e77fe700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c20 0x7fd0d00a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000c00 tx=0x7fd0d8005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.507+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd0e0021020 con 0x7fd0e8102f80 2024-06-02T23:10:10.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.728+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fd0e80628d0 con 0x7fd0d00a2c20 2024-06-02T23:10:10.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.742+0000 7fd0e4ff9700 1 -- 172.21.15.52:0/3210228990 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fd0e80628d0 con 0x7fd0d00a2c20 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.a smithi052 running (2m) 38s ago 2m 19.0M - 19.0.0-2946-g3d89accf b045ac545f82 ff3a6c7f7e1e 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.b smithi123 running (2m) 40s ago 2m 25.9M - 19.0.0-2946-g3d89accf b045ac545f82 e52435dd9cf1 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.c smithi174 running (3m) 40s ago 3m 21.8M - 19.0.0-2946-g3d89accf b045ac545f82 2ed17d9c3842 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.d smithi052 running (3m) 38s ago 3m 13.5M - 19.0.0-2946-g3d89accf b045ac545f82 bbe4fc56ae06 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.e smithi123 running (3m) 40s ago 3m 20.2M - 19.0.0-2946-g3d89accf b045ac545f82 558a5a69bcdc 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.f smithi174 running (3m) 40s ago 3m 19.9M - 19.0.0-2946-g3d89accf b045ac545f82 27cc2db1695f 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.g smithi052 running (3m) 38s ago 3m 20.4M - 19.0.0-2946-g3d89accf b045ac545f82 ce1e151382b0 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.h smithi123 running (3m) 40s ago 3m 19.0M - 19.0.0-2946-g3d89accf b045ac545f82 da6cf24d91a3 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.i smithi174 running (3m) 40s ago 3m 19.1M - 19.0.0-2946-g3d89accf b045ac545f82 a7cd1c89f433 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.j smithi052 running (3m) 38s ago 3m 14.3M - 19.0.0-2946-g3d89accf b045ac545f82 5bd0191c1dbb 2024-06-02T23:10:10.743 INFO:teuthology.orchestra.run.smithi052.stdout:mds.k smithi123 running (3m) 40s ago 3m 18.0M - 19.0.0-2946-g3d89accf b045ac545f82 43be9a76630d 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mds.l smithi174 running (3m) 40s ago 3m 18.8M - 19.0.0-2946-g3d89accf b045ac545f82 d579169b9159 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.x smithi052 *:9283,8765 running (10m) 38s ago 10m 516M - 19.0.0-2946-g3d89accf b045ac545f82 b64f2c3c3485 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.y smithi123 *:8443,8765 running (8m) 40s ago 8m 454M - 19.0.0-2946-g3d89accf b045ac545f82 fd83e0704dfe 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mgr.z smithi174 *:8443,8765 running (8m) 40s ago 8m 453M - 19.0.0-2946-g3d89accf b045ac545f82 3a242d6759ea 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mon.a smithi052 running (10m) 38s ago 10m 50.0M 2048M 19.0.0-2946-g3d89accf b045ac545f82 a0857bbe9f85 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mon.b smithi123 running (9m) 40s ago 9m 42.5M 2048M 19.0.0-2946-g3d89accf b045ac545f82 f4179c996544 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:mon.c smithi174 running (9m) 40s ago 9m 39.8M 2048M 19.0.0-2946-g3d89accf b045ac545f82 6521b80193e7 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 smithi052 running (8m) 38s ago 8m 59.2M 4273M 19.0.0-2946-g3d89accf b045ac545f82 c09cb70c2a3b 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 smithi123 running (8m) 40s ago 8m 46.0M 4272M 19.0.0-2946-g3d89accf b045ac545f82 9f50060f8296 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 smithi174 running (7m) 40s ago 7m 57.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e1a5f09d058b 2024-06-02T23:10:10.744 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 smithi052 running (7m) 38s ago 7m 58.1M 4273M 19.0.0-2946-g3d89accf b045ac545f82 a17a66af3593 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 smithi123 running (6m) 40s ago 6m 43.9M 4272M 19.0.0-2946-g3d89accf b045ac545f82 ecdcdb6db14d 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 smithi174 running (6m) 40s ago 6m 55.4M 4272M 19.0.0-2946-g3d89accf b045ac545f82 6ec5aa085ddb 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 smithi052 running (6m) 38s ago 6m 45.0M 4273M 19.0.0-2946-g3d89accf b045ac545f82 3ca98a353a70 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 smithi123 running (5m) 40s ago 5m 54.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 e283f4e5f1d6 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 smithi174 running (5m) 40s ago 5m 46.6M 4272M 19.0.0-2946-g3d89accf b045ac545f82 dea4eddb67e2 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 smithi052 running (4m) 38s ago 4m 44.2M 4273M 19.0.0-2946-g3d89accf b045ac545f82 de75b9ac8b08 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 smithi123 running (4m) 40s ago 4m 44.5M 4272M 19.0.0-2946-g3d89accf b045ac545f82 694d9661983e 2024-06-02T23:10:10.745 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 smithi174 running (3m) 40s ago 3m 53.2M 4272M 19.0.0-2946-g3d89accf b045ac545f82 d24f9ac7caea 2024-06-02T23:10:10.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c20 msgr2=0x7fd0d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:10.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c20 0x7fd0d00a50e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8000c00 tx=0x7fd0d8005fb0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 msgr2=0x7fd0e819f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 0x7fd0e819f950 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fd0e000ada0 tx=0x7fd0e000c330 comp rx=0 tx=0).stop 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 shutdown_connections 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c20 0x7fd0d00a50e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8102f80 0x7fd0e819f950 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e81013e0 0x7fd0e806b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 --2- 172.21.15.52:0/3210228990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e81025e0 0x7fd0e819d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.744+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 >> 172.21.15.52:0/3210228990 conn(0x7fd0e80fc980 msgr2=0x7fd0e8105950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:10.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.745+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 shutdown_connections 2024-06-02T23:10:10.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:10.745+0000 7fd0edc89700 1 -- 172.21.15.52:0/3210228990 wait complete. 2024-06-02T23:10:11.132 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:10 smithi052 ceph-mon[32340]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:10 smithi174 ceph-mon[35994]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:11.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:10 smithi123 ceph-mon[35977]: pgmap v380: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:11.439 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch ls' 2024-06-02T23:10:11.689 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:12.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:11 smithi052 ceph-mon[32340]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:11 smithi174 ceph-mon[35994]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:11 smithi123 ceph-mon[35977]: from='client.15066 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:13.080 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:12 smithi052 ceph-mon[32340]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:12 smithi174 ceph-mon[35994]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:12 smithi123 ceph-mon[35977]: pgmap v381: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:13.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 -- 172.21.15.52:0/111604164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 msgr2=0x7f4b48109830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 --2- 172.21.15.52:0/111604164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48109830 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000b0d0 tx=0x7f4b4000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 -- 172.21.15.52:0/111604164 shutdown_connections 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 --2- 172.21.15.52:0/111604164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48109830 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 --2- 172.21.15.52:0/111604164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b481013e0 0x7f4b48101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 --2- 172.21.15.52:0/111604164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481025e0 0x7f4b48102a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 -- 172.21.15.52:0/111604164 >> 172.21.15.52:0/111604164 conn(0x7f4b480fc980 msgr2=0x7f4b480fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 -- 172.21.15.52:0/111604164 shutdown_connections 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.342+0000 7f4b4df47700 1 -- 172.21.15.52:0/111604164 wait complete. 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 Processor -- start 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 -- start start 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481013e0 0x7f4b48105370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b481025e0 0x7f4b481039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:13.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810cef0 con 0x7f4b48102f80 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810cd70 con 0x7f4b481013e0 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b4df47700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4810d070 con 0x7f4b481025e0 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b47fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b47fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48103f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35104/0 (socket says 172.21.15.52:35104) 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b47fff700 1 -- 172.21.15.52:0/2884219866 learned_addr learned my addr 172.21.15.52:0/2884219866 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.343+0000 7f4b477fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481013e0 0x7f4b48105370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b46ffd700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b481025e0 0x7f4b481039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b47fff700 1 -- 172.21.15.52:0/2884219866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b481025e0 msgr2=0x7f4b481039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b47fff700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b481025e0 0x7f4b481039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b47fff700 1 -- 172.21.15.52:0/2884219866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481013e0 msgr2=0x7f4b48105370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:13.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b47fff700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481013e0 0x7f4b48105370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b47fff700 1 -- 172.21.15.52:0/2884219866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b481047b0 con 0x7f4b48102f80 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b477fe700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481013e0 0x7f4b48105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b47fff700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48103f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009fd0 tx=0x7f4b40004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4001f030 con 0x7f4b48102f80 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b481a26a0 con 0x7f4b48102f80 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b40007a80 con 0x7f4b48102f80 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40004770 con 0x7f4b48102f80 2024-06-02T23:10:13.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.344+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b481a2b70 con 0x7f4b48102f80 2024-06-02T23:10:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.345+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b48069180 con 0x7f4b48102f80 2024-06-02T23:10:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.347+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b40003a00 con 0x7f4b48102f80 2024-06-02T23:10:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.347+0000 7f4b44ff9700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b300a2c20 0x7f4b300a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.348+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4b4000f020 con 0x7f4b48102f80 2024-06-02T23:10:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.348+0000 7f4b477fe700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b300a2c20 0x7f4b300a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:13.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.348+0000 7f4b477fe700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b300a2c20 0x7f4b300a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b48102440 tx=0x7f4b38009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:13.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.351+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4b40029020 con 0x7f4b48102f80 2024-06-02T23:10:13.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.565+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f4b480feab0 con 0x7f4b300a2c20 2024-06-02T23:10:13.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.571+0000 7f4b44ff9700 1 -- 172.21.15.52:0/2884219866 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f4b480feab0 con 0x7f4b300a2c20 2024-06-02T23:10:13.571 INFO:teuthology.orchestra.run.smithi052.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-06-02T23:10:13.571 INFO:teuthology.orchestra.run.smithi052.stdout:mds.all 12/12 43s ago 3m smithi052=a;smithi052=d;smithi052=g;smithi052=j;smithi123=b;smithi123=e;smithi123=h;smithi123=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-06-02T23:10:13.571 INFO:teuthology.orchestra.run.smithi052.stdout:mgr 3/3 43s ago 8m smithi052=x;smithi123=y;smithi174=z;count:3 2024-06-02T23:10:13.571 INFO:teuthology.orchestra.run.smithi052.stdout:mon 3/3 43s ago 9m smithi052:172.21.15.52=a;smithi123:172.21.15.123=b;smithi174:172.21.15.174=c;count:3 2024-06-02T23:10:13.571 INFO:teuthology.orchestra.run.smithi052.stdout:osd 12 43s ago - 2024-06-02T23:10:13.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b300a2c20 msgr2=0x7f4b300a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:13.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b300a2c20 0x7f4b300a50e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4b48102440 tx=0x7f4b38009190 comp rx=0 tx=0).stop 2024-06-02T23:10:13.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 msgr2=0x7f4b48103f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:13.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48103f00 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009fd0 tx=0x7f4b40004e40 comp rx=0 tx=0).stop 2024-06-02T23:10:13.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 shutdown_connections 2024-06-02T23:10:13.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b300a2c20 0x7f4b300a50e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b48102f80 0x7f4b48103f00 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b481013e0 0x7f4b48105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 --2- 172.21.15.52:0/2884219866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b481025e0 0x7f4b481039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:13.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 >> 172.21.15.52:0/2884219866 conn(0x7f4b480fc980 msgr2=0x7f4b480fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:13.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 shutdown_connections 2024-06-02T23:10:13.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:13.573+0000 7f4b4df47700 1 -- 172.21.15.52:0/2884219866 wait complete. 2024-06-02T23:10:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:13 smithi174 ceph-mon[35994]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:14.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:13 smithi052 ceph-mon[32340]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:13 smithi123 ceph-mon[35977]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:14.351 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch host ls' 2024-06-02T23:10:14.605 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:15.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:14 smithi052 ceph-mon[32340]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:15.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:14 smithi174 ceph-mon[35994]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:15.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:14 smithi123 ceph-mon[35977]: pgmap v382: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 -- 172.21.15.52:0/3229281614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100540 msgr2=0x7f29b81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 --2- 172.21.15.52:0/3229281614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100540 0x7f29b81009a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f29ac009990 tx=0x7f29ac009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 -- 172.21.15.52:0/3229281614 shutdown_connections 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 --2- 172.21.15.52:0/3229281614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100ee0 0x7f29b810aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 --2- 172.21.15.52:0/3229281614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 0x7f29b8103ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 --2- 172.21.15.52:0/3229281614 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100540 0x7f29b81009a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 -- 172.21.15.52:0/3229281614 >> 172.21.15.52:0/3229281614 conn(0x7f29b80fc120 msgr2=0x7f29b80fe540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 -- 172.21.15.52:0/3229281614 shutdown_connections 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.368+0000 7f29bd84c700 1 -- 172.21.15.52:0/3229281614 wait complete. 2024-06-02T23:10:16.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 Processor -- start 2024-06-02T23:10:16.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 -- start start 2024-06-02T23:10:16.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 0x7f29b8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:16.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100ee0 0x7f29b8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:16.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 0x7f29b819fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810df60 con 0x7f29b8100ee0 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810dde0 con 0x7f29b8100540 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.369+0000 7f29bd84c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29b810e0e0 con 0x7f29b8103ac0 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 0x7f29b8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 0x7f29b8198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48946/0 (socket says 172.21.15.52:48946) 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 -- 172.21.15.52:0/773361152 learned_addr learned my addr 172.21.15.52:0/773361152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b77fe700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 0x7f29b819fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b67fc700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100ee0 0x7f29b8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 -- 172.21.15.52:0/773361152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 msgr2=0x7f29b819fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 0x7f29b819fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 -- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100ee0 msgr2=0x7f29b8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100ee0 0x7f29b8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 -- 172.21.15.52:0/773361152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29b81a0260 con 0x7f29b8100540 2024-06-02T23:10:16.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b67fc700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100ee0 0x7f29b8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b77fe700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 0x7f29b819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29b6ffd700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 0x7f29b8198600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e970 tx=0x7f29a800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8004220 con 0x7f29b8100540 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29b81a0550 con 0x7f29b8100540 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f29a80043c0 con 0x7f29b8100540 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29a8003d20 con 0x7f29b8100540 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.370+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29b81a0a90 con 0x7f29b8100540 2024-06-02T23:10:16.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.371+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29b8069180 con 0x7f29b8100540 2024-06-02T23:10:16.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.373+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f29a800c7e0 con 0x7f29b8100540 2024-06-02T23:10:16.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.373+0000 7f29bc84a700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f29a00a2c20 0x7f29a00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:16.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.374+0000 7f29b67fc700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f29a00a2c20 0x7f29a00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:16.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.374+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f29a80cf230 con 0x7f29b8100540 2024-06-02T23:10:16.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.374+0000 7f29b67fc700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f29a00a2c20 0x7f29a00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b6b0 tx=0x7f29ac01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:16.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.377+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f29a809c250 con 0x7f29b8100540 2024-06-02T23:10:16.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.592+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f29b80628d0 con 0x7f29a00a2c20 2024-06-02T23:10:16.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.593+0000 7f29bc84a700 1 -- 172.21.15.52:0/773361152 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f29b80628d0 con 0x7f29a00a2c20 2024-06-02T23:10:16.593 INFO:teuthology.orchestra.run.smithi052.stdout:HOST ADDR LABELS STATUS 2024-06-02T23:10:16.594 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 172.21.15.52 2024-06-02T23:10:16.594 INFO:teuthology.orchestra.run.smithi052.stdout:smithi123 172.21.15.123 2024-06-02T23:10:16.594 INFO:teuthology.orchestra.run.smithi052.stdout:smithi174 172.21.15.174 2024-06-02T23:10:16.594 INFO:teuthology.orchestra.run.smithi052.stdout:3 hosts in cluster 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f29a00a2c20 msgr2=0x7f29a00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f29a00a2c20 0x7f29a00a50e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b6b0 tx=0x7f29ac01a040 comp rx=0 tx=0).stop 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 msgr2=0x7f29b8198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 0x7f29b8198600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f29a800e970 tx=0x7f29a800ec80 comp rx=0 tx=0).stop 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 shutdown_connections 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f29a00a2c20 0x7f29a00a50e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29b8103ac0 0x7f29b819fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29b8100540 0x7f29b8198600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 --2- 172.21.15.52:0/773361152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f29b8100ee0 0x7f29b8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.595+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 >> 172.21.15.52:0/773361152 conn(0x7f29b80fc120 msgr2=0x7f29b8106c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:16.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.596+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 shutdown_connections 2024-06-02T23:10:16.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:16.596+0000 7f29bd84c700 1 -- 172.21.15.52:0/773361152 wait complete. 2024-06-02T23:10:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:16 smithi174 ceph-mon[35994]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:16 smithi174 ceph-mon[35994]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:16 smithi052 ceph-mon[32340]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:16 smithi052 ceph-mon[32340]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:17.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:16 smithi123 ceph-mon[35977]: pgmap v383: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:16 smithi123 ceph-mon[35977]: from='client.24806 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:17.409 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph orch device ls' 2024-06-02T23:10:17.659 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:19.146 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:18 smithi052 ceph-mon[32340]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:19.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:18 smithi123 ceph-mon[35977]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:18 smithi174 ceph-mon[35994]: pgmap v384: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 -- 172.21.15.52:0/2482448258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141025e0 msgr2=0x7f9014102a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 --2- 172.21.15.52:0/2482448258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141025e0 0x7f9014102a40 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f90100099c0 tx=0x7f9010009cd0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 -- 172.21.15.52:0/2482448258 shutdown_connections 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 --2- 172.21.15.52:0/2482448258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9014102f80 0x7f9014109830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 --2- 172.21.15.52:0/2482448258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90141013e0 0x7f9014101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 --2- 172.21.15.52:0/2482448258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141025e0 0x7f9014102a40 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 -- 172.21.15.52:0/2482448258 >> 172.21.15.52:0/2482448258 conn(0x7f90140fc960 msgr2=0x7f90140fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 -- 172.21.15.52:0/2482448258 shutdown_connections 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.506+0000 7f901bbad700 1 -- 172.21.15.52:0/2482448258 wait complete. 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 Processor -- start 2024-06-02T23:10:19.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 -- start start 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 0x7f9014105510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90141025e0 0x7f9014103bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9014102f80 0x7f90141040f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f901410ceb0 con 0x7f90141013e0 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f901410cd30 con 0x7f9014102f80 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901bbad700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f901410d030 con 0x7f90141025e0 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f9019949700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 0x7f9014105510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f9019949700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 0x7f9014105510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34006/0 (socket says 172.21.15.52:34006) 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f9019949700 1 -- 172.21.15.52:0/2711274878 learned_addr learned my addr 172.21.15.52:0/2711274878 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f901a14a700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9014102f80 0x7f90141040f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.507+0000 7f9019148700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90141025e0 0x7f9014103bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:19.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9019949700 1 -- 172.21.15.52:0/2711274878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90141025e0 msgr2=0x7f9014103bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9019949700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90141025e0 0x7f9014103bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9019949700 1 -- 172.21.15.52:0/2711274878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9014102f80 msgr2=0x7f90141040f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9019949700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9014102f80 0x7f90141040f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9019949700 1 -- 172.21.15.52:0/2711274878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90141049c0 con 0x7f90141013e0 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9019949700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 0x7f9014105510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f900800c970 tx=0x7f900800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008004220 con 0x7f90141013e0 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9008007d50 con 0x7f90141013e0 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90141a27a0 con 0x7f90141013e0 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9008003d00 con 0x7f90141013e0 2024-06-02T23:10:19.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.508+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90141a2cb0 con 0x7f90141013e0 2024-06-02T23:10:19.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.510+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f901410d670 con 0x7f90141013e0 2024-06-02T23:10:19.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.511+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90080078a0 con 0x7f90141013e0 2024-06-02T23:10:19.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.512+0000 7f9006ffd700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90000a2c20 0x7f90000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:19.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.512+0000 7f9019148700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90000a2c20 0x7f90000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:19.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.512+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f90080cecb0 con 0x7f90141013e0 2024-06-02T23:10:19.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.512+0000 7f9019148700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90000a2c20 0x7f90000a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9010009fd0 tx=0x7f9010005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:19.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.517+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9008014030 con 0x7f90141013e0 2024-06-02T23:10:19.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.735+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f901406b2d0 con 0x7f90000a2c20 2024-06-02T23:10:19.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.736+0000 7f9006ffd700 1 -- 172.21.15.52:0/2711274878 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f901406b2d0 con 0x7f90000a2c20 2024-06-02T23:10:19.736 INFO:teuthology.orchestra.run.smithi052.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-06-02T23:10:19.736 INFO:teuthology.orchestra.run.smithi052.stdout:smithi052 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513010N480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-06-02T23:10:19.736 INFO:teuthology.orchestra.run.smithi052.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-06-02T23:10:19.736 INFO:teuthology.orchestra.run.smithi052.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-06-02T23:10:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90000a2c20 msgr2=0x7f90000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90000a2c20 0x7f90000a50e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9010009fd0 tx=0x7f9010005fb0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 msgr2=0x7f9014105510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 0x7f9014105510 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f900800c970 tx=0x7f900800cc80 comp rx=0 tx=0).stop 2024-06-02T23:10:19.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 shutdown_connections 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90000a2c20 0x7f90000a50e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9014102f80 0x7f90141040f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90141013e0 0x7f9014105510 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 --2- 172.21.15.52:0/2711274878 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90141025e0 0x7f9014103bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 >> 172.21.15.52:0/2711274878 conn(0x7f90140fc960 msgr2=0x7f901406abc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 shutdown_connections 2024-06-02T23:10:19.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:19.739+0000 7f901bbad700 1 -- 172.21.15.52:0/2711274878 wait complete. 2024-06-02T23:10:20.509 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-06-02T23:10:20.518 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:10:20.518 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph fs dump' 2024-06-02T23:10:20.771 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:21.144 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:20 smithi052 ceph-mon[32340]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:21.144 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:20 smithi052 ceph-mon[32340]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:20 smithi123 ceph-mon[35977]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:20 smithi123 ceph-mon[35977]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:20 smithi174 ceph-mon[35994]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-06-02T23:10:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:20 smithi174 ceph-mon[35994]: pgmap v385: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 -- 172.21.15.52:0/1588426665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 msgr2=0x7f3810107ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 --2- 172.21.15.52:0/1588426665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810107ca0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f37fc009990 tx=0x7f37fc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 -- 172.21.15.52:0/1588426665 shutdown_connections 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 --2- 172.21.15.52:0/1588426665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38101081e0 0x7f381010a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 --2- 172.21.15.52:0/1588426665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3810102f80 0x7f3810105370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 --2- 172.21.15.52:0/1588426665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810107ca0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 -- 172.21.15.52:0/1588426665 >> 172.21.15.52:0/1588426665 conn(0x7f38100fc9b0 msgr2=0x7f38100fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 -- 172.21.15.52:0/1588426665 shutdown_connections 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 -- 172.21.15.52:0/1588426665 wait complete. 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.695+0000 7f3816df7700 1 Processor -- start 2024-06-02T23:10:22.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 -- start start 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3810102f80 0x7f3810198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810198ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38101081e0 0x7f381019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f381010dfb0 con 0x7f38101058b0 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f381010de30 con 0x7f3810102f80 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3816df7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f381010e130 con 0x7f38101081e0 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810198ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810198ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34032/0 (socket says 172.21.15.52:34032) 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 -- 172.21.15.52:0/2603782044 learned_addr learned my addr 172.21.15.52:0/2603782044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3814b93700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3810102f80 0x7f3810198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f3815394700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38101081e0 0x7f381019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 -- 172.21.15.52:0/2603782044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38101081e0 msgr2=0x7f381019fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:22.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38101081e0 0x7f381019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 -- 172.21.15.52:0/2603782044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3810102f80 msgr2=0x7f3810198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3810102f80 0x7f3810198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 -- 172.21.15.52:0/2603782044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38101a02c0 con 0x7f38101058b0 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.696+0000 7f380ffff700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810198ba0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005910 tx=0x7f37fc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.697+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc026070 con 0x7f38101058b0 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.697+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37fc00be80 con 0x7f38101058b0 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.697+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38101a0550 con 0x7f38101058b0 2024-06-02T23:10:22.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.697+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37fc017770 con 0x7f38101058b0 2024-06-02T23:10:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.697+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38101a09a0 con 0x7f38101058b0 2024-06-02T23:10:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.698+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3810069180 con 0x7f38101058b0 2024-06-02T23:10:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.699+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37fc029070 con 0x7f38101058b0 2024-06-02T23:10:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.700+0000 7f380dffb700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38000a2c20 0x7f38000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.700+0000 7f3814b93700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38000a2c20 0x7f38000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:22.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.700+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f37fc0d07b0 con 0x7f38101058b0 2024-06-02T23:10:22.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.700+0000 7f3814b93700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38000a2c20 0x7f38000a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3804009bf0 tx=0x7f3804009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:22.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.704+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37fc030070 con 0x7f38101058b0 2024-06-02T23:10:22.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.955+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f381002dcd0 con 0x7f38101058b0 2024-06-02T23:10:22.955 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:22 smithi052 ceph-mon[32340]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:22.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.956+0000 7f380dffb700 1 -- 172.21.15.52:0/2603782044 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) ==== 76+0+3096 (secure 0 0 0) 0x7f37fc013070 con 0x7f38101058b0 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:e27 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:legacy client fscid: 1 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:Filesystem 'cephfs' (1) 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:fs_name cephfs 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 27 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-06-02T23:07:33.296248+0000 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-06-02T23:07:54.810487+0000 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:tableserver 0 2024-06-02T23:10:22.956 INFO:teuthology.orchestra.run.smithi052.stdout:root 0 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:session_timeout 300 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:session_autoclose 300 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:max_file_size 1099511627776 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:max_xattr_size 65536 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:required_client_features {} 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure 0 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:last_failure_osd_epoch 0 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:max_mds 5 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:in 0,1,2,3,4 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:up {0=24470,1=24451,2=24449,3=24436,4=24434} 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:failed 2024-06-02T23:10:22.957 INFO:teuthology.orchestra.run.smithi052.stdout:damaged 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:stopped 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:data_pools [3] 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:metadata_pool 2 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:inline_data disabled 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:balancer 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:bal_rank_mask -1 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:standby_count_wanted 0 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:qdb_cluster leader: 24470 members: 24470,24449,24451,24436,24434 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.b{0:24470} state up:active seq 6 addr [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.f{0:24430} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/2486297365,v1:172.21.15.174:6835/2486297365] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.i{1:24451} state up:active seq 12 addr [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.k{1:24407} state up:standby-replay seq 1 addr [v2:172.21.15.123:6832/1267486579,v1:172.21.15.123:6833/1267486579] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.h{2:24449} state up:active seq 13 addr [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.958 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.l{2:24400} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/224943979,v1:172.21.15.174:6833/224943979] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.c{3:24436} state up:active seq 14 addr [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.a{3:14616} state up:standby-replay seq 1 addr [v2:172.21.15.52:6840/4238066654,v1:172.21.15.52:6841/4238066654] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.e{4:24434} state up:active seq 16 addr [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.g{4:14604} state up:standby-replay seq 1 addr [v2:172.21.15.52:6838/2175393174,v1:172.21.15.52:6839/2175393174] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:Standby daemons: 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.52:6834/3011548911,v1:172.21.15.52:6835/3011548911] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.959 INFO:teuthology.orchestra.run.smithi052.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.52:6836/4013141842,v1:172.21.15.52:6837/4013141842] compat {c=[1],r=[1],i=[1fff]}] 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38000a2c20 msgr2=0x7f38000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38000a2c20 0x7f38000a50e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f3804009bf0 tx=0x7f3804009260 comp rx=0 tx=0).stop 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 msgr2=0x7f3810198ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810198ba0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f37fc005910 tx=0x7f37fc009580 comp rx=0 tx=0).stop 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 shutdown_connections 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38000a2c20 0x7f38000a50e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38101081e0 0x7f381019fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3810102f80 0x7f3810198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 --2- 172.21.15.52:0/2603782044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f38101058b0 0x7f3810198ba0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:22.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 >> 172.21.15.52:0/2603782044 conn(0x7f38100fc9b0 msgr2=0x7f38101038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:22.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 shutdown_connections 2024-06-02T23:10:22.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:22.958+0000 7f3816df7700 1 -- 172.21.15.52:0/2603782044 wait complete. 2024-06-02T23:10:22.961 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:10:23.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:22 smithi123 ceph-mon[35977]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:23.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:22 smithi174 ceph-mon[35994]: pgmap v386: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:23.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- bash -c 'ceph osd dump' 2024-06-02T23:10:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:24.101 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:23 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2603782044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-02T23:10:24.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:23 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2603782044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-02T23:10:24.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:23 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2603782044' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-06-02T23:10:25.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:24 smithi123 ceph-mon[35977]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:25.285 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:24 smithi052 ceph-mon[32340]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:25.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:24 smithi174 ceph-mon[35994]: pgmap v387: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 -- 172.21.15.52:0/690162743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc102f90 msgr2=0x7f61cc109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 --2- 172.21.15.52:0/690162743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc102f90 0x7f61cc109840 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f61c400b0d0 tx=0x7f61c400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 -- 172.21.15.52:0/690162743 shutdown_connections 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 --2- 172.21.15.52:0/690162743 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc102f90 0x7f61cc109840 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 --2- 172.21.15.52:0/690162743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61cc1013f0 0x7f61cc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 --2- 172.21.15.52:0/690162743 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61cc1025f0 0x7f61cc102a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 -- 172.21.15.52:0/690162743 >> 172.21.15.52:0/690162743 conn(0x7f61cc0fc9b0 msgr2=0x7f61cc0fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 -- 172.21.15.52:0/690162743 shutdown_connections 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.657+0000 7f61d327b700 1 -- 172.21.15.52:0/690162743 wait complete. 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 Processor -- start 2024-06-02T23:10:25.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 -- start start 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 0x7f61cc1985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61cc1025f0 0x7f61cc198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61cc102f90 0x7f61cc19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10cee0 con 0x7f61cc1013f0 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10cd60 con 0x7f61cc1025f0 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d327b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61cc10d060 con 0x7f61cc102f90 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d1017700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 0x7f61cc1985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d1017700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 0x7f61cc1985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59140/0 (socket says 172.21.15.52:59140) 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d1017700 1 -- 172.21.15.52:0/1030500886 learned_addr learned my addr 172.21.15.52:0/1030500886 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d1818700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61cc102f90 0x7f61cc19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.658+0000 7f61d0816700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61cc1025f0 0x7f61cc198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:25.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d1017700 1 -- 172.21.15.52:0/1030500886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61cc102f90 msgr2=0x7f61cc19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d1017700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61cc102f90 0x7f61cc19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d1017700 1 -- 172.21.15.52:0/1030500886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61cc1025f0 msgr2=0x7f61cc198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d1017700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61cc1025f0 0x7f61cc198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d1017700 1 -- 172.21.15.52:0/1030500886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61cc1a0250 con 0x7f61cc1013f0 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d1017700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 0x7f61cc1985f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f61c800d9b0 tx=0x7f61c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8017030 con 0x7f61cc1013f0 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61cc1a0540 con 0x7f61cc1013f0 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61c80044d0 con 0x7f61cc1013f0 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c80039e0 con 0x7f61cc1013f0 2024-06-02T23:10:25.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.659+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61cc1a0b50 con 0x7f61cc1013f0 2024-06-02T23:10:25.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.660+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61cc069180 con 0x7f61cc1013f0 2024-06-02T23:10:25.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.661+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61c801a070 con 0x7f61cc1013f0 2024-06-02T23:10:25.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.662+0000 7f61be7fc700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61b80a2c20 0x7f61b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:25.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.662+0000 7f61d0816700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61b80a2c20 0x7f61b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:25.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.662+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f61c80ced40 con 0x7f61cc1013f0 2024-06-02T23:10:25.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.663+0000 7f61d0816700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61b80a2c20 0x7f61b80a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006fd0 tx=0x7f61c0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:25.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.666+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f61c809bc70 con 0x7f61cc1013f0 2024-06-02T23:10:25.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.873+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f61cc1a1030 con 0x7f61cc1013f0 2024-06-02T23:10:25.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.874+0000 7f61be7fc700 1 -- 172.21.15.52:0/1030500886 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5257 (secure 0 0 0) 0x7f61c80bb020 con 0x7f61cc1013f0 2024-06-02T23:10:25.876 INFO:teuthology.orchestra.run.smithi052.stdout:epoch 90 2024-06-02T23:10:25.876 INFO:teuthology.orchestra.run.smithi052.stdout:fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:10:25.876 INFO:teuthology.orchestra.run.smithi052.stdout:created 2024-06-02T22:59:11.730596+0000 2024-06-02T23:10:25.876 INFO:teuthology.orchestra.run.smithi052.stdout:modified 2024-06-02T23:08:11.318645+0000 2024-06-02T23:10:25.876 INFO:teuthology.orchestra.run.smithi052.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-06-02T23:10:25.876 INFO:teuthology.orchestra.run.smithi052.stdout:crush_version 26 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:full_ratio 0.95 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:backfillfull_ratio 0.9 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:nearfull_ratio 0.85 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:require_min_compat_client luminous 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:min_compat_client luminous 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:require_osd_release squid 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:stretch_mode_enabled false 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 88 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:max_osd 12 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:osd.0 up in weight 1 up_from 8 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6802/1323575430,v1:172.21.15.52:6803/1323575430] [v2:172.21.15.52:6804/1323575430,v1:172.21.15.52:6805/1323575430] exists,up 01ccc644-7329-48bb-9a2c-f48aafdfad4f 2024-06-02T23:10:25.877 INFO:teuthology.orchestra.run.smithi052.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/338548510,v1:172.21.15.123:6801/338548510] [v2:172.21.15.123:6802/338548510,v1:172.21.15.123:6803/338548510] exists,up 08319da8-7871-414b-ad99-e708bcd4bed9 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.2 up in weight 1 up_from 18 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/2598349360,v1:172.21.15.174:6801/2598349360] [v2:172.21.15.174:6802/2598349360,v1:172.21.15.174:6803/2598349360] exists,up 4248f78e-ee22-4cd7-92e9-e449a51cb4b1 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6810/2267151243,v1:172.21.15.52:6811/2267151243] [v2:172.21.15.52:6812/2267151243,v1:172.21.15.52:6813/2267151243] exists,up e32ed956-2af4-430f-ac3c-e349190ebfa3 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.4 up in weight 1 up_from 33 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/3937434833,v1:172.21.15.123:6809/3937434833] [v2:172.21.15.123:6810/3937434833,v1:172.21.15.123:6811/3937434833] exists,up a9734ce9-e810-4891-a831-1afaef0f45b6 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.5 up in weight 1 up_from 38 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/1358350436,v1:172.21.15.174:6809/1358350436] [v2:172.21.15.174:6810/1358350436,v1:172.21.15.174:6811/1358350436] exists,up b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.6 up in weight 1 up_from 46 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6818/2952400783,v1:172.21.15.52:6819/2952400783] [v2:172.21.15.52:6820/2952400783,v1:172.21.15.52:6821/2952400783] exists,up 3667d476-c6ce-4999-8954-4ade4fbe51a1 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.7 up in weight 1 up_from 51 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/550996005,v1:172.21.15.123:6817/550996005] [v2:172.21.15.123:6818/550996005,v1:172.21.15.123:6819/550996005] exists,up 0ef49c6a-2339-4f91-9474-0c747c53a20d 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.8 up in weight 1 up_from 59 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/186727045,v1:172.21.15.174:6817/186727045] [v2:172.21.15.174:6818/186727045,v1:172.21.15.174:6819/186727045] exists,up 27a74c0f-68e7-4c68-aaf3-69859a1b47e3 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.9 up in weight 1 up_from 67 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.52:6826/3771079645,v1:172.21.15.52:6827/3771079645] [v2:172.21.15.52:6828/3771079645,v1:172.21.15.52:6829/3771079645] exists,up 482dc129-e665-4a35-add6-b40d6359d13a 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.10 up in weight 1 up_from 72 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/4143797452,v1:172.21.15.123:6825/4143797452] [v2:172.21.15.123:6826/4143797452,v1:172.21.15.123:6827/4143797452] exists,up 8bea87b2-167c-4b06-939d-59e6a5c69b07 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:osd.11 up in weight 1 up_from 80 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/190382346,v1:172.21.15.174:6825/190382346] [v2:172.21.15.174:6826/190382346,v1:172.21.15.174:6827/190382346] exists,up d9af8f08-98ca-4c39-8571-2a6fa0c08075 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.10 [8,11] 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.1b [2,4] 2024-06-02T23:10:25.878 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 2.1d [8,11,2,1] 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.6 [4,6] 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.24 [4,9] 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.25 [5,9] 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:pg_upmap_items 3.2c [4,9] 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/540942425 expires 2024-06-03T23:00:09.671662+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/160135232 expires 2024-06-03T23:00:09.671662+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/2656327338 expires 2024-06-03T23:00:09.671662+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/2620214668 expires 2024-06-03T22:59:43.024153+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6800/540942425 expires 2024-06-03T23:00:09.671662+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1498162594 expires 2024-06-03T22:59:43.024153+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/3265496994 expires 2024-06-03T22:59:43.024153+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/752611388 expires 2024-06-03T22:59:43.024153+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:6801/2620214668 expires 2024-06-03T22:59:43.024153+0000 2024-06-02T23:10:25.879 INFO:teuthology.orchestra.run.smithi052.stdout:blocklist 172.21.15.52:0/1697426194 expires 2024-06-03T23:00:09.671662+0000 2024-06-02T23:10:25.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61b80a2c20 msgr2=0x7f61b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61b80a2c20 0x7f61b80a50e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f61c0006fd0 tx=0x7f61c0008040 comp rx=0 tx=0).stop 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 msgr2=0x7f61cc1985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 0x7f61cc1985f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f61c800d9b0 tx=0x7f61c800dcc0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 shutdown_connections 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61b80a2c20 0x7f61b80a50e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61cc102f90 0x7f61cc19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61cc1013f0 0x7f61cc1985f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 --2- 172.21.15.52:0/1030500886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61cc1025f0 0x7f61cc198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.877+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 >> 172.21.15.52:0/1030500886 conn(0x7f61cc0fc9b0 msgr2=0x7f61cc0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.878+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 shutdown_connections 2024-06-02T23:10:25.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:25.878+0000 7f61d327b700 1 -- 172.21.15.52:0/1030500886 wait complete. 2024-06-02T23:10:26.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:25 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1030500886' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-02T23:10:26.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:25 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1030500886' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-02T23:10:26.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:25 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1030500886' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-06-02T23:10:26.688 INFO:teuthology.run_tasks:Running task fs.ready... 2024-06-02T23:10:26.700 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:10:26.961 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:27.208 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:26 smithi052 ceph-mon[32340]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:27.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:26 smithi123 ceph-mon[35977]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:27.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:26 smithi174 ceph-mon[35994]: pgmap v388: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2042740034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64107400 msgr2=0x7f2f641077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2042740034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64107400 0x7f2f641077e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009990 tx=0x7f2f54009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2042740034 shutdown_connections 2024-06-02T23:10:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2042740034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 0x7f2f6410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2042740034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64107400 0x7f2f641077e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2042740034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f64101390 0x7f2f64101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2042740034 >> 172.21.15.52:0/2042740034 conn(0x7f2f64078340 msgr2=0x7f2f64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2042740034 shutdown_connections 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.639+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2042740034 wait complete. 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.640+0000 7f2f69c8d700 1 Processor -- start 2024-06-02T23:10:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.640+0000 7f2f69c8d700 1 -- start start 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f69c8d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 0x7f2f64103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f69c8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 0x7f2f641035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f69c8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f64107400 0x7f2f64103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f69c8d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f641115f0 con 0x7f2f64101390 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f69c8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64111470 con 0x7f2f64107400 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f69c8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f64111770 con 0x7f2f64101d50 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 0x7f2f64103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 0x7f2f64103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59158/0 (socket says 172.21.15.52:59158) 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 -- 172.21.15.52:0/2137197016 learned_addr learned my addr 172.21.15.52:0/2137197016 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f63fff700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f64107400 0x7f2f64103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f5affd700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 0x7f2f641035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:28.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 -- 172.21.15.52:0/2137197016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 msgr2=0x7f2f641035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 0x7f2f641035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 -- 172.21.15.52:0/2137197016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f64107400 msgr2=0x7f2f64103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f64107400 0x7f2f64103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 -- 172.21.15.52:0/2137197016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f641a70c0 con 0x7f2f64101390 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f5affd700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 0x7f2f641035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f637fe700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 0x7f2f64103070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2f54000c00 tx=0x7f2f540047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.641+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54027070 con 0x7f2f64101390 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.642+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f54004010 con 0x7f2f64101390 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.642+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f641a7350 con 0x7f2f64101390 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.642+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5400f460 con 0x7f2f64101390 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.642+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f641a77a0 con 0x7f2f64101390 2024-06-02T23:10:28.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.642+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f64069180 con 0x7f2f64101390 2024-06-02T23:10:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.644+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2f5402a070 con 0x7f2f64101390 2024-06-02T23:10:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.645+0000 7f2f617fa700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f500a2c20 0x7f2f500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.645+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2f540d0710 con 0x7f2f64101390 2024-06-02T23:10:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.645+0000 7f2f5affd700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f500a2c20 0x7f2f500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.646+0000 7f2f5affd700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f500a2c20 0x7f2f500a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c005fd0 tx=0x7f2f4c005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:28.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.649+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f54031070 con 0x7f2f64101390 2024-06-02T23:10:28.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.900+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f641a7b10 con 0x7f2f64101390 2024-06-02T23:10:28.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.901+0000 7f2f617fa700 1 -- 172.21.15.52:0/2137197016 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f2f54013070 con 0x7f2f64101390 2024-06-02T23:10:28.901 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:28.901 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:07:54.810487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.903+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f500a2c20 msgr2=0x7f2f500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.903+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f500a2c20 0x7f2f500a50e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c005fd0 tx=0x7f2f4c005e20 comp rx=0 tx=0).stop 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.903+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 msgr2=0x7f2f64103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.903+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 0x7f2f64103070 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2f54000c00 tx=0x7f2f540047b0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 shutdown_connections 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f500a2c20 0x7f2f500a50e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f64107400 0x7f2f64103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f64101390 0x7f2f64103070 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 --2- 172.21.15.52:0/2137197016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f64101d50 0x7f2f641035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 >> 172.21.15.52:0/2137197016 conn(0x7f2f64078340 msgr2=0x7f2f641006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 shutdown_connections 2024-06-02T23:10:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:28.904+0000 7f2f69c8d700 1 -- 172.21.15.52:0/2137197016 wait complete. 2024-06-02T23:10:28.904 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:10:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:28 smithi123 ceph-mon[35977]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:28 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2137197016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:10:29.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:28 smithi052 ceph-mon[32340]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:29.281 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:28 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2137197016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:10:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:28 smithi174 ceph-mon[35994]: pgmap v389: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:28 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2137197016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:10:29.732 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:10:29.990 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:31.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:30 smithi052 ceph-mon[32340]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:31.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:30 smithi123 ceph-mon[35977]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:30 smithi174 ceph-mon[35994]: pgmap v390: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:31.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 -- 172.21.15.52:0/1084685354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08103120 msgr2=0x7f7e08107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1084685354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08103120 0x7f7e08107130 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc0099c0 tx=0x7f7dfc009cd0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 -- 172.21.15.52:0/1084685354 shutdown_connections 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1084685354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e08107860 0x7f7e0810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1084685354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08102770 0x7f7e08102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1084685354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08103120 0x7f7e08107130 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 -- 172.21.15.52:0/1084685354 >> 172.21.15.52:0/1084685354 conn(0x7f7e08079600 msgr2=0x7f7e08079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 -- 172.21.15.52:0/1084685354 shutdown_connections 2024-06-02T23:10:31.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.666+0000 7f7e0db90700 1 -- 172.21.15.52:0/1084685354 wait complete. 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 Processor -- start 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 -- start start 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08102770 0x7f7e08074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 0x7f7e08074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e08107860 0x7f7e08075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7e08112640 con 0x7f7e08102770 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e081124c0 con 0x7f7e08103120 2024-06-02T23:10:31.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e0db90700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e081127c0 con 0x7f7e08107860 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 0x7f7e08074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08102770 0x7f7e08074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 0x7f7e08074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56846/0 (socket says 172.21.15.52:56846) 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.667+0000 7f7e06ffd700 1 -- 172.21.15.52:0/1885905170 learned_addr learned my addr 172.21.15.52:0/1885905170 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e07fff700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e08107860 0x7f7e08075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e06ffd700 1 -- 172.21.15.52:0/1885905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e08107860 msgr2=0x7f7e08075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e06ffd700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e08107860 0x7f7e08075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e06ffd700 1 -- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08102770 msgr2=0x7f7e08074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e06ffd700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08102770 0x7f7e08074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e06ffd700 1 -- 172.21.15.52:0/1885905170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e081a7010 con 0x7f7e08103120 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e077fe700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08102770 0x7f7e08074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e06ffd700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 0x7f7e08074cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc006010 tx=0x7f7dfc009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc026070 con 0x7f7e08103120 2024-06-02T23:10:31.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e081a72a0 con 0x7f7e08103120 2024-06-02T23:10:31.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e081a7750 con 0x7f7e08103120 2024-06-02T23:10:31.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dfc004ca0 con 0x7f7e08103120 2024-06-02T23:10:31.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.668+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc003d00 con 0x7f7e08103120 2024-06-02T23:10:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.669+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e08069180 con 0x7f7e08103120 2024-06-02T23:10:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.671+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7dfc029070 con 0x7f7e08103120 2024-06-02T23:10:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.671+0000 7f7e04ff9700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7df00a2c20 0x7f7df00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.672+0000 7f7e077fe700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7df00a2c20 0x7f7df00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.672+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7dfc0cfad0 con 0x7f7e08103120 2024-06-02T23:10:31.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.672+0000 7f7e077fe700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7df00a2c20 0x7f7df00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7df8007fd0 tx=0x7f7df80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:31.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.676+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7dfc013070 con 0x7f7e08103120 2024-06-02T23:10:31.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.931+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e0802dca0 con 0x7f7e08103120 2024-06-02T23:10:31.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.932+0000 7f7e04ff9700 1 -- 172.21.15.52:0/1885905170 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11939 (secure 0 0 0) 0x7f7dfc09caf0 con 0x7f7e08103120 2024-06-02T23:10:31.931 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:31.932 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:07:54.810487+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.934+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7df00a2c20 msgr2=0x7f7df00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.934+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7df00a2c20 0x7f7df00a50e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7df8007fd0 tx=0x7f7df80073d0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 msgr2=0x7f7e08074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 0x7f7e08074cd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc006010 tx=0x7f7dfc009f80 comp rx=0 tx=0).stop 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 shutdown_connections 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7df00a2c20 0x7f7df00a50e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e08107860 0x7f7e08075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e08102770 0x7f7e08074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 --2- 172.21.15.52:0/1885905170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e08103120 0x7f7e08074cd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:31.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 >> 172.21.15.52:0/1885905170 conn(0x7f7e08079600 msgr2=0x7f7e081007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:31.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 shutdown_connections 2024-06-02T23:10:31.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:31.935+0000 7f7e0db90700 1 -- 172.21.15.52:0/1885905170 wait complete. 2024-06-02T23:10:31.935 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 27 2024-06-02T23:10:32.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:10:32.953 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:33.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:32 smithi052 ceph-mon[32340]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:33.077 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:32 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1885905170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:10:33.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:32 smithi123 ceph-mon[35977]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:33.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:32 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1885905170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:10:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:32 smithi174 ceph-mon[35994]: pgmap v391: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:32 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1885905170' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:10:34.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:10:34.260 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:10:34.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:10:34.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.623+0000 7f53b86d7700 1 -- 172.21.15.52:0/3049367352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b006a170 msgr2=0x7f53b006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:34.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.623+0000 7f53b86d7700 1 --2- 172.21.15.52:0/3049367352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b006a170 0x7f53b006a5f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f53a0009990 tx=0x7f53a0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 -- 172.21.15.52:0/3049367352 shutdown_connections 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 --2- 172.21.15.52:0/3049367352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53b006ab30 0x7f53b010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 --2- 172.21.15.52:0/3049367352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b0108500 0x7f53b01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 --2- 172.21.15.52:0/3049367352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b006a170 0x7f53b006a5f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 -- 172.21.15.52:0/3049367352 >> 172.21.15.52:0/3049367352 conn(0x7f53b0078340 msgr2=0x7f53b0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 -- 172.21.15.52:0/3049367352 shutdown_connections 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 -- 172.21.15.52:0/3049367352 wait complete. 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 Processor -- start 2024-06-02T23:10:34.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.624+0000 7f53b86d7700 1 -- start start 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b86d7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53b006a170 0x7f53b0101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b86d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b006ab30 0x7f53b01019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b86d7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 0x7f53b0101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b86d7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f53b0112610 con 0x7f53b0108500 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b86d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f53b0112490 con 0x7f53b006ab30 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b86d7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f53b0112790 con 0x7f53b006a170 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b6c74700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 0x7f53b0101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b6c74700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 0x7f53b0101ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46000/0 (socket says 172.21.15.52:46000) 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b6c74700 1 -- 172.21.15.52:0/2597892671 learned_addr learned my addr 172.21.15.52:0/2597892671 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b5c72700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b006ab30 0x7f53b01019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:34.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b6473700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53b006a170 0x7f53b0101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b5c72700 1 -- 172.21.15.52:0/2597892671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53b006a170 msgr2=0x7f53b0101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b5c72700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53b006a170 0x7f53b0101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b5c72700 1 -- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 msgr2=0x7f53b0101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b5c72700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 0x7f53b0101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b5c72700 1 -- 172.21.15.52:0/2597892671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b01a7090 con 0x7f53b006ab30 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.625+0000 7f53b6c74700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 0x7f53b0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.626+0000 7f53b5c72700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b006ab30 0x7f53b01019a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53a00059d0 tx=0x7f53a00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.626+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0027070 con 0x7f53b006ab30 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.626+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b01a7320 con 0x7f53b006ab30 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.626+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f53a0022470 con 0x7f53b006ab30 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.626+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a0017630 con 0x7f53b006ab30 2024-06-02T23:10:34.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.626+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b01a7770 con 0x7f53b006ab30 2024-06-02T23:10:34.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.627+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b0069180 con 0x7f53b006ab30 2024-06-02T23:10:34.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.628+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f53a002a070 con 0x7f53b006ab30 2024-06-02T23:10:34.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.629+0000 7f53a77fe700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f539c0a2c20 0x7f539c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:34.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.629+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f53a00cfce0 con 0x7f53b006ab30 2024-06-02T23:10:34.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.629+0000 7f53b6473700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f539c0a2c20 0x7f539c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:34.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.629+0000 7f53b6473700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f539c0a2c20 0x7f539c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009550 tx=0x7f53ac0091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:34.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.633+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f53a0013070 con 0x7f53b006ab30 2024-06-02T23:10:34.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.843+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f53b002dca0 con 0x7f53b006ab30 2024-06-02T23:10:34.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.845+0000 7f53a77fe700 1 -- 172.21.15.52:0/2597892671 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7f53a009cce0 con 0x7f53b006ab30 2024-06-02T23:10:34.844 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:34.844 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:10:34.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f539c0a2c20 msgr2=0x7f539c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f539c0a2c20 0x7f539c0a50e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f53ac009550 tx=0x7f53ac0091f0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b006ab30 msgr2=0x7f53b01019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b006ab30 0x7f53b01019a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f53a00059d0 tx=0x7f53a00047b0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 shutdown_connections 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f539c0a2c20 0x7f539c0a50e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f53b0108500 0x7f53b0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f53b006a170 0x7f53b0101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 --2- 172.21.15.52:0/2597892671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f53b006ab30 0x7f53b01019a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 >> 172.21.15.52:0/2597892671 conn(0x7f53b0078340 msgr2=0x7f53b0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 shutdown_connections 2024-06-02T23:10:34.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:34.847+0000 7f53b86d7700 1 -- 172.21.15.52:0/2597892671 wait complete. 2024-06-02T23:10:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2597892671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:10:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:10:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:10:35.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2597892671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:10:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:10:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:10:35.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: pgmap v392: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2597892671' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:10:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:10:35.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 27, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-02T23:07:33.296248+0000', 'modified': '2024-06-02T23:07:54.810487+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24451, 'mds_2': 24449, 'mds_3': 24436, 'mds_4': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/2175393174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2175393174}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2175393174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/4238066654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 4238066654}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 4238066654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/224943979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 224943979}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 224943979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/1267486579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1267486579}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1267486579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/2486297365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 2486297365}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 2486297365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.123:6835/3092837185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3092837185}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3092837185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.174:6837/457473811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 457473811}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 457473811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6837/785283486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 785283486}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 785283486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.174:6839/2115890873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2115890873}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2115890873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/2043673684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2043673684}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2043673684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24470, 24449, 24451, 24436, 24434]} 2024-06-02T23:10:35.540 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-02T23:10:35.540 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-02T23:10:35.796 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:37.259 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:37 smithi052 ceph-mon[32340]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:37 smithi174 ceph-mon[35994]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:37.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 -- 172.21.15.52:0/3100254963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006a170 msgr2=0x7f5a7006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/3100254963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006a170 0x7f5a7006a5f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009990 tx=0x7f5a58009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 -- 172.21.15.52:0/3100254963 shutdown_connections 2024-06-02T23:10:37.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/3100254963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006ab30 0x7f5a7010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/3100254963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a70108500 0x7f5a701088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/3100254963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006a170 0x7f5a7006a5f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 -- 172.21.15.52:0/3100254963 >> 172.21.15.52:0/3100254963 conn(0x7f5a70078340 msgr2=0x7f5a70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:37.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.495+0000 7f5a75f7d700 1 -- 172.21.15.52:0/3100254963 shutdown_connections 2024-06-02T23:10:37.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.496+0000 7f5a75f7d700 1 -- 172.21.15.52:0/3100254963 wait complete. 2024-06-02T23:10:37.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.496+0000 7f5a75f7d700 1 Processor -- start 2024-06-02T23:10:37.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.496+0000 7f5a75f7d700 1 -- start start 2024-06-02T23:10:37.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a75f7d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006a170 0x7f5a7019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a75f7d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 0x7f5a7019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a75f7d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a70108500 0x7f5a70198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a75f7d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5a70112630 con 0x7f5a7006ab30 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a75f7d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a701124b0 con 0x7f5a7006a170 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a75f7d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5a701127b0 con 0x7f5a70108500 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 0x7f5a7019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 0x7f5a7019ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46024/0 (socket says 172.21.15.52:46024) 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 -- 172.21.15.52:0/4160856626 learned_addr learned my addr 172.21.15.52:0/4160856626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006a170 0x7f5a7019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a70108500 0x7f5a70198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 -- 172.21.15.52:0/4160856626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a70108500 msgr2=0x7f5a70198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a70108500 0x7f5a70198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 -- 172.21.15.52:0/4160856626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006a170 msgr2=0x7f5a7019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006a170 0x7f5a7019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 -- 172.21.15.52:0/4160856626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a701990f0 con 0x7f5a7006ab30 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006a170 0x7f5a7019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 0x7f5a7019ed10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5a58000c00 tx=0x7f5a58004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.497+0000 7f5a6d7fa700 1 -- 172.21.15.52:0/4160856626 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a58027070 con 0x7f5a7006ab30 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.498+0000 7f5a6d7fa700 1 -- 172.21.15.52:0/4160856626 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a58022470 con 0x7f5a7006ab30 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.498+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a70199380 con 0x7f5a7006ab30 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.498+0000 7f5a6d7fa700 1 -- 172.21.15.52:0/4160856626 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5800b610 con 0x7f5a7006ab30 2024-06-02T23:10:37.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.498+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a701a56a0 con 0x7f5a7006ab30 2024-06-02T23:10:37.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.500+0000 7f5a6d7fa700 1 -- 172.21.15.52:0/4160856626 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5a5802a070 con 0x7f5a7006ab30 2024-06-02T23:10:37.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.501+0000 7f5a6d7fa700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.501+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.501+0000 7f5a6d7fa700 1 -- 172.21.15.52:0/4160856626 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5a580cfc80 con 0x7f5a7006ab30 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.501+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5a70199ad0 tx=0x7f5a60009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.501+0000 7f5a80800b80 1 Processor -- start 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.501+0000 7f5a80800b80 1 -- start start 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff0d5f0 0x55c41ff0d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff07990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 0x55c41ff082f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a80800b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c41fe597c0 con 0x55c41ff07ed0 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a80800b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c41fe016d0 con 0x55c41ff0d5f0 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a80800b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c41fe55780 con 0x55c41ff0df10 2024-06-02T23:10:37.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 0x55c41ff082f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 0x55c41ff082f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46030/0 (socket says 172.21.15.52:46030) 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3139846895 learned_addr learned my addr 172.21.15.52:0/3139846895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff0d5f0 0x55c41ff0d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff07990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3139846895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 msgr2=0x55c41ff07990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff07990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3139846895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff0d5f0 msgr2=0x55c41ff0d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff0d5f0 0x55c41ff0d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3139846895 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41fdf7f00 con 0x55c41ff07ed0 2024-06-02T23:10:37.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 0x55c41ff082f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5a68011680 tx=0x7f5a68009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8638008474764905 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:37.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a65ffb700 1 -- 172.21.15.52:0/3139846895 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68019030 con 0x55c41ff07ed0 2024-06-02T23:10:37.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a65ffb700 1 -- 172.21.15.52:0/3139846895 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a680093a0 con 0x55c41ff07ed0 2024-06-02T23:10:37.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.502+0000 7f5a65ffb700 1 -- 172.21.15.52:0/3139846895 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68007500 con 0x55c41ff07ed0 2024-06-02T23:10:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 -- 172.21.15.52:0/3139846895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 msgr2=0x55c41ff082f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 0x55c41ff082f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5a68011680 tx=0x7f5a68009920 comp rx=0 tx=0).stop 2024-06-02T23:10:37.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 -- 172.21.15.52:0/3139846895 shutdown_connections 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff07ed0 0x55c41ff082f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff0d5f0 0x55c41ff0d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3139846895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff07990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 -- 172.21.15.52:0/3139846895 >> 172.21.15.52:0/3139846895 conn(0x55c41fe32c70 msgr2=0x55c41ff02800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 -- 172.21.15.52:0/3139846895 shutdown_connections 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 -- 172.21.15.52:0/3139846895 wait complete. 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.503+0000 7f5a80800b80 1 -- start start 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 Processor -- start 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff13ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff11520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff14980 con 0x55c41ff0df10 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c41fdf9ba0 con 0x55c41ff07ed0 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff14b00 con 0x55c41ff0d5f0 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a80800b80 10 client.0 _ll_register_callbacks cb 0x55c41fdf38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff11520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff11520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46034/0 (socket says 172.21.15.52:46034) 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 -- 172.21.15.52:0/870739835 learned_addr learned my addr 172.21.15.52:0/870739835 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff13ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 -- 172.21.15.52:0/870739835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 msgr2=0x55c41ff10fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 -- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 msgr2=0x55c41ff13ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff13ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6ffff700 1 -- 172.21.15.52:0/870739835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41fda8880 con 0x55c41ff0df10 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff13ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.504+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff11520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5a68012010 tx=0x7f5a68003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccc7aa605dea03f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a4effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a4effd700 1 -- 172.21.15.52:0/870739835 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68004cd0 con 0x55c41ff0df10 2024-06-02T23:10:37.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a4effd700 1 -- 172.21.15.52:0/870739835 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5a68007500 con 0x55c41ff0df10 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c41fe66cc0 con 0x55c41ff0df10 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a4effd700 1 -- 172.21.15.52:0/870739835 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a6802e070 con 0x55c41ff0df10 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a80800b80 10 client.15126 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c41ff1f250 con 0x55c41ff0df10 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a4effd700 1 -- 172.21.15.52:0/870739835 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c41fe66cc0 con 0x55c41ff0df10 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a4effd700 1 -- 172.21.15.52:0/870739835 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5a680050e0 con 0x55c41ff0df10 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a80800b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.505+0000 7f5a80800b80 10 client.15126 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a80800b80 20 client.15126 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a80800b80 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff21470 0x55c41ff228e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a80800b80 4 client.15126 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:37.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c41fe61240 con 0x55c41ff21470 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff21470 0x55c41ff228e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff21470 0x55c41ff228e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=42fd117ddd1b9216 server_cookie=6e19bc1bfeb46d46 in_seq=0 out_seq=0 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.506+0000 7f5a4effd700 10 client.15126 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a4effd700 1 -- 172.21.15.52:0/870739835 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c41fe61240 con 0x55c41ff21470 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a4effd700 10 client.15126 handle_command_reply: tid=0 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 client.15126 shutdown 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 msgr2=0x55c41ff11520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff11520 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5a68012010 tx=0x7f5a68003d60 comp rx=0 tx=0).stop 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 shutdown_connections 2024-06-02T23:10:37.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff11520 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff21470 0x55c41ff228e0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff13ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 --2- 172.21.15.52:0/870739835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a4effd700 0 client.15126 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 >> 172.21.15.52:0/870739835 conn(0x55c41fe32c70 msgr2=0x55c41ff06980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 shutdown_connections 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.507+0000 7f5a80800b80 1 -- 172.21.15.52:0/870739835 wait complete. 2024-06-02T23:10:37.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.508+0000 7f5a80800b80 20 client.15126 trim_cache size 0 max 16384 2024-06-02T23:10:37.514 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:37 smithi123 ceph-mon[35977]: pgmap v393: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:37.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 Processor -- start 2024-06-02T23:10:37.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 -- start start 2024-06-02T23:10:37.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff07990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 0x55c41ff20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff20c20 con 0x55c41ff0d5f0 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff212f0 con 0x55c41ff07ed0 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a80800b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff219c0 con 0x55c41ff0df10 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a66ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 0x55c41ff20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a66ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 0x55c41ff20da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46048/0 (socket says 172.21.15.52:46048) 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a66ffd700 1 -- 172.21.15.52:0/2232509263 learned_addr learned my addr 172.21.15.52:0/2232509263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff07990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.529+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a66ffd700 1 -- 172.21.15.52:0/2232509263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 msgr2=0x55c41ff108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a66ffd700 1 -- 172.21.15.52:0/2232509263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 msgr2=0x55c41ff07990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff07990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a66ffd700 1 -- 172.21.15.52:0/2232509263 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c41ff034b0 con 0x55c41ff0d5f0 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 0x55c41ff20da0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5a580cdb20 tx=0x7f5a58087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d977dca135dac6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a657fa700 1 -- 172.21.15.52:0/2232509263 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5809c5a0 con 0x55c41ff0d5f0 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a657fa700 1 -- 172.21.15.52:0/2232509263 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a580a0950 con 0x55c41ff0d5f0 2024-06-02T23:10:37.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.530+0000 7f5a657fa700 1 -- 172.21.15.52:0/2232509263 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a5809ca60 con 0x55c41ff0d5f0 2024-06-02T23:10:37.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 -- 172.21.15.52:0/2232509263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 msgr2=0x55c41ff20da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 0x55c41ff20da0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5a580cdb20 tx=0x7f5a58087040 comp rx=0 tx=0).stop 2024-06-02T23:10:37.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 -- 172.21.15.52:0/2232509263 shutdown_connections 2024-06-02T23:10:37.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0df10 0x55c41ff108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff07990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 --2- 172.21.15.52:0/2232509263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0d5f0 0x55c41ff20da0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 -- 172.21.15.52:0/2232509263 >> 172.21.15.52:0/2232509263 conn(0x55c41fe32c70 msgr2=0x55c41ff0e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 -- 172.21.15.52:0/2232509263 shutdown_connections 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 -- 172.21.15.52:0/2232509263 wait complete. 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.531+0000 7f5a80800b80 1 -- start start 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 Processor -- start 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff0fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff10750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff22760 con 0x55c41ff0df10 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff22090 con 0x55c41ff07ed0 2024-06-02T23:10:37.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c41ff22e30 con 0x55c41ff0d5f0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a80800b80 10 client.0 _ll_register_callbacks cb 0x55c41fdf38d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff10750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff10750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46064/0 (socket says 172.21.15.52:46064) 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3138118994 learned_addr learned my addr 172.21.15.52:0/3138118994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a66ffd700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff0fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3138118994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 msgr2=0x55c41ff10210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 msgr2=0x55c41ff0fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff0fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6ffff700 1 -- 172.21.15.52:0/3138118994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c41ff29280 con 0x55c41ff0df10 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.532+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff0fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a6ffff700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff10750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009ea0 tx=0x7f5a680098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8b5e5365908db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a4effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:37.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a4effd700 1 -- 172.21.15.52:0/3138118994 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a68019040 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a4effd700 1 -- 172.21.15.52:0/3138118994 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5a68005720 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a580a0950 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a4effd700 1 -- 172.21.15.52:0/3138118994 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5a68007500 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a80800b80 10 client.15132 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55c41ff8cf80 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a4effd700 1 -- 172.21.15.52:0/3138118994 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5a580a0950 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a4effd700 1 -- 172.21.15.52:0/3138118994 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5a68033710 con 0x55c41ff0df10 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.533+0000 7f5a80800b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a80800b80 10 client.15132 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a80800b80 20 client.15132 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff91c10 0x55c41ff90920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a80800b80 4 client.15132 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:37.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "status"}) -- 0x55c41fcdb200 con 0x55c41ff91c10 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff91c10 0x55c41ff90920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a6f7fe700 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff91c10 0x55c41ff90920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4430cc78aec0e4 server_cookie=db33be6357656499 in_seq=0 out_seq=0 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.534+0000 7f5a4effd700 10 client.15132 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a4effd700 1 -- 172.21.15.52:0/3138118994 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c41fcdb200 con 0x55c41ff91c10 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a4effd700 10 client.15132 handle_command_reply: tid=0 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 client.15132 shutdown 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 msgr2=0x55c41ff10750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff10750 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009ea0 tx=0x7f5a680098a0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 shutdown_connections 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41ff0df10 0x55c41ff10750 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c41ff91c10 0x55c41ff90920 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41ff07ed0 0x55c41ff0fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a4effd700 0 client.15132 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 --2- 172.21.15.52:0/3138118994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41ff0d5f0 0x55c41ff10210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 >> 172.21.15.52:0/3138118994 conn(0x55c41fe32c70 msgr2=0x55c41fefbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 shutdown_connections 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 1 -- 172.21.15.52:0/3138118994 wait complete. 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.535+0000 7f5a80800b80 20 client.15132 trim_cache size 0 max 16384 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24434, 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:37.539 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 166.26471762200001, 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 224.017492836 2024-06-02T23:10:37.540 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a5c0a2c20 msgr2=0x7f5a5c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5a70199ad0 tx=0x7f5a60009260 comp rx=0 tx=0).stop 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 msgr2=0x7f5a7019ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 0x7f5a7019ed10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5a58000c00 tx=0x7f5a58004790 comp rx=0 tx=0).stop 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 shutdown_connections 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a5c0a2c20 0x7f5a5c0a50e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5a70108500 0x7f5a70198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a7006a170 0x7f5a7019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 --2- 172.21.15.52:0/4160856626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5a7006ab30 0x7f5a7019ed10 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 >> 172.21.15.52:0/4160856626 conn(0x7f5a70078340 msgr2=0x7f5a70100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:37.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 shutdown_connections 2024-06-02T23:10:37.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:37.536+0000 7f5a75f7d700 1 -- 172.21.15.52:0/4160856626 wait complete. 2024-06-02T23:10:38.245 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-02T23:10:38.500 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:39 smithi174 ceph-mon[35994]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:39.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:39 smithi052 ceph-mon[32340]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:39 smithi123 ceph-mon[35977]: pgmap v394: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:40.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 -- 172.21.15.52:0/2664899777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101380 msgr2=0x7f5d00101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 --2- 172.21.15.52:0/2664899777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101380 0x7f5d00101800 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009990 tx=0x7f5ce8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 -- 172.21.15.52:0/2664899777 shutdown_connections 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 --2- 172.21.15.52:0/2664899777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101d40 0x7f5d0010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 --2- 172.21.15.52:0/2664899777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d001073f0 0x7f5d001077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 --2- 172.21.15.52:0/2664899777 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101380 0x7f5d00101800 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 -- 172.21.15.52:0/2664899777 >> 172.21.15.52:0/2664899777 conn(0x7f5d00078300 msgr2=0x7f5d00078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 -- 172.21.15.52:0/2664899777 shutdown_connections 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.231+0000 7f5d06053700 1 -- 172.21.15.52:0/2664899777 wait complete. 2024-06-02T23:10:40.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 Processor -- start 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 -- start start 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101380 0x7f5d001998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 0x7f5d00199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d001073f0 0x7f5d001a4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111510 con 0x7f5d00101d40 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111390 con 0x7f5d00101380 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5d06053700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d00111690 con 0x7f5d001073f0 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 0x7f5d00199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 0x7f5d00199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46084/0 (socket says 172.21.15.52:46084) 2024-06-02T23:10:40.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5cfeffd700 1 -- 172.21.15.52:0/705061717 learned_addr learned my addr 172.21.15.52:0/705061717 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101380 0x7f5d001998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.232+0000 7f5cfffff700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d001073f0 0x7f5d001a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfeffd700 1 -- 172.21.15.52:0/705061717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d001073f0 msgr2=0x7f5d001a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d001073f0 0x7f5d001a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfeffd700 1 -- 172.21.15.52:0/705061717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101380 msgr2=0x7f5d001998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101380 0x7f5d001998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfeffd700 1 -- 172.21.15.52:0/705061717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d001a4e90 con 0x7f5d00101d40 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 0x7f5d00199e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8003ab0 tx=0x7f5ce8003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfcff9700 1 -- 172.21.15.52:0/705061717 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8027070 con 0x7f5d00101d40 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d001a5060 con 0x7f5d00101d40 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfcff9700 1 -- 172.21.15.52:0/705061717 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce800bdc0 con 0x7f5d00101d40 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5cfcff9700 1 -- 172.21.15.52:0/705061717 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8022d40 con 0x7f5d00101d40 2024-06-02T23:10:40.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.233+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d001a5560 con 0x7f5d00101d40 2024-06-02T23:10:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.235+0000 7f5cfcff9700 1 -- 172.21.15.52:0/705061717 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ce802a070 con 0x7f5d00101d40 2024-06-02T23:10:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.236+0000 7f5cfcff9700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cec0a2c20 0x7f5cec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.236+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cec0a2c20 0x7f5cec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.236+0000 7f5cfcff9700 1 -- 172.21.15.52:0/705061717 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ce80cfd50 con 0x7f5d00101d40 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cec0a2c20 0x7f5cec0a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e270 tx=0x7f5cf0009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 Processor -- start 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 -- start start 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 0x55fa618bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618be2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fa6180f7c0 con 0x55fa618c3f10 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa617b76d0 con 0x55fa618c35f0 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa6180b780 con 0x55fa618bded0 2024-06-02T23:10:40.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 0x55fa618bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 0x55fa618bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46088/0 (socket says 172.21.15.52:46088) 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5cfeffd700 1 -- 172.21.15.52:0/1495999458 learned_addr learned my addr 172.21.15.52:0/1495999458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5cfffff700 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618be2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.237+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cfeffd700 1 -- 172.21.15.52:0/1495999458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 msgr2=0x55fa618be2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618be2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cfeffd700 1 -- 172.21.15.52:0/1495999458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 msgr2=0x55fa618c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cfeffd700 1 -- 172.21.15.52:0/1495999458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa617adf00 con 0x55fa618c3f10 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 0x55fa618bd990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8098ad0 tx=0x7f5ce800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a25a42101de7fc59 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cf5ffb700 1 -- 172.21.15.52:0/1495999458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce809bfd0 con 0x55fa618c3f10 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cf5ffb700 1 -- 172.21.15.52:0/1495999458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce809cd70 con 0x55fa618c3f10 2024-06-02T23:10:40.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.238+0000 7f5cf5ffb700 1 -- 172.21.15.52:0/1495999458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8003680 con 0x55fa618c3f10 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 -- 172.21.15.52:0/1495999458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 msgr2=0x55fa618bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 0x55fa618bd990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8098ad0 tx=0x7f5ce800b540 comp rx=0 tx=0).stop 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 -- 172.21.15.52:0/1495999458 shutdown_connections 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618be2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/1495999458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c3f10 0x55fa618bd990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 -- 172.21.15.52:0/1495999458 >> 172.21.15.52:0/1495999458 conn(0x55fa617e8c70 msgr2=0x55fa618b8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:40.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 -- 172.21.15.52:0/1495999458 shutdown_connections 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 -- 172.21.15.52:0/1495999458 wait complete. 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 -- start start 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.239+0000 7f5d108d6b80 1 Processor -- start 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618c9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fa618ca6a0 con 0x55fa618bded0 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa617afba0 con 0x55fa618c35f0 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa618ca820 con 0x55fa618c3f10 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618c9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5d108d6b80 10 client.0 _ll_register_callbacks cb 0x55fa617a98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618c9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46098/0 (socket says 172.21.15.52:46098) 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 -- 172.21.15.52:0/2430103773 learned_addr learned my addr 172.21.15.52:0/2430103773 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:40.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cfffff700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 -- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 msgr2=0x55fa618c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 -- 172.21.15.52:0/2430103773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 msgr2=0x55fa618c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 -- 172.21.15.52:0/2430103773 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa6175e880 con 0x55fa618bded0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618c9cf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00159e0 tx=0x7f5cf0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8795cc07d8b72749 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.240+0000 7f5cdeffd700 1 -- 172.21.15.52:0/2430103773 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0025020 con 0x55fa618bded0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5cdeffd700 1 -- 172.21.15.52:0/2430103773 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cf0010040 con 0x55fa618bded0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce809cd70 con 0x55fa618bded0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5cdeffd700 1 -- 172.21.15.52:0/2430103773 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5cf0013670 con 0x55fa618bded0 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 10 client.15144 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:40.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa618d5050 con 0x55fa618bded0 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5cdeffd700 1 -- 172.21.15.52:0/2430103773 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce809cd70 con 0x55fa618bded0 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5cdeffd700 1 -- 172.21.15.52:0/2430103773 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5cf0014070 con 0x55fa618bded0 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 10 client.15144 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 20 client.15144 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa618d76d0 0x55fa618d8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 4 client.15144 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.241+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa6181ccc0 con 0x55fa618d76d0 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.242+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa618d76d0 0x55fa618d8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.242+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa618d76d0 0x55fa618d8b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=12cd17ce6277e221 server_cookie=d828b7ee51e7a4a8 in_seq=0 out_seq=0 2024-06-02T23:10:40.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.242+0000 7f5cdeffd700 10 client.15144 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:10:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5cdeffd700 1 -- 172.21.15.52:0/2430103773 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa6181ccc0 con 0x55fa618d76d0 2024-06-02T23:10:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5cdeffd700 10 client.15144 handle_command_reply: tid=0 2024-06-02T23:10:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 client.15144 shutdown 2024-06-02T23:10:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 msgr2=0x55fa618c9cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618c9cf0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00159e0 tx=0x7f5cf0004ad0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 shutdown_connections 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa618d76d0 0x55fa618d8b40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618c9cf0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/2430103773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5cdeffd700 0 client.15144 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 >> 172.21.15.52:0/2430103773 conn(0x55fa617e8c70 msgr2=0x55fa618b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 shutdown_connections 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 1 -- 172.21.15.52:0/2430103773 wait complete. 2024-06-02T23:10:40.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.243+0000 7f5d108d6b80 20 client.15144 trim_cache size 0 max 16384 2024-06-02T23:10:40.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 Processor -- start 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 -- start start 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618bd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 0x55fa618be2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c3f10 0x55fa618b24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fa618d6e80 con 0x55fa618c35f0 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa618d7550 con 0x55fa618c3f10 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa618d7c20 con 0x55fa618bded0 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 0x55fa618be2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 0x55fa618be2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46110/0 (socket says 172.21.15.52:46110) 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 -- 172.21.15.52:0/3658313749 learned_addr learned my addr 172.21.15.52:0/3658313749 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 -- 172.21.15.52:0/3658313749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 msgr2=0x55fa618bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfffff700 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c3f10 0x55fa618b24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 -- 172.21.15.52:0/3658313749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c3f10 msgr2=0x55fa618b24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c3f10 0x55fa618b24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 -- 172.21.15.52:0/3658313749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa618ba030 con 0x55fa618c35f0 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 0x55fa618be2b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ce809c600 tx=0x7f5ce8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04256bf059425e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.265+0000 7f5cf57fa700 1 -- 172.21.15.52:0/3658313749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80a5250 con 0x55fa618c35f0 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5cf57fa700 1 -- 172.21.15.52:0/3658313749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce809c670 con 0x55fa618c35f0 2024-06-02T23:10:40.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5cf57fa700 1 -- 172.21.15.52:0/3658313749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce809aa00 con 0x55fa618c35f0 2024-06-02T23:10:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 -- 172.21.15.52:0/3658313749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 msgr2=0x55fa618be2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 0x55fa618be2b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ce809c600 tx=0x7f5ce8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 -- 172.21.15.52:0/3658313749 shutdown_connections 2024-06-02T23:10:40.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c3f10 0x55fa618b24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618bded0 0x55fa618bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/3658313749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618c35f0 0x55fa618be2b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 -- 172.21.15.52:0/3658313749 >> 172.21.15.52:0/3658313749 conn(0x55fa617e8c70 msgr2=0x55fa618b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 -- 172.21.15.52:0/3658313749 shutdown_connections 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.266+0000 7f5d108d6b80 1 -- 172.21.15.52:0/3658313749 wait complete. 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 -- start start 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 Processor -- start 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618e23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618e0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618e0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fa618d89c0 con 0x55fa618bded0 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa618d82f0 con 0x55fa618c35f0 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa618d9090 con 0x55fa618c3f10 2024-06-02T23:10:40.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5d108d6b80 10 client.0 _ll_register_callbacks cb 0x55fa617a98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618e23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618e23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46124/0 (socket says 172.21.15.52:46124) 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5cff7fe700 1 -- 172.21.15.52:0/533604575 learned_addr learned my addr 172.21.15.52:0/533604575 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5cfffff700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618e0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.267+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618e0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cff7fe700 1 -- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 msgr2=0x55fa618e0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618e0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cff7fe700 1 -- 172.21.15.52:0/533604575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 msgr2=0x55fa618e0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618e0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cff7fe700 1 -- 172.21.15.52:0/533604575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa618e1240 con 0x55fa618bded0 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cfffff700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618e0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cff7fe700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618e23e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007590 tx=0x7f5cf002df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bacdf829692015 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:40.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cdeffd700 1 -- 172.21.15.52:0/533604575 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf003a060 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce809c670 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cdeffd700 1 -- 172.21.15.52:0/533604575 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cf002c610 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cdeffd700 1 -- 172.21.15.52:0/533604575 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f5cf0025040 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5d108d6b80 10 client.15150 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55fa61947da0 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cdeffd700 1 -- 172.21.15.52:0/533604575 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce809c670 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.268+0000 7f5cdeffd700 1 -- 172.21.15.52:0/533604575 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f5cf00050f0 con 0x55fa618bded0 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 10 client.15150 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 10 client.15150 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 20 client.15150 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa6194c8c0 0x55fa6194b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:40.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 4 client.15150 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "status"}) -- 0x55fa618be860 con 0x55fa6194c8c0 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa6194c8c0 0x55fa6194b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5cfeffd700 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa6194c8c0 0x55fa6194b5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94677fcfdc54c5e5 server_cookie=4880bf79573394f1 in_seq=0 out_seq=0 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5cdeffd700 10 client.15150 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5cdeffd700 1 -- 172.21.15.52:0/533604575 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55fa618be860 con 0x55fa6194c8c0 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5cdeffd700 10 client.15150 handle_command_reply: tid=0 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.269+0000 7f5d108d6b80 1 client.15150 shutdown 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 msgr2=0x55fa618e23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618e23e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007590 tx=0x7f5cf002df50 comp rx=0 tx=0).stop 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 shutdown_connections 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55fa6194c8c0 0x55fa6194b5d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa618c3f10 0x55fa618e0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa618bded0 0x55fa618e23e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5cdeffd700 0 client.15150 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 --2- 172.21.15.52:0/533604575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa618c35f0 0x55fa618e0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 >> 172.21.15.52:0/533604575 conn(0x55fa617e8c70 msgr2=0x55fa618c8820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 shutdown_connections 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 1 -- 172.21.15.52:0/533604575 wait complete. 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d108d6b80 20 client.15150 trim_cache size 0 max 16384 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24436, 2024-06-02T23:10:40.274 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 170.00764907000001, 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 221.24499955900001 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:40.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cec0a2c20 msgr2=0x7f5cec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d06053700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cec0a2c20 0x7f5cec0a50e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e270 tx=0x7f5cf0009040 comp rx=0 tx=0).stop 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 msgr2=0x7f5d00199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.270+0000 7f5d06053700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 0x7f5d00199e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8003ab0 tx=0x7f5ce8003b90 comp rx=0 tx=0).stop 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 shutdown_connections 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cec0a2c20 0x7f5cec0a50e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d001073f0 0x7f5d001a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d00101380 0x7f5d001998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 --2- 172.21.15.52:0/705061717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d00101d40 0x7f5d00199e10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 >> 172.21.15.52:0/705061717 conn(0x7f5d00078300 msgr2=0x7f5d000ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 shutdown_connections 2024-06-02T23:10:40.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:40.271+0000 7f5d06053700 1 -- 172.21.15.52:0/705061717 wait complete. 2024-06-02T23:10:41.042 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-02T23:10:41.298 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:41.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:41 smithi174 ceph-mon[35994]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:41.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:41 smithi052 ceph-mon[32340]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:41.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:41 smithi123 ceph-mon[35977]: pgmap v395: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.984+0000 7f0aa2dab700 1 -- 172.21.15.52:0/2117398988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c101d60 msgr2=0x7f0a9c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.984+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/2117398988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c101d60 0x7f0a9c10e240 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b0a0 tx=0x7f0a9400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 -- 172.21.15.52:0/2117398988 shutdown_connections 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/2117398988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c101d60 0x7f0a9c10e240 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/2117398988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9c107400 0x7f0a9c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/2117398988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a9c1013c0 0x7f0a9c101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 -- 172.21.15.52:0/2117398988 >> 172.21.15.52:0/2117398988 conn(0x7f0a9c078330 msgr2=0x7f0a9c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 -- 172.21.15.52:0/2117398988 shutdown_connections 2024-06-02T23:10:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 -- 172.21.15.52:0/2117398988 wait complete. 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 Processor -- start 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.985+0000 7f0aa2dab700 1 -- start start 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a9c1013c0 0x7f0a9c19d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9c101d60 0x7f0a9c19dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 0x7f0a9c197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c111500 con 0x7f0a9c107400 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c111380 con 0x7f0a9c101d60 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9c111680 con 0x7f0a9c1013c0 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 0x7f0a9c197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 0x7f0a9c197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46148/0 (socket says 172.21.15.52:46148) 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 -- 172.21.15.52:0/593481437 learned_addr learned my addr 172.21.15.52:0/593481437 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9c101d60 0x7f0a9c19dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a9c1013c0 0x7f0a9c19d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 -- 172.21.15.52:0/593481437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a9c1013c0 msgr2=0x7f0a9c19d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a9c1013c0 0x7f0a9c19d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 -- 172.21.15.52:0/593481437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9c101d60 msgr2=0x7f0a9c19dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9c101d60 0x7f0a9c19dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 -- 172.21.15.52:0/593481437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a9c198020 con 0x7f0a9c107400 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa1348700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 0x7f0a9c197840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b830 tx=0x7f0a94004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0a99ffb700 1 -- 172.21.15.52:0/593481437 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94021070 con 0x7f0a9c107400 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a9c1982b0 con 0x7f0a9c107400 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0a99ffb700 1 -- 172.21.15.52:0/593481437 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a94007e30 con 0x7f0a9c107400 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.986+0000 7f0a99ffb700 1 -- 172.21.15.52:0/593481437 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a940047d0 con 0x7f0a9c107400 2024-06-02T23:10:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.987+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a9c06abc0 con 0x7f0a9c107400 2024-06-02T23:10:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.989+0000 7f0a99ffb700 1 -- 172.21.15.52:0/593481437 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a94007980 con 0x7f0a9c107400 2024-06-02T23:10:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.989+0000 7f0a99ffb700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a880a2c20 0x7f0a880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.990+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a880a2c20 0x7f0a880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.990+0000 7f0a99ffb700 1 -- 172.21.15.52:0/593481437 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a9400f020 con 0x7f0a9c107400 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.990+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a880a2c20 0x7f0a880a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005fd0 tx=0x7f0a8c005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.990+0000 7f0aad62eb80 1 Processor -- start 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.990+0000 7f0aad62eb80 1 -- start start 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c0049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593bffe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bfff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55593bf507c0 con 0x55593c0045f0 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55593bef86d0 con 0x55593c004f10 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55593bf4c780 con 0x55593bffeed0 2024-06-02T23:10:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c0049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c0049d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46164/0 (socket says 172.21.15.52:46164) 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 -- 172.21.15.52:0/195354384 learned_addr learned my addr 172.21.15.52:0/195354384 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593bffe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa1348700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bfff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 -- 172.21.15.52:0/195354384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 msgr2=0x55593bfff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bfff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 -- 172.21.15.52:0/195354384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 msgr2=0x55593bffe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593bffe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 -- 172.21.15.52:0/195354384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593beeef00 con 0x55593c0045f0 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593bffe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa1348700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bfff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c0049d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00ec00 tx=0x7f0a8c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1832d75705b57c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0a92ffd700 1 -- 172.21.15.52:0/195354384 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c011a80 con 0x55593c0045f0 2024-06-02T23:10:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0a92ffd700 1 -- 172.21.15.52:0/195354384 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c023070 con 0x55593c0045f0 2024-06-02T23:10:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.991+0000 7f0a92ffd700 1 -- 172.21.15.52:0/195354384 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c0103d0 con 0x55593c0045f0 2024-06-02T23:10:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 -- 172.21.15.52:0/195354384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 msgr2=0x55593c0049d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c0049d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c00ec00 tx=0x7f0a8c011000 comp rx=0 tx=0).stop 2024-06-02T23:10:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 -- 172.21.15.52:0/195354384 shutdown_connections 2024-06-02T23:10:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bfff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c0049d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/195354384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593bffe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 -- 172.21.15.52:0/195354384 >> 172.21.15.52:0/195354384 conn(0x55593bf29c70 msgr2=0x55593bff9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 -- 172.21.15.52:0/195354384 shutdown_connections 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 -- 172.21.15.52:0/195354384 wait complete. 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.992+0000 7f0aad62eb80 1 -- start start 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 Processor -- start 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c00ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c007f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c004f10 0x55593c008470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55593c00b920 con 0x55593c0045f0 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55593bef0ba0 con 0x55593bffeed0 2024-06-02T23:10:42.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55593c00baa0 con 0x55593c004f10 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aad62eb80 10 client.0 _ll_register_callbacks cb 0x55593beea8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c007f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c007f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46176/0 (socket says 172.21.15.52:46176) 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0a9bfff700 1 -- 172.21.15.52:0/3095947850 learned_addr learned my addr 172.21.15.52:0/3095947850 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aa1348700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c004f10 0x55593c008470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.993+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c00ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a9bfff700 1 -- 172.21.15.52:0/3095947850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c004f10 msgr2=0x55593c008470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c004f10 0x55593c008470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a9bfff700 1 -- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 msgr2=0x55593c00ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c00ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a9bfff700 1 -- 172.21.15.52:0/3095947850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593be9f880 con 0x55593c0045f0 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c00ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0aa1348700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c004f10 0x55593c008470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c007f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007fa0 tx=0x7f0a840127e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c8170a0da32f26 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a7bfff700 1 -- 172.21.15.52:0/3095947850 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84010930 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a7bfff700 1 -- 172.21.15.52:0/3095947850 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a84010ad0 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a8c023070 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a7bfff700 1 -- 172.21.15.52:0/3095947850 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a84010cd0 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0aad62eb80 10 client.15162 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55593c0161d0 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.994+0000 7f0a7bfff700 1 -- 172.21.15.52:0/3095947850 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a8c023070 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0a7bfff700 1 -- 172.21.15.52:0/3095947850 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a84019070 con 0x55593c0045f0 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aad62eb80 10 client.15162 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aad62eb80 10 client.15162 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aad62eb80 20 client.15162 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c018870 0x55593c019ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aad62eb80 4 client.15162 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55593bf5dcc0 con 0x55593c018870 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.995+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c018870 0x55593c019ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c018870 0x55593c019ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b5debfd30e8f33bc server_cookie=ee6ad607166b39b7 in_seq=0 out_seq=0 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0a7bfff700 10 client.15162 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0a7bfff700 1 -- 172.21.15.52:0/3095947850 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55593bf5dcc0 con 0x55593c018870 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0a7bfff700 10 client.15162 handle_command_reply: tid=0 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0aad62eb80 1 client.15162 shutdown 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 msgr2=0x55593c007f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.996+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c007f30 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007fa0 tx=0x7f0a840127e0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 shutdown_connections 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c018870 0x55593c019ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c004f10 0x55593c008470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c00ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/3095947850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c007f30 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 >> 172.21.15.52:0/3095947850 conn(0x55593bf29c70 msgr2=0x55593bffd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0a7bfff700 0 client.15162 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 shutdown_connections 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 1 -- 172.21.15.52:0/3095947850 wait complete. 2024-06-02T23:10:43.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:42.997+0000 7f0aad62eb80 20 client.15162 trim_cache size 0 max 16384 2024-06-02T23:10:43.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.018+0000 7f0aad62eb80 1 Processor -- start 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.018+0000 7f0aad62eb80 1 -- start start 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bffe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c023140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593c023680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55593c018020 con 0x55593c0045f0 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55593c0186f0 con 0x55593c004f10 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55593c018dc0 con 0x55593bffeed0 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c023140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c023140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46178/0 (socket says 172.21.15.52:46178) 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 -- 172.21.15.52:0/2404743376 learned_addr learned my addr 172.21.15.52:0/2404743376 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aa1348700 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593c023680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bffe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 -- 172.21.15.52:0/2404743376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 msgr2=0x55593bffe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bffe990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 -- 172.21.15.52:0/2404743376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 msgr2=0x55593c023680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593c023680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 -- 172.21.15.52:0/2404743376 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55593c0040a0 con 0x55593c0045f0 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0aa1348700 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593c023680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c023140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007c40 tx=0x7f0a84026420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5b87514cb3f1332 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.019+0000 7f0a927fc700 1 -- 172.21.15.52:0/2404743376 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a84010e70 con 0x55593c0045f0 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.020+0000 7f0a927fc700 1 -- 172.21.15.52:0/2404743376 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a84026b90 con 0x55593c0045f0 2024-06-02T23:10:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.020+0000 7f0a927fc700 1 -- 172.21.15.52:0/2404743376 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a840054d0 con 0x55593c0045f0 2024-06-02T23:10:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.020+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2404743376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 msgr2=0x55593c023140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.020+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c023140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0a84007c40 tx=0x7f0a84026420 comp rx=0 tx=0).stop 2024-06-02T23:10:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2404743376 shutdown_connections 2024-06-02T23:10:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593c004f10 0x55593c023680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593bffeed0 0x55593bffe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2404743376 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c0045f0 0x55593c023140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2404743376 >> 172.21.15.52:0/2404743376 conn(0x55593bf29c70 msgr2=0x55593c006820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2404743376 shutdown_connections 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2404743376 wait complete. 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 -- start start 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.021+0000 7f0aad62eb80 1 Processor -- start 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c023160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c0045f0 0x55593c0236a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 0x55593c01e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55593c019b60 con 0x55593c004f10 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55593c019490 con 0x55593bffeed0 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55593c01a230 con 0x55593c0045f0 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 0x55593c01e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aad62eb80 10 client.0 _ll_register_callbacks cb 0x55593beea8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 0x55593c01e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46192/0 (socket says 172.21.15.52:46192) 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 -- 172.21.15.52:0/2329530352 learned_addr learned my addr 172.21.15.52:0/2329530352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c023160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0a9bfff700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c0045f0 0x55593c0236a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 -- 172.21.15.52:0/2329530352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c0045f0 msgr2=0x55593c0236a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c0045f0 0x55593c0236a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 -- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 msgr2=0x55593c023160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c023160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.022+0000 7f0aa1348700 1 -- 172.21.15.52:0/2329530352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55593c01e650 con 0x55593c004f10 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aa1348700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 0x55593c01e110 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a94099330 tx=0x7f0a9400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652376d6e211cd7d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0a7bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0a7bfff700 1 -- 172.21.15.52:0/2329530352 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9409d5d0 con 0x55593c004f10 2024-06-02T23:10:43.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0a7bfff700 1 -- 172.21.15.52:0/2329530352 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a9402ddb0 con 0x55593c004f10 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a84026b90 con 0x55593c004f10 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0a7bfff700 1 -- 172.21.15.52:0/2329530352 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0a9409ced0 con 0x55593c004f10 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 10 client.15168 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55593c084860 con 0x55593c004f10 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0a7bfff700 1 -- 172.21.15.52:0/2329530352 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a84026b90 con 0x55593c004f10 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0a7bfff700 1 -- 172.21.15.52:0/2329530352 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0a94011070 con 0x55593c004f10 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 10 client.15168 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 10 client.15168 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 20 client.15168 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c089490 0x55593c0881a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:43.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 4 client.15168 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:43.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.023+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "status"}) -- 0x55593bf58240 con 0x55593c089490 2024-06-02T23:10:43.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.024+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c089490 0x55593c0881a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:43.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.024+0000 7f0a7bfff700 10 client.15168 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:10:43.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.024+0000 7f0aa0b47700 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c089490 0x55593c0881a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=303f6d3a673b3733 server_cookie=715565ce75c9d5b0 in_seq=0 out_seq=0 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.024+0000 7f0a7bfff700 1 -- 172.21.15.52:0/2329530352 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55593bf58240 con 0x55593c089490 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.024+0000 7f0a7bfff700 10 client.15168 handle_command_reply: tid=0 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.024+0000 7f0aad62eb80 1 client.15168 shutdown 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 msgr2=0x55593c01e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 0x55593c01e110 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f0a94099330 tx=0x7f0a9400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 shutdown_connections 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55593c089490 0x55593c0881a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0a7bfff700 0 client.15168 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55593c004f10 0x55593c01e110 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55593bffeed0 0x55593c023160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 --2- 172.21.15.52:0/2329530352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55593c0045f0 0x55593c0236a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 >> 172.21.15.52:0/2329530352 conn(0x55593bf29c70 msgr2=0x55593bff2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 shutdown_connections 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 1 -- 172.21.15.52:0/2329530352 wait complete. 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aad62eb80 20 client.15168 trim_cache size 0 max 16384 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-06-02T23:10:43.030 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24449, 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 173.772032389, 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 221.16129719 2024-06-02T23:10:43.031 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a880a2c20 msgr2=0x7f0a880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a880a2c20 0x7f0a880a50e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c005fd0 tx=0x7f0a8c005ee0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.025+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 msgr2=0x7f0a9c197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 0x7f0a9c197840 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400b830 tx=0x7f0a94004ce0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 shutdown_connections 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a880a2c20 0x7f0a880a50e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a9c107400 0x7f0a9c197840 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a9c1013c0 0x7f0a9c19d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 --2- 172.21.15.52:0/593481437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a9c101d60 0x7f0a9c19dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 >> 172.21.15.52:0/593481437 conn(0x7f0a9c078330 msgr2=0x7f0a9c100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:43.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 shutdown_connections 2024-06-02T23:10:43.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:43.026+0000 7f0aa2dab700 1 -- 172.21.15.52:0/593481437 wait complete. 2024-06-02T23:10:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:43 smithi174 ceph-mon[35994]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:43.398 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:43 smithi052 ceph-mon[32340]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:43.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:43 smithi123 ceph-mon[35977]: pgmap v396: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:43.729 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-02T23:10:43.979 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:45.324 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:45 smithi052 ceph-mon[32340]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:45 smithi174 ceph-mon[35994]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:45.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:45 smithi123 ceph-mon[35977]: pgmap v397: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 -- 172.21.15.52:0/2086616442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040107400 msgr2=0x7ff0401077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 --2- 172.21.15.52:0/2086616442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040107400 0x7ff0401077e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff030009990 tx=0x7ff030009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 -- 172.21.15.52:0/2086616442 shutdown_connections 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 --2- 172.21.15.52:0/2086616442 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101d50 0x7ff04010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 --2- 172.21.15.52:0/2086616442 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040107400 0x7ff0401077e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 --2- 172.21.15.52:0/2086616442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff040101390 0x7ff040101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 -- 172.21.15.52:0/2086616442 >> 172.21.15.52:0/2086616442 conn(0x7ff040078340 msgr2=0x7ff040078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 -- 172.21.15.52:0/2086616442 shutdown_connections 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.696+0000 7ff04633d700 1 -- 172.21.15.52:0/2086616442 wait complete. 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 Processor -- start 2024-06-02T23:10:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 -- start start 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101390 0x7ff04019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 0x7ff04019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff040107400 0x7ff040198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff040111620 con 0x7ff040101d50 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401114a0 con 0x7ff040101390 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff04633d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0401117a0 con 0x7ff040107400 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 0x7ff04019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 0x7ff04019edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54156/0 (socket says 172.21.15.52:54156) 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3100221685 learned_addr learned my addr 172.21.15.52:0/3100221685 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03ffff700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101390 0x7ff04019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff0448da700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff040107400 0x7ff040198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3100221685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff040107400 msgr2=0x7ff040198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff040107400 0x7ff040198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3100221685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101390 msgr2=0x7ff04019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101390 0x7ff04019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3100221685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0401991b0 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.697+0000 7ff03ffff700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101390 0x7ff04019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.698+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 0x7ff04019edd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff03400b530 tx=0x7ff03400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.698+0000 7ff03d7fa700 1 -- 172.21.15.52:0/3100221685 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034015020 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.698+0000 7ff03d7fa700 1 -- 172.21.15.52:0/3100221685 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff034011ce0 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.698+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0401994a0 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.698+0000 7ff03d7fa700 1 -- 172.21.15.52:0/3100221685 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034010d60 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.698+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0401a5760 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.700+0000 7ff03d7fa700 1 -- 172.21.15.52:0/3100221685 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff034018020 con 0x7ff040101d50 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.701+0000 7ff03d7fa700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0280a2c20 0x7ff0280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.701+0000 7ff03ffff700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0280a2c20 0x7ff0280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.701+0000 7ff03d7fa700 1 -- 172.21.15.52:0/3100221685 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff0340ce8b0 con 0x7ff040101d50 2024-06-02T23:10:45.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.701+0000 7ff050bc0b80 1 Processor -- start 2024-06-02T23:10:45.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.701+0000 7ff050bc0b80 1 -- start start 2024-06-02T23:10:45.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.701+0000 7ff03ffff700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0280a2c20 0x7ff0280a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff030009750 tx=0x7ff03001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:45.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.702+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b83464e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.702+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b834648210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.702+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834648b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.702+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b83453af10 con 0x55b83464e830 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.702+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b834599800 con 0x55b834648750 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.702+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b834501410 con 0x55b83464df10 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834648b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834648b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44080/0 (socket says 172.21.15.52:44080) 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b834648210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 -- 172.21.15.52:0/1454343754 learned_addr learned my addr 172.21.15.52:0/1454343754 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff03ffff700 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b83464e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 -- 172.21.15.52:0/1454343754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 msgr2=0x55b83464e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b83464e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 -- 172.21.15.52:0/1454343754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 msgr2=0x55b834648210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b834648210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 -- 172.21.15.52:0/1454343754 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8344e8880 con 0x55b834648750 2024-06-02T23:10:45.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b834648210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:45.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0448da700 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834648b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff038009f20 tx=0x7ff038009f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b97cceda7c5ce0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:45.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.703+0000 7ff0267fc700 1 -- 172.21.15.52:0/1454343754 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03801a070 con 0x55b834648750 2024-06-02T23:10:45.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff0267fc700 1 -- 172.21.15.52:0/1454343754 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff038015410 con 0x55b834648750 2024-06-02T23:10:45.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff0267fc700 1 -- 172.21.15.52:0/1454343754 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03800a660 con 0x55b834648750 2024-06-02T23:10:45.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 -- 172.21.15.52:0/1454343754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 msgr2=0x55b834648b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834648b70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff038009f20 tx=0x7ff038009f50 comp rx=0 tx=0).stop 2024-06-02T23:10:45.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 -- 172.21.15.52:0/1454343754 shutdown_connections 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834648b70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b83464e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/1454343754 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b834648210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 -- 172.21.15.52:0/1454343754 >> 172.21.15.52:0/1454343754 conn(0x55b834575030 msgr2=0x55b8346431d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 -- 172.21.15.52:0/1454343754 shutdown_connections 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.704+0000 7ff050bc0b80 1 -- 172.21.15.52:0/1454343754 wait complete. 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 -- start start 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 Processor -- start 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 0x55b8346546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464df10 0x55b83464c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464e830 0x55b83464ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b834539f70 con 0x55b834648750 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b8345957c0 con 0x55b83464df10 2024-06-02T23:10:45.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b834655020 con 0x55b83464e830 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff050bc0b80 10 client.0 _ll_register_callbacks cb 0x55b83452f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 0x55b8346546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff03ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 0x55b8346546c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54180/0 (socket says 172.21.15.52:54180) 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff03ffff700 1 -- 172.21.15.52:0/574346495 learned_addr learned my addr 172.21.15.52:0/574346495 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.705+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464df10 0x55b83464c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff0448da700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464e830 0x55b83464ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff03ffff700 1 -- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464e830 msgr2=0x55b83464ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff03ffff700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464e830 0x55b83464ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff03ffff700 1 -- 172.21.15.52:0/574346495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464df10 msgr2=0x55b83464c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff03ffff700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464df10 0x55b83464c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff03ffff700 1 -- 172.21.15.52:0/574346495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83455f700 con 0x55b834648750 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff03ffff700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 0x55b8346546c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff03000bfd0 tx=0x7ff030009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cd6d54b6567fcd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:45.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff01f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff01f7fe700 1 -- 172.21.15.52:0/574346495 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff030023820 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff01f7fe700 1 -- 172.21.15.52:0/574346495 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff0300239c0 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff038015410 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff01f7fe700 1 -- 172.21.15.52:0/574346495 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff030023bc0 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff050bc0b80 10 client.15180 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.706+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8346681f0 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff01f7fe700 1 -- 172.21.15.52:0/574346495 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff038015410 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff01f7fe700 1 -- 172.21.15.52:0/574346495 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff03002e3f0 con 0x55b834648750 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff050bc0b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff050bc0b80 10 client.15180 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff050bc0b80 20 client.15180 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b83466a880 0x55b83466bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff050bc0b80 4 client.15180 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:45.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b8345a6d00 con 0x55b83466a880 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.707+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b83466a880 0x55b83466bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.708+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b83466a880 0x55b83466bcf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=800863984d65e0cf server_cookie=e58da3e0d5d0fdfe in_seq=0 out_seq=0 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.708+0000 7ff01f7fe700 10 client.15180 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.708+0000 7ff01f7fe700 1 -- 172.21.15.52:0/574346495 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b8345a6d00 con 0x55b83466a880 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.708+0000 7ff01f7fe700 10 client.15180 handle_command_reply: tid=0 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.708+0000 7ff050bc0b80 1 client.15180 shutdown 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 msgr2=0x55b8346546c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 0x55b8346546c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff03000bfd0 tx=0x7ff030009310 comp rx=0 tx=0).stop 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 shutdown_connections 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b83466a880 0x55b83466bcf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464e830 0x55b83464ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b834648750 0x55b8346546c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff01f7fe700 0 client.15180 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:45.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/574346495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464df10 0x55b83464c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 >> 172.21.15.52:0/574346495 conn(0x55b834575030 msgr2=0x55b8346426b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:45.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 shutdown_connections 2024-06-02T23:10:45.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 1 -- 172.21.15.52:0/574346495 wait complete. 2024-06-02T23:10:45.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.709+0000 7ff050bc0b80 20 client.15180 trim_cache size 0 max 16384 2024-06-02T23:10:45.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 Processor -- start 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 -- start start 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b834648750 0x55b834648210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 0x55b83463f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464e830 0x55b8346588e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b83466a030 con 0x55b83464df10 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.730+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b83466a700 con 0x55b83464e830 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b83466add0 con 0x55b834648750 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 0x55b83463f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 0x55b83463f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54192/0 (socket says 172.21.15.52:54192) 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3924716488 learned_addr learned my addr 172.21.15.52:0/3924716488 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03ffff700 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b834648750 0x55b834648210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff0448da700 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464e830 0x55b8346588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3924716488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b834648750 msgr2=0x55b834648210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b834648750 0x55b834648210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3924716488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464e830 msgr2=0x55b8346588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464e830 0x55b8346588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 -- 172.21.15.52:0/3924716488 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83464ae00 con 0x55b83464df10 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff0448da700 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464e830 0x55b8346588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 0x55b83463f320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff0340975a0 tx=0x7ff0340cc2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d787b6d2bfe1cb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff025ffb700 1 -- 172.21.15.52:0/3924716488 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0340b3070 con 0x55b83464df10 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff025ffb700 1 -- 172.21.15.52:0/3924716488 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff03409f4f0 con 0x55b83464df10 2024-06-02T23:10:45.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.731+0000 7ff025ffb700 1 -- 172.21.15.52:0/3924716488 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03409b3a0 con 0x55b83464df10 2024-06-02T23:10:45.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 -- 172.21.15.52:0/3924716488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 msgr2=0x55b83463f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 0x55b83463f320 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff0340975a0 tx=0x7ff0340cc2b0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 -- 172.21.15.52:0/3924716488 shutdown_connections 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b83464e830 0x55b8346588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b834648750 0x55b834648210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/3924716488 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464df10 0x55b83463f320 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 -- 172.21.15.52:0/3924716488 >> 172.21.15.52:0/3924716488 conn(0x55b834575030 msgr2=0x55b8346581d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 -- 172.21.15.52:0/3924716488 shutdown_connections 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 -- 172.21.15.52:0/3924716488 wait complete. 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.732+0000 7ff050bc0b80 1 -- start start 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 Processor -- start 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834666c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b834667160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b8346676a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b83466bb70 con 0x55b83464e830 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b83466b4a0 con 0x55b834648750 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b83466c240 con 0x55b83464df10 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff050bc0b80 10 client.0 _ll_register_callbacks cb 0x55b83452f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b8346676a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b8346676a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54200/0 (socket says 172.21.15.52:54200) 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 -- 172.21.15.52:0/322470456 learned_addr learned my addr 172.21.15.52:0/322470456 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff03f7fe700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b834667160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff03ffff700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834666c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 -- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 msgr2=0x55b834667160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b834667160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 -- 172.21.15.52:0/322470456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 msgr2=0x55b834666c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834666c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 -- 172.21.15.52:0/322470456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b834642830 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff0448da700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b8346676a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c700 tx=0x7ff03800cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b781491233c66045 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.733+0000 7ff01f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff01f7fe700 1 -- 172.21.15.52:0/322470456 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038024070 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff01f7fe700 1 -- 172.21.15.52:0/322470456 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff038015410 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03409f4f0 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff01f7fe700 1 -- 172.21.15.52:0/322470456 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff03801a040 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 10 client.15186 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55b8346d6430 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff01f7fe700 1 -- 172.21.15.52:0/322470456 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff03409f4f0 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff01f7fe700 1 -- 172.21.15.52:0/322470456 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff03801f900 con 0x55b83464e830 2024-06-02T23:10:45.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 10 client.15186 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 10 client.15186 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 20 client.15186 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b8346dac30 0x55b8346d9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 4 client.15186 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.734+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "status"}) -- 0x55b8345a1280 con 0x55b8346dac30 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff03ffff700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b8346dac30 0x55b8346d9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff03ffff700 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b8346dac30 0x55b8346d9940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=deb2cd6b5ed0d87f server_cookie=d60c17590f74b6c6 in_seq=0 out_seq=0 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff01f7fe700 10 client.15186 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff01f7fe700 1 -- 172.21.15.52:0/322470456 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b8345a1280 con 0x55b8346dac30 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff01f7fe700 10 client.15186 handle_command_reply: tid=0 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 client.15186 shutdown 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 msgr2=0x55b8346676a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b8346676a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c700 tx=0x7ff03800cef0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 shutdown_connections 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b8346dac30 0x55b8346d9940 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b83464e830 0x55b8346676a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b834648750 0x55b834666c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff01f7fe700 0 client.15186 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 --2- 172.21.15.52:0/322470456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b83464df10 0x55b834667160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 >> 172.21.15.52:0/322470456 conn(0x55b834575030 msgr2=0x55b83464c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.735+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 shutdown_connections 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff050bc0b80 1 -- 172.21.15.52:0/322470456 wait complete. 2024-06-02T23:10:45.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff050bc0b80 20 client.15186 trim_cache size 0 max 16384 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24451, 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 177.49046905, 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:45.742 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 221.06302195000001 2024-06-02T23:10:45.743 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:45.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0280a2c20 msgr2=0x7ff0280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0280a2c20 0x7ff0280a50e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff030009750 tx=0x7ff03001a040 comp rx=0 tx=0).stop 2024-06-02T23:10:45.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 msgr2=0x7ff04019edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:45.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 0x7ff04019edd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7ff03400b530 tx=0x7ff03400b8f0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 shutdown_connections 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0280a2c20 0x7ff0280a50e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff040107400 0x7ff040198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff040101390 0x7ff04019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 --2- 172.21.15.52:0/3100221685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff040101d50 0x7ff04019edd0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 >> 172.21.15.52:0/3100221685 conn(0x7ff040078340 msgr2=0x7ff04006a310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 shutdown_connections 2024-06-02T23:10:45.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:45.736+0000 7ff04633d700 1 -- 172.21.15.52:0/3100221685 wait complete. 2024-06-02T23:10:46.435 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-02T23:10:46.684 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:47 smithi174 ceph-mon[35994]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:47 smithi052 ceph-mon[32340]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:47.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:47 smithi123 ceph-mon[35977]: pgmap v398: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:48.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 -- 172.21.15.52:0/4254102417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0107840 msgr2=0x7fdff010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 --2- 172.21.15.52:0/4254102417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0107840 0x7fdff010f3c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b0d0 tx=0x7fdfe800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 -- 172.21.15.52:0/4254102417 shutdown_connections 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 --2- 172.21.15.52:0/4254102417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0107840 0x7fdff010f3c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 --2- 172.21.15.52:0/4254102417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0102740 0x7fdff0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 --2- 172.21.15.52:0/4254102417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 0x7fdff0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 -- 172.21.15.52:0/4254102417 >> 172.21.15.52:0/4254102417 conn(0x7fdff00795f0 msgr2=0x7fdff0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 -- 172.21.15.52:0/4254102417 shutdown_connections 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 -- 172.21.15.52:0/4254102417 wait complete. 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 Processor -- start 2024-06-02T23:10:48.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.414+0000 7fdff852b700 1 -- start start 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff852b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 0x7fdff019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff852b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 0x7fdff019ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff852b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0107840 0x7fdff01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff852b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdff01126e0 con 0x7fdff0102740 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff852b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0112560 con 0x7fdff0107840 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff852b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0112860 con 0x7fdff01030f0 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 0x7fdff019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 0x7fdff019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54204/0 (socket says 172.21.15.52:54204) 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 -- 172.21.15.52:0/361906597 learned_addr learned my addr 172.21.15.52:0/361906597 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 -- 172.21.15.52:0/361906597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 msgr2=0x7fdff019ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 0x7fdff019ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0107840 0x7fdff01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 0x7fdff019ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 -- 172.21.15.52:0/361906597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0107840 msgr2=0x7fdff01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0107840 0x7fdff01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 -- 172.21.15.52:0/361906597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff0199100 con 0x7fdff0102740 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0107840 0x7fdff01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 0x7fdff019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.415+0000 7fdff62c7700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 0x7fdff019e790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b530 tx=0x7fdfec00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.416+0000 7fdfe77fe700 1 -- 172.21.15.52:0/361906597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec015020 con 0x7fdff0102740 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.416+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff01993f0 con 0x7fdff0102740 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.416+0000 7fdfe77fe700 1 -- 172.21.15.52:0/361906597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfec011ce0 con 0x7fdff0102740 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.416+0000 7fdfe77fe700 1 -- 172.21.15.52:0/361906597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec010d60 con 0x7fdff0102740 2024-06-02T23:10:48.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.416+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01a56e0 con 0x7fdff0102740 2024-06-02T23:10:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.418+0000 7fdfe77fe700 1 -- 172.21.15.52:0/361906597 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdfec018020 con 0x7fdff0102740 2024-06-02T23:10:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.419+0000 7fdfe77fe700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdfdc0a2c20 0x7fdfdc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.419+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdfdc0a2c20 0x7fdfdc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.419+0000 7fdfe77fe700 1 -- 172.21.15.52:0/361906597 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdfec0ce900 con 0x7fdff0102740 2024-06-02T23:10:48.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdfdc0a2c20 0x7fdfdc0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0005fd0 tx=0x7fdfe0005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:48.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 Processor -- start 2024-06-02T23:10:48.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 -- start start 2024-06-02T23:10:48.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 0x55bc62e08e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e09340 0x55bc62e02da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e03700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc62d507c0 con 0x55bc62e08a20 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc62cf86d0 con 0x55bc62e09340 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.420+0000 7fe002daeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc62d4c780 con 0x55bc62e032e0 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 0x55bc62e08e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 0x55bc62e08e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54216/0 (socket says 172.21.15.52:54216) 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 -- 172.21.15.52:0/1963872195 learned_addr learned my addr 172.21.15.52:0/1963872195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e09340 0x55bc62e02da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e03700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 -- 172.21.15.52:0/1963872195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 msgr2=0x55bc62e03700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e03700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 -- 172.21.15.52:0/1963872195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e09340 msgr2=0x55bc62e02da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e09340 0x55bc62e02da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 -- 172.21.15.52:0/1963872195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc62ceef00 con 0x55bc62e08a20 2024-06-02T23:10:48.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdff62c7700 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 0x55bc62e08e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdfec097590 tx=0x7fdfec09ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d30db83b0aca9a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:48.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdfe4ff9700 1 -- 172.21.15.52:0/1963872195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec09b830 con 0x55bc62e08a20 2024-06-02T23:10:48.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdfe4ff9700 1 -- 172.21.15.52:0/1963872195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfec09b130 con 0x55bc62e08a20 2024-06-02T23:10:48.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.421+0000 7fdfe4ff9700 1 -- 172.21.15.52:0/1963872195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec09f160 con 0x55bc62e08a20 2024-06-02T23:10:48.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 -- 172.21.15.52:0/1963872195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 msgr2=0x55bc62e08e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 0x55bc62e08e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fdfec097590 tx=0x7fdfec09ab10 comp rx=0 tx=0).stop 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 -- 172.21.15.52:0/1963872195 shutdown_connections 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e03700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e08a20 0x55bc62e08e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1963872195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e09340 0x55bc62e02da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 -- 172.21.15.52:0/1963872195 >> 172.21.15.52:0/1963872195 conn(0x55bc62d29c70 msgr2=0x55bc62df9800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 -- 172.21.15.52:0/1963872195 shutdown_connections 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.422+0000 7fe002daeb80 1 -- 172.21.15.52:0/1963872195 wait complete. 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 -- start start 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 Processor -- start 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e0f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e0c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e0c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e0fd20 con 0x55bc62e09340 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc62cf0ba0 con 0x55bc62e08a20 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e0fea0 con 0x55bc62e032e0 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fe002daeb80 10 client.0 _ll_register_callbacks cb 0x55bc62cea8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e0c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e0c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54222/0 (socket says 172.21.15.52:54222) 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff5ac6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e0c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 -- 172.21.15.52:0/914415220 learned_addr learned my addr 172.21.15.52:0/914415220 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff62c7700 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e0f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 -- 172.21.15.52:0/914415220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 msgr2=0x55bc62e0f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e0f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 -- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 msgr2=0x55bc62e0c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e0c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 -- 172.21.15.52:0/914415220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc62c9f880 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e0c990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80094e0 tx=0x7fdfe8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80284991294cc5ea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.423+0000 7fdfd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/914415220 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80076e0 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/914415220 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdfe8007880 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfec09b130 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/914415220 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfe802e070 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 10 client.15198 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc62e1a670 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/914415220 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfec09b130 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/914415220 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdfe8011070 con 0x55bc62e09340 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 10 client.15198 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 10 client.15198 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T23:10:48.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 20 client.15198 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e1ccb0 0x55bc62e1e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 4 client.15198 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.424+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc62d5dcc0 con 0x55bc62e1ccb0 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.425+0000 7fdff62c7700 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e1ccb0 0x55bc62e1e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.425+0000 7fdff62c7700 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e1ccb0 0x55bc62e1e120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda6b7b26234e4f2 server_cookie=7462c092ad186179 in_seq=0 out_seq=0 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.425+0000 7fdfd9ffb700 10 client.15198 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/914415220 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc62d5dcc0 con 0x55bc62e1ccb0 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fdfd9ffb700 10 client.15198 handle_command_reply: tid=0 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 client.15198 shutdown 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 msgr2=0x55bc62e0c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e0c990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdfe80094e0 tx=0x7fdfe8009510 comp rx=0 tx=0).stop 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 shutdown_connections 2024-06-02T23:10:48.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e1ccb0 0x55bc62e1e120 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e0c990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e0f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 --2- 172.21.15.52:0/914415220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e0c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fdfd9ffb700 0 client.15198 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 >> 172.21.15.52:0/914415220 conn(0x55bc62d29c70 msgr2=0x55bc62dfd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 shutdown_connections 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 1 -- 172.21.15.52:0/914415220 wait complete. 2024-06-02T23:10:48.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.426+0000 7fe002daeb80 20 client.15198 trim_cache size 0 max 16384 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 Processor -- start 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 -- start start 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e097e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e09d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e27580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e1c460 con 0x55bc62e09340 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e1cb30 con 0x55bc62e08a20 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fe002daeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e1d200 con 0x55bc62e032e0 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e27580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e27580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54238/0 (socket says 172.21.15.52:54238) 2024-06-02T23:10:48.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 -- 172.21.15.52:0/1275739205 learned_addr learned my addr 172.21.15.52:0/1275739205 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 -- 172.21.15.52:0/1275739205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 msgr2=0x55bc62e097e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e097e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 -- 172.21.15.52:0/1275739205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 msgr2=0x55bc62e09d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e09d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.448+0000 7fdff6ac8700 1 -- 172.21.15.52:0/1275739205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc62e1f080 con 0x55bc62e09340 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e27580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b0d0 tx=0x7fdfe8006080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ac9e06d0c65887 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fdfdbfff700 1 -- 172.21.15.52:0/1275739205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe8004e40 con 0x55bc62e09340 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fdfdbfff700 1 -- 172.21.15.52:0/1275739205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfe80059e0 con 0x55bc62e09340 2024-06-02T23:10:48.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fdfdbfff700 1 -- 172.21.15.52:0/1275739205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfe80074a0 con 0x55bc62e09340 2024-06-02T23:10:48.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 -- 172.21.15.52:0/1275739205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 msgr2=0x55bc62e27580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e27580 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdfe800b0d0 tx=0x7fdfe8006080 comp rx=0 tx=0).stop 2024-06-02T23:10:48.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 -- 172.21.15.52:0/1275739205 shutdown_connections 2024-06-02T23:10:48.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e09340 0x55bc62e27580 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e032e0 0x55bc62e097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 --2- 172.21.15.52:0/1275739205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e09d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 -- 172.21.15.52:0/1275739205 >> 172.21.15.52:0/1275739205 conn(0x55bc62d29c70 msgr2=0x55bc62e0ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 -- 172.21.15.52:0/1275739205 shutdown_connections 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.449+0000 7fe002daeb80 1 -- 172.21.15.52:0/1275739205 wait complete. 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 -- start start 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 Processor -- start 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 0x55bc62e27780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e27cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e09340 0x55bc62e22550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e1dfa0 con 0x55bc62e032e0 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e1d8d0 con 0x55bc62e08a20 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc62e1e670 con 0x55bc62e09340 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fe002daeb80 10 client.0 _ll_register_callbacks cb 0x55bc62cea8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:10:48.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff62c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 0x55bc62e27780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff62c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 0x55bc62e27780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54240/0 (socket says 172.21.15.52:54240) 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff62c7700 1 -- 172.21.15.52:0/2888096963 learned_addr learned my addr 172.21.15.52:0/2888096963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff6ac8700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e09340 0x55bc62e22550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e27cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff62c7700 1 -- 172.21.15.52:0/2888096963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e09340 msgr2=0x55bc62e22550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.450+0000 7fdff62c7700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e09340 0x55bc62e22550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdff62c7700 1 -- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 msgr2=0x55bc62e27cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdff62c7700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e27cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdff62c7700 1 -- 172.21.15.52:0/2888096963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc62e22d90 con 0x55bc62e032e0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e27cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdff62c7700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 0x55bc62e27780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdfec09ae30 tx=0x7fdfec096d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38721320b0ef3cb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdfd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/2888096963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfec09b830 con 0x55bc62e032e0 2024-06-02T23:10:48.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/2888096963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdfec09f160 con 0x55bc62e032e0 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdfe80059e0 con 0x55bc62e032e0 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/2888096963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdfec067020 con 0x55bc62e032e0 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fe002daeb80 10 client.15204 fetch_fsmap learned FSMap version 27 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55bc62e89000 con 0x55bc62e032e0 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/2888096963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdfe80059e0 con 0x55bc62e032e0 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/2888096963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdfec0bb0b0 con 0x55bc62e032e0 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fe002daeb80 10 client.15204 fetch_fsmap finished waiting for FSMap version 27 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.451+0000 7fe002daeb80 10 client.15204 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fe002daeb80 20 client.15204 populate_metadata read hostname 'smithi052' 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fe002daeb80 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e8d650 0x55bc62e8c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fe002daeb80 4 client.15204 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "status"}) -- 0x55bc62d58240 con 0x55bc62e8d650 2024-06-02T23:10:48.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e8d650 0x55bc62e8c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fdff5ac6700 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e8d650 0x55bc62e8c360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf7dc61956efb6b server_cookie=238d6d15a5b4307b in_seq=0 out_seq=0 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.452+0000 7fdfd9ffb700 10 client.15204 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fdfd9ffb700 1 -- 172.21.15.52:0/2888096963 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55bc62d58240 con 0x55bc62e8d650 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fdfd9ffb700 10 client.15204 handle_command_reply: tid=0 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 client.15204 shutdown 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 msgr2=0x55bc62e27780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 0x55bc62e27780 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fdfec09ae30 tx=0x7fdfec096d60 comp rx=0 tx=0).stop 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 shutdown_connections 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc62e8d650 0x55bc62e8c360 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc62e09340 0x55bc62e22550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc62e032e0 0x55bc62e27780 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 --2- 172.21.15.52:0/2888096963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc62e08a20 0x55bc62e27cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 >> 172.21.15.52:0/2888096963 conn(0x55bc62d29c70 msgr2=0x55bc62df2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fdfd9ffb700 0 client.15204 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 shutdown_connections 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 1 -- 172.21.15.52:0/2888096963 wait complete. 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.453+0000 7fe002daeb80 20 client.15204 trim_cache size 0 max 16384 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24470, 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:10:48.459 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 194.250975259, 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 27, 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 88, 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 212.94997178 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdfdc0a2c20 msgr2=0x7fdfdc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdfdc0a2c20 0x7fdfdc0a50e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0005fd0 tx=0x7fdfe0005dc0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 msgr2=0x7fdff019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 0x7fdff019e790 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfec00b530 tx=0x7fdfec00b8f0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 shutdown_connections 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdfdc0a2c20 0x7fdfdc0a50e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdff0107840 0x7fdff01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdff0102740 0x7fdff019e790 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 --2- 172.21.15.52:0/361906597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdff01030f0 0x7fdff019ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 >> 172.21.15.52:0/361906597 conn(0x7fdff00795f0 msgr2=0x7fdff0100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 shutdown_connections 2024-06-02T23:10:48.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:48.454+0000 7fdff852b700 1 -- 172.21.15.52:0/361906597 wait complete. 2024-06-02T23:10:49.184 INFO:teuthology.run_tasks:Running task sequential... 2024-06-02T23:10:49.191 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-06-02T23:10:49.191 INFO:teuthology.task.sequential:In sequential, running task print... 2024-06-02T23:10:49.191 INFO:teuthology.task.print:Enabling mgr modules 2024-06-02T23:10:49.191 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-06-02T23:10:49.232 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-06-02T23:10:49.232 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-06-02T23:10:49.233 INFO:tasks.ceph_fuse:client.0 config is {'mount_subvol_num': 0} 2024-06-02T23:10:49.233 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-06-02T23:10:49.233 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:49.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:49.233 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-02T23:10:49.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:49.251 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-06-02T23:10:49.321 INFO:teuthology.orchestra.run.smithi052.stderr:Cannot find device "ceph-brx" 2024-06-02T23:10:49.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:10:49.323 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-06-02T23:10:49.323 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs ls 2024-06-02T23:10:49.385 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:49 smithi052 ceph-mon[32340]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:49 smithi174 ceph-mon[35994]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:49 smithi123 ceph-mon[35977]: pgmap v399: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:49.612 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:51.192 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:51 smithi052 ceph-mon[32340]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 -- 172.21.15.52:0/3123863509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 msgr2=0x7f97901089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 --2- 172.21.15.52:0/3123863509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f97901089d0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9780009990 tx=0x7f9780009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 -- 172.21.15.52:0/3123863509 shutdown_connections 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 --2- 172.21.15.52:0/3123863509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9790102ea0 0x7f979010f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 --2- 172.21.15.52:0/3123863509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f97901089d0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 --2- 172.21.15.52:0/3123863509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97901024e0 0x7f9790102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 -- 172.21.15.52:0/3123863509 >> 172.21.15.52:0/3123863509 conn(0x7f9790079640 msgr2=0x7f9790079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 -- 172.21.15.52:0/3123863509 shutdown_connections 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.299+0000 7f9795d8e700 1 -- 172.21.15.52:0/3123863509 wait complete. 2024-06-02T23:10:51.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.300+0000 7f9795d8e700 1 Processor -- start 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.300+0000 7f9795d8e700 1 -- start start 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.300+0000 7f9795d8e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97901024e0 0x7f9790074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.300+0000 7f9795d8e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9790102ea0 0x7f9790074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.300+0000 7f9795d8e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f9790075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.300+0000 7f9795d8e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9790112640 con 0x7f97901085f0 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f9795d8e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97901124c0 con 0x7f9790102ea0 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f9795d8e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97901127c0 con 0x7f97901024e0 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f9790075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f9790075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54262/0 (socket says 172.21.15.52:54262) 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 -- 172.21.15.52:0/1931862420 learned_addr learned my addr 172.21.15.52:0/1931862420 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978effd700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9790102ea0 0x7f9790074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:51.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978f7fe700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97901024e0 0x7f9790074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 -- 172.21.15.52:0/1931862420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97901024e0 msgr2=0x7f9790074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97901024e0 0x7f9790074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 -- 172.21.15.52:0/1931862420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9790102ea0 msgr2=0x7f9790074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9790102ea0 0x7f9790074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 -- 172.21.15.52:0/1931862420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97901a6f80 con 0x7f97901085f0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978f7fe700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97901024e0 0x7f9790074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978effd700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9790102ea0 0x7f9790074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978ffff700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f9790075210 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f978800cca0 tx=0x7f978800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9788016020 con 0x7f97901085f0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97901a7270 con 0x7f97901085f0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97901a77b0 con 0x7f97901085f0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f978800aba0 con 0x7f97901085f0 2024-06-02T23:10:51.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.301+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978801b660 con 0x7f97901085f0 2024-06-02T23:10:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.303+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c005330 con 0x7f97901085f0 2024-06-02T23:10:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.304+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9788019020 con 0x7f97901085f0 2024-06-02T23:10:51.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.304+0000 7f978cff9700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97780a2c20 0x7f97780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.304+0000 7f978f7fe700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97780a2c20 0x7f97780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.304+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f97880ceb10 con 0x7f97901085f0 2024-06-02T23:10:51.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.305+0000 7f978f7fe700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97780a2c20 0x7f97780a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9780000c00 tx=0x7f9780005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.309+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9788020030 con 0x7f97901085f0 2024-06-02T23:10:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:51 smithi174 ceph-mon[35994]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:51 smithi123 ceph-mon[35977]: pgmap v400: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:10:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.559+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f977c005e30 con 0x7f97901085f0 2024-06-02T23:10:51.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.560+0000 7f978cff9700 1 -- 172.21.15.52:0/1931862420 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v27) ==== 53+0+73 (secure 0 0 0) 0x7f978809baa0 con 0x7f97901085f0 2024-06-02T23:10:51.560 INFO:teuthology.orchestra.run.smithi052.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-06-02T23:10:51.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97780a2c20 msgr2=0x7f97780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97780a2c20 0x7f97780a50e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9780000c00 tx=0x7f9780005fb0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 msgr2=0x7f9790075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f9790075210 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f978800cca0 tx=0x7f978800a060 comp rx=0 tx=0).stop 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 shutdown_connections 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97780a2c20 0x7f97780a50e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97901085f0 0x7f9790075210 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97901024e0 0x7f9790074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 --2- 172.21.15.52:0/1931862420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9790102ea0 0x7f9790074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 >> 172.21.15.52:0/1931862420 conn(0x7f9790079640 msgr2=0x7f9790101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 shutdown_connections 2024-06-02T23:10:51.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:51.563+0000 7f9795d8e700 1 -- 172.21.15.52:0/1931862420 wait complete. 2024-06-02T23:10:52.212 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:52 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1931862420' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi052.front.sepia.ceph.com 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-06-02T23:10:52.240 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-06-02T23:10:52.241 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-06-02T23:10:52.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:52.241 DEBUG:teuthology.orchestra.run.smithi052:> ip addr 2024-06-02T23:10:52.299 INFO:teuthology.orchestra.run.smithi052.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: inet 127.0.0.1/8 scope host lo 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 ::1/128 scope host 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d2 brd ff:ff:ff:ff:ff:ff 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f0 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:6b:fd:d3 brd ff:ff:ff:ff:ff:ff 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp7s0f1 2024-06-02T23:10:52.300 INFO:teuthology.orchestra.run.smithi052.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:de brd ff:ff:ff:ff:ff:ff 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f0 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: inet 172.21.15.52/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft 42073sec preferred_lft 42073sec 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: inet6 fe80::ec4:7aff:febd:15de/64 scope link noprefixroute 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: valid_lft forever preferred_lft forever 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: link/ether 0c:c4:7a:bd:15:df brd ff:ff:ff:ff:ff:ff 2024-06-02T23:10:52.301 INFO:teuthology.orchestra.run.smithi052.stdout: altname enp1s0f1 2024-06-02T23:10:52.301 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add name ceph-brx type bridge 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr flush dev ceph-brx 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set ceph-brx up 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-06-02T23:10:52.302 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-02T23:10:52.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T23:10:52.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:52 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1931862420' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T23:10:52.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T23:10:52.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:52 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1931862420' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-06-02T23:10:52.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:52.512 DEBUG:teuthology.orchestra.run.smithi052:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-06-02T23:10:52.538 INFO:teuthology.orchestra.run.smithi052.stdout:1 2024-06-02T23:10:52.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:52.539 DEBUG:teuthology.orchestra.run.smithi052:> route 2024-06-02T23:10:52.618 INFO:teuthology.orchestra.run.smithi052.stdout:Kernel IP routing table 2024-06-02T23:10:52.618 INFO:teuthology.orchestra.run.smithi052.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-06-02T23:10:52.618 INFO:teuthology.orchestra.run.smithi052.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-06-02T23:10:52.618 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-06-02T23:10:52.618 INFO:teuthology.orchestra.run.smithi052.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-06-02T23:10:52.619 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T23:10:52.619 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-02T23:10:52.619 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-06-02T23:10:52.619 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-06-02T23:10:52.619 DEBUG:teuthology.orchestra.run.smithi052:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-06-02T23:10:52.619 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-02T23:10:52.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:52 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T23:10:52.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:52 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T23:10:52.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:52.919 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-02T23:10:52.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:52.936 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list-id 2024-06-02T23:10:52.991 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T23:10:52.991 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-02T23:10:52.991 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:52.991 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-06-02T23:10:52.991 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-02T23:10:53.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T23:10:53.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T23:10:53.112 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-06-02T23:10:53.112 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T23:10:53.112 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-02T23:10:53.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-06-02T23:10:53.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-06-02T23:10:53.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-06-02T23:10:53.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-06-02T23:10:53.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-06-02T23:10:53.113 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-02T23:10:53.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T23:10:53.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T23:10:53.245 DEBUG:teuthology.orchestra.run.smithi052:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-06-02T23:10:53.246 DEBUG:teuthology.orchestra.run.smithi052:> set -e 2024-06-02T23:10:53.246 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set brx.0 up 2024-06-02T23:10:53.246 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link set dev brx.0 master ceph-brx 2024-06-02T23:10:53.246 DEBUG:teuthology.orchestra.run.smithi052:> ') 2024-06-02T23:10:53.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-06-02T23:10:53.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T23:10:53.327 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mount_subvol_num': 0} 2024-06-02T23:10:53.327 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-02T23:10:53.327 DEBUG:teuthology.orchestra.run.smithi052:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:53.343 INFO:teuthology.orchestra.run.smithi052.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-06-02T23:10:53.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-06-02T23:10:53.343 DEBUG:teuthology.orchestra.run.smithi052:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:53 smithi174 ceph-mon[35994]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:53.399 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-06-02T23:10:53.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:53 smithi052 ceph-mon[32340]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:53.425 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-06-02T23:10:53.483 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-06-02T23:10:53.484 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/70853edc510fdeea84bf634935dde3bbfd5a1d7eb53093b0c63db17d7152099f/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/1d8e72f423afa01c8a7601532200a9d8578833f35ba3b89d64e95ecc90cdac8e/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c061a6ed7cb25f3583cf016c06a1ca84f3bcda232f7968367d8ea735ddc8b690/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/7450634d266c5495c8e0f1e80cc60c4098d1aec9f0924b2d13f080c9cd748678/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/533dec7f1566cb7fa12591be4f5208cc68cb2e650a31b5dc9c79f444898a99af/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d2e5afc77ff58be785fc5c2061ea648baa45b30b3dc114e84fbf6fa75be2b348/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/4bfc5105a78aa9f096693b1e639424cac177c9b1e36c7879bc23219d6fd862ba/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c13c9940b6c2636f325f1a071898b484675e8dbef0fb5aa0c7c285daf0dcb067/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/34d46adee9527392e080494bef62ca29e9d22907499c229849b51a62a390dbfb/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/f1f415d274e5122cc0164e69dba8c45e8e5a225f2e1349729b2161a417e570cb/merged 2024-06-02T23:10:53.485 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-06-02T23:10:53.486 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:53.486 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:53.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:53.486 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-06-02T23:10:53.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:53 smithi123 ceph-mon[35977]: pgmap v401: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:10:53.539 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-06-02T23:10:53.539 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc --client_fs=cephfs) 2024-06-02T23:10:53.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-06-02T23:10:53.607 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-06-02T23:10:53.665 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-06-02T23:10:53.666 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/70853edc510fdeea84bf634935dde3bbfd5a1d7eb53093b0c63db17d7152099f/merged 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/1d8e72f423afa01c8a7601532200a9d8578833f35ba3b89d64e95ecc90cdac8e/merged 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c061a6ed7cb25f3583cf016c06a1ca84f3bcda232f7968367d8ea735ddc8b690/merged 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/7450634d266c5495c8e0f1e80cc60c4098d1aec9f0924b2d13f080c9cd748678/merged 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/533dec7f1566cb7fa12591be4f5208cc68cb2e650a31b5dc9c79f444898a99af/merged 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d2e5afc77ff58be785fc5c2061ea648baa45b30b3dc114e84fbf6fa75be2b348/merged 2024-06-02T23:10:53.667 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/4bfc5105a78aa9f096693b1e639424cac177c9b1e36c7879bc23219d6fd862ba/merged 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c13c9940b6c2636f325f1a071898b484675e8dbef0fb5aa0c7c285daf0dcb067/merged 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/34d46adee9527392e080494bef62ca29e9d22907499c229849b51a62a390dbfb/merged 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/f1f415d274e5122cc0164e69dba8c45e8e5a225f2e1349729b2161a417e570cb/merged 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:53.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:53.668 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-06-02T23:10:53.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:2024-06-02T23:10:53.688+0000 7fddfb016480 -1 init, newargv = 0x5589463b7820 newargc=15 2024-06-02T23:10:53.689 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[87498]: starting ceph client 2024-06-02T23:10:53.710 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[87498]: starting fuse 2024-06-02T23:10:54.807 DEBUG:teuthology.orchestra.run.smithi052:> sudo modprobe fuse 2024-06-02T23:10:54.840 DEBUG:teuthology.orchestra.run.smithi052:> cat /proc/self/mounts | awk '{print $2}' 2024-06-02T23:10:54.901 INFO:teuthology.orchestra.run.smithi052.stdout:/proc 2024-06-02T23:10:54.901 INFO:teuthology.orchestra.run.smithi052.stdout:/sys 2024-06-02T23:10:54.901 INFO:teuthology.orchestra.run.smithi052.stdout:/dev 2024-06-02T23:10:54.901 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/security 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/shm 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/pts 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/run 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/cgroup 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/pstore 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/bpf 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/ 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/selinux 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/mqueue 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/dev/hugepages 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/debug 2024-06-02T23:10:54.902 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/tracing 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/fs/fuse/connections 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/sys/kernel/config 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-sysctl.service 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/nfs/rpc_pipefs 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/1000 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/ceph 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/70853edc510fdeea84bf634935dde3bbfd5a1d7eb53093b0c63db17d7152099f/merged 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/1d8e72f423afa01c8a7601532200a9d8578833f35ba3b89d64e95ecc90cdac8e/merged 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/run/user/0 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/proc/sys/fs/binfmt_misc 2024-06-02T23:10:54.903 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c061a6ed7cb25f3583cf016c06a1ca84f3bcda232f7968367d8ea735ddc8b690/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/7450634d266c5495c8e0f1e80cc60c4098d1aec9f0924b2d13f080c9cd748678/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/533dec7f1566cb7fa12591be4f5208cc68cb2e650a31b5dc9c79f444898a99af/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/d2e5afc77ff58be785fc5c2061ea648baa45b30b3dc114e84fbf6fa75be2b348/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/4bfc5105a78aa9f096693b1e639424cac177c9b1e36c7879bc23219d6fd862ba/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/c13c9940b6c2636f325f1a071898b484675e8dbef0fb5aa0c7c285daf0dcb067/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/34d46adee9527392e080494bef62ca29e9d22907499c229849b51a62a390dbfb/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/var/lib/containers/storage/overlay/f1f415d274e5122cc0164e69dba8c45e8e5a225f2e1349729b2161a417e570cb/merged 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run.smithi052.stdout:/home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:54.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:54.905 DEBUG:teuthology.orchestra.run.smithi052:> ls /sys/fs/fuse/connections 2024-06-02T23:10:54.961 INFO:teuthology.orchestra.run.smithi052.stdout:119 2024-06-02T23:10:54.961 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [119] 2024-06-02T23:10:54.961 DEBUG:teuthology.orchestra.run.smithi052:> sudo stdin-killer -- python3 -c ' 2024-06-02T23:10:54.961 DEBUG:teuthology.orchestra.run.smithi052:> import glob 2024-06-02T23:10:54.961 DEBUG:teuthology.orchestra.run.smithi052:> import re 2024-06-02T23:10:54.961 DEBUG:teuthology.orchestra.run.smithi052:> import os 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> import subprocess 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> def _find_admin_socket(client_name): 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> files = glob.glob(asok_path) 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> # Given a non-glob path, it better be there 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> if "*" not in asok_path: 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> assert(len(files) == 1) 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> return files[0] 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> for f in files: 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-06-02T23:10:54.962 DEBUG:teuthology.orchestra.run.smithi052:> if os.path.exists("/proc/{0}".format(pid)): 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> contents = proc_f.read() 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> if mountpoint in contents: 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> return f 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> print(_find_admin_socket("client.0")) 2024-06-02T23:10:54.963 DEBUG:teuthology.orchestra.run.smithi052:> ' 2024-06-02T23:10:55.091 INFO:teuthology.orchestra.run.smithi052.stdout:/var/run/ceph/ceph-client.0.87498.asok 2024-06-02T23:10:55.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-06-02T23:10:55.103 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87498.asok 2024-06-02T23:10:55.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:55.103 DEBUG:teuthology.orchestra.run.smithi052:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87498.asok status 2024-06-02T23:10:55.245 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:10:55.245 INFO:teuthology.orchestra.run.smithi052.stdout: "metadata": { 2024-06-02T23:10:55.245 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_sha1": "3d89accfe3201286f000e065a1faea2a88c7a6ba", 2024-06-02T23:10:55.245 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_version": "ceph version 19.0.0-2946-g3d89accf (3d89accfe3201286f000e065a1faea2a88c7a6ba) squid (rc)", 2024-06-02T23:10:55.245 INFO:teuthology.orchestra.run.smithi052.stdout: "entity_id": "0", 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": "smithi052", 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "pid": "87498", 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "root": "/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc" 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_count": 0, 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "dentry_pinned_count": 0, 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 15216, 2024-06-02T23:10:55.246 INFO:teuthology.orchestra.run.smithi052.stdout: "inst": { 2024-06-02T23:10:55.247 INFO:teuthology.orchestra.run.smithi052.stdout: "name": { 2024-06-02T23:10:55.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "client", 2024-06-02T23:10:55.247 INFO:teuthology.orchestra.run.smithi052.stdout: "num": 15216 2024-06-02T23:10:55.247 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:10:55.247 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-06-02T23:10:55.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3803340859 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": { 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "192.168.0.1:0", 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3803340859 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "inst_str": "client.15216 192.168.0.1:0/3803340859", 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "addr_str": "192.168.0.1:0/3803340859", 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "inode_count": 5, 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_epoch": 27, 2024-06-02T23:10:55.248 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch": 90, 2024-06-02T23:10:55.249 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_epoch_barrier": 0, 2024-06-02T23:10:55.249 INFO:teuthology.orchestra.run.smithi052.stdout: "blocklisted": false, 2024-06-02T23:10:55.249 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs" 2024-06-02T23:10:55.249 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:10:55.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:55.253 DEBUG:teuthology.orchestra.run.smithi052:> stat --file-system '--printf=%T 2024-06-02T23:10:55.253 DEBUG:teuthology.orchestra.run.smithi052:> ' -- /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:55.270 INFO:teuthology.orchestra.run.smithi052.stdout:fuseblk 2024-06-02T23:10:55.270 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:55.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:10:55.270 DEBUG:teuthology.orchestra.run.smithi052:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-06-02T23:10:55.339 INFO:teuthology.run_tasks:Running task exec... 2024-06-02T23:10:55.350 INFO:teuthology.task.exec:Executing custom commands... 2024-06-02T23:10:55.350 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:10:55.350 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-06-02T23:10:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:55 smithi174 ceph-mon[35994]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:55.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:55 smithi052 ceph-mon[32340]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:55.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2343766664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c68100900 msgr2=0x7f4c68100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:55.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2343766664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c68100900 0x7f4c68100d20 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4c58009900 tx=0x7f4c5802a410 comp rx=0 tx=0).stop 2024-06-02T23:10:55.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2343766664 shutdown_connections 2024-06-02T23:10:55.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2343766664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c681024b0 0x7f4c68108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2343766664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c68101af0 0x7f4c68101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2343766664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c68100900 0x7f4c68100d20 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.485+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2343766664 >> 172.21.15.52:0/2343766664 conn(0x7f4c680fbe90 msgr2=0x7f4c680fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.486+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2343766664 shutdown_connections 2024-06-02T23:10:55.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.486+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2343766664 wait complete. 2024-06-02T23:10:55.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.486+0000 7f4c6cafc640 1 Processor -- start 2024-06-02T23:10:55.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.486+0000 7f4c6cafc640 1 -- start start 2024-06-02T23:10:55.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.486+0000 7f4c6cafc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c68100900 0x7f4c68197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:55.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.486+0000 7f4c6cafc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c68101af0 0x7f4c68198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6cafc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 0x7f4c6819f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6cafc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6810d590 con 0x7f4c681024b0 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6cafc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6810d410 con 0x7f4c68100900 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6cafc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c6810d710 con 0x7f4c68101af0 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 0x7f4c6819f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 0x7f4c6819f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38612/0 (socket says 172.21.15.52:38612) 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 -- 172.21.15.52:0/2077944588 learned_addr learned my addr 172.21.15.52:0/2077944588 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6659c640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c68100900 0x7f4c68197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c65d9b640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c68101af0 0x7f4c68198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 -- 172.21.15.52:0/2077944588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c68101af0 msgr2=0x7f4c68198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:55.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c68101af0 0x7f4c68198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 -- 172.21.15.52:0/2077944588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c68100900 msgr2=0x7f4c68197c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c68100900 0x7f4c68197c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 -- 172.21.15.52:0/2077944588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c6819f830 con 0x7f4c681024b0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c65d9b640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c68101af0 0x7f4c68198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c66d9d640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 0x7f4c6819f190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000b970 tx=0x7f4c6000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c60019070 con 0x7f4c681024b0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c60014440 con 0x7f4c681024b0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c6819fb20 con 0x7f4c681024b0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c600075d0 con 0x7f4c681024b0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.487+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c681a0070 con 0x7f4c681024b0 2024-06-02T23:10:55.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.489+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c68107c20 con 0x7f4c681024b0 2024-06-02T23:10:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.490+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c6001c070 con 0x7f4c681024b0 2024-06-02T23:10:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.490+0000 7f4c577fe640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c500a2c30 0x7f4c500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.490+0000 7f4c6659c640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c500a2c30 0x7f4c500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.491+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4c600ce670 con 0x7f4c681024b0 2024-06-02T23:10:55.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.491+0000 7f4c6659c640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c500a2c30 0x7f4c500a50f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4c58004770 tx=0x7f4c58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:55.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.495+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4c6009afb0 con 0x7f4c681024b0 2024-06-02T23:10:55.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:55 smithi123 ceph-mon[35977]: pgmap v402: 97 pgs: 97 active+clean; 644 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-06-02T23:10:55.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:55.720+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f4c680625a0 con 0x7f4c681024b0 2024-06-02T23:10:56.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.100+0000 7f4c577fe640 1 -- 172.21.15.52:0/2077944588 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v28) ==== 117+0+0 (secure 0 0 0) 0x7f4c6009f2b0 con 0x7f4c681024b0 2024-06-02T23:10:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c500a2c30 msgr2=0x7f4c500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c500a2c30 0x7f4c500a50f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4c58004770 tx=0x7f4c58000bc0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 msgr2=0x7f4c6819f190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 0x7f4c6819f190 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4c6000b970 tx=0x7f4c6000be30 comp rx=0 tx=0).stop 2024-06-02T23:10:56.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 shutdown_connections 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c500a2c30 0x7f4c500a50f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c68101af0 0x7f4c68198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c681024b0 0x7f4c6819f190 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 --2- 172.21.15.52:0/2077944588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c68100900 0x7f4c68197c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 >> 172.21.15.52:0/2077944588 conn(0x7f4c680fbe90 msgr2=0x7f4c68104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 shutdown_connections 2024-06-02T23:10:56.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.103+0000 7f4c6cafc640 1 -- 172.21.15.52:0/2077944588 wait complete. 2024-06-02T23:10:56.114 INFO:teuthology.run_tasks:Running task check-counter... 2024-06-02T23:10:56.126 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-06-02T23:10:56.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-02T23:10:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 -- 172.21.15.52:0/3699711668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c1008f0 msgr2=0x7f694c100d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 --2- 172.21.15.52:0/3699711668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c1008f0 0x7f694c100d10 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6948009900 tx=0x7f694802a410 comp rx=0 tx=0).stop 2024-06-02T23:10:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 -- 172.21.15.52:0/3699711668 shutdown_connections 2024-06-02T23:10:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 --2- 172.21.15.52:0/3699711668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f694c1024a0 0x7f694c108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 --2- 172.21.15.52:0/3699711668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c101ae0 0x7f694c101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 --2- 172.21.15.52:0/3699711668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c1008f0 0x7f694c100d10 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 -- 172.21.15.52:0/3699711668 >> 172.21.15.52:0/3699711668 conn(0x7f694c0fbe80 msgr2=0x7f694c0fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 -- 172.21.15.52:0/3699711668 shutdown_connections 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.273+0000 7f6954672640 1 -- 172.21.15.52:0/3699711668 wait complete. 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 Processor -- start 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 -- start start 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f694c1008f0 0x7f694c197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 0x7f694c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c1024a0 0x7f694c19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10d500 con 0x7f694c101ae0 2024-06-02T23:10:56.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10d380 con 0x7f694c1024a0 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6954672640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f694c10d680 con 0x7f694c1008f0 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 0x7f694c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 0x7f694c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38628/0 (socket says 172.21.15.52:38628) 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 -- 172.21.15.52:0/1549385797 learned_addr learned my addr 172.21.15.52:0/1549385797 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6952c0f640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c1024a0 0x7f694c19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 -- 172.21.15.52:0/1549385797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f694c1008f0 msgr2=0x7f694c197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f695240e640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f694c1008f0 0x7f694c197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f694c1008f0 0x7f694c197c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 -- 172.21.15.52:0/1549385797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c1024a0 msgr2=0x7f694c19f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c1024a0 0x7f694c19f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.274+0000 7f6951c0d640 1 -- 172.21.15.52:0/1549385797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f694c19f880 con 0x7f694c101ae0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.275+0000 7f6951c0d640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 0x7f694c1981b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d950 tx=0x7f693c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.275+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c017070 con 0x7f694c101ae0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.275+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f693c009c30 con 0x7f694c101ae0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.275+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f693c00f620 con 0x7f694c101ae0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.275+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f694c19fb70 con 0x7f694c101ae0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.275+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f694c1a00c0 con 0x7f694c101ae0 2024-06-02T23:10:56.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.276+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f694c107ce0 con 0x7f694c101ae0 2024-06-02T23:10:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.277+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f693c0109a0 con 0x7f694c101ae0 2024-06-02T23:10:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.278+0000 7f69437fe640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69380a2b90 0x7f69380a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.278+0000 7f695240e640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69380a2b90 0x7f69380a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.278+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f693c0ced10 con 0x7f694c101ae0 2024-06-02T23:10:56.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.278+0000 7f695240e640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69380a2b90 0x7f69380a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6948006010 tx=0x7f6948000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:56.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.281+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f693c009de0 con 0x7f694c101ae0 2024-06-02T23:10:56.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:56 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2077944588' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T23:10:56.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:56 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2077944588' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T23:10:56.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.466+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f694c02d400 con 0x7f694c101ae0 2024-06-02T23:10:56.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.468+0000 7f69437fe640 1 -- 172.21.15.52:0/1549385797 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7f693c09bc40 con 0x7f694c101ae0 2024-06-02T23:10:56.469 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:56.469 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:10:56.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69380a2b90 msgr2=0x7f69380a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69380a2b90 0x7f69380a5050 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6948006010 tx=0x7f6948000bc0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 msgr2=0x7f694c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 0x7f694c1981b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f693c00d950 tx=0x7f693c00de10 comp rx=0 tx=0).stop 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 shutdown_connections 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69380a2b90 0x7f69380a5050 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f694c1024a0 0x7f694c19f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f694c101ae0 0x7f694c1981b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 --2- 172.21.15.52:0/1549385797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f694c1008f0 0x7f694c197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 >> 172.21.15.52:0/1549385797 conn(0x7f694c0fbe80 msgr2=0x7f694c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 shutdown_connections 2024-06-02T23:10:56.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.470+0000 7f6954672640 1 -- 172.21.15.52:0/1549385797 wait complete. 2024-06-02T23:10:56.482 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-02T23:02:34.195730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-02T23:07:26.794014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-02T23:07:30.138193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-02T23:10:56.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-02T23:10:56.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:56 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2077944588' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1493229737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 msgr2=0x7f0b48104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1493229737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48104870 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0b38009900 tx=0x7f0b3802a410 comp rx=0 tx=0).stop 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1493229737 shutdown_connections 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1493229737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b481076e0 0x7f0b48109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1493229737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b48104db0 0x7f0b481071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1493229737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48104870 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1493229737 >> 172.21.15.52:0/1493229737 conn(0x7f0b480fbe90 msgr2=0x7f0b480fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1493229737 shutdown_connections 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.589+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1493229737 wait complete. 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 Processor -- start 2024-06-02T23:10:56.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 -- start start 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b48104db0 0x7f0b481981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b481076e0 0x7f0b4819f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4810d530 con 0x7f0b48102480 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4810d3b0 con 0x7f0b48104db0 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b4ddee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b4810d6b0 con 0x7f0b481076e0 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b477fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b477fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38648/0 (socket says 172.21.15.52:38648) 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.590+0000 7f0b477fe640 1 -- 172.21.15.52:0/1533776941 learned_addr learned my addr 172.21.15.52:0/1533776941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b46ffd640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b48104db0 0x7f0b481981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b47fff640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b481076e0 0x7f0b4819f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b477fe640 1 -- 172.21.15.52:0/1533776941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b481076e0 msgr2=0x7f0b4819f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b477fe640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b481076e0 0x7f0b4819f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b477fe640 1 -- 172.21.15.52:0/1533776941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b48104db0 msgr2=0x7f0b481981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b477fe640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b48104db0 0x7f0b481981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b477fe640 1 -- 172.21.15.52:0/1533776941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b4819f8e0 con 0x7f0b48102480 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b477fe640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48197ca0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0b38004770 tx=0x7f0b380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b3803f030 con 0x7f0b48102480 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b4819fb70 con 0x7f0b48102480 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b38003cb0 con 0x7f0b48102480 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b38035070 con 0x7f0b48102480 2024-06-02T23:10:56.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.591+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b481a0060 con 0x7f0b48102480 2024-06-02T23:10:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.592+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b480fd980 con 0x7f0b48102480 2024-06-02T23:10:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.594+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b38042030 con 0x7f0b48102480 2024-06-02T23:10:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.594+0000 7f0b44ff9640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b300a2b90 0x7f0b300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.594+0000 7f0b46ffd640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b300a2b90 0x7f0b300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.594+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0b380efb00 con 0x7f0b48102480 2024-06-02T23:10:56.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.595+0000 7f0b46ffd640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b300a2b90 0x7f0b300a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b481991a0 tx=0x7f0b3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:56.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.598+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0b380bc440 con 0x7f0b48102480 2024-06-02T23:10:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.784+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0b4802d470 con 0x7f0b48102480 2024-06-02T23:10:56.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.784+0000 7f0b44ff9640 1 -- 172.21.15.52:0/1533776941 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7f0b380c0740 con 0x7f0b48102480 2024-06-02T23:10:56.785 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b300a2b90 msgr2=0x7f0b300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b300a2b90 0x7f0b300a5050 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0b481991a0 tx=0x7f0b3c00b390 comp rx=0 tx=0).stop 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 msgr2=0x7f0b48197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48197ca0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0b38004770 tx=0x7f0b380042f0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 shutdown_connections 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b300a2b90 0x7f0b300a5050 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b481076e0 0x7f0b4819f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b48104db0 0x7f0b481981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 --2- 172.21.15.52:0/1533776941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b48102480 0x7f0b48197ca0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 >> 172.21.15.52:0/1533776941 conn(0x7f0b480fbe90 msgr2=0x7f0b48102ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 shutdown_connections 2024-06-02T23:10:56.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.786+0000 7f0b4ddee640 1 -- 172.21.15.52:0/1533776941 wait complete. 2024-06-02T23:10:56.798 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-02T23:10:56.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 -- 172.21.15.52:0/2053965664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa4101a00 msgr2=0x7f2fa4101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 --2- 172.21.15.52:0/2053965664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa4101a00 0x7f2fa4101e20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009900 tx=0x7f2f9402a410 comp rx=0 tx=0).stop 2024-06-02T23:10:56.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 -- 172.21.15.52:0/2053965664 shutdown_connections 2024-06-02T23:10:56.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 --2- 172.21.15.52:0/2053965664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa41035b0 0x7f2fa4109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 --2- 172.21.15.52:0/2053965664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4102bf0 0x7f2fa4103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 --2- 172.21.15.52:0/2053965664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa4101a00 0x7f2fa4101e20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 -- 172.21.15.52:0/2053965664 >> 172.21.15.52:0/2053965664 conn(0x7f2fa40fcf70 msgr2=0x7f2fa40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 -- 172.21.15.52:0/2053965664 shutdown_connections 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 -- 172.21.15.52:0/2053965664 wait complete. 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 Processor -- start 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.905+0000 7f2fab14f640 1 -- start start 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4101a00 0x7f2fa4197c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa4102bf0 0x7f2fa41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 0x7f2fa419f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa410d530 con 0x7f2fa41035b0 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa410d3b0 con 0x7f2fa4101a00 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa410d6b0 con 0x7f2fa4102bf0 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 0x7f2fa419f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 0x7f2fa419f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38676/0 (socket says 172.21.15.52:38676) 2024-06-02T23:10:56.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 -- 172.21.15.52:0/3749035839 learned_addr learned my addr 172.21.15.52:0/3749035839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa3fff640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa4102bf0 0x7f2fa41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa8eeb640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4101a00 0x7f2fa4197c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 -- 172.21.15.52:0/3749035839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa4102bf0 msgr2=0x7f2fa41981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa4102bf0 0x7f2fa41981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 -- 172.21.15.52:0/3749035839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4101a00 msgr2=0x7f2fa4197c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4101a00 0x7f2fa4197c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 -- 172.21.15.52:0/3749035839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa419f880 con 0x7f2fa41035b0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa8eeb640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4101a00 0x7f2fa4197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa96ec640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 0x7f2fa419f1e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00eaa0 tx=0x7f2f9c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c00cdb0 con 0x7f2fa41035b0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f9c005020 con 0x7f2fa41035b0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c0116d0 con 0x7f2fa41035b0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.906+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa419fb70 con 0x7f2fa41035b0 2024-06-02T23:10:56.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.907+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa41a00c0 con 0x7f2fa41035b0 2024-06-02T23:10:56.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.908+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa4108e40 con 0x7f2fa41035b0 2024-06-02T23:10:56.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.909+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2f9c0051d0 con 0x7f2fa41035b0 2024-06-02T23:10:56.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.910+0000 7f2fa1ffb640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f8c0a2b90 0x7f2f8c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:56.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.910+0000 7f2fa8eeb640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f8c0a2b90 0x7f2f8c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:56.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.910+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2f9c0cf260 con 0x7f2fa41035b0 2024-06-02T23:10:56.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.910+0000 7f2fa8eeb640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f8c0a2b90 0x7f2f8c0a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f94006010 tx=0x7f2f94032c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:56.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:56.915+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2f9c016030 con 0x7f2fa41035b0 2024-06-02T23:10:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.098+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2fa40625a0 con 0x7f2fa41035b0 2024-06-02T23:10:57.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.099+0000 7f2fa1ffb640 1 -- 172.21.15.52:0/3749035839 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f2f9c09c1f0 con 0x7f2fa41035b0 2024-06-02T23:10:57.099 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-06-02T23:10:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f8c0a2b90 msgr2=0x7f2f8c0a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f8c0a2b90 0x7f2f8c0a5050 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2f94006010 tx=0x7f2f94032c40 comp rx=0 tx=0).stop 2024-06-02T23:10:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 msgr2=0x7f2fa419f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 0x7f2fa419f1e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c00eaa0 tx=0x7f2f9c00ef60 comp rx=0 tx=0).stop 2024-06-02T23:10:57.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 shutdown_connections 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2f8c0a2b90 0x7f2f8c0a5050 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2fa4102bf0 0x7f2fa41981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2fa41035b0 0x7f2fa419f1e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 --2- 172.21.15.52:0/3749035839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa4101a00 0x7f2fa4197c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 >> 172.21.15.52:0/3749035839 conn(0x7f2fa40fcf70 msgr2=0x7f2fa4106010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 shutdown_connections 2024-06-02T23:10:57.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.101+0000 7f2fab14f640 1 -- 172.21.15.52:0/3749035839 wait complete. 2024-06-02T23:10:57.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-02T23:10:57.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 -- 172.21.15.52:0/5560318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c100900 msgr2=0x7f245c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 --2- 172.21.15.52:0/5560318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c100900 0x7f245c100d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f244c009900 tx=0x7f244c02a410 comp rx=0 tx=0).stop 2024-06-02T23:10:57.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 -- 172.21.15.52:0/5560318 shutdown_connections 2024-06-02T23:10:57.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 --2- 172.21.15.52:0/5560318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c101af0 0x7f245c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 --2- 172.21.15.52:0/5560318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c1024b0 0x7f245c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 --2- 172.21.15.52:0/5560318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c100900 0x7f245c100d20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 -- 172.21.15.52:0/5560318 >> 172.21.15.52:0/5560318 conn(0x7f245c0fbe90 msgr2=0x7f245c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 -- 172.21.15.52:0/5560318 shutdown_connections 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 -- 172.21.15.52:0/5560318 wait complete. 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 Processor -- start 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.222+0000 7f24625b7640 1 -- start start 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c100900 0x7f245c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 0x7f245c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c1024b0 0x7f245c19e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10c420 con 0x7f245c101af0 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10c2a0 con 0x7f245c100900 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f245c10c5a0 con 0x7f245c1024b0 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 0x7f245c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 0x7f245c1970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38692/0 (socket says 172.21.15.52:38692) 2024-06-02T23:10:57.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 -- 172.21.15.52:0/2305214057 learned_addr learned my addr 172.21.15.52:0/2305214057 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245bfff640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c100900 0x7f245c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 -- 172.21.15.52:0/2305214057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c1024b0 msgr2=0x7f245c19e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c1024b0 0x7f245c19e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 -- 172.21.15.52:0/2305214057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c100900 msgr2=0x7f245c196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c100900 0x7f245c196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 -- 172.21.15.52:0/2305214057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c19e780 con 0x7f245c101af0 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245bfff640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c100900 0x7f245c196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f245b7fe640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 0x7f245c1970a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f245000e9f0 tx=0x7f245000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:57.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245000cd80 con 0x7f245c101af0 2024-06-02T23:10:57.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24500054d0 con 0x7f245c101af0 2024-06-02T23:10:57.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c19ea70 con 0x7f245c101af0 2024-06-02T23:10:57.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.223+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450010490 con 0x7f245c101af0 2024-06-02T23:10:57.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.224+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c19ef70 con 0x7f245c101af0 2024-06-02T23:10:57.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.224+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f245c107d40 con 0x7f245c101af0 2024-06-02T23:10:57.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.226+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2450005000 con 0x7f245c101af0 2024-06-02T23:10:57.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.226+0000 7f24597fa640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2b90 0x7f24440a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:57.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.227+0000 7f245bfff640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2b90 0x7f24440a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:57.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.227+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f24500cf740 con 0x7f245c101af0 2024-06-02T23:10:57.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.227+0000 7f245bfff640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2b90 0x7f24440a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f244c006010 tx=0x7f244c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:57.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.230+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2450053850 con 0x7f245c101af0 2024-06-02T23:10:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:57 smithi174 ceph-mon[35994]: pgmap v403: 97 pgs: 97 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-02T23:10:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:57 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2077944588' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-02T23:10:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:57 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:10:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:57 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1549385797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:10:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:57 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1533776941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:10:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:57 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3749035839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T23:10:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:57 smithi052 ceph-mon[32340]: pgmap v403: 97 pgs: 97 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-02T23:10:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:57 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2077944588' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-02T23:10:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:57 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:10:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:57 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1549385797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:10:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:57 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1533776941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:10:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:57 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3749035839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T23:10:57.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.418+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f245c02d530 con 0x7f245c101af0 2024-06-02T23:10:57.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.418+0000 7f24597fa640 1 -- 172.21.15.52:0/2305214057 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f24500052f0 con 0x7f245c101af0 2024-06-02T23:10:57.419 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-06-02T23:10:57.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2b90 msgr2=0x7f24440a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2b90 0x7f24440a5050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f244c006010 tx=0x7f244c000bc0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 msgr2=0x7f245c1970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 0x7f245c1970a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f245000e9f0 tx=0x7f245000edf0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 shutdown_connections 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2b90 0x7f24440a5050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c1024b0 0x7f245c19e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c101af0 0x7f245c1970a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 --2- 172.21.15.52:0/2305214057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c100900 0x7f245c196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 >> 172.21.15.52:0/2305214057 conn(0x7f245c0fbe90 msgr2=0x7f245c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 shutdown_connections 2024-06-02T23:10:57.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:57.420+0000 7f24625b7640 1 -- 172.21.15.52:0/2305214057 wait complete. 2024-06-02T23:10:57.431 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-06-02T23:10:57.431 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:10:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:57 smithi123 ceph-mon[35977]: pgmap v403: 97 pgs: 97 active+clean; 646 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 10 op/s 2024-06-02T23:10:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:57 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2077944588' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-06-02T23:10:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:57 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:10:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:57 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1549385797' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:10:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:57 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1533776941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:10:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:57 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3749035839' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T23:10:57.683 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:10:58.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:58 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2305214057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T23:10:58.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:58 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2305214057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T23:10:58.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:58 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2305214057' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T23:10:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:10:59 smithi174 ceph-mon[35994]: pgmap v404: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:10:59.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:10:59 smithi052 ceph-mon[32340]: pgmap v404: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:10:59.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 -- 172.21.15.52:0/2625465603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 msgr2=0x7f2b8c100c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:59.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2625465603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c100c50 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2b80009990 tx=0x7f2b80009ca0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 -- 172.21.15.52:0/2625465603 shutdown_connections 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2625465603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b8c10ec80 0x7f2b8c111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2625465603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c100c50 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2625465603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b8c101190 0x7f2b8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 -- 172.21.15.52:0/2625465603 >> 172.21.15.52:0/2625465603 conn(0x7f2b8c077590 msgr2=0x7f2b8c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 -- 172.21.15.52:0/2625465603 shutdown_connections 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 -- 172.21.15.52:0/2625465603 wait complete. 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.444+0000 7f2b9349d700 1 Processor -- start 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 -- start start 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b8c101190 0x7f2b8c19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:59.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b8c10ec80 0x7f2b8c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c114860 con 0x7f2b8c100870 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1146e0 con 0x7f2b8c10ec80 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b9349d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b8c1149e0 con 0x7f2b8c101190 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38710/0 (socket says 172.21.15.52:38710) 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 -- 172.21.15.52:0/2911089224 learned_addr learned my addr 172.21.15.52:0/2911089224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b90a38700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b8c101190 0x7f2b8c19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91a3a700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b8c10ec80 0x7f2b8c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 -- 172.21.15.52:0/2911089224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b8c101190 msgr2=0x7f2b8c19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b8c101190 0x7f2b8c19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 -- 172.21.15.52:0/2911089224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b8c10ec80 msgr2=0x7f2b8c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b8c10ec80 0x7f2b8c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.445+0000 7f2b91239700 1 -- 172.21.15.52:0/2911089224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b8c19fd40 con 0x7f2b8c100870 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.446+0000 7f2b91239700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c19ad30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b80003ab0 tx=0x7f2b80003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.446+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80027070 con 0x7f2b8c100870 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.446+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b8000bdc0 con 0x7f2b8c100870 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.446+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b8c19ffd0 con 0x7f2b8c100870 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.446+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b80022d40 con 0x7f2b8c100870 2024-06-02T23:10:59.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.446+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b8c1a78b0 con 0x7f2b8c100870 2024-06-02T23:10:59.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.447+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b8c1019a0 con 0x7f2b8c100870 2024-06-02T23:10:59.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.449+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b8002a070 con 0x7f2b8c100870 2024-06-02T23:10:59.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.449+0000 7f2b7e7fc700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b780a2c20 0x7f2b780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:10:59.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.450+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b800d0510 con 0x7f2b8c100870 2024-06-02T23:10:59.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.450+0000 7f2b90a38700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b780a2c20 0x7f2b780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:10:59.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.450+0000 7f2b90a38700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b780a2c20 0x7f2b780a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c103840 tx=0x7f2b88007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:10:59.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.453+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2b80031020 con 0x7f2b8c100870 2024-06-02T23:10:59.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:10:59 smithi123 ceph-mon[35977]: pgmap v404: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:10:59.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.703+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b8c02dca0 con 0x7f2b8c100870 2024-06-02T23:10:59.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.704+0000 7f2b7e7fc700 1 -- 172.21.15.52:0/2911089224 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f2b80013070 con 0x7f2b8c100870 2024-06-02T23:10:59.705 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:10:59.705 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:10:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b780a2c20 msgr2=0x7f2b780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b780a2c20 0x7f2b780a50e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c103840 tx=0x7f2b88007400 comp rx=0 tx=0).stop 2024-06-02T23:10:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 msgr2=0x7f2b8c19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:10:59.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c19ad30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f2b80003ab0 tx=0x7f2b80003ae0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 shutdown_connections 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b780a2c20 0x7f2b780a50e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b8c10ec80 0x7f2b8c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b8c100870 0x7f2b8c19ad30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 --2- 172.21.15.52:0/2911089224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b8c101190 0x7f2b8c19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 >> 172.21.15.52:0/2911089224 conn(0x7f2b8c077590 msgr2=0x7f2b8c110050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 shutdown_connections 2024-06-02T23:10:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:10:59.707+0000 7f2b9349d700 1 -- 172.21.15.52:0/2911089224 wait complete. 2024-06-02T23:10:59.709 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:11:00.338 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:00 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2911089224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:00.368 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-06-02T23:11:00.368 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-06-02T23:11:00.368 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-02T23:11:00.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:00 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2911089224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:00.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 -- 172.21.15.52:0/3272873560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74441008e0 msgr2=0x7f7444100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 --2- 172.21.15.52:0/3272873560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74441008e0 0x7f7444100d00 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7434009900 tx=0x7f743402a410 comp rx=0 tx=0).stop 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 -- 172.21.15.52:0/3272873560 shutdown_connections 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 --2- 172.21.15.52:0/3272873560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7444102490 0x7f7444108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 --2- 172.21.15.52:0/3272873560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 0x7f7444101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 --2- 172.21.15.52:0/3272873560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f74441008e0 0x7f7444100d00 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 -- 172.21.15.52:0/3272873560 >> 172.21.15.52:0/3272873560 conn(0x7f74440fbe50 msgr2=0x7f74440fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 -- 172.21.15.52:0/3272873560 shutdown_connections 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.478+0000 7f7449212640 1 -- 172.21.15.52:0/3272873560 wait complete. 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 Processor -- start 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 -- start start 2024-06-02T23:11:00.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74441008e0 0x7f7444197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 0x7f74441981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 0x7f744419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f744410c410 con 0x7f7444102490 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f744410c290 con 0x7f7444101ad0 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7449212640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f744410c590 con 0x7f74441008e0 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f744359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 0x7f744419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f744359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 0x7f744419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38736/0 (socket says 172.21.15.52:38736) 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f744359e640 1 -- 172.21.15.52:0/1016458948 learned_addr learned my addr 172.21.15.52:0/1016458948 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f7442d9d640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74441008e0 0x7f7444197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.479+0000 7f743bfff640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 0x7f74441981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f744359e640 1 -- 172.21.15.52:0/1016458948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74441008e0 msgr2=0x7f7444197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f744359e640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74441008e0 0x7f7444197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f744359e640 1 -- 172.21.15.52:0/1016458948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 msgr2=0x7f74441981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f744359e640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 0x7f74441981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f744359e640 1 -- 172.21.15.52:0/1016458948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f744419f890 con 0x7f7444102490 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f743bfff640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 0x7f74441981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f744359e640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 0x7f744419f1f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f743c00eaa0 tx=0x7f743c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c00cdb0 con 0x7f7444102490 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f743c005020 con 0x7f7444102490 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f744419fb80 con 0x7f7444102490 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743c0116d0 con 0x7f7444102490 2024-06-02T23:11:00.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.480+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74441a00a0 con 0x7f7444102490 2024-06-02T23:11:00.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.481+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74441a0270 con 0x7f7444102490 2024-06-02T23:11:00.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.482+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f743c0051d0 con 0x7f7444102490 2024-06-02T23:11:00.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.483+0000 7f7440d99640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f74300a2b90 0x7f74300a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.483+0000 7f7442d9d640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f74300a2b90 0x7f74300a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.483+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f743c0cf680 con 0x7f7444102490 2024-06-02T23:11:00.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.483+0000 7f7442d9d640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f74300a2b90 0x7f74300a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7434004770 tx=0x7f7434000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:00.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.486+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f743c016030 con 0x7f7444102490 2024-06-02T23:11:00.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:00 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2911089224' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:00.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.667+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f74441a0440 con 0x7f74300a2b90 2024-06-02T23:11:00.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.675+0000 7f7440d99640 1 -- 172.21.15.52:0/1016458948 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259073 (secure 0 0 0) 0x7f74441a0440 con 0x7f74300a2b90 2024-06-02T23:11:00.678 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:00.680 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:11:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f74300a2b90 msgr2=0x7f74300a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f74300a2b90 0x7f74300a5050 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7434004770 tx=0x7f7434000bc0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 msgr2=0x7f744419f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 0x7f744419f1f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f743c00eaa0 tx=0x7f743c00ef60 comp rx=0 tx=0).stop 2024-06-02T23:11:00.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 shutdown_connections 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f74300a2b90 0x7f74300a5050 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7444102490 0x7f744419f1f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7444101ad0 0x7f74441981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 --2- 172.21.15.52:0/1016458948 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74441008e0 0x7f7444197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 >> 172.21.15.52:0/1016458948 conn(0x7f74440fbe50 msgr2=0x7f74440fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 shutdown_connections 2024-06-02T23:11:00.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.678+0000 7f7449212640 1 -- 172.21.15.52:0/1016458948 wait complete. 2024-06-02T23:11:00.689 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-06-02T23:10:59.812799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2027,"num_read_kb":1143,"num_write":456,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369640,"kb_used_data":12168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572184,"statfs":{"total":1151940427776,"available":1151561916416,"internally_reserved":0,"allocated":12460032,"data_stored":7749945,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6774,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002845"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802175+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:09:39.802175+0000","last_peered":"2024-06-02T23:09:39.802175+0000","last_clean":"2024-06-02T23:09:39.802175+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:09:39.802175+0000","last_undegraded":"2024-06-02T23:09:39.802175+0000","last_fullsized":"2024-06-02T23:09:39.802175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788231+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:09:39.788231+0000","last_peered":"2024-06-02T23:09:39.788231+0000","last_clean":"2024-06-02T23:09:39.788231+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:09:39.788231+0000","last_undegraded":"2024-06-02T23:09:39.788231+0000","last_fullsized":"2024-06-02T23:09:39.788231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794300+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:09:39.794300+0000","last_peered":"2024-06-02T23:09:39.794300+0000","last_clean":"2024-06-02T23:09:39.794300+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:09:39.794300+0000","last_undegraded":"2024-06-02T23:09:39.794300+0000","last_fullsized":"2024-06-02T23:09:39.794300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788417+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:09:39.788417+0000","last_peered":"2024-06-02T23:09:39.788417+0000","last_clean":"2024-06-02T23:09:39.788417+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:09:39.788417+0000","last_undegraded":"2024-06-02T23:09:39.788417+0000","last_fullsized":"2024-06-02T23:09:39.788417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.777964+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:09:39.777964+0000","last_peered":"2024-06-02T23:09:39.777964+0000","last_clean":"2024-06-02T23:09:39.777964+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:09:39.777964+0000","last_undegraded":"2024-06-02T23:09:39.777964+0000","last_fullsized":"2024-06-02T23:09:39.777964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782448+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:09:39.782448+0000","last_peered":"2024-06-02T23:09:39.782448+0000","last_clean":"2024-06-02T23:09:39.782448+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:09:39.782448+0000","last_undegraded":"2024-06-02T23:09:39.782448+0000","last_fullsized":"2024-06-02T23:09:39.782448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786918+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:09:39.786918+0000","last_peered":"2024-06-02T23:09:39.786918+0000","last_clean":"2024-06-02T23:09:39.786918+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:09:39.786918+0000","last_undegraded":"2024-06-02T23:09:39.786918+0000","last_fullsized":"2024-06-02T23:09:39.786918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786735+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:09:39.786735+0000","last_peered":"2024-06-02T23:09:39.786735+0000","last_clean":"2024-06-02T23:09:39.786735+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:09:39.786735+0000","last_undegraded":"2024-06-02T23:09:39.786735+0000","last_fullsized":"2024-06-02T23:09:39.786735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801527+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:09:39.801527+0000","last_peered":"2024-06-02T23:09:39.801527+0000","last_clean":"2024-06-02T23:09:39.801527+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:09:39.801527+0000","last_undegraded":"2024-06-02T23:09:39.801527+0000","last_fullsized":"2024-06-02T23:09:39.801527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789699+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:09:39.789699+0000","last_peered":"2024-06-02T23:09:39.789699+0000","last_clean":"2024-06-02T23:09:39.789699+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:09:39.789699+0000","last_undegraded":"2024-06-02T23:09:39.789699+0000","last_fullsized":"2024-06-02T23:09:39.789699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797457+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:09:39.797457+0000","last_peered":"2024-06-02T23:09:39.797457+0000","last_clean":"2024-06-02T23:09:39.797457+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:09:39.797457+0000","last_undegraded":"2024-06-02T23:09:39.797457+0000","last_fullsized":"2024-06-02T23:09:39.797457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.787757+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:09:39.787757+0000","last_peered":"2024-06-02T23:09:39.787757+0000","last_clean":"2024-06-02T23:09:39.787757+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:09:39.787757+0000","last_undegraded":"2024-06-02T23:09:39.787757+0000","last_fullsized":"2024-06-02T23:09:39.787757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786117+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:09:39.786117+0000","last_peered":"2024-06-02T23:09:39.786117+0000","last_clean":"2024-06-02T23:09:39.786117+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:09:39.786117+0000","last_undegraded":"2024-06-02T23:09:39.786117+0000","last_fullsized":"2024-06-02T23:09:39.786117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788799+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:09:39.788799+0000","last_peered":"2024-06-02T23:09:39.788799+0000","last_clean":"2024-06-02T23:09:39.788799+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:09:39.788799+0000","last_undegraded":"2024-06-02T23:09:39.788799+0000","last_fullsized":"2024-06-02T23:09:39.788799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791625+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:09:39.791625+0000","last_peered":"2024-06-02T23:09:39.791625+0000","last_clean":"2024-06-02T23:09:39.791625+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:09:39.791625+0000","last_undegraded":"2024-06-02T23:09:39.791625+0000","last_fullsized":"2024-06-02T23:09:39.791625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800433+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:09:39.800433+0000","last_peered":"2024-06-02T23:09:39.800433+0000","last_clean":"2024-06-02T23:09:39.800433+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:09:39.800433+0000","last_undegraded":"2024-06-02T23:09:39.800433+0000","last_fullsized":"2024-06-02T23:09:39.800433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802531+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:09:39.802531+0000","last_peered":"2024-06-02T23:09:39.802531+0000","last_clean":"2024-06-02T23:09:39.802531+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:09:39.802531+0000","last_undegraded":"2024-06-02T23:09:39.802531+0000","last_fullsized":"2024-06-02T23:09:39.802531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.803680+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:09:39.803680+0000","last_peered":"2024-06-02T23:09:39.803680+0000","last_clean":"2024-06-02T23:09:39.803680+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:09:39.803680+0000","last_undegraded":"2024-06-02T23:09:39.803680+0000","last_fullsized":"2024-06-02T23:09:39.803680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790081+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:09:39.790081+0000","last_peered":"2024-06-02T23:09:39.790081+0000","last_clean":"2024-06-02T23:09:39.790081+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:09:39.790081+0000","last_undegraded":"2024-06-02T23:09:39.790081+0000","last_fullsized":"2024-06-02T23:09:39.790081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":227,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:55.121314+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:10:55.121314+0000","last_peered":"2024-06-02T23:10:55.121314+0000","last_clean":"2024-06-02T23:10:55.121314+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:10:55.121314+0000","last_undegraded":"2024-06-02T23:10:55.121314+0000","last_fullsized":"2024-06-02T23:10:55.121314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788800+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:09:39.788800+0000","last_peered":"2024-06-02T23:09:39.788800+0000","last_clean":"2024-06-02T23:09:39.788800+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:09:39.788800+0000","last_undegraded":"2024-06-02T23:09:39.788800+0000","last_fullsized":"2024-06-02T23:09:39.788800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794320+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:09:39.794320+0000","last_peered":"2024-06-02T23:09:39.794320+0000","last_clean":"2024-06-02T23:09:39.794320+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:09:39.794320+0000","last_undegraded":"2024-06-02T23:09:39.794320+0000","last_fullsized":"2024-06-02T23:09:39.794320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215945+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:14.215945+0000","last_peered":"2024-06-02T23:08:14.215945+0000","last_clean":"2024-06-02T23:08:14.215945+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:14.215945+0000","last_undegraded":"2024-06-02T23:08:14.215945+0000","last_fullsized":"2024-06-02T23:08:14.215945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802474+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:09:39.802474+0000","last_peered":"2024-06-02T23:09:39.802474+0000","last_clean":"2024-06-02T23:09:39.802474+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:09:39.802474+0000","last_undegraded":"2024-06-02T23:09:39.802474+0000","last_fullsized":"2024-06-02T23:09:39.802474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335772+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335772+0000","last_peered":"2024-06-02T23:08:11.335772+0000","last_clean":"2024-06-02T23:08:11.335772+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335772+0000","last_undegraded":"2024-06-02T23:08:11.335772+0000","last_fullsized":"2024-06-02T23:08:11.335772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800976+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:09:39.800976+0000","last_peered":"2024-06-02T23:09:39.800976+0000","last_clean":"2024-06-02T23:09:39.800976+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:09:39.800976+0000","last_undegraded":"2024-06-02T23:09:39.800976+0000","last_fullsized":"2024-06-02T23:09:39.800976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":235,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:57.687377+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:10:57.687377+0000","last_peered":"2024-06-02T23:10:57.687377+0000","last_clean":"2024-06-02T23:10:57.687377+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:10:57.687377+0000","last_undegraded":"2024-06-02T23:10:57.687377+0000","last_fullsized":"2024-06-02T23:10:57.687377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771608+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:09:39.771608+0000","last_peered":"2024-06-02T23:09:39.771608+0000","last_clean":"2024-06-02T23:09:39.771608+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:09:39.771608+0000","last_undegraded":"2024-06-02T23:09:39.771608+0000","last_fullsized":"2024-06-02T23:09:39.771608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449603+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449603+0000","last_peered":"2024-06-02T23:08:11.449603+0000","last_clean":"2024-06-02T23:08:11.449603+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449603+0000","last_undegraded":"2024-06-02T23:08:11.449603+0000","last_fullsized":"2024-06-02T23:08:11.449603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795377+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:09:39.795377+0000","last_peered":"2024-06-02T23:09:39.795377+0000","last_clean":"2024-06-02T23:09:39.795377+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:09:39.795377+0000","last_undegraded":"2024-06-02T23:09:39.795377+0000","last_fullsized":"2024-06-02T23:09:39.795377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332946+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332946+0000","last_peered":"2024-06-02T23:08:11.332946+0000","last_clean":"2024-06-02T23:08:11.332946+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332946+0000","last_undegraded":"2024-06-02T23:08:11.332946+0000","last_fullsized":"2024-06-02T23:08:11.332946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785226+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:09:39.785226+0000","last_peered":"2024-06-02T23:09:39.785226+0000","last_clean":"2024-06-02T23:09:39.785226+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:09:39.785226+0000","last_undegraded":"2024-06-02T23:09:39.785226+0000","last_fullsized":"2024-06-02T23:09:39.785226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329286+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329286+0000","last_peered":"2024-06-02T23:08:11.329286+0000","last_clean":"2024-06-02T23:08:11.329286+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329286+0000","last_undegraded":"2024-06-02T23:08:11.329286+0000","last_fullsized":"2024-06-02T23:08:11.329286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.779022+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:09:39.779022+0000","last_peered":"2024-06-02T23:09:39.779022+0000","last_clean":"2024-06-02T23:09:39.779022+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:09:39.779022+0000","last_undegraded":"2024-06-02T23:09:39.779022+0000","last_fullsized":"2024-06-02T23:09:39.779022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":405,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:56.684971+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:10:56.684971+0000","last_peered":"2024-06-02T23:10:56.684971+0000","last_clean":"2024-06-02T23:10:56.684971+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:10:56.684971+0000","last_undegraded":"2024-06-02T23:10:56.684971+0000","last_fullsized":"2024-06-02T23:10:56.684971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":371,"num_read_kb":371,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802707+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:09:39.802707+0000","last_peered":"2024-06-02T23:09:39.802707+0000","last_clean":"2024-06-02T23:09:39.802707+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:09:39.802707+0000","last_undegraded":"2024-06-02T23:09:39.802707+0000","last_fullsized":"2024-06-02T23:09:39.802707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791331+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:09:39.791331+0000","last_peered":"2024-06-02T23:09:39.791331+0000","last_clean":"2024-06-02T23:09:39.791331+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:09:39.791331+0000","last_undegraded":"2024-06-02T23:09:39.791331+0000","last_fullsized":"2024-06-02T23:09:39.791331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330069+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.330069+0000","last_peered":"2024-06-02T23:08:11.330069+0000","last_clean":"2024-06-02T23:08:11.330069+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.330069+0000","last_undegraded":"2024-06-02T23:08:11.330069+0000","last_fullsized":"2024-06-02T23:08:11.330069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801687+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:09:39.801687+0000","last_peered":"2024-06-02T23:09:39.801687+0000","last_clean":"2024-06-02T23:09:39.801687+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:09:39.801687+0000","last_undegraded":"2024-06-02T23:09:39.801687+0000","last_fullsized":"2024-06-02T23:09:39.801687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":446,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:56.685703+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:10:56.685703+0000","last_peered":"2024-06-02T23:10:56.685703+0000","last_clean":"2024-06-02T23:10:56.685703+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:10:56.685703+0000","last_undegraded":"2024-06-02T23:10:56.685703+0000","last_fullsized":"2024-06-02T23:10:56.685703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":382,"num_read_kb":255,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794578+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:09:39.794578+0000","last_peered":"2024-06-02T23:09:39.794578+0000","last_clean":"2024-06-02T23:09:39.794578+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:09:39.794578+0000","last_undegraded":"2024-06-02T23:09:39.794578+0000","last_fullsized":"2024-06-02T23:09:39.794578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332605+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332605+0000","last_peered":"2024-06-02T23:08:11.332605+0000","last_clean":"2024-06-02T23:08:11.332605+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332605+0000","last_undegraded":"2024-06-02T23:08:11.332605+0000","last_fullsized":"2024-06-02T23:08:11.332605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.778013+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:09:39.778013+0000","last_peered":"2024-06-02T23:09:39.778013+0000","last_clean":"2024-06-02T23:09:39.778013+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:09:39.778013+0000","last_undegraded":"2024-06-02T23:09:39.778013+0000","last_fullsized":"2024-06-02T23:09:39.778013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329465+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329465+0000","last_peered":"2024-06-02T23:08:11.329465+0000","last_clean":"2024-06-02T23:08:11.329465+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329465+0000","last_undegraded":"2024-06-02T23:08:11.329465+0000","last_fullsized":"2024-06-02T23:08:11.329465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792897+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:09:39.792897+0000","last_peered":"2024-06-02T23:09:39.792897+0000","last_clean":"2024-06-02T23:09:39.792897+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:09:39.792897+0000","last_undegraded":"2024-06-02T23:09:39.792897+0000","last_fullsized":"2024-06-02T23:09:39.792897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":220,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.173048+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:10:54.173048+0000","last_peered":"2024-06-02T23:10:54.173048+0000","last_clean":"2024-06-02T23:10:54.173048+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:10:54.173048+0000","last_undegraded":"2024-06-02T23:10:54.173048+0000","last_fullsized":"2024-06-02T23:10:54.173048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786680+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:09:39.786680+0000","last_peered":"2024-06-02T23:09:39.786680+0000","last_clean":"2024-06-02T23:09:39.786680+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:09:39.786680+0000","last_undegraded":"2024-06-02T23:09:39.786680+0000","last_fullsized":"2024-06-02T23:09:39.786680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215918+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:14.215918+0000","last_peered":"2024-06-02T23:08:14.215918+0000","last_clean":"2024-06-02T23:08:14.215918+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:14.215918+0000","last_undegraded":"2024-06-02T23:08:14.215918+0000","last_fullsized":"2024-06-02T23:08:14.215918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800649+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:09:39.800649+0000","last_peered":"2024-06-02T23:09:39.800649+0000","last_clean":"2024-06-02T23:09:39.800649+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:09:39.800649+0000","last_undegraded":"2024-06-02T23:09:39.800649+0000","last_fullsized":"2024-06-02T23:09:39.800649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:45.676951+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:09:45.676951+0000","last_peered":"2024-06-02T23:09:45.676951+0000","last_clean":"2024-06-02T23:09:45.676951+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:09:45.676951+0000","last_undegraded":"2024-06-02T23:09:45.676951+0000","last_fullsized":"2024-06-02T23:09:45.676951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.783215+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:09:39.783215+0000","last_peered":"2024-06-02T23:09:39.783215+0000","last_clean":"2024-06-02T23:09:39.783215+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:09:39.783215+0000","last_undegraded":"2024-06-02T23:09:39.783215+0000","last_fullsized":"2024-06-02T23:09:39.783215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328539+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328539+0000","last_peered":"2024-06-02T23:08:11.328539+0000","last_clean":"2024-06-02T23:08:11.328539+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328539+0000","last_undegraded":"2024-06-02T23:08:11.328539+0000","last_fullsized":"2024-06-02T23:08:11.328539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785214+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:09:39.785214+0000","last_peered":"2024-06-02T23:09:39.785214+0000","last_clean":"2024-06-02T23:09:39.785214+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:09:39.785214+0000","last_undegraded":"2024-06-02T23:09:39.785214+0000","last_fullsized":"2024-06-02T23:09:39.785214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":219,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.120458+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:10:54.120458+0000","last_peered":"2024-06-02T23:10:54.120458+0000","last_clean":"2024-06-02T23:10:54.120458+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:10:54.120458+0000","last_undegraded":"2024-06-02T23:10:54.120458+0000","last_fullsized":"2024-06-02T23:10:54.120458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791877+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:09:39.791877+0000","last_peered":"2024-06-02T23:09:39.791877+0000","last_clean":"2024-06-02T23:09:39.791877+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:09:39.791877+0000","last_undegraded":"2024-06-02T23:09:39.791877+0000","last_fullsized":"2024-06-02T23:09:39.791877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792620+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:09:39.792620+0000","last_peered":"2024-06-02T23:09:39.792620+0000","last_clean":"2024-06-02T23:09:39.792620+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:09:39.792620+0000","last_undegraded":"2024-06-02T23:09:39.792620+0000","last_fullsized":"2024-06-02T23:09:39.792620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":222,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:58.179159+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:10:58.179159+0000","last_peered":"2024-06-02T23:10:58.179159+0000","last_clean":"2024-06-02T23:10:58.179159+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:10:58.179159+0000","last_undegraded":"2024-06-02T23:10:58.179159+0000","last_fullsized":"2024-06-02T23:10:58.179159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794336+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:09:39.794336+0000","last_peered":"2024-06-02T23:09:39.794336+0000","last_clean":"2024-06-02T23:09:39.794336+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:09:39.794336+0000","last_undegraded":"2024-06-02T23:09:39.794336+0000","last_fullsized":"2024-06-02T23:09:39.794336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:45.814046+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:45.814046+0000","last_peered":"2024-06-02T23:08:45.814046+0000","last_clean":"2024-06-02T23:08:45.814046+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:45.814046+0000","last_undegraded":"2024-06-02T23:08:45.814046+0000","last_fullsized":"2024-06-02T23:08:45.814046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771357+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:09:39.771357+0000","last_peered":"2024-06-02T23:09:39.771357+0000","last_clean":"2024-06-02T23:09:39.771357+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:09:39.771357+0000","last_undegraded":"2024-06-02T23:09:39.771357+0000","last_fullsized":"2024-06-02T23:09:39.771357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788062+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:09:39.788062+0000","last_peered":"2024-06-02T23:09:39.788062+0000","last_clean":"2024-06-02T23:09:39.788062+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:09:39.788062+0000","last_undegraded":"2024-06-02T23:09:39.788062+0000","last_fullsized":"2024-06-02T23:09:39.788062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'19","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.236340+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:10:54.236340+0000","last_peered":"2024-06-02T23:10:54.236340+0000","last_clean":"2024-06-02T23:10:54.236340+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:10:54.236340+0000","last_undegraded":"2024-06-02T23:10:54.236340+0000","last_fullsized":"2024-06-02T23:10:54.236340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790001+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:09:39.790001+0000","last_peered":"2024-06-02T23:09:39.790001+0000","last_clean":"2024-06-02T23:09:39.790001+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:09:39.790001+0000","last_undegraded":"2024-06-02T23:09:39.790001+0000","last_fullsized":"2024-06-02T23:09:39.790001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450463+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450463+0000","last_peered":"2024-06-02T23:08:11.450463+0000","last_clean":"2024-06-02T23:08:11.450463+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450463+0000","last_undegraded":"2024-06-02T23:08:11.450463+0000","last_fullsized":"2024-06-02T23:08:11.450463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792568+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:09:39.792568+0000","last_peered":"2024-06-02T23:09:39.792568+0000","last_clean":"2024-06-02T23:09:39.792568+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:09:39.792568+0000","last_undegraded":"2024-06-02T23:09:39.792568+0000","last_fullsized":"2024-06-02T23:09:39.792568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334494+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334494+0000","last_peered":"2024-06-02T23:08:11.334494+0000","last_clean":"2024-06-02T23:08:11.334494+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334494+0000","last_undegraded":"2024-06-02T23:08:11.334494+0000","last_fullsized":"2024-06-02T23:08:11.334494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782299+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:09:39.782299+0000","last_peered":"2024-06-02T23:09:39.782299+0000","last_clean":"2024-06-02T23:09:39.782299+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:09:39.782299+0000","last_undegraded":"2024-06-02T23:09:39.782299+0000","last_fullsized":"2024-06-02T23:09:39.782299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748925+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748925+0000","last_peered":"2024-06-02T23:08:11.748925+0000","last_clean":"2024-06-02T23:08:11.748925+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748925+0000","last_undegraded":"2024-06-02T23:08:11.748925+0000","last_fullsized":"2024-06-02T23:08:11.748925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792513+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:09:39.792513+0000","last_peered":"2024-06-02T23:09:39.792513+0000","last_clean":"2024-06-02T23:09:39.792513+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:09:39.792513+0000","last_undegraded":"2024-06-02T23:09:39.792513+0000","last_fullsized":"2024-06-02T23:09:39.792513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.275353+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:10:54.275353+0000","last_peered":"2024-06-02T23:10:54.275353+0000","last_clean":"2024-06-02T23:10:54.275353+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:10:54.275353+0000","last_undegraded":"2024-06-02T23:10:54.275353+0000","last_fullsized":"2024-06-02T23:10:54.275353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786323+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:09:39.786323+0000","last_peered":"2024-06-02T23:09:39.786323+0000","last_clean":"2024-06-02T23:09:39.786323+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:09:39.786323+0000","last_undegraded":"2024-06-02T23:09:39.786323+0000","last_fullsized":"2024-06-02T23:09:39.786323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334806+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334806+0000","last_peered":"2024-06-02T23:08:11.334806+0000","last_clean":"2024-06-02T23:08:11.334806+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334806+0000","last_undegraded":"2024-06-02T23:08:11.334806+0000","last_fullsized":"2024-06-02T23:08:11.334806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797923+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:09:39.797923+0000","last_peered":"2024-06-02T23:09:39.797923+0000","last_clean":"2024-06-02T23:09:39.797923+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:09:39.797923+0000","last_undegraded":"2024-06-02T23:09:39.797923+0000","last_fullsized":"2024-06-02T23:09:39.797923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":228,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:58.251201+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:10:58.251201+0000","last_peered":"2024-06-02T23:10:58.251201+0000","last_clean":"2024-06-02T23:10:58.251201+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:10:58.251201+0000","last_undegraded":"2024-06-02T23:10:58.251201+0000","last_fullsized":"2024-06-02T23:10:58.251201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2193,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797984+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:09:39.797984+0000","last_peered":"2024-06-02T23:09:39.797984+0000","last_clean":"2024-06-02T23:09:39.797984+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:09:39.797984+0000","last_undegraded":"2024-06-02T23:09:39.797984+0000","last_fullsized":"2024-06-02T23:09:39.797984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330301+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330301+0000","last_peered":"2024-06-02T23:08:11.330301+0000","last_clean":"2024-06-02T23:08:11.330301+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330301+0000","last_undegraded":"2024-06-02T23:08:11.330301+0000","last_fullsized":"2024-06-02T23:08:11.330301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789936+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:09:39.789936+0000","last_peered":"2024-06-02T23:09:39.789936+0000","last_clean":"2024-06-02T23:09:39.789936+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:09:39.789936+0000","last_undegraded":"2024-06-02T23:09:39.789936+0000","last_fullsized":"2024-06-02T23:09:39.789936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333595+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333595+0000","last_peered":"2024-06-02T23:08:11.333595+0000","last_clean":"2024-06-02T23:08:11.333595+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333595+0000","last_undegraded":"2024-06-02T23:08:11.333595+0000","last_fullsized":"2024-06-02T23:08:11.333595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795458+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:09:39.795458+0000","last_peered":"2024-06-02T23:09:39.795458+0000","last_clean":"2024-06-02T23:09:39.795458+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:09:39.795458+0000","last_undegraded":"2024-06-02T23:09:39.795458+0000","last_fullsized":"2024-06-02T23:09:39.795458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790898+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:09:39.790898+0000","last_peered":"2024-06-02T23:09:39.790898+0000","last_clean":"2024-06-02T23:09:39.790898+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:09:39.790898+0000","last_undegraded":"2024-06-02T23:09:39.790898+0000","last_fullsized":"2024-06-02T23:09:39.790898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785496+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:09:39.785496+0000","last_peered":"2024-06-02T23:09:39.785496+0000","last_clean":"2024-06-02T23:09:39.785496+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:09:39.785496+0000","last_undegraded":"2024-06-02T23:09:39.785496+0000","last_fullsized":"2024-06-02T23:09:39.785496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785576+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:09:39.785576+0000","last_peered":"2024-06-02T23:09:39.785576+0000","last_clean":"2024-06-02T23:09:39.785576+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:09:39.785576+0000","last_undegraded":"2024-06-02T23:09:39.785576+0000","last_fullsized":"2024-06-02T23:09:39.785576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791535+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:09:39.791535+0000","last_peered":"2024-06-02T23:09:39.791535+0000","last_clean":"2024-06-02T23:09:39.791535+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:09:39.791535+0000","last_undegraded":"2024-06-02T23:09:39.791535+0000","last_fullsized":"2024-06-02T23:09:39.791535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792313+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:09:39.792313+0000","last_peered":"2024-06-02T23:09:39.792313+0000","last_clean":"2024-06-02T23:09:39.792313+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:09:39.792313+0000","last_undegraded":"2024-06-02T23:09:39.792313+0000","last_fullsized":"2024-06-02T23:09:39.792313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.774868+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:09:39.774868+0000","last_peered":"2024-06-02T23:09:39.774868+0000","last_clean":"2024-06-02T23:09:39.774868+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:09:39.774868+0000","last_undegraded":"2024-06-02T23:09:39.774868+0000","last_fullsized":"2024-06-02T23:09:39.774868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794273+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:09:39.794273+0000","last_peered":"2024-06-02T23:09:39.794273+0000","last_clean":"2024-06-02T23:09:39.794273+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:09:39.794273+0000","last_undegraded":"2024-06-02T23:09:39.794273+0000","last_fullsized":"2024-06-02T23:09:39.794273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788292+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:09:39.788292+0000","last_peered":"2024-06-02T23:09:39.788292+0000","last_clean":"2024-06-02T23:09:39.788292+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:09:39.788292+0000","last_undegraded":"2024-06-02T23:09:39.788292+0000","last_fullsized":"2024-06-02T23:09:39.788292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786749+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:09:39.786749+0000","last_peered":"2024-06-02T23:09:39.786749+0000","last_clean":"2024-06-02T23:09:39.786749+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:09:39.786749+0000","last_undegraded":"2024-06-02T23:09:39.786749+0000","last_fullsized":"2024-06-02T23:09:39.786749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789933+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:09:39.789933+0000","last_peered":"2024-06-02T23:09:39.789933+0000","last_clean":"2024-06-02T23:09:39.789933+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:09:39.789933+0000","last_undegraded":"2024-06-02T23:09:39.789933+0000","last_fullsized":"2024-06-02T23:09:39.789933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800745+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:09:39.800745+0000","last_peered":"2024-06-02T23:09:39.800745+0000","last_clean":"2024-06-02T23:09:39.800745+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:09:39.800745+0000","last_undegraded":"2024-06-02T23:09:39.800745+0000","last_fullsized":"2024-06-02T23:09:39.800745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792409+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:09:39.792409+0000","last_peered":"2024-06-02T23:09:39.792409+0000","last_clean":"2024-06-02T23:09:39.792409+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:09:39.792409+0000","last_undegraded":"2024-06-02T23:09:39.792409+0000","last_fullsized":"2024-06-02T23:09:39.792409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.776917+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:09:39.776917+0000","last_peered":"2024-06-02T23:09:39.776917+0000","last_clean":"2024-06-02T23:09:39.776917+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:09:39.776917+0000","last_undegraded":"2024-06-02T23:09:39.776917+0000","last_fullsized":"2024-06-02T23:09:39.776917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1901,"num_read_kb":1034,"num_write":273,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":227403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645375,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1138821,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332175,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074331,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":548352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":482429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":548481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1073025,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:11:00.694 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-02T23:11:00.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 -- 172.21.15.52:0/1514701128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5798103650 msgr2=0x7f5798109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 --2- 172.21.15.52:0/1514701128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5798103650 0x7f5798109ee0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f5790009cd0 tx=0x7f579002a8e0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 -- 172.21.15.52:0/1514701128 shutdown_connections 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 --2- 172.21.15.52:0/1514701128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5798103650 0x7f5798109ee0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 --2- 172.21.15.52:0/1514701128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102c90 0x7f5798103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 --2- 172.21.15.52:0/1514701128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57980ffcc0 0x7f57981000e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 -- 172.21.15.52:0/1514701128 >> 172.21.15.52:0/1514701128 conn(0x7f57980fb610 msgr2=0x7f57980fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 -- 172.21.15.52:0/1514701128 shutdown_connections 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 -- 172.21.15.52:0/1514701128 wait complete. 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.835+0000 7f579ce9a640 1 Processor -- start 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 -- start start 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 0x7f5798197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5798102c90 0x7f5798198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798103650 0x7f579819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f579810d520 con 0x7f57980ffcc0 2024-06-02T23:11:00.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f579810d3a0 con 0x7f5798103650 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579ce9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f579810d6a0 con 0x7f5798102c90 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 0x7f5798197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 0x7f5798197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38752/0 (socket says 172.21.15.52:38752) 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 -- 172.21.15.52:0/2782209053 learned_addr learned my addr 172.21.15.52:0/2782209053 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f5796d9d640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798103650 0x7f579819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f578dd9b640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5798102c90 0x7f5798198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 -- 172.21.15.52:0/2782209053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5798102c90 msgr2=0x7f5798198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5798102c90 0x7f5798198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 -- 172.21.15.52:0/2782209053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798103650 msgr2=0x7f579819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798103650 0x7f579819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.836+0000 7f579659c640 1 -- 172.21.15.52:0/2782209053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579819f810 con 0x7f57980ffcc0 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.837+0000 7f579659c640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 0x7f5798197c40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f578800e9c0 tx=0x7f578800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.837+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f578800cd80 con 0x7f57980ffcc0 2024-06-02T23:11:00.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.837+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579819fb00 con 0x7f57980ffcc0 2024-06-02T23:11:00.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.837+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f57880035f0 con 0x7f57980ffcc0 2024-06-02T23:11:00.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.837+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5788010690 con 0x7f57980ffcc0 2024-06-02T23:11:00.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.837+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57981a00a0 con 0x7f57980ffcc0 2024-06-02T23:11:00.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.838+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5798108ee0 con 0x7f57980ffcc0 2024-06-02T23:11:00.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.839+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f57880037a0 con 0x7f57980ffcc0 2024-06-02T23:11:00.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.840+0000 7f578ffff640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57840a2b90 0x7f57840a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:00.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.840+0000 7f578dd9b640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57840a2b90 0x7f57840a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:00.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.840+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f57880cf4c0 con 0x7f57980ffcc0 2024-06-02T23:11:00.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.840+0000 7f578dd9b640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57840a2b90 0x7f57840a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5798199100 tx=0x7f5780005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:00.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:00.844+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f578809c3f0 con 0x7f57980ffcc0 2024-06-02T23:11:01.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.027+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5798061de0 con 0x7f57840a2b90 2024-06-02T23:11:01.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.034+0000 7f578ffff640 1 -- 172.21.15.52:0/2782209053 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259073 (secure 0 0 0) 0x7f5798061de0 con 0x7f57840a2b90 2024-06-02T23:11:01.037 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:01.039 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:11:01.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57840a2b90 msgr2=0x7f57840a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57840a2b90 0x7f57840a5050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f5798199100 tx=0x7f5780005f90 comp rx=0 tx=0).stop 2024-06-02T23:11:01.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 msgr2=0x7f5798197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 0x7f5798197c40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f578800e9c0 tx=0x7f578800ee80 comp rx=0 tx=0).stop 2024-06-02T23:11:01.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 shutdown_connections 2024-06-02T23:11:01.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f57840a2b90 0x7f57840a5050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5798102c90 0x7f5798198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798103650 0x7f579819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 --2- 172.21.15.52:0/2782209053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f57980ffcc0 0x7f5798197c40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.037+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 >> 172.21.15.52:0/2782209053 conn(0x7f57980fb610 msgr2=0x7f57981060b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.038+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 shutdown_connections 2024-06-02T23:11:01.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.038+0000 7f579ce9a640 1 -- 172.21.15.52:0/2782209053 wait complete. 2024-06-02T23:11:01.049 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-06-02T23:10:59.812799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2027,"num_read_kb":1143,"num_write":456,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369640,"kb_used_data":12168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572184,"statfs":{"total":1151940427776,"available":1151561916416,"internally_reserved":0,"allocated":12460032,"data_stored":7749945,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6774,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002845"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802175+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:09:39.802175+0000","last_peered":"2024-06-02T23:09:39.802175+0000","last_clean":"2024-06-02T23:09:39.802175+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:09:39.802175+0000","last_undegraded":"2024-06-02T23:09:39.802175+0000","last_fullsized":"2024-06-02T23:09:39.802175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788231+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:09:39.788231+0000","last_peered":"2024-06-02T23:09:39.788231+0000","last_clean":"2024-06-02T23:09:39.788231+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:09:39.788231+0000","last_undegraded":"2024-06-02T23:09:39.788231+0000","last_fullsized":"2024-06-02T23:09:39.788231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794300+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:09:39.794300+0000","last_peered":"2024-06-02T23:09:39.794300+0000","last_clean":"2024-06-02T23:09:39.794300+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:09:39.794300+0000","last_undegraded":"2024-06-02T23:09:39.794300+0000","last_fullsized":"2024-06-02T23:09:39.794300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788417+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:09:39.788417+0000","last_peered":"2024-06-02T23:09:39.788417+0000","last_clean":"2024-06-02T23:09:39.788417+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:09:39.788417+0000","last_undegraded":"2024-06-02T23:09:39.788417+0000","last_fullsized":"2024-06-02T23:09:39.788417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.777964+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:09:39.777964+0000","last_peered":"2024-06-02T23:09:39.777964+0000","last_clean":"2024-06-02T23:09:39.777964+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:09:39.777964+0000","last_undegraded":"2024-06-02T23:09:39.777964+0000","last_fullsized":"2024-06-02T23:09:39.777964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782448+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:09:39.782448+0000","last_peered":"2024-06-02T23:09:39.782448+0000","last_clean":"2024-06-02T23:09:39.782448+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:09:39.782448+0000","last_undegraded":"2024-06-02T23:09:39.782448+0000","last_fullsized":"2024-06-02T23:09:39.782448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786918+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:09:39.786918+0000","last_peered":"2024-06-02T23:09:39.786918+0000","last_clean":"2024-06-02T23:09:39.786918+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:09:39.786918+0000","last_undegraded":"2024-06-02T23:09:39.786918+0000","last_fullsized":"2024-06-02T23:09:39.786918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786735+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:09:39.786735+0000","last_peered":"2024-06-02T23:09:39.786735+0000","last_clean":"2024-06-02T23:09:39.786735+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:09:39.786735+0000","last_undegraded":"2024-06-02T23:09:39.786735+0000","last_fullsized":"2024-06-02T23:09:39.786735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801527+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:09:39.801527+0000","last_peered":"2024-06-02T23:09:39.801527+0000","last_clean":"2024-06-02T23:09:39.801527+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:09:39.801527+0000","last_undegraded":"2024-06-02T23:09:39.801527+0000","last_fullsized":"2024-06-02T23:09:39.801527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789699+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:09:39.789699+0000","last_peered":"2024-06-02T23:09:39.789699+0000","last_clean":"2024-06-02T23:09:39.789699+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:09:39.789699+0000","last_undegraded":"2024-06-02T23:09:39.789699+0000","last_fullsized":"2024-06-02T23:09:39.789699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797457+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:09:39.797457+0000","last_peered":"2024-06-02T23:09:39.797457+0000","last_clean":"2024-06-02T23:09:39.797457+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:09:39.797457+0000","last_undegraded":"2024-06-02T23:09:39.797457+0000","last_fullsized":"2024-06-02T23:09:39.797457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.787757+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:09:39.787757+0000","last_peered":"2024-06-02T23:09:39.787757+0000","last_clean":"2024-06-02T23:09:39.787757+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:09:39.787757+0000","last_undegraded":"2024-06-02T23:09:39.787757+0000","last_fullsized":"2024-06-02T23:09:39.787757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786117+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:09:39.786117+0000","last_peered":"2024-06-02T23:09:39.786117+0000","last_clean":"2024-06-02T23:09:39.786117+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:09:39.786117+0000","last_undegraded":"2024-06-02T23:09:39.786117+0000","last_fullsized":"2024-06-02T23:09:39.786117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788799+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:09:39.788799+0000","last_peered":"2024-06-02T23:09:39.788799+0000","last_clean":"2024-06-02T23:09:39.788799+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:09:39.788799+0000","last_undegraded":"2024-06-02T23:09:39.788799+0000","last_fullsized":"2024-06-02T23:09:39.788799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791625+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:09:39.791625+0000","last_peered":"2024-06-02T23:09:39.791625+0000","last_clean":"2024-06-02T23:09:39.791625+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:09:39.791625+0000","last_undegraded":"2024-06-02T23:09:39.791625+0000","last_fullsized":"2024-06-02T23:09:39.791625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800433+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:09:39.800433+0000","last_peered":"2024-06-02T23:09:39.800433+0000","last_clean":"2024-06-02T23:09:39.800433+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:09:39.800433+0000","last_undegraded":"2024-06-02T23:09:39.800433+0000","last_fullsized":"2024-06-02T23:09:39.800433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802531+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:09:39.802531+0000","last_peered":"2024-06-02T23:09:39.802531+0000","last_clean":"2024-06-02T23:09:39.802531+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:09:39.802531+0000","last_undegraded":"2024-06-02T23:09:39.802531+0000","last_fullsized":"2024-06-02T23:09:39.802531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.803680+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:09:39.803680+0000","last_peered":"2024-06-02T23:09:39.803680+0000","last_clean":"2024-06-02T23:09:39.803680+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:09:39.803680+0000","last_undegraded":"2024-06-02T23:09:39.803680+0000","last_fullsized":"2024-06-02T23:09:39.803680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790081+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:09:39.790081+0000","last_peered":"2024-06-02T23:09:39.790081+0000","last_clean":"2024-06-02T23:09:39.790081+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:09:39.790081+0000","last_undegraded":"2024-06-02T23:09:39.790081+0000","last_fullsized":"2024-06-02T23:09:39.790081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":227,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:55.121314+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:10:55.121314+0000","last_peered":"2024-06-02T23:10:55.121314+0000","last_clean":"2024-06-02T23:10:55.121314+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:10:55.121314+0000","last_undegraded":"2024-06-02T23:10:55.121314+0000","last_fullsized":"2024-06-02T23:10:55.121314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788800+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:09:39.788800+0000","last_peered":"2024-06-02T23:09:39.788800+0000","last_clean":"2024-06-02T23:09:39.788800+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:09:39.788800+0000","last_undegraded":"2024-06-02T23:09:39.788800+0000","last_fullsized":"2024-06-02T23:09:39.788800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794320+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:09:39.794320+0000","last_peered":"2024-06-02T23:09:39.794320+0000","last_clean":"2024-06-02T23:09:39.794320+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:09:39.794320+0000","last_undegraded":"2024-06-02T23:09:39.794320+0000","last_fullsized":"2024-06-02T23:09:39.794320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215945+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:14.215945+0000","last_peered":"2024-06-02T23:08:14.215945+0000","last_clean":"2024-06-02T23:08:14.215945+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:14.215945+0000","last_undegraded":"2024-06-02T23:08:14.215945+0000","last_fullsized":"2024-06-02T23:08:14.215945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802474+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:09:39.802474+0000","last_peered":"2024-06-02T23:09:39.802474+0000","last_clean":"2024-06-02T23:09:39.802474+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:09:39.802474+0000","last_undegraded":"2024-06-02T23:09:39.802474+0000","last_fullsized":"2024-06-02T23:09:39.802474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335772+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335772+0000","last_peered":"2024-06-02T23:08:11.335772+0000","last_clean":"2024-06-02T23:08:11.335772+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335772+0000","last_undegraded":"2024-06-02T23:08:11.335772+0000","last_fullsized":"2024-06-02T23:08:11.335772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800976+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:09:39.800976+0000","last_peered":"2024-06-02T23:09:39.800976+0000","last_clean":"2024-06-02T23:09:39.800976+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:09:39.800976+0000","last_undegraded":"2024-06-02T23:09:39.800976+0000","last_fullsized":"2024-06-02T23:09:39.800976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":235,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:57.687377+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:10:57.687377+0000","last_peered":"2024-06-02T23:10:57.687377+0000","last_clean":"2024-06-02T23:10:57.687377+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:10:57.687377+0000","last_undegraded":"2024-06-02T23:10:57.687377+0000","last_fullsized":"2024-06-02T23:10:57.687377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771608+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:09:39.771608+0000","last_peered":"2024-06-02T23:09:39.771608+0000","last_clean":"2024-06-02T23:09:39.771608+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:09:39.771608+0000","last_undegraded":"2024-06-02T23:09:39.771608+0000","last_fullsized":"2024-06-02T23:09:39.771608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449603+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449603+0000","last_peered":"2024-06-02T23:08:11.449603+0000","last_clean":"2024-06-02T23:08:11.449603+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449603+0000","last_undegraded":"2024-06-02T23:08:11.449603+0000","last_fullsized":"2024-06-02T23:08:11.449603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795377+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:09:39.795377+0000","last_peered":"2024-06-02T23:09:39.795377+0000","last_clean":"2024-06-02T23:09:39.795377+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:09:39.795377+0000","last_undegraded":"2024-06-02T23:09:39.795377+0000","last_fullsized":"2024-06-02T23:09:39.795377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332946+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332946+0000","last_peered":"2024-06-02T23:08:11.332946+0000","last_clean":"2024-06-02T23:08:11.332946+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332946+0000","last_undegraded":"2024-06-02T23:08:11.332946+0000","last_fullsized":"2024-06-02T23:08:11.332946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785226+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:09:39.785226+0000","last_peered":"2024-06-02T23:09:39.785226+0000","last_clean":"2024-06-02T23:09:39.785226+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:09:39.785226+0000","last_undegraded":"2024-06-02T23:09:39.785226+0000","last_fullsized":"2024-06-02T23:09:39.785226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329286+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329286+0000","last_peered":"2024-06-02T23:08:11.329286+0000","last_clean":"2024-06-02T23:08:11.329286+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329286+0000","last_undegraded":"2024-06-02T23:08:11.329286+0000","last_fullsized":"2024-06-02T23:08:11.329286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.779022+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:09:39.779022+0000","last_peered":"2024-06-02T23:09:39.779022+0000","last_clean":"2024-06-02T23:09:39.779022+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:09:39.779022+0000","last_undegraded":"2024-06-02T23:09:39.779022+0000","last_fullsized":"2024-06-02T23:09:39.779022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":405,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:56.684971+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:10:56.684971+0000","last_peered":"2024-06-02T23:10:56.684971+0000","last_clean":"2024-06-02T23:10:56.684971+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:10:56.684971+0000","last_undegraded":"2024-06-02T23:10:56.684971+0000","last_fullsized":"2024-06-02T23:10:56.684971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":371,"num_read_kb":371,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802707+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:09:39.802707+0000","last_peered":"2024-06-02T23:09:39.802707+0000","last_clean":"2024-06-02T23:09:39.802707+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:09:39.802707+0000","last_undegraded":"2024-06-02T23:09:39.802707+0000","last_fullsized":"2024-06-02T23:09:39.802707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791331+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:09:39.791331+0000","last_peered":"2024-06-02T23:09:39.791331+0000","last_clean":"2024-06-02T23:09:39.791331+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:09:39.791331+0000","last_undegraded":"2024-06-02T23:09:39.791331+0000","last_fullsized":"2024-06-02T23:09:39.791331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330069+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.330069+0000","last_peered":"2024-06-02T23:08:11.330069+0000","last_clean":"2024-06-02T23:08:11.330069+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.330069+0000","last_undegraded":"2024-06-02T23:08:11.330069+0000","last_fullsized":"2024-06-02T23:08:11.330069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801687+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:09:39.801687+0000","last_peered":"2024-06-02T23:09:39.801687+0000","last_clean":"2024-06-02T23:09:39.801687+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:09:39.801687+0000","last_undegraded":"2024-06-02T23:09:39.801687+0000","last_fullsized":"2024-06-02T23:09:39.801687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":446,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:56.685703+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:10:56.685703+0000","last_peered":"2024-06-02T23:10:56.685703+0000","last_clean":"2024-06-02T23:10:56.685703+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:10:56.685703+0000","last_undegraded":"2024-06-02T23:10:56.685703+0000","last_fullsized":"2024-06-02T23:10:56.685703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":382,"num_read_kb":255,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794578+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:09:39.794578+0000","last_peered":"2024-06-02T23:09:39.794578+0000","last_clean":"2024-06-02T23:09:39.794578+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:09:39.794578+0000","last_undegraded":"2024-06-02T23:09:39.794578+0000","last_fullsized":"2024-06-02T23:09:39.794578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332605+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332605+0000","last_peered":"2024-06-02T23:08:11.332605+0000","last_clean":"2024-06-02T23:08:11.332605+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332605+0000","last_undegraded":"2024-06-02T23:08:11.332605+0000","last_fullsized":"2024-06-02T23:08:11.332605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.778013+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:09:39.778013+0000","last_peered":"2024-06-02T23:09:39.778013+0000","last_clean":"2024-06-02T23:09:39.778013+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:09:39.778013+0000","last_undegraded":"2024-06-02T23:09:39.778013+0000","last_fullsized":"2024-06-02T23:09:39.778013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329465+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329465+0000","last_peered":"2024-06-02T23:08:11.329465+0000","last_clean":"2024-06-02T23:08:11.329465+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329465+0000","last_undegraded":"2024-06-02T23:08:11.329465+0000","last_fullsized":"2024-06-02T23:08:11.329465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792897+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:09:39.792897+0000","last_peered":"2024-06-02T23:09:39.792897+0000","last_clean":"2024-06-02T23:09:39.792897+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:09:39.792897+0000","last_undegraded":"2024-06-02T23:09:39.792897+0000","last_fullsized":"2024-06-02T23:09:39.792897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":220,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.173048+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:10:54.173048+0000","last_peered":"2024-06-02T23:10:54.173048+0000","last_clean":"2024-06-02T23:10:54.173048+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:10:54.173048+0000","last_undegraded":"2024-06-02T23:10:54.173048+0000","last_fullsized":"2024-06-02T23:10:54.173048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786680+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:09:39.786680+0000","last_peered":"2024-06-02T23:09:39.786680+0000","last_clean":"2024-06-02T23:09:39.786680+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:09:39.786680+0000","last_undegraded":"2024-06-02T23:09:39.786680+0000","last_fullsized":"2024-06-02T23:09:39.786680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215918+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:14.215918+0000","last_peered":"2024-06-02T23:08:14.215918+0000","last_clean":"2024-06-02T23:08:14.215918+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:14.215918+0000","last_undegraded":"2024-06-02T23:08:14.215918+0000","last_fullsized":"2024-06-02T23:08:14.215918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800649+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:09:39.800649+0000","last_peered":"2024-06-02T23:09:39.800649+0000","last_clean":"2024-06-02T23:09:39.800649+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:09:39.800649+0000","last_undegraded":"2024-06-02T23:09:39.800649+0000","last_fullsized":"2024-06-02T23:09:39.800649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:45.676951+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:09:45.676951+0000","last_peered":"2024-06-02T23:09:45.676951+0000","last_clean":"2024-06-02T23:09:45.676951+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:09:45.676951+0000","last_undegraded":"2024-06-02T23:09:45.676951+0000","last_fullsized":"2024-06-02T23:09:45.676951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.783215+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:09:39.783215+0000","last_peered":"2024-06-02T23:09:39.783215+0000","last_clean":"2024-06-02T23:09:39.783215+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:09:39.783215+0000","last_undegraded":"2024-06-02T23:09:39.783215+0000","last_fullsized":"2024-06-02T23:09:39.783215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328539+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328539+0000","last_peered":"2024-06-02T23:08:11.328539+0000","last_clean":"2024-06-02T23:08:11.328539+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328539+0000","last_undegraded":"2024-06-02T23:08:11.328539+0000","last_fullsized":"2024-06-02T23:08:11.328539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785214+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:09:39.785214+0000","last_peered":"2024-06-02T23:09:39.785214+0000","last_clean":"2024-06-02T23:09:39.785214+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:09:39.785214+0000","last_undegraded":"2024-06-02T23:09:39.785214+0000","last_fullsized":"2024-06-02T23:09:39.785214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":219,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.120458+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:10:54.120458+0000","last_peered":"2024-06-02T23:10:54.120458+0000","last_clean":"2024-06-02T23:10:54.120458+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:10:54.120458+0000","last_undegraded":"2024-06-02T23:10:54.120458+0000","last_fullsized":"2024-06-02T23:10:54.120458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791877+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:09:39.791877+0000","last_peered":"2024-06-02T23:09:39.791877+0000","last_clean":"2024-06-02T23:09:39.791877+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:09:39.791877+0000","last_undegraded":"2024-06-02T23:09:39.791877+0000","last_fullsized":"2024-06-02T23:09:39.791877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792620+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:09:39.792620+0000","last_peered":"2024-06-02T23:09:39.792620+0000","last_clean":"2024-06-02T23:09:39.792620+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:09:39.792620+0000","last_undegraded":"2024-06-02T23:09:39.792620+0000","last_fullsized":"2024-06-02T23:09:39.792620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":222,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:58.179159+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:10:58.179159+0000","last_peered":"2024-06-02T23:10:58.179159+0000","last_clean":"2024-06-02T23:10:58.179159+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:10:58.179159+0000","last_undegraded":"2024-06-02T23:10:58.179159+0000","last_fullsized":"2024-06-02T23:10:58.179159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794336+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:09:39.794336+0000","last_peered":"2024-06-02T23:09:39.794336+0000","last_clean":"2024-06-02T23:09:39.794336+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:09:39.794336+0000","last_undegraded":"2024-06-02T23:09:39.794336+0000","last_fullsized":"2024-06-02T23:09:39.794336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:45.814046+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:45.814046+0000","last_peered":"2024-06-02T23:08:45.814046+0000","last_clean":"2024-06-02T23:08:45.814046+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:45.814046+0000","last_undegraded":"2024-06-02T23:08:45.814046+0000","last_fullsized":"2024-06-02T23:08:45.814046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771357+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:09:39.771357+0000","last_peered":"2024-06-02T23:09:39.771357+0000","last_clean":"2024-06-02T23:09:39.771357+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:09:39.771357+0000","last_undegraded":"2024-06-02T23:09:39.771357+0000","last_fullsized":"2024-06-02T23:09:39.771357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788062+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:09:39.788062+0000","last_peered":"2024-06-02T23:09:39.788062+0000","last_clean":"2024-06-02T23:09:39.788062+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:09:39.788062+0000","last_undegraded":"2024-06-02T23:09:39.788062+0000","last_fullsized":"2024-06-02T23:09:39.788062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'19","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.236340+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:10:54.236340+0000","last_peered":"2024-06-02T23:10:54.236340+0000","last_clean":"2024-06-02T23:10:54.236340+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:10:54.236340+0000","last_undegraded":"2024-06-02T23:10:54.236340+0000","last_fullsized":"2024-06-02T23:10:54.236340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790001+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:09:39.790001+0000","last_peered":"2024-06-02T23:09:39.790001+0000","last_clean":"2024-06-02T23:09:39.790001+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:09:39.790001+0000","last_undegraded":"2024-06-02T23:09:39.790001+0000","last_fullsized":"2024-06-02T23:09:39.790001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450463+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450463+0000","last_peered":"2024-06-02T23:08:11.450463+0000","last_clean":"2024-06-02T23:08:11.450463+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450463+0000","last_undegraded":"2024-06-02T23:08:11.450463+0000","last_fullsized":"2024-06-02T23:08:11.450463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792568+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:09:39.792568+0000","last_peered":"2024-06-02T23:09:39.792568+0000","last_clean":"2024-06-02T23:09:39.792568+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:09:39.792568+0000","last_undegraded":"2024-06-02T23:09:39.792568+0000","last_fullsized":"2024-06-02T23:09:39.792568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334494+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334494+0000","last_peered":"2024-06-02T23:08:11.334494+0000","last_clean":"2024-06-02T23:08:11.334494+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334494+0000","last_undegraded":"2024-06-02T23:08:11.334494+0000","last_fullsized":"2024-06-02T23:08:11.334494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782299+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:09:39.782299+0000","last_peered":"2024-06-02T23:09:39.782299+0000","last_clean":"2024-06-02T23:09:39.782299+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:09:39.782299+0000","last_undegraded":"2024-06-02T23:09:39.782299+0000","last_fullsized":"2024-06-02T23:09:39.782299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748925+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748925+0000","last_peered":"2024-06-02T23:08:11.748925+0000","last_clean":"2024-06-02T23:08:11.748925+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748925+0000","last_undegraded":"2024-06-02T23:08:11.748925+0000","last_fullsized":"2024-06-02T23:08:11.748925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792513+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:09:39.792513+0000","last_peered":"2024-06-02T23:09:39.792513+0000","last_clean":"2024-06-02T23:09:39.792513+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:09:39.792513+0000","last_undegraded":"2024-06-02T23:09:39.792513+0000","last_fullsized":"2024-06-02T23:09:39.792513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.275353+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:10:54.275353+0000","last_peered":"2024-06-02T23:10:54.275353+0000","last_clean":"2024-06-02T23:10:54.275353+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:10:54.275353+0000","last_undegraded":"2024-06-02T23:10:54.275353+0000","last_fullsized":"2024-06-02T23:10:54.275353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786323+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:09:39.786323+0000","last_peered":"2024-06-02T23:09:39.786323+0000","last_clean":"2024-06-02T23:09:39.786323+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:09:39.786323+0000","last_undegraded":"2024-06-02T23:09:39.786323+0000","last_fullsized":"2024-06-02T23:09:39.786323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334806+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334806+0000","last_peered":"2024-06-02T23:08:11.334806+0000","last_clean":"2024-06-02T23:08:11.334806+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334806+0000","last_undegraded":"2024-06-02T23:08:11.334806+0000","last_fullsized":"2024-06-02T23:08:11.334806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797923+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:09:39.797923+0000","last_peered":"2024-06-02T23:09:39.797923+0000","last_clean":"2024-06-02T23:09:39.797923+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:09:39.797923+0000","last_undegraded":"2024-06-02T23:09:39.797923+0000","last_fullsized":"2024-06-02T23:09:39.797923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":228,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:58.251201+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:10:58.251201+0000","last_peered":"2024-06-02T23:10:58.251201+0000","last_clean":"2024-06-02T23:10:58.251201+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:10:58.251201+0000","last_undegraded":"2024-06-02T23:10:58.251201+0000","last_fullsized":"2024-06-02T23:10:58.251201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2193,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797984+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:09:39.797984+0000","last_peered":"2024-06-02T23:09:39.797984+0000","last_clean":"2024-06-02T23:09:39.797984+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:09:39.797984+0000","last_undegraded":"2024-06-02T23:09:39.797984+0000","last_fullsized":"2024-06-02T23:09:39.797984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330301+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330301+0000","last_peered":"2024-06-02T23:08:11.330301+0000","last_clean":"2024-06-02T23:08:11.330301+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330301+0000","last_undegraded":"2024-06-02T23:08:11.330301+0000","last_fullsized":"2024-06-02T23:08:11.330301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789936+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:09:39.789936+0000","last_peered":"2024-06-02T23:09:39.789936+0000","last_clean":"2024-06-02T23:09:39.789936+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:09:39.789936+0000","last_undegraded":"2024-06-02T23:09:39.789936+0000","last_fullsized":"2024-06-02T23:09:39.789936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333595+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333595+0000","last_peered":"2024-06-02T23:08:11.333595+0000","last_clean":"2024-06-02T23:08:11.333595+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333595+0000","last_undegraded":"2024-06-02T23:08:11.333595+0000","last_fullsized":"2024-06-02T23:08:11.333595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795458+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:09:39.795458+0000","last_peered":"2024-06-02T23:09:39.795458+0000","last_clean":"2024-06-02T23:09:39.795458+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:09:39.795458+0000","last_undegraded":"2024-06-02T23:09:39.795458+0000","last_fullsized":"2024-06-02T23:09:39.795458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790898+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:09:39.790898+0000","last_peered":"2024-06-02T23:09:39.790898+0000","last_clean":"2024-06-02T23:09:39.790898+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:09:39.790898+0000","last_undegraded":"2024-06-02T23:09:39.790898+0000","last_fullsized":"2024-06-02T23:09:39.790898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785496+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:09:39.785496+0000","last_peered":"2024-06-02T23:09:39.785496+0000","last_clean":"2024-06-02T23:09:39.785496+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:09:39.785496+0000","last_undegraded":"2024-06-02T23:09:39.785496+0000","last_fullsized":"2024-06-02T23:09:39.785496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785576+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:09:39.785576+0000","last_peered":"2024-06-02T23:09:39.785576+0000","last_clean":"2024-06-02T23:09:39.785576+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:09:39.785576+0000","last_undegraded":"2024-06-02T23:09:39.785576+0000","last_fullsized":"2024-06-02T23:09:39.785576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791535+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:09:39.791535+0000","last_peered":"2024-06-02T23:09:39.791535+0000","last_clean":"2024-06-02T23:09:39.791535+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:09:39.791535+0000","last_undegraded":"2024-06-02T23:09:39.791535+0000","last_fullsized":"2024-06-02T23:09:39.791535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792313+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:09:39.792313+0000","last_peered":"2024-06-02T23:09:39.792313+0000","last_clean":"2024-06-02T23:09:39.792313+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:09:39.792313+0000","last_undegraded":"2024-06-02T23:09:39.792313+0000","last_fullsized":"2024-06-02T23:09:39.792313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.774868+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:09:39.774868+0000","last_peered":"2024-06-02T23:09:39.774868+0000","last_clean":"2024-06-02T23:09:39.774868+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:09:39.774868+0000","last_undegraded":"2024-06-02T23:09:39.774868+0000","last_fullsized":"2024-06-02T23:09:39.774868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794273+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:09:39.794273+0000","last_peered":"2024-06-02T23:09:39.794273+0000","last_clean":"2024-06-02T23:09:39.794273+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:09:39.794273+0000","last_undegraded":"2024-06-02T23:09:39.794273+0000","last_fullsized":"2024-06-02T23:09:39.794273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788292+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:09:39.788292+0000","last_peered":"2024-06-02T23:09:39.788292+0000","last_clean":"2024-06-02T23:09:39.788292+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:09:39.788292+0000","last_undegraded":"2024-06-02T23:09:39.788292+0000","last_fullsized":"2024-06-02T23:09:39.788292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786749+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:09:39.786749+0000","last_peered":"2024-06-02T23:09:39.786749+0000","last_clean":"2024-06-02T23:09:39.786749+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:09:39.786749+0000","last_undegraded":"2024-06-02T23:09:39.786749+0000","last_fullsized":"2024-06-02T23:09:39.786749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789933+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:09:39.789933+0000","last_peered":"2024-06-02T23:09:39.789933+0000","last_clean":"2024-06-02T23:09:39.789933+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:09:39.789933+0000","last_undegraded":"2024-06-02T23:09:39.789933+0000","last_fullsized":"2024-06-02T23:09:39.789933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800745+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:09:39.800745+0000","last_peered":"2024-06-02T23:09:39.800745+0000","last_clean":"2024-06-02T23:09:39.800745+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:09:39.800745+0000","last_undegraded":"2024-06-02T23:09:39.800745+0000","last_fullsized":"2024-06-02T23:09:39.800745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792409+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:09:39.792409+0000","last_peered":"2024-06-02T23:09:39.792409+0000","last_clean":"2024-06-02T23:09:39.792409+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:09:39.792409+0000","last_undegraded":"2024-06-02T23:09:39.792409+0000","last_fullsized":"2024-06-02T23:09:39.792409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.776917+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:09:39.776917+0000","last_peered":"2024-06-02T23:09:39.776917+0000","last_clean":"2024-06-02T23:09:39.776917+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:09:39.776917+0000","last_undegraded":"2024-06-02T23:09:39.776917+0000","last_fullsized":"2024-06-02T23:09:39.776917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1901,"num_read_kb":1034,"num_write":273,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":227403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645375,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1138821,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332175,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074331,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":548352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":482429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":548481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1073025,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:11:01.053 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-06-02T23:11:01.053 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-06-02T23:11:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 -- 172.21.15.52:0/4152187970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0102460 msgr2=0x7fbac0104850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 --2- 172.21.15.52:0/4152187970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0102460 0x7fbac0104850 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbab8009900 tx=0x7fbab802a410 comp rx=0 tx=0).stop 2024-06-02T23:11:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 -- 172.21.15.52:0/4152187970 shutdown_connections 2024-06-02T23:11:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 --2- 172.21.15.52:0/4152187970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac01076c0 0x7fbac0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 --2- 172.21.15.52:0/4152187970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbac0104d90 0x7fbac0107180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 --2- 172.21.15.52:0/4152187970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0102460 0x7fbac0104850 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 -- 172.21.15.52:0/4152187970 >> 172.21.15.52:0/4152187970 conn(0x7fbac00fbe50 msgr2=0x7fbac00fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 -- 172.21.15.52:0/4152187970 shutdown_connections 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 -- 172.21.15.52:0/4152187970 wait complete. 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 Processor -- start 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.196+0000 7fbac77ce640 1 -- start start 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbac0102460 0x7fbac0101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 0x7fbac0100410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac01076c0 0x7fbac0100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010d510 con 0x7fbac0104d90 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010d390 con 0x7fbac0102460 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010d690 con 0x7fbac01076c0 2024-06-02T23:11:01.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 0x7fbac0100410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 0x7fbac0100410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38780/0 (socket says 172.21.15.52:38780) 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 -- 172.21.15.52:0/3906116196 learned_addr learned my addr 172.21.15.52:0/3906116196 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac556a640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbac0102460 0x7fbac0101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 -- 172.21.15.52:0/3906116196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac01076c0 msgr2=0x7fbac0100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac5d6b640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac01076c0 0x7fbac0100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac01076c0 0x7fbac0100950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 -- 172.21.15.52:0/3906116196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbac0102460 msgr2=0x7fbac0101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbac0102460 0x7fbac0101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 -- 172.21.15.52:0/3906116196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac01011e0 con 0x7fbac0104d90 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac4d69640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 0x7fbac0100410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c970 tx=0x7fbab400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4014070 con 0x7fbac0104d90 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbab4004440 con 0x7fbac0104d90 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac01a1d10 con 0x7fbac0104d90 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4003e50 con 0x7fbac0104d90 2024-06-02T23:11:01.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.197+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a21f0 con 0x7fbac0104d90 2024-06-02T23:11:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.198+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac00fd940 con 0x7fbac0104d90 2024-06-02T23:11:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.200+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbab4017070 con 0x7fbac0104d90 2024-06-02T23:11:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.200+0000 7fbabe7fc640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbaa80a2db0 0x7fbaa80a5270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.200+0000 7fbac556a640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbaa80a2db0 0x7fbaa80a5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.200+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbab40cf290 con 0x7fbac0104d90 2024-06-02T23:11:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.201+0000 7fbac556a640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbaa80a2db0 0x7fbaa80a5270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbab8000c00 tx=0x7fbab8005e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:01.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.204+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbab401e030 con 0x7fbac0104d90 2024-06-02T23:11:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.384+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbac0061de0 con 0x7fbaa80a2db0 2024-06-02T23:11:01.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.391+0000 7fbabe7fc640 1 -- 172.21.15.52:0/3906116196 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259073 (secure 0 0 0) 0x7fbac0061de0 con 0x7fbaa80a2db0 2024-06-02T23:11:01.394 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:11:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbaa80a2db0 msgr2=0x7fbaa80a5270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbaa80a2db0 0x7fbaa80a5270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fbab8000c00 tx=0x7fbab8005e40 comp rx=0 tx=0).stop 2024-06-02T23:11:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 msgr2=0x7fbac0100410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 0x7fbac0100410 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c970 tx=0x7fbab400ce30 comp rx=0 tx=0).stop 2024-06-02T23:11:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 shutdown_connections 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbaa80a2db0 0x7fbaa80a5270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbac01076c0 0x7fbac0100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbac0104d90 0x7fbac0100410 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 --2- 172.21.15.52:0/3906116196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbac0102460 0x7fbac0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 >> 172.21.15.52:0/3906116196 conn(0x7fbac00fbe50 msgr2=0x7fbac0102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 shutdown_connections 2024-06-02T23:11:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:01.395+0000 7fbac77ce640 1 -- 172.21.15.52:0/3906116196 wait complete. 2024-06-02T23:11:01.398 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:01 smithi174 ceph-mon[35994]: pgmap v405: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:01.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:01 smithi052 ceph-mon[32340]: pgmap v405: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:01.406 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-06-02T23:10:59.812799+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2027,"num_read_kb":1143,"num_write":456,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369640,"kb_used_data":12168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572184,"statfs":{"total":1151940427776,"available":1151561916416,"internally_reserved":0,"allocated":12460032,"data_stored":7749945,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6774,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":65,"num_write":21,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002845"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802175+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:09:39.802175+0000","last_peered":"2024-06-02T23:09:39.802175+0000","last_clean":"2024-06-02T23:09:39.802175+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:09:39.802175+0000","last_undegraded":"2024-06-02T23:09:39.802175+0000","last_fullsized":"2024-06-02T23:09:39.802175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788231+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:09:39.788231+0000","last_peered":"2024-06-02T23:09:39.788231+0000","last_clean":"2024-06-02T23:09:39.788231+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:09:39.788231+0000","last_undegraded":"2024-06-02T23:09:39.788231+0000","last_fullsized":"2024-06-02T23:09:39.788231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794300+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:09:39.794300+0000","last_peered":"2024-06-02T23:09:39.794300+0000","last_clean":"2024-06-02T23:09:39.794300+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:09:39.794300+0000","last_undegraded":"2024-06-02T23:09:39.794300+0000","last_fullsized":"2024-06-02T23:09:39.794300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788417+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:09:39.788417+0000","last_peered":"2024-06-02T23:09:39.788417+0000","last_clean":"2024-06-02T23:09:39.788417+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:09:39.788417+0000","last_undegraded":"2024-06-02T23:09:39.788417+0000","last_fullsized":"2024-06-02T23:09:39.788417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.777964+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:09:39.777964+0000","last_peered":"2024-06-02T23:09:39.777964+0000","last_clean":"2024-06-02T23:09:39.777964+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:09:39.777964+0000","last_undegraded":"2024-06-02T23:09:39.777964+0000","last_fullsized":"2024-06-02T23:09:39.777964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782448+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:09:39.782448+0000","last_peered":"2024-06-02T23:09:39.782448+0000","last_clean":"2024-06-02T23:09:39.782448+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:09:39.782448+0000","last_undegraded":"2024-06-02T23:09:39.782448+0000","last_fullsized":"2024-06-02T23:09:39.782448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786918+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:09:39.786918+0000","last_peered":"2024-06-02T23:09:39.786918+0000","last_clean":"2024-06-02T23:09:39.786918+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:09:39.786918+0000","last_undegraded":"2024-06-02T23:09:39.786918+0000","last_fullsized":"2024-06-02T23:09:39.786918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786735+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:09:39.786735+0000","last_peered":"2024-06-02T23:09:39.786735+0000","last_clean":"2024-06-02T23:09:39.786735+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:09:39.786735+0000","last_undegraded":"2024-06-02T23:09:39.786735+0000","last_fullsized":"2024-06-02T23:09:39.786735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801527+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:09:39.801527+0000","last_peered":"2024-06-02T23:09:39.801527+0000","last_clean":"2024-06-02T23:09:39.801527+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:09:39.801527+0000","last_undegraded":"2024-06-02T23:09:39.801527+0000","last_fullsized":"2024-06-02T23:09:39.801527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789699+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:09:39.789699+0000","last_peered":"2024-06-02T23:09:39.789699+0000","last_clean":"2024-06-02T23:09:39.789699+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:09:39.789699+0000","last_undegraded":"2024-06-02T23:09:39.789699+0000","last_fullsized":"2024-06-02T23:09:39.789699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797457+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:09:39.797457+0000","last_peered":"2024-06-02T23:09:39.797457+0000","last_clean":"2024-06-02T23:09:39.797457+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:09:39.797457+0000","last_undegraded":"2024-06-02T23:09:39.797457+0000","last_fullsized":"2024-06-02T23:09:39.797457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.787757+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:09:39.787757+0000","last_peered":"2024-06-02T23:09:39.787757+0000","last_clean":"2024-06-02T23:09:39.787757+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:09:39.787757+0000","last_undegraded":"2024-06-02T23:09:39.787757+0000","last_fullsized":"2024-06-02T23:09:39.787757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786117+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:09:39.786117+0000","last_peered":"2024-06-02T23:09:39.786117+0000","last_clean":"2024-06-02T23:09:39.786117+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:09:39.786117+0000","last_undegraded":"2024-06-02T23:09:39.786117+0000","last_fullsized":"2024-06-02T23:09:39.786117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788799+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:09:39.788799+0000","last_peered":"2024-06-02T23:09:39.788799+0000","last_clean":"2024-06-02T23:09:39.788799+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:09:39.788799+0000","last_undegraded":"2024-06-02T23:09:39.788799+0000","last_fullsized":"2024-06-02T23:09:39.788799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791625+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:09:39.791625+0000","last_peered":"2024-06-02T23:09:39.791625+0000","last_clean":"2024-06-02T23:09:39.791625+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:09:39.791625+0000","last_undegraded":"2024-06-02T23:09:39.791625+0000","last_fullsized":"2024-06-02T23:09:39.791625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800433+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:09:39.800433+0000","last_peered":"2024-06-02T23:09:39.800433+0000","last_clean":"2024-06-02T23:09:39.800433+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:09:39.800433+0000","last_undegraded":"2024-06-02T23:09:39.800433+0000","last_fullsized":"2024-06-02T23:09:39.800433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802531+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:09:39.802531+0000","last_peered":"2024-06-02T23:09:39.802531+0000","last_clean":"2024-06-02T23:09:39.802531+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:09:39.802531+0000","last_undegraded":"2024-06-02T23:09:39.802531+0000","last_fullsized":"2024-06-02T23:09:39.802531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.803680+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:09:39.803680+0000","last_peered":"2024-06-02T23:09:39.803680+0000","last_clean":"2024-06-02T23:09:39.803680+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:09:39.803680+0000","last_undegraded":"2024-06-02T23:09:39.803680+0000","last_fullsized":"2024-06-02T23:09:39.803680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790081+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:09:39.790081+0000","last_peered":"2024-06-02T23:09:39.790081+0000","last_clean":"2024-06-02T23:09:39.790081+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:09:39.790081+0000","last_undegraded":"2024-06-02T23:09:39.790081+0000","last_fullsized":"2024-06-02T23:09:39.790081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":227,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:55.121314+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:10:55.121314+0000","last_peered":"2024-06-02T23:10:55.121314+0000","last_clean":"2024-06-02T23:10:55.121314+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:10:55.121314+0000","last_undegraded":"2024-06-02T23:10:55.121314+0000","last_fullsized":"2024-06-02T23:10:55.121314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":186,"num_read_kb":186,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788800+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:09:39.788800+0000","last_peered":"2024-06-02T23:09:39.788800+0000","last_clean":"2024-06-02T23:09:39.788800+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:09:39.788800+0000","last_undegraded":"2024-06-02T23:09:39.788800+0000","last_fullsized":"2024-06-02T23:09:39.788800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794320+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:09:39.794320+0000","last_peered":"2024-06-02T23:09:39.794320+0000","last_clean":"2024-06-02T23:09:39.794320+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:09:39.794320+0000","last_undegraded":"2024-06-02T23:09:39.794320+0000","last_fullsized":"2024-06-02T23:09:39.794320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215945+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:14.215945+0000","last_peered":"2024-06-02T23:08:14.215945+0000","last_clean":"2024-06-02T23:08:14.215945+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:14.215945+0000","last_undegraded":"2024-06-02T23:08:14.215945+0000","last_fullsized":"2024-06-02T23:08:14.215945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802474+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:09:39.802474+0000","last_peered":"2024-06-02T23:09:39.802474+0000","last_clean":"2024-06-02T23:09:39.802474+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:09:39.802474+0000","last_undegraded":"2024-06-02T23:09:39.802474+0000","last_fullsized":"2024-06-02T23:09:39.802474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335772+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335772+0000","last_peered":"2024-06-02T23:08:11.335772+0000","last_clean":"2024-06-02T23:08:11.335772+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335772+0000","last_undegraded":"2024-06-02T23:08:11.335772+0000","last_fullsized":"2024-06-02T23:08:11.335772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800976+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:09:39.800976+0000","last_peered":"2024-06-02T23:09:39.800976+0000","last_clean":"2024-06-02T23:09:39.800976+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:09:39.800976+0000","last_undegraded":"2024-06-02T23:09:39.800976+0000","last_fullsized":"2024-06-02T23:09:39.800976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":235,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:57.687377+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:10:57.687377+0000","last_peered":"2024-06-02T23:10:57.687377+0000","last_clean":"2024-06-02T23:10:57.687377+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:10:57.687377+0000","last_undegraded":"2024-06-02T23:10:57.687377+0000","last_fullsized":"2024-06-02T23:10:57.687377+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":187,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771608+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:09:39.771608+0000","last_peered":"2024-06-02T23:09:39.771608+0000","last_clean":"2024-06-02T23:09:39.771608+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:09:39.771608+0000","last_undegraded":"2024-06-02T23:09:39.771608+0000","last_fullsized":"2024-06-02T23:09:39.771608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449603+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449603+0000","last_peered":"2024-06-02T23:08:11.449603+0000","last_clean":"2024-06-02T23:08:11.449603+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449603+0000","last_undegraded":"2024-06-02T23:08:11.449603+0000","last_fullsized":"2024-06-02T23:08:11.449603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795377+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:09:39.795377+0000","last_peered":"2024-06-02T23:09:39.795377+0000","last_clean":"2024-06-02T23:09:39.795377+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:09:39.795377+0000","last_undegraded":"2024-06-02T23:09:39.795377+0000","last_fullsized":"2024-06-02T23:09:39.795377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332946+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332946+0000","last_peered":"2024-06-02T23:08:11.332946+0000","last_clean":"2024-06-02T23:08:11.332946+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332946+0000","last_undegraded":"2024-06-02T23:08:11.332946+0000","last_fullsized":"2024-06-02T23:08:11.332946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785226+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:09:39.785226+0000","last_peered":"2024-06-02T23:09:39.785226+0000","last_clean":"2024-06-02T23:09:39.785226+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:09:39.785226+0000","last_undegraded":"2024-06-02T23:09:39.785226+0000","last_fullsized":"2024-06-02T23:09:39.785226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329286+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329286+0000","last_peered":"2024-06-02T23:08:11.329286+0000","last_clean":"2024-06-02T23:08:11.329286+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329286+0000","last_undegraded":"2024-06-02T23:08:11.329286+0000","last_fullsized":"2024-06-02T23:08:11.329286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.779022+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:09:39.779022+0000","last_peered":"2024-06-02T23:09:39.779022+0000","last_clean":"2024-06-02T23:09:39.779022+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:09:39.779022+0000","last_undegraded":"2024-06-02T23:09:39.779022+0000","last_fullsized":"2024-06-02T23:09:39.779022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":405,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:56.684971+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:10:56.684971+0000","last_peered":"2024-06-02T23:10:56.684971+0000","last_clean":"2024-06-02T23:10:56.684971+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:10:56.684971+0000","last_undegraded":"2024-06-02T23:10:56.684971+0000","last_fullsized":"2024-06-02T23:10:56.684971+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":371,"num_read_kb":371,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802707+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:09:39.802707+0000","last_peered":"2024-06-02T23:09:39.802707+0000","last_clean":"2024-06-02T23:09:39.802707+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:09:39.802707+0000","last_undegraded":"2024-06-02T23:09:39.802707+0000","last_fullsized":"2024-06-02T23:09:39.802707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791331+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:09:39.791331+0000","last_peered":"2024-06-02T23:09:39.791331+0000","last_clean":"2024-06-02T23:09:39.791331+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:09:39.791331+0000","last_undegraded":"2024-06-02T23:09:39.791331+0000","last_fullsized":"2024-06-02T23:09:39.791331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330069+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.330069+0000","last_peered":"2024-06-02T23:08:11.330069+0000","last_clean":"2024-06-02T23:08:11.330069+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.330069+0000","last_undegraded":"2024-06-02T23:08:11.330069+0000","last_fullsized":"2024-06-02T23:08:11.330069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801687+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:09:39.801687+0000","last_peered":"2024-06-02T23:09:39.801687+0000","last_clean":"2024-06-02T23:09:39.801687+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:09:39.801687+0000","last_undegraded":"2024-06-02T23:09:39.801687+0000","last_fullsized":"2024-06-02T23:09:39.801687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":446,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:56.685703+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:10:56.685703+0000","last_peered":"2024-06-02T23:10:56.685703+0000","last_clean":"2024-06-02T23:10:56.685703+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:10:56.685703+0000","last_undegraded":"2024-06-02T23:10:56.685703+0000","last_fullsized":"2024-06-02T23:10:56.685703+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":382,"num_read_kb":255,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794578+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:09:39.794578+0000","last_peered":"2024-06-02T23:09:39.794578+0000","last_clean":"2024-06-02T23:09:39.794578+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:09:39.794578+0000","last_undegraded":"2024-06-02T23:09:39.794578+0000","last_fullsized":"2024-06-02T23:09:39.794578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332605+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332605+0000","last_peered":"2024-06-02T23:08:11.332605+0000","last_clean":"2024-06-02T23:08:11.332605+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332605+0000","last_undegraded":"2024-06-02T23:08:11.332605+0000","last_fullsized":"2024-06-02T23:08:11.332605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.778013+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:09:39.778013+0000","last_peered":"2024-06-02T23:09:39.778013+0000","last_clean":"2024-06-02T23:09:39.778013+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:09:39.778013+0000","last_undegraded":"2024-06-02T23:09:39.778013+0000","last_fullsized":"2024-06-02T23:09:39.778013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329465+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329465+0000","last_peered":"2024-06-02T23:08:11.329465+0000","last_clean":"2024-06-02T23:08:11.329465+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329465+0000","last_undegraded":"2024-06-02T23:08:11.329465+0000","last_fullsized":"2024-06-02T23:08:11.329465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792897+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:09:39.792897+0000","last_peered":"2024-06-02T23:09:39.792897+0000","last_clean":"2024-06-02T23:09:39.792897+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:09:39.792897+0000","last_undegraded":"2024-06-02T23:09:39.792897+0000","last_fullsized":"2024-06-02T23:09:39.792897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":220,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.173048+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:10:54.173048+0000","last_peered":"2024-06-02T23:10:54.173048+0000","last_clean":"2024-06-02T23:10:54.173048+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:10:54.173048+0000","last_undegraded":"2024-06-02T23:10:54.173048+0000","last_fullsized":"2024-06-02T23:10:54.173048+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786680+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:09:39.786680+0000","last_peered":"2024-06-02T23:09:39.786680+0000","last_clean":"2024-06-02T23:09:39.786680+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:09:39.786680+0000","last_undegraded":"2024-06-02T23:09:39.786680+0000","last_fullsized":"2024-06-02T23:09:39.786680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215918+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:14.215918+0000","last_peered":"2024-06-02T23:08:14.215918+0000","last_clean":"2024-06-02T23:08:14.215918+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:14.215918+0000","last_undegraded":"2024-06-02T23:08:14.215918+0000","last_fullsized":"2024-06-02T23:08:14.215918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800649+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:09:39.800649+0000","last_peered":"2024-06-02T23:09:39.800649+0000","last_clean":"2024-06-02T23:09:39.800649+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:09:39.800649+0000","last_undegraded":"2024-06-02T23:09:39.800649+0000","last_fullsized":"2024-06-02T23:09:39.800649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:45.676951+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:09:45.676951+0000","last_peered":"2024-06-02T23:09:45.676951+0000","last_clean":"2024-06-02T23:09:45.676951+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:09:45.676951+0000","last_undegraded":"2024-06-02T23:09:45.676951+0000","last_fullsized":"2024-06-02T23:09:45.676951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.783215+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:09:39.783215+0000","last_peered":"2024-06-02T23:09:39.783215+0000","last_clean":"2024-06-02T23:09:39.783215+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:09:39.783215+0000","last_undegraded":"2024-06-02T23:09:39.783215+0000","last_fullsized":"2024-06-02T23:09:39.783215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328539+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328539+0000","last_peered":"2024-06-02T23:08:11.328539+0000","last_clean":"2024-06-02T23:08:11.328539+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328539+0000","last_undegraded":"2024-06-02T23:08:11.328539+0000","last_fullsized":"2024-06-02T23:08:11.328539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785214+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:09:39.785214+0000","last_peered":"2024-06-02T23:09:39.785214+0000","last_clean":"2024-06-02T23:09:39.785214+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:09:39.785214+0000","last_undegraded":"2024-06-02T23:09:39.785214+0000","last_fullsized":"2024-06-02T23:09:39.785214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":219,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.120458+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:10:54.120458+0000","last_peered":"2024-06-02T23:10:54.120458+0000","last_clean":"2024-06-02T23:10:54.120458+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:10:54.120458+0000","last_undegraded":"2024-06-02T23:10:54.120458+0000","last_fullsized":"2024-06-02T23:10:54.120458+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":186,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791877+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:09:39.791877+0000","last_peered":"2024-06-02T23:09:39.791877+0000","last_clean":"2024-06-02T23:09:39.791877+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:09:39.791877+0000","last_undegraded":"2024-06-02T23:09:39.791877+0000","last_fullsized":"2024-06-02T23:09:39.791877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792620+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:09:39.792620+0000","last_peered":"2024-06-02T23:09:39.792620+0000","last_clean":"2024-06-02T23:09:39.792620+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:09:39.792620+0000","last_undegraded":"2024-06-02T23:09:39.792620+0000","last_fullsized":"2024-06-02T23:09:39.792620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":222,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:58.179159+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:10:58.179159+0000","last_peered":"2024-06-02T23:10:58.179159+0000","last_clean":"2024-06-02T23:10:58.179159+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:10:58.179159+0000","last_undegraded":"2024-06-02T23:10:58.179159+0000","last_fullsized":"2024-06-02T23:10:58.179159+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":189,"num_read_kb":189,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794336+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:09:39.794336+0000","last_peered":"2024-06-02T23:09:39.794336+0000","last_clean":"2024-06-02T23:09:39.794336+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:09:39.794336+0000","last_undegraded":"2024-06-02T23:09:39.794336+0000","last_fullsized":"2024-06-02T23:09:39.794336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:45.814046+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:45.814046+0000","last_peered":"2024-06-02T23:08:45.814046+0000","last_clean":"2024-06-02T23:08:45.814046+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:45.814046+0000","last_undegraded":"2024-06-02T23:08:45.814046+0000","last_fullsized":"2024-06-02T23:08:45.814046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771357+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:09:39.771357+0000","last_peered":"2024-06-02T23:09:39.771357+0000","last_clean":"2024-06-02T23:09:39.771357+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:09:39.771357+0000","last_undegraded":"2024-06-02T23:09:39.771357+0000","last_fullsized":"2024-06-02T23:09:39.771357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788062+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:09:39.788062+0000","last_peered":"2024-06-02T23:09:39.788062+0000","last_clean":"2024-06-02T23:09:39.788062+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:09:39.788062+0000","last_undegraded":"2024-06-02T23:09:39.788062+0000","last_fullsized":"2024-06-02T23:09:39.788062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'19","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.236340+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:10:54.236340+0000","last_peered":"2024-06-02T23:10:54.236340+0000","last_clean":"2024-06-02T23:10:54.236340+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:10:54.236340+0000","last_undegraded":"2024-06-02T23:10:54.236340+0000","last_fullsized":"2024-06-02T23:10:54.236340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790001+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:09:39.790001+0000","last_peered":"2024-06-02T23:09:39.790001+0000","last_clean":"2024-06-02T23:09:39.790001+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:09:39.790001+0000","last_undegraded":"2024-06-02T23:09:39.790001+0000","last_fullsized":"2024-06-02T23:09:39.790001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450463+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450463+0000","last_peered":"2024-06-02T23:08:11.450463+0000","last_clean":"2024-06-02T23:08:11.450463+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450463+0000","last_undegraded":"2024-06-02T23:08:11.450463+0000","last_fullsized":"2024-06-02T23:08:11.450463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792568+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:09:39.792568+0000","last_peered":"2024-06-02T23:09:39.792568+0000","last_clean":"2024-06-02T23:09:39.792568+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:09:39.792568+0000","last_undegraded":"2024-06-02T23:09:39.792568+0000","last_fullsized":"2024-06-02T23:09:39.792568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334494+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334494+0000","last_peered":"2024-06-02T23:08:11.334494+0000","last_clean":"2024-06-02T23:08:11.334494+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334494+0000","last_undegraded":"2024-06-02T23:08:11.334494+0000","last_fullsized":"2024-06-02T23:08:11.334494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782299+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:09:39.782299+0000","last_peered":"2024-06-02T23:09:39.782299+0000","last_clean":"2024-06-02T23:09:39.782299+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:09:39.782299+0000","last_undegraded":"2024-06-02T23:09:39.782299+0000","last_fullsized":"2024-06-02T23:09:39.782299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748925+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748925+0000","last_peered":"2024-06-02T23:08:11.748925+0000","last_clean":"2024-06-02T23:08:11.748925+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748925+0000","last_undegraded":"2024-06-02T23:08:11.748925+0000","last_fullsized":"2024-06-02T23:08:11.748925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792513+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:09:39.792513+0000","last_peered":"2024-06-02T23:09:39.792513+0000","last_clean":"2024-06-02T23:09:39.792513+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:09:39.792513+0000","last_undegraded":"2024-06-02T23:09:39.792513+0000","last_fullsized":"2024-06-02T23:09:39.792513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.275353+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:10:54.275353+0000","last_peered":"2024-06-02T23:10:54.275353+0000","last_clean":"2024-06-02T23:10:54.275353+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:10:54.275353+0000","last_undegraded":"2024-06-02T23:10:54.275353+0000","last_fullsized":"2024-06-02T23:10:54.275353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786323+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:09:39.786323+0000","last_peered":"2024-06-02T23:09:39.786323+0000","last_clean":"2024-06-02T23:09:39.786323+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:09:39.786323+0000","last_undegraded":"2024-06-02T23:09:39.786323+0000","last_fullsized":"2024-06-02T23:09:39.786323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334806+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334806+0000","last_peered":"2024-06-02T23:08:11.334806+0000","last_clean":"2024-06-02T23:08:11.334806+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334806+0000","last_undegraded":"2024-06-02T23:08:11.334806+0000","last_fullsized":"2024-06-02T23:08:11.334806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797923+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:09:39.797923+0000","last_peered":"2024-06-02T23:09:39.797923+0000","last_clean":"2024-06-02T23:09:39.797923+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:09:39.797923+0000","last_undegraded":"2024-06-02T23:09:39.797923+0000","last_fullsized":"2024-06-02T23:09:39.797923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":228,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:58.251201+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:10:58.251201+0000","last_peered":"2024-06-02T23:10:58.251201+0000","last_clean":"2024-06-02T23:10:58.251201+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:10:58.251201+0000","last_undegraded":"2024-06-02T23:10:58.251201+0000","last_fullsized":"2024-06-02T23:10:58.251201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2193,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797984+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:09:39.797984+0000","last_peered":"2024-06-02T23:09:39.797984+0000","last_clean":"2024-06-02T23:09:39.797984+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:09:39.797984+0000","last_undegraded":"2024-06-02T23:09:39.797984+0000","last_fullsized":"2024-06-02T23:09:39.797984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330301+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330301+0000","last_peered":"2024-06-02T23:08:11.330301+0000","last_clean":"2024-06-02T23:08:11.330301+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330301+0000","last_undegraded":"2024-06-02T23:08:11.330301+0000","last_fullsized":"2024-06-02T23:08:11.330301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789936+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:09:39.789936+0000","last_peered":"2024-06-02T23:09:39.789936+0000","last_clean":"2024-06-02T23:09:39.789936+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:09:39.789936+0000","last_undegraded":"2024-06-02T23:09:39.789936+0000","last_fullsized":"2024-06-02T23:09:39.789936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333595+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333595+0000","last_peered":"2024-06-02T23:08:11.333595+0000","last_clean":"2024-06-02T23:08:11.333595+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333595+0000","last_undegraded":"2024-06-02T23:08:11.333595+0000","last_fullsized":"2024-06-02T23:08:11.333595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795458+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:09:39.795458+0000","last_peered":"2024-06-02T23:09:39.795458+0000","last_clean":"2024-06-02T23:09:39.795458+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:09:39.795458+0000","last_undegraded":"2024-06-02T23:09:39.795458+0000","last_fullsized":"2024-06-02T23:09:39.795458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790898+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:09:39.790898+0000","last_peered":"2024-06-02T23:09:39.790898+0000","last_clean":"2024-06-02T23:09:39.790898+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:09:39.790898+0000","last_undegraded":"2024-06-02T23:09:39.790898+0000","last_fullsized":"2024-06-02T23:09:39.790898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785496+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:09:39.785496+0000","last_peered":"2024-06-02T23:09:39.785496+0000","last_clean":"2024-06-02T23:09:39.785496+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:09:39.785496+0000","last_undegraded":"2024-06-02T23:09:39.785496+0000","last_fullsized":"2024-06-02T23:09:39.785496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785576+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:09:39.785576+0000","last_peered":"2024-06-02T23:09:39.785576+0000","last_clean":"2024-06-02T23:09:39.785576+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:09:39.785576+0000","last_undegraded":"2024-06-02T23:09:39.785576+0000","last_fullsized":"2024-06-02T23:09:39.785576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791535+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:09:39.791535+0000","last_peered":"2024-06-02T23:09:39.791535+0000","last_clean":"2024-06-02T23:09:39.791535+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:09:39.791535+0000","last_undegraded":"2024-06-02T23:09:39.791535+0000","last_fullsized":"2024-06-02T23:09:39.791535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792313+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:09:39.792313+0000","last_peered":"2024-06-02T23:09:39.792313+0000","last_clean":"2024-06-02T23:09:39.792313+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:09:39.792313+0000","last_undegraded":"2024-06-02T23:09:39.792313+0000","last_fullsized":"2024-06-02T23:09:39.792313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.774868+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:09:39.774868+0000","last_peered":"2024-06-02T23:09:39.774868+0000","last_clean":"2024-06-02T23:09:39.774868+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:09:39.774868+0000","last_undegraded":"2024-06-02T23:09:39.774868+0000","last_fullsized":"2024-06-02T23:09:39.774868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794273+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:09:39.794273+0000","last_peered":"2024-06-02T23:09:39.794273+0000","last_clean":"2024-06-02T23:09:39.794273+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:09:39.794273+0000","last_undegraded":"2024-06-02T23:09:39.794273+0000","last_fullsized":"2024-06-02T23:09:39.794273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788292+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:09:39.788292+0000","last_peered":"2024-06-02T23:09:39.788292+0000","last_clean":"2024-06-02T23:09:39.788292+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:09:39.788292+0000","last_undegraded":"2024-06-02T23:09:39.788292+0000","last_fullsized":"2024-06-02T23:09:39.788292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786749+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:09:39.786749+0000","last_peered":"2024-06-02T23:09:39.786749+0000","last_clean":"2024-06-02T23:09:39.786749+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:09:39.786749+0000","last_undegraded":"2024-06-02T23:09:39.786749+0000","last_fullsized":"2024-06-02T23:09:39.786749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789933+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:09:39.789933+0000","last_peered":"2024-06-02T23:09:39.789933+0000","last_clean":"2024-06-02T23:09:39.789933+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:09:39.789933+0000","last_undegraded":"2024-06-02T23:09:39.789933+0000","last_fullsized":"2024-06-02T23:09:39.789933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800745+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:09:39.800745+0000","last_peered":"2024-06-02T23:09:39.800745+0000","last_clean":"2024-06-02T23:09:39.800745+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:09:39.800745+0000","last_undegraded":"2024-06-02T23:09:39.800745+0000","last_fullsized":"2024-06-02T23:09:39.800745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792409+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:09:39.792409+0000","last_peered":"2024-06-02T23:09:39.792409+0000","last_clean":"2024-06-02T23:09:39.792409+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:09:39.792409+0000","last_undegraded":"2024-06-02T23:09:39.792409+0000","last_fullsized":"2024-06-02T23:09:39.792409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.776917+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:09:39.776917+0000","last_peered":"2024-06-02T23:09:39.776917+0000","last_clean":"2024-06-02T23:09:39.776917+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:09:39.776917+0000","last_undegraded":"2024-06-02T23:09:39.776917+0000","last_fullsized":"2024-06-02T23:09:39.776917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1901,"num_read_kb":1034,"num_write":273,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":227403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383739,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645375,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1138821,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808900,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332175,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074331,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495700,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":548352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738481,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":482429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":548481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182499,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920861,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757337,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1073025,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:11:01.410 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:11:01.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:01 smithi123 ceph-mon[35977]: pgmap v405: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:01.692 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:02 smithi174 ceph-mon[35994]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:02 smithi174 ceph-mon[35994]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:02 smithi174 ceph-mon[35994]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:02 smithi052 ceph-mon[32340]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:02 smithi052 ceph-mon[32340]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:02 smithi052 ceph-mon[32340]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:02 smithi123 ceph-mon[35977]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:02 smithi123 ceph-mon[35977]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:02.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:02 smithi123 ceph-mon[35977]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:03.383 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:03 smithi052 ceph-mon[32340]: pgmap v406: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:03 smithi174 ceph-mon[35994]: pgmap v406: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:03.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 -- 172.21.15.52:0/319945277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 msgr2=0x7f41e0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:03.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 --2- 172.21.15.52:0/319945277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0107110 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f41d40099c0 tx=0x7f41d4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 -- 172.21.15.52:0/319945277 shutdown_connections 2024-06-02T23:11:03.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 --2- 172.21.15.52:0/319945277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41e0107840 0x7f41e010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 --2- 172.21.15.52:0/319945277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e0102740 0x7f41e0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 --2- 172.21.15.52:0/319945277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0107110 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 -- 172.21.15.52:0/319945277 >> 172.21.15.52:0/319945277 conn(0x7f41e00795f0 msgr2=0x7f41e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:03.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 -- 172.21.15.52:0/319945277 shutdown_connections 2024-06-02T23:11:03.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 -- 172.21.15.52:0/319945277 wait complete. 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.399+0000 7f41e587c700 1 Processor -- start 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 -- start start 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e0102740 0x7f41e0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41e0107840 0x7f41e0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f41e0112620 con 0x7f41e01030f0 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41e01124a0 con 0x7f41e0102740 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41e587c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41e01127a0 con 0x7f41e0107840 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38796/0 (socket says 172.21.15.52:38796) 2024-06-02T23:11:03.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 -- 172.21.15.52:0/2865468056 learned_addr learned my addr 172.21.15.52:0/2865468056 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41df7fe700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41e0107840 0x7f41e0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41deffd700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e0102740 0x7f41e0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 -- 172.21.15.52:0/2865468056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41e0107840 msgr2=0x7f41e0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41e0107840 0x7f41e0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 -- 172.21.15.52:0/2865468056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e0102740 msgr2=0x7f41e0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e0102740 0x7f41e0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.400+0000 7f41de7fc700 1 -- 172.21.15.52:0/2865468056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41e01a7000 con 0x7f41e01030f0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.401+0000 7f41de7fc700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0074cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41d4006010 tx=0x7f41d4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.401+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4027070 con 0x7f41e01030f0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.401+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f41d4003f50 con 0x7f41e01030f0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.401+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41e01a7290 con 0x7f41e01030f0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.401+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41d4022cd0 con 0x7f41e01030f0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.401+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41e01a76e0 con 0x7f41e01030f0 2024-06-02T23:11:03.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.402+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41e0069180 con 0x7f41e01030f0 2024-06-02T23:11:03.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.403+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f41d402a070 con 0x7f41e01030f0 2024-06-02T23:11:03.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.404+0000 7f41e487a700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41c80a2c20 0x7f41c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:03.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.404+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f41d40d0170 con 0x7f41e01030f0 2024-06-02T23:11:03.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.404+0000 7f41deffd700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41c80a2c20 0x7f41c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:03.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.404+0000 7f41deffd700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41c80a2c20 0x7f41c80a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41d0005950 tx=0x7f41d00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:03.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.407+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f41d4031080 con 0x7f41e01030f0 2024-06-02T23:11:03.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:03 smithi123 ceph-mon[35977]: pgmap v406: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:03.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.659+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41e002dd40 con 0x7f41e01030f0 2024-06-02T23:11:03.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.660+0000 7f41e487a700 1 -- 172.21.15.52:0/2865468056 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f41d4013070 con 0x7f41e01030f0 2024-06-02T23:11:03.660 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:03.661 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.662+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41c80a2c20 msgr2=0x7f41c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.662+0000 7f41e587c700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41c80a2c20 0x7f41c80a50e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f41d0005950 tx=0x7f41d00058e0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.662+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 msgr2=0x7f41e0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.662+0000 7f41e587c700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0074cd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f41d4006010 tx=0x7f41d4009f80 comp rx=0 tx=0).stop 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 shutdown_connections 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41c80a2c20 0x7f41c80a50e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41e0107840 0x7f41e0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41e0102740 0x7f41e0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 --2- 172.21.15.52:0/2865468056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f41e01030f0 0x7f41e0074cd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 >> 172.21.15.52:0/2865468056 conn(0x7f41e00795f0 msgr2=0x7f41e0108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:03.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 shutdown_connections 2024-06-02T23:11:03.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:03.663+0000 7f41e587c700 1 -- 172.21.15.52:0/2865468056 wait complete. 2024-06-02T23:11:03.664 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:11:04.310 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:11:04.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:04 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2865468056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:04.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:04 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2865468056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:04.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:04 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2865468056' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:04.564 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:05 smithi174 ceph-mon[35994]: pgmap v407: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:11:05.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:05 smithi052 ceph-mon[32340]: pgmap v407: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:11:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:05 smithi123 ceph-mon[35977]: pgmap v407: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 -- 172.21.15.52:0/93678490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8101390 msgr2=0x7f8ce8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 --2- 172.21.15.52:0/93678490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8101390 0x7f8ce8101810 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc009990 tx=0x7f8cdc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 -- 172.21.15.52:0/93678490 shutdown_connections 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 --2- 172.21.15.52:0/93678490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 0x7f8ce810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 --2- 172.21.15.52:0/93678490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce8107400 0x7f8ce81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 --2- 172.21.15.52:0/93678490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8101390 0x7f8ce8101810 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 -- 172.21.15.52:0/93678490 >> 172.21.15.52:0/93678490 conn(0x7f8ce8078340 msgr2=0x7f8ce8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:06.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 -- 172.21.15.52:0/93678490 shutdown_connections 2024-06-02T23:11:06.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.320+0000 7f8cee90d700 1 -- 172.21.15.52:0/93678490 wait complete. 2024-06-02T23:11:06.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.321+0000 7f8cee90d700 1 Processor -- start 2024-06-02T23:11:06.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.321+0000 7f8cee90d700 1 -- start start 2024-06-02T23:11:06.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8cee90d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce8101390 0x7f8ce819d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8cee90d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 0x7f8ce819dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8cee90d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 0x7f8ce8197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8cee90d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce81115a0 con 0x7f8ce8107400 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8cee90d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8111420 con 0x7f8ce8101d50 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8cee90d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce8111720 con 0x7f8ce8101390 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 0x7f8ce8197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 0x7f8ce8197940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55548/0 (socket says 172.21.15.52:55548) 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 -- 172.21.15.52:0/1758244297 learned_addr learned my addr 172.21.15.52:0/1758244297 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ce77fe700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 0x7f8ce819dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ce7fff700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce8101390 0x7f8ce819d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 -- 172.21.15.52:0/1758244297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce8101390 msgr2=0x7f8ce819d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce8101390 0x7f8ce819d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 -- 172.21.15.52:0/1758244297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 msgr2=0x7f8ce819dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 0x7f8ce819dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 -- 172.21.15.52:0/1758244297 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce81980a0 con 0x7f8ce8107400 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ce77fe700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 0x7f8ce819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ceceaa700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 0x7f8ce8197940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000cfb0 tx=0x7f8ce000e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.322+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000c2e0 con 0x7f8ce8107400 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.323+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ce000a600 con 0x7f8ce8107400 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.323+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce8198390 con 0x7f8ce8107400 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.323+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce000c480 con 0x7f8ce8107400 2024-06-02T23:11:06.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.323+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce806a170 con 0x7f8ce8107400 2024-06-02T23:11:06.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.324+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce8069180 con 0x7f8ce8107400 2024-06-02T23:11:06.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.325+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ce000ea40 con 0x7f8ce8107400 2024-06-02T23:11:06.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.325+0000 7f8ce57fa700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8cd00a2c20 0x7f8cd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:06.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.326+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8ce00d0cf0 con 0x7f8ce8107400 2024-06-02T23:11:06.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.326+0000 7f8ce7fff700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8cd00a2c20 0x7f8cd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:06.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.326+0000 7f8ce7fff700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8cd00a2c20 0x7f8cd00a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006fd0 tx=0x7f8cd8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:06.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.330+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8ce009dc20 con 0x7f8ce8107400 2024-06-02T23:11:06.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.539+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ce806aba0 con 0x7f8ce8107400 2024-06-02T23:11:06.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.541+0000 7f8ce57fa700 1 -- 172.21.15.52:0/1758244297 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7f8ce009d520 con 0x7f8ce8107400 2024-06-02T23:11:06.542 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:06.542 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:11:06.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8cd00a2c20 msgr2=0x7f8cd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8cd00a2c20 0x7f8cd00a50e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8cd8006fd0 tx=0x7f8cd8008040 comp rx=0 tx=0).stop 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 msgr2=0x7f8ce8197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 0x7f8ce8197940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8ce000cfb0 tx=0x7f8ce000e330 comp rx=0 tx=0).stop 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 shutdown_connections 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8cd00a2c20 0x7f8cd00a50e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ce8107400 0x7f8ce8197940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ce8101390 0x7f8ce819d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 --2- 172.21.15.52:0/1758244297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ce8101d50 0x7f8ce819dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 >> 172.21.15.52:0/1758244297 conn(0x7f8ce8078340 msgr2=0x7f8ce8100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 shutdown_connections 2024-06-02T23:11:06.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:06.543+0000 7f8cee90d700 1 -- 172.21.15.52:0/1758244297 wait complete. 2024-06-02T23:11:07.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:07 smithi052 ceph-mon[32340]: pgmap v408: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:11:07.224 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:07 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1758244297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:11:07.255 DEBUG:tasks.fwd_scrub:Yielding 2024-06-02T23:11:07.255 INFO:teuthology.run_tasks:Running task background_exec... 2024-06-02T23:11:07.267 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:11:07.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-06-02T23:11:07.268 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-06-02T23:11:07.268 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:11:07.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:11:07.298 INFO:teuthology.run_tasks:Running task quiescer... 2024-06-02T23:11:07.311 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-06-02T23:11:07.311 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:11:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:07 smithi174 ceph-mon[35994]: pgmap v408: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:11:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:07 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1758244297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:11:07.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:07 smithi123 ceph-mon[35977]: pgmap v408: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-06-02T23:11:07.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:07 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1758244297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:11:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:09 smithi174 ceph-mon[35994]: pgmap v409: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-06-02T23:11:09.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:09 smithi052 ceph-mon[32340]: pgmap v409: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-06-02T23:11:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:09 smithi123 ceph-mon[35977]: pgmap v409: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.1 KiB/s wr, 12 op/s 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 -- 172.21.15.52:0/1067275370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa27806b620 msgr2=0x7fa27810f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 --2- 172.21.15.52:0/1067275370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa27806b620 0x7fa27810f350 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b0d0 tx=0x7fa27000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 -- 172.21.15.52:0/1067275370 shutdown_connections 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 --2- 172.21.15.52:0/1067275370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa27806b620 0x7fa27810f350 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 --2- 172.21.15.52:0/1067275370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa278108580 0x7fa278108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 --2- 172.21.15.52:0/1067275370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa27806ac60 0x7fa27806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 -- 172.21.15.52:0/1067275370 >> 172.21.15.52:0/1067275370 conn(0x7fa278078430 msgr2=0x7fa278078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 -- 172.21.15.52:0/1067275370 shutdown_connections 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.896+0000 7fa27ec40700 1 -- 172.21.15.52:0/1067275370 wait complete. 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 Processor -- start 2024-06-02T23:11:09.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 -- start start 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa27806ac60 0x7fa278101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa27806b620 0x7fa278101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 0x7fa278101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa278112670 con 0x7fa278108580 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781124f0 con 0x7fa27806ac60 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27ec40700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781127f0 con 0x7fa27806b620 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27d1dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 0x7fa278101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27d1dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 0x7fa278101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55566/0 (socket says 172.21.15.52:55566) 2024-06-02T23:11:09.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27d1dd700 1 -- 172.21.15.52:0/2936504095 learned_addr learned my addr 172.21.15.52:0/2936504095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa27806ac60 0x7fa278101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa277fff700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa27806b620 0x7fa278101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27d1dd700 1 -- 172.21.15.52:0/2936504095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa27806b620 msgr2=0x7fa278101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa27806b620 0x7fa278101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.897+0000 7fa27d1dd700 1 -- 172.21.15.52:0/2936504095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa27806ac60 msgr2=0x7fa278101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa27806ac60 0x7fa278101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa27d1dd700 1 -- 172.21.15.52:0/2936504095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2781a7030 con 0x7fa278108580 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 0x7fa278101e80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa2700095c0 tx=0x7fa270007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa275ffb700 1 -- 172.21.15.52:0/2936504095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27001f070 con 0x7fa278108580 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa275ffb700 1 -- 172.21.15.52:0/2936504095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa270007e20 con 0x7fa278108580 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2781a72c0 con 0x7fa278108580 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa275ffb700 1 -- 172.21.15.52:0/2936504095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27001b4a0 con 0x7fa278108580 2024-06-02T23:11:09.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.898+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2781a7710 con 0x7fa278108580 2024-06-02T23:11:09.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.900+0000 7fa275ffb700 1 -- 172.21.15.52:0/2936504095 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa270022070 con 0x7fa278108580 2024-06-02T23:11:09.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.901+0000 7fa275ffb700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa2600a2c20 0x7fa2600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.902+0000 7fa275ffb700 1 -- 172.21.15.52:0/2936504095 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa27000f020 con 0x7fa278108580 2024-06-02T23:11:09.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.902+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa2600a2c20 0x7fa2600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.902+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa2600a2c20 0x7fa2600a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa268005950 tx=0x7fa26800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.902+0000 7fa2894c3b80 1 Processor -- start 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.902+0000 7fa2894c3b80 1 -- start start 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fdfe0 0x560cdd8fe3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd8f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560cdd7eafe0 con 0x560cdd8f8820 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560cdd8498d0 con 0x560cdd8fe900 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560cdd7b14e0 con 0x560cdd8fdfe0 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55570/0 (socket says 172.21.15.52:55570) 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 -- 172.21.15.52:0/3065253291 learned_addr learned my addr 172.21.15.52:0/3065253291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fdfe0 0x560cdd8fe3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa277fff700 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd8f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 -- 172.21.15.52:0/3065253291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fdfe0 msgr2=0x560cdd8fe3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fdfe0 0x560cdd8fe3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 -- 172.21.15.52:0/3065253291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 msgr2=0x560cdd8f82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd8f82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 -- 172.21.15.52:0/3065253291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdd798950 con 0x560cdd8f8820 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fdfe0 0x560cdd8fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f8c40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2700a5160 tx=0x7fa27009c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc481aae12827463 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa25effd700 1 -- 172.21.15.52:0/3065253291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2700a0e60 con 0x560cdd8f8820 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa25effd700 1 -- 172.21.15.52:0/3065253291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2700cf040 con 0x560cdd8f8820 2024-06-02T23:11:09.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.903+0000 7fa25effd700 1 -- 172.21.15.52:0/3065253291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa27009cd10 con 0x560cdd8f8820 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3065253291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 msgr2=0x560cdd8f8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f8c40 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa2700a5160 tx=0x7fa27009c540 comp rx=0 tx=0).stop 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3065253291 shutdown_connections 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f8c40 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fdfe0 0x560cdd8fe3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3065253291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd8f82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3065253291 >> 172.21.15.52:0/3065253291 conn(0x560cdd825100 msgr2=0x560cdd8f32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3065253291 shutdown_connections 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.904+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3065253291 wait complete. 2024-06-02T23:11:09.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 -- start start 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 Processor -- start 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd904670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd901970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd901eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560cdd7ea040 con 0x560cdd8f8820 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560cdd845890 con 0x560cdd8fdfe0 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560cdd905090 con 0x560cdd8fe900 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa2894c3b80 10 client.0 _ll_register_callbacks cb 0x560cdd7df630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa27c9dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd904670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa27c9dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd904670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55578/0 (socket says 172.21.15.52:55578) 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa27c9dc700 1 -- 172.21.15.52:0/4196591963 learned_addr learned my addr 172.21.15.52:0/4196591963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd901eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.905+0000 7fa277fff700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd901970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa27c9dc700 1 -- 172.21.15.52:0/4196591963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 msgr2=0x560cdd901eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd901eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa27c9dc700 1 -- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 msgr2=0x560cdd901970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd901970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa27c9dc700 1 -- 172.21.15.52:0/4196591963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdd80f7d0 con 0x560cdd8f8820 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd904670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa2680114c0 tx=0x7fa2680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24e4259969706c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa257fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa257fff700 1 -- 172.21.15.52:0/4196591963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa268004a30 con 0x560cdd8f8820 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa257fff700 1 -- 172.21.15.52:0/4196591963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa268004bd0 con 0x560cdd8f8820 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2700cf040 con 0x560cdd8f8820 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa257fff700 1 -- 172.21.15.52:0/4196591963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa268004dd0 con 0x560cdd8f8820 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa2894c3b80 10 client.15294 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.906+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560cdd90f7f0 con 0x560cdd8f8820 2024-06-02T23:11:09.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa257fff700 1 -- 172.21.15.52:0/4196591963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa2700cf040 con 0x560cdd8f8820 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa257fff700 1 -- 172.21.15.52:0/4196591963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa26801f870 con 0x560cdd8f8820 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa2894c3b80 10 client.15294 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa2894c3b80 10 client.15294 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa2894c3b80 20 client.15294 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd911e80 0x560cdd914290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa2894c3b80 4 client.15294 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cdd856dd0 con 0x560cdd911e80 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.907+0000 7fa277fff700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd911e80 0x560cdd914290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa277fff700 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd911e80 0x560cdd914290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c10e3f629891107d server_cookie=41b239d4311d7d44 in_seq=0 out_seq=0 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa257fff700 10 client.15294 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa257fff700 1 -- 172.21.15.52:0/4196591963 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560cdd856dd0 con 0x560cdd911e80 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa257fff700 10 client.15294 handle_command_reply: tid=0 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa2894c3b80 1 client.15294 shutdown 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 msgr2=0x560cdd904670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.908+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd904670 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa2680114c0 tx=0x7fa2680047c0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 shutdown_connections 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd911e80 0x560cdd914290 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd901eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd904670 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/4196591963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd901970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 >> 172.21.15.52:0/4196591963 conn(0x560cdd825100 msgr2=0x560cdd8f2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa257fff700 0 client.15294 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 shutdown_connections 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 1 -- 172.21.15.52:0/4196591963 wait complete. 2024-06-02T23:11:09.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.909+0000 7fa2894c3b80 20 client.15294 trim_cache size 0 max 16384 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 Processor -- start 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 -- start start 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd903a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd8f2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781aec00 con 0x560cdd8f8820 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781b0740 con 0x560cdd8fdfe0 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781af2d0 con 0x560cdd8fe900 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa27c9dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa27c9dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55592/0 (socket says 172.21.15.52:55592) 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa27c9dc700 1 -- 172.21.15.52:0/2269645389 learned_addr learned my addr 172.21.15.52:0/2269645389 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa277fff700 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd903a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.930+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd8f2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa27c9dc700 1 -- 172.21.15.52:0/2269645389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 msgr2=0x560cdd8f2f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd8f2f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa27c9dc700 1 -- 172.21.15.52:0/2269645389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 msgr2=0x560cdd903a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd903a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa27c9dc700 1 -- 172.21.15.52:0/2269645389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cdd8fdae0 con 0x560cdd8f8820 2024-06-02T23:11:09.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f82e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa268000c00 tx=0x7fa26800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626f445c642683ce server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa25e7fc700 1 -- 172.21.15.52:0/2269645389 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26800b870 con 0x560cdd8f8820 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa25e7fc700 1 -- 172.21.15.52:0/2269645389 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa268023070 con 0x560cdd8f8820 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.931+0000 7fa25e7fc700 1 -- 172.21.15.52:0/2269645389 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2680051b0 con 0x560cdd8f8820 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 -- 172.21.15.52:0/2269645389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 msgr2=0x560cdd8f82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f82e0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa268000c00 tx=0x7fa26800b540 comp rx=0 tx=0).stop 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 -- 172.21.15.52:0/2269645389 shutdown_connections 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8fe900 0x560cdd8f2f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8f8820 0x560cdd8f82e0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/2269645389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fdfe0 0x560cdd903a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 -- 172.21.15.52:0/2269645389 >> 172.21.15.52:0/2269645389 conn(0x560cdd825100 msgr2=0x560cdd8ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 -- 172.21.15.52:0/2269645389 shutdown_connections 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 -- 172.21.15.52:0/2269645389 wait complete. 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 -- start start 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.932+0000 7fa2894c3b80 1 Processor -- start 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8f8820 0x560cdd90a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 0x560cdd90a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd9072b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781b0070 con 0x560cdd8fdfe0 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781af9a0 con 0x560cdd8fe900 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa2781acbc0 con 0x560cdd8f8820 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 10 client.0 _ll_register_callbacks cb 0x560cdd7df630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 0x560cdd90a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 0x560cdd90a820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55598/0 (socket says 172.21.15.52:55598) 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 -- 172.21.15.52:0/3730745378 learned_addr learned my addr 172.21.15.52:0/3730745378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8f8820 0x560cdd90a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa27d1dd700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd9072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 -- 172.21.15.52:0/3730745378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8f8820 msgr2=0x560cdd90a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8f8820 0x560cdd90a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 -- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 msgr2=0x560cdd9072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd9072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 -- 172.21.15.52:0/3730745378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cdd90aec0 con 0x560cdd8fdfe0 2024-06-02T23:11:09.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa277fff700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 0x560cdd90a820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004790 tx=0x7fa26c0139c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc173850408d191 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa257fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa257fff700 1 -- 172.21.15.52:0/3730745378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa26c004220 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa268023070 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.933+0000 7fa257fff700 1 -- 172.21.15.52:0/3730745378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa26c0043c0 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa257fff700 1 -- 172.21.15.52:0/3730745378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa26c003680 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 10 client.15300 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x560cdd97d1f0 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa257fff700 1 -- 172.21.15.52:0/3730745378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa268023070 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa257fff700 1 -- 172.21.15.52:0/3730745378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa26c01d740 con 0x560cdd8fdfe0 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 10 client.15300 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 10 client.15300 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 20 client.15300 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:09.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd97d930 0x560cdd97fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:09.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 4 client.15300 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:11:09.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560cdd851350 con 0x560cdd97d930 2024-06-02T23:11:09.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.934+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd97d930 0x560cdd97fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:09.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.935+0000 7fa27c9dc700 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd97d930 0x560cdd97fd50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e288ceeb7d8d3bd server_cookie=90cf8bcb1089a9c3 in_seq=0 out_seq=0 2024-06-02T23:11:09.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.935+0000 7fa257fff700 10 client.15300 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa257fff700 1 -- 172.21.15.52:0/3730745378 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560cdd851350 con 0x560cdd97d930 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa257fff700 10 client.15300 handle_command_reply: tid=0 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 client.15300 shutdown 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 msgr2=0x560cdd90a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 0x560cdd90a820 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa26c004790 tx=0x7fa26c0139c0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 shutdown_connections 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560cdd97d930 0x560cdd97fd50 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560cdd8fe900 0x560cdd9072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560cdd8f8820 0x560cdd90a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 --2- 172.21.15.52:0/3730745378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560cdd8fdfe0 0x560cdd90a820 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 >> 172.21.15.52:0/3730745378 conn(0x560cdd825100 msgr2=0x560cdd8fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.936+0000 7fa257fff700 0 client.15300 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 shutdown_connections 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa2894c3b80 1 -- 172.21.15.52:0/3730745378 wait complete. 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa2894c3b80 20 client.15300 trim_cache size 0 max 16384 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "cfc1b970-5267-4cce-b643-67125b233eac", 2024-06-02T23:11:09.941 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa2600a2c20 msgr2=0x7fa2600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa2600a2c20 0x7fa2600a50e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa268005950 tx=0x7fa26800b410 comp rx=0 tx=0).stop 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 msgr2=0x7fa278101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 0x7fa278101e80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fa2700095c0 tx=0x7fa270007a20 comp rx=0 tx=0).stop 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 shutdown_connections 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa2600a2c20 0x7fa2600a50e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa278108580 0x7fa278101e80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa27806ac60 0x7fa278101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 --2- 172.21.15.52:0/2936504095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa27806b620 0x7fa278101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:09.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.937+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 >> 172.21.15.52:0/2936504095 conn(0x7fa278078430 msgr2=0x7fa2780fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:09.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.938+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 shutdown_connections 2024-06-02T23:11:09.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:09.938+0000 7fa27ec40700 1 -- 172.21.15.52:0/2936504095 wait complete. 2024-06-02T23:11:11.059 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:11:11.284 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:11 smithi052 ceph-mon[32340]: pgmap v410: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-02T23:11:11.284 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:11 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:11:11.284 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:11 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:11:11.285 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:11 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:11:11.285 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:11 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:11:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:11 smithi174 ceph-mon[35994]: pgmap v410: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-02T23:11:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:11 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:11:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:11 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:11:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:11 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:11:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:11 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:11:11.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 -- 172.21.15.52:0/4199439024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c101440 msgr2=0x7efe9c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:11.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 --2- 172.21.15.52:0/4199439024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c101440 0x7efe9c10f330 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7efe9400b0a0 tx=0x7efe9400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 -- 172.21.15.52:0/4199439024 shutdown_connections 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 --2- 172.21.15.52:0/4199439024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c101440 0x7efe9c10f330 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 --2- 172.21.15.52:0/4199439024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9c108510 0x7efe9c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 --2- 172.21.15.52:0/4199439024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe9c100a80 0x7efe9c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 -- 172.21.15.52:0/4199439024 >> 172.21.15.52:0/4199439024 conn(0x7efe9c077a00 msgr2=0x7efe9c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 -- 172.21.15.52:0/4199439024 shutdown_connections 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.502+0000 7efea38e7700 1 -- 172.21.15.52:0/4199439024 wait complete. 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 Processor -- start 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 -- start start 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe9c100a80 0x7efe9c101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9c101440 0x7efe9c1021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 0x7efe9c1026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:11.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c112620 con 0x7efe9c108510 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1124a0 con 0x7efe9c101440 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea38e7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe9c1127a0 con 0x7efe9c100a80 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea1e84700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 0x7efe9c1026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea1e84700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 0x7efe9c1026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55620/0 (socket says 172.21.15.52:55620) 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea1e84700 1 -- 172.21.15.52:0/3512597766 learned_addr learned my addr 172.21.15.52:0/3512597766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.503+0000 7efea1683700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe9c100a80 0x7efe9c101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea0e82700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9c101440 0x7efe9c1021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea1e84700 1 -- 172.21.15.52:0/3512597766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe9c100a80 msgr2=0x7efe9c101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea1e84700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe9c100a80 0x7efe9c101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea1e84700 1 -- 172.21.15.52:0/3512597766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9c101440 msgr2=0x7efe9c1021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea1e84700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9c101440 0x7efe9c1021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea1e84700 1 -- 172.21.15.52:0/3512597766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe9c1a6ff0 con 0x7efe9c108510 2024-06-02T23:11:11.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea1e84700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 0x7efe9c1026e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe9400b830 tx=0x7efe94004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:11.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94021070 con 0x7efe9c108510 2024-06-02T23:11:11.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe94007e30 con 0x7efe9c108510 2024-06-02T23:11:11.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe94004700 con 0x7efe9c108510 2024-06-02T23:11:11.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe9c1a7280 con 0x7efe9c108510 2024-06-02T23:11:11.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.504+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe9c1a76e0 con 0x7efe9c108510 2024-06-02T23:11:11.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:11 smithi123 ceph-mon[35977]: pgmap v410: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-02T23:11:11.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:11 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:11:11.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:11 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:11:11.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:11 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:11:11.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:11 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:11:11.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.506+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe9c103140 con 0x7efe9c108510 2024-06-02T23:11:11.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.507+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe94003de0 con 0x7efe9c108510 2024-06-02T23:11:11.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.508+0000 7efe927fc700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe880a2ca0 0x7efe880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:11.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.508+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efe9400f020 con 0x7efe9c108510 2024-06-02T23:11:11.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.508+0000 7efea1683700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe880a2ca0 0x7efe880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:11.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.508+0000 7efea1683700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe880a2ca0 0x7efe880a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe9800abd0 tx=0x7efe9800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:11.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.512+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efe9402a020 con 0x7efe9c108510 2024-06-02T23:11:11.537 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:11.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.720+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efe9c0628d0 con 0x7efe880a2ca0 2024-06-02T23:11:11.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.728+0000 7efe927fc700 1 -- 172.21.15.52:0/3512597766 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259068 (secure 0 0 0) 0x7efe9c0628d0 con 0x7efe880a2ca0 2024-06-02T23:11:11.731 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:11.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe880a2ca0 msgr2=0x7efe880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:11.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe880a2ca0 0x7efe880a5160 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7efe9800abd0 tx=0x7efe9800a380 comp rx=0 tx=0).stop 2024-06-02T23:11:11.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 msgr2=0x7efe9c1026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:11.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 0x7efe9c1026e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7efe9400b830 tx=0x7efe94004ce0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 shutdown_connections 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe880a2ca0 0x7efe880a5160 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe9c108510 0x7efe9c1026e0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe9c100a80 0x7efe9c101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 --2- 172.21.15.52:0/3512597766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe9c101440 0x7efe9c1021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 >> 172.21.15.52:0/3512597766 conn(0x7efe9c077a00 msgr2=0x7efe9c0ff130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 shutdown_connections 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:11.731+0000 7efea38e7700 1 -- 172.21.15.52:0/3512597766 wait complete. 2024-06-02T23:11:11.734 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:11:12.452 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-06-02T23:11:09.815113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2127,"num_read_kb":1193,"num_write":456,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369640,"kb_used_data":12168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572184,"statfs":{"total":1151940427776,"available":1151561916416,"internally_reserved":0,"allocated":12460032,"data_stored":7749945,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":116,"num_read_kb":56,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002734"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802175+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:09:39.802175+0000","last_peered":"2024-06-02T23:09:39.802175+0000","last_clean":"2024-06-02T23:09:39.802175+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:09:39.802175+0000","last_undegraded":"2024-06-02T23:09:39.802175+0000","last_fullsized":"2024-06-02T23:09:39.802175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788231+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:09:39.788231+0000","last_peered":"2024-06-02T23:09:39.788231+0000","last_clean":"2024-06-02T23:09:39.788231+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:09:39.788231+0000","last_undegraded":"2024-06-02T23:09:39.788231+0000","last_fullsized":"2024-06-02T23:09:39.788231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794300+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:09:39.794300+0000","last_peered":"2024-06-02T23:09:39.794300+0000","last_clean":"2024-06-02T23:09:39.794300+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:09:39.794300+0000","last_undegraded":"2024-06-02T23:09:39.794300+0000","last_fullsized":"2024-06-02T23:09:39.794300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788417+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:09:39.788417+0000","last_peered":"2024-06-02T23:09:39.788417+0000","last_clean":"2024-06-02T23:09:39.788417+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:09:39.788417+0000","last_undegraded":"2024-06-02T23:09:39.788417+0000","last_fullsized":"2024-06-02T23:09:39.788417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.777964+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:09:39.777964+0000","last_peered":"2024-06-02T23:09:39.777964+0000","last_clean":"2024-06-02T23:09:39.777964+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:09:39.777964+0000","last_undegraded":"2024-06-02T23:09:39.777964+0000","last_fullsized":"2024-06-02T23:09:39.777964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782448+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:09:39.782448+0000","last_peered":"2024-06-02T23:09:39.782448+0000","last_clean":"2024-06-02T23:09:39.782448+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:09:39.782448+0000","last_undegraded":"2024-06-02T23:09:39.782448+0000","last_fullsized":"2024-06-02T23:09:39.782448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786918+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:09:39.786918+0000","last_peered":"2024-06-02T23:09:39.786918+0000","last_clean":"2024-06-02T23:09:39.786918+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:09:39.786918+0000","last_undegraded":"2024-06-02T23:09:39.786918+0000","last_fullsized":"2024-06-02T23:09:39.786918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786735+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:09:39.786735+0000","last_peered":"2024-06-02T23:09:39.786735+0000","last_clean":"2024-06-02T23:09:39.786735+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:09:39.786735+0000","last_undegraded":"2024-06-02T23:09:39.786735+0000","last_fullsized":"2024-06-02T23:09:39.786735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801527+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:09:39.801527+0000","last_peered":"2024-06-02T23:09:39.801527+0000","last_clean":"2024-06-02T23:09:39.801527+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:09:39.801527+0000","last_undegraded":"2024-06-02T23:09:39.801527+0000","last_fullsized":"2024-06-02T23:09:39.801527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789699+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:09:39.789699+0000","last_peered":"2024-06-02T23:09:39.789699+0000","last_clean":"2024-06-02T23:09:39.789699+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:09:39.789699+0000","last_undegraded":"2024-06-02T23:09:39.789699+0000","last_fullsized":"2024-06-02T23:09:39.789699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797457+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:09:39.797457+0000","last_peered":"2024-06-02T23:09:39.797457+0000","last_clean":"2024-06-02T23:09:39.797457+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:09:39.797457+0000","last_undegraded":"2024-06-02T23:09:39.797457+0000","last_fullsized":"2024-06-02T23:09:39.797457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.787757+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:09:39.787757+0000","last_peered":"2024-06-02T23:09:39.787757+0000","last_clean":"2024-06-02T23:09:39.787757+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:09:39.787757+0000","last_undegraded":"2024-06-02T23:09:39.787757+0000","last_fullsized":"2024-06-02T23:09:39.787757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786117+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:09:39.786117+0000","last_peered":"2024-06-02T23:09:39.786117+0000","last_clean":"2024-06-02T23:09:39.786117+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:09:39.786117+0000","last_undegraded":"2024-06-02T23:09:39.786117+0000","last_fullsized":"2024-06-02T23:09:39.786117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788799+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:09:39.788799+0000","last_peered":"2024-06-02T23:09:39.788799+0000","last_clean":"2024-06-02T23:09:39.788799+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:09:39.788799+0000","last_undegraded":"2024-06-02T23:09:39.788799+0000","last_fullsized":"2024-06-02T23:09:39.788799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791625+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:09:39.791625+0000","last_peered":"2024-06-02T23:09:39.791625+0000","last_clean":"2024-06-02T23:09:39.791625+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:09:39.791625+0000","last_undegraded":"2024-06-02T23:09:39.791625+0000","last_fullsized":"2024-06-02T23:09:39.791625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800433+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:09:39.800433+0000","last_peered":"2024-06-02T23:09:39.800433+0000","last_clean":"2024-06-02T23:09:39.800433+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:09:39.800433+0000","last_undegraded":"2024-06-02T23:09:39.800433+0000","last_fullsized":"2024-06-02T23:09:39.800433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802531+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:09:39.802531+0000","last_peered":"2024-06-02T23:09:39.802531+0000","last_clean":"2024-06-02T23:09:39.802531+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:09:39.802531+0000","last_undegraded":"2024-06-02T23:09:39.802531+0000","last_fullsized":"2024-06-02T23:09:39.802531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.803680+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:09:39.803680+0000","last_peered":"2024-06-02T23:09:39.803680+0000","last_clean":"2024-06-02T23:09:39.803680+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:09:39.803680+0000","last_undegraded":"2024-06-02T23:09:39.803680+0000","last_fullsized":"2024-06-02T23:09:39.803680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790081+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:09:39.790081+0000","last_peered":"2024-06-02T23:09:39.790081+0000","last_clean":"2024-06-02T23:09:39.790081+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:09:39.790081+0000","last_undegraded":"2024-06-02T23:09:39.790081+0000","last_fullsized":"2024-06-02T23:09:39.790081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:05.134939+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:11:05.134939+0000","last_peered":"2024-06-02T23:11:05.134939+0000","last_clean":"2024-06-02T23:11:05.134939+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:11:05.134939+0000","last_undegraded":"2024-06-02T23:11:05.134939+0000","last_fullsized":"2024-06-02T23:11:05.134939+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":196,"num_read_kb":196,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788800+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:09:39.788800+0000","last_peered":"2024-06-02T23:09:39.788800+0000","last_clean":"2024-06-02T23:09:39.788800+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:09:39.788800+0000","last_undegraded":"2024-06-02T23:09:39.788800+0000","last_fullsized":"2024-06-02T23:09:39.788800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794320+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:09:39.794320+0000","last_peered":"2024-06-02T23:09:39.794320+0000","last_clean":"2024-06-02T23:09:39.794320+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:09:39.794320+0000","last_undegraded":"2024-06-02T23:09:39.794320+0000","last_fullsized":"2024-06-02T23:09:39.794320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215945+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:14.215945+0000","last_peered":"2024-06-02T23:08:14.215945+0000","last_clean":"2024-06-02T23:08:14.215945+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:14.215945+0000","last_undegraded":"2024-06-02T23:08:14.215945+0000","last_fullsized":"2024-06-02T23:08:14.215945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802474+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:09:39.802474+0000","last_peered":"2024-06-02T23:09:39.802474+0000","last_clean":"2024-06-02T23:09:39.802474+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:09:39.802474+0000","last_undegraded":"2024-06-02T23:09:39.802474+0000","last_fullsized":"2024-06-02T23:09:39.802474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335772+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335772+0000","last_peered":"2024-06-02T23:08:11.335772+0000","last_clean":"2024-06-02T23:08:11.335772+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335772+0000","last_undegraded":"2024-06-02T23:08:11.335772+0000","last_fullsized":"2024-06-02T23:08:11.335772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800976+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:09:39.800976+0000","last_peered":"2024-06-02T23:09:39.800976+0000","last_clean":"2024-06-02T23:09:39.800976+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:09:39.800976+0000","last_undegraded":"2024-06-02T23:09:39.800976+0000","last_fullsized":"2024-06-02T23:09:39.800976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":245,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:07.702985+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:11:07.702985+0000","last_peered":"2024-06-02T23:11:07.702985+0000","last_clean":"2024-06-02T23:11:07.702985+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:11:07.702985+0000","last_undegraded":"2024-06-02T23:11:07.702985+0000","last_fullsized":"2024-06-02T23:11:07.702985+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771608+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:09:39.771608+0000","last_peered":"2024-06-02T23:09:39.771608+0000","last_clean":"2024-06-02T23:09:39.771608+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:09:39.771608+0000","last_undegraded":"2024-06-02T23:09:39.771608+0000","last_fullsized":"2024-06-02T23:09:39.771608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449603+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449603+0000","last_peered":"2024-06-02T23:08:11.449603+0000","last_clean":"2024-06-02T23:08:11.449603+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449603+0000","last_undegraded":"2024-06-02T23:08:11.449603+0000","last_fullsized":"2024-06-02T23:08:11.449603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795377+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:09:39.795377+0000","last_peered":"2024-06-02T23:09:39.795377+0000","last_clean":"2024-06-02T23:09:39.795377+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:09:39.795377+0000","last_undegraded":"2024-06-02T23:09:39.795377+0000","last_fullsized":"2024-06-02T23:09:39.795377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332946+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332946+0000","last_peered":"2024-06-02T23:08:11.332946+0000","last_clean":"2024-06-02T23:08:11.332946+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332946+0000","last_undegraded":"2024-06-02T23:08:11.332946+0000","last_fullsized":"2024-06-02T23:08:11.332946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785226+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:09:39.785226+0000","last_peered":"2024-06-02T23:09:39.785226+0000","last_clean":"2024-06-02T23:09:39.785226+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:09:39.785226+0000","last_undegraded":"2024-06-02T23:09:39.785226+0000","last_fullsized":"2024-06-02T23:09:39.785226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329286+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329286+0000","last_peered":"2024-06-02T23:08:11.329286+0000","last_clean":"2024-06-02T23:08:11.329286+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329286+0000","last_undegraded":"2024-06-02T23:08:11.329286+0000","last_fullsized":"2024-06-02T23:08:11.329286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.779022+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:09:39.779022+0000","last_peered":"2024-06-02T23:09:39.779022+0000","last_clean":"2024-06-02T23:09:39.779022+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:09:39.779022+0000","last_undegraded":"2024-06-02T23:09:39.779022+0000","last_fullsized":"2024-06-02T23:09:39.779022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":425,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:06.700976+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:11:06.700976+0000","last_peered":"2024-06-02T23:11:06.700976+0000","last_clean":"2024-06-02T23:11:06.700976+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:11:06.700976+0000","last_undegraded":"2024-06-02T23:11:06.700976+0000","last_fullsized":"2024-06-02T23:11:06.700976+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":391,"num_read_kb":391,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802707+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:09:39.802707+0000","last_peered":"2024-06-02T23:09:39.802707+0000","last_clean":"2024-06-02T23:09:39.802707+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:09:39.802707+0000","last_undegraded":"2024-06-02T23:09:39.802707+0000","last_fullsized":"2024-06-02T23:09:39.802707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791331+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:09:39.791331+0000","last_peered":"2024-06-02T23:09:39.791331+0000","last_clean":"2024-06-02T23:09:39.791331+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:09:39.791331+0000","last_undegraded":"2024-06-02T23:09:39.791331+0000","last_fullsized":"2024-06-02T23:09:39.791331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330069+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.330069+0000","last_peered":"2024-06-02T23:08:11.330069+0000","last_clean":"2024-06-02T23:08:11.330069+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.330069+0000","last_undegraded":"2024-06-02T23:08:11.330069+0000","last_fullsized":"2024-06-02T23:08:11.330069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801687+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:09:39.801687+0000","last_peered":"2024-06-02T23:09:39.801687+0000","last_clean":"2024-06-02T23:09:39.801687+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:09:39.801687+0000","last_undegraded":"2024-06-02T23:09:39.801687+0000","last_fullsized":"2024-06-02T23:09:39.801687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":466,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:06.701051+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:11:06.701051+0000","last_peered":"2024-06-02T23:11:06.701051+0000","last_clean":"2024-06-02T23:11:06.701051+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:11:06.701051+0000","last_undegraded":"2024-06-02T23:11:06.701051+0000","last_fullsized":"2024-06-02T23:11:06.701051+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":402,"num_read_kb":265,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794578+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:09:39.794578+0000","last_peered":"2024-06-02T23:09:39.794578+0000","last_clean":"2024-06-02T23:09:39.794578+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:09:39.794578+0000","last_undegraded":"2024-06-02T23:09:39.794578+0000","last_fullsized":"2024-06-02T23:09:39.794578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332605+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332605+0000","last_peered":"2024-06-02T23:08:11.332605+0000","last_clean":"2024-06-02T23:08:11.332605+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332605+0000","last_undegraded":"2024-06-02T23:08:11.332605+0000","last_fullsized":"2024-06-02T23:08:11.332605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.778013+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:09:39.778013+0000","last_peered":"2024-06-02T23:09:39.778013+0000","last_clean":"2024-06-02T23:09:39.778013+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:09:39.778013+0000","last_undegraded":"2024-06-02T23:09:39.778013+0000","last_fullsized":"2024-06-02T23:09:39.778013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329465+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329465+0000","last_peered":"2024-06-02T23:08:11.329465+0000","last_clean":"2024-06-02T23:08:11.329465+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329465+0000","last_undegraded":"2024-06-02T23:08:11.329465+0000","last_fullsized":"2024-06-02T23:08:11.329465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792897+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:09:39.792897+0000","last_peered":"2024-06-02T23:09:39.792897+0000","last_clean":"2024-06-02T23:09:39.792897+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:09:39.792897+0000","last_undegraded":"2024-06-02T23:09:39.792897+0000","last_fullsized":"2024-06-02T23:09:39.792897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":230,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:04.190512+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:11:04.190512+0000","last_peered":"2024-06-02T23:11:04.190512+0000","last_clean":"2024-06-02T23:11:04.190512+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:11:04.190512+0000","last_undegraded":"2024-06-02T23:11:04.190512+0000","last_fullsized":"2024-06-02T23:11:04.190512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786680+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:09:39.786680+0000","last_peered":"2024-06-02T23:09:39.786680+0000","last_clean":"2024-06-02T23:09:39.786680+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:09:39.786680+0000","last_undegraded":"2024-06-02T23:09:39.786680+0000","last_fullsized":"2024-06-02T23:09:39.786680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215918+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:14.215918+0000","last_peered":"2024-06-02T23:08:14.215918+0000","last_clean":"2024-06-02T23:08:14.215918+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:14.215918+0000","last_undegraded":"2024-06-02T23:08:14.215918+0000","last_fullsized":"2024-06-02T23:08:14.215918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800649+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:09:39.800649+0000","last_peered":"2024-06-02T23:09:39.800649+0000","last_clean":"2024-06-02T23:09:39.800649+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:09:39.800649+0000","last_undegraded":"2024-06-02T23:09:39.800649+0000","last_fullsized":"2024-06-02T23:09:39.800649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:45.676951+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:09:45.676951+0000","last_peered":"2024-06-02T23:09:45.676951+0000","last_clean":"2024-06-02T23:09:45.676951+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:09:45.676951+0000","last_undegraded":"2024-06-02T23:09:45.676951+0000","last_fullsized":"2024-06-02T23:09:45.676951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.783215+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:09:39.783215+0000","last_peered":"2024-06-02T23:09:39.783215+0000","last_clean":"2024-06-02T23:09:39.783215+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:09:39.783215+0000","last_undegraded":"2024-06-02T23:09:39.783215+0000","last_fullsized":"2024-06-02T23:09:39.783215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328539+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328539+0000","last_peered":"2024-06-02T23:08:11.328539+0000","last_clean":"2024-06-02T23:08:11.328539+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328539+0000","last_undegraded":"2024-06-02T23:08:11.328539+0000","last_fullsized":"2024-06-02T23:08:11.328539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785214+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:09:39.785214+0000","last_peered":"2024-06-02T23:09:39.785214+0000","last_clean":"2024-06-02T23:09:39.785214+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:09:39.785214+0000","last_undegraded":"2024-06-02T23:09:39.785214+0000","last_fullsized":"2024-06-02T23:09:39.785214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":229,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:04.134342+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:11:04.134342+0000","last_peered":"2024-06-02T23:11:04.134342+0000","last_clean":"2024-06-02T23:11:04.134342+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:11:04.134342+0000","last_undegraded":"2024-06-02T23:11:04.134342+0000","last_fullsized":"2024-06-02T23:11:04.134342+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791877+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:09:39.791877+0000","last_peered":"2024-06-02T23:09:39.791877+0000","last_clean":"2024-06-02T23:09:39.791877+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:09:39.791877+0000","last_undegraded":"2024-06-02T23:09:39.791877+0000","last_fullsized":"2024-06-02T23:09:39.791877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792620+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:09:39.792620+0000","last_peered":"2024-06-02T23:09:39.792620+0000","last_clean":"2024-06-02T23:09:39.792620+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:09:39.792620+0000","last_undegraded":"2024-06-02T23:09:39.792620+0000","last_fullsized":"2024-06-02T23:09:39.792620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":232,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:08.196123+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:11:08.196123+0000","last_peered":"2024-06-02T23:11:08.196123+0000","last_clean":"2024-06-02T23:11:08.196123+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:11:08.196123+0000","last_undegraded":"2024-06-02T23:11:08.196123+0000","last_fullsized":"2024-06-02T23:11:08.196123+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":199,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794336+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:09:39.794336+0000","last_peered":"2024-06-02T23:09:39.794336+0000","last_clean":"2024-06-02T23:09:39.794336+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:09:39.794336+0000","last_undegraded":"2024-06-02T23:09:39.794336+0000","last_fullsized":"2024-06-02T23:09:39.794336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:45.814046+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:45.814046+0000","last_peered":"2024-06-02T23:08:45.814046+0000","last_clean":"2024-06-02T23:08:45.814046+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:45.814046+0000","last_undegraded":"2024-06-02T23:08:45.814046+0000","last_fullsized":"2024-06-02T23:08:45.814046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771357+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:09:39.771357+0000","last_peered":"2024-06-02T23:09:39.771357+0000","last_clean":"2024-06-02T23:09:39.771357+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:09:39.771357+0000","last_undegraded":"2024-06-02T23:09:39.771357+0000","last_fullsized":"2024-06-02T23:09:39.771357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788062+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:09:39.788062+0000","last_peered":"2024-06-02T23:09:39.788062+0000","last_clean":"2024-06-02T23:09:39.788062+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:09:39.788062+0000","last_undegraded":"2024-06-02T23:09:39.788062+0000","last_fullsized":"2024-06-02T23:09:39.788062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'19","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.236340+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:10:54.236340+0000","last_peered":"2024-06-02T23:10:54.236340+0000","last_clean":"2024-06-02T23:10:54.236340+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:10:54.236340+0000","last_undegraded":"2024-06-02T23:10:54.236340+0000","last_fullsized":"2024-06-02T23:10:54.236340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790001+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:09:39.790001+0000","last_peered":"2024-06-02T23:09:39.790001+0000","last_clean":"2024-06-02T23:09:39.790001+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:09:39.790001+0000","last_undegraded":"2024-06-02T23:09:39.790001+0000","last_fullsized":"2024-06-02T23:09:39.790001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450463+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450463+0000","last_peered":"2024-06-02T23:08:11.450463+0000","last_clean":"2024-06-02T23:08:11.450463+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450463+0000","last_undegraded":"2024-06-02T23:08:11.450463+0000","last_fullsized":"2024-06-02T23:08:11.450463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792568+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:09:39.792568+0000","last_peered":"2024-06-02T23:09:39.792568+0000","last_clean":"2024-06-02T23:09:39.792568+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:09:39.792568+0000","last_undegraded":"2024-06-02T23:09:39.792568+0000","last_fullsized":"2024-06-02T23:09:39.792568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334494+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334494+0000","last_peered":"2024-06-02T23:08:11.334494+0000","last_clean":"2024-06-02T23:08:11.334494+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334494+0000","last_undegraded":"2024-06-02T23:08:11.334494+0000","last_fullsized":"2024-06-02T23:08:11.334494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782299+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:09:39.782299+0000","last_peered":"2024-06-02T23:09:39.782299+0000","last_clean":"2024-06-02T23:09:39.782299+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:09:39.782299+0000","last_undegraded":"2024-06-02T23:09:39.782299+0000","last_fullsized":"2024-06-02T23:09:39.782299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748925+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748925+0000","last_peered":"2024-06-02T23:08:11.748925+0000","last_clean":"2024-06-02T23:08:11.748925+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748925+0000","last_undegraded":"2024-06-02T23:08:11.748925+0000","last_fullsized":"2024-06-02T23:08:11.748925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792513+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:09:39.792513+0000","last_peered":"2024-06-02T23:09:39.792513+0000","last_clean":"2024-06-02T23:09:39.792513+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:09:39.792513+0000","last_undegraded":"2024-06-02T23:09:39.792513+0000","last_fullsized":"2024-06-02T23:09:39.792513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.275353+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:10:54.275353+0000","last_peered":"2024-06-02T23:10:54.275353+0000","last_clean":"2024-06-02T23:10:54.275353+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:10:54.275353+0000","last_undegraded":"2024-06-02T23:10:54.275353+0000","last_fullsized":"2024-06-02T23:10:54.275353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786323+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:09:39.786323+0000","last_peered":"2024-06-02T23:09:39.786323+0000","last_clean":"2024-06-02T23:09:39.786323+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:09:39.786323+0000","last_undegraded":"2024-06-02T23:09:39.786323+0000","last_fullsized":"2024-06-02T23:09:39.786323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334806+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334806+0000","last_peered":"2024-06-02T23:08:11.334806+0000","last_clean":"2024-06-02T23:08:11.334806+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334806+0000","last_undegraded":"2024-06-02T23:08:11.334806+0000","last_fullsized":"2024-06-02T23:08:11.334806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797923+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:09:39.797923+0000","last_peered":"2024-06-02T23:09:39.797923+0000","last_clean":"2024-06-02T23:09:39.797923+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:09:39.797923+0000","last_undegraded":"2024-06-02T23:09:39.797923+0000","last_fullsized":"2024-06-02T23:09:39.797923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:08.267055+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:11:08.267055+0000","last_peered":"2024-06-02T23:11:08.267055+0000","last_clean":"2024-06-02T23:11:08.267055+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:11:08.267055+0000","last_undegraded":"2024-06-02T23:11:08.267055+0000","last_fullsized":"2024-06-02T23:11:08.267055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2193,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797984+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:09:39.797984+0000","last_peered":"2024-06-02T23:09:39.797984+0000","last_clean":"2024-06-02T23:09:39.797984+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:09:39.797984+0000","last_undegraded":"2024-06-02T23:09:39.797984+0000","last_fullsized":"2024-06-02T23:09:39.797984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330301+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330301+0000","last_peered":"2024-06-02T23:08:11.330301+0000","last_clean":"2024-06-02T23:08:11.330301+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330301+0000","last_undegraded":"2024-06-02T23:08:11.330301+0000","last_fullsized":"2024-06-02T23:08:11.330301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789936+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:09:39.789936+0000","last_peered":"2024-06-02T23:09:39.789936+0000","last_clean":"2024-06-02T23:09:39.789936+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:09:39.789936+0000","last_undegraded":"2024-06-02T23:09:39.789936+0000","last_fullsized":"2024-06-02T23:09:39.789936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333595+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333595+0000","last_peered":"2024-06-02T23:08:11.333595+0000","last_clean":"2024-06-02T23:08:11.333595+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333595+0000","last_undegraded":"2024-06-02T23:08:11.333595+0000","last_fullsized":"2024-06-02T23:08:11.333595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795458+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:09:39.795458+0000","last_peered":"2024-06-02T23:09:39.795458+0000","last_clean":"2024-06-02T23:09:39.795458+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:09:39.795458+0000","last_undegraded":"2024-06-02T23:09:39.795458+0000","last_fullsized":"2024-06-02T23:09:39.795458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790898+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:09:39.790898+0000","last_peered":"2024-06-02T23:09:39.790898+0000","last_clean":"2024-06-02T23:09:39.790898+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:09:39.790898+0000","last_undegraded":"2024-06-02T23:09:39.790898+0000","last_fullsized":"2024-06-02T23:09:39.790898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785496+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:09:39.785496+0000","last_peered":"2024-06-02T23:09:39.785496+0000","last_clean":"2024-06-02T23:09:39.785496+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:09:39.785496+0000","last_undegraded":"2024-06-02T23:09:39.785496+0000","last_fullsized":"2024-06-02T23:09:39.785496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785576+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:09:39.785576+0000","last_peered":"2024-06-02T23:09:39.785576+0000","last_clean":"2024-06-02T23:09:39.785576+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:09:39.785576+0000","last_undegraded":"2024-06-02T23:09:39.785576+0000","last_fullsized":"2024-06-02T23:09:39.785576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791535+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:09:39.791535+0000","last_peered":"2024-06-02T23:09:39.791535+0000","last_clean":"2024-06-02T23:09:39.791535+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:09:39.791535+0000","last_undegraded":"2024-06-02T23:09:39.791535+0000","last_fullsized":"2024-06-02T23:09:39.791535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792313+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:09:39.792313+0000","last_peered":"2024-06-02T23:09:39.792313+0000","last_clean":"2024-06-02T23:09:39.792313+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:09:39.792313+0000","last_undegraded":"2024-06-02T23:09:39.792313+0000","last_fullsized":"2024-06-02T23:09:39.792313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.774868+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:09:39.774868+0000","last_peered":"2024-06-02T23:09:39.774868+0000","last_clean":"2024-06-02T23:09:39.774868+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:09:39.774868+0000","last_undegraded":"2024-06-02T23:09:39.774868+0000","last_fullsized":"2024-06-02T23:09:39.774868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794273+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:09:39.794273+0000","last_peered":"2024-06-02T23:09:39.794273+0000","last_clean":"2024-06-02T23:09:39.794273+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:09:39.794273+0000","last_undegraded":"2024-06-02T23:09:39.794273+0000","last_fullsized":"2024-06-02T23:09:39.794273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788292+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:09:39.788292+0000","last_peered":"2024-06-02T23:09:39.788292+0000","last_clean":"2024-06-02T23:09:39.788292+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:09:39.788292+0000","last_undegraded":"2024-06-02T23:09:39.788292+0000","last_fullsized":"2024-06-02T23:09:39.788292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786749+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:09:39.786749+0000","last_peered":"2024-06-02T23:09:39.786749+0000","last_clean":"2024-06-02T23:09:39.786749+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:09:39.786749+0000","last_undegraded":"2024-06-02T23:09:39.786749+0000","last_fullsized":"2024-06-02T23:09:39.786749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789933+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:09:39.789933+0000","last_peered":"2024-06-02T23:09:39.789933+0000","last_clean":"2024-06-02T23:09:39.789933+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:09:39.789933+0000","last_undegraded":"2024-06-02T23:09:39.789933+0000","last_fullsized":"2024-06-02T23:09:39.789933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800745+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:09:39.800745+0000","last_peered":"2024-06-02T23:09:39.800745+0000","last_clean":"2024-06-02T23:09:39.800745+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:09:39.800745+0000","last_undegraded":"2024-06-02T23:09:39.800745+0000","last_fullsized":"2024-06-02T23:09:39.800745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792409+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:09:39.792409+0000","last_peered":"2024-06-02T23:09:39.792409+0000","last_clean":"2024-06-02T23:09:39.792409+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:09:39.792409+0000","last_undegraded":"2024-06-02T23:09:39.792409+0000","last_fullsized":"2024-06-02T23:09:39.792409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.776917+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:09:39.776917+0000","last_peered":"2024-06-02T23:09:39.776917+0000","last_clean":"2024-06-02T23:09:39.776917+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:09:39.776917+0000","last_undegraded":"2024-06-02T23:09:39.776917+0000","last_fullsized":"2024-06-02T23:09:39.776917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2001,"num_read_kb":1084,"num_write":273,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":227403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383741,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645377,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1138821,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808902,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332177,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074331,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495702,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":548352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":482429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411435,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":548481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182501,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920863,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757339,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1073025,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:11:12.457 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:11:13.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:13 smithi052 ceph-mon[32340]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:13.368 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:13 smithi052 ceph-mon[32340]: pgmap v411: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:11:13.451 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:13.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:13 smithi123 ceph-mon[35977]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:13.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:13 smithi123 ceph-mon[35977]: pgmap v411: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:11:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:13 smithi174 ceph-mon[35994]: from='client.15306 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:13 smithi174 ceph-mon[35994]: pgmap v411: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:11:14.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 -- 172.21.15.52:0/5975478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006abc0 msgr2=0x7fbc3006afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/5975478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006abc0 0x7fbc3006afa0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009990 tx=0x7fbc24009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 -- 172.21.15.52:0/5975478 shutdown_connections 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/5975478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc3010ef70 0x7fbc30111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/5975478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006abc0 0x7fbc3006afa0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/5975478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc3006b4e0 0x7fbc3010ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 -- 172.21.15.52:0/5975478 >> 172.21.15.52:0/5975478 conn(0x7fbc30078340 msgr2=0x7fbc30078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 -- 172.21.15.52:0/5975478 shutdown_connections 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.586+0000 7fbc36e3c700 1 -- 172.21.15.52:0/5975478 wait complete. 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 Processor -- start 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 -- start start 2024-06-02T23:11:14.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc3006abc0 0x7fbc3019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 0x7fbc3019b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc3010ef70 0x7fbc3019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc30114840 con 0x7fbc3006b4e0 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301146c0 con 0x7fbc3006abc0 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc36e3c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301149c0 con 0x7fbc3010ef70 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 0x7fbc3019b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 0x7fbc3019b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36914/0 (socket says 172.21.15.52:36914) 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 -- 172.21.15.52:0/3153040059 learned_addr learned my addr 172.21.15.52:0/3153040059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc353d9700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc3010ef70 0x7fbc3019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc3006abc0 0x7fbc3019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 -- 172.21.15.52:0/3153040059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc3010ef70 msgr2=0x7fbc3019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc3010ef70 0x7fbc3019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 -- 172.21.15.52:0/3153040059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc3006abc0 msgr2=0x7fbc3019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc3006abc0 0x7fbc3019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.587+0000 7fbc2ffff700 1 -- 172.21.15.52:0/3153040059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc3019fd80 con 0x7fbc3006b4e0 2024-06-02T23:11:14.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.588+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 0x7fbc3019b2b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d950 tx=0x7fbc1c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:14.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.588+0000 7fbc2dffb700 1 -- 172.21.15.52:0/3153040059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c016070 con 0x7fbc3006b4e0 2024-06-02T23:11:14.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.588+0000 7fbc2dffb700 1 -- 172.21.15.52:0/3153040059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc1c00fd40 con 0x7fbc3006b4e0 2024-06-02T23:11:14.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.588+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc301a0070 con 0x7fbc3006b4e0 2024-06-02T23:11:14.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.588+0000 7fbc2dffb700 1 -- 172.21.15.52:0/3153040059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc1c00e3b0 con 0x7fbc3006b4e0 2024-06-02T23:11:14.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.588+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc301a78f0 con 0x7fbc3006b4e0 2024-06-02T23:11:14.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.590+0000 7fbc2dffb700 1 -- 172.21.15.52:0/3153040059 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc1c019070 con 0x7fbc3006b4e0 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.591+0000 7fbc2dffb700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc200a2c20 0x7fbc200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.591+0000 7fbc2dffb700 1 -- 172.21.15.52:0/3153040059 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbc1c0cea90 con 0x7fbc3006b4e0 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.591+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc200a2c20 0x7fbc200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.591+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc200a2c20 0x7fbc200a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006010 tx=0x7fbc2400b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 Processor -- start 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 -- start start 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb1976260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557eb1868f60 con 0x557eb19767a0 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557eb18c7850 con 0x557eb197bf60 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557eb182f460 con 0x557eb197c880 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36922/0 (socket says 172.21.15.52:36922) 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 -- 172.21.15.52:0/3127546184 learned_addr learned my addr 172.21.15.52:0/3127546184 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb1976260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 -- 172.21.15.52:0/3127546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 msgr2=0x557eb1976260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb1976260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 -- 172.21.15.52:0/3127546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 msgr2=0x557eb197c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc353d9700 1 -- 172.21.15.52:0/3127546184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb18168d0 con 0x557eb19767a0 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.592+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb1976260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.593+0000 7fbc353d9700 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976bc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbc28003ea0 tx=0x7fbc28009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcadc425f5a0493 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.593+0000 7fbc1affd700 1 -- 172.21.15.52:0/3127546184 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc280045f0 con 0x557eb19767a0 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.593+0000 7fbc1affd700 1 -- 172.21.15.52:0/3127546184 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2801a070 con 0x557eb19767a0 2024-06-02T23:11:14.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.593+0000 7fbc1affd700 1 -- 172.21.15.52:0/3127546184 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc28015680 con 0x557eb19767a0 2024-06-02T23:11:14.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.593+0000 7fbc416bfb80 1 -- 172.21.15.52:0/3127546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 msgr2=0x557eb1976bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976bc0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbc28003ea0 tx=0x7fbc28009c20 comp rx=0 tx=0).stop 2024-06-02T23:11:14.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 -- 172.21.15.52:0/3127546184 shutdown_connections 2024-06-02T23:11:14.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976bc0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/3127546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb1976260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 -- 172.21.15.52:0/3127546184 >> 172.21.15.52:0/3127546184 conn(0x557eb18a3080 msgr2=0x557eb1971220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 -- 172.21.15.52:0/3127546184 shutdown_connections 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 -- 172.21.15.52:0/3127546184 wait complete. 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 -- start start 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.594+0000 7fbc416bfb80 1 Processor -- start 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1982570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb197fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557eb1867fc0 con 0x557eb19767a0 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557eb18c3810 con 0x557eb197bf60 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557eb1983260 con 0x557eb197c880 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc416bfb80 10 client.0 _ll_register_callbacks cb 0x557eb185d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:14.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1982570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1982570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36934/0 (socket says 172.21.15.52:36934) 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 -- 172.21.15.52:0/1041773206 learned_addr learned my addr 172.21.15.52:0/1041773206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc353d9700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb197fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 -- 172.21.15.52:0/1041773206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 msgr2=0x557eb197fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb197fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 -- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 msgr2=0x557eb197f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 -- 172.21.15.52:0/1041773206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb188d750 con 0x557eb19767a0 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc353d9700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb197fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1982570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009ff0 tx=0x7fbc2401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cbfc5fe11c1ed37 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.595+0000 7fbc0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:14.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc0ffff700 1 -- 172.21.15.52:0/1041773206 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24024610 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc0ffff700 1 -- 172.21.15.52:0/1041773206 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbc240247b0 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc2801a070 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc0ffff700 1 -- 172.21.15.52:0/1041773206 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc240249b0 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 10 client.15318 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557eb198d9a0 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc0ffff700 1 -- 172.21.15.52:0/1041773206 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc2801a070 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc0ffff700 1 -- 172.21.15.52:0/1041773206 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc24039a40 con 0x557eb19767a0 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 20 client.15318 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb1990030 0x557eb1992420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 4 client.15318 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.596+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eb18d4d50 con 0x557eb1990030 2024-06-02T23:11:14.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.597+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb1990030 0x557eb1992420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.597+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb1990030 0x557eb1992420 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867fe179d15aaf20 server_cookie=ffe53524c2772451 in_seq=0 out_seq=0 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.597+0000 7fbc0ffff700 10 client.15318 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc0ffff700 1 -- 172.21.15.52:0/1041773206 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557eb18d4d50 con 0x557eb1990030 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc0ffff700 10 client.15318 handle_command_reply: tid=0 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 client.15318 shutdown 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 msgr2=0x557eb1982570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1982570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009ff0 tx=0x7fbc2401a050 comp rx=0 tx=0).stop 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 shutdown_connections 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb1990030 0x557eb1992420 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb197fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1982570 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/1041773206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 >> 172.21.15.52:0/1041773206 conn(0x557eb18a3080 msgr2=0x557eb1970700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:14.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc0ffff700 0 client.15318 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:14.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 shutdown_connections 2024-06-02T23:11:14.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 1 -- 172.21.15.52:0/1041773206 wait complete. 2024-06-02T23:11:14.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.598+0000 7fbc416bfb80 20 client.15318 trim_cache size 0 max 16384 2024-06-02T23:11:14.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 Processor -- start 2024-06-02T23:11:14.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 -- start start 2024-06-02T23:11:14.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb19817b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301aecf0 con 0x557eb19767a0 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301b0830 con 0x557eb197bf60 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301af3c0 con 0x557eb197c880 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc34bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc34bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36944/0 (socket says 172.21.15.52:36944) 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc34bd8700 1 -- 172.21.15.52:0/993365987 learned_addr learned my addr 172.21.15.52:0/993365987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.620+0000 7fbc353d9700 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb19817b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc34bd8700 1 -- 172.21.15.52:0/993365987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 msgr2=0x557eb19817b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb19817b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc34bd8700 1 -- 172.21.15.52:0/993365987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 msgr2=0x557eb197d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc34bd8700 1 -- 172.21.15.52:0/993365987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb197e850 con 0x557eb19767a0 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009750 tx=0x7fbc240094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e41e38f567f8cda server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc1a7fc700 1 -- 172.21.15.52:0/993365987 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc240306d0 con 0x557eb19767a0 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc1a7fc700 1 -- 172.21.15.52:0/993365987 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc24035070 con 0x557eb19767a0 2024-06-02T23:11:14.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.621+0000 7fbc1a7fc700 1 -- 172.21.15.52:0/993365987 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24030c60 con 0x557eb19767a0 2024-06-02T23:11:14.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 -- 172.21.15.52:0/993365987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 msgr2=0x557eb1976260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976260 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbc24009750 tx=0x7fbc240094b0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 -- 172.21.15.52:0/993365987 shutdown_connections 2024-06-02T23:11:14.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197c880 0x557eb19817b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1976260 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/993365987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197bf60 0x557eb197d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 -- 172.21.15.52:0/993365987 >> 172.21.15.52:0/993365987 conn(0x557eb18a3080 msgr2=0x557eb19810a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 -- 172.21.15.52:0/993365987 shutdown_connections 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 -- 172.21.15.52:0/993365987 wait complete. 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 -- start start 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.622+0000 7fbc416bfb80 1 Processor -- start 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1988930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197bf60 0x557eb1988e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197c880 0x557eb19974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301b0160 con 0x557eb19767a0 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301afa90 con 0x557eb197c880 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc301accb0 con 0x557eb197bf60 2024-06-02T23:11:14.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc416bfb80 10 client.0 _ll_register_callbacks cb 0x557eb185d5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1988930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1988930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36948/0 (socket says 172.21.15.52:36948) 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 -- 172.21.15.52:0/185215483 learned_addr learned my addr 172.21.15.52:0/185215483 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc353d9700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197c880 0x557eb19974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197bf60 0x557eb1988e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 -- 172.21.15.52:0/185215483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197bf60 msgr2=0x557eb1988e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197bf60 0x557eb1988e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 -- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197c880 msgr2=0x557eb19974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197c880 0x557eb19974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 -- 172.21.15.52:0/185215483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb1997d00 con 0x557eb19767a0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197bf60 0x557eb1988e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc34bd8700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1988930 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc24024690 tx=0x7fbc24009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda7b4b963f89df9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.623+0000 7fbc0ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:14.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc0ffff700 1 -- 172.21.15.52:0/185215483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc24039810 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc0ffff700 1 -- 172.21.15.52:0/185215483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbc240399b0 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc24035070 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc0ffff700 1 -- 172.21.15.52:0/185215483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbc24039bd0 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 10 client.15324 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557eb19fa980 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc0ffff700 1 -- 172.21.15.52:0/185215483 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc24035070 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc0ffff700 1 -- 172.21.15.52:0/185215483 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbc24030450 con 0x557eb19767a0 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 10 client.15324 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 10 client.15324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 20 client.15324 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb19fafa0 0x557eb19fd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 4 client.15324 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:11:14.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.624+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557eb18cf2d0 con 0x557eb19fafa0 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb19fafa0 0x557eb19fd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc2ffff700 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb19fafa0 0x557eb19fd3c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bb60a0ba9c3698c server_cookie=5bbf3a24e0092267 in_seq=0 out_seq=0 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc0ffff700 10 client.15324 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc0ffff700 1 -- 172.21.15.52:0/185215483 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557eb18cf2d0 con 0x557eb19fafa0 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc0ffff700 10 client.15324 handle_command_reply: tid=0 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 client.15324 shutdown 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 msgr2=0x557eb1988930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1988930 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fbc24024690 tx=0x7fbc24009950 comp rx=0 tx=0).stop 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 shutdown_connections 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557eb19fafa0 0x557eb19fd3c0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc0ffff700 0 client.15324 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557eb197c880 0x557eb19974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557eb19767a0 0x557eb1988930 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 --2- 172.21.15.52:0/185215483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb197bf60 0x557eb1988e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 >> 172.21.15.52:0/185215483 conn(0x557eb18a3080 msgr2=0x557eb197f980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 shutdown_connections 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.625+0000 7fbc416bfb80 1 -- 172.21.15.52:0/185215483 wait complete. 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc416bfb80 20 client.15324 trim_cache size 0 max 16384 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "cfc1b970-5267-4cce-b643-67125b233eac": { 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:11:14.632 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "cfc1b970-5267-4cce-b643-67125b233eac", 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc200a2c20 msgr2=0x7fbc200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc200a2c20 0x7fbc200a50e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbc24006010 tx=0x7fbc2400b560 comp rx=0 tx=0).stop 2024-06-02T23:11:14.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 msgr2=0x7fbc3019b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 0x7fbc3019b2b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c00d950 tx=0x7fbc1c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 shutdown_connections 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc200a2c20 0x7fbc200a50e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc3010ef70 0x7fbc3019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc3006abc0 0x7fbc3019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 --2- 172.21.15.52:0/3153040059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc3006b4e0 0x7fbc3019b2b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 >> 172.21.15.52:0/3153040059 conn(0x7fbc30078340 msgr2=0x7fbc301106d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 shutdown_connections 2024-06-02T23:11:14.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:14.626+0000 7fbc36e3c700 1 -- 172.21.15.52:0/3153040059 wait complete. 2024-06-02T23:11:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:15 smithi052 ceph-mon[32340]: pgmap v412: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-02T23:11:15.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:15 smithi123 ceph-mon[35977]: pgmap v412: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-02T23:11:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:15 smithi174 ceph-mon[35994]: pgmap v412: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-02T23:11:15.738 INFO:tasks.cephfs.filesystem:scrub status for tag:cfc1b970-5267-4cce-b643-67125b233eac - {'path': '/', 'tag': 'cfc1b970-5267-4cce-b643-67125b233eac', 'options': 'recursive,force'} 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 -- 172.21.15.52:0/1876388420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101d30 msgr2=0x7f331410e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 --2- 172.21.15.52:0/1876388420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101d30 0x7f331410e1d0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f330c00b0a0 tx=0x7f330c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 -- 172.21.15.52:0/1876388420 shutdown_connections 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 --2- 172.21.15.52:0/1876388420 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101d30 0x7f331410e1d0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 --2- 172.21.15.52:0/1876388420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314107420 0x7f3314107800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 --2- 172.21.15.52:0/1876388420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3314101370 0x7f33141017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 -- 172.21.15.52:0/1876388420 >> 172.21.15.52:0/1876388420 conn(0x7f3314078340 msgr2=0x7f3314078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:16.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 -- 172.21.15.52:0/1876388420 shutdown_connections 2024-06-02T23:11:16.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.245+0000 7f3319fe3700 1 -- 172.21.15.52:0/1876388420 wait complete. 2024-06-02T23:11:16.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 Processor -- start 2024-06-02T23:11:16.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 -- start start 2024-06-02T23:11:16.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 0x7f331419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314101d30 0x7f331419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3314107420 0x7f3314198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3314111590 con 0x7f3314101370 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3314111410 con 0x7f3314101d30 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3319fe3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3314111710 con 0x7f3314107420 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f33137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 0x7f331419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f33137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 0x7f331419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36962/0 (socket says 172.21.15.52:36962) 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f33137fe700 1 -- 172.21.15.52:0/313357991 learned_addr learned my addr 172.21.15.52:0/313357991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3312ffd700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314101d30 0x7f331419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.246+0000 7f3313fff700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3314107420 0x7f3314198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f33137fe700 1 -- 172.21.15.52:0/313357991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3314107420 msgr2=0x7f3314198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:16.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f33137fe700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3314107420 0x7f3314198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f33137fe700 1 -- 172.21.15.52:0/313357991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314101d30 msgr2=0x7f331419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f33137fe700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314101d30 0x7f331419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f33137fe700 1 -- 172.21.15.52:0/313357991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33141990f0 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f3312ffd700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314101d30 0x7f331419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f33137fe700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 0x7f331419e7d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f330400b530 tx=0x7f330400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304015020 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3304011ce0 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33141993e0 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3304010e40 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.247+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33141a56a0 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.248+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3314069180 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.249+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3304018020 con 0x7f3314101370 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.250+0000 7f3310ff9700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32fc0a2c20 0x7f32fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:16.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.250+0000 7f3312ffd700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32fc0a2c20 0x7f32fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:16.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.250+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33040ce8f0 con 0x7f3314101370 2024-06-02T23:11:16.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.251+0000 7f3312ffd700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32fc0a2c20 0x7f32fc0a50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3314078980 tx=0x7f3308005f30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:16.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.255+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f330401f020 con 0x7f3314101370 2024-06-02T23:11:16.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.459+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3314106170 con 0x7f32fc0a2c20 2024-06-02T23:11:16.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.466+0000 7f3310ff9700 1 -- 172.21.15.52:0/313357991 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259068 (secure 0 0 0) 0x7f3314106170 con 0x7f32fc0a2c20 2024-06-02T23:11:16.470 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32fc0a2c20 msgr2=0x7f32fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32fc0a2c20 0x7f32fc0a50e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3314078980 tx=0x7f3308005f30 comp rx=0 tx=0).stop 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 msgr2=0x7f331419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 0x7f331419e7d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f330400b530 tx=0x7f330400b840 comp rx=0 tx=0).stop 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 shutdown_connections 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32fc0a2c20 0x7f32fc0a50e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3314107420 0x7f3314198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3314101370 0x7f331419e7d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 --2- 172.21.15.52:0/313357991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3314101d30 0x7f331419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:16.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.469+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 >> 172.21.15.52:0/313357991 conn(0x7f3314078340 msgr2=0x7f331406ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:16.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.470+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 shutdown_connections 2024-06-02T23:11:16.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:16.470+0000 7f3319fe3700 1 -- 172.21.15.52:0/313357991 wait complete. 2024-06-02T23:11:16.473 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:11:17.148 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-06-02T23:11:15.816631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2184,"num_read_kb":1220,"num_write":456,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369640,"kb_used_data":12168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572184,"statfs":{"total":1151940427776,"available":1151561916416,"internally_reserved":0,"allocated":12460032,"data_stored":7749945,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":122,"num_read_kb":57,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002943"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802175+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:09:39.802175+0000","last_peered":"2024-06-02T23:09:39.802175+0000","last_clean":"2024-06-02T23:09:39.802175+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:09:39.802175+0000","last_undegraded":"2024-06-02T23:09:39.802175+0000","last_fullsized":"2024-06-02T23:09:39.802175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788231+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:09:39.788231+0000","last_peered":"2024-06-02T23:09:39.788231+0000","last_clean":"2024-06-02T23:09:39.788231+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:09:39.788231+0000","last_undegraded":"2024-06-02T23:09:39.788231+0000","last_fullsized":"2024-06-02T23:09:39.788231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794300+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:09:39.794300+0000","last_peered":"2024-06-02T23:09:39.794300+0000","last_clean":"2024-06-02T23:09:39.794300+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:09:39.794300+0000","last_undegraded":"2024-06-02T23:09:39.794300+0000","last_fullsized":"2024-06-02T23:09:39.794300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788417+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:09:39.788417+0000","last_peered":"2024-06-02T23:09:39.788417+0000","last_clean":"2024-06-02T23:09:39.788417+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:09:39.788417+0000","last_undegraded":"2024-06-02T23:09:39.788417+0000","last_fullsized":"2024-06-02T23:09:39.788417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.777964+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:09:39.777964+0000","last_peered":"2024-06-02T23:09:39.777964+0000","last_clean":"2024-06-02T23:09:39.777964+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:09:39.777964+0000","last_undegraded":"2024-06-02T23:09:39.777964+0000","last_fullsized":"2024-06-02T23:09:39.777964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782448+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:09:39.782448+0000","last_peered":"2024-06-02T23:09:39.782448+0000","last_clean":"2024-06-02T23:09:39.782448+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:09:39.782448+0000","last_undegraded":"2024-06-02T23:09:39.782448+0000","last_fullsized":"2024-06-02T23:09:39.782448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786918+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:09:39.786918+0000","last_peered":"2024-06-02T23:09:39.786918+0000","last_clean":"2024-06-02T23:09:39.786918+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:09:39.786918+0000","last_undegraded":"2024-06-02T23:09:39.786918+0000","last_fullsized":"2024-06-02T23:09:39.786918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786735+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:09:39.786735+0000","last_peered":"2024-06-02T23:09:39.786735+0000","last_clean":"2024-06-02T23:09:39.786735+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:09:39.786735+0000","last_undegraded":"2024-06-02T23:09:39.786735+0000","last_fullsized":"2024-06-02T23:09:39.786735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801527+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:09:39.801527+0000","last_peered":"2024-06-02T23:09:39.801527+0000","last_clean":"2024-06-02T23:09:39.801527+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:09:39.801527+0000","last_undegraded":"2024-06-02T23:09:39.801527+0000","last_fullsized":"2024-06-02T23:09:39.801527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789699+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:09:39.789699+0000","last_peered":"2024-06-02T23:09:39.789699+0000","last_clean":"2024-06-02T23:09:39.789699+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:09:39.789699+0000","last_undegraded":"2024-06-02T23:09:39.789699+0000","last_fullsized":"2024-06-02T23:09:39.789699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797457+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:09:39.797457+0000","last_peered":"2024-06-02T23:09:39.797457+0000","last_clean":"2024-06-02T23:09:39.797457+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:09:39.797457+0000","last_undegraded":"2024-06-02T23:09:39.797457+0000","last_fullsized":"2024-06-02T23:09:39.797457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.787757+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:09:39.787757+0000","last_peered":"2024-06-02T23:09:39.787757+0000","last_clean":"2024-06-02T23:09:39.787757+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:09:39.787757+0000","last_undegraded":"2024-06-02T23:09:39.787757+0000","last_fullsized":"2024-06-02T23:09:39.787757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786117+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:09:39.786117+0000","last_peered":"2024-06-02T23:09:39.786117+0000","last_clean":"2024-06-02T23:09:39.786117+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:09:39.786117+0000","last_undegraded":"2024-06-02T23:09:39.786117+0000","last_fullsized":"2024-06-02T23:09:39.786117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788799+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:09:39.788799+0000","last_peered":"2024-06-02T23:09:39.788799+0000","last_clean":"2024-06-02T23:09:39.788799+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:09:39.788799+0000","last_undegraded":"2024-06-02T23:09:39.788799+0000","last_fullsized":"2024-06-02T23:09:39.788799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791625+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:09:39.791625+0000","last_peered":"2024-06-02T23:09:39.791625+0000","last_clean":"2024-06-02T23:09:39.791625+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:09:39.791625+0000","last_undegraded":"2024-06-02T23:09:39.791625+0000","last_fullsized":"2024-06-02T23:09:39.791625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800433+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:09:39.800433+0000","last_peered":"2024-06-02T23:09:39.800433+0000","last_clean":"2024-06-02T23:09:39.800433+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:09:39.800433+0000","last_undegraded":"2024-06-02T23:09:39.800433+0000","last_fullsized":"2024-06-02T23:09:39.800433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802531+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:09:39.802531+0000","last_peered":"2024-06-02T23:09:39.802531+0000","last_clean":"2024-06-02T23:09:39.802531+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:09:39.802531+0000","last_undegraded":"2024-06-02T23:09:39.802531+0000","last_fullsized":"2024-06-02T23:09:39.802531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.803680+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:09:39.803680+0000","last_peered":"2024-06-02T23:09:39.803680+0000","last_clean":"2024-06-02T23:09:39.803680+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:09:39.803680+0000","last_undegraded":"2024-06-02T23:09:39.803680+0000","last_fullsized":"2024-06-02T23:09:39.803680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790081+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:09:39.790081+0000","last_peered":"2024-06-02T23:09:39.790081+0000","last_clean":"2024-06-02T23:09:39.790081+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:09:39.790081+0000","last_undegraded":"2024-06-02T23:09:39.790081+0000","last_fullsized":"2024-06-02T23:09:39.790081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:10.140868+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:11:10.140868+0000","last_peered":"2024-06-02T23:11:10.140868+0000","last_clean":"2024-06-02T23:11:10.140868+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:11:10.140868+0000","last_undegraded":"2024-06-02T23:11:10.140868+0000","last_fullsized":"2024-06-02T23:11:10.140868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":202,"num_read_kb":202,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788800+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:09:39.788800+0000","last_peered":"2024-06-02T23:09:39.788800+0000","last_clean":"2024-06-02T23:09:39.788800+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:09:39.788800+0000","last_undegraded":"2024-06-02T23:09:39.788800+0000","last_fullsized":"2024-06-02T23:09:39.788800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794320+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:09:39.794320+0000","last_peered":"2024-06-02T23:09:39.794320+0000","last_clean":"2024-06-02T23:09:39.794320+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:09:39.794320+0000","last_undegraded":"2024-06-02T23:09:39.794320+0000","last_fullsized":"2024-06-02T23:09:39.794320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215945+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:14.215945+0000","last_peered":"2024-06-02T23:08:14.215945+0000","last_clean":"2024-06-02T23:08:14.215945+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:14.215945+0000","last_undegraded":"2024-06-02T23:08:14.215945+0000","last_fullsized":"2024-06-02T23:08:14.215945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802474+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:09:39.802474+0000","last_peered":"2024-06-02T23:09:39.802474+0000","last_clean":"2024-06-02T23:09:39.802474+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:09:39.802474+0000","last_undegraded":"2024-06-02T23:09:39.802474+0000","last_fullsized":"2024-06-02T23:09:39.802474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335772+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335772+0000","last_peered":"2024-06-02T23:08:11.335772+0000","last_clean":"2024-06-02T23:08:11.335772+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335772+0000","last_undegraded":"2024-06-02T23:08:11.335772+0000","last_fullsized":"2024-06-02T23:08:11.335772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800976+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:09:39.800976+0000","last_peered":"2024-06-02T23:09:39.800976+0000","last_clean":"2024-06-02T23:09:39.800976+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:09:39.800976+0000","last_undegraded":"2024-06-02T23:09:39.800976+0000","last_fullsized":"2024-06-02T23:09:39.800976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":250,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:12.710528+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:11:12.710528+0000","last_peered":"2024-06-02T23:11:12.710528+0000","last_clean":"2024-06-02T23:11:12.710528+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:11:12.710528+0000","last_undegraded":"2024-06-02T23:11:12.710528+0000","last_fullsized":"2024-06-02T23:11:12.710528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771608+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:09:39.771608+0000","last_peered":"2024-06-02T23:09:39.771608+0000","last_clean":"2024-06-02T23:09:39.771608+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:09:39.771608+0000","last_undegraded":"2024-06-02T23:09:39.771608+0000","last_fullsized":"2024-06-02T23:09:39.771608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449603+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449603+0000","last_peered":"2024-06-02T23:08:11.449603+0000","last_clean":"2024-06-02T23:08:11.449603+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449603+0000","last_undegraded":"2024-06-02T23:08:11.449603+0000","last_fullsized":"2024-06-02T23:08:11.449603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795377+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:09:39.795377+0000","last_peered":"2024-06-02T23:09:39.795377+0000","last_clean":"2024-06-02T23:09:39.795377+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:09:39.795377+0000","last_undegraded":"2024-06-02T23:09:39.795377+0000","last_fullsized":"2024-06-02T23:09:39.795377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332946+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332946+0000","last_peered":"2024-06-02T23:08:11.332946+0000","last_clean":"2024-06-02T23:08:11.332946+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332946+0000","last_undegraded":"2024-06-02T23:08:11.332946+0000","last_fullsized":"2024-06-02T23:08:11.332946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785226+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:09:39.785226+0000","last_peered":"2024-06-02T23:09:39.785226+0000","last_clean":"2024-06-02T23:09:39.785226+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:09:39.785226+0000","last_undegraded":"2024-06-02T23:09:39.785226+0000","last_fullsized":"2024-06-02T23:09:39.785226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329286+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329286+0000","last_peered":"2024-06-02T23:08:11.329286+0000","last_clean":"2024-06-02T23:08:11.329286+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329286+0000","last_undegraded":"2024-06-02T23:08:11.329286+0000","last_fullsized":"2024-06-02T23:08:11.329286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.779022+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:09:39.779022+0000","last_peered":"2024-06-02T23:09:39.779022+0000","last_clean":"2024-06-02T23:09:39.779022+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:09:39.779022+0000","last_undegraded":"2024-06-02T23:09:39.779022+0000","last_fullsized":"2024-06-02T23:09:39.779022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":435,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:11.708526+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:11:11.708526+0000","last_peered":"2024-06-02T23:11:11.708526+0000","last_clean":"2024-06-02T23:11:11.708526+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:11:11.708526+0000","last_undegraded":"2024-06-02T23:11:11.708526+0000","last_fullsized":"2024-06-02T23:11:11.708526+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":401,"num_read_kb":401,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802707+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:09:39.802707+0000","last_peered":"2024-06-02T23:09:39.802707+0000","last_clean":"2024-06-02T23:09:39.802707+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:09:39.802707+0000","last_undegraded":"2024-06-02T23:09:39.802707+0000","last_fullsized":"2024-06-02T23:09:39.802707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791331+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:09:39.791331+0000","last_peered":"2024-06-02T23:09:39.791331+0000","last_clean":"2024-06-02T23:09:39.791331+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:09:39.791331+0000","last_undegraded":"2024-06-02T23:09:39.791331+0000","last_fullsized":"2024-06-02T23:09:39.791331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330069+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.330069+0000","last_peered":"2024-06-02T23:08:11.330069+0000","last_clean":"2024-06-02T23:08:11.330069+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.330069+0000","last_undegraded":"2024-06-02T23:08:11.330069+0000","last_fullsized":"2024-06-02T23:08:11.330069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801687+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:09:39.801687+0000","last_peered":"2024-06-02T23:09:39.801687+0000","last_clean":"2024-06-02T23:09:39.801687+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:09:39.801687+0000","last_undegraded":"2024-06-02T23:09:39.801687+0000","last_fullsized":"2024-06-02T23:09:39.801687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":477,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:11.708629+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:11:11.708629+0000","last_peered":"2024-06-02T23:11:11.708629+0000","last_clean":"2024-06-02T23:11:11.708629+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:11:11.708629+0000","last_undegraded":"2024-06-02T23:11:11.708629+0000","last_fullsized":"2024-06-02T23:11:11.708629+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":413,"num_read_kb":271,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794578+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:09:39.794578+0000","last_peered":"2024-06-02T23:09:39.794578+0000","last_clean":"2024-06-02T23:09:39.794578+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:09:39.794578+0000","last_undegraded":"2024-06-02T23:09:39.794578+0000","last_fullsized":"2024-06-02T23:09:39.794578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332605+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332605+0000","last_peered":"2024-06-02T23:08:11.332605+0000","last_clean":"2024-06-02T23:08:11.332605+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332605+0000","last_undegraded":"2024-06-02T23:08:11.332605+0000","last_fullsized":"2024-06-02T23:08:11.332605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.778013+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:09:39.778013+0000","last_peered":"2024-06-02T23:09:39.778013+0000","last_clean":"2024-06-02T23:09:39.778013+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:09:39.778013+0000","last_undegraded":"2024-06-02T23:09:39.778013+0000","last_fullsized":"2024-06-02T23:09:39.778013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329465+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329465+0000","last_peered":"2024-06-02T23:08:11.329465+0000","last_clean":"2024-06-02T23:08:11.329465+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329465+0000","last_undegraded":"2024-06-02T23:08:11.329465+0000","last_fullsized":"2024-06-02T23:08:11.329465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792897+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:09:39.792897+0000","last_peered":"2024-06-02T23:09:39.792897+0000","last_clean":"2024-06-02T23:09:39.792897+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:09:39.792897+0000","last_undegraded":"2024-06-02T23:09:39.792897+0000","last_fullsized":"2024-06-02T23:09:39.792897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:14.205970+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:11:14.205970+0000","last_peered":"2024-06-02T23:11:14.205970+0000","last_clean":"2024-06-02T23:11:14.205970+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:11:14.205970+0000","last_undegraded":"2024-06-02T23:11:14.205970+0000","last_fullsized":"2024-06-02T23:11:14.205970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786680+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:09:39.786680+0000","last_peered":"2024-06-02T23:09:39.786680+0000","last_clean":"2024-06-02T23:09:39.786680+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:09:39.786680+0000","last_undegraded":"2024-06-02T23:09:39.786680+0000","last_fullsized":"2024-06-02T23:09:39.786680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215918+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:14.215918+0000","last_peered":"2024-06-02T23:08:14.215918+0000","last_clean":"2024-06-02T23:08:14.215918+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:14.215918+0000","last_undegraded":"2024-06-02T23:08:14.215918+0000","last_fullsized":"2024-06-02T23:08:14.215918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800649+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:09:39.800649+0000","last_peered":"2024-06-02T23:09:39.800649+0000","last_clean":"2024-06-02T23:09:39.800649+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:09:39.800649+0000","last_undegraded":"2024-06-02T23:09:39.800649+0000","last_fullsized":"2024-06-02T23:09:39.800649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:45.676951+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:09:45.676951+0000","last_peered":"2024-06-02T23:09:45.676951+0000","last_clean":"2024-06-02T23:09:45.676951+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:09:45.676951+0000","last_undegraded":"2024-06-02T23:09:45.676951+0000","last_fullsized":"2024-06-02T23:09:45.676951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.783215+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:09:39.783215+0000","last_peered":"2024-06-02T23:09:39.783215+0000","last_clean":"2024-06-02T23:09:39.783215+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:09:39.783215+0000","last_undegraded":"2024-06-02T23:09:39.783215+0000","last_fullsized":"2024-06-02T23:09:39.783215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328539+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328539+0000","last_peered":"2024-06-02T23:08:11.328539+0000","last_clean":"2024-06-02T23:08:11.328539+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328539+0000","last_undegraded":"2024-06-02T23:08:11.328539+0000","last_fullsized":"2024-06-02T23:08:11.328539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785214+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:09:39.785214+0000","last_peered":"2024-06-02T23:09:39.785214+0000","last_clean":"2024-06-02T23:09:39.785214+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:09:39.785214+0000","last_undegraded":"2024-06-02T23:09:39.785214+0000","last_fullsized":"2024-06-02T23:09:39.785214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":234,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:09.140312+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:11:09.140312+0000","last_peered":"2024-06-02T23:11:09.140312+0000","last_clean":"2024-06-02T23:11:09.140312+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:11:09.140312+0000","last_undegraded":"2024-06-02T23:11:09.140312+0000","last_fullsized":"2024-06-02T23:11:09.140312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":201,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791877+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:09:39.791877+0000","last_peered":"2024-06-02T23:09:39.791877+0000","last_clean":"2024-06-02T23:09:39.791877+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:09:39.791877+0000","last_undegraded":"2024-06-02T23:09:39.791877+0000","last_fullsized":"2024-06-02T23:09:39.791877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792620+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:09:39.792620+0000","last_peered":"2024-06-02T23:09:39.792620+0000","last_clean":"2024-06-02T23:09:39.792620+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:09:39.792620+0000","last_undegraded":"2024-06-02T23:09:39.792620+0000","last_fullsized":"2024-06-02T23:09:39.792620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:13.203794+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:11:13.203794+0000","last_peered":"2024-06-02T23:11:13.203794+0000","last_clean":"2024-06-02T23:11:13.203794+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:11:13.203794+0000","last_undegraded":"2024-06-02T23:11:13.203794+0000","last_fullsized":"2024-06-02T23:11:13.203794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794336+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:09:39.794336+0000","last_peered":"2024-06-02T23:09:39.794336+0000","last_clean":"2024-06-02T23:09:39.794336+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:09:39.794336+0000","last_undegraded":"2024-06-02T23:09:39.794336+0000","last_fullsized":"2024-06-02T23:09:39.794336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:45.814046+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:45.814046+0000","last_peered":"2024-06-02T23:08:45.814046+0000","last_clean":"2024-06-02T23:08:45.814046+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:45.814046+0000","last_undegraded":"2024-06-02T23:08:45.814046+0000","last_fullsized":"2024-06-02T23:08:45.814046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771357+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:09:39.771357+0000","last_peered":"2024-06-02T23:09:39.771357+0000","last_clean":"2024-06-02T23:09:39.771357+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:09:39.771357+0000","last_undegraded":"2024-06-02T23:09:39.771357+0000","last_fullsized":"2024-06-02T23:09:39.771357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788062+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:09:39.788062+0000","last_peered":"2024-06-02T23:09:39.788062+0000","last_clean":"2024-06-02T23:09:39.788062+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:09:39.788062+0000","last_undegraded":"2024-06-02T23:09:39.788062+0000","last_fullsized":"2024-06-02T23:09:39.788062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'19","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.236340+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:10:54.236340+0000","last_peered":"2024-06-02T23:10:54.236340+0000","last_clean":"2024-06-02T23:10:54.236340+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:10:54.236340+0000","last_undegraded":"2024-06-02T23:10:54.236340+0000","last_fullsized":"2024-06-02T23:10:54.236340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790001+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:09:39.790001+0000","last_peered":"2024-06-02T23:09:39.790001+0000","last_clean":"2024-06-02T23:09:39.790001+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:09:39.790001+0000","last_undegraded":"2024-06-02T23:09:39.790001+0000","last_fullsized":"2024-06-02T23:09:39.790001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450463+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450463+0000","last_peered":"2024-06-02T23:08:11.450463+0000","last_clean":"2024-06-02T23:08:11.450463+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450463+0000","last_undegraded":"2024-06-02T23:08:11.450463+0000","last_fullsized":"2024-06-02T23:08:11.450463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792568+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:09:39.792568+0000","last_peered":"2024-06-02T23:09:39.792568+0000","last_clean":"2024-06-02T23:09:39.792568+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:09:39.792568+0000","last_undegraded":"2024-06-02T23:09:39.792568+0000","last_fullsized":"2024-06-02T23:09:39.792568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334494+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334494+0000","last_peered":"2024-06-02T23:08:11.334494+0000","last_clean":"2024-06-02T23:08:11.334494+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334494+0000","last_undegraded":"2024-06-02T23:08:11.334494+0000","last_fullsized":"2024-06-02T23:08:11.334494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782299+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:09:39.782299+0000","last_peered":"2024-06-02T23:09:39.782299+0000","last_clean":"2024-06-02T23:09:39.782299+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:09:39.782299+0000","last_undegraded":"2024-06-02T23:09:39.782299+0000","last_fullsized":"2024-06-02T23:09:39.782299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748925+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748925+0000","last_peered":"2024-06-02T23:08:11.748925+0000","last_clean":"2024-06-02T23:08:11.748925+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748925+0000","last_undegraded":"2024-06-02T23:08:11.748925+0000","last_fullsized":"2024-06-02T23:08:11.748925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792513+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:09:39.792513+0000","last_peered":"2024-06-02T23:09:39.792513+0000","last_clean":"2024-06-02T23:09:39.792513+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:09:39.792513+0000","last_undegraded":"2024-06-02T23:09:39.792513+0000","last_fullsized":"2024-06-02T23:09:39.792513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.275353+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:10:54.275353+0000","last_peered":"2024-06-02T23:10:54.275353+0000","last_clean":"2024-06-02T23:10:54.275353+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:10:54.275353+0000","last_undegraded":"2024-06-02T23:10:54.275353+0000","last_fullsized":"2024-06-02T23:10:54.275353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786323+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:09:39.786323+0000","last_peered":"2024-06-02T23:09:39.786323+0000","last_clean":"2024-06-02T23:09:39.786323+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:09:39.786323+0000","last_undegraded":"2024-06-02T23:09:39.786323+0000","last_fullsized":"2024-06-02T23:09:39.786323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334806+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334806+0000","last_peered":"2024-06-02T23:08:11.334806+0000","last_clean":"2024-06-02T23:08:11.334806+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334806+0000","last_undegraded":"2024-06-02T23:08:11.334806+0000","last_fullsized":"2024-06-02T23:08:11.334806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797923+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:09:39.797923+0000","last_peered":"2024-06-02T23:09:39.797923+0000","last_clean":"2024-06-02T23:09:39.797923+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:09:39.797923+0000","last_undegraded":"2024-06-02T23:09:39.797923+0000","last_fullsized":"2024-06-02T23:09:39.797923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:13.276163+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:11:13.276163+0000","last_peered":"2024-06-02T23:11:13.276163+0000","last_clean":"2024-06-02T23:11:13.276163+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:11:13.276163+0000","last_undegraded":"2024-06-02T23:11:13.276163+0000","last_fullsized":"2024-06-02T23:11:13.276163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2193,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797984+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:09:39.797984+0000","last_peered":"2024-06-02T23:09:39.797984+0000","last_clean":"2024-06-02T23:09:39.797984+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:09:39.797984+0000","last_undegraded":"2024-06-02T23:09:39.797984+0000","last_fullsized":"2024-06-02T23:09:39.797984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330301+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330301+0000","last_peered":"2024-06-02T23:08:11.330301+0000","last_clean":"2024-06-02T23:08:11.330301+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330301+0000","last_undegraded":"2024-06-02T23:08:11.330301+0000","last_fullsized":"2024-06-02T23:08:11.330301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789936+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:09:39.789936+0000","last_peered":"2024-06-02T23:09:39.789936+0000","last_clean":"2024-06-02T23:09:39.789936+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:09:39.789936+0000","last_undegraded":"2024-06-02T23:09:39.789936+0000","last_fullsized":"2024-06-02T23:09:39.789936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333595+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333595+0000","last_peered":"2024-06-02T23:08:11.333595+0000","last_clean":"2024-06-02T23:08:11.333595+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333595+0000","last_undegraded":"2024-06-02T23:08:11.333595+0000","last_fullsized":"2024-06-02T23:08:11.333595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795458+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:09:39.795458+0000","last_peered":"2024-06-02T23:09:39.795458+0000","last_clean":"2024-06-02T23:09:39.795458+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:09:39.795458+0000","last_undegraded":"2024-06-02T23:09:39.795458+0000","last_fullsized":"2024-06-02T23:09:39.795458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790898+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:09:39.790898+0000","last_peered":"2024-06-02T23:09:39.790898+0000","last_clean":"2024-06-02T23:09:39.790898+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:09:39.790898+0000","last_undegraded":"2024-06-02T23:09:39.790898+0000","last_fullsized":"2024-06-02T23:09:39.790898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785496+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:09:39.785496+0000","last_peered":"2024-06-02T23:09:39.785496+0000","last_clean":"2024-06-02T23:09:39.785496+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:09:39.785496+0000","last_undegraded":"2024-06-02T23:09:39.785496+0000","last_fullsized":"2024-06-02T23:09:39.785496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785576+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:09:39.785576+0000","last_peered":"2024-06-02T23:09:39.785576+0000","last_clean":"2024-06-02T23:09:39.785576+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:09:39.785576+0000","last_undegraded":"2024-06-02T23:09:39.785576+0000","last_fullsized":"2024-06-02T23:09:39.785576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791535+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:09:39.791535+0000","last_peered":"2024-06-02T23:09:39.791535+0000","last_clean":"2024-06-02T23:09:39.791535+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:09:39.791535+0000","last_undegraded":"2024-06-02T23:09:39.791535+0000","last_fullsized":"2024-06-02T23:09:39.791535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792313+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:09:39.792313+0000","last_peered":"2024-06-02T23:09:39.792313+0000","last_clean":"2024-06-02T23:09:39.792313+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:09:39.792313+0000","last_undegraded":"2024-06-02T23:09:39.792313+0000","last_fullsized":"2024-06-02T23:09:39.792313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.774868+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:09:39.774868+0000","last_peered":"2024-06-02T23:09:39.774868+0000","last_clean":"2024-06-02T23:09:39.774868+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:09:39.774868+0000","last_undegraded":"2024-06-02T23:09:39.774868+0000","last_fullsized":"2024-06-02T23:09:39.774868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794273+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:09:39.794273+0000","last_peered":"2024-06-02T23:09:39.794273+0000","last_clean":"2024-06-02T23:09:39.794273+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:09:39.794273+0000","last_undegraded":"2024-06-02T23:09:39.794273+0000","last_fullsized":"2024-06-02T23:09:39.794273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788292+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:09:39.788292+0000","last_peered":"2024-06-02T23:09:39.788292+0000","last_clean":"2024-06-02T23:09:39.788292+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:09:39.788292+0000","last_undegraded":"2024-06-02T23:09:39.788292+0000","last_fullsized":"2024-06-02T23:09:39.788292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786749+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:09:39.786749+0000","last_peered":"2024-06-02T23:09:39.786749+0000","last_clean":"2024-06-02T23:09:39.786749+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:09:39.786749+0000","last_undegraded":"2024-06-02T23:09:39.786749+0000","last_fullsized":"2024-06-02T23:09:39.786749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789933+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:09:39.789933+0000","last_peered":"2024-06-02T23:09:39.789933+0000","last_clean":"2024-06-02T23:09:39.789933+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:09:39.789933+0000","last_undegraded":"2024-06-02T23:09:39.789933+0000","last_fullsized":"2024-06-02T23:09:39.789933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800745+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:09:39.800745+0000","last_peered":"2024-06-02T23:09:39.800745+0000","last_clean":"2024-06-02T23:09:39.800745+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:09:39.800745+0000","last_undegraded":"2024-06-02T23:09:39.800745+0000","last_fullsized":"2024-06-02T23:09:39.800745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792409+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:09:39.792409+0000","last_peered":"2024-06-02T23:09:39.792409+0000","last_clean":"2024-06-02T23:09:39.792409+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:09:39.792409+0000","last_undegraded":"2024-06-02T23:09:39.792409+0000","last_fullsized":"2024-06-02T23:09:39.792409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.776917+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:09:39.776917+0000","last_peered":"2024-06-02T23:09:39.776917+0000","last_clean":"2024-06-02T23:09:39.776917+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:09:39.776917+0000","last_undegraded":"2024-06-02T23:09:39.776917+0000","last_fullsized":"2024-06-02T23:09:39.776917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2058,"num_read_kb":1111,"num_write":273,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":227403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645378,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1138821,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808903,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332178,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074331,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495703,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574959,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":548352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738484,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":482429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411436,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":548481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757340,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1073025,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:11:17.152 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-06-02T23:11:17.152 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph pg dump --format=json 2024-06-02T23:11:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:17 smithi052 ceph-mon[32340]: pgmap v413: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-02T23:11:17.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:17 smithi052 ceph-mon[32340]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:17.405 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:17 smithi123 ceph-mon[35977]: pgmap v413: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-02T23:11:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:17 smithi123 ceph-mon[35977]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:17.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:17 smithi174 ceph-mon[35994]: pgmap v413: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-06-02T23:11:18.052 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:17 smithi174 ceph-mon[35994]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:19.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.105+0000 7f5b2630e700 1 -- 172.21.15.52:0/2766925651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 msgr2=0x7f5b20106010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:19.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.105+0000 7f5b2630e700 1 --2- 172.21.15.52:0/2766925651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b20106010 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099c0 tx=0x7f5b14009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 -- 172.21.15.52:0/2766925651 shutdown_connections 2024-06-02T23:11:19.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 --2- 172.21.15.52:0/2766925651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b201066f0 0x7f5b2010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 --2- 172.21.15.52:0/2766925651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b20101610 0x7f5b201019f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 --2- 172.21.15.52:0/2766925651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b20106010 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 -- 172.21.15.52:0/2766925651 >> 172.21.15.52:0/2766925651 conn(0x7f5b20078430 msgr2=0x7f5b20078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:19.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 -- 172.21.15.52:0/2766925651 shutdown_connections 2024-06-02T23:11:19.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 -- 172.21.15.52:0/2766925651 wait complete. 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 Processor -- start 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.106+0000 7f5b2630e700 1 -- start start 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b2630e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b20101610 0x7f5b201030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b2630e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b201035f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b2630e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b201066f0 0x7f5b20103b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b2630e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b20111560 con 0x7f5b20101fc0 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b2630e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201113e0 con 0x7f5b20101610 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b2630e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b201116e0 con 0x7f5b201066f0 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b201035f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:19.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b201035f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36982/0 (socket says 172.21.15.52:36982) 2024-06-02T23:11:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 -- 172.21.15.52:0/3702948592 learned_addr learned my addr 172.21.15.52:0/3702948592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1ffff700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b20101610 0x7f5b201030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b248ab700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b201066f0 0x7f5b20103b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 -- 172.21.15.52:0/3702948592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b201066f0 msgr2=0x7f5b20103b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:19.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b201066f0 0x7f5b20103b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 -- 172.21.15.52:0/3702948592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b20101610 msgr2=0x7f5b201030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b20101610 0x7f5b201030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 -- 172.21.15.52:0/3702948592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b201a7100 con 0x7f5b20101fc0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.107+0000 7f5b1f7fe700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b201035f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c010 tx=0x7f5b140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.108+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14004830 con 0x7f5b20101fc0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.108+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b201a7390 con 0x7f5b20101fc0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.108+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b14005020 con 0x7f5b20101fc0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.108+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b14003dd0 con 0x7f5b20101fc0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.108+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b201a7790 con 0x7f5b20101fc0 2024-06-02T23:11:19.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.109+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b20069180 con 0x7f5b20101fc0 2024-06-02T23:11:19.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.110+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b140051d0 con 0x7f5b20101fc0 2024-06-02T23:11:19.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.111+0000 7f5b1d7fa700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b080a2c20 0x7f5b080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:19.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.111+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b140d0500 con 0x7f5b20101fc0 2024-06-02T23:11:19.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.111+0000 7f5b1ffff700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b080a2c20 0x7f5b080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.111+0000 7f5b1ffff700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b080a2c20 0x7f5b080a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005950 tx=0x7f5b100058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:19.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.114+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b14013070 con 0x7f5b20101fc0 2024-06-02T23:11:19.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.319+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 --> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5b2002ddb0 con 0x7f5b080a2c20 2024-06-02T23:11:19.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.326+0000 7f5b1d7fa700 1 -- 172.21.15.52:0/3702948592 <== mgr.14150 v2:172.21.15.52:6800/2596667410 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259068 (secure 0 0 0) 0x7f5b2002ddb0 con 0x7f5b080a2c20 2024-06-02T23:11:19.330 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.329+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b080a2c20 msgr2=0x7f5b080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.329+0000 7f5b2630e700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b080a2c20 0x7f5b080a50e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b10005950 tx=0x7f5b100058e0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.329+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 msgr2=0x7f5b201035f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.329+0000 7f5b2630e700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b201035f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5b1400c010 tx=0x7f5b140047b0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 shutdown_connections 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b080a2c20 0x7f5b080a50e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b201066f0 0x7f5b20103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b20101610 0x7f5b201030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 --2- 172.21.15.52:0/3702948592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b20101fc0 0x7f5b201035f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:19.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 >> 172.21.15.52:0/3702948592 conn(0x7f5b20078430 msgr2=0x7f5b2006afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:19.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 shutdown_connections 2024-06-02T23:11:19.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:19.330+0000 7f5b2630e700 1 -- 172.21.15.52:0/3702948592 wait complete. 2024-06-02T23:11:19.333 INFO:teuthology.orchestra.run.smithi052.stderr:dumped all 2024-06-02T23:11:19.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:19 smithi123 ceph-mon[35977]: pgmap v414: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-02T23:11:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:19 smithi174 ceph-mon[35994]: pgmap v414: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-02T23:11:19.643 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:19 smithi052 ceph-mon[32340]: pgmap v414: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-06-02T23:11:20.016 INFO:teuthology.orchestra.run.smithi052.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-06-02T23:11:17.817341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666403,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":2214,"num_read_kb":1240,"num_write":456,"num_write_kb":3100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":1181470,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":871,"ondisk_log_size":871,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369640,"kb_used_data":12168,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572184,"statfs":{"total":1151940427776,"available":1151561916416,"internally_reserved":0,"allocated":12460032,"data_stored":7749945,"data_compressed":108420,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19141,"internal_metadata":365933883},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":132,"num_read_kb":72,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003227"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802175+0000","last_change":"2024-06-02T23:07:31.213251+0000","last_active":"2024-06-02T23:09:39.802175+0000","last_peered":"2024-06-02T23:09:39.802175+0000","last_clean":"2024-06-02T23:09:39.802175+0000","last_became_active":"2024-06-02T23:07:31.212813+0000","last_became_peered":"2024-06-02T23:07:31.212813+0000","last_unstale":"2024-06-02T23:09:39.802175+0000","last_undegraded":"2024-06-02T23:09:39.802175+0000","last_fullsized":"2024-06-02T23:09:39.802175+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:49:03.088695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788231+0000","last_change":"2024-06-02T23:07:31.213779+0000","last_active":"2024-06-02T23:09:39.788231+0000","last_peered":"2024-06-02T23:09:39.788231+0000","last_clean":"2024-06-02T23:09:39.788231+0000","last_became_active":"2024-06-02T23:07:31.213453+0000","last_became_peered":"2024-06-02T23:07:31.213453+0000","last_unstale":"2024-06-02T23:09:39.788231+0000","last_undegraded":"2024-06-02T23:09:39.788231+0000","last_fullsized":"2024-06-02T23:09:39.788231+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:33:37.209812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794300+0000","last_change":"2024-06-02T23:07:31.221728+0000","last_active":"2024-06-02T23:09:39.794300+0000","last_peered":"2024-06-02T23:09:39.794300+0000","last_clean":"2024-06-02T23:09:39.794300+0000","last_became_active":"2024-06-02T23:07:31.221393+0000","last_became_peered":"2024-06-02T23:07:31.221393+0000","last_unstale":"2024-06-02T23:09:39.794300+0000","last_undegraded":"2024-06-02T23:09:39.794300+0000","last_fullsized":"2024-06-02T23:09:39.794300+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:35:57.239494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788417+0000","last_change":"2024-06-02T23:07:31.220578+0000","last_active":"2024-06-02T23:09:39.788417+0000","last_peered":"2024-06-02T23:09:39.788417+0000","last_clean":"2024-06-02T23:09:39.788417+0000","last_became_active":"2024-06-02T23:07:31.220250+0000","last_became_peered":"2024-06-02T23:07:31.220250+0000","last_unstale":"2024-06-02T23:09:39.788417+0000","last_undegraded":"2024-06-02T23:09:39.788417+0000","last_fullsized":"2024-06-02T23:09:39.788417+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:40:46.733488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":25,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.777964+0000","last_change":"2024-06-02T23:07:31.206709+0000","last_active":"2024-06-02T23:09:39.777964+0000","last_peered":"2024-06-02T23:09:39.777964+0000","last_clean":"2024-06-02T23:09:39.777964+0000","last_became_active":"2024-06-02T23:07:31.206374+0000","last_became_peered":"2024-06-02T23:07:31.206374+0000","last_unstale":"2024-06-02T23:09:39.777964+0000","last_undegraded":"2024-06-02T23:09:39.777964+0000","last_fullsized":"2024-06-02T23:09:39.777964+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:00.371323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782448+0000","last_change":"2024-06-02T23:07:31.216015+0000","last_active":"2024-06-02T23:09:39.782448+0000","last_peered":"2024-06-02T23:09:39.782448+0000","last_clean":"2024-06-02T23:09:39.782448+0000","last_became_active":"2024-06-02T23:07:31.214633+0000","last_became_peered":"2024-06-02T23:07:31.214633+0000","last_unstale":"2024-06-02T23:09:39.782448+0000","last_undegraded":"2024-06-02T23:09:39.782448+0000","last_fullsized":"2024-06-02T23:09:39.782448+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:15:00.737637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786918+0000","last_change":"2024-06-02T23:07:31.218741+0000","last_active":"2024-06-02T23:09:39.786918+0000","last_peered":"2024-06-02T23:09:39.786918+0000","last_clean":"2024-06-02T23:09:39.786918+0000","last_became_active":"2024-06-02T23:07:31.218392+0000","last_became_peered":"2024-06-02T23:07:31.218392+0000","last_unstale":"2024-06-02T23:09:39.786918+0000","last_undegraded":"2024-06-02T23:09:39.786918+0000","last_fullsized":"2024-06-02T23:09:39.786918+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:14:12.435391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786735+0000","last_change":"2024-06-02T23:07:31.205363+0000","last_active":"2024-06-02T23:09:39.786735+0000","last_peered":"2024-06-02T23:09:39.786735+0000","last_clean":"2024-06-02T23:09:39.786735+0000","last_became_active":"2024-06-02T23:07:31.204993+0000","last_became_peered":"2024-06-02T23:07:31.204993+0000","last_unstale":"2024-06-02T23:09:39.786735+0000","last_undegraded":"2024-06-02T23:09:39.786735+0000","last_fullsized":"2024-06-02T23:09:39.786735+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:05:01.639971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801527+0000","last_change":"2024-06-02T23:07:31.211770+0000","last_active":"2024-06-02T23:09:39.801527+0000","last_peered":"2024-06-02T23:09:39.801527+0000","last_clean":"2024-06-02T23:09:39.801527+0000","last_became_active":"2024-06-02T23:07:31.211342+0000","last_became_peered":"2024-06-02T23:07:31.211342+0000","last_unstale":"2024-06-02T23:09:39.801527+0000","last_undegraded":"2024-06-02T23:09:39.801527+0000","last_fullsized":"2024-06-02T23:09:39.801527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:49:42.532946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789699+0000","last_change":"2024-06-02T23:07:31.215615+0000","last_active":"2024-06-02T23:09:39.789699+0000","last_peered":"2024-06-02T23:09:39.789699+0000","last_clean":"2024-06-02T23:09:39.789699+0000","last_became_active":"2024-06-02T23:07:31.215267+0000","last_became_peered":"2024-06-02T23:07:31.215267+0000","last_unstale":"2024-06-02T23:09:39.789699+0000","last_undegraded":"2024-06-02T23:09:39.789699+0000","last_fullsized":"2024-06-02T23:09:39.789699+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:34:36.746763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797457+0000","last_change":"2024-06-02T23:07:31.222344+0000","last_active":"2024-06-02T23:09:39.797457+0000","last_peered":"2024-06-02T23:09:39.797457+0000","last_clean":"2024-06-02T23:09:39.797457+0000","last_became_active":"2024-06-02T23:07:31.221191+0000","last_became_peered":"2024-06-02T23:07:31.221191+0000","last_unstale":"2024-06-02T23:09:39.797457+0000","last_undegraded":"2024-06-02T23:09:39.797457+0000","last_fullsized":"2024-06-02T23:09:39.797457+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:26:47.569381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.787757+0000","last_change":"2024-06-02T23:07:31.212311+0000","last_active":"2024-06-02T23:09:39.787757+0000","last_peered":"2024-06-02T23:09:39.787757+0000","last_clean":"2024-06-02T23:09:39.787757+0000","last_became_active":"2024-06-02T23:07:31.211966+0000","last_became_peered":"2024-06-02T23:07:31.211966+0000","last_unstale":"2024-06-02T23:09:39.787757+0000","last_undegraded":"2024-06-02T23:09:39.787757+0000","last_fullsized":"2024-06-02T23:09:39.787757+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:35:10.479909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786117+0000","last_change":"2024-06-02T23:07:31.212449+0000","last_active":"2024-06-02T23:09:39.786117+0000","last_peered":"2024-06-02T23:09:39.786117+0000","last_clean":"2024-06-02T23:09:39.786117+0000","last_became_active":"2024-06-02T23:07:31.212115+0000","last_became_peered":"2024-06-02T23:07:31.212115+0000","last_unstale":"2024-06-02T23:09:39.786117+0000","last_undegraded":"2024-06-02T23:09:39.786117+0000","last_fullsized":"2024-06-02T23:09:39.786117+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:53:14.902952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788799+0000","last_change":"2024-06-02T23:07:31.213791+0000","last_active":"2024-06-02T23:09:39.788799+0000","last_peered":"2024-06-02T23:09:39.788799+0000","last_clean":"2024-06-02T23:09:39.788799+0000","last_became_active":"2024-06-02T23:07:31.213478+0000","last_became_peered":"2024-06-02T23:07:31.213478+0000","last_unstale":"2024-06-02T23:09:39.788799+0000","last_undegraded":"2024-06-02T23:09:39.788799+0000","last_fullsized":"2024-06-02T23:09:39.788799+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:43:18.002209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791625+0000","last_change":"2024-06-02T23:07:31.214597+0000","last_active":"2024-06-02T23:09:39.791625+0000","last_peered":"2024-06-02T23:09:39.791625+0000","last_clean":"2024-06-02T23:09:39.791625+0000","last_became_active":"2024-06-02T23:07:31.214277+0000","last_became_peered":"2024-06-02T23:07:31.214277+0000","last_unstale":"2024-06-02T23:09:39.791625+0000","last_undegraded":"2024-06-02T23:09:39.791625+0000","last_fullsized":"2024-06-02T23:09:39.791625+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:37:21.576732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800433+0000","last_change":"2024-06-02T23:07:31.217966+0000","last_active":"2024-06-02T23:09:39.800433+0000","last_peered":"2024-06-02T23:09:39.800433+0000","last_clean":"2024-06-02T23:09:39.800433+0000","last_became_active":"2024-06-02T23:07:31.217631+0000","last_became_peered":"2024-06-02T23:07:31.217631+0000","last_unstale":"2024-06-02T23:09:39.800433+0000","last_undegraded":"2024-06-02T23:09:39.800433+0000","last_fullsized":"2024-06-02T23:09:39.800433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:47:26.828956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802531+0000","last_change":"2024-06-02T23:07:31.216561+0000","last_active":"2024-06-02T23:09:39.802531+0000","last_peered":"2024-06-02T23:09:39.802531+0000","last_clean":"2024-06-02T23:09:39.802531+0000","last_became_active":"2024-06-02T23:07:31.216231+0000","last_became_peered":"2024-06-02T23:07:31.216231+0000","last_unstale":"2024-06-02T23:09:39.802531+0000","last_undegraded":"2024-06-02T23:09:39.802531+0000","last_fullsized":"2024-06-02T23:09:39.802531+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:25:23.487394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.803680+0000","last_change":"2024-06-02T23:07:31.212472+0000","last_active":"2024-06-02T23:09:39.803680+0000","last_peered":"2024-06-02T23:09:39.803680+0000","last_clean":"2024-06-02T23:09:39.803680+0000","last_became_active":"2024-06-02T23:07:31.212110+0000","last_became_peered":"2024-06-02T23:07:31.212110+0000","last_unstale":"2024-06-02T23:09:39.803680+0000","last_undegraded":"2024-06-02T23:09:39.803680+0000","last_fullsized":"2024-06-02T23:09:39.803680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:57:30.442709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790081+0000","last_change":"2024-06-02T23:07:31.220793+0000","last_active":"2024-06-02T23:09:39.790081+0000","last_peered":"2024-06-02T23:09:39.790081+0000","last_clean":"2024-06-02T23:09:39.790081+0000","last_became_active":"2024-06-02T23:07:31.220243+0000","last_became_peered":"2024-06-02T23:07:31.220243+0000","last_unstale":"2024-06-02T23:09:39.790081+0000","last_undegraded":"2024-06-02T23:09:39.790081+0000","last_fullsized":"2024-06-02T23:09:39.790081+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:45:37.893518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":248,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:15.146424+0000","last_change":"2024-06-02T23:07:27.861800+0000","last_active":"2024-06-02T23:11:15.146424+0000","last_peered":"2024-06-02T23:11:15.146424+0000","last_clean":"2024-06-02T23:11:15.146424+0000","last_became_active":"2024-06-02T23:07:27.861565+0000","last_became_peered":"2024-06-02T23:07:27.861565+0000","last_unstale":"2024-06-02T23:11:15.146424+0000","last_undegraded":"2024-06-02T23:11:15.146424+0000","last_fullsized":"2024-06-02T23:11:15.146424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:31:13.830416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":207,"num_read_kb":207,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788800+0000","last_change":"2024-06-02T23:07:31.216508+0000","last_active":"2024-06-02T23:09:39.788800+0000","last_peered":"2024-06-02T23:09:39.788800+0000","last_clean":"2024-06-02T23:09:39.788800+0000","last_became_active":"2024-06-02T23:07:31.216116+0000","last_became_peered":"2024-06-02T23:07:31.216116+0000","last_unstale":"2024-06-02T23:09:39.788800+0000","last_undegraded":"2024-06-02T23:09:39.788800+0000","last_fullsized":"2024-06-02T23:09:39.788800+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:13:22.802199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.749178+0000","last_change":"2024-06-02T23:07:27.988185+0000","last_active":"2024-06-02T23:08:11.749178+0000","last_peered":"2024-06-02T23:08:11.749178+0000","last_clean":"2024-06-02T23:08:11.749178+0000","last_became_active":"2024-06-02T23:07:27.987751+0000","last_became_peered":"2024-06-02T23:07:27.987751+0000","last_unstale":"2024-06-02T23:08:11.749178+0000","last_undegraded":"2024-06-02T23:08:11.749178+0000","last_fullsized":"2024-06-02T23:08:11.749178+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:32:21.050480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794320+0000","last_change":"2024-06-02T23:07:31.206845+0000","last_active":"2024-06-02T23:09:39.794320+0000","last_peered":"2024-06-02T23:09:39.794320+0000","last_clean":"2024-06-02T23:09:39.794320+0000","last_became_active":"2024-06-02T23:07:31.205949+0000","last_became_peered":"2024-06-02T23:07:31.205949+0000","last_unstale":"2024-06-02T23:09:39.794320+0000","last_undegraded":"2024-06-02T23:09:39.794320+0000","last_fullsized":"2024-06-02T23:09:39.794320+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:27:12.030544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215945+0000","last_change":"2024-06-02T23:08:11.365191+0000","last_active":"2024-06-02T23:08:14.215945+0000","last_peered":"2024-06-02T23:08:14.215945+0000","last_clean":"2024-06-02T23:08:14.215945+0000","last_became_active":"2024-06-02T23:08:11.336186+0000","last_became_peered":"2024-06-02T23:08:11.336186+0000","last_unstale":"2024-06-02T23:08:14.215945+0000","last_undegraded":"2024-06-02T23:08:14.215945+0000","last_fullsized":"2024-06-02T23:08:14.215945+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:33:51.721584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802474+0000","last_change":"2024-06-02T23:07:31.212673+0000","last_active":"2024-06-02T23:09:39.802474+0000","last_peered":"2024-06-02T23:09:39.802474+0000","last_clean":"2024-06-02T23:09:39.802474+0000","last_became_active":"2024-06-02T23:07:31.212300+0000","last_became_peered":"2024-06-02T23:07:31.212300+0000","last_unstale":"2024-06-02T23:09:39.802474+0000","last_undegraded":"2024-06-02T23:09:39.802474+0000","last_fullsized":"2024-06-02T23:09:39.802474+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:56:36.768089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335772+0000","last_change":"2024-06-02T23:07:27.860138+0000","last_active":"2024-06-02T23:08:11.335772+0000","last_peered":"2024-06-02T23:08:11.335772+0000","last_clean":"2024-06-02T23:08:11.335772+0000","last_became_active":"2024-06-02T23:07:27.859714+0000","last_became_peered":"2024-06-02T23:07:27.859714+0000","last_unstale":"2024-06-02T23:08:11.335772+0000","last_undegraded":"2024-06-02T23:08:11.335772+0000","last_fullsized":"2024-06-02T23:08:11.335772+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:06.612084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800976+0000","last_change":"2024-06-02T23:07:31.210692+0000","last_active":"2024-06-02T23:09:39.800976+0000","last_peered":"2024-06-02T23:09:39.800976+0000","last_clean":"2024-06-02T23:09:39.800976+0000","last_became_active":"2024-06-02T23:07:31.210252+0000","last_became_peered":"2024-06-02T23:07:31.210252+0000","last_unstale":"2024-06-02T23:09:39.800976+0000","last_undegraded":"2024-06-02T23:09:39.800976+0000","last_fullsized":"2024-06-02T23:09:39.800976+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:36:48.358686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"90'6","reported_seq":250,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:12.710528+0000","last_change":"2024-06-02T23:08:11.454887+0000","last_active":"2024-06-02T23:11:12.710528+0000","last_peered":"2024-06-02T23:11:12.710528+0000","last_clean":"2024-06-02T23:11:12.710528+0000","last_became_active":"2024-06-02T23:08:11.451055+0000","last_became_peered":"2024-06-02T23:08:11.451055+0000","last_unstale":"2024-06-02T23:11:12.710528+0000","last_undegraded":"2024-06-02T23:11:12.710528+0000","last_fullsized":"2024-06-02T23:11:12.710528+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:15:56.577963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1114,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":622,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771608+0000","last_change":"2024-06-02T23:07:31.207770+0000","last_active":"2024-06-02T23:09:39.771608+0000","last_peered":"2024-06-02T23:09:39.771608+0000","last_clean":"2024-06-02T23:09:39.771608+0000","last_became_active":"2024-06-02T23:07:31.207432+0000","last_became_peered":"2024-06-02T23:07:31.207432+0000","last_unstale":"2024-06-02T23:09:39.771608+0000","last_undegraded":"2024-06-02T23:09:39.771608+0000","last_fullsized":"2024-06-02T23:09:39.771608+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:32:11.146968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.449603+0000","last_change":"2024-06-02T23:07:27.862839+0000","last_active":"2024-06-02T23:08:11.449603+0000","last_peered":"2024-06-02T23:08:11.449603+0000","last_clean":"2024-06-02T23:08:11.449603+0000","last_became_active":"2024-06-02T23:07:27.860698+0000","last_became_peered":"2024-06-02T23:07:27.860698+0000","last_unstale":"2024-06-02T23:08:11.449603+0000","last_undegraded":"2024-06-02T23:08:11.449603+0000","last_fullsized":"2024-06-02T23:08:11.449603+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:11:33.450048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795377+0000","last_change":"2024-06-02T23:07:31.197354+0000","last_active":"2024-06-02T23:09:39.795377+0000","last_peered":"2024-06-02T23:09:39.795377+0000","last_clean":"2024-06-02T23:09:39.795377+0000","last_became_active":"2024-06-02T23:07:31.196998+0000","last_became_peered":"2024-06-02T23:07:31.196998+0000","last_unstale":"2024-06-02T23:09:39.795377+0000","last_undegraded":"2024-06-02T23:09:39.795377+0000","last_fullsized":"2024-06-02T23:09:39.795377+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:29:20.244458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332946+0000","last_change":"2024-06-02T23:07:27.857371+0000","last_active":"2024-06-02T23:08:11.332946+0000","last_peered":"2024-06-02T23:08:11.332946+0000","last_clean":"2024-06-02T23:08:11.332946+0000","last_became_active":"2024-06-02T23:07:27.856599+0000","last_became_peered":"2024-06-02T23:07:27.856599+0000","last_unstale":"2024-06-02T23:08:11.332946+0000","last_undegraded":"2024-06-02T23:08:11.332946+0000","last_fullsized":"2024-06-02T23:08:11.332946+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:37:33.173827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785226+0000","last_change":"2024-06-02T23:07:31.206865+0000","last_active":"2024-06-02T23:09:39.785226+0000","last_peered":"2024-06-02T23:09:39.785226+0000","last_clean":"2024-06-02T23:09:39.785226+0000","last_became_active":"2024-06-02T23:07:31.206582+0000","last_became_peered":"2024-06-02T23:07:31.206582+0000","last_unstale":"2024-06-02T23:09:39.785226+0000","last_undegraded":"2024-06-02T23:09:39.785226+0000","last_fullsized":"2024-06-02T23:09:39.785226+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:35.224419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329286+0000","last_change":"2024-06-02T23:07:27.859995+0000","last_active":"2024-06-02T23:08:11.329286+0000","last_peered":"2024-06-02T23:08:11.329286+0000","last_clean":"2024-06-02T23:08:11.329286+0000","last_became_active":"2024-06-02T23:07:27.859641+0000","last_became_peered":"2024-06-02T23:07:27.859641+0000","last_unstale":"2024-06-02T23:08:11.329286+0000","last_undegraded":"2024-06-02T23:08:11.329286+0000","last_fullsized":"2024-06-02T23:08:11.329286+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:18:41.724725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.779022+0000","last_change":"2024-06-02T23:07:31.216529+0000","last_active":"2024-06-02T23:09:39.779022+0000","last_peered":"2024-06-02T23:09:39.779022+0000","last_clean":"2024-06-02T23:09:39.779022+0000","last_became_active":"2024-06-02T23:07:31.216158+0000","last_became_peered":"2024-06-02T23:07:31.216158+0000","last_unstale":"2024-06-02T23:09:39.779022+0000","last_undegraded":"2024-06-02T23:09:39.779022+0000","last_fullsized":"2024-06-02T23:09:39.779022+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:34:38.751247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"90'7","reported_seq":445,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:16.716267+0000","last_change":"2024-06-02T23:07:27.863577+0000","last_active":"2024-06-02T23:11:16.716267+0000","last_peered":"2024-06-02T23:11:16.716267+0000","last_clean":"2024-06-02T23:11:16.716267+0000","last_became_active":"2024-06-02T23:07:27.863184+0000","last_became_peered":"2024-06-02T23:07:27.863184+0000","last_unstale":"2024-06-02T23:11:16.716267+0000","last_undegraded":"2024-06-02T23:11:16.716267+0000","last_fullsized":"2024-06-02T23:11:16.716267+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:38:20.723060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":411,"num_read_kb":411,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.802707+0000","last_change":"2024-06-02T23:07:31.213711+0000","last_active":"2024-06-02T23:09:39.802707+0000","last_peered":"2024-06-02T23:09:39.802707+0000","last_clean":"2024-06-02T23:09:39.802707+0000","last_became_active":"2024-06-02T23:07:31.213179+0000","last_became_peered":"2024-06-02T23:07:31.213179+0000","last_unstale":"2024-06-02T23:09:39.802707+0000","last_undegraded":"2024-06-02T23:09:39.802707+0000","last_fullsized":"2024-06-02T23:09:39.802707+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:25:12.790240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333251+0000","last_change":"2024-06-02T23:07:27.861952+0000","last_active":"2024-06-02T23:08:11.333251+0000","last_peered":"2024-06-02T23:08:11.333251+0000","last_clean":"2024-06-02T23:08:11.333251+0000","last_became_active":"2024-06-02T23:07:27.861573+0000","last_became_peered":"2024-06-02T23:07:27.861573+0000","last_unstale":"2024-06-02T23:08:11.333251+0000","last_undegraded":"2024-06-02T23:08:11.333251+0000","last_fullsized":"2024-06-02T23:08:11.333251+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:00:20.366248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791331+0000","last_change":"2024-06-02T23:07:31.208956+0000","last_active":"2024-06-02T23:09:39.791331+0000","last_peered":"2024-06-02T23:09:39.791331+0000","last_clean":"2024-06-02T23:09:39.791331+0000","last_became_active":"2024-06-02T23:07:31.208625+0000","last_became_peered":"2024-06-02T23:07:31.208625+0000","last_unstale":"2024-06-02T23:09:39.791331+0000","last_undegraded":"2024-06-02T23:09:39.791331+0000","last_fullsized":"2024-06-02T23:09:39.791331+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:38:17.229333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330069+0000","last_change":"2024-06-02T23:07:27.990313+0000","last_active":"2024-06-02T23:08:11.330069+0000","last_peered":"2024-06-02T23:08:11.330069+0000","last_clean":"2024-06-02T23:08:11.330069+0000","last_became_active":"2024-06-02T23:07:27.989909+0000","last_became_peered":"2024-06-02T23:07:27.989909+0000","last_unstale":"2024-06-02T23:08:11.330069+0000","last_undegraded":"2024-06-02T23:08:11.330069+0000","last_fullsized":"2024-06-02T23:08:11.330069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:17:45.283216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.801687+0000","last_change":"2024-06-02T23:07:31.215250+0000","last_active":"2024-06-02T23:09:39.801687+0000","last_peered":"2024-06-02T23:09:39.801687+0000","last_clean":"2024-06-02T23:09:39.801687+0000","last_became_active":"2024-06-02T23:07:31.214912+0000","last_became_peered":"2024-06-02T23:07:31.214912+0000","last_unstale":"2024-06-02T23:09:39.801687+0000","last_undegraded":"2024-06-02T23:09:39.801687+0000","last_fullsized":"2024-06-02T23:09:39.801687+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:21:55.928664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"90'37","reported_seq":487,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:16.716312+0000","last_change":"2024-06-02T23:07:27.864566+0000","last_active":"2024-06-02T23:11:16.716312+0000","last_peered":"2024-06-02T23:11:16.716312+0000","last_clean":"2024-06-02T23:11:16.716312+0000","last_became_active":"2024-06-02T23:07:27.864221+0000","last_became_peered":"2024-06-02T23:07:27.864221+0000","last_unstale":"2024-06-02T23:11:16.716312+0000","last_undegraded":"2024-06-02T23:11:16.716312+0000","last_fullsized":"2024-06-02T23:11:16.716312+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:35.910760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66880,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":423,"num_read_kb":276,"num_write":39,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794578+0000","last_change":"2024-06-02T23:07:31.209879+0000","last_active":"2024-06-02T23:09:39.794578+0000","last_peered":"2024-06-02T23:09:39.794578+0000","last_clean":"2024-06-02T23:09:39.794578+0000","last_became_active":"2024-06-02T23:07:31.209558+0000","last_became_peered":"2024-06-02T23:07:31.209558+0000","last_unstale":"2024-06-02T23:09:39.794578+0000","last_undegraded":"2024-06-02T23:09:39.794578+0000","last_fullsized":"2024-06-02T23:09:39.794578+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:43:17.449966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.332605+0000","last_change":"2024-06-02T23:07:27.860256+0000","last_active":"2024-06-02T23:08:11.332605+0000","last_peered":"2024-06-02T23:08:11.332605+0000","last_clean":"2024-06-02T23:08:11.332605+0000","last_became_active":"2024-06-02T23:07:27.859790+0000","last_became_peered":"2024-06-02T23:07:27.859790+0000","last_unstale":"2024-06-02T23:08:11.332605+0000","last_undegraded":"2024-06-02T23:08:11.332605+0000","last_fullsized":"2024-06-02T23:08:11.332605+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T11:01:28.795943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.778013+0000","last_change":"2024-06-02T23:07:31.209626+0000","last_active":"2024-06-02T23:09:39.778013+0000","last_peered":"2024-06-02T23:09:39.778013+0000","last_clean":"2024-06-02T23:09:39.778013+0000","last_became_active":"2024-06-02T23:07:31.209210+0000","last_became_peered":"2024-06-02T23:07:31.209210+0000","last_unstale":"2024-06-02T23:09:39.778013+0000","last_undegraded":"2024-06-02T23:09:39.778013+0000","last_fullsized":"2024-06-02T23:09:39.778013+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:04:09.231958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.329465+0000","last_change":"2024-06-02T23:07:27.861557+0000","last_active":"2024-06-02T23:08:11.329465+0000","last_peered":"2024-06-02T23:08:11.329465+0000","last_clean":"2024-06-02T23:08:11.329465+0000","last_became_active":"2024-06-02T23:07:27.861159+0000","last_became_peered":"2024-06-02T23:07:27.861159+0000","last_unstale":"2024-06-02T23:08:11.329465+0000","last_undegraded":"2024-06-02T23:08:11.329465+0000","last_fullsized":"2024-06-02T23:08:11.329465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:17:43.574969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792897+0000","last_change":"2024-06-02T23:07:31.215983+0000","last_active":"2024-06-02T23:09:39.792897+0000","last_peered":"2024-06-02T23:09:39.792897+0000","last_clean":"2024-06-02T23:09:39.792897+0000","last_became_active":"2024-06-02T23:07:31.215439+0000","last_became_peered":"2024-06-02T23:07:31.215439+0000","last_unstale":"2024-06-02T23:09:39.792897+0000","last_undegraded":"2024-06-02T23:09:39.792897+0000","last_fullsized":"2024-06-02T23:09:39.792897+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:50:55.557883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"90'6","reported_seq":240,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:14.205970+0000","last_change":"2024-06-02T23:07:27.861566+0000","last_active":"2024-06-02T23:11:14.205970+0000","last_peered":"2024-06-02T23:11:14.205970+0000","last_clean":"2024-06-02T23:11:14.205970+0000","last_became_active":"2024-06-02T23:07:27.861019+0000","last_became_peered":"2024-06-02T23:07:27.861019+0000","last_unstale":"2024-06-02T23:11:14.205970+0000","last_undegraded":"2024-06-02T23:11:14.205970+0000","last_fullsized":"2024-06-02T23:11:14.205970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:46.964527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786680+0000","last_change":"2024-06-02T23:07:31.219543+0000","last_active":"2024-06-02T23:09:39.786680+0000","last_peered":"2024-06-02T23:09:39.786680+0000","last_clean":"2024-06-02T23:09:39.786680+0000","last_became_active":"2024-06-02T23:07:31.219180+0000","last_became_peered":"2024-06-02T23:07:31.219180+0000","last_unstale":"2024-06-02T23:09:39.786680+0000","last_undegraded":"2024-06-02T23:09:39.786680+0000","last_fullsized":"2024-06-02T23:09:39.786680+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:40:04.555178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:14.215918+0000","last_change":"2024-06-02T23:08:11.348496+0000","last_active":"2024-06-02T23:08:14.215918+0000","last_peered":"2024-06-02T23:08:14.215918+0000","last_clean":"2024-06-02T23:08:14.215918+0000","last_became_active":"2024-06-02T23:08:11.334563+0000","last_became_peered":"2024-06-02T23:08:11.334563+0000","last_unstale":"2024-06-02T23:08:14.215918+0000","last_undegraded":"2024-06-02T23:08:14.215918+0000","last_fullsized":"2024-06-02T23:08:14.215918+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:03.881940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800649+0000","last_change":"2024-06-02T23:07:31.204769+0000","last_active":"2024-06-02T23:09:39.800649+0000","last_peered":"2024-06-02T23:09:39.800649+0000","last_clean":"2024-06-02T23:09:39.800649+0000","last_became_active":"2024-06-02T23:07:31.204286+0000","last_became_peered":"2024-06-02T23:07:31.204286+0000","last_unstale":"2024-06-02T23:09:39.800649+0000","last_undegraded":"2024-06-02T23:09:39.800649+0000","last_fullsized":"2024-06-02T23:09:39.800649+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:18:52.954139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:45.676951+0000","last_change":"2024-06-02T23:07:27.865763+0000","last_active":"2024-06-02T23:09:45.676951+0000","last_peered":"2024-06-02T23:09:45.676951+0000","last_clean":"2024-06-02T23:09:45.676951+0000","last_became_active":"2024-06-02T23:07:27.865395+0000","last_became_peered":"2024-06-02T23:07:27.865395+0000","last_unstale":"2024-06-02T23:09:45.676951+0000","last_undegraded":"2024-06-02T23:09:45.676951+0000","last_fullsized":"2024-06-02T23:09:45.676951+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:13:14.986567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.783215+0000","last_change":"2024-06-02T23:07:31.215846+0000","last_active":"2024-06-02T23:09:39.783215+0000","last_peered":"2024-06-02T23:09:39.783215+0000","last_clean":"2024-06-02T23:09:39.783215+0000","last_became_active":"2024-06-02T23:07:31.215521+0000","last_became_peered":"2024-06-02T23:07:31.215521+0000","last_unstale":"2024-06-02T23:09:39.783215+0000","last_undegraded":"2024-06-02T23:09:39.783215+0000","last_fullsized":"2024-06-02T23:09:39.783215+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:24:46.134227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.328539+0000","last_change":"2024-06-02T23:07:27.988502+0000","last_active":"2024-06-02T23:08:11.328539+0000","last_peered":"2024-06-02T23:08:11.328539+0000","last_clean":"2024-06-02T23:08:11.328539+0000","last_became_active":"2024-06-02T23:07:27.988081+0000","last_became_peered":"2024-06-02T23:07:27.988081+0000","last_unstale":"2024-06-02T23:08:11.328539+0000","last_undegraded":"2024-06-02T23:08:11.328539+0000","last_fullsized":"2024-06-02T23:08:11.328539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:51:59.755886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785214+0000","last_change":"2024-06-02T23:07:31.210928+0000","last_active":"2024-06-02T23:09:39.785214+0000","last_peered":"2024-06-02T23:09:39.785214+0000","last_clean":"2024-06-02T23:09:39.785214+0000","last_became_active":"2024-06-02T23:07:31.210554+0000","last_became_peered":"2024-06-02T23:07:31.210554+0000","last_unstale":"2024-06-02T23:09:39.785214+0000","last_undegraded":"2024-06-02T23:09:39.785214+0000","last_fullsized":"2024-06-02T23:09:39.785214+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:41:17.135529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"90'6","reported_seq":239,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:14.145909+0000","last_change":"2024-06-02T23:07:27.859051+0000","last_active":"2024-06-02T23:11:14.145909+0000","last_peered":"2024-06-02T23:11:14.145909+0000","last_clean":"2024-06-02T23:11:14.145909+0000","last_became_active":"2024-06-02T23:07:27.858514+0000","last_became_peered":"2024-06-02T23:07:27.858514+0000","last_unstale":"2024-06-02T23:11:14.145909+0000","last_undegraded":"2024-06-02T23:11:14.145909+0000","last_fullsized":"2024-06-02T23:11:14.145909+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:14:44.376536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1080,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791877+0000","last_change":"2024-06-02T23:07:31.216415+0000","last_active":"2024-06-02T23:09:39.791877+0000","last_peered":"2024-06-02T23:09:39.791877+0000","last_clean":"2024-06-02T23:09:39.791877+0000","last_became_active":"2024-06-02T23:07:31.216074+0000","last_became_peered":"2024-06-02T23:07:31.216074+0000","last_unstale":"2024-06-02T23:09:39.791877+0000","last_undegraded":"2024-06-02T23:09:39.791877+0000","last_fullsized":"2024-06-02T23:09:39.791877+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:23:58.319833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792620+0000","last_change":"2024-06-02T23:07:31.208172+0000","last_active":"2024-06-02T23:09:39.792620+0000","last_peered":"2024-06-02T23:09:39.792620+0000","last_clean":"2024-06-02T23:09:39.792620+0000","last_became_active":"2024-06-02T23:07:31.207728+0000","last_became_peered":"2024-06-02T23:07:31.207728+0000","last_unstale":"2024-06-02T23:09:39.792620+0000","last_undegraded":"2024-06-02T23:09:39.792620+0000","last_fullsized":"2024-06-02T23:09:39.792620+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:21:17.331601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":237,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:13.203794+0000","last_change":"2024-06-02T23:07:27.988024+0000","last_active":"2024-06-02T23:11:13.203794+0000","last_peered":"2024-06-02T23:11:13.203794+0000","last_clean":"2024-06-02T23:11:13.203794+0000","last_became_active":"2024-06-02T23:07:27.987657+0000","last_became_peered":"2024-06-02T23:07:27.987657+0000","last_unstale":"2024-06-02T23:11:13.203794+0000","last_undegraded":"2024-06-02T23:11:13.203794+0000","last_fullsized":"2024-06-02T23:11:13.203794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:09.876696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":204,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794336+0000","last_change":"2024-06-02T23:07:31.204903+0000","last_active":"2024-06-02T23:09:39.794336+0000","last_peered":"2024-06-02T23:09:39.794336+0000","last_clean":"2024-06-02T23:09:39.794336+0000","last_became_active":"2024-06-02T23:07:31.204605+0000","last_became_peered":"2024-06-02T23:07:31.204605+0000","last_unstale":"2024-06-02T23:09:39.794336+0000","last_undegraded":"2024-06-02T23:09:39.794336+0000","last_fullsized":"2024-06-02T23:09:39.794336+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:51:37.628362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450434+0000","last_change":"2024-06-02T23:07:27.860162+0000","last_active":"2024-06-02T23:08:11.450434+0000","last_peered":"2024-06-02T23:08:11.450434+0000","last_clean":"2024-06-02T23:08:11.450434+0000","last_became_active":"2024-06-02T23:07:27.859716+0000","last_became_peered":"2024-06-02T23:07:27.859716+0000","last_unstale":"2024-06-02T23:08:11.450434+0000","last_undegraded":"2024-06-02T23:08:11.450434+0000","last_fullsized":"2024-06-02T23:08:11.450434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:13:00.948821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":350,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:45.814046+0000","last_change":"2024-06-02T23:06:00.844556+0000","last_active":"2024-06-02T23:08:45.814046+0000","last_peered":"2024-06-02T23:08:45.814046+0000","last_clean":"2024-06-02T23:08:45.814046+0000","last_became_active":"2024-06-02T23:06:00.844063+0000","last_became_peered":"2024-06-02T23:06:00.844063+0000","last_unstale":"2024-06-02T23:08:45.814046+0000","last_undegraded":"2024-06-02T23:08:45.814046+0000","last_fullsized":"2024-06-02T23:08:45.814046+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:02:34.725618+0000","last_clean_scrub_stamp":"2024-06-02T23:02:34.725618+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:53:32.689130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.771357+0000","last_change":"2024-06-02T23:07:31.215093+0000","last_active":"2024-06-02T23:09:39.771357+0000","last_peered":"2024-06-02T23:09:39.771357+0000","last_clean":"2024-06-02T23:09:39.771357+0000","last_became_active":"2024-06-02T23:07:31.214769+0000","last_became_peered":"2024-06-02T23:07:31.214769+0000","last_unstale":"2024-06-02T23:09:39.771357+0000","last_undegraded":"2024-06-02T23:09:39.771357+0000","last_fullsized":"2024-06-02T23:09:39.771357+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:43:00.957681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333077+0000","last_change":"2024-06-02T23:07:27.862495+0000","last_active":"2024-06-02T23:08:11.333077+0000","last_peered":"2024-06-02T23:08:11.333077+0000","last_clean":"2024-06-02T23:08:11.333077+0000","last_became_active":"2024-06-02T23:07:27.860054+0000","last_became_peered":"2024-06-02T23:07:27.860054+0000","last_unstale":"2024-06-02T23:08:11.333077+0000","last_undegraded":"2024-06-02T23:08:11.333077+0000","last_fullsized":"2024-06-02T23:08:11.333077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:43:39.064375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.335914+0000","last_change":"2024-06-02T23:07:27.990401+0000","last_active":"2024-06-02T23:08:11.335914+0000","last_peered":"2024-06-02T23:08:11.335914+0000","last_clean":"2024-06-02T23:08:11.335914+0000","last_became_active":"2024-06-02T23:07:27.989892+0000","last_became_peered":"2024-06-02T23:07:27.989892+0000","last_unstale":"2024-06-02T23:08:11.335914+0000","last_undegraded":"2024-06-02T23:08:11.335914+0000","last_fullsized":"2024-06-02T23:08:11.335914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:54:54.046216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788062+0000","last_change":"2024-06-02T23:07:31.214140+0000","last_active":"2024-06-02T23:09:39.788062+0000","last_peered":"2024-06-02T23:09:39.788062+0000","last_clean":"2024-06-02T23:09:39.788062+0000","last_became_active":"2024-06-02T23:07:31.213645+0000","last_became_peered":"2024-06-02T23:07:31.213645+0000","last_unstale":"2024-06-02T23:09:39.788062+0000","last_undegraded":"2024-06-02T23:09:39.788062+0000","last_fullsized":"2024-06-02T23:09:39.788062+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:30:01.723785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"90'19","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.236340+0000","last_change":"2024-06-02T23:07:27.857608+0000","last_active":"2024-06-02T23:10:54.236340+0000","last_peered":"2024-06-02T23:10:54.236340+0000","last_clean":"2024-06-02T23:10:54.236340+0000","last_became_active":"2024-06-02T23:07:27.857265+0000","last_became_peered":"2024-06-02T23:07:27.857265+0000","last_unstale":"2024-06-02T23:10:54.236340+0000","last_undegraded":"2024-06-02T23:10:54.236340+0000","last_fullsized":"2024-06-02T23:10:54.236340+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:00:12.214329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790001+0000","last_change":"2024-06-02T23:07:31.210286+0000","last_active":"2024-06-02T23:09:39.790001+0000","last_peered":"2024-06-02T23:09:39.790001+0000","last_clean":"2024-06-02T23:09:39.790001+0000","last_became_active":"2024-06-02T23:07:31.209561+0000","last_became_peered":"2024-06-02T23:07:31.209561+0000","last_unstale":"2024-06-02T23:09:39.790001+0000","last_undegraded":"2024-06-02T23:09:39.790001+0000","last_fullsized":"2024-06-02T23:09:39.790001+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:15:47.713575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.450463+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.450463+0000","last_peered":"2024-06-02T23:08:11.450463+0000","last_clean":"2024-06-02T23:08:11.450463+0000","last_became_active":"2024-06-02T23:07:27.987791+0000","last_became_peered":"2024-06-02T23:07:27.987791+0000","last_unstale":"2024-06-02T23:08:11.450463+0000","last_undegraded":"2024-06-02T23:08:11.450463+0000","last_fullsized":"2024-06-02T23:08:11.450463+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:03:21.976260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792568+0000","last_change":"2024-06-02T23:08:11.750091+0000","last_active":"2024-06-02T23:09:39.792568+0000","last_peered":"2024-06-02T23:09:39.792568+0000","last_clean":"2024-06-02T23:09:39.792568+0000","last_became_active":"2024-06-02T23:08:11.749695+0000","last_became_peered":"2024-06-02T23:08:11.749695+0000","last_unstale":"2024-06-02T23:09:39.792568+0000","last_undegraded":"2024-06-02T23:09:39.792568+0000","last_fullsized":"2024-06-02T23:09:39.792568+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:10:47.369161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334494+0000","last_change":"2024-06-02T23:07:27.988275+0000","last_active":"2024-06-02T23:08:11.334494+0000","last_peered":"2024-06-02T23:08:11.334494+0000","last_clean":"2024-06-02T23:08:11.334494+0000","last_became_active":"2024-06-02T23:07:27.987856+0000","last_became_peered":"2024-06-02T23:07:27.987856+0000","last_unstale":"2024-06-02T23:08:11.334494+0000","last_undegraded":"2024-06-02T23:08:11.334494+0000","last_fullsized":"2024-06-02T23:08:11.334494+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T03:31:22.448852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.782299+0000","last_change":"2024-06-02T23:07:31.216417+0000","last_active":"2024-06-02T23:09:39.782299+0000","last_peered":"2024-06-02T23:09:39.782299+0000","last_clean":"2024-06-02T23:09:39.782299+0000","last_became_active":"2024-06-02T23:07:31.216070+0000","last_became_peered":"2024-06-02T23:07:31.216070+0000","last_unstale":"2024-06-02T23:09:39.782299+0000","last_undegraded":"2024-06-02T23:09:39.782299+0000","last_fullsized":"2024-06-02T23:09:39.782299+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:08:46.830010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.748925+0000","last_change":"2024-06-02T23:07:27.858938+0000","last_active":"2024-06-02T23:08:11.748925+0000","last_peered":"2024-06-02T23:08:11.748925+0000","last_clean":"2024-06-02T23:08:11.748925+0000","last_became_active":"2024-06-02T23:07:27.858568+0000","last_became_peered":"2024-06-02T23:07:27.858568+0000","last_unstale":"2024-06-02T23:08:11.748925+0000","last_undegraded":"2024-06-02T23:08:11.748925+0000","last_fullsized":"2024-06-02T23:08:11.748925+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:01:12.775729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792513+0000","last_change":"2024-06-02T23:07:31.213878+0000","last_active":"2024-06-02T23:09:39.792513+0000","last_peered":"2024-06-02T23:09:39.792513+0000","last_clean":"2024-06-02T23:09:39.792513+0000","last_became_active":"2024-06-02T23:07:31.213536+0000","last_became_peered":"2024-06-02T23:07:31.213536+0000","last_unstale":"2024-06-02T23:09:39.792513+0000","last_undegraded":"2024-06-02T23:09:39.792513+0000","last_fullsized":"2024-06-02T23:09:39.792513+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:58:43.772892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"90'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:10:54.275353+0000","last_change":"2024-06-02T23:07:27.860164+0000","last_active":"2024-06-02T23:10:54.275353+0000","last_peered":"2024-06-02T23:10:54.275353+0000","last_clean":"2024-06-02T23:10:54.275353+0000","last_became_active":"2024-06-02T23:07:27.859779+0000","last_became_peered":"2024-06-02T23:07:27.859779+0000","last_unstale":"2024-06-02T23:10:54.275353+0000","last_undegraded":"2024-06-02T23:10:54.275353+0000","last_fullsized":"2024-06-02T23:10:54.275353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:40:57.350297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786323+0000","last_change":"2024-06-02T23:07:31.205956+0000","last_active":"2024-06-02T23:09:39.786323+0000","last_peered":"2024-06-02T23:09:39.786323+0000","last_clean":"2024-06-02T23:09:39.786323+0000","last_became_active":"2024-06-02T23:07:31.205552+0000","last_became_peered":"2024-06-02T23:07:31.205552+0000","last_unstale":"2024-06-02T23:09:39.786323+0000","last_undegraded":"2024-06-02T23:09:39.786323+0000","last_fullsized":"2024-06-02T23:09:39.786323+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:57:08.806603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.334806+0000","last_change":"2024-06-02T23:07:27.861619+0000","last_active":"2024-06-02T23:08:11.334806+0000","last_peered":"2024-06-02T23:08:11.334806+0000","last_clean":"2024-06-02T23:08:11.334806+0000","last_became_active":"2024-06-02T23:07:27.860555+0000","last_became_peered":"2024-06-02T23:07:27.860555+0000","last_unstale":"2024-06-02T23:08:11.334806+0000","last_undegraded":"2024-06-02T23:08:11.334806+0000","last_fullsized":"2024-06-02T23:08:11.334806+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T08:02:54.294368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797923+0000","last_change":"2024-06-02T23:07:31.220636+0000","last_active":"2024-06-02T23:09:39.797923+0000","last_peered":"2024-06-02T23:09:39.797923+0000","last_clean":"2024-06-02T23:09:39.797923+0000","last_became_active":"2024-06-02T23:07:31.220287+0000","last_became_peered":"2024-06-02T23:07:31.220287+0000","last_unstale":"2024-06-02T23:09:39.797923+0000","last_undegraded":"2024-06-02T23:09:39.797923+0000","last_fullsized":"2024-06-02T23:09:39.797923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:14:39.761077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"90'8","reported_seq":243,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:11:13.276163+0000","last_change":"2024-06-02T23:07:27.988455+0000","last_active":"2024-06-02T23:11:13.276163+0000","last_peered":"2024-06-02T23:11:13.276163+0000","last_clean":"2024-06-02T23:11:13.276163+0000","last_became_active":"2024-06-02T23:07:27.987882+0000","last_became_peered":"2024-06-02T23:07:27.987882+0000","last_unstale":"2024-06-02T23:11:13.276163+0000","last_undegraded":"2024-06-02T23:11:13.276163+0000","last_fullsized":"2024-06-02T23:11:13.276163+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:23:10.254314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2193,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":208,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.797984+0000","last_change":"2024-06-02T23:07:31.222020+0000","last_active":"2024-06-02T23:09:39.797984+0000","last_peered":"2024-06-02T23:09:39.797984+0000","last_clean":"2024-06-02T23:09:39.797984+0000","last_became_active":"2024-06-02T23:07:31.221677+0000","last_became_peered":"2024-06-02T23:07:31.221677+0000","last_unstale":"2024-06-02T23:09:39.797984+0000","last_undegraded":"2024-06-02T23:09:39.797984+0000","last_fullsized":"2024-06-02T23:09:39.797984+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T06:58:27.860993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.330301+0000","last_change":"2024-06-02T23:07:27.864632+0000","last_active":"2024-06-02T23:08:11.330301+0000","last_peered":"2024-06-02T23:08:11.330301+0000","last_clean":"2024-06-02T23:08:11.330301+0000","last_became_active":"2024-06-02T23:07:27.864257+0000","last_became_peered":"2024-06-02T23:07:27.864257+0000","last_unstale":"2024-06-02T23:08:11.330301+0000","last_undegraded":"2024-06-02T23:08:11.330301+0000","last_fullsized":"2024-06-02T23:08:11.330301+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:02:41.386963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789936+0000","last_change":"2024-06-02T23:07:31.217816+0000","last_active":"2024-06-02T23:09:39.789936+0000","last_peered":"2024-06-02T23:09:39.789936+0000","last_clean":"2024-06-02T23:09:39.789936+0000","last_became_active":"2024-06-02T23:07:31.217460+0000","last_became_peered":"2024-06-02T23:07:31.217460+0000","last_unstale":"2024-06-02T23:09:39.789936+0000","last_undegraded":"2024-06-02T23:09:39.789936+0000","last_fullsized":"2024-06-02T23:09:39.789936+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:54:30.040984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:08:11.333595+0000","last_change":"2024-06-02T23:07:27.859649+0000","last_active":"2024-06-02T23:08:11.333595+0000","last_peered":"2024-06-02T23:08:11.333595+0000","last_clean":"2024-06-02T23:08:11.333595+0000","last_became_active":"2024-06-02T23:07:27.859178+0000","last_became_peered":"2024-06-02T23:07:27.859178+0000","last_unstale":"2024-06-02T23:08:11.333595+0000","last_undegraded":"2024-06-02T23:08:11.333595+0000","last_fullsized":"2024-06-02T23:08:11.333595+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:26.844152+0000","last_clean_scrub_stamp":"2024-06-02T23:07:26.844152+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T00:56:11.639804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.795458+0000","last_change":"2024-06-02T23:07:31.211883+0000","last_active":"2024-06-02T23:09:39.795458+0000","last_peered":"2024-06-02T23:09:39.795458+0000","last_clean":"2024-06-02T23:09:39.795458+0000","last_became_active":"2024-06-02T23:07:31.211499+0000","last_became_peered":"2024-06-02T23:07:31.211499+0000","last_unstale":"2024-06-02T23:09:39.795458+0000","last_undegraded":"2024-06-02T23:09:39.795458+0000","last_fullsized":"2024-06-02T23:09:39.795458+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:07:43.577475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.790898+0000","last_change":"2024-06-02T23:07:31.221820+0000","last_active":"2024-06-02T23:09:39.790898+0000","last_peered":"2024-06-02T23:09:39.790898+0000","last_clean":"2024-06-02T23:09:39.790898+0000","last_became_active":"2024-06-02T23:07:31.221429+0000","last_became_peered":"2024-06-02T23:07:31.221429+0000","last_unstale":"2024-06-02T23:09:39.790898+0000","last_undegraded":"2024-06-02T23:09:39.790898+0000","last_fullsized":"2024-06-02T23:09:39.790898+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T04:18:31.132896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785496+0000","last_change":"2024-06-02T23:07:31.221321+0000","last_active":"2024-06-02T23:09:39.785496+0000","last_peered":"2024-06-02T23:09:39.785496+0000","last_clean":"2024-06-02T23:09:39.785496+0000","last_became_active":"2024-06-02T23:07:31.220533+0000","last_became_peered":"2024-06-02T23:07:31.220533+0000","last_unstale":"2024-06-02T23:09:39.785496+0000","last_undegraded":"2024-06-02T23:09:39.785496+0000","last_fullsized":"2024-06-02T23:09:39.785496+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:00:22.551590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.785576+0000","last_change":"2024-06-02T23:07:31.221483+0000","last_active":"2024-06-02T23:09:39.785576+0000","last_peered":"2024-06-02T23:09:39.785576+0000","last_clean":"2024-06-02T23:09:39.785576+0000","last_became_active":"2024-06-02T23:07:31.220172+0000","last_became_peered":"2024-06-02T23:07:31.220172+0000","last_unstale":"2024-06-02T23:09:39.785576+0000","last_undegraded":"2024-06-02T23:09:39.785576+0000","last_fullsized":"2024-06-02T23:09:39.785576+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:38.798748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.791535+0000","last_change":"2024-06-02T23:07:31.209004+0000","last_active":"2024-06-02T23:09:39.791535+0000","last_peered":"2024-06-02T23:09:39.791535+0000","last_clean":"2024-06-02T23:09:39.791535+0000","last_became_active":"2024-06-02T23:07:31.208201+0000","last_became_peered":"2024-06-02T23:07:31.208201+0000","last_unstale":"2024-06-02T23:09:39.791535+0000","last_undegraded":"2024-06-02T23:09:39.791535+0000","last_fullsized":"2024-06-02T23:09:39.791535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:34:30.953127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792313+0000","last_change":"2024-06-02T23:08:11.335176+0000","last_active":"2024-06-02T23:09:39.792313+0000","last_peered":"2024-06-02T23:09:39.792313+0000","last_clean":"2024-06-02T23:09:39.792313+0000","last_became_active":"2024-06-02T23:08:11.334779+0000","last_became_peered":"2024-06-02T23:08:11.334779+0000","last_unstale":"2024-06-02T23:09:39.792313+0000","last_undegraded":"2024-06-02T23:09:39.792313+0000","last_fullsized":"2024-06-02T23:09:39.792313+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:23:50.453517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.774868+0000","last_change":"2024-06-02T23:08:11.337292+0000","last_active":"2024-06-02T23:09:39.774868+0000","last_peered":"2024-06-02T23:09:39.774868+0000","last_clean":"2024-06-02T23:09:39.774868+0000","last_became_active":"2024-06-02T23:08:11.336821+0000","last_became_peered":"2024-06-02T23:08:11.336821+0000","last_unstale":"2024-06-02T23:09:39.774868+0000","last_undegraded":"2024-06-02T23:09:39.774868+0000","last_fullsized":"2024-06-02T23:09:39.774868+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:35:15.753923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,9],"acting":[11,8,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.794273+0000","last_change":"2024-06-02T23:07:31.209835+0000","last_active":"2024-06-02T23:09:39.794273+0000","last_peered":"2024-06-02T23:09:39.794273+0000","last_clean":"2024-06-02T23:09:39.794273+0000","last_became_active":"2024-06-02T23:07:31.209146+0000","last_became_peered":"2024-06-02T23:07:31.209146+0000","last_unstale":"2024-06-02T23:09:39.794273+0000","last_undegraded":"2024-06-02T23:09:39.794273+0000","last_fullsized":"2024-06-02T23:09:39.794273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T01:49:05.193091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":23,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.788292+0000","last_change":"2024-06-02T23:07:31.215628+0000","last_active":"2024-06-02T23:09:39.788292+0000","last_peered":"2024-06-02T23:09:39.788292+0000","last_clean":"2024-06-02T23:09:39.788292+0000","last_became_active":"2024-06-02T23:07:31.215252+0000","last_became_peered":"2024-06-02T23:07:31.215252+0000","last_unstale":"2024-06-02T23:09:39.788292+0000","last_undegraded":"2024-06-02T23:09:39.788292+0000","last_fullsized":"2024-06-02T23:09:39.788292+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T10:44:55.969413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.786749+0000","last_change":"2024-06-02T23:07:31.200224+0000","last_active":"2024-06-02T23:09:39.786749+0000","last_peered":"2024-06-02T23:09:39.786749+0000","last_clean":"2024-06-02T23:09:39.786749+0000","last_became_active":"2024-06-02T23:07:31.199739+0000","last_became_peered":"2024-06-02T23:07:31.199739+0000","last_unstale":"2024-06-02T23:09:39.786749+0000","last_undegraded":"2024-06-02T23:09:39.786749+0000","last_fullsized":"2024-06-02T23:09:39.786749+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T05:37:49.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.789933+0000","last_change":"2024-06-02T23:07:31.214232+0000","last_active":"2024-06-02T23:09:39.789933+0000","last_peered":"2024-06-02T23:09:39.789933+0000","last_clean":"2024-06-02T23:09:39.789933+0000","last_became_active":"2024-06-02T23:07:31.213398+0000","last_became_peered":"2024-06-02T23:07:31.213398+0000","last_unstale":"2024-06-02T23:09:39.789933+0000","last_undegraded":"2024-06-02T23:09:39.789933+0000","last_fullsized":"2024-06-02T23:09:39.789933+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T07:12:36.096585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":31,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.800745+0000","last_change":"2024-06-02T23:07:31.218332+0000","last_active":"2024-06-02T23:09:39.800745+0000","last_peered":"2024-06-02T23:09:39.800745+0000","last_clean":"2024-06-02T23:09:39.800745+0000","last_became_active":"2024-06-02T23:07:31.217904+0000","last_became_peered":"2024-06-02T23:07:31.217904+0000","last_unstale":"2024-06-02T23:09:39.800745+0000","last_undegraded":"2024-06-02T23:09:39.800745+0000","last_fullsized":"2024-06-02T23:09:39.800745+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-03T23:25:59.840499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.792409+0000","last_change":"2024-06-02T23:07:31.207637+0000","last_active":"2024-06-02T23:09:39.792409+0000","last_peered":"2024-06-02T23:09:39.792409+0000","last_clean":"2024-06-02T23:09:39.792409+0000","last_became_active":"2024-06-02T23:07:31.207312+0000","last_became_peered":"2024-06-02T23:07:31.207312+0000","last_unstale":"2024-06-02T23:09:39.792409+0000","last_undegraded":"2024-06-02T23:09:39.792409+0000","last_fullsized":"2024-06-02T23:09:39.792409+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T09:04:51.285987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-06-02T23:09:39.776917+0000","last_change":"2024-06-02T23:08:11.337520+0000","last_active":"2024-06-02T23:09:39.776917+0000","last_peered":"2024-06-02T23:09:39.776917+0000","last_clean":"2024-06-02T23:09:39.776917+0000","last_became_active":"2024-06-02T23:08:11.336662+0000","last_became_peered":"2024-06-02T23:08:11.336662+0000","last_unstale":"2024-06-02T23:09:39.776917+0000","last_undegraded":"2024-06-02T23:09:39.776917+0000","last_fullsized":"2024-06-02T23:09:39.776917+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-06-02T23:07:30.188401+0000","last_clean_scrub_stamp":"2024-06-02T23:07:30.188401+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-06-04T02:39:14.652327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284203,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75801,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":2088,"num_read_kb":1131,"num_write":273,"num_write_kb":160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":734,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":589824,"data_stored":227403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":232,"ondisk_log_size":232,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383742,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":481555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645379,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1138821,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1599,"internal_metadata":27654593},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808904,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1011712,"data_stored":480350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":480616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332179,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1060864,"data_stored":1074331,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1589,"internal_metadata":31914443},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495703,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":480559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":548352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738485,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":482429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411436,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1077248,"data_stored":548481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182503,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1011712,"data_stored":479892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":481534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757340,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":1073025,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34501,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36140,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":69124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":69253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":122880,"data_stored":69108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-06-02T23:11:20.020 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:11:20.274 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:20.397 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:20 smithi052 ceph-mon[32340]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:20.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 -- 172.21.15.52:0/3145625796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd054101930 msgr2=0x7fd054101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 --2- 172.21.15.52:0/3145625796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd054101930 0x7fd054101db0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd048009900 tx=0x7fd04802a410 comp rx=0 tx=0).stop 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 -- 172.21.15.52:0/3145625796 shutdown_connections 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 --2- 172.21.15.52:0/3145625796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0541022f0 0x7fd054108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 --2- 172.21.15.52:0/3145625796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd054101930 0x7fd054101db0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 --2- 172.21.15.52:0/3145625796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054100740 0x7fd054100b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 -- 172.21.15.52:0/3145625796 >> 172.21.15.52:0/3145625796 conn(0x7fd0540fbcd0 msgr2=0x7fd0540fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 -- 172.21.15.52:0/3145625796 shutdown_connections 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.439+0000 7fd053fff640 1 -- 172.21.15.52:0/3145625796 wait complete. 2024-06-02T23:11:20.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 Processor -- start 2024-06-02T23:11:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 -- start start 2024-06-02T23:11:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054100740 0x7fd054197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd054101930 0x7fd054197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 0x7fd05419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d390 con 0x7fd0541022f0 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d210 con 0x7fd054100740 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd053fff640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd05410d510 con 0x7fd054101930 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 0x7fd05419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 0x7fd05419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37014/0 (socket says 172.21.15.52:37014) 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 -- 172.21.15.52:0/2158731934 learned_addr learned my addr 172.21.15.52:0/2158731934 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0527fc640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd054101930 0x7fd054197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd052ffd640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054100740 0x7fd054197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 -- 172.21.15.52:0/2158731934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd054101930 msgr2=0x7fd054197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd054101930 0x7fd054197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 -- 172.21.15.52:0/2158731934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054100740 msgr2=0x7fd054197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054100740 0x7fd054197a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.440+0000 7fd0537fe640 1 -- 172.21.15.52:0/2158731934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd05419f690 con 0x7fd0541022f0 2024-06-02T23:11:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.441+0000 7fd0537fe640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 0x7fd05419f020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e970 tx=0x7fd04c00ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.441+0000 7fd03bfff640 1 -- 172.21.15.52:0/2158731934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c00ccc0 con 0x7fd0541022f0 2024-06-02T23:11:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.441+0000 7fd03bfff640 1 -- 172.21.15.52:0/2158731934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd04c00ce60 con 0x7fd0541022f0 2024-06-02T23:11:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.441+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05419f980 con 0x7fd0541022f0 2024-06-02T23:11:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.441+0000 7fd03bfff640 1 -- 172.21.15.52:0/2158731934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04c010430 con 0x7fd0541022f0 2024-06-02T23:11:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.441+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd05419ff20 con 0x7fd0541022f0 2024-06-02T23:11:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.443+0000 7fd03bfff640 1 -- 172.21.15.52:0/2158731934 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd04c004f90 con 0x7fd0541022f0 2024-06-02T23:11:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.443+0000 7fd05c3ea740 1 Processor -- start 2024-06-02T23:11:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.443+0000 7fd05c3ea740 1 -- start start 2024-06-02T23:11:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0ba600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c7ac0114f0 con 0x55c7ac0b3000 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7abfaf870 con 0x55c7ac0b2610 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7ac00d4b0 con 0x55c7ac0b8170 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd03bfff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd03c0a2c00 0x7fd03c0a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37018/0 (socket says 172.21.15.52:37018) 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 -- 172.21.15.52:0/1040230377 learned_addr learned my addr 172.21.15.52:0/1040230377 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd0537fe640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd03c0a2c00 0x7fd03c0a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd03bfff640 1 -- 172.21.15.52:0/2158731934 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd04c0cf4c0 con 0x7fd0541022f0 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0ba600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 -- 172.21.15.52:0/1040230377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 msgr2=0x55c7ac0ba600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0ba600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 -- 172.21.15.52:0/1040230377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 msgr2=0x55c7ac0b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0b2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 -- 172.21.15.52:0/1040230377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7abfcb9b0 con 0x55c7ac0b3000 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0b7b00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd044009f20 tx=0x7fd04400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4179e341cc877f31 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.444+0000 7fd0537fe640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd03c0a2c00 0x7fd03c0a50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd04c09ba50 tx=0x7fd04c066920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd0397fa640 1 -- 172.21.15.52:0/1040230377 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044004d20 con 0x55c7ac0b3000 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd0397fa640 1 -- 172.21.15.52:0/1040230377 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0440054d0 con 0x55c7ac0b3000 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd0397fa640 1 -- 172.21.15.52:0/1040230377 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044011900 con 0x55c7ac0b3000 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1040230377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 msgr2=0x55c7ac0b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0b7b00 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd044009f20 tx=0x7fd04400f7f0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1040230377 shutdown_connections 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0ba600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0b7b00 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1040230377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1040230377 >> 172.21.15.52:0/1040230377 conn(0x55c7ac0a9870 msgr2=0x55c7ac0a9c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1040230377 shutdown_connections 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.445+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1040230377 wait complete. 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 -- start start 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 Processor -- start 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0bcd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c7ac0bb120 con 0x55c7ac0b2610 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7abf9aae0 con 0x55c7ac0b8170 2024-06-02T23:11:20.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c7ac0bb2a0 con 0x55c7ac0b3000 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd05c3ea740 10 client.0 _ll_register_callbacks cb 0x55c7abfd39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37028/0 (socket says 172.21.15.52:37028) 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 -- 172.21.15.52:0/1843587524 learned_addr learned my addr 172.21.15.52:0/1843587524 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0bcd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 -- 172.21.15.52:0/1843587524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 msgr2=0x55c7ac0bcd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0bcd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 -- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b8170 msgr2=0x55c7ac0c3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd0527fc640 1 -- 172.21.15.52:0/1843587524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7abd25210 con 0x55c7ac0b2610 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.446+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:20.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd0480096c0 tx=0x7fd048005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adaeeff8b96046ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd036ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd036ff5640 1 -- 172.21.15.52:0/1843587524 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd048035040 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0440054d0 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd036ff5640 1 -- 172.21.15.52:0/1843587524 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd048004240 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd036ff5640 1 -- 172.21.15.52:0/1843587524 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd048051050 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 10 client.15348 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7abdf1620 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd036ff5640 1 -- 172.21.15.52:0/1843587524 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0440054d0 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd036ff5640 1 -- 172.21.15.52:0/1843587524 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd04803a040 con 0x55c7ac0b2610 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 10 client.15348 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 10 client.15348 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 20 client.15348 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 4 client.15348 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.447+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ac024f10 con 0x55c7ac0c8660 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.448+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.448+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac0caad0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32285809a4b705fe server_cookie=375e91f4e6074bea in_seq=0 out_seq=0 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.448+0000 7fd036ff5640 10 client.15348 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd036ff5640 1 -- 172.21.15.52:0/1843587524 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c7ac024f10 con 0x55c7ac0c8660 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd036ff5640 10 client.15348 handle_command_reply: tid=0 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 client.15348 shutdown 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 msgr2=0x55c7ac0bc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc850 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fd0480096c0 tx=0x7fd048005df0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 shutdown_connections 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac0caad0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0bcd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd036ff5640 0 client.15348 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1843587524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc850 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 >> 172.21.15.52:0/1843587524 conn(0x55c7ac0a9870 msgr2=0x55c7ac0a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 shutdown_connections 2024-06-02T23:11:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1843587524 wait complete. 2024-06-02T23:11:20.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.449+0000 7fd05c3ea740 20 client.15348 trim_cache size 0 max 16384 2024-06-02T23:11:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.464+0000 7fd05c3ea740 1 Processor -- start 2024-06-02T23:11:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.464+0000 7fd05c3ea740 1 -- start start 2024-06-02T23:11:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0bc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541a6920 con 0x55c7ac0b3000 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541a8460 con 0x55c7ac0b2610 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541a6ff0 con 0x55c7ac0b8170 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37036/0 (socket says 172.21.15.52:37036) 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 -- 172.21.15.52:0/1144891439 learned_addr learned my addr 172.21.15.52:0/1144891439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 -- 172.21.15.52:0/1144891439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 msgr2=0x55c7ac0bc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0bc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 -- 172.21.15.52:0/1144891439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 msgr2=0x55c7ac0bc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 -- 172.21.15.52:0/1144891439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ac0a3ce0 con 0x55c7ac0b3000 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c3d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd044005cc0 tx=0x7fd044005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb83c132a006ac2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd038ff9640 1 -- 172.21.15.52:0/1144891439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0440052b0 con 0x55c7ac0b3000 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd038ff9640 1 -- 172.21.15.52:0/1144891439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd044003e20 con 0x55c7ac0b3000 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.465+0000 7fd038ff9640 1 -- 172.21.15.52:0/1144891439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd044011af0 con 0x55c7ac0b3000 2024-06-02T23:11:20.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1144891439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 msgr2=0x55c7ac0c3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c3d90 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fd044005cc0 tx=0x7fd044005040 comp rx=0 tx=0).stop 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1144891439 shutdown_connections 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b8170 0x55c7ac0bc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c3d90 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1144891439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0bc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1144891439 >> 172.21.15.52:0/1144891439 conn(0x55c7ac0a9870 msgr2=0x55c7ac0b7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1144891439 shutdown_connections 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1144891439 wait complete. 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 -- start start 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.466+0000 7fd05c3ea740 1 Processor -- start 2024-06-02T23:11:20.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541a7d90 con 0x55c7ac0b8170 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541a76c0 con 0x55c7ac0b2610 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0541a4900 con 0x55c7ac0b3000 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 10 client.0 _ll_register_callbacks cb 0x7fd0440093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37046/0 (socket says 172.21.15.52:37046) 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 -- 172.21.15.52:0/1859513873 learned_addr learned my addr 172.21.15.52:0/1859513873 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 -- 172.21.15.52:0/1859513873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 msgr2=0x55c7ac0c15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 -- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 msgr2=0x55c7ac0c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 -- 172.21.15.52:0/1859513873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ac0c2bc0 con 0x55c7ac0b8170 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd0527fc640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c25d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55c7ac0ba8a0 tx=0x7fd0480314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8812e09cfe9ebdf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd036ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd036ff5640 1 -- 172.21.15.52:0/1859513873 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04805d060 con 0x55c7ac0b8170 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd036ff5640 1 -- 172.21.15.52:0/1859513873 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd048004240 con 0x55c7ac0b8170 2024-06-02T23:11:20.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.467+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd044003e20 con 0x55c7ac0b8170 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd036ff5640 1 -- 172.21.15.52:0/1859513873 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd048035030 con 0x55c7ac0b8170 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 10 client.15354 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c7ac0c3e40 con 0x55c7ac0b8170 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd036ff5640 1 -- 172.21.15.52:0/1859513873 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd044003e20 con 0x55c7ac0b8170 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd036ff5640 1 -- 172.21.15.52:0/1859513873 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd048004e50 con 0x55c7ac0b8170 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 10 client.15354 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 10 client.15354 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 20 client.15354 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac135950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 4 client.15354 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7ac01ec20 con 0x55c7ac0c8660 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.468+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac135950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.469+0000 7fd052ffd640 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac135950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43c2de80ce315ebf server_cookie=1df2b80b5c577632 in_seq=0 out_seq=0 2024-06-02T23:11:20.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.469+0000 7fd036ff5640 10 client.15354 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.479+0000 7fd036ff5640 1 -- 172.21.15.52:0/1859513873 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c7ac01ec20 con 0x55c7ac0c8660 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.479+0000 7fd036ff5640 10 client.15354 handle_command_reply: tid=0 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.479+0000 7fd05c3ea740 1 client.15354 shutdown 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 msgr2=0x55c7ac0c25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c25d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x55c7ac0ba8a0 tx=0x7fd0480314a0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 shutdown_connections 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7ac0c8660 0x55c7ac135950 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7ac0b3000 0x55c7ac0c15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7ac0b8170 0x55c7ac0c25d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd036ff5640 0 client.15354 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 --2- 172.21.15.52:0/1859513873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7ac0b2610 0x55c7ac0c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 >> 172.21.15.52:0/1859513873 conn(0x55c7ac0a9870 msgr2=0x55c7ac0bc460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 shutdown_connections 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 1 -- 172.21.15.52:0/1859513873 wait complete. 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd05c3ea740 20 client.15354 trim_cache size 0 max 16384 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:11:20.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:11:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:11:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd03c0a2c00 msgr2=0x7fd03c0a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd03c0a2c00 0x7fd03c0a50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd04c09ba50 tx=0x7fd04c066920 comp rx=0 tx=0).stop 2024-06-02T23:11:20.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 msgr2=0x7fd05419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 0x7fd05419f020 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd04c00e970 tx=0x7fd04c00ed70 comp rx=0 tx=0).stop 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 shutdown_connections 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd03c0a2c00 0x7fd03c0a50c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd054101930 0x7fd054197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0541022f0 0x7fd05419f020 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 --2- 172.21.15.52:0/2158731934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd054100740 0x7fd054197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.480+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 >> 172.21.15.52:0/2158731934 conn(0x7fd0540fbcd0 msgr2=0x7fd054104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.481+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 shutdown_connections 2024-06-02T23:11:20.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:20.481+0000 7fd053fff640 1 -- 172.21.15.52:0/2158731934 wait complete. 2024-06-02T23:11:20.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:20 smithi123 ceph-mon[35977]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:20.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:20 smithi174 ceph-mon[35994]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-06-02T23:11:21.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:21 smithi052 ceph-mon[32340]: pgmap v415: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-02T23:11:21.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:21 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:11:21.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:21 smithi123 ceph-mon[35977]: pgmap v415: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-02T23:11:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:21 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:11:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:21 smithi174 ceph-mon[35994]: pgmap v415: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-06-02T23:11:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:21 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:11:22.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 -- 172.21.15.52:0/3795260544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30901030f0 msgr2=0x7f3090107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:22.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 --2- 172.21.15.52:0/3795260544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30901030f0 0x7f3090107110 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f30840099c0 tx=0x7f3084009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 -- 172.21.15.52:0/3795260544 shutdown_connections 2024-06-02T23:11:22.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 --2- 172.21.15.52:0/3795260544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090107840 0x7f309010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 --2- 172.21.15.52:0/3795260544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3090102740 0x7f3090102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 --2- 172.21.15.52:0/3795260544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f30901030f0 0x7f3090107110 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 -- 172.21.15.52:0/3795260544 >> 172.21.15.52:0/3795260544 conn(0x7f30900795f0 msgr2=0x7f3090079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:22.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 -- 172.21.15.52:0/3795260544 shutdown_connections 2024-06-02T23:11:22.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.078+0000 7f30949b0700 1 -- 172.21.15.52:0/3795260544 wait complete. 2024-06-02T23:11:22.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.079+0000 7f30949b0700 1 Processor -- start 2024-06-02T23:11:22.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.079+0000 7f30949b0700 1 -- start start 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.079+0000 7f30949b0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102740 0x7f309019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f30949b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30901030f0 0x7f309019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f30949b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 0x7f30901988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f30949b0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112680 con 0x7f3090107840 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f30949b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112500 con 0x7f30901030f0 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f30949b0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3090112800 con 0x7f3090102740 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 0x7f30901988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:22.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 0x7f30901988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37062/0 (socket says 172.21.15.52:37062) 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 -- 172.21.15.52:0/1925006734 learned_addr learned my addr 172.21.15.52:0/1925006734 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308dd9b700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30901030f0 0x7f309019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308e59c700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102740 0x7f309019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 -- 172.21.15.52:0/1925006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102740 msgr2=0x7f309019e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102740 0x7f309019e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 -- 172.21.15.52:0/1925006734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30901030f0 msgr2=0x7f309019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:22.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30901030f0 0x7f309019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 -- 172.21.15.52:0/1925006734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3090199040 con 0x7f3090107840 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f308ed9d700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 0x7f30901988e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f308800b970 tx=0x7f308800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088019070 con 0x7f3090107840 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3090199330 con 0x7f3090107840 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f308800cde0 con 0x7f3090107840 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.080+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3088007370 con 0x7f3090107840 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.081+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30900746c0 con 0x7f3090107840 2024-06-02T23:11:22.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.081+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3090069180 con 0x7f3090107840 2024-06-02T23:11:22.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.083+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f308801c070 con 0x7f3090107840 2024-06-02T23:11:22.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.083+0000 7f307f7fe700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30780a7030 0x7f30780a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:22.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.084+0000 7f308e59c700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30780a7030 0x7f30780a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:22.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.084+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f30880cec30 con 0x7f3090107840 2024-06-02T23:11:22.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.084+0000 7f308e59c700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30780a7030 0x7f30780a94f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3080005ea0 tx=0x7f3080005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:22.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.087+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3088023070 con 0x7f3090107840 2024-06-02T23:11:22.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.345+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3090199e10 con 0x7f3090107840 2024-06-02T23:11:22.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.345+0000 7f307f7fe700 1 -- 172.21.15.52:0/1925006734 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f308809bbc0 con 0x7f3090107840 2024-06-02T23:11:22.346 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:22.346 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:11:22.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30780a7030 msgr2=0x7f30780a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:22.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30780a7030 0x7f30780a94f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3080005ea0 tx=0x7f3080005e30 comp rx=0 tx=0).stop 2024-06-02T23:11:22.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 msgr2=0x7f30901988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:22.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 0x7f30901988e0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f308800b970 tx=0x7f308800bc80 comp rx=0 tx=0).stop 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 shutdown_connections 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30780a7030 0x7f30780a94f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3090107840 0x7f30901988e0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3090102740 0x7f309019e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 --2- 172.21.15.52:0/1925006734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f30901030f0 0x7f309019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 >> 172.21.15.52:0/1925006734 conn(0x7f30900795f0 msgr2=0x7f3090101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 shutdown_connections 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:22.348+0000 7f30949b0700 1 -- 172.21.15.52:0/1925006734 wait complete. 2024-06-02T23:11:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:11:23.002 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:11:23.259 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:23.381 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:23 smithi052 ceph-mon[32340]: pgmap v416: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:11:23.381 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:23 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1925006734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:23 smithi123 ceph-mon[35977]: pgmap v416: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:11:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:23 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1925006734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:23 smithi174 ceph-mon[35994]: pgmap v416: 97 pgs: 97 active+clean; 651 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:11:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:23 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1925006734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.182+0000 7f2113a39700 1 -- 172.21.15.52:0/901962359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 msgr2=0x7f210c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.182+0000 7f2113a39700 1 --2- 172.21.15.52:0/901962359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1077d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2108009990 tx=0x7f2108009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 -- 172.21.15.52:0/901962359 shutdown_connections 2024-06-02T23:11:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 --2- 172.21.15.52:0/901962359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f210c101d40 0x7f210c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 --2- 172.21.15.52:0/901962359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1077d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 --2- 172.21.15.52:0/901962359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f210c101380 0x7f210c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 -- 172.21.15.52:0/901962359 >> 172.21.15.52:0/901962359 conn(0x7f210c078300 msgr2=0x7f210c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 -- 172.21.15.52:0/901962359 shutdown_connections 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 -- 172.21.15.52:0/901962359 wait complete. 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 Processor -- start 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.183+0000 7f2113a39700 1 -- start start 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2113a39700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f210c101380 0x7f210c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2113a39700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f210c101d40 0x7f210c199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2113a39700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1a4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2113a39700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f210c111510 con 0x7f210c1073f0 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2113a39700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f210c111390 con 0x7f210c101d40 2024-06-02T23:11:25.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2113a39700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f210c111690 con 0x7f210c101380 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1a4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1a4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43464/0 (socket says 172.21.15.52:43464) 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 -- 172.21.15.52:0/791941684 learned_addr learned my addr 172.21.15.52:0/791941684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f21117d5700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f210c101380 0x7f210c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2110fd4700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f210c101d40 0x7f210c199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 -- 172.21.15.52:0/791941684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f210c101380 msgr2=0x7f210c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f210c101380 0x7f210c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 -- 172.21.15.52:0/791941684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f210c101d40 msgr2=0x7f210c199ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f210c101d40 0x7f210c199ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 -- 172.21.15.52:0/791941684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c1a5070 con 0x7f210c1073f0 2024-06-02T23:11:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f2111fd6700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1a4b30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2104007ee0 tx=0x7f210400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.184+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2104004bb0 con 0x7f210c1073f0 2024-06-02T23:11:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.185+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c1a52a0 con 0x7f210c1073f0 2024-06-02T23:11:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.185+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2104004d50 con 0x7f210c1073f0 2024-06-02T23:11:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.185+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21040058e0 con 0x7f210c1073f0 2024-06-02T23:11:25.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.185+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c1a57b0 con 0x7f210c1073f0 2024-06-02T23:11:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.186+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210c069180 con 0x7f210c1073f0 2024-06-02T23:11:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.187+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f210401a020 con 0x7f210c1073f0 2024-06-02T23:11:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.187+0000 7f20fe7fc700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f20f80a2ca0 0x7f20f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.188+0000 7f21117d5700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f20f80a2ca0 0x7f20f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.188+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f21040cecd0 con 0x7f210c1073f0 2024-06-02T23:11:25.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.188+0000 7f21117d5700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f20f80a2ca0 0x7f20f80a5160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f210800b5c0 tx=0x7f2108005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:25.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.191+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2104010070 con 0x7f210c1073f0 2024-06-02T23:11:25.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:25 smithi052 ceph-mon[32340]: pgmap v417: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-02T23:11:25.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.444+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f210c02dca0 con 0x7f210c1073f0 2024-06-02T23:11:25.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.445+0000 7f20fe7fc700 1 -- 172.21.15.52:0/791941684 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f210409bc60 con 0x7f210c1073f0 2024-06-02T23:11:25.445 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:25.445 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:11:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f20f80a2ca0 msgr2=0x7f20f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f20f80a2ca0 0x7f20f80a5160 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f210800b5c0 tx=0x7f2108005960 comp rx=0 tx=0).stop 2024-06-02T23:11:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 msgr2=0x7f210c1a4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1a4b30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2104007ee0 tx=0x7f210400d3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 shutdown_connections 2024-06-02T23:11:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f20f80a2ca0 0x7f20f80a5160 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f210c1073f0 0x7f210c1a4b30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f210c101380 0x7f210c199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 --2- 172.21.15.52:0/791941684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f210c101d40 0x7f210c199ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 >> 172.21.15.52:0/791941684 conn(0x7f210c078300 msgr2=0x7f210c1001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 shutdown_connections 2024-06-02T23:11:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:25.447+0000 7f2113a39700 1 -- 172.21.15.52:0/791941684 wait complete. 2024-06-02T23:11:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:11:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:25 smithi123 ceph-mon[35977]: pgmap v417: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-02T23:11:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:25 smithi174 ceph-mon[35994]: pgmap v417: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 853 B/s wr, 12 op/s 2024-06-02T23:11:26.142 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:11:26.393 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:26.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:26 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/791941684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:26.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:26 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/791941684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:26.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:26 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/791941684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:27.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:27 smithi123 ceph-mon[35977]: pgmap v418: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-06-02T23:11:27.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:27 smithi052 ceph-mon[32340]: pgmap v418: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-06-02T23:11:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:27 smithi174 ceph-mon[35994]: pgmap v418: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-06-02T23:11:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 -- 172.21.15.52:0/2415156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 msgr2=0x7f833c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:28.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 --2- 172.21.15.52:0/2415156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c105ff0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f83300099c0 tx=0x7f8330009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 -- 172.21.15.52:0/2415156265 shutdown_connections 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 --2- 172.21.15.52:0/2415156265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c1066d0 0x7f833c10e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 --2- 172.21.15.52:0/2415156265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c101680 0x7f833c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 --2- 172.21.15.52:0/2415156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c105ff0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 -- 172.21.15.52:0/2415156265 >> 172.21.15.52:0/2415156265 conn(0x7f833c0773e0 msgr2=0x7f833c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 -- 172.21.15.52:0/2415156265 shutdown_connections 2024-06-02T23:11:28.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.158+0000 7f8340fcb700 1 -- 172.21.15.52:0/2415156265 wait complete. 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.159+0000 7f8340fcb700 1 Processor -- start 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.159+0000 7f8340fcb700 1 -- start start 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.159+0000 7f8340fcb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c101680 0x7f833c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.159+0000 7f8340fcb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.159+0000 7f8340fcb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c1066d0 0x7f833c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.159+0000 7f8340fcb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1115c0 con 0x7f833c102030 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8340fcb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f833c111440 con 0x7f833c101680 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8340fcb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f833c111740 con 0x7f833c1066d0 2024-06-02T23:11:28.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43484/0 (socket says 172.21.15.52:43484) 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 -- 172.21.15.52:0/404164033 learned_addr learned my addr 172.21.15.52:0/404164033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f833a59c700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c101680 0x7f833c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f833ad9d700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c1066d0 0x7f833c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 -- 172.21.15.52:0/404164033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c1066d0 msgr2=0x7f833c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c1066d0 0x7f833c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 -- 172.21.15.52:0/404164033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c101680 msgr2=0x7f833c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c101680 0x7f833c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 -- 172.21.15.52:0/404164033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c1990c0 con 0x7f833c102030 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f8339d9b700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c19ec90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8330009fd0 tx=0x7f8330005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.160+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330026070 con 0x7f833c102030 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.161+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8330004ca0 con 0x7f833c102030 2024-06-02T23:11:28.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.161+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c199350 con 0x7f833c102030 2024-06-02T23:11:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.161+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8330003d00 con 0x7f833c102030 2024-06-02T23:11:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.161+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1a5620 con 0x7f833c102030 2024-06-02T23:11:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.162+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f833c069180 con 0x7f833c102030 2024-06-02T23:11:28.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.164+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8330029070 con 0x7f833c102030 2024-06-02T23:11:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.164+0000 7f832b7fe700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83240a2ca0 0x7f83240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:28.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.164+0000 7f833a59c700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83240a2ca0 0x7f83240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.164+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83300cfdc0 con 0x7f833c102030 2024-06-02T23:11:28.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.165+0000 7f833a59c700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83240a2ca0 0x7f83240a5160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f833c077990 tx=0x7f832c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:28.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.168+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8330030020 con 0x7f833c102030 2024-06-02T23:11:28.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.376+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f833c1a57f0 con 0x7f833c102030 2024-06-02T23:11:28.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.377+0000 7f832b7fe700 1 -- 172.21.15.52:0/404164033 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7f8330013070 con 0x7f833c102030 2024-06-02T23:11:28.378 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:28.378 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.379+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83240a2ca0 msgr2=0x7f83240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.379+0000 7f8340fcb700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83240a2ca0 0x7f83240a5160 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f833c077990 tx=0x7f832c00a380 comp rx=0 tx=0).stop 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.379+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 msgr2=0x7f833c19ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.379+0000 7f8340fcb700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c19ec90 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f8330009fd0 tx=0x7f8330005f10 comp rx=0 tx=0).stop 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 shutdown_connections 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83240a2ca0 0x7f83240a5160 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c1066d0 0x7f833c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f833c101680 0x7f833c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 --2- 172.21.15.52:0/404164033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f833c102030 0x7f833c19ec90 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:28.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 >> 172.21.15.52:0/404164033 conn(0x7f833c0773e0 msgr2=0x7f833c104fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:28.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 shutdown_connections 2024-06-02T23:11:28.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:28.380+0000 7f8340fcb700 1 -- 172.21.15.52:0/404164033 wait complete. 2024-06-02T23:11:29.162 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-06-02T23:11:29.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:29 smithi052 ceph-mon[32340]: pgmap v419: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-06-02T23:11:29.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:29 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/404164033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:11:29.419 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:29 smithi123 ceph-mon[35977]: pgmap v419: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-06-02T23:11:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:29 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/404164033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:11:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:29 smithi174 ceph-mon[35994]: pgmap v419: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.4 KiB/s wr, 14 op/s 2024-06-02T23:11:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:29 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/404164033' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:11:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:31 smithi052 ceph-mon[32340]: pgmap v420: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-02T23:11:31.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:31 smithi123 ceph-mon[35977]: pgmap v420: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-02T23:11:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:31 smithi174 ceph-mon[35994]: pgmap v420: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-02T23:11:31.995 INFO:teuthology.orchestra.run.smithi052.stdout:{"run_dir":"/var/run/ceph"} 2024-06-02T23:11:31.995 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-06-02T23:11:31.996 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 2064 2024-06-02T23:11:31.996 DEBUG:tasks.quiescer:Yielding 2024-06-02T23:11:31.996 INFO:teuthology.run_tasks:Running task workunit... 2024-06-02T23:11:32.010 INFO:tasks.workunit:Pulling workunits from ref dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-02T23:11:32.011 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-06-02T23:11:32.011 INFO:tasks.workunit:timeout=3h 2024-06-02T23:11:32.011 INFO:tasks.workunit:cleanup=True 2024-06-02T23:11:32.011 DEBUG:teuthology.orchestra.run.smithi052:> stat -- /home/ubuntu/cephtest/mnt.0 2024-06-02T23:11:32.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:11:32.032 INFO:teuthology.orchestra.run.smithi052.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:Device: 77h/119d Inode: 1 Links: 2 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:Context: system_u:object_r:fusefs_t:s0 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:Access: 2024-06-02 23:08:01.243796286 +0000 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:Modify: 2024-06-02 23:08:01.243796286 +0000 2024-06-02T23:11:32.033 INFO:teuthology.orchestra.run.smithi052.stdout:Change: 2024-06-02 23:10:55.336791728 +0000 2024-06-02T23:11:32.034 INFO:teuthology.orchestra.run.smithi052.stdout: Birth: - 2024-06-02T23:11:32.034 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-06-02T23:11:32.034 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-06-02T23:11:32.107 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 2024-06-02T23:11:32.164 INFO:tasks.workunit.client.0.smithi052.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-06-02T23:11:32.272 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:33.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:33 smithi052 ceph-mon[32340]: pgmap v421: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-02T23:11:33.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:33 smithi123 ceph-mon[35977]: pgmap v421: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-02T23:11:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 -- 172.21.15.52:0/2719613924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 msgr2=0x7effb0109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 --2- 172.21.15.52:0/2719613924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb0109be0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7effa8009ca0 tx=0x7effa802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 -- 172.21.15.52:0/2719613924 shutdown_connections 2024-06-02T23:11:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 --2- 172.21.15.52:0/2719613924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb0109be0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 --2- 172.21.15.52:0/2719613924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0104e60 0x7effb0107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 --2- 172.21.15.52:0/2719613924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effb0102530 0x7effb0104920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 -- 172.21.15.52:0/2719613924 >> 172.21.15.52:0/2719613924 conn(0x7effb00fbf20 msgr2=0x7effb00fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 -- 172.21.15.52:0/2719613924 shutdown_connections 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.577+0000 7effb599d640 1 -- 172.21.15.52:0/2719613924 wait complete. 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 Processor -- start 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 -- start start 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effb0102530 0x7effb0197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0104e60 0x7effb0198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb019f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7effb010d570 con 0x7effb0107790 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effb010d3f0 con 0x7effb0104e60 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effb599d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7effb010d6f0 con 0x7effb0102530 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb019f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb019f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43506/0 (socket says 172.21.15.52:43506) 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effaf7fe640 1 -- 172.21.15.52:0/3561403668 learned_addr learned my addr 172.21.15.52:0/3561403668 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effaeffd640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effb0102530 0x7effb0197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.578+0000 7effae7fc640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0104e60 0x7effb0198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effaf7fe640 1 -- 172.21.15.52:0/3561403668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effb0102530 msgr2=0x7effb0197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effaf7fe640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effb0102530 0x7effb0197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effaf7fe640 1 -- 172.21.15.52:0/3561403668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0104e60 msgr2=0x7effb0198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effaf7fe640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0104e60 0x7effb0198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effaf7fe640 1 -- 172.21.15.52:0/3561403668 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effb019f720 con 0x7effb0107790 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effaf7fe640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb019f0b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effa8009dc0 tx=0x7effa8003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effb499b640 1 -- 172.21.15.52:0/3561403668 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8032ec0 con 0x7effb0107790 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effb019f9b0 con 0x7effb0107790 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effb499b640 1 -- 172.21.15.52:0/3561403668 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa80044f0 con 0x7effb0107790 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effb499b640 1 -- 172.21.15.52:0/3561403668 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa8030d40 con 0x7effb0107790 2024-06-02T23:11:33.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.579+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effb019ff50 con 0x7effb0107790 2024-06-02T23:11:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.582+0000 7effb499b640 1 -- 172.21.15.52:0/3561403668 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7effa8004040 con 0x7effb0107790 2024-06-02T23:11:33.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.582+0000 7effb499b640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff980a2dd0 0x7eff980a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.582+0000 7effb499b640 1 -- 172.21.15.52:0/3561403668 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7effa80f07a0 con 0x7effb0107790 2024-06-02T23:11:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.582+0000 7effb8781740 1 Processor -- start 2024-06-02T23:11:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.582+0000 7effaeffd640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff980a2dd0 0x7eff980a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.582+0000 7effb8781740 1 -- start start 2024-06-02T23:11:33.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effb8781740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc68e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effb8781740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc693500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effb8781740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc696000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effb8781740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc5f14f0 con 0x55f4cc68ea00 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effb8781740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc58f870 con 0x55f4cc68e010 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effb8781740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc5ed4b0 con 0x55f4cc693b70 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc693500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc693500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43522/0 (socket says 172.21.15.52:43522) 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 -- 172.21.15.52:0/3868007074 learned_addr learned my addr 172.21.15.52:0/3868007074 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effaeffd640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff980a2dd0 0x7eff980a5290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effa0005d00 tx=0x7effa000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effaeffd640 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc68e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effaf7fe640 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc696000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 -- 172.21.15.52:0/3868007074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 msgr2=0x55f4cc696000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc696000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.585 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 -- 172.21.15.52:0/3868007074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 msgr2=0x55f4cc68e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc68e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 -- 172.21.15.52:0/3868007074 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4cc5ab9b0 con 0x55f4cc68ea00 2024-06-02T23:11:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.583+0000 7effae7fc640 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc693500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7effa400dbd0 tx=0x7effa400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c43d5e90213c7a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.584+0000 7eff95ffb640 1 -- 172.21.15.52:0/3868007074 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4004030 con 0x55f4cc68ea00 2024-06-02T23:11:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.584+0000 7eff95ffb640 1 -- 172.21.15.52:0/3868007074 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa40041d0 con 0x55f4cc68ea00 2024-06-02T23:11:33.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.584+0000 7eff95ffb640 1 -- 172.21.15.52:0/3868007074 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4011640 con 0x55f4cc68ea00 2024-06-02T23:11:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.584+0000 7effb8781740 1 -- 172.21.15.52:0/3868007074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 msgr2=0x55f4cc693500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.584+0000 7effb8781740 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc693500 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7effa400dbd0 tx=0x7effa400f730 comp rx=0 tx=0).stop 2024-06-02T23:11:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 -- 172.21.15.52:0/3868007074 shutdown_connections 2024-06-02T23:11:33.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc696000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc693500 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 --2- 172.21.15.52:0/3868007074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc68e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 -- 172.21.15.52:0/3868007074 >> 172.21.15.52:0/3868007074 conn(0x55f4cc689680 msgr2=0x55f4cc689a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 -- 172.21.15.52:0/3868007074 shutdown_connections 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 -- 172.21.15.52:0/3868007074 wait complete. 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 -- start start 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.585+0000 7effb8781740 1 Processor -- start 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc698250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc698790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc69f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc696b20 con 0x55f4cc68ea00 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc57aae0 con 0x55f4cc68e010 2024-06-02T23:11:33.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4cc696ca0 con 0x55f4cc693b70 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effb8781740 10 client.0 _ll_register_callbacks cb 0x55f4cc5b39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc698790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc698790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43536/0 (socket says 172.21.15.52:43536) 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 -- 172.21.15.52:0/1817596897 learned_addr learned my addr 172.21.15.52:0/1817596897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effaeffd640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc698250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effaf7fe640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc69f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 -- 172.21.15.52:0/1817596897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 msgr2=0x55f4cc69f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc69f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 -- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 msgr2=0x55f4cc698250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc698250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 -- 172.21.15.52:0/1817596897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4cc305210 con 0x55f4cc68ea00 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effaf7fe640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc69f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:33.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.586+0000 7effae7fc640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc698790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7effa4010010 tx=0x7effa400d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6634dfb099fca6c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7eff8effd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7eff8effd640 1 -- 172.21.15.52:0/1817596897 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa4003fc0 con 0x55f4cc68ea00 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa40041d0 con 0x55f4cc68ea00 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7eff8effd640 1 -- 172.21.15.52:0/1817596897 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7effa40035f0 con 0x55f4cc68ea00 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7eff8effd640 1 -- 172.21.15.52:0/1817596897 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effa40037f0 con 0x55f4cc68ea00 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7effb8781740 10 client.15384 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4cc3d1620 con 0x55f4cc68ea00 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7eff8effd640 1 -- 172.21.15.52:0/1817596897 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa40041d0 con 0x55f4cc68ea00 2024-06-02T23:11:33.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7eff8effd640 1 -- 172.21.15.52:0/1817596897 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7effa4004a20 con 0x55f4cc68ea00 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7effb8781740 10 client.15384 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7effb8781740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.587+0000 7effb8781740 20 client.15384 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.588+0000 7effb8781740 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc6a3ef0 0x55f4cc6a6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.588+0000 7effb8781740 4 client.15384 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.588+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4cc604f10 con 0x55f4cc6a3ef0 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.588+0000 7effaeffd640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc6a3ef0 0x55f4cc6a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.588+0000 7eff8effd640 10 client.15384 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.588+0000 7effaeffd640 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc6a3ef0 0x55f4cc6a6320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5459915c377ed9d server_cookie=5d9b15a66ae9dc3e in_seq=0 out_seq=0 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.589+0000 7eff8effd640 1 -- 172.21.15.52:0/1817596897 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f4cc604f10 con 0x55f4cc6a3ef0 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.589+0000 7eff8effd640 10 client.15384 handle_command_reply: tid=0 2024-06-02T23:11:33.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.589+0000 7effb8781740 1 client.15384 shutdown 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.589+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 msgr2=0x55f4cc698790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.589+0000 7effb8781740 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc698790 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7effa4010010 tx=0x7effa400d910 comp rx=0 tx=0).stop 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 shutdown_connections 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc6a3ef0 0x55f4cc6a6320 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc693b70 0x55f4cc69f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc68ea00 0x55f4cc698790 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 --2- 172.21.15.52:0/1817596897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc698250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 >> 172.21.15.52:0/1817596897 conn(0x55f4cc689680 msgr2=0x55f4cc6887e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7eff8effd640 0 client.15384 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 shutdown_connections 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 1 -- 172.21.15.52:0/1817596897 wait complete. 2024-06-02T23:11:33.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.590+0000 7effb8781740 20 client.15384 trim_cache size 0 max 16384 2024-06-02T23:11:33.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 Processor -- start 2024-06-02T23:11:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 -- start start 2024-06-02T23:11:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc687f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68ea00 0x55f4cc69f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc688490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7effb01a8bf0 con 0x55f4cc693b70 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effb01aa730 con 0x55f4cc68e010 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.615+0000 7effb8781740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7effb01a92c0 con 0x55f4cc68ea00 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc688490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc688490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43544/0 (socket says 172.21.15.52:43544) 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 -- 172.21.15.52:0/955084108 learned_addr learned my addr 172.21.15.52:0/955084108 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaeffd640 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc687f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effae7fc640 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68ea00 0x55f4cc69f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 -- 172.21.15.52:0/955084108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68ea00 msgr2=0x55f4cc69f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68ea00 0x55f4cc69f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 -- 172.21.15.52:0/955084108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 msgr2=0x55f4cc687f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc687f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 -- 172.21.15.52:0/955084108 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4cc6a0c50 con 0x55f4cc693b70 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effae7fc640 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68ea00 0x55f4cc69f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7effaf7fe640 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc688490 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7effa8031970 tx=0x7effa8003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b03e37f3c8a0ca81 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7eff957fa640 1 -- 172.21.15.52:0/955084108 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa80bd030 con 0x55f4cc693b70 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7eff957fa640 1 -- 172.21.15.52:0/955084108 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa8036030 con 0x55f4cc693b70 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.616+0000 7eff957fa640 1 -- 172.21.15.52:0/955084108 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa80c10d0 con 0x55f4cc693b70 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 -- 172.21.15.52:0/955084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 msgr2=0x55f4cc688490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc688490 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7effa8031970 tx=0x7effa8003f90 comp rx=0 tx=0).stop 2024-06-02T23:11:33.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 -- 172.21.15.52:0/955084108 shutdown_connections 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68ea00 0x55f4cc69f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc688490 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 --2- 172.21.15.52:0/955084108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68e010 0x55f4cc687f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 -- 172.21.15.52:0/955084108 >> 172.21.15.52:0/955084108 conn(0x55f4cc689680 msgr2=0x55f4cc698250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 -- 172.21.15.52:0/955084108 shutdown_connections 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 -- 172.21.15.52:0/955084108 wait complete. 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.617+0000 7effb8781740 1 -- start start 2024-06-02T23:11:33.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 Processor -- start 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68e010 0x55f4cc69c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68ea00 0x55f4cc69cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc70cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7effb01aa060 con 0x55f4cc693b70 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7effb01a9990 con 0x55f4cc68ea00 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7effb01a6bd0 con 0x55f4cc68e010 2024-06-02T23:11:33.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effb8781740 10 client.0 _ll_register_callbacks cb 0x7effa4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc70cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc70cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43556/0 (socket says 172.21.15.52:43556) 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 -- 172.21.15.52:0/2847191215 learned_addr learned my addr 172.21.15.52:0/2847191215 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaeffd640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68e010 0x55f4cc69c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 -- 172.21.15.52:0/2847191215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68e010 msgr2=0x55f4cc69c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effae7fc640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68ea00 0x55f4cc69cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68e010 0x55f4cc69c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 -- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68ea00 msgr2=0x55f4cc69cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68ea00 0x55f4cc69cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.618+0000 7effaf7fe640 1 -- 172.21.15.52:0/2847191215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4cc70d1b0 con 0x55f4cc693b70 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effaf7fe640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc70cbf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7effa80ee320 tx=0x7effa80bd890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2abf62044aadd6c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:33.625 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7eff8effd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7eff8effd640 1 -- 172.21.15.52:0/2847191215 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effa80d5030 con 0x55f4cc693b70 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7eff8effd640 1 -- 172.21.15.52:0/2847191215 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7effa80c10d0 con 0x55f4cc693b70 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effa8036030 con 0x55f4cc693b70 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7eff8effd640 1 -- 172.21.15.52:0/2847191215 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7effa80bd030 con 0x55f4cc693b70 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 10 client.15390 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4cc70e4c0 con 0x55f4cc693b70 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7eff8effd640 1 -- 172.21.15.52:0/2847191215 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7effa8036030 con 0x55f4cc693b70 2024-06-02T23:11:33.626 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7eff8effd640 1 -- 172.21.15.52:0/2847191215 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7effa80c5810 con 0x55f4cc693b70 2024-06-02T23:11:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 10 client.15390 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 20 client.15390 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc70ede0 0x55f4cc711200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:33.627 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 4 client.15390 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:11:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4cc5fec20 con 0x55f4cc70ede0 2024-06-02T23:11:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.619+0000 7effaeffd640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc70ede0 0x55f4cc711200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.620+0000 7effaeffd640 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc70ede0 0x55f4cc711200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4db5701a4f04559 server_cookie=fd30b42b9c4a6e91 in_seq=0 out_seq=0 2024-06-02T23:11:33.628 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.620+0000 7eff8effd640 10 client.15390 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:33.629 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7eff8effd640 1 -- 172.21.15.52:0/2847191215 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f4cc5fec20 con 0x55f4cc70ede0 2024-06-02T23:11:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7eff8effd640 10 client.15390 handle_command_reply: tid=0 2024-06-02T23:11:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 client.15390 shutdown 2024-06-02T23:11:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 msgr2=0x55f4cc70cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc70cbf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7effa80ee320 tx=0x7effa80bd890 comp rx=0 tx=0).stop 2024-06-02T23:11:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 shutdown_connections 2024-06-02T23:11:33.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4cc70ede0 0x55f4cc711200 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4cc693b70 0x55f4cc70cbf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4cc68ea00 0x55f4cc69cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 --2- 172.21.15.52:0/2847191215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4cc68e010 0x55f4cc69c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7eff8effd640 0 client.15390 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 >> 172.21.15.52:0/2847191215 conn(0x55f4cc689680 msgr2=0x55f4cc68c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:33.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 shutdown_connections 2024-06-02T23:11:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 1 -- 172.21.15.52:0/2847191215 wait complete. 2024-06-02T23:11:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.628+0000 7effb8781740 20 client.15390 trim_cache size 0 max 16384 2024-06-02T23:11:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:11:33.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:11:33.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:11:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff980a2dd0 msgr2=0x7eff980a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff980a2dd0 0x7eff980a5290 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7effa0005d00 tx=0x7effa000b410 comp rx=0 tx=0).stop 2024-06-02T23:11:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 msgr2=0x7effb019f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:33.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb019f0b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7effa8009dc0 tx=0x7effa8003ba0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 shutdown_connections 2024-06-02T23:11:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff980a2dd0 0x7eff980a5290 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7effb0107790 0x7effb019f0b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb0104e60 0x7effb0198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 --2- 172.21.15.52:0/3561403668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7effb0102530 0x7effb0197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 >> 172.21.15.52:0/3561403668 conn(0x7effb00fbf20 msgr2=0x7effb0102e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 shutdown_connections 2024-06-02T23:11:33.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:33.629+0000 7effb599d640 1 -- 172.21.15.52:0/3561403668 wait complete. 2024-06-02T23:11:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:33 smithi174 ceph-mon[35994]: pgmap v421: 97 pgs: 97 active+clean; 652 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.4 KiB/s wr, 13 op/s 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 -- 172.21.15.52:0/1408540180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8524102530 msgr2=0x7f85241029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 --2- 172.21.15.52:0/1408540180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8524102530 0x7f85241029b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f85180099c0 tx=0x7f8518009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 -- 172.21.15.52:0/1408540180 shutdown_connections 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 --2- 172.21.15.52:0/1408540180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 0x7f852410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 --2- 172.21.15.52:0/1408540180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f85241085d0 0x7f85241089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 --2- 172.21.15.52:0/1408540180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8524102530 0x7f85241029b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 -- 172.21.15.52:0/1408540180 >> 172.21.15.52:0/1408540180 conn(0x7f8524079640 msgr2=0x7f8524079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 -- 172.21.15.52:0/1408540180 shutdown_connections 2024-06-02T23:11:34.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.074+0000 7f8529f49700 1 -- 172.21.15.52:0/1408540180 wait complete. 2024-06-02T23:11:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 Processor -- start 2024-06-02T23:11:34.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 -- start start 2024-06-02T23:11:34.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8524102530 0x7f852419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:34.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 0x7f852419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:34.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 0x7f85241988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:34.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f85241126c0 con 0x7f85241085d0 2024-06-02T23:11:34.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112540 con 0x7f8524102ef0 2024-06-02T23:11:34.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8529f49700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8524112840 con 0x7f8524102530 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8523fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 0x7f85241988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8523fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 0x7f85241988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52506/0 (socket says 172.21.15.52:52506) 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8522ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 0x7f852419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.075+0000 7f8523fff700 1 -- 172.21.15.52:0/4294791555 learned_addr learned my addr 172.21.15.52:0/4294791555 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f85237fe700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8524102530 0x7f852419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8523fff700 1 -- 172.21.15.52:0/4294791555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8524102530 msgr2=0x7f852419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8523fff700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8524102530 0x7f852419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8523fff700 1 -- 172.21.15.52:0/4294791555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 msgr2=0x7f852419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8523fff700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 0x7f852419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8523fff700 1 -- 172.21.15.52:0/4294791555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8524199020 con 0x7f85241085d0 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8522ffd700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 0x7f852419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:11:34.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8523fff700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 0x7f85241988c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f851c00ef30 tx=0x7f851c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:34.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c0092e0 con 0x7f85241085d0 2024-06-02T23:11:34.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f851c004bb0 con 0x7f85241085d0 2024-06-02T23:11:34.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241992b0 con 0x7f85241085d0 2024-06-02T23:11:34.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c004060 con 0x7f85241085d0 2024-06-02T23:11:34.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.076+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241a55d0 con 0x7f85241085d0 2024-06-02T23:11:34.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.077+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8524069180 con 0x7f85241085d0 2024-06-02T23:11:34.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.079+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f851c01b030 con 0x7f85241085d0 2024-06-02T23:11:34.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.079+0000 7f8520ff9700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f850c0a2c20 0x7f850c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:34.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.079+0000 7f85237fe700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f850c0a2c20 0x7f850c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:34.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.079+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f851c0ceba0 con 0x7f85241085d0 2024-06-02T23:11:34.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.080+0000 7f85237fe700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f850c0a2c20 0x7f850c0a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8514005ea0 tx=0x7f8514005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:34.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.084+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f851c022020 con 0x7f85241085d0 2024-06-02T23:11:34.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.342+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f852402dca0 con 0x7f85241085d0 2024-06-02T23:11:34.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.343+0000 7f8520ff9700 1 -- 172.21.15.52:0/4294791555 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f851c010040 con 0x7f85241085d0 2024-06-02T23:11:34.343 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:34.343 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:11:34.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f850c0a2c20 msgr2=0x7f850c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:34.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f850c0a2c20 0x7f850c0a50e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8514005ea0 tx=0x7f8514005e30 comp rx=0 tx=0).stop 2024-06-02T23:11:34.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 msgr2=0x7f85241988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:34.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 0x7f85241988c0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f851c00ef30 tx=0x7f851c00c3a0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 shutdown_connections 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f850c0a2c20 0x7f850c0a50e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f85241085d0 0x7f85241988c0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8524102530 0x7f852419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 --2- 172.21.15.52:0/4294791555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8524102ef0 0x7f852419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 >> 172.21.15.52:0/4294791555 conn(0x7f8524079640 msgr2=0x7f85241010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 shutdown_connections 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:34.345+0000 7f8529f49700 1 -- 172.21.15.52:0/4294791555 wait complete. 2024-06-02T23:11:34.346 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:11:34.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-06-02T23:07:33.296248+0000', 'modified': '2024-06-02T23:10:55.722049+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24470, 'mds_1': 24451, 'mds_2': 24449, 'mds_3': 24436, 'mds_4': 24434}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14604': {'gid': 14604, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6839/2175393174', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6838', 'nonce': 2175393174}, {'type': 'v1', 'addr': '172.21.15.52:6839', 'nonce': 2175393174}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14616': {'gid': 14616, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.52:6841/4238066654', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.52:6840', 'nonce': 4238066654}, {'type': 'v1', 'addr': '172.21.15.52:6841', 'nonce': 4238066654}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24400': {'gid': 24400, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/224943979', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 224943979}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 224943979}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.123:6833/1267486579', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6832', 'nonce': 1267486579}, {'type': 'v1', 'addr': '172.21.15.123:6833', 'nonce': 1267486579}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/2486297365', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 2486297365}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 2486297365}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.123:6835/3092837185', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 3092837185}, {'type': 'v1', 'addr': '172.21.15.123:6835', 'nonce': 3092837185}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.174:6837/457473811', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 457473811}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 457473811}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.123:6837/785283486', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 785283486}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 785283486}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24451': {'gid': 24451, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.174:6839/2115890873', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 2115890873}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 2115890873}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.123:6839/2043673684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6838', 'nonce': 2043673684}, {'type': 'v1', 'addr': '172.21.15.123:6839', 'nonce': 2043673684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24470, 'qdb_cluster': [24434, 24436, 24451, 24470, 24449]} 2024-06-02T23:11:34.968 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-06-02T23:11:34.968 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.e status 2024-06-02T23:11:35.267 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:35.390 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:35 smithi052 ceph-mon[32340]: pgmap v422: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-06-02T23:11:35.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:35 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4294791555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:35.391 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:35 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:11:35.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:35 smithi123 ceph-mon[35977]: pgmap v422: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-06-02T23:11:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:35 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4294791555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:35 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:11:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:35 smithi174 ceph-mon[35994]: pgmap v422: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-06-02T23:11:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:35 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4294791555' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:35 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:11:36.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:36.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:36.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:11:36.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:11:36.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:11:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:11:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:36.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2845208471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f006abe0 msgr2=0x7f17f006b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2845208471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f006abe0 0x7f17f006b060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f17e4009990 tx=0x7f17e4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2845208471 shutdown_connections 2024-06-02T23:11:36.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2845208471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17f006b5a0 0x7f17f010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2845208471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f0108520 0x7f17f0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2845208471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f006abe0 0x7f17f006b060 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2845208471 >> 172.21.15.52:0/2845208471 conn(0x7f17f0078330 msgr2=0x7f17f0078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2845208471 shutdown_connections 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.924+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2845208471 wait complete. 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 Processor -- start 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 -- start start 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17f006abe0 0x7f17f019e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f006b5a0 0x7f17f019edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 0x7f17f0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f17f0112670 con 0x7f17f0108520 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01124f0 con 0x7f17f006abe0 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17f5d2b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17f01127f0 con 0x7f17f006b5a0 2024-06-02T23:11:36.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 0x7f17f0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 0x7f17f0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52524/0 (socket says 172.21.15.52:52524) 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17effff700 1 -- 172.21.15.52:0/2916433709 learned_addr learned my addr 172.21.15.52:0/2916433709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17eeffd700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f006b5a0 0x7f17f019edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.925+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17f006abe0 0x7f17f019e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17effff700 1 -- 172.21.15.52:0/2916433709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f006b5a0 msgr2=0x7f17f019edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17effff700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f006b5a0 0x7f17f019edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17effff700 1 -- 172.21.15.52:0/2916433709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17f006abe0 msgr2=0x7f17f019e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17effff700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17f006abe0 0x7f17f019e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17effff700 1 -- 172.21.15.52:0/2916433709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17f01991b0 con 0x7f17f0108520 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17effff700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 0x7f17f0198a50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f17e8007ee0 tx=0x7f17e800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17ecff9700 1 -- 172.21.15.52:0/2916433709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e800dbd0 con 0x7f17f0108520 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17ecff9700 1 -- 172.21.15.52:0/2916433709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e800dd70 con 0x7f17f0108520 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17ecff9700 1 -- 172.21.15.52:0/2916433709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e8005990 con 0x7f17f0108520 2024-06-02T23:11:36.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17f01994a0 con 0x7f17f0108520 2024-06-02T23:11:36.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.926+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17f01a5760 con 0x7f17f0108520 2024-06-02T23:11:36.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.929+0000 7f17ecff9700 1 -- 172.21.15.52:0/2916433709 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17e801a020 con 0x7f17f0108520 2024-06-02T23:11:36.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.930+0000 7f17ecff9700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17d80a2c20 0x7f17d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.930+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17d80a2c20 0x7f17d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.930+0000 7f17ecff9700 1 -- 172.21.15.52:0/2916433709 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17e80ce770 con 0x7f17f0108520 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.930+0000 7f18005aeb80 1 Processor -- start 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.930+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17d80a2c20 0x7f17d80a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17f0199b90 tx=0x7f17e000b5b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.930+0000 7f18005aeb80 1 -- start start 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ecd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ec7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ec7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f18005aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c419db9f10 con 0x55c419eccf10 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f18005aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c419e18800 con 0x55c419ecd830 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f18005aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c419d80410 con 0x55c419ec7750 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ecd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ecd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52536/0 (socket says 172.21.15.52:52536) 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 -- 172.21.15.52:0/1976071271 learned_addr learned my addr 172.21.15.52:0/1976071271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17eeffd700 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ec7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17effff700 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ec7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 -- 172.21.15.52:0/1976071271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 msgr2=0x55c419ec7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ec7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 -- 172.21.15.52:0/1976071271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 msgr2=0x55c419ec7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ec7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 -- 172.21.15.52:0/1976071271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c419d67880 con 0x55c419eccf10 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ecd2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f17e000fcc0 tx=0x7f17e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5d00197ff2855f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17d5ffb700 1 -- 172.21.15.52:0/1976071271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0022070 con 0x55c419eccf10 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17d5ffb700 1 -- 172.21.15.52:0/1976071271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e001e450 con 0x55c419eccf10 2024-06-02T23:11:36.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.931+0000 7f17d5ffb700 1 -- 172.21.15.52:0/1976071271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e001d610 con 0x55c419eccf10 2024-06-02T23:11:36.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 -- 172.21.15.52:0/1976071271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 msgr2=0x55c419ecd2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ecd2f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f17e000fcc0 tx=0x7f17e0012000 comp rx=0 tx=0).stop 2024-06-02T23:11:36.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 -- 172.21.15.52:0/1976071271 shutdown_connections 2024-06-02T23:11:36.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ec7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ecd2f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 --2- 172.21.15.52:0/1976071271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ec7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 -- 172.21.15.52:0/1976071271 >> 172.21.15.52:0/1976071271 conn(0x55c419df4030 msgr2=0x55c419ec21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 -- 172.21.15.52:0/1976071271 shutdown_connections 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.932+0000 7f18005aeb80 1 -- 172.21.15.52:0/1976071271 wait complete. 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 -- start start 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 Processor -- start 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ed3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ed0880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ed0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c419db8f70 con 0x55c419eccf10 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c419e147c0 con 0x55c419ecd830 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c419ed4210 con 0x55c419ec7750 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f18005aeb80 10 client.0 _ll_register_callbacks cb 0x55c419dae560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:36.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f17eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ed0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ed3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f17eeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ed0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52540/0 (socket says 172.21.15.52:52540) 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f17eeffd700 1 -- 172.21.15.52:0/198954215 learned_addr learned my addr 172.21.15.52:0/198954215 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ed3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:40270/0 (socket says 172.21.15.52:40270) 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.933+0000 7f17effff700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ed0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17eeffd700 1 -- 172.21.15.52:0/198954215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 msgr2=0x55c419ed3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17eeffd700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ed3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17eeffd700 1 -- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 msgr2=0x55c419ed0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17eeffd700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ed0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17eeffd700 1 -- 172.21.15.52:0/198954215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c419dde700 con 0x55c419eccf10 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ed3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17eeffd700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ed0880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17e4000c00 tx=0x7f17e400bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b422a183cad1f62 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:36.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17ceffd700 1 -- 172.21.15.52:0/198954215 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e4029060 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17ceffd700 1 -- 172.21.15.52:0/198954215 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17e4003ac0 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e001e450 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f17ceffd700 1 -- 172.21.15.52:0/198954215 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17e4003cc0 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f18005aeb80 10 client.15408 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.934+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c419ede970 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f17ceffd700 1 -- 172.21.15.52:0/198954215 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e001e450 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f17ceffd700 1 -- 172.21.15.52:0/198954215 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17e400f8c0 con 0x55c419eccf10 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f18005aeb80 10 client.15408 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f18005aeb80 10 client.15408 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f18005aeb80 20 client.15408 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f18005aeb80 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419ee0f70 0x55c419ee23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f18005aeb80 4 client.15408 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:36.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c419e25d00 con 0x55c419ee0f70 2024-06-02T23:11:36.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.935+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419ee0f70 0x55c419ee23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.936+0000 7f17ceffd700 10 client.15408 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:36.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.936+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419ee0f70 0x55c419ee23e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=da7ab253f0bbf8e0 server_cookie=e80adaefa12f97e2 in_seq=0 out_seq=0 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.936+0000 7f17ceffd700 1 -- 172.21.15.52:0/198954215 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c419e25d00 con 0x55c419ee0f70 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.936+0000 7f17ceffd700 10 client.15408 handle_command_reply: tid=0 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 client.15408 shutdown 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 msgr2=0x55c419ed0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ed0880 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f17e4000c00 tx=0x7f17e400bd80 comp rx=0 tx=0).stop 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 shutdown_connections 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419ee0f70 0x55c419ee23e0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ed0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ec7750 0x55c419ed3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f17ceffd700 0 client.15408 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 --2- 172.21.15.52:0/198954215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419eccf10 0x55c419ed0880 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 >> 172.21.15.52:0/198954215 conn(0x55c419df4030 msgr2=0x55c419ec16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:36.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 shutdown_connections 2024-06-02T23:11:36.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 1 -- 172.21.15.52:0/198954215 wait complete. 2024-06-02T23:11:36.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.937+0000 7f18005aeb80 20 client.15408 trim_cache size 0 max 16384 2024-06-02T23:11:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 Processor -- start 2024-06-02T23:11:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 -- start start 2024-06-02T23:11:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419ed3530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419eccf10 0x55c419ece320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ecd830 0x55c419ec1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c419ee0720 con 0x55c419ec7750 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c419ee0df0 con 0x55c419eccf10 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f18005aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c419ee14c0 con 0x55c419ecd830 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419ed3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419ed3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52542/0 (socket says 172.21.15.52:52542) 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.959+0000 7f17ef7fe700 1 -- 172.21.15.52:0/3074170697 learned_addr learned my addr 172.21.15.52:0/3074170697 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17effff700 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ecd830 0x55c419ec1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17eeffd700 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419eccf10 0x55c419ece320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17ef7fe700 1 -- 172.21.15.52:0/3074170697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ecd830 msgr2=0x55c419ec1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ecd830 0x55c419ec1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17ef7fe700 1 -- 172.21.15.52:0/3074170697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419eccf10 msgr2=0x55c419ece320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419eccf10 0x55c419ece320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17ef7fe700 1 -- 172.21.15.52:0/3074170697 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c419ece860 con 0x55c419ec7750 2024-06-02T23:11:36.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17eeffd700 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419eccf10 0x55c419ece320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:36.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419ed3530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17e0011720 tx=0x7f17e001e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1321655a9e84f0ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:36.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.960+0000 7f17d57fa700 1 -- 172.21.15.52:0/3074170697 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0022030 con 0x55c419ec7750 2024-06-02T23:11:36.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f17d57fa700 1 -- 172.21.15.52:0/3074170697 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e00118d0 con 0x55c419ec7750 2024-06-02T23:11:36.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f17d57fa700 1 -- 172.21.15.52:0/3074170697 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e00053c0 con 0x55c419ec7750 2024-06-02T23:11:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 -- 172.21.15.52:0/3074170697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 msgr2=0x55c419ed3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419ed3530 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f17e0011720 tx=0x7f17e001e720 comp rx=0 tx=0).stop 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 -- 172.21.15.52:0/3074170697 shutdown_connections 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419ecd830 0x55c419ec1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419ed3530 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 --2- 172.21.15.52:0/3074170697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419eccf10 0x55c419ece320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 -- 172.21.15.52:0/3074170697 >> 172.21.15.52:0/3074170697 conn(0x55c419df4030 msgr2=0x55c419ecdc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 -- 172.21.15.52:0/3074170697 shutdown_connections 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.961+0000 7f18005aeb80 1 -- 172.21.15.52:0/3074170697 wait complete. 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 -- start start 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 Processor -- start 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419eebd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419eccf10 0x55c419ee8a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ee8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c419ee2260 con 0x55c419ec7750 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c419ee1b90 con 0x55c419ecd830 2024-06-02T23:11:36.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c419ee2930 con 0x55c419eccf10 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f18005aeb80 10 client.0 _ll_register_callbacks cb 0x55c419dae560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.962+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419eebd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419eebd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52552/0 (socket says 172.21.15.52:52552) 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 -- 172.21.15.52:0/2237119855 learned_addr learned my addr 172.21.15.52:0/2237119855 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17eeffd700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419eccf10 0x55c419ee8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17effff700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ee8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 -- 172.21.15.52:0/2237119855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419eccf10 msgr2=0x55c419ee8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419eccf10 0x55c419ee8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 -- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 msgr2=0x55c419ee8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ee8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 -- 172.21.15.52:0/2237119855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c419ee9520 con 0x55c419ec7750 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ef7fe700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419eebd60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f17e000f640 tx=0x7f17e001e540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31166e9fab184e41 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:36.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ceffd700 1 -- 172.21.15.52:0/2237119855 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17e0022070 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f17ceffd700 1 -- 172.21.15.52:0/2237119855 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17e0011a80 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.963+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17e00118d0 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f17ceffd700 1 -- 172.21.15.52:0/2237119855 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f17e004b020 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 10 client.15414 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c419ec2010 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f17ceffd700 1 -- 172.21.15.52:0/2237119855 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17e00118d0 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f17ceffd700 1 -- 172.21.15.52:0/2237119855 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f17e00474e0 con 0x55c419ec7750 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 10 client.15414 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 10 client.15414 resolve_mds: resolved name 'e' to daemon mds.e 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 20 client.15414 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419f54e60 0x55c419f53b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 4 client.15414 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:11:36.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "status"}) -- 0x55c419e20280 con 0x55c419f54e60 2024-06-02T23:11:36.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.964+0000 7f17eeffd700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419f54e60 0x55c419f53b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f17eeffd700 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419f54e60 0x55c419f53b70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4a9d8f8e32027c2e server_cookie=90c852be5c41c69f in_seq=0 out_seq=0 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f17ceffd700 10 client.15414 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f17ceffd700 1 -- 172.21.15.52:0/2237119855 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c419e20280 con 0x55c419f54e60 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f17ceffd700 10 client.15414 handle_command_reply: tid=0 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 client.15414 shutdown 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 msgr2=0x55c419eebd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419eebd60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f17e000f640 tx=0x7f17e001e540 comp rx=0 tx=0).stop 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 shutdown_connections 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c419f54e60 0x55c419f53b70 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c419ecd830 0x55c419ee8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c419ec7750 0x55c419eebd60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 --2- 172.21.15.52:0/2237119855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c419eccf10 0x55c419ee8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 >> 172.21.15.52:0/2237119855 conn(0x55c419df4030 msgr2=0x55c419ed20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:36.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f17ceffd700 0 client.15414 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:36.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 shutdown_connections 2024-06-02T23:11:36.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 1 -- 172.21.15.52:0/2237119855 wait complete. 2024-06-02T23:11:36.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.965+0000 7f18005aeb80 20 client.15414 trim_cache size 0 max 16384 2024-06-02T23:11:36.970 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:36.971 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:11:36.971 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 4, 2024-06-02T23:11:36.971 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24434, 2024-06-02T23:11:36.971 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 225.696104795, 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 28, 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 283.44887654600001 2024-06-02T23:11:36.972 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17d80a2c20 msgr2=0x7f17d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17d80a2c20 0x7f17d80a50e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f17f0199b90 tx=0x7f17e000b5b0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 msgr2=0x7f17f0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 0x7f17f0198a50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f17e8007ee0 tx=0x7f17e800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 shutdown_connections 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17d80a2c20 0x7f17d80a50e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17f0108520 0x7f17f0198a50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17f006abe0 0x7f17f019e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 --2- 172.21.15.52:0/2916433709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17f006b5a0 0x7f17f019edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 >> 172.21.15.52:0/2916433709 conn(0x7f17f0078330 msgr2=0x7f17f01001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 shutdown_connections 2024-06-02T23:11:36.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:36.966+0000 7f17f5d2b700 1 -- 172.21.15.52:0/2916433709 wait complete. 2024-06-02T23:11:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:11:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:11:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:11:37.704 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.c status 2024-06-02T23:11:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:37 smithi174 ceph-mon[35994]: pgmap v423: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-06-02T23:11:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:37 smithi052 ceph-mon[32340]: pgmap v423: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-06-02T23:11:37.963 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:37 smithi123 ceph-mon[35977]: pgmap v423: 97 pgs: 97 active+clean; 657 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-06-02T23:11:39.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:39 smithi052 ceph-mon[32340]: pgmap v424: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-02T23:11:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 -- 172.21.15.52:0/3530370972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0101d50 msgr2=0x7fadb010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 --2- 172.21.15.52:0/3530370972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0101d50 0x7fadb010e240 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fada800b0a0 tx=0x7fada800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 -- 172.21.15.52:0/3530370972 shutdown_connections 2024-06-02T23:11:39.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 --2- 172.21.15.52:0/3530370972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0101d50 0x7fadb010e240 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 --2- 172.21.15.52:0/3530370972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadb0107400 0x7fadb01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 --2- 172.21.15.52:0/3530370972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb0101390 0x7fadb0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 -- 172.21.15.52:0/3530370972 >> 172.21.15.52:0/3530370972 conn(0x7fadb0078340 msgr2=0x7fadb0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 -- 172.21.15.52:0/3530370972 shutdown_connections 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.732+0000 7fadb61c0700 1 -- 172.21.15.52:0/3530370972 wait complete. 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.733+0000 7fadb61c0700 1 Processor -- start 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.733+0000 7fadb61c0700 1 -- start start 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadb61c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadb0101390 0x7fadb019d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadb61c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb0101d50 0x7fadb019dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadb61c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 0x7fadb01977e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadb61c0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fadb0111520 con 0x7fadb0107400 2024-06-02T23:11:39.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadb61c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01113a0 con 0x7fadb0101d50 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadb61c0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fadb01116a0 con 0x7fadb0101390 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 0x7fadb01977e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 0x7fadb01977e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52572/0 (socket says 172.21.15.52:52572) 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 -- 172.21.15.52:0/927654522 learned_addr learned my addr 172.21.15.52:0/927654522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fada6ffd700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb0101d50 0x7fadb019dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadb0101390 0x7fadb019d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 -- 172.21.15.52:0/927654522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadb0101390 msgr2=0x7fadb019d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadb0101390 0x7fadb019d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 -- 172.21.15.52:0/927654522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb0101d50 msgr2=0x7fadb019dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb0101d50 0x7fadb019dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.734+0000 7fadaffff700 1 -- 172.21.15.52:0/927654522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadb0197fc0 con 0x7fadb0107400 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.735+0000 7fadaffff700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 0x7fadb01977e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fada800b830 tx=0x7fada8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.735+0000 7fadad7fa700 1 -- 172.21.15.52:0/927654522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8021070 con 0x7fadb0107400 2024-06-02T23:11:39.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.735+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadb0198250 con 0x7fadb0107400 2024-06-02T23:11:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.735+0000 7fadad7fa700 1 -- 172.21.15.52:0/927654522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada8007e30 con 0x7fadb0107400 2024-06-02T23:11:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.735+0000 7fadad7fa700 1 -- 172.21.15.52:0/927654522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada80046a0 con 0x7fadb0107400 2024-06-02T23:11:39.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.735+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadb01a44c0 con 0x7fadb0107400 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.737+0000 7fadad7fa700 1 -- 172.21.15.52:0/927654522 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fada8007980 con 0x7fadb0107400 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.738+0000 7fadad7fa700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad9c0a2c20 0x7fad9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.738+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad9c0a2c20 0x7fad9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.738+0000 7fadad7fa700 1 -- 172.21.15.52:0/927654522 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fada80d0080 con 0x7fadb0107400 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.738+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad9c0a2c20 0x7fad9c0a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fada0005950 tx=0x7fada000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 Processor -- start 2024-06-02T23:11:39.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 -- start start 2024-06-02T23:11:39.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1909e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d194dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d195740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d7d080f10 con 0x557d7d195300 2024-06-02T23:11:39.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d7d0df800 con 0x557d7d190600 2024-06-02T23:11:39.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d7d047410 con 0x557d7d190f20 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52588/0 (socket says 172.21.15.52:52588) 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 -- 172.21.15.52:0/3579183366 learned_addr learned my addr 172.21.15.52:0/3579183366 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1909e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fada6ffd700 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d194dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 -- 172.21.15.52:0/3579183366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 msgr2=0x557d7d194dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d194dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 -- 172.21.15.52:0/3579183366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 msgr2=0x557d7d1909e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1909e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.739+0000 7fadaffff700 1 -- 172.21.15.52:0/3579183366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7d02e880 con 0x557d7d195300 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.740+0000 7fadaffff700 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d195740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fada8098da0 tx=0x7fada800b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db9e59e983dd1a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.740+0000 7fada5ffb700 1 -- 172.21.15.52:0/3579183366 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada802a070 con 0x557d7d195300 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.740+0000 7fada5ffb700 1 -- 172.21.15.52:0/3579183366 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada809d040 con 0x557d7d195300 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.740+0000 7fada5ffb700 1 -- 172.21.15.52:0/3579183366 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada80074e0 con 0x557d7d195300 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.740+0000 7fadc0a43b80 1 -- 172.21.15.52:0/3579183366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 msgr2=0x557d7d195740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.740+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d195740 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fada8098da0 tx=0x7fada800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- 172.21.15.52:0/3579183366 shutdown_connections 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d195740 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1909e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/3579183366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d194dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- 172.21.15.52:0/3579183366 >> 172.21.15.52:0/3579183366 conn(0x557d7d0bb030 msgr2=0x557d7d1891d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- 172.21.15.52:0/3579183366 shutdown_connections 2024-06-02T23:11:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- 172.21.15.52:0/3579183366 wait complete. 2024-06-02T23:11:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- start start 2024-06-02T23:11:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 Processor -- start 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d19c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d199c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d19a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d7d07ff70 con 0x557d7d195300 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d7d0db7c0 con 0x557d7d190600 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.741+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d7d19d1e0 con 0x557d7d190f20 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadc0a43b80 10 client.0 _ll_register_callbacks cb 0x557d7d075560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d19a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d19a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52594/0 (socket says 172.21.15.52:52594) 2024-06-02T23:11:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 -- 172.21.15.52:0/4140137176 learned_addr learned my addr 172.21.15.52:0/4140137176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d19c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fada6ffd700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 -- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 msgr2=0x557d7d199c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d199c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 -- 172.21.15.52:0/4140137176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 msgr2=0x557d7d19c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d19c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 -- 172.21.15.52:0/4140137176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d7d0a5700 con 0x557d7d195300 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadaffff700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d19a150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fada809c880 tx=0x7fada809c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878de62bcbcf21a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fad8effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fad8effd700 1 -- 172.21.15.52:0/4140137176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada802a070 con 0x557d7d195300 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fad8effd700 1 -- 172.21.15.52:0/4140137176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fada809ac40 con 0x557d7d195300 2024-06-02T23:11:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fada809d040 con 0x557d7d195300 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fad8effd700 1 -- 172.21.15.52:0/4140137176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fada80a5520 con 0x557d7d195300 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadc0a43b80 10 client.15426 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.742+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d7d1a7990 con 0x557d7d195300 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fad8effd700 1 -- 172.21.15.52:0/4140137176 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fada809d040 con 0x557d7d195300 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fad8effd700 1 -- 172.21.15.52:0/4140137176 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fada8011070 con 0x557d7d195300 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadc0a43b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadc0a43b80 10 client.15426 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadc0a43b80 20 client.15426 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d1a9db0 0x557d7d1ab220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadc0a43b80 4 client.15426 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d7d0ecd00 con 0x557d7d1a9db0 2024-06-02T23:11:39.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.743+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d1a9db0 0x557d7d1ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.744+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d1a9db0 0x557d7d1ab220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bab7ed8797ba0e08 server_cookie=309a9b192a0f5561 in_seq=0 out_seq=0 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.744+0000 7fad8effd700 10 client.15426 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.744+0000 7fad8effd700 1 -- 172.21.15.52:0/4140137176 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557d7d0ecd00 con 0x557d7d1a9db0 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.744+0000 7fad8effd700 10 client.15426 handle_command_reply: tid=0 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.744+0000 7fadc0a43b80 1 client.15426 shutdown 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 msgr2=0x557d7d19a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d19a150 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fada809c880 tx=0x7fada809c940 comp rx=0 tx=0).stop 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 shutdown_connections 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d19a150 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d19c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d1a9db0 0x557d7d1ab220 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/4140137176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190f20 0x557d7d199c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 >> 172.21.15.52:0/4140137176 conn(0x557d7d0bb030 msgr2=0x557d7d1967d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:39.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fad8effd700 0 client.15426 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:39.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 shutdown_connections 2024-06-02T23:11:39.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 1 -- 172.21.15.52:0/4140137176 wait complete. 2024-06-02T23:11:39.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.745+0000 7fadc0a43b80 20 client.15426 trim_cache size 0 max 16384 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 Processor -- start 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 -- start start 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1966f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 0x557d7d196fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d195300 0x557d7d1974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d7d1a9560 con 0x557d7d190f20 2024-06-02T23:11:39.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d7d1a9c30 con 0x557d7d190600 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d7d1aa300 con 0x557d7d195300 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fada6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 0x557d7d196fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fada6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 0x557d7d196fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52610/0 (socket says 172.21.15.52:52610) 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fada6ffd700 1 -- 172.21.15.52:0/482411749 learned_addr learned my addr 172.21.15.52:0/482411749 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadaffff700 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d195300 0x557d7d1974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.767+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1966f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada6ffd700 1 -- 172.21.15.52:0/482411749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d195300 msgr2=0x557d7d1974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada6ffd700 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d195300 0x557d7d1974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada6ffd700 1 -- 172.21.15.52:0/482411749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 msgr2=0x557d7d1966f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada6ffd700 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1966f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada6ffd700 1 -- 172.21.15.52:0/482411749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d7d189710 con 0x557d7d190f20 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada6ffd700 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 0x557d7d196fb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fad980044c0 tx=0x7fad9800baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94131c1c14ead786 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada57fa700 1 -- 172.21.15.52:0/482411749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98003680 con 0x557d7d190f20 2024-06-02T23:11:39.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada57fa700 1 -- 172.21.15.52:0/482411749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad98003cd0 con 0x557d7d190f20 2024-06-02T23:11:39.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.768+0000 7fada57fa700 1 -- 172.21.15.52:0/482411749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad98011d00 con 0x557d7d190f20 2024-06-02T23:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 -- 172.21.15.52:0/482411749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 msgr2=0x557d7d196fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 0x557d7d196fb0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fad980044c0 tx=0x7fad9800baf0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 -- 172.21.15.52:0/482411749 shutdown_connections 2024-06-02T23:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d195300 0x557d7d1974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190600 0x557d7d1966f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/482411749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d190f20 0x557d7d196fb0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 -- 172.21.15.52:0/482411749 >> 172.21.15.52:0/482411749 conn(0x557d7d0bb030 msgr2=0x557d7d198500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 -- 172.21.15.52:0/482411749 shutdown_connections 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 -- 172.21.15.52:0/482411749 wait complete. 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 -- start start 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.769+0000 7fadc0a43b80 1 Processor -- start 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190600 0x557d7d1b4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190f20 0x557d7d1ade70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d1ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557d7d1ab0a0 con 0x557d7d195300 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557d7d1aa9d0 con 0x557d7d190f20 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d7d1ab770 con 0x557d7d190600 2024-06-02T23:11:39.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadc0a43b80 10 client.0 _ll_register_callbacks cb 0x557d7d075560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d1ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d1ae3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52624/0 (socket says 172.21.15.52:52624) 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 -- 172.21.15.52:0/2742627721 learned_addr learned my addr 172.21.15.52:0/2742627721 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190600 0x557d7d1b4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fada6ffd700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190f20 0x557d7d1ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 -- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190600 msgr2=0x557d7d1b4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190600 0x557d7d1b4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 -- 172.21.15.52:0/2742627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190f20 msgr2=0x557d7d1ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190f20 0x557d7d1ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 -- 172.21.15.52:0/2742627721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d7d1ae920 con 0x557d7d195300 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fada6ffd700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190f20 0x557d7d1ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:39.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fadaffff700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d1ae3b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fada800b3f0 tx=0x7fada8003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c17bf63e5ce2f6f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fad8effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.770+0000 7fad8effd700 1 -- 172.21.15.52:0/2742627721 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada802a070 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad98003cd0 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fad8effd700 1 -- 172.21.15.52:0/2742627721 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fada809ac40 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fad8effd700 1 -- 172.21.15.52:0/2742627721 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fada80074e0 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 10 client.15432 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557d7d1ae920 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fad8effd700 1 -- 172.21.15.52:0/2742627721 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad98003cd0 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fad8effd700 1 -- 172.21.15.52:0/2742627721 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fada8011070 con 0x557d7d195300 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 10 client.15432 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 10 client.15432 resolve_mds: resolved name 'c' to daemon mds.c 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 20 client.15432 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d219b40 0x557d7d218850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 4 client.15432 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "status"}) -- 0x557d7d0e7280 con 0x557d7d219b40 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.771+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d219b40 0x557d7d218850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadaf7fe700 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d219b40 0x557d7d218850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4a250918625d7d3c server_cookie=5792a406da545f73 in_seq=0 out_seq=0 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fad8effd700 10 client.15432 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fad8effd700 1 -- 172.21.15.52:0/2742627721 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557d7d0e7280 con 0x557d7d219b40 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fad8effd700 10 client.15432 handle_command_reply: tid=0 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 client.15432 shutdown 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 msgr2=0x557d7d1ae3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d1ae3b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fada800b3f0 tx=0x7fada8003e60 comp rx=0 tx=0).stop 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 shutdown_connections 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557d7d195300 0x557d7d1ae3b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x557d7d219b40 0x557d7d218850 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d7d190600 0x557d7d1b4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 --2- 172.21.15.52:0/2742627721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557d7d190f20 0x557d7d1ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 >> 172.21.15.52:0/2742627721 conn(0x557d7d0bb030 msgr2=0x557d7d18da90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fad8effd700 0 client.15432 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 shutdown_connections 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.772+0000 7fadc0a43b80 1 -- 172.21.15.52:0/2742627721 wait complete. 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadc0a43b80 20 client.15432 trim_cache size 0 max 16384 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:11:39.777 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 3, 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24436, 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 229.510250326, 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 28, 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 280.74760067699998 2024-06-02T23:11:39.778 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad9c0a2c20 msgr2=0x7fad9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad9c0a2c20 0x7fad9c0a50e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fada0005950 tx=0x7fada000a300 comp rx=0 tx=0).stop 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 msgr2=0x7fadb01977e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 0x7fadb01977e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fada800b830 tx=0x7fada8004ce0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 shutdown_connections 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad9c0a2c20 0x7fad9c0a50e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fadb0107400 0x7fadb01977e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fadb0101390 0x7fadb019d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 --2- 172.21.15.52:0/927654522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fadb0101d50 0x7fadb019dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 >> 172.21.15.52:0/927654522 conn(0x7fadb0078340 msgr2=0x7fadb00ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.773+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 shutdown_connections 2024-06-02T23:11:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:39.774+0000 7fadb61c0700 1 -- 172.21.15.52:0/927654522 wait complete. 2024-06-02T23:11:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:39 smithi174 ceph-mon[35994]: pgmap v424: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-02T23:11:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:39 smithi123 ceph-mon[35977]: pgmap v424: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-06-02T23:11:40.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.h status 2024-06-02T23:11:40.730 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:41.838 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:41 smithi052 ceph-mon[32340]: pgmap v425: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-06-02T23:11:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:41 smithi174 ceph-mon[35994]: pgmap v425: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-06-02T23:11:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:41 smithi123 ceph-mon[35977]: pgmap v425: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2024-06-02T23:11:42.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 -- 172.21.15.52:0/3044984229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0101390 msgr2=0x7fbcf0101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/3044984229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0101390 0x7fbcf0101810 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fbce4009990 tx=0x7fbce4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 -- 172.21.15.52:0/3044984229 shutdown_connections 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/3044984229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101d50 0x7fbcf010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/3044984229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf0107400 0x7fbcf01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/3044984229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0101390 0x7fbcf0101810 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 -- 172.21.15.52:0/3044984229 >> 172.21.15.52:0/3044984229 conn(0x7fbcf0078340 msgr2=0x7fbcf0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 -- 172.21.15.52:0/3044984229 shutdown_connections 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 -- 172.21.15.52:0/3044984229 wait complete. 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.409+0000 7fbcf53e5700 1 Processor -- start 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 -- start start 2024-06-02T23:11:42.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101390 0x7fbcf01139b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf0101d50 0x7fbcf0111ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 0x7fbcf0112400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf0111520 con 0x7fbcf0107400 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01113a0 con 0x7fbcf0101390 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcf53e5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf01116a0 con 0x7fbcf0101d50 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 0x7fbcf0112400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 0x7fbcf0112400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52636/0 (socket says 172.21.15.52:52636) 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 -- 172.21.15.52:0/2979627694 learned_addr learned my addr 172.21.15.52:0/2979627694 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf0101d50 0x7fbcf0111ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101390 0x7fbcf01139b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 -- 172.21.15.52:0/2979627694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf0101d50 msgr2=0x7fbcf0111ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf0101d50 0x7fbcf0111ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 -- 172.21.15.52:0/2979627694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101390 msgr2=0x7fbcf01139b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101390 0x7fbcf01139b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 -- 172.21.15.52:0/2979627694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf01a7080 con 0x7fbcf0107400 2024-06-02T23:11:42.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101390 0x7fbcf01139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:42.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.410+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 0x7fbcf0112400 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbce8007d70 tx=0x7fbce800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:42.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.411+0000 7fbcd7fff700 1 -- 172.21.15.52:0/2979627694 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8016020 con 0x7fbcf0107400 2024-06-02T23:11:42.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.411+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf01a7370 con 0x7fbcf0107400 2024-06-02T23:11:42.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.411+0000 7fbcd7fff700 1 -- 172.21.15.52:0/2979627694 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce800ad40 con 0x7fbcf0107400 2024-06-02T23:11:42.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.411+0000 7fbcd7fff700 1 -- 172.21.15.52:0/2979627694 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce80115e0 con 0x7fbcf0107400 2024-06-02T23:11:42.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.411+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf01a7880 con 0x7fbcf0107400 2024-06-02T23:11:42.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.413+0000 7fbcd7fff700 1 -- 172.21.15.52:0/2979627694 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbce8019020 con 0x7fbcf0107400 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.414+0000 7fbcd7fff700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbcd80a2c20 0x7fbcd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.414+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbcd80a2c20 0x7fbcd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.414+0000 7fbcd7fff700 1 -- 172.21.15.52:0/2979627694 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbce80cefe0 con 0x7fbcf0107400 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.414+0000 7fbcffc68b80 1 Processor -- start 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 -- start start 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 0x55db53e542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e4e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e4e750 0x55db53e4eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55db53d40f10 con 0x55db53e53f10 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55db53d9f800 con 0x55db53e54830 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db53d07410 con 0x55db53e4e750 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 0x55db53e542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 0x55db53e542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52642/0 (socket says 172.21.15.52:52642) 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 -- 172.21.15.52:0/2554391609 learned_addr learned my addr 172.21.15.52:0/2554391609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e4e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbcd80a2c20 0x7fbcd80a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbce0005e10 tx=0x7fbce0005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e4e750 0x55db53e4eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 -- 172.21.15.52:0/2554391609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e4e750 msgr2=0x55db53e4eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e4e750 0x55db53e4eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 -- 172.21.15.52:0/2554391609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 msgr2=0x55db53e4e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e4e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbceeffd700 1 -- 172.21.15.52:0/2554391609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db53cee880 con 0x55db53e53f10 2024-06-02T23:11:42.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.415+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e4e750 0x55db53e4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:42.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.416+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 0x55db53e542f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbce000fd40 tx=0x7fbce0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73ca86c61bbb28b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:42.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.416+0000 7fbcd57fa700 1 -- 172.21.15.52:0/2554391609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0022070 con 0x55db53e53f10 2024-06-02T23:11:42.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.416+0000 7fbcd57fa700 1 -- 172.21.15.52:0/2554391609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce0011bb0 con 0x55db53e53f10 2024-06-02T23:11:42.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.416+0000 7fbcd57fa700 1 -- 172.21.15.52:0/2554391609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0010630 con 0x55db53e53f10 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.416+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2554391609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 msgr2=0x55db53e542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.416+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 0x55db53e542f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbce000fd40 tx=0x7fbce0012000 comp rx=0 tx=0).stop 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2554391609 shutdown_connections 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e4e750 0x55db53e4eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e53f10 0x55db53e542f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2554391609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e4e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2554391609 >> 172.21.15.52:0/2554391609 conn(0x55db53d7b030 msgr2=0x55db53e491d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2554391609 shutdown_connections 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2554391609 wait complete. 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- start start 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 Processor -- start 2024-06-02T23:11:42.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e5a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e57e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55db53d3ff70 con 0x55db53e4e750 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55db53d9b7c0 con 0x55db53e53f10 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db53e5b210 con 0x55db53e54830 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbcffc68b80 10 client.0 _ll_register_callbacks cb 0x55db53d35560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.417+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e5a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcee7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e5a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52656/0 (socket says 172.21.15.52:52656) 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 -- 172.21.15.52:0/2570757847 learned_addr learned my addr 172.21.15.52:0/2570757847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e57e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 -- 172.21.15.52:0/2570757847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 msgr2=0x55db53e57e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e57e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 -- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 msgr2=0x55db53e578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 -- 172.21.15.52:0/2570757847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db53d65700 con 0x55db53e4e750 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e5a550 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbce000ffa0 tx=0x7fbce000ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339223b2b085b03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:42.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2570757847 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0036060 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2570757847 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbce001e820 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce0011bb0 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2570757847 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbce0022050 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcffc68b80 10 client.15444 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.418+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db53e659c0 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2570757847 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce0011bb0 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2570757847 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbce003c3b0 con 0x55db53e4e750 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcffc68b80 10 client.15444 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:42.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcffc68b80 10 client.15444 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcffc68b80 20 client.15444 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53e67b80 0x55db53e68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcffc68b80 4 client.15444 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55db53dacd00 con 0x55db53e67b80 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.419+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53e67b80 0x55db53e68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.420+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53e67b80 0x55db53e68ff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7812b428c79ad243 server_cookie=3f6ab71270882732 in_seq=0 out_seq=0 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.420+0000 7fbcd2ff5700 10 client.15444 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2570757847 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55db53dacd00 con 0x55db53e67b80 2024-06-02T23:11:42.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcd2ff5700 10 client.15444 handle_command_reply: tid=0 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 client.15444 shutdown 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 msgr2=0x55db53e5a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e5a550 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fbce000ffa0 tx=0x7fbce000ffd0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 shutdown_connections 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53e67b80 0x55db53e68ff0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e57e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e5a550 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2570757847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcd2ff5700 0 client.15444 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 >> 172.21.15.52:0/2570757847 conn(0x55db53d7b030 msgr2=0x55db53e486b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 shutdown_connections 2024-06-02T23:11:42.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2570757847 wait complete. 2024-06-02T23:11:42.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.421+0000 7fbcffc68b80 20 client.15444 trim_cache size 0 max 16384 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 Processor -- start 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 -- start start 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e48e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e53f10 0x55db53e49370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e57880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55db53e67330 con 0x55db53e4e750 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55db53e67a00 con 0x55db53e54830 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db53e680d0 con 0x55db53e53f10 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e48e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e48e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52670/0 (socket says 172.21.15.52:52670) 2024-06-02T23:11:42.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 -- 172.21.15.52:0/4200878296 learned_addr learned my addr 172.21.15.52:0/4200878296 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e57880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e53f10 0x55db53e49370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 -- 172.21.15.52:0/4200878296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e53f10 msgr2=0x55db53e49370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e53f10 0x55db53e49370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 -- 172.21.15.52:0/4200878296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 msgr2=0x55db53e57880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e57880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 -- 172.21.15.52:0/4200878296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db53e67130 con 0x55db53e4e750 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e57880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.443+0000 7fbceeffd700 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e48e30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbce003c750 tx=0x7fbce000bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be31525837814a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcd4ff9700 1 -- 172.21.15.52:0/4200878296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0041020 con 0x55db53e4e750 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcd4ff9700 1 -- 172.21.15.52:0/4200878296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce003c3b0 con 0x55db53e4e750 2024-06-02T23:11:42.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcd4ff9700 1 -- 172.21.15.52:0/4200878296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0036070 con 0x55db53e4e750 2024-06-02T23:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 -- 172.21.15.52:0/4200878296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 msgr2=0x55db53e48e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e48e30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fbce003c750 tx=0x7fbce000bfd0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 -- 172.21.15.52:0/4200878296 shutdown_connections 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e54830 0x55db53e57880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e48e30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/4200878296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e53f10 0x55db53e49370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 -- 172.21.15.52:0/4200878296 >> 172.21.15.52:0/4200878296 conn(0x55db53d7b030 msgr2=0x55db53e51a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 -- 172.21.15.52:0/4200878296 shutdown_connections 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.444+0000 7fbcffc68b80 1 -- 172.21.15.52:0/4200878296 wait complete. 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 -- start start 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 Processor -- start 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e72730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e72c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e47fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55db53e68e70 con 0x55db53e4e750 2024-06-02T23:11:42.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55db53e687a0 con 0x55db53e53f10 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55db53e69540 con 0x55db53e54830 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcffc68b80 10 client.0 _ll_register_callbacks cb 0x55db53d35560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e72730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbceeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e72730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52676/0 (socket says 172.21.15.52:52676) 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbceeffd700 1 -- 172.21.15.52:0/2184253586 learned_addr learned my addr 172.21.15.52:0/2184253586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.445+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e72c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e47fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbceeffd700 1 -- 172.21.15.52:0/2184253586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 msgr2=0x55db53e47fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e47fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbceeffd700 1 -- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 msgr2=0x55db53e72c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e72c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbceeffd700 1 -- 172.21.15.52:0/2184253586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db53e48800 con 0x55db53e4e750 2024-06-02T23:11:42.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcef7fe700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e47fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbceeffd700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e72730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbce0005a40 tx=0x7fbce000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e37ce63f2bf086b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcd2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2184253586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce0036070 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2184253586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbce0004040 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbce003c3b0 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2184253586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fbce0041040 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcffc68b80 10 client.15450 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55db53e72000 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.446+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2184253586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbce003c3b0 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2184253586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fbce0004710 con 0x55db53e4e750 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcffc68b80 10 client.15450 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcffc68b80 10 client.15450 resolve_mds: resolved name 'h' to daemon mds.h 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcffc68b80 20 client.15450 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:42.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53ed76a0 0x55db53ed63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcffc68b80 4 client.15450 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "status"}) -- 0x55db53da7280 con 0x55db53ed76a0 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53ed76a0 0x55db53ed63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcee7fc700 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53ed76a0 0x55db53ed63b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9159527d390a2fc4 server_cookie=d351d30474a5b60d in_seq=0 out_seq=0 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.447+0000 7fbcd2ff5700 10 client.15450 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcd2ff5700 1 -- 172.21.15.52:0/2184253586 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55db53da7280 con 0x55db53ed76a0 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcd2ff5700 10 client.15450 handle_command_reply: tid=0 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 client.15450 shutdown 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 msgr2=0x55db53e72730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e72730 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fbce0005a40 tx=0x7fbce000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 shutdown_connections 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55db53ed76a0 0x55db53ed63b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55db53e54830 0x55db53e47fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55db53e4e750 0x55db53e72730 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcd2ff5700 0 client.15450 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 --2- 172.21.15.52:0/2184253586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55db53e53f10 0x55db53e72c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 >> 172.21.15.52:0/2184253586 conn(0x55db53d7b030 msgr2=0x55db53e576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 shutdown_connections 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 1 -- 172.21.15.52:0/2184253586 wait complete. 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcffc68b80 20 client.15450 trim_cache size 0 max 16384 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 2, 2024-06-02T23:11:42.453 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24449, 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 233.19417837399999, 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 28, 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 280.58346113699997 2024-06-02T23:11:42.454 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbcd80a2c20 msgr2=0x7fbcd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbcd80a2c20 0x7fbcd80a50e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbce0005e10 tx=0x7fbce0005d60 comp rx=0 tx=0).stop 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 msgr2=0x7fbcf0112400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.448+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 0x7fbcf0112400 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbce8007d70 tx=0x7fbce800a340 comp rx=0 tx=0).stop 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 shutdown_connections 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbcd80a2c20 0x7fbcd80a50e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcf0107400 0x7fbcf0112400 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcf0101390 0x7fbcf01139b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 --2- 172.21.15.52:0/2979627694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcf0101d50 0x7fbcf0111ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 >> 172.21.15.52:0/2979627694 conn(0x7fbcf0078340 msgr2=0x7fbcf00fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 shutdown_connections 2024-06-02T23:11:42.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:42.449+0000 7fbcf53e5700 1 -- 172.21.15.52:0/2979627694 wait complete. 2024-06-02T23:11:43.306 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.i status 2024-06-02T23:11:43.560 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:43 smithi174 ceph-mon[35994]: pgmap v426: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-06-02T23:11:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:43 smithi052 ceph-mon[32340]: pgmap v426: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-06-02T23:11:44.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:43 smithi123 ceph-mon[35977]: pgmap v426: 97 pgs: 97 active+clean; 658 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.6 KiB/s wr, 12 op/s 2024-06-02T23:11:45.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.350+0000 7f7e9dd25700 1 -- 172.21.15.52:0/2524503510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e98107870 msgr2=0x7f7e9810f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.350+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/2524503510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e98107870 0x7f7e9810f3f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f7e9000b0d0 tx=0x7f7e9000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.350+0000 7f7e9dd25700 1 -- 172.21.15.52:0/2524503510 shutdown_connections 2024-06-02T23:11:45.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.350+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/2524503510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e98107870 0x7f7e9810f3f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.350+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/2524503510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e981027a0 0x7f7e98102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.350+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/2524503510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98103150 0x7f7e98107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.351+0000 7f7e9dd25700 1 -- 172.21.15.52:0/2524503510 >> 172.21.15.52:0/2524503510 conn(0x7f7e98079570 msgr2=0x7f7e98079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:45.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.351+0000 7f7e9dd25700 1 -- 172.21.15.52:0/2524503510 shutdown_connections 2024-06-02T23:11:45.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.351+0000 7f7e9dd25700 1 -- 172.21.15.52:0/2524503510 wait complete. 2024-06-02T23:11:45.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.351+0000 7f7e9dd25700 1 Processor -- start 2024-06-02T23:11:45.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.351+0000 7f7e9dd25700 1 -- start start 2024-06-02T23:11:45.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 0x7f7e98074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e98103150 0x7f7e98074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98107870 0x7f7e98075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7e981126d0 con 0x7f7e981027a0 2024-06-02T23:11:45.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7e98112550 con 0x7f7e98107870 2024-06-02T23:11:45.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7e98112850 con 0x7f7e98103150 2024-06-02T23:11:45.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 0x7f7e98074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 0x7f7e98074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51554/0 (socket says 172.21.15.52:51554) 2024-06-02T23:11:45.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 -- 172.21.15.52:0/3697886217 learned_addr learned my addr 172.21.15.52:0/3697886217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:45.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e97fff700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98107870 0x7f7e98075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e98103150 0x7f7e98074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 -- 172.21.15.52:0/3697886217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e98103150 msgr2=0x7f7e98074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e98103150 0x7f7e98074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 -- 172.21.15.52:0/3697886217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98107870 msgr2=0x7f7e98075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98107870 0x7f7e98075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 -- 172.21.15.52:0/3697886217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e981a7030 con 0x7f7e981027a0 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e97fff700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98107870 0x7f7e98075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e977fe700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 0x7f7e98074790 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c970 tx=0x7f7e8800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e94ff9700 1 -- 172.21.15.52:0/3697886217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88007aa0 con 0x7f7e981027a0 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.352+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e981a7320 con 0x7f7e981027a0 2024-06-02T23:11:45.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.353+0000 7f7e94ff9700 1 -- 172.21.15.52:0/3697886217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e88004ce0 con 0x7f7e981027a0 2024-06-02T23:11:45.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.353+0000 7f7e94ff9700 1 -- 172.21.15.52:0/3697886217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e88005620 con 0x7f7e981027a0 2024-06-02T23:11:45.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.353+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981a7860 con 0x7f7e981027a0 2024-06-02T23:11:45.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.355+0000 7f7e94ff9700 1 -- 172.21.15.52:0/3697886217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7e88004750 con 0x7f7e981027a0 2024-06-02T23:11:45.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.355+0000 7f7e94ff9700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7e800a2c20 0x7f7e800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.356+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7e800a2c20 0x7f7e800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.356+0000 7f7e94ff9700 1 -- 172.21.15.52:0/3697886217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7e880cee80 con 0x7f7e981027a0 2024-06-02T23:11:45.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.356+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7e800a2c20 0x7f7e800a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006fd0 tx=0x7f7e8c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:45.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.356+0000 7f7ea85a8b80 1 Processor -- start 2024-06-02T23:11:45.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.356+0000 7f7ea85a8b80 1 -- start start 2024-06-02T23:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d9780639d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 0x55d97805d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d977faf7c0 con 0x55d978063f10 2024-06-02T23:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d977f576d0 con 0x55d9780635f0 2024-06-02T23:11:45.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d977fab780 con 0x55d97805ded0 2024-06-02T23:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 0x55d97805d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 0x55d97805d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51570/0 (socket says 172.21.15.52:51570) 2024-06-02T23:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 -- 172.21.15.52:0/511692573 learned_addr learned my addr 172.21.15.52:0/511692573 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e977fe700 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d9780639d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e97fff700 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 -- 172.21.15.52:0/511692573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 msgr2=0x55d97805e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 -- 172.21.15.52:0/511692573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 msgr2=0x55d9780639d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d9780639d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 -- 172.21.15.52:0/511692573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d977f4df00 con 0x55d978063f10 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e97fff700 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:45.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 0x55d97805d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d6a0 tx=0x7f7e8c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4cf959c5d92c956 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.357+0000 7f7e7dffb700 1 -- 172.21.15.52:0/511692573 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c024070 con 0x55d978063f10 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7e7dffb700 1 -- 172.21.15.52:0/511692573 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e8c00de40 con 0x55d978063f10 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7e7dffb700 1 -- 172.21.15.52:0/511692573 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c01fa90 con 0x55d978063f10 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/511692573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 msgr2=0x55d97805d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 0x55d97805d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00d6a0 tx=0x7f7e8c010000 comp rx=0 tx=0).stop 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/511692573 shutdown_connections 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d9780639d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/511692573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d978063f10 0x55d97805d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/511692573 >> 172.21.15.52:0/511692573 conn(0x55d977f88c70 msgr2=0x55d978058800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/511692573 shutdown_connections 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.358+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/511692573 wait complete. 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 -- start start 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 Processor -- start 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 0x55d978069cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d978066fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d978067500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d97806a980 con 0x55d97805ded0 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d977f4fba0 con 0x55d9780635f0 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d97806ab00 con 0x55d978063f10 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7ea85a8b80 10 client.0 _ll_register_callbacks cb 0x55d977f498d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7e977fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 0x55d978069cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7e977fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 0x55d978069cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51572/0 (socket says 172.21.15.52:51572) 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7e977fe700 1 -- 172.21.15.52:0/2046205375 learned_addr learned my addr 172.21.15.52:0/2046205375 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.359+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d978066fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e97fff700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d978067500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e977fe700 1 -- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 msgr2=0x55d978067500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e977fe700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d978067500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e977fe700 1 -- 172.21.15.52:0/2046205375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 msgr2=0x55d978066fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e977fe700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d978066fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e977fe700 1 -- 172.21.15.52:0/2046205375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d977efe880 con 0x55d97805ded0 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d978066fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:45.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e97fff700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d978067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e977fe700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 0x55d978069cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005950 tx=0x7f7e8809b100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbadd034b20237e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e76ffd700 1 -- 172.21.15.52:0/2046205375 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e880b3070 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e76ffd700 1 -- 172.21.15.52:0/2046205375 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7e88007cd0 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c00de40 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e76ffd700 1 -- 172.21.15.52:0/2046205375 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e8809bea0 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7ea85a8b80 10 client.15462 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d978075360 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e76ffd700 1 -- 172.21.15.52:0/2046205375 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e8c00de40 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.360+0000 7f7e76ffd700 1 -- 172.21.15.52:0/2046205375 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7e880a0ce0 con 0x55d97805ded0 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7ea85a8b80 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7ea85a8b80 10 client.15462 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7ea85a8b80 20 client.15462 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780779d0 0x55d978078e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7ea85a8b80 4 client.15462 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d977fbccc0 con 0x55d9780779d0 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.361+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780779d0 0x55d978078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7e76ffd700 10 client.15462 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780779d0 0x55d978078e40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5e65842c3bfc61ca server_cookie=d7f2c3bfa65b4380 in_seq=0 out_seq=0 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7e76ffd700 1 -- 172.21.15.52:0/2046205375 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d977fbccc0 con 0x55d9780779d0 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7e76ffd700 10 client.15462 handle_command_reply: tid=0 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7ea85a8b80 1 client.15462 shutdown 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 msgr2=0x55d978069cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 0x55d978069cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005950 tx=0x7f7e8809b100 comp rx=0 tx=0).stop 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.362+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 shutdown_connections 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780779d0 0x55d978078e40 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d978067500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d97805ded0 0x55d978069cc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/2046205375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d9780635f0 0x55d978066fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7e76ffd700 0 client.15462 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:11:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 >> 172.21.15.52:0/2046205375 conn(0x55d977f88c70 msgr2=0x55d97805c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 shutdown_connections 2024-06-02T23:11:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/2046205375 wait complete. 2024-06-02T23:11:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.363+0000 7f7ea85a8b80 20 client.15462 trim_cache size 0 max 16384 2024-06-02T23:11:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 Processor -- start 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 -- start start 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97805e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d978063f10 0x55d978064a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d978077180 con 0x55d9780635f0 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d978077850 con 0x55d978063f10 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.384+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d978077f20 con 0x55d97805ded0 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97805e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97805e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51578/0 (socket says 172.21.15.52:51578) 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1331608078 learned_addr learned my addr 172.21.15.52:0/1331608078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e977fe700 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e97fff700 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d978063f10 0x55d978064a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1331608078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 msgr2=0x55d97805d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1331608078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d978063f10 msgr2=0x55d978064a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d978063f10 0x55d978064a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1331608078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d978065530 con 0x55d9780635f0 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97805e2b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c01f700 tx=0x7f7e8c01f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a120537420da22 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e7d7fa700 1 -- 172.21.15.52:0/1331608078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c039070 con 0x55d9780635f0 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e7d7fa700 1 -- 172.21.15.52:0/1331608078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e8c01fce0 con 0x55d9780635f0 2024-06-02T23:11:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.385+0000 7f7e7d7fa700 1 -- 172.21.15.52:0/1331608078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c024040 con 0x55d9780635f0 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1331608078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 msgr2=0x55d97805e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97805e2b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c01f700 tx=0x7f7e8c01f690 comp rx=0 tx=0).stop 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1331608078 shutdown_connections 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d978063f10 0x55d978064a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d97805ded0 0x55d97805d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1331608078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97805e2b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1331608078 >> 172.21.15.52:0/1331608078 conn(0x55d977f88c70 msgr2=0x55d9780642f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1331608078 shutdown_connections 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1331608078 wait complete. 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.386+0000 7f7ea85a8b80 1 -- start start 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 Processor -- start 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d97805ded0 0x55d9780826e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97807caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d97807cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d978078cc0 con 0x55d9780635f0 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d9780785f0 con 0x55d97805ded0 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d978079390 con 0x55d978063f10 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7ea85a8b80 10 client.0 _ll_register_callbacks cb 0x55d977f498d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97807caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97807caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51592/0 (socket says 172.21.15.52:51592) 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1032121671 learned_addr learned my addr 172.21.15.52:0/1032121671 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e97fff700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d97807cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e977fe700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d97805ded0 0x55d9780826e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 msgr2=0x55d97807cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d97807cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1032121671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d97805ded0 msgr2=0x55d9780826e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d97805ded0 0x55d9780826e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.387+0000 7f7e96ffd700 1 -- 172.21.15.52:0/1032121671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d97807d550 con 0x55d9780635f0 2024-06-02T23:11:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e96ffd700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97807caa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c022040 tx=0x7f7e8c01f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e93ec7b28c70639 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e76ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e76ffd700 1 -- 172.21.15.52:0/1032121671 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c024030 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e76ffd700 1 -- 172.21.15.52:0/1032121671 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7e8c009600 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e8c01fce0 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e76ffd700 1 -- 172.21.15.52:0/1032121671 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7e8c039030 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7ea85a8b80 10 client.15468 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d9780e4000 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e76ffd700 1 -- 172.21.15.52:0/1032121671 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7e8c01fce0 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7e76ffd700 1 -- 172.21.15.52:0/1032121671 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7e8c013070 con 0x55d9780635f0 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7ea85a8b80 10 client.15468 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.388+0000 7f7ea85a8b80 10 client.15468 resolve_mds: resolved name 'i' to daemon mds.i 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7ea85a8b80 20 client.15468 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780e8530 0x55d9780e7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7ea85a8b80 4 client.15468 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "status"}) -- 0x55d97805e860 con 0x55d9780e8530 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7e977fe700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780e8530 0x55d9780e7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7e977fe700 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780e8530 0x55d9780e7240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=94b905f92b8c8912 server_cookie=9d4f417d876e93ae in_seq=0 out_seq=0 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7e76ffd700 10 client.15468 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7e76ffd700 1 -- 172.21.15.52:0/1032121671 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d97805e860 con 0x55d9780e8530 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7e76ffd700 10 client.15468 handle_command_reply: tid=0 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.389+0000 7f7ea85a8b80 1 client.15468 shutdown 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 msgr2=0x55d97807caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97807caa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c022040 tx=0x7f7e8c01f400 comp rx=0 tx=0).stop 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 shutdown_connections 2024-06-02T23:11:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55d9780e8530 0x55d9780e7240 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e76ffd700 0 client.15468 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d978063f10 0x55d97807cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d97805ded0 0x55d9780826e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 --2- 172.21.15.52:0/1032121671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d9780635f0 0x55d97807caa0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 >> 172.21.15.52:0/1032121671 conn(0x55d977f88c70 msgr2=0x55d978062120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 shutdown_connections 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 1 -- 172.21.15.52:0/1032121671 wait complete. 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7ea85a8b80 20 client.15468 trim_cache size 0 max 16384 2024-06-02T23:11:45.395 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 1, 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24451, 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 237.14372095600001, 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 28, 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 280.716273672 2024-06-02T23:11:45.396 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:45.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7e800a2c20 msgr2=0x7f7e800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7e800a2c20 0x7f7e800a50e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c006fd0 tx=0x7f7e8c008040 comp rx=0 tx=0).stop 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 msgr2=0x7f7e98074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 0x7f7e98074790 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f7e8800c970 tx=0x7f7e8800cd30 comp rx=0 tx=0).stop 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 shutdown_connections 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7e800a2c20 0x7f7e800a50e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7e98107870 0x7f7e98075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7e981027a0 0x7f7e98074790 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 --2- 172.21.15.52:0/3697886217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7e98103150 0x7f7e98074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.390+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 >> 172.21.15.52:0/3697886217 conn(0x7f7e98079570 msgr2=0x7f7e981089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.391+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 shutdown_connections 2024-06-02T23:11:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:45.391+0000 7f7e9dd25700 1 -- 172.21.15.52:0/3697886217 wait complete. 2024-06-02T23:11:45.739 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:11:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:45 smithi174 ceph-mon[35994]: pgmap v427: 97 pgs: 97 active+clean; 667 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-06-02T23:11:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:45 smithi052 ceph-mon[32340]: pgmap v427: 97 pgs: 97 active+clean; 667 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-06-02T23:11:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:45 smithi123 ceph-mon[35977]: pgmap v427: 97 pgs: 97 active+clean; 667 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.4 KiB/s wr, 12 op/s 2024-06-02T23:11:46.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.b status 2024-06-02T23:11:46.221 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/4191898967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0102a30 msgr2=0x7f6ce0102eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/4191898967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0102a30 0x7f6ce0102eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009900 tx=0x7f6cd002a410 comp rx=0 tx=0).stop 2024-06-02T23:11:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/4191898967 shutdown_connections 2024-06-02T23:11:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/4191898967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ce01033f0 0x7f6ce0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/4191898967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0102a30 0x7f6ce0102eb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/4191898967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce0101840 0x7f6ce0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/4191898967 >> 172.21.15.52:0/4191898967 conn(0x7f6ce00fcdb0 msgr2=0x7f6ce00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/4191898967 shutdown_connections 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.720+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/4191898967 wait complete. 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 Processor -- start 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 -- start start 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 0x7f6ce0197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ce0102a30 0x7f6ce0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce01033f0 0x7f6ce019eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010d3d0 con 0x7f6ce0101840 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010d250 con 0x7f6ce01033f0 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce6d9e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce010d550 con 0x7f6ce0102a30 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce5d9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 0x7f6ce0197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce5d9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 0x7f6ce0197a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51612/0 (socket says 172.21.15.52:51612) 2024-06-02T23:11:46.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/61089386 learned_addr learned my addr 172.21.15.52:0/61089386 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce559b640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ce0102a30 0x7f6ce0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.721+0000 7f6ce659d640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce01033f0 0x7f6ce019eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/61089386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ce0102a30 msgr2=0x7f6ce0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ce0102a30 0x7f6ce0197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/61089386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce01033f0 msgr2=0x7f6ce019eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce01033f0 0x7f6ce019eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/61089386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ce019f660 con 0x7f6ce0101840 2024-06-02T23:11:46.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce659d640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce01033f0 0x7f6ce019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 0x7f6ce0197a40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c970 tx=0x7f6cdc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6cd6ffd640 1 -- 172.21.15.52:0/61089386 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc007c10 con 0x7f6ce0101840 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ce019f950 con 0x7f6ce0101840 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6cd6ffd640 1 -- 172.21.15.52:0/61089386 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cdc007db0 con 0x7f6ce0101840 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6cd6ffd640 1 -- 172.21.15.52:0/61089386 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc00f870 con 0x7f6ce0101840 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.722+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ce019fef0 con 0x7f6ce0101840 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.724+0000 7f6cd6ffd640 1 -- 172.21.15.52:0/61089386 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cdc00fa10 con 0x7f6ce0101840 2024-06-02T23:11:46.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 Processor -- start 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 -- start start 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51c200 0x55ad7b51c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5216f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 0x55ad7b5241f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad7b47f4f0 con 0x55ad7b521d60 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad7b41d870 con 0x55ad7b51c200 2024-06-02T23:11:46.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ceadec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad7b47b4b0 con 0x55ad7b51cbf0 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6cd6ffd640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ccc0a2b80 0x7f6ccc0a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ce559b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 0x55ad7b5241f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ce559b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 0x55ad7b5241f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51618/0 (socket says 172.21.15.52:51618) 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ce559b640 1 -- 172.21.15.52:0/2766028157 learned_addr learned my addr 172.21.15.52:0/2766028157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ce659d640 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51c200 0x55ad7b51c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5216f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6ce659d640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ccc0a2b80 0x7f6ccc0a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.725+0000 7f6cd6ffd640 1 -- 172.21.15.52:0/61089386 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cdc0cf410 con 0x7f6ce0101840 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce559b640 1 -- 172.21.15.52:0/2766028157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 msgr2=0x55ad7b5216f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce559b640 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5216f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce559b640 1 -- 172.21.15.52:0/2766028157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51c200 msgr2=0x55ad7b51c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce559b640 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51c200 0x55ad7b51c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce559b640 1 -- 172.21.15.52:0/2766028157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad7b4399b0 con 0x55ad7b521d60 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce559b640 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 0x55ad7b5241f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009900 tx=0x7f6cd0032e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eee3e9e3ae94cd70 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:46.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ce659d640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ccc0a2b80 0x7f6ccc0a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55ad7b51d450 tx=0x7f6cd80093d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ccbfff640 1 -- 172.21.15.52:0/2766028157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0035070 con 0x55ad7b521d60 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ccbfff640 1 -- 172.21.15.52:0/2766028157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cd0031780 con 0x55ad7b521d60 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.726+0000 7f6ccbfff640 1 -- 172.21.15.52:0/2766028157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0045470 con 0x55ad7b521d60 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 -- 172.21.15.52:0/2766028157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 msgr2=0x55ad7b5241f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 0x55ad7b5241f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6cd0009900 tx=0x7f6cd0032e10 comp rx=0 tx=0).stop 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 -- 172.21.15.52:0/2766028157 shutdown_connections 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5216f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b521d60 0x55ad7b5241f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2766028157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51c200 0x55ad7b51c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 -- 172.21.15.52:0/2766028157 >> 172.21.15.52:0/2766028157 conn(0x55ad7b517870 msgr2=0x55ad7b517c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 -- 172.21.15.52:0/2766028157 shutdown_connections 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 -- 172.21.15.52:0/2766028157 wait complete. 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 -- start start 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.727+0000 7f6ceadec740 1 Processor -- start 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51c200 0x55ad7b526480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5269c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad7b524cb0 con 0x55ad7b51cbf0 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad7b408ae0 con 0x55ad7b521d60 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad7b524e30 con 0x55ad7b51c200 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 10 client.0 _ll_register_callbacks cb 0x55ad7b4419b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51634/0 (socket says 172.21.15.52:51634) 2024-06-02T23:11:46.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 -- 172.21.15.52:0/2608045298 learned_addr learned my addr 172.21.15.52:0/2608045298 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51c200 0x55ad7b526480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce659d640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 -- 172.21.15.52:0/2608045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51c200 msgr2=0x55ad7b526480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51c200 0x55ad7b526480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 -- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 msgr2=0x55ad7b52d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 -- 172.21.15.52:0/2608045298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad7b193210 con 0x55ad7b51cbf0 2024-06-02T23:11:46.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51c200 0x55ad7b526480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce659d640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ce559b640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5269c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00319b0 tx=0x7f6cd0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c2fe94aaa5515c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6cc97fa640 1 -- 172.21.15.52:0/2608045298 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cd0035070 con 0x55ad7b51cbf0 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6cc97fa640 1 -- 172.21.15.52:0/2608045298 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cd0003c90 con 0x55ad7b51cbf0 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.728+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cd0031780 con 0x55ad7b51cbf0 2024-06-02T23:11:46.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6cc97fa640 1 -- 172.21.15.52:0/2608045298 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6cd004a070 con 0x55ad7b51cbf0 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad7b25f620 con 0x55ad7b51cbf0 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6cc97fa640 1 -- 172.21.15.52:0/2608045298 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cd0031780 con 0x55ad7b51cbf0 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6cc97fa640 1 -- 172.21.15.52:0/2608045298 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6cd005a8c0 con 0x55ad7b51cbf0 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 10 client.15480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 20 client.15480 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b531e10 0x55ad7b534200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 4 client.15480 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad7b492f10 con 0x55ad7b531e10 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.729+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b531e10 0x55ad7b534200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.730+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b531e10 0x55ad7b534200 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3816c9b3dfcfb0d server_cookie=8de89afb8c8ddc4e in_seq=0 out_seq=0 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.730+0000 7f6cc97fa640 10 client.15480 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6cc97fa640 1 -- 172.21.15.52:0/2608045298 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad7b492f10 con 0x55ad7b531e10 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6cc97fa640 10 client.15480 handle_command_reply: tid=0 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 client.15480 shutdown 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 msgr2=0x55ad7b5269c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5269c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f6cd00319b0 tx=0x7f6cd0000bc0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 shutdown_connections 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b531e10 0x55ad7b534200 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51cbf0 0x55ad7b5269c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 --2- 172.21.15.52:0/2608045298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51c200 0x55ad7b526480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6cc97fa640 0 client.15480 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 >> 172.21.15.52:0/2608045298 conn(0x55ad7b517870 msgr2=0x55ad7b5169d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 shutdown_connections 2024-06-02T23:11:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 1 -- 172.21.15.52:0/2608045298 wait complete. 2024-06-02T23:11:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.731+0000 7f6ceadec740 20 client.15480 trim_cache size 0 max 16384 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 Processor -- start 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 -- start start 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b51a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b521d60 0x55ad7b51aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce01a8b10 con 0x55ad7b51c200 2024-06-02T23:11:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce01aa650 con 0x55ad7b51cbf0 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.746+0000 7f6ceadec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce01a91e0 con 0x55ad7b521d60 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b51a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b51a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51650/0 (socket says 172.21.15.52:51650) 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/4000667362 learned_addr learned my addr 172.21.15.52:0/4000667362 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce659d640 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b521d60 0x55ad7b51aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/4000667362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b521d60 msgr2=0x55ad7b51aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b521d60 0x55ad7b51aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/4000667362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51cbf0 msgr2=0x55ad7b52d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce559b640 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/4000667362 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad7b534570 con 0x55ad7b51c200 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce659d640 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b521d60 0x55ad7b51aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b51a550 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00a660 tx=0x7f6cdc012da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8202c4487f40275c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ccb7fe640 1 -- 172.21.15.52:0/4000667362 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc09bcd0 con 0x55ad7b51c200 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ccb7fe640 1 -- 172.21.15.52:0/4000667362 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cdc0a0080 con 0x55ad7b51c200 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.747+0000 7f6ccb7fe640 1 -- 172.21.15.52:0/4000667362 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc099fd0 con 0x55ad7b51c200 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 -- 172.21.15.52:0/4000667362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 msgr2=0x55ad7b51a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b51a550 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00a660 tx=0x7f6cdc012da0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 -- 172.21.15.52:0/4000667362 shutdown_connections 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b521d60 0x55ad7b51aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 --2- 172.21.15.52:0/4000667362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b51a550 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 -- 172.21.15.52:0/4000667362 >> 172.21.15.52:0/4000667362 conn(0x55ad7b517870 msgr2=0x55ad7b517c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 -- 172.21.15.52:0/4000667362 shutdown_connections 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 -- 172.21.15.52:0/4000667362 wait complete. 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 -- start start 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.748+0000 7f6ceadec740 1 Processor -- start 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b52b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce01a9f80 con 0x55ad7b51c200 2024-06-02T23:11:46.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce01a98b0 con 0x55ad7b521d60 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ce01a6b10 con 0x55ad7b51cbf0 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 10 client.0 _ll_register_callbacks cb 0x7f6cd8009ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b52b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b52b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51660/0 (socket says 172.21.15.52:51660) 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/3282214187 learned_addr learned my addr 172.21.15.52:0/3282214187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/3282214187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 msgr2=0x55ad7b52b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 msgr2=0x55ad7b52c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce659d640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 -- 172.21.15.52:0/3282214187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad7b52cd30 con 0x55ad7b51c200 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce659d640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ce5d9c640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b52b290 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09b9d0 tx=0x7f6cdc099e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3391a4a86f5c8a67 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6cc97fa640 1 -- 172.21.15.52:0/3282214187 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc0a48b0 con 0x55ad7b51c200 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6cc97fa640 1 -- 172.21.15.52:0/3282214187 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cdc00fd00 con 0x55ad7b51c200 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cdc0a0080 con 0x55ad7b51c200 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6cc97fa640 1 -- 172.21.15.52:0/3282214187 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6cdc098340 con 0x55ad7b51c200 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 10 client.15486 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:46.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.749+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55ad7b59cb80 con 0x55ad7b51c200 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6cc97fa640 1 -- 172.21.15.52:0/3282214187 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cdc0a0080 con 0x55ad7b51c200 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6cc97fa640 1 -- 172.21.15.52:0/3282214187 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6cdc07cc00 con 0x55ad7b51c200 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ceadec740 10 client.15486 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ceadec740 10 client.15486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ceadec740 20 client.15486 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ceadec740 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b59d4b0 0x55ad7b59f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ceadec740 4 client.15486 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad7b48cc20 con 0x55ad7b59d4b0 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.750+0000 7f6ce559b640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b59d4b0 0x55ad7b59f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.751+0000 7f6cc97fa640 10 client.15486 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:46.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.751+0000 7f6ce559b640 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b59d4b0 0x55ad7b59f8d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653dbf54dcc33c04 server_cookie=a9d1cc907255cea7 in_seq=0 out_seq=0 2024-06-02T23:11:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.757+0000 7f6cc97fa640 1 -- 172.21.15.52:0/3282214187 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ad7b48cc20 con 0x55ad7b59d4b0 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.757+0000 7f6cc97fa640 10 client.15486 handle_command_reply: tid=0 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.757+0000 7f6ceadec740 1 client.15486 shutdown 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 msgr2=0x55ad7b52b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b52b290 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc09b9d0 tx=0x7f6cdc099e40 comp rx=0 tx=0).stop 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 shutdown_connections 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ad7b59d4b0 0x55ad7b59f8d0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad7b51cbf0 0x55ad7b52b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6cc97fa640 0 client.15486 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad7b521d60 0x55ad7b52c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 --2- 172.21.15.52:0/3282214187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad7b51c200 0x55ad7b52b290 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 >> 172.21.15.52:0/3282214187 conn(0x55ad7b517870 msgr2=0x55ad7b51a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 shutdown_connections 2024-06-02T23:11:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 1 -- 172.21.15.52:0/3282214187 wait complete. 2024-06-02T23:11:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ceadec740 20 client.15486 trim_cache size 0 max 16384 2024-06-02T23:11:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:11:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:11:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ccc0a2b80 msgr2=0x7f6ccc0a5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ccc0a2b80 0x7f6ccc0a5040 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x55ad7b51d450 tx=0x7f6cd80093d0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 msgr2=0x7f6ce0197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 0x7f6ce0197a40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc00c970 tx=0x7f6cdc00ce30 comp rx=0 tx=0).stop 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 shutdown_connections 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ccc0a2b80 0x7f6ccc0a5040 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ce0102a30 0x7f6ce0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce01033f0 0x7f6ce019eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 --2- 172.21.15.52:0/61089386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ce0101840 0x7f6ce0197a40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.758+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 >> 172.21.15.52:0/61089386 conn(0x7f6ce00fcdb0 msgr2=0x7f6ce0105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.759+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 shutdown_connections 2024-06-02T23:11:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:46.759+0000 7f6ce6d9e640 1 -- 172.21.15.52:0/61089386 wait complete. 2024-06-02T23:11:46.966 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:47.734 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:47 smithi052 ceph-mon[32340]: pgmap v428: 97 pgs: 97 active+clean; 667 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-02T23:11:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:47 smithi174 ceph-mon[35994]: pgmap v428: 97 pgs: 97 active+clean; 667 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-02T23:11:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:47 smithi123 ceph-mon[35977]: pgmap v428: 97 pgs: 97 active+clean; 667 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-06-02T23:11:48.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 -- 172.21.15.52:0/666221263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1030f0 msgr2=0x7feb3c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 --2- 172.21.15.52:0/666221263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1030f0 0x7feb3c107110 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7feb300099c0 tx=0x7feb30009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 -- 172.21.15.52:0/666221263 shutdown_connections 2024-06-02T23:11:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 --2- 172.21.15.52:0/666221263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c107840 0x7feb3c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 --2- 172.21.15.52:0/666221263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c102740 0x7feb3c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 --2- 172.21.15.52:0/666221263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c1030f0 0x7feb3c107110 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 -- 172.21.15.52:0/666221263 >> 172.21.15.52:0/666221263 conn(0x7feb3c0795f0 msgr2=0x7feb3c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 -- 172.21.15.52:0/666221263 shutdown_connections 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.154+0000 7feb4118d700 1 -- 172.21.15.52:0/666221263 wait complete. 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 Processor -- start 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 -- start start 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 0x7feb3c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c1030f0 0x7feb3c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c107840 0x7feb3c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c112600 con 0x7feb3c102740 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c112480 con 0x7feb3c107840 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.155+0000 7feb4118d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c112780 con 0x7feb3c1030f0 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 0x7feb3c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 0x7feb3c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51678/0 (socket says 172.21.15.52:51678) 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 -- 172.21.15.52:0/701564983 learned_addr learned my addr 172.21.15.52:0/701564983 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3a59c700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c1030f0 0x7feb3c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3b59e700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c107840 0x7feb3c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 -- 172.21.15.52:0/701564983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c1030f0 msgr2=0x7feb3c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c1030f0 0x7feb3c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 -- 172.21.15.52:0/701564983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c107840 msgr2=0x7feb3c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c107840 0x7feb3c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 -- 172.21.15.52:0/701564983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb3c199060 con 0x7feb3c102740 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3b59e700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c107840 0x7feb3c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3a59c700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c1030f0 0x7feb3c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 0x7feb3c19e740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0060b0 tx=0x7feb2c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb23fff700 1 -- 172.21.15.52:0/701564983 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c01a070 con 0x7feb3c102740 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb23fff700 1 -- 172.21.15.52:0/701564983 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c00de20 con 0x7feb3c102740 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb3c199350 con 0x7feb3c102740 2024-06-02T23:11:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb23fff700 1 -- 172.21.15.52:0/701564983 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c014720 con 0x7feb3c102740 2024-06-02T23:11:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.156+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb3c1a5610 con 0x7feb3c102740 2024-06-02T23:11:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.158+0000 7feb23fff700 1 -- 172.21.15.52:0/701564983 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb2c01d070 con 0x7feb3c102740 2024-06-02T23:11:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.159+0000 7feb23fff700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb240a2c20 0x7feb240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.159+0000 7feb3a59c700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb240a2c20 0x7feb240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.159+0000 7feb23fff700 1 -- 172.21.15.52:0/701564983 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb2c0ce920 con 0x7feb3c102740 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb3a59c700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb240a2c20 0x7feb240a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb30006010 tx=0x7feb30009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 Processor -- start 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 -- start start 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd24e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd248af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd249030 0x55c1bd255210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd136f60 con 0x55c1bd24e470 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd195850 con 0x55c1bd248670 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.160+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd0fd460 con 0x55c1bd249030 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd24e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd24e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51680/0 (socket says 172.21.15.52:51680) 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 -- 172.21.15.52:0/49142152 learned_addr learned my addr 172.21.15.52:0/49142152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3a59c700 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd248af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 -- 172.21.15.52:0/49142152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd249030 msgr2=0x55c1bd255210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd249030 0x55c1bd255210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 -- 172.21.15.52:0/49142152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 msgr2=0x55c1bd248af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd248af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 -- 172.21.15.52:0/49142152 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bd0e48d0 con 0x55c1bd24e470 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd24e850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feb2c09b140 tx=0x7feb2c09ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6d2227ae0a2fb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb217fa700 1 -- 172.21.15.52:0/49142152 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0a3e20 con 0x55c1bd24e470 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb217fa700 1 -- 172.21.15.52:0/49142152 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c099250 con 0x55c1bd24e470 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.161+0000 7feb217fa700 1 -- 172.21.15.52:0/49142152 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2c09f120 con 0x55c1bd24e470 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 -- 172.21.15.52:0/49142152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 msgr2=0x55c1bd24e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd24e850 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7feb2c09b140 tx=0x7feb2c09ae20 comp rx=0 tx=0).stop 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 -- 172.21.15.52:0/49142152 shutdown_connections 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd249030 0x55c1bd255210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd24e850 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/49142152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd248af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 -- 172.21.15.52:0/49142152 >> 172.21.15.52:0/49142152 conn(0x55c1bd171080 msgr2=0x55c1bd23f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 -- 172.21.15.52:0/49142152 shutdown_connections 2024-06-02T23:11:48.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 -- 172.21.15.52:0/49142152 wait complete. 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.162+0000 7feb4ba10b80 1 -- start start 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 Processor -- start 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd254e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd2521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd24e470 0x55c1bd252720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd135fc0 con 0x55c1bd249030 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd191810 con 0x55c1bd24e470 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1bd255750 con 0x55c1bd248670 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb4ba10b80 10 client.0 _ll_register_callbacks cb 0x55c1bd12b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd2521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd2521e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51684/0 (socket says 172.21.15.52:51684) 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 -- 172.21.15.52:0/180617670 learned_addr learned my addr 172.21.15.52:0/180617670 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3b59e700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd24e470 0x55c1bd252720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd254e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 -- 172.21.15.52:0/180617670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 msgr2=0x55c1bd254e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd254e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 -- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd24e470 msgr2=0x55c1bd252720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd24e470 0x55c1bd252720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3a59c700 1 -- 172.21.15.52:0/180617670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bd15b750 con 0x55c1bd249030 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.163+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd254e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb3a59c700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd2521e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb30009990 tx=0x7feb30009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c22a970dec534 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:48.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb1eff5700 1 -- 172.21.15.52:0/180617670 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30003ca0 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb1eff5700 1 -- 172.21.15.52:0/180617670 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb3000f840 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb2c099250 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb1eff5700 1 -- 172.21.15.52:0/180617670 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb30037050 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 10 client.15498 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1bd25fdb0 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb1eff5700 1 -- 172.21.15.52:0/180617670 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb2c099250 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb1eff5700 1 -- 172.21.15.52:0/180617670 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feb30023410 con 0x55c1bd249030 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 20 client.15498 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:48.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd262340 0x55c1bd264750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 4 client.15498 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.164+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1bd1a2d50 con 0x55c1bd262340 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.165+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd262340 0x55c1bd264750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.165+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd262340 0x55c1bd264750 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a8966dc5b5be44b server_cookie=16c2b89438bdb16d in_seq=0 out_seq=0 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.165+0000 7feb1eff5700 10 client.15498 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb1eff5700 1 -- 172.21.15.52:0/180617670 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c1bd1a2d50 con 0x55c1bd262340 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb1eff5700 10 client.15498 handle_command_reply: tid=0 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 client.15498 shutdown 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 msgr2=0x55c1bd2521e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd2521e0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7feb30009990 tx=0x7feb30009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 shutdown_connections 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd262340 0x55c1bd264750 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd24e470 0x55c1bd252720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd254e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/180617670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd2521e0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 >> 172.21.15.52:0/180617670 conn(0x55c1bd171080 msgr2=0x55c1bd23e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb1eff5700 0 client.15498 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 shutdown_connections 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 1 -- 172.21.15.52:0/180617670 wait complete. 2024-06-02T23:11:48.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.166+0000 7feb4ba10b80 20 client.15498 trim_cache size 0 max 16384 2024-06-02T23:11:48.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 Processor -- start 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 -- start start 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd252810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd24c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd24e470 0x55c1bd23ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1aa850 con 0x55c1bd249030 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ac390 con 0x55c1bd248670 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1aaf20 con 0x55c1bd24e470 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd24c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd24c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51692/0 (socket says 172.21.15.52:51692) 2024-06-02T23:11:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.187+0000 7feb3a59c700 1 -- 172.21.15.52:0/1602131166 learned_addr learned my addr 172.21.15.52:0/1602131166 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd252810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3a59c700 1 -- 172.21.15.52:0/1602131166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd24e470 msgr2=0x55c1bd23ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3a59c700 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd24e470 0x55c1bd23ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3a59c700 1 -- 172.21.15.52:0/1602131166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 msgr2=0x55c1bd252810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3a59c700 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd252810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3a59c700 1 -- 172.21.15.52:0/1602131166 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1bd240960 con 0x55c1bd249030 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb3a59c700 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd24c570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb30023860 tx=0x7feb300235a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a242c47911d9699b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb20ff9700 1 -- 172.21.15.52:0/1602131166 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb30023a50 con 0x55c1bd249030 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb20ff9700 1 -- 172.21.15.52:0/1602131166 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30027020 con 0x55c1bd249030 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.188+0000 7feb20ff9700 1 -- 172.21.15.52:0/1602131166 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3000f740 con 0x55c1bd249030 2024-06-02T23:11:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1602131166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 msgr2=0x55c1bd24c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd24c570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7feb30023860 tx=0x7feb300235a0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1602131166 shutdown_connections 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd24e470 0x55c1bd23ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd248670 0x55c1bd252810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1602131166 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd249030 0x55c1bd24c570 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1602131166 >> 172.21.15.52:0/1602131166 conn(0x55c1bd171080 msgr2=0x55c1bd24be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1602131166 shutdown_connections 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1602131166 wait complete. 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- start start 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 Processor -- start 2024-06-02T23:11:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd269dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd249030 0x55c1bd26a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd26a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1abcc0 con 0x55c1bd24e470 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1ab5f0 con 0x55c1bd249030 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.189+0000 7feb4ba10b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb3c1a8830 con 0x55c1bd248670 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb4ba10b80 10 client.0 _ll_register_callbacks cb 0x55c1bd12b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd26a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd26a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51694/0 (socket says 172.21.15.52:51694) 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 -- 172.21.15.52:0/1325680963 learned_addr learned my addr 172.21.15.52:0/1325680963 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd269dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3a59c700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd249030 0x55c1bd26a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 -- 172.21.15.52:0/1325680963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 msgr2=0x55c1bd269dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd269dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 -- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd249030 msgr2=0x55c1bd26a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd249030 0x55c1bd26a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 -- 172.21.15.52:0/1325680963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1bd236ff0 con 0x55c1bd24e470 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb3b59e700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd26a850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feb34009cd0 tx=0x7feb340113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1625edeecfb86c75 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb1eff5700 1 -- 172.21.15.52:0/1325680963 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb34011df0 con 0x55c1bd24e470 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb30027020 con 0x55c1bd24e470 2024-06-02T23:11:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.190+0000 7feb1eff5700 1 -- 172.21.15.52:0/1325680963 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb34012450 con 0x55c1bd24e470 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb1eff5700 1 -- 172.21.15.52:0/1325680963 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7feb34012650 con 0x55c1bd24e470 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 10 client.15504 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55c1bd2cd290 con 0x55c1bd24e470 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb1eff5700 1 -- 172.21.15.52:0/1325680963 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb30027020 con 0x55c1bd24e470 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb1eff5700 1 -- 172.21.15.52:0/1325680963 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7feb3400f040 con 0x55c1bd24e470 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 10 client.15504 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 10 client.15504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 20 client.15504 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd2cd940 0x55c1bd2cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 4 client.15504 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:11:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c1bd19d2d0 con 0x55c1bd2cd940 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.191+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd2cd940 0x55c1bd2cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb3ad9d700 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd2cd940 0x55c1bd2cfd30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f941400a70c1666 server_cookie=c2c845d6c9a44bca in_seq=0 out_seq=0 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb1eff5700 10 client.15504 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb1eff5700 1 -- 172.21.15.52:0/1325680963 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c1bd19d2d0 con 0x55c1bd2cd940 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb1eff5700 10 client.15504 handle_command_reply: tid=0 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 client.15504 shutdown 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 msgr2=0x55c1bd26a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd26a850 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7feb34009cd0 tx=0x7feb340113e0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 shutdown_connections 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1bd2cd940 0x55c1bd2cfd30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1bd24e470 0x55c1bd26a850 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1bd248670 0x55c1bd269dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 --2- 172.21.15.52:0/1325680963 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1bd249030 0x55c1bd26a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 >> 172.21.15.52:0/1325680963 conn(0x55c1bd171080 msgr2=0x55c1bd24c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb1eff5700 0 client.15504 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 shutdown_connections 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 1 -- 172.21.15.52:0/1325680963 wait complete. 2024-06-02T23:11:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.192+0000 7feb4ba10b80 20 client.15504 trim_cache size 0 max 16384 2024-06-02T23:11:48.200 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:48.200 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:11:48.200 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:11:48.200 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb240a2c20 msgr2=0x7feb240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb240a2c20 0x7feb240a50e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7feb30006010 tx=0x7feb30009f10 comp rx=0 tx=0).stop 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 msgr2=0x7feb3c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 0x7feb3c19e740 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0060b0 tx=0x7feb2c007f90 comp rx=0 tx=0).stop 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 shutdown_connections 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb240a2c20 0x7feb240a50e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb3c107840 0x7feb3c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb3c102740 0x7feb3c19e740 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 --2- 172.21.15.52:0/701564983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb3c1030f0 0x7feb3c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:48.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 >> 172.21.15.52:0/701564983 conn(0x7feb3c0795f0 msgr2=0x7feb3c1007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 shutdown_connections 2024-06-02T23:11:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:48.193+0000 7feb4118d700 1 -- 172.21.15.52:0/701564983 wait complete. 2024-06-02T23:11:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:49 smithi174 ceph-mon[35994]: pgmap v429: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-06-02T23:11:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:49 smithi052 ceph-mon[32340]: pgmap v429: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-06-02T23:11:49.941 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:11:49.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:11:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:49 smithi123 ceph-mon[35977]: pgmap v429: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-06-02T23:11:50.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 -- 172.21.15.52:0/2418258403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 msgr2=0x7f7af41019b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2418258403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af41019b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7ae40099c0 tx=0x7f7ae4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 -- 172.21.15.52:0/2418258403 shutdown_connections 2024-06-02T23:11:50.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2418258403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 0x7f7af410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2418258403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af41019b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2418258403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af4101f80 0x7f7af4105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 -- 172.21.15.52:0/2418258403 >> 172.21.15.52:0/2418258403 conn(0x7f7af4078400 msgr2=0x7f7af4078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:50.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 -- 172.21.15.52:0/2418258403 shutdown_connections 2024-06-02T23:11:50.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.431+0000 7f7afaa51700 1 -- 172.21.15.52:0/2418258403 wait complete. 2024-06-02T23:11:50.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.432+0000 7f7afaa51700 1 Processor -- start 2024-06-02T23:11:50.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 -- start start 2024-06-02T23:11:50.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af4101f80 0x7f7af419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 0x7f7af4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7af4111570 con 0x7f7af41015d0 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41113f0 con 0x7f7af4101f80 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7afaa51700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7af41116f0 con 0x7f7af41066b0 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7af3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51712/0 (socket says 172.21.15.52:51712) 2024-06-02T23:11:50.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7af3fff700 1 -- 172.21.15.52:0/2563254286 learned_addr learned my addr 172.21.15.52:0/2563254286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:50.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7af37fe700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af4101f80 0x7f7af419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.433+0000 7f7af8fee700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 0x7f7af4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af3fff700 1 -- 172.21.15.52:0/2563254286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 msgr2=0x7f7af4197870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af3fff700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 0x7f7af4197870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af3fff700 1 -- 172.21.15.52:0/2563254286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af4101f80 msgr2=0x7f7af419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af3fff700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af4101f80 0x7f7af419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af3fff700 1 -- 172.21.15.52:0/2563254286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7af4197fd0 con 0x7f7af41015d0 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af8fee700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 0x7f7af4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af3fff700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af419d6b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006010 tx=0x7f7ae4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af17fa700 1 -- 172.21.15.52:0/2563254286 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4027070 con 0x7f7af41015d0 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7af4198260 con 0x7f7af41015d0 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7af41a4580 con 0x7f7af41015d0 2024-06-02T23:11:50.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af17fa700 1 -- 172.21.15.52:0/2563254286 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ae4003f50 con 0x7f7af41015d0 2024-06-02T23:11:50.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.434+0000 7f7af17fa700 1 -- 172.21.15.52:0/2563254286 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4022cd0 con 0x7f7af41015d0 2024-06-02T23:11:50.440 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:50.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.436+0000 7f7af17fa700 1 -- 172.21.15.52:0/2563254286 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7ae402a070 con 0x7f7af41015d0 2024-06-02T23:11:50.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.437+0000 7f7af17fa700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7adc0a2ca0 0x7f7adc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7af37fe700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7adc0a2ca0 0x7f7adc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7af17fa700 1 -- 172.21.15.52:0/2563254286 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7ae40cfc20 con 0x7f7af41015d0 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 Processor -- start 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 -- start start 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7af37fe700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7adc0a2ca0 0x7f7adc0a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7af4106130 tx=0x7f7ae800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5575503bcf10 con 0x5575504d2a40 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55755041b800 con 0x5575504cc960 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557550383410 con 0x5575504d2120 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.438+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cc420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51728/0 (socket says 172.21.15.52:51728) 2024-06-02T23:11:50.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 -- 172.21.15.52:0/22120279 learned_addr learned my addr 172.21.15.52:0/22120279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af8fee700 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af3fff700 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 -- 172.21.15.52:0/22120279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 msgr2=0x5575504d2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504d2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 -- 172.21.15.52:0/22120279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 msgr2=0x5575504ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ccd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 -- 172.21.15.52:0/22120279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55755036a880 con 0x5575504d2a40 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af3fff700 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7af37fe700 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cc420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8010f40 tx=0x7f7ae8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d03960d38d2a7d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7ada7fc700 1 -- 172.21.15.52:0/22120279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae800be60 con 0x5575504d2a40 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7ada7fc700 1 -- 172.21.15.52:0/22120279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ae8011db0 con 0x5575504d2a40 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.439+0000 7f7ada7fc700 1 -- 172.21.15.52:0/22120279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801c6e0 con 0x5575504d2a40 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 -- 172.21.15.52:0/22120279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 msgr2=0x5575504cc420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cc420 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8010f40 tx=0x7f7ae8013000 comp rx=0 tx=0).stop 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 -- 172.21.15.52:0/22120279 shutdown_connections 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504d2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/22120279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cc420 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 -- 172.21.15.52:0/22120279 >> 172.21.15.52:0/22120279 conn(0x5575503f7030 msgr2=0x5575504c51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 -- 172.21.15.52:0/22120279 shutdown_connections 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.440+0000 7f7b052d4b80 1 -- 172.21.15.52:0/22120279 wait complete. 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.441+0000 7f7b052d4b80 1 -- start start 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.441+0000 7f7b052d4b80 1 Processor -- start 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.441+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.441+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504de520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5575503bbf70 con 0x5575504d2a40 2024-06-02T23:11:50.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5575504177c0 con 0x5575504cc960 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575504d9420 con 0x5575504d2120 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51730/0 (socket says 172.21.15.52:51730) 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 -- 172.21.15.52:0/4262295068 learned_addr learned my addr 172.21.15.52:0/4262295068 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af3fff700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af37fe700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504de520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7b052d4b80 10 client.0 _ll_register_callbacks cb 0x5575503b1560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 -- 172.21.15.52:0/4262295068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 msgr2=0x5575504de520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504de520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 -- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 msgr2=0x5575504ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af8fee700 1 -- 172.21.15.52:0/4262295068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5575503e1700 con 0x5575504d2a40 2024-06-02T23:11:50.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.442+0000 7f7af37fe700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504de520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7af8fee700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cb540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007b20 tx=0x7f7aec007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87b7b7623e2703e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7ad37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7ad37fe700 1 -- 172.21.15.52:0/4262295068 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec01a070 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7ad37fe700 1 -- 172.21.15.52:0/4262295068 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7aec012ba0 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae8011db0 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7ad37fe700 1 -- 172.21.15.52:0/4262295068 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7aec026050 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7b052d4b80 10 client.15516 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5575504e5db0 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7ad37fe700 1 -- 172.21.15.52:0/4262295068 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ae8011db0 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.443+0000 7f7ad37fe700 1 -- 172.21.15.52:0/4262295068 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7aec0153c0 con 0x5575504d2a40 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7b052d4b80 10 client.15516 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7b052d4b80 10 client.15516 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7b052d4b80 20 client.15516 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:50.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575504e8470 0x5575504e98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7b052d4b80 4 client.15516 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557550428d00 con 0x5575504e8470 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.444+0000 7f7af3fff700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575504e8470 0x5575504e98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.445+0000 7f7af3fff700 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575504e8470 0x5575504e98e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4ded5f7cfea8868 server_cookie=686ceefbedc9bc34 in_seq=0 out_seq=0 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.445+0000 7f7ad37fe700 10 client.15516 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.445+0000 7f7ad37fe700 1 -- 172.21.15.52:0/4262295068 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557550428d00 con 0x5575504e8470 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.445+0000 7f7ad37fe700 10 client.15516 handle_command_reply: tid=0 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.445+0000 7f7b052d4b80 1 client.15516 shutdown 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.445+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 msgr2=0x5575504cb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cb540 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007b20 tx=0x7f7aec007e30 comp rx=0 tx=0).stop 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 shutdown_connections 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575504e8470 0x5575504e98e0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x5575504cb540 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7ad37fe700 0 client.15516 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/4262295068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504de520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 >> 172.21.15.52:0/4262295068 conn(0x5575503f7030 msgr2=0x5575504c46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 shutdown_connections 2024-06-02T23:11:50.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 1 -- 172.21.15.52:0/4262295068 wait complete. 2024-06-02T23:11:50.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.446+0000 7f7b052d4b80 20 client.15516 trim_cache size 0 max 16384 2024-06-02T23:11:50.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 Processor -- start 2024-06-02T23:11:50.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 -- start start 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504cc960 0x5575504ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 0x5575504e4b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504d2a40 0x5575504de3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5575504e7c20 con 0x5575504d2120 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5575504e82f0 con 0x5575504d2a40 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.478+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575504e89c0 con 0x5575504cc960 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 0x5575504e4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 0x5575504e4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51744/0 (socket says 172.21.15.52:51744) 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 -- 172.21.15.52:0/218181078 learned_addr learned my addr 172.21.15.52:0/218181078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af3fff700 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504cc960 0x5575504ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af8fee700 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504d2a40 0x5575504de3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 -- 172.21.15.52:0/218181078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504cc960 msgr2=0x5575504ccd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504cc960 0x5575504ccd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 -- 172.21.15.52:0/218181078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504d2a40 msgr2=0x5575504de3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504d2a40 0x5575504de3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 -- 172.21.15.52:0/218181078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575504c9ab0 con 0x5575504d2120 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af8fee700 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504d2a40 0x5575504de3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7af37fe700 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 0x5575504e4b60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8010f10 tx=0x7f7ae800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35713be8e4644678 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7ad9ffb700 1 -- 172.21.15.52:0/218181078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8005040 con 0x5575504d2120 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7ad9ffb700 1 -- 172.21.15.52:0/218181078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ae8005690 con 0x5575504d2120 2024-06-02T23:11:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.479+0000 7f7ad9ffb700 1 -- 172.21.15.52:0/218181078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ae801e680 con 0x5575504d2120 2024-06-02T23:11:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 -- 172.21.15.52:0/218181078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 msgr2=0x5575504e4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 0x5575504e4b60 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f7ae8010f10 tx=0x7f7ae800b480 comp rx=0 tx=0).stop 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 -- 172.21.15.52:0/218181078 shutdown_connections 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504d2a40 0x5575504de3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504cc960 0x5575504ccd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/218181078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2120 0x5575504e4b60 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 -- 172.21.15.52:0/218181078 >> 172.21.15.52:0/218181078 conn(0x5575503f7030 msgr2=0x5575504ddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 -- 172.21.15.52:0/218181078 shutdown_connections 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.480+0000 7f7b052d4b80 1 -- 172.21.15.52:0/218181078 wait complete. 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 -- start start 2024-06-02T23:11:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 Processor -- start 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504e0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504e0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x557550552d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5575504e9760 con 0x5575504d2a40 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5575504e9090 con 0x5575504cc960 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5575504e9e30 con 0x5575504d2120 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7b052d4b80 10 client.0 _ll_register_callbacks cb 0x5575503b1560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7af8fee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x557550552d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7af8fee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x557550552d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51746/0 (socket says 172.21.15.52:51746) 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7af8fee700 1 -- 172.21.15.52:0/489280596 learned_addr learned my addr 172.21.15.52:0/489280596 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7af37fe700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504e0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.481+0000 7f7af3fff700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504e0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af8fee700 1 -- 172.21.15.52:0/489280596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 msgr2=0x5575504e0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af8fee700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504e0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af8fee700 1 -- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 msgr2=0x5575504e0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af8fee700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504e0800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af8fee700 1 -- 172.21.15.52:0/489280596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557550553270 con 0x5575504d2a40 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af3fff700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af37fe700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7af8fee700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x557550552d00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007af0 tx=0x7f7aec0125d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fa7159e465e1617 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7ad37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7ad37fe700 1 -- 172.21.15.52:0/489280596 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aec01a070 con 0x5575504d2a40 2024-06-02T23:11:50.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ae8005690 con 0x5575504d2a40 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7ad37fe700 1 -- 172.21.15.52:0/489280596 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7aec004a90 con 0x5575504d2a40 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7ad37fe700 1 -- 172.21.15.52:0/489280596 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7aec031020 con 0x5575504d2a40 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7b052d4b80 10 client.15522 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.482+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557550554570 con 0x5575504d2a40 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7ad37fe700 1 -- 172.21.15.52:0/489280596 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7ae8005690 con 0x5575504d2a40 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7ad37fe700 1 -- 172.21.15.52:0/489280596 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7aec01f5d0 con 0x5575504d2a40 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7b052d4b80 10 client.15522 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7b052d4b80 10 client.15522 resolve_mds: resolved name 'b' to daemon mds.b 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7b052d4b80 20 client.15522 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:50.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575505590f0 0x557550557e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7b052d4b80 4 client.15522 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "status"}) -- 0x557550423280 con 0x5575505590f0 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7af3fff700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575505590f0 0x557550557e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.483+0000 7f7af3fff700 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575505590f0 0x557550557e00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57d38894a20b1ac2 server_cookie=7682391a64ded2f7 in_seq=0 out_seq=0 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7ad37fe700 10 client.15522 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7ad37fe700 1 -- 172.21.15.52:0/489280596 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x557550423280 con 0x5575505590f0 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7ad37fe700 10 client.15522 handle_command_reply: tid=0 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 client.15522 shutdown 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 msgr2=0x557550552d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x557550552d00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f7aec007af0 tx=0x7f7aec0125d0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 shutdown_connections 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5575505590f0 0x557550557e00 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5575504d2a40 0x557550552d00 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7ad37fe700 0 client.15522 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5575504cc960 0x5575504e0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 --2- 172.21.15.52:0/489280596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5575504d2120 0x5575504e0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 >> 172.21.15.52:0/489280596 conn(0x5575503f7030 msgr2=0x5575504d72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 shutdown_connections 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 1 -- 172.21.15.52:0/489280596 wait complete. 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.484+0000 7f7b052d4b80 20 client.15522 trim_cache size 0 max 16384 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: "cluster_fsid": "9d9fe36c-2133-11ef-bc9b-c7b262605968", 2024-06-02T23:11:50.489 INFO:teuthology.orchestra.run.smithi052.stdout: "whoami": 0, 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "id": 24470, 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "want_state": "up:active", 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "state": "up:active", 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "fs_name": "cephfs", 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "rank_uptime": 256.28120119900001, 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "mdsmap_epoch": 28, 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch": 90, 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_epoch_barrier": 90, 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout: "uptime": 274.98019768500001 2024-06-02T23:11:50.490 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:11:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7adc0a2ca0 msgr2=0x7f7adc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7adc0a2ca0 0x7f7adc0a5160 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7af4106130 tx=0x7f7ae800b410 comp rx=0 tx=0).stop 2024-06-02T23:11:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 msgr2=0x7f7af419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af419d6b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4006010 tx=0x7f7ae4009f80 comp rx=0 tx=0).stop 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 shutdown_connections 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7adc0a2ca0 0x7f7adc0a5160 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7af41066b0 0x7f7af4197870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7af41015d0 0x7f7af419d6b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 --2- 172.21.15.52:0/2563254286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7af4101f80 0x7f7af419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 >> 172.21.15.52:0/2563254286 conn(0x7f7af4078400 msgr2=0x7f7af4100af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.485+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 shutdown_connections 2024-06-02T23:11:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:50.486+0000 7f7afaa51700 1 -- 172.21.15.52:0/2563254286 wait complete. 2024-06-02T23:11:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:51 smithi174 ceph-mon[35994]: pgmap v430: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:11:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:51 smithi052 ceph-mon[32340]: pgmap v430: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:11:52.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:51 smithi123 ceph-mon[35977]: pgmap v430: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:11:52.401 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-06-02T23:11:52.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 -- 172.21.15.52:0/1273651912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 msgr2=0x7fdb6406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/1273651912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6406a5f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb58009990 tx=0x7fdb58009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 -- 172.21.15.52:0/1273651912 shutdown_connections 2024-06-02T23:11:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/1273651912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb6406ab30 0x7fdb6410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/1273651912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb64108500 0x7fdb641088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:52.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/1273651912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6406a5f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:52.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 -- 172.21.15.52:0/1273651912 >> 172.21.15.52:0/1273651912 conn(0x7fdb64078340 msgr2=0x7fdb64078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:52.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 -- 172.21.15.52:0/1273651912 shutdown_connections 2024-06-02T23:11:52.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.805+0000 7fdb6a02e700 1 -- 172.21.15.52:0/1273651912 wait complete. 2024-06-02T23:11:52.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 Processor -- start 2024-06-02T23:11:52.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 -- start start 2024-06-02T23:11:52.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:52.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb6406ab30 0x7fdb6419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:52.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb64108500 0x7fdb64198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:52.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdb64112630 con 0x7fdb6406a170 2024-06-02T23:11:52.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641124b0 con 0x7fdb64108500 2024-06-02T23:11:52.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.806+0000 7fdb6a02e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb641127b0 con 0x7fdb6406ab30 2024-06-02T23:11:52.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51766/0 (socket says 172.21.15.52:51766) 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 -- 172.21.15.52:0/3859990566 learned_addr learned my addr 172.21.15.52:0/3859990566 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb62ffd700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb6406ab30 0x7fdb6419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb63fff700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb64108500 0x7fdb64198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 -- 172.21.15.52:0/3859990566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb6406ab30 msgr2=0x7fdb6419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb6406ab30 0x7fdb6419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:52.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 -- 172.21.15.52:0/3859990566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb64108500 msgr2=0x7fdb64198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb64108500 0x7fdb64198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 -- 172.21.15.52:0/3859990566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb641990f0 con 0x7fdb6406a170 2024-06-02T23:11:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb63fff700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb64108500 0x7fdb64198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb637fe700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6419e7d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400d980 tx=0x7fdb5400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54017070 con 0x7fdb6406a170 2024-06-02T23:11:52.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb641993e0 con 0x7fdb6406a170 2024-06-02T23:11:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdb54005020 con 0x7fdb6406a170 2024-06-02T23:11:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54009b50 con 0x7fdb6406a170 2024-06-02T23:11:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.807+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb641a56a0 con 0x7fdb6406a170 2024-06-02T23:11:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.808+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb64069180 con 0x7fdb6406a170 2024-06-02T23:11:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.809+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdb54009cf0 con 0x7fdb6406a170 2024-06-02T23:11:52.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.810+0000 7fdb60ff9700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdb4c0a2c20 0x7fdb4c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:52.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.810+0000 7fdb62ffd700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdb4c0a2c20 0x7fdb4c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:52.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.810+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdb540ceee0 con 0x7fdb6406a170 2024-06-02T23:11:52.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.811+0000 7fdb62ffd700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdb4c0a2c20 0x7fdb4c0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b5c0 tx=0x7fdb58005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:52.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:52.815+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdb54020020 con 0x7fdb6406a170 2024-06-02T23:11:53.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.067+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb6402dca0 con 0x7fdb6406a170 2024-06-02T23:11:53.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.068+0000 7fdb60ff9700 1 -- 172.21.15.52:0/3859990566 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7fdb5409be70 con 0x7fdb6406a170 2024-06-02T23:11:53.067 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:11:53.067 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:11:53.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdb4c0a2c20 msgr2=0x7fdb4c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:53.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdb4c0a2c20 0x7fdb4c0a50e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fdb5800b5c0 tx=0x7fdb58005960 comp rx=0 tx=0).stop 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 msgr2=0x7fdb6419e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6419e7d0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400d980 tx=0x7fdb5400dc90 comp rx=0 tx=0).stop 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 shutdown_connections 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdb4c0a2c20 0x7fdb4c0a50e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdb64108500 0x7fdb64198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdb6406a170 0x7fdb6419e7d0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 --2- 172.21.15.52:0/3859990566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb6406ab30 0x7fdb6419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 >> 172.21.15.52:0/3859990566 conn(0x7fdb64078340 msgr2=0x7fdb64100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 shutdown_connections 2024-06-02T23:11:53.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:53.070+0000 7fdb6a02e700 1 -- 172.21.15.52:0/3859990566 wait complete. 2024-06-02T23:11:53.071 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:11:53.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:53 smithi052 ceph-mon[32340]: pgmap v431: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:53.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:53 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3859990566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:53.773 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:11:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:53 smithi174 ceph-mon[35994]: pgmap v431: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:53 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3859990566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:53 smithi123 ceph-mon[35977]: pgmap v431: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-06-02T23:11:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:53 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3859990566' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:11:54.026 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:55.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 -- 172.21.15.52:0/1315716238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70108520 msgr2=0x7f7a70108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1315716238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70108520 0x7f7a70108900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a60009990 tx=0x7f7a60009ca0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 -- 172.21.15.52:0/1315716238 shutdown_connections 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1315716238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a70102ed0 0x7f7a7010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1315716238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70108520 0x7f7a70108900 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1315716238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70102510 0x7f7a70102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.715+0000 7f7a76df0700 1 -- 172.21.15.52:0/1315716238 >> 172.21.15.52:0/1315716238 conn(0x7f7a70079640 msgr2=0x7f7a70079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 -- 172.21.15.52:0/1315716238 shutdown_connections 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 -- 172.21.15.52:0/1315716238 wait complete. 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 Processor -- start 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 -- start start 2024-06-02T23:11:55.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a70102510 0x7f7a70074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 0x7f7a70074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70108520 0x7f7a70075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a70112660 con 0x7f7a70102ed0 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701124e0 con 0x7f7a70108520 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.716+0000 7f7a76df0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701127e0 con 0x7f7a70102510 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 0x7f7a70074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 0x7f7a70074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40726/0 (socket says 172.21.15.52:40726) 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 -- 172.21.15.52:0/1282363830 learned_addr learned my addr 172.21.15.52:0/1282363830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a70102510 0x7f7a70074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a7538d700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70108520 0x7f7a70075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 -- 172.21.15.52:0/1282363830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a70102510 msgr2=0x7f7a70074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a70102510 0x7f7a70074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 -- 172.21.15.52:0/1282363830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70108520 msgr2=0x7f7a70075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70108520 0x7f7a70075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 -- 172.21.15.52:0/1282363830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a701a7030 con 0x7f7a70102ed0 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 0x7f7a70074cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400d980 tx=0x7f7a6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6dffb700 1 -- 172.21.15.52:0/1282363830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64016070 con 0x7f7a70102ed0 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6dffb700 1 -- 172.21.15.52:0/1282363830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a64004830 con 0x7f7a70102ed0 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a701a7320 con 0x7f7a70102ed0 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.717+0000 7f7a6dffb700 1 -- 172.21.15.52:0/1282363830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a64005530 con 0x7f7a70102ed0 2024-06-02T23:11:55.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.718+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a701a78b0 con 0x7f7a70102ed0 2024-06-02T23:11:55.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.720+0000 7f7a6dffb700 1 -- 172.21.15.52:0/1282363830 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a640049e0 con 0x7f7a70102ed0 2024-06-02T23:11:55.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.720+0000 7f7a6dffb700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a580a2ca0 0x7f7a580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.720+0000 7f7a6dffb700 1 -- 172.21.15.52:0/1282363830 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a640cec70 con 0x7f7a70102ed0 2024-06-02T23:11:55.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.720+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a580a2ca0 0x7f7a580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a580a2ca0 0x7f7a580a5160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b5c0 tx=0x7f7a60005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 Processor -- start 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 -- start start 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41d5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 0x562be41d0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562be4121810 con 0x562be41cff20 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562be40c9720 con 0x562be41d5640 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.721+0000 7f7a81673b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562be411d7d0 con 0x562be41d5f60 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 0x562be41d0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 0x562be41d0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40742/0 (socket says 172.21.15.52:40742) 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 -- 172.21.15.52:0/3334708326 learned_addr learned my addr 172.21.15.52:0/3334708326 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:55.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a74b8c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41d5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 -- 172.21.15.52:0/3334708326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 msgr2=0x562be41cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41cf9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 -- 172.21.15.52:0/3334708326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 msgr2=0x562be41d5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41d5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 -- 172.21.15.52:0/3334708326 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be40bff50 con 0x562be41cff20 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a7538d700 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 0x562be41d0340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a68003f90 tx=0x7f7a68009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f90053c5a4fa5ac9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a56ffd700 1 -- 172.21.15.52:0/3334708326 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68004bb0 con 0x562be41cff20 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a56ffd700 1 -- 172.21.15.52:0/3334708326 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a68004d50 con 0x562be41cff20 2024-06-02T23:11:55.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.722+0000 7f7a56ffd700 1 -- 172.21.15.52:0/3334708326 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68007cf0 con 0x562be41cff20 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 -- 172.21.15.52:0/3334708326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 msgr2=0x562be41d0340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 0x562be41d0340 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7a68003f90 tx=0x7f7a68009ba0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 -- 172.21.15.52:0/3334708326 shutdown_connections 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41cff20 0x562be41d0340 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41d5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 --2- 172.21.15.52:0/3334708326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 -- 172.21.15.52:0/3334708326 >> 172.21.15.52:0/3334708326 conn(0x562be40facc0 msgr2=0x562be41ca850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 -- 172.21.15.52:0/3334708326 shutdown_connections 2024-06-02T23:11:55.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 -- 172.21.15.52:0/3334708326 wait complete. 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 -- start start 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.723+0000 7f7a81673b80 1 Processor -- start 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41dbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5640 0x562be41d9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41d9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562be41dc9d0 con 0x562be41d5f60 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562be40c1bf0 con 0x562be41cff20 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562be41dcb50 con 0x562be41d5640 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a81673b80 10 client.0 _ll_register_callbacks cb 0x562be40bb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41d9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41d9540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40758/0 (socket says 172.21.15.52:40758) 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 -- 172.21.15.52:0/1679399559 learned_addr learned my addr 172.21.15.52:0/1679399559 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5640 0x562be41d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41dbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 -- 172.21.15.52:0/1679399559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5640 msgr2=0x562be41d9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5640 0x562be41d9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 -- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 msgr2=0x562be41dbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41dbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.724+0000 7f7a7538d700 1 -- 172.21.15.52:0/1679399559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be40708d0 con 0x562be41d5f60 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a7538d700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41d9540 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7a68004d00 tx=0x7f7a68007580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee729ecf9472954 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a4ffff700 1 -- 172.21.15.52:0/1679399559 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68022070 con 0x562be41d5f60 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a68004d50 con 0x562be41d5f60 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a4ffff700 1 -- 172.21.15.52:0/1679399559 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a68007b20 con 0x562be41d5f60 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a4ffff700 1 -- 172.21.15.52:0/1679399559 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a6802e050 con 0x562be41d5f60 2024-06-02T23:11:55.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a81673b80 10 client.15540 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562be41e7300 con 0x562be41d5f60 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a4ffff700 1 -- 172.21.15.52:0/1679399559 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a68004d50 con 0x562be41d5f60 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a4ffff700 1 -- 172.21.15.52:0/1679399559 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a68015b90 con 0x562be41d5f60 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a81673b80 10 client.15540 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a81673b80 10 client.15540 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.725+0000 7f7a81673b80 20 client.15540 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.726+0000 7f7a81673b80 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be41e97d0 0x562be41ebbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.726+0000 7f7a81673b80 4 client.15540 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.726+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562be412ed10 con 0x562be41e97d0 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.726+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be41e97d0 0x562be41ebbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.726+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be41e97d0 0x562be41ebbc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7dac19a5e4e12523 server_cookie=677c7e6a1eae7b09 in_seq=0 out_seq=0 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.726+0000 7f7a4ffff700 10 client.15540 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:55.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a4ffff700 1 -- 172.21.15.52:0/1679399559 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562be412ed10 con 0x562be41e97d0 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a4ffff700 10 client.15540 handle_command_reply: tid=0 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 client.15540 shutdown 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 msgr2=0x562be41d9540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41d9540 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7a68004d00 tx=0x7f7a68007580 comp rx=0 tx=0).stop 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 shutdown_connections 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41d9540 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be41e97d0 0x562be41ebbc0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41dbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 --2- 172.21.15.52:0/1679399559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5640 0x562be41d9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a4ffff700 0 client.15540 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 >> 172.21.15.52:0/1679399559 conn(0x562be40facc0 msgr2=0x562be41ce9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 shutdown_connections 2024-06-02T23:11:55.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.727+0000 7f7a81673b80 1 -- 172.21.15.52:0/1679399559 wait complete. 2024-06-02T23:11:55.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.728+0000 7f7a81673b80 20 client.15540 trim_cache size 0 max 16384 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 Processor -- start 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 -- start start 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41cf9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 0x562be41d37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41d6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701aed30 con 0x562be41d5640 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701b0870 con 0x562be41cff20 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a81673b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701af400 con 0x562be41d5f60 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 0x562be41d37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 0x562be41d37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40760/0 (socket says 172.21.15.52:40760) 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 -- 172.21.15.52:0/2778666049 learned_addr learned my addr 172.21.15.52:0/2778666049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 -- 172.21.15.52:0/2778666049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 msgr2=0x562be41d6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41d6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 -- 172.21.15.52:0/2778666049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 msgr2=0x562be41cf9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41cf9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41cf9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.743+0000 7f7a6ffff700 1 -- 172.21.15.52:0/2778666049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562be41d7570 con 0x562be41d5640 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 0x562be41d37d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7a64097ed0 tx=0x7f7a64086050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbd983853b44f8f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a567fc700 1 -- 172.21.15.52:0/2778666049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a640b9070 con 0x562be41d5640 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a567fc700 1 -- 172.21.15.52:0/2778666049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6409b500 con 0x562be41d5640 2024-06-02T23:11:55.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a567fc700 1 -- 172.21.15.52:0/2778666049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a640b4070 con 0x562be41d5640 2024-06-02T23:11:55.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a81673b80 1 -- 172.21.15.52:0/2778666049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 msgr2=0x562be41d37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a81673b80 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 0x562be41d37d0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7a64097ed0 tx=0x7f7a64086050 comp rx=0 tx=0).stop 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a81673b80 1 -- 172.21.15.52:0/2778666049 shutdown_connections 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a81673b80 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41d5f60 0x562be41d6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a81673b80 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41cff20 0x562be41cf9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.744+0000 7f7a81673b80 1 --2- 172.21.15.52:0/2778666049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5640 0x562be41d37d0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- 172.21.15.52:0/2778666049 >> 172.21.15.52:0/2778666049 conn(0x562be40facc0 msgr2=0x562be41d30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- 172.21.15.52:0/2778666049 shutdown_connections 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- 172.21.15.52:0/2778666049 wait complete. 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- start start 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 Processor -- start 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41cff20 0x562be41e2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41f0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41f1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701b01a0 con 0x562be41d5f60 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701afad0 con 0x562be41d5640 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a701accf0 con 0x562be41cff20 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.745+0000 7f7a81673b80 10 client.0 _ll_register_callbacks cb 0x562be40bb920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41f1020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40764/0 (socket says 172.21.15.52:40764) 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 -- 172.21.15.52:0/4065111723 learned_addr learned my addr 172.21.15.52:0/4065111723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41f0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41cff20 0x562be41e2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 -- 172.21.15.52:0/4065111723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41cff20 msgr2=0x562be41e2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41cff20 0x562be41e2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 -- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 msgr2=0x562be41f0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41f0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 -- 172.21.15.52:0/4065111723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562be41f1890 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a6ffff700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a7538d700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41f1020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c040 tx=0x7f7a68004780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fa75d73ab347630 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a4ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a4ffff700 1 -- 172.21.15.52:0/4065111723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a68022070 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a4ffff700 1 -- 172.21.15.52:0/4065111723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a680073e0 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.746+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a6409b500 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a4ffff700 1 -- 172.21.15.52:0/4065111723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a680047f0 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 10 client.15546 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x562be4254180 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a4ffff700 1 -- 172.21.15.52:0/4065111723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a6409b500 con 0x562be41d5f60 2024-06-02T23:11:55.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a4ffff700 1 -- 172.21.15.52:0/4065111723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a68015480 con 0x562be41d5f60 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 10 client.15546 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 20 client.15546 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be4254880 0x562be4256ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 4 client.15546 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562be4129290 con 0x562be4254880 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.747+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be4254880 0x562be4256ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a74b8c700 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be4254880 0x562be4256ca0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1fc4e91ef236a233 server_cookie=bb396e8082287fe1 in_seq=0 out_seq=0 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a4ffff700 10 client.15546 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a4ffff700 1 -- 172.21.15.52:0/4065111723 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562be4129290 con 0x562be4254880 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a4ffff700 10 client.15546 handle_command_reply: tid=0 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 client.15546 shutdown 2024-06-02T23:11:55.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 msgr2=0x562be41f1020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41f1020 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7a6800c040 tx=0x7f7a68004780 comp rx=0 tx=0).stop 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 shutdown_connections 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562be41d5f60 0x562be41f1020 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562be41cff20 0x562be41e2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x562be4254880 0x562be4256ca0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a4ffff700 0 client.15546 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 --2- 172.21.15.52:0/4065111723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562be41d5640 0x562be41f0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.748+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 >> 172.21.15.52:0/4065111723 conn(0x562be40facc0 msgr2=0x562be41c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 shutdown_connections 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a81673b80 1 -- 172.21.15.52:0/4065111723 wait complete. 2024-06-02T23:11:55.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a81673b80 20 client.15546 trim_cache size 0 max 16384 2024-06-02T23:11:55.753 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:11:55.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a580a2ca0 msgr2=0x7f7a580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a580a2ca0 0x7f7a580a5160 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b5c0 tx=0x7f7a60005960 comp rx=0 tx=0).stop 2024-06-02T23:11:55.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 msgr2=0x7f7a70074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 0x7f7a70074cd0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400d980 tx=0x7f7a6400dc90 comp rx=0 tx=0).stop 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 shutdown_connections 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a580a2ca0 0x7f7a580a5160 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a70108520 0x7f7a70075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a70102510 0x7f7a70074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 --2- 172.21.15.52:0/1282363830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a70102ed0 0x7f7a70074cd0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 >> 172.21.15.52:0/1282363830 conn(0x7f7a70079640 msgr2=0x7f7a70101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 shutdown_connections 2024-06-02T23:11:55.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:55.749+0000 7f7a76df0700 1 -- 172.21.15.52:0/1282363830 wait complete. 2024-06-02T23:11:55.760 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:55 smithi052 ceph-mon[32340]: pgmap v432: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:11:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:55 smithi174 ceph-mon[35994]: pgmap v432: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:11:56.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:55 smithi123 ceph-mon[35977]: pgmap v432: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:11:56.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:11:56.936 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:57 smithi052 ceph-mon[32340]: pgmap v433: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-02T23:11:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:57 smithi174 ceph-mon[35994]: pgmap v433: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-02T23:11:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:57 smithi123 ceph-mon[35977]: pgmap v433: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 -- 172.21.15.52:0/1503913124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc108101d70 msgr2=0x7fc10810e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/1503913124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc108101d70 0x7fc10810e250 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fc10000b0a0 tx=0x7fc10000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 -- 172.21.15.52:0/1503913124 shutdown_connections 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/1503913124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc108101d70 0x7fc10810e250 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/1503913124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108107410 0x7fc1081077f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/1503913124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1081013d0 0x7fc108101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 -- 172.21.15.52:0/1503913124 >> 172.21.15.52:0/1503913124 conn(0x7fc1080782c0 msgr2=0x7fc1080786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 -- 172.21.15.52:0/1503913124 shutdown_connections 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.590+0000 7fc10d6f5700 1 -- 172.21.15.52:0/1503913124 wait complete. 2024-06-02T23:11:58.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 Processor -- start 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 -- start start 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 0x7fc1081a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108101d70 0x7fc1081a0f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc108107410 0x7fc10819ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc108111540 con 0x7fc1081013d0 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081113c0 con 0x7fc108101d70 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.591+0000 7fc10d6f5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081116c0 con 0x7fc108107410 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 0x7fc1081a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 0x7fc1081a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40786/0 (socket says 172.21.15.52:40786) 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 -- 172.21.15.52:0/171212496 learned_addr learned my addr 172.21.15.52:0/171212496 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc1077fe700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc108107410 0x7fc10819ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc1067fc700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108101d70 0x7fc1081a0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 -- 172.21.15.52:0/171212496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc108107410 msgr2=0x7fc10819ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc108107410 0x7fc10819ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 -- 172.21.15.52:0/171212496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108101d70 msgr2=0x7fc1081a0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108101d70 0x7fc1081a0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 -- 172.21.15.52:0/171212496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc10819b400 con 0x7fc1081013d0 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc1067fc700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108101d70 0x7fc1081a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc106ffd700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 0x7fc1081a0a50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800d950 tx=0x7fc0f800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.592+0000 7fc0effff700 1 -- 172.21.15.52:0/171212496 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8016070 con 0x7fc1081013d0 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.593+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc10819b6f0 con 0x7fc1081013d0 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.593+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1081a79d0 con 0x7fc1081013d0 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.593+0000 7fc0effff700 1 -- 172.21.15.52:0/171212496 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0f8009df0 con 0x7fc1081013d0 2024-06-02T23:11:58.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.593+0000 7fc0effff700 1 -- 172.21.15.52:0/171212496 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f800f3c0 con 0x7fc1081013d0 2024-06-02T23:11:58.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.595+0000 7fc0effff700 1 -- 172.21.15.52:0/171212496 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0f8019070 con 0x7fc1081013d0 2024-06-02T23:11:58.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.596+0000 7fc0effff700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0f00a2ca0 0x7fc0f00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.596+0000 7fc1067fc700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0f00a2ca0 0x7fc0f00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.596+0000 7fc0effff700 1 -- 172.21.15.52:0/171212496 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc0f80ceb40 con 0x7fc1081013d0 2024-06-02T23:11:58.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.596+0000 7fc1067fc700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0f00a2ca0 0x7fc0f00a5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc10819c270 tx=0x7fc0fc00b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 Processor -- start 2024-06-02T23:11:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 -- start start 2024-06-02T23:11:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 0x5599c45a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a2880 0x5599c459c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c459cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5599c448ef60 con 0x5599c45a1f60 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599c44ed850 con 0x5599c459c7a0 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc117f78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599c4455460 con 0x5599c45a2880 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 0x5599c45a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 0x5599c45a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40800/0 (socket says 172.21.15.52:40800) 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc106ffd700 1 -- 172.21.15.52:0/3909591439 learned_addr learned my addr 172.21.15.52:0/3909591439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc1067fc700 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a2880 0x5599c459c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.597+0000 7fc1077fe700 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c459cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc106ffd700 1 -- 172.21.15.52:0/3909591439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a2880 msgr2=0x5599c459c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc106ffd700 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a2880 0x5599c459c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc106ffd700 1 -- 172.21.15.52:0/3909591439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 msgr2=0x5599c459cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc106ffd700 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c459cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc106ffd700 1 -- 172.21.15.52:0/3909591439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c443c8d0 con 0x5599c45a1f60 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc1077fe700 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c459cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc106ffd700 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 0x5599c45a2340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8097d40 tx=0x7fc0f8010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b22b6c75b92b2b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc0ed7fa700 1 -- 172.21.15.52:0/3909591439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f8020050 con 0x5599c45a1f60 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc0ed7fa700 1 -- 172.21.15.52:0/3909591439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0f809f720 con 0x5599c45a1f60 2024-06-02T23:11:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.598+0000 7fc0ed7fa700 1 -- 172.21.15.52:0/3909591439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80a3db0 con 0x5599c45a1f60 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 -- 172.21.15.52:0/3909591439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 msgr2=0x5599c45a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 0x5599c45a2340 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8097d40 tx=0x7fc0f8010830 comp rx=0 tx=0).stop 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 -- 172.21.15.52:0/3909591439 shutdown_connections 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c459cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a1f60 0x5599c45a2340 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 --2- 172.21.15.52:0/3909591439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a2880 0x5599c459c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 -- 172.21.15.52:0/3909591439 >> 172.21.15.52:0/3909591439 conn(0x5599c44c9080 msgr2=0x5599c4597220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 -- 172.21.15.52:0/3909591439 shutdown_connections 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 -- 172.21.15.52:0/3909591439 wait complete. 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 -- start start 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.599+0000 7fc117f78b80 1 Processor -- start 2024-06-02T23:11:58.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 0x5599c45a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c45a58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a2880 0x5599c45a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5599c448dfc0 con 0x5599c459c7a0 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599c44e9810 con 0x5599c45a2880 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599c45a9200 con 0x5599c45a1f60 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 10 client.0 _ll_register_callbacks cb 0x5599c44835b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 0x5599c45a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 0x5599c45a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40816/0 (socket says 172.21.15.52:40816) 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 -- 172.21.15.52:0/4237179484 learned_addr learned my addr 172.21.15.52:0/4237179484 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 -- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 msgr2=0x5599c45a58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc1077fe700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a2880 0x5599c45a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc1067fc700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c45a58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c45a58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 -- 172.21.15.52:0/4237179484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a2880 msgr2=0x5599c45a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a2880 0x5599c45a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 -- 172.21.15.52:0/4237179484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c44b3750 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc1077fe700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a2880 0x5599c45a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc106ffd700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 0x5599c45a85c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8012040 tx=0x7fc0f809b4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc314f7442381b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc0eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc0eaff5700 1 -- 172.21.15.52:0/4237179484 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0f80b3070 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc0eaff5700 1 -- 172.21.15.52:0/4237179484 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc0f80b4260 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0f809f720 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc0eaff5700 1 -- 172.21.15.52:0/4237179484 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc0f8020030 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 10 client.15558 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.600+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599c45b39b0 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc0eaff5700 1 -- 172.21.15.52:0/4237179484 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0f809f720 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc0eaff5700 1 -- 172.21.15.52:0/4237179484 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc0f80970c0 con 0x5599c459c7a0 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc117f78b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc117f78b80 10 client.15558 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc117f78b80 20 client.15558 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc117f78b80 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c45b5e80 0x5599c45b8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc117f78b80 4 client.15558 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599c44fad50 con 0x5599c45b5e80 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.601+0000 7fc1067fc700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c45b5e80 0x5599c45b8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.602+0000 7fc1067fc700 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c45b5e80 0x5599c45b8270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=60c8a7b5a7d4067c server_cookie=a61dbc8b3c1658c3 in_seq=0 out_seq=0 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.602+0000 7fc0eaff5700 10 client.15558 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.602+0000 7fc0eaff5700 1 -- 172.21.15.52:0/4237179484 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5599c44fad50 con 0x5599c45b5e80 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.602+0000 7fc0eaff5700 10 client.15558 handle_command_reply: tid=0 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.602+0000 7fc117f78b80 1 client.15558 shutdown 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 msgr2=0x5599c45a85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 0x5599c45a85c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8012040 tx=0x7fc0f809b4e0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 shutdown_connections 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a2880 0x5599c45a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c459c7a0 0x5599c45a85c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c45b5e80 0x5599c45b8270 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 --2- 172.21.15.52:0/4237179484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c45a58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 >> 172.21.15.52:0/4237179484 conn(0x5599c44c9080 msgr2=0x5599c4596700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc0eaff5700 0 client.15558 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 shutdown_connections 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 1 -- 172.21.15.52:0/4237179484 wait complete. 2024-06-02T23:11:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.603+0000 7fc117f78b80 20 client.15558 trim_cache size 0 max 16384 2024-06-02T23:11:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 Processor -- start 2024-06-02T23:11:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 -- start start 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c459c7a0 0x5599c45a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a1f60 0x5599c45a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c4596e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081aed00 con 0x5599c45a2880 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081b0840 con 0x5599c45a1f60 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.618+0000 7fc117f78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081af3d0 con 0x5599c459c7a0 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c4596e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c4596e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40828/0 (socket says 172.21.15.52:40828) 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 -- 172.21.15.52:0/2749460752 learned_addr learned my addr 172.21.15.52:0/2749460752 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc106ffd700 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c459c7a0 0x5599c45a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1067fc700 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a1f60 0x5599c45a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 -- 172.21.15.52:0/2749460752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c459c7a0 msgr2=0x5599c45a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c459c7a0 0x5599c45a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 -- 172.21.15.52:0/2749460752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a1f60 msgr2=0x5599c45a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a1f60 0x5599c45a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 -- 172.21.15.52:0/2749460752 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599c459c260 con 0x5599c45a2880 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1067fc700 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a1f60 0x5599c45a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc1077fe700 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c4596e80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc100009e00 tx=0x7fc1000090d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f33e157894f2a9f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc0ecff9700 1 -- 172.21.15.52:0/2749460752 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100003c00 con 0x5599c45a2880 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc0ecff9700 1 -- 172.21.15.52:0/2749460752 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc100003da0 con 0x5599c45a2880 2024-06-02T23:11:58.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.619+0000 7fc0ecff9700 1 -- 172.21.15.52:0/2749460752 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc100005280 con 0x5599c45a2880 2024-06-02T23:11:58.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 -- 172.21.15.52:0/2749460752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 msgr2=0x5599c4596e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c4596e80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fc100009e00 tx=0x7fc1000090d0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 -- 172.21.15.52:0/2749460752 shutdown_connections 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c4596e80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c459c7a0 0x5599c45a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 --2- 172.21.15.52:0/2749460752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c45a1f60 0x5599c45a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 -- 172.21.15.52:0/2749460752 >> 172.21.15.52:0/2749460752 conn(0x5599c44c9080 msgr2=0x5599c45a70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 -- 172.21.15.52:0/2749460752 shutdown_connections 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 -- 172.21.15.52:0/2749460752 wait complete. 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.620+0000 7fc117f78b80 1 -- start start 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 Processor -- start 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c4617b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c4618050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c45b84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081b0170 con 0x5599c45a2880 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081afaa0 con 0x5599c459c7a0 2024-06-02T23:11:58.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1081acca0 con 0x5599c45a1f60 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc117f78b80 10 client.0 _ll_register_callbacks cb 0x5599c44835b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c45b84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1067fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c4618050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1067fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c4618050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47170/0 (socket says 172.21.15.52:47170) 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c45b84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40838/0 (socket says 172.21.15.52:40838) 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc106ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c4617b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1067fc700 1 -- 172.21.15.52:0/93369404 learned_addr learned my addr 172.21.15.52:0/93369404 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 -- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 msgr2=0x5599c4618050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c4618050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 -- 172.21.15.52:0/93369404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 msgr2=0x5599c4617b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c4617b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1077fe700 1 -- 172.21.15.52:0/93369404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599c45b8cf0 con 0x5599c45a2880 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.621+0000 7fc1067fc700 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c4618050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc1077fe700 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c45b84b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc1000091f0 tx=0x7fc100003d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7931b781f9485623 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:58.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc0eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc0eaff5700 1 -- 172.21.15.52:0/93369404 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc10002a030 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc0eaff5700 1 -- 172.21.15.52:0/93369404 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc100007bd0 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc100003da0 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc0eaff5700 1 -- 172.21.15.52:0/93369404 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fc100035050 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc117f78b80 10 client.15564 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.622+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5599c4620280 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc0eaff5700 1 -- 172.21.15.52:0/93369404 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc100003da0 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc0eaff5700 1 -- 172.21.15.52:0/93369404 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fc100011070 con 0x5599c45a2880 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc117f78b80 10 client.15564 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc117f78b80 10 client.15564 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:11:58.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc117f78b80 20 client.15564 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:58.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc117f78b80 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c4620930 0x5599c4622d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc117f78b80 4 client.15564 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5599c44f52d0 con 0x5599c4620930 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.623+0000 7fc106ffd700 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c4620930 0x5599c4622d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc106ffd700 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c4620930 0x5599c4622d50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ae1ae754184f8e1 server_cookie=31a295770b7bcf9c in_seq=0 out_seq=0 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc0eaff5700 10 client.15564 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc0eaff5700 1 -- 172.21.15.52:0/93369404 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5599c44f52d0 con 0x5599c4620930 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc0eaff5700 10 client.15564 handle_command_reply: tid=0 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 client.15564 shutdown 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 msgr2=0x5599c45b84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c45b84b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc1000091f0 tx=0x7fc100003d30 comp rx=0 tx=0).stop 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 shutdown_connections 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599c45a2880 0x5599c45b84b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599c459c7a0 0x5599c4617b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5599c4620930 0x5599c4622d50 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 --2- 172.21.15.52:0/93369404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599c45a1f60 0x5599c4618050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc0eaff5700 0 client.15564 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 >> 172.21.15.52:0/93369404 conn(0x5599c44c9080 msgr2=0x5599c45a7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 shutdown_connections 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 1 -- 172.21.15.52:0/93369404 wait complete. 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.624+0000 7fc117f78b80 20 client.15564 trim_cache size 0 max 16384 2024-06-02T23:11:58.631 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0f00a2ca0 msgr2=0x7fc0f00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0f00a2ca0 0x7fc0f00a5160 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc10819c270 tx=0x7fc0fc00b2c0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 msgr2=0x7fc1081a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 0x7fc1081a0a50 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fc0f800d950 tx=0x7fc0f800dc60 comp rx=0 tx=0).stop 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 shutdown_connections 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0f00a2ca0 0x7fc0f00a5160 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc108107410 0x7fc10819ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1081013d0 0x7fc1081a0a50 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 --2- 172.21.15.52:0/171212496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc108101d70 0x7fc1081a0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 >> 172.21.15.52:0/171212496 conn(0x7fc1080782c0 msgr2=0x7fc10806ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 shutdown_connections 2024-06-02T23:11:58.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:11:58.625+0000 7fc10d6f5700 1 -- 172.21.15.52:0/171212496 wait complete. 2024-06-02T23:11:59.392 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:11:59.650 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:11:59.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 -- 172.21.15.52:0/1630682304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520100900 msgr2=0x7fe520100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 --2- 172.21.15.52:0/1630682304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520100900 0x7fe520100d20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe510009900 tx=0x7fe51002a410 comp rx=0 tx=0).stop 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 -- 172.21.15.52:0/1630682304 shutdown_connections 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 --2- 172.21.15.52:0/1630682304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520101af0 0x7fe520101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 --2- 172.21.15.52:0/1630682304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5201024b0 0x7fe520108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 --2- 172.21.15.52:0/1630682304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520100900 0x7fe520100d20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 -- 172.21.15.52:0/1630682304 >> 172.21.15.52:0/1630682304 conn(0x7fe5200fbe90 msgr2=0x7fe5200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 -- 172.21.15.52:0/1630682304 shutdown_connections 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.850+0000 7fe526812640 1 -- 172.21.15.52:0/1630682304 wait complete. 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 Processor -- start 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 -- start start 2024-06-02T23:11:59.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100900 0x7fe520197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 0x7fe520198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5201024b0 0x7fe52019f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010d510 con 0x7fe520101af0 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010d390 con 0x7fe5201024b0 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe526812640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe52010d690 con 0x7fe520100900 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.851+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 0x7fe520198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 0x7fe520198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40858/0 (socket says 172.21.15.52:40858) 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 -- 172.21.15.52:0/810845350 learned_addr learned my addr 172.21.15.52:0/810845350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51ffff640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100900 0x7fe520197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe524daf640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5201024b0 0x7fe52019f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 -- 172.21.15.52:0/810845350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100900 msgr2=0x7fe520197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100900 0x7fe520197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 -- 172.21.15.52:0/810845350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5201024b0 msgr2=0x7fe52019f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5201024b0 0x7fe52019f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 -- 172.21.15.52:0/810845350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52019f740 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 0x7fe520198060 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c890 tx=0x7fe51400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51d7fa640 1 -- 172.21.15.52:0/810845350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514004220 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52019fa30 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51d7fa640 1 -- 172.21.15.52:0/810845350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe5140043c0 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe51d7fa640 1 -- 172.21.15.52:0/810845350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514004a00 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.852+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe52019ffd0 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 Processor -- start 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe51d7fa640 1 -- 172.21.15.52:0/810845350 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe514004ba0 con 0x7fe520101af0 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 -- start start 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b974a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974aa00 0x5617b974f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b9752000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5617b96ad4f0 con 0x5617b974fb70 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5617b964b870 con 0x5617b974aa00 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.855+0000 7fe5295fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617b96a94b0 con 0x5617b974a010 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b9752000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b9752000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40864/0 (socket says 172.21.15.52:40864) 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2917155733 learned_addr learned my addr 172.21.15.52:0/2917155733 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe524daf640 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b974a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51ffff640 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974aa00 0x5617b974f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2917155733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 msgr2=0x5617b974a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b974a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2917155733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974aa00 msgr2=0x5617b974f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974aa00 0x5617b974f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2917155733 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b96679b0 con 0x5617b974fb70 2024-06-02T23:11:59.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b9752000 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5140d2ea0 tx=0x7fe5140d2f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf8e6627dd9cf00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51d7fa640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5080a2c30 0x7fe5080a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe524daf640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5080a2c30 0x7fe5080a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe5067fc640 1 -- 172.21.15.52:0/2917155733 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140e4070 con 0x5617b974fb70 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe51d7fa640 1 -- 172.21.15.52:0/810845350 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe5140ced80 con 0x7fe520101af0 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe5067fc640 1 -- 172.21.15.52:0/2917155733 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe514012af0 con 0x5617b974fb70 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.856+0000 7fe5067fc640 1 -- 172.21.15.52:0/2917155733 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140978b0 con 0x5617b974fb70 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 -- 172.21.15.52:0/2917155733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 msgr2=0x5617b9752000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe524daf640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5080a2c30 0x7fe5080a50f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5617b974b260 tx=0x7fe518007680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b9752000 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe5140d2ea0 tx=0x7fe5140d2f90 comp rx=0 tx=0).stop 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 -- 172.21.15.52:0/2917155733 shutdown_connections 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b9752000 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974aa00 0x5617b974f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2917155733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b974a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 -- 172.21.15.52:0/2917155733 >> 172.21.15.52:0/2917155733 conn(0x5617b9745680 msgr2=0x5617b9745a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 -- 172.21.15.52:0/2917155733 shutdown_connections 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.857+0000 7fe5295fb740 1 -- 172.21.15.52:0/2917155733 wait complete. 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 -- start start 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 Processor -- start 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b9758680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b9758bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974fb70 0x5617b975fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5617b9752b20 con 0x5617b974aa00 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5617b9636ae0 con 0x5617b974a010 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5617b9752ca0 con 0x5617b974fb70 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe5295fb740 10 client.0 _ll_register_callbacks cb 0x5617b966f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b9758bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b9758bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40868/0 (socket says 172.21.15.52:40868) 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 -- 172.21.15.52:0/32948161 learned_addr learned my addr 172.21.15.52:0/32948161 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe524daf640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974fb70 0x5617b975fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51ffff640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b9758680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 -- 172.21.15.52:0/32948161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974fb70 msgr2=0x5617b975fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974fb70 0x5617b975fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 -- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 msgr2=0x5617b9758680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b9758680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.858+0000 7fe51f7fe640 1 -- 172.21.15.52:0/32948161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b93c1210 con 0x5617b974aa00 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b9758bc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe5140a3d50 tx=0x7fe5140d2630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6eda1ff1abd941 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe4ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/32948161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401d040 con 0x5617b974aa00 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/32948161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe514004e90 con 0x5617b974aa00 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe514012af0 con 0x5617b974aa00 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/32948161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe514012ed0 con 0x5617b974aa00 2024-06-02T23:11:59.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 10 client.15576 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617b948d620 con 0x5617b974aa00 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/32948161 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe514012af0 con 0x5617b974aa00 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/32948161 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe51409b870 con 0x5617b974aa00 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 10 client.15576 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 10 client.15576 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 20 client.15576 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97640f0 0x5617b97664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 4 client.15576 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.859+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617b96c0f10 con 0x5617b97640f0 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.860+0000 7fe51ffff640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97640f0 0x5617b97664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.860+0000 7fe51ffff640 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97640f0 0x5617b97664e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a221a83616d0f51b server_cookie=c0089e9aebaa3da0 in_seq=0 out_seq=0 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.860+0000 7fe4ff7fe640 10 client.15576 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/32948161 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5617b96c0f10 con 0x5617b97640f0 2024-06-02T23:11:59.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe4ff7fe640 10 client.15576 handle_command_reply: tid=0 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 client.15576 shutdown 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 msgr2=0x5617b9758bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b9758bc0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe5140a3d50 tx=0x7fe5140d2630 comp rx=0 tx=0).stop 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 shutdown_connections 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97640f0 0x5617b97664e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974fb70 0x5617b975fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b9758bc0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe4ff7fe640 0 client.15576 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 --2- 172.21.15.52:0/32948161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b9758680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 >> 172.21.15.52:0/32948161 conn(0x5617b9745680 msgr2=0x5617b97447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 shutdown_connections 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 1 -- 172.21.15.52:0/32948161 wait complete. 2024-06-02T23:11:59.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.861+0000 7fe5295fb740 20 client.15576 trim_cache size 0 max 16384 2024-06-02T23:11:59.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 Processor -- start 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 -- start start 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b97409c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b975fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974fb70 0x5617b9743f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201a6a00 con 0x5617b974aa00 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201a8540 con 0x5617b974fb70 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe5295fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201a70d0 con 0x5617b974a010 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b975fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe51f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b975fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40872/0 (socket says 172.21.15.52:40872) 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2688833366 learned_addr learned my addr 172.21.15.52:0/2688833366 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:59.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe524daf640 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974fb70 0x5617b9743f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe51ffff640 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b97409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.878+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2688833366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 msgr2=0x5617b97409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b97409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2688833366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974fb70 msgr2=0x5617b9743f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974fb70 0x5617b9743f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe51f7fe640 1 -- 172.21.15.52:0/2688833366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617b974ade0 con 0x5617b974aa00 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b975fbc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe5140058c0 tx=0x7fe5140cc900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd7246ffbd615c1c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe505ffb640 1 -- 172.21.15.52:0/2688833366 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51401d070 con 0x5617b974aa00 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe505ffb640 1 -- 172.21.15.52:0/2688833366 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe51409bd10 con 0x5617b974aa00 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe505ffb640 1 -- 172.21.15.52:0/2688833366 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514009290 con 0x5617b974aa00 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe5295fb740 1 -- 172.21.15.52:0/2688833366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 msgr2=0x5617b975fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.879+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b975fbc0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe5140058c0 tx=0x7fe5140cc900 comp rx=0 tx=0).stop 2024-06-02T23:11:59.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- 172.21.15.52:0/2688833366 shutdown_connections 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974fb70 0x5617b9743f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974aa00 0x5617b975fbc0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2688833366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974a010 0x5617b97409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- 172.21.15.52:0/2688833366 >> 172.21.15.52:0/2688833366 conn(0x5617b9745680 msgr2=0x5617b9756ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- 172.21.15.52:0/2688833366 shutdown_connections 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- 172.21.15.52:0/2688833366 wait complete. 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- start start 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 Processor -- start 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b975fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974aa00 0x5617b975c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b975cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201a7e70 con 0x5617b974fb70 2024-06-02T23:11:59.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201a77a0 con 0x5617b974a010 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe5201a49e0 con 0x5617b974aa00 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe5295fb740 10 client.0 _ll_register_callbacks cb 0x7fe5140e2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe524daf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b975cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.880+0000 7fe524daf640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b975cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40878/0 (socket says 172.21.15.52:40878) 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 -- 172.21.15.52:0/2864689625 learned_addr learned my addr 172.21.15.52:0/2864689625 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe51ffff640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b975fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe51f7fe640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974aa00 0x5617b975c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 -- 172.21.15.52:0/2864689625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974aa00 msgr2=0x5617b975c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974aa00 0x5617b975c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 -- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 msgr2=0x5617b975fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b975fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 -- 172.21.15.52:0/2864689625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617b97ccc40 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe524daf640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b975cc30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe51801de10 tx=0x7fe518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fe7834a111efc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe4ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/2864689625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51801be90 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/2864689625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe5180263f0 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51409bd10 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/2864689625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe5180265f0 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe5295fb740 10 client.15582 fetch_fsmap learned FSMap version 28 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.881+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5617b97cdf60 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/2864689625 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe51409bd10 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/2864689625 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe518025070 con 0x5617b974fb70 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe5295fb740 10 client.15582 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe5295fb740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:11:59.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe5295fb740 20 client.15582 populate_metadata read hostname 'smithi052' 2024-06-02T23:11:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97ce970 0x5617b97d0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:11:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe5295fb740 4 client.15582 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:11:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5617b96bac20 con 0x5617b97ce970 2024-06-02T23:11:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.882+0000 7fe51ffff640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97ce970 0x5617b97d0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:11:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.883+0000 7fe51ffff640 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97ce970 0x5617b97d0d90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4068dccd474e02ec server_cookie=d70025c9396475c6 in_seq=0 out_seq=0 2024-06-02T23:11:59.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.883+0000 7fe4ff7fe640 10 client.15582 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe4ff7fe640 1 -- 172.21.15.52:0/2864689625 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5617b96bac20 con 0x5617b97ce970 2024-06-02T23:11:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe4ff7fe640 10 client.15582 handle_command_reply: tid=0 2024-06-02T23:11:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 client.15582 shutdown 2024-06-02T23:11:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 msgr2=0x5617b975cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b975cc30 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe51801de10 tx=0x7fe518004a40 comp rx=0 tx=0).stop 2024-06-02T23:11:59.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 shutdown_connections 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5617b97ce970 0x5617b97d0d90 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5617b974aa00 0x5617b975c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5617b974fb70 0x5617b975cc30 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 --2- 172.21.15.52:0/2864689625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5617b974a010 0x5617b975fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe4ff7fe640 0 client.15582 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 >> 172.21.15.52:0/2864689625 conn(0x5617b9745680 msgr2=0x5617b9757140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 shutdown_connections 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 1 -- 172.21.15.52:0/2864689625 wait complete. 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.886+0000 7fe5295fb740 20 client.15582 trim_cache size 0 max 16384 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:11:59.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:11:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:11:59.890 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5080a2c30 msgr2=0x7fe5080a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5080a2c30 0x7fe5080a50f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x5617b974b260 tx=0x7fe518007680 comp rx=0 tx=0).stop 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 msgr2=0x7fe520198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 0x7fe520198060 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c890 tx=0x7fe51400cd50 comp rx=0 tx=0).stop 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 shutdown_connections 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5080a2c30 0x7fe5080a50f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5201024b0 0x7fe52019f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe520101af0 0x7fe520198060 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 --2- 172.21.15.52:0/810845350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe520100900 0x7fe520197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 >> 172.21.15.52:0/810845350 conn(0x7fe5200fbe90 msgr2=0x7fe520104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 shutdown_connections 2024-06-02T23:11:59.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:11:59.887+0000 7fe526812640 1 -- 172.21.15.52:0/810845350 wait complete. 2024-06-02T23:11:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:11:59 smithi174 ceph-mon[35994]: pgmap v434: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-02T23:11:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:11:59 smithi052 ceph-mon[32340]: pgmap v434: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-02T23:12:00.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:11:59 smithi123 ceph-mon[35977]: pgmap v434: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 12 op/s 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 -- 172.21.15.52:0/467666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c01066b0 msgr2=0x7f68c010e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 --2- 172.21.15.52:0/467666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c01066b0 0x7f68c010e280 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f68b800b0d0 tx=0x7f68b800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 -- 172.21.15.52:0/467666365 shutdown_connections 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 --2- 172.21.15.52:0/467666365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c01066b0 0x7f68c010e280 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 --2- 172.21.15.52:0/467666365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c0101620 0x7f68c0101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 --2- 172.21.15.52:0/467666365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 0x7f68c0105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 -- 172.21.15.52:0/467666365 >> 172.21.15.52:0/467666365 conn(0x7f68c0078360 msgr2=0x7f68c0078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:01.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 -- 172.21.15.52:0/467666365 shutdown_connections 2024-06-02T23:12:01.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.358+0000 7f68c6d01700 1 -- 172.21.15.52:0/467666365 wait complete. 2024-06-02T23:12:01.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 Processor -- start 2024-06-02T23:12:01.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 -- start start 2024-06-02T23:12:01.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 0x7f68c019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 0x7f68c019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c01066b0 0x7f68c0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68c0111570 con 0x7f68c0101620 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01113f0 con 0x7f68c0101fd0 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c6d01700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01116f0 con 0x7f68c01066b0 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 0x7f68c019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 0x7f68c019e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40888/0 (socket says 172.21.15.52:40888) 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c4a9d700 1 -- 172.21.15.52:0/247041726 learned_addr learned my addr 172.21.15.52:0/247041726 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68c529e700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c01066b0 0x7f68c0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.359+0000 7f68bffff700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 0x7f68c019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c4a9d700 1 -- 172.21.15.52:0/247041726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c01066b0 msgr2=0x7f68c0198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c01066b0 0x7f68c0198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c4a9d700 1 -- 172.21.15.52:0/247041726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 msgr2=0x7f68c019ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 0x7f68c019ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c4a9d700 1 -- 172.21.15.52:0/247041726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c01990d0 con 0x7f68c0101620 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c529e700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c01066b0 0x7f68c0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68bffff700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 0x7f68c019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 0x7f68c019e7b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f68b000e9c0 tx=0x7f68b000ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68bdffb700 1 -- 172.21.15.52:0/247041726 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b000cbd0 con 0x7f68c0101620 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68bdffb700 1 -- 172.21.15.52:0/247041726 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b000cd70 con 0x7f68c0101620 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c01993c0 con 0x7f68c0101620 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68bdffb700 1 -- 172.21.15.52:0/247041726 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0010430 con 0x7f68c0101620 2024-06-02T23:12:01.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.360+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c01a5680 con 0x7f68c0101620 2024-06-02T23:12:01.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.363+0000 7f68bdffb700 1 -- 172.21.15.52:0/247041726 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68b0004750 con 0x7f68c0101620 2024-06-02T23:12:01.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.363+0000 7f68bdffb700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68a80a2ca0 0x7f68a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.363+0000 7f68bffff700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68a80a2ca0 0x7f68a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.363+0000 7f68bdffb700 1 -- 172.21.15.52:0/247041726 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68b00cf410 con 0x7f68c0101620 2024-06-02T23:12:01.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68bffff700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68a80a2ca0 0x7f68a80a5160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f68c0107ca0 tx=0x7f68b4005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 Processor -- start 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 -- start start 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd32296f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558bd3172810 con 0x558bd3224fd0 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558bd311a720 con 0x558bd3229c30 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68d1584b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558bd316e7d0 con 0x558bd32258f0 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.364+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32253b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32253b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40904/0 (socket says 172.21.15.52:40904) 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1646057796 learned_addr learned my addr 172.21.15.52:0/1646057796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:01.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c529e700 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68bffff700 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd32296f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1646057796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 msgr2=0x558bd32296f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd32296f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1646057796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 msgr2=0x558bd322a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1646057796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd3110f50 con 0x558bd3224fd0 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32253b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f68b0098610 tx=0x7f68b009b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef5db87a2692342 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68a6ffd700 1 -- 172.21.15.52:0/1646057796 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b009c340 con 0x558bd3224fd0 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68a6ffd700 1 -- 172.21.15.52:0/1646057796 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b009bc40 con 0x558bd3224fd0 2024-06-02T23:12:01.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.365+0000 7f68a6ffd700 1 -- 172.21.15.52:0/1646057796 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0004a40 con 0x558bd3224fd0 2024-06-02T23:12:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 -- 172.21.15.52:0/1646057796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 msgr2=0x558bd32253b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32253b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f68b0098610 tx=0x7f68b009b800 comp rx=0 tx=0).stop 2024-06-02T23:12:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 -- 172.21.15.52:0/1646057796 shutdown_connections 2024-06-02T23:12:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32253b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1646057796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd32296f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 -- 172.21.15.52:0/1646057796 >> 172.21.15.52:0/1646057796 conn(0x558bd314bcc0 msgr2=0x558bd321b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 -- 172.21.15.52:0/1646057796 shutdown_connections 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 -- 172.21.15.52:0/1646057796 wait complete. 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 -- start start 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.366+0000 7f68d1584b80 1 Processor -- start 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3224fd0 0x558bd32311e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 0x558bd322e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558bd3231dd0 con 0x558bd32258f0 2024-06-02T23:12:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558bd3112bf0 con 0x558bd3229c30 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558bd3231f50 con 0x558bd3224fd0 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68d1584b80 10 client.0 _ll_register_callbacks cb 0x558bd310c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 0x558bd322e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 0x558bd322e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40906/0 (socket says 172.21.15.52:40906) 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 -- 172.21.15.52:0/1641953862 learned_addr learned my addr 172.21.15.52:0/1641953862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3224fd0 0x558bd32311e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68c529e700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 -- 172.21.15.52:0/1641953862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3224fd0 msgr2=0x558bd32311e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3224fd0 0x558bd32311e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 -- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 msgr2=0x558bd322ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68bffff700 1 -- 172.21.15.52:0/1641953862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd30c18d0 con 0x558bd32258f0 2024-06-02T23:12:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.367+0000 7f68c529e700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f68bffff700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 0x558bd322e4e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ff70 tx=0x7f68b4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb64e5b6acd432b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f689ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f689ffff700 1 -- 172.21.15.52:0/1641953862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b4028670 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f689ffff700 1 -- 172.21.15.52:0/1641953862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68b4028810 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b009bc40 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f689ffff700 1 -- 172.21.15.52:0/1641953862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68b402f070 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f68d1584b80 10 client.15594 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558bd323c7b0 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f689ffff700 1 -- 172.21.15.52:0/1641953862 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b009bc40 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.368+0000 7f689ffff700 1 -- 172.21.15.52:0/1641953862 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f68b401f9c0 con 0x558bd32258f0 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68d1584b80 10 client.15594 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68d1584b80 10 client.15594 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:12:01.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68d1584b80 20 client.15594 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd323ee40 0x558bd3241230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68d1584b80 4 client.15594 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bd317fd10 con 0x558bd323ee40 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd323ee40 0x558bd3241230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd323ee40 0x558bd3241230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4d899cebd515765a server_cookie=659d5faae7739742 in_seq=0 out_seq=0 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.369+0000 7f689ffff700 10 client.15594 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.370+0000 7f689ffff700 1 -- 172.21.15.52:0/1641953862 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558bd317fd10 con 0x558bd323ee40 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.370+0000 7f689ffff700 10 client.15594 handle_command_reply: tid=0 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.370+0000 7f68d1584b80 1 client.15594 shutdown 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.370+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 msgr2=0x558bd322e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.370+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 0x558bd322e4e0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ff70 tx=0x7f68b4004750 comp rx=0 tx=0).stop 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 shutdown_connections 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd323ee40 0x558bd3241230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd322ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3224fd0 0x558bd32311e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f689ffff700 0 client.15594 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1641953862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd32258f0 0x558bd322e4e0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 >> 172.21.15.52:0/1641953862 conn(0x558bd314bcc0 msgr2=0x558bd321f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 shutdown_connections 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 1 -- 172.21.15.52:0/1641953862 wait complete. 2024-06-02T23:12:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.371+0000 7f68d1584b80 20 client.15594 trim_cache size 0 max 16384 2024-06-02T23:12:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 Processor -- start 2024-06-02T23:12:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 -- start start 2024-06-02T23:12:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd323b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd32258f0 0x558bd322a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3229c30 0x558bd3215510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01aa880 con 0x558bd3224fd0 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ac3c0 con 0x558bd32258f0 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68d1584b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01aaf50 con 0x558bd3229c30 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd323b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd323b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40922/0 (socket says 172.21.15.52:40922) 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1196665542 learned_addr learned my addr 172.21.15.52:0/1196665542 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.386+0000 7f68bffff700 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd32258f0 0x558bd322a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1196665542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3229c30 msgr2=0x558bd3215510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c529e700 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3229c30 0x558bd3215510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3229c30 0x558bd3215510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1196665542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd32258f0 msgr2=0x558bd322a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd32258f0 0x558bd322a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c4a9d700 1 -- 172.21.15.52:0/1196665542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bd3241890 con 0x558bd3224fd0 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd323b070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009fd0 tx=0x7f68b0005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b38738ac1a5f37 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68a67fc700 1 -- 172.21.15.52:0/1196665542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b009c340 con 0x558bd3224fd0 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68a67fc700 1 -- 172.21.15.52:0/1196665542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b0084f70 con 0x558bd3224fd0 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.387+0000 7f68a67fc700 1 -- 172.21.15.52:0/1196665542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b0099f40 con 0x558bd3224fd0 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 -- 172.21.15.52:0/1196665542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 msgr2=0x558bd323b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd323b070 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f68b0009fd0 tx=0x7f68b0005130 comp rx=0 tx=0).stop 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 -- 172.21.15.52:0/1196665542 shutdown_connections 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd3229c30 0x558bd3215510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd323b070 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 --2- 172.21.15.52:0/1196665542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd32258f0 0x558bd322a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 -- 172.21.15.52:0/1196665542 >> 172.21.15.52:0/1196665542 conn(0x558bd314bcc0 msgr2=0x558bd3214e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 -- 172.21.15.52:0/1196665542 shutdown_connections 2024-06-02T23:12:01.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 -- 172.21.15.52:0/1196665542 wait complete. 2024-06-02T23:12:01.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 -- start start 2024-06-02T23:12:01.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.388+0000 7f68d1584b80 1 Processor -- start 2024-06-02T23:12:01.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32375b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd3237af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd3232b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01abcf0 con 0x558bd3224fd0 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01ab620 con 0x558bd3229c30 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68c01a8820 con 0x558bd32258f0 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68d1584b80 10 client.0 _ll_register_callbacks cb 0x558bd310c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32375b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32375b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40930/0 (socket says 172.21.15.52:40930) 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 -- 172.21.15.52:0/286403061 learned_addr learned my addr 172.21.15.52:0/286403061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c529e700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd3232b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68bffff700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd3237af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 -- 172.21.15.52:0/286403061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 msgr2=0x558bd3237af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd3237af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 -- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 msgr2=0x558bd3232b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd3232b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 -- 172.21.15.52:0/286403061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bd3233350 con 0x558bd3224fd0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c529e700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd3232b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68bffff700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd3237af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f68c4a9d700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32375b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f68b009b940 tx=0x7f68b0005090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70719c6aa416385 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f689ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.389+0000 7f689ffff700 1 -- 172.21.15.52:0/286403061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68b00a8020 con 0x558bd3224fd0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f689ffff700 1 -- 172.21.15.52:0/286403061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68b0016df0 con 0x558bd3224fd0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68b0084f70 con 0x558bd3224fd0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f689ffff700 1 -- 172.21.15.52:0/286403061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f68b0099f40 con 0x558bd3224fd0 2024-06-02T23:12:01.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 10 client.15600 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558bd32a9760 con 0x558bd3224fd0 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f689ffff700 1 -- 172.21.15.52:0/286403061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68b0084f70 con 0x558bd3224fd0 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f689ffff700 1 -- 172.21.15.52:0/286403061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f68b0083030 con 0x558bd3224fd0 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 10 client.15600 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 10 client.15600 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 20 client.15600 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd32a99e0 0x558bd32abdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 4 client.15600 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.390+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558bd317a290 con 0x558bd32a99e0 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f68bffff700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd32a99e0 0x558bd32abdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f689ffff700 10 client.15600 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f68bffff700 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd32a99e0 0x558bd32abdd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b71befaf39bce03f server_cookie=f0212d9663eef974 in_seq=0 out_seq=0 2024-06-02T23:12:01.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f689ffff700 1 -- 172.21.15.52:0/286403061 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558bd317a290 con 0x558bd32a99e0 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f689ffff700 10 client.15600 handle_command_reply: tid=0 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f68d1584b80 1 client.15600 shutdown 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 msgr2=0x558bd32375b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.391+0000 7f68d1584b80 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32375b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f68b009b940 tx=0x7f68b0005090 comp rx=0 tx=0).stop 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 shutdown_connections 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558bd32a99e0 0x558bd32abdd0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558bd3229c30 0x558bd3232b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558bd3224fd0 0x558bd32375b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f689ffff700 0 client.15600 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 --2- 172.21.15.52:0/286403061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bd32258f0 0x558bd3237af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 >> 172.21.15.52:0/286403061 conn(0x558bd314bcc0 msgr2=0x558bd3228b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 shutdown_connections 2024-06-02T23:12:01.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 1 -- 172.21.15.52:0/286403061 wait complete. 2024-06-02T23:12:01.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68d1584b80 20 client.15600 trim_cache size 0 max 16384 2024-06-02T23:12:01.398 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68a80a2ca0 msgr2=0x7f68a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.392+0000 7f68c6d01700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68a80a2ca0 0x7f68a80a5160 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f68c0107ca0 tx=0x7f68b4005d80 comp rx=0 tx=0).stop 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 msgr2=0x7f68c019e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 0x7f68c019e7b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f68b000e9c0 tx=0x7f68b000ed80 comp rx=0 tx=0).stop 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 shutdown_connections 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68a80a2ca0 0x7f68a80a5160 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c01066b0 0x7f68c0198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68c0101620 0x7f68c019e7b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 --2- 172.21.15.52:0/247041726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c0101fd0 0x7f68c019ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:01.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 >> 172.21.15.52:0/247041726 conn(0x7f68c0078360 msgr2=0x7f68c006aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:01.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 shutdown_connections 2024-06-02T23:12:01.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:01.393+0000 7f68c6d01700 1 -- 172.21.15.52:0/247041726 wait complete. 2024-06-02T23:12:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:01 smithi174 ceph-mon[35994]: pgmap v435: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-02T23:12:01.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:01 smithi052 ceph-mon[32340]: pgmap v435: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-02T23:12:02.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:01 smithi123 ceph-mon[35977]: pgmap v435: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 341 B/s wr, 10 op/s 2024-06-02T23:12:02.091 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:12:02.345 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:03 smithi174 ceph-mon[35994]: pgmap v436: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:12:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:03 smithi052 ceph-mon[32340]: pgmap v436: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:12:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:03 smithi123 ceph-mon[35977]: pgmap v436: 97 pgs: 97 active+clean; 668 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 -- 172.21.15.52:0/1836553814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560107400 msgr2=0x7fd5601077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 --2- 172.21.15.52:0/1836553814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560107400 0x7fd5601077e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fd550009990 tx=0x7fd550009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 -- 172.21.15.52:0/1836553814 shutdown_connections 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 --2- 172.21.15.52:0/1836553814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560101d50 0x7fd56010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 --2- 172.21.15.52:0/1836553814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560107400 0x7fd5601077e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 --2- 172.21.15.52:0/1836553814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd560101390 0x7fd560101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 -- 172.21.15.52:0/1836553814 >> 172.21.15.52:0/1836553814 conn(0x7fd560078340 msgr2=0x7fd560078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 -- 172.21.15.52:0/1836553814 shutdown_connections 2024-06-02T23:12:04.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.185+0000 7fd5667c2700 1 -- 172.21.15.52:0/1836553814 wait complete. 2024-06-02T23:12:04.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 Processor -- start 2024-06-02T23:12:04.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 -- start start 2024-06-02T23:12:04.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd560101390 0x7fd56019e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 0x7fd56019ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560107400 0x7fd560198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601115a0 con 0x7fd560101d50 2024-06-02T23:12:04.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd560111420 con 0x7fd560107400 2024-06-02T23:12:04.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd5667c2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd560111720 con 0x7fd560101390 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 0x7fd56019ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 0x7fd56019ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57290/0 (socket says 172.21.15.52:57290) 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.186+0000 7fd55f7fe700 1 -- 172.21.15.52:0/2338977266 learned_addr learned my addr 172.21.15.52:0/2338977266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd560101390 0x7fd56019e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd564d5f700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560107400 0x7fd560198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55f7fe700 1 -- 172.21.15.52:0/2338977266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd560101390 msgr2=0x7fd56019e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd560101390 0x7fd56019e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55f7fe700 1 -- 172.21.15.52:0/2338977266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560107400 msgr2=0x7fd560198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560107400 0x7fd560198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55f7fe700 1 -- 172.21.15.52:0/2338977266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd560199190 con 0x7fd560101d50 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd564d5f700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560107400 0x7fd560198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 0x7fd56019ed60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd55400b530 tx=0x7fd55400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55d7fa700 1 -- 172.21.15.52:0/2338977266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554015020 con 0x7fd560101d50 2024-06-02T23:12:04.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd560199480 con 0x7fd560101d50 2024-06-02T23:12:04.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55d7fa700 1 -- 172.21.15.52:0/2338977266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd554011ce0 con 0x7fd560101d50 2024-06-02T23:12:04.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd55d7fa700 1 -- 172.21.15.52:0/2338977266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd554010e40 con 0x7fd560101d50 2024-06-02T23:12:04.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.187+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd560103320 con 0x7fd560101d50 2024-06-02T23:12:04.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.190+0000 7fd55d7fa700 1 -- 172.21.15.52:0/2338977266 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd554018020 con 0x7fd560101d50 2024-06-02T23:12:04.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.191+0000 7fd55d7fa700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5480a2c20 0x7fd5480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.191+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5480a2c20 0x7fd5480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.191+0000 7fd55d7fa700 1 -- 172.21.15.52:0/2338977266 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd5540ce890 con 0x7fd560101d50 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 Processor -- start 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 -- start start 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5480a2c20 0x7fd5480a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ffa10 tx=0x7fd550005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1edbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e0dff60 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e13e850 con 0x55dc0e1f3880 2024-06-02T23:12:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd571045b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e0a6460 con 0x55dc0e1f2f60 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1edbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1edbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57298/0 (socket says 172.21.15.52:57298) 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 -- 172.21.15.52:0/1290751644 learned_addr learned my addr 172.21.15.52:0/1290751644 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 -- 172.21.15.52:0/1290751644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 msgr2=0x55dc0e1f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 -- 172.21.15.52:0/1290751644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 msgr2=0x55dc0e1ed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1ed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.192+0000 7fd564d5f700 1 -- 172.21.15.52:0/1290751644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0e08d8d0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.193+0000 7fd55ffff700 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.193+0000 7fd564d5f700 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1edbc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd5580087c0 tx=0x7fd558008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed0749233ec2f4d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:04.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.193+0000 7fd5467fc700 1 -- 172.21.15.52:0/1290751644 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558003c00 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.193+0000 7fd5467fc700 1 -- 172.21.15.52:0/1290751644 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd558003da0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.193+0000 7fd5467fc700 1 -- 172.21.15.52:0/1290751644 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5580128b0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 -- 172.21.15.52:0/1290751644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 msgr2=0x55dc0e1edbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1edbc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fd5580087c0 tx=0x7fd558008ad0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 -- 172.21.15.52:0/1290751644 shutdown_connections 2024-06-02T23:12:04.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1edbc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 --2- 172.21.15.52:0/1290751644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1ed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 -- 172.21.15.52:0/1290751644 >> 172.21.15.52:0/1290751644 conn(0x55dc0e11a080 msgr2=0x55dc0e1e8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 -- 172.21.15.52:0/1290751644 shutdown_connections 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 -- 172.21.15.52:0/1290751644 wait complete. 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.194+0000 7fd571045b80 1 -- start start 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 Processor -- start 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1f9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e0defc0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e13a810 con 0x55dc0e1f3880 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc0e1fa260 con 0x55dc0e1f2f60 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd571045b80 10 client.0 _ll_register_callbacks cb 0x55dc0e0d45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:04.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1f9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57300/0 (socket says 172.21.15.52:57300) 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 -- 172.21.15.52:0/2801944098 learned_addr learned my addr 172.21.15.52:0/2801944098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd564d5f700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 -- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 msgr2=0x55dc0e1f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 -- 172.21.15.52:0/2801944098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 msgr2=0x55dc0e1f6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.195+0000 7fd55ffff700 1 -- 172.21.15.52:0/2801944098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0e104750 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd564d5f700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1f9620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd550015000 tx=0x7fd550009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e4f831d335669 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd53f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd53f7fe700 1 -- 172.21.15.52:0/2801944098 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550030660 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd53f7fe700 1 -- 172.21.15.52:0/2801944098 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd550030800 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd558003da0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd53f7fe700 1 -- 172.21.15.52:0/2801944098 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd550030a00 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd571045b80 10 client.15612 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc0e204a10 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.196+0000 7fd53f7fe700 1 -- 172.21.15.52:0/2801944098 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd550035070 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd53f7fe700 1 -- 172.21.15.52:0/2801944098 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd55003cc90 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd571045b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd571045b80 10 client.15612 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd571045b80 20 client.15612 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd571045b80 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e2070a0 0x55dc0e209490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd571045b80 4 client.15612 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd558003da0 con 0x55dc0e2070a0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e2070a0 0x55dc0e209490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd53f7fe700 10 client.15612 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.197+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e2070a0 0x55dc0e209490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1b11d8a7f506f5fb server_cookie=c2455f3127c6eeed in_seq=0 out_seq=0 2024-06-02T23:12:04.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd53f7fe700 1 -- 172.21.15.52:0/2801944098 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fd558003da0 con 0x55dc0e2070a0 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd53f7fe700 10 client.15612 handle_command_reply: tid=0 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 client.15612 shutdown 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 msgr2=0x55dc0e1f9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1f9620 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd550015000 tx=0x7fd550009310 comp rx=0 tx=0).stop 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 shutdown_connections 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e2070a0 0x55dc0e209490 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1f9620 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.198+0000 7fd571045b80 1 --2- 172.21.15.52:0/2801944098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.199+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 >> 172.21.15.52:0/2801944098 conn(0x55dc0e11a080 msgr2=0x55dc0e1e7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.199+0000 7fd53f7fe700 0 client.15612 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.199+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 shutdown_connections 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.199+0000 7fd571045b80 1 -- 172.21.15.52:0/2801944098 wait complete. 2024-06-02T23:12:04.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.199+0000 7fd571045b80 20 client.15612 trim_cache size 0 max 16384 2024-06-02T23:12:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 Processor -- start 2024-06-02T23:12:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 -- start start 2024-06-02T23:12:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601aecd0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601b0810 con 0x55dc0e1f2f60 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd571045b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601af3a0 con 0x55dc0e1f3880 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd55ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ed260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57306/0 (socket says 172.21.15.52:57306) 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd55ffff700 1 -- 172.21.15.52:0/2744889806 learned_addr learned my addr 172.21.15.52:0/2744889806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.215+0000 7fd564d5f700 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55ffff700 1 -- 172.21.15.52:0/2744889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 msgr2=0x55dc0e1f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55ffff700 1 -- 172.21.15.52:0/2744889806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f2f60 msgr2=0x55dc0e1edb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1edb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55ffff700 1 -- 172.21.15.52:0/2744889806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc0e1f5880 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd55ffff700 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ed260 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd550000c00 tx=0x7fd550009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdd8f84587e01b7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd545ffb700 1 -- 172.21.15.52:0/2744889806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55004a070 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd545ffb700 1 -- 172.21.15.52:0/2744889806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd550035040 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.216+0000 7fd545ffb700 1 -- 172.21.15.52:0/2744889806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55000fde0 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 -- 172.21.15.52:0/2744889806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 msgr2=0x55dc0e1ed260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ed260 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fd550000c00 tx=0x7fd550009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 -- 172.21.15.52:0/2744889806 shutdown_connections 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e1f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ed260 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 --2- 172.21.15.52:0/2744889806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1edb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 -- 172.21.15.52:0/2744889806 >> 172.21.15.52:0/2744889806 conn(0x55dc0e11a080 msgr2=0x55dc0e1f1a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 -- 172.21.15.52:0/2744889806 shutdown_connections 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.217+0000 7fd571045b80 1 -- 172.21.15.52:0/2744889806 wait complete. 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 -- start start 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 Processor -- start 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ff700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1ffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e20e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601b0140 con 0x55dc0e1f2f60 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601afa70 con 0x55dc0e1ed7a0 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd5601acc90 con 0x55dc0e1f3880 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1ffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd571045b80 10 client.0 _ll_register_callbacks cb 0x55dc0e0d45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd55f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1ffc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57308/0 (socket says 172.21.15.52:57308) 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd55f7fe700 1 -- 172.21.15.52:0/3643075073 learned_addr learned my addr 172.21.15.52:0/3643075073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.218+0000 7fd564d5f700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e20e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55ffff700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55f7fe700 1 -- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 msgr2=0x55dc0e20e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e20e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55f7fe700 1 -- 172.21.15.52:0/3643075073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1ed7a0 msgr2=0x55dc0e1ff700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ff700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55f7fe700 1 -- 172.21.15.52:0/3643075073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc0e20e970 con 0x55dc0e1f2f60 2024-06-02T23:12:04.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd564d5f700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e20e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd55f7fe700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1ffc40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd5540cc730 tx=0x7fd5540cc290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94994c4e665e9e29 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd53f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd53f7fe700 1 -- 172.21.15.52:0/3643075073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5540a3d90 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd550035040 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.219+0000 7fd53f7fe700 1 -- 172.21.15.52:0/3643075073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd55409f120 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd53f7fe700 1 -- 172.21.15.52:0/3643075073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd554066050 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 10 client.15618 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55dc0e2718d0 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd53f7fe700 1 -- 172.21.15.52:0/3643075073 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd550035040 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd53f7fe700 1 -- 172.21.15.52:0/3643075073 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd5540a3900 con 0x55dc0e1f2f60 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 10 client.15618 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:04.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 10 client.15618 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 20 client.15618 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e271fb0 0x55dc0e2743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 4 client.15618 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc0e14bd50 con 0x55dc0e271fb0 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.220+0000 7fd55ffff700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e271fb0 0x55dc0e2743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd53f7fe700 10 client.15618 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd55ffff700 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e271fb0 0x55dc0e2743d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1fc292d5bfc7a2 server_cookie=eb2ac9acdf95965e in_seq=0 out_seq=0 2024-06-02T23:12:04.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd53f7fe700 1 -- 172.21.15.52:0/3643075073 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc0e14bd50 con 0x55dc0e271fb0 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd53f7fe700 10 client.15618 handle_command_reply: tid=0 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 client.15618 shutdown 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 msgr2=0x55dc0e1ffc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1ffc40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd5540cc730 tx=0x7fd5540cc290 comp rx=0 tx=0).stop 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 shutdown_connections 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55dc0e271fb0 0x55dc0e2743d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc0e1f3880 0x55dc0e20e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dc0e1ed7a0 0x55dc0e1ff700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 --2- 172.21.15.52:0/3643075073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dc0e1f2f60 0x55dc0e1ffc40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 >> 172.21.15.52:0/3643075073 conn(0x55dc0e11a080 msgr2=0x55dc0e1f6a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd53f7fe700 0 client.15618 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 shutdown_connections 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.221+0000 7fd571045b80 1 -- 172.21.15.52:0/3643075073 wait complete. 2024-06-02T23:12:04.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd571045b80 20 client.15618 trim_cache size 0 max 16384 2024-06-02T23:12:04.229 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:04.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5480a2c20 msgr2=0x7fd5480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5480a2c20 0x7fd5480a50e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd5600ffa10 tx=0x7fd550005960 comp rx=0 tx=0).stop 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 msgr2=0x7fd56019ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 0x7fd56019ed60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fd55400b530 tx=0x7fd55400b840 comp rx=0 tx=0).stop 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 shutdown_connections 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd5480a2c20 0x7fd5480a50e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd560107400 0x7fd560198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd560101390 0x7fd56019e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 --2- 172.21.15.52:0/2338977266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd560101d50 0x7fd56019ed60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.222+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 >> 172.21.15.52:0/2338977266 conn(0x7fd560078340 msgr2=0x7fd560105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.223+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 shutdown_connections 2024-06-02T23:12:04.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:04.223+0000 7fd5667c2700 1 -- 172.21.15.52:0/2338977266 wait complete. 2024-06-02T23:12:05.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:12:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:05.740 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:05 smithi052 ceph-mon[32340]: pgmap v437: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:05 smithi174 ceph-mon[35994]: pgmap v437: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:06.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:05 smithi123 ceph-mon[35977]: pgmap v437: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:07.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.085+0000 7f5101503700 1 -- 172.21.15.52:0/1855013145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101d50 msgr2=0x7f50fc10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.085+0000 7f5101503700 1 --2- 172.21.15.52:0/1855013145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101d50 0x7f50fc10e240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b0a0 tx=0x7f50f400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 -- 172.21.15.52:0/1855013145 shutdown_connections 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 --2- 172.21.15.52:0/1855013145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101d50 0x7f50fc10e240 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 --2- 172.21.15.52:0/1855013145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 0x7f50fc1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 --2- 172.21.15.52:0/1855013145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50fc101390 0x7f50fc101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 -- 172.21.15.52:0/1855013145 >> 172.21.15.52:0/1855013145 conn(0x7f50fc078340 msgr2=0x7f50fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 -- 172.21.15.52:0/1855013145 shutdown_connections 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 -- 172.21.15.52:0/1855013145 wait complete. 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.086+0000 7f5101503700 1 Processor -- start 2024-06-02T23:12:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 -- start start 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 0x7f50fc19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50fc101d50 0x7f50fc19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 0x7f50fc19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1115a0 con 0x7f50fc101390 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc111420 con 0x7f50fc107400 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f5101503700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc111720 con 0x7f50fc101d50 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f50faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 0x7f50fc19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f50fa7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50fc101d50 0x7f50fc19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f50faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 0x7f50fc19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57332/0 (socket says 172.21.15.52:57332) 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f50faffd700 1 -- 172.21.15.52:0/219884881 learned_addr learned my addr 172.21.15.52:0/219884881 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.087+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 0x7f50fc19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50faffd700 1 -- 172.21.15.52:0/219884881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50fc101d50 msgr2=0x7f50fc19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50faffd700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50fc101d50 0x7f50fc19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50faffd700 1 -- 172.21.15.52:0/219884881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 msgr2=0x7f50fc19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50faffd700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 0x7f50fc19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50faffd700 1 -- 172.21.15.52:0/219884881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc19fd40 con 0x7f50fc101390 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 0x7f50fc19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50faffd700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 0x7f50fc19ad30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00d950 tx=0x7f50ec00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50e3fff700 1 -- 172.21.15.52:0/219884881 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec016070 con 0x7f50fc101390 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1a0030 con 0x7f50fc101390 2024-06-02T23:12:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50e3fff700 1 -- 172.21.15.52:0/219884881 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ec00fd40 con 0x7f50fc101390 2024-06-02T23:12:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f50e3fff700 1 -- 172.21.15.52:0/219884881 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00e3b0 con 0x7f50fc101390 2024-06-02T23:12:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.088+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1a78b0 con 0x7f50fc101390 2024-06-02T23:12:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.091+0000 7f50e3fff700 1 -- 172.21.15.52:0/219884881 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f50ec019070 con 0x7f50fc101390 2024-06-02T23:12:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.091+0000 7f50e3fff700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e40a2c20 0x7f50e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f50e3fff700 1 -- 172.21.15.52:0/219884881 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f50ec0cea50 con 0x7f50fc101390 2024-06-02T23:12:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e40a2c20 0x7f50e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e40a2c20 0x7f50e40a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50fc101280 tx=0x7f50f000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f510bd86b80 1 Processor -- start 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f510bd86b80 1 -- start start 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8ca340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8c4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.092+0000 7f510bd86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e5be7b6f60 con 0x55e5be8ca880 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f510bd86b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5be815850 con 0x55e5be8c9f60 2024-06-02T23:12:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f510bd86b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5be77d460 con 0x55e5be8c47a0 2024-06-02T23:12:07.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8c4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8ca340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44970/0 (socket says 172.21.15.52:44970) 2024-06-02T23:12:07.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8c4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57348/0 (socket says 172.21.15.52:57348) 2024-06-02T23:12:07.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50faffd700 1 -- 172.21.15.52:0/2659312848 learned_addr learned my addr 172.21.15.52:0/2659312848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 -- 172.21.15.52:0/2659312848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 msgr2=0x55e5be8c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 -- 172.21.15.52:0/2659312848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 msgr2=0x55e5be8ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 -- 172.21.15.52:0/2659312848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5be7648d0 con 0x55e5be8ca880 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50faffd700 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.093+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8c4260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f50f0003a20 tx=0x7f50f0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f311d8ade50d77c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.094+0000 7f50e17fa700 1 -- 172.21.15.52:0/2659312848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0004a30 con 0x55e5be8ca880 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.094+0000 7f50e17fa700 1 -- 172.21.15.52:0/2659312848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f00054d0 con 0x55e5be8ca880 2024-06-02T23:12:07.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.094+0000 7f50e17fa700 1 -- 172.21.15.52:0/2659312848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f0011ab0 con 0x55e5be8ca880 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.094+0000 7f510bd86b80 1 -- 172.21.15.52:0/2659312848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 msgr2=0x55e5be8c4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.094+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8c4260 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f50f0003a20 tx=0x7f50f0013000 comp rx=0 tx=0).stop 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 -- 172.21.15.52:0/2659312848 shutdown_connections 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2659312848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8c4260 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 -- 172.21.15.52:0/2659312848 >> 172.21.15.52:0/2659312848 conn(0x55e5be7f1080 msgr2=0x55e5be8bf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 -- 172.21.15.52:0/2659312848 shutdown_connections 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 -- 172.21.15.52:0/2659312848 wait complete. 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 -- start start 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.095+0000 7f510bd86b80 1 Processor -- start 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8cd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e5be7b5fc0 con 0x55e5be8ca880 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5be811810 con 0x55e5be8c9f60 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5be8d1260 con 0x55e5be8c47a0 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f510bd86b80 10 client.0 _ll_register_callbacks cb 0x55e5be7ab5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57352/0 (socket says 172.21.15.52:57352) 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 -- 172.21.15.52:0/2284096813 learned_addr learned my addr 172.21.15.52:0/2284096813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50faffd700 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 -- 172.21.15.52:0/2284096813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 msgr2=0x55e5be8d0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 -- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 msgr2=0x55e5be8cd880 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8cd880 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.096+0000 7f50fb7fe700 1 -- 172.21.15.52:0/2284096813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5be7db750 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8cddc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b0a0 tx=0x7f50f40076f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195a3bb400a2b9b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f50deff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f50deff5700 1 -- 172.21.15.52:0/2284096813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4007b10 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f50deff5700 1 -- 172.21.15.52:0/2284096813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f50f4007cb0 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50f00054d0 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f50deff5700 1 -- 172.21.15.52:0/2284096813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50f402f070 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f510bd86b80 10 client.15630 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.097+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5be8db9d0 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f50deff5700 1 -- 172.21.15.52:0/2284096813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50f00054d0 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f50deff5700 1 -- 172.21.15.52:0/2284096813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50f4011070 con 0x55e5be8ca880 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f510bd86b80 10 client.15630 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f510bd86b80 10 client.15630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f510bd86b80 20 client.15630 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:07.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be8de060 0x55e5be8e0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f510bd86b80 4 client.15630 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5be822d50 con 0x55e5be8de060 2024-06-02T23:12:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.098+0000 7f50faffd700 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be8de060 0x55e5be8e0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.099+0000 7f50deff5700 10 client.15630 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:07.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.099+0000 7f50faffd700 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be8de060 0x55e5be8e0480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866049921d7d4f95 server_cookie=7198eaee166d6f90 in_seq=0 out_seq=0 2024-06-02T23:12:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.099+0000 7f50deff5700 1 -- 172.21.15.52:0/2284096813 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e5be822d50 con 0x55e5be8de060 2024-06-02T23:12:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.099+0000 7f50deff5700 10 client.15630 handle_command_reply: tid=0 2024-06-02T23:12:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.099+0000 7f510bd86b80 1 client.15630 shutdown 2024-06-02T23:12:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 msgr2=0x55e5be8cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8cddc0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b0a0 tx=0x7f50f40076f0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 shutdown_connections 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be8de060 0x55e5be8e0480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8cddc0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 --2- 172.21.15.52:0/2284096813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8cd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f50deff5700 0 client.15630 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 >> 172.21.15.52:0/2284096813 conn(0x55e5be7f1080 msgr2=0x55e5be8be700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 shutdown_connections 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 1 -- 172.21.15.52:0/2284096813 wait complete. 2024-06-02T23:12:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.100+0000 7f510bd86b80 20 client.15630 trim_cache size 0 max 16384 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 Processor -- start 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 -- start start 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 0x55e5be8cddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c9f60 0x55e5be8cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8bee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1aec30 con 0x55e5be8ca880 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1b0770 con 0x55e5be8c47a0 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.115+0000 7f510bd86b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1af300 con 0x55e5be8c9f60 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 0x55e5be8cddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8bee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 0x55e5be8cddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44986/0 (socket says 172.21.15.52:44986) 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 -- 172.21.15.52:0/3721673027 learned_addr learned my addr 172.21.15.52:0/3721673027 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c9f60 0x55e5be8cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 -- 172.21.15.52:0/3721673027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c9f60 msgr2=0x55e5be8cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c9f60 0x55e5be8cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 -- 172.21.15.52:0/3721673027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 msgr2=0x55e5be8bee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8bee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 -- 172.21.15.52:0/3721673027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5be8cbdf0 con 0x55e5be8c47a0 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50faffd700 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 0x55e5be8cddf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0cc5d0 tx=0x7f50ec0cc450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d13e806177cc4dc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50e0ff9700 1 -- 172.21.15.52:0/3721673027 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0b4070 con 0x55e5be8c47a0 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50e0ff9700 1 -- 172.21.15.52:0/3721673027 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ec084070 con 0x55e5be8c47a0 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.116+0000 7f50e0ff9700 1 -- 172.21.15.52:0/3721673027 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50ec020040 con 0x55e5be8c47a0 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 -- 172.21.15.52:0/3721673027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 msgr2=0x55e5be8cddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 0x55e5be8cddf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0cc5d0 tx=0x7f50ec0cc450 comp rx=0 tx=0).stop 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 -- 172.21.15.52:0/3721673027 shutdown_connections 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8bee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c47a0 0x55e5be8cddf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3721673027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c9f60 0x55e5be8cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 -- 172.21.15.52:0/3721673027 >> 172.21.15.52:0/3721673027 conn(0x55e5be7f1080 msgr2=0x55e5be8cad20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 -- 172.21.15.52:0/3721673027 shutdown_connections 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.117+0000 7f510bd86b80 1 -- 172.21.15.52:0/3721673027 wait complete. 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 -- start start 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 Processor -- start 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8e5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8e5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1b00a0 con 0x55e5be8ca880 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1af9d0 con 0x55e5be8c9f60 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc1acbd0 con 0x55e5be8c47a0 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f510bd86b80 10 client.0 _ll_register_callbacks cb 0x55e5be7ab5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8e5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f50fa7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8e5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f50fa7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8e5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44996/0 (socket says 172.21.15.52:44996) 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8e5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57374/0 (socket says 172.21.15.52:57374) 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f50fa7fc700 1 -- 172.21.15.52:0/3443781519 learned_addr learned my addr 172.21.15.52:0/3443781519 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.118+0000 7f50faffd700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fb7fe700 1 -- 172.21.15.52:0/3443781519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 msgr2=0x55e5be8d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fb7fe700 1 -- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 msgr2=0x55e5be8e5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8e5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fb7fe700 1 -- 172.21.15.52:0/3443781519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5be8e6540 con 0x55e5be8ca880 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fa7fc700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50fb7fe700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8e5d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b070 tx=0x7f50f400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4220e7dbf461482 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:07.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50deff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:07.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50deff5700 1 -- 172.21.15.52:0/3443781519 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4005630 con 0x55e5be8ca880 2024-06-02T23:12:07.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50ec084070 con 0x55e5be8ca880 2024-06-02T23:12:07.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50deff5700 1 -- 172.21.15.52:0/3443781519 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f50f40057d0 con 0x55e5be8ca880 2024-06-02T23:12:07.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f50deff5700 1 -- 172.21.15.52:0/3443781519 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f50f40059d0 con 0x55e5be8ca880 2024-06-02T23:12:07.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f510bd86b80 10 client.15636 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:07.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.119+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e5be948e00 con 0x55e5be8ca880 2024-06-02T23:12:07.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f50deff5700 1 -- 172.21.15.52:0/3443781519 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f50ec084070 con 0x55e5be8ca880 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f50deff5700 1 -- 172.21.15.52:0/3443781519 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f50f4011040 con 0x55e5be8ca880 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f510bd86b80 10 client.15636 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f510bd86b80 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f510bd86b80 20 client.15636 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be9493f0 0x55e5be94b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f510bd86b80 4 client.15636 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5be81d2d0 con 0x55e5be9493f0 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.120+0000 7f50faffd700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be9493f0 0x55e5be94b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f50faffd700 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be9493f0 0x55e5be94b810 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ab24d87755209d server_cookie=4039ee153594d98c in_seq=0 out_seq=0 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f50deff5700 10 client.15636 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f50deff5700 1 -- 172.21.15.52:0/3443781519 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5be81d2d0 con 0x55e5be9493f0 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f50deff5700 10 client.15636 handle_command_reply: tid=0 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 client.15636 shutdown 2024-06-02T23:12:07.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 msgr2=0x55e5be8e5d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8e5d00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b070 tx=0x7f50f400b860 comp rx=0 tx=0).stop 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 shutdown_connections 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e5be9493f0 0x55e5be94b810 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5be8ca880 0x55e5be8e5d00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5be8c47a0 0x55e5be8d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 --2- 172.21.15.52:0/3443781519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5be8c9f60 0x55e5be8e5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 >> 172.21.15.52:0/3443781519 conn(0x55e5be7f1080 msgr2=0x55e5be8cf0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f50deff5700 0 client.15636 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 shutdown_connections 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 1 -- 172.21.15.52:0/3443781519 wait complete. 2024-06-02T23:12:07.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.121+0000 7f510bd86b80 20 client.15636 trim_cache size 0 max 16384 2024-06-02T23:12:07.127 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:07.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e40a2c20 msgr2=0x7f50e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e40a2c20 0x7f50e40a50e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f50fc101280 tx=0x7f50f000b3c0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 msgr2=0x7f50fc19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 0x7f50fc19ad30 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00d950 tx=0x7f50ec00dc60 comp rx=0 tx=0).stop 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 shutdown_connections 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50e40a2c20 0x7f50e40a50e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc107400 0x7f50fc19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f50fc101390 0x7f50fc19ad30 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 --2- 172.21.15.52:0/219884881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f50fc101d50 0x7f50fc19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 >> 172.21.15.52:0/219884881 conn(0x7f50fc078340 msgr2=0x7f50fc06a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 shutdown_connections 2024-06-02T23:12:07.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:07.122+0000 7f5101503700 1 -- 172.21.15.52:0/219884881 wait complete. 2024-06-02T23:12:07.401 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.9 2024-06-02T23:12:07.402 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:12:07.842 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:07 smithi052 ceph-mon[32340]: pgmap v438: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:07 smithi174 ceph-mon[35994]: pgmap v438: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:07 smithi123 ceph-mon[35977]: pgmap v438: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:08.016 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:08.873 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:12:08.873 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:12:09.685 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:09 smithi052 ceph-mon[32340]: pgmap v439: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:09.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2745636605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64108510 msgr2=0x7fca641088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:09.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2745636605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64108510 0x7fca641088f0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fca60009990 tx=0x7fca60009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:09.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2745636605 shutdown_connections 2024-06-02T23:12:09.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2745636605 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca64101440 0x7fca6410f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:09.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2745636605 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64108510 0x7fca641088f0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:09.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2745636605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64100a80 0x7fca64100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:09.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2745636605 >> 172.21.15.52:0/2745636605 conn(0x7fca64077a00 msgr2=0x7fca640fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:09.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2745636605 shutdown_connections 2024-06-02T23:12:09.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.793+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2745636605 wait complete. 2024-06-02T23:12:09.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.794+0000 7fca6b5a7700 1 Processor -- start 2024-06-02T23:12:09.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.794+0000 7fca6b5a7700 1 -- start start 2024-06-02T23:12:09.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.794+0000 7fca6b5a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 0x7fca64101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:09.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.794+0000 7fca6b5a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64101440 0x7fca641021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:09.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.794+0000 7fca6b5a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca64108510 0x7fca641026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:09.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca6b5a7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fca641126a0 con 0x7fca64100a80 2024-06-02T23:12:09.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca6b5a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fca64112520 con 0x7fca64101440 2024-06-02T23:12:09.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca6b5a7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fca64112820 con 0x7fca64108510 2024-06-02T23:12:09.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 0x7fca64101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:09.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 0x7fca64101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57384/0 (socket says 172.21.15.52:57384) 2024-06-02T23:12:09.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 -- 172.21.15.52:0/2487840853 learned_addr learned my addr 172.21.15.52:0/2487840853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:09.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca68b42700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64101440 0x7fca641021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:09.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69b44700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca64108510 0x7fca641026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:09.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 -- 172.21.15.52:0/2487840853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca64108510 msgr2=0x7fca641026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:09.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca64108510 0x7fca641026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:09.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 -- 172.21.15.52:0/2487840853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64101440 msgr2=0x7fca641021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:09.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64101440 0x7fca641021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:09.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 -- 172.21.15.52:0/2487840853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca641a70e0 con 0x7fca64100a80 2024-06-02T23:12:09.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca69343700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 0x7fca64101c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca60003ab0 tx=0x7fca60003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:09.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60027070 con 0x7fca64100a80 2024-06-02T23:12:09.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca641a7370 con 0x7fca64100a80 2024-06-02T23:12:09.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.795+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fca6000bdc0 con 0x7fca64100a80 2024-06-02T23:12:09.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.796+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60022ca0 con 0x7fca64100a80 2024-06-02T23:12:09.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.796+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca641a78a0 con 0x7fca64100a80 2024-06-02T23:12:09.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.796+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6410a9f0 con 0x7fca64100a80 2024-06-02T23:12:09.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.798+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fca6002a070 con 0x7fca64100a80 2024-06-02T23:12:09.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.798+0000 7fca5a7fc700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fca500a2c20 0x7fca500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:09.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.799+0000 7fca68b42700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fca500a2c20 0x7fca500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:09.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.799+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fca600d0810 con 0x7fca64100a80 2024-06-02T23:12:09.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.799+0000 7fca68b42700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fca500a2c20 0x7fca500a50e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca64107ba0 tx=0x7fca5400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:09.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:09.806+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fca60013070 con 0x7fca64100a80 2024-06-02T23:12:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:09 smithi174 ceph-mon[35994]: pgmap v439: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:10.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:09 smithi123 ceph-mon[35977]: pgmap v439: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:10.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.093+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca64069180 con 0x7fca64100a80 2024-06-02T23:12:10.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.094+0000 7fca5a7fc700 1 -- 172.21.15.52:0/2487840853 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7fca6009d830 con 0x7fca64100a80 2024-06-02T23:12:10.095 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:12:10.095 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fca500a2c20 msgr2=0x7fca500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fca500a2c20 0x7fca500a50e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fca64107ba0 tx=0x7fca5400b410 comp rx=0 tx=0).stop 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 msgr2=0x7fca64101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 0x7fca64101c60 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fca60003ab0 tx=0x7fca60003ae0 comp rx=0 tx=0).stop 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 shutdown_connections 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fca500a2c20 0x7fca500a50e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fca64108510 0x7fca641026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:10.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fca64100a80 0x7fca64101c60 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 --2- 172.21.15.52:0/2487840853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64101440 0x7fca641021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 >> 172.21.15.52:0/2487840853 conn(0x7fca64077a00 msgr2=0x7fca640ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 shutdown_connections 2024-06-02T23:12:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:10.097+0000 7fca6b5a7700 1 -- 172.21.15.52:0/2487840853 wait complete. 2024-06-02T23:12:10.098 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:12:10.863 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:10 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2487840853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:10.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:10 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2487840853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:10 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2487840853' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:11.738 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 146.9 --await-for 45.0' 2024-06-02T23:12:11.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 146.9 --await-for 45.0 2024-06-02T23:12:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:11 smithi174 ceph-mon[35994]: pgmap v440: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:11 smithi052 ceph-mon[32340]: pgmap v440: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:12.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:11 smithi123 ceph-mon[35977]: pgmap v440: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1463155189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c103150 msgr2=0x7f7a5c107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1463155189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c103150 0x7f7a5c107140 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f7a480099c0 tx=0x7f7a48009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1463155189 shutdown_connections 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1463155189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c107870 0x7f7a5c10f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1463155189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a5c1027a0 0x7f7a5c102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1463155189 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c103150 0x7f7a5c107140 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1463155189 >> 172.21.15.52:0/1463155189 conn(0x7f7a5c079570 msgr2=0x7f7a5c079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1463155189 shutdown_connections 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.221+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1463155189 wait complete. 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 Processor -- start 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 -- start start 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a5c1027a0 0x7f7a5c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c103150 0x7f7a5c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 0x7f7a5c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1126a0 con 0x7f7a5c107870 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c112520 con 0x7f7a5c1027a0 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.222+0000 7f7a62b9e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c112820 con 0x7f7a5c103150 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 0x7f7a5c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 0x7f7a5c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57396/0 (socket says 172.21.15.52:57396) 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 -- 172.21.15.52:0/1615352025 learned_addr learned my addr 172.21.15.52:0/1615352025 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c103150 0x7f7a5c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6093a700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a5c1027a0 0x7f7a5c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 -- 172.21.15.52:0/1615352025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c103150 msgr2=0x7f7a5c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c103150 0x7f7a5c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 -- 172.21.15.52:0/1615352025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a5c1027a0 msgr2=0x7f7a5c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a5c1027a0 0x7f7a5c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 -- 172.21.15.52:0/1615352025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c1a7080 con 0x7f7a5c107870 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a6113b700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 0x7f7a5c075210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a54007ee0 tx=0x7f7a5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.223+0000 7f7a59ffb700 1 -- 172.21.15.52:0/1615352025 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a54004bb0 con 0x7f7a5c107870 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.224+0000 7f7a59ffb700 1 -- 172.21.15.52:0/1615352025 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a54004d50 con 0x7f7a5c107870 2024-06-02T23:12:12.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.224+0000 7f7a59ffb700 1 -- 172.21.15.52:0/1615352025 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540058e0 con 0x7f7a5c107870 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.224+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c1a7370 con 0x7f7a5c107870 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.224+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1a7830 con 0x7f7a5c107870 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.226+0000 7f7a59ffb700 1 -- 172.21.15.52:0/1615352025 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a5401a020 con 0x7f7a5c107870 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.227+0000 7f7a59ffb700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a4c0a2ca0 0x7f7a4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.227+0000 7f7a59ffb700 1 -- 172.21.15.52:0/1615352025 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a540ce6c0 con 0x7f7a5c107870 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.227+0000 7f7a6093a700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a4c0a2ca0 0x7f7a4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.227+0000 7f7a6093a700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a4c0a2ca0 0x7f7a4c0a5160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005950 tx=0x7f7a5000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 Processor -- start 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 -- start start 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 0x55683fce43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fcde2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcdec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55683fbd0fe0 con 0x55683fce3fe0 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55683fc2f8d0 con 0x55683fcde820 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55683fb974e0 con 0x55683fce4900 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 0x55683fce43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 0x55683fce43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57412/0 (socket says 172.21.15.52:57412) 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6093a700 1 -- 172.21.15.52:0/3088311492 learned_addr learned my addr 172.21.15.52:0/3088311492 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fcde2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.228+0000 7f7a6113b700 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcdec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a6093a700 1 -- 172.21.15.52:0/3088311492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 msgr2=0x55683fcde2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a6093a700 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fcde2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a6093a700 1 -- 172.21.15.52:0/3088311492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 msgr2=0x55683fcdec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a6093a700 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcdec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a6093a700 1 -- 172.21.15.52:0/3088311492 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683fb7e950 con 0x55683fce3fe0 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a6093a700 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 0x55683fce43c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a50010fa0 tx=0x7f7a50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e99cb254b69acda server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3088311492 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000be60 con 0x55683fce3fe0 2024-06-02T23:12:12.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3088311492 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a50011db0 con 0x55683fce3fe0 2024-06-02T23:12:12.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.229+0000 7f7a46ffd700 1 -- 172.21.15.52:0/3088311492 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5001c770 con 0x55683fce3fe0 2024-06-02T23:12:12.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3088311492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 msgr2=0x55683fce43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 0x55683fce43c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7a50010fa0 tx=0x7f7a50013000 comp rx=0 tx=0).stop 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3088311492 shutdown_connections 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcdec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce3fe0 0x55683fce43c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3088311492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fcde2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3088311492 >> 172.21.15.52:0/3088311492 conn(0x55683fc0b100 msgr2=0x55683fcd92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3088311492 shutdown_connections 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.230+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3088311492 wait complete. 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.231+0000 7f7a6d421b80 1 -- start start 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.231+0000 7f7a6d421b80 1 Processor -- start 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.231+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fcea5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55683fbd0040 con 0x55683fcde820 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55683fc2b890 con 0x55683fce3fe0 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55683fceb2e0 con 0x55683fce4900 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6d421b80 10 client.0 _ll_register_callbacks cb 0x55683fbc5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fcea5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fcea5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57424/0 (socket says 172.21.15.52:57424) 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 -- 172.21.15.52:0/2948085465 learned_addr learned my addr 172.21.15.52:0/2948085465 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6113b700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 -- 172.21.15.52:0/2948085465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 msgr2=0x55683fce7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 -- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 msgr2=0x55683fce7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 -- 172.21.15.52:0/2948085465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683fbf57d0 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a6093a700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fcea5f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7a50010a40 tx=0x7f7a5001e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17abc338bc8cf369 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.232+0000 7f7a3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a3bfff700 1 -- 172.21.15.52:0/2948085465 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a5000be60 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a3bfff700 1 -- 172.21.15.52:0/2948085465 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a50011bb0 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a50011db0 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a3bfff700 1 -- 172.21.15.52:0/2948085465 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a5001e8f0 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a6d421b80 10 client.15654 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55683fcfe280 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.233+0000 7f7a3bfff700 1 -- 172.21.15.52:0/2948085465 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a50011db0 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a3bfff700 1 -- 172.21.15.52:0/2948085465 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a5003d420 con 0x55683fcde820 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a6d421b80 10 client.15654 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:12.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a6d421b80 10 client.15654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a6d421b80 20 client.15654 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a6d421b80 4 client.15654 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55683fc3cdd0 con 0x55683fd00800 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.234+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd02bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.235+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd02bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1d308aa64728461 server_cookie=600ae19ad498f0eb in_seq=0 out_seq=0 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.235+0000 7f7a3bfff700 10 client.15654 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a3bfff700 1 -- 172.21.15.52:0/2948085465 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55683fc3cdd0 con 0x55683fd00800 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a3bfff700 10 client.15654 handle_command_reply: tid=0 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 client.15654 shutdown 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 msgr2=0x55683fcea5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fcea5f0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7a50010a40 tx=0x7f7a5001e650 comp rx=0 tx=0).stop 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 shutdown_connections 2024-06-02T23:12:12.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd02bf0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a3bfff700 0 client.15654 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fcea5f0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2948085465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 >> 172.21.15.52:0/2948085465 conn(0x55683fc0b100 msgr2=0x55683fcd8780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 shutdown_connections 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2948085465 wait complete. 2024-06-02T23:12:12.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.236+0000 7f7a6d421b80 20 client.15654 trim_cache size 0 max 16384 2024-06-02T23:12:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 Processor -- start 2024-06-02T23:12:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 -- start start 2024-06-02T23:12:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fce5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1aa920 con 0x55683fcde820 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fce5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1ac460 con 0x55683fce3fe0 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1aaff0 con 0x55683fce4900 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.258+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45040/0 (socket says 172.21.15.52:45040) 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fce5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57426/0 (socket says 172.21.15.52:57426) 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 -- 172.21.15.52:0/2391544374 learned_addr learned my addr 172.21.15.52:0/2391544374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a6113b700 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 -- 172.21.15.52:0/2391544374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 msgr2=0x55683fce98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 -- 172.21.15.52:0/2391544374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 msgr2=0x55683fce5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fce5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 -- 172.21.15.52:0/2391544374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55683fce5ed0 con 0x55683fce3fe0 2024-06-02T23:12:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a6093a700 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fce5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:12.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce68c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800a010 tx=0x7f7a48005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6813ae2e27ff703 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.259+0000 7f7a467fc700 1 -- 172.21.15.52:0/2391544374 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a4800bae0 con 0x55683fce3fe0 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.260+0000 7f7a467fc700 1 -- 172.21.15.52:0/2391544374 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a48003d40 con 0x55683fce3fe0 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.260+0000 7f7a467fc700 1 -- 172.21.15.52:0/2391544374 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a480056a0 con 0x55683fce3fe0 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.260+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2391544374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 msgr2=0x55683fce68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.260+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce68c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800a010 tx=0x7f7a48005fb0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.260+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2391544374 shutdown_connections 2024-06-02T23:12:12.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.260+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce4900 0x55683fce98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fcde820 0x55683fce5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/2391544374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fce3fe0 0x55683fce68c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2391544374 >> 172.21.15.52:0/2391544374 conn(0x55683fc0b100 msgr2=0x55683fce9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2391544374 shutdown_connections 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 -- 172.21.15.52:0/2391544374 wait complete. 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 -- start start 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.261+0000 7f7a6d421b80 1 Processor -- start 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcf0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce3fe0 0x55683fced570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6d421b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce4900 0x55683fcedab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6d421b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1abd90 con 0x55683fce4900 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcf0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6093a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcf0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45056/0 (socket says 172.21.15.52:45056) 2024-06-02T23:12:12.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.262+0000 7f7a6093a700 1 -- 172.21.15.52:0/3755473025 learned_addr learned my addr 172.21.15.52:0/3755473025 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce3fe0 0x55683fced570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a6113b700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce4900 0x55683fcedab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c1ab6c0 con 0x55683fcde820 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c19c8c0 con 0x55683fce3fe0 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a6d421b80 10 client.0 _ll_register_callbacks cb 0x55683fbc5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 -- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 msgr2=0x55683fcf0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcf0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 -- 172.21.15.52:0/3755473025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce4900 msgr2=0x55683fcedab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce4900 0x55683fcedab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 -- 172.21.15.52:0/3755473025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55683fcee120 con 0x55683fce3fe0 2024-06-02T23:12:12.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a5bfff700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce3fe0 0x55683fced570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55683fc2b520 tx=0x7f7a48009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7ec5574fa28b61 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.263+0000 7f7a3bfff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a3bfff700 1 -- 172.21.15.52:0/3755473025 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48003b10 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a3bfff700 1 -- 172.21.15.52:0/3755473025 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a480052e0 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a48003d40 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a3bfff700 1 -- 172.21.15.52:0/3755473025 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f7a48005500 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a6d421b80 10 client.25219 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55683fd6c130 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.264+0000 7f7a3bfff700 1 -- 172.21.15.52:0/3755473025 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a480056a0 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a3bfff700 1 -- 172.21.15.52:0/3755473025 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7a48037870 con 0x55683fce3fe0 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6d421b80 10 client.25219 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6d421b80 10 client.25219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6d421b80 20 client.25219 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:12.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6d421b80 4 client.25219 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7a48003d40 con 0x55683fd00800 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.265+0000 7f7a6093a700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.266+0000 7f7a3bfff700 10 client.25219 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.266+0000 7f7a6093a700 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd6e7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db6afc96ac31ba35 server_cookie=689bb487ea657adc in_seq=0 out_seq=0 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a3bfff700 1 -- 172.21.15.52:0/3755473025 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7a48003d40 con 0x55683fd00800 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a3bfff700 10 client.25219 handle_command_reply: tid=0 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 client.25219 shutdown 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce3fe0 msgr2=0x55683fced570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce3fe0 0x55683fced570 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x55683fc2b520 tx=0x7f7a48009680 comp rx=0 tx=0).stop 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 shutdown_connections 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55683fd00800 0x55683fd6e7d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55683fce4900 0x55683fcedab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55683fcde820 0x55683fcf0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 --2- 172.21.15.52:0/3755473025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55683fce3fe0 0x55683fced570 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 >> 172.21.15.52:0/3755473025 conn(0x55683fc0b100 msgr2=0x55683fce9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a3bfff700 0 client.25219 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 shutdown_connections 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.267+0000 7f7a6d421b80 1 -- 172.21.15.52:0/3755473025 wait complete. 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a6d421b80 20 client.25219 trim_cache size 0 max 16384 2024-06-02T23:12:12.270 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:12:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:12:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "6868a3fd-33cb-4f12-baa4-22c31bfc355d", 2024-06-02T23:12:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:12:12.271 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:12:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a4c0a2ca0 msgr2=0x7f7a4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a4c0a2ca0 0x7f7a4c0a5160 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7a50005950 tx=0x7f7a5000b410 comp rx=0 tx=0).stop 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 msgr2=0x7f7a5c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 0x7f7a5c075210 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f7a54007ee0 tx=0x7f7a5400d3b0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 shutdown_connections 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.268+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a4c0a2ca0 0x7f7a4c0a5160 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.269+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a5c107870 0x7f7a5c075210 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.269+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a5c1027a0 0x7f7a5c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.269+0000 7f7a62b9e700 1 --2- 172.21.15.52:0/1615352025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a5c103150 0x7f7a5c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.269+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 >> 172.21.15.52:0/1615352025 conn(0x7f7a5c079570 msgr2=0x7f7a5c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.269+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 shutdown_connections 2024-06-02T23:12:12.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:12.269+0000 7f7a62b9e700 1 -- 172.21.15.52:0/1615352025 wait complete. 2024-06-02T23:12:12.280 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:12 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:12:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:12 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:12.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:12 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:12:12.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:12 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.981+0000 7fd1a3977640 1 -- 172.21.15.52:0/2384235081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 msgr2=0x7fd19c1000d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.981+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2384235081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c1000d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fd198009930 tx=0x7fd19802a420 comp rx=0 tx=0).stop 2024-06-02T23:12:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.981+0000 7fd1a3977640 1 -- 172.21.15.52:0/2384235081 shutdown_connections 2024-06-02T23:12:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.981+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2384235081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 0x7fd19c103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.982+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2384235081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 0x7fd19c109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.982+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2384235081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c1000d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.982+0000 7fd1a3977640 1 -- 172.21.15.52:0/2384235081 >> 172.21.15.52:0/2384235081 conn(0x7fd19c0fb600 msgr2=0x7fd19c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.982+0000 7fd1a3977640 1 -- 172.21.15.52:0/2384235081 shutdown_connections 2024-06-02T23:12:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.982+0000 7fd1a3977640 1 -- 172.21.15.52:0/2384235081 wait complete. 2024-06-02T23:12:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 Processor -- start 2024-06-02T23:12:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 -- start start 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 0x7fd19c1980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 0x7fd19c19f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10d510 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10d390 con 0x7fd19c103640 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a3977640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c10d690 con 0x7fd19c102c80 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a1713640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a1713640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57446/0 (socket says 172.21.15.52:57446) 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.983+0000 7fd1a1713640 1 -- 172.21.15.52:0/2268806811 learned_addr learned my addr 172.21.15.52:0/2268806811 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 0x7fd19c1980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 0x7fd19c19f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1713640 1 -- 172.21.15.52:0/2268806811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 msgr2=0x7fd19c1980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1713640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 0x7fd19c1980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1713640 1 -- 172.21.15.52:0/2268806811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 msgr2=0x7fd19c19f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1713640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 0x7fd19c19f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1713640 1 -- 172.21.15.52:0/2268806811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd19c19f7a0 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 0x7fd19c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 0x7fd19c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a1713640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c197b80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd1980314e0 tx=0x7fd198004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1927fc640 1 -- 172.21.15.52:0/2268806811 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19803f030 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1927fc640 1 -- 172.21.15.52:0/2268806811 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd198003cb0 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1927fc640 1 -- 172.21.15.52:0/2268806811 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd198035070 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd19c19fa30 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.984+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd19c19ff70 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.987+0000 7fd1927fc640 1 -- 172.21.15.52:0/2268806811 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd198042030 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 Processor -- start 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 -- start start 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 0x55b04f4c3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3a00 0x55b04f4c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c8b70 0x55b04f4cb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b04f4264f0 con 0x55b04f4c3010 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b04f3c4870 con 0x55b04f4c3a00 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a6765740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b04f4224b0 con 0x55b04f4c8b70 2024-06-02T23:12:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a1f14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 0x55b04f4c3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1927fc640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1880a2cb0 0x7fd1880a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a1f14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 0x55b04f4c3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57462/0 (socket says 172.21.15.52:57462) 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a1f14640 1 -- 172.21.15.52:0/3591051989 learned_addr learned my addr 172.21.15.52:0/3591051989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a1713640 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3a00 0x55b04f4c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.988+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c8b70 0x55b04f4cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1880a2cb0 0x7fd1880a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 -- 172.21.15.52:0/3591051989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c8b70 msgr2=0x55b04f4cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c8b70 0x55b04f4cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 -- 172.21.15.52:0/3591051989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3a00 msgr2=0x55b04f4c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3a00 0x55b04f4c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 -- 172.21.15.52:0/3591051989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04f3e09b0 con 0x55b04f4c3010 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1713640 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3a00 0x55b04f4c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1927fc640 1 -- 172.21.15.52:0/2268806811 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd1980ef700 con 0x7fd19c0ffcb0 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 0x55b04f4c3430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd194013b20 tx=0x7fd194011380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eda3534944de19f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1877fe640 1 -- 172.21.15.52:0/3591051989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194011cd0 con 0x55b04f4c3010 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1877fe640 1 -- 172.21.15.52:0/3591051989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1940134e0 con 0x55b04f4c3010 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1880a2cb0 0x7fd1880a5170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd194020000 tx=0x7fd19401f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.989+0000 7fd1877fe640 1 -- 172.21.15.52:0/3591051989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19401da10 con 0x55b04f4c3010 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 -- 172.21.15.52:0/3591051989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 msgr2=0x55b04f4c3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 0x55b04f4c3430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fd194013b20 tx=0x7fd194011380 comp rx=0 tx=0).stop 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 -- 172.21.15.52:0/3591051989 shutdown_connections 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c8b70 0x55b04f4cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3a00 0x55b04f4c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 --2- 172.21.15.52:0/3591051989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3010 0x55b04f4c3430 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 -- 172.21.15.52:0/3591051989 >> 172.21.15.52:0/3591051989 conn(0x55b04f4be680 msgr2=0x55b04f4bea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 -- 172.21.15.52:0/3591051989 shutdown_connections 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.990+0000 7fd1a6765740 1 -- 172.21.15.52:0/3591051989 wait complete. 2024-06-02T23:12:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 -- start start 2024-06-02T23:12:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 Processor -- start 2024-06-02T23:12:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 0x55b04f4d4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b04f4cbb20 con 0x55b04f4c8b70 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b04f3afae0 con 0x55b04f4c3010 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b04f4cbca0 con 0x55b04f4c3a00 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a6765740 10 client.0 _ll_register_callbacks cb 0x55b04f3e89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a1f14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 0x55b04f4d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a1713640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a1f14640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 0x55b04f4d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57464/0 (socket says 172.21.15.52:57464) 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a1f14640 1 -- 172.21.15.52:0/215169380 learned_addr learned my addr 172.21.15.52:0/215169380 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.991+0000 7fd1a1713640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45098/0 (socket says 172.21.15.52:45098) 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a1f14640 1 -- 172.21.15.52:0/215169380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3a00 msgr2=0x55b04f4cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a1f14640 1 -- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3010 msgr2=0x55b04f4cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a1f14640 1 -- 172.21.15.52:0/215169380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04f13a210 con 0x55b04f4c8b70 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 0x55b04f4d4800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd194009a90 tx=0x7fd194009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bcb83618d11399f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd184ff9640 1 -- 172.21.15.52:0/215169380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194034060 con 0x55b04f4c8b70 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd184ff9640 1 -- 172.21.15.52:0/215169380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd194011cd0 con 0x55b04f4c8b70 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.992+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1940134e0 con 0x55b04f4c8b70 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd184ff9640 1 -- 172.21.15.52:0/215169380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd19401d790 con 0x55b04f4c8b70 2024-06-02T23:12:12.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 10 client.15672 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b04f206620 con 0x55b04f4c8b70 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd184ff9640 1 -- 172.21.15.52:0/215169380 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1940134e0 con 0x55b04f4c8b70 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd184ff9640 1 -- 172.21.15.52:0/215169380 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd194004c90 con 0x55b04f4c8b70 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 10 client.15672 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 10 client.15672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 20 client.15672 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f4d8e70 0x55b04f4db260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 4 client.15672 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.993+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b04f439f10 con 0x55b04f4d8e70 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.994+0000 7fd1a1713640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f4d8e70 0x55b04f4db260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.994+0000 7fd184ff9640 10 client.15672 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:12.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.994+0000 7fd1a1713640 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f4d8e70 0x55b04f4db260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eba42bdd9feaca5f server_cookie=97557aa8fc7c56a0 in_seq=0 out_seq=0 2024-06-02T23:12:13.001 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd184ff9640 1 -- 172.21.15.52:0/215169380 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b04f439f10 con 0x55b04f4d8e70 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd184ff9640 10 client.15672 handle_command_reply: tid=0 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 client.15672 shutdown 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 msgr2=0x55b04f4d4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 0x55b04f4d4800 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fd194009a90 tx=0x7fd194009cc0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 shutdown_connections 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f4d8e70 0x55b04f4db260 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c8b70 0x55b04f4d4800 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 --2- 172.21.15.52:0/215169380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 >> 172.21.15.52:0/215169380 conn(0x55b04f4be680 msgr2=0x55b04f4bd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd184ff9640 0 client.15672 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 shutdown_connections 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 1 -- 172.21.15.52:0/215169380 wait complete. 2024-06-02T23:12:13.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:12.995+0000 7fd1a6765740 20 client.15672 trim_cache size 0 max 16384 2024-06-02T23:12:13.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:12 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:12:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:12 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:12 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:12:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:12 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 Processor -- start 2024-06-02T23:12:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 -- start start 2024-06-02T23:12:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a8cf0 con 0x55b04f4c3a00 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1aa830 con 0x55b04f4c8b70 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a6765740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a93c0 con 0x55b04f4c3010 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a0f12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a0f12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57468/0 (socket says 172.21.15.52:57468) 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a0f12640 1 -- 172.21.15.52:0/2031090505 learned_addr learned my addr 172.21.15.52:0/2031090505 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.027+0000 7fd1a1713640 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd1a0f12640 1 -- 172.21.15.52:0/2031090505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 msgr2=0x55b04f4cd2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd1a0f12640 1 -- 172.21.15.52:0/2031090505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 msgr2=0x55b04f4d47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd1a0f12640 1 -- 172.21.15.52:0/2031090505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b04f4be4b0 con 0x55b04f4c3a00 2024-06-02T23:12:13.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd18c008c10 tx=0x7fd18c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac7b3cbeebdc747 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:13.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd186ffd640 1 -- 172.21.15.52:0/2031090505 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c01f070 con 0x55b04f4c3a00 2024-06-02T23:12:13.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd186ffd640 1 -- 172.21.15.52:0/2031090505 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd18c01a8c0 con 0x55b04f4c3a00 2024-06-02T23:12:13.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.028+0000 7fd186ffd640 1 -- 172.21.15.52:0/2031090505 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c00f980 con 0x55b04f4c3a00 2024-06-02T23:12:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 -- 172.21.15.52:0/2031090505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 msgr2=0x55b04f4cd830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fd18c008c10 tx=0x7fd18c011810 comp rx=0 tx=0).stop 2024-06-02T23:12:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 -- 172.21.15.52:0/2031090505 shutdown_connections 2024-06-02T23:12:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4cd830 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 --2- 172.21.15.52:0/2031090505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4cd2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 -- 172.21.15.52:0/2031090505 >> 172.21.15.52:0/2031090505 conn(0x55b04f4be680 msgr2=0x55b04f4bea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 -- 172.21.15.52:0/2031090505 shutdown_connections 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 -- 172.21.15.52:0/2031090505 wait complete. 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 -- start start 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.029+0000 7fd1a6765740 1 Processor -- start 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4d1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1aa160 con 0x55b04f4c3a00 2024-06-02T23:12:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a9a90 con 0x55b04f4c8b70 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd19c1a6cd0 con 0x55b04f4c3010 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a6765740 10 client.0 _ll_register_callbacks cb 0x7fd1980bbd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4d1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4d1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57480/0 (socket says 172.21.15.52:57480) 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 -- 172.21.15.52:0/319031237 learned_addr learned my addr 172.21.15.52:0/319031237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a1713640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a1f14640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 -- 172.21.15.52:0/319031237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 msgr2=0x55b04f4d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 -- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 msgr2=0x55b04f4d1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.030+0000 7fd1a0f12640 1 -- 172.21.15.52:0/319031237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b04f541ac0 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd1a0f12640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4d1140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd18c01d040 tx=0x7fd18c01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27558db8267392 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd184ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd184ff9640 1 -- 172.21.15.52:0/319031237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd18c01f040 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd18c01a8c0 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd184ff9640 1 -- 172.21.15.52:0/319031237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd18c004040 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd184ff9640 1 -- 172.21.15.52:0/319031237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd18c01ae10 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd1a6765740 10 client.15678 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b04f542ca0 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.031+0000 7fd184ff9640 1 -- 172.21.15.52:0/319031237 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd18c01a8c0 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd184ff9640 1 -- 172.21.15.52:0/319031237 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fd18c022070 con 0x55b04f4c3a00 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a6765740 10 client.15678 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a6765740 10 client.15678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a6765740 20 client.15678 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a6765740 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f5435d0 0x55b04f5459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a6765740 4 client.15678 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:12:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b04f433c20 con 0x55b04f5435d0 2024-06-02T23:12:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a1713640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f5435d0 0x55b04f5459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd1a1713640 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f5435d0 0x55b04f5459f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f66948b09aa52369 server_cookie=d1be4bed81dfe2c9 in_seq=0 out_seq=0 2024-06-02T23:12:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.032+0000 7fd184ff9640 10 client.15678 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd184ff9640 1 -- 172.21.15.52:0/319031237 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b04f433c20 con 0x55b04f5435d0 2024-06-02T23:12:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd184ff9640 10 client.15678 handle_command_reply: tid=0 2024-06-02T23:12:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 client.15678 shutdown 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 msgr2=0x55b04f4d1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4d1140 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fd18c01d040 tx=0x7fd18c01aca0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 shutdown_connections 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b04f5435d0 0x55b04f5459f0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b04f4c8b70 0x55b04f4d1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd184ff9640 0 client.15678 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b04f4c3a00 0x55b04f4d1140 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 --2- 172.21.15.52:0/319031237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b04f4c3010 0x55b04f4d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 >> 172.21.15.52:0/319031237 conn(0x55b04f4be680 msgr2=0x55b04f4bea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 shutdown_connections 2024-06-02T23:12:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 1 -- 172.21.15.52:0/319031237 wait complete. 2024-06-02T23:12:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.036+0000 7fd1a6765740 20 client.15678 trim_cache size 0 max 16384 2024-06-02T23:12:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:12:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:12:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1880a2cb0 msgr2=0x7fd1880a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1880a2cb0 0x7fd1880a5170 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd194020000 tx=0x7fd19401f000 comp rx=0 tx=0).stop 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 msgr2=0x7fd19c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c197b80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fd1980314e0 tx=0x7fd198004340 comp rx=0 tx=0).stop 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 shutdown_connections 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1880a2cb0 0x7fd1880a5170 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd19c102c80 0x7fd19c1980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd19c103640 0x7fd19c19f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 --2- 172.21.15.52:0/2268806811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd19c0ffcb0 0x7fd19c197b80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 >> 172.21.15.52:0/2268806811 conn(0x7fd19c0fb600 msgr2=0x7fd19c106000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 shutdown_connections 2024-06-02T23:12:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:13.037+0000 7fd1a3977640 1 -- 172.21.15.52:0/2268806811 wait complete. 2024-06-02T23:12:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:12 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:12:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:12 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:12 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:12:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:12 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:13.751 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:13 smithi052 ceph-mon[32340]: pgmap v441: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:14.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:13 smithi123 ceph-mon[35977]: pgmap v441: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:14.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:13 smithi174 ceph-mon[35994]: pgmap v441: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:15.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 -- 172.21.15.52:0/1219519254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 msgr2=0x7f3ea41019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/1219519254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea41019d0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f3e940099c0 tx=0x7f3e94009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 -- 172.21.15.52:0/1219519254 shutdown_connections 2024-06-02T23:12:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/1219519254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea41066c0 0x7f3ea410e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/1219519254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea41019d0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/1219519254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ea4101fa0 0x7f3ea4105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 -- 172.21.15.52:0/1219519254 >> 172.21.15.52:0/1219519254 conn(0x7f3ea4078330 msgr2=0x7f3ea4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 -- 172.21.15.52:0/1219519254 shutdown_connections 2024-06-02T23:12:15.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.430+0000 7f3ea88e0700 1 -- 172.21.15.52:0/1219519254 wait complete. 2024-06-02T23:12:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 Processor -- start 2024-06-02T23:12:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 -- start start 2024-06-02T23:12:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea4103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea4101fa0 0x7f3ea41035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ea41066c0 0x7f3ea4103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41115d0 con 0x7f3ea41015f0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4111450 con 0x7f3ea41066c0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea88e0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea4111750 con 0x7f3ea4101fa0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea4103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea4103080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43766/0 (socket says 172.21.15.52:43766) 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea259c700 1 -- 172.21.15.52:0/709756068 learned_addr learned my addr 172.21.15.52:0/709756068 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea4101fa0 0x7f3ea41035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.431+0000 7f3ea2d9d700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ea41066c0 0x7f3ea4103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea259c700 1 -- 172.21.15.52:0/709756068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea4101fa0 msgr2=0x7f3ea41035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea259c700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea4101fa0 0x7f3ea41035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea259c700 1 -- 172.21.15.52:0/709756068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ea41066c0 msgr2=0x7f3ea4103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea259c700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ea41066c0 0x7f3ea4103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea259c700 1 -- 172.21.15.52:0/709756068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ea41a7100 con 0x7f3ea41015f0 2024-06-02T23:12:15.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea259c700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea4103080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3e937fe700 1 -- 172.21.15.52:0/709756068 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94027070 con 0x7f3ea41015f0 2024-06-02T23:12:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ea41a7390 con 0x7f3ea41015f0 2024-06-02T23:12:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3e937fe700 1 -- 172.21.15.52:0/709756068 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e94003fd0 con 0x7f3ea41015f0 2024-06-02T23:12:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3e937fe700 1 -- 172.21.15.52:0/709756068 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e94022d00 con 0x7f3ea41015f0 2024-06-02T23:12:15.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.432+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ea41a7840 con 0x7f3ea41015f0 2024-06-02T23:12:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.435+0000 7f3e937fe700 1 -- 172.21.15.52:0/709756068 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3e9402a070 con 0x7f3ea41015f0 2024-06-02T23:12:15.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.435+0000 7f3e937fe700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e8c0a2d40 0x7f3e8c0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.436+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e8c0a2d40 0x7f3e8c0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.436+0000 7f3e937fe700 1 -- 172.21.15.52:0/709756068 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3e940cfe10 con 0x7f3ea41015f0 2024-06-02T23:12:15.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.436+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e8c0a2d40 0x7f3e8c0a5200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4107d20 tx=0x7f3e9800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:15.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.436+0000 7f3eb3163b80 1 Processor -- start 2024-06-02T23:12:15.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 -- start start 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22930b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b228cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b228d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b5b21de9a0 con 0x55b5b2292cd0 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5b21868b0 con 0x55b5b22935f0 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5b21da960 con 0x55b5b228d460 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43770/0 (socket says 172.21.15.52:43770) 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 -- 172.21.15.52:0/2138115876 learned_addr learned my addr 172.21.15.52:0/2138115876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b228cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea2d9d700 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b228d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 -- 172.21.15.52:0/2138115876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 msgr2=0x55b5b228d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b228d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 -- 172.21.15.52:0/2138115876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 msgr2=0x55b5b228cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.437+0000 7f3ea259c700 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b228cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.438+0000 7f3ea259c700 1 -- 172.21.15.52:0/2138115876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b217d0e0 con 0x55b5b2292cd0 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.438+0000 7f3ea259c700 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22930b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3e9409c430 tx=0x7f3e94009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241d6f6600840dd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.438+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2138115876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9409c730 con 0x55b5b2292cd0 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.438+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2138115876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e940a0ae0 con 0x55b5b2292cd0 2024-06-02T23:12:15.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.438+0000 7f3e90ff9700 1 -- 172.21.15.52:0/2138115876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9409c990 con 0x55b5b2292cd0 2024-06-02T23:12:15.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2138115876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 msgr2=0x55b5b22930b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22930b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f3e9409c430 tx=0x7f3e94009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2138115876 shutdown_connections 2024-06-02T23:12:15.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b228d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22930b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2138115876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b228cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2138115876 >> 172.21.15.52:0/2138115876 conn(0x55b5b21b7e50 msgr2=0x55b5b22881b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2138115876 shutdown_connections 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2138115876 wait complete. 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 -- start start 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.439+0000 7f3eb3163b80 1 Processor -- start 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 0x55b5b22992d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b22965d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b22935f0 0x55b5b2296b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b5b2299d80 con 0x55b5b228d460 2024-06-02T23:12:15.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5b217ed80 con 0x55b5b2292cd0 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5b2299f00 con 0x55b5b22935f0 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3eb3163b80 10 client.0 _ll_register_callbacks cb 0x55b5b2178ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 0x55b5b22992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 0x55b5b22992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43772/0 (socket says 172.21.15.52:43772) 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 -- 172.21.15.52:0/1807459202 learned_addr learned my addr 172.21.15.52:0/1807459202 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b22965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea2d9d700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b22935f0 0x55b5b2296b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 -- 172.21.15.52:0/1807459202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b22935f0 msgr2=0x55b5b2296b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b22935f0 0x55b5b2296b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 -- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 msgr2=0x55b5b22965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b22965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 -- 172.21.15.52:0/1807459202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b212da60 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3ea259c700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 0x55b5b22992d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3e940a4de0 tx=0x7f3e9409c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3355c278b94ed9ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.440+0000 7f3e89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3e89ffb700 1 -- 172.21.15.52:0/1807459202 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940ba060 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e940a0ae0 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3e89ffb700 1 -- 172.21.15.52:0/1807459202 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3e940870b0 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3e89ffb700 1 -- 172.21.15.52:0/1807459202 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e9409aa30 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 10 client.15690 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5b2299d80 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3e89ffb700 1 -- 172.21.15.52:0/1807459202 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e940a0ae0 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3e89ffb700 1 -- 172.21.15.52:0/1807459202 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3e940b0bc0 con 0x55b5b228d460 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 10 client.15690 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:12:15.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 20 client.15690 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b22a8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 4 client.15690 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.441+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5b21ebea0 con 0x55b5b22a6a80 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.442+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b22a8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.442+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b22a8ec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d8a76a784beb4ee server_cookie=7e2a80cdb8346f35 in_seq=0 out_seq=0 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.442+0000 7f3e89ffb700 10 client.15690 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3e89ffb700 1 -- 172.21.15.52:0/1807459202 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5b21ebea0 con 0x55b5b22a6a80 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3e89ffb700 10 client.15690 handle_command_reply: tid=0 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 client.15690 shutdown 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 msgr2=0x55b5b22992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 0x55b5b22992d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f3e940a4de0 tx=0x7f3e9409c750 comp rx=0 tx=0).stop 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 shutdown_connections 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b22a8ec0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b22935f0 0x55b5b2296b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b228d460 0x55b5b22992d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3e89ffb700 0 client.15690 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/1807459202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b22965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 >> 172.21.15.52:0/1807459202 conn(0x55b5b21b7e50 msgr2=0x55b5b228bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 shutdown_connections 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 1 -- 172.21.15.52:0/1807459202 wait complete. 2024-06-02T23:12:15.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.443+0000 7f3eb3163b80 20 client.15690 trim_cache size 0 max 16384 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 Processor -- start 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 -- start start 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b22940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22ae7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b22aed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41aec40 con 0x55b5b2292cd0 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b0780 con 0x55b5b22935f0 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.468+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41af310 con 0x55b5b228d460 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22ae7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22ae7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43782/0 (socket says 172.21.15.52:43782) 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea1d9b700 1 -- 172.21.15.52:0/365881833 learned_addr learned my addr 172.21.15.52:0/365881833 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b22940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea2d9d700 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b22aed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 -- 172.21.15.52:0/365881833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 msgr2=0x55b5b22aed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b22aed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 -- 172.21.15.52:0/365881833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 msgr2=0x55b5b22ae7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22ae7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 -- 172.21.15.52:0/365881833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5b2294b40 con 0x55b5b228d460 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3ea259c700 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b22940e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e940059d0 tx=0x7f3e94009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48e8b923aa040025 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.469+0000 7f3e8bfff700 1 -- 172.21.15.52:0/365881833 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940ba070 con 0x55b5b228d460 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3e8bfff700 1 -- 172.21.15.52:0/365881833 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e9409bb80 con 0x55b5b228d460 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3e8bfff700 1 -- 172.21.15.52:0/365881833 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e940b0d20 con 0x55b5b228d460 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 -- 172.21.15.52:0/365881833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 msgr2=0x55b5b22940e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b22940e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3e940059d0 tx=0x7f3e94009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 -- 172.21.15.52:0/365881833 shutdown_connections 2024-06-02T23:12:15.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b22935f0 0x55b5b22aed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b22940e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/365881833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b2292cd0 0x55b5b22ae7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 -- 172.21.15.52:0/365881833 >> 172.21.15.52:0/365881833 conn(0x55b5b21b7e50 msgr2=0x55b5b22939d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 -- 172.21.15.52:0/365881833 shutdown_connections 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.470+0000 7f3eb3163b80 1 -- 172.21.15.52:0/365881833 wait complete. 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 -- start start 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 Processor -- start 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b229f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b229c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b22935f0 0x55b5b229c660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41b00b0 con 0x55b5b22935f0 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41af9e0 con 0x55b5b2292cd0 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ea41acc00 con 0x55b5b228d460 2024-06-02T23:12:15.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3eb3163b80 10 client.0 _ll_register_callbacks cb 0x55b5b2178ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3ea1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b229c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3ea1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b229c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35644/0 (socket says 172.21.15.52:35644) 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3ea1d9b700 1 -- 172.21.15.52:0/2274267383 learned_addr learned my addr 172.21.15.52:0/2274267383 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3ea2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b22935f0 0x55b5b229c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.471+0000 7f3ea259c700 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b229f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3ea1d9b700 1 -- 172.21.15.52:0/2274267383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 msgr2=0x55b5b229f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b229f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3ea1d9b700 1 -- 172.21.15.52:0/2274267383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b22935f0 msgr2=0x55b5b229c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b22935f0 0x55b5b229c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3ea1d9b700 1 -- 172.21.15.52:0/2274267383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5b229cea0 con 0x55b5b2292cd0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3ea1d9b700 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b229c120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005950 tx=0x7f3e980047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=851c8b8b15aacb43 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3e89ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3e89ffb700 1 -- 172.21.15.52:0/2274267383 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e98016070 con 0x55b5b2292cd0 2024-06-02T23:12:15.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e9409bb80 con 0x55b5b2292cd0 2024-06-02T23:12:15.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3e89ffb700 1 -- 172.21.15.52:0/2274267383 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3e98028580 con 0x55b5b2292cd0 2024-06-02T23:12:15.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3e89ffb700 1 -- 172.21.15.52:0/2274267383 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3e98028780 con 0x55b5b2292cd0 2024-06-02T23:12:15.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3eb3163b80 10 client.25304 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:15.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.472+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b5b23126d0 con 0x55b5b2292cd0 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3e89ffb700 1 -- 172.21.15.52:0/2274267383 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3e9409bb80 con 0x55b5b2292cd0 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3e89ffb700 1 -- 172.21.15.52:0/2274267383 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3e9802f070 con 0x55b5b2292cd0 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3eb3163b80 10 client.25304 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3eb3163b80 10 client.25304 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3eb3163b80 20 client.25304 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b2314ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3eb3163b80 4 client.25304 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.9, "await_for": 45.0}] 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.9, "await_for": 45.0}) -- 0x55b5b21e6420 con 0x55b5b22a6a80 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.473+0000 7f3ea259c700 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b2314ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.474+0000 7f3e89ffb700 10 client.25304 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:15.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.474+0000 7f3ea259c700 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b2314ef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e452644c29279e4e server_cookie=e4f3b275e2c77372 in_seq=0 out_seq=0 2024-06-02T23:12:15.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3e89ffb700 1 -- 172.21.15.52:0/2274267383 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55b5b21e6420 con 0x55b5b22a6a80 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3e89ffb700 10 client.25304 handle_command_reply: tid=0 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 client.25304 shutdown 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 msgr2=0x55b5b229c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b229c120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3e98005950 tx=0x7f3e980047c0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 shutdown_connections 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5b22a6a80 0x55b5b2314ef0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5b22935f0 0x55b5b229c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5b228d460 0x55b5b229f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 --2- 172.21.15.52:0/2274267383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5b2292cd0 0x55b5b229c120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 >> 172.21.15.52:0/2274267383 conn(0x55b5b21b7e50 msgr2=0x55b5b228ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3e89ffb700 0 client.25304 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:15.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 shutdown_connections 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 1 -- 172.21.15.52:0/2274267383 wait complete. 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.483+0000 7f3eb3163b80 20 client.25304 trim_cache size 0 max 16384 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 28, 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 2, 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout: "33d68cc8": { 2024-06-02T23:12:15.485 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 2, 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 146.9, 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:12:15.486 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:12:15.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e8c0a2d40 msgr2=0x7f3e8c0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e8c0a2d40 0x7f3e8c0a5200 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f3ea4107d20 tx=0x7f3e9800a300 comp rx=0 tx=0).stop 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 msgr2=0x7f3ea4103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea4103080 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009fd0 tx=0x7f3e940047b0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 shutdown_connections 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3e8c0a2d40 0x7f3e8c0a5200 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ea41066c0 0x7f3ea4103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ea41015f0 0x7f3ea4103080 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 --2- 172.21.15.52:0/709756068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ea4101fa0 0x7f3ea41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 >> 172.21.15.52:0/709756068 conn(0x7f3ea4078330 msgr2=0x7f3ea406a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 shutdown_connections 2024-06-02T23:12:15.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:15.484+0000 7f3ea88e0700 1 -- 172.21.15.52:0/709756068 wait complete. 2024-06-02T23:12:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:15 smithi052 ceph-mon[32340]: pgmap v442: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:15 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:12:16.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:15 smithi123 ceph-mon[35977]: pgmap v442: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:15 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:12:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:15 smithi174 ceph-mon[35994]: pgmap v442: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:15 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:12:16.703 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '33d68cc8', quiesce took 9.3 seconds. Will release after: 17.599999999999998 2024-06-02T23:12:17.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 -- 172.21.15.52:0/4177107291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 msgr2=0x7f9d9c102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 --2- 172.21.15.52:0/4177107291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c102960 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009990 tx=0x7f9d8c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 -- 172.21.15.52:0/4177107291 shutdown_connections 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 --2- 172.21.15.52:0/4177107291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c102ea0 0x7f9d9c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 --2- 172.21.15.52:0/4177107291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 0x7f9d9c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 --2- 172.21.15.52:0/4177107291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c102960 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 -- 172.21.15.52:0/4177107291 >> 172.21.15.52:0/4177107291 conn(0x7f9d9c079640 msgr2=0x7f9d9c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 -- 172.21.15.52:0/4177107291 shutdown_connections 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.241+0000 7f9da3844700 1 -- 172.21.15.52:0/4177107291 wait complete. 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 Processor -- start 2024-06-02T23:12:17.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 -- start start 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c1a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c102ea0 0x7f9d9c1a41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 0x7f9d9c10e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c112640 con 0x7f9d9c1024e0 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1124c0 con 0x7f9d9c102ea0 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da3844700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1127c0 con 0x7f9d9c1085f0 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c1a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c1a3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43812/0 (socket says 172.21.15.52:43812) 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 -- 172.21.15.52:0/2989326344 learned_addr learned my addr 172.21.15.52:0/2989326344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c102ea0 0x7f9d9c1a41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da1de1700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 0x7f9d9c10e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 -- 172.21.15.52:0/2989326344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 msgr2=0x7f9d9c10e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 0x7f9d9c10e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 -- 172.21.15.52:0/2989326344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c102ea0 msgr2=0x7f9d9c1a41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.242+0000 7f9da15e0700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c102ea0 0x7f9d9c1a41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9da15e0700 1 -- 172.21.15.52:0/2989326344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9c10ebd0 con 0x7f9d9c1024e0 2024-06-02T23:12:17.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9da1de1700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 0x7f9d9c10e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:17.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9da15e0700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c1a3c90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800c920 tx=0x7f9d9800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:17.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9d927fc700 1 -- 172.21.15.52:0/2989326344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98007b00 con 0x7f9d9c1024e0 2024-06-02T23:12:17.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9c10eec0 con 0x7f9d9c1024e0 2024-06-02T23:12:17.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9d927fc700 1 -- 172.21.15.52:0/2989326344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d98004ce0 con 0x7f9d9c1024e0 2024-06-02T23:12:17.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9d927fc700 1 -- 172.21.15.52:0/2989326344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d98005560 con 0x7f9d9c1024e0 2024-06-02T23:12:17.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.243+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d9c1abbc0 con 0x7f9d9c1024e0 2024-06-02T23:12:17.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.245+0000 7f9d927fc700 1 -- 172.21.15.52:0/2989326344 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d98004750 con 0x7f9d9c1024e0 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.246+0000 7f9d927fc700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d880a2c20 0x7f9d880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.247+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d880a2c20 0x7f9d880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.247+0000 7f9d927fc700 1 -- 172.21.15.52:0/2989326344 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d980cecc0 con 0x7f9d9c1024e0 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.247+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d880a2c20 0x7f9d880a50e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006010 tx=0x7f9d8c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.247+0000 7f9dae0c7b80 1 Processor -- start 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 -- start start 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b6ff73d810 con 0x55b6ff7f1f60 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6ff6e5720 con 0x55b6ff7ebf20 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6ff7397d0 con 0x55b6ff7f1640 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7eb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43814/0 (socket says 172.21.15.52:43814) 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 -- 172.21.15.52:0/2040410507 learned_addr learned my addr 172.21.15.52:0/2040410507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da15e0700 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da1de1700 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 -- 172.21.15.52:0/2040410507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1640 msgr2=0x55b6ff7f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 -- 172.21.15.52:0/2040410507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 msgr2=0x55b6ff7ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da0ddf700 1 -- 172.21.15.52:0/2040410507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ff6dbf50 con 0x55b6ff7f1f60 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.248+0000 7f9da1de1700 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.249+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7eb9e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009990 tx=0x7f9d8c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ed8a1a7c59458 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.249+0000 7f9d877fe700 1 -- 172.21.15.52:0/2040410507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c033070 con 0x55b6ff7f1f60 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.249+0000 7f9d877fe700 1 -- 172.21.15.52:0/2040410507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c00fcf0 con 0x55b6ff7f1f60 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.249+0000 7f9d877fe700 1 -- 172.21.15.52:0/2040410507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c02ec10 con 0x55b6ff7f1f60 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/2040410507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 msgr2=0x55b6ff7eb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7eb9e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009990 tx=0x7f9d8c01e000 comp rx=0 tx=0).stop 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/2040410507 shutdown_connections 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/2040410507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7eb9e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/2040410507 >> 172.21.15.52:0/2040410507 conn(0x55b6ff716cc0 msgr2=0x55b6ff7e6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/2040410507 shutdown_connections 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/2040410507 wait complete. 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 -- start start 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.250+0000 7f9dae0c7b80 1 Processor -- start 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b6ff7f8970 con 0x55b6ff7f1640 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6ff6ddbf0 con 0x55b6ff7ebf20 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b6ff7f8af0 con 0x55b6ff7f1f60 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9dae0c7b80 10 client.0 _ll_register_callbacks cb 0x55b6ff6d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43818/0 (socket says 172.21.15.52:43818) 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 -- 172.21.15.52:0/612140502 learned_addr learned my addr 172.21.15.52:0/612140502 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da15e0700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da1de1700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 -- 172.21.15.52:0/612140502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 msgr2=0x55b6ff7f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 -- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 msgr2=0x55b6ff7f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.251+0000 7f9da0ddf700 1 -- 172.21.15.52:0/612140502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ff68c8d0 con 0x55b6ff7f1640 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f4ff0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0036d0 tx=0x7f9d8c00ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aec13cbd6678e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9d84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9d84ff9700 1 -- 172.21.15.52:0/612140502 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c03e060 con 0x55b6ff7f1640 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c00fcf0 con 0x55b6ff7f1640 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9d84ff9700 1 -- 172.21.15.52:0/612140502 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d8c02e950 con 0x55b6ff7f1640 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9d84ff9700 1 -- 172.21.15.52:0/612140502 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d8c033040 con 0x55b6ff7f1640 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9dae0c7b80 10 client.15708 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6ff8032d0 con 0x55b6ff7f1640 2024-06-02T23:12:17.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.252+0000 7f9d84ff9700 1 -- 172.21.15.52:0/612140502 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c00fcf0 con 0x55b6ff7f1640 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9d84ff9700 1 -- 172.21.15.52:0/612140502 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9d8c045bb0 con 0x55b6ff7f1640 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9dae0c7b80 10 client.15708 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9dae0c7b80 10 client.15708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9dae0c7b80 20 client.15708 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff807b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9dae0c7b80 4 client.15708 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:17.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6ff74ad10 con 0x55b6ff8057a0 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.253+0000 7f9da15e0700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff807b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.254+0000 7f9da15e0700 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff807b90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58200a09de9aafa server_cookie=31693db422ffa803 in_seq=0 out_seq=0 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.254+0000 7f9d84ff9700 10 client.15708 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.254+0000 7f9d84ff9700 1 -- 172.21.15.52:0/612140502 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b6ff74ad10 con 0x55b6ff8057a0 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.254+0000 7f9d84ff9700 10 client.15708 handle_command_reply: tid=0 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.254+0000 7f9dae0c7b80 1 client.15708 shutdown 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 msgr2=0x55b6ff7f4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f4ff0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c0036d0 tx=0x7f9d8c00ff10 comp rx=0 tx=0).stop 2024-06-02T23:12:17.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 shutdown_connections 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff807b90 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9d84ff9700 0 client.15708 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/612140502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f4ff0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 >> 172.21.15.52:0/612140502 conn(0x55b6ff716cc0 msgr2=0x55b6ff7ea9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 shutdown_connections 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/612140502 wait complete. 2024-06-02T23:12:17.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.255+0000 7f9dae0c7b80 20 client.15708 trim_cache size 0 max 16384 2024-06-02T23:12:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 Processor -- start 2024-06-02T23:12:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 -- start start 2024-06-02T23:12:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1b2fe0 con 0x55b6ff7f1640 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1b4b20 con 0x55b6ff7f1f60 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1b36b0 con 0x55b6ff7ebf20 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43824/0 (socket says 172.21.15.52:43824) 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 -- 172.21.15.52:0/1570087014 learned_addr learned my addr 172.21.15.52:0/1570087014 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da15e0700 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da1de1700 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 -- 172.21.15.52:0/1570087014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7ebf20 msgr2=0x55b6ff7eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 -- 172.21.15.52:0/1570087014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1f60 msgr2=0x55b6ff7f2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da0ddf700 1 -- 172.21.15.52:0/1570087014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6ff7f3560 con 0x55b6ff7f1640 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.277+0000 7f9da1de1700 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.278+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f3f50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c02e560 tx=0x7f9d8c02e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa567a781daee341 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.278+0000 7f9d86ffd700 1 -- 172.21.15.52:0/1570087014 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c03e070 con 0x55b6ff7f1640 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.278+0000 7f9d86ffd700 1 -- 172.21.15.52:0/1570087014 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c0094b0 con 0x55b6ff7f1640 2024-06-02T23:12:17.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.278+0000 7f9d86ffd700 1 -- 172.21.15.52:0/1570087014 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8c045480 con 0x55b6ff7f1640 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/1570087014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 msgr2=0x55b6ff7f3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f3f50 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c02e560 tx=0x7f9d8c02e820 comp rx=0 tx=0).stop 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/1570087014 shutdown_connections 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1f60 0x55b6ff7f2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7ebf20 0x55b6ff7eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/1570087014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7f1640 0x55b6ff7f3f50 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/1570087014 >> 172.21.15.52:0/1570087014 conn(0x55b6ff716cc0 msgr2=0x55b6ff7f2400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/1570087014 shutdown_connections 2024-06-02T23:12:17.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/1570087014 wait complete. 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 -- start start 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.279+0000 7f9dae0c7b80 1 Processor -- start 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 0x55b6ff80ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1640 0x55b6ff80b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff8090a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1b4450 con 0x55b6ff7ebf20 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1b3d80 con 0x55b6ff7f1640 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d9c1b0fa0 con 0x55b6ff7f1f60 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9dae0c7b80 10 client.0 _ll_register_callbacks cb 0x55b6ff6d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:17.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 0x55b6ff80ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 0x55b6ff80ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43838/0 (socket says 172.21.15.52:43838) 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 -- 172.21.15.52:0/3862185759 learned_addr learned my addr 172.21.15.52:0/3862185759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da1de1700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff8090a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1640 0x55b6ff80b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 -- 172.21.15.52:0/3862185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 msgr2=0x55b6ff8090a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff8090a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 -- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1640 msgr2=0x55b6ff80b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1640 0x55b6ff80b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da15e0700 1 -- 172.21.15.52:0/3862185759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6ff8097c0 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da1de1700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff8090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.280+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1640 0x55b6ff80b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9da15e0700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 0x55b6ff80ab80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9d980a3c90 tx=0x7f9d98086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d0a484f67fa5764 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9d84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9d84ff9700 1 -- 172.21.15.52:0/3862185759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d9801d040 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9d84ff9700 1 -- 172.21.15.52:0/3862185759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d980995f0 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d8c0094b0 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9d84ff9700 1 -- 172.21.15.52:0/3862185759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9d9809f900 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9dae0c7b80 10 client.15714 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b6ff870b80 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.281+0000 7f9d84ff9700 1 -- 172.21.15.52:0/3862185759 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d8c0094b0 con 0x55b6ff7ebf20 2024-06-02T23:12:17.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9d84ff9700 1 -- 172.21.15.52:0/3862185759 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9d98084040 con 0x55b6ff7ebf20 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9dae0c7b80 10 client.15714 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9dae0c7b80 10 client.15714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9dae0c7b80 20 client.15714 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff873240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9dae0c7b80 4 client.15714 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6ff745290 con 0x55b6ff8057a0 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.282+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff873240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9da0ddf700 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff873240 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dad492570858431a server_cookie=3963bfe2a2075ec9 in_seq=0 out_seq=0 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9d84ff9700 10 client.15714 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9d84ff9700 1 -- 172.21.15.52:0/3862185759 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6ff745290 con 0x55b6ff8057a0 2024-06-02T23:12:17.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9d84ff9700 10 client.15714 handle_command_reply: tid=0 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 client.15714 shutdown 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 msgr2=0x55b6ff80ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 0x55b6ff80ab80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9d980a3c90 tx=0x7f9d98086040 comp rx=0 tx=0).stop 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 shutdown_connections 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b6ff8057a0 0x55b6ff873240 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b6ff7f1f60 0x55b6ff8090a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b6ff7ebf20 0x55b6ff80ab80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 --2- 172.21.15.52:0/3862185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6ff7f1640 0x55b6ff80b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 >> 172.21.15.52:0/3862185759 conn(0x55b6ff716cc0 msgr2=0x55b6ff7dfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9d84ff9700 0 client.15714 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:17.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 shutdown_connections 2024-06-02T23:12:17.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 1 -- 172.21.15.52:0/3862185759 wait complete. 2024-06-02T23:12:17.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.283+0000 7f9dae0c7b80 20 client.15714 trim_cache size 0 max 16384 2024-06-02T23:12:17.291 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:12:17.291 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:12:17.291 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:12:17.291 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:12:17.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d880a2c20 msgr2=0x7f9d880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d880a2c20 0x7f9d880a50e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c006010 tx=0x7f9d8c01a040 comp rx=0 tx=0).stop 2024-06-02T23:12:17.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 msgr2=0x7f9d9c1a3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:17.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c1a3c90 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9d9800c920 tx=0x7f9d9800cce0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 shutdown_connections 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d880a2c20 0x7f9d880a50e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d9c1085f0 0x7f9d9c10e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d9c1024e0 0x7f9d9c1a3c90 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 --2- 172.21.15.52:0/2989326344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d9c102ea0 0x7f9d9c1a41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 >> 172.21.15.52:0/2989326344 conn(0x7f9d9c079640 msgr2=0x7f9d9c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 shutdown_connections 2024-06-02T23:12:17.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:17.284+0000 7f9da3844700 1 -- 172.21.15.52:0/2989326344 wait complete. 2024-06-02T23:12:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:17 smithi052 ceph-mon[32340]: pgmap v443: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-06-02T23:12:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:17 smithi123 ceph-mon[35977]: pgmap v443: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-06-02T23:12:18.076 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:12:18.076 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:12:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:17 smithi174 ceph-mon[35994]: pgmap v443: 97 pgs: 97 active+clean; 669 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 10 op/s 2024-06-02T23:12:18.375 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:19 smithi052 ceph-mon[32340]: pgmap v444: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-02T23:12:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:19 smithi123 ceph-mon[35977]: pgmap v444: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-02T23:12:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:19 smithi174 ceph-mon[35994]: pgmap v444: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 -- 172.21.15.52:0/2307233143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 msgr2=0x7f2e441088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/2307233143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 0x7f2e441088e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2e34009990 tx=0x7f2e34009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 -- 172.21.15.52:0/2307233143 shutdown_connections 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/2307233143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406ab30 0x7f2e4410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/2307233143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 0x7f2e441088e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/2307233143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4406a170 0x7f2e4406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.224+0000 7f2e49c0d700 1 -- 172.21.15.52:0/2307233143 >> 172.21.15.52:0/2307233143 conn(0x7f2e44078340 msgr2=0x7f2e44078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.225+0000 7f2e49c0d700 1 -- 172.21.15.52:0/2307233143 shutdown_connections 2024-06-02T23:12:20.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.225+0000 7f2e49c0d700 1 -- 172.21.15.52:0/2307233143 wait complete. 2024-06-02T23:12:20.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.225+0000 7f2e49c0d700 1 Processor -- start 2024-06-02T23:12:20.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 -- start start 2024-06-02T23:12:20.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 0x7f2e44103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:20.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4406ab30 0x7f2e44101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 0x7f2e44101ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112610 con 0x7f2e44108500 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112490 con 0x7f2e4406a170 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e49c0d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44112790 con 0x7f2e4406ab30 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e437fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 0x7f2e44103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e43fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 0x7f2e44101ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:20.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e437fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 0x7f2e44103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35724/0 (socket says 172.21.15.52:35724) 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e437fe700 1 -- 172.21.15.52:0/180207439 learned_addr learned my addr 172.21.15.52:0/180207439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.226+0000 7f2e42ffd700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4406ab30 0x7f2e44101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e437fe700 1 -- 172.21.15.52:0/180207439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4406ab30 msgr2=0x7f2e44101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e437fe700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4406ab30 0x7f2e44101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e437fe700 1 -- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 msgr2=0x7f2e44101ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e437fe700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 0x7f2e44101ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e437fe700 1 -- 172.21.15.52:0/180207439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e441ab510 con 0x7f2e4406a170 2024-06-02T23:12:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e437fe700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 0x7f2e44103050 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000c00 tx=0x7f2e34003a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34027070 con 0x7f2e4406a170 2024-06-02T23:12:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e441ab740 con 0x7f2e4406a170 2024-06-02T23:12:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e3400bdc0 con 0x7f2e4406a170 2024-06-02T23:12:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34022cd0 con 0x7f2e4406a170 2024-06-02T23:12:20.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.227+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441abb90 con 0x7f2e4406a170 2024-06-02T23:12:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.229+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4410a9e0 con 0x7f2e4406a170 2024-06-02T23:12:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.230+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e3402a070 con 0x7f2e4406a170 2024-06-02T23:12:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.230+0000 7f2e40ff9700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e2c0a2c20 0x7f2e2c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:20.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.230+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2e340cfd20 con 0x7f2e4406a170 2024-06-02T23:12:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.230+0000 7f2e42ffd700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e2c0a2c20 0x7f2e2c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:20.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.231+0000 7f2e42ffd700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e2c0a2c20 0x7f2e2c0a50e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e44102bd0 tx=0x7f2e3800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:20.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.235+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2e34013070 con 0x7f2e4406a170 2024-06-02T23:12:20.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.584+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2e4410a3c0 con 0x7f2e4406a170 2024-06-02T23:12:20.586 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:12:20.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.586+0000 7f2e40ff9700 1 -- 172.21.15.52:0/180207439 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f2e34098030 con 0x7f2e4406a170 2024-06-02T23:12:20.587 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:12:20.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e2c0a2c20 msgr2=0x7f2e2c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:20.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e2c0a2c20 0x7f2e2c0a50e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2e44102bd0 tx=0x7f2e3800b410 comp rx=0 tx=0).stop 2024-06-02T23:12:20.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 msgr2=0x7f2e44103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:20.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 0x7f2e44103050 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2e34000c00 tx=0x7f2e34003a80 comp rx=0 tx=0).stop 2024-06-02T23:12:20.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 shutdown_connections 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e2c0a2c20 0x7f2e2c0a50e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e44108500 0x7f2e44101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4406a170 0x7f2e44103050 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 --2- 172.21.15.52:0/180207439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e4406ab30 0x7f2e44101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 >> 172.21.15.52:0/180207439 conn(0x7f2e44078340 msgr2=0x7f2e44100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 shutdown_connections 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:20.589+0000 7f2e49c0d700 1 -- 172.21.15.52:0/180207439 wait complete. 2024-06-02T23:12:20.590 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:12:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:20 smithi052 ceph-mon[32340]: pgmap v445: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:20.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:20 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/180207439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:21.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:20 smithi123 ceph-mon[35977]: pgmap v445: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:21.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:20 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/180207439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:20 smithi174 ceph-mon[35994]: pgmap v445: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:20 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/180207439' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:21.335 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:12:21.641 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:22 smithi174 ceph-mon[35994]: pgmap v446: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:23.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:22 smithi052 ceph-mon[32340]: pgmap v446: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:23.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:22 smithi123 ceph-mon[35977]: pgmap v446: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:23.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 -- 172.21.15.52:0/2689964566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 msgr2=0x7f6d6006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 --2- 172.21.15.52:0/2689964566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6006a5f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009990 tx=0x7f6d50009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 -- 172.21.15.52:0/2689964566 shutdown_connections 2024-06-02T23:12:23.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 --2- 172.21.15.52:0/2689964566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 0x7f6d6010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 --2- 172.21.15.52:0/2689964566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d60108500 0x7f6d601088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 --2- 172.21.15.52:0/2689964566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6006a5f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 -- 172.21.15.52:0/2689964566 >> 172.21.15.52:0/2689964566 conn(0x7f6d60078340 msgr2=0x7f6d60078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 -- 172.21.15.52:0/2689964566 shutdown_connections 2024-06-02T23:12:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.527+0000 7f6d6733d700 1 -- 172.21.15.52:0/2689964566 wait complete. 2024-06-02T23:12:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 Processor -- start 2024-06-02T23:12:23.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 -- start start 2024-06-02T23:12:23.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 0x7f6d6019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d60108500 0x7f6d60198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d60112660 con 0x7f6d6006a170 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601124e0 con 0x7f6d60108500 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d650d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d6733d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601127e0 con 0x7f6d6006ab30 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d650d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43880/0 (socket says 172.21.15.52:43880) 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d650d9700 1 -- 172.21.15.52:0/1511839508 learned_addr learned my addr 172.21.15.52:0/1511839508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.528+0000 7f6d658da700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d60108500 0x7f6d60198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d648d8700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 0x7f6d6019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d650d9700 1 -- 172.21.15.52:0/1511839508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 msgr2=0x7f6d6019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d650d9700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 0x7f6d6019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d650d9700 1 -- 172.21.15.52:0/1511839508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d60108500 msgr2=0x7f6d60198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d650d9700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d60108500 0x7f6d60198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d650d9700 1 -- 172.21.15.52:0/1511839508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d601990c0 con 0x7f6d6006a170 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d648d8700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 0x7f6d6019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d650d9700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6019e7a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ed60 tx=0x7f6d5c00c5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d567fc700 1 -- 172.21.15.52:0/1511839508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00cbc0 con 0x7f6d6006a170 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d567fc700 1 -- 172.21.15.52:0/1511839508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d5c00cd60 con 0x7f6d6006a170 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d567fc700 1 -- 172.21.15.52:0/1511839508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c018710 con 0x7f6d6006a170 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.529+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d601993b0 con 0x7f6d6006a170 2024-06-02T23:12:23.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.530+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d601a5670 con 0x7f6d6006a170 2024-06-02T23:12:23.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.532+0000 7f6d567fc700 1 -- 172.21.15.52:0/1511839508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6d5c010430 con 0x7f6d6006a170 2024-06-02T23:12:23.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.533+0000 7f6d567fc700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d4c0a2ca0 0x7f6d4c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.533+0000 7f6d567fc700 1 -- 172.21.15.52:0/1511839508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6d5c0cf270 con 0x7f6d6006a170 2024-06-02T23:12:23.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.533+0000 7f6d648d8700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d4c0a2ca0 0x7f6d4c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.534+0000 7f6d648d8700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d4c0a2ca0 0x7f6d4c0a5160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000b2e0 tx=0x7f6d5001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:23.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 Processor -- start 2024-06-02T23:12:23.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 -- start start 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a5a70 0x5648172a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 0x56481729fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5648171ed810 con 0x5648172a6390 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564817195720 con 0x5648172a0330 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5648171e97d0 con 0x5648172a5a70 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d648d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 0x56481729fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d648d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 0x56481729fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43888/0 (socket says 172.21.15.52:43888) 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d648d8700 1 -- 172.21.15.52:0/3137746884 learned_addr learned my addr 172.21.15.52:0/3137746884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d658da700 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.535+0000 7f6d650d9700 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a5a70 0x5648172a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d648d8700 1 -- 172.21.15.52:0/3137746884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a5a70 msgr2=0x5648172a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d648d8700 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a5a70 0x5648172a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d648d8700 1 -- 172.21.15.52:0/3137746884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 msgr2=0x5648172a0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d648d8700 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172a0750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d648d8700 1 -- 172.21.15.52:0/3137746884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481718bf50 con 0x5648172a6390 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d648d8700 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 0x56481729fdf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000bed0 tx=0x7f6d5001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c1f0c8ca48e798 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.536+0000 7f6d4b7fe700 1 -- 172.21.15.52:0/3137746884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000fb50 con 0x5648172a6390 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d4b7fe700 1 -- 172.21.15.52:0/3137746884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d5000fcf0 con 0x5648172a6390 2024-06-02T23:12:23.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d4b7fe700 1 -- 172.21.15.52:0/3137746884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d50004b90 con 0x5648172a6390 2024-06-02T23:12:23.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/3137746884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 msgr2=0x56481729fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 0x56481729fdf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000bed0 tx=0x7f6d5001e000 comp rx=0 tx=0).stop 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/3137746884 shutdown_connections 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172a0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a5a70 0x5648172a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/3137746884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a6390 0x56481729fdf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.537+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/3137746884 >> 172.21.15.52:0/3137746884 conn(0x5648171c6cc0 msgr2=0x564817296850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.538+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/3137746884 shutdown_connections 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.538+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/3137746884 wait complete. 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.538+0000 7f6d71bc0b80 1 -- start start 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.538+0000 7f6d71bc0b80 1 Processor -- start 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172ac180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172a9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5648172acd70 con 0x5648172a0330 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56481718dbf0 con 0x5648172a5a70 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5648172acef0 con 0x5648172a6390 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d71bc0b80 10 client.0 _ll_register_callbacks cb 0x564817187920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43904/0 (socket says 172.21.15.52:43904) 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 -- 172.21.15.52:0/443324321 learned_addr learned my addr 172.21.15.52:0/443324321 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d648d8700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d658da700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 -- 172.21.15.52:0/443324321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 msgr2=0x5648172a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172a99c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 -- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 msgr2=0x5648172a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.539+0000 7f6d650d9700 1 -- 172.21.15.52:0/443324321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481713c8d0 con 0x5648172a0330 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d648d8700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d650d9700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172ac180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c000c00 tx=0x7f6d5c09bfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2991d389a36a026 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d48ff9700 1 -- 172.21.15.52:0/443324321 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c09c1a0 con 0x5648172a0330 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d48ff9700 1 -- 172.21.15.52:0/443324321 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6d5c09aef0 con 0x5648172a0330 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d5000fcf0 con 0x5648172a0330 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d48ff9700 1 -- 172.21.15.52:0/443324321 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c09baa0 con 0x5648172a0330 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d71bc0b80 10 client.15732 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:23.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.540+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5648172b7720 con 0x5648172a0330 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.541+0000 7f6d48ff9700 1 -- 172.21.15.52:0/443324321 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d5c0b4070 con 0x5648172a0330 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d48ff9700 1 -- 172.21.15.52:0/443324321 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d5c09a960 con 0x5648172a0330 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d71bc0b80 10 client.15732 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d71bc0b80 10 client.15732 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d71bc0b80 20 client.15732 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5648172b9d60 0x5648172bc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d71bc0b80 4 client.15732 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6d5000fcf0 con 0x5648172b9d60 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.542+0000 7f6d648d8700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5648172b9d60 0x5648172bc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.543+0000 7f6d48ff9700 10 client.15732 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:23.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.543+0000 7f6d648d8700 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5648172b9d60 0x5648172bc150 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=350068b787b4e535 server_cookie=81ef0232dd39d19 in_seq=0 out_seq=0 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d48ff9700 1 -- 172.21.15.52:0/443324321 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f6d5000fcf0 con 0x5648172b9d60 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d48ff9700 10 client.15732 handle_command_reply: tid=0 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 client.15732 shutdown 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 msgr2=0x5648172ac180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172ac180 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c000c00 tx=0x7f6d5c09bfe0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 shutdown_connections 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172ac180 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5648172b9d60 0x5648172bc150 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/443324321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 >> 172.21.15.52:0/443324321 conn(0x5648171c6cc0 msgr2=0x56481729a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.544+0000 7f6d48ff9700 0 client.15732 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.545+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 shutdown_connections 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.545+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/443324321 wait complete. 2024-06-02T23:12:23.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.545+0000 7f6d71bc0b80 20 client.15732 trim_cache size 0 max 16384 2024-06-02T23:12:23.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 Processor -- start 2024-06-02T23:12:23.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 -- start start 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172b6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 0x5648172c1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601aa6d0 con 0x5648172a5a70 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ac210 con 0x5648172a0330 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601aada0 con 0x5648172a6390 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d648d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 0x5648172c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d648d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 0x5648172c1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43906/0 (socket says 172.21.15.52:43906) 2024-06-02T23:12:23.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.560+0000 7f6d648d8700 1 -- 172.21.15.52:0/4021093129 learned_addr learned my addr 172.21.15.52:0/4021093129 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d648d8700 1 -- 172.21.15.52:0/4021093129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 msgr2=0x5648172c1e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:12:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d648d8700 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c1e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d648d8700 1 -- 172.21.15.52:0/4021093129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 msgr2=0x5648172b6010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d648d8700 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172b6010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d648d8700 1 -- 172.21.15.52:0/4021093129 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648172a79f0 con 0x5648172a5a70 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d648d8700 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 0x5648172c1950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6d50004700 tx=0x7f6d50027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6cdaa3695b9ee5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d4affd700 1 -- 172.21.15.52:0/4021093129 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5000fb50 con 0x5648172a5a70 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d4affd700 1 -- 172.21.15.52:0/4021093129 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d5000fcf0 con 0x5648172a5a70 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.561+0000 7f6d4affd700 1 -- 172.21.15.52:0/4021093129 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5002f6c0 con 0x5648172a5a70 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/4021093129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 msgr2=0x5648172c1950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 0x5648172c1950 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6d50004700 tx=0x7f6d50027040 comp rx=0 tx=0).stop 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/4021093129 shutdown_connections 2024-06-02T23:12:23.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a0330 0x5648172b6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/4021093129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a5a70 0x5648172c1950 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/4021093129 >> 172.21.15.52:0/4021093129 conn(0x5648171c6cc0 msgr2=0x5648172a7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:23.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/4021093129 shutdown_connections 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.562+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/4021093129 wait complete. 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 -- start start 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 Processor -- start 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601abb40 con 0x5648172a0330 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601ab470 con 0x5648172a5a70 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d601a8720 con 0x5648172a6390 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.563+0000 7f6d71bc0b80 10 client.0 _ll_register_callbacks cb 0x564817187920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:23.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d650d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d650d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172b2360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43910/0 (socket says 172.21.15.52:43910) 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d650d9700 1 -- 172.21.15.52:0/626760328 learned_addr learned my addr 172.21.15.52:0/626760328 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d648d8700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 -- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 msgr2=0x5648172b28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172b28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 -- 172.21.15.52:0/626760328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 msgr2=0x5648172b2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172b2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 -- 172.21.15.52:0/626760328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648172c14e0 con 0x5648172a6390 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d650d9700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:23.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d648d8700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:23.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d658da700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c0d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d58015e80 tx=0x7f6d58015eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3b79b25e77a1085 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:23.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.564+0000 7f6d48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d48ff9700 1 -- 172.21.15.52:0/626760328 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d58013aa0 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d48ff9700 1 -- 172.21.15.52:0/626760328 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6d58013c40 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d5000fcf0 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d48ff9700 1 -- 172.21.15.52:0/626760328 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6d58013e40 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d71bc0b80 10 client.25279 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56481729ba70 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.565+0000 7f6d48ff9700 1 -- 172.21.15.52:0/626760328 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6d58004bb0 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d48ff9700 1 -- 172.21.15.52:0/626760328 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6d58004640 con 0x5648172a6390 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d71bc0b80 10 client.25279 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d71bc0b80 10 client.25279 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d71bc0b80 20 client.25279 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56481729c1e0 0x56481729e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:23.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d71bc0b80 4 client.25279 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f6d5000fcf0 con 0x56481729c1e0 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.566+0000 7f6d650d9700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56481729c1e0 0x56481729e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d650d9700 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56481729c1e0 0x56481729e600 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=16690089b5a11e88 server_cookie=ab14c35f93fd14c8 in_seq=0 out_seq=0 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d48ff9700 10 client.25279 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d48ff9700 1 -- 172.21.15.52:0/626760328 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f6d5000fcf0 con 0x56481729c1e0 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d48ff9700 10 client.25279 handle_command_reply: tid=0 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 client.25279 shutdown 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 msgr2=0x5648172c0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c0d40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6d58015e80 tx=0x7f6d58015eb0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 shutdown_connections 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5648172a6390 0x5648172c0d40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5648172a0330 0x5648172b2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56481729c1e0 0x56481729e600 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 --2- 172.21.15.52:0/626760328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5648172a5a70 0x5648172b28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 >> 172.21.15.52:0/626760328 conn(0x5648171c6cc0 msgr2=0x56481728fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d48ff9700 0 client.25279 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 shutdown_connections 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 1 -- 172.21.15.52:0/626760328 wait complete. 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.567+0000 7f6d71bc0b80 20 client.25279 trim_cache size 0 max 16384 2024-06-02T23:12:23.571 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d4c0a2ca0 msgr2=0x7f6d4c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d4c0a2ca0 0x7f6d4c0a5160 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6d5000b2e0 tx=0x7f6d5001a040 comp rx=0 tx=0).stop 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 msgr2=0x7f6d6019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6019e7a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c00ed60 tx=0x7f6d5c00c5b0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 shutdown_connections 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6d4c0a2ca0 0x7f6d4c0a5160 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d60108500 0x7f6d60198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6d6006a170 0x7f6d6019e7a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 --2- 172.21.15.52:0/1511839508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d6006ab30 0x7f6d6019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:23.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.568+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 >> 172.21.15.52:0/1511839508 conn(0x7f6d60078340 msgr2=0x7f6d60100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:23.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.569+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 shutdown_connections 2024-06-02T23:12:23.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:23.569+0000 7f6d6733d700 1 -- 172.21.15.52:0/1511839508 wait complete. 2024-06-02T23:12:24.372 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:12:24.753 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:25.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:24 smithi174 ceph-mon[35994]: pgmap v447: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:25.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:24 smithi052 ceph-mon[32340]: pgmap v447: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:25.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:24 smithi123 ceph-mon[35977]: pgmap v447: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.129+0000 7f9438c76640 1 -- 172.21.15.52:0/2961027441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9434102480 msgr2=0x7f9434104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.129+0000 7f9438c76640 1 --2- 172.21.15.52:0/2961027441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9434102480 0x7f9434104870 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9424009900 tx=0x7f942402a410 comp rx=0 tx=0).stop 2024-06-02T23:12:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 -- 172.21.15.52:0/2961027441 shutdown_connections 2024-06-02T23:12:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 --2- 172.21.15.52:0/2961027441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9434104db0 0x7f94341071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 --2- 172.21.15.52:0/2961027441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f94341076e0 0x7f9434109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 --2- 172.21.15.52:0/2961027441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9434102480 0x7f9434104870 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 -- 172.21.15.52:0/2961027441 >> 172.21.15.52:0/2961027441 conn(0x7f94340fbe70 msgr2=0x7f94340fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 -- 172.21.15.52:0/2961027441 shutdown_connections 2024-06-02T23:12:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 -- 172.21.15.52:0/2961027441 wait complete. 2024-06-02T23:12:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.130+0000 7f9438c76640 1 Processor -- start 2024-06-02T23:12:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 -- start start 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 0x7f9434197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9434104db0 0x7f9434198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94341076e0 0x7f943419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f943410d560 con 0x7f94341076e0 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f943410d3e0 con 0x7f9434102480 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9438c76640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f943410d6e0 con 0x7f9434104db0 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f943259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 0x7f9434197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f943259c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 0x7f9434197bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57820/0 (socket says 172.21.15.52:57820) 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f943259c640 1 -- 172.21.15.52:0/2543594678 learned_addr learned my addr 172.21.15.52:0/2543594678 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9432d9d640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94341076e0 0x7f943419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f9431d9b640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9434104db0 0x7f9434198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f943259c640 1 -- 172.21.15.52:0/2543594678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9434104db0 msgr2=0x7f9434198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f943259c640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9434104db0 0x7f9434198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.131+0000 7f943259c640 1 -- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94341076e0 msgr2=0x7f943419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.132+0000 7f943259c640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94341076e0 0x7f943419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.132+0000 7f943259c640 1 -- 172.21.15.52:0/2543594678 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f943419f7f0 con 0x7f9434102480 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.132+0000 7f9432d9d640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94341076e0 0x7f943419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.132+0000 7f943259c640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 0x7f9434197bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9424005940 tx=0x7f94240042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.132+0000 7f94237fe640 1 -- 172.21.15.52:0/2543594678 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942403f030 con 0x7f9434102480 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.132+0000 7f94237fe640 1 -- 172.21.15.52:0/2543594678 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94240318f0 con 0x7f9434102480 2024-06-02T23:12:26.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.133+0000 7f94237fe640 1 -- 172.21.15.52:0/2543594678 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9424035070 con 0x7f9434102480 2024-06-02T23:12:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.133+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f943419fa80 con 0x7f9434102480 2024-06-02T23:12:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.133+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f943419fef0 con 0x7f9434102480 2024-06-02T23:12:26.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.135+0000 7f943b987740 1 Processor -- start 2024-06-02T23:12:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 -- start start 2024-06-02T23:12:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8a00 0x5596afdcd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5596afd2b4f0 con 0x5596afdc8a00 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596afcc9870 con 0x5596afdc8010 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943b987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5596afd274b0 con 0x5596afdcdb70 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f94237fe640 1 -- 172.21.15.52:0/2543594678 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9424042030 con 0x7f9434102480 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f9431d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f943259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8a00 0x5596afdcd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f9432d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f9431d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43248/0 (socket says 172.21.15.52:43248) 2024-06-02T23:12:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.136+0000 7f9431d9b640 1 -- 172.21.15.52:0/3788570208 learned_addr learned my addr 172.21.15.52:0/3788570208 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f943259c640 1 -- 172.21.15.52:0/3788570208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 msgr2=0x5596afdd0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f943259c640 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f943259c640 1 -- 172.21.15.52:0/3788570208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 msgr2=0x5596afdc8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f943259c640 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f943259c640 1 -- 172.21.15.52:0/3788570208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596afce59b0 con 0x5596afdc8a00 2024-06-02T23:12:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f943259c640 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8a00 0x5596afdcd500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f94240ecd90 tx=0x7f94240f7540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7066f9872fab8a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f9420ff9640 1 -- 172.21.15.52:0/3788570208 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240f14e0 con 0x5596afdc8a00 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f9420ff9640 1 -- 172.21.15.52:0/3788570208 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94240fc030 con 0x5596afdc8a00 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.137+0000 7f9420ff9640 1 -- 172.21.15.52:0/3788570208 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94241003d0 con 0x5596afdc8a00 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.138+0000 7f94237fe640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f941c0a2b20 0x7f941c0a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.138+0000 7f9432d9d640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f941c0a2b20 0x7f941c0a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.138+0000 7f94237fe640 1 -- 172.21.15.52:0/2543594678 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f94240ef290 con 0x7f9434102480 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.138+0000 7f943b987740 1 -- 172.21.15.52:0/3788570208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8a00 msgr2=0x5596afdcd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.138+0000 7f943b987740 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8a00 0x5596afdcd500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f94240ecd90 tx=0x7f94240f7540 comp rx=0 tx=0).stop 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.138+0000 7f9432d9d640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f941c0a2b20 0x7f941c0a4fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5596afdc9260 tx=0x7f942c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- 172.21.15.52:0/3788570208 shutdown_connections 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8a00 0x5596afdcd500 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 --2- 172.21.15.52:0/3788570208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- 172.21.15.52:0/3788570208 >> 172.21.15.52:0/3788570208 conn(0x5596afdc3680 msgr2=0x5596afdc3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- 172.21.15.52:0/3788570208 shutdown_connections 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- 172.21.15.52:0/3788570208 wait complete. 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- start start 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 Processor -- start 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5596afd2b4f0 con 0x5596afdc8010 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5596afcb4ae0 con 0x5596afdc8a00 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.139+0000 7f943b987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5596afdd08e0 con 0x5596afdcdb70 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36114/0 (socket says 172.21.15.52:36114) 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 -- 172.21.15.52:0/2401511278 learned_addr learned my addr 172.21.15.52:0/2401511278 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943b987740 10 client.0 _ll_register_callbacks cb 0x5596afced9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9431d9b640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9432d9d640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 -- 172.21.15.52:0/2401511278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 msgr2=0x5596afdd93a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 -- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 msgr2=0x5596afdd2360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd2360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 -- 172.21.15.52:0/2401511278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596afa3f210 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9431d9b640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943259c640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd1e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f94240b84f0 tx=0x7f942403b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed93e4355ac618f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9419ffb640 1 -- 172.21.15.52:0/2401511278 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240b7dc0 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9419ffb640 1 -- 172.21.15.52:0/2401511278 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f94240b9e20 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94240fc030 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.140+0000 7f9419ffb640 1 -- 172.21.15.52:0/2401511278 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94240044e0 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 10 client.15750 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596afb0b620 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f9419ffb640 1 -- 172.21.15.52:0/2401511278 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f94240fc030 con 0x5596afdc8010 2024-06-02T23:12:26.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f9419ffb640 1 -- 172.21.15.52:0/2401511278 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f94240b8190 con 0x5596afdc8010 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 10 client.15750 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 20 client.15750 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afddd8e0 0x5596afddfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 4 client.15750 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596afd3ef10 con 0x5596afddd8e0 2024-06-02T23:12:26.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.141+0000 7f9431d9b640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afddd8e0 0x5596afddfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.142+0000 7f9419ffb640 10 client.15750 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.142+0000 7f9431d9b640 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afddd8e0 0x5596afddfcd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eace0ebdd11efe43 server_cookie=41d4024884750c94 in_seq=0 out_seq=0 2024-06-02T23:12:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f9419ffb640 1 -- 172.21.15.52:0/2401511278 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5596afd3ef10 con 0x5596afddd8e0 2024-06-02T23:12:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f9419ffb640 10 client.15750 handle_command_reply: tid=0 2024-06-02T23:12:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 client.15750 shutdown 2024-06-02T23:12:26.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 msgr2=0x5596afdd1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd1e20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f94240b84f0 tx=0x7f942403b460 comp rx=0 tx=0).stop 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 shutdown_connections 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afddd8e0 0x5596afddfcd0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afdd93a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd2360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 --2- 172.21.15.52:0/2401511278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd1e20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 >> 172.21.15.52:0/2401511278 conn(0x5596afdc3680 msgr2=0x5596afdc27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f9419ffb640 0 client.15750 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 shutdown_connections 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 1 -- 172.21.15.52:0/2401511278 wait complete. 2024-06-02T23:12:26.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.143+0000 7f943b987740 20 client.15750 trim_cache size 0 max 16384 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 Processor -- start 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 -- start start 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdc8a00 0x5596afdd9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 0x5596afdd1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94341a8a70 con 0x5596afdcdb70 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94341aa5b0 con 0x5596afdc8010 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f943b987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94341a9140 con 0x5596afdc8a00 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f9432d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 0x5596afdd1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f9432d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 0x5596afdd1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36122/0 (socket says 172.21.15.52:36122) 2024-06-02T23:12:26.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f9431d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdc8a00 0x5596afdd9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.174+0000 7f9432d9d640 1 -- 172.21.15.52:0/1934737053 learned_addr learned my addr 172.21.15.52:0/1934737053 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f943259c640 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9432d9d640 1 -- 172.21.15.52:0/1934737053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdc8a00 msgr2=0x5596afdd9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9432d9d640 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdc8a00 0x5596afdd9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9432d9d640 1 -- 172.21.15.52:0/1934737053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 msgr2=0x5596afdc6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9432d9d640 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9432d9d640 1 -- 172.21.15.52:0/1934737053 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596afde0330 con 0x5596afdcdb70 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9431d9b640 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdc8a00 0x5596afdd9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f9432d9d640 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 0x5596afdd1440 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f942c0123d0 tx=0x7f942c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=245dee5f94226eb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f941bfff640 1 -- 172.21.15.52:0/1934737053 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c02d070 con 0x5596afdcdb70 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f941bfff640 1 -- 172.21.15.52:0/1934737053 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f942c011df0 con 0x5596afdcdb70 2024-06-02T23:12:26.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.175+0000 7f941bfff640 1 -- 172.21.15.52:0/1934737053 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c028850 con 0x5596afdcdb70 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 -- 172.21.15.52:0/1934737053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 msgr2=0x5596afdd1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 0x5596afdd1440 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f942c0123d0 tx=0x7f942c017000 comp rx=0 tx=0).stop 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 -- 172.21.15.52:0/1934737053 shutdown_connections 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdc8a00 0x5596afdd9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdcdb70 0x5596afdd1440 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 --2- 172.21.15.52:0/1934737053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8010 0x5596afdc6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.176+0000 7f943b987740 1 -- 172.21.15.52:0/1934737053 >> 172.21.15.52:0/1934737053 conn(0x5596afdc3680 msgr2=0x5596afdc6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.177+0000 7f943b987740 1 -- 172.21.15.52:0/1934737053 shutdown_connections 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.177+0000 7f943b987740 1 -- 172.21.15.52:0/1934737053 wait complete. 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.178+0000 7f943b987740 1 -- start start 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.178+0000 7f943b987740 1 Processor -- start 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.178+0000 7f943b987740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.178+0000 7f943b987740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.178+0000 7f943b987740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afe46580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943b987740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f94341a9ee0 con 0x5596afdc8010 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943b987740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f94341a9810 con 0x5596afdc8a00 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943b987740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f94341a6a50 con 0x5596afdcdb70 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36138/0 (socket says 172.21.15.52:36138) 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 -- 172.21.15.52:0/1829791804 learned_addr learned my addr 172.21.15.52:0/1829791804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943b987740 10 client.0 _ll_register_callbacks cb 0x7f942c009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f9431d9b640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f9432d9d640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afe46580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 -- 172.21.15.52:0/1829791804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 msgr2=0x5596afe46580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afe46580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 -- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 msgr2=0x5596afdd6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 -- 172.21.15.52:0/1829791804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596afb0b620 con 0x5596afdc8010 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f9431d9b640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.179+0000 7f943259c640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd6340 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9424000c00 tx=0x7f94240bbba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b3d87381c36bb1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f9419ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f9419ffb640 1 -- 172.21.15.52:0/1829791804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94240d8070 con 0x5596afdc8010 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f9419ffb640 1 -- 172.21.15.52:0/1829791804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f94240b8190 con 0x5596afdc8010 2024-06-02T23:12:26.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f942c011df0 con 0x5596afdc8010 2024-06-02T23:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f9419ffb640 1 -- 172.21.15.52:0/1829791804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94240bb480 con 0x5596afdc8010 2024-06-02T23:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f943b987740 10 client.15756 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.180+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5596afdc34b0 con 0x5596afdc8010 2024-06-02T23:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f9419ffb640 1 -- 172.21.15.52:0/1829791804 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f942c011df0 con 0x5596afdc8010 2024-06-02T23:12:26.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f9419ffb640 1 -- 172.21.15.52:0/1829791804 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f94240bbd80 con 0x5596afdc8010 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f943b987740 10 client.15756 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f943b987740 10 client.15756 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f943b987740 20 client.15756 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f943b987740 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afe481b0 0x5596afe4a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f943b987740 4 client.15756 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596afd38c20 con 0x5596afe481b0 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.181+0000 7f9431d9b640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afe481b0 0x5596afe4a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.182+0000 7f9419ffb640 10 client.15756 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:26.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.182+0000 7f9431d9b640 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afe481b0 0x5596afe4a5d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89893c8cfbaa0061 server_cookie=833ef6da28873b2f in_seq=0 out_seq=0 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.185+0000 7f9419ffb640 1 -- 172.21.15.52:0/1829791804 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5596afd38c20 con 0x5596afe481b0 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.185+0000 7f9419ffb640 10 client.15756 handle_command_reply: tid=0 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.185+0000 7f943b987740 1 client.15756 shutdown 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 msgr2=0x5596afdd6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd6340 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9424000c00 tx=0x7f94240bbba0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 shutdown_connections 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5596afe481b0 0x5596afe4a5d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f9419ffb640 0 client.15756 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5596afdcdb70 0x5596afe46580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596afdc8a00 0x5596afdd6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 --2- 172.21.15.52:0/1829791804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5596afdc8010 0x5596afdd6340 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 >> 172.21.15.52:0/1829791804 conn(0x5596afdc3680 msgr2=0x5596afdc1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 shutdown_connections 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 1 -- 172.21.15.52:0/1829791804 wait complete. 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.186+0000 7f943b987740 20 client.15756 trim_cache size 0 max 16384 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:12:26.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:12:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f941c0a2b20 msgr2=0x7f941c0a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f941c0a2b20 0x7f941c0a4fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x5596afdc9260 tx=0x7f942c009500 comp rx=0 tx=0).stop 2024-06-02T23:12:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 msgr2=0x7f9434197bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 0x7f9434197bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9424005940 tx=0x7f94240042f0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 shutdown_connections 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f941c0a2b20 0x7f941c0a4fe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9434104db0 0x7f9434198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f94341076e0 0x7f943419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 --2- 172.21.15.52:0/2543594678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9434102480 0x7f9434197bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 >> 172.21.15.52:0/2543594678 conn(0x7f94340fbe70 msgr2=0x7f9434102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 shutdown_connections 2024-06-02T23:12:26.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:26.187+0000 7f9438c76640 1 -- 172.21.15.52:0/2543594678 wait complete. 2024-06-02T23:12:26.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 -- 172.21.15.52:0/3729240207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 msgr2=0x7f33d806b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 --2- 172.21.15.52:0/3729240207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d806b500 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f33cc009990 tx=0x7f33cc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 -- 172.21.15.52:0/3729240207 shutdown_connections 2024-06-02T23:12:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 --2- 172.21.15.52:0/3729240207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d810ede0 0x7f33d81111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 --2- 172.21.15.52:0/3729240207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d81033a0 0x7f33d806ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 --2- 172.21.15.52:0/3729240207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d806b500 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.584+0000 7f33dd996700 1 -- 172.21.15.52:0/3729240207 >> 172.21.15.52:0/3729240207 conn(0x7f33d8078340 msgr2=0x7f33d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.585+0000 7f33dd996700 1 -- 172.21.15.52:0/3729240207 shutdown_connections 2024-06-02T23:12:26.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.585+0000 7f33dd996700 1 -- 172.21.15.52:0/3729240207 wait complete. 2024-06-02T23:12:26.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 Processor -- start 2024-06-02T23:12:26.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 -- start start 2024-06-02T23:12:26.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d819ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 0x7f33d819b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d810ede0 0x7f33d819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33d8114850 con 0x7f33d806b0a0 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81146d0 con 0x7f33d81033a0 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33dd996700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81149d0 con 0x7f33d810ede0 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33d6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d819ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33d67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 0x7f33d819b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33d67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 0x7f33d819b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57862/0 (socket says 172.21.15.52:57862) 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33d67fc700 1 -- 172.21.15.52:0/280216806 learned_addr learned my addr 172.21.15.52:0/280216806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.586+0000 7f33d6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d819ad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36154/0 (socket says 172.21.15.52:36154) 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d77fe700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d810ede0 0x7f33d819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d67fc700 1 -- 172.21.15.52:0/280216806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d810ede0 msgr2=0x7f33d819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d67fc700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d810ede0 0x7f33d819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d67fc700 1 -- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 msgr2=0x7f33d819ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d67fc700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d819ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d67fc700 1 -- 172.21.15.52:0/280216806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d819fd70 con 0x7f33d81033a0 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33d67fc700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 0x7f33d819b2a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0057c0 tx=0x7f33cc0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33dc994700 1 -- 172.21.15.52:0/280216806 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc02f070 con 0x7f33d81033a0 2024-06-02T23:12:26.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d81a0000 con 0x7f33d81033a0 2024-06-02T23:12:26.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d8103980 con 0x7f33d81033a0 2024-06-02T23:12:26.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.587+0000 7f33dc994700 1 -- 172.21.15.52:0/280216806 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33cc02a470 con 0x7f33d81033a0 2024-06-02T23:12:26.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.588+0000 7f33dc994700 1 -- 172.21.15.52:0/280216806 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc00e640 con 0x7f33d81033a0 2024-06-02T23:12:26.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.591+0000 7f33dc994700 1 -- 172.21.15.52:0/280216806 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33cc032070 con 0x7f33d81033a0 2024-06-02T23:12:26.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.592+0000 7f33dc994700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33c00a2d20 0x7f33c00a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.592+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33c00a2d20 0x7f33c00a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.592+0000 7f33e8219b80 1 Processor -- start 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.592+0000 7f33e8219b80 1 -- start start 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f39a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f34340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33dc994700 1 -- 172.21.15.52:0/280216806 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33cc0d8d00 con 0x7f33d81033a0 2024-06-02T23:12:26.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33e8219b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5598b1e85810 con 0x5598b1f39640 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33c00a2d20 0x7f33c00a51e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33c8007fd0 tx=0x7f33c80073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33e8219b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5598b1e2d720 con 0x5598b1f33f20 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33e8219b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5598b1e817d0 con 0x5598b1f39f60 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f34340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33d77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f34340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57864/0 (socket says 172.21.15.52:57864) 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33d77fe700 1 -- 172.21.15.52:0/237911788 learned_addr learned my addr 172.21.15.52:0/237911788 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f39a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.593+0000 7f33d67fc700 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d77fe700 1 -- 172.21.15.52:0/237911788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 msgr2=0x5598b1f339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d77fe700 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f339e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d77fe700 1 -- 172.21.15.52:0/237911788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 msgr2=0x5598b1f39a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d77fe700 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f39a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d77fe700 1 -- 172.21.15.52:0/237911788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b1e23f50 con 0x5598b1f33f20 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33d77fe700 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f34340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33d00093e0 tx=0x7f33d0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c4facccdc74fb671 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.594+0000 7f33bdffb700 1 -- 172.21.15.52:0/237911788 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0019030 con 0x5598b1f33f20 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.595+0000 7f33bdffb700 1 -- 172.21.15.52:0/237911788 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33d00158f0 con 0x5598b1f33f20 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.595+0000 7f33bdffb700 1 -- 172.21.15.52:0/237911788 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0014860 con 0x5598b1f33f20 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 -- 172.21.15.52:0/237911788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 msgr2=0x5598b1f34340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f34340 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f33d00093e0 tx=0x7f33d0009920 comp rx=0 tx=0).stop 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 -- 172.21.15.52:0/237911788 shutdown_connections 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f34340 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f39a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 --2- 172.21.15.52:0/237911788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 -- 172.21.15.52:0/237911788 >> 172.21.15.52:0/237911788 conn(0x5598b1e5ecc0 msgr2=0x5598b1f2e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 -- 172.21.15.52:0/237911788 shutdown_connections 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.596+0000 7f33e8219b80 1 -- 172.21.15.52:0/237911788 wait complete. 2024-06-02T23:12:26.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.597+0000 7f33e8219b80 1 -- start start 2024-06-02T23:12:26.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.597+0000 7f33e8219b80 1 Processor -- start 2024-06-02T23:12:26.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 0x5598b1f3fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f39640 0x5598b1f3cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f3d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33d6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 0x5598b1f3fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5598b1f409d0 con 0x5598b1f33f20 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5598b1e25bf0 con 0x5598b1f39640 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5598b1f40b50 con 0x5598b1f39f60 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33d6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 0x5598b1f3fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36170/0 (socket says 172.21.15.52:36170) 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33d6ffd700 1 -- 172.21.15.52:0/3633766769 learned_addr learned my addr 172.21.15.52:0/3633766769 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33e8219b80 10 client.0 _ll_register_callbacks cb 0x5598b1e1f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33d77fe700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f3d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.598+0000 7f33d67fc700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f39640 0x5598b1f3cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d6ffd700 1 -- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 msgr2=0x5598b1f3d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f3d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d6ffd700 1 -- 172.21.15.52:0/3633766769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f39640 msgr2=0x5598b1f3cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f39640 0x5598b1f3cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d6ffd700 1 -- 172.21.15.52:0/3633766769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b1dd48d0 con 0x5598b1f33f20 2024-06-02T23:12:26.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d67fc700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f39640 0x5598b1f3cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 0x5598b1f3fce0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f33c8003910 tx=0x7f33c80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca532d990c8091e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3633766769 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8024070 con 0x5598b1f33f20 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.599+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33d00158f0 con 0x5598b1f33f20 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.600+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3633766769 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33c8011640 con 0x5598b1f33f20 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.600+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3633766769 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33c8011880 con 0x5598b1f33f20 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.600+0000 7f33e8219b80 10 client.15768 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.600+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598b1f4b1b0 con 0x5598b1f33f20 2024-06-02T23:12:26.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.600+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3633766769 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33d00158f0 con 0x5598b1f33f20 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3633766769 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f33c8020a70 con 0x5598b1f33f20 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33e8219b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33e8219b80 10 client.15768 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33e8219b80 20 client.15768 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1f4dab0 0x5598b1f4fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33e8219b80 4 client.15768 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.601+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598b1e92d10 con 0x5598b1f4dab0 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.602+0000 7f33d67fc700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1f4dab0 0x5598b1f4fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.602+0000 7f33d67fc700 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1f4dab0 0x5598b1f4fef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5de98f7fa9f361ee server_cookie=d867c4a3a7a1aea in_seq=0 out_seq=0 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.602+0000 7f33b6ffd700 10 client.15768 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:12:26.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3633766769 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5598b1e92d10 con 0x5598b1f4dab0 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33b6ffd700 10 client.15768 handle_command_reply: tid=0 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33e8219b80 1 client.15768 shutdown 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 msgr2=0x5598b1f3fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 0x5598b1f3fce0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f33c8003910 tx=0x7f33c80047c0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 shutdown_connections 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.603+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1f4dab0 0x5598b1f4fef0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f3d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33b6ffd700 0 client.15768 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f33f20 0x5598b1f3fce0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3633766769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f39640 0x5598b1f3cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 >> 172.21.15.52:0/3633766769 conn(0x5598b1e5ecc0 msgr2=0x5598b1f329d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 shutdown_connections 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 1 -- 172.21.15.52:0/3633766769 wait complete. 2024-06-02T23:12:26.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.604+0000 7f33e8219b80 20 client.15768 trim_cache size 0 max 16384 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.619+0000 7f33e8219b80 1 Processor -- start 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.619+0000 7f33e8219b80 1 -- start start 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.619+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f339e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.619+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f28510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f55800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33e8219b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81aa600 con 0x5598b1f39640 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33e8219b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81ac140 con 0x5598b1f33f20 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33e8219b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81aacd0 con 0x5598b1f39f60 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f28510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f28510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36186/0 (socket says 172.21.15.52:36186) 2024-06-02T23:12:26.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 -- 172.21.15.52:0/2586828771 learned_addr learned my addr 172.21.15.52:0/2586828771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d77fe700 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f55800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 -- 172.21.15.52:0/2586828771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 msgr2=0x5598b1f55800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f339e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f55800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 -- 172.21.15.52:0/2586828771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 msgr2=0x5598b1f339e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f339e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d67fc700 1 -- 172.21.15.52:0/2586828771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b1f2f500 con 0x5598b1f39640 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.620+0000 7f33d77fe700 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f55800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:26.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.621+0000 7f33d67fc700 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f28510 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f33cc005cd0 tx=0x7f33cc0d66a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed2ec8ff809d8bc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.621+0000 7f33bd7fa700 1 -- 172.21.15.52:0/2586828771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0c5040 con 0x5598b1f39640 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.621+0000 7f33bd7fa700 1 -- 172.21.15.52:0/2586828771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33cc0a38c0 con 0x5598b1f39640 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.621+0000 7f33bd7fa700 1 -- 172.21.15.52:0/2586828771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33cc0a5a80 con 0x5598b1f39640 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 -- 172.21.15.52:0/2586828771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 msgr2=0x5598b1f28510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f28510 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f33cc005cd0 tx=0x7f33cc0d66a0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 -- 172.21.15.52:0/2586828771 shutdown_connections 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f39f60 0x5598b1f55800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f339e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 --2- 172.21.15.52:0/2586828771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39640 0x5598b1f28510 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 -- 172.21.15.52:0/2586828771 >> 172.21.15.52:0/2586828771 conn(0x5598b1e5ecc0 msgr2=0x5598b1f27e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 -- 172.21.15.52:0/2586828771 shutdown_connections 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 -- 172.21.15.52:0/2586828771 wait complete. 2024-06-02T23:12:26.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.622+0000 7f33e8219b80 1 -- start start 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 Processor -- start 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f463e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39f60 0x5598b1f416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 0x5598b1f41c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81aba70 con 0x5598b1f39f60 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81ab3a0 con 0x5598b1f33f20 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33d81078f0 con 0x5598b1f4dab0 2024-06-02T23:12:26.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33e8219b80 10 client.0 _ll_register_callbacks cb 0x5598b1e1f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:26.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33d77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 0x5598b1f41c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.623+0000 7f33d77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 0x5598b1f41c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43330/0 (socket says 172.21.15.52:43330) 2024-06-02T23:12:26.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 -- 172.21.15.52:0/3556908417 learned_addr learned my addr 172.21.15.52:0/3556908417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f463e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d67fc700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39f60 0x5598b1f416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 -- 172.21.15.52:0/3556908417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 msgr2=0x5598b1f463e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f463e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 -- 172.21.15.52:0/3556908417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39f60 msgr2=0x5598b1f416c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39f60 0x5598b1f416c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 -- 172.21.15.52:0/3556908417 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b1f42240 con 0x5598b1f4dab0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d67fc700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39f60 0x5598b1f416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33d77fe700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 0x5598b1f41c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33d00093e0 tx=0x7f33d0016680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cea8614d667a2d5f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.624+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3556908417 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33d0016aa0 con 0x5598b1f4dab0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3556908417 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33d0016c40 con 0x5598b1f4dab0 2024-06-02T23:12:26.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33cc0a38c0 con 0x5598b1f4dab0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3556908417 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f33d002e070 con 0x5598b1f4dab0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 10 client.25312 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5598b1fb8640 con 0x5598b1f4dab0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3556908417 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33cc0a38c0 con 0x5598b1f4dab0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3556908417 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f33d0005040 con 0x5598b1f4dab0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 10 client.25312 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 10 client.25312 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 20 client.25312 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1fb89f0 0x5598b1fbae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 4 client.25312 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.625+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598b1e8d290 con 0x5598b1fb89f0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.626+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1fb89f0 0x5598b1fbae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.626+0000 7f33d6ffd700 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1fb89f0 0x5598b1fbae10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=90028589eb129727 server_cookie=25fb2ee6839c2c63 in_seq=0 out_seq=0 2024-06-02T23:12:26.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.626+0000 7f33b6ffd700 10 client.25312 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:12:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33b6ffd700 1 -- 172.21.15.52:0/3556908417 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598b1e8d290 con 0x5598b1fb89f0 2024-06-02T23:12:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33b6ffd700 10 client.25312 handle_command_reply: tid=0 2024-06-02T23:12:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 client.25312 shutdown 2024-06-02T23:12:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 msgr2=0x5598b1f41c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 0x5598b1f41c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f33d00093e0 tx=0x7f33d0016680 comp rx=0 tx=0).stop 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 shutdown_connections 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5598b1fb89f0 0x5598b1fbae10 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5598b1f4dab0 0x5598b1f41c00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5598b1f33f20 0x5598b1f463e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 --2- 172.21.15.52:0/3556908417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5598b1f39f60 0x5598b1f416c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 >> 172.21.15.52:0/3556908417 conn(0x5598b1e5ecc0 msgr2=0x5598b1f3e810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33b6ffd700 0 client.25312 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 shutdown_connections 2024-06-02T23:12:26.630 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 1 -- 172.21.15.52:0/3556908417 wait complete. 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.627+0000 7f33e8219b80 20 client.25312 trim_cache size 0 max 16384 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33c00a2d20 msgr2=0x7f33c00a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33c00a2d20 0x7f33c00a51e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f33c8007fd0 tx=0x7f33c80073d0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 msgr2=0x7f33d819b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 0x7f33d819b2a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f33cc0057c0 tx=0x7f33cc0057f0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 shutdown_connections 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33c00a2d20 0x7f33c00a51e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d810ede0 0x7f33d819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d806b0a0 0x7f33d819ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 --2- 172.21.15.52:0/280216806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d81033a0 0x7f33d819b2a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 >> 172.21.15.52:0/280216806 conn(0x7f33d8078340 msgr2=0x7f33d8100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 shutdown_connections 2024-06-02T23:12:26.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:26.628+0000 7f33dd996700 1 -- 172.21.15.52:0/280216806 wait complete. 2024-06-02T23:12:27.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:26 smithi174 ceph-mon[35994]: pgmap v448: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:27.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:26 smithi052 ceph-mon[32340]: pgmap v448: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:27.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:26 smithi123 ceph-mon[35977]: pgmap v448: 97 pgs: 97 active+clean; 670 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:27.499 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:12:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:29.136 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:28 smithi052 ceph-mon[32340]: pgmap v449: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-06-02T23:12:29.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:28 smithi123 ceph-mon[35977]: pgmap v449: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-06-02T23:12:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:28 smithi174 ceph-mon[35994]: pgmap v449: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-06-02T23:12:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.798+0000 7f7293fff700 1 -- 172.21.15.52:0/3164345890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72940092e0 con 0x7f729c101d40 2024-06-02T23:12:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 -- 172.21.15.52:0/3164345890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c101d40 msgr2=0x7f729c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 --2- 172.21.15.52:0/3164345890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c101d40 0x7f729c10e230 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f729400b0a0 tx=0x7f729400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 -- 172.21.15.52:0/3164345890 shutdown_connections 2024-06-02T23:12:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 --2- 172.21.15.52:0/3164345890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c101d40 0x7f729c10e230 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 --2- 172.21.15.52:0/3164345890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c1073f0 0x7f729c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 --2- 172.21.15.52:0/3164345890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f729c101380 0x7f729c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 -- 172.21.15.52:0/3164345890 >> 172.21.15.52:0/3164345890 conn(0x7f729c078300 msgr2=0x7f729c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 -- 172.21.15.52:0/3164345890 shutdown_connections 2024-06-02T23:12:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.799+0000 7f72a3667700 1 -- 172.21.15.52:0/3164345890 wait complete. 2024-06-02T23:12:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.800+0000 7f72a3667700 1 Processor -- start 2024-06-02T23:12:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.800+0000 7f72a3667700 1 -- start start 2024-06-02T23:12:29.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.800+0000 7f72a3667700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 0x7f729c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a3667700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f729c101d40 0x7f729c197c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a3667700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c1073f0 0x7f729c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a3667700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f729c111510 con 0x7f729c1073f0 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a3667700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f729c111390 con 0x7f729c101380 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a3667700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f729c111690 con 0x7f729c101d40 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 0x7f729c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 0x7f729c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57918/0 (socket says 172.21.15.52:57918) 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 -- 172.21.15.52:0/2515787103 learned_addr learned my addr 172.21.15.52:0/2515787103 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a0c02700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f729c101d40 0x7f729c197c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 -- 172.21.15.52:0/2515787103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f729c101d40 msgr2=0x7f729c197c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f729c101d40 0x7f729c197c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 -- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c1073f0 msgr2=0x7f729c19c770 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1c04700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c1073f0 0x7f729c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c1073f0 0x7f729c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.801+0000 7f72a1403700 1 -- 172.21.15.52:0/2515787103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f729c19ccb0 con 0x7f729c101380 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72a1c04700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c1073f0 0x7f729c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72a1403700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 0x7f729c197720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9a0 tx=0x7f729800ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72927fc700 1 -- 172.21.15.52:0/2515787103 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729800cbd0 con 0x7f729c101380 2024-06-02T23:12:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72927fc700 1 -- 172.21.15.52:0/2515787103 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f729800cd70 con 0x7f729c101380 2024-06-02T23:12:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72927fc700 1 -- 172.21.15.52:0/2515787103 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7298018750 con 0x7f729c101380 2024-06-02T23:12:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f729c19cfa0 con 0x7f729c101380 2024-06-02T23:12:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.802+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f729c19d3d0 con 0x7f729c101380 2024-06-02T23:12:29.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.806+0000 7f72927fc700 1 -- 172.21.15.52:0/2515787103 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7298010430 con 0x7f729c101380 2024-06-02T23:12:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.807+0000 7f72927fc700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72880a2ca0 0x7f72880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.807+0000 7f72927fc700 1 -- 172.21.15.52:0/2515787103 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f72980cf240 con 0x7f729c101380 2024-06-02T23:12:29.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.807+0000 7f72a0c02700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72880a2ca0 0x7f72880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.807+0000 7f72adeeab80 1 Processor -- start 2024-06-02T23:12:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 -- start start 2024-06-02T23:12:29.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72a0c02700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72880a2ca0 0x7f72880a5160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f729c078900 tx=0x7f728c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f2880 0x558e764ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764ecbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558e763def60 con 0x558e764f2880 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558e7643d850 con 0x558e764f1f60 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72adeeab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e763a5460 con 0x558e764ec7a0 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72a1403700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72a1403700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57924/0 (socket says 172.21.15.52:57924) 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.808+0000 7f72a1403700 1 -- 172.21.15.52:0/868321683 learned_addr learned my addr 172.21.15.52:0/868321683 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:29.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a0c02700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f2880 0x558e764ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a1c04700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764ecbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a0c02700 1 -- 172.21.15.52:0/868321683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 msgr2=0x558e764ecbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a0c02700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764ecbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a0c02700 1 -- 172.21.15.52:0/868321683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 msgr2=0x558e764f2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a0c02700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a0c02700 1 -- 172.21.15.52:0/868321683 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e7638c8d0 con 0x558e764f2880 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a1403700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.809+0000 7f72a1c04700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.810+0000 7f72a0c02700 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f2880 0x558e764ec260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f729c06b570 tx=0x7f728c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606bb6a07d985565 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.810+0000 7f72877fe700 1 -- 172.21.15.52:0/868321683 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c012b60 con 0x558e764f2880 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.810+0000 7f72877fe700 1 -- 172.21.15.52:0/868321683 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f728c0044d0 con 0x558e764f2880 2024-06-02T23:12:29.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.810+0000 7f72877fe700 1 -- 172.21.15.52:0/868321683 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c01f420 con 0x558e764f2880 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 -- 172.21.15.52:0/868321683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f2880 msgr2=0x558e764ec260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f2880 0x558e764ec260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f729c06b570 tx=0x7f728c014000 comp rx=0 tx=0).stop 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 -- 172.21.15.52:0/868321683 shutdown_connections 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764ecbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 --2- 172.21.15.52:0/868321683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f2880 0x558e764ec260 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 -- 172.21.15.52:0/868321683 >> 172.21.15.52:0/868321683 conn(0x558e76419080 msgr2=0x558e764e7220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.811+0000 7f72adeeab80 1 -- 172.21.15.52:0/868321683 shutdown_connections 2024-06-02T23:12:29.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.812+0000 7f72adeeab80 1 -- 172.21.15.52:0/868321683 wait complete. 2024-06-02T23:12:29.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.812+0000 7f72adeeab80 1 -- start start 2024-06-02T23:12:29.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.812+0000 7f72adeeab80 1 Processor -- start 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764f8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 0x558e764f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558e763ddfc0 con 0x558e764f6100 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558e76439810 con 0x558e764f1f60 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558e764f9260 con 0x558e764ec7a0 2024-06-02T23:12:29.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72adeeab80 10 client.0 _ll_register_callbacks cb 0x558e763d35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 0x558e764f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 0x558e764f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36228/0 (socket says 172.21.15.52:36228) 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 -- 172.21.15.52:0/3309138867 learned_addr learned my addr 172.21.15.52:0/3309138867 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1403700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a0c02700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 -- 172.21.15.52:0/3309138867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 msgr2=0x558e764f8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764f8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 -- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 msgr2=0x558e764f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1c04700 1 -- 172.21.15.52:0/3309138867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e76403750 con 0x558e764f6100 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a1403700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.813+0000 7f72a0c02700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f72a1c04700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 0x558e764f6580 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7294015000 tx=0x7f7294003cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8180d6f65d7b0f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:29.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f7284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:29.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f7284ff9700 1 -- 172.21.15.52:0/3309138867 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7294021070 con 0x558e764f6100 2024-06-02T23:12:29.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f7284ff9700 1 -- 172.21.15.52:0/3309138867 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f72940076d0 con 0x558e764f6100 2024-06-02T23:12:29.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f728c0044d0 con 0x558e764f6100 2024-06-02T23:12:29.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f7284ff9700 1 -- 172.21.15.52:0/3309138867 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f729402d070 con 0x558e764f6100 2024-06-02T23:12:29.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f72adeeab80 10 client.15783 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:29.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e76504050 con 0x558e764f6100 2024-06-02T23:12:29.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.814+0000 7f7284ff9700 1 -- 172.21.15.52:0/3309138867 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f728c0044d0 con 0x558e764f6100 2024-06-02T23:12:29.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f7284ff9700 1 -- 172.21.15.52:0/3309138867 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f7294011070 con 0x558e764f6100 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72adeeab80 10 client.15783 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72adeeab80 10 client.15783 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72adeeab80 20 client.15783 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76508ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72adeeab80 4 client.15783 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e7644ad50 con 0x558e765066e0 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.815+0000 7f72a1403700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76508ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.816+0000 7f7284ff9700 10 client.15783 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.816+0000 7f72a1403700 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76508ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c228f41714c1ab4c server_cookie=66b9ef22571f3771 in_seq=0 out_seq=0 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.816+0000 7f7284ff9700 1 -- 172.21.15.52:0/3309138867 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558e7644ad50 con 0x558e765066e0 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.816+0000 7f7284ff9700 10 client.15783 handle_command_reply: tid=0 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.816+0000 7f72adeeab80 1 client.15783 shutdown 2024-06-02T23:12:29.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 msgr2=0x558e764f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 0x558e764f6580 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f7294015000 tx=0x7f7294003cb0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 shutdown_connections 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76508ad0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f7284ff9700 0 client.15783 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f6100 0x558e764f6580 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764ec7a0 0x558e764f8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3309138867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f1f60 0x558e764f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 >> 172.21.15.52:0/3309138867 conn(0x558e76419080 msgr2=0x558e764e6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 shutdown_connections 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 1 -- 172.21.15.52:0/3309138867 wait complete. 2024-06-02T23:12:29.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.817+0000 7f72adeeab80 20 client.15783 trim_cache size 0 max 16384 2024-06-02T23:12:29.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 Processor -- start 2024-06-02T23:12:29.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 -- start start 2024-06-02T23:12:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764ec7a0 0x558e764ec260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 0x558e764f5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f6100 0x558e764f13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b3040 con 0x558e764ec7a0 2024-06-02T23:12:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b4b80 con 0x558e764f6100 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.833+0000 7f72adeeab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b3710 con 0x558e764f1f60 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a1403700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764ec7a0 0x558e764ec260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 0x558e764f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 0x558e764f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43384/0 (socket says 172.21.15.52:43384) 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 -- 172.21.15.52:0/3742633713 learned_addr learned my addr 172.21.15.52:0/3742633713 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a1c04700 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f6100 0x558e764f13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 -- 172.21.15.52:0/3742633713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f6100 msgr2=0x558e764f13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f6100 0x558e764f13e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 -- 172.21.15.52:0/3742633713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764ec7a0 msgr2=0x558e764ec260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764ec7a0 0x558e764ec260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a0c02700 1 -- 172.21.15.52:0/3742633713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e764f40a0 con 0x558e764f1f60 2024-06-02T23:12:29.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.834+0000 7f72a1403700 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764ec7a0 0x558e764ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:29.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.835+0000 7f72a0c02700 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 0x558e764f5470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f728c011d00 tx=0x7f728c009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3098c9c13b68c9ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:29.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.835+0000 7f7286ffd700 1 -- 172.21.15.52:0/3742633713 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c003900 con 0x558e764f1f60 2024-06-02T23:12:29.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.835+0000 7f7286ffd700 1 -- 172.21.15.52:0/3742633713 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f728c012e10 con 0x558e764f1f60 2024-06-02T23:12:29.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.835+0000 7f7286ffd700 1 -- 172.21.15.52:0/3742633713 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c027440 con 0x558e764f1f60 2024-06-02T23:12:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 -- 172.21.15.52:0/3742633713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 msgr2=0x558e764f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 0x558e764f5470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f728c011d00 tx=0x7f728c009640 comp rx=0 tx=0).stop 2024-06-02T23:12:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 -- 172.21.15.52:0/3742633713 shutdown_connections 2024-06-02T23:12:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764f6100 0x558e764f13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764ec7a0 0x558e764ec260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 --2- 172.21.15.52:0/3742633713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f1f60 0x558e764f5470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.836+0000 7f72adeeab80 1 -- 172.21.15.52:0/3742633713 >> 172.21.15.52:0/3742633713 conn(0x558e76419080 msgr2=0x558e764f0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.837+0000 7f72adeeab80 1 -- 172.21.15.52:0/3742633713 shutdown_connections 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.837+0000 7f72adeeab80 1 -- 172.21.15.52:0/3742633713 wait complete. 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.837+0000 7f72adeeab80 1 -- start start 2024-06-02T23:12:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.837+0000 7f72adeeab80 1 Processor -- start 2024-06-02T23:12:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764ec7a0 0x558e764fe700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 0x558e764fec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f6100 0x558e7650d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b44b0 con 0x558e764f1f60 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b3de0 con 0x558e764ec7a0 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f729c1b1000 con 0x558e764f6100 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72adeeab80 10 client.0 _ll_register_callbacks cb 0x558e765091f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a0c02700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 0x558e764fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a0c02700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 0x558e764fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36248/0 (socket says 172.21.15.52:36248) 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a0c02700 1 -- 172.21.15.52:0/2176056553 learned_addr learned my addr 172.21.15.52:0/2176056553 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a1c04700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f6100 0x558e7650d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a1403700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764ec7a0 0x558e764fe700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a0c02700 1 -- 172.21.15.52:0/2176056553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f6100 msgr2=0x558e7650d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.838+0000 7f72a0c02700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f6100 0x558e7650d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72a0c02700 1 -- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764ec7a0 msgr2=0x558e764fe700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72a0c02700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764ec7a0 0x558e764fe700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72a0c02700 1 -- 172.21.15.52:0/2176056553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e7650ded0 con 0x558e764f1f60 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72a1403700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764ec7a0 0x558e764fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72a1c04700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f6100 0x558e7650d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72a0c02700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 0x558e764fec40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f728c0043d0 tx=0x7f728c0044b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905cd9e7be5ad30a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f7284ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f7284ff9700 1 -- 172.21.15.52:0/2176056553 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f728c003f40 con 0x558e764f1f60 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f7284ff9700 1 -- 172.21.15.52:0/2176056553 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f728c03c450 con 0x558e764f1f60 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f728c012e10 con 0x558e764f1f60 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f7284ff9700 1 -- 172.21.15.52:0/2176056553 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f728c043070 con 0x558e764f1f60 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72adeeab80 10 client.15789 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.839+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x558e76570cf0 con 0x558e764f1f60 2024-06-02T23:12:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f7284ff9700 1 -- 172.21.15.52:0/2176056553 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f728c03cb00 con 0x558e764f1f60 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f7284ff9700 1 -- 172.21.15.52:0/2176056553 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f728c031070 con 0x558e764f1f60 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f72adeeab80 10 client.15789 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f72adeeab80 10 client.15789 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f72adeeab80 20 client.15789 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f72adeeab80 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76573360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f72adeeab80 4 client.15789 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.840+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f728c012e10 con 0x558e765066e0 2024-06-02T23:12:29.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f72a1403700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76573360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f7284ff9700 10 client.15789 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:29.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f72a1403700 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76573360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=701925ed45af3a62 server_cookie=70fe97d4f14b9eb8 in_seq=0 out_seq=0 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f7284ff9700 1 -- 172.21.15.52:0/2176056553 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f728c012e10 con 0x558e765066e0 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f7284ff9700 10 client.15789 handle_command_reply: tid=0 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f72adeeab80 1 client.15789 shutdown 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 msgr2=0x558e764fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.841+0000 7f72adeeab80 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 0x558e764fec40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f728c0043d0 tx=0x7f728c0044b0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 shutdown_connections 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558e765066e0 0x558e76573360 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558e764f6100 0x558e7650d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558e764ec7a0 0x558e764fe700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f7284ff9700 0 client.15789 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 --2- 172.21.15.52:0/2176056553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558e764f1f60 0x558e764fec40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 >> 172.21.15.52:0/2176056553 conn(0x558e76419080 msgr2=0x558e764e6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:29.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 shutdown_connections 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 1 -- 172.21.15.52:0/2176056553 wait complete. 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.842+0000 7f72adeeab80 20 client.15789 trim_cache size 0 max 16384 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72880a2ca0 msgr2=0x7f72880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72880a2ca0 0x7f72880a5160 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f729c078900 tx=0x7f728c009260 comp rx=0 tx=0).stop 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 msgr2=0x7f729c197720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:29.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 0x7f729c197720 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f729800e9a0 tx=0x7f729800ed60 comp rx=0 tx=0).stop 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 shutdown_connections 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72880a2ca0 0x7f72880a5160 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f729c1073f0 0x7f729c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f729c101380 0x7f729c197720 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 --2- 172.21.15.52:0/2515787103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f729c101d40 0x7f729c197c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 >> 172.21.15.52:0/2515787103 conn(0x7f729c078300 msgr2=0x7f729c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 shutdown_connections 2024-06-02T23:12:29.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:29.843+0000 7f72a3667700 1 -- 172.21.15.52:0/2515787103 wait complete. 2024-06-02T23:12:30.697 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:12:30.992 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:31.124 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:30 smithi052 ceph-mon[32340]: pgmap v450: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:30 smithi123 ceph-mon[35977]: pgmap v450: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:30 smithi174 ceph-mon[35994]: pgmap v450: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:32.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 -- 172.21.15.52:0/351261287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c06aa00 msgr2=0x7f245c06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 --2- 172.21.15.52:0/351261287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c06aa00 0x7f245c06ae80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 -- 172.21.15.52:0/351261287 shutdown_connections 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 --2- 172.21.15.52:0/351261287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06b3c0 0x7f245c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 --2- 172.21.15.52:0/351261287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c1084f0 0x7f245c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 --2- 172.21.15.52:0/351261287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c06aa00 0x7f245c06ae80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 -- 172.21.15.52:0/351261287 >> 172.21.15.52:0/351261287 conn(0x7f245c078300 msgr2=0x7f245c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 -- 172.21.15.52:0/351261287 shutdown_connections 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.923+0000 7f246093b700 1 -- 172.21.15.52:0/351261287 wait complete. 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.924+0000 7f246093b700 1 Processor -- start 2024-06-02T23:12:32.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.924+0000 7f246093b700 1 -- start start 2024-06-02T23:12:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.924+0000 7f246093b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06aa00 0x7f245c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.924+0000 7f246093b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c06b3c0 0x7f245c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f246093b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 0x7f245c10e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f246093b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f245c112650 con 0x7f245c1084f0 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f246093b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f245c1124d0 con 0x7f245c06aa00 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f246093b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f245c1127d0 con 0x7f245c06b3c0 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 0x7f245c10e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 0x7f245c10e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36260/0 (socket says 172.21.15.52:36260) 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 -- 172.21.15.52:0/820819406 learned_addr learned my addr 172.21.15.52:0/820819406 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245a59c700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06aa00 0x7f245c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f2459d9b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c06b3c0 0x7f245c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 -- 172.21.15.52:0/820819406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c06b3c0 msgr2=0x7f245c19fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c06b3c0 0x7f245c19fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 -- 172.21.15.52:0/820819406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06aa00 msgr2=0x7f245c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06aa00 0x7f245c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245ad9d700 1 -- 172.21.15.52:0/820819406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f245c10edf0 con 0x7f245c1084f0 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.925+0000 7f245a59c700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06aa00 0x7f245c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.926+0000 7f245ad9d700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 0x7f245c10e4f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f245400c370 tx=0x7f245400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.926+0000 7f244b7fe700 1 -- 172.21.15.52:0/820819406 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454004bb0 con 0x7f245c1084f0 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.926+0000 7f244b7fe700 1 -- 172.21.15.52:0/820819406 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2454004d50 con 0x7f245c1084f0 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.926+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f245c10f080 con 0x7f245c1084f0 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.926+0000 7f244b7fe700 1 -- 172.21.15.52:0/820819406 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2454004020 con 0x7f245c1084f0 2024-06-02T23:12:32.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.927+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f245c19b380 con 0x7f245c1084f0 2024-06-02T23:12:32.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.930+0000 7f244b7fe700 1 -- 172.21.15.52:0/820819406 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f245401b030 con 0x7f245c1084f0 2024-06-02T23:12:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 Processor -- start 2024-06-02T23:12:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 -- start start 2024-06-02T23:12:32.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926463340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x55592645d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592645dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55592634ff60 con 0x555926462f60 2024-06-02T23:12:32.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559263ae850 con 0x55592645d7a0 2024-06-02T23:12:32.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.931+0000 7f246b1beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555926316460 con 0x555926463880 2024-06-02T23:12:32.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f244b7fe700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2d20 0x7f24440a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x55592645d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x55592645d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43416/0 (socket says 172.21.15.52:43416) 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 -- 172.21.15.52:0/3972458675 learned_addr learned my addr 172.21.15.52:0/3972458675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245ad9d700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2d20 0x7f24440a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f244b7fe700 1 -- 172.21.15.52:0/820819406 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f24540cea40 con 0x7f245c1084f0 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 -- 172.21.15.52:0/3972458675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 msgr2=0x55592645dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f2459d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592645dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592645dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 -- 172.21.15.52:0/3972458675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 msgr2=0x555926463340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926463340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.932+0000 7f245a59c700 1 -- 172.21.15.52:0/3972458675 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559262fd8d0 con 0x555926463880 2024-06-02T23:12:32.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.933+0000 7f2459d9b700 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592645dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:32.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.933+0000 7f245ad9d700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2d20 0x7f24440a51e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f245409b000 tx=0x7f245406a280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:32.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.933+0000 7f245a59c700 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x55592645d260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f244c00afd0 tx=0x7f244c00f770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56894b55453de694 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:32.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.933+0000 7f2448ff9700 1 -- 172.21.15.52:0/3972458675 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c00fea0 con 0x555926463880 2024-06-02T23:12:32.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.933+0000 7f2448ff9700 1 -- 172.21.15.52:0/3972458675 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c005020 con 0x555926463880 2024-06-02T23:12:32.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.933+0000 7f2448ff9700 1 -- 172.21.15.52:0/3972458675 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c010910 con 0x555926463880 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 -- 172.21.15.52:0/3972458675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 msgr2=0x55592645d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x55592645d260 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f244c00afd0 tx=0x7f244c00f770 comp rx=0 tx=0).stop 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 -- 172.21.15.52:0/3972458675 shutdown_connections 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592645dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926463340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3972458675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x55592645d260 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.934+0000 7f246b1beb80 1 -- 172.21.15.52:0/3972458675 >> 172.21.15.52:0/3972458675 conn(0x55592638a080 msgr2=0x555926458220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.935+0000 7f246b1beb80 1 -- 172.21.15.52:0/3972458675 shutdown_connections 2024-06-02T23:12:32.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.935+0000 7f246b1beb80 1 -- 172.21.15.52:0/3972458675 wait complete. 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.935+0000 7f246b1beb80 1 -- start start 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.935+0000 7f246b1beb80 1 Processor -- start 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x555926469650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926466950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x555926466e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55592634efc0 con 0x555926462f60 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5559263aa810 con 0x55592645d7a0 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55592647b270 con 0x555926463880 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f246b1beb80 10 client.0 _ll_register_callbacks cb 0x5559263445b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f245a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926466950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f245a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926466950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36270/0 (socket says 172.21.15.52:36270) 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.936+0000 7f245a59c700 1 -- 172.21.15.52:0/3737894008 learned_addr learned my addr 172.21.15.52:0/3737894008 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f245a59c700 1 -- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 msgr2=0x555926466e90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:12:32.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f245a59c700 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x555926466e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f245a59c700 1 -- 172.21.15.52:0/3737894008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 msgr2=0x555926469650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f245a59c700 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x555926469650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f245a59c700 1 -- 172.21.15.52:0/3737894008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555926374750 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f245a59c700 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926466950 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f244c004da0 tx=0x7f244c004e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b0a28b1276fe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f2441ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f2441ffb700 1 -- 172.21.15.52:0/3737894008 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f244c026070 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f2441ffb700 1 -- 172.21.15.52:0/3737894008 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f244c010a50 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.937+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f244c005020 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f2441ffb700 1 -- 172.21.15.52:0/3737894008 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f244c032050 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 10 client.15798 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559264857b0 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f2441ffb700 1 -- 172.21.15.52:0/3737894008 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f244c005550 con 0x555926462f60 2024-06-02T23:12:32.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f2441ffb700 1 -- 172.21.15.52:0/3737894008 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f244c018c50 con 0x555926462f60 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 10 client.15798 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 10 client.15798 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 20 client.15798 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x555926487ce0 0x55592648a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 4 client.15798 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.938+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f244c005020 con 0x555926487ce0 2024-06-02T23:12:32.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.939+0000 7f2459d9b700 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x555926487ce0 0x55592648a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.939+0000 7f2441ffb700 10 client.15798 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.939+0000 7f2459d9b700 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x555926487ce0 0x55592648a0d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a3c78795f211a88c server_cookie=445d98d1a842adc8 in_seq=0 out_seq=0 2024-06-02T23:12:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f2441ffb700 1 -- 172.21.15.52:0/3737894008 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f244c005020 con 0x555926487ce0 2024-06-02T23:12:32.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f2441ffb700 10 client.15798 handle_command_reply: tid=0 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 client.15798 shutdown 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 msgr2=0x555926466950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926466950 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f244c004da0 tx=0x7f244c004e80 comp rx=0 tx=0).stop 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 shutdown_connections 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x555926487ce0 0x55592648a0d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f2441ffb700 0 client.15798 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x555926466e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x555926469650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3737894008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926462f60 0x555926466950 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.940+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 >> 172.21.15.52:0/3737894008 conn(0x55592638a080 msgr2=0x555926457700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.943+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 shutdown_connections 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.943+0000 7f246b1beb80 1 -- 172.21.15.52:0/3737894008 wait complete. 2024-06-02T23:12:32.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.943+0000 7f246b1beb80 20 client.15798 trim_cache size 0 max 16384 2024-06-02T23:12:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.958+0000 7f246b1beb80 1 Processor -- start 2024-06-02T23:12:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 -- start start 2024-06-02T23:12:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 0x5559264645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555926462f60 0x555926465950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x555926457e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f245c072470 con 0x55592645d7a0 2024-06-02T23:12:32.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f245c073fb0 con 0x555926462f60 2024-06-02T23:12:32.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f246b1beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f245c072b40 con 0x555926463880 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 0x5559264645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 0x5559264645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36280/0 (socket says 172.21.15.52:36280) 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 -- 172.21.15.52:0/3580391259 learned_addr learned my addr 172.21.15.52:0/3580391259 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f245a59c700 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555926462f60 0x555926465950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 -- 172.21.15.52:0/3580391259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 msgr2=0x555926457e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x555926457e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 -- 172.21.15.52:0/3580391259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555926462f60 msgr2=0x555926465950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.959+0000 7f2459d9b700 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555926462f60 0x555926465950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.960+0000 7f2459d9b700 1 -- 172.21.15.52:0/3580391259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55592645d5a0 con 0x55592645d7a0 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.960+0000 7f2459d9b700 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 0x5559264645b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbae13a25461a8c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.960+0000 7f2443fff700 1 -- 172.21.15.52:0/3580391259 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24500056d0 con 0x55592645d7a0 2024-06-02T23:12:32.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.960+0000 7f2443fff700 1 -- 172.21.15.52:0/3580391259 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f245000bc90 con 0x55592645d7a0 2024-06-02T23:12:32.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.960+0000 7f2443fff700 1 -- 172.21.15.52:0/3580391259 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245000f460 con 0x55592645d7a0 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 -- 172.21.15.52:0/3580391259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 msgr2=0x5559264645b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 0x5559264645b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2450009990 tx=0x7f2450005c70 comp rx=0 tx=0).stop 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 -- 172.21.15.52:0/3580391259 shutdown_connections 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926463880 0x555926457e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55592645d7a0 0x5559264645b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 --2- 172.21.15.52:0/3580391259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555926462f60 0x555926465950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 -- 172.21.15.52:0/3580391259 >> 172.21.15.52:0/3580391259 conn(0x55592638a080 msgr2=0x555926465240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 -- 172.21.15.52:0/3580391259 shutdown_connections 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.961+0000 7f246b1beb80 1 -- 172.21.15.52:0/3580391259 wait complete. 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 -- start start 2024-06-02T23:12:32.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 Processor -- start 2024-06-02T23:12:32.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592648d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926462f60 0x55592648d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 0x55592648aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f245c0738e0 con 0x555926463880 2024-06-02T23:12:32.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f245c073210 con 0x55592645d7a0 2024-06-02T23:12:32.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f245c070410 con 0x555926462f60 2024-06-02T23:12:32.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f2459d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 0x55592648aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f246b1beb80 10 client.0 _ll_register_callbacks cb 0x5559263445b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f2459d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592648d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f2459d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 0x55592648aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36286/0 (socket says 172.21.15.52:36286) 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f2459d9b700 1 -- 172.21.15.52:0/4124456874 learned_addr learned my addr 172.21.15.52:0/4124456874 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.962+0000 7f245a59c700 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926462f60 0x55592648d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2459d9b700 1 -- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926462f60 msgr2=0x55592648d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2459d9b700 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926462f60 0x55592648d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2459d9b700 1 -- 172.21.15.52:0/4124456874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 msgr2=0x55592648d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2459d9b700 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592648d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2459d9b700 1 -- 172.21.15.52:0/4124456874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55592648b260 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2459d9b700 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 0x55592648aa20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2450009710 tx=0x7f2450009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179a1b942ea9504a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2441ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2441ffb700 1 -- 172.21.15.52:0/4124456874 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2450031070 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f2441ffb700 1 -- 172.21.15.52:0/4124456874 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f245000b7e0 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.963+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f245000bc90 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f2441ffb700 1 -- 172.21.15.52:0/4124456874 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f245000b9e0 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 10 client.15804 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5559264f22a0 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f2441ffb700 1 -- 172.21.15.52:0/4124456874 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f245000bc90 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f2441ffb700 1 -- 172.21.15.52:0/4124456874 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f24500243f0 con 0x555926463880 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 10 client.15804 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 10 client.15804 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 20 client.15804 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5559264f2920 0x5559264f4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:32.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 4 client.15804 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.964+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5559263bbd50 con 0x5559264f2920 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f245a59c700 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5559264f2920 0x5559264f4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f2441ffb700 10 client.15804 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f245a59c700 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5559264f2920 0x5559264f4d40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6fc6bcef025d5e8 server_cookie=58dd7206015d697c in_seq=0 out_seq=0 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f2441ffb700 1 -- 172.21.15.52:0/4124456874 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5559263bbd50 con 0x5559264f2920 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f2441ffb700 10 client.15804 handle_command_reply: tid=0 2024-06-02T23:12:32.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f246b1beb80 1 client.15804 shutdown 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 msgr2=0x55592648aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f246b1beb80 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 0x55592648aa20 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2450009710 tx=0x7f2450009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 shutdown_connections 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f246b1beb80 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5559264f2920 0x5559264f4d40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.965+0000 7f246b1beb80 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555926463880 0x55592648aa20 secure :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2450009710 tx=0x7f2450009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f246b1beb80 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55592645d7a0 0x55592648d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f246b1beb80 1 --2- 172.21.15.52:0/4124456874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555926462f60 0x55592648d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 >> 172.21.15.52:0/4124456874 conn(0x55592638a080 msgr2=0x555926464370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f2441ffb700 0 client.15804 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 shutdown_connections 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f246b1beb80 1 -- 172.21.15.52:0/4124456874 wait complete. 2024-06-02T23:12:32.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.966+0000 7f246b1beb80 20 client.15804 trim_cache size 0 max 16384 2024-06-02T23:12:32.970 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:32.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2d20 msgr2=0x7f24440a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2d20 0x7f24440a51e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f245409b000 tx=0x7f245406a280 comp rx=0 tx=0).stop 2024-06-02T23:12:32.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 msgr2=0x7f245c10e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:32.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 0x7f245c10e4f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f245400c370 tx=0x7f245400c680 comp rx=0 tx=0).stop 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 shutdown_connections 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24440a2d20 0x7f24440a51e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f245c1084f0 0x7f245c10e4f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f245c06aa00 0x7f245c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 --2- 172.21.15.52:0/820819406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f245c06b3c0 0x7f245c19fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 >> 172.21.15.52:0/820819406 conn(0x7f245c078300 msgr2=0x7f245c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 shutdown_connections 2024-06-02T23:12:32.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:32.967+0000 7f246093b700 1 -- 172.21.15.52:0/820819406 wait complete. 2024-06-02T23:12:33.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:32 smithi052 ceph-mon[32340]: pgmap v451: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:33.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:32 smithi123 ceph-mon[35977]: pgmap v451: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:32 smithi174 ceph-mon[35994]: pgmap v451: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-06-02T23:12:33.678 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:12:34.002 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:34.304 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:12:35.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:34 smithi052 ceph-mon[32340]: pgmap v452: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:35.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:34 smithi123 ceph-mon[35977]: pgmap v452: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:34 smithi174 ceph-mon[35994]: pgmap v452: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:35.859 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:36 smithi123 ceph-mon[35977]: pgmap v453: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:37.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:36 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:12:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:36 smithi174 ceph-mon[35994]: pgmap v453: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:36 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:12:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:36 smithi052 ceph-mon[32340]: pgmap v453: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:36 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:12:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:38 smithi052 ceph-mon[32340]: pgmap v454: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:38 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:12:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:38 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:12:39.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:38 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:12:39.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:38 smithi123 ceph-mon[35977]: pgmap v454: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:39.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:38 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:12:39.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:38 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:12:39.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:38 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:12:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 -- 172.21.15.52:0/3777128575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 msgr2=0x7faab0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 --2- 172.21.15.52:0/3777128575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab0100d20 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7faaac009900 tx=0x7faaac02a410 comp rx=0 tx=0).stop 2024-06-02T23:12:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 -- 172.21.15.52:0/3777128575 shutdown_connections 2024-06-02T23:12:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 --2- 172.21.15.52:0/3777128575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 0x7faab0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 --2- 172.21.15.52:0/3777128575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 0x7faab0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 --2- 172.21.15.52:0/3777128575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab0100d20 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 -- 172.21.15.52:0/3777128575 >> 172.21.15.52:0/3777128575 conn(0x7faab00fbe70 msgr2=0x7faab00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 -- 172.21.15.52:0/3777128575 shutdown_connections 2024-06-02T23:12:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.330+0000 7faab7fc1640 1 -- 172.21.15.52:0/3777128575 wait complete. 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 Processor -- start 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 -- start start 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab01957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab5d5d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab01957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab5d5d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab01957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57812/0 (socket says 172.21.15.52:57812) 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab5d5d640 1 -- 172.21.15.52:0/754432036 learned_addr learned my addr 172.21.15.52:0/754432036 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:39.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 0x7faab0195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 0x7faab019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faab010b380 con 0x7faab0100900 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faab010b200 con 0x7faab0101af0 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.331+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faab010b500 con 0x7faab01024b0 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab555c640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 0x7faab0195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab655e640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 0x7faab019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab5d5d640 1 -- 172.21.15.52:0/754432036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 msgr2=0x7faab019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab5d5d640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 0x7faab019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab5d5d640 1 -- 172.21.15.52:0/754432036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 msgr2=0x7faab0195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab5d5d640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 0x7faab0195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab5d5d640 1 -- 172.21.15.52:0/754432036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab019d3d0 con 0x7faab0100900 2024-06-02T23:12:39.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab655e640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 0x7faab019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab555c640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 0x7faab0195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab5d5d640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab01957b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faaac009850 tx=0x7faaac003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faaa6ffd640 1 -- 172.21.15.52:0/754432036 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac03f070 con 0x7faab0100900 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faaa6ffd640 1 -- 172.21.15.52:0/754432036 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaac031dd0 con 0x7faab0100900 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faaa6ffd640 1 -- 172.21.15.52:0/754432036 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac035030 con 0x7faab0100900 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.332+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab019d660 con 0x7faab0100900 2024-06-02T23:12:39.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.333+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab019db00 con 0x7faab0100900 2024-06-02T23:12:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.336+0000 7faabadac740 1 Processor -- start 2024-06-02T23:12:39.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.336+0000 7faabadac740 1 -- start start 2024-06-02T23:12:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.336+0000 7faabadac740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 0x5578d0d7cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faabadac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7d210 0x5578d0d81d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faabadac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d82380 0x5578d0d84810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faabadac740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5578d0cd74f0 con 0x5578d0d7d210 2024-06-02T23:12:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faabadac740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578d0c75870 con 0x5578d0d7c820 2024-06-02T23:12:39.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faabadac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578d0cd34b0 con 0x5578d0d82380 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 0x5578d0d7cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 0x5578d0d7cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51320/0 (socket says 172.21.15.52:51320) 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 -- 172.21.15.52:0/4244903239 learned_addr learned my addr 172.21.15.52:0/4244903239 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab555c640 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d82380 0x5578d0d84810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 -- 172.21.15.52:0/4244903239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d82380 msgr2=0x5578d0d84810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d82380 0x5578d0d84810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 -- 172.21.15.52:0/4244903239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7d210 msgr2=0x5578d0d81d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7d210 0x5578d0d81d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faab655e640 1 -- 172.21.15.52:0/4244903239 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d0c919b0 con 0x5578d0d7c820 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.337+0000 7faaa6ffd640 1 -- 172.21.15.52:0/754432036 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faaac042070 con 0x7faab0100900 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.338+0000 7faab655e640 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 0x5578d0d7cc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faaa800bf00 tx=0x7faaa8008600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=389c70d16af48e6d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.338+0000 7faa9bfff640 1 -- 172.21.15.52:0/4244903239 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8008e30 con 0x5578d0d7c820 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.338+0000 7faa9bfff640 1 -- 172.21.15.52:0/4244903239 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaa8013590 con 0x5578d0d7c820 2024-06-02T23:12:39.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.338+0000 7faa9bfff640 1 -- 172.21.15.52:0/4244903239 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8005260 con 0x5578d0d7c820 2024-06-02T23:12:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.338+0000 7faaa6ffd640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa9c0a2d30 0x7faa9c0a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faaa6ffd640 1 -- 172.21.15.52:0/754432036 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7faaac0ef510 con 0x7faab0100900 2024-06-02T23:12:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faab5d5d640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa9c0a2d30 0x7faa9c0a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 -- 172.21.15.52:0/4244903239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 msgr2=0x5578d0d7cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 0x5578d0d7cc40 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faaa800bf00 tx=0x7faaa8008600 comp rx=0 tx=0).stop 2024-06-02T23:12:39.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 -- 172.21.15.52:0/4244903239 shutdown_connections 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d82380 0x5578d0d84810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7d210 0x5578d0d81d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 --2- 172.21.15.52:0/4244903239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7c820 0x5578d0d7cc40 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 -- 172.21.15.52:0/4244903239 >> 172.21.15.52:0/4244903239 conn(0x5578d0d6f680 msgr2=0x5578d0d6fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faab5d5d640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa9c0a2d30 0x7faa9c0a51f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faaac0b81a0 tx=0x7faaac005de0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 -- 172.21.15.52:0/4244903239 shutdown_connections 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.339+0000 7faabadac740 1 -- 172.21.15.52:0/4244903239 wait complete. 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.340+0000 7faabadac740 1 -- start start 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.340+0000 7faabadac740 1 Processor -- start 2024-06-02T23:12:39.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.340+0000 7faabadac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d86a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.340+0000 7faabadac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d86fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.340+0000 7faabadac740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d86a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d86a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57814/0 (socket says 172.21.15.52:57814) 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 -- 172.21.15.52:0/3272918136 learned_addr learned my addr 172.21.15.52:0/3272918136 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5578d0d85330 con 0x5578d0d7c820 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5578d0c60ae0 con 0x5578d0d82380 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5578d0d854b0 con 0x5578d0d7d210 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faabadac740 10 client.0 _ll_register_callbacks cb 0x5578d0c999b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab555c640 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d86fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 -- 172.21.15.52:0/3272918136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 msgr2=0x5578d0d86fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d86fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 -- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 msgr2=0x5578d0d8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d09eb210 con 0x5578d0d7c820 2024-06-02T23:12:39.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faab655e640 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d86a60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7faaa8005cc0 tx=0x7faaa8004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ea3f6eaa2faecc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.341+0000 7faa997fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faa997fa640 1 -- 172.21.15.52:0/3272918136 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8024070 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa8013590 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faa997fa640 1 -- 172.21.15.52:0/3272918136 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7faaa8013950 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faa997fa640 1 -- 172.21.15.52:0/3272918136 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faaa8030050 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faabadac740 10 client.15813 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578d0ab7620 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.342+0000 7faa997fa640 1 -- 172.21.15.52:0/3272918136 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaa8013590 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faa997fa640 1 -- 172.21.15.52:0/3272918136 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faaa8020610 con 0x5578d0d7c820 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faabadac740 10 client.15813 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faabadac740 10 client.15813 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faabadac740 20 client.15813 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:39.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faabadac740 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0d92c50 0x5578d0d950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faabadac740 4 client.15813 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:39.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578d0ceaf10 con 0x5578d0d92c50 2024-06-02T23:12:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.343+0000 7faab555c640 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0d92c50 0x5578d0d950d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.344+0000 7faab555c640 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0d92c50 0x5578d0d950d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41052d22f198fcc3 server_cookie=79be2224fc7092d1 in_seq=0 out_seq=0 2024-06-02T23:12:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.344+0000 7faa997fa640 10 client.15813 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faa997fa640 1 -- 172.21.15.52:0/3272918136 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5578d0ceaf10 con 0x5578d0d92c50 2024-06-02T23:12:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faa997fa640 10 client.15813 handle_command_reply: tid=0 2024-06-02T23:12:39.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 client.15813 shutdown 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 msgr2=0x5578d0d86a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d86a60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7faaa8005cc0 tx=0x7faaa8004cc0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 shutdown_connections 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0d92c50 0x5578d0d950d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d86fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 --2- 172.21.15.52:0/3272918136 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d86a60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 >> 172.21.15.52:0/3272918136 conn(0x5578d0d6f680 msgr2=0x5578d0d6e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faa997fa640 0 client.15813 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 shutdown_connections 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 1 -- 172.21.15.52:0/3272918136 wait complete. 2024-06-02T23:12:39.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.345+0000 7faabadac740 20 client.15813 trim_cache size 0 max 16384 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.376+0000 7faabadac740 1 Processor -- start 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.376+0000 7faabadac740 1 -- start start 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faabadac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7c820 0x5578d0d6a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faabadac740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7d210 0x5578d0d8df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faabadac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 0x5578d0d85fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faabadac740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faab01a68a0 con 0x5578d0d82380 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faabadac740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faab01a83e0 con 0x5578d0d7d210 2024-06-02T23:12:39.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faabadac740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faab01a6f70 con 0x5578d0d7c820 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 0x5578d0d85fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 0x5578d0d85fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57820/0 (socket says 172.21.15.52:57820) 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 -- 172.21.15.52:0/1343047022 learned_addr learned my addr 172.21.15.52:0/1343047022 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab555c640 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7d210 0x5578d0d8df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7c820 0x5578d0d6a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 -- 172.21.15.52:0/1343047022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7c820 msgr2=0x5578d0d6a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7c820 0x5578d0d6a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 -- 172.21.15.52:0/1343047022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7d210 msgr2=0x5578d0d8df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7d210 0x5578d0d8df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab655e640 1 -- 172.21.15.52:0/1343047022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578d0d6f4b0 con 0x5578d0d82380 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.377+0000 7faab555c640 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7d210 0x5578d0d8df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.378+0000 7faab655e640 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 0x5578d0d85fc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7faaa800b7a0 tx=0x7faaa8008600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46f991c439e9e03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.378+0000 7faa9b7fe640 1 -- 172.21.15.52:0/1343047022 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa802d070 con 0x5578d0d82380 2024-06-02T23:12:39.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.378+0000 7faa9b7fe640 1 -- 172.21.15.52:0/1343047022 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaa80058f0 con 0x5578d0d82380 2024-06-02T23:12:39.380 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.378+0000 7faa9b7fe640 1 -- 172.21.15.52:0/1343047022 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa8024070 con 0x5578d0d82380 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 -- 172.21.15.52:0/1343047022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 msgr2=0x5578d0d85fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 0x5578d0d85fc0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7faaa800b7a0 tx=0x7faaa8008600 comp rx=0 tx=0).stop 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 -- 172.21.15.52:0/1343047022 shutdown_connections 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d82380 0x5578d0d85fc0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d7d210 0x5578d0d8df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 --2- 172.21.15.52:0/1343047022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7c820 0x5578d0d6a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 -- 172.21.15.52:0/1343047022 >> 172.21.15.52:0/1343047022 conn(0x5578d0d6f680 msgr2=0x5578d0d858b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 -- 172.21.15.52:0/1343047022 shutdown_connections 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 -- 172.21.15.52:0/1343047022 wait complete. 2024-06-02T23:12:39.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.379+0000 7faabadac740 1 -- start start 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 Processor -- start 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d8a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d8a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faab655e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d8a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faab655e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d8a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57826/0 (socket says 172.21.15.52:57826) 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faab655e640 1 -- 172.21.15.52:0/2631390436 learned_addr learned my addr 172.21.15.52:0/2631390436 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faab01a7d10 con 0x5578d0d7c820 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faab01a7640 con 0x5578d0d82380 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faab01a4880 con 0x5578d0d7d210 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.380+0000 7faab555c640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d8a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faabadac740 10 client.0 _ll_register_callbacks cb 0x7faaa000f8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab655e640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab555c640 1 -- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 msgr2=0x5578d0d8d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab555c640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab555c640 1 -- 172.21.15.52:0/2631390436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 msgr2=0x5578d0d8a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab555c640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d8a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab555c640 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578d0d70350 con 0x5578d0d7d210 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab655e640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faab555c640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d8a690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5578d0d95920 tx=0x7faaa00035f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=750637840274aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faa997fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.381+0000 7faa997fa640 1 -- 172.21.15.52:0/2631390436 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa002a680 con 0x5578d0d7d210 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.382+0000 7faa997fa640 1 -- 172.21.15.52:0/2631390436 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7faaa002a820 con 0x5578d0d7d210 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.382+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa80058f0 con 0x5578d0d7d210 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.382+0000 7faa997fa640 1 -- 172.21.15.52:0/2631390436 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7faaa0031070 con 0x5578d0d7d210 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.382+0000 7faabadac740 10 client.25351 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:39.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.382+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5578d0dfc380 con 0x5578d0d7d210 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.382+0000 7faa997fa640 1 -- 172.21.15.52:0/2631390436 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faaa80058f0 con 0x5578d0d7d210 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faa997fa640 1 -- 172.21.15.52:0/2631390436 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7faaa001bad0 con 0x5578d0d7d210 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faabadac740 10 client.25351 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faabadac740 10 client.25351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faabadac740 20 client.25351 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faabadac740 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0dfcd80 0x5578d0dff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faabadac740 4 client.25351 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5578d0ce4c20 con 0x5578d0dfcd80 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.383+0000 7faab655e640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0dfcd80 0x5578d0dff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.384+0000 7faab655e640 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0dfcd80 0x5578d0dff1a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2ac6b5920c8fb50 server_cookie=757495449c17776 in_seq=0 out_seq=0 2024-06-02T23:12:39.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.384+0000 7faa997fa640 10 client.25351 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faa997fa640 1 -- 172.21.15.52:0/2631390436 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5578d0ce4c20 con 0x5578d0dfcd80 2024-06-02T23:12:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faa997fa640 10 client.25351 handle_command_reply: tid=0 2024-06-02T23:12:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 client.25351 shutdown 2024-06-02T23:12:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 msgr2=0x5578d0d8a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d8a690 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5578d0d95920 tx=0x7faaa00035f0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 shutdown_connections 2024-06-02T23:12:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5578d0dfcd80 0x5578d0dff1a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faa997fa640 0 client.25351 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:39.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5578d0d7d210 0x5578d0d8a690 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5578d0d82380 0x5578d0d8d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 --2- 172.21.15.52:0/2631390436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5578d0d7c820 0x5578d0d8a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.388+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 >> 172.21.15.52:0/2631390436 conn(0x5578d0d6f680 msgr2=0x5578d0d85670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 shutdown_connections 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faabadac740 1 -- 172.21.15.52:0/2631390436 wait complete. 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faabadac740 20 client.25351 trim_cache size 0 max 16384 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:12:39.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa9c0a2d30 msgr2=0x7faa9c0a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa9c0a2d30 0x7faa9c0a51f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7faaac0b81a0 tx=0x7faaac005de0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 msgr2=0x7faab01957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.389+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab01957b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7faaac009850 tx=0x7faaac003f80 comp rx=0 tx=0).stop 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 shutdown_connections 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa9c0a2d30 0x7faa9c0a51f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab01024b0 0x7faab019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faab0101af0 0x7faab0195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 --2- 172.21.15.52:0/754432036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faab0100900 0x7faab01957b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 >> 172.21.15.52:0/754432036 conn(0x7faab00fbe70 msgr2=0x7faab0104d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 shutdown_connections 2024-06-02T23:12:39.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:39.390+0000 7faab7fc1640 1 -- 172.21.15.52:0/754432036 wait complete. 2024-06-02T23:12:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:38 smithi174 ceph-mon[35994]: pgmap v454: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 11 op/s 2024-06-02T23:12:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:38 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:12:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:38 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:12:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:38 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:12:40.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1095036889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa374101390 msgr2=0x7fa3741017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1095036889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa374101390 0x7fa3741017f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fa368009990 tx=0x7fa368009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1095036889 shutdown_connections 2024-06-02T23:12:40.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1095036889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa374101d30 0x7fa37410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1095036889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3741073d0 0x7fa3741077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1095036889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa374101390 0x7fa3741017f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1095036889 >> 172.21.15.52:0/1095036889 conn(0x7fa3740782c0 msgr2=0x7fa3740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:40.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1095036889 shutdown_connections 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.745+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1095036889 wait complete. 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.746+0000 7fa37a8e3700 1 Processor -- start 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.746+0000 7fa37a8e3700 1 -- start start 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.746+0000 7fa37a8e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa374101390 0x7fa37410eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.746+0000 7fa37a8e3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa374101d30 0x7fa37410f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa37a8e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 0x7fa37410f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa37a8e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa374112680 con 0x7fa3741073d0 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa37a8e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa374112500 con 0x7fa374101390 2024-06-02T23:12:40.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa37a8e3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa374112800 con 0x7fa374101d30 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 0x7fa37410f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 0x7fa37410f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57852/0 (socket says 172.21.15.52:57852) 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 -- 172.21.15.52:0/1658369798 learned_addr learned my addr 172.21.15.52:0/1658369798 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa373fff700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa374101390 0x7fa37410eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa3737fe700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa374101d30 0x7fa37410f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 -- 172.21.15.52:0/1658369798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa374101d30 msgr2=0x7fa37410f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa374101d30 0x7fa37410f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 -- 172.21.15.52:0/1658369798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa374101390 msgr2=0x7fa37410eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa374101390 0x7fa37410eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 -- 172.21.15.52:0/1658369798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3741a6fe0 con 0x7fa3741073d0 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.747+0000 7fa378e80700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 0x7fa37410f720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa36c007d70 tx=0x7fa36c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.748+0000 7fa3717fa700 1 -- 172.21.15.52:0/1658369798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c016020 con 0x7fa3741073d0 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.748+0000 7fa3717fa700 1 -- 172.21.15.52:0/1658369798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa36c00ad40 con 0x7fa3741073d0 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.748+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3741a72d0 con 0x7fa3741073d0 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.748+0000 7fa3717fa700 1 -- 172.21.15.52:0/1658369798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c0115e0 con 0x7fa3741073d0 2024-06-02T23:12:40.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.748+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3741a7860 con 0x7fa3741073d0 2024-06-02T23:12:40.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.751+0000 7fa3717fa700 1 -- 172.21.15.52:0/1658369798 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa36c019020 con 0x7fa3741073d0 2024-06-02T23:12:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.752+0000 7fa385166b80 1 Processor -- start 2024-06-02T23:12:40.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.752+0000 7fa385166b80 1 -- start start 2024-06-02T23:12:40.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.752+0000 7fa385166b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb275a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.752+0000 7fa385166b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb26f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb275a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb275a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47904/0 (socket says 172.21.15.52:47904) 2024-06-02T23:12:40.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa3737fe700 1 -- 172.21.15.52:0/3720968643 learned_addr learned my addr 172.21.15.52:0/3720968643 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa3717fa700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2ca0 0x7fa35c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa373fff700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb26f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa3717fa700 1 -- 172.21.15.52:0/1658369798 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa36c0ceef0 con 0x7fa3741073d0 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa385166b80 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb270340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b7bb1c1810 con 0x55b7bb275f60 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b7bb169720 con 0x55b7bb26ff20 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.753+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b7bb1bd7d0 con 0x55b7bb275640 2024-06-02T23:12:40.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2ca0 0x7fa35c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa378e80700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb270340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 -- 172.21.15.52:0/3720968643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 msgr2=0x55b7bb270340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb270340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 -- 172.21.15.52:0/3720968643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 msgr2=0x55b7bb26f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb26f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 -- 172.21.15.52:0/3720968643 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bb15ff50 con 0x55b7bb275640 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa373fff700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb26f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa378e80700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb270340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:40.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.754+0000 7fa3737fe700 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb275a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa368006010 tx=0x7fa36800bbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1fe5b895c613a23 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.755+0000 7fa3737fe700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2ca0 0x7fa35c0a5160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa368005f50 tx=0x7fa368028000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.755+0000 7fa35a7fc700 1 -- 172.21.15.52:0/3720968643 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36802b020 con 0x55b7bb275640 2024-06-02T23:12:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.755+0000 7fa35a7fc700 1 -- 172.21.15.52:0/3720968643 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa368005590 con 0x55b7bb275640 2024-06-02T23:12:40.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.755+0000 7fa35a7fc700 1 -- 172.21.15.52:0/3720968643 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa368003d90 con 0x55b7bb275640 2024-06-02T23:12:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 msgr2=0x55b7bb275a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb275a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa368006010 tx=0x7fa36800bbf0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 shutdown_connections 2024-06-02T23:12:40.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb270340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb275a20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 --2- 172.21.15.52:0/3720968643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb26f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.756+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 >> 172.21.15.52:0/3720968643 conn(0x55b7bb19acc0 msgr2=0x55b7bb26a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.757+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 shutdown_connections 2024-06-02T23:12:40.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.757+0000 7fa385166b80 1 -- 172.21.15.52:0/3720968643 wait complete. 2024-06-02T23:12:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.757+0000 7fa385166b80 1 -- start start 2024-06-02T23:12:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.757+0000 7fa385166b80 1 Processor -- start 2024-06-02T23:12:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa385166b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb26ff20 0x55b7bb27bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa385166b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb275640 0x55b7bb2790e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa385166b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb279620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb275640 0x55b7bb2790e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb275640 0x55b7bb2790e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51382/0 (socket says 172.21.15.52:51382) 2024-06-02T23:12:40.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa373fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb26ff20 0x55b7bb27bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa378e80700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb279620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa373fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb26ff20 0x55b7bb27bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47920/0 (socket says 172.21.15.52:47920) 2024-06-02T23:12:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa378e80700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb279620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57864/0 (socket says 172.21.15.52:57864) 2024-06-02T23:12:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa3737fe700 1 -- 172.21.15.52:0/3947072019 learned_addr learned my addr 172.21.15.52:0/3947072019 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b7bb27c7f0 con 0x55b7bb275f60 2024-06-02T23:12:40.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b7bb161bf0 con 0x55b7bb275640 2024-06-02T23:12:40.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.758+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b7bb27c970 con 0x55b7bb26ff20 2024-06-02T23:12:40.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa378e80700 1 -- 172.21.15.52:0/3947072019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb26ff20 msgr2=0x55b7bb27bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa378e80700 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb26ff20 0x55b7bb27bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa378e80700 1 -- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb275640 msgr2=0x55b7bb2790e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa378e80700 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb275640 0x55b7bb2790e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa378e80700 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bb1108d0 con 0x55b7bb275f60 2024-06-02T23:12:40.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa378e80700 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb279620 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0cca70 tx=0x7fa36c019310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edb38e3117adf31f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:40.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa385166b80 10 client.0 _ll_register_callbacks cb 0x55b7bb15b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:40.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa368005590 con 0x55b7bb275f60 2024-06-02T23:12:40.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa3537fe700 10 client.15831 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:40.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa3537fe700 1 -- 172.21.15.52:0/3947072019 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36c09b780 con 0x55b7bb275f60 2024-06-02T23:12:40.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.759+0000 7fa3537fe700 1 -- 172.21.15.52:0/3947072019 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa36c07a170 con 0x55b7bb275f60 2024-06-02T23:12:40.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa3537fe700 1 -- 172.21.15.52:0/3947072019 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa36c09fb30 con 0x55b7bb275f60 2024-06-02T23:12:40.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 10 client.15831 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=28}) -- 0x55b7bb284fd0 con 0x55b7bb275f60 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa3537fe700 1 -- 172.21.15.52:0/3947072019 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa36c097040 con 0x55b7bb275f60 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 10 client.15831 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 10 client.15831 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 20 client.15831 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb29a7a0 0x55b7bb29cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 4 client.15831 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b7bb1ced10 con 0x55b7bb29a7a0 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.760+0000 7fa373fff700 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb29a7a0 0x55b7bb29cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.761+0000 7fa373fff700 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb29a7a0 0x55b7bb29cb90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8665d990f8019bf server_cookie=3f356e529ce72121 in_seq=0 out_seq=0 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.761+0000 7fa3537fe700 10 client.15831 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa3537fe700 1 -- 172.21.15.52:0/3947072019 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b7bb1ced10 con 0x55b7bb29a7a0 2024-06-02T23:12:40.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa3537fe700 10 client.15831 handle_command_reply: tid=0 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 client.15831 shutdown 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 msgr2=0x55b7bb279620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb279620 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0cca70 tx=0x7fa36c019310 comp rx=0 tx=0).stop 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 shutdown_connections 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb29a7a0 0x55b7bb29cb90 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb279620 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb26ff20 0x55b7bb27bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 --2- 172.21.15.52:0/3947072019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb275640 0x55b7bb2790e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 >> 172.21.15.52:0/3947072019 conn(0x55b7bb19acc0 msgr2=0x55b7bb26e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.762+0000 7fa3537fe700 0 client.15831 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.764+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 shutdown_connections 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.764+0000 7fa385166b80 1 -- 172.21.15.52:0/3947072019 wait complete. 2024-06-02T23:12:40.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.764+0000 7fa385166b80 20 client.15831 trim_cache size 0 max 16384 2024-06-02T23:12:40.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.781+0000 7fa385166b80 1 Processor -- start 2024-06-02T23:12:40.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.781+0000 7fa385166b80 1 -- start start 2024-06-02T23:12:40.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.781+0000 7fa385166b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb26f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.781+0000 7fa385166b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 0x55b7bb264510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.781+0000 7fa385166b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275f60 0x55b7bb2a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.781+0000 7fa385166b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa37406eb30 con 0x55b7bb275640 2024-06-02T23:12:40.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa385166b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa374070670 con 0x55b7bb26ff20 2024-06-02T23:12:40.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa385166b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa37406f200 con 0x55b7bb275f60 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 0x55b7bb264510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa378e80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275f60 0x55b7bb2a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa3737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 0x55b7bb264510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57872/0 (socket says 172.21.15.52:57872) 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa3737fe700 1 -- 172.21.15.52:0/2647631899 learned_addr learned my addr 172.21.15.52:0/2647631899 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa373fff700 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb26f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa378e80700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275f60 0x55b7bb2a2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47934/0 (socket says 172.21.15.52:47934) 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa373fff700 1 -- 172.21.15.52:0/2647631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275f60 msgr2=0x55b7bb2a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa373fff700 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275f60 0x55b7bb2a2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa373fff700 1 -- 172.21.15.52:0/2647631899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 msgr2=0x55b7bb264510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa373fff700 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 0x55b7bb264510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa373fff700 1 -- 172.21.15.52:0/2647631899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7bb26c080 con 0x55b7bb26ff20 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.782+0000 7fa3737fe700 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 0x55b7bb264510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.783+0000 7fa373fff700 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb26f9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa36c099a30 tx=0x7fa364009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f64bf34d8f78cbad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.783+0000 7fa359ffb700 1 -- 172.21.15.52:0/2647631899 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364003b60 con 0x55b7bb26ff20 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.783+0000 7fa359ffb700 1 -- 172.21.15.52:0/2647631899 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa364003d00 con 0x55b7bb26ff20 2024-06-02T23:12:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.783+0000 7fa359ffb700 1 -- 172.21.15.52:0/2647631899 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364010720 con 0x55b7bb26ff20 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 -- 172.21.15.52:0/2647631899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 msgr2=0x55b7bb26f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb26f9e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa36c099a30 tx=0x7fa364009260 comp rx=0 tx=0).stop 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 -- 172.21.15.52:0/2647631899 shutdown_connections 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275f60 0x55b7bb2a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb26f9e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 --2- 172.21.15.52:0/2647631899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275640 0x55b7bb264510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 -- 172.21.15.52:0/2647631899 >> 172.21.15.52:0/2647631899 conn(0x55b7bb19acc0 msgr2=0x55b7bb263e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 -- 172.21.15.52:0/2647631899 shutdown_connections 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 -- 172.21.15.52:0/2647631899 wait complete. 2024-06-02T23:12:40.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.784+0000 7fa385166b80 1 -- start start 2024-06-02T23:12:40.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 Processor -- start 2024-06-02T23:12:40.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb283440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb283980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb27ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa37406ffa0 con 0x55b7bb275f60 2024-06-02T23:12:40.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa37406f8d0 con 0x55b7bb26ff20 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa37419c620 con 0x55b7bb275640 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa373fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb283440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa385166b80 10 client.0 _ll_register_callbacks cb 0x55b7bb15b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa373fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb283440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51394/0 (socket says 172.21.15.52:51394) 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.785+0000 7fa373fff700 1 -- 172.21.15.52:0/1211418588 learned_addr learned my addr 172.21.15.52:0/1211418588 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa3737fe700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb283980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa378e80700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb27ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa373fff700 1 -- 172.21.15.52:0/1211418588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 msgr2=0x55b7bb283980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa373fff700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb283980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa373fff700 1 -- 172.21.15.52:0/1211418588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 msgr2=0x55b7bb27ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:40.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa373fff700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb27ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa373fff700 1 -- 172.21.15.52:0/1211418588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7bb27f310 con 0x55b7bb26ff20 2024-06-02T23:12:40.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa378e80700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb27ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa373fff700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb283440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa364009eb0 tx=0x7fa364003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd5c15eadbd96cf3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa3537fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa3537fe700 1 -- 172.21.15.52:0/1211418588 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa364004be0 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa364003d00 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa3537fe700 1 -- 172.21.15.52:0/1211418588 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa364004d80 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa3537fe700 1 -- 172.21.15.52:0/1211418588 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa364005040 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa385166b80 10 client.25427 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.786+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b7bb304bf0 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa3537fe700 1 -- 172.21.15.52:0/1211418588 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa364003d00 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa3537fe700 1 -- 172.21.15.52:0/1211418588 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fa364024020 con 0x55b7bb26ff20 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa385166b80 10 client.25427 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa385166b80 10 client.25427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa385166b80 20 client.25427 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:40.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa385166b80 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb3051a0 0x55b7bb307590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa385166b80 4 client.25427 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b7bb1c9290 con 0x55b7bb3051a0 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.787+0000 7fa3737fe700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb3051a0 0x55b7bb307590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa3537fe700 10 client.25427 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa3737fe700 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb3051a0 0x55b7bb307590 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1200c334981c9ecb server_cookie=8839bbad6bbc5fee in_seq=0 out_seq=0 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa3537fe700 1 -- 172.21.15.52:0/1211418588 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b7bb1c9290 con 0x55b7bb3051a0 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa3537fe700 10 client.25427 handle_command_reply: tid=0 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa385166b80 1 client.25427 shutdown 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 msgr2=0x55b7bb283440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa385166b80 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb283440 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa364009eb0 tx=0x7fa364003c40 comp rx=0 tx=0).stop 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 shutdown_connections 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.788+0000 7fa385166b80 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b7bb3051a0 0x55b7bb307590 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa3537fe700 0 client.25427 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:40.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b7bb275f60 0x55b7bb27ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b7bb26ff20 0x55b7bb283440 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 1 --2- 172.21.15.52:0/1211418588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b7bb275640 0x55b7bb283980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 >> 172.21.15.52:0/1211418588 conn(0x55b7bb19acc0 msgr2=0x55b7bb276340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 shutdown_connections 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 1 -- 172.21.15.52:0/1211418588 wait complete. 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa385166b80 20 client.25427 trim_cache size 0 max 16384 2024-06-02T23:12:40.792 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2ca0 msgr2=0x7fa35c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.789+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2ca0 0x7fa35c0a5160 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa368005f50 tx=0x7fa368028000 comp rx=0 tx=0).stop 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 msgr2=0x7fa37410f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 0x7fa37410f720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa36c007d70 tx=0x7fa36c00a340 comp rx=0 tx=0).stop 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 shutdown_connections 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2ca0 0x7fa35c0a5160 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3741073d0 0x7fa37410f720 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa374101390 0x7fa37410eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 --2- 172.21.15.52:0/1658369798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa374101d30 0x7fa37410f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:40.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 >> 172.21.15.52:0/1658369798 conn(0x7fa3740782c0 msgr2=0x7fa374100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:40.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 shutdown_connections 2024-06-02T23:12:40.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:40.790+0000 7fa37a8e3700 1 -- 172.21.15.52:0/1658369798 wait complete. 2024-06-02T23:12:41.194 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:40 smithi052 ceph-mon[32340]: pgmap v455: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:12:41.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:40 smithi123 ceph-mon[35977]: pgmap v455: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:12:41.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:40 smithi174 ceph-mon[35994]: pgmap v455: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 -- 172.21.15.52:0/3733098601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 msgr2=0x7f7dd0106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 --2- 172.21.15.52:0/3733098601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd0106020 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f7dc00099c0 tx=0x7f7dc0009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 -- 172.21.15.52:0/3733098601 shutdown_connections 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 --2- 172.21.15.52:0/3733098601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd0106750 0x7f7dd010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 --2- 172.21.15.52:0/3733098601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd0101df0 0x7f7dd01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 --2- 172.21.15.52:0/3733098601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd0106020 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.082+0000 7f7dd7758700 1 -- 172.21.15.52:0/3733098601 >> 172.21.15.52:0/3733098601 conn(0x7f7dd0077a00 msgr2=0x7f7dd00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.083+0000 7f7dd7758700 1 -- 172.21.15.52:0/3733098601 shutdown_connections 2024-06-02T23:12:42.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.083+0000 7f7dd7758700 1 -- 172.21.15.52:0/3733098601 wait complete. 2024-06-02T23:12:42.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.083+0000 7f7dd7758700 1 Processor -- start 2024-06-02T23:12:42.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 -- start start 2024-06-02T23:12:42.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd0101df0 0x7f7dd019d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:42.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd019dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:42.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd0106750 0x7f7dd0197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:42.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd0111560 con 0x7f7dd01027a0 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01113e0 con 0x7f7dd0106750 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd7758700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7dd01116e0 con 0x7f7dd0101df0 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd4cf3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd019dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd4cf3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd019dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57900/0 (socket says 172.21.15.52:57900) 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd4cf3700 1 -- 172.21.15.52:0/2516875548 learned_addr learned my addr 172.21.15.52:0/2516875548 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd54f4700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd0101df0 0x7f7dd019d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:42.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.084+0000 7f7dd5cf5700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd0106750 0x7f7dd0197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd4cf3700 1 -- 172.21.15.52:0/2516875548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd0101df0 msgr2=0x7f7dd019d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd4cf3700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd0101df0 0x7f7dd019d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd4cf3700 1 -- 172.21.15.52:0/2516875548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd0106750 msgr2=0x7f7dd0197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd4cf3700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd0106750 0x7f7dd0197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd4cf3700 1 -- 172.21.15.52:0/2516875548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dd0198040 con 0x7f7dd01027a0 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd4cf3700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd019dbd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005b40 tx=0x7f7dc00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0026070 con 0x7f7dd01027a0 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7dc0003f50 con 0x7f7dd01027a0 2024-06-02T23:12:42.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dc000fc20 con 0x7f7dd01027a0 2024-06-02T23:12:42.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.085+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dd01982d0 con 0x7f7dd01027a0 2024-06-02T23:12:42.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.086+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dd0100a80 con 0x7f7dd01027a0 2024-06-02T23:12:42.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.087+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dd0069180 con 0x7f7dd01027a0 2024-06-02T23:12:42.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.088+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7dc0029070 con 0x7f7dd01027a0 2024-06-02T23:12:42.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.089+0000 7f7dc67fc700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7dbc0a2ca0 0x7f7dbc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:42.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.089+0000 7f7dd54f4700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7dbc0a2ca0 0x7f7dbc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:42.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.089+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7dc00cf900 con 0x7f7dd01027a0 2024-06-02T23:12:42.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.090+0000 7f7dd54f4700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7dbc0a2ca0 0x7f7dbc0a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc005950 tx=0x7f7dcc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:42.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.094+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7dc005d850 con 0x7f7dd01027a0 2024-06-02T23:12:42.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.361+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dd0102b80 con 0x7f7dd01027a0 2024-06-02T23:12:42.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.362+0000 7f7dc67fc700 1 -- 172.21.15.52:0/2516875548 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f7dc0013070 con 0x7f7dd01027a0 2024-06-02T23:12:42.363 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:12:42.363 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:12:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7dbc0a2ca0 msgr2=0x7f7dbc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:42.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7dbc0a2ca0 0x7f7dbc0a5160 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7dcc005950 tx=0x7f7dcc00b410 comp rx=0 tx=0).stop 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 msgr2=0x7f7dd019dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd019dbd0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f7dc0005b40 tx=0x7f7dc00057c0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 shutdown_connections 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7dbc0a2ca0 0x7f7dbc0a5160 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7dd0106750 0x7f7dd0197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dd0101df0 0x7f7dd019d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 --2- 172.21.15.52:0/2516875548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7dd01027a0 0x7f7dd019dbd0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 >> 172.21.15.52:0/2516875548 conn(0x7f7dd0077a00 msgr2=0x7f7dd00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 shutdown_connections 2024-06-02T23:12:42.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:42.365+0000 7f7dd7758700 1 -- 172.21.15.52:0/2516875548 wait complete. 2024-06-02T23:12:42.367 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:12:43.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:42 smithi052 ceph-mon[32340]: pgmap v456: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:12:43.227 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:42 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2516875548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:43.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:42 smithi123 ceph-mon[35977]: pgmap v456: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:12:43.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:42 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2516875548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:43.263 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 33d68cc8 --release --await' 2024-06-02T23:12:43.263 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 33d68cc8 --release --await 2024-06-02T23:12:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:42 smithi174 ceph-mon[35994]: pgmap v456: 97 pgs: 97 active+clean; 671 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-06-02T23:12:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:42 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2516875548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:43.547 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:43.638 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:12:43.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:12:44.899 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:45.017 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:44 smithi052 ceph-mon[32340]: pgmap v457: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:45.195 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:44 smithi174 ceph-mon[35994]: pgmap v457: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:45.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:44 smithi123 ceph-mon[35977]: pgmap v457: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:45.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 -- 172.21.15.52:0/762288224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101380 msgr2=0x7f0a1c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 --2- 172.21.15.52:0/762288224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101380 0x7f0a1c101800 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009990 tx=0x7f0a10009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 -- 172.21.15.52:0/762288224 shutdown_connections 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 --2- 172.21.15.52:0/762288224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101d40 0x7f0a1c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 --2- 172.21.15.52:0/762288224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a1c1073f0 0x7f0a1c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 --2- 172.21.15.52:0/762288224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101380 0x7f0a1c101800 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 -- 172.21.15.52:0/762288224 >> 172.21.15.52:0/762288224 conn(0x7f0a1c078300 msgr2=0x7f0a1c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 -- 172.21.15.52:0/762288224 shutdown_connections 2024-06-02T23:12:45.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.627+0000 7f0a22448700 1 -- 172.21.15.52:0/762288224 wait complete. 2024-06-02T23:12:45.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.628+0000 7f0a22448700 1 Processor -- start 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.628+0000 7f0a22448700 1 -- start start 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a22448700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 0x7f0a1c1977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a22448700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 0x7f0a1c197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a22448700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a1c1073f0 0x7f0a1c198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 0x7f0a1c1977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 0x7f0a1c197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 0x7f0a1c1977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53256/0 (socket says 172.21.15.52:53256) 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 0x7f0a1c197d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47622/0 (socket says 172.21.15.52:47622) 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a1bfff700 1 -- 172.21.15.52:0/1122114228 learned_addr learned my addr 172.21.15.52:0/1122114228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c111530 con 0x7f0a1c101d40 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1113b0 con 0x7f0a1c101380 2024-06-02T23:12:45.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1116b0 con 0x7f0a1c1073f0 2024-06-02T23:12:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.629+0000 7f0a209e5700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a1c1073f0 0x7f0a1c198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1122114228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a1c1073f0 msgr2=0x7f0a1c198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a1c1073f0 0x7f0a1c198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1122114228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 msgr2=0x7f0a1c1977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 0x7f0a1c1977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1122114228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1c1a7030 con 0x7f0a1c101d40 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 0x7f0a1c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 0x7f0a1c197d00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009990 tx=0x7f0a1000bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a197fa700 1 -- 172.21.15.52:0/1122114228 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10027070 con 0x7f0a1c101d40 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a197fa700 1 -- 172.21.15.52:0/1122114228 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a10005da0 con 0x7f0a1c101d40 2024-06-02T23:12:45.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a197fa700 1 -- 172.21.15.52:0/1122114228 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a10022bc0 con 0x7f0a1c101d40 2024-06-02T23:12:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.630+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1c1a72c0 con 0x7f0a1c101d40 2024-06-02T23:12:45.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.631+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a1c1a7770 con 0x7f0a1c101d40 2024-06-02T23:12:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.634+0000 7f0a197fa700 1 -- 172.21.15.52:0/1122114228 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a1002a070 con 0x7f0a1c101d40 2024-06-02T23:12:45.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.635+0000 7f0a197fa700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a040a2ca0 0x7f0a040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.635+0000 7f0a197fa700 1 -- 172.21.15.52:0/1122114228 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a100cfdc0 con 0x7f0a1c101d40 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.635+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a040a2ca0 0x7f0a040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a040a2ca0 0x7f0a040a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0104e0 tx=0x7f0a0c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 Processor -- start 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 -- start start 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb5c40 0x564885cb6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb6560 0x564885cafe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cb0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564885c01910 con 0x564885cb5c40 2024-06-02T23:12:45.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564885ba9820 con 0x564885cb6560 2024-06-02T23:12:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564885bfd8d0 con 0x564885cb03d0 2024-06-02T23:12:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a209e5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cb0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb6560 0x564885cafe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a209e5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cb0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53836/0 (socket says 172.21.15.52:53836) 2024-06-02T23:12:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.636+0000 7f0a209e5700 1 -- 172.21.15.52:0/3856979631 learned_addr learned my addr 172.21.15.52:0/3856979631 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:45.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb5c40 0x564885cb6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a209e5700 1 -- 172.21.15.52:0/3856979631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb6560 msgr2=0x564885cafe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a209e5700 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb6560 0x564885cafe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a209e5700 1 -- 172.21.15.52:0/3856979631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb5c40 msgr2=0x564885cb6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a209e5700 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb5c40 0x564885cb6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a209e5700 1 -- 172.21.15.52:0/3856979631 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564885ba0050 con 0x564885cb03d0 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a209e5700 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cb0810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011680 tx=0x7f0a14009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ec5c0ded6dca126 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:45.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.637+0000 7f0a027fc700 1 -- 172.21.15.52:0/3856979631 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14019030 con 0x564885cb03d0 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.638+0000 7f0a027fc700 1 -- 172.21.15.52:0/3856979631 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a140092e0 con 0x564885cb03d0 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.638+0000 7f0a027fc700 1 -- 172.21.15.52:0/3856979631 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14007500 con 0x564885cb03d0 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.638+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/3856979631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 msgr2=0x564885cb0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.638+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cb0810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011680 tx=0x7f0a14009920 comp rx=0 tx=0).stop 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/3856979631 shutdown_connections 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cb0810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb5c40 0x564885cb6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/3856979631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb6560 0x564885cafe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/3856979631 >> 172.21.15.52:0/3856979631 conn(0x564885bdadc0 msgr2=0x564885cab120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/3856979631 shutdown_connections 2024-06-02T23:12:45.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/3856979631 wait complete. 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.639+0000 7f0a2cccbb80 1 -- start start 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 Processor -- start 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb03d0 0x564885cbc330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb6560 0x564885cb9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564885cbcfd0 con 0x564885cb03d0 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564885ba1cf0 con 0x564885cb5c40 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564885cbd150 con 0x564885cb6560 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a2cccbb80 10 client.0 _ll_register_callbacks cb 0x564885b9ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a1bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb03d0 0x564885cbc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53278/0 (socket says 172.21.15.52:53278) 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1581820975 learned_addr learned my addr 172.21.15.52:0/1581820975 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.640+0000 7f0a209e5700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb6560 0x564885cb9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1581820975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb6560 msgr2=0x564885cb9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb6560 0x564885cb9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1581820975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb03d0 msgr2=0x564885cbc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb03d0 0x564885cbc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/1581820975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564885b509d0 con 0x564885cb5c40 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb03d0 0x564885cbc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb9630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a1009cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bd9249aaa82d49 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:45.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f09fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f09fb7fe700 1 -- 172.21.15.52:0/1581820975 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a1009cd80 con 0x564885cb5c40 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a140092e0 con 0x564885cb5c40 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.641+0000 7f09fb7fe700 1 -- 172.21.15.52:0/1581820975 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a10022470 con 0x564885cb5c40 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.642+0000 7f09fb7fe700 1 -- 172.21.15.52:0/1581820975 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a1009c680 con 0x564885cb5c40 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.642+0000 7f0a2cccbb80 10 client.25445 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.642+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564885cc7750 con 0x564885cb5c40 2024-06-02T23:12:45.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.642+0000 7f09fb7fe700 1 -- 172.21.15.52:0/1581820975 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a100a0a30 con 0x564885cb5c40 2024-06-02T23:12:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f09fb7fe700 1 -- 172.21.15.52:0/1581820975 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0a100a1bc0 con 0x564885cb5c40 2024-06-02T23:12:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f0a2cccbb80 10 client.25445 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:45.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f0a2cccbb80 10 client.25445 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f0a2cccbb80 20 client.25445 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885ccc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f0a2cccbb80 4 client.25445 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.643+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0a140092e0 con 0x564885cc9e80 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.644+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885ccc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.644+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885ccc2a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31d97ea2b4e10ae server_cookie=b580cc47dfc6ac2e in_seq=0 out_seq=0 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.644+0000 7f09fb7fe700 10 client.25445 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.645+0000 7f09fb7fe700 1 -- 172.21.15.52:0/1581820975 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f0a140092e0 con 0x564885cc9e80 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.645+0000 7f09fb7fe700 10 client.25445 handle_command_reply: tid=0 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.645+0000 7f0a2cccbb80 1 client.25445 shutdown 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.645+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 msgr2=0x564885cb9630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.645+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb9630 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0a10000c00 tx=0x7f0a1009cbc0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 shutdown_connections 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885ccc2a0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb6560 0x564885cb9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f09fb7fe700 0 client.25445 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb03d0 0x564885cbc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/1581820975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb9630 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 >> 172.21.15.52:0/1581820975 conn(0x564885bdadc0 msgr2=0x564885caad10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 shutdown_connections 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/1581820975 wait complete. 2024-06-02T23:12:45.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.646+0000 7f0a2cccbb80 20 client.25445 trim_cache size 0 max 16384 2024-06-02T23:12:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 Processor -- start 2024-06-02T23:12:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 -- start start 2024-06-02T23:12:45.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cafe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885caaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c072f50 con 0x564885cb6560 2024-06-02T23:12:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c1a7db0 con 0x564885cb5c40 2024-06-02T23:12:45.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c073620 con 0x564885cb03d0 2024-06-02T23:12:45.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a1b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53294/0 (socket says 172.21.15.52:53294) 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/4157753114 learned_addr learned my addr 172.21.15.52:0/4157753114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cafe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.687+0000 7f0a209e5700 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885caaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/4157753114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 msgr2=0x564885cafe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cafe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/4157753114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 msgr2=0x564885caaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885caaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a1b7fe700 1 -- 172.21.15.52:0/4157753114 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564885cab620 con 0x564885cb5c40 2024-06-02T23:12:45.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a209e5700 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885caaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb8590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a1009c380 tx=0x7f0a1009c0a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71ab147f66f2e832 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a01ffb700 1 -- 172.21.15.52:0/4157753114 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100a0a30 con 0x564885cb5c40 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a01ffb700 1 -- 172.21.15.52:0/4157753114 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a100a5260 con 0x564885cb5c40 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.688+0000 7f0a01ffb700 1 -- 172.21.15.52:0/4157753114 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a100985a0 con 0x564885cb5c40 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/4157753114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 msgr2=0x564885cb8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb8590 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a1009c380 tx=0x7f0a1009c0a0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/4157753114 shutdown_connections 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885caaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cafe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/4157753114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cb8590 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/4157753114 >> 172.21.15.52:0/4157753114 conn(0x564885bdadc0 msgr2=0x564885caa830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:45.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.689+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/4157753114 shutdown_connections 2024-06-02T23:12:45.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.690+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/4157753114 wait complete. 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.690+0000 7f0a2cccbb80 1 -- start start 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.690+0000 7f0a2cccbb80 1 Processor -- start 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cc2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cc2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885cbf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c0743c0 con 0x564885cb6560 2024-06-02T23:12:45.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c073cf0 con 0x564885cb5c40 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1c070fa0 con 0x564885cb03d0 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885cbf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885cbf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47656/0 (socket says 172.21.15.52:47656) 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 -- 172.21.15.52:0/2768002123 learned_addr learned my addr 172.21.15.52:0/2768002123 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a2cccbb80 10 client.0 _ll_register_callbacks cb 0x564885b9ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a1b7fe700 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cc2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 -- 172.21.15.52:0/2768002123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 msgr2=0x564885cc2620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cc2620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 -- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 msgr2=0x564885cc2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:45.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cc2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 -- 172.21.15.52:0/2768002123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564885cbfc50 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.691+0000 7f0a209e5700 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885cbf410 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011650 tx=0x7f0a14009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc0dc0cf65c0a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.692+0000 7f09fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.692+0000 7f09fb7fe700 1 -- 172.21.15.52:0/2768002123 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a14003e10 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.692+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a100a5260 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.692+0000 7f09fb7fe700 1 -- 172.21.15.52:0/2768002123 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a14003fb0 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f09fb7fe700 1 -- 172.21.15.52:0/2768002123 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0a1402f050 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 10 client.15861 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x564885d355d0 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f09fb7fe700 1 -- 172.21.15.52:0/2768002123 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a140079b0 con 0x564885cb6560 2024-06-02T23:12:45.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f09fb7fe700 1 -- 172.21.15.52:0/2768002123 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0a140157a0 con 0x564885cb6560 2024-06-02T23:12:45.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 10 client.15861 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:45.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 10 client.15861 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:12:45.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 20 client.15861 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:45.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885d37a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:45.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 4 client.15861 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "33d68cc8", "release": true, "await": true}] 2024-06-02T23:12:45.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.693+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "33d68cc8", "release": true, "await": true}) -- 0x7f0a100a5260 con 0x564885cc9e80 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.694+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885d37a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.695+0000 7f0a1bfff700 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885d37a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318880ef2b704831 server_cookie=f6e948db3fbb315b in_seq=0 out_seq=0 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.695+0000 7f09fb7fe700 10 client.15861 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.697+0000 7f09fb7fe700 1 -- 172.21.15.52:0/2768002123 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f0a100a5260 con 0x564885cc9e80 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.697+0000 7f09fb7fe700 10 client.15861 handle_command_reply: tid=0 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.697+0000 7f0a2cccbb80 1 client.15861 shutdown 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.697+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 msgr2=0x564885cbf410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.697+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885cbf410 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0a14011650 tx=0x7f0a14009510 comp rx=0 tx=0).stop 2024-06-02T23:12:45.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 shutdown_connections 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564885cc9e80 0x564885d37a80 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564885cb6560 0x564885cbf410 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564885cb03d0 0x564885cc2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 --2- 172.21.15.52:0/2768002123 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564885cb5c40 0x564885cc2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 >> 172.21.15.52:0/2768002123 conn(0x564885bdadc0 msgr2=0x564885cb92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f09fb7fe700 0 client.15861 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 shutdown_connections 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 1 -- 172.21.15.52:0/2768002123 wait complete. 2024-06-02T23:12:45.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.698+0000 7f0a2cccbb80 20 client.15861 trim_cache size 0 max 16384 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 28, 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 4, 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "33d68cc8": { 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 4, 2024-06-02T23:12:45.701 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 146.9, 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:12:45.702 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:12:45.703 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:45.703 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:45.703 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:45.703 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:45.703 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:12:45.703 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:12:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.699+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a040a2ca0 msgr2=0x7f0a040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a040a2ca0 0x7f0a040a5160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c0104e0 tx=0x7f0a0c00b480 comp rx=0 tx=0).stop 2024-06-02T23:12:45.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 msgr2=0x7f0a1c197d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 0x7f0a1c197d00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009990 tx=0x7f0a1000bd80 comp rx=0 tx=0).stop 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 shutdown_connections 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a040a2ca0 0x7f0a040a5160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a1c1073f0 0x7f0a1c198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a1c101380 0x7f0a1c1977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 --2- 172.21.15.52:0/1122114228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a1c101d40 0x7f0a1c197d00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 >> 172.21.15.52:0/1122114228 conn(0x7f0a1c078300 msgr2=0x7f0a1c100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.700+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 shutdown_connections 2024-06-02T23:12:45.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:45.701+0000 7f0a22448700 1 -- 172.21.15.52:0/1122114228 wait complete. 2024-06-02T23:12:47.160 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '33d68cc8', total seconds elapsed: 39.8 2024-06-02T23:12:47.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:46 smithi123 ceph-mon[35977]: pgmap v458: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:47.352 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:46 smithi052 ceph-mon[32340]: pgmap v458: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:46 smithi174 ceph-mon[35994]: pgmap v458: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:48.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 -- 172.21.15.52:0/2753638834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c102ea0 msgr2=0x7ffa5c10f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 --2- 172.21.15.52:0/2753638834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c10f380 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400b0a0 tx=0x7ffa5400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 -- 172.21.15.52:0/2753638834 shutdown_connections 2024-06-02T23:12:48.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 --2- 172.21.15.52:0/2753638834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c10f380 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 --2- 172.21.15.52:0/2753638834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c1085f0 0x7ffa5c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 --2- 172.21.15.52:0/2753638834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c1024e0 0x7ffa5c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 -- 172.21.15.52:0/2753638834 >> 172.21.15.52:0/2753638834 conn(0x7ffa5c079640 msgr2=0x7ffa5c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:48.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 -- 172.21.15.52:0/2753638834 shutdown_connections 2024-06-02T23:12:48.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.275+0000 7ffa63652700 1 -- 172.21.15.52:0/2753638834 wait complete. 2024-06-02T23:12:48.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 Processor -- start 2024-06-02T23:12:48.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 -- start start 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c1024e0 0x7ffa5c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c1085f0 0x7ffa5c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c112640 con 0x7ffa5c1085f0 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1124c0 con 0x7ffa5c102ea0 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa63652700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1127c0 con 0x7ffa5c1024e0 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa60bed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa61bef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c1085f0 0x7ffa5c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa60bed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53316/0 (socket says 172.21.15.52:53316) 2024-06-02T23:12:48.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa60bed700 1 -- 172.21.15.52:0/2328062033 learned_addr learned my addr 172.21.15.52:0/2328062033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.276+0000 7ffa613ee700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c1024e0 0x7ffa5c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa60bed700 1 -- 172.21.15.52:0/2328062033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c1024e0 msgr2=0x7ffa5c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa60bed700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c1024e0 0x7ffa5c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa60bed700 1 -- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c1085f0 msgr2=0x7ffa5c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa60bed700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c1085f0 0x7ffa5c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa60bed700 1 -- 172.21.15.52:0/2328062033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5c199040 con 0x7ffa5c102ea0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa61bef700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c1085f0 0x7ffa5c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa60bed700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c19ec10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00d950 tx=0x7ffa4c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa527fc700 1 -- 172.21.15.52:0/2328062033 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c016070 con 0x7ffa5c102ea0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5c199330 con 0x7ffa5c102ea0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa527fc700 1 -- 172.21.15.52:0/2328062033 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c00fd40 con 0x7ffa5c102ea0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa527fc700 1 -- 172.21.15.52:0/2328062033 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00e3b0 con 0x7ffa5c102ea0 2024-06-02T23:12:48.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.277+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5c1a55a0 con 0x7ffa5c102ea0 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.280+0000 7ffa527fc700 1 -- 172.21.15.52:0/2328062033 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa4c019070 con 0x7ffa5c102ea0 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa527fc700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa480a2c20 0x7ffa480a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa613ee700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa480a2c20 0x7ffa480a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa527fc700 1 -- 172.21.15.52:0/2328062033 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c0ceee0 con 0x7ffa5c102ea0 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa6ded5b80 1 Processor -- start 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa6ded5b80 1 -- start start 2024-06-02T23:12:48.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x5592496413c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x55924963b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.281+0000 7ffa613ee700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa480a2c20 0x7ffa480a50e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005950 tx=0x7ffa5800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924963bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55924952dfe0 con 0x55924963b820 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55924958c8d0 con 0x559249641900 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592494f44e0 con 0x559249640fe0 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924963bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924963bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47686/0 (socket says 172.21.15.52:47686) 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 -- 172.21.15.52:0/1703796989 learned_addr learned my addr 172.21.15.52:0/1703796989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa613ee700 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x5592496413c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa60bed700 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x55924963b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 -- 172.21.15.52:0/1703796989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 msgr2=0x5592496413c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x5592496413c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 -- 172.21.15.52:0/1703796989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 msgr2=0x55924963b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x55924963b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 -- 172.21.15.52:0/1703796989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592494db950 con 0x55924963b820 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa613ee700 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x5592496413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa61bef700 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924963bc40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009e40 tx=0x7ffa54004a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ff399d6e42a48d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa477fe700 1 -- 172.21.15.52:0/1703796989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa54004e50 con 0x55924963b820 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.282+0000 7ffa477fe700 1 -- 172.21.15.52:0/1703796989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa54007990 con 0x55924963b820 2024-06-02T23:12:48.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa477fe700 1 -- 172.21.15.52:0/1703796989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa540055f0 con 0x55924963b820 2024-06-02T23:12:48.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1703796989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 msgr2=0x55924963bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924963bc40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009e40 tx=0x7ffa54004a30 comp rx=0 tx=0).stop 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1703796989 shutdown_connections 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924963bc40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x5592496413c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1703796989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x55924963b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1703796989 >> 172.21.15.52:0/1703796989 conn(0x559249568100 msgr2=0x5592496362a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1703796989 shutdown_connections 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.283+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1703796989 wait complete. 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 -- start start 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 Processor -- start 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55924963b820 0x5592496475f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 0x5592496448f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x559249644e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55924952d040 con 0x559249640fe0 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559249588890 con 0x559249641900 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.284+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559249648280 con 0x55924963b820 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa6ded5b80 10 client.0 _ll_register_callbacks cb 0x559249522630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 0x5592496448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 0x5592496448f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47690/0 (socket says 172.21.15.52:47690) 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 -- 172.21.15.52:0/2013075360 learned_addr learned my addr 172.21.15.52:0/2013075360 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa613ee700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55924963b820 0x5592496475f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa61bef700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x559249644e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 -- 172.21.15.52:0/2013075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55924963b820 msgr2=0x5592496475f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55924963b820 0x5592496475f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 -- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 msgr2=0x559249644e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x559249644e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 -- 172.21.15.52:0/2013075360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592495527d0 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa61bef700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x559249644e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa60bed700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 0x5592496448f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09b410 tx=0x7ffa4c09be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e93a220e2b8223 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa44ff9700 1 -- 172.21.15.52:0/2013075360 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0b4060 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa44ff9700 1 -- 172.21.15.52:0/2013075360 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c010460 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.285+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa54007990 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa44ff9700 1 -- 172.21.15.52:0/2013075360 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c020050 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 10 client.15873 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x559249652990 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa44ff9700 1 -- 172.21.15.52:0/2013075360 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa54007990 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa44ff9700 1 -- 172.21.15.52:0/2013075360 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffa4c09b970 con 0x559249640fe0 2024-06-02T23:12:48.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 20 client.15873 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559249654f20 0x559249657310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 4 client.15873 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559249599dd0 con 0x559249654f20 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.286+0000 7ffa613ee700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559249654f20 0x559249657310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.287+0000 7ffa613ee700 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559249654f20 0x559249657310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93f7756fe6a2a5b2 server_cookie=fc1e384dd790054b in_seq=0 out_seq=0 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.287+0000 7ffa44ff9700 10 client.15873 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.287+0000 7ffa44ff9700 1 -- 172.21.15.52:0/2013075360 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559249599dd0 con 0x559249654f20 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.287+0000 7ffa44ff9700 10 client.15873 handle_command_reply: tid=0 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 client.15873 shutdown 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 msgr2=0x5592496448f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 0x5592496448f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09b410 tx=0x7ffa4c09be10 comp rx=0 tx=0).stop 2024-06-02T23:12:48.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 shutdown_connections 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559249654f20 0x559249657310 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x559249644e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55924963b820 0x5592496475f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa44ff9700 0 client.15873 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/2013075360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249640fe0 0x5592496448f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 >> 172.21.15.52:0/2013075360 conn(0x559249568100 msgr2=0x559249635780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 shutdown_connections 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/2013075360 wait complete. 2024-06-02T23:12:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.288+0000 7ffa6ded5b80 20 client.15873 trim_cache size 0 max 16384 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.309+0000 7ffa6ded5b80 1 Processor -- start 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.309+0000 7ffa6ded5b80 1 -- start start 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55924963b820 0x55924963b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x559249635f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 0x559249636440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1ac980 con 0x559249641900 2024-06-02T23:12:48.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1ae4c0 con 0x55924963b820 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1ad050 con 0x559249640fe0 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa61bef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 0x559249636440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa61bef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 0x559249636440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47694/0 (socket says 172.21.15.52:47694) 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa61bef700 1 -- 172.21.15.52:0/1396864500 learned_addr learned my addr 172.21.15.52:0/1396864500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa60bed700 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x559249635f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa613ee700 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55924963b820 0x55924963b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa60bed700 1 -- 172.21.15.52:0/1396864500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55924963b820 msgr2=0x55924963b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa60bed700 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55924963b820 0x55924963b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa60bed700 1 -- 172.21.15.52:0/1396864500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 msgr2=0x559249636440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa60bed700 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 0x559249636440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa60bed700 1 -- 172.21.15.52:0/1396864500 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559249642e80 con 0x559249640fe0 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.310+0000 7ffa61bef700 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 0x559249636440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.311+0000 7ffa60bed700 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x559249635f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0980e0 tx=0x7ffa4c09b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=55ded7425a929f67 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.311+0000 7ffa46ffd700 1 -- 172.21.15.52:0/1396864500 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0b4040 con 0x559249640fe0 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.311+0000 7ffa46ffd700 1 -- 172.21.15.52:0/1396864500 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c0a4380 con 0x559249640fe0 2024-06-02T23:12:48.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.311+0000 7ffa46ffd700 1 -- 172.21.15.52:0/1396864500 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c09b970 con 0x559249640fe0 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1396864500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 msgr2=0x559249635f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x559249635f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0980e0 tx=0x7ffa4c09b610 comp rx=0 tx=0).stop 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1396864500 shutdown_connections 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559249641900 0x559249636440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55924963b820 0x55924963b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/1396864500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x559249635f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1396864500 >> 172.21.15.52:0/1396864500 conn(0x559249568100 msgr2=0x55924963fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1396864500 shutdown_connections 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/1396864500 wait complete. 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 -- start start 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.312+0000 7ffa6ded5b80 1 Processor -- start 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924964a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x55924964ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x5592496c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1addf0 con 0x55924963b820 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1ad720 con 0x559249641900 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5c1aa920 con 0x559249640fe0 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa6ded5b80 10 client.0 _ll_register_callbacks cb 0x559249522630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924964a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924964a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47702/0 (socket says 172.21.15.52:47702) 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 -- 172.21.15.52:0/159114253 learned_addr learned my addr 172.21.15.52:0/159114253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa60bed700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x55924964ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa61bef700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x5592496c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 -- 172.21.15.52:0/159114253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 msgr2=0x55924964ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x55924964ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 -- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 msgr2=0x5592496c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x5592496c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa613ee700 1 -- 172.21.15.52:0/159114253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592496c44f0 con 0x55924963b820 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.313+0000 7ffa61bef700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x5592496c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa613ee700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924964a6f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005c80 tx=0x7ffa580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1679d30f563849b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa44ff9700 1 -- 172.21.15.52:0/159114253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58016070 con 0x55924963b820 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa44ff9700 1 -- 172.21.15.52:0/159114253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa58010430 con 0x55924963b820 2024-06-02T23:12:48.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c0a4380 con 0x55924963b820 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa44ff9700 1 -- 172.21.15.52:0/159114253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7ffa58026ca0 con 0x55924963b820 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa6ded5b80 10 client.15879 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5592496c5750 con 0x55924963b820 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa44ff9700 1 -- 172.21.15.52:0/159114253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c0a4380 con 0x55924963b820 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa44ff9700 1 -- 172.21.15.52:0/159114253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7ffa5802f7a0 con 0x55924963b820 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.314+0000 7ffa6ded5b80 10 client.15879 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa6ded5b80 10 client.15879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa6ded5b80 20 client.15879 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592496c5e60 0x5592496c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa6ded5b80 4 client.15879 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559249594350 con 0x5592496c5e60 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa60bed700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592496c5e60 0x5592496c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa60bed700 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592496c5e60 0x5592496c8280 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=963cde76137115c2 server_cookie=95a6caab9beebd1f in_seq=0 out_seq=0 2024-06-02T23:12:48.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.315+0000 7ffa44ff9700 10 client.15879 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:48.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa44ff9700 1 -- 172.21.15.52:0/159114253 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559249594350 con 0x5592496c5e60 2024-06-02T23:12:48.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa44ff9700 10 client.15879 handle_command_reply: tid=0 2024-06-02T23:12:48.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 client.15879 shutdown 2024-06-02T23:12:48.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 msgr2=0x55924964a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924964a6f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005c80 tx=0x7ffa580047c0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 shutdown_connections 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592496c5e60 0x5592496c8280 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559249641900 0x5592496c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55924963b820 0x55924964a6f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 --2- 172.21.15.52:0/159114253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559249640fe0 0x55924964ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 >> 172.21.15.52:0/159114253 conn(0x559249568100 msgr2=0x5592496402d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "fe47bfc6-471a-4907-945b-33a0a4d46b04", 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:12:48.321 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:12:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa44ff9700 0 client.15879 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 shutdown_connections 2024-06-02T23:12:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 1 -- 172.21.15.52:0/159114253 wait complete. 2024-06-02T23:12:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.317+0000 7ffa6ded5b80 20 client.15879 trim_cache size 0 max 16384 2024-06-02T23:12:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa480a2c20 msgr2=0x7ffa480a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa480a2c20 0x7ffa480a50e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7ffa58005950 tx=0x7ffa5800a300 comp rx=0 tx=0).stop 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 msgr2=0x7ffa5c19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c19ec10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c00d950 tx=0x7ffa4c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 shutdown_connections 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa480a2c20 0x7ffa480a50e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa5c1085f0 0x7ffa5c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5c1024e0 0x7ffa5c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 --2- 172.21.15.52:0/2328062033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa5c102ea0 0x7ffa5c19ec10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 >> 172.21.15.52:0/2328062033 conn(0x7ffa5c079640 msgr2=0x7ffa5c100f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 shutdown_connections 2024-06-02T23:12:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:48.318+0000 7ffa63652700 1 -- 172.21.15.52:0/2328062033 wait complete. 2024-06-02T23:12:48.471 INFO:tasks.workunit.client.0.smithi052.stderr:Updating files: 91% (11554/12565) Updating files: 92% (11560/12565) Updating files: 93% (11686/12565) Updating files: 94% (11812/12565) Updating files: 95% (11937/12565) Updating files: 96% (12063/12565) Updating files: 97% (12189/12565) Updating files: 98% (12314/12565) Updating files: 99% (12440/12565) Updating files: 100% (12565/12565) Updating files: 100% (12565/12565), done. 2024-06-02T23:12:49.116 INFO:tasks.workunit.client.0.smithi052.stderr:Note: switching to 'dfcc4b3c2c55532c4c04fa47551c9df1dffdf746'. 2024-06-02T23:12:49.116 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.116 INFO:tasks.workunit.client.0.smithi052.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr:changes and commit them, and you can discard any commits you make in this 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr:state without impacting any branches by switching back to a branch. 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr:If you want to create a new branch to retain commits you create, you may 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr:do so (now or later) by using -c with the switch command. Example: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: git switch -c 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr:Or undo this operation with: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: git switch - 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.117 INFO:tasks.workunit.client.0.smithi052.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-06-02T23:12:49.118 INFO:tasks.workunit.client.0.smithi052.stderr: 2024-06-02T23:12:49.118 INFO:tasks.workunit.client.0.smithi052.stderr:HEAD is now at dfcc4b3c2c5 Merge pull request #57814 from zdover23/wip-doc-2024-06-01-backport-57804-to-squid 2024-06-02T23:12:49.121 DEBUG:teuthology.orchestra.run.smithi052:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-06-02T23:12:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:48 smithi052 ceph-mon[32340]: pgmap v459: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:48 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:12:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:48 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:48 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:12:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:48 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:49.186 INFO:tasks.workunit.client.0.smithi052.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-06-02T23:12:49.189 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-02T23:12:49.189 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-06-02T23:12:49.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:12:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:48 smithi123 ceph-mon[35977]: pgmap v459: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:48 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:12:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:48 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:48 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:12:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:48 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:48 smithi174 ceph-mon[35994]: pgmap v459: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:48 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:12:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:48 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:48 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:12:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:48 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:12:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:50 smithi052 ceph-mon[32340]: pgmap v460: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:50.994 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:50 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:12:51.106 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-06-02T23:12:51.234 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-06-02T23:12:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:50 smithi123 ceph-mon[35977]: pgmap v460: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:51.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:50 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:12:51.276 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-06-02T23:12:51.278 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-02T23:12:51.278 INFO:tasks.workunit.client.0.smithi052.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-06-02T23:12:51.320 INFO:tasks.workunit.client.0.smithi052.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-06-02T23:12:51.324 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:12:51.324 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-06-02T23:12:51.341 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-06-02T23:12:51.342 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-06-02T23:12:51.343 DEBUG:teuthology.orchestra.run.smithi052:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=dfcc4b3c2c55532c4c04fa47551c9df1dffdf746 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-06-02T23:12:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:50 smithi174 ceph-mon[35994]: pgmap v460: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:50 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:12:51.428 INFO:tasks.workunit.client.0.smithi052.stderr:+ dbench 1 2024-06-02T23:12:51.483 INFO:tasks.workunit.client.0.smithi052.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-02T23:12:51.483 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-02T23:12:51.483 INFO:tasks.workunit.client.0.smithi052.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-02T23:12:51.489 INFO:tasks.workunit.client.0.smithi052.stdout:0 of 1 processes prepared for launch 0 sec 2024-06-02T23:12:51.589 INFO:tasks.workunit.client.0.smithi052.stdout:1 of 1 processes prepared for launch 0 sec 2024-06-02T23:12:51.589 INFO:tasks.workunit.client.0.smithi052.stdout:releasing clients 2024-06-02T23:12:51.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 -- 172.21.15.52:0/1673814701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 msgr2=0x7f2ac81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 --2- 172.21.15.52:0/1673814701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac81029b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2ab80099c0 tx=0x7f2ab8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 -- 172.21.15.52:0/1673814701 shutdown_connections 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 --2- 172.21.15.52:0/1673814701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac8102ef0 0x7f2ac810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 --2- 172.21.15.52:0/1673814701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac81085d0 0x7f2ac81089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 --2- 172.21.15.52:0/1673814701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac81029b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 -- 172.21.15.52:0/1673814701 >> 172.21.15.52:0/1673814701 conn(0x7f2ac8079640 msgr2=0x7f2ac8079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 -- 172.21.15.52:0/1673814701 shutdown_connections 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.591+0000 7f2acf817700 1 -- 172.21.15.52:0/1673814701 wait complete. 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 Processor -- start 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 -- start start 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac819e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac8102ef0 0x7f2ac819edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac81085d0 0x7f2ac8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81127b0 con 0x7f2ac8102530 2024-06-02T23:12:51.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8112630 con 0x7f2ac8102ef0 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.592+0000 7f2acf817700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac8112930 con 0x7f2ac81085d0 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac819e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac819e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47720/0 (socket says 172.21.15.52:47720) 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 -- 172.21.15.52:0/4054094770 learned_addr learned my addr 172.21.15.52:0/4054094770 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2accdb2700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac8102ef0 0x7f2ac819edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acddb4700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac81085d0 0x7f2ac8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 -- 172.21.15.52:0/4054094770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac81085d0 msgr2=0x7f2ac8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac81085d0 0x7f2ac8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 -- 172.21.15.52:0/4054094770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac8102ef0 msgr2=0x7f2ac819edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac8102ef0 0x7f2ac819edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 -- 172.21.15.52:0/4054094770 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac81991f0 con 0x7f2ac8102530 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acddb4700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac81085d0 0x7f2ac8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2accdb2700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac8102ef0 0x7f2ac819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:12:51.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac819e880 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400b530 tx=0x7f2ac400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2abe7fc700 1 -- 172.21.15.52:0/4054094770 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4015020 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2abe7fc700 1 -- 172.21.15.52:0/4054094770 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac4011ce0 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac81994e0 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2abe7fc700 1 -- 172.21.15.52:0/4054094770 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4010d60 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.593+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac81a57d0 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.596+0000 7f2abe7fc700 1 -- 172.21.15.52:0/4054094770 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ac4018020 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.596+0000 7f2abe7fc700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ab40a2ca0 0x7f2ab40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.596+0000 7f2accdb2700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ab40a2ca0 0x7f2ab40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.597+0000 7f2abe7fc700 1 -- 172.21.15.52:0/4054094770 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ac40ce830 con 0x7f2ac8102530 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.597+0000 7f2accdb2700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ab40a2ca0 0x7f2ab40a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8079c90 tx=0x7f2ab8005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.597+0000 7f2ada09ab80 1 Processor -- start 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.597+0000 7f2ada09ab80 1 -- start start 2024-06-02T23:12:51.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 0x5643f8bf3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf3880 0x5643f8bed260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bed7a0 0x5643f8bedbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5643f8adff60 con 0x5643f8bf2f60 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5643f8b3e850 con 0x5643f8bf3880 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643f8aa6460 con 0x5643f8bed7a0 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 0x5643f8bf3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 0x5643f8bf3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47726/0 (socket says 172.21.15.52:47726) 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 -- 172.21.15.52:0/1596258883 learned_addr learned my addr 172.21.15.52:0/1596258883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2accdb2700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf3880 0x5643f8bed260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acddb4700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bed7a0 0x5643f8bedbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 -- 172.21.15.52:0/1596258883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bed7a0 msgr2=0x5643f8bedbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bed7a0 0x5643f8bedbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 -- 172.21.15.52:0/1596258883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf3880 msgr2=0x5643f8bed260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf3880 0x5643f8bed260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 -- 172.21.15.52:0/1596258883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f8a8d8d0 con 0x5643f8bf2f60 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2accdb2700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf3880 0x5643f8bed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acddb4700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bed7a0 0x5643f8bedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 0x5643f8bf3340 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40a3800 tx=0x7f2ac409ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac58ec34d296244c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.598+0000 7f2ab37fe700 1 -- 172.21.15.52:0/1596258883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac409b7c0 con 0x5643f8bf2f60 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ab37fe700 1 -- 172.21.15.52:0/1596258883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac409b0c0 con 0x5643f8bf2f60 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ab37fe700 1 -- 172.21.15.52:0/1596258883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac40a38a0 con 0x5643f8bf2f60 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 -- 172.21.15.52:0/1596258883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 msgr2=0x5643f8bf3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 0x5643f8bf3340 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40a3800 tx=0x7f2ac409ac00 comp rx=0 tx=0).stop 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 -- 172.21.15.52:0/1596258883 shutdown_connections 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bed7a0 0x5643f8bedbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf2f60 0x5643f8bf3340 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/1596258883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf3880 0x5643f8bed260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 -- 172.21.15.52:0/1596258883 >> 172.21.15.52:0/1596258883 conn(0x5643f8b1a080 msgr2=0x5643f8be8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 -- 172.21.15.52:0/1596258883 shutdown_connections 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.599+0000 7f2ada09ab80 1 -- 172.21.15.52:0/1596258883 wait complete. 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 -- start start 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 Processor -- start 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 0x5643f8bf9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf2f60 0x5643f8bf6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf3880 0x5643f8bf6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5643f8adefc0 con 0x5643f8bed7a0 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5643f8b3a810 con 0x5643f8bf2f60 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643f8bfa260 con 0x5643f8bf3880 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2ada09ab80 10 client.0 _ll_register_callbacks cb 0x5643f8ad45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2acd5b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 0x5643f8bf9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2acd5b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 0x5643f8bf9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47736/0 (socket says 172.21.15.52:47736) 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.600+0000 7f2acd5b3700 1 -- 172.21.15.52:0/37418487 learned_addr learned my addr 172.21.15.52:0/37418487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:51.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2accdb2700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf2f60 0x5643f8bf6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acddb4700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf3880 0x5643f8bf6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acd5b3700 1 -- 172.21.15.52:0/37418487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf3880 msgr2=0x5643f8bf6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf3880 0x5643f8bf6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acd5b3700 1 -- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf2f60 msgr2=0x5643f8bf6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf2f60 0x5643f8bf6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acd5b3700 1 -- 172.21.15.52:0/37418487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f8b04750 con 0x5643f8bed7a0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 0x5643f8bf9560 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40cc3b0 tx=0x7f2ac400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b3c4f44bc01622 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ab0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/37418487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac409b7c0 con 0x5643f8bed7a0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/37418487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ac409f120 con 0x5643f8bed7a0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac409b0c0 con 0x5643f8bed7a0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/37418487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ac40a38a0 con 0x5643f8bed7a0 2024-06-02T23:12:51.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ada09ab80 10 client.15891 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.601+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643f8c048c0 con 0x5643f8bed7a0 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/37418487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac409b0c0 con 0x5643f8bed7a0 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/37418487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ac4096e10 con 0x5643f8bed7a0 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ada09ab80 10 client.15891 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ada09ab80 10 client.15891 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ada09ab80 20 client.15891 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c09180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ada09ab80 4 client.15891 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643f8b4bd50 con 0x5643f8c06d90 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.602+0000 7f2accdb2700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c09180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.603+0000 7f2ab0ff9700 10 client.15891 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.603+0000 7f2accdb2700 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c09180 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5bba5414dce471c2 server_cookie=9ebc38e1b38685b2 in_seq=0 out_seq=0 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.603+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/37418487 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5643f8b4bd50 con 0x5643f8c06d90 2024-06-02T23:12:51.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.603+0000 7f2ab0ff9700 10 client.15891 handle_command_reply: tid=0 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 client.15891 shutdown 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 msgr2=0x5643f8bf9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 0x5643f8bf9560 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f2ac40cc3b0 tx=0x7f2ac400bf70 comp rx=0 tx=0).stop 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 shutdown_connections 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c09180 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ab0ff9700 0 client.15891 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf3880 0x5643f8bf6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bed7a0 0x5643f8bf9560 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/37418487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bf2f60 0x5643f8bf6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 >> 172.21.15.52:0/37418487 conn(0x5643f8b1a080 msgr2=0x5643f8be7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:51.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 shutdown_connections 2024-06-02T23:12:51.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 1 -- 172.21.15.52:0/37418487 wait complete. 2024-06-02T23:12:51.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.604+0000 7f2ada09ab80 20 client.15891 trim_cache size 0 max 16384 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.628+0000 7f2ada09ab80 1 Processor -- start 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.628+0000 7f2ada09ab80 1 -- start start 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8bf87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8be7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8be83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81acc50 con 0x5643f8bf3880 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81ae790 con 0x5643f8bed7a0 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81ad320 con 0x5643f8bf2f60 2024-06-02T23:12:51.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8be83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8be83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47750/0 (socket says 172.21.15.52:47750) 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 -- 172.21.15.52:0/268946050 learned_addr learned my addr 172.21.15.52:0/268946050 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2accdb2700 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8be7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8bf87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 -- 172.21.15.52:0/268946050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 msgr2=0x5643f8be7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8be7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 -- 172.21.15.52:0/268946050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 msgr2=0x5643f8bf87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8bf87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 -- 172.21.15.52:0/268946050 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643f8bf9d40 con 0x5643f8bf3880 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8bf87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.629+0000 7f2acddb4700 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8be83c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000a5c0 tx=0x7f2ac000a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12e6c5b58e19c6b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.630+0000 7f2ab2ffd700 1 -- 172.21.15.52:0/268946050 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac001a020 con 0x5643f8bf3880 2024-06-02T23:12:51.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.630+0000 7f2ab2ffd700 1 -- 172.21.15.52:0/268946050 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac00168a0 con 0x5643f8bf3880 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.630+0000 7f2ab2ffd700 1 -- 172.21.15.52:0/268946050 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0015730 con 0x5643f8bf3880 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.630+0000 7f2ada09ab80 1 -- 172.21.15.52:0/268946050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 msgr2=0x5643f8be83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.630+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8be83c0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000a5c0 tx=0x7f2ac000a5f0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 -- 172.21.15.52:0/268946050 shutdown_connections 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8be83c0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8bf87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/268946050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8be7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 -- 172.21.15.52:0/268946050 >> 172.21.15.52:0/268946050 conn(0x5643f8b1a080 msgr2=0x5643f8bf8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 -- 172.21.15.52:0/268946050 shutdown_connections 2024-06-02T23:12:51.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 -- 172.21.15.52:0/268946050 wait complete. 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 -- start start 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.631+0000 7f2ada09ab80 1 Processor -- start 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8c0a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8c0a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8c0abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81ae0c0 con 0x5643f8bf3880 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81ad9f0 con 0x5643f8bed7a0 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac81aabf0 con 0x5643f8bf2f60 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2ada09ab80 10 client.0 _ll_register_callbacks cb 0x5643f8ad45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8c0abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8c0abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47756/0 (socket says 172.21.15.52:47756) 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 -- 172.21.15.52:0/3213742673 learned_addr learned my addr 172.21.15.52:0/3213742673 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2accdb2700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8c0a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8c0a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 -- 172.21.15.52:0/3213742673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 msgr2=0x5643f8c0a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8c0a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 -- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 msgr2=0x5643f8c0a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8c0a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2acddb4700 1 -- 172.21.15.52:0/3213742673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643f8c708e0 con 0x5643f8bf3880 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.632+0000 7f2accdb2700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8c0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2acddb4700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8c0abd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009670 tx=0x7f2ac0016cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c40f8baca0fafef6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ab0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/3213742673 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0016e60 con 0x5643f8bf3880 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/3213742673 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ac001f830 con 0x5643f8bf3880 2024-06-02T23:12:51.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ac00168a0 con 0x5643f8bf3880 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/3213742673 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2ac0030070 con 0x5643f8bf3880 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ada09ab80 10 client.15897 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5643f8c71b20 con 0x5643f8bf3880 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.633+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/3213742673 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ac00168a0 con 0x5643f8bf3880 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/3213742673 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2ac00343d0 con 0x5643f8bf3880 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ada09ab80 10 client.15897 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ada09ab80 10 client.15897 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ada09ab80 20 client.15897 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c74130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ada09ab80 4 client.15897 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643f8b462d0 con 0x5643f8c06d90 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c74130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2acd5b3700 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c74130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4804ec19d63e9c57 server_cookie=77c5dea6602352bb in_seq=0 out_seq=0 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.634+0000 7f2ab0ff9700 10 client.15897 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ab0ff9700 1 -- 172.21.15.52:0/3213742673 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5643f8b462d0 con 0x5643f8c06d90 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ab0ff9700 10 client.15897 handle_command_reply: tid=0 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 client.15897 shutdown 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 msgr2=0x5643f8c0abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8c0abd0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009670 tx=0x7f2ac0016cf0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 shutdown_connections 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643f8c06d90 0x5643f8c74130 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5643f8bf3880 0x5643f8c0abd0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5643f8bed7a0 0x5643f8c0a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 --2- 172.21.15.52:0/3213742673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5643f8bf2f60 0x5643f8c0a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 >> 172.21.15.52:0/3213742673 conn(0x5643f8b1a080 msgr2=0x5643f8bf8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ab0ff9700 0 client.15897 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 shutdown_connections 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 1 -- 172.21.15.52:0/3213742673 wait complete. 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.635+0000 7f2ada09ab80 20 client.15897 trim_cache size 0 max 16384 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:12:51.640 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ab40a2ca0 msgr2=0x7f2ab40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ab40a2ca0 0x7f2ab40a5160 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ac8079c90 tx=0x7f2ab8005ab0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 msgr2=0x7f2ac819e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac819e880 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400b530 tx=0x7f2ac400b8f0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 shutdown_connections 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ab40a2ca0 0x7f2ab40a5160 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ac81085d0 0x7f2ac8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ac8102530 0x7f2ac819e880 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 --2- 172.21.15.52:0/4054094770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ac8102ef0 0x7f2ac819edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 >> 172.21.15.52:0/4054094770 conn(0x7f2ac8079640 msgr2=0x7f2ac8101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 shutdown_connections 2024-06-02T23:12:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:51.636+0000 7f2acf817700 1 -- 172.21.15.52:0/4054094770 wait complete. 2024-06-02T23:12:52.466 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:12:52.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:12:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.494+0000 7f6418676640 1 -- 172.21.15.52:0/2648385467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6410100900 msgr2=0x7f6410100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.494+0000 7f6418676640 1 --2- 172.21.15.52:0/2648385467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6410100900 0x7f6410100d20 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f640c009900 tx=0x7f640c02a410 comp rx=0 tx=0).stop 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 -- 172.21.15.52:0/2648385467 shutdown_connections 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 --2- 172.21.15.52:0/2648385467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410101af0 0x7f6410101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 --2- 172.21.15.52:0/2648385467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64101024b0 0x7f6410108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 --2- 172.21.15.52:0/2648385467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6410100900 0x7f6410100d20 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 -- 172.21.15.52:0/2648385467 >> 172.21.15.52:0/2648385467 conn(0x7f64100fbe70 msgr2=0x7f64100fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 -- 172.21.15.52:0/2648385467 shutdown_connections 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 -- 172.21.15.52:0/2648385467 wait complete. 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 Processor -- start 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 -- start start 2024-06-02T23:12:52.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.495+0000 7f6418676640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 0x7f6410104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6418676640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410101af0 0x7f6410102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6418676640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64101024b0 0x7f6410103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6418676640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f641010c470 con 0x7f64101024b0 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6418676640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f641010c2f0 con 0x7f6410101af0 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6418676640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f641010c5f0 con 0x7f6410100900 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6416412640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 0x7f6410104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410101af0 0x7f6410102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6416412640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 0x7f6410104890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53974/0 (socket says 172.21.15.52:53974) 2024-06-02T23:12:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6416412640 1 -- 172.21.15.52:0/1096728551 learned_addr learned my addr 172.21.15.52:0/1096728551 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6416c13640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64101024b0 0x7f6410103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 -- 172.21.15.52:0/1096728551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 msgr2=0x7f6410104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 0x7f6410104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 -- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64101024b0 msgr2=0x7f6410103410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64101024b0 0x7f6410103410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 -- 172.21.15.52:0/1096728551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6410103ca0 con 0x7f6410101af0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6416412640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 0x7f6410104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f6415c11640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410101af0 0x7f6410102ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9f0 tx=0x7f640000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.496+0000 7f64077fe640 1 -- 172.21.15.52:0/1096728551 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640000ccc0 con 0x7f6410101af0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.497+0000 7f64077fe640 1 -- 172.21.15.52:0/1096728551 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6400005020 con 0x7f6410101af0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.497+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64101a1cf0 con 0x7f6410101af0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.497+0000 7f64077fe640 1 -- 172.21.15.52:0/1096728551 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400004a00 con 0x7f6410101af0 2024-06-02T23:12:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.497+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64101a2190 con 0x7f6410101af0 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.499+0000 7f64077fe640 1 -- 172.21.15.52:0/1096728551 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6400004ba0 con 0x7f6410101af0 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f64077fe640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63fc0a2c30 0x7f63fc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 Processor -- start 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f6416412640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63fc0a2c30 0x7f63fc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f64077fe640 1 -- 172.21.15.52:0/1096728551 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f64000ceff0 con 0x7f6410101af0 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 -- start start 2024-06-02T23:12:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc691430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc696500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc699000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5556dc5f44f0 con 0x5556dc691010 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5556dc592870 con 0x5556dc696b70 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.500+0000 7f641b388740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556dc5f04b0 con 0x5556dc691a00 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc691430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc691430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47788/0 (socket says 172.21.15.52:47788) 2024-06-02T23:12:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 -- 172.21.15.52:0/3729313003 learned_addr learned my addr 172.21.15.52:0/3729313003 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6415c11640 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc696500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416c13640 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc699000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63fc0a2c30 0x7f63fc0a50f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c004490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 -- 172.21.15.52:0/3729313003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 msgr2=0x5556dc696500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc696500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 -- 172.21.15.52:0/3729313003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 msgr2=0x5556dc699000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc699000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 -- 172.21.15.52:0/3729313003 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556dc5ae9b0 con 0x5556dc691010 2024-06-02T23:12:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6415c11640 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc696500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6416412640 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc691430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f640c02a930 tx=0x7f640c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d326e29bf0e613 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6404ff9640 1 -- 172.21.15.52:0/3729313003 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c035050 con 0x5556dc691010 2024-06-02T23:12:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.501+0000 7f6404ff9640 1 -- 172.21.15.52:0/3729313003 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f640c003e10 con 0x5556dc691010 2024-06-02T23:12:52.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f6404ff9640 1 -- 172.21.15.52:0/3729313003 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c03c940 con 0x5556dc691010 2024-06-02T23:12:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 -- 172.21.15.52:0/3729313003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 msgr2=0x5556dc691430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc691430 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f640c02a930 tx=0x7f640c046000 comp rx=0 tx=0).stop 2024-06-02T23:12:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 -- 172.21.15.52:0/3729313003 shutdown_connections 2024-06-02T23:12:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc696500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc699000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 --2- 172.21.15.52:0/3729313003 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc691430 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.502+0000 7f641b388740 1 -- 172.21.15.52:0/3729313003 >> 172.21.15.52:0/3729313003 conn(0x5556dc68c680 msgr2=0x5556dc68ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.503+0000 7f641b388740 1 -- 172.21.15.52:0/3729313003 shutdown_connections 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.503+0000 7f641b388740 1 -- 172.21.15.52:0/3729313003 wait complete. 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.503+0000 7f641b388740 1 -- start start 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.503+0000 7f641b388740 1 Processor -- start 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.503+0000 7f641b388740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 0x5556dc69b190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f641b388740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691a00 0x5556dc69b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f641b388740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc6a2710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f641b388740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5556dc699ac0 con 0x5556dc691a00 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f641b388740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5556dc57dae0 con 0x5556dc696b70 2024-06-02T23:12:52.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f641b388740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556dc699c40 con 0x5556dc691010 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f641b388740 10 client.0 _ll_register_callbacks cb 0x5556dc5b69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 0x5556dc69b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6415c11640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691a00 0x5556dc69b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 0x5556dc69b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53982/0 (socket says 172.21.15.52:53982) 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 -- 172.21.15.52:0/4147061413 learned_addr learned my addr 172.21.15.52:0/4147061413 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416c13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc6a2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 -- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 msgr2=0x5556dc6a2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc6a2710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 -- 172.21.15.52:0/4147061413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691a00 msgr2=0x5556dc69b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691a00 0x5556dc69b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 -- 172.21.15.52:0/4147061413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556dc308210 con 0x5556dc691010 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6415c11640 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691a00 0x5556dc69b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f6416412640 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 0x5556dc69b190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f640c006010 tx=0x7f640c031f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b3fe4b2b28022c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:52.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.504+0000 7f63f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f63f9ffb640 1 -- 172.21.15.52:0/4147061413 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c058070 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c003e10 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f63f9ffb640 1 -- 172.21.15.52:0/4147061413 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f640c031990 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f63f9ffb640 1 -- 172.21.15.52:0/4147061413 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f640c064050 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f641b388740 10 client.25426 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556dc3d4620 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f63f9ffb640 1 -- 172.21.15.52:0/4147061413 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f640c003e10 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.505+0000 7f63f9ffb640 1 -- 172.21.15.52:0/4147061413 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f640c0495e0 con 0x5556dc691010 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f641b388740 10 client.25426 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f641b388740 10 client.25426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f641b388740 20 client.25426 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f641b388740 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc6a6ef0 0x5556dc6a9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f641b388740 4 client.25426 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556dc607f10 con 0x5556dc6a6ef0 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f6415c11640 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc6a6ef0 0x5556dc6a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f6415c11640 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc6a6ef0 0x5556dc6a9330 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=935556c1c212bcc0 server_cookie=5da28edec01ad01b in_seq=0 out_seq=0 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.506+0000 7f63f9ffb640 10 client.25426 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.507+0000 7f63f9ffb640 1 -- 172.21.15.52:0/4147061413 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5556dc607f10 con 0x5556dc6a6ef0 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.507+0000 7f63f9ffb640 10 client.25426 handle_command_reply: tid=0 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.507+0000 7f641b388740 1 client.25426 shutdown 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 msgr2=0x5556dc69b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 0x5556dc69b190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f640c006010 tx=0x7f640c031f60 comp rx=0 tx=0).stop 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 shutdown_connections 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc6a6ef0 0x5556dc6a9330 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc6a2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691a00 0x5556dc69b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f63f9ffb640 0 client.25426 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 --2- 172.21.15.52:0/4147061413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691010 0x5556dc69b190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 >> 172.21.15.52:0/4147061413 conn(0x5556dc68c680 msgr2=0x5556dc68b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 shutdown_connections 2024-06-02T23:12:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 1 -- 172.21.15.52:0/4147061413 wait complete. 2024-06-02T23:12:52.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.508+0000 7f641b388740 20 client.25426 trim_cache size 0 max 16384 2024-06-02T23:12:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 Processor -- start 2024-06-02T23:12:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 -- start start 2024-06-02T23:12:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc692260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc6a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc68f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64101aaed0 con 0x5556dc691010 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64101aca10 con 0x5556dc696b70 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f641b388740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64101ab5a0 con 0x5556dc691a00 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.525+0000 7f6416412640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc692260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc692260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47806/0 (socket says 172.21.15.52:47806) 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 -- 172.21.15.52:0/2239330678 learned_addr learned my addr 172.21.15.52:0/2239330678 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6415c11640 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc6a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416c13640 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc68f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 -- 172.21.15.52:0/2239330678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 msgr2=0x5556dc6a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc6a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 -- 172.21.15.52:0/2239330678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 msgr2=0x5556dc68f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc68f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 -- 172.21.15.52:0/2239330678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556dc68c4b0 con 0x5556dc691010 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6415c11640 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc6a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f6416412640 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc692260 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f640c003e60 tx=0x7f640c003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa58d226d13a8734 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f63fbfff640 1 -- 172.21.15.52:0/2239330678 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c058040 con 0x5556dc691010 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f63fbfff640 1 -- 172.21.15.52:0/2239330678 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f640c031c40 con 0x5556dc691010 2024-06-02T23:12:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.526+0000 7f63fbfff640 1 -- 172.21.15.52:0/2239330678 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640c03cae0 con 0x5556dc691010 2024-06-02T23:12:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 -- 172.21.15.52:0/2239330678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 msgr2=0x5556dc692260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc692260 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f640c003e60 tx=0x7f640c003e90 comp rx=0 tx=0).stop 2024-06-02T23:12:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 -- 172.21.15.52:0/2239330678 shutdown_connections 2024-06-02T23:12:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc6a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc696b70 0x5556dc68f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 --2- 172.21.15.52:0/2239330678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc691010 0x5556dc692260 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 -- 172.21.15.52:0/2239330678 >> 172.21.15.52:0/2239330678 conn(0x5556dc68c680 msgr2=0x5556dc696220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 -- 172.21.15.52:0/2239330678 shutdown_connections 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.527+0000 7f641b388740 1 -- 172.21.15.52:0/2239330678 wait complete. 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 -- start start 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 Processor -- start 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc691010 0x5556dc6a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc69ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc696b70 0x5556dc69f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64101ac340 con 0x5556dc696b70 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64101abc70 con 0x5556dc691010 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64101a8eb0 con 0x5556dc691a00 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f641b388740 10 client.0 _ll_register_callbacks cb 0x7f6408009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6416c13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc696b70 0x5556dc69f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6415c11640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc69ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6416c13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc696b70 0x5556dc69f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47818/0 (socket says 172.21.15.52:47818) 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6416c13640 1 -- 172.21.15.52:0/300118707 learned_addr learned my addr 172.21.15.52:0/300118707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6416412640 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc691010 0x5556dc6a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6415c11640 1 -- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc691010 msgr2=0x5556dc6a2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.528+0000 7f6415c11640 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc691010 0x5556dc6a2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f6415c11640 1 -- 172.21.15.52:0/300118707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc696b70 msgr2=0x5556dc69f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f6415c11640 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc696b70 0x5556dc69f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f6415c11640 1 -- 172.21.15.52:0/300118707 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556dc69f8b0 con 0x5556dc691a00 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f6415c11640 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc69ee30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9c0 tx=0x7f6400086040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9f2a431aa2c7c81 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f63f9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:12:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f63f9ffb640 1 -- 172.21.15.52:0/300118707 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640001c040 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f63f9ffb640 1 -- 172.21.15.52:0/300118707 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6400005560 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f640c031c40 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f63f9ffb640 1 -- 172.21.15.52:0/300118707 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6400099b80 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f641b388740 10 client.25432 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5556dc710c60 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f63f9ffb640 1 -- 172.21.15.52:0/300118707 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f640c031c40 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f63f9ffb640 1 -- 172.21.15.52:0/300118707 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f64000a40f0 con 0x5556dc691a00 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f641b388740 10 client.25432 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f641b388740 10 client.25432 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.529+0000 7f641b388740 20 client.25432 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.530+0000 7f641b388740 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc7115c0 0x5556dc7139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.530+0000 7f641b388740 4 client.25432 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.530+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556dc601c20 con 0x5556dc7115c0 2024-06-02T23:12:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.530+0000 7f6416412640 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc7115c0 0x5556dc7139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.530+0000 7f6416412640 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc7115c0 0x5556dc7139e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa39edc6f96f6354 server_cookie=206190bac943e9ec in_seq=0 out_seq=0 2024-06-02T23:12:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.530+0000 7f63f9ffb640 10 client.25432 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f63f9ffb640 1 -- 172.21.15.52:0/300118707 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5556dc601c20 con 0x5556dc7115c0 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f63f9ffb640 10 client.25432 handle_command_reply: tid=0 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 client.25432 shutdown 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 msgr2=0x5556dc69ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc69ee30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9c0 tx=0x7f6400086040 comp rx=0 tx=0).stop 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 shutdown_connections 2024-06-02T23:12:52.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5556dc7115c0 0x5556dc7139e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556dc691a00 0x5556dc69ee30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f63f9ffb640 0 client.25432 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5556dc696b70 0x5556dc69f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 --2- 172.21.15.52:0/300118707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5556dc691010 0x5556dc6a2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 >> 172.21.15.52:0/300118707 conn(0x5556dc68c680 msgr2=0x5556dc68f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 shutdown_connections 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 1 -- 172.21.15.52:0/300118707 wait complete. 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.570+0000 7f641b388740 20 client.25432 trim_cache size 0 max 16384 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:12:52.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:12:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:12:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63fc0a2c30 msgr2=0x7f63fc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63fc0a2c30 0x7f63fc0a50f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f640c000c00 tx=0x7f640c004490 comp rx=0 tx=0).stop 2024-06-02T23:12:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410101af0 msgr2=0x7f6410102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410101af0 0x7f6410102ed0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f640000e9f0 tx=0x7f640000eeb0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 shutdown_connections 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63fc0a2c30 0x7f63fc0a50f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64101024b0 0x7f6410103410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6410101af0 0x7f6410102ed0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 --2- 172.21.15.52:0/1096728551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6410100900 0x7f6410104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 >> 172.21.15.52:0/1096728551 conn(0x7f64100fbe70 msgr2=0x7f64100fd980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 shutdown_connections 2024-06-02T23:12:52.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:12:52.571+0000 7f6418676640 1 -- 172.21.15.52:0/1096728551 wait complete. 2024-06-02T23:12:52.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 921 21.77 MB/sec warmup 1 sec latency 27.075 ms 2024-06-02T23:12:52.788 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:53.194 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:52 smithi052 ceph-mon[32340]: pgmap v461: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:53.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:52 smithi123 ceph-mon[35977]: pgmap v461: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:52 smithi174 ceph-mon[35994]: pgmap v461: 97 pgs: 97 active+clean; 672 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-06-02T23:12:53.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 1641 13.09 MB/sec warmup 2 sec latency 15.018 ms 2024-06-02T23:12:54.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 2364 9.30 MB/sec warmup 3 sec latency 42.758 ms 2024-06-02T23:12:54.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/2980588104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4102380 msgr2=0x7f4fc4105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/2980588104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4102380 0x7f4fc4105ff0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40099c0 tx=0x7f4fb4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/2980588104 shutdown_connections 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/2980588104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4106720 0x7f4fc410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/2980588104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fc41019d0 0x7f4fc4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/2980588104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4102380 0x7f4fc4105ff0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/2980588104 >> 172.21.15.52:0/2980588104 conn(0x7f4fc4078340 msgr2=0x7f4fc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/2980588104 shutdown_connections 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.637+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/2980588104 wait complete. 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 Processor -- start 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 -- start start 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fc41019d0 0x7f4fc419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4102380 0x7f4fc419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:54.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 0x7f4fc41989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc4111540 con 0x7f4fc4106720 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41113c0 con 0x7f4fc4102380 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.638+0000 7f4fcc4ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4fc41116c0 con 0x7f4fc41019d0 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fcaa89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 0x7f4fc41989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4102380 0x7f4fc419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fcaa89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 0x7f4fc41989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37006/0 (socket says 172.21.15.52:37006) 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fcaa89700 1 -- 172.21.15.52:0/209486994 learned_addr learned my addr 172.21.15.52:0/209486994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fca288700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fc41019d0 0x7f4fc419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 -- 172.21.15.52:0/209486994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fc41019d0 msgr2=0x7f4fc419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fc41019d0 0x7f4fc419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 -- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 msgr2=0x7f4fc41989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 0x7f4fc41989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 -- 172.21.15.52:0/209486994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fc4199120 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fcaa89700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 0x7f4fc41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fc9a87700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4102380 0x7f4fc419ed40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098e0 tx=0x7f4fb40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4026070 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fc41993b0 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4fb400f460 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fb4009e60 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.639+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fc4102fc0 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.640+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fc406a410 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.642+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4fb4029070 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.642+0000 7f4fbb7fe700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4fb00a2c20 0x7f4fb00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.643+0000 7f4fca288700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4fb00a2c20 0x7f4fb00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.643+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4fb40cf790 con 0x7f4fc4102380 2024-06-02T23:12:54.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.643+0000 7f4fca288700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4fb00a2c20 0x7f4fb00a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0005950 tx=0x7f4fc00058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:54.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.646+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4fb4013070 con 0x7f4fc4102380 2024-06-02T23:12:54.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.933+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fc4199ef0 con 0x7f4fc4102380 2024-06-02T23:12:54.934 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:12:54.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.934+0000 7f4fbb7fe700 1 -- 172.21.15.52:0/209486994 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11939 (secure 0 0 0) 0x7f4fb409c870 con 0x7f4fc4102380 2024-06-02T23:12:54.935 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:10:55.722049+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:12:54.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4fb00a2c20 msgr2=0x7f4fb00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4fb00a2c20 0x7f4fb00a50e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0005950 tx=0x7f4fc00058e0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4102380 msgr2=0x7f4fc419ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4102380 0x7f4fc419ed40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40098e0 tx=0x7f4fb40045e0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 shutdown_connections 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4fb00a2c20 0x7f4fb00a50e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4fc4106720 0x7f4fc41989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4fc41019d0 0x7f4fc419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 --2- 172.21.15.52:0/209486994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fc4102380 0x7f4fc419ed40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 >> 172.21.15.52:0/209486994 conn(0x7f4fc4078340 msgr2=0x7f4fc406afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 shutdown_connections 2024-06-02T23:12:54.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:54.936+0000 7f4fcc4ec700 1 -- 172.21.15.52:0/209486994 wait complete. 2024-06-02T23:12:54.938 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 28 2024-06-02T23:12:55.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:54 smithi123 ceph-mon[35977]: pgmap v462: 97 pgs: 97 active+clean; 4.6 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 338 KiB/s wr, 18 op/s 2024-06-02T23:12:55.258 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:54 smithi052 ceph-mon[32340]: pgmap v462: 97 pgs: 97 active+clean; 4.6 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 338 KiB/s wr, 18 op/s 2024-06-02T23:12:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:54 smithi174 ceph-mon[35994]: pgmap v462: 97 pgs: 97 active+clean; 4.6 MiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 338 KiB/s wr, 18 op/s 2024-06-02T23:12:55.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 3287 8.84 MB/sec warmup 4 sec latency 11.011 ms 2024-06-02T23:12:55.701 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:12:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:56.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:55 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/209486994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:56.280 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:55 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/209486994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:56.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:55 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/209486994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:12:56.590 INFO:tasks.workunit.client.0.smithi052.stdout: 1 3812 7.95 MB/sec warmup 5 sec latency 15.781 ms 2024-06-02T23:12:57.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:56 smithi123 ceph-mon[35977]: pgmap v463: 97 pgs: 97 active+clean; 20 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 40 op/s 2024-06-02T23:12:57.287 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:56 smithi052 ceph-mon[32340]: pgmap v463: 97 pgs: 97 active+clean; 20 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 40 op/s 2024-06-02T23:12:57.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:56 smithi174 ceph-mon[35994]: pgmap v463: 97 pgs: 97 active+clean; 20 MiB data, 369 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 1.7 MiB/s wr, 40 op/s 2024-06-02T23:12:57.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 4281 6.72 MB/sec warmup 6 sec latency 14.523 ms 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 -- 172.21.15.52:0/2603616965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0107400 msgr2=0x7f1af01077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 --2- 172.21.15.52:0/2603616965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0107400 0x7f1af01077e0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009990 tx=0x7f1ae0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 -- 172.21.15.52:0/2603616965 shutdown_connections 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 --2- 172.21.15.52:0/2603616965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101d50 0x7f1af010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 --2- 172.21.15.52:0/2603616965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0107400 0x7f1af01077e0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 --2- 172.21.15.52:0/2603616965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0101390 0x7f1af0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 -- 172.21.15.52:0/2603616965 >> 172.21.15.52:0/2603616965 conn(0x7f1af0078340 msgr2=0x7f1af0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 -- 172.21.15.52:0/2603616965 shutdown_connections 2024-06-02T23:12:57.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.884+0000 7f1af637e700 1 -- 172.21.15.52:0/2603616965 wait complete. 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.885+0000 7f1af637e700 1 Processor -- start 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.885+0000 7f1af637e700 1 -- start start 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af637e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 0x7f1af0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af637e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0101d50 0x7f1af0199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af637e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0107400 0x7f1af01a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af637e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1af0111520 con 0x7f1af0101d50 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af637e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01113a0 con 0x7f1af0101390 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af637e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01116a0 con 0x7f1af0107400 2024-06-02T23:12:57.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 0x7f1af0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 0x7f1af0199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37592/0 (socket says 172.21.15.52:37592) 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 -- 172.21.15.52:0/1851274509 learned_addr learned my addr 172.21.15.52:0/1851274509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0101d50 0x7f1af0199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1af491b700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0107400 0x7f1af01a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 -- 172.21.15.52:0/1851274509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0107400 msgr2=0x7f1af01a4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0107400 0x7f1af01a4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 -- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0101d50 msgr2=0x7f1af0199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0101d50 0x7f1af0199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.886+0000 7f1aeffff700 1 -- 172.21.15.52:0/1851274509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1af01977f0 con 0x7f1af0101390 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.887+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 0x7f1af0199a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00057c0 tx=0x7f1ae00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.887+0000 7f1aed7fa700 1 -- 172.21.15.52:0/1851274509 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0026070 con 0x7f1af0101390 2024-06-02T23:12:57.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.887+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1af0197a80 con 0x7f1af0101390 2024-06-02T23:12:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.887+0000 7f1aed7fa700 1 -- 172.21.15.52:0/1851274509 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae0021410 con 0x7f1af0101390 2024-06-02T23:12:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.887+0000 7f1aed7fa700 1 -- 172.21.15.52:0/1851274509 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0017650 con 0x7f1af0101390 2024-06-02T23:12:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.887+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1af0197f60 con 0x7f1af0101390 2024-06-02T23:12:57.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.889+0000 7f1aed7fa700 1 -- 172.21.15.52:0/1851274509 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ae0029070 con 0x7f1af0101390 2024-06-02T23:12:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.890+0000 7f1aed7fa700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ad80a2c20 0x7f1ad80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.890+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ad80a2c20 0x7f1ad80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.890+0000 7f1aed7fa700 1 -- 172.21.15.52:0/1851274509 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1ae00cfef0 con 0x7f1af0101390 2024-06-02T23:12:57.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.891+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ad80a2c20 0x7f1ad80a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1af0078a60 tx=0x7f1ae40073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 Processor -- start 2024-06-02T23:12:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 -- start start 2024-06-02T23:12:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b93b1abf60 con 0x55b93b2bf880 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b93b20a850 con 0x55b93b2bef60 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b93b172460 con 0x55b93b2b97a0 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37048/0 (socket says 172.21.15.52:37048) 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1044055350 learned_addr learned my addr 172.21.15.52:0/1044055350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:57.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.892+0000 7f1af491b700 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1044055350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 msgr2=0x55b93b2b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1044055350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 msgr2=0x55b93b2bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1044055350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93b1598d0 con 0x55b93b2bf880 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2b9260 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4004500 tx=0x7f1ae4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd8b529b158e4311 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1ad67fc700 1 -- 172.21.15.52:0/1044055350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4024070 con 0x55b93b2bf880 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1ad67fc700 1 -- 172.21.15.52:0/1044055350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae4015070 con 0x55b93b2bf880 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.893+0000 7f1ad67fc700 1 -- 172.21.15.52:0/1044055350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40117d0 con 0x55b93b2bf880 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1044055350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 msgr2=0x55b93b2b9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2b9260 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1ae4004500 tx=0x7f1ae4013000 comp rx=0 tx=0).stop 2024-06-02T23:12:57.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1044055350 shutdown_connections 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1044055350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2b9260 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1044055350 >> 172.21.15.52:0/1044055350 conn(0x55b93b1e6080 msgr2=0x55b93b2b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1044055350 shutdown_connections 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1044055350 wait complete. 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.894+0000 7f1b00c01b80 1 -- start start 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 Processor -- start 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bf880 0x55b93b2c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b93b1aafc0 con 0x55b93b2b97a0 2024-06-02T23:12:57.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b93b206810 con 0x55b93b2bef60 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b93b2c6260 con 0x55b93b2bf880 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1b00c01b80 10 client.0 _ll_register_callbacks cb 0x55b93b1a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37614/0 (socket says 172.21.15.52:37614) 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.895+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1426908015 learned_addr learned my addr 172.21.15.52:0/1426908015 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:57.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1af491b700 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bf880 0x55b93b2c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1426908015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bf880 msgr2=0x55b93b2c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bf880 0x55b93b2c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1426908015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 msgr2=0x55b93b2c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1426908015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93b1d0750 con 0x55b93b2bef60 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2c2910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40079c0 tx=0x7f1ae40057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c472002a1f0ce422 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1acf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1426908015 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402e020 con 0x55b93b2bef60 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae4015070 con 0x55b93b2bef60 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1426908015 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1ae4003b50 con 0x55b93b2bef60 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.896+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1426908015 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ae4024030 con 0x55b93b2bef60 2024-06-02T23:12:57.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 10 client.25514 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b93b2d0a10 con 0x55b93b2bef60 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1426908015 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae4015070 con 0x55b93b2bef60 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1426908015 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1ae401f570 con 0x55b93b2bef60 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 10 client.25514 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 10 client.25514 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 20 client.25514 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b2d32d0 0x55b93b2d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 4 client.25514 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b93b217d50 con 0x55b93b2d32d0 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.897+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b2d32d0 0x55b93b2d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.898+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b2d32d0 0x55b93b2d5710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=21f0d7158daed2d4 server_cookie=f5460a90bdab0ea4 in_seq=0 out_seq=0 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.898+0000 7f1acf7fe700 10 client.25514 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:57.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1426908015 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b93b217d50 con 0x55b93b2d32d0 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1acf7fe700 10 client.25514 handle_command_reply: tid=0 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 client.25514 shutdown 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 msgr2=0x55b93b2c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2c2910 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40079c0 tx=0x7f1ae40057d0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 shutdown_connections 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bf880 0x55b93b2c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b2d32d0 0x55b93b2d5710 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1426908015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2c2910 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 >> 172.21.15.52:0/1426908015 conn(0x55b93b1e6080 msgr2=0x55b93b2b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1acf7fe700 0 client.25514 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 shutdown_connections 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1426908015 wait complete. 2024-06-02T23:12:57.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.899+0000 7f1b00c01b80 20 client.25514 trim_cache size 0 max 16384 2024-06-02T23:12:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 Processor -- start 2024-06-02T23:12:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 -- start start 2024-06-02T23:12:57.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bef60 0x55b93b2c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bf880 0x55b93b2c5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01aec00 con 0x55b93b2b97a0 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b0740 con 0x55b93b2bf880 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01af2d0 con 0x55b93b2bef60 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bef60 0x55b93b2c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1af491b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bf880 0x55b93b2c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1aeffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37072/0 (socket says 172.21.15.52:37072) 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bef60 0x55b93b2c2fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53098/0 (socket says 172.21.15.52:53098) 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.916+0000 7f1aeffff700 1 -- 172.21.15.52:0/225682843 learned_addr learned my addr 172.21.15.52:0/225682843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:57.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1af491b700 1 -- 172.21.15.52:0/225682843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bef60 msgr2=0x55b93b2c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1af491b700 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bef60 0x55b93b2c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1af491b700 1 -- 172.21.15.52:0/225682843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 msgr2=0x55b93b2b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1af491b700 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1af491b700 1 -- 172.21.15.52:0/225682843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b93b2c26e0 con 0x55b93b2bf880 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1aeffff700 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1af491b700 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bf880 0x55b93b2c5150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8007600 tx=0x7f1ae800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d45f54f42c0f729 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1ad5ffb700 1 -- 172.21.15.52:0/225682843 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8003a00 con 0x55b93b2bf880 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1ad5ffb700 1 -- 172.21.15.52:0/225682843 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae8003ba0 con 0x55b93b2bf880 2024-06-02T23:12:57.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.917+0000 7f1ad5ffb700 1 -- 172.21.15.52:0/225682843 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae8017740 con 0x55b93b2bf880 2024-06-02T23:12:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 -- 172.21.15.52:0/225682843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bf880 msgr2=0x55b93b2c5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bf880 0x55b93b2c5150 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8007600 tx=0x7f1ae800b040 comp rx=0 tx=0).stop 2024-06-02T23:12:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 -- 172.21.15.52:0/225682843 shutdown_connections 2024-06-02T23:12:57.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bf880 0x55b93b2c5150 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2b97a0 0x55b93b2b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/225682843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2bef60 0x55b93b2c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 -- 172.21.15.52:0/225682843 >> 172.21.15.52:0/225682843 conn(0x55b93b1e6080 msgr2=0x55b93b2c28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 -- 172.21.15.52:0/225682843 shutdown_connections 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 -- 172.21.15.52:0/225682843 wait complete. 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.918+0000 7f1b00c01b80 1 -- start start 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 Processor -- start 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2cb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2cba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01b0070 con 0x55b93b2bf880 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01af9a0 con 0x55b93b2bef60 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1af01acbc0 con 0x55b93b2b97a0 2024-06-02T23:12:57.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1b00c01b80 10 client.0 _ll_register_callbacks cb 0x55b93b1a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1af491b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1aef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2cb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1af491b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37078/0 (socket says 172.21.15.52:37078) 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1af491b700 1 -- 172.21.15.52:0/1118580193 learned_addr learned my addr 172.21.15.52:0/1118580193 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.919+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1118580193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 msgr2=0x55b93b2db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2db3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1118580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 msgr2=0x55b93b2cba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2cba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1aef7fe700 1 -- 172.21.15.52:0/1118580193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b93b2cbfa0 con 0x55b93b2bef60 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1aef7fe700 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2cb520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40074d0 tx=0x7f1ae4003e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1caba726202763b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1acf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1118580193 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae402e070 con 0x55b93b2bef60 2024-06-02T23:12:57.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1118580193 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1ae4003950 con 0x55b93b2bef60 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ae8003ba0 con 0x55b93b2bef60 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1118580193 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f1ae4003b90 con 0x55b93b2bef60 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1b00c01b80 10 client.25520 fetch_fsmap learned FSMap version 28 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.920+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b93b33cfb0 con 0x55b93b2bef60 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1118580193 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ae8003ba0 con 0x55b93b2bef60 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1118580193 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f1ae402a7a0 con 0x55b93b2bef60 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1b00c01b80 10 client.25520 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1b00c01b80 10 client.25520 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1b00c01b80 20 client.25520 populate_metadata read hostname 'smithi052' 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b33d5a0 0x55b93b33f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1b00c01b80 4 client.25520 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b93b2122d0 con 0x55b93b33d5a0 2024-06-02T23:12:57.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b33d5a0 0x55b93b33f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1aeffff700 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b33d5a0 0x55b93b33f9c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f35d55371b9b5e4 server_cookie=f26614d64e2f9448 in_seq=0 out_seq=0 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.921+0000 7f1acf7fe700 10 client.25520 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1acf7fe700 1 -- 172.21.15.52:0/1118580193 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b93b2122d0 con 0x55b93b33d5a0 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1acf7fe700 10 client.25520 handle_command_reply: tid=0 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 client.25520 shutdown 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 msgr2=0x55b93b2cb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2cb520 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40074d0 tx=0x7f1ae4003e80 comp rx=0 tx=0).stop 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 shutdown_connections 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b93b2bf880 0x55b93b2cba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b93b2b97a0 0x55b93b2db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b93b33d5a0 0x55b93b33f9c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 --2- 172.21.15.52:0/1118580193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b93b2bef60 0x55b93b2cb520 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 >> 172.21.15.52:0/1118580193 conn(0x55b93b1e6080 msgr2=0x55b93b2c0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:57.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1acf7fe700 0 client.25520 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:12:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 shutdown_connections 2024-06-02T23:12:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 1 -- 172.21.15.52:0/1118580193 wait complete. 2024-06-02T23:12:57.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.922+0000 7f1b00c01b80 20 client.25520 trim_cache size 0 max 16384 2024-06-02T23:12:57.928 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ad80a2c20 msgr2=0x7f1ad80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ad80a2c20 0x7f1ad80a50e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1af0078a60 tx=0x7f1ae40073d0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 msgr2=0x7f1af0199a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 0x7f1af0199a00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00057c0 tx=0x7f1ae00057f0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 shutdown_connections 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ad80a2c20 0x7f1ad80a50e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1af0107400 0x7f1af01a4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1af0101390 0x7f1af0199a00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 --2- 172.21.15.52:0/1851274509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1af0101d50 0x7f1af0199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:12:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 >> 172.21.15.52:0/1851274509 conn(0x7f1af0078340 msgr2=0x7f1af0100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:12:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 shutdown_connections 2024-06-02T23:12:57.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:12:57.923+0000 7f1af637e700 1 -- 172.21.15.52:0/1851274509 wait complete. 2024-06-02T23:12:58.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 5009 6.45 MB/sec warmup 7 sec latency 14.773 ms 2024-06-02T23:12:58.721 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:12:59.002 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:12:59.127 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:12:58 smithi052 ceph-mon[32340]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.2 MiB/s wr, 67 op/s 2024-06-02T23:12:59.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:12:58 smithi123 ceph-mon[35977]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.2 MiB/s wr, 67 op/s 2024-06-02T23:12:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:12:58 smithi174 ceph-mon[35994]: pgmap v464: 97 pgs: 97 active+clean; 24 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.2 MiB/s wr, 67 op/s 2024-06-02T23:12:59.589 INFO:tasks.workunit.client.0.smithi052.stdout: 1 5708 5.73 MB/sec warmup 8 sec latency 18.565 ms 2024-06-02T23:13:00.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 6486 5.84 MB/sec warmup 9 sec latency 22.329 ms 2024-06-02T23:13:00.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 -- 172.21.15.52:0/3148882287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec1046a0 msgr2=0x7f5bec104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3148882287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec1046a0 0x7f5bec104b20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009990 tx=0x7f5bd4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 -- 172.21.15.52:0/3148882287 shutdown_connections 2024-06-02T23:13:00.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3148882287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec105060 0x7f5bec111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3148882287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec10a6a0 0x7f5bec10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3148882287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec1046a0 0x7f5bec104b20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 -- 172.21.15.52:0/3148882287 >> 172.21.15.52:0/3148882287 conn(0x7f5bec100170 msgr2=0x7f5bec102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 -- 172.21.15.52:0/3148882287 shutdown_connections 2024-06-02T23:13:00.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 -- 172.21.15.52:0/3148882287 wait complete. 2024-06-02T23:13:00.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.814+0000 7f5bf2387700 1 Processor -- start 2024-06-02T23:13:00.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 -- start start 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 0x7f5bec19ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec105060 0x7f5bec19b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec10a6a0 0x7f5bec19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec114880 con 0x7f5bec105060 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec114700 con 0x7f5bec1046a0 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf2387700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec114a00 con 0x7f5bec10a6a0 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 0x7f5bec19ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec105060 0x7f5bec19b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 0x7f5bec19ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37650/0 (socket says 172.21.15.52:37650) 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 -- 172.21.15.52:0/3722943875 learned_addr learned my addr 172.21.15.52:0/3722943875 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec10a6a0 0x7f5bec19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 -- 172.21.15.52:0/3722943875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec10a6a0 msgr2=0x7f5bec19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec10a6a0 0x7f5bec19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 -- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec105060 msgr2=0x7f5bec19b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec105060 0x7f5bec19b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bebfff700 1 -- 172.21.15.52:0/3722943875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bec19fd40 con 0x7f5bec1046a0 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.815+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec10a6a0 0x7f5bec19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.816+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 0x7f5bec19ad30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e9a0 tx=0x7f5bdc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.816+0000 7f5be9ffb700 1 -- 172.21.15.52:0/3722943875 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc00cbd0 con 0x7f5bec1046a0 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.816+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bec1a0030 con 0x7f5bec1046a0 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.816+0000 7f5be9ffb700 1 -- 172.21.15.52:0/3722943875 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc00cd70 con 0x7f5bec1046a0 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.816+0000 7f5be9ffb700 1 -- 172.21.15.52:0/3722943875 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc018680 con 0x7f5bec1046a0 2024-06-02T23:13:00.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.816+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bec1a78b0 con 0x7f5bec1046a0 2024-06-02T23:13:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.818+0000 7f5be9ffb700 1 -- 172.21.15.52:0/3722943875 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bdc010430 con 0x7f5bec1046a0 2024-06-02T23:13:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.819+0000 7f5be9ffb700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5bd80a2c20 0x7f5bd80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.819+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5bd80a2c20 0x7f5bd80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5be9ffb700 1 -- 172.21.15.52:0/3722943875 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5bdc0cf260 con 0x7f5bec1046a0 2024-06-02T23:13:00.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 Processor -- start 2024-06-02T23:13:00.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 -- start start 2024-06-02T23:13:00.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5bd80a2c20 0x7f5bd80a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b2e0 tx=0x7f5bd4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a704340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a6fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a6febc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56026a5f0f60 con 0x56026a704880 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56026a64f850 con 0x56026a703f60 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56026a5b7460 con 0x56026a6fe7a0 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a6fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a704340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a704340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37660/0 (socket says 172.21.15.52:37660) 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a6fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37112/0 (socket says 172.21.15.52:37112) 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.820+0000 7f5bebfff700 1 -- 172.21.15.52:0/3993685227 learned_addr learned my addr 172.21.15.52:0/3993685227 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a6febc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bebfff700 1 -- 172.21.15.52:0/3993685227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 msgr2=0x56026a6febc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a6febc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bebfff700 1 -- 172.21.15.52:0/3993685227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 msgr2=0x56026a6fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a6fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bebfff700 1 -- 172.21.15.52:0/3993685227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026a59e8d0 con 0x56026a703f60 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a6febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a704340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09ba20 tx=0x7f5bdc09b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c5d22e43bc91d6d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5be27fc700 1 -- 172.21.15.52:0/3993685227 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0b4070 con 0x56026a703f60 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5be27fc700 1 -- 172.21.15.52:0/3993685227 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc09c190 con 0x56026a703f60 2024-06-02T23:13:00.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.821+0000 7f5be27fc700 1 -- 172.21.15.52:0/3993685227 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0a44d0 con 0x56026a703f60 2024-06-02T23:13:00.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3993685227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 msgr2=0x56026a704340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a704340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09ba20 tx=0x7f5bdc09b6a0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3993685227 shutdown_connections 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a6febc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a704340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3993685227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a6fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3993685227 >> 172.21.15.52:0/3993685227 conn(0x56026a62b080 msgr2=0x56026a6f9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3993685227 shutdown_connections 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3993685227 wait complete. 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.822+0000 7f5bfcc0ab80 1 -- start start 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 Processor -- start 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a70a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a707990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a707ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56026a5effc0 con 0x56026a704880 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56026a64b810 con 0x56026a703f60 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56026a70b260 con 0x56026a6fe7a0 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bfcc0ab80 10 client.0 _ll_register_callbacks cb 0x56026a5e55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bf0924700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a707ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a707990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bf0924700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a707ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37114/0 (socket says 172.21.15.52:37114) 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5be3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a707990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37670/0 (socket says 172.21.15.52:37670) 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bf0924700 1 -- 172.21.15.52:0/3587281515 learned_addr learned my addr 172.21.15.52:0/3587281515 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.823+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a70a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5be3fff700 1 -- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 msgr2=0x56026a70a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a70a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5be3fff700 1 -- 172.21.15.52:0/3587281515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 msgr2=0x56026a707ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a707ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5be3fff700 1 -- 172.21.15.52:0/3587281515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026a615750 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a707990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40059d0 tx=0x7f5bd4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4aa3f8f57c09b15 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3587281515 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4035060 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3587281515 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5bd400f460 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bdc09c190 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3587281515 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bd400f660 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bfcc0ab80 10 client.25532 fetch_fsmap learned FSMap version 28 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.824+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56026a715a70 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3587281515 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc09c190 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3587281515 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5bd403c6a0 con 0x56026a703f60 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bfcc0ab80 10 client.25532 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bfcc0ab80 10 client.25532 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:13:00.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bfcc0ab80 20 client.25532 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a718000 0x56026a71a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bfcc0ab80 4 client.25532 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56026a65cd50 con 0x56026a718000 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.825+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a718000 0x56026a71a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.826+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a718000 0x56026a71a3f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a88f901cb040f7d4 server_cookie=a99dafcb388c6cda in_seq=0 out_seq=0 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.826+0000 7f5bcb7fe700 10 client.25532 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.826+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3587281515 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56026a65cd50 con 0x56026a718000 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.826+0000 7f5bcb7fe700 10 client.25532 handle_command_reply: tid=0 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 client.25532 shutdown 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 msgr2=0x56026a707990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a707990 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5bd40059d0 tx=0x7f5bd4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 shutdown_connections 2024-06-02T23:13:00.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a704880 0x56026a707ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a718000 0x56026a71a3f0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a6fe7a0 0x56026a70a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3587281515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a703f60 0x56026a707990 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 >> 172.21.15.52:0/3587281515 conn(0x56026a62b080 msgr2=0x56026a6f8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bcb7fe700 0 client.25532 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 shutdown_connections 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3587281515 wait complete. 2024-06-02T23:13:00.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.827+0000 7f5bfcc0ab80 20 client.25532 trim_cache size 0 max 16384 2024-06-02T23:13:00.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 Processor -- start 2024-06-02T23:13:00.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 -- start start 2024-06-02T23:13:00.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a6fe260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a6f8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1aa8e0 con 0x56026a6fe7a0 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1ac420 con 0x56026a704880 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1aafb0 con 0x56026a703f60 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a6fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a6fe260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37130/0 (socket says 172.21.15.52:37130) 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bebfff700 1 -- 172.21.15.52:0/3354283479 learned_addr learned my addr 172.21.15.52:0/3354283479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.845+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a6f8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bebfff700 1 -- 172.21.15.52:0/3354283479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 msgr2=0x56026a70a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bebfff700 1 -- 172.21.15.52:0/3354283479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 msgr2=0x56026a6f8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a6f8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bebfff700 1 -- 172.21.15.52:0/3354283479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56026a6f5810 con 0x56026a6fe7a0 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a6f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a6fe260 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09ba20 tx=0x7f5bdc09b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fb4c1db098ac3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5be1ffb700 1 -- 172.21.15.52:0/3354283479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0b4040 con 0x56026a6fe7a0 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5be1ffb700 1 -- 172.21.15.52:0/3354283479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc09fe40 con 0x56026a6fe7a0 2024-06-02T23:13:00.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.846+0000 7f5be1ffb700 1 -- 172.21.15.52:0/3354283479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0a44b0 con 0x56026a6fe7a0 2024-06-02T23:13:00.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3354283479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 msgr2=0x56026a6fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a6fe260 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc09ba20 tx=0x7f5bdc09b4f0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3354283479 shutdown_connections 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a6f8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a6fe260 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3354283479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3354283479 >> 172.21.15.52:0/3354283479 conn(0x56026a62b080 msgr2=0x56026a707240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3354283479 shutdown_connections 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3354283479 wait complete. 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.847+0000 7f5bfcc0ab80 1 -- start start 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 Processor -- start 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a70bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a77ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1abd50 con 0x56026a6fe7a0 2024-06-02T23:13:00.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec1ab680 con 0x56026a704880 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bec07a890 con 0x56026a703f60 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bfcc0ab80 10 client.0 _ll_register_callbacks cb 0x56026a5e55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a70bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a70bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37142/0 (socket says 172.21.15.52:37142) 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bebfff700 1 -- 172.21.15.52:0/3423163489 learned_addr learned my addr 172.21.15.52:0/3423163489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bf0924700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a77ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.848+0000 7f5bebfff700 1 -- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 msgr2=0x56026a70c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bebfff700 1 -- 172.21.15.52:0/3423163489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 msgr2=0x56026a77ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a77ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bebfff700 1 -- 172.21.15.52:0/3423163489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56026a77b2c0 con 0x56026a6fe7a0 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bebfff700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a70bc70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc000c00 tx=0x7f5bdc099ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5be199bb011b73e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:00.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bcb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3423163489 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc0b4040 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3423163489 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5bdc0ba300 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bdc09fe40 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3423163489 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f5bdc0a4700 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bfcc0ab80 10 client.15957 fetch_fsmap learned FSMap version 28 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56026a782670 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3423163489 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bdc09fe40 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3423163489 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f5bdc09a950 con 0x56026a6fe7a0 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.849+0000 7f5bfcc0ab80 10 client.15957 fetch_fsmap finished waiting for FSMap version 28 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bfcc0ab80 10 client.15957 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bfcc0ab80 20 client.15957 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a782840 0x56026a784c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:00.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bfcc0ab80 4 client.15957 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56026a6572d0 con 0x56026a782840 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a782840 0x56026a784c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5be3fff700 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a782840 0x56026a784c30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=91888139eb986997 server_cookie=85ebc2cbdad6c63c in_seq=0 out_seq=0 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bcb7fe700 10 client.15957 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bcb7fe700 1 -- 172.21.15.52:0/3423163489 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56026a6572d0 con 0x56026a782840 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bcb7fe700 10 client.15957 handle_command_reply: tid=0 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.850+0000 7f5bfcc0ab80 1 client.15957 shutdown 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 msgr2=0x56026a70bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a70bc70 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc000c00 tx=0x7f5bdc099ed0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 shutdown_connections 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56026a704880 0x56026a77ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56026a6fe7a0 0x56026a70bc70 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56026a782840 0x56026a784c30 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 --2- 172.21.15.52:0/3423163489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56026a703f60 0x56026a70c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 >> 172.21.15.52:0/3423163489 conn(0x56026a62b080 msgr2=0x56026a6fdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bcb7fe700 0 client.15957 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 shutdown_connections 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 1 -- 172.21.15.52:0/3423163489 wait complete. 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.851+0000 7f5bfcc0ab80 20 client.15957 trim_cache size 0 max 16384 2024-06-02T23:13:00.855 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5bd80a2c20 msgr2=0x7f5bd80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5bd80a2c20 0x7f5bd80a50e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5bd400b2e0 tx=0x7f5bd4005960 comp rx=0 tx=0).stop 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 msgr2=0x7f5bec19ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 0x7f5bec19ad30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00e9a0 tx=0x7f5bdc00ecb0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 shutdown_connections 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5bd80a2c20 0x7f5bd80a50e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bec10a6a0 0x7f5bec19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bec1046a0 0x7f5bec19ad30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 --2- 172.21.15.52:0/3722943875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bec105060 0x7f5bec19b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:00.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 >> 172.21.15.52:0/3722943875 conn(0x7f5bec100170 msgr2=0x7f5bec100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:00.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 shutdown_connections 2024-06-02T23:13:00.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:00.852+0000 7f5bf2387700 1 -- 172.21.15.52:0/3722943875 wait complete. 2024-06-02T23:13:01.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:00 smithi052 ceph-mon[32340]: pgmap v465: 97 pgs: 97 active+clean; 25 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.4 MiB/s wr, 73 op/s 2024-06-02T23:13:01.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:00 smithi123 ceph-mon[35977]: pgmap v465: 97 pgs: 97 active+clean; 25 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.4 MiB/s wr, 73 op/s 2024-06-02T23:13:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:00 smithi174 ceph-mon[35994]: pgmap v465: 97 pgs: 97 active+clean; 25 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.4 MiB/s wr, 73 op/s 2024-06-02T23:13:01.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 6946 5.45 MB/sec warmup 10 sec latency 65.566 ms 2024-06-02T23:13:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:01 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:13:02.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:01 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:13:02.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:01 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:13:02.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 7420 5.35 MB/sec warmup 11 sec latency 82.867 ms 2024-06-02T23:13:03.135 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:13:03.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:02 smithi123 ceph-mon[35977]: pgmap v466: 97 pgs: 97 active+clean; 25 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.7 MiB/s wr, 82 op/s 2024-06-02T23:13:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:02 smithi174 ceph-mon[35994]: pgmap v466: 97 pgs: 97 active+clean; 25 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.7 MiB/s wr, 82 op/s 2024-06-02T23:13:03.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:02 smithi052 ceph-mon[32340]: pgmap v466: 97 pgs: 97 active+clean; 25 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 2.7 MiB/s wr, 82 op/s 2024-06-02T23:13:03.406 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:03.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 7783 4.94 MB/sec warmup 12 sec latency 10.999 ms 2024-06-02T23:13:04.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 8322 4.71 MB/sec warmup 13 sec latency 16.901 ms 2024-06-02T23:13:05.144 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:04 smithi052 ceph-mon[32340]: pgmap v467: 97 pgs: 97 active+clean; 26 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-06-02T23:13:05.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:04 smithi123 ceph-mon[35977]: pgmap v467: 97 pgs: 97 active+clean; 26 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-06-02T23:13:05.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 -- 172.21.15.52:0/869260030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f01085d0 msgr2=0x7f54f01089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 --2- 172.21.15.52:0/869260030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f01085d0 0x7f54f01089b0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009920 tx=0x7f54e4009c30 comp rx=0 tx=0).stop 2024-06-02T23:13:05.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 -- 172.21.15.52:0/869260030 shutdown_connections 2024-06-02T23:13:05.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 --2- 172.21.15.52:0/869260030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f0102ef0 0x7f54f010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 --2- 172.21.15.52:0/869260030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f01085d0 0x7f54f01089b0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 --2- 172.21.15.52:0/869260030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.257+0000 7f54f7027700 1 -- 172.21.15.52:0/869260030 >> 172.21.15.52:0/869260030 conn(0x7f54f0079640 msgr2=0x7f54f0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.258+0000 7f54f7027700 1 -- 172.21.15.52:0/869260030 shutdown_connections 2024-06-02T23:13:05.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.258+0000 7f54f7027700 1 -- 172.21.15.52:0/869260030 wait complete. 2024-06-02T23:13:05.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.258+0000 7f54f7027700 1 Processor -- start 2024-06-02T23:13:05.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.258+0000 7f54f7027700 1 -- start start 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f7027700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f7027700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 0x7f54f019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f7027700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01085d0 0x7f54f01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f7027700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01126c0 con 0x7f54f0102ef0 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f7027700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112540 con 0x7f54f0102530 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f7027700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54f0112840 con 0x7f54f01085d0 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 0x7f54f019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34434/0 (socket says 172.21.15.52:34434) 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 -- 172.21.15.52:0/989860556 learned_addr learned my addr 172.21.15.52:0/989860556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54effff700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 0x7f54f019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50880/0 (socket says 172.21.15.52:50880) 2024-06-02T23:13:05.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f55c4700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01085d0 0x7f54f01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 -- 172.21.15.52:0/989860556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01085d0 msgr2=0x7f54f01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01085d0 0x7f54f01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 -- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 msgr2=0x7f54f019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 0x7f54f019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54f4dc3700 1 -- 172.21.15.52:0/989860556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f0199100 con 0x7f54f0102530 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.259+0000 7f54effff700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 0x7f54f019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.260+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f019e7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009230 tx=0x7f54e400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.260+0000 7f54edffb700 1 -- 172.21.15.52:0/989860556 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4027070 con 0x7f54f0102530 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.260+0000 7f54edffb700 1 -- 172.21.15.52:0/989860556 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40055f0 con 0x7f54f0102530 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.260+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f0199390 con 0x7f54f0102530 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.260+0000 7f54edffb700 1 -- 172.21.15.52:0/989860556 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4004930 con 0x7f54f0102530 2024-06-02T23:13:05.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.260+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f01a56b0 con 0x7f54f0102530 2024-06-02T23:13:05.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.262+0000 7f54edffb700 1 -- 172.21.15.52:0/989860556 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54e402a070 con 0x7f54f0102530 2024-06-02T23:13:05.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.263+0000 7f54edffb700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54e00a2c20 0x7f54e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.263+0000 7f54effff700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54e00a2c20 0x7f54e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.263+0000 7f54edffb700 1 -- 172.21.15.52:0/989860556 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54e40cf940 con 0x7f54f0102530 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f54effff700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54e00a2c20 0x7f54e00a50e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54f0079c40 tx=0x7f54dc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 Processor -- start 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 -- start start 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bdd640 0x557005bdda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bddf60 0x557005bd79e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bd7f20 0x557005bd8340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557005b29810 con 0x557005bdd640 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557005ad1720 con 0x557005bddf60 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.264+0000 7f55018aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557005b257d0 con 0x557005bd7f20 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f55c4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bd7f20 0x557005bd8340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bdd640 0x557005bdda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bddf60 0x557005bd79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f55c4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bd7f20 0x557005bd8340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41188/0 (socket says 172.21.15.52:41188) 2024-06-02T23:13:05.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f55c4700 1 -- 172.21.15.52:0/3463827240 learned_addr learned my addr 172.21.15.52:0/3463827240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 -- 172.21.15.52:0/3463827240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bd7f20 msgr2=0x557005bd8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bd7f20 0x557005bd8340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 -- 172.21.15.52:0/3463827240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bddf60 msgr2=0x557005bd79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bddf60 0x557005bd79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 -- 172.21.15.52:0/3463827240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557005ac7f50 con 0x557005bdd640 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54effff700 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bddf60 0x557005bd79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bdd640 0x557005bdda20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f54e40cd4c0 tx=0x7f54e4005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8f84afc66279d00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54daffd700 1 -- 172.21.15.52:0/3463827240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40bc0f0 con 0x557005bdd640 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54daffd700 1 -- 172.21.15.52:0/3463827240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40bc290 con 0x557005bdd640 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.265+0000 7f54daffd700 1 -- 172.21.15.52:0/3463827240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4099190 con 0x557005bdd640 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 -- 172.21.15.52:0/3463827240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bdd640 msgr2=0x557005bdda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bdd640 0x557005bdda20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f54e40cd4c0 tx=0x7f54e4005a00 comp rx=0 tx=0).stop 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 -- 172.21.15.52:0/3463827240 shutdown_connections 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bd7f20 0x557005bd8340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bdd640 0x557005bdda20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 --2- 172.21.15.52:0/3463827240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bddf60 0x557005bd79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 -- 172.21.15.52:0/3463827240 >> 172.21.15.52:0/3463827240 conn(0x557005b02cc0 msgr2=0x557005bd2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 -- 172.21.15.52:0/3463827240 shutdown_connections 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.266+0000 7f55018aab80 1 -- 172.21.15.52:0/3463827240 wait complete. 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 -- start start 2024-06-02T23:13:05.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 Processor -- start 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005be10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005be15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557005be46f0 con 0x557005bddf60 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557005ac9bf0 con 0x557005bd7f20 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557005be4870 con 0x557005bdd640 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.267+0000 7f55018aab80 10 client.0 _ll_register_callbacks cb 0x557005ac3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f55c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005be15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be3db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34456/0 (socket says 172.21.15.52:34456) 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 -- 172.21.15.52:0/851421748 learned_addr learned my addr 172.21.15.52:0/851421748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54effff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005be10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 -- 172.21.15.52:0/851421748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 msgr2=0x557005be10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005be10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 -- 172.21.15.52:0/851421748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 msgr2=0x557005be15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005be15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 -- 172.21.15.52:0/851421748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557005a788d0 con 0x557005bd7f20 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f55c4700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005be15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54effff700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005be10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be3db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54e409c1f0 tx=0x7f54e409c6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6db62397eb08977 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54cffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54cffff700 1 -- 172.21.15.52:0/851421748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40b5060 con 0x557005bd7f20 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e40bc290 con 0x557005bd7f20 2024-06-02T23:13:05.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.268+0000 7f54cffff700 1 -- 172.21.15.52:0/851421748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54e40a0360 con 0x557005bd7f20 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f54cffff700 1 -- 172.21.15.52:0/851421748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54e40bc0f0 con 0x557005bd7f20 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 10 client.25550 fetch_fsmap learned FSMap version 29 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557005be46f0 con 0x557005bd7f20 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f54cffff700 1 -- 172.21.15.52:0/851421748 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40bc290 con 0x557005bd7f20 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f54cffff700 1 -- 172.21.15.52:0/851421748 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f54e40cd930 con 0x557005bd7f20 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 10 client.25550 fetch_fsmap finished waiting for FSMap version 29 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 10 client.25550 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 20 client.25550 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005bf13d0 0x557005bf37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 4 client.25550 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557005b36d10 con 0x557005bf13d0 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.269+0000 7f54effff700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005bf13d0 0x557005bf37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.270+0000 7f54cffff700 10 client.25550 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:13:05.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.270+0000 7f54effff700 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005bf13d0 0x557005bf37c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c41e6d31c2af3e77 server_cookie=fdb49306bd72eb66 in_seq=0 out_seq=0 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f54cffff700 1 -- 172.21.15.52:0/851421748 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557005b36d10 con 0x557005bf13d0 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f54cffff700 10 client.25550 handle_command_reply: tid=0 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 client.25550 shutdown 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 msgr2=0x557005be3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be3db0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f54e409c1f0 tx=0x7f54e409c6f0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 shutdown_connections 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005bf13d0 0x557005bf37c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005be15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be3db0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f54cffff700 0 client.25550 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 --2- 172.21.15.52:0/851421748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005be10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 >> 172.21.15.52:0/851421748 conn(0x557005b02cc0 msgr2=0x557005bd69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 shutdown_connections 2024-06-02T23:13:05.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 1 -- 172.21.15.52:0/851421748 wait complete. 2024-06-02T23:13:05.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.271+0000 7f55018aab80 20 client.25550 trim_cache size 0 max 16384 2024-06-02T23:13:05.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 Processor -- start 2024-06-02T23:13:05.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 -- start start 2024-06-02T23:13:05.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005bed950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bdf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bdfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01ac8c0 con 0x557005bddf60 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01ae460 con 0x557005bd7f20 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f55018aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01acf90 con 0x557005bdd640 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f54f55c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bdfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005bed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f54f55c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bdfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50908/0 (socket says 172.21.15.52:50908) 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f54f55c4700 1 -- 172.21.15.52:0/3528152174 learned_addr learned my addr 172.21.15.52:0/3528152174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.288+0000 7f54effff700 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bdf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54f4dc3700 1 -- 172.21.15.52:0/3528152174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 msgr2=0x557005bdf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bdf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54f4dc3700 1 -- 172.21.15.52:0/3528152174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 msgr2=0x557005bdfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bdfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54f4dc3700 1 -- 172.21.15.52:0/3528152174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557005bdf620 con 0x557005bd7f20 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54effff700 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bdf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005bed950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54e40cdea0 tx=0x7f54e400b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19070c03d72fe07b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54da7fc700 1 -- 172.21.15.52:0/3528152174 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40b5040 con 0x557005bd7f20 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54da7fc700 1 -- 172.21.15.52:0/3528152174 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40bc290 con 0x557005bd7f20 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.289+0000 7f54da7fc700 1 -- 172.21.15.52:0/3528152174 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4099190 con 0x557005bd7f20 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 -- 172.21.15.52:0/3528152174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 msgr2=0x557005bed950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005bed950 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f54e40cdea0 tx=0x7f54e400b560 comp rx=0 tx=0).stop 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 -- 172.21.15.52:0/3528152174 shutdown_connections 2024-06-02T23:13:05.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bdfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005bed950 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 --2- 172.21.15.52:0/3528152174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bdf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 -- 172.21.15.52:0/3528152174 >> 172.21.15.52:0/3528152174 conn(0x557005b02cc0 msgr2=0x557005bdb190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 -- 172.21.15.52:0/3528152174 shutdown_connections 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 -- 172.21.15.52:0/3528152174 wait complete. 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.290+0000 7f55018aab80 1 -- start start 2024-06-02T23:13:05.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 Processor -- start 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bea1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bf8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01add30 con 0x557005bddf60 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01ad660 con 0x557005bd7f20 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54f01aa860 con 0x557005bdd640 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f55018aab80 10 client.0 _ll_register_callbacks cb 0x557005ac3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f54f55c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bf8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f54f4dc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be9ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34468/0 (socket says 172.21.15.52:34468) 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f54f4dc3700 1 -- 172.21.15.52:0/211938992 learned_addr learned my addr 172.21.15.52:0/211938992 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.291+0000 7f54effff700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bea1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f4dc3700 1 -- 172.21.15.52:0/211938992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 msgr2=0x557005bea1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bea1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f4dc3700 1 -- 172.21.15.52:0/211938992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 msgr2=0x557005bf8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bf8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f4dc3700 1 -- 172.21.15.52:0/211938992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557005bf8ae0 con 0x557005bd7f20 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f55c4700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bf8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54f4dc3700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be9ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f54e40092a0 tx=0x7f54e40cd950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45154e635f469f13 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54cffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54cffff700 1 -- 172.21.15.52:0/211938992 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e40b5040 con 0x557005bd7f20 2024-06-02T23:13:05.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54cffff700 1 -- 172.21.15.52:0/211938992 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54e4009400 con 0x557005bd7f20 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e40bc290 con 0x557005bd7f20 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f54cffff700 1 -- 172.21.15.52:0/211938992 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f54e409c030 con 0x557005bd7f20 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f55018aab80 10 client.25556 fetch_fsmap learned FSMap version 29 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.292+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557005bf8ae0 con 0x557005bd7f20 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f54cffff700 1 -- 172.21.15.52:0/211938992 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54e40bc290 con 0x557005bd7f20 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f54cffff700 1 -- 172.21.15.52:0/211938992 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f54e40a1b00 con 0x557005bd7f20 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f55018aab80 10 client.25556 fetch_fsmap finished waiting for FSMap version 29 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f55018aab80 10 client.25556 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f55018aab80 20 client.25556 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f55018aab80 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005c5bc40 0x557005c5e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f55018aab80 4 client.25556 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557005b31290 con 0x557005c5bc40 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.293+0000 7f54effff700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005c5bc40 0x557005c5e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f54effff700 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005c5bc40 0x557005c5e030 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e8f8224b514a0bc8 server_cookie=6e8008a2f56fa1b5 in_seq=0 out_seq=0 2024-06-02T23:13:05.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f54cffff700 10 client.25556 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f54cffff700 1 -- 172.21.15.52:0/211938992 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557005b31290 con 0x557005c5bc40 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f54cffff700 10 client.25556 handle_command_reply: tid=0 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 client.25556 shutdown 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 msgr2=0x557005be9ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be9ca0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f54e40092a0 tx=0x7f54e40cd950 comp rx=0 tx=0).stop 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 shutdown_connections 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x557005c5bc40 0x557005c5e030 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557005bddf60 0x557005bf8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f54cffff700 0 client.25556 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557005bd7f20 0x557005be9ca0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 --2- 172.21.15.52:0/211938992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557005bdd640 0x557005bea1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 >> 172.21.15.52:0/211938992 conn(0x557005b02cc0 msgr2=0x557005bdf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 shutdown_connections 2024-06-02T23:13:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 1 -- 172.21.15.52:0/211938992 wait complete. 2024-06-02T23:13:05.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.294+0000 7f55018aab80 20 client.25556 trim_cache size 0 max 16384 2024-06-02T23:13:05.298 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54e00a2c20 msgr2=0x7f54e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54e00a2c20 0x7f54e00a50e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f54f0079c40 tx=0x7f54dc009310 comp rx=0 tx=0).stop 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 msgr2=0x7f54f019e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f019e7e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f54e4009230 tx=0x7f54e400fa00 comp rx=0 tx=0).stop 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 shutdown_connections 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54e00a2c20 0x7f54e00a50e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54f01085d0 0x7f54f01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54f0102530 0x7f54f019e7e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 --2- 172.21.15.52:0/989860556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54f0102ef0 0x7f54f019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 >> 172.21.15.52:0/989860556 conn(0x7f54f0079640 msgr2=0x7f54f0101220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 shutdown_connections 2024-06-02T23:13:05.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:05.295+0000 7f54f7027700 1 -- 172.21.15.52:0/989860556 wait complete. 2024-06-02T23:13:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:04 smithi174 ceph-mon[35994]: pgmap v467: 97 pgs: 97 active+clean; 26 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-06-02T23:13:05.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 8880 4.60 MB/sec warmup 14 sec latency 62.191 ms 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/1613209129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8104db0 msgr2=0x7f5cc81071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/1613209129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8104db0 0x7f5cc81071a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4009900 tx=0x7f5cb402a410 comp rx=0 tx=0).stop 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/1613209129 shutdown_connections 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/1613209129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc81076e0 0x7f5cc8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/1613209129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8104db0 0x7f5cc81071a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/1613209129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8102480 0x7f5cc8104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/1613209129 >> 172.21.15.52:0/1613209129 conn(0x7f5cc80fbe70 msgr2=0x7f5cc80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/1613209129 shutdown_connections 2024-06-02T23:13:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.671+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/1613209129 wait complete. 2024-06-02T23:13:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.672+0000 7f5ccf1f4640 1 Processor -- start 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.672+0000 7f5ccf1f4640 1 -- start start 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccf1f4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 0x7f5cc8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccf1f4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8104db0 0x7f5cc81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccf1f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc81076e0 0x7f5cc819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccf1f4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc810d530 con 0x7f5cc8102480 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccf1f4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc810d3b0 con 0x7f5cc8104db0 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccf1f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc810d6b0 con 0x7f5cc81076e0 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 0x7f5cc8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 0x7f5cc8197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50932/0 (socket says 172.21.15.52:50932) 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 -- 172.21.15.52:0/852071403 learned_addr learned my addr 172.21.15.52:0/852071403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8104db0 0x7f5cc81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5ccd791640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc81076e0 0x7f5cc819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 -- 172.21.15.52:0/852071403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc81076e0 msgr2=0x7f5cc819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc81076e0 0x7f5cc819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 -- 172.21.15.52:0/852071403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8104db0 msgr2=0x7f5cc81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8104db0 0x7f5cc81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 -- 172.21.15.52:0/852071403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc819f7d0 con 0x7f5cc8102480 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cccf90640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 0x7f5cc8197bb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00d950 tx=0x7f5cbc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.673+0000 7f5cc5ffb640 1 -- 172.21.15.52:0/852071403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc017070 con 0x7f5cc8102480 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.674+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc819fac0 con 0x7f5cc8102480 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.674+0000 7f5cc5ffb640 1 -- 172.21.15.52:0/852071403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc009c30 con 0x7f5cc8102480 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.674+0000 7f5cc5ffb640 1 -- 172.21.15.52:0/852071403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc00f620 con 0x7f5cc8102480 2024-06-02T23:13:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.674+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc81a0030 con 0x7f5cc8102480 2024-06-02T23:13:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.676+0000 7f5cc5ffb640 1 -- 172.21.15.52:0/852071403 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cbc0109a0 con 0x7f5cc8102480 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.676+0000 7f5cd1fd6740 1 Processor -- start 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 -- start start 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fe010 0x55e2095fe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e209603500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 0x55e209606000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2095614f0 con 0x55e2095fe010 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2094ff870 con 0x55e209603b70 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e20955d4b0 con 0x55e2095fea00 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc5ffb640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb80a2cb0 0x7f5cb80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5ccd791640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fe010 0x55e2095fe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 0x55e209606000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.681 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 0x55e209606000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34492/0 (socket says 172.21.15.52:34492) 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1998243616 learned_addr learned my addr 172.21.15.52:0/1998243616 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cccf90640 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e209603500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5ccd791640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb80a2cb0 0x7f5cb80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc5ffb640 1 -- 172.21.15.52:0/852071403 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cbc0cebd0 con 0x7f5cc8102480 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1998243616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 msgr2=0x55e209603500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e209603500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1998243616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fe010 msgr2=0x55e2095fe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fe010 0x55e2095fe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1998243616 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e20951b9b0 con 0x55e209603b70 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5cccf90640 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e209603500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.677+0000 7f5ccd791640 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fe010 0x55e2095fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.678+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 0x55e209606000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8069bb0 tx=0x7f5cb4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=982a22620bc87df4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.682 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.678+0000 7f5ccd791640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb80a2cb0 0x7f5cb80a5170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55e2095ff260 tx=0x7f5cc0014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.678+0000 7f5cb2ffd640 1 -- 172.21.15.52:0/1998243616 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4035030 con 0x55e209603b70 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.678+0000 7f5cb2ffd640 1 -- 172.21.15.52:0/1998243616 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb4040070 con 0x55e209603b70 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.678+0000 7f5cb2ffd640 1 -- 172.21.15.52:0/1998243616 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4003af0 con 0x55e209603b70 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1998243616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 msgr2=0x55e209606000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 0x55e209606000 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8069bb0 tx=0x7f5cb4005c00 comp rx=0 tx=0).stop 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1998243616 shutdown_connections 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e209603500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e209603b70 0x55e209606000 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1998243616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fe010 0x55e2095fe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1998243616 >> 172.21.15.52:0/1998243616 conn(0x55e2095f9680 msgr2=0x55e2095f9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1998243616 shutdown_connections 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1998243616 wait complete. 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 -- start start 2024-06-02T23:13:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 Processor -- start 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fe010 0x55e209608250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.679+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 0x55e209608790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e209606ac0 con 0x55e209603b70 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2094eaae0 con 0x55e2095fea00 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e209606c40 con 0x55e2095fe010 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cd1fd6740 10 client.0 _ll_register_callbacks cb 0x55e2095239b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 0x55e209608790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5ccd791640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 0x55e209608790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34496/0 (socket says 172.21.15.52:34496) 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 -- 172.21.15.52:0/2951763085 learned_addr learned my addr 172.21.15.52:0/2951763085 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cccf90640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fe010 0x55e209608250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 -- 172.21.15.52:0/2951763085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fe010 msgr2=0x55e209608250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fe010 0x55e209608250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 -- 172.21.15.52:0/2951763085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 msgr2=0x55e20960f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 -- 172.21.15.52:0/2951763085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e209275210 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cccf90640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fe010 0x55e209608250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 0x55e209608790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40314e0 tx=0x7f5cb40317b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a1bc89103abf269 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5ca7fff640 1 -- 172.21.15.52:0/2951763085 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4052060 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb4040070 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.680+0000 7f5ca7fff640 1 -- 172.21.15.52:0/2951763085 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cb403c850 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5ca7fff640 1 -- 172.21.15.52:0/2951763085 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5cb4035070 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 10 client.25568 fetch_fsmap learned FSMap version 29 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e209341620 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5ca7fff640 1 -- 172.21.15.52:0/2951763085 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb4040070 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5ca7fff640 1 -- 172.21.15.52:0/2951763085 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb404c030 con 0x55e2095fea00 2024-06-02T23:13:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 10 client.25568 fetch_fsmap finished waiting for FSMap version 29 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 20 client.25568 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e209613bc0 0x55e209615fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 4 client.25568 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e209574f10 con 0x55e209613bc0 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.681+0000 7f5cccf90640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e209613bc0 0x55e209615fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5ca7fff640 10 client.25568 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5cccf90640 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e209613bc0 0x55e209615fb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8728b4f3665f3f12 server_cookie=5e71ccb008a4e3ff in_seq=0 out_seq=0 2024-06-02T23:13:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5ca7fff640 1 -- 172.21.15.52:0/2951763085 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e209574f10 con 0x55e209613bc0 2024-06-02T23:13:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5ca7fff640 10 client.25568 handle_command_reply: tid=0 2024-06-02T23:13:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5cd1fd6740 1 client.25568 shutdown 2024-06-02T23:13:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 msgr2=0x55e209608790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.682+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 0x55e209608790 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40314e0 tx=0x7f5cb40317b0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 shutdown_connections 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e209613bc0 0x55e209615fb0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5ca7fff640 0 client.25568 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fea00 0x55e209608790 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/2951763085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fe010 0x55e209608250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 >> 172.21.15.52:0/2951763085 conn(0x55e2095f9680 msgr2=0x55e2095f87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 shutdown_connections 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/2951763085 wait complete. 2024-06-02T23:13:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.683+0000 7f5cd1fd6740 20 client.25568 trim_cache size 0 max 16384 2024-06-02T23:13:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 Processor -- start 2024-06-02T23:13:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 -- start start 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e2095f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 0x55e20960f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e209603b70 0x55e2095ffc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81a8c40 con 0x55e2095fea00 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81aa7e0 con 0x55e2095fe010 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.698+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81a9310 con 0x55e209603b70 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 0x55e20960f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e2095f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e2095f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34512/0 (socket says 172.21.15.52:34512) 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 0x55e20960f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50956/0 (socket says 172.21.15.52:50956) 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 -- 172.21.15.52:0/3378360525 learned_addr learned my addr 172.21.15.52:0/3378360525 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5ccd791640 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e209603b70 0x55e2095ffc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 -- 172.21.15.52:0/3378360525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e209603b70 msgr2=0x55e2095ffc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e209603b70 0x55e2095ffc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 -- 172.21.15.52:0/3378360525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 msgr2=0x55e20960f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 0x55e20960f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 -- 172.21.15.52:0/3378360525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2095f94b0 con 0x55e2095fe010 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 0x55e20960f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cccf90640 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e2095f9a60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc09b160 tx=0x7f5cbc016870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118e41ca5ec9f7b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cb27fc640 1 -- 172.21.15.52:0/3378360525 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc020040 con 0x55e2095fe010 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cb27fc640 1 -- 172.21.15.52:0/3378360525 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc082070 con 0x55e2095fe010 2024-06-02T23:13:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.699+0000 7f5cb27fc640 1 -- 172.21.15.52:0/3378360525 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc0ccbc0 con 0x55e2095fe010 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/3378360525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 msgr2=0x55e2095f9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e2095f9a60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc09b160 tx=0x7f5cbc016870 comp rx=0 tx=0).stop 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/3378360525 shutdown_connections 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e209603b70 0x55e2095ffc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2095fea00 0x55e20960f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/3378360525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e2095f9a60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/3378360525 >> 172.21.15.52:0/3378360525 conn(0x55e2095f9680 msgr2=0x55e2096032e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/3378360525 shutdown_connections 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/3378360525 wait complete. 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 -- start start 2024-06-02T23:13:05.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.700+0000 7f5cd1fd6740 1 Processor -- start 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e20960f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e20960c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81aa0b0 con 0x55e209603b70 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81a99e0 con 0x55e2095fe010 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc81a6c20 con 0x55e2095fea00 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cd1fd6740 10 client.0 _ll_register_callbacks cb 0x7f5cc0009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5ccd791640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cc7fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e20960c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5ccd791640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50972/0 (socket says 172.21.15.52:50972) 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5ccd791640 1 -- 172.21.15.52:0/1967831063 learned_addr learned my addr 172.21.15.52:0/1967831063 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cccf90640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e20960f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 msgr2=0x55e20960f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e20960f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1967831063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 msgr2=0x55e20960c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5cc7fff640 1 -- 172.21.15.52:0/1967831063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e20967c8a0 con 0x55e2095fea00 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.701+0000 7f5ccd791640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cc7fff640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e20960c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cb402a930 tx=0x7f5cb4000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d30dbbac8904ee57 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5ca7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5ca7fff640 1 -- 172.21.15.52:0/1967831063 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb405d030 con 0x55e2095fea00 2024-06-02T23:13:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc082070 con 0x55e2095fea00 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5ca7fff640 1 -- 172.21.15.52:0/1967831063 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cb40445f0 con 0x55e2095fea00 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5ca7fff640 1 -- 172.21.15.52:0/1967831063 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5cb4069050 con 0x55e2095fea00 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 10 client.25510 fetch_fsmap learned FSMap version 29 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e20967db50 con 0x55e2095fea00 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5ca7fff640 1 -- 172.21.15.52:0/1967831063 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc082070 con 0x55e2095fea00 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5ca7fff640 1 -- 172.21.15.52:0/1967831063 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5cb404c040 con 0x55e2095fea00 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 10 client.25510 fetch_fsmap finished waiting for FSMap version 29 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 10 client.25510 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 20 client.25510 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e20967e4f0 0x55e209680910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 4 client.25510 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.702+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e20956ec20 con 0x55e20967e4f0 2024-06-02T23:13:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.703+0000 7f5cccf90640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e20967e4f0 0x55e209680910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.703+0000 7f5ca7fff640 10 client.25510 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:05.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.703+0000 7f5cccf90640 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e20967e4f0 0x55e209680910 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f7687ebffd35c9 server_cookie=ef4d11019bfef9e in_seq=0 out_seq=0 2024-06-02T23:13:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5ca7fff640 1 -- 172.21.15.52:0/1967831063 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e20956ec20 con 0x55e20967e4f0 2024-06-02T23:13:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5ca7fff640 10 client.25510 handle_command_reply: tid=0 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 client.25510 shutdown 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 msgr2=0x55e20960c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e20960c150 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5cb402a930 tx=0x7f5cb4000bc0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 shutdown_connections 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e20967e4f0 0x55e209680910 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2095fea00 0x55e20960c150 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e209603b70 0x55e20960c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5ca7fff640 0 client.25510 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 --2- 172.21.15.52:0/1967831063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2095fe010 0x55e20960f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 >> 172.21.15.52:0/1967831063 conn(0x55e2095f9680 msgr2=0x55e209608230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 shutdown_connections 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 1 -- 172.21.15.52:0/1967831063 wait complete. 2024-06-02T23:13:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.724+0000 7f5cd1fd6740 20 client.25510 trim_cache size 0 max 16384 2024-06-02T23:13:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:13:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:13:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb80a2cb0 msgr2=0x7f5cb80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb80a2cb0 0x7f5cb80a5170 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x55e2095ff260 tx=0x7f5cc0014040 comp rx=0 tx=0).stop 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 msgr2=0x7f5cc8197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 0x7f5cc8197bb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc00d950 tx=0x7f5cbc00de10 comp rx=0 tx=0).stop 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 shutdown_connections 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb80a2cb0 0x7f5cb80a5170 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc81076e0 0x7f5cc819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc8104db0 0x7f5cc81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 --2- 172.21.15.52:0/852071403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc8102480 0x7f5cc8197bb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 >> 172.21.15.52:0/852071403 conn(0x7f5cc80fbe70 msgr2=0x7f5cc8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 shutdown_connections 2024-06-02T23:13:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:05.725+0000 7f5ccf1f4640 1 -- 172.21.15.52:0/852071403 wait complete. 2024-06-02T23:13:06.114 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:13:06.395 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:06.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 9490 4.41 MB/sec warmup 15 sec latency 15.140 ms 2024-06-02T23:13:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:06 smithi052 ceph-mon[32340]: pgmap v468: 97 pgs: 97 active+clean; 26 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.9 MiB/s wr, 121 op/s 2024-06-02T23:13:07.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:06 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:13:07.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:06 smithi123 ceph-mon[35977]: pgmap v468: 97 pgs: 97 active+clean; 26 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.9 MiB/s wr, 121 op/s 2024-06-02T23:13:07.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:06 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:13:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:06 smithi174 ceph-mon[35994]: pgmap v468: 97 pgs: 97 active+clean; 26 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 2.9 MiB/s wr, 121 op/s 2024-06-02T23:13:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:06 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:13:07.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 10104 4.48 MB/sec warmup 16 sec latency 15.678 ms 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 -- 172.21.15.52:0/3775203970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c4108650 msgr2=0x7f24c4108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/3775203970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c4108650 0x7f24c4108a30 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f24b4009990 tx=0x7f24b4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 -- 172.21.15.52:0/3775203970 shutdown_connections 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/3775203970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4103170 0x7f24c410f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/3775203970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c4108650 0x7f24c4108a30 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/3775203970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c41027b0 0x7f24c4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 -- 172.21.15.52:0/3775203970 >> 172.21.15.52:0/3775203970 conn(0x7f24c407a420 msgr2=0x7f24c407a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 -- 172.21.15.52:0/3775203970 shutdown_connections 2024-06-02T23:13:08.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.228+0000 7f24c8c3b700 1 -- 172.21.15.52:0/3775203970 wait complete. 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 Processor -- start 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 -- start start 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c41027b0 0x7f24c410c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 0x7f24c410cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4108650 0x7f24c41a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24c4112730 con 0x7f24c41027b0 2024-06-02T23:13:08.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41125b0 con 0x7f24c4103170 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c8c3b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41128b0 con 0x7f24c4108650 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c41027b0 0x7f24c410c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 0x7f24c410cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 0x7f24c410cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34546/0 (socket says 172.21.15.52:34546) 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.229+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1701150672 learned_addr learned my addr 172.21.15.52:0/1701150672 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4108650 0x7f24c41a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1701150672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4108650 msgr2=0x7f24c41a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4108650 0x7f24c41a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c41027b0 msgr2=0x7f24c410c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c41027b0 0x7f24c410c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1701150672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24c41a4200 con 0x7f24c4103170 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4108650 0x7f24c41a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:08.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 0x7f24c410cbf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9a0 tx=0x7f24b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:08.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24b37fe700 1 -- 172.21.15.52:0/1701150672 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b800cbd0 con 0x7f24c4103170 2024-06-02T23:13:08.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24c41a44f0 con 0x7f24c4103170 2024-06-02T23:13:08.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24c41abda0 con 0x7f24c4103170 2024-06-02T23:13:08.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24b37fe700 1 -- 172.21.15.52:0/1701150672 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24b800cd70 con 0x7f24c4103170 2024-06-02T23:13:08.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.230+0000 7f24b37fe700 1 -- 172.21.15.52:0/1701150672 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b8018680 con 0x7f24c4103170 2024-06-02T23:13:08.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.234+0000 7f24b37fe700 1 -- 172.21.15.52:0/1701150672 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f24b80188e0 con 0x7f24c4103170 2024-06-02T23:13:08.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.235+0000 7f24b37fe700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24ac0a2ca0 0x7f24ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.235+0000 7f24c259c700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24ac0a2ca0 0x7f24ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.235+0000 7f24b37fe700 1 -- 172.21.15.52:0/1701150672 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f24b80cf280 con 0x7f24c4103170 2024-06-02T23:13:08.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.235+0000 7f24d34beb80 1 Processor -- start 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.235+0000 7f24d34beb80 1 -- start start 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982a640 0x561e5982aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982af60 0x561e598249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59825340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24d34beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e59776810 con 0x561e59824f20 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24d34beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e5971e720 con 0x561e5982a640 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24d34beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e597727d0 con 0x561e5982af60 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982a640 0x561e5982aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59825340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982a640 0x561e5982aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34556/0 (socket says 172.21.15.52:34556) 2024-06-02T23:13:08.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59825340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50986/0 (socket says 172.21.15.52:50986) 2024-06-02T23:13:08.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c259c700 1 -- 172.21.15.52:0/2241111427 learned_addr learned my addr 172.21.15.52:0/2241111427 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:08.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982af60 0x561e598249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 -- 172.21.15.52:0/2241111427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982af60 msgr2=0x561e598249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982af60 0x561e598249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 -- 172.21.15.52:0/2241111427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982a640 msgr2=0x561e5982aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982a640 0x561e5982aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 -- 172.21.15.52:0/2241111427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e59714f50 con 0x561e59824f20 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c259c700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24ac0a2ca0 0x7f24ac0a5160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b2e0 tx=0x7f24b4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.236+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59825340 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0093a0 tx=0x7f24bc009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81bf48c5e38a6a38 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.237+0000 7f24b0ff9700 1 -- 172.21.15.52:0/2241111427 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc019030 con 0x561e59824f20 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.237+0000 7f24b0ff9700 1 -- 172.21.15.52:0/2241111427 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24bc009dc0 con 0x561e59824f20 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.237+0000 7f24b0ff9700 1 -- 172.21.15.52:0/2241111427 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc015440 con 0x561e59824f20 2024-06-02T23:13:08.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 -- 172.21.15.52:0/2241111427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 msgr2=0x561e59825340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59825340 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0093a0 tx=0x7f24bc009920 comp rx=0 tx=0).stop 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 -- 172.21.15.52:0/2241111427 shutdown_connections 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59825340 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982a640 0x561e5982aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2241111427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982af60 0x561e598249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 -- 172.21.15.52:0/2241111427 >> 172.21.15.52:0/2241111427 conn(0x561e5974fcc0 msgr2=0x561e5981f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 -- 172.21.15.52:0/2241111427 shutdown_connections 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 -- 172.21.15.52:0/2241111427 wait complete. 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 -- start start 2024-06-02T23:13:08.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.238+0000 7f24d34beb80 1 Processor -- start 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59830cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e5982dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e598319d0 con 0x561e59824f20 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e59716bf0 con 0x561e5982af60 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e59831b50 con 0x561e5982a640 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24d34beb80 10 client.0 _ll_register_callbacks cb 0x561e59710920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:08.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59830cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e5982dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e5982dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41312/0 (socket says 172.21.15.52:41312) 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1254277422 learned_addr learned my addr 172.21.15.52:0/1254277422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59830cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50990/0 (socket says 172.21.15.52:50990) 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1254277422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 msgr2=0x561e5982e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1254277422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 msgr2=0x561e59830cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59830cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 -- 172.21.15.52:0/1254277422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e596c58d0 con 0x561e5982a640 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e5982dfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c5b0 tx=0x7f24b809bff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54fb89ecc84b8347 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.239+0000 7f24a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1254277422 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b80b3030 con 0x561e5982a640 2024-06-02T23:13:08.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1254277422 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f24b80a4290 con 0x561e5982a640 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561e59783d10 con 0x561e5982a640 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1254277422 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24b8010970 con 0x561e5982a640 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24d34beb80 10 client.25522 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e5983c390 con 0x561e5982a640 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1254277422 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561e59783d10 con 0x561e5982a640 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1254277422 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f24b80ba170 con 0x561e5982a640 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24d34beb80 10 client.25522 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.240+0000 7f24d34beb80 10 client.25522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24d34beb80 20 client.25522 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e5983e9d0 0x561e59840dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24d34beb80 4 client.25522 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e5977e290 con 0x561e5983e9d0 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24c259c700 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e5983e9d0 0x561e59840dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24a9ffb700 10 client.25522 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.241+0000 7f24c259c700 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e5983e9d0 0x561e59840dc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e74738d44caea8a6 server_cookie=c18b76f6d033ddc5 in_seq=0 out_seq=0 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1254277422 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e5977e290 con 0x561e5983e9d0 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24a9ffb700 10 client.25522 handle_command_reply: tid=0 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 client.25522 shutdown 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 msgr2=0x561e5982dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e5982dfb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f24b800c5b0 tx=0x7f24b809bff0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 shutdown_connections 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e5983e9d0 0x561e59840dc0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59830cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24a9ffb700 0 client.25522 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1254277422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e5982dfb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 >> 172.21.15.52:0/1254277422 conn(0x561e5974fcc0 msgr2=0x561e598239d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:08.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 shutdown_connections 2024-06-02T23:13:08.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.242+0000 7f24d34beb80 1 -- 172.21.15.52:0/1254277422 wait complete. 2024-06-02T23:13:08.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.243+0000 7f24d34beb80 20 client.25522 trim_cache size 0 max 16384 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 Processor -- start 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 -- start start 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e598249e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59828730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b3170 con 0x561e59824f20 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b4d40 con 0x561e5982af60 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24d34beb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b3840 con 0x561e5982a640 2024-06-02T23:13:08.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e598249e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e598249e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50994/0 (socket says 172.21.15.52:50994) 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24c259c700 1 -- 172.21.15.52:0/2134150242 learned_addr learned my addr 172.21.15.52:0/2134150242 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.273+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59828730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c2d9d700 1 -- 172.21.15.52:0/2134150242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 msgr2=0x561e59828730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59828730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c2d9d700 1 -- 172.21.15.52:0/2134150242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 msgr2=0x561e598249e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e598249e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c2d9d700 1 -- 172.21.15.52:0/2134150242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e5982a120 con 0x561e5982af60 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c259c700 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e598249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:08.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982b400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005bb0 tx=0x7f24bc0047f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e8faff91c964e18 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24abfff700 1 -- 172.21.15.52:0/2134150242 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc007e60 con 0x561e5982af60 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24abfff700 1 -- 172.21.15.52:0/2134150242 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24bc023070 con 0x561e5982af60 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.274+0000 7f24abfff700 1 -- 172.21.15.52:0/2134150242 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24bc003a00 con 0x561e5982af60 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 -- 172.21.15.52:0/2134150242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 msgr2=0x561e5982b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982b400 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f24bc005bb0 tx=0x7f24bc0047f0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 -- 172.21.15.52:0/2134150242 shutdown_connections 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e5982b400 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e598249e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 --2- 172.21.15.52:0/2134150242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59828730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 -- 172.21.15.52:0/2134150242 >> 172.21.15.52:0/2134150242 conn(0x561e5974fcc0 msgr2=0x561e5982c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 -- 172.21.15.52:0/2134150242 shutdown_connections 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.275+0000 7f24d34beb80 1 -- 172.21.15.52:0/2134150242 wait complete. 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24d34beb80 1 -- start start 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24d34beb80 1 Processor -- start 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59837120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59837660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24d34beb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e59845bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59837120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.276+0000 7f24c259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59837120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50998/0 (socket says 172.21.15.52:50998) 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b45e0 con 0x561e59824f20 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 -- 172.21.15.52:0/1471855666 learned_addr learned my addr 172.21.15.52:0/1471855666 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:08.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b3f10 con 0x561e5982af60 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24c41b1110 con 0x561e5982a640 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24d34beb80 10 client.0 _ll_register_callbacks cb 0x561e59710920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59837660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e59845bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 -- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 msgr2=0x561e59837660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59837660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 -- 172.21.15.52:0/1471855666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 msgr2=0x561e59845bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e59845bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e598463d0 con 0x561e59824f20 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c2d9d700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e59845bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24c259c700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59837120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f24b40098e0 tx=0x7f24b4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38edfcdb3d3027a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.277+0000 7f24a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1471855666 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24b4003a90 con 0x561e59824f20 2024-06-02T23:13:08.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1471855666 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f24b4003c30 con 0x561e59824f20 2024-06-02T23:13:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24bc023070 con 0x561e59824f20 2024-06-02T23:13:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1471855666 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f24b4003e30 con 0x561e59824f20 2024-06-02T23:13:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 10 client.16011 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:08.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561e598a9100 con 0x561e59824f20 2024-06-02T23:13:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1471855666 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f24bc023070 con 0x561e59824f20 2024-06-02T23:13:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1471855666 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f24b403f070 con 0x561e59824f20 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 10 client.16011 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 10 client.16011 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 20 client.16011 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e598a97c0 0x561e598abbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 4 client.16011 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.278+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561e595f8250 con 0x561e598a97c0 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e598a97c0 0x561e598abbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24c1d9b700 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e598a97c0 0x561e598abbe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5dbf701f8fb83003 server_cookie=732fa929af58e0e4 in_seq=0 out_seq=0 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24a9ffb700 10 client.16011 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24a9ffb700 1 -- 172.21.15.52:0/1471855666 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561e595f8250 con 0x561e598a97c0 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24a9ffb700 10 client.16011 handle_command_reply: tid=0 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24d34beb80 1 client.16011 shutdown 2024-06-02T23:13:08.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 msgr2=0x561e59837120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59837120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f24b40098e0 tx=0x7f24b4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 shutdown_connections 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.279+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561e598a97c0 0x561e598abbe0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e5982af60 0x561e59845bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e59824f20 0x561e59837120 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 1 --2- 172.21.15.52:0/1471855666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e5982a640 0x561e59837660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 >> 172.21.15.52:0/1471855666 conn(0x561e5974fcc0 msgr2=0x561e5982b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24a9ffb700 0 client.16011 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 shutdown_connections 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 1 -- 172.21.15.52:0/1471855666 wait complete. 2024-06-02T23:13:08.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.280+0000 7f24d34beb80 20 client.16011 trim_cache size 0 max 16384 2024-06-02T23:13:08.284 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:13:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24ac0a2ca0 msgr2=0x7f24ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24ac0a2ca0 0x7f24ac0a5160 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f24b400b2e0 tx=0x7f24b4005960 comp rx=0 tx=0).stop 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 msgr2=0x7f24c410cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 0x7f24c410cbf0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f24b800e9a0 tx=0x7f24b800ecb0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 shutdown_connections 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24ac0a2ca0 0x7f24ac0a5160 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f24c4108650 0x7f24c41a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24c41027b0 0x7f24c410c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 --2- 172.21.15.52:0/1701150672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24c4103170 0x7f24c410cbf0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 >> 172.21.15.52:0/1701150672 conn(0x7f24c407a420 msgr2=0x7f24c410d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 shutdown_connections 2024-06-02T23:13:08.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:08.281+0000 7f24c8c3b700 1 -- 172.21.15.52:0/1701150672 wait complete. 2024-06-02T23:13:08.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 10644 4.51 MB/sec warmup 17 sec latency 15.459 ms 2024-06-02T23:13:09.087 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:13:09.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:08 smithi123 ceph-mon[35977]: pgmap v469: 97 pgs: 97 active+clean; 26 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-06-02T23:13:09.368 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:08 smithi174 ceph-mon[35994]: pgmap v469: 97 pgs: 97 active+clean; 26 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-06-02T23:13:09.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:08 smithi052 ceph-mon[32340]: pgmap v469: 97 pgs: 97 active+clean; 26 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.7 MiB/s wr, 123 op/s 2024-06-02T23:13:09.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 11010 4.35 MB/sec warmup 18 sec latency 14.151 ms 2024-06-02T23:13:10.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 11310 4.12 MB/sec warmup 19 sec latency 12.761 ms 2024-06-02T23:13:11.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 -- 172.21.15.52:0/4140153807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 msgr2=0x7f0c04101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 --2- 172.21.15.52:0/4140153807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 0x7f0c04101db0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0c000099c0 tx=0x7f0c00009cd0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 -- 172.21.15.52:0/4140153807 shutdown_connections 2024-06-02T23:13:11.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 --2- 172.21.15.52:0/4140153807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c04106720 0x7f0c0410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 --2- 172.21.15.52:0/4140153807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 0x7f0c04101db0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 --2- 172.21.15.52:0/4140153807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c04105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.198+0000 7f0c0b707700 1 -- 172.21.15.52:0/4140153807 >> 172.21.15.52:0/4140153807 conn(0x7f0c04078340 msgr2=0x7f0c04078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:11.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.199+0000 7f0c0b707700 1 -- 172.21.15.52:0/4140153807 shutdown_connections 2024-06-02T23:13:11.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.199+0000 7f0c0b707700 1 -- 172.21.15.52:0/4140153807 wait complete. 2024-06-02T23:13:11.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.199+0000 7f0c0b707700 1 Processor -- start 2024-06-02T23:13:11.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.199+0000 7f0c0b707700 1 -- start start 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c0b707700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 0x7f0c0419f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c0b707700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c0419f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c0b707700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c04106720 0x7f0c041a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c0b707700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111600 con 0x7f0c041019d0 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c0b707700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111480 con 0x7f0c04102380 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c0b707700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c04111780 con 0x7f0c04106720 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c094a3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 0x7f0c0419f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c0419f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c0419f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34590/0 (socket says 172.21.15.52:34590) 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 -- 172.21.15.52:0/2642233467 learned_addr learned my addr 172.21.15.52:0/2642233467 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c04106720 0x7f0c041a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 -- 172.21.15.52:0/2642233467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c04106720 msgr2=0x7f0c041a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c04106720 0x7f0c041a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 -- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 msgr2=0x7f0c0419f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 0x7f0c0419f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 -- 172.21.15.52:0/2642233467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c041a41e0 con 0x7f0c04102380 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.200+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c0419f9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400e940 tx=0x7f0bf400ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.201+0000 7f0bfa7fc700 1 -- 172.21.15.52:0/2642233467 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf400cbd0 con 0x7f0c04102380 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.201+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c041a44d0 con 0x7f0c04102380 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.201+0000 7f0bfa7fc700 1 -- 172.21.15.52:0/2642233467 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf400ee90 con 0x7f0c04102380 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.201+0000 7f0bfa7fc700 1 -- 172.21.15.52:0/2642233467 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf4018680 con 0x7f0c04102380 2024-06-02T23:13:11.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.201+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c041abd90 con 0x7f0c04102380 2024-06-02T23:13:11.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.203+0000 7f0bfa7fc700 1 -- 172.21.15.52:0/2642233467 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0bf4010430 con 0x7f0c04102380 2024-06-02T23:13:11.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.204+0000 7f0bfa7fc700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bf00a2c20 0x7f0bf00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.204+0000 7f0c094a3700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bf00a2c20 0x7f0bf00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.204+0000 7f0bfa7fc700 1 -- 172.21.15.52:0/2642233467 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0bf40cf300 con 0x7f0c04102380 2024-06-02T23:13:11.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c094a3700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bf00a2c20 0x7f0bf00a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0c00000c00 tx=0x7f0c00005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:11.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 Processor -- start 2024-06-02T23:13:11.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 -- start start 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558806ec5f60 con 0x558806fd9880 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558806f24850 con 0x558806fd8f60 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.205+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558806e8c460 con 0x558806fd37a0 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c094a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fd3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51036/0 (socket says 172.21.15.52:51036) 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 -- 172.21.15.52:0/412382364 learned_addr learned my addr 172.21.15.52:0/412382364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 -- 172.21.15.52:0/412382364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 msgr2=0x558806fd3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fd3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 -- 172.21.15.52:0/412382364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 msgr2=0x558806fd9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fd9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 -- 172.21.15.52:0/412382364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558806e738d0 con 0x558806fd9880 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fd3260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0bf409bac0 tx=0x7f0bf409c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51859a3aee8c2046 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.206+0000 7f0bef7fe700 1 -- 172.21.15.52:0/412382364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf409ce60 con 0x558806fd9880 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0bef7fe700 1 -- 172.21.15.52:0/412382364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf409fee0 con 0x558806fd9880 2024-06-02T23:13:11.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0bef7fe700 1 -- 172.21.15.52:0/412382364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40a4310 con 0x558806fd9880 2024-06-02T23:13:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/412382364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 msgr2=0x558806fd3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fd3260 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f0bf409bac0 tx=0x7f0bf409c230 comp rx=0 tx=0).stop 2024-06-02T23:13:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/412382364 shutdown_connections 2024-06-02T23:13:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fd3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/412382364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fd3260 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/412382364 >> 172.21.15.52:0/412382364 conn(0x558806f00080 msgr2=0x558806fce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/412382364 shutdown_connections 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.207+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/412382364 wait complete. 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 -- start start 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 Processor -- start 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fdf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fdc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fdce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558806ec4fc0 con 0x558806fd9880 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558806f20810 con 0x558806fd8f60 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558806fe0260 con 0x558806fd37a0 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c15f8ab80 10 client.0 _ll_register_callbacks cb 0x558806eba5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.208+0000 7f0c09ca4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fdce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fdc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fdce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51044/0 (socket says 172.21.15.52:51044) 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 -- 172.21.15.52:0/3235310446 learned_addr learned my addr 172.21.15.52:0/3235310446 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fdc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34608/0 (socket says 172.21.15.52:34608) 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c094a3700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fdf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 -- 172.21.15.52:0/3235310446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 msgr2=0x558806fdf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fdf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 -- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 msgr2=0x558806fdc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fdc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 -- 172.21.15.52:0/3235310446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558806eea750 con 0x558806fd9880 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fdc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fdce30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc007bf0 tx=0x7f0bfc007f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffa40060d1ca77e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0becff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:11.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0becff9700 1 -- 172.21.15.52:0/3235310446 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc012b10 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0becff9700 1 -- 172.21.15.52:0/3235310446 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0bfc012cb0 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.209+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf409fee0 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0becff9700 1 -- 172.21.15.52:0/3235310446 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bfc025050 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 10 client.16023 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558806feaa40 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0becff9700 1 -- 172.21.15.52:0/3235310446 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf409fee0 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0becff9700 1 -- 172.21.15.52:0/3235310446 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0bfc015450 con 0x558806fd9880 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 10 client.16023 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 20 client.16023 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x558806fef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 4 client.16023 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:11.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558806f31d50 con 0x558806fed120 2024-06-02T23:13:11.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.210+0000 7f0c094a3700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x558806fef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.211+0000 7f0becff9700 10 client.16023 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:11.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.211+0000 7f0c094a3700 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x558806fef540 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=645c364bb2720383 server_cookie=5814dffb24a7d226 in_seq=0 out_seq=0 2024-06-02T23:13:11.215 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:10 smithi052 ceph-mon[32340]: pgmap v470: 97 pgs: 97 active+clean; 26 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0becff9700 1 -- 172.21.15.52:0/3235310446 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558806f31d50 con 0x558806fed120 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0becff9700 10 client.16023 handle_command_reply: tid=0 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 client.16023 shutdown 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 msgr2=0x558806fdce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fdce30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc007bf0 tx=0x7f0bfc007f00 comp rx=0 tx=0).stop 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 shutdown_connections 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x558806fef540 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd9880 0x558806fdce30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0becff9700 0 client.16023 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd37a0 0x558806fdf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/3235310446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd8f60 0x558806fdc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 >> 172.21.15.52:0/3235310446 conn(0x558806f00080 msgr2=0x558806fcd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:11.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 shutdown_connections 2024-06-02T23:13:11.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/3235310446 wait complete. 2024-06-02T23:13:11.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.212+0000 7f0c15f8ab80 20 client.16023 trim_cache size 0 max 16384 2024-06-02T23:13:11.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 Processor -- start 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 -- start start 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fd3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fdce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fde830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041b31b0 con 0x558806fd8f60 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041b4d80 con 0x558806fd37a0 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.231+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041b3880 con 0x558806fd9880 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fdce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fdce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51058/0 (socket says 172.21.15.52:51058) 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c094a3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fd3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 -- 172.21.15.52:0/804992682 learned_addr learned my addr 172.21.15.52:0/804992682 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:11.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fde830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 -- 172.21.15.52:0/804992682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 msgr2=0x558806fde830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fde830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 -- 172.21.15.52:0/804992682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 msgr2=0x558806fd3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fd3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 -- 172.21.15.52:0/804992682 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558806fd8a10 con 0x558806fd8f60 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fde830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c094a3700 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fd3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fdce60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c5b0 tx=0x7f0bf4005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a903653397d2e02 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0beeffd700 1 -- 172.21.15.52:0/804992682 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf409ce60 con 0x558806fd8f60 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0beeffd700 1 -- 172.21.15.52:0/804992682 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf40a47a0 con 0x558806fd8f60 2024-06-02T23:13:11.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.232+0000 7f0beeffd700 1 -- 172.21.15.52:0/804992682 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf40ba200 con 0x558806fd8f60 2024-06-02T23:13:11.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/804992682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 msgr2=0x558806fdce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fdce60 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400c5b0 tx=0x7f0bf4005b40 comp rx=0 tx=0).stop 2024-06-02T23:13:11.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/804992682 shutdown_connections 2024-06-02T23:13:11.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fde830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fd3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/804992682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fdce60 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/804992682 >> 172.21.15.52:0/804992682 conn(0x558806f00080 msgr2=0x558806fde120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/804992682 shutdown_connections 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.233+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/804992682 wait complete. 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 -- start start 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 Processor -- start 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fe5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fe5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fe24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041b4620 con 0x558806fd8f60 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041b3f50 con 0x558806fd37a0 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c041b1150 con 0x558806fd9880 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c15f8ab80 10 client.0 _ll_register_callbacks cb 0x558806eba5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:11.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fe5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c08ca2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fe5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51064/0 (socket says 172.21.15.52:51064) 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c08ca2700 1 -- 172.21.15.52:0/721647451 learned_addr learned my addr 172.21.15.52:0/721647451 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c094a3700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fe5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.234+0000 7f0c09ca4700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fe24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c08ca2700 1 -- 172.21.15.52:0/721647451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 msgr2=0x558806fe24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fe24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c08ca2700 1 -- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 msgr2=0x558806fe5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fe5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c08ca2700 1 -- 172.21.15.52:0/721647451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558806fe2a30 con 0x558806fd8f60 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c08ca2700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fe5a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4010a00 tx=0x7f0bf409c120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fbc15c16b80b0a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0becff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0becff9700 1 -- 172.21.15.52:0/721647451 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf407e070 con 0x558806fd8f60 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0becff9700 1 -- 172.21.15.52:0/721647451 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0bf409b190 con 0x558806fd8f60 2024-06-02T23:13:11.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bf40a47a0 con 0x558806fd8f60 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0becff9700 1 -- 172.21.15.52:0/721647451 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0bf409ce60 con 0x558806fd8f60 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c15f8ab80 10 client.16029 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5588070583b0 con 0x558806fd8f60 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.235+0000 7f0becff9700 1 -- 172.21.15.52:0/721647451 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bf40a47a0 con 0x558806fd8f60 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0becff9700 1 -- 172.21.15.52:0/721647451 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f0bf4081030 con 0x558806fd8f60 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c15f8ab80 10 client.16029 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c15f8ab80 10 client.16029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c15f8ab80 20 client.16029 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x55880705a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c15f8ab80 4 client.16029 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558806f2c2d0 con 0x558806fed120 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c094a3700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x55880705a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:11.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0becff9700 10 client.16029 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.236+0000 7f0c094a3700 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x55880705a640 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be521cb80871873f server_cookie=20f4bca8c349674f in_seq=0 out_seq=0 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0becff9700 1 -- 172.21.15.52:0/721647451 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558806f2c2d0 con 0x558806fed120 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0becff9700 10 client.16029 handle_command_reply: tid=0 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 client.16029 shutdown 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 msgr2=0x558806fe5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fe5a60 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f0bf4010a00 tx=0x7f0bf409c120 comp rx=0 tx=0).stop 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 shutdown_connections 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558806fed120 0x55880705a640 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558806fd9880 0x558806fe24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558806fd37a0 0x558806fe5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 --2- 172.21.15.52:0/721647451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558806fd8f60 0x558806fe5a60 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 >> 172.21.15.52:0/721647451 conn(0x558806f00080 msgr2=0x558806fde9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0becff9700 0 client.16029 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 shutdown_connections 2024-06-02T23:13:11.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 1 -- 172.21.15.52:0/721647451 wait complete. 2024-06-02T23:13:11.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c15f8ab80 20 client.16029 trim_cache size 0 max 16384 2024-06-02T23:13:11.242 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:13:11.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bf00a2c20 msgr2=0x7f0bf00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.237+0000 7f0c0b707700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bf00a2c20 0x7f0bf00a50e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f0c00000c00 tx=0x7f0c00005fb0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 msgr2=0x7f0c0419f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c0419f9f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0bf400e940 tx=0x7f0bf400ec50 comp rx=0 tx=0).stop 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 shutdown_connections 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bf00a2c20 0x7f0bf00a50e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c04106720 0x7f0c041a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c041019d0 0x7f0c0419f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 --2- 172.21.15.52:0/2642233467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c04102380 0x7f0c0419f9f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 >> 172.21.15.52:0/2642233467 conn(0x7f0c04078340 msgr2=0x7f0c0406acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 shutdown_connections 2024-06-02T23:13:11.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:11.238+0000 7f0c0b707700 1 -- 172.21.15.52:0/2642233467 wait complete. 2024-06-02T23:13:11.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:10 smithi123 ceph-mon[35977]: pgmap v470: 97 pgs: 97 active+clean; 26 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-06-02T23:13:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:10 smithi174 ceph-mon[35994]: pgmap v470: 97 pgs: 97 active+clean; 26 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-06-02T23:13:11.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 11829 4.02 MB/sec warmup 20 sec latency 41.460 ms 2024-06-02T23:13:12.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 12497 3.98 MB/sec warmup 21 sec latency 15.885 ms 2024-06-02T23:13:13.030 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:13:13.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:13:13.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:12 smithi123 ceph-mon[35977]: pgmap v471: 97 pgs: 97 active+clean; 28 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-06-02T23:13:13.305 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:12 smithi174 ceph-mon[35994]: pgmap v471: 97 pgs: 97 active+clean; 28 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-06-02T23:13:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:12 smithi052 ceph-mon[32340]: pgmap v471: 97 pgs: 97 active+clean; 28 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 128 op/s 2024-06-02T23:13:13.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 13092 3.88 MB/sec warmup 22 sec latency 14.756 ms 2024-06-02T23:13:14.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 13859 3.98 MB/sec warmup 23 sec latency 15.268 ms 2024-06-02T23:13:15.048 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:14 smithi052 ceph-mon[32340]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 -- 172.21.15.52:0/1016527793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102ef0 msgr2=0x7efee010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 --2- 172.21.15.52:0/1016527793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102ef0 0x7efee010f3a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7efed80099c0 tx=0x7efed8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 -- 172.21.15.52:0/1016527793 shutdown_connections 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 --2- 172.21.15.52:0/1016527793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102ef0 0x7efee010f3a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 --2- 172.21.15.52:0/1016527793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee01085d0 0x7efee01089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 --2- 172.21.15.52:0/1016527793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee0102530 0x7efee01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 -- 172.21.15.52:0/1016527793 >> 172.21.15.52:0/1016527793 conn(0x7efee0079640 msgr2=0x7efee0079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 -- 172.21.15.52:0/1016527793 shutdown_connections 2024-06-02T23:13:15.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.170+0000 7efee861b700 1 -- 172.21.15.52:0/1016527793 wait complete. 2024-06-02T23:13:15.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.171+0000 7efee861b700 1 Processor -- start 2024-06-02T23:13:15.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.171+0000 7efee861b700 1 -- start start 2024-06-02T23:13:15.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.171+0000 7efee861b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 0x7efee019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.171+0000 7efee861b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee0102ef0 0x7efee019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee861b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee01085d0 0x7efee0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee861b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efee01126f0 con 0x7efee0102530 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee861b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112570 con 0x7efee01085d0 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee861b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112870 con 0x7efee0102ef0 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 0x7efee019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 0x7efee019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55384/0 (socket says 172.21.15.52:55384) 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 -- 172.21.15.52:0/2755554551 learned_addr learned my addr 172.21.15.52:0/2755554551 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee5bb6700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee0102ef0 0x7efee019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee6bb8700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee01085d0 0x7efee0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 -- 172.21.15.52:0/2755554551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee0102ef0 msgr2=0x7efee019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee0102ef0 0x7efee019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 -- 172.21.15.52:0/2755554551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee01085d0 msgr2=0x7efee0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee01085d0 0x7efee0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 -- 172.21.15.52:0/2755554551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01990b0 con 0x7efee0102530 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee63b7700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 0x7efee019e740 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efed000ef10 tx=0x7efed000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:15.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efedf7fe700 1 -- 172.21.15.52:0/2755554551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000ca70 con 0x7efee0102530 2024-06-02T23:13:15.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efedf7fe700 1 -- 172.21.15.52:0/2755554551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed0019070 con 0x7efee0102530 2024-06-02T23:13:15.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee0199340 con 0x7efee0102530 2024-06-02T23:13:15.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efedf7fe700 1 -- 172.21.15.52:0/2755554551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed0014480 con 0x7efee0102530 2024-06-02T23:13:15.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.172+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a5610 con 0x7efee0102530 2024-06-02T23:13:15.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.175+0000 7efedf7fe700 1 -- 172.21.15.52:0/2755554551 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efed0012050 con 0x7efee0102530 2024-06-02T23:13:15.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.175+0000 7efedf7fe700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efec80a2ca0 0x7efec80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.175+0000 7efedf7fe700 1 -- 172.21.15.52:0/2755554551 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efed00cefa0 con 0x7efee0102530 2024-06-02T23:13:15.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.175+0000 7efee5bb6700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efec80a2ca0 0x7efec80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.176+0000 7efee5bb6700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efec80a2ca0 0x7efec80a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efee0079c40 tx=0x7efed4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:15.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 Processor -- start 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 -- start start 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e28d0 0x560ef98e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98e7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98e7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560ef97d0fe0 con 0x560ef98e31f0 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560ef982f8d0 con 0x560ef98e75d0 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ef97974e0 con 0x560ef98e28d0 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efee5bb6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98e7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efee63b7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e28d0 0x560ef98e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efee5bb6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98e7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55386/0 (socket says 172.21.15.52:55386) 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efee5bb6700 1 -- 172.21.15.52:0/889888780 learned_addr learned my addr 172.21.15.52:0/889888780 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:15.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.177+0000 7efee6bb8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98e7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee5bb6700 1 -- 172.21.15.52:0/889888780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e28d0 msgr2=0x560ef98e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee5bb6700 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e28d0 0x560ef98e2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee5bb6700 1 -- 172.21.15.52:0/889888780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 msgr2=0x560ef98e7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee5bb6700 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98e7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee5bb6700 1 -- 172.21.15.52:0/889888780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef977e950 con 0x560ef98e31f0 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee6bb8700 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:15.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efee5bb6700 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98e7090 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efed4011da0 tx=0x7efed4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc63a7943392e0fb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efedcff9700 1 -- 172.21.15.52:0/889888780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4004a30 con 0x560ef98e31f0 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efedcff9700 1 -- 172.21.15.52:0/889888780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed40054d0 con 0x560ef98e31f0 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.178+0000 7efedcff9700 1 -- 172.21.15.52:0/889888780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4003ba0 con 0x560ef98e31f0 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 -- 172.21.15.52:0/889888780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 msgr2=0x560ef98e7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98e7090 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efed4011da0 tx=0x7efed4014000 comp rx=0 tx=0).stop 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 -- 172.21.15.52:0/889888780 shutdown_connections 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98e7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e28d0 0x560ef98e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/889888780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98e7090 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 -- 172.21.15.52:0/889888780 >> 172.21.15.52:0/889888780 conn(0x560ef980b100 msgr2=0x560ef98d92a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 -- 172.21.15.52:0/889888780 shutdown_connections 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 -- 172.21.15.52:0/889888780 wait complete. 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 -- start start 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.179+0000 7efef2e9eb80 1 Processor -- start 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef98eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98ebee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef98ec420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560ef97d0040 con 0x560ef98e31f0 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560ef982b890 con 0x560ef98e28d0 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560ef98ef700 con 0x560ef98e75d0 2024-06-02T23:13:15.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efef2e9eb80 10 client.0 _ll_register_callbacks cb 0x560ef97c5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98ebee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98ebee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55402/0 (socket says 172.21.15.52:55402) 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 -- 172.21.15.52:0/1088292862 learned_addr learned my addr 172.21.15.52:0/1088292862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee6bb8700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef98ec420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee63b7700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef98eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 -- 172.21.15.52:0/1088292862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 msgr2=0x560ef98ec420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef98ec420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 -- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 msgr2=0x560ef98eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef98eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee5bb6700 1 -- 172.21.15.52:0/1088292862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef97f57d0 con 0x560ef98e31f0 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.180+0000 7efee6bb8700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef98ec420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efee5bb6700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98ebee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efed40048e0 tx=0x7efed4004910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb17fe1f3c26e1f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:15.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efec5ffb700 1 -- 172.21.15.52:0/1088292862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4005680 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efec5ffb700 1 -- 172.21.15.52:0/1088292862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efed4003820 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed40054d0 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efec5ffb700 1 -- 172.21.15.52:0/1088292862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efed4003a20 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efef2e9eb80 10 client.16041 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ef98f9f30 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.181+0000 7efec5ffb700 1 -- 172.21.15.52:0/1088292862 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed40054d0 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efec5ffb700 1 -- 172.21.15.52:0/1088292862 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7efed40318d0 con 0x560ef98e31f0 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efef2e9eb80 10 client.16041 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efef2e9eb80 10 client.16041 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efef2e9eb80 20 client.16041 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef98fc400 0x560ef98fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efef2e9eb80 4 client.16041 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:15.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef983cdd0 con 0x560ef98fc400 2024-06-02T23:13:15.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efee63b7700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef98fc400 0x560ef98fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efee63b7700 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef98fc400 0x560ef98fe7f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee4f428a71f9e21e server_cookie=2447259264cb3ab6 in_seq=0 out_seq=0 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.182+0000 7efec5ffb700 10 client.16041 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.183+0000 7efec5ffb700 1 -- 172.21.15.52:0/1088292862 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560ef983cdd0 con 0x560ef98fc400 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.183+0000 7efec5ffb700 10 client.16041 handle_command_reply: tid=0 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.183+0000 7efef2e9eb80 1 client.16041 shutdown 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 msgr2=0x560ef98ebee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98ebee0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efed40048e0 tx=0x7efed4004910 comp rx=0 tx=0).stop 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 shutdown_connections 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef98fc400 0x560ef98fe7f0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef98ec420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef98eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/1088292862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e31f0 0x560ef98ebee0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 >> 172.21.15.52:0/1088292862 conn(0x560ef980b100 msgr2=0x560ef98e8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efec5ffb700 0 client.16041 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 shutdown_connections 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 1 -- 172.21.15.52:0/1088292862 wait complete. 2024-06-02T23:13:15.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.184+0000 7efef2e9eb80 20 client.16041 trim_cache size 0 max 16384 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.207+0000 7efef2e9eb80 1 Processor -- start 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.207+0000 7efef2e9eb80 1 -- start start 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 0x560ef98ee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98f7f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98eec60 0x560ef98f8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aa820 con 0x560ef98e28d0 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee01ac3f0 con 0x560ef98e75d0 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efee01aaef0 con 0x560ef98eec60 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 0x560ef98ee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 0x560ef98ee720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55406/0 (socket says 172.21.15.52:55406) 2024-06-02T23:13:15.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 -- 172.21.15.52:0/740511597 learned_addr learned my addr 172.21.15.52:0/740511597 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee5bb6700 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98f7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee6bb8700 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98eec60 0x560ef98f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 -- 172.21.15.52:0/740511597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98eec60 msgr2=0x560ef98f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98eec60 0x560ef98f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 -- 172.21.15.52:0/740511597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 msgr2=0x560ef98f7f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98f7f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 -- 172.21.15.52:0/740511597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef98d5890 con 0x560ef98e28d0 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efee63b7700 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 0x560ef98ee720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efed0009c10 tx=0x7efed00cca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f028ba12c3014af9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.208+0000 7efec7fff700 1 -- 172.21.15.52:0/740511597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed009c750 con 0x560ef98e28d0 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efec7fff700 1 -- 172.21.15.52:0/740511597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed00a0b00 con 0x560ef98e28d0 2024-06-02T23:13:15.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efec7fff700 1 -- 172.21.15.52:0/740511597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed00a4f30 con 0x560ef98e28d0 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 -- 172.21.15.52:0/740511597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 msgr2=0x560ef98ee720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 0x560ef98ee720 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7efed0009c10 tx=0x7efed00cca40 comp rx=0 tx=0).stop 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 -- 172.21.15.52:0/740511597 shutdown_connections 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98eec60 0x560ef98f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98e28d0 0x560ef98ee720 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/740511597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e75d0 0x560ef98f7f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.209+0000 7efef2e9eb80 1 -- 172.21.15.52:0/740511597 >> 172.21.15.52:0/740511597 conn(0x560ef980b100 msgr2=0x560ef98f7810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 -- 172.21.15.52:0/740511597 shutdown_connections 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 -- 172.21.15.52:0/740511597 wait complete. 2024-06-02T23:13:15.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 -- start start 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 Processor -- start 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef99608c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef9960e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98eec60 0x560ef995ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efee01abc90 con 0x560ef98eec60 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efee01ab5c0 con 0x560ef98e28d0 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efee01a87c0 con 0x560ef98e75d0 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efef2e9eb80 10 client.0 _ll_register_callbacks cb 0x560ef97c5630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef9960e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.210+0000 7efee6bb8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98eec60 0x560ef995ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef9960e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35464/0 (socket says 172.21.15.52:35464) 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 -- 172.21.15.52:0/37526024 learned_addr learned my addr 172.21.15.52:0/37526024 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:15.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee63b7700 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef99608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 -- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 msgr2=0x560ef99608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef99608c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 -- 172.21.15.52:0/37526024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98eec60 msgr2=0x560ef995ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98eec60 0x560ef995ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 -- 172.21.15.52:0/37526024 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef98e7390 con 0x560ef98e75d0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efee5bb6700 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef9960e00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efed40117d0 tx=0x7efed4011880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abcb407e2ac3dbd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efec5ffb700 1 -- 172.21.15.52:0/37526024 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed403c420 con 0x560ef98e75d0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed00a0b00 con 0x560ef98e75d0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efec5ffb700 1 -- 172.21.15.52:0/37526024 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efed4005430 con 0x560ef98e75d0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efec5ffb700 1 -- 172.21.15.52:0/37526024 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efed404c050 con 0x560ef98e75d0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efef2e9eb80 10 client.25561 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.211+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560ef9967f50 con 0x560ef98e75d0 2024-06-02T23:13:15.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efec5ffb700 1 -- 172.21.15.52:0/37526024 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efed00a0b00 con 0x560ef98e75d0 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efec5ffb700 1 -- 172.21.15.52:0/37526024 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7efed4003bc0 con 0x560ef98e75d0 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efef2e9eb80 10 client.25561 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efef2e9eb80 10 client.25561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efef2e9eb80 20 client.25561 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef9968630 0x560ef996aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efef2e9eb80 4 client.25561 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560ef98e9ee0 con 0x560ef9968630 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efee63b7700 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef9968630 0x560ef996aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efee63b7700 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef9968630 0x560ef996aa50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c88dedf499d2a07 server_cookie=a50c08795dd31476 in_seq=0 out_seq=0 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.212+0000 7efec5ffb700 10 client.25561 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efec5ffb700 1 -- 172.21.15.52:0/37526024 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560ef98e9ee0 con 0x560ef9968630 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efec5ffb700 10 client.25561 handle_command_reply: tid=0 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 client.25561 shutdown 2024-06-02T23:13:15.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 msgr2=0x560ef9960e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef9960e00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efed40117d0 tx=0x7efed4011880 comp rx=0 tx=0).stop 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 shutdown_connections 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560ef9968630 0x560ef996aa50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560ef98eec60 0x560ef995ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560ef98e28d0 0x560ef99608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 --2- 172.21.15.52:0/37526024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560ef98e75d0 0x560ef9960e00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 >> 172.21.15.52:0/37526024 conn(0x560ef980b100 msgr2=0x560ef98edfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efec5ffb700 0 client.25561 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 shutdown_connections 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 1 -- 172.21.15.52:0/37526024 wait complete. 2024-06-02T23:13:15.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.213+0000 7efef2e9eb80 20 client.25561 trim_cache size 0 max 16384 2024-06-02T23:13:15.219 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:13:15.219 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:13:15.219 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "6410c6f0-c37c-46ea-8b6a-8304ad909d7b", 2024-06-02T23:13:15.219 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:13:15.219 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efec80a2ca0 msgr2=0x7efec80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efec80a2ca0 0x7efec80a5160 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7efee0079c40 tx=0x7efed4009260 comp rx=0 tx=0).stop 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 msgr2=0x7efee019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 0x7efee019e740 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7efed000ef10 tx=0x7efed000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 shutdown_connections 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efec80a2ca0 0x7efec80a5160 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efee01085d0 0x7efee0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efee0102530 0x7efee019e740 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 --2- 172.21.15.52:0/2755554551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efee0102ef0 0x7efee019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.215+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 >> 172.21.15.52:0/2755554551 conn(0x7efee0079640 msgr2=0x7efee0101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.216+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 shutdown_connections 2024-06-02T23:13:15.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:15.216+0000 7efee861b700 1 -- 172.21.15.52:0/2755554551 wait complete. 2024-06-02T23:13:15.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:14 smithi123 ceph-mon[35977]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:13:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:14 smithi174 ceph-mon[35994]: pgmap v472: 97 pgs: 97 active+clean; 28 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:13:15.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 14304 4.00 MB/sec warmup 24 sec latency 17.055 ms 2024-06-02T23:13:15.866 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:13:16.131 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:13:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:15 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:15 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:13:16.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:16.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:16.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:16.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:16.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:16.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:15 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:16.604 INFO:tasks.workunit.client.0.smithi052.stdout: 1 14599 3.89 MB/sec warmup 25 sec latency 13.885 ms 2024-06-02T23:13:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:16 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:16 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:16 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:13:17.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:16 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:17.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:16 smithi052 ceph-mon[32340]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-02T23:13:17.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:16 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:16 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:16 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:13:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:16 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:16 smithi123 ceph-mon[35977]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-02T23:13:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:16 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:16 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:16 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:13:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:16 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:13:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:16 smithi174 ceph-mon[35994]: pgmap v473: 97 pgs: 97 active+clean; 28 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-02T23:13:17.617 INFO:tasks.workunit.client.0.smithi052.stdout: 1 14981 3.76 MB/sec warmup 26 sec latency 15.523 ms 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 -- 172.21.15.52:0/931278387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb4105060 msgr2=0x7f3bb4111570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/931278387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb4105060 0x7f3bb4111570 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3bac00b0a0 tx=0x7f3bac00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 -- 172.21.15.52:0/931278387 shutdown_connections 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/931278387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb4105060 0x7f3bb4111570 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/931278387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb410a6a0 0x7f3bb410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/931278387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb41046a0 0x7f3bb4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 -- 172.21.15.52:0/931278387 >> 172.21.15.52:0/931278387 conn(0x7f3bb4100190 msgr2=0x7f3bb41025b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 -- 172.21.15.52:0/931278387 shutdown_connections 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 -- 172.21.15.52:0/931278387 wait complete. 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.925+0000 7f3bb9e29700 1 Processor -- start 2024-06-02T23:13:18.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 -- start start 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 0x7f3bb41a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb4105060 0x7f3bb41a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb410a6a0 0x7f3bb419ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4114880 con 0x7f3bb41046a0 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4114700 con 0x7f3bb4105060 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb9e29700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb4114a00 con 0x7f3bb410a6a0 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 0x7f3bb41a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb4105060 0x7f3bb41a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 0x7f3bb41a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55432/0 (socket says 172.21.15.52:55432) 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 -- 172.21.15.52:0/377911239 learned_addr learned my addr 172.21.15.52:0/377911239 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb3fff700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb410a6a0 0x7f3bb419ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 -- 172.21.15.52:0/377911239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb410a6a0 msgr2=0x7f3bb419ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb410a6a0 0x7f3bb419ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 -- 172.21.15.52:0/377911239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb4105060 msgr2=0x7f3bb41a0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb4105060 0x7f3bb41a0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb37fe700 1 -- 172.21.15.52:0/377911239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb419b440 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.926+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb4105060 0x7f3bb41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb3fff700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb410a6a0 0x7f3bb419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 0x7f3bb41a0a40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9a0 tx=0x7f3ba400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb0ff9700 1 -- 172.21.15.52:0/377911239 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba400cbd0 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb0ff9700 1 -- 172.21.15.52:0/377911239 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba400cd70 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb419b730 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb0ff9700 1 -- 172.21.15.52:0/377911239 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4018680 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.927+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb41a79c0 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.930+0000 7f3bb0ff9700 1 -- 172.21.15.52:0/377911239 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ba4010430 con 0x7f3bb41046a0 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.930+0000 7f3bb0ff9700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3b9c0a2c20 0x7f3b9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.930+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3b9c0a2c20 0x7f3b9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bb0ff9700 1 -- 172.21.15.52:0/377911239 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ba40cf200 con 0x7f3bb41046a0 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3b9c0a2c20 0x7f3b9c0a50e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3bb419c260 tx=0x7f3ba8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 Processor -- start 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 -- start start 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c2292a6f60 con 0x55c2293ba880 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.931+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c229305850 con 0x55c2293b9f60 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c22926d460 con 0x55c2293b47a0 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293ba340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36402/0 (socket says 172.21.15.52:36402) 2024-06-02T23:13:18.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 -- 172.21.15.52:0/1076950790 learned_addr learned my addr 172.21.15.52:0/1076950790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb3fff700 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 -- 172.21.15.52:0/1076950790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 msgr2=0x55c2293b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 -- 172.21.15.52:0/1076950790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 msgr2=0x55c2293b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 -- 172.21.15.52:0/1076950790 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2292548d0 con 0x55c2293b9f60 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293ba340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40a41d0 tx=0x7f3ba409b6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4f9e818b205afa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3b99ffb700 1 -- 172.21.15.52:0/1076950790 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40b4070 con 0x55c2293b9f60 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3b99ffb700 1 -- 172.21.15.52:0/1076950790 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba409c190 con 0x55c2293b9f60 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.932+0000 7f3b99ffb700 1 -- 172.21.15.52:0/1076950790 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40a44d0 con 0x55c2293b9f60 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 -- 172.21.15.52:0/1076950790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 msgr2=0x55c2293ba340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293ba340 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40a41d0 tx=0x7f3ba409b6a0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 -- 172.21.15.52:0/1076950790 shutdown_connections 2024-06-02T23:13:18.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293ba340 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/1076950790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 -- 172.21.15.52:0/1076950790 >> 172.21.15.52:0/1076950790 conn(0x55c2292e1080 msgr2=0x55c2293af220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 -- 172.21.15.52:0/1076950790 shutdown_connections 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.933+0000 7f3bc46acb80 1 -- 172.21.15.52:0/1076950790 wait complete. 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 -- start start 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 Processor -- start 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293c0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c2292a5fc0 con 0x55c2293ba880 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c229301810 con 0x55c2293b9f60 2024-06-02T23:13:18.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2293c1260 con 0x55c2293b47a0 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.934+0000 7f3bc46acb80 10 client.0 _ll_register_callbacks cb 0x55c22929b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293c0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bd9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36408/0 (socket says 172.21.15.52:36408) 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293c0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35506/0 (socket says 172.21.15.52:35506) 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/857553803 learned_addr learned my addr 172.21.15.52:0/857553803 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/857553803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 msgr2=0x55c2293c0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293c0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/857553803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 msgr2=0x55c2293bdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293bdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/857553803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2292cb750 con 0x55c2293b9f60 2024-06-02T23:13:18.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bd9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80037e0 tx=0x7f3ba80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5d42d6cc2970246 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3b92ffd700 1 -- 172.21.15.52:0/857553803 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba801e800 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ba409c190 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.935+0000 7f3b92ffd700 1 -- 172.21.15.52:0/857553803 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ba801e9a0 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3b92ffd700 1 -- 172.21.15.52:0/857553803 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ba801eba0 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 10 client.25640 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c2293cbb20 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3b92ffd700 1 -- 172.21.15.52:0/857553803 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba409c190 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3b92ffd700 1 -- 172.21.15.52:0/857553803 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3ba8026020 con 0x55c2293b9f60 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 10 client.25640 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 10 client.25640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:18.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 20 client.25640 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2293ce150 0x55c2293d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 4 client.25640 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c229312d50 con 0x55c2293ce150 2024-06-02T23:13:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.936+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2293ce150 0x55c2293d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.937+0000 7f3b92ffd700 10 client.25640 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.937+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2293ce150 0x55c2293d0570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f421a06fc6ed564 server_cookie=6af80a6459062485 in_seq=0 out_seq=0 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3b92ffd700 1 -- 172.21.15.52:0/857553803 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c229312d50 con 0x55c2293ce150 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3b92ffd700 10 client.25640 handle_command_reply: tid=0 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 client.25640 shutdown 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 msgr2=0x55c2293bd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bd9a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ba80037e0 tx=0x7f3ba80047c0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 shutdown_connections 2024-06-02T23:13:18.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2293ce150 0x55c2293d0570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293ba880 0x55c2293bdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293b47a0 0x55c2293c0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/857553803 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bd9a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 >> 172.21.15.52:0/857553803 conn(0x55c2292e1080 msgr2=0x55c2293ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3b92ffd700 0 client.25640 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 shutdown_connections 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 1 -- 172.21.15.52:0/857553803 wait complete. 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.938+0000 7f3bc46acb80 20 client.25640 trim_cache size 0 max 16384 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 Processor -- start 2024-06-02T23:13:18.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 -- start start 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 0x55c2293b4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293aee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41aec80 con 0x55c2293b47a0 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b0850 con 0x55c2293b9f60 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41af350 con 0x55c2293ba880 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 0x55c2293b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 0x55c2293b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55468/0 (socket says 172.21.15.52:55468) 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bb37fe700 1 -- 172.21.15.52:0/2226761318 learned_addr learned my addr 172.21.15.52:0/2226761318 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.962+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb3fff700 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb37fe700 1 -- 172.21.15.52:0/2226761318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 msgr2=0x55c2293aee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293aee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb37fe700 1 -- 172.21.15.52:0/2226761318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 msgr2=0x55c2293bdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb37fe700 1 -- 172.21.15.52:0/2226761318 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2293b0be0 con 0x55c2293b47a0 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 0x55c2293b4260 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40ccd80 tx=0x7f3ba400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d87295c733d0ca6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3b997fa700 1 -- 172.21.15.52:0/2226761318 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40b4040 con 0x55c2293b47a0 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3b997fa700 1 -- 172.21.15.52:0/2226761318 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ba409ba90 con 0x55c2293b47a0 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.963+0000 7f3b997fa700 1 -- 172.21.15.52:0/2226761318 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba40ba3d0 con 0x55c2293b47a0 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 -- 172.21.15.52:0/2226761318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 msgr2=0x55c2293b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 0x55c2293b4260 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40ccd80 tx=0x7f3ba400ef70 comp rx=0 tx=0).stop 2024-06-02T23:13:18.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 -- 172.21.15.52:0/2226761318 shutdown_connections 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293aee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b47a0 0x55c2293b4260 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/2226761318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b9f60 0x55c2293bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 -- 172.21.15.52:0/2226761318 >> 172.21.15.52:0/2226761318 conn(0x55c2292e1080 msgr2=0x55c2293bfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 -- 172.21.15.52:0/2226761318 shutdown_connections 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 -- 172.21.15.52:0/2226761318 wait complete. 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.964+0000 7f3bc46acb80 1 -- start start 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 Processor -- start 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b47a0 0x55c2293d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 0x55c2293d60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293d65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41b00f0 con 0x55c2293b9f60 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41afa20 con 0x55c2293b47a0 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb41acc40 con 0x55c2293ba880 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bc46acb80 10 client.0 _ll_register_callbacks cb 0x55c22929b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 0x55c2293d60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 0x55c2293d60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55478/0 (socket says 172.21.15.52:55478) 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/4020112745 learned_addr learned my addr 172.21.15.52:0/4020112745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b47a0 0x55c2293d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb3fff700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293d65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/4020112745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 msgr2=0x55c2293d65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293d65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b47a0 msgr2=0x55c2293d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b47a0 0x55c2293d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.965+0000 7f3bb2ffd700 1 -- 172.21.15.52:0/4020112745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2293ab810 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3bb2ffd700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 0x55c2293d60b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009f10 tx=0x7f3ba8011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de158511b554217f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3b92ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3b92ffd700 1 -- 172.21.15.52:0/4020112745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800e8d0 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3b92ffd700 1 -- 172.21.15.52:0/4020112745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ba800ea70 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c22930d2d0 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3b92ffd700 1 -- 172.21.15.52:0/4020112745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f3ba8043070 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3bc46acb80 10 client.16065 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c229439070 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.966+0000 7f3b92ffd700 1 -- 172.21.15.52:0/4020112745 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c22930d2d0 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3b92ffd700 1 -- 172.21.15.52:0/4020112745 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f3ba8026040 con 0x55c2293b9f60 2024-06-02T23:13:18.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bc46acb80 10 client.16065 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bc46acb80 10 client.16065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bc46acb80 20 client.16065 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2294397e0 0x55c22943bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bc46acb80 4 client.16065 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2292c0f50 con 0x55c2294397e0 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2294397e0 0x55c22943bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3b92ffd700 10 client.16065 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.967+0000 7f3bb37fe700 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2294397e0 0x55c22943bc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3566d26f0dc093 server_cookie=73b28c2acf58445a in_seq=0 out_seq=0 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3b92ffd700 1 -- 172.21.15.52:0/4020112745 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c2292c0f50 con 0x55c2294397e0 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3b92ffd700 10 client.16065 handle_command_reply: tid=0 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 client.16065 shutdown 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 msgr2=0x55c2293d60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 0x55c2293d60b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009f10 tx=0x7f3ba8011bb0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 shutdown_connections 2024-06-02T23:13:18.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2294397e0 0x55c22943bc00 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3b92ffd700 0 client.16065 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2293ba880 0x55c2293d65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2293b47a0 0x55c2293d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 --2- 172.21.15.52:0/4020112745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2293b9f60 0x55c2293d60b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 >> 172.21.15.52:0/4020112745 conn(0x55c2292e1080 msgr2=0x55c2293b8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 shutdown_connections 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 1 -- 172.21.15.52:0/4020112745 wait complete. 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.968+0000 7f3bc46acb80 20 client.16065 trim_cache size 0 max 16384 2024-06-02T23:13:18.589 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: "6410c6f0-c37c-46ea-8b6a-8304ad909d7b": { 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "6410c6f0-c37c-46ea-8b6a-8304ad909d7b", 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3b9c0a2c20 msgr2=0x7f3b9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3b9c0a2c20 0x7f3b9c0a50e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f3bb419c260 tx=0x7f3ba8009260 comp rx=0 tx=0).stop 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 msgr2=0x7f3bb41a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 0x7f3bb41a0a40 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9a0 tx=0x7f3ba400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 shutdown_connections 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3b9c0a2c20 0x7f3b9c0a50e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3bb410a6a0 0x7f3bb419ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3bb41046a0 0x7f3bb41a0a40 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 --2- 172.21.15.52:0/377911239 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3bb4105060 0x7f3bb41a0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 >> 172.21.15.52:0/377911239 conn(0x7f3bb4100190 msgr2=0x7f3bb4100750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 shutdown_connections 2024-06-02T23:13:18.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:17.969+0000 7f3bb9e29700 1 -- 172.21.15.52:0/377911239 wait complete. 2024-06-02T23:13:18.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 15578 3.79 MB/sec warmup 27 sec latency 13.396 ms 2024-06-02T23:13:18.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 -- 172.21.15.52:0/248013898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8104ee0 msgr2=0x7f7fe81072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/248013898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81072d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc009860 tx=0x7f7fdc02a340 comp rx=0 tx=0).stop 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 -- 172.21.15.52:0/248013898 shutdown_connections 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/248013898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe8107810 0x7f7fe8109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/248013898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81072d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/248013898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe81025b0 0x7f7fe81049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 -- 172.21.15.52:0/248013898 >> 172.21.15.52:0/248013898 conn(0x7f7fe80fbf20 msgr2=0x7f7fe80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.824+0000 7f7fee8a5640 1 -- 172.21.15.52:0/248013898 shutdown_connections 2024-06-02T23:13:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 -- 172.21.15.52:0/248013898 wait complete. 2024-06-02T23:13:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 Processor -- start 2024-06-02T23:13:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 -- start start 2024-06-02T23:13:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe81025b0 0x7f7fe8197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8107810 0x7f7fe819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810d570 con 0x7f7fe8104ee0 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810d3f0 con 0x7f7fe8107810 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.825+0000 7f7fee8a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe810d6f0 con 0x7f7fe81025b0 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81980e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55490/0 (socket says 172.21.15.52:55490) 2024-06-02T23:13:18.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 -- 172.21.15.52:0/3856720578 learned_addr learned my addr 172.21.15.52:0/3856720578 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fece42640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8107810 0x7f7fe819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe81025b0 0x7f7fe8197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 -- 172.21.15.52:0/3856720578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe81025b0 msgr2=0x7f7fe8197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe81025b0 0x7f7fe8197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 -- 172.21.15.52:0/3856720578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8107810 msgr2=0x7f7fe819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8107810 0x7f7fe819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 -- 172.21.15.52:0/3856720578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe819f7c0 con 0x7f7fe8104ee0 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81980e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005940 tx=0x7f7fdc030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe57fa640 1 -- 172.21.15.52:0/3856720578 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc037050 con 0x7f7fe8104ee0 2024-06-02T23:13:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe819fa50 con 0x7f7fe8104ee0 2024-06-02T23:13:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe57fa640 1 -- 172.21.15.52:0/3856720578 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fdc043070 con 0x7f7fe8104ee0 2024-06-02T23:13:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fe57fa640 1 -- 172.21.15.52:0/3856720578 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc003d30 con 0x7f7fe8104ee0 2024-06-02T23:13:18.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.826+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe819ff90 con 0x7f7fe8104ee0 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7fe57fa640 1 -- 172.21.15.52:0/3856720578 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fdc004040 con 0x7f7fe8104ee0 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7ff15b5740 1 Processor -- start 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7ff15b5740 1 -- start start 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7fe57fa640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c30 0x7f7fd00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b68891430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b68896500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.829+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b68899000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562b687f44f0 con 0x562b68891010 2024-06-02T23:13:18.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b68792870 con 0x562b68891a00 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b687f04b0 con 0x562b68896b70 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fece42640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c30 0x7f7fd00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b68891430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe57fa640 1 -- 172.21.15.52:0/3856720578 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fdc0ef4e0 con 0x7f7fe8104ee0 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b68896500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b68896500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36444/0 (socket says 172.21.15.52:36444) 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 -- 172.21.15.52:0/2710047837 learned_addr learned my addr 172.21.15.52:0/2710047837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fece42640 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b68899000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 -- 172.21.15.52:0/2710047837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 msgr2=0x562b68899000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b68899000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 -- 172.21.15.52:0/2710047837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 msgr2=0x562b68891430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b68891430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 -- 172.21.15.52:0/2710047837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b687ae9b0 con 0x562b68891a00 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fece42640 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b68899000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fece42640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c30 0x7f7fd00a50f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0005fd0 tx=0x7f7fe0009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.830+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b68896500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0304b0 tx=0x7f7fdc005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d46bfa40a84a0ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7fce7fc640 1 -- 172.21.15.52:0/2710047837 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0429e0 con 0x562b68891a00 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7fce7fc640 1 -- 172.21.15.52:0/2710047837 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fdc038040 con 0x562b68891a00 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7fce7fc640 1 -- 172.21.15.52:0/2710047837 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fdc0c49e0 con 0x562b68891a00 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 -- 172.21.15.52:0/2710047837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 msgr2=0x562b68896500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b68896500 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc0304b0 tx=0x7f7fdc005750 comp rx=0 tx=0).stop 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 -- 172.21.15.52:0/2710047837 shutdown_connections 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b68899000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b68896500 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/2710047837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b68891430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 -- 172.21.15.52:0/2710047837 >> 172.21.15.52:0/2710047837 conn(0x562b6888c680 msgr2=0x562b6888ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 -- 172.21.15.52:0/2710047837 shutdown_connections 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.831+0000 7f7ff15b5740 1 -- 172.21.15.52:0/2710047837 wait complete. 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 -- start start 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 Processor -- start 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891010 0x562b6889f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 0x562b6889f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562b6889dcf0 con 0x562b68891a00 2024-06-02T23:13:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b6877dae0 con 0x562b68891010 2024-06-02T23:13:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b6889de70 con 0x562b68896b70 2024-06-02T23:13:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7ff15b5740 10 client.0 _ll_register_callbacks cb 0x562b687b69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.832+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 0x562b6889f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 0x562b6889f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55512/0 (socket says 172.21.15.52:55512) 2024-06-02T23:13:18.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe77fe640 1 -- 172.21.15.52:0/1564826979 learned_addr learned my addr 172.21.15.52:0/1564826979 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891010 0x562b6889f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fece42640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1564826979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 msgr2=0x562b688a6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1564826979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 msgr2=0x562b6889f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 0x562b6889f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1564826979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b68508210 con 0x562b68891010 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fece42640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 0x562b6889f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891010 0x562b6889f3e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x562b68892260 tx=0x7f7fd8012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23c747d951628b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:18.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1564826979 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8004020 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1564826979 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fd80041c0 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fdc038040 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1564826979 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fd8023070 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7ff15b5740 10 client.25658 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.833+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b685d4620 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1564826979 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fdc038040 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1564826979 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7fd8003970 con 0x562b68891010 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7ff15b5740 10 client.25658 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7ff15b5740 10 client.25658 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7ff15b5740 20 client.25658 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b688ab2d0 0x562b688ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7ff15b5740 4 client.25658 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b68807f10 con 0x562b688ab2d0 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b688ab2d0 0x562b688ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.834+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b688ab2d0 0x562b688ad730 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2318b814cb585ab9 server_cookie=3bbc680b2db4b666 in_seq=0 out_seq=0 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7fc77fe640 10 client.25658 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1564826979 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b68807f10 con 0x562b688ab2d0 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7fc77fe640 10 client.25658 handle_command_reply: tid=0 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 client.25658 shutdown 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891010 msgr2=0x562b6889f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891010 0x562b6889f3e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x562b68892260 tx=0x7f7fd8012b60 comp rx=0 tx=0).stop 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 shutdown_connections 2024-06-02T23:13:18.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b688ab2d0 0x562b688ad730 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891a00 0x562b6889f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7fc77fe640 0 client.25658 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.835+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1564826979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891010 0x562b6889f3e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.836+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 >> 172.21.15.52:0/1564826979 conn(0x562b6888c680 msgr2=0x562b6888b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.836+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 shutdown_connections 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.836+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1564826979 wait complete. 2024-06-02T23:13:18.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.836+0000 7f7ff15b5740 20 client.25658 trim_cache size 0 max 16384 2024-06-02T23:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.854+0000 7f7ff15b5740 1 Processor -- start 2024-06-02T23:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.854+0000 7f7ff15b5740 1 -- start start 2024-06-02T23:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.854+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b6888c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.854+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68891a00 0x562b688a6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.854+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68896b70 0x562b6888af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81a69e0 con 0x562b68891010 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81a85b0 con 0x562b68896b70 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81a70b0 con 0x562b68891a00 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b6888c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b6888c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55520/0 (socket says 172.21.15.52:55520) 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1913773380 learned_addr learned my addr 172.21.15.52:0/1913773380 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fece42640 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68896b70 0x562b6888af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68891a00 0x562b688a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1913773380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68891a00 msgr2=0x562b688a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68891a00 0x562b688a6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1913773380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68896b70 msgr2=0x562b6888af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68896b70 0x562b6888af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1913773380 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b688926a0 con 0x562b68891010 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fece42640 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68896b70 0x562b6888af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b6888c120 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80058c0 tx=0x7f7fd8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bddfa68dd4e39f00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fcdffb640 1 -- 172.21.15.52:0/1913773380 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8004460 con 0x562b68891010 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fcdffb640 1 -- 172.21.15.52:0/1913773380 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fd800fda0 con 0x562b68891010 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.855+0000 7f7fcdffb640 1 -- 172.21.15.52:0/1913773380 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8005030 con 0x562b68891010 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1913773380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 msgr2=0x562b6888c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b6888c120 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80058c0 tx=0x7f7fd8005850 comp rx=0 tx=0).stop 2024-06-02T23:13:18.857 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1913773380 shutdown_connections 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68891a00 0x562b688a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68896b70 0x562b6888af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1913773380 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b6888c120 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1913773380 >> 172.21.15.52:0/1913773380 conn(0x562b6888c680 msgr2=0x562b6889e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1913773380 shutdown_connections 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1913773380 wait complete. 2024-06-02T23:13:18.858 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.856+0000 7f7ff15b5740 1 -- start start 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 Processor -- start 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b688a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b688a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81a7e50 con 0x562b68891010 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81a7780 con 0x562b68891a00 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe81a49e0 con 0x562b68896b70 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7ff15b5740 10 client.0 _ll_register_callbacks cb 0x7f7fd8009870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b688a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b688a6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55526/0 (socket says 172.21.15.52:55526) 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1819186098 learned_addr learned my addr 172.21.15.52:0/1819186098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b688a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.861 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fece42640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1819186098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 msgr2=0x562b688a35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 msgr2=0x562b688a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b688a30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe7fff640 1 -- 172.21.15.52:0/1819186098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b688a3c60 con 0x562b68891010 2024-06-02T23:13:18.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.857+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b688a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fe7fff640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b688a6610 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009fd0 tx=0x7f7fd8005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93918d81237e4d33 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fc77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1819186098 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8004000 con 0x562b68891010 2024-06-02T23:13:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1819186098 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fd800f8f0 con 0x562b68891010 2024-06-02T23:13:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd800fda0 con 0x562b68891010 2024-06-02T23:13:18.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1819186098 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7fd8004460 con 0x562b68891010 2024-06-02T23:13:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 10 client.16083 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x562b68915110 con 0x562b68891010 2024-06-02T23:13:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1819186098 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fd800fda0 con 0x562b68891010 2024-06-02T23:13:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1819186098 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7fd80048c0 con 0x562b68891010 2024-06-02T23:13:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 10 client.16083 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:18.869 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 10 client.16083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 20 client.16083 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b68915a30 0x562b68917e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 4 client.16083 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:13:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.858+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562b68801c20 con 0x562b68915a30 2024-06-02T23:13:18.870 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.859+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b68915a30 0x562b68917e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.859+0000 7f7fc77fe640 10 client.16083 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.859+0000 7f7fe77fe640 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b68915a30 0x562b68917e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a426935330936f7 server_cookie=a04590deed406d67 in_seq=0 out_seq=0 2024-06-02T23:13:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.878+0000 7f7fc77fe640 1 -- 172.21.15.52:0/1819186098 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562b68801c20 con 0x562b68915a30 2024-06-02T23:13:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.878+0000 7f7fc77fe640 10 client.16083 handle_command_reply: tid=0 2024-06-02T23:13:18.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.878+0000 7f7ff15b5740 1 client.16083 shutdown 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 msgr2=0x562b688a6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b688a6610 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009fd0 tx=0x7f7fd8005970 comp rx=0 tx=0).stop 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 shutdown_connections 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b68915a30 0x562b68917e50 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b68896b70 0x562b688a35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7fc77fe640 0 client.16083 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b68891a00 0x562b688a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 --2- 172.21.15.52:0/1819186098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b68891010 0x562b688a6610 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 >> 172.21.15.52:0/1819186098 conn(0x562b6888c680 msgr2=0x562b6888f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 shutdown_connections 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 1 -- 172.21.15.52:0/1819186098 wait complete. 2024-06-02T23:13:18.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7ff15b5740 20 client.16083 trim_cache size 0 max 16384 2024-06-02T23:13:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:13:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:13:18.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c30 msgr2=0x7f7fd00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c30 0x7f7fd00a50f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0005fd0 tx=0x7f7fe0009f90 comp rx=0 tx=0).stop 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 msgr2=0x7f7fe81980e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.879+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81980e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc005940 tx=0x7f7fdc030e00 comp rx=0 tx=0).stop 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 shutdown_connections 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c30 0x7f7fd00a50f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe8107810 0x7f7fe819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe8104ee0 0x7f7fe81980e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 --2- 172.21.15.52:0/3856720578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe81025b0 0x7f7fe8197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 >> 172.21.15.52:0/3856720578 conn(0x7f7fe80fbf20 msgr2=0x7f7fe8102f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 shutdown_connections 2024-06-02T23:13:18.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:18.880+0000 7f7fee8a5640 1 -- 172.21.15.52:0/3856720578 wait complete. 2024-06-02T23:13:19.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:18 smithi123 ceph-mon[35977]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-02T23:13:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:18 smithi174 ceph-mon[35994]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-02T23:13:19.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:18 smithi052 ceph-mon[32340]: pgmap v474: 97 pgs: 97 active+clean; 29 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 276 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-02T23:13:19.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 16163 3.67 MB/sec warmup 28 sec latency 42.262 ms 2024-06-02T23:13:20.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 16666 3.62 MB/sec warmup 29 sec latency 82.385 ms 2024-06-02T23:13:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:20 smithi052 ceph-mon[32340]: pgmap v475: 97 pgs: 97 active+clean; 30 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-06-02T23:13:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:20 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:13:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:20 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:13:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:20 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:13:21.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:20 smithi123 ceph-mon[35977]: pgmap v475: 97 pgs: 97 active+clean; 30 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-06-02T23:13:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:20 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:13:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:20 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:13:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:20 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:13:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:20 smithi174 ceph-mon[35994]: pgmap v475: 97 pgs: 97 active+clean; 30 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-06-02T23:13:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:20 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:13:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:20 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:13:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:20 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:13:21.555 INFO:tasks.cephfs.filesystem:scrub status for tag:6410c6f0-c37c-46ea-8b6a-8304ad909d7b - {'path': '/', 'tag': '6410c6f0-c37c-46ea-8b6a-8304ad909d7b', 'options': 'recursive,force'} 2024-06-02T23:13:21.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 17473 3.72 MB/sec warmup 30 sec latency 15.959 ms 2024-06-02T23:13:22.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:21 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:13:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:21 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:13:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:21 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:13:22.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 17879 3.71 MB/sec warmup 31 sec latency 14.820 ms 2024-06-02T23:13:23.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:22 smithi123 ceph-mon[35977]: pgmap v476: 97 pgs: 97 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-02T23:13:23.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:22 smithi174 ceph-mon[35994]: pgmap v476: 97 pgs: 97 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-02T23:13:23.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:22 smithi052 ceph-mon[32340]: pgmap v476: 97 pgs: 97 active+clean; 30 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-02T23:13:23.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 18174 3.64 MB/sec warmup 32 sec latency 12.512 ms 2024-06-02T23:13:24.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 18577 3.55 MB/sec warmup 33 sec latency 14.542 ms 2024-06-02T23:13:25.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:24 smithi123 ceph-mon[35977]: pgmap v477: 97 pgs: 97 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-06-02T23:13:25.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:24 smithi174 ceph-mon[35994]: pgmap v477: 97 pgs: 97 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-06-02T23:13:25.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:24 smithi052 ceph-mon[32340]: pgmap v477: 97 pgs: 97 active+clean; 25 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.1 MiB/s wr, 159 op/s 2024-06-02T23:13:25.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 19127 3.57 MB/sec warmup 34 sec latency 107.455 ms 2024-06-02T23:13:26.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 19721 3.48 MB/sec warmup 35 sec latency 15.016 ms 2024-06-02T23:13:27.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:27 smithi123 ceph-mon[35977]: pgmap v478: 97 pgs: 97 active+clean; 29 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-06-02T23:13:27.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:27 smithi174 ceph-mon[35994]: pgmap v478: 97 pgs: 97 active+clean; 29 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-06-02T23:13:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:26 smithi052 ceph-mon[32340]: pgmap v478: 97 pgs: 97 active+clean; 29 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.5 MiB/s wr, 196 op/s 2024-06-02T23:13:27.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 20289 3.45 MB/sec warmup 36 sec latency 15.468 ms 2024-06-02T23:13:28.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 21073 3.54 MB/sec warmup 37 sec latency 10.374 ms 2024-06-02T23:13:29.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:29 smithi123 ceph-mon[35977]: pgmap v479: 97 pgs: 97 active+clean; 28 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-02T23:13:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:29 smithi174 ceph-mon[35994]: pgmap v479: 97 pgs: 97 active+clean; 28 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-02T23:13:29.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:29 smithi052 ceph-mon[32340]: pgmap v479: 97 pgs: 97 active+clean; 28 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-02T23:13:29.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 21542 3.56 MB/sec warmup 38 sec latency 14.285 ms 2024-06-02T23:13:30.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 21792 3.48 MB/sec warmup 39 sec latency 43.196 ms 2024-06-02T23:13:31.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:31 smithi123 ceph-mon[35977]: pgmap v480: 97 pgs: 97 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-02T23:13:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:31 smithi174 ceph-mon[35994]: pgmap v480: 97 pgs: 97 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-02T23:13:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:31 smithi052 ceph-mon[32340]: pgmap v480: 97 pgs: 97 active+clean; 28 MiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-02T23:13:31.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 22205 3.41 MB/sec warmup 40 sec latency 15.079 ms 2024-06-02T23:13:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 -- 172.21.15.52:0/2745522046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518101b10 msgr2=0x7f1518101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 --2- 172.21.15.52:0/2745522046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518101b10 0x7f1518101f90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f150c009900 tx=0x7f150c02a410 comp rx=0 tx=0).stop 2024-06-02T23:13:31.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 -- 172.21.15.52:0/2745522046 shutdown_connections 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 --2- 172.21.15.52:0/2745522046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15181024d0 0x7f1518108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 --2- 172.21.15.52:0/2745522046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518101b10 0x7f1518101f90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 --2- 172.21.15.52:0/2745522046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1518100920 0x7f1518100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 -- 172.21.15.52:0/2745522046 >> 172.21.15.52:0/2745522046 conn(0x7f15180fbe90 msgr2=0x7f15180fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 -- 172.21.15.52:0/2745522046 shutdown_connections 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.979+0000 7f151d2ae640 1 -- 172.21.15.52:0/2745522046 wait complete. 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 Processor -- start 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 -- start start 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 0x7f151806ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1518101b10 0x7f151819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15181024d0 0x7f151819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f151810c450 con 0x7f1518100920 2024-06-02T23:13:31.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f151810c2d0 con 0x7f15181024d0 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f151d2ae640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f151810c5d0 con 0x7f1518101b10 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 0x7f151806ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 0x7f151806ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54374/0 (socket says 172.21.15.52:54374) 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f1516ffd640 1 -- 172.21.15.52:0/1299965523 learned_addr learned my addr 172.21.15.52:0/1299965523 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f15167fc640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1518101b10 0x7f151819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f15177fe640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15181024d0 0x7f151819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.980+0000 7f1516ffd640 1 -- 172.21.15.52:0/1299965523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1518101b10 msgr2=0x7f151819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f1516ffd640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1518101b10 0x7f151819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f1516ffd640 1 -- 172.21.15.52:0/1299965523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15181024d0 msgr2=0x7f151819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f1516ffd640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15181024d0 0x7f151819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f1516ffd640 1 -- 172.21.15.52:0/1299965523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f151819f820 con 0x7f1518100920 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f1516ffd640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 0x7f151806ace0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f150800e9c0 tx=0x7f150800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f14fffff640 1 -- 172.21.15.52:0/1299965523 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150800cd80 con 0x7f1518100920 2024-06-02T23:13:31.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f151819fab0 con 0x7f1518100920 2024-06-02T23:13:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f14fffff640 1 -- 172.21.15.52:0/1299965523 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f15080035f0 con 0x7f1518100920 2024-06-02T23:13:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f14fffff640 1 -- 172.21.15.52:0/1299965523 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1508010690 con 0x7f1518100920 2024-06-02T23:13:31.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.981+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f151819ff80 con 0x7f1518100920 2024-06-02T23:13:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f14fffff640 1 -- 172.21.15.52:0/1299965523 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f15080037a0 con 0x7f1518100920 2024-06-02T23:13:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 Processor -- start 2024-06-02T23:13:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 -- start start 2024-06-02T23:13:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e09430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09a00 0x55b183e0e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e11000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b183d6c4f0 con 0x55b183e0eb70 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b183d0a870 con 0x55b183e09a00 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f151ffb6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b183d684b0 con 0x55b183e09010 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f15167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e11000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f15167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e11000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54382/0 (socket says 172.21.15.52:54382) 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.984+0000 7f15167fc640 1 -- 172.21.15.52:0/80685798 learned_addr learned my addr 172.21.15.52:0/80685798 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f14fffff640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15000a2c30 0x7f15000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f1516ffd640 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09a00 0x55b183e0e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15177fe640 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e09430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15177fe640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15000a2c30 0x7f15000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15167fc640 1 -- 172.21.15.52:0/80685798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 msgr2=0x55b183e09430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15167fc640 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e09430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15167fc640 1 -- 172.21.15.52:0/80685798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09a00 msgr2=0x55b183e0e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15167fc640 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09a00 0x55b183e0e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15167fc640 1 -- 172.21.15.52:0/80685798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b183d269b0 con 0x55b183e0eb70 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f14fffff640 1 -- 172.21.15.52:0/1299965523 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f15080cf540 con 0x7f1518100920 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15177fe640 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e09430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15167fc640 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e11000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f150c009900 tx=0x7f150c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d41d59ae5563a202 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f15177fe640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15000a2c30 0x7f15000a50f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55b183e0a260 tx=0x7f1510009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f14fd7fa640 1 -- 172.21.15.52:0/80685798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c035030 con 0x55b183e0eb70 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f14fd7fa640 1 -- 172.21.15.52:0/80685798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f150c040070 con 0x55b183e0eb70 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.985+0000 7f14fd7fa640 1 -- 172.21.15.52:0/80685798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c003a10 con 0x55b183e0eb70 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 -- 172.21.15.52:0/80685798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 msgr2=0x55b183e11000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:31.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e11000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f150c009900 tx=0x7f150c005c00 comp rx=0 tx=0).stop 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 -- 172.21.15.52:0/80685798 shutdown_connections 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e11000 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09a00 0x55b183e0e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 --2- 172.21.15.52:0/80685798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e09430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 -- 172.21.15.52:0/80685798 >> 172.21.15.52:0/80685798 conn(0x55b183e04680 msgr2=0x55b183e04a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 -- 172.21.15.52:0/80685798 shutdown_connections 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.986+0000 7f151ffb6740 1 -- 172.21.15.52:0/80685798 wait complete. 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 -- start start 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 Processor -- start 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e13250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09a00 0x55b183e13790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e1a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b183e11ac0 con 0x55b183e0eb70 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b183cf5ae0 con 0x55b183e09010 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b183e11c40 con 0x55b183e09a00 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f151ffb6740 10 client.0 _ll_register_callbacks cb 0x55b183d2e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e13250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f15177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e1a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e13250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34312/0 (socket says 172.21.15.52:34312) 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 -- 172.21.15.52:0/1848385220 learned_addr learned my addr 172.21.15.52:0/1848385220 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f15167fc640 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09a00 0x55b183e13790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 -- 172.21.15.52:0/1848385220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09a00 msgr2=0x55b183e13790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09a00 0x55b183e13790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 -- 172.21.15.52:0/1848385220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 msgr2=0x55b183e1a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:31.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e1a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.987+0000 7f1516ffd640 1 -- 172.21.15.52:0/1848385220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b183a80210 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f1516ffd640 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e13250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1508009fd0 tx=0x7f150809b730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2249feef46d8a48e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f14faff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f14faff5640 1 -- 172.21.15.52:0/1848385220 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150809c230 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f14faff5640 1 -- 172.21.15.52:0/1848385220 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f15080a45b0 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c040070 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f14faff5640 1 -- 172.21.15.52:0/1848385220 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f150809bdd0 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 10 client.25676 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b183b4c620 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f14faff5640 1 -- 172.21.15.52:0/1848385220 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f150c040070 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f14faff5640 1 -- 172.21.15.52:0/1848385220 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f15080b1040 con 0x55b183e09010 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 10 client.25676 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 10 client.25676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:31.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 20 client.25676 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e1f0a0 0x55b183e21530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 4 client.25676 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.988+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b183d7ff10 con 0x55b183e1f0a0 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.989+0000 7f15167fc640 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e1f0a0 0x55b183e21530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.989+0000 7f15167fc640 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e1f0a0 0x55b183e21530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d714ef73e2f5d6 server_cookie=58a331e2a1b4f623 in_seq=0 out_seq=0 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.989+0000 7f14faff5640 10 client.25676 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f14faff5640 1 -- 172.21.15.52:0/1848385220 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b183d7ff10 con 0x55b183e1f0a0 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f14faff5640 10 client.25676 handle_command_reply: tid=0 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 client.25676 shutdown 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 msgr2=0x55b183e13250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e13250 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1508009fd0 tx=0x7f150809b730 comp rx=0 tx=0).stop 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 shutdown_connections 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e1f0a0 0x55b183e21530 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09a00 0x55b183e13790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f14faff5640 0 client.25676 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e0eb70 0x55b183e1a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 --2- 172.21.15.52:0/1848385220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e13250 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 >> 172.21.15.52:0/1848385220 conn(0x55b183e04680 msgr2=0x55b183e037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 shutdown_connections 2024-06-02T23:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 1 -- 172.21.15.52:0/1848385220 wait complete. 2024-06-02T23:13:31.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:31.990+0000 7f151ffb6740 20 client.25676 trim_cache size 0 max 16384 2024-06-02T23:13:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 Processor -- start 2024-06-02T23:13:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 -- start start 2024-06-02T23:13:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e04a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e1a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e0eb70 0x55b183e07590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15181a8d00 con 0x55b183e09a00 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15181aa8d0 con 0x55b183e09010 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f151ffb6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15181a93d0 con 0x55b183e0eb70 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f15167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e1a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e04a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f15167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e1a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54394/0 (socket says 172.21.15.52:54394) 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.008+0000 7f15167fc640 1 -- 172.21.15.52:0/2930365021 learned_addr learned my addr 172.21.15.52:0/2930365021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15177fe640 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e0eb70 0x55b183e07590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15167fc640 1 -- 172.21.15.52:0/2930365021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e0eb70 msgr2=0x55b183e07590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15167fc640 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e0eb70 0x55b183e07590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15167fc640 1 -- 172.21.15.52:0/2930365021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 msgr2=0x55b183e04a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15167fc640 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e04a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15167fc640 1 -- 172.21.15.52:0/2930365021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b183e044b0 con 0x55b183e09a00 2024-06-02T23:13:32.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f1516ffd640 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e04a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f15167fc640 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e1a720 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55b183e05550 tx=0x7f150c031710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab50ee2bef79563 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f14fcff9640 1 -- 172.21.15.52:0/2930365021 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c035040 con 0x55b183e09a00 2024-06-02T23:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f14fcff9640 1 -- 172.21.15.52:0/2930365021 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f150c040040 con 0x55b183e09a00 2024-06-02T23:13:32.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.009+0000 7f14fcff9640 1 -- 172.21.15.52:0/2930365021 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c044700 con 0x55b183e09a00 2024-06-02T23:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 -- 172.21.15.52:0/2930365021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 msgr2=0x55b183e1a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e1a720 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x55b183e05550 tx=0x7f150c031710 comp rx=0 tx=0).stop 2024-06-02T23:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 -- 172.21.15.52:0/2930365021 shutdown_connections 2024-06-02T23:13:32.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e0eb70 0x55b183e07590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e1a720 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 --2- 172.21.15.52:0/2930365021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e09010 0x55b183e04a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 -- 172.21.15.52:0/2930365021 >> 172.21.15.52:0/2930365021 conn(0x55b183e04680 msgr2=0x55b183e0e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 -- 172.21.15.52:0/2930365021 shutdown_connections 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 -- 172.21.15.52:0/2930365021 wait complete. 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.010+0000 7f151ffb6740 1 -- start start 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 Processor -- start 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e174f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e17a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e0eb70 0x55b183e87db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f15181aa170 con 0x55b183e09a00 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15181a9aa0 con 0x55b183e0eb70 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15181a6ce0 con 0x55b183e09010 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f151ffb6740 10 client.0 _ll_register_callbacks cb 0x7f1508003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:32.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e17a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15177fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e0eb70 0x55b183e87db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f1516ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e174f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e17a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54400/0 (socket says 172.21.15.52:54400) 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 -- 172.21.15.52:0/3985546438 learned_addr learned my addr 172.21.15.52:0/3985546438 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 -- 172.21.15.52:0/3985546438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 msgr2=0x55b183e174f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e174f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 -- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e0eb70 msgr2=0x55b183e87db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e0eb70 0x55b183e87db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15167fc640 1 -- 172.21.15.52:0/3985546438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b183e180d0 con 0x55b183e09a00 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.011+0000 7f15177fe640 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e0eb70 0x55b183e87db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f15167fc640 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e17a30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f150c031b70 tx=0x7f150c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3f43e0fe05e9458 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f14faff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f14faff5640 1 -- 172.21.15.52:0/3985546438 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c035040 con 0x55b183e09a00 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f150c040040 con 0x55b183e09a00 2024-06-02T23:13:32.016 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f14faff5640 1 -- 172.21.15.52:0/3985546438 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f150c03c640 con 0x55b183e09a00 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f14faff5640 1 -- 172.21.15.52:0/3985546438 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f150c03c880 con 0x55b183e09a00 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 10 client.16101 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b183e89430 con 0x55b183e09a00 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f14faff5640 1 -- 172.21.15.52:0/3985546438 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f150c040040 con 0x55b183e09a00 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f14faff5640 1 -- 172.21.15.52:0/3985546438 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f150c044480 con 0x55b183e09a00 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 10 client.16101 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 20 client.16101 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e89a30 0x55b183e8be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 4 client.16101 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.012+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b183d79c20 con 0x55b183e89a30 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.013+0000 7f1516ffd640 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e89a30 0x55b183e8be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:32.017 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.013+0000 7f14faff5640 10 client.16101 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:32.018 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.013+0000 7f1516ffd640 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e89a30 0x55b183e8be20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8446a53862f9bd69 server_cookie=3b98295697243894 in_seq=0 out_seq=0 2024-06-02T23:13:32.032 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f14faff5640 1 -- 172.21.15.52:0/3985546438 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b183d79c20 con 0x55b183e89a30 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f14faff5640 10 client.16101 handle_command_reply: tid=0 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 client.16101 shutdown 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 msgr2=0x55b183e17a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e17a30 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f150c031b70 tx=0x7f150c009680 comp rx=0 tx=0).stop 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 shutdown_connections 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b183e89a30 0x55b183e8be20 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b183e0eb70 0x55b183e87db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f14faff5640 0 client.16101 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b183e09a00 0x55b183e17a30 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 --2- 172.21.15.52:0/3985546438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b183e09010 0x55b183e174f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 >> 172.21.15.52:0/3985546438 conn(0x55b183e04680 msgr2=0x55b183e0ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:32.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 shutdown_connections 2024-06-02T23:13:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 1 -- 172.21.15.52:0/3985546438 wait complete. 2024-06-02T23:13:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.032+0000 7f151ffb6740 20 client.16101 trim_cache size 0 max 16384 2024-06-02T23:13:32.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:13:32.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15000a2c30 msgr2=0x7f15000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15000a2c30 0x7f15000a50f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x55b183e0a260 tx=0x7f1510009500 comp rx=0 tx=0).stop 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 msgr2=0x7f151806ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 0x7f151806ace0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f150800e9c0 tx=0x7f150800ee80 comp rx=0 tx=0).stop 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 shutdown_connections 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15000a2c30 0x7f15000a50f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1518101b10 0x7f151819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15181024d0 0x7f151819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 --2- 172.21.15.52:0/1299965523 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1518100920 0x7f151806ace0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 >> 172.21.15.52:0/1299965523 conn(0x7f15180fbe90 msgr2=0x7f1518104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 shutdown_connections 2024-06-02T23:13:32.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:32.033+0000 7f151d2ae640 1 -- 172.21.15.52:0/1299965523 wait complete. 2024-06-02T23:13:32.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 22777 3.43 MB/sec warmup 41 sec latency 15.823 ms 2024-06-02T23:13:33.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:33 smithi123 ceph-mon[35977]: pgmap v481: 97 pgs: 97 active+clean; 29 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-06-02T23:13:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:33 smithi174 ceph-mon[35994]: pgmap v481: 97 pgs: 97 active+clean; 29 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-06-02T23:13:33.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:33 smithi052 ceph-mon[32340]: pgmap v481: 97 pgs: 97 active+clean; 29 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.4 MiB/s wr, 148 op/s 2024-06-02T23:13:33.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 23390 3.36 MB/sec warmup 42 sec latency 15.041 ms 2024-06-02T23:13:34.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 23907 3.34 MB/sec warmup 43 sec latency 14.615 ms 2024-06-02T23:13:35.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:35 smithi123 ceph-mon[35977]: pgmap v482: 97 pgs: 97 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-06-02T23:13:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:35 smithi174 ceph-mon[35994]: pgmap v482: 97 pgs: 97 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-06-02T23:13:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:35 smithi052 ceph-mon[32340]: pgmap v482: 97 pgs: 97 active+clean; 29 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 128 op/s 2024-06-02T23:13:35.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 24632 3.41 MB/sec warmup 44 sec latency 14.806 ms 2024-06-02T23:13:36.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 25100 3.43 MB/sec warmup 45 sec latency 14.692 ms 2024-06-02T23:13:37.118 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-06-02T23:13:37.119 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:13:37.387 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:37 smithi174 ceph-mon[35994]: pgmap v483: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-06-02T23:13:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:37 smithi052 ceph-mon[32340]: pgmap v483: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-06-02T23:13:37.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:37 smithi123 ceph-mon[35977]: pgmap v483: 97 pgs: 97 active+clean; 30 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 332 KiB/s rd, 1.6 MiB/s wr, 189 op/s 2024-06-02T23:13:37.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 25347 3.36 MB/sec warmup 46 sec latency 13.104 ms 2024-06-02T23:13:38.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 25782 3.31 MB/sec warmup 47 sec latency 24.703 ms 2024-06-02T23:13:39.349 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:39 smithi052 ceph-mon[32340]: pgmap v484: 97 pgs: 97 active+clean; 30 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-06-02T23:13:39.349 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:39 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:13:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:39 smithi174 ceph-mon[35994]: pgmap v484: 97 pgs: 97 active+clean; 30 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-06-02T23:13:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:39 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:13:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:39 smithi123 ceph-mon[35977]: pgmap v484: 97 pgs: 97 active+clean; 30 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-06-02T23:13:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:39 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:13:39.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 26277 3.33 MB/sec warmup 48 sec latency 54.852 ms 2024-06-02T23:13:40.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 26779 3.27 MB/sec warmup 49 sec latency 107.114 ms 2024-06-02T23:13:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:41 smithi052 ceph-mon[32340]: pgmap v485: 97 pgs: 97 active+clean; 31 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:13:41.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:41 smithi174 ceph-mon[35994]: pgmap v485: 97 pgs: 97 active+clean; 31 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:13:41.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:41 smithi123 ceph-mon[35977]: pgmap v485: 97 pgs: 97 active+clean; 31 MiB data, 452 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:13:41.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 27346 3.25 MB/sec warmup 50 sec latency 15.229 ms 2024-06-02T23:13:41.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.958+0000 7f96ef687700 1 -- 172.21.15.52:0/3090396631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8108520 msgr2=0x7f96e8108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:41.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.958+0000 7f96ef687700 1 --2- 172.21.15.52:0/3090396631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8108520 0x7f96e8108900 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f96e4009990 tx=0x7f96e4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:41.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 -- 172.21.15.52:0/3090396631 shutdown_connections 2024-06-02T23:13:41.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 --2- 172.21.15.52:0/3090396631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e806b520 0x7f96e810f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:41.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 --2- 172.21.15.52:0/3090396631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e8108520 0x7f96e8108900 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:41.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 --2- 172.21.15.52:0/3090396631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e806ab60 0x7f96e806afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:41.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 -- 172.21.15.52:0/3090396631 >> 172.21.15.52:0/3090396631 conn(0x7f96e8078340 msgr2=0x7f96e8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:41.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 -- 172.21.15.52:0/3090396631 shutdown_connections 2024-06-02T23:13:41.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 -- 172.21.15.52:0/3090396631 wait complete. 2024-06-02T23:13:41.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 Processor -- start 2024-06-02T23:13:41.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.959+0000 7f96ef687700 1 -- start start 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ef687700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 0x7f96e819e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ef687700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e806b520 0x7f96e819ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ef687700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e8108520 0x7f96e8198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ef687700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96e81126b0 con 0x7f96e806ab60 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ef687700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8112530 con 0x7f96e806b520 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ef687700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96e8112830 con 0x7f96e8108520 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 0x7f96e819e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 0x7f96e819e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50386/0 (socket says 172.21.15.52:50386) 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 -- 172.21.15.52:0/3466546184 learned_addr learned my addr 172.21.15.52:0/3466546184 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ecc22700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e806b520 0x7f96e819ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 -- 172.21.15.52:0/3466546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e8108520 msgr2=0x7f96e8198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:41.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96edc24700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e8108520 0x7f96e8198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e8108520 0x7f96e8198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 -- 172.21.15.52:0/3466546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e806b520 msgr2=0x7f96e819ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e806b520 0x7f96e819ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 -- 172.21.15.52:0/3466546184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e8199190 con 0x7f96e806ab60 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.960+0000 7f96ed423700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 0x7f96e819e820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f96e4000c00 tx=0x7f96e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.961+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4027070 con 0x7f96e806ab60 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.961+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96e4022470 con 0x7f96e806ab60 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.961+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e4017650 con 0x7f96e806ab60 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.961+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e8199420 con 0x7f96e806ab60 2024-06-02T23:13:41.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.961+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e81a56f0 con 0x7f96e806ab60 2024-06-02T23:13:41.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.962+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96e810aa00 con 0x7f96e806ab60 2024-06-02T23:13:41.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.963+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96e402a070 con 0x7f96e806ab60 2024-06-02T23:13:41.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.964+0000 7f96de7fc700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96d40a2ca0 0x7f96d40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:41.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.964+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96e40cfd70 con 0x7f96e806ab60 2024-06-02T23:13:41.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.964+0000 7f96ecc22700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96d40a2ca0 0x7f96d40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:41.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.964+0000 7f96ecc22700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96d40a2ca0 0x7f96d40a5160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f96e80789b0 tx=0x7f96d8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:41.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:41.969+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f96e4013070 con 0x7f96e806ab60 2024-06-02T23:13:42.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.262+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96e8199f10 con 0x7f96e806ab60 2024-06-02T23:13:42.262 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:13:42.262 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:13:42.263 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:13:42.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.262+0000 7f96de7fc700 1 -- 172.21.15.52:0/3466546184 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7f96e409cd90 con 0x7f96e806ab60 2024-06-02T23:13:42.263 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:13:42.263 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:13:42.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96d40a2ca0 msgr2=0x7f96d40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:42.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96d40a2ca0 0x7f96d40a5160 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f96e80789b0 tx=0x7f96d8009190 comp rx=0 tx=0).stop 2024-06-02T23:13:42.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 msgr2=0x7f96e819e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:42.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 0x7f96e819e820 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f96e4000c00 tx=0x7f96e40047b0 comp rx=0 tx=0).stop 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 shutdown_connections 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96d40a2ca0 0x7f96d40a5160 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e8108520 0x7f96e8198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e806ab60 0x7f96e819e820 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 --2- 172.21.15.52:0/3466546184 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e806b520 0x7f96e819ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 >> 172.21.15.52:0/3466546184 conn(0x7f96e8078340 msgr2=0x7f96e81004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 shutdown_connections 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:42.265+0000 7f96ef687700 1 -- 172.21.15.52:0/3466546184 wait complete. 2024-06-02T23:13:42.266 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:13:42.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:13:42.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:13:42.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:13:42.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:13:42.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:13:42.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:13:42.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28059 3.30 MB/sec warmup 51 sec latency 65.377 ms 2024-06-02T23:13:43.031 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-06-02T23:13:43.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-06-02T23:13:43.310 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:43 smithi174 ceph-mon[35994]: pgmap v486: 97 pgs: 97 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-02T23:13:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:43 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3466546184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:43 smithi052 ceph-mon[32340]: pgmap v486: 97 pgs: 97 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-02T23:13:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:43 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3466546184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:43.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:43 smithi123 ceph-mon[35977]: pgmap v486: 97 pgs: 97 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.3 MiB/s wr, 154 op/s 2024-06-02T23:13:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:43 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3466546184' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:43.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28488 3.32 MB/sec warmup 52 sec latency 14.705 ms 2024-06-02T23:13:44.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28810 3.28 MB/sec warmup 53 sec latency 11.973 ms 2024-06-02T23:13:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.133+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3235485830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a14100900 msgr2=0x7f6a14100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.133+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3235485830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a14100900 0x7f6a14100d20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009900 tx=0x7f6a1002a410 comp rx=0 tx=0).stop 2024-06-02T23:13:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3235485830 shutdown_connections 2024-06-02T23:13:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3235485830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a141024b0 0x7f6a14108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3235485830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14101af0 0x7f6a14101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3235485830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a14100900 0x7f6a14100d20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3235485830 >> 172.21.15.52:0/3235485830 conn(0x7f6a140fbe70 msgr2=0x7f6a140fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3235485830 shutdown_connections 2024-06-02T23:13:45.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3235485830 wait complete. 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 Processor -- start 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 -- start start 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.134+0000 7f6a1bf61640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 0x7f6a1410b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1bf61640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a14101af0 0x7f6a141098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1bf61640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a141024b0 0x7f6a14109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1bf61640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1410d550 con 0x7f6a141024b0 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1bf61640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1410d3d0 con 0x7f6a14100900 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1bf61640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a1410d6d0 con 0x7f6a14101af0 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 0x7f6a1410b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1a4fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a141024b0 0x7f6a14109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 0x7f6a1410b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60644/0 (socket says 172.21.15.52:60644) 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 -- 172.21.15.52:0/3748169109 learned_addr learned my addr 172.21.15.52:0/3748169109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a194fc640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a14101af0 0x7f6a141098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 -- 172.21.15.52:0/3748169109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a14101af0 msgr2=0x7f6a141098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a14101af0 0x7f6a141098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 -- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a141024b0 msgr2=0x7f6a14109e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a141024b0 0x7f6a14109e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 -- 172.21.15.52:0/3748169109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1410a690 con 0x7f6a14100900 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a141024b0 0x7f6a14109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 0x7f6a1410b280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a100314e0 tx=0x7f6a10031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a0affd640 1 -- 172.21.15.52:0/3748169109 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1003f030 con 0x7f6a14100900 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.135+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a141a1c80 con 0x7f6a14100900 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.136+0000 7f6a0affd640 1 -- 172.21.15.52:0/3748169109 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a100044b0 con 0x7f6a14100900 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.136+0000 7f6a0affd640 1 -- 172.21.15.52:0/3748169109 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10035070 con 0x7f6a14100900 2024-06-02T23:13:45.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.136+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a141a21d0 con 0x7f6a14100900 2024-06-02T23:13:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.138+0000 7f6a0affd640 1 -- 172.21.15.52:0/3748169109 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6a10042030 con 0x7f6a14100900 2024-06-02T23:13:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.138+0000 7f6a1ec6f740 1 Processor -- start 2024-06-02T23:13:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.138+0000 7f6a1ec6f740 1 -- start start 2024-06-02T23:13:45.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4aa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4aa9f0 0x55d05a4af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4afb60 0x55d05a4b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d05a40d4f0 con 0x55d05a4aa000 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d05a3ab870 con 0x55d05a4afb60 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d05a4094b0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4aa420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37476/0 (socket says 172.21.15.52:37476) 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 -- 172.21.15.52:0/2550226286 learned_addr learned my addr 172.21.15.52:0/2550226286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4afb60 0x55d05a4b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a0affd640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6a000a2c30 0x7f6a000a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4aa9f0 0x55d05a4af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6a000a2c30 0x7f6a000a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 -- 172.21.15.52:0/2550226286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4aa9f0 msgr2=0x55d05a4af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4aa9f0 0x55d05a4af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a0affd640 1 -- 172.21.15.52:0/3748169109 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6a100ef5e0 con 0x7f6a14100900 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 -- 172.21.15.52:0/2550226286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4afb60 msgr2=0x55d05a4b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4afb60 0x55d05a4b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a1a4fe640 1 -- 172.21.15.52:0/2550226286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d05a3c79b0 con 0x55d05a4aa000 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.139+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4aa9f0 0x55d05a4af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.140+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4aa420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c012e20 tx=0x7f6a0c0147e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e514a4cbc9766b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.140+0000 7f69fffff640 1 -- 172.21.15.52:0/2550226286 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c00e450 con 0x55d05a4aa000 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.140+0000 7f69fffff640 1 -- 172.21.15.52:0/2550226286 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a0c0128b0 con 0x55d05a4aa000 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.140+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6a000a2c30 0x7f6a000a50f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c023000 tx=0x7f6a0c021000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.140+0000 7f69fffff640 1 -- 172.21.15.52:0/2550226286 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c01f700 con 0x55d05a4aa000 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2550226286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 msgr2=0x55d05a4aa420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4aa420 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c012e20 tx=0x7f6a0c0147e0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2550226286 shutdown_connections 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4aa9f0 0x55d05a4af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4afb60 0x55d05a4b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2550226286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4aa420 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2550226286 >> 172.21.15.52:0/2550226286 conn(0x55d05a4a5670 msgr2=0x55d05a4a5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2550226286 shutdown_connections 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2550226286 wait complete. 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 -- start start 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.141+0000 7f6a1ec6f740 1 Processor -- start 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4b4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4bb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d05a4b2b10 con 0x55d05a4aa000 2024-06-02T23:13:45.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d05a396ae0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d05a4b2c90 con 0x55d05a4afb60 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 10 client.0 _ll_register_callbacks cb 0x55d05a3cf9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a19cfd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4b4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60656/0 (socket says 172.21.15.52:60656) 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a19cfd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4b4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37492/0 (socket says 172.21.15.52:37492) 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1a4fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4bb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 -- 172.21.15.52:0/2655271178 learned_addr learned my addr 172.21.15.52:0/2655271178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 -- 172.21.15.52:0/2655271178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 msgr2=0x55d05a4bb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4bb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 -- 172.21.15.52:0/2655271178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 msgr2=0x55d05a4b4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4b4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 -- 172.21.15.52:0/2655271178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d05a121210 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b47c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a04012700 tx=0x7f6a04012bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb21de430c89e938 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f69fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2655271178 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04004300 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2655271178 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6a040119a0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.142+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0c0128b0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2655271178 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a040044a0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 10 client.25700 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d05a1ed620 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2655271178 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a0c0128b0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2655271178 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6a0400f460 con 0x55d05a4aa9f0 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 10 client.25700 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:45.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 10 client.25700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 20 client.25700 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a4c0210 0x55d05a4c2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 4 client.25700 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d05a420f10 con 0x55d05a4c0210 2024-06-02T23:13:45.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.143+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a4c0210 0x55d05a4c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.144+0000 7f69fd7fa640 10 client.25700 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.144+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a4c0210 0x55d05a4c2670 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=241c48c7e96bc881 server_cookie=fab1ce9774211194 in_seq=0 out_seq=0 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.144+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2655271178 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d05a420f10 con 0x55d05a4c0210 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.144+0000 7f69fd7fa640 10 client.25700 handle_command_reply: tid=0 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.144+0000 7f6a1ec6f740 1 client.25700 shutdown 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.144+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 msgr2=0x55d05a4b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b47c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f6a04012700 tx=0x7f6a04012bc0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 shutdown_connections 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a4c0210 0x55d05a4c2670 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4bb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b47c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f69fd7fa640 0 client.25700 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2655271178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa000 0x55d05a4b4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 >> 172.21.15.52:0/2655271178 conn(0x55d05a4a5670 msgr2=0x55d05a4a4830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 shutdown_connections 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2655271178 wait complete. 2024-06-02T23:13:45.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.145+0000 7f6a1ec6f740 20 client.25700 trim_cache size 0 max 16384 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 Processor -- start 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 -- start start 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a141aaeb0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a141aca80 con 0x55d05a4aa000 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a141ab580 con 0x55d05a4afb60 2024-06-02T23:13:45.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4bb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37518/0 (socket says 172.21.15.52:37518) 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 -- 172.21.15.52:0/1885899249 learned_addr learned my addr 172.21.15.52:0/1885899249 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 -- 172.21.15.52:0/1885899249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 msgr2=0x55d05a4a8350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4a8350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 -- 172.21.15.52:0/1885899249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 msgr2=0x55d05a4b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.161+0000 7f6a194fc640 1 -- 172.21.15.52:0/1885899249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d05a4a54a0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.162+0000 7f6a194fc640 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4bb7c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6a040092e0 tx=0x7f6a0400ff50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca6c282ddc01191 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.162+0000 7f69ff7fe640 1 -- 172.21.15.52:0/1885899249 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400fb40 con 0x55d05a4aa9f0 2024-06-02T23:13:45.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.162+0000 7f69ff7fe640 1 -- 172.21.15.52:0/1885899249 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a0400fce0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.162+0000 7f69ff7fe640 1 -- 172.21.15.52:0/1885899249 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04005040 con 0x55d05a4aa9f0 2024-06-02T23:13:45.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 -- 172.21.15.52:0/3368578311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560100a80 msgr2=0x7ff560100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 --2- 172.21.15.52:0/3368578311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560100a80 0x7ff560100f00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff550009990 tx=0x7ff550009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 -- 172.21.15.52:0/3368578311 shutdown_connections 2024-06-02T23:13:45.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 --2- 172.21.15.52:0/3368578311 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560101440 0x7ff56010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 --2- 172.21.15.52:0/3368578311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560107450 0x7ff560107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 --2- 172.21.15.52:0/3368578311 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560100a80 0x7ff560100f00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 -- 172.21.15.52:0/3368578311 >> 172.21.15.52:0/3368578311 conn(0x7ff560077a30 msgr2=0x7ff5600fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.165 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:45 smithi052 ceph-mon[32340]: pgmap v487: 97 pgs: 97 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:13:45.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 -- 172.21.15.52:0/3368578311 shutdown_connections 2024-06-02T23:13:45.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.162+0000 7ff568091700 1 -- 172.21.15.52:0/3368578311 wait complete. 2024-06-02T23:13:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/1885899249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 msgr2=0x55d05a4bb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4bb7c0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f6a040092e0 tx=0x7f6a0400ff50 comp rx=0 tx=0).stop 2024-06-02T23:13:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/1885899249 shutdown_connections 2024-06-02T23:13:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4a8350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.166 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4bb7c0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/1885899249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/1885899249 >> 172.21.15.52:0/1885899249 conn(0x55d05a4a5670 msgr2=0x55d05a4b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/1885899249 shutdown_connections 2024-06-02T23:13:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/1885899249 wait complete. 2024-06-02T23:13:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 -- start start 2024-06-02T23:13:45.167 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.163+0000 7f6a1ec6f740 1 Processor -- start 2024-06-02T23:13:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.163+0000 7ff568091700 1 Processor -- start 2024-06-02T23:13:45.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 -- start start 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6a141ac320 con 0x55d05a4aa9f0 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6a141abc50 con 0x55d05a4aa000 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6a141a8e90 con 0x55d05a4afb60 2024-06-02T23:13:45.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1ec6f740 10 client.0 _ll_register_callbacks cb 0x7f6a040097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560100a80 0x7ff560101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 0x7ff5601021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560107450 0x7ff5601026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601126e0 con 0x7ff560107450 2024-06-02T23:13:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff560112560 con 0x7ff560101440 2024-06-02T23:13:45.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff568091700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff560112860 con 0x7ff560100a80 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a19cfd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a194fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a19cfd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60696/0 (socket says 172.21.15.52:60696) 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a19cfd640 1 -- 172.21.15.52:0/2387922306 learned_addr learned my addr 172.21.15.52:0/2387922306 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a194fc640 1 -- 172.21.15.52:0/2387922306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 msgr2=0x55d05a4ba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4ba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a194fc640 1 -- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 msgr2=0x55d05a4b8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff56562c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 0x7ff5601021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff56562c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 0x7ff5601021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60706/0 (socket says 172.21.15.52:60706) 2024-06-02T23:13:45.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff56562c700 1 -- 172.21.15.52:0/1478061487 learned_addr learned my addr 172.21.15.52:0/1478061487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560100a80 0x7ff560101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a194fc640 1 -- 172.21.15.52:0/2387922306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d05a4b3360 con 0x55d05a4aa9f0 2024-06-02T23:13:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a1a4fe640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.164+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff56662e700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560107450 0x7ff5601026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.164+0000 7ff56562c700 1 -- 172.21.15.52:0/1478061487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560100a80 msgr2=0x7ff560101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff56562c700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560100a80 0x7ff560101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff56562c700 1 -- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560107450 msgr2=0x7ff5601026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff56562c700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560107450 0x7ff5601026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff56562c700 1 -- 172.21.15.52:0/1478061487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5601a7030 con 0x7ff560101440 2024-06-02T23:13:45.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff56662e700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560107450 0x7ff5601026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f6a194fc640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b9100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55d05a4b2760 tx=0x7f6a04011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7205cb0deccdbbb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f69fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:45.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff56562c700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 0x7ff5601021a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff550005910 tx=0x7ff5500047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2387922306 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a0400fb40 con 0x55d05a4aa9f0 2024-06-02T23:13:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2387922306 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6a04018040 con 0x55d05a4aa9f0 2024-06-02T23:13:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a0400fce0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff556ffd700 1 -- 172.21.15.52:0/1478061487 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550027070 con 0x7ff560101440 2024-06-02T23:13:45.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff556ffd700 1 -- 172.21.15.52:0/1478061487 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff550022470 con 0x7ff560101440 2024-06-02T23:13:45.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5601a72c0 con 0x7ff560101440 2024-06-02T23:13:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2387922306 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6a04005050 con 0x55d05a4aa9f0 2024-06-02T23:13:45.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f6a1ec6f740 10 client.16128 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:45.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.165+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55d05a4bb7f0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff556ffd700 1 -- 172.21.15.52:0/1478061487 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55000b610 con 0x7ff560101440 2024-06-02T23:13:45.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.165+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5601a7770 con 0x7ff560101440 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2387922306 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6a0400fce0 con 0x55d05a4aa9f0 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2387922306 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f6a04004c10 con 0x55d05a4aa9f0 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a1ec6f740 10 client.16128 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a1ec6f740 10 client.16128 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a1ec6f740 20 client.16128 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a52b3c0 0x55d05a52d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a1ec6f740 4 client.16128 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d05a41ac20 con 0x55d05a52b3c0 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a52b3c0 0x55d05a52d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.166+0000 7f69fd7fa640 10 client.16128 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.167+0000 7f6a19cfd640 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a52b3c0 0x55d05a52d7e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae9cf7e24c3b4f46 server_cookie=c731b6c304e7fa2e in_seq=0 out_seq=0 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.168+0000 7ff556ffd700 1 -- 172.21.15.52:0/1478061487 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff55002a070 con 0x7ff560101440 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.169+0000 7ff556ffd700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff54c0a2d40 0x7ff54c0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.169+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff54c0a2d40 0x7ff54c0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.169+0000 7ff556ffd700 1 -- 172.21.15.52:0/1478061487 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff5500cfd00 con 0x7ff560101440 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.169+0000 7ff572914b80 1 Processor -- start 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.169+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff54c0a2d40 0x7ff54c0a5200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009be0 tx=0x7ff55c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 -- start start 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763590 0x561123763970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 0x56112375d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x56112375e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5611236500f0 con 0x561123763eb0 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5611236ae9e0 con 0x561123763590 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff572914b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611236165f0 con 0x56112375dce0 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 0x56112375d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56662e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x56112375e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 0x56112375d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37548/0 (socket says 172.21.15.52:37548) 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 -- 172.21.15.52:0/1027399868 learned_addr learned my addr 172.21.15.52:0/1027399868 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763590 0x561123763970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 -- 172.21.15.52:0/1027399868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 msgr2=0x56112375e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x56112375e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 -- 172.21.15.52:0/1027399868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763590 msgr2=0x561123763970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763590 0x561123763970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56562c700 1 -- 172.21.15.52:0/1027399868 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611235fda60 con 0x561123763eb0 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.170+0000 7ff56662e700 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x56112375e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.171+0000 7ff56562c700 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 0x56112375d7a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff55009c5b0 tx=0x7ff550009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb19e954581777f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.171+0000 7ff54bfff700 1 -- 172.21.15.52:0/1027399868 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55009cd20 con 0x561123763eb0 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.171+0000 7ff54bfff700 1 -- 172.21.15.52:0/1027399868 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff55009c620 con 0x561123763eb0 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.172+0000 7ff54bfff700 1 -- 172.21.15.52:0/1027399868 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55009c880 con 0x561123763eb0 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 -- 172.21.15.52:0/1027399868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 msgr2=0x56112375d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 0x56112375d7a0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff55009c5b0 tx=0x7ff550009310 comp rx=0 tx=0).stop 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 -- 172.21.15.52:0/1027399868 shutdown_connections 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x56112375e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763590 0x561123763970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 --2- 172.21.15.52:0/1027399868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763eb0 0x56112375d7a0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 -- 172.21.15.52:0/1027399868 >> 172.21.15.52:0/1027399868 conn(0x56112368a210 msgr2=0x561123758a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 -- 172.21.15.52:0/1027399868 shutdown_connections 2024-06-02T23:13:45.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 -- 172.21.15.52:0/1027399868 wait complete. 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.173+0000 7ff572914b80 1 -- start start 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 Processor -- start 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x561123769b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x561123766e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763eb0 0x561123767360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56112364f150 con 0x561123763590 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5611236aa9a0 con 0x561123763eb0 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56112376a6e0 con 0x56112375dce0 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff572914b80 10 client.0 _ll_register_callbacks cb 0x561123644740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x561123766e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff565e2d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x561123769b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x561123766e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37558/0 (socket says 172.21.15.52:37558) 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.174+0000 7ff56562c700 1 -- 172.21.15.52:0/1561696048 learned_addr learned my addr 172.21.15.52:0/1561696048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56662e700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763eb0 0x561123767360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56562c700 1 -- 172.21.15.52:0/1561696048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 msgr2=0x561123769b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56562c700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x561123769b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56562c700 1 -- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763eb0 msgr2=0x561123767360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56562c700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763eb0 0x561123767360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56562c700 1 -- 172.21.15.52:0/1561696048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611236748e0 con 0x561123763590 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56662e700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763eb0 0x561123767360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x561123769b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff56562c700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x561123766e20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff550009750 tx=0x7ff5500a0ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6008cf7b44b9954e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff5497fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff5497fa700 1 -- 172.21.15.52:0/1561696048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550022990 con 0x561123763590 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff5497fa700 1 -- 172.21.15.52:0/1561696048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff5500bc040 con 0x561123763590 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff55009c620 con 0x561123763590 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff5497fa700 1 -- 172.21.15.52:0/1561696048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff55009c880 con 0x561123763590 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff572914b80 10 client.16134 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:45.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x561123774d20 con 0x561123763590 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.175+0000 7ff5497fa700 1 -- 172.21.15.52:0/1561696048 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff55009ca20 con 0x561123763590 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff5497fa700 1 -- 172.21.15.52:0/1561696048 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff5500a1c60 con 0x561123763590 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff572914b80 10 client.16134 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff572914b80 10 client.16134 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff572914b80 20 client.16134 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff572914b80 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237795e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff572914b80 4 client.16134 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff55009c620 con 0x5611237771f0 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237795e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237795e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ccd430dbcc7f62 server_cookie=f92d87c953a92e7a in_seq=0 out_seq=0 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.176+0000 7ff5497fa700 10 client.16134 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.178+0000 7ff5497fa700 1 -- 172.21.15.52:0/1561696048 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff55009c620 con 0x5611237771f0 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.178+0000 7ff5497fa700 10 client.16134 handle_command_reply: tid=0 2024-06-02T23:13:45.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 client.16134 shutdown 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 msgr2=0x561123766e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x561123766e20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7ff550009750 tx=0x7ff5500a0ac0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 shutdown_connections 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237795e0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561123763eb0 0x561123767360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff5497fa700 0 client.16134 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56112375dce0 0x561123769b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 --2- 172.21.15.52:0/1561696048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x561123766e20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 >> 172.21.15.52:0/1561696048 conn(0x56112368a210 msgr2=0x561123757c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 shutdown_connections 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 1 -- 172.21.15.52:0/1561696048 wait complete. 2024-06-02T23:13:45.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.179+0000 7ff572914b80 20 client.16134 trim_cache size 0 max 16384 2024-06-02T23:13:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f69fd7fa640 1 -- 172.21.15.52:0/2387922306 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d05a41ac20 con 0x55d05a52b3c0 2024-06-02T23:13:45.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f69fd7fa640 10 client.16128 handle_command_reply: tid=0 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 client.16128 shutdown 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 msgr2=0x55d05a4b9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b9100 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x55d05a4b2760 tx=0x7f6a04011bd0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 shutdown_connections 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d05a52b3c0 0x55d05a52d7e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d05a4afb60 0x55d05a4ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d05a4aa9f0 0x55d05a4b9100 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 --2- 172.21.15.52:0/2387922306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d05a4aa000 0x55d05a4b8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f69fd7fa640 0 client.16128 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 >> 172.21.15.52:0/2387922306 conn(0x55d05a4a5670 msgr2=0x55d05a4a8350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 shutdown_connections 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 1 -- 172.21.15.52:0/2387922306 wait complete. 2024-06-02T23:13:45.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.183+0000 7f6a1ec6f740 20 client.16128 trim_cache size 0 max 16384 2024-06-02T23:13:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:13:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:13:45.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:13:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6a000a2c30 msgr2=0x7f6a000a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6a000a2c30 0x7f6a000a50f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c023000 tx=0x7f6a0c021000 comp rx=0 tx=0).stop 2024-06-02T23:13:45.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 msgr2=0x7f6a1410b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 0x7f6a1410b280 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6a100314e0 tx=0x7f6a10031a30 comp rx=0 tx=0).stop 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 shutdown_connections 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6a000a2c30 0x7f6a000a50f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6a14101af0 0x7f6a141098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6a141024b0 0x7f6a14109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 --2- 172.21.15.52:0/3748169109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6a14100900 0x7f6a1410b280 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 >> 172.21.15.52:0/3748169109 conn(0x7f6a140fbe70 msgr2=0x7f6a14104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 shutdown_connections 2024-06-02T23:13:45.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:45.184+0000 7f6a1bf61640 1 -- 172.21.15.52:0/3748169109 wait complete. 2024-06-02T23:13:45.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 Processor -- start 2024-06-02T23:13:45.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 -- start start 2024-06-02T23:13:45.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x561123768d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x5611237583c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x561123758900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601aeb30 con 0x561123763590 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601b0700 con 0x56112375dce0 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff572914b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601af200 con 0x561123763eb0 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x5611237583c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff565e2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x561123768d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x5611237583c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37562/0 (socket says 172.21.15.52:37562) 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff565e2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x561123768d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60734/0 (socket says 172.21.15.52:60734) 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.201+0000 7ff56562c700 1 -- 172.21.15.52:0/4168857542 learned_addr learned my addr 172.21.15.52:0/4168857542 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56662e700 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x561123758900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56562c700 1 -- 172.21.15.52:0/4168857542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 msgr2=0x561123758900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56562c700 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x561123758900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56562c700 1 -- 172.21.15.52:0/4168857542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 msgr2=0x561123768d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56562c700 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x561123768d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56562c700 1 -- 172.21.15.52:0/4168857542 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561123765390 con 0x561123763590 2024-06-02T23:13:45.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff565e2d700 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x561123768d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff56562c700 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x5611237583c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff55009a8d0 tx=0x7ff55009ab30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138b7c2e06ddc4ff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff54b7fe700 1 -- 172.21.15.52:0/4168857542 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55009c620 con 0x561123763590 2024-06-02T23:13:45.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff54b7fe700 1 -- 172.21.15.52:0/4168857542 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff55009ca20 con 0x561123763590 2024-06-02T23:13:45.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.202+0000 7ff54b7fe700 1 -- 172.21.15.52:0/4168857542 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5500b5450 con 0x561123763590 2024-06-02T23:13:45.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 -- 172.21.15.52:0/4168857542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 msgr2=0x5611237583c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x5611237583c0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7ff55009a8d0 tx=0x7ff55009ab30 comp rx=0 tx=0).stop 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 -- 172.21.15.52:0/4168857542 shutdown_connections 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x561123758900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x561123768d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 --2- 172.21.15.52:0/4168857542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x5611237583c0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 -- 172.21.15.52:0/4168857542 >> 172.21.15.52:0/4168857542 conn(0x56112368a210 msgr2=0x561123768650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 -- 172.21.15.52:0/4168857542 shutdown_connections 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 -- 172.21.15.52:0/4168857542 wait complete. 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 -- start start 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.203+0000 7ff572914b80 1 Processor -- start 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x56112376c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x56112376cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x56112376d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601affa0 con 0x561123763590 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601af8d0 con 0x56112375dce0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5601acaf0 con 0x561123763eb0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff572914b80 10 client.0 _ll_register_callbacks cb 0x561123644740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x56112376c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff56562c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x56112376cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x56112376c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60744/0 (socket says 172.21.15.52:60744) 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 -- 172.21.15.52:0/1925102527 learned_addr learned my addr 172.21.15.52:0/1925102527 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff56662e700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x56112376d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 -- 172.21.15.52:0/1925102527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 msgr2=0x56112376d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x56112376d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 -- 172.21.15.52:0/1925102527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 msgr2=0x56112376cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x56112376cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 -- 172.21.15.52:0/1925102527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611237e1fa0 con 0x56112375dce0 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff56662e700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x56112376d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:45.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff56562c700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x56112376cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff565e2d700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x56112376c9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0117d0 tx=0x7ff55c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ada9b8a40ca49bd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.204+0000 7ff5497fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff5497fa700 1 -- 172.21.15.52:0/1925102527 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff55c004240 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff55009ca20 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff5497fa700 1 -- 172.21.15.52:0/1925102527 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff55c009040 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff5497fa700 1 -- 172.21.15.52:0/1925102527 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ff55c03a070 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 10 client.25724 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5611237e3280 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff5497fa700 1 -- 172.21.15.52:0/1925102527 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff55009ca20 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff5497fa700 1 -- 172.21.15.52:0/1925102527 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7ff55c02b510 con 0x56112375dce0 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 10 client.25724 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 10 client.25724 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 20 client.25724 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:45.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237e5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:45.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 4 client.25724 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-06-02T23:13:45.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x5611236bbee0 con 0x5611237771f0 2024-06-02T23:13:45.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.205+0000 7ff56562c700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237e5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:45.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.206+0000 7ff56562c700 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237e5a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8da0092054f0a1b6 server_cookie=ff6d6ea755458b0f in_seq=0 out_seq=0 2024-06-02T23:13:45.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.206+0000 7ff5497fa700 10 client.25724 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff5497fa700 1 -- 172.21.15.52:0/1925102527 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5611236bbee0 con 0x5611237771f0 2024-06-02T23:13:45.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff5497fa700 10 client.25724 handle_command_reply: tid=0 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 client.25724 shutdown 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 msgr2=0x56112376c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x56112376c9c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff55c0117d0 tx=0x7ff55c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 shutdown_connections 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611237771f0 0x5611237e5a20 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561123763eb0 0x56112376d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56112375dce0 0x56112376c9c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 --2- 172.21.15.52:0/1925102527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561123763590 0x56112376cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff5497fa700 0 client.25724 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 >> 172.21.15.52:0/1925102527 conn(0x56112368a210 msgr2=0x56112375d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 shutdown_connections 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 1 -- 172.21.15.52:0/1925102527 wait complete. 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.279+0000 7ff572914b80 20 client.25724 trim_cache size 0 max 16384 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 6, 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "eb31abfd": { 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 6, 2024-06-02T23:13:45.281 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 130.0, 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:13:45.282 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff54c0a2d40 msgr2=0x7ff54c0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff54c0a2d40 0x7ff54c0a5200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff55c009be0 tx=0x7ff55c009260 comp rx=0 tx=0).stop 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 msgr2=0x7ff5601021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 0x7ff5601021a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff550005910 tx=0x7ff5500047b0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 shutdown_connections 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff54c0a2d40 0x7ff54c0a5200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff560107450 0x7ff5601026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff560100a80 0x7ff560101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 --2- 172.21.15.52:0/1478061487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff560101440 0x7ff5601021a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 >> 172.21.15.52:0/1478061487 conn(0x7ff560077a30 msgr2=0x7ff5601053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 shutdown_connections 2024-06-02T23:13:45.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:45.280+0000 7ff568091700 1 -- 172.21.15.52:0/1478061487 wait complete. 2024-06-02T23:13:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:45 smithi174 ceph-mon[35994]: pgmap v487: 97 pgs: 97 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:13:45.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:45 smithi123 ceph-mon[35977]: pgmap v487: 97 pgs: 97 active+clean; 31 MiB data, 476 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:13:45.618 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 3.22 MB/sec warmup 54 sec latency 351.871 ms 2024-06-02T23:13:46.021 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eb31abfd', quiesce took 8.9 seconds. Will release after: 1.0999999999999996 2024-06-02T23:13:46.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 3.16 MB/sec warmup 55 sec latency 1351.975 ms 2024-06-02T23:13:47.123 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:13:47.391 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:47 smithi174 ceph-mon[35994]: pgmap v488: 97 pgs: 97 active+clean; 32 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-02T23:13:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:47 smithi052 ceph-mon[32340]: pgmap v488: 97 pgs: 97 active+clean; 32 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-02T23:13:47.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:47 smithi123 ceph-mon[35977]: pgmap v488: 97 pgs: 97 active+clean; 32 MiB data, 504 MiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-02T23:13:47.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 3.11 MB/sec warmup 56 sec latency 2352.066 ms 2024-06-02T23:13:48.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 3.05 MB/sec warmup 57 sec latency 3352.114 ms 2024-06-02T23:13:49.078 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:49 smithi052 ceph-mon[32340]: pgmap v489: 97 pgs: 97 active+clean; 33 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-06-02T23:13:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 -- 172.21.15.52:0/1209521545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698106720 msgr2=0x7f169810e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 --2- 172.21.15.52:0/1209521545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698106720 0x7f169810e2c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f169000b0d0 tx=0x7f169000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 -- 172.21.15.52:0/1209521545 shutdown_connections 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 --2- 172.21.15.52:0/1209521545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698106720 0x7f169810e2c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 --2- 172.21.15.52:0/1209521545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 0x7f1698101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 --2- 172.21.15.52:0/1209521545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1698102380 0x7f1698105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 -- 172.21.15.52:0/1209521545 >> 172.21.15.52:0/1209521545 conn(0x7f1698078340 msgr2=0x7f1698078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 -- 172.21.15.52:0/1209521545 shutdown_connections 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.179+0000 7f16a0759700 1 -- 172.21.15.52:0/1209521545 wait complete. 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 Processor -- start 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 -- start start 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 0x7f1698103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 0x7f16981035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1698106720 0x7f1698103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111590 con 0x7f1698102380 2024-06-02T23:13:49.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111410 con 0x7f1698106720 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f16a0759700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1698111710 con 0x7f16981019d0 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 0x7f16981035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 0x7f16981035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37580/0 (socket says 172.21.15.52:37580) 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 -- 172.21.15.52:0/1697148800 learned_addr learned my addr 172.21.15.52:0/1697148800 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169ecf6700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1698106720 0x7f1698103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169e4f5700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 0x7f1698103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 -- 172.21.15.52:0/1697148800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 msgr2=0x7f1698103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 0x7f1698103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 -- 172.21.15.52:0/1697148800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1698106720 msgr2=0x7f1698103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1698106720 0x7f1698103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169dcf4700 1 -- 172.21.15.52:0/1697148800 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16981a7050 con 0x7f1698102380 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.180+0000 7f169e4f5700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 0x7f1698103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:49.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.181+0000 7f169dcf4700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 0x7f16981035d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f168800d950 tx=0x7f168800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:49.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.181+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688017070 con 0x7f1698102380 2024-06-02T23:13:49.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.181+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16981a7340 con 0x7f1698102380 2024-06-02T23:13:49.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.181+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1688009de0 con 0x7f1698102380 2024-06-02T23:13:49.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.181+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168800f3c0 con 0x7f1698102380 2024-06-02T23:13:49.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.181+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16981a78d0 con 0x7f1698102380 2024-06-02T23:13:49.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.182+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1698102ad0 con 0x7f1698102380 2024-06-02T23:13:49.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.183+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f168801a070 con 0x7f1698102380 2024-06-02T23:13:49.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.184+0000 7f168f7fe700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16840a2c20 0x7f16840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:49.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.184+0000 7f169e4f5700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16840a2c20 0x7f16840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:49.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.184+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f16880ce860 con 0x7f1698102380 2024-06-02T23:13:49.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.185+0000 7f169e4f5700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16840a2c20 0x7f16840a50e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1694005fd0 tx=0x7f1694005dc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:49.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.188+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f168809b790 con 0x7f1698102380 2024-06-02T23:13:49.308 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:49 smithi123 ceph-mon[35977]: pgmap v489: 97 pgs: 97 active+clean; 33 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-06-02T23:13:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:49 smithi174 ceph-mon[35994]: pgmap v489: 97 pgs: 97 active+clean; 33 MiB data, 505 MiB used, 1.0 TiB / 1.0 TiB avail; 191 KiB/s rd, 1.2 MiB/s wr, 127 op/s 2024-06-02T23:13:49.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.439+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f169802dca0 con 0x7f1698102380 2024-06-02T23:13:49.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.440+0000 7f168f7fe700 1 -- 172.21.15.52:0/1697148800 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7f168809b090 con 0x7f1698102380 2024-06-02T23:13:49.441 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:13:49.441 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16840a2c20 msgr2=0x7f16840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16840a2c20 0x7f16840a50e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f1694005fd0 tx=0x7f1694005dc0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 msgr2=0x7f16981035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 0x7f16981035d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f168800d950 tx=0x7f168800dc60 comp rx=0 tx=0).stop 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 shutdown_connections 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16840a2c20 0x7f16840a50e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1698106720 0x7f1698103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16981019d0 0x7f1698103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 --2- 172.21.15.52:0/1697148800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1698102380 0x7f16981035d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:49.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 >> 172.21.15.52:0/1697148800 conn(0x7f1698078340 msgr2=0x7f169806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:49.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 shutdown_connections 2024-06-02T23:13:49.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:49.442+0000 7f16a0759700 1 -- 172.21.15.52:0/1697148800 wait complete. 2024-06-02T23:13:49.444 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:13:49.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 3.00 MB/sec warmup 58 sec latency 4352.220 ms 2024-06-02T23:13:50.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:50 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1697148800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:50.220 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id eb31abfd --release --await' 2024-06-02T23:13:50.220 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id eb31abfd --release --await 2024-06-02T23:13:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:50 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1697148800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:50.475 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:50.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:50 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1697148800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:50.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 2.95 MB/sec warmup 59 sec latency 5352.314 ms 2024-06-02T23:13:51.287 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:51 smithi052 ceph-mon[32340]: pgmap v490: 97 pgs: 97 active+clean; 33 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:13:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:51 smithi174 ceph-mon[35994]: pgmap v490: 97 pgs: 97 active+clean; 33 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:13:51.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:51 smithi123 ceph-mon[35977]: pgmap v490: 97 pgs: 97 active+clean; 33 MiB data, 510 MiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:13:51.556 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:13:51.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 28971 2.90 MB/sec warmup 60 sec latency 6352.383 ms 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 -- 172.21.15.52:0/2697816671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 msgr2=0x7f27a01077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 --2- 172.21.15.52:0/2697816671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01077d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f2790009990 tx=0x7f2790009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 -- 172.21.15.52:0/2697816671 shutdown_connections 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 --2- 172.21.15.52:0/2697816671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a0101d40 0x7f27a010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 --2- 172.21.15.52:0/2697816671 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01077d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 --2- 172.21.15.52:0/2697816671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27a0101380 0x7f27a0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 -- 172.21.15.52:0/2697816671 >> 172.21.15.52:0/2697816671 conn(0x7f27a0078300 msgr2=0x7f27a0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 -- 172.21.15.52:0/2697816671 shutdown_connections 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.137+0000 7f27a60dc700 1 -- 172.21.15.52:0/2697816671 wait complete. 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 Processor -- start 2024-06-02T23:13:52.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 -- start start 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27a0101380 0x7f27a019e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a0101d40 0x7f27a019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0111590 con 0x7f27a01073f0 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0111410 con 0x7f27a0101380 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.138+0000 7f27a60dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0111710 con 0x7f27a0101d40 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27a0101380 0x7f27a019e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37584/0 (socket says 172.21.15.52:37584) 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 -- 172.21.15.52:0/768871594 learned_addr learned my addr 172.21.15.52:0/768871594 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279effd700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a0101d40 0x7f27a019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 -- 172.21.15.52:0/768871594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a0101d40 msgr2=0x7f27a019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a0101d40 0x7f27a019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 -- 172.21.15.52:0/768871594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27a0101380 msgr2=0x7f27a019e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27a0101380 0x7f27a019e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 -- 172.21.15.52:0/768871594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0199140 con 0x7f27a01073f0 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279ffff700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01989e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f279800b970 tx=0x7f279800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279cff9700 1 -- 172.21.15.52:0/768871594 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279800d860 con 0x7f27a01073f0 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279cff9700 1 -- 172.21.15.52:0/768871594 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279800e430 con 0x7f27a01073f0 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a0199430 con 0x7f27a01073f0 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.139+0000 7f279cff9700 1 -- 172.21.15.52:0/768871594 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798017660 con 0x7f27a01073f0 2024-06-02T23:13:52.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.140+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01a56a0 con 0x7f27a01073f0 2024-06-02T23:13:52.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.142+0000 7f279cff9700 1 -- 172.21.15.52:0/768871594 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f279800e5e0 con 0x7f27a01073f0 2024-06-02T23:13:52.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.143+0000 7f279cff9700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27880a2ca0 0x7f27880a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.143+0000 7f279f7fe700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27880a2ca0 0x7f27880a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.143+0000 7f279cff9700 1 -- 172.21.15.52:0/768871594 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f27980ceb40 con 0x7f27a01073f0 2024-06-02T23:13:52.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.143+0000 7f279f7fe700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27880a2ca0 0x7f27880a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2790006010 tx=0x7f279000b560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:52.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 Processor -- start 2024-06-02T23:13:52.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 -- start start 2024-06-02T23:13:52.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42c910 0x5593fa42ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa42d230 0x5593fa426b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa427060 0x5593fa427480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5593fa315060 con 0x5593fa42d230 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593fa373950 con 0x5593fa42c910 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f27b095fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593fa2db560 con 0x5593fa427060 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f279effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa42d230 0x5593fa426b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f279effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa42d230 0x5593fa426b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37586/0 (socket says 172.21.15.52:37586) 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f279effd700 1 -- 172.21.15.52:0/141152844 learned_addr learned my addr 172.21.15.52:0/141152844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f279f7fe700 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42c910 0x5593fa42ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.144+0000 7f279ffff700 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa427060 0x5593fa427480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f279f7fe700 1 -- 172.21.15.52:0/141152844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa427060 msgr2=0x5593fa427480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f279f7fe700 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa427060 0x5593fa427480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f279f7fe700 1 -- 172.21.15.52:0/141152844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa42d230 msgr2=0x5593fa426b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f279f7fe700 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa42d230 0x5593fa426b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f279f7fe700 1 -- 172.21.15.52:0/141152844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593fa2c29d0 con 0x5593fa42c910 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f279f7fe700 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42c910 0x5593fa42ccf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f27900098e0 tx=0x7f2790020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd04824d3218295 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f2785ffb700 1 -- 172.21.15.52:0/141152844 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000f560 con 0x5593fa42c910 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f2785ffb700 1 -- 172.21.15.52:0/141152844 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279000fbb0 con 0x5593fa42c910 2024-06-02T23:13:52.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.145+0000 7f2785ffb700 1 -- 172.21.15.52:0/141152844 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790024bc0 con 0x5593fa42c910 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 -- 172.21.15.52:0/141152844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42c910 msgr2=0x5593fa42ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42c910 0x5593fa42ccf0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f27900098e0 tx=0x7f2790020000 comp rx=0 tx=0).stop 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 -- 172.21.15.52:0/141152844 shutdown_connections 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa427060 0x5593fa427480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42c910 0x5593fa42ccf0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 --2- 172.21.15.52:0/141152844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa42d230 0x5593fa426b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 -- 172.21.15.52:0/141152844 >> 172.21.15.52:0/141152844 conn(0x5593fa34f180 msgr2=0x5593fa41da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 -- 172.21.15.52:0/141152844 shutdown_connections 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 -- 172.21.15.52:0/141152844 wait complete. 2024-06-02T23:13:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.146+0000 7f27b095fb80 1 -- start start 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 Processor -- start 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa432fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa4302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa430800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5593fa3140c0 con 0x5593fa427060 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593fa36f910 con 0x5593fa42d230 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593fa433b50 con 0x5593fa42c910 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f27b095fb80 10 client.0 _ll_register_callbacks cb 0x5593fa3096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa432fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa432fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37588/0 (socket says 172.21.15.52:37588) 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f279effd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa4302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f279f7fe700 1 -- 172.21.15.52:0/1413376295 learned_addr learned my addr 172.21.15.52:0/1413376295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.147+0000 7f279ffff700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa430800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279effd700 1 -- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 msgr2=0x5593fa430800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279effd700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa430800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279effd700 1 -- 172.21.15.52:0/1413376295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 msgr2=0x5593fa432fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279effd700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa432fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279effd700 1 -- 172.21.15.52:0/1413376295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593fa339850 con 0x5593fa42c910 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279ffff700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa430800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279f7fe700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa432fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f279effd700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa4302c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f279400bb40 tx=0x7f279400be50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b087a0bb6e772af7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f277effd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f277effd700 1 -- 172.21.15.52:0/1413376295 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2794003d00 con 0x5593fa42c910 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f277effd700 1 -- 172.21.15.52:0/1413376295 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2794004830 con 0x5593fa42c910 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279000fbb0 con 0x5593fa42c910 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f277effd700 1 -- 172.21.15.52:0/1413376295 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2794022070 con 0x5593fa42c910 2024-06-02T23:13:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f27b095fb80 10 client.25651 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.148+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593fa43e2d0 con 0x5593fa42c910 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f277effd700 1 -- 172.21.15.52:0/1413376295 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279000fbb0 con 0x5593fa42c910 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f277effd700 1 -- 172.21.15.52:0/1413376295 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f279401b470 con 0x5593fa42c910 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f27b095fb80 10 client.25651 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f27b095fb80 10 client.25651 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f27b095fb80 20 client.25651 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f27b095fb80 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa440910 0x5593fa442d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f27b095fb80 4 client.25651 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593fa380e50 con 0x5593fa440910 2024-06-02T23:13:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.149+0000 7f279f7fe700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa440910 0x5593fa442d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.150+0000 7f277effd700 10 client.25651 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.150+0000 7f279f7fe700 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa440910 0x5593fa442d00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c322731408dbd86e server_cookie=dae237b687e9ef43 in_seq=0 out_seq=0 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f277effd700 1 -- 172.21.15.52:0/1413376295 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5593fa380e50 con 0x5593fa440910 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f277effd700 10 client.25651 handle_command_reply: tid=0 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 client.25651 shutdown 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 msgr2=0x5593fa4302c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa4302c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f279400bb40 tx=0x7f279400be50 comp rx=0 tx=0).stop 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 shutdown_connections 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa440910 0x5593fa442d00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa430800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa432fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f277effd700 0 client.25651 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 --2- 172.21.15.52:0/1413376295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa4302c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 >> 172.21.15.52:0/1413376295 conn(0x5593fa34f180 msgr2=0x5593fa41cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:52.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 shutdown_connections 2024-06-02T23:13:52.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 1 -- 172.21.15.52:0/1413376295 wait complete. 2024-06-02T23:13:52.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.151+0000 7f27b095fb80 20 client.25651 trim_cache size 0 max 16384 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.171+0000 7f27b095fb80 1 Processor -- start 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 -- start start 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa42f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa42dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa421e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aa7b0 con 0x5593fa427060 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ac380 con 0x5593fa42d230 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f27b095fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aae80 con 0x5593fa42c910 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa42f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa42f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37598/0 (socket says 172.21.15.52:37598) 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 -- 172.21.15.52:0/928180013 learned_addr learned my addr 172.21.15.52:0/928180013 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279effd700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa42dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279ffff700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa421e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 -- 172.21.15.52:0/928180013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 msgr2=0x5593fa42dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa42dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 -- 172.21.15.52:0/928180013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 msgr2=0x5593fa421e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa421e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 -- 172.21.15.52:0/928180013 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593fa443330 con 0x5593fa427060 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279effd700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa42dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279ffff700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa421e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.172+0000 7f279f7fe700 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa42f240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27900098e0 tx=0x7f2790023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=223d31d9b6335bca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27857fa700 1 -- 172.21.15.52:0/928180013 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279000fad0 con 0x5593fa427060 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27857fa700 1 -- 172.21.15.52:0/928180013 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279000fc70 con 0x5593fa427060 2024-06-02T23:13:52.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27857fa700 1 -- 172.21.15.52:0/928180013 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790024410 con 0x5593fa427060 2024-06-02T23:13:52.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 -- 172.21.15.52:0/928180013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 msgr2=0x5593fa42f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa42f240 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f27900098e0 tx=0x7f2790023000 comp rx=0 tx=0).stop 2024-06-02T23:13:52.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 -- 172.21.15.52:0/928180013 shutdown_connections 2024-06-02T23:13:52.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa421e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa42f240 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 --2- 172.21.15.52:0/928180013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa42dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.173+0000 7f27b095fb80 1 -- 172.21.15.52:0/928180013 >> 172.21.15.52:0/928180013 conn(0x5593fa34f180 msgr2=0x5593fa421700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 -- 172.21.15.52:0/928180013 shutdown_connections 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 -- 172.21.15.52:0/928180013 wait complete. 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 -- start start 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 Processor -- start 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa4483d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa448910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa448e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01abc20 con 0x5593fa427060 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ab550 con 0x5593fa42d230 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.174+0000 7f27b095fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01a8750 con 0x5593fa42c910 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f27b095fb80 10 client.0 _ll_register_callbacks cb 0x5593fa3096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa4483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa4483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37614/0 (socket says 172.21.15.52:37614) 2024-06-02T23:13:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 -- 172.21.15.52:0/524465329 learned_addr learned my addr 172.21.15.52:0/524465329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279effd700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa448910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279ffff700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa448e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 -- 172.21.15.52:0/524465329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 msgr2=0x5593fa448910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa448910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 -- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 msgr2=0x5593fa448e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa448e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 -- 172.21.15.52:0/524465329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593fa4aa780 con 0x5593fa427060 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279ffff700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa448e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279effd700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa448910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f279f7fe700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa4483d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2790010f80 tx=0x7f279000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e13bfe8acf4c9b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f277effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f277effd700 1 -- 172.21.15.52:0/524465329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790039660 con 0x5593fa427060 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.175+0000 7f277effd700 1 -- 172.21.15.52:0/524465329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2790039800 con 0x5593fa427060 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f279000fc70 con 0x5593fa427060 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f277effd700 1 -- 172.21.15.52:0/524465329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2790039a00 con 0x5593fa427060 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 10 client.16164 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5593fa4aba30 con 0x5593fa427060 2024-06-02T23:13:52.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f277effd700 1 -- 172.21.15.52:0/524465329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f279000fc70 con 0x5593fa427060 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f277effd700 1 -- 172.21.15.52:0/524465329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f279004b400 con 0x5593fa427060 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 10 client.16164 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 10 client.16164 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 20 client.16164 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa4abc50 0x5593fa4ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 4 client.16164 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eb31abfd", "release": true, "await": true}] 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.176+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eb31abfd", "release": true, "await": true}) -- 0x5593fa37b3d0 con 0x5593fa4abc50 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.177+0000 7f279effd700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa4abc50 0x5593fa4ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.177+0000 7f277effd700 10 client.16164 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:52.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.177+0000 7f279effd700 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa4abc50 0x5593fa4ae040 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=413f752b554e28f4 server_cookie=a3807ba2026b5e6 in_seq=0 out_seq=0 2024-06-02T23:13:52.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f277effd700 1 -- 172.21.15.52:0/524465329 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5593fa37b3d0 con 0x5593fa4abc50 2024-06-02T23:13:52.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f277effd700 10 client.16164 handle_command_reply: tid=0 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 client.16164 shutdown 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 msgr2=0x5593fa4483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa4483d0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f2790010f80 tx=0x7f279000f920 comp rx=0 tx=0).stop 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 shutdown_connections 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593fa4abc50 0x5593fa4ae040 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f277effd700 0 client.16164 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593fa42d230 0x5593fa448e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593fa427060 0x5593fa4483d0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 --2- 172.21.15.52:0/524465329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593fa42c910 0x5593fa448910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 >> 172.21.15.52:0/524465329 conn(0x5593fa34f180 msgr2=0x5593fa421f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 shutdown_connections 2024-06-02T23:13:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 1 -- 172.21.15.52:0/524465329 wait complete. 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.185+0000 7f27b095fb80 20 client.16164 trim_cache size 0 max 16384 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 8, 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "eb31abfd": { 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 8, 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:13:52.187 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 130.0, 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:52.188 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27880a2ca0 msgr2=0x7f27880a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27880a2ca0 0x7f27880a5160 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2790006010 tx=0x7f279000b560 comp rx=0 tx=0).stop 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 msgr2=0x7f27a01989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01989e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f279800b970 tx=0x7f279800bc80 comp rx=0 tx=0).stop 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 shutdown_connections 2024-06-02T23:13:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27880a2ca0 0x7f27880a5160 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27a01073f0 0x7f27a01989e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27a0101380 0x7f27a019e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 --2- 172.21.15.52:0/768871594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a0101d40 0x7f27a019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:52.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 >> 172.21.15.52:0/768871594 conn(0x7f27a0078300 msgr2=0x7f27a0100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:52.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 shutdown_connections 2024-06-02T23:13:52.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:52.186+0000 7f27a60dc700 1 -- 172.21.15.52:0/768871594 wait complete. 2024-06-02T23:13:52.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 29139 2.85 MB/sec warmup 61 sec latency 6920.243 ms 2024-06-02T23:13:52.987 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'eb31abfd', total seconds elapsed: 15.9 2024-06-02T23:13:53.008 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:53.284 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:53 smithi052 ceph-mon[32340]: pgmap v491: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 815 KiB/s wr, 107 op/s 2024-06-02T23:13:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:53 smithi174 ceph-mon[35994]: pgmap v491: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 815 KiB/s wr, 107 op/s 2024-06-02T23:13:53.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:53 smithi123 ceph-mon[35977]: pgmap v491: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 815 KiB/s wr, 107 op/s 2024-06-02T23:13:53.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 29675 2.84 MB/sec warmup 62 sec latency 11.338 ms 2024-06-02T23:13:54.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 30331 2.85 MB/sec warmup 63 sec latency 12.277 ms 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 -- 172.21.15.52:0/1747979014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8108510 msgr2=0x7f54b81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 --2- 172.21.15.52:0/1747979014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8108510 0x7f54b81088f0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f54a8009990 tx=0x7f54a8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 -- 172.21.15.52:0/1747979014 shutdown_connections 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 --2- 172.21.15.52:0/1747979014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8101440 0x7f54b810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 --2- 172.21.15.52:0/1747979014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8108510 0x7f54b81088f0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 --2- 172.21.15.52:0/1747979014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100a80 0x7f54b8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.967+0000 7f54be638700 1 -- 172.21.15.52:0/1747979014 >> 172.21.15.52:0/1747979014 conn(0x7f54b8077a00 msgr2=0x7f54b80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.968+0000 7f54be638700 1 -- 172.21.15.52:0/1747979014 shutdown_connections 2024-06-02T23:13:54.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.968+0000 7f54be638700 1 -- 172.21.15.52:0/1747979014 wait complete. 2024-06-02T23:13:54.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.968+0000 7f54be638700 1 Processor -- start 2024-06-02T23:13:54.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.968+0000 7f54be638700 1 -- start start 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54be638700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100a80 0x7f54b8101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54be638700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 0x7f54b81021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54be638700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8108510 0x7f54b81026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54be638700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8112690 con 0x7f54b8101440 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 0x7f54b81021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100a80 0x7f54b8101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 0x7f54b81021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47396/0 (socket says 172.21.15.52:47396) 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54b77fe700 1 -- 172.21.15.52:0/1711137588 learned_addr learned my addr 172.21.15.52:0/1711137588 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8112510 con 0x7f54b8100a80 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54b8112810 con 0x7f54b8108510 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.969+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8108510 0x7f54b81026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b77fe700 1 -- 172.21.15.52:0/1711137588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8108510 msgr2=0x7f54b81026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:54.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b77fe700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8108510 0x7f54b81026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b77fe700 1 -- 172.21.15.52:0/1711137588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100a80 msgr2=0x7f54b8101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b77fe700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100a80 0x7f54b8101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b77fe700 1 -- 172.21.15.52:0/1711137588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b81a70f0 con 0x7f54b8101440 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8108510 0x7f54b81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b77fe700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 0x7f54b81021a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00e9a0 tx=0x7f54ac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b57fa700 1 -- 172.21.15.52:0/1711137588 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac00ca10 con 0x7f54b8101440 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b81a73e0 con 0x7f54b8101440 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b81a7970 con 0x7f54b8101440 2024-06-02T23:13:54.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b57fa700 1 -- 172.21.15.52:0/1711137588 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54ac00cbb0 con 0x7f54b8101440 2024-06-02T23:13:54.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.970+0000 7f54b57fa700 1 -- 172.21.15.52:0/1711137588 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac018680 con 0x7f54b8101440 2024-06-02T23:13:54.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.973+0000 7f54b57fa700 1 -- 172.21.15.52:0/1711137588 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54ac018820 con 0x7f54b8101440 2024-06-02T23:13:54.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54b57fa700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54a00a2ca0 0x7f54a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 Processor -- start 2024-06-02T23:13:54.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 -- start start 2024-06-02T23:13:54.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54b7fff700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54a00a2ca0 0x7f54a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54b57fa700 1 -- 172.21.15.52:0/1711137588 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54ac0cf060 con 0x7f54b8101440 2024-06-02T23:13:54.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 0x56028f3c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3c1880 0x56028f3bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3bb7a0 0x56028f3bbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56028f2adf60 con 0x56028f3c0f60 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56028f30c850 con 0x56028f3bb7a0 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56028f274460 con 0x56028f3c1880 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 0x56028f3c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 0x56028f3c1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47412/0 (socket says 172.21.15.52:47412) 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.975+0000 7f54b7fff700 1 -- 172.21.15.52:0/2835590342 learned_addr learned my addr 172.21.15.52:0/2835590342 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3bb7a0 0x56028f3bbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b77fe700 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3c1880 0x56028f3bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 -- 172.21.15.52:0/2835590342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3c1880 msgr2=0x56028f3bb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:54.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3c1880 0x56028f3bb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 -- 172.21.15.52:0/2835590342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3bb7a0 msgr2=0x56028f3bbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:54.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3bb7a0 0x56028f3bbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 -- 172.21.15.52:0/2835590342 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028f25b8d0 con 0x56028f3c0f60 2024-06-02T23:13:54.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54a00a2ca0 0x7f54a00a5160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b2e0 tx=0x7f54a8005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:54.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f54b7fff700 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 0x56028f3c1340 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f54a8028000 tx=0x7f54a8029000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9ae8a51fd3ebd5b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f549e7fc700 1 -- 172.21.15.52:0/2835590342 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8034070 con 0x56028f3c0f60 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f549e7fc700 1 -- 172.21.15.52:0/2835590342 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a800bdf0 con 0x56028f3c0f60 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.976+0000 7f549e7fc700 1 -- 172.21.15.52:0/2835590342 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a802b070 con 0x56028f3c0f60 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2835590342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 msgr2=0x56028f3c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 0x56028f3c1340 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f54a8028000 tx=0x7f54a8029000 comp rx=0 tx=0).stop 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2835590342 shutdown_connections 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3bb7a0 0x56028f3bbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c0f60 0x56028f3c1340 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2835590342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3c1880 0x56028f3bb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2835590342 >> 172.21.15.52:0/2835590342 conn(0x56028f2e8080 msgr2=0x56028f3b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2835590342 shutdown_connections 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.977+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2835590342 wait complete. 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.978+0000 7f54c8ebbb80 1 -- start start 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.978+0000 7f54c8ebbb80 1 Processor -- start 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3c7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56028f2acfc0 con 0x56028f3c1880 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56028f308810 con 0x56028f3c0f60 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56028f3c8200 con 0x56028f3bb7a0 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54c8ebbb80 10 client.0 _ll_register_callbacks cb 0x56028f2a25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49982/0 (socket says 172.21.15.52:49982) 2024-06-02T23:13:54.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 -- 172.21.15.52:0/1006791486 learned_addr learned my addr 172.21.15.52:0/1006791486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b7fff700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 -- 172.21.15.52:0/1006791486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 msgr2=0x56028f3c7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3c7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 -- 172.21.15.52:0/1006791486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 msgr2=0x56028f3c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.979+0000 7f54b77fe700 1 -- 172.21.15.52:0/1006791486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028f2d2750 con 0x56028f3c0f60 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54b77fe700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c4980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54ac010d50 tx=0x7f54ac09b890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec35555212176421 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54977fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54977fe700 1 -- 172.21.15.52:0/1006791486 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0a4500 con 0x56028f3c0f60 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a800bdf0 con 0x56028f3c0f60 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54977fe700 1 -- 172.21.15.52:0/1006791486 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54ac09baf0 con 0x56028f3c0f60 2024-06-02T23:13:54.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54977fe700 1 -- 172.21.15.52:0/1006791486 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54ac09bcf0 con 0x56028f3c0f60 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54c8ebbb80 10 client.25754 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56028f3d2890 con 0x56028f3c0f60 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54977fe700 1 -- 172.21.15.52:0/1006791486 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54a800bdf0 con 0x56028f3c0f60 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54977fe700 1 -- 172.21.15.52:0/1006791486 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f54ac09cbc0 con 0x56028f3c0f60 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54c8ebbb80 10 client.25754 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.980+0000 7f54c8ebbb80 10 client.25754 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54c8ebbb80 20 client.25754 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f3d4e70 0x56028f3d7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54c8ebbb80 4 client.25754 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56028f319d50 con 0x56028f3d4e70 2024-06-02T23:13:54.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54b7fff700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f3d4e70 0x56028f3d7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:54.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54b7fff700 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f3d4e70 0x56028f3d7260 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a4f26569b80fd62 server_cookie=ccba0ac3614c8980 in_seq=0 out_seq=0 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.981+0000 7f54977fe700 10 client.25754 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54977fe700 1 -- 172.21.15.52:0/1006791486 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56028f319d50 con 0x56028f3d4e70 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54977fe700 10 client.25754 handle_command_reply: tid=0 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 client.25754 shutdown 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 msgr2=0x56028f3c4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c4980 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f54ac010d50 tx=0x7f54ac09b890 comp rx=0 tx=0).stop 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 shutdown_connections 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f3d4e70 0x56028f3d7260 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3c7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/1006791486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c4980 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:54.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.982+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 >> 172.21.15.52:0/1006791486 conn(0x56028f2e8080 msgr2=0x56028f3b5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:54.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.983+0000 7f54977fe700 0 client.25754 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:54.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.983+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 shutdown_connections 2024-06-02T23:13:54.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.983+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/1006791486 wait complete. 2024-06-02T23:13:54.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:54.983+0000 7f54c8ebbb80 20 client.25754 trim_cache size 0 max 16384 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.009+0000 7f54c8ebbb80 1 Processor -- start 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.009+0000 7f54c8ebbb80 1 -- start start 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400d2a0 con 0x56028f3c1880 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400ee70 con 0x56028f3c0f60 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400d970 con 0x56028f3bb7a0 2024-06-02T23:13:55.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54bcbd5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54bcbd5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47428/0 (socket says 172.21.15.52:47428) 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54bcbd5700 1 -- 172.21.15.52:0/684274783 learned_addr learned my addr 172.21.15.52:0/684274783 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b7fff700 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 -- 172.21.15.52:0/684274783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 msgr2=0x56028f3bed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3bed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 -- 172.21.15.52:0/684274783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 msgr2=0x56028f3c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 -- 172.21.15.52:0/684274783 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56028f3b2810 con 0x56028f3c0f60 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.010+0000 7f54b77fe700 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c38e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f54ac010df0 tx=0x7f54ac09fdc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba1c803de1df9804 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f549dffb700 1 -- 172.21.15.52:0/684274783 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac0ba1c0 con 0x56028f3c0f60 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f549dffb700 1 -- 172.21.15.52:0/684274783 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54ac0b4070 con 0x56028f3c0f60 2024-06-02T23:13:55.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f549dffb700 1 -- 172.21.15.52:0/684274783 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ac09cbc0 con 0x56028f3c0f60 2024-06-02T23:13:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/684274783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 msgr2=0x56028f3c38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c38e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f54ac010df0 tx=0x7f54ac09fdc0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/684274783 shutdown_connections 2024-06-02T23:13:55.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3bed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/684274783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3c38e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/684274783 >> 172.21.15.52:0/684274783 conn(0x56028f2e8080 msgr2=0x56028f3bfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/684274783 shutdown_connections 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.011+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/684274783 wait complete. 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 -- start start 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 Processor -- start 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3cd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3dc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400e710 con 0x56028f3c1880 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400e040 con 0x56028f3c0f60 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54a400b240 con 0x56028f3bb7a0 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.012+0000 7f54c8ebbb80 10 client.0 _ll_register_callbacks cb 0x56028f2a25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:55.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3dc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54b77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3cdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49988/0 (socket says 172.21.15.52:49988) 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54b77fe700 1 -- 172.21.15.52:0/2547727467 learned_addr learned my addr 172.21.15.52:0/2547727467 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54b7fff700 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 -- 172.21.15.52:0/2547727467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 msgr2=0x56028f3cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 -- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 msgr2=0x56028f3cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 -- 172.21.15.52:0/2547727467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56028f3dcaa0 con 0x56028f3c1880 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54bcbd5700 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3dc230 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f54b00186a0 tx=0x7f54b00189b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717334ea3f6307e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54977fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54977fe700 1 -- 172.21.15.52:0/2547727467 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b0004bb0 con 0x56028f3c1880 2024-06-02T23:13:55.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54ac0b4070 con 0x56028f3c1880 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.013+0000 7f54977fe700 1 -- 172.21.15.52:0/2547727467 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54b0004d50 con 0x56028f3c1880 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54977fe700 1 -- 172.21.15.52:0/2547727467 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f54b0022470 con 0x56028f3c1880 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 10 client.16182 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56028f43f820 con 0x56028f3c1880 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54977fe700 1 -- 172.21.15.52:0/2547727467 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54ac0b4070 con 0x56028f3c1880 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54977fe700 1 -- 172.21.15.52:0/2547727467 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f54b0016770 con 0x56028f3c1880 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 10 client.16182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 20 client.16182 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f43ff70 0x56028f442390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 4 client.16182 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56028f3142d0 con 0x56028f43ff70 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.014+0000 7f54b7fff700 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f43ff70 0x56028f442390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54977fe700 10 client.16182 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54b7fff700 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f43ff70 0x56028f442390 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=749521a70355ef1a server_cookie=8f6a87c4f8472b0 in_seq=0 out_seq=0 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54977fe700 1 -- 172.21.15.52:0/2547727467 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56028f3142d0 con 0x56028f43ff70 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54977fe700 10 client.16182 handle_command_reply: tid=0 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 client.16182 shutdown 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 msgr2=0x56028f3dc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3dc230 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f54b00186a0 tx=0x7f54b00189b0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 shutdown_connections 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56028f43ff70 0x56028f442390 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56028f3c1880 0x56028f3dc230 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56028f3bb7a0 0x56028f3cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 --2- 172.21.15.52:0/2547727467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56028f3c0f60 0x56028f3cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 >> 172.21.15.52:0/2547727467 conn(0x56028f2e8080 msgr2=0x56028f3c0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54977fe700 0 client.16182 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 shutdown_connections 2024-06-02T23:13:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.015+0000 7f54c8ebbb80 1 -- 172.21.15.52:0/2547727467 wait complete. 2024-06-02T23:13:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54c8ebbb80 20 client.16182 trim_cache size 0 max 16384 2024-06-02T23:13:55.021 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:13:55.021 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:13:55.021 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:13:55.021 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54a00a2ca0 msgr2=0x7f54a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54a00a2ca0 0x7f54a00a5160 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f54a800b2e0 tx=0x7f54a8005fd0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 msgr2=0x7f54b81021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 0x7f54b81021a0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f54ac00e9a0 tx=0x7f54ac00ecb0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 shutdown_connections 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54a00a2ca0 0x7f54a00a5160 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54b8108510 0x7f54b81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54b8100a80 0x7f54b8101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 --2- 172.21.15.52:0/1711137588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54b8101440 0x7f54b81021a0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 >> 172.21.15.52:0/1711137588 conn(0x7f54b8077a00 msgr2=0x7f54b80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 shutdown_connections 2024-06-02T23:13:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:55.016+0000 7f54be638700 1 -- 172.21.15.52:0/1711137588 wait complete. 2024-06-02T23:13:55.335 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:55 smithi052 ceph-mon[32340]: pgmap v492: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 637 KiB/s wr, 88 op/s 2024-06-02T23:13:55.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:55 smithi174 ceph-mon[35994]: pgmap v492: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 637 KiB/s wr, 88 op/s 2024-06-02T23:13:55.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:55 smithi123 ceph-mon[35977]: pgmap v492: 97 pgs: 97 active+clean; 32 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 637 KiB/s wr, 88 op/s 2024-06-02T23:13:55.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 30889 2.84 MB/sec warmup 64 sec latency 15.800 ms 2024-06-02T23:13:55.841 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:13:55.841 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:13:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:56.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 31692 2.90 MB/sec warmup 65 sec latency 12.850 ms 2024-06-02T23:13:57.325 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:57 smithi052 ceph-mon[32340]: pgmap v493: 97 pgs: 97 active+clean; 33 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 772 KiB/s wr, 111 op/s 2024-06-02T23:13:57.403 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:57 smithi174 ceph-mon[35994]: pgmap v493: 97 pgs: 97 active+clean; 33 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 772 KiB/s wr, 111 op/s 2024-06-02T23:13:57.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:57 smithi123 ceph-mon[35977]: pgmap v493: 97 pgs: 97 active+clean; 33 MiB data, 466 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 772 KiB/s wr, 111 op/s 2024-06-02T23:13:57.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 32101 2.90 MB/sec warmup 66 sec latency 11.838 ms 2024-06-02T23:13:58.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 -- 172.21.15.52:0/1296995824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30102070 msgr2=0x7efc3010e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 --2- 172.21.15.52:0/1296995824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30102070 0x7efc3010e1e0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7efc2800b0a0 tx=0x7efc2800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 -- 172.21.15.52:0/1296995824 shutdown_connections 2024-06-02T23:13:58.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 --2- 172.21.15.52:0/1296995824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30102070 0x7efc3010e1e0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 --2- 172.21.15.52:0/1296995824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc30107500 0x7efc301078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 --2- 172.21.15.52:0/1296995824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc301016b0 0x7efc30101b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 -- 172.21.15.52:0/1296995824 >> 172.21.15.52:0/1296995824 conn(0x7efc300780f0 msgr2=0x7efc30078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 -- 172.21.15.52:0/1296995824 shutdown_connections 2024-06-02T23:13:58.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.156+0000 7efc380d2700 1 -- 172.21.15.52:0/1296995824 wait complete. 2024-06-02T23:13:58.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 Processor -- start 2024-06-02T23:13:58.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 -- start start 2024-06-02T23:13:58.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc301016b0 0x7efc30102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 0x7efc30103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30107500 0x7efc30103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efc30111590 con 0x7efc30107500 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc30111410 con 0x7efc30102070 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc380d2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc30111710 con 0x7efc301016b0 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc3566d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 0x7efc30103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc3666f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30107500 0x7efc30103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc3566d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 0x7efc30103430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50010/0 (socket says 172.21.15.52:50010) 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.157+0000 7efc3566d700 1 -- 172.21.15.52:0/2555280829 learned_addr learned my addr 172.21.15.52:0/2555280829 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc35e6e700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc301016b0 0x7efc30102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3566d700 1 -- 172.21.15.52:0/2555280829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc301016b0 msgr2=0x7efc30102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3566d700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc301016b0 0x7efc30102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3566d700 1 -- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30107500 msgr2=0x7efc30103970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3566d700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30107500 0x7efc30103970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3566d700 1 -- 172.21.15.52:0/2555280829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc301a6f40 con 0x7efc30102070 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3666f700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30107500 0x7efc30103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc3566d700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 0x7efc30103430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e9a0 tx=0x7efc2000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2000ca10 con 0x7efc30102070 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc301a7230 con 0x7efc30102070 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc2000cbb0 con 0x7efc30102070 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc20018680 con 0x7efc30102070 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.158+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc301a77c0 con 0x7efc30102070 2024-06-02T23:13:58.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.159+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc30102b20 con 0x7efc30102070 2024-06-02T23:13:58.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.161+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc20010430 con 0x7efc30102070 2024-06-02T23:13:58.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.161+0000 7efc26ffd700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc1c0a2c20 0x7efc1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.162+0000 7efc35e6e700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc1c0a2c20 0x7efc1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.162+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efc200cf580 con 0x7efc30102070 2024-06-02T23:13:58.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.162+0000 7efc35e6e700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc1c0a2c20 0x7efc1c0a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc2c007b60 tx=0x7efc2c0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.165+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7efc2009c4b0 con 0x7efc30102070 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.281+0000 7fd23249e640 1 -- 172.21.15.52:0/2244601500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c100900 msgr2=0x7fd22c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.281+0000 7fd23249e640 1 --2- 172.21.15.52:0/2244601500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c100900 0x7fd22c100d20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c02a410 comp rx=0 tx=0).stop 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.281+0000 7fd23249e640 1 -- 172.21.15.52:0/2244601500 shutdown_connections 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.281+0000 7fd23249e640 1 --2- 172.21.15.52:0/2244601500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c1024b0 0x7fd22c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.281+0000 7fd23249e640 1 --2- 172.21.15.52:0/2244601500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 0x7fd22c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.281+0000 7fd23249e640 1 --2- 172.21.15.52:0/2244601500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c100900 0x7fd22c100d20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.282+0000 7fd23249e640 1 -- 172.21.15.52:0/2244601500 >> 172.21.15.52:0/2244601500 conn(0x7fd22c0fbe70 msgr2=0x7fd22c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.284 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.282+0000 7fd23249e640 1 -- 172.21.15.52:0/2244601500 shutdown_connections 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.282+0000 7fd23249e640 1 -- 172.21.15.52:0/2244601500 wait complete. 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.282+0000 7fd23249e640 1 Processor -- start 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.282+0000 7fd23249e640 1 -- start start 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.282+0000 7fd23249e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c100900 0x7fd22c10b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd23249e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 0x7fd22c1098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd23249e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 0x7fd22c109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd23249e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c10d5b0 con 0x7fd22c1024b0 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd23249e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c10d430 con 0x7fd22c101af0 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd23249e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c10d730 con 0x7fd22c100900 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 0x7fd22c109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c100900 0x7fd22c10b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 0x7fd22c109e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47460/0 (socket says 172.21.15.52:47460) 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 -- 172.21.15.52:0/2877926392 learned_addr learned my addr 172.21.15.52:0/2877926392 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:58.285 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 0x7fd22c1098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 -- 172.21.15.52:0/2877926392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c100900 msgr2=0x7fd22c10b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c100900 0x7fd22c10b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 -- 172.21.15.52:0/2877926392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 msgr2=0x7fd22c1098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 0x7fd22c1098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 -- 172.21.15.52:0/2877926392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c10a690 con 0x7fd22c1024b0 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 0x7fd22c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd22bfff640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c100900 0x7fd22c10b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd230a3b640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 0x7fd22c109e00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd22400eb10 tx=0x7fd22400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.283+0000 7fd2297fa640 1 -- 172.21.15.52:0/2877926392 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224004b20 con 0x7fd22c1024b0 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.284+0000 7fd2297fa640 1 -- 172.21.15.52:0/2877926392 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd224004cc0 con 0x7fd22c1024b0 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.284+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c1a1d00 con 0x7fd22c1024b0 2024-06-02T23:13:58.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.284+0000 7fd2297fa640 1 -- 172.21.15.52:0/2877926392 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd224010690 con 0x7fd22c1024b0 2024-06-02T23:13:58.287 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.284+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c1a21a0 con 0x7fd22c1024b0 2024-06-02T23:13:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.286+0000 7fd2297fa640 1 -- 172.21.15.52:0/2877926392 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd224005020 con 0x7fd22c1024b0 2024-06-02T23:13:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.286+0000 7fd2351ab740 1 Processor -- start 2024-06-02T23:13:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.286+0000 7fd2351ab740 1 -- start start 2024-06-02T23:13:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd601d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 0x55cbd6022500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd6022b70 0x55cbd6025000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2351ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cbd5f804f0 con 0x55cbd6022b70 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2351ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cbd5f1e870 con 0x55cbd601da00 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2351ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cbd5f7c4b0 con 0x55cbd601d010 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 0x55cbd6022500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd601d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 0x55cbd6022500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50044/0 (socket says 172.21.15.52:50044) 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 -- 172.21.15.52:0/2394198241 learned_addr learned my addr 172.21.15.52:0/2394198241 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd6022b70 0x55cbd6025000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2297fa640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd2140a2c30 0x7fd2140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 -- 172.21.15.52:0/2394198241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 msgr2=0x55cbd601d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd601d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 -- 172.21.15.52:0/2394198241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd6022b70 msgr2=0x55cbd6025000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd6022b70 0x55cbd6025000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22bfff640 1 -- 172.21.15.52:0/2394198241 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd5f3a9b0 con 0x55cbd601da00 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd6022b70 0x55cbd6025000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd230a3b640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd2140a2c30 0x7fd2140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.287+0000 7fd2297fa640 1 -- 172.21.15.52:0/2877926392 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd2240cf740 con 0x7fd22c1024b0 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.288+0000 7fd22bfff640 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 0x55cbd6022500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c0043f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a07c72be1640f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.288+0000 7fd2127fc640 1 -- 172.21.15.52:0/2394198241 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c040070 con 0x55cbd601da00 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.288+0000 7fd230a3b640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd2140a2c30 0x7fd2140a50f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2240989a0 tx=0x7fd224066c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.288+0000 7fd2127fc640 1 -- 172.21.15.52:0/2394198241 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd21c003ca0 con 0x55cbd601da00 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.288+0000 7fd2127fc640 1 -- 172.21.15.52:0/2394198241 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c035040 con 0x55cbd601da00 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 -- 172.21.15.52:0/2394198241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 msgr2=0x55cbd6022500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 0x55cbd6022500 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009900 tx=0x7fd21c0043f0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 -- 172.21.15.52:0/2394198241 shutdown_connections 2024-06-02T23:13:58.291 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd6022b70 0x55cbd6025000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601da00 0x55cbd6022500 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 --2- 172.21.15.52:0/2394198241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd601d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 -- 172.21.15.52:0/2394198241 >> 172.21.15.52:0/2394198241 conn(0x55cbd6018680 msgr2=0x55cbd6018a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 -- 172.21.15.52:0/2394198241 shutdown_connections 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 -- 172.21.15.52:0/2394198241 wait complete. 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 -- start start 2024-06-02T23:13:58.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.289+0000 7fd2351ab740 1 Processor -- start 2024-06-02T23:13:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.295 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd602e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cbd6025ac0 con 0x55cbd601da00 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cbd5f09ae0 con 0x55cbd601d010 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cbd6025c40 con 0x55cbd6022b70 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 10 client.0 _ll_register_callbacks cb 0x55cbd5f429b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47480/0 (socket says 172.21.15.52:47480) 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50050/0 (socket says 172.21.15.52:50050) 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 -- 172.21.15.52:0/3819033749 learned_addr learned my addr 172.21.15.52:0/3819033749 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd602e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 -- 172.21.15.52:0/3819033749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 msgr2=0x55cbd602e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd602e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 -- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 msgr2=0x55cbd60272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 -- 172.21.15.52:0/3819033749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd5c94210 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd22bfff640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd21c02a930 tx=0x7fd21c031830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e032ab72bc75484 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd20b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd20b7fe640 1 -- 172.21.15.52:0/3819033749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c035070 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd20b7fe640 1 -- 172.21.15.52:0/3819033749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd21c0035f0 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.290+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd21c003ca0 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd20b7fe640 1 -- 172.21.15.52:0/3819033749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd21c040040 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 10 client.16200 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbd5d60620 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd20b7fe640 1 -- 172.21.15.52:0/3819033749 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd21c003ca0 con 0x55cbd601da00 2024-06-02T23:13:58.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd20b7fe640 1 -- 172.21.15.52:0/3819033749 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd21c004750 con 0x55cbd601da00 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 10 client.16200 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 10 client.16200 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 20 client.16200 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd6032cc0 0x55cbd60350b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 4 client.16200 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbd5f93f10 con 0x55cbd6032cc0 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.291+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd6032cc0 0x55cbd60350b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.292+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd6032cc0 0x55cbd60350b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d27d6aa805d834f6 server_cookie=b64d937be4e7de08 in_seq=0 out_seq=0 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.292+0000 7fd20b7fe640 10 client.16200 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd20b7fe640 1 -- 172.21.15.52:0/3819033749 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cbd5f93f10 con 0x55cbd6032cc0 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd20b7fe640 10 client.16200 handle_command_reply: tid=0 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 client.16200 shutdown 2024-06-02T23:13:58.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 msgr2=0x55cbd6027800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fd21c02a930 tx=0x7fd21c031830 comp rx=0 tx=0).stop 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 shutdown_connections 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd6032cc0 0x55cbd60350b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd602e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 --2- 172.21.15.52:0/3819033749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 >> 172.21.15.52:0/3819033749 conn(0x55cbd6018680 msgr2=0x55cbd60177e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd20b7fe640 0 client.16200 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 shutdown_connections 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.293+0000 7fd2351ab740 1 -- 172.21.15.52:0/3819033749 wait complete. 2024-06-02T23:13:58.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.294+0000 7fd2351ab740 20 client.16200 trim_cache size 0 max 16384 2024-06-02T23:13:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.310+0000 7fd2351ab740 1 Processor -- start 2024-06-02T23:13:58.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.310+0000 7fd2351ab740 1 -- start start 2024-06-02T23:13:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd6022b70 0x55cbd602e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd2351ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1aae30 con 0x55cbd601da00 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd2351ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1aca00 con 0x55cbd6022b70 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd2351ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ab500 con 0x55cbd601d010 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47488/0 (socket says 172.21.15.52:47488) 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd22bfff640 1 -- 172.21.15.52:0/686307412 learned_addr learned my addr 172.21.15.52:0/686307412 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.311+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd6022b70 0x55cbd602e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd22bfff640 1 -- 172.21.15.52:0/686307412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 msgr2=0x55cbd60272c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd22bfff640 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd22bfff640 1 -- 172.21.15.52:0/686307412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd6022b70 msgr2=0x55cbd602e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd22bfff640 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd6022b70 0x55cbd602e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd22bfff640 1 -- 172.21.15.52:0/686307412 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbd601e7a0 con 0x55cbd601da00 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd22bfff640 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd21c004650 tx=0x7fd21c031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0da60423e9cb5f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd211ffb640 1 -- 172.21.15.52:0/686307412 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c035040 con 0x55cbd601da00 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd211ffb640 1 -- 172.21.15.52:0/686307412 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd21c04d070 con 0x55cbd601da00 2024-06-02T23:13:58.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.312+0000 7fd211ffb640 1 -- 172.21.15.52:0/686307412 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0035f0 con 0x55cbd601da00 2024-06-02T23:13:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 -- 172.21.15.52:0/686307412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 msgr2=0x55cbd6027800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fd21c004650 tx=0x7fd21c031a30 comp rx=0 tx=0).stop 2024-06-02T23:13:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 -- 172.21.15.52:0/686307412 shutdown_connections 2024-06-02T23:13:58.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd6022b70 0x55cbd602e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd6027800 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 --2- 172.21.15.52:0/686307412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd601d010 0x55cbd60272c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 -- 172.21.15.52:0/686307412 >> 172.21.15.52:0/686307412 conn(0x55cbd6018680 msgr2=0x55cbd6013560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 -- 172.21.15.52:0/686307412 shutdown_connections 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 -- 172.21.15.52:0/686307412 wait complete. 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.313+0000 7fd2351ab740 1 -- start start 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 Processor -- start 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd602b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd602b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd609b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1ac2a0 con 0x55cbd601da00 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1abbd0 con 0x55cbd601d010 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c1a8e30 con 0x55cbd6022b70 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd2351ab740 10 client.0 _ll_register_callbacks cb 0x7fd220009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd602b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd602b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50062/0 (socket says 172.21.15.52:50062) 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd602b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22b7fe640 1 -- 172.21.15.52:0/1316506681 learned_addr learned my addr 172.21.15.52:0/1316506681 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd609b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22bfff640 1 -- 172.21.15.52:0/1316506681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 msgr2=0x55cbd609b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22bfff640 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd609b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22bfff640 1 -- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 msgr2=0x55cbd602b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22bfff640 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd602b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.314+0000 7fd22bfff640 1 -- 172.21.15.52:0/1316506681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbd602eaa0 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd22bfff640 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd602b9d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55cbd6026320 tx=0x7fd21c031930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db86536831860760 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd20b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd20b7fe640 1 -- 172.21.15.52:0/1316506681 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd21c0055c0 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd20b7fe640 1 -- 172.21.15.52:0/1316506681 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd21c0035f0 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd21c04d070 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd20b7fe640 1 -- 172.21.15.52:0/1316506681 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd21c005120 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd2351ab740 10 client.16206 fetch_fsmap learned FSMap version 30 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cbd609cf10 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd20b7fe640 1 -- 172.21.15.52:0/1316506681 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd21c04d070 con 0x55cbd601da00 2024-06-02T23:13:58.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.315+0000 7fd20b7fe640 1 -- 172.21.15.52:0/1316506681 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd21c054840 con 0x55cbd601da00 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd2351ab740 10 client.16206 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd2351ab740 10 client.16206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd2351ab740 20 client.16206 populate_metadata read hostname 'smithi052' 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd2351ab740 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd609d8e0 0x55cbd609fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd2351ab740 4 client.16206 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cbd5f8dc20 con 0x55cbd609d8e0 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd609d8e0 0x55cbd609fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd20b7fe640 10 client.16206 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:58.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.316+0000 7fd22b7fe640 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd609d8e0 0x55cbd609fd00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bc5ef2790991887 server_cookie=611bcf344d132ce2 in_seq=0 out_seq=0 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd20b7fe640 1 -- 172.21.15.52:0/1316506681 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cbd5f8dc20 con 0x55cbd609d8e0 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd20b7fe640 10 client.16206 handle_command_reply: tid=0 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 client.16206 shutdown 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 msgr2=0x55cbd602b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd602b9d0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x55cbd6026320 tx=0x7fd21c031930 comp rx=0 tx=0).stop 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 shutdown_connections 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cbd609d8e0 0x55cbd609fd00 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cbd6022b70 0x55cbd609b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cbd601da00 0x55cbd602b9d0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd20b7fe640 0 client.16206 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 --2- 172.21.15.52:0/1316506681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cbd601d010 0x55cbd602b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 >> 172.21.15.52:0/1316506681 conn(0x55cbd6018680 msgr2=0x55cbd60272c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 shutdown_connections 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 1 -- 172.21.15.52:0/1316506681 wait complete. 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.337+0000 7fd2351ab740 20 client.16206 trim_cache size 0 max 16384 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:13:58.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:13:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd2140a2c30 msgr2=0x7fd2140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd2140a2c30 0x7fd2140a50f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd2240989a0 tx=0x7fd224066c50 comp rx=0 tx=0).stop 2024-06-02T23:13:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 msgr2=0x7fd22c109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 0x7fd22c109e00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fd22400eb10 tx=0x7fd22400c2d0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 shutdown_connections 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd2140a2c30 0x7fd2140a50f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd22c1024b0 0x7fd22c109e00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd22c101af0 0x7fd22c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 --2- 172.21.15.52:0/2877926392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd22c100900 0x7fd22c10b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 >> 172.21.15.52:0/2877926392 conn(0x7fd22c0fbe70 msgr2=0x7fd22c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 shutdown_connections 2024-06-02T23:13:58.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:13:58.338+0000 7fd23249e640 1 -- 172.21.15.52:0/2877926392 wait complete. 2024-06-02T23:13:58.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.459+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc30069180 con 0x7efc30102070 2024-06-02T23:13:58.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.460+0000 7efc26ffd700 1 -- 172.21.15.52:0/2555280829 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7efc30069180 con 0x7efc30102070 2024-06-02T23:13:58.460 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:13:58.461 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc1c0a2c20 msgr2=0x7efc1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc1c0a2c20 0x7efc1c0a50e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efc2c007b60 tx=0x7efc2c0073d0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 msgr2=0x7efc30103430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 0x7efc30103430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efc2000e9a0 tx=0x7efc2000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 shutdown_connections 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc1c0a2c20 0x7efc1c0a50e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc30107500 0x7efc30103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc301016b0 0x7efc30102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 --2- 172.21.15.52:0/2555280829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc30102070 0x7efc30103430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 >> 172.21.15.52:0/2555280829 conn(0x7efc300780f0 msgr2=0x7efc30100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 shutdown_connections 2024-06-02T23:13:58.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:13:58.462+0000 7efc380d2700 1 -- 172.21.15.52:0/2555280829 wait complete. 2024-06-02T23:13:58.464 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:13:58.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 32373 2.88 MB/sec warmup 67 sec latency 45.708 ms 2024-06-02T23:13:59.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:59 smithi052 ceph-mon[32340]: pgmap v494: 97 pgs: 97 active+clean; 33 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 446 KiB/s wr, 66 op/s 2024-06-02T23:13:59.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:13:59 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2555280829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:59.180 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:13:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:59 smithi174 ceph-mon[35994]: pgmap v494: 97 pgs: 97 active+clean; 33 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 446 KiB/s wr, 66 op/s 2024-06-02T23:13:59.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:13:59 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2555280829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:59.452 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:13:59.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:59 smithi123 ceph-mon[35977]: pgmap v494: 97 pgs: 97 active+clean; 33 MiB data, 467 MiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 446 KiB/s wr, 66 op/s 2024-06-02T23:13:59.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:13:59 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2555280829' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:13:59.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 32702 2.84 MB/sec warmup 68 sec latency 15.190 ms 2024-06-02T23:14:00.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 33228 2.83 MB/sec warmup 69 sec latency 14.509 ms 2024-06-02T23:14:01.276 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:01 smithi052 ceph-mon[32340]: pgmap v495: 97 pgs: 97 active+clean; 33 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 563 KiB/s wr, 99 op/s 2024-06-02T23:14:01.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:01 smithi174 ceph-mon[35994]: pgmap v495: 97 pgs: 97 active+clean; 33 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 563 KiB/s wr, 99 op/s 2024-06-02T23:14:01.449 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:14:01.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 -- 172.21.15.52:0/2158690705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 msgr2=0x7feb0810f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 --2- 172.21.15.52:0/2158690705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb0810f360 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7feb0000b0a0 tx=0x7feb0000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 -- 172.21.15.52:0/2158690705 shutdown_connections 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 --2- 172.21.15.52:0/2158690705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb0810f360 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 --2- 172.21.15.52:0/2158690705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08108520 0x7feb08108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 --2- 172.21.15.52:0/2158690705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 0x7feb08102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 -- 172.21.15.52:0/2158690705 >> 172.21.15.52:0/2158690705 conn(0x7feb08079560 msgr2=0x7feb08079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 -- 172.21.15.52:0/2158690705 shutdown_connections 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 -- 172.21.15.52:0/2158690705 wait complete. 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.276+0000 7feb0e659700 1 Processor -- start 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 -- start start 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 0x7feb08198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb081990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08108520 0x7feb0819d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb08112650 con 0x7feb08102ed0 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb081124d0 con 0x7feb08108520 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0e659700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb081127d0 con 0x7feb08102510 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb081990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb07fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 0x7feb08198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb081990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47520/0 (socket says 172.21.15.52:47520) 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feafffff700 1 -- 172.21.15.52:0/2870903446 learned_addr learned my addr 172.21.15.52:0/2870903446 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08108520 0x7feb0819d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb07fff700 1 -- 172.21.15.52:0/2870903446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08108520 msgr2=0x7feb0819d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb07fff700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08108520 0x7feb0819d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb07fff700 1 -- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 msgr2=0x7feb081990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb07fff700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb081990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feb07fff700 1 -- 172.21.15.52:0/2870903446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0819db10 con 0x7feb08102510 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.277+0000 7feafffff700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb081990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.278+0000 7feb07fff700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 0x7feb08198b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009c50 tx=0x7feaf8009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.278+0000 7feb05ffb700 1 -- 172.21.15.52:0/2870903446 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf801e070 con 0x7feb08102510 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.278+0000 7feb05ffb700 1 -- 172.21.15.52:0/2870903446 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feaf8019410 con 0x7feb08102510 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.278+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0819de00 con 0x7feb08102510 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.278+0000 7feb05ffb700 1 -- 172.21.15.52:0/2870903446 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf8018610 con 0x7feb08102510 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.278+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb080746c0 con 0x7feb08102510 2024-06-02T23:14:01.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.281+0000 7feb05ffb700 1 -- 172.21.15.52:0/2870903446 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feaf8021070 con 0x7feb08102510 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.281+0000 7feb05ffb700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feaf40a2c20 0x7feaf40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.281+0000 7feafffff700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feaf40a2c20 0x7feaf40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.281+0000 7feb05ffb700 1 -- 172.21.15.52:0/2870903446 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feaf80d2480 con 0x7feb08102510 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.282+0000 7feafffff700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feaf40a2c20 0x7feaf40a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb08079a60 tx=0x7feaf0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.282+0000 7feb18edcb80 1 Processor -- start 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.282+0000 7feb18edcb80 1 -- start start 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.282+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628640 0x564fbc628a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628f60 0x564fbc6229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc623340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb18edcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564fbc574810 con 0x564fbc628f60 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb18edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fbc51c720 con 0x564fbc622f20 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb18edcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564fbc5707d0 con 0x564fbc628640 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628f60 0x564fbc6229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628f60 0x564fbc6229e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47532/0 (socket says 172.21.15.52:47532) 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feafffff700 1 -- 172.21.15.52:0/2355080708 learned_addr learned my addr 172.21.15.52:0/2355080708 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb07fff700 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628640 0x564fbc628a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc623340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2355080708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628640 msgr2=0x564fbc628a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628640 0x564fbc628a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2355080708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628f60 msgr2=0x564fbc6229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628f60 0x564fbc6229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2355080708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fbc512f50 con 0x564fbc622f20 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc623340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d010 tx=0x7feb00009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51023be58f4d1a88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feafe7fc700 1 -- 172.21.15.52:0/2355080708 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0000bab0 con 0x564fbc622f20 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feafe7fc700 1 -- 172.21.15.52:0/2355080708 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb0000bc50 con 0x564fbc622f20 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.283+0000 7feafe7fc700 1 -- 172.21.15.52:0/2355080708 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00007d10 con 0x564fbc622f20 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 -- 172.21.15.52:0/2355080708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 msgr2=0x564fbc623340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc623340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feb0000d010 tx=0x7feb00009df0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 -- 172.21.15.52:0/2355080708 shutdown_connections 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc623340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628640 0x564fbc628a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2355080708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628f60 0x564fbc6229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 -- 172.21.15.52:0/2355080708 >> 172.21.15.52:0/2355080708 conn(0x564fbc54dcc0 msgr2=0x564fbc61d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 -- 172.21.15.52:0/2355080708 shutdown_connections 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.284+0000 7feb18edcb80 1 -- 172.21.15.52:0/2355080708 wait complete. 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 -- start start 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 Processor -- start 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc62ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc62c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564fbc62f970 con 0x564fbc628640 2024-06-02T23:14:01.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564fbc514bf0 con 0x564fbc622f20 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564fbc62faf0 con 0x564fbc628f60 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feb18edcb80 10 client.0 _ll_register_callbacks cb 0x564fbc50e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.285+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47544/0 (socket says 172.21.15.52:47544) 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 -- 172.21.15.52:0/3500430636 learned_addr learned my addr 172.21.15.52:0/3500430636 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feb07fff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc62ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc62c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 -- 172.21.15.52:0/3500430636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 msgr2=0x564fbc62c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc62c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 -- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 msgr2=0x564fbc62ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc62ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 -- 172.21.15.52:0/3500430636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fbc4c38d0 con 0x564fbc628640 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feb07fff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc62ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feafffff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62bfd0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7feaf000eda0 tx=0x7feaf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9792a5c1db65fae server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feae77fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feae77fe700 1 -- 172.21.15.52:0/3500430636 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf001dbb0 con 0x564fbc628640 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb0000bc50 con 0x564fbc628640 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feae77fe700 1 -- 172.21.15.52:0/3500430636 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feaf0009040 con 0x564fbc628640 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.286+0000 7feae77fe700 1 -- 172.21.15.52:0/3500430636 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feaf001dd50 con 0x564fbc628640 2024-06-02T23:14:01.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 10 client.16218 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fbc63a260 con 0x564fbc628640 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feae77fe700 1 -- 172.21.15.52:0/3500430636 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb0000bc50 con 0x564fbc628640 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feae77fe700 1 -- 172.21.15.52:0/3500430636 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7feaf00274b0 con 0x564fbc628640 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 10 client.16218 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 10 client.16218 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 20 client.16218 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc63c8a0 0x564fbc63ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 4 client.16218 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fbc581d10 con 0x564fbc63c8a0 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.287+0000 7feb07fff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc63c8a0 0x564fbc63ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.288+0000 7feb07fff700 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc63c8a0 0x564fbc63ecb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dbb3142ea51b546f server_cookie=aef7e362ed062448 in_seq=0 out_seq=0 2024-06-02T23:14:01.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.288+0000 7feae77fe700 10 client.16218 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feae77fe700 1 -- 172.21.15.52:0/3500430636 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564fbc581d10 con 0x564fbc63c8a0 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feae77fe700 10 client.16218 handle_command_reply: tid=0 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 client.16218 shutdown 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 msgr2=0x564fbc62bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62bfd0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7feaf000eda0 tx=0x7feaf00047c0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 shutdown_connections 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc62c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc63c8a0 0x564fbc63ecb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc622f20 0x564fbc62ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 --2- 172.21.15.52:0/3500430636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62bfd0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 >> 172.21.15.52:0/3500430636 conn(0x564fbc54dcc0 msgr2=0x564fbc6219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feae77fe700 0 client.16218 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 shutdown_connections 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 1 -- 172.21.15.52:0/3500430636 wait complete. 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.289+0000 7feb18edcb80 20 client.16218 trim_cache size 0 max 16384 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 Processor -- start 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 -- start start 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc622f20 0x564fbc6229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 0x564fbc62a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb081aa760 con 0x564fbc628640 2024-06-02T23:14:01.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb081ac330 con 0x564fbc628f60 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb18edcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb081aae30 con 0x564fbc622f20 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb0cbf6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 0x564fbc62a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feafffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb0cbf6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 0x564fbc62a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50112/0 (socket says 172.21.15.52:50112) 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2677714169 learned_addr learned my addr 172.21.15.52:0/2677714169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.306+0000 7feb07fff700 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc622f20 0x564fbc6229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2677714169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc622f20 msgr2=0x564fbc6229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc622f20 0x564fbc6229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2677714169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 msgr2=0x564fbc62ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feb0cbf6700 1 -- 172.21.15.52:0/2677714169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fbc62a540 con 0x564fbc628f60 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 0x564fbc62a8c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb00024000 tx=0x7feb000078b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=355b167c08d6addf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feafdffb700 1 -- 172.21.15.52:0/2677714169 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb0002d070 con 0x564fbc628f60 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feafdffb700 1 -- 172.21.15.52:0/2677714169 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb00003a00 con 0x564fbc628f60 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.307+0000 7feafdffb700 1 -- 172.21.15.52:0/2677714169 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb00007d20 con 0x564fbc628f60 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 -- 172.21.15.52:0/2677714169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 msgr2=0x564fbc62a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 0x564fbc62a8c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feb00024000 tx=0x7feb000078b0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 -- 172.21.15.52:0/2677714169 shutdown_connections 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628f60 0x564fbc62a8c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc622f20 0x564fbc6229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2677714169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc628640 0x564fbc62ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 -- 172.21.15.52:0/2677714169 >> 172.21.15.52:0/2677714169 conn(0x564fbc54dcc0 msgr2=0x564fbc61d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 -- 172.21.15.52:0/2677714169 shutdown_connections 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 -- 172.21.15.52:0/2677714169 wait complete. 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.308+0000 7feb18edcb80 1 -- start start 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 Processor -- start 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 0x564fbc634db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628640 0x564fbc6352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc643570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb081abbd0 con 0x564fbc622f20 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb081ab500 con 0x564fbc628640 2024-06-02T23:14:01.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb081a8700 con 0x564fbc628f60 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb18edcb80 10 client.0 _ll_register_callbacks cb 0x564fbc50e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 0x564fbc634db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 0x564fbc634db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47574/0 (socket says 172.21.15.52:47574) 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 -- 172.21.15.52:0/2842764763 learned_addr learned my addr 172.21.15.52:0/2842764763 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feafffff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628640 0x564fbc6352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb0cbf6700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc643570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 -- 172.21.15.52:0/2842764763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 msgr2=0x564fbc643570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc643570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 -- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628640 msgr2=0x564fbc6352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628640 0x564fbc6352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feb07fff700 1 -- 172.21.15.52:0/2842764763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fbc6358e0 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.309+0000 7feafffff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628640 0x564fbc6352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb07fff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 0x564fbc634db0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009fd0 tx=0x7feaf8019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0c354cf0ce1c3e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feae77fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feae77fe700 1 -- 172.21.15.52:0/2842764763 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf809ed10 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feae77fe700 1 -- 172.21.15.52:0/2842764763 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feaf80d20b0 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb00003a00 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feae77fe700 1 -- 172.21.15.52:0/2842764763 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7feaf80ac050 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 10 client.16224 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564fbc6a6a90 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feae77fe700 1 -- 172.21.15.52:0/2842764763 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb00003a00 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feae77fe700 1 -- 172.21.15.52:0/2842764763 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7feaf80a7bd0 con 0x564fbc622f20 2024-06-02T23:14:01.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 10 client.16224 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 10 client.16224 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 20 client.16224 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc6a7140 0x564fbc6a9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 4 client.16224 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.310+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fbc57c290 con 0x564fbc6a7140 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feafffff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc6a7140 0x564fbc6a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feae77fe700 10 client.16224 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feafffff700 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc6a7140 0x564fbc6a9560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ac1517b8093ec0e4 server_cookie=fbef38943d2140d in_seq=0 out_seq=0 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feae77fe700 1 -- 172.21.15.52:0/2842764763 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fbc57c290 con 0x564fbc6a7140 2024-06-02T23:14:01.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feae77fe700 10 client.16224 handle_command_reply: tid=0 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feb18edcb80 1 client.16224 shutdown 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 msgr2=0x564fbc634db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 0x564fbc634db0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009fd0 tx=0x7feaf8019b40 comp rx=0 tx=0).stop 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 shutdown_connections 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.311+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564fbc628f60 0x564fbc643570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564fbc622f20 0x564fbc634db0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x564fbc6a7140 0x564fbc6a9560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 1 --2- 172.21.15.52:0/2842764763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564fbc628640 0x564fbc6352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 >> 172.21.15.52:0/2842764763 conn(0x564fbc54dcc0 msgr2=0x564fbc62a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feae77fe700 0 client.16224 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:14:01.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 shutdown_connections 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 1 -- 172.21.15.52:0/2842764763 wait complete. 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb18edcb80 20 client.16224 trim_cache size 0 max 16384 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feaf40a2c20 msgr2=0x7feaf40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feaf40a2c20 0x7feaf40a50e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7feb08079a60 tx=0x7feaf0009260 comp rx=0 tx=0).stop 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 msgr2=0x7feb08198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 0x7feb08198b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feaf8009c50 tx=0x7feaf8009f60 comp rx=0 tx=0).stop 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 shutdown_connections 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feaf40a2c20 0x7feaf40a50e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb08108520 0x7feb0819d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb08102510 0x7feb08198b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 --2- 172.21.15.52:0/2870903446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb08102ed0 0x7feb081990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:01.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.312+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 >> 172.21.15.52:0/2870903446 conn(0x7feb08079560 msgr2=0x7feb08101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:01.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.313+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 shutdown_connections 2024-06-02T23:14:01.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:01.313+0000 7feb0e659700 1 -- 172.21.15.52:0/2870903446 wait complete. 2024-06-02T23:14:01.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:01 smithi123 ceph-mon[35977]: pgmap v495: 97 pgs: 97 active+clean; 33 MiB data, 468 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 563 KiB/s wr, 99 op/s 2024-06-02T23:14:01.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 33856 2.83 MB/sec warmup 70 sec latency 15.092 ms 2024-06-02T23:14:02.218 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:14:02.507 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:02.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 34420 2.82 MB/sec warmup 71 sec latency 15.528 ms 2024-06-02T23:14:03.363 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:03 smithi052 ceph-mon[32340]: pgmap v496: 97 pgs: 97 active+clean; 35 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 860 KiB/s wr, 103 op/s 2024-06-02T23:14:03.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:03 smithi174 ceph-mon[35994]: pgmap v496: 97 pgs: 97 active+clean; 35 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 860 KiB/s wr, 103 op/s 2024-06-02T23:14:03.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:03 smithi123 ceph-mon[35977]: pgmap v496: 97 pgs: 97 active+clean; 35 MiB data, 470 MiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 860 KiB/s wr, 103 op/s 2024-06-02T23:14:03.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 35227 2.88 MB/sec warmup 72 sec latency 15.916 ms 2024-06-02T23:14:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 -- 172.21.15.52:0/1493035613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c10a6d0 msgr2=0x7f190c10aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 --2- 172.21.15.52:0/1493035613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c10a6d0 0x7f190c10aab0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f18fc009990 tx=0x7f18fc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 -- 172.21.15.52:0/1493035613 shutdown_connections 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 --2- 172.21.15.52:0/1493035613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c105090 0x7f190c1115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 --2- 172.21.15.52:0/1493035613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c10a6d0 0x7f190c10aab0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 --2- 172.21.15.52:0/1493035613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c1046d0 0x7f190c104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 -- 172.21.15.52:0/1493035613 >> 172.21.15.52:0/1493035613 conn(0x7f190c1001c0 msgr2=0x7f190c1025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 -- 172.21.15.52:0/1493035613 shutdown_connections 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.348+0000 7f1911f6f700 1 -- 172.21.15.52:0/1493035613 wait complete. 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 Processor -- start 2024-06-02T23:14:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 -- start start 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c1046d0 0x7f190c19ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 0x7f190c19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c10a6d0 0x7f190c19f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1148b0 con 0x7f190c105090 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f190c114730 con 0x7f190c1046d0 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f1911f6f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f190c114a30 con 0x7f190c10a6d0 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f190affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 0x7f190c19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f190affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 0x7f190c19b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50984/0 (socket says 172.21.15.52:50984) 2024-06-02T23:14:04.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f190affd700 1 -- 172.21.15.52:0/4028447416 learned_addr learned my addr 172.21.15.52:0/4028447416 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c1046d0 0x7f190c19ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.349+0000 7f190bfff700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c10a6d0 0x7f190c19f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190affd700 1 -- 172.21.15.52:0/4028447416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c10a6d0 msgr2=0x7f190c19f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190affd700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c10a6d0 0x7f190c19f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190affd700 1 -- 172.21.15.52:0/4028447416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c1046d0 msgr2=0x7f190c19ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190affd700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c1046d0 0x7f190c19ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190affd700 1 -- 172.21.15.52:0/4028447416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f190c19fd70 con 0x7f190c105090 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c1046d0 0x7f190c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190bfff700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c10a6d0 0x7f190c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f190affd700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 0x7f190c19b2a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9a0 tx=0x7f190000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f1908ff9700 1 -- 172.21.15.52:0/4028447416 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000ca10 con 0x7f190c105090 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f190c1a0060 con 0x7f190c105090 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f1908ff9700 1 -- 172.21.15.52:0/4028447416 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f190000cbb0 con 0x7f190c105090 2024-06-02T23:14:04.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f1908ff9700 1 -- 172.21.15.52:0/4028447416 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900018680 con 0x7f190c105090 2024-06-02T23:14:04.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.350+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f190c1a78e0 con 0x7f190c105090 2024-06-02T23:14:04.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.352+0000 7f1908ff9700 1 -- 172.21.15.52:0/4028447416 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1900010430 con 0x7f190c105090 2024-06-02T23:14:04.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.353+0000 7f1908ff9700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18f40a2c20 0x7f18f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.353+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18f40a2c20 0x7f18f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.353+0000 7f1908ff9700 1 -- 172.21.15.52:0/4028447416 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19000cf210 con 0x7f190c105090 2024-06-02T23:14:04.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18f40a2c20 0x7f18f40a50e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b2e0 tx=0x7f18fc01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:04.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 Processor -- start 2024-06-02T23:14:04.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 -- start start 2024-06-02T23:14:04.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb85340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55792fad6810 con 0x55792fb8a640 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55792fa7e720 con 0x55792fb84f20 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.354+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55792fad27d0 con 0x55792fb8af60 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50988/0 (socket says 172.21.15.52:50988) 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 -- 172.21.15.52:0/4086648252 learned_addr learned my addr 172.21.15.52:0/4086648252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190bfff700 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb85340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190affd700 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 -- 172.21.15.52:0/4086648252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 msgr2=0x55792fb849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 -- 172.21.15.52:0/4086648252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 msgr2=0x55792fb85340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb85340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190b7fe700 1 -- 172.21.15.52:0/4086648252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55792fa74f50 con 0x55792fb8a640 2024-06-02T23:14:04.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.355+0000 7f190bfff700 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.356+0000 7f190b7fe700 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8aa20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13b746c87c6e608 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.356+0000 7f18f1ffb700 1 -- 172.21.15.52:0/4086648252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc003680 con 0x55792fb8a640 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.356+0000 7f18f1ffb700 1 -- 172.21.15.52:0/4086648252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18fc003cd0 con 0x55792fb8a640 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.356+0000 7f18f1ffb700 1 -- 172.21.15.52:0/4086648252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc00fd30 con 0x55792fb8a640 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 -- 172.21.15.52:0/4086648252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 msgr2=0x55792fb8aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8aa20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f18fc000c00 tx=0x7f18fc01f000 comp rx=0 tx=0).stop 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 -- 172.21.15.52:0/4086648252 shutdown_connections 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb85340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8aa20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/4086648252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 -- 172.21.15.52:0/4086648252 >> 172.21.15.52:0/4086648252 conn(0x55792faafcc0 msgr2=0x55792fb7f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:04.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 -- 172.21.15.52:0/4086648252 shutdown_connections 2024-06-02T23:14:04.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.357+0000 7f191c7f2b80 1 -- 172.21.15.52:0/4086648252 wait complete. 2024-06-02T23:14:04.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 -- start start 2024-06-02T23:14:04.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 Processor -- start 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55792fb91970 con 0x55792fb8a640 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55792fa76bf0 con 0x55792fb84f20 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55792fb91af0 con 0x55792fb8af60 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f191c7f2b80 10 client.0 _ll_register_callbacks cb 0x55792fa70920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:04.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f190affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f190affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50992/0 (socket says 172.21.15.52:50992) 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f190affd700 1 -- 172.21.15.52:0/3996450083 learned_addr learned my addr 172.21.15.52:0/3996450083 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb90cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56182/0 (socket says 172.21.15.52:56182) 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.358+0000 7f190bfff700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190b7fe700 1 -- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 msgr2=0x55792fb8e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190b7fe700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb8e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190b7fe700 1 -- 172.21.15.52:0/3996450083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 msgr2=0x55792fb8dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190b7fe700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190b7fe700 1 -- 172.21.15.52:0/3996450083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55792fa258d0 con 0x55792fb84f20 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190affd700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f190b7fe700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb90cb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006010 tx=0x7f18fc00f8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb991e4414c3a184 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f18eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:04.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f18eaffd700 1 -- 172.21.15.52:0/3996450083 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc037070 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f18eaffd700 1 -- 172.21.15.52:0/3996450083 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f18fc003680 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.359+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18fc003cd0 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f18eaffd700 1 -- 172.21.15.52:0/3996450083 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18fc043050 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 10 client.25814 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55792fb91970 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f18eaffd700 1 -- 172.21.15.52:0/3996450083 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f18fc003cd0 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f18eaffd700 1 -- 172.21.15.52:0/3996450083 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f18fc024c20 con 0x55792fb84f20 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 10 client.25814 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 10 client.25814 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 20 client.25814 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fba0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 4 client.25814 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55792fae3d10 con 0x55792fb9e580 2024-06-02T23:14:04.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.360+0000 7f190affd700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fba0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.361+0000 7f18eaffd700 10 client.25814 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:14:04.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.361+0000 7f190affd700 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fba0970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cb95ec8f9167314c server_cookie=56219242938dadab in_seq=0 out_seq=0 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f18eaffd700 1 -- 172.21.15.52:0/3996450083 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55792fae3d10 con 0x55792fb9e580 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f18eaffd700 10 client.25814 handle_command_reply: tid=0 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 client.25814 shutdown 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 msgr2=0x55792fb90cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb90cb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006010 tx=0x7f18fc00f8d0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 shutdown_connections 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fba0970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f18eaffd700 0 client.25814 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb84f20 0x55792fb90cb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/3996450083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb8a640 0x55792fb8dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 >> 172.21.15.52:0/3996450083 conn(0x55792faafcc0 msgr2=0x55792fb839d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:04.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 shutdown_connections 2024-06-02T23:14:04.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 1 -- 172.21.15.52:0/3996450083 wait complete. 2024-06-02T23:14:04.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.362+0000 7f191c7f2b80 20 client.25814 trim_cache size 0 max 16384 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.379+0000 7f191c7f2b80 1 Processor -- start 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.379+0000 7f191c7f2b80 1 -- start start 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb849e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb79510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb89150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1aebb0 con 0x55792fb84f20 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b0780 con 0x55792fb8a640 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1af280 con 0x55792fb8af60 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb849e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb849e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51008/0 (socket says 172.21.15.52:51008) 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb79510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190b7fe700 1 -- 172.21.15.52:0/1680057084 learned_addr learned my addr 172.21.15.52:0/1680057084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb89150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 -- 172.21.15.52:0/1680057084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 msgr2=0x55792fb79510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb79510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 -- 172.21.15.52:0/1680057084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 msgr2=0x55792fb849e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb849e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 -- 172.21.15.52:0/1680057084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55792fb80320 con 0x55792fb8af60 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190b7fe700 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190affd700 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb79510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.380+0000 7f190bfff700 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb89150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f190400e9d0 tx=0x7f1904009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6756253ef42c457 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f18f17fa700 1 -- 172.21.15.52:0/1680057084 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904004db0 con 0x55792fb8af60 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f18f17fa700 1 -- 172.21.15.52:0/1680057084 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19040093a0 con 0x55792fb8af60 2024-06-02T23:14:04.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f18f17fa700 1 -- 172.21.15.52:0/1680057084 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1904007b30 con 0x55792fb8af60 2024-06-02T23:14:04.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 -- 172.21.15.52:0/1680057084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 msgr2=0x55792fb89150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb89150 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f190400e9d0 tx=0x7f1904009920 comp rx=0 tx=0).stop 2024-06-02T23:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 -- 172.21.15.52:0/1680057084 shutdown_connections 2024-06-02T23:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fb89150 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb849e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/1680057084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb79510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 -- 172.21.15.52:0/1680057084 >> 172.21.15.52:0/1680057084 conn(0x55792faafcc0 msgr2=0x55792fb78e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:04.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 -- 172.21.15.52:0/1680057084 shutdown_connections 2024-06-02T23:14:04.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.381+0000 7f191c7f2b80 1 -- 172.21.15.52:0/1680057084 wait complete. 2024-06-02T23:14:04.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 -- start start 2024-06-02T23:14:04.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 Processor -- start 2024-06-02T23:14:04.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb96fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb97500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fba5360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1b0020 con 0x55792fb84f20 2024-06-02T23:14:04.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1af950 con 0x55792fb8a640 2024-06-02T23:14:04.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f190c1acb70 con 0x55792fb8af60 2024-06-02T23:14:04.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f191c7f2b80 10 client.0 _ll_register_callbacks cb 0x55792fa70920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:04.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb96fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f190b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb96fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51018/0 (socket says 172.21.15.52:51018) 2024-06-02T23:14:04.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f190b7fe700 1 -- 172.21.15.52:0/2890332964 learned_addr learned my addr 172.21.15.52:0/2890332964 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:04.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.382+0000 7f190affd700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb97500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190bfff700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fba5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190b7fe700 1 -- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 msgr2=0x55792fba5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190b7fe700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fba5360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190b7fe700 1 -- 172.21.15.52:0/2890332964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 msgr2=0x55792fb97500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190b7fe700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb97500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190b7fe700 1 -- 172.21.15.52:0/2890332964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55792fba5ba0 con 0x55792fb84f20 2024-06-02T23:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f190b7fe700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb96fc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006010 tx=0x7f18fc00ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcc96548a10e70f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f18eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f18eaffd700 1 -- 172.21.15.52:0/2890332964 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18fc037070 con 0x55792fb84f20 2024-06-02T23:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f18eaffd700 1 -- 172.21.15.52:0/2890332964 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f18fc026ce0 con 0x55792fb84f20 2024-06-02T23:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19040093a0 con 0x55792fb84f20 2024-06-02T23:14:04.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f18eaffd700 1 -- 172.21.15.52:0/2890332964 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f18fc026850 con 0x55792fb84f20 2024-06-02T23:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f191c7f2b80 10 client.16242 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.383+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55792fc08a20 con 0x55792fb84f20 2024-06-02T23:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f18eaffd700 1 -- 172.21.15.52:0/2890332964 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19040093a0 con 0x55792fb84f20 2024-06-02T23:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f18eaffd700 1 -- 172.21.15.52:0/2890332964 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f18fc024670 con 0x55792fb84f20 2024-06-02T23:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f191c7f2b80 10 client.16242 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:04.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f191c7f2b80 10 client.16242 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f191c7f2b80 20 client.16242 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fc0b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f191c7f2b80 4 client.16242 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55792fade290 con 0x55792fb9e580 2024-06-02T23:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.384+0000 7f190affd700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fc0b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:04.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f18eaffd700 10 client.16242 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:14:04.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f190affd700 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fc0b0e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4d2f92e3a82a035 server_cookie=10c74811d37c5e11 in_seq=0 out_seq=0 2024-06-02T23:14:04.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f18eaffd700 1 -- 172.21.15.52:0/2890332964 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55792fade290 con 0x55792fb9e580 2024-06-02T23:14:04.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f18eaffd700 10 client.16242 handle_command_reply: tid=0 2024-06-02T23:14:04.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 client.16242 shutdown 2024-06-02T23:14:04.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 msgr2=0x55792fb96fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb96fc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f18fc006010 tx=0x7f18fc00ffb0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 shutdown_connections 2024-06-02T23:14:04.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55792fb9e580 0x55792fc0b0e0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f18eaffd700 0 client.16242 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:14:04.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55792fb8af60 0x55792fba5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55792fb84f20 0x55792fb96fc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 --2- 172.21.15.52:0/2890332964 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55792fb8a640 0x55792fb97500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 >> 172.21.15.52:0/2890332964 conn(0x55792faafcc0 msgr2=0x55792fb8c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 shutdown_connections 2024-06-02T23:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.385+0000 7f191c7f2b80 1 -- 172.21.15.52:0/2890332964 wait complete. 2024-06-02T23:14:04.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f191c7f2b80 20 client.16242 trim_cache size 0 max 16384 2024-06-02T23:14:04.400 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18f40a2c20 msgr2=0x7f18f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18f40a2c20 0x7f18f40a50e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f18fc00b2e0 tx=0x7f18fc01a040 comp rx=0 tx=0).stop 2024-06-02T23:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 msgr2=0x7f190c19b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 0x7f190c19b2a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f190000e9a0 tx=0x7f190000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 shutdown_connections 2024-06-02T23:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18f40a2c20 0x7f18f40a50e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f190c10a6d0 0x7f190c19f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f190c1046d0 0x7f190c19ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 --2- 172.21.15.52:0/4028447416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f190c105090 0x7f190c19b2a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:04.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 >> 172.21.15.52:0/4028447416 conn(0x7f190c1001c0 msgr2=0x7f190c100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:04.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.386+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 shutdown_connections 2024-06-02T23:14:04.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:04.387+0000 7f1911f6f700 1 -- 172.21.15.52:0/4028447416 wait complete. 2024-06-02T23:14:04.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 35595 2.89 MB/sec warmup 73 sec latency 15.846 ms 2024-06-02T23:14:05.210 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:05 smithi052 ceph-mon[32340]: pgmap v497: 97 pgs: 97 active+clean; 34 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 889 KiB/s wr, 105 op/s 2024-06-02T23:14:05.212 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:14:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:05 smithi174 ceph-mon[35994]: pgmap v497: 97 pgs: 97 active+clean; 34 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 889 KiB/s wr, 105 op/s 2024-06-02T23:14:05.486 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:05.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:05 smithi123 ceph-mon[35977]: pgmap v497: 97 pgs: 97 active+clean; 34 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 889 KiB/s wr, 105 op/s 2024-06-02T23:14:05.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 35895 2.87 MB/sec warmup 74 sec latency 16.678 ms 2024-06-02T23:14:06.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 36229 2.83 MB/sec warmup 75 sec latency 13.574 ms 2024-06-02T23:14:07.247 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:07 smithi052 ceph-mon[32340]: pgmap v498: 97 pgs: 97 active+clean; 35 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-02T23:14:07.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.366+0000 7efe1b3d0700 1 -- 172.21.15.52:0/221260539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 msgr2=0x7efe14101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.366+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/221260539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe14101210 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7efe10009990 tx=0x7efe10009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 -- 172.21.15.52:0/221260539 shutdown_connections 2024-06-02T23:14:07.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/221260539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe141132c0 0x7efe141156b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/221260539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe14101210 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/221260539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe14101750 0x7efe14112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 -- 172.21.15.52:0/221260539 >> 172.21.15.52:0/221260539 conn(0x7efe14076680 msgr2=0x7efe14076a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:07.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 -- 172.21.15.52:0/221260539 shutdown_connections 2024-06-02T23:14:07.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 -- 172.21.15.52:0/221260539 wait complete. 2024-06-02T23:14:07.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.367+0000 7efe1b3d0700 1 Processor -- start 2024-06-02T23:14:07.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 -- start start 2024-06-02T23:14:07.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe1419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe14101750 0x7efe1419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe141132c0 0x7efe141a3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe14118cf0 con 0x7efe14100e30 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe14118b70 con 0x7efe141132c0 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1b3d0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe14118e70 con 0x7efe14101750 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1916c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe1419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1916c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe1419f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51050/0 (socket says 172.21.15.52:51050) 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1916c700 1 -- 172.21.15.52:0/604875726 learned_addr learned my addr 172.21.15.52:0/604875726 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1996d700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe141132c0 0x7efe141a3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1896b700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe14101750 0x7efe1419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1996d700 1 -- 172.21.15.52:0/604875726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe14101750 msgr2=0x7efe1419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1996d700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe14101750 0x7efe1419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1996d700 1 -- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 msgr2=0x7efe1419f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1996d700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe1419f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1996d700 1 -- 172.21.15.52:0/604875726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe141a4160 con 0x7efe141132c0 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.368+0000 7efe1916c700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe1419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.369+0000 7efe1996d700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe141132c0 0x7efe141a3a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe140749f0 tx=0x7efe0c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.369+0000 7efe0a7fc700 1 -- 172.21.15.52:0/604875726 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0092e0 con 0x7efe141132c0 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.369+0000 7efe0a7fc700 1 -- 172.21.15.52:0/604875726 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe0c004bb0 con 0x7efe141132c0 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.369+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe141a43f0 con 0x7efe141132c0 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.369+0000 7efe0a7fc700 1 -- 172.21.15.52:0/604875726 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c004020 con 0x7efe141132c0 2024-06-02T23:14:07.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.369+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe141abcd0 con 0x7efe141132c0 2024-06-02T23:14:07.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.372+0000 7efe0a7fc700 1 -- 172.21.15.52:0/604875726 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efe0c01b030 con 0x7efe141132c0 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.372+0000 7efe0a7fc700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe000a2c20 0x7efe000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.372+0000 7efe1916c700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe000a2c20 0x7efe000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.372+0000 7efe0a7fc700 1 -- 172.21.15.52:0/604875726 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efe0c0cea90 con 0x7efe141132c0 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 Processor -- start 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 -- start start 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe1916c700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe000a2c20 0x7efe000a50e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b2e0 tx=0x7efe1001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb23506f60 0x55bb23507340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23501260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 0x55bb23501bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bb233f3f60 con 0x55bb235017a0 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb23452850 con 0x55bb23507880 2024-06-02T23:14:07.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe25c53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb233ba460 con 0x55bb23506f60 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.373+0000 7efe1996d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 0x55bb23501bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1996d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 0x55bb23501bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51062/0 (socket says 172.21.15.52:51062) 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1996d700 1 -- 172.21.15.52:0/1726225277 learned_addr learned my addr 172.21.15.52:0/1726225277 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1916c700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb23506f60 0x55bb23507340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23501260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 -- 172.21.15.52:0/1726225277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb23506f60 msgr2=0x55bb23507340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb23506f60 0x55bb23507340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 -- 172.21.15.52:0/1726225277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 msgr2=0x55bb23501bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 0x55bb23501bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 -- 172.21.15.52:0/1726225277 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb233a18d0 con 0x55bb23507880 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1916c700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb23506f60 0x55bb23507340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1996d700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 0x55bb23501bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:07.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efe1896b700 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23501260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe04009c80 tx=0x7efe04009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=724866dc95924eb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:07.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efdff7fe700 1 -- 172.21.15.52:0/1726225277 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04011cb0 con 0x55bb23507880 2024-06-02T23:14:07.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efdff7fe700 1 -- 172.21.15.52:0/1726225277 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe04011e50 con 0x55bb23507880 2024-06-02T23:14:07.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.374+0000 7efdff7fe700 1 -- 172.21.15.52:0/1726225277 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe04005240 con 0x55bb23507880 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 -- 172.21.15.52:0/1726225277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 msgr2=0x55bb23501260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23501260 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7efe04009c80 tx=0x7efe04009f90 comp rx=0 tx=0).stop 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 -- 172.21.15.52:0/1726225277 shutdown_connections 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb235017a0 0x55bb23501bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb23506f60 0x55bb23507340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 --2- 172.21.15.52:0/1726225277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23501260 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 -- 172.21.15.52:0/1726225277 >> 172.21.15.52:0/1726225277 conn(0x55bb2342e080 msgr2=0x55bb234fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 -- 172.21.15.52:0/1726225277 shutdown_connections 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.375+0000 7efe25c53b80 1 -- 172.21.15.52:0/1726225277 wait complete. 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 -- start start 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 Processor -- start 2024-06-02T23:14:07.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb2350d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb2350a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb2350aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bb233f2fc0 con 0x55bb23507880 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb2344e810 con 0x55bb23506f60 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bb2350e260 con 0x55bb235017a0 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe25c53b80 10 client.0 _ll_register_callbacks cb 0x55bb233e85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.376+0000 7efe1996d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb2350aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb2350aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51068/0 (socket says 172.21.15.52:51068) 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 -- 172.21.15.52:0/2119218514 learned_addr learned my addr 172.21.15.52:0/2119218514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1896b700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb2350a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1916c700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb2350d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 -- 172.21.15.52:0/2119218514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 msgr2=0x55bb2350d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb2350d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 -- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 msgr2=0x55bb2350a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb2350a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 -- 172.21.15.52:0/2119218514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb23418750 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1896b700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb2350a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe1996d700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb2350aee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efe0c004b80 tx=0x7efe0c09b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01aa1c4e87e084f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efdfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efdfcff9700 1 -- 172.21.15.52:0/2119218514 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0b4060 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efdfcff9700 1 -- 172.21.15.52:0/2119218514 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efe0c09a800 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.377+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe04011e50 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efdfcff9700 1 -- 172.21.15.52:0/2119218514 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe0c09b3b0 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 10 client.16254 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb23518a50 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efdfcff9700 1 -- 172.21.15.52:0/2119218514 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe0c09f760 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efdfcff9700 1 -- 172.21.15.52:0/2119218514 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7efe0c09a270 con 0x55bb23507880 2024-06-02T23:14:07.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 10 client.16254 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 10 client.16254 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 20 client.16254 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb2351b090 0x55bb2351d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 4 client.16254 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efe04011e50 con 0x55bb2351b090 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.378+0000 7efe1916c700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb2351b090 0x55bb2351d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.379+0000 7efe1916c700 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb2351b090 0x55bb2351d480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=58686123f7e2e1b0 server_cookie=64116201d230d51 in_seq=0 out_seq=0 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.379+0000 7efdfcff9700 10 client.16254 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efdfcff9700 1 -- 172.21.15.52:0/2119218514 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7efe04011e50 con 0x55bb2351b090 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efdfcff9700 10 client.16254 handle_command_reply: tid=0 2024-06-02T23:14:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 client.16254 shutdown 2024-06-02T23:14:07.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 msgr2=0x55bb2350aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb2350aee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efe0c004b80 tx=0x7efe0c09b8f0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 shutdown_connections 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb2351b090 0x55bb2351d480 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb2350aee0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb2350d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efdfcff9700 0 client.16254 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 --2- 172.21.15.52:0/2119218514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb2350a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 >> 172.21.15.52:0/2119218514 conn(0x55bb2342e080 msgr2=0x55bb234fb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 shutdown_connections 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 1 -- 172.21.15.52:0/2119218514 wait complete. 2024-06-02T23:14:07.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.380+0000 7efe25c53b80 20 client.16254 trim_cache size 0 max 16384 2024-06-02T23:14:07.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:07 smithi174 ceph-mon[35994]: pgmap v498: 97 pgs: 97 active+clean; 35 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 Processor -- start 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 -- start start 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb234fbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb234fc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb23522cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b3080 con 0x55bb23507880 2024-06-02T23:14:07.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b4c50 con 0x55bb23506f60 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.397+0000 7efe25c53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b3750 con 0x55bb235017a0 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb234fbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb234fbe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36184/0 (socket says 172.21.15.52:36184) 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 -- 172.21.15.52:0/397271384 learned_addr learned my addr 172.21.15.52:0/397271384 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1896b700 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb234fc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1996d700 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb23522cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 -- 172.21.15.52:0/397271384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 msgr2=0x55bb234fc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb234fc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 -- 172.21.15.52:0/397271384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 msgr2=0x55bb23522cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb23522cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 -- 172.21.15.52:0/397271384 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23508f30 con 0x55bb235017a0 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1996d700 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb23522cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:07.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efe1916c700 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb234fbe80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55bb2351dc50 tx=0x7efe10029040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b685d2eb3da042fd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efdfeffd700 1 -- 172.21.15.52:0/397271384 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10003880 con 0x55bb235017a0 2024-06-02T23:14:07.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efdfeffd700 1 -- 172.21.15.52:0/397271384 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe10032440 con 0x55bb235017a0 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.398+0000 7efdfeffd700 1 -- 172.21.15.52:0/397271384 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe10026690 con 0x55bb235017a0 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 -- 172.21.15.52:0/397271384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 msgr2=0x55bb234fbe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb234fbe80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55bb2351dc50 tx=0x7efe10029040 comp rx=0 tx=0).stop 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 -- 172.21.15.52:0/397271384 shutdown_connections 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23507880 0x55bb23522cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb234fbe80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 --2- 172.21.15.52:0/397271384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23506f60 0x55bb234fc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 -- 172.21.15.52:0/397271384 >> 172.21.15.52:0/397271384 conn(0x55bb2342e080 msgr2=0x55bb235092b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 -- 172.21.15.52:0/397271384 shutdown_connections 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.399+0000 7efe25c53b80 1 -- 172.21.15.52:0/397271384 wait complete. 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 -- start start 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 Processor -- start 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb23513af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23506f60 0x55bb23522400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23522940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b44f0 con 0x55bb23506f60 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b3e20 con 0x55bb23507880 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe141b1020 con 0x55bb235017a0 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.400+0000 7efe25c53b80 10 client.0 _ll_register_callbacks cb 0x55bb233e85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1916c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb23513af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1916c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb23513af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36200/0 (socket says 172.21.15.52:36200) 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1916c700 1 -- 172.21.15.52:0/656569221 learned_addr learned my addr 172.21.15.52:0/656569221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23522940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1896b700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23506f60 0x55bb23522400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 -- 172.21.15.52:0/656569221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 msgr2=0x55bb23513af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb23513af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 -- 172.21.15.52:0/656569221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23506f60 msgr2=0x55bb23522400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23506f60 0x55bb23522400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 -- 172.21.15.52:0/656569221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb23522ff0 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1916c700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb23513af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe1996d700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23522940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe0c097d80 tx=0x7efe0c000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37f18643438616b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efdfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efdfcff9700 1 -- 172.21.15.52:0/656569221 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe0c0b4260 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efdfcff9700 1 -- 172.21.15.52:0/656569221 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efe0c0bb0b0 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.401+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe10032440 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efdfcff9700 1 -- 172.21.15.52:0/656569221 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7efe0c0664c0 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 10 client.25838 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bb23585b20 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efdfcff9700 1 -- 172.21.15.52:0/656569221 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efe10032440 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efdfcff9700 1 -- 172.21.15.52:0/656569221 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7efe0c0cca80 con 0x55bb23507880 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 10 client.25838 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 10 client.25838 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:14:07.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 20 client.25838 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:07.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb23586170 0x55bb23588560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:07.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 4 client.25838 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:14:07.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bb2345fd50 con 0x55bb23586170 2024-06-02T23:14:07.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.402+0000 7efe1916c700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb23586170 0x55bb23588560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.403+0000 7efdfcff9700 10 client.25838 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.403+0000 7efe1916c700 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb23586170 0x55bb23588560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2e57b88448c7d6a3 server_cookie=cdfee19df9247576 in_seq=0 out_seq=0 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efdfcff9700 1 -- 172.21.15.52:0/656569221 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bb2345fd50 con 0x55bb23586170 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efdfcff9700 10 client.25838 handle_command_reply: tid=0 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 client.25838 shutdown 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 msgr2=0x55bb23522940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23522940 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7efe0c097d80 tx=0x7efe0c000f40 comp rx=0 tx=0).stop 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 shutdown_connections 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55bb23586170 0x55bb23588560 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23507880 0x55bb23522940 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bb235017a0 0x55bb23513af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efdfcff9700 0 client.25838 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 --2- 172.21.15.52:0/656569221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bb23506f60 0x55bb23522400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 >> 172.21.15.52:0/656569221 conn(0x55bb2342e080 msgr2=0x55bb23507c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 shutdown_connections 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 1 -- 172.21.15.52:0/656569221 wait complete. 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.404+0000 7efe25c53b80 20 client.25838 trim_cache size 0 max 16384 2024-06-02T23:14:07.408 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe000a2c20 msgr2=0x7efe000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe000a2c20 0x7efe000a50e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7efe1000b2e0 tx=0x7efe1001a040 comp rx=0 tx=0).stop 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe141132c0 msgr2=0x7efe141a3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe141132c0 0x7efe141a3a20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efe140749f0 tx=0x7efe0c00ef90 comp rx=0 tx=0).stop 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 shutdown_connections 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efe000a2c20 0x7efe000a50e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe141132c0 0x7efe141a3a20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe14100e30 0x7efe1419f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 --2- 172.21.15.52:0/604875726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe14101750 0x7efe1419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 >> 172.21.15.52:0/604875726 conn(0x7efe14076680 msgr2=0x7efe14110d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:07.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 shutdown_connections 2024-06-02T23:14:07.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:07.405+0000 7efe1b3d0700 1 -- 172.21.15.52:0/604875726 wait complete. 2024-06-02T23:14:07.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:07 smithi123 ceph-mon[35977]: pgmap v498: 97 pgs: 97 active+clean; 35 MiB data, 521 MiB used, 1.0 TiB / 1.0 TiB avail; 245 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-02T23:14:07.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 36740 2.82 MB/sec warmup 76 sec latency 17.772 ms 2024-06-02T23:14:08.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:14:08.485 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:08.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 37357 2.83 MB/sec warmup 77 sec latency 14.291 ms 2024-06-02T23:14:09.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:09 smithi052 ceph-mon[32340]: pgmap v499: 97 pgs: 97 active+clean; 36 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-06-02T23:14:09.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:09 smithi174 ceph-mon[35994]: pgmap v499: 97 pgs: 97 active+clean; 36 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-06-02T23:14:09.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:09 smithi123 ceph-mon[35977]: pgmap v499: 97 pgs: 97 active+clean; 36 MiB data, 509 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.3 MiB/s wr, 133 op/s 2024-06-02T23:14:09.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 37938 2.81 MB/sec warmup 78 sec latency 38.821 ms 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.435+0000 7fafc4451700 1 -- 172.21.15.52:0/528267263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc107450 msgr2=0x7fafbc107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.435+0000 7fafc4451700 1 --2- 172.21.15.52:0/528267263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc107450 0x7fafbc107830 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009990 tx=0x7fafb8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.435+0000 7fafc4451700 1 -- 172.21.15.52:0/528267263 shutdown_connections 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.435+0000 7fafc4451700 1 --2- 172.21.15.52:0/528267263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafbc102ee0 0x7fafbc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.435+0000 7fafc4451700 1 --2- 172.21.15.52:0/528267263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc107450 0x7fafbc107830 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.436+0000 7fafc4451700 1 --2- 172.21.15.52:0/528267263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc102540 0x7fafbc1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.436+0000 7fafc4451700 1 -- 172.21.15.52:0/528267263 >> 172.21.15.52:0/528267263 conn(0x7fafbc0795f0 msgr2=0x7fafbc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.436+0000 7fafc4451700 1 -- 172.21.15.52:0/528267263 shutdown_connections 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.436+0000 7fafc4451700 1 -- 172.21.15.52:0/528267263 wait complete. 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 Processor -- start 2024-06-02T23:14:10.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 -- start start 2024-06-02T23:14:10.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafbc102540 0x7fafbc19e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 0x7fafbc19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc107450 0x7fafbc198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1126e0 con 0x7fafbc107450 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc112560 con 0x7fafbc102ee0 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc4451700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc112860 con 0x7fafbc102540 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc19ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 0x7fafbc19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc29ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc107450 0x7fafbc198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc29ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc107450 0x7fafbc198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51110/0 (socket says 172.21.15.52:51110) 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.437+0000 7fafc19ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 0x7fafbc19ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56274/0 (socket says 172.21.15.52:56274) 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafbc102540 0x7fafbc19e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc29ee700 1 -- 172.21.15.52:0/1902088224 learned_addr learned my addr 172.21.15.52:0/1902088224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc19ec700 1 -- 172.21.15.52:0/1902088224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafbc102540 msgr2=0x7fafbc19e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc19ec700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafbc102540 0x7fafbc19e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc19ec700 1 -- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc107450 msgr2=0x7fafbc198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc19ec700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc107450 0x7fafbc198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc19ec700 1 -- 172.21.15.52:0/1902088224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafbc199180 con 0x7fafbc102ee0 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc19ec700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 0x7fafbc19ed50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fafac009db0 tx=0x7fafac00c5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafb37fe700 1 -- 172.21.15.52:0/1902088224 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac01f070 con 0x7fafbc102ee0 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.438+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafbc199470 con 0x7fafbc102ee0 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.439+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafbc0746c0 con 0x7fafbc102ee0 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.439+0000 7fafb37fe700 1 -- 172.21.15.52:0/1902088224 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafac00cd20 con 0x7fafbc102ee0 2024-06-02T23:14:10.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.439+0000 7fafb37fe700 1 -- 172.21.15.52:0/1902088224 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafac018600 con 0x7fafbc102ee0 2024-06-02T23:14:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.442+0000 7fafb37fe700 1 -- 172.21.15.52:0/1902088224 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafac022070 con 0x7fafbc102ee0 2024-06-02T23:14:10.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.442+0000 7fafb37fe700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafa80a2ca0 0x7fafa80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.443+0000 7fafb37fe700 1 -- 172.21.15.52:0/1902088224 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fafac0d29f0 con 0x7fafbc102ee0 2024-06-02T23:14:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.443+0000 7fafc21ed700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafa80a2ca0 0x7fafa80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.443+0000 7fafcecd4b80 1 Processor -- start 2024-06-02T23:14:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.443+0000 7fafcecd4b80 1 -- start start 2024-06-02T23:14:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.443+0000 7fafc21ed700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafa80a2ca0 0x7fafa80a5160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fafbc1021d0 tx=0x7fafb801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c53340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c53880 0x55b4a0c4d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.444 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0b3ff60 con 0x55b4a0c52f60 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0b9e850 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0b06460 con 0x55b4a0c53880 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c53340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc29ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c53340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51114/0 (socket says 172.21.15.52:51114) 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc21ed700 1 -- 172.21.15.52:0/272480316 learned_addr learned my addr 172.21.15.52:0/272480316 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc19ec700 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c53880 0x55b4a0c4d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc29ee700 1 -- 172.21.15.52:0/272480316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c53880 msgr2=0x55b4a0c4d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc29ee700 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c53880 0x55b4a0c4d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc29ee700 1 -- 172.21.15.52:0/272480316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 msgr2=0x55b4a0c53340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc29ee700 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c53340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc29ee700 1 -- 172.21.15.52:0/272480316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a0aed8d0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.444+0000 7fafc19ec700 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c53880 0x55b4a0c4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.445+0000 7fafc29ee700 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4dbc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e650 tx=0x7fafb4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ce3d90b31ef682d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:10.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.445+0000 7fafb0d70700 1 -- 172.21.15.52:0/272480316 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4019030 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.445+0000 7fafb0d70700 1 -- 172.21.15.52:0/272480316 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb40092e0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.445+0000 7fafb0d70700 1 -- 172.21.15.52:0/272480316 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb40047d0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 -- 172.21.15.52:0/272480316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 msgr2=0x55b4a0c4dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4dbc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fafb400e650 tx=0x7fafb4009920 comp rx=0 tx=0).stop 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 -- 172.21.15.52:0/272480316 shutdown_connections 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4dbc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c53340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/272480316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c53880 0x55b4a0c4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 -- 172.21.15.52:0/272480316 >> 172.21.15.52:0/272480316 conn(0x55b4a0b7a080 msgr2=0x55b4a0c48220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 -- 172.21.15.52:0/272480316 shutdown_connections 2024-06-02T23:14:10.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.446+0000 7fafcecd4b80 1 -- 172.21.15.52:0/272480316 wait complete. 2024-06-02T23:14:10.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.447+0000 7fafcecd4b80 1 -- start start 2024-06-02T23:14:10.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.447+0000 7fafcecd4b80 1 Processor -- start 2024-06-02T23:14:10.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.447+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c595d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.447+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c53880 0x55b4a0c56e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56296/0 (socket says 172.21.15.52:56296) 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc29ee700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c53880 0x55b4a0c56e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 -- 172.21.15.52:0/3642494132 learned_addr learned my addr 172.21.15.52:0/3642494132 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0b3efc0 con 0x55b4a0c53880 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0b9a810 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b4a0c5a260 con 0x55b4a0c52f60 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc19ec700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafcecd4b80 10 client.0 _ll_register_callbacks cb 0x55b4a0b345b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:10.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 -- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 msgr2=0x55b4a0c568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 -- 172.21.15.52:0/3642494132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c53880 msgr2=0x55b4a0c56e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c53880 0x55b4a0c56e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.448+0000 7fafc21ed700 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a0b64750 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafc21ed700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c595d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafb8000c00 tx=0x7fafb801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52170c5d8626029 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafa5ffb700 1 -- 172.21.15.52:0/3642494132 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb8003a90 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafa5ffb700 1 -- 172.21.15.52:0/3642494132 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fafb8003c30 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb40092e0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafa5ffb700 1 -- 172.21.15.52:0/3642494132 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fafb8003e30 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafcecd4b80 10 client.25850 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:10.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.449+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4a0c649c0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.450+0000 7fafa5ffb700 1 -- 172.21.15.52:0/3642494132 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb40092e0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.450+0000 7fafa5ffb700 1 -- 172.21.15.52:0/3642494132 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fafb8032550 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.450+0000 7fafcecd4b80 10 client.25850 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.450+0000 7fafcecd4b80 10 client.25850 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.450+0000 7fafcecd4b80 20 client.25850 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.450+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0c673d0 0x55b4a0c69810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.451+0000 7fafcecd4b80 4 client.25850 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.451+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b4a0babd50 con 0x55b4a0c673d0 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.451+0000 7fafc19ec700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0c673d0 0x55b4a0c69810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.451+0000 7fafa5ffb700 10 client.25850 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.451+0000 7fafc19ec700 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0c673d0 0x55b4a0c69810 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c5da2661d2b6581e server_cookie=e330a2e08f6bb192 in_seq=0 out_seq=0 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.452+0000 7fafa5ffb700 1 -- 172.21.15.52:0/3642494132 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b4a0babd50 con 0x55b4a0c673d0 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.452+0000 7fafa5ffb700 10 client.25850 handle_command_reply: tid=0 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.452+0000 7fafcecd4b80 1 client.25850 shutdown 2024-06-02T23:14:10.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.452+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 msgr2=0x55b4a0c595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.452+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c595d0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fafb8000c00 tx=0x7fafb801f000 comp rx=0 tx=0).stop 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 shutdown_connections 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0c673d0 0x55b4a0c69810 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c53880 0x55b4a0c56e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafa5ffb700 0 client.25850 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c595d0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/3642494132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 >> 172.21.15.52:0/3642494132 conn(0x55b4a0b7a080 msgr2=0x55b4a0c47700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 shutdown_connections 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 1 -- 172.21.15.52:0/3642494132 wait complete. 2024-06-02T23:14:10.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.453+0000 7fafcecd4b80 20 client.25850 trim_cache size 0 max 16384 2024-06-02T23:14:10.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 Processor -- start 2024-06-02T23:14:10.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 -- start start 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c58810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c47e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aa8d0 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1ac4a0 con 0x55b4a0c53880 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1aafa0 con 0x55b4a0c52f60 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafc21ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51128/0 (socket says 172.21.15.52:51128) 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafc21ed700 1 -- 172.21.15.52:0/1341969790 learned_addr learned my addr 172.21.15.52:0/1341969790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:10.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.470+0000 7fafc29ee700 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c47e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc19ec700 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c58810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc29ee700 1 -- 172.21.15.52:0/1341969790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 msgr2=0x55b4a0c58810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc29ee700 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c58810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc29ee700 1 -- 172.21.15.52:0/1341969790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 msgr2=0x55b4a0c4d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc29ee700 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc29ee700 1 -- 172.21.15.52:0/1341969790 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4a0c54e40 con 0x55b4a0c53880 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc21ed700 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:10.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafc29ee700 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c47e80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafb40094d0 tx=0x7fafb40092e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=959bbd2670d1c5a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:10.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafa7fff700 1 -- 172.21.15.52:0/1341969790 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4005510 con 0x55b4a0c53880 2024-06-02T23:14:10.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafa7fff700 1 -- 172.21.15.52:0/1341969790 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb4023020 con 0x55b4a0c53880 2024-06-02T23:14:10.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.471+0000 7fafa7fff700 1 -- 172.21.15.52:0/1341969790 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4007500 con 0x55b4a0c53880 2024-06-02T23:14:10.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 -- 172.21.15.52:0/1341969790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 msgr2=0x55b4a0c47e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c47e80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fafb40094d0 tx=0x7fafb40092e0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 -- 172.21.15.52:0/1341969790 shutdown_connections 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c47e80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c4d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/1341969790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c52f60 0x55b4a0c58810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 -- 172.21.15.52:0/1341969790 >> 172.21.15.52:0/1341969790 conn(0x55b4a0b7a080 msgr2=0x55b4a0c58100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 -- 172.21.15.52:0/1341969790 shutdown_connections 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.472+0000 7fafcecd4b80 1 -- 172.21.15.52:0/1341969790 wait complete. 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 -- start start 2024-06-02T23:14:10.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 Processor -- start 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c6f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c6cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c6d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1abd40 con 0x55b4a0c52f60 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1ab670 con 0x55b4a0c53880 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafbc1a8870 con 0x55b4a0c4d7a0 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafcecd4b80 10 client.0 _ll_register_callbacks cb 0x55b4a0b345b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:10.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafc19ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c6cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafc29ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c6d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafc19ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c6cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51144/0 (socket says 172.21.15.52:51144) 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafc19ec700 1 -- 172.21.15.52:0/237484124 learned_addr learned my addr 172.21.15.52:0/237484124 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.473+0000 7fafc21ed700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c6f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc29ee700 1 -- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c4d7a0 msgr2=0x55b4a0c6f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc29ee700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c6f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc29ee700 1 -- 172.21.15.52:0/237484124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 msgr2=0x55b4a0c6cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc29ee700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c6cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc29ee700 1 -- 172.21.15.52:0/237484124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b4a0cd0fe0 con 0x55b4a0c53880 2024-06-02T23:14:10.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc19ec700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c6cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafc29ee700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c6d250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafb40094d0 tx=0x7fafb400e1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624e811016be2bb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafa5ffb700 1 -- 172.21.15.52:0/237484124 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafb4019040 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafa5ffb700 1 -- 172.21.15.52:0/237484124 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fafb4024070 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafb4023020 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafa5ffb700 1 -- 172.21.15.52:0/237484124 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fafb4003b00 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.474+0000 7fafcecd4b80 10 client.25856 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55b4a0cd2120 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafa5ffb700 1 -- 172.21.15.52:0/237484124 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafb402c020 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafa5ffb700 1 -- 172.21.15.52:0/237484124 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fafb4007cd0 con 0x55b4a0c53880 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 10 client.25856 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 10 client.25856 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 20 client.25856 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:10.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0cd2520 0x55b4a0cd4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 4 client.25856 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.475+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fafb4023020 con 0x55b4a0cd2520 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.476+0000 7fafc21ed700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0cd2520 0x55b4a0cd4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.476+0000 7fafa5ffb700 10 client.25856 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.476+0000 7fafc21ed700 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0cd2520 0x55b4a0cd4910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59a31cd060e66850 server_cookie=5fc119747dd961fe in_seq=0 out_seq=0 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.476+0000 7fafa5ffb700 1 -- 172.21.15.52:0/237484124 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fafb4023020 con 0x55b4a0cd2520 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafa5ffb700 10 client.25856 handle_command_reply: tid=0 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 client.25856 shutdown 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 msgr2=0x55b4a0c6d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c6d250 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fafb40094d0 tx=0x7fafb400e1a0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 shutdown_connections 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55b4a0cd2520 0x55b4a0cd4910 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b4a0c53880 0x55b4a0c6d250 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b4a0c4d7a0 0x55b4a0c6f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 --2- 172.21.15.52:0/237484124 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b4a0c52f60 0x55b4a0c6cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 >> 172.21.15.52:0/237484124 conn(0x55b4a0b7a080 msgr2=0x55b4a0c58990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafa5ffb700 0 client.25856 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 shutdown_connections 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 1 -- 172.21.15.52:0/237484124 wait complete. 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.477+0000 7fafcecd4b80 20 client.25856 trim_cache size 0 max 16384 2024-06-02T23:14:10.482 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafa80a2ca0 msgr2=0x7fafa80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafa80a2ca0 0x7fafa80a5160 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fafbc1021d0 tx=0x7fafb801a040 comp rx=0 tx=0).stop 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 msgr2=0x7fafbc19ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 0x7fafbc19ed50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fafac009db0 tx=0x7fafac00c5b0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 shutdown_connections 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafa80a2ca0 0x7fafa80a5160 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafbc107450 0x7fafbc198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafbc102540 0x7fafbc19e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 --2- 172.21.15.52:0/1902088224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafbc102ee0 0x7fafbc19ed50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 >> 172.21.15.52:0/1902088224 conn(0x7fafbc0795f0 msgr2=0x7fafbc100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 shutdown_connections 2024-06-02T23:14:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:10.478+0000 7fafc4451700 1 -- 172.21.15.52:0/1902088224 wait complete. 2024-06-02T23:14:10.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 38528 2.84 MB/sec warmup 79 sec latency 35.748 ms 2024-06-02T23:14:11.239 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:14:11.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:11 smithi174 ceph-mon[35994]: pgmap v500: 97 pgs: 97 active+clean; 37 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-02T23:14:11.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:11 smithi052 ceph-mon[32340]: pgmap v500: 97 pgs: 97 active+clean; 37 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-02T23:14:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 -- 172.21.15.52:0/1829195970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a8104e20 msgr2=0x7f61a81052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 --2- 172.21.15.52:0/1829195970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a8104e20 0x7f61a81052a0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6198009900 tx=0x7f619802a410 comp rx=0 tx=0).stop 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 -- 172.21.15.52:0/1829195970 shutdown_connections 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 --2- 172.21.15.52:0/1829195970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a81057e0 0x7f61a810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 --2- 172.21.15.52:0/1829195970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a8104e20 0x7f61a81052a0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 --2- 172.21.15.52:0/1829195970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 -- 172.21.15.52:0/1829195970 >> 172.21.15.52:0/1829195970 conn(0x7f61a80ff1c0 msgr2=0x7f61a81015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 -- 172.21.15.52:0/1829195970 shutdown_connections 2024-06-02T23:14:11.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.438+0000 7f61b04bd640 1 -- 172.21.15.52:0/1829195970 wait complete. 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 Processor -- start 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 -- start start 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 0x7f61a819a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a81057e0 0x7f61a81a1140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810f740 con 0x7f61a81057e0 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810f5c0 con 0x7f61a8104e20 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.439+0000 7f61b04bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61a810f8c0 con 0x7f61a8103c30 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ada58640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 0x7f61a819a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61aea5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a81057e0 0x7f61a81a1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8199b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36274/0 (socket says 172.21.15.52:36274) 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 -- 172.21.15.52:0/230783591 learned_addr learned my addr 172.21.15.52:0/230783591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ada58640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 0x7f61a819a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56328/0 (socket says 172.21.15.52:56328) 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 -- 172.21.15.52:0/230783591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 msgr2=0x7f61a819a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 0x7f61a819a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 -- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a81057e0 msgr2=0x7f61a81a1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a81057e0 0x7f61a81a1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 -- 172.21.15.52:0/230783591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a81a17b0 con 0x7f61a8103c30 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ada58640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 0x7f61a819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61aea5a640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a81057e0 0x7f61a81a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61ae259640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8199b90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61a400e9c0 tx=0x7f61a400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:11.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f619f7fe640 1 -- 172.21.15.52:0/230783591 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a400cd80 con 0x7f61a8103c30 2024-06-02T23:14:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f619f7fe640 1 -- 172.21.15.52:0/230783591 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61a40035f0 con 0x7f61a8103c30 2024-06-02T23:14:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a81a1aa0 con 0x7f61a8103c30 2024-06-02T23:14:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.440+0000 7f619f7fe640 1 -- 172.21.15.52:0/230783591 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4010690 con 0x7f61a8103c30 2024-06-02T23:14:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.441+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a81a1ff0 con 0x7f61a8103c30 2024-06-02T23:14:11.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.443+0000 7f619f7fe640 1 -- 172.21.15.52:0/230783591 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f61a40037a0 con 0x7f61a8103c30 2024-06-02T23:14:11.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.444+0000 7f61b31d0740 1 Processor -- start 2024-06-02T23:14:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 -- start start 2024-06-02T23:14:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa8820 0x56504aaa8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aaadd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.445 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaae380 0x56504aab0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56504aa034f0 con 0x56504aaa8820 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56504a9a1870 con 0x56504aaae380 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61b31d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56504a9ff4b0 con 0x56504aaa9210 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61aea5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa8820 0x56504aaa8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61aea5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa8820 0x56504aaa8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51160/0 (socket says 172.21.15.52:51160) 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61aea5a640 1 -- 172.21.15.52:0/1459733661 learned_addr learned my addr 172.21.15.52:0/1459733661 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:11.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.445+0000 7f61ae259640 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aaadd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.446+0000 7f61ada58640 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaae380 0x56504aab0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f619f7fe640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61940a2c30 0x7f61940a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f61aea5a640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61940a2c30 0x7f61940a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f61ae259640 1 -- 172.21.15.52:0/1459733661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaae380 msgr2=0x56504aab0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f61ae259640 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaae380 0x56504aab0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f61ae259640 1 -- 172.21.15.52:0/1459733661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa8820 msgr2=0x56504aaa8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f61ae259640 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa8820 0x56504aaa8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.447+0000 7f61ae259640 1 -- 172.21.15.52:0/1459733661 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504a9bd9b0 con 0x56504aaa9210 2024-06-02T23:14:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.448+0000 7f61ae259640 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aaadd10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61a40ccf60 tx=0x7f61a40d54f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d76e09f797a538 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.448+0000 7f619f7fe640 1 -- 172.21.15.52:0/230783591 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f61a40cf460 con 0x7f61a8103c30 2024-06-02T23:14:11.448 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.448+0000 7f61aea5a640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61940a2c30 0x7f61940a50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56504aaa9a70 tx=0x7f61a0004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.448+0000 7f619cbfe640 1 -- 172.21.15.52:0/1459733661 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a4097f90 con 0x56504aaa9210 2024-06-02T23:14:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.448+0000 7f619cbfe640 1 -- 172.21.15.52:0/1459733661 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61a4003b30 con 0x56504aaa9210 2024-06-02T23:14:11.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.448+0000 7f619cbfe640 1 -- 172.21.15.52:0/1459733661 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a409b650 con 0x56504aaa9210 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 -- 172.21.15.52:0/1459733661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 msgr2=0x56504aaadd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aaadd10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f61a40ccf60 tx=0x7f61a40d54f0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 -- 172.21.15.52:0/1459733661 shutdown_connections 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aaadd10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaae380 0x56504aab0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1459733661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa8820 0x56504aaa8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.449+0000 7f61b31d0740 1 -- 172.21.15.52:0/1459733661 >> 172.21.15.52:0/1459733661 conn(0x56504aa9b680 msgr2=0x56504aa9ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.450+0000 7f61b31d0740 1 -- 172.21.15.52:0/1459733661 shutdown_connections 2024-06-02T23:14:11.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.450+0000 7f61b31d0740 1 -- 172.21.15.52:0/1459733661 wait complete. 2024-06-02T23:14:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 -- start start 2024-06-02T23:14:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 Processor -- start 2024-06-02T23:14:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaae380 0x56504aab9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56504aab1330 con 0x56504aaa9210 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56504a98cae0 con 0x56504aaa8820 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61ada58640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56504aab14b0 con 0x56504aaae380 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61ada58640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51166/0 (socket says 172.21.15.52:51166) 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61ada58640 1 -- 172.21.15.52:0/1571783055 learned_addr learned my addr 172.21.15.52:0/1571783055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61ae259640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61aea5a640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaae380 0x56504aab9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.451+0000 7f61b31d0740 10 client.0 _ll_register_callbacks cb 0x56504a9c59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61ada58640 1 -- 172.21.15.52:0/1571783055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaae380 msgr2=0x56504aab9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61ada58640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaae380 0x56504aab9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61ada58640 1 -- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 msgr2=0x56504aab29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61ada58640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61ada58640 1 -- 172.21.15.52:0/1571783055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504a717210 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61aea5a640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaae380 0x56504aab9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61ada58640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 0x56504aab2f20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6198000c00 tx=0x7f6198005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26b40cabd2881712 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f6191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f6191ffb640 1 -- 172.21.15.52:0/1571783055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6198035040 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f6191ffb640 1 -- 172.21.15.52:0/1571783055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f61980318d0 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.452+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56504aa16f10 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f6191ffb640 1 -- 172.21.15.52:0/1571783055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6198031ad0 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 10 client.16290 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56504a7e3620 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f6191ffb640 1 -- 172.21.15.52:0/1571783055 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56504aa16f10 con 0x56504aaa9210 2024-06-02T23:14:11.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f6191ffb640 1 -- 172.21.15.52:0/1571783055 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f61980035f0 con 0x56504aaa9210 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 10 client.16290 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 10 client.16290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 20 client.16290 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504aabe890 0x56504aac0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 4 client.16290 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.453+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56504aa10c20 con 0x56504aabe890 2024-06-02T23:14:11.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.454+0000 7f61ae259640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504aabe890 0x56504aac0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.455+0000 7f61ae259640 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504aabe890 0x56504aac0d10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce1f6435a581611 server_cookie=9920f77b222e24d1 in_seq=0 out_seq=0 2024-06-02T23:14:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.455+0000 7f6191ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.455+0000 7f6191ffb640 1 -- 172.21.15.52:0/1571783055 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56504aa10c20 con 0x56504aabe890 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.455+0000 7f6191ffb640 10 client.16290 handle_command_reply: tid=0 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 client.16290 shutdown 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 msgr2=0x56504aab2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 0x56504aab2f20 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f6198000c00 tx=0x7f6198005270 comp rx=0 tx=0).stop 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 shutdown_connections 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504aabe890 0x56504aac0d10 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaae380 0x56504aab9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 --2- 172.21.15.52:0/1571783055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 >> 172.21.15.52:0/1571783055 conn(0x56504aa9b680 msgr2=0x56504aa9a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f6191ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 shutdown_connections 2024-06-02T23:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 1 -- 172.21.15.52:0/1571783055 wait complete. 2024-06-02T23:14:11.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.456+0000 7f61b31d0740 20 client.16290 trim_cache size 0 max 16384 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.477+0000 7f61b31d0740 1 Processor -- start 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.477+0000 7f61b31d0740 1 -- start start 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504aab9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61b31d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81a6850 con 0x56504aaae380 2024-06-02T23:14:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61b31d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81a8420 con 0x56504aaa8820 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61b31d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81a6f20 con 0x56504aaa9210 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504aab9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504aab9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51168/0 (socket says 172.21.15.52:51168) 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 -- 172.21.15.52:0/3016637665 learned_addr learned my addr 172.21.15.52:0/3016637665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61ada58640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61ae259640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 -- 172.21.15.52:0/3016637665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 msgr2=0x56504aab2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 -- 172.21.15.52:0/3016637665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 msgr2=0x56504aab29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 -- 172.21.15.52:0/3016637665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56504aaa95f0 con 0x56504aaae380 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61ada58640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:11.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61ae259640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.478+0000 7f61aea5a640 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504aab9f20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f61a0010fa0 tx=0x7f61a0011510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65edf3283a37725c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.479+0000 7f6193fff640 1 -- 172.21.15.52:0/3016637665 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a001a070 con 0x56504aaae380 2024-06-02T23:14:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.479+0000 7f6193fff640 1 -- 172.21.15.52:0/3016637665 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61a0021b40 con 0x56504aaae380 2024-06-02T23:14:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.479+0000 7f6193fff640 1 -- 172.21.15.52:0/3016637665 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a000f990 con 0x56504aaae380 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.479+0000 7f61b31d0740 1 -- 172.21.15.52:0/3016637665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 msgr2=0x56504aab9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.479+0000 7f61b31d0740 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504aab9f20 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f61a0010fa0 tx=0x7f61a0011510 comp rx=0 tx=0).stop 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- 172.21.15.52:0/3016637665 shutdown_connections 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504aab9f20 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 --2- 172.21.15.52:0/3016637665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- 172.21.15.52:0/3016637665 >> 172.21.15.52:0/3016637665 conn(0x56504aa9b680 msgr2=0x56504aa9ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- 172.21.15.52:0/3016637665 shutdown_connections 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- 172.21.15.52:0/3016637665 wait complete. 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- start start 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 Processor -- start 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504ab275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81a7cc0 con 0x56504aaae380 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81a75f0 con 0x56504aaa8820 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61b31d0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61a81a4830 con 0x56504aaa9210 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.480+0000 7f61ae259640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56372/0 (socket says 172.21.15.52:56372) 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61b31d0740 10 client.0 _ll_register_callbacks cb 0x7f61a4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 -- 172.21.15.52:0/2736679679 learned_addr learned my addr 172.21.15.52:0/2736679679 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61aea5a640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504ab275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ada58640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 -- 172.21.15.52:0/2736679679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 msgr2=0x56504aab72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 -- 172.21.15.52:0/2736679679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 msgr2=0x56504ab275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504ab275e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 -- 172.21.15.52:0/2736679679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56504aab7920 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ada58640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61ae259640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab6d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61a40ccfe0 tx=0x7f61a40d5af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1fac88bc7206039 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f6191ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f6191ffb640 1 -- 172.21.15.52:0/2736679679 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61a409b830 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f6191ffb640 1 -- 172.21.15.52:0/2736679679 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f61a40cd0a0 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61a0021b40 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f6191ffb640 1 -- 172.21.15.52:0/2736679679 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f61a409bac0 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61b31d0740 10 client.25874 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.481+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56504ab28c10 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f6191ffb640 1 -- 172.21.15.52:0/2736679679 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61a0021b40 con 0x56504aaa8820 2024-06-02T23:14:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f6191ffb640 1 -- 172.21.15.52:0/2736679679 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f61a40b3080 con 0x56504aaa8820 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61b31d0740 10 client.25874 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61b31d0740 10 client.25874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61b31d0740 20 client.25874 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61b31d0740 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504ab29160 0x56504ab2b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61b31d0740 4 client.25874 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56504aa0af70 con 0x56504ab29160 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61ada58640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504ab29160 0x56504ab2b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f6191ffb640 10 client.25874 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.482+0000 7f61ada58640 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504ab29160 0x56504ab2b580 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3b0b762beef1f11 server_cookie=32b8942f9d2fdb8e in_seq=0 out_seq=0 2024-06-02T23:14:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.503+0000 7f6191ffb640 1 -- 172.21.15.52:0/2736679679 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56504aa0af70 con 0x56504ab29160 2024-06-02T23:14:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.503+0000 7f6191ffb640 10 client.25874 handle_command_reply: tid=0 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 client.25874 shutdown 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 msgr2=0x56504aab6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab6d70 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f61a40ccfe0 tx=0x7f61a40d5af0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 shutdown_connections 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56504ab29160 0x56504ab2b580 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56504aaa9210 0x56504aab72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56504aaae380 0x56504ab275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 --2- 172.21.15.52:0/2736679679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56504aaa8820 0x56504aab6d70 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f6191ffb640 0 client.25874 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 >> 172.21.15.52:0/2736679679 conn(0x56504aa9b680 msgr2=0x56504aaaa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 shutdown_connections 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 1 -- 172.21.15.52:0/2736679679 wait complete. 2024-06-02T23:14:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.504+0000 7f61b31d0740 20 client.25874 trim_cache size 0 max 16384 2024-06-02T23:14:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:14:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:14:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61940a2c30 msgr2=0x7f61940a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61940a2c30 0x7f61940a50f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x56504aaa9a70 tx=0x7f61a0004ab0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 msgr2=0x7f61a8199b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8199b90 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f61a400e9c0 tx=0x7f61a400ee80 comp rx=0 tx=0).stop 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 shutdown_connections 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61940a2c30 0x7f61940a50f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61a81057e0 0x7f61a81a1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61a8104e20 0x7f61a819a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 --2- 172.21.15.52:0/230783591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61a8103c30 0x7f61a8199b90 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 >> 172.21.15.52:0/230783591 conn(0x7f61a80ff1c0 msgr2=0x7f61a8108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:11.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 shutdown_connections 2024-06-02T23:14:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:11.505+0000 7f61b04bd640 1 -- 172.21.15.52:0/230783591 wait complete. 2024-06-02T23:14:11.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:11 smithi123 ceph-mon[35977]: pgmap v500: 97 pgs: 97 active+clean; 37 MiB data, 511 MiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-02T23:14:11.547 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:11.560 INFO:tasks.quiescer.fs.[cephfs]:Total duration (55.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:14:11.560 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.75 2024-06-02T23:14:11.560 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:14:11.619 INFO:tasks.workunit.client.0.smithi052.stdout: 1 39035 2.87 MB/sec warmup 80 sec latency 23.668 ms 2024-06-02T23:14:12.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 39391 2.85 MB/sec warmup 81 sec latency 52.165 ms 2024-06-02T23:14:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:13 smithi174 ceph-mon[35994]: pgmap v501: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-06-02T23:14:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:13 smithi052 ceph-mon[32340]: pgmap v501: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-06-02T23:14:13.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:13 smithi123 ceph-mon[35977]: pgmap v501: 97 pgs: 97 active+clean; 36 MiB data, 555 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-06-02T23:14:13.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 39610 2.82 MB/sec warmup 82 sec latency 71.843 ms 2024-06-02T23:14:14.557 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:14.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 40098 2.81 MB/sec warmup 83 sec latency 48.451 ms 2024-06-02T23:14:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:15 smithi174 ceph-mon[35994]: pgmap v502: 97 pgs: 97 active+clean; 36 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-06-02T23:14:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:15 smithi052 ceph-mon[32340]: pgmap v502: 97 pgs: 97 active+clean; 36 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-06-02T23:14:15.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:15 smithi123 ceph-mon[35977]: pgmap v502: 97 pgs: 97 active+clean; 36 MiB data, 556 MiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.1 MiB/s wr, 131 op/s 2024-06-02T23:14:15.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 40651 2.81 MB/sec warmup 84 sec latency 42.337 ms 2024-06-02T23:14:15.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 -- 172.21.15.52:0/2023160756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101d50 msgr2=0x7f437810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 --2- 172.21.15.52:0/2023160756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101d50 0x7f437810e240 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f437000b0a0 tx=0x7f437000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 -- 172.21.15.52:0/2023160756 shutdown_connections 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 --2- 172.21.15.52:0/2023160756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101d50 0x7f437810e240 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 --2- 172.21.15.52:0/2023160756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4378107400 0x7f43781077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 --2- 172.21.15.52:0/2023160756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101390 0x7f4378101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.910+0000 7f437f2ed700 1 -- 172.21.15.52:0/2023160756 >> 172.21.15.52:0/2023160756 conn(0x7f4378078340 msgr2=0x7f4378078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.911+0000 7f437f2ed700 1 -- 172.21.15.52:0/2023160756 shutdown_connections 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.911+0000 7f437f2ed700 1 -- 172.21.15.52:0/2023160756 wait complete. 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.911+0000 7f437f2ed700 1 Processor -- start 2024-06-02T23:14:15.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.911+0000 7f437f2ed700 1 -- start start 2024-06-02T23:14:15.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437f2ed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 0x7f4378103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437f2ed700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101d50 0x7f43781035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437f2ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4378107400 0x7f4378103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437f2ed700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4378111520 con 0x7f4378101390 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437f2ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43781113a0 con 0x7f4378107400 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437f2ed700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43781116a0 con 0x7f4378101d50 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 0x7f4378103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437c888700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101d50 0x7f43781035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 0x7f4378103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58446/0 (socket says 172.21.15.52:58446) 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 -- 172.21.15.52:0/4197033781 learned_addr learned my addr 172.21.15.52:0/4197033781 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d88a700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4378107400 0x7f4378103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 -- 172.21.15.52:0/4197033781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101d50 msgr2=0x7f43781035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101d50 0x7f43781035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 -- 172.21.15.52:0/4197033781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4378107400 msgr2=0x7f4378103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4378107400 0x7f4378103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437d089700 1 -- 172.21.15.52:0/4197033781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43781a7000 con 0x7f4378101390 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.912+0000 7f437c888700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101d50 0x7f43781035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.913+0000 7f437d089700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 0x7f4378103070 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f437400e9a0 tx=0x7f437400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.913+0000 7f436e7fc700 1 -- 172.21.15.52:0/4197033781 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437400ca10 con 0x7f4378101390 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.913+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43781a72f0 con 0x7f4378101390 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.913+0000 7f436e7fc700 1 -- 172.21.15.52:0/4197033781 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f437400cbb0 con 0x7f4378101390 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.913+0000 7f436e7fc700 1 -- 172.21.15.52:0/4197033781 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4374018680 con 0x7f4378101390 2024-06-02T23:14:15.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.913+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43781a7880 con 0x7f4378101390 2024-06-02T23:14:15.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.916+0000 7f436e7fc700 1 -- 172.21.15.52:0/4197033781 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4374010430 con 0x7f4378101390 2024-06-02T23:14:15.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.916+0000 7f436e7fc700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43640a2ca0 0x7f43640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.917+0000 7f437c888700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43640a2ca0 0x7f43640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.917+0000 7f436e7fc700 1 -- 172.21.15.52:0/4197033781 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f43740cf0c0 con 0x7f4378101390 2024-06-02T23:14:15.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.917+0000 7f437c888700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43640a2ca0 0x7f43640a5160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f43780788b0 tx=0x7f4368009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:15.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.917+0000 7f4389b70b80 1 Processor -- start 2024-06-02T23:14:15.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.917+0000 7f4389b70b80 1 -- start start 2024-06-02T23:14:15.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa7e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f4389b70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557f1a96af60 con 0x557f1aa7e880 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f4389b70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f1a9c9850 con 0x557f1aa787a0 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f4389b70b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557f1a931460 con 0x557f1aa7df60 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437c888700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437c888700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa78260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58452/0 (socket says 172.21.15.52:58452) 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437c888700 1 -- 172.21.15.52:0/4031171905 learned_addr learned my addr 172.21.15.52:0/4031171905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d88a700 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d089700 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa7e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d88a700 1 -- 172.21.15.52:0/4031171905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 msgr2=0x557f1aa7e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d88a700 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa7e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d88a700 1 -- 172.21.15.52:0/4031171905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 msgr2=0x557f1aa78260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d88a700 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa78260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.918+0000 7f437d88a700 1 -- 172.21.15.52:0/4031171905 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1a9188d0 con 0x557f1aa787a0 2024-06-02T23:14:15.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.919+0000 7f437d88a700 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78bc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f437000d010 tx=0x7f4370009df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e270951d10dcabca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:15.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.919+0000 7f43637fe700 1 -- 172.21.15.52:0/4031171905 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437000bab0 con 0x557f1aa787a0 2024-06-02T23:14:15.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.919+0000 7f43637fe700 1 -- 172.21.15.52:0/4031171905 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f437000bc50 con 0x557f1aa787a0 2024-06-02T23:14:15.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.919+0000 7f43637fe700 1 -- 172.21.15.52:0/4031171905 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4370007d10 con 0x557f1aa787a0 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 -- 172.21.15.52:0/4031171905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 msgr2=0x557f1aa78bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78bc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f437000d010 tx=0x7f4370009df0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 -- 172.21.15.52:0/4031171905 shutdown_connections 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78bc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa7e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 --2- 172.21.15.52:0/4031171905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa78260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 -- 172.21.15.52:0/4031171905 >> 172.21.15.52:0/4031171905 conn(0x557f1a9a5080 msgr2=0x557f1aa73220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 -- 172.21.15.52:0/4031171905 shutdown_connections 2024-06-02T23:14:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 -- 172.21.15.52:0/4031171905 wait complete. 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 -- start start 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.920+0000 7f4389b70b80 1 Processor -- start 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa845d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa818d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa81e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557f1a969fc0 con 0x557f1aa7df60 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f1a9c5810 con 0x557f1aa787a0 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557f1aa85200 con 0x557f1aa7e880 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f4389b70b80 10 client.0 _ll_register_callbacks cb 0x557f1a95f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa818d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437d089700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa845d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa818d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58454/0 (socket says 172.21.15.52:58454) 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 -- 172.21.15.52:0/2220126600 learned_addr learned my addr 172.21.15.52:0/2220126600 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437d88a700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa81e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 -- 172.21.15.52:0/2220126600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 msgr2=0x557f1aa81e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa81e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 -- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 msgr2=0x557f1aa845d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa845d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437c888700 1 -- 172.21.15.52:0/2220126600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1a98f750 con 0x557f1aa7df60 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.921+0000 7f437d089700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f437c888700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa818d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f43680037c0 tx=0x7f43680047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6869e151dce98967 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4360ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4360ff9700 1 -- 172.21.15.52:0/2220126600 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43680285f0 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4360ff9700 1 -- 172.21.15.52:0/2220126600 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4368028790 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437000bc50 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4360ff9700 1 -- 172.21.15.52:0/2220126600 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4368028990 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4389b70b80 10 client.16308 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f1aa8f9f0 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.922+0000 7f4360ff9700 1 -- 172.21.15.52:0/2220126600 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f437000bc50 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4360ff9700 1 -- 172.21.15.52:0/2220126600 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f436801ec40 con 0x557f1aa7df60 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4389b70b80 10 client.16308 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4389b70b80 10 client.16308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4389b70b80 20 client.16308 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4389b70b80 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aa92080 0x557f1aa94500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4389b70b80 4 client.16308 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f1a9d6d50 con 0x557f1aa92080 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f437d089700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aa92080 0x557f1aa94500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f4360ff9700 10 client.16308 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.923+0000 7f437d089700 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aa92080 0x557f1aa94500 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cdcb22986f6f6d2a server_cookie=704eaa28b7d4353 in_seq=0 out_seq=0 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4360ff9700 1 -- 172.21.15.52:0/2220126600 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f1a9d6d50 con 0x557f1aa92080 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4360ff9700 10 client.16308 handle_command_reply: tid=0 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 client.16308 shutdown 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 msgr2=0x557f1aa818d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa818d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f43680037c0 tx=0x7f43680047c0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 shutdown_connections 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aa92080 0x557f1aa94500 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa81e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa845d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 --2- 172.21.15.52:0/2220126600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa818d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.924+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 >> 172.21.15.52:0/2220126600 conn(0x557f1a9a5080 msgr2=0x557f1aa72700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.925+0000 7f4360ff9700 0 client.16308 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.925+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 shutdown_connections 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.925+0000 7f4389b70b80 1 -- 172.21.15.52:0/2220126600 wait complete. 2024-06-02T23:14:15.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.925+0000 7f4389b70b80 20 client.16308 trim_cache size 0 max 16384 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.942+0000 7f4389b70b80 1 Processor -- start 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 -- start start 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa72e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43781aa820 con 0x557f1aa7e880 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43781ac3f0 con 0x557f1aa787a0 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f4389b70b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43781aaef0 con 0x557f1aa7df60 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d88a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46656/0 (socket says 172.21.15.52:46656) 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 -- 172.21.15.52:0/1374277208 learned_addr learned my addr 172.21.15.52:0/1374277208 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437c888700 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa72e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 -- 172.21.15.52:0/1374277208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 msgr2=0x557f1aa72e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa72e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 -- 172.21.15.52:0/1374277208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 msgr2=0x557f1aa733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d089700 1 -- 172.21.15.52:0/1374277208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f1aa7fe60 con 0x557f1aa787a0 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.943+0000 7f437d88a700 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.944+0000 7f437d089700 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f437409b650 tx=0x7f43740ccac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ed9ede8cfe92b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.944+0000 7f4362ffd700 1 -- 172.21.15.52:0/1374277208 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f437409b950 con 0x557f1aa787a0 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.944+0000 7f4362ffd700 1 -- 172.21.15.52:0/1374277208 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f437409fd00 con 0x557f1aa787a0 2024-06-02T23:14:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.944+0000 7f4362ffd700 1 -- 172.21.15.52:0/1374277208 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740a4130 con 0x557f1aa787a0 2024-06-02T23:14:15.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.944+0000 7f4389b70b80 1 -- 172.21.15.52:0/1374277208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 msgr2=0x557f1aa78260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.944+0000 7f4389b70b80 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78260 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f437409b650 tx=0x7f43740ccac0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 -- 172.21.15.52:0/1374277208 shutdown_connections 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7e880 0x557f1aa733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa78260 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 --2- 172.21.15.52:0/1374277208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7df60 0x557f1aa72e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 -- 172.21.15.52:0/1374277208 >> 172.21.15.52:0/1374277208 conn(0x557f1a9a5080 msgr2=0x557f1aa801e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 -- 172.21.15.52:0/1374277208 shutdown_connections 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 -- 172.21.15.52:0/1374277208 wait complete. 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 -- start start 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.945+0000 7f4389b70b80 1 Processor -- start 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa8a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa8aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa98fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43781abc90 con 0x557f1aa7df60 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43781ab5c0 con 0x557f1aa787a0 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f437819c620 con 0x557f1aa7e880 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4389b70b80 10 client.0 _ll_register_callbacks cb 0x557f1a95f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437c888700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa8aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa8a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa8a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46670/0 (socket says 172.21.15.52:46670) 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 -- 172.21.15.52:0/482050283 learned_addr learned my addr 172.21.15.52:0/482050283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d88a700 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa98fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 -- 172.21.15.52:0/482050283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 msgr2=0x557f1aa98fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa98fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 -- 172.21.15.52:0/482050283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 msgr2=0x557f1aa8aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa8aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 -- 172.21.15.52:0/482050283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f1aa99560 con 0x557f1aa787a0 2024-06-02T23:14:15.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f437d089700 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa8a4f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f437409b8e0 tx=0x7f43740ccb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e6da779889b5dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.946+0000 7f4360ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4360ff9700 1 -- 172.21.15.52:0/482050283 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43740ccd00 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4360ff9700 1 -- 172.21.15.52:0/482050283 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4374099c50 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f437409fd00 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4360ff9700 1 -- 172.21.15.52:0/482050283 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f437409b950 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 10 client.25892 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557f1aafc600 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4360ff9700 1 -- 172.21.15.52:0/482050283 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f437409fd00 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4360ff9700 1 -- 172.21.15.52:0/482050283 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f4374086220 con 0x557f1aa787a0 2024-06-02T23:14:15.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 10 client.25892 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 10 client.25892 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 20 client.25892 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aafcd90 0x557f1aaff180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 4 client.25892 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557f1a9d12d0 con 0x557f1aafcd90 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.947+0000 7f437c888700 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aafcd90 0x557f1aaff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4360ff9700 10 client.25892 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f437c888700 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aafcd90 0x557f1aaff180 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5bde7eb16950221 server_cookie=60a32aa46dfdeac6 in_seq=0 out_seq=0 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4360ff9700 1 -- 172.21.15.52:0/482050283 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557f1a9d12d0 con 0x557f1aafcd90 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4360ff9700 10 client.25892 handle_command_reply: tid=0 2024-06-02T23:14:15.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 client.25892 shutdown 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 msgr2=0x557f1aa8a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa8a4f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f437409b8e0 tx=0x7f43740ccb30 comp rx=0 tx=0).stop 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 shutdown_connections 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f1aafcd90 0x557f1aaff180 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f1aa7e880 0x557f1aa98fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f1aa787a0 0x557f1aa8a4f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 --2- 172.21.15.52:0/482050283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f1aa7df60 0x557f1aa8aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 >> 172.21.15.52:0/482050283 conn(0x557f1a9a5080 msgr2=0x557f1aa7ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4360ff9700 0 client.25892 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 shutdown_connections 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.948+0000 7f4389b70b80 1 -- 172.21.15.52:0/482050283 wait complete. 2024-06-02T23:14:15.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f4389b70b80 20 client.25892 trim_cache size 0 max 16384 2024-06-02T23:14:15.955 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:14:15.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43640a2ca0 msgr2=0x7f43640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43640a2ca0 0x7f43640a5160 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f43780788b0 tx=0x7f4368009260 comp rx=0 tx=0).stop 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 msgr2=0x7f4378103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 0x7f4378103070 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f437400e9a0 tx=0x7f437400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 shutdown_connections 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43640a2ca0 0x7f43640a5160 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4378107400 0x7f4378103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4378101390 0x7f4378103070 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 --2- 172.21.15.52:0/4197033781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4378101d50 0x7f43781035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 >> 172.21.15.52:0/4197033781 conn(0x7f4378078340 msgr2=0x7f4378100630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 shutdown_connections 2024-06-02T23:14:15.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:15.949+0000 7f437f2ed700 1 -- 172.21.15.52:0/4197033781 wait complete. 2024-06-02T23:14:16.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 41281 2.79 MB/sec warmup 85 sec latency 25.012 ms 2024-06-02T23:14:17.353 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:17 smithi052 ceph-mon[32340]: pgmap v503: 97 pgs: 97 active+clean; 39 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-02T23:14:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:17 smithi174 ceph-mon[35994]: pgmap v503: 97 pgs: 97 active+clean; 39 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-02T23:14:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:17 smithi123 ceph-mon[35977]: pgmap v503: 97 pgs: 97 active+clean; 39 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 339 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-02T23:14:17.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 41835 2.82 MB/sec warmup 86 sec latency 14.953 ms 2024-06-02T23:14:17.810 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:14:17.811 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:14:18.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.340+0000 7f26de39e700 1 -- 172.21.15.52:0/551739229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8101f20 msgr2=0x7f26d8105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:18.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.340+0000 7f26de39e700 1 --2- 172.21.15.52:0/551739229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8101f20 0x7f26d8105f70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f26c8009990 tx=0x7f26c8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.340+0000 7f26de39e700 1 -- 172.21.15.52:0/551739229 shutdown_connections 2024-06-02T23:14:18.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.340+0000 7f26de39e700 1 --2- 172.21.15.52:0/551739229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d8106650 0x7f26d810e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.340+0000 7f26de39e700 1 --2- 172.21.15.52:0/551739229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 0x7f26d8101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.340+0000 7f26de39e700 1 --2- 172.21.15.52:0/551739229 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8101f20 0x7f26d8105f70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.341+0000 7f26de39e700 1 -- 172.21.15.52:0/551739229 >> 172.21.15.52:0/551739229 conn(0x7f26d80fce20 msgr2=0x7f26d80ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:18.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.341+0000 7f26de39e700 1 -- 172.21.15.52:0/551739229 shutdown_connections 2024-06-02T23:14:18.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.341+0000 7f26de39e700 1 -- 172.21.15.52:0/551739229 wait complete. 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 Processor -- start 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 -- start start 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 0x7f26d819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d8101f20 0x7f26d819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8106650 0x7f26d819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26d8110440 con 0x7f26d8106650 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26d81102c0 con 0x7f26d8101f20 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26de39e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26d81105c0 con 0x7f26d8101570 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26ddb9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8106650 0x7f26d819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.342+0000 7f26dcb9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d8101f20 0x7f26d819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26ddb9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8106650 0x7f26d819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58488/0 (socket says 172.21.15.52:58488) 2024-06-02T23:14:18.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26ddb9d700 1 -- 172.21.15.52:0/1309206490 learned_addr learned my addr 172.21.15.52:0/1309206490 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 0x7f26d819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 -- 172.21.15.52:0/1309206490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d8101f20 msgr2=0x7f26d819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d8101f20 0x7f26d819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 -- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8106650 msgr2=0x7f26d819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8106650 0x7f26d819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 -- 172.21.15.52:0/1309206490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26d819fde0 con 0x7f26d8101570 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26dd39c700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 0x7f26d819ae20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e9f0 tx=0x7f26d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:18.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4004220 con 0x7f26d8101570 2024-06-02T23:14:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f26d40043c0 con 0x7f26d8101570 2024-06-02T23:14:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26d4005020 con 0x7f26d8101570 2024-06-02T23:14:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.343+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26d81a00d0 con 0x7f26d8101570 2024-06-02T23:14:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.344+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26d81a7950 con 0x7f26d8101570 2024-06-02T23:14:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.345+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26d8069180 con 0x7f26d8101570 2024-06-02T23:14:18.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.346+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f26d4004750 con 0x7f26d8101570 2024-06-02T23:14:18.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.347+0000 7f26ce7fc700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26c40a2ca0 0x7f26c40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:18.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.347+0000 7f26dcb9b700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26c40a2ca0 0x7f26c40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:18.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.347+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f26d40cef00 con 0x7f26d8101570 2024-06-02T23:14:18.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.347+0000 7f26dcb9b700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26c40a2ca0 0x7f26c40a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f26c8006010 tx=0x7f26c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:18.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.351+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f26d409be90 con 0x7f26d8101570 2024-06-02T23:14:18.362 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:18.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 42394 2.82 MB/sec warmup 87 sec latency 14.711 ms 2024-06-02T23:14:18.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.661+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26d819c130 con 0x7f26d8101570 2024-06-02T23:14:18.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.662+0000 7f26ce7fc700 1 -- 172.21.15.52:0/1309206490 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7f26d409b790 con 0x7f26d8101570 2024-06-02T23:14:18.662 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:14:18.663 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:14:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26c40a2ca0 msgr2=0x7f26c40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:18.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26c40a2ca0 0x7f26c40a5160 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f26c8006010 tx=0x7f26c8005960 comp rx=0 tx=0).stop 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 msgr2=0x7f26d819ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 0x7f26d819ae20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f26d400e9f0 tx=0x7f26d400ed00 comp rx=0 tx=0).stop 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 shutdown_connections 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26c40a2ca0 0x7f26c40a5160 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f26d8106650 0x7f26d819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f26d8101570 0x7f26d819ae20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 --2- 172.21.15.52:0/1309206490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26d8101f20 0x7f26d819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 >> 172.21.15.52:0/1309206490 conn(0x7f26d80fce20 msgr2=0x7f26d80fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 shutdown_connections 2024-06-02T23:14:18.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:18.664+0000 7f26de39e700 1 -- 172.21.15.52:0/1309206490 wait complete. 2024-06-02T23:14:18.666 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:14:19.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:19 smithi052 ceph-mon[32340]: pgmap v504: 97 pgs: 97 active+clean; 38 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:14:19.183 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:19 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1309206490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:19 smithi174 ceph-mon[35994]: pgmap v504: 97 pgs: 97 active+clean; 38 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:14:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:19 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1309206490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:19.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:19 smithi123 ceph-mon[35977]: pgmap v504: 97 pgs: 97 active+clean; 38 MiB data, 547 MiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:14:19.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:19 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1309206490' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:19.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 42874 2.84 MB/sec warmup 88 sec latency 12.991 ms 2024-06-02T23:14:20.150 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 147.75 --await-for 45.0' 2024-06-02T23:14:20.150 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 147.75 --await-for 45.0 2024-06-02T23:14:20.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 43153 2.81 MB/sec warmup 89 sec latency 18.878 ms 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/3246314154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a41046a0 msgr2=0x7fd1a4104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/3246314154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a41046a0 0x7fd1a4104b20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fd194009990 tx=0x7fd194009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/3246314154 shutdown_connections 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/3246314154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a4111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/3246314154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a410a6a0 0x7fd1a410aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/3246314154 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a41046a0 0x7fd1a4104b20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.677+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/3246314154 >> 172.21.15.52:0/3246314154 conn(0x7fd1a4100170 msgr2=0x7fd1a4102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:20.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.678+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/3246314154 shutdown_connections 2024-06-02T23:14:20.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.678+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/3246314154 wait complete. 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.678+0000 7fd1ab2f4700 1 Processor -- start 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 -- start start 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41046a0 0x7fd1a419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a410a6a0 0x7fd1a419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a4114880 con 0x7fd1a410a6a0 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a4114700 con 0x7fd1a4105060 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1ab2f4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a4114a00 con 0x7fd1a41046a0 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1a888f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1a888f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46702/0 (socket says 172.21.15.52:46702) 2024-06-02T23:14:20.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1a888f700 1 -- 172.21.15.52:0/1649817353 learned_addr learned my addr 172.21.15.52:0/1649817353 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:20.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.679+0000 7fd1a9090700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41046a0 0x7fd1a419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.680+0000 7fd1a9891700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a410a6a0 0x7fd1a419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.680+0000 7fd1a888f700 1 -- 172.21.15.52:0/1649817353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41046a0 msgr2=0x7fd1a419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.680+0000 7fd1a888f700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41046a0 0x7fd1a419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.680+0000 7fd1a888f700 1 -- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a410a6a0 msgr2=0x7fd1a419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.680+0000 7fd1a888f700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a410a6a0 0x7fd1a419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.680+0000 7fd1a888f700 1 -- 172.21.15.52:0/1649817353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a419fd40 con 0x7fd1a4105060 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.681+0000 7fd1a888f700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a419b270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd194000c00 tx=0x7fd1940047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.681+0000 7fd19a7fc700 1 -- 172.21.15.52:0/1649817353 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194027070 con 0x7fd1a4105060 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.681+0000 7fd19a7fc700 1 -- 172.21.15.52:0/1649817353 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd194022470 con 0x7fd1a4105060 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.681+0000 7fd19a7fc700 1 -- 172.21.15.52:0/1649817353 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd194017650 con 0x7fd1a4105060 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.681+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a419ffd0 con 0x7fd1a4105060 2024-06-02T23:14:20.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.681+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a41a78b0 con 0x7fd1a4105060 2024-06-02T23:14:20.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.684+0000 7fd19a7fc700 1 -- 172.21.15.52:0/1649817353 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd19402a070 con 0x7fd1a4105060 2024-06-02T23:14:20.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.684+0000 7fd19a7fc700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1900a2ca0 0x7fd1900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.685+0000 7fd19a7fc700 1 -- 172.21.15.52:0/1649817353 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd1940cff00 con 0x7fd1a4105060 2024-06-02T23:14:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.685+0000 7fd1a9090700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1900a2ca0 0x7fd1900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.685+0000 7fd1a9090700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1900a2ca0 0x7fd1900a5160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007b60 tx=0x7fd1a00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.686+0000 7fd1b5b77b80 1 Processor -- start 2024-06-02T23:14:20.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.686+0000 7fd1b5b77b80 1 -- start start 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.686+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.686+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 0x55cb824a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.686+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cb82390fe0 con 0x55cb824ad110 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb823ef8d0 con 0x55cb824ac7f0 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb823574e0 con 0x55cb824a7030 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 0x55cb824a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 0x55cb824a6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58520/0 (socket says 172.21.15.52:58520) 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 -- 172.21.15.52:0/3529812522 learned_addr learned my addr 172.21.15.52:0/3529812522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:20.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a9090700 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a9891700 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 -- 172.21.15.52:0/3529812522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 msgr2=0x55cb824a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 -- 172.21.15.52:0/3529812522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 msgr2=0x55cb824acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824acbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 -- 172.21.15.52:0/3529812522 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8233e950 con 0x55cb824ad110 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a9891700 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.687+0000 7fd1a888f700 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 0x55cb824a6af0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd194098c20 tx=0x7fd194022620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d67db07c5251a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd18f7fe700 1 -- 172.21.15.52:0/3529812522 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19409c7c0 con 0x55cb824ad110 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd18f7fe700 1 -- 172.21.15.52:0/3529812522 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1940a0b70 con 0x55cb824ad110 2024-06-02T23:14:20.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd18f7fe700 1 -- 172.21.15.52:0/3529812522 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd19409cc80 con 0x55cb824ad110 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3529812522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 msgr2=0x55cb824a6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 0x55cb824a6af0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fd194098c20 tx=0x7fd194022620 comp rx=0 tx=0).stop 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3529812522 shutdown_connections 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3529812522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ad110 0x55cb824a6af0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.688+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3529812522 >> 172.21.15.52:0/3529812522 conn(0x55cb823cb100 msgr2=0x55cb824992a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3529812522 shutdown_connections 2024-06-02T23:14:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3529812522 wait complete. 2024-06-02T23:14:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 -- start start 2024-06-02T23:14:20.690 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 Processor -- start 2024-06-02T23:14:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 0x55cb824b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ac7f0 0x55cb824b0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb824b0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cb82390040 con 0x55cb824a7030 2024-06-02T23:14:20.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb823eb890 con 0x55cb824ad110 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb824b38b0 con 0x55cb824ac7f0 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.689+0000 7fd1b5b77b80 10 client.0 _ll_register_callbacks cb 0x55cb82385630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 0x55cb824b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 0x55cb824b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58532/0 (socket says 172.21.15.52:58532) 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 -- 172.21.15.52:0/3569577973 learned_addr learned my addr 172.21.15.52:0/3569577973 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9891700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb824b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a888f700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ac7f0 0x55cb824b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 -- 172.21.15.52:0/3569577973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ac7f0 msgr2=0x55cb824b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ac7f0 0x55cb824b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 -- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 msgr2=0x55cb824b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb824b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 -- 172.21.15.52:0/3569577973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb823b57d0 con 0x55cb824a7030 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9891700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb824b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1a9090700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 0x55cb824b2e30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00116f0 tx=0x7fd1a00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3918d08833e6345 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:20.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd18cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd18cff9700 1 -- 172.21.15.52:0/3569577973 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0004030 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1940a0b70 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.690+0000 7fd18cff9700 1 -- 172.21.15.52:0/3569577973 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd1a00041d0 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd18cff9700 1 -- 172.21.15.52:0/3569577973 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd1a002e070 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 10 client.16332 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb824bdfa0 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd18cff9700 1 -- 172.21.15.52:0/3569577973 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1940a0b70 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd18cff9700 1 -- 172.21.15.52:0/3569577973 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd1a001f430 con 0x55cb824a7030 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 10 client.16332 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 20 client.16332 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb824c0580 0x55cb824c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 4 client.16332 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:20.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.691+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb823fcdd0 con 0x55cb824c0580 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.692+0000 7fd1a888f700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb824c0580 0x55cb824c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.692+0000 7fd18cff9700 10 client.16332 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.692+0000 7fd1a888f700 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb824c0580 0x55cb824c29c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63455997aee61a5d server_cookie=410190e39799958a in_seq=0 out_seq=0 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd18cff9700 1 -- 172.21.15.52:0/3569577973 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cb823fcdd0 con 0x55cb824c0580 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd18cff9700 10 client.16332 handle_command_reply: tid=0 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 client.16332 shutdown 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 msgr2=0x55cb824b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 0x55cb824b2e30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fd1a00116f0 tx=0x7fd1a00047c0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 shutdown_connections 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb824c0580 0x55cb824c29c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb824b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824a7030 0x55cb824b2e30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/3569577973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ac7f0 0x55cb824b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 >> 172.21.15.52:0/3569577973 conn(0x55cb823cb100 msgr2=0x55cb82498780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd18cff9700 0 client.16332 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:20.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 shutdown_connections 2024-06-02T23:14:20.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/3569577973 wait complete. 2024-06-02T23:14:20.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.693+0000 7fd1b5b77b80 20 client.16332 trim_cache size 0 max 16384 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 Processor -- start 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 -- start start 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824b2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ac7f0 0x55cb82498f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb82499440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a41aa7f0 con 0x55cb824ac7f0 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a41ac3c0 con 0x55cb824ad110 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a41aaec0 con 0x55cb824a7030 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824b2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39964/0 (socket says 172.21.15.52:39964) 2024-06-02T23:14:20.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 -- 172.21.15.52:0/2790168526 learned_addr learned my addr 172.21.15.52:0/2790168526 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9891700 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb82499440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 -- 172.21.15.52:0/2790168526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 msgr2=0x55cb82499440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb82499440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 -- 172.21.15.52:0/2790168526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ac7f0 msgr2=0x55cb82498f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.719+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ac7f0 0x55cb82498f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.720+0000 7fd1a9090700 1 -- 172.21.15.52:0/2790168526 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb824b1030 con 0x55cb824a7030 2024-06-02T23:14:20.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.720+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824b2970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007590 tx=0x7fd1a00075c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6cde0cdd18c6c88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.720+0000 7fd18effd700 1 -- 172.21.15.52:0/2790168526 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a001f630 con 0x55cb824a7030 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.720+0000 7fd18effd700 1 -- 172.21.15.52:0/2790168526 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1a0023070 con 0x55cb824a7030 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.720+0000 7fd18effd700 1 -- 172.21.15.52:0/2790168526 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1a0003b20 con 0x55cb824a7030 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2790168526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 msgr2=0x55cb824b2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824b2970 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007590 tx=0x7fd1a00075c0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2790168526 shutdown_connections 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ad110 0x55cb82499440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824a7030 0x55cb824b2970 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2790168526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824ac7f0 0x55cb82498f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2790168526 >> 172.21.15.52:0/2790168526 conn(0x55cb823cb100 msgr2=0x55cb8249db60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2790168526 shutdown_connections 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2790168526 wait complete. 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.721+0000 7fd1b5b77b80 1 -- start start 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 Processor -- start 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ad110 0x55cb824b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 0x55cb8252b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a41abc60 con 0x55cb824b6170 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a41ab590 con 0x55cb824ac7f0 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a407a800 con 0x55cb824ad110 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1b5b77b80 10 client.0 _ll_register_callbacks cb 0x55cb82385630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.722+0000 7fd1a9891700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 0x55cb8252b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a9891700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 0x55cb8252b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58550/0 (socket says 172.21.15.52:58550) 2024-06-02T23:14:20.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a9891700 1 -- 172.21.15.52:0/2146210363 learned_addr learned my addr 172.21.15.52:0/2146210363 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:20.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a888f700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ad110 0x55cb824b5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a888f700 1 -- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 msgr2=0x55cb824b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a888f700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a888f700 1 -- 172.21.15.52:0/2146210363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 msgr2=0x55cb8252b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a888f700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 0x55cb8252b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a888f700 1 -- 172.21.15.52:0/2146210363 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb824b68a0 con 0x55cb824ad110 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.723+0000 7fd1a9891700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 0x55cb8252b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd1a888f700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ad110 0x55cb824b5ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd19400b2e0 tx=0x7fd194022620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab408d0388d48c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd18cff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd18cff9700 1 -- 172.21.15.52:0/2146210363 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1940b4070 con 0x55cb824ad110 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd18cff9700 1 -- 172.21.15.52:0/2146210363 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd1940870b0 con 0x55cb824ad110 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd1a0023070 con 0x55cb824ad110 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd18cff9700 1 -- 172.21.15.52:0/2146210363 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fd1940a0b70 con 0x55cb824ad110 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd1b5b77b80 10 client.25828 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.724+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cb8253d8b0 con 0x55cb824ad110 2024-06-02T23:14:20.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd18cff9700 1 -- 172.21.15.52:0/2146210363 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd1a0023070 con 0x55cb824ad110 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd18cff9700 1 -- 172.21.15.52:0/2146210363 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fd19409ca20 con 0x55cb824ad110 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1b5b77b80 10 client.25828 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1b5b77b80 10 client.25828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1b5b77b80 20 client.25828 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8253ded0 0x55cb825402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1b5b77b80 4 client.25828 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cb823f7350 con 0x55cb8253ded0 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.725+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8253ded0 0x55cb825402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.726+0000 7fd1a9090700 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8253ded0 0x55cb825402f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b098735f8f1dfa70 server_cookie=7cdaf937a4d57a81 in_seq=0 out_seq=0 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.726+0000 7fd18cff9700 10 client.25828 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd18cff9700 1 -- 172.21.15.52:0/2146210363 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cb823f7350 con 0x55cb8253ded0 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd18cff9700 10 client.25828 handle_command_reply: tid=0 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 client.25828 shutdown 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ad110 msgr2=0x55cb824b5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ad110 0x55cb824b5ad0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd19400b2e0 tx=0x7fd194022620 comp rx=0 tx=0).stop 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 shutdown_connections 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8253ded0 0x55cb825402f0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb824b6170 0x55cb8252b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd18cff9700 0 client.25828 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb824ac7f0 0x55cb824b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 --2- 172.21.15.52:0/2146210363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb824ad110 0x55cb824b5ad0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 >> 172.21.15.52:0/2146210363 conn(0x55cb823cb100 msgr2=0x55cb824ab270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 shutdown_connections 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 1 -- 172.21.15.52:0/2146210363 wait complete. 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.727+0000 7fd1b5b77b80 20 client.25828 trim_cache size 0 max 16384 2024-06-02T23:14:20.729 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:14:20.730 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:14:20.730 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "5dfd123d-a283-4e03-b10d-d4f7964935e4", 2024-06-02T23:14:20.730 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:14:20.730 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.728+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1900a2ca0 msgr2=0x7fd1900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.728+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1900a2ca0 0x7fd1900a5160 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd1a0007b60 tx=0x7fd1a00073d0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 msgr2=0x7fd1a419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a419b270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd194000c00 tx=0x7fd1940047b0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 shutdown_connections 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1900a2ca0 0x7fd1900a5160 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd1a410a6a0 0x7fd1a419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd1a41046a0 0x7fd1a419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 --2- 172.21.15.52:0/1649817353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd1a4105060 0x7fd1a419b270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 >> 172.21.15.52:0/1649817353 conn(0x7fd1a4100170 msgr2=0x7fd1a41007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 shutdown_connections 2024-06-02T23:14:20.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:20.729+0000 7fd1ab2f4700 1 -- 172.21.15.52:0/1649817353 wait complete. 2024-06-02T23:14:21.381 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: pgmap v505: 97 pgs: 97 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-02T23:14:21.381 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:21.381 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.381 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:21.382 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.382 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:21.382 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:21 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: pgmap v505: 97 pgs: 97 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:21 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: pgmap v505: 97 pgs: 97 active+clean; 39 MiB data, 566 MiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 1.3 MiB/s wr, 157 op/s 2024-06-02T23:14:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:21 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:21.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 43647 2.80 MB/sec warmup 90 sec latency 14.576 ms 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:14:22.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:22.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:14:22.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:14:22.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44219 2.81 MB/sec warmup 91 sec latency 16.274 ms 2024-06-02T23:14:22.745 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:14:23.237 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:23 smithi052 ceph-mon[32340]: pgmap v506: 97 pgs: 97 active+clean; 39 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:14:23.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 -- 172.21.15.52:0/1987342322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c06b0a0 msgr2=0x7fdf8c06b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 --2- 172.21.15.52:0/1987342322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c06b520 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009990 tx=0x7fdf80009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 -- 172.21.15.52:0/1987342322 shutdown_connections 2024-06-02T23:14:23.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 --2- 172.21.15.52:0/1987342322 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf8c10ed20 0x7fdf8c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 --2- 172.21.15.52:0/1987342322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c1033b0 0x7fdf8c06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 --2- 172.21.15.52:0/1987342322 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c06b520 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.275+0000 7fdf912e1700 1 -- 172.21.15.52:0/1987342322 >> 172.21.15.52:0/1987342322 conn(0x7fdf8c0780f0 msgr2=0x7fdf8c078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:23.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.276+0000 7fdf912e1700 1 -- 172.21.15.52:0/1987342322 shutdown_connections 2024-06-02T23:14:23.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.276+0000 7fdf912e1700 1 -- 172.21.15.52:0/1987342322 wait complete. 2024-06-02T23:14:23.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.276+0000 7fdf912e1700 1 Processor -- start 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 -- start start 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf8c1033b0 0x7fdf8c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c10ed20 0x7fdf8c19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c1148e0 con 0x7fdf8c10ed20 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c114760 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf912e1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c114a60 con 0x7fdf8c1033b0 2024-06-02T23:14:23.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c19ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46748/0 (socket says 172.21.15.52:46748) 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf8affd700 1 -- 172.21.15.52:0/2631707566 learned_addr learned my addr 172.21.15.52:0/2631707566 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf8b7fe700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c10ed20 0x7fdf8c19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.277+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf8c1033b0 0x7fdf8c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf8affd700 1 -- 172.21.15.52:0/2631707566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf8c1033b0 msgr2=0x7fdf8c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf8c1033b0 0x7fdf8c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf8affd700 1 -- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c10ed20 msgr2=0x7fdf8c19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c10ed20 0x7fdf8c19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf8affd700 1 -- 172.21.15.52:0/2631707566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8c19fd30 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c19ad70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00d980 tx=0x7fdf7c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf73fff700 1 -- 172.21.15.52:0/2631707566 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c016070 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8c1a0020 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf73fff700 1 -- 172.21.15.52:0/2631707566 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf7c004830 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf73fff700 1 -- 172.21.15.52:0/2631707566 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c005530 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.278+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8c1a78a0 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.281+0000 7fdf73fff700 1 -- 172.21.15.52:0/2631707566 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf7c0049e0 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.282+0000 7fdf73fff700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdf740a2d40 0x7fdf740a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.282+0000 7fdf73fff700 1 -- 172.21.15.52:0/2631707566 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdf7c0ced10 con 0x7fdf8c06b0a0 2024-06-02T23:14:23.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.282+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdf740a2d40 0x7fdf740a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.283+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdf740a2d40 0x7fdf740a5200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0788a0 tx=0x7fdf80005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:23.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.283+0000 7fdf9bb64b80 1 Processor -- start 2024-06-02T23:14:23.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.283+0000 7fdf9bb64b80 1 -- start start 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.283+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c37153f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.283+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153f600 0x55c371538f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c371539470 0x55c3715398b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c37153f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c37148a9b0 con 0x55c37153f600 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c3714328c0 con 0x55c37153ece0 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c371486970 con 0x55c371539470 2024-06-02T23:14:23.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c37153f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46750/0 (socket says 172.21.15.52:46750) 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 -- 172.21.15.52:0/207156891 learned_addr learned my addr 172.21.15.52:0/207156891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153f600 0x55c371538f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8b7fe700 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c371539470 0x55c3715398b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 -- 172.21.15.52:0/207156891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c371539470 msgr2=0x55c3715398b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c371539470 0x55c3715398b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 -- 172.21.15.52:0/207156891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153f600 msgr2=0x55c371538f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153f600 0x55c371538f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 -- 172.21.15.52:0/207156891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3714290f0 con 0x55c37153ece0 2024-06-02T23:14:23.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf8affd700 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c37153f0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0a3c80 tx=0x7fdf7c09af20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fcd71af52900d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:23.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf717fa700 1 -- 172.21.15.52:0/207156891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c09bc40 con 0x55c37153ece0 2024-06-02T23:14:23.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.284+0000 7fdf717fa700 1 -- 172.21.15.52:0/207156891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf7c0bb050 con 0x55c37153ece0 2024-06-02T23:14:23.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.285+0000 7fdf717fa700 1 -- 172.21.15.52:0/207156891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a3f40 con 0x55c37153ece0 2024-06-02T23:14:23.285 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:23.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.285+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/207156891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 msgr2=0x55c37153f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.285+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c37153f0c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0a3c80 tx=0x7fdf7c09af20 comp rx=0 tx=0).stop 2024-06-02T23:14:23.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.285+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/207156891 shutdown_connections 2024-06-02T23:14:23.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.285+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c371539470 0x55c3715398b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.285+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c37153f0c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.286+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/207156891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153f600 0x55c371538f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.286+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/207156891 >> 172.21.15.52:0/207156891 conn(0x55c371463e60 msgr2=0x55c3715341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.286+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/207156891 shutdown_connections 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.286+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/207156891 wait complete. 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.286+0000 7fdf9bb64b80 1 -- start start 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.286+0000 7fdf9bb64b80 1 Processor -- start 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 0x55c371545340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c371542640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371542b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c371546010 con 0x55c371539470 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c37142ad90 con 0x55c37153ece0 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c371546190 con 0x55c37153f600 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 0x55c371545340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf9bb64b80 10 client.0 _ll_register_callbacks cb 0x55c371424ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:23.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 0x55c371545340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58594/0 (socket says 172.21.15.52:58594) 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 -- 172.21.15.52:0/2861222254 learned_addr learned my addr 172.21.15.52:0/2861222254 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c371542640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8b7fe700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371542b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 -- 172.21.15.52:0/2861222254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 msgr2=0x55c371542b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371542b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 -- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 msgr2=0x55c371542640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c371542640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 -- 172.21.15.52:0/2861222254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3713d9a70 con 0x55c371539470 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c371542640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 0x55c371545340 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c012040 tx=0x7fdf7c09f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc675629570fc5b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.287+0000 7fdf6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf6eff5700 1 -- 172.21.15.52:0/2861222254 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0b8070 con 0x55c371539470 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf6eff5700 1 -- 172.21.15.52:0/2861222254 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdf7c0a3d20 con 0x55c371539470 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf7c0bb050 con 0x55c371539470 2024-06-02T23:14:23.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf6eff5700 1 -- 172.21.15.52:0/2861222254 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c09bc40 con 0x55c371539470 2024-06-02T23:14:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf9bb64b80 10 client.16344 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c371550920 con 0x55c371539470 2024-06-02T23:14:23.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.288+0000 7fdf6eff5700 1 -- 172.21.15.52:0/2861222254 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf7c0bb050 con 0x55c371539470 2024-06-02T23:14:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf6eff5700 1 -- 172.21.15.52:0/2861222254 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fdf7c09aa00 con 0x55c371539470 2024-06-02T23:14:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf9bb64b80 10 client.16344 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf9bb64b80 10 client.16344 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:14:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf9bb64b80 20 client.16344 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:23.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf9bb64b80 4 client.16344 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c371497eb0 con 0x55c371552df0 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf6eff5700 10 client.16344 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.289+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715551e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc77edc7d95ae253 server_cookie=f012b46877bd6819 in_seq=0 out_seq=0 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.290+0000 7fdf6eff5700 1 -- 172.21.15.52:0/2861222254 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c371497eb0 con 0x55c371552df0 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.290+0000 7fdf6eff5700 10 client.16344 handle_command_reply: tid=0 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.290+0000 7fdf9bb64b80 1 client.16344 shutdown 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 msgr2=0x55c371545340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 0x55c371545340 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c012040 tx=0x7fdf7c09f960 comp rx=0 tx=0).stop 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 shutdown_connections 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715551e0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371542b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c371539470 0x55c371545340 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2861222254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c37153ece0 0x55c371542640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf6eff5700 0 client.16344 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 >> 172.21.15.52:0/2861222254 conn(0x55c371463e60 msgr2=0x55c371537f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 shutdown_connections 2024-06-02T23:14:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2861222254 wait complete. 2024-06-02T23:14:23.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.291+0000 7fdf9bb64b80 20 client.16344 trim_cache size 0 max 16384 2024-06-02T23:14:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.318+0000 7fdf9bb64b80 1 Processor -- start 2024-06-02T23:14:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.318+0000 7fdf9bb64b80 1 -- start start 2024-06-02T23:14:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.318+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371541640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.318+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c3715338d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371533e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c1aa970 con 0x55c37153ece0 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c1ac540 con 0x55c371539470 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c1ab040 con 0x55c37153f600 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c3715338d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c3715338d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58608/0 (socket says 172.21.15.52:58608) 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 -- 172.21.15.52:0/2325398179 learned_addr learned my addr 172.21.15.52:0/2325398179 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8affd700 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371541640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8b7fe700 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371533e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 -- 172.21.15.52:0/2325398179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 msgr2=0x55c371533e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371533e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 -- 172.21.15.52:0/2325398179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 msgr2=0x55c371541640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371541640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 -- 172.21.15.52:0/2325398179 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3715461e0 con 0x55c37153ece0 2024-06-02T23:14:23.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.319+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c3715338d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005f50 tx=0x7fdf80028040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa2b56b95e02933 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.320+0000 7fdf70ff9700 1 -- 172.21.15.52:0/2325398179 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80003680 con 0x55c37153ece0 2024-06-02T23:14:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.320+0000 7fdf70ff9700 1 -- 172.21.15.52:0/2325398179 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf80003cd0 con 0x55c37153ece0 2024-06-02T23:14:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.320+0000 7fdf70ff9700 1 -- 172.21.15.52:0/2325398179 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000fd80 con 0x55c37153ece0 2024-06-02T23:14:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2325398179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 msgr2=0x55c3715338d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c3715338d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005f50 tx=0x7fdf80028040 comp rx=0 tx=0).stop 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2325398179 shutdown_connections 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c371533e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371541640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/2325398179 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c3715338d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2325398179 >> 172.21.15.52:0/2325398179 conn(0x55c371463e60 msgr2=0x55c371540f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2325398179 shutdown_connections 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/2325398179 wait complete. 2024-06-02T23:14:23.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.321+0000 7fdf9bb64b80 1 -- start start 2024-06-02T23:14:23.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf9bb64b80 1 Processor -- start 2024-06-02T23:14:23.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371556620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c371556b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf9bb64b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c3715bc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371556620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371556620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46782/0 (socket says 172.21.15.52:46782) 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf8affd700 1 -- 172.21.15.52:0/1640307941 learned_addr learned my addr 172.21.15.52:0/1640307941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.322+0000 7fdf9bb64b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c1abde0 con 0x55c37153ece0 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c1ab710 con 0x55c371539470 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c371556b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf8c107830 con 0x55c37153f600 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8b7fe700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c3715bc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf9bb64b80 10 client.0 _ll_register_callbacks cb 0x55c371424ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8affd700 1 -- 172.21.15.52:0/1640307941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 msgr2=0x55c3715bc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8affd700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c3715bc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8affd700 1 -- 172.21.15.52:0/1640307941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 msgr2=0x55c371556b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8affd700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c371556b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8affd700 1 -- 172.21.15.52:0/1640307941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3715bcc70 con 0x55c371539470 2024-06-02T23:14:23.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c371556b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8b7fe700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c3715bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf8affd700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371556620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0972e0 tx=0x7fdf7c09f8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e269d8669c58bb82 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:23.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf6eff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf6eff5700 1 -- 172.21.15.52:0/1640307941 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0a8060 con 0x55c371539470 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf6eff5700 1 -- 172.21.15.52:0/1640307941 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdf7c0b4260 con 0x55c371539470 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.323+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf80003cd0 con 0x55c371539470 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf6eff5700 1 -- 172.21.15.52:0/1640307941 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c0b8070 con 0x55c371539470 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 10 client.25934 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c3715bcc70 con 0x55c371539470 2024-06-02T23:14:23.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf6eff5700 1 -- 172.21.15.52:0/1640307941 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf80003cd0 con 0x55c371539470 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf6eff5700 1 -- 172.21.15.52:0/1640307941 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fdf7c084b20 con 0x55c371539470 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 10 client.25934 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 10 client.25934 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 20 client.25934 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715c0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 4 client.25934 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.75, "await_for": 45.0}] 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.324+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.75, "await_for": 45.0}) -- 0x55c37155b3f0 con 0x55c371552df0 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.325+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715c0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.325+0000 7fdf6eff5700 10 client.25934 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:23.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.325+0000 7fdf8a7fc700 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715c0290 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd8ab77ec601d753 server_cookie=411d5f7dda5c1c2a in_seq=0 out_seq=0 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf6eff5700 1 -- 172.21.15.52:0/1640307941 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c37155b3f0 con 0x55c371552df0 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf6eff5700 10 client.25934 handle_command_reply: tid=0 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 client.25934 shutdown 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 msgr2=0x55c371556620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371556620 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0972e0 tx=0x7fdf7c09f8f0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 shutdown_connections 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c371552df0 0x55c3715c0290 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c37153f600 0x55c3715bc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c371539470 0x55c371556620 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf6eff5700 0 client.25934 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 --2- 172.21.15.52:0/1640307941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c37153ece0 0x55c371556b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 >> 172.21.15.52:0/1640307941 conn(0x55c371463e60 msgr2=0x55c3715401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 shutdown_connections 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 1 -- 172.21.15.52:0/1640307941 wait complete. 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.388+0000 7fdf9bb64b80 20 client.25934 trim_cache size 0 max 16384 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 10, 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "a049db0d": { 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 10, 2024-06-02T23:14:23.389 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.8, 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:14:23.390 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdf740a2d40 msgr2=0x7fdf740a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdf740a2d40 0x7fdf740a5200 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fdf8c0788a0 tx=0x7fdf80005fd0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 msgr2=0x7fdf8c19ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c19ad70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00d980 tx=0x7fdf7c00dc90 comp rx=0 tx=0).stop 2024-06-02T23:14:23.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 shutdown_connections 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdf740a2d40 0x7fdf740a5200 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf8c10ed20 0x7fdf8c19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf8c06b0a0 0x7fdf8c19ad70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 --2- 172.21.15.52:0/2631707566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf8c1033b0 0x7fdf8c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 >> 172.21.15.52:0/2631707566 conn(0x7fdf8c0780f0 msgr2=0x7fdf8c1006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 shutdown_connections 2024-06-02T23:14:23.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:23.389+0000 7fdf912e1700 1 -- 172.21.15.52:0/2631707566 wait complete. 2024-06-02T23:14:23.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:23 smithi174 ceph-mon[35994]: pgmap v506: 97 pgs: 97 active+clean; 39 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:14:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:23 smithi123 ceph-mon[35977]: pgmap v506: 97 pgs: 97 active+clean; 39 MiB data, 571 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:14:23.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.78 MB/sec warmup 92 sec latency 227.300 ms 2024-06-02T23:14:24.317 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a049db0d', quiesce took 12.8 seconds. Will release after: 14.95 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 -- 172.21.15.52:0/1837684781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 msgr2=0x7fb544109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 --2- 172.21.15.52:0/1837684781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb544109b30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009ca0 tx=0x7fb53c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 -- 172.21.15.52:0/1837684781 shutdown_connections 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 --2- 172.21.15.52:0/1837684781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb544104db0 0x7fb5441071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 --2- 172.21.15.52:0/1837684781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb544109b30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 --2- 172.21.15.52:0/1837684781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb544102480 0x7fb544104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 -- 172.21.15.52:0/1837684781 >> 172.21.15.52:0/1837684781 conn(0x7fb5440fbe70 msgr2=0x7fb5440fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 -- 172.21.15.52:0/1837684781 shutdown_connections 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.602+0000 7fb54ac38640 1 -- 172.21.15.52:0/1837684781 wait complete. 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 Processor -- start 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 -- start start 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb544102480 0x7fb544197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb544104db0 0x7fb5441980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb54419f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb54410d5a0 con 0x7fb5441076e0 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb54410d420 con 0x7fb544102480 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb54ac38640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb54410d720 con 0x7fb544104db0 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb5491d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb54419f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb5491d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb54419f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38688/0 (socket says 172.21.15.52:38688) 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb5491d5640 1 -- 172.21.15.52:0/4170891547 learned_addr learned my addr 172.21.15.52:0/4170891547 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb5489d4640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb544102480 0x7fb544197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.603+0000 7fb543fff640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb544104db0 0x7fb5441980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb5491d5640 1 -- 172.21.15.52:0/4170891547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb544104db0 msgr2=0x7fb5441980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb5491d5640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb544104db0 0x7fb5441980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb5491d5640 1 -- 172.21.15.52:0/4170891547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb544102480 msgr2=0x7fb544197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb5491d5640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb544102480 0x7fb544197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb5491d5640 1 -- 172.21.15.52:0/4170891547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54419f7d0 con 0x7fb5441076e0 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb5491d5640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb54419f160 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009dc0 tx=0x7fb53c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb541ffb640 1 -- 172.21.15.52:0/4170891547 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c004040 con 0x7fb5441076e0 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb541ffb640 1 -- 172.21.15.52:0/4170891547 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb53c004690 con 0x7fb5441076e0 2024-06-02T23:14:24.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54419fa60 con 0x7fb5441076e0 2024-06-02T23:14:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb541ffb640 1 -- 172.21.15.52:0/4170891547 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53c02f4c0 con 0x7fb5441076e0 2024-06-02T23:14:24.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.604+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5441a0000 con 0x7fb5441076e0 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.606+0000 7fb541ffb640 1 -- 172.21.15.52:0/4170891547 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb53c030650 con 0x7fb5441076e0 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 Processor -- start 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 -- start start 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f978046430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f97804b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f97804bb70 0x55f97804e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f977fa94f0 con 0x55f978046010 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f977f47870 con 0x55f97804bb70 2024-06-02T23:14:24.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb54da27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f977fa54b0 con 0x55f978046a00 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb543fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f978046430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb543fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f978046430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38700/0 (socket says 172.21.15.52:38700) 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb543fff640 1 -- 172.21.15.52:0/2324504216 learned_addr learned my addr 172.21.15.52:0/2324504216 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb5489d4640 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f97804b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb543fff640 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f97804bb70 0x55f97804e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.607+0000 7fb541ffb640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb5340a2cb0 0x7fb5340a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb543fff640 1 -- 172.21.15.52:0/2324504216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 msgr2=0x55f97804b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb543fff640 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f97804b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb543fff640 1 -- 172.21.15.52:0/2324504216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f97804bb70 msgr2=0x55f97804e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb543fff640 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f97804bb70 0x55f97804e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb543fff640 1 -- 172.21.15.52:0/2324504216 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f977f639b0 con 0x55f978046010 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb5491d5640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb5340a2cb0 0x7fb5340a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb543fff640 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f978046430 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb53000bef0 tx=0x7fb53000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e6d9b94e24ea61f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb541ffb640 1 -- 172.21.15.52:0/4170891547 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb53c0f0820 con 0x7fb5441076e0 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb52effd640 1 -- 172.21.15.52:0/2324504216 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300173f0 con 0x55f978046010 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb52effd640 1 -- 172.21.15.52:0/2324504216 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb530010040 con 0x55f978046010 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.608+0000 7fb52effd640 1 -- 172.21.15.52:0/2324504216 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300167e0 con 0x55f978046010 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 -- 172.21.15.52:0/2324504216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 msgr2=0x55f978046430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb5491d5640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb5340a2cb0 0x7fb5340a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0900e0 tx=0x7fb53c090110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f978046430 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb53000bef0 tx=0x7fb53000b400 comp rx=0 tx=0).stop 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 -- 172.21.15.52:0/2324504216 shutdown_connections 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f97804b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f97804bb70 0x55f97804e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 --2- 172.21.15.52:0/2324504216 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f978046430 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 -- 172.21.15.52:0/2324504216 >> 172.21.15.52:0/2324504216 conn(0x55f978041680 msgr2=0x55f978041a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:24.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 -- 172.21.15.52:0/2324504216 shutdown_connections 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 -- 172.21.15.52:0/2324504216 wait complete. 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 -- start start 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.609+0000 7fb54da27740 1 Processor -- start 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f978050260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 0x55f9780507a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780577e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f97804eac0 con 0x55f978046a00 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f977f32ae0 con 0x55f978046010 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f97804ec40 con 0x55f97804bb70 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 10 client.0 _ll_register_callbacks cb 0x55f977f6b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 0x55f9780507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 0x55f9780507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38714/0 (socket says 172.21.15.52:38714) 2024-06-02T23:14:24.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 -- 172.21.15.52:0/3212460738 learned_addr learned my addr 172.21.15.52:0/3212460738 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb543fff640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f978050260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb543fff640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 -- 172.21.15.52:0/3212460738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 msgr2=0x55f9780577e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780577e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 -- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 msgr2=0x55f978050260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f978050260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 -- 172.21.15.52:0/3212460738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f977cbd210 con 0x55f978046a00 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb543fff640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f978050260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb5489d4640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 0x55f9780507a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb538010820 tx=0x7fb538010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f7f570a848edef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb523fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb523fff640 1 -- 172.21.15.52:0/3212460738 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53800fab0 con 0x55f978046a00 2024-06-02T23:14:24.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb523fff640 1 -- 172.21.15.52:0/3212460738 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb53800fc50 con 0x55f978046a00 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.610+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb530010040 con 0x55f978046a00 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb523fff640 1 -- 172.21.15.52:0/3212460738 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb53800fe50 con 0x55f978046a00 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 10 client.16362 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f977d89620 con 0x55f978046a00 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb523fff640 1 -- 172.21.15.52:0/3212460738 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb530010040 con 0x55f978046a00 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb523fff640 1 -- 172.21.15.52:0/3212460738 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb538025480 con 0x55f978046a00 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 10 client.16362 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 10 client.16362 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 20 client.16362 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f97805c200 0x55f97805e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 4 client.16362 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:24.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f977fbcf10 con 0x55f97805c200 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.611+0000 7fb543fff640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f97805c200 0x55f97805e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.612+0000 7fb543fff640 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f97805c200 0x55f97805e610 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d12c6595aed444c3 server_cookie=3367949e5cfec958 in_seq=0 out_seq=0 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.612+0000 7fb523fff640 10 client.16362 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb523fff640 1 -- 172.21.15.52:0/3212460738 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f977fbcf10 con 0x55f97805c200 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb523fff640 10 client.16362 handle_command_reply: tid=0 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 client.16362 shutdown 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 msgr2=0x55f9780507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 0x55f9780507a0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb538010820 tx=0x7fb538010ce0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 shutdown_connections 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f97805c200 0x55f97805e610 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780577e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb523fff640 0 client.16362 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046a00 0x55f9780507a0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 --2- 172.21.15.52:0/3212460738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f978050260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 >> 172.21.15.52:0/3212460738 conn(0x55f978041680 msgr2=0x55f9780407e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 shutdown_connections 2024-06-02T23:14:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 1 -- 172.21.15.52:0/3212460738 wait complete. 2024-06-02T23:14:24.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.613+0000 7fb54da27740 20 client.16362 trim_cache size 0 max 16384 2024-06-02T23:14:24.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.75 MB/sec warmup 93 sec latency 1227.344 ms 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.628+0000 7fb54da27740 1 Processor -- start 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.628+0000 7fb54da27740 1 -- start start 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.628+0000 7fb54da27740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f97805e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.628+0000 7fb54da27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f9780577a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.628+0000 7fb54da27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 0x55f978047c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.628+0000 7fb54da27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb5441a8c10 con 0x55f97804bb70 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb54da27740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5441aa7e0 con 0x55f978046010 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb54da27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5441a92e0 con 0x55f978046a00 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 0x55f978047c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 0x55f978047c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38720/0 (socket says 172.21.15.52:38720) 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 -- 172.21.15.52:0/3620973050 learned_addr learned my addr 172.21.15.52:0/3620973050 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f97805e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb5489d4640 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f9780577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 -- 172.21.15.52:0/3620973050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 msgr2=0x55f9780577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f9780577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 -- 172.21.15.52:0/3620973050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 msgr2=0x55f97805e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f97805e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 -- 172.21.15.52:0/3620973050 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f97804af40 con 0x55f97804bb70 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb5489d4640 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f9780577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb543fff640 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 0x55f978047c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb53000bef0 tx=0x7fb53000b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea0db83cad6c079 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb52e7fc640 1 -- 172.21.15.52:0/3620973050 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5300175f0 con 0x55f97804bb70 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb52e7fc640 1 -- 172.21.15.52:0/3620973050 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb530017c40 con 0x55f97804bb70 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.629+0000 7fb52e7fc640 1 -- 172.21.15.52:0/3620973050 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530026e30 con 0x55f97804bb70 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 -- 172.21.15.52:0/3620973050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 msgr2=0x55f978047c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 0x55f978047c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb53000bef0 tx=0x7fb53000b400 comp rx=0 tx=0).stop 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 -- 172.21.15.52:0/3620973050 shutdown_connections 2024-06-02T23:14:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f978046a00 0x55f9780577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f97804bb70 0x55f978047c80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 --2- 172.21.15.52:0/3620973050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046010 0x55f97805e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 -- 172.21.15.52:0/3620973050 >> 172.21.15.52:0/3620973050 conn(0x55f978041680 msgr2=0x55f97804b2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 -- 172.21.15.52:0/3620973050 shutdown_connections 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 -- 172.21.15.52:0/3620973050 wait complete. 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 -- start start 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.630+0000 7fb54da27740 1 Processor -- start 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f9780545e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046a00 0x55f978054b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb5441aa080 con 0x55f978046010 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5441a99b0 con 0x55f978046a00 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5441a6bf0 con 0x55f97804bb70 2024-06-02T23:14:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 10 client.0 _ll_register_callbacks cb 0x7fb53800a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f9780545e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f9780545e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38722/0 (socket says 172.21.15.52:38722) 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 -- 172.21.15.52:0/1655070657 learned_addr learned my addr 172.21.15.52:0/1655070657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb5489d4640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046a00 0x55f978054b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 -- 172.21.15.52:0/1655070657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 msgr2=0x55f9780c4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780c4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 -- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046a00 msgr2=0x55f978054b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046a00 0x55f978054b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 -- 172.21.15.52:0/1655070657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9780c5420 con 0x55f978046010 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb543fff640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f9780545e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb53000bd20 tx=0x7fb53000b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d409dc445694234a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb523fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb523fff640 1 -- 172.21.15.52:0/1655070657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb530026db0 con 0x55f978046010 2024-06-02T23:14:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb523fff640 1 -- 172.21.15.52:0/1655070657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb5300059c0 con 0x55f978046010 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.631+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb530017c40 con 0x55f978046010 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb523fff640 1 -- 172.21.15.52:0/1655070657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb530005bc0 con 0x55f978046010 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 10 client.16368 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f9780c6580 con 0x55f978046010 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb523fff640 1 -- 172.21.15.52:0/1655070657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb530017c40 con 0x55f978046010 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb523fff640 1 -- 172.21.15.52:0/1655070657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb530006190 con 0x55f978046010 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 10 client.16368 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 20 client.16368 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f9780c6be0 0x55f9780c8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 4 client.16368 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f977fb6c20 con 0x55f9780c6be0 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.632+0000 7fb5489d4640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f9780c6be0 0x55f9780c8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.633+0000 7fb5489d4640 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f9780c6be0 0x55f9780c8fd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad42a56b2fec50d7 server_cookie=5f841829348da112 in_seq=0 out_seq=0 2024-06-02T23:14:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.633+0000 7fb523fff640 10 client.16368 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:24.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb523fff640 1 -- 172.21.15.52:0/1655070657 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f977fb6c20 con 0x55f9780c6be0 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb523fff640 10 client.16368 handle_command_reply: tid=0 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 client.16368 shutdown 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 msgr2=0x55f9780545e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f9780545e0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb53000bd20 tx=0x7fb53000b360 comp rx=0 tx=0).stop 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 shutdown_connections 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f9780c6be0 0x55f9780c8fd0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f97804bb70 0x55f9780c4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb523fff640 0 client.16368 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f978046a00 0x55f978054b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 --2- 172.21.15.52:0/1655070657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f978046010 0x55f9780545e0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 >> 172.21.15.52:0/1655070657 conn(0x55f978041680 msgr2=0x55f978047c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:24.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 shutdown_connections 2024-06-02T23:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 1 -- 172.21.15.52:0/1655070657 wait complete. 2024-06-02T23:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.650+0000 7fb54da27740 20 client.16368 trim_cache size 0 max 16384 2024-06-02T23:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:14:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb5340a2cb0 msgr2=0x7fb5340a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb5340a2cb0 0x7fb5340a5170 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb53c0900e0 tx=0x7fb53c090110 comp rx=0 tx=0).stop 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 msgr2=0x7fb54419f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb54419f160 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009dc0 tx=0x7fb53c003ba0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 shutdown_connections 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb5340a2cb0 0x7fb5340a5170 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb544104db0 0x7fb5441980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb5441076e0 0x7fb54419f160 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 --2- 172.21.15.52:0/4170891547 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb544102480 0x7fb544197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 >> 172.21.15.52:0/4170891547 conn(0x7fb5440fbe70 msgr2=0x7fb544102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 shutdown_connections 2024-06-02T23:14:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:24.651+0000 7fb54ac38640 1 -- 172.21.15.52:0/4170891547 wait complete. 2024-06-02T23:14:25.360 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:25 smithi052 ceph-mon[32340]: pgmap v507: 97 pgs: 97 active+clean; 40 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:14:25.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:25 smithi174 ceph-mon[35994]: pgmap v507: 97 pgs: 97 active+clean; 40 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:14:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:25 smithi123 ceph-mon[35977]: pgmap v507: 97 pgs: 97 active+clean; 40 MiB data, 580 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:14:25.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.72 MB/sec warmup 94 sec latency 2227.423 ms 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 -- 172.21.15.52:0/1308946377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c101440 msgr2=0x7fe50c10f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 --2- 172.21.15.52:0/1308946377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c101440 0x7fe50c10f330 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe504009990 tx=0x7fe504009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 -- 172.21.15.52:0/1308946377 shutdown_connections 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 --2- 172.21.15.52:0/1308946377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c101440 0x7fe50c10f330 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 --2- 172.21.15.52:0/1308946377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe50c108510 0x7fe50c1088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 --2- 172.21.15.52:0/1308946377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50c100a80 0x7fe50c100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 -- 172.21.15.52:0/1308946377 >> 172.21.15.52:0/1308946377 conn(0x7fe50c077a00 msgr2=0x7fe50c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 -- 172.21.15.52:0/1308946377 shutdown_connections 2024-06-02T23:14:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.914+0000 7fe512216700 1 -- 172.21.15.52:0/1308946377 wait complete. 2024-06-02T23:14:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 Processor -- start 2024-06-02T23:14:25.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 -- start start 2024-06-02T23:14:25.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50c100a80 0x7fe50c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe50c101440 0x7fe50c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 0x7fe50c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c112640 con 0x7fe50c108510 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1124c0 con 0x7fe50c100a80 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe512216700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1127c0 con 0x7fe50c101440 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 0x7fe50c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 0x7fe50c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38746/0 (socket says 172.21.15.52:38746) 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe50bfff700 1 -- 172.21.15.52:0/3521592244 learned_addr learned my addr 172.21.15.52:0/3521592244 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe502ffd700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe50c101440 0x7fe50c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.915+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50c100a80 0x7fe50c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe50bfff700 1 -- 172.21.15.52:0/3521592244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe50c101440 msgr2=0x7fe50c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe50bfff700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe50c101440 0x7fe50c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe50bfff700 1 -- 172.21.15.52:0/3521592244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50c100a80 msgr2=0x7fe50c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe50bfff700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50c100a80 0x7fe50c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe50bfff700 1 -- 172.21.15.52:0/3521592244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c19fd80 con 0x7fe50c108510 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe50bfff700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 0x7fe50c19f640 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe504005910 tx=0x7fe5040047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:25.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe5097fa700 1 -- 172.21.15.52:0/3521592244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504004830 con 0x7fe50c108510 2024-06-02T23:14:25.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c1a0010 con 0x7fe50c108510 2024-06-02T23:14:25.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe5097fa700 1 -- 172.21.15.52:0/3521592244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe504005020 con 0x7fe50c108510 2024-06-02T23:14:25.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe5097fa700 1 -- 172.21.15.52:0/3521592244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504003cc0 con 0x7fe50c108510 2024-06-02T23:14:25.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.916+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c1a78f0 con 0x7fe50c108510 2024-06-02T23:14:25.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.918+0000 7fe5097fa700 1 -- 172.21.15.52:0/3521592244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe5040051d0 con 0x7fe50c108510 2024-06-02T23:14:25.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.919+0000 7fe5097fa700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4f80a2ca0 0x7fe4f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.919+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4f80a2ca0 0x7fe4f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.919+0000 7fe5097fa700 1 -- 172.21.15.52:0/3521592244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe5040cfcc0 con 0x7fe50c108510 2024-06-02T23:14:25.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4f80a2ca0 0x7fe4f80a5160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc007fd0 tx=0x7fe4fc00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 Processor -- start 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 -- start start 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c986a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c9809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c981340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55856c8d2810 con 0x55856c986f60 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55856c87a720 con 0x55856c980f20 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55856c8ce7d0 con 0x55856c986640 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe502ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c9809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe502ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c9809e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38754/0 (socket says 172.21.15.52:38754) 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe502ffd700 1 -- 172.21.15.52:0/3184490579 learned_addr learned my addr 172.21.15.52:0/3184490579 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c986a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe50bfff700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c981340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.920+0000 7fe502ffd700 1 -- 172.21.15.52:0/3184490579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 msgr2=0x55856c986a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe502ffd700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c986a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe502ffd700 1 -- 172.21.15.52:0/3184490579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 msgr2=0x55856c981340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe502ffd700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c981340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe502ffd700 1 -- 172.21.15.52:0/3184490579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55856c870f50 con 0x55856c986f60 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe50bfff700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c981340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c986a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe502ffd700 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c9809e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400bd30 tx=0x7fe4f40106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c0c7a187e41cb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe501ffb700 1 -- 172.21.15.52:0/3184490579 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f4004030 con 0x55856c986f60 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe501ffb700 1 -- 172.21.15.52:0/3184490579 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4f4003680 con 0x55856c986f60 2024-06-02T23:14:25.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.921+0000 7fe501ffb700 1 -- 172.21.15.52:0/3184490579 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4f40116a0 con 0x55856c986f60 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 -- 172.21.15.52:0/3184490579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 msgr2=0x55856c9809e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c9809e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fe4f400bd30 tx=0x7fe4f40106b0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 -- 172.21.15.52:0/3184490579 shutdown_connections 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c981340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c986a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/3184490579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c9809e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 -- 172.21.15.52:0/3184490579 >> 172.21.15.52:0/3184490579 conn(0x55856c8abcc0 msgr2=0x55856c97b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 -- 172.21.15.52:0/3184490579 shutdown_connections 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 -- 172.21.15.52:0/3184490579 wait complete. 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 -- start start 2024-06-02T23:14:25.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.922+0000 7fe51ca99b80 1 Processor -- start 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c98cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c98a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55856c98d9d0 con 0x55856c986f60 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55856c872bf0 con 0x55856c986640 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55856c98db50 con 0x55856c980f20 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 10 client.0 _ll_register_callbacks cb 0x55856c86c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38768/0 (socket says 172.21.15.52:38768) 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 -- 172.21.15.52:0/72182262 learned_addr learned my addr 172.21.15.52:0/72182262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe502ffd700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c98a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c98cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 -- 172.21.15.52:0/72182262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 msgr2=0x55856c98cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c98cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 -- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 msgr2=0x55856c98a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c98a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 -- 172.21.15.52:0/72182262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55856c8218d0 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe50bfff700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98a540 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe504005510 tx=0x7fe50409cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e5b94e403e2b2a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe4eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe4eaffd700 1 -- 172.21.15.52:0/72182262 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50409cc80 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4f4003680 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.923+0000 7fe4eaffd700 1 -- 172.21.15.52:0/72182262 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe5040049d0 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe4eaffd700 1 -- 172.21.15.52:0/72182262 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe50409c580 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 10 client.16380 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55856c998270 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe4eaffd700 1 -- 172.21.15.52:0/72182262 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4f4003680 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe4eaffd700 1 -- 172.21.15.52:0/72182262 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fe504098360 con 0x55856c986f60 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 10 client.16380 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:25.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 20 client.16380 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856c99cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 4 client.16380 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55856c8dfd10 con 0x55856c99a8c0 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.924+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856c99cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.925+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856c99cce0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ceecc070f6f3000 server_cookie=567a58d41ad38895 in_seq=0 out_seq=0 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.925+0000 7fe4eaffd700 10 client.16380 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe4eaffd700 1 -- 172.21.15.52:0/72182262 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55856c8dfd10 con 0x55856c99a8c0 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe4eaffd700 10 client.16380 handle_command_reply: tid=0 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 client.16380 shutdown 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 msgr2=0x55856c98a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98a540 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe504005510 tx=0x7fe50409cac0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 shutdown_connections 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856c99cce0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe4eaffd700 0 client.16380 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98a540 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c98cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/72182262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c98a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 >> 172.21.15.52:0/72182262 conn(0x55856c8abcc0 msgr2=0x55856c97f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 shutdown_connections 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 1 -- 172.21.15.52:0/72182262 wait complete. 2024-06-02T23:14:25.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.926+0000 7fe51ca99b80 20 client.16380 trim_cache size 0 max 16384 2024-06-02T23:14:25.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 Processor -- start 2024-06-02T23:14:25.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 -- start start 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c9809e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c98a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c987a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1aeb20 con 0x55856c986f60 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1b06f0 con 0x55856c980f20 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.947+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1af1f0 con 0x55856c986640 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c987a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c987a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38784/0 (socket says 172.21.15.52:38784) 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 -- 172.21.15.52:0/1100385902 learned_addr learned my addr 172.21.15.52:0/1100385902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c9809e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe502ffd700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c98a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 -- 172.21.15.52:0/1100385902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 msgr2=0x55856c98a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c98a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 -- 172.21.15.52:0/1100385902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 msgr2=0x55856c9809e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c9809e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 -- 172.21.15.52:0/1100385902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55856c97d4c0 con 0x55856c986f60 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c9809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe502ffd700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c98a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe50bfff700 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c987a50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe5040989e0 tx=0x7fe50409aa90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f61c4c939f67100 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe5017fa700 1 -- 172.21.15.52:0/1100385902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504098820 con 0x55856c986f60 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe5017fa700 1 -- 172.21.15.52:0/1100385902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50409cc80 con 0x55856c986f60 2024-06-02T23:14:25.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.948+0000 7fe5017fa700 1 -- 172.21.15.52:0/1100385902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50409b9d0 con 0x55856c986f60 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 -- 172.21.15.52:0/1100385902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 msgr2=0x55856c987a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c987a50 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fe5040989e0 tx=0x7fe50409aa90 comp rx=0 tx=0).stop 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 -- 172.21.15.52:0/1100385902 shutdown_connections 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c987a50 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c980f20 0x55856c9809e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/1100385902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c986640 0x55856c98a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 -- 172.21.15.52:0/1100385902 >> 172.21.15.52:0/1100385902 conn(0x55856c8abcc0 msgr2=0x55856c987340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 -- 172.21.15.52:0/1100385902 shutdown_connections 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 -- 172.21.15.52:0/1100385902 wait complete. 2024-06-02T23:14:25.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.949+0000 7fe51ca99b80 1 -- start start 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 Processor -- start 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c992e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c993350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1aff90 con 0x55856c986f60 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1af8c0 con 0x55856c986640 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c1acae0 con 0x55856c980f20 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe51ca99b80 10 client.0 _ll_register_callbacks cb 0x55856c86c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38790/0 (socket says 172.21.15.52:38790) 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 -- 172.21.15.52:0/2212559930 learned_addr learned my addr 172.21.15.52:0/2212559930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe502ffd700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c993350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c992e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 -- 172.21.15.52:0/2212559930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 msgr2=0x55856c992e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c992e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 -- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 msgr2=0x55856c993350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.950+0000 7fe50bfff700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c993350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe50bfff700 1 -- 172.21.15.52:0/2212559930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55856c990320 con 0x55856c986f60 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c992e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe50bfff700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98fde0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe5040a4c30 tx=0x7fe504003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8511ae69835ebb09 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe4eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe4eaffd700 1 -- 172.21.15.52:0/2212559930 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe504098820 con 0x55856c986f60 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe4eaffd700 1 -- 172.21.15.52:0/2212559930 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe50409b9d0 con 0x55856c986f60 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe50409cc80 con 0x55856c986f60 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe4eaffd700 1 -- 172.21.15.52:0/2212559930 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe5040a0930 con 0x55856c986f60 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 10 client.16386 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:25.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55856c990320 con 0x55856c986f60 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe4eaffd700 1 -- 172.21.15.52:0/2212559930 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe50409cc80 con 0x55856c986f60 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe4eaffd700 1 -- 172.21.15.52:0/2212559930 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fe5040bc1b0 con 0x55856c986f60 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 10 client.16386 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 20 client.16386 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856ca07e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.951+0000 7fe51ca99b80 4 client.16386 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55856c8da290 con 0x55856c99a8c0 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856ca07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe50b7fe700 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856ca07e80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e714f04083023350 server_cookie=43b6efb6af8cfa4f in_seq=0 out_seq=0 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe4eaffd700 10 client.16386 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe4eaffd700 1 -- 172.21.15.52:0/2212559930 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55856c8da290 con 0x55856c99a8c0 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe4eaffd700 10 client.16386 handle_command_reply: tid=0 2024-06-02T23:14:25.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.952+0000 7fe51ca99b80 1 client.16386 shutdown 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 msgr2=0x55856c98fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98fde0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe5040a4c30 tx=0x7fe504003700 comp rx=0 tx=0).stop 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 shutdown_connections 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55856c99a8c0 0x55856ca07e80 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55856c986f60 0x55856c98fde0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55856c980f20 0x55856c992e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe4eaffd700 0 client.16386 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 --2- 172.21.15.52:0/2212559930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55856c986640 0x55856c993350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 >> 172.21.15.52:0/2212559930 conn(0x55856c8abcc0 msgr2=0x55856c987340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 shutdown_connections 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 1 -- 172.21.15.52:0/2212559930 wait complete. 2024-06-02T23:14:25.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe51ca99b80 20 client.16386 trim_cache size 0 max 16384 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: "5dfd123d-a283-4e03-b10d-d4f7964935e4": { 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "5dfd123d-a283-4e03-b10d-d4f7964935e4", 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:25.958 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4f80a2ca0 msgr2=0x7fe4f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4f80a2ca0 0x7fe4f80a5160 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc007fd0 tx=0x7fe4fc00c040 comp rx=0 tx=0).stop 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 msgr2=0x7fe50c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 0x7fe50c19f640 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fe504005910 tx=0x7fe5040047b0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 shutdown_connections 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4f80a2ca0 0x7fe4f80a5160 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe50c108510 0x7fe50c19f640 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe50c100a80 0x7fe50c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.953+0000 7fe512216700 1 --2- 172.21.15.52:0/3521592244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe50c101440 0x7fe50c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.954+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 >> 172.21.15.52:0/3521592244 conn(0x7fe50c077a00 msgr2=0x7fe50c0fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.954+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 shutdown_connections 2024-06-02T23:14:25.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:25.954+0000 7fe512216700 1 -- 172.21.15.52:0/3521592244 wait complete. 2024-06-02T23:14:26.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.70 MB/sec warmup 95 sec latency 3227.521 ms 2024-06-02T23:14:26.691 INFO:tasks.cephfs.filesystem:scrub status for tag:5dfd123d-a283-4e03-b10d-d4f7964935e4 - {'path': '/', 'tag': '5dfd123d-a283-4e03-b10d-d4f7964935e4', 'options': 'recursive,force'} 2024-06-02T23:14:27.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:27 smithi174 ceph-mon[35994]: pgmap v508: 97 pgs: 97 active+clean; 38 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-02T23:14:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:27 smithi052 ceph-mon[32340]: pgmap v508: 97 pgs: 97 active+clean; 38 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-02T23:14:27.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:27 smithi123 ceph-mon[35977]: pgmap v508: 97 pgs: 97 active+clean; 38 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 321 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-02T23:14:27.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.67 MB/sec warmup 96 sec latency 4227.626 ms 2024-06-02T23:14:28.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.64 MB/sec warmup 97 sec latency 5227.729 ms 2024-06-02T23:14:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:29 smithi174 ceph-mon[35994]: pgmap v509: 97 pgs: 97 active+clean; 38 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 960 KiB/s wr, 129 op/s 2024-06-02T23:14:29.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:29 smithi052 ceph-mon[32340]: pgmap v509: 97 pgs: 97 active+clean; 38 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 960 KiB/s wr, 129 op/s 2024-06-02T23:14:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:29 smithi123 ceph-mon[35977]: pgmap v509: 97 pgs: 97 active+clean; 38 MiB data, 604 MiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 960 KiB/s wr, 129 op/s 2024-06-02T23:14:29.620 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.61 MB/sec warmup 98 sec latency 6227.845 ms 2024-06-02T23:14:30.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.59 MB/sec warmup 99 sec latency 7227.960 ms 2024-06-02T23:14:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:31 smithi174 ceph-mon[35994]: pgmap v510: 97 pgs: 97 active+clean; 34 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 871 KiB/s wr, 127 op/s 2024-06-02T23:14:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:14:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:14:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:14:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:14:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:31 smithi052 ceph-mon[32340]: pgmap v510: 97 pgs: 97 active+clean; 34 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 871 KiB/s wr, 127 op/s 2024-06-02T23:14:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:14:31.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:14:31.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:14:31.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:14:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:31 smithi123 ceph-mon[35977]: pgmap v510: 97 pgs: 97 active+clean; 34 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 871 KiB/s wr, 127 op/s 2024-06-02T23:14:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:14:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:14:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:14:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:14:31.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.56 MB/sec warmup 100 sec latency 8228.054 ms 2024-06-02T23:14:32.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.54 MB/sec warmup 101 sec latency 9228.192 ms 2024-06-02T23:14:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:33 smithi174 ceph-mon[35994]: pgmap v511: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 626 KiB/s wr, 113 op/s 2024-06-02T23:14:33.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:33 smithi052 ceph-mon[32340]: pgmap v511: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 626 KiB/s wr, 113 op/s 2024-06-02T23:14:33.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:33 smithi123 ceph-mon[35977]: pgmap v511: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 181 KiB/s rd, 626 KiB/s wr, 113 op/s 2024-06-02T23:14:33.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.51 MB/sec warmup 102 sec latency 10228.247 ms 2024-06-02T23:14:34.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.49 MB/sec warmup 103 sec latency 11228.283 ms 2024-06-02T23:14:35.395 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:35 smithi174 ceph-mon[35994]: pgmap v512: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 295 KiB/s wr, 75 op/s 2024-06-02T23:14:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:35 smithi052 ceph-mon[32340]: pgmap v512: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 295 KiB/s wr, 75 op/s 2024-06-02T23:14:35.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:35 smithi123 ceph-mon[35977]: pgmap v512: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 295 KiB/s wr, 75 op/s 2024-06-02T23:14:35.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.46 MB/sec warmup 104 sec latency 12228.377 ms 2024-06-02T23:14:36.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.44 MB/sec warmup 105 sec latency 13228.488 ms 2024-06-02T23:14:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:37 smithi174 ceph-mon[35994]: pgmap v513: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 279 KiB/s wr, 74 op/s 2024-06-02T23:14:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:37 smithi052 ceph-mon[32340]: pgmap v513: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 279 KiB/s wr, 74 op/s 2024-06-02T23:14:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:37 smithi123 ceph-mon[35977]: pgmap v513: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 279 KiB/s wr, 74 op/s 2024-06-02T23:14:37.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.42 MB/sec warmup 106 sec latency 14228.591 ms 2024-06-02T23:14:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 -- 172.21.15.52:0/4034086772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b40759e0 msgr2=0x7f64b4073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 --2- 172.21.15.52:0/4034086772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b40759e0 0x7f64b4073e30 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009900 tx=0x7f64b002a410 comp rx=0 tx=0).stop 2024-06-02T23:14:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 -- 172.21.15.52:0/4034086772 shutdown_connections 2024-06-02T23:14:37.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 --2- 172.21.15.52:0/4034086772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64b41098c0 0x7f64b410bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 --2- 172.21.15.52:0/4034086772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b4074400 0x7f64b4074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 --2- 172.21.15.52:0/4034086772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b40759e0 0x7f64b4073e30 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 -- 172.21.15.52:0/4034086772 >> 172.21.15.52:0/4034086772 conn(0x7f64b40fcf90 msgr2=0x7f64b40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 -- 172.21.15.52:0/4034086772 shutdown_connections 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 -- 172.21.15.52:0/4034086772 wait complete. 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.752+0000 7f64bba5d640 1 Processor -- start 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 -- start start 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 0x7f64b4199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b40759e0 0x7f64b419a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64b41098c0 0x7f64b41a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64b410f710 con 0x7f64b4074400 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64b410f590 con 0x7f64b40759e0 2024-06-02T23:14:37.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64bba5d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64b410f890 con 0x7f64b41098c0 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 0x7f64b4199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 0x7f64b4199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42858/0 (socket says 172.21.15.52:42858) 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 -- 172.21.15.52:0/861385010 learned_addr learned my addr 172.21.15.52:0/861385010 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64b41098c0 0x7f64b41a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b40759e0 0x7f64b419a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 -- 172.21.15.52:0/861385010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64b41098c0 msgr2=0x7f64b41a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64b41098c0 0x7f64b41a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 -- 172.21.15.52:0/861385010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b40759e0 msgr2=0x7f64b419a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b40759e0 0x7f64b419a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b97f9640 1 -- 172.21.15.52:0/861385010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64b41a1960 con 0x7f64b4074400 2024-06-02T23:14:37.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.753+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b40759e0 0x7f64b419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.754+0000 7f64b97f9640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 0x7f64b4199d40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f64b0004770 tx=0x7f64b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.754+0000 7f64aa7fc640 1 -- 172.21.15.52:0/861385010 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b003f030 con 0x7f64b4074400 2024-06-02T23:14:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.754+0000 7f64aa7fc640 1 -- 172.21.15.52:0/861385010 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64b0003cb0 con 0x7f64b4074400 2024-06-02T23:14:37.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.754+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64b41a1bf0 con 0x7f64b4074400 2024-06-02T23:14:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.754+0000 7f64aa7fc640 1 -- 172.21.15.52:0/861385010 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64b0035070 con 0x7f64b4074400 2024-06-02T23:14:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.754+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64b41a20e0 con 0x7f64b4074400 2024-06-02T23:14:37.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.756+0000 7f64aa7fc640 1 -- 172.21.15.52:0/861385010 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f64b0042030 con 0x7f64b4074400 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 Processor -- start 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 -- start start 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4eef430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4ef4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4ef7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5560a4e524f0 con 0x5560a4eefa00 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560a4df0870 con 0x5560a4eef010 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64be769740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560a4e4e4b0 con 0x5560a4ef4b70 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64b97f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4ef4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64b9ffa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4eef430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64aa7fc640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f64a00a2c30 0x7f64a00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64b97f9640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4ef4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42870/0 (socket says 172.21.15.52:42870) 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64b97f9640 1 -- 172.21.15.52:0/4039835926 learned_addr learned my addr 172.21.15.52:0/4039835926 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.757+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4ef7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f64a00a2c30 0x7f64a00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64aa7fc640 1 -- 172.21.15.52:0/861385010 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f64b00ef9b0 con 0x7f64b4074400 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b8ff8640 1 -- 172.21.15.52:0/4039835926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 msgr2=0x5560a4eef430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4eef430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b8ff8640 1 -- 172.21.15.52:0/4039835926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 msgr2=0x5560a4ef4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4ef4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b8ff8640 1 -- 172.21.15.52:0/4039835926 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a4e0c9b0 con 0x5560a4ef4b70 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b97f9640 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4ef4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4ef7000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64a4005fd0 tx=0x7f64a400f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86d18f1434cd2f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f64a00a2c30 0x7f64a00a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5560a4ef0260 tx=0x7f64ac004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:37.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f649f7fe640 1 -- 172.21.15.52:0/4039835926 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a4005020 con 0x5560a4ef4b70 2024-06-02T23:14:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f649f7fe640 1 -- 172.21.15.52:0/4039835926 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a40047a0 con 0x5560a4ef4b70 2024-06-02T23:14:37.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.758+0000 7f649f7fe640 1 -- 172.21.15.52:0/4039835926 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40116a0 con 0x5560a4ef4b70 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 -- 172.21.15.52:0/4039835926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 msgr2=0x5560a4ef7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4ef7000 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64a4005fd0 tx=0x7f64a400f730 comp rx=0 tx=0).stop 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 -- 172.21.15.52:0/4039835926 shutdown_connections 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4ef7000 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4ef4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 --2- 172.21.15.52:0/4039835926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4eef430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 -- 172.21.15.52:0/4039835926 >> 172.21.15.52:0/4039835926 conn(0x5560a4eea680 msgr2=0x5560a4eeaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 -- 172.21.15.52:0/4039835926 shutdown_connections 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 -- 172.21.15.52:0/4039835926 wait complete. 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.759+0000 7f64be769740 1 -- start start 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 Processor -- start 2024-06-02T23:14:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f00720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5560a4ef7b20 con 0x5560a4ef4b70 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5560a4ddbae0 con 0x5560a4eef010 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5560a4ef7ca0 con 0x5560a4eefa00 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64be769740 10 client.0 _ll_register_callbacks cb 0x5560a4e149b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64b9ffa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f00720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64b9ffa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f00720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42874/0 (socket says 172.21.15.52:42874) 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64b9ffa640 1 -- 172.21.15.52:0/2986323531 learned_addr learned my addr 172.21.15.52:0/2986323531 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.760+0000 7f64b97f9640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64b9ffa640 1 -- 172.21.15.52:0/2986323531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 msgr2=0x5560a4ef96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64b9ffa640 1 -- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 msgr2=0x5560a4ef91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64b9ffa640 1 -- 172.21.15.52:0/2986323531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a4b66210 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f00720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f64ac003dc0 tx=0x7f64ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beefcca7e73f2f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f649cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f649cff9640 1 -- 172.21.15.52:0/2986323531 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac01a040 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f649cff9640 1 -- 172.21.15.52:0/2986323531 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f64ac004030 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64a40047a0 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f649cff9640 1 -- 172.21.15.52:0/2986323531 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64ac0142f0 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64be769740 10 client.16398 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560a4c32620 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.761+0000 7f649cff9640 1 -- 172.21.15.52:0/2986323531 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64a40047a0 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f649cff9640 1 -- 172.21.15.52:0/2986323531 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f64ac020040 con 0x5560a4ef4b70 2024-06-02T23:14:37.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64be769740 10 client.16398 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64be769740 10 client.16398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64be769740 20 client.16398 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64be769740 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f05130 0x5560a4f075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64be769740 4 client.16398 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560a4e65f10 con 0x5560a4f05130 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64b97f9640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f05130 0x5560a4f075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f64b97f9640 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f05130 0x5560a4f075c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f26e911ea83c5920 server_cookie=cf2f0823f4c62b6a in_seq=0 out_seq=0 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.762+0000 7f649cff9640 10 client.16398 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f649cff9640 1 -- 172.21.15.52:0/2986323531 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5560a4e65f10 con 0x5560a4f05130 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f649cff9640 10 client.16398 handle_command_reply: tid=0 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 client.16398 shutdown 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 msgr2=0x5560a4f00720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f00720 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f64ac003dc0 tx=0x7f64ac004a40 comp rx=0 tx=0).stop 2024-06-02T23:14:37.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 shutdown_connections 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f05130 0x5560a4f075c0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f00720 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f649cff9640 0 client.16398 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 --2- 172.21.15.52:0/2986323531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.763+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 >> 172.21.15.52:0/2986323531 conn(0x5560a4eea680 msgr2=0x5560a4ee97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.764+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 shutdown_connections 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.764+0000 7f64be769740 1 -- 172.21.15.52:0/2986323531 wait complete. 2024-06-02T23:14:37.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.764+0000 7f64be769740 20 client.16398 trim_cache size 0 max 16384 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 Processor -- start 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 -- start start 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f006e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64b41aae80 con 0x5560a4ef4b70 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64b41aca50 con 0x5560a4eef010 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64be769740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64b41ab550 con 0x5560a4eefa00 2024-06-02T23:14:37.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42890/0 (socket says 172.21.15.52:42890) 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 -- 172.21.15.52:0/3763680782 learned_addr learned my addr 172.21.15.52:0/3763680782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b97f9640 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 -- 172.21.15.52:0/3763680782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 msgr2=0x5560a4ef1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 -- 172.21.15.52:0/3763680782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 msgr2=0x5560a4ef0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 -- 172.21.15.52:0/3763680782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560a4ee9f30 con 0x5560a4ef4b70 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b97f9640 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.780+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f006e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f64ac003fd0 tx=0x7f64ac005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a80b1688406608 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f649effd640 1 -- 172.21.15.52:0/3763680782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac03a070 con 0x5560a4ef4b70 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f649effd640 1 -- 172.21.15.52:0/3763680782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64ac026070 con 0x5560a4ef4b70 2024-06-02T23:14:37.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f649effd640 1 -- 172.21.15.52:0/3763680782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac01a040 con 0x5560a4ef4b70 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 -- 172.21.15.52:0/3763680782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 msgr2=0x5560a4f006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f006e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f64ac003fd0 tx=0x7f64ac005bd0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 -- 172.21.15.52:0/3763680782 shutdown_connections 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4eefa00 0x5560a4ef1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4ef4b70 0x5560a4f006e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 --2- 172.21.15.52:0/3763680782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4ef0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 -- 172.21.15.52:0/3763680782 >> 172.21.15.52:0/3763680782 conn(0x5560a4eea680 msgr2=0x5560a4eeaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 -- 172.21.15.52:0/3763680782 shutdown_connections 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.781+0000 7f64be769740 1 -- 172.21.15.52:0/3763680782 wait complete. 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 -- start start 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 Processor -- start 2024-06-02T23:14:37.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4efd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4efdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4f6de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f64b41ac2f0 con 0x5560a4eefa00 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f64b41abc20 con 0x5560a4eef010 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f64b41a8e60 con 0x5560a4ef4b70 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64be769740 10 client.0 _ll_register_callbacks cb 0x7f64ac014f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64b8ff8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4efdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64b8ff8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4efdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42896/0 (socket says 172.21.15.52:42896) 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64b8ff8640 1 -- 172.21.15.52:0/3935649176 learned_addr learned my addr 172.21.15.52:0/3935649176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64b97f9640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4efd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.782+0000 7f64b9ffa640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4f6de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64b8ff8640 1 -- 172.21.15.52:0/3935649176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 msgr2=0x5560a4f6de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4f6de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64b8ff8640 1 -- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 msgr2=0x5560a4efd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4efd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64b8ff8640 1 -- 172.21.15.52:0/3935649176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560a4eea4b0 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64b8ff8640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4efdc80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f64a400b620 tx=0x7f64a400b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291273bb508ed27b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f649cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f649cff9640 1 -- 172.21.15.52:0/3935649176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64a40053e0 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f649cff9640 1 -- 172.21.15.52:0/3935649176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f64a4005580 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64ac026070 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f649cff9640 1 -- 172.21.15.52:0/3935649176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f64a4004020 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64be769740 10 client.16404 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5560a4ef06d0 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.783+0000 7f649cff9640 1 -- 172.21.15.52:0/3935649176 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f64ac026070 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f649cff9640 1 -- 172.21.15.52:0/3935649176 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f64a40035f0 con 0x5560a4eefa00 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64be769740 10 client.16404 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:37.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64be769740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64be769740 20 client.16404 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64be769740 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f6fbc0 0x5560a4f71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64be769740 4 client.16404 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560a4e5fc20 con 0x5560a4f6fbc0 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64b97f9640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f6fbc0 0x5560a4f71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f649cff9640 10 client.16404 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:37.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.784+0000 7f64b97f9640 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f6fbc0 0x5560a4f71fe0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be965bf4088782dd server_cookie=ca3509de976515b8 in_seq=0 out_seq=0 2024-06-02T23:14:37.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f649cff9640 1 -- 172.21.15.52:0/3935649176 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5560a4e5fc20 con 0x5560a4f6fbc0 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f649cff9640 10 client.16404 handle_command_reply: tid=0 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 client.16404 shutdown 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 msgr2=0x5560a4efdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4efdc80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f64a400b620 tx=0x7f64a400b7e0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 shutdown_connections 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5560a4f6fbc0 0x5560a4f71fe0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5560a4ef4b70 0x5560a4f6de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f649cff9640 0 client.16404 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5560a4eefa00 0x5560a4efdc80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 --2- 172.21.15.52:0/3935649176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5560a4eef010 0x5560a4efd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 >> 172.21.15.52:0/3935649176 conn(0x5560a4eea680 msgr2=0x5560a4ef0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 shutdown_connections 2024-06-02T23:14:37.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 1 -- 172.21.15.52:0/3935649176 wait complete. 2024-06-02T23:14:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.800+0000 7f64be769740 20 client.16404 trim_cache size 0 max 16384 2024-06-02T23:14:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:14:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:14:37.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f64a00a2c30 msgr2=0x7f64a00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f64a00a2c30 0x7f64a00a50f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x5560a4ef0260 tx=0x7f64ac004ab0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 msgr2=0x7f64b4199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 0x7f64b4199d40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f64b0004770 tx=0x7f64b00042f0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 shutdown_connections 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f64a00a2c30 0x7f64a00a50f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f64b41098c0 0x7f64b41a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64b40759e0 0x7f64b419a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 --2- 172.21.15.52:0/861385010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f64b4074400 0x7f64b4199d40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 >> 172.21.15.52:0/861385010 conn(0x7f64b40fcf90 msgr2=0x7f64b41057c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:37.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 shutdown_connections 2024-06-02T23:14:37.803 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:37.801+0000 7f64bba5d640 1 -- 172.21.15.52:0/861385010 wait complete. 2024-06-02T23:14:38.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.39 MB/sec warmup 107 sec latency 15228.673 ms 2024-06-02T23:14:39.268 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:14:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:39 smithi174 ceph-mon[35994]: pgmap v514: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-06-02T23:14:39.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:39 smithi052 ceph-mon[32340]: pgmap v514: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-06-02T23:14:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:39 smithi123 ceph-mon[35977]: pgmap v514: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 20 op/s 2024-06-02T23:14:39.535 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:39.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.37 MB/sec warmup 108 sec latency 16228.719 ms 2024-06-02T23:14:40.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.35 MB/sec warmup 109 sec latency 17228.821 ms 2024-06-02T23:14:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 -- 172.21.15.52:0/618262759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 msgr2=0x7fb5701077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:41.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 --2- 172.21.15.52:0/618262759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701077e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb560009990 tx=0x7fb560009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 -- 172.21.15.52:0/618262759 shutdown_connections 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 --2- 172.21.15.52:0/618262759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb570101d50 0x7fb57010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 --2- 172.21.15.52:0/618262759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701077e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 --2- 172.21.15.52:0/618262759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101390 0x7fb570101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 -- 172.21.15.52:0/618262759 >> 172.21.15.52:0/618262759 conn(0x7fb570078340 msgr2=0x7fb570078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 -- 172.21.15.52:0/618262759 shutdown_connections 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.280+0000 7fb5768c7700 1 -- 172.21.15.52:0/618262759 wait complete. 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 Processor -- start 2024-06-02T23:14:41.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 -- start start 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb570101390 0x7fb5701999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101d50 0x7fb570199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb570111550 con 0x7fb570107400 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701113d0 con 0x7fb570101d50 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb5768c7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb5701116d0 con 0x7fb570101390 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb56f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101d50 0x7fb570199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb574e64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb56f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101d50 0x7fb570199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:54470/0 (socket says 172.21.15.52:54470) 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb574e64700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701a4a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42916/0 (socket says 172.21.15.52:42916) 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb56f7fe700 1 -- 172.21.15.52:0/987369734 learned_addr learned my addr 172.21.15.52:0/987369734 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:41.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.281+0000 7fb56ffff700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb570101390 0x7fb5701999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb574e64700 1 -- 172.21.15.52:0/987369734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb570101390 msgr2=0x7fb5701999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb574e64700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb570101390 0x7fb5701999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb574e64700 1 -- 172.21.15.52:0/987369734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101d50 msgr2=0x7fb570199f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb574e64700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101d50 0x7fb570199f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb574e64700 1 -- 172.21.15.52:0/987369734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5701a4f90 con 0x7fb570107400 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb574e64700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701a4a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb568007f10 tx=0x7fb56800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568004db0 con 0x7fb570107400 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb568009e20 con 0x7fb570107400 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5701a51c0 con 0x7fb570107400 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb568015400 con 0x7fb570107400 2024-06-02T23:14:41.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.282+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5701a56d0 con 0x7fb570107400 2024-06-02T23:14:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.283+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb570102ab0 con 0x7fb570107400 2024-06-02T23:14:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.284+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb56800db20 con 0x7fb570107400 2024-06-02T23:14:41.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.285+0000 7fb56d7fa700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb55c0a2c20 0x7fb55c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.285+0000 7fb56ffff700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb55c0a2c20 0x7fb55c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.285+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb5680ced90 con 0x7fb570107400 2024-06-02T23:14:41.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.286+0000 7fb56ffff700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb55c0a2c20 0x7fb55c0a50e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb56000b5c0 tx=0x7fb560005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:41.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.289+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb568010070 con 0x7fb570107400 2024-06-02T23:14:41.289 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:41 smithi052 ceph-mon[32340]: pgmap v515: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-02T23:14:41.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:41 smithi174 ceph-mon[35994]: pgmap v515: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-02T23:14:41.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:41 smithi123 ceph-mon[35977]: pgmap v515: 97 pgs: 97 active+clean; 34 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-02T23:14:41.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.546+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb57002dca0 con 0x7fb570107400 2024-06-02T23:14:41.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.547+0000 7fb56d7fa700 1 -- 172.21.15.52:0/987369734 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7fb56809bd20 con 0x7fb570107400 2024-06-02T23:14:41.548 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:14:41.548 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:14:41.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb55c0a2c20 msgr2=0x7fb55c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:41.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb55c0a2c20 0x7fb55c0a50e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fb56000b5c0 tx=0x7fb560005960 comp rx=0 tx=0).stop 2024-06-02T23:14:41.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 msgr2=0x7fb5701a4a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:41.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701a4a50 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb568007f10 tx=0x7fb56800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 shutdown_connections 2024-06-02T23:14:41.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb55c0a2c20 0x7fb55c0a50e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb570107400 0x7fb5701a4a50 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb570101390 0x7fb5701999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 --2- 172.21.15.52:0/987369734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb570101d50 0x7fb570199f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:41.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 >> 172.21.15.52:0/987369734 conn(0x7fb570078340 msgr2=0x7fb5701001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 shutdown_connections 2024-06-02T23:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:41.549+0000 7fb5768c7700 1 -- 172.21.15.52:0/987369734 wait complete. 2024-06-02T23:14:41.553 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:14:41.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.33 MB/sec warmup 110 sec latency 18228.886 ms 2024-06-02T23:14:42.116 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id a049db0d --release --await' 2024-06-02T23:14:42.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id a049db0d --release --await 2024-06-02T23:14:42.386 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:42.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:42 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/987369734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:42.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:42 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:14:42.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:42 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/987369734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:42.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:42 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:14:42.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:42 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/987369734' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:42.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:42 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:14:42.621 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.31 MB/sec warmup 111 sec latency 19228.969 ms 2024-06-02T23:14:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:43 smithi174 ceph-mon[35994]: pgmap v516: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-06-02T23:14:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:14:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:14:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:14:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:43 smithi052 ceph-mon[32340]: pgmap v516: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-06-02T23:14:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:14:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:14:43.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:14:43.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:43 smithi123 ceph-mon[35977]: pgmap v516: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 16 op/s 2024-06-02T23:14:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:14:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:14:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:14:43.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44660 2.29 MB/sec warmup 112 sec latency 20229.008 ms 2024-06-02T23:14:44.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 -- 172.21.15.52:0/459135643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa910107840 msgr2=0x7fa91010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 --2- 172.21.15.52:0/459135643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa910107840 0x7fa91010f3c0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa90800b0d0 tx=0x7fa90800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 -- 172.21.15.52:0/459135643 shutdown_connections 2024-06-02T23:14:44.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 --2- 172.21.15.52:0/459135643 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa910107840 0x7fa91010f3c0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 --2- 172.21.15.52:0/459135643 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa910102740 0x7fa910102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 --2- 172.21.15.52:0/459135643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa9101030f0 0x7fa910107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.143+0000 7fa914df4700 1 -- 172.21.15.52:0/459135643 >> 172.21.15.52:0/459135643 conn(0x7fa9100795f0 msgr2=0x7fa910079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:44.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.144+0000 7fa914df4700 1 -- 172.21.15.52:0/459135643 shutdown_connections 2024-06-02T23:14:44.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.144+0000 7fa914df4700 1 -- 172.21.15.52:0/459135643 wait complete. 2024-06-02T23:14:44.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.144+0000 7fa914df4700 1 Processor -- start 2024-06-02T23:14:44.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 -- start start 2024-06-02T23:14:44.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa910102740 0x7fa91019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 0x7fa91019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa910107840 0x7fa9101989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa910112700 con 0x7fa9101030f0 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa910112580 con 0x7fa910102740 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa914df4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa910112880 con 0x7fa910107840 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 0x7fa91019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 0x7fa91019ed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59050/0 (socket says 172.21.15.52:59050) 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 -- 172.21.15.52:0/3516224378 learned_addr learned my addr 172.21.15.52:0/3516224378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa90e59c700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa910102740 0x7fa91019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa90ed9d700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa910107840 0x7fa9101989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 -- 172.21.15.52:0/3516224378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa910107840 msgr2=0x7fa9101989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa910107840 0x7fa9101989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 -- 172.21.15.52:0/3516224378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa910102740 msgr2=0x7fa91019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa910102740 0x7fa91019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 -- 172.21.15.52:0/3516224378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa910199100 con 0x7fa9101030f0 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.145+0000 7fa905d9b700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 0x7fa91019ed20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b530 tx=0x7fa8f800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.146+0000 7fa907fff700 1 -- 172.21.15.52:0/3516224378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8015020 con 0x7fa9101030f0 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.146+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9101993f0 con 0x7fa9101030f0 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.146+0000 7fa907fff700 1 -- 172.21.15.52:0/3516224378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8f8011ce0 con 0x7fa9101030f0 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.146+0000 7fa907fff700 1 -- 172.21.15.52:0/3516224378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8010d60 con 0x7fa9101030f0 2024-06-02T23:14:44.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.146+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9101a56b0 con 0x7fa9101030f0 2024-06-02T23:14:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.148+0000 7fa907fff700 1 -- 172.21.15.52:0/3516224378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8f8018020 con 0x7fa9101030f0 2024-06-02T23:14:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.149+0000 7fa907fff700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8fc0a2c20 0x7fa8fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.149+0000 7fa90e59c700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8fc0a2c20 0x7fa8fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.149+0000 7fa907fff700 1 -- 172.21.15.52:0/3516224378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f80ce910 con 0x7fa9101030f0 2024-06-02T23:14:44.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.149+0000 7fa90e59c700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8fc0a2c20 0x7fa8fc0a50e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa900006fd0 tx=0x7fa900008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:44.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 Processor -- start 2024-06-02T23:14:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 -- start start 2024-06-02T23:14:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4601b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 0x560e46014e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46015810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560e45f66910 con 0x560e4601b560 2024-06-02T23:14:44.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e45f0e820 con 0x560e4601ac40 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa91f677b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e45f628d0 con 0x560e460153d0 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 0x560e46014e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa90e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4601b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 0x560e46014e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59060/0 (socket says 172.21.15.52:59060) 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa90e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4601b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38768/0 (socket says 172.21.15.52:38768) 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.150+0000 7fa905d9b700 1 -- 172.21.15.52:0/2546921918 learned_addr learned my addr 172.21.15.52:0/2546921918 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90ed9d700 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46015810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90e59c700 1 -- 172.21.15.52:0/2546921918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 msgr2=0x560e46015810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90e59c700 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46015810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90e59c700 1 -- 172.21.15.52:0/2546921918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 msgr2=0x560e46014e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90e59c700 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 0x560e46014e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90e59c700 1 -- 172.21.15.52:0/2546921918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e45f05050 con 0x560e4601ac40 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa905d9b700 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 0x560e46014e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa90e59c700 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4601b020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa900009fb0 tx=0x7fa900010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d4a149bbff6dd53 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:44.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa904d99700 1 -- 172.21.15.52:0/2546921918 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900024070 con 0x560e4601ac40 2024-06-02T23:14:44.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa904d99700 1 -- 172.21.15.52:0/2546921918 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9000148e0 con 0x560e4601ac40 2024-06-02T23:14:44.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.151+0000 7fa904d99700 1 -- 172.21.15.52:0/2546921918 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90001f950 con 0x560e4601ac40 2024-06-02T23:14:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 -- 172.21.15.52:0/2546921918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 msgr2=0x560e4601b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4601b020 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa900009fb0 tx=0x7fa900010000 comp rx=0 tx=0).stop 2024-06-02T23:14:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 -- 172.21.15.52:0/2546921918 shutdown_connections 2024-06-02T23:14:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46015810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4601b020 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2546921918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601b560 0x560e46014e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 -- 172.21.15.52:0/2546921918 >> 172.21.15.52:0/2546921918 conn(0x560e45f3fdc0 msgr2=0x560e46010120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 -- 172.21.15.52:0/2546921918 shutdown_connections 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.152+0000 7fa91f677b80 1 -- 172.21.15.52:0/2546921918 wait complete. 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 -- start start 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 Processor -- start 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46021280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e4601e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4601eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560e460263e0 con 0x560e4601ac40 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e45f06cf0 con 0x560e4601b560 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e46026560 con 0x560e460153d0 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa91f677b80 10 client.0 _ll_register_callbacks cb 0x560e45f00a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:44.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e4601e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e4601e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59074/0 (socket says 172.21.15.52:59074) 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa905d9b700 1 -- 172.21.15.52:0/2785122608 learned_addr learned my addr 172.21.15.52:0/2785122608 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.153+0000 7fa90e59c700 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46021280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa905d9b700 1 -- 172.21.15.52:0/2785122608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 msgr2=0x560e46021280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa905d9b700 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46021280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa905d9b700 1 -- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 msgr2=0x560e4601eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa905d9b700 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4601eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa905d9b700 1 -- 172.21.15.52:0/2785122608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e45eb59d0 con 0x560e4601ac40 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa905d9b700 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e4601e580 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800bcf0 tx=0x7fa8f80105d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a6ede0cdfb4e50 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa8f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/2785122608 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80b4060 con 0x560e4601ac40 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/2785122608 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f809b3a0 con 0x560e4601ac40 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9000148e0 con 0x560e4601ac40 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/2785122608 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8f809f4f0 con 0x560e4601ac40 2024-06-02T23:14:44.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa91f677b80 10 client.16422 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e46030c90 con 0x560e4601ac40 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.154+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/2785122608 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa9000148e0 con 0x560e4601ac40 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/2785122608 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fa8f809a590 con 0x560e4601ac40 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa91f677b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa91f677b80 10 client.16422 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa91f677b80 20 client.16422 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e46033730 0x560e46035b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa91f677b80 4 client.16422 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e45f73e10 con 0x560e46033730 2024-06-02T23:14:44.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.155+0000 7fa90e59c700 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e46033730 0x560e46035b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa8f5ffb700 10 client.16422 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa90e59c700 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e46033730 0x560e46035b70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0a243395751419e server_cookie=5746dd78fdd25019 in_seq=0 out_seq=0 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/2785122608 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560e45f73e10 con 0x560e46033730 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa8f5ffb700 10 client.16422 handle_command_reply: tid=0 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa91f677b80 1 client.16422 shutdown 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 msgr2=0x560e4601e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e4601e580 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800bcf0 tx=0x7fa8f80105d0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 shutdown_connections 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e46033730 0x560e46035b70 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.156+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4601eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e46021280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa91f677b80 1 --2- 172.21.15.52:0/2785122608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e4601e580 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 >> 172.21.15.52:0/2785122608 conn(0x560e45f3fdc0 msgr2=0x560e46013e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:44.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa8f5ffb700 0 client.16422 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:44.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 shutdown_connections 2024-06-02T23:14:44.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa91f677b80 1 -- 172.21.15.52:0/2785122608 wait complete. 2024-06-02T23:14:44.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.157+0000 7fa91f677b80 20 client.16422 trim_cache size 0 max 16384 2024-06-02T23:14:44.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.177+0000 7fa91f677b80 1 Processor -- start 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.177+0000 7fa91f677b80 1 -- start start 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 0x560e4601c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4600f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e4601b560 0x560e4600fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa91f677b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101aa8e0 con 0x560e460153d0 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa91f677b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ac4b0 con 0x560e4601ac40 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa91f677b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101aafb0 con 0x560e4601b560 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 0x560e4601c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 0x560e4601c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59086/0 (socket says 172.21.15.52:59086) 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 -- 172.21.15.52:0/604031266 learned_addr learned my addr 172.21.15.52:0/604031266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 -- 172.21.15.52:0/604031266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e4601b560 msgr2=0x560e4600fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e4601b560 0x560e4600fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa905d9b700 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4600f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 -- 172.21.15.52:0/604031266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 msgr2=0x560e4600f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4600f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 -- 172.21.15.52:0/604031266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e46014de0 con 0x560e460153d0 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa90e59c700 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 0x560e4601c110 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa90000dbe0 tx=0x7fa90001f510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=432d6cbd29c77123 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.178+0000 7fa8f7fff700 1 -- 172.21.15.52:0/604031266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa900024040 con 0x560e460153d0 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa8f7fff700 1 -- 172.21.15.52:0/604031266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90001f580 con 0x560e460153d0 2024-06-02T23:14:44.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa8f7fff700 1 -- 172.21.15.52:0/604031266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90002b640 con 0x560e460153d0 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 -- 172.21.15.52:0/604031266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 msgr2=0x560e4601c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 0x560e4601c110 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa90000dbe0 tx=0x7fa90001f510 comp rx=0 tx=0).stop 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 -- 172.21.15.52:0/604031266 shutdown_connections 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e4601b560 0x560e4600fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e460153d0 0x560e4601c110 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 --2- 172.21.15.52:0/604031266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601ac40 0x560e4600f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 -- 172.21.15.52:0/604031266 >> 172.21.15.52:0/604031266 conn(0x560e45f3fdc0 msgr2=0x560e4601ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 -- 172.21.15.52:0/604031266 shutdown_connections 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.179+0000 7fa91f677b80 1 -- 172.21.15.52:0/604031266 wait complete. 2024-06-02T23:14:44.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 -- start start 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 Processor -- start 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e4603b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e46039d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4603a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101abd50 con 0x560e4601ac40 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101ab680 con 0x560e4601b560 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9101a8880 con 0x560e460153d0 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa91f677b80 10 client.0 _ll_register_callbacks cb 0x560e45f00a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e46039d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa905d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e46039d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59100/0 (socket says 172.21.15.52:59100) 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa90e59c700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e4603b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa905d9b700 1 -- 172.21.15.52:0/1157353312 learned_addr learned my addr 172.21.15.52:0/1157353312 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.180+0000 7fa90ed9d700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4603a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa905d9b700 1 -- 172.21.15.52:0/1157353312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 msgr2=0x560e4603b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa905d9b700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e4603b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa905d9b700 1 -- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 msgr2=0x560e4603a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa905d9b700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4603a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa905d9b700 1 -- 172.21.15.52:0/1157353312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4603a940 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa905d9b700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e46039d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8f809ae40 tx=0x7fa8f80cc410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f43faa3d2e9c7b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa8f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/1157353312 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f80a3db0 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/1157353312 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f80661f0 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90001f580 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/1157353312 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa8f8096e90 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa91f677b80 10 client.16428 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560e4609f240 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.181+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/1157353312 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90001f580 con 0x560e4601ac40 2024-06-02T23:14:44.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/1157353312 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fa8f809b3a0 con 0x560e4601ac40 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa91f677b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa91f677b80 10 client.16428 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa91f677b80 20 client.16428 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa91f677b80 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e4609f900 0x560e460a1d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa91f677b80 4 client.16428 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a049db0d", "release": true, "await": true}] 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a049db0d", "release": true, "await": true}) -- 0x560e45f6e390 con 0x560e4609f900 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa90e59c700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e4609f900 0x560e460a1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa90e59c700 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e4609f900 0x560e460a1d20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294723f455f6ee2b server_cookie=4cbfc488d1023fed in_seq=0 out_seq=0 2024-06-02T23:14:44.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.182+0000 7fa8f5ffb700 10 client.16428 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa8f5ffb700 1 -- 172.21.15.52:0/1157353312 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560e45f6e390 con 0x560e4609f900 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa8f5ffb700 10 client.16428 handle_command_reply: tid=0 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 client.16428 shutdown 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 msgr2=0x560e46039d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e46039d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fa8f809ae40 tx=0x7fa8f80cc410 comp rx=0 tx=0).stop 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 shutdown_connections 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560e4609f900 0x560e460a1d20 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e4601b560 0x560e4603a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e460153d0 0x560e4603b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa8f5ffb700 0 client.16428 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:44.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 --2- 172.21.15.52:0/1157353312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560e4601ac40 0x560e46039d80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.189+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 >> 172.21.15.52:0/1157353312 conn(0x560e45f3fdc0 msgr2=0x560e4600f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 shutdown_connections 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa91f677b80 1 -- 172.21.15.52:0/1157353312 wait complete. 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa91f677b80 20 client.16428 trim_cache size 0 max 16384 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 12, 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:14:44.191 INFO:teuthology.orchestra.run.smithi052.stdout: "a049db0d": { 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 12, 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.8, 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:14:44.192 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:44.193 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:44.193 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:44.193 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:44.193 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:44.193 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8fc0a2c20 msgr2=0x7fa8fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8fc0a2c20 0x7fa8fc0a50e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa900006fd0 tx=0x7fa900008040 comp rx=0 tx=0).stop 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 msgr2=0x7fa91019ed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 0x7fa91019ed20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800b530 tx=0x7fa8f800b8f0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 shutdown_connections 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8fc0a2c20 0x7fa8fc0a50e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa910107840 0x7fa9101989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa910102740 0x7fa91019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 --2- 172.21.15.52:0/3516224378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa9101030f0 0x7fa91019ed20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 >> 172.21.15.52:0/3516224378 conn(0x7fa9100795f0 msgr2=0x7fa910100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:44.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 shutdown_connections 2024-06-02T23:14:44.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:44.190+0000 7fa914df4700 1 -- 172.21.15.52:0/3516224378 wait complete. 2024-06-02T23:14:44.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 44944 2.28 MB/sec warmup 113 sec latency 20798.739 ms 2024-06-02T23:14:45.009 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a049db0d', total seconds elapsed: 33.4 2024-06-02T23:14:45.009 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.75 2024-06-02T23:14:45.009 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:14:45.287 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:45 smithi174 ceph-mon[35994]: pgmap v517: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-06-02T23:14:45.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:45 smithi052 ceph-mon[32340]: pgmap v517: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-06-02T23:14:45.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:45 smithi123 ceph-mon[35977]: pgmap v517: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.2 KiB/s wr, 14 op/s 2024-06-02T23:14:45.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 45569 2.31 MB/sec warmup 114 sec latency 10.880 ms 2024-06-02T23:14:46.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 45960 2.30 MB/sec warmup 115 sec latency 62.712 ms 2024-06-02T23:14:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:47 smithi174 ceph-mon[35994]: pgmap v518: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-02T23:14:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:47 smithi052 ceph-mon[32340]: pgmap v518: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-02T23:14:47.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:47 smithi123 ceph-mon[35977]: pgmap v518: 97 pgs: 97 active+clean; 34 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-06-02T23:14:47.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 46441 2.32 MB/sec warmup 116 sec latency 15.395 ms 2024-06-02T23:14:48.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 -- 172.21.15.52:0/1354121729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea8108500 msgr2=0x7faea81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:48.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 --2- 172.21.15.52:0/1354121729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea8108500 0x7faea81088e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7faea4009990 tx=0x7faea4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 -- 172.21.15.52:0/1354121729 shutdown_connections 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 --2- 172.21.15.52:0/1354121729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea806ab30 0x7faea810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 --2- 172.21.15.52:0/1354121729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea8108500 0x7faea81088e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 --2- 172.21.15.52:0/1354121729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea806a170 0x7faea806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 -- 172.21.15.52:0/1354121729 >> 172.21.15.52:0/1354121729 conn(0x7faea8078340 msgr2=0x7faea8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 -- 172.21.15.52:0/1354121729 shutdown_connections 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.360+0000 7faeb0291700 1 -- 172.21.15.52:0/1354121729 wait complete. 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 Processor -- start 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 -- start start 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea806a170 0x7faea819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:48.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 0x7faea819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:48.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea8108500 0x7faea819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:48.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faea8112630 con 0x7faea806ab30 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faea81124b0 con 0x7faea8108500 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faeb0291700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faea81127b0 con 0x7faea806a170 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faead82c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 0x7faea819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faead82c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 0x7faea819b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59124/0 (socket says 172.21.15.52:59124) 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.361+0000 7faead82c700 1 -- 172.21.15.52:0/1715213529 learned_addr learned my addr 172.21.15.52:0/1715213529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faeae02d700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea806a170 0x7faea819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faeae82e700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea8108500 0x7faea819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faead82c700 1 -- 172.21.15.52:0/1715213529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea806a170 msgr2=0x7faea819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faead82c700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea806a170 0x7faea819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faead82c700 1 -- 172.21.15.52:0/1715213529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea8108500 msgr2=0x7faea819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faead82c700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea8108500 0x7faea819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faead82c700 1 -- 172.21.15.52:0/1715213529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faea819fe50 con 0x7faea806ab30 2024-06-02T23:14:48.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faead82c700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 0x7faea819b380 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d980 tx=0x7fae9800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:48.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae98017070 con 0x7faea806ab30 2024-06-02T23:14:48.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faea81a0140 con 0x7faea806ab30 2024-06-02T23:14:48.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae98005020 con 0x7faea806ab30 2024-06-02T23:14:48.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae9800fa90 con 0x7faea806ab30 2024-06-02T23:14:48.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.362+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faea81a79c0 con 0x7faea806ab30 2024-06-02T23:14:48.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.363+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faea8069180 con 0x7faea806ab30 2024-06-02T23:14:48.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.365+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae980051d0 con 0x7faea806ab30 2024-06-02T23:14:48.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.365+0000 7fae9f7fe700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae940a2c20 0x7fae940a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:48.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.366+0000 7faeae02d700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae940a2c20 0x7fae940a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:48.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.366+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fae980cf290 con 0x7faea806ab30 2024-06-02T23:14:48.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.366+0000 7faeae02d700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae940a2c20 0x7fae940a50e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faea40098e0 tx=0x7faea4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:48.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.370+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fae98020020 con 0x7faea806ab30 2024-06-02T23:14:48.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 46722 2.30 MB/sec warmup 117 sec latency 13.806 ms 2024-06-02T23:14:48.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.650+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faea81a7b90 con 0x7faea806ab30 2024-06-02T23:14:48.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.651+0000 7fae9f7fe700 1 -- 172.21.15.52:0/1715213529 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7fae9809c220 con 0x7faea806ab30 2024-06-02T23:14:48.651 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:14:48.652 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.653+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae940a2c20 msgr2=0x7fae940a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.653+0000 7faeb0291700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae940a2c20 0x7fae940a50e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7faea40098e0 tx=0x7faea4005960 comp rx=0 tx=0).stop 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.653+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 msgr2=0x7faea819b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.653+0000 7faeb0291700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 0x7faea819b380 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fae9800d980 tx=0x7fae9800dc90 comp rx=0 tx=0).stop 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 shutdown_connections 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae940a2c20 0x7fae940a50e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faea8108500 0x7faea819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faea806a170 0x7faea819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 --2- 172.21.15.52:0/1715213529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faea806ab30 0x7faea819b380 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:48.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 >> 172.21.15.52:0/1715213529 conn(0x7faea8078340 msgr2=0x7faea810d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:48.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 shutdown_connections 2024-06-02T23:14:48.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:48.654+0000 7faeb0291700 1 -- 172.21.15.52:0/1715213529 wait complete. 2024-06-02T23:14:48.655 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:14:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:49 smithi052 ceph-mon[32340]: pgmap v519: 97 pgs: 97 active+clean; 35 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 151 KiB/s wr, 24 op/s 2024-06-02T23:14:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:49 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1715213529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:49.439 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 147.75 --await-for 45.0' 2024-06-02T23:14:49.440 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 147.75 --await-for 45.0 2024-06-02T23:14:49.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:49 smithi123 ceph-mon[35977]: pgmap v519: 97 pgs: 97 active+clean; 35 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 151 KiB/s wr, 24 op/s 2024-06-02T23:14:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:49 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1715213529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:49.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 47220 2.30 MB/sec warmup 118 sec latency 15.119 ms 2024-06-02T23:14:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:49 smithi174 ceph-mon[35994]: pgmap v519: 97 pgs: 97 active+clean; 35 MiB data, 597 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 151 KiB/s wr, 24 op/s 2024-06-02T23:14:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:49 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1715213529' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:14:49.726 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:50.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 47794 2.31 MB/sec warmup 119 sec latency 15.551 ms 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 -- 172.21.15.52:0/3237754050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9901076e0 msgr2=0x7fb990109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 --2- 172.21.15.52:0/3237754050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9901076e0 0x7fb990109b30 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb988009ca0 tx=0x7fb98802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 -- 172.21.15.52:0/3237754050 shutdown_connections 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 --2- 172.21.15.52:0/3237754050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb990104db0 0x7fb9901071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 --2- 172.21.15.52:0/3237754050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb9901076e0 0x7fb990109b30 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 --2- 172.21.15.52:0/3237754050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990102480 0x7fb990104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 -- 172.21.15.52:0/3237754050 >> 172.21.15.52:0/3237754050 conn(0x7fb9900fbe70 msgr2=0x7fb9900fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 -- 172.21.15.52:0/3237754050 shutdown_connections 2024-06-02T23:14:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.902+0000 7fb99829e640 1 -- 172.21.15.52:0/3237754050 wait complete. 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 Processor -- start 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 -- start start 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990102480 0x7fb990101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 0x7fb990100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9901076e0 0x7fb990100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010d530 con 0x7fb990104db0 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010d3b0 con 0x7fb990102480 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb99829e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010d6b0 con 0x7fb9901076e0 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.903+0000 7fb995839640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 0x7fb990100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 0x7fb990100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59130/0 (socket says 172.21.15.52:59130) 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb99603a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990102480 0x7fb990101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 -- 172.21.15.52:0/249328078 learned_addr learned my addr 172.21.15.52:0/249328078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb99683b640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9901076e0 0x7fb990100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 -- 172.21.15.52:0/249328078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9901076e0 msgr2=0x7fb990100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9901076e0 0x7fb990100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 -- 172.21.15.52:0/249328078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990102480 msgr2=0x7fb990101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990102480 0x7fb990101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 -- 172.21.15.52:0/249328078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb990101200 con 0x7fb990104db0 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb995839640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 0x7fb990100430 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb98000c970 tx=0x7fb98000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb9877fe640 1 -- 172.21.15.52:0/249328078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980014070 con 0x7fb990104db0 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9901a1c10 con 0x7fb990104db0 2024-06-02T23:14:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb9877fe640 1 -- 172.21.15.52:0/249328078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb980004440 con 0x7fb990104db0 2024-06-02T23:14:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb9877fe640 1 -- 172.21.15.52:0/249328078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb980003e50 con 0x7fb990104db0 2024-06-02T23:14:50.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.904+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9901a21b0 con 0x7fb990104db0 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.906+0000 7fb9877fe640 1 -- 172.21.15.52:0/249328078 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb980017070 con 0x7fb990104db0 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.907+0000 7fb99afa9740 1 Processor -- start 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.907+0000 7fb99afa9740 1 -- start start 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf4d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 0x55beecf52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99afa9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55beeceb04f0 con 0x55beecf52b70 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99afa9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55beece4e870 con 0x55beecf4da00 2024-06-02T23:14:50.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99afa9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55beeceac4b0 con 0x55beecf4d010 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 0x55beecf52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb995839640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb9877fe640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb97c0a2dd0 0x7fb97c0a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 0x55beecf52500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38824/0 (socket says 172.21.15.52:38824) 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 -- 172.21.15.52:0/1634488277 learned_addr learned my addr 172.21.15.52:0/1634488277 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99683b640 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf4d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99683b640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb97c0a2dd0 0x7fb97c0a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 -- 172.21.15.52:0/1634488277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 msgr2=0x55beecf4d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf4d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 -- 172.21.15.52:0/1634488277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 msgr2=0x55beecf55000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf55000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 -- 172.21.15.52:0/1634488277 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beece6a9b0 con 0x55beecf4da00 2024-06-02T23:14:50.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb9877fe640 1 -- 172.21.15.52:0/249328078 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb9800ceb20 con 0x7fb990104db0 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.908+0000 7fb99603a640 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 0x55beecf52500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009cf0 tx=0x7fb98c011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=242070430ad364b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.909+0000 7fb99683b640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb97c0a2dd0 0x7fb97c0a5290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55beecf4e260 tx=0x7fb98802a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.909+0000 7fb984bfe640 1 -- 172.21.15.52:0/1634488277 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c004230 con 0x55beecf4da00 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.909+0000 7fb984bfe640 1 -- 172.21.15.52:0/1634488277 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb98c0043d0 con 0x55beecf4da00 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.909+0000 7fb984bfe640 1 -- 172.21.15.52:0/1634488277 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c019890 con 0x55beecf4da00 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.909+0000 7fb99afa9740 1 -- 172.21.15.52:0/1634488277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 msgr2=0x55beecf52500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.909+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 0x55beecf52500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb98c009cf0 tx=0x7fb98c011810 comp rx=0 tx=0).stop 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- 172.21.15.52:0/1634488277 shutdown_connections 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf55000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4da00 0x55beecf52500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1634488277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- 172.21.15.52:0/1634488277 >> 172.21.15.52:0/1634488277 conn(0x55beecf48680 msgr2=0x55beecf48a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:50.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- 172.21.15.52:0/1634488277 shutdown_connections 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- 172.21.15.52:0/1634488277 wait complete. 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- start start 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 Processor -- start 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55beecf55b20 con 0x55beecf4da00 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55beece39ae0 con 0x55beecf52b70 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.910+0000 7fb99afa9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55beecf55ca0 con 0x55beecf4d010 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb99afa9740 10 client.0 _ll_register_callbacks cb 0x55beece729b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59150/0 (socket says 172.21.15.52:59150) 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb99683b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 -- 172.21.15.52:0/1359266505 learned_addr learned my addr 172.21.15.52:0/1359266505 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb99603a640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 -- 172.21.15.52:0/1359266505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 msgr2=0x55beecf572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 -- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 msgr2=0x55beecf5e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 -- 172.21.15.52:0/1359266505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beecbc4210 con 0x55beecf4da00 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb99683b640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb995839640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 0x55beecf577f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb98009b0b0 tx=0x7fb9800d21f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcf6dddcf30d7f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb979ffb640 1 -- 172.21.15.52:0/1359266505 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98009b180 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb979ffb640 1 -- 172.21.15.52:0/1359266505 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb980097100 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.911+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb98c0043d0 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb979ffb640 1 -- 172.21.15.52:0/1359266505 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb98009ad10 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 10 client.16446 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beecc90620 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb979ffb640 1 -- 172.21.15.52:0/1359266505 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb98c0043d0 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb979ffb640 1 -- 172.21.15.52:0/1359266505 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb980097300 con 0x55beecf4da00 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 10 client.16446 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 20 client.16446 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecf63250 0x55beecf65640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 4 client.16446 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55beecec3f10 con 0x55beecf63250 2024-06-02T23:14:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.912+0000 7fb99603a640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecf63250 0x55beecf65640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.913+0000 7fb99603a640 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecf63250 0x55beecf65640 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=197c37bfee6fb94b server_cookie=c25fdb6b3d0085cc in_seq=0 out_seq=0 2024-06-02T23:14:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.913+0000 7fb979ffb640 10 client.16446 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.913+0000 7fb979ffb640 1 -- 172.21.15.52:0/1359266505 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55beecec3f10 con 0x55beecf63250 2024-06-02T23:14:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.913+0000 7fb979ffb640 10 client.16446 handle_command_reply: tid=0 2024-06-02T23:14:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 client.16446 shutdown 2024-06-02T23:14:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 msgr2=0x55beecf577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 0x55beecf577f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb98009b0b0 tx=0x7fb9800d21f0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 shutdown_connections 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecf63250 0x55beecf65640 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 --2- 172.21.15.52:0/1359266505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 >> 172.21.15.52:0/1359266505 conn(0x55beecf48680 msgr2=0x55beecf477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb979ffb640 0 client.16446 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 shutdown_connections 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 1 -- 172.21.15.52:0/1359266505 wait complete. 2024-06-02T23:14:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.914+0000 7fb99afa9740 20 client.16446 trim_cache size 0 max 16384 2024-06-02T23:14:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.929+0000 7fb99afa9740 1 Processor -- start 2024-06-02T23:14:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.929+0000 7fb99afa9740 1 -- start start 2024-06-02T23:14:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99afa9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901aaec0 con 0x55beecf4d010 2024-06-02T23:14:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99afa9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901aca90 con 0x55beecf52b70 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99afa9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901ab590 con 0x55beecf4da00 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59164/0 (socket says 172.21.15.52:59164) 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 -- 172.21.15.52:0/4039346379 learned_addr learned my addr 172.21.15.52:0/4039346379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99683b640 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb995839640 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 -- 172.21.15.52:0/4039346379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 msgr2=0x55beecf577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 -- 172.21.15.52:0/4039346379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 msgr2=0x55beecf5e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 -- 172.21.15.52:0/4039346379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55beecf49350 con 0x55beecf4d010 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb99603a640 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 0x55beecf572b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb98c000c00 tx=0x7fb98c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529d04f53aee6ace server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb97bfff640 1 -- 172.21.15.52:0/4039346379 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c0052a0 con 0x55beecf4d010 2024-06-02T23:14:50.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb97bfff640 1 -- 172.21.15.52:0/4039346379 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb98c005a70 con 0x55beecf4d010 2024-06-02T23:14:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.930+0000 7fb97bfff640 1 -- 172.21.15.52:0/4039346379 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c004da0 con 0x55beecf4d010 2024-06-02T23:14:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 -- 172.21.15.52:0/4039346379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 msgr2=0x55beecf572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 0x55beecf572b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb98c000c00 tx=0x7fb98c005850 comp rx=0 tx=0).stop 2024-06-02T23:14:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 -- 172.21.15.52:0/4039346379 shutdown_connections 2024-06-02T23:14:50.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf52b70 0x55beecf5e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 --2- 172.21.15.52:0/4039346379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf4d010 0x55beecf572b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 -- 172.21.15.52:0/4039346379 >> 172.21.15.52:0/4039346379 conn(0x55beecf48680 msgr2=0x55beecf43560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 -- 172.21.15.52:0/4039346379 shutdown_connections 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 -- 172.21.15.52:0/4039346379 wait complete. 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 -- start start 2024-06-02T23:14:50.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.931+0000 7fb99afa9740 1 Processor -- start 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 0x55beecf5e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf5afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf5b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901ac330 con 0x55beecf52b70 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901abc60 con 0x55beecf4d010 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9901a8ea0 con 0x55beecf4da00 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99afa9740 10 client.0 _ll_register_callbacks cb 0x7fb980003850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:50.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99683b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf5b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 0x55beecf5e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 0x55beecf5e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38854/0 (socket says 172.21.15.52:38854) 2024-06-02T23:14:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 -- 172.21.15.52:0/2927292267 learned_addr learned my addr 172.21.15.52:0/2927292267 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99683b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf5b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59180/0 (socket says 172.21.15.52:59180) 2024-06-02T23:14:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb995839640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf5afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 -- 172.21.15.52:0/2927292267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 msgr2=0x55beecf5afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf5afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 -- 172.21.15.52:0/2927292267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 msgr2=0x55beecf5b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf5b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99603a640 1 -- 172.21.15.52:0/2927292267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55beecf5ba60 con 0x55beecf4d010 2024-06-02T23:14:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.932+0000 7fb99683b640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf5b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:50.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb99603a640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 0x55beecf5e4f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb98c01b040 tx=0x7fb98c0096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42ca6a00e2c230c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb979ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb979ffb640 1 -- 172.21.15.52:0/2927292267 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98c005360 con 0x55beecf4d010 2024-06-02T23:14:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb98c005a70 con 0x55beecf4d010 2024-06-02T23:14:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb979ffb640 1 -- 172.21.15.52:0/2927292267 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb98c004b20 con 0x55beecf4d010 2024-06-02T23:14:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb979ffb640 1 -- 172.21.15.52:0/2927292267 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb98c004d20 con 0x55beecf4d010 2024-06-02T23:14:50.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb99afa9740 10 client.26015 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55beecfccef0 con 0x55beecf4d010 2024-06-02T23:14:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb979ffb640 1 -- 172.21.15.52:0/2927292267 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb98c005a70 con 0x55beecf4d010 2024-06-02T23:14:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb979ffb640 1 -- 172.21.15.52:0/2927292267 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb98c004060 con 0x55beecf4d010 2024-06-02T23:14:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb99afa9740 10 client.26015 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.933+0000 7fb99afa9740 10 client.26015 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:50.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb99afa9740 20 client.26015 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb99afa9740 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecfcd790 0x55beecfcfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb99afa9740 4 client.26015 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:14:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55beecf484d0 con 0x55beecfcd790 2024-06-02T23:14:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb995839640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecfcd790 0x55beecfcfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb995839640 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecfcd790 0x55beecfcfbb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f387192c86f5ec2 server_cookie=2288eacf8bbcb534 in_seq=0 out_seq=0 2024-06-02T23:14:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.934+0000 7fb979ffb640 10 client.26015 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb979ffb640 1 -- 172.21.15.52:0/2927292267 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55beecf484d0 con 0x55beecfcd790 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb979ffb640 10 client.26015 handle_command_reply: tid=0 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 client.26015 shutdown 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 msgr2=0x55beecf5e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 0x55beecf5e4f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb98c01b040 tx=0x7fb98c0096d0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 shutdown_connections 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55beecfcd790 0x55beecfcfbb0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55beecf4da00 0x55beecf5afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55beecf52b70 0x55beecf5b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb979ffb640 0 client.26015 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 --2- 172.21.15.52:0/2927292267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55beecf4d010 0x55beecf5e4f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 >> 172.21.15.52:0/2927292267 conn(0x55beecf48680 msgr2=0x55beecf572b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 shutdown_connections 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 1 -- 172.21.15.52:0/2927292267 wait complete. 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.954+0000 7fb99afa9740 20 client.26015 trim_cache size 0 max 16384 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:14:50.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb97c0a2dd0 msgr2=0x7fb97c0a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb97c0a2dd0 0x7fb97c0a5290 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x55beecf4e260 tx=0x7fb98802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 msgr2=0x7fb990100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 0x7fb990100430 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb98000c970 tx=0x7fb98000ce30 comp rx=0 tx=0).stop 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 shutdown_connections 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb97c0a2dd0 0x7fb97c0a5290 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9901076e0 0x7fb990100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb990104db0 0x7fb990100430 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 --2- 172.21.15.52:0/249328078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb990102480 0x7fb990101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 >> 172.21.15.52:0/249328078 conn(0x7fb9900fbe70 msgr2=0x7fb990102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 shutdown_connections 2024-06-02T23:14:50.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:14:50.955+0000 7fb99829e640 1 -- 172.21.15.52:0/249328078 wait complete. 2024-06-02T23:14:51.396 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:51 smithi052 ceph-mon[32340]: pgmap v520: 97 pgs: 97 active+clean; 37 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 475 KiB/s wr, 39 op/s 2024-06-02T23:14:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:51 smithi123 ceph-mon[35977]: pgmap v520: 97 pgs: 97 active+clean; 37 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 475 KiB/s wr, 39 op/s 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 -- 172.21.15.52:0/2471601127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d8101fa0 msgr2=0x7f68d8105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 --2- 172.21.15.52:0/2471601127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d8101fa0 0x7f68d8105fe0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f68c0009990 tx=0x7f68c0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 -- 172.21.15.52:0/2471601127 shutdown_connections 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 --2- 172.21.15.52:0/2471601127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 0x7f68d810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 --2- 172.21.15.52:0/2471601127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d81015f0 0x7f68d81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 --2- 172.21.15.52:0/2471601127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d8101fa0 0x7f68d8105fe0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 -- 172.21.15.52:0/2471601127 >> 172.21.15.52:0/2471601127 conn(0x7f68d8078330 msgr2=0x7f68d8078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:51.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 -- 172.21.15.52:0/2471601127 shutdown_connections 2024-06-02T23:14:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.530+0000 7f68de2d3700 1 -- 172.21.15.52:0/2471601127 wait complete. 2024-06-02T23:14:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 Processor -- start 2024-06-02T23:14:51.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 -- start start 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d81015f0 0x7f68d8103080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8101fa0 0x7f68d81035c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 0x7f68d8103b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68d8111550 con 0x7f68d81066c0 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81113d0 con 0x7f68d81015f0 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68de2d3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81116d0 con 0x7f68d8101fa0 2024-06-02T23:14:51.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68dc870700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 0x7f68d8103b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d81015f0 0x7f68d8103080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68dc870700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 0x7f68d8103b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59188/0 (socket says 172.21.15.52:59188) 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68dc870700 1 -- 172.21.15.52:0/1475800906 learned_addr learned my addr 172.21.15.52:0/1475800906 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.531+0000 7f68d7fff700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8101fa0 0x7f68d81035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68cffff700 1 -- 172.21.15.52:0/1475800906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8101fa0 msgr2=0x7f68d81035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68cffff700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8101fa0 0x7f68d81035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68cffff700 1 -- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 msgr2=0x7f68d8103b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68cffff700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 0x7f68d8103b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68cffff700 1 -- 172.21.15.52:0/1475800906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68d81a7040 con 0x7f68d81015f0 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68cffff700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d81015f0 0x7f68d8103080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d750 tx=0x7f68c800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68d5ffb700 1 -- 172.21.15.52:0/1475800906 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c8016070 con 0x7f68d81015f0 2024-06-02T23:14:51.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68d81a7330 con 0x7f68d81015f0 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68d5ffb700 1 -- 172.21.15.52:0/1475800906 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68c8005020 con 0x7f68d81015f0 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68d5ffb700 1 -- 172.21.15.52:0/1475800906 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c800f9f0 con 0x7f68d81015f0 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.532+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68d81a7840 con 0x7f68d81015f0 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.534+0000 7f68d5ffb700 1 -- 172.21.15.52:0/1475800906 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f68c80051d0 con 0x7f68d81015f0 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.535+0000 7f68d5ffb700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68c40a2d40 0x7f68c40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.535+0000 7f68d7fff700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68c40a2d40 0x7f68c40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.535+0000 7f68d5ffb700 1 -- 172.21.15.52:0/1475800906 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68c80ce960 con 0x7f68d81015f0 2024-06-02T23:14:51.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.536+0000 7f68d7fff700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68c40a2d40 0x7f68c40a5200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f68c0006010 tx=0x7f68c0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.536+0000 7f68e8b56b80 1 Processor -- start 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.536+0000 7f68e8b56b80 1 -- start start 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea45a0 0x556532ea4980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e9e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 0x556532e9f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556532d91100 con 0x556532e9ecf0 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556532def9f0 con 0x556532ea45a0 2024-06-02T23:14:51.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556532d57600 con 0x556532ea4ec0 2024-06-02T23:14:51.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 0x556532e9f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 0x556532e9f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59202/0 (socket says 172.21.15.52:59202) 2024-06-02T23:14:51.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 -- 172.21.15.52:0/1875435076 learned_addr learned my addr 172.21.15.52:0/1875435076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:51.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68cffff700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea45a0 0x556532ea4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68d7fff700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e9e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 -- 172.21.15.52:0/1875435076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 msgr2=0x556532e9e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e9e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 -- 172.21.15.52:0/1875435076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea45a0 msgr2=0x556532ea4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea45a0 0x556532ea4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68dc870700 1 -- 172.21.15.52:0/1875435076 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556532d3ea70 con 0x556532e9ecf0 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68cffff700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea45a0 0x556532ea4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.537+0000 7f68d7fff700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e9e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.538+0000 7f68dc870700 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 0x556532e9f110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e6a0 tx=0x7f68d0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=825fcb5f34c4b7bc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.538+0000 7f68ce7fc700 1 -- 172.21.15.52:0/1875435076 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0019030 con 0x556532e9ecf0 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.538+0000 7f68ce7fc700 1 -- 172.21.15.52:0/1875435076 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68d00093a0 con 0x556532e9ecf0 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.538+0000 7f68ce7fc700 1 -- 172.21.15.52:0/1875435076 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0007500 con 0x556532e9ecf0 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1875435076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 msgr2=0x556532e9f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 0x556532e9f110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f68d000e6a0 tx=0x7f68d0009920 comp rx=0 tx=0).stop 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1875435076 shutdown_connections 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532e9ecf0 0x556532e9f110 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea45a0 0x556532ea4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1875435076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e9e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1875435076 >> 172.21.15.52:0/1875435076 conn(0x556532dcb220 msgr2=0x556532e99aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1875435076 shutdown_connections 2024-06-02T23:14:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1875435076 wait complete. 2024-06-02T23:14:51.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.539+0000 7f68e8b56b80 1 -- start start 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 Processor -- start 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532e9ecf0 0x556532eaab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532ea7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532ea8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556532d90160 con 0x556532ea45a0 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556532deb9b0 con 0x556532e9ecf0 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556532eab8a0 con 0x556532ea4ec0 2024-06-02T23:14:51.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68e8b56b80 10 client.0 _ll_register_callbacks cb 0x556532d85750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532ea7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68cffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532e9ecf0 0x556532eaab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532ea7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59210/0 (socket says 172.21.15.52:59210) 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68d7fff700 1 -- 172.21.15.52:0/645948921 learned_addr learned my addr 172.21.15.52:0/645948921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68dc870700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532ea8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68cffff700 1 -- 172.21.15.52:0/645948921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 msgr2=0x556532ea8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68cffff700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532ea8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.540+0000 7f68cffff700 1 -- 172.21.15.52:0/645948921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 msgr2=0x556532ea7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68cffff700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532ea7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68cffff700 1 -- 172.21.15.52:0/645948921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556532db58f0 con 0x556532e9ecf0 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68d7fff700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532ea7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:51.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68dc870700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532ea8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68cffff700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532e9ecf0 0x556532eaab30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d720 tx=0x7f68c8005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd36554cff5dde88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68b77fe700 1 -- 172.21.15.52:0/645948921 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c801f050 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68b77fe700 1 -- 172.21.15.52:0/645948921 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68c80b4160 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556532dfcef0 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68b77fe700 1 -- 172.21.15.52:0/645948921 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68c809f370 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68e8b56b80 10 client.26027 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556532eb5f50 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.541+0000 7f68b77fe700 1 -- 172.21.15.52:0/645948921 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556532dfcef0 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68b77fe700 1 -- 172.21.15.52:0/645948921 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f68c80a39d0 con 0x556532e9ecf0 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68e8b56b80 10 client.26027 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68e8b56b80 10 client.26027 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68e8b56b80 20 client.26027 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:51.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532eb8660 0x556532ebaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68e8b56b80 4 client.26027 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556532df7470 con 0x556532eb8660 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68d7fff700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532eb8660 0x556532ebaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68b77fe700 10 client.26027 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.542+0000 7f68d7fff700 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532eb8660 0x556532ebaaa0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ed63b1a2a2c16db server_cookie=a8f6df362b97e21d in_seq=0 out_seq=0 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68b77fe700 1 -- 172.21.15.52:0/645948921 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556532df7470 con 0x556532eb8660 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68b77fe700 10 client.26027 handle_command_reply: tid=0 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 client.26027 shutdown 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532e9ecf0 msgr2=0x556532eaab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532e9ecf0 0x556532eaab30 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d720 tx=0x7f68c8005640 comp rx=0 tx=0).stop 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 shutdown_connections 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532eb8660 0x556532ebaaa0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532ea8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532e9ecf0 0x556532eaab30 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/645948921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532ea7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 >> 172.21.15.52:0/645948921 conn(0x556532dcb220 msgr2=0x556532e98c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.543+0000 7f68b77fe700 0 client.26027 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.544+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 shutdown_connections 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.544+0000 7f68e8b56b80 1 -- 172.21.15.52:0/645948921 wait complete. 2024-06-02T23:14:51.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.544+0000 7f68e8b56b80 20 client.26027 trim_cache size 0 max 16384 2024-06-02T23:14:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 Processor -- start 2024-06-02T23:14:51.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 -- start start 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532e9e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e993d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532eb4a50 0x556532e99910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81aec10 con 0x556532ea45a0 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81b07e0 con 0x556532eb4a50 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81af2e0 con 0x556532ea4ec0 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532e9e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68cffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532e9e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59216/0 (socket says 172.21.15.52:59216) 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68cffff700 1 -- 172.21.15.52:0/2281762417 learned_addr learned my addr 172.21.15.52:0/2281762417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68d7fff700 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68dc870700 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532eb4a50 0x556532e99910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68d7fff700 1 -- 172.21.15.52:0/2281762417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532eb4a50 msgr2=0x556532e99910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68d7fff700 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532eb4a50 0x556532e99910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68d7fff700 1 -- 172.21.15.52:0/2281762417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 msgr2=0x556532e9e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.568+0000 7f68d7fff700 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532e9e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.569+0000 7f68d7fff700 1 -- 172.21.15.52:0/2281762417 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556532ea63a0 con 0x556532ea4ec0 2024-06-02T23:14:51.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.569+0000 7f68cffff700 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532e9e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.569+0000 7f68d7fff700 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e993d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68c0005cd0 tx=0x7f68c0028040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=856f621c91694627 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.569+0000 7f68cdffb700 1 -- 172.21.15.52:0/2281762417 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0003680 con 0x556532ea4ec0 2024-06-02T23:14:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.569+0000 7f68cdffb700 1 -- 172.21.15.52:0/2281762417 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68c0003cd0 con 0x556532ea4ec0 2024-06-02T23:14:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.569+0000 7f68cdffb700 1 -- 172.21.15.52:0/2281762417 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c000fc60 con 0x556532ea4ec0 2024-06-02T23:14:51.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 -- 172.21.15.52:0/2281762417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 msgr2=0x556532e993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e993d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f68c0005cd0 tx=0x7f68c0028040 comp rx=0 tx=0).stop 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 -- 172.21.15.52:0/2281762417 shutdown_connections 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532eb4a50 0x556532e99910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532ea45a0 0x556532e9e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/2281762417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea4ec0 0x556532e993d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 -- 172.21.15.52:0/2281762417 >> 172.21.15.52:0/2281762417 conn(0x556532dcb220 msgr2=0x556532ea9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 -- 172.21.15.52:0/2281762417 shutdown_connections 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 -- 172.21.15.52:0/2281762417 wait complete. 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.570+0000 7f68e8b56b80 1 -- start start 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 Processor -- start 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea45a0 0x556532eb0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 0x556532eb1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 0x556532eadb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81b0080 con 0x556532eb1640 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81af9b0 con 0x556532ea4ec0 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68dc870700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 0x556532eadb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68d7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 0x556532eb1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68d81acbd0 con 0x556532ea45a0 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68dc870700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 0x556532eadb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59226/0 (socket says 172.21.15.52:59226) 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68dc870700 1 -- 172.21.15.52:0/1532765479 learned_addr learned my addr 172.21.15.52:0/1532765479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68cffff700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea45a0 0x556532eb0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68d7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 0x556532eb1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38922/0 (socket says 172.21.15.52:38922) 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68e8b56b80 10 client.0 _ll_register_callbacks cb 0x556532d85750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68dc870700 1 -- 172.21.15.52:0/1532765479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea45a0 msgr2=0x556532eb0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.571+0000 7f68dc870700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea45a0 0x556532eb0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68dc870700 1 -- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 msgr2=0x556532eb1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68dc870700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 0x556532eb1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68dc870700 1 -- 172.21.15.52:0/1532765479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556532eae2e0 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68d7fff700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 0x556532eb1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68dc870700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 0x556532eadb90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f68d000ed40 tx=0x7f68d0020690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e78cd119ff5c4a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68b77fe700 1 -- 172.21.15.52:0/1532765479 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0020c10 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c0003cd0 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68b77fe700 1 -- 172.21.15.52:0/1532765479 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68d0020db0 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68b77fe700 1 -- 172.21.15.52:0/1532765479 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f68d0015450 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68e8b56b80 10 client.16470 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556532f23d30 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.572+0000 7f68b77fe700 1 -- 172.21.15.52:0/1532765479 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68c0003cd0 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68b77fe700 1 -- 172.21.15.52:0/1532765479 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f68d0027070 con 0x556532eb1640 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68e8b56b80 10 client.16470 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68e8b56b80 10 client.16470 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68e8b56b80 20 client.16470 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:51.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532f24310 0x556532f26730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:51.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68e8b56b80 4 client.16470 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.75, "await_for": 45.0}] 2024-06-02T23:14:51.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.75, "await_for": 45.0}) -- 0x556532dab0f0 con 0x556532f24310 2024-06-02T23:14:51.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68cffff700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532f24310 0x556532f26730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:51.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68b77fe700 10 client.16470 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:51.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.573+0000 7f68cffff700 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532f24310 0x556532f26730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f14d2f81bfa30b3b server_cookie=695fccb07b7130da in_seq=0 out_seq=0 2024-06-02T23:14:51.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68b77fe700 1 -- 172.21.15.52:0/1532765479 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x556532dab0f0 con 0x556532f24310 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68b77fe700 10 client.16470 handle_command_reply: tid=0 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 client.16470 shutdown 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 msgr2=0x556532eadb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 0x556532eadb90 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f68d000ed40 tx=0x7f68d0020690 comp rx=0 tx=0).stop 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 shutdown_connections 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556532f24310 0x556532f26730 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556532eb1640 0x556532eadb90 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556532ea45a0 0x556532eb0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 --2- 172.21.15.52:0/1532765479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556532ea4ec0 0x556532eb1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 >> 172.21.15.52:0/1532765479 conn(0x556532dcb220 msgr2=0x556532ea76e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68b77fe700 0 client.16470 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:51.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 shutdown_connections 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 1 -- 172.21.15.52:0/1532765479 wait complete. 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.638+0000 7f68e8b56b80 20 client.16470 trim_cache size 0 max 16384 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 30, 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 14, 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "5251b677": { 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 14, 2024-06-02T23:14:51.640 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.8, 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:51.641 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:51.642 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:51.642 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:51.642 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:14:51.642 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:14:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68c40a2d40 msgr2=0x7f68c40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68c40a2d40 0x7f68c40a5200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f68c0006010 tx=0x7f68c0005960 comp rx=0 tx=0).stop 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d81015f0 msgr2=0x7f68d8103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d81015f0 0x7f68d8103080 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f68c800d750 tx=0x7f68c800da60 comp rx=0 tx=0).stop 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 shutdown_connections 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68c40a2d40 0x7f68c40a5200 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f68d81066c0 0x7f68d8103b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68d81015f0 0x7f68d8103080 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 --2- 172.21.15.52:0/1475800906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68d8101fa0 0x7f68d81035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 >> 172.21.15.52:0/1475800906 conn(0x7f68d8078330 msgr2=0x7f68d806a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 shutdown_connections 2024-06-02T23:14:51.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:51.639+0000 7f68de2d3700 1 -- 172.21.15.52:0/1475800906 wait complete. 2024-06-02T23:14:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:51 smithi174 ceph-mon[35994]: pgmap v520: 97 pgs: 97 active+clean; 37 MiB data, 599 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 475 KiB/s wr, 39 op/s 2024-06-02T23:14:52.425 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5251b677', quiesce took 7.4 seconds. Will release after: 20.35 2024-06-02T23:14:52.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 1 sec latency 1011.863 ms 2024-06-02T23:14:53.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:53 smithi123 ceph-mon[35977]: pgmap v521: 97 pgs: 97 active+clean; 38 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 706 KiB/s wr, 78 op/s 2024-06-02T23:14:53.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 2 sec latency 2011.980 ms 2024-06-02T23:14:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:53 smithi174 ceph-mon[35994]: pgmap v521: 97 pgs: 97 active+clean; 38 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 706 KiB/s wr, 78 op/s 2024-06-02T23:14:53.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:53 smithi052 ceph-mon[32340]: pgmap v521: 97 pgs: 97 active+clean; 38 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 706 KiB/s wr, 78 op/s 2024-06-02T23:14:54.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 3 sec latency 3012.039 ms 2024-06-02T23:14:55.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:55 smithi123 ceph-mon[35977]: pgmap v522: 97 pgs: 97 active+clean; 38 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 715 KiB/s wr, 80 op/s 2024-06-02T23:14:55.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 4 sec latency 4012.167 ms 2024-06-02T23:14:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:55 smithi174 ceph-mon[35994]: pgmap v522: 97 pgs: 97 active+clean; 38 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 715 KiB/s wr, 80 op/s 2024-06-02T23:14:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:55 smithi052 ceph-mon[32340]: pgmap v522: 97 pgs: 97 active+clean; 38 MiB data, 623 MiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 715 KiB/s wr, 80 op/s 2024-06-02T23:14:56.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 5 sec latency 5012.269 ms 2024-06-02T23:14:56.693 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:14:56.960 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:14:57.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:57 smithi052 ceph-mon[32340]: pgmap v523: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 813 KiB/s wr, 107 op/s 2024-06-02T23:14:57.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:57 smithi123 ceph-mon[35977]: pgmap v523: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 813 KiB/s wr, 107 op/s 2024-06-02T23:14:57.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 6 sec latency 6012.356 ms 2024-06-02T23:14:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:57 smithi174 ceph-mon[35994]: pgmap v523: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 813 KiB/s wr, 107 op/s 2024-06-02T23:14:58.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 7 sec latency 7012.406 ms 2024-06-02T23:14:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.701+0000 7f7966636700 1 -- 172.21.15.52:0/3930684333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 msgr2=0x7f796006ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.701+0000 7f7966636700 1 --2- 172.21.15.52:0/3930684333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796006ae80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f7954009990 tx=0x7f7954009ca0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.701+0000 7f7966636700 1 -- 172.21.15.52:0/3930684333 shutdown_connections 2024-06-02T23:14:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 --2- 172.21.15.52:0/3930684333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 0x7f796010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 --2- 172.21.15.52:0/3930684333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79601084f0 0x7f79601088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 --2- 172.21.15.52:0/3930684333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796006ae80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 -- 172.21.15.52:0/3930684333 >> 172.21.15.52:0/3930684333 conn(0x7f7960078300 msgr2=0x7f7960078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 -- 172.21.15.52:0/3930684333 shutdown_connections 2024-06-02T23:14:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 -- 172.21.15.52:0/3930684333 wait complete. 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.702+0000 7f7966636700 1 Processor -- start 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 -- start start 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 0x7f796019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79601084f0 0x7f79601989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7960112650 con 0x7f796006aa00 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79601124d0 con 0x7f79601084f0 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7966636700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79601127d0 con 0x7f796006b3c0 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f795f7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 0x7f796019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796019e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48036/0 (socket says 172.21.15.52:48036) 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f795ffff700 1 -- 172.21.15.52:0/1752712624 learned_addr learned my addr 172.21.15.52:0/1752712624 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.703+0000 7f7964bd3700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79601084f0 0x7f79601989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795f7fe700 1 -- 172.21.15.52:0/1752712624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79601084f0 msgr2=0x7f79601989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795f7fe700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79601084f0 0x7f79601989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795f7fe700 1 -- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 msgr2=0x7f796019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795f7fe700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795f7fe700 1 -- 172.21.15.52:0/1752712624 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7960199110 con 0x7f796006b3c0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795f7fe700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 0x7f796019ed30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f7954005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795d7fa700 1 -- 172.21.15.52:0/1752712624 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954003680 con 0x7f796006b3c0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79601993a0 con 0x7f796006b3c0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795d7fa700 1 -- 172.21.15.52:0/1752712624 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7954004ce0 con 0x7f796006b3c0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f795d7fa700 1 -- 172.21.15.52:0/1752712624 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7954003cc0 con 0x7f796006b3c0 2024-06-02T23:14:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.704+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7960101380 con 0x7f796006b3c0 2024-06-02T23:14:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.707+0000 7f795d7fa700 1 -- 172.21.15.52:0/1752712624 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7954005020 con 0x7f796006b3c0 2024-06-02T23:14:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.708+0000 7f795d7fa700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f79480a70b0 0x7f79480a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.708+0000 7f795d7fa700 1 -- 172.21.15.52:0/1752712624 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f79540cfd20 con 0x7f796006b3c0 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.708+0000 7f795ffff700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f79480a70b0 0x7f79480a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.708+0000 7f7970eb9b80 1 Processor -- start 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.708+0000 7f7970eb9b80 1 -- start start 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b38340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b32260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f372a24f60 con 0x55f372b327a0 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f372a83850 con 0x55f372b37f60 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3729eb460 con 0x55f372b38880 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f795ffff700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f79480a70b0 0x7f79480a9570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7960199af0 tx=0x7f7950007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f795ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b38340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f795ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b38340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33922/0 (socket says 172.21.15.52:33922) 2024-06-02T23:14:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f795ffff700 1 -- 172.21.15.52:0/232723748 learned_addr learned my addr 172.21.15.52:0/232723748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f795f7fe700 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b32260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.709+0000 7f7964bd3700 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795ffff700 1 -- 172.21.15.52:0/232723748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 msgr2=0x55f372b32260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795ffff700 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b32260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795ffff700 1 -- 172.21.15.52:0/232723748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 msgr2=0x55f372b32bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795ffff700 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795ffff700 1 -- 172.21.15.52:0/232723748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3729d28d0 con 0x55f372b37f60 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795f7fe700 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b32260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f795ffff700 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b38340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7950005fd0 tx=0x7f7950013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ff1e60aa4cddc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f79467fc700 1 -- 172.21.15.52:0/232723748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950011db0 con 0x55f372b37f60 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f79467fc700 1 -- 172.21.15.52:0/232723748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7950010e60 con 0x55f372b37f60 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.710+0000 7f79467fc700 1 -- 172.21.15.52:0/232723748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795001d710 con 0x55f372b37f60 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 -- 172.21.15.52:0/232723748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 msgr2=0x55f372b38340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b38340 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7950005fd0 tx=0x7f7950013000 comp rx=0 tx=0).stop 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 -- 172.21.15.52:0/232723748 shutdown_connections 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b38340 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/232723748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b32260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.711+0000 7f7970eb9b80 1 -- 172.21.15.52:0/232723748 >> 172.21.15.52:0/232723748 conn(0x55f372a5f080 msgr2=0x55f372b2d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.712+0000 7f7970eb9b80 1 -- 172.21.15.52:0/232723748 shutdown_connections 2024-06-02T23:14:58.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.712+0000 7f7970eb9b80 1 -- 172.21.15.52:0/232723748 wait complete. 2024-06-02T23:14:58.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.712+0000 7f7970eb9b80 1 -- start start 2024-06-02T23:14:58.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.712+0000 7f7970eb9b80 1 Processor -- start 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b3e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b3b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b3be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f372a23fc0 con 0x55f372b327a0 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f372a7f810 con 0x55f372b37f60 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f372b3f260 con 0x55f372b38880 2024-06-02T23:14:58.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7970eb9b80 10 client.0 _ll_register_callbacks cb 0x55f372a195b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b3e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b3e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48052/0 (socket says 172.21.15.52:48052) 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f795ffff700 1 -- 172.21.15.52:0/3794832393 learned_addr learned my addr 172.21.15.52:0/3794832393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7964bd3700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b3be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f795f7fe700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b3b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7964bd3700 1 -- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 msgr2=0x55f372b3b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7964bd3700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b3b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7964bd3700 1 -- 172.21.15.52:0/3794832393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 msgr2=0x55f372b3e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7964bd3700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b3e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.713+0000 7f7964bd3700 1 -- 172.21.15.52:0/3794832393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f372a49750 con 0x55f372b38880 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f7964bd3700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b3be50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79580040c0 tx=0x7f79580040f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=407317f9068aa1af server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f793f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f793f7fe700 1 -- 172.21.15.52:0/3794832393 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795801a070 con 0x55f372b38880 2024-06-02T23:14:58.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f793f7fe700 1 -- 172.21.15.52:0/3794832393 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7958004db0 con 0x55f372b38880 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7950010e60 con 0x55f372b38880 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f793f7fe700 1 -- 172.21.15.52:0/3794832393 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7958007500 con 0x55f372b38880 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f7970eb9b80 10 client.25948 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.714+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f372b49ac0 con 0x55f372b38880 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f793f7fe700 1 -- 172.21.15.52:0/3794832393 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7950010e60 con 0x55f372b38880 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f793f7fe700 1 -- 172.21.15.52:0/3794832393 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f79580168e0 con 0x55f372b38880 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f7970eb9b80 10 client.25948 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:58.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f7970eb9b80 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f7970eb9b80 20 client.25948 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372b4c1d0 0x55f372b4e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f7970eb9b80 4 client.25948 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.715+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f372a90d50 con 0x55f372b4c1d0 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.716+0000 7f795ffff700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372b4c1d0 0x55f372b4e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.716+0000 7f793f7fe700 10 client.25948 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:58.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.716+0000 7f795ffff700 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372b4c1d0 0x55f372b4e5f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12bf41c849f2f257 server_cookie=3350d68233238129 in_seq=0 out_seq=0 2024-06-02T23:14:58.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f793f7fe700 1 -- 172.21.15.52:0/3794832393 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f372a90d50 con 0x55f372b4c1d0 2024-06-02T23:14:58.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f793f7fe700 10 client.25948 handle_command_reply: tid=0 2024-06-02T23:14:58.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 client.25948 shutdown 2024-06-02T23:14:58.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 msgr2=0x55f372b3be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b3be50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f79580040c0 tx=0x7f79580040f0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 shutdown_connections 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372b4c1d0 0x55f372b4e5f0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b38880 0x55f372b3be50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b3e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3794832393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b37f60 0x55f372b3b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 >> 172.21.15.52:0/3794832393 conn(0x55f372a5f080 msgr2=0x55f372b2c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.717+0000 7f793f7fe700 0 client.25948 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.718+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 shutdown_connections 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.718+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3794832393 wait complete. 2024-06-02T23:14:58.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.718+0000 7f7970eb9b80 20 client.25948 trim_cache size 0 max 16384 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.751+0000 7f7970eb9b80 1 Processor -- start 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.751+0000 7f7970eb9b80 1 -- start start 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b3a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b3e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79601aa790 con 0x55f372b327a0 2024-06-02T23:14:58.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79601ac360 con 0x55f372b38880 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79601aae60 con 0x55f372b37f60 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48054/0 (socket says 172.21.15.52:48054) 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 -- 172.21.15.52:0/1406245577 learned_addr learned my addr 172.21.15.52:0/1406245577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7964bd3700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b3e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795f7fe700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b3a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 -- 172.21.15.52:0/1406245577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 msgr2=0x55f372b3a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b3a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 -- 172.21.15.52:0/1406245577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 msgr2=0x55f372b3e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b3e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 -- 172.21.15.52:0/1406245577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f372b38680 con 0x55f372b327a0 2024-06-02T23:14:58.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7964bd3700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b3e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:58.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795f7fe700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b3a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:14:58.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f795ffff700 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32260 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f79500059c0 tx=0x7f7950011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41666d9651f46d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:58.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.752+0000 7f7945ffb700 1 -- 172.21.15.52:0/1406245577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950005040 con 0x55f372b327a0 2024-06-02T23:14:58.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7945ffb700 1 -- 172.21.15.52:0/1406245577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79500046f0 con 0x55f372b327a0 2024-06-02T23:14:58.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7945ffb700 1 -- 172.21.15.52:0/1406245577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795001e680 con 0x55f372b327a0 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 -- 172.21.15.52:0/1406245577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 msgr2=0x55f372b32260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32260 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f79500059c0 tx=0x7f7950011bb0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 -- 172.21.15.52:0/1406245577 shutdown_connections 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b3e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b32260 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/1406245577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b3a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 -- 172.21.15.52:0/1406245577 >> 172.21.15.52:0/1406245577 conn(0x55f372a5f080 msgr2=0x55f372b372f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 -- 172.21.15.52:0/1406245577 shutdown_connections 2024-06-02T23:14:58.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.753+0000 7f7970eb9b80 1 -- 172.21.15.52:0/1406245577 wait complete. 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 -- start start 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 Processor -- start 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b44920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b53490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b539d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f79601abc00 con 0x55f372b327a0 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f79601ab530 con 0x55f372b38880 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79601a8750 con 0x55f372b37f60 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.754+0000 7f7970eb9b80 10 client.0 _ll_register_callbacks cb 0x55f372a195b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b44920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b44920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48066/0 (socket says 172.21.15.52:48066) 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 -- 172.21.15.52:0/3733125847 learned_addr learned my addr 172.21.15.52:0/3733125847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f7964bd3700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795f7fe700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b53490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 -- 172.21.15.52:0/3733125847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 msgr2=0x55f372b53490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b53490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 -- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 msgr2=0x55f372b539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 -- 172.21.15.52:0/3733125847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f372b53f10 con 0x55f372b327a0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f795ffff700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b44920 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f79500108c0 tx=0x7f79500108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90f20fabc645c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f793f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f793f7fe700 1 -- 172.21.15.52:0/3733125847 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79500054d0 con 0x55f372b327a0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f793f7fe700 1 -- 172.21.15.52:0/3733125847 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f79500048a0 con 0x55f372b327a0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.755+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79500046f0 con 0x55f372b327a0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f793f7fe700 1 -- 172.21.15.52:0/3733125847 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7950004aa0 con 0x55f372b327a0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 10 client.16488 fetch_fsmap learned FSMap version 30 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f372bb6a70 con 0x55f372b327a0 2024-06-02T23:14:58.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f793f7fe700 1 -- 172.21.15.52:0/3733125847 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f79500046f0 con 0x55f372b327a0 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f793f7fe700 1 -- 172.21.15.52:0/3733125847 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7950050480 con 0x55f372b327a0 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 10 client.16488 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 10 client.16488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 20 client.16488 populate_metadata read hostname 'smithi052' 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372bb6f20 0x55f372bb9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 4 client.16488 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f372a8b2d0 con 0x55f372bb6f20 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.756+0000 7f795f7fe700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372bb6f20 0x55f372bb9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f795f7fe700 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372bb6f20 0x55f372bb9340 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5276abba90313a3 server_cookie=de6cf3318bf45719 in_seq=0 out_seq=0 2024-06-02T23:14:58.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f793f7fe700 10 client.16488 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f793f7fe700 1 -- 172.21.15.52:0/3733125847 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f372a8b2d0 con 0x55f372bb6f20 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f793f7fe700 10 client.16488 handle_command_reply: tid=0 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 client.16488 shutdown 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 msgr2=0x55f372b44920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b44920 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f79500108c0 tx=0x7f79500108f0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 shutdown_connections 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f372bb6f20 0x55f372bb9340 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f372b38880 0x55f372b539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f372b327a0 0x55f372b44920 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 --2- 172.21.15.52:0/3733125847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f372b37f60 0x55f372b53490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 >> 172.21.15.52:0/3733125847 conn(0x55f372a5f080 msgr2=0x55f372b3b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f793f7fe700 0 client.16488 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 shutdown_connections 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.757+0000 7f7970eb9b80 1 -- 172.21.15.52:0/3733125847 wait complete. 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7970eb9b80 20 client.16488 trim_cache size 0 max 16384 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:14:58.764 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:14:58.765 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:14:58.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f79480a70b0 msgr2=0x7f79480a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f79480a70b0 0x7f79480a9570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7960199af0 tx=0x7f7950007480 comp rx=0 tx=0).stop 2024-06-02T23:14:58.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 msgr2=0x7f796019ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:14:58.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 0x7f796019ed30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f7954000c00 tx=0x7f7954005f10 comp rx=0 tx=0).stop 2024-06-02T23:14:58.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 shutdown_connections 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f79480a70b0 0x7f79480a9570 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79601084f0 0x7f79601989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f796006aa00 0x7f796019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 --2- 172.21.15.52:0/1752712624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f796006b3c0 0x7f796019ed30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 >> 172.21.15.52:0/1752712624 conn(0x7f7960078300 msgr2=0x7f79601000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 shutdown_connections 2024-06-02T23:14:58.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:14:58.758+0000 7f7966636700 1 -- 172.21.15.52:0/1752712624 wait complete. 2024-06-02T23:14:59.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:14:59 smithi052 ceph-mon[32340]: pgmap v524: 97 pgs: 97 active+clean; 36 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 797 KiB/s wr, 98 op/s 2024-06-02T23:14:59.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:14:59 smithi123 ceph-mon[35977]: pgmap v524: 97 pgs: 97 active+clean; 36 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 797 KiB/s wr, 98 op/s 2024-06-02T23:14:59.525 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:14:59.525 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:14:59.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 8 sec latency 8012.459 ms 2024-06-02T23:14:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:14:59 smithi174 ceph-mon[35994]: pgmap v524: 97 pgs: 97 active+clean; 36 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 797 KiB/s wr, 98 op/s 2024-06-02T23:14:59.774 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:00.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 9 sec latency 9012.545 ms 2024-06-02T23:15:01.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:01 smithi052 ceph-mon[32340]: pgmap v525: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 731 KiB/s wr, 97 op/s 2024-06-02T23:15:01.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:01 smithi123 ceph-mon[35977]: pgmap v525: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 731 KiB/s wr, 97 op/s 2024-06-02T23:15:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 -- 172.21.15.52:0/2129764726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17381066b0 msgr2=0x7f173810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 --2- 172.21.15.52:0/2129764726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17381066b0 0x7f173810e280 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f173000b0d0 tx=0x7f173000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 -- 172.21.15.52:0/2129764726 shutdown_connections 2024-06-02T23:15:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 --2- 172.21.15.52:0/2129764726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17381066b0 0x7f173810e280 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 --2- 172.21.15.52:0/2129764726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1738101620 0x7f1738101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 --2- 172.21.15.52:0/2129764726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1738101fd0 0x7f1738105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.599+0000 7f173dc31700 1 -- 172.21.15.52:0/2129764726 >> 172.21.15.52:0/2129764726 conn(0x7f1738078340 msgr2=0x7f1738078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:01.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.600+0000 7f173dc31700 1 -- 172.21.15.52:0/2129764726 shutdown_connections 2024-06-02T23:15:01.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.600+0000 7f173dc31700 1 -- 172.21.15.52:0/2129764726 wait complete. 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 Processor -- start 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 -- start start 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 0x7f1738199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1738101fd0 0x7f1738199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17381066b0 0x7f17381a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1738111550 con 0x7f1738101620 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17381113d0 con 0x7f1738101fd0 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f173dc31700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17381116d0 con 0x7f17381066b0 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f17377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 0x7f1738199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f17377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 0x7f1738199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48094/0 (socket says 172.21.15.52:48094) 2024-06-02T23:15:01.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.601+0000 7f17377fe700 1 -- 172.21.15.52:0/3698032765 learned_addr learned my addr 172.21.15.52:0/3698032765 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f1736ffd700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1738101fd0 0x7f1738199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f17377fe700 1 -- 172.21.15.52:0/3698032765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17381066b0 msgr2=0x7f17381a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f1737fff700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17381066b0 0x7f17381a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f17377fe700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17381066b0 0x7f17381a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f17377fe700 1 -- 172.21.15.52:0/3698032765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1738101fd0 msgr2=0x7f1738199dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f17377fe700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1738101fd0 0x7f1738199dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f17377fe700 1 -- 172.21.15.52:0/3698032765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17381a4e50 con 0x7f1738101620 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f1737fff700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17381066b0 0x7f17381a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f17377fe700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 0x7f1738199890 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f172800c9b0 tx=0x7f172800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728007b00 con 0x7f1738101620 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17381a5020 con 0x7f1738101620 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1728007ca0 con 0x7f1738101620 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1728018660 con 0x7f1738101620 2024-06-02T23:15:01.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.602+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17381a5560 con 0x7f1738101620 2024-06-02T23:15:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.603+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1738069180 con 0x7f1738101620 2024-06-02T23:15:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.605+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f172800f450 con 0x7f1738101620 2024-06-02T23:15:01.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.605+0000 7f1734ff9700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17200a2c20 0x7f17200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:01.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.606+0000 7f1736ffd700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17200a2c20 0x7f17200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:01.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.606+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17280cf290 con 0x7f1738101620 2024-06-02T23:15:01.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.606+0000 7f1736ffd700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17200a2c20 0x7f17200a50e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1738107aa0 tx=0x7f172c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:01.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.609+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1728014030 con 0x7f1738101620 2024-06-02T23:15:01.622 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 10 sec latency 10012.592 ms 2024-06-02T23:15:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:01 smithi174 ceph-mon[35994]: pgmap v525: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 731 KiB/s wr, 97 op/s 2024-06-02T23:15:01.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.862+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f173802dca0 con 0x7f1738101620 2024-06-02T23:15:01.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.863+0000 7f1734ff9700 1 -- 172.21.15.52:0/3698032765 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7f172809c2b0 con 0x7f1738101620 2024-06-02T23:15:01.863 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:15:01.863 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:15:01.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17200a2c20 msgr2=0x7f17200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:01.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17200a2c20 0x7f17200a50e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f1738107aa0 tx=0x7f172c00a380 comp rx=0 tx=0).stop 2024-06-02T23:15:01.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 msgr2=0x7f1738199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:01.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 0x7f1738199890 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f172800c9b0 tx=0x7f172800ccc0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 shutdown_connections 2024-06-02T23:15:01.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17200a2c20 0x7f17200a50e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17381066b0 0x7f17381a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1738101620 0x7f1738199890 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 --2- 172.21.15.52:0/3698032765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1738101fd0 0x7f1738199dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 >> 172.21.15.52:0/3698032765 conn(0x7f1738078340 msgr2=0x7f17380ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 shutdown_connections 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:01.865+0000 7f173dc31700 1 -- 172.21.15.52:0/3698032765 wait complete. 2024-06-02T23:15:01.866 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:15:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:02 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3698032765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:15:02.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 11 sec latency 11012.655 ms 2024-06-02T23:15:02.630 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:02 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3698032765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:15:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:02 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3698032765' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:15:02.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:15:02.916 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:03.350 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:03 smithi052 ceph-mon[32340]: pgmap v526: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 421 KiB/s wr, 82 op/s 2024-06-02T23:15:03.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:03 smithi123 ceph-mon[35977]: pgmap v526: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 421 KiB/s wr, 82 op/s 2024-06-02T23:15:03.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 12 sec latency 12012.697 ms 2024-06-02T23:15:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:03 smithi174 ceph-mon[35994]: pgmap v526: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 421 KiB/s wr, 82 op/s 2024-06-02T23:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.055+0000 7f70c7d77640 1 -- 172.21.15.52:0/1746218449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0100900 msgr2=0x7f70c0100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 --2- 172.21.15.52:0/1746218449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0100900 0x7f70c0100d20 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009900 tx=0x7f70bc02a410 comp rx=0 tx=0).stop 2024-06-02T23:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 -- 172.21.15.52:0/1746218449 shutdown_connections 2024-06-02T23:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 --2- 172.21.15.52:0/1746218449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c0101af0 0x7f70c0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 --2- 172.21.15.52:0/1746218449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c01024b0 0x7f70c0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 --2- 172.21.15.52:0/1746218449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0100900 0x7f70c0100d20 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 -- 172.21.15.52:0/1746218449 >> 172.21.15.52:0/1746218449 conn(0x7f70c00fbe70 msgr2=0x7f70c00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 -- 172.21.15.52:0/1746218449 shutdown_connections 2024-06-02T23:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 -- 172.21.15.52:0/1746218449 wait complete. 2024-06-02T23:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 Processor -- start 2024-06-02T23:15:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.056+0000 7f70c7d77640 1 -- start start 2024-06-02T23:15:04.058 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c0100900 0x7f70c0197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 0x7f70c0198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c01024b0 0x7f70c019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010c420 con 0x7f70c0101af0 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010c2a0 con 0x7f70c0100900 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c010c5a0 con 0x7f70c01024b0 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 0x7f70c0198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 0x7f70c0198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47920/0 (socket says 172.21.15.52:47920) 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 -- 172.21.15.52:0/3152286213 learned_addr learned my addr 172.21.15.52:0/3152286213 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5b13640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c0100900 0x7f70c0197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c6314640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c01024b0 0x7f70c019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 -- 172.21.15.52:0/3152286213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c01024b0 msgr2=0x7f70c019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.059 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c01024b0 0x7f70c019f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 -- 172.21.15.52:0/3152286213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c0100900 msgr2=0x7f70c0197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c0100900 0x7f70c0197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 -- 172.21.15.52:0/3152286213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c019f800 con 0x7f70c0101af0 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c5312640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 0x7f70c0198120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f70b000d770 tx=0x7f70b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70b6ffd640 1 -- 172.21.15.52:0/3152286213 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0017070 con 0x7f70c0101af0 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70b6ffd640 1 -- 172.21.15.52:0/3152286213 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b00054d0 con 0x7f70c0101af0 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c019faf0 con 0x7f70c0101af0 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.057+0000 7f70b6ffd640 1 -- 172.21.15.52:0/3152286213 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0004cf0 con 0x7f70c0101af0 2024-06-02T23:15:04.060 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.058+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c01a0040 con 0x7f70c0101af0 2024-06-02T23:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.060+0000 7f70b6ffd640 1 -- 172.21.15.52:0/3152286213 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f70b0004e90 con 0x7f70c0101af0 2024-06-02T23:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.060+0000 7f70cab67740 1 Processor -- start 2024-06-02T23:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 -- start start 2024-06-02T23:15:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53010 0x5604aeb53430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aeb58500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb58b70 0x5604aeb5b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5604aeab64f0 con 0x5604aeb53a00 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604aea54870 con 0x5604aeb53010 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70cab67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604aeab24b0 con 0x5604aeb58b70 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aeb58500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aeb58500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47926/0 (socket says 172.21.15.52:47926) 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 -- 172.21.15.52:0/316495904 learned_addr learned my addr 172.21.15.52:0/316495904 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5312640 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb58b70 0x5604aeb5b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c6314640 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53010 0x5604aeb53430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 -- 172.21.15.52:0/316495904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb58b70 msgr2=0x5604aeb5b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb58b70 0x5604aeb5b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 -- 172.21.15.52:0/316495904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53010 msgr2=0x5604aeb53430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53010 0x5604aeb53430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.061+0000 7f70c5b13640 1 -- 172.21.15.52:0/316495904 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604aea709b0 con 0x5604aeb53a00 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70c5b13640 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aeb58500 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009900 tx=0x7f70bc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b087da0a8bf1784 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70b6ffd640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f70ac0a2cb0 0x7f70ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70abfff640 1 -- 172.21.15.52:0/316495904 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc035030 con 0x5604aeb53a00 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70abfff640 1 -- 172.21.15.52:0/316495904 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70bc040070 con 0x5604aeb53a00 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70c6314640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f70ac0a2cb0 0x7f70ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70abfff640 1 -- 172.21.15.52:0/316495904 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc003af0 con 0x5604aeb53a00 2024-06-02T23:15:04.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.062+0000 7f70b6ffd640 1 -- 172.21.15.52:0/3152286213 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f70b00cea00 con 0x7f70c0101af0 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 -- 172.21.15.52:0/316495904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 msgr2=0x5604aeb58500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70c6314640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f70ac0a2cb0 0x7f70ac0a5170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5604aeb54260 tx=0x7f70b8009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aeb58500 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f70bc009900 tx=0x7f70bc005c00 comp rx=0 tx=0).stop 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 -- 172.21.15.52:0/316495904 shutdown_connections 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb58b70 0x5604aeb5b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aeb58500 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 --2- 172.21.15.52:0/316495904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53010 0x5604aeb53430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 -- 172.21.15.52:0/316495904 >> 172.21.15.52:0/316495904 conn(0x5604aeb4e680 msgr2=0x5604aeb4ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 -- 172.21.15.52:0/316495904 shutdown_connections 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 -- 172.21.15.52:0/316495904 wait complete. 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 -- start start 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.063+0000 7f70cab67740 1 Processor -- start 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 0x5604aeb5d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53a00 0x5604aeb5d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aeb647b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5604aeb5bac0 con 0x5604aeb53010 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604aea3fae0 con 0x5604aeb58b70 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5604aeb5bc40 con 0x5604aeb53a00 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70cab67740 10 client.0 _ll_register_callbacks cb 0x5604aea789b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 0x5604aeb5d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 0x5604aeb5d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47930/0 (socket says 172.21.15.52:47930) 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 -- 172.21.15.52:0/1583343345 learned_addr learned my addr 172.21.15.52:0/1583343345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5312640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53a00 0x5604aeb5d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c6314640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aeb647b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 -- 172.21.15.52:0/1583343345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53a00 msgr2=0x5604aeb5d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53a00 0x5604aeb5d770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 -- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 msgr2=0x5604aeb647b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aeb647b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c5b13640 1 -- 172.21.15.52:0/1583343345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604ae7ca210 con 0x5604aeb53010 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.064+0000 7f70c6314640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aeb647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70c5b13640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 0x5604aeb5d230 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f70bc02a930 tx=0x7f70bc0317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=735fb0ced124c76d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:04.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70a97fa640 1 -- 172.21.15.52:0/1583343345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc052060 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70a97fa640 1 -- 172.21.15.52:0/1583343345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f70bc03c850 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70bc040070 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70a97fa640 1 -- 172.21.15.52:0/1583343345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70bc035030 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70cab67740 10 client.16506 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604ae896620 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70a97fa640 1 -- 172.21.15.52:0/1583343345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70bc040070 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.065+0000 7f70a97fa640 1 -- 172.21.15.52:0/1583343345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f70bc04c030 con 0x5604aeb53010 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70cab67740 10 client.16506 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70cab67740 10 client.16506 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:04.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70cab67740 20 client.16506 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70cab67740 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aeb68ca0 0x5604aeb6b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70cab67740 4 client.16506 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604aeac9f10 con 0x5604aeb68ca0 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70c5312640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aeb68ca0 0x5604aeb6b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70c5312640 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aeb68ca0 0x5604aeb6b090 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25903607a780f1d server_cookie=e2250021cdc6e849 in_seq=0 out_seq=0 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.066+0000 7f70a97fa640 10 client.16506 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70a97fa640 1 -- 172.21.15.52:0/1583343345 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5604aeac9f10 con 0x5604aeb68ca0 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70a97fa640 10 client.16506 handle_command_reply: tid=0 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 client.16506 shutdown 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 msgr2=0x5604aeb5d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 0x5604aeb5d230 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f70bc02a930 tx=0x7f70bc0317b0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 shutdown_connections 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aeb68ca0 0x5604aeb6b090 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53a00 0x5604aeb5d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aeb647b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70a97fa640 0 client.16506 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 --2- 172.21.15.52:0/1583343345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53010 0x5604aeb5d230 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 >> 172.21.15.52:0/1583343345 conn(0x5604aeb4e680 msgr2=0x5604aeb4d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 shutdown_connections 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.067+0000 7f70cab67740 1 -- 172.21.15.52:0/1583343345 wait complete. 2024-06-02T23:15:04.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.068+0000 7f70cab67740 20 client.16506 trim_cache size 0 max 16384 2024-06-02T23:15:04.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 Processor -- start 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 -- start start 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aeb5c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53a00 0x5604aeb5cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 0x5604aeb5d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01a8cc0 con 0x5604aeb58b70 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01aa890 con 0x5604aeb53a00 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.083+0000 7f70cab67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01a9390 con 0x5604aeb53010 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 0x5604aeb5d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 0x5604aeb5d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47934/0 (socket says 172.21.15.52:47934) 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 -- 172.21.15.52:0/1119305986 learned_addr learned my addr 172.21.15.52:0/1119305986 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c5312640 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53a00 0x5604aeb5cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c5b13640 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aeb5c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 -- 172.21.15.52:0/1119305986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 msgr2=0x5604aeb5c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aeb5c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 -- 172.21.15.52:0/1119305986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53a00 msgr2=0x5604aeb5cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53a00 0x5604aeb5cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 -- 172.21.15.52:0/1119305986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604aeb537a0 con 0x5604aeb58b70 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c5b13640 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aeb5c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70c6314640 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 0x5604aeb5d2d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f70b8004970 tx=0x7f70b8015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829a7262c92fc97b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70ab7fe640 1 -- 172.21.15.52:0/1119305986 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b801b450 con 0x5604aeb58b70 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70ab7fe640 1 -- 172.21.15.52:0/1119305986 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b801baa0 con 0x5604aeb58b70 2024-06-02T23:15:04.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.084+0000 7f70ab7fe640 1 -- 172.21.15.52:0/1119305986 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b801cad0 con 0x5604aeb58b70 2024-06-02T23:15:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 -- 172.21.15.52:0/1119305986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 msgr2=0x5604aeb5d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 0x5604aeb5d2d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f70b8004970 tx=0x7f70b8015000 comp rx=0 tx=0).stop 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 -- 172.21.15.52:0/1119305986 shutdown_connections 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb58b70 0x5604aeb5d2d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb53a00 0x5604aeb5cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 --2- 172.21.15.52:0/1119305986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aeb5c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 -- 172.21.15.52:0/1119305986 >> 172.21.15.52:0/1119305986 conn(0x5604aeb4e680 msgr2=0x5604aeb515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 -- 172.21.15.52:0/1119305986 shutdown_connections 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 -- 172.21.15.52:0/1119305986 wait complete. 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.085+0000 7f70cab67740 1 -- start start 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 Processor -- start 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aebceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aebcf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aebd6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01aa130 con 0x5604aeb53a00 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01a9a60 con 0x5604aeb58b70 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70c01a6ca0 con 0x5604aeb53010 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70cab67740 10 client.0 _ll_register_callbacks cb 0x7f70b80098e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aebcf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aebcf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47940/0 (socket says 172.21.15.52:47940) 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 -- 172.21.15.52:0/549067082 learned_addr learned my addr 172.21.15.52:0/549067082 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5b13640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aebceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c6314640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aebd6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 -- 172.21.15.52:0/549067082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 msgr2=0x5604aebceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aebceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 -- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 msgr2=0x5604aebd6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.086+0000 7f70c5312640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aebd6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70c5312640 1 -- 172.21.15.52:0/549067082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604aebd6a10 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70c6314640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aebd6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70c5b13640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aebceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70c5312640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aebcf3e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f70b00976f0 tx=0x7f70b00d2430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa552ef3d3a8e8cb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70a97fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70a97fa640 1 -- 172.21.15.52:0/549067082 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70b0020040 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70a97fa640 1 -- 172.21.15.52:0/549067082 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f70b0099590 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70b801baa0 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70a97fa640 1 -- 172.21.15.52:0/549067082 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f70b009abf0 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70cab67740 10 client.16512 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5604aebd7d90 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70a97fa640 1 -- 172.21.15.52:0/549067082 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70b801baa0 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70a97fa640 1 -- 172.21.15.52:0/549067082 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f70b0097900 con 0x5604aeb53a00 2024-06-02T23:15:04.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70cab67740 10 client.16512 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70cab67740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.087+0000 7f70cab67740 20 client.16512 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.088+0000 7f70cab67740 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aebd87d0 0x5604aebdabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.088+0000 7f70cab67740 4 client.16512 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.088+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604aeac3c20 con 0x5604aebd87d0 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.088+0000 7f70c5b13640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aebd87d0 0x5604aebdabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.088+0000 7f70c5b13640 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aebd87d0 0x5604aebdabf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70e30a5acfef4b58 server_cookie=e5d8733d338d4488 in_seq=0 out_seq=0 2024-06-02T23:15:04.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.088+0000 7f70a97fa640 10 client.16512 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.108+0000 7f70a97fa640 1 -- 172.21.15.52:0/549067082 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5604aeac3c20 con 0x5604aebd87d0 2024-06-02T23:15:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.108+0000 7f70a97fa640 10 client.16512 handle_command_reply: tid=0 2024-06-02T23:15:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.108+0000 7f70cab67740 1 client.16512 shutdown 2024-06-02T23:15:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 msgr2=0x5604aebcf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aebcf3e0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f70b00976f0 tx=0x7f70b00d2430 comp rx=0 tx=0).stop 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 shutdown_connections 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5604aebd87d0 0x5604aebdabf0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604aeb58b70 0x5604aebd6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5604aeb53a00 0x5604aebcf3e0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 --2- 172.21.15.52:0/549067082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5604aeb53010 0x5604aebceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 >> 172.21.15.52:0/549067082 conn(0x5604aeb4e680 msgr2=0x5604aeb4cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70a97fa640 0 client.16512 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 shutdown_connections 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 1 -- 172.21.15.52:0/549067082 wait complete. 2024-06-02T23:15:04.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.109+0000 7f70cab67740 20 client.16512 trim_cache size 0 max 16384 2024-06-02T23:15:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:15:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:15:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:15:04.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f70ac0a2cb0 msgr2=0x7f70ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f70ac0a2cb0 0x7f70ac0a5170 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x5604aeb54260 tx=0x7f70b8009500 comp rx=0 tx=0).stop 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 msgr2=0x7f70c0198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 0x7f70c0198120 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f70b000d770 tx=0x7f70b000dc30 comp rx=0 tx=0).stop 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 shutdown_connections 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f70ac0a2cb0 0x7f70ac0a5170 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70c01024b0 0x7f70c019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70c0101af0 0x7f70c0198120 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 --2- 172.21.15.52:0/3152286213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f70c0100900 0x7f70c0197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 >> 172.21.15.52:0/3152286213 conn(0x7f70c00fbe70 msgr2=0x7f70c00fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 shutdown_connections 2024-06-02T23:15:04.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:04.110+0000 7f70c7d77640 1 -- 172.21.15.52:0/3152286213 wait complete. 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/1457195471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee0108570 msgr2=0x7f2ee0108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/1457195471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee0108570 0x7f2ee0108950 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0009990 tx=0x7f2ed0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/1457195471 shutdown_connections 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/1457195471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee006b5c0 0x7f2ee010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/1457195471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee0108570 0x7f2ee0108950 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/1457195471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006ac00 0x7f2ee006b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/1457195471 >> 172.21.15.52:0/1457195471 conn(0x7f2ee00773e0 msgr2=0x7f2ee00777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/1457195471 shutdown_connections 2024-06-02T23:15:04.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.573+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/1457195471 wait complete. 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.574+0000 7f2ee4e2d700 1 Processor -- start 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.574+0000 7f2ee4e2d700 1 -- start start 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ee4e2d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 0x7f2ee01014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ee4e2d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006b5c0 0x7f2ee0101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ee4e2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee0108570 0x7f2ee0101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ee4e2d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee0112670 con 0x7f2ee006ac00 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ee4e2d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01124f0 con 0x7f2ee0108570 2024-06-02T23:15:04.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ee4e2d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01127f0 con 0x7f2ee006b5c0 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 0x7f2ee01014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 0x7f2ee01014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47960/0 (socket says 172.21.15.52:47960) 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 -- 172.21.15.52:0/3838892642 learned_addr learned my addr 172.21.15.52:0/3838892642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006b5c0 0x7f2ee0101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee0108570 0x7f2ee0101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 -- 172.21.15.52:0/3838892642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006b5c0 msgr2=0x7f2ee0101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006b5c0 0x7f2ee0101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 -- 172.21.15.52:0/3838892642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee0108570 msgr2=0x7f2ee0101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee0108570 0x7f2ee0101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 -- 172.21.15.52:0/3838892642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee01a7050 con 0x7f2ee006ac00 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006b5c0 0x7f2ee0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee0108570 0x7f2ee0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:04.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.575+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 0x7f2ee01014f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0000c00 tx=0x7f2ed0003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.576+0000 7f2ed7fff700 1 -- 172.21.15.52:0/3838892642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0027070 con 0x7f2ee006ac00 2024-06-02T23:15:04.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.576+0000 7f2ed7fff700 1 -- 172.21.15.52:0/3838892642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed000bdc0 con 0x7f2ee006ac00 2024-06-02T23:15:04.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.576+0000 7f2ed7fff700 1 -- 172.21.15.52:0/3838892642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed000f460 con 0x7f2ee006ac00 2024-06-02T23:15:04.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.576+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee01a72e0 con 0x7f2ee006ac00 2024-06-02T23:15:04.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.576+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee01a76e0 con 0x7f2ee006ac00 2024-06-02T23:15:04.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.578+0000 7f2ed7fff700 1 -- 172.21.15.52:0/3838892642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ed002a070 con 0x7f2ee006ac00 2024-06-02T23:15:04.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.579+0000 7f2ed7fff700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ecc0a2c20 0x7f2ecc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.579+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ecc0a2c20 0x7f2ecc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.579+0000 7f2ed7fff700 1 -- 172.21.15.52:0/3838892642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ed00cff30 con 0x7f2ee006ac00 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.579+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ecc0a2c20 0x7f2ecc0a50e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005fd0 tx=0x7f2ec8005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.579+0000 7f2eef6b0b80 1 Processor -- start 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 -- start start 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c95ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c9589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c959340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bf9c8aa810 con 0x55bf9c958f20 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bf9c852720 con 0x55bf9c95ef60 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf9c8a67d0 con 0x55bf9c95e640 2024-06-02T23:15:04.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c959340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c959340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47970/0 (socket says 172.21.15.52:47970) 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 -- 172.21.15.52:0/3918138197 learned_addr learned my addr 172.21.15.52:0/3918138197 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c95ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c9589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 -- 172.21.15.52:0/3918138197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 msgr2=0x55bf9c95ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c95ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 -- 172.21.15.52:0/3918138197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 msgr2=0x55bf9c9589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c9589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2eded9d700 1 -- 172.21.15.52:0/3918138197 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9c848f50 con 0x55bf9c958f20 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.580+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c9589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.581+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c959340 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800d300 tx=0x7f2ed8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d94f74d7f9c5bdd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.581+0000 7f2ed4d99700 1 -- 172.21.15.52:0/3918138197 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8016020 con 0x55bf9c958f20 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.581+0000 7f2ed4d99700 1 -- 172.21.15.52:0/3918138197 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed800de30 con 0x55bf9c958f20 2024-06-02T23:15:04.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.581+0000 7f2ed4d99700 1 -- 172.21.15.52:0/3918138197 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8011910 con 0x55bf9c958f20 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3918138197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 msgr2=0x55bf9c959340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c959340 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800d300 tx=0x7f2ed8007f90 comp rx=0 tx=0).stop 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3918138197 shutdown_connections 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c959340 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c95ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3918138197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c9589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3918138197 >> 172.21.15.52:0/3918138197 conn(0x55bf9c883cc0 msgr2=0x55bf9c953850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3918138197 shutdown_connections 2024-06-02T23:15:04.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3918138197 wait complete. 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- start start 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 Processor -- start 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c958f20 0x55bf9c964c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 0x55bf9c961f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c962450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bf9c9659d0 con 0x55bf9c95e640 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bf9c84abf0 con 0x55bf9c95ef60 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bf9c965b50 con 0x55bf9c958f20 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.582+0000 7f2eef6b0b80 10 client.0 _ll_register_callbacks cb 0x55bf9c844920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 0x55bf9c961f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 0x55bf9c961f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47972/0 (socket says 172.21.15.52:47972) 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 -- 172.21.15.52:0/3182647231 learned_addr learned my addr 172.21.15.52:0/3182647231 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c958f20 0x55bf9c964c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c962450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 -- 172.21.15.52:0/3182647231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c958f20 msgr2=0x55bf9c964c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c958f20 0x55bf9c964c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 -- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 msgr2=0x55bf9c962450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c962450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 -- 172.21.15.52:0/3182647231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9c7f98d0 con 0x55bf9c95e640 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 0x55bf9c961f10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80117a0 tx=0x7f2ec80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61fa6542f433f977 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3182647231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8023070 con 0x55bf9c95e640 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3182647231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ec801f830 con 0x55bf9c95e640 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed800de30 con 0x55bf9c95e640 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3182647231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ec802f050 con 0x55bf9c95e640 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2eef6b0b80 10 client.16524 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.583+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf9c970300 con 0x55bf9c95e640 2024-06-02T23:15:04.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3182647231 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed800de30 con 0x55bf9c95e640 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3182647231 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2ec800f040 con 0x55bf9c95e640 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2eef6b0b80 10 client.16524 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2eef6b0b80 10 client.16524 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2eef6b0b80 20 client.16524 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c974d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2eef6b0b80 4 client.16524 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bf9c8b7d10 con 0x55bf9c9728e0 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.584+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c974d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c974d00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a202a78a3082bf43 server_cookie=4808f6e268ee9576 in_seq=0 out_seq=0 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2ec5ffb700 10 client.16524 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3182647231 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bf9c8b7d10 con 0x55bf9c9728e0 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2ec5ffb700 10 client.16524 handle_command_reply: tid=0 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2eef6b0b80 1 client.16524 shutdown 2024-06-02T23:15:04.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 msgr2=0x55bf9c961f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 0x55bf9c961f10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80117a0 tx=0x7f2ec80047c0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 shutdown_connections 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c974d00 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.585+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c962450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c958f20 0x55bf9c964c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2ec5ffb700 0 client.16524 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3182647231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c95e640 0x55bf9c961f10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 >> 172.21.15.52:0/3182647231 conn(0x55bf9c883cc0 msgr2=0x55bf9c9579d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 shutdown_connections 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3182647231 wait complete. 2024-06-02T23:15:04.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.586+0000 7f2eef6b0b80 20 client.16524 trim_cache size 0 max 16384 2024-06-02T23:15:04.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 Processor -- start 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 -- start start 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c9589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95e640 0x55bf9c9539f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95ef60 0x55bf9c94d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01aeb70 con 0x55bf9c958f20 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01b0740 con 0x55bf9c95e640 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01af240 con 0x55bf9c95ef60 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c9589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c9589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47986/0 (socket says 172.21.15.52:47986) 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2ede59c700 1 -- 172.21.15.52:0/1888182321 learned_addr learned my addr 172.21.15.52:0/1888182321 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95e640 0x55bf9c9539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.601+0000 7f2eded9d700 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95ef60 0x55bf9c94d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ede59c700 1 -- 172.21.15.52:0/1888182321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95ef60 msgr2=0x55bf9c94d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ede59c700 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95ef60 0x55bf9c94d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ede59c700 1 -- 172.21.15.52:0/1888182321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95e640 msgr2=0x55bf9c9539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ede59c700 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95e640 0x55bf9c9539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ede59c700 1 -- 172.21.15.52:0/1888182321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf9c9604b0 con 0x55bf9c958f20 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95e640 0x55bf9c9539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ede59c700 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c9589e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0098cb0 tx=0x7f2ed00cd930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d8f149f62ea46a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ec7fff700 1 -- 172.21.15.52:0/1888182321 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed009c850 con 0x55bf9c958f20 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ec7fff700 1 -- 172.21.15.52:0/1888182321 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed00ba070 con 0x55bf9c958f20 2024-06-02T23:15:04.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.602+0000 7f2ec7fff700 1 -- 172.21.15.52:0/1888182321 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed009ab50 con 0x55bf9c958f20 2024-06-02T23:15:04.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/1888182321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 msgr2=0x55bf9c9589e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c9589e0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0098cb0 tx=0x7f2ed00cd930 comp rx=0 tx=0).stop 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/1888182321 shutdown_connections 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95ef60 0x55bf9c94d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c9589e0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/1888182321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95e640 0x55bf9c9539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/1888182321 >> 172.21.15.52:0/1888182321 conn(0x55bf9c883cc0 msgr2=0x55bf9c94ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/1888182321 shutdown_connections 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/1888182321 wait complete. 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.603+0000 7f2eef6b0b80 1 -- start start 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 Processor -- start 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c97a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c97a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c97adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01affe0 con 0x55bf9c958f20 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01af910 con 0x55bf9c95ef60 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee01acb30 con 0x55bf9c95e640 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eef6b0b80 10 client.0 _ll_register_callbacks cb 0x55bf9c844920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c97a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c97a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48002/0 (socket says 172.21.15.52:48002) 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 -- 172.21.15.52:0/3769595500 learned_addr learned my addr 172.21.15.52:0/3769595500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c97adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c97a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 -- 172.21.15.52:0/3769595500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 msgr2=0x55bf9c97a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c97a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 -- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 msgr2=0x55bf9c97adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c97adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2ede59c700 1 -- 172.21.15.52:0/3769595500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf9c9dc720 con 0x55bf9c958f20 2024-06-02T23:15:04.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.604+0000 7f2eded9d700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c97adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2ede59c700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c97a370 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00cddd0 tx=0x7f2ed0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43158778e88d1dbd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2ec5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3769595500 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed009c850 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3769595500 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ed009ab50 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed00ba070 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3769595500 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f2ed009cab0 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2eef6b0b80 10 client.16530 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55bf9c9dda00 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.605+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3769595500 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed00ba070 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3769595500 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f2ed00b0ce0 con 0x55bf9c958f20 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2eef6b0b80 10 client.16530 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2eef6b0b80 10 client.16530 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:15:04.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2eef6b0b80 20 client.16530 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c9e00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2eef6b0b80 4 client.16530 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bf9c8b2290 con 0x55bf9c9728e0 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c9e00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2ec5ffb700 10 client.16530 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.606+0000 7f2ed5d9b700 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c9e00c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f0addffd0be4ff25 server_cookie=794064f00045c1fe in_seq=0 out_seq=0 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2ec5ffb700 1 -- 172.21.15.52:0/3769595500 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bf9c8b2290 con 0x55bf9c9728e0 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2ec5ffb700 10 client.16530 handle_command_reply: tid=0 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 client.16530 shutdown 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 msgr2=0x55bf9c97a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c97a370 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2ed00cddd0 tx=0x7f2ed0009310 comp rx=0 tx=0).stop 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 shutdown_connections 2024-06-02T23:15:04.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55bf9c9728e0 0x55bf9c9e00c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bf9c95ef60 0x55bf9c97adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bf9c958f20 0x55bf9c97a370 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 --2- 172.21.15.52:0/3769595500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bf9c95e640 0x55bf9c97a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 >> 172.21.15.52:0/3769595500 conn(0x55bf9c883cc0 msgr2=0x55bf9c960800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2ec5ffb700 0 client.16530 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 shutdown_connections 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 1 -- 172.21.15.52:0/3769595500 wait complete. 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2eef6b0b80 20 client.16530 trim_cache size 0 max 16384 2024-06-02T23:15:04.612 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ecc0a2c20 msgr2=0x7f2ecc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ecc0a2c20 0x7f2ecc0a50e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2ec8005fd0 tx=0x7f2ec8005ee0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.607+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 msgr2=0x7f2ee01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 0x7f2ee01014f0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f2ed0000c00 tx=0x7f2ed0003b30 comp rx=0 tx=0).stop 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 shutdown_connections 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ecc0a2c20 0x7f2ecc0a50e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee0108570 0x7f2ee0101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee006ac00 0x7f2ee01014f0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 --2- 172.21.15.52:0/3838892642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee006b5c0 0x7f2ee0101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 >> 172.21.15.52:0/3838892642 conn(0x7f2ee00773e0 msgr2=0x7f2ee0100040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 shutdown_connections 2024-06-02T23:15:04.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:04.608+0000 7f2ee4e2d700 1 -- 172.21.15.52:0/3838892642 wait complete. 2024-06-02T23:15:04.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 13 sec latency 13012.741 ms 2024-06-02T23:15:05.388 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:05 smithi052 ceph-mon[32340]: pgmap v527: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 190 KiB/s wr, 42 op/s 2024-06-02T23:15:05.421 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:15:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:05 smithi123 ceph-mon[35977]: pgmap v527: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 190 KiB/s wr, 42 op/s 2024-06-02T23:15:05.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 14 sec latency 14012.773 ms 2024-06-02T23:15:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:05 smithi174 ceph-mon[35994]: pgmap v527: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 190 KiB/s wr, 42 op/s 2024-06-02T23:15:05.677 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:06.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 15 sec latency 15012.869 ms 2024-06-02T23:15:07.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:07 smithi052 ceph-mon[32340]: pgmap v528: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 182 KiB/s wr, 42 op/s 2024-06-02T23:15:07.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:07 smithi123 ceph-mon[35977]: pgmap v528: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 182 KiB/s wr, 42 op/s 2024-06-02T23:15:07.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 -- 172.21.15.52:0/1160908887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 msgr2=0x7fc8dc107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/1160908887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc107110 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80099c0 tx=0x7fc8c8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 -- 172.21.15.52:0/1160908887 shutdown_connections 2024-06-02T23:15:07.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/1160908887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8dc107840 0x7fc8dc10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/1160908887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 0x7fc8dc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/1160908887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc107110 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 -- 172.21.15.52:0/1160908887 >> 172.21.15.52:0/1160908887 conn(0x7fc8dc0795f0 msgr2=0x7fc8dc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 -- 172.21.15.52:0/1160908887 shutdown_connections 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 -- 172.21.15.52:0/1160908887 wait complete. 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 Processor -- start 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.478+0000 7fc8e2b03700 1 -- start start 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 0x7fc8dc19e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc19ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8dc107840 0x7fc8dc1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1126d0 con 0x7fc8dc1030f0 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc112550 con 0x7fc8dc107840 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc112850 con 0x7fc8dc102740 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc19ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc19ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48010/0 (socket says 172.21.15.52:48010) 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 -- 172.21.15.52:0/2864520826 learned_addr learned my addr 172.21.15.52:0/2864520826 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8dc107840 0x7fc8dc1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e089f700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 0x7fc8dc19e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 -- 172.21.15.52:0/2864520826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 msgr2=0x7fc8dc19e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 0x7fc8dc19e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 -- 172.21.15.52:0/2864520826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8dc107840 msgr2=0x7fc8dc1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8dc107840 0x7fc8dc1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 -- 172.21.15.52:0/2864520826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8dc199110 con 0x7fc8dc1030f0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e089f700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 0x7fc8dc19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc19ed30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009990 tx=0x7fc8c80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8d9ffb700 1 -- 172.21.15.52:0/2864520826 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8027070 con 0x7fc8dc1030f0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8d9ffb700 1 -- 172.21.15.52:0/2864520826 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8c8004050 con 0x7fc8dc1030f0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8d9ffb700 1 -- 172.21.15.52:0/2864520826 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8022dc0 con 0x7fc8dc1030f0 2024-06-02T23:15:07.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.479+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8dc1993a0 con 0x7fc8dc1030f0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.480+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8dc1a56c0 con 0x7fc8dc1030f0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.482+0000 7fc8d9ffb700 1 -- 172.21.15.52:0/2864520826 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc8c802a070 con 0x7fc8dc1030f0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.482+0000 7fc8d9ffb700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc8cc0a2ca0 0x7fc8cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.483+0000 7fc8d9ffb700 1 -- 172.21.15.52:0/2864520826 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc8c80cff50 con 0x7fc8dc1030f0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.483+0000 7fc8e089f700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc8cc0a2ca0 0x7fc8cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.483+0000 7fc8e089f700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc8cc0a2ca0 0x7fc8cc0a5160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc199af0 tx=0x7fc8d0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.483+0000 7fc8ed386b80 1 Processor -- start 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.483+0000 7fc8ed386b80 1 -- start start 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x564741595260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564741487f60 con 0x5647415957a0 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5647414e6850 con 0x56474159af60 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56474144e460 con 0x56474159b880 2024-06-02T23:15:07.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48012/0 (socket says 172.21.15.52:48012) 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 -- 172.21.15.52:0/448459745 learned_addr learned my addr 172.21.15.52:0/448459745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e089f700 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 -- 172.21.15.52:0/448459745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 msgr2=0x564741595260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x564741595260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 -- 172.21.15.52:0/448459745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 msgr2=0x56474159b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 -- 172.21.15.52:0/448459745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647414358d0 con 0x5647415957a0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e089f700 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595bc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400a730 tx=0x7fc8d40078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed816a65e0e1082 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8c6ffd700 1 -- 172.21.15.52:0/448459745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4003a00 con 0x5647415957a0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8c6ffd700 1 -- 172.21.15.52:0/448459745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8d4003ba0 con 0x5647415957a0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.484+0000 7fc8c6ffd700 1 -- 172.21.15.52:0/448459745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4014600 con 0x5647415957a0 2024-06-02T23:15:07.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 -- 172.21.15.52:0/448459745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 msgr2=0x564741595bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595bc0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400a730 tx=0x7fc8d40078d0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 -- 172.21.15.52:0/448459745 shutdown_connections 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595bc0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/448459745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x564741595260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 -- 172.21.15.52:0/448459745 >> 172.21.15.52:0/448459745 conn(0x5647414c2080 msgr2=0x564741590220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 -- 172.21.15.52:0/448459745 shutdown_connections 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.485+0000 7fc8ed386b80 1 -- 172.21.15.52:0/448459745 wait complete. 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 -- start start 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 Processor -- start 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647415957a0 0x5647415a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 0x56474159e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x56474159ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564741486fc0 con 0x56474159af60 2024-06-02T23:15:07.573 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5647414e2810 con 0x56474159b880 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647415a2200 con 0x5647415957a0 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8ed386b80 10 client.0 _ll_register_callbacks cb 0x56474147c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 0x56474159e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8dbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 0x56474159e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48028/0 (socket says 172.21.15.52:48028) 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8dbfff700 1 -- 172.21.15.52:0/3390008046 learned_addr learned my addr 172.21.15.52:0/3390008046 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.486+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x56474159ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8e089f700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647415957a0 0x5647415a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8dbfff700 1 -- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647415957a0 msgr2=0x5647415a15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647415957a0 0x5647415a15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8dbfff700 1 -- 172.21.15.52:0/3390008046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 msgr2=0x56474159ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x56474159ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8dbfff700 1 -- 172.21.15.52:0/3390008046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647414ac750 con 0x56474159af60 2024-06-02T23:15:07.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 0x56474159e8f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000c00 tx=0x7fc8c8022750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258bba1edaa21220 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3390008046 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c809c1d0 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3390008046 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc8c80a4fc0 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d4003ba0 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3390008046 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8c809cf70 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8ed386b80 10 client.16542 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5647415ac8f0 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3390008046 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8d4003ba0 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.487+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3390008046 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fc8c809dc30 con 0x56474159af60 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8ed386b80 10 client.16542 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8ed386b80 10 client.16542 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8ed386b80 20 client.16542 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5647415aedc0 0x5647415b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8ed386b80 4 client.16542 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647414f3d50 con 0x5647415aedc0 2024-06-02T23:15:07.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8e089f700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5647415aedc0 0x5647415b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8e089f700 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5647415aedc0 0x5647415b11b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b991f8d8d68bc544 server_cookie=9f3f8f488eb434ff in_seq=0 out_seq=0 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.488+0000 7fc8bbfff700 10 client.16542 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3390008046 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647414f3d50 con 0x5647415aedc0 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8bbfff700 10 client.16542 handle_command_reply: tid=0 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 client.16542 shutdown 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 msgr2=0x56474159e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 0x56474159e8f0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8000c00 tx=0x7fc8c8022750 comp rx=0 tx=0).stop 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 shutdown_connections 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x56474159ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5647415aedc0 0x5647415b11b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647415957a0 0x5647415a15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3390008046 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56474159af60 0x56474159e8f0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8bbfff700 0 client.16542 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 >> 172.21.15.52:0/3390008046 conn(0x5647414c2080 msgr2=0x56474158f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 shutdown_connections 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.489+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3390008046 wait complete. 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.490+0000 7fc8ed386b80 20 client.16542 trim_cache size 0 max 16384 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 Processor -- start 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 -- start start 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x5647415a0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ac970 con 0x5647415957a0 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ae540 con 0x56474159af60 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ad040 con 0x56474159b880 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48038/0 (socket says 172.21.15.52:48038) 2024-06-02T23:15:07.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 -- 172.21.15.52:0/1595741876 learned_addr learned my addr 172.21.15.52:0/1595741876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x5647415a0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 -- 172.21.15.52:0/1595741876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 msgr2=0x5647415a0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x5647415a0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 -- 172.21.15.52:0/1595741876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 msgr2=0x56474159d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8e089f700 1 -- 172.21.15.52:0/1595741876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56474159f7f0 con 0x5647415957a0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.505+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8e089f700 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595260 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0007210 tx=0x7fc8d0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3fd11a3e756aed server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8c67fc700 1 -- 172.21.15.52:0/1595741876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d002e070 con 0x5647415957a0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8c67fc700 1 -- 172.21.15.52:0/1595741876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8d0029440 con 0x5647415957a0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8c67fc700 1 -- 172.21.15.52:0/1595741876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0020660 con 0x5647415957a0 2024-06-02T23:15:07.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 -- 172.21.15.52:0/1595741876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 msgr2=0x564741595260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595260 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0007210 tx=0x7fc8d0012000 comp rx=0 tx=0).stop 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 -- 172.21.15.52:0/1595741876 shutdown_connections 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159b880 0x5647415a0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x564741595260 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/1595741876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159af60 0x56474159d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 -- 172.21.15.52:0/1595741876 >> 172.21.15.52:0/1595741876 conn(0x5647414c2080 msgr2=0x5647415a0120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.506+0000 7fc8ed386b80 1 -- 172.21.15.52:0/1595741876 shutdown_connections 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 -- 172.21.15.52:0/1595741876 wait complete. 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 -- start start 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 Processor -- start 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x5647415b6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159af60 0x5647415b6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x5647415b7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1adde0 con 0x5647415957a0 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1ad710 con 0x56474159b880 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8dc1aa930 con 0x56474159af60 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8ed386b80 10 client.0 _ll_register_callbacks cb 0x56474147c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8e089f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x5647415b6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8e089f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x5647415b6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48042/0 (socket says 172.21.15.52:48042) 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.507+0000 7fc8e089f700 1 -- 172.21.15.52:0/3386710890 learned_addr learned my addr 172.21.15.52:0/3386710890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159af60 0x5647415b6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x5647415b7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e089f700 1 -- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159af60 msgr2=0x5647415b6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e089f700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159af60 0x5647415b6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e089f700 1 -- 172.21.15.52:0/3386710890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 msgr2=0x5647415b7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:07.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e089f700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x5647415b7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e089f700 1 -- 172.21.15.52:0/3386710890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564741618ad0 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e10a0700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x5647415b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8e089f700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x5647415b6700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00048a0 tx=0x7fc8d00298a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6923d10fb8fc37 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3386710890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d002e040 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3386710890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc8d0013a00 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d0029440 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3386710890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc8d0013c00 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8ed386b80 10 client.16548 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.508+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x564741619db0 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3386710890 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc8d0029440 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3386710890 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fc8d0031030 con 0x5647415957a0 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8ed386b80 10 client.16548 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8ed386b80 10 client.16548 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8ed386b80 20 client.16548 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:07.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56474161a490 0x56474161c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8ed386b80 4 client.16548 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647414ee2d0 con 0x56474161a490 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56474161a490 0x56474161c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8bbfff700 10 client.16548 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.509+0000 7fc8dbfff700 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56474161a490 0x56474161c8b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=289dcf0afe912724 server_cookie=2232931cc3e6c21d in_seq=0 out_seq=0 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8bbfff700 1 -- 172.21.15.52:0/3386710890 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647414ee2d0 con 0x56474161a490 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8bbfff700 10 client.16548 handle_command_reply: tid=0 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 client.16548 shutdown 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 msgr2=0x5647415b6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x5647415b6700 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00048a0 tx=0x7fc8d00298a0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 shutdown_connections 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56474159b880 0x5647415b7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647415957a0 0x5647415b6700 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x56474161a490 0x56474161c8b0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 --2- 172.21.15.52:0/3386710890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56474159af60 0x5647415b6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8bbfff700 0 client.16548 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 >> 172.21.15.52:0/3386710890 conn(0x5647414c2080 msgr2=0x5647415a09b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 shutdown_connections 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 1 -- 172.21.15.52:0/3386710890 wait complete. 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.510+0000 7fc8ed386b80 20 client.16548 trim_cache size 0 max 16384 2024-06-02T23:15:07.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc8cc0a2ca0 msgr2=0x7fc8cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc8cc0a2ca0 0x7fc8cc0a5160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc199af0 tx=0x7fc8d0008040 comp rx=0 tx=0).stop 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 msgr2=0x7fc8dc19ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc19ed30 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8009990 tx=0x7fc8c80047b0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 shutdown_connections 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc8cc0a2ca0 0x7fc8cc0a5160 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8dc107840 0x7fc8dc1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8dc102740 0x7fc8dc19e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 --2- 172.21.15.52:0/2864520826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc8dc1030f0 0x7fc8dc19ed30 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 >> 172.21.15.52:0/2864520826 conn(0x7fc8dc0795f0 msgr2=0x7fc8dc1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 shutdown_connections 2024-06-02T23:15:07.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:07.511+0000 7fc8e2b03700 1 -- 172.21.15.52:0/2864520826 wait complete. 2024-06-02T23:15:07.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 16 sec latency 16013.362 ms 2024-06-02T23:15:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:07 smithi174 ceph-mon[35994]: pgmap v528: 97 pgs: 97 active+clean; 36 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 182 KiB/s wr, 42 op/s 2024-06-02T23:15:08.420 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:15:08.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 17 sec latency 17013.406 ms 2024-06-02T23:15:08.674 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:09.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:09 smithi052 ceph-mon[32340]: pgmap v529: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 84 KiB/s wr, 16 op/s 2024-06-02T23:15:09.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:09 smithi123 ceph-mon[35977]: pgmap v529: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 84 KiB/s wr, 16 op/s 2024-06-02T23:15:09.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 18 sec latency 18013.451 ms 2024-06-02T23:15:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:09 smithi174 ceph-mon[35994]: pgmap v529: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 84 KiB/s wr, 16 op/s 2024-06-02T23:15:10.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.480+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/3649000306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101620 msgr2=0x7f7b04101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.480+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/3649000306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101620 0x7f7b04101a00 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f7b000099c0 tx=0x7f7b00009cd0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/3649000306 shutdown_connections 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/3649000306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b041066b0 0x7f7b0410e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/3649000306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101620 0x7f7b04101a00 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/3649000306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101fd0 0x7f7b04105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/3649000306 >> 172.21.15.52:0/3649000306 conn(0x7f7b04078340 msgr2=0x7f7b04078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/3649000306 shutdown_connections 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/3649000306 wait complete. 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 Processor -- start 2024-06-02T23:15:10.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.481+0000 7f7b0bd2b700 1 -- start start 2024-06-02T23:15:10.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b0bd2b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101620 0x7f7b04199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b0bd2b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 0x7f7b04199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b0bd2b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b041066b0 0x7f7b041a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b0bd2b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04111540 con 0x7f7b04101fd0 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b0bd2b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041113c0 con 0x7f7b04101620 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b0bd2b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041116c0 con 0x7f7b041066b0 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 0x7f7b04199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 0x7f7b04199ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48068/0 (socket says 172.21.15.52:48068) 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 -- 172.21.15.52:0/1370285422 learned_addr learned my addr 172.21.15.52:0/1370285422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101620 0x7f7b04199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 -- 172.21.15.52:0/1370285422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b041066b0 msgr2=0x7f7b041a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b041066b0 0x7f7b041a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 -- 172.21.15.52:0/1370285422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101620 msgr2=0x7f7b04199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101620 0x7f7b04199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 -- 172.21.15.52:0/1370285422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b041a4f20 con 0x7f7b04101fd0 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101620 0x7f7b04199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7b092c6700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 0x7f7b04199ea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c9e0 tx=0x7f7af400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7afaffd700 1 -- 172.21.15.52:0/1370285422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4007aa0 con 0x7f7b04101fd0 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.482+0000 7f7afaffd700 1 -- 172.21.15.52:0/1370285422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af4007c40 con 0x7f7b04101fd0 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.483+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b041a5150 con 0x7f7b04101fd0 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.483+0000 7f7afaffd700 1 -- 172.21.15.52:0/1370285422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4018660 con 0x7f7b04101fd0 2024-06-02T23:15:10.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.483+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a5690 con 0x7f7b04101fd0 2024-06-02T23:15:10.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.485+0000 7f7afaffd700 1 -- 172.21.15.52:0/1370285422 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7af400f450 con 0x7f7b04101fd0 2024-06-02T23:15:10.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.485+0000 7f7afaffd700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7af00a2c20 0x7f7af00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.486+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7af00a2c20 0x7f7af00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.486+0000 7f7afaffd700 1 -- 172.21.15.52:0/1370285422 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7af40ceff0 con 0x7f7b04101fd0 2024-06-02T23:15:10.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.486+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7af00a2c20 0x7f7af00a50e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005b40 tx=0x7f7b00005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:10.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.486+0000 7f7b165aeb80 1 Processor -- start 2024-06-02T23:15:10.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.486+0000 7f7b165aeb80 1 -- start start 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694573a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 0x55669456d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669456e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5566944bf810 con 0x556694573f60 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556694467720 con 0x55669456df20 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566944bb7d0 con 0x556694573640 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 0x55669456d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 0x55669456d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48082/0 (socket says 172.21.15.52:48082) 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 -- 172.21.15.52:0/2404440486 learned_addr learned my addr 172.21.15.52:0/2404440486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b0a2c8700 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669456e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694573a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 -- 172.21.15.52:0/2404440486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 msgr2=0x556694573a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694573a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 -- 172.21.15.52:0/2404440486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 msgr2=0x55669456e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669456e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 -- 172.21.15.52:0/2404440486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55669445df50 con 0x556694573f60 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7b092c6700 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 0x55669456d9e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7af409b580 tx=0x7f7af409c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2820110a81fdf8fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7aeffff700 1 -- 172.21.15.52:0/2404440486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40a44f0 con 0x556694573f60 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7aeffff700 1 -- 172.21.15.52:0/2404440486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af4099920 con 0x556694573f60 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.487+0000 7f7aeffff700 1 -- 172.21.15.52:0/2404440486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af409bd40 con 0x556694573f60 2024-06-02T23:15:10.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2404440486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 msgr2=0x55669456d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 0x55669456d9e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7af409b580 tx=0x7f7af409c000 comp rx=0 tx=0).stop 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2404440486 shutdown_connections 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669456e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694573a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2404440486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573f60 0x55669456d9e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2404440486 >> 172.21.15.52:0/2404440486 conn(0x556694498cc0 msgr2=0x556694568850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2404440486 shutdown_connections 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.488+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2404440486 wait complete. 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 -- start start 2024-06-02T23:15:10.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 Processor -- start 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55669456df20 0x556694579d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x556694577050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694577590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55669457a9d0 con 0x556694573640 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55669445fbf0 con 0x556694573f60 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55669457ab50 con 0x55669456df20 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b165aeb80 10 client.0 _ll_register_callbacks cb 0x556694459920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x556694577050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x556694577050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48090/0 (socket says 172.21.15.52:48090) 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.489+0000 7f7b092c6700 1 -- 172.21.15.52:0/1217105817 learned_addr learned my addr 172.21.15.52:0/1217105817 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55669456df20 0x556694579d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b0a2c8700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694577590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b092c6700 1 -- 172.21.15.52:0/1217105817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55669456df20 msgr2=0x556694579d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b092c6700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55669456df20 0x556694579d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b092c6700 1 -- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 msgr2=0x556694577590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b092c6700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694577590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b092c6700 1 -- 172.21.15.52:0/1217105817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55669440e8d0 con 0x556694573640 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b0a2c8700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694577590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b092c6700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x556694577050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ccb70 tx=0x7f7af4099d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89f45c59ec1ebed server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7aed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7aed7fa700 1 -- 172.21.15.52:0/1217105817 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40b8070 con 0x556694573640 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7aed7fa700 1 -- 172.21.15.52:0/1217105817 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7af40a4060 con 0x556694573640 2024-06-02T23:15:10.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af4099920 con 0x556694573640 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7aed7fa700 1 -- 172.21.15.52:0/1217105817 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7af40a4260 con 0x556694573640 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b165aeb80 10 client.16560 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556694585300 con 0x556694573640 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.490+0000 7f7aed7fa700 1 -- 172.21.15.52:0/1217105817 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af4099920 con 0x556694573640 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7aed7fa700 1 -- 172.21.15.52:0/1217105817 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7af4020030 con 0x556694573640 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b165aeb80 10 client.16560 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b165aeb80 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b165aeb80 20 client.16560 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945877d0 0x556694589bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b165aeb80 4 client.16560 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566944ccd10 con 0x5566945877d0 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945877d0 0x556694589bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945877d0 0x556694589bc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bed2da220e958667 server_cookie=63d288f596f2e61a in_seq=0 out_seq=0 2024-06-02T23:15:10.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.491+0000 7f7aed7fa700 10 client.16560 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7aed7fa700 1 -- 172.21.15.52:0/1217105817 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566944ccd10 con 0x5566945877d0 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7aed7fa700 10 client.16560 handle_command_reply: tid=0 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 client.16560 shutdown 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 msgr2=0x556694577050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x556694577050 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7af40ccb70 tx=0x7f7af4099d90 comp rx=0 tx=0).stop 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 shutdown_connections 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945877d0 0x556694589bc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694577590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55669456df20 0x556694579d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/1217105817 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x556694577050 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7aed7fa700 0 client.16560 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.492+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 >> 172.21.15.52:0/1217105817 conn(0x556694498cc0 msgr2=0x5566945685f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:10.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.493+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 shutdown_connections 2024-06-02T23:15:10.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.493+0000 7f7b165aeb80 1 -- 172.21.15.52:0/1217105817 wait complete. 2024-06-02T23:15:10.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.493+0000 7f7b165aeb80 20 client.16560 trim_cache size 0 max 16384 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 Processor -- start 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 -- start start 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 0x55669456d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694574c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694562510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041acaa0 con 0x55669456df20 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ae670 con 0x556694573f60 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ad170 con 0x556694573640 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 0x55669456d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 0x55669456d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48102/0 (socket says 172.21.15.52:48102) 2024-06-02T23:15:10.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 -- 172.21.15.52:0/2575447624 learned_addr learned my addr 172.21.15.52:0/2575447624 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 -- 172.21.15.52:0/2575447624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 msgr2=0x556694574c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694574c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 -- 172.21.15.52:0/2575447624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 msgr2=0x556694562510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694562510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.508+0000 7f7b09ac7700 1 -- 172.21.15.52:0/2575447624 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55669457aa50 con 0x55669456df20 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 0x55669456d9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005f50 tx=0x7f7b0001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2e8ed7568df26e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7aef7fe700 1 -- 172.21.15.52:0/2575447624 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0000bde0 con 0x55669456df20 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7aef7fe700 1 -- 172.21.15.52:0/2575447624 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b0000f910 con 0x55669456df20 2024-06-02T23:15:10.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7aef7fe700 1 -- 172.21.15.52:0/2575447624 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000315b0 con 0x55669456df20 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2575447624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 msgr2=0x55669456d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 0x55669456d9e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005f50 tx=0x7f7b0001f000 comp rx=0 tx=0).stop 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2575447624 shutdown_connections 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556694573f60 0x556694562510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55669456df20 0x55669456d9e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/2575447624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573640 0x556694574c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2575447624 >> 172.21.15.52:0/2575447624 conn(0x556694498cc0 msgr2=0x556694561e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2575447624 shutdown_connections 2024-06-02T23:15:10.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.509+0000 7f7b165aeb80 1 -- 172.21.15.52:0/2575447624 wait complete. 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 -- start start 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 Processor -- start 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669458d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x55669458a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573f60 0x55669458aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041adf10 con 0x556694573640 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041ad840 con 0x55669456df20 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041aaa40 con 0x556694573f60 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b165aeb80 10 client.0 _ll_register_callbacks cb 0x556694459920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x55669458a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b09ac7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669458d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b092c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x55669458a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48118/0 (socket says 172.21.15.52:48118) 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b09ac7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669458d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34630/0 (socket says 172.21.15.52:34630) 2024-06-02T23:15:10.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.510+0000 7f7b092c6700 1 -- 172.21.15.52:0/847983231 learned_addr learned my addr 172.21.15.52:0/847983231 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b0a2c8700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573f60 0x55669458aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b092c6700 1 -- 172.21.15.52:0/847983231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573f60 msgr2=0x55669458aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b092c6700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573f60 0x55669458aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b092c6700 1 -- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 msgr2=0x55669458d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b092c6700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669458d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b092c6700 1 -- 172.21.15.52:0/847983231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55669458b170 con 0x556694573640 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b0a2c8700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573f60 0x55669458aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b092c6700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x55669458a580 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c9b0 tx=0x7f7af409be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640dd592804b62b9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7aed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7aed7fa700 1 -- 172.21.15.52:0/847983231 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40b8070 con 0x556694573640 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7aed7fa700 1 -- 172.21.15.52:0/847983231 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7af40860b0 con 0x556694573640 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b0000f910 con 0x556694573640 2024-06-02T23:15:10.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7aed7fa700 1 -- 172.21.15.52:0/847983231 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7af4097b20 con 0x556694573640 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b165aeb80 10 client.16566 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5566945f1fd0 con 0x556694573640 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.511+0000 7f7aed7fa700 1 -- 172.21.15.52:0/847983231 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7b0000f910 con 0x556694573640 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7aed7fa700 1 -- 172.21.15.52:0/847983231 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f7af4020070 con 0x556694573640 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b165aeb80 10 client.16566 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b165aeb80 10 client.16566 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b165aeb80 20 client.16566 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945f2390 0x5566945f4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b165aeb80 4 client.16566 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5566944c7290 con 0x5566945f2390 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945f2390 0x5566945f4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7b09ac7700 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945f2390 0x5566945f4780 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=81e76375f4ca3141 server_cookie=c32755448a040bf8 in_seq=0 out_seq=0 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.512+0000 7f7aed7fa700 10 client.16566 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:15:10.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7aed7fa700 1 -- 172.21.15.52:0/847983231 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5566944c7290 con 0x5566945f2390 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7aed7fa700 10 client.16566 handle_command_reply: tid=0 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 client.16566 shutdown 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 msgr2=0x55669458a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x55669458a580 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c9b0 tx=0x7f7af409be80 comp rx=0 tx=0).stop 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 shutdown_connections 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5566945f2390 0x5566945f4780 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556694573f60 0x55669458aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55669456df20 0x55669458d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7aed7fa700 0 client.16566 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 --2- 172.21.15.52:0/847983231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556694573640 0x55669458a580 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 >> 172.21.15.52:0/847983231 conn(0x556694498cc0 msgr2=0x556694574a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 shutdown_connections 2024-06-02T23:15:10.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 1 -- 172.21.15.52:0/847983231 wait complete. 2024-06-02T23:15:10.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.513+0000 7f7b165aeb80 20 client.16566 trim_cache size 0 max 16384 2024-06-02T23:15:10.519 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:15:10.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7af00a2c20 msgr2=0x7f7af00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7af00a2c20 0x7f7af00a50e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7b00005b40 tx=0x7f7b00005ab0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 msgr2=0x7f7b04199ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 0x7f7b04199ea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7af400c9e0 tx=0x7f7af400cda0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 shutdown_connections 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7af00a2c20 0x7f7af00a50e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b041066b0 0x7f7b041a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101620 0x7f7b04199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 --2- 172.21.15.52:0/1370285422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101fd0 0x7f7b04199ea0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 >> 172.21.15.52:0/1370285422 conn(0x7f7b04078340 msgr2=0x7f7b04105840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 shutdown_connections 2024-06-02T23:15:10.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:10.514+0000 7f7b0bd2b700 1 -- 172.21.15.52:0/1370285422 wait complete. 2024-06-02T23:15:10.623 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 19 sec latency 19013.537 ms 2024-06-02T23:15:11.251 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:15:11.501 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:11.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:11 smithi123 ceph-mon[35977]: pgmap v530: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-06-02T23:15:11.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 20 sec latency 20013.650 ms 2024-06-02T23:15:11.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:11 smithi052 ceph-mon[32340]: pgmap v530: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-06-02T23:15:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:11 smithi174 ceph-mon[35994]: pgmap v530: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 63 KiB/s wr, 15 op/s 2024-06-02T23:15:12.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 21 sec latency 21013.746 ms 2024-06-02T23:15:12.777 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:15:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.315+0000 7fb7924d8700 1 -- 172.21.15.52:0/2172982445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 msgr2=0x7fb78c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.315+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2172982445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c101800 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb780009990 tx=0x7fb780009ca0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 -- 172.21.15.52:0/2172982445 shutdown_connections 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2172982445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c101d40 0x7fb78c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2172982445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb78c1073f0 0x7fb78c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2172982445 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c101800 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 -- 172.21.15.52:0/2172982445 >> 172.21.15.52:0/2172982445 conn(0x7fb78c078300 msgr2=0x7fb78c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 -- 172.21.15.52:0/2172982445 shutdown_connections 2024-06-02T23:15:13.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.316+0000 7fb7924d8700 1 -- 172.21.15.52:0/2172982445 wait complete. 2024-06-02T23:15:13.317 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:13 smithi052 ceph-mon[32340]: pgmap v531: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 Processor -- start 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 -- start start 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c101d40 0x7fb78c19b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb78c1073f0 0x7fb78c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c111510 con 0x7fb78c101380 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c111390 con 0x7fb78c1073f0 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb7924d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c111690 con 0x7fb78c101d40 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb78bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb78bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c19ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48134/0 (socket says 172.21.15.52:48134) 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.317+0000 7fb78bfff700 1 -- 172.21.15.52:0/2505506819 learned_addr learned my addr 172.21.15.52:0/2505506819 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c101d40 0x7fb78c19b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb790a75700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb78c1073f0 0x7fb78c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78bfff700 1 -- 172.21.15.52:0/2505506819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c101d40 msgr2=0x7fb78c19b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78bfff700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c101d40 0x7fb78c19b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78bfff700 1 -- 172.21.15.52:0/2505506819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb78c1073f0 msgr2=0x7fb78c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78bfff700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb78c1073f0 0x7fb78c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78bfff700 1 -- 172.21.15.52:0/2505506819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78c19fdb0 con 0x7fb78c101380 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb78bfff700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c19ada0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00d950 tx=0x7fb77c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb7897fa700 1 -- 172.21.15.52:0/2505506819 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c016070 con 0x7fb78c101380 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb7897fa700 1 -- 172.21.15.52:0/2505506819 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb77c00fd40 con 0x7fb78c101380 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb7897fa700 1 -- 172.21.15.52:0/2505506819 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c00e3b0 con 0x7fb78c101380 2024-06-02T23:15:13.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.318+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78c1a00a0 con 0x7fb78c101380 2024-06-02T23:15:13.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.319+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb78c1a7920 con 0x7fb78c101380 2024-06-02T23:15:13.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.322+0000 7fb7897fa700 1 -- 172.21.15.52:0/2505506819 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb77c019070 con 0x7fb78c101380 2024-06-02T23:15:13.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.323+0000 7fb7897fa700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7740a2ca0 0x7fb7740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.323+0000 7fb7897fa700 1 -- 172.21.15.52:0/2505506819 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb77c0cead0 con 0x7fb78c101380 2024-06-02T23:15:13.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.323+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7740a2ca0 0x7fb7740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.324+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7740a2ca0 0x7fb7740a5160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b5c0 tx=0x7fb780005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:13.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.324+0000 7fb79cd5bb80 1 Processor -- start 2024-06-02T23:15:13.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.324+0000 7fb79cd5bb80 1 -- start start 2024-06-02T23:15:13.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a5640 0x556c661a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 0x556c6619f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c6619ff20 0x556c661a0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb78b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 0x556c6619f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb78b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 0x556c6619f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48136/0 (socket says 172.21.15.52:48136) 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb78b7fe700 1 -- 172.21.15.52:0/364831385 learned_addr learned my addr 172.21.15.52:0/364831385 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb78bfff700 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a5640 0x556c661a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556c660f1810 con 0x556c661a5f60 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c66099720 con 0x556c6619ff20 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb790a75700 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c6619ff20 0x556c661a0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.325+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c660ed7d0 con 0x556c661a5640 2024-06-02T23:15:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb78b7fe700 1 -- 172.21.15.52:0/364831385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a5640 msgr2=0x556c661a5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a5640 0x556c661a5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb78b7fe700 1 -- 172.21.15.52:0/364831385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c6619ff20 msgr2=0x556c661a0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c6619ff20 0x556c661a0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb78b7fe700 1 -- 172.21.15.52:0/364831385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c6608ff50 con 0x556c661a5f60 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 0x556c6619f9e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb780009990 tx=0x7fb78001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e8b8b649ce373d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.326+0000 7fb7727fc700 1 -- 172.21.15.52:0/364831385 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780034070 con 0x556c661a5f60 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb7727fc700 1 -- 172.21.15.52:0/364831385 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb78000be40 con 0x556c661a5f60 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb7727fc700 1 -- 172.21.15.52:0/364831385 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78002f940 con 0x556c661a5f60 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 msgr2=0x556c6619f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 0x556c6619f9e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb780009990 tx=0x7fb78001f000 comp rx=0 tx=0).stop 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 shutdown_connections 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c6619ff20 0x556c661a0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a5640 0x556c661a5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/364831385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5f60 0x556c6619f9e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 >> 172.21.15.52:0/364831385 conn(0x556c660cacc0 msgr2=0x556c6619a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 shutdown_connections 2024-06-02T23:15:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.327+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/364831385 wait complete. 2024-06-02T23:15:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.328+0000 7fb79cd5bb80 1 -- start start 2024-06-02T23:15:13.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.328+0000 7fb79cd5bb80 1 Processor -- start 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.328+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 0x556c661abf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 0x556c661abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 0x556c661abf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48142/0 (socket says 172.21.15.52:48142) 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a97d0 0x556c661a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556c661ac970 con 0x556c6619ff20 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c66091bf0 con 0x556c661a5640 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c661acaf0 con 0x556c661a97d0 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 -- 172.21.15.52:0/3503213069 learned_addr learned my addr 172.21.15.52:0/3503213069 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:13.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 10 client.0 _ll_register_callbacks cb 0x556c6608b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 -- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a97d0 msgr2=0x556c661a9c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a97d0 0x556c661a9c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 -- 172.21.15.52:0/3503213069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 msgr2=0x556c661a9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 -- 172.21.15.52:0/3503213069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c660408d0 con 0x556c6619ff20 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb78bfff700 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 0x556c661abf40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb77c09b290 tx=0x7fb77c09bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76226833c1467c74 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb76b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb76b7fe700 1 -- 172.21.15.52:0/3503213069 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77c020070 con 0x556c6619ff20 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb76b7fe700 1 -- 172.21.15.52:0/3503213069 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb77c010710 con 0x556c6619ff20 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.329+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556c660fed10 con 0x556c6619ff20 2024-06-02T23:15:13.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb76b7fe700 1 -- 172.21.15.52:0/3503213069 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb77c0a3f70 con 0x556c6619ff20 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 10 client.16578 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c661acdf0 con 0x556c6619ff20 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb76b7fe700 1 -- 172.21.15.52:0/3503213069 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x556c660fed10 con 0x556c6619ff20 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb76b7fe700 1 -- 172.21.15.52:0/3503213069 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb77c09a750 con 0x556c6619ff20 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 10 client.16578 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 10 client.16578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 20 client.16578 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c661b1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 4 client.16578 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c660f9290 con 0x556c661af380 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.330+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c661b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.331+0000 7fb76b7fe700 10 client.16578 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:15:13.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.331+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c661b1820 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fb50dd1686863fed server_cookie=93d9ede741be9e9f in_seq=0 out_seq=0 2024-06-02T23:15:13.335 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb76b7fe700 1 -- 172.21.15.52:0/3503213069 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556c660f9290 con 0x556c661af380 2024-06-02T23:15:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb76b7fe700 10 client.16578 handle_command_reply: tid=0 2024-06-02T23:15:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 client.16578 shutdown 2024-06-02T23:15:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 msgr2=0x556c661abf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 0x556c661abf40 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb77c09b290 tx=0x7fb77c09bb50 comp rx=0 tx=0).stop 2024-06-02T23:15:13.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 shutdown_connections 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c661b1820 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb76b7fe700 0 client.16578 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c661a97d0 0x556c661a9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c6619ff20 0x556c661abf40 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/3503213069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.332+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 >> 172.21.15.52:0/3503213069 conn(0x556c660cacc0 msgr2=0x556c6619a5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.333+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 shutdown_connections 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.333+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/3503213069 wait complete. 2024-06-02T23:15:13.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.333+0000 7fb79cd5bb80 20 client.16578 trim_cache size 0 max 16384 2024-06-02T23:15:13.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.357+0000 7fb79cd5bb80 1 Processor -- start 2024-06-02T23:15:13.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 -- start start 2024-06-02T23:15:13.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c6619f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 0x556c661a4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb790a75700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 0x556c661a4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb790a75700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 0x556c661a4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48144/0 (socket says 172.21.15.52:48144) 2024-06-02T23:15:13.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb790a75700 1 -- 172.21.15.52:0/303056465 learned_addr learned my addr 172.21.15.52:0/303056465 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:13.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1aa820 con 0x556c661a97d0 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1ac3f0 con 0x556c661a5640 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1aaef0 con 0x556c6619ff20 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.358+0000 7fb78bfff700 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c6619f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb790a75700 1 -- 172.21.15.52:0/303056465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 msgr2=0x556c6619f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb790a75700 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c6619f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb790a75700 1 -- 172.21.15.52:0/303056465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 msgr2=0x556c661a7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb790a75700 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb790a75700 1 -- 172.21.15.52:0/303056465 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c661bed00 con 0x556c661a97d0 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:13.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.359+0000 7fb790a75700 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 0x556c661a4c30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb784009a40 tx=0x7fb784003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5487cde5f10fae server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:13.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb771ffb700 1 -- 172.21.15.52:0/303056465 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78401a070 con 0x556c661a97d0 2024-06-02T23:15:13.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb771ffb700 1 -- 172.21.15.52:0/303056465 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb784016430 con 0x556c661a97d0 2024-06-02T23:15:13.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb771ffb700 1 -- 172.21.15.52:0/303056465 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb784007640 con 0x556c661a97d0 2024-06-02T23:15:13.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 msgr2=0x556c661a4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 0x556c661a4c30 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fb784009a40 tx=0x7fb784003f90 comp rx=0 tx=0).stop 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 shutdown_connections 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a97d0 0x556c661a4c30 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c6619f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/303056465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a5640 0x556c661a7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.360+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 >> 172.21.15.52:0/303056465 conn(0x556c660cacc0 msgr2=0x556c66193e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 shutdown_connections 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/303056465 wait complete. 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 -- start start 2024-06-02T23:15:13.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 Processor -- start 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c661c1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 0x556c661c1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.361+0000 7fb79cd5bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a97d0 0x556c662230c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb79cd5bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1abc90 con 0x556c661a5640 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb79cd5bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c1ab5c0 con 0x556c661a97d0 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb79cd5bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb78c10c040 con 0x556c6619ff20 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb79cd5bb80 10 client.0 _ll_register_callbacks cb 0x556c6608b920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 0x556c661c1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb790a75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a97d0 0x556c662230c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 0x556c661c1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48150/0 (socket says 172.21.15.52:48150) 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 -- 172.21.15.52:0/1528951807 learned_addr learned my addr 172.21.15.52:0/1528951807 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78bfff700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c661c1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 -- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 msgr2=0x556c661c1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c661c1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 -- 172.21.15.52:0/1528951807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a97d0 msgr2=0x556c662230c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a97d0 0x556c662230c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 -- 172.21.15.52:0/1528951807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c66223680 con 0x556c661a5640 2024-06-02T23:15:13.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb790a75700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a97d0 0x556c662230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb78b7fe700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 0x556c661c1b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb7800037f0 tx=0x7fb7800092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8153abe9336472c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.362+0000 7fb76b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb76b7fe700 1 -- 172.21.15.52:0/1528951807 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780034040 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb76b7fe700 1 -- 172.21.15.52:0/1528951807 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb780047480 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb784016430 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb76b7fe700 1 -- 172.21.15.52:0/1528951807 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb780009590 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb79cd5bb80 10 client.16584 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x556c66224960 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb76b7fe700 1 -- 172.21.15.52:0/1528951807 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb784016430 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.363+0000 7fb76b7fe700 1 -- 172.21.15.52:0/1528951807 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fb78000f460 con 0x556c661a5640 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb79cd5bb80 10 client.16584 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb79cd5bb80 10 client.16584 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb79cd5bb80 20 client.16584 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:13.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c66226ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb79cd5bb80 4 client.16584 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c65f73250 con 0x556c661af380 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb78bfff700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c66226ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb76b7fe700 10 client.16584 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.364+0000 7fb78bfff700 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c66226ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=71e56b89525c3c9 server_cookie=e27e0219738320af in_seq=0 out_seq=0 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb76b7fe700 1 -- 172.21.15.52:0/1528951807 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c65f73250 con 0x556c661af380 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb76b7fe700 10 client.16584 handle_command_reply: tid=0 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 client.16584 shutdown 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 msgr2=0x556c661c1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 0x556c661c1b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fb7800037f0 tx=0x7fb7800092b0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 shutdown_connections 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c661af380 0x556c66226ba0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c661a97d0 0x556c662230c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c6619ff20 0x556c661c1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 --2- 172.21.15.52:0/1528951807 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c661a5640 0x556c661c1b50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 >> 172.21.15.52:0/1528951807 conn(0x556c660cacc0 msgr2=0x556c661a4ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb76b7fe700 0 client.16584 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 shutdown_connections 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 1 -- 172.21.15.52:0/1528951807 wait complete. 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.365+0000 7fb79cd5bb80 20 client.16584 trim_cache size 0 max 16384 2024-06-02T23:15:13.370 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7740a2ca0 msgr2=0x7fb7740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7740a2ca0 0x7fb7740a5160 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb78000b5c0 tx=0x7fb780005960 comp rx=0 tx=0).stop 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 msgr2=0x7fb78c19ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c19ada0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb77c00d950 tx=0x7fb77c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 shutdown_connections 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7740a2ca0 0x7fb7740a5160 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb78c1073f0 0x7fb78c19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb78c101380 0x7fb78c19ada0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 --2- 172.21.15.52:0/2505506819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb78c101d40 0x7fb78c19b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 >> 172.21.15.52:0/2505506819 conn(0x7fb78c078300 msgr2=0x7fb78c06aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 shutdown_connections 2024-06-02T23:15:13.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:13.366+0000 7fb7924d8700 1 -- 172.21.15.52:0/2505506819 wait complete. 2024-06-02T23:15:13.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:13 smithi123 ceph-mon[35977]: pgmap v531: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-06-02T23:15:13.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 22 sec latency 22013.791 ms 2024-06-02T23:15:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:13 smithi174 ceph-mon[35994]: pgmap v531: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-06-02T23:15:14.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 23 sec latency 23013.829 ms 2024-06-02T23:15:14.801 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:15:15.445 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:15 smithi174 ceph-mon[35994]: pgmap v532: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-06-02T23:15:15.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:15 smithi123 ceph-mon[35977]: pgmap v532: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-06-02T23:15:15.528 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:15 smithi052 ceph-mon[32340]: pgmap v532: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-06-02T23:15:15.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 24 sec latency 24013.884 ms 2024-06-02T23:15:15.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 -- 172.21.15.52:0/3761546309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102530 msgr2=0x7f4d881029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3761546309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102530 0x7f4d881029b0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0099c0 tx=0x7f4d7c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 -- 172.21.15.52:0/3761546309 shutdown_connections 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3761546309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d88102ef0 0x7f4d8810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3761546309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d881085d0 0x7f4d881089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3761546309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102530 0x7f4d881029b0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 -- 172.21.15.52:0/3761546309 >> 172.21.15.52:0/3761546309 conn(0x7f4d88079640 msgr2=0x7f4d88079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 -- 172.21.15.52:0/3761546309 shutdown_connections 2024-06-02T23:15:15.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.934+0000 7f4d8df55700 1 -- 172.21.15.52:0/3761546309 wait complete. 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.935+0000 7f4d8df55700 1 Processor -- start 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.935+0000 7f4d8df55700 1 -- start start 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d8df55700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88102530 0x7f4d88074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d8df55700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 0x7f4d88074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d8df55700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d881085d0 0x7f4d88075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d8df55700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d881126c0 con 0x7f4d88102ef0 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d8df55700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88112540 con 0x7f4d881085d0 2024-06-02T23:15:15.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d8df55700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d88112840 con 0x7f4d88102530 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 0x7f4d88074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 0x7f4d88074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45528/0 (socket says 172.21.15.52:45528) 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 -- 172.21.15.52:0/3738901594 learned_addr learned my addr 172.21.15.52:0/3738901594 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d877fe700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88102530 0x7f4d88074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d87fff700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d881085d0 0x7f4d88075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 -- 172.21.15.52:0/3738901594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88102530 msgr2=0x7f4d88074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88102530 0x7f4d88074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 -- 172.21.15.52:0/3738901594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d881085d0 msgr2=0x7f4d88075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d881085d0 0x7f4d88075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 -- 172.21.15.52:0/3738901594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d881a6fe0 con 0x7f4d88102ef0 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d877fe700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88102530 0x7f4d88074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.936+0000 7f4d86ffd700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 0x7f4d88074cd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0059d0 tx=0x7f4d7c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:15.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.937+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c027070 con 0x7f4d88102ef0 2024-06-02T23:15:15.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.937+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d881a7270 con 0x7f4d88102ef0 2024-06-02T23:15:15.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.937+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d7c003f50 con 0x7f4d88102ef0 2024-06-02T23:15:15.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.937+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c022d80 con 0x7f4d88102ef0 2024-06-02T23:15:15.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.937+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d881a7720 con 0x7f4d88102ef0 2024-06-02T23:15:15.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.938+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d88069180 con 0x7f4d88102ef0 2024-06-02T23:15:15.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.940+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d7c02a070 con 0x7f4d88102ef0 2024-06-02T23:15:15.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.941+0000 7f4d84ff9700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d700a2c20 0x7f4d700a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:15.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.941+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4d7c0d0410 con 0x7f4d88102ef0 2024-06-02T23:15:15.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.941+0000 7f4d877fe700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d700a2c20 0x7f4d700a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:15.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.941+0000 7f4d877fe700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d700a2c20 0x7f4d700a50e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005fd0 tx=0x7f4d7800a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:15.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:15.945+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4d7c031020 con 0x7f4d88102ef0 2024-06-02T23:15:15.968 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:16.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.207+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4d8802dca0 con 0x7f4d88102ef0 2024-06-02T23:15:16.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.208+0000 7f4d84ff9700 1 -- 172.21.15.52:0/3738901594 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11944 (secure 0 0 0) 0x7f4d7c013070 con 0x7f4d88102ef0 2024-06-02T23:15:16.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:15:16.209 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:13:05.515141+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d700a2c20 msgr2=0x7f4d700a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d700a2c20 0x7f4d700a50e0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4d78005fd0 tx=0x7f4d7800a5e0 comp rx=0 tx=0).stop 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 msgr2=0x7f4d88074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 0x7f4d88074cd0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c0059d0 tx=0x7f4d7c0057c0 comp rx=0 tx=0).stop 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 shutdown_connections 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d700a2c20 0x7f4d700a50e0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d881085d0 0x7f4d88075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d88102530 0x7f4d88074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 --2- 172.21.15.52:0/3738901594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d88102ef0 0x7f4d88074cd0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 >> 172.21.15.52:0/3738901594 conn(0x7f4d88079640 msgr2=0x7f4d881010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:16.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 shutdown_connections 2024-06-02T23:15:16.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:16.211+0000 7f4d8df55700 1 -- 172.21.15.52:0/3738901594 wait complete. 2024-06-02T23:15:16.213 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 30 2024-06-02T23:15:16.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 25 sec latency 25013.966 ms 2024-06-02T23:15:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2869639601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c107710 msgr2=0x7fdc5c109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2869639601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c107710 0x7fdc5c109b60 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fdc54009ca0 tx=0x7fdc5402a8c0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2869639601 shutdown_connections 2024-06-02T23:15:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2869639601 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c107710 0x7fdc5c109b60 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2869639601 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2869639601 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5c1024b0 0x7fdc5c1048a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2869639601 >> 172.21.15.52:0/2869639601 conn(0x7fdc5c0fbea0 msgr2=0x7fdc5c0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2869639601 shutdown_connections 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2869639601 wait complete. 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.213+0000 7fdc62a6b640 1 Processor -- start 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 -- start start 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc5c1024b0 0x7fdc5c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5c107710 0x7fdc5c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10d580 con 0x7fdc5c104de0 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10d400 con 0x7fdc5c1024b0 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc62a6b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c10d700 con 0x7fdc5c107710 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45548/0 (socket says 172.21.15.52:45548) 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 -- 172.21.15.52:0/2641335592 learned_addr learned my addr 172.21.15.52:0/2641335592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc60807640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc5c1024b0 0x7fdc5c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc61008640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5c107710 0x7fdc5c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 -- 172.21.15.52:0/2641335592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5c107710 msgr2=0x7fdc5c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5c107710 0x7fdc5c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 -- 172.21.15.52:0/2641335592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc5c1024b0 msgr2=0x7fdc5c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc5c1024b0 0x7fdc5c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 -- 172.21.15.52:0/2641335592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc5c19f7b0 con 0x7fdc5c104de0 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.214+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1980d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9c0 tx=0x7fdc5000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.215+0000 7fdc59ffb640 1 -- 172.21.15.52:0/2641335592 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5000cd80 con 0x7fdc5c104de0 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.215+0000 7fdc59ffb640 1 -- 172.21.15.52:0/2641335592 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc500054d0 con 0x7fdc5c104de0 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.215+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc5c19faa0 con 0x7fdc5c104de0 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.215+0000 7fdc59ffb640 1 -- 172.21.15.52:0/2641335592 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc50010430 con 0x7fdc5c104de0 2024-06-02T23:15:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.215+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc5c19fff0 con 0x7fdc5c104de0 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.217+0000 7fdc59ffb640 1 -- 172.21.15.52:0/2641335592 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc50004fd0 con 0x7fdc5c104de0 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 Processor -- start 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 -- start start 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78010 0x560a7ff78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78a00 0x560a7ff7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff80000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560a7fedb4f0 con 0x560a7ff7db70 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a7fe79870 con 0x560a7ff78a00 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc65778740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560a7fed74b0 con 0x560a7ff78010 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc59ffb640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc440a2c30 0x7fdc440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff80000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc61008640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc440a2c30 0x7fdc440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff80000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45562/0 (socket says 172.21.15.52:45562) 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc5bfff640 1 -- 172.21.15.52:0/426616183 learned_addr learned my addr 172.21.15.52:0/426616183 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc61008640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78010 0x560a7ff78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc60807640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78a00 0x560a7ff7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc59ffb640 1 -- 172.21.15.52:0/2641335592 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdc500cf510 con 0x7fdc5c104de0 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc61008640 1 -- 172.21.15.52:0/426616183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78a00 msgr2=0x560a7ff7d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.218+0000 7fdc61008640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78a00 0x560a7ff7d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc61008640 1 -- 172.21.15.52:0/426616183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 msgr2=0x560a7ff80000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc61008640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff80000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc61008640 1 -- 172.21.15.52:0/426616183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7fe959b0 con 0x560a7ff78010 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc60807640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78a00 0x560a7ff7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff80000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc61008640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc440a2c30 0x7fdc440a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc54031970 tx=0x7fdc54043000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc61008640 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78010 0x560a7ff78430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc54006010 tx=0x7fdc54032ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71179aa7c4fb3b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc42ffd640 1 -- 172.21.15.52:0/426616183 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5404e070 con 0x560a7ff78010 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc42ffd640 1 -- 172.21.15.52:0/426616183 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc54036040 con 0x560a7ff78010 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.219+0000 7fdc42ffd640 1 -- 172.21.15.52:0/426616183 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54004780 con 0x560a7ff78010 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 -- 172.21.15.52:0/426616183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78010 msgr2=0x560a7ff78430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78010 0x560a7ff78430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdc54006010 tx=0x7fdc54032ce0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 -- 172.21.15.52:0/426616183 shutdown_connections 2024-06-02T23:15:17.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff80000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78a00 0x560a7ff7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 --2- 172.21.15.52:0/426616183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78010 0x560a7ff78430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 -- 172.21.15.52:0/426616183 >> 172.21.15.52:0/426616183 conn(0x560a7ff73680 msgr2=0x560a7ff73a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 -- 172.21.15.52:0/426616183 shutdown_connections 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 -- 172.21.15.52:0/426616183 wait complete. 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 -- start start 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.220+0000 7fdc65778740 1 Processor -- start 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff82260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7ff827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560a7ff80ac0 con 0x560a7ff7db70 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a7fe64ae0 con 0x560a7ff78010 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560a7ff80c40 con 0x560a7ff78a00 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc65778740 10 client.0 _ll_register_callbacks cb 0x560a7fe9d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff897e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45574/0 (socket says 172.21.15.52:45574) 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 -- 172.21.15.52:0/3411059261 learned_addr learned my addr 172.21.15.52:0/3411059261 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7ff827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc60807640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff82260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 -- 172.21.15.52:0/3411059261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 msgr2=0x560a7ff827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7ff827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 -- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 msgr2=0x560a7ff82260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff82260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 -- 172.21.15.52:0/3411059261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7fbef210 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc61008640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff897e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdc54004af0 tx=0x7fdc54004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d0df183a8bc8b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.221+0000 7fdc3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3411059261 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5404e040 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3411059261 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdc54003e00 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc54036040 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3411059261 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc54004250 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 10 client.16602 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a7fcbb620 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3411059261 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc54036040 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3411059261 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fdc54053450 con 0x560a7ff7db70 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 10 client.16602 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:17.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 10 client.16602 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 20 client.16602 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7ff8dd20 0x560a7ff90160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 4 client.16602 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.222+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a7feeef10 con 0x560a7ff8dd20 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.223+0000 7fdc60807640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7ff8dd20 0x560a7ff90160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.223+0000 7fdc60807640 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7ff8dd20 0x560a7ff90160 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1978daac350ec52 server_cookie=42009a696334383 in_seq=0 out_seq=0 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.223+0000 7fdc3bfff640 10 client.16602 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3411059261 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560a7feeef10 con 0x560a7ff8dd20 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc3bfff640 10 client.16602 handle_command_reply: tid=0 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 client.16602 shutdown 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 msgr2=0x560a7ff897e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff897e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fdc54004af0 tx=0x7fdc54004790 comp rx=0 tx=0).stop 2024-06-02T23:15:17.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 shutdown_connections 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7ff8dd20 0x560a7ff90160 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7ff827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff7db70 0x560a7ff897e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 --2- 172.21.15.52:0/3411059261 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff82260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc3bfff640 0 client.16602 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.224+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 >> 172.21.15.52:0/3411059261 conn(0x560a7ff73680 msgr2=0x560a7ff727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.225+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 shutdown_connections 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.225+0000 7fdc65778740 1 -- 172.21.15.52:0/3411059261 wait complete. 2024-06-02T23:15:17.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.225+0000 7fdc65778740 20 client.16602 trim_cache size 0 max 16384 2024-06-02T23:15:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 Processor -- start 2024-06-02T23:15:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 -- start start 2024-06-02T23:15:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff79c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 0x560a7ff7a160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.242 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff7db70 0x560a7ff76670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1a8c70 con 0x560a7ff78a00 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1aa840 con 0x560a7ff78010 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc65778740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1a9340 con 0x560a7ff7db70 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.240+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 0x560a7ff7a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 0x560a7ff7a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45586/0 (socket says 172.21.15.52:45586) 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 -- 172.21.15.52:0/4057486448 learned_addr learned my addr 172.21.15.52:0/4057486448 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc60807640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff79c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc61008640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff7db70 0x560a7ff76670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 -- 172.21.15.52:0/4057486448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff7db70 msgr2=0x560a7ff76670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff7db70 0x560a7ff76670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 -- 172.21.15.52:0/4057486448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 msgr2=0x560a7ff79c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff79c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 -- 172.21.15.52:0/4057486448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a7ff6dae0 con 0x560a7ff78a00 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc61008640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff7db70 0x560a7ff76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc60807640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff79c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 0x560a7ff7a160 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000cfd0 tx=0x7fdc500cceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6b046e2e3651e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc427fc640 1 -- 172.21.15.52:0/4057486448 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5009c440 con 0x560a7ff78a00 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc427fc640 1 -- 172.21.15.52:0/4057486448 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc500a00f0 con 0x560a7ff78a00 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.241+0000 7fdc427fc640 1 -- 172.21.15.52:0/4057486448 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc500983b0 con 0x560a7ff78a00 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 -- 172.21.15.52:0/4057486448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 msgr2=0x560a7ff7a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 0x560a7ff7a160 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000cfd0 tx=0x7fdc500cceb0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 -- 172.21.15.52:0/4057486448 shutdown_connections 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff7db70 0x560a7ff76670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78a00 0x560a7ff7a160 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 --2- 172.21.15.52:0/4057486448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff78010 0x560a7ff79c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 -- 172.21.15.52:0/4057486448 >> 172.21.15.52:0/4057486448 conn(0x560a7ff73680 msgr2=0x560a7ff7d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 -- 172.21.15.52:0/4057486448 shutdown_connections 2024-06-02T23:15:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 -- 172.21.15.52:0/4057486448 wait complete. 2024-06-02T23:15:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 -- start start 2024-06-02T23:15:17.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.242+0000 7fdc65778740 1 Processor -- start 2024-06-02T23:15:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 0x560a7fff3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7fff4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff7db70 0x560a7fffb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1aa0e0 con 0x560a7ff78010 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1a9a10 con 0x560a7ff7db70 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc5c1a6c50 con 0x560a7ff78a00 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 10 client.0 _ll_register_callbacks cb 0x7fdc5009bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 0x560a7fff3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 0x560a7fff3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45602/0 (socket says 172.21.15.52:45602) 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 -- 172.21.15.52:0/3336725048 learned_addr learned my addr 172.21.15.52:0/3336725048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 -- 172.21.15.52:0/3336725048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 msgr2=0x560a7fff4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7fff4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 -- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff7db70 msgr2=0x560a7fffb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff7db70 0x560a7fffb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 -- 172.21.15.52:0/3336725048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a7fffb640 con 0x560a7ff78010 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc60807640 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 0x560a7fff3ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x560a7ff78e00 tx=0x7fdc4c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fef42902a3a9846 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3336725048 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c005220 con 0x560a7ff78010 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3336725048 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdc4c0053c0 con 0x560a7ff78010 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.243+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc500a00f0 con 0x560a7ff78010 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3336725048 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c02d070 con 0x560a7ff78010 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 10 client.16608 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x560a7fffc9c0 con 0x560a7ff78010 2024-06-02T23:15:17.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3336725048 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc500a00f0 con 0x560a7ff78010 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3336725048 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7fdc4c025030 con 0x560a7ff78010 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 10 client.16608 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 10 client.16608 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 20 client.16608 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7fffd480 0x560a7ffff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 4 client.16608 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a7fee8c20 con 0x560a7fffd480 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.244+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7fffd480 0x560a7ffff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.245+0000 7fdc5bfff640 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7fffd480 0x560a7ffff8c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0fbe87b10cae749 server_cookie=cf23552574cc324d in_seq=0 out_seq=0 2024-06-02T23:15:17.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.245+0000 7fdc3bfff640 10 client.16608 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc3bfff640 1 -- 172.21.15.52:0/3336725048 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560a7fee8c20 con 0x560a7fffd480 2024-06-02T23:15:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc3bfff640 10 client.16608 handle_command_reply: tid=0 2024-06-02T23:15:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 client.16608 shutdown 2024-06-02T23:15:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 msgr2=0x560a7fff3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 0x560a7fff3ad0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x560a7ff78e00 tx=0x7fdc4c010800 comp rx=0 tx=0).stop 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 shutdown_connections 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560a7fffd480 0x560a7ffff8c0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a7ff78a00 0x560a7fff4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a7ff7db70 0x560a7fffb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 --2- 172.21.15.52:0/3336725048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560a7ff78010 0x560a7fff3ad0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc3bfff640 0 client.16608 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 >> 172.21.15.52:0/3336725048 conn(0x560a7ff73680 msgr2=0x560a7ff79c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 shutdown_connections 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 1 -- 172.21.15.52:0/3336725048 wait complete. 2024-06-02T23:15:17.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.260+0000 7fdc65778740 20 client.16608 trim_cache size 0 max 16384 2024-06-02T23:15:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:15:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:15:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:15:17.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc440a2c30 msgr2=0x7fdc440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc440a2c30 0x7fdc440a50f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdc54031970 tx=0x7fdc54043000 comp rx=0 tx=0).stop 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 msgr2=0x7fdc5c1980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1980d0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdc5000e9c0 tx=0x7fdc5000ee80 comp rx=0 tx=0).stop 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 shutdown_connections 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc440a2c30 0x7fdc440a50f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc5c107710 0x7fdc5c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc5c104de0 0x7fdc5c1980d0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 --2- 172.21.15.52:0/2641335592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc5c1024b0 0x7fdc5c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 >> 172.21.15.52:0/2641335592 conn(0x7fdc5c0fbea0 msgr2=0x7fdc5c102df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 shutdown_connections 2024-06-02T23:15:17.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:17.261+0000 7fdc62a6b640 1 -- 172.21.15.52:0/2641335592 wait complete. 2024-06-02T23:15:17.362 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:17 smithi052 ceph-mon[32340]: pgmap v533: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-06-02T23:15:17.363 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:17 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3738901594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:15:17.390 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 5251b677 --release --await' 2024-06-02T23:15:17.390 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 5251b677 --release --await 2024-06-02T23:15:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:17 smithi123 ceph-mon[35977]: pgmap v533: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-06-02T23:15:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:17 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3738901594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:15:17.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 26 sec latency 26014.032 ms 2024-06-02T23:15:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:17 smithi174 ceph-mon[35994]: pgmap v533: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 5.9 KiB/s wr, 15 op/s 2024-06-02T23:15:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:17 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3738901594' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:15:18.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 27 sec latency 27014.133 ms 2024-06-02T23:15:18.943 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:18.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 -- 172.21.15.52:0/2616693772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc107840 msgr2=0x7f4afc10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 --2- 172.21.15.52:0/2616693772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc107840 0x7f4afc10f3c0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f4af400b0d0 tx=0x7f4af400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 -- 172.21.15.52:0/2616693772 shutdown_connections 2024-06-02T23:15:19.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 --2- 172.21.15.52:0/2616693772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc107840 0x7f4afc10f3c0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 --2- 172.21.15.52:0/2616693772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 0x7f4afc102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 --2- 172.21.15.52:0/2616693772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc1030f0 0x7f4afc107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 -- 172.21.15.52:0/2616693772 >> 172.21.15.52:0/2616693772 conn(0x7f4afc0795f0 msgr2=0x7f4afc079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:19.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 -- 172.21.15.52:0/2616693772 shutdown_connections 2024-06-02T23:15:19.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.947+0000 7f4b02230700 1 -- 172.21.15.52:0/2616693772 wait complete. 2024-06-02T23:15:19.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.948+0000 7f4b02230700 1 Processor -- start 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 -- start start 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 0x7f4afc1a08d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 0x7f4afc1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc107840 0x7f4afc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc112630 con 0x7f4afc1030f0 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1124b0 con 0x7f4afc107840 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4b02230700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1127b0 con 0x7f4afc102740 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 0x7f4afc1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 0x7f4afc1a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45626/0 (socket says 172.21.15.52:45626) 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4afaffd700 1 -- 172.21.15.52:0/2360776678 learned_addr learned my addr 172.21.15.52:0/2360776678 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 0x7f4afc1a08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.949+0000 7f4afbfff700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc107840 0x7f4afc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afaffd700 1 -- 172.21.15.52:0/2360776678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 msgr2=0x7f4afc1a08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 0x7f4afc1a08d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afaffd700 1 -- 172.21.15.52:0/2360776678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc107840 msgr2=0x7f4afc19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc107840 0x7f4afc19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afaffd700 1 -- 172.21.15.52:0/2360776678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4afc19b1f0 con 0x7f4afc1030f0 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 0x7f4afc1a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afbfff700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc107840 0x7f4afc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 0x7f4afc1a0e10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4af000e9a0 tx=0x7f4af000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4af8ff9700 1 -- 172.21.15.52:0/2360776678 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af000ca10 con 0x7f4afc1030f0 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4afc19b4e0 con 0x7f4afc1030f0 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4af8ff9700 1 -- 172.21.15.52:0/2360776678 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af000cbb0 con 0x7f4afc1030f0 2024-06-02T23:15:19.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4af8ff9700 1 -- 172.21.15.52:0/2360776678 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0018680 con 0x7f4afc1030f0 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.950+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4afc1a77a0 con 0x7f4afc1030f0 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.954+0000 7f4af8ff9700 1 -- 172.21.15.52:0/2360776678 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4af0010430 con 0x7f4afc1030f0 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.954+0000 7f4b0cab3b80 1 Processor -- start 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4af8ff9700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ae40a2c20 0x7f4ae40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 -- start start 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afbfff700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ae40a2c20 0x7f4ae40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4af8ff9700 1 -- 172.21.15.52:0/2360776678 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4af00cf1e0 con 0x7f4afc1030f0 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd40340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd40880 0x55c8afd3a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c8afc2cf60 con 0x55c8afd3ff60 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8afc8b850 con 0x55c8afd40880 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8afbf3460 con 0x55c8afd3a7a0 2024-06-02T23:15:19.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd40340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd40340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45632/0 (socket says 172.21.15.52:45632) 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2186737741 learned_addr learned my addr 172.21.15.52:0/2186737741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afbfff700 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd40880 0x55c8afd3a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.955+0000 7f4afbfff700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ae40a2c20 0x7f4ae40a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009fd0 tx=0x7f4af4009ba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2186737741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3a7a0 msgr2=0x55c8afd3abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2186737741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd40880 msgr2=0x55c8afd3a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd40880 0x55c8afd3a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2186737741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8afbda8d0 con 0x55c8afd3ff60 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd40340 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0115f0 tx=0x7f4aec011900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd766bb15d5acbef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4ae1ffb700 1 -- 172.21.15.52:0/2186737741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec004220 con 0x55c8afd3ff60 2024-06-02T23:15:19.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4ae1ffb700 1 -- 172.21.15.52:0/2186737741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aec0043c0 con 0x55c8afd3ff60 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.956+0000 7f4ae1ffb700 1 -- 172.21.15.52:0/2186737741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec019970 con 0x55c8afd3ff60 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2186737741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 msgr2=0x55c8afd40340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd40340 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0115f0 tx=0x7f4aec011900 comp rx=0 tx=0).stop 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2186737741 shutdown_connections 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd40340 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2186737741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd40880 0x55c8afd3a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2186737741 >> 172.21.15.52:0/2186737741 conn(0x55c8afc67080 msgr2=0x55c8afd35220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2186737741 shutdown_connections 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2186737741 wait complete. 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.957+0000 7f4b0cab3b80 1 -- start start 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 Processor -- start 2024-06-02T23:15:19.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd46590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd40880 0x55c8afd43dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c8afc2bfc0 con 0x55c8afd3ff60 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8afc87810 con 0x55c8afd3a7a0 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8afd47260 con 0x55c8afd40880 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4b0cab3b80 10 client.0 _ll_register_callbacks cb 0x55c8afc215b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd43890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45646/0 (socket says 172.21.15.52:45646) 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4afaffd700 1 -- 172.21.15.52:0/166490457 learned_addr learned my addr 172.21.15.52:0/166490457 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd46590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.958+0000 7f4afbfff700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd40880 0x55c8afd43dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afaffd700 1 -- 172.21.15.52:0/166490457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd40880 msgr2=0x55c8afd43dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afaffd700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd40880 0x55c8afd43dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afaffd700 1 -- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 msgr2=0x55c8afd46590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afaffd700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd46590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afaffd700 1 -- 172.21.15.52:0/166490457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8afc51750 con 0x55c8afd3ff60 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd46590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4afaffd700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd43890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4af00ccd10 tx=0x7f4af0010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30fb36cbb5a8aedd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4adaffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4adaffd700 1 -- 172.21.15.52:0/166490457 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00b4060 con 0x55c8afd3ff60 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4adaffd700 1 -- 172.21.15.52:0/166490457 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4af000cd60 con 0x55c8afd3ff60 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aec0043c0 con 0x55c8afd3ff60 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4adaffd700 1 -- 172.21.15.52:0/166490457 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4af009fe20 con 0x55c8afd3ff60 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4b0cab3b80 10 client.16620 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8afd51a60 con 0x55c8afd3ff60 2024-06-02T23:15:19.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.959+0000 7f4adaffd700 1 -- 172.21.15.52:0/166490457 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4aec0043c0 con 0x55c8afd3ff60 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4adaffd700 1 -- 172.21.15.52:0/166490457 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f4af009aec0 con 0x55c8afd3ff60 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4b0cab3b80 10 client.16620 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4b0cab3b80 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4b0cab3b80 20 client.16620 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afd53ff0 0x55c8afd56420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4b0cab3b80 4 client.16620 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8afc98d50 con 0x55c8afd53ff0 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.960+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afd53ff0 0x55c8afd56420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.961+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afd53ff0 0x55c8afd56420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e07774a803608ad server_cookie=b7505acc9bb3add6 in_seq=0 out_seq=0 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.961+0000 7f4adaffd700 10 client.16620 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.961+0000 7f4adaffd700 1 -- 172.21.15.52:0/166490457 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c8afc98d50 con 0x55c8afd53ff0 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.961+0000 7f4adaffd700 10 client.16620 handle_command_reply: tid=0 2024-06-02T23:15:19.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.961+0000 7f4b0cab3b80 1 client.16620 shutdown 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 msgr2=0x55c8afd43890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd43890 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f4af00ccd10 tx=0x7f4af0010a00 comp rx=0 tx=0).stop 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 shutdown_connections 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afd53ff0 0x55c8afd56420 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd40880 0x55c8afd43dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd46590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/166490457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd3ff60 0x55c8afd43890 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4adaffd700 0 client.16620 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 >> 172.21.15.52:0/166490457 conn(0x55c8afc67080 msgr2=0x55c8afd34700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 shutdown_connections 2024-06-02T23:15:19.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/166490457 wait complete. 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.962+0000 7f4b0cab3b80 20 client.16620 trim_cache size 0 max 16384 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 Processor -- start 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 -- start start 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd34e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1aeaf0 con 0x55c8afd40880 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1b06c0 con 0x55c8afd3a7a0 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1af1c0 con 0x55c8afd3ff60 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd34e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4afaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd34e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53366/0 (socket says 172.21.15.52:53366) 2024-06-02T23:15:19.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4afaffd700 1 -- 172.21.15.52:0/800234509 learned_addr learned my addr 172.21.15.52:0/800234509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.978+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afbfff700 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afaffd700 1 -- 172.21.15.52:0/800234509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 msgr2=0x55c8afd3ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afaffd700 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afaffd700 1 -- 172.21.15.52:0/800234509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 msgr2=0x55c8afd353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afaffd700 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afaffd700 1 -- 172.21.15.52:0/800234509 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8afd41e00 con 0x55c8afd3ff60 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4afaffd700 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd34e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4af009ba00 tx=0x7f4af0099f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a271883a8d8a043a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4ae17fa700 1 -- 172.21.15.52:0/800234509 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af0097ac0 con 0x55c8afd3ff60 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4ae17fa700 1 -- 172.21.15.52:0/800234509 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af00977c0 con 0x55c8afd3ff60 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.979+0000 7f4ae17fa700 1 -- 172.21.15.52:0/800234509 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4af00a44b0 con 0x55c8afd3ff60 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/800234509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 msgr2=0x55c8afd34e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd34e80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4af009ba00 tx=0x7f4af0099f80 comp rx=0 tx=0).stop 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/800234509 shutdown_connections 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd3ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/800234509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd34e80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/800234509 >> 172.21.15.52:0/800234509 conn(0x55c8afc67080 msgr2=0x55c8afd42180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/800234509 shutdown_connections 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/800234509 wait complete. 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.980+0000 7f4b0cab3b80 1 -- start start 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 Processor -- start 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd4c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd4ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd5aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1aff60 con 0x55c8afd40880 2024-06-02T23:15:19.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1af890 con 0x55c8afd3a7a0 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4afc1acab0 con 0x55c8afd3ff60 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4b0cab3b80 10 client.0 _ll_register_callbacks cb 0x55c8afc215b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4afbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd5aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4afbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd5aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45672/0 (socket says 172.21.15.52:45672) 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4afb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd4c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4afbfff700 1 -- 172.21.15.52:0/2141389146 learned_addr learned my addr 172.21.15.52:0/2141389146 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.981+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd4ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2141389146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 msgr2=0x55c8afd4ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd4ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2141389146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 msgr2=0x55c8afd5aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd5aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd4ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:19.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afb7fe700 1 -- 172.21.15.52:0/2141389146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8afd5b4a0 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afbfff700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd5aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4afb7fe700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd4c550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005920 tx=0x7f4aec009440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6a7e73797ddfb46 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4adaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4adaffd700 1 -- 172.21.15.52:0/2141389146 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aec003680 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4af00977c0 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4adaffd700 1 -- 172.21.15.52:0/2141389146 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4aec003820 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4adaffd700 1 -- 172.21.15.52:0/2141389146 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f4aec03a070 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4b0cab3b80 10 client.26162 fetch_fsmap learned FSMap version 30 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.982+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55c8afdc6d90 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4adaffd700 1 -- 172.21.15.52:0/2141389146 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4af00977c0 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4adaffd700 1 -- 172.21.15.52:0/2141389146 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 30) ==== 8366+0+0 (secure 0 0 0) 0x7f4aec019a40 con 0x55c8afd3a7a0 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4b0cab3b80 10 client.26162 fetch_fsmap finished waiting for FSMap version 30 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4b0cab3b80 10 client.26162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:19.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4b0cab3b80 20 client.26162 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afdc7340 0x55c8afdc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4b0cab3b80 4 client.26162 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8afc932d0 con 0x55c8afdc7340 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afdc7340 0x55c8afdc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4afaffd700 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afdc7340 0x55c8afdc9760 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9323ad65ca11097 server_cookie=45afe144251e37bc in_seq=0 out_seq=0 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.983+0000 7f4adaffd700 10 client.26162 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4adaffd700 1 -- 172.21.15.52:0/2141389146 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8afc932d0 con 0x55c8afdc7340 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4adaffd700 10 client.26162 handle_command_reply: tid=0 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 client.26162 shutdown 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 msgr2=0x55c8afd4c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd4c550 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005920 tx=0x7f4aec009440 comp rx=0 tx=0).stop 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 shutdown_connections 2024-06-02T23:15:19.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8afdc7340 0x55c8afdc9760 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8afd40880 0x55c8afd5aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8afd3a7a0 0x55c8afd4c550 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 --2- 172.21.15.52:0/2141389146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8afd3ff60 0x55c8afd4ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 >> 172.21.15.52:0/2141389146 conn(0x55c8afc67080 msgr2=0x55c8afd40c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4adaffd700 0 client.26162 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 shutdown_connections 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 1 -- 172.21.15.52:0/2141389146 wait complete. 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.984+0000 7f4b0cab3b80 20 client.26162 trim_cache size 0 max 16384 2024-06-02T23:15:19.049 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:15:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ae40a2c20 msgr2=0x7f4ae40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ae40a2c20 0x7f4ae40a50e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4af4009fd0 tx=0x7f4af4009ba0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 msgr2=0x7f4afc1a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 0x7f4afc1a0e10 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f4af000e9a0 tx=0x7f4af000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 shutdown_connections 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ae40a2c20 0x7f4ae40a50e0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4afc107840 0x7f4afc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4afc102740 0x7f4afc1a08d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 --2- 172.21.15.52:0/2360776678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4afc1030f0 0x7f4afc1a0e10 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 >> 172.21.15.52:0/2360776678 conn(0x7f4afc0795f0 msgr2=0x7f4afc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 shutdown_connections 2024-06-02T23:15:19.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:18.985+0000 7f4b02230700 1 -- 172.21.15.52:0/2360776678 wait complete. 2024-06-02T23:15:19.201 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:19 smithi052 ceph-mon[32340]: pgmap v534: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-06-02T23:15:19.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:19 smithi123 ceph-mon[35977]: pgmap v534: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-06-02T23:15:19.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 28 sec latency 28014.249 ms 2024-06-02T23:15:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:19 smithi174 ceph-mon[35994]: pgmap v534: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-06-02T23:15:20.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48470 0.00 MB/sec execute 29 sec latency 29014.340 ms 2024-06-02T23:15:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.361+0000 7f8e3afbe700 1 -- 172.21.15.52:0/4075915845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 msgr2=0x7f8e34107800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.361+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/4075915845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34107800 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f8e24009990 tx=0x7f8e24009ca0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 -- 172.21.15.52:0/4075915845 shutdown_connections 2024-06-02T23:15:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/4075915845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e34101d30 0x7f8e3410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/4075915845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34107800 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/4075915845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e34101370 0x7f8e341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 -- 172.21.15.52:0/4075915845 >> 172.21.15.52:0/4075915845 conn(0x7f8e34078340 msgr2=0x7f8e34078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 -- 172.21.15.52:0/4075915845 shutdown_connections 2024-06-02T23:15:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 -- 172.21.15.52:0/4075915845 wait complete. 2024-06-02T23:15:21.363 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:21 smithi052 ceph-mon[32340]: pgmap v535: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 Processor -- start 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.362+0000 7f8e3afbe700 1 -- start start 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3afbe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e34101370 0x7f8e3419e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3afbe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e34101d30 0x7f8e3419ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3afbe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3afbe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e34111560 con 0x7f8e34107420 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3afbe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341113e0 con 0x7f8e34101d30 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3afbe700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341116e0 con 0x7f8e34101370 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45686/0 (socket says 172.21.15.52:45686) 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 -- 172.21.15.52:0/25402287 learned_addr learned my addr 172.21.15.52:0/25402287 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e33fff700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e34101d30 0x7f8e3419ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e34101370 0x7f8e3419e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 -- 172.21.15.52:0/25402287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e34101370 msgr2=0x7f8e3419e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e34101370 0x7f8e3419e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 -- 172.21.15.52:0/25402287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e34101d30 msgr2=0x7f8e3419ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e34101d30 0x7f8e3419ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 -- 172.21.15.52:0/25402287 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e34199170 con 0x7f8e34107420 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e3955b700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34198a10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c007ee0 tx=0x7f8e2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.363+0000 7f8e31ffb700 1 -- 172.21.15.52:0/25402287 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00ddd0 con 0x7f8e34107420 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.364+0000 7f8e31ffb700 1 -- 172.21.15.52:0/25402287 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e2c004020 con 0x7f8e34107420 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.364+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e34199460 con 0x7f8e34107420 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.364+0000 7f8e31ffb700 1 -- 172.21.15.52:0/25402287 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c015660 con 0x7f8e34107420 2024-06-02T23:15:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.364+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e341a56d0 con 0x7f8e34107420 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.366+0000 7f8e31ffb700 1 -- 172.21.15.52:0/25402287 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e2c0041d0 con 0x7f8e34107420 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.367+0000 7f8e31ffb700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e200a2ca0 0x7f8e200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.367+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e200a2ca0 0x7f8e200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.367+0000 7f8e31ffb700 1 -- 172.21.15.52:0/25402287 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e2c0ce9e0 con 0x7f8e34107420 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.367+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e200a2ca0 0x7f8e200a5160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e2400b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 Processor -- start 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 -- start start 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d2f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2fe20 0x557546d29710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 0x557546d2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557546c1c060 con 0x557546d29c50 2024-06-02T23:15:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557546c7a950 con 0x557546d2f500 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e45841b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557546be2560 con 0x557546d2fe20 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 0x557546d2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 0x557546d2a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45694/0 (socket says 172.21.15.52:45694) 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 -- 172.21.15.52:0/4218376350 learned_addr learned my addr 172.21.15.52:0/4218376350 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d2f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e33fff700 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2fe20 0x557546d29710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 -- 172.21.15.52:0/4218376350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2fe20 msgr2=0x557546d29710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2fe20 0x557546d29710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 -- 172.21.15.52:0/4218376350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 msgr2=0x557546d2f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d2f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.368+0000 7f8e3955b700 1 -- 172.21.15.52:0/4218376350 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557546bc99d0 con 0x557546d29c50 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.369+0000 7f8e3955b700 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 0x557546d2a070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c097c40 tx=0x7f8e2c09abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38c263e6e150d34b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.369+0000 7f8e2affd700 1 -- 172.21.15.52:0/4218376350 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c09b270 con 0x557546d29c50 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.369+0000 7f8e2affd700 1 -- 172.21.15.52:0/4218376350 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e2c09f620 con 0x557546d29c50 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.369+0000 7f8e2affd700 1 -- 172.21.15.52:0/4218376350 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004e30 con 0x557546d29c50 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.369+0000 7f8e45841b80 1 -- 172.21.15.52:0/4218376350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 msgr2=0x557546d2a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.369+0000 7f8e45841b80 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 0x557546d2a070 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c097c40 tx=0x7f8e2c09abd0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 -- 172.21.15.52:0/4218376350 shutdown_connections 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d29c50 0x557546d2a070 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d2f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 --2- 172.21.15.52:0/4218376350 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2fe20 0x557546d29710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 -- 172.21.15.52:0/4218376350 >> 172.21.15.52:0/4218376350 conn(0x557546c56180 msgr2=0x557546d24a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 -- 172.21.15.52:0/4218376350 shutdown_connections 2024-06-02T23:15:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 -- 172.21.15.52:0/4218376350 wait complete. 2024-06-02T23:15:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 -- start start 2024-06-02T23:15:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.370+0000 7f8e45841b80 1 Processor -- start 2024-06-02T23:15:21.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d35bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 0x557546d32eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2fe20 0x557546d333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557546c1b0c0 con 0x557546d2f500 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557546c76910 con 0x557546d2fe20 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557546d367a0 con 0x557546d29c50 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e45841b80 10 client.0 _ll_register_callbacks cb 0x557546c106b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 0x557546d32eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 0x557546d32eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45706/0 (socket says 172.21.15.52:45706) 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 -- 172.21.15.52:0/2130246021 learned_addr learned my addr 172.21.15.52:0/2130246021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e3955b700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2fe20 0x557546d333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d35bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 -- 172.21.15.52:0/2130246021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 msgr2=0x557546d35bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d35bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 -- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2fe20 msgr2=0x557546d333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2fe20 0x557546d333f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 -- 172.21.15.52:0/2130246021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557546c40850 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e33fff700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 0x557546d32eb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c011540 tx=0x7f8e1c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1a587b06e4e0fb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.371+0000 7f8e13fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e13fff700 1 -- 172.21.15.52:0/2130246021 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c011e60 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2c09f620 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e13fff700 1 -- 172.21.15.52:0/2130246021 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e1c014650 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e13fff700 1 -- 172.21.15.52:0/2130246021 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e1c024070 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 10 client.16638 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557546d41000 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e13fff700 1 -- 172.21.15.52:0/2130246021 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e2c09f620 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e13fff700 1 -- 172.21.15.52:0/2130246021 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8e1c01d9f0 con 0x557546d2f500 2024-06-02T23:15:21.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 10 client.16638 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 10 client.16638 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 20 client.16638 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546d45ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 4 client.16638 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.372+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557546c87e50 con 0x557546d436e0 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.373+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546d45ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.373+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546d45ad0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=746944c1049e6271 server_cookie=fb22e4d1f3316e71 in_seq=0 out_seq=0 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.373+0000 7f8e13fff700 10 client.16638 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e13fff700 1 -- 172.21.15.52:0/2130246021 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557546c87e50 con 0x557546d436e0 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e13fff700 10 client.16638 handle_command_reply: tid=0 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 client.16638 shutdown 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 msgr2=0x557546d32eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 0x557546d32eb0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c011540 tx=0x7f8e1c009f90 comp rx=0 tx=0).stop 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 shutdown_connections 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546d45ad0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e13fff700 0 client.16638 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2fe20 0x557546d333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d35bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 --2- 172.21.15.52:0/2130246021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2f500 0x557546d32eb0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 >> 172.21.15.52:0/2130246021 conn(0x557546c56180 msgr2=0x557546d23bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 shutdown_connections 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 1 -- 172.21.15.52:0/2130246021 wait complete. 2024-06-02T23:15:21.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.374+0000 7f8e45841b80 20 client.16638 trim_cache size 0 max 16384 2024-06-02T23:15:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.394+0000 7f8e45841b80 1 Processor -- start 2024-06-02T23:15:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.394+0000 7f8e45841b80 1 -- start start 2024-06-02T23:15:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.394+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d29c50 0x557546d309c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.394+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2f500 0x557546d356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d24330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e45841b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341acad0 con 0x557546d2fe20 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e45841b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ae670 con 0x557546d29c50 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e45841b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ad1a0 con 0x557546d2f500 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d24330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d24330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45712/0 (socket says 172.21.15.52:45712) 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 -- 172.21.15.52:0/1342021073 learned_addr learned my addr 172.21.15.52:0/1342021073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d29c50 0x557546d309c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 -- 172.21.15.52:0/1342021073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2f500 msgr2=0x557546d356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2f500 0x557546d356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 -- 172.21.15.52:0/1342021073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d29c50 msgr2=0x557546d309c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d29c50 0x557546d309c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 -- 172.21.15.52:0/1342021073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557546d1c4a0 con 0x557546d2fe20 2024-06-02T23:15:21.396 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e3955b700 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d24330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c09e040 tx=0x7f8e2c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fe2ed1777185f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:21.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e2a7fc700 1 -- 172.21.15.52:0/1342021073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0b4230 con 0x557546d2fe20 2024-06-02T23:15:21.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e2a7fc700 1 -- 172.21.15.52:0/1342021073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e2c09b270 con 0x557546d2fe20 2024-06-02T23:15:21.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.395+0000 7f8e2a7fc700 1 -- 172.21.15.52:0/1342021073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004e30 con 0x557546d2fe20 2024-06-02T23:15:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 -- 172.21.15.52:0/1342021073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 msgr2=0x557546d24330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d24330 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c09e040 tx=0x7f8e2c009570 comp rx=0 tx=0).stop 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 -- 172.21.15.52:0/1342021073 shutdown_connections 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d24330 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d29c50 0x557546d309c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1342021073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d2f500 0x557546d356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 -- 172.21.15.52:0/1342021073 >> 172.21.15.52:0/1342021073 conn(0x557546c56180 msgr2=0x557546d32ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 -- 172.21.15.52:0/1342021073 shutdown_connections 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 -- 172.21.15.52:0/1342021073 wait complete. 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.396+0000 7f8e45841b80 1 -- start start 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 Processor -- start 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d3bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d3c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341adf40 con 0x557546d2fe20 2024-06-02T23:15:21.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341ad870 con 0x557546d2f500 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e341aaa70 con 0x557546d29c50 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e45841b80 10 client.0 _ll_register_callbacks cb 0x557546d32320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d38a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45728/0 (socket says 172.21.15.52:45728) 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 -- 172.21.15.52:0/1552753677 learned_addr learned my addr 172.21.15.52:0/1552753677 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d3bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e33fff700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d3c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 -- 172.21.15.52:0/1552753677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 msgr2=0x557546d3bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d3bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 -- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 msgr2=0x557546d3c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d3c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.397+0000 7f8e3955b700 1 -- 172.21.15.52:0/1552753677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557546d392d0 con 0x557546d2fe20 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e3955b700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d38a90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0a39b0 tx=0x7f8e2c0cc560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c588ccb59a4f6813 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:21.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e13fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e13fff700 1 -- 172.21.15.52:0/1552753677 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c0b4230 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2c09b270 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e13fff700 1 -- 172.21.15.52:0/1552753677 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e2c09b4d0 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e13fff700 1 -- 172.21.15.52:0/1552753677 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f8e2c09b6d0 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 10 client.16644 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557546db3350 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e13fff700 1 -- 172.21.15.52:0/1552753677 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e2c09b270 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e13fff700 1 -- 172.21.15.52:0/1552753677 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f8e2c09f120 con 0x557546d2fe20 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 10 client.16644 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 10 client.16644 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 20 client.16644 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546db5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:21.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 4 client.16644 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5251b677", "release": true, "await": true}] 2024-06-02T23:15:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.398+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5251b677", "release": true, "await": true}) -- 0x557546c823d0 con 0x557546d436e0 2024-06-02T23:15:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.399+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546db5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.399+0000 7f8e38d5a700 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546db5590 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72c5a114ce900d0b server_cookie=ca59d6e276f16506 in_seq=0 out_seq=0 2024-06-02T23:15:21.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.399+0000 7f8e13fff700 10 client.16644 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e13fff700 1 -- 172.21.15.52:0/1552753677 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x557546c823d0 con 0x557546d436e0 2024-06-02T23:15:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e13fff700 10 client.16644 handle_command_reply: tid=0 2024-06-02T23:15:21.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 client.16644 shutdown 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 msgr2=0x557546d38a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d38a90 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0a39b0 tx=0x7f8e2c0cc560 comp rx=0 tx=0).stop 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 shutdown_connections 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557546d436e0 0x557546db5590 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557546d2fe20 0x557546d38a90 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557546d29c50 0x557546d3bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 --2- 172.21.15.52:0/1552753677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557546d2f500 0x557546d3c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 >> 172.21.15.52:0/1552753677 conn(0x557546c56180 msgr2=0x557546d28700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e13fff700 0 client.16644 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 shutdown_connections 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 1 -- 172.21.15.52:0/1552753677 wait complete. 2024-06-02T23:15:21.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.405+0000 7f8e45841b80 20 client.16644 trim_cache size 0 max 16384 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 31, 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 16, 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "5251b677": { 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 16, 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:15:21.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.8, 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:21.408 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e200a2ca0 msgr2=0x7f8e200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e200a2ca0 0x7f8e200a5160 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8e24006010 tx=0x7f8e2400b540 comp rx=0 tx=0).stop 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 msgr2=0x7f8e34198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34198a10 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c007ee0 tx=0x7f8e2c00d3b0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 shutdown_connections 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e200a2ca0 0x7f8e200a5160 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e34107420 0x7f8e34198a10 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e34101370 0x7f8e3419e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 --2- 172.21.15.52:0/25402287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e34101d30 0x7f8e3419ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:21.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 >> 172.21.15.52:0/25402287 conn(0x7f8e34078340 msgr2=0x7f8e34105750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:21.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 shutdown_connections 2024-06-02T23:15:21.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:21.406+0000 7f8e3afbe700 1 -- 172.21.15.52:0/25402287 wait complete. 2024-06-02T23:15:21.493 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:15:21.493 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:15:21.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:21 smithi123 ceph-mon[35977]: pgmap v535: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:15:21.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 48532 0.00 MB/sec execute 30 sec latency 29791.138 ms 2024-06-02T23:15:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:21 smithi174 ceph-mon[35994]: pgmap v535: 97 pgs: 97 active+clean; 36 MiB data, 611 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.6 KiB/s wr, 11 op/s 2024-06-02T23:15:22.144 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5251b677', total seconds elapsed: 37.1 2024-06-02T23:15:22.204 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:22 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:15:22.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:22 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:15:22.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 49139 0.18 MB/sec execute 31 sec latency 14.812 ms 2024-06-02T23:15:22.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:22 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:15:23.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:23 smithi052 ceph-mon[32340]: pgmap v536: 97 pgs: 97 active+clean; 36 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-06-02T23:15:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:23 smithi123 ceph-mon[35977]: pgmap v536: 97 pgs: 97 active+clean; 36 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-06-02T23:15:23.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 49688 0.33 MB/sec execute 32 sec latency 14.901 ms 2024-06-02T23:15:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:23 smithi174 ceph-mon[35994]: pgmap v536: 97 pgs: 97 active+clean; 36 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 4.0 KiB/s wr, 13 op/s 2024-06-02T23:15:24.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 -- 172.21.15.52:0/1381265692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406b5a0 msgr2=0x7efe0410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 --2- 172.21.15.52:0/1381265692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406b5a0 0x7efe0410f320 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efdfc00b0a0 tx=0x7efdfc00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 -- 172.21.15.52:0/1381265692 shutdown_connections 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 --2- 172.21.15.52:0/1381265692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406b5a0 0x7efe0410f320 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 --2- 172.21.15.52:0/1381265692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04108500 0x7efe041088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 --2- 172.21.15.52:0/1381265692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406abe0 0x7efe0406b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 -- 172.21.15.52:0/1381265692 >> 172.21.15.52:0/1381265692 conn(0x7efe04078360 msgr2=0x7efe04078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 -- 172.21.15.52:0/1381265692 shutdown_connections 2024-06-02T23:15:24.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.105+0000 7efe095c7700 1 -- 172.21.15.52:0/1381265692 wait complete. 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 Processor -- start 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 -- start start 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 0x7efe04101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406b5a0 0x7efe041019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04108500 0x7efe04101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe04112630 con 0x7efe0406abe0 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe041124b0 con 0x7efe04108500 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe095c7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe041127b0 con 0x7efe0406b5a0 2024-06-02T23:15:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 0x7efe04101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 0x7efe04101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36222/0 (socket says 172.21.15.52:36222) 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.106+0000 7efe02ffd700 1 -- 172.21.15.52:0/2144495644 learned_addr learned my addr 172.21.15.52:0/2144495644 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe037fe700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04108500 0x7efe04101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe027fc700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406b5a0 0x7efe041019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe02ffd700 1 -- 172.21.15.52:0/2144495644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406b5a0 msgr2=0x7efe041019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe02ffd700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406b5a0 0x7efe041019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe02ffd700 1 -- 172.21.15.52:0/2144495644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04108500 msgr2=0x7efe04101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe02ffd700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04108500 0x7efe04101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe02ffd700 1 -- 172.21.15.52:0/2144495644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe041a7000 con 0x7efe0406abe0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe027fc700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406b5a0 0x7efe041019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe02ffd700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 0x7efe04101490 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efdf400e9a0 tx=0x7efdf400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efdebfff700 1 -- 172.21.15.52:0/2144495644 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf400cbd0 con 0x7efe0406abe0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe041a72f0 con 0x7efe0406abe0 2024-06-02T23:15:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efdebfff700 1 -- 172.21.15.52:0/2144495644 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf400cd70 con 0x7efe0406abe0 2024-06-02T23:15:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efdebfff700 1 -- 172.21.15.52:0/2144495644 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf4018680 con 0x7efe0406abe0 2024-06-02T23:15:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.107+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe041a7880 con 0x7efe0406abe0 2024-06-02T23:15:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.110+0000 7efdebfff700 1 -- 172.21.15.52:0/2144495644 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efdf40188e0 con 0x7efe0406abe0 2024-06-02T23:15:24.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.110+0000 7efdebfff700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdec0a2c20 0x7efdec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe027fc700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdec0a2c20 0x7efdec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efdebfff700 1 -- 172.21.15.52:0/2144495644 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efdf40cf280 con 0x7efe0406abe0 2024-06-02T23:15:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe027fc700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdec0a2c20 0x7efdec0a50e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe04078960 tx=0x7efdf8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe13e4ab80 1 Processor -- start 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe13e4ab80 1 -- start start 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 0x55a01bc45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.111+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc463c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a01bb97890 con 0x55a01bc45fa0 2024-06-02T23:15:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a01bb3f7a0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a01bb93850 con 0x55a01bc4b6c0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 0x55a01bc45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 0x55a01bc45a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44482/0 (socket says 172.21.15.52:44482) 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 -- 172.21.15.52:0/2093425457 learned_addr learned my addr 172.21.15.52:0/2093425457 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe037fe700 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe02ffd700 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 -- 172.21.15.52:0/2093425457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 msgr2=0x55a01bc4baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 -- 172.21.15.52:0/2093425457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 msgr2=0x55a01bc463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 -- 172.21.15.52:0/2093425457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01bb35fd0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efe027fc700 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 0x55a01bc45a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdf8011f80 tx=0x7efdf8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4191f771a5d1e52a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.112+0000 7efde97fa700 1 -- 172.21.15.52:0/2093425457 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8012dc0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efde97fa700 1 -- 172.21.15.52:0/2093425457 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf801f8d0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efde97fa700 1 -- 172.21.15.52:0/2093425457 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf801d720 con 0x55a01bc4bfe0 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 -- 172.21.15.52:0/2093425457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 msgr2=0x55a01bc45a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 0x55a01bc45a60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7efdf8011f80 tx=0x7efdf8014000 comp rx=0 tx=0).stop 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 -- 172.21.15.52:0/2093425457 shutdown_connections 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/2093425457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4bfe0 0x55a01bc45a60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 -- 172.21.15.52:0/2093425457 >> 172.21.15.52:0/2093425457 conn(0x55a01bb70d40 msgr2=0x55a01bc408d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 -- 172.21.15.52:0/2093425457 shutdown_connections 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.113+0000 7efe13e4ab80 1 -- 172.21.15.52:0/2093425457 wait complete. 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 -- start start 2024-06-02T23:15:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 Processor -- start 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc51df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4bfe0 0x55a01bc4f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a01bc52a50 con 0x55a01bc45fa0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a01bb37c70 con 0x55a01bc4b6c0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a01bc52bd0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe13e4ab80 10 client.0 _ll_register_callbacks cb 0x55a01bb319a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe027fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe027fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44486/0 (socket says 172.21.15.52:44486) 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe027fc700 1 -- 172.21.15.52:0/3566870943 learned_addr learned my addr 172.21.15.52:0/3566870943 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.114+0000 7efe037fe700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4bfe0 0x55a01bc4f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc51df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe027fc700 1 -- 172.21.15.52:0/3566870943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4bfe0 msgr2=0x55a01bc4f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe027fc700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4bfe0 0x55a01bc4f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe027fc700 1 -- 172.21.15.52:0/3566870943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 msgr2=0x55a01bc51df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe027fc700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc51df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe027fc700 1 -- 172.21.15.52:0/3566870943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01bae6950 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc51df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe027fc700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4f0f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efdf801f680 tx=0x7efdf801f6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d21e45a03b9cc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efde6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efde6ff5700 1 -- 172.21.15.52:0/3566870943 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf8036060 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efde6ff5700 1 -- 172.21.15.52:0/3566870943 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efdf801fc90 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf801f8d0 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efde6ff5700 1 -- 172.21.15.52:0/3566870943 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efdf8005050 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe13e4ab80 10 client.26192 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.115+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a01bc5d3b0 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efde6ff5700 1 -- 172.21.15.52:0/3566870943 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdf801f8d0 con 0x55a01bc4b6c0 2024-06-02T23:15:24.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efde6ff5700 1 -- 172.21.15.52:0/3566870943 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7efdf8031470 con 0x55a01bc4b6c0 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe13e4ab80 10 client.26192 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe13e4ab80 10 client.26192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe13e4ab80 20 client.26192 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bc5f930 0x55a01bc61d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe13e4ab80 4 client.26192 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a01bba4d90 con 0x55a01bc5f930 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bc5f930 0x55a01bc61d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bc5f930 0x55a01bc61d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7162ee1866711922 server_cookie=6aecf31828e542dc in_seq=0 out_seq=0 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.116+0000 7efde6ff5700 10 client.26192 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.117+0000 7efde6ff5700 1 -- 172.21.15.52:0/3566870943 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a01bba4d90 con 0x55a01bc5f930 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.117+0000 7efde6ff5700 10 client.26192 handle_command_reply: tid=0 2024-06-02T23:15:24.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.117+0000 7efe13e4ab80 1 client.26192 shutdown 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 msgr2=0x55a01bc4f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4f0f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7efdf801f680 tx=0x7efdf801f6b0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 shutdown_connections 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bc5f930 0x55a01bc61d20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4bfe0 0x55a01bc4f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc45fa0 0x55a01bc51df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3566870943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4f0f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efde6ff5700 0 client.26192 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 >> 172.21.15.52:0/3566870943 conn(0x55a01bb70d40 msgr2=0x55a01bc40670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 shutdown_connections 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3566870943 wait complete. 2024-06-02T23:15:24.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.118+0000 7efe13e4ab80 20 client.26192 trim_cache size 0 max 16384 2024-06-02T23:15:24.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 Processor -- start 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 -- start start 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc45a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc3a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe041aed30 con 0x55a01bc4bfe0 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b08d0 con 0x55a01bc45fa0 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe041af400 con 0x55a01bc4b6c0 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc3a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc3a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36246/0 (socket says 172.21.15.52:36246) 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe037fe700 1 -- 172.21.15.52:0/99542603 learned_addr learned my addr 172.21.15.52:0/99542603 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.140+0000 7efe027fc700 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe02ffd700 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc45a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe037fe700 1 -- 172.21.15.52:0/99542603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 msgr2=0x55a01bc4cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe037fe700 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe037fe700 1 -- 172.21.15.52:0/99542603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 msgr2=0x55a01bc45a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe037fe700 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc45a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe037fe700 1 -- 172.21.15.52:0/99542603 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a01bc4bcf0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efe037fe700 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc3a590 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efdfc015000 tx=0x7efdfc009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7151f983b48f492 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efde8ff9700 1 -- 172.21.15.52:0/99542603 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc00bab0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efde8ff9700 1 -- 172.21.15.52:0/99542603 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdfc00bc50 con 0x55a01bc4bfe0 2024-06-02T23:15:24.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.141+0000 7efde8ff9700 1 -- 172.21.15.52:0/99542603 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc007c90 con 0x55a01bc4bfe0 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 -- 172.21.15.52:0/99542603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 msgr2=0x55a01bc3a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc3a590 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7efdfc015000 tx=0x7efdfc009520 comp rx=0 tx=0).stop 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 -- 172.21.15.52:0/99542603 shutdown_connections 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc3a590 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc45a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/99542603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc4cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 -- 172.21.15.52:0/99542603 >> 172.21.15.52:0/99542603 conn(0x55a01bb70d40 msgr2=0x55a01bc39e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 -- 172.21.15.52:0/99542603 shutdown_connections 2024-06-02T23:15:24.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.142+0000 7efe13e4ab80 1 -- 172.21.15.52:0/99542603 wait complete. 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 -- start start 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 Processor -- start 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc586b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc54e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc55440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efe041b01a0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efe041afad0 con 0x55a01bc45fa0 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe041accf0 con 0x55a01bc4b6c0 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe13e4ab80 10 client.0 _ll_register_callbacks cb 0x55a01bb319a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc55440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe02ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc586b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc55440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36248/0 (socket says 172.21.15.52:36248) 2024-06-02T23:15:24.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 -- 172.21.15.52:0/3842015958 learned_addr learned my addr 172.21.15.52:0/3842015958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe027fc700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc54e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 -- 172.21.15.52:0/3842015958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 msgr2=0x55a01bc54e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc54e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 -- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 msgr2=0x55a01bc586b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc586b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe037fe700 1 -- 172.21.15.52:0/3842015958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a01bcca300 con 0x55a01bc4bfe0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.143+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efe037fe700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc55440 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009590 tx=0x7efdfc00be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=492d6417615846f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efde6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efde6ff5700 1 -- 172.21.15.52:0/3842015958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdfc0074a0 con 0x55a01bc4bfe0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efde6ff5700 1 -- 172.21.15.52:0/3842015958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efdfc003c10 con 0x55a01bc4bfe0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdfc00bc50 con 0x55a01bc4bfe0 2024-06-02T23:15:24.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efde6ff5700 1 -- 172.21.15.52:0/3842015958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efdfc006000 con 0x55a01bc4bfe0 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efe13e4ab80 10 client.16662 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55a01bccb460 con 0x55a01bc4bfe0 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.144+0000 7efde6ff5700 1 -- 172.21.15.52:0/3842015958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efdfc00bc50 con 0x55a01bc4bfe0 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efde6ff5700 1 -- 172.21.15.52:0/3842015958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7efdfc011070 con 0x55a01bc4bfe0 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe13e4ab80 10 client.16662 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe13e4ab80 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe13e4ab80 20 client.16662 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bccba50 0x55a01bccde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe13e4ab80 4 client.16662 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a01bb9f310 con 0x55a01bccba50 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bccba50 0x55a01bccde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efde6ff5700 10 client.16662 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:24.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.145+0000 7efe02ffd700 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bccba50 0x55a01bccde40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=794229b5e879f53c server_cookie=1eec84f1786663ae in_seq=0 out_seq=0 2024-06-02T23:15:24.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.146+0000 7efde6ff5700 1 -- 172.21.15.52:0/3842015958 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a01bb9f310 con 0x55a01bccba50 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.146+0000 7efde6ff5700 10 client.16662 handle_command_reply: tid=0 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.146+0000 7efe13e4ab80 1 client.16662 shutdown 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 msgr2=0x55a01bc55440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc55440 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7efdfc009590 tx=0x7efdfc00be40 comp rx=0 tx=0).stop 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 shutdown_connections 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a01bccba50 0x55a01bccde40 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a01bc4bfe0 0x55a01bc55440 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a01bc45fa0 0x55a01bc586b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 --2- 172.21.15.52:0/3842015958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a01bc4b6c0 0x55a01bc54e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efde6ff5700 0 client.16662 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 >> 172.21.15.52:0/3842015958 conn(0x55a01bb70d40 msgr2=0x55a01bc3a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:24.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 shutdown_connections 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 1 -- 172.21.15.52:0/3842015958 wait complete. 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.147+0000 7efe13e4ab80 20 client.16662 trim_cache size 0 max 16384 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "3a178d61-b8a5-424c-bbc6-620458c2a8e7", 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:15:24.152 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdec0a2c20 msgr2=0x7efdec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdec0a2c20 0x7efdec0a50e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe04078960 tx=0x7efdf8009260 comp rx=0 tx=0).stop 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 msgr2=0x7efe04101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 0x7efe04101490 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efdf400e9a0 tx=0x7efdf400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 shutdown_connections 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efdec0a2c20 0x7efdec0a50e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe04108500 0x7efe04101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efe0406abe0 0x7efe04101490 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 --2- 172.21.15.52:0/2144495644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe0406b5a0 0x7efe041019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 >> 172.21.15.52:0/2144495644 conn(0x7efe04078360 msgr2=0x7efe04100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:24.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 shutdown_connections 2024-06-02T23:15:24.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:24.148+0000 7efe095c7700 1 -- 172.21.15.52:0/2144495644 wait complete. 2024-06-02T23:15:24.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:15:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:15:24.160+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000208 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD/ auth{2=1} v8346 ap=2 f(v288 m2024-06-02T23:14:50.047317+0000 5=5+0) n(v11 rc2024-06-02T23:14:50.047317+0000 b814326 6=5+1) (inest sync->mix g=2) caps={15216=pAsLsXsFs/-@7198} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb89c3b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000208:[<0x10000000009/WORD v8>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x10000000003/client.0 v6>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v22>,<0x10000000001/sv_0 v45>,<0x10000000000/qa v32>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000208:[<0x10000000009/WORD v8346>,<0x10000000008/~dmtmp v6760>,<0x10000000007/client0 v288>,<0x10000000006/clients v270>,<0x10000000005/tmp v269>,<0x10000000003/client.0 v259>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v265>,<0x10000000001/sv_0 v283>,<0x10000000000/qa v263>,<0x1/volumes v260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:14:50.035533+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:14:50.047317+0000 b814326 6=5+1)","memory_value.dirstat":"f(v288 m2024-06-02T23:14:50.047317+0000 5=5+0)","memory_value.rstat":"n(v11 rc2024-06-02T23:14:50.047317+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:15:24.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:15:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:15:24.165+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v8350 ap=2 f(v420 m2024-06-02T23:14:51.508335+0000 24=24+0) n(v12 rc2024-06-02T23:14:51.508335+0000 b5230239 25=24+1) (inest sync->mix g=1,2,3,4) caps={15216=pAsLsXsFs/-@8635} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ac2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x10000000003/client.0 v6>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v22>,<0x10000000001/sv_0 v45>,<0x10000000000/qa v32>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v8350>,<0x10000000008/~dmtmp v6762>,<0x10000000007/client0 v288>,<0x10000000006/clients v270>,<0x10000000005/tmp v269>,<0x10000000003/client.0 v259>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v265>,<0x10000000001/sv_0 v283>,<0x10000000000/qa v263>,<0x1/volumes v260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:14:51.494132+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:14:51.508335+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v420 m2024-06-02T23:14:51.508335+0000 24=24+0)","memory_value.rstat":"n(v12 rc2024-06-02T23:14:51.508335+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:15:24.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:15:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:15:24.168+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000020f [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PWRPNT/ auth{2=1} v8348 ap=2 f(v336 m2024-06-02T23:14:49.487557+0000 7=7+0) n(v10 rc2024-06-02T23:14:49.487557+0000 b1462582 8=7+1) (inest sync->mix g=2) caps={15216=pAsLsXsFs/-@7165} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ca1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000020f:[<0x10000000009/PWRPNT v230>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x10000000003/client.0 v6>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v22>,<0x10000000001/sv_0 v45>,<0x10000000000/qa v32>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x1000000020f:[<0x10000000009/PWRPNT v8348>,<0x10000000008/~dmtmp v6768>,<0x10000000007/client0 v288>,<0x10000000006/clients v270>,<0x10000000005/tmp v269>,<0x10000000003/client.0 v259>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v265>,<0x10000000001/sv_0 v283>,<0x10000000000/qa v263>,<0x1/volumes v260>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:14:49.461451+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:14:49.487557+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v336 m2024-06-02T23:14:49.487557+0000 7=7+0)","memory_value.rstat":"n(v10 rc2024-06-02T23:14:49.487557+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:15:24.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 50054 0.37 MB/sec execute 33 sec latency 24.172 ms 2024-06-02T23:15:25.061 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:15:25.335 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:25.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: pgmap v537: 97 pgs: 97 active+clean; 36 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: pgmap v537: 97 pgs: 97 active+clean; 36 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.624 INFO:tasks.workunit.client.0.smithi052.stdout: 1 50353 0.36 MB/sec execute 34 sec latency 15.150 ms 2024-06-02T23:15:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: pgmap v537: 97 pgs: 97 active+clean; 36 MiB data, 612 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 6.7 KiB/s wr, 12 op/s 2024-06-02T23:15:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:15:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000020f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:15:25.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:15:26.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 50869 0.41 MB/sec execute 35 sec latency 16.866 ms 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 -- 172.21.15.52:0/841265626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 msgr2=0x7f2bb410e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/841265626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb410e210 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00b0a0 tx=0x7f2bac00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 -- 172.21.15.52:0/841265626 shutdown_connections 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/841265626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb410e210 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/841265626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb41073d0 0x7f2bb41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/841265626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb4101390 0x7f2bb41017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 -- 172.21.15.52:0/841265626 >> 172.21.15.52:0/841265626 conn(0x7f2bb40782c0 msgr2=0x7f2bb40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:27.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 -- 172.21.15.52:0/841265626 shutdown_connections 2024-06-02T23:15:27.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 -- 172.21.15.52:0/841265626 wait complete. 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 Processor -- start 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.133+0000 7f2bba3f0700 1 -- start start 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bba3f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 0x7f2bb4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bba3f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bba3f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41073d0 0x7f2bb4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bba3f0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4111500 con 0x7f2bb4101d30 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bba3f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4111380 con 0x7f2bb41073d0 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bba3f0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb4111680 con 0x7f2bb4101390 2024-06-02T23:15:27.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 0x7f2bb4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb898d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41073d0 0x7f2bb4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 0x7f2bb4103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54906/0 (socket says 172.21.15.52:54906) 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb3fff700 1 -- 172.21.15.52:0/3067260956 learned_addr learned my addr 172.21.15.52:0/3067260956 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb37fe700 1 -- 172.21.15.52:0/3067260956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 msgr2=0x7f2bb4103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 0x7f2bb4103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb37fe700 1 -- 172.21.15.52:0/3067260956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41073d0 msgr2=0x7f2bb4103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41073d0 0x7f2bb4103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb37fe700 1 -- 172.21.15.52:0/3067260956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb41a7070 con 0x7f2bb4101d30 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.134+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 0x7f2bb4103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.135+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb4103580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800d750 tx=0x7f2ba800da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.135+0000 7f2bb17fa700 1 -- 172.21.15.52:0/3067260956 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8014030 con 0x7f2bb4101d30 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.135+0000 7f2bb17fa700 1 -- 172.21.15.52:0/3067260956 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba80044d0 con 0x7f2bb4101d30 2024-06-02T23:15:27.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.135+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb41a7360 con 0x7f2bb4101d30 2024-06-02T23:15:27.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.135+0000 7f2bb17fa700 1 -- 172.21.15.52:0/3067260956 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8005420 con 0x7f2bb4101d30 2024-06-02T23:15:27.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.135+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb41a78a0 con 0x7f2bb4101d30 2024-06-02T23:15:27.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.137+0000 7f2bb17fa700 1 -- 172.21.15.52:0/3067260956 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ba8004750 con 0x7f2bb4101d30 2024-06-02T23:15:27.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.138+0000 7f2bb17fa700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b9c0a2c20 0x7f2b9c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.138+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b9c0a2c20 0x7f2b9c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.139+0000 7f2bb17fa700 1 -- 172.21.15.52:0/3067260956 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ba80ceca0 con 0x7f2bb4101d30 2024-06-02T23:15:27.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.139+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b9c0a2c20 0x7f2b9c0a50e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40104e0 tx=0x7f2ba400b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:27.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.139+0000 7f2bc4c73b80 1 Processor -- start 2024-06-02T23:15:27.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.139+0000 7f2bc4c73b80 1 -- start start 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.139+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbde6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5650dbcd8f60 con 0x5650dbdebf60 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5650dbd37850 con 0x5650dbde67a0 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650dbc9f460 con 0x5650dbdec880 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36270/0 (socket says 172.21.15.52:36270) 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 -- 172.21.15.52:0/2018969831 learned_addr learned my addr 172.21.15.52:0/2018969831 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb898d700 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbde6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 -- 172.21.15.52:0/2018969831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 msgr2=0x5650dbde6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 -- 172.21.15.52:0/2018969831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 msgr2=0x5650dbde6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbde6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 -- 172.21.15.52:0/2018969831 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650dbc868d0 con 0x5650dbdebf60 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.140+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdec340 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40116e0 tx=0x7f2ba4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ba8a49e8afd71cf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.141+0000 7f2b9a7fc700 1 -- 172.21.15.52:0/2018969831 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba401e620 con 0x5650dbdebf60 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.141+0000 7f2b9a7fc700 1 -- 172.21.15.52:0/2018969831 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba401ec70 con 0x5650dbdebf60 2024-06-02T23:15:27.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.141+0000 7f2b9a7fc700 1 -- 172.21.15.52:0/2018969831 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba401cc50 con 0x5650dbdebf60 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.141+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2018969831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 msgr2=0x5650dbdec340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.141+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdec340 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40116e0 tx=0x7f2ba4013000 comp rx=0 tx=0).stop 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2018969831 shutdown_connections 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbde6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdec340 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2018969831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2018969831 >> 172.21.15.52:0/2018969831 conn(0x5650dbd13080 msgr2=0x5650dbde1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:27.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2018969831 shutdown_connections 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2018969831 wait complete. 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.142+0000 7f2bc4c73b80 1 -- start start 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 Processor -- start 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf2560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdef860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbdefda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5650dbcd7fc0 con 0x5650dbdebf60 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5650dbd33810 con 0x5650dbde67a0 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650dbdf3260 con 0x5650dbdec880 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bc4c73b80 10 client.0 _ll_register_callbacks cb 0x5650dbccd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdef860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bb898d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbdefda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bb37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdef860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36272/0 (socket says 172.21.15.52:36272) 2024-06-02T23:15:27.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bb898d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbdefda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54928/0 (socket says 172.21.15.52:54928) 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bb37fe700 1 -- 172.21.15.52:0/2407704659 learned_addr learned my addr 172.21.15.52:0/2407704659 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.143+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb898d700 1 -- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 msgr2=0x5650dbdf2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb898d700 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb898d700 1 -- 172.21.15.52:0/2407704659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 msgr2=0x5650dbdef860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb898d700 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdef860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb898d700 1 -- 172.21.15.52:0/2407704659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650dbcfd750 con 0x5650dbdec880 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bb898d700 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbdefda0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00b070 tx=0x7f2bac004c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11c8cc23e481c24d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:27.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2b937fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2b937fe700 1 -- 172.21.15.52:0/2407704659 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac0078f0 con 0x5650dbdec880 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba401ec70 con 0x5650dbdec880 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2b937fe700 1 -- 172.21.15.52:0/2407704659 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2bac007a90 con 0x5650dbdec880 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.144+0000 7f2b937fe700 1 -- 172.21.15.52:0/2407704659 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2bac007cd0 con 0x5650dbdec880 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 10 client.26080 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5650dbdfd9d0 con 0x5650dbdec880 2024-06-02T23:15:27.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2b937fe700 1 -- 172.21.15.52:0/2407704659 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba401ec70 con 0x5650dbdec880 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2b937fe700 1 -- 172.21.15.52:0/2407704659 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2bac011070 con 0x5650dbdec880 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 10 client.26080 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 10 client.26080 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 20 client.26080 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe00060 0x5650dbe02450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 4 client.26080 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.145+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650dbd44d50 con 0x5650dbe00060 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.146+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe00060 0x5650dbe02450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.146+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe00060 0x5650dbe02450 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f4a80b82b69072d server_cookie=11f02a64951e029a in_seq=0 out_seq=0 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.146+0000 7f2b937fe700 10 client.26080 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2b937fe700 1 -- 172.21.15.52:0/2407704659 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650dbd44d50 con 0x5650dbe00060 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2b937fe700 10 client.26080 handle_command_reply: tid=0 2024-06-02T23:15:27.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 client.26080 shutdown 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 msgr2=0x5650dbdefda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbdefda0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2bac00b070 tx=0x7f2bac004c40 comp rx=0 tx=0).stop 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 shutdown_connections 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe00060 0x5650dbe02450 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbdefda0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2b937fe700 0 client.26080 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/2407704659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdebf60 0x5650dbdef860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 >> 172.21.15.52:0/2407704659 conn(0x5650dbd13080 msgr2=0x5650dbde0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 shutdown_connections 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/2407704659 wait complete. 2024-06-02T23:15:27.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.147+0000 7f2bc4c73b80 20 client.26080 trim_cache size 0 max 16384 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.170+0000 7f2bc4c73b80 1 Processor -- start 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.170+0000 7f2bc4c73b80 1 -- start start 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 0x5650dbde6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbdebf60 0x5650dbdf17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41aa820 con 0x5650dbde67a0 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41ac3c0 con 0x5650dbdebf60 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41aaef0 con 0x5650dbdec880 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 0x5650dbde6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbdebf60 0x5650dbdf17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 0x5650dbde6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36274/0 (socket says 172.21.15.52:36274) 2024-06-02T23:15:27.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb3fff700 1 -- 172.21.15.52:0/221119876 learned_addr learned my addr 172.21.15.52:0/221119876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb898d700 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb37fe700 1 -- 172.21.15.52:0/221119876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 msgr2=0x5650dbde0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb37fe700 1 -- 172.21.15.52:0/221119876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 msgr2=0x5650dbde6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 0x5650dbde6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb37fe700 1 -- 172.21.15.52:0/221119876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650dbdee7c0 con 0x5650dbdebf60 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.171+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 0x5650dbde6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.172+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbdebf60 0x5650dbdf17a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800ddb0 tx=0x7f2ba8097040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5303428f2a01c615 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.172+0000 7f2b99ffb700 1 -- 172.21.15.52:0/221119876 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba80b4070 con 0x5650dbdebf60 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.172+0000 7f2b99ffb700 1 -- 172.21.15.52:0/221119876 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba80997d0 con 0x5650dbdebf60 2024-06-02T23:15:27.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.172+0000 7f2b99ffb700 1 -- 172.21.15.52:0/221119876 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba801d040 con 0x5650dbdebf60 2024-06-02T23:15:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.172+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/221119876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbdebf60 msgr2=0x5650dbdf17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.172+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbdebf60 0x5650dbdf17a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800ddb0 tx=0x7f2ba8097040 comp rx=0 tx=0).stop 2024-06-02T23:15:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/221119876 shutdown_connections 2024-06-02T23:15:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdec880 0x5650dbde0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbde67a0 0x5650dbde6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/221119876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbdebf60 0x5650dbdf17a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/221119876 >> 172.21.15.52:0/221119876 conn(0x5650dbd13080 msgr2=0x5650dbdf1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/221119876 shutdown_connections 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/221119876 wait complete. 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 -- start start 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 Processor -- start 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdebf60 0x5650dbe07490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.173+0000 7f2bc4c73b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdec880 0x5650dbe079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41abc90 con 0x5650dbdec880 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41ab5c0 con 0x5650dbde67a0 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bc4c73b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb419c760 con 0x5650dbdebf60 2024-06-02T23:15:27.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bc4c73b80 10 client.0 _ll_register_callbacks cb 0x5650dbccd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdec880 0x5650dbe079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44542/0 (socket says 172.21.15.52:44542) 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb3fff700 1 -- 172.21.15.52:0/3468344721 learned_addr learned my addr 172.21.15.52:0/3468344721 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdebf60 0x5650dbe07490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 -- 172.21.15.52:0/3468344721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdebf60 msgr2=0x5650dbe07490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdebf60 0x5650dbe07490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 -- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 msgr2=0x5650dbdf8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 -- 172.21.15.52:0/3468344721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650dbe07f90 con 0x5650dbdec880 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb37fe700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdebf60 0x5650dbe07490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2bb898d700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdec880 0x5650dbe079d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2bac028f20 tx=0x7f2bac004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac253f5ad259166 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:27.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2b937fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.174+0000 7f2b937fe700 1 -- 172.21.15.52:0/3468344721 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bac005e70 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ba80997d0 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2b937fe700 1 -- 172.21.15.52:0/3468344721 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2bac006010 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2b937fe700 1 -- 172.21.15.52:0/3468344721 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f2bac043050 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 10 client.16680 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5650dbe6aad0 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2b937fe700 1 -- 172.21.15.52:0/3468344721 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba80997d0 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2b937fe700 1 -- 172.21.15.52:0/3468344721 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7f2bac011040 con 0x5650dbdec880 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 10 client.16680 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 10 client.16680 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 20 client.16680 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe6b1b0 0x5650dbe6d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 4 client.16680 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:15:27.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.175+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5650dbd3f2d0 con 0x5650dbe6b1b0 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe6b1b0 0x5650dbe6d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2bb3fff700 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe6b1b0 0x5650dbe6d5d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77bd82681684ad21 server_cookie=4cd0a8fbfacb223c in_seq=0 out_seq=0 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2b937fe700 10 client.16680 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2b937fe700 1 -- 172.21.15.52:0/3468344721 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5650dbd3f2d0 con 0x5650dbe6b1b0 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2b937fe700 10 client.16680 handle_command_reply: tid=0 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2bc4c73b80 1 client.16680 shutdown 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdec880 msgr2=0x5650dbe079d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.176+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdec880 0x5650dbe079d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f2bac028f20 tx=0x7f2bac004d60 comp rx=0 tx=0).stop 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 shutdown_connections 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650dbe6b1b0 0x5650dbe6d5d0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650dbdec880 0x5650dbe079d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650dbde67a0 0x5650dbdf8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 --2- 172.21.15.52:0/3468344721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650dbdebf60 0x5650dbe07490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2b937fe700 0 client.16680 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 >> 172.21.15.52:0/3468344721 conn(0x5650dbd13080 msgr2=0x5650dbdf1920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 shutdown_connections 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 1 -- 172.21.15.52:0/3468344721 wait complete. 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bc4c73b80 20 client.16680 trim_cache size 0 max 16384 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stdout: "3a178d61-b8a5-424c-bbc6-620458c2a8e7": { 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:15:27.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "3a178d61-b8a5-424c-bbc6-620458c2a8e7", 2024-06-02T23:15:27.183 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:15:27.183 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:27.183 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:15:27.183 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b9c0a2c20 msgr2=0x7f2b9c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b9c0a2c20 0x7f2b9c0a50e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2ba40104e0 tx=0x7f2ba400b480 comp rx=0 tx=0).stop 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 msgr2=0x7f2bb4103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.177+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb4103580 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2ba800d750 tx=0x7f2ba800da60 comp rx=0 tx=0).stop 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 shutdown_connections 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b9c0a2c20 0x7f2b9c0a50e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41073d0 0x7f2bb4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4101390 0x7f2bb4103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 --2- 172.21.15.52:0/3067260956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4101d30 0x7f2bb4103580 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:27.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 >> 172.21.15.52:0/3067260956 conn(0x7f2bb40782c0 msgr2=0x7f2bb4100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:27.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 shutdown_connections 2024-06-02T23:15:27.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:15:27.178+0000 7f2bba3f0700 1 -- 172.21.15.52:0/3067260956 wait complete. 2024-06-02T23:15:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:27 smithi052 ceph-mon[32340]: pgmap v538: 97 pgs: 97 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 373 KiB/s wr, 54 op/s 2024-06-02T23:15:27.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:27 smithi123 ceph-mon[35977]: pgmap v538: 97 pgs: 97 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 373 KiB/s wr, 54 op/s 2024-06-02T23:15:27.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 51465 0.49 MB/sec execute 36 sec latency 68.173 ms 2024-06-02T23:15:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:27 smithi174 ceph-mon[35994]: pgmap v538: 97 pgs: 97 active+clean; 39 MiB data, 626 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 373 KiB/s wr, 54 op/s 2024-06-02T23:15:27.996 INFO:tasks.cephfs.filesystem:scrub status for tag:3a178d61-b8a5-424c-bbc6-620458c2a8e7 - {'path': '/', 'tag': '3a178d61-b8a5-424c-bbc6-620458c2a8e7', 'options': 'recursive,force'} 2024-06-02T23:15:28.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 52091 0.52 MB/sec execute 37 sec latency 14.352 ms 2024-06-02T23:15:29.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:29 smithi052 ceph-mon[32340]: pgmap v539: 97 pgs: 97 active+clean; 40 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 523 KiB/s wr, 58 op/s 2024-06-02T23:15:29.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:29 smithi123 ceph-mon[35977]: pgmap v539: 97 pgs: 97 active+clean; 40 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 523 KiB/s wr, 58 op/s 2024-06-02T23:15:29.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 52857 0.67 MB/sec execute 38 sec latency 14.901 ms 2024-06-02T23:15:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:29 smithi174 ceph-mon[35994]: pgmap v539: 97 pgs: 97 active+clean; 40 MiB data, 627 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 523 KiB/s wr, 58 op/s 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 -- 172.21.15.52:0/4155588959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa501024b0 msgr2=0x7ffa50108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 --2- 172.21.15.52:0/4155588959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa501024b0 0x7ffa50108d40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009ca0 tx=0x7ffa4802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 -- 172.21.15.52:0/4155588959 shutdown_connections 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 --2- 172.21.15.52:0/4155588959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa501024b0 0x7ffa50108d40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 --2- 172.21.15.52:0/4155588959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50101af0 0x7ffa50101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 --2- 172.21.15.52:0/4155588959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa50100900 0x7ffa50100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 -- 172.21.15.52:0/4155588959 >> 172.21.15.52:0/4155588959 conn(0x7ffa500fbe70 msgr2=0x7ffa500fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 -- 172.21.15.52:0/4155588959 shutdown_connections 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 -- 172.21.15.52:0/4155588959 wait complete. 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 Processor -- start 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.358+0000 7ffa5869e640 1 -- start start 2024-06-02T23:15:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50100900 0x7ffa50197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 0x7ffa501980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa501024b0 0x7ffa5019f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010c410 con 0x7ffa50101af0 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010c290 con 0x7ffa50100900 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa5010c590 con 0x7ffa501024b0 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 0x7ffa501980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 0x7ffa501980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36298/0 (socket says 172.21.15.52:36298) 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50100900 0x7ffa50197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa55c39640 1 -- 172.21.15.52:0/2021292993 learned_addr learned my addr 172.21.15.52:0/2021292993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa501024b0 0x7ffa5019f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 -- 172.21.15.52:0/2021292993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa501024b0 msgr2=0x7ffa5019f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa501024b0 0x7ffa5019f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 -- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 msgr2=0x7ffa501980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 0x7ffa501980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 -- 172.21.15.52:0/2021292993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5019f790 con 0x7ffa50100900 2024-06-02T23:15:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa501024b0 0x7ffa5019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa55c39640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 0x7ffa501980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5643a640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50100900 0x7ffa50197b70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa50101950 tx=0x7ffa4c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa477fe640 1 -- 172.21.15.52:0/2021292993 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c017070 con 0x7ffa50100900 2024-06-02T23:15:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa477fe640 1 -- 172.21.15.52:0/2021292993 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c009c30 con 0x7ffa50100900 2024-06-02T23:15:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5019fa80 con 0x7ffa50100900 2024-06-02T23:15:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.359+0000 7ffa477fe640 1 -- 172.21.15.52:0/2021292993 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c00f620 con 0x7ffa50100900 2024-06-02T23:15:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.360+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa5019ffd0 con 0x7ffa50100900 2024-06-02T23:15:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.362+0000 7ffa477fe640 1 -- 172.21.15.52:0/2021292993 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa4c01a070 con 0x7ffa50100900 2024-06-02T23:15:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.362+0000 7ffa5b3ad740 1 Processor -- start 2024-06-02T23:15:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.362+0000 7ffa5b3ad740 1 -- start start 2024-06-02T23:15:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.362+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f1743430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.362+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1748500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c9f16a64f0 con 0x55c9f1748b70 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9f1644870 con 0x55c9f1743010 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9f16a24b0 con 0x55c9f1743a00 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa56c3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f1743430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa56c3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f1743430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44564/0 (socket says 172.21.15.52:44564) 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa56c3b640 1 -- 172.21.15.52:0/3895136141 learned_addr learned my addr 172.21.15.52:0/3895136141 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa5643a640 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1748500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 -- 172.21.15.52:0/3895136141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 msgr2=0x55c9f1748500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1748500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 -- 172.21.15.52:0/3895136141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 msgr2=0x55c9f1743430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f1743430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 -- 172.21.15.52:0/3895136141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f16609b0 con 0x55c9f1748b70 2024-06-02T23:15:30.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa5643a640 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1748500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f1743430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa477fe640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa3c0a2c30 0x7ffa3c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa3c0a2c30 0x7ffa3c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa55c39640 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174b000 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009fd0 tx=0x7ffa40011950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d880541ec0279c87 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa477fe640 1 -- 172.21.15.52:0/2021292993 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c0cea90 con 0x7ffa50100900 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.363+0000 7ffa44ff9640 1 -- 172.21.15.52:0/3895136141 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40005020 con 0x55c9f1748b70 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa44ff9640 1 -- 172.21.15.52:0/3895136141 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa400047a0 con 0x55c9f1748b70 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa44ff9640 1 -- 172.21.15.52:0/3895136141 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000f860 con 0x55c9f1748b70 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3895136141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 msgr2=0x55c9f174b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa3c0a2c30 0x7ffa3c0a50f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa48006010 tx=0x7ffa48005c50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174b000 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009fd0 tx=0x7ffa40011950 comp rx=0 tx=0).stop 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3895136141 shutdown_connections 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1748500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174b000 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3895136141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f1743430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.364+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3895136141 >> 172.21.15.52:0/3895136141 conn(0x55c9f173e680 msgr2=0x55c9f173ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3895136141 shutdown_connections 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3895136141 wait complete. 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 -- start start 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 Processor -- start 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f174d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1748b70 0x55c9f17547a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c9f174bb20 con 0x55c9f1743a00 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c9f162fae0 con 0x55c9f1743010 2024-06-02T23:15:30.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c9f174bca0 con 0x55c9f1748b70 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5b3ad740 10 client.0 _ll_register_callbacks cb 0x55c9f16689b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f174d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa5643a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f174d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36320/0 (socket says 172.21.15.52:36320) 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa55c39640 1 -- 172.21.15.52:0/1030241411 learned_addr learned my addr 172.21.15.52:0/1030241411 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.365+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1748b70 0x55c9f17547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5643a640 1 -- 172.21.15.52:0/1030241411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1748b70 msgr2=0x55c9f17547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1748b70 0x55c9f17547a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5643a640 1 -- 172.21.15.52:0/1030241411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 msgr2=0x55c9f174d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f174d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5643a640 1 -- 172.21.15.52:0/1030241411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f13ba210 con 0x55c9f1743010 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1748b70 0x55c9f17547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174d220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0cc610 tx=0x7ffa4c09ae60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b785353236e4c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:30.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1030241411 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c021050 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1030241411 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c09a770 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa400047a0 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1030241411 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c09ba20 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5b3ad740 10 client.26228 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c9f1486620 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.366+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1030241411 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa400047a0 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1030241411 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ffa4c0bb0b0 con 0x55c9f1743010 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa5b3ad740 10 client.26228 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa5b3ad740 10 client.26228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa5b3ad740 20 client.26228 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f1758fe0 0x55c9f175b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa5b3ad740 4 client.26228 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:30.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f16b9f10 con 0x55c9f1758fe0 2024-06-02T23:15:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa55c39640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f1758fe0 0x55c9f175b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa39ffb640 10 client.26228 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:30.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.367+0000 7ffa55c39640 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f1758fe0 0x55c9f175b470 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4280e98b30c877f server_cookie=1606a01841d728d4 in_seq=0 out_seq=0 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1030241411 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c9f16b9f10 con 0x55c9f1758fe0 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa39ffb640 10 client.26228 handle_command_reply: tid=0 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 client.26228 shutdown 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 msgr2=0x55c9f174d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174d220 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0cc610 tx=0x7ffa4c09ae60 comp rx=0 tx=0).stop 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 shutdown_connections 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f1758fe0 0x55c9f175b470 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1748b70 0x55c9f17547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f174d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1030241411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174d220 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa39ffb640 0 client.26228 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 >> 172.21.15.52:0/1030241411 conn(0x55c9f173e680 msgr2=0x55c9f173d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 shutdown_connections 2024-06-02T23:15:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1030241411 wait complete. 2024-06-02T23:15:30.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.368+0000 7ffa5b3ad740 20 client.26228 trim_cache size 0 max 16384 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 Processor -- start 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 -- start start 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1754760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501aaf30 con 0x55c9f1748b70 2024-06-02T23:15:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501acad0 con 0x55c9f1743010 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.385+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ab600 con 0x55c9f1743a00 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36332/0 (socket says 172.21.15.52:36332) 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa5643a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 -- 172.21.15.52:0/3835642030 learned_addr learned my addr 172.21.15.52:0/3835642030 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa55c39640 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1754760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 -- 172.21.15.52:0/3835642030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 msgr2=0x55c9f1754760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1754760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 -- 172.21.15.52:0/3835642030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 msgr2=0x55c9f174ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 -- 172.21.15.52:0/3835642030 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f175baf0 con 0x55c9f1748b70 2024-06-02T23:15:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa5643a640 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174d370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffa48031650 tx=0x7ffa48040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fd637cc328c8f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa3bfff640 1 -- 172.21.15.52:0/3835642030 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48031740 con 0x55c9f1748b70 2024-06-02T23:15:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa3bfff640 1 -- 172.21.15.52:0/3835642030 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa48036050 con 0x55c9f1748b70 2024-06-02T23:15:30.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.386+0000 7ffa3bfff640 1 -- 172.21.15.52:0/3835642030 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48030900 con 0x55c9f1748b70 2024-06-02T23:15:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3835642030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 msgr2=0x55c9f174d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174d370 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7ffa48031650 tx=0x7ffa48040000 comp rx=0 tx=0).stop 2024-06-02T23:15:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3835642030 shutdown_connections 2024-06-02T23:15:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743a00 0x55c9f1754760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1748b70 0x55c9f174d370 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/3835642030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1743010 0x55c9f174ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3835642030 >> 172.21.15.52:0/3835642030 conn(0x55c9f173e680 msgr2=0x55c9f1748210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3835642030 shutdown_connections 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/3835642030 wait complete. 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.387+0000 7ffa5b3ad740 1 -- start start 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 Processor -- start 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743010 0x55c9f17516d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f1751c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1748b70 0x55c9f17c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501ac3a0 con 0x55c9f1743a00 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501abcd0 con 0x55c9f1748b70 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa501a8f10 con 0x55c9f1743010 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f1751c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f1751c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36342/0 (socket says 172.21.15.52:36342) 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 -- 172.21.15.52:0/1366111722 learned_addr learned my addr 172.21.15.52:0/1366111722 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743010 0x55c9f17516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5b3ad740 10 client.0 _ll_register_callbacks cb 0x7ffa4c0d3040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1748b70 0x55c9f17c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 -- 172.21.15.52:0/1366111722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743010 msgr2=0x55c9f17516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743010 0x55c9f17516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 -- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1748b70 msgr2=0x55c9f17c1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1748b70 0x55c9f17c1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa55c39640 1 -- 172.21.15.52:0/1366111722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f173e4b0 con 0x55c9f1743a00 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa56c3b640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1748b70 0x55c9f17c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.388+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743010 0x55c9f17516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa55c39640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f1751c10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005fd0 tx=0x7ffa400093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c319440e5347923 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:30.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1366111722 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40011250 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1366111722 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4001f040 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa48036050 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1366111722 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ffa4000f600 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 10 client.16698 fetch_fsmap learned FSMap version 31 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c9f17c3460 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1366111722 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa48036050 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1366111722 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 31) ==== 8362+0+0 (secure 0 0 0) 0x7ffa40004f40 con 0x55c9f1743a00 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 10 client.16698 fetch_fsmap finished waiting for FSMap version 31 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 10 client.16698 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 20 client.16698 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:30.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f17c3ca0 0x55c9f17c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 4 client.16698 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:15:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.389+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9f16b3c20 con 0x55c9f17c3ca0 2024-06-02T23:15:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.390+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f17c3ca0 0x55c9f17c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.390+0000 7ffa39ffb640 10 client.16698 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:30.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.390+0000 7ffa5643a640 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f17c3ca0 0x55c9f17c60c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=141da6677cb966b4 server_cookie=81b214769df4de55 in_seq=0 out_seq=0 2024-06-02T23:15:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa39ffb640 1 -- 172.21.15.52:0/1366111722 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c9f16b3c20 con 0x55c9f17c3ca0 2024-06-02T23:15:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa39ffb640 10 client.16698 handle_command_reply: tid=0 2024-06-02T23:15:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 client.16698 shutdown 2024-06-02T23:15:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 msgr2=0x55c9f1751c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.405 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f1751c10 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7ffa40005fd0 tx=0x7ffa400093e0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 shutdown_connections 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c9f17c3ca0 0x55c9f17c60c0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c9f1748b70 0x55c9f17c1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c9f1743a00 0x55c9f1751c10 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 --2- 172.21.15.52:0/1366111722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c9f1743010 0x55c9f17516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa39ffb640 0 client.16698 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.404+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 >> 172.21.15.52:0/1366111722 conn(0x55c9f173e680 msgr2=0x55c9f174ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 shutdown_connections 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5b3ad740 1 -- 172.21.15.52:0/1366111722 wait complete. 2024-06-02T23:15:30.406 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5b3ad740 20 client.16698 trim_cache size 0 max 16384 2024-06-02T23:15:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:15:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:15:30.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa3c0a2c30 msgr2=0x7ffa3c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa3c0a2c30 0x7ffa3c0a50f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ffa48006010 tx=0x7ffa48005c50 comp rx=0 tx=0).stop 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50100900 msgr2=0x7ffa50197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50100900 0x7ffa50197b70 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ffa50101950 tx=0x7ffa4c00dde0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 shutdown_connections 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa3c0a2c30 0x7ffa3c0a50f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa501024b0 0x7ffa5019f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa50101af0 0x7ffa501980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 --2- 172.21.15.52:0/2021292993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa50100900 0x7ffa50197b70 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 >> 172.21.15.52:0/2021292993 conn(0x7ffa500fbe70 msgr2=0x7ffa500fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 shutdown_connections 2024-06-02T23:15:30.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:30.405+0000 7ffa5869e640 1 -- 172.21.15.52:0/2021292993 wait complete. 2024-06-02T23:15:30.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 53308 0.77 MB/sec execute 39 sec latency 14.566 ms 2024-06-02T23:15:31.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:31 smithi123 ceph-mon[35977]: pgmap v540: 97 pgs: 97 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 731 KiB/s wr, 78 op/s 2024-06-02T23:15:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:15:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:15:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:15:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:15:31.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 53631 0.78 MB/sec execute 40 sec latency 14.790 ms 2024-06-02T23:15:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:31 smithi174 ceph-mon[35994]: pgmap v540: 97 pgs: 97 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 731 KiB/s wr, 78 op/s 2024-06-02T23:15:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:15:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:15:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:15:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:15:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:31 smithi052 ceph-mon[32340]: pgmap v540: 97 pgs: 97 active+clean; 40 MiB data, 628 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 731 KiB/s wr, 78 op/s 2024-06-02T23:15:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:15:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:15:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:15:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:15:32.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:32 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:15:32.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 54019 0.78 MB/sec execute 41 sec latency 13.701 ms 2024-06-02T23:15:32.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:32 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:15:32.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:32 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:15:33.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:33 smithi123 ceph-mon[35977]: pgmap v541: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-06-02T23:15:33.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 54619 0.87 MB/sec execute 42 sec latency 14.545 ms 2024-06-02T23:15:33.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:33 smithi174 ceph-mon[35994]: pgmap v541: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-06-02T23:15:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:33 smithi052 ceph-mon[32340]: pgmap v541: 97 pgs: 97 active+clean; 40 MiB data, 614 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.0 MiB/s wr, 120 op/s 2024-06-02T23:15:34.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 55215 0.86 MB/sec execute 43 sec latency 14.817 ms 2024-06-02T23:15:35.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:35 smithi123 ceph-mon[35977]: pgmap v542: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-06-02T23:15:35.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 55742 0.89 MB/sec execute 44 sec latency 14.827 ms 2024-06-02T23:15:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:35 smithi174 ceph-mon[35994]: pgmap v542: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-06-02T23:15:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:35 smithi052 ceph-mon[32340]: pgmap v542: 97 pgs: 97 active+clean; 40 MiB data, 618 MiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 1.0 MiB/s wr, 122 op/s 2024-06-02T23:15:36.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 56533 1.02 MB/sec execute 45 sec latency 12.011 ms 2024-06-02T23:15:37.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:37 smithi123 ceph-mon[35977]: pgmap v543: 97 pgs: 97 active+clean; 42 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:15:37.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 56946 1.07 MB/sec execute 46 sec latency 15.243 ms 2024-06-02T23:15:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:37 smithi174 ceph-mon[35994]: pgmap v543: 97 pgs: 97 active+clean; 42 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:15:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:37 smithi052 ceph-mon[32340]: pgmap v543: 97 pgs: 97 active+clean; 42 MiB data, 647 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:15:38.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 57233 1.07 MB/sec execute 47 sec latency 11.864 ms 2024-06-02T23:15:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:39 smithi123 ceph-mon[35977]: pgmap v544: 97 pgs: 97 active+clean; 42 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:15:39.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 57661 1.07 MB/sec execute 48 sec latency 13.824 ms 2024-06-02T23:15:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:39 smithi174 ceph-mon[35994]: pgmap v544: 97 pgs: 97 active+clean; 42 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:15:39.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:39 smithi052 ceph-mon[32340]: pgmap v544: 97 pgs: 97 active+clean; 42 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:15:40.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 58259 1.13 MB/sec execute 49 sec latency 12.332 ms 2024-06-02T23:15:41.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:41 smithi123 ceph-mon[35977]: pgmap v545: 97 pgs: 97 active+clean; 41 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:15:41.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 58921 1.12 MB/sec execute 50 sec latency 13.942 ms 2024-06-02T23:15:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:41 smithi174 ceph-mon[35994]: pgmap v545: 97 pgs: 97 active+clean; 41 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:15:41.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:41 smithi052 ceph-mon[32340]: pgmap v545: 97 pgs: 97 active+clean; 41 MiB data, 657 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:15:42.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 59400 1.15 MB/sec execute 51 sec latency 15.529 ms 2024-06-02T23:15:43.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:43 smithi123 ceph-mon[35977]: pgmap v546: 97 pgs: 97 active+clean; 44 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:15:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:43 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3639211364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c100900 msgr2=0x7f3c5c100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3639211364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c100900 0x7f3c5c100d20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3c58009900 tx=0x7f3c5802a410 comp rx=0 tx=0).stop 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3639211364 shutdown_connections 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3639211364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c101af0 0x7f3c5c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3639211364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c5c1024b0 0x7f3c5c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3639211364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c100900 0x7f3c5c100d20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3639211364 >> 172.21.15.52:0/3639211364 conn(0x7f3c5c0fbe70 msgr2=0x7f3c5c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:43.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3639211364 shutdown_connections 2024-06-02T23:15:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.514+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3639211364 wait complete. 2024-06-02T23:15:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 Processor -- start 2024-06-02T23:15:43.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 -- start start 2024-06-02T23:15:43.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 0x7f3c5c197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c5c101af0 0x7f3c5c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c1024b0 0x7f3c5c19f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c10c420 con 0x7f3c5c101af0 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c10c2a0 con 0x7f3c5c1024b0 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c63dd5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c10c5a0 con 0x7f3c5c100900 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c61b71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 0x7f3c5c197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c61b71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 0x7f3c5c197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38774/0 (socket says 172.21.15.52:38774) 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c61b71640 1 -- 172.21.15.52:0/3693319058 learned_addr learned my addr 172.21.15.52:0/3693319058 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.515+0000 7f3c61370640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c5c101af0 0x7f3c5c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c62372640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c1024b0 0x7f3c5c19f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c61b71640 1 -- 172.21.15.52:0/3693319058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c1024b0 msgr2=0x7f3c5c19f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c61b71640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c1024b0 0x7f3c5c19f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c61b71640 1 -- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c5c101af0 msgr2=0x7f3c5c198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c61b71640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c5c101af0 0x7f3c5c198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c61b71640 1 -- 172.21.15.52:0/3693319058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c5c19f7d0 con 0x7f3c5c100900 2024-06-02T23:15:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c61b71640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 0x7f3c5c197c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c580314e0 tx=0x7f3c58004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c52ffd640 1 -- 172.21.15.52:0/3693319058 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5803f030 con 0x7f3c5c100900 2024-06-02T23:15:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c52ffd640 1 -- 172.21.15.52:0/3693319058 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c58003aa0 con 0x7f3c5c100900 2024-06-02T23:15:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c5c19fa60 con 0x7f3c5c100900 2024-06-02T23:15:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c52ffd640 1 -- 172.21.15.52:0/3693319058 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c58035070 con 0x7f3c5c100900 2024-06-02T23:15:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.516+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c5c19ff50 con 0x7f3c5c100900 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 Processor -- start 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c52ffd640 1 -- 172.21.15.52:0/3693319058 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3c58042030 con 0x7f3c5c100900 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 -- start start 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50b4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5651e50124f0 con 0x5651e50b4b70 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651e4fb0870 con 0x5651e50afa00 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.519+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651e500e4b0 con 0x5651e50af010 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c61b71640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50b4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c61b71640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50b4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38110/0 (socket says 172.21.15.52:38110) 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c61b71640 1 -- 172.21.15.52:0/2310265469 learned_addr learned my addr 172.21.15.52:0/2310265469 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c61370640 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 -- 172.21.15.52:0/2310265469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 msgr2=0x5651e50b4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50b4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 -- 172.21.15.52:0/2310265469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 msgr2=0x5651e50b7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50b7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 -- 172.21.15.52:0/2310265469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e4fcc9b0 con 0x5651e50af010 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c61b71640 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c52ffd640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3c480a2cb0 0x7f3c480a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c62372640 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50af430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009f00 tx=0x7f3c54011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ad15dd1c389acfe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c61b71640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3c480a2cb0 0x7f3c480a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.520+0000 7f3c52ffd640 1 -- 172.21.15.52:0/3693319058 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3c580ef760 con 0x7f3c5c100900 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.521+0000 7f3c47fff640 1 -- 172.21.15.52:0/2310265469 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54004d20 con 0x5651e50af010 2024-06-02T23:15:43.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.521+0000 7f3c47fff640 1 -- 172.21.15.52:0/2310265469 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c54004030 con 0x5651e50af010 2024-06-02T23:15:43.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.521+0000 7f3c47fff640 1 -- 172.21.15.52:0/2310265469 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54005260 con 0x5651e50af010 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.521+0000 7f3c66bc2740 1 -- 172.21.15.52:0/2310265469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 msgr2=0x5651e50af430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.521+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50af430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009f00 tx=0x7f3c54011810 comp rx=0 tx=0).stop 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.521+0000 7f3c61b71640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3c480a2cb0 0x7f3c480a5170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3c58000c00 tx=0x7f3c580dc040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 -- 172.21.15.52:0/2310265469 shutdown_connections 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50b7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50b4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/2310265469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50af430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 -- 172.21.15.52:0/2310265469 >> 172.21.15.52:0/2310265469 conn(0x5651e50aa680 msgr2=0x5651e50aaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 -- 172.21.15.52:0/2310265469 shutdown_connections 2024-06-02T23:15:43.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 -- 172.21.15.52:0/2310265469 wait complete. 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 -- start start 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.522+0000 7f3c66bc2740 1 Processor -- start 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50b9250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50afa00 0x5651e50b9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5651e50b7b20 con 0x5651e50b4b70 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5651e4f9bae0 con 0x5651e50af010 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651e50b7ca0 con 0x5651e50afa00 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50b9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c66bc2740 10 client.0 _ll_register_callbacks cb 0x5651e4fd49b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50b9250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38122/0 (socket says 172.21.15.52:38122) 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 -- 172.21.15.52:0/3030649284 learned_addr learned my addr 172.21.15.52:0/3030649284 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c61370640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50afa00 0x5651e50b9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 -- 172.21.15.52:0/3030649284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50afa00 msgr2=0x5651e50b9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50afa00 0x5651e50b9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 -- 172.21.15.52:0/3030649284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 msgr2=0x5651e50c07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50c07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 -- 172.21.15.52:0/3030649284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e4d26210 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c62372640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50b9250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c54003ea0 tx=0x7f3c54000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c690d5d7034e3151 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.523+0000 7f3c457fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c457fa640 1 -- 172.21.15.52:0/3030649284 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5400fb10 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c54004030 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c457fa640 1 -- 172.21.15.52:0/3030649284 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3c5400fcb0 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c457fa640 1 -- 172.21.15.52:0/3030649284 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3c54034460 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c66bc2740 10 client.26246 fetch_fsmap learned FSMap version 32 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5651e4df2620 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.524+0000 7f3c457fa640 1 -- 172.21.15.52:0/3030649284 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c54004030 con 0x5651e50af010 2024-06-02T23:15:43.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c457fa640 1 -- 172.21.15.52:0/3030649284 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3c54018070 con 0x5651e50af010 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c66bc2740 10 client.26246 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c66bc2740 10 client.26246 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c66bc2740 20 client.26246 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e50c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c66bc2740 4 client.26246 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651e5025f10 con 0x5651e50c4f00 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c61370640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e50c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c457fa640 10 client.26246 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:43.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.525+0000 7f3c61370640 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e50c72f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9980fa8f5fc990 server_cookie=1aeafe82082a74d7 in_seq=0 out_seq=0 2024-06-02T23:15:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c457fa640 1 -- 172.21.15.52:0/3030649284 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5651e5025f10 con 0x5651e50c4f00 2024-06-02T23:15:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c457fa640 10 client.26246 handle_command_reply: tid=0 2024-06-02T23:15:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 client.26246 shutdown 2024-06-02T23:15:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 msgr2=0x5651e50b9250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50b9250 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3c54003ea0 tx=0x7f3c54000f40 comp rx=0 tx=0).stop 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 shutdown_connections 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e50c72f0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50afa00 0x5651e50b9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50c07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.526+0000 7f3c457fa640 0 client.26246 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.527+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3030649284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50b9250 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.527+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 >> 172.21.15.52:0/3030649284 conn(0x5651e50aa680 msgr2=0x5651e50a97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.527+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 shutdown_connections 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.527+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3030649284 wait complete. 2024-06-02T23:15:43.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.527+0000 7f3c66bc2740 20 client.26246 trim_cache size 0 max 16384 2024-06-02T23:15:43.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.545+0000 7f3c66bc2740 1 Processor -- start 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.545+0000 7f3c66bc2740 1 -- start start 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50ad590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a6af0 con 0x5651e50b4b70 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a8630 con 0x5651e50afa00 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a71c0 con 0x5651e50af010 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c61370640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50ad590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46200/0 (socket says 172.21.15.52:46200) 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c61370640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50c0790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38138/0 (socket says 172.21.15.52:38138) 2024-06-02T23:15:43.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 -- 172.21.15.52:0/3071225450 learned_addr learned my addr 172.21.15.52:0/3071225450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 -- 172.21.15.52:0/3071225450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 msgr2=0x5651e50b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 -- 172.21.15.52:0/3071225450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 msgr2=0x5651e50c0790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50c0790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 -- 172.21.15.52:0/3071225450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e50aa4b0 con 0x5651e50b4b70 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c61370640 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c62372640 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50ad590 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009d40 tx=0x7f3c54014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e6bcd02754d27e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.546+0000 7f3c477fe640 1 -- 172.21.15.52:0/3071225450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54004680 con 0x5651e50b4b70 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.547+0000 7f3c477fe640 1 -- 172.21.15.52:0/3071225450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c5402d070 con 0x5651e50b4b70 2024-06-02T23:15:43.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.547+0000 7f3c477fe640 1 -- 172.21.15.52:0/3071225450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c54004c60 con 0x5651e50b4b70 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.547+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3071225450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 msgr2=0x5651e50ad590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50ad590 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3c54009d40 tx=0x7f3c54014040 comp rx=0 tx=0).stop 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3071225450 shutdown_connections 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50b4b70 0x5651e50ad590 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50afa00 0x5651e50c0790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3071225450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50af010 0x5651e50b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3071225450 >> 172.21.15.52:0/3071225450 conn(0x5651e50aa680 msgr2=0x5651e50aaa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3071225450 shutdown_connections 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3071225450 wait complete. 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 -- start start 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.548+0000 7f3c66bc2740 1 Processor -- start 2024-06-02T23:15:43.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50c03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 0x5651e50bce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50b4b70 0x5651e50bd380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a7f60 con 0x5651e50afa00 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a7890 con 0x5651e50af010 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c5c1a4ad0 con 0x5651e50b4b70 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c62372640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50c03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 0x5651e50bce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c66bc2740 10 client.0 _ll_register_callbacks cb 0x7f3c4c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 0x5651e50bce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46202/0 (socket says 172.21.15.52:46202) 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c62372640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50c03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38140/0 (socket says 172.21.15.52:38140) 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 -- 172.21.15.52:0/3465119582 learned_addr learned my addr 172.21.15.52:0/3465119582 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c62372640 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50b4b70 0x5651e50bd380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.552 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 -- 172.21.15.52:0/3465119582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50b4b70 msgr2=0x5651e50bd380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50b4b70 0x5651e50bd380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 -- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 msgr2=0x5651e50c03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50c03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.549+0000 7f3c61370640 1 -- 172.21.15.52:0/3465119582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e50bd8c0 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c61370640 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 0x5651e50bce40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c1990c0 tx=0x7f3c4c004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a929a5d935eab66f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c457fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c457fa640 1 -- 172.21.15.52:0/3465119582 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c4c02b060 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c457fa640 1 -- 172.21.15.52:0/3465119582 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3c4c003e00 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c5402d070 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c457fa640 1 -- 172.21.15.52:0/3465119582 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3c4c020410 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c66bc2740 10 client.16716 fetch_fsmap learned FSMap version 32 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.550+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5651e512ec50 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c457fa640 1 -- 172.21.15.52:0/3465119582 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3c5402d070 con 0x5651e50afa00 2024-06-02T23:15:43.553 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c457fa640 1 -- 172.21.15.52:0/3465119582 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3c4c025030 con 0x5651e50afa00 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c66bc2740 10 client.16716 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c66bc2740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c66bc2740 20 client.16716 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e5131650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c66bc2740 4 client.16716 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5651e501fc20 con 0x5651e50c4f00 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.551+0000 7f3c62372640 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e5131650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.552+0000 7f3c457fa640 10 client.16716 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:43.554 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.552+0000 7f3c62372640 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e5131650 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96c1541281ea2b07 server_cookie=27d1c9d0dceab3e9 in_seq=0 out_seq=0 2024-06-02T23:15:43.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c457fa640 1 -- 172.21.15.52:0/3465119582 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5651e501fc20 con 0x5651e50c4f00 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c457fa640 10 client.16716 handle_command_reply: tid=0 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 client.16716 shutdown 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 msgr2=0x5651e50bce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 0x5651e50bce40 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3c5c1990c0 tx=0x7f3c4c004340 comp rx=0 tx=0).stop 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 shutdown_connections 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5651e50c4f00 0x5651e5131650 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651e50b4b70 0x5651e50bd380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5651e50afa00 0x5651e50bce40 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 --2- 172.21.15.52:0/3465119582 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5651e50af010 0x5651e50c03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 >> 172.21.15.52:0/3465119582 conn(0x5651e50aa680 msgr2=0x5651e50b8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c457fa640 0 client.16716 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:43.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 shutdown_connections 2024-06-02T23:15:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 1 -- 172.21.15.52:0/3465119582 wait complete. 2024-06-02T23:15:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.573+0000 7f3c66bc2740 20 client.16716 trim_cache size 0 max 16384 2024-06-02T23:15:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:15:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:15:43.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3c480a2cb0 msgr2=0x7f3c480a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3c480a2cb0 0x7f3c480a5170 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3c58000c00 tx=0x7f3c580dc040 comp rx=0 tx=0).stop 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 msgr2=0x7f3c5c197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 0x7f3c5c197c00 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3c580314e0 tx=0x7f3c58004270 comp rx=0 tx=0).stop 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 shutdown_connections 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3c480a2cb0 0x7f3c480a5170 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.575 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3c5c1024b0 0x7f3c5c19f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3c5c101af0 0x7f3c5c198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 --2- 172.21.15.52:0/3693319058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c5c100900 0x7f3c5c197c00 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 >> 172.21.15.52:0/3693319058 conn(0x7f3c5c0fbe70 msgr2=0x7f3c5c069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 shutdown_connections 2024-06-02T23:15:43.576 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:43.574+0000 7f3c63dd5640 1 -- 172.21.15.52:0/3693319058 wait complete. 2024-06-02T23:15:43.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 60086 1.25 MB/sec execute 52 sec latency 18.959 ms 2024-06-02T23:15:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:43 smithi174 ceph-mon[35994]: pgmap v546: 97 pgs: 97 active+clean; 44 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:15:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:43 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:15:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:43 smithi052 ceph-mon[32340]: pgmap v546: 97 pgs: 97 active+clean; 44 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:15:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:43 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:15:44.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:15:44.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 60559 1.31 MB/sec execute 53 sec latency 15.535 ms 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:15:44.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:15:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:15:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:15:44.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:15:44.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:15:44.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:15:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:15:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:15:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:15:45.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 60798 1.29 MB/sec execute 54 sec latency 16.221 ms 2024-06-02T23:15:45.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:45 smithi123 ceph-mon[35977]: pgmap v547: 97 pgs: 97 active+clean; 43 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:15:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:45 smithi174 ceph-mon[35994]: pgmap v547: 97 pgs: 97 active+clean; 43 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:15:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:45 smithi052 ceph-mon[32340]: pgmap v547: 97 pgs: 97 active+clean; 43 MiB data, 663 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:15:46.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 61243 1.28 MB/sec execute 55 sec latency 14.787 ms 2024-06-02T23:15:47.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 61826 1.33 MB/sec execute 56 sec latency 14.478 ms 2024-06-02T23:15:47.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:47 smithi123 ceph-mon[35977]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-02T23:15:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:47 smithi174 ceph-mon[35994]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-02T23:15:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:47 smithi052 ceph-mon[32340]: pgmap v548: 97 pgs: 97 active+clean; 44 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.6 MiB/s wr, 190 op/s 2024-06-02T23:15:48.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 62471 1.32 MB/sec execute 57 sec latency 14.888 ms 2024-06-02T23:15:49.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 62944 1.34 MB/sec execute 58 sec latency 15.437 ms 2024-06-02T23:15:49.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:49 smithi123 ceph-mon[35977]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-06-02T23:15:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:49 smithi174 ceph-mon[35994]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-06-02T23:15:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:49 smithi052 ceph-mon[32340]: pgmap v549: 97 pgs: 97 active+clean; 44 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.3 MiB/s wr, 136 op/s 2024-06-02T23:15:50.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 63647 1.42 MB/sec execute 59 sec latency 15.620 ms 2024-06-02T23:15:51.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 64126 1.47 MB/sec execute 60 sec latency 14.253 ms 2024-06-02T23:15:51.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:51 smithi123 ceph-mon[35977]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-02T23:15:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:51 smithi174 ceph-mon[35994]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-02T23:15:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:51 smithi052 ceph-mon[32340]: pgmap v550: 97 pgs: 97 active+clean; 44 MiB data, 674 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.3 MiB/s wr, 149 op/s 2024-06-02T23:15:52.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 64411 1.45 MB/sec execute 61 sec latency 11.765 ms 2024-06-02T23:15:53.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 64912 1.46 MB/sec execute 62 sec latency 14.142 ms 2024-06-02T23:15:53.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:53 smithi052 ceph-mon[32340]: pgmap v551: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-02T23:15:53.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:53 smithi123 ceph-mon[35977]: pgmap v551: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-02T23:15:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:53 smithi174 ceph-mon[35994]: pgmap v551: 97 pgs: 97 active+clean; 45 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-02T23:15:54.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 65482 1.49 MB/sec execute 63 sec latency 15.639 ms 2024-06-02T23:15:55.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 66175 1.49 MB/sec execute 64 sec latency 14.950 ms 2024-06-02T23:15:55.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:55 smithi123 ceph-mon[35977]: pgmap v552: 97 pgs: 97 active+clean; 44 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-06-02T23:15:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:55 smithi174 ceph-mon[35994]: pgmap v552: 97 pgs: 97 active+clean; 44 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-06-02T23:15:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:55 smithi052 ceph-mon[32340]: pgmap v552: 97 pgs: 97 active+clean; 44 MiB data, 680 MiB used, 1.0 TiB / 1.0 TiB avail; 218 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-06-02T23:15:56.625 INFO:tasks.workunit.client.0.smithi052.stdout: 1 66795 1.56 MB/sec execute 65 sec latency 14.535 ms 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 -- 172.21.15.52:0/1034181996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 msgr2=0x7fe1981071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 --2- 172.21.15.52:0/1034181996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe1981071a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009900 tx=0x7fe18c02a410 comp rx=0 tx=0).stop 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 -- 172.21.15.52:0/1034181996 shutdown_connections 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 --2- 172.21.15.52:0/1034181996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1981076e0 0x7fe198109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 --2- 172.21.15.52:0/1034181996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe1981071a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 --2- 172.21.15.52:0/1034181996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe198102480 0x7fe198104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 -- 172.21.15.52:0/1034181996 >> 172.21.15.52:0/1034181996 conn(0x7fe1980fbe70 msgr2=0x7fe1980fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 -- 172.21.15.52:0/1034181996 shutdown_connections 2024-06-02T23:15:56.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.674+0000 7fe19c883640 1 -- 172.21.15.52:0/1034181996 wait complete. 2024-06-02T23:15:56.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 Processor -- start 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 -- start start 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe198102480 0x7fe198101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe198100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1981076e0 0x7fe198100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe19810d530 con 0x7fe198104db0 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe19810d3b0 con 0x7fe1981076e0 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.675+0000 7fe19c883640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe19810d6b0 con 0x7fe198102480 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe198100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe198100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48230/0 (socket says 172.21.15.52:48230) 2024-06-02T23:15:56.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 -- 172.21.15.52:0/2113604991 learned_addr learned my addr 172.21.15.52:0/2113604991 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe196d9d640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1981076e0 0x7fe198100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe19659c640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe198102480 0x7fe198101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 -- 172.21.15.52:0/2113604991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe198102480 msgr2=0x7fe198101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe198102480 0x7fe198101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 -- 172.21.15.52:0/2113604991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1981076e0 msgr2=0x7fe198100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1981076e0 0x7fe198100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 -- 172.21.15.52:0/2113604991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe198101200 con 0x7fe198104db0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe196d9d640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1981076e0 0x7fe198100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe195d9b640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe198100430 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0314e0 tx=0x7fe18c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe1877fe640 1 -- 172.21.15.52:0/2113604991 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c03f030 con 0x7fe198104db0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe1877fe640 1 -- 172.21.15.52:0/2113604991 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe18c003aa0 con 0x7fe198104db0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe1877fe640 1 -- 172.21.15.52:0/2113604991 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c035070 con 0x7fe198104db0 2024-06-02T23:15:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1981a1c70 con 0x7fe198104db0 2024-06-02T23:15:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.676+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1981a21c0 con 0x7fe198104db0 2024-06-02T23:15:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.679+0000 7fe1877fe640 1 -- 172.21.15.52:0/2113604991 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe18c042030 con 0x7fe198104db0 2024-06-02T23:15:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.679+0000 7fe19f58a740 1 Processor -- start 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.679+0000 7fe19f58a740 1 -- start start 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d4a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d4f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d52000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19f58a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2cad4f0 con 0x55b5e2d4a010 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19f58a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2c4b870 con 0x55b5e2d4fb70 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19f58a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2ca94b0 con 0x55b5e2d4aa00 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d4a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d4a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48234/0 (socket says 172.21.15.52:48234) 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d52000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 -- 172.21.15.52:0/449968809 learned_addr learned my addr 172.21.15.52:0/449968809 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe19659c640 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d4f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe1877fe640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe1800a2cb0 0x7fe1800a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 -- 172.21.15.52:0/449968809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 msgr2=0x55b5e2d4f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d4f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 -- 172.21.15.52:0/449968809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 msgr2=0x55b5e2d52000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d52000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 -- 172.21.15.52:0/449968809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e2c679b0 con 0x55b5e2d4a010 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe195d9b640 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d52000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe1800a2cb0 0x7fe1800a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe1877fe640 1 -- 172.21.15.52:0/2113604991 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe18c0ef410 con 0x7fe198104db0 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.680+0000 7fe196d9d640 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d4a430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe190011940 tx=0x7fe190011e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4674f2be363e17e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.681+0000 7fe184ff9640 1 -- 172.21.15.52:0/449968809 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe190017070 con 0x55b5e2d4a010 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.681+0000 7fe184ff9640 1 -- 172.21.15.52:0/449968809 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1900046a0 con 0x55b5e2d4a010 2024-06-02T23:15:56.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.681+0000 7fe184ff9640 1 -- 172.21.15.52:0/449968809 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe19001c430 con 0x55b5e2d4a010 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 -- 172.21.15.52:0/449968809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 msgr2=0x55b5e2d4a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d4a430 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fe190011940 tx=0x7fe190011e00 comp rx=0 tx=0).stop 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe196d9d640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe1800a2cb0 0x7fe1800a5170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe190021000 tx=0x7fe190022000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 -- 172.21.15.52:0/449968809 shutdown_connections 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d4f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d52000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 --2- 172.21.15.52:0/449968809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d4a430 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 -- 172.21.15.52:0/449968809 >> 172.21.15.52:0/449968809 conn(0x55b5e2d45680 msgr2=0x55b5e2d45a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 -- 172.21.15.52:0/449968809 shutdown_connections 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 -- 172.21.15.52:0/449968809 wait complete. 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 -- start start 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.682+0000 7fe19f58a740 1 Processor -- start 2024-06-02T23:15:56.688 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d54310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d54850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d5b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2d52b20 con 0x55b5e2d4a010 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2c36ae0 con 0x55b5e2d4aa00 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5e2d52ca0 con 0x55b5e2d4fb70 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 10 client.0 _ll_register_callbacks cb 0x55b5e2c6f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d54310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d54310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48250/0 (socket says 172.21.15.52:48250) 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 -- 172.21.15.52:0/2890743087 learned_addr learned my addr 172.21.15.52:0/2890743087 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe196d9d640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d5b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe195d9b640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d54850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 -- 172.21.15.52:0/2890743087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 msgr2=0x55b5e2d5b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d5b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 -- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4aa00 msgr2=0x55b5e2d54850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d54850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 -- 172.21.15.52:0/2890743087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e29c1210 con 0x55b5e2d4a010 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19659c640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d54310 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe188012700 tx=0x7fe188012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b19463dac400b1a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe17dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe17dffb640 1 -- 172.21.15.52:0/2890743087 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18800f8c0 con 0x55b5e2d4a010 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe17dffb640 1 -- 172.21.15.52:0/2890743087 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe18800fa60 con 0x55b5e2d4a010 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.683+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1900046a0 con 0x55b5e2d4a010 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe17dffb640 1 -- 172.21.15.52:0/2890743087 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe18800fc60 con 0x55b5e2d4a010 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 10 client.16728 fetch_fsmap learned FSMap version 32 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b5e2a8d620 con 0x55b5e2d4a010 2024-06-02T23:15:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe17dffb640 1 -- 172.21.15.52:0/2890743087 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe1900046a0 con 0x55b5e2d4a010 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe17dffb640 1 -- 172.21.15.52:0/2890743087 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe188027420 con 0x55b5e2d4a010 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 10 client.16728 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 10 client.16728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 20 client.16728 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2d60240 0x55b5e2d62690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 4 client.16728 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5e2cc0f10 con 0x55b5e2d60240 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.684+0000 7fe195d9b640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2d60240 0x55b5e2d62690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe17dffb640 10 client.16728 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe195d9b640 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2d60240 0x55b5e2d62690 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46d5fee07b49dda7 server_cookie=6745cfa3937061f0 in_seq=0 out_seq=0 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe17dffb640 1 -- 172.21.15.52:0/2890743087 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5e2cc0f10 con 0x55b5e2d60240 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe17dffb640 10 client.16728 handle_command_reply: tid=0 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 client.16728 shutdown 2024-06-02T23:15:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 msgr2=0x55b5e2d54310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d54310 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe188012700 tx=0x7fe188012bc0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 shutdown_connections 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2d60240 0x55b5e2d62690 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d5b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d54850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 --2- 172.21.15.52:0/2890743087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d54310 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe17dffb640 0 client.16728 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.685+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 >> 172.21.15.52:0/2890743087 conn(0x55b5e2d45680 msgr2=0x55b5e2d447e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.686+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 shutdown_connections 2024-06-02T23:15:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.686+0000 7fe19f58a740 1 -- 172.21.15.52:0/2890743087 wait complete. 2024-06-02T23:15:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.686+0000 7fe19f58a740 20 client.16728 trim_cache size 0 max 16384 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.703+0000 7fe19f58a740 1 Processor -- start 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 -- start start 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4a010 0x55b5e2d535e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d5b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d53b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981aaf50 con 0x55b5e2d4aa00 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981aca90 con 0x55b5e2d4a010 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19f58a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981ab620 con 0x55b5e2d4fb70 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d5b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d5b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48254/0 (socket says 172.21.15.52:48254) 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 -- 172.21.15.52:0/219189447 learned_addr learned my addr 172.21.15.52:0/219189447 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe19659c640 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4a010 0x55b5e2d535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe196d9d640 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d53b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 -- 172.21.15.52:0/219189447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 msgr2=0x55b5e2d53b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d53b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 -- 172.21.15.52:0/219189447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4a010 msgr2=0x55b5e2d535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4a010 0x55b5e2d535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 -- 172.21.15.52:0/219189447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5e2d52df0 con 0x55b5e2d4aa00 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.704+0000 7fe195d9b640 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d5b850 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0b81b0 tx=0x7fe18c0ece10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210d9ccf852e27af server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe17ffff640 1 -- 172.21.15.52:0/219189447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c049040 con 0x55b5e2d4aa00 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe17ffff640 1 -- 172.21.15.52:0/219189447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe18c0bbd50 con 0x55b5e2d4aa00 2024-06-02T23:15:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe17ffff640 1 -- 172.21.15.52:0/219189447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0ba0e0 con 0x55b5e2d4aa00 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 -- 172.21.15.52:0/219189447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 msgr2=0x55b5e2d5b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d5b850 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0b81b0 tx=0x7fe18c0ece10 comp rx=0 tx=0).stop 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 -- 172.21.15.52:0/219189447 shutdown_connections 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4fb70 0x55b5e2d53b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d5b850 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 --2- 172.21.15.52:0/219189447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4a010 0x55b5e2d535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 -- 172.21.15.52:0/219189447 >> 172.21.15.52:0/219189447 conn(0x55b5e2d45680 msgr2=0x55b5e2d48560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.705+0000 7fe19f58a740 1 -- 172.21.15.52:0/219189447 shutdown_connections 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 -- 172.21.15.52:0/219189447 wait complete. 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 -- start start 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 Processor -- start 2024-06-02T23:15:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d586d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d58c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2dc8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981ac3c0 con 0x55b5e2d4a010 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981abcf0 con 0x55b5e2d4fb70 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1981a8f50 con 0x55b5e2d4aa00 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19f58a740 10 client.0 _ll_register_callbacks cb 0x7fe188009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d586d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48256/0 (socket says 172.21.15.52:48256) 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe19659c640 1 -- 172.21.15.52:0/1931268495 learned_addr learned my addr 172.21.15.52:0/1931268495 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe196d9d640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2dc8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.706+0000 7fe195d9b640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d58c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19659c640 1 -- 172.21.15.52:0/1931268495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 msgr2=0x55b5e2d58c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19659c640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d58c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19659c640 1 -- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 msgr2=0x55b5e2dc8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19659c640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2dc8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19659c640 1 -- 172.21.15.52:0/1931268495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5e2dc94a0 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe196d9d640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2dc8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19659c640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d586d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe188027ad0 tx=0x7fe188027420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13abdbd7a12d289f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe17dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe17dffb640 1 -- 172.21.15.52:0/1931268495 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18800f790 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe17dffb640 1 -- 172.21.15.52:0/1931268495 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe18800f930 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe18c0bbd50 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe17dffb640 1 -- 172.21.15.52:0/1931268495 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe188036050 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19f58a740 10 client.16734 fetch_fsmap learned FSMap version 32 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b5e2dca760 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.707+0000 7fe17dffb640 1 -- 172.21.15.52:0/1931268495 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe18c0bbd50 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe17dffb640 1 -- 172.21.15.52:0/1931268495 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe188003990 con 0x55b5e2d4a010 2024-06-02T23:15:56.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe19f58a740 10 client.16734 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe19f58a740 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe19f58a740 20 client.16734 populate_metadata read hostname 'smithi052' 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe19f58a740 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2dcb0b0 0x55b5e2dcd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe19f58a740 4 client.16734 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5e2cbac20 con 0x55b5e2dcb0b0 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe195d9b640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2dcb0b0 0x55b5e2dcd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe17dffb640 10 client.16734 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:56.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.708+0000 7fe195d9b640 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2dcb0b0 0x55b5e2dcd4d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adf7b5a50b90b055 server_cookie=2f6318b5684ab8df in_seq=0 out_seq=0 2024-06-02T23:15:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.726+0000 7fe17dffb640 1 -- 172.21.15.52:0/1931268495 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b5e2cbac20 con 0x55b5e2dcb0b0 2024-06-02T23:15:56.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.726+0000 7fe17dffb640 10 client.16734 handle_command_reply: tid=0 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.726+0000 7fe19f58a740 1 client.16734 shutdown 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 msgr2=0x55b5e2d586d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d586d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe188027ad0 tx=0x7fe188027420 comp rx=0 tx=0).stop 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 shutdown_connections 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5e2dcb0b0 0x55b5e2dcd4d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5e2d4aa00 0x55b5e2d58c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5e2d4fb70 0x55b5e2dc8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 --2- 172.21.15.52:0/1931268495 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5e2d4a010 0x55b5e2d586d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe17dffb640 0 client.16734 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 >> 172.21.15.52:0/1931268495 conn(0x55b5e2d45680 msgr2=0x55b5e2d43f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 shutdown_connections 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 1 -- 172.21.15.52:0/1931268495 wait complete. 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19f58a740 20 client.16734 trim_cache size 0 max 16384 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:15:56.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:15:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe1800a2cb0 msgr2=0x7fe1800a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe1800a2cb0 0x7fe1800a5170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe190021000 tx=0x7fe190022000 comp rx=0 tx=0).stop 2024-06-02T23:15:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 msgr2=0x7fe198100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:15:56.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe198100430 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fe18c0314e0 tx=0x7fe18c004270 comp rx=0 tx=0).stop 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 shutdown_connections 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe1800a2cb0 0x7fe1800a5170 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1981076e0 0x7fe198100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.727+0000 7fe19c883640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe198104db0 0x7fe198100430 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.728+0000 7fe19c883640 1 --2- 172.21.15.52:0/2113604991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe198102480 0x7fe198101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.728+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 >> 172.21.15.52:0/2113604991 conn(0x7fe1980fbe70 msgr2=0x7fe198102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.728+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 shutdown_connections 2024-06-02T23:15:56.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:15:56.728+0000 7fe19c883640 1 -- 172.21.15.52:0/2113604991 wait complete. 2024-06-02T23:15:57.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 67266 1.56 MB/sec execute 66 sec latency 18.865 ms 2024-06-02T23:15:57.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:57 smithi123 ceph-mon[35977]: pgmap v553: 97 pgs: 97 active+clean; 46 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:15:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:57 smithi174 ceph-mon[35994]: pgmap v553: 97 pgs: 97 active+clean; 46 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:15:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:57 smithi052 ceph-mon[32340]: pgmap v553: 97 pgs: 97 active+clean; 46 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 284 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:15:57.997 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:15:58.269 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:15:58.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 67743 1.61 MB/sec execute 67 sec latency 14.636 ms 2024-06-02T23:15:59.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 68027 1.59 MB/sec execute 68 sec latency 14.967 ms 2024-06-02T23:15:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:15:59 smithi123 ceph-mon[35977]: pgmap v554: 97 pgs: 97 active+clean; 47 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-02T23:15:59.795 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:15:59 smithi052 ceph-mon[32340]: pgmap v554: 97 pgs: 97 active+clean; 47 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-02T23:15:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:15:59 smithi174 ceph-mon[35994]: pgmap v554: 97 pgs: 97 active+clean; 47 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-02T23:16:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 -- 172.21.15.52:0/3560090864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f418010a6a0 msgr2=0x7f418010aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 --2- 172.21.15.52:0/3560090864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f418010a6a0 0x7f418010aa80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4174009990 tx=0x7f4174009ca0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 -- 172.21.15.52:0/3560090864 shutdown_connections 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 --2- 172.21.15.52:0/3560090864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4180105060 0x7f4180111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 --2- 172.21.15.52:0/3560090864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f418010a6a0 0x7f418010aa80 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 --2- 172.21.15.52:0/3560090864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41801046a0 0x7f4180104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 -- 172.21.15.52:0/3560090864 >> 172.21.15.52:0/3560090864 conn(0x7f4180100170 msgr2=0x7f4180102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 -- 172.21.15.52:0/3560090864 shutdown_connections 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.047+0000 7f41871f8700 1 -- 172.21.15.52:0/3560090864 wait complete. 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 Processor -- start 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 -- start start 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41801046a0 0x7f418019ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4180105060 0x7f418019b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 0x7f418019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4180114880 con 0x7f4180105060 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4180114700 con 0x7f418010a6a0 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f41871f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4180114a00 con 0x7f41801046a0 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f4185795700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 0x7f418019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f4185795700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 0x7f418019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36154/0 (socket says 172.21.15.52:36154) 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f4185795700 1 -- 172.21.15.52:0/1771896435 learned_addr learned my addr 172.21.15.52:0/1771896435 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f4184f94700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41801046a0 0x7f418019ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.048+0000 7f417ffff700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4180105060 0x7f418019b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f4185795700 1 -- 172.21.15.52:0/1771896435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41801046a0 msgr2=0x7f418019ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f4185795700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41801046a0 0x7f418019ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f4185795700 1 -- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4180105060 msgr2=0x7f418019b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f4185795700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4180105060 0x7f418019b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f4185795700 1 -- 172.21.15.52:0/1771896435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418019fd40 con 0x7f418010a6a0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f4185795700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 0x7f418019f600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4178007ee0 tx=0x7f417800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f417dffb700 1 -- 172.21.15.52:0/1771896435 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417800dd20 con 0x7f418010a6a0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f417dffb700 1 -- 172.21.15.52:0/1771896435 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4178004020 con 0x7f418010a6a0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.049+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41801a0030 con 0x7f418010a6a0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.050+0000 7f417dffb700 1 -- 172.21.15.52:0/1771896435 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178015660 con 0x7f418010a6a0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.050+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a78b0 con 0x7f418010a6a0 2024-06-02T23:16:00.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.052+0000 7f417dffb700 1 -- 172.21.15.52:0/1771896435 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4178003a00 con 0x7f418010a6a0 2024-06-02T23:16:00.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.053+0000 7f417dffb700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41700a2ca0 0x7f41700a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.053+0000 7f417dffb700 1 -- 172.21.15.52:0/1771896435 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f41780ce8f0 con 0x7f418010a6a0 2024-06-02T23:16:00.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.053+0000 7f4184f94700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41700a2ca0 0x7f41700a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.054+0000 7f4184f94700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41700a2ca0 0x7f41700a5160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f41740098e0 tx=0x7f4174005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:00.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.054+0000 7f4191a7bb80 1 Processor -- start 2024-06-02T23:16:00.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 -- start start 2024-06-02T23:16:00.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b418f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555b41847810 con 0x555b418fbf60 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b417ef720 con 0x555b418f5f20 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b418437d0 con 0x555b418fb640 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b418f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b418f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48274/0 (socket says 172.21.15.52:48274) 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 -- 172.21.15.52:0/27363450 learned_addr learned my addr 172.21.15.52:0/27363450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4184f94700 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f4185795700 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 -- 172.21.15.52:0/27363450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 msgr2=0x555b418fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 -- 172.21.15.52:0/27363450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 msgr2=0x555b418f6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.055+0000 7f417ffff700 1 -- 172.21.15.52:0/27363450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b417e5f50 con 0x555b418fbf60 2024-06-02T23:16:00.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.056+0000 7f417ffff700 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b418f59e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f416c009d20 tx=0x7f416c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73487d7b6b5437e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:00.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.056+0000 7f416affd700 1 -- 172.21.15.52:0/27363450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c01e070 con 0x555b418fbf60 2024-06-02T23:16:00.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.056+0000 7f416affd700 1 -- 172.21.15.52:0/27363450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f416c019ae0 con 0x555b418fbf60 2024-06-02T23:16:00.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.056+0000 7f416affd700 1 -- 172.21.15.52:0/27363450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c00eac0 con 0x555b418fbf60 2024-06-02T23:16:00.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 -- 172.21.15.52:0/27363450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 msgr2=0x555b418f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b418f59e0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f416c009d20 tx=0x7f416c011720 comp rx=0 tx=0).stop 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 -- 172.21.15.52:0/27363450 shutdown_connections 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/27363450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b418f59e0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 -- 172.21.15.52:0/27363450 >> 172.21.15.52:0/27363450 conn(0x555b41820cc0 msgr2=0x555b418f0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 -- 172.21.15.52:0/27363450 shutdown_connections 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 -- 172.21.15.52:0/27363450 wait complete. 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 -- start start 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.057+0000 7f4191a7bb80 1 Processor -- start 2024-06-02T23:16:00.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b41901c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fb640 0x555b418fef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4184f94700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b41901c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4184f94700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b41901c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36160/0 (socket says 172.21.15.52:36160) 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4184f94700 1 -- 172.21.15.52:0/3550155079 learned_addr learned my addr 172.21.15.52:0/3550155079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f417ffff700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fb640 0x555b418fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fbf60 0x555b418ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555b419029d0 con 0x555b418fb640 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555b417e7bf0 con 0x555b418f5f20 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b41902b50 con 0x555b418fbf60 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4191a7bb80 10 client.0 _ll_register_callbacks cb 0x555b417e1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f4185795700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fbf60 0x555b418ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f417ffff700 1 -- 172.21.15.52:0/3550155079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fbf60 msgr2=0x555b418ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.058+0000 7f417ffff700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fbf60 0x555b418ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f417ffff700 1 -- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 msgr2=0x555b41901c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f417ffff700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b41901c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f417ffff700 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b417968d0 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f417ffff700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fb640 0x555b418fef80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f416c009920 tx=0x7f416c019920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27c806fcdb934a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f415ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f415ffff700 1 -- 172.21.15.52:0/3550155079 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c01e050 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f415ffff700 1 -- 172.21.15.52:0/3550155079 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f416c004030 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416c019ae0 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f415ffff700 1 -- 172.21.15.52:0/3550155079 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f416c004250 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f4191a7bb80 10 client.16746 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b4190d2c0 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.059+0000 7f415ffff700 1 -- 172.21.15.52:0/3550155079 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f416c019ae0 con 0x555b418fb640 2024-06-02T23:16:00.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f415ffff700 1 -- 172.21.15.52:0/3550155079 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f416c02d030 con 0x555b418fb640 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4191a7bb80 10 client.16746 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4191a7bb80 10 client.16746 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4191a7bb80 20 client.16746 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b41911c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4191a7bb80 4 client.16746 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b41854d10 con 0x555b4190f840 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4184f94700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b41911c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f4184f94700 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b41911c30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0a6bd72633d6c7 server_cookie=fdde62deaee48801 in_seq=0 out_seq=0 2024-06-02T23:16:00.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.060+0000 7f415ffff700 10 client.16746 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:00.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f415ffff700 1 -- 172.21.15.52:0/3550155079 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555b41854d10 con 0x555b4190f840 2024-06-02T23:16:00.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f415ffff700 10 client.16746 handle_command_reply: tid=0 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 client.16746 shutdown 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fb640 msgr2=0x555b418fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fb640 0x555b418fef80 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f416c009920 tx=0x7f416c019920 comp rx=0 tx=0).stop 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 shutdown_connections 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b41911c30 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fbf60 0x555b418ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f415ffff700 0 client.16746 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b41901c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/3550155079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fb640 0x555b418fef80 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 >> 172.21.15.52:0/3550155079 conn(0x555b41820cc0 msgr2=0x555b418f49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 shutdown_connections 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 1 -- 172.21.15.52:0/3550155079 wait complete. 2024-06-02T23:16:00.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.062+0000 7f4191a7bb80 20 client.16746 trim_cache size 0 max 16384 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 Processor -- start 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 -- start start 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fcb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b41917350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f41801aec80 con 0x555b418fbf60 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b07c0 con 0x555b418f5f20 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41801af350 con 0x555b418fb640 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4184f94700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4185795700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b41917350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f417ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fcb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4185795700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b41917350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48300/0 (socket says 172.21.15.52:48300) 2024-06-02T23:16:00.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f4185795700 1 -- 172.21.15.52:0/1667513994 learned_addr learned my addr 172.21.15.52:0/1667513994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:00.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f4184f94700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36172/0 (socket says 172.21.15.52:36172) 2024-06-02T23:16:00.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.089+0000 7f417ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fcb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43958/0 (socket says 172.21.15.52:43958) 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f417ffff700 1 -- 172.21.15.52:0/1667513994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 msgr2=0x555b418f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f417ffff700 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f417ffff700 1 -- 172.21.15.52:0/1667513994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 msgr2=0x555b41917350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f417ffff700 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b41917350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f417ffff700 1 -- 172.21.15.52:0/1667513994 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b418fb0f0 con 0x555b418fb640 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f4185795700 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b41917350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f417ffff700 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fcb10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f416c0048b0 tx=0x7f416c001e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0fa8e0d1f211d9b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f416a7fc700 1 -- 172.21.15.52:0/1667513994 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c01e040 con 0x555b418fb640 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f416a7fc700 1 -- 172.21.15.52:0/1667513994 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f416c01f040 con 0x555b418fb640 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.090+0000 7f416a7fc700 1 -- 172.21.15.52:0/1667513994 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416c0051a0 con 0x555b418fb640 2024-06-02T23:16:00.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 -- 172.21.15.52:0/1667513994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 msgr2=0x555b418fcb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fcb10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f416c0048b0 tx=0x7f416c001e50 comp rx=0 tx=0).stop 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 -- 172.21.15.52:0/1667513994 shutdown_connections 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418fbf60 0x555b41917350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418f5f20 0x555b418f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/1667513994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b418fcb10 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 -- 172.21.15.52:0/1667513994 >> 172.21.15.52:0/1667513994 conn(0x555b41820cc0 msgr2=0x555b418fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 -- 172.21.15.52:0/1667513994 shutdown_connections 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.091+0000 7f4191a7bb80 1 -- 172.21.15.52:0/1667513994 wait complete. 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 -- start start 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 Processor -- start 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418f5f20 0x555b41907e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b419083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 0x555b41904e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f41801b00f0 con 0x555b418f5f20 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f41801afa20 con 0x555b418fbf60 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41801acc20 con 0x555b418fb640 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4191a7bb80 10 client.0 _ll_register_callbacks cb 0x555b417e1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4185795700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 0x555b41904e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4184f94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418f5f20 0x555b41907e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4185795700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 0x555b41904e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36184/0 (socket says 172.21.15.52:36184) 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f4185795700 1 -- 172.21.15.52:0/263329159 learned_addr learned my addr 172.21.15.52:0/263329159 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.092+0000 7f417ffff700 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b419083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4185795700 1 -- 172.21.15.52:0/263329159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 msgr2=0x555b419083b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4185795700 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b419083b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4185795700 1 -- 172.21.15.52:0/263329159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418f5f20 msgr2=0x555b41907e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4185795700 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418f5f20 0x555b41907e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4185795700 1 -- 172.21.15.52:0/263329159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b41908a00 con 0x555b418fbf60 2024-06-02T23:16:00.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4185795700 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 0x555b41904e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4178097590 tx=0x7f41780cc2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c9f6e97bbaed37c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f415ffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f415ffff700 1 -- 172.21.15.52:0/263329159 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41780a3df0 con 0x555b418fbf60 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f416c01f040 con 0x555b418fbf60 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.093+0000 7f415ffff700 1 -- 172.21.15.52:0/263329159 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f41780bb0b0 con 0x555b418fbf60 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f415ffff700 1 -- 172.21.15.52:0/263329159 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f417809f160 con 0x555b418fbf60 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 10 client.26288 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555b4197a8b0 con 0x555b418fbf60 2024-06-02T23:16:00.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f415ffff700 1 -- 172.21.15.52:0/263329159 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f416c01f040 con 0x555b418fbf60 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f415ffff700 1 -- 172.21.15.52:0/263329159 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4178004bd0 con 0x555b418fbf60 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 10 client.26288 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 10 client.26288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 20 client.26288 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b4197cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 4 client.26288 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555b4184f290 con 0x555b4190f840 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.094+0000 7f4184f94700 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b4197cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f415ffff700 10 client.26288 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4184f94700 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b4197cf70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2503a09b8e3ca8b7 server_cookie=93b511a7da62759e in_seq=0 out_seq=0 2024-06-02T23:16:00.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f415ffff700 1 -- 172.21.15.52:0/263329159 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555b4184f290 con 0x555b4190f840 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f415ffff700 10 client.26288 handle_command_reply: tid=0 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 client.26288 shutdown 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 msgr2=0x555b41904e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 0x555b41904e40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4178097590 tx=0x7f41780cc2f0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 shutdown_connections 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555b4190f840 0x555b4197cf70 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555b418fbf60 0x555b41904e40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555b418f5f20 0x555b41907e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f415ffff700 0 client.26288 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 --2- 172.21.15.52:0/263329159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b418fb640 0x555b419083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 >> 172.21.15.52:0/263329159 conn(0x555b41820cc0 msgr2=0x555b418e9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 shutdown_connections 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 1 -- 172.21.15.52:0/263329159 wait complete. 2024-06-02T23:16:00.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.095+0000 7f4191a7bb80 20 client.26288 trim_cache size 0 max 16384 2024-06-02T23:16:00.100 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:16:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:16:00.100 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:16:00.100 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41700a2ca0 msgr2=0x7f41700a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41700a2ca0 0x7f41700a5160 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f41740098e0 tx=0x7f4174005960 comp rx=0 tx=0).stop 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 msgr2=0x7f418019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 0x7f418019f600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f4178007ee0 tx=0x7f417800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 shutdown_connections 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f41700a2ca0 0x7f41700a5160 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f418010a6a0 0x7f418019f600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41801046a0 0x7f418019ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 --2- 172.21.15.52:0/1771896435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4180105060 0x7f418019b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 >> 172.21.15.52:0/1771896435 conn(0x7f4180100170 msgr2=0x7f4180100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:00.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 shutdown_connections 2024-06-02T23:16:00.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:00.096+0000 7f41871f8700 1 -- 172.21.15.52:0/1771896435 wait complete. 2024-06-02T23:16:00.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 68556 1.59 MB/sec execute 69 sec latency 14.336 ms 2024-06-02T23:16:00.834 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:16:00.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:16:01.115 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:01.496 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:01 smithi052 ceph-mon[32340]: pgmap v555: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-02T23:16:01.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 69106 1.62 MB/sec execute 70 sec latency 17.730 ms 2024-06-02T23:16:01.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:01 smithi123 ceph-mon[35977]: pgmap v555: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-02T23:16:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:01 smithi174 ceph-mon[35994]: pgmap v555: 97 pgs: 97 active+clean; 46 MiB data, 698 MiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 1.4 MiB/s wr, 164 op/s 2024-06-02T23:16:02.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 69779 1.62 MB/sec execute 71 sec latency 14.832 ms 2024-06-02T23:16:03.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 -- 172.21.15.52:0/40588812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac06b3c0 msgr2=0x7fbaac10f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 --2- 172.21.15.52:0/40588812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac06b3c0 0x7fbaac10f310 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b0a0 tx=0x7fbaa400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 -- 172.21.15.52:0/40588812 shutdown_connections 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 --2- 172.21.15.52:0/40588812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac06b3c0 0x7fbaac10f310 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 --2- 172.21.15.52:0/40588812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac1084f0 0x7fbaac1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 --2- 172.21.15.52:0/40588812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaac06aa00 0x7fbaac06ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 -- 172.21.15.52:0/40588812 >> 172.21.15.52:0/40588812 conn(0x7fbaac078300 msgr2=0x7fbaac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 -- 172.21.15.52:0/40588812 shutdown_connections 2024-06-02T23:16:03.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.126+0000 7fbab479b700 1 -- 172.21.15.52:0/40588812 wait complete. 2024-06-02T23:16:03.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 Processor -- start 2024-06-02T23:16:03.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 -- start start 2024-06-02T23:16:03.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 0x7fbaac19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaac06b3c0 0x7fbaac19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 0x7fbaac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac112620 con 0x7fbaac1084f0 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1124a0 con 0x7fbaac06aa00 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.127+0000 7fbab479b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbaac1127a0 con 0x7fbaac06b3c0 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 0x7fbaac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2537700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 0x7fbaac19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2537700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 0x7fbaac19e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36202/0 (socket says 172.21.15.52:36202) 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 0x7fbaac198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48326/0 (socket says 172.21.15.52:48326) 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2537700 1 -- 172.21.15.52:0/3367953876 learned_addr learned my addr 172.21.15.52:0/3367953876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab1d36700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaac06b3c0 0x7fbaac19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 -- 172.21.15.52:0/3367953876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaac06b3c0 msgr2=0x7fbaac19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:03.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaac06b3c0 0x7fbaac19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 -- 172.21.15.52:0/3367953876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 msgr2=0x7fbaac19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 0x7fbaac19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 -- 172.21.15.52:0/3367953876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaac1990e0 con 0x7fbaac1084f0 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2537700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 0x7fbaac19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab2d38700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 0x7fbaac198980 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b070 tx=0x7fbaa4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa4021070 con 0x7fbaac1084f0 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaac199370 con 0x7fbaac1084f0 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbaa4007e30 con 0x7fbaac1084f0 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa40045c0 con 0x7fbaac1084f0 2024-06-02T23:16:03.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.128+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaac1a5690 con 0x7fbaac1084f0 2024-06-02T23:16:03.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.130+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaac069180 con 0x7fbaac1084f0 2024-06-02T23:16:03.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.131+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbaa4004760 con 0x7fbaac1084f0 2024-06-02T23:16:03.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.132+0000 7fbaa37fe700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba980a2c20 0x7fba980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.132+0000 7fbab2537700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba980a2c20 0x7fba980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.132+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbaa400f020 con 0x7fbaac1084f0 2024-06-02T23:16:03.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.132+0000 7fbab2537700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba980a2c20 0x7fba980a50e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbaac199ac0 tx=0x7fbaa8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:03.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.136+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbaa402a020 con 0x7fbaac1084f0 2024-06-02T23:16:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.446+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaac199eb0 con 0x7fbaac1084f0 2024-06-02T23:16:03.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.447+0000 7fbaa37fe700 1 -- 172.21.15.52:0/3367953876 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fbaa409d6b0 con 0x7fbaac1084f0 2024-06-02T23:16:03.448 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:16:03.448 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:16:03.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba980a2c20 msgr2=0x7fba980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba980a2c20 0x7fba980a50e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbaac199ac0 tx=0x7fbaa8009260 comp rx=0 tx=0).stop 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 msgr2=0x7fbaac198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 0x7fbaac198980 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fbaa400b070 tx=0x7fbaa4004d00 comp rx=0 tx=0).stop 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 shutdown_connections 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba980a2c20 0x7fba980a50e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbaac1084f0 0x7fbaac198980 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbaac06aa00 0x7fbaac19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 --2- 172.21.15.52:0/3367953876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbaac06b3c0 0x7fbaac19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 >> 172.21.15.52:0/3367953876 conn(0x7fbaac078300 msgr2=0x7fbaac1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 shutdown_connections 2024-06-02T23:16:03.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:03.450+0000 7fbab479b700 1 -- 172.21.15.52:0/3367953876 wait complete. 2024-06-02T23:16:03.452 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:16:03.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 70335 1.67 MB/sec execute 72 sec latency 60.969 ms 2024-06-02T23:16:03.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:03 smithi123 ceph-mon[35977]: pgmap v556: 97 pgs: 97 active+clean; 45 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-02T23:16:03.820 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:03 smithi052 ceph-mon[32340]: pgmap v556: 97 pgs: 97 active+clean; 45 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-02T23:16:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:03 smithi174 ceph-mon[35994]: pgmap v556: 97 pgs: 97 active+clean; 45 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 318 KiB/s rd, 1.5 MiB/s wr, 188 op/s 2024-06-02T23:16:04.242 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:16:04.520 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:04.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 70807 1.68 MB/sec execute 73 sec latency 11.562 ms 2024-06-02T23:16:04.645 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:04 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3367953876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:16:04.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:04 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3367953876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:16:04.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:04 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3367953876' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:16:05.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 71278 1.71 MB/sec execute 74 sec latency 15.109 ms 2024-06-02T23:16:05.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:05 smithi123 ceph-mon[35977]: pgmap v557: 97 pgs: 97 active+clean; 44 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:16:05.775 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:05 smithi052 ceph-mon[32340]: pgmap v557: 97 pgs: 97 active+clean; 44 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:16:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:05 smithi174 ceph-mon[35994]: pgmap v557: 97 pgs: 97 active+clean; 44 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 235 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 -- 172.21.15.52:0/1008371972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c102540 msgr2=0x7f331c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 --2- 172.21.15.52:0/1008371972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c102540 0x7f331c1029a0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f3310009990 tx=0x7f3310009ca0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 -- 172.21.15.52:0/1008371972 shutdown_connections 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 --2- 172.21.15.52:0/1008371972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331c102ee0 0x7f331c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 --2- 172.21.15.52:0/1008371972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c107450 0x7f331c107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 --2- 172.21.15.52:0/1008371972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c102540 0x7f331c1029a0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 -- 172.21.15.52:0/1008371972 >> 172.21.15.52:0/1008371972 conn(0x7f331c0795f0 msgr2=0x7f331c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 -- 172.21.15.52:0/1008371972 shutdown_connections 2024-06-02T23:16:06.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.338+0000 7f3320f88700 1 -- 172.21.15.52:0/1008371972 wait complete. 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 Processor -- start 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 -- start start 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 0x7f331c19ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331c102ee0 0x7f331c19b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 0x7f331c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f331c112680 con 0x7f331c107450 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f331c112500 con 0x7f331c102ee0 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.339+0000 7f3320f88700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f331c112800 con 0x7f331c102540 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 0x7f331c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 0x7f331c19ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 0x7f331c19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56324/0 (socket says 172.21.15.52:56324) 2024-06-02T23:16:06.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331ad9d700 1 -- 172.21.15.52:0/610508943 learned_addr learned my addr 172.21.15.52:0/610508943 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 0x7f331c19ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52878/0 (socket says 172.21.15.52:52878) 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f3319d9b700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331c102ee0 0x7f331c19b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 -- 172.21.15.52:0/610508943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331c102ee0 msgr2=0x7f331c19b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331c102ee0 0x7f331c19b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 -- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 msgr2=0x7f331c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 0x7f331c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 -- 172.21.15.52:0/610508943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f331c19fd40 con 0x7f331c102540 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331ad9d700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 0x7f331c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.340+0000 7f331a59c700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 0x7f331c19ad80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f330c00e9c0 tx=0x7f330c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.341+0000 7f330b7fe700 1 -- 172.21.15.52:0/610508943 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c004220 con 0x7f331c102540 2024-06-02T23:16:06.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.341+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f331c1a0030 con 0x7f331c102540 2024-06-02T23:16:06.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.341+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f331c1a78b0 con 0x7f331c102540 2024-06-02T23:16:06.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.341+0000 7f330b7fe700 1 -- 172.21.15.52:0/610508943 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f330c0043c0 con 0x7f331c102540 2024-06-02T23:16:06.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.341+0000 7f330b7fe700 1 -- 172.21.15.52:0/610508943 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f330c003c40 con 0x7f331c102540 2024-06-02T23:16:06.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.343+0000 7f330b7fe700 1 -- 172.21.15.52:0/610508943 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f330c018430 con 0x7f331c102540 2024-06-02T23:16:06.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.344+0000 7f330b7fe700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33040a2ca0 0x7f33040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.344+0000 7f3319d9b700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33040a2ca0 0x7f33040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.344+0000 7f330b7fe700 1 -- 172.21.15.52:0/610508943 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f330c0cefa0 con 0x7f331c102540 2024-06-02T23:16:06.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.345+0000 7f3319d9b700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33040a2ca0 0x7f33040a5160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f331000b2e0 tx=0x7f3310005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:06.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.345+0000 7f332b80bb80 1 Processor -- start 2024-06-02T23:16:06.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.345+0000 7f332b80bb80 1 -- start start 2024-06-02T23:16:06.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.345+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aa640 0x55dead6aaa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.345+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aaf60 0x55dead6a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 0x55dead6a5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f332b80bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dead5f6810 con 0x55dead6a4f20 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f332b80bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dead59e720 con 0x55dead6aa640 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 0x55dead6a5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 0x55dead6a5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56340/0 (socket says 172.21.15.52:56340) 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 -- 172.21.15.52:0/404175264 learned_addr learned my addr 172.21.15.52:0/404175264 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f332b80bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dead5f27d0 con 0x55dead6aaf60 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f3319d9b700 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aaf60 0x55dead6a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331a59c700 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aa640 0x55dead6aaa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 -- 172.21.15.52:0/404175264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aaf60 msgr2=0x55dead6a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aaf60 0x55dead6a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 -- 172.21.15.52:0/404175264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aa640 msgr2=0x55dead6aaa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aa640 0x55dead6aaa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 -- 172.21.15.52:0/404175264 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dead594f50 con 0x55dead6a4f20 2024-06-02T23:16:06.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331a59c700 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aa640 0x55dead6aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:06.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.346+0000 7f331ad9d700 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 0x55dead6a5340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3314009550 tx=0x7f3314009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25d5ac27470d9ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.347+0000 7f3308ff9700 1 -- 172.21.15.52:0/404175264 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331401b020 con 0x55dead6a4f20 2024-06-02T23:16:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.347+0000 7f3308ff9700 1 -- 172.21.15.52:0/404175264 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314007bb0 con 0x55dead6a4f20 2024-06-02T23:16:06.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.347+0000 7f3308ff9700 1 -- 172.21.15.52:0/404175264 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314017d20 con 0x55dead6a4f20 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.347+0000 7f332b80bb80 1 -- 172.21.15.52:0/404175264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 msgr2=0x55dead6a5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.347+0000 7f332b80bb80 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 0x55dead6a5340 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3314009550 tx=0x7f3314009920 comp rx=0 tx=0).stop 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 -- 172.21.15.52:0/404175264 shutdown_connections 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6a4f20 0x55dead6a5340 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aa640 0x55dead6aaa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 --2- 172.21.15.52:0/404175264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aaf60 0x55dead6a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 -- 172.21.15.52:0/404175264 >> 172.21.15.52:0/404175264 conn(0x55dead5cfcc0 msgr2=0x55dead69f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 -- 172.21.15.52:0/404175264 shutdown_connections 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 -- 172.21.15.52:0/404175264 wait complete. 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 -- start start 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.348+0000 7f332b80bb80 1 Processor -- start 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ae4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f3319d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f3319d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52886/0 (socket says 172.21.15.52:52886) 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:54004/0 (socket says 172.21.15.52:54004) 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ae4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ae4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56342/0 (socket says 172.21.15.52:56342) 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f3319d9b700 1 -- 172.21.15.52:0/3769077396 learned_addr learned my addr 172.21.15.52:0/3769077396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dead6b1a20 con 0x55dead6aaf60 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dead596bf0 con 0x55dead6a4f20 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dead6b1ba0 con 0x55dead6aa640 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f332b80bb80 10 client.0 _ll_register_callbacks cb 0x55dead590920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 -- 172.21.15.52:0/3769077396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 msgr2=0x55dead6adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 -- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 msgr2=0x55dead6b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.349+0000 7f331ad9d700 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dead5458d0 con 0x55dead6aaf60 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314007bb0 con 0x55dead6aaf60 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f331ad9d700 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ae4f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3314009550 tx=0x7f3314008fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677535e76278bdc9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f3301ffb700 10 client.16770 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f3319d9b700 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f3301ffb700 1 -- 172.21.15.52:0/3769077396 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331401b040 con 0x55dead6aaf60 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f3301ffb700 1 -- 172.21.15.52:0/3769077396 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3314027dc0 con 0x55dead6aaf60 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f3301ffb700 1 -- 172.21.15.52:0/3769077396 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3314017460 con 0x55dead6aaf60 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f332b80bb80 10 client.16770 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=32}) -- 0x55dead6bc260 con 0x55dead6aaf60 2024-06-02T23:16:06.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.350+0000 7f3301ffb700 1 -- 172.21.15.52:0/3769077396 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f33140179a0 con 0x55dead6aaf60 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f332b80bb80 10 client.16770 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f332b80bb80 10 client.16770 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f332b80bb80 20 client.16770 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f332b80bb80 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead6c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f332b80bb80 4 client.16770 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dead603d10 con 0x55dead6be730 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f331a59c700 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead6c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f331a59c700 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead6c0b20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=63e8b3e2755295b3 server_cookie=4060490f613913f7 in_seq=0 out_seq=0 2024-06-02T23:16:06.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.351+0000 7f3301ffb700 10 client.16770 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:16:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f3301ffb700 1 -- 172.21.15.52:0/3769077396 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dead603d10 con 0x55dead6be730 2024-06-02T23:16:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f3301ffb700 10 client.16770 handle_command_reply: tid=0 2024-06-02T23:16:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 client.16770 shutdown 2024-06-02T23:16:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 msgr2=0x55dead6ae4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ae4f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3314009550 tx=0x7f3314008fa0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 shutdown_connections 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ae4f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead6c0b20 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 --2- 172.21.15.52:0/3769077396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f3301ffb700 0 client.16770 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 >> 172.21.15.52:0/3769077396 conn(0x55dead5cfcc0 msgr2=0x55dead6a39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 shutdown_connections 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 1 -- 172.21.15.52:0/3769077396 wait complete. 2024-06-02T23:16:06.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.352+0000 7f332b80bb80 20 client.16770 trim_cache size 0 max 16384 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 Processor -- start 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 -- start start 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6a0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ac8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc00d260 con 0x55dead6aaf60 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc00eda0 con 0x55dead6a4f20 2024-06-02T23:16:06.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f332b80bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc00d930 con 0x55dead6aa640 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ac8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:54006/0 (socket says 172.21.15.52:54006) 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331a59c700 1 -- 172.21.15.52:0/199526330 learned_addr learned my addr 172.21.15.52:0/199526330 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f3319d9b700 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6a0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331ad9d700 1 -- 172.21.15.52:0/199526330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 msgr2=0x55dead6a0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331ad9d700 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6a0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331ad9d700 1 -- 172.21.15.52:0/199526330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 msgr2=0x55dead6a49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331ad9d700 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6a49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331ad9d700 1 -- 172.21.15.52:0/199526330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dead6b1490 con 0x55dead6aaf60 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.371+0000 7f331a59c700 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.372+0000 7f331ad9d700 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ac8a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3314017620 tx=0x7f3314016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159e90ee64004845 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.372+0000 7f3303fff700 1 -- 172.21.15.52:0/199526330 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f331401b040 con 0x55dead6aaf60 2024-06-02T23:16:06.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.372+0000 7f3303fff700 1 -- 172.21.15.52:0/199526330 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314007bb0 con 0x55dead6aaf60 2024-06-02T23:16:06.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.372+0000 7f3303fff700 1 -- 172.21.15.52:0/199526330 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3314007880 con 0x55dead6aaf60 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 -- 172.21.15.52:0/199526330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 msgr2=0x55dead6ac8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ac8a0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3314017620 tx=0x7f3314016000 comp rx=0 tx=0).stop 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 -- 172.21.15.52:0/199526330 shutdown_connections 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aaf60 0x55dead6ac8a0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6a4f20 0x55dead6a49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 --2- 172.21.15.52:0/199526330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6aa640 0x55dead6a0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 -- 172.21.15.52:0/199526330 >> 172.21.15.52:0/199526330 conn(0x55dead5cfcc0 msgr2=0x55dead69f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 -- 172.21.15.52:0/199526330 shutdown_connections 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 -- 172.21.15.52:0/199526330 wait complete. 2024-06-02T23:16:06.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 -- start start 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.373+0000 7f332b80bb80 1 Processor -- start 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6a4f20 0x55dead6b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 0x55dead6b73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aaf60 0x55dead6c5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc00e6d0 con 0x55dead6aa640 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc00e000 con 0x55dead6aaf60 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32fc00b200 con 0x55dead6a4f20 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f332b80bb80 10 client.0 _ll_register_callbacks cb 0x7f330c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 0x55dead6b73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 0x55dead6b73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56360/0 (socket says 172.21.15.52:56360) 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f331ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aaf60 0x55dead6c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 -- 172.21.15.52:0/2861329958 learned_addr learned my addr 172.21.15.52:0/2861329958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f331a59c700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6a4f20 0x55dead6b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 -- 172.21.15.52:0/2861329958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6a4f20 msgr2=0x55dead6b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6a4f20 0x55dead6b6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 -- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aaf60 msgr2=0x55dead6c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aaf60 0x55dead6c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 -- 172.21.15.52:0/2861329958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dead6c5770 con 0x55dead6aa640 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f331a59c700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6a4f20 0x55dead6b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f331ad9d700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aaf60 0x55dead6c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3319d9b700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 0x55dead6b73d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3310005cd0 tx=0x7f3310009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c3dd6166e6f1d47 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.374+0000 7f3301ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f3301ffb700 1 -- 172.21.15.52:0/2861329958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3310003ce0 con 0x55dead6aa640 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f3301ffb700 1 -- 172.21.15.52:0/2861329958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f331000f460 con 0x55dead6aa640 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3314007bb0 con 0x55dead6aa640 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f3301ffb700 1 -- 172.21.15.52:0/2861329958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f331000f660 con 0x55dead6aa640 2024-06-02T23:16:06.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 10 client.16776 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dead7288c0 con 0x55dead6aa640 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f3301ffb700 1 -- 172.21.15.52:0/2861329958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3314007bb0 con 0x55dead6aa640 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f3301ffb700 1 -- 172.21.15.52:0/2861329958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f33100267f0 con 0x55dead6aa640 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 10 client.16776 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 10 client.16776 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 20 client.16776 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead72b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 4 client.16776 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.375+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dead5fe290 con 0x55dead6be730 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f331a59c700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead72b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f3301ffb700 10 client.16776 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f331a59c700 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead72b090 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d32b4970311af378 server_cookie=4e80f2caad852528 in_seq=0 out_seq=0 2024-06-02T23:16:06.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f3301ffb700 1 -- 172.21.15.52:0/2861329958 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dead5fe290 con 0x55dead6be730 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f3301ffb700 10 client.16776 handle_command_reply: tid=0 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 client.16776 shutdown 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 msgr2=0x55dead6b73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 0x55dead6b73d0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3310005cd0 tx=0x7f3310009ca0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 shutdown_connections 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dead6be730 0x55dead72b090 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dead6aaf60 0x55dead6c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dead6a4f20 0x55dead6b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 --2- 172.21.15.52:0/2861329958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dead6aa640 0x55dead6b73d0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 >> 172.21.15.52:0/2861329958 conn(0x55dead5cfcc0 msgr2=0x55dead6ac860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f3301ffb700 0 client.16776 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 shutdown_connections 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 1 -- 172.21.15.52:0/2861329958 wait complete. 2024-06-02T23:16:06.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.376+0000 7f332b80bb80 20 client.16776 trim_cache size 0 max 16384 2024-06-02T23:16:06.382 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:16:06.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33040a2ca0 msgr2=0x7f33040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.382 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33040a2ca0 0x7f33040a5160 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f331000b2e0 tx=0x7f3310005960 comp rx=0 tx=0).stop 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 msgr2=0x7f331c19ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 0x7f331c19ad80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f330c00e9c0 tx=0x7f330c00ecd0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 shutdown_connections 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33040a2ca0 0x7f33040a5160 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f331c107450 0x7f331c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f331c102540 0x7f331c19ad80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 --2- 172.21.15.52:0/610508943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f331c102ee0 0x7f331c19b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 >> 172.21.15.52:0/610508943 conn(0x7f331c0795f0 msgr2=0x7f331c1003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 shutdown_connections 2024-06-02T23:16:06.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:06.377+0000 7f3320f88700 1 -- 172.21.15.52:0/610508943 wait complete. 2024-06-02T23:16:06.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 71558 1.69 MB/sec execute 75 sec latency 19.139 ms 2024-06-02T23:16:07.110 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:16:07.383 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:07.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 72059 1.70 MB/sec execute 76 sec latency 15.290 ms 2024-06-02T23:16:07.749 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:07 smithi052 ceph-mon[32340]: pgmap v558: 97 pgs: 97 active+clean; 45 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-06-02T23:16:07.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:07 smithi123 ceph-mon[35977]: pgmap v558: 97 pgs: 97 active+clean; 45 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-06-02T23:16:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:07 smithi174 ceph-mon[35994]: pgmap v558: 97 pgs: 97 active+clean; 45 MiB data, 693 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.6 MiB/s wr, 174 op/s 2024-06-02T23:16:08.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 72649 1.72 MB/sec execute 77 sec latency 14.582 ms 2024-06-02T23:16:09.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 -- 172.21.15.52:0/3378878208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 msgr2=0x7f5cc4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 --2- 172.21.15.52:0/3378878208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4101810 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f5cac009990 tx=0x7f5cac009ca0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 -- 172.21.15.52:0/3378878208 shutdown_connections 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 --2- 172.21.15.52:0/3378878208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc4101d50 0x7f5cc410e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 --2- 172.21.15.52:0/3378878208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4107400 0x7f5cc41077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 --2- 172.21.15.52:0/3378878208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4101810 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 -- 172.21.15.52:0/3378878208 >> 172.21.15.52:0/3378878208 conn(0x7f5cc4078340 msgr2=0x7f5cc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 -- 172.21.15.52:0/3378878208 shutdown_connections 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.191+0000 7f5cca779700 1 -- 172.21.15.52:0/3378878208 wait complete. 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.192+0000 7f5cca779700 1 Processor -- start 2024-06-02T23:16:09.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.192+0000 7f5cca779700 1 -- start start 2024-06-02T23:16:09.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.192+0000 7f5cca779700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.192+0000 7f5cca779700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4101d50 0x7f5cc41035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cca779700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc4107400 0x7f5cc4103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cca779700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc4111520 con 0x7f5cc4101390 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cca779700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41113a0 con 0x7f5cc4107400 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cca779700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cc41116a0 con 0x7f5cc4101d50 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56384/0 (socket says 172.21.15.52:56384) 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1782495583 learned_addr learned my addr 172.21.15.52:0/1782495583 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc4107400 0x7f5cc4103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4101d50 0x7f5cc41035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1782495583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4101d50 msgr2=0x7f5cc41035b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4101d50 0x7f5cc41035b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1782495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc4107400 msgr2=0x7f5cc4103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc4107400 0x7f5cc4103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1782495583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cc41a7080 con 0x7f5cc4101390 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4101d50 0x7f5cc41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.193+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4103070 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e9a0 tx=0x7f5cb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.194+0000 7f5cc1ffb700 1 -- 172.21.15.52:0/1782495583 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400cbd0 con 0x7f5cc4101390 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.194+0000 7f5cc1ffb700 1 -- 172.21.15.52:0/1782495583 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb400cd70 con 0x7f5cc4101390 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.194+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cc41a7370 con 0x7f5cc4101390 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.194+0000 7f5cc1ffb700 1 -- 172.21.15.52:0/1782495583 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4018680 con 0x7f5cc4101390 2024-06-02T23:16:09.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.194+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cc41a78b0 con 0x7f5cc4101390 2024-06-02T23:16:09.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.197+0000 7f5cc1ffb700 1 -- 172.21.15.52:0/1782495583 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5cb40188e0 con 0x7f5cc4101390 2024-06-02T23:16:09.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.198+0000 7f5cc1ffb700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb00a2d70 0x7f5cb00a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.198+0000 7f5cc1ffb700 1 -- 172.21.15.52:0/1782495583 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cb40cefa0 con 0x7f5cc4101390 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.198+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb00a2d70 0x7f5cb00a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.198+0000 7f5cd4ffcb80 1 Processor -- start 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 -- start start 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb00a2d70 0x7f5cb00a5230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b2e0 tx=0x7f5cac01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded53a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d2dec9f810 con 0x55d2ded53640 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d2dec47720 con 0x55d2ded53f60 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2dec9b7d0 con 0x55d2ded4df20 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded53a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded53a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56394/0 (socket says 172.21.15.52:56394) 2024-06-02T23:16:09.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1821845658 learned_addr learned my addr 172.21.15.52:0/1821845658 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1821845658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 msgr2=0x55d2ded4e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1821845658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 msgr2=0x55d2ded4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cc3fff700 1 -- 172.21.15.52:0/1821845658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2dec3df50 con 0x55d2ded53640 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.199+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:09.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.200+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:16:09.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.200+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded53a20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40982a0 tx=0x7f5cb409b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18151052bd718b59 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.200+0000 7f5cba7fc700 1 -- 172.21.15.52:0/1821845658 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb409b8d0 con 0x55d2ded53640 2024-06-02T23:16:09.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.200+0000 7f5cba7fc700 1 -- 172.21.15.52:0/1821845658 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb409fc80 con 0x55d2ded53640 2024-06-02T23:16:09.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.200+0000 7f5cba7fc700 1 -- 172.21.15.52:0/1821845658 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb40a4310 con 0x55d2ded53640 2024-06-02T23:16:09.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/1821845658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 msgr2=0x55d2ded53a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded53a20 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5cb40982a0 tx=0x7f5cb409b230 comp rx=0 tx=0).stop 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/1821845658 shutdown_connections 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded53a20 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/1821845658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/1821845658 >> 172.21.15.52:0/1821845658 conn(0x55d2dec78cc0 msgr2=0x55d2ded48850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/1821845658 shutdown_connections 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/1821845658 wait complete. 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.201+0000 7f5cd4ffcb80 1 -- start start 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 Processor -- start 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded4df20 0x55d2ded59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded56fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 0x55d2ded57520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d2ded5a9d0 con 0x55d2ded53f60 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d2dec3fbf0 con 0x55d2ded4df20 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2ded5ab50 con 0x55d2ded53640 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cd4ffcb80 10 client.0 _ll_register_callbacks cb 0x55d2dec39920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cbbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded56fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cc8d16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 0x55d2ded57520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded4df20 0x55d2ded59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cbbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded56fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52940/0 (socket says 172.21.15.52:52940) 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cc8d16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 0x55d2ded57520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56398/0 (socket says 172.21.15.52:56398) 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.202+0000 7f5cbbfff700 1 -- 172.21.15.52:0/4061968919 learned_addr learned my addr 172.21.15.52:0/4061968919 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 msgr2=0x55d2ded56fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded56fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4061968919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 msgr2=0x55d2ded57520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 0x55d2ded57520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4061968919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2debee8d0 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 0x55d2ded57520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded4df20 0x55d2ded59c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5cb409b5d0 tx=0x7f5cb40ccb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8996c1efead4380d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5ca37fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4061968919 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb4099bd0 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4061968919 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cb40a40b0 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cb409fc80 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4061968919 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5cb40ba2b0 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cd4ffcb80 10 client.26324 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d2ded65300 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.203+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4061968919 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cb409fc80 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4061968919 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5cb40b4ad0 con 0x55d2ded4df20 2024-06-02T23:16:09.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cd4ffcb80 10 client.26324 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cd4ffcb80 10 client.26324 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cd4ffcb80 20 client.26324 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2ded69bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cd4ffcb80 4 client.26324 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2decacd10 con 0x55d2ded677d0 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2ded69bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2ded69bc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=765a6d8afc338cd3 server_cookie=617a5eb14bd59e65 in_seq=0 out_seq=0 2024-06-02T23:16:09.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.204+0000 7f5ca37fe700 10 client.26324 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4061968919 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d2decacd10 con 0x55d2ded677d0 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5ca37fe700 10 client.26324 handle_command_reply: tid=0 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 client.26324 shutdown 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded4df20 msgr2=0x55d2ded59c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded4df20 0x55d2ded59c90 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5cb409b5d0 tx=0x7f5cb40ccb10 comp rx=0 tx=0).stop 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 shutdown_connections 2024-06-02T23:16:09.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53f60 0x55d2ded57520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded4df20 0x55d2ded59c90 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2ded69bc0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4061968919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded56fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5ca37fe700 0 client.26324 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 >> 172.21.15.52:0/4061968919 conn(0x55d2dec78cc0 msgr2=0x55d2ded4c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 shutdown_connections 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.205+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4061968919 wait complete. 2024-06-02T23:16:09.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.206+0000 7f5cd4ffcb80 20 client.26324 trim_cache size 0 max 16384 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.223+0000 7f5cd4ffcb80 1 Processor -- start 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.223+0000 7f5cd4ffcb80 1 -- start start 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded54a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400d2a0 con 0x55d2ded53640 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400ede0 con 0x55d2ded53f60 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400d970 con 0x55d2ded4df20 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cbbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded54a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc8d16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cbbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded54a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56402/0 (socket says 172.21.15.52:56402) 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cbbfff700 1 -- 172.21.15.52:0/3088547086 learned_addr learned my addr 172.21.15.52:0/3088547086 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc8d16700 1 -- 172.21.15.52:0/3088547086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 msgr2=0x55d2ded4d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc8d16700 1 -- 172.21.15.52:0/3088547086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 msgr2=0x55d2ded54a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded54a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cc8d16700 1 -- 172.21.15.52:0/3088547086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ded57f80 con 0x55d2ded53f60 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.224+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded54a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.225+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6cbe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007850 tx=0x7f5cbc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ed4d9c9152de33 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.225+0000 7f5cb9ffb700 1 -- 172.21.15.52:0/3088547086 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc004db0 con 0x55d2ded53f60 2024-06-02T23:16:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.225+0000 7f5cb9ffb700 1 -- 172.21.15.52:0/3088547086 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc004030 con 0x55d2ded53f60 2024-06-02T23:16:09.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.225+0000 7f5cb9ffb700 1 -- 172.21.15.52:0/3088547086 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cbc005280 con 0x55d2ded53f60 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/3088547086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 msgr2=0x55d2ded6cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6cbe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc007850 tx=0x7f5cbc009920 comp rx=0 tx=0).stop 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/3088547086 shutdown_connections 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6cbe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded4df20 0x55d2ded4d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/3088547086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded53640 0x55d2ded54a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/3088547086 >> 172.21.15.52:0/3088547086 conn(0x55d2dec78cc0 msgr2=0x55d2ded54340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/3088547086 shutdown_connections 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/3088547086 wait complete. 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 -- start start 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.226+0000 7f5cd4ffcb80 1 Processor -- start 2024-06-02T23:16:09.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 0x55d2ded6cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded6d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400e710 con 0x55d2ded4df20 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400e040 con 0x55d2ded53f60 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ca400b240 con 0x55d2ded53640 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 10 client.0 _ll_register_callbacks cb 0x55d2dec39920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 0x55d2ded6cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 0x55d2ded6cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56408/0 (socket says 172.21.15.52:56408) 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4170146627 learned_addr learned my addr 172.21.15.52:0/4170146627 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded6d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 msgr2=0x55d2ded6d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded6d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4170146627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 msgr2=0x55d2ded6a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 -- 172.21.15.52:0/4170146627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ded6ade0 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc8d16700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cc3fff700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 0x55d2ded6cfa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4097d30 tx=0x7f5cb409a790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ce2d4328c28e608 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5ca37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4170146627 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cb409fc80 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4170146627 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5cb4097620 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.227+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5cbc004030 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4170146627 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5cb409b8d0 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 10 client.16794 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d2dedd1da0 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4170146627 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5cbc004030 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4170146627 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5cb4097070 con 0x55d2ded4df20 2024-06-02T23:16:09.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 10 client.16794 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:09.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 10 client.16794 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:16:09.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 20 client.16794 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:09.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2dedd4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 4 client.16794 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:16:09.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.228+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2deca7290 con 0x55d2ded677d0 2024-06-02T23:16:09.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2dedd4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5ca37fe700 10 client.16794 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:16:09.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cbbfff700 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2dedd4270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e3c9928ffe47c96a server_cookie=e45307bc7cf96e54 in_seq=0 out_seq=0 2024-06-02T23:16:09.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5ca37fe700 1 -- 172.21.15.52:0/4170146627 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2deca7290 con 0x55d2ded677d0 2024-06-02T23:16:09.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5ca37fe700 10 client.16794 handle_command_reply: tid=0 2024-06-02T23:16:09.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 client.16794 shutdown 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 msgr2=0x55d2ded6cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 0x55d2ded6cfa0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f5cb4097d30 tx=0x7f5cb409a790 comp rx=0 tx=0).stop 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 shutdown_connections 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2ded53f60 0x55d2ded6a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2ded4df20 0x55d2ded6cfa0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d2ded677d0 0x55d2dedd4270 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 --2- 172.21.15.52:0/4170146627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ded53640 0x55d2ded6d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5ca37fe700 0 client.16794 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.229+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 >> 172.21.15.52:0/4170146627 conn(0x55d2dec78cc0 msgr2=0x55d2ded54340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 shutdown_connections 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cd4ffcb80 1 -- 172.21.15.52:0/4170146627 wait complete. 2024-06-02T23:16:09.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cd4ffcb80 20 client.16794 trim_cache size 0 max 16384 2024-06-02T23:16:09.235 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:16:09.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb00a2d70 msgr2=0x7f5cb00a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb00a2d70 0x7f5cb00a5230 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5cac00b2e0 tx=0x7f5cac01a040 comp rx=0 tx=0).stop 2024-06-02T23:16:09.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 msgr2=0x7f5cc4103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4103070 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400e9a0 tx=0x7f5cb400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 shutdown_connections 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cb00a2d70 0x7f5cb00a5230 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cc4107400 0x7f5cc4103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cc4101390 0x7f5cc4103070 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 --2- 172.21.15.52:0/1782495583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cc4101d50 0x7f5cc41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.230+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 >> 172.21.15.52:0/1782495583 conn(0x7f5cc4078340 msgr2=0x7f5cc4100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.231+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 shutdown_connections 2024-06-02T23:16:09.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:09.231+0000 7f5cca779700 1 -- 172.21.15.52:0/1782495583 wait complete. 2024-06-02T23:16:09.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 73338 1.72 MB/sec execute 78 sec latency 22.040 ms 2024-06-02T23:16:09.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:09 smithi123 ceph-mon[35977]: pgmap v559: 97 pgs: 97 active+clean; 46 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-02T23:16:09.817 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:09 smithi052 ceph-mon[32340]: pgmap v559: 97 pgs: 97 active+clean; 46 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 -- 172.21.15.52:0/4066820221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d41076e0 msgr2=0x7f12d4109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 --2- 172.21.15.52:0/4066820221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d41076e0 0x7f12d4109b30 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009ca0 tx=0x7f12cc02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 -- 172.21.15.52:0/4066820221 shutdown_connections 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 --2- 172.21.15.52:0/4066820221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12d4104db0 0x7f12d41071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 --2- 172.21.15.52:0/4066820221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d41076e0 0x7f12d4109b30 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 --2- 172.21.15.52:0/4066820221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d4102480 0x7f12d4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 -- 172.21.15.52:0/4066820221 >> 172.21.15.52:0/4066820221 conn(0x7f12d40fbe70 msgr2=0x7f12d40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 -- 172.21.15.52:0/4066820221 shutdown_connections 2024-06-02T23:16:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 -- 172.21.15.52:0/4066820221 wait complete. 2024-06-02T23:16:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 Processor -- start 2024-06-02T23:16:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.826+0000 7f12da861640 1 -- start start 2024-06-02T23:16:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12da861640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12d4102480 0x7f12d4199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12da861640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 0x7f12d419a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12da861640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d41076e0 0x7f12d41a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12da861640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410d550 con 0x7f12d4104db0 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12da861640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410d3d0 con 0x7f12d41076e0 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12da861640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12d410d6d0 con 0x7f12d4102480 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 0x7f12d419a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 0x7f12d419a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56422/0 (socket says 172.21.15.52:56422) 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 -- 172.21.15.52:0/3955510293 learned_addr learned my addr 172.21.15.52:0/3955510293 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d41076e0 0x7f12d41a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12d4102480 0x7f12d4199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 -- 172.21.15.52:0/3955510293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12d4102480 msgr2=0x7f12d4199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12d4102480 0x7f12d4199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 -- 172.21.15.52:0/3955510293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d41076e0 msgr2=0x7f12d41a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d41076e0 0x7f12d41a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 -- 172.21.15.52:0/3955510293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d41a19b0 con 0x7f12d4104db0 2024-06-02T23:16:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d41076e0 0x7f12d41a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.827+0000 7f12d37fe640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 0x7f12d419a2e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9f0 tx=0x7f12c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.828+0000 7f12d17fa640 1 -- 172.21.15.52:0/3955510293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c800ccc0 con 0x7f12d4104db0 2024-06-02T23:16:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.828+0000 7f12d17fa640 1 -- 172.21.15.52:0/3955510293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12c80054d0 con 0x7f12d4104db0 2024-06-02T23:16:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.828+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d41a1ca0 con 0x7f12d4104db0 2024-06-02T23:16:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.828+0000 7f12d17fa640 1 -- 172.21.15.52:0/3955510293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8010430 con 0x7f12d4104db0 2024-06-02T23:16:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.828+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d41a21a0 con 0x7f12d4104db0 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 Processor -- start 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d17fa640 1 -- 172.21.15.52:0/3955510293 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12c8005020 con 0x7f12d4104db0 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 -- start start 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e610430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e615500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e618000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55691e5734f0 con 0x55691e610010 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55691e511870 con 0x55691e610a00 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12dd568740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55691e56f4b0 con 0x55691e615b70 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e610430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e610430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56438/0 (socket says 172.21.15.52:56438) 2024-06-02T23:16:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 -- 172.21.15.52:0/2214140926 learned_addr learned my addr 172.21.15.52:0/2214140926 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d3fff640 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e615500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d37fe640 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e618000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 -- 172.21.15.52:0/2214140926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 msgr2=0x55691e618000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e618000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 -- 172.21.15.52:0/2214140926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 msgr2=0x55691e615500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e615500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.831+0000 7f12d8dfe640 1 -- 172.21.15.52:0/2214140926 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691e52d9b0 con 0x55691e610010 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12d17fa640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12bc0a2c30 0x7f12bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12bc0a2c30 0x7f12bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12d17fa640 1 -- 172.21.15.52:0/3955510293 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f12c80cf460 con 0x7f12d4104db0 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e610430 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009ca0 tx=0x7f12cc009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6a6a6d09392e08 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12ba7fc640 1 -- 172.21.15.52:0/2214140926 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc031d40 con 0x55691e610010 2024-06-02T23:16:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12ba7fc640 1 -- 172.21.15.52:0/2214140926 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12cc036030 con 0x55691e610010 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.832+0000 7f12ba7fc640 1 -- 172.21.15.52:0/2214140926 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0305c0 con 0x55691e610010 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 -- 172.21.15.52:0/2214140926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 msgr2=0x55691e610430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e610430 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f12cc009ca0 tx=0x7f12cc009a20 comp rx=0 tx=0).stop 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12bc0a2c30 0x7f12bc0a50f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55691e610e00 tx=0x7f12c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 -- 172.21.15.52:0/2214140926 shutdown_connections 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e618000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e615500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 --2- 172.21.15.52:0/2214140926 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e610430 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 -- 172.21.15.52:0/2214140926 >> 172.21.15.52:0/2214140926 conn(0x55691e60b680 msgr2=0x55691e60ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 -- 172.21.15.52:0/2214140926 shutdown_connections 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 -- 172.21.15.52:0/2214140926 wait complete. 2024-06-02T23:16:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 -- start start 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.833+0000 7f12dd568740 1 Processor -- start 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e61a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e61a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e621680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55691e618940 con 0x55691e610010 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55691e4fcae0 con 0x55691e610a00 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55691e618ac0 con 0x55691e615b70 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12dd568740 10 client.0 _ll_register_callbacks cb 0x55691e5359b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e61a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e61a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56452/0 (socket says 172.21.15.52:56452) 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 -- 172.21.15.52:0/995961425 learned_addr learned my addr 172.21.15.52:0/995961425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d37fe640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e61a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.840 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e621680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 -- 172.21.15.52:0/995961425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 msgr2=0x55691e621680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e621680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 -- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 msgr2=0x55691e61a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e61a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 -- 172.21.15.52:0/995961425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691e287210 con 0x55691e610010 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e621680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d37fe640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e61a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12d3fff640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e61a100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f12c40045f0 tx=0x7f12c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3284019e2729a7a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.834+0000 7f12b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12b37fe640 1 -- 172.21.15.52:0/995961425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4005020 con 0x55691e610010 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12b37fe640 1 -- 172.21.15.52:0/995961425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f12c40051c0 con 0x55691e610010 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12cc036030 con 0x55691e610010 2024-06-02T23:16:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12b37fe640 1 -- 172.21.15.52:0/995961425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f12c40053c0 con 0x55691e610010 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12dd568740 10 client.16806 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55691e353620 con 0x55691e610010 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12b37fe640 1 -- 172.21.15.52:0/995961425 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12cc036030 con 0x55691e610010 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12b37fe640 1 -- 172.21.15.52:0/995961425 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f12c4027070 con 0x55691e610010 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12dd568740 10 client.16806 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12dd568740 10 client.16806 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.835+0000 7f12dd568740 20 client.16806 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.836+0000 7f12dd568740 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e626030 0x55691e628460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.836+0000 7f12dd568740 4 client.16806 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.836+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55691e586f10 con 0x55691e626030 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.836+0000 7f12d37fe640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e626030 0x55691e628460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.836+0000 7f12d37fe640 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e626030 0x55691e628460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=138e45e574e4fa8d server_cookie=bd8f75f034d4f8a7 in_seq=0 out_seq=0 2024-06-02T23:16:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.836+0000 7f12b37fe640 10 client.16806 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:09.843 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12b37fe640 1 -- 172.21.15.52:0/995961425 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55691e586f10 con 0x55691e626030 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12b37fe640 10 client.16806 handle_command_reply: tid=0 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 client.16806 shutdown 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 msgr2=0x55691e61a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e61a100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f12c40045f0 tx=0x7f12c4004730 comp rx=0 tx=0).stop 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 shutdown_connections 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e626030 0x55691e628460 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e615b70 0x55691e621680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e610a00 0x55691e61a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12b37fe640 0 client.16806 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 --2- 172.21.15.52:0/995961425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610010 0x55691e61a100 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.837+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 >> 172.21.15.52:0/995961425 conn(0x55691e60b680 msgr2=0x55691e60a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.838+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 shutdown_connections 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.838+0000 7f12dd568740 1 -- 172.21.15.52:0/995961425 wait complete. 2024-06-02T23:16:09.844 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.838+0000 7f12dd568740 20 client.16806 trim_cache size 0 max 16384 2024-06-02T23:16:09.862 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 Processor -- start 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 -- start start 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e617f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e621640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e60ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41aaeb0 con 0x55691e610a00 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41ac9f0 con 0x55691e615b70 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12dd568740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41ab580 con 0x55691e610010 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e621640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e621640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56454/0 (socket says 172.21.15.52:56454) 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.855+0000 7f12d37fe640 1 -- 172.21.15.52:0/2499957567 learned_addr learned my addr 172.21.15.52:0/2499957567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d3fff640 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e617f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e60ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.863 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d37fe640 1 -- 172.21.15.52:0/2499957567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 msgr2=0x55691e617f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d37fe640 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e617f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d37fe640 1 -- 172.21.15.52:0/2499957567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 msgr2=0x55691e60ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d37fe640 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e60ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d37fe640 1 -- 172.21.15.52:0/2499957567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55691e618c10 con 0x55691e610a00 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12d37fe640 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e621640 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f12c80985e0 tx=0x7f12c80048b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa30532d0cc7219 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12b9ffb640 1 -- 172.21.15.52:0/2499957567 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c8099e30 con 0x55691e610a00 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12b9ffb640 1 -- 172.21.15.52:0/2499957567 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f12c80046c0 con 0x55691e610a00 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.856+0000 7f12b9ffb640 1 -- 172.21.15.52:0/2499957567 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c800ce60 con 0x55691e610a00 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 -- 172.21.15.52:0/2499957567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 msgr2=0x55691e621640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e621640 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f12c80985e0 tx=0x7f12c80048b0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 -- 172.21.15.52:0/2499957567 shutdown_connections 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e60ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.864 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e621640 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 --2- 172.21.15.52:0/2499957567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e617f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 -- 172.21.15.52:0/2499957567 >> 172.21.15.52:0/2499957567 conn(0x55691e60b680 msgr2=0x55691e60e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 -- 172.21.15.52:0/2499957567 shutdown_connections 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 -- 172.21.15.52:0/2499957567 wait complete. 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.857+0000 7f12dd568740 1 -- start start 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 Processor -- start 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e61e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e61ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e68ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41ac320 con 0x55691e610a00 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41abc50 con 0x55691e615b70 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41a8e90 con 0x55691e610010 2024-06-02T23:16:09.865 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12dd568740 10 client.0 _ll_register_callbacks cb 0x7f12c4009ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e61ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e61ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56464/0 (socket says 172.21.15.52:56464) 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 -- 172.21.15.52:0/3382904853 learned_addr learned my addr 172.21.15.52:0/3382904853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d8dfe640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e68ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e61e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 -- 172.21.15.52:0/3382904853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 msgr2=0x55691e61e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e61e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 -- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 msgr2=0x55691e68ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e68ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d37fe640 1 -- 172.21.15.52:0/3382904853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55691e68f270 con 0x55691e610a00 2024-06-02T23:16:09.866 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.858+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e61e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12d37fe640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e61ea00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f12c809bbb0 tx=0x7f12c800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1a8ad972bbcdb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12b37fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12b37fe640 1 -- 172.21.15.52:0/3382904853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c80b9060 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12b37fe640 1 -- 172.21.15.52:0/3382904853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f12c80a44d0 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55691e580c20 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12b37fe640 1 -- 172.21.15.52:0/3382904853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f12c80046c0 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 10 client.16812 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55691e6904e0 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12b37fe640 1 -- 172.21.15.52:0/3382904853 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55691e580c20 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12b37fe640 1 -- 172.21.15.52:0/3382904853 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f12c80979b0 con 0x55691e610a00 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 10 client.16812 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 10 client.16812 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 20 client.16812 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:09.867 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e690e90 0x55691e6932b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 4 client.16812 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:16:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.859+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55691e57af70 con 0x55691e690e90 2024-06-02T23:16:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.860+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e690e90 0x55691e6932b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.860+0000 7f12d3fff640 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e690e90 0x55691e6932b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfa098d3d2ca967 server_cookie=e2de5dc6de4769b9 in_seq=0 out_seq=0 2024-06-02T23:16:09.868 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.860+0000 7f12b37fe640 10 client.16812 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12b37fe640 1 -- 172.21.15.52:0/3382904853 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55691e57af70 con 0x55691e690e90 2024-06-02T23:16:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12b37fe640 10 client.16812 handle_command_reply: tid=0 2024-06-02T23:16:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 client.16812 shutdown 2024-06-02T23:16:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 msgr2=0x55691e61ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e61ea00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f12c809bbb0 tx=0x7f12c800ce60 comp rx=0 tx=0).stop 2024-06-02T23:16:09.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 shutdown_connections 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55691e690e90 0x55691e6932b0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55691e615b70 0x55691e68ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55691e610a00 0x55691e61ea00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12dd568740 1 --2- 172.21.15.52:0/3382904853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55691e610010 0x55691e61e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.879+0000 7f12b37fe640 0 client.16812 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 >> 172.21.15.52:0/3382904853 conn(0x55691e60b680 msgr2=0x55691e619d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 shutdown_connections 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12dd568740 1 -- 172.21.15.52:0/3382904853 wait complete. 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12dd568740 20 client.16812 trim_cache size 0 max 16384 2024-06-02T23:16:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:16:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:16:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:16:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12bc0a2c30 msgr2=0x7f12bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12da861640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12bc0a2c30 0x7f12bc0a50f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55691e610e00 tx=0x7f12c4005f90 comp rx=0 tx=0).stop 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 msgr2=0x7f12d419a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12da861640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 0x7f12d419a2e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f12c800e9f0 tx=0x7f12c800eeb0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.880+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 shutdown_connections 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12bc0a2c30 0x7f12bc0a50f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12d41076e0 0x7f12d41a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f12d4104db0 0x7f12d419a2e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 --2- 172.21.15.52:0/3955510293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12d4102480 0x7f12d4199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 >> 172.21.15.52:0/3955510293 conn(0x7f12d40fbe70 msgr2=0x7f12d4101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 shutdown_connections 2024-06-02T23:16:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:09.881+0000 7f12da861640 1 -- 172.21.15.52:0/3955510293 wait complete. 2024-06-02T23:16:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:09 smithi174 ceph-mon[35994]: pgmap v559: 97 pgs: 97 active+clean; 46 MiB data, 712 MiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-02T23:16:10.020 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:16:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:10.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 73936 1.77 MB/sec execute 79 sec latency 14.904 ms 2024-06-02T23:16:11.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 74488 1.81 MB/sec execute 80 sec latency 12.277 ms 2024-06-02T23:16:11.685 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:11 smithi052 ceph-mon[32340]: pgmap v560: 97 pgs: 97 active+clean; 45 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-06-02T23:16:11.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:11 smithi123 ceph-mon[35977]: pgmap v560: 97 pgs: 97 active+clean; 45 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-06-02T23:16:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:11 smithi174 ceph-mon[35994]: pgmap v560: 97 pgs: 97 active+clean; 45 MiB data, 696 MiB used, 1.0 TiB / 1.0 TiB avail; 264 KiB/s rd, 1.4 MiB/s wr, 154 op/s 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 -- 172.21.15.52:0/4291720703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb428108500 msgr2=0x7fb4281088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/4291720703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb428108500 0x7fb4281088e0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b0a0 tx=0x7fb42400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 -- 172.21.15.52:0/4291720703 shutdown_connections 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/4291720703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806b560 0x7fb42810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/4291720703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb428108500 0x7fb4281088e0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/4291720703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb42806aba0 0x7fb42806b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 -- 172.21.15.52:0/4291720703 >> 172.21.15.52:0/4291720703 conn(0x7fb428077400 msgr2=0x7fb428077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:12.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 -- 172.21.15.52:0/4291720703 shutdown_connections 2024-06-02T23:16:12.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.081+0000 7fb42fdc0700 1 -- 172.21.15.52:0/4291720703 wait complete. 2024-06-02T23:16:12.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 Processor -- start 2024-06-02T23:16:12.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 -- start start 2024-06-02T23:16:12.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 0x7fb42819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb42806b560 0x7fb42819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb428108500 0x7fb428198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb428112630 con 0x7fb42806b560 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281124b0 con 0x7fb42806aba0 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42fdc0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281127b0 con 0x7fb428108500 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 0x7fb42819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42e35d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb428108500 0x7fb428198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 0x7fb42819e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:54118/0 (socket says 172.21.15.52:54118) 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42db5c700 1 -- 172.21.15.52:0/2731969108 learned_addr learned my addr 172.21.15.52:0/2731969108 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.082+0000 7fb42d35b700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb42806b560 0x7fb42819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42db5c700 1 -- 172.21.15.52:0/2731969108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb428108500 msgr2=0x7fb428198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42db5c700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb428108500 0x7fb428198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42db5c700 1 -- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb42806b560 msgr2=0x7fb42819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42db5c700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb42806b560 0x7fb42819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42db5c700 1 -- 172.21.15.52:0/2731969108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4281990d0 con 0x7fb42806aba0 2024-06-02T23:16:12.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42db5c700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 0x7fb42819e7b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b070 tx=0x7fb424004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:12.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb41effd700 1 -- 172.21.15.52:0/2731969108 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424021070 con 0x7fb42806aba0 2024-06-02T23:16:12.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb41effd700 1 -- 172.21.15.52:0/2731969108 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb424007e30 con 0x7fb42806aba0 2024-06-02T23:16:12.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb428199360 con 0x7fb42806aba0 2024-06-02T23:16:12.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb41effd700 1 -- 172.21.15.52:0/2731969108 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4240046c0 con 0x7fb42806aba0 2024-06-02T23:16:12.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.083+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4281a5680 con 0x7fb42806aba0 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.085+0000 7fb41effd700 1 -- 172.21.15.52:0/2731969108 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb424007980 con 0x7fb42806aba0 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.086+0000 7fb41effd700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb4140a2ca0 0x7fb4140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.086+0000 7fb41effd700 1 -- 172.21.15.52:0/2731969108 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb42400f020 con 0x7fb42806aba0 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.086+0000 7fb42d35b700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb4140a2ca0 0x7fb4140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 Processor -- start 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb42d35b700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb4140a2ca0 0x7fb4140a5160 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb428077ae0 tx=0x7fb418009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:12.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 -- start start 2024-06-02T23:16:12.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a27fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a2799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a27a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55917a1cb810 con 0x55917a27f640 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55917a173720 con 0x55917a279f20 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb43a643b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55917a1c77d0 con 0x55917a27ff60 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.087+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a27fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a27fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56494/0 (socket says 172.21.15.52:56494) 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 -- 172.21.15.52:0/3575851900 learned_addr learned my addr 172.21.15.52:0/3575851900 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42e35d700 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a27a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42d35b700 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a2799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 -- 172.21.15.52:0/3575851900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 msgr2=0x55917a2799e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a2799e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 -- 172.21.15.52:0/3575851900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 msgr2=0x55917a27a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a27a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 -- 172.21.15.52:0/3575851900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917a169f50 con 0x55917a27f640 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb42db5c700 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a27fa20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb424099130 tx=0x7fb42400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75154875bc73305 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb413fff700 1 -- 172.21.15.52:0/3575851900 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424007c70 con 0x55917a27f640 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb413fff700 1 -- 172.21.15.52:0/3575851900 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb42409d3d0 con 0x55917a27f640 2024-06-02T23:16:12.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.088+0000 7fb413fff700 1 -- 172.21.15.52:0/3575851900 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb424004860 con 0x55917a27f640 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 -- 172.21.15.52:0/3575851900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 msgr2=0x55917a27fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a27fa20 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fb424099130 tx=0x7fb42400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 -- 172.21.15.52:0/3575851900 shutdown_connections 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a27a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a27fa20 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 --2- 172.21.15.52:0/3575851900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a2799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 -- 172.21.15.52:0/3575851900 >> 172.21.15.52:0/3575851900 conn(0x55917a1a4cc0 msgr2=0x55917a274850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 -- 172.21.15.52:0/3575851900 shutdown_connections 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.089+0000 7fb43a643b80 1 -- 172.21.15.52:0/3575851900 wait complete. 2024-06-02T23:16:12.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.090+0000 7fb43a643b80 1 -- start start 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.090+0000 7fb43a643b80 1 Processor -- start 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.090+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a285d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.090+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a2830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a283620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42d35b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a2830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42d35b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a2830e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56498/0 (socket says 172.21.15.52:56498) 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a285d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42d35b700 1 -- 172.21.15.52:0/2144907630 learned_addr learned my addr 172.21.15.52:0/2144907630 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55917a2866f0 con 0x55917a27f640 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55917a16bbf0 con 0x55917a279f20 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55917a286870 con 0x55917a27ff60 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb43a643b80 10 client.0 _ll_register_callbacks cb 0x55917a165920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:12.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 -- 172.21.15.52:0/2144907630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 msgr2=0x55917a283620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a283620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 -- 172.21.15.52:0/2144907630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 msgr2=0x55917a2830e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a2830e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917a11a8d0 con 0x55917a279f20 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42d35b700 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a2830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb42db5c700 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a285d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb42409c980 tx=0x7fb424007c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1fa051382e1c999 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb4117fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb4117fa700 1 -- 172.21.15.52:0/2144907630 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42407f070 con 0x55917a279f20 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb4117fa700 1 -- 172.21.15.52:0/2144907630 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb42409cf30 con 0x55917a279f20 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.091+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb42409d3d0 con 0x55917a279f20 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb4117fa700 1 -- 172.21.15.52:0/2144907630 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb424003c00 con 0x55917a279f20 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 10 client.26360 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55917a2866f0 con 0x55917a279f20 2024-06-02T23:16:12.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb4117fa700 1 -- 172.21.15.52:0/2144907630 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb42409d3d0 con 0x55917a279f20 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb4117fa700 1 -- 172.21.15.52:0/2144907630 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb424011070 con 0x55917a279f20 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 10 client.26360 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 10 client.26360 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 20 client.26360 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a293390 0x55917a295820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 4 client.26360 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.092+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55917a1d8d10 con 0x55917a293390 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.093+0000 7fb42d35b700 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a293390 0x55917a295820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.093+0000 7fb42d35b700 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a293390 0x55917a295820 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a74a126342d535c server_cookie=b4a4b7040146b9a4 in_seq=0 out_seq=0 2024-06-02T23:16:12.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.093+0000 7fb4117fa700 10 client.26360 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:16:12.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb4117fa700 1 -- 172.21.15.52:0/2144907630 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55917a1d8d10 con 0x55917a293390 2024-06-02T23:16:12.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb4117fa700 10 client.26360 handle_command_reply: tid=0 2024-06-02T23:16:12.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 client.26360 shutdown 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 msgr2=0x55917a285d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a285d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb42409c980 tx=0x7fb424007c70 comp rx=0 tx=0).stop 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 shutdown_connections 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a293390 0x55917a295820 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27ff60 0x55917a283620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a279f20 0x55917a285d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb4117fa700 0 client.26360 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 --2- 172.21.15.52:0/2144907630 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a2830e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 >> 172.21.15.52:0/2144907630 conn(0x55917a1a4cc0 msgr2=0x55917a2745f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 shutdown_connections 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 1 -- 172.21.15.52:0/2144907630 wait complete. 2024-06-02T23:16:12.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.094+0000 7fb43a643b80 20 client.26360 trim_cache size 0 max 16384 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 Processor -- start 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 -- start start 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a279f20 0x55917a2799e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a28f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a282040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281aca30 con 0x55917a27f640 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281ae570 con 0x55917a27ff60 2024-06-02T23:16:12.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb43a643b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281ad100 con 0x55917a279f20 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a28f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a279f20 0x55917a2799e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a28f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56514/0 (socket says 172.21.15.52:56514) 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 -- 172.21.15.52:0/4168440795 learned_addr learned my addr 172.21.15.52:0/4168440795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42e35d700 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a282040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 -- 172.21.15.52:0/4168440795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a279f20 msgr2=0x55917a2799e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a279f20 0x55917a2799e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 -- 172.21.15.52:0/4168440795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 msgr2=0x55917a282040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a282040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42d35b700 1 -- 172.21.15.52:0/4168440795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55917a2859d0 con 0x55917a27f640 2024-06-02T23:16:12.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.112+0000 7fb42e35d700 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a282040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:16:12.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.113+0000 7fb42d35b700 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a28f9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb4180117d0 tx=0x7fb418017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b89c1a933ec77de server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:12.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.113+0000 7fb4137fe700 1 -- 172.21.15.52:0/4168440795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb418003b60 con 0x55917a27f640 2024-06-02T23:16:12.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.113+0000 7fb4137fe700 1 -- 172.21.15.52:0/4168440795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb418004ce0 con 0x55917a27f640 2024-06-02T23:16:12.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.113+0000 7fb4137fe700 1 -- 172.21.15.52:0/4168440795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4180055a0 con 0x55917a27f640 2024-06-02T23:16:12.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.113+0000 7fb43a643b80 1 -- 172.21.15.52:0/4168440795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 msgr2=0x55917a28f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.113+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a28f9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fb4180117d0 tx=0x7fb418017000 comp rx=0 tx=0).stop 2024-06-02T23:16:12.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 -- 172.21.15.52:0/4168440795 shutdown_connections 2024-06-02T23:16:12.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a282040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a279f20 0x55917a2799e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4168440795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a27f640 0x55917a28f9c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 -- 172.21.15.52:0/4168440795 >> 172.21.15.52:0/4168440795 conn(0x55917a1a4cc0 msgr2=0x55917a26de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 -- 172.21.15.52:0/4168440795 shutdown_connections 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 -- 172.21.15.52:0/4168440795 wait complete. 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 -- start start 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.114+0000 7fb43a643b80 1 Processor -- start 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 0x55917a28bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27f640 0x55917a28c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a29a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281adea0 con 0x55917a279f20 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281ad7d0 con 0x55917a27ff60 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb4281aa9d0 con 0x55917a27f640 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb43a643b80 10 client.0 _ll_register_callbacks cb 0x55917a165920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 0x55917a28bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 0x55917a28bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56526/0 (socket says 172.21.15.52:56526) 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 -- 172.21.15.52:0/4106102285 learned_addr learned my addr 172.21.15.52:0/4106102285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42d35b700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27f640 0x55917a28c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42e35d700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a29a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 -- 172.21.15.52:0/4106102285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27f640 msgr2=0x55917a28c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27f640 0x55917a28c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 -- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 msgr2=0x55917a29a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:12.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a29a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 -- 172.21.15.52:0/4106102285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55917a29aba0 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb42db5c700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 0x55917a28bd10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb42409d280 tx=0x7fb4240cdf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4064d899b7fe7185 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb4117fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.115+0000 7fb4117fa700 1 -- 172.21.15.52:0/4106102285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb42407f070 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb4117fa700 1 -- 172.21.15.52:0/4106102285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb42409c330 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb418004ce0 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb4117fa700 1 -- 172.21.15.52:0/4106102285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb42409c630 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 10 client.16830 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55917a29aba0 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb4117fa700 1 -- 172.21.15.52:0/4106102285 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb4240a9060 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb4117fa700 1 -- 172.21.15.52:0/4106102285 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb424011070 con 0x55917a279f20 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 10 client.16830 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 10 client.16830 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:16:12.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 20 client.16830 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a2fdd60 0x55917a300180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 4 client.16830 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.116+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb418004ce0 con 0x55917a2fdd60 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb42d35b700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a2fdd60 0x55917a300180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb42d35b700 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a2fdd60 0x55917a300180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf92d0c3a147c86b server_cookie=c341554fdf8db040 in_seq=0 out_seq=0 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb4117fa700 10 client.16830 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb4117fa700 1 -- 172.21.15.52:0/4106102285 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb418004ce0 con 0x55917a2fdd60 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb4117fa700 10 client.16830 handle_command_reply: tid=0 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 client.16830 shutdown 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 msgr2=0x55917a28bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 0x55917a28bd10 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb42409d280 tx=0x7fb4240cdf60 comp rx=0 tx=0).stop 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 shutdown_connections 2024-06-02T23:16:12.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55917a2fdd60 0x55917a300180 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55917a27ff60 0x55917a29a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55917a279f20 0x55917a28bd10 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb4117fa700 0 client.16830 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 --2- 172.21.15.52:0/4106102285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55917a27f640 0x55917a28c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 >> 172.21.15.52:0/4106102285 conn(0x55917a1a4cc0 msgr2=0x55917a280340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.117+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 shutdown_connections 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb43a643b80 1 -- 172.21.15.52:0/4106102285 wait complete. 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb43a643b80 20 client.16830 trim_cache size 0 max 16384 2024-06-02T23:16:12.126 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb4140a2ca0 msgr2=0x7fb4140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb4140a2ca0 0x7fb4140a5160 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb428077ae0 tx=0x7fb418009260 comp rx=0 tx=0).stop 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 msgr2=0x7fb42819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 0x7fb42819e7b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb42400b070 tx=0x7fb424004d00 comp rx=0 tx=0).stop 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 shutdown_connections 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb4140a2ca0 0x7fb4140a5160 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb428108500 0x7fb428198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb42806aba0 0x7fb42819e7b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 --2- 172.21.15.52:0/2731969108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb42806b560 0x7fb42819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:12.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 >> 172.21.15.52:0/2731969108 conn(0x7fb428077400 msgr2=0x7fb4281001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:12.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 shutdown_connections 2024-06-02T23:16:12.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:12.118+0000 7fb42fdc0700 1 -- 172.21.15.52:0/2731969108 wait complete. 2024-06-02T23:16:12.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 74860 1.80 MB/sec execute 81 sec latency 16.877 ms 2024-06-02T23:16:12.978 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:16:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:13.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 75140 1.78 MB/sec execute 82 sec latency 15.212 ms 2024-06-02T23:16:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:13 smithi052 ceph-mon[32340]: pgmap v561: 97 pgs: 97 active+clean; 47 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-06-02T23:16:13.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:13 smithi123 ceph-mon[35977]: pgmap v561: 97 pgs: 97 active+clean; 47 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-06-02T23:16:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:13 smithi174 ceph-mon[35994]: pgmap v561: 97 pgs: 97 active+clean; 47 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 1.4 MiB/s wr, 172 op/s 2024-06-02T23:16:14.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 75686 1.79 MB/sec execute 83 sec latency 11.641 ms 2024-06-02T23:16:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 -- 172.21.15.52:0/1444396335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c1073f0 msgr2=0x7f1a3c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 --2- 172.21.15.52:0/1444396335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c1073f0 0x7f1a3c1077d0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009990 tx=0x7f1a38009ca0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 -- 172.21.15.52:0/1444396335 shutdown_connections 2024-06-02T23:16:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 --2- 172.21.15.52:0/1444396335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a3c101d40 0x7f1a3c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 --2- 172.21.15.52:0/1444396335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c1073f0 0x7f1a3c1077d0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 --2- 172.21.15.52:0/1444396335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 0x7f1a3c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 -- 172.21.15.52:0/1444396335 >> 172.21.15.52:0/1444396335 conn(0x7f1a3c078300 msgr2=0x7f1a3c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 -- 172.21.15.52:0/1444396335 shutdown_connections 2024-06-02T23:16:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.075+0000 7f1a44386700 1 -- 172.21.15.52:0/1444396335 wait complete. 2024-06-02T23:16:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 Processor -- start 2024-06-02T23:16:15.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 -- start start 2024-06-02T23:16:15.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 0x7f1a3c19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 0x7f1a3c19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a3c1073f0 0x7f1a3c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111510 con 0x7f1a3c101d40 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111390 con 0x7f1a3c101380 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a44386700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c111690 con 0x7f1a3c1073f0 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a41921700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 0x7f1a3c19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a41921700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 0x7f1a3c19ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35860/0 (socket says 172.21.15.52:35860) 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a42122700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 0x7f1a3c19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.076+0000 7f1a41921700 1 -- 172.21.15.52:0/3331537876 learned_addr learned my addr 172.21.15.52:0/3331537876 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a42923700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a3c1073f0 0x7f1a3c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a41921700 1 -- 172.21.15.52:0/3331537876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a3c1073f0 msgr2=0x7f1a3c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a41921700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a3c1073f0 0x7f1a3c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a41921700 1 -- 172.21.15.52:0/3331537876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 msgr2=0x7f1a3c19e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a41921700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 0x7f1a3c19e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a41921700 1 -- 172.21.15.52:0/3331537876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a3c199030 con 0x7f1a3c101d40 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a42122700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 0x7f1a3c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a41921700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 0x7f1a3c19ec50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e9a0 tx=0x7f1a2c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a337fe700 1 -- 172.21.15.52:0/3331537876 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c00cbd0 con 0x7f1a3c101d40 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a3c199320 con 0x7f1a3c101d40 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a337fe700 1 -- 172.21.15.52:0/3331537876 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a2c00cd70 con 0x7f1a3c101d40 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a337fe700 1 -- 172.21.15.52:0/3331537876 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c018680 con 0x7f1a3c101d40 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.077+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a3c1a55e0 con 0x7f1a3c101d40 2024-06-02T23:16:15.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.080+0000 7f1a337fe700 1 -- 172.21.15.52:0/3331537876 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a2c010430 con 0x7f1a3c101d40 2024-06-02T23:16:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.081+0000 7f1a337fe700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a280a2c20 0x7f1a280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.081+0000 7f1a42122700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a280a2c20 0x7f1a280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.081+0000 7f1a337fe700 1 -- 172.21.15.52:0/3331537876 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1a2c0cf280 con 0x7f1a3c101d40 2024-06-02T23:16:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.081+0000 7f1a4ec09b80 1 Processor -- start 2024-06-02T23:16:15.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.081+0000 7f1a42122700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a280a2c20 0x7f1a280a50e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b2e0 tx=0x7f1a3801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:15.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 -- start start 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561b30940810 con 0x561b309f7c30 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561b308e8720 con 0x561b309f38f0 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b3093c7d0 con 0x561b309f2fd0 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35870/0 (socket says 172.21.15.52:35870) 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 -- 172.21.15.52:0/2828690225 learned_addr learned my addr 172.21.15.52:0/2828690225 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42122700 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a41921700 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 -- 172.21.15.52:0/2828690225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 msgr2=0x561b309f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 -- 172.21.15.52:0/2828690225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 msgr2=0x561b309f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.082+0000 7f1a42923700 1 -- 172.21.15.52:0/2828690225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b308def50 con 0x561b309f7c30 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.083+0000 7f1a42923700 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f8090 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009440 tx=0x7f1a34009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93962ddc02e33886 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.083+0000 7f1a30d70700 1 -- 172.21.15.52:0/2828690225 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34013750 con 0x561b309f7c30 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.083+0000 7f1a30d70700 1 -- 172.21.15.52:0/2828690225 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a34013da0 con 0x561b309f7c30 2024-06-02T23:16:15.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.083+0000 7f1a30d70700 1 -- 172.21.15.52:0/2828690225 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a34015e30 con 0x561b309f7c30 2024-06-02T23:16:15.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/2828690225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 msgr2=0x561b309f8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f8090 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1a34009440 tx=0x7f1a34009920 comp rx=0 tx=0).stop 2024-06-02T23:16:15.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/2828690225 shutdown_connections 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f8090 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/2828690225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/2828690225 >> 172.21.15.52:0/2828690225 conn(0x561b30919cc0 msgr2=0x561b309e9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/2828690225 shutdown_connections 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/2828690225 wait complete. 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 -- start start 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.084+0000 7f1a4ec09b80 1 Processor -- start 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 0x561b309ff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 0x561b309fc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f7c30 0x561b309fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561b309ffdd0 con 0x561b309f38f0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561b308e0bf0 con 0x561b309f2fd0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561b309fff50 con 0x561b309f7c30 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a4ec09b80 10 client.0 _ll_register_callbacks cb 0x561b308da920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 0x561b309fc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a42122700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 0x561b309ff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 0x561b309fc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35874/0 (socket says 172.21.15.52:35874) 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 -- 172.21.15.52:0/1110219417 learned_addr learned my addr 172.21.15.52:0/1110219417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a42122700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 0x561b309ff0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41790/0 (socket says 172.21.15.52:41790) 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a42923700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f7c30 0x561b309fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 -- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f7c30 msgr2=0x561b309fc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f7c30 0x561b309fc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 -- 172.21.15.52:0/1110219417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 msgr2=0x561b309ff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 0x561b309ff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 -- 172.21.15.52:0/1110219417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b3088f8d0 con 0x561b309f38f0 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a42923700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f7c30 0x561b309fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:16:15.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a42122700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 0x561b309ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a41921700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 0x561b309fc460 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e970 tx=0x7f1a2c09c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b8d5d2aed8ba5ff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.085+0000 7f1a25ffb700 1 -- 172.21.15.52:0/1110219417 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c09c210 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a25ffb700 1 -- 172.21.15.52:0/1110219417 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1a2c09af60 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a34013da0 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a25ffb700 1 -- 172.21.15.52:0/1110219417 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a2c09bb10 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 10 client.16842 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561b30a0a6c0 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a25ffb700 1 -- 172.21.15.52:0/1110219417 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a2c064020 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a25ffb700 1 -- 172.21.15.52:0/1110219417 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1a2c09a9d0 con 0x561b309f38f0 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 10 client.16842 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 10 client.16842 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:16:15.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 20 client.16842 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a0cc50 0x561b30a0f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 4 client.16842 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561b3094dd10 con 0x561b30a0cc50 2024-06-02T23:16:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.086+0000 7f1a42122700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a0cc50 0x561b30a0f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.087+0000 7f1a42122700 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a0cc50 0x561b30a0f040 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6ad5343c7b5dc4d1 server_cookie=452d4d70adba4193 in_seq=0 out_seq=0 2024-06-02T23:16:15.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.087+0000 7f1a25ffb700 10 client.16842 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a25ffb700 1 -- 172.21.15.52:0/1110219417 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561b3094dd10 con 0x561b30a0cc50 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a25ffb700 10 client.16842 handle_command_reply: tid=0 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 client.16842 shutdown 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 msgr2=0x561b309fc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 0x561b309fc460 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e970 tx=0x7f1a2c09c050 comp rx=0 tx=0).stop 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 shutdown_connections 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a0cc50 0x561b30a0f040 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f7c30 0x561b309fc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a25ffb700 0 client.16842 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:16:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f2fd0 0x561b309ff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/1110219417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f38f0 0x561b309fc460 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 >> 172.21.15.52:0/1110219417 conn(0x561b30919cc0 msgr2=0x561b309ed9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 shutdown_connections 2024-06-02T23:16:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/1110219417 wait complete. 2024-06-02T23:16:15.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.088+0000 7f1a4ec09b80 20 client.16842 trim_cache size 0 max 16384 2024-06-02T23:16:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 Processor -- start 2024-06-02T23:16:15.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 -- start start 2024-06-02T23:16:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309e3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b30a148f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1aa8f0 con 0x561b309f7c30 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ac430 con 0x561b309f38f0 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1aafc0 con 0x561b309f2fd0 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a42923700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b30a148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a42122700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a42923700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b30a148f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35886/0 (socket says 172.21.15.52:35886) 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a42122700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309fbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34040/0 (socket says 172.21.15.52:34040) 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a42923700 1 -- 172.21.15.52:0/3770626996 learned_addr learned my addr 172.21.15.52:0/3770626996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.106+0000 7f1a41921700 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309e3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a42122700 1 -- 172.21.15.52:0/3770626996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 msgr2=0x561b309e3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a42122700 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309e3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a42122700 1 -- 172.21.15.52:0/3770626996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 msgr2=0x561b30a148f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a42122700 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b30a148f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a42122700 1 -- 172.21.15.52:0/3770626996 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561b309fd380 con 0x561b309f2fd0 2024-06-02T23:16:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a42122700 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309fbd60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a380059d0 tx=0x7f1a3801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b9ecd338ee8e0561 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a27fff700 1 -- 172.21.15.52:0/3770626996 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38004df0 con 0x561b309f2fd0 2024-06-02T23:16:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a27fff700 1 -- 172.21.15.52:0/3770626996 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a380268e0 con 0x561b309f2fd0 2024-06-02T23:16:15.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.107+0000 7f1a27fff700 1 -- 172.21.15.52:0/3770626996 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a38025870 con 0x561b309f2fd0 2024-06-02T23:16:15.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/3770626996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 msgr2=0x561b309fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309fbd60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1a380059d0 tx=0x7f1a3801f000 comp rx=0 tx=0).stop 2024-06-02T23:16:15.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/3770626996 shutdown_connections 2024-06-02T23:16:15.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b30a148f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b309fbd60 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/3770626996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309e3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/3770626996 >> 172.21.15.52:0/3770626996 conn(0x561b30919cc0 msgr2=0x561b309e2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/3770626996 shutdown_connections 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/3770626996 wait complete. 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 -- start start 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.108+0000 7f1a4ec09b80 1 Processor -- start 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b30a05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f9a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1abd60 con 0x561b309f7c30 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1ab690 con 0x561b309f38f0 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1a3c1a88b0 con 0x561b309f2fd0 2024-06-02T23:16:15.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a4ec09b80 10 client.0 _ll_register_callbacks cb 0x561b308da920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a42122700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b30a05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a42122700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b30a05aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34054/0 (socket says 172.21.15.52:34054) 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a42122700 1 -- 172.21.15.52:0/236710329 learned_addr learned my addr 172.21.15.52:0/236710329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a42923700 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 -- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 msgr2=0x561b30a05aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b30a05aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 -- 172.21.15.52:0/236710329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 msgr2=0x561b309f9a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f9a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 -- 172.21.15.52:0/236710329 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561b30a768c0 con 0x561b309f38f0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a41921700 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f93a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cfd0 tx=0x7f1a2c09a020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e5a8e139ce3bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.109+0000 7f1a25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a25ffb700 1 -- 172.21.15.52:0/236710329 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a2c010970 con 0x561b309f38f0 2024-06-02T23:16:15.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a25ffb700 1 -- 172.21.15.52:0/236710329 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1a2c09b170 con 0x561b309f38f0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a380268e0 con 0x561b309f38f0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a25ffb700 1 -- 172.21.15.52:0/236710329 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1a2c09c210 con 0x561b309f38f0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 10 client.26381 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561b30a77aa0 con 0x561b309f38f0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a25ffb700 1 -- 172.21.15.52:0/236710329 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a380268e0 con 0x561b309f38f0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a25ffb700 1 -- 172.21.15.52:0/236710329 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1a2c0a1050 con 0x561b309f38f0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 10 client.26381 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 10 client.26381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 20 client.26381 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a781b0 0x561b30a7a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 4 client.26381 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.110+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561b30948290 con 0x561b30a781b0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a42122700 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a781b0 0x561b30a7a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a42122700 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a781b0 0x561b30a7a5d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e2f89a5cd2f85aa server_cookie=548e5250c5449bbf in_seq=0 out_seq=0 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a25ffb700 10 client.26381 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:16:15.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a25ffb700 1 -- 172.21.15.52:0/236710329 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561b30948290 con 0x561b30a781b0 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a25ffb700 10 client.26381 handle_command_reply: tid=0 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 client.26381 shutdown 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 msgr2=0x561b309f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f93a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00cfd0 tx=0x7f1a2c09a020 comp rx=0 tx=0).stop 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 shutdown_connections 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561b30a781b0 0x561b30a7a5d0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561b309f7c30 0x561b309f9a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561b309f2fd0 0x561b30a05aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 --2- 172.21.15.52:0/236710329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561b309f38f0 0x561b309f93a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 >> 172.21.15.52:0/236710329 conn(0x561b30919cc0 msgr2=0x561b309f65f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.111+0000 7f1a25ffb700 0 client.26381 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 shutdown_connections 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a4ec09b80 1 -- 172.21.15.52:0/236710329 wait complete. 2024-06-02T23:16:15.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a4ec09b80 20 client.26381 trim_cache size 0 max 16384 2024-06-02T23:16:15.117 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a280a2c20 msgr2=0x7f1a280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a280a2c20 0x7f1a280a50e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f1a3800b2e0 tx=0x7f1a3801a040 comp rx=0 tx=0).stop 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 msgr2=0x7f1a3c19ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 0x7f1a3c19ec50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1a2c00e9a0 tx=0x7f1a2c00ecb0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 shutdown_connections 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a280a2c20 0x7f1a280a50e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1a3c1073f0 0x7f1a3c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a3c101380 0x7f1a3c19e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 --2- 172.21.15.52:0/3331537876 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1a3c101d40 0x7f1a3c19ec50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:15.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 >> 172.21.15.52:0/3331537876 conn(0x7f1a3c078300 msgr2=0x7f1a3c1006c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:15.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 shutdown_connections 2024-06-02T23:16:15.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:15.112+0000 7f1a44386700 1 -- 172.21.15.52:0/3331537876 wait complete. 2024-06-02T23:16:15.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 76236 1.80 MB/sec execute 84 sec latency 23.552 ms 2024-06-02T23:16:15.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:15 smithi123 ceph-mon[35977]: pgmap v562: 97 pgs: 97 active+clean; 46 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:16:15.875 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:15 smithi052 ceph-mon[32340]: pgmap v562: 97 pgs: 97 active+clean; 46 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:16:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:15 smithi174 ceph-mon[35994]: pgmap v562: 97 pgs: 97 active+clean; 46 MiB data, 699 MiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:16:15.905 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:16:16.186 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:16.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 76904 1.80 MB/sec execute 85 sec latency 17.124 ms 2024-06-02T23:16:17.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 77469 1.85 MB/sec execute 86 sec latency 88.020 ms 2024-06-02T23:16:17.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:17 smithi052 ceph-mon[32340]: pgmap v563: 97 pgs: 97 active+clean; 46 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-06-02T23:16:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:17 smithi123 ceph-mon[35977]: pgmap v563: 97 pgs: 97 active+clean; 46 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-06-02T23:16:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:17 smithi174 ceph-mon[35994]: pgmap v563: 97 pgs: 97 active+clean; 46 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.5 MiB/s wr, 164 op/s 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 -- 172.21.15.52:0/319296299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc106720 msgr2=0x7f3ecc10e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 --2- 172.21.15.52:0/319296299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc106720 0x7f3ecc10e2c0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b0d0 tx=0x7f3ec400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 -- 172.21.15.52:0/319296299 shutdown_connections 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 --2- 172.21.15.52:0/319296299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc106720 0x7f3ecc10e2c0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 --2- 172.21.15.52:0/319296299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc1019d0 0x7f3ecc101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 --2- 172.21.15.52:0/319296299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ecc102380 0x7f3ecc105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 -- 172.21.15.52:0/319296299 >> 172.21.15.52:0/319296299 conn(0x7f3ecc078340 msgr2=0x7f3ecc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:18.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 -- 172.21.15.52:0/319296299 shutdown_connections 2024-06-02T23:16:18.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.375+0000 7f3ed2626700 1 -- 172.21.15.52:0/319296299 wait complete. 2024-06-02T23:16:18.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 Processor -- start 2024-06-02T23:16:18.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 -- start start 2024-06-02T23:16:18.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ecc1019d0 0x7f3ecc19d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc102380 0x7f3ecc19db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc106720 0x7f3ecc197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc111590 con 0x7f3ecc102380 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc111410 con 0x7f3ecc106720 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed2626700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc111710 con 0x7f3ecc1019d0 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ecb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc102380 0x7f3ecc19db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ed0bc3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc106720 0x7f3ecc197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ecb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc102380 0x7f3ecc19db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35916/0 (socket says 172.21.15.52:35916) 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ecb7fe700 1 -- 172.21.15.52:0/650906930 learned_addr learned my addr 172.21.15.52:0/650906930 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.376+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ecc1019d0 0x7f3ecc19d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/650906930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ecc1019d0 msgr2=0x7f3ecc19d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ecc1019d0 0x7f3ecc19d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc102380 msgr2=0x7f3ecc19db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc102380 0x7f3ecc19db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/650906930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ecc197ff0 con 0x7f3ecc106720 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc106720 0x7f3ecc197810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b0d0 tx=0x7f3ec4007a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ec97fa700 1 -- 172.21.15.52:0/650906930 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec401f070 con 0x7f3ecc106720 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ecc198280 con 0x7f3ecc106720 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ec97fa700 1 -- 172.21.15.52:0/650906930 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec4004c50 con 0x7f3ecc106720 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ec97fa700 1 -- 172.21.15.52:0/650906930 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec401b690 con 0x7f3ecc106720 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.377+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ecc06ab60 con 0x7f3ecc106720 2024-06-02T23:16:18.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.380+0000 7f3ec97fa700 1 -- 172.21.15.52:0/650906930 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ec4022070 con 0x7f3ecc106720 2024-06-02T23:16:18.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.381+0000 7f3ec97fa700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3eb40a2c20 0x7f3eb40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.381+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3eb40a2c20 0x7f3eb40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.381+0000 7f3ec97fa700 1 -- 172.21.15.52:0/650906930 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ec400f020 con 0x7f3ecc106720 2024-06-02T23:16:18.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.381+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3eb40a2c20 0x7f3eb40a50e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009c60 tx=0x7f3ebc009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:18.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 Processor -- start 2024-06-02T23:16:18.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 -- start start 2024-06-02T23:16:18.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291d7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 0x5597291d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291d2750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55972911f810 con 0x5597291d8390 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597290c7720 con 0x5597291d2330 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55972911b7d0 con 0x5597291d7a70 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3ecb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 0x5597291d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3ecb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 0x5597291d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35920/0 (socket says 172.21.15.52:35920) 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3ecb7fe700 1 -- 172.21.15.52:0/3293157898 learned_addr learned my addr 172.21.15.52:0/3293157898 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291d2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.382+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291d7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/3293157898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 msgr2=0x5597291d7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291d7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/3293157898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 msgr2=0x5597291d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 0x5597291d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/3293157898 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597290bdf50 con 0x5597291d2330 2024-06-02T23:16:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ecb7fe700 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 0x5597291d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291d2750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40cda90 tx=0x7f3ec4006230 comp rx=0 tx=0).ready entity=mon.2 client_cookie=516b2100dadc6767 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3eb27fc700 1 -- 172.21.15.52:0/3293157898 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4005a90 con 0x5597291d2330 2024-06-02T23:16:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3eb27fc700 1 -- 172.21.15.52:0/3293157898 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec409cfb0 con 0x5597291d2330 2024-06-02T23:16:18.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.383+0000 7f3eb27fc700 1 -- 172.21.15.52:0/3293157898 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40074e0 con 0x5597291d2330 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 -- 172.21.15.52:0/3293157898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 msgr2=0x5597291d2750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291d2750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40cda90 tx=0x7f3ec4006230 comp rx=0 tx=0).stop 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 -- 172.21.15.52:0/3293157898 shutdown_connections 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291d2750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291d7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/3293157898 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d8390 0x5597291d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 -- 172.21.15.52:0/3293157898 >> 172.21.15.52:0/3293157898 conn(0x5597290f8cc0 msgr2=0x5597291c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 -- 172.21.15.52:0/3293157898 shutdown_connections 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 -- 172.21.15.52:0/3293157898 wait complete. 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.384+0000 7f3edcea9b80 1 -- start start 2024-06-02T23:16:18.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 Processor -- start 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 0x5597291de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291db4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d8390 0x5597291db9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597291deaf0 con 0x5597291d2330 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597290bfbf0 con 0x5597291d8390 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597291dec70 con 0x5597291d7a70 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3edcea9b80 10 client.0 _ll_register_callbacks cb 0x5597290b9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 0x5597291de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 0x5597291de1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35936/0 (socket says 172.21.15.52:35936) 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1148876012 learned_addr learned my addr 172.21.15.52:0/1148876012 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecb7fe700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291db4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d8390 0x5597291db9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1148876012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 msgr2=0x5597291db4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291db4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d8390 msgr2=0x5597291db9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d8390 0x5597291db9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.385+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1148876012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55972906e8d0 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 0x5597291de1b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc003900 tx=0x7f3ebc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f30ce85bdcb08da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3eab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1148876012 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc01d620 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1148876012 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ebc01d7c0 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ec409cfb0 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1148876012 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ebc01d9c0 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3edcea9b80 10 client.16860 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5597291cda40 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1148876012 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ec409cfb0 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1148876012 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3ebc027420 con 0x5597291d2330 2024-06-02T23:16:18.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3edcea9b80 10 client.16860 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.386+0000 7f3edcea9b80 10 client.16860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3edcea9b80 20 client.16860 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d00d0 0x5597291d0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3edcea9b80 4 client.16860 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55972912cd10 con 0x5597291d00d0 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3ecb7fe700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d00d0 0x5597291d0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3ecb7fe700 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d00d0 0x5597291d0540 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3196567943b5e1c9 server_cookie=46e2b5cb0090bcfd in_seq=0 out_seq=0 2024-06-02T23:16:18.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.387+0000 7f3eab7fe700 10 client.16860 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1148876012 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55972912cd10 con 0x5597291d00d0 2024-06-02T23:16:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3eab7fe700 10 client.16860 handle_command_reply: tid=0 2024-06-02T23:16:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 client.16860 shutdown 2024-06-02T23:16:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 msgr2=0x5597291de1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 0x5597291de1b0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc003900 tx=0x7f3ebc0047c0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 shutdown_connections 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d00d0 0x5597291d0540 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d8390 0x5597291db9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3eab7fe700 0 client.16860 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d2330 0x5597291de1b0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1148876012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d7a70 0x5597291db4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.388+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 >> 172.21.15.52:0/1148876012 conn(0x5597290f8cc0 msgr2=0x5597291cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.389+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 shutdown_connections 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.389+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1148876012 wait complete. 2024-06-02T23:16:18.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.389+0000 7f3edcea9b80 20 client.16860 trim_cache size 0 max 16384 2024-06-02T23:16:18.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 Processor -- start 2024-06-02T23:16:18.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 -- start start 2024-06-02T23:16:18.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 0x5597291d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d2330 0x5597291d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1aa730 con 0x5597291d7a70 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ac270 con 0x5597291d00d0 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1aae00 con 0x5597291d2330 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3ed0bc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 0x5597291d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3ecbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 0x5597291d1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41864/0 (socket says 172.21.15.52:41864) 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1976044891 learned_addr learned my addr 172.21.15.52:0/1976044891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.405+0000 7f3ecb7fe700 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d2330 0x5597291d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1976044891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d2330 msgr2=0x5597291d90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d2330 0x5597291d90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1976044891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 msgr2=0x5597291cd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291cd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3ecbfff700 1 -- 172.21.15.52:0/1976044891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597291ca2f0 con 0x5597291d00d0 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 0x5597291d1df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0058f0 tx=0x7f3ebc009370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d5885f1e823465c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3eb1ffb700 1 -- 172.21.15.52:0/1976044891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc003ef0 con 0x5597291d00d0 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3eb1ffb700 1 -- 172.21.15.52:0/1976044891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc01d620 con 0x5597291d00d0 2024-06-02T23:16:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.406+0000 7f3eb1ffb700 1 -- 172.21.15.52:0/1976044891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ebc01db60 con 0x5597291d00d0 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1976044891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 msgr2=0x5597291d1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 0x5597291d1df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0058f0 tx=0x7f3ebc009370 comp rx=0 tx=0).stop 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1976044891 shutdown_connections 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291cd0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d00d0 0x5597291d1df0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1976044891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d2330 0x5597291d90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1976044891 >> 172.21.15.52:0/1976044891 conn(0x5597290f8cc0 msgr2=0x5597291cc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1976044891 shutdown_connections 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1976044891 wait complete. 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.407+0000 7f3edcea9b80 1 -- start start 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 Processor -- start 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d00d0 0x5597291ee670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291eb380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1abba0 con 0x5597291d7a70 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1ab4d0 con 0x5597291d2330 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ecc1a86f0 con 0x5597291d00d0 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291eb380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291eb380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35964/0 (socket says 172.21.15.52:35964) 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/1827788600 learned_addr learned my addr 172.21.15.52:0/1827788600 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3edcea9b80 10 client.0 _ll_register_callbacks cb 0x5597290b9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ecb7fe700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d00d0 0x5597291ee670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/1827788600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d00d0 msgr2=0x5597291ee670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d00d0 0x5597291ee670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 msgr2=0x5597291eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.408+0000 7f3ed0bc3700 1 -- 172.21.15.52:0/1827788600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597291ebbc0 con 0x5597291d7a70 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3ed0bc3700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291eb380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40a4f60 tx=0x7f3ec40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e54cece2a68c0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3eab7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1827788600 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ec40ba060 con 0x5597291d7a70 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ebc01d620 con 0x5597291d7a70 2024-06-02T23:16:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1827788600 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ec40074e0 con 0x5597291d7a70 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1827788600 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ec409cfb0 con 0x5597291d7a70 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3edcea9b80 10 client.16866 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559729251f80 con 0x5597291d7a70 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1827788600 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ebc01d620 con 0x5597291d7a70 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1827788600 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3ec4011070 con 0x5597291d7a70 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3edcea9b80 10 client.16866 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3edcea9b80 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.409+0000 7f3edcea9b80 20 client.16866 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.410+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d8390 0x559729254400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.410+0000 7f3edcea9b80 4 client.16866 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.410+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559729127290 con 0x5597291d8390 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.410+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d8390 0x559729254400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.410+0000 7f3eab7fe700 10 client.16866 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.410+0000 7f3ecbfff700 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d8390 0x559729254400 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e709eed0de2f109 server_cookie=f36c96f06ea47ed9 in_seq=0 out_seq=0 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3eab7fe700 1 -- 172.21.15.52:0/1827788600 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559729127290 con 0x5597291d8390 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3eab7fe700 10 client.16866 handle_command_reply: tid=0 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 client.16866 shutdown 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 msgr2=0x5597291eb380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291eb380 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f3ec40a4f60 tx=0x7f3ec40099e0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 shutdown_connections 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5597291d8390 0x559729254400 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597291d7a70 0x5597291eb380 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597291d00d0 0x5597291ee670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 --2- 172.21.15.52:0/1827788600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597291d2330 0x5597291eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 >> 172.21.15.52:0/1827788600 conn(0x5597290f8cc0 msgr2=0x5597291d8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.411+0000 7f3eab7fe700 0 client.16866 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 shutdown_connections 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3edcea9b80 1 -- 172.21.15.52:0/1827788600 wait complete. 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3edcea9b80 20 client.16866 trim_cache size 0 max 16384 2024-06-02T23:16:18.418 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3eb40a2c20 msgr2=0x7f3eb40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3eb40a2c20 0x7f3eb40a50e0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009c60 tx=0x7f3ebc009260 comp rx=0 tx=0).stop 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc106720 msgr2=0x7f3ecc197810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc106720 0x7f3ecc197810 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3ec400b0d0 tx=0x7f3ec4007a00 comp rx=0 tx=0).stop 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 shutdown_connections 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3eb40a2c20 0x7f3eb40a50e0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ecc106720 0x7f3ecc197810 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ecc1019d0 0x7f3ecc19d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 --2- 172.21.15.52:0/650906930 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ecc102380 0x7f3ecc19db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.412+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 >> 172.21.15.52:0/650906930 conn(0x7f3ecc078340 msgr2=0x7f3ecc0ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.413+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 shutdown_connections 2024-06-02T23:16:18.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:18.413+0000 7f3ed2626700 1 -- 172.21.15.52:0/650906930 wait complete. 2024-06-02T23:16:18.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 77900 1.85 MB/sec execute 87 sec latency 15.987 ms 2024-06-02T23:16:19.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 78385 1.87 MB/sec execute 88 sec latency 12.446 ms 2024-06-02T23:16:19.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:19 smithi123 ceph-mon[35977]: pgmap v564: 97 pgs: 97 active+clean; 46 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-02T23:16:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:19 smithi174 ceph-mon[35994]: pgmap v564: 97 pgs: 97 active+clean; 46 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-02T23:16:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:19 smithi052 ceph-mon[32340]: pgmap v564: 97 pgs: 97 active+clean; 46 MiB data, 715 MiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-06-02T23:16:20.076 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:16:20.077 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:16:20.351 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:20.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 78665 1.86 MB/sec execute 89 sec latency 12.187 ms 2024-06-02T23:16:21.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 79155 1.86 MB/sec execute 90 sec latency 14.946 ms 2024-06-02T23:16:21.741 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:21 smithi052 ceph-mon[32340]: pgmap v565: 97 pgs: 97 active+clean; 48 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-06-02T23:16:21.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:21 smithi123 ceph-mon[35977]: pgmap v565: 97 pgs: 97 active+clean; 48 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-06-02T23:16:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:21 smithi174 ceph-mon[35994]: pgmap v565: 97 pgs: 97 active+clean; 48 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2024-06-02T23:16:22.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 79698 1.87 MB/sec execute 91 sec latency 40.098 ms 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 -- 172.21.15.52:0/3106255741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8102480 msgr2=0x7fcef8104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 --2- 172.21.15.52:0/3106255741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8102480 0x7fcef8104870 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef402a410 comp rx=0 tx=0).stop 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 -- 172.21.15.52:0/3106255741 shutdown_connections 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 --2- 172.21.15.52:0/3106255741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef8104db0 0x7fcef81071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 --2- 172.21.15.52:0/3106255741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 --2- 172.21.15.52:0/3106255741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8102480 0x7fcef8104870 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 -- 172.21.15.52:0/3106255741 >> 172.21.15.52:0/3106255741 conn(0x7fcef80fbe70 msgr2=0x7fcef80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 -- 172.21.15.52:0/3106255741 shutdown_connections 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.986+0000 7fcf00012640 1 -- 172.21.15.52:0/3106255741 wait complete. 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 Processor -- start 2024-06-02T23:16:22.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 -- start start 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef8102480 0x7fcef8101df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 0x7fcef8100430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8100970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcef810d560 con 0x7fcef8104db0 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcef810d3e0 con 0x7fcef81076e0 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcf00012640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcef810d6e0 con 0x7fcef8102480 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcefd5ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 0x7fcef8100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcefe5af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8100970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcefddae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef8102480 0x7fcef8101df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcefd5ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 0x7fcef8100430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36040/0 (socket says 172.21.15.52:36040) 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcefe5af640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8100970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41920/0 (socket says 172.21.15.52:41920) 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.987+0000 7fcefd5ad640 1 -- 172.21.15.52:0/1426781375 learned_addr learned my addr 172.21.15.52:0/1426781375 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:22.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefd5ad640 1 -- 172.21.15.52:0/1426781375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef8102480 msgr2=0x7fcef8101df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef8102480 0x7fcef8101df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefd5ad640 1 -- 172.21.15.52:0/1426781375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 msgr2=0x7fcef8100970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8100970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefd5ad640 1 -- 172.21.15.52:0/1426781375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef8101200 con 0x7fcef8104db0 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 0x7fcef8100430 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcee800c970 tx=0x7fcee800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fceeeffd640 1 -- 172.21.15.52:0/1426781375 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8007bb0 con 0x7fcef8104db0 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fceeeffd640 1 -- 172.21.15.52:0/1426781375 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcee8007d50 con 0x7fcef8104db0 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef81a1cf0 con 0x7fcef8104db0 2024-06-02T23:16:22.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fceeeffd640 1 -- 172.21.15.52:0/1426781375 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8018730 con 0x7fcef8104db0 2024-06-02T23:16:22.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.988+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef81a2190 con 0x7fcef8104db0 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fceeeffd640 1 -- 172.21.15.52:0/1426781375 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcee800f450 con 0x7fcef8104db0 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 Processor -- start 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 -- start start 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40c9e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40ca3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40ca6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc40c014f0 con 0x55bc40c9ea00 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc40b9f870 con 0x55bc40ca3b70 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.991+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc40bfd4b0 con 0x55bc40c9e010 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefd5ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40ca6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40ca3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40ca3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36052/0 (socket says 172.21.15.52:36052) 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 -- 172.21.15.52:0/1113568467 learned_addr learned my addr 172.21.15.52:0/1113568467 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40c9e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fceeeffd640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcee40a2cb0 0x7fcee40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 -- 172.21.15.52:0/1113568467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 msgr2=0x55bc40c9e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40c9e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 -- 172.21.15.52:0/1113568467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 msgr2=0x55bc40ca6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:22.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40ca6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcee40a2cb0 0x7fcee40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 -- 172.21.15.52:0/1113568467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc40bbb9b0 con 0x55bc40c9ea00 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fceeeffd640 1 -- 172.21.15.52:0/1426781375 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcee80cef80 con 0x7fcef8104db0 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.992+0000 7fcefddae640 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40ca3500 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef4031c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834f3ce981156a3b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcee3fff640 1 -- 172.21.15.52:0/1113568467 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4035030 con 0x55bc40c9ea00 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcee40a2cb0 0x7fcee40a5170 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcef00049d0 tx=0x7fcef0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcee3fff640 1 -- 172.21.15.52:0/1113568467 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcef4040070 con 0x55bc40c9ea00 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcee3fff640 1 -- 172.21.15.52:0/1113568467 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4003bb0 con 0x55bc40c9ea00 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1113568467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 msgr2=0x55bc40ca3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40ca3500 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fcef4009900 tx=0x7fcef4031c30 comp rx=0 tx=0).stop 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1113568467 shutdown_connections 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.993+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40ca6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40ca3500 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1113568467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40c9e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1113568467 >> 172.21.15.52:0/1113568467 conn(0x55bc40c99680 msgr2=0x55bc40c99a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1113568467 shutdown_connections 2024-06-02T23:16:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1113568467 wait complete. 2024-06-02T23:16:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 -- start start 2024-06-02T23:16:22.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 Processor -- start 2024-06-02T23:16:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40ca8230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.994+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40c9ea00 0x55bc40ca8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 0x55bc40caf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc40ca6ac0 con 0x55bc40ca3b70 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc40b8aae0 con 0x55bc40c9ea00 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc40ca6c40 con 0x55bc40c9e010 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcf02dfb740 10 client.0 _ll_register_callbacks cb 0x55bc40bc39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 0x55bc40caf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 0x55bc40caf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36054/0 (socket says 172.21.15.52:36054) 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 -- 172.21.15.52:0/1584222328 learned_addr learned my addr 172.21.15.52:0/1584222328 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefddae640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40ca8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40c9ea00 0x55bc40ca8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 -- 172.21.15.52:0/1584222328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 msgr2=0x55bc40ca8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40ca8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 -- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40c9ea00 msgr2=0x55bc40ca8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40c9ea00 0x55bc40ca8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 -- 172.21.15.52:0/1584222328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc40915210 con 0x55bc40ca3b70 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40c9ea00 0x55bc40ca8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcefe5af640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 0x55bc40caf7b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcef000fd30 tx=0x7fcef0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93707588aaa935bc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcee17fa640 1 -- 172.21.15.52:0/1584222328 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef000fe20 con 0x55bc40ca3b70 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcee17fa640 1 -- 172.21.15.52:0/1584222328 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcef0018050 con 0x55bc40ca3b70 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.995+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef4040070 con 0x55bc40ca3b70 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcee17fa640 1 -- 172.21.15.52:0/1584222328 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcef001e930 con 0x55bc40ca3b70 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 10 client.16896 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc409e1620 con 0x55bc40ca3b70 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcee17fa640 1 -- 172.21.15.52:0/1584222328 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcef4040070 con 0x55bc40ca3b70 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcee17fa640 1 -- 172.21.15.52:0/1584222328 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcef001a070 con 0x55bc40ca3b70 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 10 client.16896 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 20 client.16896 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40cb6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 4 client.16896 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.996+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc40c14f10 con 0x55bc40cb4120 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.997+0000 7fcefddae640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40cb6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.997+0000 7fcefddae640 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40cb6580 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7fb18c88c695e server_cookie=ae9a3cb165a72019 in_seq=0 out_seq=0 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.997+0000 7fcee17fa640 10 client.16896 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcee17fa640 1 -- 172.21.15.52:0/1584222328 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc40c14f10 con 0x55bc40cb4120 2024-06-02T23:16:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcee17fa640 10 client.16896 handle_command_reply: tid=0 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 client.16896 shutdown 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 msgr2=0x55bc40caf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 0x55bc40caf7b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fcef000fd30 tx=0x7fcef0009500 comp rx=0 tx=0).stop 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 shutdown_connections 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40cb6580 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcee17fa640 0 client.16896 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40ca3b70 0x55bc40caf7b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40c9ea00 0x55bc40ca8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/1584222328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40ca8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 >> 172.21.15.52:0/1584222328 conn(0x55bc40c99680 msgr2=0x55bc40c987e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 shutdown_connections 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 1 -- 172.21.15.52:0/1584222328 wait complete. 2024-06-02T23:16:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:22.998+0000 7fcf02dfb740 20 client.16896 trim_cache size 0 max 16384 2024-06-02T23:16:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 Processor -- start 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 -- start start 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 0x55bc40c9fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9ea00 0x55bc40ca0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40caf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81aaed0 con 0x55bc40c9e010 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81aca10 con 0x55bc40ca3b70 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81ab5a0 con 0x55bc40c9ea00 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcefddae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 0x55bc40c9fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcefddae640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 0x55bc40c9fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36066/0 (socket says 172.21.15.52:36066) 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcefddae640 1 -- 172.21.15.52:0/590119502 learned_addr learned my addr 172.21.15.52:0/590119502 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcefe5af640 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40caf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.020+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9ea00 0x55bc40ca0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcefddae640 1 -- 172.21.15.52:0/590119502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9ea00 msgr2=0x55bc40ca0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcefddae640 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9ea00 0x55bc40ca0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcefddae640 1 -- 172.21.15.52:0/590119502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 msgr2=0x55bc40caf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcefddae640 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40caf770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcefddae640 1 -- 172.21.15.52:0/590119502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc40c9e7a0 con 0x55bc40c9e010 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcefddae640 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 0x55bc40c9fc20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcef402a450 tx=0x7fcef4031da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6198ae8b57754188 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcee37fe640 1 -- 172.21.15.52:0/590119502 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef4035040 con 0x55bc40c9e010 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcee37fe640 1 -- 172.21.15.52:0/590119502 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcef4040070 con 0x55bc40c9e010 2024-06-02T23:16:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.021+0000 7fcee37fe640 1 -- 172.21.15.52:0/590119502 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcef403c440 con 0x55bc40c9e010 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 -- 172.21.15.52:0/590119502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 msgr2=0x55bc40c9fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 0x55bc40c9fc20 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcef402a450 tx=0x7fcef4031da0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 -- 172.21.15.52:0/590119502 shutdown_connections 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9ea00 0x55bc40ca0160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40caf770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/590119502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9e010 0x55bc40c9fc20 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 -- 172.21.15.52:0/590119502 >> 172.21.15.52:0/590119502 conn(0x55bc40c99680 msgr2=0x55bc40ca8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 -- 172.21.15.52:0/590119502 shutdown_connections 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 -- 172.21.15.52:0/590119502 wait complete. 2024-06-02T23:16:23.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 -- start start 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.022+0000 7fcf02dfb740 1 Processor -- start 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40cac460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40cac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40d1ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81ac340 con 0x55bc40c9ea00 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81abc70 con 0x55bc40ca3b70 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcef81a8eb0 con 0x55bc40c9e010 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcf02dfb740 10 client.0 _ll_register_callbacks cb 0x7fcef0012f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40cac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40cac9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36078/0 (socket says 172.21.15.52:36078) 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 -- 172.21.15.52:0/3622111782 learned_addr learned my addr 172.21.15.52:0/3622111782 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefddae640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40cac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefe5af640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40d1ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 -- 172.21.15.52:0/3622111782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 msgr2=0x55bc40cac460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40cac460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 -- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 msgr2=0x55bc40d1ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40d1ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 -- 172.21.15.52:0/3622111782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc40caf9d0 con 0x55bc40c9ea00 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefe5af640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40d1ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcefd5ad640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40cac9a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcee80981e0 tx=0x7fcee800a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544c5c69f15e51b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcee17fa640 1 -- 172.21.15.52:0/3622111782 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee809b350 con 0x55bc40c9ea00 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.023+0000 7fcee17fa640 1 -- 172.21.15.52:0/3622111782 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcee800f740 con 0x55bc40c9ea00 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcef4040070 con 0x55bc40c9ea00 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcee17fa640 1 -- 172.21.15.52:0/3622111782 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcee809b170 con 0x55bc40c9ea00 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 10 client.16902 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc40d1e4a0 con 0x55bc40c9ea00 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcee17fa640 1 -- 172.21.15.52:0/3622111782 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcef4040070 con 0x55bc40c9ea00 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcee17fa640 1 -- 172.21.15.52:0/3622111782 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcee8020030 con 0x55bc40c9ea00 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 10 client.16902 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 10 client.16902 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 20 client.16902 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40d20f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 4 client.16902 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bc40c0ec20 con 0x55bc40cb4120 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.024+0000 7fcefddae640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40d20f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.025+0000 7fcee17fa640 10 client.16902 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.025+0000 7fcefddae640 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40d20f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea9c7dac658f5c27 server_cookie=d89235d5ebd3807f in_seq=0 out_seq=0 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcee17fa640 1 -- 172.21.15.52:0/3622111782 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55bc40c0ec20 con 0x55bc40cb4120 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcee17fa640 10 client.16902 handle_command_reply: tid=0 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 client.16902 shutdown 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 msgr2=0x55bc40cac9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40cac9a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fcee80981e0 tx=0x7fcee800a550 comp rx=0 tx=0).stop 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 shutdown_connections 2024-06-02T23:16:23.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc40cb4120 0x55bc40d20f10 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc40ca3b70 0x55bc40d1ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc40c9ea00 0x55bc40cac9a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcee17fa640 0 client.16902 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 --2- 172.21.15.52:0/3622111782 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc40c9e010 0x55bc40cac460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 >> 172.21.15.52:0/3622111782 conn(0x55bc40c99680 msgr2=0x55bc40c9fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 shutdown_connections 2024-06-02T23:16:23.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 1 -- 172.21.15.52:0/3622111782 wait complete. 2024-06-02T23:16:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.042+0000 7fcf02dfb740 20 client.16902 trim_cache size 0 max 16384 2024-06-02T23:16:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:16:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:16:23.045 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:16:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcee40a2cb0 msgr2=0x7fcee40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.046 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcee40a2cb0 0x7fcee40a5170 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fcef00049d0 tx=0x7fcef0004a40 comp rx=0 tx=0).stop 2024-06-02T23:16:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 msgr2=0x7fcef8100430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 0x7fcef8100430 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fcee800c970 tx=0x7fcee800ce30 comp rx=0 tx=0).stop 2024-06-02T23:16:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 shutdown_connections 2024-06-02T23:16:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcee40a2cb0 0x7fcee40a5170 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.047 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcef81076e0 0x7fcef8100970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcef8104db0 0x7fcef8100430 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 --2- 172.21.15.52:0/1426781375 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef8102480 0x7fcef8101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 >> 172.21.15.52:0/1426781375 conn(0x7fcef80fbe70 msgr2=0x7fcef8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 shutdown_connections 2024-06-02T23:16:23.048 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:23.043+0000 7fcf00012640 1 -- 172.21.15.52:0/1426781375 wait complete. 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 -- 172.21.15.52:0/3454779768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 msgr2=0x7f83e0105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 --2- 172.21.15.52:0/3454779768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e0105ff0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f83d40099c0 tx=0x7f83d4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 -- 172.21.15.52:0/3454779768 shutdown_connections 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 --2- 172.21.15.52:0/3454779768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e0106720 0x7f83e010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 --2- 172.21.15.52:0/3454779768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83e01019d0 0x7f83e0101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 --2- 172.21.15.52:0/3454779768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e0105ff0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 -- 172.21.15.52:0/3454779768 >> 172.21.15.52:0/3454779768 conn(0x7f83e0078340 msgr2=0x7f83e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 -- 172.21.15.52:0/3454779768 shutdown_connections 2024-06-02T23:16:23.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.619+0000 7f83e5dac700 1 -- 172.21.15.52:0/3454779768 wait complete. 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 Processor -- start 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 -- start start 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e01019d0 0x7f83e019d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e019dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83e0106720 0x7f83e01978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83e0111570 con 0x7f83e0102380 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01113f0 con 0x7f83e0106720 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83e5dac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01116f0 con 0x7f83e01019d0 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e019dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e019dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35978/0 (socket says 172.21.15.52:35978) 2024-06-02T23:16:23.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83deffd700 1 -- 172.21.15.52:0/1385681604 learned_addr learned my addr 172.21.15.52:0/1385681604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.356 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:16:23.356 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:16:23.356 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "e9592e3a-5c67-4704-a965-f9b9cadbcfeb", 2024-06-02T23:16:23.356 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:16:23.356 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83df7fe700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e01019d0 0x7f83e019d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.620+0000 7f83dffff700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83e0106720 0x7f83e01978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83deffd700 1 -- 172.21.15.52:0/1385681604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e01019d0 msgr2=0x7f83e019d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83deffd700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e01019d0 0x7f83e019d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83deffd700 1 -- 172.21.15.52:0/1385681604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83e0106720 msgr2=0x7f83e01978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83deffd700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83e0106720 0x7f83e01978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83deffd700 1 -- 172.21.15.52:0/1385681604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e0198010 con 0x7f83e0102380 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83deffd700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e019dc30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009fd0 tx=0x7f83d4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83dcff9700 1 -- 172.21.15.52:0/1385681604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4026070 con 0x7f83e0102380 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e01982a0 con 0x7f83e0102380 2024-06-02T23:16:23.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83dcff9700 1 -- 172.21.15.52:0/1385681604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83d4004ca0 con 0x7f83e0102380 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83dcff9700 1 -- 172.21.15.52:0/1385681604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d4003d00 con 0x7f83e0102380 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.621+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e01a45c0 con 0x7f83e0102380 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.623+0000 7f83dcff9700 1 -- 172.21.15.52:0/1385681604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f83d4029070 con 0x7f83e0102380 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.624+0000 7f83dcff9700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83c80a2c20 0x7f83c80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.624+0000 7f83df7fe700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83c80a2c20 0x7f83c80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.624+0000 7f83dcff9700 1 -- 172.21.15.52:0/1385681604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83d40cf9f0 con 0x7f83e0102380 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83df7fe700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83c80a2c20 0x7f83c80a50e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f83d00060b0 tx=0x7f83d000c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 Processor -- start 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 -- start start 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bb98cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bb92c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc6bae2890 con 0x55bc6bb991f0 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc6ba8a7a0 con 0x55bc6bb988d0 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.625+0000 7f83f062fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc6bade850 con 0x55bc6bb931b0 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bb92c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bb92c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35984/0 (socket says 172.21.15.52:35984) 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 -- 172.21.15.52:0/670468780 learned_addr learned my addr 172.21.15.52:0/670468780 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83df7fe700 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bb98cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83dffff700 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 -- 172.21.15.52:0/670468780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 msgr2=0x55bc6bb935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 -- 172.21.15.52:0/670468780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 msgr2=0x55bc6bb98cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bb98cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 -- 172.21.15.52:0/670468780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6ba80fd0 con 0x55bc6bb991f0 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83df7fe700 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bb98cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83deffd700 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bb92c70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f83d409c2a0 tx=0x7f83d4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e609c569fedc0cbd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83c5ffb700 1 -- 172.21.15.52:0/670468780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d409ca10 con 0x55bc6bb991f0 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83c5ffb700 1 -- 172.21.15.52:0/670468780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83d409c310 con 0x55bc6bb991f0 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.626+0000 7f83c5ffb700 1 -- 172.21.15.52:0/670468780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40a4cc0 con 0x55bc6bb991f0 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 -- 172.21.15.52:0/670468780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 msgr2=0x55bc6bb92c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bb92c70 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f83d409c2a0 tx=0x7f83d4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 -- 172.21.15.52:0/670468780 shutdown_connections 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bb98cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 --2- 172.21.15.52:0/670468780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bb92c70 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 -- 172.21.15.52:0/670468780 >> 172.21.15.52:0/670468780 conn(0x55bc6babbd40 msgr2=0x55bc6bb8b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 -- 172.21.15.52:0/670468780 shutdown_connections 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.627+0000 7f83f062fb80 1 -- 172.21.15.52:0/670468780 wait complete. 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 -- start start 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 Processor -- start 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bb9ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bb9c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bb9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55bc6bae2890 con 0x55bc6bb988d0 2024-06-02T23:16:23.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bc6ba82c70 con 0x55bc6bb931b0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bc6bb9f980 con 0x55bc6bb991f0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83f062fb80 10 client.0 _ll_register_callbacks cb 0x55bc6ba7c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bb9c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83df7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bb9ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bb9c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35992/0 (socket says 172.21.15.52:35992) 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.628+0000 7f83deffd700 1 -- 172.21.15.52:0/2066602535 learned_addr learned my addr 172.21.15.52:0/2066602535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83dffff700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bb9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83deffd700 1 -- 172.21.15.52:0/2066602535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 msgr2=0x55bc6bb9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83deffd700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bb9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83deffd700 1 -- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 msgr2=0x55bc6bb9ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83deffd700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bb9ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83deffd700 1 -- 172.21.15.52:0/2066602535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6ba31950 con 0x55bc6bb988d0 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83df7fe700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bb9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:23.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83deffd700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bb9c2c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f83d4098720 tx=0x7f83d4004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebea6eb528a2722 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83beffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83beffd700 1 -- 172.21.15.52:0/2066602535 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40b9020 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83beffd700 1 -- 172.21.15.52:0/2066602535 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83d40bc0d0 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d409c310 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83beffd700 1 -- 172.21.15.52:0/2066602535 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83d40a06c0 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83f062fb80 10 client.16878 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc6bbaa110 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.629+0000 7f83beffd700 1 -- 172.21.15.52:0/2066602535 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83d409c310 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83beffd700 1 -- 172.21.15.52:0/2066602535 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f83d407f020 con 0x55bc6bb988d0 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83f062fb80 10 client.16878 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83f062fb80 10 client.16878 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83f062fb80 20 client.16878 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bbac5e0 0x55bc6bbae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83f062fb80 4 client.16878 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:23.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bc6baefd90 con 0x55bc6bbac5e0 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83df7fe700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bbac5e0 0x55bc6bbae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83df7fe700 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bbac5e0 0x55bc6bbae9d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aef2a57ebce0530 server_cookie=fbd4bfa629956e15 in_seq=0 out_seq=0 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.630+0000 7f83beffd700 10 client.16878 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83beffd700 1 -- 172.21.15.52:0/2066602535 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55bc6baefd90 con 0x55bc6bbac5e0 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83beffd700 10 client.16878 handle_command_reply: tid=0 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 client.16878 shutdown 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 msgr2=0x55bc6bb9c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bb9c2c0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f83d4098720 tx=0x7f83d4004f00 comp rx=0 tx=0).stop 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 shutdown_connections 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bbac5e0 0x55bc6bbae9d0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bb9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bb9ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83beffd700 0 client.16878 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2066602535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bb9c2c0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 >> 172.21.15.52:0/2066602535 conn(0x55bc6babbd40 msgr2=0x55bc6bb8b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 shutdown_connections 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.631+0000 7f83f062fb80 1 -- 172.21.15.52:0/2066602535 wait complete. 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.632+0000 7f83f062fb80 20 client.16878 trim_cache size 0 max 16384 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 Processor -- start 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 -- start start 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb99da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bbb4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bbb47b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01aa7e0 con 0x55bc6bb991f0 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01ac320 con 0x55bc6bb988d0 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83f062fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01aaeb0 con 0x55bc6bb931b0 2024-06-02T23:16:23.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bbb47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83deffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bbb4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83deffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bbb4270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41902/0 (socket says 172.21.15.52:41902) 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bbb47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36006/0 (socket says 172.21.15.52:36006) 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83deffd700 1 -- 172.21.15.52:0/3761734682 learned_addr learned my addr 172.21.15.52:0/3761734682 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.661+0000 7f83df7fe700 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb99da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83deffd700 1 -- 172.21.15.52:0/3761734682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 msgr2=0x55bc6bb99da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83deffd700 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb99da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83deffd700 1 -- 172.21.15.52:0/3761734682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 msgr2=0x55bc6bbb47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83deffd700 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bbb47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83deffd700 1 -- 172.21.15.52:0/3761734682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bc6bb8c960 con 0x55bc6bb988d0 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83df7fe700 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb99da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:23.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83dffff700 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bbb47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83deffd700 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bbb4270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f83d409c2a0 tx=0x7f83d40b5360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=402ffb831b958f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83c57fa700 1 -- 172.21.15.52:0/3761734682 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40b9070 con 0x55bc6bb988d0 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83c57fa700 1 -- 172.21.15.52:0/3761734682 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83d407c070 con 0x55bc6bb988d0 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.662+0000 7f83c57fa700 1 -- 172.21.15.52:0/3761734682 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d409a3b0 con 0x55bc6bb988d0 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 -- 172.21.15.52:0/3761734682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 msgr2=0x55bc6bbb4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bbb4270 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f83d409c2a0 tx=0x7f83d40b5360 comp rx=0 tx=0).stop 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 -- 172.21.15.52:0/3761734682 shutdown_connections 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb991f0 0x55bc6bbb47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb931b0 0x55bc6bb99da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 --2- 172.21.15.52:0/3761734682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb988d0 0x55bc6bbb4270 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 -- 172.21.15.52:0/3761734682 >> 172.21.15.52:0/3761734682 conn(0x55bc6babbd40 msgr2=0x55bc6bb99690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 -- 172.21.15.52:0/3761734682 shutdown_connections 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.663+0000 7f83f062fb80 1 -- 172.21.15.52:0/3761734682 wait complete. 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 -- start start 2024-06-02T23:16:23.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 Processor -- start 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bba5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bba1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bba2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01abc50 con 0x55bc6bb988d0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01ab580 con 0x55bc6bb931b0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83e01a8780 con 0x55bc6bb991f0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83f062fb80 10 client.0 _ll_register_callbacks cb 0x55bc6ba7c9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bba1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83deffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bba1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36016/0 (socket says 172.21.15.52:36016) 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.664+0000 7f83deffd700 1 -- 172.21.15.52:0/2872424720 learned_addr learned my addr 172.21.15.52:0/2872424720 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83dffff700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bba2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83df7fe700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bba5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83deffd700 1 -- 172.21.15.52:0/2872424720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 msgr2=0x55bc6bba2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83deffd700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bba2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83deffd700 1 -- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 msgr2=0x55bc6bba5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83deffd700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bba5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83deffd700 1 -- 172.21.15.52:0/2872424720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bc6bc17250 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83deffd700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bba1c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f83d409c010 tx=0x7f83d4009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4ec40ec71ac63d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83beffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83beffd700 1 -- 172.21.15.52:0/2872424720 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83d40b9070 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83d407c070 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83beffd700 1 -- 172.21.15.52:0/2872424720 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83d409b810 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83beffd700 1 -- 172.21.15.52:0/2872424720 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f83d409c310 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83f062fb80 10 client.16884 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55bc6bc17250 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.665+0000 7f83beffd700 1 -- 172.21.15.52:0/2872424720 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83d407c070 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83beffd700 1 -- 172.21.15.52:0/2872424720 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f83d407f070 con 0x55bc6bb988d0 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83f062fb80 10 client.16884 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83f062fb80 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:23.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83f062fb80 20 client.16884 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bc18700 0x55bc6bc1aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83f062fb80 4 client.16884 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55bc6baea310 con 0x55bc6bc18700 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83df7fe700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bc18700 0x55bc6bc1aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83df7fe700 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bc18700 0x55bc6bc1aaf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e2148983ddaaf0 server_cookie=aa5af6c44a945c96 in_seq=0 out_seq=0 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.666+0000 7f83beffd700 10 client.16884 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83beffd700 1 -- 172.21.15.52:0/2872424720 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55bc6baea310 con 0x55bc6bc18700 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83beffd700 10 client.16884 handle_command_reply: tid=0 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 client.16884 shutdown 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 msgr2=0x55bc6bba1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bba1c10 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f83d409c010 tx=0x7f83d4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 shutdown_connections 2024-06-02T23:16:23.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55bc6bc18700 0x55bc6bc1aaf0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bc6bb991f0 0x55bc6bba2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bc6bb931b0 0x55bc6bba5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 --2- 172.21.15.52:0/2872424720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55bc6bb988d0 0x55bc6bba1c10 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83beffd700 0 client.16884 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 >> 172.21.15.52:0/2872424720 conn(0x55bc6babbd40 msgr2=0x55bc6bb99610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 shutdown_connections 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.667+0000 7f83f062fb80 1 -- 172.21.15.52:0/2872424720 wait complete. 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83f062fb80 20 client.16884 trim_cache size 0 max 16384 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83c80a2c20 msgr2=0x7f83c80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83c80a2c20 0x7f83c80a50e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f83d00060b0 tx=0x7f83d000c040 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 msgr2=0x7f83e019dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e019dc30 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f83d4009fd0 tx=0x7f83d4005f10 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 shutdown_connections 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83c80a2c20 0x7f83c80a50e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83e0106720 0x7f83e01978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83e01019d0 0x7f83e019d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 --2- 172.21.15.52:0/1385681604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83e0102380 0x7f83e019dc30 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:23.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 >> 172.21.15.52:0/1385681604 conn(0x7f83e0078340 msgr2=0x7f83e00ffa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:23.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 shutdown_connections 2024-06-02T23:16:23.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:22.668+0000 7f83e5dac700 1 -- 172.21.15.52:0/1385681604 wait complete. 2024-06-02T23:16:23.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 80230 1.86 MB/sec execute 92 sec latency 76.058 ms 2024-06-02T23:16:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: pgmap v566: 97 pgs: 97 active+clean; 49 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-02T23:16:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:23.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: pgmap v566: 97 pgs: 97 active+clean; 49 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-02T23:16:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:23.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: pgmap v566: 97 pgs: 97 active+clean; 49 MiB data, 721 MiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-02T23:16:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:23.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:24.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 80678 1.86 MB/sec execute 93 sec latency 119.425 ms 2024-06-02T23:16:24.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:24 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:16:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:24 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:24 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:16:24.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:24 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:24 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:16:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:24 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:24 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:16:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:24 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:24 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:16:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:24 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:24 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:16:24.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:24 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:16:25.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 81378 1.91 MB/sec execute 94 sec latency 95.418 ms 2024-06-02T23:16:25.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:25 smithi123 ceph-mon[35977]: pgmap v567: 97 pgs: 97 active+clean; 49 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-02T23:16:25.759 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:25 smithi052 ceph-mon[32340]: pgmap v567: 97 pgs: 97 active+clean; 49 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-02T23:16:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:25 smithi174 ceph-mon[35994]: pgmap v567: 97 pgs: 97 active+clean; 49 MiB data, 722 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-02T23:16:26.342 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:16:26.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 81781 1.93 MB/sec execute 95 sec latency 22.157 ms 2024-06-02T23:16:26.660 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:27.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 82065 1.92 MB/sec execute 96 sec latency 13.536 ms 2024-06-02T23:16:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:27 smithi123 ceph-mon[35977]: pgmap v568: 97 pgs: 97 active+clean; 49 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:16:27.807 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:27 smithi052 ceph-mon[32340]: pgmap v568: 97 pgs: 97 active+clean; 49 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:16:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:27 smithi174 ceph-mon[35994]: pgmap v568: 97 pgs: 97 active+clean; 49 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:16:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 -- 172.21.15.52:0/1944869393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc1073f0 msgr2=0x7f36dc1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1944869393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc1073f0 0x7f36dc1077d0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f36d00098f0 tx=0x7f36d0009c00 comp rx=0 tx=0).stop 2024-06-02T23:16:28.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 -- 172.21.15.52:0/1944869393 shutdown_connections 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1944869393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc101d40 0x7f36dc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1944869393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc1073f0 0x7f36dc1077d0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1944869393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36dc101380 0x7f36dc101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 -- 172.21.15.52:0/1944869393 >> 172.21.15.52:0/1944869393 conn(0x7f36dc078300 msgr2=0x7f36dc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.420+0000 7f36e2aea700 1 -- 172.21.15.52:0/1944869393 shutdown_connections 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.421+0000 7f36e2aea700 1 -- 172.21.15.52:0/1944869393 wait complete. 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.421+0000 7f36e2aea700 1 Processor -- start 2024-06-02T23:16:28.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.421+0000 7f36e2aea700 1 -- start start 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.421+0000 7f36e2aea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 0x7f36dc19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.421+0000 7f36e2aea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36dc101d40 0x7f36dc19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.421+0000 7f36e2aea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc1073f0 0x7f36dc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e2aea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc111540 con 0x7f36dc101380 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e2aea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1113c0 con 0x7f36dc101d40 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e2aea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1116c0 con 0x7f36dc1073f0 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 0x7f36dc19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 0x7f36dc19e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47224/0 (socket says 172.21.15.52:47224) 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 -- 172.21.15.52:0/1699500880 learned_addr learned my addr 172.21.15.52:0/1699500880 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36dbfff700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36dc101d40 0x7f36dc19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e1087700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc1073f0 0x7f36dc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 -- 172.21.15.52:0/1699500880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc1073f0 msgr2=0x7f36dc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc1073f0 0x7f36dc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 -- 172.21.15.52:0/1699500880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36dc101d40 msgr2=0x7f36dc19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36dc101d40 0x7f36dc19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 -- 172.21.15.52:0/1699500880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc1990d0 con 0x7f36dc101380 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e0886700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 0x7f36dc19e7b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f36d000f990 tx=0x7f36d000f9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36d9ffb700 1 -- 172.21.15.52:0/1699500880 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0027070 con 0x7f36dc101380 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc199360 con 0x7f36dc101380 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36d9ffb700 1 -- 172.21.15.52:0/1699500880 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d0005570 con 0x7f36dc101380 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.422+0000 7f36d9ffb700 1 -- 172.21.15.52:0/1699500880 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0004930 con 0x7f36dc101380 2024-06-02T23:16:28.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.423+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a5680 con 0x7f36dc101380 2024-06-02T23:16:28.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.425+0000 7f36d9ffb700 1 -- 172.21.15.52:0/1699500880 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36d002a070 con 0x7f36dc101380 2024-06-02T23:16:28.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.426+0000 7f36d9ffb700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36cc0a2c20 0x7f36cc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.426+0000 7f36dbfff700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36cc0a2c20 0x7f36cc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.426+0000 7f36d9ffb700 1 -- 172.21.15.52:0/1699500880 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36d00cfaa0 con 0x7f36dc101380 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36dbfff700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36cc0a2c20 0x7f36cc0a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009750 tx=0x7f36c8009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 Processor -- start 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 -- start start 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0bfd0 0x557098d0c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 0x557098d106f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d11090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557098c59810 con 0x557098d10c30 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557098c01720 con 0x557098d0c8f0 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.427+0000 7f36ed36db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557098c557d0 con 0x557098d0bfd0 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 0x557098d106f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36e1087700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d11090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 0x557098d106f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34770/0 (socket says 172.21.15.52:34770) 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 -- 172.21.15.52:0/3280159596 learned_addr learned my addr 172.21.15.52:0/3280159596 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36e0886700 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0bfd0 0x557098d0c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 -- 172.21.15.52:0/3280159596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0bfd0 msgr2=0x557098d0c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0bfd0 0x557098d0c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 -- 172.21.15.52:0/3280159596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 msgr2=0x557098d11090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d11090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 -- 172.21.15.52:0/3280159596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557098bf7f50 con 0x557098d0c8f0 2024-06-02T23:16:28.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36dbfff700 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 0x557098d106f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009a60 tx=0x7f36c8014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0d98465cdc7108 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36c6ffd700 1 -- 172.21.15.52:0/3280159596 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c8026430 con 0x557098d0c8f0 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36c6ffd700 1 -- 172.21.15.52:0/3280159596 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36c801c8c0 con 0x557098d0c8f0 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.428+0000 7f36c6ffd700 1 -- 172.21.15.52:0/3280159596 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36c801d8b0 con 0x557098d0c8f0 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 -- 172.21.15.52:0/3280159596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 msgr2=0x557098d106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 0x557098d106f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009a60 tx=0x7f36c8014000 comp rx=0 tx=0).stop 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 -- 172.21.15.52:0/3280159596 shutdown_connections 2024-06-02T23:16:28.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d11090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0bfd0 0x557098d0c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 --2- 172.21.15.52:0/3280159596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0c8f0 0x557098d106f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 -- 172.21.15.52:0/3280159596 >> 172.21.15.52:0/3280159596 conn(0x557098c32cc0 msgr2=0x557098d02850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 -- 172.21.15.52:0/3280159596 shutdown_connections 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.429+0000 7f36ed36db80 1 -- 172.21.15.52:0/3280159596 wait complete. 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 -- start start 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 Processor -- start 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d15560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d15aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557098d18d70 con 0x557098d10c30 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557098bf9bf0 con 0x557098d0bfd0 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557098d18ef0 con 0x557098d0c8f0 2024-06-02T23:16:28.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36ed36db80 10 client.0 _ll_register_callbacks cb 0x557098bf3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36e0886700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d15560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.430+0000 7f36e1087700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d15aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34772/0 (socket says 172.21.15.52:34772) 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 -- 172.21.15.52:0/1783821899 learned_addr learned my addr 172.21.15.52:0/1783821899 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 -- 172.21.15.52:0/1783821899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 msgr2=0x557098d15560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d15560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 -- 172.21.15.52:0/1783821899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 msgr2=0x557098d15aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d15aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 -- 172.21.15.52:0/1783821899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557098ba88d0 con 0x557098d0bfd0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36e0886700 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d181f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009810 tx=0x7f36d009bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=123b48769b37ea1b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:16:28.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36bbfff700 1 -- 172.21.15.52:0/1783821899 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d00a0770 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36bbfff700 1 -- 172.21.15.52:0/1783821899 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36d0009400 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36c801c8c0 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36bbfff700 1 -- 172.21.15.52:0/1783821899 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36d00a4fa0 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36ed36db80 10 client.26444 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.431+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557098d23760 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36bbfff700 1 -- 172.21.15.52:0/1783821899 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d00b5110 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36bbfff700 1 -- 172.21.15.52:0/1783821899 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f36d00a4b10 con 0x557098d0bfd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36ed36db80 10 client.26444 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36ed36db80 10 client.26444 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36ed36db80 20 client.26444 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36ed36db80 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d25cd0 0x557098d280e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36ed36db80 4 client.26444 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f36c801c8c0 con 0x557098d25cd0 2024-06-02T23:16:28.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.432+0000 7f36dbfff700 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d25cd0 0x557098d280e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.433+0000 7f36dbfff700 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d25cd0 0x557098d280e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6445bb2aab2e97d server_cookie=53503882a21c7947 in_seq=0 out_seq=0 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.433+0000 7f36bbfff700 10 client.26444 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.433+0000 7f36bbfff700 1 -- 172.21.15.52:0/1783821899 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f36c801c8c0 con 0x557098d25cd0 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.433+0000 7f36bbfff700 10 client.26444 handle_command_reply: tid=0 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.433+0000 7f36ed36db80 1 client.26444 shutdown 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.433+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 msgr2=0x557098d181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d181f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009810 tx=0x7f36d009bda0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 shutdown_connections 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d25cd0 0x557098d280e0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d10c30 0x557098d15aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d181f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 --2- 172.21.15.52:0/1783821899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d15560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 >> 172.21.15.52:0/1783821899 conn(0x557098c32cc0 msgr2=0x557098d069d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:28.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36bbfff700 0 client.26444 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:28.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 shutdown_connections 2024-06-02T23:16:28.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 1 -- 172.21.15.52:0/1783821899 wait complete. 2024-06-02T23:16:28.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.434+0000 7f36ed36db80 20 client.26444 trim_cache size 0 max 16384 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.456+0000 7f36ed36db80 1 Processor -- start 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 -- start start 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 0x557098cfbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098cfc340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d0c8f0 0x557098d2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1aa630 con 0x557098d0c8f0 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ac170 con 0x557098d0bfd0 2024-06-02T23:16:28.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36ed36db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1aad00 con 0x557098c32cc0 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 0x557098cfbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098cfc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 0x557098cfbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53346/0 (socket says 172.21.15.52:53346) 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098cfc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34784/0 (socket says 172.21.15.52:34784) 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 -- 172.21.15.52:0/2097760175 learned_addr learned my addr 172.21.15.52:0/2097760175 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 -- 172.21.15.52:0/2097760175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 msgr2=0x557098cfc340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e1087700 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d0c8f0 0x557098d2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098cfc340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 -- 172.21.15.52:0/2097760175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d0c8f0 msgr2=0x557098d2d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d0c8f0 0x557098d2d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e0886700 1 -- 172.21.15.52:0/2097760175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557098d04280 con 0x557098c32cc0 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36dbfff700 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098cfc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.457+0000 7f36e1087700 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d0c8f0 0x557098d2d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.458+0000 7f36e0886700 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 0x557098cfbe00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f36d00cd620 tx=0x7f36d000b9d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4cde4bb9f93ddf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.458+0000 7f36c67fc700 1 -- 172.21.15.52:0/2097760175 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0009400 con 0x557098c32cc0 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.458+0000 7f36c67fc700 1 -- 172.21.15.52:0/2097760175 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d00a4b10 con 0x557098c32cc0 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.458+0000 7f36c67fc700 1 -- 172.21.15.52:0/2097760175 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d009c620 con 0x557098c32cc0 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 -- 172.21.15.52:0/2097760175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 msgr2=0x557098cfbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 0x557098cfbe00 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f36d00cd620 tx=0x7f36d000b9d0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 -- 172.21.15.52:0/2097760175 shutdown_connections 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098d0c8f0 0x557098d2d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098c32cc0 0x557098cfbe00 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 --2- 172.21.15.52:0/2097760175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098cfc340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 -- 172.21.15.52:0/2097760175 >> 172.21.15.52:0/2097760175 conn(0x557098d06f80 msgr2=0x557098d13e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 -- 172.21.15.52:0/2097760175 shutdown_connections 2024-06-02T23:16:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 -- 172.21.15.52:0/2097760175 wait complete. 2024-06-02T23:16:28.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 -- start start 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.459+0000 7f36ed36db80 1 Processor -- start 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098c32cc0 0x557098d1e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d1e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d1c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1abaa0 con 0x557098c32cc0 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ab3d0 con 0x557098d0bfd0 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1a85f0 con 0x557098d0c8f0 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36ed36db80 10 client.0 _ll_register_callbacks cb 0x557098bf3920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098c32cc0 0x557098d1e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d1e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d1e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34786/0 (socket says 172.21.15.52:34786) 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36dbfff700 1 -- 172.21.15.52:0/730809694 learned_addr learned my addr 172.21.15.52:0/730809694 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:28.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e1087700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d1c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 -- 172.21.15.52:0/730809694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 msgr2=0x557098d1c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d1c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 -- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 msgr2=0x557098d1e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d1e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 -- 172.21.15.52:0/730809694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557098d1cf00 con 0x557098c32cc0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36dbfff700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d1e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e1087700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d1c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36e0886700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098c32cc0 0x557098d1e220 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f36d00cd620 tx=0x7f36d00baf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c736e408add607 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.460+0000 7f36bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36bbfff700 1 -- 172.21.15.52:0/730809694 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d009c820 con 0x557098c32cc0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36bbfff700 1 -- 172.21.15.52:0/730809694 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36d00bc0c0 con 0x557098c32cc0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d00a4b10 con 0x557098c32cc0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36bbfff700 1 -- 172.21.15.52:0/730809694 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36d00bc300 con 0x557098c32cc0 2024-06-02T23:16:28.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 10 client.16917 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557098d1cf00 con 0x557098c32cc0 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36bbfff700 1 -- 172.21.15.52:0/730809694 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36d00a4b10 con 0x557098c32cc0 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36bbfff700 1 -- 172.21.15.52:0/730809694 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f36d0005040 con 0x557098c32cc0 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 10 client.16917 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 10 client.16917 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 20 client.16917 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d10c30 0x557098d935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 4 client.16917 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.461+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557098c66d10 con 0x557098d10c30 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36dbfff700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d10c30 0x557098d935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36bbfff700 10 client.16917 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36dbfff700 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d10c30 0x557098d935f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fd8a5fb919157c2 server_cookie=daa425df32d7c1de in_seq=0 out_seq=0 2024-06-02T23:16:28.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36bbfff700 1 -- 172.21.15.52:0/730809694 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557098c66d10 con 0x557098d10c30 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36bbfff700 10 client.16917 handle_command_reply: tid=0 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 client.16917 shutdown 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098c32cc0 msgr2=0x557098d1e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098c32cc0 0x557098d1e220 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f36d00cd620 tx=0x7f36d00baf70 comp rx=0 tx=0).stop 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 shutdown_connections 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557098d10c30 0x557098d935f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557098d0c8f0 0x557098d1c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557098c32cc0 0x557098d1e220 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 --2- 172.21.15.52:0/730809694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557098d0bfd0 0x557098d1e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 >> 172.21.15.52:0/730809694 conn(0x557098d06f80 msgr2=0x557098d13a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.462+0000 7f36bbfff700 0 client.16917 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 shutdown_connections 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36ed36db80 1 -- 172.21.15.52:0/730809694 wait complete. 2024-06-02T23:16:28.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36ed36db80 20 client.16917 trim_cache size 0 max 16384 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: "e9592e3a-5c67-4704-a965-f9b9cadbcfeb": { 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "e9592e3a-5c67-4704-a965-f9b9cadbcfeb", 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:16:28.468 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:16:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36cc0a2c20 msgr2=0x7f36cc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36cc0a2c20 0x7f36cc0a50e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009750 tx=0x7f36c8009310 comp rx=0 tx=0).stop 2024-06-02T23:16:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 msgr2=0x7f36dc19e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.463+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 0x7f36dc19e7b0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f36d000f990 tx=0x7f36d000f9c0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 shutdown_connections 2024-06-02T23:16:28.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36cc0a2c20 0x7f36cc0a50e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc1073f0 0x7f36dc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36dc101380 0x7f36dc19e7b0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 --2- 172.21.15.52:0/1699500880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36dc101d40 0x7f36dc19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 >> 172.21.15.52:0/1699500880 conn(0x7f36dc078300 msgr2=0x7f36dc105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 shutdown_connections 2024-06-02T23:16:28.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:16:28.464+0000 7f36e2aea700 1 -- 172.21.15.52:0/1699500880 wait complete. 2024-06-02T23:16:28.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 82462 1.91 MB/sec execute 97 sec latency 12.826 ms 2024-06-02T23:16:29.431 INFO:tasks.cephfs.filesystem:scrub status for tag:e9592e3a-5c67-4704-a965-f9b9cadbcfeb - {'path': '/', 'tag': 'e9592e3a-5c67-4704-a965-f9b9cadbcfeb', 'options': 'recursive,force'} 2024-06-02T23:16:29.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 83037 1.93 MB/sec execute 98 sec latency 14.858 ms 2024-06-02T23:16:29.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:29 smithi123 ceph-mon[35977]: pgmap v569: 97 pgs: 97 active+clean; 49 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-02T23:16:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:29 smithi174 ceph-mon[35994]: pgmap v569: 97 pgs: 97 active+clean; 49 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-02T23:16:29.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:29 smithi052 ceph-mon[32340]: pgmap v569: 97 pgs: 97 active+clean; 49 MiB data, 747 MiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 161 op/s 2024-06-02T23:16:30.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 83618 1.92 MB/sec execute 99 sec latency 14.912 ms 2024-06-02T23:16:31.626 INFO:tasks.workunit.client.0.smithi052.stdout: 1 84135 1.92 MB/sec execute 100 sec latency 15.278 ms 2024-06-02T23:16:31.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:31 smithi123 ceph-mon[35977]: pgmap v570: 97 pgs: 97 active+clean; 51 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-02T23:16:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:16:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:16:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:16:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:16:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:31 smithi174 ceph-mon[35994]: pgmap v570: 97 pgs: 97 active+clean; 51 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-02T23:16:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:16:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:16:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:16:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:16:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:31 smithi052 ceph-mon[32340]: pgmap v570: 97 pgs: 97 active+clean; 51 MiB data, 770 MiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 167 op/s 2024-06-02T23:16:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:16:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:16:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:16:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:16:32.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 84925 1.97 MB/sec execute 101 sec latency 14.902 ms 2024-06-02T23:16:33.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 85379 1.99 MB/sec execute 102 sec latency 14.689 ms 2024-06-02T23:16:33.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:33 smithi123 ceph-mon[35977]: pgmap v571: 97 pgs: 97 active+clean; 50 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-02T23:16:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:33 smithi174 ceph-mon[35994]: pgmap v571: 97 pgs: 97 active+clean; 50 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-02T23:16:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:33 smithi052 ceph-mon[32340]: pgmap v571: 97 pgs: 97 active+clean; 50 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 163 op/s 2024-06-02T23:16:34.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 85633 1.98 MB/sec execute 103 sec latency 14.378 ms 2024-06-02T23:16:35.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 86050 1.96 MB/sec execute 104 sec latency 13.439 ms 2024-06-02T23:16:35.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:35 smithi123 ceph-mon[35977]: pgmap v572: 97 pgs: 97 active+clean; 50 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-06-02T23:16:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:35 smithi174 ceph-mon[35994]: pgmap v572: 97 pgs: 97 active+clean; 50 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-06-02T23:16:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:35 smithi052 ceph-mon[32340]: pgmap v572: 97 pgs: 97 active+clean; 50 MiB data, 794 MiB used, 1.0 TiB / 1.0 TiB avail; 225 KiB/s rd, 1.0 MiB/s wr, 140 op/s 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 -- 172.21.15.52:0/3527861599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 msgr2=0x7ff668101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 --2- 172.21.15.52:0/3527861599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668101f70 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009900 tx=0x7ff65c02a410 comp rx=0 tx=0).stop 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 -- 172.21.15.52:0/3527861599 shutdown_connections 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 --2- 172.21.15.52:0/3527861599 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6681024b0 0x7ff668108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 --2- 172.21.15.52:0/3527861599 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668101f70 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 --2- 172.21.15.52:0/3527861599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff668100900 0x7ff668100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 -- 172.21.15.52:0/3527861599 >> 172.21.15.52:0/3527861599 conn(0x7ff6680fbe70 msgr2=0x7ff6680fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 -- 172.21.15.52:0/3527861599 shutdown_connections 2024-06-02T23:16:36.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.144+0000 7ff66cc79640 1 -- 172.21.15.52:0/3527861599 wait complete. 2024-06-02T23:16:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 Processor -- start 2024-06-02T23:16:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 -- start start 2024-06-02T23:16:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668100900 0x7ff668197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6681024b0 0x7ff66819f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810c430 con 0x7ff668101af0 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810c2b0 con 0x7ff668100900 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.145+0000 7ff66cc79640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff66810c5b0 con 0x7ff6681024b0 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668198130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49248/0 (socket says 172.21.15.52:49248) 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 -- 172.21.15.52:0/176512325 learned_addr learned my addr 172.21.15.52:0/176512325 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff66659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668100900 0x7ff668197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff666d9d640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6681024b0 0x7ff66819f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 -- 172.21.15.52:0/176512325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6681024b0 msgr2=0x7ff66819f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6681024b0 0x7ff66819f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 -- 172.21.15.52:0/176512325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668100900 msgr2=0x7ff668197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668100900 0x7ff668197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 -- 172.21.15.52:0/176512325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66819f810 con 0x7ff668101af0 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff666d9d640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6681024b0 0x7ff66819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff665d9b640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668198130 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff65c005940 tx=0x7ff65c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff6577fe640 1 -- 172.21.15.52:0/176512325 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c03f030 con 0x7ff668101af0 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff6577fe640 1 -- 172.21.15.52:0/176512325 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff65c003cb0 con 0x7ff668101af0 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66819faa0 con 0x7ff668101af0 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff6577fe640 1 -- 172.21.15.52:0/176512325 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c035070 con 0x7ff668101af0 2024-06-02T23:16:36.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.146+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff66819ff90 con 0x7ff668101af0 2024-06-02T23:16:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.149+0000 7ff6577fe640 1 -- 172.21.15.52:0/176512325 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff65c042030 con 0x7ff668101af0 2024-06-02T23:16:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.149+0000 7ff66f983740 1 Processor -- start 2024-06-02T23:16:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.149+0000 7ff66f983740 1 -- start start 2024-06-02T23:16:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.149+0000 7ff66f983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b469430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.149+0000 7ff66f983740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b46e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66f983740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b471000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66f983740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56180b3cc4f0 con 0x56180b46eb70 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66f983740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56180b36a870 con 0x56180b469a00 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66f983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56180b3c84b0 con 0x56180b469010 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b471000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b46e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b471000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49262/0 (socket says 172.21.15.52:49262) 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff665d9b640 1 -- 172.21.15.52:0/977640938 learned_addr learned my addr 172.21.15.52:0/977640938 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff666d9d640 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b469430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 -- 172.21.15.52:0/977640938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 msgr2=0x56180b469430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff6577fe640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6500a2c30 0x7ff6500a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b469430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 -- 172.21.15.52:0/977640938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 msgr2=0x56180b471000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b471000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 -- 172.21.15.52:0/977640938 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180b3869b0 con 0x56180b469a00 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff666d9d640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6500a2c30 0x7ff6500a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff6577fe640 1 -- 172.21.15.52:0/176512325 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff65c0ef5d0 con 0x7ff668101af0 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.150+0000 7ff66659c640 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b46e500 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff658009bd0 tx=0x7ff658011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e60c7e2b64303789 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff654ff9640 1 -- 172.21.15.52:0/977640938 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658004230 con 0x56180b469a00 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff654ff9640 1 -- 172.21.15.52:0/977640938 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff658009e40 con 0x56180b469a00 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff654ff9640 1 -- 172.21.15.52:0/977640938 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff658019790 con 0x56180b469a00 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 -- 172.21.15.52:0/977640938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 msgr2=0x56180b46e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b46e500 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ff658009bd0 tx=0x7ff658011810 comp rx=0 tx=0).stop 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff666d9d640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6500a2c30 0x7ff6500a50f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x56180b46a260 tx=0x7ff660004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 -- 172.21.15.52:0/977640938 shutdown_connections 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b471000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b46e500 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 --2- 172.21.15.52:0/977640938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b469430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.151+0000 7ff66f983740 1 -- 172.21.15.52:0/977640938 >> 172.21.15.52:0/977640938 conn(0x56180b464680 msgr2=0x56180b464a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 -- 172.21.15.52:0/977640938 shutdown_connections 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 -- 172.21.15.52:0/977640938 wait complete. 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 -- start start 2024-06-02T23:16:36.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 Processor -- start 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469010 0x56180b4732c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b473800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b46eb70 0x56180b47a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56180b471b20 con 0x56180b469010 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56180b355ae0 con 0x56180b469a00 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56180b471ca0 con 0x56180b46eb70 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66f983740 10 client.0 _ll_register_callbacks cb 0x56180b38e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469010 0x56180b4732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b473800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469010 0x56180b4732c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49268/0 (socket says 172.21.15.52:49268) 2024-06-02T23:16:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.152+0000 7ff66659c640 1 -- 172.21.15.52:0/891017972 learned_addr learned my addr 172.21.15.52:0/891017972 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff666d9d640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b46eb70 0x56180b47a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff665d9b640 1 -- 172.21.15.52:0/891017972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b46eb70 msgr2=0x56180b47a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff665d9b640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b46eb70 0x56180b47a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff665d9b640 1 -- 172.21.15.52:0/891017972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469010 msgr2=0x56180b4732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff665d9b640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469010 0x56180b4732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff665d9b640 1 -- 172.21.15.52:0/891017972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180b0e0210 con 0x56180b469a00 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff666d9d640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b46eb70 0x56180b47a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff665d9b640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b473800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0ed150 tx=0x7ff65c009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b7e2fc704ef0f90 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff64dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff64dffb640 1 -- 172.21.15.52:0/891017972 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c049040 con 0x56180b469a00 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56180b3dff10 con 0x56180b469a00 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff64dffb640 1 -- 172.21.15.52:0/891017972 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff65c0b84d0 con 0x56180b469a00 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff64dffb640 1 -- 172.21.15.52:0/891017972 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff65c0bb9a0 con 0x56180b469a00 2024-06-02T23:16:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff66f983740 10 client.26462 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56180b1ac620 con 0x56180b469a00 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.153+0000 7ff64dffb640 1 -- 172.21.15.52:0/891017972 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56180b3dff10 con 0x56180b469a00 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff64dffb640 1 -- 172.21.15.52:0/891017972 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff65c0bc0c0 con 0x56180b469a00 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66f983740 10 client.26462 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66f983740 10 client.26462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66f983740 20 client.26462 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66f983740 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b47f110 0x56180b481540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66f983740 4 client.26462 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56180b3d9c20 con 0x56180b47f110 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66659c640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b47f110 0x56180b481540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff64dffb640 10 client.26462 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.154+0000 7ff66659c640 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b47f110 0x56180b481540 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12b9f0490fb524f6 server_cookie=4a25ff07e07e22f8 in_seq=0 out_seq=0 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff64dffb640 1 -- 172.21.15.52:0/891017972 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56180b3d9c20 con 0x56180b47f110 2024-06-02T23:16:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff64dffb640 10 client.26462 handle_command_reply: tid=0 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 client.26462 shutdown 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 msgr2=0x56180b473800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b473800 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0ed150 tx=0x7ff65c009290 comp rx=0 tx=0).stop 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 shutdown_connections 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b47f110 0x56180b481540 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b46eb70 0x56180b47a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.155+0000 7ff66f983740 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469a00 0x56180b473800 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.156+0000 7ff66f983740 1 --2- 172.21.15.52:0/891017972 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469010 0x56180b4732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.156+0000 7ff64dffb640 0 client.26462 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.156+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 >> 172.21.15.52:0/891017972 conn(0x56180b464680 msgr2=0x56180b4637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.156+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 shutdown_connections 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.156+0000 7ff66f983740 1 -- 172.21.15.52:0/891017972 wait complete. 2024-06-02T23:16:36.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.156+0000 7ff66f983740 20 client.26462 trim_cache size 0 max 16384 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 Processor -- start 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 -- start start 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469010 0x56180b462f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469a00 0x56180b47a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b463490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6681a8be0 con 0x56180b46eb70 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6681aa720 con 0x56180b469010 2024-06-02T23:16:36.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66f983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6681a92b0 con 0x56180b469a00 2024-06-02T23:16:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff666d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b463490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff666d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b463490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49284/0 (socket says 172.21.15.52:49284) 2024-06-02T23:16:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff666d9d640 1 -- 172.21.15.52:0/4277263248 learned_addr learned my addr 172.21.15.52:0/4277263248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:36.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff66659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469010 0x56180b462f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff665d9b640 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469a00 0x56180b47a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff665d9b640 1 -- 172.21.15.52:0/4277263248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469010 msgr2=0x56180b462f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff665d9b640 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469010 0x56180b462f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff665d9b640 1 -- 172.21.15.52:0/4277263248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 msgr2=0x56180b463490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff665d9b640 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b463490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff665d9b640 1 -- 172.21.15.52:0/4277263248 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180b481bc0 con 0x56180b469a00 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.172+0000 7ff666d9d640 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b463490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff665d9b640 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469a00 0x56180b47a800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0b8830 tx=0x7ff65c0ba370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d612b551f434cbc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff64ffff640 1 -- 172.21.15.52:0/4277263248 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c049070 con 0x56180b469a00 2024-06-02T23:16:36.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff64ffff640 1 -- 172.21.15.52:0/4277263248 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff65c0bc560 con 0x56180b469a00 2024-06-02T23:16:36.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff64ffff640 1 -- 172.21.15.52:0/4277263248 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c003e60 con 0x56180b469a00 2024-06-02T23:16:36.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 -- 172.21.15.52:0/4277263248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469a00 msgr2=0x56180b47a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469a00 0x56180b47a800 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0b8830 tx=0x7ff65c0ba370 comp rx=0 tx=0).stop 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 -- 172.21.15.52:0/4277263248 shutdown_connections 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469a00 0x56180b47a800 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b46eb70 0x56180b463490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 --2- 172.21.15.52:0/4277263248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b469010 0x56180b462f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.173+0000 7ff66f983740 1 -- 172.21.15.52:0/4277263248 >> 172.21.15.52:0/4277263248 conn(0x56180b464680 msgr2=0x56180b46ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 -- 172.21.15.52:0/4277263248 shutdown_connections 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 -- 172.21.15.52:0/4277263248 wait complete. 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 -- start start 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 Processor -- start 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b477520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 0x56180b477a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b46eb70 0x56180b4e7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6681aa050 con 0x56180b469a00 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6681a9980 con 0x56180b46eb70 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6681a6bc0 con 0x56180b469010 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66f983740 10 client.0 _ll_register_callbacks cb 0x7ff65c031980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 0x56180b477a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff665d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 0x56180b477a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49288/0 (socket says 172.21.15.52:49288) 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff666d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b46eb70 0x56180b4e7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff665d9b640 1 -- 172.21.15.52:0/990252870 learned_addr learned my addr 172.21.15.52:0/990252870 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.174+0000 7ff66659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b477520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff665d9b640 1 -- 172.21.15.52:0/990252870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 msgr2=0x56180b477520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff665d9b640 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b477520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff665d9b640 1 -- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b46eb70 msgr2=0x56180b4e7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff665d9b640 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b46eb70 0x56180b4e7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff665d9b640 1 -- 172.21.15.52:0/990252870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180b468b70 con 0x56180b469a00 2024-06-02T23:16:36.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff665d9b640 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 0x56180b477a60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0f2e50 tx=0x7ff65c04bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ff350673e7b9d70 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff64dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff64dffb640 1 -- 172.21.15.52:0/990252870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff65c049040 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff65c0bc560 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff64dffb640 1 -- 172.21.15.52:0/990252870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff65c03bdb0 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff64dffb640 1 -- 172.21.15.52:0/990252870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff65c0bb7c0 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff66f983740 10 client.16935 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56180b468b70 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.175+0000 7ff64dffb640 1 -- 172.21.15.52:0/990252870 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff65c0bc560 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff64dffb640 1 -- 172.21.15.52:0/990252870 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff65c0bb2b0 con 0x56180b469a00 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66f983740 10 client.16935 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66f983740 10 client.16935 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66f983740 20 client.16935 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:36.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66f983740 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b4e97d0 0x56180b4ebbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66f983740 4 client.16935 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:16:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56180b3d3f70 con 0x56180b4e97d0 2024-06-02T23:16:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66659c640 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b4e97d0 0x56180b4ebbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff66659c640 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b4e97d0 0x56180b4ebbf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d7abf7555acbfc server_cookie=29bccd342d959b24 in_seq=0 out_seq=0 2024-06-02T23:16:36.182 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.176+0000 7ff64dffb640 10 client.16935 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff64dffb640 1 -- 172.21.15.52:0/990252870 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56180b3d3f70 con 0x56180b4e97d0 2024-06-02T23:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff64dffb640 10 client.16935 handle_command_reply: tid=0 2024-06-02T23:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 client.16935 shutdown 2024-06-02T23:16:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 msgr2=0x56180b477a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 0x56180b477a60 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7ff65c0f2e50 tx=0x7ff65c04bfb0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 shutdown_connections 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56180b4e97d0 0x56180b4ebbf0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56180b46eb70 0x56180b4e7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff64dffb640 0 client.16935 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56180b469a00 0x56180b477a60 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 --2- 172.21.15.52:0/990252870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180b469010 0x56180b477520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 >> 172.21.15.52:0/990252870 conn(0x56180b464680 msgr2=0x56180b467360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 shutdown_connections 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 1 -- 172.21.15.52:0/990252870 wait complete. 2024-06-02T23:16:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.201+0000 7ff66f983740 20 client.16935 trim_cache size 0 max 16384 2024-06-02T23:16:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:16:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:16:36.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6500a2c30 msgr2=0x7ff6500a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6500a2c30 0x7ff6500a50f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x56180b46a260 tx=0x7ff660004a40 comp rx=0 tx=0).stop 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 msgr2=0x7ff668198130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668198130 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7ff65c005940 tx=0x7ff65c0042f0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 shutdown_connections 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6500a2c30 0x7ff6500a50f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6681024b0 0x7ff66819f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff668101af0 0x7ff668198130 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 --2- 172.21.15.52:0/176512325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff668100900 0x7ff668197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 >> 172.21.15.52:0/176512325 conn(0x7ff6680fbe70 msgr2=0x7ff6680fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 shutdown_connections 2024-06-02T23:16:36.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:36.202+0000 7ff66cc79640 1 -- 172.21.15.52:0/176512325 wait complete. 2024-06-02T23:16:36.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 86626 1.99 MB/sec execute 105 sec latency 21.178 ms 2024-06-02T23:16:37.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 87237 1.97 MB/sec execute 106 sec latency 15.239 ms 2024-06-02T23:16:37.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:37 smithi123 ceph-mon[35977]: pgmap v573: 97 pgs: 97 active+clean; 52 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-02T23:16:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:37 smithi174 ceph-mon[35994]: pgmap v573: 97 pgs: 97 active+clean; 52 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-02T23:16:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:37 smithi052 ceph-mon[32340]: pgmap v573: 97 pgs: 97 active+clean; 52 MiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 296 KiB/s rd, 1.4 MiB/s wr, 173 op/s 2024-06-02T23:16:38.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 87768 1.98 MB/sec execute 107 sec latency 18.826 ms 2024-06-02T23:16:39.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 88485 2.02 MB/sec execute 108 sec latency 15.259 ms 2024-06-02T23:16:39.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:39 smithi123 ceph-mon[35977]: pgmap v574: 97 pgs: 97 active+clean; 53 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-02T23:16:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:39 smithi174 ceph-mon[35994]: pgmap v574: 97 pgs: 97 active+clean; 53 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-02T23:16:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:39 smithi052 ceph-mon[32340]: pgmap v574: 97 pgs: 97 active+clean; 53 MiB data, 793 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 1.2 MiB/s wr, 151 op/s 2024-06-02T23:16:40.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 88951 2.04 MB/sec execute 109 sec latency 14.603 ms 2024-06-02T23:16:41.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 89213 2.02 MB/sec execute 110 sec latency 15.267 ms 2024-06-02T23:16:41.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:41 smithi123 ceph-mon[35977]: pgmap v575: 97 pgs: 97 active+clean; 55 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-02T23:16:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:41 smithi174 ceph-mon[35994]: pgmap v575: 97 pgs: 97 active+clean; 55 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-02T23:16:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:41 smithi052 ceph-mon[32340]: pgmap v575: 97 pgs: 97 active+clean; 55 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-02T23:16:42.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 89665 2.01 MB/sec execute 111 sec latency 15.377 ms 2024-06-02T23:16:43.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 90225 2.03 MB/sec execute 112 sec latency 14.137 ms 2024-06-02T23:16:43.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:43 smithi123 ceph-mon[35977]: pgmap v576: 97 pgs: 97 active+clean; 53 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:16:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:43 smithi174 ceph-mon[35994]: pgmap v576: 97 pgs: 97 active+clean; 53 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:16:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:43 smithi052 ceph-mon[32340]: pgmap v576: 97 pgs: 97 active+clean; 53 MiB data, 812 MiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:16:44.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 90808 2.02 MB/sec execute 113 sec latency 15.659 ms 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:16:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:16:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:16:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:16:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:16:44.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:16:44.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:16:44.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:16:44.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:16:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:16:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:16:44.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:16:45.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 91326 2.02 MB/sec execute 114 sec latency 15.702 ms 2024-06-02T23:16:45.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:45 smithi123 ceph-mon[35977]: pgmap v577: 97 pgs: 97 active+clean; 54 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:16:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:45 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:16:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:45 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:16:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:45 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:16:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:45 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:16:45.794 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:45 smithi174 ceph-mon[35994]: pgmap v577: 97 pgs: 97 active+clean; 54 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:16:45.795 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:45 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:16:45.795 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:45 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:16:45.795 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:45 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:16:45.795 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:45 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:16:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:45 smithi052 ceph-mon[32340]: pgmap v577: 97 pgs: 97 active+clean; 54 MiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 223 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:16:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:45 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:16:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:45 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:16:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:45 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:16:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:45 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:16:46.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 92044 2.06 MB/sec execute 115 sec latency 11.367 ms 2024-06-02T23:16:47.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 92521 2.08 MB/sec execute 116 sec latency 14.943 ms 2024-06-02T23:16:47.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:47 smithi123 ceph-mon[35977]: pgmap v578: 97 pgs: 97 active+clean; 55 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-02T23:16:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:47 smithi174 ceph-mon[35994]: pgmap v578: 97 pgs: 97 active+clean; 55 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-02T23:16:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:47 smithi052 ceph-mon[32340]: pgmap v578: 97 pgs: 97 active+clean; 55 MiB data, 821 MiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.5 MiB/s wr, 175 op/s 2024-06-02T23:16:48.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 92771 2.07 MB/sec execute 117 sec latency 15.298 ms 2024-06-02T23:16:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 -- 172.21.15.52:0/3076356852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c101af0 msgr2=0x7f527c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 --2- 172.21.15.52:0/3076356852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c101af0 0x7f527c101f70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5264009860 tx=0x7f526402a340 comp rx=0 tx=0).stop 2024-06-02T23:16:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 -- 172.21.15.52:0/3076356852 shutdown_connections 2024-06-02T23:16:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 --2- 172.21.15.52:0/3076356852 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 --2- 172.21.15.52:0/3076356852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c101af0 0x7f527c101f70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 --2- 172.21.15.52:0/3076356852 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c100900 0x7f527c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 -- 172.21.15.52:0/3076356852 >> 172.21.15.52:0/3076356852 conn(0x7f527c0fbe70 msgr2=0x7f527c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 -- 172.21.15.52:0/3076356852 shutdown_connections 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.304+0000 7f5281d5a640 1 -- 172.21.15.52:0/3076356852 wait complete. 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 Processor -- start 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 -- start start 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c100900 0x7f527c069b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c101af0 0x7f527c19c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f527c10c410 con 0x7f527c1024b0 2024-06-02T23:16:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f527c10c290 con 0x7f527c100900 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5281d5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f527c10c590 con 0x7f527c101af0 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c19ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39630/0 (socket says 172.21.15.52:39630) 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 -- 172.21.15.52:0/65926913 learned_addr learned my addr 172.21.15.52:0/65926913 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c100900 0x7f527c069b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f5273fff640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c101af0 0x7f527c19c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 -- 172.21.15.52:0/65926913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c101af0 msgr2=0x7f527c19c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c101af0 0x7f527c19c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 -- 172.21.15.52:0/65926913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c100900 msgr2=0x7f527c069b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c100900 0x7f527c069b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 -- 172.21.15.52:0/65926913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f527c19f7d0 con 0x7f527c1024b0 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.305+0000 7f527bfff640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c19ef10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f527400efc0 tx=0x7f527400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.306+0000 7f52797fa640 1 -- 172.21.15.52:0/65926913 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5274003970 con 0x7f527c1024b0 2024-06-02T23:16:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.306+0000 7f52797fa640 1 -- 172.21.15.52:0/65926913 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5274004020 con 0x7f527c1024b0 2024-06-02T23:16:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.306+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f527c19fac0 con 0x7f527c1024b0 2024-06-02T23:16:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.306+0000 7f52797fa640 1 -- 172.21.15.52:0/65926913 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527400b940 con 0x7f527c1024b0 2024-06-02T23:16:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.306+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527c19ff90 con 0x7f527c1024b0 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.308+0000 7f52797fa640 1 -- 172.21.15.52:0/65926913 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f527401d020 con 0x7f527c1024b0 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.308+0000 7f5284a64740 1 Processor -- start 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 -- start start 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cbc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cbf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4c1a4f0 con 0x55ddf4cb7010 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4bb8870 con 0x55ddf4cbcb70 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5284a64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4c164b0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39634/0 (socket says 172.21.15.52:39634) 2024-06-02T23:16:49.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 -- 172.21.15.52:0/1594911872 learned_addr learned my addr 172.21.15.52:0/1594911872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f52797fa640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f52680a2c30 0x7f52680a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f527b7fe640 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cbc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cbf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f527bfff640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f52680a2c30 0x7f52680a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 -- 172.21.15.52:0/1594911872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 msgr2=0x55ddf4cbc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cbc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 -- 172.21.15.52:0/1594911872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cbcb70 msgr2=0x55ddf4cbf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cbf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.309+0000 7f5273fff640 1 -- 172.21.15.52:0/1594911872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf4bd49b0 con 0x55ddf4cb7010 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.310+0000 7f52797fa640 1 -- 172.21.15.52:0/65926913 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f52740cef50 con 0x7f527c1024b0 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.310+0000 7f5273fff640 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb7430 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5264005940 tx=0x7f5264005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b64f28693498dc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.310+0000 7f527bfff640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f52680a2c30 0x7f52680a50f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f52740981b0 tx=0x7f527406e8a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:49.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.310+0000 7f5271ffb640 1 -- 172.21.15.52:0/1594911872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264037040 con 0x55ddf4cb7010 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.310+0000 7f5271ffb640 1 -- 172.21.15.52:0/1594911872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52640044d0 con 0x55ddf4cb7010 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.310+0000 7f5271ffb640 1 -- 172.21.15.52:0/1594911872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004920 con 0x55ddf4cb7010 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 -- 172.21.15.52:0/1594911872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 msgr2=0x55ddf4cb7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb7430 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f5264005940 tx=0x7f5264005ab0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 -- 172.21.15.52:0/1594911872 shutdown_connections 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cbc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cbf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 --2- 172.21.15.52:0/1594911872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb7430 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 -- 172.21.15.52:0/1594911872 >> 172.21.15.52:0/1594911872 conn(0x55ddf4cb2680 msgr2=0x55ddf4cb2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 -- 172.21.15.52:0/1594911872 shutdown_connections 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 -- 172.21.15.52:0/1594911872 wait complete. 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 -- start start 2024-06-02T23:16:49.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.311+0000 7f5284a64740 1 Processor -- start 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc1290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cc8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4cbfa60 con 0x55ddf4cbcb70 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4ba3ae0 con 0x55ddf4cb7010 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddf4cbfbe0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5284a64740 10 client.0 _ll_register_callbacks cb 0x55ddf4bdc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cc8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cc8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39636/0 (socket says 172.21.15.52:39636) 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 -- 172.21.15.52:0/3015137804 learned_addr learned my addr 172.21.15.52:0/3015137804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f527b7fe640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 -- 172.21.15.52:0/3015137804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 msgr2=0x55ddf4cc17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 -- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 msgr2=0x55ddf4cc1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc1290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 -- 172.21.15.52:0/3015137804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf492e210 con 0x55ddf4cbcb70 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f5273fff640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cc8810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f52640304b0 tx=0x7f5264005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22cb8ab036d2650 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f525affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.312+0000 7f525affd640 1 -- 172.21.15.52:0/3015137804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264054060 con 0x55ddf4cbcb70 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f525affd640 1 -- 172.21.15.52:0/3015137804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f52640307e0 con 0x55ddf4cbcb70 2024-06-02T23:16:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52640044d0 con 0x55ddf4cbcb70 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f525affd640 1 -- 172.21.15.52:0/3015137804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5264037070 con 0x55ddf4cbcb70 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 10 client.16947 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ddf49fa620 con 0x55ddf4cbcb70 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f525affd640 1 -- 172.21.15.52:0/3015137804 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52640044d0 con 0x55ddf4cbcb70 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f525affd640 1 -- 172.21.15.52:0/3015137804 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f52640325d0 con 0x55ddf4cbcb70 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 10 client.16947 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 10 client.16947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 20 client.16947 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4cccce0 0x55ddf4ccf0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 4 client.16947 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf4c2df10 con 0x55ddf4cccce0 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.313+0000 7f527b7fe640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4cccce0 0x55ddf4ccf0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.314+0000 7f525affd640 10 client.16947 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.314+0000 7f527b7fe640 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4cccce0 0x55ddf4ccf0d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85162d7bd1b1ac41 server_cookie=eb560c178f1348d3 in_seq=0 out_seq=0 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.314+0000 7f525affd640 1 -- 172.21.15.52:0/3015137804 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ddf4c2df10 con 0x55ddf4cccce0 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.314+0000 7f525affd640 10 client.16947 handle_command_reply: tid=0 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 client.16947 shutdown 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 msgr2=0x55ddf4cc8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cc8810 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f52640304b0 tx=0x7f5264005f90 comp rx=0 tx=0).stop 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 shutdown_connections 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4cccce0 0x55ddf4ccf0d0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cc8810 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f525affd640 0 client.16947 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 --2- 172.21.15.52:0/3015137804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 >> 172.21.15.52:0/3015137804 conn(0x55ddf4cb2680 msgr2=0x55ddf4cb17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 shutdown_connections 2024-06-02T23:16:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 1 -- 172.21.15.52:0/3015137804 wait complete. 2024-06-02T23:16:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.315+0000 7f5284a64740 20 client.16947 trim_cache size 0 max 16384 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.331+0000 7f5284a64740 1 Processor -- start 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.331+0000 7f5284a64740 1 -- start start 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5284a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5284a64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5284a64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cb8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5284a64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1a8b90 con 0x55ddf4cbcb70 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5284a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1aa6d0 con 0x55ddf4cb7010 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5284a64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1a9260 con 0x55ddf4cb7a00 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.333 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cb8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cb8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39650/0 (socket says 172.21.15.52:39650) 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 -- 172.21.15.52:0/2270901956 learned_addr learned my addr 172.21.15.52:0/2270901956 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f527b7fe640 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 -- 172.21.15.52:0/2270901956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 msgr2=0x55ddf4cc87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 -- 172.21.15.52:0/2270901956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 msgr2=0x55ddf4cb1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 -- 172.21.15.52:0/2270901956 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf4cb3350 con 0x55ddf4cbcb70 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f5273fff640 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cb8c80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5264009250 tx=0x7f5264005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b45fac8919171dc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.332+0000 7f52717fa640 1 -- 172.21.15.52:0/2270901956 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004e50 con 0x55ddf4cbcb70 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f52717fa640 1 -- 172.21.15.52:0/2270901956 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5264004ff0 con 0x55ddf4cbcb70 2024-06-02T23:16:49.334 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f52717fa640 1 -- 172.21.15.52:0/2270901956 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5264004430 con 0x55ddf4cbcb70 2024-06-02T23:16:49.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 -- 172.21.15.52:0/2270901956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 msgr2=0x55ddf4cb8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cb8c80 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f5264009250 tx=0x7f5264005dc0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 -- 172.21.15.52:0/2270901956 shutdown_connections 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4cb8c80 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 --2- 172.21.15.52:0/2270901956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cb1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 -- 172.21.15.52:0/2270901956 >> 172.21.15.52:0/2270901956 conn(0x55ddf4cb2680 msgr2=0x55ddf4cb2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.333+0000 7f5284a64740 1 -- 172.21.15.52:0/2270901956 shutdown_connections 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 -- 172.21.15.52:0/2270901956 wait complete. 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 -- start start 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 Processor -- start 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc58c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4d35940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1aa000 con 0x55ddf4cbcb70 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1a9930 con 0x55ddf4cb7010 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f527c1a6b70 con 0x55ddf4cb7a00 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.334+0000 7f5284a64740 10 client.0 _ll_register_callbacks cb 0x7f526c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4d35940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56754/0 (socket says 172.21.15.52:56754) 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 -- 172.21.15.52:0/3346287746 learned_addr learned my addr 172.21.15.52:0/3346287746 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f5273fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4d35940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39664/0 (socket says 172.21.15.52:39664) 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f5273fff640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 -- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 msgr2=0x55ddf4cc5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 -- 172.21.15.52:0/3346287746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 msgr2=0x55ddf4d35940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4d35940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 -- 172.21.15.52:0/3346287746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf4cc8990 con 0x55ddf4cb7a00 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f5273fff640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4d35940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f527b7fe640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc58c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55ddf4cb7e00 tx=0x7f526c0206c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1b1cf90c29d1c094 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f525affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f525affd640 1 -- 172.21.15.52:0/3346287746 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f526c010840 con 0x55ddf4cb7a00 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f525affd640 1 -- 172.21.15.52:0/3346287746 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f526c0109e0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.335+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5264004ff0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f525affd640 1 -- 172.21.15.52:0/3346287746 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f526c010be0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 10 client.26335 fetch_fsmap learned FSMap version 32 2024-06-02T23:16:49.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ddf4d36ea0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f525affd640 1 -- 172.21.15.52:0/3346287746 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5264004ff0 con 0x55ddf4cb7a00 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f525affd640 1 -- 172.21.15.52:0/3346287746 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f526c025030 con 0x55ddf4cb7a00 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 10 client.26335 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 10 client.26335 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 20 client.26335 populate_metadata read hostname 'smithi052' 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4d378d0 0x55ddf4d39cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 4 client.26335 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ddf4c27c20 con 0x55ddf4d378d0 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.336+0000 7f5273fff640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4d378d0 0x55ddf4d39cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.337+0000 7f525affd640 10 client.26335 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:49.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.337+0000 7f5273fff640 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4d378d0 0x55ddf4d39cf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dbeb847316e3b55 server_cookie=86a0fc77cbbc76ae in_seq=0 out_seq=0 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f525affd640 1 -- 172.21.15.52:0/3346287746 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ddf4c27c20 con 0x55ddf4d378d0 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f525affd640 10 client.26335 handle_command_reply: tid=0 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 client.26335 shutdown 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 msgr2=0x55ddf4cc58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc58c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55ddf4cb7e00 tx=0x7f526c0206c0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 shutdown_connections 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ddf4d378d0 0x55ddf4d39cf0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf4cb7a00 0x55ddf4cc58c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ddf4cbcb70 0x55ddf4d35940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f525affd640 0 client.26335 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 --2- 172.21.15.52:0/3346287746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ddf4cb7010 0x55ddf4cc5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 >> 172.21.15.52:0/3346287746 conn(0x55ddf4cb2680 msgr2=0x55ddf4cb8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 shutdown_connections 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 1 -- 172.21.15.52:0/3346287746 wait complete. 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.355+0000 7f5284a64740 20 client.26335 trim_cache size 0 max 16384 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:16:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:16:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f52680a2c30 msgr2=0x7f52680a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f52680a2c30 0x7f52680a50f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f52740981b0 tx=0x7f527406e8a0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 msgr2=0x7f527c19ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:16:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c19ef10 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f527400efc0 tx=0x7f527400c800 comp rx=0 tx=0).stop 2024-06-02T23:16:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 shutdown_connections 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f52680a2c30 0x7f52680a50f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f527c101af0 0x7f527c19c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f527c1024b0 0x7f527c19ef10 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 --2- 172.21.15.52:0/65926913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f527c100900 0x7f527c069b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 >> 172.21.15.52:0/65926913 conn(0x7f527c0fbe70 msgr2=0x7f527c104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 shutdown_connections 2024-06-02T23:16:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:16:49.356+0000 7f5281d5a640 1 -- 172.21.15.52:0/65926913 wait complete. 2024-06-02T23:16:49.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 93211 2.06 MB/sec execute 118 sec latency 14.968 ms 2024-06-02T23:16:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:49 smithi123 ceph-mon[35977]: pgmap v579: 97 pgs: 97 active+clean; 55 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-02T23:16:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:49 smithi174 ceph-mon[35994]: pgmap v579: 97 pgs: 97 active+clean; 55 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-02T23:16:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:49 smithi052 ceph-mon[32340]: pgmap v579: 97 pgs: 97 active+clean; 55 MiB data, 806 MiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-02T23:16:50.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 93788 2.07 MB/sec execute 119 sec latency 14.738 ms 2024-06-02T23:16:51.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 94433 2.06 MB/sec execute 120 sec latency 15.008 ms 2024-06-02T23:16:51.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:51 smithi123 ceph-mon[35977]: pgmap v580: 97 pgs: 97 active+clean; 56 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-02T23:16:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:51 smithi174 ceph-mon[35994]: pgmap v580: 97 pgs: 97 active+clean; 56 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-02T23:16:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:51 smithi052 ceph-mon[32340]: pgmap v580: 97 pgs: 97 active+clean; 56 MiB data, 807 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 170 op/s 2024-06-02T23:16:52.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 95028 2.09 MB/sec execute 121 sec latency 11.495 ms 2024-06-02T23:16:53.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 95602 2.10 MB/sec execute 122 sec latency 15.836 ms 2024-06-02T23:16:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:53 smithi123 ceph-mon[35977]: pgmap v581: 97 pgs: 97 active+clean; 51 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-06-02T23:16:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:53 smithi174 ceph-mon[35994]: pgmap v581: 97 pgs: 97 active+clean; 51 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-06-02T23:16:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:53 smithi052 ceph-mon[32340]: pgmap v581: 97 pgs: 97 active+clean; 51 MiB data, 801 MiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 142 op/s 2024-06-02T23:16:54.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 96090 2.12 MB/sec execute 123 sec latency 14.700 ms 2024-06-02T23:16:55.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 96372 2.10 MB/sec execute 124 sec latency 12.182 ms 2024-06-02T23:16:55.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:55 smithi123 ceph-mon[35977]: pgmap v582: 97 pgs: 97 active+clean; 51 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-02T23:16:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:55 smithi174 ceph-mon[35994]: pgmap v582: 97 pgs: 97 active+clean; 51 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-02T23:16:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:55 smithi052 ceph-mon[32340]: pgmap v582: 97 pgs: 97 active+clean; 51 MiB data, 810 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.3 MiB/s wr, 155 op/s 2024-06-02T23:16:56.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 96872 2.10 MB/sec execute 125 sec latency 10.421 ms 2024-06-02T23:16:57.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 97428 2.11 MB/sec execute 126 sec latency 15.168 ms 2024-06-02T23:16:57.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:57 smithi123 ceph-mon[35977]: pgmap v583: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-06-02T23:16:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:57 smithi174 ceph-mon[35994]: pgmap v583: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-06-02T23:16:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:57 smithi052 ceph-mon[32340]: pgmap v583: 97 pgs: 97 active+clean; 53 MiB data, 834 MiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.6 MiB/s wr, 172 op/s 2024-06-02T23:16:58.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 98113 2.11 MB/sec execute 127 sec latency 15.806 ms 2024-06-02T23:16:59.433 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:16:59.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 98713 2.13 MB/sec execute 128 sec latency 11.919 ms 2024-06-02T23:16:59.708 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:16:59.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:16:59 smithi123 ceph-mon[35977]: pgmap v584: 97 pgs: 97 active+clean; 54 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:16:59.833 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:16:59 smithi052 ceph-mon[32340]: pgmap v584: 97 pgs: 97 active+clean; 54 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:16:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:16:59 smithi174 ceph-mon[35994]: pgmap v584: 97 pgs: 97 active+clean; 54 MiB data, 853 MiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:17:00.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 99180 2.13 MB/sec execute 129 sec latency 14.705 ms 2024-06-02T23:17:01.539 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:01 smithi052 ceph-mon[32340]: pgmap v585: 97 pgs: 97 active+clean; 53 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:17:01.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 99648 2.15 MB/sec execute 130 sec latency 65.534 ms 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 -- 172.21.15.52:0/647533618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 msgr2=0x7fe01810f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 --2- 172.21.15.52:0/647533618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 0x7fe01810f3f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe01000b0d0 tx=0x7fe01000b3e0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 -- 172.21.15.52:0/647533618 shutdown_connections 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 --2- 172.21.15.52:0/647533618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 0x7fe01810f3f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 --2- 172.21.15.52:0/647533618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe018102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 --2- 172.21.15.52:0/647533618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018103150 0x7fe018107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 -- 172.21.15.52:0/647533618 >> 172.21.15.52:0/647533618 conn(0x7fe018079570 msgr2=0x7fe018079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 -- 172.21.15.52:0/647533618 shutdown_connections 2024-06-02T23:17:01.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.663+0000 7fe02027e700 1 -- 172.21.15.52:0/647533618 wait complete. 2024-06-02T23:17:01.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.664+0000 7fe02027e700 1 Processor -- start 2024-06-02T23:17:01.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.664+0000 7fe02027e700 1 -- start start 2024-06-02T23:17:01.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.664+0000 7fe02027e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe01819e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.664+0000 7fe02027e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018103150 0x7fe01819ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.664+0000 7fe02027e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 0x7fe0181988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe01819e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e81b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 0x7fe0181988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe02027e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe018112660 con 0x7fe018107870 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe01819e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58494/0 (socket says 172.21.15.52:58494) 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 -- 172.21.15.52:0/2803325683 learned_addr learned my addr 172.21.15.52:0/2803325683 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181124e0 con 0x7fe0181027a0 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181127e0 con 0x7fe018103150 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01d819700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018103150 0x7fe01819ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 -- 172.21.15.52:0/2803325683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018103150 msgr2=0x7fe01819ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018103150 0x7fe01819ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 -- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 msgr2=0x7fe0181988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 0x7fe0181988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 -- 172.21.15.52:0/2803325683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe018199020 con 0x7fe0181027a0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe01e01a700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe01819e6b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe01400d980 tx=0x7fe01400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe00f7fe700 1 -- 172.21.15.52:0/2803325683 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014016070 con 0x7fe0181027a0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe018199310 con 0x7fe0181027a0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe00f7fe700 1 -- 172.21.15.52:0/2803325683 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe014004830 con 0x7fe0181027a0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe00f7fe700 1 -- 172.21.15.52:0/2803325683 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014005530 con 0x7fe0181027a0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.665+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0181a5580 con 0x7fe0181027a0 2024-06-02T23:17:01.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.668+0000 7fe00f7fe700 1 -- 172.21.15.52:0/2803325683 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe0140049e0 con 0x7fe0181027a0 2024-06-02T23:17:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.669+0000 7fe00f7fe700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe0040a2c20 0x7fe0040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.669+0000 7fe01d819700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe0040a2c20 0x7fe0040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.669+0000 7fe00f7fe700 1 -- 172.21.15.52:0/2803325683 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe0140ced90 con 0x7fe0181027a0 2024-06-02T23:17:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.669+0000 7fe02ab01b80 1 Processor -- start 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.669+0000 7fe01d819700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe0040a2c20 0x7fe0040a50e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe018108c90 tx=0x7fe008009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.669+0000 7fe02ab01b80 1 -- start start 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 0x5561aa0f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0edf20 0x5561aa0ee340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5561aa03f810 con 0x5561aa0edf20 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5561a9fe7720 con 0x5561aa0f3f60 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561aa03b7d0 con 0x5561aa0f3640 2024-06-02T23:17:01.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 0x5561aa0f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01d819700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e81b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0edf20 0x5561aa0ee340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 0x5561aa0f3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47362/0 (socket says 172.21.15.52:47362) 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 -- 172.21.15.52:0/142479799 learned_addr learned my addr 172.21.15.52:0/142479799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01d819700 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0ed9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58498/0 (socket says 172.21.15.52:58498) 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 -- 172.21.15.52:0/142479799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 msgr2=0x5561aa0ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 -- 172.21.15.52:0/142479799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0edf20 msgr2=0x5561aa0ee340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0edf20 0x5561aa0ee340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 -- 172.21.15.52:0/142479799 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561a9fddf50 con 0x5561aa0f3640 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.670+0000 7fe01e01a700 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 0x5561aa0f3a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe0140a3d60 tx=0x7fe014004dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=546744da3e1713d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.671+0000 7fe00cd70700 1 -- 172.21.15.52:0/142479799 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014022350 con 0x5561aa0f3640 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.671+0000 7fe00cd70700 1 -- 172.21.15.52:0/142479799 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe01409bd20 con 0x5561aa0f3640 2024-06-02T23:17:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.671+0000 7fe00cd70700 1 -- 172.21.15.52:0/142479799 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0140a4060 con 0x5561aa0f3640 2024-06-02T23:17:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.671+0000 7fe02ab01b80 1 -- 172.21.15.52:0/142479799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 msgr2=0x5561aa0f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.671+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 0x5561aa0f3a20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fe0140a3d60 tx=0x7fe014004dd0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- 172.21.15.52:0/142479799 shutdown_connections 2024-06-02T23:17:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0edf20 0x5561aa0ee340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0f3640 0x5561aa0f3a20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/142479799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- 172.21.15.52:0/142479799 >> 172.21.15.52:0/142479799 conn(0x5561aa018cc0 msgr2=0x5561aa0e8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- 172.21.15.52:0/142479799 shutdown_connections 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- 172.21.15.52:0/142479799 wait complete. 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- start start 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 Processor -- start 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0f9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5561aa0fa970 con 0x5561aa0f3640 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5561a9fdfbf0 con 0x5561aa0f3f60 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.672+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5561aa0faaf0 con 0x5561aa0edf20 2024-06-02T23:17:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe02ab01b80 10 client.0 _ll_register_callbacks cb 0x5561a9fd9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0f6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34036/0 (socket says 172.21.15.52:34036) 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 -- 172.21.15.52:0/1792583367 learned_addr learned my addr 172.21.15.52:0/1792583367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01e81b700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01e01a700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0f9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 -- 172.21.15.52:0/1792583367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 msgr2=0x5561aa0f9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0f9cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 -- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 msgr2=0x5561aa0f7510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0f7510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 -- 172.21.15.52:0/1792583367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561a9f8e8d0 con 0x5561aa0f3640 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe01d819700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0f6fd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe00801f8e0 tx=0x7fe0080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c85a8c28400ab68 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe001ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe001ffb700 1 -- 172.21.15.52:0/1792583367 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008004320 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe001ffb700 1 -- 172.21.15.52:0/1792583367 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe0080044c0 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.673+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe01409bd20 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe001ffb700 1 -- 172.21.15.52:0/1792583367 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe008003680 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 10 client.16965 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5561aa1052d0 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe001ffb700 1 -- 172.21.15.52:0/1792583367 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe01409bd20 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe001ffb700 1 -- 172.21.15.52:0/1792583367 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe008026070 con 0x5561aa0f3640 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 20 client.16965 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa1078b0 0x5561aa109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 4 client.16965 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.674+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561aa04cd10 con 0x5561aa1078b0 2024-06-02T23:17:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.675+0000 7fe01e01a700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa1078b0 0x5561aa109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.675+0000 7fe01e01a700 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa1078b0 0x5561aa109cd0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da12b6849cd241a0 server_cookie=9bf9594daecc7b9 in_seq=0 out_seq=0 2024-06-02T23:17:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.675+0000 7fe001ffb700 10 client.16965 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe001ffb700 1 -- 172.21.15.52:0/1792583367 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5561aa04cd10 con 0x5561aa1078b0 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe001ffb700 10 client.16965 handle_command_reply: tid=0 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 client.16965 shutdown 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 msgr2=0x5561aa0f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0f6fd0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fe00801f8e0 tx=0x7fe0080047c0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 shutdown_connections 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa1078b0 0x5561aa109cd0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0f7510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0f9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/1792583367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0f6fd0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 >> 172.21.15.52:0/1792583367 conn(0x5561aa018cc0 msgr2=0x5561aa0e85f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe001ffb700 0 client.16965 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 shutdown_connections 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 1 -- 172.21.15.52:0/1792583367 wait complete. 2024-06-02T23:17:01.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.676+0000 7fe02ab01b80 20 client.16965 trim_cache size 0 max 16384 2024-06-02T23:17:01.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 Processor -- start 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 -- start start 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0e2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa10f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181aa820 con 0x5561aa0f3640 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181ac360 con 0x5561aa0f3f60 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.697+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181aaef0 con 0x5561aa0edf20 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0e2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0e2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34050/0 (socket says 172.21.15.52:34050) 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 -- 172.21.15.52:0/107074751 learned_addr learned my addr 172.21.15.52:0/107074751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:01.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01e81b700 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa10f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 -- 172.21.15.52:0/107074751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 msgr2=0x5561aa0ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 -- 172.21.15.52:0/107074751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 msgr2=0x5561aa10f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa10f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 -- 172.21.15.52:0/107074751 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561aa0e9470 con 0x5561aa0f3640 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01e01a700 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe01d819700 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0e2510 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe008011c90 tx=0x7fe008009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558d3a0bff8849d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe003fff700 1 -- 172.21.15.52:0/107074751 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe008004320 con 0x5561aa0f3640 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe003fff700 1 -- 172.21.15.52:0/107074751 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe008037070 con 0x5561aa0f3640 2024-06-02T23:17:01.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.698+0000 7fe003fff700 1 -- 172.21.15.52:0/107074751 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0080334e0 con 0x5561aa0f3640 2024-06-02T23:17:01.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 -- 172.21.15.52:0/107074751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 msgr2=0x5561aa0e2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0e2510 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fe008011c90 tx=0x7fe008009350 comp rx=0 tx=0).stop 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 -- 172.21.15.52:0/107074751 shutdown_connections 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa10f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/107074751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa0e2510 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 -- 172.21.15.52:0/107074751 >> 172.21.15.52:0/107074751 conn(0x5561aa018cc0 msgr2=0x5561aa0e1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 -- 172.21.15.52:0/107074751 shutdown_connections 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.699+0000 7fe02ab01b80 1 -- 172.21.15.52:0/107074751 wait complete. 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 -- start start 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 Processor -- start 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa1002f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181abc90 con 0x5561aa0f3640 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181ab5c0 con 0x5561aa0f3f60 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe0181a87c0 con 0x5561aa0edf20 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe02ab01b80 10 client.0 _ll_register_callbacks cb 0x5561a9fd9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ffdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47394/0 (socket says 172.21.15.52:47394) 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 -- 172.21.15.52:0/2469248535 learned_addr learned my addr 172.21.15.52:0/2469248535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01d819700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa1002f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e81b700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 -- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 msgr2=0x5561aa0fcd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0fcd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 -- 172.21.15.52:0/2469248535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 msgr2=0x5561aa1002f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:01.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa1002f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.700+0000 7fe01e01a700 1 -- 172.21.15.52:0/2469248535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561aa1009a0 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe01e01a700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ffdb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe014097a80 tx=0x7fe0140ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3789fecdab47cdd1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe001ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe001ffb700 1 -- 172.21.15.52:0/2469248535 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe014064070 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe001ffb700 1 -- 172.21.15.52:0/2469248535 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe0140056d0 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe008037070 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe001ffb700 1 -- 172.21.15.52:0/2469248535 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe0140978c0 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe02ab01b80 10 client.26353 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5561aa172b70 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.701+0000 7fe001ffb700 1 -- 172.21.15.52:0/2469248535 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe008037070 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe001ffb700 1 -- 172.21.15.52:0/2469248535 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe0140a3e00 con 0x5561aa0edf20 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 10 client.26353 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 10 client.26353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:01.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 20 client.26353 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa172e60 0x5561aa175250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 4 client.26353 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561aa047290 con 0x5561aa172e60 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe01d819700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa172e60 0x5561aa175250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe01d819700 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa172e60 0x5561aa175250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b35199fc97657d9d server_cookie=ac000a7ebd6481af in_seq=0 out_seq=0 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe001ffb700 10 client.26353 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe001ffb700 1 -- 172.21.15.52:0/2469248535 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5561aa047290 con 0x5561aa172e60 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe001ffb700 10 client.26353 handle_command_reply: tid=0 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 client.26353 shutdown 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 msgr2=0x5561aa0ffdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ffdb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe014097a80 tx=0x7fe0140ce040 comp rx=0 tx=0).stop 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 shutdown_connections 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5561aa172e60 0x5561aa175250 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561aa0f3f60 0x5561aa0fcd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5561aa0edf20 0x5561aa0ffdb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe02ab01b80 1 --2- 172.21.15.52:0/2469248535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5561aa0f3640 0x5561aa1002f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.702+0000 7fe001ffb700 0 client.26353 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 >> 172.21.15.52:0/2469248535 conn(0x5561aa018cc0 msgr2=0x5561aa0f58e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 shutdown_connections 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02ab01b80 1 -- 172.21.15.52:0/2469248535 wait complete. 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02ab01b80 20 client.26353 trim_cache size 0 max 16384 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:17:01.707 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe0040a2c20 msgr2=0x7fe0040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02027e700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe0040a2c20 0x7fe0040a50e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe018108c90 tx=0x7fe008009260 comp rx=0 tx=0).stop 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 msgr2=0x7fe01819e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.703+0000 7fe02027e700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe01819e6b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe01400d980 tx=0x7fe01400dc90 comp rx=0 tx=0).stop 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 shutdown_connections 2024-06-02T23:17:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe0040a2c20 0x7fe0040a50e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe018107870 0x7fe0181988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe0181027a0 0x7fe01819e6b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 --2- 172.21.15.52:0/2803325683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe018103150 0x7fe01819ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 >> 172.21.15.52:0/2803325683 conn(0x7fe018079570 msgr2=0x7fe0181004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 shutdown_connections 2024-06-02T23:17:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:01.704+0000 7fe02027e700 1 -- 172.21.15.52:0/2803325683 wait complete. 2024-06-02T23:17:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:01 smithi174 ceph-mon[35994]: pgmap v585: 97 pgs: 97 active+clean; 53 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:17:02.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:01 smithi123 ceph-mon[35977]: pgmap v585: 97 pgs: 97 active+clean; 53 MiB data, 880 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:17:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 -- 172.21.15.52:0/1573282004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c100e10 msgr2=0x7fd38c101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 --2- 172.21.15.52:0/1573282004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c100e10 0x7fd38c101230 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fd380009900 tx=0x7fd38002a410 comp rx=0 tx=0).stop 2024-06-02T23:17:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 -- 172.21.15.52:0/1573282004 shutdown_connections 2024-06-02T23:17:02.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 --2- 172.21.15.52:0/1573282004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c101b20 0x7fd38c101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 --2- 172.21.15.52:0/1573282004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38c1024e0 0x7fd38c108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 --2- 172.21.15.52:0/1573282004 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c100e10 0x7fd38c101230 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 -- 172.21.15.52:0/1573282004 >> 172.21.15.52:0/1573282004 conn(0x7fd38c0fb600 msgr2=0x7fd38c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 -- 172.21.15.52:0/1573282004 shutdown_connections 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 -- 172.21.15.52:0/1573282004 wait complete. 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.461+0000 7fd393110640 1 Processor -- start 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 -- start start 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c100e10 0x7fd38c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38c101b20 0x7fd38c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 0x7fd38c19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c10c480 con 0x7fd38c1024e0 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c10c300 con 0x7fd38c101b20 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd393110640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c10c600 con 0x7fd38c100e10 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 0x7fd38c19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 0x7fd38c19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34072/0 (socket says 172.21.15.52:34072) 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 -- 172.21.15.52:0/4203899052 learned_addr learned my addr 172.21.15.52:0/4203899052 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd390eac640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c100e10 0x7fd38c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd38bfff640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38c101b20 0x7fd38c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 -- 172.21.15.52:0/4203899052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c100e10 msgr2=0x7fd38c100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c100e10 0x7fd38c100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 -- 172.21.15.52:0/4203899052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38c101b20 msgr2=0x7fd38c19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38c101b20 0x7fd38c19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd3916ad640 1 -- 172.21.15.52:0/4203899052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38c19f810 con 0x7fd38c1024e0 2024-06-02T23:17:02.464 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.462+0000 7fd390eac640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c100e10 0x7fd38c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.463+0000 7fd3916ad640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 0x7fd38c19ef50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd38400eaa0 tx=0x7fd38400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.463+0000 7fd389ffb640 1 -- 172.21.15.52:0/4203899052 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38400cdb0 con 0x7fd38c1024e0 2024-06-02T23:17:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.463+0000 7fd389ffb640 1 -- 172.21.15.52:0/4203899052 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd384005020 con 0x7fd38c1024e0 2024-06-02T23:17:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.463+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38c196710 con 0x7fd38c1024e0 2024-06-02T23:17:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.463+0000 7fd389ffb640 1 -- 172.21.15.52:0/4203899052 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3840116d0 con 0x7fd38c1024e0 2024-06-02T23:17:02.465 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.463+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd38c196ce0 con 0x7fd38c1024e0 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 Processor -- start 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd389ffb640 1 -- 172.21.15.52:0/4203899052 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3840051d0 con 0x7fd38c1024e0 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 -- start start 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af50430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af55500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af58000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56357aeb34f0 con 0x56357af50010 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56357ae51870 con 0x56357af55b70 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.466+0000 7fd395e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56357aeaf4b0 con 0x56357af50a00 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af50430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af50430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34084/0 (socket says 172.21.15.52:34084) 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 -- 172.21.15.52:0/4150582223 learned_addr learned my addr 172.21.15.52:0/4150582223 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd390eac640 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af58000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 -- 172.21.15.52:0/4150582223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 msgr2=0x56357af55500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af55500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 -- 172.21.15.52:0/4150582223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 msgr2=0x56357af58000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af58000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 -- 172.21.15.52:0/4150582223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357ae6d9b0 con 0x56357af50010 2024-06-02T23:17:02.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd390eac640 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd389ffb640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2cb0 0x7fd37c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd38bfff640 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af50430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd37800d980 tx=0x7fd37800d6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb6d17030089e4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd390eac640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2cb0 0x7fd37c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.467+0000 7fd389ffb640 1 -- 172.21.15.52:0/4203899052 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd3840cf100 con 0x7fd38c1024e0 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.468+0000 7fd376ffd640 1 -- 172.21.15.52:0/4150582223 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3780147f0 con 0x56357af50010 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.468+0000 7fd376ffd640 1 -- 172.21.15.52:0/4150582223 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd378014e40 con 0x56357af50010 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.468+0000 7fd376ffd640 1 -- 172.21.15.52:0/4150582223 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801dcd0 con 0x56357af50010 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.468+0000 7fd395e1d740 1 -- 172.21.15.52:0/4150582223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 msgr2=0x56357af50430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.468+0000 7fd390eac640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2cb0 0x7fd37c0a5170 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd380004770 tx=0x7fd380005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.468+0000 7fd395e1d740 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af50430 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd37800d980 tx=0x7fd37800d6d0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 -- 172.21.15.52:0/4150582223 shutdown_connections 2024-06-02T23:17:02.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af55500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af58000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 --2- 172.21.15.52:0/4150582223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af50430 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 -- 172.21.15.52:0/4150582223 >> 172.21.15.52:0/4150582223 conn(0x56357af4b680 msgr2=0x56357af4ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 -- 172.21.15.52:0/4150582223 shutdown_connections 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 -- 172.21.15.52:0/4150582223 wait complete. 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 -- start start 2024-06-02T23:17:02.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 Processor -- start 2024-06-02T23:17:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.469+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af61800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd395e1d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56357af58b20 con 0x56357af50010 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd395e1d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56357ae3cae0 con 0x56357af55b70 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd395e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56357af58ca0 con 0x56357af50a00 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd395e1d740 10 client.0 _ll_register_callbacks cb 0x56357ae759b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd390eac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af61800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd390eac640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34092/0 (socket says 172.21.15.52:34092) 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd390eac640 1 -- 172.21.15.52:0/1851237592 learned_addr learned my addr 172.21.15.52:0/1851237592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd38bfff640 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 -- 172.21.15.52:0/1851237592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 msgr2=0x56357af5a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af5a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 -- 172.21.15.52:0/1851237592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 msgr2=0x56357af5a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 -- 172.21.15.52:0/1851237592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357abc7210 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd3916ad640 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af61800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd3840ccfa0 tx=0x7fd38409fd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6634878aec8d0043 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd36bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.470+0000 7fd36bfff640 1 -- 172.21.15.52:0/1851237592 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38401f040 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd36bfff640 1 -- 172.21.15.52:0/1851237592 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd38409ae90 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd378014e40 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd36bfff640 1 -- 172.21.15.52:0/1851237592 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd38409b090 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 10 client.26516 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56357ac93620 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd36bfff640 1 -- 172.21.15.52:0/1851237592 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd378014e40 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd36bfff640 1 -- 172.21.15.52:0/1851237592 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd38400f040 con 0x56357af55b70 2024-06-02T23:17:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 10 client.26516 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 10 client.26516 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 20 client.26516 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357af660f0 0x56357af68500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 4 client.26516 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.471+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56357aec6f10 con 0x56357af660f0 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.472+0000 7fd390eac640 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357af660f0 0x56357af68500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.472+0000 7fd36bfff640 10 client.26516 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.472+0000 7fd390eac640 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357af660f0 0x56357af68500 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=705b59bab9541ae7 server_cookie=2825f3f486cf4ea8 in_seq=0 out_seq=0 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd36bfff640 1 -- 172.21.15.52:0/1851237592 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56357aec6f10 con 0x56357af660f0 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd36bfff640 10 client.26516 handle_command_reply: tid=0 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 client.26516 shutdown 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 msgr2=0x56357af61800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af61800 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd3840ccfa0 tx=0x7fd38409fd40 comp rx=0 tx=0).stop 2024-06-02T23:17:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 shutdown_connections 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357af660f0 0x56357af68500 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50a00 0x56357af5a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af61800 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd36bfff640 0 client.26516 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 --2- 172.21.15.52:0/1851237592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 >> 172.21.15.52:0/1851237592 conn(0x56357af4b680 msgr2=0x56357af4a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 shutdown_connections 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 1 -- 172.21.15.52:0/1851237592 wait complete. 2024-06-02T23:17:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.473+0000 7fd395e1d740 20 client.26516 trim_cache size 0 max 16384 2024-06-02T23:17:02.482 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:17:02.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:17:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 Processor -- start 2024-06-02T23:17:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 -- start start 2024-06-02T23:17:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af617c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af5a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1a6bb0 con 0x56357af50a00 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1a86f0 con 0x56357af55b70 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd395e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1a7280 con 0x56357af50010 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af617c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd390eac640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af617c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34108/0 (socket says 172.21.15.52:34108) 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.494+0000 7fd38bfff640 1 -- 172.21.15.52:0/3402135847 learned_addr learned my addr 172.21.15.52:0/3402135847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd3916ad640 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af5a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd38bfff640 1 -- 172.21.15.52:0/3402135847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50010 msgr2=0x56357af5a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd38bfff640 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd38bfff640 1 -- 172.21.15.52:0/3402135847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 msgr2=0x56357af5a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd38bfff640 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af5a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd38bfff640 1 -- 172.21.15.52:0/3402135847 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56357af54fa0 con 0x56357af50a00 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd38bfff640 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af617c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e7b0 tx=0x7fd37800d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8f500d2d672c0c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd3767fc640 1 -- 172.21.15.52:0/3402135847 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd37801d5b0 con 0x56357af50a00 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd3767fc640 1 -- 172.21.15.52:0/3402135847 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37801d920 con 0x56357af50a00 2024-06-02T23:17:02.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.495+0000 7fd3767fc640 1 -- 172.21.15.52:0/3402135847 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd378014a10 con 0x56357af50a00 2024-06-02T23:17:02.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 -- 172.21.15.52:0/3402135847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 msgr2=0x56357af617c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af617c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd37800e7b0 tx=0x7fd37800d500 comp rx=0 tx=0).stop 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 -- 172.21.15.52:0/3402135847 shutdown_connections 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af55b70 0x56357af5a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af617c0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3402135847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af50010 0x56357af5a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 -- 172.21.15.52:0/3402135847 >> 172.21.15.52:0/3402135847 conn(0x56357af4b680 msgr2=0x56357af4ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 -- 172.21.15.52:0/3402135847 shutdown_connections 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 -- 172.21.15.52:0/3402135847 wait complete. 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.496+0000 7fd395e1d740 1 -- start start 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 Processor -- start 2024-06-02T23:17:02.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af50010 0x56357af616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af5e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af55b70 0x56357af5e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1a8020 con 0x56357af50a00 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1a7950 con 0x56357af50010 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd38c1a4b90 con 0x56357af55b70 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd395e1d740 10 client.0 _ll_register_callbacks cb 0x7fd384097ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd390eac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af50010 0x56357af616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af5e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd390eac640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af50010 0x56357af616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58588/0 (socket says 172.21.15.52:58588) 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd38bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af5e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34122/0 (socket says 172.21.15.52:34122) 2024-06-02T23:17:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd390eac640 1 -- 172.21.15.52:0/3820082843 learned_addr learned my addr 172.21.15.52:0/3820082843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.497+0000 7fd3916ad640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af55b70 0x56357af5e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd3916ad640 1 -- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af50010 msgr2=0x56357af616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd3916ad640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af50010 0x56357af616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd3916ad640 1 -- 172.21.15.52:0/3820082843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 msgr2=0x56357af5e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd3916ad640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af5e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd3916ad640 1 -- 172.21.15.52:0/3820082843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56357afd7580 con 0x56357af55b70 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd38bfff640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af5e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd3916ad640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af55b70 0x56357af5e690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd38409b9d0 tx=0x7fd3840a4670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d22a5c53ed811459 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd36bfff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd36bfff640 1 -- 172.21.15.52:0/3820082843 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38407e020 con 0x56357af55b70 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd36bfff640 1 -- 172.21.15.52:0/3820082843 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd3840979a0 con 0x56357af55b70 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd37801d920 con 0x56357af55b70 2024-06-02T23:17:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd36bfff640 1 -- 172.21.15.52:0/3820082843 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd38401f070 con 0x56357af55b70 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd395e1d740 10 client.26368 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.498+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56357afd88a0 con 0x56357af55b70 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd36bfff640 1 -- 172.21.15.52:0/3820082843 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd37801d920 con 0x56357af55b70 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd36bfff640 1 -- 172.21.15.52:0/3820082843 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd3840a0ed0 con 0x56357af55b70 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd395e1d740 10 client.26368 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd395e1d740 10 client.26368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd395e1d740 20 client.26368 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357afd9270 0x56357afdb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd395e1d740 4 client.26368 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56357aec0c20 con 0x56357afd9270 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd390eac640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357afd9270 0x56357afdb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd36bfff640 10 client.26368 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.499+0000 7fd390eac640 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357afd9270 0x56357afdb690 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974ab64f76e5c836 server_cookie=dae95084aba8c67c in_seq=0 out_seq=0 2024-06-02T23:17:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd36bfff640 1 -- 172.21.15.52:0/3820082843 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56357aec0c20 con 0x56357afd9270 2024-06-02T23:17:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd36bfff640 10 client.26368 handle_command_reply: tid=0 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 client.26368 shutdown 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af55b70 msgr2=0x56357af5e690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af55b70 0x56357af5e690 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd38409b9d0 tx=0x7fd3840a4670 comp rx=0 tx=0).stop 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 shutdown_connections 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56357afd9270 0x56357afdb690 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56357af55b70 0x56357af5e690 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56357af50a00 0x56357af5e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 --2- 172.21.15.52:0/3820082843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56357af50010 0x56357af616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd36bfff640 0 client.26368 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 >> 172.21.15.52:0/3820082843 conn(0x56357af4b680 msgr2=0x56357af5a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 shutdown_connections 2024-06-02T23:17:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 1 -- 172.21.15.52:0/3820082843 wait complete. 2024-06-02T23:17:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.521+0000 7fd395e1d740 20 client.26368 trim_cache size 0 max 16384 2024-06-02T23:17:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:17:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:17:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2cb0 msgr2=0x7fd37c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2cb0 0x7fd37c0a5170 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fd380004770 tx=0x7fd380005e50 comp rx=0 tx=0).stop 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 msgr2=0x7fd38c19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 0x7fd38c19ef50 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fd38400eaa0 tx=0x7fd38400ef60 comp rx=0 tx=0).stop 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 shutdown_connections 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2cb0 0x7fd37c0a5170 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd38c1024e0 0x7fd38c19ef50 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd38c101b20 0x7fd38c19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 --2- 172.21.15.52:0/4203899052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd38c100e10 0x7fd38c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 >> 172.21.15.52:0/4203899052 conn(0x7fd38c0fb600 msgr2=0x7fd38c104ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 shutdown_connections 2024-06-02T23:17:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:02.522+0000 7fd393110640 1 -- 172.21.15.52:0/4203899052 wait complete. 2024-06-02T23:17:02.627 INFO:tasks.workunit.client.0.smithi052.stdout: 1 99917 2.14 MB/sec execute 131 sec latency 22.007 ms 2024-06-02T23:17:02.777 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:03.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 100364 2.13 MB/sec execute 132 sec latency 14.516 ms 2024-06-02T23:17:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:03 smithi174 ceph-mon[35994]: pgmap v586: 97 pgs: 97 active+clean; 55 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-02T23:17:03.896 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:03 smithi052 ceph-mon[32340]: pgmap v586: 97 pgs: 97 active+clean; 55 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-02T23:17:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:03 smithi123 ceph-mon[35977]: pgmap v586: 97 pgs: 97 active+clean; 55 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-02T23:17:04.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 100884 2.14 MB/sec execute 133 sec latency 71.108 ms 2024-06-02T23:17:04.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 -- 172.21.15.52:0/1757740872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0264108500 msgr2=0x7f02641088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 --2- 172.21.15.52:0/1757740872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0264108500 0x7f02641088e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f0260009990 tx=0x7f0260009ca0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 -- 172.21.15.52:0/1757740872 shutdown_connections 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 --2- 172.21.15.52:0/1757740872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f026406ab30 0x7f026410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 --2- 172.21.15.52:0/1757740872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0264108500 0x7f02641088e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 --2- 172.21.15.52:0/1757740872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f026406a170 0x7f026406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 -- 172.21.15.52:0/1757740872 >> 172.21.15.52:0/1757740872 conn(0x7f0264078340 msgr2=0x7f0264078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 -- 172.21.15.52:0/1757740872 shutdown_connections 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.641+0000 7f026bbc7700 1 -- 172.21.15.52:0/1757740872 wait complete. 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 Processor -- start 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 -- start start 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 0x7f026419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:04.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f026406ab30 0x7f026419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0264108500 0x7f02641989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0264112630 con 0x7f026406a170 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02641124b0 con 0x7f0264108500 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f026bbc7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02641127b0 con 0x7f026406ab30 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f0269963700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 0x7f026419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.642+0000 7f0269963700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 0x7f026419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51600/0 (socket says 172.21.15.52:51600) 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 -- 172.21.15.52:0/739731140 learned_addr learned my addr 172.21.15.52:0/739731140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f026a164700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0264108500 0x7f02641989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269162700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f026406ab30 0x7f026419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 -- 172.21.15.52:0/739731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f026406ab30 msgr2=0x7f026419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f026406ab30 0x7f026419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 -- 172.21.15.52:0/739731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0264108500 msgr2=0x7f02641989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0264108500 0x7f02641989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 -- 172.21.15.52:0/739731140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0264199110 con 0x7f026406a170 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f026a164700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0264108500 0x7f02641989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f0269963700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 0x7f026419e7a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0260000c00 tx=0x7f0260004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0260027070 con 0x7f026406a170 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0260022470 con 0x7f026406a170 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02641993a0 con 0x7f026406a170 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026000b610 con 0x7f026406a170 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.643+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02641a5670 con 0x7f026406a170 2024-06-02T23:17:04.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.644+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f026410a9e0 con 0x7f026406a170 2024-06-02T23:17:04.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.647+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f026002a070 con 0x7f026406a170 2024-06-02T23:17:04.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.647+0000 7f025affd700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02500a2ca0 0x7f02500a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:04.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.647+0000 7f0269162700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02500a2ca0 0x7f02500a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:04.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.647+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f02600cfe20 con 0x7f026406a170 2024-06-02T23:17:04.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.648+0000 7f0269162700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02500a2ca0 0x7f02500a5160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f02640789b0 tx=0x7f0254009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:04.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.652+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0260013070 con 0x7f026406a170 2024-06-02T23:17:04.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.942+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0264199e90 con 0x7f026406a170 2024-06-02T23:17:04.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.943+0000 7f025affd700 1 -- 172.21.15.52:0/739731140 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f026009ce40 con 0x7f026406a170 2024-06-02T23:17:04.942 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:17:04.942 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02500a2ca0 msgr2=0x7f02500a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02500a2ca0 0x7f02500a5160 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f02640789b0 tx=0x7f0254009260 comp rx=0 tx=0).stop 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 msgr2=0x7f026419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 0x7f026419e7a0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f0260000c00 tx=0x7f0260004790 comp rx=0 tx=0).stop 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 shutdown_connections 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02500a2ca0 0x7f02500a5160 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0264108500 0x7f02641989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f026406a170 0x7f026419e7a0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 --2- 172.21.15.52:0/739731140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f026406ab30 0x7f026419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:04.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.945+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 >> 172.21.15.52:0/739731140 conn(0x7f0264078340 msgr2=0x7f02641000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:04.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.946+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 shutdown_connections 2024-06-02T23:17:04.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:04.946+0000 7f026bbc7700 1 -- 172.21.15.52:0/739731140 wait complete. 2024-06-02T23:17:04.946 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:17:05.216 INFO:tasks.quiescer.fs.[cephfs]:Total duration (48.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:17:05.216 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.05 2024-06-02T23:17:05.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:17:05.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 101502 2.13 MB/sec execute 134 sec latency 14.966 ms 2024-06-02T23:17:05.736 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:17:05.762 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:05 smithi052 ceph-mon[32340]: pgmap v587: 97 pgs: 97 active+clean; 55 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-02T23:17:05.762 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:05 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/739731140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:05.826 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:05 smithi174 ceph-mon[35994]: pgmap v587: 97 pgs: 97 active+clean; 55 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-02T23:17:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:05 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/739731140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:06.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:05 smithi123 ceph-mon[35977]: pgmap v587: 97 pgs: 97 active+clean; 55 MiB data, 866 MiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-02T23:17:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:05 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/739731140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:06.586 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:06.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 101966 2.13 MB/sec execute 135 sec latency 14.874 ms 2024-06-02T23:17:07.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 102674 2.16 MB/sec execute 136 sec latency 14.634 ms 2024-06-02T23:17:07.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:07 smithi052 ceph-mon[32340]: pgmap v588: 97 pgs: 97 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:17:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:07 smithi174 ceph-mon[35994]: pgmap v588: 97 pgs: 97 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:17:08.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:07 smithi123 ceph-mon[35977]: pgmap v588: 97 pgs: 97 active+clean; 55 MiB data, 856 MiB used, 1.0 TiB / 1.0 TiB avail; 280 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 -- 172.21.15.52:0/335273377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e80 msgr2=0x7f899410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 --2- 172.21.15.52:0/335273377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e80 0x7f899410f330 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b0a0 tx=0x7f898c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 -- 172.21.15.52:0/335273377 shutdown_connections 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 --2- 172.21.15.52:0/335273377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e80 0x7f899410f330 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 --2- 172.21.15.52:0/335273377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941084f0 0x7f89941088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 --2- 172.21.15.52:0/335273377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024e0 0x7f8994102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 -- 172.21.15.52:0/335273377 >> 172.21.15.52:0/335273377 conn(0x7f8994079530 msgr2=0x7f8994079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 -- 172.21.15.52:0/335273377 shutdown_connections 2024-06-02T23:17:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 -- 172.21.15.52:0/335273377 wait complete. 2024-06-02T23:17:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.133+0000 7f8999764700 1 Processor -- start 2024-06-02T23:17:08.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 -- start start 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941024e0 0x7f899419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8994102e80 0x7f899419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 0x7f8994198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89941125f0 con 0x7f89941084f0 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8994112470 con 0x7f89941024e0 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8999764700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8994112770 con 0x7f8994102e80 2024-06-02T23:17:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 0x7f8994198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8992ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941024e0 0x7f899419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 0x7f8994198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51622/0 (socket says 172.21.15.52:51622) 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 -- 172.21.15.52:0/1104808061 learned_addr learned my addr 172.21.15.52:0/1104808061 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89927fc700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8994102e80 0x7f899419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 -- 172.21.15.52:0/1104808061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8994102e80 msgr2=0x7f899419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8994102e80 0x7f899419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 -- 172.21.15.52:0/1104808061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941024e0 msgr2=0x7f899419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941024e0 0x7f899419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f89937fe700 1 -- 172.21.15.52:0/1104808061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89941990f0 con 0x7f89941084f0 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.134+0000 7f8992ffd700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941024e0 0x7f899419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.135+0000 7f89937fe700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 0x7f8994198990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b780 tx=0x7f898c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.135+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c021070 con 0x7f89941084f0 2024-06-02T23:17:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.135+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f898c007e30 con 0x7f89941084f0 2024-06-02T23:17:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.135+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8994199380 con 0x7f89941084f0 2024-06-02T23:17:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.135+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898c004700 con 0x7f89941084f0 2024-06-02T23:17:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.135+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89941a56a0 con 0x7f89941084f0 2024-06-02T23:17:08.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.136+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8994069180 con 0x7f89941084f0 2024-06-02T23:17:08.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.139+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f898c007980 con 0x7f89941084f0 2024-06-02T23:17:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.140+0000 7f897bfff700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f897c0a2ca0 0x7f897c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.140+0000 7f8992ffd700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f897c0a2ca0 0x7f897c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.140+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f898c0cfe40 con 0x7f89941084f0 2024-06-02T23:17:08.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.140+0000 7f8992ffd700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f897c0a2ca0 0x7f897c0a5160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8984009680 tx=0x7f8984009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:08.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.146+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f898c003c00 con 0x7f89941084f0 2024-06-02T23:17:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.439+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8994199ec0 con 0x7f89941084f0 2024-06-02T23:17:08.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.440+0000 7f897bfff700 1 -- 172.21.15.52:0/1104808061 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f898c09ce60 con 0x7f89941084f0 2024-06-02T23:17:08.439 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:17:08.440 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.442+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f897c0a2ca0 msgr2=0x7f897c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.442+0000 7f8999764700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f897c0a2ca0 0x7f897c0a5160 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8984009680 tx=0x7f8984009260 comp rx=0 tx=0).stop 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.442+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 msgr2=0x7f8994198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.442+0000 7f8999764700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 0x7f8994198990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f898c00b780 tx=0x7f898c004d00 comp rx=0 tx=0).stop 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.442+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 shutdown_connections 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f897c0a2ca0 0x7f897c0a5160 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941084f0 0x7f8994198990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89941024e0 0x7f899419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 --2- 172.21.15.52:0/1104808061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8994102e80 0x7f899419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:08.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 >> 172.21.15.52:0/1104808061 conn(0x7f8994079530 msgr2=0x7f8994100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:08.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 shutdown_connections 2024-06-02T23:17:08.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:08.443+0000 7f8999764700 1 -- 172.21.15.52:0/1104808061 wait complete. 2024-06-02T23:17:08.443 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:17:08.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 103131 2.18 MB/sec execute 137 sec latency 30.125 ms 2024-06-02T23:17:08.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:08 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1104808061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:08.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:08 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1104808061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:09.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:08 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1104808061' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:09.561 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 144.05 --await-for 45.0' 2024-06-02T23:17:09.562 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 144.05 --await-for 45.0 2024-06-02T23:17:09.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 103373 2.17 MB/sec execute 138 sec latency 24.005 ms 2024-06-02T23:17:09.783 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:09 smithi052 ceph-mon[32340]: pgmap v589: 97 pgs: 97 active+clean; 55 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.812+0000 7f2476ff8700 1 -- 172.21.15.52:0/258095465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24701027a0 msgr2=0x7f2470106020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.812+0000 7f2476ff8700 1 --2- 172.21.15.52:0/258095465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24701027a0 0x7f2470106020 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f245c0099c0 tx=0x7f245c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 -- 172.21.15.52:0/258095465 shutdown_connections 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 --2- 172.21.15.52:0/258095465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 0x7f247010e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 --2- 172.21.15.52:0/258095465 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2470101df0 0x7f24701021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 --2- 172.21.15.52:0/258095465 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f24701027a0 0x7f2470106020 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 -- 172.21.15.52:0/258095465 >> 172.21.15.52:0/258095465 conn(0x7f2470077a00 msgr2=0x7f24700fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:09.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 -- 172.21.15.52:0/258095465 shutdown_connections 2024-06-02T23:17:09.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.813+0000 7f2476ff8700 1 -- 172.21.15.52:0/258095465 wait complete. 2024-06-02T23:17:09.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.814+0000 7f2476ff8700 1 Processor -- start 2024-06-02T23:17:09.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.814+0000 7f2476ff8700 1 -- start start 2024-06-02T23:17:09.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.814+0000 7f2476ff8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 0x7f24700fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2474d94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 0x7f24700fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2474d94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 0x7f24700fce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51642/0 (socket says 172.21.15.52:51642) 2024-06-02T23:17:09.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2474d94700 1 -- 172.21.15.52:0/976611118 learned_addr learned my addr 172.21.15.52:0/976611118 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24701027a0 0x7f24700fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 0x7f24700fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2470111560 con 0x7f2470101df0 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24701113e0 con 0x7f24701027a0 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24701116e0 con 0x7f2470106750 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24701027a0 0x7f24700fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2475595700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 0x7f24700fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 -- 172.21.15.52:0/976611118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 msgr2=0x7f24700fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 0x7f24700fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 -- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 msgr2=0x7f24700fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 0x7f24700fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 -- 172.21.15.52:0/976611118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24701a7060 con 0x7f24701027a0 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2475595700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 0x7f24700fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f2474d94700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 0x7f24700fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.815+0000 7f246ffff700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24701027a0 0x7f24700fd3b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f245c0098e0 tx=0x7f245c004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.816+0000 7f246dffb700 1 -- 172.21.15.52:0/976611118 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c026070 con 0x7f24701027a0 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.816+0000 7f246dffb700 1 -- 172.21.15.52:0/976611118 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f245c003c70 con 0x7f24701027a0 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.816+0000 7f246dffb700 1 -- 172.21.15.52:0/976611118 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f245c00fad0 con 0x7f24701027a0 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.816+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24701a72f0 con 0x7f24701027a0 2024-06-02T23:17:09.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.816+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24701a76f0 con 0x7f24701027a0 2024-06-02T23:17:09.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.819+0000 7f246dffb700 1 -- 172.21.15.52:0/976611118 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f245c029070 con 0x7f24701027a0 2024-06-02T23:17:09.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.820+0000 7f248187bb80 1 Processor -- start 2024-06-02T23:17:09.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 -- start start 2024-06-02T23:17:09.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f246dffb700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24600a2ca0 0x7f24600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee79a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f2475595700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24600a2ca0 0x7f24600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56080edc5810 con 0x56080ee79640 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56080ed6d720 con 0x56080ee79f60 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f248187bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56080edc17d0 con 0x56080ee73f20 2024-06-02T23:17:09.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f2474d94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee79a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f2475595700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f2474d94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee79a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51646/0 (socket says 172.21.15.52:51646) 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.821+0000 7f2474d94700 1 -- 172.21.15.52:0/90683274 learned_addr learned my addr 172.21.15.52:0/90683274 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f246dffb700 1 -- 172.21.15.52:0/976611118 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f245c0cf780 con 0x7f24701027a0 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f246ffff700 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2474d94700 1 -- 172.21.15.52:0/90683274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 msgr2=0x56080ee74340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2474d94700 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee74340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2475595700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24600a2ca0 0x7f24600a5160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2468007770 tx=0x7f2468007b30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2474d94700 1 -- 172.21.15.52:0/90683274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 msgr2=0x56080ee739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2474d94700 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2474d94700 1 -- 172.21.15.52:0/90683274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080ed63f50 con 0x56080ee79640 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f246ffff700 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f2474d94700 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee79a20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f24640117b0 tx=0x7f2464011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284db95e6c539d75 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f245affd700 1 -- 172.21.15.52:0/90683274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246401e070 con 0x56080ee79640 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f245affd700 1 -- 172.21.15.52:0/90683274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f246400f990 con 0x56080ee79640 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.822+0000 7f245affd700 1 -- 172.21.15.52:0/90683274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24640199f0 con 0x56080ee79640 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 -- 172.21.15.52:0/90683274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 msgr2=0x56080ee79a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee79a20 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f24640117b0 tx=0x7f2464011ac0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 -- 172.21.15.52:0/90683274 shutdown_connections 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee74340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee79a20 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 --2- 172.21.15.52:0/90683274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.823+0000 7f248187bb80 1 -- 172.21.15.52:0/90683274 >> 172.21.15.52:0/90683274 conn(0x56080ed9ecc0 msgr2=0x56080ee6e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.824+0000 7f248187bb80 1 -- 172.21.15.52:0/90683274 shutdown_connections 2024-06-02T23:17:09.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.824+0000 7f248187bb80 1 -- 172.21.15.52:0/90683274 wait complete. 2024-06-02T23:17:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.824+0000 7f248187bb80 1 -- start start 2024-06-02T23:17:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.824+0000 7f248187bb80 1 Processor -- start 2024-06-02T23:17:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 0x56080ee7fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee79640 0x56080ee7d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee7d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 0x56080ee7fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 0x56080ee7fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51658/0 (socket says 172.21.15.52:51658) 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 -- 172.21.15.52:0/2055699849 learned_addr learned my addr 172.21.15.52:0/2055699849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56080ee809d0 con 0x56080ee73f20 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56080ed65bf0 con 0x56080ee79f60 2024-06-02T23:17:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56080ee80b50 con 0x56080ee79640 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f248187bb80 10 client.0 _ll_register_callbacks cb 0x56080ed5f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2475595700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee7d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f246ffff700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee79640 0x56080ee7d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 -- 172.21.15.52:0/2055699849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee79640 msgr2=0x56080ee7d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee79640 0x56080ee7d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 -- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 msgr2=0x56080ee7d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee7d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.825+0000 7f2474d94700 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080ed148d0 con 0x56080ee73f20 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f2474d94700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 0x56080ee7fd20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f24640195d0 tx=0x7f246400fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57722092adaa03ca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f244ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f244ffff700 1 -- 172.21.15.52:0/2055699849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246401e040 con 0x56080ee73f20 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f244ffff700 1 -- 172.21.15.52:0/2055699849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2464026c00 con 0x56080ee73f20 2024-06-02T23:17:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f246400f990 con 0x56080ee73f20 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f244ffff700 1 -- 172.21.15.52:0/2055699849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2464005ce0 con 0x56080ee73f20 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f248187bb80 10 client.17013 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.826+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56080ee8b280 con 0x56080ee73f20 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f244ffff700 1 -- 172.21.15.52:0/2055699849 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f246402a030 con 0x56080ee73f20 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f244ffff700 1 -- 172.21.15.52:0/2055699849 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2464003a50 con 0x56080ee73f20 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f248187bb80 10 client.17013 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f248187bb80 10 client.17013 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f248187bb80 20 client.17013 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f248187bb80 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080ee8fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f248187bb80 4 client.17013 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f246400f990 con 0x56080ee8d7b0 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.827+0000 7f246ffff700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080ee8fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.828+0000 7f244ffff700 10 client.17013 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:17:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.828+0000 7f246ffff700 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080ee8fba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=24ed1dad02bac42f server_cookie=e9f011c878827f8e in_seq=0 out_seq=0 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f244ffff700 1 -- 172.21.15.52:0/2055699849 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f246400f990 con 0x56080ee8d7b0 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f244ffff700 10 client.17013 handle_command_reply: tid=0 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 client.17013 shutdown 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 msgr2=0x56080ee7fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 0x56080ee7fd20 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f24640195d0 tx=0x7f246400fdf0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 shutdown_connections 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080ee8fba0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f244ffff700 0 client.17013 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:17:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee7d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee73f20 0x56080ee7fd20 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 --2- 172.21.15.52:0/2055699849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee79640 0x56080ee7d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 >> 172.21.15.52:0/2055699849 conn(0x56080ed9ecc0 msgr2=0x56080ee729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 shutdown_connections 2024-06-02T23:17:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 1 -- 172.21.15.52:0/2055699849 wait complete. 2024-06-02T23:17:09.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.829+0000 7f248187bb80 20 client.17013 trim_cache size 0 max 16384 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.849+0000 7f248187bb80 1 Processor -- start 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.849+0000 7f248187bb80 1 -- start start 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f248187bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee739e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f248187bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee7bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f248187bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee775c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f248187bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24701aa7b0 con 0x56080ee79640 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f248187bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24701ac2f0 con 0x56080ee79f60 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f248187bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f24701aae80 con 0x56080ee73f20 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f2475595700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee775c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f2475595700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee775c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33742/0 (socket says 172.21.15.52:33742) 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f2475595700 1 -- 172.21.15.52:0/967847048 learned_addr learned my addr 172.21.15.52:0/967847048 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f246ffff700 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee7bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.850+0000 7f2474d94700 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee739e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f2475595700 1 -- 172.21.15.52:0/967847048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 msgr2=0x56080ee739e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f2475595700 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee739e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f2475595700 1 -- 172.21.15.52:0/967847048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 msgr2=0x56080ee7bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f2475595700 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee7bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f2475595700 1 -- 172.21.15.52:0/967847048 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080ee80b50 con 0x56080ee79f60 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f246ffff700 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee7bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f2475595700 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee775c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2468017770 tx=0x7f246801d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6651b1c76317ce server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f245a7fc700 1 -- 172.21.15.52:0/967847048 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246801a670 con 0x56080ee79f60 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f245a7fc700 1 -- 172.21.15.52:0/967847048 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2468020040 con 0x56080ee79f60 2024-06-02T23:17:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.851+0000 7f245a7fc700 1 -- 172.21.15.52:0/967847048 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24680259a0 con 0x56080ee79f60 2024-06-02T23:17:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 -- 172.21.15.52:0/967847048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 msgr2=0x56080ee775c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee775c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2468017770 tx=0x7f246801d000 comp rx=0 tx=0).stop 2024-06-02T23:17:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 -- 172.21.15.52:0/967847048 shutdown_connections 2024-06-02T23:17:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee775c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee739e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 --2- 172.21.15.52:0/967847048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee7bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 -- 172.21.15.52:0/967847048 >> 172.21.15.52:0/967847048 conn(0x56080ed9ecc0 msgr2=0x56080ee76eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 -- 172.21.15.52:0/967847048 shutdown_connections 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.852+0000 7f248187bb80 1 -- 172.21.15.52:0/967847048 wait complete. 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 -- start start 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 Processor -- start 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee862c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee948a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee94de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f24701abc20 con 0x56080ee79640 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f24701ab550 con 0x56080ee79f60 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f247019c800 con 0x56080ee73f20 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f248187bb80 10 client.0 _ll_register_callbacks cb 0x56080ed5f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f2474d94700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee862c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f246ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f2474d94700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee862c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:40058/0 (socket says 172.21.15.52:40058) 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f2474d94700 1 -- 172.21.15.52:0/3734697175 learned_addr learned my addr 172.21.15.52:0/3734697175 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.853+0000 7f2475595700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee94de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f2474d94700 1 -- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 msgr2=0x56080ee94de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f2474d94700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee94de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f2474d94700 1 -- 172.21.15.52:0/3734697175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 msgr2=0x56080ee948a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f2474d94700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee948a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f2474d94700 1 -- 172.21.15.52:0/3734697175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080ee95320 con 0x56080ee73f20 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f246ffff700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f2474d94700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee862c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24640195d0 tx=0x7f2464009570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97f64082f06c3f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f244ffff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f244ffff700 1 -- 172.21.15.52:0/3734697175 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2464015070 con 0x56080ee73f20 2024-06-02T23:17:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f244ffff700 1 -- 172.21.15.52:0/3734697175 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2464003680 con 0x56080ee73f20 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2468020040 con 0x56080ee73f20 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.854+0000 7f244ffff700 1 -- 172.21.15.52:0/3734697175 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2464003880 con 0x56080ee73f20 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 10 client.26398 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56080eef7ef0 con 0x56080ee73f20 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f244ffff700 1 -- 172.21.15.52:0/3734697175 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2468020040 con 0x56080ee73f20 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f244ffff700 1 -- 172.21.15.52:0/3734697175 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f246403c4f0 con 0x56080ee73f20 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 10 client.26398 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 10 client.26398 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 20 client.26398 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080eefa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 4 client.26398 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56080edd2d10 con 0x56080ee8d7b0 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.855+0000 7f246ffff700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080eefa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.856+0000 7f244ffff700 10 client.26398 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.856+0000 7f246ffff700 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080eefa520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5f8639704c32949a server_cookie=37b3e1a126786bd in_seq=0 out_seq=0 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.856+0000 7f244ffff700 1 -- 172.21.15.52:0/3734697175 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56080edd2d10 con 0x56080ee8d7b0 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.856+0000 7f244ffff700 10 client.26398 handle_command_reply: tid=0 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.856+0000 7f248187bb80 1 client.26398 shutdown 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 msgr2=0x56080ee862c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee862c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f24640195d0 tx=0x7f2464009570 comp rx=0 tx=0).stop 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 shutdown_connections 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56080ee8d7b0 0x56080eefa520 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56080ee79f60 0x56080ee94de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56080ee73f20 0x56080ee862c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 --2- 172.21.15.52:0/3734697175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56080ee79640 0x56080ee948a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 >> 172.21.15.52:0/3734697175 conn(0x56080ed9ecc0 msgr2=0x56080ee67e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f244ffff700 0 client.26398 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:17:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 shutdown_connections 2024-06-02T23:17:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 1 -- 172.21.15.52:0/3734697175 wait complete. 2024-06-02T23:17:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.857+0000 7f248187bb80 20 client.26398 trim_cache size 0 max 16384 2024-06-02T23:17:09.861 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24600a2ca0 msgr2=0x7f24600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24600a2ca0 0x7f24600a5160 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f2468007770 tx=0x7f2468007b30 comp rx=0 tx=0).stop 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24701027a0 msgr2=0x7f24700fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24701027a0 0x7f24700fd3b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f245c0098e0 tx=0x7f245c004270 comp rx=0 tx=0).stop 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 shutdown_connections 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f24600a2ca0 0x7f24600a5160 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2470106750 0x7f24700fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2470101df0 0x7f24700fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 --2- 172.21.15.52:0/976611118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24701027a0 0x7f24700fd3b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 >> 172.21.15.52:0/976611118 conn(0x7f2470077a00 msgr2=0x7f2470100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 shutdown_connections 2024-06-02T23:17:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:09.858+0000 7f2476ff8700 1 -- 172.21.15.52:0/976611118 wait complete. 2024-06-02T23:17:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:09 smithi174 ceph-mon[35994]: pgmap v589: 97 pgs: 97 active+clean; 55 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-02T23:17:10.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:09 smithi123 ceph-mon[35977]: pgmap v589: 97 pgs: 97 active+clean; 55 MiB data, 857 MiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.2 MiB/s wr, 165 op/s 2024-06-02T23:17:10.068 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:10.611 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:17:10.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 103779 2.16 MB/sec execute 139 sec latency 15.725 ms 2024-06-02T23:17:11.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 104314 2.17 MB/sec execute 140 sec latency 15.069 ms 2024-06-02T23:17:11.774 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:11 smithi052 ceph-mon[32340]: pgmap v590: 97 pgs: 97 active+clean; 56 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-02T23:17:11.782 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:11 smithi174 ceph-mon[35994]: pgmap v590: 97 pgs: 97 active+clean; 56 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-02T23:17:12.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:11 smithi123 ceph-mon[35977]: pgmap v590: 97 pgs: 97 active+clean; 56 MiB data, 862 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-02T23:17:12.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 104890 2.16 MB/sec execute 141 sec latency 37.074 ms 2024-06-02T23:17:13.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 -- 172.21.15.52:0/305687489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406aa30 msgr2=0x7f0bd406aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/305687489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406aa30 0x7f0bd406aeb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f0bc40099c0 tx=0x7f0bc4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 -- 172.21.15.52:0/305687489 shutdown_connections 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/305687489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd406b3f0 0x7f0bd410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/305687489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd41073a0 0x7f0bd4107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/305687489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406aa30 0x7f0bd406aeb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 -- 172.21.15.52:0/305687489 >> 172.21.15.52:0/305687489 conn(0x7f0bd4078390 msgr2=0x7f0bd40787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 -- 172.21.15.52:0/305687489 shutdown_connections 2024-06-02T23:17:13.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.464+0000 7f0bdbe55700 1 -- 172.21.15.52:0/305687489 wait complete. 2024-06-02T23:17:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 Processor -- start 2024-06-02T23:17:13.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 -- start start 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 0x7f0bd4101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406b3f0 0x7f0bd4101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd41073a0 0x7f0bd4101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4112680 con 0x7f0bd406b3f0 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4112500 con 0x7f0bd406aa30 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bdbe55700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd4112800 con 0x7f0bd41073a0 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bd9bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 0x7f0bd4101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bd9bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 0x7f0bd4101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33764/0 (socket says 172.21.15.52:33764) 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/2278012974 learned_addr learned my addr 172.21.15.52:0/2278012974 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406b3f0 0x7f0bd4101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.465+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd41073a0 0x7f0bd4101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/2278012974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd41073a0 msgr2=0x7f0bd4101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd41073a0 0x7f0bd4101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406b3f0 msgr2=0x7f0bd4101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406b3f0 0x7f0bd4101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/2278012974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd41a6fa0 con 0x7f0bd406aa30 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 0x7f0bd4101450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e9a0 tx=0x7f0bd000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bcaffd700 1 -- 172.21.15.52:0/2278012974 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd000ca10 con 0x7f0bd406aa30 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd41a7290 con 0x7f0bd406aa30 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bcaffd700 1 -- 172.21.15.52:0/2278012974 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bd000cbb0 con 0x7f0bd406aa30 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bcaffd700 1 -- 172.21.15.52:0/2278012974 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd0018680 con 0x7f0bd406aa30 2024-06-02T23:17:13.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.466+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd41a77a0 con 0x7f0bd406aa30 2024-06-02T23:17:13.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.469+0000 7f0bcaffd700 1 -- 172.21.15.52:0/2278012974 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0bd0018820 con 0x7f0bd406aa30 2024-06-02T23:17:13.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.470+0000 7f0bcaffd700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bc00a2cc0 0x7f0bc00a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.470+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bc00a2cc0 0x7f0bc00a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.470+0000 7f0bcaffd700 1 -- 172.21.15.52:0/2278012974 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0bd00cf180 con 0x7f0bd406aa30 2024-06-02T23:17:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bc00a2cc0 0x7f0bc00a5180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4102ee0 tx=0x7f0bc4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 Processor -- start 2024-06-02T23:17:13.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 -- start start 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4215f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd42168b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cdd41679b0 con 0x55cdd4216470 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cdd410f8c0 con 0x55cdd421c600 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.471+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cdd4163970 con 0x55cdd421bce0 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4215f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bda3f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd42168b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4215f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33780/0 (socket says 172.21.15.52:33780) 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bda3f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd42168b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51696/0 (socket says 172.21.15.52:51696) 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 -- 172.21.15.52:0/1080653285 learned_addr learned my addr 172.21.15.52:0/1080653285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 -- 172.21.15.52:0/1080653285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 msgr2=0x55cdd421c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 -- 172.21.15.52:0/1080653285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 msgr2=0x55cdd42168b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd42168b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 -- 172.21.15.52:0/1080653285 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd41060f0 con 0x55cdd421c600 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd42168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4215f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4015000 tx=0x7f0bc401a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe552e009b55243 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:13.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.472+0000 7f0bbffff700 1 -- 172.21.15.52:0/1080653285 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4009e60 con 0x55cdd421c600 2024-06-02T23:17:13.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0bbffff700 1 -- 172.21.15.52:0/1080653285 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bc4003c50 con 0x55cdd421c600 2024-06-02T23:17:13.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0bbffff700 1 -- 172.21.15.52:0/1080653285 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4023920 con 0x55cdd421c600 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 -- 172.21.15.52:0/1080653285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 msgr2=0x55cdd4215f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4215f30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0bc4015000 tx=0x7f0bc401a070 comp rx=0 tx=0).stop 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 -- 172.21.15.52:0/1080653285 shutdown_connections 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd42168b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/1080653285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4215f30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.473+0000 7f0be66d8b80 1 -- 172.21.15.52:0/1080653285 >> 172.21.15.52:0/1080653285 conn(0x55cdd4140e60 msgr2=0x55cdd42111c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.474+0000 7f0be66d8b80 1 -- 172.21.15.52:0/1080653285 shutdown_connections 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.474+0000 7f0be66d8b80 1 -- 172.21.15.52:0/1080653285 wait complete. 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.474+0000 7f0be66d8b80 1 -- start start 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.474+0000 7f0be66d8b80 1 Processor -- start 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd4222370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd421fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cdd4222fb0 con 0x55cdd4216470 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0bd9bf1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd4222370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0bd9bf1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd4222370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51712/0 (socket says 172.21.15.52:51712) 2024-06-02T23:17:13.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/3850649008 learned_addr learned my addr 172.21.15.52:0/3850649008 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd421fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cdd4107d90 con 0x55cdd421c600 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cdd4223130 con 0x55cdd421bce0 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.475+0000 7f0be66d8b80 10 client.0 _ll_register_callbacks cb 0x55cdd4101ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/3850649008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 msgr2=0x55cdd421f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 msgr2=0x55cdd421fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd421fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/3850649008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd40b6a70 con 0x55cdd4216470 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd421fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd4222370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00cccb0 tx=0x7f0bd009c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56cddec670835884 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:13.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/3850649008 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bd00ce070 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/3850649008 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0bd000cd60 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4003c50 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/3850649008 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0bd009fdc0 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0be66d8b80 10 client.17031 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.476+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdd422d850 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/3850649008 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bd00a4680 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/3850649008 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0bd0097760 con 0x55cdd4216470 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0be66d8b80 10 client.17031 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0be66d8b80 10 client.17031 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0be66d8b80 20 client.17031 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd4232280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0be66d8b80 4 client.17031 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0bc4003c50 con 0x55cdd422fe90 2024-06-02T23:17:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd4232280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.477+0000 7f0bbd7fa700 10 client.17031 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:13.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.478+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd4232280 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aba48d08d1c1ad6 server_cookie=9f11a904049a979 in_seq=0 out_seq=0 2024-06-02T23:17:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/3850649008 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f0bc4003c50 con 0x55cdd422fe90 2024-06-02T23:17:13.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0bbd7fa700 10 client.17031 handle_command_reply: tid=0 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 client.17031 shutdown 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 msgr2=0x55cdd4222370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd4222370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f0bd00cccb0 tx=0x7f0bd009c270 comp rx=0 tx=0).stop 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 shutdown_connections 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd4232280 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd421fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0bbd7fa700 0 client.17031 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd4222370 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/3850649008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd421f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.479+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 >> 172.21.15.52:0/3850649008 conn(0x55cdd4140e60 msgr2=0x55cdd4214f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.480+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 shutdown_connections 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.480+0000 7f0be66d8b80 1 -- 172.21.15.52:0/3850649008 wait complete. 2024-06-02T23:17:13.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.480+0000 7f0be66d8b80 20 client.17031 trim_cache size 0 max 16384 2024-06-02T23:17:13.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 Processor -- start 2024-06-02T23:17:13.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 -- start start 2024-06-02T23:17:13.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd421e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd4237a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4237fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41aec00 con 0x55cdd4216470 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41b0740 con 0x55cdd421c600 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41af2d0 con 0x55cdd421bce0 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd4237a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd4237a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:40102/0 (socket says 172.21.15.52:40102) 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 -- 172.21.15.52:0/2181296320 learned_addr learned my addr 172.21.15.52:0/2181296320 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:13.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4237fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd421e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 -- 172.21.15.52:0/2181296320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 msgr2=0x55cdd4237fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4237fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 -- 172.21.15.52:0/2181296320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 msgr2=0x55cdd421e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd421e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd93f0700 1 -- 172.21.15.52:0/2181296320 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdd4232860 con 0x55cdd421bce0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.518+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd421e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.519+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd4237a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b580 tx=0x7f0bc4025040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=590a9a1fa9f04f2f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.519+0000 7f0bbf7fe700 1 -- 172.21.15.52:0/2181296320 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4046070 con 0x55cdd421bce0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.519+0000 7f0bbf7fe700 1 -- 172.21.15.52:0/2181296320 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bc4003bd0 con 0x55cdd421bce0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.519+0000 7f0bbf7fe700 1 -- 172.21.15.52:0/2181296320 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc4038400 con 0x55cdd421bce0 2024-06-02T23:17:13.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2181296320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 msgr2=0x55cdd4237a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd4237a90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0bc400b580 tx=0x7f0bc4025040 comp rx=0 tx=0).stop 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2181296320 shutdown_connections 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd421c600 0x55cdd4237fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd4216470 0x55cdd421e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2181296320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd4237a90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2181296320 >> 172.21.15.52:0/2181296320 conn(0x55cdd4140e60 msgr2=0x55cdd421d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2181296320 shutdown_connections 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2181296320 wait complete. 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.520+0000 7f0be66d8b80 1 -- start start 2024-06-02T23:17:13.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 Processor -- start 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 0x55cdd4225390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd42258d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd421c600 0x55cdd4225e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41b0070 con 0x55cdd421c600 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41af9a0 con 0x55cdd4216470 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd41acbc0 con 0x55cdd421bce0 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0be66d8b80 10 client.0 _ll_register_callbacks cb 0x55cdd4101ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0bd9bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 0x55cdd4225390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0bd9bf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 0x55cdd4225390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33824/0 (socket says 172.21.15.52:33824) 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0bd9bf1700 1 -- 172.21.15.52:0/2882697155 learned_addr learned my addr 172.21.15.52:0/2882697155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd421c600 0x55cdd4225e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.521+0000 7f0bd93f0700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd42258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bda3f2700 1 -- 172.21.15.52:0/2882697155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 msgr2=0x55cdd42258d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd42258d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bda3f2700 1 -- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 msgr2=0x55cdd4225390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 0x55cdd4225390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bda3f2700 1 -- 172.21.15.52:0/2882697155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdd4212960 con 0x55cdd421c600 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 0x55cdd4225390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bda3f2700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd421c600 0x55cdd4225e10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc007ed0 tx=0x7f0bcc004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545655768eb1b4dc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bbd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/2882697155 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bcc004bb0 con 0x55cdd421c600 2024-06-02T23:17:13.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/2882697155 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0bcc004d50 con 0x55cdd421c600 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0bc4003bd0 con 0x55cdd421c600 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.522+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/2882697155 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0bcc020410 con 0x55cdd421c600 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 10 client.17034 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cdd4212550 con 0x55cdd421c600 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/2882697155 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0bc4003bd0 con 0x55cdd421c600 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/2882697155 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0bcc02c760 con 0x55cdd421c600 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 10 client.17034 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 10 client.17034 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 20 client.17034 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd42a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 4 client.17034 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.05, "await_for": 45.0}] 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.523+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.05, "await_for": 45.0}) -- 0x55cdd4174eb0 con 0x55cdd422fe90 2024-06-02T23:17:13.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.524+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd42a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:13.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.524+0000 7f0bbd7fa700 10 client.17034 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:13.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.524+0000 7f0bd9bf1700 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd42a6ad0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a41786197885582 server_cookie=874ccadb6803cfa in_seq=0 out_seq=0 2024-06-02T23:17:13.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0bbd7fa700 1 -- 172.21.15.52:0/2882697155 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55cdd4174eb0 con 0x55cdd422fe90 2024-06-02T23:17:13.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0bbd7fa700 10 client.17034 handle_command_reply: tid=0 2024-06-02T23:17:13.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 client.17034 shutdown 2024-06-02T23:17:13.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:13 smithi052 ceph-mon[32340]: pgmap v591: 97 pgs: 97 active+clean; 57 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-06-02T23:17:13.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd421c600 msgr2=0x55cdd4225e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd421c600 0x55cdd4225e10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f0bcc007ed0 tx=0x7f0bcc004320 comp rx=0 tx=0).stop 2024-06-02T23:17:13.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 shutdown_connections 2024-06-02T23:17:13.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cdd422fe90 0x55cdd42a6ad0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0bbd7fa700 0 client.17034 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cdd421c600 0x55cdd4225e10 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cdd4216470 0x55cdd4225390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.592+0000 7f0be66d8b80 1 --2- 172.21.15.52:0/2882697155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cdd421bce0 0x55cdd42258d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.593+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 >> 172.21.15.52:0/2882697155 conn(0x55cdd4140e60 msgr2=0x55cdd4211160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.593+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 shutdown_connections 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.593+0000 7f0be66d8b80 1 -- 172.21.15.52:0/2882697155 wait complete. 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.593+0000 7f0be66d8b80 20 client.17034 trim_cache size 0 max 16384 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:17:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 18, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "b117150": { 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 18, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 144.1, 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:17:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:13.596 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bc00a2cc0 msgr2=0x7f0bc00a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bc00a2cc0 0x7f0bc00a5180 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0bd4102ee0 tx=0x7f0bc4005fb0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 msgr2=0x7f0bd4101450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 0x7f0bd4101450 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0bd000e9a0 tx=0x7f0bd000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 shutdown_connections 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0bc00a2cc0 0x7f0bc00a5180 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd41073a0 0x7f0bd4101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0bd406aa30 0x7f0bd4101450 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 --2- 172.21.15.52:0/2278012974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0bd406b3f0 0x7f0bd4101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 >> 172.21.15.52:0/2278012974 conn(0x7f0bd4078390 msgr2=0x7f0bd40ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 shutdown_connections 2024-06-02T23:17:13.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:13.594+0000 7f0bdbe55700 1 -- 172.21.15.52:0/2278012974 wait complete. 2024-06-02T23:17:13.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.16 MB/sec execute 142 sec latency 105.023 ms 2024-06-02T23:17:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:13 smithi174 ceph-mon[35994]: pgmap v591: 97 pgs: 97 active+clean; 57 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-06-02T23:17:14.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:13 smithi123 ceph-mon[35977]: pgmap v591: 97 pgs: 97 active+clean; 57 MiB data, 890 MiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 1.3 MiB/s wr, 148 op/s 2024-06-02T23:17:14.417 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b117150', quiesce took 9.2 seconds. Will release after: 14.850000000000001 2024-06-02T23:17:14.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.14 MB/sec execute 143 sec latency 1105.075 ms 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 -- 172.21.15.52:0/2265929138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 msgr2=0x7f2814107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 --2- 172.21.15.52:0/2265929138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f2814107120 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f28080099c0 tx=0x7f2808009cd0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 -- 172.21.15.52:0/2265929138 shutdown_connections 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 --2- 172.21.15.52:0/2265929138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2814107850 0x7f281410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 --2- 172.21.15.52:0/2265929138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f281406ae80 0x7f281406b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 --2- 172.21.15.52:0/2265929138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f2814107120 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 -- 172.21.15.52:0/2265929138 >> 172.21.15.52:0/2265929138 conn(0x7f2814078400 msgr2=0x7f2814078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 -- 172.21.15.52:0/2265929138 shutdown_connections 2024-06-02T23:17:14.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.673+0000 7f281aa2c700 1 -- 172.21.15.52:0/2265929138 wait complete. 2024-06-02T23:17:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 Processor -- start 2024-06-02T23:17:14.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 -- start start 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281406ae80 0x7f281419e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f281419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2814107850 0x7f28141988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2814112630 con 0x7f281406b830 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28141124b0 con 0x7f2814107850 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f281aa2c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28141127b0 con 0x7f281406ae80 2024-06-02T23:17:14.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f28137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f281419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f28137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f281419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38384/0 (socket says 172.21.15.52:38384) 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f28137fe700 1 -- 172.21.15.52:0/3889370780 learned_addr learned my addr 172.21.15.52:0/3889370780 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f2818fc9700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2814107850 0x7f28141988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.674+0000 7f2813fff700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281406ae80 0x7f281419e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28137fe700 1 -- 172.21.15.52:0/3889370780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281406ae80 msgr2=0x7f281419e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28137fe700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281406ae80 0x7f281419e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28137fe700 1 -- 172.21.15.52:0/3889370780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2814107850 msgr2=0x7f28141988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28137fe700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2814107850 0x7f28141988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28137fe700 1 -- 172.21.15.52:0/3889370780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2814199050 con 0x7f281406b830 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28137fe700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f281419ec70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2808006010 tx=0x7f2808009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28117fa700 1 -- 172.21.15.52:0/3889370780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808026070 con 0x7f281406b830 2024-06-02T23:17:14.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28117fa700 1 -- 172.21.15.52:0/3889370780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2808004ca0 con 0x7f281406b830 2024-06-02T23:17:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28141992e0 con 0x7f281406b830 2024-06-02T23:17:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f28117fa700 1 -- 172.21.15.52:0/3889370780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2808003d20 con 0x7f281406b830 2024-06-02T23:17:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.675+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28141a5600 con 0x7f281406b830 2024-06-02T23:17:14.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.677+0000 7f28117fa700 1 -- 172.21.15.52:0/3889370780 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2808029070 con 0x7f281406b830 2024-06-02T23:17:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.678+0000 7f28117fa700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27fc0a2c20 0x7f27fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.678+0000 7f2813fff700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27fc0a2c20 0x7f27fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.678+0000 7f28117fa700 1 -- 172.21.15.52:0/3889370780 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f28080cf850 con 0x7f281406b830 2024-06-02T23:17:14.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f2813fff700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27fc0a2c20 0x7f27fc0a50e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f280c009550 tx=0x7f280c0091f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 Processor -- start 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 -- start start 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbd7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbd1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd17a0 0x55c94dbd1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c94dac3f60 con 0x55c94dbd6f60 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c94db22850 con 0x55c94dbd7880 2024-06-02T23:17:14.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28252afb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c94da8a460 con 0x55c94dbd17a0 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f2813fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbd7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f2813fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbd7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38396/0 (socket says 172.21.15.52:38396) 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f2813fff700 1 -- 172.21.15.52:0/207721477 learned_addr learned my addr 172.21.15.52:0/207721477 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.679+0000 7f28137fe700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbd1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2818fc9700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd17a0 0x55c94dbd1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2813fff700 1 -- 172.21.15.52:0/207721477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd17a0 msgr2=0x55c94dbd1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2813fff700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd17a0 0x55c94dbd1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2813fff700 1 -- 172.21.15.52:0/207721477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 msgr2=0x55c94dbd1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2813fff700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbd1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2813fff700 1 -- 172.21.15.52:0/207721477 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c94da718d0 con 0x55c94dbd6f60 2024-06-02T23:17:14.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f28137fe700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbd1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2818fc9700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd17a0 0x55c94dbd1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f2813fff700 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbd7340 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f280c009690 tx=0x7f280c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=670e86fab946e5dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f27fa7fc700 1 -- 172.21.15.52:0/207721477 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c01c050 con 0x55c94dbd6f60 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f27fa7fc700 1 -- 172.21.15.52:0/207721477 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c026070 con 0x55c94dbd6f60 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.680+0000 7f27fa7fc700 1 -- 172.21.15.52:0/207721477 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c015680 con 0x55c94dbd6f60 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 -- 172.21.15.52:0/207721477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 msgr2=0x55c94dbd7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbd7340 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f280c009690 tx=0x7f280c013000 comp rx=0 tx=0).stop 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 -- 172.21.15.52:0/207721477 shutdown_connections 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd17a0 0x55c94dbd1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbd7340 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 --2- 172.21.15.52:0/207721477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbd1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 -- 172.21.15.52:0/207721477 >> 172.21.15.52:0/207721477 conn(0x55c94dafe080 msgr2=0x55c94dbcc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:14.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 -- 172.21.15.52:0/207721477 shutdown_connections 2024-06-02T23:17:14.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 -- 172.21.15.52:0/207721477 wait complete. 2024-06-02T23:17:14.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 -- start start 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.681+0000 7f28252afb80 1 Processor -- start 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbdd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbdd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd7880 0x55c94dbda8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c94dac2fc0 con 0x55c94dbd6f60 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c94db1e810 con 0x55c94dbd17a0 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c94dbde260 con 0x55c94dbd7880 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28252afb80 10 client.0 _ll_register_callbacks cb 0x55c94dab85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbdd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbdd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38398/0 (socket says 172.21.15.52:38398) 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 -- 172.21.15.52:0/1573126257 learned_addr learned my addr 172.21.15.52:0/1573126257 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f2818fc9700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd7880 0x55c94dbda8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f2813fff700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbdd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 -- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd7880 msgr2=0x55c94dbda8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd7880 0x55c94dbda8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 -- 172.21.15.52:0/1573126257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 msgr2=0x55c94dbdd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbdd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 -- 172.21.15.52:0/1573126257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94dae8750 con 0x55c94dbd6f60 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f2818fc9700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd7880 0x55c94dbda8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f28137fe700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbdd8e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2808000c00 tx=0x7f2808005130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe313193269aa603 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.682+0000 7f27f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f27f37fe700 1 -- 172.21.15.52:0/1573126257 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280809c990 con 0x55c94dbd6f60 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f27f37fe700 1 -- 172.21.15.52:0/1573126257 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f28080a49e0 con 0x55c94dbd6f60 2024-06-02T23:17:14.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c026070 con 0x55c94dbd6f60 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f27f37fe700 1 -- 172.21.15.52:0/1573126257 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f280809c290 con 0x55c94dbd6f60 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 10 client.17046 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c94dbf11b0 con 0x55c94dbd6f60 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f27f37fe700 1 -- 172.21.15.52:0/1573126257 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c026070 con 0x55c94dbd6f60 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f27f37fe700 1 -- 172.21.15.52:0/1573126257 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f280809b6e0 con 0x55c94dbd6f60 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 10 client.17046 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 10 client.17046 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 20 client.17046 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dbf3720 0x55c94dbf5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 4 client.17046 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.683+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c94db2fd50 con 0x55c94dbf3720 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.684+0000 7f2813fff700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dbf3720 0x55c94dbf5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.684+0000 7f2813fff700 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dbf3720 0x55c94dbf5b40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=476ffebef5bcb016 server_cookie=ca7c69bd3215a7d2 in_seq=0 out_seq=0 2024-06-02T23:17:14.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.684+0000 7f27f37fe700 10 client.17046 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f27f37fe700 1 -- 172.21.15.52:0/1573126257 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c94db2fd50 con 0x55c94dbf3720 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f27f37fe700 10 client.17046 handle_command_reply: tid=0 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 client.17046 shutdown 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 msgr2=0x55c94dbdd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbdd8e0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f2808000c00 tx=0x7f2808005130 comp rx=0 tx=0).stop 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 shutdown_connections 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dbf3720 0x55c94dbf5b40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd7880 0x55c94dbda8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbdd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 --2- 172.21.15.52:0/1573126257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd6f60 0x55c94dbdd8e0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 >> 172.21.15.52:0/1573126257 conn(0x55c94dafe080 msgr2=0x55c94dbcb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f27f37fe700 0 client.17046 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 shutdown_connections 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 1 -- 172.21.15.52:0/1573126257 wait complete. 2024-06-02T23:17:14.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.685+0000 7f28252afb80 20 client.17046 trim_cache size 0 max 16384 2024-06-02T23:17:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.700+0000 7f28252afb80 1 Processor -- start 2024-06-02T23:17:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.700+0000 7f28252afb80 1 -- start start 2024-06-02T23:17:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.700+0000 7f28252afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 0x55c94dbda8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.700+0000 7f28252afb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbcbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.700+0000 7f28252afb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbcc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.700+0000 7f28252afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28141aa750 con 0x55c94dbd17a0 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f28252afb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28141ac290 con 0x55c94dbd7880 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f28252afb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28141aae20 con 0x55c94dbd6f60 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 0x55c94dbda8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 0x55c94dbda8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38406/0 (socket says 172.21.15.52:38406) 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 -- 172.21.15.52:0/3963895566 learned_addr learned my addr 172.21.15.52:0/3963895566 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2818fc9700 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbcc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f28137fe700 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbcbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 -- 172.21.15.52:0/3963895566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 msgr2=0x55c94dbcbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbcbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 -- 172.21.15.52:0/3963895566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 msgr2=0x55c94dbcc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbcc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 -- 172.21.15.52:0/3963895566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c94dbd8e60 con 0x55c94dbd17a0 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f2813fff700 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 0x55c94dbda8b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f280c007b70 tx=0x7f280c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1299b3dc0777e653 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:14.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f27f9ffb700 1 -- 172.21.15.52:0/3963895566 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c01c070 con 0x55c94dbd17a0 2024-06-02T23:17:14.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f27f9ffb700 1 -- 172.21.15.52:0/3963895566 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c026040 con 0x55c94dbd17a0 2024-06-02T23:17:14.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.701+0000 7f27f9ffb700 1 -- 172.21.15.52:0/3963895566 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280c034640 con 0x55c94dbd17a0 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 -- 172.21.15.52:0/3963895566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 msgr2=0x55c94dbda8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 0x55c94dbda8b0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f280c007b70 tx=0x7f280c00b040 comp rx=0 tx=0).stop 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 -- 172.21.15.52:0/3963895566 shutdown_connections 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd7880 0x55c94dbcc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd17a0 0x55c94dbda8b0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 --2- 172.21.15.52:0/3963895566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbcbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 -- 172.21.15.52:0/3963895566 >> 172.21.15.52:0/3963895566 conn(0x55c94dafe080 msgr2=0x55c94dbda1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 -- 172.21.15.52:0/3963895566 shutdown_connections 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 -- 172.21.15.52:0/3963895566 wait complete. 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 -- start start 2024-06-02T23:17:14.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.702+0000 7f28252afb80 1 Processor -- start 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbf0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbf0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd7880 0x55c94dbfa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f28141abbc0 con 0x55c94dbd7880 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28141ab4f0 con 0x55c94dbd17a0 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28141a8710 con 0x55c94dbd6f60 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28252afb80 10 client.0 _ll_register_callbacks cb 0x55c94dab85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2813fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbf0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd7880 0x55c94dbfa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2813fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbf0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33430/0 (socket says 172.21.15.52:33430) 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2813fff700 1 -- 172.21.15.52:0/4258964846 learned_addr learned my addr 172.21.15.52:0/4258964846 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f28137fe700 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbf0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 -- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 msgr2=0x55c94dbf0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbf0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 -- 172.21.15.52:0/4258964846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 msgr2=0x55c94dbf0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbf0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 -- 172.21.15.52:0/4258964846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c94dbfb060 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f2818fc9700 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd7880 0x55c94dbfa9a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x55c94dbe2940 tx=0x7f2804012df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c44718c721d259 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.703+0000 7f27f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f27f37fe700 1 -- 172.21.15.52:0/4258964846 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2804011c10 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f27f37fe700 1 -- 172.21.15.52:0/4258964846 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2804011db0 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f280c026040 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f27f37fe700 1 -- 172.21.15.52:0/4258964846 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2804004830 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 10 client.17052 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c94dc5dd60 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f27f37fe700 1 -- 172.21.15.52:0/4258964846 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f280c026040 con 0x55c94dbd7880 2024-06-02T23:17:14.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f27f37fe700 1 -- 172.21.15.52:0/4258964846 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f280401b070 con 0x55c94dbd7880 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 10 client.17052 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 10 client.17052 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 20 client.17052 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dc5e470 0x55c94dc60890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 4 client.17052 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c94db2a2d0 con 0x55c94dc5e470 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.704+0000 7f2813fff700 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dc5e470 0x55c94dc60890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f2813fff700 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dc5e470 0x55c94dc60890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9fb24289e196493e server_cookie=ed31dc0b1407019e in_seq=0 out_seq=0 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f27f37fe700 10 client.17052 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f27f37fe700 1 -- 172.21.15.52:0/4258964846 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c94db2a2d0 con 0x55c94dc5e470 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f27f37fe700 10 client.17052 handle_command_reply: tid=0 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 client.17052 shutdown 2024-06-02T23:17:14.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd7880 msgr2=0x55c94dbfa9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd7880 0x55c94dbfa9a0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x55c94dbe2940 tx=0x7f2804012df0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 shutdown_connections 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c94dbd7880 0x55c94dbfa9a0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c94dbd17a0 0x55c94dbf0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c94dc5e470 0x55c94dc60890 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 --2- 172.21.15.52:0/4258964846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c94dbd6f60 0x55c94dbf0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 >> 172.21.15.52:0/4258964846 conn(0x55c94dafe080 msgr2=0x55c94dbd7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f27f37fe700 0 client.17052 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 shutdown_connections 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 1 -- 172.21.15.52:0/4258964846 wait complete. 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.705+0000 7f28252afb80 20 client.17052 trim_cache size 0 max 16384 2024-06-02T23:17:14.709 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:17:14.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27fc0a2c20 msgr2=0x7f27fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27fc0a2c20 0x7f27fc0a50e0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f280c009550 tx=0x7f280c0091f0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 msgr2=0x7f281419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:14.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f281419ec70 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f2808006010 tx=0x7f2808009f80 comp rx=0 tx=0).stop 2024-06-02T23:17:14.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 shutdown_connections 2024-06-02T23:17:14.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27fc0a2c20 0x7f27fc0a50e0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2814107850 0x7f28141988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f281406ae80 0x7f281419e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 --2- 172.21.15.52:0/3889370780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f281406b830 0x7f281419ec70 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:14.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 >> 172.21.15.52:0/3889370780 conn(0x7f2814078400 msgr2=0x7f28140ff600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:14.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 shutdown_connections 2024-06-02T23:17:14.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:14.706+0000 7f281aa2c700 1 -- 172.21.15.52:0/3889370780 wait complete. 2024-06-02T23:17:15.508 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:17:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.619+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/1566300760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 msgr2=0x7f5fc4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.619+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/1566300760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4100d20 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009900 tx=0x7f5fb402a410 comp rx=0 tx=0).stop 2024-06-02T23:17:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/1566300760 shutdown_connections 2024-06-02T23:17:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/1566300760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc4101af0 0x7f5fc4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/1566300760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc41024b0 0x7f5fc4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/1566300760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4100d20 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/1566300760 >> 172.21.15.52:0/1566300760 conn(0x7f5fc40fbe70 msgr2=0x7f5fc40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/1566300760 shutdown_connections 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/1566300760 wait complete. 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 Processor -- start 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 -- start start 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4199e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.620+0000 7f5fc8bd5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc4101af0 0x7f5fc419a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc8bd5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc41024b0 0x7f5fc41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc8bd5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc410c400 con 0x7f5fc4100900 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc8bd5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc410c280 con 0x7f5fc4101af0 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc8bd5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc410c580 con 0x7f5fc41024b0 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4199e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4199e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38436/0 (socket says 172.21.15.52:38436) 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 -- 172.21.15.52:0/3868360291 learned_addr learned my addr 172.21.15.52:0/3868360291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc4101af0 0x7f5fc419a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc41024b0 0x7f5fc41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 -- 172.21.15.52:0/3868360291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc41024b0 msgr2=0x7f5fc41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc41024b0 0x7f5fc41a1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 -- 172.21.15.52:0/3868360291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc4101af0 msgr2=0x7f5fc419a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc4101af0 0x7f5fc419a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 -- 172.21.15.52:0/3868360291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fc41a19d0 con 0x7f5fc4100900 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc259c640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4199e10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40314e0 tx=0x7f5fb4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fb37fe640 1 -- 172.21.15.52:0/3868360291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb403f030 con 0x7f5fc4100900 2024-06-02T23:17:15.623 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fb37fe640 1 -- 172.21.15.52:0/3868360291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fb40044b0 con 0x7f5fc4100900 2024-06-02T23:17:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fc41a1c60 con 0x7f5fc4100900 2024-06-02T23:17:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.621+0000 7f5fb37fe640 1 -- 172.21.15.52:0/3868360291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4035070 con 0x7f5fc4100900 2024-06-02T23:17:15.624 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.622+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fc41a2150 con 0x7f5fc4100900 2024-06-02T23:17:15.629 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.624+0000 7f5fcb9c2740 1 Processor -- start 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.624+0000 7f5fb37fe640 1 -- 172.21.15.52:0/3868360291 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5fb4042030 con 0x7f5fc4100900 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.624+0000 7f5fcb9c2740 1 -- start start 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bff010 0x558053bff430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bffa00 0x558053c04500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c07000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558053b624f0 con 0x558053c04b70 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558053b00870 con 0x558053bff010 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558053b5e4b0 con 0x558053bffa00 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c07000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c07000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38440/0 (socket says 172.21.15.52:38440) 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2990316139 learned_addr learned my addr 172.21.15.52:0/2990316139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bff010 0x558053bff430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.630 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc259c640 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bffa00 0x558053c04500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2990316139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bffa00 msgr2=0x558053c04500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bffa00 0x558053c04500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2990316139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bff010 msgr2=0x558053bff430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bff010 0x558053bff430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2990316139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558053b1c9b0 con 0x558053c04b70 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fb37fe640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fac0a2c30 0x7f5fac0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c07000 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800ac20 tx=0x7f5fb800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64e6513e8628274f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fac0a2c30 0x7f5fac0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.625+0000 7f5fb37fe640 1 -- 172.21.15.52:0/3868360291 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5fb40ef5f0 con 0x7f5fc4100900 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fb0ff9640 1 -- 172.21.15.52:0/2990316139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb801d070 con 0x558053c04b70 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fb0ff9640 1 -- 172.21.15.52:0/2990316139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fb8018af0 con 0x558053c04b70 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fb0ff9640 1 -- 172.21.15.52:0/2990316139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8010af0 con 0x558053c04b70 2024-06-02T23:17:15.631 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2990316139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 msgr2=0x558053c07000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fac0a2c30 0x7f5fac0a50f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x558053c00260 tx=0x7f5fbc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c07000 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800ac20 tx=0x7f5fb800f790 comp rx=0 tx=0).stop 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.626+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2990316139 shutdown_connections 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bffa00 0x558053c04500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c07000 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2990316139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bff010 0x558053bff430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2990316139 >> 172.21.15.52:0/2990316139 conn(0x558053bfa680 msgr2=0x558053bfaa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2990316139 shutdown_connections 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2990316139 wait complete. 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- start start 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 Processor -- start 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c09250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.632 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c09790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c107d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558053c07b20 con 0x558053bff010 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558053aebae0 con 0x558053bffa00 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558053c07ca0 con 0x558053c04b70 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fcb9c2740 10 client.0 _ll_register_callbacks cb 0x558053b249b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c09250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c09250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38452/0 (socket says 172.21.15.52:38452) 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.627+0000 7f5fc259c640 1 -- 172.21.15.52:0/3022125397 learned_addr learned my addr 172.21.15.52:0/3022125397 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c09790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c107d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc259c640 1 -- 172.21.15.52:0/3022125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 msgr2=0x558053c107d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc259c640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c107d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc259c640 1 -- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 msgr2=0x558053c09790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc259c640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c09790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc259c640 1 -- 172.21.15.52:0/3022125397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558053876210 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fc259c640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c09250 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4042000 tx=0x7f5fb40b8150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d3b75590046a57 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/3022125397 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4049070 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/3022125397 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5fb40bc0e0 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fb8018af0 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/3022125397 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5fb40b9fc0 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fcb9c2740 10 client.17064 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558053942620 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.628+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/3022125397 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fb8018af0 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/3022125397 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5fb40c4660 con 0x558053bff010 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fcb9c2740 10 client.17064 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fcb9c2740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fcb9c2740 20 client.17064 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c15120 0x558053c17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fcb9c2740 4 client.17064 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558053b75f10 con 0x558053c15120 2024-06-02T23:17:15.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.13 MB/sec execute 144 sec latency 2105.108 ms 2024-06-02T23:17:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.629+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c15120 0x558053c17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c15120 0x558053c17590 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d57ede8da29228e2 server_cookie=fb4c23609a20b0f2 in_seq=0 out_seq=0 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fa9ffb640 10 client.17064 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/3022125397 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558053b75f10 con 0x558053c15120 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fa9ffb640 10 client.17064 handle_command_reply: tid=0 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 client.17064 shutdown 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 msgr2=0x558053c09250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c09250 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4042000 tx=0x7f5fb40b8150 comp rx=0 tx=0).stop 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 shutdown_connections 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c15120 0x558053c17590 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c107d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fa9ffb640 0 client.17064 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c09790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/3022125397 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c09250 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.630+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 >> 172.21.15.52:0/3022125397 conn(0x558053bfa680 msgr2=0x558053bf97e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.631+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 shutdown_connections 2024-06-02T23:17:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.631+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/3022125397 wait complete. 2024-06-02T23:17:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.631+0000 7f5fcb9c2740 20 client.17064 trim_cache size 0 max 16384 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 Processor -- start 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 -- start start 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bff010 0x558053c08750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c10760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c08e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41aaee0 con 0x558053c04b70 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41aca20 con 0x558053bffa00 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.646+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41ab5b0 con 0x558053bff010 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c08e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c10760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c10760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33474/0 (socket says 172.21.15.52:33474) 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c08e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38464/0 (socket says 172.21.15.52:38464) 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2010958974 learned_addr learned my addr 172.21.15.52:0/2010958974 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc259c640 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bff010 0x558053c08750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2010958974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bff010 msgr2=0x558053c08750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bff010 0x558053c08750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2010958974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 msgr2=0x558053c08e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c08e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/2010958974 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558053bfa4b0 con 0x558053bffa00 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c08e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:15.648 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c10760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800ab70 tx=0x7f5fb800a2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40d4d2fe257e4085 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fabfff640 1 -- 172.21.15.52:0/2010958974 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb801d050 con 0x558053bffa00 2024-06-02T23:17:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fabfff640 1 -- 172.21.15.52:0/2010958974 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fb801e040 con 0x558053bffa00 2024-06-02T23:17:15.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.647+0000 7f5fabfff640 1 -- 172.21.15.52:0/2010958974 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb8018a70 con 0x558053bffa00 2024-06-02T23:17:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2010958974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 msgr2=0x558053c10760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c10760 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5fb800ab70 tx=0x7f5fb800a2d0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2010958974 shutdown_connections 2024-06-02T23:17:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053c04b70 0x558053c08e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c10760 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/2010958974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053bff010 0x558053c08750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2010958974 >> 172.21.15.52:0/2010958974 conn(0x558053bfa680 msgr2=0x558053c08040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2010958974 shutdown_connections 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/2010958974 wait complete. 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.648+0000 7f5fcb9c2740 1 -- start start 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 Processor -- start 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c0d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c0d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c7de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41ac350 con 0x558053bff010 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41abc80 con 0x558053bffa00 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fc41a8ec0 con 0x558053c04b70 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fcb9c2740 10 client.0 _ll_register_callbacks cb 0x7f5fb4005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:15.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c0d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c0d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c0d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33490/0 (socket says 172.21.15.52:33490) 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/4021461831 learned_addr learned my addr 172.21.15.52:0/4021461831 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc2d9d640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c7de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/4021461831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 msgr2=0x558053c7de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c7de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/4021461831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 msgr2=0x558053c0d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c0d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 -- 172.21.15.52:0/4021461831 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558053c109e0 con 0x558053bffa00 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc259c640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c0d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fc1d9b640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c0d910 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8000c00 tx=0x7f5fb8005b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6224deeebcbe295b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.649+0000 7f5fa9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:17:15.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/4021461831 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fb803c060 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/4021461831 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5fb80189f0 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fb801e040 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/4021461831 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5fb801d040 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 10 client.26603 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558053c7f3f0 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/4021461831 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fb801e040 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/4021461831 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5fb802d070 con 0x558053bffa00 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 10 client.26603 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 20 client.26603 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c7fb20 0x558053c81f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 4 client.26603 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.650+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558053b6fc20 con 0x558053c7fb20 2024-06-02T23:17:15.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.651+0000 7f5fc259c640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c7fb20 0x558053c81f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.651+0000 7f5fc259c640 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c7fb20 0x558053c81f40 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21631e375f1d8671 server_cookie=e7b3b4876f574579 in_seq=0 out_seq=0 2024-06-02T23:17:15.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.651+0000 7f5fa9ffb640 10 client.26603 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.670+0000 7f5fa9ffb640 1 -- 172.21.15.52:0/4021461831 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558053b6fc20 con 0x558053c7fb20 2024-06-02T23:17:15.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.670+0000 7f5fa9ffb640 10 client.26603 handle_command_reply: tid=0 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.670+0000 7f5fcb9c2740 1 client.26603 shutdown 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 msgr2=0x558053c0d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c0d910 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f5fb8000c00 tx=0x7f5fb8005b60 comp rx=0 tx=0).stop 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 shutdown_connections 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558053c7fb20 0x558053c81f40 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558053c04b70 0x558053c7de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558053bffa00 0x558053c0d910 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fa9ffb640 0 client.26603 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 --2- 172.21.15.52:0/4021461831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558053bff010 0x558053c0d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 >> 172.21.15.52:0/4021461831 conn(0x558053bfa680 msgr2=0x558053bf8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 shutdown_connections 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 1 -- 172.21.15.52:0/4021461831 wait complete. 2024-06-02T23:17:15.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fcb9c2740 20 client.26603 trim_cache size 0 max 16384 2024-06-02T23:17:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:17:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:17:15.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fac0a2c30 msgr2=0x7f5fac0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fac0a2c30 0x7f5fac0a50f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x558053c00260 tx=0x7f5fbc009500 comp rx=0 tx=0).stop 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 msgr2=0x7f5fc4199e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4199e10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40314e0 tx=0x7f5fb4031a30 comp rx=0 tx=0).stop 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 shutdown_connections 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fac0a2c30 0x7f5fac0a50f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fc41024b0 0x7f5fc41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc4101af0 0x7f5fc419a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 --2- 172.21.15.52:0/3868360291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fc4100900 0x7f5fc4199e10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.671+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 >> 172.21.15.52:0/3868360291 conn(0x7f5fc40fbe70 msgr2=0x7f5fc40fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.672+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 shutdown_connections 2024-06-02T23:17:15.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:15.672+0000 7f5fc8bd5640 1 -- 172.21.15.52:0/3868360291 wait complete. 2024-06-02T23:17:15.773 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:15 smithi174 ceph-mon[35994]: pgmap v592: 97 pgs: 97 active+clean; 58 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-02T23:17:15.895 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:15 smithi052 ceph-mon[32340]: pgmap v592: 97 pgs: 97 active+clean; 58 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-02T23:17:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:15 smithi123 ceph-mon[35977]: pgmap v592: 97 pgs: 97 active+clean; 58 MiB data, 895 MiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.1 MiB/s wr, 150 op/s 2024-06-02T23:17:16.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.11 MB/sec execute 145 sec latency 3105.193 ms 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 -- 172.21.15.52:0/2231487672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec1066b0 msgr2=0x7f36ec10e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 --2- 172.21.15.52:0/2231487672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec1066b0 0x7f36ec10e280 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f36e400b0d0 tx=0x7f36e400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 -- 172.21.15.52:0/2231487672 shutdown_connections 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 --2- 172.21.15.52:0/2231487672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec1066b0 0x7f36ec10e280 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 --2- 172.21.15.52:0/2231487672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec101620 0x7f36ec101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 --2- 172.21.15.52:0/2231487672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec101fd0 0x7f36ec105fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 -- 172.21.15.52:0/2231487672 >> 172.21.15.52:0/2231487672 conn(0x7f36ec078340 msgr2=0x7f36ec078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 -- 172.21.15.52:0/2231487672 shutdown_connections 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.500+0000 7f36f4189700 1 -- 172.21.15.52:0/2231487672 wait complete. 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 Processor -- start 2024-06-02T23:17:17.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 -- start start 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 0x7f36ec19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec101fd0 0x7f36ec19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec1066b0 0x7f36ec19f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec111550 con 0x7f36ec101620 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1113d0 con 0x7f36ec1066b0 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f4189700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1116d0 con 0x7f36ec101fd0 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 0x7f36ec19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f1724700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec101fd0 0x7f36ec19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 0x7f36ec19ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38498/0 (socket says 172.21.15.52:38498) 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f1f25700 1 -- 172.21.15.52:0/1009989884 learned_addr learned my addr 172.21.15.52:0/1009989884 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.501+0000 7f36f2726700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec1066b0 0x7f36ec19f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f1f25700 1 -- 172.21.15.52:0/1009989884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec101fd0 msgr2=0x7f36ec19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f1f25700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec101fd0 0x7f36ec19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f1f25700 1 -- 172.21.15.52:0/1009989884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec1066b0 msgr2=0x7f36ec19f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f1f25700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec1066b0 0x7f36ec19f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f1f25700 1 -- 172.21.15.52:0/1009989884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ec19fd30 con 0x7f36ec101620 2024-06-02T23:17:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f1f25700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 0x7f36ec19ad20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c920 tx=0x7f36e800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:17.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36e2ffd700 1 -- 172.21.15.52:0/1009989884 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8004a30 con 0x7f36ec101620 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ec1a0020 con 0x7f36ec101620 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36e2ffd700 1 -- 172.21.15.52:0/1009989884 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e80054d0 con 0x7f36ec101620 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36e2ffd700 1 -- 172.21.15.52:0/1009989884 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e8007c30 con 0x7f36ec101620 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.502+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ec1a78a0 con 0x7f36ec101620 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.505+0000 7f36e2ffd700 1 -- 172.21.15.52:0/1009989884 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36e8004bd0 con 0x7f36ec101620 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.505+0000 7f36e2ffd700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36d80a2c20 0x7f36d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.505+0000 7f36e2ffd700 1 -- 172.21.15.52:0/1009989884 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36e80cebb0 con 0x7f36ec101620 2024-06-02T23:17:17.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.505+0000 7f36f1724700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36d80a2c20 0x7f36d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 Processor -- start 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36f1724700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36d80a2c20 0x7f36d80a50e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f36ec06bb70 tx=0x7f36dc0073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 -- start start 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55661f507810 con 0x55661f5b5f20 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55661f4af720 con 0x55661f5bb640 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55661f5037d0 con 0x55661f5bbf60 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36f2726700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36f2726700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5b6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38508/0 (socket says 172.21.15.52:38508) 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36f2726700 1 -- 172.21.15.52:0/4087517710 learned_addr learned my addr 172.21.15.52:0/4087517710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:17.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.506+0000 7f36f1724700 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36f1f25700 1 -- 172.21.15.52:0/4087517710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 msgr2=0x55661f5b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36f1f25700 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5b59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36f1f25700 1 -- 172.21.15.52:0/4087517710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 msgr2=0x55661f5b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36f1f25700 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36f1f25700 1 -- 172.21.15.52:0/4087517710 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55661f4a5f50 con 0x55661f5bb640 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36f1f25700 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5bba20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f36e80cc6e0 tx=0x7f36e809bbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac45d49c87fe48a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36d7fff700 1 -- 172.21.15.52:0/4087517710 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e801d070 con 0x55661f5bb640 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36d7fff700 1 -- 172.21.15.52:0/4087517710 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e80a40b0 con 0x55661f5bb640 2024-06-02T23:17:17.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.507+0000 7f36d7fff700 1 -- 172.21.15.52:0/4087517710 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e809b6a0 con 0x55661f5bb640 2024-06-02T23:17:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 -- 172.21.15.52:0/4087517710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 msgr2=0x55661f5bba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5bba20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f36e80cc6e0 tx=0x7f36e809bbc0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 -- 172.21.15.52:0/4087517710 shutdown_connections 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5bba20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/4087517710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 -- 172.21.15.52:0/4087517710 >> 172.21.15.52:0/4087517710 conn(0x55661f4e0cc0 msgr2=0x55661f5b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 -- 172.21.15.52:0/4087517710 shutdown_connections 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 -- 172.21.15.52:0/4087517710 wait complete. 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.508+0000 7f36fea0cb80 1 -- start start 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 Processor -- start 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5c1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 0x55661f5befd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bbf60 0x55661f5bf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55661f5c29d0 con 0x55661f5bb640 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55661f4a7bf0 con 0x55661f5bbf60 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55661f5c2b50 con 0x55661f5b5f20 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36fea0cb80 10 client.0 _ll_register_callbacks cb 0x55661f4a1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36f1724700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 0x55661f5befd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5c1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36f1724700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 0x55661f5befd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38514/0 (socket says 172.21.15.52:38514) 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36f1724700 1 -- 172.21.15.52:0/2475553092 learned_addr learned my addr 172.21.15.52:0/2475553092 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.509+0000 7f36f2726700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bbf60 0x55661f5bf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1724700 1 -- 172.21.15.52:0/2475553092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 msgr2=0x55661f5c1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1724700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5c1cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1724700 1 -- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bbf60 msgr2=0x55661f5bf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1724700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bbf60 0x55661f5bf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1724700 1 -- 172.21.15.52:0/2475553092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661f4568d0 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1f25700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36f1724700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 0x55661f5befd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f36dc010f70 tx=0x7f36dc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2120bef9556bbc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36d57fa700 1 -- 172.21.15.52:0/2475553092 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc028470 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e80a40b0 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36d57fa700 1 -- 172.21.15.52:0/2475553092 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36dc028610 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36d57fa700 1 -- 172.21.15.52:0/2475553092 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36dc028810 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36fea0cb80 10 client.17082 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55661f5cd2e0 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.510+0000 7f36d57fa700 1 -- 172.21.15.52:0/2475553092 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e80a40b0 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36d57fa700 1 -- 172.21.15.52:0/2475553092 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f36dc01faa0 con 0x55661f5bb640 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36fea0cb80 10 client.17082 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:17.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36fea0cb80 10 client.17082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36fea0cb80 20 client.17082 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f5cf920 0x55661f5d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36fea0cb80 4 client.17082 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55661f514d10 con 0x55661f5cf920 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36f1f25700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f5cf920 0x55661f5d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36f1f25700 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f5cf920 0x55661f5d1d10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7d1a4a9d9dca343e server_cookie=6aa6c9b0ad6d5429 in_seq=0 out_seq=0 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.511+0000 7f36d57fa700 10 client.17082 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36d57fa700 1 -- 172.21.15.52:0/2475553092 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55661f514d10 con 0x55661f5cf920 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36d57fa700 10 client.17082 handle_command_reply: tid=0 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 client.17082 shutdown 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 msgr2=0x55661f5befd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 0x55661f5befd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f36dc010f70 tx=0x7f36dc0047c0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 shutdown_connections 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f5cf920 0x55661f5d1d10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bbf60 0x55661f5bf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5c1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2475553092 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bb640 0x55661f5befd0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36d57fa700 0 client.17082 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 >> 172.21.15.52:0/2475553092 conn(0x55661f4e0cc0 msgr2=0x55661f5b05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 shutdown_connections 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.512+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2475553092 wait complete. 2024-06-02T23:17:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.513+0000 7f36fea0cb80 20 client.17082 trim_cache size 0 max 16384 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.528+0000 7f36fea0cb80 1 Processor -- start 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.528+0000 7f36fea0cb80 1 -- start start 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.528+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.528+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5b9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.528+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bbf60 0x55661f5d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1aed30 con 0x55661f5bbf60 2024-06-02T23:17:17.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b0870 con 0x55661f5bb640 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1af400 con 0x55661f5b5f20 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1724700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5b9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5b59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43248/0 (socket says 172.21.15.52:43248) 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 -- 172.21.15.52:0/2963338865 learned_addr learned my addr 172.21.15.52:0/2963338865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f2726700 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bbf60 0x55661f5d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 -- 172.21.15.52:0/2963338865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 msgr2=0x55661f5b9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5b9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 -- 172.21.15.52:0/2963338865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bbf60 msgr2=0x55661f5d75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bbf60 0x55661f5d75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 -- 172.21.15.52:0/2963338865 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55661f5bd560 con 0x55661f5b5f20 2024-06-02T23:17:17.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.529+0000 7f36f1f25700 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5b59e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36e8097e10 tx=0x7f36e8097040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e35b4ac96fcdcc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:17.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.533+0000 7f36d77fe700 1 -- 172.21.15.52:0/2963338865 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e80b9070 con 0x55661f5b5f20 2024-06-02T23:17:17.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.533+0000 7f36d77fe700 1 -- 172.21.15.52:0/2963338865 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e8005680 con 0x55661f5b5f20 2024-06-02T23:17:17.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.533+0000 7f36d77fe700 1 -- 172.21.15.52:0/2963338865 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e801d040 con 0x55661f5b5f20 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.533+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2963338865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 msgr2=0x55661f5b59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.533+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5b59e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f36e8097e10 tx=0x7f36e8097040 comp rx=0 tx=0).stop 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2963338865 shutdown_connections 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5bbf60 0x55661f5d75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5b5f20 0x55661f5b59e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/2963338865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5b9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2963338865 >> 172.21.15.52:0/2963338865 conn(0x55661f4e0cc0 msgr2=0x55661f5bd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:17.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2963338865 shutdown_connections 2024-06-02T23:17:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- 172.21.15.52:0/2963338865 wait complete. 2024-06-02T23:17:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- start start 2024-06-02T23:17:17.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 Processor -- start 2024-06-02T23:17:17.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5d6d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1b01a0 con 0x55661f5b5f20 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1afad0 con 0x55661f5bb640 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.534+0000 7f36fea0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36ec1accd0 con 0x55661f5bbf60 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36fea0cb80 10 client.0 _ll_register_callbacks cb 0x55661f4a1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5c81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38524/0 (socket says 172.21.15.52:38524) 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 -- 172.21.15.52:0/703853109 learned_addr learned my addr 172.21.15.52:0/703853109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1724700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f2726700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5d6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 -- 172.21.15.52:0/703853109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 msgr2=0x55661f5d6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5d6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 -- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 msgr2=0x55661f5c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 -- 172.21.15.52:0/703853109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55661f5d7540 con 0x55661f5b5f20 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36f1f25700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5c81b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c8f0 tx=0x7f36e800a470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a6407fb711e2d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36d57fa700 1 -- 172.21.15.52:0/703853109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36e801d070 con 0x55661f5b5f20 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36d57fa700 1 -- 172.21.15.52:0/703853109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36e80b4240 con 0x55661f5b5f20 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36e8005680 con 0x55661f5b5f20 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36d57fa700 1 -- 172.21.15.52:0/703853109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36e80b9050 con 0x55661f5b5f20 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36fea0cb80 10 client.17088 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.535+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55661f63a200 con 0x55661f5b5f20 2024-06-02T23:17:17.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36d57fa700 1 -- 172.21.15.52:0/703853109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36e8005680 con 0x55661f5b5f20 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36d57fa700 1 -- 172.21.15.52:0/703853109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f36e80a3c20 con 0x55661f5b5f20 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36fea0cb80 10 client.17088 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36fea0cb80 10 client.17088 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36fea0cb80 20 client.17088 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f63a3d0 0x55661f63c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36fea0cb80 4 client.17088 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55661f50f290 con 0x55661f63a3d0 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.536+0000 7f36f1724700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f63a3d0 0x55661f63c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36f1724700 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f63a3d0 0x55661f63c7c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c0bbde2dd90cd82 server_cookie=16705e7a7cd45794 in_seq=0 out_seq=0 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36d57fa700 10 client.17088 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36d57fa700 1 -- 172.21.15.52:0/703853109 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55661f50f290 con 0x55661f63a3d0 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36d57fa700 10 client.17088 handle_command_reply: tid=0 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 client.17088 shutdown 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 msgr2=0x55661f5c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5c81b0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c8f0 tx=0x7f36e800a470 comp rx=0 tx=0).stop 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 shutdown_connections 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55661f63a3d0 0x55661f63c7c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55661f5bbf60 0x55661f5d6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55661f5b5f20 0x55661f5c81b0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 --2- 172.21.15.52:0/703853109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55661f5bb640 0x55661f5c86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36d57fa700 0 client.17088 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 >> 172.21.15.52:0/703853109 conn(0x55661f4e0cc0 msgr2=0x55661f5a9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 shutdown_connections 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 1 -- 172.21.15.52:0/703853109 wait complete. 2024-06-02T23:17:17.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.537+0000 7f36fea0cb80 20 client.17088 trim_cache size 0 max 16384 2024-06-02T23:17:17.541 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:17:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36d80a2c20 msgr2=0x7f36d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36d80a2c20 0x7f36d80a50e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f36ec06bb70 tx=0x7f36dc0073d0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 msgr2=0x7f36ec19ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 0x7f36ec19ad20 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f36e800c920 tx=0x7f36e800cc30 comp rx=0 tx=0).stop 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 shutdown_connections 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36d80a2c20 0x7f36d80a50e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36ec1066b0 0x7f36ec19f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36ec101620 0x7f36ec19ad20 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 --2- 172.21.15.52:0/1009989884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36ec101fd0 0x7f36ec19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 >> 172.21.15.52:0/1009989884 conn(0x7f36ec078340 msgr2=0x7f36ec10ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 shutdown_connections 2024-06-02T23:17:17.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:17.538+0000 7f36f4189700 1 -- 172.21.15.52:0/1009989884 wait complete. 2024-06-02T23:17:17.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.10 MB/sec execute 146 sec latency 4105.278 ms 2024-06-02T23:17:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:17 smithi174 ceph-mon[35994]: pgmap v593: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-02T23:17:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:17 smithi052 ceph-mon[32340]: pgmap v593: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-02T23:17:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:17 smithi123 ceph-mon[35977]: pgmap v593: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 249 KiB/s rd, 1.2 MiB/s wr, 155 op/s 2024-06-02T23:17:18.375 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:17:18.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.08 MB/sec execute 147 sec latency 5105.321 ms 2024-06-02T23:17:18.631 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:19.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.07 MB/sec execute 148 sec latency 6105.433 ms 2024-06-02T23:17:19.809 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:19 smithi123 ceph-mon[35977]: pgmap v594: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-06-02T23:17:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:19 smithi174 ceph-mon[35994]: pgmap v594: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-06-02T23:17:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:19 smithi052 ceph-mon[32340]: pgmap v594: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 243 KiB/s rd, 1.0 MiB/s wr, 146 op/s 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 -- 172.21.15.52:0/3821927226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620113660 msgr2=0x7f4620115a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 --2- 172.21.15.52:0/3821927226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620113660 0x7f4620115a50 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f461800b0a0 tx=0x7f461800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 -- 172.21.15.52:0/3821927226 shutdown_connections 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 --2- 172.21.15.52:0/3821927226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4620113660 0x7f4620115a50 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 --2- 172.21.15.52:0/3821927226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46200746c0 0x7f4620074aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 --2- 172.21.15.52:0/3821927226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4620074fe0 0x7f4620113120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 -- 172.21.15.52:0/3821927226 >> 172.21.15.52:0/3821927226 conn(0x7f46200795f0 msgr2=0x7f4620079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 -- 172.21.15.52:0/3821927226 shutdown_connections 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 -- 172.21.15.52:0/3821927226 wait complete. 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 Processor -- start 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.459+0000 7f4627aa0700 1 -- start start 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f4627aa0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 0x7f462019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f4627aa0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4620074fe0 0x7f462019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f4627aa0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4620113660 0x7f46201a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f4627aa0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4620118c30 con 0x7f46200746c0 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f4627aa0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4620118ab0 con 0x7f4620113660 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f4627aa0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4620118db0 con 0x7f4620074fe0 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 0x7f462019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 0x7f462019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38552/0 (socket says 172.21.15.52:38552) 2024-06-02T23:17:20.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 -- 172.21.15.52:0/1446587377 learned_addr learned my addr 172.21.15.52:0/1446587377 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462603d700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4620113660 0x7f46201a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462503b700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4620074fe0 0x7f462019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 -- 172.21.15.52:0/1446587377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4620074fe0 msgr2=0x7f462019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4620074fe0 0x7f462019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 -- 172.21.15.52:0/1446587377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4620113660 msgr2=0x7f46201a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4620113660 0x7f46201a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 -- 172.21.15.52:0/1446587377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46201a40f0 con 0x7f46200746c0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.460+0000 7f462583c700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 0x7f462019f0e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d950 tx=0x7f461c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.461+0000 7f4616ffd700 1 -- 172.21.15.52:0/1446587377 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c016070 con 0x7f46200746c0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.461+0000 7f4616ffd700 1 -- 172.21.15.52:0/1446587377 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c009df0 con 0x7f46200746c0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.461+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46201a43e0 con 0x7f46200746c0 2024-06-02T23:17:20.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.461+0000 7f4616ffd700 1 -- 172.21.15.52:0/1446587377 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c00f3c0 con 0x7f46200746c0 2024-06-02T23:17:20.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.461+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46201abc60 con 0x7f46200746c0 2024-06-02T23:17:20.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.463+0000 7f4616ffd700 1 -- 172.21.15.52:0/1446587377 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f461c019070 con 0x7f46200746c0 2024-06-02T23:17:20.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.464+0000 7f4616ffd700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f460c0a2c20 0x7f460c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.464+0000 7f462503b700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f460c0a2c20 0x7f460c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.464+0000 7f4616ffd700 1 -- 172.21.15.52:0/1446587377 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f461c0cebc0 con 0x7f46200746c0 2024-06-02T23:17:20.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.464+0000 7f4632323b80 1 Processor -- start 2024-06-02T23:17:20.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 -- start start 2024-06-02T23:17:20.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f460c0a2c20 0x7f460c0a50e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f462010ffe0 tx=0x7f461000d4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:20.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d22340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d1c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d1cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fc83c0ef60 con 0x55fc83d22880 2024-06-02T23:17:20.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fc83c6d850 con 0x55fc83d21f60 2024-06-02T23:17:20.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f4632323b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fc83bd5460 con 0x55fc83d1c7a0 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d1c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d1c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38558/0 (socket says 172.21.15.52:38558) 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 -- 172.21.15.52:0/1443377218 learned_addr learned my addr 172.21.15.52:0/1443377218 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462583c700 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d22340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462603d700 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d1cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 -- 172.21.15.52:0/1443377218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 msgr2=0x55fc83d1cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d1cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 -- 172.21.15.52:0/1443377218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 msgr2=0x55fc83d22340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d22340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 -- 172.21.15.52:0/1443377218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc83bbc8d0 con 0x55fc83d22880 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462603d700 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d1cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.465+0000 7f462503b700 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d1c260 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f461000fc20 tx=0x7f4610012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb39370245c45b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f460bfff700 1 -- 172.21.15.52:0/1443377218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4610022070 con 0x55fc83d22880 2024-06-02T23:17:20.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f460bfff700 1 -- 172.21.15.52:0/1443377218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461001e450 con 0x55fc83d22880 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f460bfff700 1 -- 172.21.15.52:0/1443377218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461001d610 con 0x55fc83d22880 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f4632323b80 1 -- 172.21.15.52:0/1443377218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 msgr2=0x55fc83d1c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f4632323b80 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d1c260 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f461000fc20 tx=0x7f4610012000 comp rx=0 tx=0).stop 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f4632323b80 1 -- 172.21.15.52:0/1443377218 shutdown_connections 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f4632323b80 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d1cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.466+0000 7f4632323b80 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d22340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 --2- 172.21.15.52:0/1443377218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d1c260 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 -- 172.21.15.52:0/1443377218 >> 172.21.15.52:0/1443377218 conn(0x55fc83c49080 msgr2=0x55fc83d17220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 -- 172.21.15.52:0/1443377218 shutdown_connections 2024-06-02T23:17:20.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 -- 172.21.15.52:0/1443377218 wait complete. 2024-06-02T23:17:20.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 -- start start 2024-06-02T23:17:20.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 Processor -- start 2024-06-02T23:17:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d28590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.467+0000 7f4632323b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d25890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f4632323b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d25dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f4632323b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fc83c0dfc0 con 0x55fc83d22880 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f4632323b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fc83c69810 con 0x55fc83d21f60 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f4632323b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fc83d29260 con 0x55fc83d1c7a0 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f4632323b80 10 client.0 _ll_register_callbacks cb 0x55fc83c035b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d25dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d25dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38562/0 (socket says 172.21.15.52:38562) 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 -- 172.21.15.52:0/691981041 learned_addr learned my addr 172.21.15.52:0/691981041 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462583c700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d28590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462503b700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d25890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 -- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 msgr2=0x55fc83d28590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d28590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 -- 172.21.15.52:0/691981041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 msgr2=0x55fc83d25890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d25890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 -- 172.21.15.52:0/691981041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc83c33750 con 0x55fc83d22880 2024-06-02T23:17:20.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f462603d700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d25dd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4618004cc0 tx=0x7f4618004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e052df9b40853d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f46097fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f46097fa700 1 -- 172.21.15.52:0/691981041 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46180078f0 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f46097fa700 1 -- 172.21.15.52:0/691981041 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4618007a90 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.468+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461001e450 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f46097fa700 1 -- 172.21.15.52:0/691981041 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4618009cd0 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 10 client.17100 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc83d339b0 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f46097fa700 1 -- 172.21.15.52:0/691981041 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461001e450 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f46097fa700 1 -- 172.21.15.52:0/691981041 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4618011070 con 0x55fc83d22880 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 10 client.17100 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 10 client.17100 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 20 client.17100 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83d36160 0x55fc83d38580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 4 client.17100 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc83c7ad50 con 0x55fc83d36160 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.469+0000 7f462583c700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83d36160 0x55fc83d38580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.470+0000 7f462583c700 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83d36160 0x55fc83d38580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e65c6f0ae38e36f6 server_cookie=852c9fbdd4a6796c in_seq=0 out_seq=0 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.470+0000 7f46097fa700 10 client.17100 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.470+0000 7f46097fa700 1 -- 172.21.15.52:0/691981041 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc83c7ad50 con 0x55fc83d36160 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f46097fa700 10 client.17100 handle_command_reply: tid=0 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 client.17100 shutdown 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 msgr2=0x55fc83d25dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d25dd0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f4618004cc0 tx=0x7f4618004cf0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 shutdown_connections 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83d36160 0x55fc83d38580 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d22880 0x55fc83d25dd0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d1c7a0 0x55fc83d28590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f46097fa700 0 client.17100 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:17:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 --2- 172.21.15.52:0/691981041 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d25890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 >> 172.21.15.52:0/691981041 conn(0x55fc83c49080 msgr2=0x55fc83d16700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 shutdown_connections 2024-06-02T23:17:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 1 -- 172.21.15.52:0/691981041 wait complete. 2024-06-02T23:17:20.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.471+0000 7f4632323b80 20 client.17100 trim_cache size 0 max 16384 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.486+0000 7f4632323b80 1 Processor -- start 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.486+0000 7f4632323b80 1 -- start start 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f4632323b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d25e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f4632323b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f4632323b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d16e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f4632323b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46201b3120 con 0x55fc83d1c7a0 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f4632323b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46201b4c60 con 0x55fc83d21f60 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f4632323b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46201b37f0 con 0x55fc83d22880 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d25e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d25e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38576/0 (socket says 172.21.15.52:38576) 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 -- 172.21.15.52:0/1941384417 learned_addr learned my addr 172.21.15.52:0/1941384417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:20.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 -- 172.21.15.52:0/1941384417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 msgr2=0x55fc83d16e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d16e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 -- 172.21.15.52:0/1941384417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 msgr2=0x55fc83d23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462503b700 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d23430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 -- 172.21.15.52:0/1941384417 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc83d1c260 con 0x55fc83d1c7a0 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f462583c700 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d25e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f461c097e20 tx=0x7f461c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9af2dd05120b52 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f460b7fe700 1 -- 172.21.15.52:0/1941384417 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c082070 con 0x55fc83d1c7a0 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f460b7fe700 1 -- 172.21.15.52:0/1941384417 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c084070 con 0x55fc83d1c7a0 2024-06-02T23:17:20.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.487+0000 7f460b7fe700 1 -- 172.21.15.52:0/1941384417 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c020040 con 0x55fc83d1c7a0 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 -- 172.21.15.52:0/1941384417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 msgr2=0x55fc83d25e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d25e20 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f461c097e20 tx=0x7f461c0ce040 comp rx=0 tx=0).stop 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 -- 172.21.15.52:0/1941384417 shutdown_connections 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d16e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d25e20 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 --2- 172.21.15.52:0/1941384417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 -- 172.21.15.52:0/1941384417 >> 172.21.15.52:0/1941384417 conn(0x55fc83c49080 msgr2=0x55fc83d22d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 -- 172.21.15.52:0/1941384417 shutdown_connections 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 -- 172.21.15.52:0/1941384417 wait complete. 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 -- start start 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.488+0000 7f4632323b80 1 Processor -- start 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d2ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d3d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d3dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46201b4590 con 0x55fc83d1c7a0 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46201b3ec0 con 0x55fc83d21f60 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46201b10c0 con 0x55fc83d22880 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d2ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f4632323b80 10 client.0 _ll_register_callbacks cb 0x55fc83c035b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:20.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d2ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38592/0 (socket says 172.21.15.52:38592) 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 -- 172.21.15.52:0/2959382390 learned_addr learned my addr 172.21.15.52:0/2959382390 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462603d700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d3dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462503b700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d3d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 -- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 msgr2=0x55fc83d3dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d3dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 -- 172.21.15.52:0/2959382390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 msgr2=0x55fc83d3d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d3d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462583c700 1 -- 172.21.15.52:0/2959382390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc83d3e460 con 0x55fc83d1c7a0 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.489+0000 7f462603d700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f462583c700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d2ec20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f461c09b150 tx=0x7f461c09b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1fdfb0bcb053be server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f46097fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:20.494 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f46097fa700 1 -- 172.21.15.52:0/2959382390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f461c082070 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f46097fa700 1 -- 172.21.15.52:0/2959382390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f461c09b6b0 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f461c084070 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f46097fa700 1 -- 172.21.15.52:0/2959382390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f461c020070 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 10 client.17106 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc83da0d70 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f46097fa700 1 -- 172.21.15.52:0/2959382390 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f461c084070 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f46097fa700 1 -- 172.21.15.52:0/2959382390 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f461c0ccbb0 con 0x55fc83d1c7a0 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 10 client.17106 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 10 client.17106 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 20 client.17106 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83da1440 0x55fc83da3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:20.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 4 client.17106 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.490+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fc83c752d0 con 0x55fc83da1440 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f462503b700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83da1440 0x55fc83da3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f462503b700 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83da1440 0x55fc83da3860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ff3e31339b2bacfc server_cookie=d5c0a805dfaf0f7b in_seq=0 out_seq=0 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f46097fa700 10 client.17106 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f46097fa700 1 -- 172.21.15.52:0/2959382390 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fc83c752d0 con 0x55fc83da1440 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f46097fa700 10 client.17106 handle_command_reply: tid=0 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 client.17106 shutdown 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 msgr2=0x55fc83d2ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d2ec20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f461c09b150 tx=0x7f461c09b520 comp rx=0 tx=0).stop 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 shutdown_connections 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fc83da1440 0x55fc83da3860 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc83d22880 0x55fc83d3dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc83d1c7a0 0x55fc83d2ec20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 --2- 172.21.15.52:0/2959382390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc83d21f60 0x55fc83d3d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f46097fa700 0 client.17106 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.491+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 >> 172.21.15.52:0/2959382390 conn(0x55fc83c49080 msgr2=0x55fc83d270c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 shutdown_connections 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4632323b80 1 -- 172.21.15.52:0/2959382390 wait complete. 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4632323b80 20 client.17106 trim_cache size 0 max 16384 2024-06-02T23:17:20.497 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f460c0a2c20 msgr2=0x7f460c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f460c0a2c20 0x7f460c0a50e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f462010ffe0 tx=0x7f461000d4b0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 msgr2=0x7f462019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 0x7f462019f0e0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f461c00d950 tx=0x7f461c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 shutdown_connections 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f460c0a2c20 0x7f460c0a50e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4620113660 0x7f46201a39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46200746c0 0x7f462019f0e0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 --2- 172.21.15.52:0/1446587377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4620074fe0 0x7f462019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:20.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.492+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 >> 172.21.15.52:0/1446587377 conn(0x7f46200795f0 msgr2=0x7f4620112490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:20.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.493+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 shutdown_connections 2024-06-02T23:17:20.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:20.493+0000 7f4627aa0700 1 -- 172.21.15.52:0/1446587377 wait complete. 2024-06-02T23:17:20.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.06 MB/sec execute 149 sec latency 7105.553 ms 2024-06-02T23:17:21.323 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:17:21.575 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:21.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.04 MB/sec execute 150 sec latency 8105.674 ms 2024-06-02T23:17:21.841 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:21 smithi052 ceph-mon[32340]: pgmap v595: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 881 KiB/s wr, 112 op/s 2024-06-02T23:17:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:21 smithi174 ceph-mon[35994]: pgmap v595: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 881 KiB/s wr, 112 op/s 2024-06-02T23:17:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:21 smithi123 ceph-mon[35977]: pgmap v595: 97 pgs: 97 active+clean; 56 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 881 KiB/s wr, 112 op/s 2024-06-02T23:17:22.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.03 MB/sec execute 151 sec latency 9105.732 ms 2024-06-02T23:17:23.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 -- 172.21.15.52:0/1279411455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280100e50 msgr2=0x7f32801012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 --2- 172.21.15.52:0/1279411455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280100e50 0x7f32801012d0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f3270009990 tx=0x7f3270009ca0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 -- 172.21.15.52:0/1279411455 shutdown_connections 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 --2- 172.21.15.52:0/1279411455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3280101810 0x7f328010f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 --2- 172.21.15.52:0/1279411455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280107570 0x7f3280107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 --2- 172.21.15.52:0/1279411455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280100e50 0x7f32801012d0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 -- 172.21.15.52:0/1279411455 >> 172.21.15.52:0/1279411455 conn(0x7f32800766b0 msgr2=0x7f3280076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 -- 172.21.15.52:0/1279411455 shutdown_connections 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 -- 172.21.15.52:0/1279411455 wait complete. 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 Processor -- start 2024-06-02T23:17:23.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.462+0000 7f32876b6700 1 -- start start 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f32876b6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3280100e50 0x7f328019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f32876b6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280101810 0x7f328019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f32876b6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 0x7f3280198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f32876b6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3280112680 con 0x7f3280107570 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f32876b6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3280112500 con 0x7f3280100e50 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f32876b6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3280112800 con 0x7f3280101810 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 0x7f3280198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 0x7f3280198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38616/0 (socket says 172.21.15.52:38616) 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 -- 172.21.15.52:0/829044657 learned_addr learned my addr 172.21.15.52:0/829044657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:23.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3284c51700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280101810 0x7f328019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285452700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3280100e50 0x7f328019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 -- 172.21.15.52:0/829044657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280101810 msgr2=0x7f328019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280101810 0x7f328019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 -- 172.21.15.52:0/829044657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3280100e50 msgr2=0x7f328019e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3280100e50 0x7f328019e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 -- 172.21.15.52:0/829044657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3280199060 con 0x7f3280107570 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.463+0000 7f3285c53700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 0x7f3280198900 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3278007d70 tx=0x7f327800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.464+0000 7f32767fc700 1 -- 172.21.15.52:0/829044657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3278016020 con 0x7f3280107570 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.464+0000 7f32767fc700 1 -- 172.21.15.52:0/829044657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327800ac90 con 0x7f3280107570 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.464+0000 7f32767fc700 1 -- 172.21.15.52:0/829044657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32780115e0 con 0x7f3280107570 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.464+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3280199350 con 0x7f3280107570 2024-06-02T23:17:23.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.464+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32801a5610 con 0x7f3280107570 2024-06-02T23:17:23.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.466+0000 7f32767fc700 1 -- 172.21.15.52:0/829044657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3278019020 con 0x7f3280107570 2024-06-02T23:17:23.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.467+0000 7f32767fc700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f326c0a2ca0 0x7f326c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.467+0000 7f3285452700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f326c0a2ca0 0x7f326c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.467+0000 7f32767fc700 1 -- 172.21.15.52:0/829044657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32780cef70 con 0x7f3280107570 2024-06-02T23:17:23.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.467+0000 7f3285452700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f326c0a2ca0 0x7f326c0a5160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3280199a40 tx=0x7f327c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:23.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 Processor -- start 2024-06-02T23:17:23.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 -- start start 2024-06-02T23:17:23.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec646813b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec646856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec64686090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ec645ce810 con 0x55ec64680fd0 2024-06-02T23:17:23.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec64576720 con 0x55ec64685c30 2024-06-02T23:17:23.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3291f39b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec645ca7d0 con 0x55ec646818f0 2024-06-02T23:17:23.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3285452700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec646813b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3285452700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec646813b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38618/0 (socket says 172.21.15.52:38618) 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3285452700 1 -- 172.21.15.52:0/2682750945 learned_addr learned my addr 172.21.15.52:0/2682750945 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3284c51700 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec646856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3285c53700 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec64686090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3285452700 1 -- 172.21.15.52:0/2682750945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 msgr2=0x55ec646856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.468+0000 7f3285452700 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec646856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f3285452700 1 -- 172.21.15.52:0/2682750945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 msgr2=0x55ec64686090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f3285452700 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec64686090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f3285452700 1 -- 172.21.15.52:0/2682750945 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6456cf50 con 0x55ec64680fd0 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f3284c51700 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec646856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f3285452700 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec646813b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f327c00b640 tx=0x7f327c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55f2970e7b6d7473 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f326b7fe700 1 -- 172.21.15.52:0/2682750945 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c00be60 con 0x55ec64680fd0 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f326b7fe700 1 -- 172.21.15.52:0/2682750945 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327c01ead0 con 0x55ec64680fd0 2024-06-02T23:17:23.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.469+0000 7f326b7fe700 1 -- 172.21.15.52:0/2682750945 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c01cac0 con 0x55ec64680fd0 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 -- 172.21.15.52:0/2682750945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 msgr2=0x55ec646813b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec646813b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f327c00b640 tx=0x7f327c013000 comp rx=0 tx=0).stop 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 -- 172.21.15.52:0/2682750945 shutdown_connections 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec64686090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec646813b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2682750945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec646856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 -- 172.21.15.52:0/2682750945 >> 172.21.15.52:0/2682750945 conn(0x55ec645a7cc0 msgr2=0x55ec64677850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:23.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 -- 172.21.15.52:0/2682750945 shutdown_connections 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 -- 172.21.15.52:0/2682750945 wait complete. 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 -- start start 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.470+0000 7f3291f39b80 1 Processor -- start 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec6468d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec6468a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec6468aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ec6468ddd0 con 0x55ec64680fd0 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec6456ebf0 con 0x55ec64685c30 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec6468df50 con 0x55ec646818f0 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec6468d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3291f39b80 10 client.0 _ll_register_callbacks cb 0x55ec64568920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec6468d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38628/0 (socket says 172.21.15.52:38628) 2024-06-02T23:17:23.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 -- 172.21.15.52:0/3981641543 learned_addr learned my addr 172.21.15.52:0/3981641543 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3284c51700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec6468a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285c53700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec6468aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 -- 172.21.15.52:0/3981641543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 msgr2=0x55ec6468a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec6468a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 -- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 msgr2=0x55ec6468aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec6468aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 -- 172.21.15.52:0/3981641543 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6451d8d0 con 0x55ec64680fd0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3285452700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec6468d1e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f327c01e650 tx=0x7f327c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51cbf0f1017b4e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.471+0000 7f3268ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3268ff9700 1 -- 172.21.15.52:0/3981641543 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c036060 con 0x55ec64680fd0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3268ff9700 1 -- 172.21.15.52:0/3981641543 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f327c00be40 con 0x55ec64680fd0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c01ead0 con 0x55ec64680fd0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3268ff9700 1 -- 172.21.15.52:0/3981641543 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f327c01c6e0 con 0x55ec64680fd0 2024-06-02T23:17:23.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 10 client.17118 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec64698750 con 0x55ec64680fd0 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3268ff9700 1 -- 172.21.15.52:0/3981641543 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327c01ead0 con 0x55ec64680fd0 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3268ff9700 1 -- 172.21.15.52:0/3981641543 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f327c026a40 con 0x55ec64680fd0 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 10 client.17118 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 20 client.17118 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec6469ad30 0x55ec6469d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 4 client.17118 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.472+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec645dbd10 con 0x55ec6469ad30 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.473+0000 7f3284c51700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec6469ad30 0x55ec6469d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.473+0000 7f3268ff9700 10 client.17118 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.473+0000 7f3284c51700 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec6469ad30 0x55ec6469d120 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5c6f9ee5d6c1217 server_cookie=8d6e0901b06cbb8c in_seq=0 out_seq=0 2024-06-02T23:17:23.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3268ff9700 1 -- 172.21.15.52:0/3981641543 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec645dbd10 con 0x55ec6469ad30 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3268ff9700 10 client.17118 handle_command_reply: tid=0 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 client.17118 shutdown 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 msgr2=0x55ec6468d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec6468d1e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f327c01e650 tx=0x7f327c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 shutdown_connections 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec6469ad30 0x55ec6469d120 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64685c30 0x55ec6468aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec6468d1e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3268ff9700 0 client.17118 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 --2- 172.21.15.52:0/3981641543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec646818f0 0x55ec6468a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 >> 172.21.15.52:0/3981641543 conn(0x55ec645a7cc0 msgr2=0x55ec6467b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 shutdown_connections 2024-06-02T23:17:23.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 1 -- 172.21.15.52:0/3981641543 wait complete. 2024-06-02T23:17:23.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.474+0000 7f3291f39b80 20 client.17118 trim_cache size 0 max 16384 2024-06-02T23:17:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 Processor -- start 2024-06-02T23:17:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 -- start start 2024-06-02T23:17:23.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec64671690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec646818f0 0x55ec64689440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6467bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32801aa810 con 0x55ec64680fd0 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32801ac350 con 0x55ec646818f0 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3291f39b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32801aaee0 con 0x55ec64685c30 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec64671690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec64671690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38638/0 (socket says 172.21.15.52:38638) 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 -- 172.21.15.52:0/1351979284 learned_addr learned my addr 172.21.15.52:0/1351979284 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285c53700 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6467bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3284c51700 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec646818f0 0x55ec64689440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 -- 172.21.15.52:0/1351979284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 msgr2=0x55ec6467bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6467bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 -- 172.21.15.52:0/1351979284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec646818f0 msgr2=0x55ec64689440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec646818f0 0x55ec64689440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3285452700 1 -- 172.21.15.52:0/1351979284 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec6468de70 con 0x55ec64680fd0 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.490+0000 7f3284c51700 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec646818f0 0x55ec64689440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3285452700 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec64671690 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f327c005ee0 tx=0x7f327c005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2753271c6b0d8f60 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f326affd700 1 -- 172.21.15.52:0/1351979284 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c041020 con 0x55ec64680fd0 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f326affd700 1 -- 172.21.15.52:0/1351979284 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327c011ab0 con 0x55ec64680fd0 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f326affd700 1 -- 172.21.15.52:0/1351979284 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f327c036070 con 0x55ec64680fd0 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 -- 172.21.15.52:0/1351979284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 msgr2=0x55ec64671690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec64671690 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f327c005ee0 tx=0x7f327c005f10 comp rx=0 tx=0).stop 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 -- 172.21.15.52:0/1351979284 shutdown_connections 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6467bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec64680fd0 0x55ec64671690 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 --2- 172.21.15.52:0/1351979284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec646818f0 0x55ec64689440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.491+0000 7f3291f39b80 1 -- 172.21.15.52:0/1351979284 >> 172.21.15.52:0/1351979284 conn(0x55ec645a7cc0 msgr2=0x55ec64684af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 -- 172.21.15.52:0/1351979284 shutdown_connections 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 -- 172.21.15.52:0/1351979284 wait complete. 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 -- start start 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 Processor -- start 2024-06-02T23:17:23.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64680fd0 0x55ec646fda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 0x55ec646fdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6469d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32801abc80 con 0x55ec646818f0 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32801ab5b0 con 0x55ec64680fd0 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32801a87f0 con 0x55ec64685c30 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.492+0000 7f3291f39b80 10 client.0 _ll_register_callbacks cb 0x55ec64568920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 0x55ec646fdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 0x55ec646fdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38644/0 (socket says 172.21.15.52:38644) 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 -- 172.21.15.52:0/2622108390 learned_addr learned my addr 172.21.15.52:0/2622108390 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3285c53700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6469d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3285452700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64680fd0 0x55ec646fda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 -- 172.21.15.52:0/2622108390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 msgr2=0x55ec6469d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6469d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 -- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64680fd0 msgr2=0x55ec646fda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64680fd0 0x55ec646fda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 -- 172.21.15.52:0/2622108390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec6469dee0 con 0x55ec646818f0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3285452700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64680fd0 0x55ec646fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3284c51700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 0x55ec646fdfa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3270009990 tx=0x7f32700058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc8092038821b59 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3268ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3268ff9700 1 -- 172.21.15.52:0/2622108390 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32700050d0 con 0x55ec646818f0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3268ff9700 1 -- 172.21.15.52:0/2622108390 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3270005270 con 0x55ec646818f0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.493+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f327c011ab0 con 0x55ec646818f0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3268ff9700 1 -- 172.21.15.52:0/2622108390 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3270005470 con 0x55ec646818f0 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 10 client.17124 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:23.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec647051f0 con 0x55ec646818f0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3268ff9700 1 -- 172.21.15.52:0/2622108390 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f327c011ab0 con 0x55ec646818f0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3268ff9700 1 -- 172.21.15.52:0/2622108390 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f327000f6c0 con 0x55ec646818f0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 10 client.17124 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 20 client.17124 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec647058e0 0x55ec64707d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 4 client.17124 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec645d6290 con 0x55ec647058e0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.494+0000 7f3285452700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec647058e0 0x55ec64707d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3268ff9700 10 client.17124 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3285452700 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec647058e0 0x55ec64707d00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d791454d26781e83 server_cookie=93a18b32d6efb8b8 in_seq=0 out_seq=0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3268ff9700 1 -- 172.21.15.52:0/2622108390 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec645d6290 con 0x55ec647058e0 2024-06-02T23:17:23.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3268ff9700 10 client.17124 handle_command_reply: tid=0 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 client.17124 shutdown 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 msgr2=0x55ec646fdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 0x55ec646fdfa0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f3270009990 tx=0x7f32700058e0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 shutdown_connections 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec647058e0 0x55ec64707d00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec64685c30 0x55ec6469d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec64680fd0 0x55ec646fda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 --2- 172.21.15.52:0/2622108390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec646818f0 0x55ec646fdfa0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 >> 172.21.15.52:0/2622108390 conn(0x55ec645a7cc0 msgr2=0x55ec64671510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3268ff9700 0 client.17124 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 shutdown_connections 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.495+0000 7f3291f39b80 1 -- 172.21.15.52:0/2622108390 wait complete. 2024-06-02T23:17:23.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f3291f39b80 20 client.17124 trim_cache size 0 max 16384 2024-06-02T23:17:23.502 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:17:23.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f326c0a2ca0 msgr2=0x7f326c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f326c0a2ca0 0x7f326c0a5160 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3280199a40 tx=0x7f327c00b410 comp rx=0 tx=0).stop 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 msgr2=0x7f3280198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 0x7f3280198900 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f3278007d70 tx=0x7f327800a340 comp rx=0 tx=0).stop 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 shutdown_connections 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f326c0a2ca0 0x7f326c0a5160 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3280107570 0x7f3280198900 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3280100e50 0x7f328019e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 --2- 172.21.15.52:0/829044657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3280101810 0x7f328019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 >> 172.21.15.52:0/829044657 conn(0x7f32800766b0 msgr2=0x7f32800ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 shutdown_connections 2024-06-02T23:17:23.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:23.496+0000 7f32876b6700 1 -- 172.21.15.52:0/829044657 wait complete. 2024-06-02T23:17:23.628 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.02 MB/sec execute 152 sec latency 10105.792 ms 2024-06-02T23:17:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:23 smithi052 ceph-mon[32340]: pgmap v596: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 579 KiB/s wr, 93 op/s 2024-06-02T23:17:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:23 smithi174 ceph-mon[35994]: pgmap v596: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 579 KiB/s wr, 93 op/s 2024-06-02T23:17:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:23 smithi123 ceph-mon[35977]: pgmap v596: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 579 KiB/s wr, 93 op/s 2024-06-02T23:17:24.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 2.00 MB/sec execute 153 sec latency 11105.884 ms 2024-06-02T23:17:25.235 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:17:25.235 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:17:25.499 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:25.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.99 MB/sec execute 154 sec latency 12105.937 ms 2024-06-02T23:17:25.876 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:25 smithi052 ceph-mon[32340]: pgmap v597: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 324 KiB/s wr, 74 op/s 2024-06-02T23:17:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:25 smithi174 ceph-mon[35994]: pgmap v597: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 324 KiB/s wr, 74 op/s 2024-06-02T23:17:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:25 smithi123 ceph-mon[35977]: pgmap v597: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 324 KiB/s wr, 74 op/s 2024-06-02T23:17:26.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.98 MB/sec execute 155 sec latency 13106.026 ms 2024-06-02T23:17:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 -- 172.21.15.52:0/1012386077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 msgr2=0x7f3db810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/1012386077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db810f320 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b0a0 tx=0x7f3db000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 -- 172.21.15.52:0/1012386077 shutdown_connections 2024-06-02T23:17:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/1012386077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db810f320 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/1012386077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db8108500 0x7f3db81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/1012386077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db806a170 0x7f3db806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 -- 172.21.15.52:0/1012386077 >> 172.21.15.52:0/1012386077 conn(0x7f3db8078340 msgr2=0x7f3db8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.277+0000 7f3dbccf1700 1 -- 172.21.15.52:0/1012386077 shutdown_connections 2024-06-02T23:17:27.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.278+0000 7f3dbccf1700 1 -- 172.21.15.52:0/1012386077 wait complete. 2024-06-02T23:17:27.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.278+0000 7f3dbccf1700 1 Processor -- start 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.278+0000 7f3dbccf1700 1 -- start start 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3dbccf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db806a170 0x7f3db819e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3dbccf1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3dbccf1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db8108500 0x7f3db8198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3dbccf1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8112710 con 0x7f3db806ab30 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3dbccf1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8112590 con 0x7f3db806a170 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3dbccf1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3db8112890 con 0x7f3db8108500 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48020/0 (socket says 172.21.15.52:48020) 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2880414441 learned_addr learned my addr 172.21.15.52:0/2880414441 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db659c700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db806a170 0x7f3db819e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db8108500 0x7f3db8198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2880414441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db8108500 msgr2=0x7f3db8198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db8108500 0x7f3db8198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2880414441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db806a170 msgr2=0x7f3db819e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db806a170 0x7f3db819e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2880414441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db81991f0 con 0x7f3db806ab30 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db8108500 0x7f3db8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.279+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db819ee10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3da000b530 tx=0x7f3da000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.280+0000 7f3daf7fe700 1 -- 172.21.15.52:0/2880414441 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0015020 con 0x7f3db806ab30 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.280+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db81994e0 con 0x7f3db806ab30 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.280+0000 7f3daf7fe700 1 -- 172.21.15.52:0/2880414441 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da0011ce0 con 0x7f3db806ab30 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.280+0000 7f3daf7fe700 1 -- 172.21.15.52:0/2880414441 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0010e40 con 0x7f3db806ab30 2024-06-02T23:17:27.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.280+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db81a57a0 con 0x7f3db806ab30 2024-06-02T23:17:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.282+0000 7f3daf7fe700 1 -- 172.21.15.52:0/2880414441 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3da0018020 con 0x7f3db806ab30 2024-06-02T23:17:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.282+0000 7f3daf7fe700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3da40a2c20 0x7f3da40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.283+0000 7f3db659c700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3da40a2c20 0x7f3da40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.283+0000 7f3daf7fe700 1 -- 172.21.15.52:0/2880414441 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3da00ce890 con 0x7f3db806ab30 2024-06-02T23:17:27.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.283+0000 7f3db659c700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3da40a2c20 0x7f3da40a50e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3db8199bd0 tx=0x7f3da8009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 Processor -- start 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 -- start start 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98275cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c9827a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c9827aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c98163fe0 con 0x559c9827a5d0 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c981c28d0 con 0x559c982761f0 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c9812a4e0 con 0x559c982758d0 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c9827aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c9827aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48028/0 (socket says 172.21.15.52:48028) 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 -- 172.21.15.52:0/2726082353 learned_addr learned my addr 172.21.15.52:0/2726082353 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db659c700 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98275cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c9827a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 -- 172.21.15.52:0/2726082353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 msgr2=0x559c98275cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98275cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 -- 172.21.15.52:0/2726082353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 msgr2=0x559c9827a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c9827a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db6d9d700 1 -- 172.21.15.52:0/2726082353 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c98111950 con 0x559c9827a5d0 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.284+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c9827a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.285+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c9827aa10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b070 tx=0x7f3db0009140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc24e3114317121 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.285+0000 7f3dacff9700 1 -- 172.21.15.52:0/2726082353 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0020030 con 0x559c9827a5d0 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.285+0000 7f3dacff9700 1 -- 172.21.15.52:0/2726082353 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db0009410 con 0x559c9827a5d0 2024-06-02T23:17:27.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.285+0000 7f3dacff9700 1 -- 172.21.15.52:0/2726082353 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db001c440 con 0x559c9827a5d0 2024-06-02T23:17:27.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2726082353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 msgr2=0x559c9827aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c9827aa10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b070 tx=0x7f3db0009140 comp rx=0 tx=0).stop 2024-06-02T23:17:27.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2726082353 shutdown_connections 2024-06-02T23:17:27.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c9827aa10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98275cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2726082353 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c9827a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2726082353 >> 172.21.15.52:0/2726082353 conn(0x559c9819e100 msgr2=0x559c9826c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2726082353 shutdown_connections 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2726082353 wait complete. 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 -- start start 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.286+0000 7f3dc7574b80 1 Processor -- start 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982758d0 0x559c982819f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c9827ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c9827a5d0 0x559c9827f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c98163040 con 0x559c982761f0 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c981be890 con 0x559c982758d0 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c982826a0 con 0x559c9827a5d0 2024-06-02T23:17:27.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3dc7574b80 10 client.0 _ll_register_callbacks cb 0x559c98158630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c9827ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c9827ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48042/0 (socket says 172.21.15.52:48042) 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 -- 172.21.15.52:0/766868054 learned_addr learned my addr 172.21.15.52:0/766868054 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db659c700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982758d0 0x559c982819f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c9827a5d0 0x559c9827f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 -- 172.21.15.52:0/766868054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c9827a5d0 msgr2=0x559c9827f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c9827a5d0 0x559c9827f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 -- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982758d0 msgr2=0x559c982819f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982758d0 0x559c982819f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 -- 172.21.15.52:0/766868054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c981887d0 con 0x559c982761f0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c9827ed60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3da0097580 tx=0x7f3da0010400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e7e47a8af469a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.287+0000 7f3d9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3d9dffb700 1 -- 172.21.15.52:0/766868054 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da009f4d0 con 0x559c982761f0 2024-06-02T23:17:27.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3d9dffb700 1 -- 172.21.15.52:0/766868054 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3da0018310 con 0x559c982761f0 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db0009410 con 0x559c982761f0 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3d9dffb700 1 -- 172.21.15.52:0/766868054 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3da0018510 con 0x559c982761f0 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 10 client.17136 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c9828cdc0 con 0x559c982761f0 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3d9dffb700 1 -- 172.21.15.52:0/766868054 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db0009410 con 0x559c982761f0 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3d9dffb700 1 -- 172.21.15.52:0/766868054 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3da00a3900 con 0x559c982761f0 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 10 client.17136 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 10 client.17136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 20 client.17136 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c9828f350 0x559c98291780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.288+0000 7f3dc7574b80 4 client.17136 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.289+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c981cfdd0 con 0x559c9828f350 2024-06-02T23:17:27.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.289+0000 7f3db659c700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c9828f350 0x559c98291780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.289+0000 7f3d9dffb700 10 client.17136 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.289+0000 7f3db659c700 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c9828f350 0x559c98291780 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42cdbeb49810c8cc server_cookie=d0fa1fe5e6a35155 in_seq=0 out_seq=0 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3d9dffb700 1 -- 172.21.15.52:0/766868054 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c981cfdd0 con 0x559c9828f350 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3d9dffb700 10 client.17136 handle_command_reply: tid=0 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 client.17136 shutdown 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 msgr2=0x559c9827ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c9827ed60 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f3da0097580 tx=0x7f3da0010400 comp rx=0 tx=0).stop 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 shutdown_connections 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c9828f350 0x559c98291780 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c9827a5d0 0x559c9827f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982758d0 0x559c982819f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/766868054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c9827ed60 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.290+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 >> 172.21.15.52:0/766868054 conn(0x559c9819e100 msgr2=0x559c9827baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:27.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.291+0000 7f3d9dffb700 0 client.17136 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:27.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.291+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 shutdown_connections 2024-06-02T23:17:27.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.291+0000 7f3dc7574b80 1 -- 172.21.15.52:0/766868054 wait complete. 2024-06-02T23:17:27.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.291+0000 7f3dc7574b80 20 client.17136 trim_cache size 0 max 16384 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 Processor -- start 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 -- start start 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c9827e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c98280ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c98297050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81aa890 con 0x559c9827a5d0 2024-06-02T23:17:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ac3d0 con 0x559c982761f0 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81aaf60 con 0x559c982758d0 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c98297050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c98297050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48046/0 (socket says 172.21.15.52:48046) 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 -- 172.21.15.52:0/3151850845 learned_addr learned my addr 172.21.15.52:0/3151850845 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db659c700 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c9827e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 -- 172.21.15.52:0/3151850845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 msgr2=0x559c9827e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c9827e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 -- 172.21.15.52:0/3151850845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 msgr2=0x559c98280ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c98280ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c98280ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db6d9d700 1 -- 172.21.15.52:0/3151850845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c98291d60 con 0x559c9827a5d0 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.313+0000 7f3db659c700 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c9827e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c98297050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b830 tx=0x7f3db000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654da4c003c18fa8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3d9ffff700 1 -- 172.21.15.52:0/3151850845 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0020040 con 0x559c9827a5d0 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3d9ffff700 1 -- 172.21.15.52:0/3151850845 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db0004d90 con 0x559c9827a5d0 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3d9ffff700 1 -- 172.21.15.52:0/3151850845 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3db0025790 con 0x559c9827a5d0 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 -- 172.21.15.52:0/3151850845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 msgr2=0x559c98297050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c98297050 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3db000b830 tx=0x7f3db000b420 comp rx=0 tx=0).stop 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 -- 172.21.15.52:0/3151850845 shutdown_connections 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c9827a5d0 0x559c98297050 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c9827e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/3151850845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c982761f0 0x559c98280ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.314+0000 7f3dc7574b80 1 -- 172.21.15.52:0/3151850845 >> 172.21.15.52:0/3151850845 conn(0x559c9819e100 msgr2=0x559c98280590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 -- 172.21.15.52:0/3151850845 shutdown_connections 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 -- 172.21.15.52:0/3151850845 wait complete. 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 -- start start 2024-06-02T23:17:27.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 Processor -- start 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98277c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c982781c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c9827a5d0 0x559c98278700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81abd00 con 0x559c982761f0 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81ab630 con 0x559c9827a5d0 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3db81a8850 con 0x559c982758d0 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3dc7574b80 10 client.0 _ll_register_callbacks cb 0x559c9826be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3db5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c982781c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3db5d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c982781c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48048/0 (socket says 172.21.15.52:48048) 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2008443084 learned_addr learned my addr 172.21.15.52:0/2008443084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.315+0000 7f3db6d9d700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c9827a5d0 0x559c98278700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db659c700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98277c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2008443084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 msgr2=0x559c98277c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98277c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c9827a5d0 msgr2=0x559c98278700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c9827a5d0 0x559c98278700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db5d9b700 1 -- 172.21.15.52:0/2008443084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c982f9070 con 0x559c982761f0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3db5d9b700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c982781c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3da009ae20 tx=0x7f3da00973e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc0ab32874e0be5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3d9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3d9dffb700 1 -- 172.21.15.52:0/2008443084 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da009aef0 con 0x559c982761f0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3d9dffb700 1 -- 172.21.15.52:0/2008443084 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3da0011830 con 0x559c982761f0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3db0004d90 con 0x559c982761f0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3d9dffb700 1 -- 172.21.15.52:0/2008443084 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3da007f020 con 0x559c982761f0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3dc7574b80 10 client.17142 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c982fa3a0 con 0x559c982761f0 2024-06-02T23:17:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.316+0000 7f3d9dffb700 1 -- 172.21.15.52:0/2008443084 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3db0004d90 con 0x559c982761f0 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3d9dffb700 1 -- 172.21.15.52:0/2008443084 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3da0099fe0 con 0x559c982761f0 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3dc7574b80 10 client.17142 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3dc7574b80 10 client.17142 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3dc7574b80 20 client.17142 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c982fa950 0x559c982fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3dc7574b80 4 client.17142 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559c981ca350 con 0x559c982fa950 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3db659c700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c982fa950 0x559c982fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3db659c700 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c982fa950 0x559c982fcd70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9378f21384d856f3 server_cookie=805b69ec61a2b16c in_seq=0 out_seq=0 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.317+0000 7f3d9dffb700 10 client.17142 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.318+0000 7f3d9dffb700 1 -- 172.21.15.52:0/2008443084 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559c981ca350 con 0x559c982fa950 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.318+0000 7f3d9dffb700 10 client.17142 handle_command_reply: tid=0 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.318+0000 7f3dc7574b80 1 client.17142 shutdown 2024-06-02T23:17:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 msgr2=0x559c982781c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c982781c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f3da009ae20 tx=0x7f3da00973e0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 shutdown_connections 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c982fa950 0x559c982fcd70 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c9827a5d0 0x559c98278700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c982758d0 0x559c98277c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3d9dffb700 0 client.17142 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 --2- 172.21.15.52:0/2008443084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c982761f0 0x559c982781c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 >> 172.21.15.52:0/2008443084 conn(0x559c9819e100 msgr2=0x559c98270b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 shutdown_connections 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 1 -- 172.21.15.52:0/2008443084 wait complete. 2024-06-02T23:17:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dc7574b80 20 client.17142 trim_cache size 0 max 16384 2024-06-02T23:17:27.325 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:17:27.325 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:17:27.325 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "7a38893b-611d-4743-b4bf-af41582031df", 2024-06-02T23:17:27.325 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:17:27.325 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:17:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3da40a2c20 msgr2=0x7f3da40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3da40a2c20 0x7f3da40a50e0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3db8199bd0 tx=0x7f3da8009190 comp rx=0 tx=0).stop 2024-06-02T23:17:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 msgr2=0x7f3db819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.319+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db819ee10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3da000b530 tx=0x7f3da000b840 comp rx=0 tx=0).stop 2024-06-02T23:17:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 shutdown_connections 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3da40a2c20 0x7f3da40a50e0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db8108500 0x7f3db8198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db806a170 0x7f3db819e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 --2- 172.21.15.52:0/2880414441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db806ab30 0x7f3db819ee10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 >> 172.21.15.52:0/2880414441 conn(0x7f3db8078340 msgr2=0x7f3db8100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 shutdown_connections 2024-06-02T23:17:27.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:27.320+0000 7f3dbccf1700 1 -- 172.21.15.52:0/2880414441 wait complete. 2024-06-02T23:17:27.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.96 MB/sec execute 156 sec latency 14106.126 ms 2024-06-02T23:17:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:27 smithi123 ceph-mon[35977]: pgmap v598: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 267 KiB/s wr, 53 op/s 2024-06-02T23:17:27.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:17:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:17:27.333+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v17362 ap=3 f(v1998 m2024-06-02T23:17:10.069511+0000 3=3+0) n(v23 rc2024-06-02T23:17:10.069511+0000 b780288 4=3+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:190 nref=3)) caps={15216=pAsLsXs/-@20930} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ac1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v66>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x10000000003/client.0 v6>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v22>,<0x10000000001/sv_0 v45>,<0x10000000000/qa v32>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v17362>,<0x10000000008/~dmtmp v15596>,<0x10000000007/client0 v564>,<0x10000000006/clients v542>,<0x10000000005/tmp v522>,<0x10000000003/client.0 v500>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v501>,<0x10000000001/sv_0 v516>,<0x10000000000/qa v491>,<0x1/volumes v498>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:17:09.667551+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:17:10.069511+0000 b780288 4=3+1)","memory_value.dirstat":"f(v1998 m2024-06-02T23:17:10.069511+0000 3=3+0)","memory_value.rstat":"n(v23 rc2024-06-02T23:17:10.069511+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:17:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:27 smithi174 ceph-mon[35994]: pgmap v598: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 267 KiB/s wr, 53 op/s 2024-06-02T23:17:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:27 smithi052 ceph-mon[32340]: pgmap v598: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 267 KiB/s wr, 53 op/s 2024-06-02T23:17:27.978 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:17:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:17:28.619 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-06-02T23:17:28.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:28 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.95 MB/sec execute 157 sec latency 15106.166 ms 2024-06-02T23:17:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 -- 172.21.15.52:0/1624041157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 msgr2=0x7f96e0102020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 --2- 172.21.15.52:0/1624041157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0102020 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f96c8009900 tx=0x7f96c802a410 comp rx=0 tx=0).stop 2024-06-02T23:17:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 -- 172.21.15.52:0/1624041157 shutdown_connections 2024-06-02T23:17:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 --2- 172.21.15.52:0/1624041157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e0102560 0x7f96e0108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 --2- 172.21.15.52:0/1624041157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0102020 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 --2- 172.21.15.52:0/1624041157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e01009b0 0x7f96e0100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 -- 172.21.15.52:0/1624041157 >> 172.21.15.52:0/1624041157 conn(0x7f96e00fbf20 msgr2=0x7f96e00fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 -- 172.21.15.52:0/1624041157 shutdown_connections 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.761+0000 7f96e6790640 1 -- 172.21.15.52:0/1624041157 wait complete. 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 Processor -- start 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 -- start start 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e01009b0 0x7f96e0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102560 0x7f96e019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010c450 con 0x7f96e0101ba0 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010c2d0 con 0x7f96e01009b0 2024-06-02T23:17:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e6790640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96e010c5d0 con 0x7f96e0102560 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0198070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48058/0 (socket says 172.21.15.52:48058) 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 -- 172.21.15.52:0/1995403851 learned_addr learned my addr 172.21.15.52:0/1995403851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96dffff640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e01009b0 0x7f96e0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102560 0x7f96e019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 -- 172.21.15.52:0/1995403851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102560 msgr2=0x7f96e019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102560 0x7f96e019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 -- 172.21.15.52:0/1995403851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e01009b0 msgr2=0x7f96e0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e01009b0 0x7f96e0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.762+0000 7f96d7fff640 1 -- 172.21.15.52:0/1995403851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96e019f750 con 0x7f96e0101ba0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.763+0000 7f96d7fff640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0198070 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005940 tx=0x7f96c8004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.763+0000 7f96ddffb640 1 -- 172.21.15.52:0/1995403851 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c803f030 con 0x7f96e0101ba0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.763+0000 7f96ddffb640 1 -- 172.21.15.52:0/1995403851 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c8003cb0 con 0x7f96e0101ba0 2024-06-02T23:17:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.763+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96e019f9e0 con 0x7f96e0101ba0 2024-06-02T23:17:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.763+0000 7f96ddffb640 1 -- 172.21.15.52:0/1995403851 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c8035070 con 0x7f96e0101ba0 2024-06-02T23:17:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.763+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96e019ff20 con 0x7f96e0101ba0 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.765+0000 7f96ddffb640 1 -- 172.21.15.52:0/1995403851 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96c8042030 con 0x7f96e0101ba0 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.765+0000 7f96e9579740 1 Processor -- start 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 -- start start 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bf420 0x55d5023bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 0x55d5023c4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023c4f80 0x55d5023c7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d50231e4f0 con 0x55d5023bfe10 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d5022bc870 con 0x55d5023bf420 2024-06-02T23:17:28.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e9579740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d50231a4b0 con 0x55d5023c4f80 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 0x55d5023c4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 0x55d5023c4910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48064/0 (socket says 172.21.15.52:48064) 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 -- 172.21.15.52:0/3522303481 learned_addr learned my addr 172.21.15.52:0/3522303481 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bf420 0x55d5023bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96ddffb640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96cc0a2cb0 0x7f96cc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96d7fff640 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023c4f80 0x55d5023c7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96cc0a2cb0 0x7f96cc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96ddffb640 1 -- 172.21.15.52:0/1995403851 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96c80efa30 con 0x7f96e0101ba0 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 -- 172.21.15.52:0/3522303481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023c4f80 msgr2=0x55d5023c7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023c4f80 0x55d5023c7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 -- 172.21.15.52:0/3522303481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bf420 msgr2=0x55d5023bf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bf420 0x55d5023bf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96dffff640 1 -- 172.21.15.52:0/3522303481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5022d89b0 con 0x55d5023bfe10 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.766+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bf420 0x55d5023bf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:28.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96dffff640 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 0x55d5023c4910 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ac70 tx=0x7f96d000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeece55249d641f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96cc0a2cb0 0x7f96cc0a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55d5023c0670 tx=0x7f96d8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96d67fc640 1 -- 172.21.15.52:0/3522303481 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0005220 con 0x55d5023bfe10 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96d67fc640 1 -- 172.21.15.52:0/3522303481 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d00047a0 con 0x55d5023bfe10 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96d67fc640 1 -- 172.21.15.52:0/3522303481 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d00187e0 con 0x55d5023bfe10 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96e9579740 1 -- 172.21.15.52:0/3522303481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 msgr2=0x55d5023c4910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.767+0000 7f96e9579740 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 0x55d5023c4910 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f96d000ac70 tx=0x7f96d000f790 comp rx=0 tx=0).stop 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- 172.21.15.52:0/3522303481 shutdown_connections 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023c4f80 0x55d5023c7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bfe10 0x55d5023c4910 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 --2- 172.21.15.52:0/3522303481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bf420 0x55d5023bf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- 172.21.15.52:0/3522303481 >> 172.21.15.52:0/3522303481 conn(0x55d5023b6680 msgr2=0x55d5023b6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- 172.21.15.52:0/3522303481 shutdown_connections 2024-06-02T23:17:28.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- 172.21.15.52:0/3522303481 wait complete. 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- start start 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 Processor -- start 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bf420 0x55d5023c9900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bfe10 0x55d5023c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 0x55d5023d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5023c7f30 con 0x55d5023c4f80 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d5022a7ae0 con 0x55d5023bfe10 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5023c80b0 con 0x55d5023bf420 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e9579740 10 client.0 _ll_register_callbacks cb 0x55d5022e09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.768+0000 7f96e4d2d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 0x55d5023d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 0x55d5023d0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48074/0 (socket says 172.21.15.52:48074) 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 -- 172.21.15.52:0/2830455507 learned_addr learned my addr 172.21.15.52:0/2830455507 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96d7fff640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bfe10 0x55d5023c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96dffff640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bf420 0x55d5023c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 -- 172.21.15.52:0/2830455507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bf420 msgr2=0x55d5023c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bf420 0x55d5023c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 -- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bfe10 msgr2=0x55d5023c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bfe10 0x55d5023c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 -- 172.21.15.52:0/2830455507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d502032210 con 0x55d5023c4f80 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96dffff640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bf420 0x55d5023c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 0x55d5023d0e80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f96d8011e30 tx=0x7f96d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc071e74bea81f01 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96bf7fe640 1 -- 172.21.15.52:0/2830455507 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d8023070 con 0x55d5023c4f80 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96bf7fe640 1 -- 172.21.15.52:0/2830455507 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96d8004b20 con 0x55d5023c4f80 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d00047a0 con 0x55d5023c4f80 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96bf7fe640 1 -- 172.21.15.52:0/2830455507 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96d802e050 con 0x55d5023c4f80 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e9579740 10 client.17154 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5020fe620 con 0x55d5023c4f80 2024-06-02T23:17:28.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.769+0000 7f96bf7fe640 1 -- 172.21.15.52:0/2830455507 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d00047a0 con 0x55d5023c4f80 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96bf7fe640 1 -- 172.21.15.52:0/2830455507 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f96d801f440 con 0x55d5023c4f80 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96e9579740 10 client.17154 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96e9579740 10 client.17154 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96e9579740 20 client.17154 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96e9579740 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5023d5830 0x55d5023d7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96e9579740 4 client.17154 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d502331f10 con 0x55d5023d5830 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.770+0000 7f96dffff640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5023d5830 0x55d5023d7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.771+0000 7f96dffff640 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5023d5830 0x55d5023d7cc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e6b3386267168b server_cookie=a3daf578e8ea5398 in_seq=0 out_seq=0 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.771+0000 7f96bf7fe640 10 client.17154 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.771+0000 7f96bf7fe640 1 -- 172.21.15.52:0/2830455507 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d502331f10 con 0x55d5023d5830 2024-06-02T23:17:28.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.771+0000 7f96bf7fe640 10 client.17154 handle_command_reply: tid=0 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.771+0000 7f96e9579740 1 client.17154 shutdown 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 msgr2=0x55d5023d0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 0x55d5023d0e80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f96d8011e30 tx=0x7f96d8004a40 comp rx=0 tx=0).stop 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 shutdown_connections 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5023d5830 0x55d5023d7cc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023c4f80 0x55d5023d0e80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023bfe10 0x55d5023c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96bf7fe640 0 client.17154 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 --2- 172.21.15.52:0/2830455507 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bf420 0x55d5023c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 >> 172.21.15.52:0/2830455507 conn(0x55d5023b6680 msgr2=0x55d5023b57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 shutdown_connections 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 1 -- 172.21.15.52:0/2830455507 wait complete. 2024-06-02T23:17:28.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.772+0000 7f96e9579740 20 client.17154 trim_cache size 0 max 16384 2024-06-02T23:17:28.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 Processor -- start 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 -- start start 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023b9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023b9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96e01a6a40 con 0x55d5023bf420 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e01a8580 con 0x55d5023c4f80 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96e9579740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96e01a7110 con 0x55d5023bfe10 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023b9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023b9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48080/0 (socket says 172.21.15.52:48080) 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.787+0000 7f96dffff640 1 -- 172.21.15.52:0/1700397294 learned_addr learned my addr 172.21.15.52:0/1700397294 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023b9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96d7fff640 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96dffff640 1 -- 172.21.15.52:0/1700397294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 msgr2=0x55d5023d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96dffff640 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023d0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96dffff640 1 -- 172.21.15.52:0/1700397294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 msgr2=0x55d5023b9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96dffff640 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023b9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96dffff640 1 -- 172.21.15.52:0/1700397294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5023c06a0 con 0x55d5023bf420 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96d7fff640 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96dffff640 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023b9590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f96d000a840 tx=0x7f96d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d09e788b659d11b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96d5ffb640 1 -- 172.21.15.52:0/1700397294 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0004c50 con 0x55d5023bf420 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96d5ffb640 1 -- 172.21.15.52:0/1700397294 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d00044f0 con 0x55d5023bf420 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96d5ffb640 1 -- 172.21.15.52:0/1700397294 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0005230 con 0x55d5023bf420 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e9579740 1 -- 172.21.15.52:0/1700397294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 msgr2=0x55d5023b9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e9579740 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023b9590 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f96d000a840 tx=0x7f96d0000bc0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e9579740 1 -- 172.21.15.52:0/1700397294 shutdown_connections 2024-06-02T23:17:28.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e9579740 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e9579740 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023b9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.788+0000 7f96e9579740 1 --2- 172.21.15.52:0/1700397294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023b9590 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- 172.21.15.52:0/1700397294 >> 172.21.15.52:0/1700397294 conn(0x55d5023b6680 msgr2=0x55d5023c1270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- 172.21.15.52:0/1700397294 shutdown_connections 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- 172.21.15.52:0/1700397294 wait complete. 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- start start 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 Processor -- start 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023cf440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96e01a7eb0 con 0x55d5023bf420 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96e01a77e0 con 0x55d5023c4f80 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96e01a4a20 con 0x55d5023bfe10 2024-06-02T23:17:28.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96e9579740 10 client.0 _ll_register_callbacks cb 0x7f96d8011ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96dffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48092/0 (socket says 172.21.15.52:48092) 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.789+0000 7f96dffff640 1 -- 172.21.15.52:0/3085563183 learned_addr learned my addr 172.21.15.52:0/3085563183 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96d7fff640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96dffff640 1 -- 172.21.15.52:0/3085563183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 msgr2=0x55d5023ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96dffff640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023ce430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023cf440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96dffff640 1 -- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 msgr2=0x55d5023cf440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96dffff640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023cf440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96dffff640 1 -- 172.21.15.52:0/3085563183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5023cfa30 con 0x55d5023bf420 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96e4d2d640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96dffff640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023cdef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f96d000a840 tx=0x7f96d0005d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa388f2a25bb549 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:28.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96bf7fe640 1 -- 172.21.15.52:0/3085563183 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96d0039060 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96bf7fe640 1 -- 172.21.15.52:0/3085563183 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96d0005130 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96d00044f0 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96bf7fe640 1 -- 172.21.15.52:0/3085563183 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f96d0005330 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96e9579740 10 client.17160 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d5023d0cb0 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.790+0000 7f96bf7fe640 1 -- 172.21.15.52:0/3085563183 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96d00044f0 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96bf7fe640 1 -- 172.21.15.52:0/3085563183 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f96d0040c00 con 0x55d5023bf420 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96e9579740 10 client.17160 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96e9579740 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96e9579740 20 client.17160 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96e9579740 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5024406b0 0x55d502442aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96e9579740 4 client.17160 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:17:28.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d50232bc20 con 0x55d5024406b0 2024-06-02T23:17:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96d7fff640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5024406b0 0x55d502442aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96d7fff640 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5024406b0 0x55d502442aa0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcd0ebac7d013ad3 server_cookie=e95894eec7d4d9ee in_seq=0 out_seq=0 2024-06-02T23:17:28.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.791+0000 7f96bf7fe640 10 client.17160 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96bf7fe640 1 -- 172.21.15.52:0/3085563183 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d50232bc20 con 0x55d5024406b0 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96bf7fe640 10 client.17160 handle_command_reply: tid=0 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 client.17160 shutdown 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 msgr2=0x55d5023cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023cdef0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f96d000a840 tx=0x7f96d0005d00 comp rx=0 tx=0).stop 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 shutdown_connections 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5024406b0 0x55d502442aa0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5023bfe10 0x55d5023ce430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5023c4f80 0x55d5023cf440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 --2- 172.21.15.52:0/3085563183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5023bf420 0x55d5023cdef0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96bf7fe640 0 client.17160 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:28.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 >> 172.21.15.52:0/3085563183 conn(0x55d5023b6680 msgr2=0x55d5023b9590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 shutdown_connections 2024-06-02T23:17:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 1 -- 172.21.15.52:0/3085563183 wait complete. 2024-06-02T23:17:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.809+0000 7f96e9579740 20 client.17160 trim_cache size 0 max 16384 2024-06-02T23:17:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:17:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:17:28.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96cc0a2cb0 msgr2=0x7f96cc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96cc0a2cb0 0x7f96cc0a5170 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x55d5023c0670 tx=0x7f96d8004ab0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 msgr2=0x7f96e0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0198070 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f96c8005940 tx=0x7f96c8004430 comp rx=0 tx=0).stop 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 shutdown_connections 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96cc0a2cb0 0x7f96cc0a5170 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96e0102560 0x7f96e019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96e0101ba0 0x7f96e0198070 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 --2- 172.21.15.52:0/1995403851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96e01009b0 0x7f96e0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 >> 172.21.15.52:0/1995403851 conn(0x7f96e00fbf20 msgr2=0x7f96e00fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 shutdown_connections 2024-06-02T23:17:28.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:28.810+0000 7f96e6790640 1 -- 172.21.15.52:0/1995403851 wait complete. 2024-06-02T23:17:28.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:28.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-06-02T23:17:28.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:28 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:17:29.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:17:29.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:17:29.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-06-02T23:17:29.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:28 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:17:29.267 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:17:29.554 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:29.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.94 MB/sec execute 158 sec latency 16106.215 ms 2024-06-02T23:17:29.800 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:29 smithi052 ceph-mon[32340]: pgmap v599: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 62 KiB/s wr, 30 op/s 2024-06-02T23:17:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:29 smithi174 ceph-mon[35994]: pgmap v599: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 62 KiB/s wr, 30 op/s 2024-06-02T23:17:30.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:29 smithi123 ceph-mon[35977]: pgmap v599: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 62 KiB/s wr, 30 op/s 2024-06-02T23:17:30.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.93 MB/sec execute 159 sec latency 17106.306 ms 2024-06-02T23:17:31.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.91 MB/sec execute 160 sec latency 18106.401 ms 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4292866755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc01073f0 msgr2=0x7f6cc01077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4292866755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc01073f0 0x7f6cc01077d0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc009990 tx=0x7f6cbc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4292866755 shutdown_connections 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4292866755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc0101d40 0x7f6cc010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4292866755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc01073f0 0x7f6cc01077d0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4292866755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101380 0x7f6cc0101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4292866755 >> 172.21.15.52:0/4292866755 conn(0x7f6cc0078300 msgr2=0x7f6cc0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4292866755 shutdown_connections 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.060+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4292866755 wait complete. 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 Processor -- start 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 -- start start 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 0x7f6cc019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101d40 0x7f6cc019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc01073f0 0x7f6cc0198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0111590 con 0x7f6cc0101380 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0111410 con 0x7f6cc0101d40 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc6f52700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc0111710 con 0x7f6cc01073f0 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc4cee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 0x7f6cc019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc4cee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 0x7f6cc019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48116/0 (socket says 172.21.15.52:48116) 2024-06-02T23:17:31.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.061+0000 7f6cc4cee700 1 -- 172.21.15.52:0/4102816516 learned_addr learned my addr 172.21.15.52:0/4102816516 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101d40 0x7f6cc019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc01073f0 0x7f6cc0198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc4cee700 1 -- 172.21.15.52:0/4102816516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc01073f0 msgr2=0x7f6cc0198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc01073f0 0x7f6cc0198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc4cee700 1 -- 172.21.15.52:0/4102816516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101d40 msgr2=0x7f6cc019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101d40 0x7f6cc019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc4cee700 1 -- 172.21.15.52:0/4102816516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc01990a0 con 0x7f6cc0101380 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101d40 0x7f6cc019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc01073f0 0x7f6cc0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 0x7f6cc019e780 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cb5ffb700 1 -- 172.21.15.52:0/4102816516 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc027070 con 0x7f6cc0101380 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cb5ffb700 1 -- 172.21.15.52:0/4102816516 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cbc00bdc0 con 0x7f6cc0101380 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cb5ffb700 1 -- 172.21.15.52:0/4102816516 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cbc00f460 con 0x7f6cc0101380 2024-06-02T23:17:31.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc0199330 con 0x7f6cc0101380 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.062+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc01a5650 con 0x7f6cc0101380 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.065+0000 7f6cb5ffb700 1 -- 172.21.15.52:0/4102816516 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cbc02a070 con 0x7f6cc0101380 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.065+0000 7f6cb5ffb700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca80a2ca0 0x7f6ca80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.065+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca80a2ca0 0x7f6ca80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.065+0000 7f6cb5ffb700 1 -- 172.21.15.52:0/4102816516 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cbc0cfeb0 con 0x7f6cc0101380 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca80a2ca0 0x7f6ca80a5160 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0005d90 tx=0x7f6cb0005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cd17d5b80 1 Processor -- start 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cd17d5b80 1 -- start start 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd06a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06f60 0x558f2fd009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 0x558f2fd01340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.066+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f2fc52810 con 0x558f2fd00f20 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f2fbfa720 con 0x558f2fd06f60 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558f2fc4e7d0 con 0x558f2fd06640 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 0x558f2fd01340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 0x558f2fd01340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48128/0 (socket says 172.21.15.52:48128) 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 -- 172.21.15.52:0/825472541 learned_addr learned my addr 172.21.15.52:0/825472541 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd06a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06f60 0x558f2fd009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 -- 172.21.15.52:0/825472541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 msgr2=0x558f2fd06a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd06a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 -- 172.21.15.52:0/825472541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06f60 msgr2=0x558f2fd009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06f60 0x558f2fd009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 -- 172.21.15.52:0/825472541 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2fbf0f50 con 0x558f2fd00f20 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 0x558f2fd01340 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8007f30 tx=0x7f6cb800c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ae28b80f31b947 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:31.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6ca6ffd700 1 -- 172.21.15.52:0/825472541 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8014070 con 0x558f2fd00f20 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6ca6ffd700 1 -- 172.21.15.52:0/825472541 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb800faf0 con 0x558f2fd00f20 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.067+0000 7f6ca6ffd700 1 -- 172.21.15.52:0/825472541 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb800ea20 con 0x558f2fd00f20 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/825472541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 msgr2=0x558f2fd01340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 0x558f2fd01340 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8007f30 tx=0x7f6cb800c5a0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/825472541 shutdown_connections 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd00f20 0x558f2fd01340 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd06a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/825472541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06f60 0x558f2fd009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/825472541 >> 172.21.15.52:0/825472541 conn(0x558f2fc2bcc0 msgr2=0x558f2fcfb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/825472541 shutdown_connections 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.068+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/825472541 wait complete. 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 -- start start 2024-06-02T23:17:31.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 Processor -- start 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd00f20 0x558f2fd0ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06640 0x558f2fd09fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f2fd0d9d0 con 0x558f2fd06f60 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f2fbf2bf0 con 0x558f2fd06640 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558f2fd0db50 con 0x558f2fd00f20 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cd17d5b80 10 client.0 _ll_register_callbacks cb 0x558f2fbec920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cc54ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cc54ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48136/0 (socket says 172.21.15.52:48136) 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cc54ef700 1 -- 172.21.15.52:0/82982663 learned_addr learned my addr 172.21.15.52:0/82982663 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06640 0x558f2fd09fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.069+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd00f20 0x558f2fd0ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc54ef700 1 -- 172.21.15.52:0/82982663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd00f20 msgr2=0x558f2fd0ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd00f20 0x558f2fd0ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc54ef700 1 -- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06640 msgr2=0x558f2fd09fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06640 0x558f2fd09fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc54ef700 1 -- 172.21.15.52:0/82982663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2fba18d0 con 0x558f2fd06f60 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd00f20 0x558f2fd0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0a510 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800cf60 tx=0x7f6cb8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928d589cb136149b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6c9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6c9ffff700 1 -- 172.21.15.52:0/82982663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8014040 con 0x558f2fd06f60 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6c9ffff700 1 -- 172.21.15.52:0/82982663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cb800e8a0 con 0x558f2fd06f60 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb800faf0 con 0x558f2fd06f60 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6c9ffff700 1 -- 172.21.15.52:0/82982663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6cb800eaa0 con 0x558f2fd06f60 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cd17d5b80 10 client.17172 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558f2fd18240 con 0x558f2fd06f60 2024-06-02T23:17:31.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.070+0000 7f6c9ffff700 1 -- 172.21.15.52:0/82982663 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb800faf0 con 0x558f2fd06f60 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6c9ffff700 1 -- 172.21.15.52:0/82982663 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6cb8019a90 con 0x558f2fd06f60 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cd17d5b80 10 client.17172 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cd17d5b80 10 client.17172 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cd17d5b80 20 client.17172 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd1cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cd17d5b80 4 client.17172 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f2fc5fd10 con 0x558f2fd1a7c0 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.071+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd1cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd1cbb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218d594c078e59af server_cookie=bae77a467a7e38c9 in_seq=0 out_seq=0 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6c9ffff700 10 client.17172 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6c9ffff700 1 -- 172.21.15.52:0/82982663 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f2fc5fd10 con 0x558f2fd1a7c0 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6c9ffff700 10 client.17172 handle_command_reply: tid=0 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 client.17172 shutdown 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 msgr2=0x558f2fd0a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0a510 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800cf60 tx=0x7f6cb8007a40 comp rx=0 tx=0).stop 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 shutdown_connections 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd1cbb0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0a510 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.073+0000 7f6c9ffff700 0 client.17172 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.072+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd00f20 0x558f2fd0ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.073+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/82982663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd06640 0x558f2fd09fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.073+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 >> 172.21.15.52:0/82982663 conn(0x558f2fc2bcc0 msgr2=0x558f2fcff9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.073+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 shutdown_connections 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.073+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/82982663 wait complete. 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.073+0000 7f6cd17d5b80 20 client.17172 trim_cache size 0 max 16384 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 Processor -- start 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 -- start start 2024-06-02T23:17:31.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd009e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 0x558f2fd04730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06f60 0x558f2fd07bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01ac8a0 con 0x558f2fd06640 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01ae3e0 con 0x558f2fd00f20 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01acf70 con 0x558f2fd06f60 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 0x558f2fd04730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 0x558f2fd04730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48150/0 (socket says 172.21.15.52:48150) 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 -- 172.21.15.52:0/3848167558 learned_addr learned my addr 172.21.15.52:0/3848167558 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06f60 0x558f2fd07bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 -- 172.21.15.52:0/3848167558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06f60 msgr2=0x558f2fd07bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06f60 0x558f2fd07bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 -- 172.21.15.52:0/3848167558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 msgr2=0x558f2fd009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cb7fff700 1 -- 172.21.15.52:0/3848167558 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f2fcfc520 con 0x558f2fd06640 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.094+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.095+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 0x558f2fd04730 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000ee50 tx=0x7f6cb0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fceb890ba3451aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.095+0000 7f6ca67fc700 1 -- 172.21.15.52:0/3848167558 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0011de0 con 0x558f2fd06640 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.095+0000 7f6ca67fc700 1 -- 172.21.15.52:0/3848167558 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb0024070 con 0x558f2fd06640 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.095+0000 7f6ca67fc700 1 -- 172.21.15.52:0/3848167558 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0003e40 con 0x558f2fd06640 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.095+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/3848167558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 msgr2=0x558f2fd04730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.095+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 0x558f2fd04730 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000ee50 tx=0x7f6cb0009f90 comp rx=0 tx=0).stop 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/3848167558 shutdown_connections 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06f60 0x558f2fd07bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/3848167558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06640 0x558f2fd04730 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/3848167558 >> 172.21.15.52:0/3848167558 conn(0x558f2fc2bcc0 msgr2=0x558f2fcf4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/3848167558 shutdown_connections 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/3848167558 wait complete. 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 -- start start 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.096+0000 7f6cd17d5b80 1 Processor -- start 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd13350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01add10 con 0x558f2fd06f60 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01ad640 con 0x558f2fd00f20 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc01aa840 con 0x558f2fd06640 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 10 client.0 _ll_register_callbacks cb 0x558f2fbec920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48162/0 (socket says 172.21.15.52:48162) 2024-06-02T23:17:31.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 -- 172.21.15.52:0/502139583 learned_addr learned my addr 172.21.15.52:0/502139583 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd13350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 -- 172.21.15.52:0/502139583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 msgr2=0x558f2fd13350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd13350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 -- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 msgr2=0x558f2fd12e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd12e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 -- 172.21.15.52:0/502139583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f2fd10350 con 0x558f2fd06f60 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cb7fff700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd13350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cc54ef700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0fde0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8021470 tx=0x7f6cb800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f54d3e29702c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6c9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6c9ffff700 1 -- 172.21.15.52:0/502139583 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb803c060 con 0x558f2fd06f60 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6c9ffff700 1 -- 172.21.15.52:0/502139583 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cb800ee40 con 0x558f2fd06f60 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.097+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb0024070 con 0x558f2fd06f60 2024-06-02T23:17:31.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6c9ffff700 1 -- 172.21.15.52:0/502139583 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6cb8014030 con 0x558f2fd06f60 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 10 client.17178 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558f2fd859b0 con 0x558f2fd06f60 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6c9ffff700 1 -- 172.21.15.52:0/502139583 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb0024070 con 0x558f2fd06f60 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6c9ffff700 1 -- 172.21.15.52:0/502139583 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6cb80199f0 con 0x558f2fd06f60 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:17:31.658 INFO:teuthology.orchestra.run.smithi052.stdout: "7a38893b-611d-4743-b4bf-af41582031df": { 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "7a38893b-611d-4743-b4bf-af41582031df", 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 20 client.17178 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd88000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 4 client.17178 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558f2fc5a290 con 0x558f2fd1a7c0 2024-06-02T23:17:31.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.098+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd88000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cc4cee700 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd88000 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7835969fab932dcd server_cookie=9cdc062f6ec684ac in_seq=0 out_seq=0 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6c9ffff700 10 client.17178 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6c9ffff700 1 -- 172.21.15.52:0/502139583 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558f2fc5a290 con 0x558f2fd1a7c0 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6c9ffff700 10 client.17178 handle_command_reply: tid=0 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 client.17178 shutdown 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 msgr2=0x558f2fd0fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0fde0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8021470 tx=0x7f6cb800c9a0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 shutdown_connections 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558f2fd1a7c0 0x558f2fd88000 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f2fd06f60 0x558f2fd0fde0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f2fd00f20 0x558f2fd12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 --2- 172.21.15.52:0/502139583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f2fd06640 0x558f2fd13350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6c9ffff700 0 client.17178 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 >> 172.21.15.52:0/502139583 conn(0x558f2fc2bcc0 msgr2=0x558f2fd07a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:31.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 shutdown_connections 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 1 -- 172.21.15.52:0/502139583 wait complete. 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.099+0000 7f6cd17d5b80 20 client.17178 trim_cache size 0 max 16384 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca80a2ca0 msgr2=0x7f6ca80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca80a2ca0 0x7f6ca80a5160 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0005d90 tx=0x7f6cb0005d00 comp rx=0 tx=0).stop 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 msgr2=0x7f6cc019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 0x7f6cc019e780 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000c00 tx=0x7f6cbc003b30 comp rx=0 tx=0).stop 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 shutdown_connections 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca80a2ca0 0x7f6ca80a5160 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cc01073f0 0x7f6cc0198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cc0101380 0x7f6cc019e780 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 --2- 172.21.15.52:0/4102816516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cc0101d40 0x7f6cc019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 >> 172.21.15.52:0/4102816516 conn(0x7f6cc0078300 msgr2=0x7f6cc0100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:31.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 shutdown_connections 2024-06-02T23:17:31.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:31.100+0000 7f6cc6f52700 1 -- 172.21.15.52:0/4102816516 wait complete. 2024-06-02T23:17:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:31 smithi174 ceph-mon[35994]: pgmap v600: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-06-02T23:17:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:31 smithi052 ceph-mon[32340]: pgmap v600: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-06-02T23:17:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:31 smithi123 ceph-mon[35977]: pgmap v600: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.8 KiB/s wr, 19 op/s 2024-06-02T23:17:32.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.90 MB/sec execute 161 sec latency 19106.445 ms 2024-06-02T23:17:33.034 INFO:tasks.cephfs.filesystem:scrub status for tag:7a38893b-611d-4743-b4bf-af41582031df - {'path': '/', 'tag': '7a38893b-611d-4743-b4bf-af41582031df', 'options': 'recursive,force'} 2024-06-02T23:17:33.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.89 MB/sec execute 162 sec latency 20106.531 ms 2024-06-02T23:17:33.776 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:33 smithi052 ceph-mon[32340]: pgmap v601: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-06-02T23:17:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:33 smithi174 ceph-mon[35994]: pgmap v601: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-06-02T23:17:34.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:33 smithi123 ceph-mon[35977]: pgmap v601: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.7 KiB/s wr, 21 op/s 2024-06-02T23:17:34.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 -- 172.21.15.52:0/3381318655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a501030f0 msgr2=0x7f2a50107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 --2- 172.21.15.52:0/3381318655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a501030f0 0x7f2a50107110 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2a400099c0 tx=0x7f2a40009cd0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 -- 172.21.15.52:0/3381318655 shutdown_connections 2024-06-02T23:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 --2- 172.21.15.52:0/3381318655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a50107840 0x7f2a5010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 --2- 172.21.15.52:0/3381318655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a50102740 0x7f2a50102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 --2- 172.21.15.52:0/3381318655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a501030f0 0x7f2a50107110 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 -- 172.21.15.52:0/3381318655 >> 172.21.15.52:0/3381318655 conn(0x7f2a500795f0 msgr2=0x7f2a50079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 -- 172.21.15.52:0/3381318655 shutdown_connections 2024-06-02T23:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.014+0000 7f2a57f36700 1 -- 172.21.15.52:0/3381318655 wait complete. 2024-06-02T23:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 Processor -- start 2024-06-02T23:17:34.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 -- start start 2024-06-02T23:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 0x7f2a5019e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a501030f0 0x7f2a5019ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a50107840 0x7f2a50198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a50112620 con 0x7f2a50102740 2024-06-02T23:17:34.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501124a0 con 0x7f2a50107840 2024-06-02T23:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a57f36700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a501127a0 con 0x7f2a501030f0 2024-06-02T23:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a55cd2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 0x7f2a5019e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a55cd2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 0x7f2a5019e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45180/0 (socket says 172.21.15.52:45180) 2024-06-02T23:17:34.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a55cd2700 1 -- 172.21.15.52:0/2527323875 learned_addr learned my addr 172.21.15.52:0/2527323875 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a554d1700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a501030f0 0x7f2a5019ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a564d3700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a50107840 0x7f2a50198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a55cd2700 1 -- 172.21.15.52:0/2527323875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a501030f0 msgr2=0x7f2a5019ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.015+0000 7f2a55cd2700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a501030f0 0x7f2a5019ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a55cd2700 1 -- 172.21.15.52:0/2527323875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a50107840 msgr2=0x7f2a50198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:34.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a55cd2700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a50107840 0x7f2a50198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a55cd2700 1 -- 172.21.15.52:0/2527323875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a50199060 con 0x7f2a50102740 2024-06-02T23:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a55cd2700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 0x7f2a5019e740 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00d950 tx=0x7f2a4c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c016070 con 0x7f2a50102740 2024-06-02T23:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a50199350 con 0x7f2a50102740 2024-06-02T23:17:34.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a4c009df0 con 0x7f2a50102740 2024-06-02T23:17:34.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00f3c0 con 0x7f2a50102740 2024-06-02T23:17:34.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.016+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a501a5610 con 0x7f2a50102740 2024-06-02T23:17:34.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.017+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a50069180 con 0x7f2a50102740 2024-06-02T23:17:34.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.018+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a4c019070 con 0x7f2a50102740 2024-06-02T23:17:34.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.019+0000 7f2a46ffd700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a3c0a2c20 0x7f2a3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.019+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2a4c0cebd0 con 0x7f2a50102740 2024-06-02T23:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.019+0000 7f2a554d1700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a3c0a2c20 0x7f2a3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.019+0000 7f2a554d1700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a3c0a2c20 0x7f2a3c0a50e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a501020d0 tx=0x7f2a40005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:34.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.023+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2a4c09bb60 con 0x7f2a50102740 2024-06-02T23:17:34.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.278+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a50199e60 con 0x7f2a50102740 2024-06-02T23:17:34.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.278+0000 7f2a46ffd700 1 -- 172.21.15.52:0/2527323875 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f2a50199e60 con 0x7f2a50102740 2024-06-02T23:17:34.279 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:17:34.279 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:17:34.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a3c0a2c20 msgr2=0x7f2a3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:34.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a3c0a2c20 0x7f2a3c0a50e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2a501020d0 tx=0x7f2a40005fb0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 msgr2=0x7f2a5019e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:34.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 0x7f2a5019e740 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00d950 tx=0x7f2a4c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:17:34.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 shutdown_connections 2024-06-02T23:17:34.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a3c0a2c20 0x7f2a3c0a50e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a50107840 0x7f2a50198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a50102740 0x7f2a5019e740 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 --2- 172.21.15.52:0/2527323875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a501030f0 0x7f2a5019ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 >> 172.21.15.52:0/2527323875 conn(0x7f2a500795f0 msgr2=0x7f2a50100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 shutdown_connections 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:34.281+0000 7f2a57f36700 1 -- 172.21.15.52:0/2527323875 wait complete. 2024-06-02T23:17:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:17:34.629 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.88 MB/sec execute 163 sec latency 21106.610 ms 2024-06-02T23:17:34.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:34 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2527323875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:34.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:34 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2527323875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:35.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:34 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2527323875' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:35.065 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id b117150 --release --await' 2024-06-02T23:17:35.066 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id b117150 --release --await 2024-06-02T23:17:35.328 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:35.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.87 MB/sec execute 164 sec latency 22106.637 ms 2024-06-02T23:17:35.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:35 smithi052 ceph-mon[32340]: pgmap v602: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 KiB/s wr, 22 op/s 2024-06-02T23:17:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:35 smithi174 ceph-mon[35994]: pgmap v602: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 KiB/s wr, 22 op/s 2024-06-02T23:17:36.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:35 smithi123 ceph-mon[35977]: pgmap v602: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 8.3 KiB/s wr, 22 op/s 2024-06-02T23:17:36.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105366 1.86 MB/sec execute 165 sec latency 23106.674 ms 2024-06-02T23:17:36.783 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:17:36.783 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:17:36.783 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:17:36.783 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:17:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:17:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:17:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:17:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:17:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:17:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:17:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:17:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:17:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 -- 172.21.15.52:0/856914578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac06aa00 msgr2=0x7f02ac06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 --2- 172.21.15.52:0/856914578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac06aa00 0x7f02ac06ae80 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0294009990 tx=0x7f0294009ca0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 -- 172.21.15.52:0/856914578 shutdown_connections 2024-06-02T23:17:37.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 --2- 172.21.15.52:0/856914578 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac06b3c0 0x7f02ac10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 --2- 172.21.15.52:0/856914578 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02ac1084f0 0x7f02ac1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 --2- 172.21.15.52:0/856914578 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac06aa00 0x7f02ac06ae80 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 -- 172.21.15.52:0/856914578 >> 172.21.15.52:0/856914578 conn(0x7f02ac078300 msgr2=0x7f02ac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 -- 172.21.15.52:0/856914578 shutdown_connections 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.045+0000 7f02b0feb700 1 -- 172.21.15.52:0/856914578 wait complete. 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.046+0000 7f02b0feb700 1 Processor -- start 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.046+0000 7f02b0feb700 1 -- start start 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02b0feb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02ac06aa00 0x7f02ac19e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02b0feb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac06b3c0 0x7f02ac19ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02b0feb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 0x7f02ac198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02b0feb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac112620 con 0x7f02ac1084f0 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02b0feb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1124a0 con 0x7f02ac06b3c0 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02b0feb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1127a0 con 0x7f02ac06aa00 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 0x7f02ac198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 0x7f02ac198980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45188/0 (socket says 172.21.15.52:45188) 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 -- 172.21.15.52:0/482832539 learned_addr learned my addr 172.21.15.52:0/482832539 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac06b3c0 0x7f02ac19ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aa59c700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02ac06aa00 0x7f02ac19e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 -- 172.21.15.52:0/482832539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02ac06aa00 msgr2=0x7f02ac19e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02ac06aa00 0x7f02ac19e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 -- 172.21.15.52:0/482832539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac06b3c0 msgr2=0x7f02ac19ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac06b3c0 0x7f02ac19ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.047+0000 7f02aad9d700 1 -- 172.21.15.52:0/482832539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02ac1990e0 con 0x7f02ac1084f0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.048+0000 7f02aad9d700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 0x7f02ac198980 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f02a400d980 tx=0x7f02a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.048+0000 7f02a37fe700 1 -- 172.21.15.52:0/482832539 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a4017070 con 0x7f02ac1084f0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.048+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02ac1993d0 con 0x7f02ac1084f0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.048+0000 7f02a37fe700 1 -- 172.21.15.52:0/482832539 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f02a4005020 con 0x7f02ac1084f0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.048+0000 7f02a37fe700 1 -- 172.21.15.52:0/482832539 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a400fa90 con 0x7f02ac1084f0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.048+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02ac1a5690 con 0x7f02ac1084f0 2024-06-02T23:17:37.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.051+0000 7f02a37fe700 1 -- 172.21.15.52:0/482832539 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02a40051d0 con 0x7f02ac1084f0 2024-06-02T23:17:37.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.052+0000 7f02a37fe700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02980a2c20 0x7f02980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.052+0000 7f02aa59c700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02980a2c20 0x7f02980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.052+0000 7f02a37fe700 1 -- 172.21.15.52:0/482832539 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f02a40cee50 con 0x7f02ac1084f0 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.052+0000 7f02bb86eb80 1 Processor -- start 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.052+0000 7f02aa59c700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02980a2c20 0x7f02980a50e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02ac199ac0 tx=0x7f029c009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 -- start start 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407050 0x558fbc407430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 0x558fbc4012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc401c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558fbc34e910 con 0x558fbc407970 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558fbc2f6820 con 0x558fbc4017e0 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fbc34a8d0 con 0x558fbc407050 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 0x558fbc4012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 0x558fbc4012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45190/0 (socket says 172.21.15.52:45190) 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 -- 172.21.15.52:0/2324227733 learned_addr learned my addr 172.21.15.52:0/2324227733 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02aad9d700 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc401c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02aa59c700 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407050 0x558fbc407430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 -- 172.21.15.52:0/2324227733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407050 msgr2=0x558fbc407430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407050 0x558fbc407430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 -- 172.21.15.52:0/2324227733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 msgr2=0x558fbc401c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc401c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.053+0000 7f02a9d9b700 1 -- 172.21.15.52:0/2324227733 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbc2ed050 con 0x558fbc407970 2024-06-02T23:17:37.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.054+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 0x558fbc4012a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0294009990 tx=0x7f0294009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a40df96492d0ec6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:37.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.054+0000 7f02a0ff9700 1 -- 172.21.15.52:0/2324227733 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0294003a50 con 0x558fbc407970 2024-06-02T23:17:37.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.054+0000 7f02a0ff9700 1 -- 172.21.15.52:0/2324227733 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0294026070 con 0x558fbc407970 2024-06-02T23:17:37.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.054+0000 7f02a0ff9700 1 -- 172.21.15.52:0/2324227733 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400ba30 con 0x558fbc407970 2024-06-02T23:17:37.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.054+0000 7f02bb86eb80 1 -- 172.21.15.52:0/2324227733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 msgr2=0x558fbc4012a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.054+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 0x558fbc4012a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f0294009990 tx=0x7f0294009f90 comp rx=0 tx=0).stop 2024-06-02T23:17:37.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- 172.21.15.52:0/2324227733 shutdown_connections 2024-06-02T23:17:37.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc401c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407050 0x558fbc407430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/2324227733 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407970 0x558fbc4012a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- 172.21.15.52:0/2324227733 >> 172.21.15.52:0/2324227733 conn(0x558fbc327dc0 msgr2=0x558fbc3f8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- 172.21.15.52:0/2324227733 shutdown_connections 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- 172.21.15.52:0/2324227733 wait complete. 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- start start 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 Processor -- start 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc40d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc40a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc40af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558fbc40e380 con 0x558fbc407050 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558fbc2eecf0 con 0x558fbc4017e0 2024-06-02T23:17:37.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fbc40e500 con 0x558fbc407970 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02bb86eb80 10 client.0 _ll_register_callbacks cb 0x558fbc2e8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.055+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc40a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc40a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45192/0 (socket says 172.21.15.52:45192) 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 -- 172.21.15.52:0/1677787530 learned_addr learned my addr 172.21.15.52:0/1677787530 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02aa59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc40d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02aad9d700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc40af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 -- 172.21.15.52:0/1677787530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 msgr2=0x558fbc40af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc40af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 -- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 msgr2=0x558fbc40d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc40d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 -- 172.21.15.52:0/1677787530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbc29d9d0 con 0x558fbc407050 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02aa59c700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc40d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc40a9d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0294005850 tx=0x7f0294005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdb3df1d612d66ce server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f0291ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:37.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f0291ffb700 1 -- 172.21.15.52:0/1677787530 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029402e070 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f0291ffb700 1 -- 172.21.15.52:0/1677787530 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f029400b9d0 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0294026070 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f0291ffb700 1 -- 172.21.15.52:0/1677787530 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f029403a070 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02bb86eb80 10 client.17196 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.056+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558fbc418c80 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f0291ffb700 1 -- 172.21.15.52:0/1677787530 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0294026070 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f0291ffb700 1 -- 172.21.15.52:0/1677787530 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0294033aa0 con 0x558fbc407050 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02bb86eb80 10 client.17196 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02bb86eb80 10 client.17196 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02bb86eb80 20 client.17196 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc41d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02bb86eb80 4 client.17196 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fbc35be10 con 0x558fbc41b150 2024-06-02T23:17:37.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.057+0000 7f02aa59c700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc41d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.058+0000 7f0291ffb700 10 client.17196 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.058+0000 7f02aa59c700 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc41d540 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97b0ab6140c6d5d3 server_cookie=3f7e6b4c0b73ee0b in_seq=0 out_seq=0 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.058+0000 7f0291ffb700 1 -- 172.21.15.52:0/1677787530 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558fbc35be10 con 0x558fbc41b150 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.058+0000 7f0291ffb700 10 client.17196 handle_command_reply: tid=0 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.058+0000 7f02bb86eb80 1 client.17196 shutdown 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 msgr2=0x558fbc40a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc40a9d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f0294005850 tx=0x7f0294005280 comp rx=0 tx=0).stop 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 shutdown_connections 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc41d540 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc40af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f0291ffb700 0 client.17196 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc40d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/1677787530 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc40a9d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 >> 172.21.15.52:0/1677787530 conn(0x558fbc327dc0 msgr2=0x558fbc3fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 shutdown_connections 2024-06-02T23:17:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 1 -- 172.21.15.52:0/1677787530 wait complete. 2024-06-02T23:17:37.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.059+0000 7f02bb86eb80 20 client.17196 trim_cache size 0 max 16384 2024-06-02T23:17:37.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 Processor -- start 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 -- start start 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc4017e0 0x558fbc409b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc408480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc407970 0x558fbc3f7f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1aa6b0 con 0x558fbc407050 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1ac1f0 con 0x558fbc407970 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.079+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1aad80 con 0x558fbc4017e0 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc408480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc408480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45200/0 (socket says 172.21.15.52:45200) 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 -- 172.21.15.52:0/3440235447 learned_addr learned my addr 172.21.15.52:0/3440235447 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02aad9d700 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc407970 0x558fbc3f7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02aa59c700 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc4017e0 0x558fbc409b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 -- 172.21.15.52:0/3440235447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc4017e0 msgr2=0x558fbc409b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc4017e0 0x558fbc409b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 -- 172.21.15.52:0/3440235447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc407970 msgr2=0x558fbc3f7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc407970 0x558fbc3f7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 -- 172.21.15.52:0/3440235447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fbc408f40 con 0x558fbc407050 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc408480 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0294009ce0 tx=0x7f0294009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedb5aa14acc9e54 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f0293fff700 1 -- 172.21.15.52:0/3440235447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029402e040 con 0x558fbc407050 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f0293fff700 1 -- 172.21.15.52:0/3440235447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0294026070 con 0x558fbc407050 2024-06-02T23:17:37.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.080+0000 7f0293fff700 1 -- 172.21.15.52:0/3440235447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400bbd0 con 0x558fbc407050 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 -- 172.21.15.52:0/3440235447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 msgr2=0x558fbc408480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc408480 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f0294009ce0 tx=0x7f0294009810 comp rx=0 tx=0).stop 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 -- 172.21.15.52:0/3440235447 shutdown_connections 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc407970 0x558fbc3f7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc4017e0 0x558fbc409b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/3440235447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc408480 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 -- 172.21.15.52:0/3440235447 >> 172.21.15.52:0/3440235447 conn(0x558fbc327dc0 msgr2=0x558fbc3f7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 -- 172.21.15.52:0/3440235447 shutdown_connections 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 -- 172.21.15.52:0/3440235447 wait complete. 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 -- start start 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.081+0000 7f02bb86eb80 1 Processor -- start 2024-06-02T23:17:37.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc41ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc41ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc484a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1abb20 con 0x558fbc407050 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1ab450 con 0x558fbc4017e0 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02ac1a8670 con 0x558fbc407970 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 10 client.0 _ll_register_callbacks cb 0x558fbc2e8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc41ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc41ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45216/0 (socket says 172.21.15.52:45216) 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 -- 172.21.15.52:0/718951478 learned_addr learned my addr 172.21.15.52:0/718951478 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02aad9d700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc484a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02aa59c700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc41ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 -- 172.21.15.52:0/718951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 msgr2=0x558fbc484a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc484a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 -- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 msgr2=0x558fbc41ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc41ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 -- 172.21.15.52:0/718951478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fbc484fd0 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02a9d9b700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc41ef90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0294009350 tx=0x7f029400f490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ec7ced4649a5137 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f0291ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f0291ffb700 1 -- 172.21.15.52:0/718951478 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029400fd70 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f0291ffb700 1 -- 172.21.15.52:0/718951478 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f029400b7d0 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.082+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0294026070 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f0291ffb700 1 -- 172.21.15.52:0/718951478 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f029400f6d0 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 10 client.17202 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558fbc486280 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f0291ffb700 1 -- 172.21.15.52:0/718951478 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0294026070 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f0291ffb700 1 -- 172.21.15.52:0/718951478 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0294033a10 con 0x558fbc407050 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 10 client.17202 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 10 client.17202 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 20 client.17202 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:37.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc488730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:37.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 4 client.17202 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b117150", "release": true, "await": true}] 2024-06-02T23:17:37.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b117150", "release": true, "await": true}) -- 0x558fbc356390 con 0x558fbc41b150 2024-06-02T23:17:37.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.083+0000 7f02aa59c700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc488730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:37.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.084+0000 7f02aa59c700 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc488730 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=562027235955220 server_cookie=9e903643a176e9d9 in_seq=0 out_seq=0 2024-06-02T23:17:37.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.084+0000 7f0291ffb700 10 client.17202 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f0291ffb700 1 -- 172.21.15.52:0/718951478 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558fbc356390 con 0x558fbc41b150 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f0291ffb700 10 client.17202 handle_command_reply: tid=0 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f02bb86eb80 1 client.17202 shutdown 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 msgr2=0x558fbc41ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc41ef90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f0294009350 tx=0x7f029400f490 comp rx=0 tx=0).stop 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 shutdown_connections 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558fbc41b150 0x558fbc488730 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.090+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fbc407970 0x558fbc484a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558fbc4017e0 0x558fbc41ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02bb86eb80 1 --2- 172.21.15.52:0/718951478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558fbc407050 0x558fbc41ef90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 >> 172.21.15.52:0/718951478 conn(0x558fbc327dc0 msgr2=0x558fbc3f80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f0291ffb700 0 client.17202 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 shutdown_connections 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02bb86eb80 1 -- 172.21.15.52:0/718951478 wait complete. 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02bb86eb80 20 client.17202 trim_cache size 0 max 16384 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 20, 2024-06-02T23:17:37.092 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "b117150": { 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 20, 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 144.1, 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:17:37.093 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:17:37.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02980a2c20 msgr2=0x7f02980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02980a2c20 0x7f02980a50e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f02ac199ac0 tx=0x7f029c009180 comp rx=0 tx=0).stop 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 msgr2=0x7f02ac198980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 0x7f02ac198980 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f02a400d980 tx=0x7f02a400dc90 comp rx=0 tx=0).stop 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 shutdown_connections 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02980a2c20 0x7f02980a50e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f02ac1084f0 0x7f02ac198980 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02ac06aa00 0x7f02ac19e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 --2- 172.21.15.52:0/482832539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f02ac06b3c0 0x7f02ac19ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.091+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 >> 172.21.15.52:0/482832539 conn(0x7f02ac078300 msgr2=0x7f02ac100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.092+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 shutdown_connections 2024-06-02T23:17:37.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:37.092+0000 7f02b0feb700 1 -- 172.21.15.52:0/482832539 wait complete. 2024-06-02T23:17:37.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 105684 1.87 MB/sec execute 166 sec latency 23569.978 ms 2024-06-02T23:17:37.781 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:37 smithi052 ceph-mon[32340]: pgmap v603: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2024-06-02T23:17:37.810 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b117150', total seconds elapsed: 32.6 2024-06-02T23:17:37.810 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 24.05 2024-06-02T23:17:37.810 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:17:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:37 smithi174 ceph-mon[35994]: pgmap v603: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2024-06-02T23:17:38.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:37 smithi123 ceph-mon[35977]: pgmap v603: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 6.0 KiB/s wr, 21 op/s 2024-06-02T23:17:38.090 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:38.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 106239 1.88 MB/sec execute 167 sec latency 15.408 ms 2024-06-02T23:17:39.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 106706 1.89 MB/sec execute 168 sec latency 60.249 ms 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 -- 172.21.15.52:0/2694943460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548101390 msgr2=0x7f1548101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2694943460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548101390 0x7f1548101810 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f1538009990 tx=0x7f1538009ca0 comp rx=0 tx=0).stop 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 -- 172.21.15.52:0/2694943460 shutdown_connections 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2694943460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101d50 0x7f154810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2694943460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1548107400 0x7f15481077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2694943460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548101390 0x7f1548101810 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:39.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 -- 172.21.15.52:0/2694943460 >> 172.21.15.52:0/2694943460 conn(0x7f1548078340 msgr2=0x7f1548078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:39.808 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:39 smithi052 ceph-mon[32340]: pgmap v604: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-06-02T23:17:39.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.807+0000 7f154f8d0700 1 -- 172.21.15.52:0/2694943460 shutdown_connections 2024-06-02T23:17:39.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.808+0000 7f154f8d0700 1 -- 172.21.15.52:0/2694943460 wait complete. 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.808+0000 7f154f8d0700 1 Processor -- start 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.808+0000 7f154f8d0700 1 -- start start 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154f8d0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 0x7f154819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154f8d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1548101d50 0x7f154819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154f8d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548107400 0x7f1548198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154f8d0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1548111570 con 0x7f1548107400 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154f8d0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15481113f0 con 0x7f1548101d50 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154f8d0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15481116f0 con 0x7f1548101390 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 0x7f154819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 0x7f154819e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44944/0 (socket says 172.21.15.52:44944) 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 -- 172.21.15.52:0/2269069466 learned_addr learned my addr 172.21.15.52:0/2269069466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154ce6b700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1548101d50 0x7f154819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154de6d700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548107400 0x7f1548198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:39.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 -- 172.21.15.52:0/2269069466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1548101d50 msgr2=0x7f154819ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1548101d50 0x7f154819ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 -- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548107400 msgr2=0x7f1548198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548107400 0x7f1548198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.809+0000 7f154d66c700 1 -- 172.21.15.52:0/2269069466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15481990c0 con 0x7f1548101390 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.810+0000 7f154d66c700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 0x7f154819e7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f154400d950 tx=0x7f154400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.810+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1544016070 con 0x7f1548101390 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.810+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15481993b0 con 0x7f1548101390 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.810+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f154400fd40 con 0x7f1548101390 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.810+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154400e3b0 con 0x7f1548101390 2024-06-02T23:17:39.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.810+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15481a5670 con 0x7f1548101390 2024-06-02T23:17:39.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.811+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1548069180 con 0x7f1548101390 2024-06-02T23:17:39.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.812+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1544019070 con 0x7f1548101390 2024-06-02T23:17:39.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.813+0000 7f153e7fc700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15340a2c20 0x7f15340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:39.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.813+0000 7f154ce6b700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15340a2c20 0x7f15340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:39.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.813+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f15440cead0 con 0x7f1548101390 2024-06-02T23:17:39.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.814+0000 7f154ce6b700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15340a2c20 0x7f15340a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15480788b0 tx=0x7f153800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:39.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:39.818+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f154409ba00 con 0x7f1548101390 2024-06-02T23:17:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:39 smithi174 ceph-mon[35994]: pgmap v604: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-06-02T23:17:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:39 smithi123 ceph-mon[35977]: pgmap v604: 97 pgs: 97 active+clean; 56 MiB data, 918 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 8.4 KiB/s wr, 23 op/s 2024-06-02T23:17:40.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.104+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15481a5960 con 0x7f1548101390 2024-06-02T23:17:40.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.105+0000 7f153e7fc700 1 -- 172.21.15.52:0/2269069466 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f154409b300 con 0x7f1548101390 2024-06-02T23:17:40.105 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:17:40.105 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15340a2c20 msgr2=0x7f15340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15340a2c20 0x7f15340a50e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f15480788b0 tx=0x7f153800b540 comp rx=0 tx=0).stop 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 msgr2=0x7f154819e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 0x7f154819e7a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f154400d950 tx=0x7f154400dc60 comp rx=0 tx=0).stop 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 shutdown_connections 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15340a2c20 0x7f15340a50e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1548107400 0x7f1548198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1548101390 0x7f154819e7a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 --2- 172.21.15.52:0/2269069466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1548101d50 0x7f154819ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 >> 172.21.15.52:0/2269069466 conn(0x7f1548078340 msgr2=0x7f1548100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 shutdown_connections 2024-06-02T23:17:40.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:40.107+0000 7f154f8d0700 1 -- 172.21.15.52:0/2269069466 wait complete. 2024-06-02T23:17:40.109 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:17:40.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 106950 1.88 MB/sec execute 169 sec latency 15.155 ms 2024-06-02T23:17:40.809 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:40 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2269069466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:40.839 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 144.05 --await-for 45.0' 2024-06-02T23:17:40.839 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 144.05 --await-for 45.0 2024-06-02T23:17:40.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:40 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2269069466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:41.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:40 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2269069466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:17:41.107 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:17:41.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 107379 1.88 MB/sec execute 170 sec latency 15.728 ms 2024-06-02T23:17:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:41 smithi174 ceph-mon[35994]: pgmap v605: 97 pgs: 97 active+clean; 57 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-06-02T23:17:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:41 smithi052 ceph-mon[32340]: pgmap v605: 97 pgs: 97 active+clean; 57 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-06-02T23:17:41.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 -- 172.21.15.52:0/3051053007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff910102480 msgr2=0x7ff910104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 --2- 172.21.15.52:0/3051053007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff910102480 0x7ff910104870 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff900009900 tx=0x7ff90002a410 comp rx=0 tx=0).stop 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 -- 172.21.15.52:0/3051053007 shutdown_connections 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 --2- 172.21.15.52:0/3051053007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9101076e0 0x7ff910109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 --2- 172.21.15.52:0/3051053007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910104db0 0x7ff9101071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 --2- 172.21.15.52:0/3051053007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff910102480 0x7ff910104870 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 -- 172.21.15.52:0/3051053007 >> 172.21.15.52:0/3051053007 conn(0x7ff9100fbe70 msgr2=0x7ff9100fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 -- 172.21.15.52:0/3051053007 shutdown_connections 2024-06-02T23:17:41.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.908+0000 7ff915190640 1 -- 172.21.15.52:0/3051053007 wait complete. 2024-06-02T23:17:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 Processor -- start 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 -- start start 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910102480 0x7ff910197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff910104db0 0x7ff9101980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 0x7ff91019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010d550 con 0x7ff9101076e0 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010d3d0 con 0x7ff910102480 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff915190640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff91010d6d0 con 0x7ff910104db0 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff90f59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 0x7ff91019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff90f59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 0x7ff91019f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45246/0 (socket says 172.21.15.52:45246) 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff90f59e640 1 -- 172.21.15.52:0/2885181628 learned_addr learned my addr 172.21.15.52:0/2885181628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910102480 0x7ff910197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.909+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff910104db0 0x7ff9101980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90f59e640 1 -- 172.21.15.52:0/2885181628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff910104db0 msgr2=0x7ff9101980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90f59e640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff910104db0 0x7ff9101980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90f59e640 1 -- 172.21.15.52:0/2885181628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910102480 msgr2=0x7ff910197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90f59e640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910102480 0x7ff910197ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90f59e640 1 -- 172.21.15.52:0/2885181628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91019f7c0 con 0x7ff9101076e0 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910102480 0x7ff910197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff90f59e640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 0x7ff91019f150 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff90800eaa0 tx=0x7ff90800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff8f7fff640 1 -- 172.21.15.52:0/2885181628 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90800cdb0 con 0x7ff9101076e0 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91019fab0 con 0x7ff9101076e0 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff8f7fff640 1 -- 172.21.15.52:0/2885181628 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff908005020 con 0x7ff9101076e0 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff8f7fff640 1 -- 172.21.15.52:0/2885181628 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9080116d0 con 0x7ff9101076e0 2024-06-02T23:17:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.910+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9101a0050 con 0x7ff9101076e0 2024-06-02T23:17:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff8f7fff640 1 -- 172.21.15.52:0/2885181628 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9080051d0 con 0x7ff9101076e0 2024-06-02T23:17:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 Processor -- start 2024-06-02T23:17:41.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 -- start start 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9cd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5603bf9274f0 con 0x5603bf9cdf80 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603bf8c5870 con 0x5603bf9c8420 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff917f7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603bf9234b0 con 0x5603bf9c8e10 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.913+0000 7ff8f7fff640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff8f80a2c30 0x7ff8f80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46932/0 (socket says 172.21.15.52:46932) 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44974/0 (socket says 172.21.15.52:44974) 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90e59c640 1 -- 172.21.15.52:0/3265729243 learned_addr learned my addr 172.21.15.52:0/3265729243 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90e59c640 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90f59e640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff8f80a2c30 0x7ff8f80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff8f7fff640 1 -- 172.21.15.52:0/2885181628 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff9080cf110 con 0x7ff9101076e0 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 -- 172.21.15.52:0/3265729243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 msgr2=0x5603bf9c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9c8840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 -- 172.21.15.52:0/3265729243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 msgr2=0x5603bf9d0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 -- 172.21.15.52:0/3265729243 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603bf8e19b0 con 0x5603bf9c8e10 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90e59c640 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9cd910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9000314e0 tx=0x7ff900031580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=77c8b61e553b9a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff90f59e640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff8f80a2c30 0x7ff8f80a50f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff90809b9e0 tx=0x7ff908086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff8f57fa640 1 -- 172.21.15.52:0/3265729243 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff900035070 con 0x5603bf9c8e10 2024-06-02T23:17:41.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff8f57fa640 1 -- 172.21.15.52:0/3265729243 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9000318f0 con 0x5603bf9c8e10 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.914+0000 7ff8f57fa640 1 -- 172.21.15.52:0/3265729243 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90003c440 con 0x5603bf9c8e10 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 -- 172.21.15.52:0/3265729243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 msgr2=0x5603bf9cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9cd910 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff9000314e0 tx=0x7ff900031580 comp rx=0 tx=0).stop 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 -- 172.21.15.52:0/3265729243 shutdown_connections 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9cd910 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3265729243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 -- 172.21.15.52:0/3265729243 >> 172.21.15.52:0/3265729243 conn(0x5603bf9bf680 msgr2=0x5603bf9bfa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 -- 172.21.15.52:0/3265729243 shutdown_connections 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 -- 172.21.15.52:0/3265729243 wait complete. 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.915+0000 7ff917f7f740 1 -- start start 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 Processor -- start 2024-06-02T23:17:41.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8420 0x5603bf9d2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8e10 0x5603bf9d2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5603bf9d0ed0 con 0x5603bf9cdf80 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5603bf8b0ae0 con 0x5603bf9c8e10 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5603bf9d1050 con 0x5603bf9c8420 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff917f7f740 10 client.0 _ll_register_callbacks cb 0x5603bf8e99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45256/0 (socket says 172.21.15.52:45256) 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8e10 0x5603bf9d2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 -- 172.21.15.52:0/2893833677 learned_addr learned my addr 172.21.15.52:0/2893833677 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8420 0x5603bf9d2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 -- 172.21.15.52:0/2893833677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8420 msgr2=0x5603bf9d2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8420 0x5603bf9d2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 -- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8e10 msgr2=0x5603bf9d2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8e10 0x5603bf9d2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 -- 172.21.15.52:0/2893833677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603bf63b210 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8e10 0x5603bf9d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d9be0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff904011d30 tx=0x7ff904004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287a33da3bdec86 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.916+0000 7ff8f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/2893833677 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904004240 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9000318f0 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/2893833677 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff9040043e0 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/2893833677 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff90400fe60 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 10 client.17220 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603bf707620 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/2893833677 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9000318f0 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/2893833677 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff9040224c0 con 0x5603bf9cdf80 2024-06-02T23:17:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 10 client.17220 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 10 client.17220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 20 client.17220 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bf9de8f0 0x5603bf9e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 4 client.17220 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.917+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603bf93af10 con 0x5603bf9de8f0 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.918+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bf9de8f0 0x5603bf9e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.918+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bf9de8f0 0x5603bf9e0d90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3f16b0c8d942379 server_cookie=a1a23d993e6dcc36 in_seq=0 out_seq=0 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.918+0000 7ff8f2ff5640 10 client.17220 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/2893833677 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5603bf93af10 con 0x5603bf9de8f0 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff8f2ff5640 10 client.17220 handle_command_reply: tid=0 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 client.17220 shutdown 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 msgr2=0x5603bf9d9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d9be0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7ff904011d30 tx=0x7ff904004730 comp rx=0 tx=0).stop 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 shutdown_connections 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bf9de8f0 0x5603bf9e0d90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d9be0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff8f2ff5640 0 client.17220 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8e10 0x5603bf9d2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2893833677 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8420 0x5603bf9d2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 >> 172.21.15.52:0/2893833677 conn(0x5603bf9bf680 msgr2=0x5603bf9be7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 shutdown_connections 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 1 -- 172.21.15.52:0/2893833677 wait complete. 2024-06-02T23:17:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.919+0000 7ff917f7f740 20 client.17220 trim_cache size 0 max 16384 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 Processor -- start 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 -- start start 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 0x5603bf9bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9cdf80 0x5603bf9c2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101a6a90 con 0x5603bf9c8420 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101a85d0 con 0x5603bf9cdf80 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.937+0000 7ff917f7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101a7160 con 0x5603bf9c8e10 2024-06-02T23:17:41.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 0x5603bf9bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 0x5603bf9bf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45270/0 (socket says 172.21.15.52:45270) 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 -- 172.21.15.52:0/2957540415 learned_addr learned my addr 172.21.15.52:0/2957540415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9cdf80 0x5603bf9c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 -- 172.21.15.52:0/2957540415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 msgr2=0x5603bf9d9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 -- 172.21.15.52:0/2957540415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9cdf80 msgr2=0x5603bf9c2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9cdf80 0x5603bf9c2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 -- 172.21.15.52:0/2957540415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603bf9c8bb0 con 0x5603bf9c8420 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff90e59c640 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 0x5603bf9bf120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff9040097e0 tx=0x7ff904009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc21ba0a23fd22e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff8f4ff9640 1 -- 172.21.15.52:0/2957540415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904022630 con 0x5603bf9c8420 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff8f4ff9640 1 -- 172.21.15.52:0/2957540415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff904004070 con 0x5603bf9c8420 2024-06-02T23:17:41.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.938+0000 7ff8f4ff9640 1 -- 172.21.15.52:0/2957540415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904003870 con 0x5603bf9c8420 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 -- 172.21.15.52:0/2957540415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 msgr2=0x5603bf9bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 0x5603bf9bf120 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7ff9040097e0 tx=0x7ff904009770 comp rx=0 tx=0).stop 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 -- 172.21.15.52:0/2957540415 shutdown_connections 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9cdf80 0x5603bf9c2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 --2- 172.21.15.52:0/2957540415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9c8420 0x5603bf9bf120 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 -- 172.21.15.52:0/2957540415 >> 172.21.15.52:0/2957540415 conn(0x5603bf9bf680 msgr2=0x5603bf9ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 -- 172.21.15.52:0/2957540415 shutdown_connections 2024-06-02T23:17:41.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 -- 172.21.15.52:0/2957540415 wait complete. 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.939+0000 7ff917f7f740 1 -- start start 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 Processor -- start 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9d6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101a7f00 con 0x5603bf9cdf80 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101a7830 con 0x5603bf9c8420 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9101a4a70 con 0x5603bf9c8e10 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff917f7f740 10 client.0 _ll_register_callbacks cb 0x7ff9040115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9d6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9d6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46954/0 (socket says 172.21.15.52:46954) 2024-06-02T23:17:41.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 -- 172.21.15.52:0/3966031752 learned_addr learned my addr 172.21.15.52:0/3966031752 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 -- 172.21.15.52:0/3966031752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 msgr2=0x5603bf9d7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 -- 172.21.15.52:0/3966031752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 msgr2=0x5603bf9d82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.940+0000 7ff90e59c640 1 -- 172.21.15.52:0/3966031752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603bf9d1720 con 0x5603bf9c8420 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff90e59c640 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9d6de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff90400fc20 tx=0x7ff904009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e9d8874365ed94c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:41.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff8f2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/3966031752 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904022ad0 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/3966031752 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff904022c70 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff904004070 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/3966031752 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff904003a20 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 10 client.26732 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603bf9d9a60 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/3966031752 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff904004070 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/3966031752 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff904005460 con 0x5603bf9c8420 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 10 client.26732 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 10 client.26732 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 20 client.26732 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bfa49730 0x5603bfa4bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:41.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 4 client.26732 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:17:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603bf934c20 con 0x5603bfa49730 2024-06-02T23:17:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.941+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bfa49730 0x5603bfa4bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.942+0000 7ff8f2ff5640 10 client.26732 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:41.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.942+0000 7ff90ed9d640 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bfa49730 0x5603bfa4bb50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45cd1f42a44b5eeb server_cookie=694854807227e2e2 in_seq=0 out_seq=0 2024-06-02T23:17:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff8f2ff5640 1 -- 172.21.15.52:0/3966031752 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5603bf934c20 con 0x5603bfa49730 2024-06-02T23:17:41.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff8f2ff5640 10 client.26732 handle_command_reply: tid=0 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 client.26732 shutdown 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 msgr2=0x5603bf9d6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9d6de0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff90400fc20 tx=0x7ff904009fd0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 shutdown_connections 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5603bfa49730 0x5603bfa4bb50 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5603bf9c8e10 0x5603bf9d7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5603bf9cdf80 0x5603bf9d82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff8f2ff5640 0 client.26732 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 --2- 172.21.15.52:0/3966031752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603bf9c8420 0x5603bf9d6de0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 >> 172.21.15.52:0/3966031752 conn(0x5603bf9bf680 msgr2=0x5603bf9c2650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:41.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 shutdown_connections 2024-06-02T23:17:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 1 -- 172.21.15.52:0/3966031752 wait complete. 2024-06-02T23:17:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.958+0000 7ff917f7f740 20 client.26732 trim_cache size 0 max 16384 2024-06-02T23:17:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:17:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:17:41.960 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:17:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff8f80a2c30 msgr2=0x7ff8f80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff8f80a2c30 0x7ff8f80a50f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff90809b9e0 tx=0x7ff908086040 comp rx=0 tx=0).stop 2024-06-02T23:17:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 msgr2=0x7ff91019f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 0x7ff91019f150 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff90800eaa0 tx=0x7ff90800ef60 comp rx=0 tx=0).stop 2024-06-02T23:17:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 shutdown_connections 2024-06-02T23:17:41.961 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff8f80a2c30 0x7ff8f80a50f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff910104db0 0x7ff9101980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9101076e0 0x7ff91019f150 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 --2- 172.21.15.52:0/2885181628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff910102480 0x7ff910197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.959+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 >> 172.21.15.52:0/2885181628 conn(0x7ff9100fbe70 msgr2=0x7ff910102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.960+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 shutdown_connections 2024-06-02T23:17:41.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:41.960+0000 7ff915190640 1 -- 172.21.15.52:0/2885181628 wait complete. 2024-06-02T23:17:42.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:41 smithi123 ceph-mon[35977]: pgmap v605: 97 pgs: 97 active+clean; 57 MiB data, 923 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 130 KiB/s wr, 23 op/s 2024-06-02T23:17:42.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 107921 1.89 MB/sec execute 171 sec latency 15.574 ms 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 -- 172.21.15.52:0/617518476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d806aec0 msgr2=0x7fc4d806b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/617518476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d806aec0 0x7fc4d806b2a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40099c0 tx=0x7fc4d4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 -- 172.21.15.52:0/617518476 shutdown_connections 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/617518476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d8107910 0x7fc4d810f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/617518476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d806aec0 0x7fc4d806b2a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/617518476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d806b870 0x7fc4d81071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 -- 172.21.15.52:0/617518476 >> 172.21.15.52:0/617518476 conn(0x7fc4d80783e0 msgr2=0x7fc4d80787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:42.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 -- 172.21.15.52:0/617518476 shutdown_connections 2024-06-02T23:17:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.934+0000 7fc4e07c4700 1 -- 172.21.15.52:0/617518476 wait complete. 2024-06-02T23:17:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 Processor -- start 2024-06-02T23:17:42.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 -- start start 2024-06-02T23:17:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d806aec0 0x7fc4d819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d806b870 0x7fc4d819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 0x7fc4d8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d8112740 con 0x7fc4d8107910 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81125c0 con 0x7fc4d806aec0 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4e07c4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81128c0 con 0x7fc4d806b870 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4ded61700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 0x7fc4d8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4de560700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d806aec0 0x7fc4d819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4ded61700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 0x7fc4d8198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45290/0 (socket says 172.21.15.52:45290) 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4ded61700 1 -- 172.21.15.52:0/2097502435 learned_addr learned my addr 172.21.15.52:0/2097502435 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.935+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d806b870 0x7fc4d819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4ded61700 1 -- 172.21.15.52:0/2097502435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d806b870 msgr2=0x7fc4d819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4ded61700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d806b870 0x7fc4d819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4ded61700 1 -- 172.21.15.52:0/2097502435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d806aec0 msgr2=0x7fc4d819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4ded61700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d806aec0 0x7fc4d819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4ded61700 1 -- 172.21.15.52:0/2097502435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4d8199060 con 0x7fc4d8107910 2024-06-02T23:17:42.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4de560700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d806aec0 0x7fc4d819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4ded61700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 0x7fc4d8198900 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000ef30 tx=0x7fc4d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2097502435 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00092e0 con 0x7fc4d8107910 2024-06-02T23:17:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2097502435 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d0004bb0 con 0x7fc4d8107910 2024-06-02T23:17:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4d81992f0 con 0x7fc4d8107910 2024-06-02T23:17:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2097502435 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0004020 con 0x7fc4d8107910 2024-06-02T23:17:42.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.936+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4d81a5610 con 0x7fc4d8107910 2024-06-02T23:17:42.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.939+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2097502435 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc4d001b030 con 0x7fc4d8107910 2024-06-02T23:17:42.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.940+0000 7fc4cf7fe700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4c40a2cc0 0x7fc4c40a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.940+0000 7fc4de560700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4c40a2cc0 0x7fc4c40a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.940+0000 7fc4cf7fe700 1 -- 172.21.15.52:0/2097502435 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc4d00ceaa0 con 0x7fc4d8107910 2024-06-02T23:17:42.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.940+0000 7fc4de560700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4c40a2cc0 0x7fc4c40a5180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000c00 tx=0x7fc4d4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:42.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 Processor -- start 2024-06-02T23:17:42.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 -- start start 2024-06-02T23:17:42.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb35340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb35880 0x563f5eb35cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563f5ea829b0 con 0x563f5eb3b0f0 2024-06-02T23:17:42.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563f5ea2a8c0 con 0x563f5eb3ba10 2024-06-02T23:17:42.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f5ea7e970 con 0x563f5eb35880 2024-06-02T23:17:42.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4de560700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4de560700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45292/0 (socket says 172.21.15.52:45292) 2024-06-02T23:17:42.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4de560700 1 -- 172.21.15.52:0/2546862446 learned_addr learned my addr 172.21.15.52:0/2546862446 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4ded61700 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb35880 0x563f5eb35cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.941+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb35340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4de560700 1 -- 172.21.15.52:0/2546862446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb35880 msgr2=0x563f5eb35cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4de560700 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb35880 0x563f5eb35cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4de560700 1 -- 172.21.15.52:0/2546862446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 msgr2=0x563f5eb35340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4de560700 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb35340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4de560700 1 -- 172.21.15.52:0/2546862446 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5ea210f0 con 0x563f5eb3b0f0 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4ded61700 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb35880 0x563f5eb35cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4de560700 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3b4d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4015000 tx=0x7fc4d4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8591be38331913 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4ccff9700 1 -- 172.21.15.52:0/2546862446 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4005510 con 0x563f5eb3b0f0 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4ccff9700 1 -- 172.21.15.52:0/2546862446 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d40056b0 con 0x563f5eb3b0f0 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.942+0000 7fc4ccff9700 1 -- 172.21.15.52:0/2546862446 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4024a20 con 0x563f5eb3b0f0 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 -- 172.21.15.52:0/2546862446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 msgr2=0x563f5eb3b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3b4d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4015000 tx=0x7fc4d4020000 comp rx=0 tx=0).stop 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 -- 172.21.15.52:0/2546862446 shutdown_connections 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb35880 0x563f5eb35cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3b4d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/2546862446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb35340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 -- 172.21.15.52:0/2546862446 >> 172.21.15.52:0/2546862446 conn(0x563f5ea5be60 msgr2=0x563f5eb2c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:42.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 -- 172.21.15.52:0/2546862446 shutdown_connections 2024-06-02T23:17:42.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 -- 172.21.15.52:0/2546862446 wait complete. 2024-06-02T23:17:42.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.943+0000 7fc4eb047b80 1 -- start start 2024-06-02T23:17:42.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 Processor -- start 2024-06-02T23:17:42.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 0x563f5eb41730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3ba10 0x563f5eb3ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563f5eb423c0 con 0x563f5eb3ba10 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563f5ea22d90 con 0x563f5eb35880 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563f5eb42540 con 0x563f5eb3b0f0 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4eb047b80 10 client.0 _ll_register_callbacks cb 0x563f5ea1cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4ded61700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3ba10 0x563f5eb3ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 0x563f5eb41730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 0x563f5eb41730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46976/0 (socket says 172.21.15.52:46976) 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 -- 172.21.15.52:0/3259420410 learned_addr learned my addr 172.21.15.52:0/3259420410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 -- 172.21.15.52:0/3259420410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 msgr2=0x563f5eb3ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 -- 172.21.15.52:0/3259420410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3ba10 msgr2=0x563f5eb3ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3ba10 0x563f5eb3ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4de560700 1 -- 172.21.15.52:0/3259420410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5e9d1a70 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.944+0000 7fc4ded61700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3ba10 0x563f5eb3ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4de560700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 0x563f5eb41730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009350 tx=0x7fc4d4003a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=624ee2d6c8e30296 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3259420410 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4005510 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d40056b0 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3259420410 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc4d402c050 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3259420410 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4d4042410 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4eb047b80 10 client.26744 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f5eb4cca0 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.945+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3259420410 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d40056b0 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3259420410 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc4d402fcd0 con 0x563f5eb35880 2024-06-02T23:17:42.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4eb047b80 10 client.26744 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4eb047b80 10 client.26744 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4eb047b80 20 client.26744 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5eb4f170 0x563f5eb51560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4eb047b80 4 client.26744 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f5ea8feb0 con 0x563f5eb4f170 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5eb4f170 0x563f5eb51560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4c1ffb700 10 client.26744 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.946+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5eb4f170 0x563f5eb51560 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9656b87bd57b2d99 server_cookie=b049f9fcd0517c27 in_seq=0 out_seq=0 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.947+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3259420410 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563f5ea8feb0 con 0x563f5eb4f170 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.947+0000 7fc4c1ffb700 10 client.26744 handle_command_reply: tid=0 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.947+0000 7fc4eb047b80 1 client.26744 shutdown 2024-06-02T23:17:42.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 msgr2=0x563f5eb41730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 0x563f5eb41730 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009350 tx=0x7fc4d4003a70 comp rx=0 tx=0).stop 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 shutdown_connections 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5eb4f170 0x563f5eb51560 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb3ba10 0x563f5eb3ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb35880 0x563f5eb41730 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4c1ffb700 0 client.26744 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3259420410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb3ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 >> 172.21.15.52:0/3259420410 conn(0x563f5ea5be60 msgr2=0x563f5eb2ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 shutdown_connections 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3259420410 wait complete. 2024-06-02T23:17:42.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.948+0000 7fc4eb047b80 20 client.26744 trim_cache size 0 max 16384 2024-06-02T23:17:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 Processor -- start 2024-06-02T23:17:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 -- start start 2024-06-02T23:17:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb30630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb54ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81aa6f0 con 0x563f5eb35880 2024-06-02T23:17:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ac230 con 0x563f5eb3ba10 2024-06-02T23:17:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81aadc0 con 0x563f5eb3b0f0 2024-06-02T23:17:42.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4de560700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb30630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4ded61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb54ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4ded61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb54ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46982/0 (socket says 172.21.15.52:46982) 2024-06-02T23:17:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4ded61700 1 -- 172.21.15.52:0/3866117691 learned_addr learned my addr 172.21.15.52:0/3866117691 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4de560700 1 -- 172.21.15.52:0/3866117691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 msgr2=0x563f5eb54590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4de560700 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4de560700 1 -- 172.21.15.52:0/3866117691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 msgr2=0x563f5eb54ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4de560700 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb54ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4de560700 1 -- 172.21.15.52:0/3866117691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f5eb41370 con 0x563f5eb35880 2024-06-02T23:17:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.973+0000 7fc4ded61700 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.974+0000 7fc4de560700 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb30630 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4003760 tx=0x7fc4d4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6a88df84d75586 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:42.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.974+0000 7fc4c3fff700 1 -- 172.21.15.52:0/3866117691 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40377f0 con 0x563f5eb35880 2024-06-02T23:17:42.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.974+0000 7fc4c3fff700 1 -- 172.21.15.52:0/3866117691 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d4024ad0 con 0x563f5eb35880 2024-06-02T23:17:42.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.974+0000 7fc4c3fff700 1 -- 172.21.15.52:0/3866117691 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40056b0 con 0x563f5eb35880 2024-06-02T23:17:42.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3866117691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 msgr2=0x563f5eb30630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:42.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb30630 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4003760 tx=0x7fc4d4003b90 comp rx=0 tx=0).stop 2024-06-02T23:17:42.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3866117691 shutdown_connections 2024-06-02T23:17:42.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3ba10 0x563f5eb54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb30630 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3866117691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3866117691 >> 172.21.15.52:0/3866117691 conn(0x563f5ea5be60 msgr2=0x563f5eb2ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3866117691 shutdown_connections 2024-06-02T23:17:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3866117691 wait complete. 2024-06-02T23:17:42.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 -- start start 2024-06-02T23:17:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.975+0000 7fc4eb047b80 1 Processor -- start 2024-06-02T23:17:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb54590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3ba10 0x563f5eb52ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81abb60 con 0x563f5eb35880 2024-06-02T23:17:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81ab490 con 0x563f5eb3b0f0 2024-06-02T23:17:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4d81a8690 con 0x563f5eb3ba10 2024-06-02T23:17:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4eb047b80 10 client.0 _ll_register_callbacks cb 0x563f5ea1cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb54590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4ddd5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb54590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45316/0 (socket says 172.21.15.52:45316) 2024-06-02T23:17:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 -- 172.21.15.52:0/3764272384 learned_addr learned my addr 172.21.15.52:0/3764272384 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4ded61700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3ba10 0x563f5eb52ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 -- 172.21.15.52:0/3764272384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3ba10 msgr2=0x563f5eb52ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3ba10 0x563f5eb52ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 -- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3b0f0 msgr2=0x563f5eb54ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 -- 172.21.15.52:0/3764272384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f5eb531d0 con 0x563f5eb35880 2024-06-02T23:17:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:17:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4de560700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb54590 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400a010 tx=0x7fc4d4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78845c22919291d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.976+0000 7fc4c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3764272384 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4050060 con 0x563f5eb35880 2024-06-02T23:17:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4d4024ad0 con 0x563f5eb35880 2024-06-02T23:17:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3764272384 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc4d403c040 con 0x563f5eb35880 2024-06-02T23:17:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3764272384 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc4d4037a90 con 0x563f5eb35880 2024-06-02T23:17:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 10 client.17244 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563f5ebba510 con 0x563f5eb35880 2024-06-02T23:17:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3764272384 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc4d4024ad0 con 0x563f5eb35880 2024-06-02T23:17:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3764272384 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc4d404b470 con 0x563f5eb35880 2024-06-02T23:17:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 10 client.17244 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 10 client.17244 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:17:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 20 client.17244 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5ebba990 0x563f5ebbcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 4 client.17244 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.05, "await_for": 45.0}] 2024-06-02T23:17:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.977+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 144.05, "await_for": 45.0}) -- 0x563f5ea8a430 con 0x563f5ebba990 2024-06-02T23:17:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.978+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5ebba990 0x563f5ebbcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.978+0000 7fc4ddd5f700 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5ebba990 0x563f5ebbcd80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d25fc729d2b9adc server_cookie=be990a26e871b4f8 in_seq=0 out_seq=0 2024-06-02T23:17:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:42.978+0000 7fc4c1ffb700 10 client.17244 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4c1ffb700 1 -- 172.21.15.52:0/3764272384 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563f5ea8a430 con 0x563f5ebba990 2024-06-02T23:17:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4c1ffb700 10 client.17244 handle_command_reply: tid=0 2024-06-02T23:17:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 client.17244 shutdown 2024-06-02T23:17:43.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 msgr2=0x563f5eb54590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb54590 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400a010 tx=0x7fc4d4005e80 comp rx=0 tx=0).stop 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 shutdown_connections 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563f5ebba990 0x563f5ebbcd80 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563f5eb3ba10 0x563f5eb52ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563f5eb35880 0x563f5eb54590 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4c1ffb700 0 client.17244 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 --2- 172.21.15.52:0/3764272384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563f5eb3b0f0 0x563f5eb54ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 >> 172.21.15.52:0/3764272384 conn(0x563f5ea5be60 msgr2=0x563f5eb307b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 shutdown_connections 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 1 -- 172.21.15.52:0/3764272384 wait complete. 2024-06-02T23:17:43.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.040+0000 7fc4eb047b80 20 client.17244 trim_cache size 0 max 16384 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 22, 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout: "1cfe3ad1": { 2024-06-02T23:17:43.043 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 22, 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 144.1, 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:17:43.044 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:17:43.045 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4c40a2cc0 msgr2=0x7fc4c40a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4c40a2cc0 0x7fc4c40a5180 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4000c00 tx=0x7fc4d4005fb0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 msgr2=0x7fc4d8198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 0x7fc4d8198900 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000ef30 tx=0x7fc4d000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 shutdown_connections 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc4c40a2cc0 0x7fc4c40a5180 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc4d8107910 0x7fc4d8198900 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc4d806aec0 0x7fc4d819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 --2- 172.21.15.52:0/2097502435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4d806b870 0x7fc4d819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:43.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 >> 172.21.15.52:0/2097502435 conn(0x7fc4d80783e0 msgr2=0x7fc4d80ff350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:43.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 shutdown_connections 2024-06-02T23:17:43.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:17:43.041+0000 7fc4e07c4700 1 -- 172.21.15.52:0/2097502435 wait complete. 2024-06-02T23:17:43.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.88 MB/sec execute 172 sec latency 619.974 ms 2024-06-02T23:17:43.829 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:43 smithi052 ceph-mon[32340]: pgmap v606: 97 pgs: 97 active+clean; 59 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 388 KiB/s wr, 36 op/s 2024-06-02T23:17:43.859 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '1cfe3ad1', quiesce took 6.0 seconds. Will release after: 18.05 2024-06-02T23:17:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:43 smithi174 ceph-mon[35994]: pgmap v606: 97 pgs: 97 active+clean; 59 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 388 KiB/s wr, 36 op/s 2024-06-02T23:17:44.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:43 smithi123 ceph-mon[35977]: pgmap v606: 97 pgs: 97 active+clean; 59 MiB data, 924 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 388 KiB/s wr, 36 op/s 2024-06-02T23:17:44.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.87 MB/sec execute 173 sec latency 1620.036 ms 2024-06-02T23:17:44.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:44 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:17:44.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:44 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:17:45.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:44 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:17:45.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.86 MB/sec execute 174 sec latency 2620.121 ms 2024-06-02T23:17:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:45 smithi174 ceph-mon[35994]: pgmap v607: 97 pgs: 97 active+clean; 60 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 452 KiB/s wr, 54 op/s 2024-06-02T23:17:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:17:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:17:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:17:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:45 smithi052 ceph-mon[32340]: pgmap v607: 97 pgs: 97 active+clean; 60 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 452 KiB/s wr, 54 op/s 2024-06-02T23:17:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:17:45.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:17:45.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:17:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:45 smithi123 ceph-mon[35977]: pgmap v607: 97 pgs: 97 active+clean; 60 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 452 KiB/s wr, 54 op/s 2024-06-02T23:17:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:17:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:17:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:17:46.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.85 MB/sec execute 175 sec latency 3620.185 ms 2024-06-02T23:17:47.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.84 MB/sec execute 176 sec latency 4620.286 ms 2024-06-02T23:17:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:47 smithi174 ceph-mon[35994]: pgmap v608: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 623 KiB/s wr, 78 op/s 2024-06-02T23:17:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:47 smithi052 ceph-mon[32340]: pgmap v608: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 623 KiB/s wr, 78 op/s 2024-06-02T23:17:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:47 smithi123 ceph-mon[35977]: pgmap v608: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 623 KiB/s wr, 78 op/s 2024-06-02T23:17:48.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.83 MB/sec execute 177 sec latency 5620.359 ms 2024-06-02T23:17:49.630 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.82 MB/sec execute 178 sec latency 6620.419 ms 2024-06-02T23:17:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:49 smithi174 ceph-mon[35994]: pgmap v609: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 670 KiB/s wr, 89 op/s 2024-06-02T23:17:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:49 smithi052 ceph-mon[32340]: pgmap v609: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 670 KiB/s wr, 89 op/s 2024-06-02T23:17:50.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:49 smithi123 ceph-mon[35977]: pgmap v609: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 670 KiB/s wr, 89 op/s 2024-06-02T23:17:50.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.81 MB/sec execute 179 sec latency 7620.483 ms 2024-06-02T23:17:51.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.80 MB/sec execute 180 sec latency 8620.570 ms 2024-06-02T23:17:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:51 smithi174 ceph-mon[35994]: pgmap v610: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 668 KiB/s wr, 88 op/s 2024-06-02T23:17:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:51 smithi052 ceph-mon[32340]: pgmap v610: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 668 KiB/s wr, 88 op/s 2024-06-02T23:17:52.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:51 smithi123 ceph-mon[35977]: pgmap v610: 97 pgs: 97 active+clean; 57 MiB data, 952 MiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 668 KiB/s wr, 88 op/s 2024-06-02T23:17:52.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.79 MB/sec execute 181 sec latency 9620.603 ms 2024-06-02T23:17:53.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.78 MB/sec execute 182 sec latency 10620.648 ms 2024-06-02T23:17:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:53 smithi174 ceph-mon[35994]: pgmap v611: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 546 KiB/s wr, 85 op/s 2024-06-02T23:17:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:53 smithi052 ceph-mon[32340]: pgmap v611: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 546 KiB/s wr, 85 op/s 2024-06-02T23:17:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:53 smithi123 ceph-mon[35977]: pgmap v611: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 546 KiB/s wr, 85 op/s 2024-06-02T23:17:54.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.77 MB/sec execute 183 sec latency 11620.757 ms 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 -- 172.21.15.52:0/2799772626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13341024b0 msgr2=0x7f1334108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 --2- 172.21.15.52:0/2799772626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13341024b0 0x7f1334108d40 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ca0 tx=0x7f132c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 -- 172.21.15.52:0/2799772626 shutdown_connections 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 --2- 172.21.15.52:0/2799772626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f13341024b0 0x7f1334108d40 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 --2- 172.21.15.52:0/2799772626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334101af0 0x7f1334101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 --2- 172.21.15.52:0/2799772626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1334100900 0x7f1334100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 -- 172.21.15.52:0/2799772626 >> 172.21.15.52:0/2799772626 conn(0x7f13340fbe70 msgr2=0x7f13340fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 -- 172.21.15.52:0/2799772626 shutdown_connections 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.061+0000 7f1339eee640 1 -- 172.21.15.52:0/2799772626 wait complete. 2024-06-02T23:17:55.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 Processor -- start 2024-06-02T23:17:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 -- start start 2024-06-02T23:17:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 0x7f133410b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334101af0 0x7f13341098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13341024b0 0x7f1334109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f133410d550 con 0x7f1334100900 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f133410d3d0 con 0x7f1334101af0 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1339eee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f133410d6d0 con 0x7f13341024b0 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f13337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 0x7f133410b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f13337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 0x7f133410b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45058/0 (socket says 172.21.15.52:45058) 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f13337fe640 1 -- 172.21.15.52:0/1734059611 learned_addr learned my addr 172.21.15.52:0/1734059611 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.062+0000 7f1332ffd640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334101af0 0x7f13341098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f1333fff640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13341024b0 0x7f1334109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f13337fe640 1 -- 172.21.15.52:0/1734059611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13341024b0 msgr2=0x7f1334109e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f13337fe640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13341024b0 0x7f1334109e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f13337fe640 1 -- 172.21.15.52:0/1734059611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334101af0 msgr2=0x7f13341098c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f13337fe640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334101af0 0x7f13341098c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f13337fe640 1 -- 172.21.15.52:0/1734059611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133410a690 con 0x7f1334100900 2024-06-02T23:17:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f13337fe640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 0x7f133410b280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f132400c970 tx=0x7f132400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f1330ff9640 1 -- 172.21.15.52:0/1734059611 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324007c10 con 0x7f1334100900 2024-06-02T23:17:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13341a1be0 con 0x7f1334100900 2024-06-02T23:17:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f1330ff9640 1 -- 172.21.15.52:0/1734059611 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1324007db0 con 0x7f1334100900 2024-06-02T23:17:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f1330ff9640 1 -- 172.21.15.52:0/1734059611 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132400f7d0 con 0x7f1334100900 2024-06-02T23:17:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.063+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13341a2180 con 0x7f1334100900 2024-06-02T23:17:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f1330ff9640 1 -- 172.21.15.52:0/1734059611 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f132400fa30 con 0x7f1334100900 2024-06-02T23:17:55.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 Processor -- start 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 -- start start 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb532a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 0x55efb532f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb5332000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55efb528d4f0 con 0x55efb532aa00 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55efb522b870 con 0x55efb532a010 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f133cbf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55efb52894b0 con 0x55efb532fb70 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f13337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 0x55efb532f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f13337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 0x55efb532f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45062/0 (socket says 172.21.15.52:45062) 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f13337fe640 1 -- 172.21.15.52:0/597204225 learned_addr learned my addr 172.21.15.52:0/597204225 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f1333fff640 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb532a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.066+0000 7f1332ffd640 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb5332000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1330ff9640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f13200a2c30 0x7f13200a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f13337fe640 1 -- 172.21.15.52:0/597204225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 msgr2=0x55efb5332000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f13337fe640 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb5332000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f13337fe640 1 -- 172.21.15.52:0/597204225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 msgr2=0x55efb532a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f13337fe640 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb532a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f13337fe640 1 -- 172.21.15.52:0/597204225 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb52479b0 con 0x55efb532aa00 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1333fff640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f13200a2c30 0x7f13200a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1330ff9640 1 -- 172.21.15.52:0/1734059611 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f13240cf390 con 0x7f1334100900 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f13337fe640 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 0x55efb532f500 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1324089e80 tx=0x7f1324066880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c021c83ad3635ba server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1329ffb640 1 -- 172.21.15.52:0/597204225 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132409b610 con 0x55efb532aa00 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1333fff640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f13200a2c30 0x7f13200a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55efb532b260 tx=0x7f132c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1329ffb640 1 -- 172.21.15.52:0/597204225 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1324089270 con 0x55efb532aa00 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.067+0000 7f1329ffb640 1 -- 172.21.15.52:0/597204225 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1324089070 con 0x55efb532aa00 2024-06-02T23:17:55.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 -- 172.21.15.52:0/597204225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 msgr2=0x55efb532f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 0x55efb532f500 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f1324089e80 tx=0x7f1324066880 comp rx=0 tx=0).stop 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 -- 172.21.15.52:0/597204225 shutdown_connections 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb5332000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532aa00 0x55efb532f500 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 --2- 172.21.15.52:0/597204225 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb532a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 -- 172.21.15.52:0/597204225 >> 172.21.15.52:0/597204225 conn(0x55efb5325680 msgr2=0x55efb5325a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 -- 172.21.15.52:0/597204225 shutdown_connections 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 -- 172.21.15.52:0/597204225 wait complete. 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.068+0000 7f133cbf8740 1 -- start start 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 Processor -- start 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 0x55efb5334190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532aa00 0x55efb53346d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb533b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55efb5332ac0 con 0x55efb532a010 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55efb5216ae0 con 0x55efb532aa00 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55efb5332c40 con 0x55efb532fb70 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f133cbf8740 10 client.0 _ll_register_callbacks cb 0x55efb524f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 0x55efb5334190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 0x55efb5334190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45078/0 (socket says 172.21.15.52:45078) 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 -- 172.21.15.52:0/1150410378 learned_addr learned my addr 172.21.15.52:0/1150410378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f1333fff640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb533b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f1332ffd640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532aa00 0x55efb53346d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 -- 172.21.15.52:0/1150410378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 msgr2=0x55efb533b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb533b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 -- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532aa00 msgr2=0x55efb53346d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532aa00 0x55efb53346d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f13337fe640 1 -- 172.21.15.52:0/1150410378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb4fa1210 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.069+0000 7f1333fff640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb533b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f13337fe640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 0x55efb5334190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f13240a4360 tx=0x7f1324018cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5137250f31e8bd3d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f1312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f1312ffd640 1 -- 172.21.15.52:0/1150410378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132409b610 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f1312ffd640 1 -- 172.21.15.52:0/1150410378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1324089070 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55efb52a0f10 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f1312ffd640 1 -- 172.21.15.52:0/1150410378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f132409c3b0 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 10 client.17256 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efb506d620 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f1312ffd640 1 -- 172.21.15.52:0/1150410378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55efb52a0f10 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f1312ffd640 1 -- 172.21.15.52:0/1150410378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f13240a4400 con 0x55efb532a010 2024-06-02T23:17:55.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 10 client.17256 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 10 client.17256 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 20 client.17256 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb533fc00 0x55efb5341ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 4 client.17256 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.070+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55efb529ac20 con 0x55efb533fc00 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.071+0000 7f1332ffd640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb533fc00 0x55efb5341ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.071+0000 7f1332ffd640 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb533fc00 0x55efb5341ff0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9d1ec0f82b071c server_cookie=974fca3d11901362 in_seq=0 out_seq=0 2024-06-02T23:17:55.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.071+0000 7f1312ffd640 10 client.17256 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f1312ffd640 1 -- 172.21.15.52:0/1150410378 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55efb529ac20 con 0x55efb533fc00 2024-06-02T23:17:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f1312ffd640 10 client.17256 handle_command_reply: tid=0 2024-06-02T23:17:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 client.17256 shutdown 2024-06-02T23:17:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 msgr2=0x55efb5334190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 0x55efb5334190 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f13240a4360 tx=0x7f1324018cd0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 shutdown_connections 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb533fc00 0x55efb5341ff0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532fb70 0x55efb533b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532aa00 0x55efb53346d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1150410378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532a010 0x55efb5334190 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f1312ffd640 0 client.17256 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 >> 172.21.15.52:0/1150410378 conn(0x55efb5325680 msgr2=0x55efb53247e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 shutdown_connections 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 1 -- 172.21.15.52:0/1150410378 wait complete. 2024-06-02T23:17:55.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.072+0000 7f133cbf8740 20 client.17256 trim_cache size 0 max 16384 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 Processor -- start 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 -- start start 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5328590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb533b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5328ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13341aae80 con 0x55efb532fb70 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ac9c0 con 0x55efb532a010 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f133cbf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ab550 con 0x55efb532aa00 2024-06-02T23:17:55.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5328ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5328ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45082/0 (socket says 172.21.15.52:45082) 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 -- 172.21.15.52:0/1395170663 learned_addr learned my addr 172.21.15.52:0/1395170663 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1332ffd640 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb533b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f13337fe640 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5328590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 -- 172.21.15.52:0/1395170663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 msgr2=0x55efb533b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb533b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 -- 172.21.15.52:0/1395170663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 msgr2=0x55efb5328590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5328590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 -- 172.21.15.52:0/1395170663 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55efb53425d0 con 0x55efb532fb70 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f13337fe640 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5328590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.088+0000 7f1333fff640 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5328ad0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ca0 tx=0x7f132c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58b6926911e8429 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f13297fa640 1 -- 172.21.15.52:0/1395170663 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c052070 con 0x55efb532fb70 2024-06-02T23:17:55.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f13297fa640 1 -- 172.21.15.52:0/1395170663 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f132c036040 con 0x55efb532fb70 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f13297fa640 1 -- 172.21.15.52:0/1395170663 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c0305c0 con 0x55efb532fb70 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 -- 172.21.15.52:0/1395170663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 msgr2=0x55efb5328ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5328ad0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f132c009ca0 tx=0x7f132c041000 comp rx=0 tx=0).stop 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 -- 172.21.15.52:0/1395170663 shutdown_connections 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb533b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5328ad0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 --2- 172.21.15.52:0/1395170663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5328590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 -- 172.21.15.52:0/1395170663 >> 172.21.15.52:0/1395170663 conn(0x55efb5325680 msgr2=0x55efb532bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 -- 172.21.15.52:0/1395170663 shutdown_connections 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.089+0000 7f133cbf8740 1 -- 172.21.15.52:0/1395170663 wait complete. 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 -- start start 2024-06-02T23:17:55.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 Processor -- start 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5338740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb5338c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5339c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f13341ac2f0 con 0x55efb532fb70 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f13341abc20 con 0x55efb532a010 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13341a8e60 con 0x55efb532aa00 2024-06-02T23:17:55.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f133cbf8740 10 client.0 _ll_register_callbacks cb 0x7f132c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f1333fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5339c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f13337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5338740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f13337fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5338740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35532/0 (socket says 172.21.15.52:35532) 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f13337fe640 1 -- 172.21.15.52:0/3768186586 learned_addr learned my addr 172.21.15.52:0/3768186586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.090+0000 7f1332ffd640 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb5338c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1333fff640 1 -- 172.21.15.52:0/3768186586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 msgr2=0x55efb5338c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1333fff640 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb5338c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1333fff640 1 -- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 msgr2=0x55efb5338740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1333fff640 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5338740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1333fff640 1 -- 172.21.15.52:0/3768186586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55efb53254b0 con 0x55efb532fb70 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1333fff640 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5339c40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f132c006010 tx=0x7f132c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e62764d8b22884 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1312ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1312ffd640 1 -- 172.21.15.52:0/3768186586 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c052070 con 0x55efb532fb70 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1312ffd640 1 -- 172.21.15.52:0/3768186586 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f132c003d80 con 0x55efb532fb70 2024-06-02T23:17:55.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132c036040 con 0x55efb532fb70 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1312ffd640 1 -- 172.21.15.52:0/3768186586 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f132c0305c0 con 0x55efb532fb70 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f133cbf8740 10 client.17262 fetch_fsmap learned FSMap version 32 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55efb533b2e0 con 0x55efb532fb70 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.091+0000 7f1312ffd640 1 -- 172.21.15.52:0/3768186586 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f132c036040 con 0x55efb532fb70 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f1312ffd640 1 -- 172.21.15.52:0/3768186586 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f132c05e030 con 0x55efb532fb70 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f133cbf8740 10 client.17262 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f133cbf8740 10 client.17262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f133cbf8740 20 client.17262 populate_metadata read hostname 'smithi052' 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f133cbf8740 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb53aa8f0 0x55efb53acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f133cbf8740 4 client.17262 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55efb5294f70 con 0x55efb53aa8f0 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f13337fe640 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb53aa8f0 0x55efb53acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f1312ffd640 10 client.17262 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:55.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.092+0000 7f13337fe640 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb53aa8f0 0x55efb53acd10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d3d8c2ad2c3ea3 server_cookie=c90e79268631175a in_seq=0 out_seq=0 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f1312ffd640 1 -- 172.21.15.52:0/3768186586 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55efb5294f70 con 0x55efb53aa8f0 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f1312ffd640 10 client.17262 handle_command_reply: tid=0 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 client.17262 shutdown 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 msgr2=0x55efb5339c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5339c40 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f132c006010 tx=0x7f132c004270 comp rx=0 tx=0).stop 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 shutdown_connections 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55efb53aa8f0 0x55efb53acd10 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55efb532aa00 0x55efb5338c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55efb532fb70 0x55efb5339c40 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f1312ffd640 0 client.17262 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 --2- 172.21.15.52:0/3768186586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55efb532a010 0x55efb5338740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 >> 172.21.15.52:0/3768186586 conn(0x55efb5325680 msgr2=0x55efb5328590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 shutdown_connections 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.112+0000 7f133cbf8740 1 -- 172.21.15.52:0/3768186586 wait complete. 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f133cbf8740 20 client.17262 trim_cache size 0 max 16384 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:17:55.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f13200a2c30 msgr2=0x7f13200a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f13200a2c30 0x7f13200a50f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x55efb532b260 tx=0x7f132c0095c0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 msgr2=0x7f133410b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 0x7f133410b280 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f132400c970 tx=0x7f132400ce30 comp rx=0 tx=0).stop 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 shutdown_connections 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f13200a2c30 0x7f13200a50f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f13341024b0 0x7f1334109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1334101af0 0x7f13341098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 --2- 172.21.15.52:0/1734059611 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1334100900 0x7f133410b280 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:17:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 >> 172.21.15.52:0/1734059611 conn(0x7f13340fbe70 msgr2=0x7f1334104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:17:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 shutdown_connections 2024-06-02T23:17:55.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:17:55.113+0000 7f1339eee640 1 -- 172.21.15.52:0/1734059611 wait complete. 2024-06-02T23:17:55.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.76 MB/sec execute 184 sec latency 12620.852 ms 2024-06-02T23:17:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:55 smithi174 ceph-mon[35994]: pgmap v612: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 289 KiB/s wr, 69 op/s 2024-06-02T23:17:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:55 smithi052 ceph-mon[32340]: pgmap v612: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 289 KiB/s wr, 69 op/s 2024-06-02T23:17:56.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:55 smithi123 ceph-mon[35977]: pgmap v612: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 289 KiB/s wr, 69 op/s 2024-06-02T23:17:56.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.75 MB/sec execute 185 sec latency 13620.961 ms 2024-06-02T23:17:57.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.74 MB/sec execute 186 sec latency 14621.045 ms 2024-06-02T23:17:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:57 smithi174 ceph-mon[35994]: pgmap v613: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 224 KiB/s wr, 48 op/s 2024-06-02T23:17:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:57 smithi052 ceph-mon[32340]: pgmap v613: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 224 KiB/s wr, 48 op/s 2024-06-02T23:17:58.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:57 smithi123 ceph-mon[35977]: pgmap v613: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 224 KiB/s wr, 48 op/s 2024-06-02T23:17:58.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.73 MB/sec execute 187 sec latency 15621.164 ms 2024-06-02T23:17:59.631 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.72 MB/sec execute 188 sec latency 16621.294 ms 2024-06-02T23:17:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:17:59 smithi174 ceph-mon[35994]: pgmap v614: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-06-02T23:17:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:17:59 smithi052 ceph-mon[32340]: pgmap v614: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-06-02T23:18:00.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:17:59 smithi123 ceph-mon[35977]: pgmap v614: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 25 op/s 2024-06-02T23:18:00.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.71 MB/sec execute 189 sec latency 17621.420 ms 2024-06-02T23:18:01.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.70 MB/sec execute 190 sec latency 18621.535 ms 2024-06-02T23:18:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:01 smithi174 ceph-mon[35994]: pgmap v615: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-06-02T23:18:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:01 smithi052 ceph-mon[32340]: pgmap v615: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-06-02T23:18:01.911 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:18:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:01 smithi123 ceph-mon[35977]: pgmap v615: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.0 KiB/s wr, 15 op/s 2024-06-02T23:18:02.176 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:02.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.69 MB/sec execute 191 sec latency 19621.614 ms 2024-06-02T23:18:03.035 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:18:03.406 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:03.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.68 MB/sec execute 192 sec latency 20621.674 ms 2024-06-02T23:18:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:03 smithi174 ceph-mon[35994]: pgmap v616: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-06-02T23:18:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:03 smithi052 ceph-mon[32340]: pgmap v616: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-06-02T23:18:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:03 smithi123 ceph-mon[35977]: pgmap v616: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-06-02T23:18:04.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.68 MB/sec execute 193 sec latency 21621.704 ms 2024-06-02T23:18:05.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.67 MB/sec execute 194 sec latency 22621.736 ms 2024-06-02T23:18:05.709 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:05 smithi052 ceph-mon[32340]: pgmap v617: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-06-02T23:18:05.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 -- 172.21.15.52:0/1198641473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006b620 msgr2=0x7feb4010f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:05.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1198641473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006b620 0x7feb4010f350 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7feb3800b0d0 tx=0x7feb3800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 -- 172.21.15.52:0/1198641473 shutdown_connections 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1198641473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006b620 0x7feb4010f350 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1198641473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb40108580 0x7feb40108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1198641473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4006ac60 0x7feb4006b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 -- 172.21.15.52:0/1198641473 >> 172.21.15.52:0/1198641473 conn(0x7feb40078430 msgr2=0x7feb40078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 -- 172.21.15.52:0/1198641473 shutdown_connections 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 -- 172.21.15.52:0/1198641473 wait complete. 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 Processor -- start 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.829+0000 7feb47e2f700 1 -- start start 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb47e2f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 0x7feb40101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb47e2f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4006b620 0x7feb40101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb47e2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb40108580 0x7feb40101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:05.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb47e2f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb40112680 con 0x7feb4006ac60 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb47e2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb40112500 con 0x7feb40108580 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb47e2f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb40112800 con 0x7feb4006b620 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 0x7feb40101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 0x7feb40101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54654/0 (socket says 172.21.15.52:54654) 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 -- 172.21.15.52:0/1341033987 learned_addr learned my addr 172.21.15.52:0/1341033987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb463cc700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb40108580 0x7feb40101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb453ca700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4006b620 0x7feb40101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:05.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 -- 172.21.15.52:0/1341033987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4006b620 msgr2=0x7feb40101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:05.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4006b620 0x7feb40101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:05.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 -- 172.21.15.52:0/1341033987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb40108580 msgr2=0x7feb40101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:05.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb40108580 0x7feb40101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:05.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 -- 172.21.15.52:0/1341033987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb401a7000 con 0x7feb4006ac60 2024-06-02T23:18:05.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.830+0000 7feb45bcb700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 0x7feb40101400 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d950 tx=0x7feb3c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:05.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.831+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c016070 con 0x7feb4006ac60 2024-06-02T23:18:05.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.831+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb401a72f0 con 0x7feb4006ac60 2024-06-02T23:18:05.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.831+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb3c00fd40 con 0x7feb4006ac60 2024-06-02T23:18:05.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.831+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00e3b0 con 0x7feb4006ac60 2024-06-02T23:18:05.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.831+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb401a7880 con 0x7feb4006ac60 2024-06-02T23:18:05.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.832+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb40069180 con 0x7feb4006ac60 2024-06-02T23:18:05.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.833+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb3c019070 con 0x7feb4006ac60 2024-06-02T23:18:05.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.834+0000 7feb36ffd700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb2c0a2ca0 0x7feb2c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:05.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.834+0000 7feb453ca700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb2c0a2ca0 0x7feb2c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:05.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.834+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb3c0ce9f0 con 0x7feb4006ac60 2024-06-02T23:18:05.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.834+0000 7feb453ca700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb2c0a2ca0 0x7feb2c0a5160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feb40078a30 tx=0x7feb3000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:05.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:05.838+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7feb3c09b920 con 0x7feb4006ac60 2024-06-02T23:18:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:05 smithi174 ceph-mon[35994]: pgmap v617: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-06-02T23:18:06.010 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:05 smithi123 ceph-mon[35977]: pgmap v617: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.9 KiB/s wr, 15 op/s 2024-06-02T23:18:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.095+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feb4002dca0 con 0x7feb4006ac60 2024-06-02T23:18:06.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.096+0000 7feb36ffd700 1 -- 172.21.15.52:0/1341033987 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7feb3c09b220 con 0x7feb4006ac60 2024-06-02T23:18:06.097 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:18:06.097 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb2c0a2ca0 msgr2=0x7feb2c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb2c0a2ca0 0x7feb2c0a5160 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7feb40078a30 tx=0x7feb3000b3c0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 msgr2=0x7feb40101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 0x7feb40101400 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7feb3c00d950 tx=0x7feb3c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 shutdown_connections 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb2c0a2ca0 0x7feb2c0a5160 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.098+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb40108580 0x7feb40101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.099+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb4006ac60 0x7feb40101400 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.099+0000 7feb47e2f700 1 --2- 172.21.15.52:0/1341033987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb4006b620 0x7feb40101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.099+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 >> 172.21.15.52:0/1341033987 conn(0x7feb40078430 msgr2=0x7feb400fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.099+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 shutdown_connections 2024-06-02T23:18:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.099+0000 7feb47e2f700 1 -- 172.21.15.52:0/1341033987 wait complete. 2024-06-02T23:18:06.100 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 -- 172.21.15.52:0/3399900310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c06a170 msgr2=0x7f6b3c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 --2- 172.21.15.52:0/3399900310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c06a170 0x7f6b3c06a5f0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009990 tx=0x7f6b2c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 -- 172.21.15.52:0/3399900310 shutdown_connections 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 --2- 172.21.15.52:0/3399900310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3c06ab30 0x7f6b3c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 --2- 172.21.15.52:0/3399900310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 0x7f6b3c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 --2- 172.21.15.52:0/3399900310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c06a170 0x7f6b3c06a5f0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 -- 172.21.15.52:0/3399900310 >> 172.21.15.52:0/3399900310 conn(0x7f6b3c078340 msgr2=0x7f6b3c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 -- 172.21.15.52:0/3399900310 shutdown_connections 2024-06-02T23:18:06.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.262+0000 7f6b44550700 1 -- 172.21.15.52:0/3399900310 wait complete. 2024-06-02T23:18:06.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 Processor -- start 2024-06-02T23:18:06.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 -- start start 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3c06ab30 0x7f6b3c198d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 0x7f6b3c199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 0x7f6b3c19da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c112610 con 0x7f6b3c19d600 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c112490 con 0x7f6b3c06ab30 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b44550700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c112790 con 0x7f6b3c108500 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 0x7f6b3c19da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 0x7f6b3c19da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54666/0 (socket says 172.21.15.52:54666) 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.263+0000 7f6b42aed700 1 -- 172.21.15.52:0/4026758359 learned_addr learned my addr 172.21.15.52:0/4026758359 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b422ec700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3c06ab30 0x7f6b3c198d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 0x7f6b3c199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b42aed700 1 -- 172.21.15.52:0/4026758359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 msgr2=0x7f6b3c199270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b42aed700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 0x7f6b3c199270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b42aed700 1 -- 172.21.15.52:0/4026758359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3c06ab30 msgr2=0x7f6b3c198d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b42aed700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3c06ab30 0x7f6b3c198d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b42aed700 1 -- 172.21.15.52:0/4026758359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b3c19e120 con 0x7f6b3c19d600 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 0x7f6b3c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b42aed700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 0x7f6b3c19da80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400e940 tx=0x7f6b3400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b337fe700 1 -- 172.21.15.52:0/4026758359 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b3400cbd0 con 0x7f6b3c19d600 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b337fe700 1 -- 172.21.15.52:0/4026758359 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b3400cd70 con 0x7f6b3c19d600 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b3c19e410 con 0x7f6b3c19d600 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b337fe700 1 -- 172.21.15.52:0/4026758359 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34005690 con 0x7f6b3c19d600 2024-06-02T23:18:06.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.264+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b3c1a5ba0 con 0x7f6b3c19d600 2024-06-02T23:18:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.267+0000 7f6b337fe700 1 -- 172.21.15.52:0/4026758359 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b34021440 con 0x7f6b3c19d600 2024-06-02T23:18:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.268+0000 7f6b337fe700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b280a2c20 0x7f6b280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.268+0000 7f6b337fe700 1 -- 172.21.15.52:0/4026758359 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6b340cf350 con 0x7f6b3c19d600 2024-06-02T23:18:06.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.268+0000 7f6b422ec700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b280a2c20 0x7f6b280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.268+0000 7f6b4edd3b80 1 Processor -- start 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b422ec700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b280a2c20 0x7f6b280a50e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e270 tx=0x7f6b38009040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 -- start start 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0eca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 0x55d84c0e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0e6f20 0x55d84c0e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d84c038810 con 0x55d84c0e6f20 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d84bfe0720 con 0x55d84c0ecf60 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d84c0347d0 con 0x55d84c0ec640 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b41aeb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 0x55d84c0e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0e6f20 0x55d84c0e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b41aeb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 0x55d84c0e69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43810/0 (socket says 172.21.15.52:43810) 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b41aeb700 1 -- 172.21.15.52:0/2066861358 learned_addr learned my addr 172.21.15.52:0/2066861358 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:06.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.269+0000 7f6b422ec700 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0eca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b41aeb700 1 -- 172.21.15.52:0/2066861358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 msgr2=0x55d84c0eca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0eca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b41aeb700 1 -- 172.21.15.52:0/2066861358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0e6f20 msgr2=0x55d84c0e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0e6f20 0x55d84c0e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b41aeb700 1 -- 172.21.15.52:0/2066861358 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84bfd6f50 con 0x55d84c0ecf60 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b422ec700 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 0x55d84c0e69e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c06b210 tx=0x7f6b2c005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abe3c9f44032fa09 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b30ff9700 1 -- 172.21.15.52:0/2066861358 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c026070 con 0x55d84c0ecf60 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b30ff9700 1 -- 172.21.15.52:0/2066861358 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b2c009510 con 0x55d84c0ecf60 2024-06-02T23:18:06.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.270+0000 7f6b30ff9700 1 -- 172.21.15.52:0/2066861358 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0038e0 con 0x55d84c0ecf60 2024-06-02T23:18:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/2066861358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 msgr2=0x55d84c0e69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 0x55d84c0e69e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c06b210 tx=0x7f6b2c005c00 comp rx=0 tx=0).stop 2024-06-02T23:18:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/2066861358 shutdown_connections 2024-06-02T23:18:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0e6f20 0x55d84c0e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0eca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/2066861358 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ecf60 0x55d84c0e69e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/2066861358 >> 172.21.15.52:0/2066861358 conn(0x55d84c011cc0 msgr2=0x55d84c0e1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/2066861358 shutdown_connections 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/2066861358 wait complete. 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.271+0000 7f6b4edd3b80 1 -- start start 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 Processor -- start 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0f0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0f0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d84c0f7d80 con 0x55d84c0ecf60 2024-06-02T23:18:06.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d84bfd8bf0 con 0x55d84c0e6f20 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d84c0f7f00 con 0x55d84c0ec640 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b4edd3b80 10 client.0 _ll_register_callbacks cb 0x55d84bfd2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0f0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54692/0 (socket says 172.21.15.52:54692) 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 -- 172.21.15.52:0/3791552905 learned_addr learned my addr 172.21.15.52:0/3791552905 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b422ec700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0f0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 -- 172.21.15.52:0/3791552905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 msgr2=0x55d84c0f0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0f0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 -- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 msgr2=0x55d84c0f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b42aed700 1 -- 172.21.15.52:0/3791552905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84bf878d0 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.272+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b42aed700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0f0570 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6b3409b8e0 tx=0x7f6b3409b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec79c35e2a2f57df server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b25ffb700 1 -- 172.21.15.52:0/3791552905 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34086040 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b25ffb700 1 -- 172.21.15.52:0/3791552905 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6b340a45c0 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b2c009510 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b25ffb700 1 -- 172.21.15.52:0/3791552905 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b34067030 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b4edd3b80 10 client.17280 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d84c1026f0 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b25ffb700 1 -- 172.21.15.52:0/3791552905 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b2c009510 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b25ffb700 1 -- 172.21.15.52:0/3791552905 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6b34097a50 con 0x55d84c0ecf60 2024-06-02T23:18:06.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b4edd3b80 10 client.17280 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.273+0000 7f6b4edd3b80 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b4edd3b80 20 client.17280 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c107120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b4edd3b80 4 client.17280 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d84c045d10 con 0x55d84c104d30 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b422ec700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c107120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b25ffb700 10 client.17280 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.274+0000 7f6b422ec700 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c107120 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0b5ba8292fc7a44 server_cookie=ecb1cceb67458380 in_seq=0 out_seq=0 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b25ffb700 1 -- 172.21.15.52:0/3791552905 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d84c045d10 con 0x55d84c104d30 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b25ffb700 10 client.17280 handle_command_reply: tid=0 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 client.17280 shutdown 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 msgr2=0x55d84c0f0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0f0570 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6b3409b8e0 tx=0x7f6b3409b5f0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 shutdown_connections 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c107120 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b25ffb700 0 client.17280 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0f0570 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/3791552905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ec640 0x55d84c0f0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 >> 172.21.15.52:0/3791552905 conn(0x55d84c011cc0 msgr2=0x55d84c0e59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 shutdown_connections 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.275+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/3791552905 wait complete. 2024-06-02T23:18:06.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.276+0000 7f6b4edd3b80 20 client.17280 trim_cache size 0 max 16384 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 Processor -- start 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 -- start start 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0e69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 0x55d84c0eeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ecf60 0x55d84c0db690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1aad70 con 0x55d84c0ec640 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ac8b0 con 0x55d84c0e6f20 2024-06-02T23:18:06.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ab440 con 0x55d84c0ecf60 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 0x55d84c0eeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 0x55d84c0eeff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54696/0 (socket says 172.21.15.52:54696) 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 -- 172.21.15.52:0/290389704 learned_addr learned my addr 172.21.15.52:0/290389704 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b422ec700 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0e69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b42aed700 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ecf60 0x55d84c0db690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 -- 172.21.15.52:0/290389704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ecf60 msgr2=0x55d84c0db690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ecf60 0x55d84c0db690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 -- 172.21.15.52:0/290389704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 msgr2=0x55d84c0e69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0e69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.298+0000 7f6b41aeb700 1 -- 172.21.15.52:0/290389704 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d84c0ec120 con 0x55d84c0ec640 2024-06-02T23:18:06.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.299+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 0x55d84c0eeff0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0059d0 tx=0x7f6b2c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566026842b83140 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:06.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.299+0000 7f6b27fff700 1 -- 172.21.15.52:0/290389704 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c02e070 con 0x55d84c0ec640 2024-06-02T23:18:06.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.299+0000 7f6b27fff700 1 -- 172.21.15.52:0/290389704 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b2c009590 con 0x55d84c0ec640 2024-06-02T23:18:06.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.299+0000 7f6b27fff700 1 -- 172.21.15.52:0/290389704 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c026070 con 0x55d84c0ec640 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/290389704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 msgr2=0x55d84c0eeff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 0x55d84c0eeff0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c0059d0 tx=0x7f6b2c005c70 comp rx=0 tx=0).stop 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/290389704 shutdown_connections 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0ecf60 0x55d84c0db690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0e6f20 0x55d84c0e69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/290389704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ec640 0x55d84c0eeff0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/290389704 >> 172.21.15.52:0/290389704 conn(0x55d84c011cc0 msgr2=0x55d84c0eda70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/290389704 shutdown_connections 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/290389704 wait complete. 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 -- start start 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.300+0000 7f6b4edd3b80 1 Processor -- start 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0e6f20 0x55d84c0fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ec640 0x55d84c0fd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0fa190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1ac1e0 con 0x55d84c0ecf60 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1abb10 con 0x55d84c0ec640 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b3c1a8d30 con 0x55d84c0e6f20 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b4edd3b80 10 client.0 _ll_register_callbacks cb 0x55d84bfd2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:06.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0fa190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0fa190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54708/0 (socket says 172.21.15.52:54708) 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 -- 172.21.15.52:0/734073672 learned_addr learned my addr 172.21.15.52:0/734073672 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b422ec700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0e6f20 0x55d84c0fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b41aeb700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ec640 0x55d84c0fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 -- 172.21.15.52:0/734073672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0e6f20 msgr2=0x55d84c0fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0e6f20 0x55d84c0fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 -- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ec640 msgr2=0x55d84c0fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ec640 0x55d84c0fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 -- 172.21.15.52:0/734073672 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d84c0fa6d0 con 0x55d84c0ecf60 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b42aed700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0fa190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009fd0 tx=0x7f6b34097c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99fad591de05817 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.301+0000 7f6b25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b25ffb700 1 -- 172.21.15.52:0/734073672 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b34086040 con 0x55d84c0ecf60 2024-06-02T23:18:06.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b25ffb700 1 -- 172.21.15.52:0/734073672 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6b3409b030 con 0x55d84c0ecf60 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b2c009590 con 0x55d84c0ecf60 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b25ffb700 1 -- 172.21.15.52:0/734073672 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b340aa070 con 0x55d84c0ecf60 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 10 client.17286 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d84c170100 con 0x55d84c0ecf60 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b25ffb700 1 -- 172.21.15.52:0/734073672 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b2c009590 con 0x55d84c0ecf60 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b25ffb700 1 -- 172.21.15.52:0/734073672 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6b340a1120 con 0x55d84c0ecf60 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 10 client.17286 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 20 client.17286 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c172790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 4 client.17286 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.302+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d84c040290 con 0x55d84c104d30 2024-06-02T23:18:06.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b422ec700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c172790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b25ffb700 10 client.17286 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b422ec700 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c172790 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26eeac2de319b8e2 server_cookie=ab88a3cd0153fd66 in_seq=0 out_seq=0 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b25ffb700 1 -- 172.21.15.52:0/734073672 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d84c040290 con 0x55d84c104d30 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b25ffb700 10 client.17286 handle_command_reply: tid=0 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 client.17286 shutdown 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 msgr2=0x55d84c0fa190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0fa190 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009fd0 tx=0x7f6b34097c10 comp rx=0 tx=0).stop 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 shutdown_connections 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d84c104d30 0x55d84c172790 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d84c0ecf60 0x55d84c0fa190 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d84c0e6f20 0x55d84c0fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 --2- 172.21.15.52:0/734073672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d84c0ec640 0x55d84c0fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b25ffb700 0 client.17286 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 >> 172.21.15.52:0/734073672 conn(0x55d84c011cc0 msgr2=0x55d84c0eb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.303+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 shutdown_connections 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b4edd3b80 1 -- 172.21.15.52:0/734073672 wait complete. 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b4edd3b80 20 client.17286 trim_cache size 0 max 16384 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:18:06.310 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:18:06.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b280a2c20 msgr2=0x7f6b280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b280a2c20 0x7f6b280a50e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f6b3800e270 tx=0x7f6b38009040 comp rx=0 tx=0).stop 2024-06-02T23:18:06.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 msgr2=0x7f6b3c19da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:06.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 0x7f6b3c19da80 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6b3400e940 tx=0x7f6b3400ed00 comp rx=0 tx=0).stop 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 shutdown_connections 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b280a2c20 0x7f6b280a50e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b3c19d600 0x7f6b3c19da80 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b3c06ab30 0x7f6b3c198d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 --2- 172.21.15.52:0/4026758359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b3c108500 0x7f6b3c199270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 >> 172.21.15.52:0/4026758359 conn(0x7f6b3c078340 msgr2=0x7f6b3c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 shutdown_connections 2024-06-02T23:18:06.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:06.304+0000 7f6b44550700 1 -- 172.21.15.52:0/4026758359 wait complete. 2024-06-02T23:18:06.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.66 MB/sec execute 195 sec latency 23621.769 ms 2024-06-02T23:18:06.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:06 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1341033987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:18:06.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:06 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1341033987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:18:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:06 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1341033987' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:18:07.100 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 1cfe3ad1 --release --await' 2024-06-02T23:18:07.100 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 1cfe3ad1 --release --await 2024-06-02T23:18:07.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.65 MB/sec execute 196 sec latency 24621.820 ms 2024-06-02T23:18:07.838 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:18:07.838 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:18:07.839 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:07 smithi174 ceph-mon[35994]: pgmap v618: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 14 op/s 2024-06-02T23:18:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:07 smithi052 ceph-mon[32340]: pgmap v618: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 14 op/s 2024-06-02T23:18:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:07 smithi123 ceph-mon[35977]: pgmap v618: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 14 op/s 2024-06-02T23:18:08.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 -- 172.21.15.52:0/4253173152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c1024e0 msgr2=0x7f4a6c1048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 --2- 172.21.15.52:0/4253173152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c1024e0 0x7f4a6c1048d0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009900 tx=0x7f4a5c02a410 comp rx=0 tx=0).stop 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 -- 172.21.15.52:0/4253173152 shutdown_connections 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 --2- 172.21.15.52:0/4253173152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a6c107740 0x7f4a6c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 --2- 172.21.15.52:0/4253173152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a6c104e10 0x7f4a6c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 --2- 172.21.15.52:0/4253173152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c1024e0 0x7f4a6c1048d0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 -- 172.21.15.52:0/4253173152 >> 172.21.15.52:0/4253173152 conn(0x7f4a6c0fbe90 msgr2=0x7f4a6c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 -- 172.21.15.52:0/4253173152 shutdown_connections 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 -- 172.21.15.52:0/4253173152 wait complete. 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.205+0000 7f4a729a5640 1 Processor -- start 2024-06-02T23:18:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 -- start start 2024-06-02T23:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a6c104e10 0x7f4a6c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 0x7f4a6c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a6c19d310 0x7f4a6c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c10d570 con 0x7f4a6c107740 2024-06-02T23:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c10d3f0 con 0x7f4a6c19d310 2024-06-02T23:18:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a729a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c10d6f0 con 0x7f4a6c104e10 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 0x7f4a6c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 0x7f4a6c1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54738/0 (socket says 172.21.15.52:54738) 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/3963833363 learned_addr learned my addr 172.21.15.52:0/3963833363 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.206+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a6c104e10 0x7f4a6c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a70f42640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a6c19d310 0x7f4a6c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/3963833363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a6c104e10 msgr2=0x7f4a6c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a6c104e10 0x7f4a6c197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/3963833363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a6c19d310 msgr2=0x7f4a6c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a6c19d310 0x7f4a6c19f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/3963833363 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a6c19fda0 con 0x7f4a6c107740 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 0x7f4a6c1982f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9a0 tx=0x7f4a5400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a697fa640 1 -- 172.21.15.52:0/3963833363 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5400cd80 con 0x7f4a6c107740 2024-06-02T23:18:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a697fa640 1 -- 172.21.15.52:0/3963833363 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a540035f0 con 0x7f4a6c107740 2024-06-02T23:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a697fa640 1 -- 172.21.15.52:0/3963833363 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54010690 con 0x7f4a6c107740 2024-06-02T23:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a6c1a0090 con 0x7f4a6c107740 2024-06-02T23:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.207+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a6c1a0560 con 0x7f4a6c107740 2024-06-02T23:18:08.210 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.210+0000 7f4a697fa640 1 -- 172.21.15.52:0/3963833363 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4a540037a0 con 0x7f4a6c107740 2024-06-02T23:18:08.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 Processor -- start 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 -- start start 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a697fa640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4a580a2c30 0x7f4a580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4618430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb461d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 0x563fb4620000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563fb457b4f0 con 0x563fb461db70 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563fb4519870 con 0x563fb4618a00 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a7578d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563fb45774b0 con 0x563fb4618010 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a70f42640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4a580a2c30 0x7f4a580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a697fa640 1 -- 172.21.15.52:0/3963833363 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4a540cf410 con 0x7f4a6c107740 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a6bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4618430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a70f42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 0x563fb4620000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a6bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4618430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:50910/0 (socket says 172.21.15.52:50910) 2024-06-02T23:18:08.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a70f42640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 0x563fb4620000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54754/0 (socket says 172.21.15.52:54754) 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a6bfff640 1 -- 172.21.15.52:0/1178501271 learned_addr learned my addr 172.21.15.52:0/1178501271 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.211+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb461d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 -- 172.21.15.52:0/1178501271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 msgr2=0x563fb4618430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4618430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 -- 172.21.15.52:0/1178501271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 msgr2=0x563fb461d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb461d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 -- 172.21.15.52:0/1178501271 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fb45359b0 con 0x563fb461db70 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 0x563fb4620000 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fd0 tx=0x7f4a64011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a590b6cee97da40 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a70f42640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4a580a2c30 0x7f4a580a50f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009fd0 tx=0x7f4a64016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a627fc640 1 -- 172.21.15.52:0/1178501271 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64014e60 con 0x563fb461db70 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a627fc640 1 -- 172.21.15.52:0/1178501271 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6400ebc0 con 0x563fb461db70 2024-06-02T23:18:08.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.212+0000 7f4a627fc640 1 -- 172.21.15.52:0/1178501271 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64013a00 con 0x563fb461db70 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 -- 172.21.15.52:0/1178501271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 msgr2=0x563fb4620000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 0x563fb4620000 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4a64005fd0 tx=0x7f4a64011750 comp rx=0 tx=0).stop 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 -- 172.21.15.52:0/1178501271 shutdown_connections 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb461db70 0x563fb4620000 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb461d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 --2- 172.21.15.52:0/1178501271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4618430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 -- 172.21.15.52:0/1178501271 >> 172.21.15.52:0/1178501271 conn(0x563fb4613680 msgr2=0x563fb4613a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 -- 172.21.15.52:0/1178501271 shutdown_connections 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 -- 172.21.15.52:0/1178501271 wait complete. 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.213+0000 7f4a7578d740 1 -- start start 2024-06-02T23:18:08.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 Processor -- start 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 0x563fb4626670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb4626bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb461db70 0x563fb462dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563fb4624ed0 con 0x563fb4618010 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563fb4504ae0 con 0x563fb4618a00 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563fb4625050 con 0x563fb461db70 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a7578d740 10 client.0 _ll_register_callbacks cb 0x563fb453d9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 0x563fb4626670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 0x563fb4626670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54760/0 (socket says 172.21.15.52:54760) 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 -- 172.21.15.52:0/2297808634 learned_addr learned my addr 172.21.15.52:0/2297808634 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a70f42640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb461db70 0x563fb462dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb4626bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 -- 172.21.15.52:0/2297808634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb461db70 msgr2=0x563fb462dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb461db70 0x563fb462dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 -- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 msgr2=0x563fb4626bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb4626bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.214+0000 7f4a6bfff640 1 -- 172.21.15.52:0/2297808634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fb428f210 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 0x563fb4626670 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009230 tx=0x7f4a5c004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5bc939be81398f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/2297808634 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5c035070 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/2297808634 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4a5c003a00 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6400ebc0 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/2297808634 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a5c003c00 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a7578d740 10 client.17298 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563fb435b620 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/2297808634 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6400ebc0 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/2297808634 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4a5c03c4d0 con 0x563fb4618010 2024-06-02T23:18:08.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a7578d740 10 client.17298 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a7578d740 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.215+0000 7f4a7578d740 20 client.17298 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.216+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb4632540 0x563fb4634950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.216+0000 7f4a7578d740 4 client.17298 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.216+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fb458ef10 con 0x563fb4632540 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.216+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb4632540 0x563fb4634950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.216+0000 7f4a4b7fe640 10 client.17298 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.216+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb4632540 0x563fb4634950 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfa14635eb8ebf5d server_cookie=30e6e801cb78cc0a in_seq=0 out_seq=0 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/2297808634 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563fb458ef10 con 0x563fb4632540 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a4b7fe640 10 client.17298 handle_command_reply: tid=0 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 client.17298 shutdown 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 msgr2=0x563fb4626670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 0x563fb4626670 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c009230 tx=0x7f4a5c004440 comp rx=0 tx=0).stop 2024-06-02T23:18:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 shutdown_connections 2024-06-02T23:18:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb4632540 0x563fb4634950 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb461db70 0x563fb462dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb4618a00 0x563fb4626bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a4b7fe640 0 client.17298 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2297808634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618010 0x563fb4626670 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 >> 172.21.15.52:0/2297808634 conn(0x563fb4613680 msgr2=0x563fb46127e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 shutdown_connections 2024-06-02T23:18:08.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.217+0000 7f4a7578d740 1 -- 172.21.15.52:0/2297808634 wait complete. 2024-06-02T23:18:08.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.218+0000 7f4a7578d740 20 client.17298 trim_cache size 0 max 16384 2024-06-02T23:18:08.242 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 Processor -- start 2024-06-02T23:18:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 -- start start 2024-06-02T23:18:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4626970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb460e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1a6fd0 con 0x563fb4618a00 2024-06-02T23:18:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1a8b10 con 0x563fb461db70 2024-06-02T23:18:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.233+0000 7f4a7578d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1a76a0 con 0x563fb4618010 2024-06-02T23:18:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54768/0 (socket says 172.21.15.52:54768) 2024-06-02T23:18:08.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/2033370509 learned_addr learned my addr 172.21.15.52:0/2033370509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4626970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb460e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 -- 172.21.15.52:0/2033370509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 msgr2=0x563fb4626970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4626970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 -- 172.21.15.52:0/2033370509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 msgr2=0x563fb462dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 -- 172.21.15.52:0/2033370509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fb46134b0 con 0x563fb461db70 2024-06-02T23:18:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a70f42640 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb460e6e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a64011c50 tx=0x7f4a64005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ab4e9e48bfd4fa9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.234+0000 7f4a61ffb640 1 -- 172.21.15.52:0/2033370509 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64034070 con 0x563fb461db70 2024-06-02T23:18:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a61ffb640 1 -- 172.21.15.52:0/2033370509 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6400ebc0 con 0x563fb461db70 2024-06-02T23:18:08.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a61ffb640 1 -- 172.21.15.52:0/2033370509 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a64014e60 con 0x563fb461db70 2024-06-02T23:18:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 -- 172.21.15.52:0/2033370509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 msgr2=0x563fb460e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb460e6e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f4a64011c50 tx=0x7f4a64005bf0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 -- 172.21.15.52:0/2033370509 shutdown_connections 2024-06-02T23:18:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb460e6e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 --2- 172.21.15.52:0/2033370509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb4626970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 -- 172.21.15.52:0/2033370509 >> 172.21.15.52:0/2033370509 conn(0x563fb4613680 msgr2=0x563fb4613a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 -- 172.21.15.52:0/2033370509 shutdown_connections 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.235+0000 7f4a7578d740 1 -- 172.21.15.52:0/2033370509 wait complete. 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 -- start start 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 Processor -- start 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb462da00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb462a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1a8440 con 0x563fb4618a00 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1a7d70 con 0x563fb461db70 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4a6c1a4fb0 con 0x563fb4618010 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a7578d740 10 client.0 _ll_register_callbacks cb 0x7f4a5c005d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a70f42640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb462a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.249 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a6b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54778/0 (socket says 172.21.15.52:54778) 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a70f42640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb462a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43878/0 (socket says 172.21.15.52:43878) 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/656241921 learned_addr learned my addr 172.21.15.52:0/656241921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.236+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb462da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/656241921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 msgr2=0x563fb462da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb462da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 msgr2=0x563fb462a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:08.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb462a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a6b7fe640 1 -- 172.21.15.52:0/656241921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fb46175c0 con 0x563fb4618a00 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a70f42640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb462a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a6b7fe640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462a4a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4a5409bc30 tx=0x7f4a5409b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa547545746f327f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a4b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/656241921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a54003b10 con 0x563fb4618a00 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/656241921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4a54098280 con 0x563fb4618a00 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a6400ebc0 con 0x563fb4618a00 2024-06-02T23:18:08.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/656241921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4a5409c3a0 con 0x563fb4618a00 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a7578d740 10 client.17304 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563fb469c2d0 con 0x563fb4618a00 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/656241921 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4a6400ebc0 con 0x563fb4618a00 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/656241921 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4a54097960 con 0x563fb4618a00 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a7578d740 10 client.17304 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.237+0000 7f4a7578d740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a7578d740 20 client.17304 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:08.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a7578d740 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb469cc30 0x563fb469f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a7578d740 4 client.17304 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:18:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fb4588c20 con 0x563fb469cc30 2024-06-02T23:18:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb469cc30 0x563fb469f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a4b7fe640 10 client.17304 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:08.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.238+0000 7f4a6bfff640 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb469cc30 0x563fb469f050 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8515bae596edcae5 server_cookie=36b508b4f3c9debc in_seq=0 out_seq=0 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.252+0000 7f4a4b7fe640 1 -- 172.21.15.52:0/656241921 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563fb4588c20 con 0x563fb469cc30 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.252+0000 7f4a4b7fe640 10 client.17304 handle_command_reply: tid=0 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.252+0000 7f4a7578d740 1 client.17304 shutdown 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 msgr2=0x563fb462a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462a4a0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f4a5409bc30 tx=0x7f4a5409b8e0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 shutdown_connections 2024-06-02T23:18:08.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563fb469cc30 0x563fb469f050 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563fb461db70 0x563fb462a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563fb4618a00 0x563fb462a4a0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a4b7fe640 0 client.17304 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 --2- 172.21.15.52:0/656241921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563fb4618010 0x563fb462da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 >> 172.21.15.52:0/656241921 conn(0x563fb4613680 msgr2=0x563fb460e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 shutdown_connections 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 1 -- 172.21.15.52:0/656241921 wait complete. 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a7578d740 20 client.17304 trim_cache size 0 max 16384 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:18:08.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:18:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:18:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4a580a2c30 msgr2=0x7f4a580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4a580a2c30 0x7f4a580a50f0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f4a64009fd0 tx=0x7f4a64016000 comp rx=0 tx=0).stop 2024-06-02T23:18:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 msgr2=0x7f4a6c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:08.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 0x7f4a6c1982f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9a0 tx=0x7f4a5400ee60 comp rx=0 tx=0).stop 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 shutdown_connections 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4a580a2c30 0x7f4a580a50f0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4a6c19d310 0x7f4a6c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4a6c107740 0x7f4a6c1982f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 --2- 172.21.15.52:0/3963833363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4a6c104e10 0x7f4a6c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.253+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 >> 172.21.15.52:0/3963833363 conn(0x7f4a6c0fbe90 msgr2=0x7f4a6c102e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.254+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 shutdown_connections 2024-06-02T23:18:08.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:08.254+0000 7f4a729a5640 1 -- 172.21.15.52:0/3963833363 wait complete. 2024-06-02T23:18:08.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.64 MB/sec execute 197 sec latency 25621.859 ms 2024-06-02T23:18:09.049 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:09.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.63 MB/sec execute 198 sec latency 26621.947 ms 2024-06-02T23:18:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:09 smithi174 ceph-mon[35994]: pgmap v619: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-06-02T23:18:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:09 smithi052 ceph-mon[32340]: pgmap v619: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-06-02T23:18:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:09 smithi123 ceph-mon[35977]: pgmap v619: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-06-02T23:18:10.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108077 1.63 MB/sec execute 199 sec latency 27621.999 ms 2024-06-02T23:18:10.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 -- 172.21.15.52:0/489919186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028101440 msgr2=0x7f502810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 --2- 172.21.15.52:0/489919186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028101440 0x7f502810f330 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f502000b0a0 tx=0x7f502000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 -- 172.21.15.52:0/489919186 shutdown_connections 2024-06-02T23:18:10.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 --2- 172.21.15.52:0/489919186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028101440 0x7f502810f330 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 --2- 172.21.15.52:0/489919186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5028108510 0x7f50281088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 --2- 172.21.15.52:0/489919186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 0x7f5028100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 -- 172.21.15.52:0/489919186 >> 172.21.15.52:0/489919186 conn(0x7f5028077a00 msgr2=0x7f50280fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 -- 172.21.15.52:0/489919186 shutdown_connections 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.791+0000 7f502e120700 1 -- 172.21.15.52:0/489919186 wait complete. 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 Processor -- start 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 -- start start 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 0x7f5028101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5028101440 0x7f50281021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 0x7f50281026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5028112620 con 0x7f5028108510 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50281124a0 con 0x7f5028100a80 2024-06-02T23:18:10.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.792+0000 7f502e120700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50281127a0 con 0x7f5028101440 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 0x7f50281026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5028101440 0x7f50281021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 0x7f50281026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54798/0 (socket says 172.21.15.52:54798) 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 0x7f5028101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5027fff700 1 -- 172.21.15.52:0/3503518834 learned_addr learned my addr 172.21.15.52:0/3503518834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 -- 172.21.15.52:0/3503518834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5028101440 msgr2=0x7f50281021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5028101440 0x7f50281021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 -- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 msgr2=0x7f50281026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 0x7f50281026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 -- 172.21.15.52:0/3503518834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50281a7020 con 0x7f5028100a80 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5027fff700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 0x7f50281026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:10.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f50277fe700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 0x7f5028101c60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f501800d950 tx=0x7f501800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:10.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5024ff9700 1 -- 172.21.15.52:0/3503518834 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5018016070 con 0x7f5028100a80 2024-06-02T23:18:10.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.793+0000 7f5024ff9700 1 -- 172.21.15.52:0/3503518834 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f501800fd40 con 0x7f5028100a80 2024-06-02T23:18:10.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.794+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50281a7310 con 0x7f5028100a80 2024-06-02T23:18:10.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.794+0000 7f5024ff9700 1 -- 172.21.15.52:0/3503518834 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501800e3b0 con 0x7f5028100a80 2024-06-02T23:18:10.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.794+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50281a78a0 con 0x7f5028100a80 2024-06-02T23:18:10.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.796+0000 7f5024ff9700 1 -- 172.21.15.52:0/3503518834 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5018019070 con 0x7f5028100a80 2024-06-02T23:18:10.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.797+0000 7f5024ff9700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50140a2ca0 0x7f50140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.797+0000 7f5024ff9700 1 -- 172.21.15.52:0/3503518834 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f50180ce910 con 0x7f5028100a80 2024-06-02T23:18:10.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.797+0000 7f5026ffd700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50140a2ca0 0x7f50140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.797+0000 7f50389a3b80 1 Processor -- start 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 -- start start 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5026ffd700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50140a2ca0 0x7f50140a5160 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5028107bb0 tx=0x7f5010009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ea3296060 con 0x563ea33ac460 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ea32f4950 con 0x563ea33b1d10 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50389a3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ea325c560 con 0x563ea33b2630 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54804/0 (socket says 172.21.15.52:54804) 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 -- 172.21.15.52:0/141977771 learned_addr learned my addr 172.21.15.52:0/141977771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:10.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f50277fe700 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5026ffd700 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 -- 172.21.15.52:0/141977771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 msgr2=0x563ea33abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 -- 172.21.15.52:0/141977771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 msgr2=0x563ea33b20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.798+0000 7f5027fff700 1 -- 172.21.15.52:0/141977771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea32439d0 con 0x563ea33ac460 2024-06-02T23:18:10.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.799+0000 7f5027fff700 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33ac880 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5020009e00 tx=0x7f50200092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0d85710734d7df server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:10.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.799+0000 7f501dffb700 1 -- 172.21.15.52:0/141977771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5020021070 con 0x563ea33ac460 2024-06-02T23:18:10.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.799+0000 7f501dffb700 1 -- 172.21.15.52:0/141977771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5020004c50 con 0x563ea33ac460 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.799+0000 7f501dffb700 1 -- 172.21.15.52:0/141977771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50200052a0 con 0x563ea33ac460 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 -- 172.21.15.52:0/141977771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 msgr2=0x563ea33ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33ac880 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f5020009e00 tx=0x7f50200092e0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 -- 172.21.15.52:0/141977771 shutdown_connections 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33ac880 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 --2- 172.21.15.52:0/141977771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 -- 172.21.15.52:0/141977771 >> 172.21.15.52:0/141977771 conn(0x563ea32d0180 msgr2=0x563ea339ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 -- 172.21.15.52:0/141977771 shutdown_connections 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 -- 172.21.15.52:0/141977771 wait complete. 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 -- start start 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.800+0000 7f50389a3b80 1 Processor -- start 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33b8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ea32950c0 con 0x563ea33ac460 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ea32f0910 con 0x563ea33b1d10 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ea33b8fb0 con 0x563ea33b2630 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50389a3b80 10 client.0 _ll_register_callbacks cb 0x563ea328a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33b8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33b8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54808/0 (socket says 172.21.15.52:54808) 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 -- 172.21.15.52:0/3485282296 learned_addr learned my addr 172.21.15.52:0/3485282296 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f5027fff700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f5026ffd700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 -- 172.21.15.52:0/3485282296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 msgr2=0x563ea33b5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33b5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 -- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 msgr2=0x563ea33b5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.801+0000 7f50277fe700 1 -- 172.21.15.52:0/3485282296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea32ba850 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f50277fe700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33b8370 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f50180cc3e0 tx=0x7f501809ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6b22a449b2f253 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f5006ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f5006ffd700 1 -- 172.21.15.52:0/3485282296 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50180b4060 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f5006ffd700 1 -- 172.21.15.52:0/3485282296 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f501809f170 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5020004c50 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f5006ffd700 1 -- 172.21.15.52:0/3485282296 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5018020040 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f50389a3b80 10 client.17316 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ea33c3710 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f5006ffd700 1 -- 172.21.15.52:0/3485282296 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5020004c50 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f5006ffd700 1 -- 172.21.15.52:0/3485282296 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f501809b3a0 con 0x563ea33ac460 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f50389a3b80 10 client.17316 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:10.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.802+0000 7f50389a3b80 10 client.17316 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f50389a3b80 20 client.17316 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f50389a3b80 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea33c5d50 0x563ea33c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f50389a3b80 4 client.17316 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ea3301e50 con 0x563ea33c5d50 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f5026ffd700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea33c5d50 0x563ea33c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f5026ffd700 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea33c5d50 0x563ea33c8180 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cbf70e846200909 server_cookie=45b52a15dae878d0 in_seq=0 out_seq=0 2024-06-02T23:18:10.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.803+0000 7f5006ffd700 10 client.17316 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f5006ffd700 1 -- 172.21.15.52:0/3485282296 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ea3301e50 con 0x563ea33c5d50 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f5006ffd700 10 client.17316 handle_command_reply: tid=0 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 client.17316 shutdown 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 msgr2=0x563ea33b8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33b8370 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f50180cc3e0 tx=0x7f501809ad00 comp rx=0 tx=0).stop 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 shutdown_connections 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea33c5d50 0x563ea33c8180 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33b2630 0x563ea33b5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33ac460 0x563ea33b8370 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 --2- 172.21.15.52:0/3485282296 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b1d10 0x563ea33b5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 >> 172.21.15.52:0/3485282296 conn(0x563ea32d0180 msgr2=0x563ea339dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.804+0000 7f5006ffd700 0 client.17316 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.805+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 shutdown_connections 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.805+0000 7f50389a3b80 1 -- 172.21.15.52:0/3485282296 wait complete. 2024-06-02T23:18:10.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.805+0000 7f50389a3b80 20 client.17316 trim_cache size 0 max 16384 2024-06-02T23:18:10.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 Processor -- start 2024-06-02T23:18:10.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 -- start start 2024-06-02T23:18:10.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33abf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33b1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50281aa840 con 0x563ea33b1d10 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50281ac380 con 0x563ea33b2630 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50389a3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f50281aaf10 con 0x563ea33ac460 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54812/0 (socket says 172.21.15.52:54812) 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f5026ffd700 1 -- 172.21.15.52:0/257783969 learned_addr learned my addr 172.21.15.52:0/257783969 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f50277fe700 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f5027fff700 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.825+0000 7f5026ffd700 1 -- 172.21.15.52:0/257783969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 msgr2=0x563ea33abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f5026ffd700 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f5026ffd700 1 -- 172.21.15.52:0/257783969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 msgr2=0x563ea33b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f5026ffd700 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f5026ffd700 1 -- 172.21.15.52:0/257783969 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ea33964a0 con 0x563ea33b1d10 2024-06-02T23:18:10.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f5027fff700 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:18:10.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f5026ffd700 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33b0e50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f50100097c0 tx=0x7f5010014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539375f59dfe15ca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:10.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f501d7fa700 1 -- 172.21.15.52:0/257783969 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010003680 con 0x563ea33b1d10 2024-06-02T23:18:10.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f501d7fa700 1 -- 172.21.15.52:0/257783969 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5010003cd0 con 0x563ea33b1d10 2024-06-02T23:18:10.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.826+0000 7f501d7fa700 1 -- 172.21.15.52:0/257783969 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010029cf0 con 0x563ea33b1d10 2024-06-02T23:18:10.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 -- 172.21.15.52:0/257783969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 msgr2=0x563ea33b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33b0e50 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f50100097c0 tx=0x7f5010014000 comp rx=0 tx=0).stop 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 -- 172.21.15.52:0/257783969 shutdown_connections 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 --2- 172.21.15.52:0/257783969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33b0e50 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 -- 172.21.15.52:0/257783969 >> 172.21.15.52:0/257783969 conn(0x563ea32d0180 msgr2=0x563ea33b0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 -- 172.21.15.52:0/257783969 shutdown_connections 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 -- 172.21.15.52:0/257783969 wait complete. 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 -- start start 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.827+0000 7f50389a3b80 1 Processor -- start 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33be7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33bb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33bb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f50281abcb0 con 0x563ea33b1d10 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50281ab5e0 con 0x563ea33b2630 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f502819c660 con 0x563ea33ac460 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50389a3b80 10 client.0 _ll_register_callbacks cb 0x563ea328a6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33bb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33bb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54822/0 (socket says 172.21.15.52:54822) 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 -- 172.21.15.52:0/2616427801 learned_addr learned my addr 172.21.15.52:0/2616427801 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f50277fe700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33be7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5027fff700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33bb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 -- 172.21.15.52:0/2616427801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 msgr2=0x563ea33be7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33be7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 -- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 msgr2=0x563ea33bb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33bb780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 -- 172.21.15.52:0/2616427801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ea33bbd40 con 0x563ea33b1d10 2024-06-02T23:18:10.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5027fff700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5026ffd700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33bb240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5010005cb0 tx=0x7f5010003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff96dea3158dde12 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.828+0000 7f5006ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f5006ffd700 1 -- 172.21.15.52:0/2616427801 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010037020 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f5006ffd700 1 -- 172.21.15.52:0/2616427801 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5010003880 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5010003cd0 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f5006ffd700 1 -- 172.21.15.52:0/2616427801 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5010003a80 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 10 client.17322 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563ea34317a0 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f5006ffd700 1 -- 172.21.15.52:0/2616427801 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5010003cd0 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f5006ffd700 1 -- 172.21.15.52:0/2616427801 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5010005540 con 0x563ea33b1d10 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 10 client.17322 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 10 client.17322 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 20 client.17322 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:10.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea3431e30 0x563ea3434250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:10.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 4 client.17322 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "1cfe3ad1", "release": true, "await": true}] 2024-06-02T23:18:10.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.829+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "1cfe3ad1", "release": true, "await": true}) -- 0x563ea32fc3d0 con 0x563ea3431e30 2024-06-02T23:18:10.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.830+0000 7f50277fe700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea3431e30 0x563ea3434250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:10.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.830+0000 7f50277fe700 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea3431e30 0x563ea3434250 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c30744af1aead4 server_cookie=aa50de9c1d4ce8fc in_seq=0 out_seq=0 2024-06-02T23:18:10.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.830+0000 7f5006ffd700 10 client.17322 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f5006ffd700 1 -- 172.21.15.52:0/2616427801 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563ea32fc3d0 con 0x563ea3431e30 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f5006ffd700 10 client.17322 handle_command_reply: tid=0 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 client.17322 shutdown 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 msgr2=0x563ea33bb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33bb240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5010005cb0 tx=0x7f5010003f30 comp rx=0 tx=0).stop 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 shutdown_connections 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ea3431e30 0x563ea3434250 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ea33b2630 0x563ea33bb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ea33ac460 0x563ea33be7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f5006ffd700 0 client.17322 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 --2- 172.21.15.52:0/2616427801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ea33b1d10 0x563ea33bb240 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 >> 172.21.15.52:0/2616427801 conn(0x563ea32d0180 msgr2=0x563ea33b0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:10.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 shutdown_connections 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 1 -- 172.21.15.52:0/2616427801 wait complete. 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.837+0000 7f50389a3b80 20 client.17322 trim_cache size 0 max 16384 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 24, 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:18:10.839 INFO:teuthology.orchestra.run.smithi052.stdout: "1cfe3ad1": { 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 24, 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 144.1, 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:18:10.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:10.841 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50140a2ca0 msgr2=0x7f50140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50140a2ca0 0x7f50140a5160 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f5028107bb0 tx=0x7f5010009260 comp rx=0 tx=0).stop 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 msgr2=0x7f5028101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 0x7f5028101c60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f501800d950 tx=0x7f501800dc60 comp rx=0 tx=0).stop 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 shutdown_connections 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f50140a2ca0 0x7f50140a5160 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5028108510 0x7f50281026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5028100a80 0x7f5028101c60 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 --2- 172.21.15.52:0/3503518834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5028101440 0x7f50281021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 >> 172.21.15.52:0/3503518834 conn(0x7f5028077a00 msgr2=0x7f50280ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 shutdown_connections 2024-06-02T23:18:10.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:10.838+0000 7f502e120700 1 -- 172.21.15.52:0/3503518834 wait complete. 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 -- 172.21.15.52:0/2848388187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0100a80 msgr2=0x7f16e0100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 --2- 172.21.15.52:0/2848388187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0100a80 0x7f16e0100f00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f16dc009990 tx=0x7f16dc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 -- 172.21.15.52:0/2848388187 shutdown_connections 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 --2- 172.21.15.52:0/2848388187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0101440 0x7f16e010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 --2- 172.21.15.52:0/2848388187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0108510 0x7f16e01088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 --2- 172.21.15.52:0/2848388187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0100a80 0x7f16e0100f00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 -- 172.21.15.52:0/2848388187 >> 172.21.15.52:0/2848388187 conn(0x7f16e0077a00 msgr2=0x7f16e00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 -- 172.21.15.52:0/2848388187 shutdown_connections 2024-06-02T23:18:11.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.514+0000 7f16e7d77700 1 -- 172.21.15.52:0/2848388187 wait complete. 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 Processor -- start 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 -- start start 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100a80 0x7f16e0101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0101440 0x7f16e01021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 0x7f16e01026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f16e0112640 con 0x7f16e0108510 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01124c0 con 0x7f16e0101440 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e7d77700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16e01127c0 con 0x7f16e0100a80 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e6314700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 0x7f16e01026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.515+0000 7f16e5312700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0101440 0x7f16e01021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:11.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100a80 0x7f16e0101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e6314700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 0x7f16e01026e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54844/0 (socket says 172.21.15.52:54844) 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5312700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0101440 0x7f16e01021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43968/0 (socket says 172.21.15.52:43968) 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e6314700 1 -- 172.21.15.52:0/1042440758 learned_addr learned my addr 172.21.15.52:0/1042440758 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 -- 172.21.15.52:0/1042440758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0101440 msgr2=0x7f16e01021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0101440 0x7f16e01021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 -- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 msgr2=0x7f16e01026e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 0x7f16e01026e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 -- 172.21.15.52:0/1042440758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16e01a7000 con 0x7f16e0100a80 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e6314700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 0x7f16e01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e5b13700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100a80 0x7f16e0101c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e9f0 tx=0x7f16d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d400c9c0 con 0x7f16e0100a80 2024-06-02T23:18:11.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f16d400cb60 con 0x7f16e0100a80 2024-06-02T23:18:11.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16e01a72f0 con 0x7f16e0100a80 2024-06-02T23:18:11.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16d4018680 con 0x7f16e0100a80 2024-06-02T23:18:11.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.516+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16e01a7850 con 0x7f16e0100a80 2024-06-02T23:18:11.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.517+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16e0069180 con 0x7f16e0100a80 2024-06-02T23:18:11.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.519+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f16d4010430 con 0x7f16e0100a80 2024-06-02T23:18:11.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.520+0000 7f16d2ffd700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16cc0a2ca0 0x7f16cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:11.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.520+0000 7f16e5312700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16cc0a2ca0 0x7f16cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:11.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.520+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f16d40cf520 con 0x7f16e0100a80 2024-06-02T23:18:11.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.520+0000 7f16e5312700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16cc0a2ca0 0x7f16cc0a5160 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b2e0 tx=0x7f16dc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:11.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.523+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f16d409c450 con 0x7f16e0100a80 2024-06-02T23:18:11.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 108561 1.62 MB/sec execute 200 sec latency 27824.404 ms 2024-06-02T23:18:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:11 smithi052 ceph-mon[32340]: pgmap v620: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-06-02T23:18:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.831+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16e0103190 con 0x7f16e0100a80 2024-06-02T23:18:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.832+0000 7f16d2ffd700 1 -- 172.21.15.52:0/1042440758 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f16d409bd50 con 0x7f16e0100a80 2024-06-02T23:18:11.832 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:18:11.833 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:18:11.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16cc0a2ca0 msgr2=0x7f16cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16cc0a2ca0 0x7f16cc0a5160 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f16dc00b2e0 tx=0x7f16dc005960 comp rx=0 tx=0).stop 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100a80 msgr2=0x7f16e0101c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100a80 0x7f16e0101c60 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f16d400e9f0 tx=0x7f16d400ed00 comp rx=0 tx=0).stop 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 shutdown_connections 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f16cc0a2ca0 0x7f16cc0a5160 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f16e0108510 0x7f16e01026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16e0100a80 0x7f16e0101c60 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 --2- 172.21.15.52:0/1042440758 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16e0101440 0x7f16e01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 >> 172.21.15.52:0/1042440758 conn(0x7f16e0077a00 msgr2=0x7f16e00ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 shutdown_connections 2024-06-02T23:18:11.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:11.834+0000 7f16e7d77700 1 -- 172.21.15.52:0/1042440758 wait complete. 2024-06-02T23:18:11.836 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:18:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:11 smithi174 ceph-mon[35994]: pgmap v620: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-06-02T23:18:12.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:11 smithi123 ceph-mon[35977]: pgmap v620: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.3 KiB/s wr, 12 op/s 2024-06-02T23:18:12.049 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '1cfe3ad1', total seconds elapsed: 34.2 2024-06-02T23:18:12.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 109050 1.62 MB/sec execute 201 sec latency 14.818 ms 2024-06-02T23:18:12.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1042440758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:18:12.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:18:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1042440758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:18:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1042440758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:18:13.021 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:13.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 109766 1.65 MB/sec execute 202 sec latency 11.315 ms 2024-06-02T23:18:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:13 smithi052 ceph-mon[32340]: pgmap v621: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-02T23:18:14.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:13 smithi123 ceph-mon[35977]: pgmap v621: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-02T23:18:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:13 smithi174 ceph-mon[35994]: pgmap v621: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-06-02T23:18:14.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 110232 1.66 MB/sec execute 203 sec latency 14.603 ms 2024-06-02T23:18:14.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.888+0000 7f780d9d2700 1 -- 172.21.15.52:0/1113314048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 msgr2=0x7f7808101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.888+0000 7f780d9d2700 1 --2- 172.21.15.52:0/1113314048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808101800 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009990 tx=0x7f77fc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 -- 172.21.15.52:0/1113314048 shutdown_connections 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 --2- 172.21.15.52:0/1113314048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7808101d40 0x7f780810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 --2- 172.21.15.52:0/1113314048 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 0x7f78081077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 --2- 172.21.15.52:0/1113314048 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808101800 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 -- 172.21.15.52:0/1113314048 >> 172.21.15.52:0/1113314048 conn(0x7f7808078300 msgr2=0x7f7808078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 -- 172.21.15.52:0/1113314048 shutdown_connections 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 -- 172.21.15.52:0/1113314048 wait complete. 2024-06-02T23:18:14.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.889+0000 7f780d9d2700 1 Processor -- start 2024-06-02T23:18:14.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 -- start start 2024-06-02T23:18:14.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7808101d40 0x7f78081035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 0x7f7808103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7808111540 con 0x7f7808101380 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78081113c0 con 0x7f78081073f0 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f780d9d2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78081116c0 con 0x7f7808101d40 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52408/0 (socket says 172.21.15.52:52408) 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f78077fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 0x7f7808103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f7806ffd700 1 -- 172.21.15.52:0/2930933309 learned_addr learned my addr 172.21.15.52:0/2930933309 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.890+0000 7f78067fc700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7808101d40 0x7f78081035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f78077fe700 1 -- 172.21.15.52:0/2930933309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7808101d40 msgr2=0x7f78081035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f78077fe700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7808101d40 0x7f78081035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f78077fe700 1 -- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 msgr2=0x7f7808103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f78077fe700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f78077fe700 1 -- 172.21.15.52:0/2930933309 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78081a70d0 con 0x7f78081073f0 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f78077fe700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 0x7f7808103ae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f780000efc0 tx=0x7f780000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f780c9d0700 1 -- 172.21.15.52:0/2930933309 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000092e0 con 0x7f78081073f0 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f780c9d0700 1 -- 172.21.15.52:0/2930933309 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7800004bb0 con 0x7f78081073f0 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78081a7360 con 0x7f78081073f0 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f780c9d0700 1 -- 172.21.15.52:0/2930933309 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7800004020 con 0x7f78081073f0 2024-06-02T23:18:14.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.891+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78081a77b0 con 0x7f78081073f0 2024-06-02T23:18:14.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.894+0000 7f780c9d0700 1 -- 172.21.15.52:0/2930933309 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f780001b030 con 0x7f78081073f0 2024-06-02T23:18:14.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.895+0000 7f780c9d0700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f77f00a2c20 0x7f77f00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.895+0000 7f780c9d0700 1 -- 172.21.15.52:0/2930933309 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f78000ceb60 con 0x7f78081073f0 2024-06-02T23:18:14.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.895+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f77f00a2c20 0x7f77f00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.895+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f77f00a2c20 0x7f77f00a50e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77f8011550 tx=0x7f77f8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:14.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 Processor -- start 2024-06-02T23:18:14.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 -- start start 2024-06-02T23:18:14.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db556340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 0x5568db550260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db5507a0 0x5568db550bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5568db442f60 con 0x5568db556880 2024-06-02T23:18:14.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568db4a1850 con 0x5568db5507a0 2024-06-02T23:18:14.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7818255b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5568db409460 con 0x5568db555f60 2024-06-02T23:18:14.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f78067fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 0x5568db550260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f78067fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 0x5568db550260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52414/0 (socket says 172.21.15.52:52414) 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db556340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f78067fc700 1 -- 172.21.15.52:0/1846476794 learned_addr learned my addr 172.21.15.52:0/1846476794 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.896+0000 7f78077fe700 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db5507a0 0x5568db550bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78077fe700 1 -- 172.21.15.52:0/1846476794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 msgr2=0x5568db556340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78077fe700 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db556340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78077fe700 1 -- 172.21.15.52:0/1846476794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 msgr2=0x5568db550260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78077fe700 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 0x5568db550260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78077fe700 1 -- 172.21.15.52:0/1846476794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568db3f08d0 con 0x5568db5507a0 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78067fc700 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 0x5568db550260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f78077fe700 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db5507a0 0x5568db550bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78000cc630 tx=0x7f7800000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9697735887d00718 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f77edffb700 1 -- 172.21.15.52:0/1846476794 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780009bb20 con 0x5568db5507a0 2024-06-02T23:18:14.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f77edffb700 1 -- 172.21.15.52:0/1846476794 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f780009b420 con 0x5568db5507a0 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.897+0000 7f77edffb700 1 -- 172.21.15.52:0/1846476794 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000ccaf0 con 0x5568db5507a0 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 -- 172.21.15.52:0/1846476794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db5507a0 msgr2=0x5568db550bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db5507a0 0x5568db550bc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f78000cc630 tx=0x7f7800000f40 comp rx=0 tx=0).stop 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 -- 172.21.15.52:0/1846476794 shutdown_connections 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db5507a0 0x5568db550bc0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db556340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 --2- 172.21.15.52:0/1846476794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db556880 0x5568db550260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 -- 172.21.15.52:0/1846476794 >> 172.21.15.52:0/1846476794 conn(0x5568db47d080 msgr2=0x5568db54b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 -- 172.21.15.52:0/1846476794 shutdown_connections 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 -- 172.21.15.52:0/1846476794 wait complete. 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.898+0000 7f7818255b80 1 -- start start 2024-06-02T23:18:14.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 Processor -- start 2024-06-02T23:18:14.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db55c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db559970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db556880 0x5568db559eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5568db441fc0 con 0x5568db5507a0 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568db49d810 con 0x5568db556880 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5568db55d260 con 0x5568db555f60 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7818255b80 10 client.0 _ll_register_callbacks cb 0x5568db4375b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db55c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db55c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52418/0 (socket says 172.21.15.52:52418) 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78077fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db556880 0x5568db559eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7806ffd700 1 -- 172.21.15.52:0/2791661699 learned_addr learned my addr 172.21.15.52:0/2791661699 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78067fc700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db559970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78077fe700 1 -- 172.21.15.52:0/2791661699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 msgr2=0x5568db559970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78077fe700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db559970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78077fe700 1 -- 172.21.15.52:0/2791661699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 msgr2=0x5568db55c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78077fe700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db55c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f78077fe700 1 -- 172.21.15.52:0/2791661699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568db467750 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.899+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db55c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f78077fe700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db556880 0x5568db559eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f780009b3b0 tx=0x7f7800004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43afa99874915b9c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f77e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f77e6ffd700 1 -- 172.21.15.52:0/2791661699 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78000b9060 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f77e6ffd700 1 -- 172.21.15.52:0/2791661699 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f780009a870 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f780009b420 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f77e6ffd700 1 -- 172.21.15.52:0/2791661699 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f780009bb20 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f7818255b80 10 client.26834 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5568db567a40 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f77e6ffd700 1 -- 172.21.15.52:0/2791661699 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f780009b420 con 0x5568db556880 2024-06-02T23:18:14.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f77e6ffd700 1 -- 172.21.15.52:0/2791661699 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7800066190 con 0x5568db556880 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.900+0000 7f7818255b80 10 client.26834 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7818255b80 10 client.26834 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7818255b80 20 client.26834 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7818255b80 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db569f10 0x5568db56c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7818255b80 4 client.26834 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568db4aed50 con 0x5568db569f10 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db569f10 0x5568db56c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f77e6ffd700 10 client.26834 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.901+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db569f10 0x5568db56c300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b13d1125fb7029fa server_cookie=18c45fefa5dbe66f in_seq=0 out_seq=0 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f77e6ffd700 1 -- 172.21.15.52:0/2791661699 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568db4aed50 con 0x5568db569f10 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f77e6ffd700 10 client.26834 handle_command_reply: tid=0 2024-06-02T23:18:14.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 client.26834 shutdown 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db556880 msgr2=0x5568db559eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db556880 0x5568db559eb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f780009b3b0 tx=0x7f7800004ad0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 shutdown_connections 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db569f10 0x5568db56c300 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db556880 0x5568db559eb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db55c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f77e6ffd700 0 client.26834 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 --2- 172.21.15.52:0/2791661699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db555f60 0x5568db559970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 >> 172.21.15.52:0/2791661699 conn(0x5568db47d080 msgr2=0x5568db54a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 shutdown_connections 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 1 -- 172.21.15.52:0/2791661699 wait complete. 2024-06-02T23:18:14.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.902+0000 7f7818255b80 20 client.26834 trim_cache size 0 max 16384 2024-06-02T23:18:14.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 Processor -- start 2024-06-02T23:18:14.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 -- start start 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db550260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db557450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db54ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78081aed20 con 0x5568db5507a0 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78081b0860 con 0x5568db555f60 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.919+0000 7f7818255b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78081af3f0 con 0x5568db556880 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db550260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db550260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52428/0 (socket says 172.21.15.52:52428) 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f78077fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db54ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 -- 172.21.15.52:0/2573434967 learned_addr learned my addr 172.21.15.52:0/2573434967 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f78067fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db557450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 -- 172.21.15.52:0/2573434967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 msgr2=0x5568db54ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db54ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 -- 172.21.15.52:0/2573434967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 msgr2=0x5568db557450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db557450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 -- 172.21.15.52:0/2573434967 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568db54ccc0 con 0x5568db5507a0 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f78067fc700 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db557450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f78077fe700 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db54ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f7806ffd700 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db550260 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f77f8013010 tx=0x7f77f8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f15ae4ab47534a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f77ed7fa700 1 -- 172.21.15.52:0/2573434967 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f800e9f0 con 0x5568db5507a0 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f77ed7fa700 1 -- 172.21.15.52:0/2573434967 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77f8004830 con 0x5568db5507a0 2024-06-02T23:18:14.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.920+0000 7f77ed7fa700 1 -- 172.21.15.52:0/2573434967 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f8005280 con 0x5568db5507a0 2024-06-02T23:18:14.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 -- 172.21.15.52:0/2573434967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 msgr2=0x5568db550260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db550260 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f77f8013010 tx=0x7f77f8014000 comp rx=0 tx=0).stop 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 -- 172.21.15.52:0/2573434967 shutdown_connections 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db54ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db550260 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 --2- 172.21.15.52:0/2573434967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db557450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 -- 172.21.15.52:0/2573434967 >> 172.21.15.52:0/2573434967 conn(0x5568db47d080 msgr2=0x5568db556d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 -- 172.21.15.52:0/2573434967 shutdown_connections 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.921+0000 7f7818255b80 1 -- 172.21.15.52:0/2573434967 wait complete. 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 -- start start 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 Processor -- start 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db56c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db56ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db5d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78081b0190 con 0x5568db5507a0 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78081afac0 con 0x5568db555f60 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78081acce0 con 0x5568db556880 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7818255b80 10 client.0 _ll_register_callbacks cb 0x5568db4375b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db56c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db56c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52442/0 (socket says 172.21.15.52:52442) 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f7806ffd700 1 -- 172.21.15.52:0/1786591472 learned_addr learned my addr 172.21.15.52:0/1786591472 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f78077fe700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db5d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.922+0000 7f78067fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db56ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7806ffd700 1 -- 172.21.15.52:0/1786591472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 msgr2=0x5568db5d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7806ffd700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db5d2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7806ffd700 1 -- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 msgr2=0x5568db56ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7806ffd700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db56ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7806ffd700 1 -- 172.21.15.52:0/1786591472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568db5d2d80 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f78067fc700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db56ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f78077fe700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db5d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7806ffd700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db56c500 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e400 tx=0x7f77f80090b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e170b99459901a29 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f77e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f77e6ffd700 1 -- 172.21.15.52:0/1786591472 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77f801fc70 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f77e6ffd700 1 -- 172.21.15.52:0/1786591472 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f77f802a3c0 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.923+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77f8004830 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f77e6ffd700 1 -- 172.21.15.52:0/1786591472 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f77f801fe10 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 10 client.17346 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5568db5d3f60 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f77e6ffd700 1 -- 172.21.15.52:0/1786591472 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f77f8004830 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f77e6ffd700 1 -- 172.21.15.52:0/1786591472 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f77f8033870 con 0x5568db5507a0 2024-06-02T23:18:14.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 10 client.17346 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 10 client.17346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 20 client.17346 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db5d45d0 0x5568db5d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 4 client.17346 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568db4a92d0 con 0x5568db5d45d0 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.924+0000 7f78067fc700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db5d45d0 0x5568db5d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f78067fc700 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db5d45d0 0x5568db5d69f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=dab6a5b82f9cc1ab server_cookie=468d3b01aa0621e6 in_seq=0 out_seq=0 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f77e6ffd700 10 client.17346 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f77e6ffd700 1 -- 172.21.15.52:0/1786591472 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568db4a92d0 con 0x5568db5d45d0 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f77e6ffd700 10 client.17346 handle_command_reply: tid=0 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 client.17346 shutdown 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 msgr2=0x5568db56c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db56c500 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f77f800e400 tx=0x7f77f80090b0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 shutdown_connections 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568db556880 0x5568db5d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568db5507a0 0x5568db56c500 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5568db5d45d0 0x5568db5d69f0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 --2- 172.21.15.52:0/1786591472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568db555f60 0x5568db56ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f77e6ffd700 0 client.17346 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 >> 172.21.15.52:0/1786591472 conn(0x5568db47d080 msgr2=0x5568db558130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.925+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 shutdown_connections 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f7818255b80 1 -- 172.21.15.52:0/1786591472 wait complete. 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f7818255b80 20 client.17346 trim_cache size 0 max 16384 2024-06-02T23:18:14.931 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f77f00a2c20 msgr2=0x7f77f00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f77f00a2c20 0x7f77f00a50e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f77f8011550 tx=0x7f77f8009260 comp rx=0 tx=0).stop 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 msgr2=0x7f7808103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 0x7f7808103ae0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f780000efc0 tx=0x7f780000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 shutdown_connections 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f77f00a2c20 0x7f77f00a50e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78081073f0 0x7f7808103ae0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7808101380 0x7f7808103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 --2- 172.21.15.52:0/2930933309 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7808101d40 0x7f78081035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:14.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.926+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 >> 172.21.15.52:0/2930933309 conn(0x7f7808078300 msgr2=0x7f78081006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:14.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.927+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 shutdown_connections 2024-06-02T23:18:14.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:14.927+0000 7f780d9d2700 1 -- 172.21.15.52:0/2930933309 wait complete. 2024-06-02T23:18:15.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 110471 1.65 MB/sec execute 204 sec latency 18.327 ms 2024-06-02T23:18:15.756 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:18:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:15 smithi052 ceph-mon[32340]: pgmap v622: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-06-02T23:18:15.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:15 smithi174 ceph-mon[35994]: pgmap v622: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-06-02T23:18:16.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:15 smithi123 ceph-mon[35977]: pgmap v622: 97 pgs: 97 active+clean; 57 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 KiB/s wr, 17 op/s 2024-06-02T23:18:16.027 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:16.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 110903 1.65 MB/sec execute 205 sec latency 14.320 ms 2024-06-02T23:18:17.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 111453 1.66 MB/sec execute 206 sec latency 21.523 ms 2024-06-02T23:18:17.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.823+0000 7fde2d937700 1 -- 172.21.15.52:0/1495967360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28108520 msgr2=0x7fde28108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.823+0000 7fde2d937700 1 --2- 172.21.15.52:0/1495967360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28108520 0x7fde28108900 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fde18009990 tx=0x7fde18009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.825 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:17 smithi052 ceph-mon[32340]: pgmap v623: 97 pgs: 97 active+clean; 59 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 319 KiB/s wr, 29 op/s 2024-06-02T23:18:17.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.823+0000 7fde2d937700 1 -- 172.21.15.52:0/1495967360 shutdown_connections 2024-06-02T23:18:17.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.823+0000 7fde2d937700 1 --2- 172.21.15.52:0/1495967360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde28102ed0 0x7fde2810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.823+0000 7fde2d937700 1 --2- 172.21.15.52:0/1495967360 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28108520 0x7fde28108900 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.823+0000 7fde2d937700 1 --2- 172.21.15.52:0/1495967360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28102510 0x7fde28102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.824+0000 7fde2d937700 1 -- 172.21.15.52:0/1495967360 >> 172.21.15.52:0/1495967360 conn(0x7fde28079560 msgr2=0x7fde28079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.824+0000 7fde2d937700 1 -- 172.21.15.52:0/1495967360 shutdown_connections 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.824+0000 7fde2d937700 1 -- 172.21.15.52:0/1495967360 wait complete. 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.824+0000 7fde2d937700 1 Processor -- start 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.824+0000 7fde2d937700 1 -- start start 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde2d937700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 0x7fde2819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde2d937700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28102ed0 0x7fde2819ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde2d937700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde28108520 0x7fde281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde2d937700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde28112650 con 0x7fde28102510 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde2d937700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde281124d0 con 0x7fde28108520 2024-06-02T23:18:17.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde2d937700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde281127d0 con 0x7fde28102ed0 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 0x7fde2819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 0x7fde2819e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52446/0 (socket says 172.21.15.52:52446) 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde267fc700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28102ed0 0x7fde2819ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde277fe700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde28108520 0x7fde281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 -- 172.21.15.52:0/2626767809 learned_addr learned my addr 172.21.15.52:0/2626767809 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 -- 172.21.15.52:0/2626767809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28102ed0 msgr2=0x7fde2819ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28102ed0 0x7fde2819ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 -- 172.21.15.52:0/2626767809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde28108520 msgr2=0x7fde281988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde28108520 0x7fde281988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 -- 172.21.15.52:0/2626767809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde28199050 con 0x7fde28102510 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.825+0000 7fde26ffd700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 0x7fde2819e6e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fde18005b40 tx=0x7fde18005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.826+0000 7fde2c935700 1 -- 172.21.15.52:0/2626767809 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18027070 con 0x7fde28102510 2024-06-02T23:18:17.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.826+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde281992e0 con 0x7fde28102510 2024-06-02T23:18:17.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.826+0000 7fde2c935700 1 -- 172.21.15.52:0/2626767809 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde18005690 con 0x7fde28102510 2024-06-02T23:18:17.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.826+0000 7fde2c935700 1 -- 172.21.15.52:0/2626767809 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde18022bc0 con 0x7fde28102510 2024-06-02T23:18:17.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.826+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde281a55b0 con 0x7fde28102510 2024-06-02T23:18:17.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.828+0000 7fde2c935700 1 -- 172.21.15.52:0/2626767809 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde1802a070 con 0x7fde28102510 2024-06-02T23:18:17.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.829+0000 7fde2c935700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fde100a2c20 0x7fde100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.829+0000 7fde2c935700 1 -- 172.21.15.52:0/2626767809 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fde180cff40 con 0x7fde28102510 2024-06-02T23:18:17.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.829+0000 7fde267fc700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fde100a2c20 0x7fde100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.829+0000 7fde267fc700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fde100a2c20 0x7fde100a50e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fde28079b40 tx=0x7fde1c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:17.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 Processor -- start 2024-06-02T23:18:17.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 -- start start 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404ce390 0x563a404c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404c8330 0x563a404c8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563a40415810 con 0x563a404cda70 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a403bd720 con 0x563a404c8330 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde381bab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a404117d0 con 0x563a404ce390 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52448/0 (socket says 172.21.15.52:52448) 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde267fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404ce390 0x563a404c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde26ffd700 1 -- 172.21.15.52:0/2737924400 learned_addr learned my addr 172.21.15.52:0/2737924400 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.830+0000 7fde277fe700 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404c8330 0x563a404c8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde267fc700 1 -- 172.21.15.52:0/2737924400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404c8330 msgr2=0x563a404c8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde267fc700 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404c8330 0x563a404c8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde267fc700 1 -- 172.21.15.52:0/2737924400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 msgr2=0x563a404cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde267fc700 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde267fc700 1 -- 172.21.15.52:0/2737924400 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a403b3f50 con 0x563a404ce390 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde26ffd700 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde267fc700 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404ce390 0x563a404c7df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005950 tx=0x7fde1c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2fc8e8a59f48643 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde0dffb700 1 -- 172.21.15.52:0/2737924400 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011bb0 con 0x563a404ce390 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde0dffb700 1 -- 172.21.15.52:0/2737924400 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde1c011d50 con 0x563a404ce390 2024-06-02T23:18:17.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.831+0000 7fde0dffb700 1 -- 172.21.15.52:0/2737924400 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c01e420 con 0x563a404ce390 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 -- 172.21.15.52:0/2737924400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404ce390 msgr2=0x563a404c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404ce390 0x563a404c7df0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fde1c005950 tx=0x7fde1c013000 comp rx=0 tx=0).stop 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 -- 172.21.15.52:0/2737924400 shutdown_connections 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404c8330 0x563a404c8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 --2- 172.21.15.52:0/2737924400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404ce390 0x563a404c7df0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 -- 172.21.15.52:0/2737924400 >> 172.21.15.52:0/2737924400 conn(0x563a403eecc0 msgr2=0x563a404be850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 -- 172.21.15.52:0/2737924400 shutdown_connections 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 -- 172.21.15.52:0/2737924400 wait complete. 2024-06-02T23:18:17.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.832+0000 7fde381bab80 1 -- start start 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 Processor -- start 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404c8330 0x563a404d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 0x563a404d1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563a404d4dd0 con 0x563a404c8330 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a403b5bf0 con 0x563a404ce390 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a404d4f50 con 0x563a404cda70 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde381bab80 10 client.0 _ll_register_callbacks cb 0x563a403af920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde267fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 0x563a404d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde267fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 0x563a404d1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43750/0 (socket says 172.21.15.52:43750) 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde267fc700 1 -- 172.21.15.52:0/1780625848 learned_addr learned my addr 172.21.15.52:0/1780625848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde277fe700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.833+0000 7fde26ffd700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404c8330 0x563a404d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde267fc700 1 -- 172.21.15.52:0/1780625848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 msgr2=0x563a404d1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde267fc700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404d1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde267fc700 1 -- 172.21.15.52:0/1780625848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404c8330 msgr2=0x563a404d4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde267fc700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404c8330 0x563a404d4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde267fc700 1 -- 172.21.15.52:0/1780625848 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a403648d0 con 0x563a404cda70 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde267fc700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 0x563a404d1400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00bfe0 tx=0x7fde1c00baf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9b844855a56da5f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde06ffd700 1 -- 172.21.15.52:0/1780625848 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c036060 con 0x563a404cda70 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde06ffd700 1 -- 172.21.15.52:0/1780625848 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fde1c0054c0 con 0x563a404cda70 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1c011d50 con 0x563a404cda70 2024-06-02T23:18:17.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde06ffd700 1 -- 172.21.15.52:0/1780625848 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde1c0056c0 con 0x563a404cda70 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.834+0000 7fde381bab80 10 client.26689 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a404df6e0 con 0x563a404cda70 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde06ffd700 1 -- 172.21.15.52:0/1780625848 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde1c004c00 con 0x563a404cda70 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde06ffd700 1 -- 172.21.15.52:0/1780625848 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fde1c031b60 con 0x563a404cda70 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 10 client.26689 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 10 client.26689 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 20 client.26689 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a404e4050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 4 client.26689 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fde1c011d50 con 0x563a404e1c60 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.835+0000 7fde26ffd700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a404e4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde26ffd700 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a404e4050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=45ccf85c90586e6f server_cookie=1728a9ee80b683d9 in_seq=0 out_seq=0 2024-06-02T23:18:17.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde06ffd700 10 client.26689 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde06ffd700 1 -- 172.21.15.52:0/1780625848 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fde1c011d50 con 0x563a404e1c60 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde06ffd700 10 client.26689 handle_command_reply: tid=0 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde381bab80 1 client.26689 shutdown 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 msgr2=0x563a404d1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.836+0000 7fde381bab80 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 0x563a404d1400 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00bfe0 tx=0x7fde1c00baf0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 shutdown_connections 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404d1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404c8330 0x563a404d4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a404e4050 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde06ffd700 0 client.26689 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 --2- 172.21.15.52:0/1780625848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404cda70 0x563a404d1400 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 >> 172.21.15.52:0/1780625848 conn(0x563a403eecc0 msgr2=0x563a404c29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:17.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 shutdown_connections 2024-06-02T23:18:17.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 1 -- 172.21.15.52:0/1780625848 wait complete. 2024-06-02T23:18:17.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.837+0000 7fde381bab80 20 client.26689 trim_cache size 0 max 16384 2024-06-02T23:18:17.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 Processor -- start 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 -- start start 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404cfd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde281aa6b0 con 0x563a404cda70 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde281ac1f0 con 0x563a404ce390 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde381bab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde281aad80 con 0x563a404c8330 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde267fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde267fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404be8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52468/0 (socket says 172.21.15.52:52468) 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde267fc700 1 -- 172.21.15.52:0/1688773341 learned_addr learned my addr 172.21.15.52:0/1688773341 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde26ffd700 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.854+0000 7fde277fe700 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404cfd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde267fc700 1 -- 172.21.15.52:0/1688773341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 msgr2=0x563a404be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde267fc700 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde267fc700 1 -- 172.21.15.52:0/1688773341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 msgr2=0x563a404cfd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde267fc700 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404cfd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde267fc700 1 -- 172.21.15.52:0/1688773341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a404d4870 con 0x563a404cda70 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde267fc700 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404be8c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fde1c010f10 tx=0x7fde1c00b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1273d4e41b6c8a7c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde0d7fa700 1 -- 172.21.15.52:0/1688773341 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c041020 con 0x563a404cda70 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde0d7fa700 1 -- 172.21.15.52:0/1688773341 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde1c004710 con 0x563a404cda70 2024-06-02T23:18:17.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.855+0000 7fde0d7fa700 1 -- 172.21.15.52:0/1688773341 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c036040 con 0x563a404cda70 2024-06-02T23:18:17.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 -- 172.21.15.52:0/1688773341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 msgr2=0x563a404be8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404be8c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fde1c010f10 tx=0x7fde1c00b970 comp rx=0 tx=0).stop 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 -- 172.21.15.52:0/1688773341 shutdown_connections 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404ce390 0x563a404cfd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 --2- 172.21.15.52:0/1688773341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404cda70 0x563a404be8c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 -- 172.21.15.52:0/1688773341 >> 172.21.15.52:0/1688773341 conn(0x563a403eecc0 msgr2=0x563a404d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 -- 172.21.15.52:0/1688773341 shutdown_connections 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 -- 172.21.15.52:0/1688773341 wait complete. 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.856+0000 7fde381bab80 1 -- start start 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 Processor -- start 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 0x563a404e4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 0x563a4054a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde281abb20 con 0x563a404ce390 2024-06-02T23:18:17.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde281ab450 con 0x563a404cda70 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde281a8690 con 0x563a404c8330 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde381bab80 10 client.0 _ll_register_callbacks cb 0x563a403af920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 0x563a4054a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde267fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 0x563a404e4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde26ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde267fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 0x563a404e4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46630/0 (socket says 172.21.15.52:46630) 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 0x563a4054a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52474/0 (socket says 172.21.15.52:52474) 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde267fc700 1 -- 172.21.15.52:0/88364424 learned_addr learned my addr 172.21.15.52:0/88364424 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde26ffd700 1 -- 172.21.15.52:0/88364424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 msgr2=0x563a404e4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde26ffd700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 0x563a404e4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde26ffd700 1 -- 172.21.15.52:0/88364424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 msgr2=0x563a4054a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde26ffd700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 0x563a4054a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde26ffd700 1 -- 172.21.15.52:0/88364424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a4054aad0 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde267fc700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 0x563a404e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.857+0000 7fde277fe700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 0x563a4054a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde26ffd700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404e45f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fde1800b5c0 tx=0x7fde180a3040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ae377654bd9e17c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde06ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde06ffd700 1 -- 172.21.15.52:0/88364424 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde180a30b0 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde06ffd700 1 -- 172.21.15.52:0/88364424 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fde1809ab60 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde1c004710 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde06ffd700 1 -- 172.21.15.52:0/88364424 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde18022470 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 10 client.26695 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a4054bd50 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde06ffd700 1 -- 172.21.15.52:0/88364424 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde1c004710 con 0x563a404c8330 2024-06-02T23:18:17.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde06ffd700 1 -- 172.21.15.52:0/88364424 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fde180a4fb0 con 0x563a404c8330 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 10 client.26695 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 10 client.26695 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 20 client.26695 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a4054e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 4 client.26695 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.858+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a40422d10 con 0x563a404e1c60 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde267fc700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a4054e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde06ffd700 10 client.26695 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:18:17.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde267fc700 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a4054e450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=cd2e3b077795499f server_cookie=1efd5a721a851e0a in_seq=0 out_seq=0 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde06ffd700 1 -- 172.21.15.52:0/88364424 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a40422d10 con 0x563a404e1c60 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde06ffd700 10 client.26695 handle_command_reply: tid=0 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde381bab80 1 client.26695 shutdown 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 msgr2=0x563a404e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.859+0000 7fde381bab80 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404e45f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fde1800b5c0 tx=0x7fde180a3040 comp rx=0 tx=0).stop 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 shutdown_connections 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a404ce390 0x563a4054a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x563a404e1c60 0x563a4054e450 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a404c8330 0x563a404e45f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 --2- 172.21.15.52:0/88364424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a404cda70 0x563a404e4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde06ffd700 0 client.26695 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 >> 172.21.15.52:0/88364424 conn(0x563a403eecc0 msgr2=0x563a404d13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:17.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 shutdown_connections 2024-06-02T23:18:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 1 -- 172.21.15.52:0/88364424 wait complete. 2024-06-02T23:18:17.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde381bab80 20 client.26695 trim_cache size 0 max 16384 2024-06-02T23:18:17.866 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fde100a2c20 msgr2=0x7fde100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fde100a2c20 0x7fde100a50e0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fde28079b40 tx=0x7fde1c00b410 comp rx=0 tx=0).stop 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 msgr2=0x7fde2819e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 0x7fde2819e6e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fde18005b40 tx=0x7fde18005e30 comp rx=0 tx=0).stop 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 shutdown_connections 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fde100a2c20 0x7fde100a50e0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde28108520 0x7fde281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde28102510 0x7fde2819e6e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 --2- 172.21.15.52:0/2626767809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde28102ed0 0x7fde2819ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.860+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 >> 172.21.15.52:0/2626767809 conn(0x7fde28079560 msgr2=0x7fde281011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.861+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 shutdown_connections 2024-06-02T23:18:17.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:17.861+0000 7fde2d937700 1 -- 172.21.15.52:0/2626767809 wait complete. 2024-06-02T23:18:18.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:17 smithi123 ceph-mon[35977]: pgmap v623: 97 pgs: 97 active+clean; 59 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 319 KiB/s wr, 29 op/s 2024-06-02T23:18:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:17 smithi174 ceph-mon[35994]: pgmap v623: 97 pgs: 97 active+clean; 59 MiB data, 990 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 319 KiB/s wr, 29 op/s 2024-06-02T23:18:18.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 112063 1.66 MB/sec execute 207 sec latency 22.979 ms 2024-06-02T23:18:18.679 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:18:18.956 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:19.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 112575 1.66 MB/sec execute 208 sec latency 15.296 ms 2024-06-02T23:18:20.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:19 smithi123 ceph-mon[35977]: pgmap v624: 97 pgs: 97 active+clean; 61 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 577 KiB/s wr, 62 op/s 2024-06-02T23:18:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:19 smithi174 ceph-mon[35994]: pgmap v624: 97 pgs: 97 active+clean; 61 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 577 KiB/s wr, 62 op/s 2024-06-02T23:18:20.148 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:19 smithi052 ceph-mon[32340]: pgmap v624: 97 pgs: 97 active+clean; 61 MiB data, 991 MiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 577 KiB/s wr, 62 op/s 2024-06-02T23:18:20.632 INFO:tasks.workunit.client.0.smithi052.stdout: 1 113277 1.68 MB/sec execute 209 sec latency 87.907 ms 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.732+0000 7ff527fd3700 1 -- 172.21.15.52:0/831943509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006a170 msgr2=0x7ff52006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.732+0000 7ff527fd3700 1 --2- 172.21.15.52:0/831943509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006a170 0x7ff52006a5f0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7ff510009990 tx=0x7ff510009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 -- 172.21.15.52:0/831943509 shutdown_connections 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 --2- 172.21.15.52:0/831943509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006ab30 0x7ff52010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 --2- 172.21.15.52:0/831943509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520108500 0x7ff5201088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 --2- 172.21.15.52:0/831943509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006a170 0x7ff52006a5f0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 -- 172.21.15.52:0/831943509 >> 172.21.15.52:0/831943509 conn(0x7ff520078340 msgr2=0x7ff520078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 -- 172.21.15.52:0/831943509 shutdown_connections 2024-06-02T23:18:20.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 -- 172.21.15.52:0/831943509 wait complete. 2024-06-02T23:18:20.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.733+0000 7ff527fd3700 1 Processor -- start 2024-06-02T23:18:20.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 -- start start 2024-06-02T23:18:20.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 0x7ff52019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 0x7ff52019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520108500 0x7ff5201989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff520112630 con 0x7ff52006ab30 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201124b0 con 0x7ff52006a170 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff527fd3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201127b0 con 0x7ff520108500 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff525d6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 0x7ff52019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 0x7ff52019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff525d6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 0x7ff52019e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46650/0 (socket says 172.21.15.52:46650) 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 0x7ff52019ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52494/0 (socket says 172.21.15.52:52494) 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff525d6f700 1 -- 172.21.15.52:0/3553657846 learned_addr learned my addr 172.21.15.52:0/3553657846 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff526570700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520108500 0x7ff5201989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 -- 172.21.15.52:0/3553657846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520108500 msgr2=0x7ff5201989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520108500 0x7ff5201989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 -- 172.21.15.52:0/3553657846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 msgr2=0x7ff52019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 0x7ff52019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff52556e700 1 -- 172.21.15.52:0/3553657846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff520199150 con 0x7ff52006ab30 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.734+0000 7ff525d6f700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 0x7ff52019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.735+0000 7ff52556e700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 0x7ff52019ed70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff5100059d0 tx=0x7ff5100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.735+0000 7ff516ffd700 1 -- 172.21.15.52:0/3553657846 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510027070 con 0x7ff52006ab30 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.735+0000 7ff516ffd700 1 -- 172.21.15.52:0/3553657846 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff510022470 con 0x7ff52006ab30 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.735+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5201993e0 con 0x7ff52006ab30 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.735+0000 7ff516ffd700 1 -- 172.21.15.52:0/3553657846 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510017630 con 0x7ff52006ab30 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.735+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5201a5700 con 0x7ff52006ab30 2024-06-02T23:18:20.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.737+0000 7ff516ffd700 1 -- 172.21.15.52:0/3553657846 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff51002a070 con 0x7ff52006ab30 2024-06-02T23:18:20.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.738+0000 7ff516ffd700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff50c0a2ca0 0x7ff50c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.738+0000 7ff525d6f700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff50c0a2ca0 0x7ff50c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.738+0000 7ff516ffd700 1 -- 172.21.15.52:0/3553657846 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff5100cfc20 con 0x7ff52006ab30 2024-06-02T23:18:20.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.739+0000 7ff525d6f700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff50c0a2ca0 0x7ff50c0a5160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff520199b30 tx=0x7ff51c00b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:20.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.739+0000 7ff532856b80 1 Processor -- start 2024-06-02T23:18:20.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.739+0000 7ff532856b80 1 -- start start 2024-06-02T23:18:20.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.739+0000 7ff532856b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36f9850 0x55abe36f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff532856b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 0x55abe36fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff532856b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fe550 0x55abe36fe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff532856b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abe35e7f60 con 0x55abe36fa170 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff532856b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abe3646850 con 0x55abe36fe550 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff532856b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abe35ae460 con 0x55abe36f9850 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 0x55abe36fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 0x55abe36fe010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52500/0 (socket says 172.21.15.52:52500) 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 -- 172.21.15.52:0/2088696458 learned_addr learned my addr 172.21.15.52:0/2088696458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff526570700 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fe550 0x55abe36fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff525d6f700 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36f9850 0x55abe36f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 -- 172.21.15.52:0/2088696458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36f9850 msgr2=0x55abe36f9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36f9850 0x55abe36f9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 -- 172.21.15.52:0/2088696458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fe550 msgr2=0x55abe36fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fe550 0x55abe36fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 -- 172.21.15.52:0/2088696458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe35958d0 con 0x55abe36fa170 2024-06-02T23:18:20.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff526570700 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fe550 0x55abe36fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:20.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.740+0000 7ff52556e700 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 0x55abe36fe010 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff51009c4d0 tx=0x7ff51009c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ebca90ccd17665 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.741+0000 7ff50bfff700 1 -- 172.21.15.52:0/2088696458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff510022620 con 0x55abe36fa170 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.741+0000 7ff50bfff700 1 -- 172.21.15.52:0/2088696458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff51009cc40 con 0x55abe36fa170 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.741+0000 7ff50bfff700 1 -- 172.21.15.52:0/2088696458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5100a4c90 con 0x55abe36fa170 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.741+0000 7ff532856b80 1 -- 172.21.15.52:0/2088696458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 msgr2=0x55abe36fe010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.741+0000 7ff532856b80 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 0x55abe36fe010 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff51009c4d0 tx=0x7ff51009c390 comp rx=0 tx=0).stop 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 -- 172.21.15.52:0/2088696458 shutdown_connections 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fe550 0x55abe36fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36f9850 0x55abe36f9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 --2- 172.21.15.52:0/2088696458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fa170 0x55abe36fe010 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 -- 172.21.15.52:0/2088696458 >> 172.21.15.52:0/2088696458 conn(0x55abe3622080 msgr2=0x55abe36f0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:20.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 -- 172.21.15.52:0/2088696458 shutdown_connections 2024-06-02T23:18:20.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 -- 172.21.15.52:0/2088696458 wait complete. 2024-06-02T23:18:20.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 -- start start 2024-06-02T23:18:20.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.742+0000 7ff532856b80 1 Processor -- start 2024-06-02T23:18:20.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 0x55abe3705b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fa170 0x55abe3702e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 0x55abe3705b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fe550 0x55abe37033a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55abe35e6fc0 con 0x55abe36f9850 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55abe3642810 con 0x55abe36fa170 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abe3706430 con 0x55abe36fe550 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 0x55abe3705b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52506/0 (socket says 172.21.15.52:52506) 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 -- 172.21.15.52:0/1820848650 learned_addr learned my addr 172.21.15.52:0/1820848650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff532856b80 10 client.0 _ll_register_callbacks cb 0x55abe35dc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff52556e700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fa170 0x55abe3702e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff526570700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fe550 0x55abe37033a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 -- 172.21.15.52:0/1820848650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fe550 msgr2=0x55abe37033a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fe550 0x55abe37033a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 -- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fa170 msgr2=0x55abe3702e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fa170 0x55abe3702e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff525d6f700 1 -- 172.21.15.52:0/1820848650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe360c750 con 0x55abe36f9850 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.743+0000 7ff52556e700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fa170 0x55abe3702e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff525d6f700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 0x55abe3705b60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff51c011a20 tx=0x7ff51c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95deb0d6c6df886c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff5097fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff5097fa700 1 -- 172.21.15.52:0/1820848650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c028630 con 0x55abe36f9850 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff5097fa700 1 -- 172.21.15.52:0/1820848650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff51c01f430 con 0x55abe36f9850 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51009cc40 con 0x55abe36f9850 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff5097fa700 1 -- 172.21.15.52:0/1820848650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff51c0287d0 con 0x55abe36f9850 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff532856b80 10 client.17376 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55abe3710c20 con 0x55abe36f9850 2024-06-02T23:18:20.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.744+0000 7ff5097fa700 1 -- 172.21.15.52:0/1820848650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff51009cc40 con 0x55abe36f9850 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff5097fa700 1 -- 172.21.15.52:0/1820848650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff51c01fce0 con 0x55abe36f9850 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff532856b80 10 client.17376 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff532856b80 10 client.17376 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff532856b80 20 client.17376 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff532856b80 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3713260 0x55abe37156b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff532856b80 4 client.17376 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abe3653d50 con 0x55abe3713260 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.745+0000 7ff52556e700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3713260 0x55abe37156b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.746+0000 7ff52556e700 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3713260 0x55abe37156b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3477147b99e7d750 server_cookie=3f9c3411d1cd07c0 in_seq=0 out_seq=0 2024-06-02T23:18:20.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.746+0000 7ff5097fa700 10 client.17376 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:18:20.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.746+0000 7ff5097fa700 1 -- 172.21.15.52:0/1820848650 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55abe3653d50 con 0x55abe3713260 2024-06-02T23:18:20.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.746+0000 7ff5097fa700 10 client.17376 handle_command_reply: tid=0 2024-06-02T23:18:20.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 client.17376 shutdown 2024-06-02T23:18:20.748 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 msgr2=0x55abe3705b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 0x55abe3705b60 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff51c011a20 tx=0x7ff51c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 shutdown_connections 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3713260 0x55abe37156b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fe550 0x55abe37033a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36f9850 0x55abe3705b60 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff5097fa700 0 client.17376 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 --2- 172.21.15.52:0/1820848650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36fa170 0x55abe3702e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 >> 172.21.15.52:0/1820848650 conn(0x55abe3622080 msgr2=0x55abe36ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 shutdown_connections 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 1 -- 172.21.15.52:0/1820848650 wait complete. 2024-06-02T23:18:20.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.747+0000 7ff532856b80 20 client.17376 trim_cache size 0 max 16384 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 Processor -- start 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 -- start start 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe370f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe3704da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe371af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201aa750 con 0x55abe36fe550 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201ac290 con 0x55abe36f9850 2024-06-02T23:18:20.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff532856b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201aae20 con 0x55abe36fa170 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff526570700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe371af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff526570700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe371af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52520/0 (socket says 172.21.15.52:52520) 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff526570700 1 -- 172.21.15.52:0/2734104850 learned_addr learned my addr 172.21.15.52:0/2734104850 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff525d6f700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe370f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.768+0000 7ff52556e700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe3704da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff525d6f700 1 -- 172.21.15.52:0/2734104850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 msgr2=0x55abe3704da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff525d6f700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe3704da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff525d6f700 1 -- 172.21.15.52:0/2734104850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 msgr2=0x55abe371af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff525d6f700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe371af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff525d6f700 1 -- 172.21.15.52:0/2734104850 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abe36fedd0 con 0x55abe36f9850 2024-06-02T23:18:20.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff52556e700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe3704da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff526570700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe371af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff525d6f700 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe370f510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff51c033570 tx=0x7ff51c010ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=475d72377276ba58 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff50b7fe700 1 -- 172.21.15.52:0/2734104850 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c028630 con 0x55abe36f9850 2024-06-02T23:18:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff50b7fe700 1 -- 172.21.15.52:0/2734104850 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff51c028e20 con 0x55abe36f9850 2024-06-02T23:18:20.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.769+0000 7ff50b7fe700 1 -- 172.21.15.52:0/2734104850 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c01f690 con 0x55abe36f9850 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 -- 172.21.15.52:0/2734104850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 msgr2=0x55abe370f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe370f510 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff51c033570 tx=0x7ff51c010ed0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 -- 172.21.15.52:0/2734104850 shutdown_connections 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe371af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe370f510 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 --2- 172.21.15.52:0/2734104850 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe3704da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 -- 172.21.15.52:0/2734104850 >> 172.21.15.52:0/2734104850 conn(0x55abe3622080 msgr2=0x55abe3704690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 -- 172.21.15.52:0/2734104850 shutdown_connections 2024-06-02T23:18:20.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 -- 172.21.15.52:0/2734104850 wait complete. 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.770+0000 7ff532856b80 1 -- start start 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 Processor -- start 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe36f85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe36f8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe36f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201abbc0 con 0x55abe36fe550 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201ab4f0 con 0x55abe36f9850 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff5201a8710 con 0x55abe36fa170 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff532856b80 10 client.0 _ll_register_callbacks cb 0x55abe35dc5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe36f85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe36f85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46680/0 (socket says 172.21.15.52:46680) 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 -- 172.21.15.52:0/1643196853 learned_addr learned my addr 172.21.15.52:0/1643196853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff526570700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe36f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff52556e700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe36f8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 -- 172.21.15.52:0/1643196853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 msgr2=0x55abe36f8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe36f8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 -- 172.21.15.52:0/1643196853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 msgr2=0x55abe36f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe36f9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.771+0000 7ff525d6f700 1 -- 172.21.15.52:0/1643196853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abe3709f00 con 0x55abe36f9850 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff525d6f700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe36f85e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c010 tx=0x7ff51c028660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=869d5607ec834bf0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff5097fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff5097fa700 1 -- 172.21.15.52:0/1643196853 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff51c033990 con 0x55abe36f9850 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff5097fa700 1 -- 172.21.15.52:0/1643196853 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff51c005040 con 0x55abe36f9850 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff51c028e20 con 0x55abe36f9850 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff5097fa700 1 -- 172.21.15.52:0/1643196853 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff51c028c40 con 0x55abe36f9850 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff532856b80 10 client.26876 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:20.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55abe3782bc0 con 0x55abe36f9850 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff5097fa700 1 -- 172.21.15.52:0/1643196853 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff51c028e20 con 0x55abe36f9850 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.772+0000 7ff5097fa700 1 -- 172.21.15.52:0/1643196853 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff51c044070 con 0x55abe36f9850 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff532856b80 10 client.26876 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff532856b80 10 client.26876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff532856b80 20 client.26876 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff532856b80 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3783130 0x55abe3785550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff532856b80 4 client.26876 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abe364e2d0 con 0x55abe3783130 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff52556e700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3783130 0x55abe3785550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff5097fa700 10 client.26876 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.773+0000 7ff52556e700 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3783130 0x55abe3785550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77581bfb15cb4a37 server_cookie=280a5660dd3105c3 in_seq=0 out_seq=0 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff5097fa700 1 -- 172.21.15.52:0/1643196853 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abe364e2d0 con 0x55abe3783130 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff5097fa700 10 client.26876 handle_command_reply: tid=0 2024-06-02T23:18:20.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 client.26876 shutdown 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 msgr2=0x55abe36f85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe36f85e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff51c00c010 tx=0x7ff51c028660 comp rx=0 tx=0).stop 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 shutdown_connections 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55abe3783130 0x55abe3785550 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55abe36fe550 0x55abe36f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55abe36f9850 0x55abe36f85e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff5097fa700 0 client.26876 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 --2- 172.21.15.52:0/1643196853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abe36fa170 0x55abe36f8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 >> 172.21.15.52:0/1643196853 conn(0x55abe3622080 msgr2=0x55abe3702680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 shutdown_connections 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 1 -- 172.21.15.52:0/1643196853 wait complete. 2024-06-02T23:18:20.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.774+0000 7ff532856b80 20 client.26876 trim_cache size 0 max 16384 2024-06-02T23:18:20.779 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:18:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff50c0a2ca0 msgr2=0x7ff50c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff50c0a2ca0 0x7ff50c0a5160 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff520199b30 tx=0x7ff51c00b480 comp rx=0 tx=0).stop 2024-06-02T23:18:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 msgr2=0x7ff52019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:20.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 0x7ff52019ed70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7ff5100059d0 tx=0x7ff5100047b0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 shutdown_connections 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff50c0a2ca0 0x7ff50c0a5160 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff520108500 0x7ff5201989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff52006a170 0x7ff52019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 --2- 172.21.15.52:0/3553657846 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff52006ab30 0x7ff52019ed70 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 >> 172.21.15.52:0/3553657846 conn(0x7ff520078340 msgr2=0x7ff520100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 shutdown_connections 2024-06-02T23:18:20.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:20.775+0000 7ff527fd3700 1 -- 172.21.15.52:0/3553657846 wait complete. 2024-06-02T23:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 -- 172.21.15.52:0/4285167502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f0101af0 msgr2=0x7f95f0101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 --2- 172.21.15.52:0/4285167502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f0101af0 0x7f95f0101f70 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f95e0009ca0 tx=0x7f95e002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 -- 172.21.15.52:0/4285167502 shutdown_connections 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 --2- 172.21.15.52:0/4285167502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f01024b0 0x7f95f0108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 --2- 172.21.15.52:0/4285167502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f0101af0 0x7f95f0101f70 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 --2- 172.21.15.52:0/4285167502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f0100900 0x7f95f0100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 -- 172.21.15.52:0/4285167502 >> 172.21.15.52:0/4285167502 conn(0x7f95f00fbe70 msgr2=0x7f95f00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 -- 172.21.15.52:0/4285167502 shutdown_connections 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.357+0000 7f95f7134640 1 -- 172.21.15.52:0/4285167502 wait complete. 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 Processor -- start 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 -- start start 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 0x7f95f0069c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f0101af0 0x7f95f019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f01024b0 0x7f95f019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010c430 con 0x7f95f01024b0 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010c2b0 con 0x7f95f0101af0 2024-06-02T23:18:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f7134640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95f010c5b0 con 0x7f95f0100900 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 0x7f95f0069c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f56d1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f01024b0 0x7f95f019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 0x7f95f0069c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43860/0 (socket says 172.21.15.52:43860) 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f4ed0640 1 -- 172.21.15.52:0/3595565940 learned_addr learned my addr 172.21.15.52:0/3595565940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95effff640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f0101af0 0x7f95f019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f56d1640 1 -- 172.21.15.52:0/3595565940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 msgr2=0x7f95f0069c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f56d1640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 0x7f95f0069c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f56d1640 1 -- 172.21.15.52:0/3595565940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f0101af0 msgr2=0x7f95f019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f56d1640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f0101af0 0x7f95f019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f56d1640 1 -- 172.21.15.52:0/3595565940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f01966e0 con 0x7f95f01024b0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.358+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 0x7f95f0069c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.359+0000 7f95f56d1640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f01024b0 0x7f95f019ef00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f95e800e970 tx=0x7f95e800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.359+0000 7f95edffb640 1 -- 172.21.15.52:0/3595565940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800ccc0 con 0x7f95f01024b0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.359+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f01969d0 con 0x7f95f01024b0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.359+0000 7f95edffb640 1 -- 172.21.15.52:0/3595565940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95e8005020 con 0x7f95f01024b0 2024-06-02T23:18:21.361 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.359+0000 7f95edffb640 1 -- 172.21.15.52:0/3595565940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e800ce60 con 0x7f95f01024b0 2024-06-02T23:18:21.362 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.359+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f0196f20 con 0x7f95f01024b0 2024-06-02T23:18:21.366 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 Processor -- start 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95edffb640 1 -- 172.21.15.52:0/3595565940 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f95e80051d0 con 0x7f95f01024b0 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 -- start start 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257716430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f25771b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f25771e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f2576794f0 con 0x55f257716a00 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f257617870 con 0x55f257716010 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.362+0000 7f95f9e41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2576754b0 con 0x55f25771bb70 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f25771b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257716430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f25771b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52540/0 (socket says 172.21.15.52:52540) 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 -- 172.21.15.52:0/67859771 learned_addr learned my addr 172.21.15.52:0/67859771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:21.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95effff640 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f25771e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 -- 172.21.15.52:0/67859771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 msgr2=0x55f25771e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f25771e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 -- 172.21.15.52:0/67859771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 msgr2=0x55f257716430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257716430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 -- 172.21.15.52:0/67859771 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2576339b0 con 0x55f257716a00 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95effff640 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257716430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95edffb640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f95dc0a2c30 0x7f95dc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95effff640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f95dc0a2c30 0x7f95dc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f25771b500 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f95d8010770 tx=0x7f95d8010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbf6589ee491c8f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95edffb640 1 -- 172.21.15.52:0/3595565940 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f95e80ceea0 con 0x7f95f01024b0 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95e6ffd640 1 -- 172.21.15.52:0/67859771 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800e930 con 0x55f257716a00 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95e6ffd640 1 -- 172.21.15.52:0/67859771 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d8011440 con 0x55f257716a00 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.363+0000 7f95e6ffd640 1 -- 172.21.15.52:0/67859771 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8019630 con 0x55f257716a00 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 -- 172.21.15.52:0/67859771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 msgr2=0x55f25771b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95effff640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f95dc0a2c30 0x7f95dc0a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f95f0069990 tx=0x7f95e0005c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f25771b500 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f95d8010770 tx=0x7f95d8010c30 comp rx=0 tx=0).stop 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 -- 172.21.15.52:0/67859771 shutdown_connections 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f25771e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f25771b500 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 --2- 172.21.15.52:0/67859771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257716430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 -- 172.21.15.52:0/67859771 >> 172.21.15.52:0/67859771 conn(0x55f257711680 msgr2=0x55f257711a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 -- 172.21.15.52:0/67859771 shutdown_connections 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.364+0000 7f95f9e41740 1 -- 172.21.15.52:0/67859771 wait complete. 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 -- start start 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 Processor -- start 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257720220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f257720760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f2577277a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f25771eb20 con 0x55f257716a00 2024-06-02T23:18:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f257602ae0 con 0x55f257716010 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f25771eca0 con 0x55f25771bb70 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f9e41740 10 client.0 _ll_register_callbacks cb 0x55f25763b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257720220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f257720760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257720220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46712/0 (socket says 172.21.15.52:46712) 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f4ed0640 1 -- 172.21.15.52:0/3333027053 learned_addr learned my addr 172.21.15.52:0/3333027053 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.365+0000 7f95f56d1640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f2577277a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f4ed0640 1 -- 172.21.15.52:0/3333027053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 msgr2=0x55f2577277a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f2577277a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f4ed0640 1 -- 172.21.15.52:0/3333027053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 msgr2=0x55f257720760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f257720760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f4ed0640 1 -- 172.21.15.52:0/3333027053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f25738d210 con 0x55f257716010 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f56d1640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f2577277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95effff640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f257720760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257720220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000c00 tx=0x7f95d8010430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=72f285abd48365df server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95cffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95cffff640 1 -- 172.21.15.52:0/3333027053 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8027030 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95cffff640 1 -- 172.21.15.52:0/3333027053 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f95d801f070 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d8011440 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95cffff640 1 -- 172.21.15.52:0/3333027053 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f95d8010d60 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f9e41740 10 client.26888 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f257459620 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.366+0000 7f95cffff640 1 -- 172.21.15.52:0/3333027053 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95d8011440 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95cffff640 1 -- 172.21.15.52:0/3333027053 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f95d800ea50 con 0x55f257716010 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95f9e41740 10 client.26888 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95f9e41740 10 client.26888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95f9e41740 20 client.26888 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95f9e41740 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f25772e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95f9e41740 4 client.26888 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f25768cf10 con 0x55f25772bd40 2024-06-02T23:18:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95effff640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f25772e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95cffff640 10 client.26888 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.367+0000 7f95effff640 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f25772e130 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc90ac328a0ee3b server_cookie=59314231fd1e1a8a in_seq=0 out_seq=0 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.368+0000 7f95cffff640 1 -- 172.21.15.52:0/3333027053 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f25768cf10 con 0x55f25772bd40 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.368+0000 7f95cffff640 10 client.26888 handle_command_reply: tid=0 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.368+0000 7f95f9e41740 1 client.26888 shutdown 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 msgr2=0x55f257720220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257720220 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f95d8000c00 tx=0x7f95d8010430 comp rx=0 tx=0).stop 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 shutdown_connections 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f25772e130 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95cffff640 0 client.26888 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f25771bb70 0x55f2577277a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716a00 0x55f257720760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 --2- 172.21.15.52:0/3333027053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716010 0x55f257720220 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 >> 172.21.15.52:0/3333027053 conn(0x55f257711680 msgr2=0x55f2577107e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 shutdown_connections 2024-06-02T23:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 1 -- 172.21.15.52:0/3333027053 wait complete. 2024-06-02T23:18:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.369+0000 7f95f9e41740 20 client.26888 trim_cache size 0 max 16384 2024-06-02T23:18:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 Processor -- start 2024-06-02T23:18:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 -- start start 2024-06-02T23:18:21.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716010 0x55f25771f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 0x55f257727760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f25771bb70 0x55f25771fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01aafe0 con 0x55f25771bb70 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01acb20 con 0x55f257716a00 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f9e41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ab6b0 con 0x55f257716010 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 0x55f257727760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95effff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 0x55f257727760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46716/0 (socket says 172.21.15.52:46716) 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95effff640 1 -- 172.21.15.52:0/1551681334 learned_addr learned my addr 172.21.15.52:0/1551681334 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f56d1640 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f25771bb70 0x55f25771fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.385+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716010 0x55f25771f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95effff640 1 -- 172.21.15.52:0/1551681334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716010 msgr2=0x55f25771f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95effff640 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716010 0x55f25771f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95effff640 1 -- 172.21.15.52:0/1551681334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f25771bb70 msgr2=0x55f25771fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95effff640 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f25771bb70 0x55f25771fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95effff640 1 -- 172.21.15.52:0/1551681334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2577114b0 con 0x55f257716a00 2024-06-02T23:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95effff640 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 0x55f257727760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95e0032010 tx=0x7f95e003d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e003905c93ff4c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95e67fc640 1 -- 172.21.15.52:0/1551681334 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0003e00 con 0x55f257716a00 2024-06-02T23:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95e67fc640 1 -- 172.21.15.52:0/1551681334 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95e005c030 con 0x55f257716a00 2024-06-02T23:18:21.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.386+0000 7f95e67fc640 1 -- 172.21.15.52:0/1551681334 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e0030b30 con 0x55f257716a00 2024-06-02T23:18:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 -- 172.21.15.52:0/1551681334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 msgr2=0x55f257727760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 0x55f257727760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f95e0032010 tx=0x7f95e003d000 comp rx=0 tx=0).stop 2024-06-02T23:18:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 -- 172.21.15.52:0/1551681334 shutdown_connections 2024-06-02T23:18:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f25771bb70 0x55f25771fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f257716a00 0x55f257727760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1551681334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716010 0x55f25771f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 -- 172.21.15.52:0/1551681334 >> 172.21.15.52:0/1551681334 conn(0x55f257711680 msgr2=0x55f257714360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 -- 172.21.15.52:0/1551681334 shutdown_connections 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 -- 172.21.15.52:0/1551681334 wait complete. 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 -- start start 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.387+0000 7f95f9e41740 1 Processor -- start 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 0x55f257724680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716a00 0x55f257724bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f25771bb70 0x55f2577949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ac450 con 0x55f257716010 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01abd80 con 0x55f25771bb70 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01a8fc0 con 0x55f257716a00 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f9e41740 10 client.0 _ll_register_callbacks cb 0x7f95e0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:21.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 0x55f257724680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 0x55f257724680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52556/0 (socket says 172.21.15.52:52556) 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 -- 172.21.15.52:0/1691051104 learned_addr learned my addr 172.21.15.52:0/1691051104 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f56d1640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f25771bb70 0x55f2577949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95effff640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716a00 0x55f257724bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 -- 172.21.15.52:0/1691051104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716a00 msgr2=0x55f257724bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716a00 0x55f257724bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 -- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f25771bb70 msgr2=0x55f2577949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f25771bb70 0x55f2577949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f4ed0640 1 -- 172.21.15.52:0/1691051104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f257794ef0 con 0x55f257716010 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.388+0000 7f95f56d1640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f25771bb70 0x55f2577949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f4ed0640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 0x55f257724680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f95d800ee10 tx=0x7f95d800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dbcdad4dc93f961 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95cffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:21.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95cffff640 1 -- 172.21.15.52:0/1691051104 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8027040 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95cffff640 1 -- 172.21.15.52:0/1691051104 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f95d800e7d0 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e005c030 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95cffff640 1 -- 172.21.15.52:0/1691051104 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f95d800e9d0 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 10 client.17400 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f2577961b0 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95cffff640 1 -- 172.21.15.52:0/1691051104 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f95e005c030 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95cffff640 1 -- 172.21.15.52:0/1691051104 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f95d801a430 con 0x55f257716010 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 10 client.17400 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 10 client.17400 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 20 client.17400 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f257798c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 4 client.17400 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:18:21.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.389+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f257686c20 con 0x55f25772bd40 2024-06-02T23:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.390+0000 7f95effff640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f257798c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.390+0000 7f95effff640 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f257798c20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab82736254c37f17 server_cookie=9ccfc993a839aee in_seq=0 out_seq=0 2024-06-02T23:18:21.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.390+0000 7f95cffff640 10 client.17400 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95cffff640 1 -- 172.21.15.52:0/1691051104 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f257686c20 con 0x55f25772bd40 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95cffff640 10 client.17400 handle_command_reply: tid=0 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 client.17400 shutdown 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 msgr2=0x55f257724680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 0x55f257724680 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f95d800ee10 tx=0x7f95d800bdd0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 shutdown_connections 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f25772bd40 0x55f257798c20 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f257716a00 0x55f257724bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f25771bb70 0x55f2577949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 --2- 172.21.15.52:0/1691051104 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f257716010 0x55f257724680 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95cffff640 0 client.17400 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 >> 172.21.15.52:0/1691051104 conn(0x55f257711680 msgr2=0x55f25770ff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 shutdown_connections 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 1 -- 172.21.15.52:0/1691051104 wait complete. 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.407+0000 7f95f9e41740 20 client.17400 trim_cache size 0 max 16384 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:18:21.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:18:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f95dc0a2c30 msgr2=0x7f95dc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f95dc0a2c30 0x7f95dc0a50f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f95f0069990 tx=0x7f95e0005c30 comp rx=0 tx=0).stop 2024-06-02T23:18:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f01024b0 msgr2=0x7f95f019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f01024b0 0x7f95f019ef00 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f95e800e970 tx=0x7f95e800ee30 comp rx=0 tx=0).stop 2024-06-02T23:18:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 shutdown_connections 2024-06-02T23:18:21.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f95dc0a2c30 0x7f95dc0a50f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f95f01024b0 0x7f95f019ef00 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95f0101af0 0x7f95f019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 --2- 172.21.15.52:0/3595565940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95f0100900 0x7f95f0069c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 >> 172.21.15.52:0/3595565940 conn(0x7f95f00fbe70 msgr2=0x7f95f0104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 shutdown_connections 2024-06-02T23:18:21.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:21.408+0000 7f95f7134640 1 -- 172.21.15.52:0/3595565940 wait complete. 2024-06-02T23:18:21.524 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:18:21.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 113668 1.69 MB/sec execute 210 sec latency 15.938 ms 2024-06-02T23:18:21.801 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:21 smithi052 ceph-mon[32340]: pgmap v625: 97 pgs: 97 active+clean; 61 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 794 KiB/s wr, 82 op/s 2024-06-02T23:18:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:21 smithi123 ceph-mon[35977]: pgmap v625: 97 pgs: 97 active+clean; 61 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 794 KiB/s wr, 82 op/s 2024-06-02T23:18:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:21 smithi174 ceph-mon[35994]: pgmap v625: 97 pgs: 97 active+clean; 61 MiB data, 992 MiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 794 KiB/s wr, 82 op/s 2024-06-02T23:18:22.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 113943 1.69 MB/sec execute 211 sec latency 13.198 ms 2024-06-02T23:18:23.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 114278 1.68 MB/sec execute 212 sec latency 21.704 ms 2024-06-02T23:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 -- 172.21.15.52:0/249389408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1050101380 msgr2=0x7f1050101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 --2- 172.21.15.52:0/249389408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1050101380 0x7f1050101800 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1044009990 tx=0x7f1044009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 -- 172.21.15.52:0/249389408 shutdown_connections 2024-06-02T23:18:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 --2- 172.21.15.52:0/249389408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101d40 0x7f105010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 --2- 172.21.15.52:0/249389408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f10501073f0 0x7f10501077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 --2- 172.21.15.52:0/249389408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1050101380 0x7f1050101800 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 -- 172.21.15.52:0/249389408 >> 172.21.15.52:0/249389408 conn(0x7f1050078300 msgr2=0x7f1050078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 -- 172.21.15.52:0/249389408 shutdown_connections 2024-06-02T23:18:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.637+0000 7f1054e92700 1 -- 172.21.15.52:0/249389408 wait complete. 2024-06-02T23:18:23.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 Processor -- start 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 -- start start 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101380 0x7f1050103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 0x7f10501035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 0x7f1050103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1050111540 con 0x7f10501073f0 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10501113c0 con 0x7f1050101d40 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f1054e92700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10501116c0 con 0x7f1050101380 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 0x7f1050103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 0x7f10501035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101380 0x7f1050103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 0x7f1050103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52576/0 (socket says 172.21.15.52:52576) 2024-06-02T23:18:23.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 0x7f10501035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46738/0 (socket says 172.21.15.52:46738) 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101380 0x7f1050103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43926/0 (socket says 172.21.15.52:43926) 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.638+0000 7f104ed9d700 1 -- 172.21.15.52:0/4025419150 learned_addr learned my addr 172.21.15.52:0/4025419150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104dd9b700 1 -- 172.21.15.52:0/4025419150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101380 msgr2=0x7f1050103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104dd9b700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101380 0x7f1050103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104dd9b700 1 -- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 msgr2=0x7f1050103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104dd9b700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 0x7f1050103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104dd9b700 1 -- 172.21.15.52:0/4025419150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10501a7070 con 0x7f1050101d40 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104ed9d700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 0x7f1050103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f104dd9b700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 0x7f10501035a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10440059d0 tx=0x7f1044004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f103f7fe700 1 -- 172.21.15.52:0/4025419150 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044027070 con 0x7f1050101d40 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10501a7300 con 0x7f1050101d40 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f103f7fe700 1 -- 172.21.15.52:0/4025419150 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1044022470 con 0x7f1050101d40 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10501a77b0 con 0x7f1050101d40 2024-06-02T23:18:23.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.639+0000 7f103f7fe700 1 -- 172.21.15.52:0/4025419150 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104400b610 con 0x7f1050101d40 2024-06-02T23:18:23.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.642+0000 7f103f7fe700 1 -- 172.21.15.52:0/4025419150 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f104402a070 con 0x7f1050101d40 2024-06-02T23:18:23.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.643+0000 7f103f7fe700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10380a2c20 0x7f10380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.643+0000 7f103f7fe700 1 -- 172.21.15.52:0/4025419150 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f10440cfd50 con 0x7f1050101d40 2024-06-02T23:18:23.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.643+0000 7f104e59c700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10380a2c20 0x7f10380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.643+0000 7f105f715b80 1 Processor -- start 2024-06-02T23:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 -- start start 2024-06-02T23:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104e59c700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10380a2c20 0x7f10380a50e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1040009be0 tx=0x7f1040009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x559834442340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834442880 0x55983443c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x55983443cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55983432ef60 con 0x559834441f60 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55983438d850 con 0x55983443c7a0 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f105f715b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5598342f5460 con 0x559834442880 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x559834442340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104dd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834442880 0x55983443c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x559834442340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52580/0 (socket says 172.21.15.52:52580) 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104e59c700 1 -- 172.21.15.52:0/1004063529 learned_addr learned my addr 172.21.15.52:0/1004063529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104ed9d700 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x55983443cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104dd9b700 1 -- 172.21.15.52:0/1004063529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 msgr2=0x55983443cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104dd9b700 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x55983443cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104dd9b700 1 -- 172.21.15.52:0/1004063529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 msgr2=0x559834442340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104dd9b700 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x559834442340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104dd9b700 1 -- 172.21.15.52:0/1004063529 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598342dc8d0 con 0x559834442880 2024-06-02T23:18:23.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.644+0000 7f104e59c700 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x559834442340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.645+0000 7f104dd9b700 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834442880 0x55983443c260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10440cd8d0 tx=0x7f104409c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a42789002e8b4f92 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.645+0000 7f103cff9700 1 -- 172.21.15.52:0/1004063529 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440095d0 con 0x559834442880 2024-06-02T23:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.645+0000 7f103cff9700 1 -- 172.21.15.52:0/1004063529 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10440a5250 con 0x559834442880 2024-06-02T23:18:23.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.645+0000 7f103cff9700 1 -- 172.21.15.52:0/1004063529 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440a4dc0 con 0x559834442880 2024-06-02T23:18:23.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 -- 172.21.15.52:0/1004063529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834442880 msgr2=0x55983443c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834442880 0x55983443c260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10440cd8d0 tx=0x7f104409c660 comp rx=0 tx=0).stop 2024-06-02T23:18:23.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 -- 172.21.15.52:0/1004063529 shutdown_connections 2024-06-02T23:18:23.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x55983443cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x559834442340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 --2- 172.21.15.52:0/1004063529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834442880 0x55983443c260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 -- 172.21.15.52:0/1004063529 >> 172.21.15.52:0/1004063529 conn(0x559834369080 msgr2=0x559834437220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 -- 172.21.15.52:0/1004063529 shutdown_connections 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 -- 172.21.15.52:0/1004063529 wait complete. 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 -- start start 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.646+0000 7f105f715b80 1 Processor -- start 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x5598344485e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 0x5598344458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834445e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55983432dfc0 con 0x559834442880 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559834389810 con 0x559834441f60 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559834449010 con 0x55983443c7a0 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f105f715b80 10 client.0 _ll_register_callbacks cb 0x5598343235b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 0x5598344458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834445e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x5598344485e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 0x5598344458e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46746/0 (socket says 172.21.15.52:46746) 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104dd9b700 1 -- 172.21.15.52:0/3625727562 learned_addr learned my addr 172.21.15.52:0/3625727562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104ed9d700 1 -- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 msgr2=0x5598344485e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104ed9d700 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x5598344485e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104ed9d700 1 -- 172.21.15.52:0/3625727562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 msgr2=0x5598344458e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104ed9d700 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 0x5598344458e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104ed9d700 1 -- 172.21.15.52:0/3625727562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559834353750 con 0x559834442880 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.647+0000 7f104dd9b700 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 0x5598344458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f104ed9d700 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834445e20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1048007ca0 tx=0x7f1048009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a84854afa844588 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f1035ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f1035ffb700 1 -- 172.21.15.52:0/3625727562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104801a070 con 0x559834442880 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10440a5250 con 0x559834442880 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f1035ffb700 1 -- 172.21.15.52:0/3625727562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f104801e410 con 0x559834442880 2024-06-02T23:18:23.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f1035ffb700 1 -- 172.21.15.52:0/3625727562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1048024050 con 0x559834442880 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f105f715b80 10 client.17412 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.648+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5598344537c0 con 0x559834442880 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f1035ffb700 1 -- 172.21.15.52:0/3625727562 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f10440a5250 con 0x559834442880 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f1035ffb700 1 -- 172.21.15.52:0/3625727562 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1048004020 con 0x559834442880 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f105f715b80 10 client.17412 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f105f715b80 10 client.17412 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f105f715b80 20 client.17412 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f105f715b80 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x559834455e00 0x5598344581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f105f715b80 4 client.17412 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55983439ad50 con 0x559834455e00 2024-06-02T23:18:23.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.649+0000 7f104e59c700 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x559834455e00 0x5598344581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.650+0000 7f1035ffb700 10 client.17412 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.650+0000 7f104e59c700 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x559834455e00 0x5598344581f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=631aee286da8fd76 server_cookie=82960d6d88413ba5 in_seq=0 out_seq=0 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.650+0000 7f1035ffb700 1 -- 172.21.15.52:0/3625727562 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55983439ad50 con 0x559834455e00 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.650+0000 7f1035ffb700 10 client.17412 handle_command_reply: tid=0 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.650+0000 7f105f715b80 1 client.17412 shutdown 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 msgr2=0x559834445e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834445e20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1048007ca0 tx=0x7f1048009510 comp rx=0 tx=0).stop 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 shutdown_connections 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x559834455e00 0x5598344581f0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834445e20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x5598344485e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f1035ffb700 0 client.17412 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:18:23.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 --2- 172.21.15.52:0/3625727562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834441f60 0x5598344458e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 >> 172.21.15.52:0/3625727562 conn(0x559834369080 msgr2=0x559834436700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:23.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 shutdown_connections 2024-06-02T23:18:23.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 1 -- 172.21.15.52:0/3625727562 wait complete. 2024-06-02T23:18:23.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.651+0000 7f105f715b80 20 client.17412 trim_cache size 0 max 16384 2024-06-02T23:18:23.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.668+0000 7f105f715b80 1 Processor -- start 2024-06-02T23:18:23.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.668+0000 7f105f715b80 1 -- start start 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f105f715b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x559834445e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f105f715b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834441f60 0x5598344412d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f105f715b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834448120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f105f715b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10501aeb40 con 0x559834442880 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f105f715b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10501b0680 con 0x55983443c7a0 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f105f715b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10501af210 con 0x559834441f60 2024-06-02T23:18:23.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834448120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x559834445e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x559834445e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46762/0 (socket says 172.21.15.52:46762) 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104e59c700 1 -- 172.21.15.52:0/2930807875 learned_addr learned my addr 172.21.15.52:0/2930807875 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104dd9b700 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834441f60 0x5598344412d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104ed9d700 1 -- 172.21.15.52:0/2930807875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834441f60 msgr2=0x5598344412d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104ed9d700 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834441f60 0x5598344412d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104ed9d700 1 -- 172.21.15.52:0/2930807875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 msgr2=0x559834445e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104ed9d700 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x559834445e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104ed9d700 1 -- 172.21.15.52:0/2930807875 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598344389a0 con 0x559834442880 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.669+0000 7f104e59c700 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x559834445e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.670+0000 7f104ed9d700 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834448120 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f104800b010 tx=0x7f104800efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789dd6aa227b10c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:23.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.670+0000 7f1037fff700 1 -- 172.21.15.52:0/2930807875 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f104801a070 con 0x559834442880 2024-06-02T23:18:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.670+0000 7f1037fff700 1 -- 172.21.15.52:0/2930807875 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f104802d070 con 0x559834442880 2024-06-02T23:18:23.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.670+0000 7f1037fff700 1 -- 172.21.15.52:0/2930807875 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1048004e50 con 0x559834442880 2024-06-02T23:18:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 -- 172.21.15.52:0/2930807875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 msgr2=0x559834448120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834448120 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f104800b010 tx=0x7f104800efd0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 -- 172.21.15.52:0/2930807875 shutdown_connections 2024-06-02T23:18:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834442880 0x559834448120 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55983443c7a0 0x559834445e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 --2- 172.21.15.52:0/2930807875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559834441f60 0x5598344412d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 -- 172.21.15.52:0/2930807875 >> 172.21.15.52:0/2930807875 conn(0x559834369080 msgr2=0x55983443fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 -- 172.21.15.52:0/2930807875 shutdown_connections 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 -- 172.21.15.52:0/2930807875 wait complete. 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 -- start start 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 Processor -- start 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.671+0000 7f105f715b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x55983444e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f105f715b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x55983444e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f105f715b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 0x55983445cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f105f715b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f10501affb0 con 0x559834441f60 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f105f715b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f10501af8e0 con 0x559834442880 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f105f715b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10501acb00 con 0x55983443c7a0 2024-06-02T23:18:23.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f105f715b80 10 client.0 _ll_register_callbacks cb 0x5598343235b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x55983444e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 0x55983445cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 0x55983445cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46768/0 (socket says 172.21.15.52:46768) 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104ed9d700 1 -- 172.21.15.52:0/889701848 learned_addr learned my addr 172.21.15.52:0/889701848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104e59c700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x55983444e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 -- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 msgr2=0x55983444e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x55983444e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 -- 172.21.15.52:0/889701848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 msgr2=0x55983445cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 0x55983445cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 -- 172.21.15.52:0/889701848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55983445d130 con 0x559834441f60 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104ed9d700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 0x55983445cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104e59c700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x55983444e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:23.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f1035ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f104dd9b700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x55983444e770 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f10440a4d20 tx=0x7f1044009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207a39286e94a22c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f1035ffb700 1 -- 172.21.15.52:0/889701848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10440a4fc0 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.672+0000 7f1035ffb700 1 -- 172.21.15.52:0/889701848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f10440a1bb0 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f104802d070 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f1035ffb700 1 -- 172.21.15.52:0/889701848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f10440095d0 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 10 client.17418 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5598344c0270 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f1035ffb700 1 -- 172.21.15.52:0/889701848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f104802d070 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f1035ffb700 1 -- 172.21.15.52:0/889701848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f104409bc10 con 0x559834441f60 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 10 client.17418 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 10 client.17418 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 20 client.17418 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5598344c0590 0x5598344c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:23.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 4 client.17418 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.673+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598343952d0 con 0x5598344c0590 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f104e59c700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5598344c0590 0x5598344c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f104e59c700 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5598344c0590 0x5598344c29b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be700456de9c96ae server_cookie=20ac3dbb95616f83 in_seq=0 out_seq=0 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f1035ffb700 10 client.17418 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f1035ffb700 1 -- 172.21.15.52:0/889701848 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598343952d0 con 0x5598344c0590 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f1035ffb700 10 client.17418 handle_command_reply: tid=0 2024-06-02T23:18:23.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 client.17418 shutdown 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 msgr2=0x55983444e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x55983444e770 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f10440a4d20 tx=0x7f1044009310 comp rx=0 tx=0).stop 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 shutdown_connections 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5598344c0590 0x5598344c29b0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f1035ffb700 0 client.17418 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559834442880 0x55983445cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55983443c7a0 0x55983444e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.674+0000 7f105f715b80 1 --2- 172.21.15.52:0/889701848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559834441f60 0x55983444e770 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 >> 172.21.15.52:0/889701848 conn(0x559834369080 msgr2=0x5598344412d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 shutdown_connections 2024-06-02T23:18:23.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f105f715b80 1 -- 172.21.15.52:0/889701848 wait complete. 2024-06-02T23:18:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f105f715b80 20 client.17418 trim_cache size 0 max 16384 2024-06-02T23:18:23.679 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:18:23.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10380a2c20 msgr2=0x7f10380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10380a2c20 0x7f10380a50e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f1040009be0 tx=0x7f1040009260 comp rx=0 tx=0).stop 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 msgr2=0x7f10501035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 0x7f10501035a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f10440059d0 tx=0x7f1044004790 comp rx=0 tx=0).stop 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 shutdown_connections 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f10380a2c20 0x7f10380a50e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f10501073f0 0x7f1050103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1050101380 0x7f1050103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 --2- 172.21.15.52:0/4025419150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1050101d40 0x7f10501035a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.675+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 >> 172.21.15.52:0/4025419150 conn(0x7f1050078300 msgr2=0x7f1050100680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.676+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 shutdown_connections 2024-06-02T23:18:23.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:23.676+0000 7f1054e92700 1 -- 172.21.15.52:0/4025419150 wait complete. 2024-06-02T23:18:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:23 smithi052 ceph-mon[32340]: pgmap v626: 97 pgs: 97 active+clean; 60 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 997 KiB/s wr, 115 op/s 2024-06-02T23:18:24.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:23 smithi123 ceph-mon[35977]: pgmap v626: 97 pgs: 97 active+clean; 60 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 997 KiB/s wr, 115 op/s 2024-06-02T23:18:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:23 smithi174 ceph-mon[35994]: pgmap v626: 97 pgs: 97 active+clean; 60 MiB data, 1019 MiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 997 KiB/s wr, 115 op/s 2024-06-02T23:18:24.379 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:18:24.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 114772 1.68 MB/sec execute 213 sec latency 15.524 ms 2024-06-02T23:18:24.663 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:25.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 115388 1.69 MB/sec execute 214 sec latency 15.283 ms 2024-06-02T23:18:25.806 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:25 smithi052 ceph-mon[32340]: pgmap v627: 97 pgs: 97 active+clean; 57 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-06-02T23:18:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:25 smithi123 ceph-mon[35977]: pgmap v627: 97 pgs: 97 active+clean; 57 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-06-02T23:18:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:25 smithi174 ceph-mon[35994]: pgmap v627: 97 pgs: 97 active+clean; 57 MiB data, 1020 MiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 1.0 MiB/s wr, 119 op/s 2024-06-02T23:18:26.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 115918 1.69 MB/sec execute 215 sec latency 99.048 ms 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.655+0000 7f9f00731700 1 -- 172.21.15.52:0/3312841707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 msgr2=0x7f9ef81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.655+0000 7f9f00731700 1 --2- 172.21.15.52:0/3312841707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 0x7f9ef81077e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4009990 tx=0x7f9ef4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 -- 172.21.15.52:0/3312841707 shutdown_connections 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 --2- 172.21.15.52:0/3312841707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 --2- 172.21.15.52:0/3312841707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 0x7f9ef81077e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 --2- 172.21.15.52:0/3312841707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 0x7f9ef8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 -- 172.21.15.52:0/3312841707 >> 172.21.15.52:0/3312841707 conn(0x7f9ef8078340 msgr2=0x7f9ef8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 -- 172.21.15.52:0/3312841707 shutdown_connections 2024-06-02T23:18:26.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 -- 172.21.15.52:0/3312841707 wait complete. 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.656+0000 7f9f00731700 1 Processor -- start 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 -- start start 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 0x7f9ef8103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef81035b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 0x7f9ef8103af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81115a0 con 0x7f9ef8107400 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8111420 con 0x7f9ef8101d50 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9f00731700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef8111720 con 0x7f9ef8101390 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 0x7f9ef8103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efecce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 0x7f9ef8103af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef81035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef81035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44052/0 (socket says 172.21.15.52:44052) 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 -- 172.21.15.52:0/2907770785 learned_addr learned my addr 172.21.15.52:0/2907770785 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 -- 172.21.15.52:0/2907770785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 msgr2=0x7f9ef8103070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 0x7f9ef8103070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 -- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 msgr2=0x7f9ef8103af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 0x7f9ef8103af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efdccc700 1 -- 172.21.15.52:0/2907770785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef81a7130 con 0x7f9ef8101d50 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.657+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 0x7f9ef8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:26.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.658+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef81035b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e9c0 tx=0x7f9ee800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.659+0000 7f9eef7fe700 1 -- 172.21.15.52:0/2907770785 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee800cbd0 con 0x7f9ef8101d50 2024-06-02T23:18:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.659+0000 7f9eef7fe700 1 -- 172.21.15.52:0/2907770785 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ee800ee60 con 0x7f9ef8101d50 2024-06-02T23:18:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.659+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef81a7420 con 0x7f9ef8101d50 2024-06-02T23:18:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.659+0000 7f9eef7fe700 1 -- 172.21.15.52:0/2907770785 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80055b0 con 0x7f9ef8101d50 2024-06-02T23:18:26.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.659+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef81a7960 con 0x7f9ef8101d50 2024-06-02T23:18:26.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.662+0000 7f9eef7fe700 1 -- 172.21.15.52:0/2907770785 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9ee8004750 con 0x7f9ef8101d50 2024-06-02T23:18:26.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.663+0000 7f9eef7fe700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9ee40a2ca0 0x7f9ee40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.663+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9ee40a2ca0 0x7f9ee40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.663+0000 7f9eef7fe700 1 -- 172.21.15.52:0/2907770785 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9ee80cf230 con 0x7f9ef8101d50 2024-06-02T23:18:26.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9ee40a2ca0 0x7f9ee40a5160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400b2e0 tx=0x7f9ef401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:26.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 Processor -- start 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 -- start start 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf65f0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55edf64e2f60 con 0x55edf65f5f60 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55edf6541850 con 0x55edf65f6880 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.664+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55edf64a9460 con 0x55edf65f07a0 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57740/0 (socket says 172.21.15.52:57740) 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 -- 172.21.15.52:0/2638970077 learned_addr learned my addr 172.21.15.52:0/2638970077 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:26.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efecce700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf65f0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 -- 172.21.15.52:0/2638970077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 msgr2=0x55edf65f0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf65f0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 -- 172.21.15.52:0/2638970077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 msgr2=0x55edf65f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 -- 172.21.15.52:0/2638970077 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf64908d0 con 0x55edf65f5f60 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efecce700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf65f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.665+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f6340 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4015000 tx=0x7f9ef401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd35cc9f395f7a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9eecff9700 1 -- 172.21.15.52:0/2638970077 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4003880 con 0x55edf65f5f60 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9eecff9700 1 -- 172.21.15.52:0/2638970077 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ef4005690 con 0x55edf65f5f60 2024-06-02T23:18:26.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9eecff9700 1 -- 172.21.15.52:0/2638970077 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef400fd90 con 0x55edf65f5f60 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2638970077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 msgr2=0x55edf65f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f6340 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4015000 tx=0x7f9ef401f000 comp rx=0 tx=0).stop 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2638970077 shutdown_connections 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf65f0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f6340 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2638970077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.666+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2638970077 >> 172.21.15.52:0/2638970077 conn(0x55edf651d080 msgr2=0x55edf65eb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:26.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2638970077 shutdown_connections 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2638970077 wait complete. 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 -- start start 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 Processor -- start 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 0x55edf65fc600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f6880 0x55edf65f9e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55edf64e1fc0 con 0x55edf65f5f60 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55edf653d810 con 0x55edf65f07a0 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55edf65fd200 con 0x55edf65f6880 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.667+0000 7f9f0afb4b80 10 client.0 _ll_register_callbacks cb 0x55edf64d75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 0x55edf65fc600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 0x55edf65fc600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44072/0 (socket says 172.21.15.52:44072) 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 -- 172.21.15.52:0/4162613109 learned_addr learned my addr 172.21.15.52:0/4162613109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efecce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f6880 0x55edf65f9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efdccc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 -- 172.21.15.52:0/4162613109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f6880 msgr2=0x55edf65f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f6880 0x55edf65f9e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 -- 172.21.15.52:0/4162613109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 msgr2=0x55edf65f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 -- 172.21.15.52:0/4162613109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf6507750 con 0x55edf65f07a0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efecce700 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f6880 0x55edf65f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 0x55edf65fc600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40098e0 tx=0x7f9ef4003d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=600e829bb108fe39 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9ee1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/4162613109 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4037070 con 0x55edf65f07a0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/4162613109 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9ef400f870 con 0x55edf65f07a0 2024-06-02T23:18:26.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.668+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef4005690 con 0x55edf65f07a0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/4162613109 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ef4043050 con 0x55edf65f07a0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 10 client.26924 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55edf66079f0 con 0x55edf65f07a0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/4162613109 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ef4005690 con 0x55edf65f07a0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/4162613109 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9ef4024bd0 con 0x55edf65f07a0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 10 client.26924 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 10 client.26924 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 20 client.26924 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6609ec0 0x55edf660c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 4 client.26924 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edf654ed50 con 0x55edf6609ec0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.669+0000 7f9efdccc700 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6609ec0 0x55edf660c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.670+0000 7f9ee1ffb700 10 client.26924 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:26.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.670+0000 7f9efdccc700 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6609ec0 0x55edf660c2b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ab8e819f83a1b3b server_cookie=f19b8dd6708f8d78 in_seq=0 out_seq=0 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.671+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/4162613109 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55edf654ed50 con 0x55edf6609ec0 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.671+0000 7f9ee1ffb700 10 client.26924 handle_command_reply: tid=0 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.671+0000 7f9f0afb4b80 1 client.26924 shutdown 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.671+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 msgr2=0x55edf65fc600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.671+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 0x55edf65fc600 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9ef40098e0 tx=0x7f9ef4003d00 comp rx=0 tx=0).stop 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 shutdown_connections 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6609ec0 0x55edf660c2b0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f6880 0x55edf65f9e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f07a0 0x55edf65fc600 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9ee1ffb700 0 client.26924 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/4162613109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf65f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 >> 172.21.15.52:0/4162613109 conn(0x55edf651d080 msgr2=0x55edf65ea700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:26.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 shutdown_connections 2024-06-02T23:18:26.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/4162613109 wait complete. 2024-06-02T23:18:26.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.672+0000 7f9f0afb4b80 20 client.26924 trim_cache size 0 max 16384 2024-06-02T23:18:26.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 Processor -- start 2024-06-02T23:18:26.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 -- start start 2024-06-02T23:18:26.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 0x55edf65f8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f5f60 0x55edf65eae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65eb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81aed00 con 0x55edf65f07a0 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81b0840 con 0x55edf65f6880 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81af3d0 con 0x55edf65f5f60 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 0x55edf65f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9efe4cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 0x55edf65f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57762/0 (socket says 172.21.15.52:57762) 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9efe4cd700 1 -- 172.21.15.52:0/1649783208 learned_addr learned my addr 172.21.15.52:0/1649783208 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.690+0000 7f9efecce700 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65eb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efdccc700 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f5f60 0x55edf65eae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efe4cd700 1 -- 172.21.15.52:0/1649783208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f5f60 msgr2=0x55edf65eae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f5f60 0x55edf65eae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efe4cd700 1 -- 172.21.15.52:0/1649783208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 msgr2=0x55edf65eb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65eb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efe4cd700 1 -- 172.21.15.52:0/1649783208 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edf6606530 con 0x55edf65f07a0 2024-06-02T23:18:26.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efdccc700 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f5f60 0x55edf65eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:26.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 0x55edf65f8820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4025fd0 tx=0x7f9ef400b3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534ce8dcef65c2ed server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:26.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9ee3fff700 1 -- 172.21.15.52:0/1649783208 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4037070 con 0x55edf65f07a0 2024-06-02T23:18:26.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9ee3fff700 1 -- 172.21.15.52:0/1649783208 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ef4005690 con 0x55edf65f07a0 2024-06-02T23:18:26.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.691+0000 7f9ee3fff700 1 -- 172.21.15.52:0/1649783208 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef4024440 con 0x55edf65f07a0 2024-06-02T23:18:26.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/1649783208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 msgr2=0x55edf65f8820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 0x55edf65f8820 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f9ef4025fd0 tx=0x7f9ef400b3f0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/1649783208 shutdown_connections 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf65eb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f07a0 0x55edf65f8820 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/1649783208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f5f60 0x55edf65eae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/1649783208 >> 172.21.15.52:0/1649783208 conn(0x55edf651d080 msgr2=0x55edf65f8110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/1649783208 shutdown_connections 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/1649783208 wait complete. 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 -- start start 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.692+0000 7f9f0afb4b80 1 Processor -- start 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf660f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf660ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf660cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81b0170 con 0x55edf65f5f60 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81afaa0 con 0x55edf65f6880 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef81accc0 con 0x55edf65f07a0 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9f0afb4b80 10 client.0 _ll_register_callbacks cb 0x55edf64d75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efecce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf660cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf660ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf660ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57770/0 (socket says 172.21.15.52:57770) 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 -- 172.21.15.52:0/2125174910 learned_addr learned my addr 172.21.15.52:0/2125174910 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf660f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 -- 172.21.15.52:0/2125174910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 msgr2=0x55edf660f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf660f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 -- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 msgr2=0x55edf660cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf660cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efdccc700 1 -- 172.21.15.52:0/2125174910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edf660d500 con 0x55edf65f5f60 2024-06-02T23:18:26.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.693+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf660f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9efdccc700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf660ca80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9ee809ba50 tx=0x7f9ee809c320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6090ffdb9cd8f33f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9ee1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/2125174910 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee80a4730 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/2125174910 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9ee8099b60 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef4005690 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/2125174910 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9ee8099dc0 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 10 client.17436 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55edf6674540 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/2125174910 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9ef4005690 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/2125174910 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9ee80ba210 con 0x55edf65f5f60 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 10 client.17436 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 10 client.17436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:26.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 20 client.17436 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.694+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6674c00 0x55edf6677020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.695+0000 7f9f0afb4b80 4 client.17436 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.695+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edf65492d0 con 0x55edf6674c00 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.695+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6674c00 0x55edf6677020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.695+0000 7f9efe4cd700 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6674c00 0x55edf6677020 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a77dc4e4c61e908b server_cookie=eefe6c824715ec82 in_seq=0 out_seq=0 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.695+0000 7f9ee1ffb700 10 client.17436 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9ee1ffb700 1 -- 172.21.15.52:0/2125174910 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edf65492d0 con 0x55edf6674c00 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9ee1ffb700 10 client.17436 handle_command_reply: tid=0 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 client.17436 shutdown 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 msgr2=0x55edf660ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf660ca80 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f9ee809ba50 tx=0x7f9ee809c320 comp rx=0 tx=0).stop 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 shutdown_connections 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55edf6674c00 0x55edf6677020 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55edf65f6880 0x55edf660cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9ee1ffb700 0 client.17436 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edf65f07a0 0x55edf660f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 --2- 172.21.15.52:0/2125174910 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55edf65f5f60 0x55edf660ca80 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 >> 172.21.15.52:0/2125174910 conn(0x55edf651d080 msgr2=0x55edf6605db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 shutdown_connections 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 1 -- 172.21.15.52:0/2125174910 wait complete. 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.696+0000 7f9f0afb4b80 20 client.17436 trim_cache size 0 max 16384 2024-06-02T23:18:26.700 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9ee40a2ca0 msgr2=0x7f9ee40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9ee40a2ca0 0x7f9ee40a5160 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9ef400b2e0 tx=0x7f9ef401a040 comp rx=0 tx=0).stop 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 msgr2=0x7f9ef81035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef81035b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800e9c0 tx=0x7f9ee800ecd0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 shutdown_connections 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9ee40a2ca0 0x7f9ee40a5160 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9ef8107400 0x7f9ef8103af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9ef8101390 0x7f9ef8103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 --2- 172.21.15.52:0/2907770785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ef8101d50 0x7f9ef81035b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 >> 172.21.15.52:0/2907770785 conn(0x7f9ef8078340 msgr2=0x7f9ef8100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:26.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 shutdown_connections 2024-06-02T23:18:26.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:26.697+0000 7f9f00731700 1 -- 172.21.15.52:0/2907770785 wait complete. 2024-06-02T23:18:27.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 116530 1.71 MB/sec execute 216 sec latency 15.012 ms 2024-06-02T23:18:28.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:27 smithi123 ceph-mon[35977]: pgmap v628: 97 pgs: 97 active+clean; 59 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:18:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:27 smithi174 ceph-mon[35994]: pgmap v628: 97 pgs: 97 active+clean; 59 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:18:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:27 smithi052 ceph-mon[32340]: pgmap v628: 97 pgs: 97 active+clean; 59 MiB data, 1003 MiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:18:28.459 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:18:28.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:18:28.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 117073 1.72 MB/sec execute 217 sec latency 12.306 ms 2024-06-02T23:18:28.741 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:29.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 117429 1.72 MB/sec execute 218 sec latency 13.246 ms 2024-06-02T23:18:30.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:29 smithi123 ceph-mon[35977]: pgmap v629: 97 pgs: 97 active+clean; 59 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-02T23:18:30.027 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:29 smithi052 ceph-mon[32340]: pgmap v629: 97 pgs: 97 active+clean; 59 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-02T23:18:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:29 smithi174 ceph-mon[35994]: pgmap v629: 97 pgs: 97 active+clean; 59 MiB data, 1008 MiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.3 MiB/s wr, 160 op/s 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 -- 172.21.15.52:0/1697301731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 msgr2=0x7fa8f410f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 --2- 172.21.15.52:0/1697301731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f410f3b0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00b0d0 tx=0x7fa8ec00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 -- 172.21.15.52:0/1697301731 shutdown_connections 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 --2- 172.21.15.52:0/1697301731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f410f3b0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 --2- 172.21.15.52:0/1697301731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f4102740 0x7fa8f4102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 --2- 172.21.15.52:0/1697301731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f41030f0 0x7fa8f4107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 -- 172.21.15.52:0/1697301731 >> 172.21.15.52:0/1697301731 conn(0x7fa8f40795b0 msgr2=0x7fa8f40799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 -- 172.21.15.52:0/1697301731 shutdown_connections 2024-06-02T23:18:30.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.427+0000 7fa8fb622700 1 -- 172.21.15.52:0/1697301731 wait complete. 2024-06-02T23:18:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.428+0000 7fa8fb622700 1 Processor -- start 2024-06-02T23:18:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.428+0000 7fa8fb622700 1 -- start start 2024-06-02T23:18:30.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.428+0000 7fa8fb622700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f4102740 0x7fa8f4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.428+0000 7fa8fb622700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 0x7fa8f4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.428+0000 7fa8fb622700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 0x7fa8f4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8fb622700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4112600 con 0x7fa8f4107830 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f9bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 0x7fa8f4074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44098/0 (socket says 172.21.15.52:44098) 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 -- 172.21.15.52:0/3727963132 learned_addr learned my addr 172.21.15.52:0/3727963132 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4112480 con 0x7fa8f41030f0 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8f4112780 con 0x7fa8f4102740 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f93be700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f4102740 0x7fa8f4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 -- 172.21.15.52:0/3727963132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f4102740 msgr2=0x7fa8f4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f4102740 0x7fa8f4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 -- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 msgr2=0x7fa8f4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 -- 172.21.15.52:0/3727963132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8f41ab440 con 0x7fa8f41030f0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 0x7fa8f4074cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d950 tx=0x7fa8e400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8ea7fc700 1 -- 172.21.15.52:0/3727963132 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e4016070 con 0x7fa8f41030f0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8f41ab6d0 con 0x7fa8f41030f0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8ea7fc700 1 -- 172.21.15.52:0/3727963132 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8e4009df0 con 0x7fa8f41030f0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8ea7fc700 1 -- 172.21.15.52:0/3727963132 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8e400f3c0 con 0x7fa8f41030f0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.429+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8f41abc60 con 0x7fa8f41030f0 2024-06-02T23:18:30.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.432+0000 7fa8ea7fc700 1 -- 172.21.15.52:0/3727963132 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8e4019070 con 0x7fa8f41030f0 2024-06-02T23:18:30.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.433+0000 7fa8ea7fc700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8e00a2c20 0x7fa8e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.433+0000 7fa8ea7fc700 1 -- 172.21.15.52:0/3727963132 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8e40cea70 con 0x7fa8f41030f0 2024-06-02T23:18:30.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.433+0000 7fa8f93be700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8e00a2c20 0x7fa8e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 Processor -- start 2024-06-02T23:18:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 -- start start 2024-06-02T23:18:30.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa8f93be700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8e00a2c20 0x7fa8e00a50e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009bf0 tx=0x7fa8f0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6116aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6116fe0 0x563bc6110a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc61113c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563bc6062890 con 0x563bc6110fa0 2024-06-02T23:18:30.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563bc600a7a0 con 0x563bc61166c0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563bc605e850 con 0x563bc6116fe0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa8f8bbd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6116fe0 0x563bc6110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa8f93be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6116aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa8f8bbd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6116fe0 0x563bc6110a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35804/0 (socket says 172.21.15.52:35804) 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa8f8bbd700 1 -- 172.21.15.52:0/2851809990 learned_addr learned my addr 172.21.15.52:0/2851809990 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.434+0000 7fa8f9bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc61113c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8f93be700 1 -- 172.21.15.52:0/2851809990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6116fe0 msgr2=0x563bc6110a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8f93be700 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6116fe0 0x563bc6110a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8f93be700 1 -- 172.21.15.52:0/2851809990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 msgr2=0x563bc61113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8f93be700 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc61113c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8f93be700 1 -- 172.21.15.52:0/2851809990 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc6000fd0 con 0x563bc61166c0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8f93be700 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6116aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0011d80 tx=0x7fa8f0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=617c07c6a5d67f1e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:30.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8df7fe700 1 -- 172.21.15.52:0/2851809990 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0012dc0 con 0x563bc61166c0 2024-06-02T23:18:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8df7fe700 1 -- 172.21.15.52:0/2851809990 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8f0011e70 con 0x563bc61166c0 2024-06-02T23:18:30.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.435+0000 7fa8df7fe700 1 -- 172.21.15.52:0/2851809990 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f001f420 con 0x563bc61166c0 2024-06-02T23:18:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 -- 172.21.15.52:0/2851809990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 msgr2=0x563bc6116aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6116aa0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0011d80 tx=0x7fa8f0014000 comp rx=0 tx=0).stop 2024-06-02T23:18:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 -- 172.21.15.52:0/2851809990 shutdown_connections 2024-06-02T23:18:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc61113c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6116aa0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/2851809990 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6116fe0 0x563bc6110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 -- 172.21.15.52:0/2851809990 >> 172.21.15.52:0/2851809990 conn(0x563bc603bd40 msgr2=0x563bc610b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 -- 172.21.15.52:0/2851809990 shutdown_connections 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 -- 172.21.15.52:0/2851809990 wait complete. 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 -- start start 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.436+0000 7fa905ea5b80 1 Processor -- start 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 0x563bc611cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc611a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563bc611da50 con 0x563bc6116fe0 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563bc6002c70 con 0x563bc6110fa0 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563bc611dbd0 con 0x563bc61166c0 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa905ea5b80 10 client.0 _ll_register_callbacks cb 0x563bc5ffc9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:30.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 0x563bc611cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f9bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc611a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 0x563bc611cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44114/0 (socket says 172.21.15.52:44114) 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 -- 172.21.15.52:0/45754174 learned_addr learned my addr 172.21.15.52:0/45754174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 -- 172.21.15.52:0/45754174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 msgr2=0x563bc611a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 -- 172.21.15.52:0/45754174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 msgr2=0x563bc611a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc611a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f93be700 1 -- 172.21.15.52:0/45754174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc5fb1950 con 0x563bc6110fa0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.437+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc611a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8f93be700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 0x563bc611cd40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0028680 tx=0x7fa8f00286b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4508f25188cb9f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8dcff9700 1 -- 172.21.15.52:0/45754174 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0028910 con 0x563bc6110fa0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8dcff9700 1 -- 172.21.15.52:0/45754174 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f0028ab0 con 0x563bc6110fa0 2024-06-02T23:18:30.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8f0011e70 con 0x563bc6110fa0 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8dcff9700 1 -- 172.21.15.52:0/45754174 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8f0028cb0 con 0x563bc6110fa0 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa905ea5b80 10 client.26942 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bc61283b0 con 0x563bc6110fa0 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8dcff9700 1 -- 172.21.15.52:0/45754174 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8f0011e70 con 0x563bc6110fa0 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.438+0000 7fa8dcff9700 1 -- 172.21.15.52:0/45754174 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa8f0033420 con 0x563bc6110fa0 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa905ea5b80 10 client.26942 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa905ea5b80 10 client.26942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa905ea5b80 20 client.26942 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc612a880 0x563bc612cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa905ea5b80 4 client.26942 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bc606fd90 con 0x563bc612a880 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc612a880 0x563bc612cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa8dcff9700 10 client.26942 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:30.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.439+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc612a880 0x563bc612cc70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa07dc704c236bc server_cookie=a22452bc74d8d1d9 in_seq=0 out_seq=0 2024-06-02T23:18:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa8dcff9700 1 -- 172.21.15.52:0/45754174 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563bc606fd90 con 0x563bc612a880 2024-06-02T23:18:30.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa8dcff9700 10 client.26942 handle_command_reply: tid=0 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 client.26942 shutdown 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 msgr2=0x563bc611cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 0x563bc611cd40 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0028680 tx=0x7fa8f00286b0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 shutdown_connections 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc612a880 0x563bc612cc70 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc611a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6110fa0 0x563bc611cd40 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/45754174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa8dcff9700 0 client.26942 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 >> 172.21.15.52:0/45754174 conn(0x563bc603bd40 msgr2=0x563bc610b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 shutdown_connections 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 1 -- 172.21.15.52:0/45754174 wait complete. 2024-06-02T23:18:30.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.440+0000 7fa905ea5b80 20 client.26942 trim_cache size 0 max 16384 2024-06-02T23:18:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 Processor -- start 2024-06-02T23:18:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 -- start start 2024-06-02T23:18:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6110fa0 0x563bc6110a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6119040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc6132270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800d2a0 con 0x563bc6116fe0 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800ede0 con 0x563bc61166c0 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800d970 con 0x563bc6110fa0 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc6132270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc6132270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57816/0 (socket says 172.21.15.52:57816) 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f8bbd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6119040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 -- 172.21.15.52:0/3359026122 learned_addr learned my addr 172.21.15.52:0/3359026122 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:30.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f93be700 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6110fa0 0x563bc6110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 -- 172.21.15.52:0/3359026122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6110fa0 msgr2=0x563bc6110a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6110fa0 0x563bc6110a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 -- 172.21.15.52:0/3359026122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 msgr2=0x563bc6119040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6119040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f9bbf700 1 -- 172.21.15.52:0/3359026122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bc610c4f0 con 0x563bc6116fe0 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.465+0000 7fa8f93be700 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6110fa0 0x563bc6110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.466+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6119040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.466+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc6132270 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004af0 tx=0x7fa8ec004b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663f33b265bab893 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.466+0000 7fa8deffd700 1 -- 172.21.15.52:0/3359026122 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec006020 con 0x563bc6116fe0 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.466+0000 7fa8deffd700 1 -- 172.21.15.52:0/3359026122 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8ec007990 con 0x563bc6116fe0 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.466+0000 7fa8deffd700 1 -- 172.21.15.52:0/3359026122 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec01cc30 con 0x563bc6116fe0 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 -- 172.21.15.52:0/3359026122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 msgr2=0x563bc6132270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc6132270 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec004af0 tx=0x7fa8ec004b20 comp rx=0 tx=0).stop 2024-06-02T23:18:30.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 -- 172.21.15.52:0/3359026122 shutdown_connections 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6116fe0 0x563bc6132270 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc6110fa0 0x563bc6110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/3359026122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc61166c0 0x563bc6119040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 -- 172.21.15.52:0/3359026122 >> 172.21.15.52:0/3359026122 conn(0x563bc603bd40 msgr2=0x563bc6118930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 -- 172.21.15.52:0/3359026122 shutdown_connections 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 -- 172.21.15.52:0/3359026122 wait complete. 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 -- start start 2024-06-02T23:18:30.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.467+0000 7fa905ea5b80 1 Processor -- start 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc6123550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6116fe0 0x563bc6120400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800e710 con 0x563bc6110fa0 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800e040 con 0x563bc6116fe0 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d800b240 con 0x563bc61166c0 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc6123550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa905ea5b80 10 client.0 _ll_register_callbacks cb 0x563bc5ffc9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc6123550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57828/0 (socket says 172.21.15.52:57828) 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 -- 172.21.15.52:0/4002231880 learned_addr learned my addr 172.21.15.52:0/4002231880 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6116fe0 0x563bc6120400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 -- 172.21.15.52:0/4002231880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 msgr2=0x563bc611fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 -- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6116fe0 msgr2=0x563bc6120400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6116fe0 0x563bc6120400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f93be700 1 -- 172.21.15.52:0/4002231880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bc611d540 con 0x563bc6110fa0 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.468+0000 7fa8f9bbf700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6116fe0 0x563bc6120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa8f93be700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc6123550 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0028680 tx=0x7fa8f0033d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da529ac66117c150 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:30.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa8dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa8dcff9700 1 -- 172.21.15.52:0/4002231880 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8f0011e70 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa8dcff9700 1 -- 172.21.15.52:0/4002231880 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8f0028930 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8ec007990 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa8dcff9700 1 -- 172.21.15.52:0/4002231880 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa8f0005030 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa905ea5b80 10 client.17454 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563bc6195f60 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.469+0000 7fa8dcff9700 1 -- 172.21.15.52:0/4002231880 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8ec007990 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa8dcff9700 1 -- 172.21.15.52:0/4002231880 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa8f0004bb0 con 0x563bc6110fa0 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa905ea5b80 10 client.17454 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa905ea5b80 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa905ea5b80 20 client.17454 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc6196640 0x563bc6198a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:30.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa905ea5b80 4 client.17454 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563bc606a310 con 0x563bc6196640 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc6196640 0x563bc6198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa8f8bbd700 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc6196640 0x563bc6198a60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da1b669c1170ed4e server_cookie=ba270dad4b4c88ca in_seq=0 out_seq=0 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.470+0000 7fa8dcff9700 10 client.17454 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa8dcff9700 1 -- 172.21.15.52:0/4002231880 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563bc606a310 con 0x563bc6196640 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa8dcff9700 10 client.17454 handle_command_reply: tid=0 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 client.17454 shutdown 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 msgr2=0x563bc6123550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc6123550 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0028680 tx=0x7fa8f0033d10 comp rx=0 tx=0).stop 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 shutdown_connections 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563bc6196640 0x563bc6198a60 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bc6116fe0 0x563bc6120400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bc6110fa0 0x563bc6123550 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 --2- 172.21.15.52:0/4002231880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bc61166c0 0x563bc611fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 >> 172.21.15.52:0/4002231880 conn(0x563bc603bd40 msgr2=0x563bc61151b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa8dcff9700 0 client.17454 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.471+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 shutdown_connections 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa905ea5b80 1 -- 172.21.15.52:0/4002231880 wait complete. 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa905ea5b80 20 client.17454 trim_cache size 0 max 16384 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "1cac2c81-ef82-4cab-8dd1-77463fd2ddae", 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:18:30.475 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:18:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8e00a2c20 msgr2=0x7fa8e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8e00a2c20 0x7fa8e00a50e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009bf0 tx=0x7fa8f0009260 comp rx=0 tx=0).stop 2024-06-02T23:18:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 msgr2=0x7fa8f4074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 0x7fa8f4074cd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa8e400d950 tx=0x7fa8e400dc60 comp rx=0 tx=0).stop 2024-06-02T23:18:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 shutdown_connections 2024-06-02T23:18:30.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8e00a2c20 0x7fa8e00a50e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8f4107830 0x7fa8f4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8f4102740 0x7fa8f4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 --2- 172.21.15.52:0/3727963132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8f41030f0 0x7fa8f4074cd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:30.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.472+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 >> 172.21.15.52:0/3727963132 conn(0x7fa8f40795b0 msgr2=0x7fa8f41005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:30.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.473+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 shutdown_connections 2024-06-02T23:18:30.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:30.473+0000 7fa8fb622700 1 -- 172.21.15.52:0/3727963132 wait complete. 2024-06-02T23:18:30.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 117685 1.72 MB/sec execute 219 sec latency 28.398 ms 2024-06-02T23:18:31.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:30 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:18:31.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:31.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:30 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:18:31.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.119 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:30 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:18:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.149 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:18:31.431 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:18:31.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 118193 1.72 MB/sec execute 220 sec latency 11.268 ms 2024-06-02T23:18:31.884 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: pgmap v630: 97 pgs: 97 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:18:31.885 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:31.886 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: pgmap v630: 97 pgs: 97 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:18:32.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: pgmap v630: 97 pgs: 97 active+clean; 59 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.3 MiB/s wr, 143 op/s 2024-06-02T23:18:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:18:32.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 118737 1.72 MB/sec execute 221 sec latency 15.082 ms 2024-06-02T23:18:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/1644274000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b81063b0 msgr2=0x7fc0b8106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/1644274000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b81063b0 0x7fc0b8106790 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8009990 tx=0x7fc0a8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/1644274000 shutdown_connections 2024-06-02T23:18:33.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/1644274000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b810e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/1644274000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b81063b0 0x7fc0b8106790 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/1644274000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0b8101670 0x7fc0b8101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/1644274000 >> 172.21.15.52:0/1644274000 conn(0x7fc0b8078250 msgr2=0x7fc0b8078660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/1644274000 shutdown_connections 2024-06-02T23:18:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.150+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/1644274000 wait complete. 2024-06-02T23:18:33.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.151+0000 7fc0bdaf4700 1 Processor -- start 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.151+0000 7fc0bdaf4700 1 -- start start 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0bdaf4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b8101670 0x7fc0b819e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0bdaf4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b819ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0bdaf4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0b81063b0 0x7fc0b8198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0bdaf4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81116a0 con 0x7fc0b8101670 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0bdaf4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8111520 con 0x7fc0b8102030 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0bdaf4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b8111820 con 0x7fc0b81063b0 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b819ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b8101670 0x7fc0b819e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b819ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44144/0 (socket says 172.21.15.52:44144) 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/457062333 learned_addr learned my addr 172.21.15.52:0/457062333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b8101670 0x7fc0b819e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57838/0 (socket says 172.21.15.52:57838) 2024-06-02T23:18:33.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0b81063b0 0x7fc0b8198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/457062333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0b81063b0 msgr2=0x7fc0b8198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0b81063b0 0x7fc0b8198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b8101670 msgr2=0x7fc0b819e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b8101670 0x7fc0b819e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.152+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/457062333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0b81991c0 con 0x7fc0b8102030 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.153+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b819ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000e9c0 tx=0x7fc0a000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.153+0000 7fc0b4ff9700 1 -- 172.21.15.52:0/457062333 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a000cbd0 con 0x7fc0b8102030 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.153+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0b81994b0 con 0x7fc0b8102030 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.153+0000 7fc0b4ff9700 1 -- 172.21.15.52:0/457062333 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0a000ee60 con 0x7fc0b8102030 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.153+0000 7fc0b4ff9700 1 -- 172.21.15.52:0/457062333 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00055b0 con 0x7fc0b8102030 2024-06-02T23:18:33.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.153+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0b81a5770 con 0x7fc0b8102030 2024-06-02T23:18:33.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.155+0000 7fc0b4ff9700 1 -- 172.21.15.52:0/457062333 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc0a0021440 con 0x7fc0b8102030 2024-06-02T23:18:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.156+0000 7fc0b4ff9700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0a40a2c20 0x7fc0a40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.156+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0a40a2c20 0x7fc0a40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.156+0000 7fc0b4ff9700 1 -- 172.21.15.52:0/457062333 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc0a00cf320 con 0x7fc0b8102030 2024-06-02T23:18:33.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0a40a2c20 0x7fc0a40a50e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8199ba0 tx=0x7fc0a800b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 Processor -- start 2024-06-02T23:18:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 -- start start 2024-06-02T23:18:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236563b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 0x563b2365a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b2365b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563b235a3810 con 0x563b236568f0 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b2354b720 con 0x563b2365ac00 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.157+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b2359f7d0 con 0x563b23655fd0 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 0x563b2365a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 0x563b2365a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57840/0 (socket says 172.21.15.52:57840) 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/3145383221 learned_addr learned my addr 172.21.15.52:0/3145383221 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/3145383221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 msgr2=0x563b236563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/3145383221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 msgr2=0x563b2365b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b2365b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/3145383221 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b23541f50 con 0x563b236568f0 2024-06-02T23:18:33.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.158+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 0x563b2365a6c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0098ec0 tx=0x7fc0a000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ccd8a38012915d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.159+0000 7fc0adffb700 1 -- 172.21.15.52:0/3145383221 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a00b3070 con 0x563b236568f0 2024-06-02T23:18:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.159+0000 7fc0adffb700 1 -- 172.21.15.52:0/3145383221 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0a009aad0 con 0x563b236568f0 2024-06-02T23:18:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.159+0000 7fc0adffb700 1 -- 172.21.15.52:0/3145383221 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0087060 con 0x563b236568f0 2024-06-02T23:18:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.159+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3145383221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 msgr2=0x563b2365a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.159+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 0x563b2365a6c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fc0a0098ec0 tx=0x7fc0a000cf90 comp rx=0 tx=0).stop 2024-06-02T23:18:33.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3145383221 shutdown_connections 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b2365b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3145383221 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b236568f0 0x563b2365a6c0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3145383221 >> 172.21.15.52:0/3145383221 conn(0x563b2357ccc0 msgr2=0x563b2364c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3145383221 shutdown_connections 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3145383221 wait complete. 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 -- start start 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.160+0000 7fc0c8377b80 1 Processor -- start 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236621b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b2365f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563b23662dc0 con 0x563b2365ac00 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563b23543bf0 con 0x563b236568f0 2024-06-02T23:18:33.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b23662f40 con 0x563b23655fd0 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0c8377b80 10 client.0 _ll_register_callbacks cb 0x563b2353d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b2365f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44168/0 (socket says 172.21.15.52:44168) 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236621b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b6ffd700 1 -- 172.21.15.52:0/1409734573 learned_addr learned my addr 172.21.15.52:0/1409734573 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:33.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b2365f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57856/0 (socket says 172.21.15.52:57856) 2024-06-02T23:18:33.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 -- 172.21.15.52:0/1409734573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 msgr2=0x563b236621b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236621b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 -- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 msgr2=0x563b2365f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b7fff700 1 -- 172.21.15.52:0/1409734573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b234f28d0 con 0x563b2365ac00 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.161+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b2365f9f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0011cc0 tx=0x7fc0b0009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6ffc86869a3e63 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc096ffd700 1 -- 172.21.15.52:0/1409734573 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0004db0 con 0x563b2365ac00 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc096ffd700 1 -- 172.21.15.52:0/1409734573 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc0b0003a00 con 0x563b2365ac00 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a009aad0 con 0x563b2365ac00 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc096ffd700 1 -- 172.21.15.52:0/1409734573 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0b0024050 con 0x563b2365ac00 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc0c8377b80 10 client.17466 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b2366d7d0 con 0x563b2365ac00 2024-06-02T23:18:33.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.162+0000 7fc096ffd700 1 -- 172.21.15.52:0/1409734573 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0b000fcb0 con 0x563b2365ac00 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc096ffd700 1 -- 172.21.15.52:0/1409734573 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc0b0004350 con 0x563b2365ac00 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0c8377b80 10 client.17466 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0c8377b80 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0c8377b80 20 client.17466 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b2366fdb0 0x563b236721d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0c8377b80 4 client.17466 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc0a009aad0 con 0x563b2366fdb0 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b2366fdb0 0x563b236721d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b2366fdb0 0x563b236721d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fabda2db8530dd9 server_cookie=9ac563b93f192a6c in_seq=0 out_seq=0 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.163+0000 7fc096ffd700 10 client.17466 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc096ffd700 1 -- 172.21.15.52:0/1409734573 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fc0a009aad0 con 0x563b2366fdb0 2024-06-02T23:18:33.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc096ffd700 10 client.17466 handle_command_reply: tid=0 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 client.17466 shutdown 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 msgr2=0x563b2365f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b2365f9f0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0011cc0 tx=0x7fc0b0009400 comp rx=0 tx=0).stop 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 shutdown_connections 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b2366fdb0 0x563b236721d0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b2365f9f0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b236621b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1409734573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 >> 172.21.15.52:0/1409734573 conn(0x563b2357ccc0 msgr2=0x563b236509d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc096ffd700 0 client.17466 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 shutdown_connections 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1409734573 wait complete. 2024-06-02T23:18:33.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.164+0000 7fc0c8377b80 20 client.17466 trim_cache size 0 max 16384 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 Processor -- start 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 -- start start 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b2365eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b23677aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81aa880 con 0x563b2365ac00 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81ac3c0 con 0x563b236568f0 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81aaf50 con 0x563b23655fd0 2024-06-02T23:18:33.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b2365eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0b7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b23677aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0b6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b2365eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35876/0 (socket says 172.21.15.52:35876) 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.187+0000 7fc0b77fe700 1 -- 172.21.15.52:0/3715000570 learned_addr learned my addr 172.21.15.52:0/3715000570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b7fff700 1 -- 172.21.15.52:0/3715000570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 msgr2=0x563b2365eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b2365eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b7fff700 1 -- 172.21.15.52:0/3715000570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 msgr2=0x563b2365a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b7fff700 1 -- 172.21.15.52:0/3715000570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b236464d0 con 0x563b2365ac00 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b2365eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:33.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b23677aa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0011ff0 tx=0x7fc0b00095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a2952729536faf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0ad7fa700 1 -- 172.21.15.52:0/3715000570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b002c070 con 0x563b2365ac00 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0ad7fa700 1 -- 172.21.15.52:0/3715000570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0b0005c60 con 0x563b2365ac00 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.188+0000 7fc0ad7fa700 1 -- 172.21.15.52:0/3715000570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0b000fa60 con 0x563b2365ac00 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3715000570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 msgr2=0x563b23677aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b23677aa0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc0b0011ff0 tx=0x7fc0b00095e0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3715000570 shutdown_connections 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b2365ac00 0x563b23677aa0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b23655fd0 0x563b2365eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/3715000570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b236568f0 0x563b2365a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3715000570 >> 172.21.15.52:0/3715000570 conn(0x563b2357ccc0 msgr2=0x563b23659ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3715000570 shutdown_connections 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.189+0000 7fc0c8377b80 1 -- 172.21.15.52:0/3715000570 wait complete. 2024-06-02T23:18:33.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 -- start start 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 Processor -- start 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 0x563b236662d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b236568f0 0x563b23666810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b236d9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81abcf0 con 0x563b23655fd0 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81ab620 con 0x563b2365ac00 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0b81a8860 con 0x563b236568f0 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0c8377b80 10 client.0 _ll_register_callbacks cb 0x563b2353d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 0x563b236662d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0b77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 0x563b236662d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57878/0 (socket says 172.21.15.52:57878) 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0b77fe700 1 -- 172.21.15.52:0/1298668228 learned_addr learned my addr 172.21.15.52:0/1298668228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b236568f0 0x563b23666810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.190+0000 7fc0b7fff700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b236d9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0b77fe700 1 -- 172.21.15.52:0/1298668228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b236568f0 msgr2=0x563b23666810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b236568f0 0x563b23666810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0b77fe700 1 -- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 msgr2=0x563b236d9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b236d9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0b77fe700 1 -- 172.21.15.52:0/1298668228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b236670d0 con 0x563b23655fd0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0b77fe700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 0x563b236662d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8006010 tx=0x7fc0a8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=208e2c1c29abb8c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc096ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc096ffd700 1 -- 172.21.15.52:0/1298668228 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a8037030 con 0x563b23655fd0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc096ffd700 1 -- 172.21.15.52:0/1298668228 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc0a8003950 con 0x563b23655fd0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0b0005c60 con 0x563b23655fd0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc096ffd700 1 -- 172.21.15.52:0/1298668228 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc0a8003b50 con 0x563b23655fd0 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0c8377b80 10 client.17472 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:33.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.191+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563b236db0d0 con 0x563b23655fd0 2024-06-02T23:18:33.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc096ffd700 1 -- 172.21.15.52:0/1298668228 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc0b0005c60 con 0x563b23655fd0 2024-06-02T23:18:33.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc096ffd700 1 -- 172.21.15.52:0/1298668228 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc0a803e4d0 con 0x563b23655fd0 2024-06-02T23:18:33.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0c8377b80 10 client.17472 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:33.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0c8377b80 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:33.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0c8377b80 20 client.17472 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:33.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b236db7e0 0x563b236ddc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:33.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0c8377b80 4 client.17472 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:18:33.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b235b0d10 con 0x563b236db7e0 2024-06-02T23:18:33.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.192+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b236db7e0 0x563b236ddc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0b6ffd700 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b236db7e0 0x563b236ddc00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e717751a4db7e6b1 server_cookie=f161a9303fd28341 in_seq=0 out_seq=0 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc096ffd700 10 client.17472 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc096ffd700 1 -- 172.21.15.52:0/1298668228 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x563b235b0d10 con 0x563b236db7e0 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc096ffd700 10 client.17472 handle_command_reply: tid=0 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 client.17472 shutdown 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 msgr2=0x563b236662d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 0x563b236662d0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc0a8006010 tx=0x7fc0a8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 shutdown_connections 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563b236db7e0 0x563b236ddc00 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563b2365ac00 0x563b236d9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563b23655fd0 0x563b236662d0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 --2- 172.21.15.52:0/1298668228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b236568f0 0x563b23666810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc096ffd700 0 client.17472 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 >> 172.21.15.52:0/1298668228 conn(0x563b2357ccc0 msgr2=0x563b23660ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 shutdown_connections 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 1 -- 172.21.15.52:0/1298668228 wait complete. 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.193+0000 7fc0c8377b80 20 client.17472 trim_cache size 0 max 16384 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stdout: "1cac2c81-ef82-4cab-8dd1-77463fd2ddae": { 2024-06-02T23:18:33.197 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:18:33.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1cac2c81-ef82-4cab-8dd1-77463fd2ddae", 2024-06-02T23:18:33.198 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:18:33.198 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:33.198 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:18:33.198 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0a40a2c20 msgr2=0x7fc0a40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0a40a2c20 0x7fc0a40a50e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8199ba0 tx=0x7fc0a800b260 comp rx=0 tx=0).stop 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 msgr2=0x7fc0b819ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b819ede0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc0a000e9c0 tx=0x7fc0a000ecd0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 shutdown_connections 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc0a40a2c20 0x7fc0a40a50e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0b81063b0 0x7fc0b8198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc0b8101670 0x7fc0b819e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 --2- 172.21.15.52:0/457062333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0b8102030 0x7fc0b819ede0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 >> 172.21.15.52:0/457062333 conn(0x7fc0b8078250 msgr2=0x7fc0b806aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:33.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 shutdown_connections 2024-06-02T23:18:33.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:18:33.194+0000 7fc0bdaf4700 1 -- 172.21.15.52:0/457062333 wait complete. 2024-06-02T23:18:33.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 119398 1.72 MB/sec execute 222 sec latency 42.050 ms 2024-06-02T23:18:34.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:33 smithi123 ceph-mon[35977]: pgmap v631: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-02T23:18:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:33 smithi174 ceph-mon[35994]: pgmap v631: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-02T23:18:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:33 smithi052 ceph-mon[32340]: pgmap v631: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.4 MiB/s wr, 163 op/s 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 -- 172.21.15.52:0/2947939128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c101a50 msgr2=0x7ff20c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2947939128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c101a50 0x7ff20c101e70 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009900 tx=0x7ff1fc02a410 comp rx=0 tx=0).stop 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 -- 172.21.15.52:0/2947939128 shutdown_connections 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2947939128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff20c103600 0x7ff20c109e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2947939128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c102c40 0x7ff20c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2947939128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c101a50 0x7ff20c101e70 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 -- 172.21.15.52:0/2947939128 >> 172.21.15.52:0/2947939128 conn(0x7ff20c0fcfc0 msgr2=0x7ff20c0ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 -- 172.21.15.52:0/2947939128 shutdown_connections 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.509+0000 7ff2116d4640 1 -- 172.21.15.52:0/2947939128 wait complete. 2024-06-02T23:18:34.509 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 Processor -- start 2024-06-02T23:18:34.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 -- start start 2024-06-02T23:18:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c101a50 0x7ff20c075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff20c102c40 0x7ff20c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 0x7ff20c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10d580 con 0x7ff20c103600 2024-06-02T23:18:34.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10d400 con 0x7ff20c101a50 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff2116d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c10d700 con 0x7ff20c102c40 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff20b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 0x7ff20c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff20b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 0x7ff20c0742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33218/0 (socket says 172.21.15.52:33218) 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff20affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c101a50 0x7ff20c075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff20b7fe640 1 -- 172.21.15.52:0/2817770761 learned_addr learned my addr 172.21.15.52:0/2817770761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.510+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff20c102c40 0x7ff20c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20b7fe640 1 -- 172.21.15.52:0/2817770761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff20c102c40 msgr2=0x7ff20c073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20b7fe640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff20c102c40 0x7ff20c073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20b7fe640 1 -- 172.21.15.52:0/2817770761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c101a50 msgr2=0x7ff20c075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20b7fe640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c101a50 0x7ff20c075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20b7fe640 1 -- 172.21.15.52:0/2817770761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff20c074b60 con 0x7ff20c103600 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20affd640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c101a50 0x7ff20c075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff20b7fe640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 0x7ff20c0742d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff20400eb10 tx=0x7ff20400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:34.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff1f3fff640 1 -- 172.21.15.52:0/2817770761 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204004b20 con 0x7ff20c103600 2024-06-02T23:18:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff20c1a1d10 con 0x7ff20c103600 2024-06-02T23:18:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff1f3fff640 1 -- 172.21.15.52:0/2817770761 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff204004cc0 con 0x7ff20c103600 2024-06-02T23:18:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff1f3fff640 1 -- 172.21.15.52:0/2817770761 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff204010690 con 0x7ff20c103600 2024-06-02T23:18:34.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.511+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff20c1a21f0 con 0x7ff20c103600 2024-06-02T23:18:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.513+0000 7ff1f3fff640 1 -- 172.21.15.52:0/2817770761 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff204005020 con 0x7ff20c103600 2024-06-02T23:18:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 Processor -- start 2024-06-02T23:18:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 -- start start 2024-06-02T23:18:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819ed010 0x5586819ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 0x5586819f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586819504f0 con 0x5586819eda00 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5586818ee870 con 0x5586819f2b70 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff2143e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55868194c4b0 con 0x5586819ed010 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff1f3fff640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff20affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 0x5586819f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff20affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 0x5586819f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33222/0 (socket says 172.21.15.52:33222) 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff20affd640 1 -- 172.21.15.52:0/1906264253 learned_addr learned my addr 172.21.15.52:0/1906264253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.514+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819ed010 0x5586819ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20b7fe640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20affd640 1 -- 172.21.15.52:0/1906264253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819ed010 msgr2=0x5586819ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20affd640 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819ed010 0x5586819ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20affd640 1 -- 172.21.15.52:0/1906264253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 msgr2=0x5586819f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20affd640 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20affd640 1 -- 172.21.15.52:0/1906264253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55868190a9b0 con 0x5586819eda00 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff1f3fff640 1 -- 172.21.15.52:0/2817770761 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff2040cf7f0 con 0x7ff20c103600 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20affd640 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 0x5586819f2500 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009900 tx=0x7ff1fc032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4201e3ee0c2ff71a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff20b7fe640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff204000f80 tx=0x7ff20400efd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff1f17fa640 1 -- 172.21.15.52:0/1906264253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc035070 con 0x5586819eda00 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff1f17fa640 1 -- 172.21.15.52:0/1906264253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1fc0035f0 con 0x5586819eda00 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.515+0000 7ff1f17fa640 1 -- 172.21.15.52:0/1906264253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1fc0044e0 con 0x5586819eda00 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 -- 172.21.15.52:0/1906264253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 msgr2=0x5586819f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 0x5586819f2500 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff1fc009900 tx=0x7ff1fc032eb0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 -- 172.21.15.52:0/1906264253 shutdown_connections 2024-06-02T23:18:34.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819eda00 0x5586819f2500 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1906264253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819ed010 0x5586819ed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 -- 172.21.15.52:0/1906264253 >> 172.21.15.52:0/1906264253 conn(0x5586819e8680 msgr2=0x5586819e8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 -- 172.21.15.52:0/1906264253 shutdown_connections 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 -- 172.21.15.52:0/1906264253 wait complete. 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.516+0000 7ff2143e3740 1 -- start start 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 Processor -- start 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 0x5586819f7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586819f5ac0 con 0x5586819ed010 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5586818d9ae0 con 0x5586819f2b70 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586819f5c40 con 0x5586819eda00 2024-06-02T23:18:34.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff2143e3740 10 client.0 _ll_register_callbacks cb 0x5586819129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 0x5586819f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 0x5586819f7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33232/0 (socket says 172.21.15.52:33232) 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1840122585 learned_addr learned my addr 172.21.15.52:0/1840122585 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20affd640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1840122585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 msgr2=0x5586819f7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 msgr2=0x5586819fe7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819fe7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1840122585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558681664210 con 0x5586819ed010 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 0x5586819f7230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff200004530 tx=0x7ff2000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeababd2d6906af2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.517+0000 7ff1eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1840122585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000f650 con 0x5586819ed010 2024-06-02T23:18:34.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1fc0035f0 con 0x5586819ed010 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1840122585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff20000f7f0 con 0x5586819ed010 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1840122585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff200012020 con 0x5586819ed010 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 10 client.17484 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558681730620 con 0x5586819ed010 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1840122585 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1fc0035f0 con 0x5586819ed010 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1840122585 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff200027910 con 0x5586819ed010 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 10 client.17484 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 10 client.17484 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 20 client.17484 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a034c0 0x558681a05960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 4 client.17484 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.518+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558681963f10 con 0x558681a034c0 2024-06-02T23:18:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.519+0000 7ff20affd640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a034c0 0x558681a05960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.519+0000 7ff20affd640 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a034c0 0x558681a05960 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b2f9935b72b5656 server_cookie=6576554161d6981b in_seq=0 out_seq=0 2024-06-02T23:18:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.519+0000 7ff1eeff5640 10 client.17484 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1840122585 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558681963f10 con 0x558681a034c0 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff1eeff5640 10 client.17484 handle_command_reply: tid=0 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 client.17484 shutdown 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 msgr2=0x5586819f7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 0x5586819f7230 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7ff200004530 tx=0x7ff2000046c0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 shutdown_connections 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a034c0 0x558681a05960 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819f2b70 0x5586819fe7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1840122585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819ed010 0x5586819f7230 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 >> 172.21.15.52:0/1840122585 conn(0x5586819e8680 msgr2=0x5586819e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff1eeff5640 0 client.17484 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 shutdown_connections 2024-06-02T23:18:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 1 -- 172.21.15.52:0/1840122585 wait complete. 2024-06-02T23:18:34.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.520+0000 7ff2143e3740 20 client.17484 trim_cache size 0 max 16384 2024-06-02T23:18:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 Processor -- start 2024-06-02T23:18:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 -- start start 2024-06-02T23:18:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x5586819fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819eba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1aaf80 con 0x5586819f2b70 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1acac0 con 0x5586819eb590 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff2143e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1ab650 con 0x5586819eda00 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff20affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x5586819fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff20affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x5586819fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33234/0 (socket says 172.21.15.52:33234) 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff20affd640 1 -- 172.21.15.52:0/2590019410 learned_addr learned my addr 172.21.15.52:0/2590019410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.536+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819eba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 -- 172.21.15.52:0/2590019410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 msgr2=0x5586819f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 -- 172.21.15.52:0/2590019410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 msgr2=0x5586819fe770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x5586819fe770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 -- 172.21.15.52:0/2590019410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586819ed7a0 con 0x5586819eb590 2024-06-02T23:18:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20affd640 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x5586819fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819eba40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff200012d10 tx=0x7ff200005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fc9f3b668182eb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff1f0ff9640 1 -- 172.21.15.52:0/2590019410 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200009530 con 0x5586819eb590 2024-06-02T23:18:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff1f0ff9640 1 -- 172.21.15.52:0/2590019410 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff200027a20 con 0x5586819eb590 2024-06-02T23:18:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.537+0000 7ff1f0ff9640 1 -- 172.21.15.52:0/2590019410 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff200004730 con 0x5586819eb590 2024-06-02T23:18:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 -- 172.21.15.52:0/2590019410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 msgr2=0x5586819eba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819eba40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff200012d10 tx=0x7ff200005f90 comp rx=0 tx=0).stop 2024-06-02T23:18:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 -- 172.21.15.52:0/2590019410 shutdown_connections 2024-06-02T23:18:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819eba40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x5586819fe770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 --2- 172.21.15.52:0/2590019410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 -- 172.21.15.52:0/2590019410 >> 172.21.15.52:0/2590019410 conn(0x5586819e8680 msgr2=0x5586819f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 -- 172.21.15.52:0/2590019410 shutdown_connections 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 -- 172.21.15.52:0/2590019410 wait complete. 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 -- start start 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.538+0000 7ff2143e3740 1 Processor -- start 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819fb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x558681a6c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1ac3f0 con 0x5586819f2b70 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1abd20 con 0x5586819eb590 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff20c1a8f60 con 0x5586819eda00 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 10 client.0 _ll_register_callbacks cb 0x7ff200009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819fb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819fb6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55388/0 (socket says 172.21.15.52:55388) 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1111588856 learned_addr learned my addr 172.21.15.52:0/1111588856 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x558681a6c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20affd640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1111588856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 msgr2=0x5586819fbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819fbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1111588856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 msgr2=0x558681a6c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x558681a6c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 -- 172.21.15.52:0/1111588856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558681a6c770 con 0x5586819eb590 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20affd640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff20a7fc640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819fb6b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff200027500 tx=0x7ff200003d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6992a6aa0faedfad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff1eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1111588856 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20000f450 con 0x5586819eb590 2024-06-02T23:18:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1111588856 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff20000f5f0 con 0x5586819eb590 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.539+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff200027a20 con 0x5586819eb590 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1111588856 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff20000f7f0 con 0x5586819eb590 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 10 client.26981 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558681a6d8d0 con 0x5586819eb590 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1111588856 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff200027a20 con 0x5586819eb590 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1111588856 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff200011960 con 0x5586819eb590 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 10 client.26981 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 10 client.26981 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 20 client.26981 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a6e1d0 0x558681a705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 4 client.26981 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55868195dc20 con 0x558681a6e1d0 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.540+0000 7ff20affd640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a6e1d0 0x558681a705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.541+0000 7ff1eeff5640 10 client.26981 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.541+0000 7ff20affd640 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a6e1d0 0x558681a705f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe64eb472db7ce server_cookie=928278571d4daff9 in_seq=0 out_seq=0 2024-06-02T23:18:34.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff1eeff5640 1 -- 172.21.15.52:0/1111588856 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55868195dc20 con 0x558681a6e1d0 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff1eeff5640 10 client.26981 handle_command_reply: tid=0 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 client.26981 shutdown 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 msgr2=0x5586819fb6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819fb6b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff200027500 tx=0x7ff200003d40 comp rx=0 tx=0).stop 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 shutdown_connections 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558681a6e1d0 0x558681a705f0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586819eda00 0x5586819fbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff1eeff5640 0 client.26981 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586819f2b70 0x558681a6c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 --2- 172.21.15.52:0/1111588856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586819eb590 0x5586819fb6b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 >> 172.21.15.52:0/1111588856 conn(0x5586819e8680 msgr2=0x5586819f6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:34.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 shutdown_connections 2024-06-02T23:18:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 1 -- 172.21.15.52:0/1111588856 wait complete. 2024-06-02T23:18:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.560+0000 7ff2143e3740 20 client.26981 trim_cache size 0 max 16384 2024-06-02T23:18:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:18:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:18:34.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1f40a2c30 msgr2=0x7ff1f40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff204000f80 tx=0x7ff20400efd0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 msgr2=0x7ff20c0742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 0x7ff20c0742d0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff20400eb10 tx=0x7ff20400c2d0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 shutdown_connections 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1f40a2c30 0x7ff1f40a50f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff20c102c40 0x7ff20c073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff20c103600 0x7ff20c0742d0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 --2- 172.21.15.52:0/2817770761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff20c101a50 0x7ff20c075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 >> 172.21.15.52:0/2817770761 conn(0x7ff20c0fcfc0 msgr2=0x7ff20c105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 shutdown_connections 2024-06-02T23:18:34.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:34.561+0000 7ff2116d4640 1 -- 172.21.15.52:0/2817770761 wait complete. 2024-06-02T23:18:34.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 119969 1.73 MB/sec execute 223 sec latency 35.441 ms 2024-06-02T23:18:35.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 120411 1.74 MB/sec execute 224 sec latency 64.144 ms 2024-06-02T23:18:35.782 INFO:tasks.cephfs.filesystem:scrub status for tag:1cac2c81-ef82-4cab-8dd1-77463fd2ddae - {'path': '/', 'tag': '1cac2c81-ef82-4cab-8dd1-77463fd2ddae', 'options': 'recursive,force'} 2024-06-02T23:18:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:35 smithi123 ceph-mon[35977]: pgmap v632: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-06-02T23:18:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:35 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:18:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:35 smithi174 ceph-mon[35994]: pgmap v632: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-06-02T23:18:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:35 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:18:36.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:35 smithi052 ceph-mon[32340]: pgmap v632: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-06-02T23:18:36.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:35 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:18:36.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 120883 1.75 MB/sec execute 225 sec latency 15.114 ms 2024-06-02T23:18:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:18:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:18:37.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:18:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:18:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:18:37.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:18:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:18:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:18:37.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:18:37.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 121148 1.75 MB/sec execute 226 sec latency 15.758 ms 2024-06-02T23:18:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:37 smithi123 ceph-mon[35977]: pgmap v633: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-02T23:18:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:37 smithi174 ceph-mon[35994]: pgmap v633: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-02T23:18:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:37 smithi052 ceph-mon[32340]: pgmap v633: 97 pgs: 97 active+clean; 61 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 1.4 MiB/s wr, 178 op/s 2024-06-02T23:18:38.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 121581 1.74 MB/sec execute 227 sec latency 15.624 ms 2024-06-02T23:18:39.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 122166 1.75 MB/sec execute 228 sec latency 15.267 ms 2024-06-02T23:18:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:39 smithi123 ceph-mon[35977]: pgmap v634: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-02T23:18:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:39 smithi174 ceph-mon[35994]: pgmap v634: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-02T23:18:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:39 smithi052 ceph-mon[32340]: pgmap v634: 97 pgs: 97 active+clean; 60 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.2 MiB/s wr, 147 op/s 2024-06-02T23:18:40.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 122809 1.75 MB/sec execute 229 sec latency 15.797 ms 2024-06-02T23:18:41.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 123390 1.76 MB/sec execute 230 sec latency 15.412 ms 2024-06-02T23:18:41.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:41 smithi052 ceph-mon[32340]: pgmap v635: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-06-02T23:18:42.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:41 smithi123 ceph-mon[35977]: pgmap v635: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-06-02T23:18:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:41 smithi174 ceph-mon[35994]: pgmap v635: 97 pgs: 97 active+clean; 62 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.4 MiB/s wr, 158 op/s 2024-06-02T23:18:42.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 123963 1.77 MB/sec execute 231 sec latency 14.850 ms 2024-06-02T23:18:43.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 124444 1.78 MB/sec execute 232 sec latency 14.713 ms 2024-06-02T23:18:44.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:43 smithi123 ceph-mon[35977]: pgmap v636: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-02T23:18:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:43 smithi174 ceph-mon[35994]: pgmap v636: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-02T23:18:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:43 smithi052 ceph-mon[32340]: pgmap v636: 97 pgs: 97 active+clean; 63 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-02T23:18:44.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 124725 1.78 MB/sec execute 233 sec latency 12.286 ms 2024-06-02T23:18:45.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 125162 1.77 MB/sec execute 234 sec latency 26.079 ms 2024-06-02T23:18:45.943 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:45 smithi174 ceph-mon[35994]: pgmap v637: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:18:45.944 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:45 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:18:46.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:45 smithi123 ceph-mon[35977]: pgmap v637: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:18:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:45 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:18:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:45 smithi052 ceph-mon[32340]: pgmap v637: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:18:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:45 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:18:46.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 125718 1.78 MB/sec execute 235 sec latency 15.121 ms 2024-06-02T23:18:47.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:18:47.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:18:47.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:46 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:18:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:18:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:18:47.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:46 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:18:47.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:18:47.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:18:47.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:46 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:18:47.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 126372 1.78 MB/sec execute 236 sec latency 14.634 ms 2024-06-02T23:18:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 -- 172.21.15.52:0/3968692284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4100900 msgr2=0x7f17b4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3968692284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4100900 0x7f17b4100d20 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f17b0009900 tx=0x7f17b002a410 comp rx=0 tx=0).stop 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 -- 172.21.15.52:0/3968692284 shutdown_connections 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3968692284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b4101af0 0x7f17b4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3968692284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17b41024b0 0x7f17b4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3968692284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4100900 0x7f17b4100d20 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 -- 172.21.15.52:0/3968692284 >> 172.21.15.52:0/3968692284 conn(0x7f17b40fbe70 msgr2=0x7f17b40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 -- 172.21.15.52:0/3968692284 shutdown_connections 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 -- 172.21.15.52:0/3968692284 wait complete. 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.659+0000 7f17bbf89640 1 Processor -- start 2024-06-02T23:18:47.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 -- start start 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b4100900 0x7f17b4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 0x7f17b4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17b41024b0 0x7f17b419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410c400 con 0x7f17b4101af0 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410c280 con 0x7f17b41024b0 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17bbf89640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17b410c580 con 0x7f17b4100900 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 0x7f17b4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 0x7f17b4198100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47004/0 (socket says 172.21.15.52:47004) 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 -- 172.21.15.52:0/3782569790 learned_addr learned my addr 172.21.15.52:0/3782569790 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9d25640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b4100900 0x7f17b4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17ba526640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17b41024b0 0x7f17b419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 -- 172.21.15.52:0/3782569790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b4100900 msgr2=0x7f17b4197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b4100900 0x7f17b4197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 -- 172.21.15.52:0/3782569790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17b41024b0 msgr2=0x7f17b419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17b41024b0 0x7f17b419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 -- 172.21.15.52:0/3782569790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b419f7e0 con 0x7f17b4101af0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.660+0000 7f17b9524640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 0x7f17b4198100 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f17a400d950 tx=0x7f17a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.661+0000 7f17aaffd640 1 -- 172.21.15.52:0/3782569790 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4017030 con 0x7f17b4101af0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.661+0000 7f17aaffd640 1 -- 172.21.15.52:0/3782569790 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17a4004020 con 0x7f17b4101af0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.661+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b419fad0 con 0x7f17b4101af0 2024-06-02T23:18:47.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.661+0000 7f17aaffd640 1 -- 172.21.15.52:0/3782569790 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40047a0 con 0x7f17b4101af0 2024-06-02T23:18:47.664 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.661+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b41a0070 con 0x7f17b4101af0 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17aaffd640 1 -- 172.21.15.52:0/3782569790 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f17a40041d0 con 0x7f17b4101af0 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17bed74740 1 Processor -- start 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17bed74740 1 -- start start 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17bed74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17bed74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be72910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17bed74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be75410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.664+0000 7f17bed74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ee4bdcc4f0 con 0x55ee4be6de10 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17bed74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee4bd6a870 con 0x55ee4be6d420 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17bed74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee4bdc84b0 con 0x55ee4be72f80 2024-06-02T23:18:47.667 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17b9d25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be72910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17b9d25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be72910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47012/0 (socket says 172.21.15.52:47012) 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17b9d25640 1 -- 172.21.15.52:0/2781597374 learned_addr learned my addr 172.21.15.52:0/2781597374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17b9524640 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be75410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 -- 172.21.15.52:0/2781597374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 msgr2=0x55ee4be75410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be75410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 -- 172.21.15.52:0/2781597374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 msgr2=0x55ee4be72910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be72910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 -- 172.21.15.52:0/2781597374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4bd869b0 con 0x55ee4be6d420 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17b9d25640 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be72910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17aaffd640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17a00a2cb0 0x7f17a00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17b9d25640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17a00a2cb0 0x7f17a00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17ba526640 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be6d840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009e50 tx=0x7f17ac0107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b6a408fabd56c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f17aaffd640 1 -- 172.21.15.52:0/3782569790 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17a40ceab0 con 0x7f17b4101af0 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.665+0000 7f179ffff640 1 -- 172.21.15.52:0/2781597374 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac003b70 con 0x55ee4be6d420 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f179ffff640 1 -- 172.21.15.52:0/2781597374 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17ac015070 con 0x55ee4be6d420 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f179ffff640 1 -- 172.21.15.52:0/2781597374 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac011740 con 0x55ee4be6d420 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 -- 172.21.15.52:0/2781597374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 msgr2=0x55ee4be6d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be6d840 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009e50 tx=0x7f17ac0107a0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17b9d25640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17a00a2cb0 0x7f17a00a5170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55ee4be6e210 tx=0x7f17b0005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 -- 172.21.15.52:0/2781597374 shutdown_connections 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be75410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be72910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 --2- 172.21.15.52:0/2781597374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be6d840 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 -- 172.21.15.52:0/2781597374 >> 172.21.15.52:0/2781597374 conn(0x55ee4be64680 msgr2=0x55ee4be64a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:47.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 -- 172.21.15.52:0/2781597374 shutdown_connections 2024-06-02T23:18:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.666+0000 7f17bed74740 1 -- 172.21.15.52:0/2781597374 wait complete. 2024-06-02T23:18:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 -- start start 2024-06-02T23:18:47.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 Processor -- start 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 0x55ee4be7ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ee4be75f30 con 0x55ee4be72f80 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ee4bd55ae0 con 0x55ee4be6d420 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ee4be760b0 con 0x55ee4be6de10 2024-06-02T23:18:47.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17bed74740 10 client.0 _ll_register_callbacks cb 0x55ee4bd8e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.667+0000 7f17ba526640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 0x55ee4be7ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17b9d25640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 0x55ee4be7ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47020/0 (socket says 172.21.15.52:47020) 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17b9d25640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37408/0 (socket says 172.21.15.52:37408) 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 -- 172.21.15.52:0/1474182675 learned_addr learned my addr 172.21.15.52:0/1474182675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17b9524640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 -- 172.21.15.52:0/1474182675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6de10 msgr2=0x55ee4be77c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 -- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 msgr2=0x55ee4be776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 -- 172.21.15.52:0/1474182675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4bae0210 con 0x55ee4be72f80 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17b9d25640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:47.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17ba526640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 0x55ee4be7ec70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005b20 tx=0x7f17ac004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22ef88b0ac5bd24 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f179d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f179d7fa640 1 -- 172.21.15.52:0/1474182675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac02c060 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f179d7fa640 1 -- 172.21.15.52:0/1474182675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17ac003b80 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.668+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17ac015070 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f179d7fa640 1 -- 172.21.15.52:0/1474182675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17ac003d80 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 10 client.17502 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee4bbac620 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f179d7fa640 1 -- 172.21.15.52:0/1474182675 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17ac015070 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f179d7fa640 1 -- 172.21.15.52:0/1474182675 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f17ac011440 con 0x55ee4be72f80 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 10 client.17502 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 20 client.17502 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4be83270 0x55ee4be85660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 4 client.17502 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:18:47.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee4bddff10 con 0x55ee4be83270 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.669+0000 7f17b9d25640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4be83270 0x55ee4be85660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.670+0000 7f179d7fa640 10 client.17502 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.670+0000 7f17b9d25640 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4be83270 0x55ee4be85660 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c113ade661b04d05 server_cookie=40da6958830c0634 in_seq=0 out_seq=0 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.670+0000 7f179d7fa640 1 -- 172.21.15.52:0/1474182675 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ee4bddff10 con 0x55ee4be83270 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.670+0000 7f179d7fa640 10 client.17502 handle_command_reply: tid=0 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.670+0000 7f17bed74740 1 client.17502 shutdown 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 msgr2=0x55ee4be7ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 0x55ee4be7ec70 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f17ac005b20 tx=0x7f17ac004230 comp rx=0 tx=0).stop 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 shutdown_connections 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4be83270 0x55ee4be85660 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be72f80 0x55ee4be7ec70 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 --2- 172.21.15.52:0/1474182675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f179d7fa640 0 client.17502 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:47.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 >> 172.21.15.52:0/1474182675 conn(0x55ee4be64680 msgr2=0x55ee4be637e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 shutdown_connections 2024-06-02T23:18:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 1 -- 172.21.15.52:0/1474182675 wait complete. 2024-06-02T23:18:47.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.671+0000 7f17bed74740 20 client.17502 trim_cache size 0 max 16384 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 Processor -- start 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 -- start start 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be7eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41a6af0 con 0x55ee4be6d420 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41a8630 con 0x55ee4be6de10 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17bed74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41a71c0 con 0x55ee4be72f80 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17ba526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be7eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17b9524640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17ba526640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be7eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33408/0 (socket says 172.21.15.52:33408) 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17b9d25640 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17ba526640 1 -- 172.21.15.52:0/633167873 learned_addr learned my addr 172.21.15.52:0/633167873 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17b9524640 1 -- 172.21.15.52:0/633167873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 msgr2=0x55ee4be7eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17b9524640 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be7eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17b9524640 1 -- 172.21.15.52:0/633167873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6d420 msgr2=0x55ee4be776f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.691+0000 7f17b9524640 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.692+0000 7f17b9524640 1 -- 172.21.15.52:0/633167873 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee4be65350 con 0x55ee4be6de10 2024-06-02T23:18:47.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.692+0000 7f17b9524640 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17a40cc630 tx=0x7f17a40cedb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a69161c08b26d7d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.694+0000 7f179f7fe640 1 -- 172.21.15.52:0/633167873 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a40d5070 con 0x55ee4be6de10 2024-06-02T23:18:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.694+0000 7f179f7fe640 1 -- 172.21.15.52:0/633167873 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17a409ba40 con 0x55ee4be6de10 2024-06-02T23:18:47.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.694+0000 7f179f7fe640 1 -- 172.21.15.52:0/633167873 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17a4097090 con 0x55ee4be6de10 2024-06-02T23:18:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 -- 172.21.15.52:0/633167873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6de10 msgr2=0x55ee4be77c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f17a40cc630 tx=0x7f17a40cedb0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 -- 172.21.15.52:0/633167873 shutdown_connections 2024-06-02T23:18:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be72f80 0x55ee4be7eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be6de10 0x55ee4be77c30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 --2- 172.21.15.52:0/633167873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6d420 0x55ee4be776f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 -- 172.21.15.52:0/633167873 >> 172.21.15.52:0/633167873 conn(0x55ee4be64680 msgr2=0x55ee4be64a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 -- 172.21.15.52:0/633167873 shutdown_connections 2024-06-02T23:18:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 -- 172.21.15.52:0/633167873 wait complete. 2024-06-02T23:18:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 -- start start 2024-06-02T23:18:47.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.695+0000 7f17bed74740 1 Processor -- start 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6d420 0x55ee4be7b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be7be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be72f80 0x55ee4beebdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41a7f60 con 0x55ee4be6de10 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41a7890 con 0x55ee4be72f80 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f17b41a4ad0 con 0x55ee4be6d420 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17bed74740 10 client.0 _ll_register_callbacks cb 0x7f17b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9524640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be7be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6d420 0x55ee4be7b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17ba526640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be72f80 0x55ee4beebdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9524640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be7be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47036/0 (socket says 172.21.15.52:47036) 2024-06-02T23:18:47.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9524640 1 -- 172.21.15.52:0/2513625827 learned_addr learned my addr 172.21.15.52:0/2513625827 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 -- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be72f80 msgr2=0x55ee4beebdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be72f80 0x55ee4beebdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 -- 172.21.15.52:0/2513625827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 msgr2=0x55ee4be7be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be7be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 -- 172.21.15.52:0/2513625827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee4be7eea0 con 0x55ee4be6d420 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17ba526640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be72f80 0x55ee4beebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9524640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be7be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f17b9d25640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6d420 0x55ee4be7b950 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55ee4be6e880 tx=0x7f17b0009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5cd52cb5ebb7d1f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.696+0000 7f179d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f179d7fa640 1 -- 172.21.15.52:0/2513625827 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17b0035040 con 0x55ee4be6d420 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f179d7fa640 1 -- 172.21.15.52:0/2513625827 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f17b0053480 con 0x55ee4be6d420 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f17a409ba40 con 0x55ee4be6d420 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f179d7fa640 1 -- 172.21.15.52:0/2513625827 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f17b0053680 con 0x55ee4be6d420 2024-06-02T23:18:47.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 10 client.26839 fetch_fsmap learned FSMap version 32 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ee4beed3b0 con 0x55ee4be6d420 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f179d7fa640 1 -- 172.21.15.52:0/2513625827 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f17a409ba40 con 0x55ee4be6d420 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f179d7fa640 1 -- 172.21.15.52:0/2513625827 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f17b0050050 con 0x55ee4be6d420 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 10 client.26839 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 10 client.26839 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 20 client.26839 populate_metadata read hostname 'smithi052' 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4beedd80 0x55ee4bef0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 4 client.26839 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ee4be72c80 con 0x55ee4beedd80 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.697+0000 7f17b9524640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4beedd80 0x55ee4bef0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.698+0000 7f179d7fa640 10 client.26839 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:47.701 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.698+0000 7f17b9524640 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4beedd80 0x55ee4bef0170 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2742dee6919cb37 server_cookie=289e7a8fd68788f1 in_seq=0 out_seq=0 2024-06-02T23:18:47.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.716+0000 7f179d7fa640 1 -- 172.21.15.52:0/2513625827 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ee4be72c80 con 0x55ee4beedd80 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.716+0000 7f179d7fa640 10 client.26839 handle_command_reply: tid=0 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 client.26839 shutdown 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6d420 msgr2=0x55ee4be7b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6d420 0x55ee4be7b950 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x55ee4be6e880 tx=0x7f17b0009260 comp rx=0 tx=0).stop 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 shutdown_connections 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ee4beedd80 0x55ee4bef0170 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ee4be72f80 0x55ee4beebdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f179d7fa640 0 client.26839 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ee4be6de10 0x55ee4be7be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 --2- 172.21.15.52:0/2513625827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ee4be6d420 0x55ee4be7b950 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 >> 172.21.15.52:0/2513625827 conn(0x55ee4be64680 msgr2=0x55ee4be6f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 shutdown_connections 2024-06-02T23:18:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 1 -- 172.21.15.52:0/2513625827 wait complete. 2024-06-02T23:18:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.717+0000 7f17bed74740 20 client.26839 trim_cache size 0 max 16384 2024-06-02T23:18:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:18:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:18:47.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:18:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17a00a2cb0 msgr2=0x7f17a00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17a00a2cb0 0x7f17a00a5170 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x55ee4be6e210 tx=0x7f17b0005fd0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 msgr2=0x7f17b4198100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:18:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 0x7f17b4198100 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f17a400d950 tx=0x7f17a400de10 comp rx=0 tx=0).stop 2024-06-02T23:18:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 shutdown_connections 2024-06-02T23:18:47.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f17a00a2cb0 0x7f17a00a5170 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17b41024b0 0x7f17b419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f17b4101af0 0x7f17b4198100 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 --2- 172.21.15.52:0/3782569790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f17b4100900 0x7f17b4197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:18:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 >> 172.21.15.52:0/3782569790 conn(0x7f17b40fbe70 msgr2=0x7f17b40fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:18:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 shutdown_connections 2024-06-02T23:18:47.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:18:47.718+0000 7f17bbf89640 1 -- 172.21.15.52:0/3782569790 wait complete. 2024-06-02T23:18:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:47 smithi123 ceph-mon[35977]: pgmap v638: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-02T23:18:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:47 smithi174 ceph-mon[35994]: pgmap v638: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-02T23:18:48.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:47 smithi052 ceph-mon[32340]: pgmap v638: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-02T23:18:48.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 126954 1.79 MB/sec execute 237 sec latency 16.257 ms 2024-06-02T23:18:49.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 127517 1.80 MB/sec execute 238 sec latency 15.142 ms 2024-06-02T23:18:50.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:49 smithi123 ceph-mon[35977]: pgmap v639: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-06-02T23:18:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:49 smithi174 ceph-mon[35994]: pgmap v639: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-06-02T23:18:50.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:49 smithi052 ceph-mon[32340]: pgmap v639: 97 pgs: 97 active+clean; 62 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.3 MiB/s wr, 145 op/s 2024-06-02T23:18:50.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 128000 1.81 MB/sec execute 239 sec latency 14.188 ms 2024-06-02T23:18:51.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 128276 1.80 MB/sec execute 240 sec latency 13.489 ms 2024-06-02T23:18:52.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:51 smithi123 ceph-mon[35977]: pgmap v640: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:18:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:51 smithi174 ceph-mon[35994]: pgmap v640: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:18:52.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:51 smithi052 ceph-mon[32340]: pgmap v640: 97 pgs: 97 active+clean; 64 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:18:52.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 128770 1.80 MB/sec execute 241 sec latency 14.626 ms 2024-06-02T23:18:53.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 129343 1.81 MB/sec execute 242 sec latency 11.322 ms 2024-06-02T23:18:54.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:53 smithi123 ceph-mon[35977]: pgmap v641: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-02T23:18:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:53 smithi174 ceph-mon[35994]: pgmap v641: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-02T23:18:54.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:53 smithi052 ceph-mon[32340]: pgmap v641: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-02T23:18:54.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 130022 1.81 MB/sec execute 243 sec latency 14.944 ms 2024-06-02T23:18:55.633 INFO:tasks.workunit.client.0.smithi052.stdout: 1 130520 1.82 MB/sec execute 244 sec latency 19.312 ms 2024-06-02T23:18:56.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:55 smithi123 ceph-mon[35977]: pgmap v642: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:18:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:55 smithi174 ceph-mon[35994]: pgmap v642: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:18:56.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:55 smithi052 ceph-mon[32340]: pgmap v642: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:18:56.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 131062 1.83 MB/sec execute 245 sec latency 14.689 ms 2024-06-02T23:18:57.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 131551 1.84 MB/sec execute 246 sec latency 14.661 ms 2024-06-02T23:18:58.009 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:57 smithi123 ceph-mon[35977]: pgmap v643: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 187 op/s 2024-06-02T23:18:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:57 smithi174 ceph-mon[35994]: pgmap v643: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 187 op/s 2024-06-02T23:18:58.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:57 smithi052 ceph-mon[32340]: pgmap v643: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 1.4 MiB/s wr, 187 op/s 2024-06-02T23:18:58.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 131823 1.83 MB/sec execute 247 sec latency 13.521 ms 2024-06-02T23:18:59.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 132313 1.83 MB/sec execute 248 sec latency 14.644 ms 2024-06-02T23:19:00.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:18:59 smithi123 ceph-mon[35977]: pgmap v644: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:19:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:18:59 smithi174 ceph-mon[35994]: pgmap v644: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:19:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:18:59 smithi052 ceph-mon[32340]: pgmap v644: 97 pgs: 97 active+clean; 65 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 240 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:19:00.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 132809 1.84 MB/sec execute 249 sec latency 84.258 ms 2024-06-02T23:19:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1065397622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 msgr2=0x7fbbd8101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1065397622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8101f60 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc009900 tx=0x7fbbcc02a410 comp rx=0 tx=0).stop 2024-06-02T23:19:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1065397622 shutdown_connections 2024-06-02T23:19:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1065397622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbd81024a0 0x7fbbd8108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1065397622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8101f60 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1065397622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81008f0 0x7fbbd8100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1065397622 >> 172.21.15.52:0/1065397622 conn(0x7fbbd80fbe80 msgr2=0x7fbbd80fe2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1065397622 shutdown_connections 2024-06-02T23:19:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.820+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1065397622 wait complete. 2024-06-02T23:19:00.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.821+0000 7fbbdd04c640 1 Processor -- start 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.821+0000 7fbbdd04c640 1 -- start start 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbdd04c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbd81008f0 0x7fbbd8196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbdd04c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbdd04c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81024a0 0x7fbbd819dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbdd04c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd810c420 con 0x7fbbd8101ae0 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbdd04c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd810c2a0 con 0x7fbbd81024a0 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbdd04c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd810c5a0 con 0x7fbbd81008f0 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44630/0 (socket says 172.21.15.52:44630) 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 -- 172.21.15.52:0/1892102744 learned_addr learned my addr 172.21.15.52:0/1892102744 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbd81008f0 0x7fbbd8196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81024a0 0x7fbbd819dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 -- 172.21.15.52:0/1892102744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbd81008f0 msgr2=0x7fbbd8196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbd81008f0 0x7fbbd8196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 -- 172.21.15.52:0/1892102744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81024a0 msgr2=0x7fbbd819dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81024a0 0x7fbbd819dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 -- 172.21.15.52:0/1892102744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd819e650 con 0x7fbbd8101ae0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81024a0 0x7fbbd819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.822+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8196f70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc005940 tx=0x7fbbcc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.823+0000 7fbbbffff640 1 -- 172.21.15.52:0/1892102744 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc03f030 con 0x7fbbd8101ae0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.823+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd819e8e0 con 0x7fbbd8101ae0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.823+0000 7fbbbffff640 1 -- 172.21.15.52:0/1892102744 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbcc0318f0 con 0x7fbbd8101ae0 2024-06-02T23:19:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.823+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd819ee20 con 0x7fbbd8101ae0 2024-06-02T23:19:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.823+0000 7fbbbffff640 1 -- 172.21.15.52:0/1892102744 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc049060 con 0x7fbbd8101ae0 2024-06-02T23:19:00.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.825+0000 7fbbbffff640 1 -- 172.21.15.52:0/1892102744 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbbcc042030 con 0x7fbbd8101ae0 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.825+0000 7fbbdfe30740 1 Processor -- start 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 -- start start 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939bc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939bf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5567939184f0 con 0x5567939b7220 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5567938b6870 con 0x5567939bcd80 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5567939144b0 con 0x5567939b7c10 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44634/0 (socket says 172.21.15.52:44634) 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 -- 172.21.15.52:0/1523677830 learned_addr learned my addr 172.21.15.52:0/1523677830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939bf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbbffff640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbc00a2c30 0x7fbbc00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 -- 172.21.15.52:0/1523677830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 msgr2=0x5567939bc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939bc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 -- 172.21.15.52:0/1523677830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 msgr2=0x5567939bf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939bf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 -- 172.21.15.52:0/1523677830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567938d29b0 con 0x5567939b7220 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbbffff640 1 -- 172.21.15.52:0/1892102744 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbbcc0ef890 con 0x7fbbd8101ae0 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.826+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbc00a2c30 0x7fbbc00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.827+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b7640 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0010620 tx=0x7fbbd0010ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9806ea997e0a01 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.827+0000 7fbbbd7fa640 1 -- 172.21.15.52:0/1523677830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0019070 con 0x5567939b7220 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.827+0000 7fbbbd7fa640 1 -- 172.21.15.52:0/1523677830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd0022070 con 0x5567939b7220 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.827+0000 7fbbbd7fa640 1 -- 172.21.15.52:0/1523677830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd0026c10 con 0x5567939b7220 2024-06-02T23:19:00.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbc00a2c30 0x7fbbc00a50f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0021000 tx=0x7fbbd001f000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1523677830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 msgr2=0x5567939b7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b7640 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0010620 tx=0x7fbbd0010ae0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1523677830 shutdown_connections 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939bc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939bf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1523677830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b7640 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1523677830 >> 172.21.15.52:0/1523677830 conn(0x5567939b2890 msgr2=0x5567939b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1523677830 shutdown_connections 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1523677830 wait complete. 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- start start 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 Processor -- start 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939bcd80 0x5567939caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5567939b0350 con 0x5567939b7220 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5567938a1ae0 con 0x5567939b7c10 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.828+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5567939b04d0 con 0x5567939bcd80 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbdfe30740 10 client.0 _ll_register_callbacks cb 0x5567938da9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53174/0 (socket says 172.21.15.52:53174) 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 -- 172.21.15.52:0/1794673306 learned_addr learned my addr 172.21.15.52:0/1794673306 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939bcd80 0x5567939caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 -- 172.21.15.52:0/1794673306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939bcd80 msgr2=0x5567939caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939bcd80 0x5567939caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 -- 172.21.15.52:0/1794673306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 msgr2=0x5567939c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 -- 172.21.15.52:0/1794673306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55679362c210 con 0x5567939b7c10 2024-06-02T23:19:00.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd759e640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939bcd80 0x5567939caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbd659c640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c8510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0b8580 tx=0x7fbbcc009620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f22a630ab0f2cb12 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbbaff5640 1 -- 172.21.15.52:0/1794673306 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc035040 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbbaff5640 1 -- 172.21.15.52:0/1794673306 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbbcc0b8790 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbd0022070 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbbaff5640 1 -- 172.21.15.52:0/1794673306 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbbcc0bc820 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbdfe30740 10 client.27011 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.829+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5567936f8620 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbbaff5640 1 -- 172.21.15.52:0/1794673306 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbd0022070 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbbaff5640 1 -- 172.21.15.52:0/1794673306 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbbcc0c00d0 con 0x5567939b7c10 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbdfe30740 10 client.27011 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbdfe30740 10 client.27011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbdfe30740 20 client.27011 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:00.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5567939cf4a0 0x5567939d1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbdfe30740 4 client.27011 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55679392bf10 con 0x5567939cf4a0 2024-06-02T23:19:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5567939cf4a0 0x5567939d1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbbaff5640 10 client.27011 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:00.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.830+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5567939cf4a0 0x5567939d1940 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87eeeece8f4a1e0 server_cookie=d4ab9fcf4c66cb1e in_seq=0 out_seq=0 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbbaff5640 1 -- 172.21.15.52:0/1794673306 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55679392bf10 con 0x5567939cf4a0 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbbaff5640 10 client.27011 handle_command_reply: tid=0 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 client.27011 shutdown 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 msgr2=0x5567939c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c8510 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0b8580 tx=0x7fbbcc009620 comp rx=0 tx=0).stop 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 shutdown_connections 2024-06-02T23:19:00.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5567939cf4a0 0x5567939d1940 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939bcd80 0x5567939caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c8510 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/1794673306 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 >> 172.21.15.52:0/1794673306 conn(0x5567939b2890 msgr2=0x5567939af7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbbaff5640 0 client.27011 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 shutdown_connections 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 1 -- 172.21.15.52:0/1794673306 wait complete. 2024-06-02T23:19:00.838 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.832+0000 7fbbdfe30740 20 client.27011 trim_cache size 0 max 16384 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.847+0000 7fbbdfe30740 1 Processor -- start 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.847+0000 7fbbdfe30740 1 -- start start 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939b0e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939b1350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81a6a50 con 0x5567939b7220 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81a8590 con 0x5567939bcd80 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81a7120 con 0x5567939b7c10 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44664/0 (socket says 172.21.15.52:44664) 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 -- 172.21.15.52:0/2225825889 learned_addr learned my addr 172.21.15.52:0/2225825889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd759e640 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939b1350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 -- 172.21.15.52:0/2225825889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 msgr2=0x5567939b0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd659c640 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939b0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939b0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 -- 172.21.15.52:0/2225825889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 msgr2=0x5567939b1350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939b1350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 -- 172.21.15.52:0/2225825889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567939abfe0 con 0x5567939b7220 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b08d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x5567939b8010 tx=0x7fbbc80125d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=900e2143df1dc8f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbbcff9640 1 -- 172.21.15.52:0/2225825889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc8004250 con 0x5567939b7220 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbbcff9640 1 -- 172.21.15.52:0/2225825889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc80043f0 con 0x5567939b7220 2024-06-02T23:19:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.848+0000 7fbbbcff9640 1 -- 172.21.15.52:0/2225825889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc80035f0 con 0x5567939b7220 2024-06-02T23:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 -- 172.21.15.52:0/2225825889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 msgr2=0x5567939b08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b08d0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x5567939b8010 tx=0x7fbbc80125d0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 -- 172.21.15.52:0/2225825889 shutdown_connections 2024-06-02T23:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7c10 0x5567939b0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939bcd80 0x5567939b1350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/2225825889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939b7220 0x5567939b08d0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 -- 172.21.15.52:0/2225825889 >> 172.21.15.52:0/2225825889 conn(0x5567939b2890 msgr2=0x5567939b5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 -- 172.21.15.52:0/2225825889 shutdown_connections 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 -- 172.21.15.52:0/2225825889 wait complete. 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.849+0000 7fbbdfe30740 1 -- start start 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 Processor -- start 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7220 0x5567939c6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939bcd80 0x556793a38210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81a7ec0 con 0x5567939bcd80 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81a77f0 con 0x5567939b7c10 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd81a4a30 con 0x5567939b7220 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbdfe30740 10 client.0 _ll_register_callbacks cb 0x7fbbcc0bbfe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939bcd80 0x556793a38210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd759e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939bcd80 0x556793a38210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44680/0 (socket says 172.21.15.52:44680) 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd759e640 1 -- 172.21.15.52:0/3365404410 learned_addr learned my addr 172.21.15.52:0/3365404410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7220 0x5567939c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 -- 172.21.15.52:0/3365404410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7220 msgr2=0x5567939c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7220 0x5567939c6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 -- 172.21.15.52:0/3365404410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939bcd80 msgr2=0x556793a38210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939bcd80 0x556793a38210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 -- 172.21.15.52:0/3365404410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556793a38750 con 0x5567939b7c10 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbd659c640 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c6560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0bc0b0 tx=0x7fbbcc0c4f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e53fa725d6f23ea0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.850+0000 7fbbbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbbaff5640 1 -- 172.21.15.52:0/3365404410 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbcc035070 con 0x5567939b7c10 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbbaff5640 1 -- 172.21.15.52:0/3365404410 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbbcc0c4b00 con 0x5567939b7c10 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbc80043f0 con 0x5567939b7c10 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbbaff5640 1 -- 172.21.15.52:0/3365404410 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbbcc0c04d0 con 0x5567939b7c10 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 10 client.27017 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:00.855 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556793a39a10 con 0x5567939b7c10 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbbaff5640 1 -- 172.21.15.52:0/3365404410 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc80043f0 con 0x5567939b7c10 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbbaff5640 1 -- 172.21.15.52:0/3365404410 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbbcc0b8030 con 0x5567939b7c10 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 10 client.27017 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 10 client.27017 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 20 client.27017 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556793a3a230 0x556793a3c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 4 client.27017 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556793925c20 con 0x556793a3a230 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.851+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556793a3a230 0x556793a3c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.852+0000 7fbbd6d9d640 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556793a3a230 0x556793a3c650 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e14c0f458bd0b3d3 server_cookie=6676a58bd503ee2b in_seq=0 out_seq=0 2024-06-02T23:19:00.856 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.852+0000 7fbbbaff5640 10 client.27017 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.877+0000 7fbbbaff5640 1 -- 172.21.15.52:0/3365404410 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556793925c20 con 0x556793a3a230 2024-06-02T23:19:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.877+0000 7fbbbaff5640 10 client.27017 handle_command_reply: tid=0 2024-06-02T23:19:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.877+0000 7fbbdfe30740 1 client.27017 shutdown 2024-06-02T23:19:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.877+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 msgr2=0x5567939c6560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.877+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c6560 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0bc0b0 tx=0x7fbbcc0c4f10 comp rx=0 tx=0).stop 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 shutdown_connections 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556793a3a230 0x556793a3c650 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5567939bcd80 0x556793a38210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5567939b7c10 0x5567939c6560 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbbaff5640 0 client.27017 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 --2- 172.21.15.52:0/3365404410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5567939b7220 0x5567939c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 >> 172.21.15.52:0/3365404410 conn(0x5567939b2890 msgr2=0x5567939aef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 shutdown_connections 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 1 -- 172.21.15.52:0/3365404410 wait complete. 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdfe30740 20 client.27017 trim_cache size 0 max 16384 2024-06-02T23:19:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbc00a2c30 msgr2=0x7fbbc00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbc00a2c30 0x7fbbc00a50f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0021000 tx=0x7fbbd001f000 comp rx=0 tx=0).stop 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 msgr2=0x7fbbd8196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8196f70 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc005940 tx=0x7fbbcc0042f0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 shutdown_connections 2024-06-02T23:19:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbc00a2c30 0x7fbbc00a50f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbd81024a0 0x7fbbd819dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbd8101ae0 0x7fbbd8196f70 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 --2- 172.21.15.52:0/1892102744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbd81008f0 0x7fbbd8196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.878+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 >> 172.21.15.52:0/1892102744 conn(0x7fbbd80fbe80 msgr2=0x7fbbd8104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.879+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 shutdown_connections 2024-06-02T23:19:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:00.879+0000 7fbbdd04c640 1 -- 172.21.15.52:0/1892102744 wait complete. 2024-06-02T23:19:01.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 133464 1.83 MB/sec execute 250 sec latency 16.711 ms 2024-06-02T23:19:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:01 smithi123 ceph-mon[35977]: pgmap v645: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:19:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:01 smithi174 ceph-mon[35994]: pgmap v645: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:19:02.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:01 smithi052 ceph-mon[32340]: pgmap v645: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:19:02.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 134035 1.85 MB/sec execute 251 sec latency 14.785 ms 2024-06-02T23:19:03.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 134605 1.85 MB/sec execute 252 sec latency 15.476 ms 2024-06-02T23:19:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:03 smithi123 ceph-mon[35977]: pgmap v646: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:19:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:03 smithi174 ceph-mon[35994]: pgmap v646: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:19:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:03 smithi052 ceph-mon[32340]: pgmap v646: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:19:04.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 135082 1.86 MB/sec execute 253 sec latency 14.979 ms 2024-06-02T23:19:05.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 135333 1.86 MB/sec execute 254 sec latency 15.452 ms 2024-06-02T23:19:05.783 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:19:06.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:05 smithi123 ceph-mon[35977]: pgmap v647: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-02T23:19:06.051 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:05 smithi174 ceph-mon[35994]: pgmap v647: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-02T23:19:06.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:05 smithi052 ceph-mon[32340]: pgmap v647: 97 pgs: 97 active+clean; 67 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 197 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-02T23:19:06.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 135765 1.85 MB/sec execute 255 sec latency 14.950 ms 2024-06-02T23:19:07.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 136331 1.86 MB/sec execute 256 sec latency 15.731 ms 2024-06-02T23:19:07.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:07 smithi052 ceph-mon[32340]: pgmap v648: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-06-02T23:19:08.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:07 smithi123 ceph-mon[35977]: pgmap v648: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-06-02T23:19:08.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 -- 172.21.15.52:0/1772500117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 msgr2=0x7f54e006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 --2- 172.21.15.52:0/1772500117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e006a5f0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 -- 172.21.15.52:0/1772500117 shutdown_connections 2024-06-02T23:19:08.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 --2- 172.21.15.52:0/1772500117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e006ab30 0x7f54e010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 --2- 172.21.15.52:0/1772500117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e0108500 0x7f54e01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 --2- 172.21.15.52:0/1772500117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e006a5f0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 -- 172.21.15.52:0/1772500117 >> 172.21.15.52:0/1772500117 conn(0x7f54e0078340 msgr2=0x7f54e0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:08.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 -- 172.21.15.52:0/1772500117 shutdown_connections 2024-06-02T23:19:08.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.008+0000 7f54e6161700 1 -- 172.21.15.52:0/1772500117 wait complete. 2024-06-02T23:19:08.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 Processor -- start 2024-06-02T23:19:08.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 -- start start 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e0101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e006ab30 0x7f54e01019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e0108500 0x7f54e0101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0112630 con 0x7f54e006a170 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01124b0 con 0x7f54e006ab30 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54e6161700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01127b0 con 0x7f54e0108500 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e0101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e0101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44198/0 (socket says 172.21.15.52:44198) 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54df7fe700 1 -- 172.21.15.52:0/1042191965 learned_addr learned my addr 172.21.15.52:0/1042191965 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54dffff700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e0108500 0x7f54e0101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.009+0000 7f54deffd700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e006ab30 0x7f54e01019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54df7fe700 1 -- 172.21.15.52:0/1042191965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e0108500 msgr2=0x7f54e0101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e0108500 0x7f54e0101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54df7fe700 1 -- 172.21.15.52:0/1042191965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e006ab30 msgr2=0x7f54e01019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e006ab30 0x7f54e01019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54df7fe700 1 -- 172.21.15.52:0/1042191965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01a7090 con 0x7f54e006a170 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54dffff700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e0108500 0x7f54e0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e0101460 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e9a0 tx=0x7f54d800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54dcff9700 1 -- 172.21.15.52:0/1042191965 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d800cbd0 con 0x7f54e006a170 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a7380 con 0x7f54e006a170 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54dcff9700 1 -- 172.21.15.52:0/1042191965 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d800cd70 con 0x7f54e006a170 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54dcff9700 1 -- 172.21.15.52:0/1042191965 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8018680 con 0x7f54e006a170 2024-06-02T23:19:08.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.010+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a78e0 con 0x7f54e006a170 2024-06-02T23:19:08.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.013+0000 7f54dcff9700 1 -- 172.21.15.52:0/1042191965 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f54d8010430 con 0x7f54e006a170 2024-06-02T23:19:08.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.014+0000 7f54dcff9700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54c80a2ca0 0x7f54c80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.014+0000 7f54deffd700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54c80a2ca0 0x7f54c80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.014+0000 7f54dcff9700 1 -- 172.21.15.52:0/1042191965 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54d80cf180 con 0x7f54e006a170 2024-06-02T23:19:08.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54deffd700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54c80a2ca0 0x7f54c80a5160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b2e0 tx=0x7f54d4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:08.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 Processor -- start 2024-06-02T23:19:08.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 -- start start 2024-06-02T23:19:08.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5b640 0x556ba5d5ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5bf60 0x556ba5d559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 0x556ba5d56340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556ba5ca7810 con 0x556ba5d5b640 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556ba5c4f720 con 0x556ba5d55f20 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ba5ca37d0 con 0x556ba5d5bf60 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 0x556ba5d56340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 0x556ba5d56340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36728/0 (socket says 172.21.15.52:36728) 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54dffff700 1 -- 172.21.15.52:0/442855297 learned_addr learned my addr 172.21.15.52:0/442855297 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54df7fe700 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5b640 0x556ba5d5ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.015+0000 7f54deffd700 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5bf60 0x556ba5d559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54dffff700 1 -- 172.21.15.52:0/442855297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5bf60 msgr2=0x556ba5d559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54dffff700 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5bf60 0x556ba5d559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54dffff700 1 -- 172.21.15.52:0/442855297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5b640 msgr2=0x556ba5d5ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54dffff700 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5b640 0x556ba5d5ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54dffff700 1 -- 172.21.15.52:0/442855297 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba5c45f50 con 0x556ba5d55f20 2024-06-02T23:19:08.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54dffff700 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 0x556ba5d56340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009e60 tx=0x7f54d00077e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ccb11269c7afdff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:08.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54c5ffb700 1 -- 172.21.15.52:0/442855297 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0014610 con 0x556ba5d55f20 2024-06-02T23:19:08.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54c5ffb700 1 -- 172.21.15.52:0/442855297 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d0019070 con 0x556ba5d55f20 2024-06-02T23:19:08.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.016+0000 7f54c5ffb700 1 -- 172.21.15.52:0/442855297 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0013a40 con 0x556ba5d55f20 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 -- 172.21.15.52:0/442855297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 msgr2=0x556ba5d56340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 0x556ba5d56340 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009e60 tx=0x7f54d00077e0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 -- 172.21.15.52:0/442855297 shutdown_connections 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d55f20 0x556ba5d56340 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5b640 0x556ba5d5ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/442855297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5bf60 0x556ba5d559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 -- 172.21.15.52:0/442855297 >> 172.21.15.52:0/442855297 conn(0x556ba5c80cc0 msgr2=0x556ba5d50850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 -- 172.21.15.52:0/442855297 shutdown_connections 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.017+0000 7f54f09e4b80 1 -- 172.21.15.52:0/442855297 wait complete. 2024-06-02T23:19:08.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 -- start start 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 Processor -- start 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d61cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d5f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556ba5d629d0 con 0x556ba5d5bf60 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556ba5c47bf0 con 0x556ba5d5b640 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556ba5d62b50 con 0x556ba5d55f20 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54f09e4b80 10 client.0 _ll_register_callbacks cb 0x556ba5c41920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54deffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d5f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54deffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36738/0 (socket says 172.21.15.52:36738) 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54deffd700 1 -- 172.21.15.52:0/1759667236 learned_addr learned my addr 172.21.15.52:0/1759667236 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.018+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d61cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54deffd700 1 -- 172.21.15.52:0/1759667236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 msgr2=0x556ba5d61cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54deffd700 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d61cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54deffd700 1 -- 172.21.15.52:0/1759667236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 msgr2=0x556ba5d5f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54deffd700 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d5f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54deffd700 1 -- 172.21.15.52:0/1759667236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba5bf68d0 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54deffd700 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5eff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674d3764dfac22c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54beffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54beffd700 1 -- 172.21.15.52:0/1759667236 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d4035060 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54beffd700 1 -- 172.21.15.52:0/1759667236 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54d400f460 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d0019070 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.019+0000 7f54beffd700 1 -- 172.21.15.52:0/1759667236 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54d400f6a0 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 10 client.27029 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556ba5d6d2f0 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54beffd700 1 -- 172.21.15.52:0/1759667236 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d0019070 con 0x556ba5d5b640 2024-06-02T23:19:08.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54beffd700 1 -- 172.21.15.52:0/1759667236 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f54d40265f0 con 0x556ba5d5b640 2024-06-02T23:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 10 client.27029 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:08.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 10 client.27029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 20 client.27029 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5d6f930 0x556ba5d71d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 4 client.27029 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ba5cb4d10 con 0x556ba5d6f930 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.020+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5d6f930 0x556ba5d71d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.021+0000 7f54beffd700 10 client.27029 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:08.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.021+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5d6f930 0x556ba5d71d20 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=381452419bf80841 server_cookie=f7bbc17f304dfd9b in_seq=0 out_seq=0 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.021+0000 7f54beffd700 1 -- 172.21.15.52:0/1759667236 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x556ba5cb4d10 con 0x556ba5d6f930 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.021+0000 7f54beffd700 10 client.27029 handle_command_reply: tid=0 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.021+0000 7f54f09e4b80 1 client.27029 shutdown 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 msgr2=0x556ba5d5eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5eff0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f54d4009990 tx=0x7f54d4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 shutdown_connections 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5d6f930 0x556ba5d71d20 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d5f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54beffd700 0 client.27029 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d61cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1759667236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5eff0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 >> 172.21.15.52:0/1759667236 conn(0x556ba5c80cc0 msgr2=0x556ba5d549d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 shutdown_connections 2024-06-02T23:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1759667236 wait complete. 2024-06-02T23:19:08.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.022+0000 7f54f09e4b80 20 client.27029 trim_cache size 0 max 16384 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 Processor -- start 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 -- start start 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d559e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d4a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01aec80 con 0x556ba5d5bf60 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b07c0 con 0x556ba5d5b640 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.049+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01af350 con 0x556ba5d55f20 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d4a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54deffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54dffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d4a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44224/0 (socket says 172.21.15.52:44224) 2024-06-02T23:19:08.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54dffff700 1 -- 172.21.15.52:0/649856316 learned_addr learned my addr 172.21.15.52:0/649856316 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:08.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:58402/0 (socket says 172.21.15.52:58402) 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 -- 172.21.15.52:0/649856316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 msgr2=0x556ba5d5df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 -- 172.21.15.52:0/649856316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 msgr2=0x556ba5d4a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d4a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 -- 172.21.15.52:0/649856316 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ba5d5d560 con 0x556ba5d55f20 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54dffff700 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d4a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54df7fe700 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d559e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54d80ccd00 tx=0x7f54d80ccb80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dc357bf905efdaf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.050+0000 7f54c57fa700 1 -- 172.21.15.52:0/649856316 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d80ce040 con 0x556ba5d55f20 2024-06-02T23:19:08.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54c57fa700 1 -- 172.21.15.52:0/649856316 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d809c110 con 0x556ba5d55f20 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54c57fa700 1 -- 172.21.15.52:0/649856316 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8099d10 con 0x556ba5d55f20 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 -- 172.21.15.52:0/649856316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 msgr2=0x556ba5d559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d559e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54d80ccd00 tx=0x7f54d80ccb80 comp rx=0 tx=0).stop 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 -- 172.21.15.52:0/649856316 shutdown_connections 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d5bf60 0x556ba5d4a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d55f20 0x556ba5d559e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/649856316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5b640 0x556ba5d5df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 -- 172.21.15.52:0/649856316 >> 172.21.15.52:0/649856316 conn(0x556ba5c80cc0 msgr2=0x556ba5d5c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 -- 172.21.15.52:0/649856316 shutdown_connections 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.051+0000 7f54f09e4b80 1 -- 172.21.15.52:0/649856316 wait complete. 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 -- start start 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 Processor -- start 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 0x556ba5d68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5b640 0x556ba5d68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5bf60 0x556ba5d64e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b00f0 con 0x556ba5d55f20 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01afa20 con 0x556ba5d5bf60 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01acc40 con 0x556ba5d5b640 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54f09e4b80 10 client.0 _ll_register_callbacks cb 0x556ba5c41920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 0x556ba5d68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54df7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 0x556ba5d68020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44236/0 (socket says 172.21.15.52:44236) 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54df7fe700 1 -- 172.21.15.52:0/1642484323 learned_addr learned my addr 172.21.15.52:0/1642484323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54dffff700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5bf60 0x556ba5d64e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.052+0000 7f54deffd700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5b640 0x556ba5d68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54df7fe700 1 -- 172.21.15.52:0/1642484323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5b640 msgr2=0x556ba5d68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5b640 0x556ba5d68560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54df7fe700 1 -- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5bf60 msgr2=0x556ba5d64e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:08.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5bf60 0x556ba5d64e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54df7fe700 1 -- 172.21.15.52:0/1642484323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ba5d654c0 con 0x556ba5d55f20 2024-06-02T23:19:08.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54df7fe700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 0x556ba5d68020 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f54d800c5b0 tx=0x7f54d8005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44dd63b5323da5f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:08.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54beffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:08.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54beffd700 1 -- 172.21.15.52:0/1642484323 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d80b9060 con 0x556ba5d55f20 2024-06-02T23:19:08.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54beffd700 1 -- 172.21.15.52:0/1642484323 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54d8099d10 con 0x556ba5d55f20 2024-06-02T23:19:08.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d809c110 con 0x556ba5d55f20 2024-06-02T23:19:08.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54beffd700 1 -- 172.21.15.52:0/1642484323 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f54d80ce070 con 0x556ba5d55f20 2024-06-02T23:19:08.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54f09e4b80 10 client.17544 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x556ba5ddaf40 con 0x556ba5d55f20 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.053+0000 7f54beffd700 1 -- 172.21.15.52:0/1642484323 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f54d809c110 con 0x556ba5d55f20 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54beffd700 1 -- 172.21.15.52:0/1642484323 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f54d80a41f0 con 0x556ba5d55f20 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 10 client.17544 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 20 client.17544 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5ddb580 0x556ba5ddd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 4 client.17544 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556ba5caf290 con 0x556ba5ddb580 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54deffd700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5ddb580 0x556ba5ddd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54beffd700 10 client.17544 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54deffd700 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5ddb580 0x556ba5ddd9a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad528ce4d7f8a82c server_cookie=671f5060d5304f6a in_seq=0 out_seq=0 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54beffd700 1 -- 172.21.15.52:0/1642484323 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556ba5caf290 con 0x556ba5ddb580 2024-06-02T23:19:08.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54beffd700 10 client.17544 handle_command_reply: tid=0 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.054+0000 7f54f09e4b80 1 client.17544 shutdown 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 msgr2=0x556ba5d68020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 0x556ba5d68020 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f54d800c5b0 tx=0x7f54d8005d50 comp rx=0 tx=0).stop 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 shutdown_connections 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x556ba5ddb580 0x556ba5ddd9a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556ba5d5bf60 0x556ba5d64e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556ba5d55f20 0x556ba5d68020 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 --2- 172.21.15.52:0/1642484323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556ba5d5b640 0x556ba5d68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 >> 172.21.15.52:0/1642484323 conn(0x556ba5c80cc0 msgr2=0x556ba5d5a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54beffd700 0 client.17544 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 shutdown_connections 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 1 -- 172.21.15.52:0/1642484323 wait complete. 2024-06-02T23:19:08.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.055+0000 7f54f09e4b80 20 client.17544 trim_cache size 0 max 16384 2024-06-02T23:19:08.061 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:19:08.061 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:19:08.061 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:19:08.061 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54c80a2ca0 msgr2=0x7f54c80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54c80a2ca0 0x7f54c80a5160 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f54d400b2e0 tx=0x7f54d4005960 comp rx=0 tx=0).stop 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 msgr2=0x7f54e0101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e0101460 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f54d800e9a0 tx=0x7f54d800ecb0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 shutdown_connections 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54c80a2ca0 0x7f54c80a5160 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f54e0108500 0x7f54e0101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54e006a170 0x7f54e0101460 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 --2- 172.21.15.52:0/1042191965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e006ab30 0x7f54e01019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:08.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 >> 172.21.15.52:0/1042191965 conn(0x7f54e0078340 msgr2=0x7f54e01000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:08.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 shutdown_connections 2024-06-02T23:19:08.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:08.056+0000 7f54e6161700 1 -- 172.21.15.52:0/1042191965 wait complete. 2024-06-02T23:19:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:07 smithi174 ceph-mon[35994]: pgmap v648: 97 pgs: 97 active+clean; 68 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.5 MiB/s wr, 183 op/s 2024-06-02T23:19:08.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 136912 1.86 MB/sec execute 257 sec latency 64.334 ms 2024-06-02T23:19:09.167 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:19:09.167 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:19:09.449 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:09.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 137402 1.86 MB/sec execute 258 sec latency 15.505 ms 2024-06-02T23:19:09.823 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:09 smithi052 ceph-mon[32340]: pgmap v649: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-06-02T23:19:10.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:09 smithi123 ceph-mon[35977]: pgmap v649: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-06-02T23:19:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:09 smithi174 ceph-mon[35994]: pgmap v649: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.3 MiB/s wr, 144 op/s 2024-06-02T23:19:10.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 138124 1.88 MB/sec execute 259 sec latency 17.821 ms 2024-06-02T23:19:10.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:10 smithi052 ceph-mon[32340]: pgmap v650: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-06-02T23:19:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:10 smithi174 ceph-mon[35994]: pgmap v650: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-06-02T23:19:11.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.248+0000 7f35c65a9700 1 -- 172.21.15.52:0/4261698770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0108510 msgr2=0x7f35c01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.248+0000 7f35c65a9700 1 --2- 172.21.15.52:0/4261698770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0108510 0x7f35c01088f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f35b0009990 tx=0x7f35b0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 -- 172.21.15.52:0/4261698770 shutdown_connections 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 --2- 172.21.15.52:0/4261698770 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c0101440 0x7f35c010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 --2- 172.21.15.52:0/4261698770 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0108510 0x7f35c01088f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 --2- 172.21.15.52:0/4261698770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0100a80 0x7f35c0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 -- 172.21.15.52:0/4261698770 >> 172.21.15.52:0/4261698770 conn(0x7f35c0077a00 msgr2=0x7f35c00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 -- 172.21.15.52:0/4261698770 shutdown_connections 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 -- 172.21.15.52:0/4261698770 wait complete. 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.249+0000 7f35c65a9700 1 Processor -- start 2024-06-02T23:19:11.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 -- start start 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 0x7f35c019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0101440 0x7f35c019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c0108510 0x7f35c0198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35c0112620 con 0x7f35c0100a80 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01124a0 con 0x7f35c0101440 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c65a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35c01127a0 con 0x7f35c0108510 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 0x7f35c019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:11.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 0x7f35c019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44254/0 (socket says 172.21.15.52:44254) 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 -- 172.21.15.52:0/825129775 learned_addr learned my addr 172.21.15.52:0/825129775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35b7fff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0101440 0x7f35c019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35c4b46700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c0108510 0x7f35c0198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 -- 172.21.15.52:0/825129775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c0108510 msgr2=0x7f35c0198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c0108510 0x7f35c0198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 -- 172.21.15.52:0/825129775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0101440 msgr2=0x7f35c019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0101440 0x7f35c019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35bffff700 1 -- 172.21.15.52:0/825129775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c01990c0 con 0x7f35c0100a80 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.250+0000 7f35b7fff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0101440 0x7f35c019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.251+0000 7f35bffff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 0x7f35c019e7a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000c00 tx=0x7f35b0004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.251+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b0027070 con 0x7f35c0100a80 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.251+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35b0022470 con 0x7f35c0100a80 2024-06-02T23:19:11.253 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.251+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35b000b610 con 0x7f35c0100a80 2024-06-02T23:19:11.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.251+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c0199350 con 0x7f35c0100a80 2024-06-02T23:19:11.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.251+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c01a5670 con 0x7f35c0100a80 2024-06-02T23:19:11.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.252+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c010a9f0 con 0x7f35c0100a80 2024-06-02T23:19:11.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.254+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35b002a070 con 0x7f35c0100a80 2024-06-02T23:19:11.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.254+0000 7f35bdffb700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2ca0 0x7f35ac0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:11.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.255+0000 7f35b7fff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2ca0 0x7f35ac0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:11.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.255+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f35b00d00a0 con 0x7f35c0100a80 2024-06-02T23:19:11.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.255+0000 7f35b7fff700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2ca0 0x7f35ac0a5160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f35c0107c80 tx=0x7f35a8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:11.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:10 smithi123 ceph-mon[35977]: pgmap v650: 97 pgs: 97 active+clean; 69 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.4 MiB/s wr, 174 op/s 2024-06-02T23:19:11.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.259+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f35b0013070 con 0x7f35c0100a80 2024-06-02T23:19:11.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.533+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35c0069180 con 0x7f35c0100a80 2024-06-02T23:19:11.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.534+0000 7f35bdffb700 1 -- 172.21.15.52:0/825129775 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f35b009d0c0 con 0x7f35c0100a80 2024-06-02T23:19:11.535 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:19:11.535 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:19:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2ca0 msgr2=0x7f35ac0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2ca0 0x7f35ac0a5160 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f35c0107c80 tx=0x7f35a8009260 comp rx=0 tx=0).stop 2024-06-02T23:19:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 msgr2=0x7f35c019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:11.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 0x7f35c019e7a0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f35b0000c00 tx=0x7f35b0004790 comp rx=0 tx=0).stop 2024-06-02T23:19:11.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 shutdown_connections 2024-06-02T23:19:11.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f35ac0a2ca0 0x7f35ac0a5160 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35c0108510 0x7f35c0198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35c0100a80 0x7f35c019e7a0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 --2- 172.21.15.52:0/825129775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35c0101440 0x7f35c019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:11.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 >> 172.21.15.52:0/825129775 conn(0x7f35c0077a00 msgr2=0x7f35c00ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:11.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 shutdown_connections 2024-06-02T23:19:11.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:11.537+0000 7f35c65a9700 1 -- 172.21.15.52:0/825129775 wait complete. 2024-06-02T23:19:11.540 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:19:11.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 138569 1.88 MB/sec execute 260 sec latency 16.670 ms 2024-06-02T23:19:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/825129775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:19:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/825129775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:19:12.204 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:19:12.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/825129775' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:19:12.478 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:12.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 138807 1.88 MB/sec execute 261 sec latency 13.270 ms 2024-06-02T23:19:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:13 smithi052 ceph-mon[32340]: pgmap v651: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:19:13.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:13 smithi123 ceph-mon[35977]: pgmap v651: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:19:13.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:13 smithi174 ceph-mon[35994]: pgmap v651: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 266 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:19:13.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 139232 1.87 MB/sec execute 262 sec latency 11.935 ms 2024-06-02T23:19:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 -- 172.21.15.52:0/3734072065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228104db0 msgr2=0x7f92281071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 --2- 172.21.15.52:0/3734072065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228104db0 0x7f92281071a0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f921c009900 tx=0x7f921c02a410 comp rx=0 tx=0).stop 2024-06-02T23:19:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 -- 172.21.15.52:0/3734072065 shutdown_connections 2024-06-02T23:19:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 --2- 172.21.15.52:0/3734072065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92281076e0 0x7f9228109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 --2- 172.21.15.52:0/3734072065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228104db0 0x7f92281071a0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 --2- 172.21.15.52:0/3734072065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9228102480 0x7f9228104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 -- 172.21.15.52:0/3734072065 >> 172.21.15.52:0/3734072065 conn(0x7f92280fbe70 msgr2=0x7f92280fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 -- 172.21.15.52:0/3734072065 shutdown_connections 2024-06-02T23:19:13.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 -- 172.21.15.52:0/3734072065 wait complete. 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 Processor -- start 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.977+0000 7f922e3c1640 1 -- start start 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922e3c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 0x7f9228197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922e3c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9228104db0 0x7f9228198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922e3c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92281076e0 0x7f922819f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922e3c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f922810d520 con 0x7f9228102480 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922e3c1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f922810d3a0 con 0x7f92281076e0 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922e3c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f922810d6a0 con 0x7f9228104db0 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 0x7f9228197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922c95e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92281076e0 0x7f922819f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 0x7f9228197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41088/0 (socket says 172.21.15.52:41088) 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 -- 172.21.15.52:0/1517270627 learned_addr learned my addr 172.21.15.52:0/1517270627 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f92277fe640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9228104db0 0x7f9228198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 -- 172.21.15.52:0/1517270627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9228104db0 msgr2=0x7f9228198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9228104db0 0x7f9228198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 -- 172.21.15.52:0/1517270627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92281076e0 msgr2=0x7f922819f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92281076e0 0x7f922819f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 -- 172.21.15.52:0/1517270627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f922819f7e0 con 0x7f9228102480 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f922c95e640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92281076e0 0x7f922819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.978+0000 7f9227fff640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 0x7f9228197bc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9f0 tx=0x7f921800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.979+0000 7f92257fa640 1 -- 172.21.15.52:0/1517270627 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921800cd80 con 0x7f9228102480 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.979+0000 7f92257fa640 1 -- 172.21.15.52:0/1517270627 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92180108e0 con 0x7f9228102480 2024-06-02T23:19:13.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.979+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f922819fad0 con 0x7f9228102480 2024-06-02T23:19:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.979+0000 7f92257fa640 1 -- 172.21.15.52:0/1517270627 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9218018920 con 0x7f9228102480 2024-06-02T23:19:13.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.979+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f922819fff0 con 0x7f9228102480 2024-06-02T23:19:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.981+0000 7f92257fa640 1 -- 172.21.15.52:0/1517270627 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9218010a90 con 0x7f9228102480 2024-06-02T23:19:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.981+0000 7f92311a9740 1 Processor -- start 2024-06-02T23:19:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 -- start start 2024-06-02T23:19:13.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de32430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de37500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de37b70 0x55c06de3a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c06dd954f0 con 0x55c06de37b70 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c06dd33870 con 0x55c06de32010 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92311a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c06dd914b0 con 0x55c06de32a00 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de32430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de32430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33668/0 (socket says 172.21.15.52:33668) 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 -- 172.21.15.52:0/3436909117 learned_addr learned my addr 172.21.15.52:0/3436909117 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92277fe640 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de37b70 0x55c06de3a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f9227fff640 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de37500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 -- 172.21.15.52:0/3436909117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 msgr2=0x55c06de37500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de37500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 -- 172.21.15.52:0/3436909117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de37b70 msgr2=0x55c06de3a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de37b70 0x55c06de3a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f92257fa640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92100a2c30 0x7f92100a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f922c95e640 1 -- 172.21.15.52:0/3436909117 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06dd4f9b0 con 0x55c06de32010 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.982+0000 7f9227fff640 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.983+0000 7f922c95e640 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de32430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f92200117e0 tx=0x7f9220011ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68ff2c8bed90386a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.983+0000 7f92257fa640 1 -- 172.21.15.52:0/1517270627 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f92180cf2e0 con 0x7f9228102480 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.983+0000 7f922c95e640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92100a2c30 0x7f92100a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.983+0000 7f920e7fc640 1 -- 172.21.15.52:0/3436909117 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220011290 con 0x55c06de32010 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.983+0000 7f920e7fc640 1 -- 172.21.15.52:0/3436909117 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9220003bd0 con 0x55c06de32010 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.983+0000 7f920e7fc640 1 -- 172.21.15.52:0/3436909117 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922001a3f0 con 0x55c06de32010 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- 172.21.15.52:0/3436909117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 msgr2=0x55c06de32430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f922c95e640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92100a2c30 0x7f92100a50f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f922001b000 tx=0x7f922001a610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de32430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f92200117e0 tx=0x7f9220011ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- 172.21.15.52:0/3436909117 shutdown_connections 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de37500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de37b70 0x55c06de3a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- 172.21.15.52:0/3436909117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de32430 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- 172.21.15.52:0/3436909117 >> 172.21.15.52:0/3436909117 conn(0x55c06de2d680 msgr2=0x55c06de2da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- 172.21.15.52:0/3436909117 shutdown_connections 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- 172.21.15.52:0/3436909117 wait complete. 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- start start 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 Processor -- start 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 0x55c06de3c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de3c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de437d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c06de3ab20 con 0x55c06de32010 2024-06-02T23:19:13.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.984+0000 7f92311a9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c06dd1eae0 con 0x55c06de37b70 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f92311a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c06de3aca0 con 0x55c06de32a00 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f92311a9740 10 client.0 _ll_register_callbacks cb 0x55c06dd579b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 0x55c06de3c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 0x55c06de3c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41120/0 (socket says 172.21.15.52:41120) 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 -- 172.21.15.52:0/2945815676 learned_addr learned my addr 172.21.15.52:0/2945815676 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f922c95e640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de437d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f92277fe640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de3c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 -- 172.21.15.52:0/2945815676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 msgr2=0x55c06de3c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de3c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 -- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 msgr2=0x55c06de437d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de437d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 -- 172.21.15.52:0/2945815676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06daa9210 con 0x55c06de32010 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f92277fe640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de3c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:13.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f9227fff640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 0x55c06de3c2c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9c0 tx=0x7f9218010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e42de9e3740387b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:13.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.985+0000 7f92077fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92077fe640 1 -- 172.21.15.52:0/2945815676 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92180a4550 con 0x55c06de32010 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92077fe640 1 -- 172.21.15.52:0/2945815676 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9218010430 con 0x55c06de32010 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9220003bd0 con 0x55c06de32010 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92077fe640 1 -- 172.21.15.52:0/2945815676 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9218010650 con 0x55c06de32010 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 10 client.17562 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c06db75620 con 0x55c06de32010 2024-06-02T23:19:13.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92077fe640 1 -- 172.21.15.52:0/2945815676 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9220003bd0 con 0x55c06de32010 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92077fe640 1 -- 172.21.15.52:0/2945815676 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9218097b70 con 0x55c06de32010 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 10 client.17562 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 10 client.17562 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 20 client.17562 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06de480a0 0x55c06de4a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 4 client.17562 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c06dda8f10 con 0x55c06de480a0 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.986+0000 7f92277fe640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06de480a0 0x55c06de4a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.987+0000 7f92277fe640 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06de480a0 0x55c06de4a4f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5aeca621616edcf server_cookie=983e1e36cb6d489b in_seq=0 out_seq=0 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.987+0000 7f92077fe640 10 client.17562 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92077fe640 1 -- 172.21.15.52:0/2945815676 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c06dda8f10 con 0x55c06de480a0 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92077fe640 10 client.17562 handle_command_reply: tid=0 2024-06-02T23:19:13.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 client.17562 shutdown 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 msgr2=0x55c06de3c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 0x55c06de3c2c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9c0 tx=0x7f9218010f20 comp rx=0 tx=0).stop 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 shutdown_connections 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06de480a0 0x55c06de4a4f0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32a00 0x55c06de3c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de437d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 --2- 172.21.15.52:0/2945815676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32010 0x55c06de3c2c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92077fe640 0 client.17562 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 >> 172.21.15.52:0/2945815676 conn(0x55c06de2d680 msgr2=0x55c06de2c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 shutdown_connections 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 1 -- 172.21.15.52:0/2945815676 wait complete. 2024-06-02T23:19:13.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:13.988+0000 7f92311a9740 20 client.17562 trim_cache size 0 max 16384 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 Processor -- start 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 -- start start 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de3bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de43790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de37b70 0x55c06de3c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92281a8d80 con 0x55c06de32a00 2024-06-02T23:19:14.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92281aa8c0 con 0x55c06de32010 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92311a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92281a9450 con 0x55c06de37b70 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de43790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de43790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41124/0 (socket says 172.21.15.52:41124) 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 -- 172.21.15.52:0/982533217 learned_addr learned my addr 172.21.15.52:0/982533217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f9227fff640 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de3bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f922c95e640 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de37b70 0x55c06de3c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 -- 172.21.15.52:0/982533217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de37b70 msgr2=0x55c06de3c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de37b70 0x55c06de3c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 -- 172.21.15.52:0/982533217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 msgr2=0x55c06de3bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de3bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.006+0000 7f92277fe640 1 -- 172.21.15.52:0/982533217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c06de2d4b0 con 0x55c06de32a00 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92277fe640 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de43790 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9228069bb0 tx=0x7f921c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d825f5b5e7c14ee server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f920dffb640 1 -- 172.21.15.52:0/982533217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c035040 con 0x55c06de32a00 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f920dffb640 1 -- 172.21.15.52:0/982533217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f921c004240 con 0x55c06de32a00 2024-06-02T23:19:14.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f920dffb640 1 -- 172.21.15.52:0/982533217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f921c046660 con 0x55c06de32a00 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 -- 172.21.15.52:0/982533217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 msgr2=0x55c06de43790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de43790 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f9228069bb0 tx=0x7f921c031850 comp rx=0 tx=0).stop 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 -- 172.21.15.52:0/982533217 shutdown_connections 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de37b70 0x55c06de3c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de43790 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 --2- 172.21.15.52:0/982533217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de32010 0x55c06de3bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.007+0000 7f92311a9740 1 -- 172.21.15.52:0/982533217 >> 172.21.15.52:0/982533217 conn(0x55c06de2d680 msgr2=0x55c06de2da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 -- 172.21.15.52:0/982533217 shutdown_connections 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 -- 172.21.15.52:0/982533217 wait complete. 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 -- start start 2024-06-02T23:19:14.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 Processor -- start 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32010 0x55c06de40910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de41e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92281aa1f0 con 0x55c06de32a00 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92281a9b20 con 0x55c06de37b70 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92281a6d60 con 0x55c06de32010 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92311a9740 10 client.0 _ll_register_callbacks cb 0x7f9218009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f92277fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f922c95e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de41e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.008+0000 7f9227fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32010 0x55c06de40910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de41e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33700/0 (socket says 172.21.15.52:33700) 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 -- 172.21.15.52:0/3781304883 learned_addr learned my addr 172.21.15.52:0/3781304883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 -- 172.21.15.52:0/3781304883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32010 msgr2=0x55c06de40910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32010 0x55c06de40910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 -- 172.21.15.52:0/3781304883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 msgr2=0x55c06de40e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de40e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 -- 172.21.15.52:0/3781304883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c06de42400 con 0x55c06de37b70 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92277fe640 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f922c95e640 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de41e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9220011ce0 tx=0x7f9220018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c58d3fe8e28bcdd5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92077fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92077fe640 1 -- 172.21.15.52:0/3781304883 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9220038070 con 0x55c06de37b70 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f921c004240 con 0x55c06de37b70 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92077fe640 1 -- 172.21.15.52:0/3781304883 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9220011210 con 0x55c06de37b70 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92077fe640 1 -- 172.21.15.52:0/3781304883 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9220044050 con 0x55c06de37b70 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92311a9740 10 client.27059 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.009+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c06de43800 con 0x55c06de37b70 2024-06-02T23:19:14.014 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92077fe640 1 -- 172.21.15.52:0/3781304883 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f921c004240 con 0x55c06de37b70 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92077fe640 1 -- 172.21.15.52:0/3781304883 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9220033040 con 0x55c06de37b70 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92311a9740 10 client.27059 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92311a9740 10 client.27059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92311a9740 20 client.27059 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92311a9740 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06deb30d0 0x55c06deb54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92311a9740 4 client.27059 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c06dda2c20 con 0x55c06deb30d0 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f9227fff640 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06deb30d0 0x55c06deb54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f92077fe640 10 client.27059 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:14.015 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.010+0000 7f9227fff640 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06deb30d0 0x55c06deb54c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=231143c078a9be6a server_cookie=87d17aa37451749f in_seq=0 out_seq=0 2024-06-02T23:19:14.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92077fe640 1 -- 172.21.15.52:0/3781304883 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c06dda2c20 con 0x55c06deb30d0 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92077fe640 10 client.27059 handle_command_reply: tid=0 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 client.27059 shutdown 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 msgr2=0x55c06de41e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de41e10 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9220011ce0 tx=0x7f9220018000 comp rx=0 tx=0).stop 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 shutdown_connections 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c06deb30d0 0x55c06deb54c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92077fe640 0 client.27059 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c06de37b70 0x55c06de41e10 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c06de32a00 0x55c06de40e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 --2- 172.21.15.52:0/3781304883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c06de32010 0x55c06de40910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 >> 172.21.15.52:0/3781304883 conn(0x55c06de2d680 msgr2=0x55c06de3bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 shutdown_connections 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 1 -- 172.21.15.52:0/3781304883 wait complete. 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.036+0000 7f92311a9740 20 client.27059 trim_cache size 0 max 16384 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:19:14.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:19:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92100a2c30 msgr2=0x7f92100a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92100a2c30 0x7f92100a50f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f922001b000 tx=0x7f922001a610 comp rx=0 tx=0).stop 2024-06-02T23:19:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 msgr2=0x7f9228197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 0x7f9228197bc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f921800e9f0 tx=0x7f921800eeb0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 shutdown_connections 2024-06-02T23:19:14.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92100a2c30 0x7f92100a50f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9228104db0 0x7f9228198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92281076e0 0x7f922819f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 --2- 172.21.15.52:0/1517270627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9228102480 0x7f9228197bc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 >> 172.21.15.52:0/1517270627 conn(0x7f92280fbe70 msgr2=0x7f9228102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 shutdown_connections 2024-06-02T23:19:14.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:14.037+0000 7f922e3c1640 1 -- 172.21.15.52:0/1517270627 wait complete. 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 -- 172.21.15.52:0/664483673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38101390 msgr2=0x7fea38101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 --2- 172.21.15.52:0/664483673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38101390 0x7fea38101810 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009990 tx=0x7fea2c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 -- 172.21.15.52:0/664483673 shutdown_connections 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 --2- 172.21.15.52:0/664483673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101d50 0x7fea3810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 --2- 172.21.15.52:0/664483673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38107400 0x7fea381077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 --2- 172.21.15.52:0/664483673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38101390 0x7fea38101810 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 -- 172.21.15.52:0/664483673 >> 172.21.15.52:0/664483673 conn(0x7fea38078340 msgr2=0x7fea38078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 -- 172.21.15.52:0/664483673 shutdown_connections 2024-06-02T23:19:14.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.172+0000 7fea3dffe700 1 -- 172.21.15.52:0/664483673 wait complete. 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.173+0000 7fea3dffe700 1 Processor -- start 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.173+0000 7fea3dffe700 1 -- start start 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea3dffe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101390 0x7fea3819e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea3dffe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38101d50 0x7fea3819ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea3dffe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38107400 0x7fea38198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea3dffe700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea38111540 con 0x7fea38107400 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea3dffe700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea381113c0 con 0x7fea38101390 2024-06-02T23:19:14.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea3dffe700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea381116c0 con 0x7fea38101d50 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38107400 0x7fea38198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea36ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38101d50 0x7fea3819ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38107400 0x7fea38198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41150/0 (socket says 172.21.15.52:41150) 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea37fff700 1 -- 172.21.15.52:0/2212028854 learned_addr learned my addr 172.21.15.52:0/2212028854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea377fe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101390 0x7fea3819e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea377fe700 1 -- 172.21.15.52:0/2212028854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38101d50 msgr2=0x7fea3819ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea377fe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38101d50 0x7fea3819ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea377fe700 1 -- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38107400 msgr2=0x7fea38198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea377fe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38107400 0x7fea38198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.174+0000 7fea377fe700 1 -- 172.21.15.52:0/2212028854 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea381990d0 con 0x7fea38101390 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.175+0000 7fea377fe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101390 0x7fea3819e7b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fea2800d750 tx=0x7fea2800da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.175+0000 7fea34ff9700 1 -- 172.21.15.52:0/2212028854 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28016070 con 0x7fea38101390 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.175+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea381993c0 con 0x7fea38101390 2024-06-02T23:19:14.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.175+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea381a5680 con 0x7fea38101390 2024-06-02T23:19:14.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.175+0000 7fea34ff9700 1 -- 172.21.15.52:0/2212028854 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea28005020 con 0x7fea38101390 2024-06-02T23:19:14.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.175+0000 7fea34ff9700 1 -- 172.21.15.52:0/2212028854 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2801f060 con 0x7fea38101390 2024-06-02T23:19:14.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.177+0000 7fea34ff9700 1 -- 172.21.15.52:0/2212028854 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fea2800fa50 con 0x7fea38101390 2024-06-02T23:19:14.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.178+0000 7fea34ff9700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fea200a2c20 0x7fea200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.178+0000 7fea36ffd700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fea200a2c20 0x7fea200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.178+0000 7fea34ff9700 1 -- 172.21.15.52:0/2212028854 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fea280ce9e0 con 0x7fea38101390 2024-06-02T23:19:14.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.179+0000 7fea36ffd700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fea200a2c20 0x7fea200a50e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00b2e0 tx=0x7fea2c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.179+0000 7fea48881b80 1 Processor -- start 2024-06-02T23:19:14.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.179+0000 7fea48881b80 1 -- start start 2024-06-02T23:19:14.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.179+0000 7fea48881b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.179+0000 7fea48881b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ebdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea48881b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657eca210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea48881b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563657dabf60 con 0x563657ec3470 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea48881b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563657e0a850 con 0x563657ebe030 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea48881b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563657d72460 con 0x563657ebd670 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41166/0 (socket says 172.21.15.52:41166) 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 -- 172.21.15.52:0/3629763032 learned_addr learned my addr 172.21.15.52:0/3629763032 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea37fff700 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657eca210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea36ffd700 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ebdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 -- 172.21.15.52:0/3629763032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 msgr2=0x563657ebdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ebdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 -- 172.21.15.52:0/3629763032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 msgr2=0x563657eca210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657eca210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 -- 172.21.15.52:0/3629763032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563657d598d0 con 0x563657ec3470 2024-06-02T23:19:14.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea377fe700 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec3850 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fea280a4890 tx=0x7fea2809bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=801d57188c0f058a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea1dffb700 1 -- 172.21.15.52:0/3629763032 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2809a450 con 0x563657ec3470 2024-06-02T23:19:14.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea1dffb700 1 -- 172.21.15.52:0/3629763032 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea280bc0c0 con 0x563657ec3470 2024-06-02T23:19:14.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.180+0000 7fea1dffb700 1 -- 172.21.15.52:0/3629763032 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28098920 con 0x563657ec3470 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 -- 172.21.15.52:0/3629763032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 msgr2=0x563657ec3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec3850 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fea280a4890 tx=0x7fea2809bdc0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 -- 172.21.15.52:0/3629763032 shutdown_connections 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657eca210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec3850 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 --2- 172.21.15.52:0/3629763032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ebdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 -- 172.21.15.52:0/3629763032 >> 172.21.15.52:0/3629763032 conn(0x563657de6080 msgr2=0x563657eb4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 -- 172.21.15.52:0/3629763032 shutdown_connections 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.181+0000 7fea48881b80 1 -- 172.21.15.52:0/3629763032 wait complete. 2024-06-02T23:19:14.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 -- start start 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 Processor -- start 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ec7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563657daafc0 con 0x563657ec3470 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563657e06810 con 0x563657ebe030 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563657eca750 con 0x563657ebd670 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.182+0000 7fea48881b80 10 client.0 _ll_register_callbacks cb 0x563657da05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea36ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ec7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41168/0 (socket says 172.21.15.52:41168) 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 -- 172.21.15.52:0/3811289687 learned_addr learned my addr 172.21.15.52:0/3811289687 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea377fe700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 -- 172.21.15.52:0/3811289687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 msgr2=0x563657ec9f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec9f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 -- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 msgr2=0x563657ec7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ec7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 -- 172.21.15.52:0/3811289687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563657dd0750 con 0x563657ec3470 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea377fe700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea36ffd700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ec7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea37fff700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec77a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fea30014e10 tx=0x7fea3000e610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1571d5b8f345ca20 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:14.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea16ffd700 1 -- 172.21.15.52:0/3811289687 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea300090d0 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea280bc0c0 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea16ffd700 1 -- 172.21.15.52:0/3811289687 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fea30009270 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea16ffd700 1 -- 172.21.15.52:0/3811289687 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea30003a00 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea48881b80 10 client.17580 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.183+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563657ed4f90 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea16ffd700 1 -- 172.21.15.52:0/3811289687 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea280bc0c0 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea16ffd700 1 -- 172.21.15.52:0/3811289687 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fea3001e5c0 con 0x563657ec3470 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea48881b80 10 client.17580 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea48881b80 10 client.17580 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea48881b80 20 client.17580 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea48881b80 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657ed7620 0x563657ed9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea48881b80 4 client.17580 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563657e17d50 con 0x563657ed7620 2024-06-02T23:19:14.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea377fe700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657ed7620 0x563657ed9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea377fe700 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657ed7620 0x563657ed9a30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b5377eff480a3b6e server_cookie=262d096914d61dcf in_seq=0 out_seq=0 2024-06-02T23:19:14.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.184+0000 7fea16ffd700 10 client.17580 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea16ffd700 1 -- 172.21.15.52:0/3811289687 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563657e17d50 con 0x563657ed7620 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea16ffd700 10 client.17580 handle_command_reply: tid=0 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 client.17580 shutdown 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 msgr2=0x563657ec77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec77a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fea30014e10 tx=0x7fea3000e610 comp rx=0 tx=0).stop 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 shutdown_connections 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ec77a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657ed7620 0x563657ed9a30 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 --2- 172.21.15.52:0/3811289687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ec7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea16ffd700 0 client.17580 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:19:14.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.185+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 >> 172.21.15.52:0/3811289687 conn(0x563657de6080 msgr2=0x563657eb3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.186+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 shutdown_connections 2024-06-02T23:19:14.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.186+0000 7fea48881b80 1 -- 172.21.15.52:0/3811289687 wait complete. 2024-06-02T23:19:14.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.186+0000 7fea48881b80 20 client.17580 trim_cache size 0 max 16384 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 Processor -- start 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 -- start start 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec6400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 0x563657eb3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ec3470 0x563657eb43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea381aa820 con 0x563657ebe030 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea381ac360 con 0x563657ec3470 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea48881b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea381aaef0 con 0x563657ebd670 2024-06-02T23:19:14.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 0x563657eb3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea36ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 0x563657eb3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41176/0 (socket says 172.21.15.52:41176) 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea36ffd700 1 -- 172.21.15.52:0/1315513689 learned_addr learned my addr 172.21.15.52:0/1315513689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea37fff700 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ec3470 0x563657eb43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.204+0000 7fea377fe700 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea36ffd700 1 -- 172.21.15.52:0/1315513689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 msgr2=0x563657ec6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea36ffd700 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec6400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea36ffd700 1 -- 172.21.15.52:0/1315513689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ec3470 msgr2=0x563657eb43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea36ffd700 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ec3470 0x563657eb43c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea36ffd700 1 -- 172.21.15.52:0/1315513689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563657eb4e40 con 0x563657ebe030 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea377fe700 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:14.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea36ffd700 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 0x563657eb3e80 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fea2c000c00 tx=0x7fea2c022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d8b9378b6fba50 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea1d7fa700 1 -- 172.21.15.52:0/1315513689 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c036070 con 0x563657ebe030 2024-06-02T23:19:14.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea1d7fa700 1 -- 172.21.15.52:0/1315513689 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea2c003d30 con 0x563657ebe030 2024-06-02T23:19:14.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.205+0000 7fea1d7fa700 1 -- 172.21.15.52:0/1315513689 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c00fc70 con 0x563657ebe030 2024-06-02T23:19:14.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 -- 172.21.15.52:0/1315513689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 msgr2=0x563657eb3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 0x563657eb3e80 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fea2c000c00 tx=0x7fea2c022000 comp rx=0 tx=0).stop 2024-06-02T23:19:14.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 -- 172.21.15.52:0/1315513689 shutdown_connections 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ec3470 0x563657eb43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ec6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 --2- 172.21.15.52:0/1315513689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ebe030 0x563657eb3e80 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 -- 172.21.15.52:0/1315513689 >> 172.21.15.52:0/1315513689 conn(0x563657de6080 msgr2=0x563657ec2a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 -- 172.21.15.52:0/1315513689 shutdown_connections 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.206+0000 7fea48881b80 1 -- 172.21.15.52:0/1315513689 wait complete. 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 -- start start 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 Processor -- start 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ecfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ed0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ede6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fea381abc90 con 0x563657ec3470 2024-06-02T23:19:14.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fea381ab5c0 con 0x563657ebe030 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea381a87c0 con 0x563657ebd670 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea48881b80 10 client.0 _ll_register_callbacks cb 0x563657da05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ede6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea37fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ede6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41178/0 (socket says 172.21.15.52:41178) 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea37fff700 1 -- 172.21.15.52:0/2829548283 learned_addr learned my addr 172.21.15.52:0/2829548283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.207+0000 7fea36ffd700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ed0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea377fe700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ecfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea37fff700 1 -- 172.21.15.52:0/2829548283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 msgr2=0x563657ecfbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea37fff700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ecfbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea37fff700 1 -- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 msgr2=0x563657ed0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea37fff700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ed0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea37fff700 1 -- 172.21.15.52:0/2829548283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563657edef30 con 0x563657ec3470 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea36ffd700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ed0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:14.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea37fff700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ede6f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fea3001ea30 tx=0x7fea3000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07d493a40b32eb3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea16ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea16ffd700 1 -- 172.21.15.52:0/2829548283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea30003e30 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea16ffd700 1 -- 172.21.15.52:0/2829548283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fea300090d0 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2c003d30 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea16ffd700 1 -- 172.21.15.52:0/2829548283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fea300092f0 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea48881b80 10 client.17586 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.208+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563657f41db0 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea16ffd700 1 -- 172.21.15.52:0/2829548283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fea2c003d30 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea16ffd700 1 -- 172.21.15.52:0/2829548283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fea30007a30 con 0x563657ec3470 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea48881b80 10 client.17586 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea48881b80 10 client.17586 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea48881b80 20 client.17586 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea48881b80 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657f42540 0x563657f44960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:14.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea48881b80 4 client.17586 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:19:14.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563657e122d0 con 0x563657f42540 2024-06-02T23:19:14.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.209+0000 7fea377fe700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657f42540 0x563657f44960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea16ffd700 10 client.17586 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea377fe700 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657f42540 0x563657f44960 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9cf2d41da0f3a93 server_cookie=f61ec1d2e216ce96 in_seq=0 out_seq=0 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea16ffd700 1 -- 172.21.15.52:0/2829548283 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563657e122d0 con 0x563657f42540 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea16ffd700 10 client.17586 handle_command_reply: tid=0 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 client.17586 shutdown 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 msgr2=0x563657ede6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ede6f0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fea3001ea30 tx=0x7fea3000ea10 comp rx=0 tx=0).stop 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 shutdown_connections 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563657ec3470 0x563657ede6f0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563657ebd670 0x563657ecfbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x563657f42540 0x563657f44960 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea16ffd700 0 client.17586 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:19:14.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 --2- 172.21.15.52:0/2829548283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563657ebe030 0x563657ed0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 >> 172.21.15.52:0/2829548283 conn(0x563657de6080 msgr2=0x563657ec6400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 shutdown_connections 2024-06-02T23:19:14.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 1 -- 172.21.15.52:0/2829548283 wait complete. 2024-06-02T23:19:14.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.210+0000 7fea48881b80 20 client.17586 trim_cache size 0 max 16384 2024-06-02T23:19:14.215 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fea200a2c20 msgr2=0x7fea200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fea200a2c20 0x7fea200a50e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fea2c00b2e0 tx=0x7fea2c005960 comp rx=0 tx=0).stop 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101390 msgr2=0x7fea3819e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101390 0x7fea3819e7b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fea2800d750 tx=0x7fea2800da60 comp rx=0 tx=0).stop 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 shutdown_connections 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fea200a2c20 0x7fea200a50e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fea38107400 0x7fea38198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fea38101390 0x7fea3819e7b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 --2- 172.21.15.52:0/2212028854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea38101d50 0x7fea3819ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:14.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 >> 172.21.15.52:0/2212028854 conn(0x7fea38078340 msgr2=0x7fea381006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:14.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 shutdown_connections 2024-06-02T23:19:14.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:14.211+0000 7fea3dffe700 1 -- 172.21.15.52:0/2212028854 wait complete. 2024-06-02T23:19:14.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 139755 1.88 MB/sec execute 263 sec latency 24.378 ms 2024-06-02T23:19:15.031 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:19:15.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:15 smithi123 ceph-mon[35977]: pgmap v652: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-02T23:19:15.303 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:15.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:15 smithi174 ceph-mon[35994]: pgmap v652: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-02T23:19:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:15 smithi052 ceph-mon[32340]: pgmap v652: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 131 op/s 2024-06-02T23:19:15.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 140370 1.88 MB/sec execute 264 sec latency 14.392 ms 2024-06-02T23:19:16.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 140916 1.88 MB/sec execute 265 sec latency 15.446 ms 2024-06-02T23:19:17.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:17 smithi052 ceph-mon[32340]: pgmap v653: 97 pgs: 97 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 -- 172.21.15.52:0/2292069443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4102e50 msgr2=0x7f8ed410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 --2- 172.21.15.52:0/2292069443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4102e50 0x7f8ed410f310 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc00b0d0 tx=0x7f8ecc00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 -- 172.21.15.52:0/2292069443 shutdown_connections 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 --2- 172.21.15.52:0/2292069443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4102e50 0x7f8ed410f310 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 --2- 172.21.15.52:0/2292069443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4108540 0x7f8ed4108920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 --2- 172.21.15.52:0/2292069443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102490 0x7f8ed4102910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 -- 172.21.15.52:0/2292069443 >> 172.21.15.52:0/2292069443 conn(0x7f8ed40fdf40 msgr2=0x7f8ed4100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 -- 172.21.15.52:0/2292069443 shutdown_connections 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.108+0000 7f8edc662700 1 -- 172.21.15.52:0/2292069443 wait complete. 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 Processor -- start 2024-06-02T23:19:17.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 -- start start 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 0x7f8ed4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102e50 0x7f8ed4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 0x7f8ed4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4112620 con 0x7f8ed4108540 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41124a0 con 0x7f8ed4102490 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.109+0000 7f8edc662700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41127a0 con 0x7f8ed4102e50 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 0x7f8ed4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 0x7f8ed4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 0x7f8ed4075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41196/0 (socket says 172.21.15.52:41196) 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8ed9bfd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102e50 0x7f8ed4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 0x7f8ed4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33784/0 (socket says 172.21.15.52:33784) 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 -- 172.21.15.52:0/3516129772 learned_addr learned my addr 172.21.15.52:0/3516129772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 -- 172.21.15.52:0/3516129772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102e50 msgr2=0x7f8ed4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102e50 0x7f8ed4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 -- 172.21.15.52:0/3516129772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 msgr2=0x7f8ed4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 0x7f8ed4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 -- 172.21.15.52:0/3516129772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed41a6f50 con 0x7f8ed4108540 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 0x7f8ed4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102e50 0x7f8ed4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edabff700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 0x7f8ed4075210 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0095c0 tx=0x7f8ecc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8ecb7fe700 1 -- 172.21.15.52:0/3516129772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc01f070 con 0x7f8ed4108540 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41a71e0 con 0x7f8ed4108540 2024-06-02T23:19:17.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a7690 con 0x7f8ed4108540 2024-06-02T23:19:17.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.110+0000 7f8ecb7fe700 1 -- 172.21.15.52:0/3516129772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ecc007e20 con 0x7f8ed4108540 2024-06-02T23:19:17.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.111+0000 7f8ecb7fe700 1 -- 172.21.15.52:0/3516129772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc01b4a0 con 0x7f8ed4108540 2024-06-02T23:19:17.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.113+0000 7f8ecb7fe700 1 -- 172.21.15.52:0/3516129772 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8ecc022070 con 0x7f8ed4108540 2024-06-02T23:19:17.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.114+0000 7f8ecb7fe700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.114+0000 7f8ecb7fe700 1 -- 172.21.15.52:0/3516129772 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8ecc00f020 con 0x7f8ed4108540 2024-06-02T23:19:17.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.114+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.114+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0007f50 tx=0x7f8ed00073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:17.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 Processor -- start 2024-06-02T23:19:17.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 -- start start 2024-06-02T23:19:17.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfb1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfab260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfabbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5606bce9df60 con 0x5606bcfb0f60 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5606bcefc850 con 0x5606bcfab7a0 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606bce64460 con 0x5606bcfb1880 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfb1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfb1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41200/0 (socket says 172.21.15.52:41200) 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8eda3fe700 1 -- 172.21.15.52:0/4105728503 learned_addr learned my addr 172.21.15.52:0/4105728503 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfab260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.115+0000 7f8edabff700 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfabbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ed9bfd700 1 -- 172.21.15.52:0/4105728503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 msgr2=0x5606bcfabbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfabbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ed9bfd700 1 -- 172.21.15.52:0/4105728503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 msgr2=0x5606bcfb1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfb1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ed9bfd700 1 -- 172.21.15.52:0/4105728503 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bce4b8d0 con 0x5606bcfb1880 2024-06-02T23:19:17.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfab260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40761d0 tx=0x7f8ec4010730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dad6ae949d682709 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:17.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ec8ff9700 1 -- 172.21.15.52:0/4105728503 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec401c480 con 0x5606bcfb1880 2024-06-02T23:19:17.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ec8ff9700 1 -- 172.21.15.52:0/4105728503 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec401cad0 con 0x5606bcfb1880 2024-06-02T23:19:17.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.116+0000 7f8ec8ff9700 1 -- 172.21.15.52:0/4105728503 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4011b90 con 0x5606bcfb1880 2024-06-02T23:19:17.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/4105728503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 msgr2=0x5606bcfab260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfab260 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40761d0 tx=0x7f8ec4010730 comp rx=0 tx=0).stop 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/4105728503 shutdown_connections 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfabbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfb1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/4105728503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfab260 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/4105728503 >> 172.21.15.52:0/4105728503 conn(0x5606bced8080 msgr2=0x5606bcfa6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/4105728503 shutdown_connections 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/4105728503 wait complete. 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.117+0000 7f8ee6ee5b80 1 -- start start 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 Processor -- start 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfb75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb0f60 0x5606bcfb4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5606bce9cfc0 con 0x5606bcfb1880 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5606bcef8810 con 0x5606bcfab7a0 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606bcfb8260 con 0x5606bcfb0f60 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ee6ee5b80 10 client.0 _ll_register_callbacks cb 0x5606bce925b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfb75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8edabff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8ed9bfd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb0f60 0x5606bcfb4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfb75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33792/0 (socket says 172.21.15.52:33792) 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8eda3fe700 1 -- 172.21.15.52:0/2336098171 learned_addr learned my addr 172.21.15.52:0/2336098171 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8edabff700 1 -- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb0f60 msgr2=0x5606bcfb4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8edabff700 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb0f60 0x5606bcfb4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8edabff700 1 -- 172.21.15.52:0/2336098171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 msgr2=0x5606bcfb75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8edabff700 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfb75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.118+0000 7f8edabff700 1 -- 172.21.15.52:0/2336098171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bcec2750 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8edabff700 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb4e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0cdbb0 tx=0x7f8ecc09d140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1144aa9ba12736d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ebdffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ebdffb700 1 -- 172.21.15.52:0/2336098171 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0a5530 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ebdffb700 1 -- 172.21.15.52:0/2336098171 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8ecc003c00 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec401cad0 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ebdffb700 1 -- 172.21.15.52:0/2336098171 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ecc003e20 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 10 client.17598 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606bcfc2a00 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ebdffb700 1 -- 172.21.15.52:0/2336098171 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec401cad0 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ebdffb700 1 -- 172.21.15.52:0/2336098171 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8ecc011070 con 0x5606bcfb1880 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 10 client.17598 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 10 client.17598 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 20 client.17598 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:17.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bcfc4fc0 0x5606bcfc73e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 4 client.17598 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.119+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606bcf09d50 con 0x5606bcfc4fc0 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.120+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bcfc4fc0 0x5606bcfc73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.120+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bcfc4fc0 0x5606bcfc73e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7233ce70859efb68 server_cookie=9d266b5c960e0ca0 in_seq=0 out_seq=0 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.120+0000 7f8ebdffb700 10 client.17598 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ebdffb700 1 -- 172.21.15.52:0/2336098171 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5606bcf09d50 con 0x5606bcfc4fc0 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ebdffb700 10 client.17598 handle_command_reply: tid=0 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 client.17598 shutdown 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 msgr2=0x5606bcfb4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb4e40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0cdbb0 tx=0x7f8ecc09d140 comp rx=0 tx=0).stop 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 shutdown_connections 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb4e40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfb75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bcfc4fc0 0x5606bcfc73e0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/2336098171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb0f60 0x5606bcfb4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ebdffb700 0 client.17598 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 >> 172.21.15.52:0/2336098171 conn(0x5606bced8080 msgr2=0x5606bcfa5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 shutdown_connections 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/2336098171 wait complete. 2024-06-02T23:19:17.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.121+0000 7f8ee6ee5b80 20 client.17598 trim_cache size 0 max 16384 2024-06-02T23:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 Processor -- start 2024-06-02T23:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 -- start start 2024-06-02T23:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfab7a0 0x5606bcfa5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 0x5606bcfa63c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb1d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb800d2a0 con 0x5606bcfb1880 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb800ede0 con 0x5606bcfb0f60 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb800d970 con 0x5606bcfab7a0 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfab7a0 0x5606bcfa5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ed9bfd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 0x5606bcfa63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8edabff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb1d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfab7a0 0x5606bcfa5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:58600/0 (socket says 172.21.15.52:58600) 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8ed9bfd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 0x5606bcfa63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33804/0 (socket says 172.21.15.52:33804) 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.138+0000 7f8eda3fe700 1 -- 172.21.15.52:0/3530572417 learned_addr learned my addr 172.21.15.52:0/3530572417 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ed9bfd700 1 -- 172.21.15.52:0/3530572417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfab7a0 msgr2=0x5606bcfa5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfab7a0 0x5606bcfa5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ed9bfd700 1 -- 172.21.15.52:0/3530572417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 msgr2=0x5606bcfb1d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb1d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ed9bfd700 1 -- 172.21.15.52:0/3530572417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606bcfa79a0 con 0x5606bcfb0f60 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8edabff700 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 0x5606bcfa63c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005d90 tx=0x7f8ec4005950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd13f9f3e0b1f243 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ebffff700 1 -- 172.21.15.52:0/3530572417 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4029bc0 con 0x5606bcfb0f60 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ebffff700 1 -- 172.21.15.52:0/3530572417 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec40044d0 con 0x5606bcfb0f60 2024-06-02T23:19:17.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.139+0000 7f8ebffff700 1 -- 172.21.15.52:0/3530572417 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4005530 con 0x5606bcfb0f60 2024-06-02T23:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/3530572417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 msgr2=0x5606bcfa63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 0x5606bcfa63c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005d90 tx=0x7f8ec4005950 comp rx=0 tx=0).stop 2024-06-02T23:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/3530572417 shutdown_connections 2024-06-02T23:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb1880 0x5606bcfb1d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfab7a0 0x5606bcfa5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/3530572417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfb0f60 0x5606bcfa63c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/3530572417 >> 172.21.15.52:0/3530572417 conn(0x5606bced8080 msgr2=0x5606bcfb31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/3530572417 shutdown_connections 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/3530572417 wait complete. 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 -- start start 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.140+0000 7f8ee6ee5b80 1 Processor -- start 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfbd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfbdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfcbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb800e710 con 0x5606bcfb0f60 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb800e040 con 0x5606bcfab7a0 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb800b240 con 0x5606bcfb1880 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ee6ee5b80 10 client.0 _ll_register_callbacks cb 0x5606bce925b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:17.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfbd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8ed9bfd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfbdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfbd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33806/0 (socket says 172.21.15.52:33806) 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 -- 172.21.15.52:0/839547902 learned_addr learned my addr 172.21.15.52:0/839547902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8edabff700 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfcbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 -- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 msgr2=0x5606bcfcbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfcbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 -- 172.21.15.52:0/839547902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 msgr2=0x5606bcfbdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:17.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfbdac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.141+0000 7f8eda3fe700 1 -- 172.21.15.52:0/839547902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606bcfcc2c0 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8eda3fe700 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfbd580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00042f0 tx=0x7f8ed00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e04da5b6924b3dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ebdffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ebdffb700 1 -- 172.21.15.52:0/839547902 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ed002d070 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ebdffb700 1 -- 172.21.15.52:0/839547902 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8ed0011440 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec40044d0 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ebdffb700 1 -- 172.21.15.52:0/839547902 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8ed0039050 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 10 client.27095 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5606bd02f500 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ebdffb700 1 -- 172.21.15.52:0/839547902 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8ec40044d0 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ebdffb700 1 -- 172.21.15.52:0/839547902 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8ed0029520 con 0x5606bcfab7a0 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 10 client.27095 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 10 client.27095 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:19:17.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 20 client.27095 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.142+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bd02fc40 0x5606bd032060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ee6ee5b80 4 client.27095 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5606bcf042d0 con 0x5606bd02fc40 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bd02fc40 0x5606bd032060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ed9bfd700 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bd02fc40 0x5606bd032060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a699907bbd55ae47 server_cookie=2e6fac781189f59e in_seq=0 out_seq=0 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ebdffb700 10 client.27095 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ebdffb700 1 -- 172.21.15.52:0/839547902 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5606bcf042d0 con 0x5606bd02fc40 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ebdffb700 10 client.27095 handle_command_reply: tid=0 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ee6ee5b80 1 client.27095 shutdown 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 msgr2=0x5606bcfbd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.143+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfbd580 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8ed00042f0 tx=0x7f8ed00047c0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 shutdown_connections 2024-06-02T23:19:17.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5606bd02fc40 0x5606bd032060 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606bcfb1880 0x5606bcfcbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5606bcfab7a0 0x5606bcfbd580 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ebdffb700 0 client.27095 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 --2- 172.21.15.52:0/839547902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5606bcfb0f60 0x5606bcfbdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 >> 172.21.15.52:0/839547902 conn(0x5606bced8080 msgr2=0x5606bcfb2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 shutdown_connections 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 1 -- 172.21.15.52:0/839547902 wait complete. 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8ee6ee5b80 20 client.27095 trim_cache size 0 max 16384 2024-06-02T23:19:17.150 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ec00a2ca0 msgr2=0x7f8ec00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8edc662700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0007f50 tx=0x7f8ed00073d0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 msgr2=0x7f8ed4075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.144+0000 7f8edc662700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 0x7f8ed4075210 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0095c0 tx=0x7f8ecc007a20 comp rx=0 tx=0).stop 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 shutdown_connections 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8ec00a2ca0 0x7f8ec00a5160 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ed4108540 0x7f8ed4075210 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ed4102490 0x7f8ed4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 --2- 172.21.15.52:0/3516129772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ed4102e50 0x7f8ed4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:17.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 >> 172.21.15.52:0/3516129772 conn(0x7f8ed40fdf40 msgr2=0x7f8ed40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:17.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 shutdown_connections 2024-06-02T23:19:17.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:17.145+0000 7f8edc662700 1 -- 172.21.15.52:0/3516129772 wait complete. 2024-06-02T23:19:17.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:17 smithi123 ceph-mon[35977]: pgmap v653: 97 pgs: 97 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-02T23:19:17.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:17 smithi174 ceph-mon[35994]: pgmap v653: 97 pgs: 97 active+clean; 66 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 293 KiB/s rd, 1.5 MiB/s wr, 179 op/s 2024-06-02T23:19:17.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 141656 1.90 MB/sec execute 266 sec latency 15.717 ms 2024-06-02T23:19:17.844 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:19:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:18.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 142057 1.90 MB/sec execute 267 sec latency 15.080 ms 2024-06-02T23:19:19.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:19 smithi123 ceph-mon[35977]: pgmap v654: 97 pgs: 97 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:19:19.341 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:19 smithi052 ceph-mon[32340]: pgmap v654: 97 pgs: 97 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:19:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:19 smithi174 ceph-mon[35994]: pgmap v654: 97 pgs: 97 active+clean; 65 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.2 MiB/s wr, 140 op/s 2024-06-02T23:19:19.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 142341 1.90 MB/sec execute 268 sec latency 14.523 ms 2024-06-02T23:19:19.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.903+0000 7f4c8f082700 1 -- 172.21.15.52:0/3452088347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88101d50 msgr2=0x7f4c8810e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.903+0000 7f4c8f082700 1 --2- 172.21.15.52:0/3452088347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88101d50 0x7f4c8810e240 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000b0a0 tx=0x7f4c8000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 -- 172.21.15.52:0/3452088347 shutdown_connections 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 --2- 172.21.15.52:0/3452088347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88101d50 0x7f4c8810e240 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 --2- 172.21.15.52:0/3452088347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88107400 0x7f4c881077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 --2- 172.21.15.52:0/3452088347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88101390 0x7f4c88101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 -- 172.21.15.52:0/3452088347 >> 172.21.15.52:0/3452088347 conn(0x7f4c88078340 msgr2=0x7f4c88078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 -- 172.21.15.52:0/3452088347 shutdown_connections 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 -- 172.21.15.52:0/3452088347 wait complete. 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 Processor -- start 2024-06-02T23:19:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.904+0000 7f4c8f082700 1 -- start start 2024-06-02T23:19:19.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8f082700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88101390 0x7f4c8819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8f082700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88101d50 0x7f4c8819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8f082700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 0x7f4c881987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8f082700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881115a0 con 0x7f4c88107400 2024-06-02T23:19:19.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8f082700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c88111420 con 0x7f4c88101390 2024-06-02T23:19:19.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8f082700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c88111720 con 0x7f4c88101d50 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 0x7f4c881987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 0x7f4c881987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41256/0 (socket says 172.21.15.52:41256) 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 -- 172.21.15.52:0/888711844 learned_addr learned my addr 172.21.15.52:0/888711844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88101390 0x7f4c8819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c87fff700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88101d50 0x7f4c8819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 -- 172.21.15.52:0/888711844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88101d50 msgr2=0x7f4c8819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88101d50 0x7f4c8819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 -- 172.21.15.52:0/888711844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88101390 msgr2=0x7f4c8819e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88101390 0x7f4c8819e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8d61f700 1 -- 172.21.15.52:0/888711844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c88198fd0 con 0x7f4c88107400 2024-06-02T23:19:19.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.905+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88101390 0x7f4c8819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.906+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 0x7f4c881987f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000b070 tx=0x7f4c80009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.906+0000 7f4c85ffb700 1 -- 172.21.15.52:0/888711844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80026070 con 0x7f4c88107400 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.906+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c88199260 con 0x7f4c88107400 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.906+0000 7f4c85ffb700 1 -- 172.21.15.52:0/888711844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c80007770 con 0x7f4c88107400 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.906+0000 7f4c85ffb700 1 -- 172.21.15.52:0/888711844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004280 con 0x7f4c88107400 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.906+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c88102fa0 con 0x7f4c88107400 2024-06-02T23:19:19.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.909+0000 7f4c85ffb700 1 -- 172.21.15.52:0/888711844 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4c80007920 con 0x7f4c88107400 2024-06-02T23:19:19.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.910+0000 7f4c85ffb700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c780a2c20 0x7f4c780a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.910+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c780a2c20 0x7f4c780a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.910+0000 7f4c85ffb700 1 -- 172.21.15.52:0/888711844 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4c8000f030 con 0x7f4c88107400 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.910+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c780a2c20 0x7f4c780a50e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009680 tx=0x7f4c7c012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 Processor -- start 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 -- start start 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bef60 0x55ad342bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bf880 0x55ad342b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad341abf60 con 0x55ad342b97a0 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad3420a850 con 0x55ad342bf880 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c99905b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad34172460 con 0x55ad342bef60 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342b9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41264/0 (socket says 172.21.15.52:41264) 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 -- 172.21.15.52:0/3846658050 learned_addr learned my addr 172.21.15.52:0/3846658050 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bef60 0x55ad342bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c87fff700 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bf880 0x55ad342b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 -- 172.21.15.52:0/3846658050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bef60 msgr2=0x55ad342bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bef60 0x55ad342bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 -- 172.21.15.52:0/3846658050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bf880 msgr2=0x55ad342b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bf880 0x55ad342b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.911+0000 7f4c8d61f700 1 -- 172.21.15.52:0/3846658050 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad341598d0 con 0x55ad342b97a0 2024-06-02T23:19:19.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.912+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342b9bc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4c800d5f50 tx=0x7f4c8000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e6bc7e9f2988db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:19.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.912+0000 7f4c72ffd700 1 -- 172.21.15.52:0/3846658050 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800bde60 con 0x55ad342b97a0 2024-06-02T23:19:19.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.912+0000 7f4c72ffd700 1 -- 172.21.15.52:0/3846658050 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c800ba070 con 0x55ad342b97a0 2024-06-02T23:19:19.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.912+0000 7f4c72ffd700 1 -- 172.21.15.52:0/3846658050 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80007cb0 con 0x55ad342b97a0 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 -- 172.21.15.52:0/3846658050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 msgr2=0x55ad342b9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342b9bc0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f4c800d5f50 tx=0x7f4c8000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 -- 172.21.15.52:0/3846658050 shutdown_connections 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342b9bc0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bef60 0x55ad342bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 --2- 172.21.15.52:0/3846658050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bf880 0x55ad342b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 -- 172.21.15.52:0/3846658050 >> 172.21.15.52:0/3846658050 conn(0x55ad341e6080 msgr2=0x55ad342b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 -- 172.21.15.52:0/3846658050 shutdown_connections 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 -- 172.21.15.52:0/3846658050 wait complete. 2024-06-02T23:19:19.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.913+0000 7f4c99905b80 1 -- start start 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 Processor -- start 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 0x55ad342c2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ad341aafc0 con 0x55ad342b97a0 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ad34206810 con 0x55ad342bef60 2024-06-02T23:19:19.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ad342c6010 con 0x55ad342bf880 2024-06-02T23:19:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c99905b80 10 client.0 _ll_register_callbacks cb 0x55ad341a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c8ce1e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c87fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 0x55ad342c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c8ce1e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41270/0 (socket says 172.21.15.52:41270) 2024-06-02T23:19:19.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c87fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 0x55ad342c2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33818/0 (socket says 172.21.15.52:33818) 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c8ce1e700 1 -- 172.21.15.52:0/4102313112 learned_addr learned my addr 172.21.15.52:0/4102313112 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.914+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c87fff700 1 -- 172.21.15.52:0/4102313112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 msgr2=0x55ad342c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c87fff700 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c87fff700 1 -- 172.21.15.52:0/4102313112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 msgr2=0x55ad342c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c87fff700 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c87fff700 1 -- 172.21.15.52:0/4102313112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad341d0750 con 0x55ad342bef60 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c87fff700 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 0x55ad342c2880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c74012740 tx=0x7f4c74012a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbc0245938276e36 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c67fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c67fff700 1 -- 172.21.15.52:0/4102313112 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c74004020 con 0x55ad342bef60 2024-06-02T23:19:19.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c67fff700 1 -- 172.21.15.52:0/4102313112 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4c740041c0 con 0x55ad342bef60 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c800ba070 con 0x55ad342bef60 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c67fff700 1 -- 172.21.15.52:0/4102313112 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c74023070 con 0x55ad342bef60 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c99905b80 10 client.27107 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.915+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad342d0780 con 0x55ad342bef60 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c67fff700 1 -- 172.21.15.52:0/4102313112 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c800ba070 con 0x55ad342bef60 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c67fff700 1 -- 172.21.15.52:0/4102313112 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c7400f4c0 con 0x55ad342bef60 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c99905b80 10 client.27107 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c99905b80 10 client.27107 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c99905b80 20 client.27107 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c99905b80 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad342d3150 0x55ad342d55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c99905b80 4 client.27107 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:19.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad34217d50 con 0x55ad342d3150 2024-06-02T23:19:19.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.916+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad342d3150 0x55ad342d55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad342d3150 0x55ad342d55d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80c470523e44bd54 server_cookie=90c0704351155e09 in_seq=0 out_seq=0 2024-06-02T23:19:19.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c67fff700 10 client.27107 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c67fff700 1 -- 172.21.15.52:0/4102313112 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ad34217d50 con 0x55ad342d3150 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c67fff700 10 client.27107 handle_command_reply: tid=0 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c99905b80 1 client.27107 shutdown 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 msgr2=0x55ad342c2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.917+0000 7f4c99905b80 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 0x55ad342c2880 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4c74012740 tx=0x7f4c74012a50 comp rx=0 tx=0).stop 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 shutdown_connections 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad342d3150 0x55ad342d55d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342c5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c67fff700 0 client.27107 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:19:19.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 --2- 172.21.15.52:0/4102313112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342bef60 0x55ad342c2880 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 >> 172.21.15.52:0/4102313112 conn(0x55ad341e6080 msgr2=0x55ad342b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:19.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 shutdown_connections 2024-06-02T23:19:19.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 1 -- 172.21.15.52:0/4102313112 wait complete. 2024-06-02T23:19:19.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.918+0000 7f4c99905b80 20 client.27107 trim_cache size 0 max 16384 2024-06-02T23:19:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 Processor -- start 2024-06-02T23:19:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 -- start start 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342b97a0 0x55ad342b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 0x55ad342b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342b43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881aa5a0 con 0x55ad342bf880 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881ac0e0 con 0x55ad342cf0f0 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c99905b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881aac70 con 0x55ad342b97a0 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c87fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 0x55ad342b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342b43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c87fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 0x55ad342b3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41284/0 (socket says 172.21.15.52:41284) 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c87fff700 1 -- 172.21.15.52:0/914851411 learned_addr learned my addr 172.21.15.52:0/914851411 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342b43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33824/0 (socket says 172.21.15.52:33824) 2024-06-02T23:19:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.935+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342b97a0 0x55ad342b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8d61f700 1 -- 172.21.15.52:0/914851411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342b97a0 msgr2=0x55ad342b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342b97a0 0x55ad342b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8d61f700 1 -- 172.21.15.52:0/914851411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 msgr2=0x55ad342b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 0x55ad342b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8d61f700 1 -- 172.21.15.52:0/914851411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad342d5c50 con 0x55ad342cf0f0 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342b97a0 0x55ad342b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c87fff700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 0x55ad342b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342b43c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55ad342c5e20 tx=0x7f4c800099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fa306a9f18b4a05 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c727fc700 1 -- 172.21.15.52:0/914851411 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80088070 con 0x55ad342cf0f0 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c727fc700 1 -- 172.21.15.52:0/914851411 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c800ba040 con 0x55ad342cf0f0 2024-06-02T23:19:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.936+0000 7f4c727fc700 1 -- 172.21.15.52:0/914851411 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800bde60 con 0x55ad342cf0f0 2024-06-02T23:19:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 -- 172.21.15.52:0/914851411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 msgr2=0x55ad342b43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342b43c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x55ad342c5e20 tx=0x7f4c800099e0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 -- 172.21.15.52:0/914851411 shutdown_connections 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342b43c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342b97a0 0x55ad342b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 --2- 172.21.15.52:0/914851411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342bf880 0x55ad342b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 -- 172.21.15.52:0/914851411 >> 172.21.15.52:0/914851411 conn(0x55ad341e6080 msgr2=0x55ad342c1170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 -- 172.21.15.52:0/914851411 shutdown_connections 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 -- 172.21.15.52:0/914851411 wait complete. 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 -- start start 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.937+0000 7f4c99905b80 1 Processor -- start 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342cb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342da710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881aba10 con 0x55ad342b97a0 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881ab340 con 0x55ad342cf0f0 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c881a8560 con 0x55ad342bf880 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c99905b80 10 client.0 _ll_register_callbacks cb 0x55ad341a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342cb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342cb8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41292/0 (socket says 172.21.15.52:41292) 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 -- 172.21.15.52:0/1413098199 learned_addr learned my addr 172.21.15.52:0/1413098199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8d61f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342dac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33838/0 (socket says 172.21.15.52:33838) 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c87fff700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342da710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 -- 172.21.15.52:0/1413098199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 msgr2=0x55ad342da710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342da710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 -- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 msgr2=0x55ad342dac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:19.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342dac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 -- 172.21.15.52:0/1413098199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad342db320 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8d61f700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.938+0000 7f4c8ce1e700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342cb8a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x55ad342d5e50 tx=0x7f4c7c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39536faa444344e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c67fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c67fff700 1 -- 172.21.15.52:0/1413098199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c01f0d0 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c67fff700 1 -- 172.21.15.52:0/1413098199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4c7c01f820 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c800ba040 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c67fff700 1 -- 172.21.15.52:0/1413098199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f4c7c01f270 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c99905b80 10 client.17622 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ad3433dda0 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c67fff700 1 -- 172.21.15.52:0/1413098199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4c800ba040 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c67fff700 1 -- 172.21.15.52:0/1413098199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f4c7c03eae0 con 0x55ad342b97a0 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c99905b80 10 client.17622 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:19.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.939+0000 7f4c99905b80 10 client.17622 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c99905b80 20 client.17622 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c99905b80 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad3433e510 0x55ad34340930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c99905b80 4 client.17622 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad342122d0 con 0x55ad3433e510 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c87fff700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad3433e510 0x55ad34340930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c67fff700 10 client.17622 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c87fff700 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad3433e510 0x55ad34340930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3be4eda775ca29a7 server_cookie=46b3b9ca6807a328 in_seq=0 out_seq=0 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c67fff700 1 -- 172.21.15.52:0/1413098199 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad342122d0 con 0x55ad3433e510 2024-06-02T23:19:19.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c67fff700 10 client.17622 handle_command_reply: tid=0 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.940+0000 7f4c99905b80 1 client.17622 shutdown 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 msgr2=0x55ad342cb8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342cb8a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x55ad342d5e50 tx=0x7f4c7c004750 comp rx=0 tx=0).stop 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 shutdown_connections 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ad3433e510 0x55ad34340930 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ad342cf0f0 0x55ad342dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ad342b97a0 0x55ad342cb8a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 --2- 172.21.15.52:0/1413098199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ad342bf880 0x55ad342da710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 >> 172.21.15.52:0/1413098199 conn(0x55ad341e6080 msgr2=0x55ad342bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c67fff700 0 client.17622 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 shutdown_connections 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 1 -- 172.21.15.52:0/1413098199 wait complete. 2024-06-02T23:19:19.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.941+0000 7f4c99905b80 20 client.17622 trim_cache size 0 max 16384 2024-06-02T23:19:19.948 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c780a2c20 msgr2=0x7f4c780a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c780a2c20 0x7f4c780a50e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009680 tx=0x7f4c7c012040 comp rx=0 tx=0).stop 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 msgr2=0x7f4c881987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 0x7f4c881987f0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000b070 tx=0x7f4c80009c00 comp rx=0 tx=0).stop 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 shutdown_connections 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4c780a2c20 0x7f4c780a50e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4c88107400 0x7f4c881987f0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c88101390 0x7f4c8819e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 --2- 172.21.15.52:0/888711844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88101d50 0x7f4c8819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:19.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 >> 172.21.15.52:0/888711844 conn(0x7f4c88078340 msgr2=0x7f4c88105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 shutdown_connections 2024-06-02T23:19:19.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:19.942+0000 7f4c8f082700 1 -- 172.21.15.52:0/888711844 wait complete. 2024-06-02T23:19:20.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 142700 1.90 MB/sec execute 269 sec latency 16.029 ms 2024-06-02T23:19:20.691 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:19:20.955 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:21 smithi123 ceph-mon[35977]: pgmap v655: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-02T23:19:21.277 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:21 smithi052 ceph-mon[32340]: pgmap v655: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-02T23:19:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:21 smithi174 ceph-mon[35994]: pgmap v655: 97 pgs: 97 active+clean; 67 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 283 KiB/s rd, 1.3 MiB/s wr, 166 op/s 2024-06-02T23:19:21.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 143192 1.90 MB/sec execute 270 sec latency 15.277 ms 2024-06-02T23:19:22.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 143820 1.90 MB/sec execute 271 sec latency 14.959 ms 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 -- 172.21.15.52:0/166851884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 msgr2=0x7f727c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 --2- 172.21.15.52:0/166851884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1077e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f7278009990 tx=0x7f7278009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 -- 172.21.15.52:0/166851884 shutdown_connections 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 --2- 172.21.15.52:0/166851884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101d50 0x7f727c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 --2- 172.21.15.52:0/166851884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1077e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 --2- 172.21.15.52:0/166851884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101390 0x7f727c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 -- 172.21.15.52:0/166851884 >> 172.21.15.52:0/166851884 conn(0x7f727c078340 msgr2=0x7f727c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 -- 172.21.15.52:0/166851884 shutdown_connections 2024-06-02T23:19:22.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.792+0000 7f7283da5700 1 -- 172.21.15.52:0/166851884 wait complete. 2024-06-02T23:19:22.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.793+0000 7f7283da5700 1 Processor -- start 2024-06-02T23:19:22.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.793+0000 7f7283da5700 1 -- start start 2024-06-02T23:19:22.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.793+0000 7f7283da5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 0x7f727c197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7283da5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101d50 0x7f727c197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7283da5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7283da5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1115a0 con 0x7f727c107400 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7283da5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727c111420 con 0x7f727c101d50 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7283da5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f727c111720 con 0x7f727c101390 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7282342700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 0x7f727c197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 0x7f727c197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:58678/0 (socket says 172.21.15.52:58678) 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 -- 172.21.15.52:0/1697890120 learned_addr learned my addr 172.21.15.52:0/1697890120 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281340700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101d50 0x7f727c197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 -- 172.21.15.52:0/1697890120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101d50 msgr2=0x7f727c197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101d50 0x7f727c197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.797 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 -- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 msgr2=0x7f727c1982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 -- 172.21.15.52:0/1697890120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f727c1a70a0 con 0x7f727c101390 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281340700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101d50 0x7f727c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7282342700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.794+0000 7f7281b41700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 0x7f727c197830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7278005910 tx=0x7f72780047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.795+0000 7f7272ffd700 1 -- 172.21.15.52:0/1697890120 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278004830 con 0x7f727c101390 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.795+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f727c1a7330 con 0x7f727c101390 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.795+0000 7f7272ffd700 1 -- 172.21.15.52:0/1697890120 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7278005020 con 0x7f727c101390 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.795+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f727c1a7780 con 0x7f727c101390 2024-06-02T23:19:22.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.795+0000 7f7272ffd700 1 -- 172.21.15.52:0/1697890120 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7278003c80 con 0x7f727c101390 2024-06-02T23:19:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.798+0000 7f7272ffd700 1 -- 172.21.15.52:0/1697890120 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f72780051d0 con 0x7f727c101390 2024-06-02T23:19:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.799+0000 7f7272ffd700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72680a2c20 0x7f72680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.799+0000 7f7281340700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72680a2c20 0x7f72680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.799+0000 7f7272ffd700 1 -- 172.21.15.52:0/1697890120 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f72780cfd80 con 0x7f727c101390 2024-06-02T23:19:22.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 Processor -- start 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f7281340700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72680a2c20 0x7f72680a50e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f727c199990 tx=0x7f726c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 -- start start 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832640 0x55f34e832a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e82c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 0x55f34e82d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f34e77e810 con 0x55f34e82cf20 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f34e726720 con 0x55f34e832f60 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f728e628b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f34e77a7d0 con 0x55f34e832640 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f7282342700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 0x55f34e82d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f7281b41700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832640 0x55f34e832a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f7282342700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 0x55f34e82d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41316/0 (socket says 172.21.15.52:41316) 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f7282342700 1 -- 172.21.15.52:0/3714617986 learned_addr learned my addr 172.21.15.52:0/3714617986 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:22.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.800+0000 7f7281340700 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e82c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7282342700 1 -- 172.21.15.52:0/3714617986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832640 msgr2=0x55f34e832a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7282342700 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832640 0x55f34e832a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7282342700 1 -- 172.21.15.52:0/3714617986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 msgr2=0x55f34e82c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7282342700 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e82c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7282342700 1 -- 172.21.15.52:0/3714617986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34e71cf50 con 0x55f34e82cf20 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7281b41700 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832640 0x55f34e832a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7281340700 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e82c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7282342700 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 0x55f34e82d340 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7274007740 tx=0x7f7274009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b9e03745771fb28 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7267fff700 1 -- 172.21.15.52:0/3714617986 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274019020 con 0x55f34e82cf20 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7267fff700 1 -- 172.21.15.52:0/3714617986 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7274015d00 con 0x55f34e82cf20 2024-06-02T23:19:22.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.801+0000 7f7267fff700 1 -- 172.21.15.52:0/3714617986 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7274014d80 con 0x55f34e82cf20 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 -- 172.21.15.52:0/3714617986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 msgr2=0x55f34e82d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 0x55f34e82d340 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7274007740 tx=0x7f7274009f90 comp rx=0 tx=0).stop 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 -- 172.21.15.52:0/3714617986 shutdown_connections 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e82cf20 0x55f34e82d340 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832640 0x55f34e832a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 --2- 172.21.15.52:0/3714617986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e82c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 -- 172.21.15.52:0/3714617986 >> 172.21.15.52:0/3714617986 conn(0x55f34e757cc0 msgr2=0x55f34e827850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 -- 172.21.15.52:0/3714617986 shutdown_connections 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.802+0000 7f728e628b80 1 -- 172.21.15.52:0/3714617986 wait complete. 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 -- start start 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 Processor -- start 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e838cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e835fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e836510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f34e8399d0 con 0x55f34e832640 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f34e71ebf0 con 0x55f34e82cf20 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f728e628b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f34e839b50 con 0x55f34e832f60 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f7281340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e835fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.803+0000 7f7281b41700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e838cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281b41700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e838cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33868/0 (socket says 172.21.15.52:33868) 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e835fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41322/0 (socket says 172.21.15.52:41322) 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281b41700 1 -- 172.21.15.52:0/1833431722 learned_addr learned my addr 172.21.15.52:0/1833431722 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f728e628b80 10 client.0 _ll_register_callbacks cb 0x55f34e718920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7282342700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e836510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 -- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 msgr2=0x55f34e836510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e836510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 -- 172.21.15.52:0/1833431722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 msgr2=0x55f34e838cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e838cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 -- 172.21.15.52:0/1833431722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34e6cd8d0 con 0x55f34e832640 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281b41700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e838cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7282342700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e836510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f7281340700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e835fd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f726c005950 tx=0x7f726c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e881988286849dda server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.804+0000 7f72657fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f72657fa700 1 -- 172.21.15.52:0/1833431722 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c004a30 con 0x55f34e832640 2024-06-02T23:19:22.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f72657fa700 1 -- 172.21.15.52:0/1833431722 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f726c004bd0 con 0x55f34e832640 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7274015d00 con 0x55f34e832640 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f72657fa700 1 -- 172.21.15.52:0/1833431722 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f726c004e10 con 0x55f34e832640 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f728e628b80 10 client.17634 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f34e844380 con 0x55f34e832640 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.805+0000 7f72657fa700 1 -- 172.21.15.52:0/1833431722 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7274015d00 con 0x55f34e832640 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f72657fa700 1 -- 172.21.15.52:0/1833431722 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f726c01ea90 con 0x55f34e832640 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f728e628b80 10 client.17634 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f728e628b80 10 client.17634 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f728e628b80 20 client.17634 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f728e628b80 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8469c0 0x55f34e848db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f728e628b80 4 client.17634 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f34e78bd10 con 0x55f34e8469c0 2024-06-02T23:19:22.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f7281b41700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8469c0 0x55f34e848db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f72657fa700 10 client.17634 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:19:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.806+0000 7f7281b41700 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8469c0 0x55f34e848db0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5f796983ba1874b0 server_cookie=d5ea8b6e5b29455b in_seq=0 out_seq=0 2024-06-02T23:19:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f72657fa700 1 -- 172.21.15.52:0/1833431722 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f34e78bd10 con 0x55f34e8469c0 2024-06-02T23:19:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f72657fa700 10 client.17634 handle_command_reply: tid=0 2024-06-02T23:19:22.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 client.17634 shutdown 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 msgr2=0x55f34e835fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e835fd0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f726c005950 tx=0x7f726c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 shutdown_connections 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8469c0 0x55f34e848db0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e836510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e838cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f72657fa700 0 client.17634 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 --2- 172.21.15.52:0/1833431722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e835fd0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.807+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 >> 172.21.15.52:0/1833431722 conn(0x55f34e757cc0 msgr2=0x55f34e82b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.808+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 shutdown_connections 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.808+0000 7f728e628b80 1 -- 172.21.15.52:0/1833431722 wait complete. 2024-06-02T23:19:22.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.808+0000 7f728e628b80 20 client.17634 trim_cache size 0 max 16384 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.823+0000 7f728e628b80 1 Processor -- start 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.823+0000 7f728e628b80 1 -- start start 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f728e628b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e82c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f728e628b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e833a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f728e628b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e84e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f728e628b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1aeb70 con 0x55f34e832640 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f728e628b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1b06b0 con 0x55f34e82cf20 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f728e628b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1af240 con 0x55f34e832f60 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e833a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e833a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41324/0 (socket says 172.21.15.52:41324) 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 -- 172.21.15.52:0/1262295515 learned_addr learned my addr 172.21.15.52:0/1262295515 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281b41700 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e82c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7282342700 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e84e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 -- 172.21.15.52:0/1262295515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 msgr2=0x55f34e84e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e84e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 -- 172.21.15.52:0/1262295515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 msgr2=0x55f34e82c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e82c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 -- 172.21.15.52:0/1262295515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f34e828520 con 0x55f34e832640 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7282342700 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e84e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.824+0000 7f7281340700 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e833a50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b4f0 tx=0x7f726c0287b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273bc717ee29807b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.825+0000 7f72677fe700 1 -- 172.21.15.52:0/1262295515 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c004a30 con 0x55f34e832640 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.825+0000 7f72677fe700 1 -- 172.21.15.52:0/1262295515 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f726c004bd0 con 0x55f34e832640 2024-06-02T23:19:22.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.825+0000 7f72677fe700 1 -- 172.21.15.52:0/1262295515 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c005620 con 0x55f34e832640 2024-06-02T23:19:22.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.825+0000 7f728e628b80 1 -- 172.21.15.52:0/1262295515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 msgr2=0x55f34e833a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.825+0000 7f728e628b80 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e833a50 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f726c00b4f0 tx=0x7f726c0287b0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.825+0000 7f728e628b80 1 -- 172.21.15.52:0/1262295515 shutdown_connections 2024-06-02T23:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e832f60 0x55f34e84e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e82cf20 0x55f34e82c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 --2- 172.21.15.52:0/1262295515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e833a50 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- 172.21.15.52:0/1262295515 >> 172.21.15.52:0/1262295515 conn(0x55f34e757cc0 msgr2=0x55f34e833340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- 172.21.15.52:0/1262295515 shutdown_connections 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- 172.21.15.52:0/1262295515 wait complete. 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- start start 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 Processor -- start 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e82cf20 0x55f34e83f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e84e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e84e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1affe0 con 0x55f34e832640 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1af910 con 0x55f34e832f60 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.826+0000 7f728e628b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f727c1acb30 con 0x55f34e82cf20 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f728e628b80 10 client.0 _ll_register_callbacks cb 0x55f34e718920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e84e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e84e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41332/0 (socket says 172.21.15.52:41332) 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 -- 172.21.15.52:0/59449110 learned_addr learned my addr 172.21.15.52:0/59449110 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281b41700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e82cf20 0x55f34e83f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7282342700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e84e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 -- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e82cf20 msgr2=0x55f34e83f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e82cf20 0x55f34e83f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 -- 172.21.15.52:0/59449110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 msgr2=0x55f34e84e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e84e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 -- 172.21.15.52:0/59449110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f34e84ef10 con 0x55f34e832640 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7282342700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e84e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f7281340700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e84e100 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f726c000c00 tx=0x7f726c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56ee344505cd8a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f72657fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f72657fa700 1 -- 172.21.15.52:0/59449110 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c005670 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f72657fa700 1 -- 172.21.15.52:0/59449110 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f726c004020 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.827+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726c004bd0 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f72657fa700 1 -- 172.21.15.52:0/59449110 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f726c04d050 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 10 client.17640 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f34e8b1710 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f72657fa700 1 -- 172.21.15.52:0/59449110 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f726c004bd0 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f72657fa700 1 -- 172.21.15.52:0/59449110 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f726c003680 con 0x55f34e832640 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 10 client.17640 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 10 client.17640 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 20 client.17640 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:22.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8b1d00 0x55f34e8b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 4 client.17640 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.828+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f34e786290 con 0x55f34e8b1d00 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f7281b41700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8b1d00 0x55f34e8b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f72657fa700 10 client.17640 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f7281b41700 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8b1d00 0x55f34e8b40f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6332eafdbb35d8d server_cookie=cab52daa4ec0918e in_seq=0 out_seq=0 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f72657fa700 1 -- 172.21.15.52:0/59449110 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f34e786290 con 0x55f34e8b1d00 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f72657fa700 10 client.17640 handle_command_reply: tid=0 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 client.17640 shutdown 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 msgr2=0x55f34e84e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e84e100 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f726c000c00 tx=0x7f726c013000 comp rx=0 tx=0).stop 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 shutdown_connections 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55f34e8b1d00 0x55f34e8b40f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f34e832f60 0x55f34e84e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f34e82cf20 0x55f34e83f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 --2- 172.21.15.52:0/59449110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f34e832640 0x55f34e84e100 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f72657fa700 0 client.17640 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.829+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 >> 172.21.15.52:0/59449110 conn(0x55f34e757cc0 msgr2=0x55f34e821510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 shutdown_connections 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f728e628b80 1 -- 172.21.15.52:0/59449110 wait complete. 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f728e628b80 20 client.17640 trim_cache size 0 max 16384 2024-06-02T23:19:22.836 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72680a2c20 msgr2=0x7f72680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72680a2c20 0x7f72680a50e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f727c199990 tx=0x7f726c00b410 comp rx=0 tx=0).stop 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 msgr2=0x7f727c197830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 0x7f727c197830 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7278005910 tx=0x7f72780047b0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 shutdown_connections 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f72680a2c20 0x7f72680a50e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f727c107400 0x7f727c1982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f727c101390 0x7f727c197830 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 --2- 172.21.15.52:0/1697890120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727c101d50 0x7f727c197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:22.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 >> 172.21.15.52:0/1697890120 conn(0x7f727c078340 msgr2=0x7f727c100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:22.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 shutdown_connections 2024-06-02T23:19:22.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:22.830+0000 7f7283da5700 1 -- 172.21.15.52:0/1697890120 wait complete. 2024-06-02T23:19:23.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:23 smithi052 ceph-mon[32340]: pgmap v656: 97 pgs: 97 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:19:23.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:23 smithi174 ceph-mon[35994]: pgmap v656: 97 pgs: 97 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:19:23.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:23 smithi123 ceph-mon[35977]: pgmap v656: 97 pgs: 97 active+clean; 68 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 238 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:19:23.585 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:19:23.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 144382 1.90 MB/sec execute 272 sec latency 15.188 ms 2024-06-02T23:19:23.854 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:24.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 145175 1.92 MB/sec execute 273 sec latency 11.368 ms 2024-06-02T23:19:25.282 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:25 smithi052 ceph-mon[32340]: pgmap v657: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-06-02T23:19:25.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:25 smithi174 ceph-mon[35994]: pgmap v657: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-06-02T23:19:25.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:25 smithi123 ceph-mon[35977]: pgmap v657: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 137 op/s 2024-06-02T23:19:25.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 145485 1.92 MB/sec execute 274 sec latency 157.115 ms 2024-06-02T23:19:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 -- 172.21.15.52:0/437471821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318102740 msgr2=0x7fa318102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 --2- 172.21.15.52:0/437471821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318102740 0x7fa318102b20 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa3140099c0 tx=0x7fa314009cd0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 -- 172.21.15.52:0/437471821 shutdown_connections 2024-06-02T23:19:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 --2- 172.21.15.52:0/437471821 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa318107840 0x7fa31810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 --2- 172.21.15.52:0/437471821 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318102740 0x7fa318102b20 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 --2- 172.21.15.52:0/437471821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa3181030f0 0x7fa318107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 -- 172.21.15.52:0/437471821 >> 172.21.15.52:0/437471821 conn(0x7fa3180795f0 msgr2=0x7fa318079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 -- 172.21.15.52:0/437471821 shutdown_connections 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.673+0000 7fa320678700 1 -- 172.21.15.52:0/437471821 wait complete. 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 Processor -- start 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 -- start start 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa318102740 0x7fa318074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3181030f0 0x7fa318074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 0x7fa318075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa318112610 con 0x7fa318107840 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa318112490 con 0x7fa3181030f0 2024-06-02T23:19:25.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.674+0000 7fa320678700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa318112790 con 0x7fa318102740 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31ec15700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 0x7fa318075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3181030f0 0x7fa318074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa318102740 0x7fa318074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31ec15700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 0x7fa318075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35242/0 (socket says 172.21.15.52:35242) 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31ec15700 1 -- 172.21.15.52:0/982484518 learned_addr learned my addr 172.21.15.52:0/982484518 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 -- 172.21.15.52:0/982484518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3181030f0 msgr2=0x7fa318074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3181030f0 0x7fa318074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 -- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 msgr2=0x7fa318075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 0x7fa318075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 -- 172.21.15.52:0/982484518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3181a7010 con 0x7fa318102740 2024-06-02T23:19:25.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31ec15700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 0x7fa318075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31dc13700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3181030f0 0x7fa318074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa31e414700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa318102740 0x7fa318074790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa31400b580 tx=0x7fa314005280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa30f7fe700 1 -- 172.21.15.52:0/982484518 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314026070 con 0x7fa318102740 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa30f7fe700 1 -- 172.21.15.52:0/982484518 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa3140056c0 con 0x7fa318102740 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3181a72a0 con 0x7fa318102740 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.675+0000 7fa30f7fe700 1 -- 172.21.15.52:0/982484518 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31400f9c0 con 0x7fa318102740 2024-06-02T23:19:25.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.676+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3181a76f0 con 0x7fa318102740 2024-06-02T23:19:25.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.678+0000 7fa30f7fe700 1 -- 172.21.15.52:0/982484518 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa314029070 con 0x7fa318102740 2024-06-02T23:19:25.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.678+0000 7fa30f7fe700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3040a2c20 0x7fa3040a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.679+0000 7fa30f7fe700 1 -- 172.21.15.52:0/982484518 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa3140cf730 con 0x7fa318102740 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.679+0000 7fa31dc13700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3040a2c20 0x7fa3040a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.679+0000 7fa31dc13700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3040a2c20 0x7fa3040a50e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa318102090 tx=0x7fa308009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 Processor -- start 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 -- start start 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661640 0x559d7f661a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661f60 0x559d7f65b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559d7f5ad810 con 0x559d7f661f60 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d7f555720 con 0x559d7f65bf20 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d7f5a97d0 con 0x559d7f661640 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661f60 0x559d7f65b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31e414700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661640 0x559d7f661a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31ec15700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661f60 0x559d7f65b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35248/0 (socket says 172.21.15.52:35248) 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31dc13700 1 -- 172.21.15.52:0/4287094352 learned_addr learned my addr 172.21.15.52:0/4287094352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31ec15700 1 -- 172.21.15.52:0/4287094352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661640 msgr2=0x559d7f661a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31ec15700 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661640 0x559d7f661a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31ec15700 1 -- 172.21.15.52:0/4287094352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661f60 msgr2=0x559d7f65b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.680+0000 7fa31ec15700 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661f60 0x559d7f65b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.681+0000 7fa31ec15700 1 -- 172.21.15.52:0/4287094352 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7f54bf50 con 0x559d7f65bf20 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.681+0000 7fa31ec15700 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65c340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa310009430 tx=0x7fa310009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4faf4ba3bc61e018 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.681+0000 7fa30cff9700 1 -- 172.21.15.52:0/4287094352 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310019070 con 0x559d7f65bf20 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.681+0000 7fa30cff9700 1 -- 172.21.15.52:0/4287094352 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa310004db0 con 0x559d7f65bf20 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.681+0000 7fa30cff9700 1 -- 172.21.15.52:0/4287094352 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3100053e0 con 0x559d7f65bf20 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 -- 172.21.15.52:0/4287094352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 msgr2=0x559d7f65c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65c340 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa310009430 tx=0x7fa310009920 comp rx=0 tx=0).stop 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 -- 172.21.15.52:0/4287094352 shutdown_connections 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65c340 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661640 0x559d7f661a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/4287094352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661f60 0x559d7f65b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 -- 172.21.15.52:0/4287094352 >> 172.21.15.52:0/4287094352 conn(0x559d7f586cc0 msgr2=0x559d7f656850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:25.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 -- 172.21.15.52:0/4287094352 shutdown_connections 2024-06-02T23:19:25.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 -- 172.21.15.52:0/4287094352 wait complete. 2024-06-02T23:19:25.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.682+0000 7fa32aefbb80 1 -- start start 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 Processor -- start 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f65bf20 0x559d7f667cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f664fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f661f60 0x559d7f665510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559d7f6689d0 con 0x559d7f661640 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559d7f54dbf0 con 0x559d7f661f60 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d7f668b50 con 0x559d7f65bf20 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f664fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f664fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35254/0 (socket says 172.21.15.52:35254) 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa31dc13700 1 -- 172.21.15.52:0/1791495155 learned_addr learned my addr 172.21.15.52:0/1791495155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:25.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa31e414700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f65bf20 0x559d7f667cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa32aefbb80 10 client.0 _ll_register_callbacks cb 0x559d7f547920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.683+0000 7fa31ec15700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f661f60 0x559d7f665510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31e414700 1 -- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f661f60 msgr2=0x559d7f665510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31e414700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f661f60 0x559d7f665510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31e414700 1 -- 172.21.15.52:0/1791495155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 msgr2=0x559d7f664fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31e414700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f664fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31e414700 1 -- 172.21.15.52:0/1791495155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7f4fc8d0 con 0x559d7f65bf20 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31dc13700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f664fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa31e414700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f65bf20 0x559d7f667cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3140098e0 tx=0x7fa31409b950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a25c605a34049bbf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa301ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa301ffb700 1 -- 172.21.15.52:0/1791495155 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31409c6f0 con 0x559d7f65bf20 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa310004db0 con 0x559d7f65bf20 2024-06-02T23:19:25.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa301ffb700 1 -- 172.21.15.52:0/1791495155 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa3140a4740 con 0x559d7f65bf20 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa301ffb700 1 -- 172.21.15.52:0/1791495155 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa3140b8020 con 0x559d7f65bf20 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa32aefbb80 10 client.26974 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559d7f6689d0 con 0x559d7f65bf20 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.684+0000 7fa301ffb700 1 -- 172.21.15.52:0/1791495155 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa310004db0 con 0x559d7f65bf20 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa301ffb700 1 -- 172.21.15.52:0/1791495155 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa3140baba0 con 0x559d7f65bf20 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa32aefbb80 10 client.26974 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa32aefbb80 10 client.26974 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa32aefbb80 20 client.26974 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6757c0 0x559d7f677c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa32aefbb80 4 client.26974 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d7f5bad10 con 0x559d7f6757c0 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa31dc13700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6757c0 0x559d7f677c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa31dc13700 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6757c0 0x559d7f677c40 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d7aa0020577ae2 server_cookie=92fa2c6d810f4275 in_seq=0 out_seq=0 2024-06-02T23:19:25.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.685+0000 7fa301ffb700 10 client.26974 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa301ffb700 1 -- 172.21.15.52:0/1791495155 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559d7f5bad10 con 0x559d7f6757c0 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa301ffb700 10 client.26974 handle_command_reply: tid=0 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 client.26974 shutdown 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f65bf20 msgr2=0x559d7f667cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f65bf20 0x559d7f667cd0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa3140098e0 tx=0x7fa31409b950 comp rx=0 tx=0).stop 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 shutdown_connections 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6757c0 0x559d7f677c40 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f661f60 0x559d7f665510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f65bf20 0x559d7f667cd0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa301ffb700 0 client.26974 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1791495155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f664fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 >> 172.21.15.52:0/1791495155 conn(0x559d7f586cc0 msgr2=0x559d7f65a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:25.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.686+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 shutdown_connections 2024-06-02T23:19:25.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.687+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1791495155 wait complete. 2024-06-02T23:19:25.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.687+0000 7fa32aefbb80 20 client.26974 trim_cache size 0 max 16384 2024-06-02T23:19:25.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 Processor -- start 2024-06-02T23:19:25.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 -- start start 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f64fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f650340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181aec30 con 0x559d7f661640 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b0770 con 0x559d7f65bf20 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181af300 con 0x559d7f661f60 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f64fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f64fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35260/0 (socket says 172.21.15.52:35260) 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 -- 172.21.15.52:0/3664028633 learned_addr learned my addr 172.21.15.52:0/3664028633 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31e414700 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31ec15700 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f650340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 -- 172.21.15.52:0/3664028633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 msgr2=0x559d7f650340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f650340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 -- 172.21.15.52:0/3664028633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 msgr2=0x559d7f65b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.703+0000 7fa31dc13700 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.704+0000 7fa31dc13700 1 -- 172.21.15.52:0/3664028633 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d7f6782c0 con 0x559d7f661640 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.704+0000 7fa31dc13700 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f64fe00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa3080094f0 tx=0x7fa308014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c31741324b18c70e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.704+0000 7fa303fff700 1 -- 172.21.15.52:0/3664028633 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308029650 con 0x559d7f661640 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.704+0000 7fa303fff700 1 -- 172.21.15.52:0/3664028633 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa308029ca0 con 0x559d7f661640 2024-06-02T23:19:25.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.704+0000 7fa303fff700 1 -- 172.21.15.52:0/3664028633 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30801ec50 con 0x559d7f661640 2024-06-02T23:19:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 -- 172.21.15.52:0/3664028633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 msgr2=0x559d7f64fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f64fe00 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa3080094f0 tx=0x7fa308014000 comp rx=0 tx=0).stop 2024-06-02T23:19:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 -- 172.21.15.52:0/3664028633 shutdown_connections 2024-06-02T23:19:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f650340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f65b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/3664028633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f64fe00 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 -- 172.21.15.52:0/3664028633 >> 172.21.15.52:0/3664028633 conn(0x559d7f586cc0 msgr2=0x559d7f656380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 -- 172.21.15.52:0/3664028633 shutdown_connections 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 -- 172.21.15.52:0/3664028633 wait complete. 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 -- start start 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.705+0000 7fa32aefbb80 1 Processor -- start 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f66e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f66e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f67cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181b00a0 con 0x559d7f661640 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181af9d0 con 0x559d7f65bf20 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3181acbf0 con 0x559d7f661f60 2024-06-02T23:19:25.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 10 client.0 _ll_register_callbacks cb 0x559d7f547920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f66e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31e414700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f66e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f66e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35270/0 (socket says 172.21.15.52:35270) 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 -- 172.21.15.52:0/1508668021 learned_addr learned my addr 172.21.15.52:0/1508668021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31ec15700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f67cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 -- 172.21.15.52:0/1508668021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 msgr2=0x559d7f67cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f67cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 -- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 msgr2=0x559d7f66e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f66e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 -- 172.21.15.52:0/1508668021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d7f67d500 con 0x559d7f661640 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31e414700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f66e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa31dc13700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f66e710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa30801e480 tx=0x7fa30801e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7015a34178f77f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:25.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa301ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa301ffb700 1 -- 172.21.15.52:0/1508668021 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30801e830 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa301ffb700 1 -- 172.21.15.52:0/1508668021 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa30801e9d0 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.706+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa308029ca0 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa301ffb700 1 -- 172.21.15.52:0/1508668021 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa30801ebd0 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 10 client.17658 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559d7f6e0140 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa301ffb700 1 -- 172.21.15.52:0/1508668021 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa308029ca0 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa301ffb700 1 -- 172.21.15.52:0/1508668021 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa308004750 con 0x559d7f661640 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 10 client.17658 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 20 client.17658 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6e0840 0x559d7f6e2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 4 client.17658 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:19:25.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d7f5b5290 con 0x559d7f6e0840 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.707+0000 7fa31e414700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6e0840 0x559d7f6e2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa31e414700 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6e0840 0x559d7f6e2c60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44690e21089a454f server_cookie=7fc1213fc1d927ca in_seq=0 out_seq=0 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa301ffb700 10 client.17658 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa301ffb700 1 -- 172.21.15.52:0/1508668021 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d7f5b5290 con 0x559d7f6e0840 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa301ffb700 10 client.17658 handle_command_reply: tid=0 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 client.17658 shutdown 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 msgr2=0x559d7f66e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f66e710 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa30801e480 tx=0x7fa30801e4b0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 shutdown_connections 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559d7f6e0840 0x559d7f6e2c60 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa301ffb700 0 client.17658 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d7f661f60 0x559d7f67cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559d7f65bf20 0x559d7f66e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 --2- 172.21.15.52:0/1508668021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559d7f661640 0x559d7f66e710 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 >> 172.21.15.52:0/1508668021 conn(0x559d7f586cc0 msgr2=0x559d7f64fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:25.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 shutdown_connections 2024-06-02T23:19:25.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.708+0000 7fa32aefbb80 1 -- 172.21.15.52:0/1508668021 wait complete. 2024-06-02T23:19:25.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa32aefbb80 20 client.17658 trim_cache size 0 max 16384 2024-06-02T23:19:25.715 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3040a2c20 msgr2=0x7fa3040a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3040a2c20 0x7fa3040a50e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa318102090 tx=0x7fa308009260 comp rx=0 tx=0).stop 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa318102740 msgr2=0x7fa318074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa318102740 0x7fa318074790 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa31400b580 tx=0x7fa314005280 comp rx=0 tx=0).stop 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 shutdown_connections 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3040a2c20 0x7fa3040a50e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa318107840 0x7fa318075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa318102740 0x7fa318074790 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 --2- 172.21.15.52:0/982484518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3181030f0 0x7fa318074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 >> 172.21.15.52:0/982484518 conn(0x7fa3180795f0 msgr2=0x7fa318100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 shutdown_connections 2024-06-02T23:19:25.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:25.709+0000 7fa320678700 1 -- 172.21.15.52:0/982484518 wait complete. 2024-06-02T23:19:26.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 145821 1.92 MB/sec execute 275 sec latency 18.481 ms 2024-06-02T23:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/1898530284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 msgr2=0x7f6dd4101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/1898530284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd4101f70 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009900 tx=0x7f6dc402a410 comp rx=0 tx=0).stop 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/1898530284 shutdown_connections 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/1898530284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dd41024b0 0x7f6dd4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/1898530284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd4101f70 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/1898530284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd4100900 0x7f6dd4100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/1898530284 >> 172.21.15.52:0/1898530284 conn(0x7f6dd40fbe70 msgr2=0x7f6dd40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/1898530284 shutdown_connections 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/1898530284 wait complete. 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.137+0000 7f6ddbbb0640 1 Processor -- start 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 -- start start 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dd4100900 0x7f6dd4069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd419c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd41024b0 0x7f6dd419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd410c480 con 0x7f6dd4101af0 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd410c300 con 0x7f6dd41024b0 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6ddbbb0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd410c600 con 0x7f6dd4100900 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd419c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dd4100900 0x7f6dd4069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd419c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35292/0 (socket says 172.21.15.52:35292) 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 -- 172.21.15.52:0/3494362622 learned_addr learned my addr 172.21.15.52:0/3494362622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dda14d640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd41024b0 0x7f6dd419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 -- 172.21.15.52:0/3494362622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dd4100900 msgr2=0x7f6dd4069be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dd4100900 0x7f6dd4069be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 -- 172.21.15.52:0/3494362622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd41024b0 msgr2=0x7f6dd419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd41024b0 0x7f6dd419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 -- 172.21.15.52:0/3494362622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd419f7c0 con 0x7f6dd4101af0 2024-06-02T23:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.138+0000 7f6dd914b640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd419c8c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40314e0 tx=0x7f6dc4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.139+0000 7f6dcaffd640 1 -- 172.21.15.52:0/3494362622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc403f070 con 0x7f6dd4101af0 2024-06-02T23:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.139+0000 7f6dcaffd640 1 -- 172.21.15.52:0/3494362622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dc4031c40 con 0x7f6dd4101af0 2024-06-02T23:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.139+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd419fa50 con 0x7f6dd4101af0 2024-06-02T23:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.139+0000 7f6dcaffd640 1 -- 172.21.15.52:0/3494362622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4035030 con 0x7f6dd4101af0 2024-06-02T23:19:27.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.139+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd419ff90 con 0x7f6dd4101af0 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.141+0000 7f6dcaffd640 1 -- 172.21.15.52:0/3494362622 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6dc4042070 con 0x7f6dd4101af0 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 Processor -- start 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 -- start start 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef04430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 0x555ffef09500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ffee674f0 con 0x555ffef04a00 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ffee05870 con 0x555ffef04010 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dde99e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555ffee634b0 con 0x555ffef09b70 2024-06-02T23:19:27.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dcaffd640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6dc00a2cb0 0x7f6dc00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 0x555ffef09500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 0x555ffef09500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35306/0 (socket says 172.21.15.52:35306) 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dd994c640 1 -- 172.21.15.52:0/1042148385 learned_addr learned my addr 172.21.15.52:0/1042148385 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dd914b640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dda14d640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef04430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.142+0000 7f6dda14d640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6dc00a2cb0 0x7f6dc00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd994c640 1 -- 172.21.15.52:0/1042148385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 msgr2=0x555ffef0c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd994c640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd994c640 1 -- 172.21.15.52:0/1042148385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 msgr2=0x555ffef04430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd994c640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef04430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd994c640 1 -- 172.21.15.52:0/1042148385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffee219b0 con 0x555ffef04a00 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dda14d640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef04430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:27.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd914b640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dcaffd640 1 -- 172.21.15.52:0/3494362622 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6dc40ef6b0 con 0x7f6dd4101af0 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dda14d640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6dc00a2cb0 0x7f6dc00a5170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x555ffef05260 tx=0x7f6dcc009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dd994c640 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 0x555ffef09500 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009c80 tx=0x7f6dd0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce48211e09cf50c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dbffff640 1 -- 172.21.15.52:0/1042148385 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd001e070 con 0x555ffef04a00 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dbffff640 1 -- 172.21.15.52:0/1042148385 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd00198e0 con 0x555ffef04a00 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.143+0000 7f6dbffff640 1 -- 172.21.15.52:0/1042148385 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd000e8b0 con 0x555ffef04a00 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 -- 172.21.15.52:0/1042148385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 msgr2=0x555ffef09500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 0x555ffef09500 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009c80 tx=0x7f6dd0011810 comp rx=0 tx=0).stop 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 -- 172.21.15.52:0/1042148385 shutdown_connections 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04a00 0x555ffef09500 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 --2- 172.21.15.52:0/1042148385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef04430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 -- 172.21.15.52:0/1042148385 >> 172.21.15.52:0/1042148385 conn(0x555ffeeff680 msgr2=0x555ffeeffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 -- 172.21.15.52:0/1042148385 shutdown_connections 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 -- 172.21.15.52:0/1042148385 wait complete. 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 -- start start 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.144+0000 7f6dde99e740 1 Processor -- start 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef0e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef0e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 0x555ffef15830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ffef0cb20 con 0x555ffef09b70 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ffedf0ae0 con 0x555ffef04010 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555ffef0cca0 con 0x555ffef04a00 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dde99e740 10 client.0 _ll_register_callbacks cb 0x555ffee299b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 0x555ffef15830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef0e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 0x555ffef15830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35320/0 (socket says 172.21.15.52:35320) 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef0e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41780/0 (socket says 172.21.15.52:41780) 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 -- 172.21.15.52:0/3675950606 learned_addr learned my addr 172.21.15.52:0/3675950606 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dd914b640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef0e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 -- 172.21.15.52:0/3675950606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 msgr2=0x555ffef0e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef0e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 -- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 msgr2=0x555ffef0e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef0e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dda14d640 1 -- 172.21.15.52:0/3675950606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffeb7b210 con 0x555ffef09b70 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.145+0000 7f6dd994c640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef0e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dda14d640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 0x555ffef15830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc019180 tx=0x7f6dcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61676cac148319e0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:27.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/3675950606 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc02d060 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/3675950606 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6dcc004730 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd00198e0 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/3675950606 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6dcc020460 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 10 client.17670 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ffec47620 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/3675950606 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd00198e0 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/3675950606 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6dcc027030 con 0x555ffef09b70 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 10 client.17670 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 10 client.17670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 20 client.17670 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.146+0000 7f6dde99e740 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef1a180 0x555ffef1c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.147+0000 7f6dde99e740 4 client.17670 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.147+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ffee7af10 con 0x555ffef1a180 2024-06-02T23:19:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.147+0000 7f6dd994c640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef1a180 0x555ffef1c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.147+0000 7f6dd994c640 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef1a180 0x555ffef1c5f0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c8e8489af075a06 server_cookie=241f8af1953866a9 in_seq=0 out_seq=0 2024-06-02T23:19:27.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.147+0000 7f6dbd7fa640 10 client.17670 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/3675950606 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ffee7af10 con 0x555ffef1a180 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dbd7fa640 10 client.17670 handle_command_reply: tid=0 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 client.17670 shutdown 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 msgr2=0x555ffef15830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 0x555ffef15830 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc019180 tx=0x7f6dcc004a40 comp rx=0 tx=0).stop 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 shutdown_connections 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef1a180 0x555ffef1c5f0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef09b70 0x555ffef15830 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 --2- 172.21.15.52:0/3675950606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04010 0x555ffef0e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dbd7fa640 0 client.17670 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 >> 172.21.15.52:0/3675950606 conn(0x555ffeeff680 msgr2=0x555ffeefe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 shutdown_connections 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.148+0000 7f6dde99e740 1 -- 172.21.15.52:0/3675950606 wait complete. 2024-06-02T23:19:27.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.149+0000 7f6dde99e740 20 client.17670 trim_cache size 0 max 16384 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.167+0000 7f6dde99e740 1 Processor -- start 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.167+0000 7f6dde99e740 1 -- start start 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef0d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04a00 0x555ffef157f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dde99e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41a8d30 con 0x555ffef04010 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dde99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41aa870 con 0x555ffef04a00 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dde99e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41a9400 con 0x555ffef09b70 2024-06-02T23:19:27.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef0d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef0d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35324/0 (socket says 172.21.15.52:35324) 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 -- 172.21.15.52:0/2093399416 learned_addr learned my addr 172.21.15.52:0/2093399416 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dda14d640 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd914b640 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04a00 0x555ffef157f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 -- 172.21.15.52:0/2093399416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 msgr2=0x555ffef0d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 -- 172.21.15.52:0/2093399416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04a00 msgr2=0x555ffef157f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04a00 0x555ffef157f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 -- 172.21.15.52:0/2093399416 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ffef05290 con 0x555ffef04010 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd914b640 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04a00 0x555ffef157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dd994c640 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef0d8d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0000be0 tx=0x7f6dd0019a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3d09cc99a8c2a90 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:27.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dbf7fe640 1 -- 172.21.15.52:0/2093399416 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0029070 con 0x555ffef04010 2024-06-02T23:19:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dbf7fe640 1 -- 172.21.15.52:0/2093399416 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd001f040 con 0x555ffef04010 2024-06-02T23:19:27.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.168+0000 7f6dbf7fe640 1 -- 172.21.15.52:0/2093399416 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd001e040 con 0x555ffef04010 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 -- 172.21.15.52:0/2093399416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 msgr2=0x555ffef0d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef0d8d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0000be0 tx=0x7f6dd0019a10 comp rx=0 tx=0).stop 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 -- 172.21.15.52:0/2093399416 shutdown_connections 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef09b70 0x555ffef0d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef04a00 0x555ffef157f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 --2- 172.21.15.52:0/2093399416 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef0d8d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 -- 172.21.15.52:0/2093399416 >> 172.21.15.52:0/2093399416 conn(0x555ffeeff680 msgr2=0x555ffef09270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 -- 172.21.15.52:0/2093399416 shutdown_connections 2024-06-02T23:19:27.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 -- 172.21.15.52:0/2093399416 wait complete. 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.169+0000 7f6dde99e740 1 -- start start 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 Processor -- start 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef152a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef157e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef09b70 0x555ffef11e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41aa1a0 con 0x555ffef04010 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41a9ad0 con 0x555ffef09b70 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd41a6d10 con 0x555ffef04a00 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dde99e740 10 client.0 _ll_register_callbacks cb 0x7f6dcc019040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef152a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dd994c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef152a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35332/0 (socket says 172.21.15.52:35332) 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dd994c640 1 -- 172.21.15.52:0/4151510540 learned_addr learned my addr 172.21.15.52:0/4151510540 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dda14d640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef09b70 0x555ffef11e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dd914b640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef157e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dd994c640 1 -- 172.21.15.52:0/4151510540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 msgr2=0x555ffef157e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.170+0000 7f6dd994c640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef157e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dd994c640 1 -- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef09b70 msgr2=0x555ffef11e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dd994c640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef09b70 0x555ffef11e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dd994c640 1 -- 172.21.15.52:0/4151510540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ffef12660 con 0x555ffef04010 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dd994c640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef152a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009c50 tx=0x7f6dd0019910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233e9d2f3a892409 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/4151510540 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd001e040 con 0x555ffef04010 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/4151510540 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6dd00050c0 con 0x555ffef04010 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dd001f040 con 0x555ffef04010 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/4151510540 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6dd0029040 con 0x555ffef04010 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dde99e740 10 client.17676 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555ffef83d30 con 0x555ffef04010 2024-06-02T23:19:27.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/4151510540 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6dd001f040 con 0x555ffef04010 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.171+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/4151510540 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6dd0025550 con 0x555ffef04010 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dde99e740 10 client.17676 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dde99e740 10 client.17676 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dde99e740 20 client.17676 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dde99e740 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef844e0 0x555ffef86900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dde99e740 4 client.17676 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ffee74c20 con 0x555ffef844e0 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dd914b640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef844e0 0x555ffef86900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dbd7fa640 10 client.17676 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:27.177 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.172+0000 7f6dd914b640 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef844e0 0x555ffef86900 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd5bd8cbfcac1ac server_cookie=fa01324b6118fee7 in_seq=0 out_seq=0 2024-06-02T23:19:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.194+0000 7f6dbd7fa640 1 -- 172.21.15.52:0/4151510540 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555ffee74c20 con 0x555ffef844e0 2024-06-02T23:19:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.194+0000 7f6dbd7fa640 10 client.17676 handle_command_reply: tid=0 2024-06-02T23:19:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.194+0000 7f6dde99e740 1 client.17676 shutdown 2024-06-02T23:19:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 msgr2=0x555ffef152a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef152a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6dd0009c50 tx=0x7f6dd0019910 comp rx=0 tx=0).stop 2024-06-02T23:19:27.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 shutdown_connections 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ffef844e0 0x555ffef86900 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ffef04a00 0x555ffef157e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dbd7fa640 0 client.17676 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ffef09b70 0x555ffef11e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 --2- 172.21.15.52:0/4151510540 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ffef04010 0x555ffef152a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 >> 172.21.15.52:0/4151510540 conn(0x555ffeeff680 msgr2=0x555ffef05be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 shutdown_connections 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 1 -- 172.21.15.52:0/4151510540 wait complete. 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6dde99e740 20 client.17676 trim_cache size 0 max 16384 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:19:27.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:19:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:19:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6dc00a2cb0 msgr2=0x7f6dc00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.195+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6dc00a2cb0 0x7f6dc00a5170 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x555ffef05260 tx=0x7f6dcc009500 comp rx=0 tx=0).stop 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 msgr2=0x7f6dd419c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd419c8c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40314e0 tx=0x7f6dc4003c10 comp rx=0 tx=0).stop 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 shutdown_connections 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6dc00a2cb0 0x7f6dc00a5170 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6dd41024b0 0x7f6dd419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6dd4101af0 0x7f6dd419c8c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 --2- 172.21.15.52:0/3494362622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6dd4100900 0x7f6dd4069be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 >> 172.21.15.52:0/3494362622 conn(0x7f6dd40fbe70 msgr2=0x7f6dd4104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 shutdown_connections 2024-06-02T23:19:27.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:27.196+0000 7f6ddbbb0640 1 -- 172.21.15.52:0/3494362622 wait complete. 2024-06-02T23:19:27.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:27 smithi174 ceph-mon[35994]: pgmap v658: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-02T23:19:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:27 smithi052 ceph-mon[32340]: pgmap v658: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-02T23:19:27.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:27 smithi123 ceph-mon[35977]: pgmap v658: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 1.4 MiB/s wr, 188 op/s 2024-06-02T23:19:27.625 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:19:27.625 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:19:27.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 146086 1.92 MB/sec execute 276 sec latency 15.932 ms 2024-06-02T23:19:27.891 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:28.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 146627 1.92 MB/sec execute 277 sec latency 13.179 ms 2024-06-02T23:19:29.289 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:29 smithi052 ceph-mon[32340]: pgmap v659: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:19:29.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:29 smithi174 ceph-mon[35994]: pgmap v659: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:19:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:29 smithi123 ceph-mon[35977]: pgmap v659: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:19:29.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 147173 1.92 MB/sec execute 278 sec latency 33.727 ms 2024-06-02T23:19:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 -- 172.21.15.52:0/1643626211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f8108500 msgr2=0x7f80f81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/1643626211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f8108500 0x7f80f81088e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f80e8009990 tx=0x7f80e8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 -- 172.21.15.52:0/1643626211 shutdown_connections 2024-06-02T23:19:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/1643626211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806ab30 0x7f80f810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/1643626211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f8108500 0x7f80f81088e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/1643626211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f806a170 0x7f80f806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 -- 172.21.15.52:0/1643626211 >> 172.21.15.52:0/1643626211 conn(0x7f80f8078340 msgr2=0x7f80f8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 -- 172.21.15.52:0/1643626211 shutdown_connections 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 -- 172.21.15.52:0/1643626211 wait complete. 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 Processor -- start 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.547+0000 7f80fe5fa700 1 -- start start 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fe5fa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806a170 0x7f80f8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fe5fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 0x7f80f81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fe5fa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f8108500 0x7f80f8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fe5fa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8112610 con 0x7f80f806ab30 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fe5fa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8112490 con 0x7f80f8108500 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fe5fa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80f8112790 con 0x7f80f806a170 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 0x7f80f81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806a170 0x7f80f8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 0x7f80f81019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35364/0 (socket says 172.21.15.52:35364) 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80effff700 1 -- 172.21.15.52:0/4106550367 learned_addr learned my addr 172.21.15.52:0/4106550367 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f8108500 0x7f80f8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 -- 172.21.15.52:0/4106550367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806a170 msgr2=0x7f80f8101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806a170 0x7f80f8101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 -- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 msgr2=0x7f80f81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 0x7f80f81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 -- 172.21.15.52:0/4106550367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80f81a70b0 con 0x7f80f8108500 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80effff700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 0x7f80f81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80f7fff700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806a170 0x7f80f8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.548+0000 7f80fcb97700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f8108500 0x7f80f8101ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80f0007f10 tx=0x7f80f000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.549+0000 7f80f5ffb700 1 -- 172.21.15.52:0/4106550367 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0004db0 con 0x7f80f8108500 2024-06-02T23:19:29.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.549+0000 7f80f5ffb700 1 -- 172.21.15.52:0/4106550367 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f0003a00 con 0x7f80f8108500 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.549+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80f81a73a0 con 0x7f80f8108500 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.549+0000 7f80f5ffb700 1 -- 172.21.15.52:0/4106550367 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0004570 con 0x7f80f8108500 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.549+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80f81a7890 con 0x7f80f8108500 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.552+0000 7f80f5ffb700 1 -- 172.21.15.52:0/4106550367 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f80f001c020 con 0x7f80f8108500 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.552+0000 7f80f5ffb700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80e40a2c20 0x7f80e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f80f7fff700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80e40a2c20 0x7f80e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f80f5ffb700 1 -- 172.21.15.52:0/4106550367 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f80f00ce6e0 con 0x7f80f8108500 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f80f7fff700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80e40a2c20 0x7f80e40a50e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f80e80098e0 tx=0x7f80e8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 Processor -- start 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 -- start start 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9d8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9d33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fded924890 con 0x55fded9d2fa0 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fded8cc7a0 con 0x55fded9d8fe0 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.553+0000 7f8108e7db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fded920850 con 0x55fded9d86c0 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9d8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41806/0 (socket says 172.21.15.52:41806) 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80fcb97700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9d33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9d8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52892/0 (socket says 172.21.15.52:52892) 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 -- 172.21.15.52:0/3057007368 learned_addr learned my addr 172.21.15.52:0/3057007368 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 -- 172.21.15.52:0/3057007368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 msgr2=0x55fded9d8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9d8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 -- 172.21.15.52:0/3057007368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 msgr2=0x55fded9d33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9d33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 -- 172.21.15.52:0/3057007368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fded8c2fd0 con 0x55fded9d8fe0 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80effff700 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9d2a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f80e0011650 tx=0x7f80e0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c0f320602ff3ed2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80ee7fc700 1 -- 172.21.15.52:0/3057007368 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e0004a30 con 0x55fded9d8fe0 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80ee7fc700 1 -- 172.21.15.52:0/3057007368 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e00054d0 con 0x55fded9d8fe0 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.554+0000 7f80ee7fc700 1 -- 172.21.15.52:0/3057007368 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e00199f0 con 0x55fded9d8fe0 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 -- 172.21.15.52:0/3057007368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 msgr2=0x55fded9d2a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9d2a60 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f80e0011650 tx=0x7f80e0009f90 comp rx=0 tx=0).stop 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 -- 172.21.15.52:0/3057007368 shutdown_connections 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9d33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9d8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3057007368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9d2a60 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 -- 172.21.15.52:0/3057007368 >> 172.21.15.52:0/3057007368 conn(0x55fded8fdd40 msgr2=0x55fded9cd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 -- 172.21.15.52:0/3057007368 shutdown_connections 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 -- 172.21.15.52:0/3057007368 wait complete. 2024-06-02T23:19:29.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.555+0000 7f8108e7db80 1 -- start start 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 Processor -- start 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fded9dfa50 con 0x55fded9d86c0 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fded8c4c70 con 0x55fded9d2fa0 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fded9dfbd0 con 0x55fded9d8fe0 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f80effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f8108e7db80 10 client.0 _ll_register_callbacks cb 0x55fded8be9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f80effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9dc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35388/0 (socket says 172.21.15.52:35388) 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f80effff700 1 -- 172.21.15.52:0/2015567959 learned_addr learned my addr 172.21.15.52:0/2015567959 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f80f7fff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.556+0000 7f80fcb97700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80f7fff700 1 -- 172.21.15.52:0/2015567959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 msgr2=0x55fded9dc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80f7fff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9dc5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80f7fff700 1 -- 172.21.15.52:0/2015567959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 msgr2=0x55fded9dc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80f7fff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9dc090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80f7fff700 1 -- 172.21.15.52:0/2015567959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fded873950 con 0x55fded9d2fa0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80effff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80fcb97700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80f7fff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9ded90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006010 tx=0x7f80e8009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cc1fa32b31adb4e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80d77fe700 1 -- 172.21.15.52:0/2015567959 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80e800f870 con 0x55fded9d2fa0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80d77fe700 1 -- 172.21.15.52:0/2015567959 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f80e800fa10 con 0x55fded9d2fa0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80e00054d0 con 0x55fded9d2fa0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80d77fe700 1 -- 172.21.15.52:0/2015567959 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80e800fc10 con 0x55fded9d2fa0 2024-06-02T23:19:29.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f8108e7db80 10 client.27179 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fded9ea390 con 0x55fded9d2fa0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.557+0000 7f80d77fe700 1 -- 172.21.15.52:0/2015567959 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80e00054d0 con 0x55fded9d2fa0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f80d77fe700 1 -- 172.21.15.52:0/2015567959 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f80e8022c40 con 0x55fded9d2fa0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f8108e7db80 10 client.27179 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f8108e7db80 10 client.27179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f8108e7db80 20 client.27179 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f8108e7db80 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fded9ec9d0 0x55fded9eedf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f8108e7db80 4 client.27179 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fded931d90 con 0x55fded9ec9d0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f80effff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fded9ec9d0 0x55fded9eedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f80effff700 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fded9ec9d0 0x55fded9eedf0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e566ffe604beb6d server_cookie=8da0b7b22e778c45 in_seq=0 out_seq=0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.558+0000 7f80d77fe700 10 client.27179 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.559+0000 7f80d77fe700 1 -- 172.21.15.52:0/2015567959 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fded931d90 con 0x55fded9ec9d0 2024-06-02T23:19:29.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.559+0000 7f80d77fe700 10 client.27179 handle_command_reply: tid=0 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.559+0000 7f8108e7db80 1 client.27179 shutdown 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.559+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 msgr2=0x55fded9ded90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.559+0000 7f8108e7db80 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9ded90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f80e8006010 tx=0x7f80e8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 shutdown_connections 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fded9ec9d0 0x55fded9eedf0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9ded90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f80d77fe700 0 client.27179 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 --2- 172.21.15.52:0/2015567959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9dc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 >> 172.21.15.52:0/2015567959 conn(0x55fded8fdd40 msgr2=0x55fded9d1a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 shutdown_connections 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 1 -- 172.21.15.52:0/2015567959 wait complete. 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.560+0000 7f8108e7db80 20 client.27179 trim_cache size 0 max 16384 2024-06-02T23:19:29.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 Processor -- start 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 -- start start 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9d2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9c7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81aec40 con 0x55fded9d86c0 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b0780 con 0x55fded9d2fa0 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f8108e7db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81af310 con 0x55fded9d8fe0 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9d2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80fcb97700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35394/0 (socket says 172.21.15.52:35394) 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9d2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41814/0 (socket says 172.21.15.52:41814) 2024-06-02T23:19:29.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80effff700 1 -- 172.21.15.52:0/3194824345 learned_addr learned my addr 172.21.15.52:0/3194824345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80fcb97700 1 -- 172.21.15.52:0/3194824345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 msgr2=0x55fded9d2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80fcb97700 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9d2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80fcb97700 1 -- 172.21.15.52:0/3194824345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 msgr2=0x55fded9c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80fcb97700 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80fcb97700 1 -- 172.21.15.52:0/3194824345 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fded9ce580 con 0x55fded9d8fe0 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80f7fff700 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.584+0000 7f80effff700 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.585+0000 7f80fcb97700 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9d9ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80f0007ee0 tx=0x7f80f0009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a3491e839dce5b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.585+0000 7f80edffb700 1 -- 172.21.15.52:0/3194824345 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f00b5070 con 0x55fded9d8fe0 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.585+0000 7f80edffb700 1 -- 172.21.15.52:0/3194824345 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f00a3be0 con 0x55fded9d8fe0 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.585+0000 7f80edffb700 1 -- 172.21.15.52:0/3194824345 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0019040 con 0x55fded9d8fe0 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 -- 172.21.15.52:0/3194824345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 msgr2=0x55fded9d9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9d9ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f80f0007ee0 tx=0x7f80f0009310 comp rx=0 tx=0).stop 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 -- 172.21.15.52:0/3194824345 shutdown_connections 2024-06-02T23:19:29.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d8fe0 0x55fded9d9ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d2fa0 0x55fded9d2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 --2- 172.21.15.52:0/3194824345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d86c0 0x55fded9c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 -- 172.21.15.52:0/3194824345 >> 172.21.15.52:0/3194824345 conn(0x55fded8fdd40 msgr2=0x55fded9d93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 -- 172.21.15.52:0/3194824345 shutdown_connections 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 -- 172.21.15.52:0/3194824345 wait complete. 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 -- start start 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.586+0000 7f8108e7db80 1 Processor -- start 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9e4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9e53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9e1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81b00b0 con 0x55fded9d2fa0 2024-06-02T23:19:29.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81af9e0 con 0x55fded9d8fe0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80f81acbe0 con 0x55fded9d86c0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 10 client.0 _ll_register_callbacks cb 0x55fded8be9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9e4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80f7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9e4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35398/0 (socket says 172.21.15.52:35398) 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80f7fff700 1 -- 172.21.15.52:0/485720351 learned_addr learned my addr 172.21.15.52:0/485720351 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9e1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 -- 172.21.15.52:0/485720351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 msgr2=0x55fded9e53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9e53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 -- 172.21.15.52:0/485720351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 msgr2=0x55fded9e4e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9e4e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 -- 172.21.15.52:0/485720351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fded9e23a0 con 0x55fded9d8fe0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80f7fff700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80fcb97700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9e1e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009440 tx=0x7f80f0009ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7acf3c918c11cb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:29.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80d77fe700 1 -- 172.21.15.52:0/485720351 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f00b5030 con 0x55fded9d8fe0 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f80d77fe700 1 -- 172.21.15.52:0/485720351 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f80f009b1d0 con 0x55fded9d8fe0 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.587+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f80f00a3be0 con 0x55fded9d8fe0 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f80d77fe700 1 -- 172.21.15.52:0/485720351 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f80f0099330 con 0x55fded9d8fe0 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "ce4d6f04-9842-491d-b7ae-b9b66ec65b32", 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:19:29.704 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 10 client.27185 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fdeda57ee0 con 0x55fded9d8fe0 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f80d77fe700 1 -- 172.21.15.52:0/485720351 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f80f00a3be0 con 0x55fded9d8fe0 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f80d77fe700 1 -- 172.21.15.52:0/485720351 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f80f00b9cd0 con 0x55fded9d8fe0 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 10 client.27185 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 10 client.27185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 20 client.27185 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fdeda585c0 0x55fdeda5a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 4 client.27185 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fded92c310 con 0x55fdeda585c0 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.588+0000 7f80f7fff700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fdeda585c0 0x55fdeda5a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:29.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.589+0000 7f80f7fff700 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fdeda585c0 0x55fdeda5a9e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4779a7ec6a19dfc7 server_cookie=3a21ecd429fff1db in_seq=0 out_seq=0 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.589+0000 7f80d77fe700 10 client.27185 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f80d77fe700 1 -- 172.21.15.52:0/485720351 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fded92c310 con 0x55fdeda585c0 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f80d77fe700 10 client.27185 handle_command_reply: tid=0 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 client.27185 shutdown 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 msgr2=0x55fded9e1e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9e1e60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009440 tx=0x7f80f0009ea0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 shutdown_connections 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fdeda585c0 0x55fdeda5a9e0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fded9d8fe0 0x55fded9e1e60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fded9d2fa0 0x55fded9e4e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f80d77fe700 0 client.27185 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 --2- 172.21.15.52:0/485720351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fded9d86c0 0x55fded9e53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 >> 172.21.15.52:0/485720351 conn(0x55fded8fdd40 msgr2=0x55fded9d71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:29.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 shutdown_connections 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 1 -- 172.21.15.52:0/485720351 wait complete. 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.590+0000 7f8108e7db80 20 client.27185 trim_cache size 0 max 16384 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80e40a2c20 msgr2=0x7f80e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80e40a2c20 0x7f80e40a50e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f80e80098e0 tx=0x7f80e8005960 comp rx=0 tx=0).stop 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f8108500 msgr2=0x7f80f8101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f8108500 0x7f80f8101ee0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f80f0007f10 tx=0x7f80f000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 shutdown_connections 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80e40a2c20 0x7f80e40a50e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80f8108500 0x7f80f8101ee0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f80f806a170 0x7f80f8101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 --2- 172.21.15.52:0/4106550367 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f80f806ab30 0x7f80f81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 >> 172.21.15.52:0/4106550367 conn(0x7f80f8078340 msgr2=0x7f80f8100290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 shutdown_connections 2024-06-02T23:19:29.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:29.591+0000 7f80fe5fa700 1 -- 172.21.15.52:0/4106550367 wait complete. 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:30.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:30 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:19:30.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:30.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:30 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:30.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:30.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:30 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:30.590 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:19:30.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 147824 1.92 MB/sec execute 279 sec latency 15.045 ms 2024-06-02T23:19:30.899 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:31.298 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:31.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:31 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:19:31.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:31 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.299 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:31 smithi052 ceph-mon[32340]: pgmap v660: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-02T23:19:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:31 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:19:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:31 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:31 smithi174 ceph-mon[35994]: pgmap v660: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-02T23:19:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:31 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:19:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:31 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:19:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:31 smithi123 ceph-mon[35977]: pgmap v660: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 286 KiB/s rd, 1.3 MiB/s wr, 162 op/s 2024-06-02T23:19:31.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 148426 1.93 MB/sec execute 280 sec latency 15.071 ms 2024-06-02T23:19:32.634 INFO:tasks.workunit.client.0.smithi052.stdout: 1 148963 1.94 MB/sec execute 281 sec latency 15.941 ms 2024-06-02T23:19:32.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 -- 172.21.15.52:0/2019429487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102ee0 msgr2=0x7f091c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 --2- 172.21.15.52:0/2019429487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102ee0 0x7f091c10f3c0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f091400b0a0 tx=0x7f091400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 -- 172.21.15.52:0/2019429487 shutdown_connections 2024-06-02T23:19:32.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 --2- 172.21.15.52:0/2019429487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102ee0 0x7f091c10f3c0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 --2- 172.21.15.52:0/2019429487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c107450 0x7f091c107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 --2- 172.21.15.52:0/2019429487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f091c102540 0x7f091c1029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 -- 172.21.15.52:0/2019429487 >> 172.21.15.52:0/2019429487 conn(0x7f091c0795f0 msgr2=0x7f091c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:32.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 -- 172.21.15.52:0/2019429487 shutdown_connections 2024-06-02T23:19:32.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.645+0000 7f092280c700 1 -- 172.21.15.52:0/2019429487 wait complete. 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 Processor -- start 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 -- start start 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 0x7f091c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 0x7f091c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f091c107450 0x7f091c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f091c112690 con 0x7f091c102540 2024-06-02T23:19:32.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f091c112510 con 0x7f091c102ee0 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f092280c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f091c112810 con 0x7f091c107450 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f091bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 0x7f091c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 0x7f091c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.646+0000 7f091bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 0x7f091c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35410/0 (socket says 172.21.15.52:35410) 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 0x7f091c19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41838/0 (socket says 172.21.15.52:41838) 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091bfff700 1 -- 172.21.15.52:0/3952885679 learned_addr learned my addr 172.21.15.52:0/3952885679 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f0920da9700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f091c107450 0x7f091c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 -- 172.21.15.52:0/3952885679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f091c107450 msgr2=0x7f091c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f091c107450 0x7f091c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 -- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 msgr2=0x7f091c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 0x7f091c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 -- 172.21.15.52:0/3952885679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f091c198ff0 con 0x7f091c102ee0 2024-06-02T23:19:32.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091bfff700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 0x7f091c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:32.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f091b7fe700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 0x7f091c19ec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0910007bc0 tx=0x7f0910007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:32.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f09197fa700 1 -- 172.21.15.52:0/3952885679 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091001c070 con 0x7f091c102ee0 2024-06-02T23:19:32.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f091c1992e0 con 0x7f091c102ee0 2024-06-02T23:19:32.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f09197fa700 1 -- 172.21.15.52:0/3952885679 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09100044e0 con 0x7f091c102ee0 2024-06-02T23:19:32.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f09197fa700 1 -- 172.21.15.52:0/3952885679 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f091000c640 con 0x7f091c102ee0 2024-06-02T23:19:32.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.647+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f091c0746c0 con 0x7f091c102ee0 2024-06-02T23:19:32.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.650+0000 7f09197fa700 1 -- 172.21.15.52:0/3952885679 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0910004030 con 0x7f091c102ee0 2024-06-02T23:19:32.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.650+0000 7f09197fa700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09040a2ca0 0x7f09040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.651+0000 7f091bfff700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09040a2ca0 0x7f09040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.651+0000 7f09197fa700 1 -- 172.21.15.52:0/3952885679 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09100d1e70 con 0x7f091c102ee0 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.651+0000 7f091bfff700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09040a2ca0 0x7f09040a5160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f090c009680 tx=0x7f090c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.651+0000 7f092d08fb80 1 Processor -- start 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.651+0000 7f092d08fb80 1 -- start start 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 0x55934511aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x5593451149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559345114f20 0x559345115340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f092d08fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559345066810 con 0x559345114f20 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f092d08fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55934500e720 con 0x55934511a640 2024-06-02T23:19:32.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f092d08fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593450627d0 con 0x55934511af60 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 0x55934511aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 0x55934511aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41854/0 (socket says 172.21.15.52:41854) 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 -- 172.21.15.52:0/3300468329 learned_addr learned my addr 172.21.15.52:0/3300468329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f0920da9700 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559345114f20 0x559345115340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091b7fe700 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x5593451149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 -- 172.21.15.52:0/3300468329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 msgr2=0x5593451149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x5593451149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 -- 172.21.15.52:0/3300468329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559345114f20 msgr2=0x559345115340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559345114f20 0x559345115340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 -- 172.21.15.52:0/3300468329 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559345004f50 con 0x55934511a640 2024-06-02T23:19:32.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.652+0000 7f091bfff700 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 0x55934511aa20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e720 tx=0x7f090c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=865e9cc1419433c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:32.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f09027fc700 1 -- 172.21.15.52:0/3300468329 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c011e60 con 0x55934511a640 2024-06-02T23:19:32.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f09027fc700 1 -- 172.21.15.52:0/3300468329 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f090c0044d0 con 0x55934511a640 2024-06-02T23:19:32.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f09027fc700 1 -- 172.21.15.52:0/3300468329 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c01e680 con 0x55934511a640 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 -- 172.21.15.52:0/3300468329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 msgr2=0x55934511aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 0x55934511aa20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e720 tx=0x7f090c014000 comp rx=0 tx=0).stop 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 -- 172.21.15.52:0/3300468329 shutdown_connections 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559345114f20 0x559345115340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511a640 0x55934511aa20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 --2- 172.21.15.52:0/3300468329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x5593451149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 -- 172.21.15.52:0/3300468329 >> 172.21.15.52:0/3300468329 conn(0x55934503fcc0 msgr2=0x55934510f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.653+0000 7f092d08fb80 1 -- 172.21.15.52:0/3300468329 shutdown_connections 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.654+0000 7f092d08fb80 1 -- 172.21.15.52:0/3300468329 wait complete. 2024-06-02T23:19:32.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.654+0000 7f092d08fb80 1 -- start start 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.654+0000 7f092d08fb80 1 Processor -- start 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.654+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345120cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x55934511e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f092d08fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5593451219d0 con 0x55934511a640 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f092d08fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559345006bf0 con 0x55934511af60 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f092d08fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559345121b50 con 0x559345114f20 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f092d08fb80 10 client.0 _ll_register_callbacks cb 0x559345000920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35440/0 (socket says 172.21.15.52:35440) 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f091b7fe700 1 -- 172.21.15.52:0/2756107652 learned_addr learned my addr 172.21.15.52:0/2756107652 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x55934511e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f091bfff700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345120cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 -- 172.21.15.52:0/2756107652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 msgr2=0x559345120cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345120cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 -- 172.21.15.52:0/2756107652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 msgr2=0x55934511dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 -- 172.21.15.52:0/2756107652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559344fb58d0 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f091b7fe700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f0920da9700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x55934511e510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f091400d010 tx=0x7f091400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e69710bbf61c9b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.655+0000 7f08fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2756107652 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09140076e0 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2756107652 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0914007880 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f090c0044d0 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2756107652 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f091402e070 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 10 client.27197 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55934512c300 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2756107652 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f090c0044d0 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2756107652 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0914011070 con 0x55934511af60 2024-06-02T23:19:32.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 10 client.27197 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 10 client.27197 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 20 client.27197 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934512e940 0x559345130d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 4 client.27197 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.656+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559345073d10 con 0x55934512e940 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.657+0000 7f091bfff700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934512e940 0x559345130d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.657+0000 7f091bfff700 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934512e940 0x559345130d30 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7caa06b70a975e04 server_cookie=1a6c8f99b0ddad46 in_seq=0 out_seq=0 2024-06-02T23:19:32.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.657+0000 7f08fb7fe700 10 client.27197 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2756107652 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559345073d10 con 0x55934512e940 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f08fb7fe700 10 client.27197 handle_command_reply: tid=0 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 client.27197 shutdown 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 msgr2=0x55934511e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x55934511e510 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f091400d010 tx=0x7f091400bf60 comp rx=0 tx=0).stop 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 shutdown_connections 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934512e940 0x559345130d30 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x55934511e510 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345120cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f08fb7fe700 0 client.27197 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2756107652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 >> 172.21.15.52:0/2756107652 conn(0x55934503fcc0 msgr2=0x55934510f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 shutdown_connections 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 1 -- 172.21.15.52:0/2756107652 wait complete. 2024-06-02T23:19:32.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.658+0000 7f092d08fb80 20 client.27197 trim_cache size 0 max 16384 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 Processor -- start 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 -- start start 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559345114f20 0x5593451149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x559345136590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1aa740 con 0x55934511a640 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1ac280 con 0x559345114f20 2024-06-02T23:19:32.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f092d08fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1aae10 con 0x55934511af60 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559345114f20 0x5593451149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35450/0 (socket says 172.21.15.52:35450) 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559345114f20 0x5593451149e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41862/0 (socket says 172.21.15.52:41862) 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091b7fe700 1 -- 172.21.15.52:0/942547535 learned_addr learned my addr 172.21.15.52:0/942547535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f0920da9700 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x559345136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091b7fe700 1 -- 172.21.15.52:0/942547535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 msgr2=0x559345136590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.684+0000 7f091b7fe700 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x559345136590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f091b7fe700 1 -- 172.21.15.52:0/942547535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559345114f20 msgr2=0x5593451149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f091b7fe700 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559345114f20 0x5593451149e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f091b7fe700 1 -- 172.21.15.52:0/942547535 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559345131360 con 0x55934511a640 2024-06-02T23:19:32.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f091b7fe700 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511ba50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f091009e400 tx=0x7f0910089040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d609f81c78dbb9c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f0901ffb700 1 -- 172.21.15.52:0/942547535 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100cfe60 con 0x55934511a640 2024-06-02T23:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f0901ffb700 1 -- 172.21.15.52:0/942547535 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0910004320 con 0x55934511a640 2024-06-02T23:19:32.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.685+0000 7f0901ffb700 1 -- 172.21.15.52:0/942547535 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100a7370 con 0x55934511a640 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 -- 172.21.15.52:0/942547535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 msgr2=0x55934511ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511ba50 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f091009e400 tx=0x7f0910089040 comp rx=0 tx=0).stop 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 -- 172.21.15.52:0/942547535 shutdown_connections 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55934511af60 0x559345136590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559345114f20 0x5593451149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 --2- 172.21.15.52:0/942547535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x55934511ba50 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 -- 172.21.15.52:0/942547535 >> 172.21.15.52:0/942547535 conn(0x55934503fcc0 msgr2=0x55934511b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 -- 172.21.15.52:0/942547535 shutdown_connections 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 -- 172.21.15.52:0/942547535 wait complete. 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 -- start start 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.686+0000 7f092d08fb80 1 Processor -- start 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345126e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x559345127380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x559345123e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1abbb0 con 0x55934511a640 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1ab4e0 con 0x55934511af60 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f091c1a8700 con 0x559345114f20 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f092d08fb80 10 client.0 _ll_register_callbacks cb 0x559345000920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x559345127380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x559345127380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:35458/0 (socket says 172.21.15.52:35458) 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 -- 172.21.15.52:0/2243991940 learned_addr learned my addr 172.21.15.52:0/2243991940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091bfff700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345126e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f0920da9700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x559345123e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 -- 172.21.15.52:0/2243991940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 msgr2=0x559345126e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345126e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 -- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 msgr2=0x559345123e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x559345123e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091b7fe700 1 -- 172.21.15.52:0/2243991940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55934511a140 con 0x55934511a640 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.687+0000 7f091bfff700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345126e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f091b7fe700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x559345127380 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f09100cf9f0 tx=0x7f091009df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=debd05bb9c96514f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f08fb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2243991940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09100cfe60 con 0x55934511a640 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2243991940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f091009ca00 con 0x55934511a640 2024-06-02T23:19:32.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55934506e290 con 0x55934511a640 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2243991940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f091009e700 con 0x55934511a640 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 10 client.17712 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559345199b00 con 0x55934511a640 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2243991940 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55934506e290 con 0x55934511a640 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2243991940 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0910016030 con 0x55934511a640 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 10 client.17712 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 10 client.17712 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 20 client.17712 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.688+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934519a180 0x55934519c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f092d08fb80 4 client.17712 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559344ee8250 con 0x55934519a180 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f091bfff700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934519a180 0x55934519c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f091bfff700 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934519a180 0x55934519c5a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc34166a0e8a8d79 server_cookie=c1ea5e87fc2242b6 in_seq=0 out_seq=0 2024-06-02T23:19:32.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f08fb7fe700 10 client.17712 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f08fb7fe700 1 -- 172.21.15.52:0/2243991940 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559344ee8250 con 0x55934519a180 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f08fb7fe700 10 client.17712 handle_command_reply: tid=0 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.689+0000 7f092d08fb80 1 client.17712 shutdown 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 msgr2=0x559345127380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x559345127380 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f09100cf9f0 tx=0x7f091009df70 comp rx=0 tx=0).stop 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 shutdown_connections 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55934519a180 0x55934519c5a0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55934511af60 0x559345123e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559345114f20 0x559345126e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 --2- 172.21.15.52:0/2243991940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55934511a640 0x559345127380 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 >> 172.21.15.52:0/2243991940 conn(0x55934503fcc0 msgr2=0x55934511b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f08fb7fe700 0 client.17712 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:32.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 shutdown_connections 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 1 -- 172.21.15.52:0/2243991940 wait complete. 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092d08fb80 20 client.17712 trim_cache size 0 max 16384 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: "ce4d6f04-9842-491d-b7ae-b9b66ec65b32": { 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "ce4d6f04-9842-491d-b7ae-b9b66ec65b32", 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:32.695 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09040a2ca0 msgr2=0x7f09040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092280c700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09040a2ca0 0x7f09040a5160 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f090c009680 tx=0x7f090c009260 comp rx=0 tx=0).stop 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 msgr2=0x7f091c19ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.690+0000 7f092280c700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 0x7f091c19ec10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0910007bc0 tx=0x7f0910007ed0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 shutdown_connections 2024-06-02T23:19:32.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09040a2ca0 0x7f09040a5160 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f091c107450 0x7f091c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f091c102540 0x7f091c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 --2- 172.21.15.52:0/3952885679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f091c102ee0 0x7f091c19ec10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 >> 172.21.15.52:0/3952885679 conn(0x7f091c0795f0 msgr2=0x7f091c100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 shutdown_connections 2024-06-02T23:19:32.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:32.691+0000 7f092280c700 1 -- 172.21.15.52:0/3952885679 wait complete. 2024-06-02T23:19:33.339 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:33 smithi052 ceph-mon[32340]: pgmap v661: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-06-02T23:19:33.366 INFO:tasks.cephfs.filesystem:scrub status for tag:ce4d6f04-9842-491d-b7ae-b9b66ec65b32 - {'path': '/', 'tag': 'ce4d6f04-9842-491d-b7ae-b9b66ec65b32', 'options': 'recursive,force'} 2024-06-02T23:19:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:33 smithi174 ceph-mon[35994]: pgmap v661: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-06-02T23:19:33.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:33 smithi123 ceph-mon[35977]: pgmap v661: 97 pgs: 97 active+clean; 69 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 151 op/s 2024-06-02T23:19:33.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 149341 1.94 MB/sec execute 282 sec latency 14.880 ms 2024-06-02T23:19:34.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 149606 1.94 MB/sec execute 283 sec latency 12.766 ms 2024-06-02T23:19:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:35 smithi174 ceph-mon[35994]: pgmap v662: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-06-02T23:19:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:35 smithi052 ceph-mon[32340]: pgmap v662: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-06-02T23:19:35.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:35 smithi123 ceph-mon[35977]: pgmap v662: 97 pgs: 97 active+clean; 70 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.0 MiB/s wr, 137 op/s 2024-06-02T23:19:35.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 150097 1.94 MB/sec execute 284 sec latency 20.467 ms 2024-06-02T23:19:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:19:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:19:36.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:36 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:19:36.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:19:36.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:19:36.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:36 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:19:36.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:19:36.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:19:36.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:36 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:19:36.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 150672 1.94 MB/sec execute 285 sec latency 14.729 ms 2024-06-02T23:19:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:37 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:19:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:37 smithi174 ceph-mon[35994]: pgmap v663: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-06-02T23:19:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:37 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:19:37.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:37 smithi052 ceph-mon[32340]: pgmap v663: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-06-02T23:19:37.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:37 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:19:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:37 smithi123 ceph-mon[35977]: pgmap v663: 97 pgs: 97 active+clean; 71 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 1.4 MiB/s wr, 189 op/s 2024-06-02T23:19:37.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 151355 1.94 MB/sec execute 286 sec latency 15.350 ms 2024-06-02T23:19:38.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 151942 1.95 MB/sec execute 287 sec latency 14.803 ms 2024-06-02T23:19:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:39 smithi174 ceph-mon[35994]: pgmap v664: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-02T23:19:39.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:39 smithi052 ceph-mon[32340]: pgmap v664: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-02T23:19:39.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:39 smithi123 ceph-mon[35977]: pgmap v664: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 148 op/s 2024-06-02T23:19:39.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 152397 1.95 MB/sec execute 288 sec latency 16.025 ms 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/1974486763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 msgr2=0x7f8e08100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/1974486763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08100d20 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f8df8009900 tx=0x7f8df802a410 comp rx=0 tx=0).stop 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/1974486763 shutdown_connections 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/1974486763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081024b0 0x7f8e08108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/1974486763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 0x7f8e08101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/1974486763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08100d20 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/1974486763 >> 172.21.15.52:0/1974486763 conn(0x7f8e080fbe70 msgr2=0x7f8e080fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:40.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/1974486763 shutdown_connections 2024-06-02T23:19:40.295 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.294+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/1974486763 wait complete. 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 Processor -- start 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 -- start start 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 0x7f8e0819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081024b0 0x7f8e081a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0810c400 con 0x7f8e08100900 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0810c280 con 0x7f8e08101af0 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0cfe3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0810c580 con 0x7f8e081024b0 2024-06-02T23:19:40.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56152/0 (socket says 172.21.15.52:56152) 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e0659c640 1 -- 172.21.15.52:0/2231016340 learned_addr learned my addr 172.21.15.52:0/2231016340 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.295+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 0x7f8e0819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081024b0 0x7f8e081a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e05d9b640 1 -- 172.21.15.52:0/2231016340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081024b0 msgr2=0x7f8e081a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081024b0 0x7f8e081a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e05d9b640 1 -- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 msgr2=0x7f8e08199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e05d9b640 1 -- 172.21.15.52:0/2231016340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081a19e0 con 0x7f8e08101af0 2024-06-02T23:19:40.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e0659c640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 0x7f8e0819a300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00e9f0 tx=0x7f8dfc00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8df77fe640 1 -- 172.21.15.52:0/2231016340 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc00cd80 con 0x7f8e08101af0 2024-06-02T23:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8df77fe640 1 -- 172.21.15.52:0/2231016340 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8dfc0035f0 con 0x7f8e08101af0 2024-06-02T23:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081a1cd0 con 0x7f8e08101af0 2024-06-02T23:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8df77fe640 1 -- 172.21.15.52:0/2231016340 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dfc010670 con 0x7f8e08101af0 2024-06-02T23:19:40.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.296+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081a21d0 con 0x7f8e08101af0 2024-06-02T23:19:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.299+0000 7f8df77fe640 1 -- 172.21.15.52:0/2231016340 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8dfc0037a0 con 0x7f8e08101af0 2024-06-02T23:19:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.299+0000 7f8e0fdcd740 1 Processor -- start 2024-06-02T23:19:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 -- start start 2024-06-02T23:19:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043eea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 0x55a043eef500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ef2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a043e4d4f0 con 0x55a043eeaa00 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a043deb870 con 0x55a043eefb70 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a043e494b0 con 0x55a043eea010 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8df77fe640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8df00a2c30 0x7f8df00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 0x55a043eef500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e06d9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043eea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 0x55a043eef500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56154/0 (socket says 172.21.15.52:56154) 2024-06-02T23:19:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e05d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ef2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 -- 172.21.15.52:0/1236526279 learned_addr learned my addr 172.21.15.52:0/1236526279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8df00a2c30 0x7f8df00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8df77fe640 1 -- 172.21.15.52:0/2231016340 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8dfc0cf340 con 0x7f8e08101af0 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 -- 172.21.15.52:0/1236526279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 msgr2=0x55a043eea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043eea430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 -- 172.21.15.52:0/1236526279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 msgr2=0x55a043ef2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ef2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e0659c640 1 -- 172.21.15.52:0/1236526279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a043e079b0 con 0x55a043eeaa00 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043eea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.300+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ef2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.301+0000 7f8e0659c640 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 0x55a043eef500 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8df8006010 tx=0x7f8df8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd039798b9c26a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.301+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8df00a2c30 0x7f8df00a50f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55a043eeb260 tx=0x7f8e00009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.301+0000 7f8df4ff9640 1 -- 172.21.15.52:0/1236526279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8035070 con 0x55a043eeaa00 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.301+0000 7f8df4ff9640 1 -- 172.21.15.52:0/1236526279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df80318c0 con 0x55a043eeaa00 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.301+0000 7f8df4ff9640 1 -- 172.21.15.52:0/1236526279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8045470 con 0x55a043eeaa00 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/1236526279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 msgr2=0x55a043eef500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 0x55a043eef500 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8df8006010 tx=0x7f8df8005be0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/1236526279 shutdown_connections 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ef2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eeaa00 0x55a043eef500 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/1236526279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043eea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/1236526279 >> 172.21.15.52:0/1236526279 conn(0x55a043ee5680 msgr2=0x55a043ee5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/1236526279 shutdown_connections 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/1236526279 wait complete. 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- start start 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 Processor -- start 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ef4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043ef4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043efb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a043ef2ac0 con 0x55a043eea010 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a043dd6ae0 con 0x55a043eefb70 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.302+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a043ef2c40 con 0x55a043eeaa00 2024-06-02T23:19:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e0fdcd740 10 client.0 _ll_register_callbacks cb 0x55a043e0f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ef4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043efb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ef4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56156/0 (socket says 172.21.15.52:56156) 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e0659c640 1 -- 172.21.15.52:0/4059532937 learned_addr learned my addr 172.21.15.52:0/4059532937 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043ef4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 -- 172.21.15.52:0/4059532937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 msgr2=0x55a043ef4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043ef4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 -- 172.21.15.52:0/4059532937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 msgr2=0x55a043ef4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ef4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 -- 172.21.15.52:0/4059532937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a043b61210 con 0x55a043eefb70 2024-06-02T23:19:40.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e0659c640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ef4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043efb7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e00005fd0 tx=0x7f8e00004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c50cb44a75f800e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8dedffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8dedffb640 1 -- 172.21.15.52:0/4059532937 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00003c40 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df80318c0 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.303+0000 7f8dedffb640 1 -- 172.21.15.52:0/4059532937 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e00003de0 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8dedffb640 1 -- 172.21.15.52:0/4059532937 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e0002f020 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 10 client.27215 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a043c2d620 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8dedffb640 1 -- 172.21.15.52:0/4059532937 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df80318c0 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8dedffb640 1 -- 172.21.15.52:0/4059532937 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8e00011840 con 0x55a043eefb70 2024-06-02T23:19:40.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 10 client.27215 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 10 client.27215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 20 client.27215 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f000a0 0x55a043f02510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 4 client.27215 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a043e60f10 con 0x55a043f000a0 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.304+0000 7f8e0659c640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f000a0 0x55a043f02510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.305+0000 7f8dedffb640 10 client.27215 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.305+0000 7f8e0659c640 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f000a0 0x55a043f02510 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf9c805cf8ef839e server_cookie=a081eb59e0ff8633 in_seq=0 out_seq=0 2024-06-02T23:19:40.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.305+0000 7f8dedffb640 1 -- 172.21.15.52:0/4059532937 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a043e60f10 con 0x55a043f000a0 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.305+0000 7f8dedffb640 10 client.27215 handle_command_reply: tid=0 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 client.27215 shutdown 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 msgr2=0x55a043efb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043efb7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8e00005fd0 tx=0x7f8e00004ab0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 shutdown_connections 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f000a0 0x55a043f02510 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043ef4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043efb7b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/4059532937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ef4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8dedffb640 0 client.27215 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 >> 172.21.15.52:0/4059532937 conn(0x55a043ee5680 msgr2=0x55a043ee47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:40.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 shutdown_connections 2024-06-02T23:19:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/4059532937 wait complete. 2024-06-02T23:19:40.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.306+0000 7f8e0fdcd740 20 client.27215 trim_cache size 0 max 16384 2024-06-02T23:19:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 Processor -- start 2024-06-02T23:19:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 -- start start 2024-06-02T23:19:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ee3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043efb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ee4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081aaf90 con 0x55a043eea010 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081acad0 con 0x55a043eefb70 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081ab660 con 0x55a043eeaa00 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ee3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ee3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56166/0 (socket says 172.21.15.52:56166) 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 -- 172.21.15.52:0/3231799259 learned_addr learned my addr 172.21.15.52:0/3231799259 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ee4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e05d9b640 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043efb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 -- 172.21.15.52:0/3231799259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 msgr2=0x55a043efb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043efb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 -- 172.21.15.52:0/3231799259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 msgr2=0x55a043ee4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ee4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.323+0000 7f8e0659c640 1 -- 172.21.15.52:0/3231799259 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a043eea7a0 con 0x55a043eea010 2024-06-02T23:19:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.324+0000 7f8e0659c640 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ee3f50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8df8032eb0 tx=0x7f8df8048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd164ad990ffcd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.324+0000 7f8deffff640 1 -- 172.21.15.52:0/3231799259 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8056070 con 0x55a043eea010 2024-06-02T23:19:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.324+0000 7f8deffff640 1 -- 172.21.15.52:0/3231799259 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df8051ae0 con 0x55a043eea010 2024-06-02T23:19:40.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.324+0000 7f8deffff640 1 -- 172.21.15.52:0/3231799259 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8035040 con 0x55a043eea010 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/3231799259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 msgr2=0x55a043ee3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ee3f50 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f8df8032eb0 tx=0x7f8df8048040 comp rx=0 tx=0).stop 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/3231799259 shutdown_connections 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eeaa00 0x55a043efb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eefb70 0x55a043ee4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/3231799259 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eea010 0x55a043ee3f50 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/3231799259 >> 172.21.15.52:0/3231799259 conn(0x55a043ee5680 msgr2=0x55a043eebc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:40.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/3231799259 shutdown_connections 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/3231799259 wait complete. 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- start start 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 Processor -- start 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043efb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eeaa00 0x55a043ef82b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 0x55a043ef8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081ac400 con 0x55a043eefb70 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081abd30 con 0x55a043eeaa00 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081a8f70 con 0x55a043eea010 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.325+0000 7f8e0fdcd740 10 client.0 _ll_register_callbacks cb 0x7f8e00009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 0x55a043ef8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 0x55a043ef8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56170/0 (socket says 172.21.15.52:56170) 2024-06-02T23:19:40.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e05d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eeaa00 0x55a043ef82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 -- 172.21.15.52:0/69331095 learned_addr learned my addr 172.21.15.52:0/69331095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e0659c640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043efb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 -- 172.21.15.52:0/69331095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 msgr2=0x55a043efb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043efb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 -- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eeaa00 msgr2=0x55a043ef82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eeaa00 0x55a043ef82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 -- 172.21.15.52:0/69331095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a043f68e90 con 0x55a043eefb70 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e0659c640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043efb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e06d9d640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 0x55a043ef8890 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8e00013010 tx=0x7f8e000098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f21b5cb3fc610e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8dedffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:40.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8dedffb640 1 -- 172.21.15.52:0/69331095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00039060 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8dedffb640 1 -- 172.21.15.52:0/69331095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e00011990 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8df8051ae0 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8dedffb640 1 -- 172.21.15.52:0/69331095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e00004b20 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e0fdcd740 10 client.17730 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.326+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a043f6a1b0 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8dedffb640 1 -- 172.21.15.52:0/69331095 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8df8051ae0 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8dedffb640 1 -- 172.21.15.52:0/69331095 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8e00040480 con 0x55a043eefb70 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0fdcd740 10 client.17730 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0fdcd740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0fdcd740 20 client.17730 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f6ab00 0x55a043f6cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0fdcd740 4 client.17730 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a043e5ac20 con 0x55a043f6ab00 2024-06-02T23:19:40.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.327+0000 7f8e0659c640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f6ab00 0x55a043f6cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.328+0000 7f8e0659c640 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f6ab00 0x55a043f6cf20 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c3987fe193ecfa server_cookie=59a238db6bfe247d in_seq=0 out_seq=0 2024-06-02T23:19:40.332 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.328+0000 7f8dedffb640 10 client.17730 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8dedffb640 1 -- 172.21.15.52:0/69331095 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a043e5ac20 con 0x55a043f6ab00 2024-06-02T23:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8dedffb640 10 client.17730 handle_command_reply: tid=0 2024-06-02T23:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 client.17730 shutdown 2024-06-02T23:19:40.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 msgr2=0x55a043ef8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 0x55a043ef8890 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f8e00013010 tx=0x7f8e000098e0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 shutdown_connections 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a043f6ab00 0x55a043f6cf20 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a043eefb70 0x55a043ef8890 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a043eeaa00 0x55a043ef82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 --2- 172.21.15.52:0/69331095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a043eea010 0x55a043efb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 >> 172.21.15.52:0/69331095 conn(0x55a043ee5680 msgr2=0x55a043eebc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8dedffb640 0 client.17730 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 shutdown_connections 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.350+0000 7f8e0fdcd740 1 -- 172.21.15.52:0/69331095 wait complete. 2024-06-02T23:19:40.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0fdcd740 20 client.17730 trim_cache size 0 max 16384 2024-06-02T23:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:19:40.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8df00a2c30 msgr2=0x7f8df00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8df00a2c30 0x7f8df00a50f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x55a043eeb260 tx=0x7f8e00009500 comp rx=0 tx=0).stop 2024-06-02T23:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 msgr2=0x7f8e0819a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:40.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 0x7f8e0819a300 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8dfc00e9f0 tx=0x7f8dfc00eeb0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 shutdown_connections 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8df00a2c30 0x7f8df00a50f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e081024b0 0x7f8e081a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08101af0 0x7f8e0819a300 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 --2- 172.21.15.52:0/2231016340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e08100900 0x7f8e08199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 >> 172.21.15.52:0/2231016340 conn(0x7f8e080fbe70 msgr2=0x7f8e080fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 shutdown_connections 2024-06-02T23:19:40.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:40.351+0000 7f8e0cfe3640 1 -- 172.21.15.52:0/2231016340 wait complete. 2024-06-02T23:19:40.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 152861 1.96 MB/sec execute 289 sec latency 18.996 ms 2024-06-02T23:19:41.378 INFO:tasks.quiescer.fs.[cephfs]:Total duration (52.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:19:41.379 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.3 2024-06-02T23:19:41.379 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:19:41.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:41 smithi174 ceph-mon[35994]: pgmap v665: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-02T23:19:41.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:41 smithi052 ceph-mon[32340]: pgmap v665: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-02T23:19:41.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:41 smithi123 ceph-mon[35977]: pgmap v665: 97 pgs: 97 active+clean; 72 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 1.2 MiB/s wr, 167 op/s 2024-06-02T23:19:41.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 153133 1.95 MB/sec execute 290 sec latency 13.264 ms 2024-06-02T23:19:41.645 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:42.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 153606 1.95 MB/sec execute 291 sec latency 15.705 ms 2024-06-02T23:19:43.116 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:43 smithi052 ceph-mon[32340]: pgmap v666: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-02T23:19:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:43 smithi174 ceph-mon[35994]: pgmap v666: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-02T23:19:43.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 -- 172.21.15.52:0/2882111349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e40106720 msgr2=0x7f4e4010e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/2882111349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e40106720 0x7f4e4010e2c0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800b0d0 tx=0x7f4e3800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 -- 172.21.15.52:0/2882111349 shutdown_connections 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/2882111349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e40106720 0x7f4e4010e2c0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/2882111349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e401019d0 0x7f4e40101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/2882111349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e40102380 0x7f4e40105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 -- 172.21.15.52:0/2882111349 >> 172.21.15.52:0/2882111349 conn(0x7f4e40078340 msgr2=0x7f4e40078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 -- 172.21.15.52:0/2882111349 shutdown_connections 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 -- 172.21.15.52:0/2882111349 wait complete. 2024-06-02T23:19:43.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.475+0000 7f4e45f3b700 1 Processor -- start 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 -- start start 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 0x7f4e40103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e40102380 0x7f4e401035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e40106720 0x7f4e40103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4e401115b0 con 0x7f4e401019d0 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40111430 con 0x7f4e40106720 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e45f3b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e40111730 con 0x7f4e40102380 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 0x7f4e40103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 0x7f4e40103090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56188/0 (socket says 172.21.15.52:56188) 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 -- 172.21.15.52:0/3417977164 learned_addr learned my addr 172.21.15.52:0/3417977164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:43.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3effd700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e40102380 0x7f4e401035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3ffff700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e40106720 0x7f4e40103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 -- 172.21.15.52:0/3417977164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e40102380 msgr2=0x7f4e401035d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e40102380 0x7f4e401035d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 -- 172.21.15.52:0/3417977164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e40106720 msgr2=0x7f4e40103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e40106720 0x7f4e40103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3f7fe700 1 -- 172.21.15.52:0/3417977164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e401a70b0 con 0x7f4e401019d0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.476+0000 7f4e3ffff700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e40106720 0x7f4e40103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.477+0000 7f4e3f7fe700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 0x7f4e40103090 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000e9a0 tx=0x7f4e3000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.477+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e3000ca10 con 0x7f4e401019d0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.477+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e401a73a0 con 0x7f4e401019d0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.477+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4e3000cbb0 con 0x7f4e401019d0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.477+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401a7930 con 0x7f4e401019d0 2024-06-02T23:19:43.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.477+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30018680 con 0x7f4e401019d0 2024-06-02T23:19:43.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.478+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e40102ad0 con 0x7f4e401019d0 2024-06-02T23:19:43.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.479+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4e30010430 con 0x7f4e401019d0 2024-06-02T23:19:43.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.480+0000 7f4e3cff9700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4e280a2c20 0x7f4e280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:43.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.480+0000 7f4e3effd700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4e280a2c20 0x7f4e280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:43.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.480+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4e300cf200 con 0x7f4e401019d0 2024-06-02T23:19:43.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.481+0000 7f4e3effd700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4e280a2c20 0x7f4e280a50e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4e40107cb0 tx=0x7f4e3400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:43.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.485+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4e3009c130 con 0x7f4e401019d0 2024-06-02T23:19:43.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:43 smithi123 ceph-mon[35977]: pgmap v666: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 1.3 MiB/s wr, 165 op/s 2024-06-02T23:19:43.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 154110 1.96 MB/sec execute 292 sec latency 46.648 ms 2024-06-02T23:19:43.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.775+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e4002dca0 con 0x7f4e401019d0 2024-06-02T23:19:43.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.776+0000 7f4e3cff9700 1 -- 172.21.15.52:0/3417977164 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f4e3009ba30 con 0x7f4e401019d0 2024-06-02T23:19:43.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:19:43.776 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.778+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4e280a2c20 msgr2=0x7f4e280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.778+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4e280a2c20 0x7f4e280a50e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f4e40107cb0 tx=0x7f4e3400b3c0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.778+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 msgr2=0x7f4e40103090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.778+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 0x7f4e40103090 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000e9a0 tx=0x7f4e3000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 shutdown_connections 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4e280a2c20 0x7f4e280a50e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4e40106720 0x7f4e40103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4e401019d0 0x7f4e40103090 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 --2- 172.21.15.52:0/3417977164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e40102380 0x7f4e401035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 >> 172.21.15.52:0/3417977164 conn(0x7f4e40078340 msgr2=0x7f4e4006aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 shutdown_connections 2024-06-02T23:19:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:43.779+0000 7f4e45f3b700 1 -- 172.21.15.52:0/3417977164 wait complete. 2024-06-02T23:19:43.779 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:19:44.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:44 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3417977164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:19:44.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:44 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3417977164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:19:44.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:44 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3417977164' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:19:44.571 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 146.3 --await-for 45.0' 2024-06-02T23:19:44.571 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 146.3 --await-for 45.0 2024-06-02T23:19:44.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 154737 1.95 MB/sec execute 293 sec latency 15.053 ms 2024-06-02T23:19:44.887 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:19:45.286 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:45 smithi052 ceph-mon[32340]: pgmap v667: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-02T23:19:45.294 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:45 smithi174 ceph-mon[35994]: pgmap v667: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-02T23:19:45.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:45 smithi123 ceph-mon[35977]: pgmap v667: 97 pgs: 97 active+clean; 73 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-02T23:19:45.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155293 1.97 MB/sec execute 294 sec latency 15.666 ms 2024-06-02T23:19:46.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155856 1.97 MB/sec execute 295 sec latency 61.400 ms 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1813679623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 msgr2=0x7f0e9c108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1813679623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c108900 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0e980098f0 tx=0x7f0e98009c00 comp rx=0 tx=0).stop 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1813679623 shutdown_connections 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1813679623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9c06b420 0x7f0e9c10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1813679623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c108900 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1813679623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06aa60 0x7f0e9c06aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1813679623 >> 172.21.15.52:0/1813679623 conn(0x7f0e9c0783d0 msgr2=0x7f0e9c0787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1813679623 shutdown_connections 2024-06-02T23:19:46.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.720+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1813679623 wait complete. 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 Processor -- start 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 -- start start 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9c06aa60 0x7f0e9c19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06b420 0x7f0e9c19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c112630 con 0x7f0e9c06aa60 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1124b0 con 0x7f0e9c108520 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.721+0000 7f0ea3fea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1127b0 con 0x7f0e9c06b420 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea1585700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06b420 0x7f0e9c19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea1585700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06b420 0x7f0e9c19ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34450/0 (socket says 172.21.15.52:34450) 2024-06-02T23:19:46.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9c06aa60 0x7f0e9c19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea1585700 1 -- 172.21.15.52:0/1301899379 learned_addr learned my addr 172.21.15.52:0/1301899379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:46.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37686/0 (socket says 172.21.15.52:37686) 2024-06-02T23:19:46.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 -- 172.21.15.52:0/1301899379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06b420 msgr2=0x7f0e9c19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06b420 0x7f0e9c19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 -- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9c06aa60 msgr2=0x7f0e9c19e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9c06aa60 0x7f0e9c19e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 -- 172.21.15.52:0/1301899379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e9c199070 con 0x7f0e9c108520 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0ea2587700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c198910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400efc0 tx=0x7f0e9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.722+0000 7f0e92ffd700 1 -- 172.21.15.52:0/1301899379 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940092e0 con 0x7f0e9c108520 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.723+0000 7f0e92ffd700 1 -- 172.21.15.52:0/1301899379 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e94004bb0 con 0x7f0e9c108520 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.723+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e9c199300 con 0x7f0e9c108520 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.723+0000 7f0e92ffd700 1 -- 172.21.15.52:0/1301899379 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e94004020 con 0x7f0e9c108520 2024-06-02T23:19:46.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.723+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e9c101380 con 0x7f0e9c108520 2024-06-02T23:19:46.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.726+0000 7f0e92ffd700 1 -- 172.21.15.52:0/1301899379 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0e9401b030 con 0x7f0e9c108520 2024-06-02T23:19:46.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 Processor -- start 2024-06-02T23:19:46.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 -- start start 2024-06-02T23:19:46.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0e92ffd700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0e880a2d40 0x7f0e880a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335e2970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2eb0 0x5628335dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335dcce0 0x5628335dd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0ea1585700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0e880a2d40 0x7f0e880a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5628334cf0f0 con 0x5628335dcce0 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56283352d9e0 con 0x5628335e2eb0 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0eae86db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628334955f0 con 0x5628335e2590 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.727+0000 7f0e92ffd700 1 -- 172.21.15.52:0/1301899379 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0e940ce970 con 0x7f0e9c108520 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335e2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335e2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34456/0 (socket says 172.21.15.52:34456) 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 -- 172.21.15.52:0/106556347 learned_addr learned my addr 172.21.15.52:0/106556347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:46.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea2587700 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335dcce0 0x5628335dd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1585700 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2eb0 0x5628335dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 -- 172.21.15.52:0/106556347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2eb0 msgr2=0x5628335dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2eb0 0x5628335dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 -- 172.21.15.52:0/106556347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335dcce0 msgr2=0x5628335dd100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335dcce0 0x5628335dd100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 -- 172.21.15.52:0/106556347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283347ca60 con 0x5628335e2590 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1585700 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2eb0 0x5628335dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335e2970 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005e90 tx=0x7f0e98005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee43badf8861b4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0ea1585700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0e880a2d40 0x7f0e880a5200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c078960 tx=0x7f0e8c0119c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:46.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0e87fff700 1 -- 172.21.15.52:0/106556347 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98026070 con 0x5628335e2590 2024-06-02T23:19:46.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.728+0000 7f0e87fff700 1 -- 172.21.15.52:0/106556347 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e9800f910 con 0x5628335e2590 2024-06-02T23:19:46.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0e87fff700 1 -- 172.21.15.52:0/106556347 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e980038e0 con 0x5628335e2590 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 -- 172.21.15.52:0/106556347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 msgr2=0x5628335e2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335e2970 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e98005e90 tx=0x7f0e98005dc0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 -- 172.21.15.52:0/106556347 shutdown_connections 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335dcce0 0x5628335dd100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335e2970 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 --2- 172.21.15.52:0/106556347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2eb0 0x5628335dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.729+0000 7f0eae86db80 1 -- 172.21.15.52:0/106556347 >> 172.21.15.52:0/106556347 conn(0x562833509210 msgr2=0x5628335d7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.730+0000 7f0eae86db80 1 -- 172.21.15.52:0/106556347 shutdown_connections 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.730+0000 7f0eae86db80 1 -- 172.21.15.52:0/106556347 wait complete. 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.730+0000 7f0eae86db80 1 -- start start 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.730+0000 7f0eae86db80 1 Processor -- start 2024-06-02T23:19:46.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 0x5628335e8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2590 0x5628335e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335e6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5628334ce150 con 0x5628335e2eb0 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5628335299a0 con 0x5628335e2590 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628335e9830 con 0x5628335dcce0 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 0x5628335e8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0eae86db80 10 client.0 _ll_register_callbacks cb 0x5628334c3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea2587700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1585700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2590 0x5628335e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 0x5628335e8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34472/0 (socket says 172.21.15.52:34472) 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 -- 172.21.15.52:0/1490242853 learned_addr learned my addr 172.21.15.52:0/1490242853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 -- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2590 msgr2=0x5628335e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2590 0x5628335e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 -- 172.21.15.52:0/1490242853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 msgr2=0x5628335e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.731+0000 7f0ea1d86700 1 -- 172.21.15.52:0/1490242853 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628334f38e0 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 0x5628335e8be0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006010 tx=0x7f0e98005290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c51c5d127dd6b986 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0e857fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0e857fa700 1 -- 172.21.15.52:0/1490242853 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98026040 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0e857fa700 1 -- 172.21.15.52:0/1490242853 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0e98003680 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9800f910 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0e857fa700 1 -- 172.21.15.52:0/1490242853 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e98005900 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0eae86db80 10 client.27061 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5628335f3f70 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0e857fa700 1 -- 172.21.15.52:0/1490242853 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e9800f910 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0e857fa700 1 -- 172.21.15.52:0/1490242853 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0e98029070 con 0x5628335dcce0 2024-06-02T23:19:46.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0eae86db80 10 client.27061 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.732+0000 7f0eae86db80 10 client.27061 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0eae86db80 20 client.27061 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628335f6440 0x5628335f8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0eae86db80 4 client.27061 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283353aee0 con 0x5628335f6440 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0ea1585700 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628335f6440 0x5628335f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0ea1585700 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628335f6440 0x5628335f8830 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9a0538cb31d1ff3 server_cookie=72c7cf5becdfc293 in_seq=0 out_seq=0 2024-06-02T23:19:46.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.733+0000 7f0e857fa700 10 client.27061 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.734+0000 7f0e857fa700 1 -- 172.21.15.52:0/1490242853 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56283353aee0 con 0x5628335f6440 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.734+0000 7f0e857fa700 10 client.27061 handle_command_reply: tid=0 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 client.27061 shutdown 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 msgr2=0x5628335e8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 0x5628335e8be0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0e98006010 tx=0x7f0e98005290 comp rx=0 tx=0).stop 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 shutdown_connections 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628335f6440 0x5628335f8830 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335dcce0 0x5628335e8be0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1490242853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335e2590 0x5628335e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0e857fa700 0 client.27061 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 >> 172.21.15.52:0/1490242853 conn(0x562833509210 msgr2=0x5628335d6c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 shutdown_connections 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 1 -- 172.21.15.52:0/1490242853 wait complete. 2024-06-02T23:19:46.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.735+0000 7f0eae86db80 20 client.27061 trim_cache size 0 max 16384 2024-06-02T23:19:46.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.760+0000 7f0eae86db80 1 Processor -- start 2024-06-02T23:19:46.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.760+0000 7f0eae86db80 1 -- start start 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0eae86db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1aa740 con 0x5628335e2eb0 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0eae86db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1ac280 con 0x5628335dcce0 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0eae86db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1aae10 con 0x5628335e2590 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335d7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38148/0 (socket says 172.21.15.52:38148) 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 -- 172.21.15.52:0/1098300513 learned_addr learned my addr 172.21.15.52:0/1098300513 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea1585700 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 -- 172.21.15.52:0/1098300513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 msgr2=0x5628335d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 -- 172.21.15.52:0/1098300513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 msgr2=0x5628335dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 -- 172.21.15.52:0/1098300513 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628335e6e10 con 0x5628335e2eb0 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea1585700 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.761+0000 7f0ea2587700 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335d7900 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0e9409b220 tx=0x7f0e94009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=211feb5168a51db0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.762+0000 7f0e877fe700 1 -- 172.21.15.52:0/1098300513 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e940b4070 con 0x5628335e2eb0 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.762+0000 7f0e877fe700 1 -- 172.21.15.52:0/1098300513 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e9409b990 con 0x5628335e2eb0 2024-06-02T23:19:46.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.762+0000 7f0e877fe700 1 -- 172.21.15.52:0/1098300513 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9409b4f0 con 0x5628335e2eb0 2024-06-02T23:19:46.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 -- 172.21.15.52:0/1098300513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 msgr2=0x5628335d7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335d7900 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f0e9409b220 tx=0x7f0e94009480 comp rx=0 tx=0).stop 2024-06-02T23:19:46.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 -- 172.21.15.52:0/1098300513 shutdown_connections 2024-06-02T23:19:46.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2eb0 0x5628335d7900 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 --2- 172.21.15.52:0/1098300513 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2590 0x5628335d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 -- 172.21.15.52:0/1098300513 >> 172.21.15.52:0/1098300513 conn(0x562833509210 msgr2=0x5628335e4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 -- 172.21.15.52:0/1098300513 shutdown_connections 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 -- 172.21.15.52:0/1098300513 wait complete. 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 -- start start 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.763+0000 7f0eae86db80 1 Processor -- start 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335fb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2590 0x5628335fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2eb0 0x5628335f9e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1abbb0 con 0x5628335e2590 2024-06-02T23:19:46.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1ab4e0 con 0x5628335dcce0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e9c1a8700 con 0x5628335e2eb0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0eae86db80 10 client.0 _ll_register_callbacks cb 0x5628334c3740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0ea1d86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335fb970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37726/0 (socket says 172.21.15.52:37726) 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0ea1d86700 1 -- 172.21.15.52:0/2067543190 learned_addr learned my addr 172.21.15.52:0/2067543190 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0ea1585700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2590 0x5628335fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.764+0000 7f0ea2587700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2eb0 0x5628335f9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1d86700 1 -- 172.21.15.52:0/2067543190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2eb0 msgr2=0x5628335f9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2eb0 0x5628335f9e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1d86700 1 -- 172.21.15.52:0/2067543190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2590 msgr2=0x5628335fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2590 0x5628335fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1d86700 1 -- 172.21.15.52:0/2067543190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628335fa520 con 0x5628335dcce0 2024-06-02T23:19:46.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea2587700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2eb0 0x5628335f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1585700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2590 0x5628335fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0ea1d86700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335fb970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800b2b0 tx=0x7f0e9800fa10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13b7b8d2b6afbbdf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0e857fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0e857fa700 1 -- 172.21.15.52:0/2067543190 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98005900 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0e857fa700 1 -- 172.21.15.52:0/2067543190 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0e98005300 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9409b990 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0e857fa700 1 -- 172.21.15.52:0/2067543190 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0e98046070 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0eae86db80 10 client.27245 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.765+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562833661870 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0e857fa700 1 -- 172.21.15.52:0/2067543190 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0e9409b990 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0e857fa700 1 -- 172.21.15.52:0/2067543190 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0e98029040 con 0x5628335dcce0 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0eae86db80 10 client.27245 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0eae86db80 10 client.27245 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:19:46.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0eae86db80 20 client.27245 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0eae86db80 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562833661f30 0x562833664350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0eae86db80 4 client.27245 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.3, "await_for": 45.0}] 2024-06-02T23:19:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.3, "await_for": 45.0}) -- 0x562833535460 con 0x562833661f30 2024-06-02T23:19:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.766+0000 7f0ea1585700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562833661f30 0x562833664350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.767+0000 7f0ea1585700 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562833661f30 0x562833664350 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a30c1be855e2f7 server_cookie=8dfa5547240258b3 in_seq=0 out_seq=0 2024-06-02T23:19:46.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.767+0000 7f0e857fa700 10 client.27245 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:46.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0e857fa700 1 -- 172.21.15.52:0/2067543190 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562833535460 con 0x562833661f30 2024-06-02T23:19:46.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0e857fa700 10 client.27245 handle_command_reply: tid=0 2024-06-02T23:19:46.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 client.27245 shutdown 2024-06-02T23:19:46.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 msgr2=0x5628335fb970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335fb970 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800b2b0 tx=0x7f0e9800fa10 comp rx=0 tx=0).stop 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 shutdown_connections 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562833661f30 0x562833664350 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628335e2eb0 0x5628335f9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628335dcce0 0x5628335fb970 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 --2- 172.21.15.52:0/2067543190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628335e2590 0x5628335fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 >> 172.21.15.52:0/2067543190 conn(0x562833509210 msgr2=0x5628335e57b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.857+0000 7f0e857fa700 0 client.27245 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 shutdown_connections 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0eae86db80 1 -- 172.21.15.52:0/2067543190 wait complete. 2024-06-02T23:19:46.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0eae86db80 20 client.27245 trim_cache size 0 max 16384 2024-06-02T23:19:46.859 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:19:46.859 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:19:46.859 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:19:46.859 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 26, 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "534f2182": { 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 26, 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 146.3, 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:19:46.860 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:19:46.861 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:19:46.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0e880a2d40 msgr2=0x7f0e880a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0e880a2d40 0x7f0e880a5200 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c078960 tx=0x7f0e8c0119c0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 msgr2=0x7f0e9c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:46.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.858+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c198910 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0e9400efc0 tx=0x7f0e9400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 shutdown_connections 2024-06-02T23:19:46.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0e880a2d40 0x7f0e880a5200 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c108520 0x7f0e9c198910 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0e9c06aa60 0x7f0e9c19e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 --2- 172.21.15.52:0/1301899379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e9c06b420 0x7f0e9c19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:46.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 >> 172.21.15.52:0/1301899379 conn(0x7f0e9c0783d0 msgr2=0x7f0e9c0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:46.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 shutdown_connections 2024-06-02T23:19:46.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:19:46.859+0000 7f0ea3fea700 1 -- 172.21.15.52:0/1301899379 wait complete. 2024-06-02T23:19:47.326 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:47 smithi123 ceph-mon[35977]: pgmap v668: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-02T23:19:47.327 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:47 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:19:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:47 smithi174 ceph-mon[35994]: pgmap v668: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-02T23:19:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:47 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:19:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:47 smithi052 ceph-mon[32340]: pgmap v668: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 1.5 MiB/s wr, 189 op/s 2024-06-02T23:19:47.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:47 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:19:47.630 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '534f2182', quiesce took 6.3 seconds. Will release after: 20.0 2024-06-02T23:19:47.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.96 MB/sec execute 296 sec latency 750.008 ms 2024-06-02T23:19:48.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.96 MB/sec execute 297 sec latency 1750.053 ms 2024-06-02T23:19:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: pgmap v669: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-02T23:19:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:48 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: pgmap v669: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-02T23:19:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:48 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: pgmap v669: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 142 op/s 2024-06-02T23:19:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:48 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:49.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.95 MB/sec execute 298 sec latency 2750.158 ms 2024-06-02T23:19:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:19:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:19:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:49 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:19:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:19:50.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:49 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:50.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:19:50.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:19:50.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:49 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:19:50.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.94 MB/sec execute 299 sec latency 3750.211 ms 2024-06-02T23:19:51.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.94 MB/sec execute 300 sec latency 4750.331 ms 2024-06-02T23:19:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:51 smithi123 ceph-mon[35977]: pgmap v670: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:19:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:51 smithi174 ceph-mon[35994]: pgmap v670: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:19:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:51 smithi052 ceph-mon[32340]: pgmap v670: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:19:52.635 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.93 MB/sec execute 301 sec latency 5750.442 ms 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.454+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/2016711301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d581076e0 msgr2=0x7f7d58109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.454+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/2016711301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d581076e0 0x7f7d58109b30 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009ca0 tx=0x7f7d5002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/2016711301 shutdown_connections 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/2016711301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d58104db0 0x7f7d581071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/2016711301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d581076e0 0x7f7d58109b30 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/2016711301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 0x7f7d58104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/2016711301 >> 172.21.15.52:0/2016711301 conn(0x7f7d580fbe90 msgr2=0x7f7d580fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/2016711301 shutdown_connections 2024-06-02T23:19:53.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.455+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/2016711301 wait complete. 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 Processor -- start 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 -- start start 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 0x7f7d58197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 0x7f7d58198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d581076e0 0x7f7d5819f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810d550 con 0x7f7d58104db0 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810d3d0 con 0x7f7d58102480 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5cf5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5810d6d0 con 0x7f7d581076e0 2024-06-02T23:19:53.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 0x7f7d58198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 0x7f7d58198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38168/0 (socket says 172.21.15.52:38168) 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 -- 172.21.15.52:0/117705924 learned_addr learned my addr 172.21.15.52:0/117705924 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d581076e0 0x7f7d5819f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5659c640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 0x7f7d58197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 -- 172.21.15.52:0/117705924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d581076e0 msgr2=0x7f7d5819f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d581076e0 0x7f7d5819f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 -- 172.21.15.52:0/117705924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 msgr2=0x7f7d58197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 0x7f7d58197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d4ffff640 1 -- 172.21.15.52:0/117705924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5819f730 con 0x7f7d58104db0 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d581076e0 0x7f7d5819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.456+0000 7f7d5659c640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 0x7f7d58197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.457+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 0x7f7d58198050 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000e9f0 tx=0x7f7d4000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:53.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.457+0000 7f7d4f7fe640 1 -- 172.21.15.52:0/117705924 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000ccc0 con 0x7f7d58104db0 2024-06-02T23:19:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.457+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5819fa20 con 0x7f7d58104db0 2024-06-02T23:19:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.457+0000 7f7d4f7fe640 1 -- 172.21.15.52:0/117705924 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d400054d0 con 0x7f7d58104db0 2024-06-02T23:19:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.457+0000 7f7d4f7fe640 1 -- 172.21.15.52:0/117705924 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d40010430 con 0x7f7d58104db0 2024-06-02T23:19:53.459 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.457+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5819ff70 con 0x7f7d58104db0 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.460+0000 7f7d4f7fe640 1 -- 172.21.15.52:0/117705924 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7d40005020 con 0x7f7d58104db0 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.460+0000 7f7d5fc63740 1 Processor -- start 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.460+0000 7f7d5fc63740 1 -- start start 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54db430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca54e0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54e0b70 0x561ca54e3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561ca543e4f0 con 0x561ca54db010 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ca53dc870 con 0x561ca54dba00 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ca543a4b0 con 0x561ca54e0b70 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54db430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38178/0 (socket says 172.21.15.52:38178) 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 -- 172.21.15.52:0/2625548747 learned_addr learned my addr 172.21.15.52:0/2625548747 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d5659c640 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca54e0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d4f7fe640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7d440a2c30 0x7f7d440a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54e0b70 0x561ca54e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 -- 172.21.15.52:0/2625548747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54e0b70 msgr2=0x561ca54e3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54e0b70 0x561ca54e3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 -- 172.21.15.52:0/2625548747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 msgr2=0x561ca54e0500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca54e0500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 -- 172.21.15.52:0/2625548747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca53f89b0 con 0x561ca54db010 2024-06-02T23:19:53.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d4f7fe640 1 -- 172.21.15.52:0/117705924 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7d400cf540 con 0x7f7d58104db0 2024-06-02T23:19:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.461+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7d440a2c30 0x7f7d440a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.462+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54db430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009a60 tx=0x7f7d500093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b535e396750c9412 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.462+0000 7f7d4cff9640 1 -- 172.21.15.52:0/2625548747 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50031cc0 con 0x561ca54db010 2024-06-02T23:19:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.462+0000 7f7d4cff9640 1 -- 172.21.15.52:0/2625548747 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d50036040 con 0x561ca54db010 2024-06-02T23:19:53.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.462+0000 7f7d4cff9640 1 -- 172.21.15.52:0/2625548747 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50030820 con 0x561ca54db010 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 -- 172.21.15.52:0/2625548747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 msgr2=0x561ca54db430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7d440a2c30 0x7f7d440a50f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d5004c000 tx=0x7f7d50049000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54db430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f7d50009a60 tx=0x7f7d500093e0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 -- 172.21.15.52:0/2625548747 shutdown_connections 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54e0b70 0x561ca54e3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca54e0500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.467 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/2625548747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54db430 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 -- 172.21.15.52:0/2625548747 >> 172.21.15.52:0/2625548747 conn(0x561ca54d6680 msgr2=0x561ca54d6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 -- 172.21.15.52:0/2625548747 shutdown_connections 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 -- 172.21.15.52:0/2625548747 wait complete. 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 -- start start 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 Processor -- start 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca54e5190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 0x561ca54e56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.463+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561ca54e39a0 con 0x561ca54dba00 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ca53c7ae0 con 0x561ca54e0b70 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ca54e3b20 con 0x561ca54db010 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d5fc63740 10 client.0 _ll_register_callbacks cb 0x561ca54009b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:53.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 0x561ca54e56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 0x561ca54e56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38184/0 (socket says 172.21.15.52:38184) 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 -- 172.21.15.52:0/724187903 learned_addr learned my addr 172.21.15.52:0/724187903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d5659c640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca54e5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 -- 172.21.15.52:0/724187903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 msgr2=0x561ca54e5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca54e5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 -- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 msgr2=0x561ca54ec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54ec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 -- 172.21.15.52:0/724187903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca5152210 con 0x561ca54dba00 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 0x561ca54e56d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d400987a0 tx=0x7f7d400881a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1474cb25ade9832 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d3dffb640 1 -- 172.21.15.52:0/724187903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400046c0 con 0x561ca54dba00 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d3dffb640 1 -- 172.21.15.52:0/724187903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7d400a45b0 con 0x561ca54dba00 2024-06-02T23:19:53.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.464+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d50036040 con 0x561ca54dba00 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d3dffb640 1 -- 172.21.15.52:0/724187903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d40005310 con 0x561ca54dba00 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 10 client.17766 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ca521e620 con 0x561ca54dba00 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d3dffb640 1 -- 172.21.15.52:0/724187903 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d50036040 con 0x561ca54dba00 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d3dffb640 1 -- 172.21.15.52:0/724187903 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7d400b1030 con 0x561ca54dba00 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 10 client.17766 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 10 client.17766 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 20 client.17766 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca54f0fd0 0x561ca54f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 4 client.17766 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ca5451f10 con 0x561ca54f0fd0 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.465+0000 7f7d5659c640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca54f0fd0 0x561ca54f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.466+0000 7f7d3dffb640 10 client.17766 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.466+0000 7f7d5659c640 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca54f0fd0 0x561ca54f33c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b739e6c417d5c9ca server_cookie=de0955b7759321c9 in_seq=0 out_seq=0 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.466+0000 7f7d3dffb640 1 -- 172.21.15.52:0/724187903 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561ca5451f10 con 0x561ca54f0fd0 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.466+0000 7f7d3dffb640 10 client.17766 handle_command_reply: tid=0 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.466+0000 7f7d5fc63740 1 client.17766 shutdown 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 msgr2=0x561ca54e56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 0x561ca54e56d0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f7d400987a0 tx=0x7f7d400881a0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 shutdown_connections 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca54f0fd0 0x561ca54f33c0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54dba00 0x561ca54e56d0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/724187903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca54e5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d3dffb640 0 client.17766 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 >> 172.21.15.52:0/724187903 conn(0x561ca54d6680 msgr2=0x561ca54d57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:53.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 shutdown_connections 2024-06-02T23:19:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 1 -- 172.21.15.52:0/724187903 wait complete. 2024-06-02T23:19:53.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.467+0000 7f7d5fc63740 20 client.17766 trim_cache size 0 max 16384 2024-06-02T23:19:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 Processor -- start 2024-06-02T23:19:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 -- start start 2024-06-02T23:19:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54dba00 0x561ca54d9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54f7c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a8c20 con 0x561ca54db010 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581aa760 con 0x561ca54e0b70 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.482+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a92f0 con 0x561ca54dba00 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38200/0 (socket says 172.21.15.52:38200) 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 -- 172.21.15.52:0/520611332 learned_addr learned my addr 172.21.15.52:0/520611332 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54f7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54dba00 0x561ca54d9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 -- 172.21.15.52:0/520611332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54dba00 msgr2=0x561ca54d9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54dba00 0x561ca54d9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 -- 172.21.15.52:0/520611332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 msgr2=0x561ca54f7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54f7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 -- 172.21.15.52:0/520611332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca54f3730 con 0x561ca54db010 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54dba00 0x561ca54d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:19:53.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d5659c640 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54d19c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48014900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a491169ad0844e27 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d3ffff640 1 -- 172.21.15.52:0/520611332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48014190 con 0x561ca54db010 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d3ffff640 1 -- 172.21.15.52:0/520611332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d480035f0 con 0x561ca54db010 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.483+0000 7f7d3ffff640 1 -- 172.21.15.52:0/520611332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d480126b0 con 0x561ca54db010 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 -- 172.21.15.52:0/520611332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 msgr2=0x561ca54d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54d19c0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7d48000c00 tx=0x7f7d48014900 comp rx=0 tx=0).stop 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 -- 172.21.15.52:0/520611332 shutdown_connections 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54dba00 0x561ca54d9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54e0b70 0x561ca54f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/520611332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54db010 0x561ca54d19c0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 -- 172.21.15.52:0/520611332 >> 172.21.15.52:0/520611332 conn(0x561ca54d6680 msgr2=0x561ca54d9530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 -- 172.21.15.52:0/520611332 shutdown_connections 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 -- 172.21.15.52:0/520611332 wait complete. 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 -- start start 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.484+0000 7f7d5fc63740 1 Processor -- start 2024-06-02T23:19:53.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca5556bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca5557100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 0x561ca555e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581aa090 con 0x561ca54e0b70 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a99c0 con 0x561ca54dba00 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7d581a6c00 con 0x561ca54db010 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 10 client.0 _ll_register_callbacks cb 0x7f7d40004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 0x561ca555e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 0x561ca555e140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38202/0 (socket says 172.21.15.52:38202) 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 -- 172.21.15.52:0/3598690610 learned_addr learned my addr 172.21.15.52:0/3598690610 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d4ffff640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca5557100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5659c640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca5556bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 -- 172.21.15.52:0/3598690610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 msgr2=0x561ca5556bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca5556bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 -- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 msgr2=0x561ca5557100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca5557100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 -- 172.21.15.52:0/3598690610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca555e730 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5659c640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca5556bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d56d9d640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 0x561ca555e140 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7d50031970 tx=0x7f7d50031ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b9e5bfa64fbfa5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d3dffb640 1 -- 172.21.15.52:0/3598690610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d50063060 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d3dffb640 1 -- 172.21.15.52:0/3598690610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7d50030c90 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.485+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d480035f0 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d3dffb640 1 -- 172.21.15.52:0/3598690610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7d50030e90 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 10 client.17772 fetch_fsmap learned FSMap version 32 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561ca555fa40 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d3dffb640 1 -- 172.21.15.52:0/3598690610 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7d480035f0 con 0x561ca54e0b70 2024-06-02T23:19:53.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d3dffb640 1 -- 172.21.15.52:0/3598690610 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7d500525a0 con 0x561ca54e0b70 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 10 client.17772 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 10 client.17772 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 20 client.17772 populate_metadata read hostname 'smithi052' 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca55601c0 0x561ca55625e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 4 client.17772 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561ca544bc20 con 0x561ca55601c0 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.486+0000 7f7d5659c640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca55601c0 0x561ca55625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.487+0000 7f7d5659c640 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca55601c0 0x561ca55625e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1d520f4377bca41 server_cookie=5e66fca8fd10eb48 in_seq=0 out_seq=0 2024-06-02T23:19:53.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.487+0000 7f7d3dffb640 10 client.17772 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d3dffb640 1 -- 172.21.15.52:0/3598690610 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561ca544bc20 con 0x561ca55601c0 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d3dffb640 10 client.17772 handle_command_reply: tid=0 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 client.17772 shutdown 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 msgr2=0x561ca555e140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 0x561ca555e140 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7d50031970 tx=0x7f7d50031ef0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 shutdown_connections 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561ca55601c0 0x561ca55625e0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca54e0b70 0x561ca555e140 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca54dba00 0x561ca5557100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d3dffb640 0 client.17772 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 --2- 172.21.15.52:0/3598690610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca54db010 0x561ca5556bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.510+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 >> 172.21.15.52:0/3598690610 conn(0x561ca54d6680 msgr2=0x561ca54d9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 shutdown_connections 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5fc63740 1 -- 172.21.15.52:0/3598690610 wait complete. 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5fc63740 20 client.17772 trim_cache size 0 max 16384 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:19:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7d440a2c30 msgr2=0x7f7d440a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7d440a2c30 0x7f7d440a50f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d5004c000 tx=0x7f7d50049000 comp rx=0 tx=0).stop 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 msgr2=0x7f7d58198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 0x7f7d58198050 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f7d4000e9f0 tx=0x7f7d4000eeb0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 shutdown_connections 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7d440a2c30 0x7f7d440a50f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7d581076e0 0x7f7d5819f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7d58104db0 0x7f7d58198050 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 --2- 172.21.15.52:0/117705924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d58102480 0x7f7d58197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:19:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 >> 172.21.15.52:0/117705924 conn(0x7f7d580fbe90 msgr2=0x7f7d58102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:19:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 shutdown_connections 2024-06-02T23:19:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:19:53.511+0000 7f7d5cf5a640 1 -- 172.21.15.52:0/117705924 wait complete. 2024-06-02T23:19:53.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.92 MB/sec execute 302 sec latency 6750.550 ms 2024-06-02T23:19:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:53 smithi123 ceph-mon[35977]: pgmap v671: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-06-02T23:19:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:53 smithi174 ceph-mon[35994]: pgmap v671: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-06-02T23:19:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:53 smithi052 ceph-mon[32340]: pgmap v671: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 1.1 MiB/s wr, 125 op/s 2024-06-02T23:19:54.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.92 MB/sec execute 303 sec latency 7750.657 ms 2024-06-02T23:19:55.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.91 MB/sec execute 304 sec latency 8750.744 ms 2024-06-02T23:19:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:55 smithi123 ceph-mon[35977]: pgmap v672: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 804 KiB/s wr, 97 op/s 2024-06-02T23:19:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:55 smithi174 ceph-mon[35994]: pgmap v672: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 804 KiB/s wr, 97 op/s 2024-06-02T23:19:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:55 smithi052 ceph-mon[32340]: pgmap v672: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 804 KiB/s wr, 97 op/s 2024-06-02T23:19:56.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.91 MB/sec execute 305 sec latency 9750.781 ms 2024-06-02T23:19:57.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.90 MB/sec execute 306 sec latency 10750.842 ms 2024-06-02T23:19:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:57 smithi123 ceph-mon[35977]: pgmap v673: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 759 KiB/s wr, 98 op/s 2024-06-02T23:19:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:57 smithi174 ceph-mon[35994]: pgmap v673: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 759 KiB/s wr, 98 op/s 2024-06-02T23:19:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:57 smithi052 ceph-mon[32340]: pgmap v673: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 759 KiB/s wr, 98 op/s 2024-06-02T23:19:58.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.89 MB/sec execute 307 sec latency 11750.949 ms 2024-06-02T23:19:59.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.89 MB/sec execute 308 sec latency 12751.009 ms 2024-06-02T23:19:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:19:59 smithi123 ceph-mon[35977]: pgmap v674: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 440 KiB/s wr, 48 op/s 2024-06-02T23:19:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:19:59 smithi174 ceph-mon[35994]: pgmap v674: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 440 KiB/s wr, 48 op/s 2024-06-02T23:19:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:19:59 smithi052 ceph-mon[32340]: pgmap v674: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 440 KiB/s wr, 48 op/s 2024-06-02T23:20:00.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.88 MB/sec execute 309 sec latency 13751.099 ms 2024-06-02T23:20:00.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:00 smithi123 ceph-mon[35977]: overall HEALTH_OK 2024-06-02T23:20:00.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:00 smithi174 ceph-mon[35994]: overall HEALTH_OK 2024-06-02T23:20:00.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:00 smithi052 ceph-mon[32340]: overall HEALTH_OK 2024-06-02T23:20:01.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.87 MB/sec execute 310 sec latency 14751.191 ms 2024-06-02T23:20:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:01 smithi123 ceph-mon[35977]: pgmap v675: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 346 KiB/s wr, 41 op/s 2024-06-02T23:20:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:01 smithi174 ceph-mon[35994]: pgmap v675: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 346 KiB/s wr, 41 op/s 2024-06-02T23:20:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:01 smithi052 ceph-mon[32340]: pgmap v675: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 346 KiB/s wr, 41 op/s 2024-06-02T23:20:02.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.87 MB/sec execute 311 sec latency 15751.258 ms 2024-06-02T23:20:03.367 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:20:03.633 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:03.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.86 MB/sec execute 312 sec latency 16751.304 ms 2024-06-02T23:20:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:03 smithi052 ceph-mon[32340]: pgmap v676: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 119 KiB/s wr, 24 op/s 2024-06-02T23:20:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:03 smithi123 ceph-mon[35977]: pgmap v676: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 119 KiB/s wr, 24 op/s 2024-06-02T23:20:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:03 smithi174 ceph-mon[35994]: pgmap v676: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 119 KiB/s wr, 24 op/s 2024-06-02T23:20:04.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.86 MB/sec execute 313 sec latency 17751.332 ms 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 -- 172.21.15.52:0/492993654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 msgr2=0x7f90e0107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 --2- 172.21.15.52:0/492993654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e0107110 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f90d40099c0 tx=0x7f90d4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 -- 172.21.15.52:0/492993654 shutdown_connections 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 --2- 172.21.15.52:0/492993654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e0107840 0x7f90e010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 --2- 172.21.15.52:0/492993654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e0102740 0x7f90e0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 --2- 172.21.15.52:0/492993654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e0107110 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 -- 172.21.15.52:0/492993654 >> 172.21.15.52:0/492993654 conn(0x7f90e00795f0 msgr2=0x7f90e0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:05.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 -- 172.21.15.52:0/492993654 shutdown_connections 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 -- 172.21.15.52:0/492993654 wait complete. 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 Processor -- start 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.605+0000 7f90e7ced700 1 -- start start 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e0102740 0x7f90e019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e0107840 0x7f90e0198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0112600 con 0x7f90e01030f0 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0112480 con 0x7f90e0102740 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0112780 con 0x7f90e0107840 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e019ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44568/0 (socket says 172.21.15.52:44568) 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 -- 172.21.15.52:0/2704897585 learned_addr learned my addr 172.21.15.52:0/2704897585 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5a89700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e0102740 0x7f90e019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e628a700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e0107840 0x7f90e0198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 -- 172.21.15.52:0/2704897585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e0107840 msgr2=0x7f90e0198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e0107840 0x7f90e0198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 -- 172.21.15.52:0/2704897585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e0102740 msgr2=0x7f90e019e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e0102740 0x7f90e019e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 -- 172.21.15.52:0/2704897585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e0198fe0 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e5288700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e019ebc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006010 tx=0x7f90d4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90d2ffd700 1 -- 172.21.15.52:0/2704897585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4027070 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90d2ffd700 1 -- 172.21.15.52:0/2704897585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d4004050 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90d2ffd700 1 -- 172.21.15.52:0/2704897585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d4022e10 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.606+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e0199270 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.607+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e01a54e0 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.609+0000 7f90d2ffd700 1 -- 172.21.15.52:0/2704897585 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f90d402a070 con 0x7f90e01030f0 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.609+0000 7f90d2ffd700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90cc0a2ca0 0x7f90cc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.609+0000 7f90d2ffd700 1 -- 172.21.15.52:0/2704897585 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f90d40cfe20 con 0x7f90e01030f0 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.609+0000 7f90e5a89700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90cc0a2ca0 0x7f90cc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90e5a89700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90cc0a2ca0 0x7f90cc0a5160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005ea0 tx=0x7f90dc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 Processor -- start 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 -- start start 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c491750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c48b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c48bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557a0c379f60 con 0x557a0c491c90 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557a0c3d8850 con 0x557a0c491370 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.610+0000 7f90f2570b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a0c340460 con 0x557a0c48bbb0 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c48b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c48b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44570/0 (socket says 172.21.15.52:44570) 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 -- 172.21.15.52:0/3106996326 learned_addr learned my addr 172.21.15.52:0/3106996326 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5a89700 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c491750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 -- 172.21.15.52:0/3106996326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 msgr2=0x557a0c48bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e628a700 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c48bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c48bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 -- 172.21.15.52:0/3106996326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 msgr2=0x557a0c491750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c491750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 -- 172.21.15.52:0/3106996326 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0c3278d0 con 0x557a0c491c90 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90e5288700 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c48b670 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f90d409c6d0 tx=0x7f90d409c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dbfb8ea3515d3c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90cbfff700 1 -- 172.21.15.52:0/3106996326 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d409ce40 con 0x557a0c491c90 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90cbfff700 1 -- 172.21.15.52:0/3106996326 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d409c740 con 0x557a0c491c90 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.611+0000 7f90cbfff700 1 -- 172.21.15.52:0/3106996326 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d409cc00 con 0x557a0c491c90 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 -- 172.21.15.52:0/3106996326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 msgr2=0x557a0c48b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c48b670 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f90d409c6d0 tx=0x7f90d409c590 comp rx=0 tx=0).stop 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 -- 172.21.15.52:0/3106996326 shutdown_connections 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c48bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c491750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3106996326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c48b670 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 -- 172.21.15.52:0/3106996326 >> 172.21.15.52:0/3106996326 conn(0x557a0c3b4080 msgr2=0x557a0c482220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 -- 172.21.15.52:0/3106996326 shutdown_connections 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 -- 172.21.15.52:0/3106996326 wait complete. 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 -- start start 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.612+0000 7f90f2570b80 1 Processor -- start 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c48bbb0 0x557a0c497a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c494d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c4952a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557a0c378fc0 con 0x557a0c491c90 2024-06-02T23:20:05.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557a0c3d4810 con 0x557a0c48bbb0 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557a0c498430 con 0x557a0c491370 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90f2570b80 10 client.0 _ll_register_callbacks cb 0x557a0c36e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c4952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c4952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44576/0 (socket says 172.21.15.52:44576) 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 -- 172.21.15.52:0/4125685286 learned_addr learned my addr 172.21.15.52:0/4125685286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e5288700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c494d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e5a89700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c48bbb0 0x557a0c497a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 -- 172.21.15.52:0/4125685286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 msgr2=0x557a0c494d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c494d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 -- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c48bbb0 msgr2=0x557a0c497a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c48bbb0 0x557a0c497a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 -- 172.21.15.52:0/4125685286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0c39e750 con 0x557a0c491c90 2024-06-02T23:20:05.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e5a89700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c48bbb0 0x557a0c497a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90e628a700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c4952a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f90d800f930 tx=0x7f90d800fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a555356668d119 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.613+0000 7f90c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90c97fa700 1 -- 172.21.15.52:0/4125685286 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d800d070 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90c97fa700 1 -- 172.21.15.52:0/4125685286 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f90d8003c00 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90d409c740 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90c97fa700 1 -- 172.21.15.52:0/4125685286 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f90d8010040 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 10 client.17784 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557a0c4a2be0 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90c97fa700 1 -- 172.21.15.52:0/4125685286 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90d409c740 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90c97fa700 1 -- 172.21.15.52:0/4125685286 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f90d8013050 con 0x557a0c491c90 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 10 client.17784 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 10 client.17784 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:05.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 20 client.17784 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c4a5270 0x557a0c4a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 4 client.17784 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.614+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a0c3e5d50 con 0x557a0c4a5270 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.615+0000 7f90e5a89700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c4a5270 0x557a0c4a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.615+0000 7f90e5a89700 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c4a5270 0x557a0c4a7660 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3255c98672eccb4f server_cookie=fca0c523cb5e3dad in_seq=0 out_seq=0 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.615+0000 7f90c97fa700 10 client.17784 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90c97fa700 1 -- 172.21.15.52:0/4125685286 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557a0c3e5d50 con 0x557a0c4a5270 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90c97fa700 10 client.17784 handle_command_reply: tid=0 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 client.17784 shutdown 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 msgr2=0x557a0c4952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c4952a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f90d800f930 tx=0x7f90d800fc40 comp rx=0 tx=0).stop 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 shutdown_connections 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c4a5270 0x557a0c4a7660 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c4952a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c48bbb0 0x557a0c497a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 --2- 172.21.15.52:0/4125685286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c494d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90c97fa700 0 client.17784 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 >> 172.21.15.52:0/4125685286 conn(0x557a0c3b4080 msgr2=0x557a0c481700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 shutdown_connections 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 1 -- 172.21.15.52:0/4125685286 wait complete. 2024-06-02T23:20:05.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.616+0000 7f90f2570b80 20 client.17784 trim_cache size 0 max 16384 2024-06-02T23:20:05.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.85 MB/sec execute 314 sec latency 18751.362 ms 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 Processor -- start 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 -- start start 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 0x557a0c481e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c4823c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491c90 0x557a0c492130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01aa5b0 con 0x557a0c48bbb0 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01ac0f0 con 0x557a0c491c90 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90f2570b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01aac80 con 0x557a0c491370 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.637+0000 7f90e5a89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 0x557a0c481e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 0x557a0c481e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44582/0 (socket says 172.21.15.52:44582) 2024-06-02T23:20:05.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 -- 172.21.15.52:0/3680041058 learned_addr learned my addr 172.21.15.52:0/3680041058 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e628a700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491c90 0x557a0c492130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5288700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c4823c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 -- 172.21.15.52:0/3680041058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 msgr2=0x557a0c4823c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c4823c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 -- 172.21.15.52:0/3680041058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491c90 msgr2=0x557a0c492130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491c90 0x557a0c492130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 -- 172.21.15.52:0/3680041058 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a0c493cc0 con 0x557a0c48bbb0 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e628a700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491c90 0x557a0c492130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:05.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5288700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c4823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:05.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90e5a89700 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 0x557a0c481e80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x557a0c4a7ed0 tx=0x7f90dc00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76d54afc7f08a4a0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:05.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90cb7fe700 1 -- 172.21.15.52:0/3680041058 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc020b40 con 0x557a0c48bbb0 2024-06-02T23:20:05.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90cb7fe700 1 -- 172.21.15.52:0/3680041058 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90dc020ce0 con 0x557a0c48bbb0 2024-06-02T23:20:05.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.638+0000 7f90cb7fe700 1 -- 172.21.15.52:0/3680041058 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90dc0264b0 con 0x557a0c48bbb0 2024-06-02T23:20:05.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 -- 172.21.15.52:0/3680041058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 msgr2=0x557a0c481e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 0x557a0c481e80 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x557a0c4a7ed0 tx=0x7f90dc00af90 comp rx=0 tx=0).stop 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 -- 172.21.15.52:0/3680041058 shutdown_connections 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491c90 0x557a0c492130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c48bbb0 0x557a0c481e80 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 --2- 172.21.15.52:0/3680041058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c491370 0x557a0c4823c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 -- 172.21.15.52:0/3680041058 >> 172.21.15.52:0/3680041058 conn(0x557a0c3b4080 msgr2=0x557a0c494d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 -- 172.21.15.52:0/3680041058 shutdown_connections 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 -- 172.21.15.52:0/3680041058 wait complete. 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 -- start start 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.639+0000 7f90f2570b80 1 Processor -- start 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c49d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c49dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c49a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01aba20 con 0x557a0c491c90 2024-06-02T23:20:05.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01ab350 con 0x557a0c491370 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90e01a8570 con 0x557a0c48bbb0 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90f2570b80 10 client.0 _ll_register_callbacks cb 0x557a0c36e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c49a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c49a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44596/0 (socket says 172.21.15.52:44596) 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 -- 172.21.15.52:0/1642798660 learned_addr learned my addr 172.21.15.52:0/1642798660 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e5a89700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c49d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e5288700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c49dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 -- 172.21.15.52:0/1642798660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 msgr2=0x557a0c49d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c49d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 -- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 msgr2=0x557a0c49dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c49dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 -- 172.21.15.52:0/1642798660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a0c49ac10 con 0x557a0c491c90 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e5288700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c49dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90e628a700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c49a650 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f90d800f900 tx=0x7f90d8003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff410fd293eb63f5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:05.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.640+0000 7f90c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90c97fa700 1 -- 172.21.15.52:0/1642798660 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d800d040 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90c97fa700 1 -- 172.21.15.52:0/1642798660 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f90d8004ae0 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90dc020ce0 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90c97fa700 1 -- 172.21.15.52:0/1642798660 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f90d802e030 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 10 client.17790 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557a0c510780 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90c97fa700 1 -- 172.21.15.52:0/1642798660 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f90dc020ce0 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90c97fa700 1 -- 172.21.15.52:0/1642798660 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f90d8031070 con 0x557a0c491c90 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 10 client.17790 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 10 client.17790 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 20 client.17790 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c510e00 0x557a0c513220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 4 client.17790 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:20:05.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.641+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557a0c3e02d0 con 0x557a0c510e00 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90e5a89700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c510e00 0x557a0c513220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90e5a89700 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c510e00 0x557a0c513220 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f87ec2b4b24ab85a server_cookie=284c990b58758ad0 in_seq=0 out_seq=0 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90c97fa700 10 client.17790 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90c97fa700 1 -- 172.21.15.52:0/1642798660 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557a0c3e02d0 con 0x557a0c510e00 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90c97fa700 10 client.17790 handle_command_reply: tid=0 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90f2570b80 1 client.17790 shutdown 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 msgr2=0x557a0c49a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.642+0000 7f90f2570b80 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c49a650 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f90d800f900 tx=0x7f90d8003c90 comp rx=0 tx=0).stop 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 shutdown_connections 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557a0c510e00 0x557a0c513220 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557a0c491c90 0x557a0c49a650 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557a0c48bbb0 0x557a0c49d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90c97fa700 0 client.17790 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 --2- 172.21.15.52:0/1642798660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557a0c491370 0x557a0c49dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 >> 172.21.15.52:0/1642798660 conn(0x557a0c3b4080 msgr2=0x557a0c492130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 shutdown_connections 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 1 -- 172.21.15.52:0/1642798660 wait complete. 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90f2570b80 20 client.17790 trim_cache size 0 max 16384 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:20:05.648 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90cc0a2ca0 msgr2=0x7f90cc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90cc0a2ca0 0x7f90cc0a5160 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f90dc005ea0 tx=0x7f90dc005e30 comp rx=0 tx=0).stop 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 msgr2=0x7f90e019ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e019ebc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f90d4006010 tx=0x7f90d4003680 comp rx=0 tx=0).stop 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 shutdown_connections 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f90cc0a2ca0 0x7f90cc0a5160 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90e0107840 0x7f90e0198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90e0102740 0x7f90e019e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 --2- 172.21.15.52:0/2704897585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f90e01030f0 0x7f90e019ebc0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:05.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.643+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 >> 172.21.15.52:0/2704897585 conn(0x7f90e00795f0 msgr2=0x7f90e01004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:05.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.644+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 shutdown_connections 2024-06-02T23:20:05.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:05.644+0000 7f90e7ced700 1 -- 172.21.15.52:0/2704897585 wait complete. 2024-06-02T23:20:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:05 smithi123 ceph-mon[35977]: pgmap v677: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-06-02T23:20:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:05 smithi174 ceph-mon[35994]: pgmap v677: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-06-02T23:20:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:05 smithi052 ceph-mon[32340]: pgmap v677: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 62 KiB/s wr, 15 op/s 2024-06-02T23:20:06.423 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:20:06.423 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:20:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 -- 172.21.15.52:0/3518984387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4104db0 msgr2=0x7f2bb41071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/3518984387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4104db0 0x7f2bb41071a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009900 tx=0x7f2b9c02a410 comp rx=0 tx=0).stop 2024-06-02T23:20:06.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 -- 172.21.15.52:0/3518984387 shutdown_connections 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/3518984387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41076e0 0x7f2bb4109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/3518984387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4104db0 0x7f2bb41071a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/3518984387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4102480 0x7f2bb4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 -- 172.21.15.52:0/3518984387 >> 172.21.15.52:0/3518984387 conn(0x7f2bb40fbe70 msgr2=0x7f2bb40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 -- 172.21.15.52:0/3518984387 shutdown_connections 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.602+0000 7f2bb9c95640 1 -- 172.21.15.52:0/3518984387 wait complete. 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 Processor -- start 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 -- start start 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 0x7f2bb4197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4104db0 0x7f2bb4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41076e0 0x7f2bb419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb410d540 con 0x7f2bb4102480 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb410d3c0 con 0x7f2bb41076e0 2024-06-02T23:20:06.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb9c95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb410d6c0 con 0x7f2bb4104db0 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 0x7f2bb4197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 0x7f2bb4197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44618/0 (socket says 172.21.15.52:44618) 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 -- 172.21.15.52:0/535129824 learned_addr learned my addr 172.21.15.52:0/535129824 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb3fff640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41076e0 0x7f2bb419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4104db0 0x7f2bb4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 -- 172.21.15.52:0/535129824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4104db0 msgr2=0x7f2bb4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4104db0 0x7f2bb4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 -- 172.21.15.52:0/535129824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41076e0 msgr2=0x7f2bb419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.603+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41076e0 0x7f2bb419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb37fe640 1 -- 172.21.15.52:0/535129824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bb419f770 con 0x7f2bb4102480 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 0x7f2bb4197b50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400d770 tx=0x7f2ba400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb0ff9640 1 -- 172.21.15.52:0/535129824 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4017070 con 0x7f2bb4102480 2024-06-02T23:20:06.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb0ff9640 1 -- 172.21.15.52:0/535129824 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba40054d0 con 0x7f2bb4102480 2024-06-02T23:20:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb0ff9640 1 -- 172.21.15.52:0/535129824 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba4004cf0 con 0x7f2bb4102480 2024-06-02T23:20:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bb419fa60 con 0x7f2bb4102480 2024-06-02T23:20:06.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.604+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bb419ff30 con 0x7f2bb4102480 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.606+0000 7f2bb0ff9640 1 -- 172.21.15.52:0/535129824 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ba4004e90 con 0x7f2bb4102480 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 Processor -- start 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 -- start start 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75168840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c7516d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c7516df80 0x555c75170410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555c750c74f0 con 0x555c75168420 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c75065870 con 0x555c7516df80 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c750c34b0 con 0x555c75168e10 2024-06-02T23:20:06.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bb0ff9640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ba00a2c30 0x7f2ba00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75168840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75168840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44632/0 (socket says 172.21.15.52:44632) 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.607+0000 7f2bb3fff640 1 -- 172.21.15.52:0/595605460 learned_addr learned my addr 172.21.15.52:0/595605460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c7516df80 0x555c75170410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb0ff9640 1 -- 172.21.15.52:0/535129824 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ba40cea80 con 0x7f2bb4102480 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb3fff640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ba00a2c30 0x7f2ba00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c7516d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/595605460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 msgr2=0x555c7516d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c7516d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/595605460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 msgr2=0x555c75168840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75168840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/595605460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c750819b0 con 0x555c7516df80 2024-06-02T23:20:06.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb3fff640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ba00a2c30 0x7f2ba00a50f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x555c75169670 tx=0x7f2bac007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c7516df80 0x555c75170410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009230 tx=0x7f2b9c032cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27eb16985684fc2e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2ba9ffb640 1 -- 172.21.15.52:0/595605460 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c03f070 con 0x555c7516df80 2024-06-02T23:20:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2ba9ffb640 1 -- 172.21.15.52:0/595605460 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b9c031780 con 0x555c7516df80 2024-06-02T23:20:06.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.608+0000 7f2ba9ffb640 1 -- 172.21.15.52:0/595605460 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c035070 con 0x555c7516df80 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 -- 172.21.15.52:0/595605460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c7516df80 msgr2=0x555c75170410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c7516df80 0x555c75170410 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009230 tx=0x7f2b9c032cc0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 -- 172.21.15.52:0/595605460 shutdown_connections 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c7516d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c7516df80 0x555c75170410 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/595605460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75168840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 -- 172.21.15.52:0/595605460 >> 172.21.15.52:0/595605460 conn(0x555c7515f680 msgr2=0x555c7515fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:06.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 -- 172.21.15.52:0/595605460 shutdown_connections 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.609+0000 7f2bbc99b740 1 -- 172.21.15.52:0/595605460 wait complete. 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 -- start start 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 Processor -- start 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c751726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c75172c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 0x555c75179be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555c75170ed0 con 0x555c7516df80 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c75050ae0 con 0x555c75168420 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c75171050 con 0x555c75168e10 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bbc99b740 10 client.0 _ll_register_callbacks cb 0x555c750899b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c751726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 0x555c75179be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c751726d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37030/0 (socket says 172.21.15.52:37030) 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bb37fe640 1 -- 172.21.15.52:0/1463621318 learned_addr learned my addr 172.21.15.52:0/1463621318 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 0x555c75179be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44642/0 (socket says 172.21.15.52:44642) 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.610+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c75172c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb37fe640 1 -- 172.21.15.52:0/1463621318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 msgr2=0x555c75172c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c75172c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb37fe640 1 -- 172.21.15.52:0/1463621318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 msgr2=0x555c75179be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 0x555c75179be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb37fe640 1 -- 172.21.15.52:0/1463621318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c74ddb210 con 0x555c75168420 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb3fff640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 0x555c75179be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c751726d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009fd0 tx=0x7f2ba4004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbf247276e75299b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2b92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2b92ffd640 1 -- 172.21.15.52:0/1463621318 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40b4060 con 0x555c75168420 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2b92ffd640 1 -- 172.21.15.52:0/1463621318 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ba4097980 con 0x555c75168420 2024-06-02T23:20:06.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9c031780 con 0x555c75168420 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2b92ffd640 1 -- 172.21.15.52:0/1463621318 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2ba4020040 con 0x555c75168420 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bbc99b740 10 client.27278 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c74ea7620 con 0x555c75168420 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.611+0000 7f2b92ffd640 1 -- 172.21.15.52:0/1463621318 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ba407b020 con 0x555c75168420 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2b92ffd640 1 -- 172.21.15.52:0/1463621318 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2ba407e020 con 0x555c75168420 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bbc99b740 10 client.27278 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bbc99b740 10 client.27278 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bbc99b740 20 client.27278 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c7517e430 0x555c75180880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bbc99b740 4 client.27278 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2b9c031780 con 0x555c7517e430 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c7517e430 0x555c75180880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2b92ffd640 10 client.27278 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.612+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c7517e430 0x555c75180880 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6efd8ca254438891 server_cookie=25c911e250467b62 in_seq=0 out_seq=0 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.613+0000 7f2b92ffd640 1 -- 172.21.15.52:0/1463621318 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f2b9c031780 con 0x555c7517e430 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.613+0000 7f2b92ffd640 10 client.27278 handle_command_reply: tid=0 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.613+0000 7f2bbc99b740 1 client.27278 shutdown 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.613+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 msgr2=0x555c751726d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.613+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c751726d0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4009fd0 tx=0x7f2ba4004790 comp rx=0 tx=0).stop 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 shutdown_connections 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c7517e430 0x555c75180880 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c75168e10 0x555c75172c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c7516df80 0x555c75179be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2b92ffd640 0 client.27278 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/1463621318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c751726d0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 >> 172.21.15.52:0/1463621318 conn(0x555c7515f680 msgr2=0x555c7515e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:06.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 shutdown_connections 2024-06-02T23:20:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 1 -- 172.21.15.52:0/1463621318 wait complete. 2024-06-02T23:20:06.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.614+0000 7f2bbc99b740 20 client.27278 trim_cache size 0 max 16384 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.631+0000 7f2bbc99b740 1 Processor -- start 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.631+0000 7f2bbc99b740 1 -- start start 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c75170370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 0x555c75179ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c75162ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41a6940 con 0x555c75168e10 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41a8480 con 0x555c75168420 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41a7010 con 0x555c7516df80 2024-06-02T23:20:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 0x555c75179ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 0x555c75179ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44652/0 (socket says 172.21.15.52:44652) 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/306840098 learned_addr learned my addr 172.21.15.52:0/306840098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c75170370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb3fff640 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c75162ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/306840098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 msgr2=0x555c75162ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c75162ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/306840098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 msgr2=0x555c75170370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c75170370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 -- 172.21.15.52:0/306840098 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c7515f4b0 con 0x555c75168e10 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.632+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 0x555c75179ba0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009230 tx=0x7f2b9c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5005cb4198b09e1d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2ba97fa640 1 -- 172.21.15.52:0/306840098 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c035040 con 0x555c75168e10 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2ba97fa640 1 -- 172.21.15.52:0/306840098 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b9c052070 con 0x555c75168e10 2024-06-02T23:20:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2ba97fa640 1 -- 172.21.15.52:0/306840098 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b9c03f040 con 0x555c75168e10 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2bbc99b740 1 -- 172.21.15.52:0/306840098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 msgr2=0x555c75179ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 0x555c75179ba0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c009230 tx=0x7f2b9c009260 comp rx=0 tx=0).stop 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2bbc99b740 1 -- 172.21.15.52:0/306840098 shutdown_connections 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c75162ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168e10 0x555c75179ba0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.633+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/306840098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168420 0x555c75170370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- 172.21.15.52:0/306840098 >> 172.21.15.52:0/306840098 conn(0x555c7515f680 msgr2=0x555c75162590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- 172.21.15.52:0/306840098 shutdown_connections 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- 172.21.15.52:0/306840098 wait complete. 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- start start 2024-06-02T23:20:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 Processor -- start 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75176a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168e10 0x555c75176f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c751e71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41a7db0 con 0x555c75168420 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41a76e0 con 0x555c75168e10 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.634+0000 7f2bbc99b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bb41a4920 con 0x555c7516df80 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bbc99b740 10 client.0 _ll_register_callbacks cb 0x7f2ba40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75176a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75176a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44666/0 (socket says 172.21.15.52:44666) 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 -- 172.21.15.52:0/620142559 learned_addr learned my addr 172.21.15.52:0/620142559 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168e10 0x555c75176f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb3fff640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c751e71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 -- 172.21.15.52:0/620142559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 msgr2=0x555c751e71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c751e71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 -- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168e10 msgr2=0x555c75176f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168e10 0x555c75176f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 -- 172.21.15.52:0/620142559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c751e76e0 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bb37fe640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75176a50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4097770 tx=0x7f2ba4005b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28418d382e3726c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2b92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2b92ffd640 1 -- 172.21.15.52:0/620142559 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ba40b4040 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2b92ffd640 1 -- 172.21.15.52:0/620142559 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ba409a970 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.635+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b9c052070 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2b92ffd640 1 -- 172.21.15.52:0/620142559 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2ba4005180 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 10 client.17808 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c751e8980 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2b92ffd640 1 -- 172.21.15.52:0/620142559 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b9c052070 con 0x555c75168420 2024-06-02T23:20:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2b92ffd640 1 -- 172.21.15.52:0/620142559 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2ba407e040 con 0x555c75168420 2024-06-02T23:20:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 10 client.17808 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 10 client.17808 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 20 client.17808 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c751e9360 0x555c751eb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 4 client.17808 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:20:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555c750daf10 con 0x555c751e9360 2024-06-02T23:20:06.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.85 MB/sec execute 315 sec latency 19751.393 ms 2024-06-02T23:20:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.636+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c751e9360 0x555c751eb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.637+0000 7f2bb2ffd640 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c751e9360 0x555c751eb780 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ab8d53dee8d0d0b server_cookie=2546b351f433753d in_seq=0 out_seq=0 2024-06-02T23:20:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.637+0000 7f2b92ffd640 10 client.17808 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2b92ffd640 1 -- 172.21.15.52:0/620142559 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555c750daf10 con 0x555c751e9360 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2b92ffd640 10 client.17808 handle_command_reply: tid=0 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 client.17808 shutdown 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 msgr2=0x555c75176a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75176a50 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f2ba4097770 tx=0x7f2ba4005b20 comp rx=0 tx=0).stop 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 shutdown_connections 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555c751e9360 0x555c751eb780 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c7516df80 0x555c751e71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2b92ffd640 0 client.17808 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c75168e10 0x555c75176f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 --2- 172.21.15.52:0/620142559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c75168420 0x555c75176a50 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 >> 172.21.15.52:0/620142559 conn(0x555c7515f680 msgr2=0x555c75162590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 shutdown_connections 2024-06-02T23:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 1 -- 172.21.15.52:0/620142559 wait complete. 2024-06-02T23:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.657+0000 7f2bbc99b740 20 client.17808 trim_cache size 0 max 16384 2024-06-02T23:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:20:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ba00a2c30 msgr2=0x7f2ba00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ba00a2c30 0x7f2ba00a50f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x555c75169670 tx=0x7f2bac007520 comp rx=0 tx=0).stop 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 msgr2=0x7f2bb4197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 0x7f2bb4197b50 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f2ba400d770 tx=0x7f2ba400dc30 comp rx=0 tx=0).stop 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 shutdown_connections 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ba00a2c30 0x7f2ba00a50f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bb4104db0 0x7f2bb4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2bb41076e0 0x7f2bb419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 --2- 172.21.15.52:0/535129824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2bb4102480 0x7f2bb4197b50 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 >> 172.21.15.52:0/535129824 conn(0x7f2bb40fbe70 msgr2=0x7f2bb4102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 shutdown_connections 2024-06-02T23:20:06.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:06.658+0000 7f2bb9c95640 1 -- 172.21.15.52:0/535129824 wait complete. 2024-06-02T23:20:06.686 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:07.632 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:20:07.636 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.84 MB/sec execute 316 sec latency 20751.448 ms 2024-06-02T23:20:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:07 smithi123 ceph-mon[35977]: pgmap v678: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 63 KiB/s wr, 16 op/s 2024-06-02T23:20:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:07 smithi174 ceph-mon[35994]: pgmap v678: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 63 KiB/s wr, 16 op/s 2024-06-02T23:20:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:07 smithi052 ceph-mon[32340]: pgmap v678: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 63 KiB/s wr, 16 op/s 2024-06-02T23:20:08.229 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:08.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.83 MB/sec execute 317 sec latency 21751.486 ms 2024-06-02T23:20:09.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 -- 172.21.15.52:0/411204960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 msgr2=0x7f596410f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:09.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 --2- 172.21.15.52:0/411204960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f596410f3c0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f595c00b0d0 tx=0x7f595c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 -- 172.21.15.52:0/411204960 shutdown_connections 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 --2- 172.21.15.52:0/411204960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f596410f3c0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 --2- 172.21.15.52:0/411204960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5964102740 0x7f5964102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 --2- 172.21.15.52:0/411204960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 0x7f5964107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 -- 172.21.15.52:0/411204960 >> 172.21.15.52:0/411204960 conn(0x7f59640795f0 msgr2=0x7f5964079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 -- 172.21.15.52:0/411204960 shutdown_connections 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.442+0000 7f5969e5f700 1 -- 172.21.15.52:0/411204960 wait complete. 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 Processor -- start 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 -- start start 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5964102740 0x7f596419e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 0x7f596419ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f59641988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5964112680 con 0x7f5964107840 2024-06-02T23:20:09.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5964112500 con 0x7f5964102740 2024-06-02T23:20:09.214 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:20:09.214 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:20:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.443+0000 7f5969e5f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5964112800 con 0x7f59641030f0 2024-06-02T23:20:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f595bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 0x7f596419ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f59637fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5964102740 0x7f596419e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:09.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f595bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 0x7f596419ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:55208/0 (socket says 172.21.15.52:55208) 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f595bfff700 1 -- 172.21.15.52:0/1000999175 learned_addr learned my addr 172.21.15.52:0/1000999175 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f59641988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 -- 172.21.15.52:0/1000999175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 msgr2=0x7f596419ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 0x7f596419ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 -- 172.21.15.52:0/1000999175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5964102740 msgr2=0x7f596419e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5964102740 0x7f596419e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 -- 172.21.15.52:0/1000999175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5964199020 con 0x7f5964107840 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.444+0000 7f5963fff700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f59641988c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f595c009fd0 tx=0x7f595c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.445+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c028070 con 0x7f5964107840 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.445+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59641992b0 con 0x7f5964107840 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.445+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595c004c50 con 0x7f5964107840 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.445+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c005660 con 0x7f5964107840 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.445+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59640746c0 con 0x7f5964107840 2024-06-02T23:20:09.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.446+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5964069180 con 0x7f5964107840 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.450+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f595c02b070 con 0x7f5964107840 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.450+0000 7f59617fa700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59500a2c20 0x7f59500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.451+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f595c00f020 con 0x7f5964107840 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.451+0000 7f59637fe700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59500a2c20 0x7f59500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.451+0000 7f59637fe700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59500a2c20 0x7f59500a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5954005950 tx=0x7f59540058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.459+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f595c032020 con 0x7f5964107840 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.723+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5964199df0 con 0x7f5964107840 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.723+0000 7f59617fa700 1 -- 172.21.15.52:0/1000999175 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f595c01a3c0 con 0x7f5964107840 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.725+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59500a2c20 msgr2=0x7f59500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.725+0000 7f5969e5f700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59500a2c20 0x7f59500a50e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f5954005950 tx=0x7f59540058e0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 msgr2=0x7f59641988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f59641988c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f595c009fd0 tx=0x7f595c007a00 comp rx=0 tx=0).stop 2024-06-02T23:20:09.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 shutdown_connections 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59500a2c20 0x7f59500a50e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5964107840 0x7f59641988c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5964102740 0x7f596419e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 --2- 172.21.15.52:0/1000999175 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59641030f0 0x7f596419ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 >> 172.21.15.52:0/1000999175 conn(0x7f59640795f0 msgr2=0x7f5964100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 shutdown_connections 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:08.726+0000 7f5969e5f700 1 -- 172.21.15.52:0/1000999175 wait complete. 2024-06-02T23:20:09.217 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:20:09.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.83 MB/sec execute 318 sec latency 22751.542 ms 2024-06-02T23:20:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:09 smithi052 ceph-mon[32340]: pgmap v679: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-06-02T23:20:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:09 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1000999175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:09 smithi123 ceph-mon[35977]: pgmap v679: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-06-02T23:20:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:09 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1000999175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:09 smithi174 ceph-mon[35994]: pgmap v679: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-06-02T23:20:09.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:09 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1000999175' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:10.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.82 MB/sec execute 319 sec latency 23751.625 ms 2024-06-02T23:20:10.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:20:11.414 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:11.631 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:11 smithi052 ceph-mon[32340]: pgmap v680: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-06-02T23:20:11.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.82 MB/sec execute 320 sec latency 24751.668 ms 2024-06-02T23:20:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:11 smithi123 ceph-mon[35977]: pgmap v680: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-06-02T23:20:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:11 smithi174 ceph-mon[35994]: pgmap v680: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.3 KiB/s wr, 13 op/s 2024-06-02T23:20:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 -- 172.21.15.52:0/3496867808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba28108500 msgr2=0x7fba281088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/3496867808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba28108500 0x7fba281088e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fba18009990 tx=0x7fba18009ca0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 -- 172.21.15.52:0/3496867808 shutdown_connections 2024-06-02T23:20:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/3496867808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba2806b580 0x7fba2810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/3496867808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba28108500 0x7fba281088e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/3496867808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba2806abc0 0x7fba2806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 -- 172.21.15.52:0/3496867808 >> 172.21.15.52:0/3496867808 conn(0x7fba28078340 msgr2=0x7fba28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 -- 172.21.15.52:0/3496867808 shutdown_connections 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.920+0000 7fba2cd8c700 1 -- 172.21.15.52:0/3496867808 wait complete. 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 Processor -- start 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 -- start start 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba2806abc0 0x7fba28101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 0x7fba281019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba28108500 0x7fba28101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fba28112690 con 0x7fba2806b580 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba28112510 con 0x7fba2806abc0 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2cd8c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fba28112810 con 0x7fba28108500 2024-06-02T23:20:12.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 0x7fba281019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 0x7fba281019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44692/0 (socket says 172.21.15.52:44692) 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 -- 172.21.15.52:0/1801776393 learned_addr learned my addr 172.21.15.52:0/1801776393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba2659c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba2806abc0 0x7fba28101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba26d9d700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba28108500 0x7fba28101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 -- 172.21.15.52:0/1801776393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba28108500 msgr2=0x7fba28101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba28108500 0x7fba28101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 -- 172.21.15.52:0/1801776393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba2806abc0 msgr2=0x7fba28101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba2806abc0 0x7fba28101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 -- 172.21.15.52:0/1801776393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba281a7130 con 0x7fba2806b580 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.921+0000 7fba25d9b700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 0x7fba281019d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00b530 tx=0x7fba1c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.922+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c015020 con 0x7fba2806b580 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.922+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba281a7420 con 0x7fba2806b580 2024-06-02T23:20:12.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.922+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fba1c011ce0 con 0x7fba2806b580 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.922+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba281a79b0 con 0x7fba2806b580 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.922+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c010e40 con 0x7fba2806b580 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.924+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba2810a9e0 con 0x7fba2806b580 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.924+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fba1c018020 con 0x7fba2806b580 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.925+0000 7fba177fe700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba100a2c20 0x7fba100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.925+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fba1c0ce870 con 0x7fba2806b580 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.925+0000 7fba2659c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba100a2c20 0x7fba100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.926+0000 7fba2659c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba100a2c20 0x7fba100a50e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fba180098e0 tx=0x7fba18005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:12.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:11.930+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fba1c01f020 con 0x7fba2806b580 2024-06-02T23:20:12.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.184+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba281029c0 con 0x7fba2806b580 2024-06-02T23:20:12.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.185+0000 7fba177fe700 1 -- 172.21.15.52:0/1801776393 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fba1c010380 con 0x7fba2806b580 2024-06-02T23:20:12.185 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:20:12.185 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba100a2c20 msgr2=0x7fba100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba100a2c20 0x7fba100a50e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fba180098e0 tx=0x7fba18005960 comp rx=0 tx=0).stop 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 msgr2=0x7fba281019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 0x7fba281019d0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fba1c00b530 tx=0x7fba1c00b840 comp rx=0 tx=0).stop 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 shutdown_connections 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fba100a2c20 0x7fba100a50e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fba28108500 0x7fba28101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba2806abc0 0x7fba28101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 --2- 172.21.15.52:0/1801776393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fba2806b580 0x7fba281019d0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 >> 172.21.15.52:0/1801776393 conn(0x7fba28078340 msgr2=0x7fba28100100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 shutdown_connections 2024-06-02T23:20:12.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:12.187+0000 7fba2cd8c700 1 -- 172.21.15.52:0/1801776393 wait complete. 2024-06-02T23:20:12.189 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:20:12.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.81 MB/sec execute 321 sec latency 25751.704 ms 2024-06-02T23:20:12.710 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1801776393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:12.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1801776393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1801776393' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:13.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.81 MB/sec execute 322 sec latency 26751.769 ms 2024-06-02T23:20:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:13 smithi052 ceph-mon[32340]: pgmap v681: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-06-02T23:20:13.719 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 534f2182 --release --await' 2024-06-02T23:20:13.720 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 534f2182 --release --await 2024-06-02T23:20:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:13 smithi123 ceph-mon[35977]: pgmap v681: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-06-02T23:20:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:13 smithi174 ceph-mon[35994]: pgmap v681: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 4.7 KiB/s wr, 14 op/s 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/1088704111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14102510 msgr2=0x7f2b14102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/1088704111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14102510 0x7f2b14102990 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009990 tx=0x7f2b04009ca0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/1088704111 shutdown_connections 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/1088704111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 0x7f2b1410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/1088704111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b14108510 0x7f2b141088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/1088704111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14102510 0x7f2b14102990 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/1088704111 >> 172.21.15.52:0/1088704111 conn(0x7f2b140795b0 msgr2=0x7f2b140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/1088704111 shutdown_connections 2024-06-02T23:20:14.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.233+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/1088704111 wait complete. 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 Processor -- start 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 -- start start 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b14102510 0x7f2b1419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 0x7f2b1419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 0x7f2b141989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141126e0 con 0x7f2b14108510 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b14112560 con 0x7f2b14102510 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b1b6c3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b14112860 con 0x7f2b14102ed0 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 0x7f2b141989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 0x7f2b141989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43536/0 (socket says 172.21.15.52:43536) 2024-06-02T23:20:14.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.234+0000 7f2b19c60700 1 -- 172.21.15.52:0/2219596774 learned_addr learned my addr 172.21.15.52:0/2219596774 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 0x7f2b1419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b1945f700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b14102510 0x7f2b1419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b19c60700 1 -- 172.21.15.52:0/2219596774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 msgr2=0x7f2b1419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b19c60700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 0x7f2b1419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b19c60700 1 -- 172.21.15.52:0/2219596774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b14102510 msgr2=0x7f2b1419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b19c60700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b14102510 0x7f2b1419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b19c60700 1 -- 172.21.15.52:0/2219596774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b14199140 con 0x7f2b14108510 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 0x7f2b1419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b19c60700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 0x7f2b141989e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00b970 tx=0x7f2b0c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b0a7fc700 1 -- 172.21.15.52:0/2219596774 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c019070 con 0x7f2b14108510 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b14199430 con 0x7f2b14108510 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b0a7fc700 1 -- 172.21.15.52:0/2219596774 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c00cde0 con 0x7f2b14108510 2024-06-02T23:20:14.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.235+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b141a56f0 con 0x7f2b14108510 2024-06-02T23:20:14.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.236+0000 7f2b0a7fc700 1 -- 172.21.15.52:0/2219596774 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c007370 con 0x7f2b14108510 2024-06-02T23:20:14.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.238+0000 7f2b0a7fc700 1 -- 172.21.15.52:0/2219596774 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b0c01c070 con 0x7f2b14108510 2024-06-02T23:20:14.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.239+0000 7f2b0a7fc700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2c20 0x7f2b000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.239+0000 7f2b0a7fc700 1 -- 172.21.15.52:0/2219596774 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b0c0ce520 con 0x7f2b14108510 2024-06-02T23:20:14.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.239+0000 7f2b1945f700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2c20 0x7f2b000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 Processor -- start 2024-06-02T23:20:14.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 -- start start 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b1945f700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2c20 0x7f2b000a50e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b14199b20 tx=0x7f2b1000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8440 0x555c5adc8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8d60 0x555c5add5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5add5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555c5acbaf60 con 0x555c5add59a0 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c5ad19850 con 0x555c5adc8440 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c5ac81460 con 0x555c5adc8d60 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5add5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5add5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43538/0 (socket says 172.21.15.52:43538) 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b19c60700 1 -- 172.21.15.52:0/162892300 learned_addr learned my addr 172.21.15.52:0/162892300 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.240+0000 7f2b1945f700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8440 0x555c5adc8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b19c60700 1 -- 172.21.15.52:0/162892300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8d60 msgr2=0x555c5add5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8d60 0x555c5add5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b19c60700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8d60 0x555c5add5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b19c60700 1 -- 172.21.15.52:0/162892300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8440 msgr2=0x555c5adc8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b19c60700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8440 0x555c5adc8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b19c60700 1 -- 172.21.15.52:0/162892300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c5ac688d0 con 0x555c5add59a0 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b1945f700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8440 0x555c5adc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8d60 0x555c5add5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2b19c60700 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5add5e00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c097210 tx=0x7f2b0c004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f56265544bdf3c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2aff7fe700 1 -- 172.21.15.52:0/162892300 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c09f160 con 0x555c5add59a0 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2aff7fe700 1 -- 172.21.15.52:0/162892300 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c09e030 con 0x555c5add59a0 2024-06-02T23:20:14.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.241+0000 7f2aff7fe700 1 -- 172.21.15.52:0/162892300 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0a37f0 con 0x555c5add59a0 2024-06-02T23:20:14.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 -- 172.21.15.52:0/162892300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 msgr2=0x555c5add5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5add5e00 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c097210 tx=0x7f2b0c004be0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 -- 172.21.15.52:0/162892300 shutdown_connections 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5add5e00 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8440 0x555c5adc8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/162892300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8d60 0x555c5add5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 -- 172.21.15.52:0/162892300 >> 172.21.15.52:0/162892300 conn(0x555c5acf5080 msgr2=0x555c5adc3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 -- 172.21.15.52:0/162892300 shutdown_connections 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 -- 172.21.15.52:0/162892300 wait complete. 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.242+0000 7f2b25f46b80 1 -- start start 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 Processor -- start 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5addcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5adda2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5adda7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555c5acb9fc0 con 0x555c5add59a0 2024-06-02T23:20:14.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555c5ad15810 con 0x555c5adc8d60 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555c5addda90 con 0x555c5adc8440 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b25f46b80 10 client.0 _ll_register_callbacks cb 0x555c5acaf5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5adda7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5adda7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43554/0 (socket says 172.21.15.52:43554) 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b19c60700 1 -- 172.21.15.52:0/624827723 learned_addr learned my addr 172.21.15.52:0/624827723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5adda2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.243+0000 7f2b1945f700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5addcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b19c60700 1 -- 172.21.15.52:0/624827723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 msgr2=0x555c5addcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b19c60700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5addcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b19c60700 1 -- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 msgr2=0x555c5adda2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b19c60700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5adda2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b19c60700 1 -- 172.21.15.52:0/624827723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c5acdf750 con 0x555c5add59a0 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b19c60700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5adda7f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00bfd0 tx=0x7f2b0c009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c949ffcb0d5c9a3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:14.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2afcff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2afcff9700 1 -- 172.21.15.52:0/624827723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0b9030 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b0c09e030 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2afcff9700 1 -- 172.21.15.52:0/624827723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b0c0146f0 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2afcff9700 1 -- 172.21.15.52:0/624827723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c0ce0b0 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b25f46b80 10 client.17832 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.244+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c5ade8230 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2afcff9700 1 -- 172.21.15.52:0/624827723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c09e030 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2afcff9700 1 -- 172.21.15.52:0/624827723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b0c096b00 con 0x555c5add59a0 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b25f46b80 10 client.17832 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b25f46b80 10 client.17832 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b25f46b80 20 client.17832 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5adea700 0x555c5adecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b25f46b80 4 client.17832 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:14.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c5ad26d50 con 0x555c5adea700 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.245+0000 7f2b1945f700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5adea700 0x555c5adecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.246+0000 7f2b1945f700 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5adea700 0x555c5adecaf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9efed9695eee43a1 server_cookie=860b8437a3fef395 in_seq=0 out_seq=0 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.246+0000 7f2afcff9700 10 client.17832 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.246+0000 7f2afcff9700 1 -- 172.21.15.52:0/624827723 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555c5ad26d50 con 0x555c5adea700 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.246+0000 7f2afcff9700 10 client.17832 handle_command_reply: tid=0 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 client.17832 shutdown 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 msgr2=0x555c5adda7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5adda7f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00bfd0 tx=0x7f2b0c009970 comp rx=0 tx=0).stop 2024-06-02T23:20:14.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 shutdown_connections 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5add59a0 0x555c5adda7f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5addcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5adea700 0x555c5adecaf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/624827723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5adda2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2afcff9700 0 client.17832 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 >> 172.21.15.52:0/624827723 conn(0x555c5acf5080 msgr2=0x555c5add6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 shutdown_connections 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 1 -- 172.21.15.52:0/624827723 wait complete. 2024-06-02T23:20:14.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.247+0000 7f2b25f46b80 20 client.17832 trim_cache size 0 max 16384 2024-06-02T23:20:14.256 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 Processor -- start 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 -- start start 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8440 0x555c5add9210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5addcaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5add59a0 0x555c5adefb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141aa8d0 con 0x555c5adc8440 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141ac410 con 0x555c5adc8d60 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.263+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141aafa0 con 0x555c5add59a0 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b1945f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8440 0x555c5add9210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5addcaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5add59a0 0x555c5adefb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b1945f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8440 0x555c5add9210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43568/0 (socket says 172.21.15.52:43568) 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5addcaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46062/0 (socket says 172.21.15.52:46062) 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b1945f700 1 -- 172.21.15.52:0/329329673 learned_addr learned my addr 172.21.15.52:0/329329673 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5add59a0 0x555c5adefb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36782/0 (socket says 172.21.15.52:36782) 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 -- 172.21.15.52:0/329329673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5add59a0 msgr2=0x555c5adefb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5add59a0 0x555c5adefb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 -- 172.21.15.52:0/329329673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8440 msgr2=0x555c5add9210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8440 0x555c5add9210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 -- 172.21.15.52:0/329329673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c5add8820 con 0x555c5adc8d60 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.264+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5addcaa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b14079db0 tx=0x7f2b04005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a17d404346198f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2afeffd700 1 -- 172.21.15.52:0/329329673 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04028070 con 0x555c5adc8d60 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2afeffd700 1 -- 172.21.15.52:0/329329673 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b040050b0 con 0x555c5adc8d60 2024-06-02T23:20:14.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2afeffd700 1 -- 172.21.15.52:0/329329673 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0400ba10 con 0x555c5adc8d60 2024-06-02T23:20:14.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 -- 172.21.15.52:0/329329673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 msgr2=0x555c5addcaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5addcaa0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2b14079db0 tx=0x7f2b04005040 comp rx=0 tx=0).stop 2024-06-02T23:20:14.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 -- 172.21.15.52:0/329329673 shutdown_connections 2024-06-02T23:20:14.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5add59a0 0x555c5adefb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8440 0x555c5add9210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/329329673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5adc8d60 0x555c5addcaa0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.265+0000 7f2b25f46b80 1 -- 172.21.15.52:0/329329673 >> 172.21.15.52:0/329329673 conn(0x555c5acf5080 msgr2=0x555c5adc7ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.266+0000 7f2b25f46b80 1 -- 172.21.15.52:0/329329673 shutdown_connections 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.266+0000 7f2b25f46b80 1 -- 172.21.15.52:0/329329673 wait complete. 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.266+0000 7f2b25f46b80 1 -- start start 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.266+0000 7f2b25f46b80 1 Processor -- start 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5adf0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 0x555c5aded5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5add59a0 0x555c5adedae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141abd40 con 0x555c5adc8d60 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141ab670 con 0x555c5add59a0 2024-06-02T23:20:14.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b141a8890 con 0x555c5adc8440 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b25f46b80 10 client.0 _ll_register_callbacks cb 0x555c5acaf5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 0x555c5aded5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b19c60700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5add59a0 0x555c5adedae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 0x555c5aded5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43582/0 (socket says 172.21.15.52:43582) 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 -- 172.21.15.52:0/1831117973 learned_addr learned my addr 172.21.15.52:0/1831117973 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b1945f700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5adf0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 -- 172.21.15.52:0/1831117973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 msgr2=0x555c5adf0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5adf0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 -- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5add59a0 msgr2=0x555c5adedae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5add59a0 0x555c5adedae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b18c5e700 1 -- 172.21.15.52:0/1831117973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c5adee2d0 con 0x555c5adc8d60 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.267+0000 7f2b19c60700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5add59a0 0x555c5adedae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:14.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b18c5e700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 0x555c5aded5a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009ce0 tx=0x7f2b040056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9cb96aee301a60 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2afcff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2afcff9700 1 -- 172.21.15.52:0/1831117973 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04028040 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2afcff9700 1 -- 172.21.15.52:0/1831117973 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b04003c60 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b040050b0 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2afcff9700 1 -- 172.21.15.52:0/1831117973 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b04003e60 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b25f46b80 10 client.17838 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555c5ae55010 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2afcff9700 1 -- 172.21.15.52:0/1831117973 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0400bc80 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2afcff9700 1 -- 172.21.15.52:0/1831117973 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b0402da80 con 0x555c5adc8d60 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b25f46b80 10 client.17838 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b25f46b80 10 client.17838 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.268+0000 7f2b25f46b80 20 client.17838 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:14.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5ae55740 0x555c5ae57b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2b25f46b80 4 client.17838 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f2b040050b0 con 0x555c5ae55740 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2b1945f700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5ae55740 0x555c5ae57b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2b1945f700 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5ae55740 0x555c5ae57b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=639ef269a1ffbd6d server_cookie=6810b6a6a24db29e in_seq=0 out_seq=0 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2afcff9700 10 client.17838 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2afcff9700 1 -- 172.21.15.52:0/1831117973 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f2b040050b0 con 0x555c5ae55740 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2afcff9700 10 client.17838 handle_command_reply: tid=0 2024-06-02T23:20:14.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.269+0000 7f2b25f46b80 1 client.17838 shutdown 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 msgr2=0x555c5aded5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 0x555c5aded5a0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f2b04009ce0 tx=0x7f2b040056b0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 shutdown_connections 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555c5ae55740 0x555c5ae57b60 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555c5add59a0 0x555c5adedae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555c5adc8440 0x555c5adf0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 --2- 172.21.15.52:0/1831117973 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555c5adc8d60 0x555c5aded5a0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 >> 172.21.15.52:0/1831117973 conn(0x555c5acf5080 msgr2=0x555c5add48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:14.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2afcff9700 0 client.17838 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:20:14.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 shutdown_connections 2024-06-02T23:20:14.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 1 -- 172.21.15.52:0/1831117973 wait complete. 2024-06-02T23:20:14.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.270+0000 7f2b25f46b80 20 client.17838 trim_cache size 0 max 16384 2024-06-02T23:20:14.275 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:20:14.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2c20 msgr2=0x7f2b000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2c20 0x7f2b000a50e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2b14199b20 tx=0x7f2b1000b410 comp rx=0 tx=0).stop 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 msgr2=0x7f2b141989e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 0x7f2b141989e0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00b970 tx=0x7f2b0c00bc80 comp rx=0 tx=0).stop 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 shutdown_connections 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2c20 0x7f2b000a50e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b14108510 0x7f2b141989e0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b14102510 0x7f2b1419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 --2- 172.21.15.52:0/2219596774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b14102ed0 0x7f2b1419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 >> 172.21.15.52:0/2219596774 conn(0x7f2b140795b0 msgr2=0x7f2b14101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 shutdown_connections 2024-06-02T23:20:14.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:14.271+0000 7f2b1b6c3700 1 -- 172.21.15.52:0/2219596774 wait complete. 2024-06-02T23:20:14.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.80 MB/sec execute 323 sec latency 27751.802 ms 2024-06-02T23:20:15.273 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:20:15.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.79 MB/sec execute 324 sec latency 28751.849 ms 2024-06-02T23:20:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:15 smithi052 ceph-mon[32340]: pgmap v682: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-06-02T23:20:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:15 smithi123 ceph-mon[35977]: pgmap v682: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-06-02T23:20:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:15 smithi174 ceph-mon[35994]: pgmap v682: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 4.5 KiB/s wr, 13 op/s 2024-06-02T23:20:16.184 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:16.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.79 MB/sec execute 325 sec latency 29751.925 ms 2024-06-02T23:20:17.595 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:17 smithi052 ceph-mon[32340]: pgmap v683: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-06-02T23:20:17.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155873 1.78 MB/sec execute 326 sec latency 30751.969 ms 2024-06-02T23:20:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:17 smithi123 ceph-mon[35977]: pgmap v683: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-06-02T23:20:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:17 smithi174 ceph-mon[35994]: pgmap v683: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.4 KiB/s wr, 14 op/s 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 -- 172.21.15.52:0/138300661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 msgr2=0x7f2b181029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 --2- 172.21.15.52:0/138300661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b181029b0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f2b080099c0 tx=0x7f2b08009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 -- 172.21.15.52:0/138300661 shutdown_connections 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 --2- 172.21.15.52:0/138300661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b18102ef0 0x7f2b1810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 --2- 172.21.15.52:0/138300661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b181085d0 0x7f2b181089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 --2- 172.21.15.52:0/138300661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b181029b0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 -- 172.21.15.52:0/138300661 >> 172.21.15.52:0/138300661 conn(0x7f2b18079640 msgr2=0x7f2b18079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 -- 172.21.15.52:0/138300661 shutdown_connections 2024-06-02T23:20:18.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.175+0000 7f2b1d868700 1 -- 172.21.15.52:0/138300661 wait complete. 2024-06-02T23:20:18.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 Processor -- start 2024-06-02T23:20:18.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 -- start start 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b1819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b18102ef0 0x7f2b1819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b181085d0 0x7f2b18198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181126c0 con 0x7f2b18102530 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18112540 con 0x7f2b18102ef0 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b1d868700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b18112840 con 0x7f2b181085d0 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b1819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b1819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43598/0 (socket says 172.21.15.52:43598) 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1630124236 learned_addr learned my addr 172.21.15.52:0/1630124236 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.176+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b18102ef0 0x7f2b1819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b177fe700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b181085d0 0x7f2b18198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1630124236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b181085d0 msgr2=0x7f2b18198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b181085d0 0x7f2b18198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1630124236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b18102ef0 msgr2=0x7f2b1819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b18102ef0 0x7f2b1819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1630124236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b18199080 con 0x7f2b18102530 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b177fe700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b181085d0 0x7f2b18198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b1819e760 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00d950 tx=0x7f2b0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b1c866700 1 -- 172.21.15.52:0/1630124236 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c016070 con 0x7f2b18102530 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b1c866700 1 -- 172.21.15.52:0/1630124236 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c009df0 con 0x7f2b18102530 2024-06-02T23:20:18.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b18199370 con 0x7f2b18102530 2024-06-02T23:20:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b1c866700 1 -- 172.21.15.52:0/1630124236 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c00f3c0 con 0x7f2b18102530 2024-06-02T23:20:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.177+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b181a5630 con 0x7f2b18102530 2024-06-02T23:20:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.180+0000 7f2b1c866700 1 -- 172.21.15.52:0/1630124236 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b0c019070 con 0x7f2b18102530 2024-06-02T23:20:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.180+0000 7f2b1c866700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2ca0 0x7f2b000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.180+0000 7f2b1c866700 1 -- 172.21.15.52:0/1630124236 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b0c0cebe0 con 0x7f2b18102530 2024-06-02T23:20:18.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.180+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2ca0 0x7f2b000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2ca0 0x7f2b000a5160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b18079c40 tx=0x7f2b08005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 Processor -- start 2024-06-02T23:20:18.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 -- start start 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f0b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0e5d810 con 0x55c0d0f11f60 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0e05720 con 0x55c0d0f0bf20 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0e597d0 con 0x55c0d0f11640 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.181+0000 7f2b167fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f0b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f0b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43614/0 (socket says 172.21.15.52:43614) 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 -- 172.21.15.52:0/1664065241 learned_addr learned my addr 172.21.15.52:0/1664065241 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b177fe700 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 -- 172.21.15.52:0/1664065241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 msgr2=0x55c0d0f11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f11a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 -- 172.21.15.52:0/1664065241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 msgr2=0x55c0d0f0c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 -- 172.21.15.52:0/1664065241 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d0dfbf50 con 0x55c0d0f11f60 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b177fe700 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f0b9e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2b08015000 tx=0x7f2b0801a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6de66616ab0e993 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2afdffb700 1 -- 172.21.15.52:0/1664065241 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08003e50 con 0x55c0d0f11f60 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2afdffb700 1 -- 172.21.15.52:0/1664065241 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b08005690 con 0x55c0d0f11f60 2024-06-02T23:20:18.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.182+0000 7f2afdffb700 1 -- 172.21.15.52:0/1664065241 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b08023b00 con 0x55c0d0f11f60 2024-06-02T23:20:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1664065241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 msgr2=0x55c0d0f0b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f0b9e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2b08015000 tx=0x7f2b0801a070 comp rx=0 tx=0).stop 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1664065241 shutdown_connections 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1664065241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f0b9e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1664065241 >> 172.21.15.52:0/1664065241 conn(0x55c0d0e36cc0 msgr2=0x55c0d0f06850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1664065241 shutdown_connections 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1664065241 wait complete. 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.183+0000 7f2b280ebb80 1 -- start start 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 Processor -- start 2024-06-02T23:20:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f17cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f14fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f15510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0f18910 con 0x55c0d0f11f60 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0dfdbf0 con 0x55c0d0f0bf20 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0d0f18a90 con 0x55c0d0f11640 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b280ebb80 10 client.0 _ll_register_callbacks cb 0x55c0d0df7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f14fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b177fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f14fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36838/0 (socket says 172.21.15.52:36838) 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b177fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f15510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43628/0 (socket says 172.21.15.52:43628) 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 -- 172.21.15.52:0/3136928112 learned_addr learned my addr 172.21.15.52:0/3136928112 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f17cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 -- 172.21.15.52:0/3136928112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 msgr2=0x55c0d0f17cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f17cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 -- 172.21.15.52:0/3136928112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 msgr2=0x55c0d0f15510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f15510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b167fc700 1 -- 172.21.15.52:0/3136928112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d0dac8d0 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.184+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2b167fc700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f14fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b08030040 tx=0x7f2b08005aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c667125818540e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2af6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3136928112 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800b990 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3136928112 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b0800bb30 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b08005690 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3136928112 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b0800bd50 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2b280ebb80 10 client.27142 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0d0f18910 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3136928112 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b08005690 con 0x55c0d0f11640 2024-06-02T23:20:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.185+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3136928112 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b08042530 con 0x55c0d0f11640 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b280ebb80 10 client.27142 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b280ebb80 10 client.27142 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b280ebb80 20 client.27142 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f25510 0x55c0d0f27900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b280ebb80 4 client.27142 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0d0e6ad10 con 0x55c0d0f25510 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f25510 0x55c0d0f27900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2af6ffd700 10 client.27142 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.186+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f25510 0x55c0d0f27900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=69ee1502f39450c5 server_cookie=321fe247ce02f787 in_seq=0 out_seq=0 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3136928112 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0d0e6ad10 con 0x55c0d0f25510 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2af6ffd700 10 client.27142 handle_command_reply: tid=0 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 client.27142 shutdown 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 msgr2=0x55c0d0f14fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f14fd0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2b08030040 tx=0x7f2b08005aa0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 shutdown_connections 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f11f60 0x55c0d0f15510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f17cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f25510 0x55c0d0f27900 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3136928112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f14fd0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2af6ffd700 0 client.27142 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 >> 172.21.15.52:0/3136928112 conn(0x55c0d0e36cc0 msgr2=0x55c0d0f0a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 shutdown_connections 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3136928112 wait complete. 2024-06-02T23:20:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.187+0000 7f2b280ebb80 20 client.27142 trim_cache size 0 max 16384 2024-06-02T23:20:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.202+0000 7f2b280ebb80 1 Processor -- start 2024-06-02T23:20:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 -- start start 2024-06-02T23:20:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f12b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f2cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181aca90 con 0x55c0d0f0bf20 2024-06-02T23:20:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181ae5d0 con 0x55c0d0f11f60 2024-06-02T23:20:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181ad160 con 0x55c0d0f11640 2024-06-02T23:20:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43636/0 (socket says 172.21.15.52:43636) 2024-06-02T23:20:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1853577720 learned_addr learned my addr 172.21.15.52:0/1853577720 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f12b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1853577720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 msgr2=0x55c0d0f12b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f12b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1853577720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 msgr2=0x55c0d0f2cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f2cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 -- 172.21.15.52:0/1853577720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0d0f15ed0 con 0x55c0d0f0bf20 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b167fc700 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.203+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0b9e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c09b110 tx=0x7f2b0c097040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef39dc16425ca7e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2afd7fa700 1 -- 172.21.15.52:0/1853577720 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c020040 con 0x55c0d0f0bf20 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2afd7fa700 1 -- 172.21.15.52:0/1853577720 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c0a4080 con 0x55c0d0f0bf20 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2afd7fa700 1 -- 172.21.15.52:0/1853577720 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c09b8d0 con 0x55c0d0f0bf20 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1853577720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 msgr2=0x55c0d0f0b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0b9e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c09b110 tx=0x7f2b0c097040 comp rx=0 tx=0).stop 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1853577720 shutdown_connections 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f2cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f0b9e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/1853577720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f12b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1853577720 >> 172.21.15.52:0/1853577720 conn(0x55c0d0e36cc0 msgr2=0x55c0d0f12400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1853577720 shutdown_connections 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.204+0000 7f2b280ebb80 1 -- 172.21.15.52:0/1853577720 wait complete. 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 -- start start 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 Processor -- start 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f2c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f2ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f1cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181adf00 con 0x55c0d0f0bf20 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181ad830 con 0x55c0d0f11f60 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b181aaa30 con 0x55c0d0f11640 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b280ebb80 10 client.0 _ll_register_callbacks cb 0x55c0d0df7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f2c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b16ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f2c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43652/0 (socket says 172.21.15.52:43652) 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.205+0000 7f2b16ffd700 1 -- 172.21.15.52:0/3866551088 learned_addr learned my addr 172.21.15.52:0/3866551088 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b167fc700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f2ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b177fe700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f1cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b16ffd700 1 -- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 msgr2=0x55c0d0f2ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f2ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b16ffd700 1 -- 172.21.15.52:0/3866551088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 msgr2=0x55c0d0f1cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f1cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b16ffd700 1 -- 172.21.15.52:0/3866551088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0d0f1d280 con 0x55c0d0f0bf20 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b177fe700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f1cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b16ffd700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f2c770 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c09bcc0 tx=0x7f2b0c099770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a9d83cb4bfc28d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2af6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3866551088 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b0c0b9060 con 0x55c0d0f0bf20 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3866551088 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b0c0b4210 con 0x55c0d0f0bf20 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b0c0a4080 con 0x55c0d0f0bf20 2024-06-02T23:20:18.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3866551088 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b0c0b4070 con 0x55c0d0f0bf20 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b280ebb80 10 client.17856 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.206+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c0d0f1e4f0 con 0x55c0d0f0bf20 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3866551088 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b0c0a4080 con 0x55c0d0f0bf20 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3866551088 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b0c0ccb70 con 0x55c0d0f0bf20 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b280ebb80 10 client.17856 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b280ebb80 10 client.17856 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b280ebb80 20 client.17856 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f98b30 0x55c0d0f9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b280ebb80 4 client.17856 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0d0e65290 con 0x55c0d0f98b30 2024-06-02T23:20:18.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b167fc700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f98b30 0x55c0d0f9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2b167fc700 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f98b30 0x55c0d0f9af20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ae193f8b3ed33ef3 server_cookie=f11eb2d5e866fd53 in_seq=0 out_seq=0 2024-06-02T23:20:18.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.207+0000 7f2af6ffd700 10 client.17856 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:20:18.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2af6ffd700 1 -- 172.21.15.52:0/3866551088 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0d0e65290 con 0x55c0d0f98b30 2024-06-02T23:20:18.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2af6ffd700 10 client.17856 handle_command_reply: tid=0 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 client.17856 shutdown 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 msgr2=0x55c0d0f2c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f2c770 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c09bcc0 tx=0x7f2b0c099770 comp rx=0 tx=0).stop 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 shutdown_connections 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0d0f11f60 0x55c0d0f1cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0d0f0bf20 0x55c0d0f2c770 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55c0d0f98b30 0x55c0d0f9af20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 --2- 172.21.15.52:0/3866551088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0d0f11640 0x55c0d0f2ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2af6ffd700 0 client.17856 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 >> 172.21.15.52:0/3866551088 conn(0x55c0d0e36cc0 msgr2=0x55c0d0effe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 shutdown_connections 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 1 -- 172.21.15.52:0/3866551088 wait complete. 2024-06-02T23:20:18.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.208+0000 7f2b280ebb80 20 client.17856 trim_cache size 0 max 16384 2024-06-02T23:20:18.214 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:20:18.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2ca0 msgr2=0x7f2b000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.214 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2ca0 0x7f2b000a5160 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2b18079c40 tx=0x7f2b08005fb0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 msgr2=0x7f2b1819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b1819e760 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c00d950 tx=0x7f2b0c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 shutdown_connections 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b000a2ca0 0x7f2b000a5160 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b181085d0 0x7f2b18198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b18102530 0x7f2b1819e760 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 --2- 172.21.15.52:0/1630124236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b18102ef0 0x7f2b1819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 >> 172.21.15.52:0/1630124236 conn(0x7f2b18079640 msgr2=0x7f2b181011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 shutdown_connections 2024-06-02T23:20:18.215 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.209+0000 7f2b1d868700 1 -- 172.21.15.52:0/1630124236 wait complete. 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 -- 172.21.15.52:0/1852760598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec101d80 msgr2=0x7fb1ec10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/1852760598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec101d80 0x7fb1ec10e240 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b0a0 tx=0x7fb1e400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 -- 172.21.15.52:0/1852760598 shutdown_connections 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/1852760598 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec101d80 0x7fb1ec10e240 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/1852760598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec107440 0x7fb1ec107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/1852760598 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.400+0000 7fb1f29ee700 1 -- 172.21.15.52:0/1852760598 >> 172.21.15.52:0/1852760598 conn(0x7fb1ec077400 msgr2=0x7fb1ec077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 -- 172.21.15.52:0/1852760598 shutdown_connections 2024-06-02T23:20:18.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 -- 172.21.15.52:0/1852760598 wait complete. 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 Processor -- start 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 -- start start 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec19e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec101d80 0x7fb1ec19ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ec107440 0x7fb1ec1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec111580 con 0x7fb1ec1013c0 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec111400 con 0x7fb1ec101d80 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1f29ee700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec111700 con 0x7fb1ec107440 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.401+0000 7fb1ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec19e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec19e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43666/0 (socket says 172.21.15.52:43666) 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 -- 172.21.15.52:0/2246266210 learned_addr learned my addr 172.21.15.52:0/2246266210 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec101d80 0x7fb1ec19ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ec107440 0x7fb1ec1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 -- 172.21.15.52:0/2246266210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ec107440 msgr2=0x7fb1ec1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ec107440 0x7fb1ec1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 -- 172.21.15.52:0/2246266210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec101d80 msgr2=0x7fb1ec19ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec101d80 0x7fb1ec19ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 -- 172.21.15.52:0/2246266210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1ec199030 con 0x7fb1ec1013c0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec19e710 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00c9b0 tx=0x7fb1dc00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1e9ffb700 1 -- 172.21.15.52:0/2246266210 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc007b00 con 0x7fb1ec1013c0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1ec199320 con 0x7fb1ec1013c0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1e9ffb700 1 -- 172.21.15.52:0/2246266210 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1dc00ce50 con 0x7fb1ec1013c0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1e9ffb700 1 -- 172.21.15.52:0/2246266210 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc018660 con 0x7fb1ec1013c0 2024-06-02T23:20:18.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.402+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1ec1a55e0 con 0x7fb1ec1013c0 2024-06-02T23:20:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.405+0000 7fb1e9ffb700 1 -- 172.21.15.52:0/2246266210 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb1dc00f450 con 0x7fb1ec1013c0 2024-06-02T23:20:18.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.405+0000 7fb1e9ffb700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb1d80a2ca0 0x7fb1d80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.405+0000 7fb1e9ffb700 1 -- 172.21.15.52:0/2246266210 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb1dc0cee40 con 0x7fb1ec1013c0 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb1d80a2ca0 0x7fb1d80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb1d80a2ca0 0x7fb1d80a5160 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec06bce0 tx=0x7fb1d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 Processor -- start 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 -- start start 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ce6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3ce0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5608d3bcf060 con 0x5608d3ce7230 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608d3c2d950 con 0x5608d3ce1060 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5608d3b95560 con 0x5608d3ce6910 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.406+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3ce0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3ce0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43674/0 (socket says 172.21.15.52:43674) 2024-06-02T23:20:18.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 -- 172.21.15.52:0/1137396149 learned_addr learned my addr 172.21.15.52:0/1137396149 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ce6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 -- 172.21.15.52:0/1137396149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 msgr2=0x5608d3ce6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ce6cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 -- 172.21.15.52:0/1137396149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 msgr2=0x5608d3ce1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 -- 172.21.15.52:0/1137396149 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d3b7c9d0 con 0x5608d3ce7230 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3ce0b20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4011da0 tx=0x7fb1d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ac04d96634c3729 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e27fc700 1 -- 172.21.15.52:0/1137396149 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4004a30 con 0x5608d3ce7230 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e27fc700 1 -- 172.21.15.52:0/1137396149 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1d40054d0 con 0x5608d3ce7230 2024-06-02T23:20:18.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.407+0000 7fb1e27fc700 1 -- 172.21.15.52:0/1137396149 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1d4003c10 con 0x5608d3ce7230 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 -- 172.21.15.52:0/1137396149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 msgr2=0x5608d3ce0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3ce0b20 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4011da0 tx=0x7fb1d4014000 comp rx=0 tx=0).stop 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 -- 172.21.15.52:0/1137396149 shutdown_connections 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ce6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/1137396149 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3ce0b20 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 -- 172.21.15.52:0/1137396149 >> 172.21.15.52:0/1137396149 conn(0x5608d3c09180 msgr2=0x5608d3cd7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 -- 172.21.15.52:0/1137396149 shutdown_connections 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 -- 172.21.15.52:0/1137396149 wait complete. 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.408+0000 7fb1fd271b80 1 -- start start 2024-06-02T23:20:18.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 Processor -- start 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cecf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cea210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cea750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5608d3bce0c0 con 0x5608d3ce7230 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608d3c29910 con 0x5608d3ce6910 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5608d3cedc10 con 0x5608d3ce1060 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1fd271b80 10 client.0 _ll_register_callbacks cb 0x5608d3bc36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cea750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1ebfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cecf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cea750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43688/0 (socket says 172.21.15.52:43688) 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/2351286475 learned_addr learned my addr 172.21.15.52:0/2351286475 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cea210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/2351286475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 msgr2=0x5608d3cecf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cecf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 msgr2=0x5608d3cea210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cea210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/2351286475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d3bf3850 con 0x5608d3ce7230 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.409+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cea750 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4015000 tx=0x7fb1e400bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e103b1afef0474b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/2351286475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e40074e0 con 0x5608d3ce7230 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/2351286475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb1e4007680 con 0x5608d3ce7230 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1d40054d0 con 0x5608d3ce7230 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/2351286475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb1e402e070 con 0x5608d3ce7230 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 10 client.17868 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:18.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608d3cf8350 con 0x5608d3ce7230 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/2351286475 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1d40054d0 con 0x5608d3ce7230 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/2351286475 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb1e4011070 con 0x5608d3ce7230 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 10 client.17868 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 10 client.17868 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 20 client.17868 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3cfcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 4 client.17868 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.410+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608d3c3ae50 con 0x5608d3cfa9e0 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.411+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3cfcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.411+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3cfcdf0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fca0e6b9dc97b509 server_cookie=6ad66ee89cbebe8a in_seq=0 out_seq=0 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.411+0000 7fb1cb7fe700 10 client.17868 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/2351286475 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5608d3c3ae50 con 0x5608d3cfa9e0 2024-06-02T23:20:18.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1cb7fe700 10 client.17868 handle_command_reply: tid=0 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 client.17868 shutdown 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 msgr2=0x5608d3cea750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cea750 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4015000 tx=0x7fb1e400bf60 comp rx=0 tx=0).stop 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 shutdown_connections 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3cfcdf0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cea750 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1cb7fe700 0 client.17868 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cecf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/2351286475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cea210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.412+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 >> 172.21.15.52:0/2351286475 conn(0x5608d3c09180 msgr2=0x5608d3cdb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.413+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 shutdown_connections 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.413+0000 7fb1fd271b80 1 -- 172.21.15.52:0/2351286475 wait complete. 2024-06-02T23:20:18.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.413+0000 7fb1fd271b80 20 client.17868 trim_cache size 0 max 16384 2024-06-02T23:20:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.433+0000 7fb1fd271b80 1 Processor -- start 2024-06-02T23:20:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 -- start start 2024-06-02T23:20:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ceca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3d026c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1aa720 con 0x5608d3ce7230 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1ac260 con 0x5608d3ce1060 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1aadf0 con 0x5608d3ce6910 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3d026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3d026c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43696/0 (socket says 172.21.15.52:43696) 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/8564799 learned_addr learned my addr 172.21.15.52:0/8564799 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ceca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/8564799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 msgr2=0x5608d3ceca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ceca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/8564799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 msgr2=0x5608d3ce7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/8564799 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d3cd8f60 con 0x5608d3ce7230 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:18.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.434+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ceca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.435+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3d026c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b3f0 tx=0x7fb1e4004f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3dc18defe65c16 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.435+0000 7fb1e1ffb700 1 -- 172.21.15.52:0/8564799 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4037020 con 0x5608d3ce7230 2024-06-02T23:20:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.435+0000 7fb1e1ffb700 1 -- 172.21.15.52:0/8564799 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e4027c00 con 0x5608d3ce7230 2024-06-02T23:20:18.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.435+0000 7fb1e1ffb700 1 -- 172.21.15.52:0/8564799 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4007660 con 0x5608d3ce7230 2024-06-02T23:20:18.439 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.435+0000 7fb1fd271b80 1 -- 172.21.15.52:0/8564799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 msgr2=0x5608d3d026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.435+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3d026c0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb1e400b3f0 tx=0x7fb1e4004f30 comp rx=0 tx=0).stop 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- 172.21.15.52:0/8564799 shutdown_connections 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3d026c0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce1060 0x5608d3ce7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/8564799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce6910 0x5608d3ceca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- 172.21.15.52:0/8564799 >> 172.21.15.52:0/8564799 conn(0x5608d3c09180 msgr2=0x5608d3ce76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- 172.21.15.52:0/8564799 shutdown_connections 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- 172.21.15.52:0/8564799 wait complete. 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- start start 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 Processor -- start 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cf2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cf3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cefea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1abb90 con 0x5608d3ce7230 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1ab4c0 con 0x5608d3ce6910 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.436+0000 7fb1fd271b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb1ec1a86e0 con 0x5608d3ce1060 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1fd271b80 10 client.0 _ll_register_callbacks cb 0x5608d3bc36b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cefea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cefea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43708/0 (socket says 172.21.15.52:43708) 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/4192586826 learned_addr learned my addr 172.21.15.52:0/4192586826 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cf2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1e3fff700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cf3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/4192586826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 msgr2=0x5608d3cf2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cf2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 msgr2=0x5608d3cf3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:18.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cf3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 -- 172.21.15.52:0/4192586826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d3cf0490 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cf2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1f0f8b700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cefea0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006070 tx=0x7fb1e4006210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912d9e7bcf4ba44e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/4192586826 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e4005e50 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/4192586826 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb1e400f070 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.437+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1e4027c00 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/4192586826 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb1e4027690 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 10 client.17874 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608d3d65fa0 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/4192586826 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1e4027c00 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/4192586826 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb1e4011050 con 0x5608d3ce7230 2024-06-02T23:20:18.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 10 client.17874 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 10 client.17874 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 20 client.17874 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3d685c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 4 client.17874 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "534f2182", "release": true, "await": true}] 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "534f2182", "release": true, "await": true}) -- 0x5608d3c353d0 con 0x5608d3cfa9e0 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.438+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3d685c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.439+0000 7fb1ebfff700 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3d685c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b401126894bd9189 server_cookie=9d010b36eb0ccf7d in_seq=0 out_seq=0 2024-06-02T23:20:18.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.439+0000 7fb1cb7fe700 10 client.17874 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:18.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1cb7fe700 1 -- 172.21.15.52:0/4192586826 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5608d3c353d0 con 0x5608d3cfa9e0 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1cb7fe700 10 client.17874 handle_command_reply: tid=0 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 client.17874 shutdown 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 msgr2=0x5608d3cefea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cefea0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fb1e4006070 tx=0x7fb1e4006210 comp rx=0 tx=0).stop 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 shutdown_connections 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5608d3cfa9e0 0x5608d3d685c0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d3ce7230 0x5608d3cefea0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d3ce1060 0x5608d3cf2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1cb7fe700 0 client.17874 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 --2- 172.21.15.52:0/4192586826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d3ce6910 0x5608d3cf3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 >> 172.21.15.52:0/4192586826 conn(0x5608d3c09180 msgr2=0x5608d3cdb700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 shutdown_connections 2024-06-02T23:20:18.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 1 -- 172.21.15.52:0/4192586826 wait complete. 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.447+0000 7fb1fd271b80 20 client.17874 trim_cache size 0 max 16384 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 28, 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "534f2182": { 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 28, 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:20:18.449 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 146.3, 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:18.450 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb1d80a2ca0 msgr2=0x7fb1d80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb1d80a2ca0 0x7fb1d80a5160 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec06bce0 tx=0x7fb1d4009260 comp rx=0 tx=0).stop 2024-06-02T23:20:18.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 msgr2=0x7fb1ec19e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec19e710 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fb1dc00c9b0 tx=0x7fb1dc00ccc0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 shutdown_connections 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb1d80a2ca0 0x7fb1d80a5160 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb1ec107440 0x7fb1ec1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb1ec1013c0 0x7fb1ec19e710 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 --2- 172.21.15.52:0/2246266210 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1ec101d80 0x7fb1ec19ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 >> 172.21.15.52:0/2246266210 conn(0x7fb1ec077400 msgr2=0x7fb1ec1056d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 shutdown_connections 2024-06-02T23:20:18.452 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:18.448+0000 7fb1f29ee700 1 -- 172.21.15.52:0/2246266210 wait complete. 2024-06-02T23:20:18.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 155922 1.78 MB/sec execute 327 sec latency 31557.795 ms 2024-06-02T23:20:19.113 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:20:19.394 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:19.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 156393 1.79 MB/sec execute 328 sec latency 13.217 ms 2024-06-02T23:20:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:19 smithi123 ceph-mon[35977]: pgmap v684: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.759+0000 7f191e2a5640 1 -- 172.21.15.52:0/461156574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19181024b0 msgr2=0x7f1918108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 --2- 172.21.15.52:0/461156574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19181024b0 0x7f1918108d40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f1910009ca0 tx=0x7f191002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 -- 172.21.15.52:0/461156574 shutdown_connections 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 --2- 172.21.15.52:0/461156574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1918101af0 0x7f1918101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 --2- 172.21.15.52:0/461156574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19181024b0 0x7f1918108d40 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 --2- 172.21.15.52:0/461156574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 -- 172.21.15.52:0/461156574 >> 172.21.15.52:0/461156574 conn(0x7f19180fbe70 msgr2=0x7f19180fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 -- 172.21.15.52:0/461156574 shutdown_connections 2024-06-02T23:20:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 -- 172.21.15.52:0/461156574 wait complete. 2024-06-02T23:20:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 Processor -- start 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.760+0000 7f191e2a5640 1 -- start start 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191e2a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191e2a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1918101af0 0x7f1918198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191e2a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19181024b0 0x7f191819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191e2a5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f191810d550 con 0x7f1918101af0 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191e2a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f191810d3d0 con 0x7f1918100900 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191e2a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f191810d6d0 con 0x7f19181024b0 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f190ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1918101af0 0x7f1918198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918197b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46180/0 (socket says 172.21.15.52:46180) 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f191c842640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19181024b0 0x7f191819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 -- 172.21.15.52:0/2985840712 learned_addr learned my addr 172.21.15.52:0/2985840712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 -- 172.21.15.52:0/2985840712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19181024b0 msgr2=0x7f191819f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19181024b0 0x7f191819f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 -- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1918101af0 msgr2=0x7f1918198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1918101af0 0x7f1918198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 -- 172.21.15.52:0/2985840712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191819f770 con 0x7f1918100900 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1917fff640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918197b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f190800e9c0 tx=0x7f190800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.761+0000 7f1915ffb640 1 -- 172.21.15.52:0/2985840712 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190800cd80 con 0x7f1918100900 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.762+0000 7f1915ffb640 1 -- 172.21.15.52:0/2985840712 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19080035f0 con 0x7f1918100900 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.762+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191819fa60 con 0x7f1918100900 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.762+0000 7f1915ffb640 1 -- 172.21.15.52:0/2985840712 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908010690 con 0x7f1918100900 2024-06-02T23:20:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.762+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191819ffb0 con 0x7f1918100900 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.764+0000 7f1915ffb640 1 -- 172.21.15.52:0/2985840712 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19080037a0 con 0x7f1918100900 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.764+0000 7f1920fb2740 1 Processor -- start 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 -- start start 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d782430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d787500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d787b70 0x56193d78a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56193d6e54f0 con 0x56193d782010 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56193d683870 con 0x56193d787b70 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1920fb2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56193d6e14b0 con 0x56193d782a00 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d782430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d782430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43742/0 (socket says 172.21.15.52:43742) 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 -- 172.21.15.52:0/1473054712 learned_addr learned my addr 172.21.15.52:0/1473054712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f190ffff640 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d787b70 0x56193d78a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1917fff640 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d787500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1915ffb640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19040a2c30 0x7f19040a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 -- 172.21.15.52:0/1473054712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 msgr2=0x56193d787500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d787500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 -- 172.21.15.52:0/1473054712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d787b70 msgr2=0x56193d78a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d787b70 0x56193d78a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f191c842640 1 -- 172.21.15.52:0/1473054712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193d69f9b0 con 0x56193d782010 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.765+0000 7f1915ffb640 1 -- 172.21.15.52:0/2985840712 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19080cf410 con 0x7f1918100900 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.766+0000 7f191c842640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19040a2c30 0x7f19040a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.766+0000 7f191c842640 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d782430 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1910009ca0 tx=0x7f1910009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0641f5e99011e87 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.766+0000 7f190e7fc640 1 -- 172.21.15.52:0/1473054712 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910031750 con 0x56193d782010 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.766+0000 7f190e7fc640 1 -- 172.21.15.52:0/1473054712 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1910036030 con 0x56193d782010 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.766+0000 7f190e7fc640 1 -- 172.21.15.52:0/1473054712 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19100307e0 con 0x56193d782010 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 -- 172.21.15.52:0/1473054712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 msgr2=0x56193d782430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f191c842640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19040a2c30 0x7f19040a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f191004a000 tx=0x7f191004b000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d782430 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f1910009ca0 tx=0x7f1910009a20 comp rx=0 tx=0).stop 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 -- 172.21.15.52:0/1473054712 shutdown_connections 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d787500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d787b70 0x56193d78a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 --2- 172.21.15.52:0/1473054712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d782430 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 -- 172.21.15.52:0/1473054712 >> 172.21.15.52:0/1473054712 conn(0x56193d77d680 msgr2=0x56193d77da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 -- 172.21.15.52:0/1473054712 shutdown_connections 2024-06-02T23:20:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 -- 172.21.15.52:0/1473054712 wait complete. 2024-06-02T23:20:19.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 -- start start 2024-06-02T23:20:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.767+0000 7f1920fb2740 1 Processor -- start 2024-06-02T23:20:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d78c130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d78c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d793640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56193d78a9a0 con 0x56193d782010 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56193d66eae0 con 0x56193d782a00 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56193d78ab20 con 0x56193d787b70 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 10 client.0 _ll_register_callbacks cb 0x56193d6a79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d78c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d78c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43750/0 (socket says 172.21.15.52:43750) 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 -- 172.21.15.52:0/4257245937 learned_addr learned my addr 172.21.15.52:0/4257245937 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f190ffff640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d78c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f191c842640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d793640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 -- 172.21.15.52:0/4257245937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 msgr2=0x56193d793640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d793640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 -- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 msgr2=0x56193d78c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d78c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 -- 172.21.15.52:0/4257245937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193d3f9210 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f191c842640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d793640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1917fff640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d78c130 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1908000c00 tx=0x7f190809b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caae0538a2fccc38 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f18f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f18f77fe640 1 -- 172.21.15.52:0/4257245937 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908003b30 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f18f77fe640 1 -- 172.21.15.52:0/4257245937 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19080a4420 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.768+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1910036030 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f18f77fe640 1 -- 172.21.15.52:0/4257245937 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19080b9050 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 10 client.17886 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56193d4c5620 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f18f77fe640 1 -- 172.21.15.52:0/4257245937 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1910036030 con 0x56193d782010 2024-06-02T23:20:19.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f18f77fe640 1 -- 172.21.15.52:0/4257245937 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f19080b4070 con 0x56193d782010 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 10 client.17886 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 20 client.17886 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d797ed0 0x56193d79a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 4 client.17886 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56193d6f8f10 con 0x56193d797ed0 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.769+0000 7f190ffff640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d797ed0 0x56193d79a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.770+0000 7f190ffff640 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d797ed0 0x56193d79a320 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe5bbaf27752cbb server_cookie=f6e1667d349da827 in_seq=0 out_seq=0 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.770+0000 7f18f77fe640 10 client.17886 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f18f77fe640 1 -- 172.21.15.52:0/4257245937 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56193d6f8f10 con 0x56193d797ed0 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f18f77fe640 10 client.17886 handle_command_reply: tid=0 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 client.17886 shutdown 2024-06-02T23:20:19.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 msgr2=0x56193d78c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d78c130 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f1908000c00 tx=0x7f190809b780 comp rx=0 tx=0).stop 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 shutdown_connections 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d797ed0 0x56193d79a320 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d793640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d78c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 --2- 172.21.15.52:0/4257245937 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d78c130 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f18f77fe640 0 client.17886 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 >> 172.21.15.52:0/4257245937 conn(0x56193d77d680 msgr2=0x56193d77c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 shutdown_connections 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 1 -- 172.21.15.52:0/4257245937 wait complete. 2024-06-02T23:20:19.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.771+0000 7f1920fb2740 20 client.17886 trim_cache size 0 max 16384 2024-06-02T23:20:19.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 Processor -- start 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 -- start start 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 0x56193d787270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d7935d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d787b70 0x56193d78af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19181a8be0 con 0x56193d787b70 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19181aa720 con 0x56193d782010 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.787+0000 7f1920fb2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19181a92b0 con 0x56193d782a00 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f191c842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d787b70 0x56193d78af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 0x56193d787270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 0x56193d787270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46204/0 (socket says 172.21.15.52:46204) 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f191c842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d787b70 0x56193d78af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43754/0 (socket says 172.21.15.52:43754) 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 -- 172.21.15.52:0/205538959 learned_addr learned my addr 172.21.15.52:0/205538959 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:19.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f190ffff640 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d7935d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 -- 172.21.15.52:0/205538959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 msgr2=0x56193d7935d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d7935d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 -- 172.21.15.52:0/205538959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d787b70 msgr2=0x56193d78af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d787b70 0x56193d78af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 -- 172.21.15.52:0/205538959 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56193d79a9a0 con 0x56193d782010 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f1917fff640 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 0x56193d787270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f190809bbd0 tx=0x7f190809b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e3c62c484007126 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f190dffb640 1 -- 172.21.15.52:0/205538959 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190809b5a0 con 0x56193d782010 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f190dffb640 1 -- 172.21.15.52:0/205538959 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1908003b30 con 0x56193d782010 2024-06-02T23:20:19.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.788+0000 7f190dffb640 1 -- 172.21.15.52:0/205538959 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19080a4680 con 0x56193d782010 2024-06-02T23:20:19.793 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 -- 172.21.15.52:0/205538959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 msgr2=0x56193d787270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 0x56193d787270 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f190809bbd0 tx=0x7f190809b8b0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 -- 172.21.15.52:0/205538959 shutdown_connections 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d782a00 0x56193d7935d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d787b70 0x56193d78af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 --2- 172.21.15.52:0/205538959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782010 0x56193d787270 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 -- 172.21.15.52:0/205538959 >> 172.21.15.52:0/205538959 conn(0x56193d77d680 msgr2=0x56193d77da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 -- 172.21.15.52:0/205538959 shutdown_connections 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 -- 172.21.15.52:0/205538959 wait complete. 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.789+0000 7f1920fb2740 1 -- start start 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 Processor -- start 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d790330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d790870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.794 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d800bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19181aa050 con 0x56193d782010 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19181a9980 con 0x56193d782a00 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19181a6bc0 con 0x56193d787b70 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1920fb2740 10 client.0 _ll_register_callbacks cb 0x7f19080046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d790330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d790330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43764/0 (socket says 172.21.15.52:43764) 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 -- 172.21.15.52:0/3828476958 learned_addr learned my addr 172.21.15.52:0/3828476958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f190ffff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d790870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f191c842640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d800bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 -- 172.21.15.52:0/3828476958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 msgr2=0x56193d800bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d800bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 -- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 msgr2=0x56193d790870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:19.795 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d790870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f1917fff640 1 -- 172.21.15.52:0/3828476958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56193d77d4b0 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.790+0000 7f190ffff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d790870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1917fff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d790330 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1908018000 tx=0x7f19080a46d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3129a36284df0aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f18f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f18f77fe640 1 -- 172.21.15.52:0/3828476958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190807f070 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f18f77fe640 1 -- 172.21.15.52:0/3828476958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19080d2d50 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56193d6f2c20 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f18f77fe640 1 -- 172.21.15.52:0/3828476958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f190809b5a0 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 10 client.17892 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56193d802040 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f18f77fe640 1 -- 172.21.15.52:0/3828476958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56193d6f2c20 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f18f77fe640 1 -- 172.21.15.52:0/3828476958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f19080b4070 con 0x56193d782010 2024-06-02T23:20:19.796 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 10 client.17892 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 10 client.17892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 20 client.17892 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d802810 0x56193d804c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 4 client.17892 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.791+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56193d6ecf70 con 0x56193d802810 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.792+0000 7f190ffff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d802810 0x56193d804c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.792+0000 7f190ffff640 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d802810 0x56193d804c30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9019a358c63630c server_cookie=ed58562f4fa6fdd3 in_seq=0 out_seq=0 2024-06-02T23:20:19.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.792+0000 7f18f77fe640 10 client.17892 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.810+0000 7f18f77fe640 1 -- 172.21.15.52:0/3828476958 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56193d6ecf70 con 0x56193d802810 2024-06-02T23:20:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.810+0000 7f18f77fe640 10 client.17892 handle_command_reply: tid=0 2024-06-02T23:20:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.810+0000 7f1920fb2740 1 client.17892 shutdown 2024-06-02T23:20:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 msgr2=0x56193d790330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d790330 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f1908018000 tx=0x7f19080a46d0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 shutdown_connections 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56193d802810 0x56193d804c30 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56193d787b70 0x56193d800bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56193d782a00 0x56193d790870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 --2- 172.21.15.52:0/3828476958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56193d782010 0x56193d790330 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f18f77fe640 0 client.17892 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 >> 172.21.15.52:0/3828476958 conn(0x56193d77d680 msgr2=0x56193d78aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 shutdown_connections 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 1 -- 172.21.15.52:0/3828476958 wait complete. 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f1920fb2740 20 client.17892 trim_cache size 0 max 16384 2024-06-02T23:20:19.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:20:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:20:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:20:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19040a2c30 msgr2=0x7f19040a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.811+0000 7f191e2a5640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19040a2c30 0x7f19040a50f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f191004a000 tx=0x7f191004b000 comp rx=0 tx=0).stop 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 msgr2=0x7f1918197b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918197b50 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f190800e9c0 tx=0x7f190800ee80 comp rx=0 tx=0).stop 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 shutdown_connections 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19040a2c30 0x7f19040a50f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19181024b0 0x7f191819f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1918101af0 0x7f1918198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 --2- 172.21.15.52:0/2985840712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1918100900 0x7f1918197b50 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 >> 172.21.15.52:0/2985840712 conn(0x7f19180fbe70 msgr2=0x7f1918104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 shutdown_connections 2024-06-02T23:20:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:19.812+0000 7f191e2a5640 1 -- 172.21.15.52:0/2985840712 wait complete. 2024-06-02T23:20:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:19 smithi174 ceph-mon[35994]: pgmap v684: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-06-02T23:20:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:19 smithi052 ceph-mon[32340]: pgmap v684: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 3.5 KiB/s wr, 13 op/s 2024-06-02T23:20:19.969 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '534f2182', total seconds elapsed: 38.6 2024-06-02T23:20:19.970 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.3 2024-06-02T23:20:19.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:20:20.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 156653 1.78 MB/sec execute 329 sec latency 14.949 ms 2024-06-02T23:20:21.140 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:21.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 157079 1.78 MB/sec execute 330 sec latency 15.608 ms 2024-06-02T23:20:21.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:21 smithi052 ceph-mon[32340]: pgmap v685: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-06-02T23:20:21.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:21 smithi123 ceph-mon[35977]: pgmap v685: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-06-02T23:20:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:21 smithi174 ceph-mon[35994]: pgmap v685: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 61 KiB/s wr, 17 op/s 2024-06-02T23:20:22.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.345+0000 7eff2339c700 1 -- 172.21.15.52:0/3588665083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 msgr2=0x7eff1c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.345+0000 7eff2339c700 1 --2- 172.21.15.52:0/3588665083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 0x7eff1c10e240 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7eff1400b0a0 tx=0x7eff1400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 -- 172.21.15.52:0/3588665083 shutdown_connections 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 --2- 172.21.15.52:0/3588665083 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 0x7eff1c10e240 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 --2- 172.21.15.52:0/3588665083 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff1c107400 0x7eff1c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 --2- 172.21.15.52:0/3588665083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 -- 172.21.15.52:0/3588665083 >> 172.21.15.52:0/3588665083 conn(0x7eff1c078340 msgr2=0x7eff1c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 -- 172.21.15.52:0/3588665083 shutdown_connections 2024-06-02T23:20:22.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.346+0000 7eff2339c700 1 -- 172.21.15.52:0/3588665083 wait complete. 2024-06-02T23:20:22.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 Processor -- start 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 -- start start 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c19e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 0x7eff1c19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff1c107400 0x7eff1c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c111520 con 0x7eff1c101d50 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1113a0 con 0x7eff1c101390 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff2339c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1116a0 con 0x7eff1c107400 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff21138700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c19e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff21138700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c19e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46218/0 (socket says 172.21.15.52:46218) 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff21138700 1 -- 172.21.15.52:0/2101574993 learned_addr learned my addr 172.21.15.52:0/2101574993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff21939700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff1c107400 0x7eff1c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff20937700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 0x7eff1c19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.347+0000 7eff21138700 1 -- 172.21.15.52:0/2101574993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff1c107400 msgr2=0x7eff1c196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff21138700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff1c107400 0x7eff1c196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff21138700 1 -- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 msgr2=0x7eff1c19ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff21138700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 0x7eff1c19ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff21138700 1 -- 172.21.15.52:0/2101574993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1c197350 con 0x7eff1c101390 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff21138700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c19e780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c920 tx=0x7eff1800cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff127fc700 1 -- 172.21.15.52:0/2101574993 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18007b00 con 0x7eff1c101390 2024-06-02T23:20:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff127fc700 1 -- 172.21.15.52:0/2101574993 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff18004ce0 con 0x7eff1c101390 2024-06-02T23:20:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1c197640 con 0x7eff1c101390 2024-06-02T23:20:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff127fc700 1 -- 172.21.15.52:0/2101574993 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18005550 con 0x7eff1c101390 2024-06-02T23:20:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.348+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff1c1b4400 con 0x7eff1c101390 2024-06-02T23:20:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.351+0000 7eff127fc700 1 -- 172.21.15.52:0/2101574993 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff18004750 con 0x7eff1c101390 2024-06-02T23:20:22.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.352+0000 7eff127fc700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff080a2c20 0x7eff080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.352+0000 7eff20937700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff080a2c20 0x7eff080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.352+0000 7eff127fc700 1 -- 172.21.15.52:0/2101574993 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7eff180cee90 con 0x7eff1c101390 2024-06-02T23:20:22.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff20937700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff080a2c20 0x7eff080a50e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7eff0c005fd0 tx=0x7eff0c005f60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:22.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 Processor -- start 2024-06-02T23:20:22.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 -- start start 2024-06-02T23:20:22.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9338a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a93329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9332f20 0x5564a9333340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5564a9284810 con 0x5564a9338640 2024-06-02T23:20:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5564a922c720 con 0x5564a9338f60 2024-06-02T23:20:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5564a92807d0 con 0x5564a9332f20 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff21138700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9338a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff21138700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9338a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43794/0 (socket says 172.21.15.52:43794) 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.353+0000 7eff21138700 1 -- 172.21.15.52:0/774983274 learned_addr learned my addr 172.21.15.52:0/774983274 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff20937700 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a93329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21939700 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9332f20 0x5564a9333340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21138700 1 -- 172.21.15.52:0/774983274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9332f20 msgr2=0x5564a9333340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21138700 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9332f20 0x5564a9333340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21138700 1 -- 172.21.15.52:0/774983274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 msgr2=0x5564a93329e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21138700 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a93329e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21138700 1 -- 172.21.15.52:0/774983274 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a9222f50 con 0x5564a9338640 2024-06-02T23:20:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff20937700 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a93329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff21138700 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9338a20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7eff180a3e60 tx=0x7eff1809be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dd1adea0a9bb5f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff077fe700 1 -- 172.21.15.52:0/774983274 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1801d040 con 0x5564a9338640 2024-06-02T23:20:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff077fe700 1 -- 172.21.15.52:0/774983274 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff180a4390 con 0x5564a9338640 2024-06-02T23:20:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.354+0000 7eff077fe700 1 -- 172.21.15.52:0/774983274 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18007ca0 con 0x5564a9338640 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/774983274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 msgr2=0x5564a9338a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9338a20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7eff180a3e60 tx=0x7eff1809be20 comp rx=0 tx=0).stop 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/774983274 shutdown_connections 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9332f20 0x5564a9333340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9338a20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/774983274 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a93329e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/774983274 >> 172.21.15.52:0/774983274 conn(0x5564a925dcc0 msgr2=0x5564a932d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/774983274 shutdown_connections 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.355+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/774983274 wait complete. 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 -- start start 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 Processor -- start 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a933ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338640 0x5564a933bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5564a933f9d0 con 0x5564a9332f20 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5564a9224bf0 con 0x5564a9338640 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5564a933fb50 con 0x5564a9338f60 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff2dc1fb80 10 client.0 _ll_register_callbacks cb 0x5564a921e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.356+0000 7eff21138700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a933ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a933ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43808/0 (socket says 172.21.15.52:43808) 2024-06-02T23:20:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 -- 172.21.15.52:0/4059771657 learned_addr learned my addr 172.21.15.52:0/4059771657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff20937700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338640 0x5564a933bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21939700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 -- 172.21.15.52:0/4059771657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 msgr2=0x5564a933c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 -- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338640 msgr2=0x5564a933bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338640 0x5564a933bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 -- 172.21.15.52:0/4059771657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a91d38d0 con 0x5564a9332f20 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff20937700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338640 0x5564a933bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff21138700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a933ece0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7eff18097b30 tx=0x7eff180973c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b979afb362186fd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff04ff9700 1 -- 172.21.15.52:0/4059771657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1801d070 con 0x5564a9332f20 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff180a4390 con 0x5564a9332f20 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.357+0000 7eff04ff9700 1 -- 172.21.15.52:0/4059771657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7eff18004e90 con 0x5564a9332f20 2024-06-02T23:20:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff04ff9700 1 -- 172.21.15.52:0/4059771657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff180997c0 con 0x5564a9332f20 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 10 client.17904 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564a933f9d0 con 0x5564a9332f20 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff04ff9700 1 -- 172.21.15.52:0/4059771657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff180a4390 con 0x5564a9332f20 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff04ff9700 1 -- 172.21.15.52:0/4059771657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7eff180a3f90 con 0x5564a9332f20 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 10 client.17904 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 10 client.17904 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 20 client.17904 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a934ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 4 client.17904 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.358+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564a9291d10 con 0x5564a934c610 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.359+0000 7eff20937700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a934ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.359+0000 7eff20937700 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a934ea00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c24c1b539bfef86d server_cookie=e5c9a4181b4574c7 in_seq=0 out_seq=0 2024-06-02T23:20:22.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.359+0000 7eff04ff9700 10 client.17904 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff04ff9700 1 -- 172.21.15.52:0/4059771657 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5564a9291d10 con 0x5564a934c610 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff04ff9700 10 client.17904 handle_command_reply: tid=0 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 client.17904 shutdown 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 msgr2=0x5564a933ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a933ece0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7eff18097b30 tx=0x7eff180973c0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 shutdown_connections 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a934ea00 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a933ece0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff04ff9700 0 client.17904 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/4059771657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338640 0x5564a933bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 >> 172.21.15.52:0/4059771657 conn(0x5564a925dcc0 msgr2=0x5564a932d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 shutdown_connections 2024-06-02T23:20:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.360+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/4059771657 wait complete. 2024-06-02T23:20:22.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.361+0000 7eff2dc1fb80 20 client.17904 trim_cache size 0 max 16384 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 Processor -- start 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 -- start start 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9332f20 0x5564a93329e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9327510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1bb930 con 0x5564a9338640 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1bd470 con 0x5564a9332f20 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.378+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1bc000 con 0x5564a9338f60 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21939700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff20937700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9327510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21939700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37048/0 (socket says 172.21.15.52:37048) 2024-06-02T23:20:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9332f20 0x5564a93329e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21939700 1 -- 172.21.15.52:0/716905082 learned_addr learned my addr 172.21.15.52:0/716905082 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 -- 172.21.15.52:0/716905082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 msgr2=0x5564a933d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 -- 172.21.15.52:0/716905082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 msgr2=0x5564a9327510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9327510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 -- 172.21.15.52:0/716905082 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564a933cf20 con 0x5564a9332f20 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff20937700 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9327510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff21138700 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9332f20 0x5564a93329e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff180ba2b0 tx=0x7eff1809b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df9d177e86723d93 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff06ffd700 1 -- 172.21.15.52:0/716905082 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff1801d070 con 0x5564a9332f20 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff06ffd700 1 -- 172.21.15.52:0/716905082 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff180056f0 con 0x5564a9332f20 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.379+0000 7eff06ffd700 1 -- 172.21.15.52:0/716905082 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff18007ca0 con 0x5564a9332f20 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/716905082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9332f20 msgr2=0x5564a93329e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9332f20 0x5564a93329e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7eff180ba2b0 tx=0x7eff1809b8a0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/716905082 shutdown_connections 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338f60 0x5564a933d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9332f20 0x5564a93329e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/716905082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9338640 0x5564a9327510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/716905082 >> 172.21.15.52:0/716905082 conn(0x5564a925dcc0 msgr2=0x5564a9326e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:22.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/716905082 shutdown_connections 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.380+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/716905082 wait complete. 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 -- start start 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 Processor -- start 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a9344e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338640 0x5564a9345350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a9353250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1bcda0 con 0x5564a9332f20 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1bc6d0 con 0x5564a9338f60 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff1c1b98d0 con 0x5564a9338640 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff2dc1fb80 10 client.0 _ll_register_callbacks cb 0x5564a921e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff21138700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a9344e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff21939700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a9353250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff21138700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a9344e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43824/0 (socket says 172.21.15.52:43824) 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.381+0000 7eff21138700 1 -- 172.21.15.52:0/3201280759 learned_addr learned my addr 172.21.15.52:0/3201280759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff20937700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338640 0x5564a9345350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff21939700 1 -- 172.21.15.52:0/3201280759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338640 msgr2=0x5564a9345350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff21939700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338640 0x5564a9345350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff21939700 1 -- 172.21.15.52:0/3201280759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 msgr2=0x5564a9344e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff21939700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a9344e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff21939700 1 -- 172.21.15.52:0/3201280759 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564a9353810 con 0x5564a9338f60 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff20937700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338640 0x5564a9345350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff21939700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a9353250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5564a933d560 tx=0x7eff14009540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27848c36c0111716 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff04ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff04ff9700 1 -- 172.21.15.52:0/3201280759 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140078e0 con 0x5564a9338f60 2024-06-02T23:20:22.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff04ff9700 1 -- 172.21.15.52:0/3201280759 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7eff14007a80 con 0x5564a9338f60 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff180056f0 con 0x5564a9338f60 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff04ff9700 1 -- 172.21.15.52:0/3201280759 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7eff14007c80 con 0x5564a9338f60 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.382+0000 7eff2dc1fb80 10 client.27377 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5564a93b68b0 con 0x5564a9338f60 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff04ff9700 1 -- 172.21.15.52:0/3201280759 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff180056f0 con 0x5564a9338f60 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff04ff9700 1 -- 172.21.15.52:0/3201280759 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7eff14011070 con 0x5564a9338f60 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 10 client.27377 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 10 client.27377 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 20 client.27377 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a93b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 4 client.27377 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564a928c290 con 0x5564a934c610 2024-06-02T23:20:22.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.383+0000 7eff21138700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a93b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:22.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff04ff9700 10 client.27377 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff21138700 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a93b9060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6cf1dd23423ccff server_cookie=d0e766a74c6ec9b1 in_seq=0 out_seq=0 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff04ff9700 1 -- 172.21.15.52:0/3201280759 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564a928c290 con 0x5564a934c610 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff04ff9700 10 client.27377 handle_command_reply: tid=0 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 client.27377 shutdown 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 msgr2=0x5564a9353250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a9353250 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x5564a933d560 tx=0x7eff14009540 comp rx=0 tx=0).stop 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 shutdown_connections 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5564a934c610 0x5564a93b9060 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5564a9338f60 0x5564a9353250 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5564a9332f20 0x5564a9344e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff04ff9700 0 client.27377 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:20:22.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 --2- 172.21.15.52:0/3201280759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5564a9338640 0x5564a9345350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 >> 172.21.15.52:0/3201280759 conn(0x5564a925dcc0 msgr2=0x5564a9339b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 shutdown_connections 2024-06-02T23:20:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 1 -- 172.21.15.52:0/3201280759 wait complete. 2024-06-02T23:20:22.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.384+0000 7eff2dc1fb80 20 client.27377 trim_cache size 0 max 16384 2024-06-02T23:20:22.390 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff080a2c20 msgr2=0x7eff080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff080a2c20 0x7eff080a50e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7eff0c005fd0 tx=0x7eff0c005f60 comp rx=0 tx=0).stop 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 msgr2=0x7eff1c19e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c19e780 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7eff1800c920 tx=0x7eff1800cc30 comp rx=0 tx=0).stop 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 shutdown_connections 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff080a2c20 0x7eff080a50e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff1c107400 0x7eff1c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff1c101390 0x7eff1c19e780 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 --2- 172.21.15.52:0/2101574993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff1c101d50 0x7eff1c19ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 >> 172.21.15.52:0/2101574993 conn(0x7eff1c078340 msgr2=0x7eff1c06a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:22.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 shutdown_connections 2024-06-02T23:20:22.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:22.385+0000 7eff2339c700 1 -- 172.21.15.52:0/2101574993 wait complete. 2024-06-02T23:20:22.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 157623 1.79 MB/sec execute 331 sec latency 62.881 ms 2024-06-02T23:20:23.520 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:23 smithi052 ceph-mon[32340]: pgmap v686: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 164 KiB/s wr, 28 op/s 2024-06-02T23:20:23.549 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:20:23.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 158235 1.78 MB/sec execute 332 sec latency 15.887 ms 2024-06-02T23:20:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:23 smithi123 ceph-mon[35977]: pgmap v686: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 164 KiB/s wr, 28 op/s 2024-06-02T23:20:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:23 smithi174 ceph-mon[35994]: pgmap v686: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 164 KiB/s wr, 28 op/s 2024-06-02T23:20:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.107+0000 7fc14cfec700 1 -- 172.21.15.52:0/3541234460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148106b90 msgr2=0x7fc148106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.107+0000 7fc14cfec700 1 --2- 172.21.15.52:0/3541234460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148106b90 0x7fc148106f70 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fc138009990 tx=0x7fc138009ca0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.107+0000 7fc14cfec700 1 -- 172.21.15.52:0/3541234460 shutdown_connections 2024-06-02T23:20:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.107+0000 7fc14cfec700 1 --2- 172.21.15.52:0/3541234460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148101510 0x7fc14810d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.107+0000 7fc14cfec700 1 --2- 172.21.15.52:0/3541234460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148106b90 0x7fc148106f70 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.108+0000 7fc14cfec700 1 --2- 172.21.15.52:0/3541234460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148100b50 0x7fc148100fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.108+0000 7fc14cfec700 1 -- 172.21.15.52:0/3541234460 >> 172.21.15.52:0/3541234460 conn(0x7fc148077a00 msgr2=0x7fc1480fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.108+0000 7fc14cfec700 1 -- 172.21.15.52:0/3541234460 shutdown_connections 2024-06-02T23:20:24.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.108+0000 7fc14cfec700 1 -- 172.21.15.52:0/3541234460 wait complete. 2024-06-02T23:20:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 Processor -- start 2024-06-02T23:20:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 -- start start 2024-06-02T23:20:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 0x7fc148197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148101510 0x7fc148197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 0x7fc148197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:24.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148106b90 0x7fc148198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc148111560 con 0x7fc148100b50 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481113e0 con 0x7fc148101510 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14cfec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1481116e0 con 0x7fc148106b90 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 0x7fc148197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57482/0 (socket says 172.21.15.52:57482) 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc14659c700 1 -- 172.21.15.52:0/44747050 learned_addr learned my addr 172.21.15.52:0/44747050 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.109+0000 7fc145d9b700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148101510 0x7fc148197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc146d9d700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148106b90 0x7fc148198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc145d9b700 1 -- 172.21.15.52:0/44747050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148106b90 msgr2=0x7fc148198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc145d9b700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148106b90 0x7fc148198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc145d9b700 1 -- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 msgr2=0x7fc148197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc145d9b700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 0x7fc148197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc145d9b700 1 -- 172.21.15.52:0/44747050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1481a6fa0 con 0x7fc148101510 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc14659c700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 0x7fc148197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc146d9d700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148106b90 0x7fc148198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc145d9b700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148101510 0x7fc148197cd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00ea20 tx=0x7fc13c00ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c004220 con 0x7fc148101510 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1481a7290 con 0x7fc148101510 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc13c00ce20 con 0x7fc148101510 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1481a7820 con 0x7fc148101510 2024-06-02T23:20:24.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.110+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c003d60 con 0x7fc148101510 2024-06-02T23:20:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.111+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc148069180 con 0x7fc148101510 2024-06-02T23:20:24.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.113+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc13c00c970 con 0x7fc148101510 2024-06-02T23:20:24.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.113+0000 7fc1377fe700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1300a2ca0 0x7fc1300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:24.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.114+0000 7fc14659c700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1300a2ca0 0x7fc1300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:24.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.114+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc13c0cf290 con 0x7fc148101510 2024-06-02T23:20:24.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.114+0000 7fc14659c700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1300a2ca0 0x7fc1300a5160 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc138006010 tx=0x7fc13800b630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:24.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.119+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc13c09c1c0 con 0x7fc148101510 2024-06-02T23:20:24.134 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.403+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc14802dca0 con 0x7fc148101510 2024-06-02T23:20:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.404+0000 7fc1377fe700 1 -- 172.21.15.52:0/44747050 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fc13c09bac0 con 0x7fc148101510 2024-06-02T23:20:24.405 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:20:24.405 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:20:24.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.406+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1300a2ca0 msgr2=0x7fc1300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:24.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.406+0000 7fc14cfec700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1300a2ca0 0x7fc1300a5160 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc138006010 tx=0x7fc13800b630 comp rx=0 tx=0).stop 2024-06-02T23:20:24.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148101510 msgr2=0x7fc148197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:24.407 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148101510 0x7fc148197cd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc13c00ea20 tx=0x7fc13c00ed30 comp rx=0 tx=0).stop 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 shutdown_connections 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1300a2ca0 0x7fc1300a5160 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc148106b90 0x7fc148198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc148100b50 0x7fc148197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 --2- 172.21.15.52:0/44747050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc148101510 0x7fc148197cd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 >> 172.21.15.52:0/44747050 conn(0x7fc148077a00 msgr2=0x7fc148103a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 shutdown_connections 2024-06-02T23:20:24.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:24.407+0000 7fc14cfec700 1 -- 172.21.15.52:0/44747050 wait complete. 2024-06-02T23:20:24.409 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:20:24.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 158684 1.78 MB/sec execute 333 sec latency 51.172 ms 2024-06-02T23:20:24.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:24 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/44747050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:24.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:24 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/44747050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:24.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:24 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/44747050' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:25.556 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 146.3 --await-for 45.0' 2024-06-02T23:20:25.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 146.3 --await-for 45.0 2024-06-02T23:20:25.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 159397 1.80 MB/sec execute 334 sec latency 16.249 ms 2024-06-02T23:20:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:25 smithi123 ceph-mon[35977]: pgmap v687: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 206 KiB/s wr, 29 op/s 2024-06-02T23:20:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:25 smithi174 ceph-mon[35994]: pgmap v687: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 206 KiB/s wr, 29 op/s 2024-06-02T23:20:25.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:25 smithi052 ceph-mon[32340]: pgmap v687: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 206 KiB/s wr, 29 op/s 2024-06-02T23:20:26.068 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:26.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 159789 1.80 MB/sec execute 335 sec latency 52.866 ms 2024-06-02T23:20:27.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160055 1.80 MB/sec execute 336 sec latency 15.407 ms 2024-06-02T23:20:27.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:27 smithi123 ceph-mon[35977]: pgmap v688: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 502 KiB/s wr, 83 op/s 2024-06-02T23:20:27.762 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:27 smithi052 ceph-mon[32340]: pgmap v688: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 502 KiB/s wr, 83 op/s 2024-06-02T23:20:27.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 -- 172.21.15.52:0/3427547546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006ab30 msgr2=0x7f7f5010f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/3427547546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006ab30 0x7f7f5010f320 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009f90 tx=0x7f7f4800b330 comp rx=0 tx=0).stop 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 -- 172.21.15.52:0/3427547546 shutdown_connections 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/3427547546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006ab30 0x7f7f5010f320 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/3427547546 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 0x7f7f501088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/3427547546 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5006a170 0x7f7f5006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.795+0000 7f7f54d9f700 1 -- 172.21.15.52:0/3427547546 >> 172.21.15.52:0/3427547546 conn(0x7f7f50078340 msgr2=0x7f7f50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.796+0000 7f7f54d9f700 1 -- 172.21.15.52:0/3427547546 shutdown_connections 2024-06-02T23:20:27.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.796+0000 7f7f54d9f700 1 -- 172.21.15.52:0/3427547546 wait complete. 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.796+0000 7f7f54d9f700 1 Processor -- start 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.796+0000 7f7f54d9f700 1 -- start start 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f54d9f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 0x7f7f50101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f54d9f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5006ab30 0x7f7f501019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f54d9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 0x7f7f50101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f54d9f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f50112630 con 0x7f7f5006ab30 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f54d9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501124b0 con 0x7f7f50108500 2024-06-02T23:20:27.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f54d9f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f501127b0 con 0x7f7f5006a170 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 0x7f7f50101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 0x7f7f50101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38564/0 (socket says 172.21.15.52:38564) 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 -- 172.21.15.52:0/772320329 learned_addr learned my addr 172.21.15.52:0/772320329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 0x7f7f50101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5006ab30 0x7f7f501019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 -- 172.21.15.52:0/772320329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 msgr2=0x7f7f50101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 0x7f7f50101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 -- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5006ab30 msgr2=0x7f7f501019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5006ab30 0x7f7f501019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4e59c700 1 -- 172.21.15.52:0/772320329 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f501a7010 con 0x7f7f5006a170 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.797+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 0x7f7f50101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:27.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.798+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 0x7f7f50101460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9c0 tx=0x7f7f4000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.798+0000 7f7f3f7fe700 1 -- 172.21.15.52:0/772320329 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4000cbd0 con 0x7f7f5006a170 2024-06-02T23:20:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.798+0000 7f7f3f7fe700 1 -- 172.21.15.52:0/772320329 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4000ee60 con 0x7f7f5006a170 2024-06-02T23:20:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.798+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f501a7300 con 0x7f7f5006a170 2024-06-02T23:20:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.798+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f501a7840 con 0x7f7f5006a170 2024-06-02T23:20:27.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.798+0000 7f7f3f7fe700 1 -- 172.21.15.52:0/772320329 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400055b0 con 0x7f7f5006a170 2024-06-02T23:20:27.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.801+0000 7f7f3f7fe700 1 -- 172.21.15.52:0/772320329 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f40021440 con 0x7f7f5006a170 2024-06-02T23:20:27.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.802+0000 7f7f3f7fe700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.802+0000 7f7f3f7fe700 1 -- 172.21.15.52:0/772320329 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7f400cf070 con 0x7f7f5006a170 2024-06-02T23:20:27.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.802+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.802+0000 7f7f5f622b80 1 Processor -- start 2024-06-02T23:20:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.802+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009750 tx=0x7f7f44009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 -- start start 2024-06-02T23:20:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef8726d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87267260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 0x55ef87267bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef87159f60 con 0x55ef872677a0 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef871b8850 con 0x55ef8726d880 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef87120460 con 0x55ef8726cf60 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f4ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 0x55ef87267bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f4dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87267260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f4ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 0x55ef87267bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57504/0 (socket says 172.21.15.52:57504) 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f4dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87267260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42040/0 (socket says 172.21.15.52:42040) 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3452985710 learned_addr learned my addr 172.21.15.52:0/3452985710 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:27.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.803+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef8726d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3452985710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 msgr2=0x55ef8726d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef8726d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3452985710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 msgr2=0x55ef87267260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87267260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3452985710 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef871078d0 con 0x55ef872677a0 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87267260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 0x55ef87267bc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c10 tx=0x7f7f48004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1807a15ad9fa1b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f3cff9700 1 -- 172.21.15.52:0/3452985710 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4800b9f0 con 0x55ef872677a0 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f3cff9700 1 -- 172.21.15.52:0/3452985710 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4800bb90 con 0x55ef872677a0 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.804+0000 7f7f3cff9700 1 -- 172.21.15.52:0/3452985710 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480052a0 con 0x55ef872677a0 2024-06-02T23:20:27.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3452985710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 msgr2=0x55ef87267bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 0x55ef87267bc0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c10 tx=0x7f7f48004f90 comp rx=0 tx=0).stop 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3452985710 shutdown_connections 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef872677a0 0x55ef87267bc0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef8726d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3452985710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87267260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3452985710 >> 172.21.15.52:0/3452985710 conn(0x55ef87194080 msgr2=0x55ef87262220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3452985710 shutdown_connections 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.805+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3452985710 wait complete. 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 -- start start 2024-06-02T23:20:27.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 Processor -- start 2024-06-02T23:20:27.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef87273530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef87270830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726d880 0x55ef87270d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef87158fc0 con 0x55ef8726cf60 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef871b4810 con 0x55ef872677a0 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef87274200 con 0x55ef8726d880 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f5f622b80 10 client.0 _ll_register_callbacks cb 0x55ef8714e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef87273530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef87273530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42048/0 (socket says 172.21.15.52:42048) 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f4e59c700 1 -- 172.21.15.52:0/3447389301 learned_addr learned my addr 172.21.15.52:0/3447389301 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef87270830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.806+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726d880 0x55ef87270d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f4e59c700 1 -- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726d880 msgr2=0x55ef87270d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726d880 0x55ef87270d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f4e59c700 1 -- 172.21.15.52:0/3447389301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 msgr2=0x55ef87270830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef87270830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f4e59c700 1 -- 172.21.15.52:0/3447389301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef8717e750 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef87273530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f40097d10 tx=0x7f7f4009c080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=baa1faba5622b60a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3447389301 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f400b9060 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3447389301 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7f400a40e0 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4800bb90 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3447389301 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f4009fcb0 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f5f622b80 10 client.27395 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.807+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef8727e990 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3447389301 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4800bb90 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3447389301 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7f40086300 con 0x55ef872677a0 2024-06-02T23:20:27.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f5f622b80 10 client.27395 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f5f622b80 10 client.27395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f5f622b80 20 client.27395 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef87283370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f5f622b80 4 client.27395 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef871c5d50 con 0x55ef87280f20 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef87283370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef87283370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=18de3e8a6a3f4eb2 server_cookie=af89105860fec454 in_seq=0 out_seq=0 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.808+0000 7f7f35ffb700 10 client.27395 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3447389301 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef871c5d50 con 0x55ef87280f20 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f35ffb700 10 client.27395 handle_command_reply: tid=0 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 client.27395 shutdown 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 msgr2=0x55ef87273530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef87273530 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7f40097d10 tx=0x7f7f4009c080 comp rx=0 tx=0).stop 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 shutdown_connections 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef87283370 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726d880 0x55ef87270d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f35ffb700 0 client.27395 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef87273530 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3447389301 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef87270830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.809+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 >> 172.21.15.52:0/3447389301 conn(0x55ef87194080 msgr2=0x55ef87261700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.810+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 shutdown_connections 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.810+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3447389301 wait complete. 2024-06-02T23:20:27.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.810+0000 7f7f5f622b80 20 client.27395 trim_cache size 0 max 16384 2024-06-02T23:20:27.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.826+0000 7f7f5f622b80 1 Processor -- start 2024-06-02T23:20:27.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.826+0000 7f7f5f622b80 1 -- start start 2024-06-02T23:20:27.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef872677a0 0x55ef87267260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef8726e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87261e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000d250 con 0x55ef8726cf60 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000ed90 con 0x55ef8726d880 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000d920 con 0x55ef872677a0 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef8726e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef8726e2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57526/0 (socket says 172.21.15.52:57526) 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef872677a0 0x55ef87267260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87261e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 -- 172.21.15.52:0/2533462804 learned_addr learned my addr 172.21.15.52:0/2533462804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:27.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 -- 172.21.15.52:0/2533462804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef872677a0 msgr2=0x55ef87267260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef872677a0 0x55ef87267260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 -- 172.21.15.52:0/2533462804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 msgr2=0x55ef87261e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87261e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 -- 172.21.15.52:0/2533462804 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef87263980 con 0x55ef8726cf60 2024-06-02T23:20:27.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.827+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef8726e2d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7f500789b0 tx=0x7f7f44014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91d778316b06688 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f37fff700 1 -- 172.21.15.52:0/2533462804 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4401e050 con 0x55ef8726cf60 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f37fff700 1 -- 172.21.15.52:0/2533462804 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4402b3e0 con 0x55ef8726cf60 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f37fff700 1 -- 172.21.15.52:0/2533462804 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4402a470 con 0x55ef8726cf60 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 -- 172.21.15.52:0/2533462804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 msgr2=0x55ef8726e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef8726e2d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7f500789b0 tx=0x7f7f44014000 comp rx=0 tx=0).stop 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 -- 172.21.15.52:0/2533462804 shutdown_connections 2024-06-02T23:20:27.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef8726d880 0x55ef87261e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef872677a0 0x55ef87267260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/2533462804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726cf60 0x55ef8726e2d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 -- 172.21.15.52:0/2533462804 >> 172.21.15.52:0/2533462804 conn(0x55ef87194080 msgr2=0x55ef872728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 -- 172.21.15.52:0/2533462804 shutdown_connections 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.828+0000 7f7f5f622b80 1 -- 172.21.15.52:0/2533462804 wait complete. 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 -- start start 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 Processor -- start 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef872888b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef87288df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 0x55ef87289330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000e6c0 con 0x55ef8726d880 2024-06-02T23:20:27.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000dff0 con 0x55ef872677a0 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f3000b1f0 con 0x55ef8726cf60 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.829+0000 7f7f5f622b80 10 client.0 _ll_register_callbacks cb 0x55ef8714e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 0x55ef87289330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 0x55ef87289330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57530/0 (socket says 172.21.15.52:57530) 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3382858460 learned_addr learned my addr 172.21.15.52:0/3382858460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef87288df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef872888b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 msgr2=0x55ef87288df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef87288df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3382858460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 msgr2=0x55ef872888b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef872888b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 -- 172.21.15.52:0/3382858460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef872eac50 con 0x55ef8726d880 2024-06-02T23:20:27.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4dd9b700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef87288df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f4ed9d700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 0x55ef87289330 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c40 tx=0x7f7f48007490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a9ecf6c360d3a4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f35ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.830+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3382858460 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48034060 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3382858460 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7f4802f460 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4402b3e0 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3382858460 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7f4802f660 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 10 client.17934 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef872ebf20 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3382858460 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f4402b3e0 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3382858460 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7f48011070 con 0x55ef8726d880 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 10 client.17934 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 10 client.17934 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 20 client.17934 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:27.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef872ee100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 4 client.17934 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef871c02d0 con 0x55ef87280f20 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.831+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef872ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f35ffb700 10 client.17934 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f4e59c700 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef872ee100 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1d668b4478e8af1c server_cookie=88ef33a97c970380 in_seq=0 out_seq=0 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f35ffb700 1 -- 172.21.15.52:0/3382858460 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef871c02d0 con 0x55ef87280f20 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f35ffb700 10 client.17934 handle_command_reply: tid=0 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 client.17934 shutdown 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 msgr2=0x55ef87289330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 0x55ef87289330 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009c40 tx=0x7f7f48007490 comp rx=0 tx=0).stop 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 shutdown_connections 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55ef87280f20 0x55ef872ee100 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f35ffb700 0 client.17934 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:20:27.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef8726d880 0x55ef87289330 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef872677a0 0x55ef872888b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 --2- 172.21.15.52:0/3382858460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef8726cf60 0x55ef87288df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 >> 172.21.15.52:0/3382858460 conn(0x55ef87194080 msgr2=0x55ef872728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 shutdown_connections 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 1 -- 172.21.15.52:0/3382858460 wait complete. 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.832+0000 7f7f5f622b80 20 client.17934 trim_cache size 0 max 16384 2024-06-02T23:20:27.838 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 msgr2=0x7f7f380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7f44009750 tx=0x7f7f44009310 comp rx=0 tx=0).stop 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 msgr2=0x7f7f50101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 0x7f7f50101460 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000e9c0 tx=0x7f7f4000ecd0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 shutdown_connections 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f50108500 0x7f7f50101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f5006a170 0x7f7f50101460 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 --2- 172.21.15.52:0/772320329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f5006ab30 0x7f7f501019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:27.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 >> 172.21.15.52:0/772320329 conn(0x7f7f50078340 msgr2=0x7f7f50100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:27.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 shutdown_connections 2024-06-02T23:20:27.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:27.833+0000 7f7f54d9f700 1 -- 172.21.15.52:0/772320329 wait complete. 2024-06-02T23:20:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:27 smithi174 ceph-mon[35994]: pgmap v688: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 502 KiB/s wr, 83 op/s 2024-06-02T23:20:28.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1871849231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c101df0 msgr2=0x7f0b9c1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1871849231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c101df0 0x7f0b9c1021d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0099c0 tx=0x7f0b8c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1871849231 shutdown_connections 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1871849231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b9c106750 0x7f0b9c10e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1871849231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c101df0 0x7f0b9c1021d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1871849231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c1027a0 0x7f0b9c106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1871849231 >> 172.21.15.52:0/1871849231 conn(0x7f0b9c077a30 msgr2=0x7f0b9c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1871849231 shutdown_connections 2024-06-02T23:20:28.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.603+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1871849231 wait complete. 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 Processor -- start 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 -- start start 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b9c101df0 0x7f0b9c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c1027a0 0x7f0b9c19ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c106750 0x7f0b9c198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c111520 con 0x7f0b9c1027a0 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1113a0 con 0x7f0b9c106750 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.604+0000 7f0ba18bf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1116a0 con 0x7f0b9c101df0 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c1027a0 0x7f0b9c19ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c1027a0 0x7f0b9c19ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57550/0 (socket says 172.21.15.52:57550) 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c106750 0x7f0b9c198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/1576740389 learned_addr learned my addr 172.21.15.52:0/1576740389 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9affd700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b9c101df0 0x7f0b9c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/1576740389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b9c101df0 msgr2=0x7f0b9c19e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b9c101df0 0x7f0b9c19e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c1027a0 msgr2=0x7f0b9c19ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c1027a0 0x7f0b9c19ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/1576740389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b9c198fe0 con 0x7f0b9c106750 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c106750 0x7f0b9c198800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400efc0 tx=0x7f0b9400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0ba08bd700 1 -- 172.21.15.52:0/1576740389 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b940092e0 con 0x7f0b9c106750 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b9c199270 con 0x7f0b9c106750 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0ba08bd700 1 -- 172.21.15.52:0/1576740389 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b94004bb0 con 0x7f0b9c106750 2024-06-02T23:20:28.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.605+0000 7f0ba08bd700 1 -- 172.21.15.52:0/1576740389 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b94004020 con 0x7f0b9c106750 2024-06-02T23:20:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.606+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b9c0fcda0 con 0x7f0b9c106750 2024-06-02T23:20:28.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.608+0000 7f0ba08bd700 1 -- 172.21.15.52:0/1576740389 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0b9401b030 con 0x7f0b9c106750 2024-06-02T23:20:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.609+0000 7f0ba08bd700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b840a2d40 0x7f0b840a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.609+0000 7f0b9affd700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b840a2d40 0x7f0b840a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.609+0000 7f0ba08bd700 1 -- 172.21.15.52:0/1576740389 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0b940ce9f0 con 0x7f0b9c106750 2024-06-02T23:20:28.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.609+0000 7f0b9affd700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b840a2d40 0x7f0b840a5200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c003760 tx=0x7f0b8c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 Processor -- start 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 -- start start 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf16f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef3ce689a0 con 0x55ef3cf1d5f0 2024-06-02T23:20:28.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef3ce108b0 con 0x55ef3cf17460 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0bac142b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef3ce64960 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0b9a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf16f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0b9a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf16f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57566/0 (socket says 172.21.15.52:57566) 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/3221713314 learned_addr learned my addr 172.21.15.52:0/3221713314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.610+0000 7f0b9affd700 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/3221713314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 msgr2=0x55ef3cf1d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/3221713314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 msgr2=0x55ef3cf16f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf16f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/3221713314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3ce070e0 con 0x55ef3cf17460 2024-06-02T23:20:28.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf178a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b940cc570 tx=0x7f0b9409acf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbad5d8376fb26ce server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b81ffb700 1 -- 172.21.15.52:0/3221713314 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9409ba10 con 0x55ef3cf17460 2024-06-02T23:20:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b81ffb700 1 -- 172.21.15.52:0/3221713314 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b9409b310 con 0x55ef3cf17460 2024-06-02T23:20:28.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.611+0000 7f0b81ffb700 1 -- 172.21.15.52:0/3221713314 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b940bb0c0 con 0x55ef3cf17460 2024-06-02T23:20:28.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 -- 172.21.15.52:0/3221713314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 msgr2=0x55ef3cf178a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf178a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0b940cc570 tx=0x7f0b9409acf0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 -- 172.21.15.52:0/3221713314 shutdown_connections 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf178a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 --2- 172.21.15.52:0/3221713314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf16f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 -- 172.21.15.52:0/3221713314 >> 172.21.15.52:0/3221713314 conn(0x55ef3ce41e50 msgr2=0x55ef3cf121b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 -- 172.21.15.52:0/3221713314 shutdown_connections 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 -- 172.21.15.52:0/3221713314 wait complete. 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.612+0000 7f0bac142b80 1 -- start start 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 Processor -- start 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf17460 0x55ef3cf23400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf20700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf20c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef3cf24060 con 0x55ef3cf1d5f0 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef3ce08d80 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef3cf241e0 con 0x55ef3cf17460 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf20c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0bac142b80 10 client.0 _ll_register_callbacks cb 0x55ef3ce02ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf20c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57570/0 (socket says 172.21.15.52:57570) 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9b7fe700 1 -- 172.21.15.52:0/2911474574 learned_addr learned my addr 172.21.15.52:0/2911474574 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf20700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9affd700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf17460 0x55ef3cf23400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/2911474574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf17460 msgr2=0x55ef3cf23400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf17460 0x55ef3cf23400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/2911474574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 msgr2=0x55ef3cf20c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf20c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.613+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/2911474574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3cdb7a60 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf20700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b900129e0 tx=0x7f0b90012cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2f95332b1a89f0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b7affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b7affd700 1 -- 172.21.15.52:0/2911474574 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90004050 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b7affd700 1 -- 172.21.15.52:0/2911474574 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0b900041f0 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b9409b310 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b7affd700 1 -- 172.21.15.52:0/2911474574 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b900043f0 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0bac142b80 10 client.27413 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef3cf2ea50 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.614+0000 7f0b7affd700 1 -- 172.21.15.52:0/2911474574 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b9409b310 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0b7affd700 1 -- 172.21.15.52:0/2911474574 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0b900039f0 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0bac142b80 10 client.27413 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0bac142b80 10 client.27413 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0bac142b80 20 client.27413 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf310e0 0x55ef3cf334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0bac142b80 4 client.27413 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef3ce75ea0 con 0x55ef3cf310e0 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0b9affd700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf310e0 0x55ef3cf334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0b9affd700 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf310e0 0x55ef3cf334d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b791478e67c40ff server_cookie=fa8d950ce0368fe6 in_seq=0 out_seq=0 2024-06-02T23:20:28.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.615+0000 7f0b7affd700 10 client.27413 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:28.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0b7affd700 1 -- 172.21.15.52:0/2911474574 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef3ce75ea0 con 0x55ef3cf310e0 2024-06-02T23:20:28.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0b7affd700 10 client.27413 handle_command_reply: tid=0 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 client.27413 shutdown 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1ccd0 msgr2=0x55ef3cf20700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf20700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0b900129e0 tx=0x7f0b90012cf0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 shutdown_connections 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf310e0 0x55ef3cf334d0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf20c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf17460 0x55ef3cf23400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2911474574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf20700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0b7affd700 0 client.27413 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.616+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 >> 172.21.15.52:0/2911474574 conn(0x55ef3ce41e50 msgr2=0x55ef3cf11da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.617+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 shutdown_connections 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.617+0000 7f0bac142b80 1 -- 172.21.15.52:0/2911474574 wait complete. 2024-06-02T23:20:28.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.617+0000 7f0bac142b80 20 client.27413 trim_cache size 0 max 16384 2024-06-02T23:20:28.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160361 1.80 MB/sec execute 337 sec latency 30.554 ms 2024-06-02T23:20:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 Processor -- start 2024-06-02T23:20:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 -- start start 2024-06-02T23:20:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf2d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf38da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1aeb40 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1b0680 con 0x55ef3cf17460 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.638+0000 7f0bac142b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1af210 con 0x55ef3cf1d5f0 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57572/0 (socket says 172.21.15.52:57572) 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/607955078 learned_addr learned my addr 172.21.15.52:0/607955078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf2d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf38da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/607955078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1d5f0 msgr2=0x55ef3cf38da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf38da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/607955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 msgr2=0x55ef3cf2d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf2d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 -- 172.21.15.52:0/607955078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef3cf123b0 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9affd700 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf2d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:28.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1c0f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0b90003e60 tx=0x7f0b90009990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a965966ff40e03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.639+0000 7f0b817fa700 1 -- 172.21.15.52:0/607955078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b90003fd0 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0b817fa700 1 -- 172.21.15.52:0/607955078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b90003b30 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0b817fa700 1 -- 172.21.15.52:0/607955078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b9000fdb0 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 -- 172.21.15.52:0/607955078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 msgr2=0x55ef3cf1c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1c0f0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f0b90003e60 tx=0x7f0b90009990 comp rx=0 tx=0).stop 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 -- 172.21.15.52:0/607955078 shutdown_connections 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf38da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf17460 0x55ef3cf2d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 --2- 172.21.15.52:0/607955078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf1c0f0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 -- 172.21.15.52:0/607955078 >> 172.21.15.52:0/607955078 conn(0x55ef3ce41e50 msgr2=0x55ef3cf1e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 -- 172.21.15.52:0/607955078 shutdown_connections 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.640+0000 7f0bac142b80 1 -- 172.21.15.52:0/607955078 wait complete. 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 -- start start 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 Processor -- start 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 0x55ef3cf29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf27c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf28180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1affb0 con 0x55ef3cf17460 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1af8e0 con 0x55ef3cf1d5f0 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0b9c1acb00 con 0x55ef3cf1ccd0 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0bac142b80 10 client.0 _ll_register_callbacks cb 0x55ef3ce02ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0b9affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 0x55ef3cf29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0b9affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 0x55ef3cf29890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57578/0 (socket says 172.21.15.52:57578) 2024-06-02T23:20:28.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0b9affd700 1 -- 172.21.15.52:0/2321614333 learned_addr learned my addr 172.21.15.52:0/2321614333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.641+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf27c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9b7fe700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf28180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9affd700 1 -- 172.21.15.52:0/2321614333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 msgr2=0x55ef3cf27c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9affd700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf27c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9affd700 1 -- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1d5f0 msgr2=0x55ef3cf28180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9affd700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf28180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9affd700 1 -- 172.21.15.52:0/2321614333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef3cf12580 con 0x55ef3cf17460 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b9affd700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 0x55ef3cf29890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0098e0 tx=0x7f0b8c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3d731c07abfd41 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b7affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b7affd700 1 -- 172.21.15.52:0/2321614333 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b8c00b9a0 con 0x55ef3cf17460 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b90003b30 con 0x55ef3cf17460 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b7affd700 1 -- 172.21.15.52:0/2321614333 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0b8c00bb40 con 0x55ef3cf17460 2024-06-02T23:20:28.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0b7affd700 1 -- 172.21.15.52:0/2321614333 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0b8c042070 con 0x55ef3cf17460 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0bac142b80 10 client.17952 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.642+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ef3cf9c970 con 0x55ef3cf17460 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0b7affd700 1 -- 172.21.15.52:0/2321614333 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0b90003b30 con 0x55ef3cf17460 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0b7affd700 1 -- 172.21.15.52:0/2321614333 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0b8c023ce0 con 0x55ef3cf17460 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0bac142b80 10 client.17952 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0bac142b80 10 client.17952 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0bac142b80 20 client.17952 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf9d0f0 0x55ef3cf9f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0bac142b80 4 client.17952 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.3, "await_for": 45.0}] 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 146.3, "await_for": 45.0}) -- 0x55ef3ce70420 con 0x55ef3cf9d0f0 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf9d0f0 0x55ef3cf9f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0b9a7fc700 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf9d0f0 0x55ef3cf9f510 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e9d059b137715 server_cookie=7fb7033c1190b03d in_seq=0 out_seq=0 2024-06-02T23:20:28.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.643+0000 7f0b7affd700 10 client.17952 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:28.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0b7affd700 1 -- 172.21.15.52:0/2321614333 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ef3ce70420 con 0x55ef3cf9d0f0 2024-06-02T23:20:28.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0b7affd700 10 client.17952 handle_command_reply: tid=0 2024-06-02T23:20:28.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 client.17952 shutdown 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 msgr2=0x55ef3cf29890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 0x55ef3cf29890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c0098e0 tx=0x7f0b8c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 shutdown_connections 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef3cf9d0f0 0x55ef3cf9f510 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef3cf1d5f0 0x55ef3cf28180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef3cf17460 0x55ef3cf29890 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0b7affd700 0 client.17952 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 --2- 172.21.15.52:0/2321614333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef3cf1ccd0 0x55ef3cf27c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.711+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 >> 172.21.15.52:0/2321614333 conn(0x55ef3ce41e50 msgr2=0x55ef3cf1c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 shutdown_connections 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0bac142b80 1 -- 172.21.15.52:0/2321614333 wait complete. 2024-06-02T23:20:28.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0bac142b80 20 client.17952 trim_cache size 0 max 16384 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 30, 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "9923b3d7": { 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 30, 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:20:28.714 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 146.3, 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:28.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b840a2d40 msgr2=0x7f0b840a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b840a2d40 0x7f0b840a5200 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0b8c003760 tx=0x7f0b8c005fb0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c106750 msgr2=0x7f0b9c198800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c106750 0x7f0b9c198800 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0b9400efc0 tx=0x7f0b9400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 shutdown_connections 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0b840a2d40 0x7f0b840a5200 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b9c106750 0x7f0b9c198800 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0b9c101df0 0x7f0b9c19e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 --2- 172.21.15.52:0/1576740389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0b9c1027a0 0x7f0b9c19ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 >> 172.21.15.52:0/1576740389 conn(0x7f0b9c077a30 msgr2=0x7f0b9c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 shutdown_connections 2024-06-02T23:20:28.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:28.712+0000 7f0ba18bf700 1 -- 172.21.15.52:0/1576740389 wait complete. 2024-06-02T23:20:29.063 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:20:29.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.79 MB/sec execute 338 sec latency 951.046 ms 2024-06-02T23:20:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:29 smithi052 ceph-mon[32340]: pgmap v689: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 642 KiB/s wr, 89 op/s 2024-06-02T23:20:29.689 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9923b3d7', quiesce took 9.7 seconds. Will release after: 16.6 2024-06-02T23:20:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:29 smithi123 ceph-mon[35977]: pgmap v689: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 642 KiB/s wr, 89 op/s 2024-06-02T23:20:29.777 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:29 smithi174 ceph-mon[35994]: pgmap v689: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 642 KiB/s wr, 89 op/s 2024-06-02T23:20:30.637 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.79 MB/sec execute 339 sec latency 1951.095 ms 2024-06-02T23:20:31.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 -- 172.21.15.52:0/2353366623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738102770 msgr2=0x7f2738102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 --2- 172.21.15.52:0/2353366623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738102770 0x7f2738102b50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f27340099c0 tx=0x7f2734009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 -- 172.21.15.52:0/2353366623 shutdown_connections 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 --2- 172.21.15.52:0/2353366623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 0x7f273810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 --2- 172.21.15.52:0/2353366623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738102770 0x7f2738102b50 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 --2- 172.21.15.52:0/2353366623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738103120 0x7f2738107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 -- 172.21.15.52:0/2353366623 >> 172.21.15.52:0/2353366623 conn(0x7f2738079600 msgr2=0x7f2738079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 -- 172.21.15.52:0/2353366623 shutdown_connections 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 -- 172.21.15.52:0/2353366623 wait complete. 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.441+0000 7f273fd4e700 1 Processor -- start 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 -- start start 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738102770 0x7f2738074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 0x7f2738074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 0x7f2738075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2738112650 con 0x7f2738103120 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27381124d0 con 0x7f2738107860 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273fd4e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27381127d0 con 0x7f2738102770 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 0x7f2738074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 0x7f2738074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57594/0 (socket says 172.21.15.52:57594) 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 -- 172.21.15.52:0/4040104712 learned_addr learned my addr 172.21.15.52:0/4040104712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273daea700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738102770 0x7f2738074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273e2eb700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 0x7f2738075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 -- 172.21.15.52:0/4040104712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738102770 msgr2=0x7f2738074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738102770 0x7f2738074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 -- 172.21.15.52:0/4040104712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 msgr2=0x7f2738075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 0x7f2738075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 -- 172.21.15.52:0/4040104712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27381a6ff0 con 0x7f2738103120 2024-06-02T23:20:31.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273e2eb700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 0x7f2738075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273daea700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738102770 0x7f2738074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.442+0000 7f273d2e9700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 0x7f2738074cd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f272800d980 tx=0x7f272800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.443+0000 7f272effd700 1 -- 172.21.15.52:0/4040104712 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728017070 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.443+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27381a72e0 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.443+0000 7f272effd700 1 -- 172.21.15.52:0/4040104712 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2728004830 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.443+0000 7f272effd700 1 -- 172.21.15.52:0/4040104712 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2728005440 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.443+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27381a7820 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.445+0000 7f272effd700 1 -- 172.21.15.52:0/4040104712 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27280055e0 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.446+0000 7f272effd700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27240a2c20 0x7f27240a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.446+0000 7f273daea700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27240a2c20 0x7f27240a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.446+0000 7f272effd700 1 -- 172.21.15.52:0/4040104712 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f27280cefd0 con 0x7f2738103120 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.446+0000 7f273daea700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27240a2c20 0x7f27240a50e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27340059d0 tx=0x7f2734005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:31.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.446+0000 7f274a5d1b80 1 Processor -- start 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 -- start start 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfaea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaeabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f3cf9dcf60 con 0x55f3cfaea7a0 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f3cfa3b850 con 0x55f3cfaf0880 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3cf9a3460 con 0x55f3cfaeff60 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaeabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaeabc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57598/0 (socket says 172.21.15.52:57598) 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 -- 172.21.15.52:0/3881438007 learned_addr learned my addr 172.21.15.52:0/3881438007 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273daea700 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfaea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 -- 172.21.15.52:0/3881438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaeff60 msgr2=0x55f3cfaf0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 -- 172.21.15.52:0/3881438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 msgr2=0x55f3cfaea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfaea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.447+0000 7f273e2eb700 1 -- 172.21.15.52:0/3881438007 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3cf98a8d0 con 0x55f3cfaea7a0 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.448+0000 7f273e2eb700 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaeabc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2730009440 tx=0x7f2730009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd6b5f26b2bd45c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.448+0000 7f2723fff700 1 -- 172.21.15.52:0/3881438007 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730015640 con 0x55f3cfaea7a0 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.448+0000 7f2723fff700 1 -- 172.21.15.52:0/3881438007 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2730015c90 con 0x55f3cfaea7a0 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.448+0000 7f2723fff700 1 -- 172.21.15.52:0/3881438007 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2730014d50 con 0x55f3cfaea7a0 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.448+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3881438007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 msgr2=0x55f3cfaeabc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.448+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaeabc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f2730009440 tx=0x7f2730009920 comp rx=0 tx=0).stop 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3881438007 shutdown_connections 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaeabc0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3881438007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfaea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3881438007 >> 172.21.15.52:0/3881438007 conn(0x55f3cfa17080 msgr2=0x55f3cfae5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:31.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3881438007 shutdown_connections 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3881438007 wait complete. 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 -- start start 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 Processor -- start 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaf6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.449+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f3cf9dbfc0 con 0x55f3cfaeff60 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f3cfa37810 con 0x55f3cfaea7a0 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3cfaf7260 con 0x55f3cfaf0880 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 10 client.0 _ll_register_callbacks cb 0x55f3cf9d15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57606/0 (socket says 172.21.15.52:57606) 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 -- 172.21.15.52:0/3275348090 learned_addr learned my addr 172.21.15.52:0/3275348090 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273daea700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaf6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273e2eb700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 -- 172.21.15.52:0/3275348090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 msgr2=0x55f3cfaf3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 -- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 msgr2=0x55f3cfaf6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaf6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 -- 172.21.15.52:0/3275348090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3cfa01750 con 0x55f3cfaeff60 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf3890 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2728009520 tx=0x7f2728004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195343106d632fe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f27217fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f27217fa700 1 -- 172.21.15.52:0/3275348090 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280b4060 con 0x55f3cfaeff60 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f27217fa700 1 -- 172.21.15.52:0/3275348090 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f272800e350 con 0x55f3cfaeff60 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2730015c90 con 0x55f3cfaeff60 2024-06-02T23:20:31.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f27217fa700 1 -- 172.21.15.52:0/3275348090 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f272800e550 con 0x55f3cfaeff60 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 10 client.17964 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.450+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3cfb019e0 con 0x55f3cfaeff60 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f27217fa700 1 -- 172.21.15.52:0/3275348090 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2730015c90 con 0x55f3cfaeff60 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f27217fa700 1 -- 172.21.15.52:0/3275348090 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f272809ba60 con 0x55f3cfaeff60 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f274a5d1b80 10 client.17964 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f274a5d1b80 10 client.17964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f274a5d1b80 20 client.17964 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb03f70 0x55f3cfb06360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f274a5d1b80 4 client.17964 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3cfa48d50 con 0x55f3cfb03f70 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.451+0000 7f273daea700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb03f70 0x55f3cfb06360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f27217fa700 10 client.17964 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f273daea700 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb03f70 0x55f3cfb06360 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86e4f19425cee6b1 server_cookie=cad99aed7d31b1b7 in_seq=0 out_seq=0 2024-06-02T23:20:31.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f27217fa700 1 -- 172.21.15.52:0/3275348090 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f3cfa48d50 con 0x55f3cfb03f70 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f27217fa700 10 client.17964 handle_command_reply: tid=0 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f274a5d1b80 1 client.17964 shutdown 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 msgr2=0x55f3cfaf3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.452+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf3890 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f2728009520 tx=0x7f2728004e20 comp rx=0 tx=0).stop 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 shutdown_connections 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb03f70 0x55f3cfb06360 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaf6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f27217fa700 0 client.17964 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3275348090 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf3890 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 >> 172.21.15.52:0/3275348090 conn(0x55f3cfa17080 msgr2=0x55f3cfae4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 shutdown_connections 2024-06-02T23:20:31.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3275348090 wait complete. 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.453+0000 7f274a5d1b80 20 client.17964 trim_cache size 0 max 16384 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 Processor -- start 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 -- start start 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaea260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27381aed30 con 0x55f3cfaeff60 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27381b0870 con 0x55f3cfaea7a0 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27381af400 con 0x55f3cfaf0880 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57620/0 (socket says 172.21.15.52:57620) 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f273d2e9700 1 -- 172.21.15.52:0/3245399398 learned_addr learned my addr 172.21.15.52:0/3245399398 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:31.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f273daea700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaea260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f273e2eb700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.468+0000 7f273d2e9700 1 -- 172.21.15.52:0/3245399398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 msgr2=0x55f3cfaf27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273d2e9700 1 -- 172.21.15.52:0/3245399398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 msgr2=0x55f3cfaea260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaea260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273d2e9700 1 -- 172.21.15.52:0/3245399398 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3cfb069c0 con 0x55f3cfaeff60 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273daea700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273e2eb700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f273d2e9700 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf14f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f272800e6f0 tx=0x7f2728099d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241a7122f78a4573 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f27237fe700 1 -- 172.21.15.52:0/3245399398 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27280a4470 con 0x55f3cfaeff60 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f27237fe700 1 -- 172.21.15.52:0/3245399398 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272800e350 con 0x55f3cfaeff60 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f27237fe700 1 -- 172.21.15.52:0/3245399398 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272809cb30 con 0x55f3cfaeff60 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3245399398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 msgr2=0x55f3cfaf14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.469+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf14f0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f272800e6f0 tx=0x7f2728099d10 comp rx=0 tx=0).stop 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3245399398 shutdown_connections 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaf0880 0x55f3cfaf27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaea7a0 0x55f3cfaea260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/3245399398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfaf14f0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3245399398 >> 172.21.15.52:0/3245399398 conn(0x55f3cfa17080 msgr2=0x55f3cfaf50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3245399398 shutdown_connections 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- 172.21.15.52:0/3245399398 wait complete. 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- start start 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 Processor -- start 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaea7a0 0x55f3cfafcd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfb0b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfb0bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27381b01a0 con 0x55f3cfaeff60 2024-06-02T23:20:31.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27381afad0 con 0x55f3cfaf0880 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27381accf0 con 0x55f3cfaea7a0 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.470+0000 7f274a5d1b80 10 client.0 _ll_register_callbacks cb 0x55f3cf9d15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfb0b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfb0b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57628/0 (socket says 172.21.15.52:57628) 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 -- 172.21.15.52:0/951930204 learned_addr learned my addr 172.21.15.52:0/951930204 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273e2eb700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfb0bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273daea700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaea7a0 0x55f3cfafcd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 -- 172.21.15.52:0/951930204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaea7a0 msgr2=0x55f3cfafcd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaea7a0 0x55f3cfafcd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 -- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 msgr2=0x55f3cfb0bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfb0bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 -- 172.21.15.52:0/951930204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3cfb0c4a0 con 0x55f3cfaeff60 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273e2eb700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfb0bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:31.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f273d2e9700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfb0b6a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2728099ab0 tx=0x7f272809b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c0c3d201ed6ab8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f27217fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f27217fa700 1 -- 172.21.15.52:0/951930204 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f272809fbb0 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f27217fa700 1 -- 172.21.15.52:0/951930204 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2728009d50 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f272800e350 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f27217fa700 1 -- 172.21.15.52:0/951930204 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f27280a4470 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f274a5d1b80 10 client.17970 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.471+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f3cfb6ec70 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f27217fa700 1 -- 172.21.15.52:0/951930204 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f272800e350 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f27217fa700 1 -- 172.21.15.52:0/951930204 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2728097550 con 0x55f3cfaeff60 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f274a5d1b80 10 client.17970 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f274a5d1b80 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f274a5d1b80 20 client.17970 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:31.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb6efe0 0x55f3cfb713d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:31.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f274a5d1b80 4 client.17970 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:20:31.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f3cfa432d0 con 0x55f3cfb6efe0 2024-06-02T23:20:31.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f273daea700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb6efe0 0x55f3cfb713d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:31.491 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:20:31.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f27217fa700 10 client.17970 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:31.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.472+0000 7f273daea700 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb6efe0 0x55f3cfb713d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=939c5a132d4c2572 server_cookie=732f0ec9e02d62c9 in_seq=0 out_seq=0 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f27217fa700 1 -- 172.21.15.52:0/951930204 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f3cfa432d0 con 0x55f3cfb6efe0 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f27217fa700 10 client.17970 handle_command_reply: tid=0 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 client.17970 shutdown 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 msgr2=0x55f3cfb0b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfb0b6a0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f2728099ab0 tx=0x7f272809b980 comp rx=0 tx=0).stop 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 shutdown_connections 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f3cfb6efe0 0x55f3cfb713d0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f3cfaf0880 0x55f3cfb0bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3cfaea7a0 0x55f3cfafcd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 --2- 172.21.15.52:0/951930204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f3cfaeff60 0x55f3cfb0b6a0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f27217fa700 0 client.17970 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 >> 172.21.15.52:0/951930204 conn(0x55f3cfa17080 msgr2=0x55f3cfaf5950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 shutdown_connections 2024-06-02T23:20:31.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 1 -- 172.21.15.52:0/951930204 wait complete. 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.473+0000 7f274a5d1b80 20 client.17970 trim_cache size 0 max 16384 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27240a2c20 msgr2=0x7f27240a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27240a2c20 0x7f27240a50e0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f27340059d0 tx=0x7f2734005940 comp rx=0 tx=0).stop 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 msgr2=0x7f2738074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 0x7f2738074cd0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f272800d980 tx=0x7f272800dd40 comp rx=0 tx=0).stop 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 shutdown_connections 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f27240a2c20 0x7f27240a50e0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2738107860 0x7f2738075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2738102770 0x7f2738074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 --2- 172.21.15.52:0/4040104712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2738103120 0x7f2738074cd0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 >> 172.21.15.52:0/4040104712 conn(0x7f2738079600 msgr2=0x7f2738100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 shutdown_connections 2024-06-02T23:20:31.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:31.474+0000 7f273fd4e700 1 -- 172.21.15.52:0/4040104712 wait complete. 2024-06-02T23:20:31.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.78 MB/sec execute 340 sec latency 2951.187 ms 2024-06-02T23:20:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:31 smithi123 ceph-mon[35977]: pgmap v690: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 904 KiB/s wr, 116 op/s 2024-06-02T23:20:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:31 smithi174 ceph-mon[35994]: pgmap v690: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 904 KiB/s wr, 116 op/s 2024-06-02T23:20:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:31 smithi052 ceph-mon[32340]: pgmap v690: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 166 KiB/s rd, 904 KiB/s wr, 116 op/s 2024-06-02T23:20:32.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.78 MB/sec execute 341 sec latency 3951.289 ms 2024-06-02T23:20:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 -- 172.21.15.52:0/3630060188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 msgr2=0x7f36bc107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 --2- 172.21.15.52:0/3630060188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc107240 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009900 tx=0x7f36b002a410 comp rx=0 tx=0).stop 2024-06-02T23:20:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 -- 172.21.15.52:0/3630060188 shutdown_connections 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 --2- 172.21.15.52:0/3630060188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 0x7f36bc109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 --2- 172.21.15.52:0/3630060188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc107240 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 --2- 172.21.15.52:0/3630060188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36bc102520 0x7f36bc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 -- 172.21.15.52:0/3630060188 >> 172.21.15.52:0/3630060188 conn(0x7f36bc0fbe70 msgr2=0x7f36bc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 -- 172.21.15.52:0/3630060188 shutdown_connections 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 -- 172.21.15.52:0/3630060188 wait complete. 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.915+0000 7f36c17c1640 1 Processor -- start 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 -- start start 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36bc102520 0x7f36bc199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 0x7f36bc1a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc10d530 con 0x7f36bc104e50 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc10d3b0 con 0x7f36bc102520 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36c17c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc10d6b0 con 0x7f36bc107780 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57646/0 (socket says 172.21.15.52:57646) 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1972853724 learned_addr learned my addr 172.21.15.52:0/1972853724 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 0x7f36bc1a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36baffd640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36bc102520 0x7f36bc199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1972853724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 msgr2=0x7f36bc1a1370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 0x7f36bc1a1370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1972853724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36bc102520 msgr2=0x7f36bc199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36bc102520 0x7f36bc199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1972853724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc1a19e0 con 0x7f36bc104e50 2024-06-02T23:20:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.916+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 0x7f36bc1a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.917+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc19a300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36b00314e0 tx=0x7f36b0031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.917+0000 7f36a3fff640 1 -- 172.21.15.52:0/1972853724 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b003f030 con 0x7f36bc104e50 2024-06-02T23:20:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.917+0000 7f36a3fff640 1 -- 172.21.15.52:0/1972853724 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b00044b0 con 0x7f36bc104e50 2024-06-02T23:20:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.917+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc1a1c70 con 0x7f36bc104e50 2024-06-02T23:20:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.917+0000 7f36a3fff640 1 -- 172.21.15.52:0/1972853724 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0035070 con 0x7f36bc104e50 2024-06-02T23:20:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.917+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1a21b0 con 0x7f36bc104e50 2024-06-02T23:20:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.919+0000 7f36a3fff640 1 -- 172.21.15.52:0/1972853724 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f36b0042030 con 0x7f36bc104e50 2024-06-02T23:20:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.919+0000 7f36c45aa740 1 Processor -- start 2024-06-02T23:20:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.919+0000 7f36c45aa740 1 -- start start 2024-06-02T23:20:32.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 0x559917b13430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13a00 0x559917b18500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36c45aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559917a764f0 con 0x559917b13010 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36c45aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559917a14870 con 0x559917b18b70 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36c45aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559917a724b0 con 0x559917b13a00 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 0x559917b13430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 0x559917b13430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57654/0 (socket says 172.21.15.52:57654) 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1930697011 learned_addr learned my addr 172.21.15.52:0/1930697011 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36a3fff640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36a40a2cb0 0x7f36a40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36baffd640 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13a00 0x559917b18500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36a40a2cb0 0x7f36a40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36a3fff640 1 -- 172.21.15.52:0/1972853724 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36b00ef640 con 0x7f36bc104e50 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1930697011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13a00 msgr2=0x559917b18500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13a00 0x559917b18500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1930697011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 msgr2=0x559917b1b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1930697011 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559917a309b0 con 0x559917b13010 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.920+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 0x559917b13430 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f36b4014a90 tx=0x7f36b4014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec73f295aeaa6464 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.921+0000 7f36a17fa640 1 -- 172.21.15.52:0/1930697011 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4014530 con 0x559917b13010 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.921+0000 7f36a17fa640 1 -- 172.21.15.52:0/1930697011 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b4023070 con 0x559917b13010 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.921+0000 7f36a17fa640 1 -- 172.21.15.52:0/1930697011 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400cce0 con 0x559917b13010 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.921+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36a40a2cb0 0x7f36a40a5170 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36b4022000 tx=0x7f36b4020000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.921+0000 7f36c45aa740 1 -- 172.21.15.52:0/1930697011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 msgr2=0x559917b13430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.921+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 0x559917b13430 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f36b4014a90 tx=0x7f36b4014f50 comp rx=0 tx=0).stop 2024-06-02T23:20:32.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- 172.21.15.52:0/1930697011 shutdown_connections 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13a00 0x559917b18500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1930697011 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13010 0x559917b13430 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- 172.21.15.52:0/1930697011 >> 172.21.15.52:0/1930697011 conn(0x559917b0e680 msgr2=0x559917b0ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- 172.21.15.52:0/1930697011 shutdown_connections 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- 172.21.15.52:0/1930697011 wait complete. 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- start start 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 Processor -- start 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b1d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b24830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559917b1bb20 con 0x559917b18b70 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599179ffae0 con 0x559917b13a00 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559917b1bca0 con 0x559917b13010 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36c45aa740 10 client.0 _ll_register_callbacks cb 0x559917a389b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:32.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b24830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b24830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57664/0 (socket says 172.21.15.52:57664) 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1447556759 learned_addr learned my addr 172.21.15.52:0/1447556759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b1d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.922+0000 7f36baffd640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1447556759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 msgr2=0x559917b1d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b1d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 msgr2=0x559917b1d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b1d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36bb7fe640 1 -- 172.21.15.52:0/1447556759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55991778a210 con 0x559917b18b70 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b1d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b24830 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f36b4004a30 tx=0x7f36b4004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3f92ca727e3b6c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f369eff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:32.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f369eff5640 1 -- 172.21.15.52:0/1447556759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4037060 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f369eff5640 1 -- 172.21.15.52:0/1447556759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36b4014510 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b4023070 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f369eff5640 1 -- 172.21.15.52:0/1447556759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36b4009900 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36c45aa740 10 client.17982 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559917856620 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.923+0000 7f369eff5640 1 -- 172.21.15.52:0/1447556759 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b4023070 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f369eff5640 1 -- 172.21.15.52:0/1447556759 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f36b400f040 con 0x559917b18b70 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36c45aa740 10 client.17982 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36c45aa740 10 client.17982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36c45aa740 20 client.17982 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b2b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36c45aa740 4 client.17982 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:32.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559917a89f10 con 0x559917b28e10 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.924+0000 7f36baffd640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b2b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.925+0000 7f36baffd640 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b2b200 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed1a5dc3079da8b6 server_cookie=5c0d054145756e74 in_seq=0 out_seq=0 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.925+0000 7f369eff5640 10 client.17982 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.925+0000 7f369eff5640 1 -- 172.21.15.52:0/1447556759 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559917a89f10 con 0x559917b28e10 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.925+0000 7f369eff5640 10 client.17982 handle_command_reply: tid=0 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 client.17982 shutdown 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 msgr2=0x559917b24830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b24830 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f36b4004a30 tx=0x7f36b4004a60 comp rx=0 tx=0).stop 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 shutdown_connections 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b2b200 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b24830 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b1d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f369eff5640 0 client.17982 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:32.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1447556759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b1d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 >> 172.21.15.52:0/1447556759 conn(0x559917b0e680 msgr2=0x559917b0d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 shutdown_connections 2024-06-02T23:20:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 1 -- 172.21.15.52:0/1447556759 wait complete. 2024-06-02T23:20:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.926+0000 7f36c45aa740 20 client.17982 trim_cache size 0 max 16384 2024-06-02T23:20:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.941+0000 7f36c45aa740 1 Processor -- start 2024-06-02T23:20:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.941+0000 7f36c45aa740 1 -- start start 2024-06-02T23:20:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.941+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b0ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 0x559917b247f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36c45aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1aaeb0 con 0x559917b13a00 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36c45aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1ac9f0 con 0x559917b18b70 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36c45aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1ab580 con 0x559917b13010 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 0x559917b247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 0x559917b247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57680/0 (socket says 172.21.15.52:57680) 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1506070128 learned_addr learned my addr 172.21.15.52:0/1506070128 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36baffd640 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b0ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1506070128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 msgr2=0x559917b0ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b0ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1506070128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 msgr2=0x559917b1d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 -- 172.21.15.52:0/1506070128 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559917b2b7e0 con 0x559917b13a00 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36baffd640 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 0x559917b247f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f36b0005940 tx=0x7f36b008ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fbfdfe25d0a0e61 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36a0ff9640 1 -- 172.21.15.52:0/1506070128 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0049040 con 0x559917b13a00 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36a0ff9640 1 -- 172.21.15.52:0/1506070128 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b0086f50 con 0x559917b13a00 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.942+0000 7f36a0ff9640 1 -- 172.21.15.52:0/1506070128 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b00c4ae0 con 0x559917b13a00 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 -- 172.21.15.52:0/1506070128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 msgr2=0x559917b247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 0x559917b247f0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f36b0005940 tx=0x7f36b008ef30 comp rx=0 tx=0).stop 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 -- 172.21.15.52:0/1506070128 shutdown_connections 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b18b70 0x559917b1d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b13a00 0x559917b247f0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 --2- 172.21.15.52:0/1506070128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b0ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 -- 172.21.15.52:0/1506070128 >> 172.21.15.52:0/1506070128 conn(0x559917b0e680 msgr2=0x559917b182f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 -- 172.21.15.52:0/1506070128 shutdown_connections 2024-06-02T23:20:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 -- 172.21.15.52:0/1506070128 wait complete. 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 -- start start 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.943+0000 7f36c45aa740 1 Processor -- start 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b245e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1ac320 con 0x559917b18b70 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1abc50 con 0x559917b13a00 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1a8e90 con 0x559917b13010 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 10 client.0 _ll_register_callbacks cb 0x7f36b401ef50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b20b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57688/0 (socket says 172.21.15.52:57688) 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 -- 172.21.15.52:0/105095644 learned_addr learned my addr 172.21.15.52:0/105095644 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36baffd640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b245e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 -- 172.21.15.52:0/105095644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 msgr2=0x559917b240a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b240a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 -- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 msgr2=0x559917b245e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b245e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 -- 172.21.15.52:0/105095644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559917b211b0 con 0x559917b18b70 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36ba7fc640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36bb7fe640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b20b40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009ac0 tx=0x7f36b4014720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a65ec9bc991826 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f369eff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f369eff5640 1 -- 172.21.15.52:0/105095644 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4042030 con 0x559917b18b70 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f369eff5640 1 -- 172.21.15.52:0/105095644 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36b402f070 con 0x559917b18b70 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.944+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b0086f50 con 0x559917b18b70 2024-06-02T23:20:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f369eff5640 1 -- 172.21.15.52:0/105095644 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f36b4037040 con 0x559917b18b70 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 10 client.17988 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559917b925b0 con 0x559917b18b70 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f369eff5640 1 -- 172.21.15.52:0/105095644 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f36b0086f50 con 0x559917b18b70 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f369eff5640 1 -- 172.21.15.52:0/105095644 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f36b400cab0 con 0x559917b18b70 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 10 client.17988 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 20 client.17988 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b94f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 4 client.17988 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559917a83c20 con 0x559917b28e10 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.945+0000 7f36baffd640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b94f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.946+0000 7f36baffd640 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b94f70 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5942095e9c2ac server_cookie=1f612b14cb2294e9 in_seq=0 out_seq=0 2024-06-02T23:20:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.946+0000 7f369eff5640 10 client.17988 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f369eff5640 1 -- 172.21.15.52:0/105095644 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559917a83c20 con 0x559917b28e10 2024-06-02T23:20:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f369eff5640 10 client.17988 handle_command_reply: tid=0 2024-06-02T23:20:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 client.17988 shutdown 2024-06-02T23:20:32.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 msgr2=0x559917b20b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b20b40 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009ac0 tx=0x7f36b4014720 comp rx=0 tx=0).stop 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 shutdown_connections 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917b28e10 0x559917b94f70 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917b18b70 0x559917b20b40 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917b13a00 0x559917b245e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 --2- 172.21.15.52:0/105095644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917b13010 0x559917b240a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f369eff5640 0 client.17988 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 >> 172.21.15.52:0/105095644 conn(0x559917b0e680 msgr2=0x559917b18220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 shutdown_connections 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 1 -- 172.21.15.52:0/105095644 wait complete. 2024-06-02T23:20:32.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c45aa740 20 client.17988 trim_cache size 0 max 16384 2024-06-02T23:20:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:20:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:20:32.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36a40a2cb0 msgr2=0x7f36a40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36a40a2cb0 0x7f36a40a5170 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f36b4022000 tx=0x7f36b4020000 comp rx=0 tx=0).stop 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 msgr2=0x7f36bc19a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc19a300 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f36b00314e0 tx=0x7f36b0031a30 comp rx=0 tx=0).stop 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 shutdown_connections 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36a40a2cb0 0x7f36a40a5170 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36bc107780 0x7f36bc1a1370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f36bc104e50 0x7f36bc19a300 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 --2- 172.21.15.52:0/1972853724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f36bc102520 0x7f36bc199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.967+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 >> 172.21.15.52:0/1972853724 conn(0x7f36bc0fbe70 msgr2=0x7f36bc101cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.968+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 shutdown_connections 2024-06-02T23:20:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:32.968+0000 7f36c17c1640 1 -- 172.21.15.52:0/1972853724 wait complete. 2024-06-02T23:20:33.103 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:20:33.103 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:20:33.358 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:33.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.77 MB/sec execute 342 sec latency 4951.416 ms 2024-06-02T23:20:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:33 smithi123 ceph-mon[35977]: pgmap v691: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 943 KiB/s wr, 130 op/s 2024-06-02T23:20:33.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:33 smithi052 ceph-mon[32340]: pgmap v691: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 943 KiB/s wr, 130 op/s 2024-06-02T23:20:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:33 smithi174 ceph-mon[35994]: pgmap v691: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 943 KiB/s wr, 130 op/s 2024-06-02T23:20:34.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.76 MB/sec execute 343 sec latency 5951.454 ms 2024-06-02T23:20:35.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 -- 172.21.15.52:0/1567127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c107400 msgr2=0x7f8e4c1077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1567127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c107400 0x7f8e4c1077e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009990 tx=0x7f8e3c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 -- 172.21.15.52:0/1567127697 shutdown_connections 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1567127697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e4c101d50 0x7f8e4c10e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1567127697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c107400 0x7f8e4c1077e0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1567127697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c101390 0x7f8e4c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.001+0000 7f8e528b9700 1 -- 172.21.15.52:0/1567127697 >> 172.21.15.52:0/1567127697 conn(0x7f8e4c078340 msgr2=0x7f8e4c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.002+0000 7f8e528b9700 1 -- 172.21.15.52:0/1567127697 shutdown_connections 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.002+0000 7f8e528b9700 1 -- 172.21.15.52:0/1567127697 wait complete. 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.002+0000 7f8e528b9700 1 Processor -- start 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.002+0000 7f8e528b9700 1 -- start start 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.002+0000 7f8e528b9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 0x7f8e4c19e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.002+0000 7f8e528b9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e4c101d50 0x7f8e4c19ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e528b9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c107400 0x7f8e4c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e528b9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1115f0 con 0x7f8e4c101390 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e528b9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c111470 con 0x7f8e4c101d50 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e528b9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c111770 con 0x7f8e4c107400 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 0x7f8e4c19e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 0x7f8e4c19e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50696/0 (socket says 172.21.15.52:50696) 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 -- 172.21.15.52:0/1982820310 learned_addr learned my addr 172.21.15.52:0/1982820310 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e43fff700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e4c101d50 0x7f8e4c19ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e50e56700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c107400 0x7f8e4c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 -- 172.21.15.52:0/1982820310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c107400 msgr2=0x7f8e4c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c107400 0x7f8e4c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 -- 172.21.15.52:0/1982820310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e4c101d50 msgr2=0x7f8e4c19ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e4c101d50 0x7f8e4c19ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 -- 172.21.15.52:0/1982820310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1991c0 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e50e56700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c107400 0x7f8e4c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 0x7f8e4c19e8a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c005910 tx=0x7f8e3c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e49ffb700 1 -- 172.21.15.52:0/1982820310 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c027070 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e49ffb700 1 -- 172.21.15.52:0/1982820310 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e3c003f10 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c199450 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.003+0000 7f8e49ffb700 1 -- 172.21.15.52:0/1982820310 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c022d00 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.004+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a5770 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.006+0000 7f8e49ffb700 1 -- 172.21.15.52:0/1982820310 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8e3c02a070 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.006+0000 7f8e49ffb700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e380a2ca0 0x7f8e380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e43fff700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e380a2ca0 0x7f8e380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e49ffb700 1 -- 172.21.15.52:0/1982820310 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e3c0cfe00 con 0x7f8e4c101390 2024-06-02T23:20:35.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 Processor -- start 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e43fff700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e380a2ca0 0x7f8e380a5160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c078940 tx=0x7f8e34009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 -- start start 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69e43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561eb6935890 con 0x561eb69e3fa0 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561eb68dd7a0 con 0x561eb69e9fe0 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561eb6931850 con 0x561eb69e96c0 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.007+0000 7f8e50e56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69e43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69e43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50700/0 (socket says 172.21.15.52:50700) 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 -- 172.21.15.52:0/322036335 learned_addr learned my addr 172.21.15.52:0/322036335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:35.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e43fff700 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 -- 172.21.15.52:0/322036335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 msgr2=0x561eb69e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 -- 172.21.15.52:0/322036335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 msgr2=0x561eb69e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 -- 172.21.15.52:0/322036335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb68d3fd0 con 0x561eb69e3fa0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e43fff700 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e50e56700 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69e43c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009d50 tx=0x7f8e440077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f1ee2d6ad02ad42 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e427fc700 1 -- 172.21.15.52:0/322036335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44019070 con 0x561eb69e3fa0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e427fc700 1 -- 172.21.15.52:0/322036335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e44014ab0 con 0x561eb69e3fa0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.008+0000 7f8e427fc700 1 -- 172.21.15.52:0/322036335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44013980 con 0x561eb69e3fa0 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/322036335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 msgr2=0x561eb69e43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69e43c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009d50 tx=0x7f8e440077e0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/322036335 shutdown_connections 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69e43c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/322036335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/322036335 >> 172.21.15.52:0/322036335 conn(0x561eb690ed40 msgr2=0x561eb69de8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/322036335 shutdown_connections 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/322036335 wait complete. 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.009+0000 7f8e5d13cb80 1 -- start start 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 Processor -- start 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69efd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69ed050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69ed590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561eb69f0a50 con 0x561eb69e3fa0 2024-06-02T23:20:35.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561eb68d5c70 con 0x561eb69e9fe0 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561eb69f0bd0 con 0x561eb69e96c0 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e5d13cb80 10 client.0 _ll_register_callbacks cb 0x561eb68cf9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69efd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69efd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50716/0 (socket says 172.21.15.52:50716) 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 -- 172.21.15.52:0/4006634890 learned_addr learned my addr 172.21.15.52:0/4006634890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e43fff700 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69ed050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 -- 172.21.15.52:0/4006634890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 msgr2=0x561eb69ed050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69ed050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 -- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 msgr2=0x561eb69ed590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69ed590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 -- 172.21.15.52:0/4006634890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb6884950 con 0x561eb69e3fa0 2024-06-02T23:20:35.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69efd50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c000c00 tx=0x7f8e3c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb18a8a2136f681d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.010+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/4006634890 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c09cdc0 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/4006634890 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e3c09a9c0 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e44014ab0 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/4006634890 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c0870d0 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 10 client.18000 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561eb69fb380 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/4006634890 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e44014ab0 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/4006634890 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8e3c003a60 con 0x561eb69e3fa0 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 10 client.18000 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 20 client.18000 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb69fd8f0 0x561eb69ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 4 client.18000 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561eb6942d90 con 0x561eb69fd8f0 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.011+0000 7f8e43fff700 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb69fd8f0 0x561eb69ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.012+0000 7f8e43fff700 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb69fd8f0 0x561eb69ffd50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d26a175c9ca27e8f server_cookie=d6645ece558a7f98 in_seq=0 out_seq=0 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.012+0000 7f8e2b7fe700 10 client.18000 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.012+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/4006634890 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561eb6942d90 con 0x561eb69fd8f0 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.012+0000 7f8e2b7fe700 10 client.18000 handle_command_reply: tid=0 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.012+0000 7f8e5d13cb80 1 client.18000 shutdown 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 msgr2=0x561eb69efd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69efd50 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c000c00 tx=0x7f8e3c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 shutdown_connections 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb69fd8f0 0x561eb69ffd50 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69ed590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69efd50 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/4006634890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69ed050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 >> 172.21.15.52:0/4006634890 conn(0x561eb690ed40 msgr2=0x561eb69e2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e2b7fe700 0 client.18000 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 shutdown_connections 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/4006634890 wait complete. 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.013+0000 7f8e5d13cb80 20 client.18000 trim_cache size 0 max 16384 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 Processor -- start 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 -- start start 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e3fa0 0x561eb69e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 0x561eb69eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1aca20 con 0x561eb69e96c0 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1ae560 con 0x561eb69e9fe0 2024-06-02T23:20:35.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1ad0f0 con 0x561eb69e3fa0 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 0x561eb69eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 0x561eb69eac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50728/0 (socket says 172.21.15.52:50728) 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 -- 172.21.15.52:0/315861443 learned_addr learned my addr 172.21.15.52:0/315861443 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e3fa0 0x561eb69e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 -- 172.21.15.52:0/315861443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e3fa0 msgr2=0x561eb69e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e3fa0 0x561eb69e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 -- 172.21.15.52:0/315861443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 msgr2=0x561eb69d8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e50e56700 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69d8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e43fff700 1 -- 172.21.15.52:0/315861443 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561eb69df5a0 con 0x561eb69e96c0 2024-06-02T23:20:35.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.034+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e3fa0 0x561eb69e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e43fff700 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 0x561eb69eac50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x561eb69ee620 tx=0x7f8e34013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f26c4e757f7c306 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e41ffb700 1 -- 172.21.15.52:0/315861443 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3401b040 con 0x561eb69e96c0 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e41ffb700 1 -- 172.21.15.52:0/315861443 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e34015b70 con 0x561eb69e96c0 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e41ffb700 1 -- 172.21.15.52:0/315861443 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e34029aa0 con 0x561eb69e96c0 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/315861443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 msgr2=0x561eb69eac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 0x561eb69eac50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x561eb69ee620 tx=0x7f8e34013000 comp rx=0 tx=0).stop 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/315861443 shutdown_connections 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e3fa0 0x561eb69e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/315861443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e96c0 0x561eb69eac50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/315861443 >> 172.21.15.52:0/315861443 conn(0x561eb690ed40 msgr2=0x561eb69d7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.035+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/315861443 shutdown_connections 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/315861443 wait complete. 2024-06-02T23:20:35.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 -- start start 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 Processor -- start 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69f6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1ade90 con 0x561eb69e3fa0 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1ad7c0 con 0x561eb69e9fe0 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1aa9c0 con 0x561eb69e96c0 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e5d13cb80 10 client.0 _ll_register_callbacks cb 0x561eb68cf9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69f6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69f6070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50740/0 (socket says 172.21.15.52:50740) 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.036+0000 7f8e4bfff700 1 -- 172.21.15.52:0/204750929 learned_addr learned my addr 172.21.15.52:0/204750929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e50e56700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e43fff700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e4bfff700 1 -- 172.21.15.52:0/204750929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 msgr2=0x561eb69f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e4bfff700 1 -- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 msgr2=0x561eb69f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e4bfff700 1 -- 172.21.15.52:0/204750929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561eb69f34d0 con 0x561eb69e3fa0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e50e56700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e4bfff700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69f6070 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c099090 tx=0x7f8e3c0a50f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62d3f77788e95ca9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e2b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/204750929 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c080060 con 0x561eb69e3fa0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/204750929 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8e3c022470 con 0x561eb69e3fa0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e34015b70 con 0x561eb69e3fa0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/204750929 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c09cdc0 con 0x561eb69e3fa0 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e5d13cb80 10 client.18006 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:35.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561eb69f34d0 con 0x561eb69e3fa0 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.037+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/204750929 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8e34015b70 con 0x561eb69e3fa0 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/204750929 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8e3c09b580 con 0x561eb69e3fa0 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e5d13cb80 10 client.18006 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e5d13cb80 10 client.18006 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e5d13cb80 20 client.18006 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb6a69240 0x561eb6a6b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e5d13cb80 4 client.18006 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561eb693d310 con 0x561eb6a69240 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e43fff700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb6a69240 0x561eb6a6b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e2b7fe700 10 client.18006 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.038+0000 7f8e43fff700 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb6a69240 0x561eb6a6b630 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=297287ea96a8f719 server_cookie=e5f9e7625ba7c65c in_seq=0 out_seq=0 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.039+0000 7f8e2b7fe700 1 -- 172.21.15.52:0/204750929 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561eb693d310 con 0x561eb6a69240 2024-06-02T23:20:35.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.039+0000 7f8e2b7fe700 10 client.18006 handle_command_reply: tid=0 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.039+0000 7f8e5d13cb80 1 client.18006 shutdown 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 msgr2=0x561eb69f6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69f6070 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c099090 tx=0x7f8e3c0a50f0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 shutdown_connections 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561eb6a69240 0x561eb6a6b630 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561eb69e9fe0 0x561eb69f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561eb69e3fa0 0x561eb69f6070 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e2b7fe700 0 client.18006 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 --2- 172.21.15.52:0/204750929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561eb69e96c0 0x561eb69f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 >> 172.21.15.52:0/204750929 conn(0x561eb690ed40 msgr2=0x561eb69e81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:35.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 shutdown_connections 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "6beecc3c-192a-40f3-9599-8f625640a6bd", 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 1 -- 172.21.15.52:0/204750929 wait complete. 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e5d13cb80 20 client.18006 trim_cache size 0 max 16384 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e380a2ca0 msgr2=0x7f8e380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e380a2ca0 0x7f8e380a5160 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c078940 tx=0x7f8e34009190 comp rx=0 tx=0).stop 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 msgr2=0x7f8e4c19e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 0x7f8e4c19e8a0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c005910 tx=0x7f8e3c0047b0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 shutdown_connections 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8e380a2ca0 0x7f8e380a5160 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8e4c107400 0x7f8e4c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8e4c101390 0x7f8e4c19e8a0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.040+0000 7f8e528b9700 1 --2- 172.21.15.52:0/1982820310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e4c101d50 0x7f8e4c19ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.041+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 >> 172.21.15.52:0/1982820310 conn(0x7f8e4c078340 msgr2=0x7f8e4c1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.041+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 shutdown_connections 2024-06-02T23:20:35.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:35.041+0000 7f8e528b9700 1 -- 172.21.15.52:0/1982820310 wait complete. 2024-06-02T23:20:35.486 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:20:35.054+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000229 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX/ auth{1=1,2=1,3=1,4=1} v26186 ap=3 f(v1509 m2024-06-02T23:20:28.339693+0000 20=20+0) n(v37 rc2024-06-02T23:20:28.339693+0000 b5201920 21=20+1) (inest sync->mix g=1,2,3,4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:312 nref=3)) caps={15216=pAsLsXs/-@27508} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ac2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000229:[<0x10000000009/PARADOX v108>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x10000000003/client.0 v6>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v22>,<0x10000000001/sv_0 v45>,<0x10000000000/qa v32>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000229:[<0x10000000009/PARADOX v26186>,<0x10000000008/~dmtmp v24432>,<0x10000000007/client0 v863>,<0x10000000006/clients v848>,<0x10000000005/tmp v823>,<0x10000000003/client.0 v798>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v790>,<0x10000000001/sv_0 v802>,<0x10000000000/qa v774>,<0x1/volumes v785>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:20:28.329107+0000 21=21+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:20:28.339693+0000 b5201920 21=20+1)","memory_value.dirstat":"f(v1509 m2024-06-02T23:20:28.339693+0000 20=20+0)","memory_value.rstat":"n(v37 rc2024-06-02T23:20:28.339693+0000 b5201920 21=20+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:20:35.486 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:20:35.056+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000225 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM/ auth{3=1} v26182 ap=3 f(v3182 m2024-06-02T23:20:28.288224+0000 3=3+0) n(v35 rc2024-06-02T23:20:28.288224+0000 b780288 4=3+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:313 nref=3)) caps={15216=pAsLsXs/-@32097} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ac1600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000225:[<0x10000000009/PM v66>,<0x10000000008/~dmtmp v30>,<0x10000000007/client0 v14>,<0x10000000006/clients v4>,<0x10000000005/tmp v4>,<0x10000000003/client.0 v6>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v22>,<0x10000000001/sv_0 v45>,<0x10000000000/qa v32>,<0x1/volumes v32>]//[]","memoryvalue":"(2)0x10000000225:[<0x10000000009/PM v26182>,<0x10000000008/~dmtmp v24432>,<0x10000000007/client0 v863>,<0x10000000006/clients v848>,<0x10000000005/tmp v823>,<0x10000000003/client.0 v798>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v790>,<0x10000000001/sv_0 v802>,<0x10000000000/qa v774>,<0x1/volumes v785>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:20:27.886858+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:20:28.288224+0000 b780288 4=3+1)","memory_value.dirstat":"f(v3182 m2024-06-02T23:20:28.288224+0000 3=3+0)","memory_value.rstat":"n(v35 rc2024-06-02T23:20:28.288224+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:20:35.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.76 MB/sec execute 344 sec latency 6951.566 ms 2024-06-02T23:20:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-mon[35977]: pgmap v692: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 838 KiB/s wr, 118 op/s 2024-06-02T23:20:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:35 smithi174 ceph-mon[35994]: pgmap v692: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 838 KiB/s wr, 118 op/s 2024-06-02T23:20:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:35 smithi052 ceph-mon[32340]: pgmap v692: 97 pgs: 97 active+clean; 76 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 838 KiB/s wr, 118 op/s 2024-06-02T23:20:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:35.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:35.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:20:36.199 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:36.572 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:20:36.572 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.572 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:36.572 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.573 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:36 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.75 MB/sec execute 345 sec latency 7951.650 ms 2024-06-02T23:20:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:20:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:36.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:20:36.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:36 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:20:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:36.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000229 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000225 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/PM) see mds.b log and `damage ls` output for details 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:36.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:36 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:20:37.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:37 smithi052 ceph-mon[32340]: pgmap v693: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 802 KiB/s wr, 125 op/s 2024-06-02T23:20:37.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.75 MB/sec execute 346 sec latency 8951.697 ms 2024-06-02T23:20:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:37 smithi123 ceph-mon[35977]: pgmap v693: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 802 KiB/s wr, 125 op/s 2024-06-02T23:20:37.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 -- 172.21.15.52:0/1571836111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101d50 msgr2=0x7f350c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1571836111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101d50 0x7f350c10e240 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3504009990 tx=0x7f3504009ca0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 -- 172.21.15.52:0/1571836111 shutdown_connections 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1571836111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101d50 0x7f350c10e240 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1571836111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f350c107400 0x7f350c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1571836111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f350c101390 0x7f350c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 -- 172.21.15.52:0/1571836111 >> 172.21.15.52:0/1571836111 conn(0x7f350c078340 msgr2=0x7f350c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 -- 172.21.15.52:0/1571836111 shutdown_connections 2024-06-02T23:20:37.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.875+0000 7f3512e4f700 1 -- 172.21.15.52:0/1571836111 wait complete. 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 Processor -- start 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 -- start start 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 0x7f350c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f350c101d50 0x7f350c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f350c107400 0x7f350c198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f350c111550 con 0x7f350c101390 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1113d0 con 0x7f350c107400 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.876+0000 7f3512e4f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1116d0 con 0x7f350c101d50 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 0x7f350c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 0x7f350c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50760/0 (socket says 172.21.15.52:50760) 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 -- 172.21.15.52:0/1098370243 learned_addr learned my addr 172.21.15.52:0/1098370243 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f350bfff700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f350c101d50 0x7f350c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f35113ec700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f350c107400 0x7f350c198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 -- 172.21.15.52:0/1098370243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f350c101d50 msgr2=0x7f350c19ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f350c101d50 0x7f350c19ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 -- 172.21.15.52:0/1098370243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f350c107400 msgr2=0x7f350c198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f350c107400 0x7f350c198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 -- 172.21.15.52:0/1098370243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f350c1990b0 con 0x7f350c101390 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3510beb700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 0x7f350c19e740 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00d980 tx=0x7f34fc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3509ffb700 1 -- 172.21.15.52:0/1098370243 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc016070 con 0x7f350c101390 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3509ffb700 1 -- 172.21.15.52:0/1098370243 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34fc004830 con 0x7f350c101390 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f350c1993a0 con 0x7f350c101390 2024-06-02T23:20:37.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.877+0000 7f3509ffb700 1 -- 172.21.15.52:0/1098370243 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc005530 con 0x7f350c101390 2024-06-02T23:20:37.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.878+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f350c1a5610 con 0x7f350c101390 2024-06-02T23:20:37.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.880+0000 7f3509ffb700 1 -- 172.21.15.52:0/1098370243 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f34fc0049e0 con 0x7f350c101390 2024-06-02T23:20:37.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.880+0000 7f3509ffb700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f34f40a2c20 0x7f34f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.880+0000 7f350bfff700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f34f40a2c20 0x7f34f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.880+0000 7f3509ffb700 1 -- 172.21.15.52:0/1098370243 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f34fc0ced00 con 0x7f350c101390 2024-06-02T23:20:37.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.881+0000 7f350bfff700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f34f40a2c20 0x7f34f40a50e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f350c078940 tx=0x7f3500009180 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.881+0000 7f351d6d2b80 1 Processor -- start 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.881+0000 7f351d6d2b80 1 -- start start 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af8f850 0x562b3af8fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af90170 0x562b3af94010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af94990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562b3ae7df60 con 0x562b3af94550 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3aedc850 con 0x562b3af90170 2024-06-02T23:20:37.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b3ae44460 con 0x562b3af8f850 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af94990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af94990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50774/0 (socket says 172.21.15.52:50774) 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 -- 172.21.15.52:0/95902854 learned_addr learned my addr 172.21.15.52:0/95902854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f3510beb700 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af8f850 0x562b3af8fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f350bfff700 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af90170 0x562b3af94010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 -- 172.21.15.52:0/95902854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af8f850 msgr2=0x562b3af8fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af8f850 0x562b3af8fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 -- 172.21.15.52:0/95902854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af90170 msgr2=0x562b3af94010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af90170 0x562b3af94010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 -- 172.21.15.52:0/95902854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3ae2b8d0 con 0x562b3af94550 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f350bfff700 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af90170 0x562b3af94010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.882+0000 7f35113ec700 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af94990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3504015000 tx=0x7f3504009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd1bff7f3e2220 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f34f2ffd700 1 -- 172.21.15.52:0/95902854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504027070 con 0x562b3af94550 2024-06-02T23:20:37.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f34f2ffd700 1 -- 172.21.15.52:0/95902854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f35040095c0 con 0x562b3af94550 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f34f2ffd700 1 -- 172.21.15.52:0/95902854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f350400ba30 con 0x562b3af94550 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 -- 172.21.15.52:0/95902854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 msgr2=0x562b3af94990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af94990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3504015000 tx=0x7f3504009f90 comp rx=0 tx=0).stop 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 -- 172.21.15.52:0/95902854 shutdown_connections 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af94990 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af8f850 0x562b3af8fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/95902854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af90170 0x562b3af94010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 -- 172.21.15.52:0/95902854 >> 172.21.15.52:0/95902854 conn(0x562b3aeb8080 msgr2=0x562b3af86220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 -- 172.21.15.52:0/95902854 shutdown_connections 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.883+0000 7f351d6d2b80 1 -- 172.21.15.52:0/95902854 wait complete. 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 -- start start 2024-06-02T23:20:37.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 Processor -- start 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3af9ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 0x562b3af98d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af94550 0x562b3af992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562b3ae7cfc0 con 0x562b3af90170 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3aed8810 con 0x562b3af8f850 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562b3af9c620 con 0x562b3af94550 2024-06-02T23:20:37.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f351d6d2b80 10 client.0 _ll_register_callbacks cb 0x562b3ae725b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.884+0000 7f350bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 0x562b3af98d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 0x562b3af98d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50778/0 (socket says 172.21.15.52:50778) 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 -- 172.21.15.52:0/2352880439 learned_addr learned my addr 172.21.15.52:0/2352880439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f35113ec700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af94550 0x562b3af992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f3510beb700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3af9ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 -- 172.21.15.52:0/2352880439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af94550 msgr2=0x562b3af992b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af94550 0x562b3af992b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 -- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 msgr2=0x562b3af9ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3af9ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 -- 172.21.15.52:0/2352880439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3aea2750 con 0x562b3af90170 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f350bfff700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 0x562b3af98d70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3500015820 tx=0x7f3500004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7093055ac63ed906 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f34ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f34ebfff700 1 -- 172.21.15.52:0/2352880439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500026020 con 0x562b3af90170 2024-06-02T23:20:37.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f34ebfff700 1 -- 172.21.15.52:0/2352880439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3500022830 con 0x562b3af90170 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562b3aee9d50 con 0x562b3af90170 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f34ebfff700 1 -- 172.21.15.52:0/2352880439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f350001c030 con 0x562b3af90170 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f351d6d2b80 10 client.18018 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.885+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b3afa6e30 con 0x562b3af90170 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f34ebfff700 1 -- 172.21.15.52:0/2352880439 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x562b3aee9d50 con 0x562b3af90170 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f34ebfff700 1 -- 172.21.15.52:0/2352880439 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3500007b60 con 0x562b3af90170 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f351d6d2b80 10 client.18018 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f351d6d2b80 10 client.18018 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f351d6d2b80 20 client.18018 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3afa9460 0x562b3afab850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f351d6d2b80 4 client.18018 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b3aee42d0 con 0x562b3afa9460 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.886+0000 7f3510beb700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3afa9460 0x562b3afab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.887+0000 7f34ebfff700 10 client.18018 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.887+0000 7f3510beb700 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3afa9460 0x562b3afab850 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9dfbd0e186e9214 server_cookie=8110a01cd0bf2dab in_seq=0 out_seq=0 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.887+0000 7f34ebfff700 1 -- 172.21.15.52:0/2352880439 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562b3aee42d0 con 0x562b3afa9460 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.887+0000 7f34ebfff700 10 client.18018 handle_command_reply: tid=0 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 client.18018 shutdown 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 msgr2=0x562b3af98d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 0x562b3af98d70 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f3500015820 tx=0x7f3500004b40 comp rx=0 tx=0).stop 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 shutdown_connections 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3afa9460 0x562b3afab850 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af94550 0x562b3af992b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3af9ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/2352880439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af90170 0x562b3af98d70 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 >> 172.21.15.52:0/2352880439 conn(0x562b3aeb8080 msgr2=0x562b3af95a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f34ebfff700 0 client.18018 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:37.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 shutdown_connections 2024-06-02T23:20:37.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 1 -- 172.21.15.52:0/2352880439 wait complete. 2024-06-02T23:20:37.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.888+0000 7f351d6d2b80 20 client.18018 trim_cache size 0 max 16384 2024-06-02T23:20:37.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:37 smithi174 ceph-mon[35994]: pgmap v693: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 802 KiB/s wr, 125 op/s 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 Processor -- start 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 -- start start 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 0x562b3afb1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3afb1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af94550 0x562b3af93330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1aa760 con 0x562b3af8f850 2024-06-02T23:20:37.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1ac2a0 con 0x562b3af94550 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1aae30 con 0x562b3af90170 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 0x562b3afb1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 0x562b3afb1100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50786/0 (socket says 172.21.15.52:50786) 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 -- 172.21.15.52:0/4096859713 learned_addr learned my addr 172.21.15.52:0/4096859713 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f35113ec700 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af94550 0x562b3af93330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 -- 172.21.15.52:0/4096859713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 msgr2=0x562b3afb1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f350bfff700 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3afb1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3afb1640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 -- 172.21.15.52:0/4096859713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af94550 msgr2=0x562b3af93330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af94550 0x562b3af93330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 -- 172.21.15.52:0/4096859713 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3af86700 con 0x562b3af8f850 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f35113ec700 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af94550 0x562b3af93330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.909+0000 7f3510beb700 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 0x562b3afb1100 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f34fc097f60 tx=0x7f34fc007640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4178c9d1bc31d4d0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:37.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f34f27fc700 1 -- 172.21.15.52:0/4096859713 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc09bc90 con 0x562b3af8f850 2024-06-02T23:20:37.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f34f27fc700 1 -- 172.21.15.52:0/4096859713 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34fc09b590 con 0x562b3af8f850 2024-06-02T23:20:37.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f34f27fc700 1 -- 172.21.15.52:0/4096859713 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34fc099890 con 0x562b3af8f850 2024-06-02T23:20:37.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4096859713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 msgr2=0x562b3afb1100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 0x562b3afb1100 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f34fc097f60 tx=0x7f34fc007640 comp rx=0 tx=0).stop 2024-06-02T23:20:37.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4096859713 shutdown_connections 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af94550 0x562b3af93330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af8f850 0x562b3afb1100 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4096859713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3afb1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4096859713 >> 172.21.15.52:0/4096859713 conn(0x562b3aeb8080 msgr2=0x562b3af8aae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4096859713 shutdown_connections 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.910+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4096859713 wait complete. 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 -- start start 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 Processor -- start 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3afa20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3af9d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af9d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1abbd0 con 0x562b3af94550 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1ab500 con 0x562b3af8f850 2024-06-02T23:20:37.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f350c1a8700 con 0x562b3af90170 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f351d6d2b80 10 client.0 _ll_register_callbacks cb 0x562b3ae725b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f35113ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af9d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f35113ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af9d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50788/0 (socket says 172.21.15.52:50788) 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f35113ec700 1 -- 172.21.15.52:0/4118885915 learned_addr learned my addr 172.21.15.52:0/4118885915 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f3510beb700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3afa20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.911+0000 7f350bfff700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3af9d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f35113ec700 1 -- 172.21.15.52:0/4118885915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 msgr2=0x562b3af9d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f35113ec700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3af9d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f35113ec700 1 -- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 msgr2=0x562b3afa20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f35113ec700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3afa20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f35113ec700 1 -- 172.21.15.52:0/4118885915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3af9e200 con 0x562b3af94550 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f35113ec700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af9d9c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3504000c00 tx=0x7f3504003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b5a0332b7cbe7d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f34ebfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:37.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f34ebfff700 1 -- 172.21.15.52:0/4118885915 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3504031070 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34fc09b590 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f34ebfff700 1 -- 172.21.15.52:0/4118885915 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f35040052b0 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f34ebfff700 1 -- 172.21.15.52:0/4118885915 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3504027040 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f351d6d2b80 10 client.18024 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.912+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562b3b014190 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f34ebfff700 1 -- 172.21.15.52:0/4118885915 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34fc09b590 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f34ebfff700 1 -- 172.21.15.52:0/4118885915 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f350400f580 con 0x562b3af94550 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f351d6d2b80 10 client.18024 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f351d6d2b80 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f351d6d2b80 20 client.18024 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:37.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3b0143b0 0x562b3b0167a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f351d6d2b80 4 client.18024 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562b3ae97f50 con 0x562b3b0143b0 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f3510beb700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3b0143b0 0x562b3b0167a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f34ebfff700 10 client.18024 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.913+0000 7f3510beb700 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3b0143b0 0x562b3b0167a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8947e21577b6b4d server_cookie=eb12e73f9fd01a1e in_seq=0 out_seq=0 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f34ebfff700 1 -- 172.21.15.52:0/4118885915 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x562b3ae97f50 con 0x562b3b0143b0 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f34ebfff700 10 client.18024 handle_command_reply: tid=0 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 client.18024 shutdown 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 msgr2=0x562b3af9d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af9d9c0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f3504000c00 tx=0x7f3504003ef0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 shutdown_connections 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562b3b0143b0 0x562b3b0167a0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562b3af94550 0x562b3af9d9c0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3af8f850 0x562b3afa20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 --2- 172.21.15.52:0/4118885915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562b3af90170 0x562b3af9d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 >> 172.21.15.52:0/4118885915 conn(0x562b3aeb8080 msgr2=0x562b3af9a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f34ebfff700 0 client.18024 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 shutdown_connections 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 1 -- 172.21.15.52:0/4118885915 wait complete. 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.914+0000 7f351d6d2b80 20 client.18024 trim_cache size 0 max 16384 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:20:37.920 INFO:teuthology.orchestra.run.smithi052.stdout: "6beecc3c-192a-40f3-9599-8f625640a6bd": { 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "6beecc3c-192a-40f3-9599-8f625640a6bd", 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:20:37.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f34f40a2c20 msgr2=0x7f34f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f34f40a2c20 0x7f34f40a50e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f350c078940 tx=0x7f3500009180 comp rx=0 tx=0).stop 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 msgr2=0x7f350c19e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 0x7f350c19e740 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00d980 tx=0x7f34fc00dc90 comp rx=0 tx=0).stop 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 shutdown_connections 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f34f40a2c20 0x7f34f40a50e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f350c107400 0x7f350c198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f350c101390 0x7f350c19e740 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 --2- 172.21.15.52:0/1098370243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f350c101d50 0x7f350c19ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 >> 172.21.15.52:0/1098370243 conn(0x7f350c078340 msgr2=0x7f350c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 shutdown_connections 2024-06-02T23:20:37.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:37.915+0000 7f3512e4f700 1 -- 172.21.15.52:0/1098370243 wait complete. 2024-06-02T23:20:38.600 INFO:tasks.cephfs.filesystem:scrub status for tag:6beecc3c-192a-40f3-9599-8f625640a6bd - {'path': '/', 'tag': '6beecc3c-192a-40f3-9599-8f625640a6bd', 'options': 'recursive,force'} 2024-06-02T23:20:38.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.74 MB/sec execute 347 sec latency 9951.782 ms 2024-06-02T23:20:39.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.74 MB/sec execute 348 sec latency 10951.853 ms 2024-06-02T23:20:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:39 smithi123 ceph-mon[35977]: pgmap v694: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 509 KiB/s wr, 75 op/s 2024-06-02T23:20:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:39 smithi174 ceph-mon[35994]: pgmap v694: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 509 KiB/s wr, 75 op/s 2024-06-02T23:20:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:39 smithi052 ceph-mon[32340]: pgmap v694: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 509 KiB/s wr, 75 op/s 2024-06-02T23:20:40.638 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.73 MB/sec execute 349 sec latency 11951.949 ms 2024-06-02T23:20:41.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.73 MB/sec execute 350 sec latency 12952.053 ms 2024-06-02T23:20:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:41 smithi123 ceph-mon[35977]: pgmap v695: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 372 KiB/s wr, 72 op/s 2024-06-02T23:20:41.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:20:41.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:20:41.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:20:41.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:20:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:41 smithi174 ceph-mon[35994]: pgmap v695: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 372 KiB/s wr, 72 op/s 2024-06-02T23:20:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:20:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:20:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:20:41.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:20:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:41 smithi052 ceph-mon[32340]: pgmap v695: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 372 KiB/s wr, 72 op/s 2024-06-02T23:20:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:20:41.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:20:41.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:20:41.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:20:42.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.72 MB/sec execute 351 sec latency 13952.140 ms 2024-06-02T23:20:43.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.72 MB/sec execute 352 sec latency 14952.259 ms 2024-06-02T23:20:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:43 smithi123 ceph-mon[35977]: pgmap v696: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 110 KiB/s wr, 46 op/s 2024-06-02T23:20:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:43 smithi174 ceph-mon[35994]: pgmap v696: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 110 KiB/s wr, 46 op/s 2024-06-02T23:20:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:43 smithi052 ceph-mon[32340]: pgmap v696: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 110 KiB/s wr, 46 op/s 2024-06-02T23:20:44.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.71 MB/sec execute 353 sec latency 15952.321 ms 2024-06-02T23:20:45.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.71 MB/sec execute 354 sec latency 16952.445 ms 2024-06-02T23:20:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:45 smithi123 ceph-mon[35977]: pgmap v697: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-06-02T23:20:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:45 smithi174 ceph-mon[35994]: pgmap v697: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-06-02T23:20:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:45 smithi052 ceph-mon[32340]: pgmap v697: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-06-02T23:20:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 -- 172.21.15.52:0/2924510402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664100c80 msgr2=0x7fd6641010e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 --2- 172.21.15.52:0/2924510402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664100c80 0x7fd6641010e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009900 tx=0x7fd64c02a410 comp rx=0 tx=0).stop 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 -- 172.21.15.52:0/2924510402 shutdown_connections 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 --2- 172.21.15.52:0/2924510402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664101620 0x7fd664101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 --2- 172.21.15.52:0/2924510402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664100c80 0x7fd6641010e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 --2- 172.21.15.52:0/2924510402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd664100320 0x7fd664100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 -- 172.21.15.52:0/2924510402 >> 172.21.15.52:0/2924510402 conn(0x7fd6640fbf20 msgr2=0x7fd6640fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 -- 172.21.15.52:0/2924510402 shutdown_connections 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 -- 172.21.15.52:0/2924510402 wait complete. 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 Processor -- start 2024-06-02T23:20:46.070 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.069+0000 7fd669722640 1 -- start start 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd669722640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664100320 0x7fd664199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd669722640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd664100c80 0x7fd66419a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd669722640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 0x7fd6641a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd669722640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410f750 con 0x7fd664101620 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd669722640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410f5d0 con 0x7fd664100320 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd669722640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd66410f8d0 con 0x7fd664100c80 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 0x7fd6641a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 0x7fd6641a1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47608/0 (socket says 172.21.15.52:47608) 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 -- 172.21.15.52:0/21223423 learned_addr learned my addr 172.21.15.52:0/21223423 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd65bfff640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd664100c80 0x7fd66419a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd662ffd640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664100320 0x7fd664199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 -- 172.21.15.52:0/21223423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd664100c80 msgr2=0x7fd66419a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd664100c80 0x7fd66419a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 -- 172.21.15.52:0/21223423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664100320 msgr2=0x7fd664199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664100320 0x7fd664199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 -- 172.21.15.52:0/21223423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6641a19c0 con 0x7fd664101620 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.070+0000 7fd6637fe640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 0x7fd6641a1320 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00eaa0 tx=0x7fd65c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.071+0000 7fd660ff9640 1 -- 172.21.15.52:0/21223423 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c00cdb0 con 0x7fd664101620 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.071+0000 7fd660ff9640 1 -- 172.21.15.52:0/21223423 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd65c005020 con 0x7fd664101620 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.071+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6641a1cb0 con 0x7fd664101620 2024-06-02T23:20:46.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.071+0000 7fd660ff9640 1 -- 172.21.15.52:0/21223423 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd65c0116d0 con 0x7fd664101620 2024-06-02T23:20:46.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.071+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6641a21b0 con 0x7fd664101620 2024-06-02T23:20:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd660ff9640 1 -- 172.21.15.52:0/21223423 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd65c0051d0 con 0x7fd664101620 2024-06-02T23:20:46.080 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 Processor -- start 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 -- start start 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f246d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f2472500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f2475000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5585f23d04f0 con 0x5585f2472b70 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5585f236e870 con 0x5585f246d010 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.074+0000 7fd66c435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585f23cc4b0 con 0x5585f246da00 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f2475000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f2475000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47620/0 (socket says 172.21.15.52:47620) 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 -- 172.21.15.52:0/3933063002 learned_addr learned my addr 172.21.15.52:0/3933063002 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:46.081 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f246d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd662ffd640 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f2472500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd660ff9640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6500a2cb0 0x7fd6500a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 -- 172.21.15.52:0/3933063002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 msgr2=0x5585f2472500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f2472500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 -- 172.21.15.52:0/3933063002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 msgr2=0x5585f246d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f246d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd6637fe640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6500a2cb0 0x7fd6500a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 -- 172.21.15.52:0/3933063002 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f238a9b0 con 0x5585f2472b70 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd660ff9640 1 -- 172.21.15.52:0/21223423 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd65c0cf190 con 0x7fd664101620 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.075+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f2475000 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005ec0 tx=0x7fd64c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f991c451bf299c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd6597fa640 1 -- 172.21.15.52:0/3933063002 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c035040 con 0x5585f2472b70 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd6597fa640 1 -- 172.21.15.52:0/3933063002 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd64c0317f0 con 0x5585f2472b70 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd6597fa640 1 -- 172.21.15.52:0/3933063002 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c003cf0 con 0x5585f2472b70 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 -- 172.21.15.52:0/3933063002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 msgr2=0x5585f2475000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd6637fe640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6500a2cb0 0x7fd6500a5170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd65c09ba60 tx=0x7fd65c06eb00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f2475000 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005ec0 tx=0x7fd64c005e30 comp rx=0 tx=0).stop 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 -- 172.21.15.52:0/3933063002 shutdown_connections 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f2472500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f2475000 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 --2- 172.21.15.52:0/3933063002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f246d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.076+0000 7fd66c435740 1 -- 172.21.15.52:0/3933063002 >> 172.21.15.52:0/3933063002 conn(0x5585f2468680 msgr2=0x5585f2468a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 -- 172.21.15.52:0/3933063002 shutdown_connections 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 -- 172.21.15.52:0/3933063002 wait complete. 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 -- start start 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 Processor -- start 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f2477260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f24777a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f247e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5585f2475ac0 con 0x5585f246d010 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5585f2359ae0 con 0x5585f246da00 2024-06-02T23:20:46.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585f2475c40 con 0x5585f2472b70 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd66c435740 10 client.0 _ll_register_callbacks cb 0x5585f23929b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f2477260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f2477260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47624/0 (socket says 172.21.15.52:47624) 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd65bfff640 1 -- 172.21.15.52:0/705156265 learned_addr learned my addr 172.21.15.52:0/705156265 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.077+0000 7fd662ffd640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f24777a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f247e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 -- 172.21.15.52:0/705156265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 msgr2=0x5585f247e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f247e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 -- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 msgr2=0x5585f24777a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f24777a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 -- 172.21.15.52:0/705156265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f20e4210 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd65bfff640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f2477260 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005de0 tx=0x7fd64c005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8b3bf31deeb9f6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd64a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd64a7fc640 1 -- 172.21.15.52:0/705156265 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c053060 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd64a7fc640 1 -- 172.21.15.52:0/705156265 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd64c03e970 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd64c0317f0 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd64a7fc640 1 -- 172.21.15.52:0/705156265 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd64c035040 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd66c435740 10 client.18036 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5585f21b0620 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.078+0000 7fd64a7fc640 1 -- 172.21.15.52:0/705156265 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd64c0317f0 con 0x5585f246d010 2024-06-02T23:20:46.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd64a7fc640 1 -- 172.21.15.52:0/705156265 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd64c04bc30 con 0x5585f246d010 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd66c435740 10 client.18036 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd66c435740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd66c435740 20 client.18036 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd66c435740 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f2482cd0 0x5585f24850c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd66c435740 4 client.18036 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585f23e3f10 con 0x5585f2482cd0 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.079+0000 7fd662ffd640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f2482cd0 0x5585f24850c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.080+0000 7fd64a7fc640 10 client.18036 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:46.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.080+0000 7fd662ffd640 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f2482cd0 0x5585f24850c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c98454907789c277 server_cookie=5a5ae565a1aae4c4 in_seq=0 out_seq=0 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd64a7fc640 1 -- 172.21.15.52:0/705156265 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5585f23e3f10 con 0x5585f2482cd0 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd64a7fc640 10 client.18036 handle_command_reply: tid=0 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 client.18036 shutdown 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 msgr2=0x5585f2477260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f2477260 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fd64c005de0 tx=0x7fd64c005d50 comp rx=0 tx=0).stop 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 shutdown_connections 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f2482cd0 0x5585f24850c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f247e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f24777a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd64a7fc640 0 client.18036 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 --2- 172.21.15.52:0/705156265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f2477260 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 >> 172.21.15.52:0/705156265 conn(0x5585f2468680 msgr2=0x5585f24677e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 shutdown_connections 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 1 -- 172.21.15.52:0/705156265 wait complete. 2024-06-02T23:20:46.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.081+0000 7fd66c435740 20 client.18036 trim_cache size 0 max 16384 2024-06-02T23:20:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 Processor -- start 2024-06-02T23:20:46.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 -- start start 2024-06-02T23:20:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f246b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f247e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f246bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641aae90 con 0x5585f246d010 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641ac9d0 con 0x5585f246da00 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd66c435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641ab560 con 0x5585f2472b70 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.097+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f246b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f246b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47636/0 (socket says 172.21.15.52:47636) 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 -- 172.21.15.52:0/4123729410 learned_addr learned my addr 172.21.15.52:0/4123729410 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd662ffd640 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f247e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 -- 172.21.15.52:0/4123729410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 msgr2=0x5585f246bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f246bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 -- 172.21.15.52:0/4123729410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 msgr2=0x5585f247e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f247e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 -- 172.21.15.52:0/4123729410 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585f24684b0 con 0x5585f246d010 2024-06-02T23:20:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd662ffd640 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f247e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd65bfff640 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f246b590 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009850 tx=0x7fd64c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1879dd016f7c101 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd658ff9640 1 -- 172.21.15.52:0/4123729410 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c03e070 con 0x5585f246d010 2024-06-02T23:20:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd658ff9640 1 -- 172.21.15.52:0/4123729410 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd64c03e210 con 0x5585f246d010 2024-06-02T23:20:46.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.098+0000 7fd658ff9640 1 -- 172.21.15.52:0/4123729410 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c03e770 con 0x5585f246d010 2024-06-02T23:20:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 -- 172.21.15.52:0/4123729410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 msgr2=0x5585f246b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f246b590 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fd64c009850 tx=0x7fd64c005fd0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 -- 172.21.15.52:0/4123729410 shutdown_connections 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f2472b70 0x5585f246bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246da00 0x5585f247e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 --2- 172.21.15.52:0/4123729410 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f246d010 0x5585f246b590 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 -- 172.21.15.52:0/4123729410 >> 172.21.15.52:0/4123729410 conn(0x5585f2468680 msgr2=0x5585f2472290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 -- 172.21.15.52:0/4123729410 shutdown_connections 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 -- 172.21.15.52:0/4123729410 wait complete. 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.099+0000 7fd66c435740 1 -- start start 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 Processor -- start 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f247b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f247bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f24eb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641ac300 con 0x5585f2472b70 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641abc30 con 0x5585f246d010 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6641a8e70 con 0x5585f246da00 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd66c435740 10 client.0 _ll_register_callbacks cb 0x7fd64c032d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f24eb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f24eb860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47644/0 (socket says 172.21.15.52:47644) 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 -- 172.21.15.52:0/3003253993 learned_addr learned my addr 172.21.15.52:0/3003253993 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f247b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd662ffd640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f247bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 -- 172.21.15.52:0/3003253993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 msgr2=0x5585f247bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f247bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 -- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 msgr2=0x5585f247b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f247b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.100+0000 7fd65bfff640 1 -- 172.21.15.52:0/3003253993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585f24ebda0 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd65bfff640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f24eb860 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd64c04a580 tx=0x7fd64c031610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26e3b9c5c60c2e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd64a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd64a7fc640 1 -- 172.21.15.52:0/3003253993 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd64c05e020 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd64a7fc640 1 -- 172.21.15.52:0/3003253993 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd64c03e770 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd64c03e210 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd64a7fc640 1 -- 172.21.15.52:0/3003253993 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd64c06e020 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 10 client.18042 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5585f24ecfc0 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd64a7fc640 1 -- 172.21.15.52:0/3003253993 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd64c03e210 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd64a7fc640 1 -- 172.21.15.52:0/3003253993 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd64c04b710 con 0x5585f2472b70 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 10 client.18042 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 20 client.18042 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:46.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f24ed7c0 0x5585f24efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 4 client.18042 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:20:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.101+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5585f23ddc20 con 0x5585f24ed7c0 2024-06-02T23:20:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.102+0000 7fd662ffd640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f24ed7c0 0x5585f24efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.102+0000 7fd64a7fc640 10 client.18042 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:46.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.102+0000 7fd662ffd640 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f24ed7c0 0x5585f24efbe0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ce2d209ab2ffa3 server_cookie=1531390450682031 in_seq=0 out_seq=0 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd64a7fc640 1 -- 172.21.15.52:0/3003253993 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5585f23ddc20 con 0x5585f24ed7c0 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd64a7fc640 10 client.18042 handle_command_reply: tid=0 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 client.18042 shutdown 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 msgr2=0x5585f24eb860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f24eb860 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd64c04a580 tx=0x7fd64c031610 comp rx=0 tx=0).stop 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 shutdown_connections 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585f24ed7c0 0x5585f24efbe0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5585f246da00 0x5585f247bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5585f2472b70 0x5585f24eb860 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 --2- 172.21.15.52:0/3003253993 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5585f246d010 0x5585f247b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd64a7fc640 0 client.18042 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 >> 172.21.15.52:0/3003253993 conn(0x5585f2468680 msgr2=0x5585f246b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 shutdown_connections 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 1 -- 172.21.15.52:0/3003253993 wait complete. 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd66c435740 20 client.18042 trim_cache size 0 max 16384 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:20:46.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:20:46.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6500a2cb0 msgr2=0x7fd6500a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6500a2cb0 0x7fd6500a5170 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fd65c09ba60 tx=0x7fd65c06eb00 comp rx=0 tx=0).stop 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 msgr2=0x7fd6641a1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 0x7fd6641a1320 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fd65c00eaa0 tx=0x7fd65c00ef60 comp rx=0 tx=0).stop 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 shutdown_connections 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6500a2cb0 0x7fd6500a5170 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd664100c80 0x7fd66419a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd664101620 0x7fd6641a1320 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 --2- 172.21.15.52:0/21223423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd664100320 0x7fd664199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 >> 172.21.15.52:0/21223423 conn(0x7fd6640fbf20 msgr2=0x7fd6640fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 shutdown_connections 2024-06-02T23:20:46.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:46.120+0000 7fd669722640 1 -- 172.21.15.52:0/21223423 wait complete. 2024-06-02T23:20:46.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:20:46.544 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:46.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.71 MB/sec execute 355 sec latency 17952.522 ms 2024-06-02T23:20:47.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.70 MB/sec execute 356 sec latency 18952.570 ms 2024-06-02T23:20:47.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:47 smithi052 ceph-mon[32340]: pgmap v698: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-06-02T23:20:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:47 smithi174 ceph-mon[35994]: pgmap v698: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-06-02T23:20:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:47 smithi123 ceph-mon[35977]: pgmap v698: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 13 KiB/s wr, 26 op/s 2024-06-02T23:20:48.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 -- 172.21.15.52:0/2748930946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d81030f0 msgr2=0x7f97d8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 --2- 172.21.15.52:0/2748930946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d81030f0 0x7f97d8107110 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f97c80099c0 tx=0x7f97c8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 -- 172.21.15.52:0/2748930946 shutdown_connections 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 --2- 172.21.15.52:0/2748930946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8107840 0x7f97d810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 --2- 172.21.15.52:0/2748930946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d8102740 0x7f97d8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 --2- 172.21.15.52:0/2748930946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d81030f0 0x7f97d8107110 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 -- 172.21.15.52:0/2748930946 >> 172.21.15.52:0/2748930946 conn(0x7f97d80795f0 msgr2=0x7f97d8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 -- 172.21.15.52:0/2748930946 shutdown_connections 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 -- 172.21.15.52:0/2748930946 wait complete. 2024-06-02T23:20:48.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 Processor -- start 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.257+0000 7f97e0459700 1 -- start start 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97e0459700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8102740 0x7f97d819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97e0459700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d81030f0 0x7f97d819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97e0459700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 0x7f97d8198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97e0459700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8112600 con 0x7f97d8107840 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97e0459700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8112480 con 0x7f97d8102740 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97e0459700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97d8112780 con 0x7f97d81030f0 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 0x7f97d8198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:48.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 0x7f97d8198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47662/0 (socket says 172.21.15.52:47662) 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 -- 172.21.15.52:0/3935411849 learned_addr learned my addr 172.21.15.52:0/3935411849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97dd9f4700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d81030f0 0x7f97d819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de1f5700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8102740 0x7f97d819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 -- 172.21.15.52:0/3935411849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d81030f0 msgr2=0x7f97d819ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d81030f0 0x7f97d819ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 -- 172.21.15.52:0/3935411849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8102740 msgr2=0x7f97d819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8102740 0x7f97d819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 -- 172.21.15.52:0/3935411849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d8198fe0 con 0x7f97d8107840 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de1f5700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8102740 0x7f97d819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:48.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.258+0000 7f97de9f6700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 0x7f97d8198880 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f97d0007d70 tx=0x7f97d000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:48.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.259+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0016020 con 0x7f97d8107840 2024-06-02T23:20:48.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.259+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97d000ad40 con 0x7f97d8107840 2024-06-02T23:20:48.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.259+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d81992d0 con 0x7f97d8107840 2024-06-02T23:20:48.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.259+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d00115e0 con 0x7f97d8107840 2024-06-02T23:20:48.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.259+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d81a5590 con 0x7f97d8107840 2024-06-02T23:20:48.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.260+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d8069180 con 0x7f97d8107840 2024-06-02T23:20:48.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.261+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97d0019020 con 0x7f97d8107840 2024-06-02T23:20:48.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.261+0000 7f97cf7fe700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97c40a2c20 0x7f97c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:48.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.262+0000 7f97de1f5700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97c40a2c20 0x7f97c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:48.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.262+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f97d00cefe0 con 0x7f97d8107840 2024-06-02T23:20:48.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.262+0000 7f97de1f5700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97c40a2c20 0x7f97c40a50e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f97d81999c0 tx=0x7f97d4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:48.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.266+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f97d0020020 con 0x7f97d8107840 2024-06-02T23:20:48.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.519+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97d802dca0 con 0x7f97d8107840 2024-06-02T23:20:48.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.520+0000 7f97cf7fe700 1 -- 172.21.15.52:0/3935411849 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f97d009bf70 con 0x7f97d8107840 2024-06-02T23:20:48.521 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:20:48.521 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:20:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97c40a2c20 msgr2=0x7f97c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97c40a2c20 0x7f97c40a50e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f97d81999c0 tx=0x7f97d4009260 comp rx=0 tx=0).stop 2024-06-02T23:20:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 msgr2=0x7f97d8198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 0x7f97d8198880 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f97d0007d70 tx=0x7f97d000a340 comp rx=0 tx=0).stop 2024-06-02T23:20:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 shutdown_connections 2024-06-02T23:20:48.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97c40a2c20 0x7f97c40a50e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97d8107840 0x7f97d8198880 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97d8102740 0x7f97d819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 --2- 172.21.15.52:0/3935411849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97d81030f0 0x7f97d819ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 >> 172.21.15.52:0/3935411849 conn(0x7f97d80795f0 msgr2=0x7f97d8100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 shutdown_connections 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:48.523+0000 7f97e0459700 1 -- 172.21.15.52:0/3935411849 wait complete. 2024-06-02T23:20:48.524 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:20:48.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.70 MB/sec execute 357 sec latency 19952.679 ms 2024-06-02T23:20:49.214 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 9923b3d7 --release --await' 2024-06-02T23:20:49.214 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 9923b3d7 --release --await 2024-06-02T23:20:49.469 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:20:49.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.69 MB/sec execute 358 sec latency 20952.723 ms 2024-06-02T23:20:49.809 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:49 smithi123 ceph-mon[35977]: pgmap v699: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-06-02T23:20:49.809 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:49 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3935411849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:49 smithi174 ceph-mon[35994]: pgmap v699: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-06-02T23:20:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:49 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3935411849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:49 smithi052 ceph-mon[32340]: pgmap v699: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 6.8 KiB/s wr, 18 op/s 2024-06-02T23:20:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:49 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3935411849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:20:50.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160363 1.69 MB/sec execute 359 sec latency 21952.770 ms 2024-06-02T23:20:50.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:50 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:20:50.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:50 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:50 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:50 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:50 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:50 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:20:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:50 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:50 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:50 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:50 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:51.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:50 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:20:51.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:50 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:51.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:50 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:51.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:50 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:51.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:50 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:51.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 -- 172.21.15.52:0/2955114267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006b7b0 msgr2=0x7fde101070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 --2- 172.21.15.52:0/2955114267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006b7b0 0x7fde101070d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fde040099c0 tx=0x7fde04009cd0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 -- 172.21.15.52:0/2955114267 shutdown_connections 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 --2- 172.21.15.52:0/2955114267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 0x7fde1010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 --2- 172.21.15.52:0/2955114267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1006ae00 0x7fde1006b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 --2- 172.21.15.52:0/2955114267 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006b7b0 0x7fde101070d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 -- 172.21.15.52:0/2955114267 >> 172.21.15.52:0/2955114267 conn(0x7fde100773e0 msgr2=0x7fde100777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 -- 172.21.15.52:0/2955114267 shutdown_connections 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.273+0000 7fde14cc1700 1 -- 172.21.15.52:0/2955114267 wait complete. 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.274+0000 7fde14cc1700 1 Processor -- start 2024-06-02T23:20:51.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.274+0000 7fde14cc1700 1 -- start start 2024-06-02T23:20:51.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 0x7fde101014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1006b7b0 0x7fde10101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 0x7fde10101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde10112680 con 0x7fde1006ae00 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde10112500 con 0x7fde1006b7b0 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde10112800 con 0x7fde101077b0 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 0x7fde101014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 0x7fde101014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47680/0 (socket says 172.21.15.52:47680) 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 -- 172.21.15.52:0/560922603 learned_addr learned my addr 172.21.15.52:0/560922603 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1006b7b0 0x7fde10101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 0x7fde10101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 -- 172.21.15.52:0/560922603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 msgr2=0x7fde10101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 0x7fde10101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 -- 172.21.15.52:0/560922603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1006b7b0 msgr2=0x7fde10101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1006b7b0 0x7fde10101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 -- 172.21.15.52:0/560922603 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde101a70d0 con 0x7fde1006ae00 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 0x7fde10101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde0e59c700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 0x7fde101014f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fde000060b0 tx=0x7fde00007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fddff7fe700 1 -- 172.21.15.52:0/560922603 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0001a070 con 0x7fde1006ae00 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fddff7fe700 1 -- 172.21.15.52:0/560922603 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde0000de20 con 0x7fde1006ae00 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde101a73c0 con 0x7fde1006ae00 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.275+0000 7fddff7fe700 1 -- 172.21.15.52:0/560922603 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde00014720 con 0x7fde1006ae00 2024-06-02T23:20:51.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.276+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde101a7920 con 0x7fde1006ae00 2024-06-02T23:20:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.278+0000 7fddff7fe700 1 -- 172.21.15.52:0/560922603 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fde0001d070 con 0x7fde1006ae00 2024-06-02T23:20:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.278+0000 7fddff7fe700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fddf80a2ca0 0x7fddf80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.279+0000 7fddff7fe700 1 -- 172.21.15.52:0/560922603 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fde000ce900 con 0x7fde1006ae00 2024-06-02T23:20:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.279+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fddf80a2ca0 0x7fddf80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.279+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fddf80a2ca0 0x7fddf80a5160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fde04000c00 tx=0x7fde04005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:51.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.279+0000 7fde1f544b80 1 Processor -- start 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 -- start start 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc578e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc51710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc52070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56064fb44060 con 0x56064fc57500 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56064fba2950 con 0x56064fc57e20 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde1f544b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56064fb0a560 con 0x56064fc51c50 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47686/0 (socket says 172.21.15.52:47686) 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 -- 172.21.15.52:0/2781859793 learned_addr learned my addr 172.21.15.52:0/2781859793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc51710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc52070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 -- 172.21.15.52:0/2781859793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 msgr2=0x56064fc52070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc52070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 -- 172.21.15.52:0/2781859793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 msgr2=0x56064fc51710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc51710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0e59c700 1 -- 172.21.15.52:0/2781859793 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064faf19d0 con 0x56064fc57500 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.280+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc52070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.281+0000 7fde0e59c700 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc578e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fde00015e60 tx=0x7fde0009abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3aae529c2e4d55 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.281+0000 7fddfcff9700 1 -- 172.21.15.52:0/2781859793 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0009f4e0 con 0x56064fc57500 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.281+0000 7fddfcff9700 1 -- 172.21.15.52:0/2781859793 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde000a3da0 con 0x56064fc57500 2024-06-02T23:20:51.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.281+0000 7fddfcff9700 1 -- 172.21.15.52:0/2781859793 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0009b590 con 0x56064fc57500 2024-06-02T23:20:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 -- 172.21.15.52:0/2781859793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 msgr2=0x56064fc578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc578e0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fde00015e60 tx=0x7fde0009abb0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 -- 172.21.15.52:0/2781859793 shutdown_connections 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc52070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc578e0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2781859793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc51710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 -- 172.21.15.52:0/2781859793 >> 172.21.15.52:0/2781859793 conn(0x56064fb7e180 msgr2=0x56064fc4ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 -- 172.21.15.52:0/2781859793 shutdown_connections 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 -- 172.21.15.52:0/2781859793 wait complete. 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 -- start start 2024-06-02T23:20:51.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.282+0000 7fde1f544b80 1 Processor -- start 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 0x56064fc5daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc5ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc5b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56064fb430c0 con 0x56064fc51c50 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56064fb9e910 con 0x56064fc57e20 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56064fc5e800 con 0x56064fc57500 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde1f544b80 10 client.0 _ll_register_callbacks cb 0x56064fb386b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 0x56064fc5daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0dd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc5ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 0x56064fc5daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47698/0 (socket says 172.21.15.52:47698) 2024-06-02T23:20:51.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 -- 172.21.15.52:0/4084376855 learned_addr learned my addr 172.21.15.52:0/4084376855 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc5b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 -- 172.21.15.52:0/4084376855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 msgr2=0x56064fc5ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc5ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 -- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 msgr2=0x56064fc5b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc5b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 -- 172.21.15.52:0/4084376855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064fb68850 con 0x56064fc51c50 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc5ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.283+0000 7fde0e59c700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 0x56064fc5daf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fde00015c40 tx=0x7fde00097500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef37fe71cd67aa78 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fddf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:51.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fddf5ffb700 1 -- 172.21.15.52:0/4084376855 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0009f4e0 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fddf5ffb700 1 -- 172.21.15.52:0/4084376855 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fde000a3910 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde000a3da0 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fddf5ffb700 1 -- 172.21.15.52:0/4084376855 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde000a3b10 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 10 client.18060 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56064fc68f40 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fddf5ffb700 1 -- 172.21.15.52:0/4084376855 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde000a3da0 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fddf5ffb700 1 -- 172.21.15.52:0/4084376855 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fde00004ad0 con 0x56064fc51c50 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 10 client.18060 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 10 client.18060 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 20 client.18060 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fc6b410 0x56064fc6d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 4 client.18060 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.284+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56064fbafe50 con 0x56064fc6b410 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.285+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fc6b410 0x56064fc6d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.285+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fc6b410 0x56064fc6d800 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=293b7d52dc546096 server_cookie=7b9f04fa9b85a89b in_seq=0 out_seq=0 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.285+0000 7fddf5ffb700 10 client.18060 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fddf5ffb700 1 -- 172.21.15.52:0/4084376855 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56064fbafe50 con 0x56064fc6b410 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fddf5ffb700 10 client.18060 handle_command_reply: tid=0 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 client.18060 shutdown 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 msgr2=0x56064fc5daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 0x56064fc5daf0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fde00015c40 tx=0x7fde00097500 comp rx=0 tx=0).stop 2024-06-02T23:20:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 shutdown_connections 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fc6b410 0x56064fc6d800 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc5b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc51c50 0x56064fc5daf0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4084376855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc5ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 >> 172.21.15.52:0/4084376855 conn(0x56064fb7e180 msgr2=0x56064fc4bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fddf5ffb700 0 client.18060 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 shutdown_connections 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 1 -- 172.21.15.52:0/4084376855 wait complete. 2024-06-02T23:20:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.286+0000 7fde1f544b80 20 client.18060 trim_cache size 0 max 16384 2024-06-02T23:20:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 Processor -- start 2024-06-02T23:20:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 -- start start 2024-06-02T23:20:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc4c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc4c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde101aeca0 con 0x56064fc57500 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde101b07e0 con 0x56064fc57e20 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde1f544b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde101af370 con 0x56064fc51c50 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde0dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc4c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde0dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc4c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47702/0 (socket says 172.21.15.52:47702) 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde0dd9b700 1 -- 172.21.15.52:0/4274034385 learned_addr learned my addr 172.21.15.52:0/4274034385 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc4c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.307+0000 7fde0e59c700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0dd9b700 1 -- 172.21.15.52:0/4274034385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 msgr2=0x56064fc5cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc5cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0dd9b700 1 -- 172.21.15.52:0/4274034385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 msgr2=0x56064fc4c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc4c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0dd9b700 1 -- 172.21.15.52:0/4274034385 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064fc5de60 con 0x56064fc57500 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc4c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:51.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0e59c700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:20:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc4c330 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fde04005950 tx=0x7fde04022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8832aa49cbcb2cfe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fddf7fff700 1 -- 172.21.15.52:0/4274034385 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04010c00 con 0x56064fc57500 2024-06-02T23:20:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fddf7fff700 1 -- 172.21.15.52:0/4274034385 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde04010da0 con 0x56064fc57500 2024-06-02T23:20:51.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.308+0000 7fddf7fff700 1 -- 172.21.15.52:0/4274034385 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04024670 con 0x56064fc57500 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 -- 172.21.15.52:0/4274034385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 msgr2=0x56064fc4c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc4c330 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fde04005950 tx=0x7fde04022000 comp rx=0 tx=0).stop 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 -- 172.21.15.52:0/4274034385 shutdown_connections 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc57e20 0x56064fc4c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc51c50 0x56064fc5cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 --2- 172.21.15.52:0/4274034385 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57500 0x56064fc4c330 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 -- 172.21.15.52:0/4274034385 >> 172.21.15.52:0/4274034385 conn(0x56064fb7e180 msgr2=0x56064fc5c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 -- 172.21.15.52:0/4274034385 shutdown_connections 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 -- 172.21.15.52:0/4274034385 wait complete. 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 -- start start 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.309+0000 7fde1f544b80 1 Processor -- start 2024-06-02T23:20:51.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc51c50 0x56064fc70930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc6dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 0x56064fc6e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fde101b0110 con 0x56064fc57e20 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde101afa40 con 0x56064fc51c50 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fde101acc40 con 0x56064fc57500 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 10 client.0 _ll_register_callbacks cb 0x56064fb386b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 0x56064fc6e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 0x56064fc6e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47712/0 (socket says 172.21.15.52:47712) 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 -- 172.21.15.52:0/2612404961 learned_addr learned my addr 172.21.15.52:0/2612404961 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0e59c700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc51c50 0x56064fc70930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0dd9b700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc6dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 -- 172.21.15.52:0/2612404961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 msgr2=0x56064fc6dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc6dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 -- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc51c50 msgr2=0x56064fc70930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc51c50 0x56064fc70930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 -- 172.21.15.52:0/2612404961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56064fc6e7d0 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde0ed9d700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 0x56064fc6e210 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fde08010a50 tx=0x7fde08010d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a1f38eb07271160 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fddf5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fddf5ffb700 1 -- 172.21.15.52:0/2612404961 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0800fd20 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fddf5ffb700 1 -- 172.21.15.52:0/2612404961 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fde0801a3f0 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.310+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde04010da0 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fddf5ffb700 1 -- 172.21.15.52:0/2612404961 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fde0800de90 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 10 client.18066 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56064fcd5b40 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fddf5ffb700 1 -- 172.21.15.52:0/2612404961 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fde04010da0 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fddf5ffb700 1 -- 172.21.15.52:0/2612404961 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fde080265f0 con 0x56064fc57e20 2024-06-02T23:20:51.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 10 client.18066 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 10 client.18066 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 20 client.18066 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fcd6300 0x56064fcd8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 4 client.18066 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9923b3d7", "release": true, "await": true}] 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9923b3d7", "release": true, "await": true}) -- 0x56064fbaa3d0 con 0x56064fcd6300 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.311+0000 7fde0e59c700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fcd6300 0x56064fcd8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.312+0000 7fde0e59c700 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fcd6300 0x56064fcd8720 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a5e8ddf01f08959 server_cookie=ffb3bdf5642d61e3 in_seq=0 out_seq=0 2024-06-02T23:20:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.312+0000 7fddf5ffb700 10 client.18066 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.317+0000 7fddf5ffb700 1 -- 172.21.15.52:0/2612404961 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56064fbaa3d0 con 0x56064fcd6300 2024-06-02T23:20:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.317+0000 7fddf5ffb700 10 client.18066 handle_command_reply: tid=0 2024-06-02T23:20:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.317+0000 7fde1f544b80 1 client.18066 shutdown 2024-06-02T23:20:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.317+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 msgr2=0x56064fc6e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.317+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 0x56064fc6e210 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fde08010a50 tx=0x7fde08010d60 comp rx=0 tx=0).stop 2024-06-02T23:20:51.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 shutdown_connections 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56064fcd6300 0x56064fcd8720 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56064fc57e20 0x56064fc6e210 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56064fc51c50 0x56064fc70930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 --2- 172.21.15.52:0/2612404961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064fc57500 0x56064fc6dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fddf5ffb700 0 client.18066 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 >> 172.21.15.52:0/2612404961 conn(0x56064fb7e180 msgr2=0x56064fc50700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 shutdown_connections 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 1 -- 172.21.15.52:0/2612404961 wait complete. 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde1f544b80 20 client.18066 trim_cache size 0 max 16384 2024-06-02T23:20:51.319 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 32, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "9923b3d7": { 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 32, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 146.3, 2024-06-02T23:20:51.320 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:20:51.321 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fddf80a2ca0 msgr2=0x7fddf80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde14cc1700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fddf80a2ca0 0x7fddf80a5160 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fde04000c00 tx=0x7fde04005fd0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 msgr2=0x7fde101014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.318+0000 7fde14cc1700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 0x7fde101014f0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fde000060b0 tx=0x7fde00007f90 comp rx=0 tx=0).stop 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 shutdown_connections 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fddf80a2ca0 0x7fddf80a5160 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fde101077b0 0x7fde10101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fde1006ae00 0x7fde101014f0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 --2- 172.21.15.52:0/560922603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde1006b7b0 0x7fde10101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 >> 172.21.15.52:0/560922603 conn(0x7fde100773e0 msgr2=0x7fde10106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 shutdown_connections 2024-06-02T23:20:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:20:51.319+0000 7fde14cc1700 1 -- 172.21.15.52:0/560922603 wait complete. 2024-06-02T23:20:51.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 160530 1.68 MB/sec execute 360 sec latency 22632.835 ms 2024-06-02T23:20:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:51 smithi052 ceph-mon[32340]: pgmap v700: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-06-02T23:20:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:20:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:20:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:51 smithi174 ceph-mon[35994]: pgmap v700: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-06-02T23:20:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:20:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:20:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:51 smithi123 ceph-mon[35977]: pgmap v700: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 7.2 KiB/s wr, 16 op/s 2024-06-02T23:20:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:20:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:20:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:20:52.051 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9923b3d7', total seconds elapsed: 32.1 2024-06-02T23:20:52.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 161089 1.69 MB/sec execute 361 sec latency 14.929 ms 2024-06-02T23:20:53.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 161701 1.69 MB/sec execute 362 sec latency 15.598 ms 2024-06-02T23:20:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:53 smithi174 ceph-mon[35994]: pgmap v701: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-06-02T23:20:53.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:53 smithi052 ceph-mon[32340]: pgmap v701: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-06-02T23:20:54.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:53 smithi123 ceph-mon[35977]: pgmap v701: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.2 KiB/s wr, 14 op/s 2024-06-02T23:20:54.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 162223 1.69 MB/sec execute 363 sec latency 14.933 ms 2024-06-02T23:20:55.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 162946 1.70 MB/sec execute 364 sec latency 14.168 ms 2024-06-02T23:20:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:55 smithi174 ceph-mon[35994]: pgmap v702: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-06-02T23:20:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:55 smithi052 ceph-mon[32340]: pgmap v702: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-06-02T23:20:56.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:55 smithi123 ceph-mon[35977]: pgmap v702: 97 pgs: 97 active+clean; 71 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 19 op/s 2024-06-02T23:20:56.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 163397 1.71 MB/sec execute 365 sec latency 14.680 ms 2024-06-02T23:20:57.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 163636 1.71 MB/sec execute 366 sec latency 13.324 ms 2024-06-02T23:20:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:57 smithi174 ceph-mon[35994]: pgmap v703: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 374 KiB/s wr, 50 op/s 2024-06-02T23:20:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:57 smithi052 ceph-mon[32340]: pgmap v703: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 374 KiB/s wr, 50 op/s 2024-06-02T23:20:58.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:57 smithi123 ceph-mon[35977]: pgmap v703: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 374 KiB/s wr, 50 op/s 2024-06-02T23:20:58.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 164043 1.70 MB/sec execute 367 sec latency 12.639 ms 2024-06-02T23:20:59.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 -- 172.21.15.52:0/1574356275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 msgr2=0x7f8c78109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/1574356275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c78109b30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009be0 tx=0x7f8c7002a6c0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 -- 172.21.15.52:0/1574356275 shutdown_connections 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/1574356275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c78109b30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/1574356275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78104db0 0x7f8c781071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/1574356275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78102480 0x7f8c78104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 -- 172.21.15.52:0/1574356275 >> 172.21.15.52:0/1574356275 conn(0x7f8c780fbe70 msgr2=0x7f8c780fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 -- 172.21.15.52:0/1574356275 shutdown_connections 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 -- 172.21.15.52:0/1574356275 wait complete. 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 Processor -- start 2024-06-02T23:20:59.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.221+0000 7f8c7e89e640 1 -- start start 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7e89e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78102480 0x7f8c78197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7e89e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78104db0 0x7f8c781980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7e89e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c7819f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7e89e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7810d550 con 0x7f8c781076e0 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7e89e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7810d3d0 con 0x7f8c78102480 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7e89e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c7810d6d0 con 0x7f8c78104db0 2024-06-02T23:20:59.223 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c7819f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c7819f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37660/0 (socket says 172.21.15.52:37660) 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/331424100 learned_addr learned my addr 172.21.15.52:0/331424100 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c77fff640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78102480 0x7f8c78197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c777fe640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78104db0 0x7f8c781980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/331424100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78104db0 msgr2=0x7f8c781980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78104db0 0x7f8c781980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/331424100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78102480 msgr2=0x7f8c78197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78102480 0x7f8c78197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.222+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/331424100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7819f7b0 con 0x7f8c781076e0 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.223+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c7819f140 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8c70034040 tx=0x7f8c70030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.223+0000 7f8c757fa640 1 -- 172.21.15.52:0/331424100 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70041020 con 0x7f8c781076e0 2024-06-02T23:20:59.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.223+0000 7f8c757fa640 1 -- 172.21.15.52:0/331424100 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c70037040 con 0x7f8c781076e0 2024-06-02T23:20:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.223+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c7819fa40 con 0x7f8c781076e0 2024-06-02T23:20:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.223+0000 7f8c757fa640 1 -- 172.21.15.52:0/331424100 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c70036070 con 0x7f8c781076e0 2024-06-02T23:20:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.223+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c7819ff80 con 0x7f8c781076e0 2024-06-02T23:20:59.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.225+0000 7f8c757fa640 1 -- 172.21.15.52:0/331424100 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c70044020 con 0x7f8c781076e0 2024-06-02T23:20:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 Processor -- start 2024-06-02T23:20:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 -- start start 2024-06-02T23:20:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c757fa640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2cb0 0x7f8c600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7f8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c777fe640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2cb0 0x7f8c600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e7fb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e7564f0 con 0x55fc5e7f3010 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e6f4870 con 0x55fc5e7f8b70 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c815ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e7524b0 con 0x55fc5e7f3a00 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c757fa640 1 -- 172.21.15.52:0/331424100 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8c700efb80 con 0x7f8c781076e0 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7f3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37666/0 (socket says 172.21.15.52:37666) 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 -- 172.21.15.52:0/3126770169 learned_addr learned my addr 172.21.15.52:0/3126770169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c777fe640 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e7fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 -- 172.21.15.52:0/3126770169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 msgr2=0x55fc5e7f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 -- 172.21.15.52:0/3126770169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 msgr2=0x55fc5e7fb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.227+0000 7f8c77fff640 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e7fb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.228+0000 7f8c77fff640 1 -- 172.21.15.52:0/3126770169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5e7109b0 con 0x55fc5e7f3010 2024-06-02T23:20:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.228+0000 7f8c777fe640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2cb0 0x7f8c600a5170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8c781990a0 tx=0x7f8c6c009b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.228+0000 7f8c77fff640 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7f3430 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009d20 tx=0x7f8c68011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5827d85bdf6a21c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.228+0000 7f8c5e7fc640 1 -- 172.21.15.52:0/3126770169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68019620 con 0x55fc5e7f3010 2024-06-02T23:20:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.228+0000 7f8c5e7fc640 1 -- 172.21.15.52:0/3126770169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c68019c70 con 0x55fc5e7f3010 2024-06-02T23:20:59.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.228+0000 7f8c5e7fc640 1 -- 172.21.15.52:0/3126770169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68018c00 con 0x55fc5e7f3010 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 -- 172.21.15.52:0/3126770169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 msgr2=0x55fc5e7f3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7f3430 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009d20 tx=0x7f8c68011810 comp rx=0 tx=0).stop 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 -- 172.21.15.52:0/3126770169 shutdown_connections 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e7fb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3126770169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7f3430 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 -- 172.21.15.52:0/3126770169 >> 172.21.15.52:0/3126770169 conn(0x55fc5e7ee680 msgr2=0x55fc5e7eea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 -- 172.21.15.52:0/3126770169 shutdown_connections 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 -- 172.21.15.52:0/3126770169 wait complete. 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.229+0000 7f8c815ae740 1 -- start start 2024-06-02T23:20:59.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 Processor -- start 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e8047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e7fbb20 con 0x55fc5e7f8b70 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e6dfae0 con 0x55fc5e7f3010 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fc5e7fbca0 con 0x55fc5e7f3a00 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c815ae740 10 client.0 _ll_register_callbacks cb 0x55fc5e7189b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e8047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e8047d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37674/0 (socket says 172.21.15.52:37674) 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/1000411379 learned_addr learned my addr 172.21.15.52:0/1000411379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.230+0000 7f8c77fff640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/1000411379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 msgr2=0x55fc5e7fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3010 msgr2=0x55fc5e7fd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/1000411379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5e46a210 con 0x55fc5e7f8b70 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c777fe640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c77fff640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e8047d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8c700b8de0 tx=0x7f8c700bc2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26fa21d7439f786d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c577fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c577fe640 1 -- 172.21.15.52:0/1000411379 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700bc410 con 0x55fc5e7f8b70 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c68019c70 con 0x55fc5e7f8b70 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c577fe640 1 -- 172.21.15.52:0/1000411379 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8c70003d80 con 0x55fc5e7f8b70 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c577fe640 1 -- 172.21.15.52:0/1000411379 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c700c07c0 con 0x55fc5e7f8b70 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c815ae740 10 client.18078 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.231+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc5e536620 con 0x55fc5e7f8b70 2024-06-02T23:20:59.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c577fe640 1 -- 172.21.15.52:0/1000411379 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c68019c70 con 0x55fc5e7f8b70 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c577fe640 1 -- 172.21.15.52:0/1000411379 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8c70004670 con 0x55fc5e7f8b70 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c815ae740 10 client.18078 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c815ae740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c815ae740 20 client.18078 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c815ae740 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e809100 0x55fc5e80b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c815ae740 4 client.18078 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc5e769f10 con 0x55fc5e809100 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.232+0000 7f8c77fff640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e809100 0x55fc5e80b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.233+0000 7f8c77fff640 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e809100 0x55fc5e80b5a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dd3dba188a1816e server_cookie=72a36881c1b8796a in_seq=0 out_seq=0 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.233+0000 7f8c577fe640 10 client.18078 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.233+0000 7f8c577fe640 1 -- 172.21.15.52:0/1000411379 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fc5e769f10 con 0x55fc5e809100 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.233+0000 7f8c577fe640 10 client.18078 handle_command_reply: tid=0 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.233+0000 7f8c815ae740 1 client.18078 shutdown 2024-06-02T23:20:59.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 msgr2=0x55fc5e8047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e8047d0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8c700b8de0 tx=0x7f8c700bc2a0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 shutdown_connections 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e809100 0x55fc5e80b5a0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e8047d0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 --2- 172.21.15.52:0/1000411379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 >> 172.21.15.52:0/1000411379 conn(0x55fc5e7ee680 msgr2=0x55fc5e7ed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c577fe640 0 client.18078 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 shutdown_connections 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 1 -- 172.21.15.52:0/1000411379 wait complete. 2024-06-02T23:20:59.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.234+0000 7f8c815ae740 20 client.18078 trim_cache size 0 max 16384 2024-06-02T23:20:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 Processor -- start 2024-06-02T23:20:59.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 -- start start 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e804790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781a8bd0 con 0x55fc5e7f3010 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781aa710 con 0x55fc5e7f8b70 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c815ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781a92a0 con 0x55fc5e7f3a00 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e804790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c77fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37688/0 (socket says 172.21.15.52:37688) 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.253+0000 7f8c77fff640 1 -- 172.21.15.52:0/3438695089 learned_addr learned my addr 172.21.15.52:0/3438695089 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c777fe640 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c77fff640 1 -- 172.21.15.52:0/3438695089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 msgr2=0x55fc5e7fd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c77fff640 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c77fff640 1 -- 172.21.15.52:0/3438695089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 msgr2=0x55fc5e804790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c77fff640 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e804790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c77fff640 1 -- 172.21.15.52:0/3438695089 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc5e7e8ae0 con 0x55fc5e7f3010 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e804790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c77fff640 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800a010 tx=0x7f8c68005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31653a7d14329bc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c5dffb640 1 -- 172.21.15.52:0/3438695089 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68018670 con 0x55fc5e7f3010 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c5dffb640 1 -- 172.21.15.52:0/3438695089 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c68018d80 con 0x55fc5e7f3010 2024-06-02T23:20:59.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.254+0000 7f8c5dffb640 1 -- 172.21.15.52:0/3438695089 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68003ab0 con 0x55fc5e7f3010 2024-06-02T23:20:59.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 -- 172.21.15.52:0/3438695089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 msgr2=0x55fc5e7fd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800a010 tx=0x7f8c68005f90 comp rx=0 tx=0).stop 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 -- 172.21.15.52:0/3438695089 shutdown_connections 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3a00 0x55fc5e7fd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f8b70 0x55fc5e804790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 --2- 172.21.15.52:0/3438695089 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f3010 0x55fc5e7fd250 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 -- 172.21.15.52:0/3438695089 >> 172.21.15.52:0/3438695089 conn(0x55fc5e7ee680 msgr2=0x55fc5e7f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 -- 172.21.15.52:0/3438695089 shutdown_connections 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 -- 172.21.15.52:0/3438695089 wait complete. 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.255+0000 7f8c815ae740 1 -- start start 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 Processor -- start 2024-06-02T23:20:59.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3010 0x55fc5e8040d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3a00 0x55fc5e804610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e800cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781aa040 con 0x55fc5e7f8b70 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781a9970 con 0x55fc5e7f3a00 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781a6bb0 con 0x55fc5e7f3010 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c815ae740 10 client.0 _ll_register_callbacks cb 0x7f8c700b8050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e800cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e800cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37698/0 (socket says 172.21.15.52:37698) 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/687717955 learned_addr learned my addr 172.21.15.52:0/687717955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c777fe640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3a00 0x55fc5e804610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c77fff640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3010 0x55fc5e8040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/687717955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3010 msgr2=0x55fc5e8040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3010 0x55fc5e8040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3a00 msgr2=0x55fc5e804610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3a00 0x55fc5e804610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c7ce3b640 1 -- 172.21.15.52:0/687717955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc5e801460 con 0x55fc5e7f8b70 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c777fe640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3a00 0x55fc5e804610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.256+0000 7f8c77fff640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3010 0x55fc5e8040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c7ce3b640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e800cf0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8c700bc110 tx=0x7f8c70009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2f9f7e9065986b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c577fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c577fe640 1 -- 172.21.15.52:0/687717955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c700a3060 con 0x55fc5e7f8b70 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c577fe640 1 -- 172.21.15.52:0/687717955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8c700edb70 con 0x55fc5e7f8b70 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c68018d80 con 0x55fc5e7f8b70 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c577fe640 1 -- 172.21.15.52:0/687717955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8c700bc410 con 0x55fc5e7f8b70 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 10 client.18084 fetch_fsmap learned FSMap version 32 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55fc5e872b60 con 0x55fc5e7f8b70 2024-06-02T23:20:59.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c577fe640 1 -- 172.21.15.52:0/687717955 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c68018d80 con 0x55fc5e7f8b70 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c577fe640 1 -- 172.21.15.52:0/687717955 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8c700bb860 con 0x55fc5e7f8b70 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 10 client.18084 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 10 client.18084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 20 client.18084 populate_metadata read hostname 'smithi052' 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e873350 0x55fc5e875770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 4 client.18084 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.257+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc5e763c20 con 0x55fc5e873350 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.258+0000 7f8c77fff640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e873350 0x55fc5e875770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.258+0000 7f8c577fe640 10 client.18084 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:59.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.258+0000 7f8c77fff640 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e873350 0x55fc5e875770 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad8ef637535451c3 server_cookie=f4132d1a5da365a8 in_seq=0 out_seq=0 2024-06-02T23:20:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.277+0000 7f8c577fe640 1 -- 172.21.15.52:0/687717955 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fc5e763c20 con 0x55fc5e873350 2024-06-02T23:20:59.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.277+0000 7f8c577fe640 10 client.18084 handle_command_reply: tid=0 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.277+0000 7f8c815ae740 1 client.18084 shutdown 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 msgr2=0x55fc5e800cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e800cf0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8c700bc110 tx=0x7f8c70009500 comp rx=0 tx=0).stop 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 shutdown_connections 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fc5e873350 0x55fc5e875770 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fc5e7f8b70 0x55fc5e800cf0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fc5e7f3a00 0x55fc5e804610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 --2- 172.21.15.52:0/687717955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fc5e7f3010 0x55fc5e8040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c577fe640 0 client.18084 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 >> 172.21.15.52:0/687717955 conn(0x55fc5e7ee680 msgr2=0x55fc5e7f4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 shutdown_connections 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 1 -- 172.21.15.52:0/687717955 wait complete. 2024-06-02T23:20:59.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c815ae740 20 client.18084 trim_cache size 0 max 16384 2024-06-02T23:20:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:20:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:20:59.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2cb0 msgr2=0x7f8c600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.278+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2cb0 0x7f8c600a5170 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8c781990a0 tx=0x7f8c6c009b10 comp rx=0 tx=0).stop 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 msgr2=0x7f8c7819f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c7819f140 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f8c70034040 tx=0x7f8c70030f90 comp rx=0 tx=0).stop 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 shutdown_connections 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2cb0 0x7f8c600a5170 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78104db0 0x7f8c781980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c781076e0 0x7f8c7819f140 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 --2- 172.21.15.52:0/331424100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78102480 0x7f8c78197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 >> 172.21.15.52:0/331424100 conn(0x7f8c780fbe70 msgr2=0x7f8c78102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:20:59.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 shutdown_connections 2024-06-02T23:20:59.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:20:59.279+0000 7f8c7e89e640 1 -- 172.21.15.52:0/331424100 wait complete. 2024-06-02T23:20:59.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 164574 1.71 MB/sec execute 368 sec latency 17.903 ms 2024-06-02T23:20:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:20:59 smithi174 ceph-mon[35994]: pgmap v704: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-06-02T23:20:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:20:59 smithi052 ceph-mon[32340]: pgmap v704: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-06-02T23:21:00.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:20:59 smithi123 ceph-mon[35977]: pgmap v704: 97 pgs: 97 active+clean; 73 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 478 KiB/s wr, 54 op/s 2024-06-02T23:21:00.639 INFO:tasks.workunit.client.0.smithi052.stdout: 1 165162 1.71 MB/sec execute 369 sec latency 14.895 ms 2024-06-02T23:21:01.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 165731 1.71 MB/sec execute 370 sec latency 12.757 ms 2024-06-02T23:21:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:01 smithi174 ceph-mon[35994]: pgmap v705: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 778 KiB/s wr, 88 op/s 2024-06-02T23:21:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:01 smithi052 ceph-mon[32340]: pgmap v705: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 778 KiB/s wr, 88 op/s 2024-06-02T23:21:02.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:01 smithi123 ceph-mon[35977]: pgmap v705: 97 pgs: 97 active+clean; 75 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 778 KiB/s wr, 88 op/s 2024-06-02T23:21:02.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 166471 1.72 MB/sec execute 371 sec latency 13.973 ms 2024-06-02T23:21:03.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 166892 1.73 MB/sec execute 372 sec latency 15.503 ms 2024-06-02T23:21:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:03 smithi174 ceph-mon[35994]: pgmap v706: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 957 KiB/s wr, 107 op/s 2024-06-02T23:21:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:03 smithi052 ceph-mon[32340]: pgmap v706: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 957 KiB/s wr, 107 op/s 2024-06-02T23:21:04.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:03 smithi123 ceph-mon[35977]: pgmap v706: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 957 KiB/s wr, 107 op/s 2024-06-02T23:21:04.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 167166 1.72 MB/sec execute 373 sec latency 12.868 ms 2024-06-02T23:21:05.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 167509 1.72 MB/sec execute 374 sec latency 63.888 ms 2024-06-02T23:21:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:05 smithi174 ceph-mon[35994]: pgmap v707: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 991 KiB/s wr, 118 op/s 2024-06-02T23:21:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:05 smithi052 ceph-mon[32340]: pgmap v707: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 991 KiB/s wr, 118 op/s 2024-06-02T23:21:06.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:05 smithi123 ceph-mon[35977]: pgmap v707: 97 pgs: 97 active+clean; 74 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 991 KiB/s wr, 118 op/s 2024-06-02T23:21:06.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 168017 1.72 MB/sec execute 375 sec latency 14.655 ms 2024-06-02T23:21:07.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 168652 1.73 MB/sec execute 376 sec latency 14.693 ms 2024-06-02T23:21:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:07 smithi174 ceph-mon[35994]: pgmap v708: 97 pgs: 97 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-02T23:21:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:07 smithi052 ceph-mon[32340]: pgmap v708: 97 pgs: 97 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-02T23:21:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:07 smithi123 ceph-mon[35977]: pgmap v708: 97 pgs: 97 active+clean; 76 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.4 MiB/s wr, 151 op/s 2024-06-02T23:21:08.601 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:21:08.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 169219 1.73 MB/sec execute 377 sec latency 15.026 ms 2024-06-02T23:21:08.881 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:09.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 169996 1.74 MB/sec execute 378 sec latency 14.256 ms 2024-06-02T23:21:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:09 smithi174 ceph-mon[35994]: pgmap v709: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-06-02T23:21:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:09 smithi052 ceph-mon[32340]: pgmap v709: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-06-02T23:21:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:09 smithi123 ceph-mon[35977]: pgmap v709: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.2 MiB/s wr, 132 op/s 2024-06-02T23:21:10.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 170370 1.74 MB/sec execute 379 sec latency 15.143 ms 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 -- 172.21.15.52:0/1869627702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56601084f0 msgr2=0x7f56601088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 --2- 172.21.15.52:0/1869627702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56601084f0 0x7f56601088d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f565c009990 tx=0x7f565c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 -- 172.21.15.52:0/1869627702 shutdown_connections 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 --2- 172.21.15.52:0/1869627702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f566006b3c0 0x7f566010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 --2- 172.21.15.52:0/1869627702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f56601084f0 0x7f56601088d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 --2- 172.21.15.52:0/1869627702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006aa00 0x7f566006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 -- 172.21.15.52:0/1869627702 >> 172.21.15.52:0/1869627702 conn(0x7f5660078300 msgr2=0x7f5660078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:10.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 -- 172.21.15.52:0/1869627702 shutdown_connections 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.833+0000 7f56681b7700 1 -- 172.21.15.52:0/1869627702 wait complete. 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 Processor -- start 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 -- start start 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 0x7f566019e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b3c0 0x7f566019eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56601084f0 0x7f5660198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112680 con 0x7f566006aa00 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112500 con 0x7f566006b3c0 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f56681b7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5660112800 con 0x7f56601084f0 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f5665f53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 0x7f566019e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f5665f53700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 0x7f566019e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60010/0 (socket says 172.21.15.52:60010) 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f5665f53700 1 -- 172.21.15.52:0/1809922901 learned_addr learned my addr 172.21.15.52:0/1809922901 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:10.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.834+0000 7f5665752700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b3c0 0x7f566019eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5666754700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56601084f0 0x7f5660198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665f53700 1 -- 172.21.15.52:0/1809922901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56601084f0 msgr2=0x7f5660198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665f53700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56601084f0 0x7f5660198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665f53700 1 -- 172.21.15.52:0/1809922901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b3c0 msgr2=0x7f566019eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665f53700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b3c0 0x7f566019eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665f53700 1 -- 172.21.15.52:0/1809922901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5660199180 con 0x7f566006aa00 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665752700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b3c0 0x7f566019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5665f53700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 0x7f566019e860 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f565c003ab0 tx=0x7f565c003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5656ffd700 1 -- 172.21.15.52:0/1809922901 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c027070 con 0x7f566006aa00 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5660199410 con 0x7f566006aa00 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5656ffd700 1 -- 172.21.15.52:0/1809922901 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565c00bdc0 con 0x7f566006aa00 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f5656ffd700 1 -- 172.21.15.52:0/1809922901 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c022bf0 con 0x7f566006aa00 2024-06-02T23:21:10.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.835+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5660101380 con 0x7f566006aa00 2024-06-02T23:21:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.838+0000 7f5656ffd700 1 -- 172.21.15.52:0/1809922901 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f565c02a070 con 0x7f566006aa00 2024-06-02T23:21:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.839+0000 7f5656ffd700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f564c0a7030 0x7f564c0a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.839+0000 7f5665752700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f564c0a7030 0x7f564c0a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.839+0000 7f5656ffd700 1 -- 172.21.15.52:0/1809922901 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f565c0cfed0 con 0x7f566006aa00 2024-06-02T23:21:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.839+0000 7f5665752700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f564c0a7030 0x7f564c0a94f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5660078940 tx=0x7f56500073d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:10.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 Processor -- start 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 -- start start 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff64260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff64bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffe56f60 con 0x55b5fff6a880 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffeb5850 con 0x55b5fff647a0 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffe1d460 con 0x55b5fff69f60 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5666754700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff64bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5665752700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff64260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5666754700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff64bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50742/0 (socket says 172.21.15.52:50742) 2024-06-02T23:21:10.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5666754700 1 -- 172.21.15.52:0/3892235826 learned_addr learned my addr 172.21.15.52:0/3892235826 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.840+0000 7f5665f53700 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f5666754700 1 -- 172.21.15.52:0/3892235826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 msgr2=0x55b5fff6a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f5666754700 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f5666754700 1 -- 172.21.15.52:0/3892235826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 msgr2=0x55b5fff64260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f5666754700 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff64260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f5666754700 1 -- 172.21.15.52:0/3892235826 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5ffe048d0 con 0x55b5fff647a0 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f5666754700 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff64bc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f565800a430 tx=0x7f5658009650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72dd4a6471322a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f564bfff700 1 -- 172.21.15.52:0/3892235826 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658014620 con 0x55b5fff647a0 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f564bfff700 1 -- 172.21.15.52:0/3892235826 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5658019070 con 0x55b5fff647a0 2024-06-02T23:21:10.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.841+0000 7f564bfff700 1 -- 172.21.15.52:0/3892235826 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658013aa0 con 0x55b5fff647a0 2024-06-02T23:21:10.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3892235826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 msgr2=0x55b5fff64bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff64bc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f565800a430 tx=0x7f5658009650 comp rx=0 tx=0).stop 2024-06-02T23:21:10.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3892235826 shutdown_connections 2024-06-02T23:21:10.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff64bc0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3892235826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff64260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3892235826 >> 172.21.15.52:0/3892235826 conn(0x55b5ffe91080 msgr2=0x55b5fff5f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3892235826 shutdown_connections 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.842+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3892235826 wait complete. 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 -- start start 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 Processor -- start 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff6deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffe55fc0 con 0x55b5fff6a880 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b5ffeb1810 con 0x55b5fff647a0 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5fff71260 con 0x55b5fff69f60 2024-06-02T23:21:10.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5672a3ab80 10 client.0 _ll_register_callbacks cb 0x55b5ffe4b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5666754700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff6deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5666754700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff6deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60022/0 (socket says 172.21.15.52:60022) 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5666754700 1 -- 172.21.15.52:0/2307565737 learned_addr learned my addr 172.21.15.52:0/2307565737 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5665f53700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff70670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.843+0000 7f5665752700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5666754700 1 -- 172.21.15.52:0/2307565737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 msgr2=0x55b5fff6d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5666754700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5666754700 1 -- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 msgr2=0x55b5fff70670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5666754700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff70670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5666754700 1 -- 172.21.15.52:0/2307565737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5ffe7b750 con 0x55b5fff6a880 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5666754700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff6deb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f56580136c0 tx=0x7f5658013950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b7de335ddaabaa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f56497fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:10.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f56497fa700 1 -- 172.21.15.52:0/2307565737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658027470 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f56497fa700 1 -- 172.21.15.52:0/2307565737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5658027610 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5658019070 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f56497fa700 1 -- 172.21.15.52:0/2307565737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5658027810 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5672a3ab80 10 client.18096 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.844+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b5fff7ba60 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f56497fa700 1 -- 172.21.15.52:0/2307565737 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5658019070 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f56497fa700 1 -- 172.21.15.52:0/2307565737 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f565801d8d0 con 0x55b5fff6a880 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5672a3ab80 10 client.18096 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5672a3ab80 10 client.18096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5672a3ab80 20 client.18096 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fff7e0a0 0x55b5fff80490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5672a3ab80 4 client.18096 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5ffec2d50 con 0x55b5fff7e0a0 2024-06-02T23:21:10.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5665f53700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fff7e0a0 0x55b5fff80490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f56497fa700 10 client.18096 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.845+0000 7f5665f53700 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fff7e0a0 0x55b5fff80490 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3def034f3604a32 server_cookie=c27c77bde946bf7 in_seq=0 out_seq=0 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.846+0000 7f56497fa700 1 -- 172.21.15.52:0/2307565737 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b5ffec2d50 con 0x55b5fff7e0a0 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.846+0000 7f56497fa700 10 client.18096 handle_command_reply: tid=0 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.846+0000 7f5672a3ab80 1 client.18096 shutdown 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 msgr2=0x55b5fff6deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff6deb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f56580136c0 tx=0x7f5658013950 comp rx=0 tx=0).stop 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 shutdown_connections 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fff7e0a0 0x55b5fff80490 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff6deb0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff647a0 0x55b5fff70670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/2307565737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff69f60 0x55b5fff6d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f56497fa700 0 client.18096 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:10.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 >> 172.21.15.52:0/2307565737 conn(0x55b5ffe91080 msgr2=0x55b5fff5e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:10.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 shutdown_connections 2024-06-02T23:21:10.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 1 -- 172.21.15.52:0/2307565737 wait complete. 2024-06-02T23:21:10.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.847+0000 7f5672a3ab80 20 client.18096 trim_cache size 0 max 16384 2024-06-02T23:21:10.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.870+0000 7f5672a3ab80 1 Processor -- start 2024-06-02T23:21:10.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.870+0000 7f5672a3ab80 1 -- start start 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.870+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff64260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.870+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 0x55b5fff6b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.870+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff5ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56601aa7d0 con 0x55b5fff6a880 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56601ac310 con 0x55b5fff69f60 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56601aaea0 con 0x55b5fff647a0 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5665752700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 0x55b5fff6b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff5ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5665752700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 0x55b5fff6b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50756/0 (socket says 172.21.15.52:50756) 2024-06-02T23:21:10.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5665752700 1 -- 172.21.15.52:0/1987041594 learned_addr learned my addr 172.21.15.52:0/1987041594 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5665f53700 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff64260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 -- 172.21.15.52:0/1987041594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 msgr2=0x55b5fff64260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff64260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 -- 172.21.15.52:0/1987041594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 msgr2=0x55b5fff6b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 0x55b5fff6b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 -- 172.21.15.52:0/1987041594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5fff6bf00 con 0x55b5fff6a880 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5665752700 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 0x55b5fff6b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.871+0000 7f5666754700 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff5ee80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f56580070b0 tx=0x7f5658009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d3d594b786e2407 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.872+0000 7f564b7fe700 1 -- 172.21.15.52:0/1987041594 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658009a50 con 0x55b5fff6a880 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.872+0000 7f564b7fe700 1 -- 172.21.15.52:0/1987041594 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565800e070 con 0x55b5fff6a880 2024-06-02T23:21:10.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.872+0000 7f564b7fe700 1 -- 172.21.15.52:0/1987041594 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565801d420 con 0x55b5fff6a880 2024-06-02T23:21:10.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.872+0000 7f5672a3ab80 1 -- 172.21.15.52:0/1987041594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 msgr2=0x55b5fff5ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.872+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff5ee80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f56580070b0 tx=0x7f5658009650 comp rx=0 tx=0).stop 2024-06-02T23:21:10.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 -- 172.21.15.52:0/1987041594 shutdown_connections 2024-06-02T23:21:10.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff6a880 0x55b5fff5ee80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff64260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/1987041594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff69f60 0x55b5fff6b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 -- 172.21.15.52:0/1987041594 >> 172.21.15.52:0/1987041594 conn(0x55b5ffe91080 msgr2=0x55b5fff6fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 -- 172.21.15.52:0/1987041594 shutdown_connections 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 -- 172.21.15.52:0/1987041594 wait complete. 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 -- start start 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.873+0000 7f5672a3ab80 1 Processor -- start 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff811b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 0x55b5fff816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff6a880 0x55b5fff81c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f56601abc40 con 0x55b5fff69f60 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56601ab570 con 0x55b5fff6a880 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f56601a8790 con 0x55b5fff647a0 2024-06-02T23:21:10.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5672a3ab80 10 client.0 _ll_register_callbacks cb 0x55b5ffe4b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 0x55b5fff816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 0x55b5fff816f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60034/0 (socket says 172.21.15.52:60034) 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 -- 172.21.15.52:0/3253915693 learned_addr learned my addr 172.21.15.52:0/3253915693 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665f53700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff811b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5666754700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff6a880 0x55b5fff81c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 -- 172.21.15.52:0/3253915693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 msgr2=0x55b5fff811b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff811b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 -- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff6a880 msgr2=0x55b5fff81c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff6a880 0x55b5fff81c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 -- 172.21.15.52:0/3253915693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5fffe79e0 con 0x55b5fff69f60 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f5665752700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 0x55b5fff816f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f56500036d0 tx=0x7f56500047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785b1230e4a418dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.874+0000 7f56497fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f56497fa700 1 -- 172.21.15.52:0/3253915693 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5650003de0 con 0x55b5fff69f60 2024-06-02T23:21:10.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f56497fa700 1 -- 172.21.15.52:0/3253915693 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5650011440 con 0x55b5fff69f60 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f565800e070 con 0x55b5fff69f60 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f56497fa700 1 -- 172.21.15.52:0/3253915693 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f5650032070 con 0x55b5fff69f60 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f5672a3ab80 10 client.18102 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b5fffe8cc0 con 0x55b5fff69f60 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.875+0000 7f56497fa700 1 -- 172.21.15.52:0/3253915693 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f565800e070 con 0x55b5fff69f60 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f56497fa700 1 -- 172.21.15.52:0/3253915693 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f565002b7a0 con 0x55b5fff69f60 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5672a3ab80 10 client.18102 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5672a3ab80 10 client.18102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5672a3ab80 20 client.18102 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fffe9430 0x55b5fffeb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5672a3ab80 4 client.18102 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5ffebd2d0 con 0x55b5fffe9430 2024-06-02T23:21:10.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5665f53700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fffe9430 0x55b5fffeb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f5665f53700 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fffe9430 0x55b5fffeb850 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac5d39478758a22 server_cookie=1aee856e573a7ab8 in_seq=0 out_seq=0 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.876+0000 7f56497fa700 10 client.18102 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f56497fa700 1 -- 172.21.15.52:0/3253915693 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b5ffebd2d0 con 0x55b5fffe9430 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f56497fa700 10 client.18102 handle_command_reply: tid=0 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 client.18102 shutdown 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 msgr2=0x55b5fff816f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 0x55b5fff816f0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f56500036d0 tx=0x7f56500047c0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 shutdown_connections 2024-06-02T23:21:10.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b5fffe9430 0x55b5fffeb850 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b5fff6a880 0x55b5fff81c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5fff647a0 0x55b5fff811b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f56497fa700 0 client.18102 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 --2- 172.21.15.52:0/3253915693 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b5fff69f60 0x55b5fff816f0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 >> 172.21.15.52:0/3253915693 conn(0x55b5ffe91080 msgr2=0x55b5fff6fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 shutdown_connections 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 1 -- 172.21.15.52:0/3253915693 wait complete. 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.877+0000 7f5672a3ab80 20 client.18102 trim_cache size 0 max 16384 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:21:10.882 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:21:10.883 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:21:10.883 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:21:10.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f564c0a7030 msgr2=0x7f564c0a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f564c0a7030 0x7f564c0a94f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f5660078940 tx=0x7f56500073d0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 msgr2=0x7f566019e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:10.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 0x7f566019e860 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f565c003ab0 tx=0x7f565c003ae0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 shutdown_connections 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f564c0a7030 0x7f564c0a94f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f56601084f0 0x7f5660198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f566006aa00 0x7f566019e860 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 --2- 172.21.15.52:0/1809922901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f566006b3c0 0x7f566019eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 >> 172.21.15.52:0/1809922901 conn(0x7f5660078300 msgr2=0x7f5660100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 shutdown_connections 2024-06-02T23:21:10.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:10.878+0000 7f56681b7700 1 -- 172.21.15.52:0/1809922901 wait complete. 2024-06-02T23:21:11.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 170656 1.74 MB/sec execute 380 sec latency 15.709 ms 2024-06-02T23:21:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:11 smithi052 ceph-mon[32340]: pgmap v710: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-02T23:21:11.770 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:21:11.770 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:21:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:11 smithi174 ceph-mon[35994]: pgmap v710: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-02T23:21:12.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:11 smithi123 ceph-mon[35977]: pgmap v710: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 306 KiB/s rd, 1.3 MiB/s wr, 173 op/s 2024-06-02T23:21:12.084 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 -- 172.21.15.52:0/3416375057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34741024b0 msgr2=0x7f3474108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 --2- 172.21.15.52:0/3416375057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34741024b0 0x7f3474108d40 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f346c009ca0 tx=0x7f346c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 -- 172.21.15.52:0/3416375057 shutdown_connections 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 --2- 172.21.15.52:0/3416375057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3474101af0 0x7f3474101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 --2- 172.21.15.52:0/3416375057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f34741024b0 0x7f3474108d40 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 --2- 172.21.15.52:0/3416375057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474100900 0x7f3474100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 -- 172.21.15.52:0/3416375057 >> 172.21.15.52:0/3416375057 conn(0x7f34740fbe70 msgr2=0x7f34740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 -- 172.21.15.52:0/3416375057 shutdown_connections 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 -- 172.21.15.52:0/3416375057 wait complete. 2024-06-02T23:21:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.381+0000 7f3479f22640 1 Processor -- start 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 -- start start 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 0x7f3474197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474101af0 0x7f34741980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741024b0 0x7f347419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f347410c410 con 0x7f3474100900 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f347410c290 con 0x7f3474101af0 2024-06-02T23:21:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3479f22640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f347410c590 con 0x7f34741024b0 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 0x7f3474197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 0x7f3474197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60058/0 (socket says 172.21.15.52:60058) 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 -- 172.21.15.52:0/321968087 learned_addr learned my addr 172.21.15.52:0/321968087 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3472ffd640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474101af0 0x7f34741980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3473fff640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741024b0 0x7f347419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 -- 172.21.15.52:0/321968087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741024b0 msgr2=0x7f347419f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741024b0 0x7f347419f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 -- 172.21.15.52:0/321968087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474101af0 msgr2=0x7f34741980c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474101af0 0x7f34741980c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 -- 172.21.15.52:0/321968087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f347419f750 con 0x7f3474100900 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f3472ffd640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474101af0 0x7f34741980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.382+0000 7f34737fe640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 0x7f3474197b80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f346400e9c0 tx=0x7f346400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.383+0000 7f3470ff9640 1 -- 172.21.15.52:0/321968087 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346400cde0 con 0x7f3474100900 2024-06-02T23:21:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.383+0000 7f3470ff9640 1 -- 172.21.15.52:0/321968087 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f34640108e0 con 0x7f3474100900 2024-06-02T23:21:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.383+0000 7f3470ff9640 1 -- 172.21.15.52:0/321968087 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3464018990 con 0x7f3474100900 2024-06-02T23:21:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.383+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f347419fa40 con 0x7f3474100900 2024-06-02T23:21:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.383+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347419ffe0 con 0x7f3474100900 2024-06-02T23:21:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f3470ff9640 1 -- 172.21.15.52:0/321968087 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f346401f020 con 0x7f3474100900 2024-06-02T23:21:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 Processor -- start 2024-06-02T23:21:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 -- start start 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c6247c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62481500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c62484000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562c623df4f0 con 0x562c6247ca00 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562c6237d870 con 0x562c6247c010 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f347cc33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562c623db4b0 con 0x562c62481b70 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f34737fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62481500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f3473fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c6247c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f3473fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c6247c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50784/0 (socket says 172.21.15.52:50784) 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f34737fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62481500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60064/0 (socket says 172.21.15.52:60064) 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f3473fff640 1 -- 172.21.15.52:0/2812088204 learned_addr learned my addr 172.21.15.52:0/2812088204 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.386+0000 7f3472ffd640 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c62484000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3473fff640 1 -- 172.21.15.52:0/2812088204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 msgr2=0x562c62484000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3473fff640 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c62484000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3473fff640 1 -- 172.21.15.52:0/2812088204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 msgr2=0x562c62481500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3473fff640 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62481500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3473fff640 1 -- 172.21.15.52:0/2812088204 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c623999b0 con 0x562c6247c010 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3472ffd640 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c62484000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3470ff9640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f345c0a2cb0 0x7f345c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f34737fe640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f345c0a2cb0 0x7f345c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3473fff640 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c6247c430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f346c009ca0 tx=0x7f346c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5698f31d4cd7643 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3470ff9640 1 -- 172.21.15.52:0/321968087 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f34640cf150 con 0x7f3474100900 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3459ffb640 1 -- 172.21.15.52:0/2812088204 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c031650 con 0x562c6247c010 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3459ffb640 1 -- 172.21.15.52:0/2812088204 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f346c036030 con 0x562c6247c010 2024-06-02T23:21:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.387+0000 7f3459ffb640 1 -- 172.21.15.52:0/2812088204 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346c004440 con 0x562c6247c010 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 -- 172.21.15.52:0/2812088204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 msgr2=0x562c6247c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f34737fe640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f345c0a2cb0 0x7f345c0a5170 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f34640ccff0 tx=0x7f34640d2c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c6247c430 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f346c009ca0 tx=0x7f346c009a20 comp rx=0 tx=0).stop 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 -- 172.21.15.52:0/2812088204 shutdown_connections 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c62484000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62481500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 --2- 172.21.15.52:0/2812088204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c6247c430 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 -- 172.21.15.52:0/2812088204 >> 172.21.15.52:0/2812088204 conn(0x562c62477680 msgr2=0x562c62477a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 -- 172.21.15.52:0/2812088204 shutdown_connections 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 -- 172.21.15.52:0/2812088204 wait complete. 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.388+0000 7f347cc33740 1 -- start start 2024-06-02T23:21:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 Processor -- start 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c62486130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62486670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c6248d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562c62484940 con 0x562c6247ca00 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562c62368ae0 con 0x562c6247c010 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562c62484ac0 con 0x562c62481b70 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f347cc33740 10 client.0 _ll_register_callbacks cb 0x562c623a19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62486670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62486670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60070/0 (socket says 172.21.15.52:60070) 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 -- 172.21.15.52:0/1864486334 learned_addr learned my addr 172.21.15.52:0/1864486334 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3473fff640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c62486130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3473fff640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c6248d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.394 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 -- 172.21.15.52:0/1864486334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 msgr2=0x562c6248d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c6248d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 -- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 msgr2=0x562c62486130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c62486130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 -- 172.21.15.52:0/1864486334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c620f3210 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3472ffd640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62486670 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f34680129f0 tx=0x7f3468012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeb0e128e294f07 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.389+0000 7f3452ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f3452ffd640 1 -- 172.21.15.52:0/1864486334 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f346800f9e0 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f3452ffd640 1 -- 172.21.15.52:0/1864486334 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f346800fb80 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f346c036030 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f3452ffd640 1 -- 172.21.15.52:0/1864486334 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f346800fd80 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 10 client.18114 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c621bf620 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f3452ffd640 1 -- 172.21.15.52:0/1864486334 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f346c036030 con 0x562c6247ca00 2024-06-02T23:21:12.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f3452ffd640 1 -- 172.21.15.52:0/1864486334 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f34680104c0 con 0x562c6247ca00 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 10 client.18114 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 10 client.18114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 20 client.18114 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c62492000 0x562c62494460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 4 client.18114 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c623f2f10 con 0x562c62492000 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.390+0000 7f3473fff640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c62492000 0x562c62494460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.391+0000 7f3473fff640 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c62492000 0x562c62494460 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed696a9792081bf4 server_cookie=ac883886144fa6e6 in_seq=0 out_seq=0 2024-06-02T23:21:12.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.391+0000 7f3452ffd640 10 client.18114 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f3452ffd640 1 -- 172.21.15.52:0/1864486334 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562c623f2f10 con 0x562c62492000 2024-06-02T23:21:12.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f3452ffd640 10 client.18114 handle_command_reply: tid=0 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 client.18114 shutdown 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 msgr2=0x562c62486670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62486670 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f34680129f0 tx=0x7f3468012eb0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 shutdown_connections 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c62492000 0x562c62494460 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c62481b70 0x562c6248d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c62486670 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f3452ffd640 0 client.18114 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 --2- 172.21.15.52:0/1864486334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247c010 0x562c62486130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 >> 172.21.15.52:0/1864486334 conn(0x562c62477680 msgr2=0x562c624767e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 shutdown_connections 2024-06-02T23:21:12.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 1 -- 172.21.15.52:0/1864486334 wait complete. 2024-06-02T23:21:12.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.392+0000 7f347cc33740 20 client.18114 trim_cache size 0 max 16384 2024-06-02T23:21:12.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 Processor -- start 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 -- start start 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c62483f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 0x562c6248d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c62481b70 0x562c6247a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f34741a8bb0 con 0x562c62481b70 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34741aa6f0 con 0x562c6247ca00 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f347cc33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34741a9280 con 0x562c6247c010 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 0x562c6248d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f3473fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c62481b70 0x562c6247a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 0x562c6248d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50808/0 (socket says 172.21.15.52:50808) 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.410+0000 7f3472ffd640 1 -- 172.21.15.52:0/3953665755 learned_addr learned my addr 172.21.15.52:0/3953665755 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:12.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3473fff640 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c62483f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3472ffd640 1 -- 172.21.15.52:0/3953665755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 msgr2=0x562c62483f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3472ffd640 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c62483f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3472ffd640 1 -- 172.21.15.52:0/3953665755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c62481b70 msgr2=0x562c6247a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3472ffd640 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c62481b70 0x562c6247a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3472ffd640 1 -- 172.21.15.52:0/3953665755 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c6247d290 con 0x562c6247ca00 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f3472ffd640 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 0x562c6248d600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3468012420 tx=0x7f3468009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f24b9b7aba6ead3a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f34597fa640 1 -- 172.21.15.52:0/3953665755 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468010ba0 con 0x562c6247ca00 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f34597fa640 1 -- 172.21.15.52:0/3953665755 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3468010d40 con 0x562c6247ca00 2024-06-02T23:21:12.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.411+0000 7f34597fa640 1 -- 172.21.15.52:0/3953665755 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468005430 con 0x562c6247ca00 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 -- 172.21.15.52:0/3953665755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 msgr2=0x562c6248d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 0x562c6248d600 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3468012420 tx=0x7f3468009b50 comp rx=0 tx=0).stop 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 -- 172.21.15.52:0/3953665755 shutdown_connections 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c62481b70 0x562c6247a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c6247ca00 0x562c6248d600 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 --2- 172.21.15.52:0/3953665755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c62483f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 -- 172.21.15.52:0/3953665755 >> 172.21.15.52:0/3953665755 conn(0x562c62477680 msgr2=0x562c6247dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:12.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 -- 172.21.15.52:0/3953665755 shutdown_connections 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 -- 172.21.15.52:0/3953665755 wait complete. 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 -- start start 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.412+0000 7f347cc33740 1 Processor -- start 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c6248a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c6248ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c62481b70 0x562c6248bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f34741aa020 con 0x562c6247ca00 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f34741a9950 con 0x562c62481b70 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f34741a6b90 con 0x562c6247c010 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f347cc33740 10 client.0 _ll_register_callbacks cb 0x7f3468009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c6248ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c6248ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60076/0 (socket says 172.21.15.52:60076) 2024-06-02T23:21:12.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 -- 172.21.15.52:0/171304335 learned_addr learned my addr 172.21.15.52:0/171304335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3473fff640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c6248a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3473fff640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c62481b70 0x562c6248bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 -- 172.21.15.52:0/171304335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 msgr2=0x562c6248a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c6248a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 -- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c62481b70 msgr2=0x562c6248bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c62481b70 0x562c6248bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 -- 172.21.15.52:0/171304335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c6248c260 con 0x562c6247ca00 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3472ffd640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c6248ac60 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f34680109b0 tx=0x7f346800fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ca4e749aa071f05 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3452ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.413+0000 7f3452ffd640 1 -- 172.21.15.52:0/171304335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3468035060 con 0x562c6247ca00 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f3452ffd640 1 -- 172.21.15.52:0/171304335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f34680055c0 con 0x562c6247ca00 2024-06-02T23:21:12.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3468010d40 con 0x562c6247ca00 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f3452ffd640 1 -- 172.21.15.52:0/171304335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3468010b50 con 0x562c6247ca00 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 10 client.18120 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562c6248d570 con 0x562c6247ca00 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f3452ffd640 1 -- 172.21.15.52:0/171304335 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3468010d40 con 0x562c6247ca00 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f3452ffd640 1 -- 172.21.15.52:0/171304335 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f346802f070 con 0x562c6247ca00 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 10 client.18120 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 10 client.18120 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 20 client.18120 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c624fd040 0x562c624ff460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 4 client.18120 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562c623ecc20 con 0x562c624fd040 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.414+0000 7f3473fff640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c624fd040 0x562c624ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:12.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.415+0000 7f3452ffd640 10 client.18120 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:12.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.415+0000 7f3473fff640 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c624fd040 0x562c624ff460 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebd442d6499297e5 server_cookie=3278631db1857618 in_seq=0 out_seq=0 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f3452ffd640 1 -- 172.21.15.52:0/171304335 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562c623ecc20 con 0x562c624fd040 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f3452ffd640 10 client.18120 handle_command_reply: tid=0 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 client.18120 shutdown 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 msgr2=0x562c6248ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c6248ac60 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f34680109b0 tx=0x7f346800fc10 comp rx=0 tx=0).stop 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 shutdown_connections 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562c624fd040 0x562c624ff460 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562c62481b70 0x562c6248bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f3452ffd640 0 client.18120 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562c6247ca00 0x562c6248ac60 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 --2- 172.21.15.52:0/171304335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562c6247c010 0x562c6248a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 >> 172.21.15.52:0/171304335 conn(0x562c62477680 msgr2=0x562c6247a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 shutdown_connections 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 1 -- 172.21.15.52:0/171304335 wait complete. 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.438+0000 7f347cc33740 20 client.18120 trim_cache size 0 max 16384 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:21:12.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:21:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f345c0a2cb0 msgr2=0x7f345c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f345c0a2cb0 0x7f345c0a5170 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f34640ccff0 tx=0x7f34640d2c30 comp rx=0 tx=0).stop 2024-06-02T23:21:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 msgr2=0x7f3474197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 0x7f3474197b80 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f346400e9c0 tx=0x7f346400ee80 comp rx=0 tx=0).stop 2024-06-02T23:21:12.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 shutdown_connections 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f345c0a2cb0 0x7f345c0a5170 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f34741024b0 0x7f347419f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3474101af0 0x7f34741980c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 --2- 172.21.15.52:0/321968087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3474100900 0x7f3474197b80 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 >> 172.21.15.52:0/321968087 conn(0x7f34740fbe70 msgr2=0x7f34740fd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 shutdown_connections 2024-06-02T23:21:12.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:12.439+0000 7f3479f22640 1 -- 172.21.15.52:0/321968087 wait complete. 2024-06-02T23:21:12.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 170897 1.74 MB/sec execute 381 sec latency 46.563 ms 2024-06-02T23:21:13.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 171367 1.74 MB/sec execute 382 sec latency 61.611 ms 2024-06-02T23:21:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:13 smithi174 ceph-mon[35994]: pgmap v711: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-02T23:21:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:13 smithi052 ceph-mon[32340]: pgmap v711: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-02T23:21:14.011 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:13 smithi123 ceph-mon[35977]: pgmap v711: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 257 KiB/s rd, 1.3 MiB/s wr, 152 op/s 2024-06-02T23:21:14.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 171849 1.74 MB/sec execute 383 sec latency 63.066 ms 2024-06-02T23:21:15.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 172473 1.74 MB/sec execute 384 sec latency 15.695 ms 2024-06-02T23:21:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:15 smithi052 ceph-mon[32340]: pgmap v712: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:21:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:15 smithi174 ceph-mon[35994]: pgmap v712: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:21:15.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.923+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/2389545002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d4101d40 msgr2=0x7fa8d410e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:15.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.923+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/2389545002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d4101d40 0x7fa8d410e200 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc00b0a0 tx=0x7fa8cc00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:21:15.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/2389545002 shutdown_connections 2024-06-02T23:21:15.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/2389545002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d4101d40 0x7fa8d410e200 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:15.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/2389545002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d41073c0 0x7fa8d41077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:15.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/2389545002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101380 0x7fa8d4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:15.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/2389545002 >> 172.21.15.52:0/2389545002 conn(0x7fa8d4078330 msgr2=0x7fa8d4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:15.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/2389545002 shutdown_connections 2024-06-02T23:21:15.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/2389545002 wait complete. 2024-06-02T23:21:15.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.924+0000 7fa8dc6f4700 1 Processor -- start 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 -- start start 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d4101380 0x7fa8d419ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 0x7fa8d419b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d41073c0 0x7fa8d419f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d4111550 con 0x7fa8d41073c0 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41113d0 con 0x7fa8d4101d40 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dc6f4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa8d41116d0 con 0x7fa8d4101380 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dac91700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d41073c0 0x7fa8d419f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8d9c8f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 0x7fa8d419b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8d9c8f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 0x7fa8d419b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49458/0 (socket says 172.21.15.52:49458) 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8d9c8f700 1 -- 172.21.15.52:0/1769704292 learned_addr learned my addr 172.21.15.52:0/1769704292 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:15.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8da490700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d4101380 0x7fa8d419ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dac91700 1 -- 172.21.15.52:0/1769704292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d4101380 msgr2=0x7fa8d419ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dac91700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d4101380 0x7fa8d419ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dac91700 1 -- 172.21.15.52:0/1769704292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 msgr2=0x7fa8d419b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dac91700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 0x7fa8d419b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.925+0000 7fa8dac91700 1 -- 172.21.15.52:0/1769704292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8d419fd20 con 0x7fa8d41073c0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8d9c8f700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 0x7fa8d419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8dac91700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d41073c0 0x7fa8d419f5e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0062a0 tx=0x7fa8cc004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc021070 con 0x7fa8d41073c0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8d419ffb0 con 0x7fa8d41073c0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa8cc004e30 con 0x7fa8d41073c0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8cc004620 con 0x7fa8d41073c0 2024-06-02T23:21:15.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.926+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8d41a7890 con 0x7fa8d41073c0 2024-06-02T23:21:15.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.927+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8d4102a70 con 0x7fa8d41073c0 2024-06-02T23:21:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.929+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa8cc007980 con 0x7fa8d41073c0 2024-06-02T23:21:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.930+0000 7fa8cb7fe700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8c00a2ca0 0x7fa8c00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:15.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.930+0000 7fa8da490700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8c00a2ca0 0x7fa8c00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.930+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa8cc0cfd80 con 0x7fa8d41073c0 2024-06-02T23:21:15.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.931+0000 7fa8da490700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8c00a2ca0 0x7fa8c00a5160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005950 tx=0x7fa8d000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:15.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:15.934+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa8cc09cda0 con 0x7fa8d41073c0 2024-06-02T23:21:16.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:15 smithi123 ceph-mon[35977]: pgmap v712: 97 pgs: 97 active+clean; 78 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 220 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:21:16.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.229+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa8d41a7ac0 con 0x7fa8d41073c0 2024-06-02T23:21:16.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.229+0000 7fa8cb7fe700 1 -- 172.21.15.52:0/1769704292 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fa8cc09c6a0 con 0x7fa8d41073c0 2024-06-02T23:21:16.229 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:21:16.229 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:21:16.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8c00a2ca0 msgr2=0x7fa8c00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:16.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8c00a2ca0 0x7fa8c00a5160 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0005950 tx=0x7fa8d000a300 comp rx=0 tx=0).stop 2024-06-02T23:21:16.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d41073c0 msgr2=0x7fa8d419f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d41073c0 0x7fa8d419f5e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fa8cc0062a0 tx=0x7fa8cc004c80 comp rx=0 tx=0).stop 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 shutdown_connections 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa8c00a2ca0 0x7fa8c00a5160 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa8d41073c0 0x7fa8d419f5e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa8d4101380 0x7fa8d419ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 --2- 172.21.15.52:0/1769704292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa8d4101d40 0x7fa8d419b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.232+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 >> 172.21.15.52:0/1769704292 conn(0x7fa8d4078330 msgr2=0x7fa8d406aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.233+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 shutdown_connections 2024-06-02T23:21:16.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:16.233+0000 7fa8dc6f4700 1 -- 172.21.15.52:0/1769704292 wait complete. 2024-06-02T23:21:16.233 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:21:16.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 173019 1.75 MB/sec execute 385 sec latency 63.886 ms 2024-06-02T23:21:16.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:16 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1769704292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:21:16.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:16 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1769704292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:21:17.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:16 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1769704292' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:21:17.076 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:21:17.351 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:17.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 173598 1.75 MB/sec execute 386 sec latency 13.087 ms 2024-06-02T23:21:17.798 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:17 smithi052 ceph-mon[32340]: pgmap v713: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:21:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:17 smithi174 ceph-mon[35994]: pgmap v713: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:21:18.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:17 smithi123 ceph-mon[35977]: pgmap v713: 97 pgs: 97 active+clean; 77 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 1.5 MiB/s wr, 172 op/s 2024-06-02T23:21:18.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 174054 1.76 MB/sec execute 387 sec latency 15.144 ms 2024-06-02T23:21:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 -- 172.21.15.52:0/602546447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101390 msgr2=0x7ffa60101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/602546447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101390 0x7ffa60101810 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c009990 tx=0x7ffa4c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 -- 172.21.15.52:0/602546447 shutdown_connections 2024-06-02T23:21:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/602546447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101d50 0x7ffa6010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/602546447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 0x7ffa601077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/602546447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101390 0x7ffa60101810 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 -- 172.21.15.52:0/602546447 >> 172.21.15.52:0/602546447 conn(0x7ffa60078340 msgr2=0x7ffa60078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:19.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 -- 172.21.15.52:0/602546447 shutdown_connections 2024-06-02T23:21:19.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.193+0000 7ffa66ba2700 1 -- 172.21.15.52:0/602546447 wait complete. 2024-06-02T23:21:19.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa66ba2700 1 Processor -- start 2024-06-02T23:21:19.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa66ba2700 1 -- start start 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa66ba2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101390 0x7ffa6019f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa66ba2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 0x7ffa6019f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa66ba2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 0x7ffa601a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa66ba2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa60111520 con 0x7ffa60101d50 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.194+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 0x7ffa6019f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 0x7ffa6019f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41920/0 (socket says 172.21.15.52:41920) 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4056226556 learned_addr learned my addr 172.21.15.52:0/4056226556 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601113a0 con 0x7ffa60101390 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101390 0x7ffa6019f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa6513f700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 0x7ffa601a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601116a0 con 0x7ffa60107400 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4056226556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 msgr2=0x7ffa601a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 0x7ffa601a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4056226556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101390 msgr2=0x7ffa6019f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101390 0x7ffa6019f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4056226556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601a4130 con 0x7ffa60101d50 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa6513f700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 0x7ffa601a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101390 0x7ffa6019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 0x7ffa6019f910 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0057d0 tx=0x7ffa4c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.195+0000 7ffa5dffb700 1 -- 172.21.15.52:0/4056226556 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c026070 con 0x7ffa60101d50 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.196+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa601a43c0 con 0x7ffa60101d50 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.196+0000 7ffa5dffb700 1 -- 172.21.15.52:0/4056226556 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c021410 con 0x7ffa60101d50 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.196+0000 7ffa5dffb700 1 -- 172.21.15.52:0/4056226556 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c017650 con 0x7ffa60101d50 2024-06-02T23:21:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.196+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601abca0 con 0x7ffa60101d50 2024-06-02T23:21:19.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.198+0000 7ffa5dffb700 1 -- 172.21.15.52:0/4056226556 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffa4c029070 con 0x7ffa60101d50 2024-06-02T23:21:19.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.199+0000 7ffa5dffb700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa500a2c20 0x7ffa500a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.199+0000 7ffa5dffb700 1 -- 172.21.15.52:0/4056226556 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c0cfc90 con 0x7ffa60101d50 2024-06-02T23:21:19.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.199+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa500a2c20 0x7ffa500a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.199+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa500a2c20 0x7ffa500a50e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009fd0 tx=0x7ffa54009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 Processor -- start 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 -- start start 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb96f60 0x5597fbb97340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597fba83f60 con 0x5597fbb917a0 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597fbae2850 con 0x5597fbb97880 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa71425b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597fba4a460 con 0x5597fbb96f60 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6493e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb96f60 0x5597fbb97340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41926/0 (socket says 172.21.15.52:41926) 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 -- 172.21.15.52:0/3863501013 learned_addr learned my addr 172.21.15.52:0/3863501013 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 -- 172.21.15.52:0/3863501013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb96f60 msgr2=0x5597fbb97340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb96f60 0x5597fbb97340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 -- 172.21.15.52:0/3863501013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 msgr2=0x5597fbb91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6513f700 1 -- 172.21.15.52:0/3863501013 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fba318d0 con 0x5597fbb917a0 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.200+0000 7ffa6493e700 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb96f60 0x5597fbb97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:19.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.201+0000 7ffa6513f700 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91bc0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009510 tx=0x7ffa58009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5580f117b2e1d5b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.201+0000 7ffa4affd700 1 -- 172.21.15.52:0/3863501013 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa5801b020 con 0x5597fbb917a0 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.201+0000 7ffa4affd700 1 -- 172.21.15.52:0/3863501013 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa5800eba0 con 0x5597fbb917a0 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.201+0000 7ffa4affd700 1 -- 172.21.15.52:0/3863501013 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa58017d00 con 0x5597fbb917a0 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 -- 172.21.15.52:0/3863501013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 msgr2=0x5597fbb91bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91bc0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009510 tx=0x7ffa58009920 comp rx=0 tx=0).stop 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 -- 172.21.15.52:0/3863501013 shutdown_connections 2024-06-02T23:21:19.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91bc0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb96f60 0x5597fbb97340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 --2- 172.21.15.52:0/3863501013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 -- 172.21.15.52:0/3863501013 >> 172.21.15.52:0/3863501013 conn(0x5597fbabe080 msgr2=0x5597fbb8c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 -- 172.21.15.52:0/3863501013 shutdown_connections 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 -- 172.21.15.52:0/3863501013 wait complete. 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 -- start start 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.202+0000 7ffa71425b80 1 Processor -- start 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbb9d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb9a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb97880 0x5597fbb9ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597fba82fc0 con 0x5597fbb97880 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597fbade810 con 0x5597fbb96f60 2024-06-02T23:21:19.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597fbb9e2b0 con 0x5597fbb917a0 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa71425b80 10 client.0 _ll_register_callbacks cb 0x5597fba785b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa6493e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbb9d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa6513f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb97880 0x5597fbb9ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb9a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa6493e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbb9d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54826/0 (socket says 172.21.15.52:54826) 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa6493e700 1 -- 172.21.15.52:0/847445215 learned_addr learned my addr 172.21.15.52:0/847445215 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 -- 172.21.15.52:0/847445215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 msgr2=0x5597fbb9d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbb9d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 -- 172.21.15.52:0/847445215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb97880 msgr2=0x5597fbb9ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb97880 0x5597fbb9ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 -- 172.21.15.52:0/847445215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fbaa8750 con 0x5597fbb96f60 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb9a7f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09c570 tx=0x7ffa4c009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92169843ca75310d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:19.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.203+0000 7ffa3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa3ffff700 1 -- 172.21.15.52:0/847445215 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c09c5e0 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa3ffff700 1 -- 172.21.15.52:0/847445215 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c09c840 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa5800eba0 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa3ffff700 1 -- 172.21.15.52:0/847445215 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c0a0990 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 10 client.27596 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5597fbba8970 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa3ffff700 1 -- 172.21.15.52:0/847445215 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa5800eba0 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa3ffff700 1 -- 172.21.15.52:0/847445215 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ffa4c085b70 con 0x5597fbb96f60 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 10 client.27596 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 10 client.27596 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 20 client.27596 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbbaaf80 0x5597fbbad3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 4 client.27596 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:19.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597fbaefd50 con 0x5597fbbaaf80 2024-06-02T23:21:19.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.204+0000 7ffa6493e700 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbbaaf80 0x5597fbbad3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.205+0000 7ffa6493e700 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbbaaf80 0x5597fbbad3a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=23ebfd26a4c288c6 server_cookie=72553ea92bb6555f in_seq=0 out_seq=0 2024-06-02T23:21:19.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.205+0000 7ffa3ffff700 10 client.27596 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.205+0000 7ffa3ffff700 1 -- 172.21.15.52:0/847445215 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597fbaefd50 con 0x5597fbbaaf80 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.205+0000 7ffa3ffff700 10 client.27596 handle_command_reply: tid=0 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 client.27596 shutdown 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 msgr2=0x5597fbb9a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb9a7f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c09c570 tx=0x7ffa4c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 shutdown_connections 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb97880 0x5597fbb9ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbb9d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbbaaf80 0x5597fbbad3a0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 --2- 172.21.15.52:0/847445215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb9a7f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa3ffff700 0 client.27596 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:21:19.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 >> 172.21.15.52:0/847445215 conn(0x5597fbabe080 msgr2=0x5597fbb8b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:19.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 shutdown_connections 2024-06-02T23:21:19.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 1 -- 172.21.15.52:0/847445215 wait complete. 2024-06-02T23:21:19.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.206+0000 7ffa71425b80 20 client.27596 trim_cache size 0 max 16384 2024-06-02T23:21:19.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.223+0000 7ffa71425b80 1 Processor -- start 2024-06-02T23:21:19.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.223+0000 7ffa71425b80 1 -- start start 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb99970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb97880 0x5597fbb9c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa71425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b2fa0 con 0x5597fbb917a0 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa71425b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b4ae0 con 0x5597fbb96f60 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa71425b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b3670 con 0x5597fbb97880 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa6493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb99970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.224 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa6493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41942/0 (socket says 172.21.15.52:41942) 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa6493e700 1 -- 172.21.15.52:0/1932402464 learned_addr learned my addr 172.21.15.52:0/1932402464 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 -- 172.21.15.52:0/1932402464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb97880 msgr2=0x5597fbb9c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa6513f700 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb97880 0x5597fbb9c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb97880 0x5597fbb9c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 -- 172.21.15.52:0/1932402464 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 msgr2=0x5597fbb91260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 -- 172.21.15.52:0/1932402464 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597fbb9b0b0 con 0x5597fbb96f60 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa6513f700 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb97880 0x5597fbb9c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.224+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb99970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c098a40 tx=0x7ffa4c086fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e071eedd2e421d8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa4a7fc700 1 -- 172.21.15.52:0/1932402464 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0a51c0 con 0x5597fbb96f60 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa4a7fc700 1 -- 172.21.15.52:0/1932402464 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c085b70 con 0x5597fbb96f60 2024-06-02T23:21:19.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa4a7fc700 1 -- 172.21.15.52:0/1932402464 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c09c840 con 0x5597fbb96f60 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 -- 172.21.15.52:0/1932402464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 msgr2=0x5597fbb99970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb99970 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c098a40 tx=0x7ffa4c086fe0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 -- 172.21.15.52:0/1932402464 shutdown_connections 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb97880 0x5597fbb9c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb917a0 0x5597fbb91260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 --2- 172.21.15.52:0/1932402464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb96f60 0x5597fbb99970 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 -- 172.21.15.52:0/1932402464 >> 172.21.15.52:0/1932402464 conn(0x5597fbabe080 msgr2=0x5597fbb9c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:19.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 -- 172.21.15.52:0/1932402464 shutdown_connections 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.225+0000 7ffa71425b80 1 -- 172.21.15.52:0/1932402464 wait complete. 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 -- start start 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 Processor -- start 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbba3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 0x5597fbb9edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb9f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b4410 con 0x5597fbb96f60 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b3d40 con 0x5597fbb97880 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa601b0f60 con 0x5597fbb917a0 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa71425b80 10 client.0 _ll_register_callbacks cb 0x5597fba785b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.226+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 0x5597fbb9edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa6493e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbba3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 0x5597fbb9edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41956/0 (socket says 172.21.15.52:41956) 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4053513177 learned_addr learned my addr 172.21.15.52:0/4053513177 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa6513f700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb9f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4053513177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 msgr2=0x5597fbba3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbba3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 msgr2=0x5597fbb9f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb9f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 -- 172.21.15.52:0/4053513177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597fbb9f870 con 0x5597fbb96f60 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbba3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa5ffff700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 0x5597fbb9edc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c098fb0 tx=0x7ffa4c0218a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ac2afd71c47fb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa3ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa3ffff700 1 -- 172.21.15.52:0/4053513177 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4c0a4f30 con 0x5597fbb96f60 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa4c085b70 con 0x5597fbb96f60 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa3ffff700 1 -- 172.21.15.52:0/4053513177 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffa4c00fb40 con 0x5597fbb96f60 2024-06-02T23:21:19.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.227+0000 7ffa3ffff700 1 -- 172.21.15.52:0/4053513177 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ffa4c09c5e0 con 0x5597fbb96f60 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 10 client.18144 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5597fbb9f870 con 0x5597fbb96f60 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa3ffff700 1 -- 172.21.15.52:0/4053513177 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffa4c09ca50 con 0x5597fbb96f60 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa3ffff700 1 -- 172.21.15.52:0/4053513177 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ffa4c0a1c40 con 0x5597fbb96f60 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 10 client.18144 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 10 client.18144 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 20 client.18144 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbc15b70 0x5597fbc17f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 4 client.18144 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ffa4c085b70 con 0x5597fbc15b70 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.228+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbc15b70 0x5597fbc17f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa6493e700 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbc15b70 0x5597fbc17f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d11971b8f68ff173 server_cookie=a359d89fb1d4f15a in_seq=0 out_seq=0 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa3ffff700 10 client.18144 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:21:19.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa3ffff700 1 -- 172.21.15.52:0/4053513177 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ffa4c085b70 con 0x5597fbc15b70 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa3ffff700 10 client.18144 handle_command_reply: tid=0 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 client.18144 shutdown 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 msgr2=0x5597fbb9edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 0x5597fbb9edc0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c098fb0 tx=0x7ffa4c0218a0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 shutdown_connections 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5597fbc15b70 0x5597fbc17f60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597fbb97880 0x5597fbb9f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa3ffff700 0 client.18144 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597fbb917a0 0x5597fbba3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 --2- 172.21.15.52:0/4053513177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597fbb96f60 0x5597fbb9edc0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 >> 172.21.15.52:0/4053513177 conn(0x5597fbabe080 msgr2=0x5597fbb95970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 shutdown_connections 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 1 -- 172.21.15.52:0/4053513177 wait complete. 2024-06-02T23:21:19.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.229+0000 7ffa71425b80 20 client.18144 trim_cache size 0 max 16384 2024-06-02T23:21:19.234 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:21:19.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa500a2c20 msgr2=0x7ffa500a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa500a2c20 0x7ffa500a50e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7ffa54009fd0 tx=0x7ffa54009260 comp rx=0 tx=0).stop 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 msgr2=0x7ffa6019f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 0x7ffa6019f910 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c0057d0 tx=0x7ffa4c005800 comp rx=0 tx=0).stop 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 shutdown_connections 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffa500a2c20 0x7ffa500a50e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa60107400 0x7ffa601a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa60101390 0x7ffa6019f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 --2- 172.21.15.52:0/4056226556 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffa60101d50 0x7ffa6019f910 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 >> 172.21.15.52:0/4056226556 conn(0x7ffa60078340 msgr2=0x7ffa601004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 shutdown_connections 2024-06-02T23:21:19.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:19.230+0000 7ffa66ba2700 1 -- 172.21.15.52:0/4056226556 wait complete. 2024-06-02T23:21:19.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 174273 1.76 MB/sec execute 388 sec latency 30.044 ms 2024-06-02T23:21:19.808 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:19 smithi123 ceph-mon[35977]: pgmap v714: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-06-02T23:21:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:19 smithi174 ceph-mon[35994]: pgmap v714: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-06-02T23:21:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:19 smithi052 ceph-mon[32340]: pgmap v714: 97 pgs: 97 active+clean; 77 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 1.1 MiB/s wr, 139 op/s 2024-06-02T23:21:19.929 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:21:20.201 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:20.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 174649 1.75 MB/sec execute 389 sec latency 14.808 ms 2024-06-02T23:21:21.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 175216 1.76 MB/sec execute 390 sec latency 12.682 ms 2024-06-02T23:21:21.739 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:21 smithi052 ceph-mon[32340]: pgmap v715: 97 pgs: 97 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-02T23:21:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:21 smithi174 ceph-mon[35994]: pgmap v715: 97 pgs: 97 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-02T23:21:22.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 -- 172.21.15.52:0/1538182382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 msgr2=0x7fe76c06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1538182382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 0x7fe76c06ae80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fe75c009990 tx=0x7fe75c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 -- 172.21.15.52:0/1538182382 shutdown_connections 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1538182382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe76c06b3c0 0x7fe76c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1538182382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c1084f0 0x7fe76c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1538182382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 0x7fe76c06ae80 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 -- 172.21.15.52:0/1538182382 >> 172.21.15.52:0/1538182382 conn(0x7fe76c078300 msgr2=0x7fe76c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 -- 172.21.15.52:0/1538182382 shutdown_connections 2024-06-02T23:21:22.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.005+0000 7fe7741ea700 1 -- 172.21.15.52:0/1538182382 wait complete. 2024-06-02T23:21:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:21 smithi123 ceph-mon[35977]: pgmap v715: 97 pgs: 97 active+clean; 79 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 285 KiB/s rd, 1.3 MiB/s wr, 167 op/s 2024-06-02T23:21:22.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 Processor -- start 2024-06-02T23:21:22.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 -- start start 2024-06-02T23:21:22.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 0x7fe76c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 0x7fe76c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe76c1084f0 0x7fe76c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c112600 con 0x7fe76c06aa00 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.006+0000 7fe7741ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c112480 con 0x7fe76c06b3c0 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe7741ea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c112780 con 0x7fe76c1084f0 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 0x7fe76c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 0x7fe76c19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49548/0 (socket says 172.21.15.52:49548) 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 -- 172.21.15.52:0/1219356396 learned_addr learned my addr 172.21.15.52:0/1219356396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771f86700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 0x7fe76c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe772787700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe76c1084f0 0x7fe76c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 -- 172.21.15.52:0/1219356396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe76c1084f0 msgr2=0x7fe76c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe76c1084f0 0x7fe76c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 -- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 msgr2=0x7fe76c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 0x7fe76c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 -- 172.21.15.52:0/1219356396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76c1990c0 con 0x7fe76c06b3c0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.007+0000 7fe771785700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 0x7fe76c19ece0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe75c000c00 tx=0x7fe75c004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.008+0000 7fe762ffd700 1 -- 172.21.15.52:0/1219356396 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c027070 con 0x7fe76c06b3c0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.008+0000 7fe762ffd700 1 -- 172.21.15.52:0/1219356396 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c022470 con 0x7fe76c06b3c0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.008+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76c199350 con 0x7fe76c06b3c0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.008+0000 7fe762ffd700 1 -- 172.21.15.52:0/1219356396 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00b610 con 0x7fe76c06b3c0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.008+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe76c101380 con 0x7fe76c06b3c0 2024-06-02T23:21:22.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.010+0000 7fe762ffd700 1 -- 172.21.15.52:0/1219356396 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe75c02a070 con 0x7fe76c06b3c0 2024-06-02T23:21:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.011+0000 7fe762ffd700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7580a2ca0 0x7fe7580a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.011+0000 7fe762ffd700 1 -- 172.21.15.52:0/1219356396 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe75c0cfc30 con 0x7fe76c06b3c0 2024-06-02T23:21:22.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.011+0000 7fe771f86700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7580a2ca0 0x7fe7580a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 Processor -- start 2024-06-02T23:21:22.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe771f86700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7580a2ca0 0x7fe7580a5160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe768009680 tx=0x7fe768009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:22.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 -- start start 2024-06-02T23:21:22.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e13640 0x55a3c9e13a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 0x55a3c9e0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a3c9d5f810 con 0x55a3c9e13f60 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3c9d07720 con 0x55a3c9e13640 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.012+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a3c9d5b7d0 con 0x55a3c9e0df20 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 0x55a3c9e0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 0x55a3c9e0d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41992/0 (socket says 172.21.15.52:41992) 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 -- 172.21.15.52:0/3034511996 learned_addr learned my addr 172.21.15.52:0/3034511996 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771f86700 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e13640 0x55a3c9e13a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe772787700 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 -- 172.21.15.52:0/3034511996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 msgr2=0x55a3c9e0e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 -- 172.21.15.52:0/3034511996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e13640 msgr2=0x55a3c9e13a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e13640 0x55a3c9e13a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 -- 172.21.15.52:0/3034511996 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c9cfdf50 con 0x55a3c9e13f60 2024-06-02T23:21:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe771785700 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 0x55a3c9e0d9e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0989b0 tx=0x7fe75c09bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3c7ff66af97380 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe757fff700 1 -- 172.21.15.52:0/3034511996 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0a0900 con 0x55a3c9e13f60 2024-06-02T23:21:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe757fff700 1 -- 172.21.15.52:0/3034511996 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c0a5130 con 0x55a3c9e13f60 2024-06-02T23:21:22.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.013+0000 7fe757fff700 1 -- 172.21.15.52:0/3034511996 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0a4ca0 con 0x55a3c9e13f60 2024-06-02T23:21:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 -- 172.21.15.52:0/3034511996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 msgr2=0x55a3c9e0d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 0x55a3c9e0d9e0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fe75c0989b0 tx=0x7fe75c09bf30 comp rx=0 tx=0).stop 2024-06-02T23:21:22.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 -- 172.21.15.52:0/3034511996 shutdown_connections 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e13640 0x55a3c9e13a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/3034511996 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13f60 0x55a3c9e0d9e0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 -- 172.21.15.52:0/3034511996 >> 172.21.15.52:0/3034511996 conn(0x55a3c9d38cc0 msgr2=0x55a3c9e08850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 -- 172.21.15.52:0/3034511996 shutdown_connections 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.014+0000 7fe77ea6db80 1 -- 172.21.15.52:0/3034511996 wait complete. 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.015+0000 7fe77ea6db80 1 -- start start 2024-06-02T23:21:22.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.015+0000 7fe77ea6db80 1 Processor -- start 2024-06-02T23:21:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e17230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e178d0 0x55a3c9e23bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a3c9e1a6f0 con 0x55a3c9e13640 2024-06-02T23:21:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a3c9cffbf0 con 0x55a3c9e0df20 2024-06-02T23:21:22.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a3c9e1a870 con 0x55a3c9e178d0 2024-06-02T23:21:22.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe77ea6db80 10 client.0 _ll_register_callbacks cb 0x55a3c9cf9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:22.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe772787700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e178d0 0x55a3c9e23bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe772787700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e178d0 0x55a3c9e23bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54872/0 (socket says 172.21.15.52:54872) 2024-06-02T23:21:22.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe772787700 1 -- 172.21.15.52:0/2379827526 learned_addr learned my addr 172.21.15.52:0/2379827526 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:22.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771785700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e17230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771f86700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771f86700 1 -- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e178d0 msgr2=0x55a3c9e23bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771f86700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e178d0 0x55a3c9e23bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771f86700 1 -- 172.21.15.52:0/2379827526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 msgr2=0x55a3c9e17230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771f86700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e17230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771f86700 1 -- 172.21.15.52:0/2379827526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c9cae8d0 con 0x55a3c9e0df20 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.016+0000 7fe771785700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:22.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe771f86700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e19f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe76c199aa0 tx=0x7fe7680047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4721205a8324c4d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe7557fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe7557fa700 1 -- 172.21.15.52:0/2379827526 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768004050 con 0x55a3c9e0df20 2024-06-02T23:21:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe7557fa700 1 -- 172.21.15.52:0/2379827526 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe7680041f0 con 0x55a3c9e0df20 2024-06-02T23:21:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c0a5130 con 0x55a3c9e0df20 2024-06-02T23:21:22.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe7557fa700 1 -- 172.21.15.52:0/2379827526 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe7680043f0 con 0x55a3c9e0df20 2024-06-02T23:21:22.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe77ea6db80 10 client.27614 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:22.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3c9e25670 con 0x55a3c9e0df20 2024-06-02T23:21:22.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.017+0000 7fe7557fa700 1 -- 172.21.15.52:0/2379827526 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c0a5130 con 0x55a3c9e0df20 2024-06-02T23:21:22.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe7557fa700 1 -- 172.21.15.52:0/2379827526 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe768018040 con 0x55a3c9e0df20 2024-06-02T23:21:22.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe77ea6db80 10 client.27614 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:22.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe77ea6db80 10 client.27614 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe77ea6db80 20 client.27614 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e2a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe77ea6db80 4 client.27614 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a3c9d6cd10 con 0x55a3c9e27cb0 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.018+0000 7fe771785700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e2a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.019+0000 7fe7557fa700 10 client.27614 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:21:22.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.019+0000 7fe771785700 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e2a0e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8251ccd707798c5e server_cookie=644a9bb02b3dbffe in_seq=0 out_seq=0 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe7557fa700 1 -- 172.21.15.52:0/2379827526 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a3c9d6cd10 con 0x55a3c9e27cb0 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe7557fa700 10 client.27614 handle_command_reply: tid=0 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 client.27614 shutdown 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 msgr2=0x55a3c9e19f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e19f30 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe76c199aa0 tx=0x7fe7680047c0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 shutdown_connections 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e2a0e0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e178d0 0x55a3c9e23bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe7557fa700 0 client.27614 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e19f30 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/2379827526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e17230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 >> 172.21.15.52:0/2379827526 conn(0x55a3c9d38cc0 msgr2=0x55a3c9e085f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 shutdown_connections 2024-06-02T23:21:22.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 1 -- 172.21.15.52:0/2379827526 wait complete. 2024-06-02T23:21:22.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.020+0000 7fe77ea6db80 20 client.27614 trim_cache size 0 max 16384 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 Processor -- start 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 -- start start 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e12c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e178d0 0x55a3c9e239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1aebf0 con 0x55a3c9e13640 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1b0730 con 0x55a3c9e178d0 2024-06-02T23:21:22.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1af2c0 con 0x55a3c9e0df20 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe771785700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e12c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe771785700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e12c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42012/0 (socket says 172.21.15.52:42012) 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe771785700 1 -- 172.21.15.52:0/563932568 learned_addr learned my addr 172.21.15.52:0/563932568 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe772787700 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e178d0 0x55a3c9e239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.037+0000 7fe771f86700 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771785700 1 -- 172.21.15.52:0/563932568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 msgr2=0x55a3c9e0d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771785700 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771785700 1 -- 172.21.15.52:0/563932568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e178d0 msgr2=0x55a3c9e239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771785700 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e178d0 0x55a3c9e239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771785700 1 -- 172.21.15.52:0/563932568 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3c9e08d70 con 0x55a3c9e13640 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771f86700 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:22.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe771785700 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e12c40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe75c022940 tx=0x7fe75c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c801fb12541026d9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe7577fe700 1 -- 172.21.15.52:0/563932568 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0b5260 con 0x55a3c9e13640 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe7577fe700 1 -- 172.21.15.52:0/563932568 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c0b5400 con 0x55a3c9e13640 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.038+0000 7fe7577fe700 1 -- 172.21.15.52:0/563932568 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0a4ca0 con 0x55a3c9e13640 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 -- 172.21.15.52:0/563932568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 msgr2=0x55a3c9e12c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e12c40 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fe75c022940 tx=0x7fe75c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 -- 172.21.15.52:0/563932568 shutdown_connections 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e178d0 0x55a3c9e239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e0df20 0x55a3c9e0d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/563932568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e13640 0x55a3c9e12c40 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 -- 172.21.15.52:0/563932568 >> 172.21.15.52:0/563932568 conn(0x55a3c9d38cc0 msgr2=0x55a3c9e11730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 -- 172.21.15.52:0/563932568 shutdown_connections 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.039+0000 7fe77ea6db80 1 -- 172.21.15.52:0/563932568 wait complete. 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 -- start start 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 Processor -- start 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e1fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e13640 0x55a3c9e20250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 0x55a3c9e2edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1b0060 con 0x55a3c9e178d0 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1af990 con 0x55a3c9e0df20 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe76c1acb90 con 0x55a3c9e13640 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe77ea6db80 10 client.0 _ll_register_callbacks cb 0x55a3c9cf9920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.040+0000 7fe772787700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 0x55a3c9e2edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 0x55a3c9e2edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42014/0 (socket says 172.21.15.52:42014) 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 -- 172.21.15.52:0/346685414 learned_addr learned my addr 172.21.15.52:0/346685414 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe771f86700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e1fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe771785700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e13640 0x55a3c9e20250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 -- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e13640 msgr2=0x55a3c9e20250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e13640 0x55a3c9e20250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 -- 172.21.15.52:0/346685414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 msgr2=0x55a3c9e1fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e1fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 -- 172.21.15.52:0/346685414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3c9e2f630 con 0x55a3c9e178d0 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe771f86700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e1fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:22.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe772787700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 0x55a3c9e2edc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe764004660 tx=0x7fe764004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd06669b6a7f60e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:22.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe7557fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:22.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe7557fa700 1 -- 172.21.15.52:0/346685414 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764007710 con 0x55a3c9e178d0 2024-06-02T23:21:22.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75c0b5400 con 0x55a3c9e178d0 2024-06-02T23:21:22.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.041+0000 7fe7557fa700 1 -- 172.21.15.52:0/346685414 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe7640078b0 con 0x55a3c9e178d0 2024-06-02T23:21:22.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe7557fa700 1 -- 172.21.15.52:0/346685414 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fe764028070 con 0x55a3c9e178d0 2024-06-02T23:21:22.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 10 client.18162 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:22.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a3c9e9ac50 con 0x55a3c9e178d0 2024-06-02T23:21:22.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe7557fa700 1 -- 172.21.15.52:0/346685414 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe75c0b5400 con 0x55a3c9e178d0 2024-06-02T23:21:22.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe7557fa700 1 -- 172.21.15.52:0/346685414 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fe764021b60 con 0x55a3c9e178d0 2024-06-02T23:21:22.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 10 client.18162 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:22.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 10 client.18162 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:21:22.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 20 client.18162 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e9d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 4 client.18162 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a3c9d67290 con 0x55a3c9e27cb0 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.042+0000 7fe771f86700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e9d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe771f86700 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e9d340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1232b07cb2f70dce server_cookie=ec3b47269e8f0914 in_seq=0 out_seq=0 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe7557fa700 10 client.18162 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe7557fa700 1 -- 172.21.15.52:0/346685414 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a3c9d67290 con 0x55a3c9e27cb0 2024-06-02T23:21:22.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe7557fa700 10 client.18162 handle_command_reply: tid=0 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 client.18162 shutdown 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 msgr2=0x55a3c9e2edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 0x55a3c9e2edc0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fe764004660 tx=0x7fe764004690 comp rx=0 tx=0).stop 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 shutdown_connections 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a3c9e178d0 0x55a3c9e2edc0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3c9e0df20 0x55a3c9e1fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55a3c9e27cb0 0x55a3c9e9d340 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 --2- 172.21.15.52:0/346685414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a3c9e13640 0x55a3c9e20250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe7557fa700 0 client.18162 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:21:22.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 >> 172.21.15.52:0/346685414 conn(0x55a3c9d38cc0 msgr2=0x55a3c9e15b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:22.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 shutdown_connections 2024-06-02T23:21:22.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 1 -- 172.21.15.52:0/346685414 wait complete. 2024-06-02T23:21:22.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.043+0000 7fe77ea6db80 20 client.18162 trim_cache size 0 max 16384 2024-06-02T23:21:22.051 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:21:22.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7580a2ca0 msgr2=0x7fe7580a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7580a2ca0 0x7fe7580a5160 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fe768009680 tx=0x7fe768009260 comp rx=0 tx=0).stop 2024-06-02T23:21:22.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 msgr2=0x7fe76c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 0x7fe76c19ece0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe75c000c00 tx=0x7fe75c004790 comp rx=0 tx=0).stop 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 shutdown_connections 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe7580a2ca0 0x7fe7580a5160 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe76c1084f0 0x7fe76c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe76c06aa00 0x7fe76c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 --2- 172.21.15.52:0/1219356396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe76c06b3c0 0x7fe76c19ece0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:22.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 >> 172.21.15.52:0/1219356396 conn(0x7fe76c078300 msgr2=0x7fe76c1000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:22.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 shutdown_connections 2024-06-02T23:21:22.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:22.044+0000 7fe7741ea700 1 -- 172.21.15.52:0/1219356396 wait complete. 2024-06-02T23:21:22.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 175803 1.76 MB/sec execute 391 sec latency 12.900 ms 2024-06-02T23:21:22.810 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:21:23.090 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:23.640 INFO:tasks.workunit.client.0.smithi052.stdout: 1 176313 1.76 MB/sec execute 392 sec latency 16.234 ms 2024-06-02T23:21:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:23 smithi174 ceph-mon[35994]: pgmap v716: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-02T23:21:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:23 smithi052 ceph-mon[32340]: pgmap v716: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-02T23:21:24.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:23 smithi123 ceph-mon[35977]: pgmap v716: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 131 op/s 2024-06-02T23:21:24.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 177089 1.77 MB/sec execute 393 sec latency 15.087 ms 2024-06-02T23:21:24.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 -- 172.21.15.52:0/1948350857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 msgr2=0x7f54741077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 --2- 172.21.15.52:0/1948350857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f54741077b0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f5470009990 tx=0x7f5470009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 -- 172.21.15.52:0/1948350857 shutdown_connections 2024-06-02T23:21:24.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 --2- 172.21.15.52:0/1948350857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5474101d30 0x7f547410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 --2- 172.21.15.52:0/1948350857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f54741077b0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 --2- 172.21.15.52:0/1948350857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f54741017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.925+0000 7f547b423700 1 -- 172.21.15.52:0/1948350857 >> 172.21.15.52:0/1948350857 conn(0x7f54740782c0 msgr2=0x7f54740786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:24.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 -- 172.21.15.52:0/1948350857 shutdown_connections 2024-06-02T23:21:24.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 -- 172.21.15.52:0/1948350857 wait complete. 2024-06-02T23:21:24.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 Processor -- start 2024-06-02T23:21:24.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 -- start start 2024-06-02T23:21:24.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f5474103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5474101d30 0x7f5474103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f5474103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54741114c0 con 0x7f54741073d0 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5474111340 con 0x7f5474101390 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.926+0000 7f547b423700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5474111640 con 0x7f5474101d30 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f5474103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54791bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f5474103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54791bf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f5474103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35246/0 (socket says 172.21.15.52:35246) 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54791bf700 1 -- 172.21.15.52:0/3721766872 learned_addr learned my addr 172.21.15.52:0/3721766872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f5474103ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58952/0 (socket says 172.21.15.52:58952) 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54789be700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5474101d30 0x7f5474103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 -- 172.21.15.52:0/3721766872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5474101d30 msgr2=0x7f5474103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5474101d30 0x7f5474103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 -- 172.21.15.52:0/3721766872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 msgr2=0x7f5474103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f5474103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 -- 172.21.15.52:0/3721766872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54741a7050 con 0x7f54741073d0 2024-06-02T23:21:24.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54791bf700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f5474103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:24.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f54799c0700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f5474103ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f546c00cc50 tx=0x7f546c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:24.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f546a7fc700 1 -- 172.21.15.52:0/3721766872 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c016020 con 0x7f54741073d0 2024-06-02T23:21:24.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.927+0000 7f546a7fc700 1 -- 172.21.15.52:0/3721766872 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546c012420 con 0x7f54741073d0 2024-06-02T23:21:24.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.928+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54741a7340 con 0x7f54741073d0 2024-06-02T23:21:24.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.928+0000 7f546a7fc700 1 -- 172.21.15.52:0/3721766872 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0115a0 con 0x7f54741073d0 2024-06-02T23:21:24.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.928+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54741a7880 con 0x7f54741073d0 2024-06-02T23:21:24.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.930+0000 7f546a7fc700 1 -- 172.21.15.52:0/3721766872 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f546c019020 con 0x7f54741073d0 2024-06-02T23:21:24.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.931+0000 7f546a7fc700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54600a2c20 0x7f54600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.931+0000 7f54791bf700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54600a2c20 0x7f54600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.931+0000 7f546a7fc700 1 -- 172.21.15.52:0/3721766872 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f546c0ced90 con 0x7f54741073d0 2024-06-02T23:21:24.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.931+0000 7f54791bf700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54600a2c20 0x7f54600a50e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5470006010 tx=0x7f5470005bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:24.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.932+0000 7f5485ca6b80 1 Processor -- start 2024-06-02T23:21:24.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.932+0000 7f5485ca6b80 1 -- start start 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7640 0x55d08e0a7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7f60 0x55d08e0a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0a2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d08dff3810 con 0x55d08e0a1f20 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d08df9b720 con 0x55d08e0a7f60 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d08dfef7d0 con 0x55d08e0a7640 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0a2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0a2340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58962/0 (socket says 172.21.15.52:58962) 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54789be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7f60 0x55d08e0a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 -- 172.21.15.52:0/2388408762 learned_addr learned my addr 172.21.15.52:0/2388408762 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54791bf700 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7640 0x55d08e0a7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 -- 172.21.15.52:0/2388408762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7640 msgr2=0x55d08e0a7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7640 0x55d08e0a7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 -- 172.21.15.52:0/2388408762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7f60 msgr2=0x55d08e0a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7f60 0x55d08e0a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.933+0000 7f54799c0700 1 -- 172.21.15.52:0/2388408762 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d08df91f50 con 0x55d08e0a1f20 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.934+0000 7f54799c0700 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0a2340 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f546c09b320 tx=0x7f546c007a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962ded30e9a908c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.934+0000 7f545f7fe700 1 -- 172.21.15.52:0/2388408762 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0b4070 con 0x55d08e0a1f20 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.934+0000 7f545f7fe700 1 -- 172.21.15.52:0/2388408762 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546c09b620 con 0x55d08e0a1f20 2024-06-02T23:21:24.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.934+0000 7f545f7fe700 1 -- 172.21.15.52:0/2388408762 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c09b880 con 0x55d08e0a1f20 2024-06-02T23:21:24.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2388408762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 msgr2=0x55d08e0a2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0a2340 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f546c09b320 tx=0x7f546c007a60 comp rx=0 tx=0).stop 2024-06-02T23:21:24.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2388408762 shutdown_connections 2024-06-02T23:21:24.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0a2340 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7640 0x55d08e0a7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2388408762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7f60 0x55d08e0a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2388408762 >> 172.21.15.52:0/2388408762 conn(0x55d08dfcccc0 msgr2=0x55d08e09c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2388408762 shutdown_connections 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2388408762 wait complete. 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 -- start start 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.935+0000 7f5485ca6b80 1 Processor -- start 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a1f20 0x55d08e0adc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 0x55d08e0aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d08e0ae9d0 con 0x55d08e0a7640 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d08df93bf0 con 0x55d08e0a1f20 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d08e0aeb50 con 0x55d08e0a7f60 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f5485ca6b80 10 client.0 _ll_register_callbacks cb 0x55d08df8d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:24.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f54799c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f54789be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 0x55d08e0aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f54789be700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 0x55d08e0aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58964/0 (socket says 172.21.15.52:58964) 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f54799c0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33964/0 (socket says 172.21.15.52:33964) 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f54789be700 1 -- 172.21.15.52:0/1263573390 learned_addr learned my addr 172.21.15.52:0/1263573390 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.936+0000 7f54791bf700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a1f20 0x55d08e0adc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54799c0700 1 -- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a1f20 msgr2=0x55d08e0adc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54799c0700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a1f20 0x55d08e0adc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54799c0700 1 -- 172.21.15.52:0/1263573390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 msgr2=0x55d08e0aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54799c0700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 0x55d08e0aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54799c0700 1 -- 172.21.15.52:0/1263573390 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d08df428d0 con 0x55d08e0a7f60 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54789be700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 0x55d08e0aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f54799c0700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0ab4a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f546c00cc20 tx=0x7f546c012c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dfd1cbeb0e06ca0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f545cff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:21:24.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f545cff9700 1 -- 172.21.15.52:0/1263573390 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546c0b8070 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f545cff9700 1 -- 172.21.15.52:0/1263573390 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f546c09b880 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f546c09b620 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f545cff9700 1 -- 172.21.15.52:0/1263573390 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f546c0b4050 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f5485ca6b80 10 client.27403 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.937+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d08e0ae9d0 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f545cff9700 1 -- 172.21.15.52:0/1263573390 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f546c09b620 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f545cff9700 1 -- 172.21.15.52:0/1263573390 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f546c084bb0 con 0x55d08e0a7f60 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f5485ca6b80 10 client.27403 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f5485ca6b80 10 client.27403 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f5485ca6b80 20 client.27403 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e0bb5f0 0x55d08e0bd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f5485ca6b80 4 client.27403 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:24.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d08e000d10 con 0x55d08e0bb5f0 2024-06-02T23:21:24.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f54791bf700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e0bb5f0 0x55d08e0bd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f545cff9700 10 client.27403 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.938+0000 7f54791bf700 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e0bb5f0 0x55d08e0bd9e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b37a9fb82130187a server_cookie=404ac026d5f0872c in_seq=0 out_seq=0 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.939+0000 7f545cff9700 1 -- 172.21.15.52:0/1263573390 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d08e000d10 con 0x55d08e0bb5f0 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.939+0000 7f545cff9700 10 client.27403 handle_command_reply: tid=0 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.939+0000 7f5485ca6b80 1 client.27403 shutdown 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 msgr2=0x55d08e0ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0ab4a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f546c00cc20 tx=0x7f546c012c80 comp rx=0 tx=0).stop 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 shutdown_connections 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e0bb5f0 0x55d08e0bd9e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0ab4a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a1f20 0x55d08e0adc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/1263573390 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7640 0x55d08e0aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f545cff9700 0 client.27403 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:21:24.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 >> 172.21.15.52:0/1263573390 conn(0x55d08dfcccc0 msgr2=0x55d08e0a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:24.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 shutdown_connections 2024-06-02T23:21:24.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 1 -- 172.21.15.52:0/1263573390 wait complete. 2024-06-02T23:21:24.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.940+0000 7f5485ca6b80 20 client.27403 trim_cache size 0 max 16384 2024-06-02T23:21:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.955+0000 7f5485ca6b80 1 Processor -- start 2024-06-02T23:21:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 -- start start 2024-06-02T23:21:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a1f20 0x55d08e0a19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e096690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54741aec80 con 0x55d08e0a7f60 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54741b07c0 con 0x55d08e0a7640 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54741af350 con 0x55d08e0a1f20 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e096690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54799c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e096690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35270/0 (socket says 172.21.15.52:35270) 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54799c0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58968/0 (socket says 172.21.15.52:58968) 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 -- 172.21.15.52:0/2054733263 learned_addr learned my addr 172.21.15.52:0/2054733263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:24.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54791bf700 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a1f20 0x55d08e0a19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 -- 172.21.15.52:0/2054733263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a1f20 msgr2=0x55d08e0a19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a1f20 0x55d08e0a19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 -- 172.21.15.52:0/2054733263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7f60 msgr2=0x55d08e0c3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.956+0000 7f54789be700 1 -- 172.21.15.52:0/2054733263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d08e0ae460 con 0x55d08e0a7640 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.957+0000 7f54789be700 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e096690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5464004580 tx=0x7f546400bba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb98fc2603442932 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.957+0000 7f545effd700 1 -- 172.21.15.52:0/2054733263 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464003940 con 0x55d08e0a7640 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.957+0000 7f545effd700 1 -- 172.21.15.52:0/2054733263 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5464020070 con 0x55d08e0a7640 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.957+0000 7f545effd700 1 -- 172.21.15.52:0/2054733263 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464011e30 con 0x55d08e0a7640 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2054733263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 msgr2=0x55d08e096690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e096690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5464004580 tx=0x7f546400bba0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2054733263 shutdown_connections 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a1f20 0x55d08e0a19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/2054733263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e096690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2054733263 >> 172.21.15.52:0/2054733263 conn(0x55d08dfcccc0 msgr2=0x55d08e0a89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2054733263 shutdown_connections 2024-06-02T23:21:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 -- 172.21.15.52:0/2054733263 wait complete. 2024-06-02T23:21:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 -- start start 2024-06-02T23:21:24.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.958+0000 7f5485ca6b80 1 Processor -- start 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f54741b00f0 con 0x55d08e0a1f20 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54741afa20 con 0x55d08e0a7640 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f54741acc20 con 0x55d08e0a7f60 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f5485ca6b80 10 client.0 _ll_register_callbacks cb 0x55d08df8d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0b4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58972/0 (socket says 172.21.15.52:58972) 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54789be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 -- 172.21.15.52:0/4085176186 learned_addr learned my addr 172.21.15.52:0/4085176186 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54799c0700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 -- 172.21.15.52:0/4085176186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 msgr2=0x55d08e0c3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 -- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 msgr2=0x55d08e0c2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e0c2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54791bf700 1 -- 172.21.15.52:0/4085176186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d08e125040 con 0x55d08e0a1f20 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54789be700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e0c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.959+0000 7f54799c0700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f54791bf700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0b4690 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x55d08e0be010 tx=0x7f5470009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d987524f203fe91a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:24.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f545cff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f545cff9700 1 -- 172.21.15.52:0/4085176186 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5470024e50 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f545cff9700 1 -- 172.21.15.52:0/4085176186 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f547000f460 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5464020070 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f545cff9700 1 -- 172.21.15.52:0/4085176186 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f547000f660 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f5485ca6b80 10 client.18180 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d08e126160 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.960+0000 7f545cff9700 1 -- 172.21.15.52:0/4085176186 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5464020070 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f545cff9700 1 -- 172.21.15.52:0/4085176186 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f5470047810 con 0x55d08e0a1f20 2024-06-02T23:21:24.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 10 client.18180 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 10 client.18180 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 20 client.18180 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e1268f0 0x55d08e128d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 4 client.18180 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d08dffb290 con 0x55d08e1268f0 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f54789be700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e1268f0 0x55d08e128d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f54789be700 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e1268f0 0x55d08e128d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d5e3769b399af9b server_cookie=f5ffea1c10c9ce90 in_seq=0 out_seq=0 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f545cff9700 10 client.18180 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f545cff9700 1 -- 172.21.15.52:0/4085176186 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d08dffb290 con 0x55d08e1268f0 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f545cff9700 10 client.18180 handle_command_reply: tid=0 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.961+0000 7f5485ca6b80 1 client.18180 shutdown 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 msgr2=0x55d08e0b4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0b4690 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x55d08e0be010 tx=0x7f5470009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 shutdown_connections 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55d08e1268f0 0x55d08e128d10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d08e0a7f60 0x55d08e0c3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d08e0a1f20 0x55d08e0b4690 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 --2- 172.21.15.52:0/4085176186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d08e0a7640 0x55d08e0c2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 >> 172.21.15.52:0/4085176186 conn(0x55d08dfcccc0 msgr2=0x55d08e0a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f545cff9700 0 client.18180 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 shutdown_connections 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 1 -- 172.21.15.52:0/4085176186 wait complete. 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.962+0000 7f5485ca6b80 20 client.18180 trim_cache size 0 max 16384 2024-06-02T23:21:24.968 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:21:24.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54600a2c20 msgr2=0x7f54600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54600a2c20 0x7f54600a50e0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5470006010 tx=0x7f5470005bc0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 msgr2=0x7f5474103ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:24.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f5474103ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f546c00cc50 tx=0x7f546c00cf60 comp rx=0 tx=0).stop 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 shutdown_connections 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f54600a2c20 0x7f54600a50e0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f54741073d0 0x7f5474103ac0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5474101390 0x7f5474103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 --2- 172.21.15.52:0/3721766872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5474101d30 0x7f5474103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 >> 172.21.15.52:0/3721766872 conn(0x7f54740782c0 msgr2=0x7f5474100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 shutdown_connections 2024-06-02T23:21:24.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:24.963+0000 7f547b423700 1 -- 172.21.15.52:0/3721766872 wait complete. 2024-06-02T23:21:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 -- 172.21.15.52:0/658356253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61941024b0 msgr2=0x7f6194108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 --2- 172.21.15.52:0/658356253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61941024b0 0x7f6194108d40 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f618c009ca0 tx=0x7f618c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 -- 172.21.15.52:0/658356253 shutdown_connections 2024-06-02T23:21:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 --2- 172.21.15.52:0/658356253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f61941024b0 0x7f6194108d40 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 --2- 172.21.15.52:0/658356253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6194101af0 0x7f6194101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 --2- 172.21.15.52:0/658356253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 0x7f6194100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 -- 172.21.15.52:0/658356253 >> 172.21.15.52:0/658356253 conn(0x7f61940fbe70 msgr2=0x7f61940fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 -- 172.21.15.52:0/658356253 shutdown_connections 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.541+0000 7f619abd5640 1 -- 172.21.15.52:0/658356253 wait complete. 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 Processor -- start 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 -- start start 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 0x7f6194199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 0x7f619419a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941024b0 0x7f61941a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f619410d510 con 0x7f6194101af0 2024-06-02T23:21:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f619410d390 con 0x7f61941024b0 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f619abd5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f619410d690 con 0x7f6194100900 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 0x7f619419a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 0x7f619419a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58992/0 (socket says 172.21.15.52:58992) 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 -- 172.21.15.52:0/190493183 learned_addr learned my addr 172.21.15.52:0/190493183 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6199172640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941024b0 0x7f61941a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6198971640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 0x7f6194199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 -- 172.21.15.52:0/190493183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 msgr2=0x7f6194199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 0x7f6194199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 -- 172.21.15.52:0/190493183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941024b0 msgr2=0x7f61941a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941024b0 0x7f61941a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6193fff640 1 -- 172.21.15.52:0/190493183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61941a1970 con 0x7f6194101af0 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.542+0000 7f6198971640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 0x7f6194199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.543+0000 7f6193fff640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 0x7f619419a290 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f618000e9f0 tx=0x7f618000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.543+0000 7f6191ffb640 1 -- 172.21.15.52:0/190493183 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618000cd80 con 0x7f6194101af0 2024-06-02T23:21:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.543+0000 7f6191ffb640 1 -- 172.21.15.52:0/190493183 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61800108e0 con 0x7f6194101af0 2024-06-02T23:21:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.543+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61941a1c60 con 0x7f6194101af0 2024-06-02T23:21:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.543+0000 7f6191ffb640 1 -- 172.21.15.52:0/190493183 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180018920 con 0x7f6194101af0 2024-06-02T23:21:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.543+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61941a21b0 con 0x7f6194101af0 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.546+0000 7f6191ffb640 1 -- 172.21.15.52:0/190493183 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6180010a90 con 0x7f6194101af0 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.546+0000 7f619d9c1740 1 Processor -- start 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.546+0000 7f619d9c1740 1 -- start start 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.546+0000 7f6191ffb640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61840a2c30 0x7f61840a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52010 0x56096df52430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df57b70 0x56096df5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6199172640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61840a2c30 0x7f61840a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6191ffb640 1 -- 172.21.15.52:0/190493183 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f61800cf390 con 0x7f6194101af0 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f619d9c1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56096deb54f0 con 0x56096df52a00 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f619d9c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56096de53870 con 0x56096df57b70 2024-06-02T23:21:25.549 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f619d9c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56096deb14b0 con 0x56096df52010 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df57500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59006/0 (socket says 172.21.15.52:59006) 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 -- 172.21.15.52:0/179458882 learned_addr learned my addr 172.21.15.52:0/179458882 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6199172640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df57b70 0x56096df5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6198971640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52010 0x56096df52430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 -- 172.21.15.52:0/179458882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52010 msgr2=0x56096df52430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52010 0x56096df52430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 -- 172.21.15.52:0/179458882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df57b70 msgr2=0x56096df5a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df57b70 0x56096df5a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 -- 172.21.15.52:0/179458882 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096de6f9b0 con 0x56096df52a00 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6198971640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52010 0x56096df52430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6199172640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df57b70 0x56096df5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6199172640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61840a2c30 0x7f61840a50f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f618c005e30 tx=0x7f618c031530 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f6193fff640 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df57500 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f618009bbb0 tx=0x7f618004a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ac297bfd0128f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.547+0000 7f617effd640 1 -- 172.21.15.52:0/179458882 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618009c320 con 0x56096df52a00 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f617effd640 1 -- 172.21.15.52:0/179458882 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f618009bc20 con 0x56096df52a00 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f617effd640 1 -- 172.21.15.52:0/179458882 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6180098420 con 0x56096df52a00 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 -- 172.21.15.52:0/179458882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 msgr2=0x56096df57500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df57500 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f618009bbb0 tx=0x7f618004a7f0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 -- 172.21.15.52:0/179458882 shutdown_connections 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df57b70 0x56096df5a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df57500 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 --2- 172.21.15.52:0/179458882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52010 0x56096df52430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 -- 172.21.15.52:0/179458882 >> 172.21.15.52:0/179458882 conn(0x56096df4d680 msgr2=0x56096df4da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 -- 172.21.15.52:0/179458882 shutdown_connections 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.548+0000 7f619d9c1740 1 -- 172.21.15.52:0/179458882 wait complete. 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 -- start start 2024-06-02T23:21:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 Processor -- start 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df5c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df5c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df63710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56096df5aac0 con 0x56096df57b70 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56096de3eae0 con 0x56096df52010 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56096df5ac40 con 0x56096df52a00 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f619d9c1740 10 client.0 _ll_register_callbacks cb 0x56096de779b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f6199172640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df63710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f6198971640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df5c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f6199172640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df63710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59020/0 (socket says 172.21.15.52:59020) 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f6198971640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df5c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35316/0 (socket says 172.21.15.52:35316) 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f6199172640 1 -- 172.21.15.52:0/3385172628 learned_addr learned my addr 172.21.15.52:0/3385172628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.549+0000 7f6193fff640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df5c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6199172640 1 -- 172.21.15.52:0/3385172628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 msgr2=0x56096df5c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6199172640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df5c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6199172640 1 -- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 msgr2=0x56096df5c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6199172640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df5c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6199172640 1 -- 172.21.15.52:0/3385172628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096dbc9210 con 0x56096df57b70 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6193fff640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df5c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6199172640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df63710 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f618c009ca0 tx=0x7f618c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ab9acaa3d0785da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6173fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6173fff640 1 -- 172.21.15.52:0/3385172628 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c051020 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6173fff640 1 -- 172.21.15.52:0/3385172628 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f618c004780 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618009bc20 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f6173fff640 1 -- 172.21.15.52:0/3385172628 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f618c0305c0 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f619d9c1740 10 client.18192 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.550+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56096dc95620 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f6173fff640 1 -- 172.21.15.52:0/3385172628 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f618009bc20 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f6173fff640 1 -- 172.21.15.52:0/3385172628 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f618c04d6f0 con 0x56096df57b70 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f619d9c1740 10 client.18192 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f619d9c1740 10 client.18192 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f619d9c1740 20 client.18192 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096df68040 0x56096df6a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f619d9c1740 4 client.18192 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56096dec8f10 con 0x56096df68040 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.551+0000 7f6198971640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096df68040 0x56096df6a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.552+0000 7f6173fff640 10 client.18192 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.552+0000 7f6198971640 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096df68040 0x56096df6a450 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1809470c320a496c server_cookie=5b94b97393e55c12 in_seq=0 out_seq=0 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f6173fff640 1 -- 172.21.15.52:0/3385172628 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56096dec8f10 con 0x56096df68040 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f6173fff640 10 client.18192 handle_command_reply: tid=0 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 client.18192 shutdown 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 msgr2=0x56096df63710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df63710 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f618c009ca0 tx=0x7f618c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 shutdown_connections 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096df68040 0x56096df6a450 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df5c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f6173fff640 0 client.18192 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df63710 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3385172628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df5c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 >> 172.21.15.52:0/3385172628 conn(0x56096df4d680 msgr2=0x56096df4c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 shutdown_connections 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 1 -- 172.21.15.52:0/3385172628 wait complete. 2024-06-02T23:21:25.559 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.553+0000 7f619d9c1740 20 client.18192 trim_cache size 0 max 16384 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 Processor -- start 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 -- start start 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df6fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df636d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df702a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61941aaf70 con 0x56096df57b70 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61941acab0 con 0x56096df52010 2024-06-02T23:21:25.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f619d9c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61941ab640 con 0x56096df52a00 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f6199172640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df702a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f6198971640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df6fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f6193fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df636d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f6199172640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df702a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59028/0 (socket says 172.21.15.52:59028) 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.571+0000 7f6199172640 1 -- 172.21.15.52:0/3623390863 learned_addr learned my addr 172.21.15.52:0/3623390863 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6193fff640 1 -- 172.21.15.52:0/3623390863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 msgr2=0x56096df6fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6193fff640 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df6fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6193fff640 1 -- 172.21.15.52:0/3623390863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 msgr2=0x56096df702a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6193fff640 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df702a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6193fff640 1 -- 172.21.15.52:0/3623390863 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56096df6aad0 con 0x56096df52a00 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6198971640 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df6fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6199172640 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f6193fff640 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df636d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f618009b5f0 tx=0x7f618009b580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be3ecbce20746587 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f617e7fc640 1 -- 172.21.15.52:0/3623390863 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618009bc20 con 0x56096df52a00 2024-06-02T23:21:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f617e7fc640 1 -- 172.21.15.52:0/3623390863 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f618009ffd0 con 0x56096df52a00 2024-06-02T23:21:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.572+0000 7f617e7fc640 1 -- 172.21.15.52:0/3623390863 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618009b2d0 con 0x56096df52a00 2024-06-02T23:21:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 -- 172.21.15.52:0/3623390863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 msgr2=0x56096df636d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df636d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f618009b5f0 tx=0x7f618009b580 comp rx=0 tx=0).stop 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 -- 172.21.15.52:0/3623390863 shutdown_connections 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df52a00 0x56096df636d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df57b70 0x56096df702a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 --2- 172.21.15.52:0/3623390863 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df6fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 -- 172.21.15.52:0/3623390863 >> 172.21.15.52:0/3623390863 conn(0x56096df4d680 msgr2=0x56096df53b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 -- 172.21.15.52:0/3623390863 shutdown_connections 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 -- 172.21.15.52:0/3623390863 wait complete. 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 -- start start 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.573+0000 7f619d9c1740 1 Processor -- start 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df60bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df57b70 0x56096dfd0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61941ac3e0 con 0x56096df52a00 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61941abd10 con 0x56096df52010 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61941a8f50 con 0x56096df57b70 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 10 client.0 _ll_register_callbacks cb 0x7f618c031f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df60bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df60bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59044/0 (socket says 172.21.15.52:59044) 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6198971640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 -- 172.21.15.52:0/304029294 learned_addr learned my addr 172.21.15.52:0/304029294 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6199172640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df57b70 0x56096dfd0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 -- 172.21.15.52:0/304029294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df57b70 msgr2=0x56096dfd0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df57b70 0x56096dfd0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 -- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 msgr2=0x56096df606b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df606b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 -- 172.21.15.52:0/304029294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56096df5ad80 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6198971640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6193fff640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df60bf0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f61800985f0 tx=0x7f618004a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ebbd7d7a1d4a05 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6173fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6173fff640 1 -- 172.21.15.52:0/304029294 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61800b9060 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f6173fff640 1 -- 172.21.15.52:0/304029294 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6180010430 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.574+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f618009ffd0 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f6173fff640 1 -- 172.21.15.52:0/304029294 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f618009bc20 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 10 client.18198 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56096df4cf30 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f6173fff640 1 -- 172.21.15.52:0/304029294 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f618009ffd0 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f6173fff640 1 -- 172.21.15.52:0/304029294 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6180018bc0 con 0x56096df52a00 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 10 client.18198 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 10 client.18198 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 20 client.18198 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096dfd24f0 0x56096dfd48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 4 client.18198 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:21:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56096dec2c20 con 0x56096dfd24f0 2024-06-02T23:21:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.575+0000 7f6198971640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096dfd24f0 0x56096dfd48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.576+0000 7f6173fff640 10 client.18198 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.576+0000 7f6198971640 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096dfd24f0 0x56096dfd48e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de23e19992d478de server_cookie=a6397265f227fb2e in_seq=0 out_seq=0 2024-06-02T23:21:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.602+0000 7f6173fff640 1 -- 172.21.15.52:0/304029294 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56096dec2c20 con 0x56096dfd24f0 2024-06-02T23:21:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.602+0000 7f6173fff640 10 client.18198 handle_command_reply: tid=0 2024-06-02T23:21:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.602+0000 7f619d9c1740 1 client.18198 shutdown 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 msgr2=0x56096df60bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df60bf0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f61800985f0 tx=0x7f618004a920 comp rx=0 tx=0).stop 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 shutdown_connections 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56096dfd24f0 0x56096dfd48e0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f6173fff640 0 client.18198 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56096df57b70 0x56096dfd0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56096df52a00 0x56096df60bf0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 --2- 172.21.15.52:0/304029294 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56096df52010 0x56096df606b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 >> 172.21.15.52:0/304029294 conn(0x56096df4d680 msgr2=0x56096df53b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 shutdown_connections 2024-06-02T23:21:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 1 -- 172.21.15.52:0/304029294 wait complete. 2024-06-02T23:21:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.603+0000 7f619d9c1740 20 client.18198 trim_cache size 0 max 16384 2024-06-02T23:21:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:21:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:21:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:21:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.604+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61840a2c30 msgr2=0x7f61840a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.604+0000 7f619abd5640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61840a2c30 0x7f61840a50f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f618c005e30 tx=0x7f618c031530 comp rx=0 tx=0).stop 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.604+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 msgr2=0x7f619419a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.604+0000 7f619abd5640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 0x7f619419a290 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f618000e9f0 tx=0x7f618000eeb0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.604+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 shutdown_connections 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61840a2c30 0x7f61840a50f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941024b0 0x7f61941a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6194101af0 0x7f619419a290 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 --2- 172.21.15.52:0/190493183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6194100900 0x7f6194199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 >> 172.21.15.52:0/190493183 conn(0x7f61940fbe70 msgr2=0x7f6194109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 shutdown_connections 2024-06-02T23:21:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:25.605+0000 7f619abd5640 1 -- 172.21.15.52:0/190493183 wait complete. 2024-06-02T23:21:25.621 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:21:25.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 177460 1.77 MB/sec execute 394 sec latency 16.696 ms 2024-06-02T23:21:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:25 smithi174 ceph-mon[35994]: pgmap v717: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-06-02T23:21:25.899 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:25.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:25 smithi052 ceph-mon[32340]: pgmap v717: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-06-02T23:21:26.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:25 smithi123 ceph-mon[35977]: pgmap v717: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 224 KiB/s rd, 1.0 MiB/s wr, 128 op/s 2024-06-02T23:21:26.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 177757 1.77 MB/sec execute 395 sec latency 14.345 ms 2024-06-02T23:21:27.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 -- 172.21.15.52:0/1423376980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406a170 msgr2=0x7f615406a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 --2- 172.21.15.52:0/1423376980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406a170 0x7f615406a5d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f61440099c0 tx=0x7f6144009cd0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 -- 172.21.15.52:0/1423376980 shutdown_connections 2024-06-02T23:21:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 --2- 172.21.15.52:0/1423376980 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f615406ab10 0x7f615410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 --2- 172.21.15.52:0/1423376980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61541084e0 0x7f61541088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 --2- 172.21.15.52:0/1423376980 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406a170 0x7f615406a5d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 -- 172.21.15.52:0/1423376980 >> 172.21.15.52:0/1423376980 conn(0x7f61540782c0 msgr2=0x7f61540786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:27.634 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:27 smithi052 ceph-mon[32340]: pgmap v718: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-06-02T23:21:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 -- 172.21.15.52:0/1423376980 shutdown_connections 2024-06-02T23:21:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.633+0000 7f615c121700 1 -- 172.21.15.52:0/1423376980 wait complete. 2024-06-02T23:21:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 Processor -- start 2024-06-02T23:21:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 -- start start 2024-06-02T23:21:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 0x7f615419e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406ab10 0x7f615419ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61541084e0 0x7f6154198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6154112640 con 0x7f615406ab10 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61541124c0 con 0x7f615406a170 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f61596bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406ab10 0x7f615419ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f6159ebd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 0x7f615419e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615c121700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61541127c0 con 0x7f61541084e0 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f6159ebd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 0x7f615419e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35352/0 (socket says 172.21.15.52:35352) 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f6159ebd700 1 -- 172.21.15.52:0/201884228 learned_addr learned my addr 172.21.15.52:0/201884228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.634+0000 7f615a6be700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61541084e0 0x7f6154198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f6159ebd700 1 -- 172.21.15.52:0/201884228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61541084e0 msgr2=0x7f6154198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f6159ebd700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61541084e0 0x7f6154198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f6159ebd700 1 -- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406ab10 msgr2=0x7f615419ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f6159ebd700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406ab10 0x7f615419ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f6159ebd700 1 -- 172.21.15.52:0/201884228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61541990a0 con 0x7f615406a170 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f61596bc700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406ab10 0x7f615419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f6159ebd700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 0x7f615419e780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f615000e9a0 tx=0x7f615000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f614affd700 1 -- 172.21.15.52:0/201884228 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615000ca10 con 0x7f615406a170 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6154199390 con 0x7f615406a170 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f614affd700 1 -- 172.21.15.52:0/201884228 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f615000cbb0 con 0x7f615406a170 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f614affd700 1 -- 172.21.15.52:0/201884228 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6150018680 con 0x7f615406a170 2024-06-02T23:21:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.635+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61541a5650 con 0x7f615406a170 2024-06-02T23:21:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.638+0000 7f614affd700 1 -- 172.21.15.52:0/201884228 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6150010430 con 0x7f615406a170 2024-06-02T23:21:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f614affd700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61400a2c20 0x7f61400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f61596bc700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61400a2c20 0x7f61400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f614affd700 1 -- 172.21.15.52:0/201884228 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f61500cf110 con 0x7f615406a170 2024-06-02T23:21:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f61669a4b80 1 Processor -- start 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f61669a4b80 1 -- start start 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f61596bc700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61400a2c20 0x7f61400a50e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6154078930 tx=0x7f6144005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.639+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebedaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebfa210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61669a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e92eadbf60 con 0x55e92ebed670 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61669a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e92eb3a850 con 0x55e92ebee030 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61669a4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e92eaa2460 con 0x55e92ebf3470 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61596bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebedaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebfa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61596bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebedaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59064/0 (socket says 172.21.15.52:59064) 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f61596bc700 1 -- 172.21.15.52:0/18826976 learned_addr learned my addr 172.21.15.52:0/18826976 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f6159ebd700 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 -- 172.21.15.52:0/18826976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 msgr2=0x55e92ebf3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 -- 172.21.15.52:0/18826976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 msgr2=0x55e92ebedaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebedaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 -- 172.21.15.52:0/18826976 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92ea898d0 con 0x55e92ebee030 2024-06-02T23:21:27.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f615a6be700 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebfa210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f614c00f410 tx=0x7f614c00f720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cce41fa1f640a2d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:27.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 178032 1.77 MB/sec execute 396 sec latency 14.326 ms 2024-06-02T23:21:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.640+0000 7f613ffff700 1 -- 172.21.15.52:0/18826976 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c018030 con 0x55e92ebee030 2024-06-02T23:21:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f613ffff700 1 -- 172.21.15.52:0/18826976 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f614c004bb0 con 0x55e92ebee030 2024-06-02T23:21:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f613ffff700 1 -- 172.21.15.52:0/18826976 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c005020 con 0x55e92ebee030 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 -- 172.21.15.52:0/18826976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 msgr2=0x55e92ebfa210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebfa210 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f614c00f410 tx=0x7f614c00f720 comp rx=0 tx=0).stop 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 -- 172.21.15.52:0/18826976 shutdown_connections 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebfa210 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 --2- 172.21.15.52:0/18826976 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebedaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 -- 172.21.15.52:0/18826976 >> 172.21.15.52:0/18826976 conn(0x55e92eb16080 msgr2=0x55e92ebe4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 -- 172.21.15.52:0/18826976 shutdown_connections 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.641+0000 7f61669a4b80 1 -- 172.21.15.52:0/18826976 wait complete. 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 -- start start 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 Processor -- start 2024-06-02T23:21:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebed670 0x55e92ebf9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 0x55e92ebf7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e92eadafc0 con 0x55e92ebee030 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e92eb36810 con 0x55e92ebed670 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.642+0000 7f61669a4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e92ebfa750 con 0x55e92ebf3470 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61669a4b80 10 client.0 _ll_register_callbacks cb 0x55e92ead05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 0x55e92ebf7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 0x55e92ebf7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59070/0 (socket says 172.21.15.52:59070) 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 -- 172.21.15.52:0/1097005164 learned_addr learned my addr 172.21.15.52:0/1097005164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f615a6be700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f6159ebd700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebed670 0x55e92ebf9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 -- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 msgr2=0x55e92ebf76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 -- 172.21.15.52:0/1097005164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebed670 msgr2=0x55e92ebf9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebed670 0x55e92ebf9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 -- 172.21.15.52:0/1097005164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92eb00750 con 0x55e92ebee030 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f615a6be700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f6159ebd700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebed670 0x55e92ebf9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f61596bc700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 0x55e92ebf7180 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6144000c00 tx=0x7f6144009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d42c20efed0b3c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.643+0000 7f613d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f613d7fa700 1 -- 172.21.15.52:0/1097005164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144023460 con 0x55e92ebee030 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f613d7fa700 1 -- 172.21.15.52:0/1097005164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6144023600 con 0x55e92ebee030 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614c004bb0 con 0x55e92ebee030 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f613d7fa700 1 -- 172.21.15.52:0/1097005164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6144023800 con 0x55e92ebee030 2024-06-02T23:21:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 10 client.18210 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e92ec04db0 con 0x55e92ebee030 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f613d7fa700 1 -- 172.21.15.52:0/1097005164 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f614c004bb0 con 0x55e92ebee030 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f613d7fa700 1 -- 172.21.15.52:0/1097005164 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f614403b950 con 0x55e92ebee030 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 10 client.18210 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 10 client.18210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 20 client.18210 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec07440 0x55e92ec09860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 4 client.18210 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.644+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e92eb47d50 con 0x55e92ec07440 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.645+0000 7f6159ebd700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec07440 0x55e92ec09860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.645+0000 7f6159ebd700 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec07440 0x55e92ec09860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=2f38e846454feb9e server_cookie=c19def46c91b94f8 in_seq=0 out_seq=0 2024-06-02T23:21:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.645+0000 7f613d7fa700 10 client.18210 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:21:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f613d7fa700 1 -- 172.21.15.52:0/1097005164 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e92eb47d50 con 0x55e92ec07440 2024-06-02T23:21:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f613d7fa700 10 client.18210 handle_command_reply: tid=0 2024-06-02T23:21:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 client.18210 shutdown 2024-06-02T23:21:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 msgr2=0x55e92ebf7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 0x55e92ebf7180 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6144000c00 tx=0x7f6144009cd0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 shutdown_connections 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec07440 0x55e92ec09860 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebf76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebed670 0x55e92ebf9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f613d7fa700 0 client.18210 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 --2- 172.21.15.52:0/1097005164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebee030 0x55e92ebf7180 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 >> 172.21.15.52:0/1097005164 conn(0x55e92eb16080 msgr2=0x55e92ebe3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 shutdown_connections 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 1 -- 172.21.15.52:0/1097005164 wait complete. 2024-06-02T23:21:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.646+0000 7f61669a4b80 20 client.18210 trim_cache size 0 max 16384 2024-06-02T23:21:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.667+0000 7f61669a4b80 1 Processor -- start 2024-06-02T23:21:27.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.667+0000 7f61669a4b80 1 -- start start 2024-06-02T23:21:27.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebf60e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebf16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebe8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61669a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61541aa7c0 con 0x55e92ebed670 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61669a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61541ac300 con 0x55e92ebee030 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61669a4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61541aae90 con 0x55e92ebf3470 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61596bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebf16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61596bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebf16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35376/0 (socket says 172.21.15.52:35376) 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f61596bc700 1 -- 172.21.15.52:0/113520120 learned_addr learned my addr 172.21.15.52:0/113520120 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f6159ebd700 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebf60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.668+0000 7f615a6be700 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebe8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f6159ebd700 1 -- 172.21.15.52:0/113520120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 msgr2=0x55e92ebe8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f6159ebd700 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebe8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f6159ebd700 1 -- 172.21.15.52:0/113520120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 msgr2=0x55e92ebf16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f6159ebd700 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebf16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f6159ebd700 1 -- 172.21.15.52:0/113520120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e92ec09ee0 con 0x55e92ebed670 2024-06-02T23:21:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f61596bc700 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebf16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f6159ebd700 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebf60e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f615000cfd0 tx=0x7f615000c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a535fc4063e586 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f613f7fe700 1 -- 172.21.15.52:0/113520120 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f615009fd50 con 0x55e92ebed670 2024-06-02T23:21:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f613f7fe700 1 -- 172.21.15.52:0/113520120 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61500a4610 con 0x55e92ebed670 2024-06-02T23:21:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.669+0000 7f613f7fe700 1 -- 172.21.15.52:0/113520120 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61500a43e0 con 0x55e92ebed670 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 -- 172.21.15.52:0/113520120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 msgr2=0x55e92ebf60e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebf60e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f615000cfd0 tx=0x7f615000c480 comp rx=0 tx=0).stop 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 -- 172.21.15.52:0/113520120 shutdown_connections 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebf3470 0x55e92ebe8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebf60e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 --2- 172.21.15.52:0/113520120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebee030 0x55e92ebf16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.670+0000 7f61669a4b80 1 -- 172.21.15.52:0/113520120 >> 172.21.15.52:0/113520120 conn(0x55e92eb16080 msgr2=0x55e92ebe8250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.671+0000 7f61669a4b80 1 -- 172.21.15.52:0/113520120 shutdown_connections 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.671+0000 7f61669a4b80 1 -- 172.21.15.52:0/113520120 wait complete. 2024-06-02T23:21:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.671+0000 7f61669a4b80 1 -- start start 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.671+0000 7f61669a4b80 1 Processor -- start 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebecc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebee030 0x55e92ebff9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebf3470 0x55e92ebffee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f61541abc30 con 0x55e92ebed670 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f61541ab560 con 0x55e92ebf3470 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61541a8760 con 0x55e92ebee030 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61669a4b80 10 client.0 _ll_register_callbacks cb 0x55e92ead05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f6159ebd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebecc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f615a6be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebf3470 0x55e92ebffee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f61596bc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebee030 0x55e92ebff9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f6159ebd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebecc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59088/0 (socket says 172.21.15.52:59088) 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f6159ebd700 1 -- 172.21.15.52:0/54437957 learned_addr learned my addr 172.21.15.52:0/54437957 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f615a6be700 1 -- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebee030 msgr2=0x55e92ebff9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f615a6be700 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebee030 0x55e92ebff9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f615a6be700 1 -- 172.21.15.52:0/54437957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 msgr2=0x55e92ebecc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f615a6be700 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebecc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.672+0000 7f615a6be700 1 -- 172.21.15.52:0/54437957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e92ec00420 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.673+0000 7f615a6be700 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebf3470 0x55e92ebffee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f614c00a040 tx=0x7f614c003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe076a2c397d4d12 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.673+0000 7f613d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.673+0000 7f613d7fa700 1 -- 172.21.15.52:0/54437957 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614c0054e0 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.673+0000 7f613d7fa700 1 -- 172.21.15.52:0/54437957 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f614c005680 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.673+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f61500a4610 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f613d7fa700 1 -- 172.21.15.52:0/54437957 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f614c0058a0 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 10 client.27674 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e92ec6e0e0 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f613d7fa700 1 -- 172.21.15.52:0/54437957 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f61500a4610 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f613d7fa700 1 -- 172.21.15.52:0/54437957 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f614c024020 con 0x55e92ebf3470 2024-06-02T23:21:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 10 client.27674 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 10 client.27674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 20 client.27674 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec6e3f0 0x55e92ec707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 4 client.27674 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.674+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e92eb422d0 con 0x55e92ec6e3f0 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.675+0000 7f6159ebd700 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec6e3f0 0x55e92ec707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.675+0000 7f613d7fa700 10 client.27674 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.675+0000 7f6159ebd700 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec6e3f0 0x55e92ec707e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e374c76b81d16306 server_cookie=eceb59b26af20a41 in_seq=0 out_seq=0 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.675+0000 7f613d7fa700 1 -- 172.21.15.52:0/54437957 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e92eb422d0 con 0x55e92ec6e3f0 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.675+0000 7f613d7fa700 10 client.27674 handle_command_reply: tid=0 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.675+0000 7f61669a4b80 1 client.27674 shutdown 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebf3470 msgr2=0x55e92ebffee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebf3470 0x55e92ebffee0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f614c00a040 tx=0x7f614c003c30 comp rx=0 tx=0).stop 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 shutdown_connections 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e92ec6e3f0 0x55e92ec707e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e92ebf3470 0x55e92ebffee0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e92ebed670 0x55e92ebecc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f613d7fa700 0 client.27674 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 --2- 172.21.15.52:0/54437957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e92ebee030 0x55e92ebff9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 >> 172.21.15.52:0/54437957 conn(0x55e92eb16080 msgr2=0x55e92ebf9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 shutdown_connections 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 1 -- 172.21.15.52:0/54437957 wait complete. 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.676+0000 7f61669a4b80 20 client.27674 trim_cache size 0 max 16384 2024-06-02T23:21:27.679 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61400a2c20 msgr2=0x7f61400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61400a2c20 0x7f61400a50e0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6154078930 tx=0x7f6144005fb0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 msgr2=0x7f615419e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 0x7f615419e780 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f615000e9a0 tx=0x7f615000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 shutdown_connections 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f61400a2c20 0x7f61400a50e0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61541084e0 0x7f6154198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f615406a170 0x7f615419e780 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 --2- 172.21.15.52:0/201884228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f615406ab10 0x7f615419ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:27.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 >> 172.21.15.52:0/201884228 conn(0x7f61540782c0 msgr2=0x7f61540ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:27.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 shutdown_connections 2024-06-02T23:21:27.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:27.677+0000 7f615c121700 1 -- 172.21.15.52:0/201884228 wait complete. 2024-06-02T23:21:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:27 smithi174 ceph-mon[35994]: pgmap v718: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-06-02T23:21:28.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:27 smithi123 ceph-mon[35977]: pgmap v718: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 292 KiB/s rd, 1.4 MiB/s wr, 171 op/s 2024-06-02T23:21:28.481 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:21:28.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 178545 1.77 MB/sec execute 397 sec latency 13.328 ms 2024-06-02T23:21:28.811 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:29.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 179142 1.77 MB/sec execute 398 sec latency 15.016 ms 2024-06-02T23:21:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:29 smithi174 ceph-mon[35994]: pgmap v719: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-02T23:21:29.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:29 smithi052 ceph-mon[32340]: pgmap v719: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-02T23:21:30.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:29 smithi123 ceph-mon[35977]: pgmap v719: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 1.2 MiB/s wr, 130 op/s 2024-06-02T23:21:30.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 -- 172.21.15.52:0/1563635313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 msgr2=0x7fac5c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 --2- 172.21.15.52:0/1563635313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c1012b0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fac500099c0 tx=0x7fac50009cd0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 -- 172.21.15.52:0/1563635313 shutdown_connections 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 --2- 172.21.15.52:0/1563635313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac5c1017f0 0x7fac5c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 --2- 172.21.15.52:0/1563635313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac5c107500 0x7fac5c1078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 --2- 172.21.15.52:0/1563635313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c1012b0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 -- 172.21.15.52:0/1563635313 >> 172.21.15.52:0/1563635313 conn(0x7fac5c076730 msgr2=0x7fac5c076b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 -- 172.21.15.52:0/1563635313 shutdown_connections 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.630+0000 7fac619a9700 1 -- 172.21.15.52:0/1563635313 wait complete. 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 Processor -- start 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 -- start start 2024-06-02T23:21:30.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac5c1017f0 0x7fac5c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac5c107500 0x7fac5c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c112650 con 0x7fac5c100e30 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1124d0 con 0x7fac5c107500 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac619a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1127d0 con 0x7fac5c1017f0 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac5affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac5affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c19e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59096/0 (socket says 172.21.15.52:59096) 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac5affd700 1 -- 172.21.15.52:0/1496516970 learned_addr learned my addr 172.21.15.52:0/1496516970 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.631+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac5c107500 0x7fac5c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac5c1017f0 0x7fac5c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5affd700 1 -- 172.21.15.52:0/1496516970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac5c1017f0 msgr2=0x7fac5c19ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5affd700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac5c1017f0 0x7fac5c19ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5affd700 1 -- 172.21.15.52:0/1496516970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac5c107500 msgr2=0x7fac5c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5affd700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac5c107500 0x7fac5c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5affd700 1 -- 172.21.15.52:0/1496516970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac5c199050 con 0x7fac5c100e30 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac5affd700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c19e730 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00e9a0 tx=0x7fac4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac609a7700 1 -- 172.21.15.52:0/1496516970 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c00ca10 con 0x7fac5c100e30 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac609a7700 1 -- 172.21.15.52:0/1496516970 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac4c00cbb0 con 0x7fac5c100e30 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac609a7700 1 -- 172.21.15.52:0/1496516970 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4c018680 con 0x7fac5c100e30 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac5c199340 con 0x7fac5c100e30 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.632+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac5c1a5600 con 0x7fac5c100e30 2024-06-02T23:21:30.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.635+0000 7fac609a7700 1 -- 172.21.15.52:0/1496516970 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac4c010430 con 0x7fac5c100e30 2024-06-02T23:21:30.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.635+0000 7fac609a7700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac440a2c20 0x7fac440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.636+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac440a2c20 0x7fac440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.636+0000 7fac609a7700 1 -- 172.21.15.52:0/1496516970 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fac4c0cf190 con 0x7fac5c100e30 2024-06-02T23:21:30.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.636+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac440a2c20 0x7fac440a50e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac5c076d90 tx=0x7fac50005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:30.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.636+0000 7fac6c22cb80 1 Processor -- start 2024-06-02T23:21:30.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.636+0000 7fac6c22cb80 1 -- start start 2024-06-02T23:21:30.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe16a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe11340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5568efd62810 con 0x5568efe16f60 2024-06-02T23:21:30.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568efd0a720 con 0x5568efe16640 2024-06-02T23:21:30.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5568efd5e7d0 con 0x5568efe10f20 2024-06-02T23:21:30.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe109e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59108/0 (socket says 172.21.15.52:59108) 2024-06-02T23:21:30.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 -- 172.21.15.52:0/855347314 learned_addr learned my addr 172.21.15.52:0/855347314 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:30.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5affd700 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe16a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe11340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 -- 172.21.15.52:0/855347314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 msgr2=0x5568efe11340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe11340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 -- 172.21.15.52:0/855347314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 msgr2=0x5568efe16a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe16a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 -- 172.21.15.52:0/855347314 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568efd00f50 con 0x5568efe16f60 2024-06-02T23:21:30.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe11340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:21:30.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.637+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe109e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fac50015000 tx=0x7fac5001a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681cb444c88d5fb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:30.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.638+0000 7fac41ffb700 1 -- 172.21.15.52:0/855347314 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500056e0 con 0x5568efe16f60 2024-06-02T23:21:30.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.638+0000 7fac41ffb700 1 -- 172.21.15.52:0/855347314 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac50003e50 con 0x5568efe16f60 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.638+0000 7fac41ffb700 1 -- 172.21.15.52:0/855347314 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac50023b00 con 0x5568efe16f60 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.638+0000 7fac6c22cb80 1 -- 172.21.15.52:0/855347314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 msgr2=0x5568efe109e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.638+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe109e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fac50015000 tx=0x7fac5001a070 comp rx=0 tx=0).stop 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 -- 172.21.15.52:0/855347314 shutdown_connections 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe11340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe16a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/855347314 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe109e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 -- 172.21.15.52:0/855347314 >> 172.21.15.52:0/855347314 conn(0x5568efd3bcc0 msgr2=0x5568efe0b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 -- 172.21.15.52:0/855347314 shutdown_connections 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 -- 172.21.15.52:0/855347314 wait complete. 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 -- start start 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.639+0000 7fac6c22cb80 1 Processor -- start 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe1cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe19fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe1a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5568efe1d9b0 con 0x5568efe16f60 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5568efd02bf0 con 0x5568efe16640 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5568efe1db30 con 0x5568efe10f20 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac6c22cb80 10 client.0 _ll_register_callbacks cb 0x5568efcfc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe1a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe19fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe1a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59120/0 (socket says 172.21.15.52:59120) 2024-06-02T23:21:30.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5b7fe700 1 -- 172.21.15.52:0/2732491719 learned_addr learned my addr 172.21.15.52:0/2732491719 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5affd700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe1cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5a7fc700 1 -- 172.21.15.52:0/2732491719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 msgr2=0x5568efe1cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe1cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5a7fc700 1 -- 172.21.15.52:0/2732491719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 msgr2=0x5568efe1a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe1a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5a7fc700 1 -- 172.21.15.52:0/2732491719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568efcb18d0 con 0x5568efe16640 2024-06-02T23:21:30.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.640+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:30.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 179725 1.77 MB/sec execute 399 sec latency 48.070 ms 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe19fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fac50030040 tx=0x7fac5000b9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=609e6fbd80fbf0a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac3affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac3affd700 1 -- 172.21.15.52:0/2732491719 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac500056e0 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac3affd700 1 -- 172.21.15.52:0/2732491719 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fac50003c50 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac50003e50 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac3affd700 1 -- 172.21.15.52:0/2732491719 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac5000bdc0 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac6c22cb80 10 client.27686 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.641+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5568efe1d9b0 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac3affd700 1 -- 172.21.15.52:0/2732491719 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac50003e50 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac3affd700 1 -- 172.21.15.52:0/2732491719 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fac50010470 con 0x5568efe16640 2024-06-02T23:21:30.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac6c22cb80 10 client.27686 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac6c22cb80 10 client.27686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac6c22cb80 20 client.27686 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe2a2e0 0x5568efe2c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac6c22cb80 4 client.27686 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568efd6fd10 con 0x5568efe2a2e0 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac5affd700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe2a2e0 0x5568efe2c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac3affd700 10 client.27686 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.642+0000 7fac5affd700 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe2a2e0 0x5568efe2c6d0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b50e8653c4f29731 server_cookie=86f0d03ebc2e8386 in_seq=0 out_seq=0 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.643+0000 7fac3affd700 1 -- 172.21.15.52:0/2732491719 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5568efd6fd10 con 0x5568efe2a2e0 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.643+0000 7fac3affd700 10 client.27686 handle_command_reply: tid=0 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.643+0000 7fac6c22cb80 1 client.27686 shutdown 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.643+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 msgr2=0x5568efe19fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.643+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe19fc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fac50030040 tx=0x7fac5000b9c0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 shutdown_connections 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe2a2e0 0x5568efe2c6d0 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe1a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe1cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2732491719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe19fc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 >> 172.21.15.52:0/2732491719 conn(0x5568efd3bcc0 msgr2=0x5568efe0f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac3affd700 0 client.27686 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 shutdown_connections 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2732491719 wait complete. 2024-06-02T23:21:30.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.644+0000 7fac6c22cb80 20 client.27686 trim_cache size 0 max 16384 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 Processor -- start 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 -- start start 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe109e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe173a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1acab0 con 0x5568efe16f60 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1ae5f0 con 0x5568efe16640 2024-06-02T23:21:30.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1ad180 con 0x5568efe10f20 2024-06-02T23:21:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe178e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59126/0 (socket says 172.21.15.52:59126) 2024-06-02T23:21:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 -- 172.21.15.52:0/3833167415 learned_addr learned my addr 172.21.15.52:0/3833167415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe173a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5affd700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe109e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 -- 172.21.15.52:0/3833167415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 msgr2=0x5568efe109e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe109e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 -- 172.21.15.52:0/3833167415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 msgr2=0x5568efe173a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe173a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5b7fe700 1 -- 172.21.15.52:0/3833167415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568efe160e0 con 0x5568efe16f60 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.661+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe173a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.662+0000 7fac5affd700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.662+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe178e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fac54004a30 tx=0x7fac54009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d9d88ab6c80f32 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.662+0000 7fac417fa700 1 -- 172.21.15.52:0/3833167415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54004db0 con 0x5568efe16f60 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.662+0000 7fac417fa700 1 -- 172.21.15.52:0/3833167415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac54017c50 con 0x5568efe16f60 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.662+0000 7fac417fa700 1 -- 172.21.15.52:0/3833167415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac540058e0 con 0x5568efe16f60 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 -- 172.21.15.52:0/3833167415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 msgr2=0x5568efe178e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe178e0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fac54004a30 tx=0x7fac54009920 comp rx=0 tx=0).stop 2024-06-02T23:21:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 -- 172.21.15.52:0/3833167415 shutdown_connections 2024-06-02T23:21:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe178e0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe109e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/3833167415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe173a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 -- 172.21.15.52:0/3833167415 >> 172.21.15.52:0/3833167415 conn(0x5568efd3bcc0 msgr2=0x5568efe0b380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:30.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 -- 172.21.15.52:0/3833167415 shutdown_connections 2024-06-02T23:21:30.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 -- 172.21.15.52:0/3833167415 wait complete. 2024-06-02T23:21:30.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 -- start start 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.663+0000 7fac6c22cb80 1 Processor -- start 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe22e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe23360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe30f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1adf20 con 0x5568efe16f60 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1ad850 con 0x5568efe16640 2024-06-02T23:21:30.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fac5c1aaa50 con 0x5568efe10f20 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac6c22cb80 10 client.0 _ll_register_callbacks cb 0x5568efcfc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe30f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe30f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59130/0 (socket says 172.21.15.52:59130) 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 -- 172.21.15.52:0/2421002205 learned_addr learned my addr 172.21.15.52:0/2421002205 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe23360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5affd700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe22e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 -- 172.21.15.52:0/2421002205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 msgr2=0x5568efe22e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe22e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 -- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 msgr2=0x5568efe23360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe23360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 -- 172.21.15.52:0/2421002205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568efe31510 con 0x5568efe16f60 2024-06-02T23:21:30.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5a7fc700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac5b7fe700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe30f20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fac5400b010 tx=0x7fac54004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e35af79b4529518 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.664+0000 7fac3affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac3affd700 1 -- 172.21.15.52:0/2421002205 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac54005020 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac3affd700 1 -- 172.21.15.52:0/2421002205 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fac54017e00 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac54017c50 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac3affd700 1 -- 172.21.15.52:0/2421002205 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fac54005770 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 10 client.18234 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5568efe945c0 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac3affd700 1 -- 172.21.15.52:0/2421002205 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac54017c50 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac3affd700 1 -- 172.21.15.52:0/2421002205 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fac54032030 con 0x5568efe16f60 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 10 client.18234 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 20 client.18234 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:30.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.665+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe94c30 0x5568efe97050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac6c22cb80 4 client.18234 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5568efd6a290 con 0x5568efe94c30 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac5affd700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe94c30 0x5568efe97050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac5affd700 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe94c30 0x5568efe97050 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f74c44798325934 server_cookie=b3311ea0eee20289 in_seq=0 out_seq=0 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac3affd700 10 client.18234 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac3affd700 1 -- 172.21.15.52:0/2421002205 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5568efd6a290 con 0x5568efe94c30 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.666+0000 7fac3affd700 10 client.18234 handle_command_reply: tid=0 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 client.18234 shutdown 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 msgr2=0x5568efe30f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe30f20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fac5400b010 tx=0x7fac54004d00 comp rx=0 tx=0).stop 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 shutdown_connections 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5568efe94c30 0x5568efe97050 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5568efe16f60 0x5568efe30f20 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5568efe10f20 0x5568efe22e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 --2- 172.21.15.52:0/2421002205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5568efe16640 0x5568efe23360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 >> 172.21.15.52:0/2421002205 conn(0x5568efd3bcc0 msgr2=0x5568efe173a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac3affd700 0 client.18234 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 shutdown_connections 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 1 -- 172.21.15.52:0/2421002205 wait complete. 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.667+0000 7fac6c22cb80 20 client.18234 trim_cache size 0 max 16384 2024-06-02T23:21:30.673 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac440a2c20 msgr2=0x7fac440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac440a2c20 0x7fac440a50e0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fac5c076d90 tx=0x7fac50005fb0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 msgr2=0x7fac5c19e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c19e730 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00e9a0 tx=0x7fac4c00ecb0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 shutdown_connections 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac440a2c20 0x7fac440a50e0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fac5c107500 0x7fac5c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fac5c100e30 0x7fac5c19e730 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 --2- 172.21.15.52:0/1496516970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fac5c1017f0 0x7fac5c19ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:30.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.668+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 >> 172.21.15.52:0/1496516970 conn(0x7fac5c076730 msgr2=0x7fac5c0feee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:30.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.669+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 shutdown_connections 2024-06-02T23:21:30.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:30.669+0000 7fac619a9700 1 -- 172.21.15.52:0/1496516970 wait complete. 2024-06-02T23:21:31.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 180264 1.78 MB/sec execute 400 sec latency 15.826 ms 2024-06-02T23:21:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:31 smithi052 ceph-mon[32340]: pgmap v720: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-02T23:21:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:31 smithi174 ceph-mon[35994]: pgmap v720: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-02T23:21:32.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:31 smithi123 ceph-mon[35977]: pgmap v720: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-02T23:21:32.363 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:21:32.364 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:21:32.638 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:32.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 180726 1.78 MB/sec execute 401 sec latency 15.635 ms 2024-06-02T23:21:33.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 181196 1.79 MB/sec execute 402 sec latency 12.825 ms 2024-06-02T23:21:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:33 smithi174 ceph-mon[35994]: pgmap v721: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:21:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:33 smithi052 ceph-mon[32340]: pgmap v721: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:21:34.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:33 smithi123 ceph-mon[35977]: pgmap v721: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:21:34.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 -- 172.21.15.52:0/1383657057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 msgr2=0x7fa7141017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 --2- 172.21.15.52:0/1383657057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 0x7fa7141017f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa708009990 tx=0x7fa708009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 -- 172.21.15.52:0/1383657057 shutdown_connections 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 --2- 172.21.15.52:0/1383657057 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa714101d30 0x7fa71410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 --2- 172.21.15.52:0/1383657057 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7141073d0 0x7fa7141077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 --2- 172.21.15.52:0/1383657057 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 0x7fa7141017f0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 -- 172.21.15.52:0/1383657057 >> 172.21.15.52:0/1383657057 conn(0x7fa7140782c0 msgr2=0x7fa7140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 -- 172.21.15.52:0/1383657057 shutdown_connections 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 -- 172.21.15.52:0/1383657057 wait complete. 2024-06-02T23:21:34.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.514+0000 7fa719eed700 1 Processor -- start 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 -- start start 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 0x7fa714199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa714101d30 0x7fa714199ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 0x7fa7141a49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141114e0 con 0x7fa714101390 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa714111360 con 0x7fa7141073d0 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa719eed700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa714111660 con 0x7fa714101d30 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 0x7fa714199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa713fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 0x7fa7141a49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa713fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 0x7fa7141a49e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48744/0 (socket says 172.21.15.52:48744) 2024-06-02T23:21:34.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.515+0000 7fa713fff700 1 -- 172.21.15.52:0/2351710399 learned_addr learned my addr 172.21.15.52:0/2351710399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa714101d30 0x7fa714199ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa713fff700 1 -- 172.21.15.52:0/2351710399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa714101d30 msgr2=0x7fa714199ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa713fff700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa714101d30 0x7fa714199ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa713fff700 1 -- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 msgr2=0x7fa714199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa713fff700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 0x7fa714199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa713fff700 1 -- 172.21.15.52:0/2351710399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7141a4f20 con 0x7fa7141073d0 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa713fff700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 0x7fa7141a49e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009ee0 tx=0x7fa70c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa710ff9700 1 -- 172.21.15.52:0/2351710399 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00dd20 con 0x7fa7141073d0 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa710ff9700 1 -- 172.21.15.52:0/2351710399 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa70c004020 con 0x7fa7141073d0 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7141977b0 con 0x7fa7141073d0 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa710ff9700 1 -- 172.21.15.52:0/2351710399 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0127b0 con 0x7fa7141073d0 2024-06-02T23:21:34.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.516+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa714197c70 con 0x7fa7141073d0 2024-06-02T23:21:34.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.519+0000 7fa710ff9700 1 -- 172.21.15.52:0/2351710399 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa70c0041d0 con 0x7fa7141073d0 2024-06-02T23:21:34.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.520+0000 7fa710ff9700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6fc0a2c20 0x7fa6fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.520+0000 7fa7137fe700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6fc0a2c20 0x7fa6fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.520+0000 7fa710ff9700 1 -- 172.21.15.52:0/2351710399 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa70c0d2aa0 con 0x7fa7141073d0 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.520+0000 7fa724770b80 1 Processor -- start 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.520+0000 7fa724770b80 1 -- start start 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.520+0000 7fa7137fe700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6fc0a2c20 0x7fa6fc0a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa704009be0 tx=0x7fa704009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa724770b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa724770b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd6062e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa724770b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd606c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa724770b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564acd4f8fe0 con 0x564acd606820 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa724770b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564acd5578d0 con 0x564acd60c900 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa724770b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564acd4bf4e0 con 0x564acd60bfe0 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd6062e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa713fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd606c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd6062e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48756/0 (socket says 172.21.15.52:48756) 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 -- 172.21.15.52:0/2469770130 learned_addr learned my addr 172.21.15.52:0/2469770130 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:34.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa7137fe700 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 -- 172.21.15.52:0/2469770130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 msgr2=0x564acd60c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 -- 172.21.15.52:0/2469770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 msgr2=0x564acd606c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd606c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 -- 172.21.15.52:0/2469770130 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564acd4a6950 con 0x564acd60c900 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa713fff700 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd606c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.521+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd6062e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa708009990 tx=0x7fa708005c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10deb479d6baec39 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa6f9ffb700 1 -- 172.21.15.52:0/2469770130 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708027070 con 0x564acd60c900 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa6f9ffb700 1 -- 172.21.15.52:0/2469770130 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7080056a0 con 0x564acd60c900 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa6f9ffb700 1 -- 172.21.15.52:0/2469770130 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708003b40 con 0x564acd60c900 2024-06-02T23:21:34.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 -- 172.21.15.52:0/2469770130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 msgr2=0x564acd6062e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd6062e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa708009990 tx=0x7fa708005c70 comp rx=0 tx=0).stop 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 -- 172.21.15.52:0/2469770130 shutdown_connections 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd606c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 --2- 172.21.15.52:0/2469770130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd6062e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.522+0000 7fa724770b80 1 -- 172.21.15.52:0/2469770130 >> 172.21.15.52:0/2469770130 conn(0x564acd533100 msgr2=0x564acd6012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 -- 172.21.15.52:0/2469770130 shutdown_connections 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 -- 172.21.15.52:0/2469770130 wait complete. 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 -- start start 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 Processor -- start 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd612610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd60fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564acd4f8040 con 0x564acd606820 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.523+0000 7fa724770b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564acd553890 con 0x564acd60c900 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa724770b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564acd613280 con 0x564acd60bfe0 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa724770b80 10 client.0 _ll_register_callbacks cb 0x564acd4ed630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd612610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd60fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd612610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37876/0 (socket says 172.21.15.52:37876) 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa7137fe700 1 -- 172.21.15.52:0/532242206 learned_addr learned my addr 172.21.15.52:0/532242206 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa712ffd700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 -- 172.21.15.52:0/532242206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 msgr2=0x564acd60f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 -- 172.21.15.52:0/532242206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 msgr2=0x564acd612610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd612610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 -- 172.21.15.52:0/532242206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564acd51d7d0 con 0x564acd60c900 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa7137fe700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd612610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa713fff700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd60fe50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00d700 tx=0x7fa70c09f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75ef190ff0f22fd7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa6f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.524+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/532242206 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70c0232f0 con 0x564acd60c900 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/532242206 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa70c0a7b10 con 0x564acd60c900 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7080056a0 con 0x564acd60c900 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/532242206 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa70c09fa30 con 0x564acd60c900 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 10 client.27704 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:34.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564acd61d9c0 con 0x564acd60c900 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/532242206 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7080056a0 con 0x564acd60c900 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/532242206 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa70c024030 con 0x564acd60c900 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 10 client.27704 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 10 client.27704 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 20 client.27704 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd61ff50 0x564acd622360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 4 client.27704 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564acd564dd0 con 0x564acd61ff50 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.525+0000 7fa7137fe700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd61ff50 0x564acd622360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.526+0000 7fa7137fe700 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd61ff50 0x564acd622360 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c08a5ade74e224 server_cookie=7492cf09d263b6b6 in_seq=0 out_seq=0 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.526+0000 7fa6f2ffd700 10 client.27704 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/532242206 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564acd564dd0 con 0x564acd61ff50 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa6f2ffd700 10 client.27704 handle_command_reply: tid=0 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 client.27704 shutdown 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 msgr2=0x564acd60fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd60fe50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00d700 tx=0x7fa70c09f870 comp rx=0 tx=0).stop 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 shutdown_connections 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd61ff50 0x564acd622360 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd60fe50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd612610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 --2- 172.21.15.52:0/532242206 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60bfe0 0x564acd60f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 >> 172.21.15.52:0/532242206 conn(0x564acd533100 msgr2=0x564acd600780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.527+0000 7fa6f2ffd700 0 client.27704 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.528+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 shutdown_connections 2024-06-02T23:21:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.528+0000 7fa724770b80 1 -- 172.21.15.52:0/532242206 wait complete. 2024-06-02T23:21:34.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.528+0000 7fa724770b80 20 client.27704 trim_cache size 0 max 16384 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 Processor -- start 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 -- start start 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd60d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60bfe0 0x564acd600f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60c900 0x564acd601440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141aa820 con 0x564acd606820 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141ac360 con 0x564acd60bfe0 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.550+0000 7fa724770b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141aaef0 con 0x564acd60c900 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd60d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60bfe0 0x564acd600f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa7137fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd60d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37890/0 (socket says 172.21.15.52:37890) 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa7137fe700 1 -- 172.21.15.52:0/36063377 learned_addr learned my addr 172.21.15.52:0/36063377 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa713fff700 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60c900 0x564acd601440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 -- 172.21.15.52:0/36063377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60c900 msgr2=0x564acd601440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60c900 0x564acd601440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 -- 172.21.15.52:0/36063377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 msgr2=0x564acd60d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd60d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 -- 172.21.15.52:0/36063377 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564acd6029c0 con 0x564acd60bfe0 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa7137fe700 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd60d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa712ffd700 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60bfe0 0x564acd600f00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7080036c0 tx=0x7fa70800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fae3202ebf232e12 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa6f97fa700 1 -- 172.21.15.52:0/36063377 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708031070 con 0x564acd60bfe0 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.551+0000 7fa6f97fa700 1 -- 172.21.15.52:0/36063377 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa708028040 con 0x564acd60bfe0 2024-06-02T23:21:34.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa6f97fa700 1 -- 172.21.15.52:0/36063377 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708027040 con 0x564acd60bfe0 2024-06-02T23:21:34.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 -- 172.21.15.52:0/36063377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60bfe0 msgr2=0x564acd600f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60bfe0 0x564acd600f00 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa7080036c0 tx=0x7fa70800b390 comp rx=0 tx=0).stop 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 -- 172.21.15.52:0/36063377 shutdown_connections 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd60c900 0x564acd601440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd606820 0x564acd60d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 --2- 172.21.15.52:0/36063377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60bfe0 0x564acd600f00 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 -- 172.21.15.52:0/36063377 >> 172.21.15.52:0/36063377 conn(0x564acd533100 msgr2=0x564acd60cce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 -- 172.21.15.52:0/36063377 shutdown_connections 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.552+0000 7fa724770b80 1 -- 172.21.15.52:0/36063377 wait complete. 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 -- start start 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 Processor -- start 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd606820 0x564acd618850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 0x564acd618d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd615630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141abc90 con 0x564acd60bfe0 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141ab5c0 con 0x564acd60c900 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7141a87e0 con 0x564acd606820 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.553+0000 7fa724770b80 10 client.0 _ll_register_callbacks cb 0x564acd4ed630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 0x564acd618d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 0x564acd618d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37894/0 (socket says 172.21.15.52:37894) 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 -- 172.21.15.52:0/2236545665 learned_addr learned my addr 172.21.15.52:0/2236545665 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa713fff700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd615630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa7137fe700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd606820 0x564acd618850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 -- 172.21.15.52:0/2236545665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd606820 msgr2=0x564acd618850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd606820 0x564acd618850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 -- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 msgr2=0x564acd615630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd615630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 -- 172.21.15.52:0/2236545665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564acd615e40 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa712ffd700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 0x564acd618d90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa708025040 tx=0x7fa7080056b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45c3d426e02c54c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa6f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.554+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/2236545665 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708039070 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/2236545665 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa708003790 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa708028040 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/2236545665 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa708027070 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 10 client.18252 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x564acd68b850 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/2236545665 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa708028040 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/2236545665 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa70803c020 con 0x564acd60bfe0 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 10 client.18252 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 20 client.18252 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:34.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd68ba80 0x564acd68de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:34.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 4 client.18252 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:21:34.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.555+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564acd55f350 con 0x564acd68ba80 2024-06-02T23:21:34.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.556+0000 7fa7137fe700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd68ba80 0x564acd68de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:34.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.556+0000 7fa7137fe700 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd68ba80 0x564acd68de70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32422459dfb946de server_cookie=fd92ca29e9e5c461 in_seq=0 out_seq=0 2024-06-02T23:21:34.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.556+0000 7fa6f2ffd700 10 client.18252 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:34.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.557+0000 7fa6f2ffd700 1 -- 172.21.15.52:0/2236545665 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564acd55f350 con 0x564acd68ba80 2024-06-02T23:21:34.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.557+0000 7fa6f2ffd700 10 client.18252 handle_command_reply: tid=0 2024-06-02T23:21:34.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.557+0000 7fa724770b80 1 client.18252 shutdown 2024-06-02T23:21:34.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 msgr2=0x564acd618d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 0x564acd618d90 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7fa708025040 tx=0x7fa7080056b0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 shutdown_connections 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564acd68ba80 0x564acd68de70 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564acd60c900 0x564acd615630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564acd606820 0x564acd618850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 --2- 172.21.15.52:0/2236545665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564acd60bfe0 0x564acd618d90 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 >> 172.21.15.52:0/2236545665 conn(0x564acd533100 msgr2=0x564acd611140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa6f2ffd700 0 client.18252 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 shutdown_connections 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 1 -- 172.21.15.52:0/2236545665 wait complete. 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.558+0000 7fa724770b80 20 client.18252 trim_cache size 0 max 16384 2024-06-02T23:21:34.562 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:21:34.563 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:21:34.563 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "6c4da0b2-656f-4e60-84b0-648e650c2a82", 2024-06-02T23:21:34.563 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:21:34.563 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6fc0a2c20 msgr2=0x7fa6fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6fc0a2c20 0x7fa6fc0a50e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa704009be0 tx=0x7fa704009260 comp rx=0 tx=0).stop 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 msgr2=0x7fa7141a49e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 0x7fa7141a49e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009ee0 tx=0x7fa70c00d3b0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 shutdown_connections 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6fc0a2c20 0x7fa6fc0a50e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7141073d0 0x7fa7141a49e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa714101390 0x7fa714199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 --2- 172.21.15.52:0/2351710399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa714101d30 0x7fa714199ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 >> 172.21.15.52:0/2351710399 conn(0x7fa7140782c0 msgr2=0x7fa714100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 shutdown_connections 2024-06-02T23:21:34.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:34.559+0000 7fa719eed700 1 -- 172.21.15.52:0/2351710399 wait complete. 2024-06-02T23:21:34.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 181445 1.79 MB/sec execute 403 sec latency 33.261 ms 2024-06-02T23:21:35.315 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:21:35.597 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:21:35.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 181856 1.78 MB/sec execute 404 sec latency 14.884 ms 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: pgmap v722: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:21:35.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.860 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: pgmap v722: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:21:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:21:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:35.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: pgmap v722: 97 pgs: 97 active+clean; 82 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 219 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:21:36.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 182385 1.79 MB/sec execute 405 sec latency 68.674 ms 2024-06-02T23:21:37.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 182950 1.79 MB/sec execute 406 sec latency 69.077 ms 2024-06-02T23:21:37.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 -- 172.21.15.52:0/1034536736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 msgr2=0x7f81f0101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 --2- 172.21.15.52:0/1034536736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 0x7f81f0101800 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f81dc009990 tx=0x7f81dc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 -- 172.21.15.52:0/1034536736 shutdown_connections 2024-06-02T23:21:37.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 --2- 172.21.15.52:0/1034536736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81f0101d40 0x7f81f010e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 --2- 172.21.15.52:0/1034536736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f01073f0 0x7f81f01077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 --2- 172.21.15.52:0/1034536736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 0x7f81f0101800 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 -- 172.21.15.52:0/1034536736 >> 172.21.15.52:0/1034536736 conn(0x7f81f0078300 msgr2=0x7f81f0078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 -- 172.21.15.52:0/1034536736 shutdown_connections 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.516+0000 7f81f71ea700 1 -- 172.21.15.52:0/1034536736 wait complete. 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 Processor -- start 2024-06-02T23:21:37.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 -- start start 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 0x7f81f01082b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 0x7f81f01087f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81f01073f0 0x7f81f0108d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81f0111560 con 0x7f81f0101380 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01113e0 con 0x7f81f01073f0 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f71ea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01116e0 con 0x7f81f0101d40 2024-06-02T23:21:37.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81f4f86700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 0x7f81f01082b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.517+0000 7f81effff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 0x7f81f01087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 0x7f81f01087f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:55486/0 (socket says 172.21.15.52:55486) 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 -- 172.21.15.52:0/200418512 learned_addr learned my addr 172.21.15.52:0/200418512 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81f5787700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81f01073f0 0x7f81f0108d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 -- 172.21.15.52:0/200418512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81f01073f0 msgr2=0x7f81f0108d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81f01073f0 0x7f81f0108d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 -- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 msgr2=0x7f81f01082b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 0x7f81f01082b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 -- 172.21.15.52:0/200418512 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81f01ab430 con 0x7f81f0101d40 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81effff700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 0x7f81f01087f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0047b0 tx=0x7f81dc00bfb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:37.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81edffb700 1 -- 172.21.15.52:0/200418512 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc027070 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81f01ab6c0 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81edffb700 1 -- 172.21.15.52:0/200418512 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81dc022470 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.518+0000 7f81edffb700 1 -- 172.21.15.52:0/200418512 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc00b610 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.519+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81f01abb70 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.521+0000 7f81edffb700 1 -- 172.21.15.52:0/200418512 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f81dc02a070 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.522+0000 7f81edffb700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81e00a2c20 0x7f81e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.522+0000 7f81f4f86700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81e00a2c20 0x7f81e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.522+0000 7f81edffb700 1 -- 172.21.15.52:0/200418512 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f81dc0cfd10 con 0x7f81f0101d40 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 Processor -- start 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f81f4f86700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81e00a2c20 0x7f81e00a50e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009be0 tx=0x7f81e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 -- start start 2024-06-02T23:21:37.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a7da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a78340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561a639c9810 con 0x561a63a7d640 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561a63971720 con 0x561a63a77f20 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f8201a6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a639c57d0 con 0x561a63a7df60 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f81f5787700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a78340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f81f5787700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a78340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48814/0 (socket says 172.21.15.52:48814) 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f81effff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.523+0000 7f81f5787700 1 -- 172.21.15.52:0/586954680 learned_addr learned my addr 172.21.15.52:0/586954680 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:37.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81f4f86700 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a7da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81effff700 1 -- 172.21.15.52:0/586954680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 msgr2=0x561a63a78340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81effff700 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a78340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81effff700 1 -- 172.21.15.52:0/586954680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 msgr2=0x561a63a7da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81effff700 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a7da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81effff700 1 -- 172.21.15.52:0/586954680 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a63967f50 con 0x561a63a7df60 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81f4f86700 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81effff700 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a779e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81dc09c2d0 tx=0x7f81dc09c4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd893559bff9edbf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81daffd700 1 -- 172.21.15.52:0/586954680 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc09ccd0 con 0x561a63a7df60 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81daffd700 1 -- 172.21.15.52:0/586954680 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81dc0bc040 con 0x561a63a7df60 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.524+0000 7f81daffd700 1 -- 172.21.15.52:0/586954680 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81dc09c830 con 0x561a63a7df60 2024-06-02T23:21:37.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 -- 172.21.15.52:0/586954680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 msgr2=0x561a63a779e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a779e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f81dc09c2d0 tx=0x7f81dc09c4c0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 -- 172.21.15.52:0/586954680 shutdown_connections 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a78340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a7da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 --2- 172.21.15.52:0/586954680 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a779e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 -- 172.21.15.52:0/586954680 >> 172.21.15.52:0/586954680 conn(0x561a639a2cc0 msgr2=0x561a63a72850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 -- 172.21.15.52:0/586954680 shutdown_connections 2024-06-02T23:21:37.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 -- 172.21.15.52:0/586954680 wait complete. 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 -- start start 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.525+0000 7f8201a6db80 1 Processor -- start 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a83ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a80fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a81520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561a63a846f0 con 0x561a63a7d640 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561a63969bf0 con 0x561a63a77f20 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a63a84870 con 0x561a63a7df60 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f8201a6db80 10 client.0 _ll_register_callbacks cb 0x561a63963920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a80fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a80fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37928/0 (socket says 172.21.15.52:37928) 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81effff700 1 -- 172.21.15.52:0/3257320633 learned_addr learned my addr 172.21.15.52:0/3257320633 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a83ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f5787700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a81520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 -- 172.21.15.52:0/3257320633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 msgr2=0x561a63a81520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a81520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 -- 172.21.15.52:0/3257320633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 msgr2=0x561a63a80fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a80fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 -- 172.21.15.52:0/3257320633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a639188d0 con 0x561a63a77f20 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81effff700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a80fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:37.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81f4f86700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a83ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f81e40117d0 tx=0x7f81e40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=362503f197b87325 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.526+0000 7f81cffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f81cffff700 1 -- 172.21.15.52:0/3257320633 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e401ee30 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81dc0bc040 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f81cffff700 1 -- 172.21.15.52:0/3257320633 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f81e401f430 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f81cffff700 1 -- 172.21.15.52:0/3257320633 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f81e401f630 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 10 client.27722 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a63a8f000 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f81cffff700 1 -- 172.21.15.52:0/3257320633 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81dc0bc040 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f81cffff700 1 -- 172.21.15.52:0/3257320633 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f81e4033830 con 0x561a63a77f20 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 10 client.27722 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 10 client.27722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 20 client.27722 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:37.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63a91590 0x561a63a939d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 4 client.27722 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.527+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a639d6d10 con 0x561a63a91590 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.528+0000 7f81effff700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63a91590 0x561a63a939d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.528+0000 7f81cffff700 10 client.27722 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.528+0000 7f81effff700 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63a91590 0x561a63a939d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f62e3e0ad336a7b server_cookie=68d3c22da9951d6c in_seq=0 out_seq=0 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f81cffff700 1 -- 172.21.15.52:0/3257320633 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a639d6d10 con 0x561a63a91590 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f81cffff700 10 client.27722 handle_command_reply: tid=0 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 client.27722 shutdown 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 msgr2=0x561a63a83ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a83ce0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f81e40117d0 tx=0x7f81e40047c0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 shutdown_connections 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63a91590 0x561a63a939d0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63a81520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a83ce0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f81cffff700 0 client.27722 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 --2- 172.21.15.52:0/3257320633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a80fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 >> 172.21.15.52:0/3257320633 conn(0x561a639a2cc0 msgr2=0x561a63a725f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 shutdown_connections 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 1 -- 172.21.15.52:0/3257320633 wait complete. 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.529+0000 7f8201a6db80 20 client.27722 trim_cache size 0 max 16384 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.552+0000 7f8201a6db80 1 Processor -- start 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.552+0000 7f8201a6db80 1 -- start start 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a77f20 0x561a63a779e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a7d640 0x561a63a6c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 0x561a63a992a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f8201a6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b3050 con 0x561a63a7df60 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f8201a6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b4b90 con 0x561a63a7d640 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f8201a6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b3720 con 0x561a63a77f20 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 0x561a63a992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 0x561a63a992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37938/0 (socket says 172.21.15.52:37938) 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a7d640 0x561a63a6c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 -- 172.21.15.52:0/2316782749 learned_addr learned my addr 172.21.15.52:0/2316782749 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f4f86700 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a77f20 0x561a63a779e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 -- 172.21.15.52:0/2316782749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a77f20 msgr2=0x561a63a779e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a77f20 0x561a63a779e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 -- 172.21.15.52:0/2316782749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a7d640 msgr2=0x561a63a6c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a7d640 0x561a63a6c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 -- 172.21.15.52:0/2316782749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a63a94050 con 0x561a63a7df60 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81effff700 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a7d640 0x561a63a6c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:37.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.553+0000 7f81f5787700 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 0x561a63a992a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f81e8004390 tx=0x7f81e8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569a945ea29c9e3f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.554+0000 7f81da7fc700 1 -- 172.21.15.52:0/2316782749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8007500 con 0x561a63a7df60 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.554+0000 7f81da7fc700 1 -- 172.21.15.52:0/2316782749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e8007b50 con 0x561a63a7df60 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.554+0000 7f81da7fc700 1 -- 172.21.15.52:0/2316782749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e8017bc0 con 0x561a63a7df60 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.554+0000 7f8201a6db80 1 -- 172.21.15.52:0/2316782749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 msgr2=0x561a63a992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.554+0000 7f8201a6db80 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 0x561a63a992a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f81e8004390 tx=0x7f81e8009920 comp rx=0 tx=0).stop 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 -- 172.21.15.52:0/2316782749 shutdown_connections 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7df60 0x561a63a992a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a77f20 0x561a63a779e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 --2- 172.21.15.52:0/2316782749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a7d640 0x561a63a6c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 -- 172.21.15.52:0/2316782749 >> 172.21.15.52:0/2316782749 conn(0x561a639a2cc0 msgr2=0x561a63a7e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 -- 172.21.15.52:0/2316782749 shutdown_connections 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 -- 172.21.15.52:0/2316782749 wait complete. 2024-06-02T23:21:37.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 -- start start 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.555+0000 7f8201a6db80 1 Processor -- start 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a85f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63afb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b44c0 con 0x561a63a7d640 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b3df0 con 0x561a63a77f20 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81f01b1010 con 0x561a63a7df60 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f8201a6db80 10 client.0 _ll_register_callbacks cb 0x561a63963920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81effff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a864d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37946/0 (socket says 172.21.15.52:37946) 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81effff700 1 -- 172.21.15.52:0/6075567 learned_addr learned my addr 172.21.15.52:0/6075567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:37.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a85f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f5787700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63afb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 -- 172.21.15.52:0/6075567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 msgr2=0x561a63afb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63afb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 -- 172.21.15.52:0/6075567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 msgr2=0x561a63a864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 -- 172.21.15.52:0/6075567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a63afb7b0 con 0x561a63a77f20 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81effff700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81f4f86700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a85f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f81e4016040 tx=0x7f81e401ef30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb9bbb98ec075973 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.556+0000 7f81cffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:37.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f81cffff700 1 -- 172.21.15.52:0/6075567 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81e401f430 con 0x561a63a77f20 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: "6c4da0b2-656f-4e60-84b0-648e650c2a82": { 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "6c4da0b2-656f-4e60-84b0-648e650c2a82", 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:21:37.683 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:21:37.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f81cffff700 1 -- 172.21.15.52:0/6075567 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f81e4033630 con 0x561a63a77f20 2024-06-02T23:21:37.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81e8007b50 con 0x561a63a77f20 2024-06-02T23:21:37.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f81cffff700 1 -- 172.21.15.52:0/6075567 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f81e4033830 con 0x561a63a77f20 2024-06-02T23:21:37.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 10 client.27728 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:37.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a63afcac0 con 0x561a63a77f20 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f81cffff700 1 -- 172.21.15.52:0/6075567 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f81e8007b50 con 0x561a63a77f20 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f81cffff700 1 -- 172.21.15.52:0/6075567 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f81e40465c0 con 0x561a63a77f20 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 10 client.27728 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 10 client.27728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 20 client.27728 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63afd190 0x561a63aff5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 4 client.27728 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.557+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561a639d1290 con 0x561a63afd190 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f81effff700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63afd190 0x561a63aff5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f81effff700 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63afd190 0x561a63aff5b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=471cc8b8c909be5f server_cookie=17d18a271c44801 in_seq=0 out_seq=0 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f81cffff700 10 client.27728 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f81cffff700 1 -- 172.21.15.52:0/6075567 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561a639d1290 con 0x561a63afd190 2024-06-02T23:21:37.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f81cffff700 10 client.27728 handle_command_reply: tid=0 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 client.27728 shutdown 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 msgr2=0x561a63a85f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a85f90 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f81e4016040 tx=0x7f81e401ef30 comp rx=0 tx=0).stop 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 shutdown_connections 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a63afd190 0x561a63aff5b0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a63a7df60 0x561a63afb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a63a77f20 0x561a63a85f90 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 --2- 172.21.15.52:0/6075567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a63a7d640 0x561a63a864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f81cffff700 0 client.27728 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 >> 172.21.15.52:0/6075567 conn(0x561a639a2cc0 msgr2=0x561a63a7c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.558+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 shutdown_connections 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f8201a6db80 1 -- 172.21.15.52:0/6075567 wait complete. 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f8201a6db80 20 client.27728 trim_cache size 0 max 16384 2024-06-02T23:21:37.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81e00a2c20 msgr2=0x7f81e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81e00a2c20 0x7f81e00a50e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f81e4009be0 tx=0x7f81e4009260 comp rx=0 tx=0).stop 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 msgr2=0x7f81f01087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 0x7f81f01087f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f81dc0047b0 tx=0x7f81dc00bfb0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 shutdown_connections 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81e00a2c20 0x7f81e00a50e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81f01073f0 0x7f81f0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f81f0101380 0x7f81f01082b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 --2- 172.21.15.52:0/200418512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81f0101d40 0x7f81f01087f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 >> 172.21.15.52:0/200418512 conn(0x7f81f0078300 msgr2=0x7f81f01004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 shutdown_connections 2024-06-02T23:21:37.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:21:37.559+0000 7f81f71ea700 1 -- 172.21.15.52:0/200418512 wait complete. 2024-06-02T23:21:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:37 smithi174 ceph-mon[35994]: pgmap v723: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-06-02T23:21:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:37 smithi052 ceph-mon[32340]: pgmap v723: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-06-02T23:21:38.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:37 smithi123 ceph-mon[35977]: pgmap v723: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 1.5 MiB/s wr, 176 op/s 2024-06-02T23:21:38.358 INFO:tasks.cephfs.filesystem:scrub status for tag:6c4da0b2-656f-4e60-84b0-648e650c2a82 - {'path': '/', 'tag': '6c4da0b2-656f-4e60-84b0-648e650c2a82', 'options': 'recursive,force'} 2024-06-02T23:21:38.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 183483 1.79 MB/sec execute 407 sec latency 14.686 ms 2024-06-02T23:21:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 -- 172.21.15.52:0/459348451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74101a30 msgr2=0x7f9d74101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 --2- 172.21.15.52:0/459348451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74101a30 0x7f9d74101e50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009900 tx=0x7f9d7002a410 comp rx=0 tx=0).stop 2024-06-02T23:21:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 -- 172.21.15.52:0/459348451 shutdown_connections 2024-06-02T23:21:38.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 --2- 172.21.15.52:0/459348451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 0x7f9d74109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 --2- 172.21.15.52:0/459348451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74102c20 0x7f9d741030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 --2- 172.21.15.52:0/459348451 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74101a30 0x7f9d74101e50 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 -- 172.21.15.52:0/459348451 >> 172.21.15.52:0/459348451 conn(0x7f9d740fcfc0 msgr2=0x7f9d740ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 -- 172.21.15.52:0/459348451 shutdown_connections 2024-06-02T23:21:38.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 -- 172.21.15.52:0/459348451 wait complete. 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.710+0000 7f9d7b342640 1 Processor -- start 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 -- start start 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74101a30 0x7f9d74197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 0x7f9d741980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 0x7f9d7419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7410d560 con 0x7f9d74102c20 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7410d3e0 con 0x7f9d74101a30 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d7b342640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d7410d6e0 con 0x7f9d741035e0 2024-06-02T23:21:38.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 0x7f9d741980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d790de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74101a30 0x7f9d74197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 0x7f9d741980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37968/0 (socket says 172.21.15.52:37968) 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 -- 172.21.15.52:0/61003958 learned_addr learned my addr 172.21.15.52:0/61003958 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d798df640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 0x7f9d7419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 -- 172.21.15.52:0/61003958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 msgr2=0x7f9d7419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 0x7f9d7419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 -- 172.21.15.52:0/61003958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74101a30 msgr2=0x7f9d74197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74101a30 0x7f9d74197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 -- 172.21.15.52:0/61003958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d7419f7a0 con 0x7f9d74102c20 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d798df640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 0x7f9d7419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d790de640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74101a30 0x7f9d74197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.711+0000 7f9d788dd640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 0x7f9d741980c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400e9f0 tx=0x7f9d6400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.712+0000 7f9d6a7fc640 1 -- 172.21.15.52:0/61003958 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6400cd80 con 0x7f9d74102c20 2024-06-02T23:21:38.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.712+0000 7f9d6a7fc640 1 -- 172.21.15.52:0/61003958 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d640108e0 con 0x7f9d74102c20 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.712+0000 7f9d6a7fc640 1 -- 172.21.15.52:0/61003958 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64018920 con 0x7f9d74102c20 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.712+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d7419fa90 con 0x7f9d74102c20 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.712+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d741a0030 con 0x7f9d74102c20 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.714+0000 7f9d6a7fc640 1 -- 172.21.15.52:0/61003958 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d64010a90 con 0x7f9d74102c20 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 Processor -- start 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 -- start start 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23010 0x560007d23430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d28500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d2b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560007c864f0 con 0x560007d23a00 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560007c24870 con 0x560007d28b70 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d7e053740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560007c824b0 con 0x560007d23010 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d6a7fc640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d600a2cb0 0x7f9d600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d790de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d28500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d798df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23010 0x560007d23430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d788dd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d2b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d798df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23010 0x560007d23430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:55560/0 (socket says 172.21.15.52:55560) 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d798df640 1 -- 172.21.15.52:0/2399597712 learned_addr learned my addr 172.21.15.52:0/2399597712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d798df640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d600a2cb0 0x7f9d600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.715+0000 7f9d6a7fc640 1 -- 172.21.15.52:0/61003958 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d640cf350 con 0x7f9d74102c20 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d790de640 1 -- 172.21.15.52:0/2399597712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23010 msgr2=0x560007d23430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d790de640 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23010 0x560007d23430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d790de640 1 -- 172.21.15.52:0/2399597712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 msgr2=0x560007d2b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d790de640 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d2b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d790de640 1 -- 172.21.15.52:0/2399597712 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560007c409b0 con 0x560007d23a00 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d798df640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d600a2cb0 0x7f9d600a5170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009270 tx=0x7f9d6c009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:38.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d790de640 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d28500 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009230 tx=0x7f9d70032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fff868781b4ddca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d5f7fe640 1 -- 172.21.15.52:0/2399597712 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70035070 con 0x560007d23a00 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d5f7fe640 1 -- 172.21.15.52:0/2399597712 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d700044d0 con 0x560007d23a00 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.716+0000 7f9d5f7fe640 1 -- 172.21.15.52:0/2399597712 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70003e20 con 0x560007d23a00 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 -- 172.21.15.52:0/2399597712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 msgr2=0x560007d28500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d28500 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f9d70009230 tx=0x7f9d70032eb0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 -- 172.21.15.52:0/2399597712 shutdown_connections 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d2b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d28500 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2399597712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23010 0x560007d23430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 -- 172.21.15.52:0/2399597712 >> 172.21.15.52:0/2399597712 conn(0x560007d1e680 msgr2=0x560007d1ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 -- 172.21.15.52:0/2399597712 shutdown_connections 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 -- 172.21.15.52:0/2399597712 wait complete. 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.717+0000 7f9d7e053740 1 -- start start 2024-06-02T23:21:38.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 Processor -- start 2024-06-02T23:21:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d2d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23a00 0x560007d2d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d34800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560007d2bac0 con 0x560007d23010 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560007c0fae0 con 0x560007d28b70 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560007d2bc40 con 0x560007d23a00 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d790de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d2d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d7e053740 10 client.0 _ll_register_callbacks cb 0x560007c489b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d790de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d2d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37990/0 (socket says 172.21.15.52:37990) 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d798df640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d34800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d790de640 1 -- 172.21.15.52:0/1999937580 learned_addr learned my addr 172.21.15.52:0/1999937580 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.718+0000 7f9d788dd640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23a00 0x560007d2d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d798df640 1 -- 172.21.15.52:0/1999937580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23a00 msgr2=0x560007d2d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d798df640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23a00 0x560007d2d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d798df640 1 -- 172.21.15.52:0/1999937580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 msgr2=0x560007d2d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d798df640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d2d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d798df640 1 -- 172.21.15.52:0/1999937580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56000799a210 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d790de640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d798df640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d34800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x560007d24260 tx=0x7f9d6c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c5f81e5d314ffb6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d5cff9640 1 -- 172.21.15.52:0/1999937580 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c027070 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d700044d0 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.719+0000 7f9d5cff9640 1 -- 172.21.15.52:0/1999937580 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d6c004790 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d5cff9640 1 -- 172.21.15.52:0/1999937580 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d6c033050 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 10 client.27740 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560007a66620 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d5cff9640 1 -- 172.21.15.52:0/1999937580 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d700044d0 con 0x560007d28b70 2024-06-02T23:21:38.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d5cff9640 1 -- 172.21.15.52:0/1999937580 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d6c00f4b0 con 0x560007d28b70 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 10 client.27740 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 10 client.27740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 20 client.27740 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007d390f0 0x560007d3b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 4 client.27740 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560007c99f10 con 0x560007d390f0 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.720+0000 7f9d790de640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007d390f0 0x560007d3b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d5cff9640 10 client.27740 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d790de640 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007d390f0 0x560007d3b540 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4415b7f6cb6b9d30 server_cookie=5008351e558f4f02 in_seq=0 out_seq=0 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d5cff9640 1 -- 172.21.15.52:0/1999937580 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560007c99f10 con 0x560007d390f0 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d5cff9640 10 client.27740 handle_command_reply: tid=0 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d7e053740 1 client.27740 shutdown 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 msgr2=0x560007d34800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.721+0000 7f9d7e053740 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d34800 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x560007d24260 tx=0x7f9d6c009a20 comp rx=0 tx=0).stop 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 shutdown_connections 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007d390f0 0x560007d3b540 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d23a00 0x560007d2d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d28b70 0x560007d34800 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d5cff9640 0 client.27740 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 --2- 172.21.15.52:0/1999937580 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d2d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 >> 172.21.15.52:0/1999937580 conn(0x560007d1e680 msgr2=0x560007d1d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 shutdown_connections 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 1 -- 172.21.15.52:0/1999937580 wait complete. 2024-06-02T23:21:38.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.722+0000 7f9d7e053740 20 client.27740 trim_cache size 0 max 16384 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 Processor -- start 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 -- start start 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23010 0x560007d2af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d347c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007d1cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d741a8d00 con 0x560007d23a00 2024-06-02T23:21:38.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d741aa840 con 0x560007d23010 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d7e053740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d741a93d0 con 0x560007d28b70 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d798df640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007d1cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d790de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23010 0x560007d2af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d347c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37994/0 (socket says 172.21.15.52:37994) 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 -- 172.21.15.52:0/2516368107 learned_addr learned my addr 172.21.15.52:0/2516368107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 -- 172.21.15.52:0/2516368107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 msgr2=0x560007d1cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007d1cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 -- 172.21.15.52:0/2516368107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23010 msgr2=0x560007d2af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23010 0x560007d2af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d788dd640 1 -- 172.21.15.52:0/2516368107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560007d23de0 con 0x560007d23a00 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.742+0000 7f9d798df640 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007d1cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.743+0000 7f9d788dd640 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d347c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9d64098550 tx=0x7f9d64025b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6e6da3c606a81bb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:38.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.743+0000 7f9d5effd640 1 -- 172.21.15.52:0/2516368107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6409ff30 con 0x560007d23a00 2024-06-02T23:21:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.743+0000 7f9d5effd640 1 -- 172.21.15.52:0/2516368107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d640a47f0 con 0x560007d23a00 2024-06-02T23:21:38.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.743+0000 7f9d5effd640 1 -- 172.21.15.52:0/2516368107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d640a4360 con 0x560007d23a00 2024-06-02T23:21:38.746 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 -- 172.21.15.52:0/2516368107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 msgr2=0x560007d347c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d347c0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f9d64098550 tx=0x7f9d64025b40 comp rx=0 tx=0).stop 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 -- 172.21.15.52:0/2516368107 shutdown_connections 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007d1cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23a00 0x560007d347c0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 --2- 172.21.15.52:0/2516368107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23010 0x560007d2af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 -- 172.21.15.52:0/2516368107 >> 172.21.15.52:0/2516368107 conn(0x560007d1e680 msgr2=0x560007d24c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 -- 172.21.15.52:0/2516368107 shutdown_connections 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 -- 172.21.15.52:0/2516368107 wait complete. 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 -- start start 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.744+0000 7f9d7e053740 1 Processor -- start 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d31800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23a00 0x560007d31d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007da1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d741aa170 con 0x560007d23010 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d741a9aa0 con 0x560007d23a00 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d741a6ce0 con 0x560007d28b70 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d790de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d31800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d7e053740 10 client.0 _ll_register_callbacks cb 0x7f9d6c009e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d790de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d31800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38010/0 (socket says 172.21.15.52:38010) 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d790de640 1 -- 172.21.15.52:0/3904211664 learned_addr learned my addr 172.21.15.52:0/3904211664 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d798df640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007da1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d788dd640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23a00 0x560007d31d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d798df640 1 -- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23a00 msgr2=0x560007d31d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d798df640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23a00 0x560007d31d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d798df640 1 -- 172.21.15.52:0/3904211664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 msgr2=0x560007d31800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d798df640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d31800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.745+0000 7f9d798df640 1 -- 172.21.15.52:0/3904211664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560007da6810 con 0x560007d28b70 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d798df640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007da1e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c011c40 tx=0x7f9d6c0146a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=473460b462a2665 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d5cff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d5cff9640 1 -- 172.21.15.52:0/3904211664 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6c027040 con 0x560007d28b70 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d5cff9640 1 -- 172.21.15.52:0/3904211664 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d6c00f420 con 0x560007d28b70 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d640a47f0 con 0x560007d28b70 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d5cff9640 1 -- 172.21.15.52:0/3904211664 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d6c046050 con 0x560007d28b70 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d7e053740 10 client.27505 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560007da7b30 con 0x560007d28b70 2024-06-02T23:21:38.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.746+0000 7f9d5cff9640 1 -- 172.21.15.52:0/3904211664 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d6c00fd50 con 0x560007d28b70 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d5cff9640 1 -- 172.21.15.52:0/3904211664 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d6c02cb50 con 0x560007d28b70 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d7e053740 10 client.27505 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d7e053740 10 client.27505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d7e053740 20 client.27505 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d7e053740 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007da84a0 0x560007daa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d7e053740 4 client.27505 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:21:38.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9d640a47f0 con 0x560007da84a0 2024-06-02T23:21:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d790de640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007da84a0 0x560007daa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d5cff9640 10 client.27505 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:38.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.747+0000 7f9d790de640 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007da84a0 0x560007daa8c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc181cd8d5bec391 server_cookie=c15b5e2da59dd87a in_seq=0 out_seq=0 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d5cff9640 1 -- 172.21.15.52:0/3904211664 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f9d640a47f0 con 0x560007da84a0 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d5cff9640 10 client.27505 handle_command_reply: tid=0 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 client.27505 shutdown 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 msgr2=0x560007da1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007da1e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c011c40 tx=0x7f9d6c0146a0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 shutdown_connections 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560007da84a0 0x560007daa8c0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560007d28b70 0x560007da1e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560007d23a00 0x560007d31d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d5cff9640 0 client.27505 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 --2- 172.21.15.52:0/3904211664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560007d23010 0x560007d31800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 >> 172.21.15.52:0/3904211664 conn(0x560007d1e680 msgr2=0x560007d24c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 shutdown_connections 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.767+0000 7f9d7e053740 1 -- 172.21.15.52:0/3904211664 wait complete. 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7e053740 20 client.27505 trim_cache size 0 max 16384 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:21:38.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d600a2cb0 msgr2=0x7f9d600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d600a2cb0 0x7f9d600a5170 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c009270 tx=0x7f9d6c009be0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 msgr2=0x7f9d741980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 0x7f9d741980c0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f9d6400e9f0 tx=0x7f9d6400eeb0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 shutdown_connections 2024-06-02T23:21:38.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d600a2cb0 0x7f9d600a5170 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d741035e0 0x7f9d7419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d74102c20 0x7f9d741980c0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 --2- 172.21.15.52:0/61003958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d74101a30 0x7f9d74197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.768+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 >> 172.21.15.52:0/61003958 conn(0x7f9d740fcfc0 msgr2=0x7f9d74105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.769+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 shutdown_connections 2024-06-02T23:21:38.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:38.769+0000 7f9d7b342640 1 -- 172.21.15.52:0/61003958 wait complete. 2024-06-02T23:21:39.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 184219 1.80 MB/sec execute 408 sec latency 12.688 ms 2024-06-02T23:21:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:39 smithi174 ceph-mon[35994]: pgmap v724: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-02T23:21:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:39 smithi052 ceph-mon[32340]: pgmap v724: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-02T23:21:40.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:39 smithi123 ceph-mon[35977]: pgmap v724: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.1 MiB/s wr, 132 op/s 2024-06-02T23:21:40.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 184680 1.81 MB/sec execute 409 sec latency 14.958 ms 2024-06-02T23:21:41.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 184935 1.80 MB/sec execute 410 sec latency 14.519 ms 2024-06-02T23:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:41 smithi174 ceph-mon[35994]: pgmap v725: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-02T23:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:21:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:21:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:41 smithi052 ceph-mon[32340]: pgmap v725: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-02T23:21:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:21:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:21:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:21:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:21:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:41 smithi123 ceph-mon[35977]: pgmap v725: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 311 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-06-02T23:21:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:21:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:21:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:21:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:21:42.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 185393 1.80 MB/sec execute 411 sec latency 15.173 ms 2024-06-02T23:21:43.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 185967 1.81 MB/sec execute 412 sec latency 14.596 ms 2024-06-02T23:21:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:43 smithi174 ceph-mon[35994]: pgmap v726: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:21:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:43 smithi052 ceph-mon[32340]: pgmap v726: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:21:44.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:43 smithi123 ceph-mon[35977]: pgmap v726: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 1.2 MiB/s wr, 144 op/s 2024-06-02T23:21:44.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 186633 1.80 MB/sec execute 413 sec latency 11.875 ms 2024-06-02T23:21:45.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 187217 1.81 MB/sec execute 414 sec latency 16.984 ms 2024-06-02T23:21:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:45 smithi174 ceph-mon[35994]: pgmap v727: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-02T23:21:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:45 smithi052 ceph-mon[32340]: pgmap v727: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-02T23:21:46.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:45 smithi123 ceph-mon[35977]: pgmap v727: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 149 op/s 2024-06-02T23:21:46.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 187805 1.82 MB/sec execute 415 sec latency 14.905 ms 2024-06-02T23:21:47.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 188297 1.82 MB/sec execute 416 sec latency 14.712 ms 2024-06-02T23:21:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:47 smithi174 ceph-mon[35994]: pgmap v728: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-02T23:21:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:47 smithi052 ceph-mon[32340]: pgmap v728: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-02T23:21:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:47 smithi123 ceph-mon[35977]: pgmap v728: 97 pgs: 97 active+clean; 81 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 329 KiB/s rd, 1.4 MiB/s wr, 197 op/s 2024-06-02T23:21:48.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 188592 1.82 MB/sec execute 417 sec latency 11.487 ms 2024-06-02T23:21:49.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 189136 1.82 MB/sec execute 418 sec latency 11.330 ms 2024-06-02T23:21:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:49 smithi174 ceph-mon[35994]: pgmap v729: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-02T23:21:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:49 smithi052 ceph-mon[32340]: pgmap v729: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-02T23:21:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:49 smithi123 ceph-mon[35977]: pgmap v729: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-02T23:21:50.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 189713 1.82 MB/sec execute 419 sec latency 43.783 ms 2024-06-02T23:21:51.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 190361 1.82 MB/sec execute 420 sec latency 16.301 ms 2024-06-02T23:21:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/361607114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 msgr2=0x7f3a08101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/361607114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08101e50 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009900 tx=0x7f39f802a410 comp rx=0 tx=0).stop 2024-06-02T23:21:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/361607114 shutdown_connections 2024-06-02T23:21:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/361607114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a08102c20 0x7f3a081030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/361607114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a081035e0 0x7f3a08109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.871 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/361607114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08101e50 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/361607114 >> 172.21.15.52:0/361607114 conn(0x7f3a080fcfc0 msgr2=0x7f3a080ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/361607114 shutdown_connections 2024-06-02T23:21:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.870+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/361607114 wait complete. 2024-06-02T23:21:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 Processor -- start 2024-06-02T23:21:51.872 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 -- start start 2024-06-02T23:21:51.873 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a08102c20 0x7f3a08073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a081035e0 0x7f3a080742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0810d580 con 0x7f3a08101a30 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0810d400 con 0x7f3a08102c20 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.871+0000 7f3a0e3ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a0810d700 con 0x7f3a081035e0 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a08102c20 0x7f3a08073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08075700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36574/0 (socket says 172.21.15.52:36574) 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 -- 172.21.15.52:0/2637954429 learned_addr learned my addr 172.21.15.52:0/2637954429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a081035e0 0x7f3a080742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 -- 172.21.15.52:0/2637954429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a081035e0 msgr2=0x7f3a080742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.874 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a081035e0 0x7f3a080742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 -- 172.21.15.52:0/2637954429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a08102c20 msgr2=0x7f3a08073d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a08102c20 0x7f3a08073d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 -- 172.21.15.52:0/2637954429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a08074b90 con 0x7f3a08101a30 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08075700 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f39f80314e0 tx=0x7f39f8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a057fa640 1 -- 172.21.15.52:0/2637954429 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f803f070 con 0x7f3a08101a30 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a057fa640 1 -- 172.21.15.52:0/2637954429 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39f8031c40 con 0x7f3a08101a30 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a081a1c90 con 0x7f3a08101a30 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a057fa640 1 -- 172.21.15.52:0/2637954429 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8035030 con 0x7f3a08101a30 2024-06-02T23:21:51.875 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.872+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a081a2150 con 0x7f3a08101a30 2024-06-02T23:21:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a057fa640 1 -- 172.21.15.52:0/2637954429 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f39f8042070 con 0x7f3a08101a30 2024-06-02T23:21:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 Processor -- start 2024-06-02T23:21:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 -- start start 2024-06-02T23:21:51.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 0x559a78f4d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4da00 0x559a78f52500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f52b70 0x559a78f55000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559a78eb04f0 con 0x559a78f4d010 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a78e4e870 con 0x559a78f52b70 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a111d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a78eac4b0 con 0x559a78f4da00 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a0c98b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 0x559a78f4d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a0c98b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 0x559a78f4d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36576/0 (socket says 172.21.15.52:36576) 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.875+0000 7f3a0c98b640 1 -- 172.21.15.52:0/3625914368 learned_addr learned my addr 172.21.15.52:0/3625914368 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f52b70 0x559a78f55000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a07fff640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4da00 0x559a78f52500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 -- 172.21.15.52:0/3625914368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4da00 msgr2=0x559a78f52500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4da00 0x559a78f52500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 -- 172.21.15.52:0/3625914368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 msgr2=0x559a78f4d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 0x559a78f4d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 -- 172.21.15.52:0/3625914368 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a78e6a9b0 con 0x559a78f52b70 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a07fff640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4da00 0x559a78f52500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 0x559a78f4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a057fa640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f39f00a2c30 0x7f39f00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f39f00a2c30 0x7f39f00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a077fe640 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f52b70 0x559a78f55000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009bf0 tx=0x7f39fc011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e59e06ed3d3d052c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f3a057fa640 1 -- 172.21.15.52:0/2637954429 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f39f80ef780 con 0x7f3a08101a30 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f39ee7fc640 1 -- 172.21.15.52:0/3625914368 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc01e070 con 0x559a78f52b70 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f39ee7fc640 1 -- 172.21.15.52:0/3625914368 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39fc009da0 con 0x559a78f52b70 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.876+0000 7f39ee7fc640 1 -- 172.21.15.52:0/3625914368 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0187d0 con 0x559a78f52b70 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 -- 172.21.15.52:0/3625914368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f52b70 msgr2=0x559a78f55000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f39f00a2c30 0x7f39f00a50f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x559a78f4e260 tx=0x7f3a000096a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:51.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f52b70 0x559a78f55000 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f39fc009bf0 tx=0x7f39fc011810 comp rx=0 tx=0).stop 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 -- 172.21.15.52:0/3625914368 shutdown_connections 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4da00 0x559a78f52500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f52b70 0x559a78f55000 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 --2- 172.21.15.52:0/3625914368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4d010 0x559a78f4d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 -- 172.21.15.52:0/3625914368 >> 172.21.15.52:0/3625914368 conn(0x559a78f48680 msgr2=0x559a78f48a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 -- 172.21.15.52:0/3625914368 shutdown_connections 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 -- 172.21.15.52:0/3625914368 wait complete. 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.877+0000 7f3a111d9740 1 -- start start 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 Processor -- start 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f57220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f57760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f5e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559a78f55b20 con 0x559a78f52b70 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559a78e39ae0 con 0x559a78f4da00 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559a78f55ca0 con 0x559a78f4d010 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a111d9740 10 client.0 _ll_register_callbacks cb 0x559a78e729b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f57760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a0c98b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f5e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f57760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:52374/0 (socket says 172.21.15.52:52374) 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 -- 172.21.15.52:0/2661586295 learned_addr learned my addr 172.21.15.52:0/2661586295 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f57220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 -- 172.21.15.52:0/2661586295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 msgr2=0x559a78f57220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f57220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 -- 172.21.15.52:0/2661586295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 msgr2=0x559a78f5e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f5e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a077fe640 1 -- 172.21.15.52:0/2661586295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a78bc4210 con 0x559a78f4da00 2024-06-02T23:21:51.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.878+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f5e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f3a077fe640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f57760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005d70 tx=0x7f39fc005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1755cbd8a61d5151 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f39e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f39e77fe640 1 -- 172.21.15.52:0/2661586295 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc01e040 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39fc009da0 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f39e77fe640 1 -- 172.21.15.52:0/2661586295 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f39fc0050c0 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f39e77fe640 1 -- 172.21.15.52:0/2661586295 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f39fc0052c0 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f3a111d9740 10 client.27758 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a78c90620 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f39e77fe640 1 -- 172.21.15.52:0/2661586295 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39fc009da0 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f39e77fe640 1 -- 172.21.15.52:0/2661586295 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f39fc021030 con 0x559a78f4da00 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f3a111d9740 10 client.27758 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.879+0000 7f3a111d9740 10 client.27758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f3a111d9740 20 client.27758 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:51.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78f62ec0 0x559a78f652b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f3a111d9740 4 client.27758 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a78ec3f10 con 0x559a78f62ec0 2024-06-02T23:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78f62ec0 0x559a78f652b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f39e77fe640 10 client.27758 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:51.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.880+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78f62ec0 0x559a78f652b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e95fe89689c20086 server_cookie=996a135aa0939237 in_seq=0 out_seq=0 2024-06-02T23:21:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f39e77fe640 1 -- 172.21.15.52:0/2661586295 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559a78ec3f10 con 0x559a78f62ec0 2024-06-02T23:21:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f39e77fe640 10 client.27758 handle_command_reply: tid=0 2024-06-02T23:21:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 client.27758 shutdown 2024-06-02T23:21:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 msgr2=0x559a78f57760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f57760 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f39fc005d70 tx=0x7f39fc005da0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 shutdown_connections 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78f62ec0 0x559a78f652b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f5e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f57760 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2661586295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f57220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f39e77fe640 0 client.27758 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.881+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 >> 172.21.15.52:0/2661586295 conn(0x559a78f48680 msgr2=0x559a78f477e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.882+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 shutdown_connections 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.882+0000 7f3a111d9740 1 -- 172.21.15.52:0/2661586295 wait complete. 2024-06-02T23:21:51.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.882+0000 7f3a111d9740 20 client.27758 trim_cache size 0 max 16384 2024-06-02T23:21:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:51 smithi174 ceph-mon[35994]: pgmap v730: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-02T23:21:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:51 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:21:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:51 smithi052 ceph-mon[32340]: pgmap v730: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-02T23:21:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:51 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 Processor -- start 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 -- start start 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f4b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f5e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f4b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081aaea0 con 0x559a78f52b70 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081ac9e0 con 0x559a78f4da00 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a111d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081ab570 con 0x559a78f4d010 2024-06-02T23:21:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f4b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f4b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36588/0 (socket says 172.21.15.52:36588) 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 -- 172.21.15.52:0/2985371455 learned_addr learned my addr 172.21.15.52:0/2985371455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a077fe640 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f5e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a07fff640 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f4b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 -- 172.21.15.52:0/2985371455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 msgr2=0x559a78f4b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f4b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 -- 172.21.15.52:0/2985371455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 msgr2=0x559a78f5e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f5e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.900+0000 7f3a0c98b640 1 -- 172.21.15.52:0/2985371455 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a78f484b0 con 0x559a78f52b70 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.901+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f4b8a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3a0001e5e0 tx=0x7f3a00019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d58c88eca4c3aa0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.901+0000 7f39edffb640 1 -- 172.21.15.52:0/2985371455 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004b20 con 0x559a78f52b70 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.901+0000 7f39edffb640 1 -- 172.21.15.52:0/2985371455 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a00004cc0 con 0x559a78f52b70 2024-06-02T23:21:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.901+0000 7f39edffb640 1 -- 172.21.15.52:0/2985371455 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00005420 con 0x559a78f52b70 2024-06-02T23:21:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 -- 172.21.15.52:0/2985371455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 msgr2=0x559a78f4b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f4b8a0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f3a0001e5e0 tx=0x7f3a00019000 comp rx=0 tx=0).stop 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 -- 172.21.15.52:0/2985371455 shutdown_connections 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f52b70 0x559a78f4b8a0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4da00 0x559a78f5e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 --2- 172.21.15.52:0/2985371455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f4d010 0x559a78f4b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 -- 172.21.15.52:0/2985371455 >> 172.21.15.52:0/2985371455 conn(0x559a78f48680 msgr2=0x559a78f4ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 -- 172.21.15.52:0/2985371455 shutdown_connections 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 -- 172.21.15.52:0/2985371455 wait complete. 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 -- start start 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.902+0000 7f3a111d9740 1 Processor -- start 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 0x559a78f5b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 0x559a78f5bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f52b70 0x559a78f5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081ac310 con 0x559a78f4da00 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081abc40 con 0x559a78f4d010 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a081a8e80 con 0x559a78f52b70 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 10 client.0 _ll_register_callbacks cb 0x7f39fc01c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 0x559a78f5bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a07fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 0x559a78f5b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 0x559a78f5bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36592/0 (socket says 172.21.15.52:36592) 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a07fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 0x559a78f5b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:52398/0 (socket says 172.21.15.52:52398) 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 -- 172.21.15.52:0/466053763 learned_addr learned my addr 172.21.15.52:0/466053763 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f52b70 0x559a78f5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 -- 172.21.15.52:0/466053763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f52b70 msgr2=0x559a78f5cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f52b70 0x559a78f5cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 -- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 msgr2=0x559a78f5b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 0x559a78f5b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 -- 172.21.15.52:0/466053763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a78f4e6d0 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a0c98b640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f52b70 0x559a78f5cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a07fff640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 0x559a78f5b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a077fe640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 0x559a78f5bde0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0196e0 tx=0x7f39fc019970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2fc2dbd687b5d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f39e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f39e77fe640 1 -- 172.21.15.52:0/466053763 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc01e040 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f39e77fe640 1 -- 172.21.15.52:0/466053763 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f39fc003a00 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.903+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a00004cc0 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f39e77fe640 1 -- 172.21.15.52:0/466053763 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f39fc009da0 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 10 client.18306 fetch_fsmap learned FSMap version 32 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559a78f43fe0 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f39e77fe640 1 -- 172.21.15.52:0/466053763 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a00004cc0 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f39e77fe640 1 -- 172.21.15.52:0/466053763 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f39fc021040 con 0x559a78f4da00 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 10 client.18306 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:21:51.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 10 client.18306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 20 client.18306 populate_metadata read hostname 'smithi052' 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78fcdb50 0x559a78fcff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 4 client.18306 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559a78ebdc20 con 0x559a78fcdb50 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.904+0000 7f3a07fff640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78fcdb50 0x559a78fcff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.905+0000 7f39e77fe640 10 client.18306 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.905+0000 7f3a07fff640 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78fcdb50 0x559a78fcff40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75f6a65bd06eebd7 server_cookie=8c671c6fd95de3f0 in_seq=0 out_seq=0 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.927+0000 7f39e77fe640 1 -- 172.21.15.52:0/466053763 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559a78ebdc20 con 0x559a78fcdb50 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.927+0000 7f39e77fe640 10 client.18306 handle_command_reply: tid=0 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.927+0000 7f3a111d9740 1 client.18306 shutdown 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.927+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 msgr2=0x559a78f5bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.927+0000 7f3a111d9740 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 0x559a78f5bde0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f39fc0196e0 tx=0x7f39fc019970 comp rx=0 tx=0).stop 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.927+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 shutdown_connections 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559a78fcdb50 0x559a78fcff40 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559a78f52b70 0x559a78f5cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559a78f4da00 0x559a78f5bde0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f39e77fe640 0 client.18306 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 --2- 172.21.15.52:0/466053763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559a78f4d010 0x559a78f5b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 >> 172.21.15.52:0/466053763 conn(0x559a78f48680 msgr2=0x559a78f570f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 shutdown_connections 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 1 -- 172.21.15.52:0/466053763 wait complete. 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a111d9740 20 client.18306 trim_cache size 0 max 16384 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:21:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:21:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f39f00a2c30 msgr2=0x7f39f00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f39f00a2c30 0x7f39f00a50f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x559a78f4e260 tx=0x7f3a000096a0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 msgr2=0x7f3a08075700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:21:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08075700 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f39f80314e0 tx=0x7f39f8003c10 comp rx=0 tx=0).stop 2024-06-02T23:21:51.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 shutdown_connections 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f39f00a2c30 0x7f39f00a50f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a081035e0 0x7f3a080742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a08102c20 0x7f3a08073d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 --2- 172.21.15.52:0/2637954429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a08101a30 0x7f3a08075700 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 >> 172.21.15.52:0/2637954429 conn(0x7f3a080fcfc0 msgr2=0x7f3a08106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 shutdown_connections 2024-06-02T23:21:51.931 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:21:51.928+0000 7f3a0e3ee640 1 -- 172.21.15.52:0/2637954429 wait complete. 2024-06-02T23:21:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:51 smithi123 ceph-mon[35977]: pgmap v730: 97 pgs: 97 active+clean; 83 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 282 KiB/s rd, 1.4 MiB/s wr, 168 op/s 2024-06-02T23:21:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:51 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:21:52.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 190971 1.83 MB/sec execute 421 sec latency 17.422 ms 2024-06-02T23:21:53.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:52 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:52 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:52 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:21:53.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:52 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:21:53.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:52 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:52 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:52 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:52 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:21:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:52 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:21:53.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:52 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:52 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:52 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:52 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:21:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:52 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:21:53.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:52 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:21:53.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 191528 1.84 MB/sec execute 422 sec latency 13.796 ms 2024-06-02T23:21:54.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:53 smithi123 ceph-mon[35977]: pgmap v731: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:21:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:53 smithi174 ceph-mon[35994]: pgmap v731: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:21:54.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:53 smithi052 ceph-mon[32340]: pgmap v731: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.3 MiB/s wr, 159 op/s 2024-06-02T23:21:54.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 191918 1.84 MB/sec execute 423 sec latency 12.661 ms 2024-06-02T23:21:55.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 192235 1.83 MB/sec execute 424 sec latency 12.980 ms 2024-06-02T23:21:56.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:55 smithi123 ceph-mon[35977]: pgmap v732: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-02T23:21:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:55 smithi174 ceph-mon[35994]: pgmap v732: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-02T23:21:56.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:55 smithi052 ceph-mon[32340]: pgmap v732: 97 pgs: 97 active+clean; 84 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 1.2 MiB/s wr, 145 op/s 2024-06-02T23:21:56.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 192793 1.83 MB/sec execute 425 sec latency 11.671 ms 2024-06-02T23:21:57.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 193428 1.84 MB/sec execute 426 sec latency 15.082 ms 2024-06-02T23:21:58.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:57 smithi123 ceph-mon[35977]: pgmap v733: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-06-02T23:21:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:57 smithi174 ceph-mon[35994]: pgmap v733: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-06-02T23:21:58.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:57 smithi052 ceph-mon[32340]: pgmap v733: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 315 KiB/s rd, 1.6 MiB/s wr, 191 op/s 2024-06-02T23:21:58.641 INFO:tasks.workunit.client.0.smithi052.stdout: 1 194019 1.84 MB/sec execute 427 sec latency 16.854 ms 2024-06-02T23:21:59.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 194834 1.85 MB/sec execute 428 sec latency 14.591 ms 2024-06-02T23:22:00.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:21:59 smithi123 ceph-mon[35977]: pgmap v734: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:22:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:21:59 smithi174 ceph-mon[35994]: pgmap v734: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:22:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:21:59 smithi052 ceph-mon[32340]: pgmap v734: 97 pgs: 97 active+clean; 85 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:22:00.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 195226 1.85 MB/sec execute 429 sec latency 16.858 ms 2024-06-02T23:22:01.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 195527 1.85 MB/sec execute 430 sec latency 12.315 ms 2024-06-02T23:22:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:01 smithi123 ceph-mon[35977]: pgmap v735: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-06-02T23:22:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:01 smithi174 ceph-mon[35994]: pgmap v735: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-06-02T23:22:02.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:01 smithi052 ceph-mon[32340]: pgmap v735: 97 pgs: 97 active+clean; 86 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 1.5 MiB/s wr, 152 op/s 2024-06-02T23:22:02.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 195923 1.85 MB/sec execute 431 sec latency 17.091 ms 2024-06-02T23:22:03.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 196506 1.85 MB/sec execute 432 sec latency 16.148 ms 2024-06-02T23:22:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:03 smithi123 ceph-mon[35977]: pgmap v736: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-02T23:22:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:03 smithi174 ceph-mon[35994]: pgmap v736: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-02T23:22:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:03 smithi052 ceph-mon[32340]: pgmap v736: 97 pgs: 97 active+clean; 85 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.4 MiB/s wr, 175 op/s 2024-06-02T23:22:04.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 197100 1.85 MB/sec execute 433 sec latency 16.587 ms 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 -- 172.21.15.52:0/968195230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 msgr2=0x7f1f64108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/968195230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f64108d40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009be0 tx=0x7f1f5c02a6c0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 -- 172.21.15.52:0/968195230 shutdown_connections 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/968195230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f64101af0 0x7f1f64101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/968195230 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f64108d40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/968195230 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64100900 0x7f1f64100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 -- 172.21.15.52:0/968195230 >> 172.21.15.52:0/968195230 conn(0x7f1f640fbe70 msgr2=0x7f1f640fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.024+0000 7f1f68c9a640 1 -- 172.21.15.52:0/968195230 shutdown_connections 2024-06-02T23:22:05.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.025+0000 7f1f68c9a640 1 -- 172.21.15.52:0/968195230 wait complete. 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.025+0000 7f1f68c9a640 1 Processor -- start 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.025+0000 7f1f68c9a640 1 -- start start 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.025+0000 7f1f68c9a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f64100900 0x7f1f64069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f68c9a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 0x7f1f6419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f68c9a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f6419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f68c9a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6410c430 con 0x7f1f64101af0 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f68c9a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6410c2b0 con 0x7f1f641024b0 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f68c9a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f6410c5b0 con 0x7f1f64100900 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 0x7f1f6419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 0x7f1f6419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55326/0 (socket says 172.21.15.52:55326) 2024-06-02T23:22:05.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f62d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f6419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 -- 172.21.15.52:0/2005358227 learned_addr learned my addr 172.21.15.52:0/2005358227 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f6259c640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f64100900 0x7f1f64069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 -- 172.21.15.52:0/2005358227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f64100900 msgr2=0x7f1f64069be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f64100900 0x7f1f64069be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 -- 172.21.15.52:0/2005358227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 msgr2=0x7f1f6419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f6419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 -- 172.21.15.52:0/2005358227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f6419f770 con 0x7f1f64101af0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f6419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 0x7f1f6419c870 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c970 tx=0x7f1f5800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f537fe640 1 -- 172.21.15.52:0/2005358227 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f58007bb0 con 0x7f1f64101af0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f6419fa60 con 0x7f1f64101af0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f537fe640 1 -- 172.21.15.52:0/2005358227 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f58007d50 con 0x7f1f64101af0 2024-06-02T23:22:05.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.026+0000 7f1f537fe640 1 -- 172.21.15.52:0/2005358227 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5800f450 con 0x7f1f64101af0 2024-06-02T23:22:05.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.027+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f6419ffb0 con 0x7f1f64101af0 2024-06-02T23:22:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.029+0000 7f1f537fe640 1 -- 172.21.15.52:0/2005358227 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f58004f20 con 0x7f1f64101af0 2024-06-02T23:22:05.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.029+0000 7f1f6b9a2740 1 Processor -- start 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 -- start start 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf7b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf80910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf83410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a62beda4f0 con 0x55a62bf7be10 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a62be78870 con 0x55a62bf80f80 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a62bed64b0 con 0x55a62bf7b420 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf80910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf83410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf83410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42452/0 (socket says 172.21.15.52:42452) 2024-06-02T23:22:05.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 -- 172.21.15.52:0/825453076 learned_addr learned my addr 172.21.15.52:0/825453076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf80910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55330/0 (socket says 172.21.15.52:55330) 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf7b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f537fe640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f4c0a2c30 0x7f1f4c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 -- 172.21.15.52:0/825453076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 msgr2=0x55a62bf7b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf7b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 -- 172.21.15.52:0/825453076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 msgr2=0x55a62bf80910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf80910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f61d9b640 1 -- 172.21.15.52:0/825453076 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62be949b0 con 0x55a62bf80f80 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.030+0000 7f1f6259c640 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f4c0a2c30 0x7f1f4c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f537fe640 1 -- 172.21.15.52:0/2005358227 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1f580cecf0 con 0x7f1f64101af0 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf83410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f580839c0 tx=0x7f1f580662b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50bd6dc7c5f06d3b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:05.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f4c0a2c30 0x7f1f4c0a50f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006010 tx=0x7f1f5c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f50ff9640 1 -- 172.21.15.52:0/825453076 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801d070 con 0x55a62bf80f80 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f50ff9640 1 -- 172.21.15.52:0/825453076 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f5807fe60 con 0x55a62bf80f80 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.031+0000 7f1f50ff9640 1 -- 172.21.15.52:0/825453076 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5809b170 con 0x55a62bf80f80 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/825453076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 msgr2=0x55a62bf83410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf83410 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1f580839c0 tx=0x7f1f580662b0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/825453076 shutdown_connections 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf83410 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf80910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/825453076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf7b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/825453076 >> 172.21.15.52:0/825453076 conn(0x55a62bf72680 msgr2=0x55a62bf72a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/825453076 shutdown_connections 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/825453076 wait complete. 2024-06-02T23:22:05.038 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 -- start start 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.032+0000 7f1f6b9a2740 1 Processor -- start 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf8cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a62bf83e70 con 0x55a62bf7be10 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a62be63ae0 con 0x55a62bf80f80 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a62bf83ff0 con 0x55a62bf7b420 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6b9a2740 10 client.0 _ll_register_callbacks cb 0x55a62be9c9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55336/0 (socket says 172.21.15.52:55336) 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 -- 172.21.15.52:0/1915143157 learned_addr learned my addr 172.21.15.52:0/1915143157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf8cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.039 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f6259c640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 -- 172.21.15.52:0/1915143157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 msgr2=0x55a62bf85610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 -- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 msgr2=0x55a62bf8cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf8cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 -- 172.21.15.52:0/1915143157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62bbee210 con 0x55a62bf7be10 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1f580cc870 tx=0x7f1f58007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe58cb52f5bef56 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f49ffb640 1 -- 172.21.15.52:0/1915143157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801d040 con 0x55a62bf7be10 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.033+0000 7f1f49ffb640 1 -- 172.21.15.52:0/1915143157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1f580996d0 con 0x55a62bf7be10 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a62beedf10 con 0x55a62bf7be10 2024-06-02T23:22:05.040 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f49ffb640 1 -- 172.21.15.52:0/1915143157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f5809af90 con 0x55a62bf7be10 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 10 client.18318 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a62bcba620 con 0x55a62bf7be10 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f49ffb640 1 -- 172.21.15.52:0/1915143157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55a62beedf10 con 0x55a62bf7be10 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f49ffb640 1 -- 172.21.15.52:0/1915143157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1f580ccce0 con 0x55a62bf7be10 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 10 client.18318 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 10 client.18318 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 20 client.18318 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bf91010 0x55a62bf93400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.041 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 4 client.18318 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a62bee7c20 con 0x55a62bf91010 2024-06-02T23:22:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.034+0000 7f1f6259c640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bf91010 0x55a62bf93400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.035+0000 7f1f6259c640 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bf91010 0x55a62bf93400 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae0e05437cc2117 server_cookie=9b2c063d61edb0fd in_seq=0 out_seq=0 2024-06-02T23:22:05.042 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.035+0000 7f1f49ffb640 10 client.18318 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f49ffb640 1 -- 172.21.15.52:0/1915143157 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a62bee7c20 con 0x55a62bf91010 2024-06-02T23:22:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f49ffb640 10 client.18318 handle_command_reply: tid=0 2024-06-02T23:22:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 client.18318 shutdown 2024-06-02T23:22:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 msgr2=0x55a62bf85b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1f580cc870 tx=0x7f1f58007730 comp rx=0 tx=0).stop 2024-06-02T23:22:05.043 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 shutdown_connections 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bf91010 0x55a62bf93400 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bf8cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1915143157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f49ffb640 0 client.18318 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 >> 172.21.15.52:0/1915143157 conn(0x55a62bf72680 msgr2=0x55a62bf717e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 shutdown_connections 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1915143157 wait complete. 2024-06-02T23:22:05.044 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.036+0000 7f1f6b9a2740 20 client.18318 trim_cache size 0 max 16384 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 Processor -- start 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 -- start start 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 0x55a62bf8cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a6aa0 con 0x55a62bf80f80 2024-06-02T23:22:05.056 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a85e0 con 0x55a62bf7b420 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a7170 con 0x55a62bf7be10 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 0x55a62bf8cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 0x55a62bf8cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55352/0 (socket says 172.21.15.52:55352) 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 -- 172.21.15.52:0/1467563405 learned_addr learned my addr 172.21.15.52:0/1467563405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f6259c640 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 -- 172.21.15.52:0/1467563405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7be10 msgr2=0x55a62bf85b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 -- 172.21.15.52:0/1467563405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf7b420 msgr2=0x55a62bf85610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 -- 172.21.15.52:0/1467563405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a62bcba620 con 0x55a62bf80f80 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.055+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 0x55a62bf8cae0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009be0 tx=0x7f1f5c040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69fbfa1a9406569 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:05.057 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f4bfff640 1 -- 172.21.15.52:0/1467563405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c036040 con 0x55a62bf80f80 2024-06-02T23:22:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f4bfff640 1 -- 172.21.15.52:0/1467563405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f5c050030 con 0x55a62bf80f80 2024-06-02T23:22:05.058 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f4bfff640 1 -- 172.21.15.52:0/1467563405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c04c6a0 con 0x55a62bf80f80 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1467563405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 msgr2=0x55a62bf8cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 0x55a62bf8cae0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009be0 tx=0x7f1f5c040000 comp rx=0 tx=0).stop 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1467563405 shutdown_connections 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7be10 0x55a62bf85b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf80f80 0x55a62bf8cae0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/1467563405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf7b420 0x55a62bf85610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1467563405 >> 172.21.15.52:0/1467563405 conn(0x55a62bf72680 msgr2=0x55a62bf72a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1467563405 shutdown_connections 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.056+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/1467563405 wait complete. 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 -- start start 2024-06-02T23:22:05.061 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 Processor -- start 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf89d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bff9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a7f10 con 0x55a62bf7be10 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a7840 con 0x55a62bf80f80 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f641a4a80 con 0x55a62bf7b420 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6b9a2740 10 client.0 _ll_register_callbacks cb 0x7f1f54009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf89d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f61d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf89d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55364/0 (socket says 172.21.15.52:55364) 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f61d9b640 1 -- 172.21.15.52:0/4124482473 learned_addr learned my addr 172.21.15.52:0/4124482473 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f62d9d640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bff9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.062 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.057+0000 7f1f6259c640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f61d9b640 1 -- 172.21.15.52:0/4124482473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 msgr2=0x55a62bf897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f61d9b640 1 -- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 msgr2=0x55a62bff9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bff9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f61d9b640 1 -- 172.21.15.52:0/4124482473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a62bf8cdf0 con 0x55a62bf7be10 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f61d9b640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf89d20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1f580058c0 tx=0x7f1f58097810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a068d63a03f04c7f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f49ffb640 1 -- 172.21.15.52:0/4124482473 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f5801d070 con 0x55a62bf7be10 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f5c050030 con 0x55a62bf7be10 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f49ffb640 1 -- 172.21.15.52:0/4124482473 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1f58099930 con 0x55a62bf7be10 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f49ffb640 1 -- 172.21.15.52:0/4124482473 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f5809b170 con 0x55a62bf7be10 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 10 client.18324 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:05.063 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a62bf8cdf0 con 0x55a62bf7be10 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f49ffb640 1 -- 172.21.15.52:0/4124482473 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f5c050030 con 0x55a62bf7be10 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f49ffb640 1 -- 172.21.15.52:0/4124482473 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1f580ccce0 con 0x55a62bf7be10 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 10 client.18324 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 10 client.18324 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 20 client.18324 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bffb5a0 0x55a62bffd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 4 client.18324 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.058+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a62bee1f70 con 0x55a62bffb5a0 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.059+0000 7f1f6259c640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bffb5a0 0x55a62bffd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.059+0000 7f1f6259c640 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bffb5a0 0x55a62bffd9c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23f6b210226e50d0 server_cookie=ea3ba83ebc5a5ff7 in_seq=0 out_seq=0 2024-06-02T23:22:05.064 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.059+0000 7f1f49ffb640 10 client.18324 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f49ffb640 1 -- 172.21.15.52:0/4124482473 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a62bee1f70 con 0x55a62bffb5a0 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f49ffb640 10 client.18324 handle_command_reply: tid=0 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 client.18324 shutdown 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 msgr2=0x55a62bf89d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf89d20 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f1f580058c0 tx=0x7f1f58097810 comp rx=0 tx=0).stop 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 shutdown_connections 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a62bffb5a0 0x55a62bffd9c0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.082 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a62bf80f80 0x55a62bff9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a62bf7be10 0x55a62bf89d20 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f49ffb640 0 client.18324 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 --2- 172.21.15.52:0/4124482473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a62bf7b420 0x55a62bf897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 >> 172.21.15.52:0/4124482473 conn(0x55a62bf72680 msgr2=0x55a62bf7d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 shutdown_connections 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 1 -- 172.21.15.52:0/4124482473 wait complete. 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.081+0000 7f1f6b9a2740 20 client.18324 trim_cache size 0 max 16384 2024-06-02T23:22:05.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:22:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:22:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:22:05.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f4c0a2c30 msgr2=0x7f1f4c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f4c0a2c30 0x7f1f4c0a50f0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c006010 tx=0x7f1f5c009500 comp rx=0 tx=0).stop 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 msgr2=0x7f1f6419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 0x7f1f6419c870 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f1f5800c970 tx=0x7f1f5800ce30 comp rx=0 tx=0).stop 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 shutdown_connections 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f4c0a2c30 0x7f1f4c0a50f0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f641024b0 0x7f1f6419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f64101af0 0x7f1f6419c870 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 --2- 172.21.15.52:0/2005358227 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f64100900 0x7f1f64069be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 >> 172.21.15.52:0/2005358227 conn(0x7f1f640fbe70 msgr2=0x7f1f64104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:05.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 shutdown_connections 2024-06-02T23:22:05.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:05.082+0000 7f1f68c9a640 1 -- 172.21.15.52:0/2005358227 wait complete. 2024-06-02T23:22:05.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 197601 1.85 MB/sec execute 434 sec latency 22.208 ms 2024-06-02T23:22:06.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:05 smithi123 ceph-mon[35977]: pgmap v737: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:22:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:05 smithi174 ceph-mon[35994]: pgmap v737: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:22:06.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:05 smithi052 ceph-mon[32340]: pgmap v737: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 146 op/s 2024-06-02T23:22:06.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 198384 1.86 MB/sec execute 435 sec latency 15.770 ms 2024-06-02T23:22:07.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 198790 1.87 MB/sec execute 436 sec latency 15.278 ms 2024-06-02T23:22:08.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:07 smithi123 ceph-mon[35977]: pgmap v738: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-06-02T23:22:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:07 smithi174 ceph-mon[35994]: pgmap v738: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-06-02T23:22:08.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:07 smithi052 ceph-mon[32340]: pgmap v738: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 1.6 MiB/s wr, 192 op/s 2024-06-02T23:22:08.359 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:22:08.622 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:08.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 199090 1.87 MB/sec execute 437 sec latency 11.993 ms 2024-06-02T23:22:09.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 199419 1.86 MB/sec execute 438 sec latency 70.526 ms 2024-06-02T23:22:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:09 smithi052 ceph-mon[32340]: pgmap v739: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:22:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:09 smithi123 ceph-mon[35977]: pgmap v739: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:22:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:09 smithi174 ceph-mon[35994]: pgmap v739: 97 pgs: 97 active+clean; 86 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:22:10.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 199890 1.86 MB/sec execute 439 sec latency 83.373 ms 2024-06-02T23:22:11.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 200417 1.87 MB/sec execute 440 sec latency 60.282 ms 2024-06-02T23:22:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:11 smithi052 ceph-mon[32340]: pgmap v740: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:22:12.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:11 smithi123 ceph-mon[35977]: pgmap v740: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:22:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:11 smithi174 ceph-mon[35994]: pgmap v740: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.4 MiB/s wr, 155 op/s 2024-06-02T23:22:12.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 201030 1.86 MB/sec execute 441 sec latency 15.180 ms 2024-06-02T23:22:12.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 -- 172.21.15.52:0/1256494971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4107870 msgr2=0x7f40e410f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1256494971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4107870 0x7f40e410f3f0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b0d0 tx=0x7f40dc00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 -- 172.21.15.52:0/1256494971 shutdown_connections 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1256494971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4107870 0x7f40e410f3f0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1256494971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1256494971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e4103150 0x7f40e4107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 -- 172.21.15.52:0/1256494971 >> 172.21.15.52:0/1256494971 conn(0x7f40e4079570 msgr2=0x7f40e4079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 -- 172.21.15.52:0/1256494971 shutdown_connections 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 -- 172.21.15.52:0/1256494971 wait complete. 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 Processor -- start 2024-06-02T23:22:12.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.913+0000 7f40ebbee700 1 -- start start 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ebbee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ebbee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4103150 0x7f40e4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ebbee700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e4107870 0x7f40e4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ebbee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40e4112660 con 0x7f40e4103150 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ebbee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41124e0 con 0x7f40e41027a0 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ebbee700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41127e0 con 0x7f40e4107870 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e9189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4103150 0x7f40e4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42496/0 (socket says 172.21.15.52:42496) 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 -- 172.21.15.52:0/1906285344 learned_addr learned my addr 172.21.15.52:0/1906285344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40ea18b700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e4107870 0x7f40e4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 -- 172.21.15.52:0/1906285344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e4107870 msgr2=0x7f40e4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e4107870 0x7f40e4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 -- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4103150 msgr2=0x7f40e4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4103150 0x7f40e4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 -- 172.21.15.52:0/1906285344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40e41a6f90 con 0x7f40e41027a0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.914+0000 7f40e998a700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4074790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e9a0 tx=0x7f40e000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.915+0000 7f40daffd700 1 -- 172.21.15.52:0/1906285344 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e000cbd0 con 0x7f40e41027a0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.915+0000 7f40daffd700 1 -- 172.21.15.52:0/1906285344 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e000cd70 con 0x7f40e41027a0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.915+0000 7f40daffd700 1 -- 172.21.15.52:0/1906285344 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0018680 con 0x7f40e41027a0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.915+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40e41a7280 con 0x7f40e41027a0 2024-06-02T23:22:12.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.915+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40e41a7770 con 0x7f40e41027a0 2024-06-02T23:22:12.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.917+0000 7f40daffd700 1 -- 172.21.15.52:0/1906285344 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40e0010430 con 0x7f40e41027a0 2024-06-02T23:22:12.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.918+0000 7f40daffd700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d00a2c20 0x7f40d00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.918+0000 7f40e9189700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d00a2c20 0x7f40d00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.918+0000 7f40daffd700 1 -- 172.21.15.52:0/1906285344 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40e00cf280 con 0x7f40e41027a0 2024-06-02T23:22:12.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40e9189700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d00a2c20 0x7f40d00a50e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f40e4108c90 tx=0x7f40d400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:12.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 Processor -- start 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 -- start start 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bef60 0x5619a12bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 0x5619a12b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12b97a0 0x5619a12b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5619a11abf60 con 0x5619a12bf880 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5619a120a850 con 0x5619a12b97a0 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40f6471b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5619a1172460 con 0x5619a12bef60 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40e9189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 0x5619a12b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40e9189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 0x5619a12b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55404/0 (socket says 172.21.15.52:55404) 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40e998a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bef60 0x5619a12bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.919+0000 7f40e9189700 1 -- 172.21.15.52:0/4287130675 learned_addr learned my addr 172.21.15.52:0/4287130675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40ea18b700 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12b97a0 0x5619a12b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40e9189700 1 -- 172.21.15.52:0/4287130675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bef60 msgr2=0x5619a12bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40e9189700 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bef60 0x5619a12bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40e9189700 1 -- 172.21.15.52:0/4287130675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12b97a0 msgr2=0x5619a12b9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40e9189700 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12b97a0 0x5619a12b9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40e9189700 1 -- 172.21.15.52:0/4287130675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a11598d0 con 0x5619a12bf880 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40e9189700 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 0x5619a12b9260 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f40d4005e00 tx=0x7f40d4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46fe0c0485bed5c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40cffff700 1 -- 172.21.15.52:0/4287130675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4016070 con 0x5619a12bf880 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40cffff700 1 -- 172.21.15.52:0/4287130675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40d4010c60 con 0x5619a12bf880 2024-06-02T23:22:12.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.920+0000 7f40cffff700 1 -- 172.21.15.52:0/4287130675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4025d90 con 0x5619a12bf880 2024-06-02T23:22:12.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 -- 172.21.15.52:0/4287130675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 msgr2=0x5619a12b9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.925 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 0x5619a12b9260 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f40d4005e00 tx=0x7f40d4012000 comp rx=0 tx=0).stop 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 -- 172.21.15.52:0/4287130675 shutdown_connections 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12b97a0 0x5619a12b9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bef60 0x5619a12bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 --2- 172.21.15.52:0/4287130675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bf880 0x5619a12b9260 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 -- 172.21.15.52:0/4287130675 >> 172.21.15.52:0/4287130675 conn(0x5619a11e6080 msgr2=0x5619a12b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 -- 172.21.15.52:0/4287130675 shutdown_connections 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 -- 172.21.15.52:0/4287130675 wait complete. 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.921+0000 7f40f6471b80 1 -- start start 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 Processor -- start 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 0x5619a12c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bef60 0x5619a12c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bf880 0x5619a12c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5619a11aafc0 con 0x5619a12b97a0 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5619a1206810 con 0x5619a12bef60 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5619a12c6260 con 0x5619a12bf880 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40f6471b80 10 client.0 _ll_register_callbacks cb 0x5619a11a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 0x5619a12c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 0x5619a12c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55412/0 (socket says 172.21.15.52:55412) 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 -- 172.21.15.52:0/3669554171 learned_addr learned my addr 172.21.15.52:0/3669554171 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e9189700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bef60 0x5619a12c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40ea18b700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bf880 0x5619a12c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 -- 172.21.15.52:0/3669554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bf880 msgr2=0x5619a12c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bf880 0x5619a12c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 -- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bef60 msgr2=0x5619a12c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bef60 0x5619a12c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e998a700 1 -- 172.21.15.52:0/3669554171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a11d0750 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.922+0000 7f40e9189700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bef60 0x5619a12c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40e998a700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 0x5619a12c5610 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f40e0097f70 tx=0x7f40e009c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=706f4bfbf40fa66b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3669554171 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0010970 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3669554171 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40e009af60 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40d4010c60 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3669554171 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f40e009c210 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40f6471b80 10 client.18336 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5619a12d0a10 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.923+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3669554171 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40d4010c60 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3669554171 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f40e00ba2a0 con 0x5619a12b97a0 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40f6471b80 10 client.18336 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40f6471b80 10 client.18336 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:12.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40f6471b80 20 client.18336 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a12d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40f6471b80 4 client.18336 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619a1217d50 con 0x5619a12d3050 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.924+0000 7f40e9189700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a12d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.925+0000 7f40cd7fa700 10 client.18336 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.925+0000 7f40e9189700 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a12d5440 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d567c8924ba65d3d server_cookie=5b382d6d1d7d8e52 in_seq=0 out_seq=0 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.925+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3669554171 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619a1217d50 con 0x5619a12d3050 2024-06-02T23:22:12.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.925+0000 7f40cd7fa700 10 client.18336 handle_command_reply: tid=0 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.925+0000 7f40f6471b80 1 client.18336 shutdown 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 msgr2=0x5619a12c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 0x5619a12c5610 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f40e0097f70 tx=0x7f40e009c050 comp rx=0 tx=0).stop 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 shutdown_connections 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a12d5440 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12bf880 0x5619a12c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12b97a0 0x5619a12c5610 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40cd7fa700 0 client.18336 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:12.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3669554171 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bef60 0x5619a12c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 >> 172.21.15.52:0/3669554171 conn(0x5619a11e6080 msgr2=0x5619a12b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:12.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.926+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 shutdown_connections 2024-06-02T23:22:12.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.927+0000 7f40f6471b80 1 -- 172.21.15.52:0/3669554171 wait complete. 2024-06-02T23:22:12.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.927+0000 7f40f6471b80 20 client.18336 trim_cache size 0 max 16384 2024-06-02T23:22:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 Processor -- start 2024-06-02T23:22:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 -- start start 2024-06-02T23:22:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12c4850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41aecd0 con 0x5619a12bef60 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41b0810 con 0x5619a12bf880 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40f6471b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41af3a0 con 0x5619a12b97a0 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40e9189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40e998a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40e9189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12c4850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55426/0 (socket says 172.21.15.52:55426) 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40ea18b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40e9189700 1 -- 172.21.15.52:0/1338567768 learned_addr learned my addr 172.21.15.52:0/1338567768 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40ea18b700 1 -- 172.21.15.52:0/1338567768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 msgr2=0x5619a12b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40ea18b700 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40ea18b700 1 -- 172.21.15.52:0/1338567768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 msgr2=0x5619a12c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40ea18b700 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40ea18b700 1 -- 172.21.15.52:0/1338567768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619a12b4e10 con 0x5619a12bf880 2024-06-02T23:22:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.952+0000 7f40e9189700 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.953+0000 7f40ea18b700 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12b3e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b770 tx=0x7f40dc009e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0cb61a5cdc54096 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.953+0000 7f40cf7fe700 1 -- 172.21.15.52:0/1338567768 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc02b070 con 0x5619a12bf880 2024-06-02T23:22:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.953+0000 7f40cf7fe700 1 -- 172.21.15.52:0/1338567768 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40dc0074e0 con 0x5619a12bf880 2024-06-02T23:22:12.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.953+0000 7f40cf7fe700 1 -- 172.21.15.52:0/1338567768 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40dc01d680 con 0x5619a12bf880 2024-06-02T23:22:12.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 -- 172.21.15.52:0/1338567768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 msgr2=0x5619a12b3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12b3e80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f40dc00b770 tx=0x7f40dc009e20 comp rx=0 tx=0).stop 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 -- 172.21.15.52:0/1338567768 shutdown_connections 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12b3e80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 --2- 172.21.15.52:0/1338567768 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 -- 172.21.15.52:0/1338567768 >> 172.21.15.52:0/1338567768 conn(0x5619a11e6080 msgr2=0x5619a12c4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 -- 172.21.15.52:0/1338567768 shutdown_connections 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 -- 172.21.15.52:0/1338567768 wait complete. 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 -- start start 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.954+0000 7f40f6471b80 1 Processor -- start 2024-06-02T23:22:12.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12da7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12dadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41b0140 con 0x5619a12bef60 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41afa70 con 0x5619a12bf880 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40e41acc70 con 0x5619a12b97a0 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40f6471b80 10 client.0 _ll_register_callbacks cb 0x5619a11a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e998a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e998a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:45900/0 (socket says 172.21.15.52:45900) 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e998a700 1 -- 172.21.15.52:0/3954590883 learned_addr learned my addr 172.21.15.52:0/3954590883 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12da7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40ea18b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12dadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 -- 172.21.15.52:0/3954590883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 msgr2=0x5619a12cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 -- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 msgr2=0x5619a12dadd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12dadd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 -- 172.21.15.52:0/3954590883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619a133cd30 con 0x5619a12bef60 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e998a700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40ea18b700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12dadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40e9189700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12da7c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f40d400ffc0 tx=0x7f40d4010690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c2fcbf71b3556b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.955+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3954590883 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40d4016040 con 0x5619a12bef60 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40dc0074e0 con 0x5619a12bef60 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3954590883 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40d4025830 con 0x5619a12bef60 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3954590883 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f40d4041070 con 0x5619a12bef60 2024-06-02T23:22:12.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 10 client.18342 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5619a133de80 con 0x5619a12bef60 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3954590883 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40dc0074e0 con 0x5619a12bef60 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3954590883 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f40d40046f0 con 0x5619a12bef60 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 10 client.18342 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 10 client.18342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 20 client.18342 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a13402c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 4 client.18342 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619a12122d0 con 0x5619a12d3050 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.956+0000 7f40e998a700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a13402c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.957+0000 7f40e998a700 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a13402c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a17e041e2aa4e26 server_cookie=60c6f513c8ed7da6 in_seq=0 out_seq=0 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.957+0000 7f40cd7fa700 10 client.18342 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40cd7fa700 1 -- 172.21.15.52:0/3954590883 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619a12122d0 con 0x5619a12d3050 2024-06-02T23:22:12.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40cd7fa700 10 client.18342 handle_command_reply: tid=0 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 client.18342 shutdown 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 msgr2=0x5619a12da7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12da7c0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f40d400ffc0 tx=0x7f40d4010690 comp rx=0 tx=0).stop 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 shutdown_connections 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619a12d3050 0x5619a13402c0 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619a12bf880 0x5619a12dadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619a12b97a0 0x5619a12cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 --2- 172.21.15.52:0/3954590883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619a12bef60 0x5619a12da7c0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40cd7fa700 0 client.18342 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 >> 172.21.15.52:0/3954590883 conn(0x5619a11e6080 msgr2=0x5619a12be310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 shutdown_connections 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 1 -- 172.21.15.52:0/3954590883 wait complete. 2024-06-02T23:22:12.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.958+0000 7f40f6471b80 20 client.18342 trim_cache size 0 max 16384 2024-06-02T23:22:12.964 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:22:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:22:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:22:12.964 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d00a2c20 msgr2=0x7f40d00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d00a2c20 0x7f40d00a50e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f40e4108c90 tx=0x7f40d400a300 comp rx=0 tx=0).stop 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 msgr2=0x7f40e4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4074790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f40e000e9a0 tx=0x7f40e000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 shutdown_connections 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d00a2c20 0x7f40d00a50e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40e4107870 0x7f40e4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40e41027a0 0x7f40e4074790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 --2- 172.21.15.52:0/1906285344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40e4103150 0x7f40e4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 >> 172.21.15.52:0/1906285344 conn(0x7f40e4079570 msgr2=0x7f40e41004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 shutdown_connections 2024-06-02T23:22:12.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:12.959+0000 7f40ebbee700 1 -- 172.21.15.52:0/1906285344 wait complete. 2024-06-02T23:22:13.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 201638 1.87 MB/sec execute 442 sec latency 36.235 ms 2024-06-02T23:22:13.793 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:22:13.793 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:22:13.817 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:13 smithi052 ceph-mon[32340]: pgmap v741: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-02T23:22:14.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:13 smithi123 ceph-mon[35977]: pgmap v741: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-02T23:22:14.077 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:13 smithi174 ceph-mon[35994]: pgmap v741: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 305 KiB/s rd, 1.4 MiB/s wr, 176 op/s 2024-06-02T23:22:14.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 202169 1.88 MB/sec execute 443 sec latency 15.315 ms 2024-06-02T23:22:15.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 202550 1.88 MB/sec execute 444 sec latency 14.412 ms 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 -- 172.21.15.52:0/1647770188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 msgr2=0x7fa36c06a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1647770188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c06a5f0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fa358009990 tx=0x7fa358009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 -- 172.21.15.52:0/1647770188 shutdown_connections 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1647770188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa36c06ab30 0x7fa36c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1647770188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36c108500 0x7fa36c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1647770188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c06a5f0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:15.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 -- 172.21.15.52:0/1647770188 >> 172.21.15.52:0/1647770188 conn(0x7fa36c078340 msgr2=0x7fa36c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:15.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 -- 172.21.15.52:0/1647770188 shutdown_connections 2024-06-02T23:22:15.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.850+0000 7fa372cbf700 1 -- 172.21.15.52:0/1647770188 wait complete. 2024-06-02T23:22:15.852 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:15 smithi052 ceph-mon[32340]: pgmap v742: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:22:15.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 Processor -- start 2024-06-02T23:22:15.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 -- start start 2024-06-02T23:22:15.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:15.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36c06ab30 0x7fa36c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:15.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa36c108500 0x7fa36c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c112660 con 0x7fa36c06a170 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1124e0 con 0x7fa36c06ab30 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa372cbf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa36c1127e0 con 0x7fa36c108500 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa370a5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa370a5b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39972/0 (socket says 172.21.15.52:39972) 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa370a5b700 1 -- 172.21.15.52:0/1112124270 learned_addr learned my addr 172.21.15.52:0/1112124270 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa37125c700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa36c108500 0x7fa36c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.851+0000 7fa36bfff700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36c06ab30 0x7fa36c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa370a5b700 1 -- 172.21.15.52:0/1112124270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa36c108500 msgr2=0x7fa36c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa370a5b700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa36c108500 0x7fa36c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa370a5b700 1 -- 172.21.15.52:0/1112124270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36c06ab30 msgr2=0x7fa36c19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa370a5b700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36c06ab30 0x7fa36c19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:15.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa370a5b700 1 -- 172.21.15.52:0/1112124270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa36c199120 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa370a5b700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c19e800 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e9a0 tx=0x7fa36000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa36000ca10 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa36000cbb0 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa36c199410 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa360018680 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.852+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa36c1a56d0 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.853+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa36c069180 con 0x7fa36c06a170 2024-06-02T23:22:15.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.855+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa360010430 con 0x7fa36c06a170 2024-06-02T23:22:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.856+0000 7fa369ffb700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2c20 0x7fa35c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.856+0000 7fa36bfff700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2c20 0x7fa35c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:15.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.856+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa3600cf330 con 0x7fa36c06a170 2024-06-02T23:22:15.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.856+0000 7fa36bfff700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2c20 0x7fa35c0a50e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa36c078a20 tx=0x7fa358005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:15.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:15.860+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa36009c2c0 con 0x7fa36c06a170 2024-06-02T23:22:16.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:15 smithi123 ceph-mon[35977]: pgmap v742: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:22:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:15 smithi174 ceph-mon[35994]: pgmap v742: 97 pgs: 97 active+clean; 88 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.2 MiB/s wr, 143 op/s 2024-06-02T23:22:16.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.149+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa36c199ef0 con 0x7fa36c06a170 2024-06-02T23:22:16.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.150+0000 7fa369ffb700 1 -- 172.21.15.52:0/1112124270 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fa36009bbc0 con 0x7fa36c06a170 2024-06-02T23:22:16.150 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:22:16.151 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:22:16.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2c20 msgr2=0x7fa35c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2c20 0x7fa35c0a50e0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fa36c078a20 tx=0x7fa358005fd0 comp rx=0 tx=0).stop 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 msgr2=0x7fa36c19e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c19e800 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fa36000e9a0 tx=0x7fa36000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 shutdown_connections 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa35c0a2c20 0x7fa35c0a50e0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa36c108500 0x7fa36c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa36c06a170 0x7fa36c19e800 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 --2- 172.21.15.52:0/1112124270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36c06ab30 0x7fa36c19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.152+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 >> 172.21.15.52:0/1112124270 conn(0x7fa36c078340 msgr2=0x7fa36c100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.153+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 shutdown_connections 2024-06-02T23:22:16.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:16.153+0000 7fa372cbf700 1 -- 172.21.15.52:0/1112124270 wait complete. 2024-06-02T23:22:16.154 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:22:16.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 202851 1.88 MB/sec execute 445 sec latency 13.194 ms 2024-06-02T23:22:16.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:16 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1112124270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:16.935 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:22:17.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:16 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1112124270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:16 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1112124270' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:17.250 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:17.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 203390 1.88 MB/sec execute 446 sec latency 11.564 ms 2024-06-02T23:22:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:17 smithi052 ceph-mon[32340]: pgmap v743: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-02T23:22:18.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:17 smithi123 ceph-mon[35977]: pgmap v743: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-02T23:22:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:17 smithi174 ceph-mon[35994]: pgmap v743: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-02T23:22:18.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 -- 172.21.15.52:0/3047446151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce8104e10 msgr2=0x7f2ce8107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/3047446151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce8104e10 0x7f2ce8107200 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f2cd8009900 tx=0x7f2cd802a410 comp rx=0 tx=0).stop 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 -- 172.21.15.52:0/3047446151 shutdown_connections 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/3047446151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8107740 0x7f2ce8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/3047446151 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce8104e10 0x7f2ce8107200 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/3047446151 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce81024e0 0x7f2ce81048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 -- 172.21.15.52:0/3047446151 >> 172.21.15.52:0/3047446151 conn(0x7f2ce80fbe70 msgr2=0x7f2ce80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 -- 172.21.15.52:0/3047446151 shutdown_connections 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.182+0000 7f2cefcdd640 1 -- 172.21.15.52:0/3047446151 wait complete. 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 Processor -- start 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 -- start start 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 0x7f2ce8101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 0x7f2ce8100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce8107740 0x7f2ce81009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810d500 con 0x7f2ce81024e0 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810d380 con 0x7f2ce8104e10 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cefcdd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce810d680 con 0x7f2ce8107740 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 0x7f2ce8101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 0x7f2ce8100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2cee27a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce8107740 0x7f2ce81009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 0x7f2ce8100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48232/0 (socket says 172.21.15.52:48232) 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 0x7f2ce8101e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39990/0 (socket says 172.21.15.52:39990) 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 -- 172.21.15.52:0/1141501201 learned_addr learned my addr 172.21.15.52:0/1141501201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 -- 172.21.15.52:0/1141501201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce8107740 msgr2=0x7f2ce81009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce8107740 0x7f2ce81009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 -- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 msgr2=0x7f2ce8101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 0x7f2ce8101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ced278640 1 -- 172.21.15.52:0/1141501201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce8101260 con 0x7f2ce8104e10 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.183+0000 7f2ceda79640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 0x7f2ce8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.184+0000 7f2ced278640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 0x7f2ce8100490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80314e0 tx=0x7f2cd8031a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.184+0000 7f2cdeffd640 1 -- 172.21.15.52:0/1141501201 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd803f030 con 0x7f2ce8104e10 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.184+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce81a1c00 con 0x7f2ce8104e10 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.184+0000 7f2cdeffd640 1 -- 172.21.15.52:0/1141501201 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cd8003aa0 con 0x7f2ce8104e10 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.184+0000 7f2cdeffd640 1 -- 172.21.15.52:0/1141501201 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8035070 con 0x7f2ce8104e10 2024-06-02T23:22:18.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.184+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce81a21a0 con 0x7f2ce8104e10 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.186+0000 7f2cdeffd640 1 -- 172.21.15.52:0/1141501201 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2cd8042030 con 0x7f2ce8104e10 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 Processor -- start 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 -- start start 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9ed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9f2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9f5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557b4b9504f0 con 0x557b4b9ed010 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b4b8ee870 con 0x557b4b9f2b70 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b4b94c4b0 con 0x557b4b9eda00 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2cee27a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9ed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9f2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2ced278640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9f2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38468/0 (socket says 172.21.15.52:38468) 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2ceda79640 1 -- 172.21.15.52:0/3663422751 learned_addr learned my addr 172.21.15.52:0/3663422751 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.187+0000 7f2ceda79640 1 -- 172.21.15.52:0/3663422751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 msgr2=0x557b4b9f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2ceda79640 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2ceda79640 1 -- 172.21.15.52:0/3663422751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 msgr2=0x557b4b9ed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2ceda79640 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9ed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2ceda79640 1 -- 172.21.15.52:0/3663422751 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4b90a9b0 con 0x557b4b9eda00 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2cdeffd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd40a2c30 0x7f2cd40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2ceda79640 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9f2500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400f4c0 tx=0x7f2ce400f980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e3520c4eb22b5c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2cee27a640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd40a2c30 0x7f2cd40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2cdeffd640 1 -- 172.21.15.52:0/1141501201 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2cd80ef520 con 0x7f2ce8104e10 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2cd3fff640 1 -- 172.21.15.52:0/3663422751 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4005220 con 0x557b4b9eda00 2024-06-02T23:22:18.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2cd3fff640 1 -- 172.21.15.52:0/3663422751 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ce40047a0 con 0x557b4b9eda00 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.188+0000 7f2cd3fff640 1 -- 172.21.15.52:0/3663422751 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce4010ac0 con 0x557b4b9eda00 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 -- 172.21.15.52:0/3663422751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 msgr2=0x557b4b9f2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9f2500 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2ce400f4c0 tx=0x7f2ce400f980 comp rx=0 tx=0).stop 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cee27a640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd40a2c30 0x7f2cd40a50f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x557b4b9ee260 tx=0x7f2ce0009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 -- 172.21.15.52:0/3663422751 shutdown_connections 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9f2500 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/3663422751 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9ed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 -- 172.21.15.52:0/3663422751 >> 172.21.15.52:0/3663422751 conn(0x557b4b9e8680 msgr2=0x557b4b9e8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 -- 172.21.15.52:0/3663422751 shutdown_connections 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 -- 172.21.15.52:0/3663422751 wait complete. 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 -- start start 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.189+0000 7f2cf29ea740 1 Processor -- start 2024-06-02T23:22:18.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9ed010 0x557b4b9f7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 0x557b4b9f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fe790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557b4b9f5b20 con 0x557b4b9eda00 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b4b8d9ae0 con 0x557b4b9f2b70 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b4b9f5ca0 con 0x557b4b9ed010 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cf29ea740 10 client.0 _ll_register_callbacks cb 0x557b4b9129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 0x557b4b9f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 0x557b4b9f77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40014/0 (socket says 172.21.15.52:40014) 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 -- 172.21.15.52:0/2182455064 learned_addr learned my addr 172.21.15.52:0/2182455064 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9ed010 0x557b4b9f7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 -- 172.21.15.52:0/2182455064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9ed010 msgr2=0x557b4b9f7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9ed010 0x557b4b9f7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 -- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 msgr2=0x557b4b9fe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ced278640 1 -- 172.21.15.52:0/2182455064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4b664210 con 0x557b4b9eda00 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9ed010 0x557b4b9f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.190+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2ced278640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 0x557b4b9f77c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80c44f0 tx=0x7f2cd8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d892b91baf2ae27 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2182455064 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8049040 con 0x557b4b9eda00 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2182455064 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2cd8003c50 con 0x557b4b9eda00 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ce40047a0 con 0x557b4b9eda00 2024-06-02T23:22:18.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2182455064 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2cd80bbb80 con 0x557b4b9eda00 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cf29ea740 10 client.18360 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b4b730620 con 0x557b4b9eda00 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2182455064 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ce40047a0 con 0x557b4b9eda00 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.191+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2182455064 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2cd80bc010 con 0x557b4b9eda00 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cf29ea740 10 client.18360 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cf29ea740 10 client.18360 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cf29ea740 20 client.18360 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba030c0 0x557b4ba05580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cf29ea740 4 client.18360 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b4b963f10 con 0x557b4ba030c0 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba030c0 0x557b4ba05580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba030c0 0x557b4ba05580 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=709edcd958f0df23 server_cookie=93007843bbaff1fd in_seq=0 out_seq=0 2024-06-02T23:22:18.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.192+0000 7f2cd17fa640 10 client.18360 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2182455064 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b4b963f10 con 0x557b4ba030c0 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cd17fa640 10 client.18360 handle_command_reply: tid=0 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 client.18360 shutdown 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 msgr2=0x557b4b9f77c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 0x557b4b9f77c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80c44f0 tx=0x7f2cd8009260 comp rx=0 tx=0).stop 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 shutdown_connections 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba030c0 0x557b4ba05580 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9eda00 0x557b4b9f77c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cd17fa640 0 client.18360 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2182455064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9ed010 0x557b4b9f7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 >> 172.21.15.52:0/2182455064 conn(0x557b4b9e8680 msgr2=0x557b4b9e77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 shutdown_connections 2024-06-02T23:22:18.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2182455064 wait complete. 2024-06-02T23:22:18.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.193+0000 7f2cf29ea740 20 client.18360 trim_cache size 0 max 16384 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 Processor -- start 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 -- start start 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9eebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9eda00 0x557b4b9fe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9f2b70 0x557b4b9ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81aae30 con 0x557b4b9ed010 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81ac970 con 0x557b4b9eda00 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.210+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81ab500 con 0x557b4b9f2b70 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9eebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9eebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40016/0 (socket says 172.21.15.52:40016) 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ceda79640 1 -- 172.21.15.52:0/2644470102 learned_addr learned my addr 172.21.15.52:0/2644470102 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:18.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9eda00 0x557b4b9fe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9f2b70 0x557b4b9ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 -- 172.21.15.52:0/2644470102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9f2b70 msgr2=0x557b4b9ef120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9f2b70 0x557b4b9ef120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 -- 172.21.15.52:0/2644470102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 msgr2=0x557b4b9eebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9eebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 -- 172.21.15.52:0/2644470102 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b4b9ee6d0 con 0x557b4b9eda00 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9f2b70 0x557b4b9ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2ced278640 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9eda00 0x557b4b9fe750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80bbab0 tx=0x7f2cd80ba2c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25bae4d68b209381 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.211+0000 7f2cd37fe640 1 -- 172.21.15.52:0/2644470102 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd809f070 con 0x557b4b9eda00 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cd37fe640 1 -- 172.21.15.52:0/2644470102 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cd80bf020 con 0x557b4b9eda00 2024-06-02T23:22:18.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cd37fe640 1 -- 172.21.15.52:0/2644470102 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd8049070 con 0x557b4b9eda00 2024-06-02T23:22:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2644470102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9eda00 msgr2=0x557b4b9fe750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9eda00 0x557b4b9fe750 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80bbab0 tx=0x7f2cd80ba2c0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2644470102 shutdown_connections 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9f2b70 0x557b4b9ef120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9eda00 0x557b4b9fe750 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2644470102 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9eebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2644470102 >> 172.21.15.52:0/2644470102 conn(0x557b4b9e8680 msgr2=0x557b4b9f2240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2644470102 shutdown_connections 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.212+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2644470102 wait complete. 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 -- start start 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 Processor -- start 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9fe380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9fae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81ac2a0 con 0x557b4b9ed010 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81abbd0 con 0x557b4b9f2b70 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81a8e10 con 0x557b4b9eda00 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cf29ea740 10 client.0 _ll_register_callbacks cb 0x7f2cd804cfb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9fe380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2cee27a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2ceda79640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9fe380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40028/0 (socket says 172.21.15.52:40028) 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.213+0000 7f2ceda79640 1 -- 172.21.15.52:0/2030728695 learned_addr learned my addr 172.21.15.52:0/2030728695 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2ced278640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9fae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cee27a640 1 -- 172.21.15.52:0/2030728695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 msgr2=0x557b4b9fae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9fae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cee27a640 1 -- 172.21.15.52:0/2030728695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 msgr2=0x557b4b9fe380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9fe380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cee27a640 1 -- 172.21.15.52:0/2030728695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b4b9fe9b0 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cee27a640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fb3d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009f50 tx=0x7f2ce0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce81994f02ed2226 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cd17fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2030728695 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce002d060 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2030728695 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ce000fc40 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cd80bf020 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2030728695 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2ce000fe40 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cf29ea740 10 client.27824 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.214+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557b4ba6ce90 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2030728695 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cd80bf020 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2030728695 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2ce00057b0 con 0x557b4b9f2b70 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cf29ea740 10 client.27824 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cf29ea740 10 client.27824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:18.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cf29ea740 20 client.27824 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba6d850 0x557b4ba6fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cf29ea740 4 client.27824 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:22:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b4b95dc20 con 0x557b4ba6d850 2024-06-02T23:22:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba6d850 0x557b4ba6fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.216+0000 7f2cd17fa640 10 client.27824 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:18.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.215+0000 7f2ceda79640 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba6d850 0x557b4ba6fc70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d74e521150c291e server_cookie=6665c9fcafca49c4 in_seq=0 out_seq=0 2024-06-02T23:22:18.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cd17fa640 1 -- 172.21.15.52:0/2030728695 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557b4b95dc20 con 0x557b4ba6d850 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cd17fa640 10 client.27824 handle_command_reply: tid=0 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 client.27824 shutdown 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 msgr2=0x557b4b9fb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fb3d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009f50 tx=0x7f2ce0004a40 comp rx=0 tx=0).stop 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 shutdown_connections 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b4ba6d850 0x557b4ba6fc70 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b4b9eda00 0x557b4b9fae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cd17fa640 0 client.27824 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b4b9f2b70 0x557b4b9fb3d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 --2- 172.21.15.52:0/2030728695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b4b9ed010 0x557b4b9fe380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 >> 172.21.15.52:0/2030728695 conn(0x557b4b9e8680 msgr2=0x557b4b9f1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 shutdown_connections 2024-06-02T23:22:18.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 1 -- 172.21.15.52:0/2030728695 wait complete. 2024-06-02T23:22:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.235+0000 7f2cf29ea740 20 client.27824 trim_cache size 0 max 16384 2024-06-02T23:22:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:22:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:22:18.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd40a2c30 msgr2=0x7f2cd40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd40a2c30 0x7f2cd40a50f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x557b4b9ee260 tx=0x7f2ce0009500 comp rx=0 tx=0).stop 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 msgr2=0x7f2ce8100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 0x7f2ce8100490 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2cd80314e0 tx=0x7f2cd8031a30 comp rx=0 tx=0).stop 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 shutdown_connections 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd40a2c30 0x7f2cd40a50f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce8107740 0x7f2ce81009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce8104e10 0x7f2ce8100490 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 --2- 172.21.15.52:0/1141501201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce81024e0 0x7f2ce8101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 >> 172.21.15.52:0/1141501201 conn(0x7f2ce80fbe70 msgr2=0x7f2ce8102da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 shutdown_connections 2024-06-02T23:22:18.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:18.236+0000 7f2cefcdd640 1 -- 172.21.15.52:0/1141501201 wait complete. 2024-06-02T23:22:18.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 204000 1.88 MB/sec execute 447 sec latency 15.563 ms 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 -- 172.21.15.52:0/1779440577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 msgr2=0x7f87341017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 --2- 172.21.15.52:0/1779440577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f87341017f0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f8720009990 tx=0x7f8720009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 -- 172.21.15.52:0/1779440577 shutdown_connections 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 --2- 172.21.15.52:0/1779440577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8734101d30 0x7f873410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 --2- 172.21.15.52:0/1779440577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f87341073d0 0x7f87341077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 --2- 172.21.15.52:0/1779440577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f87341017f0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 -- 172.21.15.52:0/1779440577 >> 172.21.15.52:0/1779440577 conn(0x7f87340782c0 msgr2=0x7f87340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:19.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 -- 172.21.15.52:0/1779440577 shutdown_connections 2024-06-02T23:22:19.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.086+0000 7f873aaf1700 1 -- 172.21.15.52:0/1779440577 wait complete. 2024-06-02T23:22:19.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 Processor -- start 2024-06-02T23:22:19.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 -- start start 2024-06-02T23:22:19.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f873419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8734101d30 0x7f873419dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87341073d0 0x7f8734197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87341114e0 con 0x7f8734101390 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8734111360 con 0x7f87341073d0 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873aaf1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8734111660 con 0x7f8734101d30 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873888d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f873419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f8733fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8734101d30 0x7f873419dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873888d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f873419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40054/0 (socket says 172.21.15.52:40054) 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.087+0000 7f873888d700 1 -- 172.21.15.52:0/4198218040 learned_addr learned my addr 172.21.15.52:0/4198218040 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873908e700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87341073d0 0x7f8734197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873888d700 1 -- 172.21.15.52:0/4198218040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8734101d30 msgr2=0x7f873419dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873888d700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8734101d30 0x7f873419dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873888d700 1 -- 172.21.15.52:0/4198218040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87341073d0 msgr2=0x7f8734197860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873888d700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87341073d0 0x7f8734197860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873888d700 1 -- 172.21.15.52:0/4198218040 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8734197fc0 con 0x7f8734101390 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f8733fff700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8734101d30 0x7f873419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873888d700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f873419d6a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f872800d980 tx=0x7f872800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f8731ffb700 1 -- 172.21.15.52:0/4198218040 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728016070 con 0x7f8734101390 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87341982b0 con 0x7f8734101390 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f8731ffb700 1 -- 172.21.15.52:0/4198218040 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8728004830 con 0x7f8734101390 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.088+0000 7f8731ffb700 1 -- 172.21.15.52:0/4198218040 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728005530 con 0x7f8734101390 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.089+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873406a170 con 0x7f8734101390 2024-06-02T23:22:19.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.091+0000 7f8731ffb700 1 -- 172.21.15.52:0/4198218040 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f87280049e0 con 0x7f8734101390 2024-06-02T23:22:19.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.092+0000 7f8731ffb700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87240a2ca0 0x7f87240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.092+0000 7f8733fff700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87240a2ca0 0x7f87240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.092+0000 7f8731ffb700 1 -- 172.21.15.52:0/4198218040 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f87280cec90 con 0x7f8734101390 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.092+0000 7f8745374b80 1 Processor -- start 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 -- start start 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8733fff700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87240a2ca0 0x7f87240a5160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f87200098e0 tx=0x7f8720005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0640 0x55ec90ee0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0f60 0x55ec90eda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90edb340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ec90e2c810 con 0x55ec90ee0f60 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec90dd4720 con 0x55ec90edaf20 2024-06-02T23:22:19.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8745374b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec90e287d0 con 0x55ec90ee0640 2024-06-02T23:22:19.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8733fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0f60 0x55ec90eda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8733fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0f60 0x55ec90eda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40056/0 (socket says 172.21.15.52:40056) 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f8733fff700 1 -- 172.21.15.52:0/1201114460 learned_addr learned my addr 172.21.15.52:0/1201114460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873908e700 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90edb340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873888d700 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0640 0x55ec90ee0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873908e700 1 -- 172.21.15.52:0/1201114460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0640 msgr2=0x55ec90ee0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873908e700 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0640 0x55ec90ee0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873908e700 1 -- 172.21.15.52:0/1201114460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0f60 msgr2=0x55ec90eda9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873908e700 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0f60 0x55ec90eda9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.093+0000 7f873908e700 1 -- 172.21.15.52:0/1201114460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec90dcaf50 con 0x55ec90edaf20 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.094+0000 7f873908e700 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90edb340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f872c011580 tx=0x7f872c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29afa7db6fac8602 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.094+0000 7f871effd700 1 -- 172.21.15.52:0/1201114460 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c011e80 con 0x55ec90edaf20 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.094+0000 7f871effd700 1 -- 172.21.15.52:0/1201114460 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c004bb0 con 0x55ec90edaf20 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.094+0000 7f871effd700 1 -- 172.21.15.52:0/1201114460 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c004690 con 0x55ec90edaf20 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 -- 172.21.15.52:0/1201114460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 msgr2=0x55ec90edb340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90edb340 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f872c011580 tx=0x7f872c009920 comp rx=0 tx=0).stop 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 -- 172.21.15.52:0/1201114460 shutdown_connections 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90edb340 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0640 0x55ec90ee0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 --2- 172.21.15.52:0/1201114460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0f60 0x55ec90eda9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 -- 172.21.15.52:0/1201114460 >> 172.21.15.52:0/1201114460 conn(0x55ec90e05cc0 msgr2=0x55ec90ed5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 -- 172.21.15.52:0/1201114460 shutdown_connections 2024-06-02T23:22:19.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 -- 172.21.15.52:0/1201114460 wait complete. 2024-06-02T23:22:19.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 -- start start 2024-06-02T23:22:19.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.095+0000 7f8745374b80 1 Processor -- start 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90ee6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0640 0x55ec90ee4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ec90ee79d0 con 0x55ec90ee0640 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec90dccbf0 con 0x55ec90edaf20 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec90ee7b50 con 0x55ec90ee0f60 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8745374b80 10 client.0 _ll_register_callbacks cb 0x55ec90dc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90ee6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90ee6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48296/0 (socket says 172.21.15.52:48296) 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 -- 172.21.15.52:0/2067895412 learned_addr learned my addr 172.21.15.52:0/2067895412 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873908e700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8733fff700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0640 0x55ec90ee4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 -- 172.21.15.52:0/2067895412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 msgr2=0x55ec90ee45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 -- 172.21.15.52:0/2067895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0640 msgr2=0x55ec90ee4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0640 0x55ec90ee4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 -- 172.21.15.52:0/2067895412 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec90d7b8d0 con 0x55ec90edaf20 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873908e700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f873888d700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90ee6d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87280ccb30 tx=0x7f872809af00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d702f2f8a94aabb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.096+0000 7f8713fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8713fff700 1 -- 172.21.15.52:0/2067895412 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87280b4060 con 0x55ec90edaf20 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8713fff700 1 -- 172.21.15.52:0/2067895412 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f87280a3d00 con 0x55ec90edaf20 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872c004bb0 con 0x55ec90edaf20 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8713fff700 1 -- 172.21.15.52:0/2067895412 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8728022350 con 0x55ec90edaf20 2024-06-02T23:22:19.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 10 client.27836 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec90ef23c0 con 0x55ec90edaf20 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8713fff700 1 -- 172.21.15.52:0/2067895412 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c004bb0 con 0x55ec90edaf20 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8713fff700 1 -- 172.21.15.52:0/2067895412 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f87280b4200 con 0x55ec90edaf20 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 10 client.27836 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 10 client.27836 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 20 client.27836 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90ef6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 4 client.27836 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.097+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec90e39d10 con 0x55ec90ef4950 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.098+0000 7f8733fff700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90ef6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.098+0000 7f8733fff700 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90ef6d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2bcf6b584de3bfac server_cookie=3d04c61a94c1d6b2 in_seq=0 out_seq=0 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.098+0000 7f8713fff700 10 client.27836 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8713fff700 1 -- 172.21.15.52:0/2067895412 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec90e39d10 con 0x55ec90ef4950 2024-06-02T23:22:19.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8713fff700 10 client.27836 handle_command_reply: tid=0 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 client.27836 shutdown 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 msgr2=0x55ec90ee6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90ee6d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f87280ccb30 tx=0x7f872809af00 comp rx=0 tx=0).stop 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 shutdown_connections 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90ef6d40 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90edaf20 0x55ec90ee6d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8713fff700 0 client.27836 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 --2- 172.21.15.52:0/2067895412 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90ee0640 0x55ec90ee4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 >> 172.21.15.52:0/2067895412 conn(0x55ec90e05cc0 msgr2=0x55ec90ed99d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 shutdown_connections 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 1 -- 172.21.15.52:0/2067895412 wait complete. 2024-06-02T23:22:19.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.099+0000 7f8745374b80 20 client.27836 trim_cache size 0 max 16384 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 Processor -- start 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 -- start start 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eda9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90ecf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87341aec80 con 0x55ec90edaf20 2024-06-02T23:22:19.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b07c0 con 0x55ec90ee0640 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.117+0000 7f8745374b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87341af350 con 0x55ec90ee0f60 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f873888d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eda9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90ecf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f873888d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eda9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40082/0 (socket says 172.21.15.52:40082) 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f873888d700 1 -- 172.21.15.52:0/3829124435 learned_addr learned my addr 172.21.15.52:0/3829124435 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f873908e700 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 -- 172.21.15.52:0/3829124435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 msgr2=0x55ec90ee1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 -- 172.21.15.52:0/3829124435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 msgr2=0x55ec90eda9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eda9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 -- 172.21.15.52:0/3829124435 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec90ed5380 con 0x55ec90ee0640 2024-06-02T23:22:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f8733fff700 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90ecf690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87200059d0 tx=0x7f8720021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25235f9d5627a3c6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f871e7fc700 1 -- 172.21.15.52:0/3829124435 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872000f460 con 0x55ec90ee0640 2024-06-02T23:22:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f871e7fc700 1 -- 172.21.15.52:0/3829124435 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872000fab0 con 0x55ec90ee0640 2024-06-02T23:22:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.118+0000 7f871e7fc700 1 -- 172.21.15.52:0/3829124435 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872003a820 con 0x55ec90ee0640 2024-06-02T23:22:19.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 -- 172.21.15.52:0/3829124435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 msgr2=0x55ec90ecf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90ecf690 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f87200059d0 tx=0x7f8720021000 comp rx=0 tx=0).stop 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 -- 172.21.15.52:0/3829124435 shutdown_connections 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eda9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 --2- 172.21.15.52:0/3829124435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90ecf690 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 -- 172.21.15.52:0/3829124435 >> 172.21.15.52:0/3829124435 conn(0x55ec90e05cc0 msgr2=0x55ec90ee1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 -- 172.21.15.52:0/3829124435 shutdown_connections 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.119+0000 7f8745374b80 1 -- 172.21.15.52:0/3829124435 wait complete. 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 -- start start 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 Processor -- start 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eece10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90eed350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f87341b00f0 con 0x55ec90edaf20 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f873888d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eece10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f873888d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eece10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40094/0 (socket says 172.21.15.52:40094) 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f873888d700 1 -- 172.21.15.52:0/738909658 learned_addr learned my addr 172.21.15.52:0/738909658 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f87341afa20 con 0x55ec90ee0640 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.120+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f87341acc20 con 0x55ec90ee0f60 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8733fff700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90eed350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8745374b80 10 client.0 _ll_register_callbacks cb 0x55ec90dc6920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873908e700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873888d700 1 -- 172.21.15.52:0/738909658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 msgr2=0x55ec90ee9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873888d700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873888d700 1 -- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 msgr2=0x55ec90eed350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873888d700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90eed350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873888d700 1 -- 172.21.15.52:0/738909658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec90eea320 con 0x55ec90edaf20 2024-06-02T23:22:19.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873908e700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8733fff700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90eed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f873888d700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eece10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f872809b4b0 tx=0x7f8728004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd3d32b184bb5f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8713fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8713fff700 1 -- 172.21.15.52:0/738909658 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8728004420 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8713fff700 1 -- 172.21.15.52:0/738909658 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f87280b4060 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f872000fab0 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.121+0000 7f8713fff700 1 -- 172.21.15.52:0/738909658 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f872809bc20 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 10 client.18384 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55ec90f5fbd0 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8713fff700 1 -- 172.21.15.52:0/738909658 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872000fab0 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8713fff700 1 -- 172.21.15.52:0/738909658 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f8728084ab0 con 0x55ec90edaf20 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 10 client.18384 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 10 client.18384 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:22:19.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 20 client.18384 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90f622b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 4 client.18384 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:22:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.122+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec90e34290 con 0x55ec90ef4950 2024-06-02T23:22:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.123+0000 7f8733fff700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90f622b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.123+0000 7f8713fff700 10 client.18384 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:22:19.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.123+0000 7f8733fff700 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90f622b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d01a5311a022228 server_cookie=fa45e1829a9d0bb9 in_seq=0 out_seq=0 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.123+0000 7f8713fff700 1 -- 172.21.15.52:0/738909658 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec90e34290 con 0x55ec90ef4950 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.123+0000 7f8713fff700 10 client.18384 handle_command_reply: tid=0 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.123+0000 7f8745374b80 1 client.18384 shutdown 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 msgr2=0x55ec90eece10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eece10 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f872809b4b0 tx=0x7f8728004750 comp rx=0 tx=0).stop 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 shutdown_connections 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec90ee0f60 0x55ec90ee9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec90edaf20 0x55ec90eece10 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55ec90ef4950 0x55ec90f622b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 --2- 172.21.15.52:0/738909658 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec90ee0640 0x55ec90eed350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8713fff700 0 client.18384 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:22:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 >> 172.21.15.52:0/738909658 conn(0x55ec90e05cc0 msgr2=0x55ec90ee1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 shutdown_connections 2024-06-02T23:22:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 1 -- 172.21.15.52:0/738909658 wait complete. 2024-06-02T23:22:19.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.124+0000 7f8745374b80 20 client.18384 trim_cache size 0 max 16384 2024-06-02T23:22:19.129 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87240a2ca0 msgr2=0x7f87240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87240a2ca0 0x7f87240a5160 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f87200098e0 tx=0x7f8720005960 comp rx=0 tx=0).stop 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 msgr2=0x7f873419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f873419d6a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f872800d980 tx=0x7f872800dc90 comp rx=0 tx=0).stop 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 shutdown_connections 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87240a2ca0 0x7f87240a5160 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87341073d0 0x7f8734197860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8734101390 0x7f873419d6a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 --2- 172.21.15.52:0/4198218040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8734101d30 0x7f873419dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 >> 172.21.15.52:0/4198218040 conn(0x7f87340782c0 msgr2=0x7f8734100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 shutdown_connections 2024-06-02T23:22:19.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:19.125+0000 7f873aaf1700 1 -- 172.21.15.52:0/4198218040 wait complete. 2024-06-02T23:22:19.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 204599 1.88 MB/sec execute 448 sec latency 14.800 ms 2024-06-02T23:22:19.937 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:22:20.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:19 smithi123 ceph-mon[35977]: pgmap v744: 97 pgs: 97 active+clean; 90 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-06-02T23:22:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:19 smithi174 ceph-mon[35994]: pgmap v744: 97 pgs: 97 active+clean; 90 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-06-02T23:22:20.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:19 smithi052 ceph-mon[32340]: pgmap v744: 97 pgs: 97 active+clean; 90 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-06-02T23:22:20.203 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:20.646 INFO:tasks.workunit.client.0.smithi052.stdout: 1 205216 1.89 MB/sec execute 449 sec latency 15.118 ms 2024-06-02T23:22:21.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 205803 1.89 MB/sec execute 450 sec latency 10.520 ms 2024-06-02T23:22:21.822 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:21 smithi052 ceph-mon[32340]: pgmap v745: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-02T23:22:22.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:21 smithi123 ceph-mon[35977]: pgmap v745: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3950289526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0108510 msgr2=0x7f7fe01088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3950289526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0108510 0x7f7fe01088f0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009990 tx=0x7f7fd4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3950289526 shutdown_connections 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3950289526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0101440 0x7f7fe010f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3950289526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0108510 0x7f7fe01088f0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3950289526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0100a80 0x7f7fe0100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3950289526 >> 172.21.15.52:0/3950289526 conn(0x7f7fe0077a00 msgr2=0x7f7fe00fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3950289526 shutdown_connections 2024-06-02T23:22:22.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.102+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3950289526 wait complete. 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 Processor -- start 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 -- start start 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 0x7f7fe0101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0101440 0x7f7fe01021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0108510 0x7f7fe01026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe0112620 con 0x7f7fe0100a80 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01124a0 con 0x7f7fe0108510 2024-06-02T23:22:22.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe6ae8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01127a0 con 0x7f7fe0101440 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 0x7f7fe0101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 0x7f7fe0101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40114/0 (socket says 172.21.15.52:40114) 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe5085700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0108510 0x7f7fe01026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fe4884700 1 -- 172.21.15.52:0/3466421873 learned_addr learned my addr 172.21.15.52:0/3466421873 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.103+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0101440 0x7f7fe01021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe5085700 1 -- 172.21.15.52:0/3466421873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0101440 msgr2=0x7f7fe01021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe5085700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0101440 0x7f7fe01021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe5085700 1 -- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 msgr2=0x7f7fe0101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe5085700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 0x7f7fe0101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe5085700 1 -- 172.21.15.52:0/3466421873 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe01a7140 con 0x7f7fe0108510 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe4884700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 0x7f7fe0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0101440 0x7f7fe01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:22.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe5085700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0108510 0x7f7fe01026e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800efc0 tx=0x7f7fd800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:22.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fddffb700 1 -- 172.21.15.52:0/3466421873 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd80092e0 con 0x7f7fe0108510 2024-06-02T23:22:22.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe01a73d0 con 0x7f7fe0108510 2024-06-02T23:22:22.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fddffb700 1 -- 172.21.15.52:0/3466421873 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fd8004bb0 con 0x7f7fe0108510 2024-06-02T23:22:22.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.104+0000 7f7fddffb700 1 -- 172.21.15.52:0/3466421873 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8004020 con 0x7f7fe0108510 2024-06-02T23:22:22.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.105+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe01a7910 con 0x7f7fe0108510 2024-06-02T23:22:22.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.107+0000 7f7fddffb700 1 -- 172.21.15.52:0/3466421873 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fd801b030 con 0x7f7fe0108510 2024-06-02T23:22:22.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.108+0000 7f7fddffb700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c20 0x7f7fd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.108+0000 7f7fe4884700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c20 0x7f7fd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.108+0000 7f7fddffb700 1 -- 172.21.15.52:0/3466421873 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fd80cea70 con 0x7f7fe0108510 2024-06-02T23:22:22.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.108+0000 7f7fe4884700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c20 0x7f7fd00a50e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006010 tx=0x7f7fd400b630 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:22.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 Processor -- start 2024-06-02T23:22:22.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 -- start start 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecaca0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecaca0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562ecab92f60 con 0x562ecaca5f60 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ecabf1850 con 0x562ecaca07a0 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ecab59460 con 0x562ecaca6880 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7fdffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecaca0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40130/0 (socket says 172.21.15.52:40130) 2024-06-02T23:22:22.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7fe4884700 1 -- 172.21.15.52:0/3144607841 learned_addr learned my addr 172.21.15.52:0/3144607841 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:22.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.109+0000 7f7fe5085700 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecaca0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fdffff700 1 -- 172.21.15.52:0/3144607841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 msgr2=0x562ecaca0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecaca0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fdffff700 1 -- 172.21.15.52:0/3144607841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 msgr2=0x562ecaca6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fdffff700 1 -- 172.21.15.52:0/3144607841 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ecab408d0 con 0x562ecaca6880 2024-06-02T23:22:22.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecaca0260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7fe01036f0 tx=0x7f7fcc010730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d99e31a2db349b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fcaffd700 1 -- 172.21.15.52:0/3144607841 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc01c480 con 0x562ecaca6880 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fcaffd700 1 -- 172.21.15.52:0/3144607841 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fcc01cad0 con 0x562ecaca6880 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.110+0000 7f7fcaffd700 1 -- 172.21.15.52:0/3144607841 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011b90 con 0x562ecaca6880 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3144607841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 msgr2=0x562ecaca0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecaca0260 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7fe01036f0 tx=0x7f7fcc010730 comp rx=0 tx=0).stop 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3144607841 shutdown_connections 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecaca0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3144607841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecaca0260 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3144607841 >> 172.21.15.52:0/3144607841 conn(0x562ecabcd080 msgr2=0x562ecac9b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.111+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3144607841 shutdown_connections 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.112+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3144607841 wait complete. 2024-06-02T23:22:22.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.112+0000 7f7ff136bb80 1 -- start start 2024-06-02T23:22:22.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.112+0000 7f7ff136bb80 1 Processor -- start 2024-06-02T23:22:22.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecacac5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca6880 0x562ecaca9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562ecab91fc0 con 0x562ecaca5f60 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562ecabed810 con 0x562ecaca6880 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562ecacad260 con 0x562ecaca07a0 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7ff136bb80 10 client.0 _ll_register_callbacks cb 0x562ecab875b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7fdffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecacac5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecacac5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38584/0 (socket says 172.21.15.52:38584) 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7fdffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40146/0 (socket says 172.21.15.52:40146) 2024-06-02T23:22:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7fdffff700 1 -- 172.21.15.52:0/3980170960 learned_addr learned my addr 172.21.15.52:0/3980170960 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.113+0000 7f7fe5085700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca6880 0x562ecaca9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe4884700 1 -- 172.21.15.52:0/3980170960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca6880 msgr2=0x562ecaca9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe4884700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca6880 0x562ecaca9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe4884700 1 -- 172.21.15.52:0/3980170960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 msgr2=0x562ecaca98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe4884700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe4884700 1 -- 172.21.15.52:0/3980170960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ecabb7750 con 0x562ecaca07a0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe5085700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca6880 0x562ecaca9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fe4884700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecacac5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4005cd0 tx=0x7f7fd4009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7384b89c4f1439d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fbffff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fbffff700 1 -- 172.21.15.52:0/3980170960 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400f870 con 0x562ecaca07a0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fbffff700 1 -- 172.21.15.52:0/3980170960 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fd400fa10 con 0x562ecaca07a0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc01cad0 con 0x562ecaca07a0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.114+0000 7f7fbffff700 1 -- 172.21.15.52:0/3980170960 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7fd400fc10 con 0x562ecaca07a0 2024-06-02T23:22:22.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.115+0000 7f7ff136bb80 10 client.27601 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.115+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ecacb79b0 con 0x562ecaca07a0 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.115+0000 7f7fbffff700 1 -- 172.21.15.52:0/3980170960 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fcc01cad0 con 0x562ecaca07a0 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7fbffff700 1 -- 172.21.15.52:0/3980170960 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7fd4037400 con 0x562ecaca07a0 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7ff136bb80 10 client.27601 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7ff136bb80 10 client.27601 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7ff136bb80 20 client.27601 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecacba440 0x562ecacbc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7ff136bb80 4 client.27601 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ecabfed50 con 0x562ecacba440 2024-06-02T23:22:22.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.116+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecacba440 0x562ecacbc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.117+0000 7f7fdffff700 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecacba440 0x562ecacbc8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ec6591b6cef08885 server_cookie=e50455676af821d2 in_seq=0 out_seq=0 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.117+0000 7f7fbffff700 10 client.27601 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.117+0000 7f7fbffff700 1 -- 172.21.15.52:0/3980170960 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562ecabfed50 con 0x562ecacba440 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.117+0000 7f7fbffff700 10 client.27601 handle_command_reply: tid=0 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.117+0000 7f7ff136bb80 1 client.27601 shutdown 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 msgr2=0x562ecacac5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecacac5d0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4005cd0 tx=0x7f7fd4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 shutdown_connections 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca6880 0x562ecaca9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecacba440 0x562ecacbc8a0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecacac5d0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/3980170960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecaca98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 >> 172.21.15.52:0/3980170960 conn(0x562ecabcd080 msgr2=0x562ecac9a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7fbffff700 0 client.27601 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 shutdown_connections 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 1 -- 172.21.15.52:0/3980170960 wait complete. 2024-06-02T23:22:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.118+0000 7f7ff136bb80 20 client.27601 trim_cache size 0 max 16384 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.135+0000 7f7ff136bb80 1 Processor -- start 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.135+0000 7f7ff136bb80 1 -- start start 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecaca7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 0x562ecac9ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 0x562ecac9b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01aecf0 con 0x562ecaca6880 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b0830 con 0x562ecaca5f60 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01af3c0 con 0x562ecaca07a0 2024-06-02T23:22:22.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 0x562ecac9ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fe5085700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 0x562ecac9b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 0x562ecac9ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48344/0 (socket says 172.21.15.52:48344) 2024-06-02T23:22:22.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fe5085700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 0x562ecac9b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40152/0 (socket says 172.21.15.52:40152) 2024-06-02T23:22:22.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 -- 172.21.15.52:0/1990506617 learned_addr learned my addr 172.21.15.52:0/1990506617 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:22.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecaca7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 -- 172.21.15.52:0/1990506617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 msgr2=0x562ecaca7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecaca7370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 -- 172.21.15.52:0/1990506617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 msgr2=0x562ecac9b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 0x562ecac9b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 -- 172.21.15.52:0/1990506617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ecaca78b0 con 0x562ecaca5f60 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fe5085700 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 0x562ecac9b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.136+0000 7f7fdffff700 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 0x562ecac9ae80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005d90 tx=0x7f7fcc005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a9a240f190251580 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.137+0000 7f7fca7fc700 1 -- 172.21.15.52:0/1990506617 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc004220 con 0x562ecaca5f60 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.137+0000 7f7fca7fc700 1 -- 172.21.15.52:0/1990506617 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fcc0043c0 con 0x562ecaca5f60 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.137+0000 7f7fca7fc700 1 -- 172.21.15.52:0/1990506617 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc004c40 con 0x562ecaca5f60 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.137+0000 7f7ff136bb80 1 -- 172.21.15.52:0/1990506617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 msgr2=0x562ecac9ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.137+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 0x562ecac9ae80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc005d90 tx=0x7f7fcc005ce0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 -- 172.21.15.52:0/1990506617 shutdown_connections 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca6880 0x562ecac9b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca07a0 0x562ecaca7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/1990506617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca5f60 0x562ecac9ae80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 -- 172.21.15.52:0/1990506617 >> 172.21.15.52:0/1990506617 conn(0x562ecabcd080 msgr2=0x562ecaca6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:22.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 -- 172.21.15.52:0/1990506617 shutdown_connections 2024-06-02T23:22:22.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 -- 172.21.15.52:0/1990506617 wait complete. 2024-06-02T23:22:22.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 -- start start 2024-06-02T23:22:22.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.138+0000 7f7ff136bb80 1 Processor -- start 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecacc2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecacb2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecacb2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b0160 con 0x562ecaca5f60 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01afa90 con 0x562ecaca07a0 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01accb0 con 0x562ecaca6880 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7ff136bb80 10 client.0 _ll_register_callbacks cb 0x562ecab875b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:22.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecacc2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecacc2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48348/0 (socket says 172.21.15.52:48348) 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 -- 172.21.15.52:0/2964372155 learned_addr learned my addr 172.21.15.52:0/2964372155 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe5085700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecacb2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fdffff700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecacb2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 -- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 msgr2=0x562ecacb2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecacb2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 -- 172.21.15.52:0/2964372155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 msgr2=0x562ecacb2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecacb2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 -- 172.21.15.52:0/2964372155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ecacb30a0 con 0x562ecaca07a0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fe4884700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecacc2660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009750 tx=0x7f7fd4037d40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93a94ac2d033c860 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.139+0000 7f7fbffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7fbffff700 1 -- 172.21.15.52:0/2964372155 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4049060 con 0x562ecaca07a0 2024-06-02T23:22:22.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fcc0043c0 con 0x562ecaca07a0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7fbffff700 1 -- 172.21.15.52:0/2964372155 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fd403a460 con 0x562ecaca07a0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7fbffff700 1 -- 172.21.15.52:0/2964372155 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7fd4005130 con 0x562ecaca07a0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7ff136bb80 10 client.27860 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x562ecad2c950 con 0x562ecaca07a0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7fbffff700 1 -- 172.21.15.52:0/2964372155 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fcc0043c0 con 0x562ecaca07a0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.140+0000 7f7fbffff700 1 -- 172.21.15.52:0/2964372155 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7fd4043030 con 0x562ecaca07a0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 10 client.27860 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 10 client.27860 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 20 client.27860 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecad2d000 0x562ecad2f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 4 client.27860 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562ecabf92d0 con 0x562ecad2d000 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7fdffff700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecad2d000 0x562ecad2f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:22.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7fdffff700 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecad2d000 0x562ecad2f420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b6f9628901855f7a server_cookie=b7acb079f63c3adc in_seq=0 out_seq=0 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7fbffff700 10 client.27860 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7fbffff700 1 -- 172.21.15.52:0/2964372155 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562ecabf92d0 con 0x562ecad2d000 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7fbffff700 10 client.27860 handle_command_reply: tid=0 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.141+0000 7f7ff136bb80 1 client.27860 shutdown 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 msgr2=0x562ecacc2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecacc2660 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009750 tx=0x7f7fd4037d40 comp rx=0 tx=0).stop 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 shutdown_connections 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x562ecad2d000 0x562ecad2f420 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562ecaca6880 0x562ecacb2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7fbffff700 0 client.27860 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562ecaca07a0 0x562ecacc2660 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 --2- 172.21.15.52:0/2964372155 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562ecaca5f60 0x562ecacb2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 >> 172.21.15.52:0/2964372155 conn(0x562ecabcd080 msgr2=0x562ecacab100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:22.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 shutdown_connections 2024-06-02T23:22:22.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 1 -- 172.21.15.52:0/2964372155 wait complete. 2024-06-02T23:22:22.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7ff136bb80 20 client.27860 trim_cache size 0 max 16384 2024-06-02T23:22:22.146 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:22:22.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c20 msgr2=0x7f7fd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c20 0x7f7fd00a50e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4006010 tx=0x7f7fd400b630 comp rx=0 tx=0).stop 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0108510 msgr2=0x7f7fe01026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.142+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0108510 0x7f7fe01026e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800efc0 tx=0x7f7fd800c3a0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 shutdown_connections 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fd00a2c20 0x7f7fd00a50e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe0108510 0x7f7fe01026e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fe0100a80 0x7f7fe0101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 --2- 172.21.15.52:0/3466421873 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fe0101440 0x7f7fe01021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 >> 172.21.15.52:0/3466421873 conn(0x7f7fe0077a00 msgr2=0x7f7fe00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 shutdown_connections 2024-06-02T23:22:22.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:22.143+0000 7f7fe6ae8700 1 -- 172.21.15.52:0/3466421873 wait complete. 2024-06-02T23:22:22.148 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:21 smithi174 ceph-mon[35994]: pgmap v745: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 1.3 MiB/s wr, 147 op/s 2024-06-02T23:22:22.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 206093 1.89 MB/sec execute 451 sec latency 57.201 ms 2024-06-02T23:22:22.969 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:22:23.240 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:23.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 206381 1.89 MB/sec execute 452 sec latency 13.706 ms 2024-06-02T23:22:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:23 smithi123 ceph-mon[35977]: pgmap v746: 97 pgs: 97 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-02T23:22:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:23 smithi174 ceph-mon[35994]: pgmap v746: 97 pgs: 97 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-02T23:22:24.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:23 smithi052 ceph-mon[32340]: pgmap v746: 97 pgs: 97 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 298 KiB/s rd, 1.3 MiB/s wr, 172 op/s 2024-06-02T23:22:24.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 206905 1.89 MB/sec execute 453 sec latency 14.912 ms 2024-06-02T23:22:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 -- 172.21.15.52:0/4166850789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25581073f0 msgr2=0x7f25581077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 --2- 172.21.15.52:0/4166850789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25581073f0 0x7f25581077d0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f2548009990 tx=0x7f2548009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 -- 172.21.15.52:0/4166850789 shutdown_connections 2024-06-02T23:22:25.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 --2- 172.21.15.52:0/4166850789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2558101d40 0x7f255810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 --2- 172.21.15.52:0/4166850789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25581073f0 0x7f25581077d0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 --2- 172.21.15.52:0/4166850789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 0x7f2558101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 -- 172.21.15.52:0/4166850789 >> 172.21.15.52:0/4166850789 conn(0x7f2558078300 msgr2=0x7f2558078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 -- 172.21.15.52:0/4166850789 shutdown_connections 2024-06-02T23:22:25.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.162+0000 7f255ec10700 1 -- 172.21.15.52:0/4166850789 wait complete. 2024-06-02T23:22:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 Processor -- start 2024-06-02T23:22:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 -- start start 2024-06-02T23:22:25.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 0x7f255819d800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2558101d40 0x7f255819dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25581073f0 0x7f25581979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2558111560 con 0x7f2558101d40 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25581113e0 con 0x7f2558101380 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255ec10700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25581116e0 con 0x7f25581073f0 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2558101d40 0x7f255819dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2558101d40 0x7f255819dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56892/0 (socket says 172.21.15.52:56892) 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f2557fff700 1 -- 172.21.15.52:0/3638175430 learned_addr learned my addr 172.21.15.52:0/3638175430 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 0x7f255819d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.163+0000 7f255d1ad700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25581073f0 0x7f25581979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255c9ac700 1 -- 172.21.15.52:0/3638175430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25581073f0 msgr2=0x7f25581979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25581073f0 0x7f25581979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255c9ac700 1 -- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2558101d40 msgr2=0x7f255819dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2558101d40 0x7f255819dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255c9ac700 1 -- 172.21.15.52:0/3638175430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2558198120 con 0x7f2558101380 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 0x7f255819d800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c00 tx=0x7f25480047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f2555ffb700 1 -- 172.21.15.52:0/3638175430 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548026070 con 0x7f2558101380 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f2555ffb700 1 -- 172.21.15.52:0/3638175430 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2548021410 con 0x7f2558101380 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25581983b0 con 0x7f2558101380 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f2555ffb700 1 -- 172.21.15.52:0/3638175430 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254800b610 con 0x7f2558101380 2024-06-02T23:22:25.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.164+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f255806aa00 con 0x7f2558101380 2024-06-02T23:22:25.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.167+0000 7f2555ffb700 1 -- 172.21.15.52:0/3638175430 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2548029070 con 0x7f2558101380 2024-06-02T23:22:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.168+0000 7f2555ffb700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25400a2c20 0x7f25400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.168+0000 7f2555ffb700 1 -- 172.21.15.52:0/3638175430 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f25480cfca0 con 0x7f2558101380 2024-06-02T23:22:25.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.168+0000 7f2557fff700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25400a2c20 0x7f25400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.169+0000 7f2557fff700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25400a2c20 0x7f25400a50e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2558078a30 tx=0x7f254c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.169+0000 7f2569493b80 1 Processor -- start 2024-06-02T23:22:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.169+0000 7f2569493b80 1 -- start start 2024-06-02T23:22:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.169+0000 7f2569493b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda00340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f2569493b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6ed9fa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f2569493b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6ed9fabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f2569493b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f6ed8ecf60 con 0x55f6eda00880 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f2569493b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6ed94b850 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f2569493b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6ed8b3460 con 0x55f6ed9fff60 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda00340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6ed9fa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda00340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:57908/0 (socket says 172.21.15.52:57908) 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 -- 172.21.15.52:0/3046808834 learned_addr learned my addr 172.21.15.52:0/3046808834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255d1ad700 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6ed9fabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 -- 172.21.15.52:0/3046808834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 msgr2=0x55f6ed9fabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6ed9fabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 -- 172.21.15.52:0/3046808834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 msgr2=0x55f6ed9fa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6ed9fa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 -- 172.21.15.52:0/3046808834 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6ed89a8d0 con 0x55f6ed9fff60 2024-06-02T23:22:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255d1ad700 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6ed9fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:25.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda00340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2548098a20 tx=0x7f254809c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbdd0211a5f54138 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.170+0000 7f253effd700 1 -- 172.21.15.52:0/3046808834 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25480bc040 con 0x55f6ed9fff60 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f253effd700 1 -- 172.21.15.52:0/3046808834 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f254809c5c0 con 0x55f6ed9fff60 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f253effd700 1 -- 172.21.15.52:0/3046808834 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2548021930 con 0x55f6ed9fff60 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f2569493b80 1 -- 172.21.15.52:0/3046808834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 msgr2=0x55f6eda00340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f2569493b80 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda00340 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2548098a20 tx=0x7f254809c4b0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f2569493b80 1 -- 172.21.15.52:0/3046808834 shutdown_connections 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f2569493b80 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6ed9fabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f2569493b80 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda00340 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.171+0000 7f2569493b80 1 --2- 172.21.15.52:0/3046808834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6ed9fa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 -- 172.21.15.52:0/3046808834 >> 172.21.15.52:0/3046808834 conn(0x55f6ed927080 msgr2=0x55f6ed9f5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 -- 172.21.15.52:0/3046808834 shutdown_connections 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 -- 172.21.15.52:0/3046808834 wait complete. 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 -- start start 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 Processor -- start 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda06680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.172+0000 7f2569493b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda03980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2569493b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6eda03ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2569493b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f6ed8ebfc0 con 0x55f6eda00880 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda06680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2569493b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6ed947810 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2569493b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6eda07010 con 0x55f6ed9fff60 2024-06-02T23:22:25.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda06680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39380/0 (socket says 172.21.15.52:39380) 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 -- 172.21.15.52:0/2753296381 learned_addr learned my addr 172.21.15.52:0/2753296381 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2569493b80 10 client.0 _ll_register_callbacks cb 0x55f6ed8e15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255d1ad700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6eda03ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 -- 172.21.15.52:0/2753296381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 msgr2=0x55f6eda03980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2557fff700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda03980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda03980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 -- 172.21.15.52:0/2753296381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 msgr2=0x55f6eda03ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6eda03ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 -- 172.21.15.52:0/2753296381 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6ed911750 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2557fff700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda03980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f255c9ac700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda06680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f254800b2e0 tx=0x7f25480a0970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b91f56900c5d80b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2537fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.173+0000 7f2537fff700 1 -- 172.21.15.52:0/2753296381 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25480ba060 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2537fff700 1 -- 172.21.15.52:0/2753296381 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f254809c820 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f254809c5c0 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2537fff700 1 -- 172.21.15.52:0/2753296381 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f25480b4070 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 10 client.27872 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f6eda19f80 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2537fff700 1 -- 172.21.15.52:0/2753296381 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f254809c5c0 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2537fff700 1 -- 172.21.15.52:0/2753296381 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f254809bb30 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 10 client.27872 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 10 client.27872 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 20 client.27872 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda1c450 0x55f6eda1e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 4 client.27872 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6ed958d50 con 0x55f6eda1c450 2024-06-02T23:22:25.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.174+0000 7f2557fff700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda1c450 0x55f6eda1e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.175+0000 7f2537fff700 10 client.27872 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:22:25.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.175+0000 7f2557fff700 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda1c450 0x55f6eda1e840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=50aacaed373748d9 server_cookie=1f0f6810dcb5296d in_seq=0 out_seq=0 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.175+0000 7f2537fff700 1 -- 172.21.15.52:0/2753296381 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f6ed958d50 con 0x55f6eda1c450 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.175+0000 7f2537fff700 10 client.27872 handle_command_reply: tid=0 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 client.27872 shutdown 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 msgr2=0x55f6eda06680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda06680 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f254800b2e0 tx=0x7f25480a0970 comp rx=0 tx=0).stop 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 shutdown_connections 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda1c450 0x55f6eda1e840 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6eda00880 0x55f6eda03ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda06680 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 --2- 172.21.15.52:0/2753296381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fff60 0x55f6eda03980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 >> 172.21.15.52:0/2753296381 conn(0x55f6ed927080 msgr2=0x55f6ed9f4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:25.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2537fff700 0 client.27872 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:22:25.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 shutdown_connections 2024-06-02T23:22:25.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 1 -- 172.21.15.52:0/2753296381 wait complete. 2024-06-02T23:22:25.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.176+0000 7f2569493b80 20 client.27872 trim_cache size 0 max 16384 2024-06-02T23:22:25.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 Processor -- start 2024-06-02T23:22:25.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 -- start start 2024-06-02T23:22:25.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda02900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6eda00880 0x55f6eda061c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25581aa7a0 con 0x55f6ed9fff60 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25581ac2e0 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2569493b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25581aae70 con 0x55f6eda00880 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56926/0 (socket says 172.21.15.52:56926) 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda02900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 -- 172.21.15.52:0/3714811984 learned_addr learned my addr 172.21.15.52:0/3714811984 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f255d1ad700 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6eda00880 0x55f6eda061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 -- 172.21.15.52:0/3714811984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6eda00880 msgr2=0x55f6eda061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6eda00880 0x55f6eda061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 -- 172.21.15.52:0/3714811984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 msgr2=0x55f6eda02900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda02900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 -- 172.21.15.52:0/3714811984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6eda1ed10 con 0x55f6ed9fff60 2024-06-02T23:22:25.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f255c9ac700 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda02900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.196+0000 7f2557fff700 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda014f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f254c013010 tx=0x7f254c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d81befd8c3d9fc1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f253e7fc700 1 -- 172.21.15.52:0/3714811984 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c02f030 con 0x55f6ed9fff60 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f253e7fc700 1 -- 172.21.15.52:0/3714811984 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f254c01e4d0 con 0x55f6ed9fff60 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f253e7fc700 1 -- 172.21.15.52:0/3714811984 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c029620 con 0x55f6ed9fff60 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f2569493b80 1 -- 172.21.15.52:0/3714811984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 msgr2=0x55f6eda014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f2569493b80 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda014f0 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f254c013010 tx=0x7f254c017000 comp rx=0 tx=0).stop 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f2569493b80 1 -- 172.21.15.52:0/3714811984 shutdown_connections 2024-06-02T23:22:25.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f2569493b80 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6eda00880 0x55f6eda061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f2569493b80 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda02900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.197+0000 7f2569493b80 1 --2- 172.21.15.52:0/3714811984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda014f0 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.198+0000 7f2569493b80 1 -- 172.21.15.52:0/3714811984 >> 172.21.15.52:0/3714811984 conn(0x55f6ed927080 msgr2=0x55f6ed9fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.198+0000 7f2569493b80 1 -- 172.21.15.52:0/3714811984 shutdown_connections 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.198+0000 7f2569493b80 1 -- 172.21.15.52:0/3714811984 wait complete. 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.198+0000 7f2569493b80 1 -- start start 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.198+0000 7f2569493b80 1 Processor -- start 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda0c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda0c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6eda00880 0x55f6eda22fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25581abc10 con 0x55f6ed9fff60 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25581ab540 con 0x55f6eda00880 2024-06-02T23:22:25.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25581a8740 con 0x55f6ed9fa7a0 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2569493b80 10 client.0 _ll_register_callbacks cb 0x55f6ed8e15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda0c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda0c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56928/0 (socket says 172.21.15.52:56928) 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 -- 172.21.15.52:0/2189248006 learned_addr learned my addr 172.21.15.52:0/2189248006 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f255d1ad700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6eda00880 0x55f6eda22fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 -- 172.21.15.52:0/2189248006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fa7a0 msgr2=0x55f6eda0c260 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda0c260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 -- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6eda00880 msgr2=0x55f6eda22fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6eda00880 0x55f6eda22fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f2557fff700 1 -- 172.21.15.52:0/2189248006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6eda23500 con 0x55f6ed9fff60 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.199+0000 7f255d1ad700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6eda00880 0x55f6eda22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2557fff700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda0c7a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f254c032000 tx=0x7f254c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3efc173934ebb3c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2537fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:25.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2537fff700 1 -- 172.21.15.52:0/2189248006 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f254c02f050 con 0x55f6ed9fff60 2024-06-02T23:22:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2537fff700 1 -- 172.21.15.52:0/2189248006 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f254c005510 con 0x55f6ed9fff60 2024-06-02T23:22:25.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f254c01e4d0 con 0x55f6ed9fff60 2024-06-02T23:22:25.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2537fff700 1 -- 172.21.15.52:0/2189248006 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f254c004750 con 0x55f6ed9fff60 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2569493b80 10 client.18420 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55f6eda86680 con 0x55f6ed9fff60 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.200+0000 7f2537fff700 1 -- 172.21.15.52:0/2189248006 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f254c01e4d0 con 0x55f6ed9fff60 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2537fff700 1 -- 172.21.15.52:0/2189248006 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f254c02b8d0 con 0x55f6ed9fff60 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 10 client.18420 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 10 client.18420 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 20 client.18420 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda86cb0 0x55f6eda890d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 4 client.18420 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f6ed9532d0 con 0x55f6eda86cb0 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f255c9ac700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda86cb0 0x55f6eda890d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2537fff700 10 client.18420 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f255c9ac700 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda86cb0 0x55f6eda890d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f036706bcb478bc1 server_cookie=79f598d76e363a23 in_seq=0 out_seq=0 2024-06-02T23:22:25.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2537fff700 1 -- 172.21.15.52:0/2189248006 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f6ed9532d0 con 0x55f6eda86cb0 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2537fff700 10 client.18420 handle_command_reply: tid=0 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.201+0000 7f2569493b80 1 client.18420 shutdown 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 msgr2=0x55f6eda0c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda0c7a0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f254c032000 tx=0x7f254c011bf0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 shutdown_connections 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55f6eda86cb0 0x55f6eda890d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6eda00880 0x55f6eda22fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6ed9fa7a0 0x55f6eda0c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 --2- 172.21.15.52:0/2189248006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f6ed9fff60 0x55f6eda0c7a0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 >> 172.21.15.52:0/2189248006 conn(0x55f6ed927080 msgr2=0x55f6eda01370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2537fff700 0 client.18420 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 shutdown_connections 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 1 -- 172.21.15.52:0/2189248006 wait complete. 2024-06-02T23:22:25.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.202+0000 7f2569493b80 20 client.18420 trim_cache size 0 max 16384 2024-06-02T23:22:25.206 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:22:25.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25400a2c20 msgr2=0x7f25400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25400a2c20 0x7f25400a50e0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f2558078a30 tx=0x7f254c009260 comp rx=0 tx=0).stop 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 msgr2=0x7f255819d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 0x7f255819d800 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2548000c00 tx=0x7f25480047b0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 shutdown_connections 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25400a2c20 0x7f25400a50e0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f25581073f0 0x7f25581979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2558101380 0x7f255819d800 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 --2- 172.21.15.52:0/3638175430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2558101d40 0x7f255819dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 >> 172.21.15.52:0/3638175430 conn(0x7f2558078300 msgr2=0x7f25581001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 shutdown_connections 2024-06-02T23:22:25.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:25.203+0000 7f255ec10700 1 -- 172.21.15.52:0/3638175430 wait complete. 2024-06-02T23:22:25.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 207497 1.89 MB/sec execute 454 sec latency 15.377 ms 2024-06-02T23:22:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:25 smithi123 ceph-mon[35977]: pgmap v747: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-06-02T23:22:26.048 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:25 smithi052 ceph-mon[32340]: pgmap v747: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-06-02T23:22:26.080 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:22:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:25 smithi174 ceph-mon[35994]: pgmap v747: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 1.1 MiB/s wr, 142 op/s 2024-06-02T23:22:26.354 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:26.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 208094 1.89 MB/sec execute 455 sec latency 15.458 ms 2024-06-02T23:22:27.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 208719 1.90 MB/sec execute 456 sec latency 14.797 ms 2024-06-02T23:22:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:27 smithi052 ceph-mon[32340]: pgmap v748: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-02T23:22:28.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:27 smithi123 ceph-mon[35977]: pgmap v748: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-02T23:22:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:27 smithi174 ceph-mon[35994]: pgmap v748: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 1.5 MiB/s wr, 181 op/s 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 -- 172.21.15.52:0/2657345100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101390 msgr2=0x7fa6d41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 --2- 172.21.15.52:0/2657345100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101390 0x7fa6d41017f0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009990 tx=0x7fa6c8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 -- 172.21.15.52:0/2657345100 shutdown_connections 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 --2- 172.21.15.52:0/2657345100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101d30 0x7fa6d410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 --2- 172.21.15.52:0/2657345100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6d41073d0 0x7fa6d41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 --2- 172.21.15.52:0/2657345100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101390 0x7fa6d41017f0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.219+0000 7fa6d9352700 1 -- 172.21.15.52:0/2657345100 >> 172.21.15.52:0/2657345100 conn(0x7fa6d40782c0 msgr2=0x7fa6d40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.220+0000 7fa6d9352700 1 -- 172.21.15.52:0/2657345100 shutdown_connections 2024-06-02T23:22:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.220+0000 7fa6d9352700 1 -- 172.21.15.52:0/2657345100 wait complete. 2024-06-02T23:22:28.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 Processor -- start 2024-06-02T23:22:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 -- start start 2024-06-02T23:22:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 0x7fa6d4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101d30 0x7fa6d4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6d41073d0 0x7fa6d4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.221 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41114e0 con 0x7fa6d4101d30 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4111360 con 0x7fa6d4101390 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.221+0000 7fa6d9352700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d4111660 con 0x7fa6d41073d0 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d27fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101d30 0x7fa6d4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 0x7fa6d4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 0x7fa6d4103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39414/0 (socket says 172.21.15.52:39414) 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/4155941056 learned_addr learned my addr 172.21.15.52:0/4155941056 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d37fe700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6d41073d0 0x7fa6d4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/4155941056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6d41073d0 msgr2=0x7fa6d4103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6d41073d0 0x7fa6d4103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101d30 msgr2=0x7fa6d4103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101d30 0x7fa6d4103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/4155941056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6d41a7080 con 0x7fa6d4101390 2024-06-02T23:22:28.222 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 0x7fa6d4103040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400e9c0 tx=0x7fa6c400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:28.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.222+0000 7fa6bbfff700 1 -- 172.21.15.52:0/4155941056 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c400cbd0 con 0x7fa6d4101390 2024-06-02T23:22:28.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.223+0000 7fa6bbfff700 1 -- 172.21.15.52:0/4155941056 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c400ee60 con 0x7fa6d4101390 2024-06-02T23:22:28.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.223+0000 7fa6bbfff700 1 -- 172.21.15.52:0/4155941056 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40055b0 con 0x7fa6d4101390 2024-06-02T23:22:28.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.223+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6d41a7370 con 0x7fa6d4101390 2024-06-02T23:22:28.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.223+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6d41a77d0 con 0x7fa6d4101390 2024-06-02T23:22:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.226+0000 7fa6bbfff700 1 -- 172.21.15.52:0/4155941056 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa6c4004750 con 0x7fa6d4101390 2024-06-02T23:22:28.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.226+0000 7fa6bbfff700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6bc0a2ca0 0x7fa6bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6bc0a2ca0 0x7fa6bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6bbfff700 1 -- 172.21.15.52:0/4155941056 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa6c40cf190 con 0x7fa6d4101390 2024-06-02T23:22:28.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 Processor -- start 2024-06-02T23:22:28.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 -- start start 2024-06-02T23:22:28.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6bc0a2ca0 0x7fa6bc0a5160 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800b2e0 tx=0x7fa6c8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:28.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 0x563c59b03340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b03880 0x563c59afd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59afd7a0 0x563c59afdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563c599eff60 con 0x563c59b02f60 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c59a4e850 con 0x563c59b03880 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c599b6460 con 0x563c59afd7a0 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 0x563c59b03340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 0x563c59b03340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56972/0 (socket says 172.21.15.52:56972) 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/3241779408 learned_addr learned my addr 172.21.15.52:0/3241779408 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.227+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b03880 0x563c59afd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d37fe700 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59afd7a0 0x563c59afdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d27fc700 1 -- 172.21.15.52:0/3241779408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59afd7a0 msgr2=0x563c59afdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59afd7a0 0x563c59afdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d27fc700 1 -- 172.21.15.52:0/3241779408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 msgr2=0x563c59b03340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 0x563c59b03340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d27fc700 1 -- 172.21.15.52:0/3241779408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c5999d8d0 con 0x563c59b03880 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 0x563c59b03340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b03880 0x563c59afd260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80059d0 tx=0x7fa6c801f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71a498f813dcdc80 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6b97fa700 1 -- 172.21.15.52:0/3241779408 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800bc40 con 0x563c59b03880 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6b97fa700 1 -- 172.21.15.52:0/3241779408 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c800bde0 con 0x563c59b03880 2024-06-02T23:22:28.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.228+0000 7fa6b97fa700 1 -- 172.21.15.52:0/3241779408 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800fa20 con 0x563c59b03880 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3241779408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b03880 msgr2=0x563c59afd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b03880 0x563c59afd260 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa6c80059d0 tx=0x7fa6c801f000 comp rx=0 tx=0).stop 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3241779408 shutdown_connections 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59afd7a0 0x563c59afdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b02f60 0x563c59b03340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3241779408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b03880 0x563c59afd260 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3241779408 >> 172.21.15.52:0/3241779408 conn(0x563c59a2a080 msgr2=0x563c59af8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3241779408 shutdown_connections 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3241779408 wait complete. 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.229+0000 7fa6e3bd5b80 1 -- start start 2024-06-02T23:22:28.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 Processor -- start 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b09630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b06930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b06e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563c599eefc0 con 0x563c59b03880 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563c59a4a810 con 0x563c59afd7a0 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563c59b0a200 con 0x563c59b02f60 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6e3bd5b80 10 client.0 _ll_register_callbacks cb 0x563c599e45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b09630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b06e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b09630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39436/0 (socket says 172.21.15.52:39436) 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b06e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56982/0 (socket says 172.21.15.52:56982) 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/3721555337 learned_addr learned my addr 172.21.15.52:0/3721555337 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:28.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b06930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 msgr2=0x563c59b06930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b06930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/3721555337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 msgr2=0x563c59b06e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b06e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.230+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/3721555337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c59a14750 con 0x563c59afd7a0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b09630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40ccd10 tx=0x7fa6c409b400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce5054a4be8c092a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/3721555337 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40b3070 con 0x563c59afd7a0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/3721555337 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa6c4016df0 con 0x563c59afd7a0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c800bde0 con 0x563c59afd7a0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/3721555337 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa6c407f050 con 0x563c59afd7a0 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 10 client.27890 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:28.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563c59b14a20 con 0x563c59afd7a0 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/3721555337 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c800bde0 con 0x563c59afd7a0 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/3721555337 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa6c40ba260 con 0x563c59afd7a0 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 10 client.27890 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 10 client.27890 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 20 client.27890 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b19450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 4 client.27890 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.231+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c59a5bd50 con 0x563c59b17060 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.232+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b19450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.232+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b19450 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c79dec19837aa91e server_cookie=d71724d5848d5355 in_seq=0 out_seq=0 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.232+0000 7fa6b6ff5700 10 client.27890 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/3721555337 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563c59a5bd50 con 0x563c59b17060 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6b6ff5700 10 client.27890 handle_command_reply: tid=0 2024-06-02T23:22:28.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 client.27890 shutdown 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 msgr2=0x563c59b09630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b09630 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40ccd10 tx=0x7fa6c409b400 comp rx=0 tx=0).stop 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 shutdown_connections 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b19450 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b06e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b09630 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6b6ff5700 0 client.27890 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/3721555337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b06930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 >> 172.21.15.52:0/3721555337 conn(0x563c59a2a080 msgr2=0x563c59af7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 shutdown_connections 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/3721555337 wait complete. 2024-06-02T23:22:28.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.233+0000 7fa6e3bd5b80 20 client.27890 trim_cache size 0 max 16384 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 Processor -- start 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 -- start start 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59afd7a0 0x563c59afd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 0x563c59b043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b03880 0x563c59af7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41aeb60 con 0x563c59afd7a0 2024-06-02T23:22:28.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41b06a0 con 0x563c59b02f60 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41af230 con 0x563c59b03880 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59afd7a0 0x563c59afd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 0x563c59b043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 0x563c59b043b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39442/0 (socket says 172.21.15.52:39442) 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59afd7a0 0x563c59afd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56988/0 (socket says 172.21.15.52:56988) 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 -- 172.21.15.52:0/314500344 learned_addr learned my addr 172.21.15.52:0/314500344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d37fe700 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b03880 0x563c59af7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 -- 172.21.15.52:0/314500344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b03880 msgr2=0x563c59af7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b03880 0x563c59af7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 -- 172.21.15.52:0/314500344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59afd7a0 msgr2=0x563c59afd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59afd7a0 0x563c59afd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.250+0000 7fa6d27fc700 1 -- 172.21.15.52:0/314500344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c59b07870 con 0x563c59b02f60 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.251+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 0x563c59b043b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8000c00 tx=0x7fa6c800f600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99c923816151f139 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:28.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.251+0000 7fa6b8ff9700 1 -- 172.21.15.52:0/314500344 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c800f670 con 0x563c59b02f60 2024-06-02T23:22:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.251+0000 7fa6b8ff9700 1 -- 172.21.15.52:0/314500344 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c8003680 con 0x563c59b02f60 2024-06-02T23:22:28.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.251+0000 7fa6b8ff9700 1 -- 172.21.15.52:0/314500344 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c80306e0 con 0x563c59b02f60 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/314500344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 msgr2=0x563c59b043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 0x563c59b043b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8000c00 tx=0x7fa6c800f600 comp rx=0 tx=0).stop 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/314500344 shutdown_connections 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b03880 0x563c59af7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59afd7a0 0x563c59afd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/314500344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59b02f60 0x563c59b043b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/314500344 >> 172.21.15.52:0/314500344 conn(0x563c59a2a080 msgr2=0x563c59b03ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:28.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/314500344 shutdown_connections 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/314500344 wait complete. 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 -- start start 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.252+0000 7fa6e3bd5b80 1 Processor -- start 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b0f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b0fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b1e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41affd0 con 0x563c59b03880 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41af900 con 0x563c59afd7a0 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6d41acb20 con 0x563c59b02f60 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6e3bd5b80 10 client.0 _ll_register_callbacks cb 0x563c599e45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b1e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b1e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57004/0 (socket says 172.21.15.52:57004) 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d37fe700 1 -- 172.21.15.52:0/386749522 learned_addr learned my addr 172.21.15.52:0/386749522 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:28.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b0f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b0fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 msgr2=0x563c59b0fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b0fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/386749522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 msgr2=0x563c59b1e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b1e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 -- 172.21.15.52:0/386749522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c59b1e830 con 0x563c59afd7a0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d37fe700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b1e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6d2ffd700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b0f6a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40983f0 tx=0x7fa6c40097b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=792714cd04f2aec6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.253+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/386749522 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c40b3040 con 0x563c59afd7a0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/386749522 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa6c40ba260 con 0x563c59afd7a0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6c8003680 con 0x563c59afd7a0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/386749522 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa6c409b7f0 con 0x563c59afd7a0 2024-06-02T23:22:28.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 10 client.27896 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563c59b81700 con 0x563c59afd7a0 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/386749522 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa6c8003680 con 0x563c59afd7a0 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/386749522 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa6c4097770 con 0x563c59afd7a0 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 10 client.27896 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 10 client.27896 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 20 client.27896 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b83e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 4 client.27896 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563c59a562d0 con 0x563c59b17060 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.254+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b83e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6b6ff5700 10 client.27896 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6d27fc700 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b83e40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=702d3df293c8bae8 server_cookie=4a188c637af0d1dd in_seq=0 out_seq=0 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6b6ff5700 1 -- 172.21.15.52:0/386749522 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563c59a562d0 con 0x563c59b17060 2024-06-02T23:22:28.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6b6ff5700 10 client.27896 handle_command_reply: tid=0 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 client.27896 shutdown 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 msgr2=0x563c59b0f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b0f6a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa6c40983f0 tx=0x7fa6c40097b0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 shutdown_connections 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563c59b17060 0x563c59b83e40 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563c59b03880 0x563c59b1e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563c59afd7a0 0x563c59b0f6a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 --2- 172.21.15.52:0/386749522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563c59b02f60 0x563c59b0fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6b6ff5700 0 client.27896 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 >> 172.21.15.52:0/386749522 conn(0x563c59a2a080 msgr2=0x563c59b08160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 shutdown_connections 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 1 -- 172.21.15.52:0/386749522 wait complete. 2024-06-02T23:22:28.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.255+0000 7fa6e3bd5b80 20 client.27896 trim_cache size 0 max 16384 2024-06-02T23:22:28.260 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:22:28.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6bc0a2ca0 msgr2=0x7fa6bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6bc0a2ca0 0x7fa6bc0a5160 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fa6c800b2e0 tx=0x7fa6c8005960 comp rx=0 tx=0).stop 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 msgr2=0x7fa6d4103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 0x7fa6d4103040 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400e9c0 tx=0x7fa6c400ecd0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 shutdown_connections 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa6bc0a2ca0 0x7fa6bc0a5160 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6d41073d0 0x7fa6d4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6d4101390 0x7fa6d4103040 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 --2- 172.21.15.52:0/4155941056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa6d4101d30 0x7fa6d4103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 >> 172.21.15.52:0/4155941056 conn(0x7fa6d40782c0 msgr2=0x7fa6d406a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 shutdown_connections 2024-06-02T23:22:28.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:28.256+0000 7fa6d9352700 1 -- 172.21.15.52:0/4155941056 wait complete. 2024-06-02T23:22:28.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 209234 1.91 MB/sec execute 457 sec latency 68.222 ms 2024-06-02T23:22:29.188 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:22:29.459 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:29.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 209614 1.91 MB/sec execute 458 sec latency 14.796 ms 2024-06-02T23:22:29.737 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 12.4 2024-06-02T23:22:29.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:22:29.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:29 smithi052 ceph-mon[32340]: pgmap v749: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:22:30.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:29 smithi123 ceph-mon[35977]: pgmap v749: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:22:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:29 smithi174 ceph-mon[35994]: pgmap v749: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 233 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:22:30.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 209887 1.90 MB/sec execute 459 sec latency 14.181 ms 2024-06-02T23:22:30.676 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:30.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:30 smithi052 ceph-mon[32340]: pgmap v750: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:22:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:30 smithi174 ceph-mon[35994]: pgmap v750: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:22:31.256 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:30 smithi123 ceph-mon[35977]: pgmap v750: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.4 MiB/s wr, 162 op/s 2024-06-02T23:22:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 -- 172.21.15.52:0/1431971848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e41024e0 msgr2=0x7fb7e41048d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1431971848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e41024e0 0x7fb7e41048d0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009900 tx=0x7fb7d802a410 comp rx=0 tx=0).stop 2024-06-02T23:22:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 -- 172.21.15.52:0/1431971848 shutdown_connections 2024-06-02T23:22:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1431971848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7e4104e10 0x7fb7e4107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1431971848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 0x7fb7e4109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1431971848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e41024e0 0x7fb7e41048d0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 -- 172.21.15.52:0/1431971848 >> 172.21.15.52:0/1431971848 conn(0x7fb7e40fbe70 msgr2=0x7fb7e40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 -- 172.21.15.52:0/1431971848 shutdown_connections 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.337+0000 7fb7ead41640 1 -- 172.21.15.52:0/1431971848 wait complete. 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 Processor -- start 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 -- start start 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7e41024e0 0x7fb7e4197c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 0x7fb7e4198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 0x7fb7e419f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e410d520 con 0x7fb7e4104e10 2024-06-02T23:22:31.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e410d3a0 con 0x7fb7e4107740 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7ead41640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e410d6a0 con 0x7fb7e41024e0 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 0x7fb7e4198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 0x7fb7e4198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57024/0 (socket says 172.21.15.52:57024) 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 -- 172.21.15.52:0/1396979103 learned_addr learned my addr 172.21.15.52:0/1396979103 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e8add640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7e41024e0 0x7fb7e4197c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e92de640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 0x7fb7e419f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 -- 172.21.15.52:0/1396979103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7e41024e0 msgr2=0x7fb7e4197c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7e41024e0 0x7fb7e4197c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 -- 172.21.15.52:0/1396979103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 msgr2=0x7fb7e419f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 0x7fb7e419f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 -- 172.21.15.52:0/1396979103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7e419f800 con 0x7fb7e4104e10 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e92de640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 0x7fb7e419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.338+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 0x7fb7e4198170 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000d770 tx=0x7fb7d000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.339+0000 7fb7e1ffb640 1 -- 172.21.15.52:0/1396979103 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0017070 con 0x7fb7e4104e10 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.339+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7e419faf0 con 0x7fb7e4104e10 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.339+0000 7fb7e1ffb640 1 -- 172.21.15.52:0/1396979103 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7d0010460 con 0x7fb7e4104e10 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.339+0000 7fb7e1ffb640 1 -- 172.21.15.52:0/1396979103 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000f620 con 0x7fb7e4104e10 2024-06-02T23:22:31.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.339+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7e41a0090 con 0x7fb7e4104e10 2024-06-02T23:22:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.342+0000 7fb7e1ffb640 1 -- 172.21.15.52:0/1396979103 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb7d001a070 con 0x7fb7e4104e10 2024-06-02T23:22:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.342+0000 7fb7eda45740 1 Processor -- start 2024-06-02T23:22:31.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.342+0000 7fb7eda45740 1 -- start start 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48477840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a4847c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a4847cf80 0x563a4847f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7eda45740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563a483d64f0 con 0x563a4847cf80 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7eda45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a48374870 con 0x563a48477420 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7eda45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a483d24b0 con 0x563a48477e10 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48477840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a4847cf80 0x563a4847f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48477840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39492/0 (socket says 172.21.15.52:39492) 2024-06-02T23:22:31.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 -- 172.21.15.52:0/1910650656 learned_addr learned my addr 172.21.15.52:0/1910650656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e8add640 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a4847c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 -- 172.21.15.52:0/1910650656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 msgr2=0x563a4847c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a4847c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 -- 172.21.15.52:0/1910650656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a4847cf80 msgr2=0x563a4847f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a4847cf80 0x563a4847f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e92de640 1 -- 172.21.15.52:0/1910650656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a483909b0 con 0x563a48477420 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e1ffb640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7d40a2c30 0x7fb7d40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a4847cf80 0x563a4847f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.343+0000 7fb7e8add640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7d40a2c30 0x7fb7d40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7e92de640 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48477840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc011810 tx=0x7fb7dc011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f0e7a4562c14a19 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.349 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7e1ffb640 1 -- 172.21.15.52:0/1396979103 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb7d00ce950 con 0x7fb7e4104e10 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7ceffd640 1 -- 172.21.15.52:0/1910650656 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc011310 con 0x563a48477420 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7ceffd640 1 -- 172.21.15.52:0/1910650656 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7dc00f9a0 con 0x563a48477420 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7ceffd640 1 -- 172.21.15.52:0/1910650656 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc005650 con 0x563a48477420 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7eda45740 1 -- 172.21.15.52:0/1910650656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 msgr2=0x563a48477840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7eda45740 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48477840 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc011810 tx=0x7fb7dc011cd0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.344+0000 7fb7e8add640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7d40a2c30 0x7fb7d40a50f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006010 tx=0x7fb7d8000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- 172.21.15.52:0/1910650656 shutdown_connections 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a4847c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a4847cf80 0x563a4847f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 --2- 172.21.15.52:0/1910650656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48477840 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- 172.21.15.52:0/1910650656 >> 172.21.15.52:0/1910650656 conn(0x563a4846e680 msgr2=0x563a4846ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- 172.21.15.52:0/1910650656 shutdown_connections 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- 172.21.15.52:0/1910650656 wait complete. 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- start start 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 Processor -- start 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a48481690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a48481bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a4847cf80 0x563a48488c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563a4847ff30 con 0x563a48477420 2024-06-02T23:22:31.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a4835fae0 con 0x563a4847cf80 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.345+0000 7fb7eda45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563a484800b0 con 0x563a48477e10 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7eda45740 10 client.0 _ll_register_callbacks cb 0x563a483989b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a48481690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a48481690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57038/0 (socket says 172.21.15.52:57038) 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 -- 172.21.15.52:0/3576334020 learned_addr learned my addr 172.21.15.52:0/3576334020 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a48481bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e92de640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a4847cf80 0x563a48488c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 -- 172.21.15.52:0/3576334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 msgr2=0x563a48481bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a48481bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 -- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a4847cf80 msgr2=0x563a48488c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a4847cf80 0x563a48488c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 -- 172.21.15.52:0/3576334020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a480ea210 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7e8add640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a48481690 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb7d802a930 tx=0x7fb7d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c0c63c392a2917e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7c3fff640 1 -- 172.21.15.52:0/3576334020 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d8035040 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7c3fff640 1 -- 172.21.15.52:0/3576334020 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb7d8004240 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.346+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7dc00f9a0 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7c3fff640 1 -- 172.21.15.52:0/3576334020 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb7d8004440 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 10 client.18450 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a481b6620 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7c3fff640 1 -- 172.21.15.52:0/3576334020 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7dc00f9a0 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7c3fff640 1 -- 172.21.15.52:0/3576334020 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d8044070 con 0x563a48477420 2024-06-02T23:22:31.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 10 client.18450 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 10 client.18450 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 20 client.18450 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a4848d510 0x563a4848f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 4 client.18450 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a483e9f10 con 0x563a4848d510 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.347+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a4848d510 0x563a4848f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.348+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a4848d510 0x563a4848f900 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9edd6eaf3bd2cd6 server_cookie=42a7d471a63ac847 in_seq=0 out_seq=0 2024-06-02T23:22:31.354 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.348+0000 7fb7c3fff640 10 client.18450 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.355 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7c3fff640 1 -- 172.21.15.52:0/3576334020 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563a483e9f10 con 0x563a4848d510 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7c3fff640 10 client.18450 handle_command_reply: tid=0 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 client.18450 shutdown 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 msgr2=0x563a48481690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a48481690 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fb7d802a930 tx=0x7fb7d8009260 comp rx=0 tx=0).stop 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 shutdown_connections 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a4848d510 0x563a4848f900 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a48477e10 0x563a48481bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a4847cf80 0x563a48488c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 --2- 172.21.15.52:0/3576334020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a48481690 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7c3fff640 0 client.18450 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.356 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 >> 172.21.15.52:0/3576334020 conn(0x563a4846e680 msgr2=0x563a4846d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 shutdown_connections 2024-06-02T23:22:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 1 -- 172.21.15.52:0/3576334020 wait complete. 2024-06-02T23:22:31.357 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.349+0000 7fb7eda45740 20 client.18450 trim_cache size 0 max 16384 2024-06-02T23:22:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.367+0000 7fb7eda45740 1 Processor -- start 2024-06-02T23:22:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.367+0000 7fb7eda45740 1 -- start start 2024-06-02T23:22:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a484819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477e10 0x563a48488bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a4846cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.368 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7eda45740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41a6ad0 con 0x563a48477420 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7eda45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41a8610 con 0x563a48477e10 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7eda45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41a71a0 con 0x563a4847cf80 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e8add640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a484819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e8add640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a484819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57040/0 (socket says 172.21.15.52:57040) 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a4846cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e8add640 1 -- 172.21.15.52:0/510095839 learned_addr learned my addr 172.21.15.52:0/510095839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477e10 0x563a48488bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 -- 172.21.15.52:0/510095839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477e10 msgr2=0x563a48488bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477e10 0x563a48488bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 -- 172.21.15.52:0/510095839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 msgr2=0x563a484819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a484819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 -- 172.21.15.52:0/510095839 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a4848fc30 con 0x563a4847cf80 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477e10 0x563a48488bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e8add640 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a484819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.368+0000 7fb7e92de640 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a4846cf50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00f790 tx=0x7fb7dc011150 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32c7956b4f27eaf6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7ce7fc640 1 -- 172.21.15.52:0/510095839 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc011340 con 0x563a4847cf80 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7ce7fc640 1 -- 172.21.15.52:0/510095839 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7dc01f040 con 0x563a4847cf80 2024-06-02T23:22:31.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7ce7fc640 1 -- 172.21.15.52:0/510095839 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7dc00fdc0 con 0x563a4847cf80 2024-06-02T23:22:31.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 -- 172.21.15.52:0/510095839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 msgr2=0x563a4846cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a4846cf50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc00f790 tx=0x7fb7dc011150 comp rx=0 tx=0).stop 2024-06-02T23:22:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 -- 172.21.15.52:0/510095839 shutdown_connections 2024-06-02T23:22:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a4846cf50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477e10 0x563a48488bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 --2- 172.21.15.52:0/510095839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477420 0x563a484819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 -- 172.21.15.52:0/510095839 >> 172.21.15.52:0/510095839 conn(0x563a4846e680 msgr2=0x563a484812a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 -- 172.21.15.52:0/510095839 shutdown_connections 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.369+0000 7fb7eda45740 1 -- 172.21.15.52:0/510095839 wait complete. 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 -- start start 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 Processor -- start 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48485a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 0x563a48485fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a484f61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41a7f40 con 0x563a48477e10 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41a7870 con 0x563a48477420 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb7e41a4ab0 con 0x563a4847cf80 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7eda45740 10 client.0 _ll_register_callbacks cb 0x7fb7d8032e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 0x563a48485fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7e8add640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48485a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7e3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 0x563a48485fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57048/0 (socket says 172.21.15.52:57048) 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7e3fff640 1 -- 172.21.15.52:0/572798652 learned_addr learned my addr 172.21.15.52:0/572798652 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.370+0000 7fb7e8add640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48485a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39526/0 (socket says 172.21.15.52:39526) 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e92de640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a484f61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e3fff640 1 -- 172.21.15.52:0/572798652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 msgr2=0x563a484f61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a484f61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e3fff640 1 -- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 msgr2=0x563a48485a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48485a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e3fff640 1 -- 172.21.15.52:0/572798652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a484f6710 con 0x563a48477e10 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e8add640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48485a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e92de640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a484f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:31.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7e3fff640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 0x563a48485fa0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00cc480 tx=0x7fb7d00d4550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0df2c1e3418cce0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7c3fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7c3fff640 1 -- 172.21.15.52:0/572798652 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00b3020 con 0x563a48477e10 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7c3fff640 1 -- 172.21.15.52:0/572798652 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb7d00994e0 con 0x563a48477e10 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7dc01f040 con 0x563a48477e10 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7c3fff640 1 -- 172.21.15.52:0/572798652 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb7d0021040 con 0x563a48477e10 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7eda45740 10 client.18456 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.371+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563a484f79f0 con 0x563a48477e10 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7c3fff640 1 -- 172.21.15.52:0/572798652 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb7dc01f040 con 0x563a48477e10 2024-06-02T23:22:31.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7c3fff640 1 -- 172.21.15.52:0/572798652 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb7d0097850 con 0x563a48477e10 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7eda45740 10 client.18456 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7eda45740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7eda45740 20 client.18456 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7eda45740 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a484f83f0 0x563a484fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7eda45740 4 client.18456 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a483e3c20 con 0x563a484f83f0 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7e8add640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a484f83f0 0x563a484fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7e8add640 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a484f83f0 0x563a484fa810 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82df6d7d299eeeeb server_cookie=a3eeb52d89b95cff in_seq=0 out_seq=0 2024-06-02T23:22:31.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.372+0000 7fb7c3fff640 10 client.18456 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7c3fff640 1 -- 172.21.15.52:0/572798652 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563a483e3c20 con 0x563a484f83f0 2024-06-02T23:22:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7c3fff640 10 client.18456 handle_command_reply: tid=0 2024-06-02T23:22:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 client.18456 shutdown 2024-06-02T23:22:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 msgr2=0x563a48485fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 0x563a48485fa0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fb7d00cc480 tx=0x7fb7d00d4550 comp rx=0 tx=0).stop 2024-06-02T23:22:31.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 shutdown_connections 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563a484f83f0 0x563a484fa810 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563a4847cf80 0x563a484f61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563a48477e10 0x563a48485fa0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7c3fff640 0 client.18456 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 --2- 172.21.15.52:0/572798652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a48477420 0x563a48485a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 >> 172.21.15.52:0/572798652 conn(0x563a4846e680 msgr2=0x563a48471360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 shutdown_connections 2024-06-02T23:22:31.392 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 1 -- 172.21.15.52:0/572798652 wait complete. 2024-06-02T23:22:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.391+0000 7fb7eda45740 20 client.18456 trim_cache size 0 max 16384 2024-06-02T23:22:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:22:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:22:31.393 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:22:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7d40a2c30 msgr2=0x7fb7d40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7d40a2c30 0x7fb7d40a50f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006010 tx=0x7fb7d8000bc0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 msgr2=0x7fb7e4198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 0x7fb7e4198170 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000d770 tx=0x7fb7d000db70 comp rx=0 tx=0).stop 2024-06-02T23:22:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 shutdown_connections 2024-06-02T23:22:31.395 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb7d40a2c30 0x7fb7d40a50f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7e4107740 0x7fb7e419f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb7e4104e10 0x7fb7e4198170 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 --2- 172.21.15.52:0/1396979103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7e41024e0 0x7fb7e4197c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 >> 172.21.15.52:0/1396979103 conn(0x7fb7e40fbe70 msgr2=0x7fb7e4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 shutdown_connections 2024-06-02T23:22:31.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:31.392+0000 7fb7ead41640 1 -- 172.21.15.52:0/1396979103 wait complete. 2024-06-02T23:22:31.642 INFO:tasks.workunit.client.0.smithi052.stdout: 1 210389 1.90 MB/sec execute 460 sec latency 15.841 ms 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 -- 172.21.15.52:0/2875545919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 msgr2=0x7fa5381088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/2875545919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 0x7fa5381088d0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fa528009990 tx=0x7fa528009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 -- 172.21.15.52:0/2875545919 shutdown_connections 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/2875545919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 0x7fa53810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/2875545919 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 0x7fa5381088d0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/2875545919 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 -- 172.21.15.52:0/2875545919 >> 172.21.15.52:0/2875545919 conn(0x7fa538078300 msgr2=0x7fa538078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 -- 172.21.15.52:0/2875545919 shutdown_connections 2024-06-02T23:22:31.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.817+0000 7fa53e4d0700 1 -- 172.21.15.52:0/2875545919 wait complete. 2024-06-02T23:22:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.818+0000 7fa53e4d0700 1 Processor -- start 2024-06-02T23:22:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.818+0000 7fa53e4d0700 1 -- start start 2024-06-02T23:22:31.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53e4d0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53e4d0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 0x7fa53819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53e4d0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 0x7fa538198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa537fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa537fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39544/0 (socket says 172.21.15.52:39544) 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53e4d0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa538112600 con 0x7fa5381084f0 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa537fff700 1 -- 172.21.15.52:0/1262927969 learned_addr learned my addr 172.21.15.52:0/1262927969 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa538112480 con 0x7fa53806aa00 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa538112780 con 0x7fa53806b3c0 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 0x7fa538198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 0x7fa53819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa537fff700 1 -- 172.21.15.52:0/1262927969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 msgr2=0x7fa53819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.819+0000 7fa537fff700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 0x7fa53819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa537fff700 1 -- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 msgr2=0x7fa538198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa537fff700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 0x7fa538198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa537fff700 1 -- 172.21.15.52:0/1262927969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5381990a0 con 0x7fa53806aa00 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 0x7fa53819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa537fff700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53819e780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa5280057d0 tx=0x7fa528005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa5357fa700 1 -- 172.21.15.52:0/1262927969 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528026070 con 0x7fa53806aa00 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa5357fa700 1 -- 172.21.15.52:0/1262927969 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa528021410 con 0x7fa53806aa00 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa538199330 con 0x7fa53806aa00 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa538101380 con 0x7fa53806aa00 2024-06-02T23:22:31.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.820+0000 7fa5357fa700 1 -- 172.21.15.52:0/1262927969 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528017650 con 0x7fa53806aa00 2024-06-02T23:22:31.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.823+0000 7fa5357fa700 1 -- 172.21.15.52:0/1262927969 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa528029070 con 0x7fa53806aa00 2024-06-02T23:22:31.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.824+0000 7fa5357fa700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5200a2ca0 0x7fa5200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.824+0000 7fa5357fa700 1 -- 172.21.15.52:0/1262927969 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa5280cfc90 con 0x7fa53806aa00 2024-06-02T23:22:31.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.824+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5200a2ca0 0x7fa5200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 Processor -- start 2024-06-02T23:22:31.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 -- start start 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5200a2ca0 0x7fa5200a5160 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5380789e0 tx=0x7fa52c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e19a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e14340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561c62d65810 con 0x561c62e19f60 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561c62d0d720 con 0x561c62e13f20 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa548d53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c62d617d0 con 0x561c62e19640 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e139e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57070/0 (socket says 172.21.15.52:57070) 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa5377fe700 1 -- 172.21.15.52:0/1896627738 learned_addr learned my addr 172.21.15.52:0/1896627738 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.825+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e14340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa537fff700 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e19a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa53ca6d700 1 -- 172.21.15.52:0/1896627738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 msgr2=0x561c62e19a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e19a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa53ca6d700 1 -- 172.21.15.52:0/1896627738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 msgr2=0x561c62e139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa53ca6d700 1 -- 172.21.15.52:0/1896627738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c62d03f50 con 0x561c62e13f20 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa537fff700 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e14340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa530010510 tx=0x7fa53000d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4666a1aa5e6607f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa51e7fc700 1 -- 172.21.15.52:0/1896627738 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530004bb0 con 0x561c62e13f20 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa51e7fc700 1 -- 172.21.15.52:0/1896627738 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa530004d50 con 0x561c62e13f20 2024-06-02T23:22:31.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.826+0000 7fa51e7fc700 1 -- 172.21.15.52:0/1896627738 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530007500 con 0x561c62e13f20 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 -- 172.21.15.52:0/1896627738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 msgr2=0x561c62e14340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e14340 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa530010510 tx=0x7fa53000d040 comp rx=0 tx=0).stop 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 -- 172.21.15.52:0/1896627738 shutdown_connections 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e14340 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e19a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1896627738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 -- 172.21.15.52:0/1896627738 >> 172.21.15.52:0/1896627738 conn(0x561c62d3ecc0 msgr2=0x561c62e0e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 -- 172.21.15.52:0/1896627738 shutdown_connections 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.827+0000 7fa548d53b80 1 -- 172.21.15.52:0/1896627738 wait complete. 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 -- start start 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 Processor -- start 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e1fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e1d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e1d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561c62e209d0 con 0x561c62e19f60 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561c62d05bf0 con 0x561c62e13f20 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.828+0000 7fa548d53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561c62e20b50 con 0x561c62e19640 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa548d53b80 10 client.0 _ll_register_callbacks cb 0x561c62cff920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e1fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e1fd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39570/0 (socket says 172.21.15.52:39570) 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 -- 172.21.15.52:0/1024462571 learned_addr learned my addr 172.21.15.52:0/1024462571 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e1d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e1d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 -- 172.21.15.52:0/1024462571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 msgr2=0x561c62e1d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e1d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 -- 172.21.15.52:0/1024462571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 msgr2=0x561c62e1d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e1d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 -- 172.21.15.52:0/1024462571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c62cb48d0 con 0x561c62e13f20 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e1d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e1d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa537fff700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e1fd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa528005cb0 tx=0x7fa5280218a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1020a06d5b87c10 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa5177fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa5177fe700 1 -- 172.21.15.52:0/1024462571 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52809c620 con 0x561c62e13f20 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.829+0000 7fa5177fe700 1 -- 172.21.15.52:0/1024462571 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa52809a920 con 0x561c62e13f20 2024-06-02T23:22:31.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa530004d50 con 0x561c62e13f20 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa5177fe700 1 -- 172.21.15.52:0/1024462571 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa5280a09d0 con 0x561c62e13f20 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 10 client.27926 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c62e2b2f0 con 0x561c62e13f20 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa5177fe700 1 -- 172.21.15.52:0/1024462571 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa530004d50 con 0x561c62e13f20 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa5177fe700 1 -- 172.21.15.52:0/1024462571 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa5280a4d70 con 0x561c62e13f20 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 10 client.27926 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 10 client.27926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 20 client.27926 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e2db60 0x561c62e2ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 4 client.27926 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c62d72d10 con 0x561c62e2db60 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.830+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e2db60 0x561c62e2ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.831+0000 7fa5377fe700 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e2db60 0x561c62e2ffa0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c482caeb3ba842fd server_cookie=69559d7e4258b4b6 in_seq=0 out_seq=0 2024-06-02T23:22:31.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.831+0000 7fa5177fe700 10 client.27926 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.831+0000 7fa5177fe700 1 -- 172.21.15.52:0/1024462571 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561c62d72d10 con 0x561c62e2db60 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa5177fe700 10 client.27926 handle_command_reply: tid=0 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 client.27926 shutdown 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 msgr2=0x561c62e1fd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e1fd40 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa528005cb0 tx=0x7fa5280218a0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 shutdown_connections 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e2db60 0x561c62e2ffa0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19f60 0x561c62e1d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e1fd40 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 --2- 172.21.15.52:0/1024462571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e1d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 >> 172.21.15.52:0/1024462571 conn(0x561c62d3ecc0 msgr2=0x561c62e129d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa5177fe700 0 client.27926 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 shutdown_connections 2024-06-02T23:22:31.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 1 -- 172.21.15.52:0/1024462571 wait complete. 2024-06-02T23:22:31.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.832+0000 7fa548d53b80 20 client.27926 trim_cache size 0 max 16384 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 Processor -- start 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 -- start start 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 0x561c62e1bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19f60 0x561c62e08510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381aa7f0 con 0x561c62e19640 2024-06-02T23:22:31.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ac330 con 0x561c62e13f20 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa548d53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381aaec0 con 0x561c62e19f60 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 0x561c62e1bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa5377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 0x561c62e1bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57090/0 (socket says 172.21.15.52:57090) 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa5377fe700 1 -- 172.21.15.52:0/2607770825 learned_addr learned my addr 172.21.15.52:0/2607770825 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa537fff700 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.851+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19f60 0x561c62e08510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa5377fe700 1 -- 172.21.15.52:0/2607770825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19f60 msgr2=0x561c62e08510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa5377fe700 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19f60 0x561c62e08510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa5377fe700 1 -- 172.21.15.52:0/2607770825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 msgr2=0x561c62e139e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa5377fe700 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e139e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa5377fe700 1 -- 172.21.15.52:0/2607770825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c62e1f980 con 0x561c62e19640 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa537fff700 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa5377fe700 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 0x561c62e1bfa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009eb0 tx=0x7fa52c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a60de7a71dd139 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa51dffb700 1 -- 172.21.15.52:0/2607770825 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c035020 con 0x561c62e19640 2024-06-02T23:22:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa51dffb700 1 -- 172.21.15.52:0/2607770825 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa52c031b00 con 0x561c62e19640 2024-06-02T23:22:31.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.852+0000 7fa51dffb700 1 -- 172.21.15.52:0/2607770825 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52c024b80 con 0x561c62e19640 2024-06-02T23:22:31.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 -- 172.21.15.52:0/2607770825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 msgr2=0x561c62e1bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 0x561c62e1bfa0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fa52c009eb0 tx=0x7fa52c01f000 comp rx=0 tx=0).stop 2024-06-02T23:22:31.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 -- 172.21.15.52:0/2607770825 shutdown_connections 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19f60 0x561c62e08510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e13f20 0x561c62e139e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 --2- 172.21.15.52:0/2607770825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e19640 0x561c62e1bfa0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 -- 172.21.15.52:0/2607770825 >> 172.21.15.52:0/2607770825 conn(0x561c62d3ecc0 msgr2=0x561c62e07e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 -- 172.21.15.52:0/2607770825 shutdown_connections 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 -- 172.21.15.52:0/2607770825 wait complete. 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.853+0000 7fa548d53b80 1 -- start start 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 Processor -- start 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 0x561c62e25e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e26380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e19f60 0x561c62e34940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381abc60 con 0x561c62e13f20 2024-06-02T23:22:31.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ab590 con 0x561c62e19f60 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381a8790 con 0x561c62e19640 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa548d53b80 10 client.0 _ll_register_callbacks cb 0x561c62cff920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 0x561c62e25e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 0x561c62e25e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57104/0 (socket says 172.21.15.52:57104) 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 -- 172.21.15.52:0/264729042 learned_addr learned my addr 172.21.15.52:0/264729042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa53ca6d700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e19f60 0x561c62e34940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa5377fe700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e26380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 -- 172.21.15.52:0/264729042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 msgr2=0x561c62e26380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e26380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 -- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e19f60 msgr2=0x561c62e34940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e19f60 0x561c62e34940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa537fff700 1 -- 172.21.15.52:0/264729042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c62e34e80 con 0x561c62e13f20 2024-06-02T23:22:31.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.854+0000 7fa5377fe700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e26380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa537fff700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 0x561c62e25e40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa5280a5170 tx=0x7fa52809cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=327118be745b9b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa5177fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa5177fe700 1 -- 172.21.15.52:0/264729042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52809bf80 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa5177fe700 1 -- 172.21.15.52:0/264729042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa52809c880 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa52c031b00 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa5177fe700 1 -- 172.21.15.52:0/264729042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fa52809c620 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa548d53b80 10 client.18474 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561c62e34e80 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa5177fe700 1 -- 172.21.15.52:0/264729042 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa52c031b00 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.855+0000 7fa5177fe700 1 -- 172.21.15.52:0/264729042 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fa52809ba70 con 0x561c62e13f20 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 10 client.18474 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 10 client.18474 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 20 client.18474 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:31.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e98350 0x561c62e9a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 4 client.18474 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561c62d6d290 con 0x561c62e98350 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa5377fe700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e98350 0x561c62e9a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa5377fe700 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e98350 0x561c62e9a740 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ca6604fcb6c9ff server_cookie=65d7b4af51d1784b in_seq=0 out_seq=0 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa5177fe700 10 client.18474 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa5177fe700 1 -- 172.21.15.52:0/264729042 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561c62d6d290 con 0x561c62e98350 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa5177fe700 10 client.18474 handle_command_reply: tid=0 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.856+0000 7fa548d53b80 1 client.18474 shutdown 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 msgr2=0x561c62e25e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 0x561c62e25e40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fa5280a5170 tx=0x7fa52809cbd0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 shutdown_connections 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561c62e98350 0x561c62e9a740 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561c62e19f60 0x561c62e34940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561c62e13f20 0x561c62e25e40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 --2- 172.21.15.52:0/264729042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561c62e19640 0x561c62e26380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 >> 172.21.15.52:0/264729042 conn(0x561c62d3ecc0 msgr2=0x561c62e1a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa5177fe700 0 client.18474 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 shutdown_connections 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 1 -- 172.21.15.52:0/264729042 wait complete. 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa548d53b80 20 client.18474 trim_cache size 0 max 16384 2024-06-02T23:22:31.861 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5200a2ca0 msgr2=0x7fa5200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5200a2ca0 0x7fa5200a5160 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa5380789e0 tx=0x7fa52c009260 comp rx=0 tx=0).stop 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 msgr2=0x7fa53819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.857+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53819e780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa5280057d0 tx=0x7fa528005800 comp rx=0 tx=0).stop 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 shutdown_connections 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5200a2ca0 0x7fa5200a5160 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa5381084f0 0x7fa538198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa53806aa00 0x7fa53819e780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 --2- 172.21.15.52:0/1262927969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa53806b3c0 0x7fa53819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 >> 172.21.15.52:0/1262927969 conn(0x7fa538078300 msgr2=0x7fa5381001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:31.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 shutdown_connections 2024-06-02T23:22:31.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:31.858+0000 7fa53e4d0700 1 -- 172.21.15.52:0/1262927969 wait complete. 2024-06-02T23:22:32.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 210964 1.90 MB/sec execute 461 sec latency 18.397 ms 2024-06-02T23:22:33.288 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:33 smithi052 ceph-mon[32340]: pgmap v751: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-06-02T23:22:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:33 smithi174 ceph-mon[35994]: pgmap v751: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-06-02T23:22:33.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:33 smithi123 ceph-mon[35977]: pgmap v751: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 1.4 MiB/s wr, 157 op/s 2024-06-02T23:22:33.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 211620 1.90 MB/sec execute 462 sec latency 15.047 ms 2024-06-02T23:22:33.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 -- 172.21.15.52:0/2259504646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102ef0 msgr2=0x7fbf3010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 --2- 172.21.15.52:0/2259504646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102ef0 0x7fbf3010f3a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800b0d0 tx=0x7fbf2800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 -- 172.21.15.52:0/2259504646 shutdown_connections 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 --2- 172.21.15.52:0/2259504646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102ef0 0x7fbf3010f3a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 --2- 172.21.15.52:0/2259504646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 0x7fbf301089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 --2- 172.21.15.52:0/2259504646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102530 0x7fbf301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 -- 172.21.15.52:0/2259504646 >> 172.21.15.52:0/2259504646 conn(0x7fbf30079640 msgr2=0x7fbf30079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 -- 172.21.15.52:0/2259504646 shutdown_connections 2024-06-02T23:22:33.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.698+0000 7fbf37387700 1 -- 172.21.15.52:0/2259504646 wait complete. 2024-06-02T23:22:33.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 Processor -- start 2024-06-02T23:22:33.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 -- start start 2024-06-02T23:22:33.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 0x7fbf30074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:33.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102ef0 0x7fbf30074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 0x7fbf30075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf301126c0 con 0x7fbf30102530 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30112540 con 0x7fbf301085d0 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf37387700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf30112840 con 0x7fbf30102ef0 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf35123700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 0x7fbf30074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf34922700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102ef0 0x7fbf30074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf35123700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 0x7fbf30074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57136/0 (socket says 172.21.15.52:57136) 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf35924700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 0x7fbf30075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.699+0000 7fbf35123700 1 -- 172.21.15.52:0/1616555343 learned_addr learned my addr 172.21.15.52:0/1616555343 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf34922700 1 -- 172.21.15.52:0/1616555343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 msgr2=0x7fbf30075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf34922700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 0x7fbf30075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf34922700 1 -- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 msgr2=0x7fbf30074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf34922700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 0x7fbf30074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:33.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf34922700 1 -- 172.21.15.52:0/1616555343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf301a6fe0 con 0x7fbf30102ef0 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf35123700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 0x7fbf30074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf35924700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 0x7fbf30075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf34922700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102ef0 0x7fbf30074cd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d950 tx=0x7fbf2000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf20016070 con 0x7fbf30102ef0 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf2000fd40 con 0x7fbf30102ef0 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf301a72d0 con 0x7fbf30102ef0 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2000e3b0 con 0x7fbf30102ef0 2024-06-02T23:22:33.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.700+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf301a7860 con 0x7fbf30102ef0 2024-06-02T23:22:33.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.701+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf30103c40 con 0x7fbf30102ef0 2024-06-02T23:22:33.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.703+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf20019070 con 0x7fbf30102ef0 2024-06-02T23:22:33.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.704+0000 7fbf267fc700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf1c0a2ca0 0x7fbf1c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:33.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.704+0000 7fbf35123700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf1c0a2ca0 0x7fbf1c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:33.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.704+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbf200ceae0 con 0x7fbf30102ef0 2024-06-02T23:22:33.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.705+0000 7fbf35123700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf1c0a2ca0 0x7fbf1c0a5160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009680 tx=0x7fbf2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:33.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:33.707+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fbf2009ba10 con 0x7fbf30102ef0 2024-06-02T23:22:33.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.000+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf30075cc0 con 0x7fbf30102ef0 2024-06-02T23:22:34.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.001+0000 7fbf267fc700 1 -- 172.21.15.52:0/1616555343 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fbf2009b310 con 0x7fbf30102ef0 2024-06-02T23:22:34.000 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:22:34.000 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:22:34.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf1c0a2ca0 msgr2=0x7fbf1c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:34.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf1c0a2ca0 0x7fbf1c0a5160 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009680 tx=0x7fbf2c009260 comp rx=0 tx=0).stop 2024-06-02T23:22:34.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102ef0 msgr2=0x7fbf30074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102ef0 0x7fbf30074cd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d950 tx=0x7fbf2000dc60 comp rx=0 tx=0).stop 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 shutdown_connections 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf1c0a2ca0 0x7fbf1c0a5160 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf301085d0 0x7fbf30075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf30102530 0x7fbf30074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.003+0000 7fbf37387700 1 --2- 172.21.15.52:0/1616555343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf30102ef0 0x7fbf30074cd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.004+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 >> 172.21.15.52:0/1616555343 conn(0x7fbf30079640 msgr2=0x7fbf301010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.004+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 shutdown_connections 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:34.004+0000 7fbf37387700 1 -- 172.21.15.52:0/1616555343 wait complete. 2024-06-02T23:22:34.004 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:22:34.193 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:22:34.193 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:22:34.328 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:34 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1616555343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:34.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:34 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1616555343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:34.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:34 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1616555343' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:34.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 212217 1.91 MB/sec execute 463 sec latency 12.605 ms 2024-06-02T23:22:34.746 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 132.4 --await-for 45.0' 2024-06-02T23:22:34.747 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 132.4 --await-for 45.0 2024-06-02T23:22:34.817 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:35.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:35 smithi174 ceph-mon[35994]: pgmap v752: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-02T23:22:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:35 smithi052 ceph-mon[32340]: pgmap v752: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-02T23:22:35.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:35 smithi123 ceph-mon[35977]: pgmap v752: 97 pgs: 97 active+clean; 94 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-02T23:22:35.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 212665 1.91 MB/sec execute 464 sec latency 14.948 ms 2024-06-02T23:22:36.006 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:36.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 213133 1.92 MB/sec execute 465 sec latency 19.721 ms 2024-06-02T23:22:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 -- 172.21.15.52:0/333462483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101380 msgr2=0x7fd72c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 --2- 172.21.15.52:0/333462483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101380 0x7fd72c101800 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fd720009990 tx=0x7fd720009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 -- 172.21.15.52:0/333462483 shutdown_connections 2024-06-02T23:22:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 --2- 172.21.15.52:0/333462483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101d40 0x7fd72c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 --2- 172.21.15.52:0/333462483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c1073f0 0x7fd72c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 --2- 172.21.15.52:0/333462483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101380 0x7fd72c101800 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 -- 172.21.15.52:0/333462483 >> 172.21.15.52:0/333462483 conn(0x7fd72c078300 msgr2=0x7fd72c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:37.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.114+0000 7fd730f16700 1 -- 172.21.15.52:0/333462483 shutdown_connections 2024-06-02T23:22:37.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.115+0000 7fd730f16700 1 -- 172.21.15.52:0/333462483 wait complete. 2024-06-02T23:22:37.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.115+0000 7fd730f16700 1 Processor -- start 2024-06-02T23:22:37.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.115+0000 7fd730f16700 1 -- start start 2024-06-02T23:22:37.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd730f16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101380 0x7fd72c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd730f16700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 0x7fd72c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd730f16700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c1073f0 0x7fd72c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd730f16700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c111510 con 0x7fd72c101d40 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd730f16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c111390 con 0x7fd72c101380 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd730f16700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c111690 con 0x7fd72c1073f0 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 0x7fd72c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 0x7fd72c19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48490/0 (socket says 172.21.15.52:48490) 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 -- 172.21.15.52:0/2949652240 learned_addr learned my addr 172.21.15.52:0/2949652240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101380 0x7fd72c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c1073f0 0x7fd72c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 -- 172.21.15.52:0/2949652240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c1073f0 msgr2=0x7fd72c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c1073f0 0x7fd72c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 -- 172.21.15.52:0/2949652240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101380 msgr2=0x7fd72c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101380 0x7fd72c19e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 -- 172.21.15.52:0/2949652240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c199060 con 0x7fd72c101d40 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101380 0x7fd72c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.116+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 0x7fd72c19ec80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd720000c00 tx=0x7fd720004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.117+0000 7fd71b7fe700 1 -- 172.21.15.52:0/2949652240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720027070 con 0x7fd72c101d40 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.117+0000 7fd71b7fe700 1 -- 172.21.15.52:0/2949652240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd720022470 con 0x7fd72c101d40 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.117+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1992f0 con 0x7fd72c101d40 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.117+0000 7fd71b7fe700 1 -- 172.21.15.52:0/2949652240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72000b610 con 0x7fd72c101d40 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.117+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c102f90 con 0x7fd72c101d40 2024-06-02T23:22:37.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.119+0000 7fd71b7fe700 1 -- 172.21.15.52:0/2949652240 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd72002a070 con 0x7fd72c101d40 2024-06-02T23:22:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.120+0000 7fd71b7fe700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd7140a2c20 0x7fd7140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.120+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd7140a2c20 0x7fd7140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.120+0000 7fd71b7fe700 1 -- 172.21.15.52:0/2949652240 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd7200cfcd0 con 0x7fd72c101d40 2024-06-02T23:22:37.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.120+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd7140a2c20 0x7fd7140a50e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd72c199a40 tx=0x7fd71c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:37.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 Processor -- start 2024-06-02T23:22:37.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 -- start start 2024-06-02T23:22:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 0x55e6d2040aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d203aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d203afa0 0x55e6d203b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e6d1f8c890 con 0x55e6d20406c0 2024-06-02T23:22:37.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e6d1f347a0 con 0x55e6d203afa0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd73b799b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6d1f88850 con 0x55e6d2040fe0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 0x55e6d2040aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 0x55e6d2040aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48502/0 (socket says 172.21.15.52:48502) 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd72a59c700 1 -- 172.21.15.52:0/2858188234 learned_addr learned my addr 172.21.15.52:0/2858188234 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.121+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d203aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd72a59c700 1 -- 172.21.15.52:0/2858188234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 msgr2=0x55e6d203aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d203aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd72a59c700 1 -- 172.21.15.52:0/2858188234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d203afa0 msgr2=0x55e6d203b3c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d203afa0 0x55e6d203b3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd72a59c700 1 -- 172.21.15.52:0/2858188234 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d1f2afd0 con 0x55e6d20406c0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 0x55e6d2040aa0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00e930 tx=0x7fd71c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84471a369efcde5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd718ff9700 1 -- 172.21.15.52:0/2858188234 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c017a10 con 0x55e6d20406c0 2024-06-02T23:22:37.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd718ff9700 1 -- 172.21.15.52:0/2858188234 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd71c026070 con 0x55e6d20406c0 2024-06-02T23:22:37.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.122+0000 7fd718ff9700 1 -- 172.21.15.52:0/2858188234 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c021e60 con 0x55e6d20406c0 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 -- 172.21.15.52:0/2858188234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 msgr2=0x55e6d2040aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 0x55e6d2040aa0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fd71c00e930 tx=0x7fd71c014000 comp rx=0 tx=0).stop 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 -- 172.21.15.52:0/2858188234 shutdown_connections 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d203afa0 0x55e6d203b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d20406c0 0x55e6d2040aa0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2858188234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d203aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 -- 172.21.15.52:0/2858188234 >> 172.21.15.52:0/2858188234 conn(0x55e6d1f65d40 msgr2=0x55e6d20358d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 -- 172.21.15.52:0/2858188234 shutdown_connections 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 -- 172.21.15.52:0/2858188234 wait complete. 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.123+0000 7fd73b799b80 1 -- start start 2024-06-02T23:22:37.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 Processor -- start 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 0x55e6d2046d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d20406c0 0x55e6d2044030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d2040fe0 0x55e6d2044570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e6d2047a50 con 0x55e6d203afa0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e6d1f2cc70 con 0x55e6d2040fe0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6d2047bd0 con 0x55e6d20406c0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd73b799b80 10 client.0 _ll_register_callbacks cb 0x55e6d1f269a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 0x55e6d2046d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 0x55e6d2046d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48514/0 (socket says 172.21.15.52:48514) 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 -- 172.21.15.52:0/2531468887 learned_addr learned my addr 172.21.15.52:0/2531468887 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d2040fe0 0x55e6d2044570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d20406c0 0x55e6d2044030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 -- 172.21.15.52:0/2531468887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d20406c0 msgr2=0x55e6d2044030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d20406c0 0x55e6d2044030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 -- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d2040fe0 msgr2=0x55e6d2044570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.124+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d2040fe0 0x55e6d2044570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd72a59c700 1 -- 172.21.15.52:0/2531468887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d1edb950 con 0x55e6d203afa0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d2040fe0 0x55e6d2044570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d20406c0 0x55e6d2044030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd72a59c700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 0x55e6d2046d30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd71c007b70 tx=0x7fd71c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21384c29e93fe5de server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd711ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd711ffb700 1 -- 172.21.15.52:0/2531468887 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c021620 con 0x55e6d203afa0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd711ffb700 1 -- 172.21.15.52:0/2531468887 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd71c0217c0 con 0x55e6d203afa0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71c026070 con 0x55e6d203afa0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd711ffb700 1 -- 172.21.15.52:0/2531468887 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd71c03b070 con 0x55e6d203afa0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd73b799b80 10 client.18492 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6d2052330 con 0x55e6d203afa0 2024-06-02T23:22:37.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.125+0000 7fd711ffb700 1 -- 172.21.15.52:0/2531468887 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd71c026070 con 0x55e6d203afa0 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd711ffb700 1 -- 172.21.15.52:0/2531468887 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd71c034810 con 0x55e6d203afa0 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd73b799b80 10 client.18492 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd73b799b80 10 client.18492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd73b799b80 20 client.18492 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d2054910 0x55e6d2056d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd73b799b80 4 client.18492 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6d1f99d90 con 0x55e6d2054910 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d2054910 0x55e6d2056d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd729d9b700 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d2054910 0x55e6d2056d00 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8537370643204dec server_cookie=c915d7dfcdcb23db in_seq=0 out_seq=0 2024-06-02T23:22:37.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.126+0000 7fd711ffb700 10 client.18492 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd711ffb700 1 -- 172.21.15.52:0/2531468887 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e6d1f99d90 con 0x55e6d2054910 2024-06-02T23:22:37.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd711ffb700 10 client.18492 handle_command_reply: tid=0 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd73b799b80 1 client.18492 shutdown 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 msgr2=0x55e6d2046d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 0x55e6d2046d30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fd71c007b70 tx=0x7fd71c004ce0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 shutdown_connections 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.127+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d2054910 0x55e6d2056d00 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d2040fe0 0x55e6d2044570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d203afa0 0x55e6d2046d30 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 1 --2- 172.21.15.52:0/2531468887 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d20406c0 0x55e6d2044030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 >> 172.21.15.52:0/2531468887 conn(0x55e6d1f65d40 msgr2=0x55e6d2035670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd711ffb700 0 client.18492 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 shutdown_connections 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 1 -- 172.21.15.52:0/2531468887 wait complete. 2024-06-02T23:22:37.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.128+0000 7fd73b799b80 20 client.18492 trim_cache size 0 max 16384 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.152+0000 7fd73b799b80 1 Processor -- start 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.152+0000 7fd73b799b80 1 -- start start 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d203aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d202f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d2059dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd73b799b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1aeb90 con 0x55e6d2059940 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd73b799b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1b06d0 con 0x55e6d20406c0 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd73b799b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1af260 con 0x55e6d2040fe0 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d2059dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d203aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d2059dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48526/0 (socket says 172.21.15.52:48526) 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72ad9d700 1 -- 172.21.15.52:0/416990652 learned_addr learned my addr 172.21.15.52:0/416990652 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:37.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d203aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60238/0 (socket says 172.21.15.52:60238) 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd729d9b700 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d202f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 -- 172.21.15.52:0/416990652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 msgr2=0x55e6d202f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d202f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 -- 172.21.15.52:0/416990652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 msgr2=0x55e6d2059dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d2059dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 -- 172.21.15.52:0/416990652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6d20364f0 con 0x55e6d20406c0 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d2059dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd72a59c700 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d203aa60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009fd0 tx=0x7fd71c005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b1cca4611d5ae4a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.153+0000 7fd713fff700 1 -- 172.21.15.52:0/416990652 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c034680 con 0x55e6d20406c0 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd713fff700 1 -- 172.21.15.52:0/416990652 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd71c026040 con 0x55e6d20406c0 2024-06-02T23:22:37.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd713fff700 1 -- 172.21.15.52:0/416990652 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71c022070 con 0x55e6d20406c0 2024-06-02T23:22:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 -- 172.21.15.52:0/416990652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 msgr2=0x55e6d203aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d203aa60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009fd0 tx=0x7fd71c005eb0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 -- 172.21.15.52:0/416990652 shutdown_connections 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d2059dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d203aa60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 --2- 172.21.15.52:0/416990652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d202f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 -- 172.21.15.52:0/416990652 >> 172.21.15.52:0/416990652 conn(0x55e6d1f65d40 msgr2=0x55e6d2041a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.154+0000 7fd73b799b80 1 -- 172.21.15.52:0/416990652 shutdown_connections 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 -- 172.21.15.52:0/416990652 wait complete. 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 -- start start 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 Processor -- start 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d20571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d2057700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d20c0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1b0000 con 0x55e6d2059940 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1af930 con 0x55e6d20406c0 2024-06-02T23:22:37.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.155+0000 7fd73b799b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1acb30 con 0x55e6d2040fe0 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d20571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd73b799b80 10 client.0 _ll_register_callbacks cb 0x55e6d1f269a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d20c0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d20571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60240/0 (socket says 172.21.15.52:60240) 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d20c0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48528/0 (socket says 172.21.15.52:48528) 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 -- 172.21.15.52:0/3494395626 learned_addr learned my addr 172.21.15.52:0/3494395626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd729d9b700 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d2057700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 -- 172.21.15.52:0/3494395626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 msgr2=0x55e6d2057700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d2057700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 -- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 msgr2=0x55e6d20571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d20571c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 -- 172.21.15.52:0/3494395626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6d20401c0 con 0x55e6d2059940 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd72ad9d700 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d20c0aa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd724010db0 tx=0x7fd72400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c014c89a28cd766 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:37.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd711ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd711ffb700 1 -- 172.21.15.52:0/3494395626 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724020060 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd711ffb700 1 -- 172.21.15.52:0/3494395626 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd72401b400 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.156+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd71c026040 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd711ffb700 1 -- 172.21.15.52:0/3494395626 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd72401b600 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 10 client.18498 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55e6d20c8310 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd711ffb700 1 -- 172.21.15.52:0/3494395626 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd71c026040 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd711ffb700 1 -- 172.21.15.52:0/3494395626 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd72401a3a0 con 0x55e6d2059940 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 10 client.18498 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 10 client.18498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 20 client.18498 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d20c8a80 0x55e6d20caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 4 client.18498 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:22:37.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e6d1f94310 con 0x55e6d20c8a80 2024-06-02T23:22:37.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.157+0000 7fd72a59c700 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d20c8a80 0x55e6d20caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.158+0000 7fd72a59c700 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d20c8a80 0x55e6d20caea0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a462858f6ed5bfca server_cookie=bcd9f55c1f921125 in_seq=0 out_seq=0 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.158+0000 7fd711ffb700 10 client.18498 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd711ffb700 1 -- 172.21.15.52:0/3494395626 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e6d1f94310 con 0x55e6d20c8a80 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd711ffb700 10 client.18498 handle_command_reply: tid=0 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 client.18498 shutdown 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 msgr2=0x55e6d20c0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d20c0aa0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fd724010db0 tx=0x7fd72400f6a0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 shutdown_connections 2024-06-02T23:22:37.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e6d20c8a80 0x55e6d20caea0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e6d2059940 0x55e6d20c0aa0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e6d20406c0 0x55e6d20571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 --2- 172.21.15.52:0/3494395626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6d2040fe0 0x55e6d2057700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.159+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 >> 172.21.15.52:0/3494395626 conn(0x55e6d1f65d40 msgr2=0x55e6d202ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd711ffb700 0 client.18498 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 shutdown_connections 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd73b799b80 1 -- 172.21.15.52:0/3494395626 wait complete. 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd73b799b80 20 client.18498 trim_cache size 0 max 16384 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:22:37.162 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "1675aa7d-9859-4db8-82de-a8960e113286", 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd7140a2c20 msgr2=0x7fd7140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd730f16700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd7140a2c20 0x7fd7140a50e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd72c199a40 tx=0x7fd71c00b040 comp rx=0 tx=0).stop 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 msgr2=0x7fd72c19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.160+0000 7fd730f16700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 0x7fd72c19ec80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fd720000c00 tx=0x7fd720004790 comp rx=0 tx=0).stop 2024-06-02T23:22:37.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 shutdown_connections 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd7140a2c20 0x7fd7140a50e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd72c1073f0 0x7fd72c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd72c101380 0x7fd72c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 --2- 172.21.15.52:0/2949652240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd72c101d40 0x7fd72c19ec80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 >> 172.21.15.52:0/2949652240 conn(0x7fd72c078300 msgr2=0x7fd72c100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 shutdown_connections 2024-06-02T23:22:37.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:37.161+0000 7fd730f16700 1 -- 172.21.15.52:0/2949652240 wait complete. 2024-06-02T23:22:37.292 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:37 smithi052 ceph-mon[32340]: pgmap v753: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-02T23:22:37.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:37 smithi174 ceph-mon[35994]: pgmap v753: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-02T23:22:37.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:37 smithi123 ceph-mon[35977]: pgmap v753: 97 pgs: 97 active+clean; 89 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.5 MiB/s wr, 168 op/s 2024-06-02T23:22:37.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 213412 1.91 MB/sec execute 466 sec latency 14.621 ms 2024-06-02T23:22:38.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:22:38.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:38.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:38.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.304 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.305 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:38 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:22:38.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:38.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:38.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:38 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:38.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:38 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:38.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 213922 1.91 MB/sec execute 467 sec latency 21.196 ms 2024-06-02T23:22:38.978 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:22:39.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 -- 172.21.15.52:0/3367047392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf01046d0 msgr2=0x7fdcf0104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:39 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:39.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:39 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:39.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:39 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:22:39.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:39 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:39.234 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:39 smithi052 ceph-mon[32340]: pgmap v754: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-06-02T23:22:39.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/3367047392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf01046d0 0x7fdcf0104b50 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fdce0009990 tx=0x7fdce0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 -- 172.21.15.52:0/3367047392 shutdown_connections 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/3367047392 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf0105090 0x7fdcf01115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/3367047392 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf010a6d0 0x7fdcf010aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/3367047392 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf01046d0 0x7fdcf0104b50 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 -- 172.21.15.52:0/3367047392 >> 172.21.15.52:0/3367047392 conn(0x7fdcf01001c0 msgr2=0x7fdcf01025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 -- 172.21.15.52:0/3367047392 shutdown_connections 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.233+0000 7fdcf7be1700 1 -- 172.21.15.52:0/3367047392 wait complete. 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.234+0000 7fdcf7be1700 1 Processor -- start 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.234+0000 7fdcf7be1700 1 -- start start 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.234+0000 7fdcf7be1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf01046d0 0x7fdcf01a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.234+0000 7fdcf7be1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf0105090 0x7fdcf01a0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf7be1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 0x7fdcf019aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf7be1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114910 con 0x7fdcf010a6d0 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf7be1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114790 con 0x7fdcf01046d0 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf7be1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcf0114a90 con 0x7fdcf0105090 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 0x7fdcf019aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 0x7fdcf019aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48548/0 (socket says 172.21.15.52:48548) 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 -- 172.21.15.52:0/1446088632 learned_addr learned my addr 172.21.15.52:0/1446088632 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf517c700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf0105090 0x7fdcf01a0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 -- 172.21.15.52:0/1446088632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf0105090 msgr2=0x7fdcf01a0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf0105090 0x7fdcf01a0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 -- 172.21.15.52:0/1446088632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf01046d0 msgr2=0x7fdcf01a0910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf01046d0 0x7fdcf01a0910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 -- 172.21.15.52:0/1446088632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcf019b230 con 0x7fdcf010a6d0 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf517c700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf0105090 0x7fdcf01a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:39.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.235+0000 7fdcf617e700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 0x7fdcf019aad0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fdce8007ee0 tx=0x7fdce800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.236+0000 7fdce6ffd700 1 -- 172.21.15.52:0/1446088632 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce800dbd0 con 0x7fdcf010a6d0 2024-06-02T23:22:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.236+0000 7fdce6ffd700 1 -- 172.21.15.52:0/1446088632 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce800dd70 con 0x7fdcf010a6d0 2024-06-02T23:22:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.236+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcf019b520 con 0x7fdcf010a6d0 2024-06-02T23:22:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.236+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcf01a77e0 con 0x7fdcf010a6d0 2024-06-02T23:22:39.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.236+0000 7fdce6ffd700 1 -- 172.21.15.52:0/1446088632 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8005990 con 0x7fdcf010a6d0 2024-06-02T23:22:39.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.240+0000 7fdce6ffd700 1 -- 172.21.15.52:0/1446088632 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdce801a020 con 0x7fdcf010a6d0 2024-06-02T23:22:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.241+0000 7fdce6ffd700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdcdc0a2d40 0x7fdcdc0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.241+0000 7fdce6ffd700 1 -- 172.21.15.52:0/1446088632 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdce80ce630 con 0x7fdcf010a6d0 2024-06-02T23:22:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.241+0000 7fdcf597d700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdcdc0a2d40 0x7fdcdc0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.241+0000 7fdd02464b80 1 Processor -- start 2024-06-02T23:22:39.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.241+0000 7fdd02464b80 1 -- start start 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991543180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639915436c0 0x56399153cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56399153d4f0 0x56399153d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdd02464b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5639914270f0 con 0x5639915436c0 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdd02464b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639914859e0 con 0x56399153d4f0 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdd02464b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5639913ed5f0 con 0x563991542da0 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf517c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639915436c0 0x56399153cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf517c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639915436c0 0x56399153cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48562/0 (socket says 172.21.15.52:48562) 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf517c700 1 -- 172.21.15.52:0/116578847 learned_addr learned my addr 172.21.15.52:0/116578847 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:39.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf597d700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdcdc0a2d40 0x7fdcdc0a5200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009a70 tx=0x7fdcec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56399153d4f0 0x56399153d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf597d700 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991543180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 -- 172.21.15.52:0/116578847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 msgr2=0x563991543180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991543180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 -- 172.21.15.52:0/116578847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639915436c0 msgr2=0x56399153cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639915436c0 0x56399153cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 -- 172.21.15.52:0/116578847 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639913d4a60 con 0x56399153d4f0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.242+0000 7fdcf617e700 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56399153d4f0 0x56399153d910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdce80cc160 tx=0x7fdce809b640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a58fbd8fe98ff0fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.243+0000 7fdcdbfff700 1 -- 172.21.15.52:0/116578847 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce80a3b30 con 0x56399153d4f0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.243+0000 7fdcdbfff700 1 -- 172.21.15.52:0/116578847 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce8009390 con 0x56399153d4f0 2024-06-02T23:22:39.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.243+0000 7fdcdbfff700 1 -- 172.21.15.52:0/116578847 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce809b380 con 0x56399153d4f0 2024-06-02T23:22:39.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 -- 172.21.15.52:0/116578847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56399153d4f0 msgr2=0x56399153d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56399153d4f0 0x56399153d910 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdce80cc160 tx=0x7fdce809b640 comp rx=0 tx=0).stop 2024-06-02T23:22:39.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 -- 172.21.15.52:0/116578847 shutdown_connections 2024-06-02T23:22:39.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56399153d4f0 0x56399153d910 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991543180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 --2- 172.21.15.52:0/116578847 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639915436c0 0x56399153cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 -- 172.21.15.52:0/116578847 >> 172.21.15.52:0/116578847 conn(0x563991461210 msgr2=0x56399152fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 -- 172.21.15.52:0/116578847 shutdown_connections 2024-06-02T23:22:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.244+0000 7fdd02464b80 1 -- 172.21.15.52:0/116578847 wait complete. 2024-06-02T23:22:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 -- start start 2024-06-02T23:22:39.245 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 Processor -- start 2024-06-02T23:22:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x563991549440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991546740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x563991546c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563991426150 con 0x56399153d4f0 2024-06-02T23:22:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdcf597d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x563991549440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.246 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639914819a0 con 0x5639915436c0 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.245+0000 7fdd02464b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56399154a0f0 con 0x563991542da0 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x563991549440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48566/0 (socket says 172.21.15.52:48566) 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 -- 172.21.15.52:0/3682781031 learned_addr learned my addr 172.21.15.52:0/3682781031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf517c700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991546740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdd02464b80 10 client.0 _ll_register_callbacks cb 0x56399141b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf617e700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x563991546c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 -- 172.21.15.52:0/3682781031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 msgr2=0x563991546740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991546740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 -- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 msgr2=0x563991546c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x563991546c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf597d700 1 -- 172.21.15.52:0/3682781031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56399144b8e0 con 0x56399153d4f0 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf517c700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991546740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.246+0000 7fdcf617e700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x563991546c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:39.247 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x563991549440 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdcec01d6e0 tx=0x7fdcec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc82269cd764f30f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdcd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3682781031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcec02a060 con 0x56399153d4f0 2024-06-02T23:22:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3682781031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdcec010840 con 0x56399153d4f0 2024-06-02T23:22:39.248 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce8009390 con 0x56399153d4f0 2024-06-02T23:22:39.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3682781031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdcec010a80 con 0x56399153d4f0 2024-06-02T23:22:39.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdd02464b80 10 client.18510 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:39.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.247+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x563991554890 con 0x56399153d4f0 2024-06-02T23:22:39.249 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3682781031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce8009390 con 0x56399153d4f0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3682781031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdcec024030 con 0x56399153d4f0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdd02464b80 10 client.18510 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdd02464b80 10 client.18510 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdd02464b80 20 client.18510 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x563991559350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdd02464b80 4 client.18510 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563991492ee0 con 0x563991556ed0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.248+0000 7fdcf517c700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x563991559350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.249+0000 7fdcd97fa700 10 client.18510 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.249+0000 7fdcf517c700 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x563991559350 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32555a4139dc525d server_cookie=ec4bcb7097072489 in_seq=0 out_seq=0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3682781031 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563991492ee0 con 0x563991556ed0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdcd97fa700 10 client.18510 handle_command_reply: tid=0 2024-06-02T23:22:39.250 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 client.18510 shutdown 2024-06-02T23:22:39.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 msgr2=0x563991549440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x563991549440 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fdcec01d6e0 tx=0x7fdcec0047c0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 shutdown_connections 2024-06-02T23:22:39.251 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x563991559350 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x563991546c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x563991549440 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdcd97fa700 0 client.18510 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3682781031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563991542da0 0x563991546740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.250+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 >> 172.21.15.52:0/3682781031 conn(0x563991461210 msgr2=0x56399152ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.251+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 shutdown_connections 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.251+0000 7fdd02464b80 1 -- 172.21.15.52:0/3682781031 wait complete. 2024-06-02T23:22:39.252 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.251+0000 7fdd02464b80 20 client.18510 trim_cache size 0 max 16384 2024-06-02T23:22:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.287+0000 7fdd02464b80 1 Processor -- start 2024-06-02T23:22:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 -- start start 2024-06-02T23:22:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x56399152f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563991542da0 0x56399152f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639915436c0 0x56399155ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd400d2b0 con 0x56399153d4f0 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd400edf0 con 0x563991542da0 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdd02464b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd400d980 con 0x5639915436c0 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf597d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x56399152f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf517c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563991542da0 0x56399152f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf597d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x56399152f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48576/0 (socket says 172.21.15.52:48576) 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf597d700 1 -- 172.21.15.52:0/361704577 learned_addr learned my addr 172.21.15.52:0/361704577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf617e700 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639915436c0 0x56399155ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.289 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf517c700 1 -- 172.21.15.52:0/361704577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639915436c0 msgr2=0x56399155ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf517c700 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639915436c0 0x56399155ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf517c700 1 -- 172.21.15.52:0/361704577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 msgr2=0x56399152f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf517c700 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x56399152f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.288+0000 7fdcf517c700 1 -- 172.21.15.52:0/361704577 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563991547ca0 con 0x563991542da0 2024-06-02T23:22:39.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.289+0000 7fdcf517c700 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563991542da0 0x56399152f900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdce000b2e0 tx=0x7fdce001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01f82b589016ad0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.289+0000 7fdcdb7fe700 1 -- 172.21.15.52:0/361704577 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce00053c0 con 0x563991542da0 2024-06-02T23:22:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.289+0000 7fdcdb7fe700 1 -- 172.21.15.52:0/361704577 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce0005560 con 0x563991542da0 2024-06-02T23:22:39.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.289+0000 7fdcdb7fe700 1 -- 172.21.15.52:0/361704577 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce000f8c0 con 0x563991542da0 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 -- 172.21.15.52:0/361704577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563991542da0 msgr2=0x56399152f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563991542da0 0x56399152f900 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdce000b2e0 tx=0x7fdce001a040 comp rx=0 tx=0).stop 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 -- 172.21.15.52:0/361704577 shutdown_connections 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639915436c0 0x56399155ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56399153d4f0 0x56399152f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 --2- 172.21.15.52:0/361704577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563991542da0 0x56399152f900 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 -- 172.21.15.52:0/361704577 >> 172.21.15.52:0/361704577 conn(0x563991461210 msgr2=0x563991534020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:39.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 -- 172.21.15.52:0/361704577 shutdown_connections 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 -- 172.21.15.52:0/361704577 wait complete. 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 -- start start 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.290+0000 7fdd02464b80 1 Processor -- start 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56399153d4f0 0x56399154f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563991542da0 0x56399154fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x56399154dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd400e720 con 0x563991542da0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd400e050 con 0x5639915436c0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdcd400b250 con 0x56399153d4f0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdd02464b80 10 client.0 _ll_register_callbacks cb 0x56399141b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf517c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563991542da0 0x56399154fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf617e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x56399154dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf517c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563991542da0 0x56399154fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48578/0 (socket says 172.21.15.52:48578) 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf517c700 1 -- 172.21.15.52:0/3624733113 learned_addr learned my addr 172.21.15.52:0/3624733113 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56399153d4f0 0x56399154f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf617e700 1 -- 172.21.15.52:0/3624733113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56399153d4f0 msgr2=0x56399154f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf617e700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56399153d4f0 0x56399154f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf617e700 1 -- 172.21.15.52:0/3624733113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563991542da0 msgr2=0x56399154fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf617e700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563991542da0 0x56399154fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf617e700 1 -- 172.21.15.52:0/3624733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56399154e550 con 0x5639915436c0 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.291+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56399153d4f0 0x56399154f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcf617e700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x56399154dd10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d7b0 tx=0x7fdce809adc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb93643384e5152a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3624733113 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdce8097160 con 0x5639915436c0 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3624733113 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdce80a38a0 con 0x5639915436c0 2024-06-02T23:22:39.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdce0005560 con 0x5639915436c0 2024-06-02T23:22:39.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3624733113 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fdce8009390 con 0x5639915436c0 2024-06-02T23:22:39.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 10 client.27968 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:39.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639915c2920 con 0x5639915436c0 2024-06-02T23:22:39.298 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3624733113 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdce0005560 con 0x5639915436c0 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3624733113 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fdce809a310 con 0x5639915436c0 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 10 client.27968 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 10 client.27968 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 20 client.27968 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x5639915c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 4 client.27968 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.4, "await_for": 45.0}] 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.292+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 132.4, "await_for": 45.0}) -- 0x56399148d460 con 0x563991556ed0 2024-06-02T23:22:39.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.293+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x5639915c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:39.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.293+0000 7fdcd97fa700 10 client.27968 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:39.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.293+0000 7fdcf597d700 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x5639915c4c60 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b83cd64c66763c server_cookie=dcd8ab0c362c4a23 in_seq=0 out_seq=0 2024-06-02T23:22:39.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.354+0000 7fdcd97fa700 1 -- 172.21.15.52:0/3624733113 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56399148d460 con 0x563991556ed0 2024-06-02T23:22:39.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.354+0000 7fdcd97fa700 10 client.27968 handle_command_reply: tid=0 2024-06-02T23:22:39.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 client.27968 shutdown 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 34, 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bc4c2cd2": { 2024-06-02T23:22:39.356 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 34, 2024-06-02T23:22:39.357 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:22:39.357 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 132.4, 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:39.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 msgr2=0x56399154dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x56399154dd10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fdce800d7b0 tx=0x7fdce809adc0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 shutdown_connections 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563991556ed0 0x5639915c4c60 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639915436c0 0x56399154dd10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56399153d4f0 0x56399154f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 --2- 172.21.15.52:0/3624733113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563991542da0 0x56399154fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdcd97fa700 0 client.27968 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 >> 172.21.15.52:0/3624733113 conn(0x563991461210 msgr2=0x563991552230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 shutdown_connections 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 1 -- 172.21.15.52:0/3624733113 wait complete. 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.355+0000 7fdd02464b80 20 client.27968 trim_cache size 0 max 16384 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdcdc0a2d40 msgr2=0x7fdcdc0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdcdc0a2d40 0x7fdcdc0a5200 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdcec009a70 tx=0x7fdcec009260 comp rx=0 tx=0).stop 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 msgr2=0x7fdcf019aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 0x7fdcf019aad0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fdce8007ee0 tx=0x7fdce800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 shutdown_connections 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdcdc0a2d40 0x7fdcdc0a5200 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdcf010a6d0 0x7fdcf019aad0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf01046d0 0x7fdcf01a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 --2- 172.21.15.52:0/1446088632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdcf0105090 0x7fdcf01a0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:39.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 >> 172.21.15.52:0/1446088632 conn(0x7fdcf01001c0 msgr2=0x7fdcf0100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:39.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 shutdown_connections 2024-06-02T23:22:39.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:39.356+0000 7fdcf7be1700 1 -- 172.21.15.52:0/1446088632 wait complete. 2024-06-02T23:22:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:39 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:39 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:39 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:22:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:39 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:39.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:39 smithi174 ceph-mon[35994]: pgmap v754: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-06-02T23:22:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:39 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:39 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:39 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:22:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:39 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:22:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:39 smithi123 ceph-mon[35977]: pgmap v754: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 1.2 MiB/s wr, 126 op/s 2024-06-02T23:22:39.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.92 MB/sec execute 468 sec latency 272.317 ms 2024-06-02T23:22:40.362 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bc4c2cd2', quiesce took 10.6 seconds. Will release after: 1.8000000000000007 2024-06-02T23:22:40.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.91 MB/sec execute 469 sec latency 1272.356 ms 2024-06-02T23:22:41.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:41 smithi174 ceph-mon[35994]: pgmap v755: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-02T23:22:41.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:41 smithi052 ceph-mon[32340]: pgmap v755: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-02T23:22:41.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:41 smithi123 ceph-mon[35977]: pgmap v755: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 301 KiB/s rd, 1.5 MiB/s wr, 184 op/s 2024-06-02T23:22:41.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.91 MB/sec execute 470 sec latency 2272.430 ms 2024-06-02T23:22:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 -- 172.21.15.52:0/601491705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 msgr2=0x7fda8406a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 --2- 172.21.15.52:0/601491705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8406a5f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fda78009990 tx=0x7fda78009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 -- 172.21.15.52:0/601491705 shutdown_connections 2024-06-02T23:22:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 --2- 172.21.15.52:0/601491705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda8406ab30 0x7fda8410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 --2- 172.21.15.52:0/601491705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda84108500 0x7fda841088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 --2- 172.21.15.52:0/601491705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8406a5f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 -- 172.21.15.52:0/601491705 >> 172.21.15.52:0/601491705 conn(0x7fda84078340 msgr2=0x7fda84078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 -- 172.21.15.52:0/601491705 shutdown_connections 2024-06-02T23:22:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.015+0000 7fda88b81700 1 -- 172.21.15.52:0/601491705 wait complete. 2024-06-02T23:22:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 Processor -- start 2024-06-02T23:22:42.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 -- start start 2024-06-02T23:22:42.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8406ab30 0x7fda8419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda84108500 0x7fda84198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda84112690 con 0x7fda8406a170 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda84112510 con 0x7fda8406ab30 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda88b81700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fda84112810 con 0x7fda84108500 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda8259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda8259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48594/0 (socket says 172.21.15.52:48594) 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.016+0000 7fda8259c700 1 -- 172.21.15.52:0/520778401 learned_addr learned my addr 172.21.15.52:0/520778401 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda81d9b700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8406ab30 0x7fda8419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda82d9d700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda84108500 0x7fda84198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda8259c700 1 -- 172.21.15.52:0/520778401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda84108500 msgr2=0x7fda84198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda8259c700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda84108500 0x7fda84198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda8259c700 1 -- 172.21.15.52:0/520778401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8406ab30 msgr2=0x7fda8419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda8259c700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8406ab30 0x7fda8419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda8259c700 1 -- 172.21.15.52:0/520778401 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda84199090 con 0x7fda8406a170 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda82d9d700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda84108500 0x7fda84198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda81d9b700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8406ab30 0x7fda8419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda8259c700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8419e770 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fda74007e20 tx=0x7fda7400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda737fe700 1 -- 172.21.15.52:0/520778401 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74016020 con 0x7fda8406a170 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda737fe700 1 -- 172.21.15.52:0/520778401 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda7400ae00 con 0x7fda8406a170 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda84199380 con 0x7fda8406a170 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda737fe700 1 -- 172.21.15.52:0/520778401 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74011700 con 0x7fda8406a170 2024-06-02T23:22:42.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.017+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda841a5640 con 0x7fda8406a170 2024-06-02T23:22:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.019+0000 7fda737fe700 1 -- 172.21.15.52:0/520778401 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fda74019020 con 0x7fda8406a170 2024-06-02T23:22:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.020+0000 7fda737fe700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fda6c0a2ca0 0x7fda6c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.020+0000 7fda81d9b700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fda6c0a2ca0 0x7fda6c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.021+0000 7fda737fe700 1 -- 172.21.15.52:0/520778401 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fda740cee20 con 0x7fda8406a170 2024-06-02T23:22:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.021+0000 7fda81d9b700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fda6c0a2ca0 0x7fda6c0a5160 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fda78006010 tx=0x7fda7800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.021+0000 7fda93404b80 1 Processor -- start 2024-06-02T23:22:42.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.021+0000 7fda93404b80 1 -- start start 2024-06-02T23:22:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda93404b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2b370 0x565172e2b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda93404b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 0x565172e25670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda93404b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e25ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda93404b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x565172d13f60 con 0x565172e2bc90 2024-06-02T23:22:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda93404b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565172d72850 con 0x565172e25bb0 2024-06-02T23:22:42.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda93404b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565172cda460 con 0x565172e2b370 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 0x565172e25670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 0x565172e25670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48600/0 (socket says 172.21.15.52:48600) 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 -- 172.21.15.52:0/4175957031 learned_addr learned my addr 172.21.15.52:0/4175957031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda8259c700 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2b370 0x565172e2b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda82d9d700 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e25ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 -- 172.21.15.52:0/4175957031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2b370 msgr2=0x565172e2b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2b370 0x565172e2b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 -- 172.21.15.52:0/4175957031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 msgr2=0x565172e25ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e25ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 -- 172.21.15.52:0/4175957031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565172cc18d0 con 0x565172e2bc90 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda8259c700 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2b370 0x565172e2b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.022+0000 7fda81d9b700 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 0x565172e25670 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fda78000c00 tx=0x7fda7801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b31d1b792b7ba0e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda70ff9700 1 -- 172.21.15.52:0/4175957031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800bad0 con 0x565172e2bc90 2024-06-02T23:22:42.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda70ff9700 1 -- 172.21.15.52:0/4175957031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda78005570 con 0x565172e2bc90 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda70ff9700 1 -- 172.21.15.52:0/4175957031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800fa20 con 0x565172e2bc90 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 -- 172.21.15.52:0/4175957031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 msgr2=0x565172e25670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 0x565172e25670 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fda78000c00 tx=0x7fda7801f000 comp rx=0 tx=0).stop 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 -- 172.21.15.52:0/4175957031 shutdown_connections 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e25ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2b370 0x565172e2b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 --2- 172.21.15.52:0/4175957031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2bc90 0x565172e25670 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.023+0000 7fda93404b80 1 -- 172.21.15.52:0/4175957031 >> 172.21.15.52:0/4175957031 conn(0x565172d4e080 msgr2=0x565172e1c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 -- 172.21.15.52:0/4175957031 shutdown_connections 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 -- 172.21.15.52:0/4175957031 wait complete. 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 -- start start 2024-06-02T23:22:42.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 Processor -- start 2024-06-02T23:22:42.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e31970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e2ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e2f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x565172d12fc0 con 0x565172e25bb0 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565172d6e810 con 0x565172e2b370 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.024+0000 7fda93404b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565172e32620 con 0x565172e2bc90 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda93404b80 10 client.0 _ll_register_callbacks cb 0x565172d085b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e31970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e31970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48608/0 (socket says 172.21.15.52:48608) 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 -- 172.21.15.52:0/3419077670 learned_addr learned my addr 172.21.15.52:0/3419077670 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda81d9b700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e2ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda82d9d700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e2f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 -- 172.21.15.52:0/3419077670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 msgr2=0x565172e2f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e2f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 -- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 msgr2=0x565172e2ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e2ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 -- 172.21.15.52:0/3419077670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565172d38750 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda8259c700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e31970 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fda74000f80 tx=0x7fda740193e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19069ddf9eb39e13 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda69ffb700 1 -- 172.21.15.52:0/3419077670 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740b3020 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda69ffb700 1 -- 172.21.15.52:0/3419077670 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fda7409b8b0 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.025+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda78005570 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda69ffb700 1 -- 172.21.15.52:0/3419077670 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda7409fa00 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 10 client.18528 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565172e3cd30 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda69ffb700 1 -- 172.21.15.52:0/3419077670 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda78005570 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda69ffb700 1 -- 172.21.15.52:0/3419077670 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fda74023060 con 0x565172e25bb0 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 10 client.18528 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 10 client.18528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 20 client.18528 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:42.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172e3f390 0x565172e417c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 4 client.18528 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565172d7fd50 con 0x565172e3f390 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.026+0000 7fda81d9b700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172e3f390 0x565172e417c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.027+0000 7fda81d9b700 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172e3f390 0x565172e417c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04be128ca29d2e1 server_cookie=d06f4aae33660696 in_seq=0 out_seq=0 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.027+0000 7fda69ffb700 10 client.18528 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda69ffb700 1 -- 172.21.15.52:0/3419077670 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565172d7fd50 con 0x565172e3f390 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda69ffb700 10 client.18528 handle_command_reply: tid=0 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 client.18528 shutdown 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 msgr2=0x565172e31970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e31970 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fda74000f80 tx=0x7fda740193e0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 shutdown_connections 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172e3f390 0x565172e417c0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e2f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e31970 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 --2- 172.21.15.52:0/3419077670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e2ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 >> 172.21.15.52:0/3419077670 conn(0x565172d4e080 msgr2=0x565172e1b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda69ffb700 0 client.18528 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 shutdown_connections 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 1 -- 172.21.15.52:0/3419077670 wait complete. 2024-06-02T23:22:42.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.028+0000 7fda93404b80 20 client.18528 trim_cache size 0 max 16384 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.049+0000 7fda93404b80 1 Processor -- start 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.049+0000 7fda93404b80 1 -- start start 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda93404b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e20ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda93404b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 0x565172e1be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda93404b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e1c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda93404b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda841aa800 con 0x565172e2b370 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda93404b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda841ac340 con 0x565172e25bb0 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda93404b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fda841aaed0 con 0x565172e2bc90 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 0x565172e1be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 0x565172e1be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48614/0 (socket says 172.21.15.52:48614) 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 -- 172.21.15.52:0/2396835841 learned_addr learned my addr 172.21.15.52:0/2396835841 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:42.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda8259c700 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e20ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 -- 172.21.15.52:0/2396835841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 msgr2=0x565172e1c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e1c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 -- 172.21.15.52:0/2396835841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 msgr2=0x565172e20ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e20ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 -- 172.21.15.52:0/2396835841 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565172e21020 con 0x565172e2b370 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 0x565172e1be80 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fda78000c00 tx=0x7fda78031040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2b04745db71374f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda6bfff700 1 -- 172.21.15.52:0/2396835841 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda78005530 con 0x565172e2b370 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda6bfff700 1 -- 172.21.15.52:0/2396835841 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda780056d0 con 0x565172e2b370 2024-06-02T23:22:42.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.050+0000 7fda6bfff700 1 -- 172.21.15.52:0/2396835841 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7800fcf0 con 0x565172e2b370 2024-06-02T23:22:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 -- 172.21.15.52:0/2396835841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 msgr2=0x565172e1be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 0x565172e1be80 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7fda78000c00 tx=0x7fda78031040 comp rx=0 tx=0).stop 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 -- 172.21.15.52:0/2396835841 shutdown_connections 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e1c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e25bb0 0x565172e20ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 --2- 172.21.15.52:0/2396835841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e2b370 0x565172e1be80 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 -- 172.21.15.52:0/2396835841 >> 172.21.15.52:0/2396835841 conn(0x565172d4e080 msgr2=0x565172e2c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 -- 172.21.15.52:0/2396835841 shutdown_connections 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 -- 172.21.15.52:0/2396835841 wait complete. 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.051+0000 7fda93404b80 1 -- start start 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 Processor -- start 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e380e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e35480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e35a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fda841abc70 con 0x565172e25bb0 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fda841ab5a0 con 0x565172e2b370 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fda841a87c0 con 0x565172e2bc90 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda93404b80 10 client.0 _ll_register_callbacks cb 0x565172d085b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e380e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e380e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48618/0 (socket says 172.21.15.52:48618) 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 -- 172.21.15.52:0/2859063554 learned_addr learned my addr 172.21.15.52:0/2859063554 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e35480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda82d9d700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e35a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 -- 172.21.15.52:0/2859063554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 msgr2=0x565172e35a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e35a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 -- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 msgr2=0x565172e35480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e35480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 -- 172.21.15.52:0/2859063554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565172e36290 con 0x565172e25bb0 2024-06-02T23:22:42.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e35480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda8259c700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e380e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fda74098020 tx=0x7fda74012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765a4dff6c060c00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.052+0000 7fda69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda69ffb700 1 -- 172.21.15.52:0/2859063554 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda740b3070 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda69ffb700 1 -- 172.21.15.52:0/2859063554 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fda74097470 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda780056d0 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda69ffb700 1 -- 172.21.15.52:0/2859063554 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fda740afe60 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 10 client.18534 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x565172eaa180 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda69ffb700 1 -- 172.21.15.52:0/2859063554 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fda780056d0 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda69ffb700 1 -- 172.21.15.52:0/2859063554 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fda7409a510 con 0x565172e25bb0 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 10 client.18534 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 10 client.18534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:42.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 20 client.18534 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172eaa680 0x565172eacaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 4 client.18534 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.053+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565172d7a2d0 con 0x565172eaa680 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172eaa680 0x565172eacaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda81d9b700 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172eaa680 0x565172eacaa0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65ac1e3477374936 server_cookie=d7fa264f6fed2f82 in_seq=0 out_seq=0 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda69ffb700 10 client.18534 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda69ffb700 1 -- 172.21.15.52:0/2859063554 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x565172d7a2d0 con 0x565172eaa680 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda69ffb700 10 client.18534 handle_command_reply: tid=0 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 client.18534 shutdown 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 msgr2=0x565172e380e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e380e0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fda74098020 tx=0x7fda74012d20 comp rx=0 tx=0).stop 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 shutdown_connections 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565172eaa680 0x565172eacaa0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565172e2bc90 0x565172e35a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565172e25bb0 0x565172e380e0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda69ffb700 0 client.18534 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 --2- 172.21.15.52:0/2859063554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565172e2b370 0x565172e35480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.054+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 >> 172.21.15.52:0/2859063554 conn(0x565172d4e080 msgr2=0x565172e304a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 shutdown_connections 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda93404b80 1 -- 172.21.15.52:0/2859063554 wait complete. 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda93404b80 20 client.18534 trim_cache size 0 max 16384 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:22:42.059 INFO:teuthology.orchestra.run.smithi052.stdout: "1675aa7d-9859-4db8-82de-a8960e113286": { 2024-06-02T23:22:42.060 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:22:42.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "1675aa7d-9859-4db8-82de-a8960e113286", 2024-06-02T23:22:42.060 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:22:42.060 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:42.060 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:42.060 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fda6c0a2ca0 msgr2=0x7fda6c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fda6c0a2ca0 0x7fda6c0a5160 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fda78006010 tx=0x7fda7800b540 comp rx=0 tx=0).stop 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 msgr2=0x7fda8419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8419e770 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fda74007e20 tx=0x7fda7400a340 comp rx=0 tx=0).stop 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 shutdown_connections 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fda6c0a2ca0 0x7fda6c0a5160 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fda84108500 0x7fda84198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fda8406a170 0x7fda8419e770 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 --2- 172.21.15.52:0/520778401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fda8406ab30 0x7fda8419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 >> 172.21.15.52:0/520778401 conn(0x7fda84078340 msgr2=0x7fda84100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:42.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 shutdown_connections 2024-06-02T23:22:42.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:42.055+0000 7fda88b81700 1 -- 172.21.15.52:0/520778401 wait complete. 2024-06-02T23:22:42.163 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:22:42.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.90 MB/sec execute 471 sec latency 3272.489 ms 2024-06-02T23:22:43.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:43 smithi174 ceph-mon[35994]: pgmap v756: 97 pgs: 97 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-02T23:22:43.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:43 smithi052 ceph-mon[32340]: pgmap v756: 97 pgs: 97 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-02T23:22:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:43 smithi123 ceph-mon[35977]: pgmap v756: 97 pgs: 97 active+clean; 91 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-06-02T23:22:43.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.90 MB/sec execute 472 sec latency 4272.540 ms 2024-06-02T23:22:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 -- 172.21.15.52:0/81456379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c104dd0 msgr2=0x7feb9c1071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 --2- 172.21.15.52:0/81456379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c104dd0 0x7feb9c1071c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7feb84009900 tx=0x7feb8402a410 comp rx=0 tx=0).stop 2024-06-02T23:22:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 -- 172.21.15.52:0/81456379 shutdown_connections 2024-06-02T23:22:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 --2- 172.21.15.52:0/81456379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb9c107700 0x7feb9c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 --2- 172.21.15.52:0/81456379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c104dd0 0x7feb9c1071c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 --2- 172.21.15.52:0/81456379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb9c1024a0 0x7feb9c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 -- 172.21.15.52:0/81456379 >> 172.21.15.52:0/81456379 conn(0x7feb9c0fbeb0 msgr2=0x7feb9c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 -- 172.21.15.52:0/81456379 shutdown_connections 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 -- 172.21.15.52:0/81456379 wait complete. 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 Processor -- start 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.493+0000 7feba1510640 1 -- start start 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feba1510640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb9c1024a0 0x7feb9c101e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feba1510640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb9c104dd0 0x7feb9c100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feba1510640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 0x7feb9c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feba1510640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c10d560 con 0x7feb9c107700 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feba1510640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c10d3e0 con 0x7feb9c104dd0 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feba1510640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c10d6e0 con 0x7feb9c1024a0 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 0x7feb9c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 0x7feb9c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42348/0 (socket says 172.21.15.52:42348) 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 -- 172.21.15.52:0/4220887291 learned_addr learned my addr 172.21.15.52:0/4220887291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb93fff640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb9c104dd0 0x7feb9c100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9affd640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb9c1024a0 0x7feb9c101e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 -- 172.21.15.52:0/4220887291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb9c1024a0 msgr2=0x7feb9c101e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb9c1024a0 0x7feb9c101e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 -- 172.21.15.52:0/4220887291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb9c104dd0 msgr2=0x7feb9c100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb9c104dd0 0x7feb9c100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 -- 172.21.15.52:0/4220887291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb9c101220 con 0x7feb9c107700 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb9b7fe640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 0x7feb9c100990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7feb94009fd0 tx=0x7feb9400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb98ff9640 1 -- 172.21.15.52:0/4220887291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb94017070 con 0x7feb9c107700 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.494+0000 7feb98ff9640 1 -- 172.21.15.52:0/4220887291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb94003e20 con 0x7feb9c107700 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.495+0000 7feb98ff9640 1 -- 172.21.15.52:0/4220887291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb940106c0 con 0x7feb9c107700 2024-06-02T23:22:44.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.495+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb9c1a1cf0 con 0x7feb9c107700 2024-06-02T23:22:44.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.495+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb9c1a21c0 con 0x7feb9c107700 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.497+0000 7feb98ff9640 1 -- 172.21.15.52:0/4220887291 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7feb94003970 con 0x7feb9c107700 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.497+0000 7feba4218740 1 Processor -- start 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.497+0000 7feba4218740 1 -- start start 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feba4218740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e49430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feba4218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e4e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feba4218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e4eb70 0x560d04e51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feba4218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560d04dac4f0 con 0x560d04e4eb70 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feba4218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560d04d4a870 con 0x560d04e49a00 2024-06-02T23:22:44.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feba4218740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d04da84b0 con 0x560d04e49010 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb98ff9640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb880a2c30 0x7feb880a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e4eb70 0x560d04e51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb93fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e49430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb9affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e4e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb93fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e49430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33596/0 (socket says 172.21.15.52:33596) 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb93fff640 1 -- 172.21.15.52:0/2372304962 learned_addr learned my addr 172.21.15.52:0/2372304962 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb9b7fe640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb880a2c30 0x7feb880a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb98ff9640 1 -- 172.21.15.52:0/4220887291 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb940cee20 con 0x7feb9c107700 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb9affd640 1 -- 172.21.15.52:0/2372304962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 msgr2=0x560d04e49430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb9affd640 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e49430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb9affd640 1 -- 172.21.15.52:0/2372304962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e4eb70 msgr2=0x560d04e51000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.498+0000 7feb9affd640 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e4eb70 0x560d04e51000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb9affd640 1 -- 172.21.15.52:0/2372304962 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d04d669b0 con 0x560d04e49a00 2024-06-02T23:22:44.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb93fff640 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e49430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb9b7fe640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb880a2c30 0x7feb880a50f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feb94098020 tx=0x7feb9406a5d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb9affd640 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e4e500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00cb10 tx=0x7feb8c00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=623e0f37a7bb798f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb917fa640 1 -- 172.21.15.52:0/2372304962 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004020 con 0x560d04e49a00 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb917fa640 1 -- 172.21.15.52:0/2372304962 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c0041c0 con 0x560d04e49a00 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.499+0000 7feb917fa640 1 -- 172.21.15.52:0/2372304962 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c010690 con 0x560d04e49a00 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 -- 172.21.15.52:0/2372304962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 msgr2=0x560d04e4e500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e4e500 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00cb10 tx=0x7feb8c00e350 comp rx=0 tx=0).stop 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 -- 172.21.15.52:0/2372304962 shutdown_connections 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e4eb70 0x560d04e51000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e4e500 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 --2- 172.21.15.52:0/2372304962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e49430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 -- 172.21.15.52:0/2372304962 >> 172.21.15.52:0/2372304962 conn(0x560d04e44680 msgr2=0x560d04e44a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:44.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 -- 172.21.15.52:0/2372304962 shutdown_connections 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 -- 172.21.15.52:0/2372304962 wait complete. 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 -- start start 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.500+0000 7feba4218740 1 Processor -- start 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e53340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e53880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e4eb70 0x560d04e5a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560d04e51b20 con 0x560d04e49010 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560d04d35ae0 con 0x560d04e49a00 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560d04e51ca0 con 0x560d04e4eb70 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feba4218740 10 client.0 _ll_register_callbacks cb 0x560d04d6e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e53340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e53340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42370/0 (socket says 172.21.15.52:42370) 2024-06-02T23:22:44.506 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 -- 172.21.15.52:0/1766962925 learned_addr learned my addr 172.21.15.52:0/1766962925 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb9affd640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e53880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 -- 172.21.15.52:0/1766962925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e4eb70 msgr2=0x560d04e5a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e4eb70 0x560d04e5a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 -- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 msgr2=0x560d04e53880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e53880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 -- 172.21.15.52:0/1766962925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d04ac0210 con 0x560d04e49010 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb9affd640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb93fff640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e53340 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7feb84006010 tx=0x7feb840092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27ac4dc4239a72a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.501+0000 7feb827fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feb827fc640 1 -- 172.21.15.52:0/1766962925 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb84035040 con 0x560d04e49010 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feb827fc640 1 -- 172.21.15.52:0/1766962925 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb840046c0 con 0x560d04e49010 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c0041c0 con 0x560d04e49010 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feb827fc640 1 -- 172.21.15.52:0/1766962925 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb840318b0 con 0x560d04e49010 2024-06-02T23:22:44.507 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 10 client.18546 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d04b8c620 con 0x560d04e49010 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feb827fc640 1 -- 172.21.15.52:0/1766962925 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c0041c0 con 0x560d04e49010 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feb827fc640 1 -- 172.21.15.52:0/1766962925 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feb8403d860 con 0x560d04e49010 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 10 client.18546 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 10 client.18546 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 20 client.18546 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04e5f150 0x560d04e615c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 4 client.18546 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d04dbff10 con 0x560d04e5f150 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.502+0000 7feb9affd640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04e5f150 0x560d04e615c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.503+0000 7feb827fc640 10 client.18546 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:44.508 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.503+0000 7feb9affd640 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04e5f150 0x560d04e615c0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86d752212946f665 server_cookie=de644ba31bb5fc5f in_seq=0 out_seq=0 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feb827fc640 1 -- 172.21.15.52:0/1766962925 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560d04dbff10 con 0x560d04e5f150 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feb827fc640 10 client.18546 handle_command_reply: tid=0 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 client.18546 shutdown 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 msgr2=0x560d04e53340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e53340 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7feb84006010 tx=0x7feb840092d0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 shutdown_connections 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04e5f150 0x560d04e615c0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e4eb70 0x560d04e5a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e49a00 0x560d04e53880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feb827fc640 0 client.18546 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 --2- 172.21.15.52:0/1766962925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e53340 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 >> 172.21.15.52:0/1766962925 conn(0x560d04e44680 msgr2=0x560d04e437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 shutdown_connections 2024-06-02T23:22:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 1 -- 172.21.15.52:0/1766962925 wait complete. 2024-06-02T23:22:44.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.504+0000 7feba4218740 20 client.18546 trim_cache size 0 max 16384 2024-06-02T23:22:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.519+0000 7feba4218740 1 Processor -- start 2024-06-02T23:22:44.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.519+0000 7feba4218740 1 -- start start 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feba4218740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e53660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feba4218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 0x560d04e5a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feba4218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e47590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feba4218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1aaeb0 con 0x560d04e49a00 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feba4218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1ac9f0 con 0x560d04e4eb70 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feba4218740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1ab580 con 0x560d04e49010 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 0x560d04e5a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 0x560d04e5a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42380/0 (socket says 172.21.15.52:42380) 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 -- 172.21.15.52:0/1969722475 learned_addr learned my addr 172.21.15.52:0/1969722475 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:44.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb93fff640 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e47590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb93fff640 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e53660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 -- 172.21.15.52:0/1969722475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 msgr2=0x560d04e53660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e53660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 -- 172.21.15.52:0/1969722475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 msgr2=0x560d04e47590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e47590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 -- 172.21.15.52:0/1969722475 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d04e4dfc0 con 0x560d04e49a00 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.520+0000 7feb9affd640 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 0x560d04e5a850 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00ca60 tx=0x7feb8c005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4b73ba0807e2b0c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feb90ff9640 1 -- 172.21.15.52:0/1969722475 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c004090 con 0x560d04e49a00 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feb90ff9640 1 -- 172.21.15.52:0/1969722475 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c0054f0 con 0x560d04e49a00 2024-06-02T23:22:44.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feb90ff9640 1 -- 172.21.15.52:0/1969722475 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8c003dc0 con 0x560d04e49a00 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 -- 172.21.15.52:0/1969722475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 msgr2=0x560d04e5a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 0x560d04e5a850 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00ca60 tx=0x7feb8c005fa0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 -- 172.21.15.52:0/1969722475 shutdown_connections 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e47590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49a00 0x560d04e5a850 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 --2- 172.21.15.52:0/1969722475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49010 0x560d04e53660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 -- 172.21.15.52:0/1969722475 >> 172.21.15.52:0/1969722475 conn(0x560d04e44680 msgr2=0x560d04e52f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 -- 172.21.15.52:0/1969722475 shutdown_connections 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.521+0000 7feba4218740 1 -- 172.21.15.52:0/1969722475 wait complete. 2024-06-02T23:22:44.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 -- start start 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 Processor -- start 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e57b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49a00 0x560d04e58050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e59060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1ac320 con 0x560d04e49010 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1abc50 con 0x560d04e4eb70 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb9c1a8e90 con 0x560d04e49a00 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feba4218740 10 client.0 _ll_register_callbacks cb 0x7feb84031580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e57b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e57b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42382/0 (socket says 172.21.15.52:42382) 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 -- 172.21.15.52:0/2245225812 learned_addr learned my addr 172.21.15.52:0/2245225812 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb9affd640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49a00 0x560d04e58050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e59060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 -- 172.21.15.52:0/2245225812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49a00 msgr2=0x560d04e58050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49a00 0x560d04e58050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 -- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 msgr2=0x560d04e59060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e59060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.522+0000 7feb93fff640 1 -- 172.21.15.52:0/2245225812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d04e59650 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb93fff640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e57b10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7feb8402a450 tx=0x7feb840058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d8d80c273ff2e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb827fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb827fc640 1 -- 172.21.15.52:0/2245225812 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb8405b030 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb827fc640 1 -- 172.21.15.52:0/2245225812 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7feb84005480 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb8c0054f0 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb827fc640 1 -- 172.21.15.52:0/2245225812 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7feb84035070 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 10 client.18552 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560d04e5a960 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb827fc640 1 -- 172.21.15.52:0/2245225812 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7feb8c0054f0 con 0x560d04e49010 2024-06-02T23:22:44.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feb827fc640 1 -- 172.21.15.52:0/2245225812 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7feb84062580 con 0x560d04e49010 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 10 client.18552 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 10 client.18552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 20 client.18552 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04eca380 0x560d04ecc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 4 client.18552 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.523+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d04db9c20 con 0x560d04eca380 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.524+0000 7feb9affd640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04eca380 0x560d04ecc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.524+0000 7feb9affd640 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04eca380 0x560d04ecc7a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6b25988136a7d4 server_cookie=14c0087cc3dac212 in_seq=0 out_seq=0 2024-06-02T23:22:44.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.524+0000 7feb827fc640 10 client.18552 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.543+0000 7feb827fc640 1 -- 172.21.15.52:0/2245225812 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x560d04db9c20 con 0x560d04eca380 2024-06-02T23:22:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.543+0000 7feb827fc640 10 client.18552 handle_command_reply: tid=0 2024-06-02T23:22:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.543+0000 7feba4218740 1 client.18552 shutdown 2024-06-02T23:22:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.543+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 msgr2=0x560d04e57b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e57b10 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7feb8402a450 tx=0x7feb840058b0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.544 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 shutdown_connections 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560d04eca380 0x560d04ecc7a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560d04e49a00 0x560d04e58050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560d04e4eb70 0x560d04e59060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 --2- 172.21.15.52:0/2245225812 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560d04e49010 0x560d04e57b10 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feb827fc640 0 client.18552 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 >> 172.21.15.52:0/2245225812 conn(0x560d04e44680 msgr2=0x560d04e47590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 shutdown_connections 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 1 -- 172.21.15.52:0/2245225812 wait complete. 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba4218740 20 client.18552 trim_cache size 0 max 16384 2024-06-02T23:22:44.545 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:22:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:22:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:22:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb880a2c30 msgr2=0x7feb880a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.546 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb880a2c30 0x7feb880a50f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7feb94098020 tx=0x7feb9406a5d0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 msgr2=0x7feb9c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 0x7feb9c100990 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7feb94009fd0 tx=0x7feb9400b260 comp rx=0 tx=0).stop 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 shutdown_connections 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7feb880a2c30 0x7feb880a50f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7feb9c107700 0x7feb9c100990 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb9c104dd0 0x7feb9c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 --2- 172.21.15.52:0/4220887291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb9c1024a0 0x7feb9c101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.544+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 >> 172.21.15.52:0/4220887291 conn(0x7feb9c0fbeb0 msgr2=0x7feb9c102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.545+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 shutdown_connections 2024-06-02T23:22:44.547 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:44.545+0000 7feba1510640 1 -- 172.21.15.52:0/4220887291 wait complete. 2024-06-02T23:22:44.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.90 MB/sec execute 473 sec latency 5272.601 ms 2024-06-02T23:22:45.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:45 smithi174 ceph-mon[35994]: pgmap v757: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1017 KiB/s wr, 151 op/s 2024-06-02T23:22:45.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:45 smithi052 ceph-mon[32340]: pgmap v757: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1017 KiB/s wr, 151 op/s 2024-06-02T23:22:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:45 smithi123 ceph-mon[35977]: pgmap v757: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 1017 KiB/s wr, 151 op/s 2024-06-02T23:22:45.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.89 MB/sec execute 474 sec latency 6272.705 ms 2024-06-02T23:22:45.686 INFO:tasks.cephfs.filesystem:scrub status for tag:1675aa7d-9859-4db8-82de-a8960e113286 - {'path': '/', 'tag': '1675aa7d-9859-4db8-82de-a8960e113286', 'options': 'recursive,force'} 2024-06-02T23:22:45.760 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:46.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:46 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:22:46.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:46 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:22:46.265 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:46 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:22:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:46 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:22:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:46 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:22:46.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:46 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:22:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:46 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:22:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:46 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:22:46.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:46 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:22:46.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.89 MB/sec execute 475 sec latency 7272.757 ms 2024-06-02T23:22:47.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:47 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:22:47.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:47 smithi052 ceph-mon[32340]: pgmap v758: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 913 KiB/s wr, 133 op/s 2024-06-02T23:22:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:47 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:22:47.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:47 smithi174 ceph-mon[35994]: pgmap v758: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 913 KiB/s wr, 133 op/s 2024-06-02T23:22:47.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 -- 172.21.15.52:0/1459461748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 msgr2=0x7f8c78101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/1459461748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c78101470 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f8c68009990 tx=0x7f8c68009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 -- 172.21.15.52:0/1459461748 shutdown_connections 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/1459461748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78105f30 0x7f8c7810da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/1459461748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c78101470 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/1459461748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78101a40 0x7f8c78105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 -- 172.21.15.52:0/1459461748 >> 172.21.15.52:0/1459461748 conn(0x7f8c780766b0 msgr2=0x7f8c78076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 -- 172.21.15.52:0/1459461748 shutdown_connections 2024-06-02T23:22:47.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.506+0000 7f8c7df1c700 1 -- 172.21.15.52:0/1459461748 wait complete. 2024-06-02T23:22:47.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:47 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:22:47.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:47 smithi123 ceph-mon[35977]: pgmap v758: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 913 KiB/s wr, 133 op/s 2024-06-02T23:22:47.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 Processor -- start 2024-06-02T23:22:47.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 -- start start 2024-06-02T23:22:47.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c7819adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:47.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78101a40 0x7f8c7819b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78105f30 0x7f8c7819f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781104f0 con 0x7f8c78101090 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78110370 con 0x7f8c78105f30 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c7df1c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78110670 con 0x7f8c78101a40 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c777fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c7819adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c777fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c7819adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42400/0 (socket says 172.21.15.52:42400) 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.507+0000 7f8c777fe700 1 -- 172.21.15.52:0/4283771515 learned_addr learned my addr 172.21.15.52:0/4283771515 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c77fff700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78105f30 0x7f8c7819f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c76ffd700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78101a40 0x7f8c7819b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c777fe700 1 -- 172.21.15.52:0/4283771515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78101a40 msgr2=0x7f8c7819b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c777fe700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78101a40 0x7f8c7819b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c777fe700 1 -- 172.21.15.52:0/4283771515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78105f30 msgr2=0x7f8c7819f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:47.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c777fe700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78105f30 0x7f8c7819f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c777fe700 1 -- 172.21.15.52:0/4283771515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c7819fdd0 con 0x7f8c78101090 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c777fe700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c7819adc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c00 tx=0x7f8c68003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68027070 con 0x7f8c78101090 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8c6800bdc0 con 0x7f8c78101090 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c781a0060 con 0x7f8c78101090 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68022cd0 con 0x7f8c78101090 2024-06-02T23:22:47.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.508+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a7940 con 0x7f8c78101090 2024-06-02T23:22:47.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.510+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c78069180 con 0x7f8c78101090 2024-06-02T23:22:47.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.511+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8c6802a070 con 0x7f8c78101090 2024-06-02T23:22:47.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.511+0000 7f8c74ff9700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2c20 0x7f8c600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:47.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.511+0000 7f8c76ffd700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2c20 0x7f8c600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:47.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.511+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8c680cfd20 con 0x7f8c78101090 2024-06-02T23:22:47.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.512+0000 7f8c76ffd700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2c20 0x7f8c600a50e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8c7810df20 tx=0x7f8c6c00b510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:47.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.516+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8c68013070 con 0x7f8c78101090 2024-06-02T23:22:47.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.88 MB/sec execute 476 sec latency 8272.823 ms 2024-06-02T23:22:47.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.769+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c781a7b10 con 0x7f8c78101090 2024-06-02T23:22:47.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.770+0000 7f8c74ff9700 1 -- 172.21.15.52:0/4283771515 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f8c68098030 con 0x7f8c78101090 2024-06-02T23:22:47.770 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:22:47.770 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:22:47.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2c20 msgr2=0x7f8c600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:47.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2c20 0x7f8c600a50e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f8c7810df20 tx=0x7f8c6c00b510 comp rx=0 tx=0).stop 2024-06-02T23:22:47.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 msgr2=0x7f8c7819adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:47.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c7819adc0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f8c68000c00 tx=0x7f8c68003a80 comp rx=0 tx=0).stop 2024-06-02T23:22:47.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 shutdown_connections 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8c600a2c20 0x7f8c600a50e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8c78105f30 0x7f8c7819f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8c78101090 0x7f8c7819adc0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 --2- 172.21.15.52:0/4283771515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8c78101a40 0x7f8c7819b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 >> 172.21.15.52:0/4283771515 conn(0x7f8c780766b0 msgr2=0x7f8c780fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 shutdown_connections 2024-06-02T23:22:47.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:47.772+0000 7f8c7df1c700 1 -- 172.21.15.52:0/4283771515 wait complete. 2024-06-02T23:22:47.774 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:22:48.386 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:48 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4283771515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:48.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:48 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4283771515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:48.416 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id bc4c2cd2 --release --await' 2024-06-02T23:22:48.417 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id bc4c2cd2 --release --await 2024-06-02T23:22:48.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:48 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4283771515' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:22:48.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.88 MB/sec execute 477 sec latency 9272.865 ms 2024-06-02T23:22:48.672 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:22:49.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:49 smithi174 ceph-mon[35994]: pgmap v759: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 615 KiB/s wr, 92 op/s 2024-06-02T23:22:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:49 smithi052 ceph-mon[32340]: pgmap v759: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 615 KiB/s wr, 92 op/s 2024-06-02T23:22:49.408 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:49 smithi123 ceph-mon[35977]: pgmap v759: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 615 KiB/s wr, 92 op/s 2024-06-02T23:22:49.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214292 1.88 MB/sec execute 478 sec latency 10272.956 ms 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 -- 172.21.15.52:0/77111798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f081084f0 msgr2=0x7f2f081088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 --2- 172.21.15.52:0/77111798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f081084f0 0x7f2f081088d0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8009990 tx=0x7f2ef8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 -- 172.21.15.52:0/77111798 shutdown_connections 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 --2- 172.21.15.52:0/77111798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 0x7f2f0810f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 --2- 172.21.15.52:0/77111798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f081084f0 0x7f2f081088d0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 --2- 172.21.15.52:0/77111798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0806aa00 0x7f2f0806ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 -- 172.21.15.52:0/77111798 >> 172.21.15.52:0/77111798 conn(0x7f2f08078300 msgr2=0x7f2f08078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:50.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 -- 172.21.15.52:0/77111798 shutdown_connections 2024-06-02T23:22:50.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.475+0000 7f2f0e646700 1 -- 172.21.15.52:0/77111798 wait complete. 2024-06-02T23:22:50.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.476+0000 7f2f0e646700 1 Processor -- start 2024-06-02T23:22:50.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.476+0000 7f2f0e646700 1 -- start start 2024-06-02T23:22:50.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.476+0000 7f2f0e646700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 0x7f2f0819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.476+0000 7f2f0e646700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 0x7f2f0819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.476+0000 7f2f0e646700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f081084f0 0x7f2f081989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f0e646700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f08112620 con 0x7f2f0806aa00 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f0e646700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f081124a0 con 0x7f2f0806b3c0 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f0e646700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f081127a0 con 0x7f2f081084f0 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 0x7f2f0819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 0x7f2f0819e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42420/0 (socket says 172.21.15.52:42420) 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 -- 172.21.15.52:0/846868364 learned_addr learned my addr 172.21.15.52:0/846868364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2efffff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 0x7f2f0819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f081084f0 0x7f2f081989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 -- 172.21.15.52:0/846868364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f081084f0 msgr2=0x7f2f081989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f081084f0 0x7f2f081989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 -- 172.21.15.52:0/846868364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 msgr2=0x7f2f0819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 0x7f2f0819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 -- 172.21.15.52:0/846868364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f08199100 con 0x7f2f0806aa00 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2efffff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 0x7f2f0819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.477+0000 7f2f07fff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 0x7f2f0819e7e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8000c00 tx=0x7f2ef80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.478+0000 7f2f05ffb700 1 -- 172.21.15.52:0/846868364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8027070 con 0x7f2f0806aa00 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.478+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f08199390 con 0x7f2f0806aa00 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.478+0000 7f2f05ffb700 1 -- 172.21.15.52:0/846868364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef8022470 con 0x7f2f0806aa00 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.478+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f081a56b0 con 0x7f2f0806aa00 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.478+0000 7f2f05ffb700 1 -- 172.21.15.52:0/846868364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8017650 con 0x7f2f0806aa00 2024-06-02T23:22:50.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.480+0000 7f2f05ffb700 1 -- 172.21.15.52:0/846868364 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ef802a070 con 0x7f2f0806aa00 2024-06-02T23:22:50.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.481+0000 7f2f05ffb700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ef40a2ca0 0x7f2ef40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.481+0000 7f2efffff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ef40a2ca0 0x7f2ef40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.481+0000 7f2f05ffb700 1 -- 172.21.15.52:0/846868364 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ef80cfcf0 con 0x7f2f0806aa00 2024-06-02T23:22:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 Processor -- start 2024-06-02T23:22:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2efffff700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ef40a2ca0 0x7f2ef40a5160 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2f08078970 tx=0x7f2ef000b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 -- start start 2024-06-02T23:22:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abe9fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abea0480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5566abd8e060 con 0x5566abea0060 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5566abdec950 con 0x5566abea6230 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566abd54560 con 0x5566abea5910 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abea0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abea0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42422/0 (socket says 172.21.15.52:42422) 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/559469603 learned_addr learned my addr 172.21.15.52:0/559469603 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2efffff700 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abe9fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f07fff700 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/559469603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 msgr2=0x5566abea5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/559469603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 msgr2=0x5566abe9fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abe9fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.482+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/559469603 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566abd3b9d0 con 0x5566abea0060 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.483+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abea0480 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009440 tx=0x7f2f00009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7e04dbf8ecca21 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:50.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.483+0000 7f2efe7fc700 1 -- 172.21.15.52:0/559469603 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00013750 con 0x5566abea0060 2024-06-02T23:22:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.483+0000 7f2efe7fc700 1 -- 172.21.15.52:0/559469603 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f00013da0 con 0x5566abea0060 2024-06-02T23:22:50.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.483+0000 7f2efe7fc700 1 -- 172.21.15.52:0/559469603 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f000143e0 con 0x5566abea0060 2024-06-02T23:22:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.483+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/559469603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 msgr2=0x5566abea0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.483+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abea0480 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009440 tx=0x7f2f00009920 comp rx=0 tx=0).stop 2024-06-02T23:22:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/559469603 shutdown_connections 2024-06-02T23:22:50.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abea0480 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/559469603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abe9fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/559469603 >> 172.21.15.52:0/559469603 conn(0x5566abdc8180 msgr2=0x5566abe96a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/559469603 shutdown_connections 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/559469603 wait complete. 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 -- start start 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.484+0000 7f2f18ec9b80 1 Processor -- start 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abeabec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abea9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5566abd8d0c0 con 0x5566abea0060 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5566abde8910 con 0x5566abea6230 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566abeacc10 con 0x5566abea5910 2024-06-02T23:22:50.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f18ec9b80 10 client.0 _ll_register_callbacks cb 0x5566abd826b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abeabec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abeabec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42430/0 (socket says 172.21.15.52:42430) 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 -- 172.21.15.52:0/2409329955 learned_addr learned my addr 172.21.15.52:0/2409329955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abea9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2efffff700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 -- 172.21.15.52:0/2409329955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 msgr2=0x5566abea91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 -- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 msgr2=0x5566abea9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abea9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 -- 172.21.15.52:0/2409329955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566abdb2850 con 0x5566abea0060 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2f07fff700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abeabec0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80cd820 tx=0x7f2ef809cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21289ddd5ebec8e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:50.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.485+0000 7f2ee77fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2ee77fe700 1 -- 172.21.15.52:0/2409329955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef809bf70 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2ee77fe700 1 -- 172.21.15.52:0/2409329955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ef8085da0 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f00013da0 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2ee77fe700 1 -- 172.21.15.52:0/2409329955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2ef809cd10 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 10 client.18570 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566abeb7320 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2ee77fe700 1 -- 172.21.15.52:0/2409329955 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f00013da0 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2ee77fe700 1 -- 172.21.15.52:0/2409329955 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2ef80cdce0 con 0x5566abea0060 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 10 client.18570 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 10 client.18570 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 20 client.18570 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abebbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 4 client.18570 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:50.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.486+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566abdf9e50 con 0x5566abeb9890 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.487+0000 7f2efffff700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abebbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.487+0000 7f2efffff700 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abebbcb0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21f64a9dd960d029 server_cookie=90a999fc1ad4d6c6 in_seq=0 out_seq=0 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.487+0000 7f2ee77fe700 10 client.18570 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2ee77fe700 1 -- 172.21.15.52:0/2409329955 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5566abdf9e50 con 0x5566abeb9890 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2ee77fe700 10 client.18570 handle_command_reply: tid=0 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 client.18570 shutdown 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 msgr2=0x5566abeabec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abeabec0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f2ef80cd820 tx=0x7f2ef809cb50 comp rx=0 tx=0).stop 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 shutdown_connections 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abebbcb0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2ee77fe700 0 client.18570 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea6230 0x5566abea9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea0060 0x5566abeabec0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2409329955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abea91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 >> 172.21.15.52:0/2409329955 conn(0x5566abdc8180 msgr2=0x5566abe95bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:50.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 shutdown_connections 2024-06-02T23:22:50.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2409329955 wait complete. 2024-06-02T23:22:50.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.488+0000 7f2f18ec9b80 20 client.18570 trim_cache size 0 max 16384 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 Processor -- start 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 -- start start 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abe9af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abe96330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abe96870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800d2a0 con 0x5566abea6230 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800ede0 con 0x5566abea0060 2024-06-02T23:22:50.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800d970 con 0x5566abea5910 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abe96870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abe96870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42432/0 (socket says 172.21.15.52:42432) 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/2747268055 learned_addr learned my addr 172.21.15.52:0/2747268055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f07fff700 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abe9af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/2747268055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 msgr2=0x5566abe96330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2efffff700 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abe96330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abe96330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/2747268055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 msgr2=0x5566abe9af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abe9af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/2747268055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566abe9b4d0 con 0x5566abea6230 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f07fff700 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abe9af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.509+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abe96870 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2f00000f80 tx=0x7f2f00004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=588b2ebad126947a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2efdffb700 1 -- 172.21.15.52:0/2747268055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00009e60 con 0x5566abea6230 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2efdffb700 1 -- 172.21.15.52:0/2747268055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f00013a00 con 0x5566abea6230 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2efdffb700 1 -- 172.21.15.52:0/2747268055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00015a40 con 0x5566abea6230 2024-06-02T23:22:50.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2747268055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 msgr2=0x5566abe96870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abe96870 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f2f00000f80 tx=0x7f2f00004b40 comp rx=0 tx=0).stop 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2747268055 shutdown_connections 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abe96870 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abe9af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/2747268055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abe96330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2747268055 >> 172.21.15.52:0/2747268055 conn(0x5566abdc8180 msgr2=0x5566abea6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2747268055 shutdown_connections 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.510+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/2747268055 wait complete. 2024-06-02T23:22:50.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 -- start start 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 Processor -- start 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abeb2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abeb26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abeaef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800e710 con 0x5566abea6230 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800e040 con 0x5566abea0060 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee800b240 con 0x5566abea5910 2024-06-02T23:22:50.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f18ec9b80 10 client.0 _ll_register_callbacks cb 0x5566abd826b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f0cbe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abeaef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f0cbe3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abeaef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42444/0 (socket says 172.21.15.52:42444) 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/592539604 learned_addr learned my addr 172.21.15.52:0/592539604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.511+0000 7f2efffff700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abeb26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f07fff700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abeb2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/592539604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 msgr2=0x5566abeb26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abeb26c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 msgr2=0x5566abeb2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abeb2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f0cbe3700 1 -- 172.21.15.52:0/592539604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566abeaf6f0 con 0x5566abea6230 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f0cbe3700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abeaef00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009960 tx=0x7f2f00015440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5aa5be6104b0d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2ee77fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:50.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2ee77fe700 1 -- 172.21.15.52:0/592539604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f00013db0 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2ee77fe700 1 -- 172.21.15.52:0/592539604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2f00015680 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f00013a00 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2ee77fe700 1 -- 172.21.15.52:0/592539604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2f0003a070 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f18ec9b80 10 client.18576 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.512+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5566abf250c0 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2ee77fe700 1 -- 172.21.15.52:0/592539604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2f00013a00 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2ee77fe700 1 -- 172.21.15.52:0/592539604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2f00020ac0 con 0x5566abea6230 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f18ec9b80 10 client.18576 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f18ec9b80 10 client.18576 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f18ec9b80 20 client.18576 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abf27520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f18ec9b80 4 client.18576 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bc4c2cd2", "release": true, "await": true}] 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bc4c2cd2", "release": true, "await": true}) -- 0x5566abdf43d0 con 0x5566abeb9890 2024-06-02T23:22:50.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f07fff700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abf27520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:50.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2f07fff700 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abf27520 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c5c5acfd545eb9 server_cookie=a62e7ce2535a76fa in_seq=0 out_seq=0 2024-06-02T23:22:50.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.513+0000 7f2ee77fe700 10 client.18576 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2ee77fe700 1 -- 172.21.15.52:0/592539604 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5566abdf43d0 con 0x5566abeb9890 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2ee77fe700 10 client.18576 handle_command_reply: tid=0 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 client.18576 shutdown 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 msgr2=0x5566abeaef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abeaef00 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009960 tx=0x7f2f00015440 comp rx=0 tx=0).stop 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 shutdown_connections 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5566abeb9890 0x5566abf27520 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5566abea6230 0x5566abeaef00 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5566abea0060 0x5566abeb2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 --2- 172.21.15.52:0/592539604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566abea5910 0x5566abeb26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 >> 172.21.15.52:0/592539604 conn(0x5566abdc8180 msgr2=0x5566abeaa9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2ee77fe700 0 client.18576 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 shutdown_connections 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.520+0000 7f2f18ec9b80 1 -- 172.21.15.52:0/592539604 wait complete. 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f18ec9b80 20 client.18576 trim_cache size 0 max 16384 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:22:50.522 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 36, 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "bc4c2cd2": { 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 36, 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:22:50.523 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 132.4, 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:22:50.524 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:22:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ef40a2ca0 msgr2=0x7f2ef40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ef40a2ca0 0x7f2ef40a5160 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f2f08078970 tx=0x7f2ef000b2c0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 msgr2=0x7f2f0819e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 0x7f2f0819e7e0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8000c00 tx=0x7f2ef80047b0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 shutdown_connections 2024-06-02T23:22:50.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ef40a2ca0 0x7f2ef40a5160 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f081084f0 0x7f2f081989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0806aa00 0x7f2f0819e7e0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 --2- 172.21.15.52:0/846868364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0806b3c0 0x7f2f0819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 >> 172.21.15.52:0/846868364 conn(0x7f2f08078300 msgr2=0x7f2f081000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 shutdown_connections 2024-06-02T23:22:50.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:22:50.521+0000 7f2f0e646700 1 -- 172.21.15.52:0/846868364 wait complete. 2024-06-02T23:22:50.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214349 1.87 MB/sec execute 479 sec latency 11152.366 ms 2024-06-02T23:22:51.261 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bc4c2cd2', total seconds elapsed: 21.5 2024-06-02T23:22:51.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:51 smithi174 ceph-mon[35994]: pgmap v760: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 488 KiB/s wr, 92 op/s 2024-06-02T23:22:51.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:51 smithi052 ceph-mon[32340]: pgmap v760: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 488 KiB/s wr, 92 op/s 2024-06-02T23:22:51.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:51 smithi123 ceph-mon[35977]: pgmap v760: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 488 KiB/s wr, 92 op/s 2024-06-02T23:22:51.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 214973 1.87 MB/sec execute 480 sec latency 15.987 ms 2024-06-02T23:22:52.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 215481 1.87 MB/sec execute 481 sec latency 14.602 ms 2024-06-02T23:22:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:53 smithi174 ceph-mon[35994]: pgmap v761: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 74 KiB/s wr, 31 op/s 2024-06-02T23:22:53.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:53 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:22:53.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:53 smithi052 ceph-mon[32340]: pgmap v761: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 74 KiB/s wr, 31 op/s 2024-06-02T23:22:53.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:53 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:22:53.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:53 smithi123 ceph-mon[35977]: pgmap v761: 97 pgs: 97 active+clean; 89 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 74 KiB/s wr, 31 op/s 2024-06-02T23:22:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:53 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:22:53.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 216189 1.88 MB/sec execute 482 sec latency 13.481 ms 2024-06-02T23:22:54.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:22:54.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:22:54.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:22:54.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:22:54.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:22:54.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:22:54.506 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:22:54.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:22:54.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:22:54.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 216664 1.89 MB/sec execute 483 sec latency 15.184 ms 2024-06-02T23:22:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:55 smithi174 ceph-mon[35994]: pgmap v762: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-06-02T23:22:55.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 216931 1.88 MB/sec execute 484 sec latency 15.863 ms 2024-06-02T23:22:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:55 smithi052 ceph-mon[32340]: pgmap v762: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-06-02T23:22:55.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:55 smithi123 ceph-mon[35977]: pgmap v762: 97 pgs: 97 active+clean; 90 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 55 KiB/s wr, 36 op/s 2024-06-02T23:22:56.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 217366 1.88 MB/sec execute 485 sec latency 15.056 ms 2024-06-02T23:22:57.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 -- 172.21.15.52:0/3212303569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8104db0 msgr2=0x7fd0e81071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 --2- 172.21.15.52:0/3212303569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8104db0 0x7fd0e81071a0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc009900 tx=0x7fd0dc02a410 comp rx=0 tx=0).stop 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 -- 172.21.15.52:0/3212303569 shutdown_connections 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 --2- 172.21.15.52:0/3212303569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e81076e0 0x7fd0e8109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 --2- 172.21.15.52:0/3212303569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e8104db0 0x7fd0e81071a0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 --2- 172.21.15.52:0/3212303569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 0x7fd0e8104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 -- 172.21.15.52:0/3212303569 >> 172.21.15.52:0/3212303569 conn(0x7fd0e80fbe70 msgr2=0x7fd0e80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 -- 172.21.15.52:0/3212303569 shutdown_connections 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 -- 172.21.15.52:0/3212303569 wait complete. 2024-06-02T23:22:57.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.641+0000 7fd0eda95640 1 Processor -- start 2024-06-02T23:22:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:57 smithi174 ceph-mon[35994]: pgmap v763: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 413 KiB/s wr, 53 op/s 2024-06-02T23:22:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 -- start start 2024-06-02T23:22:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 0x7fd0e8197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104db0 0x7fd0e81980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81076e0 0x7fd0e819f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810d530 con 0x7fd0e81076e0 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810d3b0 con 0x7fd0e8102480 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0eda95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e810d6b0 con 0x7fd0e8104db0 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81076e0 0x7fd0e819f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 0x7fd0e8197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81076e0 0x7fd0e819f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40542/0 (socket says 172.21.15.52:40542) 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e7fff640 1 -- 172.21.15.52:0/424466413 learned_addr learned my addr 172.21.15.52:0/424466413 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104db0 0x7fd0e81980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e77fe640 1 -- 172.21.15.52:0/424466413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104db0 msgr2=0x7fd0e81980f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104db0 0x7fd0e81980f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e77fe640 1 -- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81076e0 msgr2=0x7fd0e819f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81076e0 0x7fd0e819f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.642+0000 7fd0e77fe640 1 -- 172.21.15.52:0/424466413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0e819f7d0 con 0x7fd0e8102480 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.643+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 0x7fd0e8197bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800e9f0 tx=0x7fd0d800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.643+0000 7fd0e4ff9640 1 -- 172.21.15.52:0/424466413 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800ccc0 con 0x7fd0e8102480 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.643+0000 7fd0e4ff9640 1 -- 172.21.15.52:0/424466413 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0d80108e0 con 0x7fd0e8102480 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.643+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0e819fac0 con 0x7fd0e8102480 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.643+0000 7fd0e4ff9640 1 -- 172.21.15.52:0/424466413 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8018880 con 0x7fd0e8102480 2024-06-02T23:22:57.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.643+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0e819ffe0 con 0x7fd0e8102480 2024-06-02T23:22:57.646 INFO:tasks.workunit.client.0.smithi052.stdout: 1 217952 1.89 MB/sec execute 486 sec latency 13.388 ms 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0e4ff9640 1 -- 172.21.15.52:0/424466413 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd0d8010a90 con 0x7fd0e8102480 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 Processor -- start 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 -- start start 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814839430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x56281483e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56281483eb70 0x562814841000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56281479c4f0 con 0x562814839010 2024-06-02T23:22:57.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56281473a870 con 0x56281483eb70 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0f079b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628147984b0 con 0x562814839a00 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814839430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814839430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40552/0 (socket says 172.21.15.52:40552) 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.646+0000 7fd0e7fff640 1 -- 172.21.15.52:0/422554924 learned_addr learned my addr 172.21.15.52:0/422554924 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56281483eb70 0x562814841000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x56281483e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e7fff640 1 -- 172.21.15.52:0/422554924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 msgr2=0x56281483e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x56281483e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e7fff640 1 -- 172.21.15.52:0/422554924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56281483eb70 msgr2=0x562814841000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56281483eb70 0x562814841000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e7fff640 1 -- 172.21.15.52:0/422554924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628147569b0 con 0x562814839010 2024-06-02T23:22:57.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e4ff9640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c30 0x7fd0d00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x56281483e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814839430 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0011460 tx=0x7fd0e0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd604763a270185 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e4ff9640 1 -- 172.21.15.52:0/424466413 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd0d80cf360 con 0x7fd0e8102480 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c30 0x7fd0d00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0cdffb640 1 -- 172.21.15.52:0/422554924 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0012770 con 0x562814839010 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0cdffb640 1 -- 172.21.15.52:0/422554924 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e0012dc0 con 0x562814839010 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.647+0000 7fd0cdffb640 1 -- 172.21.15.52:0/422554924 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e00193a0 con 0x562814839010 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 -- 172.21.15.52:0/422554924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 msgr2=0x562814839430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0e77fe640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c30 0x7fd0d00a50f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80cd200 tx=0x7fd0d8009700 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814839430 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0011460 tx=0x7fd0e0011920 comp rx=0 tx=0).stop 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 -- 172.21.15.52:0/422554924 shutdown_connections 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x56281483e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56281483eb70 0x562814841000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 --2- 172.21.15.52:0/422554924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814839430 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 -- 172.21.15.52:0/422554924 >> 172.21.15.52:0/422554924 conn(0x562814834680 msgr2=0x562814834a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 -- 172.21.15.52:0/422554924 shutdown_connections 2024-06-02T23:22:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 -- 172.21.15.52:0/422554924 wait complete. 2024-06-02T23:22:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.648+0000 7fd0f079b740 1 -- start start 2024-06-02T23:22:57.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 Processor -- start 2024-06-02T23:22:57.652 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:57 smithi052 ceph-mon[32340]: pgmap v763: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 413 KiB/s wr, 53 op/s 2024-06-02T23:22:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814843250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814843790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x56281484a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562814841b20 con 0x56281483eb70 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562814725ae0 con 0x562814839010 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562814841ca0 con 0x562814839a00 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0f079b740 10 client.0 _ll_register_callbacks cb 0x56281475e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x56281484a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x56281484a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40568/0 (socket says 172.21.15.52:40568) 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0e7fff640 1 -- 172.21.15.52:0/4258260656 learned_addr learned my addr 172.21.15.52:0/4258260656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814843250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.649+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814843790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0e7fff640 1 -- 172.21.15.52:0/4258260656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 msgr2=0x562814843790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814843790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0e7fff640 1 -- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 msgr2=0x562814843250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814843250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0e7fff640 1 -- 172.21.15.52:0/4258260656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628144b0210 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x56281484a7d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd0e001c040 tx=0x7fd0e0014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28eef9131e8965c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/4258260656 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e002e060 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/4258260656 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd0e0004c40 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e0012dc0 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/4258260656 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0e0004e40 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0f079b740 10 client.18588 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56281457c620 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/4258260656 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e0012dc0 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.650+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/4258260656 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd0e0026040 con 0x56281483eb70 2024-06-02T23:22:57.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0f079b740 10 client.18588 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0f079b740 10 client.18588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0f079b740 20 client.18588 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0f079b740 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56281484edb0 0x5628148511a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0f079b740 4 client.18588 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628147aff10 con 0x56281484edb0 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56281484edb0 0x5628148511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0c6ffd640 10 client.18588 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.651+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56281484edb0 0x5628148511a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1c9c8f04632f4b server_cookie=5f4dbb7f536bd9b6 in_seq=0 out_seq=0 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.652+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/4258260656 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5628147aff10 con 0x56281484edb0 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.652+0000 7fd0c6ffd640 10 client.18588 handle_command_reply: tid=0 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.652+0000 7fd0f079b740 1 client.18588 shutdown 2024-06-02T23:22:57.658 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.652+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 msgr2=0x56281484a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.652+0000 7fd0f079b740 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x56281484a7d0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fd0e001c040 tx=0x7fd0e0014040 comp rx=0 tx=0).stop 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 shutdown_connections 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56281484edb0 0x5628148511a0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814843790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x56281484a7d0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0c6ffd640 0 client.18588 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 --2- 172.21.15.52:0/4258260656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814843250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 >> 172.21.15.52:0/4258260656 conn(0x562814834680 msgr2=0x5628148337e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 shutdown_connections 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 1 -- 172.21.15.52:0/4258260656 wait complete. 2024-06-02T23:22:57.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.653+0000 7fd0f079b740 20 client.18588 trim_cache size 0 max 16384 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.669+0000 7fd0f079b740 1 Processor -- start 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.669+0000 7fd0f079b740 1 -- start start 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814837360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839a00 0x56281484a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56281483eb70 0x5628148378a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0f079b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e81a8d20 con 0x562814839010 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0f079b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e81aa860 con 0x562814839a00 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0f079b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e81a93f0 con 0x56281483eb70 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814837360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814837360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40584/0 (socket says 172.21.15.52:40584) 2024-06-02T23:22:57.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 -- 172.21.15.52:0/1932841979 learned_addr learned my addr 172.21.15.52:0/1932841979 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839a00 0x56281484a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 -- 172.21.15.52:0/1932841979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56281483eb70 msgr2=0x5628148378a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56281483eb70 0x5628148378a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 -- 172.21.15.52:0/1932841979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839a00 msgr2=0x56281484a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839a00 0x56281484a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 -- 172.21.15.52:0/1932841979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628148344b0 con 0x562814839010 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814837360 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009b40 tx=0x7fd0e0009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3c1740b5481a13 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.670+0000 7fd0cd7fa640 1 -- 172.21.15.52:0/1932841979 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0039070 con 0x562814839010 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0cd7fa640 1 -- 172.21.15.52:0/1932841979 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e0005df0 con 0x562814839010 2024-06-02T23:22:57.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0cd7fa640 1 -- 172.21.15.52:0/1932841979 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e002e040 con 0x562814839010 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 -- 172.21.15.52:0/1932841979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 msgr2=0x562814837360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814837360 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0009b40 tx=0x7fd0e0009c30 comp rx=0 tx=0).stop 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 -- 172.21.15.52:0/1932841979 shutdown_connections 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56281483eb70 0x5628148378a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839a00 0x56281484a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 --2- 172.21.15.52:0/1932841979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562814839010 0x562814837360 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 -- 172.21.15.52:0/1932841979 >> 172.21.15.52:0/1932841979 conn(0x562814834680 msgr2=0x56281483ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 -- 172.21.15.52:0/1932841979 shutdown_connections 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.671+0000 7fd0f079b740 1 -- 172.21.15.52:0/1932841979 wait complete. 2024-06-02T23:22:57.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 -- start start 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 Processor -- start 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814847850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814847d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x562814848d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e81aa190 con 0x56281483eb70 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e81a9ac0 con 0x562814839010 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0e81a6d00 con 0x562814839a00 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0f079b740 10 client.0 _ll_register_callbacks cb 0x7fd0dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814847850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x562814848d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0e7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x562814848d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40600/0 (socket says 172.21.15.52:40600) 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0e7fff640 1 -- 172.21.15.52:0/3093590903 learned_addr learned my addr 172.21.15.52:0/3093590903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:22:57.677 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.672+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814847d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e7fff640 1 -- 172.21.15.52:0/3093590903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 msgr2=0x562814847d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814847d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e7fff640 1 -- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 msgr2=0x562814847850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814847850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e7fff640 1 -- 172.21.15.52:0/3093590903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562814849340 con 0x56281483eb70 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814847d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0e7fff640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x562814848d50 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0000f80 tx=0x7fd0e0011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cbed3be75a086b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3093590903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e0039070 con 0x56281483eb70 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3093590903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd0e0004a00 con 0x56281483eb70 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0e0005df0 con 0x56281483eb70 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3093590903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fd0e0029650 con 0x56281483eb70 2024-06-02T23:22:57.678 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0f079b740 10 client.18594 fetch_fsmap learned FSMap version 32 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56281484a650 con 0x56281483eb70 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.673+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3093590903 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd0e0005df0 con 0x56281483eb70 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3093590903 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fd0e000f040 con 0x56281483eb70 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0f079b740 10 client.18594 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0f079b740 10 client.18594 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0f079b740 20 client.18594 populate_metadata read hostname 'smithi052' 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0f079b740 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628148b9b50 0x5628148bbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0f079b740 4 client.18594 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5628147a9c20 con 0x5628148b9b50 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628148b9b50 0x5628148bbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0c6ffd640 10 client.18594 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:57.679 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.674+0000 7fd0e6ffd640 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628148b9b50 0x5628148bbf40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b832ff0d574ef02 server_cookie=b3129a3e48515671 in_seq=0 out_seq=0 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.693+0000 7fd0c6ffd640 1 -- 172.21.15.52:0/3093590903 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5628147a9c20 con 0x5628148b9b50 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.693+0000 7fd0c6ffd640 10 client.18594 handle_command_reply: tid=0 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.693+0000 7fd0f079b740 1 client.18594 shutdown 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 msgr2=0x562814848d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x562814848d50 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fd0e0000f80 tx=0x7fd0e0011920 comp rx=0 tx=0).stop 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 shutdown_connections 2024-06-02T23:22:57.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5628148b9b50 0x5628148bbf40 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562814839a00 0x562814847d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56281483eb70 0x562814848d50 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 --2- 172.21.15.52:0/3093590903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562814839010 0x562814847850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 >> 172.21.15.52:0/3093590903 conn(0x562814834680 msgr2=0x562814837360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0c6ffd640 0 client.18594 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 shutdown_connections 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 1 -- 172.21.15.52:0/3093590903 wait complete. 2024-06-02T23:22:57.696 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0f079b740 20 client.18594 trim_cache size 0 max 16384 2024-06-02T23:22:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:22:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:22:57.697 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c30 msgr2=0x7fd0d00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0eda95640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c30 0x7fd0d00a50f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd0d80cd200 tx=0x7fd0d8009700 comp rx=0 tx=0).stop 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 msgr2=0x7fd0e8197bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.694+0000 7fd0eda95640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 0x7fd0e8197bb0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800e9f0 tx=0x7fd0d800eeb0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 shutdown_connections 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd0d00a2c30 0x7fd0d00a50f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0e8104db0 0x7fd0e81980f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd0e81076e0 0x7fd0e819f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 --2- 172.21.15.52:0/424466413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0e8102480 0x7fd0e8197bb0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:22:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 >> 172.21.15.52:0/424466413 conn(0x7fd0e80fbe70 msgr2=0x7fd0e8102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:22:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 shutdown_connections 2024-06-02T23:22:57.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:22:57.695+0000 7fd0eda95640 1 -- 172.21.15.52:0/424466413 wait complete. 2024-06-02T23:22:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:57 smithi123 ceph-mon[35977]: pgmap v763: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 413 KiB/s wr, 53 op/s 2024-06-02T23:22:58.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 218601 1.88 MB/sec execute 487 sec latency 14.390 ms 2024-06-02T23:22:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:22:59 smithi174 ceph-mon[35994]: pgmap v764: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 525 KiB/s wr, 50 op/s 2024-06-02T23:22:59.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 219181 1.89 MB/sec execute 488 sec latency 15.102 ms 2024-06-02T23:22:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:22:59 smithi052 ceph-mon[32340]: pgmap v764: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 525 KiB/s wr, 50 op/s 2024-06-02T23:22:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:22:59 smithi123 ceph-mon[35977]: pgmap v764: 97 pgs: 97 active+clean; 92 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 525 KiB/s wr, 50 op/s 2024-06-02T23:23:00.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 219746 1.89 MB/sec execute 489 sec latency 15.763 ms 2024-06-02T23:23:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:01 smithi174 ceph-mon[35994]: pgmap v765: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 844 KiB/s wr, 103 op/s 2024-06-02T23:23:01.643 INFO:tasks.workunit.client.0.smithi052.stdout: 1 220223 1.90 MB/sec execute 490 sec latency 15.090 ms 2024-06-02T23:23:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:01 smithi052 ceph-mon[32340]: pgmap v765: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 844 KiB/s wr, 103 op/s 2024-06-02T23:23:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:01 smithi123 ceph-mon[35977]: pgmap v765: 97 pgs: 97 active+clean; 93 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 844 KiB/s wr, 103 op/s 2024-06-02T23:23:02.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 220503 1.89 MB/sec execute 491 sec latency 13.682 ms 2024-06-02T23:23:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:03 smithi174 ceph-mon[35994]: pgmap v766: 97 pgs: 97 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-02T23:23:03.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 220958 1.89 MB/sec execute 492 sec latency 15.561 ms 2024-06-02T23:23:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:03 smithi052 ceph-mon[32340]: pgmap v766: 97 pgs: 97 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-02T23:23:03.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:03 smithi123 ceph-mon[35977]: pgmap v766: 97 pgs: 97 active+clean; 94 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 1.1 MiB/s wr, 107 op/s 2024-06-02T23:23:04.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 221545 1.90 MB/sec execute 493 sec latency 11.473 ms 2024-06-02T23:23:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:05 smithi174 ceph-mon[35994]: pgmap v767: 97 pgs: 97 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:23:05.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 222172 1.89 MB/sec execute 494 sec latency 15.680 ms 2024-06-02T23:23:05.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:05 smithi052 ceph-mon[32340]: pgmap v767: 97 pgs: 97 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:23:05.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:05 smithi123 ceph-mon[35977]: pgmap v767: 97 pgs: 97 active+clean; 95 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 230 KiB/s rd, 1.2 MiB/s wr, 139 op/s 2024-06-02T23:23:06.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 222745 1.90 MB/sec execute 495 sec latency 14.833 ms 2024-06-02T23:23:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:07 smithi174 ceph-mon[35994]: pgmap v768: 97 pgs: 97 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:23:07.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 223314 1.90 MB/sec execute 496 sec latency 13.237 ms 2024-06-02T23:23:07.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:07 smithi052 ceph-mon[32340]: pgmap v768: 97 pgs: 97 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:23:07.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:07 smithi123 ceph-mon[35977]: pgmap v768: 97 pgs: 97 active+clean; 93 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 258 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:23:08.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 223806 1.91 MB/sec execute 497 sec latency 11.265 ms 2024-06-02T23:23:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:09 smithi174 ceph-mon[35994]: pgmap v769: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:23:09.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 224090 1.91 MB/sec execute 498 sec latency 14.872 ms 2024-06-02T23:23:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:09 smithi052 ceph-mon[32340]: pgmap v769: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:23:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:09 smithi123 ceph-mon[35977]: pgmap v769: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 1.2 MiB/s wr, 146 op/s 2024-06-02T23:23:10.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 224596 1.91 MB/sec execute 499 sec latency 14.605 ms 2024-06-02T23:23:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 -- 172.21.15.52:0/1105842492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88103c20 msgr2=0x7f3a88104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.797 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 --2- 172.21.15.52:0/1105842492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88103c20 0x7f3a88104040 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c009900 tx=0x7f3a7c02a410 comp rx=0 tx=0).stop 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 -- 172.21.15.52:0/1105842492 shutdown_connections 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 --2- 172.21.15.52:0/1105842492 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 0x7f3a8810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 --2- 172.21.15.52:0/1105842492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88104e10 0x7f3a88105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 --2- 172.21.15.52:0/1105842492 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88103c20 0x7f3a88104040 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 -- 172.21.15.52:0/1105842492 >> 172.21.15.52:0/1105842492 conn(0x7f3a880ff190 msgr2=0x7f3a881015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 -- 172.21.15.52:0/1105842492 shutdown_connections 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 -- 172.21.15.52:0/1105842492 wait complete. 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 Processor -- start 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.795+0000 7f3a8f142640 1 -- start start 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8f142640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88103c20 0x7f3a88199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8f142640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 0x7f3a8819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8f142640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 0x7f3a881a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8f142640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8810f750 con 0x7f3a88104e10 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8f142640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8810f5d0 con 0x7f3a88103c20 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8f142640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a8810f8d0 con 0x7f3a881057d0 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88103c20 0x7f3a88199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 0x7f3a8819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88103c20 0x7f3a88199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33728/0 (socket says 172.21.15.52:33728) 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8cede640 1 -- 172.21.15.52:0/3525629704 learned_addr learned my addr 172.21.15.52:0/3525629704 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 0x7f3a8819a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49328/0 (socket says 172.21.15.52:49328) 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 0x7f3a881a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 -- 172.21.15.52:0/3525629704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88103c20 msgr2=0x7f3a88199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88103c20 0x7f3a88199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 -- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 msgr2=0x7f3a8819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 0x7f3a8819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 -- 172.21.15.52:0/3525629704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a881a19e0 con 0x7f3a881057d0 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 0x7f3a8819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.796+0000 7f3a8d6df640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 0x7f3a881a1370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009fd0 tx=0x7f3a8000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.797+0000 7f3a85ffb640 1 -- 172.21.15.52:0/3525629704 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000ce20 con 0x7f3a881057d0 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.797+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a881a1cd0 con 0x7f3a881057d0 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.797+0000 7f3a85ffb640 1 -- 172.21.15.52:0/3525629704 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a80005020 con 0x7f3a881057d0 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.797+0000 7f3a85ffb640 1 -- 172.21.15.52:0/3525629704 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a80010690 con 0x7f3a881057d0 2024-06-02T23:23:10.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.797+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a881a2210 con 0x7f3a881057d0 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.799+0000 7f3a85ffb640 1 -- 172.21.15.52:0/3525629704 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a800051d0 con 0x7f3a881057d0 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.799+0000 7f3a91e52740 1 Processor -- start 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.799+0000 7f3a91e52740 1 -- start start 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 0x55cec5314430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314a00 0x55cec5319500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec531c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a91e52740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cec52774f0 con 0x55cec5314010 2024-06-02T23:23:10.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a91e52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cec5215870 con 0x55cec5319b70 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a91e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec52734b0 con 0x55cec5314a00 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 0x55cec5314430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 0x55cec5314430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49338/0 (socket says 172.21.15.52:49338) 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 -- 172.21.15.52:0/1419544291 learned_addr learned my addr 172.21.15.52:0/1419544291 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a8cede640 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314a00 0x55cec5319500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a85ffb640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a780a2cb0 0x7f3a780a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec531c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 -- 172.21.15.52:0/1419544291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314a00 msgr2=0x55cec5319500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314a00 0x55cec5319500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 -- 172.21.15.52:0/1419544291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 msgr2=0x55cec531c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec531c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 -- 172.21.15.52:0/1419544291 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec52319b0 con 0x55cec5314010 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a8d6df640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a780a2cb0 0x7f3a780a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a8cede640 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314a00 0x55cec5319500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a85ffb640 1 -- 172.21.15.52:0/3525629704 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3a800cf600 con 0x7f3a881057d0 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a87fff640 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 0x55cec5314430 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3a74013c30 tx=0x7f3a74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdc5fba7cb06a41 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.800+0000 7f3a72ffd640 1 -- 172.21.15.52:0/1419544291 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74013130 con 0x55cec5314010 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a72ffd640 1 -- 172.21.15.52:0/1419544291 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a7400fc50 con 0x55cec5314010 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a72ffd640 1 -- 172.21.15.52:0/1419544291 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a74005260 con 0x55cec5314010 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a8d6df640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a780a2cb0 0x7f3a780a5170 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c760 tx=0x7f3a80066b10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 -- 172.21.15.52:0/1419544291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 msgr2=0x55cec5314430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 0x55cec5314430 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f3a74013c30 tx=0x7f3a74009c00 comp rx=0 tx=0).stop 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 -- 172.21.15.52:0/1419544291 shutdown_connections 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314a00 0x55cec5319500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec531c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 --2- 172.21.15.52:0/1419544291 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314010 0x55cec5314430 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 -- 172.21.15.52:0/1419544291 >> 172.21.15.52:0/1419544291 conn(0x55cec530f680 msgr2=0x55cec530fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:10.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 -- 172.21.15.52:0/1419544291 shutdown_connections 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.801+0000 7f3a91e52740 1 -- 172.21.15.52:0/1419544291 wait complete. 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 -- start start 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 Processor -- start 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec531e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec531e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec53257d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cec531cac0 con 0x55cec5314a00 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cec5200ae0 con 0x55cec5319b70 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec531cc40 con 0x55cec5314010 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a91e52740 10 client.0 _ll_register_callbacks cb 0x55cec52399b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec531e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.802+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec531e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a87fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec531e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44854/0 (socket says 172.21.15.52:44854) 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec531e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49350/0 (socket says 172.21.15.52:49350) 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a87fff640 1 -- 172.21.15.52:0/3897944364 learned_addr learned my addr 172.21.15.52:0/3897944364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec53257d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 -- 172.21.15.52:0/3897944364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 msgr2=0x55cec531e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec531e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 -- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 msgr2=0x55cec53257d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec53257d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 -- 172.21.15.52:0/3897944364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec4f8b210 con 0x55cec5314a00 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec531e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a8cede640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec531e790 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5129ae5fca7d31f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a67fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a67fff640 1 -- 172.21.15.52:0/3897944364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c035070 con 0x55cec5314a00 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a67fff640 1 -- 172.21.15.52:0/3897944364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3a7c004040 con 0x55cec5314a00 2024-06-02T23:23:10.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.803+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a7400fc50 con 0x55cec5314a00 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a67fff640 1 -- 172.21.15.52:0/3897944364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c004240 con 0x55cec5314a00 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 10 client.18606 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cec5057620 con 0x55cec5314a00 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a67fff640 1 -- 172.21.15.52:0/3897944364 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a7400fc50 con 0x55cec5314a00 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a67fff640 1 -- 172.21.15.52:0/3897944364 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3a7c03b430 con 0x55cec5314a00 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 10 client.18606 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 10 client.18606 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 20 client.18606 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec532c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 4 client.18606 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec528af10 con 0x55cec532a150 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.804+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec532c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a67fff640 10 client.18606 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:10.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec532c540 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b080de76f9707e server_cookie=3fc30e7b3ecdf28 in_seq=0 out_seq=0 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a67fff640 1 -- 172.21.15.52:0/3897944364 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cec528af10 con 0x55cec532a150 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a67fff640 10 client.18606 handle_command_reply: tid=0 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 client.18606 shutdown 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 msgr2=0x55cec531e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec531e790 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c031d20 comp rx=0 tx=0).stop 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 shutdown_connections 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec532c540 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec53257d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec531e790 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a67fff640 0 client.18606 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3897944364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec531e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.805+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 >> 172.21.15.52:0/3897944364 conn(0x55cec530f680 msgr2=0x55cec530e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.806+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 shutdown_connections 2024-06-02T23:23:10.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.806+0000 7f3a91e52740 1 -- 172.21.15.52:0/3897944364 wait complete. 2024-06-02T23:23:10.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.806+0000 7f3a91e52740 20 client.18606 trim_cache size 0 max 16384 2024-06-02T23:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 Processor -- start 2024-06-02T23:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 -- start start 2024-06-02T23:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5315c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5325790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5316160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881aaef0 con 0x55cec5314a00 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881aca30 con 0x55cec5319b70 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a91e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881ab5c0 con 0x55cec5314010 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5325790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5325790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49366/0 (socket says 172.21.15.52:49366) 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a8cede640 1 -- 172.21.15.52:0/865664409 learned_addr learned my addr 172.21.15.52:0/865664409 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a87fff640 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5316160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.823+0000 7f3a87fff640 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5315c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a8cede640 1 -- 172.21.15.52:0/865664409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 msgr2=0x55cec5315c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a8cede640 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5315c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a8cede640 1 -- 172.21.15.52:0/865664409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 msgr2=0x55cec5316160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a8cede640 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5316160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a8cede640 1 -- 172.21.15.52:0/865664409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec530f4b0 con 0x55cec5314a00 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a87fff640 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5316160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a8cede640 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5325790 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c03ba60 tx=0x7f3a7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1dc52e1a83e7a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a727fc640 1 -- 172.21.15.52:0/865664409 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c054070 con 0x55cec5314a00 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a727fc640 1 -- 172.21.15.52:0/865664409 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a7c005850 con 0x55cec5314a00 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.824+0000 7f3a727fc640 1 -- 172.21.15.52:0/865664409 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c035070 con 0x55cec5314a00 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 -- 172.21.15.52:0/865664409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 msgr2=0x55cec5325790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5325790 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c03ba60 tx=0x7f3a7c005fd0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 -- 172.21.15.52:0/865664409 shutdown_connections 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5316160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5325790 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 --2- 172.21.15.52:0/865664409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5315c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 -- 172.21.15.52:0/865664409 >> 172.21.15.52:0/865664409 conn(0x55cec530f680 msgr2=0x55cec530fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:10.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 -- 172.21.15.52:0/865664409 shutdown_connections 2024-06-02T23:23:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 -- 172.21.15.52:0/865664409 wait complete. 2024-06-02T23:23:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 -- start start 2024-06-02T23:23:10.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.825+0000 7f3a91e52740 1 Processor -- start 2024-06-02T23:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5322750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5322c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5392df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881ac360 con 0x55cec5314a00 2024-06-02T23:23:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881abc90 con 0x55cec5319b70 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a881a8ed0 con 0x55cec5314010 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a91e52740 10 client.0 _ll_register_callbacks cb 0x7f3a800093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5322c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5322c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49380/0 (socket says 172.21.15.52:49380) 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 -- 172.21.15.52:0/3977888842 learned_addr learned my addr 172.21.15.52:0/3977888842 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5392df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5322750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 -- 172.21.15.52:0/3977888842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 msgr2=0x55cec5322750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5322750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 -- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 msgr2=0x55cec5392df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5392df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 -- 172.21.15.52:0/3977888842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec53933e0 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a8cede640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5322c90 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e7fb866cc6467d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a67fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.826+0000 7f3a67fff640 1 -- 172.21.15.52:0/3977888842 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c035030 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a67fff640 1 -- 172.21.15.52:0/3977888842 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3a7c005250 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a7c005850 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a67fff640 1 -- 172.21.15.52:0/3977888842 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a7c068050 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 10 client.18612 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cec5394690 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a67fff640 1 -- 172.21.15.52:0/3977888842 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a7c005850 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a67fff640 1 -- 172.21.15.52:0/3977888842 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3a7c0644f0 con 0x55cec5314a00 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 10 client.18612 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 10 client.18612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 20 client.18612 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec5396fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 4 client.18612 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:23:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cec5284c20 con 0x55cec532a150 2024-06-02T23:23:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.827+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec5396fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.828+0000 7f3a67fff640 10 client.18612 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.828+0000 7f3a87fff640 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec5396fe0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c231ce299ee82c2e server_cookie=a7c3145151cd95a in_seq=0 out_seq=0 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a67fff640 1 -- 172.21.15.52:0/3977888842 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cec5284c20 con 0x55cec532a150 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a67fff640 10 client.18612 handle_command_reply: tid=0 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 client.18612 shutdown 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 msgr2=0x55cec5322c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5322c90 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c000c00 tx=0x7f3a7c02a410 comp rx=0 tx=0).stop 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 shutdown_connections 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec532a150 0x55cec5396fe0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec5319b70 0x55cec5392df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec5314a00 0x55cec5322c90 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a67fff640 0 client.18612 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 --2- 172.21.15.52:0/3977888842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec5314010 0x55cec5322750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.847+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 >> 172.21.15.52:0/3977888842 conn(0x55cec530f680 msgr2=0x55cec531e180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 shutdown_connections 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a91e52740 1 -- 172.21.15.52:0/3977888842 wait complete. 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a91e52740 20 client.18612 trim_cache size 0 max 16384 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:23:10.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:23:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a780a2cb0 msgr2=0x7f3a780a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a780a2cb0 0x7f3a780a5170 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000c760 tx=0x7f3a80066b10 comp rx=0 tx=0).stop 2024-06-02T23:23:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 msgr2=0x7f3a881a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 0x7f3a881a1370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3a80009fd0 tx=0x7f3a8000ef10 comp rx=0 tx=0).stop 2024-06-02T23:23:10.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 shutdown_connections 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a780a2cb0 0x7f3a780a5170 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a881057d0 0x7f3a881a1370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a88104e10 0x7f3a8819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 --2- 172.21.15.52:0/3525629704 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a88103c20 0x7f3a88199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.848+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 >> 172.21.15.52:0/3525629704 conn(0x7f3a880ff190 msgr2=0x7f3a88108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.849+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 shutdown_connections 2024-06-02T23:23:10.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:10.849+0000 7f3a8f142640 1 -- 172.21.15.52:0/3525629704 wait complete. 2024-06-02T23:23:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:11 smithi174 ceph-mon[35994]: pgmap v770: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-02T23:23:11.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 225160 1.91 MB/sec execute 500 sec latency 21.020 ms 2024-06-02T23:23:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:11 smithi052 ceph-mon[32340]: pgmap v770: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-02T23:23:11.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:11 smithi123 ceph-mon[35977]: pgmap v770: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 320 KiB/s rd, 1.5 MiB/s wr, 191 op/s 2024-06-02T23:23:12.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 225842 1.91 MB/sec execute 501 sec latency 15.735 ms 2024-06-02T23:23:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:13 smithi174 ceph-mon[35994]: pgmap v771: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-06-02T23:23:13.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 226428 1.92 MB/sec execute 502 sec latency 14.644 ms 2024-06-02T23:23:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:13 smithi052 ceph-mon[32340]: pgmap v771: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-06-02T23:23:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:13 smithi123 ceph-mon[35977]: pgmap v771: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 217 KiB/s rd, 1.4 MiB/s wr, 142 op/s 2024-06-02T23:23:14.290 INFO:tasks.quiescer.fs.[cephfs]:Total duration (41.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:23:14.290 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.75 2024-06-02T23:23:14.290 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:23:14.564 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:14.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 226888 1.92 MB/sec execute 503 sec latency 12.414 ms 2024-06-02T23:23:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:15 smithi174 ceph-mon[35994]: pgmap v772: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-02T23:23:15.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 227360 1.92 MB/sec execute 504 sec latency 15.481 ms 2024-06-02T23:23:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:15 smithi052 ceph-mon[32340]: pgmap v772: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-02T23:23:15.687 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:23:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:15 smithi123 ceph-mon[35977]: pgmap v772: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.2 MiB/s wr, 160 op/s 2024-06-02T23:23:15.975 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:16.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 227626 1.92 MB/sec execute 505 sec latency 13.759 ms 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 -- 172.21.15.52:0/1041735679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 msgr2=0x7f40b410f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 --2- 172.21.15.52:0/1041735679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b410f310 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f40ac00b0a0 tx=0x7f40ac00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 -- 172.21.15.52:0/1041735679 shutdown_connections 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 --2- 172.21.15.52:0/1041735679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b410f310 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 --2- 172.21.15.52:0/1041735679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b41084d0 0x7f40b41088b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 --2- 172.21.15.52:0/1041735679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b406aa00 0x7f40b406ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 -- 172.21.15.52:0/1041735679 >> 172.21.15.52:0/1041735679 conn(0x7f40b4078330 msgr2=0x7f40b4078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:16.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 -- 172.21.15.52:0/1041735679 shutdown_connections 2024-06-02T23:23:16.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.655+0000 7f40bb639700 1 -- 172.21.15.52:0/1041735679 wait complete. 2024-06-02T23:23:16.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 Processor -- start 2024-06-02T23:23:16.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 -- start start 2024-06-02T23:23:16.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b406aa00 0x7f40b419e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:16.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:16.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b41084d0 0x7f40b4198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:16.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40b8bd4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:16.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40b41125d0 con 0x7f40b406b3a0 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112450 con 0x7f40b41084d0 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40bb639700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40b4112750 con 0x7f40b406aa00 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40b8bd4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b419ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34460/0 (socket says 172.21.15.52:34460) 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40b8bd4700 1 -- 172.21.15.52:0/83272709 learned_addr learned my addr 172.21.15.52:0/83272709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40b9bd6700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b41084d0 0x7f40b4198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.656+0000 7f40b93d5700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b406aa00 0x7f40b419e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40b8bd4700 1 -- 172.21.15.52:0/83272709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b406aa00 msgr2=0x7f40b419e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40b8bd4700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b406aa00 0x7f40b419e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40b8bd4700 1 -- 172.21.15.52:0/83272709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b41084d0 msgr2=0x7f40b4198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40b8bd4700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b41084d0 0x7f40b4198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40b8bd4700 1 -- 172.21.15.52:0/83272709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b41990d0 con 0x7f40b406b3a0 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40b8bd4700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b419ecf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f40a800d950 tx=0x7f40a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:16.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8016070 con 0x7f40b406b3a0 2024-06-02T23:23:16.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40a800fd40 con 0x7f40b406b3a0 2024-06-02T23:23:16.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b41993c0 con 0x7f40b406b3a0 2024-06-02T23:23:16.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.657+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a800e3b0 con 0x7f40b406b3a0 2024-06-02T23:23:16.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.658+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b41a5680 con 0x7f40b406b3a0 2024-06-02T23:23:16.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.659+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b4103b80 con 0x7f40b406b3a0 2024-06-02T23:23:16.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.660+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40a8019070 con 0x7f40b406b3a0 2024-06-02T23:23:16.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.661+0000 7f40a67fc700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40a00a2ca0 0x7f40a00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:16.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.661+0000 7f40b93d5700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40a00a2ca0 0x7f40a00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:16.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.661+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40a80cee00 con 0x7f40b406b3a0 2024-06-02T23:23:16.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.661+0000 7f40b93d5700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40a00a2ca0 0x7f40a00a5160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f40b4199ab0 tx=0x7f40b0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:16.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.666+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f40a809bd90 con 0x7f40b406b3a0 2024-06-02T23:23:16.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.972+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40b402dca0 con 0x7f40b406b3a0 2024-06-02T23:23:16.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.973+0000 7f40a67fc700 1 -- 172.21.15.52:0/83272709 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f40a809b690 con 0x7f40b406b3a0 2024-06-02T23:23:16.973 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:23:16.973 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:23:16.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40a00a2ca0 msgr2=0x7f40a00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40a00a2ca0 0x7f40a00a5160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f40b4199ab0 tx=0x7f40b0009260 comp rx=0 tx=0).stop 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 msgr2=0x7f40b419ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b419ecf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f40a800d950 tx=0x7f40a800dc60 comp rx=0 tx=0).stop 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 shutdown_connections 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40a00a2ca0 0x7f40a00a5160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40b41084d0 0x7f40b4198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40b406aa00 0x7f40b419e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 --2- 172.21.15.52:0/83272709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40b406b3a0 0x7f40b419ecf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 >> 172.21.15.52:0/83272709 conn(0x7f40b4078330 msgr2=0x7f40b40ff4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 shutdown_connections 2024-06-02T23:23:16.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:16.975+0000 7f40bb639700 1 -- 172.21.15.52:0/83272709 wait complete. 2024-06-02T23:23:16.977 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:23:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:17 smithi174 ceph-mon[35994]: pgmap v773: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-02T23:23:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:17 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/83272709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:17.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 228026 1.92 MB/sec execute 506 sec latency 67.212 ms 2024-06-02T23:23:17.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:17 smithi052 ceph-mon[32340]: pgmap v773: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-02T23:23:17.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:17 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/83272709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:17 smithi123 ceph-mon[35977]: pgmap v773: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 1.4 MiB/s wr, 166 op/s 2024-06-02T23:23:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:17 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/83272709' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:18.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 228506 1.92 MB/sec execute 507 sec latency 67.591 ms 2024-06-02T23:23:19.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:19 smithi123 ceph-mon[35977]: pgmap v774: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-06-02T23:23:19.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:19 smithi052 ceph-mon[32340]: pgmap v774: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-06-02T23:23:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:19 smithi174 ceph-mon[35994]: pgmap v774: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 216 KiB/s rd, 1.2 MiB/s wr, 133 op/s 2024-06-02T23:23:19.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 228986 1.92 MB/sec execute 508 sec latency 67.018 ms 2024-06-02T23:23:20.343 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 140.75 --await-for 45.0' 2024-06-02T23:23:20.344 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 140.75 --await-for 45.0 2024-06-02T23:23:20.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 229530 1.92 MB/sec execute 509 sec latency 21.104 ms 2024-06-02T23:23:21.538 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:21 smithi052 ceph-mon[32340]: pgmap v775: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:23:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:21 smithi174 ceph-mon[35994]: pgmap v775: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:23:21.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 230252 1.92 MB/sec execute 510 sec latency 63.167 ms 2024-06-02T23:23:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:21 smithi123 ceph-mon[35977]: pgmap v775: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 322 KiB/s rd, 1.4 MiB/s wr, 181 op/s 2024-06-02T23:23:21.790 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 -- 172.21.15.52:0/3052004256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804108620 msgr2=0x7fc804108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 --2- 172.21.15.52:0/3052004256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804108620 0x7fc804108a00 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4009990 tx=0x7fc7f4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 -- 172.21.15.52:0/3052004256 shutdown_connections 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 --2- 172.21.15.52:0/3052004256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 0x7fc80410f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 --2- 172.21.15.52:0/3052004256 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804108620 0x7fc804108a00 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 --2- 172.21.15.52:0/3052004256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc804102510 0x7fc804102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 -- 172.21.15.52:0/3052004256 >> 172.21.15.52:0/3052004256 conn(0x7fc804079640 msgr2=0x7fc804079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 -- 172.21.15.52:0/3052004256 shutdown_connections 2024-06-02T23:23:21.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.802+0000 7fc8089fc700 1 -- 172.21.15.52:0/3052004256 wait complete. 2024-06-02T23:23:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.803+0000 7fc8089fc700 1 Processor -- start 2024-06-02T23:23:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.803+0000 7fc8089fc700 1 -- start start 2024-06-02T23:23:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc8089fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804102510 0x7fc80419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc8089fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 0x7fc80419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc8089fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc804108620 0x7fc80419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc8089fc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041126d0 con 0x7fc804102510 2024-06-02T23:23:21.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc8089fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc804112550 con 0x7fc804102ed0 2024-06-02T23:23:21.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc8089fc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc804112850 con 0x7fc804108620 2024-06-02T23:23:21.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc80259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804102510 0x7fc80419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc80259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804102510 0x7fc80419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34472/0 (socket says 172.21.15.52:34472) 2024-06-02T23:23:21.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc801d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 0x7fc80419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc80259c700 1 -- 172.21.15.52:0/2374703565 learned_addr learned my addr 172.21.15.52:0/2374703565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc804108620 0x7fc80419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc801d9b700 1 -- 172.21.15.52:0/2374703565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc804108620 msgr2=0x7fc80419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc804108620 0x7fc80419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc801d9b700 1 -- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804102510 msgr2=0x7fc80419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804102510 0x7fc80419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.804+0000 7fc801d9b700 1 -- 172.21.15.52:0/2374703565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80419fd40 con 0x7fc804102ed0 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.805+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 0x7fc80419b270 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800d950 tx=0x7fc7f800dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.805+0000 7fc7f37fe700 1 -- 172.21.15.52:0/2374703565 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8017070 con 0x7fc804102ed0 2024-06-02T23:23:21.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.805+0000 7fc7f37fe700 1 -- 172.21.15.52:0/2374703565 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f8009de0 con 0x7fc804102ed0 2024-06-02T23:23:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.805+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8041a0030 con 0x7fc804102ed0 2024-06-02T23:23:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.805+0000 7fc7f37fe700 1 -- 172.21.15.52:0/2374703565 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8010d90 con 0x7fc804102ed0 2024-06-02T23:23:21.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.805+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8041a78b0 con 0x7fc804102ed0 2024-06-02T23:23:21.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.808+0000 7fc7f37fe700 1 -- 172.21.15.52:0/2374703565 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc7f801a070 con 0x7fc804102ed0 2024-06-02T23:23:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.809+0000 7fc7f37fe700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc7ec0a2ca0 0x7fc7ec0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.809+0000 7fc7f37fe700 1 -- 172.21.15.52:0/2374703565 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc7f80cea10 con 0x7fc804102ed0 2024-06-02T23:23:21.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.809+0000 7fc80259c700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc7ec0a2ca0 0x7fc7ec0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc80259c700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc7ec0a2ca0 0x7fc7ec0a5160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4006010 tx=0x7fc7f400b260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:21.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 Processor -- start 2024-06-02T23:23:21.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 -- start start 2024-06-02T23:23:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d55d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d557260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d557bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55df4d449f60 con 0x55df4d5577a0 2024-06-02T23:23:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df4d4a8850 con 0x55df4d55d880 2024-06-02T23:23:21.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc81327fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df4d410460 con 0x55df4d55cf60 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.810+0000 7fc801d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d557260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc802d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d557bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d557260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38942/0 (socket says 172.21.15.52:38942) 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 -- 172.21.15.52:0/868166718 learned_addr learned my addr 172.21.15.52:0/868166718 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc80259c700 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d55d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 -- 172.21.15.52:0/868166718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 msgr2=0x55df4d55d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d55d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 -- 172.21.15.52:0/868166718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 msgr2=0x55df4d557bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d557bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 -- 172.21.15.52:0/868166718 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4d3f78d0 con 0x55df4d55d880 2024-06-02T23:23:21.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc801d9b700 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d557260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8010880 tx=0x7fc7f800df90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86d30abcf7822779 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc7f0ff9700 1 -- 172.21.15.52:0/868166718 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8021050 con 0x55df4d55d880 2024-06-02T23:23:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc7f0ff9700 1 -- 172.21.15.52:0/868166718 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f809b240 con 0x55df4d55d880 2024-06-02T23:23:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.811+0000 7fc7f0ff9700 1 -- 172.21.15.52:0/868166718 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80bb310 con 0x55df4d55d880 2024-06-02T23:23:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 -- 172.21.15.52:0/868166718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 msgr2=0x55df4d557260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d557260 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8010880 tx=0x7fc7f800df90 comp rx=0 tx=0).stop 2024-06-02T23:23:21.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 -- 172.21.15.52:0/868166718 shutdown_connections 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d557bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d55d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 --2- 172.21.15.52:0/868166718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d557260 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 -- 172.21.15.52:0/868166718 >> 172.21.15.52:0/868166718 conn(0x55df4d484080 msgr2=0x55df4d552220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 -- 172.21.15.52:0/868166718 shutdown_connections 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.812+0000 7fc81327fb80 1 -- 172.21.15.52:0/868166718 wait complete. 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 -- start start 2024-06-02T23:23:21.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 Processor -- start 2024-06-02T23:23:21.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5635d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d5608d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d560e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55df4d448fc0 con 0x55df4d5577a0 2024-06-02T23:23:21.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df4d4a4810 con 0x55df4d55d880 2024-06-02T23:23:21.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc81327fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df4d5641a0 con 0x55df4d55cf60 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.813+0000 7fc80259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5635d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc81327fb80 10 client.0 _ll_register_callbacks cb 0x55df4d43e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5635d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34492/0 (socket says 172.21.15.52:34492) 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 -- 172.21.15.52:0/2972455176 learned_addr learned my addr 172.21.15.52:0/2972455176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d560e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d5608d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 -- 172.21.15.52:0/2972455176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 msgr2=0x55df4d5608d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d5608d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 -- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 msgr2=0x55df4d560e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d560e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 -- 172.21.15.52:0/2972455176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4d46e750 con 0x55df4d5577a0 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d560e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:21.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d5608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc80259c700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5635d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bc30 tx=0x7fc7f4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b50b4bed203e88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc7e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.814+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2972455176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4003990 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2972455176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc7f4003b30 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f809b240 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2972455176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7f4031460 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 10 client.18630 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df4d577170 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2972455176 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f400f910 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2972455176 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7f4026860 con 0x55df4d5577a0 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 10 client.18630 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:21.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 20 client.18630 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d57bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 4 client.18630 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc7f809b240 con 0x55df4d579950 2024-06-02T23:23:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.815+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d57bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.816+0000 7fc7e9ffb700 10 client.18630 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:21.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.816+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d57bd90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5998b30489c06f4d server_cookie=24fb2693b5574a5e in_seq=0 out_seq=0 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2972455176 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fc7f809b240 con 0x55df4d579950 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc7e9ffb700 10 client.18630 handle_command_reply: tid=0 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 client.18630 shutdown 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 msgr2=0x55df4d5635d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5635d0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400bc30 tx=0x7fc7f4009310 comp rx=0 tx=0).stop 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 shutdown_connections 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d57bd90 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc7e9ffb700 0 client.18630 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:21.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d560e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5635d0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2972455176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d5608d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.817+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 >> 172.21.15.52:0/2972455176 conn(0x55df4d484080 msgr2=0x55df4d551700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:21.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.818+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 shutdown_connections 2024-06-02T23:23:21.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.818+0000 7fc81327fb80 1 -- 172.21.15.52:0/2972455176 wait complete. 2024-06-02T23:23:21.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.818+0000 7fc81327fb80 20 client.18630 trim_cache size 0 max 16384 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.842+0000 7fc81327fb80 1 Processor -- start 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.842+0000 7fc81327fb80 1 -- start start 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 0x55df4d557260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d563110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d55d880 0x55df4d551e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc81327fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041aa840 con 0x55df4d55d880 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc81327fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041ac380 con 0x55df4d5577a0 2024-06-02T23:23:21.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc81327fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041aaf10 con 0x55df4d55cf60 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 0x55df4d557260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 0x55df4d557260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38948/0 (socket says 172.21.15.52:38948) 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 -- 172.21.15.52:0/982505315 learned_addr learned my addr 172.21.15.52:0/982505315 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc802d9d700 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d55d880 0x55df4d551e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc801d9b700 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d563110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 -- 172.21.15.52:0/982505315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 msgr2=0x55df4d563110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d563110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 -- 172.21.15.52:0/982505315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d55d880 msgr2=0x55df4d551e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d55d880 0x55df4d551e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 -- 172.21.15.52:0/982505315 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df4d55ee80 con 0x55df4d5577a0 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc801d9b700 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d563110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.843+0000 7fc80259c700 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 0x55df4d557260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c010 tx=0x7fc7f400b310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb37990e11673546 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.844+0000 7fc7ebfff700 1 -- 172.21.15.52:0/982505315 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4041070 con 0x55df4d5577a0 2024-06-02T23:23:21.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.844+0000 7fc7ebfff700 1 -- 172.21.15.52:0/982505315 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7f40268e0 con 0x55df4d5577a0 2024-06-02T23:23:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.844+0000 7fc7ebfff700 1 -- 172.21.15.52:0/982505315 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f4024ce0 con 0x55df4d5577a0 2024-06-02T23:23:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.844+0000 7fc81327fb80 1 -- 172.21.15.52:0/982505315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 msgr2=0x55df4d557260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.844+0000 7fc81327fb80 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 0x55df4d557260 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc7f400c010 tx=0x7fc7f400b310 comp rx=0 tx=0).stop 2024-06-02T23:23:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- 172.21.15.52:0/982505315 shutdown_connections 2024-06-02T23:23:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d55d880 0x55df4d551e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d5577a0 0x55df4d557260 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 --2- 172.21.15.52:0/982505315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d563110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- 172.21.15.52:0/982505315 >> 172.21.15.52:0/982505315 conn(0x55df4d484080 msgr2=0x55df4d55ba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- 172.21.15.52:0/982505315 shutdown_connections 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- 172.21.15.52:0/982505315 wait complete. 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- start start 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 Processor -- start 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5819b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d57eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d57f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041abcb0 con 0x55df4d5577a0 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8041ab5e0 con 0x55df4d55d880 2024-06-02T23:23:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc80410d290 con 0x55df4d55cf60 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc80259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5819b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc81327fb80 10 client.0 _ll_register_callbacks cb 0x55df4d43e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc80259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5819b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:34504/0 (socket says 172.21.15.52:34504) 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc80259c700 1 -- 172.21.15.52:0/2740271663 learned_addr learned my addr 172.21.15.52:0/2740271663 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.845+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d57f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc801d9b700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d57eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc802d9d700 1 -- 172.21.15.52:0/2740271663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 msgr2=0x55df4d57eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d57eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc802d9d700 1 -- 172.21.15.52:0/2740271663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 msgr2=0x55df4d5819b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5819b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc802d9d700 1 -- 172.21.15.52:0/2740271663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df4d57f850 con 0x55df4d55d880 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc802d9d700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d57f2e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc018470 tx=0x7fc7fc0184a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14648d6c596fbd6f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc7e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2740271663 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc004db0 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2740271663 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc7fc005020 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7f40268e0 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.846+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2740271663 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc018e20 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 10 client.28085 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55df4d5e42f0 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2740271663 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc7fc0056d0 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2740271663 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc7fc01f070 con 0x55df4d55d880 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 10 client.28085 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 10 client.28085 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 20 client.28085 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d5e6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 4 client.28085 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:23:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fc7f40268e0 con 0x55df4d579950 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.847+0000 7fc80259c700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d5e6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc7e9ffb700 10 client.28085 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc80259c700 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d5e6ac0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13331a2633275654 server_cookie=36e867cfcc4e2dbf in_seq=0 out_seq=0 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc7e9ffb700 1 -- 172.21.15.52:0/2740271663 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7fc7f40268e0 con 0x55df4d579950 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc7e9ffb700 10 client.28085 handle_command_reply: tid=0 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 client.28085 shutdown 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 msgr2=0x55df4d57f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d57f2e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc018470 tx=0x7fc7fc0184a0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 shutdown_connections 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df4d579950 0x55df4d5e6ac0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df4d55d880 0x55df4d57f2e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df4d5577a0 0x55df4d5819b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 --2- 172.21.15.52:0/2740271663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df4d55cf60 0x55df4d57eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 >> 172.21.15.52:0/2740271663 conn(0x55df4d484080 msgr2=0x55df4d55c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc7e9ffb700 0 client.28085 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:21.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 shutdown_connections 2024-06-02T23:23:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 1 -- 172.21.15.52:0/2740271663 wait complete. 2024-06-02T23:23:21.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.848+0000 7fc81327fb80 20 client.28085 trim_cache size 0 max 16384 2024-06-02T23:23:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:23:21.854 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:23:21.854 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:23:21.854 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc7ec0a2ca0 msgr2=0x7fc7ec0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc7ec0a2ca0 0x7fc7ec0a5160 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fc7f4006010 tx=0x7fc7f400b260 comp rx=0 tx=0).stop 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 msgr2=0x7fc80419b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 0x7fc80419b270 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800d950 tx=0x7fc7f800dc60 comp rx=0 tx=0).stop 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 shutdown_connections 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc7ec0a2ca0 0x7fc7ec0a5160 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc804108620 0x7fc80419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc804102510 0x7fc80419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 --2- 172.21.15.52:0/2374703565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc804102ed0 0x7fc80419b270 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 >> 172.21.15.52:0/2374703565 conn(0x7fc804079640 msgr2=0x7fc80410d910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 shutdown_connections 2024-06-02T23:23:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:21.849+0000 7fc8089fc700 1 -- 172.21.15.52:0/2374703565 wait complete. 2024-06-02T23:23:22.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 230686 1.93 MB/sec execute 511 sec latency 16.449 ms 2024-06-02T23:23:23.230 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:23:23.231 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:23:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:23 smithi174 ceph-mon[35994]: pgmap v776: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:23:23.644 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231001 1.93 MB/sec execute 512 sec latency 12.482 ms 2024-06-02T23:23:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:23 smithi052 ceph-mon[32340]: pgmap v776: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:23:23.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:23 smithi123 ceph-mon[35977]: pgmap v776: 97 pgs: 97 active+clean; 97 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 1.2 MiB/s wr, 138 op/s 2024-06-02T23:23:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3201660060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b781076e0 msgr2=0x7f2b78109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3201660060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b781076e0 0x7f2b78109b30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f2b70009ca0 tx=0x7f2b7002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3201660060 shutdown_connections 2024-06-02T23:23:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3201660060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b781076e0 0x7f2b78109b30 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3201660060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b78104db0 0x7f2b781071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3201660060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b78102480 0x7f2b78104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3201660060 >> 172.21.15.52:0/3201660060 conn(0x7f2b780fbe70 msgr2=0x7f2b780fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3201660060 shutdown_connections 2024-06-02T23:23:23.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3201660060 wait complete. 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 Processor -- start 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.949+0000 7f2b7fa6d640 1 -- start start 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7fa6d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 0x7f2b78197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7fa6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b78104db0 0x7f2b781980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7fa6d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b781076e0 0x7f2b7819f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7fa6d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810d580 con 0x7f2b78102480 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7fa6d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810d400 con 0x7f2b78104db0 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7fa6d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7810d700 con 0x7f2b781076e0 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 0x7f2b78197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 0x7f2b78197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44544/0 (socket says 172.21.15.52:44544) 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 -- 172.21.15.52:0/3472040622 learned_addr learned my addr 172.21.15.52:0/3472040622 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:23.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b78104db0 0x7f2b781980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7e00a640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b781076e0 0x7f2b7819f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 -- 172.21.15.52:0/3472040622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b781076e0 msgr2=0x7f2b7819f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b781076e0 0x7f2b7819f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 -- 172.21.15.52:0/3472040622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b78104db0 msgr2=0x7f2b781980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b78104db0 0x7f2b781980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 -- 172.21.15.52:0/3472040622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7819f780 con 0x7f2b78102480 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b7d809640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 0x7f2b78197b60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d950 tx=0x7f2b7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.950+0000 7f2b6effd640 1 -- 172.21.15.52:0/3472040622 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74017070 con 0x7f2b78102480 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.951+0000 7f2b6effd640 1 -- 172.21.15.52:0/3472040622 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b74009c30 con 0x7f2b78102480 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.951+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b7819fa70 con 0x7f2b78102480 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.951+0000 7f2b6effd640 1 -- 172.21.15.52:0/3472040622 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7400f620 con 0x7f2b78102480 2024-06-02T23:23:23.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.951+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b781a0010 con 0x7f2b78102480 2024-06-02T23:23:23.954 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 Processor -- start 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b6effd640 1 -- 172.21.15.52:0/3472040622 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b740109a0 con 0x7f2b78102480 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 -- start start 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbf2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf2a00 0x561a3fbf7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 0x561a3fbfa000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561a3fb554f0 con 0x561a3fbf2a00 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561a3faf3870 con 0x561a3fbf7b70 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.954+0000 7f2b82777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a3fb514b0 con 0x561a3fbf2010 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 0x561a3fbfa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf2a00 0x561a3fbf7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 0x561a3fbfa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55248/0 (socket says 172.21.15.52:55248) 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf2a00 0x561a3fbf7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44548/0 (socket says 172.21.15.52:44548) 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 -- 172.21.15.52:0/3524385404 learned_addr learned my addr 172.21.15.52:0/3524385404 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7e00a640 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbf2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 -- 172.21.15.52:0/3524385404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 msgr2=0x561a3fbf2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbf2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 -- 172.21.15.52:0/3524385404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf2a00 msgr2=0x561a3fbf7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf2a00 0x561a3fbf7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 -- 172.21.15.52:0/3524385404 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3fb0f9b0 con 0x561a3fbf7b70 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b6effd640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b640a2cb0 0x7f2b640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 0x561a3fbfa000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009c60 tx=0x7f2b68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7fb3d6f7f133ec3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b7e00a640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b640a2cb0 0x7f2b640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b6effd640 1 -- 172.21.15.52:0/3472040622 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b740ceb30 con 0x7f2b78102480 2024-06-02T23:23:23.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b63fff640 1 -- 172.21.15.52:0/3524385404 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004250 con 0x561a3fbf7b70 2024-06-02T23:23:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b63fff640 1 -- 172.21.15.52:0/3524385404 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b68009e10 con 0x561a3fbf7b70 2024-06-02T23:23:23.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.955+0000 7f2b63fff640 1 -- 172.21.15.52:0/3524385404 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68004cf0 con 0x561a3fbf7b70 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 -- 172.21.15.52:0/3524385404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 msgr2=0x561a3fbfa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b7e00a640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b640a2cb0 0x7f2b640a5170 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002f3a0 tx=0x7f2b70004a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 0x561a3fbfa000 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2b68009c60 tx=0x7f2b68011810 comp rx=0 tx=0).stop 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 -- 172.21.15.52:0/3524385404 shutdown_connections 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf7b70 0x561a3fbfa000 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf2a00 0x561a3fbf7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 --2- 172.21.15.52:0/3524385404 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbf2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 -- 172.21.15.52:0/3524385404 >> 172.21.15.52:0/3524385404 conn(0x561a3fbed680 msgr2=0x561a3fbeda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 -- 172.21.15.52:0/3524385404 shutdown_connections 2024-06-02T23:23:23.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.956+0000 7f2b82777740 1 -- 172.21.15.52:0/3524385404 wait complete. 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 -- start start 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 Processor -- start 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbfc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2a00 0x561a3fbfc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561a3fbfab20 con 0x561a3fbf7b70 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561a3fadeae0 con 0x561a3fbf2a00 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561a3fbfaca0 con 0x561a3fbf2010 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b82777740 10 client.0 _ll_register_callbacks cb 0x561a3fb179b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b7e00a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b7d008640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2a00 0x561a3fbfc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b7e00a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44558/0 (socket says 172.21.15.52:44558) 2024-06-02T23:23:23.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b7e00a640 1 -- 172.21.15.52:0/1420836357 learned_addr learned my addr 172.21.15.52:0/1420836357 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.957+0000 7f2b7d809640 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbfc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b7d008640 1 -- 172.21.15.52:0/1420836357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 msgr2=0x561a3fbfc250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b7d008640 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbfc250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b7d008640 1 -- 172.21.15.52:0/1420836357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 msgr2=0x561a3fc037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b7d008640 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b7d008640 1 -- 172.21.15.52:0/1420836357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3f869210 con 0x561a3fbf2a00 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b7d008640 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2a00 0x561a3fbfc790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b68012fd0 tx=0x7f2b68003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cca2be5f6e794880 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b617fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b617fa640 1 -- 172.21.15.52:0/1420836357 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b68029070 con 0x561a3fbf2a00 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b617fa640 1 -- 172.21.15.52:0/1420836357 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b68023430 con 0x561a3fbf2a00 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b68009e10 con 0x561a3fbf2a00 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b617fa640 1 -- 172.21.15.52:0/1420836357 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b68035050 con 0x561a3fbf2a00 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b82777740 10 client.28097 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:23.964 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a3f935620 con 0x561a3fbf2a00 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.958+0000 7f2b617fa640 1 -- 172.21.15.52:0/1420836357 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b68009e10 con 0x561a3fbf2a00 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b617fa640 1 -- 172.21.15.52:0/1420836357 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b68023cd0 con 0x561a3fbf2a00 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b82777740 10 client.28097 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b82777740 10 client.28097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b82777740 20 client.28097 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b82777740 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc07ea0 0x561a3fc0a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b82777740 4 client.28097 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a3fb68f10 con 0x561a3fc07ea0 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b7d809640 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc07ea0 0x561a3fc0a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b7d809640 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc07ea0 0x561a3fc0a290 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee40c408031e3aa server_cookie=47ce9f8b7d61196 in_seq=0 out_seq=0 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.959+0000 7f2b617fa640 10 client.28097 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b617fa640 1 -- 172.21.15.52:0/1420836357 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561a3fb68f10 con 0x561a3fc07ea0 2024-06-02T23:23:23.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b617fa640 10 client.28097 handle_command_reply: tid=0 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 client.28097 shutdown 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2a00 msgr2=0x561a3fbfc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2a00 0x561a3fbfc790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b68012fd0 tx=0x7f2b68003820 comp rx=0 tx=0).stop 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 shutdown_connections 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc07ea0 0x561a3fc0a290 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2a00 0x561a3fbfc790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 --2- 172.21.15.52:0/1420836357 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2010 0x561a3fbfc250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b617fa640 0 client.28097 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 >> 172.21.15.52:0/1420836357 conn(0x561a3fbed680 msgr2=0x561a3fbec7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 shutdown_connections 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 1 -- 172.21.15.52:0/1420836357 wait complete. 2024-06-02T23:23:23.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.960+0000 7f2b82777740 20 client.28097 trim_cache size 0 max 16384 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.978+0000 7f2b82777740 1 Processor -- start 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 -- start start 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fbf0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc03790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fbf0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a8cc0 con 0x561a3fbf7b70 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781aa800 con 0x561a3fbf2010 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b82777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a9390 con 0x561a3fbf2a00 2024-06-02T23:23:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7e00a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fbf0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fbf0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7e00a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fbf0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44562/0 (socket says 172.21.15.52:44562) 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7e00a640 1 -- 172.21.15.52:0/2737868361 learned_addr learned my addr 172.21.15.52:0/2737868361 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d008640 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc03790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 -- 172.21.15.52:0/2737868361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 msgr2=0x561a3fc03790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc03790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 -- 172.21.15.52:0/2737868361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 msgr2=0x561a3fbf0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fbf0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 -- 172.21.15.52:0/2737868361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a3fbed4b0 con 0x561a3fbf2010 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7e00a640 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fbf0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.979+0000 7f2b7d809640 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fbf0560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b740cc6b0 tx=0x7f2b740bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=356f512968b66c9b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b637fe640 1 -- 172.21.15.52:0/2737868361 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74020040 con 0x561a3fbf2010 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b637fe640 1 -- 172.21.15.52:0/2737868361 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b74009de0 con 0x561a3fbf2010 2024-06-02T23:23:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b637fe640 1 -- 172.21.15.52:0/2737868361 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b740995a0 con 0x561a3fbf2010 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 -- 172.21.15.52:0/2737868361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 msgr2=0x561a3fbf0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fbf0560 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2b740cc6b0 tx=0x7f2b740bb040 comp rx=0 tx=0).stop 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 -- 172.21.15.52:0/2737868361 shutdown_connections 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc03790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fbf0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 --2- 172.21.15.52:0/2737868361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fbf0560 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.980+0000 7f2b82777740 1 -- 172.21.15.52:0/2737868361 >> 172.21.15.52:0/2737868361 conn(0x561a3fbed680 msgr2=0x561a3fbf3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 -- 172.21.15.52:0/2737868361 shutdown_connections 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 -- 172.21.15.52:0/2737868361 wait complete. 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 -- start start 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 Processor -- start 2024-06-02T23:23:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fc009d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc00f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc01f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781aa130 con 0x561a3fbf7b70 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a9a60 con 0x561a3fbf2010 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b781a6ca0 con 0x561a3fbf2a00 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b82777740 10 client.0 _ll_register_callbacks cb 0x7f2b740d24e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fc009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b7e00a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc01f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b7d809640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fc009d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55282/0 (socket says 172.21.15.52:55282) 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b7d809640 1 -- 172.21.15.52:0/3665567145 learned_addr learned my addr 172.21.15.52:0/3665567145 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.981+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc00f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b7d809640 1 -- 172.21.15.52:0/3665567145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 msgr2=0x561a3fc00f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b7d809640 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc00f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b7d809640 1 -- 172.21.15.52:0/3665567145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 msgr2=0x561a3fc01f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b7d809640 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc01f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b7d809640 1 -- 172.21.15.52:0/3665567145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a3fc02510 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b7d809640 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fc009d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b7409b170 tx=0x7f2b7400bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2c7d07d4d74ef4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b617fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b617fa640 1 -- 172.21.15.52:0/3665567145 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b74020030 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b617fa640 1 -- 172.21.15.52:0/3665567145 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b7409a8c0 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b74009de0 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b617fa640 1 -- 172.21.15.52:0/3665567145 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2b7409b470 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b82777740 10 client.28103 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561a3fc03820 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.982+0000 7f2b617fa640 1 -- 172.21.15.52:0/3665567145 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b74009de0 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b617fa640 1 -- 172.21.15.52:0/3665567145 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2b740971c0 con 0x561a3fbf2010 2024-06-02T23:23:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b82777740 10 client.28103 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b82777740 10 client.28103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b82777740 20 client.28103 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b82777740 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc72f30 0x561a3fc75350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b82777740 4 client.28103 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a3fb62c20 con 0x561a3fc72f30 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc72f30 0x561a3fc75350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b617fa640 10 client.28103 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:23.983+0000 7f2b7d008640 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc72f30 0x561a3fc75350 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f3e838b2591455 server_cookie=df78c02ae3410325 in_seq=0 out_seq=0 2024-06-02T23:23:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b617fa640 1 -- 172.21.15.52:0/3665567145 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561a3fb62c20 con 0x561a3fc72f30 2024-06-02T23:23:24.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b617fa640 10 client.28103 handle_command_reply: tid=0 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 client.28103 shutdown 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 msgr2=0x561a3fc009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fc009d0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2b7409b170 tx=0x7f2b7400bf60 comp rx=0 tx=0).stop 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 shutdown_connections 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561a3fc72f30 0x561a3fc75350 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561a3fbf2a00 0x561a3fc00f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561a3fbf7b70 0x561a3fc01f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b617fa640 0 client.28103 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 --2- 172.21.15.52:0/3665567145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561a3fbf2010 0x561a3fc009d0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 >> 172.21.15.52:0/3665567145 conn(0x561a3fbed680 msgr2=0x561a3fbf0560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 shutdown_connections 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 1 -- 172.21.15.52:0/3665567145 wait complete. 2024-06-02T23:23:24.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.005+0000 7f2b82777740 20 client.28103 trim_cache size 0 max 16384 2024-06-02T23:23:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:23:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:23:24.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b640a2cb0 msgr2=0x7f2b640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b640a2cb0 0x7f2b640a5170 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2b7002f3a0 tx=0x7f2b70004a60 comp rx=0 tx=0).stop 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 msgr2=0x7f2b78197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 0x7f2b78197b60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f2b7400d950 tx=0x7f2b7400de10 comp rx=0 tx=0).stop 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 shutdown_connections 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b640a2cb0 0x7f2b640a5170 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b781076e0 0x7f2b7819f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b78104db0 0x7f2b781980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 --2- 172.21.15.52:0/3472040622 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b78102480 0x7f2b78197b60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 >> 172.21.15.52:0/3472040622 conn(0x7f2b780fbe70 msgr2=0x7f2b78102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 shutdown_connections 2024-06-02T23:23:24.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:24.006+0000 7f2b7fa6d640 1 -- 172.21.15.52:0/3472040622 wait complete. 2024-06-02T23:23:24.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 -- 172.21.15.52:0/360382190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38101d90 msgr2=0x7fec3810e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/360382190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38101d90 0x7fec3810e280 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fec3000b0a0 tx=0x7fec3000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 -- 172.21.15.52:0/360382190 shutdown_connections 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/360382190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38101d90 0x7fec3810e280 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/360382190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec38107440 0x7fec38107820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/360382190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 0x7fec38101850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 -- 172.21.15.52:0/360382190 >> 172.21.15.52:0/360382190 conn(0x7fec38078360 msgr2=0x7fec38078770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 -- 172.21.15.52:0/360382190 shutdown_connections 2024-06-02T23:23:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.351+0000 7fec3e9eb700 1 -- 172.21.15.52:0/360382190 wait complete. 2024-06-02T23:23:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.352+0000 7fec3e9eb700 1 Processor -- start 2024-06-02T23:23:24.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.352+0000 7fec3e9eb700 1 -- start start 2024-06-02T23:23:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.352+0000 7fec3e9eb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 0x7fec3819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.352+0000 7fec3e9eb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec38101d90 0x7fec3819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.352+0000 7fec3e9eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 0x7fec381987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3e9eb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec38111550 con 0x7fec38107440 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3e9eb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec381113d0 con 0x7fec381013d0 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3e9eb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec381116d0 con 0x7fec38101d90 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 0x7fec381987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec37fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 0x7fec3819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 0x7fec381987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44592/0 (socket says 172.21.15.52:44592) 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 -- 172.21.15.52:0/2073250498 learned_addr learned my addr 172.21.15.52:0/2073250498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:24.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec377fe700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec38101d90 0x7fec3819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 -- 172.21.15.52:0/2073250498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec38101d90 msgr2=0x7fec3819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec38101d90 0x7fec3819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 -- 172.21.15.52:0/2073250498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 msgr2=0x7fec3819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 0x7fec3819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 -- 172.21.15.52:0/2073250498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec38198fa0 con 0x7fec38107440 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec37fff700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 0x7fec3819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3cf88700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 0x7fec381987c0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fec3000b780 tx=0x7fec30004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:24.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec357fa700 1 -- 172.21.15.52:0/2073250498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30021070 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec38199230 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.353+0000 7fec357fa700 1 -- 172.21.15.52:0/2073250498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec30007e30 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.354+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec38103210 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.354+0000 7fec357fa700 1 -- 172.21.15.52:0/2073250498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30004700 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.356+0000 7fec357fa700 1 -- 172.21.15.52:0/2073250498 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fec3001d450 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.357+0000 7fec357fa700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fec200a2cc0 0x7fec200a5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.357+0000 7fec37fff700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fec200a2cc0 0x7fec200a5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.357+0000 7fec357fa700 1 -- 172.21.15.52:0/2073250498 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fec3000f020 con 0x7fec38107440 2024-06-02T23:23:24.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.357+0000 7fec4926eb80 1 Processor -- start 2024-06-02T23:23:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.357+0000 7fec4926eb80 1 -- start start 2024-06-02T23:23:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec37fff700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fec200a2cc0 0x7fec200a5180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fec280096d0 tx=0x7fec28012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 0x559c16c957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16c96110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec4926eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c16b88100 con 0x559c16c9bec0 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec4926eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c16be69f0 con 0x559c16c95cf0 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec4926eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c16b4e600 con 0x559c16c9b5a0 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 0x559c16c957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 0x559c16c957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44596/0 (socket says 172.21.15.52:44596) 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 -- 172.21.15.52:0/2672890308 learned_addr learned my addr 172.21.15.52:0/2672890308 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec3cf88700 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16c96110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec37fff700 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 -- 172.21.15.52:0/2672890308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 msgr2=0x559c16c9b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 -- 172.21.15.52:0/2672890308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 msgr2=0x559c16c96110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16c96110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 -- 172.21.15.52:0/2672890308 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c16b35a70 con 0x559c16c9bec0 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec3cf88700 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16c96110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.358+0000 7fec377fe700 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 0x559c16c957b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009d20 tx=0x7fec2c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813c58ab3cd37289 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.359+0000 7fec1e7fc700 1 -- 172.21.15.52:0/2672890308 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c01e070 con 0x559c16c9bec0 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.359+0000 7fec1e7fc700 1 -- 172.21.15.52:0/2672890308 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec2c019ae0 con 0x559c16c9bec0 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.359+0000 7fec1e7fc700 1 -- 172.21.15.52:0/2672890308 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c00ea40 con 0x559c16c9bec0 2024-06-02T23:23:24.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.359+0000 7fec4926eb80 1 -- 172.21.15.52:0/2672890308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 msgr2=0x559c16c957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.359+0000 7fec4926eb80 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 0x559c16c957b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009d20 tx=0x7fec2c011720 comp rx=0 tx=0).stop 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 -- 172.21.15.52:0/2672890308 shutdown_connections 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16c96110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 --2- 172.21.15.52:0/2672890308 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9bec0 0x559c16c957b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 -- 172.21.15.52:0/2672890308 >> 172.21.15.52:0/2672890308 conn(0x559c16bc2220 msgr2=0x559c16c90aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 -- 172.21.15.52:0/2672890308 shutdown_connections 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 -- 172.21.15.52:0/2672890308 wait complete. 2024-06-02T23:23:24.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.360+0000 7fec4926eb80 1 -- start start 2024-06-02T23:23:24.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 Processor -- start 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c95cf0 0x559c16ca1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9ef20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c9f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c16b87160 con 0x559c16c95cf0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c16be29b0 con 0x559c16c9bec0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c16ca28a0 con 0x559c16c9b5a0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec4926eb80 10 client.0 _ll_register_callbacks cb 0x559c16b7c750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec377fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec377fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34882/0 (socket says 172.21.15.52:34882) 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec377fe700 1 -- 172.21.15.52:0/121682985 learned_addr learned my addr 172.21.15.52:0/121682985 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec3cf88700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c9f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.361+0000 7fec37fff700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c95cf0 0x559c16ca1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec377fe700 1 -- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 msgr2=0x559c16c9f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec377fe700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c9f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec377fe700 1 -- 172.21.15.52:0/121682985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c95cf0 msgr2=0x559c16ca1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec377fe700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c95cf0 0x559c16ca1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec377fe700 1 -- 172.21.15.52:0/121682985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c16bac8f0 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec377fe700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9ef20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0198b0 tx=0x7fec2c0198e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a10d006e34fd1c2f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec177fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec177fe700 1 -- 172.21.15.52:0/121682985 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2c027020 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec177fe700 1 -- 172.21.15.52:0/121682985 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fec2c004450 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.362+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec2c019ae0 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec177fe700 1 -- 172.21.15.52:0/121682985 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fec2c01e070 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec4926eb80 10 client.27835 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c16cace50 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec177fe700 1 -- 172.21.15.52:0/121682985 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec2c019ae0 con 0x559c16c9b5a0 2024-06-02T23:23:24.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec177fe700 1 -- 172.21.15.52:0/121682985 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fec2c004b20 con 0x559c16c9b5a0 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec4926eb80 10 client.27835 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec4926eb80 10 client.27835 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec4926eb80 20 client.27835 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.363+0000 7fec4926eb80 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16caf320 0x559c16cb1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.364+0000 7fec4926eb80 4 client.27835 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.364+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c16bf3ef0 con 0x559c16caf320 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.364+0000 7fec37fff700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16caf320 0x559c16cb1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.364+0000 7fec37fff700 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16caf320 0x559c16cb1710 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a5432ee72688a2a server_cookie=19a6653c8fe794f3 in_seq=0 out_seq=0 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.364+0000 7fec177fe700 10 client.27835 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec177fe700 1 -- 172.21.15.52:0/121682985 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c16bf3ef0 con 0x559c16caf320 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec177fe700 10 client.27835 handle_command_reply: tid=0 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec4926eb80 1 client.27835 shutdown 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 msgr2=0x559c16c9ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec4926eb80 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9ef20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0198b0 tx=0x7fec2c0198e0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 shutdown_connections 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec4926eb80 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16caf320 0x559c16cb1710 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.365+0000 7fec4926eb80 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c9f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec4926eb80 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c95cf0 0x559c16ca1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec4926eb80 1 --2- 172.21.15.52:0/121682985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9b5a0 0x559c16c9ef20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec177fe700 0 client.27835 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 >> 172.21.15.52:0/121682985 conn(0x559c16bc2220 msgr2=0x559c16c8fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 shutdown_connections 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec4926eb80 1 -- 172.21.15.52:0/121682985 wait complete. 2024-06-02T23:23:24.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.366+0000 7fec4926eb80 20 client.27835 trim_cache size 0 max 16384 2024-06-02T23:23:24.379 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:24.397 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.396+0000 7fec4926eb80 1 Processor -- start 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.396+0000 7fec4926eb80 1 -- start start 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.396+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c95cf0 0x559c16c94eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.396+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16c953f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c903d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec4926eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec381aa6f0 con 0x559c16c9b5a0 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec4926eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec381ac230 con 0x559c16c9bec0 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec4926eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec381aadc0 con 0x559c16c95cf0 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16c953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16c953f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44606/0 (socket says 172.21.15.52:44606) 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec377fe700 1 -- 172.21.15.52:0/3383076489 learned_addr learned my addr 172.21.15.52:0/3383076489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c95cf0 0x559c16c94eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec3cf88700 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c903d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 -- 172.21.15.52:0/3383076489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 msgr2=0x559c16c903d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c903d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 -- 172.21.15.52:0/3383076489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 msgr2=0x559c16c953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16c953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 -- 172.21.15.52:0/3383076489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c16c9ecf0 con 0x559c16c95cf0 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec37fff700 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c95cf0 0x559c16c94eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x559c16cb1cb0 tx=0x7fec28018000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d97fe727525dd42 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec1dffb700 1 -- 172.21.15.52:0/3383076489 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2802a450 con 0x559c16c95cf0 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.397+0000 7fec1dffb700 1 -- 172.21.15.52:0/3383076489 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec2802aaa0 con 0x559c16c95cf0 2024-06-02T23:23:24.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec1dffb700 1 -- 172.21.15.52:0/3383076489 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2801c930 con 0x559c16c95cf0 2024-06-02T23:23:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 -- 172.21.15.52:0/3383076489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c95cf0 msgr2=0x559c16c94eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c95cf0 0x559c16c94eb0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x559c16cb1cb0 tx=0x7fec28018000 comp rx=0 tx=0).stop 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 -- 172.21.15.52:0/3383076489 shutdown_connections 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c9bec0 0x559c16c903d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c95cf0 0x559c16c94eb0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 --2- 172.21.15.52:0/3383076489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16c953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 -- 172.21.15.52:0/3383076489 >> 172.21.15.52:0/3383076489 conn(0x559c16bc2220 msgr2=0x559c16c947a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 -- 172.21.15.52:0/3383076489 shutdown_connections 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.398+0000 7fec4926eb80 1 -- 172.21.15.52:0/3383076489 wait complete. 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.399+0000 7fec4926eb80 1 -- start start 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.399+0000 7fec4926eb80 1 Processor -- start 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.399+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16ca4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16ca5110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec4926eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9bec0 0x559c16ca5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec4926eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fec381abb60 con 0x559c16c9b5a0 2024-06-02T23:23:24.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec4926eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fec381ab490 con 0x559c16c95cf0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec4926eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec381a86b0 con 0x559c16c9bec0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec4926eb80 10 client.0 _ll_register_callbacks cb 0x559c16b7c750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16ca4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16ca4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55332/0 (socket says 172.21.15.52:55332) 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 -- 172.21.15.52:0/4182556945 learned_addr learned my addr 172.21.15.52:0/4182556945 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec377fe700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16ca5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec3cf88700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9bec0 0x559c16ca5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 -- 172.21.15.52:0/4182556945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9bec0 msgr2=0x559c16ca5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9bec0 0x559c16ca5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 -- 172.21.15.52:0/4182556945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 msgr2=0x559c16ca5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16ca5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 -- 172.21.15.52:0/4182556945 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c16d227c0 con 0x559c16c95cf0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec37fff700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16ca4bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fec28012010 tx=0x7fec280043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=682b8b8064fac7be server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:24.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.400+0000 7fec177fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec177fe700 1 -- 172.21.15.52:0/4182556945 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec28038070 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec177fe700 1 -- 172.21.15.52:0/4182556945 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fec2802a710 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec2802aaa0 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec177fe700 1 -- 172.21.15.52:0/4182556945 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fec28044050 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 10 client.28121 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559c16d23a80 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec177fe700 1 -- 172.21.15.52:0/4182556945 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fec2802aaa0 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec177fe700 1 -- 172.21.15.52:0/4182556945 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fec28004be0 con 0x559c16c95cf0 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 10 client.28121 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 10 client.28121 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 20 client.28121 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:24.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16d241d0 0x559c16d265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 4 client.28121 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.75, "await_for": 45.0}] 2024-06-02T23:23:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.401+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.75, "await_for": 45.0}) -- 0x559c16bee470 con 0x559c16d241d0 2024-06-02T23:23:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.402+0000 7fec377fe700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16d241d0 0x559c16d265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.402+0000 7fec377fe700 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16d241d0 0x559c16d265f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278e50a9426d8b9 server_cookie=5c0d6727f7aaae95 in_seq=0 out_seq=0 2024-06-02T23:23:24.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.402+0000 7fec177fe700 10 client.28121 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:24.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.458+0000 7fec177fe700 1 -- 172.21.15.52:0/4182556945 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559c16bee470 con 0x559c16d241d0 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.458+0000 7fec177fe700 10 client.28121 handle_command_reply: tid=0 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 client.28121 shutdown 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 msgr2=0x559c16ca4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16ca4bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fec28012010 tx=0x7fec280043b0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 shutdown_connections 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559c16d241d0 0x559c16d265f0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c16c9bec0 0x559c16ca5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c16c95cf0 0x559c16ca4bd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 --2- 172.21.15.52:0/4182556945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c16c9b5a0 0x559c16ca5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec177fe700 0 client.28121 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 >> 172.21.15.52:0/4182556945 conn(0x559c16bc2220 msgr2=0x559c16c95030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 shutdown_connections 2024-06-02T23:23:24.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 1 -- 172.21.15.52:0/4182556945 wait complete. 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.459+0000 7fec4926eb80 20 client.28121 trim_cache size 0 max 16384 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 38, 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "9d5fe456": { 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 38, 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:23:24.461 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 140.8, 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:24.462 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fec200a2cc0 msgr2=0x7fec200a5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fec200a2cc0 0x7fec200a5180 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fec280096d0 tx=0x7fec28012040 comp rx=0 tx=0).stop 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 msgr2=0x7fec381987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 0x7fec381987c0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fec3000b780 tx=0x7fec30004d00 comp rx=0 tx=0).stop 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 shutdown_connections 2024-06-02T23:23:24.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fec200a2cc0 0x7fec200a5180 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fec38107440 0x7fec381987c0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec381013d0 0x7fec3819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 --2- 172.21.15.52:0/2073250498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec38101d90 0x7fec3819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:24.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 >> 172.21.15.52:0/2073250498 conn(0x7fec38078360 msgr2=0x7fec38105420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:24.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 shutdown_connections 2024-06-02T23:23:24.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:24.460+0000 7fec3e9eb700 1 -- 172.21.15.52:0/2073250498 wait complete. 2024-06-02T23:23:24.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.93 MB/sec execute 513 sec latency 207.910 ms 2024-06-02T23:23:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:25 smithi174 ceph-mon[35994]: pgmap v777: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-02T23:23:25.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.92 MB/sec execute 514 sec latency 1207.998 ms 2024-06-02T23:23:25.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:25 smithi052 ceph-mon[32340]: pgmap v777: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-02T23:23:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:25 smithi123 ceph-mon[35977]: pgmap v777: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 269 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-02T23:23:25.822 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '9d5fe456', quiesce took 11.5 seconds. Will release after: 9.25 2024-06-02T23:23:26.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.92 MB/sec execute 515 sec latency 2208.103 ms 2024-06-02T23:23:26.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.959+0000 7f097cf9c700 1 -- 172.21.15.52:0/34363443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f097806ac60 msgr2=0x7f097806b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:26.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.959+0000 7f097cf9c700 1 --2- 172.21.15.52:0/34363443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f097806ac60 0x7f097806b0e0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f096c0099c0 tx=0x7f096c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:23:26.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 -- 172.21.15.52:0/34363443 shutdown_connections 2024-06-02T23:23:26.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 --2- 172.21.15.52:0/34363443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097806b620 0x7f097810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:26.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 --2- 172.21.15.52:0/34363443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0978108580 0x7f0978108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:26.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 --2- 172.21.15.52:0/34363443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f097806ac60 0x7f097806b0e0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:26.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 -- 172.21.15.52:0/34363443 >> 172.21.15.52:0/34363443 conn(0x7f0978078430 msgr2=0x7f0978078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:26.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 -- 172.21.15.52:0/34363443 shutdown_connections 2024-06-02T23:23:26.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 -- 172.21.15.52:0/34363443 wait complete. 2024-06-02T23:23:26.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.960+0000 7f097cf9c700 1 Processor -- start 2024-06-02T23:23:26.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 -- start start 2024-06-02T23:23:26.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f097806ac60 0x7f0978101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:26.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097806b620 0x7f0978101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:26.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 0x7f0978101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:26.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0978112690 con 0x7f0978108580 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0978112510 con 0x7f097806b620 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097cf9c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0978112810 con 0x7f097806ac60 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 0x7f0978101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 0x7f0978101e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44638/0 (socket says 172.21.15.52:44638) 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 -- 172.21.15.52:0/4002249035 learned_addr learned my addr 172.21.15.52:0/4002249035 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0975d9b700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097806b620 0x7f0978101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f097659c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f097806ac60 0x7f0978101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 -- 172.21.15.52:0/4002249035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f097806ac60 msgr2=0x7f0978101400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f097806ac60 0x7f0978101400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 -- 172.21.15.52:0/4002249035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097806b620 msgr2=0x7f0978101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097806b620 0x7f0978101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 -- 172.21.15.52:0/4002249035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09781a7050 con 0x7f0978108580 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.961+0000 7f0976d9d700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 0x7f0978101e80 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0970007ee0 tx=0x7f097000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:26.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.962+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0970004bb0 con 0x7f0978108580 2024-06-02T23:23:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.962+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0970004d50 con 0x7f0978108580 2024-06-02T23:23:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.962+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09781a7340 con 0x7f0978108580 2024-06-02T23:23:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.962+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09700058e0 con 0x7f0978108580 2024-06-02T23:23:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.962+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09781a7800 con 0x7f0978108580 2024-06-02T23:23:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.963+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f097810aa10 con 0x7f0978108580 2024-06-02T23:23:26.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.964+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f097001a020 con 0x7f0978108580 2024-06-02T23:23:26.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.965+0000 7f09677fe700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09600a2c20 0x7f09600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:26.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.965+0000 7f097659c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09600a2c20 0x7f09600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:26.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.965+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09700cedd0 con 0x7f0978108580 2024-06-02T23:23:26.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.965+0000 7f097659c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09600a2c20 0x7f09600a50e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f09680045b0 tx=0x7f096800b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:26.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:26.969+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0970010070 con 0x7f0978108580 2024-06-02T23:23:27.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.219+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f097810a3f0 con 0x7f0978108580 2024-06-02T23:23:27.220 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.220+0000 7f09677fe700 1 -- 172.21.15.52:0/4002249035 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f097009bd60 con 0x7f0978108580 2024-06-02T23:23:27.220 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:23:27.220 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09600a2c20 msgr2=0x7f09600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09600a2c20 0x7f09600a50e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f09680045b0 tx=0x7f096800b3c0 comp rx=0 tx=0).stop 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 msgr2=0x7f0978101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 0x7f0978101e80 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f0970007ee0 tx=0x7f097000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 shutdown_connections 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09600a2c20 0x7f09600a50e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0978108580 0x7f0978101e80 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f097806ac60 0x7f0978101400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 --2- 172.21.15.52:0/4002249035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f097806b620 0x7f0978101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 >> 172.21.15.52:0/4002249035 conn(0x7f0978078430 msgr2=0x7f09780fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 shutdown_connections 2024-06-02T23:23:27.223 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:27.222+0000 7f097cf9c700 1 -- 172.21.15.52:0/4002249035 wait complete. 2024-06-02T23:23:27.224 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:23:27.611 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:27 smithi052 ceph-mon[32340]: pgmap v778: 97 pgs: 97 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:23:27.612 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4002249035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:27 smithi174 ceph-mon[35994]: pgmap v778: 97 pgs: 97 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:23:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4002249035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:27.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.91 MB/sec execute 516 sec latency 3208.154 ms 2024-06-02T23:23:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:27 smithi123 ceph-mon[35977]: pgmap v778: 97 pgs: 97 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 268 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:23:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4002249035' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:27.849 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:23:28.107 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:28.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.91 MB/sec execute 517 sec latency 4208.223 ms 2024-06-02T23:23:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:29 smithi174 ceph-mon[35994]: pgmap v779: 97 pgs: 97 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-02T23:23:29.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.91 MB/sec execute 518 sec latency 5208.278 ms 2024-06-02T23:23:29.646 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:29 smithi052 ceph-mon[32340]: pgmap v779: 97 pgs: 97 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-02T23:23:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:29 smithi123 ceph-mon[35977]: pgmap v779: 97 pgs: 97 active+clean; 100 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.0 MiB/s wr, 130 op/s 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 -- 172.21.15.52:0/1614668987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 msgr2=0x7f3ad01088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 --2- 172.21.15.52:0/1614668987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad01088e0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009990 tx=0x7f3acc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 -- 172.21.15.52:0/1614668987 shutdown_connections 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 --2- 172.21.15.52:0/1614668987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad006b580 0x7f3ad010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 --2- 172.21.15.52:0/1614668987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad01088e0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 --2- 172.21.15.52:0/1614668987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad006abc0 0x7f3ad006b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 -- 172.21.15.52:0/1614668987 >> 172.21.15.52:0/1614668987 conn(0x7f3ad0078340 msgr2=0x7f3ad0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 -- 172.21.15.52:0/1614668987 shutdown_connections 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.012+0000 7f3ad838f700 1 -- 172.21.15.52:0/1614668987 wait complete. 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 Processor -- start 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 -- start start 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad006abc0 0x7f3ad0101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad006b580 0x7f3ad01019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad0101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112690 con 0x7f3ad0108500 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112510 con 0x7f3ad006b580 2024-06-02T23:23:30.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad838f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad0112810 con 0x7f3ad006abc0 2024-06-02T23:23:30.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad692c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad0101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad692c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad0101f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44660/0 (socket says 172.21.15.52:44660) 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad692c700 1 -- 172.21.15.52:0/735496740 learned_addr learned my addr 172.21.15.52:0/735496740 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad612b700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad006abc0 0x7f3ad0101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.013+0000 7f3ad592a700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad006b580 0x7f3ad01019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad692c700 1 -- 172.21.15.52:0/735496740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad006abc0 msgr2=0x7f3ad0101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad692c700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad006abc0 0x7f3ad0101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad692c700 1 -- 172.21.15.52:0/735496740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad006b580 msgr2=0x7f3ad01019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad692c700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad006b580 0x7f3ad01019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad692c700 1 -- 172.21.15.52:0/735496740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad01a70f0 con 0x7f3ad0108500 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad692c700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad0101f10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800b970 tx=0x7f3ac800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ac77fe700 1 -- 172.21.15.52:0/735496740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac800d860 con 0x7f3ad0108500 2024-06-02T23:23:30.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ac77fe700 1 -- 172.21.15.52:0/735496740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ac800e430 con 0x7f3ad0108500 2024-06-02T23:23:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad01a73e0 con 0x7f3ad0108500 2024-06-02T23:23:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ac77fe700 1 -- 172.21.15.52:0/735496740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac8017660 con 0x7f3ad0108500 2024-06-02T23:23:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.014+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad01a7970 con 0x7f3ad0108500 2024-06-02T23:23:30.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.017+0000 7f3ac77fe700 1 -- 172.21.15.52:0/735496740 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ac800e5e0 con 0x7f3ad0108500 2024-06-02T23:23:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.018+0000 7f3ac77fe700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3abc0a2ca0 0x7f3abc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.018+0000 7f3ad612b700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3abc0a2ca0 0x7f3abc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.018+0000 7f3ac77fe700 1 -- 172.21.15.52:0/735496740 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ac80ceb40 con 0x7f3ad0108500 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.018+0000 7f3ad612b700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3abc0a2ca0 0x7f3abc0a5160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b5c0 tx=0x7f3acc005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 Processor -- start 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 -- start start 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 0x55dd441af340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd4409bf60 con 0x55dd441aef60 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd440fa850 con 0x55dd441af880 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd44062460 con 0x55dd441a97a0 2024-06-02T23:23:30.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ad612b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 0x55dd441af340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ad612b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 0x55dd441af340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44664/0 (socket says 172.21.15.52:44664) 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ad612b700 1 -- 172.21.15.52:0/3196080554 learned_addr learned my addr 172.21.15.52:0/3196080554 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ad592a700 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ad692c700 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.019+0000 7f3ad612b700 1 -- 172.21.15.52:0/3196080554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 msgr2=0x55dd441a9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ad612b700 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441a9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ad612b700 1 -- 172.21.15.52:0/3196080554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 msgr2=0x55dd441a9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ad612b700 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ad612b700 1 -- 172.21.15.52:0/3196080554 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd440498d0 con 0x55dd441aef60 2024-06-02T23:23:30.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ad612b700 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 0x55dd441af340 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009990 tx=0x7f3acc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827c8cdb7417deb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:30.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ac4d70700 1 -- 172.21.15.52:0/3196080554 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc034070 con 0x55dd441aef60 2024-06-02T23:23:30.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ac4d70700 1 -- 172.21.15.52:0/3196080554 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3acc024930 con 0x55dd441aef60 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.020+0000 7f3ac4d70700 1 -- 172.21.15.52:0/3196080554 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc02fa30 con 0x55dd441aef60 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/3196080554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 msgr2=0x55dd441af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 0x55dd441af340 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3acc009990 tx=0x7f3acc01f000 comp rx=0 tx=0).stop 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/3196080554 shutdown_connections 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441a9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441aef60 0x55dd441af340 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/3196080554 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/3196080554 >> 172.21.15.52:0/3196080554 conn(0x55dd440d6080 msgr2=0x55dd441a4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/3196080554 shutdown_connections 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/3196080554 wait complete. 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 -- start start 2024-06-02T23:23:30.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.021+0000 7f3ae2c12b80 1 Processor -- start 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 0x55dd441b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd4409afc0 con 0x55dd441af880 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd440f6810 con 0x55dd441aef60 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd441b6200 con 0x55dd441a97a0 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 10 client.0 _ll_register_callbacks cb 0x55dd440905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 0x55dd441b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 0x55dd441b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44666/0 (socket says 172.21.15.52:44666) 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 -- 172.21.15.52:0/928508498 learned_addr learned my addr 172.21.15.52:0/928508498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad612b700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad592a700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 -- 172.21.15.52:0/928508498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 msgr2=0x55dd441b55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441b55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 -- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 msgr2=0x55dd441b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 -- 172.21.15.52:0/928508498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd440c0750 con 0x55dd441af880 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ad692c700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 0x55dd441b2e10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80cc670 tx=0x7f3ac800e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3e2d35e51d6be9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/928508498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ac80b3070 con 0x55dd441af880 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc024930 con 0x55dd441af880 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.022+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/928508498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ac80a3bb0 con 0x55dd441af880 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/928508498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3ac809b3d0 con 0x55dd441af880 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 10 client.18690 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dd441c9170 con 0x55dd441af880 2024-06-02T23:23:30.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/928508498 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3acc024930 con 0x55dd441af880 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/928508498 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3ac801f030 con 0x55dd441af880 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 10 client.18690 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 10 client.18690 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 20 client.18690 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd441cb700 0x55dd441cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 4 client.18690 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.023+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd44107d50 con 0x55dd441cb700 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.024+0000 7f3ad612b700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd441cb700 0x55dd441cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.024+0000 7f3ad612b700 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd441cb700 0x55dd441cdb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3eb170a96bf2111a server_cookie=be72edb55cdf6782 in_seq=0 out_seq=0 2024-06-02T23:23:30.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.024+0000 7f3ab9ffb700 10 client.18690 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:23:30.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/928508498 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dd44107d50 con 0x55dd441cb700 2024-06-02T23:23:30.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ab9ffb700 10 client.18690 handle_command_reply: tid=0 2024-06-02T23:23:30.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 client.18690 shutdown 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 msgr2=0x55dd441b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 0x55dd441b2e10 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3ac80cc670 tx=0x7f3ac800e950 comp rx=0 tx=0).stop 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 shutdown_connections 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441af880 0x55dd441b2e10 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441a97a0 0x55dd441b55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd441cb700 0x55dd441cdb20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/928508498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 >> 172.21.15.52:0/928508498 conn(0x55dd440d6080 msgr2=0x55dd441a3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ab9ffb700 0 client.18690 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 shutdown_connections 2024-06-02T23:23:30.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/928508498 wait complete. 2024-06-02T23:23:30.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.025+0000 7f3ae2c12b80 20 client.18690 trim_cache size 0 max 16384 2024-06-02T23:23:30.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.040+0000 7f3ae2c12b80 1 Processor -- start 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.040+0000 7f3ae2c12b80 1 -- start start 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.040+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441aef60 0x55dd441b18d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01aa890 con 0x55dd441a97a0 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ac3d0 con 0x55dd441af880 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01aaf60 con 0x55dd441aef60 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44668/0 (socket says 172.21.15.52:44668) 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 -- 172.21.15.52:0/2702574740 learned_addr learned my addr 172.21.15.52:0/2702574740 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad692c700 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad592a700 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441aef60 0x55dd441b18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 -- 172.21.15.52:0/2702574740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441aef60 msgr2=0x55dd441b18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441aef60 0x55dd441b18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 -- 172.21.15.52:0/2702574740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 msgr2=0x55dd441a3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 -- 172.21.15.52:0/2702574740 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd441a9260 con 0x55dd441a97a0 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad592a700 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441aef60 0x55dd441b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3ad612b700 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b04f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00bad0 tx=0x7f3acc00b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a624b5dd0d0d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3abbfff700 1 -- 172.21.15.52:0/2702574740 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc034070 con 0x55dd441a97a0 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3abbfff700 1 -- 172.21.15.52:0/2702574740 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3acc035040 con 0x55dd441a97a0 2024-06-02T23:23:30.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.041+0000 7f3abbfff700 1 -- 172.21.15.52:0/2702574740 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc02f950 con 0x55dd441a97a0 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/2702574740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 msgr2=0x55dd441b04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b04f0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00bad0 tx=0x7f3acc00b7d0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/2702574740 shutdown_connections 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441af880 0x55dd441a3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b04f0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/2702574740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441aef60 0x55dd441b18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/2702574740 >> 172.21.15.52:0/2702574740 conn(0x55dd440d6080 msgr2=0x55dd441b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/2702574740 shutdown_connections 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/2702574740 wait complete. 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.042+0000 7f3ae2c12b80 1 -- start start 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 Processor -- start 2024-06-02T23:23:30.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b6c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441af880 0x55dd4422e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01abd00 con 0x55dd441a97a0 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad01ab630 con 0x55dd441aef60 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad019c6c0 con 0x55dd441af880 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ae2c12b80 10 client.0 _ll_register_callbacks cb 0x55dd440905b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44672/0 (socket says 172.21.15.52:44672) 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 -- 172.21.15.52:0/1947858265 learned_addr learned my addr 172.21.15.52:0/1947858265 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad692c700 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441af880 0x55dd4422e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 -- 172.21.15.52:0/1947858265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441af880 msgr2=0x55dd4422e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441af880 0x55dd4422e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 -- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 msgr2=0x55dd441b7140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:23:30.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b7140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.043+0000 7f3ad612b700 1 -- 172.21.15.52:0/1947858265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd4422e9f0 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ad612b700 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b6c00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3acc032040 tx=0x7f3acc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba8c4fc9b9a3c69 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/1947858265 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3acc034040 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/1947858265 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3acc024b70 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3acc035040 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/1947858265 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3acc0246c0 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ae2c12b80 10 client.18696 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55dd44235d10 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/1947858265 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3acc035040 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.044+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/1947858265 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3acc03c620 con 0x55dd441a97a0 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ae2c12b80 10 client.18696 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:30.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ae2c12b80 10 client.18696 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ae2c12b80 20 client.18696 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd44235f90 0x55dd44238380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ae2c12b80 4 client.18696 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dd441022d0 con 0x55dd44235f90 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ad592a700 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd44235f90 0x55dd44238380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ad592a700 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd44235f90 0x55dd44238380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=291dc5de1edcfd0 server_cookie=4a44da04b378a594 in_seq=0 out_seq=0 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ab9ffb700 10 client.18696 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.045+0000 7f3ab9ffb700 1 -- 172.21.15.52:0/1947858265 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dd441022d0 con 0x55dd44235f90 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ab9ffb700 10 client.18696 handle_command_reply: tid=0 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 client.18696 shutdown 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 msgr2=0x55dd441b6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b6c00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3acc032040 tx=0x7f3acc00be70 comp rx=0 tx=0).stop 2024-06-02T23:23:30.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 shutdown_connections 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd441af880 0x55dd4422e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd441a97a0 0x55dd441b6c00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55dd44235f90 0x55dd44238380 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 --2- 172.21.15.52:0/1947858265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd441aef60 0x55dd441b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ab9ffb700 0 client.18696 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 >> 172.21.15.52:0/1947858265 conn(0x55dd440d6080 msgr2=0x55dd441a8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 shutdown_connections 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 1 -- 172.21.15.52:0/1947858265 wait complete. 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.046+0000 7f3ae2c12b80 20 client.18696 trim_cache size 0 max 16384 2024-06-02T23:23:30.051 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3abc0a2ca0 msgr2=0x7f3abc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3abc0a2ca0 0x7f3abc0a5160 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00b5c0 tx=0x7f3acc005960 comp rx=0 tx=0).stop 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 msgr2=0x7f3ad0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad0101f10 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f3ac800b970 tx=0x7f3ac800bc80 comp rx=0 tx=0).stop 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 shutdown_connections 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3abc0a2ca0 0x7f3abc0a5160 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3ad0108500 0x7f3ad0101f10 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ad006abc0 0x7f3ad0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 --2- 172.21.15.52:0/735496740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ad006b580 0x7f3ad01019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:30.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 >> 172.21.15.52:0/735496740 conn(0x7f3ad0078340 msgr2=0x7f3ad01001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:30.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 shutdown_connections 2024-06-02T23:23:30.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:30.047+0000 7f3ad838f700 1 -- 172.21.15.52:0/735496740 wait complete. 2024-06-02T23:23:30.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.90 MB/sec execute 519 sec latency 6208.358 ms 2024-06-02T23:23:30.883 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:23:31.137 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:31.581 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:31 smithi052 ceph-mon[32340]: pgmap v780: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 975 KiB/s wr, 136 op/s 2024-06-02T23:23:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:31 smithi174 ceph-mon[35994]: pgmap v780: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 975 KiB/s wr, 136 op/s 2024-06-02T23:23:31.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.90 MB/sec execute 520 sec latency 7208.437 ms 2024-06-02T23:23:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:31 smithi123 ceph-mon[35977]: pgmap v780: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 239 KiB/s rd, 975 KiB/s wr, 136 op/s 2024-06-02T23:23:32.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.90 MB/sec execute 521 sec latency 8208.500 ms 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 -- 172.21.15.52:0/193836646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406b5c0 msgr2=0x7f6b8410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/193836646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406b5c0 0x7f6b8410f370 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c00b0a0 tx=0x7f6b7c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 -- 172.21.15.52:0/193836646 shutdown_connections 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/193836646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406b5c0 0x7f6b8410f370 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/193836646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b84108530 0x7f6b84108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/193836646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b8406ac00 0x7f6b8406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 -- 172.21.15.52:0/193836646 >> 172.21.15.52:0/193836646 conn(0x7f6b84077400 msgr2=0x7f6b84077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 -- 172.21.15.52:0/193836646 shutdown_connections 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 -- 172.21.15.52:0/193836646 wait complete. 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 Processor -- start 2024-06-02T23:23:33.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.041+0000 7f6b8a43b700 1 -- start start 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b8a43b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 0x7f6b8419e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b8a43b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8406b5c0 0x7f6b8419ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b8a43b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84108530 0x7f6b84198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b8a43b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84112690 con 0x7f6b8406ac00 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b8a43b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84112510 con 0x7f6b84108530 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b8a43b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b84112810 con 0x7f6b8406b5c0 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 0x7f6b8419e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 0x7f6b8419e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44682/0 (socket says 172.21.15.52:44682) 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 -- 172.21.15.52:0/2581398141 learned_addr learned my addr 172.21.15.52:0/2581398141 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b837fe700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8406b5c0 0x7f6b8419ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b889d8700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84108530 0x7f6b84198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 -- 172.21.15.52:0/2581398141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8406b5c0 msgr2=0x7f6b8419ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8406b5c0 0x7f6b8419ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 -- 172.21.15.52:0/2581398141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84108530 msgr2=0x7f6b84198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84108530 0x7f6b84198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 -- 172.21.15.52:0/2581398141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b84199090 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b889d8700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84108530 0x7f6b84198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.042+0000 7f6b83fff700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 0x7f6b8419e770 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400b530 tx=0x7f6b7400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.043+0000 7f6b817fa700 1 -- 172.21.15.52:0/2581398141 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74015020 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.043+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b84199380 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.043+0000 7f6b817fa700 1 -- 172.21.15.52:0/2581398141 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b74011ce0 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.043+0000 7f6b817fa700 1 -- 172.21.15.52:0/2581398141 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74010d60 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.043+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b841a5640 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.045+0000 7f6b817fa700 1 -- 172.21.15.52:0/2581398141 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6b74018020 con 0x7f6b8406ac00 2024-06-02T23:23:33.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.046+0000 7f6b817fa700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b6c0a2c20 0x7f6b6c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.046+0000 7f6b817fa700 1 -- 172.21.15.52:0/2581398141 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6b740ce910 con 0x7f6b8406ac00 2024-06-02T23:23:33.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.046+0000 7f6b837fe700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b6c0a2c20 0x7f6b6c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.046+0000 7f6b837fe700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b6c0a2c20 0x7f6b6c0a50e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6b78005ea0 tx=0x7f6b78005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:33.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.046+0000 7f6b94cbeb80 1 Processor -- start 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 -- start start 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 0x55d532303850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fd670 0x55d5322fdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d53230a210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5321ebf60 con 0x55d532303470 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d53224a850 con 0x55d5322fe030 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5321b2460 con 0x55d5322fd670 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 0x55d532303850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 0x55d532303850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44692/0 (socket says 172.21.15.52:44692) 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 -- 172.21.15.52:0/1380894787 learned_addr learned my addr 172.21.15.52:0/1380894787 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b837fe700 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fd670 0x55d5322fdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b889d8700 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d53230a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 -- 172.21.15.52:0/1380894787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fd670 msgr2=0x55d5322fdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fd670 0x55d5322fdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 -- 172.21.15.52:0/1380894787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 msgr2=0x55d53230a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d53230a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.047+0000 7f6b83fff700 1 -- 172.21.15.52:0/1380894787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5321998d0 con 0x55d532303470 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.048+0000 7f6b83fff700 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 0x55d532303850 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6b740975a0 tx=0x7f6b7409ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dadc1e55e72cc86 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.048+0000 7f6b6a7fc700 1 -- 172.21.15.52:0/1380894787 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b740104a0 con 0x55d532303470 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.048+0000 7f6b6a7fc700 1 -- 172.21.15.52:0/1380894787 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b7409b840 con 0x55d532303470 2024-06-02T23:23:33.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.048+0000 7f6b6a7fc700 1 -- 172.21.15.52:0/1380894787 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7409f120 con 0x55d532303470 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1380894787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 msgr2=0x55d532303850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 0x55d532303850 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f6b740975a0 tx=0x7f6b7409ac80 comp rx=0 tx=0).stop 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1380894787 shutdown_connections 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d53230a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d532303470 0x55d532303850 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1380894787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fd670 0x55d5322fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1380894787 >> 172.21.15.52:0/1380894787 conn(0x55d532226080 msgr2=0x55d5322f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1380894787 shutdown_connections 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1380894787 wait complete. 2024-06-02T23:23:33.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 -- start start 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.049+0000 7f6b94cbeb80 1 Processor -- start 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fd670 0x55d532309f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 0x55d532307240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d532307780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5321eafc0 con 0x55d5322fe030 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d532246810 con 0x55d5322fd670 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d53230a750 con 0x55d532303470 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b94cbeb80 10 client.0 _ll_register_callbacks cb 0x55d5321e05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 0x55d532307240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fd670 0x55d532309f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b837fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 0x55d532307240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44696/0 (socket says 172.21.15.52:44696) 2024-06-02T23:23:33.054 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b837fe700 1 -- 172.21.15.52:0/390328386 learned_addr learned my addr 172.21.15.52:0/390328386 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b889d8700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d532307780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b83fff700 1 -- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 msgr2=0x55d532307780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b83fff700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d532307780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b83fff700 1 -- 172.21.15.52:0/390328386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 msgr2=0x55d532307240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b83fff700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 0x55d532307240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b83fff700 1 -- 172.21.15.52:0/390328386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d532210750 con 0x55d5322fd670 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.050+0000 7f6b837fe700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 0x55d532307240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b83fff700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fd670 0x55d532309f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b740975a0 tx=0x7f6b74009450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3eb0637264a72f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b637fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b637fe700 1 -- 172.21.15.52:0/390328386 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7409f4f0 con 0x55d5322fd670 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b637fe700 1 -- 172.21.15.52:0/390328386 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6b7409f120 con 0x55d5322fd670 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7409b840 con 0x55d5322fd670 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b637fe700 1 -- 172.21.15.52:0/390328386 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b740a3db0 con 0x55d5322fd670 2024-06-02T23:23:33.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 10 client.28148 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d532314e70 con 0x55d5322fd670 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b637fe700 1 -- 172.21.15.52:0/390328386 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b7409b840 con 0x55d5322fd670 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b637fe700 1 -- 172.21.15.52:0/390328386 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6b7409a590 con 0x55d5322fd670 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 10 client.28148 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 10 client.28148 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 20 client.28148 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532317340 0x55d532319730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 4 client.28148 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.051+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d532257d50 con 0x55d532317340 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.052+0000 7f6b837fe700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532317340 0x55d532319730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.052+0000 7f6b637fe700 10 client.28148 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:23:33.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.052+0000 7f6b837fe700 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532317340 0x55d532319730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=68b465ff1458ac5b server_cookie=2cd1581814987035 in_seq=0 out_seq=0 2024-06-02T23:23:33.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b637fe700 1 -- 172.21.15.52:0/390328386 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d532257d50 con 0x55d532317340 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b637fe700 10 client.28148 handle_command_reply: tid=0 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 client.28148 shutdown 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fd670 msgr2=0x55d532309f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fd670 0x55d532309f40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f6b740975a0 tx=0x7f6b74009450 comp rx=0 tx=0).stop 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 shutdown_connections 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532317340 0x55d532319730 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d532307780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fd670 0x55d532309f40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b637fe700 0 client.28148 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/390328386 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fe030 0x55d532307240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.053+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 >> 172.21.15.52:0/390328386 conn(0x55d532226080 msgr2=0x55d5322f3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.054+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 shutdown_connections 2024-06-02T23:23:33.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.054+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/390328386 wait complete. 2024-06-02T23:23:33.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.054+0000 7f6b94cbeb80 20 client.28148 trim_cache size 0 max 16384 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 Processor -- start 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 -- start start 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d532302960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fe030 0x55d5322f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d532303470 0x55d5322f43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841aa8a0 con 0x55d5322fd670 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841ac3e0 con 0x55d532303470 2024-06-02T23:23:33.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841aaf70 con 0x55d5322fe030 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d532302960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d532302960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44698/0 (socket says 172.21.15.52:44698) 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 -- 172.21.15.52:0/1213539691 learned_addr learned my addr 172.21.15.52:0/1213539691 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b837fe700 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fe030 0x55d5322f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b889d8700 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d532303470 0x55d5322f43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 -- 172.21.15.52:0/1213539691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fe030 msgr2=0x55d5322f3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fe030 0x55d5322f3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 -- 172.21.15.52:0/1213539691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d532303470 msgr2=0x55d5322f43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d532303470 0x55d5322f43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.069+0000 7f6b83fff700 1 -- 172.21.15.52:0/1213539691 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5323057b0 con 0x55d5322fd670 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.070+0000 7f6b83fff700 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d532302960 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6b7409ae40 tx=0x7f6b74004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc5ff2f6cd48634 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.070+0000 7f6b69ffb700 1 -- 172.21.15.52:0/1213539691 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7409ac80 con 0x55d5322fd670 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.070+0000 7f6b69ffb700 1 -- 172.21.15.52:0/1213539691 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b7409b840 con 0x55d5322fd670 2024-06-02T23:23:33.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.070+0000 7f6b69ffb700 1 -- 172.21.15.52:0/1213539691 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b74099440 con 0x55d5322fd670 2024-06-02T23:23:33.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.070+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1213539691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 msgr2=0x55d532302960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.070+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d532302960 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f6b7409ae40 tx=0x7f6b74004860 comp rx=0 tx=0).stop 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1213539691 shutdown_connections 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d532303470 0x55d5322f43c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d532302960 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/1213539691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5322fe030 0x55d5322f3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1213539691 >> 172.21.15.52:0/1213539691 conn(0x55d532226080 msgr2=0x55d5322f8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1213539691 shutdown_connections 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/1213539691 wait complete. 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 -- start start 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.071+0000 7f6b94cbeb80 1 Processor -- start 2024-06-02T23:23:33.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d53231cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d532319fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d53231a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841abd10 con 0x55d5322fd670 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841ab640 con 0x55d5322fe030 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6b841a8860 con 0x55d532303470 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 10 client.0 _ll_register_callbacks cb 0x55d5321e05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d53231cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d53231cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44714/0 (socket says 172.21.15.52:44714) 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 -- 172.21.15.52:0/985503329 learned_addr learned my addr 172.21.15.52:0/985503329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b889d8700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d53231a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b837fe700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d532319fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 -- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 msgr2=0x55d53231a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d53231a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 -- 172.21.15.52:0/985503329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 msgr2=0x55d532319fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d532319fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 -- 172.21.15.52:0/985503329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d53231aa60 con 0x55d5322fd670 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b889d8700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d53231a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b83fff700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d53231cc10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6b740993f0 tx=0x7f6b7409a960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cc462948f6a048 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b637fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b637fe700 1 -- 172.21.15.52:0/985503329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b7409ac80 con 0x55d5322fd670 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b637fe700 1 -- 172.21.15.52:0/985503329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6b740661b0 con 0x55d5322fd670 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.072+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b7409b840 con 0x55d5322fd670 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b637fe700 1 -- 172.21.15.52:0/985503329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6b7409f4f0 con 0x55d5322fd670 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 10 client.18714 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:33.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55d53231aa60 con 0x55d5322fd670 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b637fe700 1 -- 172.21.15.52:0/985503329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6b7409b840 con 0x55d5322fd670 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b637fe700 1 -- 172.21.15.52:0/985503329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6b7409a000 con 0x55d5322fd670 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 10 client.18714 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 10 client.18714 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 20 client.18714 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532381b90 0x55d532383f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 4 client.18714 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5322522d0 con 0x55d532381b90 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.073+0000 7f6b837fe700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532381b90 0x55d532383f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b837fe700 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532381b90 0x55d532383f80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bfa3850f125e36bf server_cookie=a060cb1063e69452 in_seq=0 out_seq=0 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b637fe700 10 client.18714 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b637fe700 1 -- 172.21.15.52:0/985503329 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5322522d0 con 0x55d532381b90 2024-06-02T23:23:33.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b637fe700 10 client.18714 handle_command_reply: tid=0 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 client.18714 shutdown 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 msgr2=0x55d53231cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d53231cc10 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f6b740993f0 tx=0x7f6b7409a960 comp rx=0 tx=0).stop 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 shutdown_connections 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55d532381b90 0x55d532383f80 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d532303470 0x55d53231a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5322fd670 0x55d53231cc10 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 --2- 172.21.15.52:0/985503329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5322fe030 0x55d532319fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b637fe700 0 client.18714 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 >> 172.21.15.52:0/985503329 conn(0x55d532226080 msgr2=0x55d532300e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 shutdown_connections 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 1 -- 172.21.15.52:0/985503329 wait complete. 2024-06-02T23:23:33.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.074+0000 7f6b94cbeb80 20 client.18714 trim_cache size 0 max 16384 2024-06-02T23:23:33.081 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:23:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b6c0a2c20 msgr2=0x7f6b6c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b6c0a2c20 0x7f6b6c0a50e0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6b78005ea0 tx=0x7f6b78005e30 comp rx=0 tx=0).stop 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 msgr2=0x7f6b8419e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 0x7f6b8419e770 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400b530 tx=0x7f6b7400b8f0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 shutdown_connections 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6b6c0a2c20 0x7f6b6c0a50e0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b84108530 0x7f6b84198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6b8406ac00 0x7f6b8419e770 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 --2- 172.21.15.52:0/2581398141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6b8406b5c0 0x7f6b8419ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 >> 172.21.15.52:0/2581398141 conn(0x7f6b84077400 msgr2=0x7f6b841000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 shutdown_connections 2024-06-02T23:23:33.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:33.075+0000 7f6b8a43b700 1 -- 172.21.15.52:0/2581398141 wait complete. 2024-06-02T23:23:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:33 smithi174 ceph-mon[35994]: pgmap v781: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 696 KiB/s wr, 85 op/s 2024-06-02T23:23:33.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.89 MB/sec execute 522 sec latency 9208.588 ms 2024-06-02T23:23:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:33 smithi052 ceph-mon[32340]: pgmap v781: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 696 KiB/s wr, 85 op/s 2024-06-02T23:23:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:33 smithi123 ceph-mon[35977]: pgmap v781: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 696 KiB/s wr, 85 op/s 2024-06-02T23:23:33.803 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:23:34.059 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:34.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.89 MB/sec execute 523 sec latency 10208.669 ms 2024-06-02T23:23:35.073 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:23:35.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:35 smithi052 ceph-mon[32340]: pgmap v782: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 586 KiB/s wr, 81 op/s 2024-06-02T23:23:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:35 smithi174 ceph-mon[35994]: pgmap v782: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 586 KiB/s wr, 81 op/s 2024-06-02T23:23:35.646 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.88 MB/sec execute 524 sec latency 11208.725 ms 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 -- 172.21.15.52:0/2364539555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8102740 msgr2=0x7f09c8102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 --2- 172.21.15.52:0/2364539555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8102740 0x7f09c8102b20 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0099c0 tx=0x7f09bc009cd0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 -- 172.21.15.52:0/2364539555 shutdown_connections 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 --2- 172.21.15.52:0/2364539555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8107840 0x7f09c810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 --2- 172.21.15.52:0/2364539555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8102740 0x7f09c8102b20 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 --2- 172.21.15.52:0/2364539555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c81030f0 0x7f09c8107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 -- 172.21.15.52:0/2364539555 >> 172.21.15.52:0/2364539555 conn(0x7f09c80795f0 msgr2=0x7f09c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:35.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 -- 172.21.15.52:0/2364539555 shutdown_connections 2024-06-02T23:23:35.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 -- 172.21.15.52:0/2364539555 wait complete. 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 Processor -- start 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.675+0000 7f09cefc7700 1 -- start start 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cefc7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8102740 0x7f09c8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cefc7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c81030f0 0x7f09c8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cefc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 0x7f09c8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cefc7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112680 con 0x7f09c8107840 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cefc7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112500 con 0x7f09c8102740 2024-06-02T23:23:35.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cefc7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c8112800 con 0x7f09c81030f0 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 0x7f09c8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 0x7f09c8075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58812/0 (socket says 172.21.15.52:58812) 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 -- 172.21.15.52:0/4267086413 learned_addr learned my addr 172.21.15.52:0/4267086413 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09c7fff700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c81030f0 0x7f09c8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09ccd63700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8102740 0x7f09c8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 -- 172.21.15.52:0/4267086413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c81030f0 msgr2=0x7f09c8074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c81030f0 0x7f09c8074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 -- 172.21.15.52:0/4267086413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8102740 msgr2=0x7f09c8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8102740 0x7f09c8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 -- 172.21.15.52:0/4267086413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09c81a70b0 con 0x7f09c8107840 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09ccd63700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8102740 0x7f09c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.676+0000 7f09cd564700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 0x7f09c8075210 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f09c0007d70 tx=0x7f09c000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.677+0000 7f09c5ffb700 1 -- 172.21.15.52:0/4267086413 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c0016020 con 0x7f09c8107840 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.677+0000 7f09c5ffb700 1 -- 172.21.15.52:0/4267086413 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09c000ad40 con 0x7f09c8107840 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.677+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09c81a73a0 con 0x7f09c8107840 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.677+0000 7f09c5ffb700 1 -- 172.21.15.52:0/4267086413 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c00115e0 con 0x7f09c8107840 2024-06-02T23:23:35.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.677+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09c81a78e0 con 0x7f09c8107840 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.679+0000 7f09c5ffb700 1 -- 172.21.15.52:0/4267086413 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f09c0019020 con 0x7f09c8107840 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.680+0000 7f09c5ffb700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09b80a2ca0 0x7f09b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.680+0000 7f09c5ffb700 1 -- 172.21.15.52:0/4267086413 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09c00cebb0 con 0x7f09c8107840 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.680+0000 7f09ccd63700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09b80a2ca0 0x7f09b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.680+0000 7f09ccd63700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09b80a2ca0 0x7f09b80a5160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009fd0 tx=0x7f09bc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 Processor -- start 2024-06-02T23:23:35.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 -- start start 2024-06-02T23:23:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b04a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928afe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928aff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558928a50810 con 0x558928b04640 2024-06-02T23:23:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5589289f8720 con 0x558928afef20 2024-06-02T23:23:35.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09d984ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558928a4c7d0 con 0x558928b04f60 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09ccd63700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b04a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.681+0000 7f09ccd63700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b04a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58814/0 (socket says 172.21.15.52:58814) 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 -- 172.21.15.52:0/2212108696 learned_addr learned my addr 172.21.15.52:0/2212108696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09c7fff700 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928afe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09cd564700 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928aff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 -- 172.21.15.52:0/2212108696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 msgr2=0x558928afe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928afe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 -- 172.21.15.52:0/2212108696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 msgr2=0x558928aff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928aff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 -- 172.21.15.52:0/2212108696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589289eef50 con 0x558928b04640 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09ccd63700 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b04a20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0099c0 tx=0x7f09bc01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e898a003576112 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09b2ffd700 1 -- 172.21.15.52:0/2212108696 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc00f460 con 0x558928b04640 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09b2ffd700 1 -- 172.21.15.52:0/2212108696 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09bc00fab0 con 0x558928b04640 2024-06-02T23:23:35.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.682+0000 7f09b2ffd700 1 -- 172.21.15.52:0/2212108696 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09bc024a60 con 0x558928b04640 2024-06-02T23:23:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 -- 172.21.15.52:0/2212108696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 msgr2=0x558928b04a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b04a20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f09bc0099c0 tx=0x7f09bc01f000 comp rx=0 tx=0).stop 2024-06-02T23:23:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 -- 172.21.15.52:0/2212108696 shutdown_connections 2024-06-02T23:23:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928aff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b04a20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 --2- 172.21.15.52:0/2212108696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928afe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 -- 172.21.15.52:0/2212108696 >> 172.21.15.52:0/2212108696 conn(0x558928a29cc0 msgr2=0x558928af9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:35.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 -- 172.21.15.52:0/2212108696 shutdown_connections 2024-06-02T23:23:35.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 -- 172.21.15.52:0/2212108696 wait complete. 2024-06-02T23:23:35.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.683+0000 7f09d984ab80 1 -- start start 2024-06-02T23:23:35.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 Processor -- start 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b0ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04640 0x558928b07f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 0x558928b084c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558928b0b910 con 0x558928b04f60 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5589289f0bf0 con 0x558928b04640 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558928b0ba90 con 0x558928afef20 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09d984ab80 10 client.0 _ll_register_callbacks cb 0x5589289ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 0x558928b084c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 0x558928b084c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58826/0 (socket says 172.21.15.52:58826) 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 -- 172.21.15.52:0/3564514201 learned_addr learned my addr 172.21.15.52:0/3564514201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:35.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09ccd63700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b0ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09c7fff700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04640 0x558928b07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 -- 172.21.15.52:0/3564514201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 msgr2=0x558928b0ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b0ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 -- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04640 msgr2=0x558928b07f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04640 0x558928b07f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.684+0000 7f09cd564700 1 -- 172.21.15.52:0/3564514201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55892899f8d0 con 0x558928b04f60 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09cd564700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 0x558928b084c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f09c00cc730 tx=0x7f09c0007bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db0878f6a680b46 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09a7fff700 1 -- 172.21.15.52:0/3564514201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09c009b440 con 0x558928b04f60 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09a7fff700 1 -- 172.21.15.52:0/3564514201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09c00b4260 con 0x558928b04f60 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09bc00fab0 con 0x558928b04f60 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09a7fff700 1 -- 172.21.15.52:0/3564514201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f09c009f7f0 con 0x558928b04f60 2024-06-02T23:23:35.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 10 client.18726 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558928b161e0 con 0x558928b04f60 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09a7fff700 1 -- 172.21.15.52:0/3564514201 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09bc00fab0 con 0x558928b04f60 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09a7fff700 1 -- 172.21.15.52:0/3564514201 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f09c00a3c20 con 0x558928b04f60 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 10 client.18726 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 10 client.18726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 20 client.18726 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.685+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b18770 0x558928b1ab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.686+0000 7f09d984ab80 4 client.18726 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.686+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558928a5dd10 con 0x558928b18770 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.686+0000 7f09ccd63700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b18770 0x558928b1ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.686+0000 7f09ccd63700 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b18770 0x558928b1ab60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af2ce4821beb5474 server_cookie=c3de3d39579536e0 in_seq=0 out_seq=0 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.686+0000 7f09a7fff700 10 client.18726 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09a7fff700 1 -- 172.21.15.52:0/3564514201 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558928a5dd10 con 0x558928b18770 2024-06-02T23:23:35.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09a7fff700 10 client.18726 handle_command_reply: tid=0 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 client.18726 shutdown 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 msgr2=0x558928b084c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 0x558928b084c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f09c00cc730 tx=0x7f09c0007bb0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 shutdown_connections 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b18770 0x558928b1ab60 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04f60 0x558928b084c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b0ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3564514201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04640 0x558928b07f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 >> 172.21.15.52:0/3564514201 conn(0x558928a29cc0 msgr2=0x558928afd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09a7fff700 0 client.18726 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 shutdown_connections 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.687+0000 7f09d984ab80 1 -- 172.21.15.52:0/3564514201 wait complete. 2024-06-02T23:23:35.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.688+0000 7f09d984ab80 20 client.18726 trim_cache size 0 max 16384 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 Processor -- start 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 -- start start 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928afe9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928aff300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928b05340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81aed00 con 0x558928b04640 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81b0840 con 0x558928afef20 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09d984ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81af3d0 con 0x558928b04f60 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928aff300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928aff300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58828/0 (socket says 172.21.15.52:58828) 2024-06-02T23:23:35.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09c7fff700 1 -- 172.21.15.52:0/3729968471 learned_addr learned my addr 172.21.15.52:0/3729968471 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09ccd63700 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928afe9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.704+0000 7f09cd564700 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928b05340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09c7fff700 1 -- 172.21.15.52:0/3729968471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 msgr2=0x558928b05340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09c7fff700 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928b05340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09c7fff700 1 -- 172.21.15.52:0/3729968471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 msgr2=0x558928afe9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09c7fff700 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928afe9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09c7fff700 1 -- 172.21.15.52:0/3729968471 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558928b0a870 con 0x558928b04640 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09ccd63700 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928afe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:35.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09c7fff700 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928aff300 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f09b400acf0 tx=0x7f09b400a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a001d3fbb003aa84 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09b27fc700 1 -- 172.21.15.52:0/3729968471 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b401e030 con 0x558928b04640 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09b27fc700 1 -- 172.21.15.52:0/3729968471 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09b4004ce0 con 0x558928b04640 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.705+0000 7f09b27fc700 1 -- 172.21.15.52:0/3729968471 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b40055f0 con 0x558928b04640 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 -- 172.21.15.52:0/3729968471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 msgr2=0x558928aff300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928aff300 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f09b400acf0 tx=0x7f09b400a300 comp rx=0 tx=0).stop 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 -- 172.21.15.52:0/3729968471 shutdown_connections 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928b04f60 0x558928b05340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928afef20 0x558928afe9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 --2- 172.21.15.52:0/3729968471 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928aff300 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 -- 172.21.15.52:0/3729968471 >> 172.21.15.52:0/3729968471 conn(0x558928a29cc0 msgr2=0x558928b06870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 -- 172.21.15.52:0/3729968471 shutdown_connections 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 -- 172.21.15.52:0/3729968471 wait complete. 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 -- start start 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.706+0000 7f09d984ab80 1 Processor -- start 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b11460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b1fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04f60 0x558928b20120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81b0170 con 0x558928b04640 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81afaa0 con 0x558928b04f60 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09c81acca0 con 0x558928afef20 2024-06-02T23:23:35.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09d984ab80 10 client.0 _ll_register_callbacks cb 0x5589289ea920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b1fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b1fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58834/0 (socket says 172.21.15.52:58834) 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 -- 172.21.15.52:0/86875351 learned_addr learned my addr 172.21.15.52:0/86875351 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09cd564700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04f60 0x558928b20120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09ccd63700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b11460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 -- 172.21.15.52:0/86875351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 msgr2=0x558928b11460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b11460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 -- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04f60 msgr2=0x558928b20120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04f60 0x558928b20120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.707+0000 7f09c7fff700 1 -- 172.21.15.52:0/86875351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558928b207f0 con 0x558928b04640 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09c7fff700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b1fbe0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005190 tx=0x7f09b4004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cfb2a8c67619816 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09a7fff700 1 -- 172.21.15.52:0/86875351 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09b4005600 con 0x558928b04640 2024-06-02T23:23:35.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09a7fff700 1 -- 172.21.15.52:0/86875351 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09b4004020 con 0x558928b04640 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09b4004ce0 con 0x558928b04640 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09a7fff700 1 -- 172.21.15.52:0/86875351 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f09b4034070 con 0x558928b04640 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09d984ab80 10 client.18732 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558928b83230 con 0x558928b04640 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09a7fff700 1 -- 172.21.15.52:0/86875351 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09b4004ce0 con 0x558928b04640 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09a7fff700 1 -- 172.21.15.52:0/86875351 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f09b402c030 con 0x558928b04640 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09d984ab80 10 client.18732 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09d984ab80 10 client.18732 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.708+0000 7f09d984ab80 20 client.18732 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09d984ab80 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b83860 0x558928b85c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09d984ab80 4 client.18732 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558928a58290 con 0x558928b83860 2024-06-02T23:23:35.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09ccd63700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b83860 0x558928b85c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:35.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09ccd63700 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b83860 0x558928b85c80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6dcf2878bf9ba5c server_cookie=f3d94fb0309c26c9 in_seq=0 out_seq=0 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09a7fff700 10 client.18732 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09a7fff700 1 -- 172.21.15.52:0/86875351 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558928a58290 con 0x558928b83860 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09a7fff700 10 client.18732 handle_command_reply: tid=0 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09d984ab80 1 client.18732 shutdown 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 msgr2=0x558928b1fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.709+0000 7f09d984ab80 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b1fbe0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f09b4005190 tx=0x7f09b4004a00 comp rx=0 tx=0).stop 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 shutdown_connections 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x558928b83860 0x558928b85c80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558928b04f60 0x558928b20120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558928afef20 0x558928b11460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 --2- 172.21.15.52:0/86875351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558928b04640 0x558928b1fbe0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09a7fff700 0 client.18732 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:23:35.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 >> 172.21.15.52:0/86875351 conn(0x558928a29cc0 msgr2=0x558928b06830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:35.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 shutdown_connections 2024-06-02T23:23:35.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 1 -- 172.21.15.52:0/86875351 wait complete. 2024-06-02T23:23:35.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09d984ab80 20 client.18732 trim_cache size 0 max 16384 2024-06-02T23:23:35.715 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09b80a2ca0 msgr2=0x7f09b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09cefc7700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09b80a2ca0 0x7f09b80a5160 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009fd0 tx=0x7f09bc005fb0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 msgr2=0x7f09c8075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.710+0000 7f09cefc7700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 0x7f09c8075210 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f09c0007d70 tx=0x7f09c000a340 comp rx=0 tx=0).stop 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 shutdown_connections 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09b80a2ca0 0x7f09b80a5160 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f09c8107840 0x7f09c8075210 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f09c8102740 0x7f09c8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 --2- 172.21.15.52:0/4267086413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09c81030f0 0x7f09c8074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:35.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 >> 172.21.15.52:0/4267086413 conn(0x7f09c80795f0 msgr2=0x7f09c8108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:35.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 shutdown_connections 2024-06-02T23:23:35.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:35.711+0000 7f09cefc7700 1 -- 172.21.15.52:0/4267086413 wait complete. 2024-06-02T23:23:35.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:35 smithi123 ceph-mon[35977]: pgmap v782: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 586 KiB/s wr, 81 op/s 2024-06-02T23:23:35.945 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:36.442 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:23:36.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.88 MB/sec execute 525 sec latency 12208.777 ms 2024-06-02T23:23:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 -- 172.21.15.52:0/2934861391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 msgr2=0x7f9d54100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 --2- 172.21.15.52:0/2934861391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54100d20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009900 tx=0x7f9d5002a410 comp rx=0 tx=0).stop 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 -- 172.21.15.52:0/2934861391 shutdown_connections 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 --2- 172.21.15.52:0/2934861391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d541024b0 0x7f9d54108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 --2- 172.21.15.52:0/2934861391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101af0 0x7f9d54101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 --2- 172.21.15.52:0/2934861391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54100d20 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 -- 172.21.15.52:0/2934861391 >> 172.21.15.52:0/2934861391 conn(0x7f9d540fbe70 msgr2=0x7f9d540fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 -- 172.21.15.52:0/2934861391 shutdown_connections 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.096+0000 7f9d5b838640 1 -- 172.21.15.52:0/2934861391 wait complete. 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 Processor -- start 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 -- start start 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101af0 0x7f9d541980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d541024b0 0x7f9d5419f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410c430 con 0x7f9d54100900 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410c2b0 con 0x7f9d54101af0 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d5b838640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d5410c5b0 con 0x7f9d541024b0 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d595d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d595d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58840/0 (socket says 172.21.15.52:58840) 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d595d4640 1 -- 172.21.15.52:0/1835310908 learned_addr learned my addr 172.21.15.52:0/1835310908 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101af0 0x7f9d541980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.097+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d541024b0 0x7f9d5419f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d595d4640 1 -- 172.21.15.52:0/1835310908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d541024b0 msgr2=0x7f9d5419f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d541024b0 0x7f9d5419f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d595d4640 1 -- 172.21.15.52:0/1835310908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101af0 msgr2=0x7f9d541980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101af0 0x7f9d541980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d595d4640 1 -- 172.21.15.52:0/1835310908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d5419f780 con 0x7f9d54100900 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54197b60 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009230 tx=0x7f9d50004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:37.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d4a7fc640 1 -- 172.21.15.52:0/1835310908 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5003f030 con 0x7f9d54100900 2024-06-02T23:23:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d4a7fc640 1 -- 172.21.15.52:0/1835310908 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d50003cb0 con 0x7f9d54100900 2024-06-02T23:23:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d5419fa10 con 0x7f9d54100900 2024-06-02T23:23:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d4a7fc640 1 -- 172.21.15.52:0/1835310908 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50035070 con 0x7f9d54100900 2024-06-02T23:23:37.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.098+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d5419ff00 con 0x7f9d54100900 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.100+0000 7f9d4a7fc640 1 -- 172.21.15.52:0/1835310908 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9d50042030 con 0x7f9d54100900 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 Processor -- start 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 -- start start 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7891ff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789204910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789207410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c78915e4f0 con 0x55c7891ffe10 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7890fc870 con 0x55c789204f80 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d5e622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c78915a4b0 con 0x55c7891ff420 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d4a7fc640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d400a2c30 0x7f9d400a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d595d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789204910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d595d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789204910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58842/0 (socket says 172.21.15.52:58842) 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d595d4640 1 -- 172.21.15.52:0/1359529049 learned_addr learned my addr 172.21.15.52:0/1359529049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7891ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789207410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d400a2c30 0x7f9d400a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.101+0000 7f9d4a7fc640 1 -- 172.21.15.52:0/1835310908 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d500ef610 con 0x7f9d54100900 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 -- 172.21.15.52:0/1359529049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 msgr2=0x55c789207410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789207410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 -- 172.21.15.52:0/1359529049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 msgr2=0x55c789204910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789204910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 -- 172.21.15.52:0/1359529049 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7891189b0 con 0x55c7891ff420 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789204910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7891ff840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c014e30 tx=0x7f9d4c0117b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c0dedb31ce28ac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d400a2c30 0x7f9d400a50f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c011cb0 tx=0x7f9d4c016000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d3f7fe640 1 -- 172.21.15.52:0/1359529049 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c022070 con 0x55c7891ff420 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d3f7fe640 1 -- 172.21.15.52:0/1359529049 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c003e20 con 0x55c7891ff420 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.102+0000 7f9d3f7fe640 1 -- 172.21.15.52:0/1359529049 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4c004280 con 0x55c7891ff420 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 -- 172.21.15.52:0/1359529049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 msgr2=0x55c7891ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7891ff840 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c014e30 tx=0x7f9d4c0117b0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 -- 172.21.15.52:0/1359529049 shutdown_connections 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789207410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789204910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1359529049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7891ff840 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 -- 172.21.15.52:0/1359529049 >> 172.21.15.52:0/1359529049 conn(0x55c7891f6680 msgr2=0x55c7891f6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 -- 172.21.15.52:0/1359529049 shutdown_connections 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 -- 172.21.15.52:0/1359529049 wait complete. 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 -- start start 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.103+0000 7f9d5e622740 1 Processor -- start 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789210c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c789207ed0 con 0x55c7891ffe10 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c7890e7ae0 con 0x55c789204f80 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c789208050 con 0x55c7891ff420 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d5e622740 10 client.0 _ll_register_callbacks cb 0x55c7891209b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58858/0 (socket says 172.21.15.52:58858) 2024-06-02T23:23:37.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 -- 172.21.15.52:0/1174885934 learned_addr learned my addr 172.21.15.52:0/1174885934 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789210c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 -- 172.21.15.52:0/1174885934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 msgr2=0x55c7892096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 -- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 msgr2=0x55c789210c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789210c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 -- 172.21.15.52:0/1174885934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c788e72210 con 0x55c7891ffe10 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789210c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789209c00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9d44010820 tx=0x7f9d44010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfb9043a58c4921 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.104+0000 7f9d3cff9640 1 -- 172.21.15.52:0/1174885934 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4400fb30 con 0x55c7891ffe10 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d3cff9640 1 -- 172.21.15.52:0/1174885934 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d4400fcd0 con 0x55c7891ffe10 2024-06-02T23:23:37.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d4c003e20 con 0x55c7891ffe10 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d3cff9640 1 -- 172.21.15.52:0/1174885934 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d44005020 con 0x55c7891ffe10 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 10 client.18744 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c788f3e620 con 0x55c7891ffe10 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d3cff9640 1 -- 172.21.15.52:0/1174885934 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d4c003e20 con 0x55c7891ffe10 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d3cff9640 1 -- 172.21.15.52:0/1174885934 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d44025480 con 0x55c7891ffe10 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 10 client.18744 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 10 client.18744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 20 client.18744 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892155f0 0x55c789217a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 4 client.18744 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c789171f10 con 0x55c7892155f0 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.105+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892155f0 0x55c789217a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.106+0000 7f9d595d4640 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892155f0 0x55c789217a70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bbc795c5696be86 server_cookie=8d40cf6cfee318f1 in_seq=0 out_seq=0 2024-06-02T23:23:37.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.106+0000 7f9d3cff9640 10 client.18744 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:37.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d3cff9640 1 -- 172.21.15.52:0/1174885934 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c789171f10 con 0x55c7892155f0 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d3cff9640 10 client.18744 handle_command_reply: tid=0 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 client.18744 shutdown 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 msgr2=0x55c789209c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789209c00 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f9d44010820 tx=0x7f9d44010c20 comp rx=0 tx=0).stop 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 shutdown_connections 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892155f0 0x55c789217a70 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c789204f80 0x55c789210c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d3cff9640 0 client.18744 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 --2- 172.21.15.52:0/1174885934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 >> 172.21.15.52:0/1174885934 conn(0x55c7891f6680 msgr2=0x55c7891f57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 shutdown_connections 2024-06-02T23:23:37.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 1 -- 172.21.15.52:0/1174885934 wait complete. 2024-06-02T23:23:37.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.107+0000 7f9d5e622740 20 client.18744 trim_cache size 0 max 16384 2024-06-02T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 Processor -- start 2024-06-02T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 -- start start 2024-06-02T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 0x55c789210c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541a6ab0 con 0x55c789204f80 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541a85f0 con 0x55c7891ffe10 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d5e622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541a7180 con 0x55c7891ff420 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d59dd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 0x55c789210c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d59dd5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 0x55c789210c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58874/0 (socket says 172.21.15.52:58874) 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d59dd5640 1 -- 172.21.15.52:0/362603933 learned_addr learned my addr 172.21.15.52:0/362603933 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d58dd3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d595d4640 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d58dd3640 1 -- 172.21.15.52:0/362603933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 msgr2=0x55c7892096c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d58dd3640 1 -- 172.21.15.52:0/362603933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 msgr2=0x55c789210c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 0x55c789210c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d58dd3640 1 -- 172.21.15.52:0/362603933 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7891f64b0 con 0x55c7891ffe10 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.123+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 0x55c789210c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.124+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c789209c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9d440107f0 tx=0x7f9d4400a870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8a186bcc6a71855 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.124+0000 7f9d3effd640 1 -- 172.21.15.52:0/362603933 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4402b070 con 0x55c7891ffe10 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.124+0000 7f9d3effd640 1 -- 172.21.15.52:0/362603933 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d44025b30 con 0x55c7891ffe10 2024-06-02T23:23:37.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.124+0000 7f9d3effd640 1 -- 172.21.15.52:0/362603933 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d44005400 con 0x55c7891ffe10 2024-06-02T23:23:37.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- 172.21.15.52:0/362603933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 msgr2=0x55c789209c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c789209c00 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f9d440107f0 tx=0x7f9d4400a870 comp rx=0 tx=0).stop 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- 172.21.15.52:0/362603933 shutdown_connections 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c789204f80 0x55c789210c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c789209c00 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- 172.21.15.52:0/362603933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c7891ff420 0x55c7892096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- 172.21.15.52:0/362603933 >> 172.21.15.52:0/362603933 conn(0x55c7891f6680 msgr2=0x55c7891f1560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- 172.21.15.52:0/362603933 shutdown_connections 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- 172.21.15.52:0/362603933 wait complete. 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- start start 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 Processor -- start 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 0x55c78920db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c78920e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c789204f80 0x55c78927e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541a7f20 con 0x55c7891ff420 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541a7850 con 0x55c7891ffe10 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.125+0000 7f9d5e622740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d541a4a90 con 0x55c789204f80 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d5e622740 10 client.0 _ll_register_callbacks cb 0x7f9d4400a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 0x55c78920db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 0x55c78920db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58884/0 (socket says 172.21.15.52:58884) 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 -- 172.21.15.52:0/2824911840 learned_addr learned my addr 172.21.15.52:0/2824911840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c78920e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d59dd5640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c789204f80 0x55c78927e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 -- 172.21.15.52:0/2824911840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c789204f80 msgr2=0x55c78927e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c789204f80 0x55c78927e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 -- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 msgr2=0x55c78920e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c78920e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 -- 172.21.15.52:0/2824911840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c78927e880 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d595d4640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 0x55c78920db60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9d500ed190 tx=0x7f9d50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c2228c0dcbf663 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d3cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d3cff9640 1 -- 172.21.15.52:0/2824911840 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d50049040 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d3cff9640 1 -- 172.21.15.52:0/2824911840 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9d500ba1a0 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d44025b30 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d3cff9640 1 -- 172.21.15.52:0/2824911840 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f9d500c4b10 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d5e622740 10 client.18750 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.126+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c78927fbb0 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d3cff9640 1 -- 172.21.15.52:0/2824911840 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9d44025b30 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d3cff9640 1 -- 172.21.15.52:0/2824911840 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f9d5003bce0 con 0x55c7891ff420 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d5e622740 10 client.18750 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d5e622740 10 client.18750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:37.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d5e622740 20 client.18750 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d5e622740 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892804e0 0x55c789282900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d5e622740 4 client.18750 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:23:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c78916bc20 con 0x55c7892804e0 2024-06-02T23:23:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.127+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892804e0 0x55c789282900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.128+0000 7f9d58dd3640 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892804e0 0x55c789282900 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689aaee864393c40 server_cookie=eb7639518f8509b8 in_seq=0 out_seq=0 2024-06-02T23:23:37.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.128+0000 7f9d3cff9640 10 client.18750 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:37.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d3cff9640 1 -- 172.21.15.52:0/2824911840 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c78916bc20 con 0x55c7892804e0 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d3cff9640 10 client.18750 handle_command_reply: tid=0 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 client.18750 shutdown 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 msgr2=0x55c78920db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 0x55c78920db60 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f9d500ed190 tx=0x7f9d50009260 comp rx=0 tx=0).stop 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 shutdown_connections 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c7892804e0 0x55c789282900 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c789204f80 0x55c78927e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c7891ffe10 0x55c78920e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d3cff9640 0 client.18750 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 --2- 172.21.15.52:0/2824911840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c7891ff420 0x55c78920db60 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 >> 172.21.15.52:0/2824911840 conn(0x55c7891f6680 msgr2=0x55c7891f9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 shutdown_connections 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 1 -- 172.21.15.52:0/2824911840 wait complete. 2024-06-02T23:23:37.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.152+0000 7f9d5e622740 20 client.18750 trim_cache size 0 max 16384 2024-06-02T23:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:23:37.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d400a2c30 msgr2=0x7f9d400a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d400a2c30 0x7f9d400a50f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c011cb0 tx=0x7f9d4c016000 comp rx=0 tx=0).stop 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 msgr2=0x7f9d54197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54197b60 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f9d50009230 tx=0x7f9d50004340 comp rx=0 tx=0).stop 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 shutdown_connections 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9d400a2c30 0x7f9d400a50f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d541024b0 0x7f9d5419f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9d54101af0 0x7f9d541980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 --2- 172.21.15.52:0/1835310908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9d54100900 0x7f9d54197b60 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 >> 172.21.15.52:0/1835310908 conn(0x7f9d540fbe70 msgr2=0x7f9d540fd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 shutdown_connections 2024-06-02T23:23:37.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:37.153+0000 7f9d5b838640 1 -- 172.21.15.52:0/1835310908 wait complete. 2024-06-02T23:23:37.228 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:37.639 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:37 smithi052 ceph-mon[32340]: pgmap v783: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 462 KiB/s wr, 64 op/s 2024-06-02T23:23:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:37 smithi174 ceph-mon[35994]: pgmap v783: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 462 KiB/s wr, 64 op/s 2024-06-02T23:23:37.645 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.88 MB/sec execute 526 sec latency 13208.824 ms 2024-06-02T23:23:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:37 smithi123 ceph-mon[35977]: pgmap v783: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 462 KiB/s wr, 64 op/s 2024-06-02T23:23:38.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.87 MB/sec execute 527 sec latency 14215.300 ms 2024-06-02T23:23:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 -- 172.21.15.52:0/719806111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006a170 msgr2=0x7f37f006a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 --2- 172.21.15.52:0/719806111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006a170 0x7f37f006a5f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f37d8009990 tx=0x7f37d8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:23:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 -- 172.21.15.52:0/719806111 shutdown_connections 2024-06-02T23:23:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 --2- 172.21.15.52:0/719806111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f006ab30 0x7f37f010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 --2- 172.21.15.52:0/719806111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f0108500 0x7f37f01088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 --2- 172.21.15.52:0/719806111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006a170 0x7f37f006a5f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 -- 172.21.15.52:0/719806111 >> 172.21.15.52:0/719806111 conn(0x7f37f0078340 msgr2=0x7f37f0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 -- 172.21.15.52:0/719806111 shutdown_connections 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.828+0000 7f37f53ee700 1 -- 172.21.15.52:0/719806111 wait complete. 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 Processor -- start 2024-06-02T23:23:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 -- start start 2024-06-02T23:23:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f006a170 0x7f37f019e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006ab30 0x7f37f019ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f0108500 0x7f37f01989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0112690 con 0x7f37f006ab30 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0112510 con 0x7f37f0108500 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.829+0000 7f37f53ee700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37f0112810 con 0x7f37f006a170 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37ef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f0108500 0x7f37f01989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006ab30 0x7f37f019ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37ef7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f0108500 0x7f37f01989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41842/0 (socket says 172.21.15.52:41842) 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37ef7fe700 1 -- 172.21.15.52:0/3954879636 learned_addr learned my addr 172.21.15.52:0/3954879636 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37eeffd700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f006a170 0x7f37f019e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 -- 172.21.15.52:0/3954879636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f006a170 msgr2=0x7f37f019e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f006a170 0x7f37f019e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 -- 172.21.15.52:0/3954879636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f0108500 msgr2=0x7f37f01989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f0108500 0x7f37f01989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 -- 172.21.15.52:0/3954879636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f0199150 con 0x7f37f006ab30 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37eeffd700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f006a170 0x7f37f019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37e67fc700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006ab30 0x7f37f019ed70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f37d80059d0 tx=0x7f37d80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8027070 con 0x7f37f006ab30 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37d8003f90 con 0x7f37f006ab30 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.830+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f01993e0 con 0x7f37f006ab30 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.831+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37d8022d00 con 0x7f37f006ab30 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.831+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f01a5700 con 0x7f37f006ab30 2024-06-02T23:23:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.831+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37f010a9e0 con 0x7f37f006ab30 2024-06-02T23:23:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.833+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37d802a070 con 0x7f37f006ab30 2024-06-02T23:23:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.834+0000 7f37ecff9700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37dc0a2ca0 0x7f37dc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.834+0000 7f37eeffd700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37dc0a2ca0 0x7f37dc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:38.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.834+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f37d80d0670 con 0x7f37f006ab30 2024-06-02T23:23:38.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.835+0000 7f37eeffd700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37dc0a2ca0 0x7f37dc0a5160 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f37f0199b30 tx=0x7f37e0009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:38.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:38.838+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37d8031070 con 0x7f37f006ab30 2024-06-02T23:23:39.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.090+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37f0103bc0 con 0x7f37f006ab30 2024-06-02T23:23:39.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.091+0000 7f37ecff9700 1 -- 172.21.15.52:0/3954879636 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f37d8013070 con 0x7f37f006ab30 2024-06-02T23:23:39.091 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:23:39.091 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37dc0a2ca0 msgr2=0x7f37dc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37dc0a2ca0 0x7f37dc0a5160 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f37f0199b30 tx=0x7f37e0009190 comp rx=0 tx=0).stop 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006ab30 msgr2=0x7f37f019ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006ab30 0x7f37f019ed70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f37d80059d0 tx=0x7f37d80047b0 comp rx=0 tx=0).stop 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 shutdown_connections 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37dc0a2ca0 0x7f37dc0a5160 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37f0108500 0x7f37f01989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37f006a170 0x7f37f019e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 --2- 172.21.15.52:0/3954879636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37f006ab30 0x7f37f019ed70 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 >> 172.21.15.52:0/3954879636 conn(0x7f37f0078340 msgr2=0x7f37f0100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:39.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 shutdown_connections 2024-06-02T23:23:39.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:39.093+0000 7f37f53ee700 1 -- 172.21.15.52:0/3954879636 wait complete. 2024-06-02T23:23:39.095 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:23:39.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:39 smithi052 ceph-mon[32340]: pgmap v784: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 70 KiB/s wr, 26 op/s 2024-06-02T23:23:39.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:39 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3954879636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:39 smithi174 ceph-mon[35994]: pgmap v784: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 70 KiB/s wr, 26 op/s 2024-06-02T23:23:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:39 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3954879636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:39.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.87 MB/sec execute 528 sec latency 15215.328 ms 2024-06-02T23:23:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:39 smithi123 ceph-mon[35977]: pgmap v784: 97 pgs: 97 active+clean; 98 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 70 KiB/s wr, 26 op/s 2024-06-02T23:23:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:39 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3954879636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:39.970 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 9d5fe456 --release --await' 2024-06-02T23:23:39.970 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 9d5fe456 --release --await 2024-06-02T23:23:40.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 -- 172.21.15.52:0/3096550897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7006b0a0 msgr2=0x7f3a7006b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 --2- 172.21.15.52:0/3096550897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7006b0a0 0x7f3a7006b500 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f3a64009990 tx=0x7f3a64009ca0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 -- 172.21.15.52:0/3096550897 shutdown_connections 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 --2- 172.21.15.52:0/3096550897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a7010ede0 0x7f3a701111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 --2- 172.21.15.52:0/3096550897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a701033a0 0x7f3a7006ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 --2- 172.21.15.52:0/3096550897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7006b0a0 0x7f3a7006b500 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 -- 172.21.15.52:0/3096550897 >> 172.21.15.52:0/3096550897 conn(0x7f3a70078340 msgr2=0x7f3a70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 -- 172.21.15.52:0/3096550897 shutdown_connections 2024-06-02T23:23:40.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.442+0000 7f3a7782a700 1 -- 172.21.15.52:0/3096550897 wait complete. 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 Processor -- start 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 -- start start 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7006b0a0 0x7f3a7019ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a701033a0 0x7f3a7019b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 0x7f3a7019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a70114850 con 0x7f3a7010ede0 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701146d0 con 0x7f3a701033a0 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.443+0000 7f3a7782a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701149d0 con 0x7f3a7006b0a0 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 0x7f3a7019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 0x7f3a7019f630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58926/0 (socket says 172.21.15.52:58926) 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 -- 172.21.15.52:0/2000954180 learned_addr learned my addr 172.21.15.52:0/2000954180 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a755c6700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7006b0a0 0x7f3a7019ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a701033a0 0x7f3a7019b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 -- 172.21.15.52:0/2000954180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7006b0a0 msgr2=0x7f3a7019ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7006b0a0 0x7f3a7019ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 -- 172.21.15.52:0/2000954180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a701033a0 msgr2=0x7f3a7019b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a701033a0 0x7f3a7019b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 -- 172.21.15.52:0/2000954180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a7019fd70 con 0x7f3a7010ede0 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a755c6700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7006b0a0 0x7f3a7019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 0x7f3a7019f630 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3a680094f0 tx=0x7f3a6800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a627fc700 1 -- 172.21.15.52:0/2000954180 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800ddd0 con 0x7f3a7010ede0 2024-06-02T23:23:40.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a627fc700 1 -- 172.21.15.52:0/2000954180 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a68009e20 con 0x7f3a7010ede0 2024-06-02T23:23:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a701a0060 con 0x7f3a7010ede0 2024-06-02T23:23:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a627fc700 1 -- 172.21.15.52:0/2000954180 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68015400 con 0x7f3a7010ede0 2024-06-02T23:23:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.444+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a701a78e0 con 0x7f3a7010ede0 2024-06-02T23:23:40.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.447+0000 7f3a627fc700 1 -- 172.21.15.52:0/2000954180 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3a680155a0 con 0x7f3a7010ede0 2024-06-02T23:23:40.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.447+0000 7f3a627fc700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a5c0a2ca0 0x7f3a5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.447+0000 7f3a627fc700 1 -- 172.21.15.52:0/2000954180 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3a680ce9c0 con 0x7f3a7010ede0 2024-06-02T23:23:40.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.447+0000 7f3a755c6700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a5c0a2ca0 0x7f3a5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a755c6700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a5c0a2ca0 0x7f3a5c0a5160 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00bc10 tx=0x7f3a6c00b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 Processor -- start 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 -- start start 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c54a947f60 con 0x55c54aa557a0 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c54a9a6850 con 0x55c54aa5af60 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a820adb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c54a90e460 con 0x55c54aa5b880 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a75dc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a75dc7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa55bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58932/0 (socket says 172.21.15.52:58932) 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.448+0000 7f3a75dc7700 1 -- 172.21.15.52:0/3879775352 learned_addr learned my addr 172.21.15.52:0/3879775352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a755c6700 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a75dc7700 1 -- 172.21.15.52:0/3879775352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 msgr2=0x55c54aa55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa55260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a75dc7700 1 -- 172.21.15.52:0/3879775352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 msgr2=0x55c54aa5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a75dc7700 1 -- 172.21.15.52:0/3879775352 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54a8f58d0 con 0x55c54aa557a0 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa55bc0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3a680a3990 tx=0x7f3a68004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b242eeb74783ab96 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a5b7fe700 1 -- 172.21.15.52:0/3879775352 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a680bb0f0 con 0x55c54aa557a0 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a5b7fe700 1 -- 172.21.15.52:0/3879775352 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a680bb290 con 0x55c54aa557a0 2024-06-02T23:23:40.451 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.449+0000 7f3a5b7fe700 1 -- 172.21.15.52:0/3879775352 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68097160 con 0x55c54aa557a0 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 -- 172.21.15.52:0/3879775352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 msgr2=0x55c54aa55bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa55bc0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3a680a3990 tx=0x7f3a68004e40 comp rx=0 tx=0).stop 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 -- 172.21.15.52:0/3879775352 shutdown_connections 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa55bc0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 --2- 172.21.15.52:0/3879775352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 -- 172.21.15.52:0/3879775352 >> 172.21.15.52:0/3879775352 conn(0x55c54a982080 msgr2=0x55c54aa50220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 -- 172.21.15.52:0/3879775352 shutdown_connections 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 -- 172.21.15.52:0/3879775352 wait complete. 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.450+0000 7f3a820adb80 1 -- start start 2024-06-02T23:23:40.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 Processor -- start 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa615a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa5ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c54a946fc0 con 0x55c54aa557a0 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c54a9a2810 con 0x55c54aa5af60 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c54aa62260 con 0x55c54aa5b880 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a820adb80 10 client.0 _ll_register_callbacks cb 0x55c54a93c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a755c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa615a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a755c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa615a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58940/0 (socket says 172.21.15.52:58940) 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a755c6700 1 -- 172.21.15.52:0/1723620406 learned_addr learned my addr 172.21.15.52:0/1723620406 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa5ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.451+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a755c6700 1 -- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 msgr2=0x55c54aa5ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa5ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a755c6700 1 -- 172.21.15.52:0/1723620406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 msgr2=0x55c54aa5e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a755c6700 1 -- 172.21.15.52:0/1723620406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54a96c750 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa615a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011440 tx=0x7f3a6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3964059bc5c93143 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1723620406 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c028470 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1723620406 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3a6c01f430 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a680bb290 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1723620406 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a6c01f670 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a820adb80 10 client.18768 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c54aa6cac0 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.452+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1723620406 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a680bb290 con 0x55c54aa557a0 2024-06-02T23:23:40.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1723620406 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3a6c028b80 con 0x55c54aa557a0 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a820adb80 10 client.18768 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a820adb80 10 client.18768 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a820adb80 20 client.18768 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aa6f100 0x55c54aa714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a820adb80 4 client.18768 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c54a9b3d50 con 0x55c54aa6f100 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aa6f100 0x55c54aa714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a58ff9700 10 client.18768 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.453+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aa6f100 0x55c54aa714f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8ad6419e45852ba0 server_cookie=71ae707a1ffcae1e in_seq=0 out_seq=0 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1723620406 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c54a9b3d50 con 0x55c54aa6f100 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a58ff9700 10 client.18768 handle_command_reply: tid=0 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 client.18768 shutdown 2024-06-02T23:23:40.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 msgr2=0x55c54aa615a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa615a0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011440 tx=0x7f3a6c004750 comp rx=0 tx=0).stop 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 shutdown_connections 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aa6f100 0x55c54aa714f0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5b880 0x55c54aa5ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa615a0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1723620406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5af60 0x55c54aa5e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 >> 172.21.15.52:0/1723620406 conn(0x55c54a982080 msgr2=0x55c54aa4f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.454+0000 7f3a58ff9700 0 client.18768 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.455+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 shutdown_connections 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.455+0000 7f3a820adb80 1 -- 172.21.15.52:0/1723620406 wait complete. 2024-06-02T23:23:40.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.455+0000 7f3a820adb80 20 client.18768 trim_cache size 0 max 16384 2024-06-02T23:23:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 Processor -- start 2024-06-02T23:23:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 -- start start 2024-06-02T23:23:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa6b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa4fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701aec70 con 0x55c54aa557a0 2024-06-02T23:23:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701b07b0 con 0x55c54aa5b880 2024-06-02T23:23:40.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a820adb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701af340 con 0x55c54aa5af60 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a755c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa6b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a755c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa6b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58944/0 (socket says 172.21.15.52:58944) 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a755c6700 1 -- 172.21.15.52:0/1516882248 learned_addr learned my addr 172.21.15.52:0/1516882248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa4fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.471+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a755c6700 1 -- 172.21.15.52:0/1516882248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 msgr2=0x55c54aa55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa55260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a755c6700 1 -- 172.21.15.52:0/1516882248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 msgr2=0x55c54aa4fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa4fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a755c6700 1 -- 172.21.15.52:0/1516882248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54aa5b680 con 0x55c54aa557a0 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa6b380 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0058b0 tx=0x7f3a6c005f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c64ab8138df43e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a5affd700 1 -- 172.21.15.52:0/1516882248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c004020 con 0x55c54aa557a0 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a5affd700 1 -- 172.21.15.52:0/1516882248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a6c023070 con 0x55c54aa557a0 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.472+0000 7f3a5affd700 1 -- 172.21.15.52:0/1516882248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c0052a0 con 0x55c54aa557a0 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 -- 172.21.15.52:0/1516882248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 msgr2=0x55c54aa6b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa6b380 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c0058b0 tx=0x7f3a6c005f60 comp rx=0 tx=0).stop 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 -- 172.21.15.52:0/1516882248 shutdown_connections 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa4fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa6b380 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1516882248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 -- 172.21.15.52:0/1516882248 >> 172.21.15.52:0/1516882248 conn(0x55c54a982080 msgr2=0x55c54aa5d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 -- 172.21.15.52:0/1516882248 shutdown_connections 2024-06-02T23:23:40.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 -- 172.21.15.52:0/1516882248 wait complete. 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.473+0000 7f3a820adb80 1 -- start start 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 Processor -- start 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa674f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa67a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa75db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701b00e0 con 0x55c54aa557a0 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701afa10 con 0x55c54aa5b880 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a701acc30 con 0x55c54aa5af60 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a820adb80 10 client.0 _ll_register_callbacks cb 0x55c54a93c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a755c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa674f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a755c6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa674f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58952/0 (socket says 172.21.15.52:58952) 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a755c6700 1 -- 172.21.15.52:0/1180938908 learned_addr learned my addr 172.21.15.52:0/1180938908 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:40.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa67a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.474+0000 7f3a75dc7700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa75db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a755c6700 1 -- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 msgr2=0x55c54aa67a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa67a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a755c6700 1 -- 172.21.15.52:0/1180938908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 msgr2=0x55c54aa75db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa75db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a755c6700 1 -- 172.21.15.52:0/1180938908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54aa680a0 con 0x55c54aa557a0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a755c6700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa674f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c01fe90 tx=0x7f3a6c01fcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c11a77025cb63a7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a58ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1180938908 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c040020 con 0x55c54aa557a0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1180938908 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3a6c004430 con 0x55c54aa557a0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a6c023070 con 0x55c54aa557a0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1180938908 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3a6c005040 con 0x55c54aa557a0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a820adb80 10 client.18774 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.475+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c54aad92e0 con 0x55c54aa557a0 2024-06-02T23:23:40.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1180938908 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3a6c023070 con 0x55c54aa557a0 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1180938908 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3a6c03a030 con 0x55c54aa557a0 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a820adb80 10 client.18774 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a820adb80 10 client.18774 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a820adb80 20 client.18774 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aad9900 0x55c54aadbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a820adb80 4 client.18774 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:23:40.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c54a9ae2d0 con 0x55c54aad9900 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.476+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aad9900 0x55c54aadbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a58ff9700 10 client.18774 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a74dc5700 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aad9900 0x55c54aadbd20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=97bc0ddd17768011 server_cookie=ff5674c68cc36e93 in_seq=0 out_seq=0 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a58ff9700 1 -- 172.21.15.52:0/1180938908 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c54a9ae2d0 con 0x55c54aad9900 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a58ff9700 10 client.18774 handle_command_reply: tid=0 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 client.18774 shutdown 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 msgr2=0x55c54aa674f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa674f0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c01fe90 tx=0x7f3a6c01fcd0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 shutdown_connections 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55c54aad9900 0x55c54aadbd20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54aa5b880 0x55c54aa75db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54aa557a0 0x55c54aa674f0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 --2- 172.21.15.52:0/1180938908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54aa5af60 0x55c54aa67a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 >> 172.21.15.52:0/1180938908 conn(0x55c54a982080 msgr2=0x55c54aa5bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:40.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a58ff9700 0 client.18774 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:23:40.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 shutdown_connections 2024-06-02T23:23:40.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 1 -- 172.21.15.52:0/1180938908 wait complete. 2024-06-02T23:23:40.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.477+0000 7f3a820adb80 20 client.18774 trim_cache size 0 max 16384 2024-06-02T23:23:40.483 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a5c0a2ca0 msgr2=0x7f3a5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a5c0a2ca0 0x7f3a5c0a5160 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c00bc10 tx=0x7f3a6c00b5f0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 msgr2=0x7f3a7019f630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 0x7f3a7019f630 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f3a680094f0 tx=0x7f3a6800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 shutdown_connections 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3a5c0a2ca0 0x7f3a5c0a5160 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3a7010ede0 0x7f3a7019f630 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a7006b0a0 0x7f3a7019ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 --2- 172.21.15.52:0/2000954180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a701033a0 0x7f3a7019b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 >> 172.21.15.52:0/2000954180 conn(0x7f3a70078340 msgr2=0x7f3a70100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 shutdown_connections 2024-06-02T23:23:40.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:40.478+0000 7f3a7782a700 1 -- 172.21.15.52:0/2000954180 wait complete. 2024-06-02T23:23:40.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.87 MB/sec execute 529 sec latency 16215.373 ms 2024-06-02T23:23:41.166 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:23:41.238 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:41.362 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:41 smithi052 ceph-mon[32340]: pgmap v785: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-02T23:23:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:41 smithi174 ceph-mon[35994]: pgmap v785: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-02T23:23:41.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.86 MB/sec execute 530 sec latency 17215.466 ms 2024-06-02T23:23:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:41 smithi123 ceph-mon[35977]: pgmap v785: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-06-02T23:23:42.467 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:42.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231193 1.86 MB/sec execute 531 sec latency 18215.554 ms 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 -- 172.21.15.52:0/3751522497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910102ed0 msgr2=0x7fb91010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 --2- 172.21.15.52:0/3751522497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910102ed0 0x7fb91010f360 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fb90800b0a0 tx=0x7fb90800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 -- 172.21.15.52:0/3751522497 shutdown_connections 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 --2- 172.21.15.52:0/3751522497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910102ed0 0x7fb91010f360 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 --2- 172.21.15.52:0/3751522497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910108520 0x7fb910108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 --2- 172.21.15.52:0/3751522497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb910102510 0x7fb910102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 -- 172.21.15.52:0/3751522497 >> 172.21.15.52:0/3751522497 conn(0x7fb910079640 msgr2=0x7fb910079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:42.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 -- 172.21.15.52:0/3751522497 shutdown_connections 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.975+0000 7fb916898700 1 -- 172.21.15.52:0/3751522497 wait complete. 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 Processor -- start 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 -- start start 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 0x7fb910074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb910102ed0 0x7fb910074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910108520 0x7fb910075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101126a0 con 0x7fb910108520 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb910112520 con 0x7fb910102510 2024-06-02T23:23:42.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.976+0000 7fb916898700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb910112820 con 0x7fb910102ed0 2024-06-02T23:23:42.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 0x7fb910074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 0x7fb910074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41912/0 (socket says 172.21.15.52:41912) 2024-06-02T23:23:42.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910108520 0x7fb910075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90ffff700 1 -- 172.21.15.52:0/3770856808 learned_addr learned my addr 172.21.15.52:0/3770856808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb907fff700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb910102ed0 0x7fb910074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 -- 172.21.15.52:0/3770856808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb910102ed0 msgr2=0x7fb910074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:42.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb910102ed0 0x7fb910074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 -- 172.21.15.52:0/3770856808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 msgr2=0x7fb910074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 0x7fb910074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 -- 172.21.15.52:0/3770856808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9101a7000 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90ffff700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 0x7fb910074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb914e35700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910108520 0x7fb910075210 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb908009fd0 tx=0x7fb908004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90dffb700 1 -- 172.21.15.52:0/3770856808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908021070 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90dffb700 1 -- 172.21.15.52:0/3770856808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb908007b80 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9101a7290 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.977+0000 7fb90dffb700 1 -- 172.21.15.52:0/3770856808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080052a0 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.978+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9101a7740 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.980+0000 7fb90dffb700 1 -- 172.21.15.52:0/3770856808 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb908024070 con 0x7fb910108520 2024-06-02T23:23:42.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.980+0000 7fb90dffb700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8fc0a2c20 0x7fb8fc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb90ffff700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8fc0a2c20 0x7fb8fc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb90dffb700 1 -- 172.21.15.52:0/3770856808 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb90800f020 con 0x7fb910108520 2024-06-02T23:23:42.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb90ffff700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8fc0a2c20 0x7fb8fc0a50e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb9000103d0 tx=0x7fb900007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb92111bb80 1 Processor -- start 2024-06-02T23:23:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb92111bb80 1 -- start start 2024-06-02T23:23:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.981+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e8315710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8316070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb92111bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5600e8208060 con 0x5600e8315c50 2024-06-02T23:23:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb92111bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5600e8266950 con 0x5600e831b500 2024-06-02T23:23:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb92111bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600e81ce560 con 0x5600e831be20 2024-06-02T23:23:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb914e35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8316070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb914e35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8316070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58974/0 (socket says 172.21.15.52:58974) 2024-06-02T23:23:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb914e35700 1 -- 172.21.15.52:0/2001148816 learned_addr learned my addr 172.21.15.52:0/2001148816 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:42.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb907fff700 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e8315710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 -- 172.21.15.52:0/2001148816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 msgr2=0x5600e8315710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e8315710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 -- 172.21.15.52:0/2001148816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 msgr2=0x5600e8316070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8316070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 -- 172.21.15.52:0/2001148816 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e81b59d0 con 0x5600e831b500 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb914e35700 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8316070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.982+0000 7fb90ffff700 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831b8e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9000115d0 tx=0x7fb900013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c67f50b5071d4f6b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb9067fc700 1 -- 172.21.15.52:0/2001148816 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90001e620 con 0x5600e831b500 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb9067fc700 1 -- 172.21.15.52:0/2001148816 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb90001ec70 con 0x5600e831b500 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb9067fc700 1 -- 172.21.15.52:0/2001148816 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90001ccf0 con 0x5600e831b500 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 -- 172.21.15.52:0/2001148816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 msgr2=0x5600e831b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:42.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831b8e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb9000115d0 tx=0x7fb900013000 comp rx=0 tx=0).stop 2024-06-02T23:23:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 -- 172.21.15.52:0/2001148816 shutdown_connections 2024-06-02T23:23:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8316070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831b8e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2001148816 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e8315710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.983+0000 7fb92111bb80 1 -- 172.21.15.52:0/2001148816 >> 172.21.15.52:0/2001148816 conn(0x5600e8242180 msgr2=0x5600e8310a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 -- 172.21.15.52:0/2001148816 shutdown_connections 2024-06-02T23:23:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 -- 172.21.15.52:0/2001148816 wait complete. 2024-06-02T23:23:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 -- start start 2024-06-02T23:23:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 Processor -- start 2024-06-02T23:23:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8321bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e831f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5600e82070c0 con 0x5600e8315c50 2024-06-02T23:23:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5600e8262910 con 0x5600e831b500 2024-06-02T23:23:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.984+0000 7fb92111bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600e8322800 con 0x5600e831be20 2024-06-02T23:23:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb92111bb80 10 client.0 _ll_register_callbacks cb 0x5600e81fc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:42.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8321bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8321bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58984/0 (socket says 172.21.15.52:58984) 2024-06-02T23:23:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 -- 172.21.15.52:0/2781882005 learned_addr learned my addr 172.21.15.52:0/2781882005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb914e35700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e831f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb907fff700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 -- 172.21.15.52:0/2781882005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 msgr2=0x5600e831f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:42.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e831f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 -- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 msgr2=0x5600e831eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:42.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:42.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 -- 172.21.15.52:0/2781882005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e822c850 con 0x5600e8315c50 2024-06-02T23:23:42.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb914e35700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e831f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:42.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb90ffff700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8321bd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb900011010 tx=0x7fb90001c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6607f0348e37fd94 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:42.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb8ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:42.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/2781882005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900036060 con 0x5600e8315c50 2024-06-02T23:23:42.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/2781882005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb90001cb60 con 0x5600e8315c50 2024-06-02T23:23:42.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.985+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb90001ec70 con 0x5600e8315c50 2024-06-02T23:23:42.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/2781882005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb90001cd60 con 0x5600e8315c50 2024-06-02T23:23:42.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 10 client.18786 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:42.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600e832cfc0 con 0x5600e8315c50 2024-06-02T23:23:42.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/2781882005 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb90001ec70 con 0x5600e8315c50 2024-06-02T23:23:42.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/2781882005 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb9000267c0 con 0x5600e8315c50 2024-06-02T23:23:42.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 10 client.18786 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:42.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 10 client.18786 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:23:42.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 20 client.18786 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:42.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e832f540 0x5600e8331930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 4 client.18786 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600e8273e50 con 0x5600e832f540 2024-06-02T23:23:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.986+0000 7fb907fff700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e832f540 0x5600e8331930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.987+0000 7fb8ef7fe700 10 client.18786 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:42.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.987+0000 7fb907fff700 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e832f540 0x5600e8331930 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b817634f2e972d81 server_cookie=d4d89ab6ddcbc8b7 in_seq=0 out_seq=0 2024-06-02T23:23:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/2781882005 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5600e8273e50 con 0x5600e832f540 2024-06-02T23:23:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb8ef7fe700 10 client.18786 handle_command_reply: tid=0 2024-06-02T23:23:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 client.18786 shutdown 2024-06-02T23:23:43.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 msgr2=0x5600e8321bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8321bd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fb900011010 tx=0x7fb90001c630 comp rx=0 tx=0).stop 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 shutdown_connections 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e832f540 0x5600e8331930 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e831f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8321bd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 --2- 172.21.15.52:0/2781882005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e831eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 >> 172.21.15.52:0/2781882005 conn(0x5600e8242180 msgr2=0x5600e830fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:43.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb8ef7fe700 0 client.18786 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 shutdown_connections 2024-06-02T23:23:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 1 -- 172.21.15.52:0/2781882005 wait complete. 2024-06-02T23:23:43.002 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:42.988+0000 7fb92111bb80 20 client.18786 trim_cache size 0 max 16384 2024-06-02T23:23:43.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 Processor -- start 2024-06-02T23:23:43.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 -- start start 2024-06-02T23:23:43.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e8315c50 0x5600e8321710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8314e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 0x5600e8315350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101aebd0 con 0x5600e831be20 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b0710 con 0x5600e831b500 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb92111bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101af2a0 con 0x5600e8315c50 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb914e35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 0x5600e8315350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb914e35700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 0x5600e8315350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58996/0 (socket says 172.21.15.52:58996) 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb914e35700 1 -- 172.21.15.52:0/4039643661 learned_addr learned my addr 172.21.15.52:0/4039643661 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.009+0000 7fb90ffff700 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e8315c50 0x5600e8321710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb907fff700 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8314e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb914e35700 1 -- 172.21.15.52:0/4039643661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e8315c50 msgr2=0x5600e8321710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:43.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb914e35700 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e8315c50 0x5600e8321710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb914e35700 1 -- 172.21.15.52:0/4039643661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 msgr2=0x5600e8314e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb914e35700 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8314e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb914e35700 1 -- 172.21.15.52:0/4039643661 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600e8308430 con 0x5600e831be20 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb914e35700 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 0x5600e8315350 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb908099700 tx=0x7fb908009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbbd3feebc82a6e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb905ffb700 1 -- 172.21.15.52:0/4039643661 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90809d480 con 0x5600e831be20 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb905ffb700 1 -- 172.21.15.52:0/4039643661 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb90809cd80 con 0x5600e831be20 2024-06-02T23:23:43.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.010+0000 7fb905ffb700 1 -- 172.21.15.52:0/4039643661 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90809cfe0 con 0x5600e831be20 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 -- 172.21.15.52:0/4039643661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 msgr2=0x5600e8315350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 0x5600e8315350 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fb908099700 tx=0x7fb908009040 comp rx=0 tx=0).stop 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 -- 172.21.15.52:0/4039643661 shutdown_connections 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e831be20 0x5600e8315350 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e8315c50 0x5600e8321710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 --2- 172.21.15.52:0/4039643661 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8314e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 -- 172.21.15.52:0/4039643661 >> 172.21.15.52:0/4039643661 conn(0x5600e8242180 msgr2=0x5600e8314700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 -- 172.21.15.52:0/4039643661 shutdown_connections 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 -- 172.21.15.52:0/4039643661 wait complete. 2024-06-02T23:23:43.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 -- start start 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 Processor -- start 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8336c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.011+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8337190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb92111bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e83376d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb92111bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101b0040 con 0x5600e8315c50 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb92111bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101af970 con 0x5600e831b500 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb92111bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9101acb70 con 0x5600e831be20 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb92111bb80 10 client.0 _ll_register_callbacks cb 0x5600e81fc6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8336c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8336c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59010/0 (socket says 172.21.15.52:59010) 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 -- 172.21.15.52:0/1881787616 learned_addr learned my addr 172.21.15.52:0/1881787616 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb914e35700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e83376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb907fff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8337190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 -- 172.21.15.52:0/1881787616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 msgr2=0x5600e83376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e83376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 -- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 msgr2=0x5600e8337190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8337190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 -- 172.21.15.52:0/1881787616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600e8398fd0 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb907fff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8337190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb90ffff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8336c50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb90001c3f0 tx=0x7fb9000071c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3463b4472a132a8f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb8ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/1881787616 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900041030 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/1881787616 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb90002d040 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.012+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb90809cd80 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/1881787616 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb900036070 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 10 client.18792 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5600e839a2d0 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/1881787616 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb90809cd80 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/1881787616 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb90003c580 con 0x5600e8315c50 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 10 client.18792 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:43.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 10 client.18792 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 20 client.18792 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e839a9d0 0x5600e839cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 4 client.18792 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "9d5fe456", "release": true, "await": true}] 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "9d5fe456", "release": true, "await": true}) -- 0x5600e826e3d0 con 0x5600e839a9d0 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.013+0000 7fb907fff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e839a9d0 0x5600e839cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.014+0000 7fb907fff700 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e839a9d0 0x5600e839cdf0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de59804792876161 server_cookie=f0c95a7c238564f7 in_seq=0 out_seq=0 2024-06-02T23:23:43.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.014+0000 7fb8ef7fe700 10 client.18792 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb8ef7fe700 1 -- 172.21.15.52:0/1881787616 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5600e826e3d0 con 0x5600e839a9d0 2024-06-02T23:23:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb8ef7fe700 10 client.18792 handle_command_reply: tid=0 2024-06-02T23:23:43.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 client.18792 shutdown 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 msgr2=0x5600e8336c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8336c50 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fb90001c3f0 tx=0x7fb9000071c0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 shutdown_connections 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5600e839a9d0 0x5600e839cdf0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5600e831be20 0x5600e83376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5600e8315c50 0x5600e8336c50 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 --2- 172.21.15.52:0/1881787616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5600e831b500 0x5600e8337190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb8ef7fe700 0 client.18792 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 >> 172.21.15.52:0/1881787616 conn(0x5600e8242180 msgr2=0x5600e8314f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 shutdown_connections 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 1 -- 172.21.15.52:0/1881787616 wait complete. 2024-06-02T23:23:43.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.021+0000 7fb92111bb80 20 client.18792 trim_cache size 0 max 16384 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 40, 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "9d5fe456": { 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 40, 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:23:43.022 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 140.8, 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:43.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:43.024 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:43.024 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:23:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8fc0a2c20 msgr2=0x7fb8fc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:43.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8fc0a2c20 0x7fb8fc0a50e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fb9000103d0 tx=0x7fb900007480 comp rx=0 tx=0).stop 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910108520 msgr2=0x7fb910075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910108520 0x7fb910075210 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fb908009fd0 tx=0x7fb908004c60 comp rx=0 tx=0).stop 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 shutdown_connections 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8fc0a2c20 0x7fb8fc0a50e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb910108520 0x7fb910075210 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb910102510 0x7fb910074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 --2- 172.21.15.52:0/3770856808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb910102ed0 0x7fb910074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 >> 172.21.15.52:0/3770856808 conn(0x7fb910079640 msgr2=0x7fb9101011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 shutdown_connections 2024-06-02T23:23:43.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:43.022+0000 7fb916898700 1 -- 172.21.15.52:0/3770856808 wait complete. 2024-06-02T23:23:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:43 smithi174 ceph-mon[35994]: pgmap v786: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 17 op/s 2024-06-02T23:23:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:43 smithi052 ceph-mon[32340]: pgmap v786: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 17 op/s 2024-06-02T23:23:43.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 231517 1.86 MB/sec execute 532 sec latency 18585.553 ms 2024-06-02T23:23:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:43 smithi123 ceph-mon[35977]: pgmap v786: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.7 KiB/s wr, 17 op/s 2024-06-02T23:23:44.294 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '9d5fe456', total seconds elapsed: 30.0 2024-06-02T23:23:44.294 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.75 2024-06-02T23:23:44.294 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:23:44.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 232079 1.86 MB/sec execute 533 sec latency 14.463 ms 2024-06-02T23:23:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.379+0000 7fcd3c52f700 1 -- 172.21.15.52:0/2395833905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd34101d40 msgr2=0x7fcd3410e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.379+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/2395833905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd34101d40 0x7fcd3410e200 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c009990 tx=0x7fcd2c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 -- 172.21.15.52:0/2395833905 shutdown_connections 2024-06-02T23:23:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/2395833905 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd34101d40 0x7fcd3410e200 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/2395833905 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd341073c0 0x7fcd341077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/2395833905 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd34101380 0x7fcd34101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 -- 172.21.15.52:0/2395833905 >> 172.21.15.52:0/2395833905 conn(0x7fcd34078330 msgr2=0x7fcd34078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:45.380 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:45 smithi052 ceph-mon[32340]: pgmap v787: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-06-02T23:23:45.381 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:45.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 -- 172.21.15.52:0/2395833905 shutdown_connections 2024-06-02T23:23:45.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.380+0000 7fcd3c52f700 1 -- 172.21.15.52:0/2395833905 wait complete. 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 Processor -- start 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 -- start start 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 0x7fcd341999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd34101d40 0x7fcd34199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd341073c0 0x7fcd341a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd34111560 con 0x7fcd341073c0 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341113e0 con 0x7fcd34101380 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.381+0000 7fcd3c52f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd341116e0 con 0x7fcd34101d40 2024-06-02T23:23:45.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 0x7fcd341999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 0x7fcd341999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49180/0 (socket says 172.21.15.52:49180) 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/862539145 learned_addr learned my addr 172.21.15.52:0/862539145 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd341073c0 0x7fcd341a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd39aca700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd34101d40 0x7fcd34199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/862539145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd34101d40 msgr2=0x7fcd34199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd34101d40 0x7fcd34199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd341073c0 msgr2=0x7fcd341a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd341073c0 0x7fcd341a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/862539145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd341977d0 con 0x7fcd34101380 2024-06-02T23:23:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 0x7fcd341999e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c370 tx=0x7fcd3000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.382+0000 7fcd2b7fe700 1 -- 172.21.15.52:0/862539145 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000cea0 con 0x7fcd34101380 2024-06-02T23:23:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.383+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd34197a60 con 0x7fcd34101380 2024-06-02T23:23:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.383+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd34197ff0 con 0x7fcd34101380 2024-06-02T23:23:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.383+0000 7fcd2b7fe700 1 -- 172.21.15.52:0/862539145 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd300090d0 con 0x7fcd34101380 2024-06-02T23:23:45.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.383+0000 7fcd2b7fe700 1 -- 172.21.15.52:0/862539145 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30004440 con 0x7fcd34101380 2024-06-02T23:23:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.385+0000 7fcd2b7fe700 1 -- 172.21.15.52:0/862539145 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcd3001a020 con 0x7fcd34101380 2024-06-02T23:23:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.386+0000 7fcd2b7fe700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd200a2c20 0x7fcd200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.386+0000 7fcd2b7fe700 1 -- 172.21.15.52:0/862539145 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcd300cf910 con 0x7fcd34101380 2024-06-02T23:23:45.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.386+0000 7fcd39aca700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd200a2c20 0x7fcd200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.387+0000 7fcd39aca700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd200a2c20 0x7fcd200a50e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcd340789a0 tx=0x7fcd24009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.387+0000 7fcd46db2b80 1 Processor -- start 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 -- start start 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1a770 0x558615b1ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b14a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 0x558615b153f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586159fef60 con 0x558615b14fb0 2024-06-02T23:23:45.391 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558615a5d850 con 0x558615b1a770 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586159c5460 con 0x558615b1b090 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 0x558615b153f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 0x558615b153f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46226/0 (socket says 172.21.15.52:46226) 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 -- 172.21.15.52:0/2424714078 learned_addr learned my addr 172.21.15.52:0/2424714078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1a770 0x558615b1ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 -- 172.21.15.52:0/2424714078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 msgr2=0x558615b14a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:23:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b14a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 -- 172.21.15.52:0/2424714078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1a770 msgr2=0x558615b1ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1a770 0x558615b1ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.388+0000 7fcd3aacc700 1 -- 172.21.15.52:0/2424714078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586159ac8d0 con 0x558615b14fb0 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.389+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 0x558615b153f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c005c10 tx=0x7fcd2c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c06a01ede1219b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.389+0000 7fcd28ff9700 1 -- 172.21.15.52:0/2424714078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c027070 con 0x558615b14fb0 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.389+0000 7fcd28ff9700 1 -- 172.21.15.52:0/2424714078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd2c00bc90 con 0x558615b14fb0 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.389+0000 7fcd28ff9700 1 -- 172.21.15.52:0/2424714078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c003a70 con 0x558615b14fb0 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2424714078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 msgr2=0x558615b153f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 0x558615b153f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c005c10 tx=0x7fcd2c005c40 comp rx=0 tx=0).stop 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2424714078 shutdown_connections 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b14fb0 0x558615b153f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1a770 0x558615b1ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2424714078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b14a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.390+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2424714078 >> 172.21.15.52:0/2424714078 conn(0x558615a39080 msgr2=0x558615b07220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.391+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2424714078 shutdown_connections 2024-06-02T23:23:45.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.391+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2424714078 wait complete. 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.391+0000 7fcd46db2b80 1 -- start start 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.391+0000 7fcd46db2b80 1 Processor -- start 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b20e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1a770 0x558615b1e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1b090 0x558615b1e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd3a2cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b20e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586159fdfc0 con 0x558615b1b090 2024-06-02T23:23:45.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558615a59810 con 0x558615b14fb0 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd3a2cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b20e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49196/0 (socket says 172.21.15.52:49196) 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558615b21a20 con 0x558615b1a770 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/541495059 learned_addr learned my addr 172.21.15.52:0/541495059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd46db2b80 10 client.0 _ll_register_callbacks cb 0x5586159f35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1b090 0x558615b1e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.392+0000 7fcd39aca700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1a770 0x558615b1e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/541495059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1a770 msgr2=0x558615b1e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1a770 0x558615b1e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/541495059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1b090 msgr2=0x558615b1e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1b090 0x558615b1e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/541495059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558615a23750 con 0x558615b14fb0 2024-06-02T23:23:45.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b20e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcd30098600 tx=0x7fcd300bc330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e994427262df8e54 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd1dffb700 1 -- 172.21.15.52:0/541495059 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300a0550 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd1dffb700 1 -- 172.21.15.52:0/541495059 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcd300a0170 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2c00bc90 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd1dffb700 1 -- 172.21.15.52:0/541495059 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd300a4e10 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd46db2b80 10 client.28223 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.393+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558615b2c290 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd1dffb700 1 -- 172.21.15.52:0/541495059 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd30004bd0 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd1dffb700 1 -- 172.21.15.52:0/541495059 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcd300045e0 con 0x558615b14fb0 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd46db2b80 10 client.28223 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd46db2b80 10 client.28223 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:45.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd46db2b80 20 client.28223 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b2e870 0x558615b30c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd46db2b80 4 client.28223 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fcd2c00bc90 con 0x558615b2e870 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd39aca700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b2e870 0x558615b30c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd39aca700 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b2e870 0x558615b30c60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1831cfe97bcaf077 server_cookie=dbffc2a7788d6d12 in_seq=0 out_seq=0 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.394+0000 7fcd1dffb700 10 client.28223 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd1dffb700 1 -- 172.21.15.52:0/541495059 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fcd2c00bc90 con 0x558615b2e870 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd1dffb700 10 client.28223 handle_command_reply: tid=0 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd46db2b80 1 client.28223 shutdown 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 msgr2=0x558615b20e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b20e40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcd30098600 tx=0x7fcd300bc330 comp rx=0 tx=0).stop 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 shutdown_connections 2024-06-02T23:23:45.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.395+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b2e870 0x558615b30c60 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd1dffb700 0 client.28223 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1b090 0x558615b1e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b20e40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/541495059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1a770 0x558615b1e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 >> 172.21.15.52:0/541495059 conn(0x558615a39080 msgr2=0x558615b06700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 shutdown_connections 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 1 -- 172.21.15.52:0/541495059 wait complete. 2024-06-02T23:23:45.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.396+0000 7fcd46db2b80 20 client.28223 trim_cache size 0 max 16384 2024-06-02T23:23:45.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 Processor -- start 2024-06-02T23:23:45.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 -- start start 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 0x558615b1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b20980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1b090 0x558615b06e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800d250 con 0x558615b1a770 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800ed90 con 0x558615b1b090 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800d920 con 0x558615b14fb0 2024-06-02T23:23:45.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd3a2cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 0x558615b1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd3a2cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 0x558615b1bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47430/0 (socket says 172.21.15.52:47430) 2024-06-02T23:23:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/2172185909 learned_addr learned my addr 172.21.15.52:0/2172185909 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd39aca700 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b20980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.413+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1b090 0x558615b06e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/2172185909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1b090 msgr2=0x558615b06e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1b090 0x558615b06e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/2172185909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 msgr2=0x558615b20980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b20980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3a2cb700 1 -- 172.21.15.52:0/2172185909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558615b1a2f0 con 0x558615b14fb0 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1b090 0x558615b06e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 0x558615b1bb00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd300a48e0 tx=0x7fcd3009a6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de630e74790b3fc7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd1ffff700 1 -- 172.21.15.52:0/2172185909 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd300a4980 con 0x558615b14fb0 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.414+0000 7fcd1ffff700 1 -- 172.21.15.52:0/2172185909 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd300a4b90 con 0x558615b14fb0 2024-06-02T23:23:45.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd1ffff700 1 -- 172.21.15.52:0/2172185909 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30004bd0 con 0x558615b14fb0 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2172185909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 msgr2=0x558615b1bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 0x558615b1bb00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcd300a48e0 tx=0x7fcd3009a6b0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2172185909 shutdown_connections 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b1b090 0x558615b06e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b14fb0 0x558615b1bb00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/2172185909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b20980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2172185909 >> 172.21.15.52:0/2172185909 conn(0x558615a39080 msgr2=0x558615b0bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:45.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2172185909 shutdown_connections 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 -- 172.21.15.52:0/2172185909 wait complete. 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 -- start start 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.415+0000 7fcd46db2b80 1 Processor -- start 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b26f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b27460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b23d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800e6c0 con 0x558615b1a770 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800dff0 con 0x558615b14fb0 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd1800b1f0 con 0x558615b1b090 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd46db2b80 10 client.0 _ll_register_callbacks cb 0x5586159f35b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b27460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b27460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46250/0 (socket says 172.21.15.52:46250) 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 -- 172.21.15.52:0/174811415 learned_addr learned my addr 172.21.15.52:0/174811415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:45.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b26f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd3aacc700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b23d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 -- 172.21.15.52:0/174811415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 msgr2=0x558615b23d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b23d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 -- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 msgr2=0x558615b26f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b26f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 -- 172.21.15.52:0/174811415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558615b24380 con 0x558615b1a770 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd39aca700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b27460 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x558615b1f710 tx=0x7fcd240047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c2cf68b275c4f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.416+0000 7fcd1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:45.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.417+0000 7fcd1dffb700 1 -- 172.21.15.52:0/174811415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd24003b50 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.417+0000 7fcd1dffb700 1 -- 172.21.15.52:0/174811415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcd24003cf0 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.417+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd300a4b90 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.417+0000 7fcd1dffb700 1 -- 172.21.15.52:0/174811415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fcd24031470 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.417+0000 7fcd46db2b80 10 client.18810 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.417+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558615b99ed0 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd1dffb700 1 -- 172.21.15.52:0/174811415 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd300a4b90 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd1dffb700 1 -- 172.21.15.52:0/174811415 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fcd24012680 con 0x558615b1a770 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd46db2b80 10 client.18810 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd46db2b80 10 client.18810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd46db2b80 20 client.18810 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b9a610 0x558615b9ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd46db2b80 4 client.18810 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:23:45.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558615a6ad50 con 0x558615b9a610 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.418+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b9a610 0x558615b9ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd1dffb700 10 client.18810 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd3a2cb700 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b9a610 0x558615b9ca30 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c729e5a202ad6f7 server_cookie=d222ecf19d5b7a58 in_seq=0 out_seq=0 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd1dffb700 1 -- 172.21.15.52:0/174811415 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558615a6ad50 con 0x558615b9a610 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd1dffb700 10 client.18810 handle_command_reply: tid=0 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 client.18810 shutdown 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 msgr2=0x558615b27460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b27460 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x558615b1f710 tx=0x7fcd240047c0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 shutdown_connections 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558615b9a610 0x558615b9ca30 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558615b1b090 0x558615b23d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558615b14fb0 0x558615b26f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 --2- 172.21.15.52:0/174811415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558615b1a770 0x558615b27460 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 >> 172.21.15.52:0/174811415 conn(0x558615a39080 msgr2=0x558615b0bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd1dffb700 0 client.18810 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.419+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 shutdown_connections 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.420+0000 7fcd46db2b80 1 -- 172.21.15.52:0/174811415 wait complete. 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.420+0000 7fcd46db2b80 20 client.18810 trim_cache size 0 max 16384 2024-06-02T23:23:45.431 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:23:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.420+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd200a2c20 msgr2=0x7fcd200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.420+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd200a2c20 0x7fcd200a50e0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcd340789a0 tx=0x7fcd24009260 comp rx=0 tx=0).stop 2024-06-02T23:23:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.420+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 msgr2=0x7fcd341999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.420+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 0x7fcd341999e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c370 tx=0x7fcd3000c680 comp rx=0 tx=0).stop 2024-06-02T23:23:45.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 shutdown_connections 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd200a2c20 0x7fcd200a50e0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd341073c0 0x7fcd341a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd34101380 0x7fcd341999e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 --2- 172.21.15.52:0/862539145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd34101d40 0x7fcd34199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 >> 172.21.15.52:0/862539145 conn(0x7fcd34078330 msgr2=0x7fcd340ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 shutdown_connections 2024-06-02T23:23:45.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:45.421+0000 7fcd3c52f700 1 -- 172.21.15.52:0/862539145 wait complete. 2024-06-02T23:23:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:45 smithi174 ceph-mon[35994]: pgmap v787: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-06-02T23:23:45.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 232689 1.86 MB/sec execute 534 sec latency 14.763 ms 2024-06-02T23:23:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:45 smithi123 ceph-mon[35977]: pgmap v787: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.4 KiB/s wr, 16 op/s 2024-06-02T23:23:46.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 233213 1.86 MB/sec execute 535 sec latency 14.913 ms 2024-06-02T23:23:47.635 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:47 smithi052 ceph-mon[32340]: pgmap v788: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 155 KiB/s wr, 38 op/s 2024-06-02T23:23:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:47 smithi174 ceph-mon[35994]: pgmap v788: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 155 KiB/s wr, 38 op/s 2024-06-02T23:23:47.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 233924 1.87 MB/sec execute 536 sec latency 14.875 ms 2024-06-02T23:23:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:47 smithi123 ceph-mon[35977]: pgmap v788: 97 pgs: 97 active+clean; 94 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 155 KiB/s wr, 38 op/s 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 -- 172.21.15.52:0/673275099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06ab30 msgr2=0x7fa1fc10f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 --2- 172.21.15.52:0/673275099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc10f320 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fa1f400b0a0 tx=0x7fa1f400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 -- 172.21.15.52:0/673275099 shutdown_connections 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 --2- 172.21.15.52:0/673275099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc10f320 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 --2- 172.21.15.52:0/673275099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc108500 0x7fa1fc1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 --2- 172.21.15.52:0/673275099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06a170 0x7fa1fc06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:47.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 -- 172.21.15.52:0/673275099 >> 172.21.15.52:0/673275099 conn(0x7fa1fc078340 msgr2=0x7fa1fc078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:47.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 -- 172.21.15.52:0/673275099 shutdown_connections 2024-06-02T23:23:47.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.908+0000 7fa200a86700 1 -- 172.21.15.52:0/673275099 wait complete. 2024-06-02T23:23:47.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.909+0000 7fa200a86700 1 Processor -- start 2024-06-02T23:23:47.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.909+0000 7fa200a86700 1 -- start start 2024-06-02T23:23:47.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa200a86700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06a170 0x7fa1fc1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa200a86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc1a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa200a86700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc108500 0x7fa1fc19ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa200a86700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc112610 con 0x7fa1fc06a170 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa200a86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc112490 con 0x7fa1fc06ab30 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa200a86700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1fc112790 con 0x7fa1fc108500 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1fa59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06a170 0x7fa1fc1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc1a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1fad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc108500 0x7fa1fc19ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc1a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49234/0 (socket says 172.21.15.52:49234) 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1fa59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06a170 0x7fa1fc1a0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46278/0 (socket says 172.21.15.52:46278) 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 -- 172.21.15.52:0/3901012800 learned_addr learned my addr 172.21.15.52:0/3901012800 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 -- 172.21.15.52:0/3901012800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc108500 msgr2=0x7fa1fc19ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:47.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc108500 0x7fa1fc19ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 -- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06a170 msgr2=0x7fa1fc1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06a170 0x7fa1fc1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 -- 172.21.15.52:0/3901012800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1fc19b3f0 con 0x7fa1fc06ab30 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.910+0000 7fa1f9d9b700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc1a0f80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e9a0 tx=0x7fa1f000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.911+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000ca10 con 0x7fa1fc06ab30 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.911+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1fc19b6e0 con 0x7fa1fc06ab30 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.911+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa1f000cbb0 con 0x7fa1fc06ab30 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.911+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1fc1a79c0 con 0x7fa1fc06ab30 2024-06-02T23:23:47.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.911+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0018680 con 0x7fa1fc06ab30 2024-06-02T23:23:47.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.912+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1fc103bc0 con 0x7fa1fc06ab30 2024-06-02T23:23:47.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.913+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa1f0010430 con 0x7fa1fc06ab30 2024-06-02T23:23:47.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.914+0000 7fa1eb7fe700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1e40a2c20 0x7fa1e40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:47.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.914+0000 7fa1fa59c700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1e40a2c20 0x7fa1e40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:47.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.914+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa1f00cf430 con 0x7fa1fc06ab30 2024-06-02T23:23:47.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.914+0000 7fa1fa59c700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1e40a2c20 0x7fa1e40a50e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009be0 tx=0x7fa1ec009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:47.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:47.919+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fa1f009c360 con 0x7fa1fc06ab30 2024-06-02T23:23:47.935 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:23:47.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:23:48.229 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:48.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.240+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1fc1a7b90 con 0x7fa1fc06ab30 2024-06-02T23:23:48.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.241+0000 7fa1eb7fe700 1 -- 172.21.15.52:0/3901012800 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7fa1f009bc60 con 0x7fa1fc06ab30 2024-06-02T23:23:48.241 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:23:48.241 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:23:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1e40a2c20 msgr2=0x7fa1e40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1e40a2c20 0x7fa1e40a50e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa1ec009be0 tx=0x7fa1ec009260 comp rx=0 tx=0).stop 2024-06-02T23:23:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 msgr2=0x7fa1fc1a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:48.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc1a0f80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa1f000e9a0 tx=0x7fa1f000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 shutdown_connections 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1e40a2c20 0x7fa1e40a50e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1fc108500 0x7fa1fc19ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa1fc06a170 0x7fa1fc1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 --2- 172.21.15.52:0/3901012800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1fc06ab30 0x7fa1fc1a0f80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.244+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 >> 172.21.15.52:0/3901012800 conn(0x7fa1fc078340 msgr2=0x7fa1fc10d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.245+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 shutdown_connections 2024-06-02T23:23:48.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:48.245+0000 7fa200a86700 1 -- 172.21.15.52:0/3901012800 wait complete. 2024-06-02T23:23:48.245 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:23:48.582 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:48 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3901012800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:48.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:48 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3901012800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:48.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 234382 1.87 MB/sec execute 537 sec latency 50.249 ms 2024-06-02T23:23:48.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:48 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3901012800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:23:48.923 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 140.75 --await-for 45.0' 2024-06-02T23:23:48.923 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 140.75 --await-for 45.0 2024-06-02T23:23:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:49 smithi052 ceph-mon[32340]: pgmap v789: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 317 KiB/s wr, 39 op/s 2024-06-02T23:23:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:49 smithi174 ceph-mon[35994]: pgmap v789: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 317 KiB/s wr, 39 op/s 2024-06-02T23:23:49.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 234617 1.87 MB/sec execute 538 sec latency 13.898 ms 2024-06-02T23:23:49.658 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:49 smithi123 ceph-mon[35977]: pgmap v789: 97 pgs: 97 active+clean; 96 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 317 KiB/s wr, 39 op/s 2024-06-02T23:23:50.081 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:50.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 -- 172.21.15.52:0/2600165884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c1024a0 msgr2=0x7f2d4c104890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 --2- 172.21.15.52:0/2600165884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c104890 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c00dbf0 tx=0x7f2d3c02e2a0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 -- 172.21.15.52:0/2600165884 shutdown_connections 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 --2- 172.21.15.52:0/2600165884 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c107700 0x7f2d4c109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 --2- 172.21.15.52:0/2600165884 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c104dd0 0x7f2d4c1071c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 --2- 172.21.15.52:0/2600165884 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c104890 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 -- 172.21.15.52:0/2600165884 >> 172.21.15.52:0/2600165884 conn(0x7f2d4c0fbeb0 msgr2=0x7f2d4c0fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 -- 172.21.15.52:0/2600165884 shutdown_connections 2024-06-02T23:23:50.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.257+0000 7f2d521ee640 1 -- 172.21.15.52:0/2600165884 wait complete. 2024-06-02T23:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.258+0000 7f2d521ee640 1 Processor -- start 2024-06-02T23:23:50.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.258+0000 7f2d521ee640 1 -- start start 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.258+0000 7f2d521ee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d521ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c104dd0 0x7f2d4c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d521ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c107700 0x7f2d4c19cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d521ee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c10b2f0 con 0x7f2d4c1024a0 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d521ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c10b170 con 0x7f2d4c107700 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d521ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c10b470 con 0x7f2d4c104dd0 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46298/0 (socket says 172.21.15.52:46298) 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c107700 0x7f2d4c19cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/491118447 learned_addr learned my addr 172.21.15.52:0/491118447 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4affd640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c104dd0 0x7f2d4c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/491118447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c104dd0 msgr2=0x7f2d4c195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c104dd0 0x7f2d4c195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/491118447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c107700 msgr2=0x7f2d4c19cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c107700 0x7f2d4c19cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/491118447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d4c19d550 con 0x7f2d4c1024a0 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c107700 0x7f2d4c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4affd640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c104dd0 0x7f2d4c195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.259+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c195930 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c038040 tx=0x7f2d3c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.260+0000 7f2d48ff9640 1 -- 172.21.15.52:0/491118447 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c044020 con 0x7f2d4c1024a0 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.260+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d4c19d7e0 con 0x7f2d4c1024a0 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.260+0000 7f2d48ff9640 1 -- 172.21.15.52:0/491118447 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d3c00d420 con 0x7f2d4c1024a0 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.260+0000 7f2d48ff9640 1 -- 172.21.15.52:0/491118447 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c00b950 con 0x7f2d4c1024a0 2024-06-02T23:23:50.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.260+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d4c19dd80 con 0x7f2d4c1024a0 2024-06-02T23:23:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d48ff9640 1 -- 172.21.15.52:0/491118447 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2d3c047020 con 0x7f2d4c1024a0 2024-06-02T23:23:50.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d54fdd740 1 Processor -- start 2024-06-02T23:23:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d54fdd740 1 -- start start 2024-06-02T23:23:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 0x55909df2f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df34500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df37000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 0x55909df2f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.263+0000 7f2d4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 0x55909df2f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49258/0 (socket says 172.21.15.52:49258) 2024-06-02T23:23:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4bfff640 1 -- 172.21.15.52:0/3747768484 learned_addr learned my addr 172.21.15.52:0/3747768484 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55909de924f0 con 0x55909df2fa00 2024-06-02T23:23:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55909de30870 con 0x55909df2f010 2024-06-02T23:23:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55909de8e4b0 con 0x55909df34b70 2024-06-02T23:23:50.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4affd640 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df37000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d48ff9640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d340a2c30 0x7f2d340a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d48ff9640 1 -- 172.21.15.52:0/491118447 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2d3c0f36d0 con 0x7f2d4c1024a0 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4bfff640 1 -- 172.21.15.52:0/3747768484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 msgr2=0x55909df37000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df37000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df34500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4bfff640 1 -- 172.21.15.52:0/3747768484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 msgr2=0x55909df34500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df34500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.264+0000 7f2d4bfff640 1 -- 172.21.15.52:0/3747768484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909de4c9b0 con 0x55909df2f010 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.265+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d340a2c30 0x7f2d340a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.265+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df34500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.265+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 0x55909df2f430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2d44011940 tx=0x7f2d44011e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d07c2c45d0e28 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.265+0000 7f2d31ffb640 1 -- 172.21.15.52:0/3747768484 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44017070 con 0x55909df2f010 2024-06-02T23:23:50.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.265+0000 7f2d31ffb640 1 -- 172.21.15.52:0/3747768484 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d440046a0 con 0x55909df2f010 2024-06-02T23:23:50.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.265+0000 7f2d31ffb640 1 -- 172.21.15.52:0/3747768484 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d4401ce00 con 0x55909df2f010 2024-06-02T23:23:50.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d340a2c30 0x7f2d340a50f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2d44023000 tx=0x7f2d44024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 msgr2=0x55909df2f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 0x55909df2f430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2d44011940 tx=0x7f2d44011e00 comp rx=0 tx=0).stop 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 shutdown_connections 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df37000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df34500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/3747768484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2f010 0x55909df2f430 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 >> 172.21.15.52:0/3747768484 conn(0x55909df2a680 msgr2=0x55909df2aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 shutdown_connections 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 -- 172.21.15.52:0/3747768484 wait complete. 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.266+0000 7f2d54fdd740 1 -- start start 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 Processor -- start 2024-06-02T23:23:50.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df41a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2fa00 0x55909df41f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df48fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55909df37b20 con 0x55909df2f010 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55909de1bae0 con 0x55909df34b70 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55909df37ca0 con 0x55909df2fa00 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d54fdd740 10 client.0 _ll_register_callbacks cb 0x55909de549b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df48fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d4bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df48fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49264/0 (socket says 172.21.15.52:49264) 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d4bfff640 1 -- 172.21.15.52:0/1236894706 learned_addr learned my addr 172.21.15.52:0/1236894706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df41a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.267+0000 7f2d4affd640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2fa00 0x55909df41f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4bfff640 1 -- 172.21.15.52:0/1236894706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2fa00 msgr2=0x55909df41f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2fa00 0x55909df41f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4bfff640 1 -- 172.21.15.52:0/1236894706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 msgr2=0x55909df41a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df41a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4bfff640 1 -- 172.21.15.52:0/1236894706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909dba6210 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df41a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df48fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d4401cfc0 tx=0x7f2d44011620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccffe51eb43a3915 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d2affd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d2affd640 1 -- 172.21.15.52:0/1236894706 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d44017040 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d2affd640 1 -- 172.21.15.52:0/1236894706 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2d44011030 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d440046a0 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d2affd640 1 -- 172.21.15.52:0/1236894706 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d44011230 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d54fdd740 10 client.28247 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.268+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55909dc72620 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d2affd640 1 -- 172.21.15.52:0/1236894706 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d440046a0 con 0x55909df34b70 2024-06-02T23:23:50.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d2affd640 1 -- 172.21.15.52:0/1236894706 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2d4401b070 con 0x55909df34b70 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d54fdd740 10 client.28247 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d54fdd740 10 client.28247 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d54fdd740 20 client.28247 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909df4fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d54fdd740 4 client.28247 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55909dea5f10 con 0x55909df4d680 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.269+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909df4fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.270+0000 7f2d2affd640 10 client.28247 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:50.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.270+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909df4fa90 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbb49d54eed304 server_cookie=49cdb65c19c91ce5 in_seq=0 out_seq=0 2024-06-02T23:23:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.270+0000 7f2d2affd640 1 -- 172.21.15.52:0/1236894706 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55909dea5f10 con 0x55909df4d680 2024-06-02T23:23:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.270+0000 7f2d2affd640 10 client.28247 handle_command_reply: tid=0 2024-06-02T23:23:50.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.270+0000 7f2d54fdd740 1 client.28247 shutdown 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 msgr2=0x55909df48fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df48fc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f2d4401cfc0 tx=0x7f2d44011620 comp rx=0 tx=0).stop 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 shutdown_connections 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909df4fa90 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d2affd640 0 client.28247 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2fa00 0x55909df41f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df48fc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/1236894706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df41a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 >> 172.21.15.52:0/1236894706 conn(0x55909df2a680 msgr2=0x55909df297e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 shutdown_connections 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 1 -- 172.21.15.52:0/1236894706 wait complete. 2024-06-02T23:23:50.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.271+0000 7f2d54fdd740 20 client.28247 trim_cache size 0 max 16384 2024-06-02T23:23:50.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 Processor -- start 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 -- start start 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2f010 0x55909df34260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df48f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a6b50 con 0x55909df2fa00 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a8690 con 0x55909df34b70 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a7220 con 0x55909df2f010 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df48f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df48f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46326/0 (socket says 172.21.15.52:46326) 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 -- 172.21.15.52:0/579791365 learned_addr learned my addr 172.21.15.52:0/579791365 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2f010 0x55909df34260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 -- 172.21.15.52:0/579791365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2f010 msgr2=0x55909df34260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2f010 0x55909df34260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 -- 172.21.15.52:0/579791365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 msgr2=0x55909df2d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df2d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4affd640 1 -- 172.21.15.52:0/579791365 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55909df2a4b0 con 0x55909df2fa00 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.292+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2f010 0x55909df34260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.293+0000 7f2d4affd640 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df48f80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c06a760 tx=0x7f2d40010b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316913300d31d1db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.293+0000 7f2d317fa640 1 -- 172.21.15.52:0/579791365 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40010e40 con 0x55909df2fa00 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.293+0000 7f2d317fa640 1 -- 172.21.15.52:0/579791365 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d4000f8f0 con 0x55909df2fa00 2024-06-02T23:23:50.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.293+0000 7f2d317fa640 1 -- 172.21.15.52:0/579791365 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d40005440 con 0x55909df2fa00 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 -- 172.21.15.52:0/579791365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 msgr2=0x55909df48f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df48f80 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c06a760 tx=0x7f2d40010b30 comp rx=0 tx=0).stop 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 -- 172.21.15.52:0/579791365 shutdown_connections 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df34b70 0x55909df2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2fa00 0x55909df48f80 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/579791365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df2f010 0x55909df34260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 -- 172.21.15.52:0/579791365 >> 172.21.15.52:0/579791365 conn(0x55909df2a680 msgr2=0x55909df30c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:50.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 -- 172.21.15.52:0/579791365 shutdown_connections 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 -- 172.21.15.52:0/579791365 wait complete. 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.294+0000 7f2d54fdd740 1 -- start start 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 Processor -- start 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df464a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2fa00 0x55909df469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df479a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a7fc0 con 0x55909df2f010 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a78f0 con 0x55909df2fa00 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2d4c1a4b30 con 0x55909df34b70 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d54fdd740 10 client.0 _ll_register_callbacks cb 0x7f2d40009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df464a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df464a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46328/0 (socket says 172.21.15.52:46328) 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/716945845 learned_addr learned my addr 172.21.15.52:0/716945845 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4affd640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2fa00 0x55909df469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4bfff640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df479a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/716945845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 msgr2=0x55909df479a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df479a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2fa00 msgr2=0x55909df469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.295+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2fa00 0x55909df469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d4b7fe640 1 -- 172.21.15.52:0/716945845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55909df2b350 con 0x55909df2f010 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d4affd640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2fa00 0x55909df469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d4b7fe640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df464a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0f1570 tx=0x7f2d3c00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da348c21f6e95824 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d2affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d2affd640 1 -- 172.21.15.52:0/716945845 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d3c0bf8c0 con 0x55909df2f010 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d2affd640 1 -- 172.21.15.52:0/716945845 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2d3c0bc5d0 con 0x55909df2f010 2024-06-02T23:23:50.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d4000f8f0 con 0x55909df2f010 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d2affd640 1 -- 172.21.15.52:0/716945845 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2d3c00b400 con 0x55909df2f010 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d54fdd740 10 client.18831 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55909df491a0 con 0x55909df2f010 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d2affd640 1 -- 172.21.15.52:0/716945845 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2d4000f8f0 con 0x55909df2f010 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d2affd640 1 -- 172.21.15.52:0/716945845 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2d3c0c3070 con 0x55909df2f010 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d54fdd740 10 client.18831 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d54fdd740 10 client.18831 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.296+0000 7f2d54fdd740 20 client.18831 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.297+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909dfba970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.297+0000 7f2d54fdd740 4 client.18831 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.297+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55909de9fc20 con 0x55909df4d680 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.297+0000 7f2d4affd640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909dfba970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.297+0000 7f2d4affd640 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909dfba970 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a4dc65926fbe17a server_cookie=dadcee3251c42316 in_seq=0 out_seq=0 2024-06-02T23:23:50.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.297+0000 7f2d2affd640 10 client.18831 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d2affd640 1 -- 172.21.15.52:0/716945845 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55909de9fc20 con 0x55909df4d680 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d2affd640 10 client.18831 handle_command_reply: tid=0 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 client.18831 shutdown 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 msgr2=0x55909df464a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df464a0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c0f1570 tx=0x7f2d3c00d040 comp rx=0 tx=0).stop 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 shutdown_connections 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55909df4d680 0x55909dfba970 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55909df34b70 0x55909df479a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55909df2fa00 0x55909df469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 --2- 172.21.15.52:0/716945845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55909df2f010 0x55909df464a0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 >> 172.21.15.52:0/716945845 conn(0x55909df2a680 msgr2=0x55909df30c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:50.317 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d2affd640 0 client.18831 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 shutdown_connections 2024-06-02T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 1 -- 172.21.15.52:0/716945845 wait complete. 2024-06-02T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.317+0000 7f2d54fdd740 20 client.18831 trim_cache size 0 max 16384 2024-06-02T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:23:50.318 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d340a2c30 msgr2=0x7f2d340a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d340a2c30 0x7f2d340a50f0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f2d44023000 tx=0x7f2d44024000 comp rx=0 tx=0).stop 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 msgr2=0x7f2d4c195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c195930 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f2d3c038040 tx=0x7f2d3c0052f0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 shutdown_connections 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2d340a2c30 0x7f2d340a50f0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2d4c104dd0 0x7f2d4c195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.319 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d4c107700 0x7f2d4c19cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 --2- 172.21.15.52:0/491118447 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2d4c1024a0 0x7f2d4c195930 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 >> 172.21.15.52:0/491118447 conn(0x7f2d4c0fbeb0 msgr2=0x7f2d4c103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 shutdown_connections 2024-06-02T23:23:50.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:23:50.318+0000 7f2d521ee640 1 -- 172.21.15.52:0/491118447 wait complete. 2024-06-02T23:23:50.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 235021 1.87 MB/sec execute 539 sec latency 16.550 ms 2024-06-02T23:23:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:51 smithi174 ceph-mon[35994]: pgmap v790: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 598 KiB/s wr, 71 op/s 2024-06-02T23:23:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:51 smithi052 ceph-mon[32340]: pgmap v790: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 598 KiB/s wr, 71 op/s 2024-06-02T23:23:51.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 235575 1.87 MB/sec execute 540 sec latency 58.627 ms 2024-06-02T23:23:51.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:51 smithi123 ceph-mon[35977]: pgmap v790: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 598 KiB/s wr, 71 op/s 2024-06-02T23:23:52.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 236126 1.87 MB/sec execute 541 sec latency 50.941 ms 2024-06-02T23:23:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 -- 172.21.15.52:0/3942228631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 msgr2=0x7f19d4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 --2- 172.21.15.52:0/3942228631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d4100f00 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009990 tx=0x7f19c0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 -- 172.21.15.52:0/3942228631 shutdown_connections 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 --2- 172.21.15.52:0/3942228631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19d4101520 0x7f19d4111210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 --2- 172.21.15.52:0/3942228631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4103360 0x7f19d4103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 --2- 172.21.15.52:0/3942228631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d4100f00 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 -- 172.21.15.52:0/3942228631 >> 172.21.15.52:0/3942228631 conn(0x7f19d4077a00 msgr2=0x7f19d40fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 -- 172.21.15.52:0/3942228631 shutdown_connections 2024-06-02T23:23:52.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 -- 172.21.15.52:0/3942228631 wait complete. 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.924+0000 7f19daebd700 1 Processor -- start 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 -- start start 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d419ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4101520 0x7f19d419b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19d4103360 0x7f19d419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41147f0 con 0x7f19d4100a80 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19d4114670 con 0x7f19d4101520 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19daebd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19d4114970 con 0x7f19d4103360 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d419ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d419ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46346/0 (socket says 172.21.15.52:46346) 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 -- 172.21.15.52:0/1345234849 learned_addr learned my addr 172.21.15.52:0/1345234849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d3fff700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4101520 0x7f19d419b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d945a700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19d4103360 0x7f19d419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 -- 172.21.15.52:0/1345234849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19d4103360 msgr2=0x7f19d419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19d4103360 0x7f19d419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 -- 172.21.15.52:0/1345234849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4101520 msgr2=0x7f19d419b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4101520 0x7f19d419b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d8c59700 1 -- 172.21.15.52:0/1345234849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d419fd30 con 0x7f19d4100a80 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.925+0000 7f19d3fff700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4101520 0x7f19d419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.926+0000 7f19d8c59700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d419ad20 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f19c800e9a0 tx=0x7f19c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.926+0000 7f19d1ffb700 1 -- 172.21.15.52:0/1345234849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c800cbd0 con 0x7f19d4100a80 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.926+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d41a0020 con 0x7f19d4100a80 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.926+0000 7f19d1ffb700 1 -- 172.21.15.52:0/1345234849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c800cd70 con 0x7f19d4100a80 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.926+0000 7f19d1ffb700 1 -- 172.21.15.52:0/1345234849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8018680 con 0x7f19d4100a80 2024-06-02T23:23:52.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.926+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d41a78a0 con 0x7f19d4100a80 2024-06-02T23:23:52.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.929+0000 7f19d1ffb700 1 -- 172.21.15.52:0/1345234849 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f19c80188e0 con 0x7f19d4100a80 2024-06-02T23:23:52.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.930+0000 7f19d1ffb700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19c40a2c20 0x7f19c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.930+0000 7f19d3fff700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19c40a2c20 0x7f19c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.930+0000 7f19d1ffb700 1 -- 172.21.15.52:0/1345234849 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19c80cf260 con 0x7f19d4100a80 2024-06-02T23:23:52.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.930+0000 7f19d3fff700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19c40a2c20 0x7f19c40a50e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f19c000b2e0 tx=0x7f19c001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:52.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.930+0000 7f19e5740b80 1 Processor -- start 2024-06-02T23:23:52.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.930+0000 7f19e5740b80 1 -- start start 2024-06-02T23:23:52.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d15820 0x55c1e8d15c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19e5740b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8c07fe0 con 0x55c1e8d15820 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19e5740b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8c668d0 con 0x55c1e8d1afe0 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19e5740b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8bce4e0 con 0x55c1e8d1b900 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d945a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d15820 0x55c1e8d15c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d152e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47540/0 (socket says 172.21.15.52:47540) 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 -- 172.21.15.52:0/141419618 learned_addr learned my addr 172.21.15.52:0/141419618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d8c59700 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 -- 172.21.15.52:0/141419618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 msgr2=0x55c1e8d1b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 -- 172.21.15.52:0/141419618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d15820 msgr2=0x55c1e8d15c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d15820 0x55c1e8d15c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d3fff700 1 -- 172.21.15.52:0/141419618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e8bb5950 con 0x55c1e8d1b900 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.931+0000 7f19d8c59700 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.932+0000 7f19d3fff700 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d152e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009750 tx=0x7f19c001e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=536cfdf2e3761d6a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.932+0000 7f19beffd700 1 -- 172.21.15.52:0/141419618 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c000f870 con 0x55c1e8d1b900 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.932+0000 7f19beffd700 1 -- 172.21.15.52:0/141419618 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c0005050 con 0x55c1e8d1b900 2024-06-02T23:23:52.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.932+0000 7f19beffd700 1 -- 172.21.15.52:0/141419618 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c0004b20 con 0x55c1e8d1b900 2024-06-02T23:23:52.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 -- 172.21.15.52:0/141419618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 msgr2=0x55c1e8d152e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d152e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009750 tx=0x7f19c001e000 comp rx=0 tx=0).stop 2024-06-02T23:23:52.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 -- 172.21.15.52:0/141419618 shutdown_connections 2024-06-02T23:23:52.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d15820 0x55c1e8d15c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 --2- 172.21.15.52:0/141419618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d152e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 -- 172.21.15.52:0/141419618 >> 172.21.15.52:0/141419618 conn(0x55c1e8c42100 msgr2=0x55c1e8d102a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 -- 172.21.15.52:0/141419618 shutdown_connections 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.933+0000 7f19e5740b80 1 -- 172.21.15.52:0/141419618 wait complete. 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 -- start start 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 Processor -- start 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 0x55c1e8d21690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d1eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8c07040 con 0x55c1e8d1b900 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8c62890 con 0x55c1e8d1afe0 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.934+0000 7f19e5740b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c1e8d22280 con 0x55c1e8d15820 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 0x55c1e8d21690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19e5740b80 10 client.0 _ll_register_callbacks cb 0x55c1e8bfc630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 0x55c1e8d21690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47552/0 (socket says 172.21.15.52:47552) 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 -- 172.21.15.52:0/955875063 learned_addr learned my addr 172.21.15.52:0/955875063 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d945a700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d1eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d3fff700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 -- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 msgr2=0x55c1e8d1e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 -- 172.21.15.52:0/955875063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 msgr2=0x55c1e8d1eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d1eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 -- 172.21.15.52:0/955875063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e8c2c7d0 con 0x55c1e8d15820 2024-06-02T23:23:52.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d3fff700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19d8c59700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 0x55c1e8d21690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f19c80984c0 tx=0x7f19c809c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd4e1b27307bdf4a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19b3fff700 1 -- 172.21.15.52:0/955875063 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c8018dd0 con 0x55c1e8d15820 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19b3fff700 1 -- 172.21.15.52:0/955875063 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19c80862b0 con 0x55c1e8d15820 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.935+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19c0005050 con 0x55c1e8d15820 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19b3fff700 1 -- 172.21.15.52:0/955875063 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19c809baf0 con 0x55c1e8d15820 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 10 client.27991 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1e8d2cac0 con 0x55c1e8d15820 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19b3fff700 1 -- 172.21.15.52:0/955875063 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c0005050 con 0x55c1e8d15820 2024-06-02T23:23:52.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19b3fff700 1 -- 172.21.15.52:0/955875063 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f19c8097840 con 0x55c1e8d15820 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 10 client.27991 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 10 client.27991 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 20 client.27991 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d2f460 0x55c1e8d318e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 4 client.27991 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1e8c73dd0 con 0x55c1e8d2f460 2024-06-02T23:23:52.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.936+0000 7f19d3fff700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d2f460 0x55c1e8d318e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.937+0000 7f19b3fff700 10 client.27991 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:52.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.937+0000 7f19d3fff700 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d2f460 0x55c1e8d318e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b30559b17c9bc2e server_cookie=745f791b7e93d410 in_seq=0 out_seq=0 2024-06-02T23:23:52.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19b3fff700 1 -- 172.21.15.52:0/955875063 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c1e8c73dd0 con 0x55c1e8d2f460 2024-06-02T23:23:52.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19b3fff700 10 client.27991 handle_command_reply: tid=0 2024-06-02T23:23:52.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 client.27991 shutdown 2024-06-02T23:23:52.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 msgr2=0x55c1e8d21690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 0x55c1e8d21690 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f19c80984c0 tx=0x7f19c809c030 comp rx=0 tx=0).stop 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 shutdown_connections 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d2f460 0x55c1e8d318e0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d1eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d15820 0x55c1e8d21690 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 --2- 172.21.15.52:0/955875063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 >> 172.21.15.52:0/955875063 conn(0x55c1e8c42100 msgr2=0x55c1e8d0f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.938+0000 7f19b3fff700 0 client.27991 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.939+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 shutdown_connections 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.939+0000 7f19e5740b80 1 -- 172.21.15.52:0/955875063 wait complete. 2024-06-02T23:23:52.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.939+0000 7f19e5740b80 20 client.27991 trim_cache size 0 max 16384 2024-06-02T23:23:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 Processor -- start 2024-06-02T23:23:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 -- start start 2024-06-02T23:23:52.965 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d211d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41aed20 con 0x55c1e8d1afe0 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b0860 con 0x55c1e8d15820 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19e5740b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41af3f0 con 0x55c1e8d1b900 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46370/0 (socket says 172.21.15.52:46370) 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19d3fff700 1 -- 172.21.15.52:0/3055183654 learned_addr learned my addr 172.21.15.52:0/3055183654 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19d945a700 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d211d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.965+0000 7f19d8c59700 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19d3fff700 1 -- 172.21.15.52:0/3055183654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 msgr2=0x55c1e8d211d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19d3fff700 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d211d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19d3fff700 1 -- 172.21.15.52:0/3055183654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 msgr2=0x55c1e8d152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19d3fff700 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19d3fff700 1 -- 172.21.15.52:0/3055183654 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1e8d1cf00 con 0x55c1e8d1afe0 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19d3fff700 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1c5e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009350 tx=0x7f19c0019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e612c74a1d830a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19be7fc700 1 -- 172.21.15.52:0/3055183654 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c000f930 con 0x55c1e8d1afe0 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19be7fc700 1 -- 172.21.15.52:0/3055183654 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c0039470 con 0x55c1e8d1afe0 2024-06-02T23:23:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.966+0000 7f19be7fc700 1 -- 172.21.15.52:0/3055183654 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19c002f6c0 con 0x55c1e8d1afe0 2024-06-02T23:23:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 -- 172.21.15.52:0/3055183654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 msgr2=0x55c1e8d1c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1c5e0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f19c0009350 tx=0x7f19c0019040 comp rx=0 tx=0).stop 2024-06-02T23:23:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 -- 172.21.15.52:0/3055183654 shutdown_connections 2024-06-02T23:23:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1b900 0x55c1e8d211d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 --2- 172.21.15.52:0/3055183654 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d1c5e0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 -- 172.21.15.52:0/3055183654 >> 172.21.15.52:0/3055183654 conn(0x55c1e8c42100 msgr2=0x55c1e8d1a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 -- 172.21.15.52:0/3055183654 shutdown_connections 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 -- 172.21.15.52:0/3055183654 wait complete. 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 -- start start 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.967+0000 7f19e5740b80 1 Processor -- start 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d37480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41b0190 con 0x55c1e8d1b900 2024-06-02T23:23:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41afac0 con 0x55c1e8d15820 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19d41accc0 con 0x55c1e8d1afe0 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19e5740b80 10 client.0 _ll_register_callbacks cb 0x55c1e8bfc630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19d8c59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19d945a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d37480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19d945a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d37480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46380/0 (socket says 172.21.15.52:46380) 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19d3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19d945a700 1 -- 172.21.15.52:0/2001010753 learned_addr learned my addr 172.21.15.52:0/2001010753 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.968+0000 7f19d8c59700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d27e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49314/0 (socket says 172.21.15.52:49314) 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d945a700 1 -- 172.21.15.52:0/2001010753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1afe0 msgr2=0x55c1e8d36d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d945a700 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d36d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d945a700 1 -- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 msgr2=0x55c1e8d27e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d945a700 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d27e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d945a700 1 -- 172.21.15.52:0/2001010753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1e8d991c0 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d8c59700 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19d945a700 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d37480 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f19cc015fd0 tx=0x7f19cc018700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60809637b240d4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19b3fff700 1 -- 172.21.15.52:0/2001010753 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19cc004bb0 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19b3fff700 1 -- 172.21.15.52:0/2001010753 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19cc004d50 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19c0039470 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19b3fff700 1 -- 172.21.15.52:0/2001010753 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f19cc022470 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19e5740b80 10 client.18849 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.969+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55c1e8d9a440 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19b3fff700 1 -- 172.21.15.52:0/2001010753 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f19c0039470 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19b3fff700 1 -- 172.21.15.52:0/2001010753 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f19cc016740 con 0x55c1e8d1b900 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19e5740b80 10 client.18849 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:52.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19e5740b80 10 client.18849 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19e5740b80 20 client.18849 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19e5740b80 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d9ab80 0x55c1e8d9cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19e5740b80 4 client.18849 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c1e8c6e350 con 0x55c1e8d9ab80 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.970+0000 7f19d8c59700 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d9ab80 0x55c1e8d9cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.971+0000 7f19b3fff700 10 client.18849 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:52.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.971+0000 7f19d8c59700 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d9ab80 0x55c1e8d9cfa0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3bbbf72a74047dc server_cookie=b6ae6ae48f209c20 in_seq=0 out_seq=0 2024-06-02T23:23:52.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.971+0000 7f19b3fff700 1 -- 172.21.15.52:0/2001010753 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c1e8c6e350 con 0x55c1e8d9ab80 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19b3fff700 10 client.18849 handle_command_reply: tid=0 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 client.18849 shutdown 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 msgr2=0x55c1e8d37480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d37480 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f19cc015fd0 tx=0x7f19cc018700 comp rx=0 tx=0).stop 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 shutdown_connections 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c1e8d9ab80 0x55c1e8d9cfa0 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c1e8d1b900 0x55c1e8d37480 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c1e8d15820 0x55c1e8d27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 --2- 172.21.15.52:0/2001010753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c1e8d1afe0 0x55c1e8d36d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19b3fff700 0 client.18849 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 >> 172.21.15.52:0/2001010753 conn(0x55c1e8c42100 msgr2=0x55c1e8d1e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 shutdown_connections 2024-06-02T23:23:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 1 -- 172.21.15.52:0/2001010753 wait complete. 2024-06-02T23:23:52.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.972+0000 7f19e5740b80 20 client.18849 trim_cache size 0 max 16384 2024-06-02T23:23:52.978 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:23:52.978 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:23:52.978 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "ff6e8b63-dc3f-4561-ad74-ab0510544298", 2024-06-02T23:23:52.978 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:23:52.978 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:23:52.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19c40a2c20 msgr2=0x7f19c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19c40a2c20 0x7f19c40a50e0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f19c000b2e0 tx=0x7f19c001a040 comp rx=0 tx=0).stop 2024-06-02T23:23:52.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 msgr2=0x7f19d419ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:52.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d419ad20 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f19c800e9a0 tx=0x7f19c800ecb0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 shutdown_connections 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f19c40a2c20 0x7f19c40a50e0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f19d4103360 0x7f19d419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f19d4100a80 0x7f19d419ad20 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 --2- 172.21.15.52:0/1345234849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19d4101520 0x7f19d419b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 >> 172.21.15.52:0/1345234849 conn(0x7f19d4077a00 msgr2=0x7f19d40fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 shutdown_connections 2024-06-02T23:23:52.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:52.973+0000 7f19daebd700 1 -- 172.21.15.52:0/1345234849 wait complete. 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: pgmap v791: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 811 KiB/s wr, 92 op/s 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: pgmap v791: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 811 KiB/s wr, 92 op/s 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 236641 1.87 MB/sec execute 542 sec latency 30.895 ms 2024-06-02T23:23:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: pgmap v791: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 811 KiB/s wr, 92 op/s 2024-06-02T23:23:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 -- 172.21.15.52:0/993236006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3204101df0 msgr2=0x7f32041021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 --2- 172.21.15.52:0/993236006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3204101df0 0x7f32041021d0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f32000099c0 tx=0x7f3200009cd0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 -- 172.21.15.52:0/993236006 shutdown_connections 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 --2- 172.21.15.52:0/993236006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 0x7f320410e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 --2- 172.21.15.52:0/993236006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3204101df0 0x7f32041021d0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 --2- 172.21.15.52:0/993236006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32041027a0 0x7f3204106020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.158+0000 7f320bd65700 1 -- 172.21.15.52:0/993236006 >> 172.21.15.52:0/993236006 conn(0x7f3204077a30 msgr2=0x7f32040fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.159+0000 7f320bd65700 1 -- 172.21.15.52:0/993236006 shutdown_connections 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.159+0000 7f320bd65700 1 -- 172.21.15.52:0/993236006 wait complete. 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.159+0000 7f320bd65700 1 Processor -- start 2024-06-02T23:23:54.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.159+0000 7f320bd65700 1 -- start start 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320bd65700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 0x7f3204197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320bd65700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32041027a0 0x7f3204197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320bd65700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 0x7f3204198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320bd65700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3204111540 con 0x7f32041027a0 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320bd65700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32041113c0 con 0x7f3204101df0 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320bd65700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32041116c0 con 0x7f3204106750 2024-06-02T23:23:54.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 0x7f3204197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 0x7f3204197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39376/0 (socket says 172.21.15.52:39376) 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 -- 172.21.15.52:0/1708765686 learned_addr learned my addr 172.21.15.52:0/1708765686 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320a302700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 0x7f3204198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209300700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32041027a0 0x7f3204197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 -- 172.21.15.52:0/1708765686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 msgr2=0x7f3204198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 0x7f3204198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 -- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32041027a0 msgr2=0x7f3204197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32041027a0 0x7f3204197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209b01700 1 -- 172.21.15.52:0/1708765686 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32041a6f00 con 0x7f3204101df0 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f320a302700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 0x7f3204198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.160+0000 7f3209300700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32041027a0 0x7f3204197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.161+0000 7f3209b01700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 0x7f3204197690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3200006010 tx=0x7f3200004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:54.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.161+0000 7f31faffd700 1 -- 172.21.15.52:0/1708765686 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3200026070 con 0x7f3204101df0 2024-06-02T23:23:54.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.161+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32041a7190 con 0x7f3204101df0 2024-06-02T23:23:54.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.161+0000 7f31faffd700 1 -- 172.21.15.52:0/1708765686 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3200003c70 con 0x7f3204101df0 2024-06-02T23:23:54.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.161+0000 7f31faffd700 1 -- 172.21.15.52:0/1708765686 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320000fb90 con 0x7f3204101df0 2024-06-02T23:23:54.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.161+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32041a7640 con 0x7f3204101df0 2024-06-02T23:23:54.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.164+0000 7f31faffd700 1 -- 172.21.15.52:0/1708765686 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3200029070 con 0x7f3204101df0 2024-06-02T23:23:54.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.165+0000 7f31faffd700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f31f00a2d40 0x7f31f00a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.166+0000 7f31faffd700 1 -- 172.21.15.52:0/1708765686 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32000cf9e0 con 0x7f3204101df0 2024-06-02T23:23:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.166+0000 7f3209300700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f31f00a2d40 0x7f31f00a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.166+0000 7f32165e8b80 1 Processor -- start 2024-06-02T23:23:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f3209300700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f31f00a2d40 0x7f31f00a5200 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32041997f0 tx=0x7f31f4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 -- start start 2024-06-02T23:23:54.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x56335069b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x56335069c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56335058e100 con 0x56335069bcf0 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5633505ec9f0 con 0x5633506a1ec0 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f32165e8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563350554600 con 0x5633506a15a0 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f3209b01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f320a302700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x56335069c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f3209b01700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47176/0 (socket says 172.21.15.52:47176) 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f320a302700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x56335069c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49628/0 (socket says 172.21.15.52:49628) 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.167+0000 7f3209b01700 1 -- 172.21.15.52:0/2578888469 learned_addr learned my addr 172.21.15.52:0/2578888469 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209b01700 1 -- 172.21.15.52:0/2578888469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 msgr2=0x56335069b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209300700 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x56335069b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209b01700 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x56335069b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209b01700 1 -- 172.21.15.52:0/2578888469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 msgr2=0x56335069c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209b01700 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x56335069c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209b01700 1 -- 172.21.15.52:0/2578888469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56335053ba70 con 0x5633506a15a0 2024-06-02T23:23:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f320a302700 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x56335069c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f3209b01700 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a1980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32000a4950 tx=0x7f320009be60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d9d27215a74ca65 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f31effff700 1 -- 172.21.15.52:0/2578888469 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320009c9a0 con 0x5633506a15a0 2024-06-02T23:23:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f31effff700 1 -- 172.21.15.52:0/2578888469 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32000bae50 con 0x5633506a15a0 2024-06-02T23:23:54.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.168+0000 7f31effff700 1 -- 172.21.15.52:0/2578888469 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320009a630 con 0x5633506a15a0 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.169+0000 7f32165e8b80 1 -- 172.21.15.52:0/2578888469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 msgr2=0x5633506a1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.169+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a1980 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f32000a4950 tx=0x7f320009be60 comp rx=0 tx=0).stop 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.169+0000 7f32165e8b80 1 -- 172.21.15.52:0/2578888469 shutdown_connections 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.169+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x56335069c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.169+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a1980 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.169+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2578888469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x56335069b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.170+0000 7f32165e8b80 1 -- 172.21.15.52:0/2578888469 >> 172.21.15.52:0/2578888469 conn(0x5633505c8220 msgr2=0x563350696aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:54.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.170+0000 7f32165e8b80 1 -- 172.21.15.52:0/2578888469 shutdown_connections 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.170+0000 7f32165e8b80 1 -- 172.21.15.52:0/2578888469 wait complete. 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.170+0000 7f32165e8b80 1 -- start start 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.170+0000 7f32165e8b80 1 Processor -- start 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x5633506a7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x5633506a5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f32165e8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56335058d160 con 0x56335069bcf0 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f3209b01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x5633506a7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f3209b01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x5633506a7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49644/0 (socket says 172.21.15.52:49644) 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f320a302700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x5633506a5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f3209b01700 1 -- 172.21.15.52:0/2554906669 learned_addr learned my addr 172.21.15.52:0/2554906669 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:54.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5633505e89b0 con 0x5633506a1ec0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.171+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5633506a8650 con 0x5633506a15a0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209300700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f32165e8b80 10 client.0 _ll_register_callbacks cb 0x563350582750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209b01700 1 -- 172.21.15.52:0/2554906669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 msgr2=0x5633506a4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209b01700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209b01700 1 -- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 msgr2=0x5633506a5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209b01700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x5633506a5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209b01700 1 -- 172.21.15.52:0/2554906669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633505b28f0 con 0x56335069bcf0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f320a302700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x5633506a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f3209b01700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x5633506a7be0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f320000b580 tx=0x7f320009c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ee574fb99489d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.172+0000 7f31ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.173+0000 7f31ed7fa700 1 -- 172.21.15.52:0/2554906669 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32000a4e80 con 0x56335069bcf0 2024-06-02T23:23:54.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.173+0000 7f31ed7fa700 1 -- 172.21.15.52:0/2554906669 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32000a49f0 con 0x56335069bcf0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.173+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32000bae50 con 0x56335069bcf0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.173+0000 7f31ed7fa700 1 -- 172.21.15.52:0/2554906669 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f320007d070 con 0x56335069bcf0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.173+0000 7f32165e8b80 10 client.18861 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.173+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5633506b2a60 con 0x56335069bcf0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f31ed7fa700 1 -- 172.21.15.52:0/2554906669 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32000bae50 con 0x56335069bcf0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f31ed7fa700 1 -- 172.21.15.52:0/2554906669 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32000a0260 con 0x56335069bcf0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f32165e8b80 10 client.18861 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f32165e8b80 10 client.18861 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f32165e8b80 20 client.18861 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5633506b4f30 0x5633506b7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f32165e8b80 4 client.18861 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633505f9ef0 con 0x5633506b4f30 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.174+0000 7f3209300700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5633506b4f30 0x5633506b7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.175+0000 7f3209300700 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5633506b4f30 0x5633506b7320 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6411d9220f79e0d server_cookie=7499c6d965368738 in_seq=0 out_seq=0 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.175+0000 7f31ed7fa700 10 client.18861 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f31ed7fa700 1 -- 172.21.15.52:0/2554906669 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5633505f9ef0 con 0x5633506b4f30 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f31ed7fa700 10 client.18861 handle_command_reply: tid=0 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 client.18861 shutdown 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 msgr2=0x5633506a7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x5633506a7be0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f320000b580 tx=0x7f320009c370 comp rx=0 tx=0).stop 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 shutdown_connections 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5633506b4f30 0x5633506b7320 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506a1ec0 0x5633506a5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f31ed7fa700 0 client.18861 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56335069bcf0 0x5633506a7be0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2554906669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506a4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 >> 172.21.15.52:0/2554906669 conn(0x5633505c8220 msgr2=0x563350695c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:54.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.176+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 shutdown_connections 2024-06-02T23:23:54.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.177+0000 7f32165e8b80 1 -- 172.21.15.52:0/2554906669 wait complete. 2024-06-02T23:23:54.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.177+0000 7f32165e8b80 20 client.18861 trim_cache size 0 max 16384 2024-06-02T23:23:54.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 Processor -- start 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 -- start start 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a1ec0 0x563350696910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32041aeb90 con 0x5633506a1ec0 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32041b06d0 con 0x5633506bcae0 2024-06-02T23:23:54.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.201+0000 7f32165e8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32041af260 con 0x5633506a15a0 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506bcf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39410/0 (socket says 172.21.15.52:39410) 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 -- 172.21.15.52:0/2490408405 learned_addr learned my addr 172.21.15.52:0/2490408405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f3209300700 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a1ec0 0x563350696910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 -- 172.21.15.52:0/2490408405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 msgr2=0x5633506963d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506963d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 -- 172.21.15.52:0/2490408405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a1ec0 msgr2=0x563350696910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a1ec0 0x563350696910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 -- 172.21.15.52:0/2490408405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633506a57d0 con 0x5633506bcae0 2024-06-02T23:23:54.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f320a302700 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506bcf60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f31fc004540 tx=0x7f31fc009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c637b3dd4d2770b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f31ef7fe700 1 -- 172.21.15.52:0/2490408405 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00ea00 con 0x5633506bcae0 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.202+0000 7f31ef7fe700 1 -- 172.21.15.52:0/2490408405 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31fc00eba0 con 0x5633506bcae0 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f31ef7fe700 1 -- 172.21.15.52:0/2490408405 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0206a0 con 0x5633506bcae0 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 -- 172.21.15.52:0/2490408405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 msgr2=0x5633506bcf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506bcf60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f31fc004540 tx=0x7f31fc009920 comp rx=0 tx=0).stop 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 -- 172.21.15.52:0/2490408405 shutdown_connections 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506bcf60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a15a0 0x5633506963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 --2- 172.21.15.52:0/2490408405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a1ec0 0x563350696910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 -- 172.21.15.52:0/2490408405 >> 172.21.15.52:0/2490408405 conn(0x5633505c8220 msgr2=0x5633506a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 -- 172.21.15.52:0/2490408405 shutdown_connections 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.203+0000 7f32165e8b80 1 -- 172.21.15.52:0/2490408405 wait complete. 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 -- start start 2024-06-02T23:23:54.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 Processor -- start 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 0x5633506ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a1ec0 0x5633506aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32041b0000 con 0x5633506a15a0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32041af930 con 0x5633506bcae0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32041acb50 con 0x5633506a1ec0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f3209b01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 0x5633506ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f32165e8b80 10 client.0 _ll_register_callbacks cb 0x563350582750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f3209b01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 0x5633506ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49656/0 (socket says 172.21.15.52:49656) 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.204+0000 7f3209b01700 1 -- 172.21.15.52:0/747942429 learned_addr learned my addr 172.21.15.52:0/747942429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f320a302700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209300700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a1ec0 0x5633506aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209b01700 1 -- 172.21.15.52:0/747942429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a1ec0 msgr2=0x5633506aa870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209b01700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a1ec0 0x5633506aa870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209b01700 1 -- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 msgr2=0x5633506aadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209b01700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506aadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209b01700 1 -- 172.21.15.52:0/747942429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633506ab3a0 con 0x5633506a15a0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209300700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a1ec0 0x5633506aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f3209b01700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 0x5633506ade20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f320009a550 tx=0x7f32000092b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dedcadc458b93d64 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f31ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f31ed7fa700 1 -- 172.21.15.52:0/747942429 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f320009bde0 con 0x5633506a15a0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f31ed7fa700 1 -- 172.21.15.52:0/747942429 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32000a49f0 con 0x5633506a15a0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31fc00eba0 con 0x5633506a15a0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.205+0000 7f31ed7fa700 1 -- 172.21.15.52:0/747942429 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32000bae50 con 0x5633506a15a0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 10 client.18867 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5633506ab3a0 con 0x5633506a15a0 2024-06-02T23:23:54.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f31ed7fa700 1 -- 172.21.15.52:0/747942429 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f31fc00eba0 con 0x5633506a15a0 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f31ed7fa700 1 -- 172.21.15.52:0/747942429 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32000a0160 con 0x5633506a15a0 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 10 client.18867 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 10 client.18867 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 20 client.18867 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563350729430 0x56335072b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 4 client.18867 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.75, "await_for": 45.0}] 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.75, "await_for": 45.0}) -- 0x5633505f4470 con 0x563350729430 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.206+0000 7f3209300700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563350729430 0x56335072b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.207+0000 7f31ed7fa700 10 client.18867 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:54.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.207+0000 7f3209300700 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563350729430 0x56335072b850 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9d1dbf9fa3e463e server_cookie=e75406e95dd6f9c5 in_seq=0 out_seq=0 2024-06-02T23:23:54.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f31ed7fa700 1 -- 172.21.15.52:0/747942429 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5633505f4470 con 0x563350729430 2024-06-02T23:23:54.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f31ed7fa700 10 client.18867 handle_command_reply: tid=0 2024-06-02T23:23:54.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 client.18867 shutdown 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 msgr2=0x5633506ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 0x5633506ade20 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f320009a550 tx=0x7f32000092b0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 shutdown_connections 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563350729430 0x56335072b850 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5633506bcae0 0x5633506aadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5633506a15a0 0x5633506ade20 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f31ed7fa700 0 client.18867 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 --2- 172.21.15.52:0/747942429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5633506a1ec0 0x5633506aa870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.269+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 >> 172.21.15.52:0/747942429 conn(0x5633505c8220 msgr2=0x5633506a3f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 shutdown_connections 2024-06-02T23:23:54.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f32165e8b80 1 -- 172.21.15.52:0/747942429 wait complete. 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f32165e8b80 20 client.18867 trim_cache size 0 max 16384 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 42, 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: "f901e1b4": { 2024-06-02T23:23:54.271 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 42, 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 140.8, 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:23:54.272 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:23:54.273 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:54.273 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:54.273 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:54.273 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:54.273 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:54.273 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:23:54.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f31f00a2d40 msgr2=0x7f31f00a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f320bd65700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f31f00a2d40 0x7f31f00a5200 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32041997f0 tx=0x7f31f4009260 comp rx=0 tx=0).stop 2024-06-02T23:23:54.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 msgr2=0x7f3204197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.270+0000 7f320bd65700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 0x7f3204197690 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3200006010 tx=0x7f3200004270 comp rx=0 tx=0).stop 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 shutdown_connections 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f31f00a2d40 0x7f31f00a5200 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3204106750 0x7f3204198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3204101df0 0x7f3204197690 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 --2- 172.21.15.52:0/1708765686 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32041027a0 0x7f3204197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 >> 172.21.15.52:0/1708765686 conn(0x7f3204077a30 msgr2=0x7f3204104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 shutdown_connections 2024-06-02T23:23:54.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:54.271+0000 7f320bd65700 1 -- 172.21.15.52:0/1708765686 wait complete. 2024-06-02T23:23:54.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:23:54.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:23:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:54 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:23:54.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:23:54.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:23:54.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:23:54.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:54 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:23:54.652 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.88 MB/sec execute 543 sec latency 433.605 ms 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:54.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:23:54.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:54 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:23:54.766 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:23:55.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:55 smithi052 ceph-mon[32340]: pgmap v792: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 848 KiB/s wr, 102 op/s 2024-06-02T23:23:55.455 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f901e1b4', quiesce took 11.2 seconds. Will release after: 9.55 2024-06-02T23:23:55.513 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:23:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:55 smithi174 ceph-mon[35994]: pgmap v792: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 848 KiB/s wr, 102 op/s 2024-06-02T23:23:55.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.87 MB/sec execute 544 sec latency 1433.713 ms 2024-06-02T23:23:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:55 smithi123 ceph-mon[35977]: pgmap v792: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 159 KiB/s rd, 848 KiB/s wr, 102 op/s 2024-06-02T23:23:56.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.87 MB/sec execute 545 sec latency 2433.754 ms 2024-06-02T23:23:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.149+0000 7ff290666700 1 -- 172.21.15.52:0/2011804572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 msgr2=0x7ff28810f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.149+0000 7ff290666700 1 --2- 172.21.15.52:0/2011804572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff28810f330 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7ff28000b0a0 tx=0x7ff28000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 -- 172.21.15.52:0/2011804572 shutdown_connections 2024-06-02T23:23:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 --2- 172.21.15.52:0/2011804572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff28810f330 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 --2- 172.21.15.52:0/2011804572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 0x7ff2881088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 --2- 172.21.15.52:0/2011804572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff288102550 0x7ff2881029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 -- 172.21.15.52:0/2011804572 >> 172.21.15.52:0/2011804572 conn(0x7ff288079650 msgr2=0x7ff288079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 -- 172.21.15.52:0/2011804572 shutdown_connections 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 -- 172.21.15.52:0/2011804572 wait complete. 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 Processor -- start 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.150+0000 7ff290666700 1 -- start start 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff290666700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff288102550 0x7ff288074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff290666700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff288074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff290666700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 0x7ff288075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff290666700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff288112630 con 0x7ff288102f10 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff290666700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881124b0 con 0x7ff2881084f0 2024-06-02T23:23:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff290666700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881127b0 con 0x7ff288102550 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff288074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff288074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49680/0 (socket says 172.21.15.52:49680) 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 -- 172.21.15.52:0/2761382640 learned_addr learned my addr 172.21.15.52:0/2761382640 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28e402700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff288102550 0x7ff288074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 0x7ff288075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 -- 172.21.15.52:0/2761382640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff288102550 msgr2=0x7ff288074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff288102550 0x7ff288074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 -- 172.21.15.52:0/2761382640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 msgr2=0x7ff288075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 0x7ff288075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 -- 172.21.15.52:0/2761382640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2881a7020 con 0x7ff288102f10 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 0x7ff288075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.151+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff288074cd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d980 tx=0x7ff27800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.152+0000 7ff27f7fe700 1 -- 172.21.15.52:0/2761382640 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278017070 con 0x7ff288102f10 2024-06-02T23:23:57.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.152+0000 7ff27f7fe700 1 -- 172.21.15.52:0/2761382640 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff278004830 con 0x7ff288102f10 2024-06-02T23:23:57.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.152+0000 7ff27f7fe700 1 -- 172.21.15.52:0/2761382640 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278005440 con 0x7ff288102f10 2024-06-02T23:23:57.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.152+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2881a7310 con 0x7ff288102f10 2024-06-02T23:23:57.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.152+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2881a7800 con 0x7ff288102f10 2024-06-02T23:23:57.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.155+0000 7ff27f7fe700 1 -- 172.21.15.52:0/2761382640 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2780055e0 con 0x7ff288102f10 2024-06-02T23:23:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.155+0000 7ff27f7fe700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2740a2c20 0x7ff2740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.156+0000 7ff28e402700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2740a2c20 0x7ff2740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.156+0000 7ff27f7fe700 1 -- 172.21.15.52:0/2761382640 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff2780cef70 con 0x7ff288102f10 2024-06-02T23:23:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.156+0000 7ff28e402700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2740a2c20 0x7ff2740a50e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff284005950 tx=0x7ff2840058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.156+0000 7ff29aee9b80 1 Processor -- start 2024-06-02T23:23:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.156+0000 7ff29aee9b80 1 -- start start 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e13d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e137260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55654e029f60 con 0x55654e13cf60 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55654e088850 con 0x55654e1377a0 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55654dff0460 con 0x55654e13d880 2024-06-02T23:23:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e13d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e13d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49694/0 (socket says 172.21.15.52:49694) 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 -- 172.21.15.52:0/2303428613 learned_addr learned my addr 172.21.15.52:0/2303428613 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e137260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 -- 172.21.15.52:0/2303428613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 msgr2=0x55654e137260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e137260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 -- 172.21.15.52:0/2303428613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 msgr2=0x55654e137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e137bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 -- 172.21.15.52:0/2303428613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654dfd78d0 con 0x55654e13cf60 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.157+0000 7ff28e402700 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e13d340 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff28400fcd0 tx=0x7ff284012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44d4acd2ba14c64 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff27cff9700 1 -- 172.21.15.52:0/2303428613 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284022070 con 0x55654e13cf60 2024-06-02T23:23:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff27cff9700 1 -- 172.21.15.52:0/2303428613 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff284011c40 con 0x55654e13cf60 2024-06-02T23:23:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff27cff9700 1 -- 172.21.15.52:0/2303428613 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff28401d610 con 0x55654e13cf60 2024-06-02T23:23:57.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2303428613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 msgr2=0x55654e13d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e13d340 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7ff28400fcd0 tx=0x7ff284012000 comp rx=0 tx=0).stop 2024-06-02T23:23:57.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2303428613 shutdown_connections 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e13d340 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2303428613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e137260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2303428613 >> 172.21.15.52:0/2303428613 conn(0x55654e064080 msgr2=0x55654e132220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2303428613 shutdown_connections 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.158+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2303428613 wait complete. 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 -- start start 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 Processor -- start 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e1435c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13cf60 0x55654e1408c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 0x55654e140e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55654e028fc0 con 0x55654e13d880 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55654e084810 con 0x55654e1377a0 2024-06-02T23:23:57.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55654e144260 con 0x55654e13cf60 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff29aee9b80 10 client.0 _ll_register_callbacks cb 0x55654e01e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff28ec03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 0x55654e140e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff28ec03700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 0x55654e140e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49710/0 (socket says 172.21.15.52:49710) 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff28ec03700 1 -- 172.21.15.52:0/2136282364 learned_addr learned my addr 172.21.15.52:0/2136282364 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13cf60 0x55654e1408c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.159+0000 7ff28e402700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e1435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff28ec03700 1 -- 172.21.15.52:0/2136282364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13cf60 msgr2=0x55654e1408c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13cf60 0x55654e1408c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff28ec03700 1 -- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 msgr2=0x55654e1435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e1435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff28ec03700 1 -- 172.21.15.52:0/2136282364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654e04e750 con 0x55654e13d880 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 0x55654e140e00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff280015000 tx=0x7ff280004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfa60133f8b8894 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:57.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff271ffb700 1 -- 172.21.15.52:0/2136282364 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2800076e0 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff271ffb700 1 -- 172.21.15.52:0/2136282364 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff280007880 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff284011c40 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff271ffb700 1 -- 172.21.15.52:0/2136282364 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff280007a80 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff29aee9b80 10 client.18879 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55654e14e9f0 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.160+0000 7ff271ffb700 1 -- 172.21.15.52:0/2136282364 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff284011c40 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff271ffb700 1 -- 172.21.15.52:0/2136282364 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff280011070 con 0x55654e13d880 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff29aee9b80 10 client.18879 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff29aee9b80 10 client.18879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff29aee9b80 20 client.18879 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e151030 0x55654e153420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff29aee9b80 4 client.18879 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55654e095d50 con 0x55654e151030 2024-06-02T23:23:57.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.161+0000 7ff28e402700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e151030 0x55654e153420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff271ffb700 10 client.18879 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff28e402700 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e151030 0x55654e153420 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3278fd7180ab0e61 server_cookie=9e2a95dd0301d6aa in_seq=0 out_seq=0 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff271ffb700 1 -- 172.21.15.52:0/2136282364 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55654e095d50 con 0x55654e151030 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff271ffb700 10 client.18879 handle_command_reply: tid=0 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff29aee9b80 1 client.18879 shutdown 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 msgr2=0x55654e140e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.162+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 0x55654e140e00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7ff280015000 tx=0x7ff280004ce0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 shutdown_connections 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e151030 0x55654e153420 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13d880 0x55654e140e00 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e1377a0 0x55654e1435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2136282364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13cf60 0x55654e1408c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff271ffb700 0 client.18879 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 >> 172.21.15.52:0/2136282364 conn(0x55654e064080 msgr2=0x55654e131700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 shutdown_connections 2024-06-02T23:23:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2136282364 wait complete. 2024-06-02T23:23:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.163+0000 7ff29aee9b80 20 client.18879 trim_cache size 0 max 16384 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.183+0000 7ff29aee9b80 1 Processor -- start 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.183+0000 7ff29aee9b80 1 -- start start 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e1377a0 0x55654e140ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e1401e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13d880 0x55654e131e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881aebe0 con 0x55654e13cf60 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b0720 con 0x55654e13d880 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881af2b0 con 0x55654e1377a0 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e1401e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e1401e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49724/0 (socket says 172.21.15.52:49724) 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 -- 172.21.15.52:0/2867070270 learned_addr learned my addr 172.21.15.52:0/2867070270 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:57.184 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28e402700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e1377a0 0x55654e140ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13d880 0x55654e131e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 -- 172.21.15.52:0/2867070270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e1377a0 msgr2=0x55654e140ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e1377a0 0x55654e140ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 -- 172.21.15.52:0/2867070270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13d880 msgr2=0x55654e131e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13d880 0x55654e131e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 -- 172.21.15.52:0/2867070270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654e133040 con 0x55654e13cf60 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13d880 0x55654e131e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28e402700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e1377a0 0x55654e140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e1401e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff278097c60 tx=0x7ff27809b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e70bfac7eae14d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff273fff700 1 -- 172.21.15.52:0/2867070270 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2780b4070 con 0x55654e13cf60 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff273fff700 1 -- 172.21.15.52:0/2867070270 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff27809b800 con 0x55654e13cf60 2024-06-02T23:23:57.185 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.184+0000 7ff273fff700 1 -- 172.21.15.52:0/2867070270 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27809ba60 con 0x55654e13cf60 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2867070270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 msgr2=0x55654e1401e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e1401e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff278097c60 tx=0x7ff27809b230 comp rx=0 tx=0).stop 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2867070270 shutdown_connections 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13d880 0x55654e131e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e1377a0 0x55654e140ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2867070270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e13cf60 0x55654e1401e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2867070270 >> 172.21.15.52:0/2867070270 conn(0x55654e064080 msgr2=0x55654e142980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2867070270 shutdown_connections 2024-06-02T23:23:57.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.185+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2867070270 wait complete. 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 -- start start 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 Processor -- start 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 0x55654e149550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13cf60 0x55654e149a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e146520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881b0050 con 0x55654e1377a0 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881af980 con 0x55654e13cf60 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2881acb80 con 0x55654e13d880 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff29aee9b80 10 client.0 _ll_register_callbacks cb 0x55654e01e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 0x55654e149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 0x55654e149550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49738/0 (socket says 172.21.15.52:49738) 2024-06-02T23:23:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 -- 172.21.15.52:0/2540045391 learned_addr learned my addr 172.21.15.52:0/2540045391 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13cf60 0x55654e149a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e146520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 -- 172.21.15.52:0/2540045391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 msgr2=0x55654e146520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e146520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 -- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13cf60 msgr2=0x55654e149a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13cf60 0x55654e149a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28e402700 1 -- 172.21.15.52:0/2540045391 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654e146a60 con 0x55654e1377a0 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13cf60 0x55654e149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.186+0000 7ff28ec03700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff28e402700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 0x55654e149550 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff28400fc70 tx=0x7ff284011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a498c71bc3af07 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff271ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff271ffb700 1 -- 172.21.15.52:0/2540045391 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff284022040 con 0x55654e1377a0 2024-06-02T23:23:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff271ffb700 1 -- 172.21.15.52:0/2540045391 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff284005030 con 0x55654e1377a0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff27809b800 con 0x55654e1377a0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff271ffb700 1 -- 172.21.15.52:0/2540045391 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff284044050 con 0x55654e1377a0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 10 client.18885 fetch_fsmap learned FSMap version 32 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55654e1bc450 con 0x55654e1377a0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff271ffb700 1 -- 172.21.15.52:0/2540045391 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff27809b800 con 0x55654e1377a0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff271ffb700 1 -- 172.21.15.52:0/2540045391 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7ff28401e450 con 0x55654e1377a0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 10 client.18885 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 10 client.18885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 20 client.18885 populate_metadata read hostname 'smithi052' 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e1bc830 0x55654e1bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 4 client.18885 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.187+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55654e0902d0 con 0x55654e1bc830 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e1bc830 0x55654e1bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:23:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff28dc01700 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e1bc830 0x55654e1bec50 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b754d62d6846236 server_cookie=f9ca4825b7681f58 in_seq=0 out_seq=0 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff271ffb700 10 client.18885 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff271ffb700 1 -- 172.21.15.52:0/2540045391 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55654e0902d0 con 0x55654e1bc830 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff271ffb700 10 client.18885 handle_command_reply: tid=0 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 client.18885 shutdown 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 msgr2=0x55654e149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 0x55654e149550 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7ff28400fc70 tx=0x7ff284011b20 comp rx=0 tx=0).stop 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 shutdown_connections 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654e1bc830 0x55654e1bec50 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654e13d880 0x55654e146520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654e1377a0 0x55654e149550 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff271ffb700 0 client.18885 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 --2- 172.21.15.52:0/2540045391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654e13cf60 0x55654e149a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 >> 172.21.15.52:0/2540045391 conn(0x55654e064080 msgr2=0x55654e1420f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 shutdown_connections 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.188+0000 7ff29aee9b80 1 -- 172.21.15.52:0/2540045391 wait complete. 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff29aee9b80 20 client.18885 trim_cache size 0 max 16384 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stdout: "ff6e8b63-dc3f-4561-ad74-ab0510544298": { 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:23:57.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "ff6e8b63-dc3f-4561-ad74-ab0510544298", 2024-06-02T23:23:57.195 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:23:57.195 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:57.195 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:23:57.195 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:23:57.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2740a2c20 msgr2=0x7ff2740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2740a2c20 0x7ff2740a50e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff284005950 tx=0x7ff2840058e0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 msgr2=0x7ff288074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff288074cd0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7ff27800d980 tx=0x7ff27800dd40 comp rx=0 tx=0).stop 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 shutdown_connections 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2740a2c20 0x7ff2740a50e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2881084f0 0x7ff288075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff288102550 0x7ff288074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 --2- 172.21.15.52:0/2761382640 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff288102f10 0x7ff288074cd0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 >> 172.21.15.52:0/2761382640 conn(0x7ff288079650 msgr2=0x7ff288100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 shutdown_connections 2024-06-02T23:23:57.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:23:57.189+0000 7ff290666700 1 -- 172.21.15.52:0/2761382640 wait complete. 2024-06-02T23:23:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:57 smithi174 ceph-mon[35994]: pgmap v793: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-02T23:23:57.668 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:57 smithi052 ceph-mon[32340]: pgmap v793: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-02T23:23:57.668 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.87 MB/sec execute 546 sec latency 3433.812 ms 2024-06-02T23:23:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:57 smithi123 ceph-mon[35977]: pgmap v793: 97 pgs: 97 active+clean; 97 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.1 MiB/s wr, 152 op/s 2024-06-02T23:23:57.909 INFO:tasks.cephfs.filesystem:scrub status for tag:ff6e8b63-dc3f-4561-ad74-ab0510544298 - {'path': '/', 'tag': 'ff6e8b63-dc3f-4561-ad74-ab0510544298', 'options': 'recursive,force'} 2024-06-02T23:23:58.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.86 MB/sec execute 547 sec latency 4433.885 ms 2024-06-02T23:23:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:23:59 smithi174 ceph-mon[35994]: pgmap v794: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:23:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:23:59 smithi052 ceph-mon[32340]: pgmap v794: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:23:59.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.86 MB/sec execute 548 sec latency 5433.942 ms 2024-06-02T23:23:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:23:59 smithi123 ceph-mon[35977]: pgmap v794: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 1.1 MiB/s wr, 135 op/s 2024-06-02T23:24:00.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.86 MB/sec execute 549 sec latency 6434.003 ms 2024-06-02T23:24:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:01 smithi174 ceph-mon[35994]: pgmap v795: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 938 KiB/s wr, 136 op/s 2024-06-02T23:24:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:24:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:24:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:24:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:24:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:01 smithi052 ceph-mon[32340]: pgmap v795: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 938 KiB/s wr, 136 op/s 2024-06-02T23:24:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:24:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:24:01.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:24:01.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:24:01.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.85 MB/sec execute 550 sec latency 7434.040 ms 2024-06-02T23:24:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:01 smithi123 ceph-mon[35977]: pgmap v795: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 938 KiB/s wr, 136 op/s 2024-06-02T23:24:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:24:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:24:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:24:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:24:02.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.85 MB/sec execute 551 sec latency 8434.141 ms 2024-06-02T23:24:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 -- 172.21.15.52:0/1862717518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 msgr2=0x7f1f74109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 --2- 172.21.15.52:0/1862717518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f74109b30 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009ca0 tx=0x7f1f6c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 -- 172.21.15.52:0/1862717518 shutdown_connections 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 --2- 172.21.15.52:0/1862717518 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f74109b30 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 --2- 172.21.15.52:0/1862717518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f74104db0 0x7f1f741071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 --2- 172.21.15.52:0/1862717518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f74102480 0x7f1f74104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 -- 172.21.15.52:0/1862717518 >> 172.21.15.52:0/1862717518 conn(0x7f1f740fbe90 msgr2=0x7f1f740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 -- 172.21.15.52:0/1862717518 shutdown_connections 2024-06-02T23:24:03.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 -- 172.21.15.52:0/1862717518 wait complete. 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.422+0000 7f1f7902a640 1 Processor -- start 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 -- start start 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f74102480 0x7f1f74197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f74104db0 0x7f1f74198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f7419f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7410d570 con 0x7f1f741076e0 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7410d3f0 con 0x7f1f74102480 2024-06-02T23:24:03.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7902a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f7410d6f0 con 0x7f1f74104db0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f7419f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f7419f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49752/0 (socket says 172.21.15.52:49752) 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7359e640 1 -- 172.21.15.52:0/123448857 learned_addr learned my addr 172.21.15.52:0/123448857 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7259c640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f74104db0 0x7f1f74198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f74102480 0x7f1f74197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7359e640 1 -- 172.21.15.52:0/123448857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f74104db0 msgr2=0x7f1f74198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.423+0000 7f1f7359e640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f74104db0 0x7f1f74198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f7359e640 1 -- 172.21.15.52:0/123448857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f74102480 msgr2=0x7f1f74197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f7359e640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f74102480 0x7f1f74197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f7359e640 1 -- 172.21.15.52:0/123448857 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f7419f750 con 0x7f1f741076e0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f7359e640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f7419f0e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009dc0 tx=0x7f1f6c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f5bfff640 1 -- 172.21.15.52:0/123448857 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c032ec0 con 0x7f1f741076e0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f5bfff640 1 -- 172.21.15.52:0/123448857 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f6c0044f0 con 0x7f1f741076e0 2024-06-02T23:24:03.425 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f5bfff640 1 -- 172.21.15.52:0/123448857 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c030d40 con 0x7f1f741076e0 2024-06-02T23:24:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f7419f9e0 con 0x7f1f741076e0 2024-06-02T23:24:03.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.424+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f7419ff30 con 0x7f1f741076e0 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.427+0000 7f1f5bfff640 1 -- 172.21.15.52:0/123448857 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1f6c004040 con 0x7f1f741076e0 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.427+0000 7f1f7be15740 1 Processor -- start 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.427+0000 7f1f7be15740 1 -- start start 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31010 0x561535c31430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c36500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c36b70 0x561535c39000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7be15740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561535b944f0 con 0x561535c31a00 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7be15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561535b32870 con 0x561535c36b70 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7be15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561535b904b0 con 0x561535c31010 2024-06-02T23:24:03.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c36500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c36500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49762/0 (socket says 172.21.15.52:49762) 2024-06-02T23:24:03.428 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1587123258 learned_addr learned my addr 172.21.15.52:0/1587123258 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f5bfff640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f5c0a2e50 0x7f1f5c0a5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7259c640 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c36b70 0x561535c39000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1587123258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31010 msgr2=0x561535c31430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31010 0x561535c31430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1587123258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c36b70 msgr2=0x561535c39000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c36b70 0x561535c39000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1587123258 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561535b4e9b0 con 0x561535c31a00 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7259c640 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c36b70 0x561535c39000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f7359e640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f5c0a2e50 0x7f1f5c0a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c36500 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400ac10 tx=0x7f1f6400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e81d96ec3cf1538 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.428+0000 7f1f5bfff640 1 -- 172.21.15.52:0/123448857 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1f6c0f0520 con 0x7f1f741076e0 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f597fa640 1 -- 172.21.15.52:0/1587123258 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6401d070 con 0x561535c31a00 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f597fa640 1 -- 172.21.15.52:0/1587123258 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f64018af0 con 0x561535c31a00 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f597fa640 1 -- 172.21.15.52:0/1587123258 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f64010a20 con 0x561535c31a00 2024-06-02T23:24:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 -- 172.21.15.52:0/1587123258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 msgr2=0x561535c36500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c36500 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f1f6400ac10 tx=0x7f1f6400f790 comp rx=0 tx=0).stop 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7359e640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f5c0a2e50 0x7f1f5c0a5310 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0bcce0 tx=0x7f1f6c08cd20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 -- 172.21.15.52:0/1587123258 shutdown_connections 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c36b70 0x561535c39000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c36500 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1587123258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31010 0x561535c31430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 -- 172.21.15.52:0/1587123258 >> 172.21.15.52:0/1587123258 conn(0x561535c2c680 msgr2=0x561535c2ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 -- 172.21.15.52:0/1587123258 shutdown_connections 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.429+0000 7f1f7be15740 1 -- 172.21.15.52:0/1587123258 wait complete. 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 -- start start 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 Processor -- start 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 0x561535c3b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31a00 0x561535c3b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535c42890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561535c39b20 con 0x561535c31010 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561535b1dae0 con 0x561535c31a00 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561535c39ca0 con 0x561535c36b70 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7be15740 10 client.0 _ll_register_callbacks cb 0x561535b569b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 0x561535c3b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 0x561535c3b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49776/0 (socket says 172.21.15.52:49776) 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 -- 172.21.15.52:0/2145249828 learned_addr learned my addr 172.21.15.52:0/2145249828 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31a00 0x561535c3b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535c42890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 -- 172.21.15.52:0/2145249828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 msgr2=0x561535c42890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535c42890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 -- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31a00 msgr2=0x561535c3b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31a00 0x561535c3b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.430+0000 7f1f7259c640 1 -- 172.21.15.52:0/2145249828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615358a8210 con 0x561535c31010 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31a00 0x561535c3b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 0x561535c3b310 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bb10 tx=0x7f1f68004110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c05cbc990f8b8a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f56ff5640 1 -- 172.21.15.52:0/2145249828 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68005220 con 0x561535c31010 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f56ff5640 1 -- 172.21.15.52:0/2145249828 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1f680053c0 con 0x561535c31010 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f64018af0 con 0x561535c31010 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f56ff5640 1 -- 172.21.15.52:0/2145249828 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f68003e50 con 0x561535c31010 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f7be15740 10 client.18897 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561535974620 con 0x561535c31010 2024-06-02T23:24:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f56ff5640 1 -- 172.21.15.52:0/2145249828 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f64018af0 con 0x561535c31010 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f56ff5640 1 -- 172.21.15.52:0/2145249828 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1f68011840 con 0x561535c31010 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f7be15740 10 client.18897 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.431+0000 7f1f7be15740 10 client.18897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f7be15740 20 client.18897 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535c47140 0x561535c49550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f7be15740 4 client.18897 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561535ba7f10 con 0x561535c47140 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535c47140 0x561535c49550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535c47140 0x561535c49550 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2c896cfd32accb4 server_cookie=c8cb0d7d729c7d3b in_seq=0 out_seq=0 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.432+0000 7f1f56ff5640 10 client.18897 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f56ff5640 1 -- 172.21.15.52:0/2145249828 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561535ba7f10 con 0x561535c47140 2024-06-02T23:24:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f56ff5640 10 client.18897 handle_command_reply: tid=0 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 client.18897 shutdown 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 msgr2=0x561535c3b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 0x561535c3b310 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f1f6800bb10 tx=0x7f1f68004110 comp rx=0 tx=0).stop 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 shutdown_connections 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535c47140 0x561535c49550 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535c42890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31a00 0x561535c3b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2145249828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31010 0x561535c3b310 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f56ff5640 0 client.18897 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 >> 172.21.15.52:0/2145249828 conn(0x561535c2c680 msgr2=0x561535c2b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 shutdown_connections 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.433+0000 7f1f7be15740 1 -- 172.21.15.52:0/2145249828 wait complete. 2024-06-02T23:24:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.434+0000 7f1f7be15740 20 client.18897 trim_cache size 0 max 16384 2024-06-02T23:24:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 Processor -- start 2024-06-02T23:24:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 -- start start 2024-06-02T23:24:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c2bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31a00 0x561535c42850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 0x561535c3af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741a8c00 con 0x561535c36b70 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741aa740 con 0x561535c31010 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7be15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741a92d0 con 0x561535c31a00 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 0x561535c3af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 0x561535c3af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49784/0 (socket says 172.21.15.52:49784) 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 -- 172.21.15.52:0/2449634139 learned_addr learned my addr 172.21.15.52:0/2449634139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31a00 0x561535c42850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c2bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 -- 172.21.15.52:0/2449634139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31a00 msgr2=0x561535c42850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31a00 0x561535c42850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 -- 172.21.15.52:0/2449634139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 msgr2=0x561535c2bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c2bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.449+0000 7f1f7259c640 1 -- 172.21.15.52:0/2449634139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561535c3a3d0 con 0x561535c36b70 2024-06-02T23:24:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7259c640 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 0x561535c3af20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1f68004650 tx=0x7f1f68000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f1c8c4fba39061 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f58ff9640 1 -- 172.21.15.52:0/2449634139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f680055c0 con 0x561535c36b70 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f58ff9640 1 -- 172.21.15.52:0/2449634139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f68011920 con 0x561535c36b70 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f58ff9640 1 -- 172.21.15.52:0/2449634139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f68004d90 con 0x561535c36b70 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 -- 172.21.15.52:0/2449634139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 msgr2=0x561535c3af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 0x561535c3af20 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f1f68004650 tx=0x7f1f68000bc0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 -- 172.21.15.52:0/2449634139 shutdown_connections 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c31a00 0x561535c42850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c36b70 0x561535c3af20 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 --2- 172.21.15.52:0/2449634139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c2bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.450+0000 7f1f7be15740 1 -- 172.21.15.52:0/2449634139 >> 172.21.15.52:0/2449634139 conn(0x561535c2c680 msgr2=0x561535c2ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 -- 172.21.15.52:0/2449634139 shutdown_connections 2024-06-02T23:24:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 -- 172.21.15.52:0/2449634139 wait complete. 2024-06-02T23:24:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 -- start start 2024-06-02T23:24:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 Processor -- start 2024-06-02T23:24:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c3f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c3f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535cafe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741aa070 con 0x561535c31a00 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741a99a0 con 0x561535c31010 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1f741a6be0 con 0x561535c36b70 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7be15740 10 client.0 _ll_register_callbacks cb 0x7f1f68004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c3f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f72d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c3f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49800/0 (socket says 172.21.15.52:49800) 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1127360150 learned_addr learned my addr 172.21.15.52:0/1127360150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7259c640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535cafe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.451+0000 7f1f7259c640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c3f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1127360150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 msgr2=0x561535cafe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535cafe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 msgr2=0x561535c3f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c3f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f72d9d640 1 -- 172.21.15.52:0/1127360150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561535c42980 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f72d9d640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c3f8b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1f640058c0 tx=0x7f1f6400a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc3e34cedf27731 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f56ff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f56ff5640 1 -- 172.21.15.52:0/1127360150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f6401d040 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f56ff5640 1 -- 172.21.15.52:0/1127360150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1f64018880 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f68011920 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f56ff5640 1 -- 172.21.15.52:0/1127360150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f1f64018a80 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f7be15740 10 client.18903 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x561535cb1250 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f56ff5640 1 -- 172.21.15.52:0/1127360150 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1f68011920 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.452+0000 7f1f56ff5640 1 -- 172.21.15.52:0/1127360150 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f1f64004aa0 con 0x561535c31a00 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7be15740 10 client.18903 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7be15740 10 client.18903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:03.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7be15740 20 client.18903 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535cb1a90 0x561535cb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7be15740 4 client.18903 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:24:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561535ba1c20 con 0x561535cb1a90 2024-06-02T23:24:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7259c640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535cb1a90 0x561535cb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f7259c640 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535cb1a90 0x561535cb3eb0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2723dba129c18afc server_cookie=51340d319bcd7eb3 in_seq=0 out_seq=0 2024-06-02T23:24:03.458 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.453+0000 7f1f56ff5640 10 client.18903 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f56ff5640 1 -- 172.21.15.52:0/1127360150 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561535ba1c20 con 0x561535cb1a90 2024-06-02T23:24:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f56ff5640 10 client.18903 handle_command_reply: tid=0 2024-06-02T23:24:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f7be15740 1 client.18903 shutdown 2024-06-02T23:24:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 msgr2=0x561535c3f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c3f8b0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f1f640058c0 tx=0x7f1f6400a6c0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 shutdown_connections 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.474+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561535cb1a90 0x561535cb3eb0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561535c36b70 0x561535cafe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561535c31a00 0x561535c3f8b0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 1 --2- 172.21.15.52:0/1127360150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561535c31010 0x561535c3f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f56ff5640 0 client.18903 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 >> 172.21.15.52:0/1127360150 conn(0x561535c2c680 msgr2=0x561535c3af20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 shutdown_connections 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 1 -- 172.21.15.52:0/1127360150 wait complete. 2024-06-02T23:24:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7be15740 20 client.18903 trim_cache size 0 max 16384 2024-06-02T23:24:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:24:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:24:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:24:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f5c0a2e50 msgr2=0x7f1f5c0a5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f5c0a2e50 0x7f1f5c0a5310 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c0bcce0 tx=0x7f1f6c08cd20 comp rx=0 tx=0).stop 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 msgr2=0x7f1f7419f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f7419f0e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009dc0 tx=0x7f1f6c003ba0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 shutdown_connections 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1f5c0a2e50 0x7f1f5c0a5310 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1f74104db0 0x7f1f74198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1f741076e0 0x7f1f7419f0e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 --2- 172.21.15.52:0/123448857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f74102480 0x7f1f74197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 >> 172.21.15.52:0/123448857 conn(0x7f1f740fbe90 msgr2=0x7f1f74102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.475+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 shutdown_connections 2024-06-02T23:24:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:03.476+0000 7f1f7902a640 1 -- 172.21.15.52:0/123448857 wait complete. 2024-06-02T23:24:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:03 smithi174 ceph-mon[35994]: pgmap v796: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 658 KiB/s wr, 104 op/s 2024-06-02T23:24:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:03 smithi052 ceph-mon[32340]: pgmap v796: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 658 KiB/s wr, 104 op/s 2024-06-02T23:24:03.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.85 MB/sec execute 552 sec latency 9434.230 ms 2024-06-02T23:24:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:03 smithi123 ceph-mon[35977]: pgmap v796: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 658 KiB/s wr, 104 op/s 2024-06-02T23:24:04.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.84 MB/sec execute 553 sec latency 10434.354 ms 2024-06-02T23:24:05.007 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:24:05.270 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:05 smithi174 ceph-mon[35994]: pgmap v797: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 81 op/s 2024-06-02T23:24:05.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:05 smithi052 ceph-mon[32340]: pgmap v797: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 81 op/s 2024-06-02T23:24:05.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.84 MB/sec execute 554 sec latency 11434.412 ms 2024-06-02T23:24:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:05 smithi123 ceph-mon[35977]: pgmap v797: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 444 KiB/s wr, 81 op/s 2024-06-02T23:24:06.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.84 MB/sec execute 555 sec latency 12434.510 ms 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 -- 172.21.15.52:0/3344198629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 msgr2=0x7f40a4102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 --2- 172.21.15.52:0/3344198629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4102b20 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f40940099c0 tx=0x7f4094009cd0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 -- 172.21.15.52:0/3344198629 shutdown_connections 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 --2- 172.21.15.52:0/3344198629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a4107840 0x7f40a410f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 --2- 172.21.15.52:0/3344198629 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4102b20 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 --2- 172.21.15.52:0/3344198629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a41030f0 0x7f40a4107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 -- 172.21.15.52:0/3344198629 >> 172.21.15.52:0/3344198629 conn(0x7f40a40795f0 msgr2=0x7f40a4079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 -- 172.21.15.52:0/3344198629 shutdown_connections 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.074+0000 7f40a94e2700 1 -- 172.21.15.52:0/3344198629 wait complete. 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.075+0000 7f40a94e2700 1 Processor -- start 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.075+0000 7f40a94e2700 1 -- start start 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.075+0000 7f40a94e2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.075+0000 7f40a94e2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a41030f0 0x7f40a4074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.075+0000 7f40a94e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a4107840 0x7f40a4075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a94e2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4112610 con 0x7f40a4102740 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a94e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4112490 con 0x7f40a4107840 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a94e2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40a4112790 con 0x7f40a41030f0 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58972/0 (socket says 172.21.15.52:58972) 2024-06-02T23:24:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 -- 172.21.15.52:0/2133911920 learned_addr learned my addr 172.21.15.52:0/2133911920 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a37fe700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a4107840 0x7f40a4075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a27fc700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a41030f0 0x7f40a4074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 -- 172.21.15.52:0/2133911920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a41030f0 msgr2=0x7f40a4074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a41030f0 0x7f40a4074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 -- 172.21.15.52:0/2133911920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a4107840 msgr2=0x7f40a4075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a4107840 0x7f40a4075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 -- 172.21.15.52:0/2133911920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40a41a6fe0 con 0x7f40a4102740 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a27fc700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a41030f0 0x7f40a4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a2ffd700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4074790 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f40940059d0 tx=0x7f40940057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094027070 con 0x7f40a4102740 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40a41a7270 con 0x7f40a4102740 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4094003f50 con 0x7f40a4102740 2024-06-02T23:24:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4094022d80 con 0x7f40a4102740 2024-06-02T23:24:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.076+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40a41a76d0 con 0x7f40a4102740 2024-06-02T23:24:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.077+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a410aa80 con 0x7f40a4102740 2024-06-02T23:24:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.079+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f409402a070 con 0x7f40a4102740 2024-06-02T23:24:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.080+0000 7f408bfff700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f408c0a2c20 0x7f408c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.080+0000 7f40a27fc700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f408c0a2c20 0x7f408c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.080+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40940d0810 con 0x7f40a4102740 2024-06-02T23:24:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.080+0000 7f40a27fc700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f408c0a2c20 0x7f408c0a50e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4098005fd0 tx=0x7f4098005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.084+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4094013070 con 0x7f40a4102740 2024-06-02T23:24:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.336+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40a4075c70 con 0x7f40a4102740 2024-06-02T23:24:07.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.337+0000 7f408bfff700 1 -- 172.21.15.52:0/2133911920 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7f409409d7d0 con 0x7f40a4102740 2024-06-02T23:24:07.338 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:24:07.338 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f408c0a2c20 msgr2=0x7f408c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f408c0a2c20 0x7f408c0a50e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f4098005fd0 tx=0x7f4098005ee0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 msgr2=0x7f40a4074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4074790 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f40940059d0 tx=0x7f40940057c0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 shutdown_connections 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f408c0a2c20 0x7f408c0a50e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40a4107840 0x7f40a4075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40a4102740 0x7f40a4074790 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 --2- 172.21.15.52:0/2133911920 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40a41030f0 0x7f40a4074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 >> 172.21.15.52:0/2133911920 conn(0x7f40a40795f0 msgr2=0x7f40a41007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 shutdown_connections 2024-06-02T23:24:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:07.340+0000 7f40a94e2700 1 -- 172.21.15.52:0/2133911920 wait complete. 2024-06-02T23:24:07.342 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:24:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:07 smithi174 ceph-mon[35994]: pgmap v798: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 410 KiB/s wr, 73 op/s 2024-06-02T23:24:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:07 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2133911920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:24:07.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:07 smithi052 ceph-mon[32340]: pgmap v798: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 410 KiB/s wr, 73 op/s 2024-06-02T23:24:07.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:07 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2133911920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:24:07.653 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.83 MB/sec execute 556 sec latency 13434.592 ms 2024-06-02T23:24:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:07 smithi123 ceph-mon[35977]: pgmap v798: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 410 KiB/s wr, 73 op/s 2024-06-02T23:24:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:07 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2133911920' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:24:08.028 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id f901e1b4 --release --await' 2024-06-02T23:24:08.028 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id f901e1b4 --release --await 2024-06-02T23:24:08.281 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:08.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.83 MB/sec execute 557 sec latency 14434.622 ms 2024-06-02T23:24:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:09 smithi174 ceph-mon[35994]: pgmap v799: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 102 KiB/s wr, 24 op/s 2024-06-02T23:24:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:09 smithi052 ceph-mon[32340]: pgmap v799: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 102 KiB/s wr, 24 op/s 2024-06-02T23:24:09.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237043 1.83 MB/sec execute 558 sec latency 15434.749 ms 2024-06-02T23:24:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:09 smithi123 ceph-mon[35977]: pgmap v799: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 102 KiB/s wr, 24 op/s 2024-06-02T23:24:10.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 -- 172.21.15.52:0/3212032586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33781030f0 msgr2=0x7f3378107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 --2- 172.21.15.52:0/3212032586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33781030f0 0x7f3378107110 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f33680099c0 tx=0x7f3368009cd0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 -- 172.21.15.52:0/3212032586 shutdown_connections 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 --2- 172.21.15.52:0/3212032586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3378107840 0x7f337810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 --2- 172.21.15.52:0/3212032586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3378102740 0x7f3378102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 --2- 172.21.15.52:0/3212032586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33781030f0 0x7f3378107110 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 -- 172.21.15.52:0/3212032586 >> 172.21.15.52:0/3212032586 conn(0x7f33780795f0 msgr2=0x7f3378079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 -- 172.21.15.52:0/3212032586 shutdown_connections 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.116+0000 7f337c9ec700 1 -- 172.21.15.52:0/3212032586 wait complete. 2024-06-02T23:24:10.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 Processor -- start 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 -- start start 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3378102740 0x7f337819e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33781030f0 0x7f337819ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 0x7f33781988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3378112620 con 0x7f3378107840 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33781124a0 con 0x7f3378102740 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337c9ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33781127a0 con 0x7f33781030f0 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f3376d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 0x7f33781988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f3376d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 0x7f33781988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58980/0 (socket says 172.21.15.52:58980) 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f3376d9d700 1 -- 172.21.15.52:0/1630589981 learned_addr learned my addr 172.21.15.52:0/1630589981 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f3375d9b700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33781030f0 0x7f337819ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.117+0000 7f337659c700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3378102740 0x7f337819e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f3376d9d700 1 -- 172.21.15.52:0/1630589981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33781030f0 msgr2=0x7f337819ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f3376d9d700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33781030f0 0x7f337819ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f3376d9d700 1 -- 172.21.15.52:0/1630589981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3378102740 msgr2=0x7f337819e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f3376d9d700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3378102740 0x7f337819e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f3376d9d700 1 -- 172.21.15.52:0/1630589981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3378199040 con 0x7f3378107840 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f3376d9d700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 0x7f33781988e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3370007ee0 tx=0x7f337000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f336f7fe700 1 -- 172.21.15.52:0/1630589981 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3370004bb0 con 0x7f3378107840 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f336f7fe700 1 -- 172.21.15.52:0/1630589981 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3370004d50 con 0x7f3378107840 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f336f7fe700 1 -- 172.21.15.52:0/1630589981 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700058e0 con 0x7f3378107840 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3378199330 con 0x7f3378107840 2024-06-02T23:24:10.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.118+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33781a55f0 con 0x7f3378107840 2024-06-02T23:24:10.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.120+0000 7f336f7fe700 1 -- 172.21.15.52:0/1630589981 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f337001a020 con 0x7f3378107840 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.121+0000 7f336f7fe700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33640a2ca0 0x7f33640a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.121+0000 7f337659c700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33640a2ca0 0x7f33640a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.121+0000 7f336f7fe700 1 -- 172.21.15.52:0/1630589981 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33700ce670 con 0x7f3378107840 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.121+0000 7f337659c700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33640a2ca0 0x7f33640a5160 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3360005950 tx=0x7f336000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 Processor -- start 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 -- start start 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d25020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b266c70910 con 0x55b266d1f3d0 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b266c18820 con 0x55b266d24c40 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.122+0000 7f338726fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b266c6c8d0 con 0x55b266d25560 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f337659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d25020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58992/0 (socket says 172.21.15.52:58992) 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 -- 172.21.15.52:0/170902861 learned_addr learned my addr 172.21.15.52:0/170902861 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3375d9b700 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 -- 172.21.15.52:0/170902861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 msgr2=0x55b266d1ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d1ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 -- 172.21.15.52:0/170902861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 msgr2=0x55b266d25020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d25020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 -- 172.21.15.52:0/170902861 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b266c0f050 con 0x55b266d1f3d0 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f337659c700 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d25020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f3376d9d700 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1f810 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f337009ae90 tx=0x7f337009ade0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=689963b1ba5c5958 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f336cff9700 1 -- 172.21.15.52:0/170902861 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337009b600 con 0x55b266d1f3d0 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f336cff9700 1 -- 172.21.15.52:0/170902861 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3370005a80 con 0x55b266d1f3d0 2024-06-02T23:24:10.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.123+0000 7f336cff9700 1 -- 172.21.15.52:0/170902861 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337009b3c0 con 0x55b266d1f3d0 2024-06-02T23:24:10.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 -- 172.21.15.52:0/170902861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 msgr2=0x55b266d1f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1f810 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f337009ae90 tx=0x7f337009ade0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 -- 172.21.15.52:0/170902861 shutdown_connections 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1f810 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d25020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 --2- 172.21.15.52:0/170902861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d1ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 -- 172.21.15.52:0/170902861 >> 172.21.15.52:0/170902861 conn(0x55b266c49dc0 msgr2=0x55b266d1a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 -- 172.21.15.52:0/170902861 shutdown_connections 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.124+0000 7f338726fb80 1 -- 172.21.15.52:0/170902861 wait complete. 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 -- start start 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 Processor -- start 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d2b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d28690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d28bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b266d303e0 con 0x55b266d1f3d0 2024-06-02T23:24:10.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b266c10cf0 con 0x55b266d24c40 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b266d30560 con 0x55b266d25560 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f338726fb80 10 client.0 _ll_register_callbacks cb 0x55b266c0aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f337659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d2b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f337659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d2b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59004/0 (socket says 172.21.15.52:59004) 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f337659c700 1 -- 172.21.15.52:0/1180063922 learned_addr learned my addr 172.21.15.52:0/1180063922 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f3376d9d700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d28bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f3375d9b700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d28690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.125+0000 7f337659c700 1 -- 172.21.15.52:0/1180063922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 msgr2=0x55b266d28bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f337659c700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d28bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f337659c700 1 -- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 msgr2=0x55b266d28690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f337659c700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d28690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f337659c700 1 -- 172.21.15.52:0/1180063922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b266bbf9d0 con 0x55b266d1f3d0 2024-06-02T23:24:10.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f337659c700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d2b390 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f336001de30 tx=0x7f33600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa70cb997ccd22e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f335dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f335dffb700 1 -- 172.21.15.52:0/1180063922 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360016070 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f335dffb700 1 -- 172.21.15.52:0/1180063922 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3360010cd0 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3370005a80 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f335dffb700 1 -- 172.21.15.52:0/1180063922 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f336000fe80 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f338726fb80 10 client.18921 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b266d3adb0 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f335dffb700 1 -- 172.21.15.52:0/1180063922 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3370005a80 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.126+0000 7f335dffb700 1 -- 172.21.15.52:0/1180063922 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f3360032430 con 0x55b266d1f3d0 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f338726fb80 10 client.18921 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f338726fb80 10 client.18921 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f338726fb80 20 client.18921 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:10.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f338726fb80 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266d3d340 0x55b266d3f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f338726fb80 4 client.18921 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b266c7de10 con 0x55b266d3d340 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.127+0000 7f3375d9b700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266d3d340 0x55b266d3f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f3375d9b700 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266d3d340 0x55b266d3f780 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42324d3ee0129a10 server_cookie=b29daa0a4f9a0ff in_seq=0 out_seq=0 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f335dffb700 10 client.18921 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f335dffb700 1 -- 172.21.15.52:0/1180063922 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b266c7de10 con 0x55b266d3d340 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f335dffb700 10 client.18921 handle_command_reply: tid=0 2024-06-02T23:24:10.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f338726fb80 1 client.18921 shutdown 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 msgr2=0x55b266d2b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f338726fb80 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d2b390 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f336001de30 tx=0x7f33600047c0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.128+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 shutdown_connections 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266d3d340 0x55b266d3f780 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d25560 0x55b266d28bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d2b390 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 --2- 172.21.15.52:0/1180063922 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d28690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 >> 172.21.15.52:0/1180063922 conn(0x55b266c49dc0 msgr2=0x55b266d19d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:10.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f335dffb700 0 client.18921 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:10.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 shutdown_connections 2024-06-02T23:24:10.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 1 -- 172.21.15.52:0/1180063922 wait complete. 2024-06-02T23:24:10.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.129+0000 7f338726fb80 20 client.18921 trim_cache size 0 max 16384 2024-06-02T23:24:10.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 Processor -- start 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 -- start start 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d24c40 0x55b266d39680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d25560 0x55b266d261d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33781aa870 con 0x55b266d1f3d0 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33781ac3b0 con 0x55b266d25560 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f338726fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33781aaf40 con 0x55b266d24c40 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.149+0000 7f337659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59018/0 (socket says 172.21.15.52:59018) 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 -- 172.21.15.52:0/891356150 learned_addr learned my addr 172.21.15.52:0/891356150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f3375d9b700 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d24c40 0x55b266d39680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f3376d9d700 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d25560 0x55b266d261d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 -- 172.21.15.52:0/891356150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d24c40 msgr2=0x55b266d39680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d24c40 0x55b266d39680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 -- 172.21.15.52:0/891356150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d25560 msgr2=0x55b266d261d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d25560 0x55b266d261d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 -- 172.21.15.52:0/891356150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b266d279e0 con 0x55b266d1f3d0 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f337659c700 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1ee90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f33600328d0 tx=0x7f3360004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a85ff2fbc52b754 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f335ffff700 1 -- 172.21.15.52:0/891356150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360032630 con 0x55b266d1f3d0 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f335ffff700 1 -- 172.21.15.52:0/891356150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3360010710 con 0x55b266d1f3d0 2024-06-02T23:24:10.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.150+0000 7f335ffff700 1 -- 172.21.15.52:0/891356150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360004a90 con 0x55b266d1f3d0 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 -- 172.21.15.52:0/891356150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 msgr2=0x55b266d1ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1ee90 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f33600328d0 tx=0x7f3360004750 comp rx=0 tx=0).stop 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 -- 172.21.15.52:0/891356150 shutdown_connections 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d25560 0x55b266d261d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d1f3d0 0x55b266d1ee90 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 --2- 172.21.15.52:0/891356150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d24c40 0x55b266d39680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 -- 172.21.15.52:0/891356150 >> 172.21.15.52:0/891356150 conn(0x55b266c49dc0 msgr2=0x55b266d23830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 -- 172.21.15.52:0/891356150 shutdown_connections 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 -- 172.21.15.52:0/891356150 wait complete. 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 -- start start 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.151+0000 7f338726fb80 1 Processor -- start 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d1f3d0 0x55b266d35da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d327c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 0x55b266d32d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33781abce0 con 0x55b266d25560 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33781ab610 con 0x55b266d24c40 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33781a8850 con 0x55b266d1f3d0 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f338726fb80 10 client.0 _ll_register_callbacks cb 0x55b266c0aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 0x55b266d32d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 0x55b266d32d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59020/0 (socket says 172.21.15.52:59020) 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 -- 172.21.15.52:0/2385481687 learned_addr learned my addr 172.21.15.52:0/2385481687 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f337659c700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d1f3d0 0x55b266d35da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3375d9b700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d327c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 -- 172.21.15.52:0/2385481687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d1f3d0 msgr2=0x55b266d35da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d1f3d0 0x55b266d35da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 -- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 msgr2=0x55b266d327c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d327c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 -- 172.21.15.52:0/2385481687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b266d332f0 con 0x55b266d25560 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3375d9b700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f3376d9d700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 0x55b266d32d00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3370000f80 tx=0x7f3370005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3dc54633ce214f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f335dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.152+0000 7f335dffb700 1 -- 172.21.15.52:0/2385481687 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33700a38e0 con 0x55b266d25560 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f335dffb700 1 -- 172.21.15.52:0/2385481687 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33700b9b40 con 0x55b266d25560 2024-06-02T23:24:10.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360010710 con 0x55b266d25560 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f335dffb700 1 -- 172.21.15.52:0/2385481687 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f337007f050 con 0x55b266d25560 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 10 client.18927 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55b266da8e70 con 0x55b266d25560 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f335dffb700 1 -- 172.21.15.52:0/2385481687 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3360010710 con 0x55b266d25560 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f335dffb700 1 -- 172.21.15.52:0/2385481687 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f33700b9020 con 0x55b266d25560 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 10 client.18927 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 10 client.18927 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 20 client.18927 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266da9150 0x55b266dab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 4 client.18927 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f901e1b4", "release": true, "await": true}] 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f901e1b4", "release": true, "await": true}) -- 0x55b266c78390 con 0x55b266da9150 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.153+0000 7f337659c700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266da9150 0x55b266dab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.154+0000 7f337659c700 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266da9150 0x55b266dab540 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5afe1be1d5ba1943 server_cookie=46e4e759208eba0b in_seq=0 out_seq=0 2024-06-02T23:24:10.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.154+0000 7f335dffb700 10 client.18927 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f335dffb700 1 -- 172.21.15.52:0/2385481687 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55b266c78390 con 0x55b266da9150 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f335dffb700 10 client.18927 handle_command_reply: tid=0 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 client.18927 shutdown 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 msgr2=0x55b266d32d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 0x55b266d32d00 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f3370000f80 tx=0x7f3370005230 comp rx=0 tx=0).stop 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 shutdown_connections 2024-06-02T23:24:10.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b266da9150 0x55b266dab540 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b266d25560 0x55b266d32d00 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b266d1f3d0 0x55b266d35da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 --2- 172.21.15.52:0/2385481687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b266d24c40 0x55b266d327c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 >> 172.21.15.52:0/2385481687 conn(0x55b266c49dc0 msgr2=0x55b266d23830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f335dffb700 0 client.18927 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 shutdown_connections 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 1 -- 172.21.15.52:0/2385481687 wait complete. 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.161+0000 7f338726fb80 20 client.18927 trim_cache size 0 max 16384 2024-06-02T23:24:10.163 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 32, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 44, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "f901e1b4": { 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 44, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:24:10.164 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 140.8, 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:10.165 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33640a2ca0 msgr2=0x7f33640a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33640a2ca0 0x7f33640a5160 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3360005950 tx=0x7f336000a300 comp rx=0 tx=0).stop 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 msgr2=0x7f33781988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 0x7f33781988e0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f3370007ee0 tx=0x7f337000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 shutdown_connections 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33640a2ca0 0x7f33640a5160 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3378107840 0x7f33781988e0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3378102740 0x7f337819e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 --2- 172.21.15.52:0/1630589981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33781030f0 0x7f337819ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 >> 172.21.15.52:0/1630589981 conn(0x7f33780795f0 msgr2=0x7f3378108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 shutdown_connections 2024-06-02T23:24:10.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:10.162+0000 7f337c9ec700 1 -- 172.21.15.52:0/1630589981 wait complete. 2024-06-02T23:24:10.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237420 1.83 MB/sec execute 559 sec latency 15944.460 ms 2024-06-02T23:24:10.878 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f901e1b4', total seconds elapsed: 26.6 2024-06-02T23:24:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:11 smithi174 ceph-mon[35994]: pgmap v800: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-06-02T23:24:11.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:11 smithi052 ceph-mon[32340]: pgmap v800: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-06-02T23:24:11.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 237819 1.83 MB/sec execute 560 sec latency 15.170 ms 2024-06-02T23:24:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:11 smithi123 ceph-mon[35977]: pgmap v800: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-06-02T23:24:12.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 238119 1.83 MB/sec execute 561 sec latency 14.403 ms 2024-06-02T23:24:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:13 smithi174 ceph-mon[35994]: pgmap v801: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 112 KiB/s wr, 18 op/s 2024-06-02T23:24:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:13 smithi052 ceph-mon[32340]: pgmap v801: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 112 KiB/s wr, 18 op/s 2024-06-02T23:24:13.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 238487 1.83 MB/sec execute 562 sec latency 15.214 ms 2024-06-02T23:24:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:13 smithi123 ceph-mon[35977]: pgmap v801: 97 pgs: 97 active+clean; 98 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 112 KiB/s wr, 18 op/s 2024-06-02T23:24:14.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 238987 1.83 MB/sec execute 563 sec latency 15.242 ms 2024-06-02T23:24:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:15 smithi174 ceph-mon[35994]: pgmap v802: 97 pgs: 97 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 141 KiB/s wr, 21 op/s 2024-06-02T23:24:15.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:15 smithi052 ceph-mon[32340]: pgmap v802: 97 pgs: 97 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 141 KiB/s wr, 21 op/s 2024-06-02T23:24:15.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 239688 1.83 MB/sec execute 564 sec latency 14.282 ms 2024-06-02T23:24:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:15 smithi123 ceph-mon[35977]: pgmap v802: 97 pgs: 97 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 141 KiB/s wr, 21 op/s 2024-06-02T23:24:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 -- 172.21.15.52:0/939308752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 msgr2=0x7fbcfc109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 --2- 172.21.15.52:0/939308752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 0x7fbcfc109b30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fbcf4009be0 tx=0x7fbcf402a6c0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 -- 172.21.15.52:0/939308752 shutdown_connections 2024-06-02T23:24:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 --2- 172.21.15.52:0/939308752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 0x7fbcfc109b30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 --2- 172.21.15.52:0/939308752 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 --2- 172.21.15.52:0/939308752 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 0x7fbcfc104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.576+0000 7fbd0475a640 1 -- 172.21.15.52:0/939308752 >> 172.21.15.52:0/939308752 conn(0x7fbcfc0fbe70 msgr2=0x7fbcfc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 -- 172.21.15.52:0/939308752 shutdown_connections 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 -- 172.21.15.52:0/939308752 wait complete. 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 Processor -- start 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 -- start start 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 0x7fbcfc197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 0x7fbcfc19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc10d550 con 0x7fbcfc104db0 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc10d3d0 con 0x7fbcfc1076e0 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.577+0000 7fbd0475a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc10d6d0 con 0x7fbcfc102480 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39500/0 (socket says 172.21.15.52:39500) 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1469660327 learned_addr learned my addr 172.21.15.52:0/1469660327 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 0x7fbcfc19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd024f6640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 0x7fbcfc197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1469660327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 msgr2=0x7fbcfc197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 0x7fbcfc197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1469660327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 msgr2=0x7fbcfc19f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 0x7fbcfc19f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1469660327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcfc19f760 con 0x7fbcfc104db0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd024f6640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 0x7fbcfc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc198080 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e9a0 tx=0x7fbcec00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbcf37fe640 1 -- 172.21.15.52:0/1469660327 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec00cd80 con 0x7fbcfc104db0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcfc19fa50 con 0x7fbcfc104db0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbcf37fe640 1 -- 172.21.15.52:0/1469660327 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcec0035f0 con 0x7fbcfc104db0 2024-06-02T23:24:16.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbcf37fe640 1 -- 172.21.15.52:0/1469660327 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec010690 con 0x7fbcfc104db0 2024-06-02T23:24:16.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.578+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcfc19fff0 con 0x7fbcfc104db0 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbcf37fe640 1 -- 172.21.15.52:0/1469660327 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbcec0037a0 con 0x7fbcfc104db0 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 Processor -- start 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 -- start start 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f273430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f278500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f27b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a91f1d64f0 con 0x55a91f273a00 2024-06-02T23:24:16.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a91f174870 con 0x55a91f273010 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.581+0000 7fbd07465740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a91f1d24b0 con 0x55a91f278b70 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd024f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f278500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f273430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f273430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56378/0 (socket says 172.21.15.52:56378) 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 -- 172.21.15.52:0/3219151788 learned_addr learned my addr 172.21.15.52:0/3219151788 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbcf37fe640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbce80a2c30 0x7fbce80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f27b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 -- 172.21.15.52:0/3219151788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 msgr2=0x55a91f27b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f27b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 -- 172.21.15.52:0/3219151788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 msgr2=0x55a91f278500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f278500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 -- 172.21.15.52:0/3219151788 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91f1909b0 con 0x55a91f273010 2024-06-02T23:24:16.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd024f6640 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f278500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbcf37fe640 1 -- 172.21.15.52:0/1469660327 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbcec0cf360 con 0x7fbcfc104db0 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbce80a2c30 0x7fbce80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.582+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f273430 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbcf402acb0 tx=0x7fbcf4030500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c9a2d1a49a60890 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbcf0ff9640 1 -- 172.21.15.52:0/3219151788 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4036040 con 0x55a91f273010 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbce80a2c30 0x7fbce80a50f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbcf404b000 tx=0x7fbcf4048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbcf0ff9640 1 -- 172.21.15.52:0/3219151788 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcf4037040 con 0x55a91f273010 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbcf0ff9640 1 -- 172.21.15.52:0/3219151788 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf40496a0 con 0x55a91f273010 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 -- 172.21.15.52:0/3219151788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 msgr2=0x55a91f273430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f273430 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbcf402acb0 tx=0x7fbcf4030500 comp rx=0 tx=0).stop 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 -- 172.21.15.52:0/3219151788 shutdown_connections 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f27b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f278500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 --2- 172.21.15.52:0/3219151788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f273430 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.583+0000 7fbd07465740 1 -- 172.21.15.52:0/3219151788 >> 172.21.15.52:0/3219151788 conn(0x55a91f26e680 msgr2=0x55a91f26ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 -- 172.21.15.52:0/3219151788 shutdown_connections 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 -- 172.21.15.52:0/3219151788 wait complete. 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 -- start start 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 Processor -- start 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 0x55a91f27d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f27d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f278b70 0x55a91f284860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a91f27bb20 con 0x55a91f273010 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a91f15fae0 con 0x55a91f278b70 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a91f27bca0 con 0x55a91f273a00 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd07465740 10 client.0 _ll_register_callbacks cb 0x55a91f1989b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd024f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 0x55a91f27d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd024f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 0x55a91f27d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39516/0 (socket says 172.21.15.52:39516) 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd024f6640 1 -- 172.21.15.52:0/4030337498 learned_addr learned my addr 172.21.15.52:0/4030337498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.584+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f278b70 0x55a91f284860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f27d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd024f6640 1 -- 172.21.15.52:0/4030337498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 msgr2=0x55a91f27d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd024f6640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f27d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd024f6640 1 -- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f278b70 msgr2=0x55a91f284860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd024f6640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f278b70 0x55a91f284860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd024f6640 1 -- 172.21.15.52:0/4030337498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91eeea210 con 0x55a91f273010 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f27d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f278b70 0x55a91f284860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd024f6640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 0x55a91f27d2e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80136e0 tx=0x7fbcf8013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc6261954c628424 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbce5ffb640 1 -- 172.21.15.52:0/4030337498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8004460 con 0x55a91f273010 2024-06-02T23:24:16.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbce5ffb640 1 -- 172.21.15.52:0/4030337498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbcf8011960 con 0x55a91f273010 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf4037040 con 0x55a91f273010 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbce5ffb640 1 -- 172.21.15.52:0/4030337498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcf8011b60 con 0x55a91f273010 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd07465740 10 client.18939 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.585+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a91efb6620 con 0x55a91f273010 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbce5ffb640 1 -- 172.21.15.52:0/4030337498 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcf4037040 con 0x55a91f273010 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbce5ffb640 1 -- 172.21.15.52:0/4030337498 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbcf801a020 con 0x55a91f273010 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd07465740 10 client.18939 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd07465740 10 client.18939 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd07465740 20 client.18939 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd07465740 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2891c0 0x55a91f28b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd07465740 4 client.18939 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a91f1e9f10 con 0x55a91f2891c0 2024-06-02T23:24:16.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2891c0 0x55a91f28b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbce5ffb640 10 client.18939 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:16.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.586+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2891c0 0x55a91f28b660 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8701325a231ef20a server_cookie=fbcfad5b0fc181bc in_seq=0 out_seq=0 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbce5ffb640 1 -- 172.21.15.52:0/4030337498 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a91f1e9f10 con 0x55a91f2891c0 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbce5ffb640 10 client.18939 handle_command_reply: tid=0 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 client.18939 shutdown 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 msgr2=0x55a91f27d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 0x55a91f27d2e0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80136e0 tx=0x7fbcf8013ba0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 shutdown_connections 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2891c0 0x55a91f28b660 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f27d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f278b70 0x55a91f284860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbce5ffb640 0 client.18939 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:16.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 --2- 172.21.15.52:0/4030337498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273010 0x55a91f27d2e0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 >> 172.21.15.52:0/4030337498 conn(0x55a91f26e680 msgr2=0x55a91f26d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 shutdown_connections 2024-06-02T23:24:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 1 -- 172.21.15.52:0/4030337498 wait complete. 2024-06-02T23:24:16.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.588+0000 7fbd07465740 20 client.18939 trim_cache size 0 max 16384 2024-06-02T23:24:16.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 Processor -- start 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 -- start start 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f271a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f271fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f278b70 0x55a91f284820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1a8c60 con 0x55a91f278b70 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1aa7a0 con 0x55a91f273010 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd07465740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1a9330 con 0x55a91f273a00 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd024f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f271a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd024f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f271a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56394/0 (socket says 172.21.15.52:56394) 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd024f6640 1 -- 172.21.15.52:0/3572876398 learned_addr learned my addr 172.21.15.52:0/3572876398 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:16.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.605+0000 7fbd02cf7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f278b70 0x55a91f284820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f271fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd024f6640 1 -- 172.21.15.52:0/3572876398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 msgr2=0x55a91f271fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd024f6640 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f271fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd024f6640 1 -- 172.21.15.52:0/3572876398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f278b70 msgr2=0x55a91f284820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd024f6640 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f278b70 0x55a91f284820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd024f6640 1 -- 172.21.15.52:0/3572876398 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a91f277ef0 con 0x55a91f273010 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbd024f6640 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f271a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8000c00 tx=0x7fbcf8011f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70d192dcdabd46da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbce7fff640 1 -- 172.21.15.52:0/3572876398 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf80119b0 con 0x55a91f273010 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbce7fff640 1 -- 172.21.15.52:0/3572876398 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcf801c840 con 0x55a91f273010 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.606+0000 7fbce7fff640 1 -- 172.21.15.52:0/3572876398 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcf8003e80 con 0x55a91f273010 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 -- 172.21.15.52:0/3572876398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 msgr2=0x55a91f271a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f271a70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8000c00 tx=0x7fbcf8011f40 comp rx=0 tx=0).stop 2024-06-02T23:24:16.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 -- 172.21.15.52:0/3572876398 shutdown_connections 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f273a00 0x55a91f271fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f278b70 0x55a91f284820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 --2- 172.21.15.52:0/3572876398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f271a70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 -- 172.21.15.52:0/3572876398 >> 172.21.15.52:0/3572876398 conn(0x55a91f26e680 msgr2=0x55a91f271360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 -- 172.21.15.52:0/3572876398 shutdown_connections 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 -- 172.21.15.52:0/3572876398 wait complete. 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.607+0000 7fbd07465740 1 -- start start 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 Processor -- start 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f2816f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f281c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f2f1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1aa0d0 con 0x55a91f273a00 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1a9a00 con 0x55a91f273010 2024-06-02T23:24:16.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbcfc1a6c40 con 0x55a91f278b70 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd07465740 10 client.0 _ll_register_callbacks cb 0x7fbcf8005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f281c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f281c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39532/0 (socket says 172.21.15.52:39532) 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1207232042 learned_addr learned my addr 172.21.15.52:0/1207232042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd024f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f2816f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd02cf7640 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f2f1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1207232042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 msgr2=0x55a91f2f1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f2f1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 msgr2=0x55a91f2816f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f2816f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 -- 172.21.15.52:0/1207232042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a91f2f2470 con 0x55a91f273a00 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbd01cf5640 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f281c30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbcec097ff0 tx=0x7fbcec097f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c39375d97df1b679 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.608+0000 7fbce5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:16.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbce5ffb640 1 -- 172.21.15.52:0/1207232042 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcec09ff40 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbce5ffb640 1 -- 172.21.15.52:0/1207232042 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbcec0ccf40 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbcf801c840 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbce5ffb640 1 -- 172.21.15.52:0/1207232042 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fbcec09bb90 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 10 client.18945 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55a91f2f37f0 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbce5ffb640 1 -- 172.21.15.52:0/1207232042 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbcf801c840 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbce5ffb640 1 -- 172.21.15.52:0/1207232042 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fbcec0ce040 con 0x55a91f273a00 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 10 client.18945 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 10 client.18945 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 20 client.18945 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2f41a0 0x55a91f2f65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 4 client.18945 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:24:16.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a91f26e4d0 con 0x55a91f2f41a0 2024-06-02T23:24:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.609+0000 7fbd024f6640 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2f41a0 0x55a91f2f65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.610+0000 7fbce5ffb640 10 client.18945 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:16.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.610+0000 7fbd024f6640 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2f41a0 0x55a91f2f65c0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=519dc3ac52bd2fae server_cookie=ec06a91922787077 in_seq=0 out_seq=0 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.633+0000 7fbce5ffb640 1 -- 172.21.15.52:0/1207232042 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55a91f26e4d0 con 0x55a91f2f41a0 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.633+0000 7fbce5ffb640 10 client.18945 handle_command_reply: tid=0 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.633+0000 7fbd07465740 1 client.18945 shutdown 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 msgr2=0x55a91f281c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f281c30 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fbcec097ff0 tx=0x7fbcec097f60 comp rx=0 tx=0).stop 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 shutdown_connections 2024-06-02T23:24:16.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a91f2f41a0 0x55a91f2f65c0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a91f278b70 0x55a91f2f1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a91f273a00 0x55a91f281c30 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 --2- 172.21.15.52:0/1207232042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a91f273010 0x55a91f2816f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbce5ffb640 0 client.18945 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 >> 172.21.15.52:0/1207232042 conn(0x55a91f26e680 msgr2=0x55a91f26d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 shutdown_connections 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 1 -- 172.21.15.52:0/1207232042 wait complete. 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd07465740 20 client.18945 trim_cache size 0 max 16384 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:24:16.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:24:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:24:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbce80a2c30 msgr2=0x7fbce80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd0475a640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbce80a2c30 0x7fbce80a50f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fbcf404b000 tx=0x7fbcf4048000 comp rx=0 tx=0).stop 2024-06-02T23:24:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 msgr2=0x7fbcfc198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd0475a640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc198080 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00e9a0 tx=0x7fbcec00ee60 comp rx=0 tx=0).stop 2024-06-02T23:24:16.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.634+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 shutdown_connections 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbce80a2c30 0x7fbce80a50f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcfc1076e0 0x7fbcfc19f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbcfc104db0 0x7fbcfc198080 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 --2- 172.21.15.52:0/1469660327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbcfc102480 0x7fbcfc197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 >> 172.21.15.52:0/1469660327 conn(0x7fbcfc0fbe70 msgr2=0x7fbcfc102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 shutdown_connections 2024-06-02T23:24:16.638 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:16.635+0000 7fbd0475a640 1 -- 172.21.15.52:0/1469660327 wait complete. 2024-06-02T23:24:16.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 240253 1.83 MB/sec execute 565 sec latency 18.332 ms 2024-06-02T23:24:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:17 smithi174 ceph-mon[35994]: pgmap v803: 97 pgs: 97 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 502 KiB/s wr, 71 op/s 2024-06-02T23:24:17.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:17 smithi052 ceph-mon[32340]: pgmap v803: 97 pgs: 97 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 502 KiB/s wr, 71 op/s 2024-06-02T23:24:17.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 240995 1.84 MB/sec execute 566 sec latency 13.760 ms 2024-06-02T23:24:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:17 smithi123 ceph-mon[35977]: pgmap v803: 97 pgs: 97 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 502 KiB/s wr, 71 op/s 2024-06-02T23:24:18.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 241441 1.84 MB/sec execute 567 sec latency 14.886 ms 2024-06-02T23:24:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:19 smithi174 ceph-mon[35994]: pgmap v804: 97 pgs: 97 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 553 KiB/s wr, 72 op/s 2024-06-02T23:24:19.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:19 smithi052 ceph-mon[32340]: pgmap v804: 97 pgs: 97 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 553 KiB/s wr, 72 op/s 2024-06-02T23:24:19.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 241676 1.84 MB/sec execute 568 sec latency 28.233 ms 2024-06-02T23:24:19.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:19 smithi123 ceph-mon[35977]: pgmap v804: 97 pgs: 97 active+clean; 99 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 553 KiB/s wr, 72 op/s 2024-06-02T23:24:20.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 242062 1.84 MB/sec execute 569 sec latency 12.570 ms 2024-06-02T23:24:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:21 smithi174 ceph-mon[35994]: pgmap v805: 97 pgs: 97 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 878 KiB/s wr, 103 op/s 2024-06-02T23:24:21.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:21 smithi052 ceph-mon[32340]: pgmap v805: 97 pgs: 97 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 878 KiB/s wr, 103 op/s 2024-06-02T23:24:21.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 242638 1.84 MB/sec execute 570 sec latency 15.140 ms 2024-06-02T23:24:21.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:21 smithi123 ceph-mon[35977]: pgmap v805: 97 pgs: 97 active+clean; 100 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 156 KiB/s rd, 878 KiB/s wr, 103 op/s 2024-06-02T23:24:22.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 243243 1.84 MB/sec execute 571 sec latency 15.114 ms 2024-06-02T23:24:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:23 smithi174 ceph-mon[35994]: pgmap v806: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-06-02T23:24:23.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:23 smithi052 ceph-mon[32340]: pgmap v806: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-06-02T23:24:23.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 243798 1.84 MB/sec execute 572 sec latency 15.136 ms 2024-06-02T23:24:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:23 smithi123 ceph-mon[35977]: pgmap v806: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 195 KiB/s rd, 1.1 MiB/s wr, 129 op/s 2024-06-02T23:24:24.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 244534 1.85 MB/sec execute 573 sec latency 14.854 ms 2024-06-02T23:24:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:25 smithi174 ceph-mon[35994]: pgmap v807: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-06-02T23:24:25.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:25 smithi052 ceph-mon[32340]: pgmap v807: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-06-02T23:24:25.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 244995 1.85 MB/sec execute 574 sec latency 18.522 ms 2024-06-02T23:24:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:25 smithi123 ceph-mon[35977]: pgmap v807: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-06-02T23:24:26.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 245249 1.85 MB/sec execute 575 sec latency 13.016 ms 2024-06-02T23:24:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:27 smithi174 ceph-mon[35994]: pgmap v808: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-02T23:24:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:27 smithi052 ceph-mon[32340]: pgmap v808: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-02T23:24:27.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 245677 1.85 MB/sec execute 576 sec latency 14.272 ms 2024-06-02T23:24:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:27 smithi123 ceph-mon[35977]: pgmap v808: 97 pgs: 97 active+clean; 101 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.3 MiB/s wr, 182 op/s 2024-06-02T23:24:27.910 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:24:28.193 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:28.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 246240 1.85 MB/sec execute 577 sec latency 15.841 ms 2024-06-02T23:24:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:29 smithi174 ceph-mon[35994]: pgmap v809: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-06-02T23:24:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:29 smithi052 ceph-mon[32340]: pgmap v809: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-06-02T23:24:29.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 246850 1.85 MB/sec execute 578 sec latency 15.438 ms 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 -- 172.21.15.52:0/519011871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4104dd0 msgr2=0x7f0ab41071c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/519011871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4104dd0 0x7f0ab41071c0 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009900 tx=0x7f0aa402a410 comp rx=0 tx=0).stop 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 -- 172.21.15.52:0/519011871 shutdown_connections 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/519011871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab4107700 0x7f0ab4109b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/519011871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4104dd0 0x7f0ab41071c0 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/519011871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 0x7f0ab4104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 -- 172.21.15.52:0/519011871 >> 172.21.15.52:0/519011871 conn(0x7f0ab40fbeb0 msgr2=0x7f0ab40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:29.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 -- 172.21.15.52:0/519011871 shutdown_connections 2024-06-02T23:24:29.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.734+0000 7f0abb9c1640 1 -- 172.21.15.52:0/519011871 wait complete. 2024-06-02T23:24:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 Processor -- start 2024-06-02T23:24:29.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 -- start start 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 0x7f0ab4197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab4104dd0 0x7f0ab4198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4107700 0x7f0ab419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab410d540 con 0x7f0ab4107700 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab410d3c0 con 0x7f0ab41024a0 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0abb9c1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab410d6c0 con 0x7f0ab4104dd0 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0ab975d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 0x7f0ab4197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0ab9f5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4107700 0x7f0ab419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.735+0000 7f0ab975d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 0x7f0ab4197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49478/0 (socket says 172.21.15.52:49478) 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab975d640 1 -- 172.21.15.52:0/3074181928 learned_addr learned my addr 172.21.15.52:0/3074181928 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab4104dd0 0x7f0ab4198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab9f5e640 1 -- 172.21.15.52:0/3074181928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab4104dd0 msgr2=0x7f0ab4198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab9f5e640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab4104dd0 0x7f0ab4198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab9f5e640 1 -- 172.21.15.52:0/3074181928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 msgr2=0x7f0ab4197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab9f5e640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 0x7f0ab4197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab9f5e640 1 -- 172.21.15.52:0/3074181928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ab419f740 con 0x7f0ab4107700 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0ab9f5e640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4107700 0x7f0ab419f0d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0aac007db0 tx=0x7f0aac010420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0aaa7fc640 1 -- 172.21.15.52:0/3074181928 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac010d40 con 0x7f0ab4107700 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0aaa7fc640 1 -- 172.21.15.52:0/3074181928 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aac00c640 con 0x7f0ab4107700 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ab41002f0 con 0x7f0ab4107700 2024-06-02T23:24:29.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0aaa7fc640 1 -- 172.21.15.52:0/3074181928 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aac017650 con 0x7f0ab4107700 2024-06-02T23:24:29.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.736+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ab4100810 con 0x7f0ab4107700 2024-06-02T23:24:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0aaa7fc640 1 -- 172.21.15.52:0/3074181928 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0aac00c7f0 con 0x7f0ab4107700 2024-06-02T23:24:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 Processor -- start 2024-06-02T23:24:29.742 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 -- start start 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1134c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca1139d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca113c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557ca108f4f0 con 0x557ca1134820 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ca102d870 con 0x557ca1135210 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.740+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ca108b4b0 con 0x557ca113a380 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca1139d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca1139d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49488/0 (socket says 172.21.15.52:49488) 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 -- 172.21.15.52:0/4182208656 learned_addr learned my addr 172.21.15.52:0/4182208656 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1134c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 -- 172.21.15.52:0/4182208656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 msgr2=0x557ca113c810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca113c810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 -- 172.21.15.52:0/4182208656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 msgr2=0x557ca1134c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.743 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1134c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab975d640 1 -- 172.21.15.52:0/4182208656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca10499b0 con 0x557ca1135210 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1134c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.741+0000 7f0aaa7fc640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0aa00a2cb0 0x7f0aa00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.742+0000 7f0aaa7fc640 1 -- 172.21.15.52:0/3074181928 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0aac0d2cb0 con 0x7f0ab4107700 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.742+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0aa00a2cb0 0x7f0aa00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.742+0000 7f0ab975d640 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca1139d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000bda0 tx=0x7f0ab00107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d72a53415ada02a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.742+0000 7f0a9f7fe640 1 -- 172.21.15.52:0/4182208656 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab001c680 con 0x557ca1135210 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.742+0000 7f0a9f7fe640 1 -- 172.21.15.52:0/4182208656 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab00118f0 con 0x557ca1135210 2024-06-02T23:24:29.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.742+0000 7f0a9f7fe640 1 -- 172.21.15.52:0/4182208656 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab001c680 con 0x557ca1135210 2024-06-02T23:24:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.743+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0aa00a2cb0 0x7f0aa00a5170 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4004770 tx=0x7f0aa4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.743+0000 7f0abe7ab740 1 -- 172.21.15.52:0/4182208656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 msgr2=0x557ca1139d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.743+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca1139d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000bda0 tx=0x7f0ab00107a0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 -- 172.21.15.52:0/4182208656 shutdown_connections 2024-06-02T23:24:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca113c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca1139d10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/4182208656 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1134c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 -- 172.21.15.52:0/4182208656 >> 172.21.15.52:0/4182208656 conn(0x557ca1127680 msgr2=0x557ca1127a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 -- 172.21.15.52:0/4182208656 shutdown_connections 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 -- 172.21.15.52:0/4182208656 wait complete. 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 -- start start 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.744+0000 7f0abe7ab740 1 Processor -- start 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca113ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca113f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca1145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557ca113d330 con 0x557ca1134820 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ca1018ae0 con 0x557ca1135210 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ca113d4b0 con 0x557ca113a380 2024-06-02T23:24:29.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0abe7ab740 10 client.0 _ll_register_callbacks cb 0x557ca10519b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca113f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca113f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49504/0 (socket says 172.21.15.52:49504) 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/2011060500 learned_addr learned my addr 172.21.15.52:0/2011060500 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab9f5e640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca1145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca113ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/2011060500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 msgr2=0x557ca1145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca1145fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/2011060500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 msgr2=0x557ca113ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca113ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/2011060500 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca0da3210 con 0x557ca1135210 2024-06-02T23:24:29.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.745+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca113ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca113f010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4033000 tx=0x7f0aa402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c47c4b38e66f01dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0a9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2011060500 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4035040 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab00118f0 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2011060500 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0aa4004480 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2011060500 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0aa40035f0 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0abe7ab740 10 client.28367 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ca0e6f620 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.746+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2011060500 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0ab00118f0 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2011060500 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0aa4044070 con 0x557ca1135210 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0abe7ab740 10 client.28367 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0abe7ab740 10 client.28367 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:29.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0abe7ab740 20 client.28367 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca114a8d0 0x557ca114cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0abe7ab740 4 client.28367 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ca10a2f10 con 0x557ca114a8d0 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca114a8d0 0x557ca114cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca114a8d0 0x557ca114cd60 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcbee0063bd2e42d server_cookie=926b07ed1800f33d in_seq=0 out_seq=0 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.747+0000 7f0a9cff9640 10 client.28367 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2011060500 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557ca10a2f10 con 0x557ca114a8d0 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0a9cff9640 10 client.28367 handle_command_reply: tid=0 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 client.28367 shutdown 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 msgr2=0x557ca113f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca113f010 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4033000 tx=0x7f0aa402a410 comp rx=0 tx=0).stop 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 shutdown_connections 2024-06-02T23:24:29.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca114a8d0 0x557ca114cd60 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca113a380 0x557ca1145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca1135210 0x557ca113f010 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2011060500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca113ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0a9cff9640 0 client.28367 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.748+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 >> 172.21.15.52:0/2011060500 conn(0x557ca1127680 msgr2=0x557ca11267e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.749+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 shutdown_connections 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.749+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2011060500 wait complete. 2024-06-02T23:24:29.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.749+0000 7f0abe7ab740 20 client.28367 trim_cache size 0 max 16384 2024-06-02T23:24:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:29 smithi123 ceph-mon[35977]: pgmap v809: 97 pgs: 97 active+clean; 102 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 214 KiB/s rd, 1.2 MiB/s wr, 135 op/s 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.765+0000 7f0abe7ab740 1 Processor -- start 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 -- start start 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1134820 0x557ca114cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 0x557ca1145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca112a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41a6990 con 0x557ca1135210 2024-06-02T23:24:29.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41a84d0 con 0x557ca113a380 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41a7060 con 0x557ca1134820 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 0x557ca1145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab9f5e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca112a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 0x557ca1145fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56848/0 (socket says 172.21.15.52:56848) 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/1598900528 learned_addr learned my addr 172.21.15.52:0/1598900528 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab975d640 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1134820 0x557ca114cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/1598900528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1134820 msgr2=0x557ca114cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1134820 0x557ca114cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/1598900528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 msgr2=0x557ca112a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca112a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab8f5c640 1 -- 172.21.15.52:0/1598900528 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ca11274b0 con 0x557ca1135210 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.766+0000 7f0ab9f5e640 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca112a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:29.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.767+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 0x557ca1145fa0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005ec0 tx=0x7f0aa4005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5e68c1f6e8f240 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.767+0000 7f0a9effd640 1 -- 172.21.15.52:0/1598900528 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4035040 con 0x557ca1135210 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.767+0000 7f0a9effd640 1 -- 172.21.15.52:0/1598900528 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa405f020 con 0x557ca1135210 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.767+0000 7f0a9effd640 1 -- 172.21.15.52:0/1598900528 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4003d70 con 0x557ca1135210 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.767+0000 7f0abe7ab740 1 -- 172.21.15.52:0/1598900528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 msgr2=0x557ca1145fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.767+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 0x557ca1145fa0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005ec0 tx=0x7f0aa4005930 comp rx=0 tx=0).stop 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- 172.21.15.52:0/1598900528 shutdown_connections 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca112a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1135210 0x557ca1145fa0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/1598900528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1134820 0x557ca114cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- 172.21.15.52:0/1598900528 >> 172.21.15.52:0/1598900528 conn(0x557ca1127680 msgr2=0x557ca1136490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- 172.21.15.52:0/1598900528 shutdown_connections 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- 172.21.15.52:0/1598900528 wait complete. 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- start start 2024-06-02T23:24:29.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 Processor -- start 2024-06-02T23:24:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1145da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1135210 0x557ca1142890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca1142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41a7e00 con 0x557ca1134820 2024-06-02T23:24:29.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41a7730 con 0x557ca113a380 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0ab41a4970 con 0x557ca1135210 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.768+0000 7f0abe7ab740 10 client.0 _ll_register_callbacks cb 0x7f0aa4032f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1145da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1145da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56854/0 (socket says 172.21.15.52:56854) 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 -- 172.21.15.52:0/2954050407 learned_addr learned my addr 172.21.15.52:0/2954050407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab9f5e640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca1142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1135210 0x557ca1142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 -- 172.21.15.52:0/2954050407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1135210 msgr2=0x557ca1142890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1135210 0x557ca1142890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 -- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 msgr2=0x557ca1142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca1142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 -- 172.21.15.52:0/2954050407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ca11b3630 con 0x557ca1134820 2024-06-02T23:24:29.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1135210 0x557ca1142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0ab975d640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1145da0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0010ca0 tx=0x7f0ab0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7224a5ce3f725685 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0a9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2954050407 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab001c680 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2954050407 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0ab0011650 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0aa405f020 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2954050407 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f0ab0038050 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0abe7ab740 10 client.18963 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.769+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x557ca11b4850 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2954050407 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa405f020 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2954050407 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f0ab0005130 con 0x557ca1134820 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0abe7ab740 10 client.18963 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0abe7ab740 10 client.18963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:29.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0abe7ab740 20 client.18963 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca11b4ed0 0x557ca11b72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0abe7ab740 4 client.18963 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:24:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ca109cc20 con 0x557ca11b4ed0 2024-06-02T23:24:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca11b4ed0 0x557ca11b72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0ab8f5c640 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca11b4ed0 0x557ca11b72c0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e9c74ea290e8ecf server_cookie=519d318c87cc926d in_seq=0 out_seq=0 2024-06-02T23:24:29.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.770+0000 7f0a9cff9640 10 client.18963 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0a9cff9640 1 -- 172.21.15.52:0/2954050407 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557ca109cc20 con 0x557ca11b4ed0 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0a9cff9640 10 client.18963 handle_command_reply: tid=0 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 client.18963 shutdown 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 msgr2=0x557ca1145da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1145da0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0010ca0 tx=0x7f0ab0005f30 comp rx=0 tx=0).stop 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 shutdown_connections 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557ca11b4ed0 0x557ca11b72c0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ca1135210 0x557ca1142890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ca113a380 0x557ca1142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0a9cff9640 0 client.18963 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 --2- 172.21.15.52:0/2954050407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557ca1134820 0x557ca1145da0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.798 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 >> 172.21.15.52:0/2954050407 conn(0x557ca1127680 msgr2=0x557ca112a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 shutdown_connections 2024-06-02T23:24:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 1 -- 172.21.15.52:0/2954050407 wait complete. 2024-06-02T23:24:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.797+0000 7f0abe7ab740 20 client.18963 trim_cache size 0 max 16384 2024-06-02T23:24:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:24:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:24:29.799 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:24:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0aa00a2cb0 msgr2=0x7f0aa00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0aa00a2cb0 0x7f0aa00a5170 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4004770 tx=0x7f0aa4000bc0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4107700 msgr2=0x7f0ab419f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4107700 0x7f0ab419f0d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f0aac007db0 tx=0x7f0aac010420 comp rx=0 tx=0).stop 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 shutdown_connections 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0aa00a2cb0 0x7f0aa00a5170 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0ab4104dd0 0x7f0ab4198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0ab4107700 0x7f0ab419f0d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 --2- 172.21.15.52:0/3074181928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ab41024a0 0x7f0ab4197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 >> 172.21.15.52:0/3074181928 conn(0x7f0ab40fbeb0 msgr2=0x7f0ab4102de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 shutdown_connections 2024-06-02T23:24:29.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:29.798+0000 7f0abb9c1640 1 -- 172.21.15.52:0/3074181928 wait complete. 2024-06-02T23:24:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.160+0000 7f6f97fe7700 1 -- 172.21.15.52:0/2618253145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90101390 msgr2=0x7f6f90101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.160+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/2618253145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90101390 0x7f6f90101810 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009990 tx=0x7f6f80009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 -- 172.21.15.52:0/2618253145 shutdown_connections 2024-06-02T23:24:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/2618253145 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f90101d50 0x7f6f9010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/2618253145 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90107400 0x7f6f901077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/2618253145 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90101390 0x7f6f90101810 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 -- 172.21.15.52:0/2618253145 >> 172.21.15.52:0/2618253145 conn(0x7f6f90078340 msgr2=0x7f6f90078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 -- 172.21.15.52:0/2618253145 shutdown_connections 2024-06-02T23:24:30.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.161+0000 7f6f97fe7700 1 -- 172.21.15.52:0/2618253145 wait complete. 2024-06-02T23:24:30.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 Processor -- start 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 -- start start 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f90101390 0x7f6f9019d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 0x7f6f9019dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90107400 0x7f6f901978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90111550 con 0x7f6f90107400 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901113d0 con 0x7f6f90101d50 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f97fe7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901116d0 con 0x7f6f90101390 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 0x7f6f9019dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 0x7f6f9019dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49546/0 (socket says 172.21.15.52:49546) 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 -- 172.21.15.52:0/963430750 learned_addr learned my addr 172.21.15.52:0/963430750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:30.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95d83700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f90101390 0x7f6f9019d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f96584700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90107400 0x7f6f901978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 -- 172.21.15.52:0/963430750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f90101390 msgr2=0x7f6f9019d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f90101390 0x7f6f9019d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 -- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90107400 msgr2=0x7f6f901978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90107400 0x7f6f901978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.162+0000 7f6f95582700 1 -- 172.21.15.52:0/963430750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f90198030 con 0x7f6f90101d50 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f96584700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90107400 0x7f6f901978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f95582700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 0x7f6f9019dc00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000c00 tx=0x7f6f800047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f86ffd700 1 -- 172.21.15.52:0/963430750 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80027070 con 0x7f6f90101d50 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f901982c0 con 0x7f6f90101d50 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f86ffd700 1 -- 172.21.15.52:0/963430750 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f80022470 con 0x7f6f90101d50 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f86ffd700 1 -- 172.21.15.52:0/963430750 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80017650 con 0x7f6f90101d50 2024-06-02T23:24:30.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.163+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f9006a170 con 0x7f6f90101d50 2024-06-02T23:24:30.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.165+0000 7f6f86ffd700 1 -- 172.21.15.52:0/963430750 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f8002a070 con 0x7f6f90101d50 2024-06-02T23:24:30.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.166+0000 7f6f86ffd700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f7c0a2c20 0x7f6f7c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.166+0000 7f6f86ffd700 1 -- 172.21.15.52:0/963430750 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6f800cfd60 con 0x7f6f90101d50 2024-06-02T23:24:30.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.166+0000 7f6f95d83700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f7c0a2c20 0x7f6f7c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.167+0000 7f6f95d83700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f7c0a2c20 0x7f6f7c0a50e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6f900ff970 tx=0x7f6f8c010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:30.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.167+0000 7f6fa286ab80 1 Processor -- start 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.167+0000 7f6fa286ab80 1 -- start start 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e75340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cec4dc6810 con 0x55cec4e7af60 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cec4d6e720 con 0x55cec4e7a640 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec4dc27d0 con 0x55cec4e74f20 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e749e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56886/0 (socket says 172.21.15.52:56886) 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 -- 172.21.15.52:0/888939161 learned_addr learned my addr 172.21.15.52:0/888939161 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95d83700 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f96584700 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e75340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 -- 172.21.15.52:0/888939161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 msgr2=0x55cec4e75340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e75340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 -- 172.21.15.52:0/888939161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 msgr2=0x55cec4e7aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 -- 172.21.15.52:0/888939161 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec4d64f50 con 0x55cec4e7af60 2024-06-02T23:24:30.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.168+0000 7f6f95582700 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e749e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6f80098ae0 tx=0x7f6f8009cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c4fa8ce52a5cdd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:30.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.169+0000 7f6f7bfff700 1 -- 172.21.15.52:0/888939161 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800a0a30 con 0x55cec4e7af60 2024-06-02T23:24:30.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.169+0000 7f6f7bfff700 1 -- 172.21.15.52:0/888939161 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f800a5260 con 0x55cec4e7af60 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.169+0000 7f6f7bfff700 1 -- 172.21.15.52:0/888939161 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8009cb40 con 0x55cec4e7af60 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.169+0000 7f6fa286ab80 1 -- 172.21.15.52:0/888939161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 msgr2=0x55cec4e749e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.169+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e749e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f6f80098ae0 tx=0x7f6f8009cd80 comp rx=0 tx=0).stop 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 -- 172.21.15.52:0/888939161 shutdown_connections 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e75340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/888939161 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e749e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 -- 172.21.15.52:0/888939161 >> 172.21.15.52:0/888939161 conn(0x55cec4d9fcc0 msgr2=0x55cec4e6f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 -- 172.21.15.52:0/888939161 shutdown_connections 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 -- 172.21.15.52:0/888939161 wait complete. 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 -- start start 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.170+0000 7f6fa286ab80 1 Processor -- start 2024-06-02T23:24:30.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e80d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e7e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cec4e816f0 con 0x55cec4e7af60 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cec4d66bf0 con 0x55cec4e7a640 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cec4e81870 con 0x55cec4e74f20 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6fa286ab80 10 client.0 _ll_register_callbacks cb 0x55cec4d60920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f96584700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e7e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49566/0 (socket says 172.21.15.52:49566) 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f96584700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e7e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56892/0 (socket says 172.21.15.52:56892) 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 -- 172.21.15.52:0/3394555336 learned_addr learned my addr 172.21.15.52:0/3394555336 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95d83700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e80d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 -- 172.21.15.52:0/3394555336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 msgr2=0x55cec4e80d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e80d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 -- 172.21.15.52:0/3394555336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 msgr2=0x55cec4e7e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e7e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 -- 172.21.15.52:0/3394555336 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec4d158d0 con 0x55cec4e7a640 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f96584700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e7e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f95582700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7e030 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6f800a4d30 tx=0x7f6f8000b490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fbf6f6a0359c472 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.171+0000 7f6f797fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6f797fa700 1 -- 172.21.15.52:0/3394555336 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8009c8e0 con 0x55cec4e7a640 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6f797fa700 1 -- 172.21.15.52:0/3394555336 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6f8009ca80 con 0x55cec4e7a640 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f800a5260 con 0x55cec4e7a640 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6f797fa700 1 -- 172.21.15.52:0/3394555336 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f800a4dd0 con 0x55cec4e7a640 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 10 client.28385 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cec4e816f0 con 0x55cec4e7a640 2024-06-02T23:24:30.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6f797fa700 1 -- 172.21.15.52:0/3394555336 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f800a5260 con 0x55cec4e7a640 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6f797fa700 1 -- 172.21.15.52:0/3394555336 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6f800b0b10 con 0x55cec4e7a640 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 10 client.28385 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 10 client.28385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 20 client.28385 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.172+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4e8e3d0 0x55cec4e907c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.173+0000 7f6fa286ab80 4 client.28385 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.173+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cec4dd3d10 con 0x55cec4e8e3d0 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.173+0000 7f6f95d83700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4e8e3d0 0x55cec4e907c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.173+0000 7f6f797fa700 10 client.28385 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:30.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.173+0000 7f6f95d83700 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4e8e3d0 0x55cec4e907c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5ea961ba0a344d5 server_cookie=661341e55c4aec24 in_seq=0 out_seq=0 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6f797fa700 1 -- 172.21.15.52:0/3394555336 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cec4dd3d10 con 0x55cec4e8e3d0 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6f797fa700 10 client.28385 handle_command_reply: tid=0 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 client.28385 shutdown 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 msgr2=0x55cec4e7e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7e030 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6f800a4d30 tx=0x7f6f8000b490 comp rx=0 tx=0).stop 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 shutdown_connections 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4e8e3d0 0x55cec4e907c0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6f797fa700 0 client.28385 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:30.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e7e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e80d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/3394555336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7a640 0x55cec4e7e030 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.174+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 >> 172.21.15.52:0/3394555336 conn(0x55cec4d9fcc0 msgr2=0x55cec4e6f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:30.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.175+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 shutdown_connections 2024-06-02T23:24:30.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.175+0000 7f6fa286ab80 1 -- 172.21.15.52:0/3394555336 wait complete. 2024-06-02T23:24:30.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.175+0000 7f6fa286ab80 20 client.28385 trim_cache size 0 max 16384 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 Processor -- start 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 -- start start 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e749e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e8a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e8e3d0 0x55cec4e78730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901aec30 con 0x55cec4e7af60 2024-06-02T23:24:30.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b0770 con 0x55cec4e8e3d0 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901af300 con 0x55cec4e74f20 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6f95582700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e8a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6f96584700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e8e3d0 0x55cec4e78730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6f95582700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e8a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56898/0 (socket says 172.21.15.52:56898) 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6f95582700 1 -- 172.21.15.52:0/2839632010 learned_addr learned my addr 172.21.15.52:0/2839632010 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.198+0000 7f6f95d83700 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e749e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f95582700 1 -- 172.21.15.52:0/2839632010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 msgr2=0x55cec4e749e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f95582700 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e749e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f95582700 1 -- 172.21.15.52:0/2839632010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e8e3d0 msgr2=0x55cec4e78730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f95582700 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e8e3d0 0x55cec4e78730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f95582700 1 -- 172.21.15.52:0/2839632010 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cec4e7c5a0 con 0x55cec4e7af60 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f95582700 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e8a9f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6f8009a930 tx=0x7f6f8009ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac45468d3f39577 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f7b7fe700 1 -- 172.21.15.52:0/2839632010 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80098750 con 0x55cec4e7af60 2024-06-02T23:24:30.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f7b7fe700 1 -- 172.21.15.52:0/2839632010 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f800870b0 con 0x55cec4e7af60 2024-06-02T23:24:30.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.199+0000 7f6f7b7fe700 1 -- 172.21.15.52:0/2839632010 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800a1bc0 con 0x55cec4e7af60 2024-06-02T23:24:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 -- 172.21.15.52:0/2839632010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 msgr2=0x55cec4e8a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e8a9f0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f6f8009a930 tx=0x7f6f8009ab90 comp rx=0 tx=0).stop 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 -- 172.21.15.52:0/2839632010 shutdown_connections 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e8e3d0 0x55cec4e78730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e749e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/2839632010 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e7af60 0x55cec4e8a9f0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 -- 172.21.15.52:0/2839632010 >> 172.21.15.52:0/2839632010 conn(0x55cec4d9fcc0 msgr2=0x55cec4e7b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 -- 172.21.15.52:0/2839632010 shutdown_connections 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 -- 172.21.15.52:0/2839632010 wait complete. 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 -- start start 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.200+0000 7f6fa286ab80 1 Processor -- start 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e95b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7af60 0x55cec4e960c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e8e3d0 0x55cec4e96600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901b00a0 con 0x55cec4e8e3d0 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901af9d0 con 0x55cec4e7af60 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f901acbd0 con 0x55cec4e74f20 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6fa286ab80 10 client.0 _ll_register_callbacks cb 0x55cec4d60920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e95b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f96584700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e8e3d0 0x55cec4e96600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e95b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39328/0 (socket says 172.21.15.52:39328) 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 -- 172.21.15.52:0/1367009868 learned_addr learned my addr 172.21.15.52:0/1367009868 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95582700 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7af60 0x55cec4e960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 -- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7af60 msgr2=0x55cec4e960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7af60 0x55cec4e960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 -- 172.21.15.52:0/1367009868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e8e3d0 msgr2=0x55cec4e96600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e8e3d0 0x55cec4e96600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.201+0000 7f6f95d83700 1 -- 172.21.15.52:0/1367009868 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cec4ef8060 con 0x55cec4e74f20 2024-06-02T23:24:30.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6f95d83700 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e95b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009fd0 tx=0x7f6f8c004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c552ca6e6790e0c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6f797fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6f797fa700 1 -- 172.21.15.52:0/1367009868 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8c003890 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6f797fa700 1 -- 172.21.15.52:0/1367009868 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6f8c003a30 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55cec4dce290 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6f797fa700 1 -- 172.21.15.52:0/1367009868 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f6f8c003c30 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6fa286ab80 10 client.28087 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.202+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55cec4ef9260 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6f797fa700 1 -- 172.21.15.52:0/1367009868 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55cec4dce290 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6f797fa700 1 -- 172.21.15.52:0/1367009868 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f6f8c005360 con 0x55cec4e74f20 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6fa286ab80 10 client.28087 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6fa286ab80 10 client.28087 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6fa286ab80 20 client.28087 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:30.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4ef99f0 0x55cec4efbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6fa286ab80 4 client.28087 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cec4c48250 con 0x55cec4ef99f0 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6f95582700 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4ef99f0 0x55cec4efbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6f797fa700 10 client.28087 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.203+0000 7f6f95582700 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4ef99f0 0x55cec4efbe10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724ce6dde46013a9 server_cookie=7037d205809d4171 in_seq=0 out_seq=0 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6f797fa700 1 -- 172.21.15.52:0/1367009868 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55cec4c48250 con 0x55cec4ef99f0 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6f797fa700 10 client.28087 handle_command_reply: tid=0 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 client.28087 shutdown 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 msgr2=0x55cec4e95b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e95b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6f8c009fd0 tx=0x7f6f8c004750 comp rx=0 tx=0).stop 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 shutdown_connections 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cec4ef99f0 0x55cec4efbe10 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cec4e8e3d0 0x55cec4e96600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cec4e74f20 0x55cec4e95b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6f797fa700 0 client.28087 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 --2- 172.21.15.52:0/1367009868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cec4e7af60 0x55cec4e960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 >> 172.21.15.52:0/1367009868 conn(0x55cec4d9fcc0 msgr2=0x55cec4e68e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 shutdown_connections 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 1 -- 172.21.15.52:0/1367009868 wait complete. 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.204+0000 7f6fa286ab80 20 client.28087 trim_cache size 0 max 16384 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:24:30.210 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:24:30.211 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:24:30.211 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f7c0a2c20 msgr2=0x7f6f7c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f7c0a2c20 0x7f6f7c0a50e0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6f900ff970 tx=0x7f6f8c010040 comp rx=0 tx=0).stop 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 msgr2=0x7f6f9019dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 0x7f6f9019dc00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6f80000c00 tx=0x7f6f800047b0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 shutdown_connections 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f7c0a2c20 0x7f6f7c0a50e0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f90107400 0x7f6f901978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f90101390 0x7f6f9019d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 --2- 172.21.15.52:0/963430750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f90101d50 0x7f6f9019dc00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 >> 172.21.15.52:0/963430750 conn(0x7f6f90078340 msgr2=0x7f6f90100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 shutdown_connections 2024-06-02T23:24:30.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:30.205+0000 7f6f97fe7700 1 -- 172.21.15.52:0/963430750 wait complete. 2024-06-02T23:24:30.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 247352 1.85 MB/sec execute 579 sec latency 24.248 ms 2024-06-02T23:24:30.944 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:24:30.944 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:24:31.232 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:31.642 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:31 smithi052 ceph-mon[32340]: pgmap v810: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:24:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:31 smithi174 ceph-mon[35994]: pgmap v810: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:24:31.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 248070 1.86 MB/sec execute 580 sec latency 15.825 ms 2024-06-02T23:24:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:31 smithi123 ceph-mon[35977]: pgmap v810: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 290 KiB/s rd, 1.4 MiB/s wr, 165 op/s 2024-06-02T23:24:32.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 248469 1.86 MB/sec execute 581 sec latency 14.784 ms 2024-06-02T23:24:33.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 -- 172.21.15.52:0/3692061280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa58102ea0 msgr2=0x7faa5810f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 --2- 172.21.15.52:0/3692061280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa58102ea0 0x7faa5810f380 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7faa5000b0a0 tx=0x7faa5000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 -- 172.21.15.52:0/3692061280 shutdown_connections 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 --2- 172.21.15.52:0/3692061280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa58102ea0 0x7faa5810f380 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 --2- 172.21.15.52:0/3692061280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa581085f0 0x7faa581089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 --2- 172.21.15.52:0/3692061280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 0x7faa58102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 -- 172.21.15.52:0/3692061280 >> 172.21.15.52:0/3692061280 conn(0x7faa58079640 msgr2=0x7faa58079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 -- 172.21.15.52:0/3692061280 shutdown_connections 2024-06-02T23:24:33.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.132+0000 7faa5cc25700 1 -- 172.21.15.52:0/3692061280 wait complete. 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 Processor -- start 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 -- start start 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 0x7faa5819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 0x7faa5819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 0x7faa58198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faa58112660 con 0x7faa581085f0 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faa581124e0 con 0x7faa58102ea0 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5cc25700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa581127e0 con 0x7faa581024e0 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa55d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 0x7faa5819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:33.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa56d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 0x7faa58198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa55d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 0x7faa5819ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49608/0 (socket says 172.21.15.52:49608) 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa5659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 0x7faa5819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa56d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 0x7faa58198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56930/0 (socket says 172.21.15.52:56930) 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.133+0000 7faa55d9b700 1 -- 172.21.15.52:0/1083566252 learned_addr learned my addr 172.21.15.52:0/1083566252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5659c700 1 -- 172.21.15.52:0/1083566252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 msgr2=0x7faa5819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5659c700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 0x7faa5819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5659c700 1 -- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 msgr2=0x7faa58198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5659c700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 0x7faa58198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5659c700 1 -- 172.21.15.52:0/1083566252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa58199060 con 0x7faa581024e0 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa55d9b700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 0x7faa5819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa56d9d700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 0x7faa58198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5659c700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 0x7faa5819e6f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e9f0 tx=0x7faa4800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa4800c9c0 con 0x7faa581024e0 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa58199350 con 0x7faa581024e0 2024-06-02T23:24:33.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7faa4800cb60 con 0x7faa581024e0 2024-06-02T23:24:33.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa48018680 con 0x7faa581024e0 2024-06-02T23:24:33.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.134+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa580746c0 con 0x7faa581024e0 2024-06-02T23:24:33.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.135+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa58069180 con 0x7faa581024e0 2024-06-02T23:24:33.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.137+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7faa48010430 con 0x7faa581024e0 2024-06-02T23:24:33.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.137+0000 7faa477fe700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa400a2c20 0x7faa400a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:33.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.137+0000 7faa55d9b700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa400a2c20 0x7faa400a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:33.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.137+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7faa480cf2b0 con 0x7faa581024e0 2024-06-02T23:24:33.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.138+0000 7faa55d9b700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa400a2c20 0x7faa400a50e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faa58079bb0 tx=0x7faa4c012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:33.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.142+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7faa48053850 con 0x7faa581024e0 2024-06-02T23:24:33.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.434+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faa5802dca0 con 0x7faa581024e0 2024-06-02T23:24:33.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.434+0000 7faa477fe700 1 -- 172.21.15.52:0/1083566252 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11939 (secure 0 0 0) 0x7faa4809c1e0 con 0x7faa581024e0 2024-06-02T23:24:33.435 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:24:33.435 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":32,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:15:30.518538+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:24:33.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa400a2c20 msgr2=0x7faa400a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:33.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa400a2c20 0x7faa400a50e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faa58079bb0 tx=0x7faa4c012040 comp rx=0 tx=0).stop 2024-06-02T23:24:33.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 msgr2=0x7faa5819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:33.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 0x7faa5819e6f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faa4800e9f0 tx=0x7faa4800ed00 comp rx=0 tx=0).stop 2024-06-02T23:24:33.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 shutdown_connections 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7faa400a2c20 0x7faa400a50e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faa581085f0 0x7faa58198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa581024e0 0x7faa5819e6f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 --2- 172.21.15.52:0/1083566252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faa58102ea0 0x7faa5819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 >> 172.21.15.52:0/1083566252 conn(0x7faa58079640 msgr2=0x7faa58101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 shutdown_connections 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:33.437+0000 7faa5cc25700 1 -- 172.21.15.52:0/1083566252 wait complete. 2024-06-02T23:24:33.438 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 32 2024-06-02T23:24:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:33 smithi174 ceph-mon[35994]: pgmap v811: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-02T23:24:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:33 smithi052 ceph-mon[32340]: pgmap v811: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-02T23:24:33.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 248748 1.86 MB/sec execute 582 sec latency 17.818 ms 2024-06-02T23:24:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:33 smithi123 ceph-mon[35977]: pgmap v811: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 267 KiB/s rd, 1.3 MiB/s wr, 156 op/s 2024-06-02T23:24:34.124 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:24:34.395 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:34 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1083566252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:24:34.648 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:34 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1083566252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:24:34.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 249084 1.86 MB/sec execute 583 sec latency 15.273 ms 2024-06-02T23:24:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:34 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1083566252' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:24:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:35 smithi174 ceph-mon[35994]: pgmap v812: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:24:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:35 smithi052 ceph-mon[32340]: pgmap v812: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:24:35.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 249581 1.86 MB/sec execute 584 sec latency 15.020 ms 2024-06-02T23:24:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:35 smithi123 ceph-mon[35977]: pgmap v812: 97 pgs: 97 active+clean; 103 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.1 MiB/s wr, 140 op/s 2024-06-02T23:24:36.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 -- 172.21.15.52:0/3340867499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101d50 msgr2=0x7f7a2010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 --2- 172.21.15.52:0/3340867499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101d50 0x7f7a2010e240 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800b0a0 tx=0x7f7a1800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 -- 172.21.15.52:0/3340867499 shutdown_connections 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 --2- 172.21.15.52:0/3340867499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101d50 0x7f7a2010e240 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 --2- 172.21.15.52:0/3340867499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a20107400 0x7f7a201077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 --2- 172.21.15.52:0/3340867499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20101390 0x7f7a20101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 -- 172.21.15.52:0/3340867499 >> 172.21.15.52:0/3340867499 conn(0x7f7a20078340 msgr2=0x7f7a20078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 -- 172.21.15.52:0/3340867499 shutdown_connections 2024-06-02T23:24:36.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.111+0000 7f7a248e0700 1 -- 172.21.15.52:0/3340867499 wait complete. 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 Processor -- start 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 -- start start 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 0x7f7a2019e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a20101d50 0x7f7a2019ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20107400 0x7f7a20198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a20111520 con 0x7f7a20101390 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201113a0 con 0x7f7a20107400 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a248e0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201116a0 con 0x7f7a20101d50 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 0x7f7a2019e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 0x7f7a2019e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48852/0 (socket says 172.21.15.52:48852) 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a1e59c700 1 -- 172.21.15.52:0/937927723 learned_addr learned my addr 172.21.15.52:0/937927723 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a20101d50 0x7f7a2019ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.112+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20107400 0x7f7a20198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1e59c700 1 -- 172.21.15.52:0/937927723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a20101d50 msgr2=0x7f7a2019ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a20101d50 0x7f7a2019ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1e59c700 1 -- 172.21.15.52:0/937927723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20107400 msgr2=0x7f7a20198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20107400 0x7f7a20198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1e59c700 1 -- 172.21.15.52:0/937927723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a201990f0 con 0x7f7a20101390 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20107400 0x7f7a20198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 0x7f7a2019e780 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d950 tx=0x7f7a1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a0f7fe700 1 -- 172.21.15.52:0/937927723 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a10016070 con 0x7f7a20101390 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a201993e0 con 0x7f7a20101390 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a0f7fe700 1 -- 172.21.15.52:0/937927723 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a10009df0 con 0x7f7a20101390 2024-06-02T23:24:36.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a0f7fe700 1 -- 172.21.15.52:0/937927723 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1000f3c0 con 0x7f7a20101390 2024-06-02T23:24:36.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.113+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a201a5650 con 0x7f7a20101390 2024-06-02T23:24:36.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.115+0000 7f7a0f7fe700 1 -- 172.21.15.52:0/937927723 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7a10019070 con 0x7f7a20101390 2024-06-02T23:24:36.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.116+0000 7f7a0f7fe700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a080a2c20 0x7f7a080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.116+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a080a2c20 0x7f7a080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.116+0000 7f7a0f7fe700 1 -- 172.21.15.52:0/937927723 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a100ceb60 con 0x7f7a20101390 2024-06-02T23:24:36.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.117+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a080a2c20 0x7f7a080a50e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7a2006b250 tx=0x7f7a14009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:36.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.117+0000 7f7a2f163b80 1 Processor -- start 2024-06-02T23:24:36.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 -- start start 2024-06-02T23:24:36.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43fc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d43f6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f7030 0x5608d4403210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5608d42e4f60 con 0x5608d43fc470 2024-06-02T23:24:36.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608d4343850 con 0x5608d43f7030 2024-06-02T23:24:36.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5608d42ab460 con 0x5608d43f6670 2024-06-02T23:24:36.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43fc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43fc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48862/0 (socket says 172.21.15.52:48862) 2024-06-02T23:24:36.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a1e59c700 1 -- 172.21.15.52:0/1343595338 learned_addr learned my addr 172.21.15.52:0/1343595338 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f7030 0x5608d4403210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d43f6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.118+0000 7f7a1ed9d700 1 -- 172.21.15.52:0/1343595338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 msgr2=0x5608d43f6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d43f6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1ed9d700 1 -- 172.21.15.52:0/1343595338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 msgr2=0x5608d43fc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43fc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1ed9d700 1 -- 172.21.15.52:0/1343595338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d42928d0 con 0x5608d43f7030 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d43f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:36.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f7030 0x5608d4403210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800d010 tx=0x7f7a18009360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccd76f08741ad490 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:36.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.119+0000 7f7a0cff9700 1 -- 172.21.15.52:0/1343595338 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1800bcb0 con 0x5608d43f7030 2024-06-02T23:24:36.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.120+0000 7f7a0cff9700 1 -- 172.21.15.52:0/1343595338 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a180074e0 con 0x5608d43f7030 2024-06-02T23:24:36.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.120+0000 7f7a0cff9700 1 -- 172.21.15.52:0/1343595338 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a18025680 con 0x5608d43f7030 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.120+0000 7f7a2f163b80 1 -- 172.21.15.52:0/1343595338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f7030 msgr2=0x5608d4403210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.120+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f7030 0x5608d4403210 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f7a1800d010 tx=0x7f7a18009360 comp rx=0 tx=0).stop 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 -- 172.21.15.52:0/1343595338 shutdown_connections 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f7030 0x5608d4403210 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43fc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/1343595338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d43f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 -- 172.21.15.52:0/1343595338 >> 172.21.15.52:0/1343595338 conn(0x5608d431f080 msgr2=0x5608d43ed220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:36.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 -- 172.21.15.52:0/1343595338 shutdown_connections 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 -- 172.21.15.52:0/1343595338 wait complete. 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 -- start start 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.121+0000 7f7a2f163b80 1 Processor -- start 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4402f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d4400280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d44007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5608d42e3fc0 con 0x5608d43f7030 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5608d433f810 con 0x5608d43fc470 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5608d4403750 con 0x5608d43f6670 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4402f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d4400280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4402f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43024/0 (socket says 172.21.15.52:43024) 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3727178297 learned_addr learned my addr 172.21.15.52:0/3727178297 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:36.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a2f163b80 10 client.0 _ll_register_callbacks cb 0x5608d42d95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d44007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 msgr2=0x5608d44007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d44007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3727178297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 msgr2=0x5608d4400280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d4400280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3727178297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d4309750 con 0x5608d43f6670 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d44007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d4400280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.122+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4402f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7a100970d0 tx=0x7f7a1009bbc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ebf57066cd93539 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:36.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a05ffb700 1 -- 172.21.15.52:0/3727178297 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100b4060 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a05ffb700 1 -- 172.21.15.52:0/3727178297 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a1009b650 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a180074e0 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a05ffb700 1 -- 172.21.15.52:0/3727178297 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a10020040 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 10 client.28105 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608d440ded0 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a05ffb700 1 -- 172.21.15.52:0/3727178297 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a180074e0 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a05ffb700 1 -- 172.21.15.52:0/3727178297 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7a10066260 con 0x5608d43f6670 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 10 client.28105 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 10 client.28105 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 20 client.28105 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.123+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d4412960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.124+0000 7f7a2f163b80 4 client.28105 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:36.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.124+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608d4350d50 con 0x5608d4410510 2024-06-02T23:24:36.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.124+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d4412960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.124+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d4412960 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=35afb504ee6c9cd0 server_cookie=cba1496f6536aa18 in_seq=0 out_seq=0 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.124+0000 7f7a05ffb700 10 client.28105 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.125+0000 7f7a05ffb700 1 -- 172.21.15.52:0/3727178297 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5608d4350d50 con 0x5608d4410510 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.125+0000 7f7a05ffb700 10 client.28105 handle_command_reply: tid=0 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.125+0000 7f7a2f163b80 1 client.28105 shutdown 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 msgr2=0x5608d4402f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4402f80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7a100970d0 tx=0x7f7a1009bbc0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 shutdown_connections 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d4412960 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a05ffb700 0 client.28105 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d44007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4402f80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3727178297 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d4400280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 >> 172.21.15.52:0/3727178297 conn(0x5608d431f080 msgr2=0x5608d43ec700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 shutdown_connections 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3727178297 wait complete. 2024-06-02T23:24:36.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.126+0000 7f7a2f163b80 20 client.28105 trim_cache size 0 max 16384 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 Processor -- start 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 -- start start 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 0x5608d43ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f7030 0x5608d440c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43ece80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201aa7f0 con 0x5608d43fc470 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ac330 con 0x5608d43f6670 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201aaec0 con 0x5608d43f7030 2024-06-02T23:24:36.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a1ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43ece80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 0x5608d43ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a1ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43ece80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48882/0 (socket says 172.21.15.52:48882) 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a1e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 0x5608d43ff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58306/0 (socket says 172.21.15.52:58306) 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a1ed9d700 1 -- 172.21.15.52:0/3541072574 learned_addr learned my addr 172.21.15.52:0/3541072574 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.143+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f7030 0x5608d440c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3541072574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f7030 msgr2=0x5608d440c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f7030 0x5608d440c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3541072574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 msgr2=0x5608d43ece80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43ece80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1e59c700 1 -- 172.21.15.52:0/3541072574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608d43fbd30 con 0x5608d43f6670 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:36.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 0x5608d43ff1e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7a10097850 tx=0x7f7a1009f920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=823655ef71b3476 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:36.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a07fff700 1 -- 172.21.15.52:0/3541072574 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a100b4040 con 0x5608d43f6670 2024-06-02T23:24:36.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a07fff700 1 -- 172.21.15.52:0/3541072574 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a100662a0 con 0x5608d43f6670 2024-06-02T23:24:36.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.144+0000 7f7a07fff700 1 -- 172.21.15.52:0/3541072574 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a1009b650 con 0x5608d43f6670 2024-06-02T23:24:36.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3541072574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 msgr2=0x5608d43ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 0x5608d43ff1e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7a10097850 tx=0x7f7a1009f920 comp rx=0 tx=0).stop 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3541072574 shutdown_connections 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43fc470 0x5608d43ece80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43f6670 0x5608d43ff1e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/3541072574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f7030 0x5608d440c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3541072574 >> 172.21.15.52:0/3541072574 conn(0x5608d431f080 msgr2=0x5608d43f1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3541072574 shutdown_connections 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 -- 172.21.15.52:0/3541072574 wait complete. 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 -- start start 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.145+0000 7f7a2f163b80 1 Processor -- start 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4408b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d44090d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d4417860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201abc60 con 0x5608d43f7030 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201ab590 con 0x5608d43fc470 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a201a8790 con 0x5608d43f6670 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a2f163b80 10 client.0 _ll_register_callbacks cb 0x5608d42d95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d44090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d44090d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48898/0 (socket says 172.21.15.52:48898) 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 -- 172.21.15.52:0/4137019174 learned_addr learned my addr 172.21.15.52:0/4137019174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1ed9d700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d4417860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4408b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 -- 172.21.15.52:0/4137019174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 msgr2=0x5608d4408b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4408b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 -- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 msgr2=0x5608d4417860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d4417860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.146+0000 7f7a1dd9b700 1 -- 172.21.15.52:0/4137019174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608d44180a0 con 0x5608d43f7030 2024-06-02T23:24:36.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a1dd9b700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d44090d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x5608d43ede10 tx=0x7f7a140047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce9c90c3d774b7d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a05ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a05ffb700 1 -- 172.21.15.52:0/4137019174 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a14004030 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a100662a0 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a05ffb700 1 -- 172.21.15.52:0/4137019174 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7a140041d0 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a05ffb700 1 -- 172.21.15.52:0/4137019174 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f7a14039070 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a2f163b80 10 client.19005 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5608d447ae00 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a05ffb700 1 -- 172.21.15.52:0/4137019174 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7a100662a0 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.147+0000 7f7a05ffb700 1 -- 172.21.15.52:0/4137019174 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f7a140038e0 con 0x5608d43f7030 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 10 client.19005 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 10 client.19005 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 20 client.19005 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:36.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d447d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:36.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 4 client.19005 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:24:36.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608d434b2d0 con 0x5608d4410510 2024-06-02T23:24:36.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d447d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a05ffb700 10 client.19005 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a1e59c700 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d447d560 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bcff311537e4ffc5 server_cookie=5363469be7d69fe in_seq=0 out_seq=0 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a05ffb700 1 -- 172.21.15.52:0/4137019174 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608d434b2d0 con 0x5608d4410510 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a05ffb700 10 client.19005 handle_command_reply: tid=0 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.148+0000 7f7a2f163b80 1 client.19005 shutdown 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 msgr2=0x5608d44090d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d44090d0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x5608d43ede10 tx=0x7f7a140047c0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 shutdown_connections 2024-06-02T23:24:36.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5608d4410510 0x5608d447d560 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a05ffb700 0 client.19005 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5608d43fc470 0x5608d4417860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608d43f6670 0x5608d4408b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 --2- 172.21.15.52:0/4137019174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608d43f7030 0x5608d44090d0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 >> 172.21.15.52:0/4137019174 conn(0x5608d431f080 msgr2=0x5608d43ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 shutdown_connections 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 1 -- 172.21.15.52:0/4137019174 wait complete. 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a2f163b80 20 client.19005 trim_cache size 0 max 16384 2024-06-02T23:24:36.154 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a080a2c20 msgr2=0x7f7a080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.149+0000 7f7a248e0700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a080a2c20 0x7f7a080a50e0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f7a2006b250 tx=0x7f7a14009260 comp rx=0 tx=0).stop 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 msgr2=0x7f7a2019e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 0x7f7a2019e780 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f7a1000d950 tx=0x7f7a1000dc60 comp rx=0 tx=0).stop 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 shutdown_connections 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7a080a2c20 0x7f7a080a50e0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a20107400 0x7f7a20198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7a20101390 0x7f7a2019e780 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 --2- 172.21.15.52:0/937927723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a20101d50 0x7f7a2019ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:36.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 >> 172.21.15.52:0/937927723 conn(0x7f7a20078340 msgr2=0x7f7a20105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:36.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 shutdown_connections 2024-06-02T23:24:36.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:36.150+0000 7f7a248e0700 1 -- 172.21.15.52:0/937927723 wait complete. 2024-06-02T23:24:36.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 250201 1.86 MB/sec execute 585 sec latency 15.386 ms 2024-06-02T23:24:36.897 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:24:37.172 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:37.557 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:37 smithi052 ceph-mon[32340]: pgmap v813: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-02T23:24:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:37 smithi174 ceph-mon[35994]: pgmap v813: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-02T23:24:37.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 250779 1.86 MB/sec execute 586 sec latency 14.018 ms 2024-06-02T23:24:37.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:37 smithi123 ceph-mon[35977]: pgmap v813: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 307 KiB/s rd, 1.5 MiB/s wr, 182 op/s 2024-06-02T23:24:38.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 251530 1.87 MB/sec execute 587 sec latency 15.966 ms 2024-06-02T23:24:38.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.926+0000 7fb62de8d700 1 -- 172.21.15.52:0/3231000377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 msgr2=0x7fb6281029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.926+0000 7fb62de8d700 1 --2- 172.21.15.52:0/3231000377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb6281029a0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009990 tx=0x7fb61c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.926+0000 7fb62de8d700 1 -- 172.21.15.52:0/3231000377 shutdown_connections 2024-06-02T23:24:38.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.926+0000 7fb62de8d700 1 --2- 172.21.15.52:0/3231000377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.926+0000 7fb62de8d700 1 --2- 172.21.15.52:0/3231000377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 0x7fb628107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.926+0000 7fb62de8d700 1 --2- 172.21.15.52:0/3231000377 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb6281029a0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.927+0000 7fb62de8d700 1 -- 172.21.15.52:0/3231000377 >> 172.21.15.52:0/3231000377 conn(0x7fb6280795f0 msgr2=0x7fb628079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:38.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.927+0000 7fb62de8d700 1 -- 172.21.15.52:0/3231000377 shutdown_connections 2024-06-02T23:24:38.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.927+0000 7fb62de8d700 1 -- 172.21.15.52:0/3231000377 wait complete. 2024-06-02T23:24:38.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.927+0000 7fb62de8d700 1 Processor -- start 2024-06-02T23:24:38.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 -- start start 2024-06-02T23:24:38.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb62819e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 0x7fb628198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb628112690 con 0x7fb628102540 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb628112510 con 0x7fb628102ee0 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb62de8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb628112810 con 0x7fb628107450 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb626ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb6277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb62819e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb626ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58336/0 (socket says 172.21.15.52:58336) 2024-06-02T23:24:38.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb626ffd700 1 -- 172.21.15.52:0/435072879 learned_addr learned my addr 172.21.15.52:0/435072879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:38.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.928+0000 7fb627fff700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 0x7fb628198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb627fff700 1 -- 172.21.15.52:0/435072879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 msgr2=0x7fb62819ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb627fff700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62819ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb627fff700 1 -- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 msgr2=0x7fb62819e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb627fff700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb62819e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb627fff700 1 -- 172.21.15.52:0/435072879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6281990f0 con 0x7fb628107450 2024-06-02T23:24:38.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb6277fe700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb62819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:38.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb626ffd700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:38.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb627fff700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 0x7fb628198990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb62000efc0 tx=0x7fb62000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:38.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb624ff9700 1 -- 172.21.15.52:0/435072879 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6200092e0 con 0x7fb628107450 2024-06-02T23:24:38.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb624ff9700 1 -- 172.21.15.52:0/435072879 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb620004bb0 con 0x7fb628107450 2024-06-02T23:24:38.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb624ff9700 1 -- 172.21.15.52:0/435072879 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620004020 con 0x7fb628107450 2024-06-02T23:24:38.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb628199380 con 0x7fb628107450 2024-06-02T23:24:38.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.929+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6281997b0 con 0x7fb628107450 2024-06-02T23:24:38.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.932+0000 7fb624ff9700 1 -- 172.21.15.52:0/435072879 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb62001b030 con 0x7fb628107450 2024-06-02T23:24:38.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.932+0000 7fb624ff9700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb6100a2ca0 0x7fb6100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.933+0000 7fb624ff9700 1 -- 172.21.15.52:0/435072879 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb6200cea00 con 0x7fb628107450 2024-06-02T23:24:38.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.933+0000 7fb6277fe700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb6100a2ca0 0x7fb6100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.933+0000 7fb6277fe700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb6100a2ca0 0x7fb6100a5160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb628199ad0 tx=0x7fb61800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:38.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 Processor -- start 2024-06-02T23:24:38.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 -- start start 2024-06-02T23:24:38.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733640 0x559b0b733a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733f60 0x559b0b72d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559b0b67f810 con 0x559b0b72df20 2024-06-02T23:24:38.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b0b627720 con 0x559b0b733640 2024-06-02T23:24:38.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb638710b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b0b67b7d0 con 0x559b0b733f60 2024-06-02T23:24:38.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb627fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb627fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48942/0 (socket says 172.21.15.52:48942) 2024-06-02T23:24:38.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb627fff700 1 -- 172.21.15.52:0/1455306087 learned_addr learned my addr 172.21.15.52:0/1455306087 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:38.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733640 0x559b0b733a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb626ffd700 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733f60 0x559b0b72d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb6277fe700 1 -- 172.21.15.52:0/1455306087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733f60 msgr2=0x559b0b72d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733f60 0x559b0b72d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb6277fe700 1 -- 172.21.15.52:0/1455306087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 msgr2=0x559b0b72e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb6277fe700 1 -- 172.21.15.52:0/1455306087 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0b61df50 con 0x559b0b733640 2024-06-02T23:24:38.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.934+0000 7fb627fff700 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.935+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733640 0x559b0b733a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb618011010 tx=0x7fb618013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3fbca053f2ba9d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.935+0000 7fb60dffb700 1 -- 172.21.15.52:0/1455306087 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61801e620 con 0x559b0b733640 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.935+0000 7fb60dffb700 1 -- 172.21.15.52:0/1455306087 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb61801ec70 con 0x559b0b733640 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.935+0000 7fb60dffb700 1 -- 172.21.15.52:0/1455306087 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61801ccb0 con 0x559b0b733640 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 -- 172.21.15.52:0/1455306087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733640 msgr2=0x559b0b733a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733640 0x559b0b733a20 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fb618011010 tx=0x7fb618013000 comp rx=0 tx=0).stop 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 -- 172.21.15.52:0/1455306087 shutdown_connections 2024-06-02T23:24:38.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733640 0x559b0b733a20 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 --2- 172.21.15.52:0/1455306087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733f60 0x559b0b72d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 -- 172.21.15.52:0/1455306087 >> 172.21.15.52:0/1455306087 conn(0x559b0b658cc0 msgr2=0x559b0b728850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 -- 172.21.15.52:0/1455306087 shutdown_connections 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 -- 172.21.15.52:0/1455306087 wait complete. 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 -- start start 2024-06-02T23:24:38.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.936+0000 7fb638710b80 1 Processor -- start 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b72df20 0x559b0b739d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b737050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 0x559b0b737590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559b0b73a9d0 con 0x559b0b733f60 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559b0b61fbf0 con 0x559b0b72df20 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b0b73ab50 con 0x559b0b733640 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb627fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 0x559b0b737590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b737050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb638710b80 10 client.0 _ll_register_callbacks cb 0x559b0b619920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b737050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43078/0 (socket says 172.21.15.52:43078) 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb627fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 0x559b0b737590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48948/0 (socket says 172.21.15.52:48948) 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 -- 172.21.15.52:0/2646626766 learned_addr learned my addr 172.21.15.52:0/2646626766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb6277fe700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b72df20 0x559b0b739d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 -- 172.21.15.52:0/2646626766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b72df20 msgr2=0x559b0b739d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b72df20 0x559b0b739d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 -- 172.21.15.52:0/2646626766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 msgr2=0x559b0b737590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 0x559b0b737590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb626ffd700 1 -- 172.21.15.52:0/2646626766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0b5ce8d0 con 0x559b0b733640 2024-06-02T23:24:38.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.937+0000 7fb627fff700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 0x559b0b737590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb626ffd700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b737050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b750 tx=0x7fb61c0050b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5c95532facbd0171 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb606ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb606ffd700 1 -- 172.21.15.52:0/2646626766 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c003880 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb606ffd700 1 -- 172.21.15.52:0/2646626766 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb61c003a20 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb61801ec70 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb606ffd700 1 -- 172.21.15.52:0/2646626766 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb61c003c20 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb638710b80 10 client.28123 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.938+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559b0b7452d0 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb606ffd700 1 -- 172.21.15.52:0/2646626766 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb61801ec70 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb606ffd700 1 -- 172.21.15.52:0/2646626766 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb61c00f460 con 0x559b0b733640 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb638710b80 10 client.28123 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb638710b80 10 client.28123 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb638710b80 20 client.28123 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:38.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb638710b80 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b747960 0x559b0b749d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb638710b80 4 client.28123 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b0b68cd10 con 0x559b0b747960 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb6277fe700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b747960 0x559b0b749d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.939+0000 7fb606ffd700 10 client.28123 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb6277fe700 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b747960 0x559b0b749d80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b7ee1ea28484cc54 server_cookie=1ef84f370450adf7 in_seq=0 out_seq=0 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb606ffd700 1 -- 172.21.15.52:0/2646626766 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559b0b68cd10 con 0x559b0b747960 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb606ffd700 10 client.28123 handle_command_reply: tid=0 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb638710b80 1 client.28123 shutdown 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 msgr2=0x559b0b737050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb638710b80 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b737050 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb61c00b750 tx=0x7fb61c0050b0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 shutdown_connections 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb638710b80 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b733f60 0x559b0b737590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.940+0000 7fb638710b80 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b72df20 0x559b0b739d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb638710b80 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b747960 0x559b0b749d80 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb638710b80 1 --2- 172.21.15.52:0/2646626766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b737050 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb606ffd700 0 client.28123 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:24:38.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 >> 172.21.15.52:0/2646626766 conn(0x559b0b658cc0 msgr2=0x559b0b72c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:38.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 shutdown_connections 2024-06-02T23:24:38.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb638710b80 1 -- 172.21.15.52:0/2646626766 wait complete. 2024-06-02T23:24:38.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.941+0000 7fb638710b80 20 client.28123 trim_cache size 0 max 16384 2024-06-02T23:24:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 Processor -- start 2024-06-02T23:24:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 -- start start 2024-06-02T23:24:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b736950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b734a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6281aec50 con 0x559b0b72df20 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6281b0790 con 0x559b0b733f60 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb638710b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6281af320 con 0x559b0b733640 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb6277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb6277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48952/0 (socket says 172.21.15.52:48952) 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb6277fe700 1 -- 172.21.15.52:0/1974286903 learned_addr learned my addr 172.21.15.52:0/1974286903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.959+0000 7fb626ffd700 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b736950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb627fff700 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b734a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb6277fe700 1 -- 172.21.15.52:0/1974286903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 msgr2=0x559b0b736950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b736950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb6277fe700 1 -- 172.21.15.52:0/1974286903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 msgr2=0x559b0b734a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b734a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb6277fe700 1 -- 172.21.15.52:0/1974286903 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b0b731b80 con 0x559b0b72df20 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72d9e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b6a0 tx=0x7fb61800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7423dea6ed4f90a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb60d7fa700 1 -- 172.21.15.52:0/1974286903 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61801c710 con 0x559b0b72df20 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb60d7fa700 1 -- 172.21.15.52:0/1974286903 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb618005040 con 0x559b0b72df20 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.960+0000 7fb60d7fa700 1 -- 172.21.15.52:0/1974286903 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618004a60 con 0x559b0b72df20 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 -- 172.21.15.52:0/1974286903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 msgr2=0x559b0b72d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72d9e0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb61800b6a0 tx=0x7fb61800bdc0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 -- 172.21.15.52:0/1974286903 shutdown_connections 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b734a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b72d9e0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 --2- 172.21.15.52:0/1974286903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b736950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 -- 172.21.15.52:0/1974286903 >> 172.21.15.52:0/1974286903 conn(0x559b0b658cc0 msgr2=0x559b0b734340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 -- 172.21.15.52:0/1974286903 shutdown_connections 2024-06-02T23:24:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 -- 172.21.15.52:0/1974286903 wait complete. 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.961+0000 7fb638710b80 1 -- start start 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 Processor -- start 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b73fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b740350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b74e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb6281b00c0 con 0x559b0b72df20 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6281af9f0 con 0x559b0b733f60 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb6281acc10 con 0x559b0b733640 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb638710b80 10 client.0 _ll_register_callbacks cb 0x559b0b619920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b73fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b73fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48960/0 (socket says 172.21.15.52:48960) 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 -- 172.21.15.52:0/1960904650 learned_addr learned my addr 172.21.15.52:0/1960904650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb627fff700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b74e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb626ffd700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b740350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 -- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 msgr2=0x559b0b740350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b740350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 -- 172.21.15.52:0/1960904650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 msgr2=0x559b0b74e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b74e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.962+0000 7fb6277fe700 1 -- 172.21.15.52:0/1960904650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b0b74eb60 con 0x559b0b72df20 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb6277fe700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b73fe10 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb618011510 tx=0x7fb61801c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e8c9d71c95019e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb606ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb606ffd700 1 -- 172.21.15.52:0/1960904650 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618040070 con 0x559b0b72df20 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb606ffd700 1 -- 172.21.15.52:0/1960904650 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb61801c910 con 0x559b0b72df20 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb618005040 con 0x559b0b72df20 2024-06-02T23:24:38.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb606ffd700 1 -- 172.21.15.52:0/1960904650 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fb61804c050 con 0x559b0b72df20 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb638710b80 10 client.19023 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x559b0b7b1d00 con 0x559b0b72df20 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.963+0000 7fb606ffd700 1 -- 172.21.15.52:0/1960904650 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb618005040 con 0x559b0b72df20 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb606ffd700 1 -- 172.21.15.52:0/1960904650 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fb61801e420 con 0x559b0b72df20 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 10 client.19023 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 10 client.19023 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 20 client.19023 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b7b2190 0x559b0b7b45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 4 client.19023 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559b0b687290 con 0x559b0b7b2190 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb626ffd700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b7b2190 0x559b0b7b45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb606ffd700 10 client.19023 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb626ffd700 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b7b2190 0x559b0b7b45b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7ddc978f43540754 server_cookie=ce2372af39f21785 in_seq=0 out_seq=0 2024-06-02T23:24:38.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb606ffd700 1 -- 172.21.15.52:0/1960904650 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559b0b687290 con 0x559b0b7b2190 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb606ffd700 10 client.19023 handle_command_reply: tid=0 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.964+0000 7fb638710b80 1 client.19023 shutdown 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 msgr2=0x559b0b73fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b73fe10 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb618011510 tx=0x7fb61801c710 comp rx=0 tx=0).stop 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 shutdown_connections 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559b0b733f60 0x559b0b74e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559b0b72df20 0x559b0b73fe10 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559b0b7b2190 0x559b0b7b45b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 --2- 172.21.15.52:0/1960904650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b0b733640 0x559b0b740350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb606ffd700 0 client.19023 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 >> 172.21.15.52:0/1960904650 conn(0x559b0b658cc0 msgr2=0x559b0b734340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 shutdown_connections 2024-06-02T23:24:38.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 1 -- 172.21.15.52:0/1960904650 wait complete. 2024-06-02T23:24:38.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.965+0000 7fb638710b80 20 client.19023 trim_cache size 0 max 16384 2024-06-02T23:24:38.971 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:24:38.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb6100a2ca0 msgr2=0x7fb6100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb6100a2ca0 0x7fb6100a5160 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fb628199ad0 tx=0x7fb61800b410 comp rx=0 tx=0).stop 2024-06-02T23:24:38.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 msgr2=0x7fb628198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 0x7fb628198990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb62000efc0 tx=0x7fb62000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 shutdown_connections 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb6100a2ca0 0x7fb6100a5160 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb628107450 0x7fb628198990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb628102540 0x7fb62819e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 --2- 172.21.15.52:0/435072879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb628102ee0 0x7fb62819ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 >> 172.21.15.52:0/435072879 conn(0x7fb6280795f0 msgr2=0x7fb6281006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 shutdown_connections 2024-06-02T23:24:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:38.966+0000 7fb62de8d700 1 -- 172.21.15.52:0/435072879 wait complete. 2024-06-02T23:24:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:39 smithi174 ceph-mon[35994]: pgmap v814: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:24:39.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:39 smithi052 ceph-mon[32340]: pgmap v814: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:24:39.654 INFO:tasks.workunit.client.0.smithi052.stdout: 1 251956 1.87 MB/sec execute 588 sec latency 37.553 ms 2024-06-02T23:24:39.662 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:24:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:39 smithi123 ceph-mon[35977]: pgmap v814: 97 pgs: 97 active+clean; 105 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:24:39.940 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:40.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 252255 1.87 MB/sec execute 589 sec latency 15.112 ms 2024-06-02T23:24:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:41 smithi174 ceph-mon[35994]: pgmap v815: 97 pgs: 97 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:24:41.645 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:41 smithi052 ceph-mon[32340]: pgmap v815: 97 pgs: 97 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:24:41.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 252534 1.87 MB/sec execute 590 sec latency 20.053 ms 2024-06-02T23:24:41.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 -- 172.21.15.52:0/2598154132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 msgr2=0x7f55d810ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 --2- 172.21.15.52:0/2598154132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d810ea00 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f55c8009990 tx=0x7f55c8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 -- 172.21.15.52:0/2598154132 shutdown_connections 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 --2- 172.21.15.52:0/2598154132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d810ef40 0x7f55d8111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 --2- 172.21.15.52:0/2598154132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d806a1a0 0x7f55d806a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 --2- 172.21.15.52:0/2598154132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d810ea00 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 -- 172.21.15.52:0/2598154132 >> 172.21.15.52:0/2598154132 conn(0x7f55d8078340 msgr2=0x7f55d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 -- 172.21.15.52:0/2598154132 shutdown_connections 2024-06-02T23:24:41.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.672+0000 7f55dfab9700 1 -- 172.21.15.52:0/2598154132 wait complete. 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 Processor -- start 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 -- start start 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d806a1a0 0x7f55d819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d810ef40 0x7f55d819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8114850 con 0x7f55d806aac0 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81146d0 con 0x7f55d810ef40 2024-06-02T23:24:41.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dfab9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81149d0 con 0x7f55d806a1a0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dd054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dd054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d819b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48978/0 (socket says 172.21.15.52:48978) 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dd054700 1 -- 172.21.15.52:0/3244861217 learned_addr learned my addr 172.21.15.52:0/3244861217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55dd855700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d806a1a0 0x7f55d819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.673+0000 7f55de056700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d810ef40 0x7f55d819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd054700 1 -- 172.21.15.52:0/3244861217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d806a1a0 msgr2=0x7f55d819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd054700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d806a1a0 0x7f55d819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd054700 1 -- 172.21.15.52:0/3244861217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d810ef40 msgr2=0x7f55d819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd054700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d810ef40 0x7f55d819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd054700 1 -- 172.21.15.52:0/3244861217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d819fdb0 con 0x7f55d806aac0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd855700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d806a1a0 0x7f55d819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dd054700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d819b2e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f55c8003ab0 tx=0x7f55c8003ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55ceffd700 1 -- 172.21.15.52:0/3244861217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8027070 con 0x7f55d806aac0 2024-06-02T23:24:41.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81a0040 con 0x7f55d806aac0 2024-06-02T23:24:41.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55ceffd700 1 -- 172.21.15.52:0/3244861217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55c800bdc0 con 0x7f55d806aac0 2024-06-02T23:24:41.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55ceffd700 1 -- 172.21.15.52:0/3244861217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8022bf0 con 0x7f55d806aac0 2024-06-02T23:24:41.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.674+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a7920 con 0x7f55d806aac0 2024-06-02T23:24:41.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.677+0000 7f55ceffd700 1 -- 172.21.15.52:0/3244861217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f55c802a070 con 0x7f55d806aac0 2024-06-02T23:24:41.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.678+0000 7f55ceffd700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f55c40a2c20 0x7f55c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.678+0000 7f55dd855700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f55c40a2c20 0x7f55c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.678+0000 7f55ceffd700 1 -- 172.21.15.52:0/3244861217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f55c80cfe70 con 0x7f55d806aac0 2024-06-02T23:24:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55dd855700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f55c40a2c20 0x7f55c40a50e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005950 tx=0x7f55d400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 Processor -- start 2024-06-02T23:24:41.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 -- start start 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x56392702d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927027260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927027bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563926f19f60 con 0x5639270277a0 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.679+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563926f78850 con 0x56392702cf60 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563926ee0460 con 0x56392702d880 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x56392702d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x56392702d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58416/0 (socket says 172.21.15.52:58416) 2024-06-02T23:24:41.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 -- 172.21.15.52:0/3754257286 learned_addr learned my addr 172.21.15.52:0/3754257286 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55de056700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927027bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd054700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927027260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 -- 172.21.15.52:0/3754257286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 msgr2=0x563927027260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927027260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 -- 172.21.15.52:0/3754257286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 msgr2=0x563927027bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927027bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 -- 172.21.15.52:0/3754257286 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563926ec78d0 con 0x56392702cf60 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd054700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927027260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55de056700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927027bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55dd855700 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x56392702d340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f55d4011790 tx=0x7f55d4013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=284bdb4ef7277e78 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.680+0000 7f55c3fff700 1 -- 172.21.15.52:0/3754257286 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4010ea0 con 0x56392702cf60 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55c3fff700 1 -- 172.21.15.52:0/3754257286 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55d401ead0 con 0x56392702cf60 2024-06-02T23:24:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55c3fff700 1 -- 172.21.15.52:0/3754257286 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d401cbd0 con 0x56392702cf60 2024-06-02T23:24:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 -- 172.21.15.52:0/3754257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 msgr2=0x56392702d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x56392702d340 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f55d4011790 tx=0x7f55d4013000 comp rx=0 tx=0).stop 2024-06-02T23:24:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 -- 172.21.15.52:0/3754257286 shutdown_connections 2024-06-02T23:24:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927027bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x56392702d340 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/3754257286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927027260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 -- 172.21.15.52:0/3754257286 >> 172.21.15.52:0/3754257286 conn(0x563926f54080 msgr2=0x563927022220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 -- 172.21.15.52:0/3754257286 shutdown_connections 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.681+0000 7f55ea33cb80 1 -- 172.21.15.52:0/3754257286 wait complete. 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 -- start start 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 Processor -- start 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639270277a0 0x563927033630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 0x563927030930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927030e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563926f18fc0 con 0x56392702cf60 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.682+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563926f74810 con 0x5639270277a0 2024-06-02T23:24:41.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563927034260 con 0x56392702d880 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55ea33cb80 10 client.0 _ll_register_callbacks cb 0x563926f0e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 0x563927030930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 0x563927030930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48990/0 (socket says 172.21.15.52:48990) 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd054700 1 -- 172.21.15.52:0/217745946 learned_addr learned my addr 172.21.15.52:0/217745946 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639270277a0 0x563927033630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55de056700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927030e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 -- 172.21.15.52:0/217745946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 msgr2=0x563927030e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927030e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 -- 172.21.15.52:0/217745946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 msgr2=0x563927030930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 0x563927030930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 -- 172.21.15.52:0/217745946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563926f3e750 con 0x5639270277a0 2024-06-02T23:24:41.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55de056700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927030e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd054700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 0x563927030930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55dd855700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639270277a0 0x563927033630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f55d4011790 tx=0x7f55d4010ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3285f7b528a3f1a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55c17fa700 1 -- 172.21.15.52:0/217745946 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4035060 con 0x5639270277a0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d401ead0 con 0x5639270277a0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.683+0000 7f55c17fa700 1 -- 172.21.15.52:0/217745946 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f55d401e630 con 0x5639270277a0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55c17fa700 1 -- 172.21.15.52:0/217745946 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55d401e830 con 0x5639270277a0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 10 client.28445 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56392703e9c0 con 0x5639270277a0 2024-06-02T23:24:41.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55c17fa700 1 -- 172.21.15.52:0/217745946 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55d401ead0 con 0x5639270277a0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55c17fa700 1 -- 172.21.15.52:0/217745946 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f55d401cb70 con 0x5639270277a0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 10 client.28445 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 10 client.28445 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 20 client.28445 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x563927040ef0 0x5639270432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 4 client.28445 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563926f85d50 con 0x563927040ef0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.684+0000 7f55dd054700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x563927040ef0 0x5639270432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.685+0000 7f55c17fa700 10 client.28445 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.685+0000 7f55dd054700 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x563927040ef0 0x5639270432e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d1b74c30eafc80c5 server_cookie=e81dbf02515afcce in_seq=0 out_seq=0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.685+0000 7f55c17fa700 1 -- 172.21.15.52:0/217745946 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563926f85d50 con 0x563927040ef0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.685+0000 7f55c17fa700 10 client.28445 handle_command_reply: tid=0 2024-06-02T23:24:41.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.685+0000 7f55ea33cb80 1 client.28445 shutdown 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639270277a0 msgr2=0x563927033630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639270277a0 0x563927033630 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f55d4011790 tx=0x7f55d4010ea0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 shutdown_connections 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x563927040ef0 0x5639270432e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702d880 0x563927030e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5639270277a0 0x563927033630 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/217745946 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56392702cf60 0x563927030930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55c17fa700 0 client.28445 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:24:41.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 >> 172.21.15.52:0/217745946 conn(0x563926f54080 msgr2=0x563927021700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:41.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 shutdown_connections 2024-06-02T23:24:41.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 1 -- 172.21.15.52:0/217745946 wait complete. 2024-06-02T23:24:41.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.686+0000 7f55ea33cb80 20 client.28445 trim_cache size 0 max 16384 2024-06-02T23:24:41.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 Processor -- start 2024-06-02T23:24:41.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 -- start start 2024-06-02T23:24:41.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563926f54080 0x56392702e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927021e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702cf60 0x5639270223c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81aeca0 con 0x5639270277a0 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b07e0 con 0x563926f54080 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81af370 con 0x56392702cf60 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55dd054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927021e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55dd054700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927021e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49000/0 (socket says 172.21.15.52:49000) 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55dd054700 1 -- 172.21.15.52:0/226597747 learned_addr learned my addr 172.21.15.52:0/226597747 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55de056700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702cf60 0x5639270223c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.704+0000 7f55dd855700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563926f54080 0x56392702e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55de056700 1 -- 172.21.15.52:0/226597747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563926f54080 msgr2=0x56392702e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55de056700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563926f54080 0x56392702e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55de056700 1 -- 172.21.15.52:0/226597747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 msgr2=0x563927021e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55de056700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927021e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55de056700 1 -- 172.21.15.52:0/226597747 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639270275a0 con 0x56392702cf60 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55dd054700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927021e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55dd855700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563926f54080 0x56392702e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55de056700 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702cf60 0x5639270223c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55d0003f50 tx=0x7f55d0003f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba43c12a8399219 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55c37fe700 1 -- 172.21.15.52:0/226597747 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d001a070 con 0x56392702cf60 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55c37fe700 1 -- 172.21.15.52:0/226597747 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55d0004db0 con 0x56392702cf60 2024-06-02T23:24:41.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.705+0000 7f55c37fe700 1 -- 172.21.15.52:0/226597747 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0007c50 con 0x56392702cf60 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 -- 172.21.15.52:0/226597747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702cf60 msgr2=0x5639270223c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702cf60 0x5639270223c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f55d0003f50 tx=0x7f55d0003f80 comp rx=0 tx=0).stop 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 -- 172.21.15.52:0/226597747 shutdown_connections 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56392702cf60 0x5639270223c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563926f54080 0x56392702e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/226597747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5639270277a0 0x563927021e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 -- 172.21.15.52:0/226597747 >> 172.21.15.52:0/226597747 conn(0x56392702b270 msgr2=0x56392702f9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 -- 172.21.15.52:0/226597747 shutdown_connections 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 -- 172.21.15.52:0/226597747 wait complete. 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 -- start start 2024-06-02T23:24:41.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.706+0000 7f55ea33cb80 1 Processor -- start 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 0x563927042e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639270277a0 0x5639270433c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x563927043900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b0110 con 0x563926f54080 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81afa40 con 0x56392702cf60 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81acc60 con 0x5639270277a0 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 0x563927042e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55ea33cb80 10 client.0 _ll_register_callbacks cb 0x563926f0e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 0x563927042e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49008/0 (socket says 172.21.15.52:49008) 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 -- 172.21.15.52:0/208245125 learned_addr learned my addr 172.21.15.52:0/208245125 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd054700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639270277a0 0x5639270433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55de056700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x563927043900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 -- 172.21.15.52:0/208245125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639270277a0 msgr2=0x5639270433c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639270277a0 0x5639270433c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 -- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 msgr2=0x563927043900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x563927043900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 -- 172.21.15.52:0/208245125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639270a99e0 con 0x563926f54080 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd054700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639270277a0 0x5639270433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55dd855700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 0x563927042e80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f55d4004850 tx=0x7f55d401ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7b6a479910f540 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.707+0000 7f55c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55c17fa700 1 -- 172.21.15.52:0/208245125 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d4040070 con 0x563926f54080 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55c17fa700 1 -- 172.21.15.52:0/208245125 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f55d401e630 con 0x563926f54080 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d0004db0 con 0x563926f54080 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55c17fa700 1 -- 172.21.15.52:0/208245125 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f55d4035070 con 0x563926f54080 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55ea33cb80 10 client.19041 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5639270aacc0 con 0x563926f54080 2024-06-02T23:24:41.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.708+0000 7f55c17fa700 1 -- 172.21.15.52:0/208245125 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f55d0004db0 con 0x563926f54080 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55c17fa700 1 -- 172.21.15.52:0/208245125 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f55d4043020 con 0x563926f54080 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 10 client.19041 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 10 client.19041 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 20 client.19041 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56392702d880 0x5639270ad400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 4 client.19041 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563926f802d0 con 0x56392702d880 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55dd054700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56392702d880 0x5639270ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55c17fa700 10 client.19041 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55dd054700 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56392702d880 0x5639270ad400 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=93ae72bc75260304 server_cookie=b4fda9aa1f28d6a in_seq=0 out_seq=0 2024-06-02T23:24:41.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55c17fa700 1 -- 172.21.15.52:0/208245125 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563926f802d0 con 0x56392702d880 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55c17fa700 10 client.19041 handle_command_reply: tid=0 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.709+0000 7f55ea33cb80 1 client.19041 shutdown 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 msgr2=0x563927042e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 0x563927042e80 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f55d4004850 tx=0x7f55d401ec00 comp rx=0 tx=0).stop 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 shutdown_connections 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56392702d880 0x5639270ad400 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56392702cf60 0x563927043900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55c17fa700 0 client.19041 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:24:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563926f54080 0x563927042e80 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 --2- 172.21.15.52:0/208245125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5639270277a0 0x5639270433c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 >> 172.21.15.52:0/208245125 conn(0x56392702b270 msgr2=0x5639270329f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 shutdown_connections 2024-06-02T23:24:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 1 -- 172.21.15.52:0/208245125 wait complete. 2024-06-02T23:24:41.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.710+0000 7f55ea33cb80 20 client.19041 trim_cache size 0 max 16384 2024-06-02T23:24:41.718 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:24:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f55c40a2c20 msgr2=0x7f55c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f55c40a2c20 0x7f55c40a50e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f55d4005950 tx=0x7f55d400b410 comp rx=0 tx=0).stop 2024-06-02T23:24:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 msgr2=0x7f55d819b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d819b2e0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f55c8003ab0 tx=0x7f55c8003ae0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 shutdown_connections 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f55c40a2c20 0x7f55c40a50e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55d810ef40 0x7f55d819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f55d806a1a0 0x7f55d819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 --2- 172.21.15.52:0/3244861217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f55d806aac0 0x7f55d819b2e0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 >> 172.21.15.52:0/3244861217 conn(0x7f55d8078340 msgr2=0x7f55d81104b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 shutdown_connections 2024-06-02T23:24:41.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:41.711+0000 7f55dfab9700 1 -- 172.21.15.52:0/3244861217 wait complete. 2024-06-02T23:24:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:41 smithi123 ceph-mon[35977]: pgmap v815: 97 pgs: 97 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.5 MiB/s wr, 166 op/s 2024-06-02T23:24:42.556 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:24:42.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 253044 1.87 MB/sec execute 591 sec latency 16.655 ms 2024-06-02T23:24:42.840 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 -- 172.21.15.52:0/841445557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 msgr2=0x7f32d8109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/841445557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d8109b30 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f32d0009ca0 tx=0x7f32d002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 -- 172.21.15.52:0/841445557 shutdown_connections 2024-06-02T23:24:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/841445557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8104e10 0x7f32d8107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/841445557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d8109b30 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/841445557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d81024e0 0x7f32d81048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 -- 172.21.15.52:0/841445557 >> 172.21.15.52:0/841445557 conn(0x7f32d80fbe70 msgr2=0x7f32d80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 -- 172.21.15.52:0/841445557 shutdown_connections 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.906+0000 7f32dd7b0640 1 -- 172.21.15.52:0/841445557 wait complete. 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 Processor -- start 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 -- start start 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d81024e0 0x7f32d8197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8104e10 0x7f32d8198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d819f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810d530 con 0x7f32d8107740 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810d3b0 con 0x7f32d81024e0 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32dd7b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32d810d6b0 con 0x7f32d8104e10 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d819f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d819f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49020/0 (socket says 172.21.15.52:49020) 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d77fe640 1 -- 172.21.15.52:0/1348812340 learned_addr learned my addr 172.21.15.52:0/1348812340 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d81024e0 0x7f32d8197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8104e10 0x7f32d8198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d77fe640 1 -- 172.21.15.52:0/1348812340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8104e10 msgr2=0x7f32d8198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.907+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8104e10 0x7f32d8198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32d77fe640 1 -- 172.21.15.52:0/1348812340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d81024e0 msgr2=0x7f32d8197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d81024e0 0x7f32d8197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32d77fe640 1 -- 172.21.15.52:0/1348812340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32d819f770 con 0x7f32d8107740 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d819f100 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f32d0031970 tx=0x7f32d0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32bffff640 1 -- 172.21.15.52:0/1348812340 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0004040 con 0x7f32d8107740 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32bffff640 1 -- 172.21.15.52:0/1348812340 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32d0004690 con 0x7f32d8107740 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32bffff640 1 -- 172.21.15.52:0/1348812340 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d0030c60 con 0x7f32d8107740 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32d819fa00 con 0x7f32d8107740 2024-06-02T23:24:42.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.908+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32d819ffa0 con 0x7f32d8107740 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32bffff640 1 -- 172.21.15.52:0/1348812340 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f32d00041e0 con 0x7f32d8107740 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 Processor -- start 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 -- start start 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32bffff640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32c00a2cb0 0x7f32c00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 0x555893b65840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893b6d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555893ac44f0 con 0x555893b65420 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555893a62870 con 0x555893b65e10 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32e059e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555893ac04b0 con 0x555893b6af80 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32c00a2cb0 0x7f32c00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32bffff640 1 -- 172.21.15.52:0/1348812340 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32d00f0490 con 0x7f32d8107740 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.911+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 0x555893b65840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 0x555893b65840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49034/0 (socket says 172.21.15.52:49034) 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1947346084 learned_addr learned my addr 172.21.15.52:0/1947346084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893b6d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1947346084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 msgr2=0x555893b6d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893b6d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1947346084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 msgr2=0x555893b6a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1947346084 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555893a7e9b0 con 0x555893b65420 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32c00a2cb0 0x7f32c00a5170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f32d8199060 tx=0x7f32cc014950 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 0x555893b65840 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009cf0 tx=0x7f32c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9be821565a2b51f8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32bd7fa640 1 -- 172.21.15.52:0/1947346084 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8004230 con 0x555893b65420 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32bd7fa640 1 -- 172.21.15.52:0/1947346084 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80043d0 con 0x555893b65420 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.912+0000 7f32bd7fa640 1 -- 172.21.15.52:0/1947346084 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c80197e0 con 0x555893b65420 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 -- 172.21.15.52:0/1947346084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 msgr2=0x555893b65840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 0x555893b65840 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f32c8009cf0 tx=0x7f32c8011810 comp rx=0 tx=0).stop 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 -- 172.21.15.52:0/1947346084 shutdown_connections 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893b6d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 --2- 172.21.15.52:0/1947346084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65420 0x555893b65840 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 -- 172.21.15.52:0/1947346084 >> 172.21.15.52:0/1947346084 conn(0x555893b5c680 msgr2=0x555893b5ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 -- 172.21.15.52:0/1947346084 shutdown_connections 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.913+0000 7f32e059e740 1 -- 172.21.15.52:0/1947346084 wait complete. 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 -- start start 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 Processor -- start 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65420 0x555893b6f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b76e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555893b6df30 con 0x555893b6af80 2024-06-02T23:24:42.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555893a4dae0 con 0x555893b65e10 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555893b6e0b0 con 0x555893b65420 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32d67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32e059e740 10 client.0 _ll_register_callbacks cb 0x555893a869b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65420 0x555893b6f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32d67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58508/0 (socket says 172.21.15.52:58508) 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32d67fc640 1 -- 172.21.15.52:0/1915504748 learned_addr learned my addr 172.21.15.52:0/1915504748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b76e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.914+0000 7f32d67fc640 1 -- 172.21.15.52:0/1915504748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65420 msgr2=0x555893b6f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65420 0x555893b6f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32d67fc640 1 -- 172.21.15.52:0/1915504748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 msgr2=0x555893b76e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b76e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32d67fc640 1 -- 172.21.15.52:0/1915504748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558937d8210 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b76e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6fde0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f32cc014200 tx=0x7f32cc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a693a6bdb52c27 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32baff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32baff5640 1 -- 172.21.15.52:0/1915504748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc020850 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32baff5640 1 -- 172.21.15.52:0/1915504748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32cc0209f0 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c80043d0 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32baff5640 1 -- 172.21.15.52:0/1915504748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32cc020bf0 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32e059e740 10 client.28463 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5558938a4620 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.915+0000 7f32baff5640 1 -- 172.21.15.52:0/1915504748 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c80043d0 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32baff5640 1 -- 172.21.15.52:0/1915504748 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32cc02cba0 con 0x555893b65e10 2024-06-02T23:24:42.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32e059e740 10 client.28463 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32e059e740 10 client.28463 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32e059e740 20 client.28463 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32e059e740 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893b7b780 0x555893b7db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32e059e740 4 client.28463 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555893ad7f10 con 0x555893b7b780 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893b7b780 0x555893b7db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893b7b780 0x555893b7db90 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d6312a211411c2 server_cookie=d8f14d9f307c7da0 in_seq=0 out_seq=0 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.916+0000 7f32baff5640 10 client.28463 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32baff5640 1 -- 172.21.15.52:0/1915504748 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555893ad7f10 con 0x555893b7b780 2024-06-02T23:24:42.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32baff5640 10 client.28463 handle_command_reply: tid=0 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 client.28463 shutdown 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 msgr2=0x555893b6fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6fde0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f32cc014200 tx=0x7f32cc004730 comp rx=0 tx=0).stop 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 shutdown_connections 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893b7b780 0x555893b7db90 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b76e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65e10 0x555893b6fde0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 --2- 172.21.15.52:0/1915504748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65420 0x555893b6f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 >> 172.21.15.52:0/1915504748 conn(0x555893b5c680 msgr2=0x555893b5b7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32baff5640 0 client.28463 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 shutdown_connections 2024-06-02T23:24:42.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 1 -- 172.21.15.52:0/1915504748 wait complete. 2024-06-02T23:24:42.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.917+0000 7f32e059e740 20 client.28463 trim_cache size 0 max 16384 2024-06-02T23:24:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 Processor -- start 2024-06-02T23:24:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 -- start start 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b5f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65e10 0x555893b76de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b5fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81a6a60 con 0x555893b6af80 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81a85a0 con 0x555893b65420 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32e059e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81a7130 con 0x555893b65e10 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b5fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32d6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b5f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32d77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b5fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49038/0 (socket says 172.21.15.52:49038) 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32d77fe640 1 -- 172.21.15.52:0/1648076208 learned_addr learned my addr 172.21.15.52:0/1648076208 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.933+0000 7f32d67fc640 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65e10 0x555893b76de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1648076208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65e10 msgr2=0x555893b76de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65e10 0x555893b76de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1648076208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 msgr2=0x555893b5fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b5fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d6ffd640 1 -- 172.21.15.52:0/1648076208 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555893b661f0 con 0x555893b65420 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d77fe640 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b5fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b5f590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f32c801b040 tx=0x7f32c8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2330c802229e0fa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32bcff9640 1 -- 172.21.15.52:0/1648076208 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8031070 con 0x555893b65420 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32bcff9640 1 -- 172.21.15.52:0/1648076208 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c8003ac0 con 0x555893b65420 2024-06-02T23:24:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.934+0000 7f32bcff9640 1 -- 172.21.15.52:0/1648076208 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32c8004190 con 0x555893b65420 2024-06-02T23:24:42.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- 172.21.15.52:0/1648076208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 msgr2=0x555893b5f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b5f590 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f32c801b040 tx=0x7f32c8011810 comp rx=0 tx=0).stop 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- 172.21.15.52:0/1648076208 shutdown_connections 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b65e10 0x555893b76de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b6af80 0x555893b5fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- 172.21.15.52:0/1648076208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b5f590 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- 172.21.15.52:0/1648076208 >> 172.21.15.52:0/1648076208 conn(0x555893b5c680 msgr2=0x555893b671d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- 172.21.15.52:0/1648076208 shutdown_connections 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- 172.21.15.52:0/1648076208 wait complete. 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- start start 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 Processor -- start 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b73c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 0x555893b74180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893be4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81a7ed0 con 0x555893b65e10 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81a7800 con 0x555893b65420 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32d81a4a40 con 0x555893b6af80 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.935+0000 7f32e059e740 10 client.0 _ll_register_callbacks cb 0x7f32cc00b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 0x555893b74180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 0x555893b74180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49042/0 (socket says 172.21.15.52:49042) 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 -- 172.21.15.52:0/3418268651 learned_addr learned my addr 172.21.15.52:0/3418268651 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b73c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d77fe640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893be4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.941 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 -- 172.21.15.52:0/3418268651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 msgr2=0x555893be4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893be4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 -- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 msgr2=0x555893b73c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b73c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 -- 172.21.15.52:0/3418268651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555893be49a0 con 0x555893b65e10 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d77fe640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893be4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32d67fc640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 0x555893b74180 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x555893b70d80 tx=0x7f32cc020b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a019a852688f4f1d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32baff5640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32baff5640 1 -- 172.21.15.52:0/3418268651 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32cc011a90 con 0x555893b65e10 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32baff5640 1 -- 172.21.15.52:0/3418268651 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32cc00b9f0 con 0x555893b65e10 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.936+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32c8003ac0 con 0x555893b65e10 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32baff5640 1 -- 172.21.15.52:0/3418268651 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f32cc014040 con 0x555893b65e10 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 10 client.19059 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:42.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x555893be5bd0 con 0x555893b65e10 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32baff5640 1 -- 172.21.15.52:0/3418268651 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f32c8003ac0 con 0x555893b65e10 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32baff5640 1 -- 172.21.15.52:0/3418268651 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f32cc02ca30 con 0x555893b65e10 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 10 client.19059 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 10 client.19059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 20 client.19059 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893be65d0 0x555893be89f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 4 client.19059 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555893ad1c20 con 0x555893be65d0 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.937+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893be65d0 0x555893be89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.938+0000 7f32baff5640 10 client.19059 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:42.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.938+0000 7f32d6ffd640 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893be65d0 0x555893be89f0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52a6e4d44b583e4e server_cookie=d9391e97852dcd99 in_seq=0 out_seq=0 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32baff5640 1 -- 172.21.15.52:0/3418268651 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555893ad1c20 con 0x555893be65d0 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32baff5640 10 client.19059 handle_command_reply: tid=0 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 client.19059 shutdown 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 msgr2=0x555893b74180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 0x555893b74180 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x555893b70d80 tx=0x7f32cc020b70 comp rx=0 tx=0).stop 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 shutdown_connections 2024-06-02T23:24:42.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555893be65d0 0x555893be89f0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32baff5640 0 client.19059 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555893b6af80 0x555893be4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555893b65e10 0x555893b74180 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 --2- 172.21.15.52:0/3418268651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555893b65420 0x555893b73c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.965+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 >> 172.21.15.52:0/3418268651 conn(0x555893b5c680 msgr2=0x555893b5f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 shutdown_connections 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32e059e740 1 -- 172.21.15.52:0/3418268651 wait complete. 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32e059e740 20 client.19059 trim_cache size 0 max 16384 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:24:42.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32c00a2cb0 msgr2=0x7f32c00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32c00a2cb0 0x7f32c00a5170 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f32d8199060 tx=0x7f32cc014950 comp rx=0 tx=0).stop 2024-06-02T23:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 msgr2=0x7f32d819f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d819f100 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f32d0031970 tx=0x7f32d0003ba0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 shutdown_connections 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f32c00a2cb0 0x7f32c00a5170 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32d8104e10 0x7f32d8198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32d8107740 0x7f32d819f100 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 --2- 172.21.15.52:0/1348812340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32d81024e0 0x7f32d8197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 >> 172.21.15.52:0/1348812340 conn(0x7f32d80fbe70 msgr2=0x7f32d8102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 shutdown_connections 2024-06-02T23:24:42.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:42.966+0000 7f32dd7b0640 1 -- 172.21.15.52:0/1348812340 wait complete. 2024-06-02T23:24:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:43 smithi052 ceph-mon[32340]: pgmap v816: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-02T23:24:43.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 253577 1.87 MB/sec execute 592 sec latency 22.448 ms 2024-06-02T23:24:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:43 smithi123 ceph-mon[35977]: pgmap v816: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-02T23:24:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:43 smithi174 ceph-mon[35994]: pgmap v816: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 242 KiB/s rd, 1.3 MiB/s wr, 153 op/s 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 -- 172.21.15.52:0/1075611607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 msgr2=0x7f2e0010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 --2- 172.21.15.52:0/1075611607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0010f360 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f2df800b0a0 tx=0x7f2df800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 -- 172.21.15.52:0/1075611607 shutdown_connections 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 --2- 172.21.15.52:0/1075611607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0010f360 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 --2- 172.21.15.52:0/1075611607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 0x7f2e00108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 --2- 172.21.15.52:0/1075611607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e00102510 0x7f2e00102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 -- 172.21.15.52:0/1075611607 >> 172.21.15.52:0/1075611607 conn(0x7f2e00079560 msgr2=0x7f2e00079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 -- 172.21.15.52:0/1075611607 shutdown_connections 2024-06-02T23:24:44.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.650+0000 7f2e049dc700 1 -- 172.21.15.52:0/1075611607 wait complete. 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.651+0000 7f2e049dc700 1 Processor -- start 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.651+0000 7f2e049dc700 1 -- start start 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.651+0000 7f2e049dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e00102510 0x7f2e0019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.651+0000 7f2e049dc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.651+0000 7f2e049dc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 0x7f2e00198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2e049dc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e00112650 con 0x7f2e00102ed0 2024-06-02T23:24:44.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2e049dc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001124d0 con 0x7f2e00108520 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2e049dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001127d0 con 0x7f2e00102510 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56832/0 (socket says 172.21.15.52:56832) 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfdd9b700 1 -- 172.21.15.52:0/661208586 learned_addr learned my addr 172.21.15.52:0/661208586 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 0x7f2e00198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e00102510 0x7f2e0019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 -- 172.21.15.52:0/661208586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e00102510 msgr2=0x7f2e0019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e00102510 0x7f2e0019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 -- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 msgr2=0x7f2e0019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 -- 172.21.15.52:0/661208586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e00198ff0 con 0x7f2e00108520 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:44.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 0x7f2e00198890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2df800d010 tx=0x7f2df8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:44.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2def7fe700 1 -- 172.21.15.52:0/661208586 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8021070 con 0x7f2e00108520 2024-06-02T23:24:44.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.652+0000 7f2def7fe700 1 -- 172.21.15.52:0/661208586 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df8007e30 con 0x7f2e00108520 2024-06-02T23:24:44.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.653+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e00199280 con 0x7f2e00108520 2024-06-02T23:24:44.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.653+0000 7f2def7fe700 1 -- 172.21.15.52:0/661208586 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df8004590 con 0x7f2e00108520 2024-06-02T23:24:44.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.653+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e001a55a0 con 0x7f2e00108520 2024-06-02T23:24:44.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 254223 1.87 MB/sec execute 593 sec latency 16.056 ms 2024-06-02T23:24:44.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.655+0000 7f2def7fe700 1 -- 172.21.15.52:0/661208586 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2df8003c90 con 0x7f2e00108520 2024-06-02T23:24:44.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.655+0000 7f2def7fe700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2de80a2c20 0x7f2de80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.656+0000 7f2def7fe700 1 -- 172.21.15.52:0/661208586 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2df80cfe50 con 0x7f2e00108520 2024-06-02T23:24:44.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.656+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2de80a2c20 0x7f2de80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.656+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2de80a2c20 0x7f2de80a50e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2e001999d0 tx=0x7f2df0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:44.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 Processor -- start 2024-06-02T23:24:44.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 -- start start 2024-06-02T23:24:44.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55aa57e47f60 con 0x55aa57f5b880 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa57ea6850 con 0x55aa57f557a0 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.657+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa57e0e460 con 0x55aa57f5af60 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38498/0 (socket says 172.21.15.52:38498) 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfe59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2276560016 learned_addr learned my addr 172.21.15.52:0/2276560016 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2276560016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 msgr2=0x55aa57f5b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2276560016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 msgr2=0x55aa57f55260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f55260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2276560016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa57df58d0 con 0x55aa57f557a0 2024-06-02T23:24:44.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2df809c470 tx=0x7f2df8009040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba7ed0f54d98d3f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:44.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2decff9700 1 -- 172.21.15.52:0/2276560016 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df802a070 con 0x55aa57f557a0 2024-06-02T23:24:44.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2decff9700 1 -- 172.21.15.52:0/2276560016 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df809c770 con 0x55aa57f557a0 2024-06-02T23:24:44.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.658+0000 7f2decff9700 1 -- 172.21.15.52:0/2276560016 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df80074e0 con 0x55aa57f557a0 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2276560016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 msgr2=0x55aa57f55bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55bc0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2df809c470 tx=0x7f2df8009040 comp rx=0 tx=0).stop 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2276560016 shutdown_connections 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55bc0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2276560016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f55260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2276560016 >> 172.21.15.52:0/2276560016 conn(0x55aa57e82080 msgr2=0x55aa57f50220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2276560016 shutdown_connections 2024-06-02T23:24:44.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.659+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2276560016 wait complete. 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 -- start start 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 Processor -- start 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f61590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55aa57e46fc0 con 0x55aa57f5b880 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55aa57ea2810 con 0x55aa57f557a0 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa57f66420 con 0x55aa57f5af60 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2dfe59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f61590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2dfe59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f61590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38512/0 (socket says 172.21.15.52:38512) 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2dfe59c700 1 -- 172.21.15.52:0/159668535 learned_addr learned my addr 172.21.15.52:0/159668535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2e0f25fb80 10 client.0 _ll_register_callbacks cb 0x55aa57e3c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.660+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfe59c700 1 -- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 msgr2=0x55aa57f5e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfe59c700 1 -- 172.21.15.52:0/159668535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 msgr2=0x55aa57f5ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfe59c700 1 -- 172.21.15.52:0/159668535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa57e6c750 con 0x55aa57f557a0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f61590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2df00045b0 tx=0x7f2df00047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a63732304ab946e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2de5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2de5ffb700 1 -- 172.21.15.52:0/159668535 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0004830 con 0x55aa57f557a0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2de5ffb700 1 -- 172.21.15.52:0/159668535 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2df00049d0 con 0x55aa57f557a0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.661+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df809c770 con 0x55aa57f557a0 2024-06-02T23:24:44.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2de5ffb700 1 -- 172.21.15.52:0/159668535 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2df002e070 con 0x55aa57f557a0 2024-06-02T23:24:44.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 10 client.28481 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:44.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa57f70ae0 con 0x55aa57f557a0 2024-06-02T23:24:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2de5ffb700 1 -- 172.21.15.52:0/159668535 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df809c770 con 0x55aa57f557a0 2024-06-02T23:24:44.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2de5ffb700 1 -- 172.21.15.52:0/159668535 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2df0027470 con 0x55aa57f557a0 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 10 client.28481 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 10 client.28481 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 20 client.28481 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57f755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 4 client.28481 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.662+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa57eb3d50 con 0x55aa57f73170 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.663+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57f755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.663+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57f755b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bdd2258b67f80cb9 server_cookie=35e12559c9329495 in_seq=0 out_seq=0 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.663+0000 7f2de5ffb700 10 client.28481 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2de5ffb700 1 -- 172.21.15.52:0/159668535 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55aa57eb3d50 con 0x55aa57f73170 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2de5ffb700 10 client.28481 handle_command_reply: tid=0 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 client.28481 shutdown 2024-06-02T23:24:44.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 msgr2=0x55aa57f61590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f61590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2df00045b0 tx=0x7f2df00047c0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 shutdown_connections 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57f755b0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2de5ffb700 0 client.28481 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f61590 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/159668535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 >> 172.21.15.52:0/159668535 conn(0x55aa57e82080 msgr2=0x55aa57f4f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 shutdown_connections 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/159668535 wait complete. 2024-06-02T23:24:44.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.664+0000 7f2e0f25fb80 20 client.28481 trim_cache size 0 max 16384 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 Processor -- start 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 -- start start 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aa710 con 0x55aa57f5b880 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ac250 con 0x55aa57f557a0 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001aade0 con 0x55aa57f5af60 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2dfe59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2dfed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2dfe59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38516/0 (socket says 172.21.15.52:38516) 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2dfed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56862/0 (socket says 172.21.15.52:56862) 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.682+0000 7f2dfe59c700 1 -- 172.21.15.52:0/2838701299 learned_addr learned my addr 172.21.15.52:0/2838701299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfe59c700 1 -- 172.21.15.52:0/2838701299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 msgr2=0x55aa57f5ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfe59c700 1 -- 172.21.15.52:0/2838701299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 msgr2=0x55aa57f5c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfe59c700 1 -- 172.21.15.52:0/2838701299 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa57f519c0 con 0x55aa57f557a0 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004580 tx=0x7f2df0003c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=592e1a0adea88a41 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2de7fff700 1 -- 172.21.15.52:0/2838701299 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0037070 con 0x55aa57f557a0 2024-06-02T23:24:44.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2de7fff700 1 -- 172.21.15.52:0/2838701299 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df00045e0 con 0x55aa57f557a0 2024-06-02T23:24:44.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.683+0000 7f2de7fff700 1 -- 172.21.15.52:0/2838701299 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0033b90 con 0x55aa57f557a0 2024-06-02T23:24:44.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2838701299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 msgr2=0x55aa57f55260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55260 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2df0004580 tx=0x7f2df0003c40 comp rx=0 tx=0).stop 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2838701299 shutdown_connections 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5b880 0x55aa57f5c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f557a0 0x55aa57f55260 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2838701299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f5af60 0x55aa57f5ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2838701299 >> 172.21.15.52:0/2838701299 conn(0x55aa57e82080 msgr2=0x55aa57f600c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2838701299 shutdown_connections 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.684+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2838701299 wait complete. 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 -- start start 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 Processor -- start 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f557a0 0x55aa57f6bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 0x55aa57f7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 0x55aa57f7aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001abb80 con 0x55aa57f5af60 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001ab4b0 con 0x55aa57f5b880 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e001a86b0 con 0x55aa57f557a0 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2e0f25fb80 10 client.0 _ll_register_callbacks cb 0x55aa57e3c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2dfdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 0x55aa57f7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2dfed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 0x55aa57f7aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2dfdd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 0x55aa57f7a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56872/0 (socket says 172.21.15.52:56872) 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2dfdd9b700 1 -- 172.21.15.52:0/2293254509 learned_addr learned my addr 172.21.15.52:0/2293254509 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.685+0000 7f2dfed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 0x55aa57f7aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38528/0 (socket says 172.21.15.52:38528) 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f557a0 0x55aa57f6bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f557a0 msgr2=0x55aa57f6bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f557a0 0x55aa57f6bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2293254509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 msgr2=0x55aa57f7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 0x55aa57f7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfed9d700 1 -- 172.21.15.52:0/2293254509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa57f7b4b0 con 0x55aa57f5b880 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f557a0 0x55aa57f6bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfdd9b700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 0x55aa57f7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2dfed9d700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 0x55aa57f7aca0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2df8098bd0 tx=0x7f2df8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f2670b6eccc0491 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2de5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2de5ffb700 1 -- 172.21.15.52:0/2293254509 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df802a040 con 0x55aa57f5b880 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2de5ffb700 1 -- 172.21.15.52:0/2293254509 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2df80074e0 con 0x55aa57f5b880 2024-06-02T23:24:44.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2df00045e0 con 0x55aa57f5b880 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.686+0000 7f2de5ffb700 1 -- 172.21.15.52:0/2293254509 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f2df809c770 con 0x55aa57f5b880 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 10 client.28487 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x55aa57fddda0 con 0x55aa57f5b880 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2de5ffb700 1 -- 172.21.15.52:0/2293254509 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2df00045e0 con 0x55aa57f5b880 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2de5ffb700 1 -- 172.21.15.52:0/2293254509 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f2df8011070 con 0x55aa57f5b880 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 10 client.28487 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 10 client.28487 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 20 client.28487 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57fe02b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 4 client.28487 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55aa57eae2d0 con 0x55aa57f73170 2024-06-02T23:24:44.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.687+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57fe02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2de5ffb700 10 client.28487 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2dfe59c700 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57fe02b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a08486da3775148 server_cookie=edb4fac552a7538e in_seq=0 out_seq=0 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2de5ffb700 1 -- 172.21.15.52:0/2293254509 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55aa57eae2d0 con 0x55aa57f73170 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2de5ffb700 10 client.28487 handle_command_reply: tid=0 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 client.28487 shutdown 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 msgr2=0x55aa57f7aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 0x55aa57f7aca0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2df8098bd0 tx=0x7f2df8009780 comp rx=0 tx=0).stop 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 shutdown_connections 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55aa57f73170 0x55aa57fe02b0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55aa57f5b880 0x55aa57f7aca0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa57f557a0 0x55aa57f6bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2de5ffb700 0 client.28487 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:24:44.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 --2- 172.21.15.52:0/2293254509 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55aa57f5af60 0x55aa57f7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 >> 172.21.15.52:0/2293254509 conn(0x55aa57e82080 msgr2=0x55aa57f5a280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:44.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 shutdown_connections 2024-06-02T23:24:44.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 1 -- 172.21.15.52:0/2293254509 wait complete. 2024-06-02T23:24:44.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.688+0000 7f2e0f25fb80 20 client.28487 trim_cache size 0 max 16384 2024-06-02T23:24:44.696 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2de80a2c20 msgr2=0x7f2de80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2de80a2c20 0x7f2de80a50e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f2e001999d0 tx=0x7f2df0009260 comp rx=0 tx=0).stop 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 msgr2=0x7f2e00198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 0x7f2e00198890 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2df800d010 tx=0x7f2df8004d00 comp rx=0 tx=0).stop 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 shutdown_connections 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2de80a2c20 0x7f2de80a50e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e00108520 0x7f2e00198890 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e00102510 0x7f2e0019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 --2- 172.21.15.52:0/661208586 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e00102ed0 0x7f2e0019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 >> 172.21.15.52:0/661208586 conn(0x7f2e00079560 msgr2=0x7f2e001010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 shutdown_connections 2024-06-02T23:24:44.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:44.689+0000 7f2e049dc700 1 -- 172.21.15.52:0/661208586 wait complete. 2024-06-02T23:24:45.502 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:24:45.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:45 smithi052 ceph-mon[32340]: pgmap v817: 97 pgs: 97 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:24:45.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 254798 1.88 MB/sec execute 594 sec latency 15.144 ms 2024-06-02T23:24:45.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:45 smithi174 ceph-mon[35994]: pgmap v817: 97 pgs: 97 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:24:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:45 smithi123 ceph-mon[35977]: pgmap v817: 97 pgs: 97 active+clean; 107 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 1.3 MiB/s wr, 138 op/s 2024-06-02T23:24:45.816 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:46.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 255265 1.88 MB/sec execute 595 sec latency 12.258 ms 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 -- 172.21.15.52:0/228395231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 msgr2=0x7f301c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 --2- 172.21.15.52:0/228395231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 0x7f301c1088e0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f300c009990 tx=0x7f300c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 -- 172.21.15.52:0/228395231 shutdown_connections 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 --2- 172.21.15.52:0/228395231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f301c06ab30 0x7f301c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 --2- 172.21.15.52:0/228395231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 0x7f301c1088e0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 --2- 172.21.15.52:0/228395231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 -- 172.21.15.52:0/228395231 >> 172.21.15.52:0/228395231 conn(0x7f301c078340 msgr2=0x7f301c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 -- 172.21.15.52:0/228395231 shutdown_connections 2024-06-02T23:24:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.619+0000 7f30218c4700 1 -- 172.21.15.52:0/228395231 wait complete. 2024-06-02T23:24:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 Processor -- start 2024-06-02T23:24:47.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 -- start start 2024-06-02T23:24:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f301c06ab30 0x7f301c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 0x7f301c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f301c112630 con 0x7f301c108500 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1124b0 con 0x7f301c06a170 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f30218c4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f301c1127b0 con 0x7f301c06ab30 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f301affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.620+0000 7f301affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38556/0 (socket says 172.21.15.52:38556) 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 -- 172.21.15.52:0/1184360479 learned_addr learned my addr 172.21.15.52:0/1184360479 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f30127fc700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f301c06ab30 0x7f301c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301b7fe700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 0x7f301c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 -- 172.21.15.52:0/1184360479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f301c06ab30 msgr2=0x7f301c19ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f301c06ab30 0x7f301c19ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 -- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 msgr2=0x7f301c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 0x7f301c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 -- 172.21.15.52:0/1184360479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f301c199110 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f301affd700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c19e7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f300c00b2e0 tx=0x7f300c0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f30208c2700 1 -- 172.21.15.52:0/1184360479 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c026070 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f30208c2700 1 -- 172.21.15.52:0/1184360479 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f300c021410 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f30208c2700 1 -- 172.21.15.52:0/1184360479 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c017650 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.621+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f301c1993a0 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.622+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f301c101390 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.624+0000 7f30208c2700 1 -- 172.21.15.52:0/1184360479 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f300c029070 con 0x7f301c06a170 2024-06-02T23:24:47.624 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:47 smithi052 ceph-mon[32340]: pgmap v818: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-02T23:24:47.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.625+0000 7f30208c2700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30080a2c20 0x7f30080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.625+0000 7f30127fc700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30080a2c20 0x7f30080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.625+0000 7f30208c2700 1 -- 172.21.15.52:0/1184360479 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f300c0cfc90 con 0x7f301c06a170 2024-06-02T23:24:47.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f30127fc700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30080a2c20 0x7f30080a50e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f301c0789b0 tx=0x7f3004009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 Processor -- start 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 -- start start 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8a1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a1880 0x56212b89b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56212b78df60 con 0x56212b89b7a0 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56212b7ec850 con 0x56212b8a0f60 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.626+0000 7f302c147b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56212b754460 con 0x56212b8a1880 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8a1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38572/0 (socket says 172.21.15.52:38572) 2024-06-02T23:24:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301affd700 1 -- 172.21.15.52:0/1058277955 learned_addr learned my addr 172.21.15.52:0/1058277955 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301b7fe700 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f30127fc700 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a1880 0x56212b89b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301b7fe700 1 -- 172.21.15.52:0/1058277955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a1880 msgr2=0x56212b89b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301b7fe700 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a1880 0x56212b89b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301b7fe700 1 -- 172.21.15.52:0/1058277955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 msgr2=0x56212b8a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301b7fe700 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301b7fe700 1 -- 172.21.15.52:0/1058277955 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212b73b8d0 con 0x56212b89b7a0 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.627+0000 7f301affd700 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.628+0000 7f301b7fe700 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89bbc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3014009310 tx=0x7f3014009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa44b4718067d00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.628+0000 7f3011ffb700 1 -- 172.21.15.52:0/1058277955 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3014019020 con 0x56212b89b7a0 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.628+0000 7f3011ffb700 1 -- 172.21.15.52:0/1058277955 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3014015af0 con 0x56212b89b7a0 2024-06-02T23:24:47.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.628+0000 7f3011ffb700 1 -- 172.21.15.52:0/1058277955 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30140149e0 con 0x56212b89b7a0 2024-06-02T23:24:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 -- 172.21.15.52:0/1058277955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 msgr2=0x56212b89bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89bbc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f3014009310 tx=0x7f3014009920 comp rx=0 tx=0).stop 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 -- 172.21.15.52:0/1058277955 shutdown_connections 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89bbc0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 --2- 172.21.15.52:0/1058277955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a1880 0x56212b89b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 -- 172.21.15.52:0/1058277955 >> 172.21.15.52:0/1058277955 conn(0x56212b7c8080 msgr2=0x56212b896220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 -- 172.21.15.52:0/1058277955 shutdown_connections 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 -- 172.21.15.52:0/1058277955 wait complete. 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.629+0000 7f302c147b80 1 -- start start 2024-06-02T23:24:47.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 Processor -- start 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b89b7a0 0x56212b8a7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56212b78cfc0 con 0x56212b8a1880 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56212b7e8810 con 0x56212b89b7a0 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56212b8a8260 con 0x56212b8a0f60 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f302c147b80 10 client.0 _ll_register_callbacks cb 0x56212b7825b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f301b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f301b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8a4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56900/0 (socket says 172.21.15.52:56900) 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f301b7fe700 1 -- 172.21.15.52:0/1917402066 learned_addr learned my addr 172.21.15.52:0/1917402066 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.630+0000 7f301affd700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b89b7a0 0x56212b8a7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301affd700 1 -- 172.21.15.52:0/1917402066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 msgr2=0x56212b8a4830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:24:47.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301affd700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301affd700 1 -- 172.21.15.52:0/1917402066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 msgr2=0x56212b8a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301affd700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301affd700 1 -- 172.21.15.52:0/1917402066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56212b7b2750 con 0x56212b89b7a0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301b7fe700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f301affd700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b89b7a0 0x56212b8a7530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f300c09c4e0 tx=0x7f300c09ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c7ad076146ad04 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f2ffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f2ffaffd700 1 -- 172.21.15.52:0/1917402066 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c09cc50 con 0x56212b89b7a0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f2ffaffd700 1 -- 172.21.15.52:0/1917402066 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f300c0b5260 con 0x56212b89b7a0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3014015af0 con 0x56212b89b7a0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f2ffaffd700 1 -- 172.21.15.52:0/1917402066 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f300c09c550 con 0x56212b89b7a0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f302c147b80 10 client.28499 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56212b8b29b0 con 0x56212b89b7a0 2024-06-02T23:24:47.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.631+0000 7f2ffaffd700 1 -- 172.21.15.52:0/1917402066 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3014015af0 con 0x56212b89b7a0 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f2ffaffd700 1 -- 172.21.15.52:0/1917402066 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f300c09ba30 con 0x56212b89b7a0 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f302c147b80 10 client.28499 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f302c147b80 10 client.28499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f302c147b80 20 client.28499 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f302c147b80 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b8b4fc0 0x56212b8b7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f302c147b80 4 client.28499 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56212b7f9d50 con 0x56212b8b4fc0 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f30127fc700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b8b4fc0 0x56212b8b7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f2ffaffd700 10 client.28499 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:47.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.632+0000 7f30127fc700 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b8b4fc0 0x56212b8b7420 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68c9c33309373979 server_cookie=f7fe24963c336785 in_seq=0 out_seq=0 2024-06-02T23:24:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.633+0000 7f2ffaffd700 1 -- 172.21.15.52:0/1917402066 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56212b7f9d50 con 0x56212b8b4fc0 2024-06-02T23:24:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.633+0000 7f2ffaffd700 10 client.28499 handle_command_reply: tid=0 2024-06-02T23:24:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.633+0000 7f302c147b80 1 client.28499 shutdown 2024-06-02T23:24:47.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.633+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b89b7a0 msgr2=0x56212b8a7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.633+0000 7f302c147b80 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b89b7a0 0x56212b8a7530 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f300c09c4e0 tx=0x7f300c09ca90 comp rx=0 tx=0).stop 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 shutdown_connections 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b8b4fc0 0x56212b8b7420 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b89b7a0 0x56212b8a7530 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 --2- 172.21.15.52:0/1917402066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 >> 172.21.15.52:0/1917402066 conn(0x56212b7c8080 msgr2=0x56212b895700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f2ffaffd700 0 client.28499 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 shutdown_connections 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 1 -- 172.21.15.52:0/1917402066 wait complete. 2024-06-02T23:24:47.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.634+0000 7f302c147b80 20 client.28499 trim_cache size 0 max 16384 2024-06-02T23:24:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 Processor -- start 2024-06-02T23:24:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 -- start start 2024-06-02T23:24:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a1880 0x56212b895e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00d2a0 con 0x56212b89b7a0 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00ede0 con 0x56212b8a1880 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f302c147b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00d970 con 0x56212b8a0f60 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f301affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f301affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56912/0 (socket says 172.21.15.52:56912) 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f301affd700 1 -- 172.21.15.52:0/598885645 learned_addr learned my addr 172.21.15.52:0/598885645 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f301b7fe700 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a1880 0x56212b895e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.651+0000 7f30127fc700 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f301affd700 1 -- 172.21.15.52:0/598885645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 msgr2=0x56212b8a2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f301affd700 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f301affd700 1 -- 172.21.15.52:0/598885645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a1880 msgr2=0x56212b895e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f301affd700 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a1880 0x56212b895e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f301affd700 1 -- 172.21.15.52:0/598885645 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56212b8a2da0 con 0x56212b89b7a0 2024-06-02T23:24:47.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f301affd700 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89b260 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f300c006010 tx=0x7f300c09aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508ef40470323008 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f30117fa700 1 -- 172.21.15.52:0/598885645 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c0a5130 con 0x56212b89b7a0 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f30117fa700 1 -- 172.21.15.52:0/598885645 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f300c0a0900 con 0x56212b89b7a0 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.652+0000 7f30117fa700 1 -- 172.21.15.52:0/598885645 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300c09b410 con 0x56212b89b7a0 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 -- 172.21.15.52:0/598885645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 msgr2=0x56212b89b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89b260 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f300c006010 tx=0x7f300c09aa60 comp rx=0 tx=0).stop 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 -- 172.21.15.52:0/598885645 shutdown_connections 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a1880 0x56212b895e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b89b7a0 0x56212b89b260 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 --2- 172.21.15.52:0/598885645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b8a0f60 0x56212b8a2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 -- 172.21.15.52:0/598885645 >> 172.21.15.52:0/598885645 conn(0x56212b7c8080 msgr2=0x56212b8a1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 -- 172.21.15.52:0/598885645 shutdown_connections 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 -- 172.21.15.52:0/598885645 wait complete. 2024-06-02T23:24:47.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.653+0000 7f302c147b80 1 -- start start 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 Processor -- start 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b89b7a0 0x56212b8ad580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8adac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00e710 con 0x56212b8a1880 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00e040 con 0x56212b8a0f60 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ffc00b240 con 0x56212b89b7a0 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f302c147b80 10 client.0 _ll_register_callbacks cb 0x56212b7825b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f301b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f301affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b89b7a0 0x56212b8ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f30127fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f301affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b89b7a0 0x56212b8ad580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34950/0 (socket says 172.21.15.52:34950) 2024-06-02T23:24:47.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f301affd700 1 -- 172.21.15.52:0/364494079 learned_addr learned my addr 172.21.15.52:0/364494079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f30127fc700 1 -- 172.21.15.52:0/364494079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b89b7a0 msgr2=0x56212b8ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f30127fc700 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b89b7a0 0x56212b8ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f30127fc700 1 -- 172.21.15.52:0/364494079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 msgr2=0x56212b8bbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f30127fc700 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8bbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.654+0000 7f30127fc700 1 -- 172.21.15.52:0/364494079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56212b8bc260 con 0x56212b8a0f60 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f30127fc700 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8adac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56212b8b7ca0 tx=0x7f30040047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6934bac864fdd6b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:47.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f2ffaffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:47.660 INFO:tasks.workunit.client.0.smithi052.stdout: 1 255728 1.88 MB/sec execute 596 sec latency 12.563 ms 2024-06-02T23:24:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f2ffaffd700 1 -- 172.21.15.52:0/364494079 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f300401dce0 con 0x56212b8a0f60 2024-06-02T23:24:47.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f2ffaffd700 1 -- 172.21.15.52:0/364494079 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3004009040 con 0x56212b8a0f60 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f300c0a0900 con 0x56212b8a0f60 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f2ffaffd700 1 -- 172.21.15.52:0/364494079 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f3004004030 con 0x56212b8a0f60 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f302c147b80 10 client.28505 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x56212b91f440 con 0x56212b8a0f60 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f2ffaffd700 1 -- 172.21.15.52:0/364494079 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f300c0a0900 con 0x56212b8a0f60 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.655+0000 7f2ffaffd700 1 -- 172.21.15.52:0/364494079 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7f30040273a0 con 0x56212b8a0f60 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 10 client.28505 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 10 client.28505 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 20 client.28505 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b91fb00 0x56212b921f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 4 client.28505 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56212b7f42d0 con 0x56212b91fb00 2024-06-02T23:24:47.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f301affd700 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b91fb00 0x56212b921f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f2ffaffd700 10 client.28505 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f301affd700 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b91fb00 0x56212b921f20 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb46c3d338ad5776 server_cookie=1dec4feb93353e91 in_seq=0 out_seq=0 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f2ffaffd700 1 -- 172.21.15.52:0/364494079 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56212b7f42d0 con 0x56212b91fb00 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f2ffaffd700 10 client.28505 handle_command_reply: tid=0 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.656+0000 7f302c147b80 1 client.28505 shutdown 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 msgr2=0x56212b8adac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8adac0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x56212b8b7ca0 tx=0x7f30040047c0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 shutdown_connections 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56212b91fb00 0x56212b921f20 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f2ffaffd700 0 client.28505 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56212b8a1880 0x56212b8bbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56212b89b7a0 0x56212b8ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 --2- 172.21.15.52:0/364494079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56212b8a0f60 0x56212b8adac0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 >> 172.21.15.52:0/364494079 conn(0x56212b7c8080 msgr2=0x56212b8a6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 shutdown_connections 2024-06-02T23:24:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 1 -- 172.21.15.52:0/364494079 wait complete. 2024-06-02T23:24:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f302c147b80 20 client.28505 trim_cache size 0 max 16384 2024-06-02T23:24:47.665 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:24:47.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30080a2c20 msgr2=0x7f30080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f30218c4700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30080a2c20 0x7f30080a50e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f301c0789b0 tx=0x7f3004009260 comp rx=0 tx=0).stop 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 msgr2=0x7f301c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.657+0000 7f30218c4700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c19e7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f300c00b2e0 tx=0x7f300c0047b0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 shutdown_connections 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f30080a2c20 0x7f30080a50e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f301c108500 0x7f301c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f301c06a170 0x7f301c19e7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 --2- 172.21.15.52:0/1184360479 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f301c06ab30 0x7f301c19ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 >> 172.21.15.52:0/1184360479 conn(0x7f301c078340 msgr2=0x7f301c1000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 shutdown_connections 2024-06-02T23:24:47.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:47.658+0000 7f30218c4700 1 -- 172.21.15.52:0/1184360479 wait complete. 2024-06-02T23:24:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:47 smithi123 ceph-mon[35977]: pgmap v818: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-02T23:24:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:47 smithi174 ceph-mon[35994]: pgmap v818: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 1.4 MiB/s wr, 182 op/s 2024-06-02T23:24:48.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 255999 1.88 MB/sec execute 597 sec latency 15.420 ms 2024-06-02T23:24:49.397 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:24:49.398 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:24:49.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 256503 1.88 MB/sec execute 598 sec latency 15.073 ms 2024-06-02T23:24:49.686 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:49 smithi123 ceph-mon[35977]: pgmap v819: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-06-02T23:24:49.810 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:49 smithi052 ceph-mon[32340]: pgmap v819: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-06-02T23:24:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:49 smithi174 ceph-mon[35994]: pgmap v819: 97 pgs: 97 active+clean; 106 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 204 KiB/s rd, 1.1 MiB/s wr, 136 op/s 2024-06-02T23:24:50.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 257055 1.88 MB/sec execute 599 sec latency 16.257 ms 2024-06-02T23:24:51.655 INFO:tasks.workunit.client.0.smithi052.stdout: 1 cleanup 600 sec 2024-06-02T23:24:51.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:51 smithi052 ceph-mon[32340]: pgmap v820: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-06-02T23:24:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:51 smithi123 ceph-mon[35977]: pgmap v820: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-06-02T23:24:51.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/3606481025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4101390 msgr2=0x7fc6f41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/3606481025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4101390 0x7fc6f41017f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8009990 tx=0x7fc6e8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/3606481025 shutdown_connections 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/3606481025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f4101d30 0x7fc6f410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/3606481025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f41073d0 0x7fc6f41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/3606481025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f4101390 0x7fc6f41017f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/3606481025 >> 172.21.15.52:0/3606481025 conn(0x7fc6f40782c0 msgr2=0x7fc6f40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/3606481025 shutdown_connections 2024-06-02T23:24:51.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.826+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/3606481025 wait complete. 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 Processor -- start 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 -- start start 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f4101390 0x7fc6f419d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 0x7fc6f419dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f41073d0 0x7fc6f41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f4111530 con 0x7fc6f41073d0 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41113b0 con 0x7fc6f4101390 2024-06-02T23:24:51.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f8ed5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41116b0 con 0x7fc6f4101d30 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 0x7fc6f419dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f4101390 0x7fc6f419d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f41073d0 0x7fc6f41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f1d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 0x7fc6f419dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34970/0 (socket says 172.21.15.52:34970) 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/1805080505 learned_addr learned my addr 172.21.15.52:0/1805080505 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/1805080505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f4101390 msgr2=0x7fc6f419d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.827+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f4101390 0x7fc6f419d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f41073d0 msgr2=0x7fc6f41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f41073d0 0x7fc6f41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/1805080505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6f4198050 con 0x7fc6f4101d30 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f41073d0 0x7fc6f41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 0x7fc6f419dc70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005910 tx=0x7fc6e80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6e37fe700 1 -- 172.21.15.52:0/1805080505 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8027070 con 0x7fc6f4101d30 2024-06-02T23:24:51.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6e37fe700 1 -- 172.21.15.52:0/1805080505 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6e8022470 con 0x7fc6f4101d30 2024-06-02T23:24:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6f41982e0 con 0x7fc6f4101d30 2024-06-02T23:24:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6e37fe700 1 -- 172.21.15.52:0/1805080505 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e800b610 con 0x7fc6f4101d30 2024-06-02T23:24:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.828+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6f41986d0 con 0x7fc6f4101d30 2024-06-02T23:24:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.830+0000 7fc6e37fe700 1 -- 172.21.15.52:0/1805080505 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc6e802a070 con 0x7fc6f4101d30 2024-06-02T23:24:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.831+0000 7fc6e37fe700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc6dc0a2c20 0x7fc6dc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.831+0000 7fc6f259c700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc6dc0a2c20 0x7fc6dc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.831+0000 7fc6e37fe700 1 -- 172.21.15.52:0/1805080505 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc6e80cfd60 con 0x7fc6f4101d30 2024-06-02T23:24:51.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc6f259c700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc6dc0a2c20 0x7fc6dc0a50e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4198a30 tx=0x7fc6e4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:51.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 Processor -- start 2024-06-02T23:24:51.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 -- start start 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec5a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec54b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec61290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558dceb42fe0 con 0x558dcec546f0 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558dceba18d0 con 0x558dcec5a4f0 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.832+0000 7fc703758b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558dceb094e0 con 0x558dcec550b0 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec54b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec54b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56932/0 (socket says 172.21.15.52:56932) 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/2797355407 learned_addr learned my addr 172.21.15.52:0/2797355407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f259c700 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec5a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec61290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/2797355407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 msgr2=0x558dcec61290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec61290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/2797355407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 msgr2=0x558dcec5a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec5a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 -- 172.21.15.52:0/2797355407 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dceaf0950 con 0x558dcec546f0 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f259c700 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec54b70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80a4d30 tx=0x7fc6e8009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39b50de4573ff3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.833+0000 7fc6e0ff9700 1 -- 172.21.15.52:0/2797355407 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e809c680 con 0x558dcec546f0 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc6e0ff9700 1 -- 172.21.15.52:0/2797355407 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6e80a0a30 con 0x558dcec546f0 2024-06-02T23:24:51.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc6e0ff9700 1 -- 172.21.15.52:0/2797355407 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e809cb40 con 0x558dcec546f0 2024-06-02T23:24:51.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 -- 172.21.15.52:0/2797355407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 msgr2=0x558dcec54b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec54b70 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fc6e80a4d30 tx=0x7fc6e8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 -- 172.21.15.52:0/2797355407 shutdown_connections 2024-06-02T23:24:51.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec61290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec5a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 --2- 172.21.15.52:0/2797355407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec54b70 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 -- 172.21.15.52:0/2797355407 >> 172.21.15.52:0/2797355407 conn(0x558dceb7d100 msgr2=0x558dcec4b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.834+0000 7fc703758b80 1 -- 172.21.15.52:0/2797355407 shutdown_connections 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.835+0000 7fc703758b80 1 -- 172.21.15.52:0/2797355407 wait complete. 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.835+0000 7fc703758b80 1 -- start start 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.835+0000 7fc703758b80 1 Processor -- start 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.835+0000 7fc703758b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec546f0 0x558dcec63e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec550b0 0x558dcec643d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 0x558dcec5e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558dceb42040 con 0x558dcec5a4f0 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558dceb9d890 con 0x558dcec550b0 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558dcec5aed0 con 0x558dcec546f0 2024-06-02T23:24:51.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 10 client.0 _ll_register_callbacks cb 0x558dceb37630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 0x558dcec5e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 0x558dcec5e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56946/0 (socket says 172.21.15.52:56946) 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1240884819 learned_addr learned my addr 172.21.15.52:0/1240884819 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f259c700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec546f0 0x558dcec63e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec550b0 0x558dcec643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1240884819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec546f0 msgr2=0x558dcec63e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec546f0 0x558dcec63e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec550b0 msgr2=0x558dcec643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec550b0 0x558dcec643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1240884819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dceb677d0 con 0x558dcec5a4f0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 0x558dcec5e210 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec004660 tx=0x7fc6ec004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f13dceca6036c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:51.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/1240884819 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec01a070 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/1240884819 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc6ec0079d0 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.836+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6e80a0a30 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/1240884819 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc6ec007c10 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 10 client.19107 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558dcec6bca0 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/1240884819 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6e80a0a30 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/1240884819 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc6ec016b00 con 0x558dcec5a4f0 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 10 client.19107 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 10 client.19107 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 20 client.19107 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcec6e230 0x558dcec70620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 4 client.19107 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558dcebaedd0 con 0x558dcec6e230 2024-06-02T23:24:51.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.837+0000 7fc6f259c700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcec6e230 0x558dcec70620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc6f259c700 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcec6e230 0x558dcec70620 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd811927176e25d server_cookie=4bae51aa142eed5e in_seq=0 out_seq=0 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc6d9ffb700 10 client.19107 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/1240884819 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558dcebaedd0 con 0x558dcec6e230 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc6d9ffb700 10 client.19107 handle_command_reply: tid=0 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc703758b80 1 client.19107 shutdown 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 msgr2=0x558dcec5e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.838+0000 7fc703758b80 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 0x558dcec5e210 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec004660 tx=0x7fc6ec004690 comp rx=0 tx=0).stop 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 shutdown_connections 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcec6e230 0x558dcec70620 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec5a4f0 0x558dcec5e210 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec546f0 0x558dcec63e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 --2- 172.21.15.52:0/1240884819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec550b0 0x558dcec643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 >> 172.21.15.52:0/1240884819 conn(0x558dceb7d100 msgr2=0x558dcec4a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:51.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc6d9ffb700 0 client.19107 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:51.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 shutdown_connections 2024-06-02T23:24:51.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 1 -- 172.21.15.52:0/1240884819 wait complete. 2024-06-02T23:24:51.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.839+0000 7fc703758b80 20 client.19107 trim_cache size 0 max 16384 2024-06-02T23:24:51.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.861+0000 7fc703758b80 1 Processor -- start 2024-06-02T23:24:51.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.861+0000 7fc703758b80 1 -- start start 2024-06-02T23:24:51.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc703758b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec4af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc703758b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec4b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc703758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc703758b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41aa6f0 con 0x558dcec546f0 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc703758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ac230 con 0x558dcec5a4f0 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc703758b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41aadc0 con 0x558dcec550b0 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec75ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38648/0 (socket says 172.21.15.52:38648) 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1369358543 learned_addr learned my addr 172.21.15.52:0/1369358543 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f259c700 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec4af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec4b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1369358543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 msgr2=0x558dcec4b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec4b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1369358543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 msgr2=0x558dcec4af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec4af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.862+0000 7fc6f2d9d700 1 -- 172.21.15.52:0/1369358543 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558dcec4bec0 con 0x558dcec5a4f0 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.863+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec75ef0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec016ef0 tx=0x7fc6ec009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4ef101b825c776a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.863+0000 7fc6dbfff700 1 -- 172.21.15.52:0/1369358543 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec02f070 con 0x558dcec5a4f0 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.863+0000 7fc6dbfff700 1 -- 172.21.15.52:0/1369358543 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6ec0077d0 con 0x558dcec5a4f0 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.863+0000 7fc6dbfff700 1 -- 172.21.15.52:0/1369358543 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6ec009e60 con 0x558dcec5a4f0 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 -- 172.21.15.52:0/1369358543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 msgr2=0x558dcec75ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec75ef0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec016ef0 tx=0x7fc6ec009280 comp rx=0 tx=0).stop 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 -- 172.21.15.52:0/1369358543 shutdown_connections 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec75ef0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec4af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 --2- 172.21.15.52:0/1369358543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec4b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 -- 172.21.15.52:0/1369358543 >> 172.21.15.52:0/1369358543 conn(0x558dceb7d100 msgr2=0x558dcec5b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:51.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 -- 172.21.15.52:0/1369358543 shutdown_connections 2024-06-02T23:24:51.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 -- 172.21.15.52:0/1369358543 wait complete. 2024-06-02T23:24:51.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 -- start start 2024-06-02T23:24:51.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.864+0000 7fc703758b80 1 Processor -- start 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec6a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec64c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec65180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41abb60 con 0x558dcec546f0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41ab490 con 0x558dcec5a4f0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6f41a8690 con 0x558dcec550b0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc703758b80 10 client.0 _ll_register_callbacks cb 0x558dceb37630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec6a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec6a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56958/0 (socket says 172.21.15.52:56958) 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 -- 172.21.15.52:0/3281318597 learned_addr learned my addr 172.21.15.52:0/3281318597 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f2d9d700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec65180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec64c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 -- 172.21.15.52:0/3281318597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 msgr2=0x558dcec64c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec64c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 -- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 msgr2=0x558dcec65180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec65180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 -- 172.21.15.52:0/3281318597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558dcec65880 con 0x558dcec546f0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6f259c700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec6a850 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009aa0 tx=0x7fc6e40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca7ef4807fd3ced8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.865+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/3281318597 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4003b60 con 0x558dcec546f0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/3281318597 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc6e4003d00 con 0x558dcec546f0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6ec0077d0 con 0x558dcec546f0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/3281318597 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7fc6e403a050 con 0x558dcec546f0 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 10 client.19113 fetch_fsmap learned FSMap version 32 2024-06-02T23:24:51.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x558dcecde680 con 0x558dcec546f0 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/3281318597 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc6ec0077d0 con 0x558dcec546f0 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/3281318597 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 32) ==== 8354+0+0 (secure 0 0 0) 0x7fc6e401e480 con 0x558dcec546f0 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 10 client.19113 fetch_fsmap finished waiting for FSMap version 32 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 10 client.19113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 20 client.19113 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcecded70 0x558dcece1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 4 client.19113 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558dceba9350 con 0x558dcecded70 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.866+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcecded70 0x558dcece1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.867+0000 7fc6d9ffb700 10 client.19113 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.867+0000 7fc6f1d9b700 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcecded70 0x558dcece1190 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e763982cd9d16f4 server_cookie=60c62175dc6677f9 in_seq=0 out_seq=0 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc6d9ffb700 1 -- 172.21.15.52:0/3281318597 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558dceba9350 con 0x558dcecded70 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc6d9ffb700 10 client.19113 handle_command_reply: tid=0 2024-06-02T23:24:51.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 client.19113 shutdown 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 msgr2=0x558dcec6a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec6a850 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4009aa0 tx=0x7fc6e40047c0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 shutdown_connections 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558dcecded70 0x558dcece1190 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558dcec5a4f0 0x558dcec65180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc6d9ffb700 0 client.19113 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558dcec546f0 0x558dcec6a850 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 --2- 172.21.15.52:0/3281318597 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558dcec550b0 0x558dcec64c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 >> 172.21.15.52:0/3281318597 conn(0x558dceb7d100 msgr2=0x558dcec49ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 shutdown_connections 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 1 -- 172.21.15.52:0/3281318597 wait complete. 2024-06-02T23:24:51.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.868+0000 7fc703758b80 20 client.19113 trim_cache size 0 max 16384 2024-06-02T23:24:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:24:51.873 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:24:51.873 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "8832af8a-704f-4c82-b920-b11f3b62d958", 2024-06-02T23:24:51.873 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:24:51.873 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:24:51.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc6dc0a2c20 msgr2=0x7fc6dc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc6dc0a2c20 0x7fc6dc0a50e0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4198a30 tx=0x7fc6e4009260 comp rx=0 tx=0).stop 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 msgr2=0x7fc6f419dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 0x7fc6f419dc70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc6e8005910 tx=0x7fc6e80047b0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 shutdown_connections 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc6dc0a2c20 0x7fc6dc0a50e0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc6f41073d0 0x7fc6f41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6f4101390 0x7fc6f419d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 --2- 172.21.15.52:0/1805080505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6f4101d30 0x7fc6f419dc70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 >> 172.21.15.52:0/1805080505 conn(0x7fc6f40782c0 msgr2=0x7fc6f4100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 shutdown_connections 2024-06-02T23:24:51.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:51.869+0000 7fc6f8ed5700 1 -- 172.21.15.52:0/1805080505 wait complete. 2024-06-02T23:24:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:51 smithi174 ceph-mon[35994]: pgmap v820: 97 pgs: 97 active+clean; 108 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 247 KiB/s rd, 1.4 MiB/s wr, 150 op/s 2024-06-02T23:24:52.343 INFO:tasks.workunit.client.0.smithi052.stdout: 0 cleanup 601 sec 2024-06-02T23:24:52.343 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-02T23:24:52.343 INFO:tasks.workunit.client.0.smithi052.stdout: Operation Count AvgLat MaxLat 2024-06-02T23:24:52.343 INFO:tasks.workunit.client.0.smithi052.stdout: ---------------------------------------- 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: NTCreateX 36196 5.680 23569.973 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Close 26580 0.048 1.002 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Rename 1534 7.062 63.054 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Unlink 7316 6.487 18585.143 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Qpathinfo 32825 2.983 88.011 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Qfileinfo 5723 0.075 0.843 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Qfsinfo 6018 0.315 3.799 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Sfileinfo 2949 1.394 71.104 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Find 12681 10.335 157.109 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: WriteX 17884 5.153 31557.790 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: ReadX 56775 0.004 0.194 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: LockX 118 0.648 1.078 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: UnlockX 118 0.521 0.839 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: Flush 2537 2.504 18.076 2024-06-02T23:24:52.344 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-02T23:24:52.345 INFO:tasks.workunit.client.0.smithi052.stdout:Throughput 1.88074 MB/sec 1 clients 1 procs max_latency=31557.795 ms 2024-06-02T23:24:52.345 INFO:tasks.workunit.client.0.smithi052.stderr:+ dbench 10 2024-06-02T23:24:52.345 INFO:tasks.workunit.client.0.smithi052.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-06-02T23:24:52.345 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-02T23:24:52.345 INFO:tasks.workunit.client.0.smithi052.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-06-02T23:24:52.346 INFO:tasks.workunit.client.0.smithi052.stdout:9 of 10 processes prepared for launch 0 sec 2024-06-02T23:24:52.446 INFO:tasks.workunit.client.0.smithi052.stdout:10 of 10 processes prepared for launch 0 sec 2024-06-02T23:24:52.446 INFO:tasks.workunit.client.0.smithi052.stdout:releasing clients 2024-06-02T23:24:52.554 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.554 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.554 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.554 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:52.555 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:52 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.597 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:52.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:52 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:52.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:52 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:52.972 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:24:53.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 273 102.62 MB/sec warmup 1 sec latency 36.164 ms 2024-06-02T23:24:53.625 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.626 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.627 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: pgmap v821: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-02T23:24:53.627 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:53 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:24:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:24:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: pgmap v821: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-02T23:24:53.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:53 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:24:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: pgmap v821: 97 pgs: 97 active+clean; 109 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 262 KiB/s rd, 1.2 MiB/s wr, 157 op/s 2024-06-02T23:24:53.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:53 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:54.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 508 88.09 MB/sec warmup 2 sec latency 62.860 ms 2024-06-02T23:24:55.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 -- 172.21.15.52:0/3328824238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe28810a6d0 msgr2=0x7fe28810aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3328824238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe28810a6d0 0x7fe28810aab0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe27c009990 tx=0x7fe27c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 -- 172.21.15.52:0/3328824238 shutdown_connections 2024-06-02T23:24:55.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3328824238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe288105090 0x7fe2881115a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3328824238 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe28810a6d0 0x7fe28810aab0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3328824238 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe2881046d0 0x7fe288104b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 -- 172.21.15.52:0/3328824238 >> 172.21.15.52:0/3328824238 conn(0x7fe2881001c0 msgr2=0x7fe2881025e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 -- 172.21.15.52:0/3328824238 shutdown_connections 2024-06-02T23:24:55.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.163+0000 7fe28efd5700 1 -- 172.21.15.52:0/3328824238 wait complete. 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.164+0000 7fe28efd5700 1 Processor -- start 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.164+0000 7fe28efd5700 1 -- start start 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28efd5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 0x7fe28819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28efd5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe288105090 0x7fe28819b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28efd5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe28810a6d0 0x7fe28819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28efd5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe288114910 con 0x7fe288105090 2024-06-02T23:24:55.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28efd5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe288114790 con 0x7fe2881046d0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28efd5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe288114a90 con 0x7fe28810a6d0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 0x7fe28819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 0x7fe28819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39002/0 (socket says 172.21.15.52:39002) 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 -- 172.21.15.52:0/3329772484 learned_addr learned my addr 172.21.15.52:0/3329772484 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe287fff700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe288105090 0x7fe28819b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28d572700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe28810a6d0 0x7fe28819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 -- 172.21.15.52:0/3329772484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe28810a6d0 msgr2=0x7fe28819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe28810a6d0 0x7fe28819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 -- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe288105090 msgr2=0x7fe28819b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe288105090 0x7fe28819b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.165+0000 7fe28cd71700 1 -- 172.21.15.52:0/3329772484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28819fdb0 con 0x7fe2881046d0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.166+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 0x7fe28819ada0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe27c006010 tx=0x7fe27c0057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:55.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.166+0000 7fe285ffb700 1 -- 172.21.15.52:0/3329772484 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c027070 con 0x7fe2881046d0 2024-06-02T23:24:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.166+0000 7fe285ffb700 1 -- 172.21.15.52:0/3329772484 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe27c022470 con 0x7fe2881046d0 2024-06-02T23:24:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.166+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2881a0040 con 0x7fe2881046d0 2024-06-02T23:24:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.166+0000 7fe285ffb700 1 -- 172.21.15.52:0/3329772484 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c00b610 con 0x7fe2881046d0 2024-06-02T23:24:55.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.166+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2881a7920 con 0x7fe2881046d0 2024-06-02T23:24:55.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.169+0000 7fe285ffb700 1 -- 172.21.15.52:0/3329772484 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe27c02a070 con 0x7fe2881046d0 2024-06-02T23:24:55.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.170+0000 7fe285ffb700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe2780a2ca0 0x7fe2780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.170+0000 7fe287fff700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe2780a2ca0 0x7fe2780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.170+0000 7fe285ffb700 1 -- 172.21.15.52:0/3329772484 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe27c0cfd60 con 0x7fe2881046d0 2024-06-02T23:24:55.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 Processor -- start 2024-06-02T23:24:55.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe287fff700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe2780a2ca0 0x7fe2780a5160 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe288102840 tx=0x7fe274009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:55.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 -- start start 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620bf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620b9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c261fabf60 con 0x55c2620b97a0 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c26200a850 con 0x55c2620bf880 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.171+0000 7fe299858b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c261f72460 con 0x55c2620bef60 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe287fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620bf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe287fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39014/0 (socket says 172.21.15.52:39014) 2024-06-02T23:24:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620bf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38400/0 (socket says 172.21.15.52:38400) 2024-06-02T23:24:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe287fff700 1 -- 172.21.15.52:0/2454170240 learned_addr learned my addr 172.21.15.52:0/2454170240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28d572700 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620b9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28d572700 1 -- 172.21.15.52:0/2454170240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 msgr2=0x55c2620bf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28d572700 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620bf340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28d572700 1 -- 172.21.15.52:0/2454170240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 msgr2=0x55c2620b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28d572700 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28d572700 1 -- 172.21.15.52:0/2454170240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c261f598d0 con 0x55c2620b97a0 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe28cd71700 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.172+0000 7fe287fff700 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.173+0000 7fe28d572700 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620b9bc0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe280009f70 tx=0x7fe280009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b845774ed02e72b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.173+0000 7fe272ffd700 1 -- 172.21.15.52:0/2454170240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280014620 con 0x55c2620b97a0 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.173+0000 7fe272ffd700 1 -- 172.21.15.52:0/2454170240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe280019070 con 0x55c2620b97a0 2024-06-02T23:24:55.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.173+0000 7fe272ffd700 1 -- 172.21.15.52:0/2454170240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe280013980 con 0x55c2620b97a0 2024-06-02T23:24:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 -- 172.21.15.52:0/2454170240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 msgr2=0x55c2620b9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620b9bc0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fe280009f70 tx=0x7fe280009650 comp rx=0 tx=0).stop 2024-06-02T23:24:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 -- 172.21.15.52:0/2454170240 shutdown_connections 2024-06-02T23:24:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620b9bc0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620bf340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 --2- 172.21.15.52:0/2454170240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 -- 172.21.15.52:0/2454170240 >> 172.21.15.52:0/2454170240 conn(0x55c261fe6080 msgr2=0x55c2620b4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:55.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.174+0000 7fe299858b80 1 -- 172.21.15.52:0/2454170240 shutdown_connections 2024-06-02T23:24:55.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.175+0000 7fe299858b80 1 -- 172.21.15.52:0/2454170240 wait complete. 2024-06-02T23:24:55.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.175+0000 7fe299858b80 1 -- start start 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.175+0000 7fe299858b80 1 Processor -- start 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620c2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620c2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c261faafc0 con 0x55c2620b97a0 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c262006810 con 0x55c2620bf880 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2620c6260 con 0x55c2620bef60 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe299858b80 10 client.0 _ll_register_callbacks cb 0x55c261fa05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:55.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620c2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620c2840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38410/0 (socket says 172.21.15.52:38410) 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 -- 172.21.15.52:0/3494868076 learned_addr learned my addr 172.21.15.52:0/3494868076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe28d572700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620c2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 -- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 msgr2=0x55c2620c2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620c2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 -- 172.21.15.52:0/3494868076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 msgr2=0x55c2620c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620c5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.176+0000 7fe287fff700 1 -- 172.21.15.52:0/3494868076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c261fd0750 con 0x55c2620bef60 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe287fff700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620c2840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe274004580 tx=0x7fe2740047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=269cab2f54e3337d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe267fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe267fff700 1 -- 172.21.15.52:0/3494868076 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe274027420 con 0x55c2620bef60 2024-06-02T23:24:55.179 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe267fff700 1 -- 172.21.15.52:0/3494868076 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe27401d410 con 0x55c2620bef60 2024-06-02T23:24:55.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe280019070 con 0x55c2620bef60 2024-06-02T23:24:55.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe267fff700 1 -- 172.21.15.52:0/3494868076 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe27401d610 con 0x55c2620bef60 2024-06-02T23:24:55.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe299858b80 10 client.28219 fetch_fsmap learned FSMap version 34 2024-06-02T23:24:55.180 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.177+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c2620d0990 con 0x55c2620bef60 2024-06-02T23:24:55.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe267fff700 1 -- 172.21.15.52:0/3494868076 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe2740275c0 con 0x55c2620bef60 2024-06-02T23:24:55.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe267fff700 1 -- 172.21.15.52:0/3494868076 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe274032420 con 0x55c2620bef60 2024-06-02T23:24:55.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe299858b80 10 client.28219 fetch_fsmap finished waiting for FSMap version 34 2024-06-02T23:24:55.181 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe299858b80 10 client.28219 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe299858b80 20 client.28219 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe299858b80 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2620d2fd0 0x55c2620d53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe299858b80 4 client.28219 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fe280019070 con 0x55c2620d2fd0 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.178+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2620d2fd0 0x55c2620d53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.179+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2620d2fd0 0x55c2620d53c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f7b8e9fb3b8ce7 server_cookie=1c5675c3ef41b560 in_seq=0 out_seq=0 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.179+0000 7fe267fff700 10 client.28219 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe267fff700 1 -- 172.21.15.52:0/3494868076 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fe280019070 con 0x55c2620d2fd0 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe267fff700 10 client.28219 handle_command_reply: tid=0 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 client.28219 shutdown 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 msgr2=0x55c2620c2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620c2840 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fe274004580 tx=0x7fe2740047c0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.182 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 shutdown_connections 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2620d2fd0 0x55c2620d53c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bf880 0x55c2620c2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620b97a0 0x55c2620c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 --2- 172.21.15.52:0/3494868076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bef60 0x55c2620c2840 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 >> 172.21.15.52:0/3494868076 conn(0x55c261fe6080 msgr2=0x55c2620b3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.180+0000 7fe267fff700 0 client.28219 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.181+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 shutdown_connections 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.181+0000 7fe299858b80 1 -- 172.21.15.52:0/3494868076 wait complete. 2024-06-02T23:24:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.181+0000 7fe299858b80 20 client.28219 trim_cache size 0 max 16384 2024-06-02T23:24:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 Processor -- start 2024-06-02T23:24:55.216 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 -- start start 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 0x55c2620b9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bef60 0x55c2620bcee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bf880 0x55c2620b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881aebf0 con 0x55c2620bef60 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881b0760 con 0x55c2620b97a0 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe299858b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881af2c0 con 0x55c2620bf880 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 0x55c2620b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe28d572700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bf880 0x55c2620b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 0x55c2620b9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39020/0 (socket says 172.21.15.52:39020) 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe28cd71700 1 -- 172.21.15.52:0/2852502334 learned_addr learned my addr 172.21.15.52:0/2852502334 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:55.217 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.216+0000 7fe287fff700 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bef60 0x55c2620bcee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe28cd71700 1 -- 172.21.15.52:0/2852502334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bf880 msgr2=0x55c2620b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe28cd71700 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bf880 0x55c2620b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe28cd71700 1 -- 172.21.15.52:0/2852502334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bef60 msgr2=0x55c2620bcee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe28cd71700 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bef60 0x55c2620bcee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.218 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe28cd71700 1 -- 172.21.15.52:0/2852502334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2620c2610 con 0x55c2620b97a0 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe28cd71700 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 0x55c2620b9260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe27c022d10 tx=0x7fe27c087040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d015488d0a65312 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.217+0000 7fe2727fc700 1 -- 172.21.15.52:0/2852502334 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c0a09d0 con 0x55c2620b97a0 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.218+0000 7fe2727fc700 1 -- 172.21.15.52:0/2852502334 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe27c0a5200 con 0x55c2620b97a0 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.218+0000 7fe2727fc700 1 -- 172.21.15.52:0/2852502334 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27c022a90 con 0x55c2620b97a0 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 -- 172.21.15.52:0/2852502334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 msgr2=0x55c2620b9260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 0x55c2620b9260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe27c022d10 tx=0x7fe27c087040 comp rx=0 tx=0).stop 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 -- 172.21.15.52:0/2852502334 shutdown_connections 2024-06-02T23:24:55.225 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620bf880 0x55c2620b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620b97a0 0x55c2620b9260 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 --2- 172.21.15.52:0/2852502334 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bef60 0x55c2620bcee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 -- 172.21.15.52:0/2852502334 >> 172.21.15.52:0/2852502334 conn(0x55c261fe6080 msgr2=0x55c2620bc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 -- 172.21.15.52:0/2852502334 shutdown_connections 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.219+0000 7fe299858b80 1 -- 172.21.15.52:0/2852502334 wait complete. 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.220+0000 7fe299858b80 1 -- start start 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.220+0000 7fe299858b80 1 Processor -- start 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.220+0000 7fe299858b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620b97a0 0x55c2620cb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe299858b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 0x55c2620cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe299858b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bf880 0x55c2620da240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe299858b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881b0090 con 0x55c2620bf880 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe299858b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881af9c0 con 0x55c2620bef60 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe299858b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2881acbb0 con 0x55c2620b97a0 2024-06-02T23:24:55.226 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe299858b80 10 client.0 _ll_register_callbacks cb 0x55c261fa05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 0x55c2620cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 0x55c2620cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39032/0 (socket says 172.21.15.52:39032) 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 -- 172.21.15.52:0/3410421071 learned_addr learned my addr 172.21.15.52:0/3410421071 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe28cd71700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620b97a0 0x55c2620cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe28d572700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bf880 0x55c2620da240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 -- 172.21.15.52:0/3410421071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620b97a0 msgr2=0x55c2620cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620b97a0 0x55c2620cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 -- 172.21.15.52:0/3410421071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bf880 msgr2=0x55c2620da240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bf880 0x55c2620da240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 -- 172.21.15.52:0/3410421071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2620daa80 con 0x55c2620bef60 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620b97a0 0x55c2620cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.221+0000 7fe287fff700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 0x55c2620cbcb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55c2620c3e10 tx=0x7fe274032650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d23ccf69e3ba9ac8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:55.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe267fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe267fff700 1 -- 172.21.15.52:0/3410421071 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe27401d410 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe267fff700 1 -- 172.21.15.52:0/3410421071 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe274009040 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe27c0a5200 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe267fff700 1 -- 172.21.15.52:0/3410421071 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7fe274041070 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe299858b80 10 client.28541 fetch_fsmap learned FSMap version 34 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x55c26213d7b0 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.222+0000 7fe267fff700 1 -- 172.21.15.52:0/3410421071 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe27c0a5200 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe267fff700 1 -- 172.21.15.52:0/3410421071 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 34) ==== 8362+0+0 (secure 0 0 0) 0x7fe274027c20 con 0x55c2620bef60 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe299858b80 10 client.28541 fetch_fsmap finished waiting for FSMap version 34 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe299858b80 10 client.28541 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe299858b80 20 client.28541 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe299858b80 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c26213def0 0x55c262140310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe299858b80 4 client.28541 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:24:55.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c262017d50 con 0x55c26213def0 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c26213def0 0x55c262140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe28cd71700 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c26213def0 0x55c262140310 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3acd42d408c17ae server_cookie=114adbc3ce19260c in_seq=0 out_seq=0 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.223+0000 7fe267fff700 10 client.28541 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe267fff700 1 -- 172.21.15.52:0/3410421071 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c262017d50 con 0x55c26213def0 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe267fff700 10 client.28541 handle_command_reply: tid=0 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 client.28541 shutdown 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 msgr2=0x55c2620cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 0x55c2620cbcb0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x55c2620c3e10 tx=0x7fe274032650 comp rx=0 tx=0).stop 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 shutdown_connections 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c26213def0 0x55c262140310 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe267fff700 0 client.28541 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2620bf880 0x55c2620da240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2620b97a0 0x55c2620cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 --2- 172.21.15.52:0/3410421071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2620bef60 0x55c2620cbcb0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.224+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 >> 172.21.15.52:0/3410421071 conn(0x55c261fe6080 msgr2=0x55c2620bfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.225+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 shutdown_connections 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.225+0000 7fe299858b80 1 -- 172.21.15.52:0/3410421071 wait complete. 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.225+0000 7fe299858b80 20 client.28541 trim_cache size 0 max 16384 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stdout: "8832af8a-704f-4c82-b920-b11f3b62d958": { 2024-06-02T23:24:55.230 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:24:55.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "8832af8a-704f-4c82-b920-b11f3b62d958", 2024-06-02T23:24:55.231 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:24:55.231 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:55.231 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:24:55.231 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe2780a2ca0 msgr2=0x7fe2780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe2780a2ca0 0x7fe2780a5160 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fe288102840 tx=0x7fe274009260 comp rx=0 tx=0).stop 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 msgr2=0x7fe28819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 0x7fe28819ada0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe27c006010 tx=0x7fe27c0057c0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 shutdown_connections 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe2780a2ca0 0x7fe2780a5160 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe28810a6d0 0x7fe28819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe2881046d0 0x7fe28819ada0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 --2- 172.21.15.52:0/3329772484 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe288105090 0x7fe28819b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 >> 172.21.15.52:0/3329772484 conn(0x7fe2881001c0 msgr2=0x7fe288100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 shutdown_connections 2024-06-02T23:24:55.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:24:55.226+0000 7fe28efd5700 1 -- 172.21.15.52:0/3329772484 wait complete. 2024-06-02T23:24:55.415 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:55 smithi052 ceph-mon[32340]: pgmap v822: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-06-02T23:24:55.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:55 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:55.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:24:55.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:24:55.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:24:55.416 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:55 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:24:55.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 697 70.62 MB/sec warmup 3 sec latency 109.123 ms 2024-06-02T23:24:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:55 smithi123 ceph-mon[35977]: pgmap v822: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-06-02T23:24:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:55 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:24:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:24:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:24:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:55 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:24:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:55 smithi174 ceph-mon[35994]: pgmap v822: 97 pgs: 97 active+clean; 120 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 2.1 MiB/s wr, 153 op/s 2024-06-02T23:24:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:55 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:24:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:24:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:24:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:55 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.080+0000 7f4732031640 1 -- 172.21.15.52:0/804179493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 msgr2=0x7f472c108df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.080+0000 7f4732031640 1 --2- 172.21.15.52:0/804179493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c108df0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4724009be0 tx=0x7f472402a6c0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- 172.21.15.52:0/804179493 shutdown_connections 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 --2- 172.21.15.52:0/804179493 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f472c101ba0 0x7f472c102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 --2- 172.21.15.52:0/804179493 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c108df0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 --2- 172.21.15.52:0/804179493 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- 172.21.15.52:0/804179493 >> 172.21.15.52:0/804179493 conn(0x7f472c0fbf20 msgr2=0x7f472c0fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- 172.21.15.52:0/804179493 shutdown_connections 2024-06-02T23:24:56.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- 172.21.15.52:0/804179493 wait complete. 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 Processor -- start 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- start start 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c06aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f472c101ba0 0x7f472c19c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c470 con 0x7f472c1009b0 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c2f0 con 0x7f472c102560 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f4732031640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f472c10c5f0 con 0x7f472c101ba0 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.081+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c06aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c06aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43580/0 (socket says 172.21.15.52:43580) 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 -- 172.21.15.52:0/159394272 learned_addr learned my addr 172.21.15.52:0/159394272 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472bfff640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f4723fff640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f472c101ba0 0x7f472c19c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 -- 172.21.15.52:0/159394272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f472c101ba0 msgr2=0x7f472c19c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f472c101ba0 0x7f472c19c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 -- 172.21.15.52:0/159394272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 msgr2=0x7f472c19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 -- 172.21.15.52:0/159394272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f472c19f7c0 con 0x7f472c1009b0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472bfff640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f472b7fe640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c06aca0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e9a0 tx=0x7f471c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f47297fa640 1 -- 172.21.15.52:0/159394272 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c00cd80 con 0x7f472c1009b0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f472c19fab0 con 0x7f472c1009b0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f47297fa640 1 -- 172.21.15.52:0/159394272 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f471c0108e0 con 0x7f472c1009b0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f47297fa640 1 -- 172.21.15.52:0/159394272 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c0188a0 con 0x7f472c1009b0 2024-06-02T23:24:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.082+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f472c1a0050 con 0x7f472c1009b0 2024-06-02T23:24:56.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.084+0000 7f47297fa640 1 -- 172.21.15.52:0/159394272 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f471c010a90 con 0x7f472c1009b0 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 Processor -- start 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 -- start start 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c141b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1420500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1423000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e7c137e4f0 con 0x55e7c141b010 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7c131c870 con 0x55e7c141ba00 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4734e18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e7c137a4b0 con 0x55e7c1420b70 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c141b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c141b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43594/0 (socket says 172.21.15.52:43594) 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 -- 172.21.15.52:0/2533331998 learned_addr learned my addr 172.21.15.52:0/2533331998 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:56.092 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472b7fe640 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1420500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f47297fa640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47180a2cb0 0x7f47180a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f4723fff640 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1423000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 -- 172.21.15.52:0/2533331998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 msgr2=0x55e7c1423000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1423000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 -- 172.21.15.52:0/2533331998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 msgr2=0x55e7c1420500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1420500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472bfff640 1 -- 172.21.15.52:0/2533331998 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7c13389b0 con 0x55e7c141b010 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.085+0000 7f472b7fe640 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1420500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f47297fa640 1 -- 172.21.15.52:0/159394272 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f471c0cf400 con 0x7f472c1009b0 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f472bfff640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47180a2cb0 0x7f47180a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f472bfff640 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c141b430 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4724009be0 tx=0x7f4724005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de3be306e2994e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f4721ffb640 1 -- 172.21.15.52:0/2533331998 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4724036040 con 0x55e7c141b010 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f4721ffb640 1 -- 172.21.15.52:0/2533331998 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4724037040 con 0x55e7c141b010 2024-06-02T23:24:56.093 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f4721ffb640 1 -- 172.21.15.52:0/2533331998 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47240497a0 con 0x55e7c141b010 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.086+0000 7f472bfff640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47180a2cb0 0x7f47180a5170 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f472404b000 tx=0x7f4724048000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- 172.21.15.52:0/2533331998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 msgr2=0x55e7c141b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c141b430 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f4724009be0 tx=0x7f4724005eb0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- 172.21.15.52:0/2533331998 shutdown_connections 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1423000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1420500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- 172.21.15.52:0/2533331998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c141b430 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- 172.21.15.52:0/2533331998 >> 172.21.15.52:0/2533331998 conn(0x55e7c1416680 msgr2=0x55e7c1416a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- 172.21.15.52:0/2533331998 shutdown_connections 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- 172.21.15.52:0/2533331998 wait complete. 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- start start 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 Processor -- start 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c14252e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.094 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1425820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c142c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1423b20 con 0x55e7c141b010 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1307ae0 con 0x55e7c141ba00 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e7c1423ca0 con 0x55e7c1420b70 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.087+0000 7f4734e18740 10 client.0 _ll_register_callbacks cb 0x55e7c13409b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c14252e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c14252e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43598/0 (socket says 172.21.15.52:43598) 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 -- 172.21.15.52:0/2666524947 learned_addr learned my addr 172.21.15.52:0/2666524947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472bfff640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c142c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f4723fff640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1425820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 -- 172.21.15.52:0/2666524947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 msgr2=0x55e7c142c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c142c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 -- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 msgr2=0x55e7c1425820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.095 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1425820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 -- 172.21.15.52:0/2666524947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7c1092210 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f4723fff640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1425820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472bfff640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c142c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f472b7fe640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c14252e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f471c0058c0 tx=0x7f471c0094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6d816414ddd70b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f470affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f470affd640 1 -- 172.21.15.52:0/2666524947 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c010e20 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f470affd640 1 -- 172.21.15.52:0/2666524947 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f471c0cf110 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4724037040 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f470affd640 1 -- 172.21.15.52:0/2666524947 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f471c09c390 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f4734e18740 10 client.19140 fetch_fsmap learned FSMap version 35 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7c115e620 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.088+0000 7f470affd640 1 -- 172.21.15.52:0/2666524947 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4724037040 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f470affd640 1 -- 172.21.15.52:0/2666524947 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f471c0979e0 con 0x55e7c141b010 2024-06-02T23:24:56.096 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4734e18740 10 client.19140 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4734e18740 10 client.19140 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4734e18740 20 client.19140 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4734e18740 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c1431140 0x55e7c14335d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4734e18740 4 client.19140 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7c1391f10 con 0x55e7c1431140 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4723fff640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c1431140 0x55e7c14335d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f470affd640 10 client.19140 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.089+0000 7f4723fff640 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c1431140 0x55e7c14335d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a50e783e9db97410 server_cookie=16f00d7b784a7161 in_seq=0 out_seq=0 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f470affd640 1 -- 172.21.15.52:0/2666524947 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e7c1391f10 con 0x55e7c1431140 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f470affd640 10 client.19140 handle_command_reply: tid=0 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 client.19140 shutdown 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 msgr2=0x55e7c14252e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c14252e0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f471c0058c0 tx=0x7f471c0094d0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.097 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 shutdown_connections 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c1431140 0x55e7c14335d0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c142c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141ba00 0x55e7c1425820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f470affd640 0 client.19140 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 --2- 172.21.15.52:0/2666524947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141b010 0x55e7c14252e0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 >> 172.21.15.52:0/2666524947 conn(0x55e7c1416680 msgr2=0x55e7c14157e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 shutdown_connections 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 1 -- 172.21.15.52:0/2666524947 wait complete. 2024-06-02T23:24:56.098 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.090+0000 7f4734e18740 20 client.19140 trim_cache size 0 max 16384 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.105+0000 7f4734e18740 1 Processor -- start 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.105+0000 7f4734e18740 1 -- start start 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4734e18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c141b010 0x55e7c1419590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4734e18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c142c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4734e18740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c1420b70 0x55e7c1419ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4734e18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1a8c90 con 0x55e7c141ba00 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4734e18740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1aa830 con 0x55e7c1420b70 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4734e18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1a9360 con 0x55e7c141b010 2024-06-02T23:24:56.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c142c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c142c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43602/0 (socket says 172.21.15.52:43602) 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 -- 172.21.15.52:0/1495953462 learned_addr learned my addr 172.21.15.52:0/1495953462 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f472bfff640 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c1420b70 0x55e7c1419ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f472b7fe640 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c141b010 0x55e7c1419590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 -- 172.21.15.52:0/1495953462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c141b010 msgr2=0x55e7c1419590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c141b010 0x55e7c1419590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 -- 172.21.15.52:0/1495953462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c1420b70 msgr2=0x55e7c1419ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c1420b70 0x55e7c1419ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 -- 172.21.15.52:0/1495953462 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7c1433c50 con 0x55e7c141ba00 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f472b7fe640 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c141b010 0x55e7c1419590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f4723fff640 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c142c820 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x55e7c1433e80 tx=0x7f4714011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1699608f04f0ed6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:56.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f47217fa640 1 -- 172.21.15.52:0/1495953462 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714005020 con 0x55e7c141ba00 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f47217fa640 1 -- 172.21.15.52:0/1495953462 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47140047a0 con 0x55e7c141ba00 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.106+0000 7f47217fa640 1 -- 172.21.15.52:0/1495953462 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471401d7f0 con 0x55e7c141ba00 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 -- 172.21.15.52:0/1495953462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 msgr2=0x55e7c142c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c142c820 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x55e7c1433e80 tx=0x7f4714011810 comp rx=0 tx=0).stop 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 -- 172.21.15.52:0/1495953462 shutdown_connections 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c1420b70 0x55e7c1419ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c142c820 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 --2- 172.21.15.52:0/1495953462 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c141b010 0x55e7c1419590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 -- 172.21.15.52:0/1495953462 >> 172.21.15.52:0/1495953462 conn(0x55e7c1416680 msgr2=0x55e7c1416a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 -- 172.21.15.52:0/1495953462 shutdown_connections 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 -- 172.21.15.52:0/1495953462 wait complete. 2024-06-02T23:24:56.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.107+0000 7f4734e18740 1 -- start start 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 Processor -- start 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141b010 0x55e7c14298a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c1429de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1499ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1aa160 con 0x55e7c141ba00 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1a9a60 con 0x55e7c141b010 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f472c1a6c70 con 0x55e7c1420b70 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4734e18740 10 client.0 _ll_register_callbacks cb 0x7f471c00cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c1429de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c1429de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43614/0 (socket says 172.21.15.52:43614) 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 -- 172.21.15.52:0/1809979745 learned_addr learned my addr 172.21.15.52:0/1809979745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:24:56.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f472bfff640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1499ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f472b7fe640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141b010 0x55e7c14298a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 -- 172.21.15.52:0/1809979745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 msgr2=0x55e7c1499ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1499ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 -- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141b010 msgr2=0x55e7c14298a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141b010 0x55e7c14298a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 -- 172.21.15.52:0/1809979745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7c14164b0 con 0x55e7c141ba00 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f4723fff640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c1429de0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4714004770 tx=0x7f4714011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecef2de0e1a12b36 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.108+0000 7f470affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f470affd640 1 -- 172.21.15.52:0/1809979745 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140054b0 con 0x55e7c141ba00 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f470affd640 1 -- 172.21.15.52:0/1809979745 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4714004950 con 0x55e7c141ba00 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47140047a0 con 0x55e7c141ba00 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f470affd640 1 -- 172.21.15.52:0/1809979745 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f4714004b50 con 0x55e7c141ba00 2024-06-02T23:24:56.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 10 client.19146 fetch_fsmap learned FSMap version 35 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x55e7c149b540 con 0x55e7c141ba00 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f470affd640 1 -- 172.21.15.52:0/1809979745 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f47140047a0 con 0x55e7c141ba00 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f470affd640 1 -- 172.21.15.52:0/1809979745 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f4714027650 con 0x55e7c141ba00 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 10 client.19146 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 10 client.19146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 20 client.19146 populate_metadata read hostname 'smithi052' 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c149beb0 0x55e7c149e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 4 client.19146 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7c138bc20 con 0x55e7c149beb0 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.109+0000 7f472b7fe640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c149beb0 0x55e7c149e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.110+0000 7f470affd640 10 client.19146 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:56.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.110+0000 7f472b7fe640 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c149beb0 0x55e7c149e2d0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9fc119edfe4d4a3 server_cookie=2f83b264a05bb296 in_seq=0 out_seq=0 2024-06-02T23:24:56.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.370+0000 7f470affd640 1 -- 172.21.15.52:0/1809979745 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e7c138bc20 con 0x55e7c149beb0 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.370+0000 7f470affd640 10 client.19146 handle_command_reply: tid=0 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.370+0000 7f4734e18740 1 client.19146 shutdown 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 msgr2=0x55e7c1429de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c1429de0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f4714004770 tx=0x7f4714011e80 comp rx=0 tx=0).stop 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 shutdown_connections 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e7c149beb0 0x55e7c149e2d0 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e7c1420b70 0x55e7c1499ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e7c141ba00 0x55e7c1429de0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f470affd640 0 client.19146 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 --2- 172.21.15.52:0/1809979745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e7c141b010 0x55e7c14298a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 >> 172.21.15.52:0/1809979745 conn(0x55e7c1416680 msgr2=0x55e7c1419590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 shutdown_connections 2024-06-02T23:24:56.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 1 -- 172.21.15.52:0/1809979745 wait complete. 2024-06-02T23:24:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.371+0000 7f4734e18740 20 client.19146 trim_cache size 0 max 16384 2024-06-02T23:24:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:24:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:24:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47180a2cb0 msgr2=0x7f47180a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47180a2cb0 0x7f47180a5170 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f472404b000 tx=0x7f4724048000 comp rx=0 tx=0).stop 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 msgr2=0x7f472c06aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c06aca0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f471c00e9a0 tx=0x7f471c00ee60 comp rx=0 tx=0).stop 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 shutdown_connections 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47180a2cb0 0x7f47180a5170 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f472c101ba0 0x7f472c19c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f472c102560 0x7f472c19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 --2- 172.21.15.52:0/159394272 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f472c1009b0 0x7f472c06aca0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:24:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 >> 172.21.15.52:0/159394272 conn(0x7f472c0fbf20 msgr2=0x7f472c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:24:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 shutdown_connections 2024-06-02T23:24:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:24:56.372+0000 7f4732031640 1 -- 172.21.15.52:0/159394272 wait complete. 2024-06-02T23:24:56.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 807 53.67 MB/sec warmup 4 sec latency 353.170 ms 2024-06-02T23:24:56.756 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:56 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:56.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:56 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:56.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:56 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:24:57.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 904 43.40 MB/sec warmup 5 sec latency 249.703 ms 2024-06-02T23:24:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:57 smithi123 ceph-mon[35977]: pgmap v823: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 13 MiB/s wr, 233 op/s 2024-06-02T23:24:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:57 smithi174 ceph-mon[35994]: pgmap v823: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 13 MiB/s wr, 233 op/s 2024-06-02T23:24:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:57 smithi052 ceph-mon[32340]: pgmap v823: 97 pgs: 97 active+clean; 239 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 413 KiB/s rd, 13 MiB/s wr, 233 op/s 2024-06-02T23:24:58.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1019 37.96 MB/sec warmup 6 sec latency 191.279 ms 2024-06-02T23:24:58.931 INFO:tasks.cephfs.filesystem:scrub status for tag:8832af8a-704f-4c82-b920-b11f3b62d958 - {'path': '/', 'tag': '8832af8a-704f-4c82-b920-b11f3b62d958', 'options': 'recursive,force'} 2024-06-02T23:24:59.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1105 32.80 MB/sec warmup 7 sec latency 210.260 ms 2024-06-02T23:24:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:24:59 smithi123 ceph-mon[35977]: pgmap v824: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 17 MiB/s wr, 230 op/s 2024-06-02T23:24:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:24:59 smithi174 ceph-mon[35994]: pgmap v824: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 17 MiB/s wr, 230 op/s 2024-06-02T23:24:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:24:59 smithi052 ceph-mon[32340]: pgmap v824: 97 pgs: 97 active+clean; 284 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 326 KiB/s rd, 17 MiB/s wr, 230 op/s 2024-06-02T23:25:00.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1208 28.82 MB/sec warmup 8 sec latency 251.527 ms 2024-06-02T23:25:01.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1375 28.57 MB/sec warmup 9 sec latency 128.440 ms 2024-06-02T23:25:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:01 smithi123 ceph-mon[35977]: pgmap v825: 97 pgs: 97 active+clean; 295 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 317 op/s 2024-06-02T23:25:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:25:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:25:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:25:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:25:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:01 smithi174 ceph-mon[35994]: pgmap v825: 97 pgs: 97 active+clean; 295 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 317 op/s 2024-06-02T23:25:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:25:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:25:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:25:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:25:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:01 smithi052 ceph-mon[32340]: pgmap v825: 97 pgs: 97 active+clean; 295 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 18 MiB/s wr, 317 op/s 2024-06-02T23:25:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:25:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:25:01.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:25:01.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:25:02.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1516 26.08 MB/sec warmup 10 sec latency 100.170 ms 2024-06-02T23:25:03.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1629 23.80 MB/sec warmup 11 sec latency 152.773 ms 2024-06-02T23:25:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:03 smithi123 ceph-mon[35977]: pgmap v826: 97 pgs: 97 active+clean; 291 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 19 MiB/s wr, 373 op/s 2024-06-02T23:25:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:03 smithi174 ceph-mon[35994]: pgmap v826: 97 pgs: 97 active+clean; 291 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 19 MiB/s wr, 373 op/s 2024-06-02T23:25:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:03 smithi052 ceph-mon[32340]: pgmap v826: 97 pgs: 97 active+clean; 291 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 522 KiB/s rd, 19 MiB/s wr, 373 op/s 2024-06-02T23:25:04.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1810 21.94 MB/sec warmup 12 sec latency 111.116 ms 2024-06-02T23:25:05.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 1951 20.35 MB/sec warmup 13 sec latency 90.512 ms 2024-06-02T23:25:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:05 smithi123 ceph-mon[35977]: pgmap v827: 97 pgs: 97 active+clean; 291 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 19 MiB/s wr, 346 op/s 2024-06-02T23:25:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:05 smithi174 ceph-mon[35994]: pgmap v827: 97 pgs: 97 active+clean; 291 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 19 MiB/s wr, 346 op/s 2024-06-02T23:25:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:05 smithi052 ceph-mon[32340]: pgmap v827: 97 pgs: 97 active+clean; 291 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 19 MiB/s wr, 346 op/s 2024-06-02T23:25:06.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2079 19.02 MB/sec warmup 14 sec latency 134.498 ms 2024-06-02T23:25:07.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2242 18.10 MB/sec warmup 15 sec latency 103.170 ms 2024-06-02T23:25:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:07 smithi123 ceph-mon[35977]: pgmap v828: 97 pgs: 97 active+clean; 292 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 18 MiB/s wr, 382 op/s 2024-06-02T23:25:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:07 smithi174 ceph-mon[35994]: pgmap v828: 97 pgs: 97 active+clean; 292 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 18 MiB/s wr, 382 op/s 2024-06-02T23:25:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:07 smithi052 ceph-mon[32340]: pgmap v828: 97 pgs: 97 active+clean; 292 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 701 KiB/s rd, 18 MiB/s wr, 382 op/s 2024-06-02T23:25:08.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2357 17.43 MB/sec warmup 16 sec latency 105.124 ms 2024-06-02T23:25:09.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2440 16.64 MB/sec warmup 17 sec latency 113.565 ms 2024-06-02T23:25:09.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 -- 172.21.15.52:0/3277409576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878102520 msgr2=0x7f6878104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 --2- 172.21.15.52:0/3277409576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878102520 0x7f6878104910 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f6868009900 tx=0x7f686802a410 comp rx=0 tx=0).stop 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 -- 172.21.15.52:0/3277409576 shutdown_connections 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 --2- 172.21.15.52:0/3277409576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878107780 0x7f6878109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 --2- 172.21.15.52:0/3277409576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878104e50 0x7f6878107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 --2- 172.21.15.52:0/3277409576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878102520 0x7f6878104910 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 -- 172.21.15.52:0/3277409576 >> 172.21.15.52:0/3277409576 conn(0x7f68780fbe70 msgr2=0x7f68780fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 -- 172.21.15.52:0/3277409576 shutdown_connections 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 -- 172.21.15.52:0/3277409576 wait complete. 2024-06-02T23:25:09.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.479+0000 7f687d961640 1 Processor -- start 2024-06-02T23:25:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 -- start start 2024-06-02T23:25:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 0x7f6878101e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878104e50 0x7f68781004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 0x7f6878100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f687810d540 con 0x7f6878107780 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f687810d3c0 con 0x7f6878102520 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f687d961640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f687810d6c0 con 0x7f6878104e50 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f68777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 0x7f6878100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 0x7f6878101e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 0x7f6878101e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33948/0 (socket says 172.21.15.52:33948) 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f68777fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 0x7f6878100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45510/0 (socket says 172.21.15.52:45510) 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f6876ffd640 1 -- 172.21.15.52:0/702031084 learned_addr learned my addr 172.21.15.52:0/702031084 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.480+0000 7f68767fc640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878104e50 0x7f68781004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68767fc640 1 -- 172.21.15.52:0/702031084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 msgr2=0x7f6878101e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68767fc640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 0x7f6878101e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68767fc640 1 -- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 msgr2=0x7f6878100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68767fc640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 0x7f6878100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68767fc640 1 -- 172.21.15.52:0/702031084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68781012a0 con 0x7f6878104e50 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68777fe640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 0x7f6878100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f6876ffd640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 0x7f6878101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f68767fc640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878104e50 0x7f68781004d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f686c00e9c0 tx=0x7f686c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f687c95f640 1 -- 172.21.15.52:0/702031084 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00ccc0 con 0x7f6878104e50 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f687c95f640 1 -- 172.21.15.52:0/702031084 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c0108e0 con 0x7f6878104e50 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68781a1d00 con 0x7f6878104e50 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f687c95f640 1 -- 172.21.15.52:0/702031084 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c0187e0 con 0x7f6878104e50 2024-06-02T23:25:09.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.481+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68781a21e0 con 0x7f6878104e50 2024-06-02T23:25:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.484+0000 7f687c95f640 1 -- 172.21.15.52:0/702031084 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f686c010a90 con 0x7f6878104e50 2024-06-02T23:25:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 Processor -- start 2024-06-02T23:25:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 -- start start 2024-06-02T23:25:09.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f687c95f640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68600a2cb0 0x7f68600a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885716430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x55688571b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5568856794f0 con 0x556885716a00 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556885617870 con 0x55688571bb70 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f688066b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5568856754b0 con 0x556885716010 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f68777fe640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68600a2cb0 0x7f68600a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885716430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885716430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:45936/0 (socket says 172.21.15.52:45936) 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f6876ffd640 1 -- 172.21.15.52:0/3547242497 learned_addr learned my addr 172.21.15.52:0/3547242497 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:09.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.485+0000 7f687c95f640 1 -- 172.21.15.52:0/702031084 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f686c0cf2a0 con 0x7f6878104e50 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f68767fc640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x55688571b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f6876ffd640 1 -- 172.21.15.52:0/3547242497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 msgr2=0x55688571e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f68777fe640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f6876ffd640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f6876ffd640 1 -- 172.21.15.52:0/3547242497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 msgr2=0x55688571b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f6876ffd640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x55688571b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f6876ffd640 1 -- 172.21.15.52:0/3547242497 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568856339b0 con 0x556885716010 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f68767fc640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x55688571b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f68777fe640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f6876ffd640 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885716430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6868009a20 tx=0x7f6868032f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1300bc0cde7157e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f68777fe640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68600a2cb0 0x7f68600a5170 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6870005fd0 tx=0x7f6870010cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f685dffb640 1 -- 172.21.15.52:0/3547242497 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868035070 con 0x556885716010 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f685dffb640 1 -- 172.21.15.52:0/3547242497 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68680035f0 con 0x556885716010 2024-06-02T23:25:09.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.486+0000 7f685dffb640 1 -- 172.21.15.52:0/3547242497 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868031900 con 0x556885716010 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 -- 172.21.15.52:0/3547242497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 msgr2=0x556885716430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885716430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6868009a20 tx=0x7f6868032f60 comp rx=0 tx=0).stop 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 -- 172.21.15.52:0/3547242497 shutdown_connections 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x55688571b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 --2- 172.21.15.52:0/3547242497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885716430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.487+0000 7f688066b740 1 -- 172.21.15.52:0/3547242497 >> 172.21.15.52:0/3547242497 conn(0x556885711680 msgr2=0x556885711a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.488+0000 7f688066b740 1 -- 172.21.15.52:0/3547242497 shutdown_connections 2024-06-02T23:25:09.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.488+0000 7f688066b740 1 -- 172.21.15.52:0/3547242497 wait complete. 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.488+0000 7f688066b740 1 -- start start 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.488+0000 7f688066b740 1 Processor -- start 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 0x5568857201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556885716a00 0x556885720700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55688571bb70 0x556885727740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55688571eac0 con 0x556885716010 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556885602ae0 con 0x556885716a00 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55688571ec40 con 0x55688571bb70 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f688066b740 10 client.0 _ll_register_callbacks cb 0x55688563b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 0x5568857201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 0x5568857201c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45522/0 (socket says 172.21.15.52:45522) 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 -- 172.21.15.52:0/1107069382 learned_addr learned my addr 172.21.15.52:0/1107069382 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:09.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f68767fc640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556885716a00 0x556885720700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f68777fe640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55688571bb70 0x556885727740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 -- 172.21.15.52:0/1107069382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55688571bb70 msgr2=0x556885727740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55688571bb70 0x556885727740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 -- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556885716a00 msgr2=0x556885720700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556885716a00 0x556885720700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 -- 172.21.15.52:0/1107069382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55688538d210 con 0x556885716010 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6876ffd640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 0x5568857201c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6868032fd0 tx=0x7f68680317f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f309731e16aac19 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.489+0000 7f6856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f6856ffd640 1 -- 172.21.15.52:0/1107069382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868053060 con 0x556885716010 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f6856ffd640 1 -- 172.21.15.52:0/1107069382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68680041b0 con 0x556885716010 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68680035f0 con 0x556885716010 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f6856ffd640 1 -- 172.21.15.52:0/1107069382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6868035050 con 0x556885716010 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 10 client.19158 fetch_fsmap learned FSMap version 35 2024-06-02T23:25:09.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x556885459620 con 0x556885716010 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f6856ffd640 1 -- 172.21.15.52:0/1107069382 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68680035f0 con 0x556885716010 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f6856ffd640 1 -- 172.21.15.52:0/1107069382 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f6868045ab0 con 0x556885716010 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 10 client.19158 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 10 client.19158 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 20 client.19158 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.490+0000 7f688066b740 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x55688572df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.491+0000 7f688066b740 4 client.19158 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.491+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55688568cf10 con 0x55688572bba0 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.491+0000 7f68767fc640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x55688572df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.491+0000 7f6856ffd640 10 client.19158 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:09.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.491+0000 7f68767fc640 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x55688572df90 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e99cd8f3ca5f12a6 server_cookie=7cf3de44e93d575c in_seq=0 out_seq=0 2024-06-02T23:25:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.492+0000 7f6856ffd640 1 -- 172.21.15.52:0/1107069382 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55688568cf10 con 0x55688572bba0 2024-06-02T23:25:09.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.492+0000 7f6856ffd640 10 client.19158 handle_command_reply: tid=0 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.492+0000 7f688066b740 1 client.19158 shutdown 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 msgr2=0x5568857201c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 0x5568857201c0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f6868032fd0 tx=0x7f68680317f0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 shutdown_connections 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x55688572df90 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55688571bb70 0x556885727740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f6856ffd640 0 client.19158 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556885716a00 0x556885720700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 --2- 172.21.15.52:0/1107069382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716010 0x5568857201c0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 >> 172.21.15.52:0/1107069382 conn(0x556885711680 msgr2=0x5568857107e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 shutdown_connections 2024-06-02T23:25:09.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 1 -- 172.21.15.52:0/1107069382 wait complete. 2024-06-02T23:25:09.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.493+0000 7f688066b740 20 client.19158 trim_cache size 0 max 16384 2024-06-02T23:25:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 Processor -- start 2024-06-02T23:25:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 -- start start 2024-06-02T23:25:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885714b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885727700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68781aaec0 con 0x556885716a00 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68781aca60 con 0x55688571bb70 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f688066b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68781ab590 con 0x556885716010 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885727700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885727700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45530/0 (socket says 172.21.15.52:45530) 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 -- 172.21.15.52:0/2372774240 learned_addr learned my addr 172.21.15.52:0/2372774240 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68777fe640 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f6876ffd640 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885714b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 -- 172.21.15.52:0/2372774240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 msgr2=0x556885714b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885714b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 -- 172.21.15.52:0/2372774240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 msgr2=0x55688571f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.520+0000 7f68767fc640 1 -- 172.21.15.52:0/2372774240 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556885716de0 con 0x556885716a00 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.521+0000 7f68767fc640 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885727700 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f686c09b8e0 tx=0x7f686c009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84a5e9e93ede3aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.521+0000 7f685d7fa640 1 -- 172.21.15.52:0/2372774240 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c09ce60 con 0x556885716a00 2024-06-02T23:25:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.521+0000 7f685d7fa640 1 -- 172.21.15.52:0/2372774240 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c099c80 con 0x556885716a00 2024-06-02T23:25:09.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.521+0000 7f685d7fa640 1 -- 172.21.15.52:0/2372774240 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c086040 con 0x556885716a00 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 -- 172.21.15.52:0/2372774240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 msgr2=0x556885727700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885727700 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f686c09b8e0 tx=0x7f686c009700 comp rx=0 tx=0).stop 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 -- 172.21.15.52:0/2372774240 shutdown_connections 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x55688571f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885727700 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 --2- 172.21.15.52:0/2372774240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885714b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 -- 172.21.15.52:0/2372774240 >> 172.21.15.52:0/2372774240 conn(0x556885711680 msgr2=0x556885714470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 -- 172.21.15.52:0/2372774240 shutdown_connections 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 -- 172.21.15.52:0/2372774240 wait complete. 2024-06-02T23:25:09.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 -- start start 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.522+0000 7f688066b740 1 Processor -- start 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885724a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885724f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x556885725fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68781ac390 con 0x556885716a00 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68781abc90 con 0x55688571bb70 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68781a8ea0 con 0x556885716010 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f688066b740 10 client.0 _ll_register_callbacks cb 0x7f6868032eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885724f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x556885725fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885724a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885724f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45536/0 (socket says 172.21.15.52:45536) 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x556885725fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33976/0 (socket says 172.21.15.52:33976) 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f6876ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885724a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:45968/0 (socket says 172.21.15.52:45968) 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 -- 172.21.15.52:0/1717163483 learned_addr learned my addr 172.21.15.52:0/1717163483 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 -- 172.21.15.52:0/1717163483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 msgr2=0x556885724a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885724a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 -- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 msgr2=0x556885725fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x556885725fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 -- 172.21.15.52:0/1717163483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556885726590 con 0x556885716a00 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f6876ffd640 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885724a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f68767fc640 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885724f90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f686c098040 tx=0x7f686c00efb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63414fb59cac9905 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.523+0000 7f6856ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f6856ffd640 1 -- 172.21.15.52:0/1717163483 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c086040 con 0x556885716a00 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f6856ffd640 1 -- 172.21.15.52:0/1717163483 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f686c09b2a0 con 0x556885716a00 2024-06-02T23:25:09.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c099c80 con 0x556885716a00 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f6856ffd640 1 -- 172.21.15.52:0/1717163483 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f686c09ce60 con 0x556885716a00 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 10 client.19164 fetch_fsmap learned FSMap version 35 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5568857278a0 con 0x556885716a00 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f6856ffd640 1 -- 172.21.15.52:0/1717163483 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c099c80 con 0x556885716a00 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f6856ffd640 1 -- 172.21.15.52:0/1717163483 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f686c09abc0 con 0x556885716a00 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 10 client.19164 fetch_fsmap finished waiting for FSMap version 35 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 10 client.19164 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 20 client.19164 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x556885798d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 4 client.19164 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556885686c20 con 0x55688572bba0 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.524+0000 7f6876ffd640 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x556885798d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:09.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.525+0000 7f6856ffd640 10 client.19164 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:09.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.525+0000 7f6876ffd640 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x556885798d20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=536cdd6352c3456f server_cookie=13fd936389b8db6b in_seq=0 out_seq=0 2024-06-02T23:25:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f6856ffd640 1 -- 172.21.15.52:0/1717163483 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556885686c20 con 0x55688572bba0 2024-06-02T23:25:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f6856ffd640 10 client.19164 handle_command_reply: tid=0 2024-06-02T23:25:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 client.19164 shutdown 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 msgr2=0x556885724f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885724f90 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f686c098040 tx=0x7f686c00efb0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 shutdown_connections 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55688572bba0 0x556885798d20 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55688571bb70 0x556885725fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556885716a00 0x556885724f90 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 --2- 172.21.15.52:0/1717163483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556885716010 0x556885724a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 >> 172.21.15.52:0/1717163483 conn(0x556885711680 msgr2=0x556885714d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f6856ffd640 0 client.19164 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 shutdown_connections 2024-06-02T23:25:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 1 -- 172.21.15.52:0/1717163483 wait complete. 2024-06-02T23:25:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.748+0000 7f688066b740 20 client.19164 trim_cache size 0 max 16384 2024-06-02T23:25:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:25:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:25:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68600a2cb0 msgr2=0x7f68600a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68600a2cb0 0x7f68600a5170 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6870005fd0 tx=0x7f6870010cf0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878104e50 msgr2=0x7f68781004d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878104e50 0x7f68781004d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f686c00e9c0 tx=0x7f686c00ee80 comp rx=0 tx=0).stop 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 shutdown_connections 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68600a2cb0 0x7f68600a5170 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6878104e50 0x7f68781004d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6878107780 0x7f6878100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 --2- 172.21.15.52:0/702031084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6878102520 0x7f6878101e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 >> 172.21.15.52:0/702031084 conn(0x7f68780fbe70 msgr2=0x7f6878102e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 shutdown_connections 2024-06-02T23:25:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:09.749+0000 7f687d961640 1 -- 172.21.15.52:0/702031084 wait complete. 2024-06-02T23:25:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:09 smithi123 ceph-mon[35977]: pgmap v829: 97 pgs: 97 active+clean; 292 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 6.8 MiB/s wr, 305 op/s 2024-06-02T23:25:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:09 smithi174 ceph-mon[35994]: pgmap v829: 97 pgs: 97 active+clean; 292 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 6.8 MiB/s wr, 305 op/s 2024-06-02T23:25:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:09 smithi052 ceph-mon[32340]: pgmap v829: 97 pgs: 97 active+clean; 292 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 541 KiB/s rd, 6.8 MiB/s wr, 305 op/s 2024-06-02T23:25:10.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2554 16.03 MB/sec warmup 18 sec latency 217.127 ms 2024-06-02T23:25:11.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2701 16.37 MB/sec warmup 19 sec latency 118.492 ms 2024-06-02T23:25:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:11 smithi123 ceph-mon[35977]: pgmap v830: 97 pgs: 97 active+clean; 296 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.4 MiB/s wr, 301 op/s 2024-06-02T23:25:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:11 smithi174 ceph-mon[35994]: pgmap v830: 97 pgs: 97 active+clean; 296 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.4 MiB/s wr, 301 op/s 2024-06-02T23:25:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:11 smithi052 ceph-mon[32340]: pgmap v830: 97 pgs: 97 active+clean; 296 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 3.4 MiB/s wr, 301 op/s 2024-06-02T23:25:12.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 2981 16.84 MB/sec warmup 20 sec latency 132.376 ms 2024-06-02T23:25:12.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:12 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:25:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:12 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:25:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:12 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:25:13.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 3248 16.58 MB/sec warmup 21 sec latency 124.978 ms 2024-06-02T23:25:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:13 smithi123 ceph-mon[35977]: pgmap v831: 97 pgs: 97 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.2 MiB/s wr, 248 op/s 2024-06-02T23:25:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:13 smithi174 ceph-mon[35994]: pgmap v831: 97 pgs: 97 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.2 MiB/s wr, 248 op/s 2024-06-02T23:25:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:13 smithi052 ceph-mon[32340]: pgmap v831: 97 pgs: 97 active+clean; 293 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 607 KiB/s rd, 2.2 MiB/s wr, 248 op/s 2024-06-02T23:25:14.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 3365 16.40 MB/sec warmup 22 sec latency 119.608 ms 2024-06-02T23:25:15.446 INFO:tasks.workunit.client.0.smithi052.stdout: 10 3435 15.77 MB/sec warmup 23 sec latency 92.960 ms 2024-06-02T23:25:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:15 smithi123 ceph-mon[35977]: pgmap v832: 97 pgs: 97 active+clean; 302 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-06-02T23:25:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:15 smithi174 ceph-mon[35994]: pgmap v832: 97 pgs: 97 active+clean; 302 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-06-02T23:25:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:15 smithi052 ceph-mon[32340]: pgmap v832: 97 pgs: 97 active+clean; 302 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 693 KiB/s rd, 2.7 MiB/s wr, 224 op/s 2024-06-02T23:25:16.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 3507 15.15 MB/sec warmup 24 sec latency 96.386 ms 2024-06-02T23:25:17.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 3712 15.57 MB/sec warmup 25 sec latency 175.172 ms 2024-06-02T23:25:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:17 smithi123 ceph-mon[35977]: pgmap v833: 97 pgs: 97 active+clean; 309 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 3.7 MiB/s wr, 245 op/s 2024-06-02T23:25:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:17 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:25:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:17 smithi174 ceph-mon[35994]: pgmap v833: 97 pgs: 97 active+clean; 309 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 3.7 MiB/s wr, 245 op/s 2024-06-02T23:25:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:17 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:25:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:17 smithi052 ceph-mon[32340]: pgmap v833: 97 pgs: 97 active+clean; 309 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 3.7 MiB/s wr, 245 op/s 2024-06-02T23:25:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:17 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:25:18.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 3899 15.26 MB/sec warmup 26 sec latency 58.665 ms 2024-06-02T23:25:19.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4043 14.97 MB/sec warmup 27 sec latency 50.069 ms 2024-06-02T23:25:19.712 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:19 smithi052 ceph-mon[32340]: pgmap v834: 97 pgs: 97 active+clean; 311 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.8 MiB/s wr, 199 op/s 2024-06-02T23:25:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:19 smithi123 ceph-mon[35977]: pgmap v834: 97 pgs: 97 active+clean; 311 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.8 MiB/s wr, 199 op/s 2024-06-02T23:25:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:19 smithi174 ceph-mon[35994]: pgmap v834: 97 pgs: 97 active+clean; 311 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 572 KiB/s rd, 3.8 MiB/s wr, 199 op/s 2024-06-02T23:25:20.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4163 14.56 MB/sec warmup 28 sec latency 111.149 ms 2024-06-02T23:25:21.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4279 14.08 MB/sec warmup 29 sec latency 69.130 ms 2024-06-02T23:25:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:21 smithi123 ceph-mon[35977]: pgmap v835: 97 pgs: 97 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 4.9 MiB/s wr, 241 op/s 2024-06-02T23:25:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:21 smithi174 ceph-mon[35994]: pgmap v835: 97 pgs: 97 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 4.9 MiB/s wr, 241 op/s 2024-06-02T23:25:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:21 smithi052 ceph-mon[32340]: pgmap v835: 97 pgs: 97 active+clean; 312 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 867 KiB/s rd, 4.9 MiB/s wr, 241 op/s 2024-06-02T23:25:22.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4441 13.75 MB/sec warmup 30 sec latency 103.879 ms 2024-06-02T23:25:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 -- 172.21.15.52:0/111763828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c109300 msgr2=0x7fd17c10fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 --2- 172.21.15.52:0/111763828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c109300 0x7fd17c10fce0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd170009900 tx=0x7fd17002a410 comp rx=0 tx=0).stop 2024-06-02T23:25:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 -- 172.21.15.52:0/111763828 shutdown_connections 2024-06-02T23:25:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 --2- 172.21.15.52:0/111763828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c110220 0x7fd17c112610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 --2- 172.21.15.52:0/111763828 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c109300 0x7fd17c10fce0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 --2- 172.21.15.52:0/111763828 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c1089a0 0x7fd17c108dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 -- 172.21.15.52:0/111763828 >> 172.21.15.52:0/111763828 conn(0x7fd17c06ebd0 msgr2=0x7fd17c071010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 -- 172.21.15.52:0/111763828 shutdown_connections 2024-06-02T23:25:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.894+0000 7fd18182c640 1 -- 172.21.15.52:0/111763828 wait complete. 2024-06-02T23:25:22.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 Processor -- start 2024-06-02T23:25:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 -- start start 2024-06-02T23:25:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c1089a0 0x7fd17c1a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 0x7fd17c1a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.896 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c110220 0x7fd17c1adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c10b620 con 0x7fd17c110220 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c10b4a0 con 0x7fd17c1089a0 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.895+0000 7fd18182c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c10b7a0 con 0x7fd17c109300 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 0x7fd17c1a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c1089a0 0x7fd17c1a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 0x7fd17c1a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47552/0 (socket says 172.21.15.52:47552) 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c1089a0 0x7fd17c1a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35354/0 (socket says 172.21.15.52:35354) 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 -- 172.21.15.52:0/1480830469 learned_addr learned my addr 172.21.15.52:0/1480830469 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c110220 0x7fd17c1adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 -- 172.21.15.52:0/1480830469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c1089a0 msgr2=0x7fd17c1a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c1089a0 0x7fd17c1a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.897 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 -- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c110220 msgr2=0x7fd17c1adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c110220 0x7fd17c1adf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 -- 172.21.15.52:0/1480830469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd17c1ae5c0 con 0x7fd17c109300 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c110220 0x7fd17c1adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 0x7fd17c1a6ee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1700314e0 tx=0x7fd170004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.896+0000 7fd18082a640 1 -- 172.21.15.52:0/1480830469 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17003f030 con 0x7fd17c109300 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.897+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd17c1ae850 con 0x7fd17c109300 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.897+0000 7fd18082a640 1 -- 172.21.15.52:0/1480830469 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd170003aa0 con 0x7fd17c109300 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.897+0000 7fd18082a640 1 -- 172.21.15.52:0/1480830469 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd170035070 con 0x7fd17c109300 2024-06-02T23:25:22.898 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.897+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd17c1aed90 con 0x7fd17c109300 2024-06-02T23:25:22.899 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.900+0000 7fd18082a640 1 -- 172.21.15.52:0/1480830469 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd170042030 con 0x7fd17c109300 2024-06-02T23:25:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.900+0000 7fd18460f740 1 Processor -- start 2024-06-02T23:25:22.900 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 -- start start 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534010 0x55ca2b534430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 0x55ca2b539500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b539b70 0x55ca2b53c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ca2b4974f0 con 0x55ca2b534a00 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca2b435870 con 0x55ca2b539b70 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18460f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca2b4934b0 con 0x55ca2b534010 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd18082a640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1640a2cb0 0x7fd1640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd17affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 0x55ca2b539500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd17affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 0x55ca2b539500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45692/0 (socket says 172.21.15.52:45692) 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd17affd640 1 -- 172.21.15.52:0/3341233033 learned_addr learned my addr 172.21.15.52:0/3341233033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:22.901 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.901+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b539b70 0x55ca2b53c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd18082a640 1 -- 172.21.15.52:0/1480830469 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd1700ef730 con 0x7fd17c109300 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd17affd640 1 -- 172.21.15.52:0/3341233033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534010 msgr2=0x55ca2b534430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd17affd640 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534010 0x55ca2b534430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd17affd640 1 -- 172.21.15.52:0/3341233033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b539b70 msgr2=0x55ca2b53c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd17affd640 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b539b70 0x55ca2b53c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd17affd640 1 -- 172.21.15.52:0/3341233033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2b4519b0 con 0x55ca2b534a00 2024-06-02T23:25:22.902 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.902+0000 7fd17affd640 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 0x55ca2b539500 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009cf0 tx=0x7fd16c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=906ed041749c1721 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.903+0000 7fd161ffb640 1 -- 172.21.15.52:0/3341233033 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c004230 con 0x55ca2b534a00 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.903+0000 7fd161ffb640 1 -- 172.21.15.52:0/3341233033 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd16c0043d0 con 0x55ca2b534a00 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.903+0000 7fd161ffb640 1 -- 172.21.15.52:0/3341233033 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c019850 con 0x55ca2b534a00 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.903+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1640a2cb0 0x7fd1640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 -- 172.21.15.52:0/3341233033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 msgr2=0x55ca2b539500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 0x55ca2b539500 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fd16c009cf0 tx=0x7fd16c011810 comp rx=0 tx=0).stop 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1640a2cb0 0x7fd1640a5170 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd174005c40 tx=0x7fd174009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:22.904 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 -- 172.21.15.52:0/3341233033 shutdown_connections 2024-06-02T23:25:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b539b70 0x55ca2b53c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534a00 0x55ca2b539500 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 --2- 172.21.15.52:0/3341233033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534010 0x55ca2b534430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.904+0000 7fd18460f740 1 -- 172.21.15.52:0/3341233033 >> 172.21.15.52:0/3341233033 conn(0x55ca2b52f680 msgr2=0x55ca2b52fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.905+0000 7fd18460f740 1 -- 172.21.15.52:0/3341233033 shutdown_connections 2024-06-02T23:25:22.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.905+0000 7fd18460f740 1 -- 172.21.15.52:0/3341233033 wait complete. 2024-06-02T23:25:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.905+0000 7fd18460f740 1 -- start start 2024-06-02T23:25:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.906+0000 7fd18460f740 1 Processor -- start 2024-06-02T23:25:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.906+0000 7fd18460f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 0x55ca2b53e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.906+0000 7fd18460f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b53e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.908 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.906+0000 7fd18460f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b539b70 0x55ca2b545830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 0x55ca2b53e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 0x55ca2b53e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45708/0 (socket says 172.21.15.52:45708) 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 -- 172.21.15.52:0/1345525702 learned_addr learned my addr 172.21.15.52:0/1345525702 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ca2b53cb20 con 0x55ca2b534010 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca2b420ae0 con 0x55ca2b534a00 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b53e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ca2b53cca0 con 0x55ca2b539b70 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b539b70 0x55ca2b545830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd18460f740 10 client.0 _ll_register_callbacks cb 0x55ca2b4599b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 -- 172.21.15.52:0/1345525702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b539b70 msgr2=0x55ca2b545830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b539b70 0x55ca2b545830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 -- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 msgr2=0x55ca2b53e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.909 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b53e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2b1ab210 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd17affd640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 0x55ca2b53e2b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd16c01a000 tx=0x7fd16c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412b718511a728e4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.907+0000 7fd15affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd15affd640 1 -- 172.21.15.52:0/1345525702 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c027070 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd16c0043d0 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd15affd640 1 -- 172.21.15.52:0/1345525702 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd16c005240 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd15affd640 1 -- 172.21.15.52:0/1345525702 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd16c033050 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd18460f740 10 client.19173 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ca2b277620 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd15affd640 1 -- 172.21.15.52:0/1345525702 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd16c0043d0 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.908+0000 7fd15affd640 1 -- 172.21.15.52:0/1345525702 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fd16c004c90 con 0x55ca2b534010 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd18460f740 10 client.19173 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd18460f740 10 client.19173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:22.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd18460f740 20 client.19173 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd18460f740 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b549ea0 0x55ca2b54c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd18460f740 4 client.19173 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca2b4aaf10 con 0x55ca2b549ea0 2024-06-02T23:25:22.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b549ea0 0x55ca2b54c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b549ea0 0x55ca2b54c310 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0a440879c9ce83 server_cookie=7bebceccfcfe759d in_seq=0 out_seq=0 2024-06-02T23:25:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.909+0000 7fd15affd640 10 client.19173 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.910+0000 7fd15affd640 1 -- 172.21.15.52:0/1345525702 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ca2b4aaf10 con 0x55ca2b549ea0 2024-06-02T23:25:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.910+0000 7fd15affd640 10 client.19173 handle_command_reply: tid=0 2024-06-02T23:25:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.910+0000 7fd18460f740 1 client.19173 shutdown 2024-06-02T23:25:22.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 msgr2=0x55ca2b53e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 0x55ca2b53e2b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fd16c01a000 tx=0x7fd16c003b90 comp rx=0 tx=0).stop 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 shutdown_connections 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b549ea0 0x55ca2b54c310 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b539b70 0x55ca2b545830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b53e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd15affd640 0 client.19173 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 --2- 172.21.15.52:0/1345525702 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b534010 0x55ca2b53e2b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 >> 172.21.15.52:0/1345525702 conn(0x55ca2b52f680 msgr2=0x55ca2b52e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 shutdown_connections 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 1 -- 172.21.15.52:0/1345525702 wait complete. 2024-06-02T23:25:22.913 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.911+0000 7fd18460f740 20 client.19173 trim_cache size 0 max 16384 2024-06-02T23:25:22.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.944+0000 7fd18460f740 1 Processor -- start 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.944+0000 7fd18460f740 1 -- start start 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.944+0000 7fd18460f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b5392e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd18460f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b5457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd18460f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b549ea0 0x55ca2b532b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd18460f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b7af0 con 0x55ca2b539b70 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd18460f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b9750 con 0x55ca2b534a00 2024-06-02T23:25:22.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd18460f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b81c0 con 0x55ca2b549ea0 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd17affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b5392e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd17b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b549ea0 0x55ca2b532b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd17a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b5457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd17affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b5392e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35382/0 (socket says 172.21.15.52:35382) 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.945+0000 7fd17affd640 1 -- 172.21.15.52:0/420124183 learned_addr learned my addr 172.21.15.52:0/420124183 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd17b7fe640 1 -- 172.21.15.52:0/420124183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 msgr2=0x55ca2b5392e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b5392e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd17b7fe640 1 -- 172.21.15.52:0/420124183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 msgr2=0x55ca2b5457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b5457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd17b7fe640 1 -- 172.21.15.52:0/420124183 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca2b5347a0 con 0x55ca2b549ea0 2024-06-02T23:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b549ea0 0x55ca2b532b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ca2b535260 tx=0x7fd174012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff9cd0ad224b84a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd1617fa640 1 -- 172.21.15.52:0/420124183 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd17401f7b0 con 0x55ca2b549ea0 2024-06-02T23:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd1617fa640 1 -- 172.21.15.52:0/420124183 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd174023070 con 0x55ca2b549ea0 2024-06-02T23:25:22.946 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.946+0000 7fd1617fa640 1 -- 172.21.15.52:0/420124183 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd174028d60 con 0x55ca2b549ea0 2024-06-02T23:25:22.947 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 -- 172.21.15.52:0/420124183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b549ea0 msgr2=0x55ca2b532b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b549ea0 0x55ca2b532b80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55ca2b535260 tx=0x7fd174012000 comp rx=0 tx=0).stop 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 -- 172.21.15.52:0/420124183 shutdown_connections 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b549ea0 0x55ca2b532b80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b5457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 --2- 172.21.15.52:0/420124183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b534a00 0x55ca2b5392e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 -- 172.21.15.52:0/420124183 >> 172.21.15.52:0/420124183 conn(0x55ca2b52f680 msgr2=0x55ca2b532470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 -- 172.21.15.52:0/420124183 shutdown_connections 2024-06-02T23:25:22.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.948+0000 7fd18460f740 1 -- 172.21.15.52:0/420124183 wait complete. 2024-06-02T23:25:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.949+0000 7fd18460f740 1 -- start start 2024-06-02T23:25:22.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.949+0000 7fd18460f740 1 Processor -- start 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534a00 0x55ca2b542550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b542a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b549ea0 0x55ca2b5b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b8fc0 con 0x55ca2b539b70 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b88c0 con 0x55ca2b549ea0 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd17c1b5ad0 con 0x55ca2b534a00 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd18460f740 10 client.0 _ll_register_callbacks cb 0x7fd16c009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534a00 0x55ca2b542550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b542a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17a7fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b542a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45726/0 (socket says 172.21.15.52:45726) 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17a7fc640 1 -- 172.21.15.52:0/1748454110 learned_addr learned my addr 172.21.15.52:0/1748454110 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:22.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17b7fe640 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b549ea0 0x55ca2b5b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17affd640 1 -- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b549ea0 msgr2=0x55ca2b5b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17affd640 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b549ea0 0x55ca2b5b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17affd640 1 -- 172.21.15.52:0/1748454110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 msgr2=0x55ca2b542a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17affd640 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b542a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:22.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.950+0000 7fd17affd640 1 -- 172.21.15.52:0/1748454110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca2b543130 con 0x55ca2b534a00 2024-06-02T23:25:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd17affd640 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534a00 0x55ca2b542550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd16c003e00 tx=0x7fd16c003e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=11a08024b9f84d6e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd15affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:25:22.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd15affd640 1 -- 172.21.15.52:0/1748454110 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd16c027070 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd15affd640 1 -- 172.21.15.52:0/1748454110 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd16c023460 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd174023070 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd15affd640 1 -- 172.21.15.52:0/1748454110 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fd16c009ea0 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd18460f740 10 client.28273 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.951+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55ca2b5b4260 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd15affd640 1 -- 172.21.15.52:0/1748454110 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd174023070 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd15affd640 1 -- 172.21.15.52:0/1748454110 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fd16c023b10 con 0x55ca2b534a00 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd18460f740 10 client.28273 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd18460f740 10 client.28273 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd18460f740 20 client.28273 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd18460f740 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b5b4be0 0x55ca2b5b7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd18460f740 4 client.28273 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ca2b53d720 con 0x55ca2b5b4be0 2024-06-02T23:25:22.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b5b4be0 0x55ca2b5b7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd15affd640 10 client.28273 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:22.954 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:22.952+0000 7fd17a7fc640 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b5b4be0 0x55ca2b5b7000 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40eb15fc10359808 server_cookie=745241a83761f3c5 in_seq=0 out_seq=0 2024-06-02T23:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd15affd640 1 -- 172.21.15.52:0/1748454110 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ca2b53d720 con 0x55ca2b5b4be0 2024-06-02T23:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd15affd640 10 client.28273 handle_command_reply: tid=0 2024-06-02T23:25:23.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 client.28273 shutdown 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534a00 msgr2=0x55ca2b542550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534a00 0x55ca2b542550 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd16c003e00 tx=0x7fd16c003e30 comp rx=0 tx=0).stop 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 shutdown_connections 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ca2b5b4be0 0x55ca2b5b7000 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd15affd640 0 client.28273 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca2b549ea0 0x55ca2b5b2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ca2b539b70 0x55ca2b542a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.000+0000 7fd18460f740 1 --2- 172.21.15.52:0/1748454110 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ca2b534a00 0x55ca2b542550 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 >> 172.21.15.52:0/1748454110 conn(0x55ca2b52f680 msgr2=0x55ca2b532d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:23.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 shutdown_connections 2024-06-02T23:25:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18460f740 1 -- 172.21.15.52:0/1748454110 wait complete. 2024-06-02T23:25:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18460f740 20 client.28273 trim_cache size 0 max 16384 2024-06-02T23:25:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:25:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:25:23.002 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1640a2cb0 msgr2=0x7fd1640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18182c640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1640a2cb0 0x7fd1640a5170 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd174005c40 tx=0x7fd174009500 comp rx=0 tx=0).stop 2024-06-02T23:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 msgr2=0x7fd17c1a6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.001+0000 7fd18182c640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 0x7fd17c1a6ee0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1700314e0 tx=0x7fd170004270 comp rx=0 tx=0).stop 2024-06-02T23:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 shutdown_connections 2024-06-02T23:25:23.003 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd1640a2cb0 0x7fd1640a5170 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd17c109300 0x7fd17c1a6ee0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd17c110220 0x7fd17c1adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 --2- 172.21.15.52:0/1480830469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd17c1089a0 0x7fd17c1a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 >> 172.21.15.52:0/1480830469 conn(0x7fd17c06ebd0 msgr2=0x7fd17c114580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 shutdown_connections 2024-06-02T23:25:23.004 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:23.002+0000 7fd18182c640 1 -- 172.21.15.52:0/1480830469 wait complete. 2024-06-02T23:25:23.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4613 13.43 MB/sec warmup 31 sec latency 95.053 ms 2024-06-02T23:25:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:23 smithi123 ceph-mon[35977]: pgmap v836: 97 pgs: 97 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-06-02T23:25:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:23 smithi174 ceph-mon[35994]: pgmap v836: 97 pgs: 97 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-06-02T23:25:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:23 smithi052 ceph-mon[32340]: pgmap v836: 97 pgs: 97 active+clean; 315 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.9 MiB/s wr, 238 op/s 2024-06-02T23:25:24.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4795 13.32 MB/sec warmup 32 sec latency 68.339 ms 2024-06-02T23:25:25.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 4948 13.04 MB/sec warmup 33 sec latency 116.695 ms 2024-06-02T23:25:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:25 smithi123 ceph-mon[35977]: pgmap v837: 97 pgs: 97 active+clean; 309 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 4.8 MiB/s wr, 214 op/s 2024-06-02T23:25:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:25 smithi174 ceph-mon[35994]: pgmap v837: 97 pgs: 97 active+clean; 309 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 4.8 MiB/s wr, 214 op/s 2024-06-02T23:25:25.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:25 smithi052 ceph-mon[32340]: pgmap v837: 97 pgs: 97 active+clean; 309 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 982 KiB/s rd, 4.8 MiB/s wr, 214 op/s 2024-06-02T23:25:26.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 5174 13.38 MB/sec warmup 34 sec latency 57.291 ms 2024-06-02T23:25:27.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 5349 13.19 MB/sec warmup 35 sec latency 101.439 ms 2024-06-02T23:25:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:27 smithi123 ceph-mon[35977]: pgmap v838: 97 pgs: 97 active+clean; 301 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 270 op/s 2024-06-02T23:25:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:27 smithi174 ceph-mon[35994]: pgmap v838: 97 pgs: 97 active+clean; 301 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 270 op/s 2024-06-02T23:25:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:27 smithi052 ceph-mon[32340]: pgmap v838: 97 pgs: 97 active+clean; 301 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 270 op/s 2024-06-02T23:25:27.950 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-06-02T23:25:27.951 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:25:28.307 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:28.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 5539 12.93 MB/sec warmup 36 sec latency 58.008 ms 2024-06-02T23:25:28.932 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:25:29.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 5663 12.62 MB/sec warmup 37 sec latency 99.550 ms 2024-06-02T23:25:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:29 smithi123 ceph-mon[35977]: pgmap v839: 97 pgs: 97 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-06-02T23:25:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:29 smithi174 ceph-mon[35994]: pgmap v839: 97 pgs: 97 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-06-02T23:25:29.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:29 smithi052 ceph-mon[32340]: pgmap v839: 97 pgs: 97 active+clean; 295 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 972 KiB/s rd, 3.9 MiB/s wr, 260 op/s 2024-06-02T23:25:30.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 5824 12.44 MB/sec warmup 38 sec latency 136.743 ms 2024-06-02T23:25:31.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6028 12.40 MB/sec warmup 39 sec latency 83.073 ms 2024-06-02T23:25:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:31 smithi123 ceph-mon[35977]: pgmap v840: 97 pgs: 97 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-06-02T23:25:31.766 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:31 smithi052 ceph-mon[32340]: pgmap v840: 97 pgs: 97 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-06-02T23:25:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:31 smithi174 ceph-mon[35994]: pgmap v840: 97 pgs: 97 active+clean; 297 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 297 op/s 2024-06-02T23:25:32.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6164 12.23 MB/sec warmup 40 sec latency 134.132 ms 2024-06-02T23:25:33.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6293 12.09 MB/sec warmup 41 sec latency 142.948 ms 2024-06-02T23:25:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:33 smithi123 ceph-mon[35977]: pgmap v841: 97 pgs: 97 active+clean; 301 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 3.2 MiB/s wr, 298 op/s 2024-06-02T23:25:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:33 smithi174 ceph-mon[35994]: pgmap v841: 97 pgs: 97 active+clean; 301 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 3.2 MiB/s wr, 298 op/s 2024-06-02T23:25:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:33 smithi052 ceph-mon[32340]: pgmap v841: 97 pgs: 97 active+clean; 301 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 3.2 MiB/s wr, 298 op/s 2024-06-02T23:25:34.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6497 12.35 MB/sec warmup 42 sec latency 91.570 ms 2024-06-02T23:25:34.705 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:35.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6688 12.40 MB/sec warmup 43 sec latency 137.510 ms 2024-06-02T23:25:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:35 smithi123 ceph-mon[35977]: pgmap v842: 97 pgs: 97 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-06-02T23:25:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:35 smithi174 ceph-mon[35994]: pgmap v842: 97 pgs: 97 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-06-02T23:25:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:35 smithi052 ceph-mon[32340]: pgmap v842: 97 pgs: 97 active+clean; 302 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 801 KiB/s rd, 2.8 MiB/s wr, 267 op/s 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 -- 172.21.15.52:0/3562400443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab64101b30 msgr2=0x7fab64101fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 --2- 172.21.15.52:0/3562400443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab64101b30 0x7fab64101fb0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fab58009900 tx=0x7fab5802a3f0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 -- 172.21.15.52:0/3562400443 shutdown_connections 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 --2- 172.21.15.52:0/3562400443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab641024f0 0x7fab64108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 --2- 172.21.15.52:0/3562400443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab64101b30 0x7fab64101fb0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 --2- 172.21.15.52:0/3562400443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 0x7fab64100d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 -- 172.21.15.52:0/3562400443 >> 172.21.15.52:0/3562400443 conn(0x7fab640fbeb0 msgr2=0x7fab640fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:36.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 -- 172.21.15.52:0/3562400443 shutdown_connections 2024-06-02T23:25:36.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.109+0000 7fab695c8640 1 -- 172.21.15.52:0/3562400443 wait complete. 2024-06-02T23:25:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.110+0000 7fab695c8640 1 Processor -- start 2024-06-02T23:25:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.110+0000 7fab695c8640 1 -- start start 2024-06-02T23:25:36.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.110+0000 7fab695c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 0x7fab6410cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.110+0000 7fab695c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64101b30 0x7fab6410b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.110+0000 7fab695c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 0x7fab6410b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab627fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64101b30 0x7fab6410b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab627fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64101b30 0x7fab6410b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45210/0 (socket says 172.21.15.52:45210) 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab637fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 0x7fab6410b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab637fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 0x7fab6410b940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59182/0 (socket says 172.21.15.52:59182) 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 0x7fab6410cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab695c8640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab6410d5d0 con 0x7fab641024f0 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab695c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab6410d450 con 0x7fab64101b30 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab695c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab6410d750 con 0x7fab64100940 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab627fc640 1 -- 172.21.15.52:0/2023508539 learned_addr learned my addr 172.21.15.52:0/2023508539 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 -- 172.21.15.52:0/2023508539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64101b30 msgr2=0x7fab6410b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64101b30 0x7fab6410b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 -- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 msgr2=0x7fab6410b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 0x7fab6410b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 -- 172.21.15.52:0/2023508539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6410c1d0 con 0x7fab64100940 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab637fe640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 0x7fab6410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.111+0000 7fab62ffd640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 0x7fab6410cdc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab5400e9c0 tx=0x7fab5400edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.112+0000 7fab4bfff640 1 -- 172.21.15.52:0/2023508539 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5400ccc0 con 0x7fab64100940 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.112+0000 7fab4bfff640 1 -- 172.21.15.52:0/2023508539 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab540108e0 con 0x7fab64100940 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.112+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab641a1cc0 con 0x7fab64100940 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.112+0000 7fab4bfff640 1 -- 172.21.15.52:0/2023508539 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540187a0 con 0x7fab64100940 2024-06-02T23:25:36.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.112+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab641a2190 con 0x7fab64100940 2024-06-02T23:25:36.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.115+0000 7fab4bfff640 1 -- 172.21.15.52:0/2023508539 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fab5401f020 con 0x7fab64100940 2024-06-02T23:25:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.115+0000 7fab6c3b1740 1 Processor -- start 2024-06-02T23:25:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.115+0000 7fab6c3b1740 1 -- start start 2024-06-02T23:25:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59eb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f1c594e4f0 con 0x55f1c59eba00 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1c58ec870 con 0x55f1c59eb010 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1c594a4b0 con 0x55f1c59f0b70 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab637fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59eb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab637fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59eb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45214/0 (socket says 172.21.15.52:45214) 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab637fe640 1 -- 172.21.15.52:0/1936131474 learned_addr learned my addr 172.21.15.52:0/1936131474 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab627fc640 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab4bfff640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab4c0a2c30 0x7fab4c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab62ffd640 1 -- 172.21.15.52:0/1936131474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 msgr2=0x55f1c59f3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab62ffd640 1 -- 172.21.15.52:0/1936131474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 msgr2=0x55f1c59eb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59eb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab62ffd640 1 -- 172.21.15.52:0/1936131474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c59089b0 con 0x55f1c59eba00 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab637fe640 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab637fe640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab4c0a2c30 0x7fab4c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.116+0000 7fab4bfff640 1 -- 172.21.15.52:0/2023508539 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fab540cefa0 con 0x7fab64100940 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.117+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f0500 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fab540ccb20 tx=0x7fab54022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5debf8036620e273 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.117+0000 7fab497fa640 1 -- 172.21.15.52:0/1936131474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54010e20 con 0x55f1c59eba00 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.117+0000 7fab637fe640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab4c0a2c30 0x7fab4c0a50f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55f1c59ec260 tx=0x7fab5c004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.117+0000 7fab497fa640 1 -- 172.21.15.52:0/1936131474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab54008830 con 0x55f1c59eba00 2024-06-02T23:25:36.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.117+0000 7fab497fa640 1 -- 172.21.15.52:0/1936131474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540a44a0 con 0x55f1c59eba00 2024-06-02T23:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1936131474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 msgr2=0x55f1c59f0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f0500 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fab540ccb20 tx=0x7fab54022000 comp rx=0 tx=0).stop 2024-06-02T23:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1936131474 shutdown_connections 2024-06-02T23:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f0500 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1936131474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59eb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1936131474 >> 172.21.15.52:0/1936131474 conn(0x55f1c59e6680 msgr2=0x55f1c59e6a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1936131474 shutdown_connections 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1936131474 wait complete. 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.118+0000 7fab6c3b1740 1 -- start start 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 Processor -- start 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59fc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f1c59f3ac0 con 0x55f1c59eba00 2024-06-02T23:25:36.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1c58d7ae0 con 0x55f1c59eb010 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1c59f3c40 con 0x55f1c59f0b70 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45216/0 (socket says 172.21.15.52:45216) 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 -- 172.21.15.52:0/1958625470 learned_addr learned my addr 172.21.15.52:0/1958625470 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab6c3b1740 10 client.0 _ll_register_callbacks cb 0x55f1c59109b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab627fc640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab637fe640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59fc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 -- 172.21.15.52:0/1958625470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 msgr2=0x55f1c59fc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59fc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 -- 172.21.15.52:0/1958625470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 msgr2=0x55f1c59f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f5730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab62ffd640 1 -- 172.21.15.52:0/1958625470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c5662210 con 0x55f1c59eb010 2024-06-02T23:25:36.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.119+0000 7fab637fe640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab62ffd640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f51f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fab5409b530 tx=0x7fab54009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10401924ff658604 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab46ff5640 1 -- 172.21.15.52:0/1958625470 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54010e20 con 0x55f1c59eb010 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab46ff5640 1 -- 172.21.15.52:0/1958625470 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fab54099b30 con 0x55f1c59eb010 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab54008830 con 0x55f1c59eb010 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab46ff5640 1 -- 172.21.15.52:0/1958625470 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fab5409b830 con 0x55f1c59eb010 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab6c3b1740 10 client.28604 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f1c572e620 con 0x55f1c59eb010 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab46ff5640 1 -- 172.21.15.52:0/1958625470 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab5409fbe0 con 0x55f1c59eb010 2024-06-02T23:25:36.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab46ff5640 1 -- 172.21.15.52:0/1958625470 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fab5409ba90 con 0x55f1c59eb010 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab6c3b1740 10 client.28604 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab6c3b1740 10 client.28604 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.120+0000 7fab6c3b1740 20 client.28604 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.121+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a00c60 0x55f1c5a03050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.121+0000 7fab6c3b1740 4 client.28604 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.121+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fab54008830 con 0x55f1c5a00c60 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.121+0000 7fab627fc640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a00c60 0x55f1c5a03050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.121+0000 7fab627fc640 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a00c60 0x55f1c5a03050 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=554ed5a6001f1216 server_cookie=6fa29e107be86d9 in_seq=0 out_seq=0 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.121+0000 7fab46ff5640 10 client.28604 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab46ff5640 1 -- 172.21.15.52:0/1958625470 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fab54008830 con 0x55f1c5a00c60 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab46ff5640 10 client.28604 handle_command_reply: tid=0 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 client.28604 shutdown 2024-06-02T23:25:36.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 msgr2=0x55f1c59f51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f51f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fab5409b530 tx=0x7fab54009f70 comp rx=0 tx=0).stop 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 shutdown_connections 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a00c60 0x55f1c5a03050 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59fc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/1958625470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f51f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 >> 172.21.15.52:0/1958625470 conn(0x55f1c59e6680 msgr2=0x55f1c59e57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab46ff5640 0 client.28604 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 shutdown_connections 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 1 -- 172.21.15.52:0/1958625470 wait complete. 2024-06-02T23:25:36.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.123+0000 7fab6c3b1740 20 client.28604 trim_cache size 0 max 16384 2024-06-02T23:25:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 Processor -- start 2024-06-02T23:25:36.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 -- start start 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59fc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab641aae60 con 0x55f1c59eba00 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab641acac0 con 0x55f1c59eb010 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab641ab530 con 0x55f1c59f0b70 2024-06-02T23:25:36.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab627fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59fc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab627fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59fc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59200/0 (socket says 172.21.15.52:59200) 2024-06-02T23:25:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab627fc640 1 -- 172.21.15.52:0/3785359054 learned_addr learned my addr 172.21.15.52:0/3785359054 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab62ffd640 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.149+0000 7fab637fe640 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab62ffd640 1 -- 172.21.15.52:0/3785359054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 msgr2=0x55f1c59f5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab62ffd640 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab62ffd640 1 -- 172.21.15.52:0/3785359054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 msgr2=0x55f1c59fc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab62ffd640 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59fc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab62ffd640 1 -- 172.21.15.52:0/3785359054 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1c59e64b0 con 0x55f1c59eb010 2024-06-02T23:25:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab627fc640 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:36.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab62ffd640 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f4e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fab540a3f70 tx=0x7fab5400efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbed788e575892a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab48ff9640 1 -- 172.21.15.52:0/3785359054 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54010e20 con 0x55f1c59eb010 2024-06-02T23:25:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab48ff9640 1 -- 172.21.15.52:0/3785359054 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab5409fbe0 con 0x55f1c59eb010 2024-06-02T23:25:36.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.150+0000 7fab48ff9640 1 -- 172.21.15.52:0/3785359054 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab54010a90 con 0x55f1c59eb010 2024-06-02T23:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 -- 172.21.15.52:0/3785359054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 msgr2=0x55f1c59f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f4e00 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fab540a3f70 tx=0x7fab5400efb0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 -- 172.21.15.52:0/3785359054 shutdown_connections 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59f0b70 0x55f1c59f5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59eba00 0x55f1c59fc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/3785359054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eb010 0x55f1c59f4e00 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 -- 172.21.15.52:0/3785359054 >> 172.21.15.52:0/3785359054 conn(0x55f1c59e6680 msgr2=0x55f1c59e16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 -- 172.21.15.52:0/3785359054 shutdown_connections 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 -- 172.21.15.52:0/3785359054 wait complete. 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.151+0000 7fab6c3b1740 1 -- start start 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 Processor -- start 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 0x55f1c59f9330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eba00 0x55f1c59f9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59f0b70 0x55f1c5a697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fab641ac330 con 0x55f1c59f0b70 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab641abc30 con 0x55f1c59eba00 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fab641a8e40 con 0x55f1c59eb010 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab6c3b1740 10 client.0 _ll_register_callbacks cb 0x7fab5c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 0x55f1c59f9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 0x55f1c59f9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33236/0 (socket says 172.21.15.52:33236) 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 -- 172.21.15.52:0/2240000831 learned_addr learned my addr 172.21.15.52:0/2240000831 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab627fc640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eba00 0x55f1c59f9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab637fe640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59f0b70 0x55f1c5a697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 -- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eba00 msgr2=0x55f1c59f9870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eba00 0x55f1c59f9870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 -- 172.21.15.52:0/2240000831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59f0b70 msgr2=0x55f1c5a697e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59f0b70 0x55f1c5a697e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.159 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.152+0000 7fab62ffd640 1 -- 172.21.15.52:0/2240000831 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1c59fc940 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab62ffd640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 0x55f1c59f9330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab5409b9f0 tx=0x7fab5409bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4cc7f62984c5b9da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab46ff5640 1 -- 172.21.15.52:0/2240000831 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab540b8070 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab46ff5640 1 -- 172.21.15.52:0/2240000831 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fab540860b0 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab5409fbe0 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab46ff5640 1 -- 172.21.15.52:0/2240000831 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fab540d2cd0 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab6c3b1740 10 client.28291 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55f1c59fc940 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab46ff5640 1 -- 172.21.15.52:0/2240000831 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fab540a44a0 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.153+0000 7fab46ff5640 1 -- 172.21.15.52:0/2240000831 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fab5409ac80 con 0x55f1c59eb010 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab6c3b1740 10 client.28291 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab6c3b1740 10 client.28291 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab6c3b1740 20 client.28291 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:36.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a6b1f0 0x55f1c5a6d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab6c3b1740 4 client.28291 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:25:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fab5409fbe0 con 0x55f1c5a6b1f0 2024-06-02T23:25:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab627fc640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a6b1f0 0x55f1c5a6d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab46ff5640 10 client.28291 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:36.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.154+0000 7fab627fc640 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a6b1f0 0x55f1c5a6d610 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c546cda841b7c568 server_cookie=83a9249e1ca3c244 in_seq=0 out_seq=0 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab46ff5640 1 -- 172.21.15.52:0/2240000831 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fab5409fbe0 con 0x55f1c5a6b1f0 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab46ff5640 10 client.28291 handle_command_reply: tid=0 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 client.28291 shutdown 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 msgr2=0x55f1c59f9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 0x55f1c59f9330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab5409b9f0 tx=0x7fab5409bf30 comp rx=0 tx=0).stop 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 shutdown_connections 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f1c5a6b1f0 0x55f1c5a6d610 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1c59f0b70 0x55f1c5a697e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab46ff5640 0 client.28291 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1c59eba00 0x55f1c59f9870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 --2- 172.21.15.52:0/2240000831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1c59eb010 0x55f1c59f9330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 >> 172.21.15.52:0/2240000831 conn(0x55f1c59e6680 msgr2=0x55f1c59f4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 shutdown_connections 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 1 -- 172.21.15.52:0/2240000831 wait complete. 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.188+0000 7fab6c3b1740 20 client.28291 trim_cache size 0 max 16384 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:25:36.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:25:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:25:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab4c0a2c30 msgr2=0x7fab4c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab4c0a2c30 0x7fab4c0a50f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x55f1c59ec260 tx=0x7fab5c004a40 comp rx=0 tx=0).stop 2024-06-02T23:25:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 msgr2=0x7fab6410cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:36.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 0x7fab6410cdc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fab5400e9c0 tx=0x7fab5400edc0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 shutdown_connections 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fab4c0a2c30 0x7fab4c0a50f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fab641024f0 0x7fab6410b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab64101b30 0x7fab6410b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 --2- 172.21.15.52:0/2023508539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fab64100940 0x7fab6410cdc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 >> 172.21.15.52:0/2023508539 conn(0x7fab640fbeb0 msgr2=0x7fab64104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.189+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 shutdown_connections 2024-06-02T23:25:36.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:36.190+0000 7fab695c8640 1 -- 172.21.15.52:0/2023508539 wait complete. 2024-06-02T23:25:36.447 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6870 12.40 MB/sec warmup 44 sec latency 194.478 ms 2024-06-02T23:25:37.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 6998 12.28 MB/sec warmup 45 sec latency 221.064 ms 2024-06-02T23:25:37.511 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:37 smithi052 ceph-mon[32340]: pgmap v843: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 325 op/s 2024-06-02T23:25:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:37 smithi123 ceph-mon[35977]: pgmap v843: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 325 op/s 2024-06-02T23:25:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:37 smithi174 ceph-mon[35994]: pgmap v843: 97 pgs: 97 active+clean; 313 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 325 op/s 2024-06-02T23:25:38.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7133 12.23 MB/sec warmup 46 sec latency 81.503 ms 2024-06-02T23:25:38.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 -- 172.21.15.52:0/3200770861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220102ed0 msgr2=0x7fb22010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:38.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 --2- 172.21.15.52:0/3200770861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220102ed0 0x7fb22010f360 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fb21800b0a0 tx=0x7fb21800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:25:38.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 -- 172.21.15.52:0/3200770861 shutdown_connections 2024-06-02T23:25:38.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 --2- 172.21.15.52:0/3200770861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220102ed0 0x7fb22010f360 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:38.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 --2- 172.21.15.52:0/3200770861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220108520 0x7fb220108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:38.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 --2- 172.21.15.52:0/3200770861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 0x7fb220102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:38.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.958+0000 7fb224db5700 1 -- 172.21.15.52:0/3200770861 >> 172.21.15.52:0/3200770861 conn(0x7fb220079560 msgr2=0x7fb220079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:38.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.959+0000 7fb224db5700 1 -- 172.21.15.52:0/3200770861 shutdown_connections 2024-06-02T23:25:38.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.959+0000 7fb224db5700 1 -- 172.21.15.52:0/3200770861 wait complete. 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 Processor -- start 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 -- start start 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 0x7fb22019e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 0x7fb22019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 0x7fb220198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 0x7fb22019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 0x7fb22019ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45258/0 (socket says 172.21.15.52:45258) 2024-06-02T23:25:38.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 0x7fb220198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 0x7fb22019e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21e59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 0x7fb22019e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33246/0 (socket says 172.21.15.52:33246) 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 0x7fb220198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59220/0 (socket says 172.21.15.52:59220) 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb220112650 con 0x7fb220108520 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb21dd9b700 1 -- 172.21.15.52:0/830475676 learned_addr learned my addr 172.21.15.52:0/830475676 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201124d0 con 0x7fb220102ed0 2024-06-02T23:25:38.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.960+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb2201127d0 con 0x7fb220102510 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21dd9b700 1 -- 172.21.15.52:0/830475676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 msgr2=0x7fb22019e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21dd9b700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 0x7fb22019e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21dd9b700 1 -- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 msgr2=0x7fb220198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21dd9b700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 0x7fb220198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21dd9b700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb220198ff0 con 0x7fb220102ed0 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21ed9d700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 0x7fb220198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb21dd9b700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 0x7fb22019ec10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb21400e9a0 tx=0x7fb21400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb21400ca10 con 0x7fb220102ed0 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb21400cbb0 con 0x7fb220102ed0 2024-06-02T23:25:38.963 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2201992e0 con 0x7fb220102ed0 2024-06-02T23:25:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.961+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb214018680 con 0x7fb220102ed0 2024-06-02T23:25:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.962+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2201a55a0 con 0x7fb220102ed0 2024-06-02T23:25:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.963+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb220069180 con 0x7fb220102ed0 2024-06-02T23:25:38.964 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.964+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb214010430 con 0x7fb220102ed0 2024-06-02T23:25:38.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.965+0000 7fb20f7fe700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2080a2ca0 0x7fb2080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:38.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.965+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb2140cf140 con 0x7fb220102ed0 2024-06-02T23:25:38.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.965+0000 7fb21e59c700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2080a2ca0 0x7fb2080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:38.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.966+0000 7fb21e59c700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2080a2ca0 0x7fb2080a5160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb2201999d0 tx=0x7fb210009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:38.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:38.972+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb21409c0d0 con 0x7fb220102ed0 2024-06-02T23:25:39.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.403+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb22002dca0 con 0x7fb220102ed0 2024-06-02T23:25:39.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.404+0000 7fb20f7fe700 1 -- 172.21.15.52:0/830475676 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7fb21409b9d0 con 0x7fb220102ed0 2024-06-02T23:25:39.405 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:25:39.405 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.407+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2080a2ca0 msgr2=0x7fb2080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.407+0000 7fb224db5700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2080a2ca0 0x7fb2080a5160 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb2201999d0 tx=0x7fb210009260 comp rx=0 tx=0).stop 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.407+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 msgr2=0x7fb22019ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.407+0000 7fb224db5700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 0x7fb22019ec10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb21400e9a0 tx=0x7fb21400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.407+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 shutdown_connections 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2080a2ca0 0x7fb2080a5160 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb220108520 0x7fb220198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb220102510 0x7fb22019e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 --2- 172.21.15.52:0/830475676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb220102ed0 0x7fb22019ec10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 >> 172.21.15.52:0/830475676 conn(0x7fb220079560 msgr2=0x7fb2201010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 shutdown_connections 2024-06-02T23:25:39.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.408+0000 7fb224db5700 1 -- 172.21.15.52:0/830475676 wait complete. 2024-06-02T23:25:39.409 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:25:39.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7302 12.28 MB/sec warmup 47 sec latency 82.940 ms 2024-06-02T23:25:39.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 -- 172.21.15.52:0/4221428837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810de50 msgr2=0x7fbea810e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 --2- 172.21.15.52:0/4221428837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810de50 0x7fbea810e230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fbe98009990 tx=0x7fbe98009ca0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 -- 172.21.15.52:0/4221428837 shutdown_connections 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 --2- 172.21.15.52:0/4221428837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea8109e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 --2- 172.21.15.52:0/4221428837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810de50 0x7fbea810e230 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 --2- 172.21.15.52:0/4221428837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea810e770 0x7fbea8109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 -- 172.21.15.52:0/4221428837 >> 172.21.15.52:0/4221428837 conn(0x7fbea806d9e0 msgr2=0x7fbea806ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 -- 172.21.15.52:0/4221428837 shutdown_connections 2024-06-02T23:25:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.538+0000 7fbead0f0700 1 -- 172.21.15.52:0/4221428837 wait complete. 2024-06-02T23:25:39.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.539+0000 7fbead0f0700 1 Processor -- start 2024-06-02T23:25:39.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 -- start start 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea81a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea810de50 0x7fbea81a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810e770 0x7fbea81a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8114ce0 con 0x7fbea81099d0 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8114b60 con 0x7fbea810e770 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbead0f0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbea8114e60 con 0x7fbea810de50 2024-06-02T23:25:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbea6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea81a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbea6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea81a3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59238/0 (socket says 172.21.15.52:59238) 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2658527714 learned_addr learned my addr 172.21.15.52:0/2658527714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbea759e700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810e770 0x7fbea81a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.540+0000 7fbea659c700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea810de50 0x7fbea81a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2658527714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea810de50 msgr2=0x7fbea81a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea810de50 0x7fbea81a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2658527714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810e770 msgr2=0x7fbea81a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810e770 0x7fbea81a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2658527714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbea81a8590 con 0x7fbea81099d0 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea759e700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810e770 0x7fbea81a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea81a3580 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fbe98005910 tx=0x7fbe980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:39.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbe8ffff700 1 -- 172.21.15.52:0/2658527714 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98027070 con 0x7fbea81099d0 2024-06-02T23:25:39.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbe8ffff700 1 -- 172.21.15.52:0/2658527714 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe98003f10 con 0x7fbea81099d0 2024-06-02T23:25:39.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbe8ffff700 1 -- 172.21.15.52:0/2658527714 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe98022c40 con 0x7fbea81099d0 2024-06-02T23:25:39.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbea81a8820 con 0x7fbea81099d0 2024-06-02T23:25:39.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.541+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbea81b0100 con 0x7fbea81099d0 2024-06-02T23:25:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.545+0000 7fbe8ffff700 1 -- 172.21.15.52:0/2658527714 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe9802a070 con 0x7fbea81099d0 2024-06-02T23:25:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.546+0000 7fbe8ffff700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe900a2c20 0x7fbe900a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.546+0000 7fbe8ffff700 1 -- 172.21.15.52:0/2658527714 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe980cfe70 con 0x7fbea81099d0 2024-06-02T23:25:39.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.546+0000 7fbea659c700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe900a2c20 0x7fbe900a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.546+0000 7fbeb7973b80 1 Processor -- start 2024-06-02T23:25:39.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.546+0000 7fbeb7973b80 1 -- start start 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4cf60 0x563ad0c4d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 0x563ad0c47260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbea659c700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe900a2c20 0x7fbe900a50e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbea806e1f0 tx=0x7fbe9c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbeb7973b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ad0b39f60 con 0x563ad0c477a0 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbeb7973b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ad0b98850 con 0x563ad0c4cf60 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbeb7973b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ad0b00460 con 0x563ad0c4d880 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 0x563ad0c47260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 0x563ad0c47260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33274/0 (socket says 172.21.15.52:33274) 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbea659c700 1 -- 172.21.15.52:0/633545502 learned_addr learned my addr 172.21.15.52:0/633545502 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbea759e700 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.547+0000 7fbea659c700 1 -- 172.21.15.52:0/633545502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4cf60 msgr2=0x563ad0c4d340 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.548+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4cf60 0x563ad0c4d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.548+0000 7fbea659c700 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4cf60 0x563ad0c4d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.548+0000 7fbea659c700 1 -- 172.21.15.52:0/633545502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 msgr2=0x563ad0c47bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.548+0000 7fbea659c700 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.548+0000 7fbea659c700 1 -- 172.21.15.52:0/633545502 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad0ae78d0 con 0x563ad0c4d880 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.548+0000 7fbea759e700 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.549+0000 7fbea659c700 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 0x563ad0c47260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00ea40 tx=0x7fbe9c011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaf33073dc1a1a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.549+0000 7fbe8d7fa700 1 -- 172.21.15.52:0/633545502 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c01b040 con 0x563ad0c4d880 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.549+0000 7fbe8d7fa700 1 -- 172.21.15.52:0/633545502 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe9c024070 con 0x563ad0c4d880 2024-06-02T23:25:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.549+0000 7fbe8d7fa700 1 -- 172.21.15.52:0/633545502 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c013690 con 0x563ad0c4d880 2024-06-02T23:25:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 -- 172.21.15.52:0/633545502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 msgr2=0x563ad0c47260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 0x563ad0c47260 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00ea40 tx=0x7fbe9c011000 comp rx=0 tx=0).stop 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 -- 172.21.15.52:0/633545502 shutdown_connections 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4cf60 0x563ad0c4d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/633545502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4d880 0x563ad0c47260 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 -- 172.21.15.52:0/633545502 >> 172.21.15.52:0/633545502 conn(0x563ad0b74080 msgr2=0x563ad0c42220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 -- 172.21.15.52:0/633545502 shutdown_connections 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.550+0000 7fbeb7973b80 1 -- 172.21.15.52:0/633545502 wait complete. 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.551+0000 7fbeb7973b80 1 -- start start 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.551+0000 7fbeb7973b80 1 Processor -- start 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.551+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c53670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.551+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c50970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c50eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c50970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c50970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33276/0 (socket says 172.21.15.52:33276) 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 -- 172.21.15.52:0/3452876359 learned_addr learned my addr 172.21.15.52:0/3452876359 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563ad0b38fc0 con 0x563ad0c477a0 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ad0b94810 con 0x563ad0c4d880 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563ad0c54260 con 0x563ad0c4cf60 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c53670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea759e700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c50eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbeb7973b80 10 client.0 _ll_register_callbacks cb 0x563ad0b2e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:39.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 -- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 msgr2=0x563ad0c50eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c50eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 -- 172.21.15.52:0/3452876359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 msgr2=0x563ad0c53670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c53670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbea659c700 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad0b5e750 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.552+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe9c024070 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbea659c700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c50970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c016bb0 tx=0x7fbe9c016be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d0d826787a2e2f0b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbe8aff5700 10 client.28306 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbe8aff5700 1 -- 172.21.15.52:0/3452876359 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c02f020 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbe8aff5700 1 -- 172.21.15.52:0/3452876359 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe9c028830 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbe8aff5700 1 -- 172.21.15.52:0/3452876359 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbe9c028a30 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbeb7973b80 10 client.28306 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=37}) -- 0x563ad0c66f60 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbe8aff5700 1 -- 172.21.15.52:0/3452876359 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fbe9c03c060 con 0x563ad0c4cf60 2024-06-02T23:25:39.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbeb7973b80 10 client.28306 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.553+0000 7fbeb7973b80 10 client.28306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbeb7973b80 20 client.28306 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0c69430 0x563ad0c6b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbeb7973b80 4 client.28306 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ad0ba5d50 con 0x563ad0c69430 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0c69430 0x563ad0c6b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0c69430 0x563ad0c6b820 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac63f03af90b40e server_cookie=61ead7f8fa6885a9 in_seq=0 out_seq=0 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.554+0000 7fbe8aff5700 10 client.28306 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.555+0000 7fbe8aff5700 1 -- 172.21.15.52:0/3452876359 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563ad0ba5d50 con 0x563ad0c69430 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.555+0000 7fbe8aff5700 10 client.28306 handle_command_reply: tid=0 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.555+0000 7fbeb7973b80 1 client.28306 shutdown 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.555+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 msgr2=0x563ad0c50970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.555+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c50970 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c016bb0 tx=0x7fbe9c016be0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 shutdown_connections 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0c69430 0x563ad0c6b820 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c50eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c53670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/3452876359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c50970 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 >> 172.21.15.52:0/3452876359 conn(0x563ad0b74080 msgr2=0x563ad0c41700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbe8aff5700 0 client.28306 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:39.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 shutdown_connections 2024-06-02T23:25:39.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 1 -- 172.21.15.52:0/3452876359 wait complete. 2024-06-02T23:25:39.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.556+0000 7fbeb7973b80 20 client.28306 trim_cache size 0 max 16384 2024-06-02T23:25:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.590+0000 7fbeb7973b80 1 Processor -- start 2024-06-02T23:25:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 -- start start 2024-06-02T23:25:39.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:39 smithi052 ceph-mon[32340]: pgmap v844: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.8 MiB/s wr, 229 op/s 2024-06-02T23:25:39.592 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:39 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/830475676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c4e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c528b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b2ed0 con 0x563ad0c477a0 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b4b30 con 0x563ad0c4d880 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbeb7973b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b35a0 con 0x563ad0c4cf60 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c4e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c4e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33290/0 (socket says 172.21.15.52:33290) 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.591+0000 7fbea659c700 1 -- 172.21.15.52:0/106901378 learned_addr learned my addr 172.21.15.52:0/106901378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea759e700 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea659c700 1 -- 172.21.15.52:0/106901378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 msgr2=0x563ad0c528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea659c700 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea659c700 1 -- 172.21.15.52:0/106901378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 msgr2=0x563ad0c47260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea659c700 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea659c700 1 -- 172.21.15.52:0/106901378 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ad0c4d680 con 0x563ad0c4cf60 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbea659c700 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c4e510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00ee00 tx=0x7fbe9c02c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74db06e15bd721cb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbe8cff9700 1 -- 172.21.15.52:0/106901378 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c041070 con 0x563ad0c4cf60 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbe8cff9700 1 -- 172.21.15.52:0/106901378 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe9c024070 con 0x563ad0c4cf60 2024-06-02T23:25:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.592+0000 7fbe8cff9700 1 -- 172.21.15.52:0/106901378 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c02f070 con 0x563ad0c4cf60 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 -- 172.21.15.52:0/106901378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 msgr2=0x563ad0c4e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c4e510 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00ee00 tx=0x7fbe9c02c000 comp rx=0 tx=0).stop 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 -- 172.21.15.52:0/106901378 shutdown_connections 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c4d880 0x563ad0c528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c477a0 0x563ad0c47260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/106901378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c4e510 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.593+0000 7fbeb7973b80 1 -- 172.21.15.52:0/106901378 >> 172.21.15.52:0/106901378 conn(0x563ad0b74080 msgr2=0x563ad0c521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.594+0000 7fbeb7973b80 1 -- 172.21.15.52:0/106901378 shutdown_connections 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.594+0000 7fbeb7973b80 1 -- 172.21.15.52:0/106901378 wait complete. 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.594+0000 7fbeb7973b80 1 -- start start 2024-06-02T23:25:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.594+0000 7fbeb7973b80 1 Processor -- start 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 0x563ad0c6e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c6ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbeb7973b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 0x563ad0c6bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c6ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 0x563ad0c6e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c6ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33292/0 (socket says 172.21.15.52:33292) 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 0x563ad0c6e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45300/0 (socket says 172.21.15.52:45300) 2024-06-02T23:25:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 0x563ad0c6bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 0x563ad0c6bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59292/0 (socket says 172.21.15.52:59292) 2024-06-02T23:25:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbea659c700 1 -- 172.21.15.52:0/2934182374 learned_addr learned my addr 172.21.15.52:0/2934182374 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b43a0 con 0x563ad0c4d880 2024-06-02T23:25:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.595+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbea81b3ca0 con 0x563ad0c477a0 2024-06-02T23:25:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbea811ce60 con 0x563ad0c4cf60 2024-06-02T23:25:39.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2934182374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 msgr2=0x563ad0c6ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c6ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbeb7973b80 10 client.0 _ll_register_callbacks cb 0x563ad0b2e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:39.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2934182374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 msgr2=0x563ad0c6bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:39.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 0x563ad0c6bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea6d9d700 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ad0c6c560 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe9c024070 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea759e700 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 0x563ad0c6bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbea6d9d700 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 0x563ad0c6e7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe980cd9f0 tx=0x7fbe98003700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67b8609954be38c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbe8aff5700 10 client.28631 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.596+0000 7fbe8aff5700 1 -- 172.21.15.52:0/2934182374 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe980cde60 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbe8aff5700 1 -- 172.21.15.52:0/2934182374 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe9809aa90 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbe8aff5700 1 -- 172.21.15.52:0/2934182374 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbe980a0b40 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 10 client.28631 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=37}) -- 0x563ad0cd38d0 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbe8aff5700 1 -- 172.21.15.52:0/2934182374 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fbe98061ce0 con 0x563ad0c477a0 2024-06-02T23:25:39.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 10 client.28631 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 10 client.28631 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 20 client.28631 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0cd3eb0 0x563ad0cd62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 4 client.28631 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563ad0ba02d0 con 0x563ad0cd3eb0 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.597+0000 7fbea659c700 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0cd3eb0 0x563ad0cd62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.598+0000 7fbea659c700 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0cd3eb0 0x563ad0cd62d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d71bd0236b3b3e server_cookie=f2f4c13579a20550 in_seq=0 out_seq=0 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.598+0000 7fbe8aff5700 10 client.28631 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.598+0000 7fbe8aff5700 1 -- 172.21.15.52:0/2934182374 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563ad0ba02d0 con 0x563ad0cd3eb0 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.598+0000 7fbe8aff5700 10 client.28631 handle_command_reply: tid=0 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.598+0000 7fbeb7973b80 1 client.28631 shutdown 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 msgr2=0x563ad0c6e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 0x563ad0c6e7d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbe980cd9f0 tx=0x7fbe98003700 comp rx=0 tx=0).stop 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 shutdown_connections 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563ad0cd3eb0 0x563ad0cd62d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563ad0c4d880 0x563ad0c6bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ad0c477a0 0x563ad0c6e7d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 --2- 172.21.15.52:0/2934182374 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563ad0c4cf60 0x563ad0c6ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 >> 172.21.15.52:0/2934182374 conn(0x563ad0b74080 msgr2=0x563ad0c4c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.599+0000 7fbe8aff5700 0 client.28631 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.600+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 shutdown_connections 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.600+0000 7fbeb7973b80 1 -- 172.21.15.52:0/2934182374 wait complete. 2024-06-02T23:25:39.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.600+0000 7fbeb7973b80 20 client.28631 trim_cache size 0 max 16384 2024-06-02T23:25:39.604 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:25:39.604 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:25:39.604 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:25:39.604 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:25:39.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe900a2c20 msgr2=0x7fbe900a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe900a2c20 0x7fbe900a50e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbea806e1f0 tx=0x7fbe9c009190 comp rx=0 tx=0).stop 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 msgr2=0x7fbea81a3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea81a3580 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fbe98005910 tx=0x7fbe980047b0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 shutdown_connections 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe900a2c20 0x7fbe900a50e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbea810e770 0x7fbea81a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbea81099d0 0x7fbea81a3580 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 --2- 172.21.15.52:0/2658527714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbea810de50 0x7fbea81a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:39.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.601+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 >> 172.21.15.52:0/2658527714 conn(0x7fbea806d9e0 msgr2=0x7fbea810cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:39.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.602+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 shutdown_connections 2024-06-02T23:25:39.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:39.602+0000 7fbead0f0700 1 -- 172.21.15.52:0/2658527714 wait complete. 2024-06-02T23:25:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:39 smithi123 ceph-mon[35977]: pgmap v844: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.8 MiB/s wr, 229 op/s 2024-06-02T23:25:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:39 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/830475676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:39 smithi174 ceph-mon[35994]: pgmap v844: 97 pgs: 97 active+clean; 319 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 672 KiB/s rd, 3.8 MiB/s wr, 229 op/s 2024-06-02T23:25:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:39 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/830475676' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:40.065 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 130 --await-for 45.0' 2024-06-02T23:25:40.065 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 130 --await-for 45.0 2024-06-02T23:25:40.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7441 12.27 MB/sec warmup 48 sec latency 115.664 ms 2024-06-02T23:25:40.748 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:25:40.748 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:25:40.891 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:41.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7558 12.08 MB/sec warmup 49 sec latency 75.013 ms 2024-06-02T23:25:41.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:41 smithi052 ceph-mon[32340]: pgmap v845: 97 pgs: 97 active+clean; 322 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-06-02T23:25:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:41 smithi123 ceph-mon[35977]: pgmap v845: 97 pgs: 97 active+clean; 322 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-06-02T23:25:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:41 smithi174 ceph-mon[35994]: pgmap v845: 97 pgs: 97 active+clean; 322 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 788 KiB/s rd, 4.5 MiB/s wr, 242 op/s 2024-06-02T23:25:42.116 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:42.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7698 11.94 MB/sec warmup 50 sec latency 68.653 ms 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 -- 172.21.15.52:0/3034558879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107850 msgr2=0x7fbe5010f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 --2- 172.21.15.52:0/3034558879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107850 0x7fbe5010f3d0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b0d0 tx=0x7fbe4800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 -- 172.21.15.52:0/3034558879 shutdown_connections 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 --2- 172.21.15.52:0/3034558879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe50107850 0x7fbe5010f3d0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 --2- 172.21.15.52:0/3034558879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006a3d0 0x7fbe5006a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 --2- 172.21.15.52:0/3034558879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe5006ad80 0x7fbe50107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 -- 172.21.15.52:0/3034558879 >> 172.21.15.52:0/3034558879 conn(0x7fbe50078330 msgr2=0x7fbe50078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:43.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 -- 172.21.15.52:0/3034558879 shutdown_connections 2024-06-02T23:25:43.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.370+0000 7fbe5827c700 1 -- 172.21.15.52:0/3034558879 wait complete. 2024-06-02T23:25:43.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 Processor -- start 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 -- start start 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 0x7fbe50198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006ad80 0x7fbe50198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe50107850 0x7fbe5019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe56018700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 0x7fbe50198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50112620 con 0x7fbe5006a3d0 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501124a0 con 0x7fbe5006ad80 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe5827c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501127a0 con 0x7fbe50107850 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe55817700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006ad80 0x7fbe50198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe56018700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 0x7fbe50198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59302/0 (socket says 172.21.15.52:59302) 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.371+0000 7fbe56018700 1 -- 172.21.15.52:0/4242103141 learned_addr learned my addr 172.21.15.52:0/4242103141 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56819700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe50107850 0x7fbe5019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56018700 1 -- 172.21.15.52:0/4242103141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe50107850 msgr2=0x7fbe5019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56018700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe50107850 0x7fbe5019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56018700 1 -- 172.21.15.52:0/4242103141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006ad80 msgr2=0x7fbe50198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56018700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006ad80 0x7fbe50198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56018700 1 -- 172.21.15.52:0/4242103141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5019d760 con 0x7fbe5006a3d0 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe55817700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006ad80 0x7fbe50198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe56018700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 0x7fbe50198750 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e9a0 tx=0x7fbe4c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe477fe700 1 -- 172.21.15.52:0/4242103141 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c024070 con 0x7fbe5006a3d0 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe477fe700 1 -- 172.21.15.52:0/4242103141 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe4c00cb90 con 0x7fbe5006a3d0 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe477fe700 1 -- 172.21.15.52:0/4242103141 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c01fb70 con 0x7fbe5006a3d0 2024-06-02T23:25:43.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.372+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5019da50 con 0x7fbe5006a3d0 2024-06-02T23:25:43.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.373+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe50101390 con 0x7fbe5006a3d0 2024-06-02T23:25:43.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.376+0000 7fbe477fe700 1 -- 172.21.15.52:0/4242103141 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe4c027070 con 0x7fbe5006a3d0 2024-06-02T23:25:43.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.377+0000 7fbe477fe700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe3c0a2d40 0x7fbe3c0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.377+0000 7fbe62affb80 1 Processor -- start 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.377+0000 7fbe55817700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe3c0a2d40 0x7fbe3c0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.377+0000 7fbe477fe700 1 -- 172.21.15.52:0/4242103141 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe4c0d75c0 con 0x7fbe5006a3d0 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 -- start start 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe55817700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe3c0a2d40 0x7fbe3c0a5200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009680 tx=0x7fbe40009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07530b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d074cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5649d069e9a0 con 0x5649d0752cd0 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649d06468b0 con 0x5649d07535f0 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe62affb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5649d069a960 con 0x5649d074d460 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe56819700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe56018700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe56018700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07530b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59318/0 (socket says 172.21.15.52:59318) 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe55817700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d074cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.378+0000 7fbe56018700 1 -- 172.21.15.52:0/74283943 learned_addr learned my addr 172.21.15.52:0/74283943 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe55817700 1 -- 172.21.15.52:0/74283943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 msgr2=0x5649d074d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe55817700 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe55817700 1 -- 172.21.15.52:0/74283943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 msgr2=0x5649d07530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe55817700 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe55817700 1 -- 172.21.15.52:0/74283943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d063d0e0 con 0x5649d07535f0 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe55817700 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d074cf20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009870 tx=0x7fbe40015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bacb958fac64b86c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe44d70700 1 -- 172.21.15.52:0/74283943 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4001edf0 con 0x5649d07535f0 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe44d70700 1 -- 172.21.15.52:0/74283943 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe4001f8d0 con 0x5649d07535f0 2024-06-02T23:25:43.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.379+0000 7fbe44d70700 1 -- 172.21.15.52:0/74283943 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4001c950 con 0x5649d07535f0 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 -- 172.21.15.52:0/74283943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 msgr2=0x5649d074cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d074cf20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009870 tx=0x7fbe40015000 comp rx=0 tx=0).stop 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 -- 172.21.15.52:0/74283943 shutdown_connections 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 --2- 172.21.15.52:0/74283943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d074cf20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.380+0000 7fbe62affb80 1 -- 172.21.15.52:0/74283943 >> 172.21.15.52:0/74283943 conn(0x5649d0677e50 msgr2=0x5649d07481b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.381+0000 7fbe62affb80 1 -- 172.21.15.52:0/74283943 shutdown_connections 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.381+0000 7fbe62affb80 1 -- 172.21.15.52:0/74283943 wait complete. 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.381+0000 7fbe62affb80 1 -- start start 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.381+0000 7fbe62affb80 1 Processor -- start 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.381+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d0759350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d0752cd0 0x5649d0756650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 0x5649d0756b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d0759350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d0759350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45352/0 (socket says 172.21.15.52:45352) 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 0x5649d0756b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe55817700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d0752cd0 0x5649d0756650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe55817700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d0752cd0 0x5649d0756650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33338/0 (socket says 172.21.15.52:33338) 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56819700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 0x5649d0756b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59320/0 (socket says 172.21.15.52:59320) 2024-06-02T23:25:43.385 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe62affb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5649d075a060 con 0x5649d07535f0 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe62affb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5649d063ed80 con 0x5649d074d460 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe62affb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5649d075a1e0 con 0x5649d0752cd0 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 -- 172.21.15.52:0/4142124789 learned_addr learned my addr 172.21.15.52:0/4142124789 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe62affb80 10 client.0 _ll_register_callbacks cb 0x5649d0638ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 -- 172.21.15.52:0/4142124789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d0752cd0 msgr2=0x5649d0756650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d0752cd0 0x5649d0756650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 -- 172.21.15.52:0/4142124789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 msgr2=0x5649d0756b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 0x5649d0756b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56018700 1 -- 172.21.15.52:0/4142124789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d05eda60 con 0x5649d074d460 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.382+0000 7fbe56819700 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 0x5649d0756b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4001f8d0 con 0x5649d074d460 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe56018700 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d0759350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009230 tx=0x7fbe4c0a4330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa2af505abcf5fd4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:43.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe39ffb700 10 client.28643 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe39ffb700 1 -- 172.21.15.52:0/4142124789 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0a3df0 con 0x5649d074d460 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe39ffb700 1 -- 172.21.15.52:0/4142124789 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe4c0d70b0 con 0x5649d074d460 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe39ffb700 1 -- 172.21.15.52:0/4142124789 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbe4c0d72b0 con 0x5649d074d460 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe62affb80 10 client.28643 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=37}) -- 0x5649d07648e0 con 0x5649d074d460 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe39ffb700 1 -- 172.21.15.52:0/4142124789 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fbe4c00c6e0 con 0x5649d074d460 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.383+0000 7fbe62affb80 10 client.28643 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe62affb80 10 client.28643 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe62affb80 20 client.28643 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe62affb80 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07691f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe62affb80 4 client.28643 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649d06abea0 con 0x5649d0766e00 2024-06-02T23:25:43.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe55817700 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07691f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe39ffb700 10 client.28643 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.384+0000 7fbe55817700 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07691f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=395da087fe36bd74 server_cookie=1c0ca1f092858117 in_seq=0 out_seq=0 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe39ffb700 1 -- 172.21.15.52:0/4142124789 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5649d06abea0 con 0x5649d0766e00 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe39ffb700 10 client.28643 handle_command_reply: tid=0 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 client.28643 shutdown 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 msgr2=0x5649d0759350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d0759350 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c009230 tx=0x7fbe4c0a4330 comp rx=0 tx=0).stop 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 shutdown_connections 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07691f0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe39ffb700 0 client.28643 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d07535f0 0x5649d0756b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d0759350 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 --2- 172.21.15.52:0/4142124789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d0752cd0 0x5649d0756650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 >> 172.21.15.52:0/4142124789 conn(0x5649d0677e50 msgr2=0x5649d0747da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:43.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 shutdown_connections 2024-06-02T23:25:43.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.385+0000 7fbe62affb80 1 -- 172.21.15.52:0/4142124789 wait complete. 2024-06-02T23:25:43.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.386+0000 7fbe62affb80 20 client.28643 trim_cache size 0 max 16384 2024-06-02T23:25:43.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.416+0000 7fbe62affb80 1 Processor -- start 2024-06-02T23:25:43.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.416+0000 7fbe62affb80 1 -- start start 2024-06-02T23:25:43.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07542a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d0751ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe62affb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501aa560 con 0x5649d0752cd0 2024-06-02T23:25:43.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe62affb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501ac1c0 con 0x5649d07535f0 2024-06-02T23:25:43.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe62affb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501aac30 con 0x5649d074d460 2024-06-02T23:25:43.418 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56819700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d0751ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe55817700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56819700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d0751ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45362/0 (socket says 172.21.15.52:45362) 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56819700 1 -- 172.21.15.52:0/3215992188 learned_addr learned my addr 172.21.15.52:0/3215992188 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56018700 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe55817700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07542a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59336/0 (socket says 172.21.15.52:59336) 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56018700 1 -- 172.21.15.52:0/3215992188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 msgr2=0x5649d0751ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56018700 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d0751ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56018700 1 -- 172.21.15.52:0/3215992188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 msgr2=0x5649d07542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56018700 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.417+0000 7fbe56018700 1 -- 172.21.15.52:0/3215992188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649d0754be0 con 0x5649d074d460 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.418+0000 7fbe56018700 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074cf20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0ac530 tx=0x7fbe4c0a6000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3b508dd95e1b4a5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:43.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.418+0000 7fbe3bfff700 1 -- 172.21.15.52:0/3215992188 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0a3df0 con 0x5649d074d460 2024-06-02T23:25:43.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.418+0000 7fbe3bfff700 1 -- 172.21.15.52:0/3215992188 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe4001f8d0 con 0x5649d074d460 2024-06-02T23:25:43.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.418+0000 7fbe3bfff700 1 -- 172.21.15.52:0/3215992188 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0d7110 con 0x5649d074d460 2024-06-02T23:25:43.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 -- 172.21.15.52:0/3215992188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 msgr2=0x5649d074cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074cf20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0ac530 tx=0x7fbe4c0a6000 comp rx=0 tx=0).stop 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 -- 172.21.15.52:0/3215992188 shutdown_connections 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d07535f0 0x5649d0751ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d074d460 0x5649d074cf20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 --2- 172.21.15.52:0/3215992188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d07542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 -- 172.21.15.52:0/3215992188 >> 172.21.15.52:0/3215992188 conn(0x5649d0677e50 msgr2=0x5649d0751790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 -- 172.21.15.52:0/3215992188 shutdown_connections 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 -- 172.21.15.52:0/3215992188 wait complete. 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.419+0000 7fbe62affb80 1 -- start start 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 Processor -- start 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d075fc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d075c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d07535f0 0x5649d075c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501aba30 con 0x5649d0752cd0 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501ab330 con 0x5649d074d460 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501a8500 con 0x5649d07535f0 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe56018700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d075fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe62affb80 10 client.0 _ll_register_callbacks cb 0x5649d0638ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe55817700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d075c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe56018700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d075fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45364/0 (socket says 172.21.15.52:45364) 2024-06-02T23:25:43.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe56018700 1 -- 172.21.15.52:0/174238729 learned_addr learned my addr 172.21.15.52:0/174238729 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe56819700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d07535f0 0x5649d075c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.420+0000 7fbe56018700 1 -- 172.21.15.52:0/174238729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d07535f0 msgr2=0x5649d075c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe56018700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d07535f0 0x5649d075c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe56018700 1 -- 172.21.15.52:0/174238729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 msgr2=0x5649d075c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe56018700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d075c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe56018700 1 -- 172.21.15.52:0/174238729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649d075d230 con 0x5649d074d460 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe55817700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d075c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe56819700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d07535f0 0x5649d075c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe56018700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d075fc80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0a0250 tx=0x7fbe4c005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdaee64cf46b5cba server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe39ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:25:43.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe39ffb700 1 -- 172.21.15.52:0/174238729 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c0d7110 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe39ffb700 1 -- 172.21.15.52:0/174238729 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe4c09fb40 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe4001f8d0 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe39ffb700 1 -- 172.21.15.52:0/174238729 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fbe4c0a3df0 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe62affb80 10 client.28649 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.421+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5649d07d2b30 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.422+0000 7fbe39ffb700 1 -- 172.21.15.52:0/174238729 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe4001f8d0 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.422+0000 7fbe39ffb700 1 -- 172.21.15.52:0/174238729 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fbe4c08a070 con 0x5649d074d460 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.422+0000 7fbe62affb80 10 client.28649 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.422+0000 7fbe62affb80 10 client.28649 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.422+0000 7fbe62affb80 20 client.28649 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.423+0000 7fbe62affb80 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07d5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.423+0000 7fbe62affb80 4 client.28649 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}] 2024-06-02T23:25:43.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.423+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 130.0, "await_for": 45.0}) -- 0x5649d06a6420 con 0x5649d0766e00 2024-06-02T23:25:43.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.423+0000 7fbe55817700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07d5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:43.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.423+0000 7fbe55817700 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07d5060 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efedac2cf9d29440 server_cookie=2c57c119b92e8ea7 in_seq=0 out_seq=0 2024-06-02T23:25:43.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.423+0000 7fbe39ffb700 10 client.28649 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:43.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7828 11.75 MB/sec warmup 51 sec latency 92.277 ms 2024-06-02T23:25:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:43 smithi123 ceph-mon[35977]: pgmap v846: 97 pgs: 97 active+clean; 325 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.6 MiB/s wr, 248 op/s 2024-06-02T23:25:43.757 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:43 smithi052 ceph-mon[32340]: pgmap v846: 97 pgs: 97 active+clean; 325 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.6 MiB/s wr, 248 op/s 2024-06-02T23:25:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.759+0000 7fbe39ffb700 1 -- 172.21.15.52:0/174238729 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5649d06a6420 con 0x5649d0766e00 2024-06-02T23:25:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.759+0000 7fbe39ffb700 10 client.28649 handle_command_reply: tid=0 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.759+0000 7fbe62affb80 1 client.28649 shutdown 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 msgr2=0x5649d075fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d075fc80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c0a0250 tx=0x7fbe4c005bb0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 shutdown_connections 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5649d0766e00 0x5649d07d5060 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5649d07535f0 0x5649d075c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5649d074d460 0x5649d075fc80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 --2- 172.21.15.52:0/174238729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5649d0752cd0 0x5649d075c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe39ffb700 0 client.28649 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 >> 172.21.15.52:0/174238729 conn(0x5649d0677e50 msgr2=0x5649d0752020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 shutdown_connections 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 1 -- 172.21.15.52:0/174238729 wait complete. 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.760+0000 7fbe62affb80 20 client.28649 trim_cache size 0 max 16384 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 37, 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 46, 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout: "479d83de": { 2024-06-02T23:25:43.762 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 46, 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 130.0, 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:25:43.763 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe3c0a2d40 msgr2=0x7fbe3c0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe3c0a2d40 0x7fbe3c0a5200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009680 tx=0x7fbe40009260 comp rx=0 tx=0).stop 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 msgr2=0x7fbe50198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 0x7fbe50198750 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c00e9a0 tx=0x7fbe4c00ecb0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 shutdown_connections 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe3c0a2d40 0x7fbe3c0a5200 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe50107850 0x7fbe5019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe5006a3d0 0x7fbe50198750 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 --2- 172.21.15.52:0/4242103141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe5006ad80 0x7fbe50198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 >> 172.21.15.52:0/4242103141 conn(0x7fbe50078330 msgr2=0x7fbe500ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 shutdown_connections 2024-06-02T23:25:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:43.761+0000 7fbe5827c700 1 -- 172.21.15.52:0/4242103141 wait complete. 2024-06-02T23:25:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:43 smithi174 ceph-mon[35994]: pgmap v846: 97 pgs: 97 active+clean; 325 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.6 MiB/s wr, 248 op/s 2024-06-02T23:25:44.414 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '479d83de', quiesce took 16.5 seconds. Will release after: -6.5 2024-06-02T23:25:44.414 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:25:44.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 11.53 MB/sec warmup 52 sec latency 1023.118 ms 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 -- 172.21.15.52:0/3407326801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880107840 msgr2=0x7f488010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 --2- 172.21.15.52:0/3407326801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880107840 0x7f488010f3c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f487800b0d0 tx=0x7f487800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 -- 172.21.15.52:0/3407326801 shutdown_connections 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 --2- 172.21.15.52:0/3407326801 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880107840 0x7f488010f3c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 --2- 172.21.15.52:0/3407326801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4880102740 0x7f4880102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 --2- 172.21.15.52:0/3407326801 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f48801030f0 0x7f4880107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 -- 172.21.15.52:0/3407326801 >> 172.21.15.52:0/3407326801 conn(0x7f48800795f0 msgr2=0x7f4880079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 -- 172.21.15.52:0/3407326801 shutdown_connections 2024-06-02T23:25:45.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.037+0000 7f4884e8d700 1 -- 172.21.15.52:0/3407326801 wait complete. 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 Processor -- start 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 -- start start 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 0x7f4880074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801030f0 0x7f4880074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4880107840 0x7f4880075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4880112630 con 0x7f4880102740 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48801124b0 con 0x7f48801030f0 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f4884e8d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f48801127b0 con 0x7f4880107840 2024-06-02T23:25:45.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f487e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 0x7f4880074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f487e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 0x7f4880074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39892/0 (socket says 172.21.15.52:39892) 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f487e59c700 1 -- 172.21.15.52:0/485573974 learned_addr learned my addr 172.21.15.52:0/485573974 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.038+0000 7f487dd9b700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801030f0 0x7f4880074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487ed9d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4880107840 0x7f4880075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487e59c700 1 -- 172.21.15.52:0/485573974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4880107840 msgr2=0x7f4880075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487e59c700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4880107840 0x7f4880075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487e59c700 1 -- 172.21.15.52:0/485573974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801030f0 msgr2=0x7f4880074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487e59c700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801030f0 0x7f4880074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487e59c700 1 -- 172.21.15.52:0/485573974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48801a6f60 con 0x7f4880102740 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f487e59c700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 0x7f4880074790 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f487000d950 tx=0x7f487000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4870016070 con 0x7f4880102740 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4870009df0 con 0x7f4880102740 2024-06-02T23:25:45.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48801a7250 con 0x7f4880102740 2024-06-02T23:25:45.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487000f3c0 con 0x7f4880102740 2024-06-02T23:25:45.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.039+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48801a7790 con 0x7f4880102740 2024-06-02T23:25:45.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.040+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4880069180 con 0x7f4880102740 2024-06-02T23:25:45.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.041+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4870019070 con 0x7f4880102740 2024-06-02T23:25:45.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.042+0000 7f486f7fe700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f48680a2c20 0x7f48680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:45.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.042+0000 7f487dd9b700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f48680a2c20 0x7f48680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:45.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.043+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f48700cec30 con 0x7f4880102740 2024-06-02T23:25:45.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.043+0000 7f487dd9b700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f48680a2c20 0x7f48680a50e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4880108d40 tx=0x7f487400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:45.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.046+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f487009bb60 con 0x7f4880102740 2024-06-02T23:25:45.142 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:45.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.320+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f488002de60 con 0x7f4880102740 2024-06-02T23:25:45.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.321+0000 7f486f7fe700 1 -- 172.21.15.52:0/485573974 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7f487009b460 con 0x7f4880102740 2024-06-02T23:25:45.322 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:25:45.322 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:25:45.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f48680a2c20 msgr2=0x7f48680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f48680a2c20 0x7f48680a50e0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4880108d40 tx=0x7f487400b3c0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 msgr2=0x7f4880074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 0x7f4880074790 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f487000d950 tx=0x7f487000dc60 comp rx=0 tx=0).stop 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 shutdown_connections 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f48680a2c20 0x7f48680a50e0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4880107840 0x7f4880075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4880102740 0x7f4880074790 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 --2- 172.21.15.52:0/485573974 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48801030f0 0x7f4880074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 >> 172.21.15.52:0/485573974 conn(0x7f48800795f0 msgr2=0x7f4880100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 shutdown_connections 2024-06-02T23:25:45.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:45.324+0000 7f4884e8d700 1 -- 172.21.15.52:0/485573974 wait complete. 2024-06-02T23:25:45.326 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:25:45.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 11.31 MB/sec warmup 53 sec latency 2023.168 ms 2024-06-02T23:25:45.596 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:45 smithi052 ceph-mon[32340]: pgmap v847: 97 pgs: 97 active+clean; 326 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 4.6 MiB/s wr, 214 op/s 2024-06-02T23:25:45.596 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:45 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/485573974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:45 smithi123 ceph-mon[35977]: pgmap v847: 97 pgs: 97 active+clean; 326 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 4.6 MiB/s wr, 214 op/s 2024-06-02T23:25:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:45 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/485573974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:45 smithi174 ceph-mon[35994]: pgmap v847: 97 pgs: 97 active+clean; 326 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 759 KiB/s rd, 4.6 MiB/s wr, 214 op/s 2024-06-02T23:25:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:45 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/485573974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:46.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 11.10 MB/sec warmup 54 sec latency 3023.274 ms 2024-06-02T23:25:47.159 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:25:47.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 10.90 MB/sec warmup 55 sec latency 4023.323 ms 2024-06-02T23:25:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 -- 172.21.15.52:0/158517553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006b770 msgr2=0x7faca0107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 --2- 172.21.15.52:0/158517553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006b770 0x7faca0107130 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fac880099c0 tx=0x7fac88009cd0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 -- 172.21.15.52:0/158517553 shutdown_connections 2024-06-02T23:25:47.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 --2- 172.21.15.52:0/158517553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0107860 0x7faca010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 --2- 172.21.15.52:0/158517553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006adc0 0x7faca006b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 --2- 172.21.15.52:0/158517553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006b770 0x7faca0107130 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 -- 172.21.15.52:0/158517553 >> 172.21.15.52:0/158517553 conn(0x7faca0078340 msgr2=0x7faca0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 -- 172.21.15.52:0/158517553 shutdown_connections 2024-06-02T23:25:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.670+0000 7faca560c700 1 -- 172.21.15.52:0/158517553 wait complete. 2024-06-02T23:25:47.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.671+0000 7faca560c700 1 Processor -- start 2024-06-02T23:25:47.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.671+0000 7faca560c700 1 -- start start 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.671+0000 7faca560c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 0x7faca0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.671+0000 7faca560c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006b770 0x7faca0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.671+0000 7faca560c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0107860 0x7faca01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7faca560c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7faca0112670 con 0x7faca006adc0 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7faca560c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7faca01124f0 con 0x7faca006b770 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7faca560c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faca01127f0 con 0x7faca0107860 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 0x7faca0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 0x7faca0101840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39908/0 (socket says 172.21.15.52:39908) 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 -- 172.21.15.52:0/1949239466 learned_addr learned my addr 172.21.15.52:0/1949239466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9e7fc700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006b770 0x7faca0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:47.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9f7fe700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0107860 0x7faca01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 -- 172.21.15.52:0/1949239466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0107860 msgr2=0x7faca01022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0107860 0x7faca01022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 -- 172.21.15.52:0/1949239466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006b770 msgr2=0x7faca0101d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006b770 0x7faca0101d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 -- 172.21.15.52:0/1949239466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faca01a7020 con 0x7faca006adc0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9e7fc700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006b770 0x7faca0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.672+0000 7fac9effd700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 0x7faca0101840 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fac9000d950 tx=0x7fac9000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.673+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac90016070 con 0x7faca006adc0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.673+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faca01a7310 con 0x7faca006adc0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.673+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fac9000fd40 con 0x7faca006adc0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.673+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac9000e3b0 con 0x7faca006adc0 2024-06-02T23:25:47.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.673+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faca01a7870 con 0x7faca006adc0 2024-06-02T23:25:47.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.674+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faca0103be0 con 0x7faca006adc0 2024-06-02T23:25:47.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.675+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fac90019070 con 0x7faca006adc0 2024-06-02T23:25:47.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.676+0000 7fac97fff700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac8c0a2c20 0x7fac8c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:47.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.676+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fac900cee80 con 0x7faca006adc0 2024-06-02T23:25:47.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.676+0000 7fac9e7fc700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac8c0a2c20 0x7fac8c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:47.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.676+0000 7fac9e7fc700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac8c0a2c20 0x7fac8c0a50e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7faca0103280 tx=0x7fac88005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:47.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.680+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fac9009be10 con 0x7faca006adc0 2024-06-02T23:25:47.702 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:47 smithi123 ceph-mon[35977]: pgmap v848: 97 pgs: 97 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-06-02T23:25:47.833 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:47 smithi052 ceph-mon[32340]: pgmap v848: 97 pgs: 97 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-06-02T23:25:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:47 smithi174 ceph-mon[35994]: pgmap v848: 97 pgs: 97 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 976 KiB/s rd, 5.0 MiB/s wr, 271 op/s 2024-06-02T23:25:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.948+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faca002dca0 con 0x7faca006adc0 2024-06-02T23:25:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.949+0000 7fac97fff700 1 -- 172.21.15.52:0/1949239466 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7fac9009b710 con 0x7faca006adc0 2024-06-02T23:25:47.949 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:25:47.949 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:25:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac8c0a2c20 msgr2=0x7fac8c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac8c0a2c20 0x7fac8c0a50e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7faca0103280 tx=0x7fac88005fb0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 msgr2=0x7faca0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 0x7faca0101840 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fac9000d950 tx=0x7fac9000dc60 comp rx=0 tx=0).stop 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 shutdown_connections 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fac8c0a2c20 0x7fac8c0a50e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faca0107860 0x7faca01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7faca006adc0 0x7faca0101840 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 --2- 172.21.15.52:0/1949239466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7faca006b770 0x7faca0101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 >> 172.21.15.52:0/1949239466 conn(0x7faca0078340 msgr2=0x7faca00ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 shutdown_connections 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:47.951+0000 7faca560c700 1 -- 172.21.15.52:0/1949239466 wait complete. 2024-06-02T23:25:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:25:48.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 10.70 MB/sec warmup 56 sec latency 5023.395 ms 2024-06-02T23:25:48.752 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:48 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1949239466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:48.753 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 479d83de --release --await' 2024-06-02T23:25:48.753 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 479d83de --release --await 2024-06-02T23:25:48.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:48 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1949239466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:48.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:48 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1949239466' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 -- 172.21.15.52:0/55759158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f0101a20 msgr2=0x7f40f0101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 --2- 172.21.15.52:0/55759158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f0101a20 0x7f40f0101e40 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009900 tx=0x7f40e002a410 comp rx=0 tx=0).stop 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 -- 172.21.15.52:0/55759158 shutdown_connections 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 --2- 172.21.15.52:0/55759158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40f01035d0 0x7f40f0109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 --2- 172.21.15.52:0/55759158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0102c10 0x7f40f0103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 --2- 172.21.15.52:0/55759158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f0101a20 0x7f40f0101e40 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 -- 172.21.15.52:0/55759158 >> 172.21.15.52:0/55759158 conn(0x7f40f00fcf90 msgr2=0x7f40f00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 -- 172.21.15.52:0/55759158 shutdown_connections 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 -- 172.21.15.52:0/55759158 wait complete. 2024-06-02T23:25:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.285+0000 7f40f5c95640 1 Processor -- start 2024-06-02T23:25:49.287 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 -- start start 2024-06-02T23:25:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0101a20 0x7f40f0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40f0102c10 0x7f40f01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 0x7f40f019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.288 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010d520 con 0x7f40f01035d0 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010d3a0 con 0x7f40f0101a20 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40f5c95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40f010d6a0 con 0x7f40f0102c10 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 0x7f40f019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 0x7f40f019f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39918/0 (socket says 172.21.15.52:39918) 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 -- 172.21.15.52:0/1649025106 learned_addr learned my addr 172.21.15.52:0/1649025106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40eeffd640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40f0102c10 0x7f40f01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0101a20 0x7f40f0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 -- 172.21.15.52:0/1649025106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40f0102c10 msgr2=0x7f40f01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40f0102c10 0x7f40f01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.289 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 -- 172.21.15.52:0/1649025106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0101a20 msgr2=0x7f40f0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0101a20 0x7f40f0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 -- 172.21.15.52:0/1649025106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40f019f7b0 con 0x7f40f01035d0 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0101a20 0x7f40f0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.286+0000 7f40effff640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 0x7f40f019f140 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f40e800eaa0 tx=0x7f40e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.287+0000 7f40ecff9640 1 -- 172.21.15.52:0/1649025106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e800cdb0 con 0x7f40f01035d0 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.287+0000 7f40ecff9640 1 -- 172.21.15.52:0/1649025106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e8005020 con 0x7f40f01035d0 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.287+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40f019faa0 con 0x7f40f01035d0 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.287+0000 7f40ecff9640 1 -- 172.21.15.52:0/1649025106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e80116d0 con 0x7f40f01035d0 2024-06-02T23:25:49.290 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.287+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40f019ffa0 con 0x7f40f01035d0 2024-06-02T23:25:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.289+0000 7f40ecff9640 1 -- 172.21.15.52:0/1649025106 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f40e80051d0 con 0x7f40f01035d0 2024-06-02T23:25:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.289+0000 7f40f899c740 1 Processor -- start 2024-06-02T23:25:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.289+0000 7f40f899c740 1 -- start start 2024-06-02T23:25:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40f899c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33b010 0x555bfb33b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40f899c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb340500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40f899c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 0x555bfb343000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40f899c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555bfb29e4f0 con 0x555bfb340b70 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40f899c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555bfb23c870 con 0x555bfb33b010 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40f899c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555bfb29a4b0 con 0x555bfb33ba00 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 0x555bfb343000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 0x555bfb343000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39930/0 (socket says 172.21.15.52:39930) 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 -- 172.21.15.52:0/1743593201 learned_addr learned my addr 172.21.15.52:0/1743593201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40ecff9640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d80a2c30 0x7f40d80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33b010 0x555bfb33b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb340500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40ecff9640 1 -- 172.21.15.52:0/1649025106 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40e80cf1e0 con 0x7f40f01035d0 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40effff640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d80a2c30 0x7f40d80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 -- 172.21.15.52:0/1743593201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 msgr2=0x555bfb340500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb340500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 -- 172.21.15.52:0/1743593201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33b010 msgr2=0x555bfb33b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33b010 0x555bfb33b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 -- 172.21.15.52:0/1743593201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfb2589b0 con 0x555bfb340b70 2024-06-02T23:25:49.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40eeffd640 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 0x555bfb343000 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x555bfb33c260 tx=0x7f40e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fa829bb93ff893 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40effff640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d80a2c30 0x7f40d80a50f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f40e8098420 tx=0x7f40e8086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.290+0000 7f40d5ffb640 1 -- 172.21.15.52:0/1743593201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4012d60 con 0x555bfb340b70 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40d5ffb640 1 -- 172.21.15.52:0/1743593201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e4010770 con 0x555bfb340b70 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40d5ffb640 1 -- 172.21.15.52:0/1743593201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4003830 con 0x555bfb340b70 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 -- 172.21.15.52:0/1743593201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 msgr2=0x555bfb343000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 0x555bfb343000 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x555bfb33c260 tx=0x7f40e4004730 comp rx=0 tx=0).stop 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 -- 172.21.15.52:0/1743593201 shutdown_connections 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb340500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb340b70 0x555bfb343000 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 --2- 172.21.15.52:0/1743593201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33b010 0x555bfb33b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 -- 172.21.15.52:0/1743593201 >> 172.21.15.52:0/1743593201 conn(0x555bfb336680 msgr2=0x555bfb336a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 -- 172.21.15.52:0/1743593201 shutdown_connections 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.291+0000 7f40f899c740 1 -- 172.21.15.52:0/1743593201 wait complete. 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 -- start start 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 Processor -- start 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3452b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb3457f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb340b70 0x555bfb34c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555bfb343b20 con 0x555bfb33b010 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555bfb227ae0 con 0x555bfb340b70 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555bfb343ca0 con 0x555bfb33ba00 2024-06-02T23:25:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40f899c740 10 client.0 _ll_register_callbacks cb 0x555bfb2609b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40eeffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39932/0 (socket says 172.21.15.52:39932) 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40eeffd640 1 -- 172.21.15.52:0/3426965463 learned_addr learned my addr 172.21.15.52:0/3426965463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb3457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.292+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb340b70 0x555bfb34c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40eeffd640 1 -- 172.21.15.52:0/3426965463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 msgr2=0x555bfb3457f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb3457f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40eeffd640 1 -- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb340b70 msgr2=0x555bfb34c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb340b70 0x555bfb34c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40eeffd640 1 -- 172.21.15.52:0/3426965463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfafb2210 con 0x555bfb33b010 2024-06-02T23:25:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb3457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3452b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bfd0 tx=0x7f40e4001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686ea486ce31ace server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40ceffd640 1 -- 172.21.15.52:0/3426965463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e4005220 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.293+0000 7f40ceffd640 1 -- 172.21.15.52:0/3426965463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40e40053c0 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e4010770 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40ceffd640 1 -- 172.21.15.52:0/3426965463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f40e4004460 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 10 client.19260 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555bfb07e620 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40ceffd640 1 -- 172.21.15.52:0/3426965463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e4010770 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40ceffd640 1 -- 172.21.15.52:0/3426965463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f40e4029030 con 0x555bfb33b010 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 10 client.19260 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 10 client.19260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 20 client.19260 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb350e70 0x555bfb353260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 4 client.19260 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.294+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555bfb2b1f10 con 0x555bfb350e70 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.295+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb350e70 0x555bfb353260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.295+0000 7f40ceffd640 10 client.19260 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.295+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb350e70 0x555bfb353260 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2ed86cbc4a61fb2 server_cookie=174c58b4cd769cdf in_seq=0 out_seq=0 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40ceffd640 1 -- 172.21.15.52:0/3426965463 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555bfb2b1f10 con 0x555bfb350e70 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40ceffd640 10 client.19260 handle_command_reply: tid=0 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 client.19260 shutdown 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 msgr2=0x555bfb3452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3452b0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f40e400bfd0 tx=0x7f40e4001e50 comp rx=0 tx=0).stop 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 shutdown_connections 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb350e70 0x555bfb353260 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb33ba00 0x555bfb3457f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40ceffd640 0 client.19260 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb340b70 0x555bfb34c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 --2- 172.21.15.52:0/3426965463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3452b0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 >> 172.21.15.52:0/3426965463 conn(0x555bfb336680 msgr2=0x555bfb3357e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 shutdown_connections 2024-06-02T23:25:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 1 -- 172.21.15.52:0/3426965463 wait complete. 2024-06-02T23:25:49.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.296+0000 7f40f899c740 20 client.19260 trim_cache size 0 max 16384 2024-06-02T23:25:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 Processor -- start 2024-06-02T23:25:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 -- start start 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb334f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb335490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01a8d50 con 0x555bfb33b010 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01aa9b0 con 0x555bfb33ba00 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40f899c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01a9420 con 0x555bfb336680 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb335490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb335490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39940/0 (socket says 172.21.15.52:39940) 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3718756783 learned_addr learned my addr 172.21.15.52:0/3718756783 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3718756783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 msgr2=0x555bfb334f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb334f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.312+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb334f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3718756783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 msgr2=0x555bfb3359d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3718756783 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555bfb3364b0 con 0x555bfb33b010 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb334f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb335490 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009900 tx=0x7f40e0005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6bfcf10da819c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40d57fa640 1 -- 172.21.15.52:0/3718756783 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0035050 con 0x555bfb33b010 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40d57fa640 1 -- 172.21.15.52:0/3718756783 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e0005040 con 0x555bfb33b010 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.313+0000 7f40d57fa640 1 -- 172.21.15.52:0/3718756783 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e00049b0 con 0x555bfb33b010 2024-06-02T23:25:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- 172.21.15.52:0/3718756783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 msgr2=0x555bfb335490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb335490 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f40e0009900 tx=0x7f40e0005d50 comp rx=0 tx=0).stop 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- 172.21.15.52:0/3718756783 shutdown_connections 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3359d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb335490 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- 172.21.15.52:0/3718756783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb334f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- 172.21.15.52:0/3718756783 >> 172.21.15.52:0/3718756783 conn(0x555bfb340640 msgr2=0x555bfb344ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- 172.21.15.52:0/3718756783 shutdown_connections 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- 172.21.15.52:0/3718756783 wait complete. 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- start start 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 Processor -- start 2024-06-02T23:25:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb3b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3b73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3be430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01aa220 con 0x555bfb33b010 2024-06-02T23:25:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01a9b20 con 0x555bfb33ba00 2024-06-02T23:25:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40f01a6d30 con 0x555bfb336680 2024-06-02T23:25:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.314+0000 7f40f899c740 10 client.0 _ll_register_callbacks cb 0x7f40e400bf00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3b73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3b73f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39952/0 (socket says 172.21.15.52:39952) 2024-06-02T23:25:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3971986688 learned_addr learned my addr 172.21.15.52:0/3971986688 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb3b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3be430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3971986688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 msgr2=0x555bfb3b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb3b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 msgr2=0x555bfb3be430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3be430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 -- 172.21.15.52:0/3971986688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555bfb33a5c0 con 0x555bfb33b010 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ef7fe640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3b73f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005940 tx=0x7f40e0032a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de26a6fdfea3b3c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ceffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ceffd640 1 -- 172.21.15.52:0/3971986688 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40e0051070 con 0x555bfb33b010 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40e0005040 con 0x555bfb33b010 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ceffd640 1 -- 172.21.15.52:0/3971986688 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f40e0032250 con 0x555bfb33b010 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.315+0000 7f40ceffd640 1 -- 172.21.15.52:0/3971986688 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f40e005d050 con 0x555bfb33b010 2024-06-02T23:25:49.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 10 client.19266 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x555bfb3bfad0 con 0x555bfb33b010 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40ceffd640 1 -- 172.21.15.52:0/3971986688 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f40e0005040 con 0x555bfb33b010 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40ceffd640 1 -- 172.21.15.52:0/3971986688 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f40e0031580 con 0x555bfb33b010 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 10 client.19266 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 10 client.19266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 20 client.19266 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb340b70 0x555bfb3c2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 4 client.19266 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555bfb2abc20 con 0x555bfb340b70 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb340b70 0x555bfb3c2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40ceffd640 10 client.19266 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:49.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.316+0000 7f40eeffd640 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb340b70 0x555bfb3c2520 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=680cc40a9c03bd82 server_cookie=2555e66b3fbe0586 in_seq=0 out_seq=0 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40ceffd640 1 -- 172.21.15.52:0/3971986688 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555bfb2abc20 con 0x555bfb340b70 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40ceffd640 10 client.19266 handle_command_reply: tid=0 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 client.19266 shutdown 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 msgr2=0x555bfb3b73f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3b73f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f40e0005940 tx=0x7f40e0032a90 comp rx=0 tx=0).stop 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 shutdown_connections 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555bfb340b70 0x555bfb3c2520 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555bfb33ba00 0x555bfb3be430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555bfb33b010 0x555bfb3b73f0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40ceffd640 0 client.19266 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 --2- 172.21.15.52:0/3971986688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555bfb336680 0x555bfb3b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 >> 172.21.15.52:0/3971986688 conn(0x555bfb340640 msgr2=0x555bfb339260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 shutdown_connections 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.343+0000 7f40f899c740 1 -- 172.21.15.52:0/3971986688 wait complete. 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f899c740 20 client.19266 trim_cache size 0 max 16384 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:25:49.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:25:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d80a2c30 msgr2=0x7f40d80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d80a2c30 0x7f40d80a50f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f40e8098420 tx=0x7f40e8086040 comp rx=0 tx=0).stop 2024-06-02T23:25:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 msgr2=0x7f40f019f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:49.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 0x7f40f019f140 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f40e800eaa0 tx=0x7f40e800ef60 comp rx=0 tx=0).stop 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 shutdown_connections 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f40d80a2c30 0x7f40d80a50f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f40f0102c10 0x7f40f01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f40f01035d0 0x7f40f019f140 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 --2- 172.21.15.52:0/1649025106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40f0101a20 0x7f40f0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 >> 172.21.15.52:0/1649025106 conn(0x7f40f00fcf90 msgr2=0x7f40f0105f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 shutdown_connections 2024-06-02T23:25:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:25:49.344+0000 7f40f5c95640 1 -- 172.21.15.52:0/1649025106 wait complete. 2024-06-02T23:25:49.448 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 10.52 MB/sec warmup 57 sec latency 6023.491 ms 2024-06-02T23:25:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:49 smithi123 ceph-mon[35977]: pgmap v849: 97 pgs: 97 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-06-02T23:25:49.774 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:49 smithi174 ceph-mon[35994]: pgmap v849: 97 pgs: 97 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-06-02T23:25:49.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:49 smithi052 ceph-mon[32340]: pgmap v849: 97 pgs: 97 active+clean; 321 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.5 MiB/s wr, 203 op/s 2024-06-02T23:25:50.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 10.33 MB/sec warmup 58 sec latency 7023.603 ms 2024-06-02T23:25:50.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 -- 172.21.15.52:0/649453909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac1085f0 msgr2=0x7f9bac1089d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/649453909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac1085f0 0x7f9bac1089d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009990 tx=0x7f9b9c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 -- 172.21.15.52:0/649453909 shutdown_connections 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/649453909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac102ea0 0x7f9bac10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/649453909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac1085f0 0x7f9bac1089d0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/649453909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1024e0 0x7f9bac102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 -- 172.21.15.52:0/649453909 >> 172.21.15.52:0/649453909 conn(0x7f9bac079640 msgr2=0x7f9bac079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 -- 172.21.15.52:0/649453909 shutdown_connections 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.934+0000 7f9bb1efd700 1 -- 172.21.15.52:0/649453909 wait complete. 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.935+0000 7f9bb1efd700 1 Processor -- start 2024-06-02T23:25:50.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.935+0000 7f9bb1efd700 1 -- start start 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bb1efd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac1024e0 0x7f9bac074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bb1efd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 0x7f9bac074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bb1efd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1085f0 0x7f9bac075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bb1efd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac112640 con 0x7f9bac102ea0 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bb1efd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1124c0 con 0x7f9bac1024e0 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bb1efd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1127c0 con 0x7f9bac1085f0 2024-06-02T23:25:50.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 0x7f9bac074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 0x7f9bac074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39968/0 (socket says 172.21.15.52:39968) 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/821932579 learned_addr learned my addr 172.21.15.52:0/821932579 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac1024e0 0x7f9bac074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9babfff700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1085f0 0x7f9bac075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/821932579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1085f0 msgr2=0x7f9bac075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1085f0 0x7f9bac075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/821932579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac1024e0 msgr2=0x7f9bac074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac1024e0 0x7f9bac074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/821932579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9bac1a6f80 con 0x7f9bac102ea0 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9babfff700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1085f0 0x7f9bac075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac1024e0 0x7f9bac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.936+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 0x7f9bac074cd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c9b0 tx=0x7f9b9400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.937+0000 7f9ba97fa700 1 -- 172.21.15.52:0/821932579 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94007b00 con 0x7f9bac102ea0 2024-06-02T23:25:50.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.937+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9bac1a7270 con 0x7f9bac102ea0 2024-06-02T23:25:50.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.937+0000 7f9ba97fa700 1 -- 172.21.15.52:0/821932579 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b94007ca0 con 0x7f9bac102ea0 2024-06-02T23:25:50.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.937+0000 7f9ba97fa700 1 -- 172.21.15.52:0/821932579 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b94018730 con 0x7f9bac102ea0 2024-06-02T23:25:50.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.937+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9bac1a7800 con 0x7f9bac102ea0 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.939+0000 7f9ba97fa700 1 -- 172.21.15.52:0/821932579 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9b9400f450 con 0x7f9bac102ea0 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.940+0000 7f9ba97fa700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9b980a2c20 0x7f9b980a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.940+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9b980a2c20 0x7f9b980a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.940+0000 7f9ba97fa700 1 -- 172.21.15.52:0/821932579 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9b940cf130 con 0x7f9bac102ea0 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.940+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9b980a2c20 0x7f9b980a50e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006010 tx=0x7f9b9c00b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 Processor -- start 2024-06-02T23:25:50.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 -- start start 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43985340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f4397f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f4397fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f43871f60 con 0x558f43984f60 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f438d0850 con 0x558f4397f7a0 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558f43838460 con 0x558f43985880 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43985340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43985340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39978/0 (socket says 172.21.15.52:39978) 2024-06-02T23:25:50.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 -- 172.21.15.52:0/2418219562 learned_addr learned my addr 172.21.15.52:0/2418219562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f4397f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 -- 172.21.15.52:0/2418219562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 msgr2=0x558f4397f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f4397f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 -- 172.21.15.52:0/2418219562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 msgr2=0x558f4397fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f4397fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.941+0000 7f9bab7fe700 1 -- 172.21.15.52:0/2418219562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f4381f8d0 con 0x558f43984f60 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43985340 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009990 tx=0x7f9b9c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb21cb9e632497c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9ba1ffb700 1 -- 172.21.15.52:0/2418219562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00bbd0 con 0x558f43984f60 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9ba1ffb700 1 -- 172.21.15.52:0/2418219562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b9c00bd70 con 0x558f43984f60 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9ba1ffb700 1 -- 172.21.15.52:0/2418219562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c00fb10 con 0x558f43984f60 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bbc780b80 1 -- 172.21.15.52:0/2418219562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 msgr2=0x558f43985340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43985340 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009990 tx=0x7f9b9c01f000 comp rx=0 tx=0).stop 2024-06-02T23:25:50.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bbc780b80 1 -- 172.21.15.52:0/2418219562 shutdown_connections 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f4397fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43985340 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.942+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/2418219562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f4397f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- 172.21.15.52:0/2418219562 >> 172.21.15.52:0/2418219562 conn(0x558f438ac080 msgr2=0x558f4397a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- 172.21.15.52:0/2418219562 shutdown_connections 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- 172.21.15.52:0/2418219562 wait complete. 2024-06-02T23:25:50.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- start start 2024-06-02T23:25:50.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 Processor -- start 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f4398b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f439888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f43988e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558f43870fc0 con 0x558f43984f60 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558f438cc810 con 0x558f43985880 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558f4398c200 con 0x558f4397f7a0 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.943+0000 7f9bbc780b80 10 client.0 _ll_register_callbacks cb 0x558f438665b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f439888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f439888d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39988/0 (socket says 172.21.15.52:39988) 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/4282307618 learned_addr learned my addr 172.21.15.52:0/4282307618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9babfff700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f43988e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f4398b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/4282307618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 msgr2=0x558f4398b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f4398b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 msgr2=0x558f43988e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f43988e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/4282307618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f43896750 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f439888d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400afd0 tx=0x7f9b9400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1335f2f9a8ee3dfa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9b8affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9b8affd700 1 -- 172.21.15.52:0/4282307618 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940b3020 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9b8affd700 1 -- 172.21.15.52:0/4282307618 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9b9400f740 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b9c00bd70 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.944+0000 7f9b8affd700 1 -- 172.21.15.52:0/4282307618 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9b9409fd10 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 10 client.19278 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558f439969c0 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9b8affd700 1 -- 172.21.15.52:0/4282307618 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b9c00bd70 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9b8affd700 1 -- 172.21.15.52:0/4282307618 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f9b940a41d0 con 0x558f43984f60 2024-06-02T23:25:50.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 10 client.19278 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 10 client.19278 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 20 client.19278 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f4399b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 4 client.19278 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f438ddd50 con 0x558f43999000 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.945+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f4399b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.946+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f4399b3f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e82db65c018b8ae3 server_cookie=91dfc7adaddfe1d3 in_seq=0 out_seq=0 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.946+0000 7f9b8affd700 10 client.19278 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.946+0000 7f9b8affd700 1 -- 172.21.15.52:0/4282307618 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558f438ddd50 con 0x558f43999000 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.946+0000 7f9b8affd700 10 client.19278 handle_command_reply: tid=0 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.946+0000 7f9bbc780b80 1 client.19278 shutdown 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 msgr2=0x558f439888d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f439888d0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400afd0 tx=0x7f9b9400faf0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 shutdown_connections 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f4399b3f0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f43988e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9b8affd700 0 client.19278 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f4398b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/4282307618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f439888d0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 >> 172.21.15.52:0/4282307618 conn(0x558f438ac080 msgr2=0x558f43979700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 shutdown_connections 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 1 -- 172.21.15.52:0/4282307618 wait complete. 2024-06-02T23:25:50.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.947+0000 7f9bbc780b80 20 client.19278 trim_cache size 0 max 16384 2024-06-02T23:25:50.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.968+0000 7f9bbc780b80 1 Processor -- start 2024-06-02T23:25:50.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.968+0000 7f9bbc780b80 1 -- start start 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f43988900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43988e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f43979e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1aeb10 con 0x558f43984f60 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1b0770 con 0x558f4397f7a0 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1af1e0 con 0x558f43985880 2024-06-02T23:25:50.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43988e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9bab7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f43988900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43988e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39998/0 (socket says 172.21.15.52:39998) 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/269459224 learned_addr learned my addr 172.21.15.52:0/269459224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9babfff700 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f43979e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/269459224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 msgr2=0x558f43979e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f43979e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/269459224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 msgr2=0x558f43988900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f43988900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.969+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/269459224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f43984a60 con 0x558f43984f60 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.970+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43988e40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9b9409b660 tx=0x7f9b94099e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11f8bd3f7abe870 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:50.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.970+0000 7f9ba17fa700 1 -- 172.21.15.52:0/269459224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940b3070 con 0x558f43984f60 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.970+0000 7f9ba17fa700 1 -- 172.21.15.52:0/269459224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b940a45d0 con 0x558f43984f60 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.970+0000 7f9ba17fa700 1 -- 172.21.15.52:0/269459224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940a4140 con 0x558f43984f60 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 -- 172.21.15.52:0/269459224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 msgr2=0x558f43988e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43988e40 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9b9409b660 tx=0x7f9b94099e70 comp rx=0 tx=0).stop 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 -- 172.21.15.52:0/269459224 shutdown_connections 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f43985880 0x558f43979e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f4397f7a0 0x558f43988900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/269459224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43988e40 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 -- 172.21.15.52:0/269459224 >> 172.21.15.52:0/269459224 conn(0x558f438ac080 msgr2=0x558f439871e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 -- 172.21.15.52:0/269459224 shutdown_connections 2024-06-02T23:25:50.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 -- 172.21.15.52:0/269459224 wait complete. 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 -- start start 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.971+0000 7f9bbc780b80 1 Processor -- start 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f439914f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43991a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f4398ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1affe0 con 0x558f43984f60 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1af8e0 con 0x558f43985880 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9bac1acad0 con 0x558f4397f7a0 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 10 client.0 _ll_register_callbacks cb 0x558f438665b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43991a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43991a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40006/0 (socket says 172.21.15.52:40006) 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/659157551 learned_addr learned my addr 172.21.15.52:0/659157551 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f439914f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9babfff700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f4398ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/659157551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 msgr2=0x558f439914f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f439914f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 msgr2=0x558f4398ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f4398ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 -- 172.21.15.52:0/659157551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f4398d240 con 0x558f43984f60 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9ba2ffd700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43991a30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9b940074a0 tx=0x7f9b9400faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eebb94f7034572 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:50.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9b8affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9b8affd700 1 -- 172.21.15.52:0/659157551 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b940b3070 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9b8affd700 1 -- 172.21.15.52:0/659157551 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9b940ba140 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.972+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b940a45d0 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9b8affd700 1 -- 172.21.15.52:0/659157551 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f9b940ba340 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 10 client.19284 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558f43a03630 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9b8affd700 1 -- 172.21.15.52:0/659157551 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9b940a45d0 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9b8affd700 1 -- 172.21.15.52:0/659157551 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f9b9409a820 con 0x558f43984f60 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 10 client.19284 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 10 client.19284 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 20 client.19284 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f43a05aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 4 client.19284 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:25:50.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f438d82d0 con 0x558f43999000 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.973+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f43a05aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bab7fe700 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f43a05aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b424f2097a8b9938 server_cookie=4e9b59b7b7d22cc5 in_seq=0 out_seq=0 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9b8affd700 10 client.19284 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9b8affd700 1 -- 172.21.15.52:0/659157551 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f438d82d0 con 0x558f43999000 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9b8affd700 10 client.19284 handle_command_reply: tid=0 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 client.19284 shutdown 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 msgr2=0x558f43991a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43991a30 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9b940074a0 tx=0x7f9b9400faf0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 shutdown_connections 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x558f43999000 0x558f43a05aa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558f43985880 0x558f4398ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558f4397f7a0 0x558f439914f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 --2- 172.21.15.52:0/659157551 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558f43984f60 0x558f43991a30 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 >> 172.21.15.52:0/659157551 conn(0x558f438ac080 msgr2=0x558f439842e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9b8affd700 0 client.19284 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:25:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 shutdown_connections 2024-06-02T23:25:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 1 -- 172.21.15.52:0/659157551 wait complete. 2024-06-02T23:25:50.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.974+0000 7f9bbc780b80 20 client.19284 trim_cache size 0 max 16384 2024-06-02T23:25:50.981 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9b980a2c20 msgr2=0x7f9b980a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9b980a2c20 0x7f9b980a50e0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006010 tx=0x7f9b9c00b540 comp rx=0 tx=0).stop 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 msgr2=0x7f9bac074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 0x7f9bac074cd0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f9b9400c9b0 tx=0x7f9b9400cd70 comp rx=0 tx=0).stop 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 shutdown_connections 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9b980a2c20 0x7f9b980a50e0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9bac1085f0 0x7f9bac075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bac1024e0 0x7f9bac074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 --2- 172.21.15.52:0/821932579 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9bac102ea0 0x7f9bac074cd0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 >> 172.21.15.52:0/821932579 conn(0x7f9bac079640 msgr2=0x7f9bac106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 shutdown_connections 2024-06-02T23:25:50.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:50.975+0000 7f9bb1efd700 1 -- 172.21.15.52:0/821932579 wait complete. 2024-06-02T23:25:51.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 10.16 MB/sec warmup 59 sec latency 8023.701 ms 2024-06-02T23:25:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:51 smithi123 ceph-mon[35977]: pgmap v850: 97 pgs: 97 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-06-02T23:25:51.777 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:51 smithi052 ceph-mon[32340]: pgmap v850: 97 pgs: 97 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-06-02T23:25:51.808 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:25:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:51 smithi174 ceph-mon[35994]: pgmap v850: 97 pgs: 97 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 667 KiB/s rd, 2.8 MiB/s wr, 198 op/s 2024-06-02T23:25:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:52.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7837 9.99 MB/sec warmup 60 sec latency 9023.750 ms 2024-06-02T23:25:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 -- 172.21.15.52:0/258646403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece81030f0 msgr2=0x7fece8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 --2- 172.21.15.52:0/258646403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece81030f0 0x7fece8107110 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0099c0 tx=0x7fecdc009cd0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 -- 172.21.15.52:0/258646403 shutdown_connections 2024-06-02T23:25:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 --2- 172.21.15.52:0/258646403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8107840 0x7fece810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 --2- 172.21.15.52:0/258646403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece8102740 0x7fece8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 --2- 172.21.15.52:0/258646403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece81030f0 0x7fece8107110 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 -- 172.21.15.52:0/258646403 >> 172.21.15.52:0/258646403 conn(0x7fece80795f0 msgr2=0x7fece8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:53.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.320+0000 7feced10e700 1 -- 172.21.15.52:0/258646403 shutdown_connections 2024-06-02T23:25:53.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.321+0000 7feced10e700 1 -- 172.21.15.52:0/258646403 wait complete. 2024-06-02T23:25:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.321+0000 7feced10e700 1 Processor -- start 2024-06-02T23:25:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.321+0000 7feced10e700 1 -- start start 2024-06-02T23:25:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7feced10e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 0x7fece819e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7feced10e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece81030f0 0x7fece819eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7feced10e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8107840 0x7fece8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7feced10e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fece8112620 con 0x7fece8102740 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7feced10e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fece81124a0 con 0x7fece8107840 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7feced10e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fece81127a0 con 0x7fece81030f0 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 0x7fece819e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 0x7fece819e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40020/0 (socket says 172.21.15.52:40020) 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 -- 172.21.15.52:0/3973987689 learned_addr learned my addr 172.21.15.52:0/3973987689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece659c700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece81030f0 0x7fece819eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece759e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8107840 0x7fece8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 -- 172.21.15.52:0/3973987689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece81030f0 msgr2=0x7fece819eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece81030f0 0x7fece819eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 -- 172.21.15.52:0/3973987689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8107840 msgr2=0x7fece8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8107840 0x7fece8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 -- 172.21.15.52:0/3973987689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fece8199080 con 0x7fece8102740 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.322+0000 7fece6d9d700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 0x7fece819e760 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fecd800d980 tx=0x7fecd800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.323+0000 7feccffff700 1 -- 172.21.15.52:0/3973987689 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8017070 con 0x7fece8102740 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.323+0000 7feccffff700 1 -- 172.21.15.52:0/3973987689 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecd8005020 con 0x7fece8102740 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.323+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fece8199370 con 0x7fece8102740 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.323+0000 7feccffff700 1 -- 172.21.15.52:0/3973987689 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd800fa90 con 0x7fece8102740 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.323+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fece81a5630 con 0x7fece8102740 2024-06-02T23:25:53.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.325+0000 7feccffff700 1 -- 172.21.15.52:0/3973987689 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fecd80051d0 con 0x7fece8102740 2024-06-02T23:25:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.326+0000 7feccffff700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fecd00a2c20 0x7fecd00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.326+0000 7fece659c700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fecd00a2c20 0x7fecd00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.326+0000 7feccffff700 1 -- 172.21.15.52:0/3973987689 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fecd80ceeb0 con 0x7fece8102740 2024-06-02T23:25:53.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fece659c700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fecd00a2c20 0x7fecd00a50e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fece8102110 tx=0x7fecdc005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 Processor -- start 2024-06-02T23:25:53.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 -- start start 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b44020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 0x558ce5b3de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b3e3d0 0x558ce5b3e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558ce5a8f910 con 0x558ce5b44560 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ce5a37820 con 0x558ce5b3e3d0 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fecf7991b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ce5a8b8d0 con 0x558ce5b43c40 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fece659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 0x558ce5b3de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fece659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 0x558ce5b3de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40032/0 (socket says 172.21.15.52:40032) 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fece659c700 1 -- 172.21.15.52:0/1222059706 learned_addr learned my addr 172.21.15.52:0/1222059706 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fece759e700 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b3e3d0 0x558ce5b3e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.327+0000 7fece6d9d700 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b44020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7fece659c700 1 -- 172.21.15.52:0/1222059706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 msgr2=0x558ce5b44020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7fece659c700 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b44020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7fece659c700 1 -- 172.21.15.52:0/1222059706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b3e3d0 msgr2=0x558ce5b3e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7fece659c700 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b3e3d0 0x558ce5b3e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7fece659c700 1 -- 172.21.15.52:0/1222059706 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce5a2e050 con 0x558ce5b44560 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7fece659c700 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 0x558ce5b3de90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fecdc015000 tx=0x7fecdc020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddb9d2282e57a58 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7feccd7fa700 1 -- 172.21.15.52:0/1222059706 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc003db0 con 0x558ce5b44560 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7feccd7fa700 1 -- 172.21.15.52:0/1222059706 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecdc00bc40 con 0x558ce5b44560 2024-06-02T23:25:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.328+0000 7feccd7fa700 1 -- 172.21.15.52:0/1222059706 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecdc023900 con 0x558ce5b44560 2024-06-02T23:25:53.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 -- 172.21.15.52:0/1222059706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 msgr2=0x558ce5b3de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 0x558ce5b3de90 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fecdc015000 tx=0x7fecdc020000 comp rx=0 tx=0).stop 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 -- 172.21.15.52:0/1222059706 shutdown_connections 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b3e3d0 0x558ce5b3e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b44020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1222059706 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b44560 0x558ce5b3de90 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 -- 172.21.15.52:0/1222059706 >> 172.21.15.52:0/1222059706 conn(0x558ce5a68dc0 msgr2=0x558ce5b39120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 -- 172.21.15.52:0/1222059706 shutdown_connections 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 -- 172.21.15.52:0/1222059706 wait complete. 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 -- start start 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.329+0000 7fecf7991b80 1 Processor -- start 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b47560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b47aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558ce5b4af10 con 0x558ce5b3e3d0 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ce5a2fcf0 con 0x558ce5b44560 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ce5b4b090 con 0x558ce5b43c40 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fecf7991b80 10 client.0 _ll_register_callbacks cb 0x558ce5a29a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4a260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40040/0 (socket says 172.21.15.52:40040) 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 -- 172.21.15.52:0/1346218024 learned_addr learned my addr 172.21.15.52:0/1346218024 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece659c700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b47560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece759e700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b47aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 -- 172.21.15.52:0/1346218024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 msgr2=0x558ce5b47560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b47560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 -- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 msgr2=0x558ce5b47aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b47aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 -- 172.21.15.52:0/1346218024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce59de9d0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7fece6d9d700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4a260 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fecd80cca30 tx=0x7fecd8005690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb380fd44e6cd03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7feccaff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.330+0000 7feccaff5700 1 -- 172.21.15.52:0/1346218024 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8014030 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7feccaff5700 1 -- 172.21.15.52:0/1346218024 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fecd809b9a0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecdc00bc40 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7feccaff5700 1 -- 172.21.15.52:0/1346218024 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fecd80b9050 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 10 client.19296 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558ce5b557a0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7feccaff5700 1 -- 172.21.15.52:0/1346218024 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecdc00bc40 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7feccaff5700 1 -- 172.21.15.52:0/1346218024 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fecd80a3f20 con 0x558ce5b3e3d0 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 10 client.19296 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:53.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 10 client.19296 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 20 client.19296 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5b57d80 0x558ce5b5a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 4 client.19296 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce5a9ce10 con 0x558ce5b57d80 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.331+0000 7fece659c700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5b57d80 0x558ce5b5a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.332+0000 7fece659c700 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5b57d80 0x558ce5b5a1b0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68bf9e8255820b44 server_cookie=bbe659a4b07826a0 in_seq=0 out_seq=0 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.332+0000 7feccaff5700 10 client.19296 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7feccaff5700 1 -- 172.21.15.52:0/1346218024 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558ce5a9ce10 con 0x558ce5b57d80 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7feccaff5700 10 client.19296 handle_command_reply: tid=0 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 client.19296 shutdown 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 msgr2=0x558ce5b4a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4a260 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fecd80cca30 tx=0x7fecd8005690 comp rx=0 tx=0).stop 2024-06-02T23:25:53.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 shutdown_connections 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5b57d80 0x558ce5b5a1b0 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b47aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4a260 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 --2- 172.21.15.52:0/1346218024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b47560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7feccaff5700 0 client.19296 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 >> 172.21.15.52:0/1346218024 conn(0x558ce5a68dc0 msgr2=0x558ce5b3ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 shutdown_connections 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 1 -- 172.21.15.52:0/1346218024 wait complete. 2024-06-02T23:25:53.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.333+0000 7fecf7991b80 20 client.19296 trim_cache size 0 max 16384 2024-06-02T23:25:53.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 Processor -- start 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 -- start start 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b466e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b5fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b60000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fece81aca50 con 0x558ce5b3e3d0 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fece81ae6b0 con 0x558ce5b44560 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fecf7991b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fece81ad120 con 0x558ce5b43c40 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b466e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40042/0 (socket says 172.21.15.52:40042) 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.355+0000 7fece6d9d700 1 -- 172.21.15.52:0/586857786 learned_addr learned my addr 172.21.15.52:0/586857786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:53.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece759e700 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b60000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece659c700 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b5fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece6d9d700 1 -- 172.21.15.52:0/586857786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 msgr2=0x558ce5b5fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece6d9d700 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b5fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece6d9d700 1 -- 172.21.15.52:0/586857786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 msgr2=0x558ce5b60000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece6d9d700 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b60000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece6d9d700 1 -- 172.21.15.52:0/586857786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce5b45ad0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fece6d9d700 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b466e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fecd809b6d0 tx=0x7fecd80ba240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820ce2e32fd60f8f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fecccff9700 1 -- 172.21.15.52:0/586857786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8014070 con 0x558ce5b3e3d0 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fecccff9700 1 -- 172.21.15.52:0/586857786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecd8065020 con 0x558ce5b3e3d0 2024-06-02T23:25:53.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.356+0000 7fecccff9700 1 -- 172.21.15.52:0/586857786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd80a3f20 con 0x558ce5b3e3d0 2024-06-02T23:25:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 -- 172.21.15.52:0/586857786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 msgr2=0x558ce5b466e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b466e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fecd809b6d0 tx=0x7fecd80ba240 comp rx=0 tx=0).stop 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 -- 172.21.15.52:0/586857786 shutdown_connections 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b60000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b466e0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 --2- 172.21.15.52:0/586857786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b5fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 -- 172.21.15.52:0/586857786 >> 172.21.15.52:0/586857786 conn(0x558ce5a68dc0 msgr2=0x558ce5b44ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 -- 172.21.15.52:0/586857786 shutdown_connections 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 -- 172.21.15.52:0/586857786 wait complete. 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 -- start start 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.357+0000 7fecf7991b80 1 Processor -- start 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b4d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b4dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fece81adf20 con 0x558ce5b3e3d0 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fece81ad820 con 0x558ce5b44560 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fece81aaa10 con 0x558ce5b43c40 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fecf7991b80 10 client.0 _ll_register_callbacks cb 0x558ce5a29a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40050/0 (socket says 172.21.15.52:40050) 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 -- 172.21.15.52:0/4117028181 learned_addr learned my addr 172.21.15.52:0/4117028181 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece759e700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b4dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece659c700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b4d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 -- 172.21.15.52:0/4117028181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 msgr2=0x558ce5b4d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b4d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 -- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 msgr2=0x558ce5b4dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:53.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b4dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 -- 172.21.15.52:0/4117028181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce5bc2da0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece759e700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b4dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7fece6d9d700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4d400 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fecd8098110 tx=0x7fecd800fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c247926ac8b0d54 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.358+0000 7feccaff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7feccaff5700 1 -- 172.21.15.52:0/4117028181 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecd8014070 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7feccaff5700 1 -- 172.21.15.52:0/4117028181 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fecd80092a0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecd8065020 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7feccaff5700 1 -- 172.21.15.52:0/4117028181 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fecd80a43b0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 10 client.19302 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558ce5bc40e0 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7feccaff5700 1 -- 172.21.15.52:0/4117028181 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fecd8065020 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7feccaff5700 1 -- 172.21.15.52:0/4117028181 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fecd8004a30 con 0x558ce5b3e3d0 2024-06-02T23:25:53.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 10 client.19302 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 10 client.19302 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 20 client.19302 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5bc4530 0x558ce5bc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 4 client.19302 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "479d83de", "release": true, "await": true}] 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "479d83de", "release": true, "await": true}) -- 0x558ce5a97390 con 0x558ce5bc4530 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.359+0000 7fece659c700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5bc4530 0x558ce5bc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.360+0000 7feccaff5700 10 client.19302 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:53.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.360+0000 7fece659c700 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5bc4530 0x558ce5bc6920 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a2e11222415f7fb server_cookie=d048f08a882fcdac in_seq=0 out_seq=0 2024-06-02T23:25:53.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.397+0000 7feccaff5700 1 -- 172.21.15.52:0/4117028181 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x558ce5a97390 con 0x558ce5bc4530 2024-06-02T23:25:53.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7feccaff5700 10 client.19302 handle_command_reply: tid=0 2024-06-02T23:25:53.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 client.19302 shutdown 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 msgr2=0x558ce5b4d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4d400 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fecd8098110 tx=0x7fecd800fc30 comp rx=0 tx=0).stop 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 shutdown_connections 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558ce5bc4530 0x558ce5bc6920 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558ce5b44560 0x558ce5b4dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558ce5b3e3d0 0x558ce5b4d400 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 --2- 172.21.15.52:0/4117028181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce5b43c40 0x558ce5b4d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7feccaff5700 0 client.19302 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 >> 172.21.15.52:0/4117028181 conn(0x558ce5a68dc0 msgr2=0x558ce5b38830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 shutdown_connections 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 1 -- 172.21.15.52:0/4117028181 wait complete. 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.398+0000 7fecf7991b80 20 client.19302 trim_cache size 0 max 16384 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 37, 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 48, 2024-06-02T23:25:53.400 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "479d83de": { 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 48, 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 130.0, 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:25:53.401 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:25:53.402 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fecd00a2c20 msgr2=0x7fecd00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fecd00a2c20 0x7fecd00a50e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fece8102110 tx=0x7fecdc005fb0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 msgr2=0x7fece819e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 0x7fece819e760 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fecd800d980 tx=0x7fecd800dc90 comp rx=0 tx=0).stop 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 shutdown_connections 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fecd00a2c20 0x7fecd00a50e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fece8107840 0x7fece8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fece8102740 0x7fece819e760 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 --2- 172.21.15.52:0/3973987689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fece81030f0 0x7fece819eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.399+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 >> 172.21.15.52:0/3973987689 conn(0x7fece80795f0 msgr2=0x7fece8100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:53.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.400+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 shutdown_connections 2024-06-02T23:25:53.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:53.400+0000 7feced10e700 1 -- 172.21.15.52:0/3973987689 wait complete. 2024-06-02T23:25:53.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 7845 9.83 MB/sec warmup 61 sec latency 9957.974 ms 2024-06-02T23:25:53.810 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:53 smithi052 ceph-mon[32340]: pgmap v851: 97 pgs: 97 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-06-02T23:25:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:53 smithi174 ceph-mon[35994]: pgmap v851: 97 pgs: 97 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-06-02T23:25:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:53 smithi123 ceph-mon[35977]: pgmap v851: 97 pgs: 97 active+clean; 318 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 558 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-06-02T23:25:54.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 8018 9.76 MB/sec warmup 62 sec latency 100.076 ms 2024-06-02T23:25:54.731 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '479d83de', total seconds elapsed: 26.8 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 -- 172.21.15.52:0/3407877424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 msgr2=0x7f5ab0100fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 --2- 172.21.15.52:0/3407877424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab0100fd0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009990 tx=0x7f5aa0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 -- 172.21.15.52:0/3407877424 shutdown_connections 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 --2- 172.21.15.52:0/3407877424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0101510 0x7f5ab010d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 --2- 172.21.15.52:0/3407877424 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab0106c00 0x7f5ab0106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 --2- 172.21.15.52:0/3407877424 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab0100fd0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 -- 172.21.15.52:0/3407877424 >> 172.21.15.52:0/3407877424 conn(0x7f5ab0077990 msgr2=0x7f5ab00fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 -- 172.21.15.52:0/3407877424 shutdown_connections 2024-06-02T23:25:55.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.018+0000 7f5ab7433700 1 -- 172.21.15.52:0/3407877424 wait complete. 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.019+0000 7f5ab7433700 1 Processor -- start 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.019+0000 7f5ab7433700 1 -- start start 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.019+0000 7f5ab7433700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab019e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.019+0000 7f5ab7433700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab0101510 0x7f5ab019ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.019+0000 7f5ab7433700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0106c00 0x7f5ab01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab7433700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0112700 con 0x7f5ab0100b50 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab7433700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0112580 con 0x7f5ab0106c00 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab7433700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab0112880 con 0x7f5ab0101510 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab019e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab019e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53266/0 (socket says 172.21.15.52:53266) 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab51cf700 1 -- 172.21.15.52:0/1971083514 learned_addr learned my addr 172.21.15.52:0/1971083514 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0106c00 0x7f5ab01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab0101510 0x7f5ab019ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 -- 172.21.15.52:0/1971083514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab0101510 msgr2=0x7f5ab019ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab0101510 0x7f5ab019ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 -- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 msgr2=0x7f5ab019e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab019e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 -- 172.21.15.52:0/1971083514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ab0199100 con 0x7f5ab0106c00 2024-06-02T23:25:55.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.020+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0106c00 0x7f5ab01989a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800efc0 tx=0x7f5aa800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.021+0000 7f5aa67fc700 1 -- 172.21.15.52:0/1971083514 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80093a0 con 0x7f5ab0106c00 2024-06-02T23:25:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.021+0000 7f5aa67fc700 1 -- 172.21.15.52:0/1971083514 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aa800c940 con 0x7f5ab0106c00 2024-06-02T23:25:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.021+0000 7f5aa67fc700 1 -- 172.21.15.52:0/1971083514 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8004480 con 0x7f5ab0106c00 2024-06-02T23:25:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.021+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ab0199390 con 0x7f5ab0106c00 2024-06-02T23:25:55.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.021+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ab01a56b0 con 0x7f5ab0106c00 2024-06-02T23:25:55.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.024+0000 7f5aa67fc700 1 -- 172.21.15.52:0/1971083514 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5aa801b030 con 0x7f5ab0106c00 2024-06-02T23:25:55.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.025+0000 7f5aa67fc700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a9c0a2ca0 0x7f5a9c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.025+0000 7f5aa67fc700 1 -- 172.21.15.52:0/1971083514 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5aa80ce640 con 0x7f5ab0106c00 2024-06-02T23:25:55.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.025+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a9c0a2ca0 0x7f5a9c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 Processor -- start 2024-06-02T23:25:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 -- start start 2024-06-02T23:25:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4c640 0x55deb5a4ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a47340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a9c0a2ca0 0x7f5a9c0a5160 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0199ae0 tx=0x7f5aac009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ab49ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ab49ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53272/0 (socket says 172.21.15.52:53272) 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55deb5998810 con 0x55deb5a4cf60 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ab49ce700 1 -- 172.21.15.52:0/2035015560 learned_addr learned my addr 172.21.15.52:0/2035015560 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55deb5940720 con 0x55deb5a4c640 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55deb59947d0 con 0x55deb5a46f20 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a47340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.026+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4c640 0x55deb5a4ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5ab49ce700 1 -- 172.21.15.52:0/2035015560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 msgr2=0x55deb5a47340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a47340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5ab49ce700 1 -- 172.21.15.52:0/2035015560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4c640 msgr2=0x55deb5a4ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4c640 0x55deb5a4ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5ab49ce700 1 -- 172.21.15.52:0/2035015560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb5936f50 con 0x55deb5a4cf60 2024-06-02T23:25:55.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a469e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009990 tx=0x7f5aa0005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2437c27e00a55adf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:55.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5a9b7fe700 1 -- 172.21.15.52:0/2035015560 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0027070 con 0x55deb5a4cf60 2024-06-02T23:25:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5a9b7fe700 1 -- 172.21.15.52:0/2035015560 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aa00056a0 con 0x55deb5a4cf60 2024-06-02T23:25:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.027+0000 7f5a9b7fe700 1 -- 172.21.15.52:0/2035015560 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0003a20 con 0x55deb5a4cf60 2024-06-02T23:25:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 msgr2=0x55deb5a469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a469e0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f5aa0009990 tx=0x7f5aa0005bc0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 shutdown_connections 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a47340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4c640 0x55deb5a4ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/2035015560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a469e0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 >> 172.21.15.52:0/2035015560 conn(0x55deb5971cc0 msgr2=0x55deb5a41850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 shutdown_connections 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.028+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/2035015560 wait complete. 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.029+0000 7f5ac1cb6b80 1 -- start start 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.029+0000 7f5ac1cb6b80 1 Processor -- start 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a52c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4c640 0x55deb5a4ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a50520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a52c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55deb5a53970 con 0x55deb5a4cf60 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55deb5938bf0 con 0x55deb5a46f20 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a52c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38338/0 (socket says 172.21.15.52:38338) 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab51cf700 1 -- 172.21.15.52:0/3332091774 learned_addr learned my addr 172.21.15.52:0/3332091774 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4c640 0x55deb5a4ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55deb5a53af0 con 0x55deb5a4c640 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ac1cb6b80 10 client.0 _ll_register_callbacks cb 0x55deb5932920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab49ce700 1 -- 172.21.15.52:0/3332091774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 msgr2=0x55deb5a52c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a52c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a50520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab49ce700 1 -- 172.21.15.52:0/3332091774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 msgr2=0x55deb5a50520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a50520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab49ce700 1 -- 172.21.15.52:0/3332091774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb58e78d0 con 0x55deb5a4c640 2024-06-02T23:25:55.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.030+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a50520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4c640 0x55deb5a4ffe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00098e0 tx=0x7f5aa000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2183c4801da889d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5a98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5a98ff9700 1 -- 172.21.15.52:0/3332091774 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa0030020 con 0x55deb5a4c640 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aa00056a0 con 0x55deb5a4c640 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5a98ff9700 1 -- 172.21.15.52:0/3332091774 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5aa0003890 con 0x55deb5a4c640 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5a98ff9700 1 -- 172.21.15.52:0/3332091774 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5aa0027070 con 0x55deb5a4c640 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5ac1cb6b80 10 client.28396 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:55.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.031+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55deb5a53970 con 0x55deb5a4c640 2024-06-02T23:25:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5a98ff9700 1 -- 172.21.15.52:0/3332091774 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aa00056a0 con 0x55deb5a4c640 2024-06-02T23:25:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5a98ff9700 1 -- 172.21.15.52:0/3332091774 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f5aa0021070 con 0x55deb5a4c640 2024-06-02T23:25:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ac1cb6b80 10 client.28396 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ac1cb6b80 10 client.28396 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:25:55.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ac1cb6b80 20 client.28396 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5a60880 0x55deb5a62cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ac1cb6b80 4 client.28396 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55deb59a5d10 con 0x55deb5a60880 2024-06-02T23:25:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.032+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5a60880 0x55deb5a62cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.033+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5a60880 0x55deb5a62cc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e4e0b231a42693b7 server_cookie=678b8da457d38fdc in_seq=0 out_seq=0 2024-06-02T23:25:55.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.033+0000 7f5a98ff9700 10 client.28396 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:25:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5a98ff9700 1 -- 172.21.15.52:0/3332091774 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55deb59a5d10 con 0x55deb5a60880 2024-06-02T23:25:55.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5a98ff9700 10 client.28396 handle_command_reply: tid=0 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 client.28396 shutdown 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4c640 msgr2=0x55deb5a4ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4c640 0x55deb5a4ffe0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00098e0 tx=0x7f5aa000b390 comp rx=0 tx=0).stop 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 shutdown_connections 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4cf60 0x55deb5a50520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a52c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5a60880 0x55deb5a62cc0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/3332091774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4c640 0x55deb5a4ffe0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 >> 172.21.15.52:0/3332091774 conn(0x55deb5971cc0 msgr2=0x55deb5a459d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:55.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.034+0000 7f5a98ff9700 0 client.28396 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:25:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.035+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 shutdown_connections 2024-06-02T23:25:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.035+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/3332091774 wait complete. 2024-06-02T23:25:55.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.035+0000 7f5ac1cb6b80 20 client.28396 trim_cache size 0 max 16384 2024-06-02T23:25:55.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.059+0000 7f5ac1cb6b80 1 Processor -- start 2024-06-02T23:25:55.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.059+0000 7f5ac1cb6b80 1 -- start start 2024-06-02T23:25:55.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4cf60 0x55deb5a68aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01aec30 con 0x55deb5a4c640 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01b0890 con 0x55deb5a4cf60 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01af300 con 0x55deb5a46f20 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab49ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a469e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37052/0 (socket says 172.21.15.52:37052) 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 -- 172.21.15.52:0/4076386393 learned_addr learned my addr 172.21.15.52:0/4076386393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4cf60 0x55deb5a68aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 -- 172.21.15.52:0/4076386393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4cf60 msgr2=0x55deb5a68aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4cf60 0x55deb5a68aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 -- 172.21.15.52:0/4076386393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 msgr2=0x55deb5a68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a68560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab51cf700 1 -- 172.21.15.52:0/4076386393 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55deb5a4e550 con 0x55deb5a46f20 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.060+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.061+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a469e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0096d0 tx=0x7f5aac014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab579b075bacb28 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.061+0000 7f5a9affd700 1 -- 172.21.15.52:0/4076386393 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac021d60 con 0x55deb5a46f20 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.061+0000 7f5a9affd700 1 -- 172.21.15.52:0/4076386393 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aac01f680 con 0x55deb5a46f20 2024-06-02T23:25:55.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.061+0000 7f5a9affd700 1 -- 172.21.15.52:0/4076386393 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aac033640 con 0x55deb5a46f20 2024-06-02T23:25:55.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/4076386393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 msgr2=0x55deb5a469e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a469e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5aac0096d0 tx=0x7f5aac014000 comp rx=0 tx=0).stop 2024-06-02T23:25:55.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/4076386393 shutdown_connections 2024-06-02T23:25:55.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a4cf60 0x55deb5a68aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a46f20 0x55deb5a469e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/4076386393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/4076386393 >> 172.21.15.52:0/4076386393 conn(0x55deb5971cc0 msgr2=0x55deb5a41380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/4076386393 shutdown_connections 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.062+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/4076386393 wait complete. 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.063+0000 7f5ac1cb6b80 1 -- start start 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.063+0000 7f5ac1cb6b80 1 Processor -- start 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.063+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a56390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.063+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a568d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.063+0000 7f5ac1cb6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4cf60 0x55deb5ac9530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01b0100 con 0x55deb5a4c640 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01afa00 con 0x55deb5a46f20 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ac1cb6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ab01acbf0 con 0x55deb5a4cf60 2024-06-02T23:25:55.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab49ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4cf60 0x55deb5ac9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab51cf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a56390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab49ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a568d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53292/0 (socket says 172.21.15.52:53292) 2024-06-02T23:25:55.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab49ce700 1 -- 172.21.15.52:0/1159930026 learned_addr learned my addr 172.21.15.52:0/1159930026 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ac1cb6b80 10 client.0 _ll_register_callbacks cb 0x55deb5932920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 -- 172.21.15.52:0/1159930026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 msgr2=0x55deb5a56390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a56390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 -- 172.21.15.52:0/1159930026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 msgr2=0x55deb5a568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 -- 172.21.15.52:0/1159930026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55deb5ac9a70 con 0x55deb5a4cf60 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab49ce700 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.064+0000 7f5ab59d0700 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4cf60 0x55deb5ac9530 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8097360 tx=0x7f5aa8012620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97ce7cc80b8cd112 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5a98ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5a98ff9700 1 -- 172.21.15.52:0/1159930026 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa80b7070 con 0x55deb5a4cf60 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5aac01f680 con 0x55deb5a4cf60 2024-06-02T23:25:55.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5a98ff9700 1 -- 172.21.15.52:0/1159930026 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5aa80a3650 con 0x55deb5a4cf60 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5a98ff9700 1 -- 172.21.15.52:0/1159930026 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5aa8080050 con 0x55deb5a4cf60 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 10 client.28402 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55deb5acad60 con 0x55deb5a4cf60 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5a98ff9700 1 -- 172.21.15.52:0/1159930026 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5aac01f680 con 0x55deb5a4cf60 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5a98ff9700 1 -- 172.21.15.52:0/1159930026 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f5aa8096c50 con 0x55deb5a4cf60 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 10 client.28402 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 10 client.28402 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 20 client.28402 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5acb420 0x55deb5acd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 4 client.28402 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.065+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55deb59a0290 con 0x55deb5acb420 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.066+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5acb420 0x55deb5acd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.066+0000 7f5a98ff9700 10 client.28402 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:25:55.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.066+0000 7f5ab51cf700 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5acb420 0x55deb5acd840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=94e2ce3b191a3251 server_cookie=6528cf299fcfce81 in_seq=0 out_seq=0 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.066+0000 7f5a98ff9700 1 -- 172.21.15.52:0/1159930026 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55deb59a0290 con 0x55deb5acb420 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.066+0000 7f5a98ff9700 10 client.28402 handle_command_reply: tid=0 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.066+0000 7f5ac1cb6b80 1 client.28402 shutdown 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4cf60 msgr2=0x55deb5ac9530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4cf60 0x55deb5ac9530 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8097360 tx=0x7f5aa8012620 comp rx=0 tx=0).stop 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 shutdown_connections 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55deb5acb420 0x55deb5acd840 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55deb5a4cf60 0x55deb5ac9530 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5a98ff9700 0 client.28402 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55deb5a46f20 0x55deb5a56390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 --2- 172.21.15.52:0/1159930026 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55deb5a4c640 0x55deb5a568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 >> 172.21.15.52:0/1159930026 conn(0x55deb5971cc0 msgr2=0x55deb5a3ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:55.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 shutdown_connections 2024-06-02T23:25:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 1 -- 172.21.15.52:0/1159930026 wait complete. 2024-06-02T23:25:55.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.067+0000 7f5ac1cb6b80 20 client.28402 trim_cache size 0 max 16384 2024-06-02T23:25:55.071 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a9c0a2ca0 msgr2=0x7f5a9c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a9c0a2ca0 0x7f5a9c0a5160 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0199ae0 tx=0x7f5aac009260 comp rx=0 tx=0).stop 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0106c00 msgr2=0x7f5ab01989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0106c00 0x7f5ab01989a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800efc0 tx=0x7f5aa800c3a0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 shutdown_connections 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5a9c0a2ca0 0x7f5a9c0a5160 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ab0106c00 0x7f5ab01989a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ab0100b50 0x7f5ab019e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 --2- 172.21.15.52:0/1971083514 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ab0101510 0x7f5ab019ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:55.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 >> 172.21.15.52:0/1971083514 conn(0x7f5ab0077990 msgr2=0x7f5ab00ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 shutdown_connections 2024-06-02T23:25:55.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:55.068+0000 7f5ab7433700 1 -- 172.21.15.52:0/1971083514 wait complete. 2024-06-02T23:25:55.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 8184 9.70 MB/sec warmup 63 sec latency 59.029 ms 2024-06-02T23:25:55.799 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:55 smithi052 ceph-mon[32340]: pgmap v852: 97 pgs: 97 active+clean; 319 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-06-02T23:25:55.843 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:25:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:55 smithi174 ceph-mon[35994]: pgmap v852: 97 pgs: 97 active+clean; 319 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-06-02T23:25:56.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:55 smithi123 ceph-mon[35977]: pgmap v852: 97 pgs: 97 active+clean; 319 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 331 KiB/s rd, 1.3 MiB/s wr, 120 op/s 2024-06-02T23:25:56.229 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:56.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 8320 9.64 MB/sec warmup 64 sec latency 97.794 ms 2024-06-02T23:25:56.620 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:56 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:25:56.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:56 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:25:57.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:56 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:25:57.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 8487 9.62 MB/sec warmup 65 sec latency 104.977 ms 2024-06-02T23:25:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: pgmap v853: 97 pgs: 97 active+clean; 320 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:25:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: pgmap v853: 97 pgs: 97 active+clean; 320 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-02T23:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:25:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:25:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:25:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:25:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:25:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: pgmap v853: 97 pgs: 97 active+clean; 320 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 1.3 MiB/s wr, 139 op/s 2024-06-02T23:25:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:25:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:25:58.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 8670 9.74 MB/sec warmup 66 sec latency 207.780 ms 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.595+0000 7f8d782e0700 1 -- 172.21.15.52:0/53238718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70103150 msgr2=0x7f8d70107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.595+0000 7f8d782e0700 1 --2- 172.21.15.52:0/53238718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70103150 0x7f8d70107140 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f8d600099c0 tx=0x7f8d60009cd0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.595+0000 7f8d782e0700 1 -- 172.21.15.52:0/53238718 shutdown_connections 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.595+0000 7f8d782e0700 1 --2- 172.21.15.52:0/53238718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d70107870 0x7f8d7010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.595+0000 7f8d782e0700 1 --2- 172.21.15.52:0/53238718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d701027a0 0x7f8d70102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.595+0000 7f8d782e0700 1 --2- 172.21.15.52:0/53238718 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70103150 0x7f8d70107140 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.596+0000 7f8d782e0700 1 -- 172.21.15.52:0/53238718 >> 172.21.15.52:0/53238718 conn(0x7f8d70079570 msgr2=0x7f8d70079980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:58.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.596+0000 7f8d782e0700 1 -- 172.21.15.52:0/53238718 shutdown_connections 2024-06-02T23:25:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.597+0000 7f8d782e0700 1 -- 172.21.15.52:0/53238718 wait complete. 2024-06-02T23:25:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.597+0000 7f8d782e0700 1 Processor -- start 2024-06-02T23:25:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 -- start start 2024-06-02T23:25:58.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d701027a0 0x7f8d7019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d70103150 0x7f8d7019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70107870 0x7f8d70198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d701027a0 0x7f8d7019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d701027a0 0x7f8d7019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38382/0 (socket says 172.21.15.52:38382) 2024-06-02T23:25:58.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d7607c700 1 -- 172.21.15.52:0/4009765610 learned_addr learned my addr 172.21.15.52:0/4009765610 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:58.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d7587b700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d70103150 0x7f8d7019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8d70112650 con 0x7f8d70107870 2024-06-02T23:25:58.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701124d0 con 0x7f8d701027a0 2024-06-02T23:25:58.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701127d0 con 0x7f8d70103150 2024-06-02T23:25:58.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.598+0000 7f8d7687d700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70107870 0x7f8d70198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7687d700 1 -- 172.21.15.52:0/4009765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d70103150 msgr2=0x7f8d7019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7687d700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d70103150 0x7f8d7019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7687d700 1 -- 172.21.15.52:0/4009765610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d701027a0 msgr2=0x7f8d7019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7687d700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d701027a0 0x7f8d7019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7687d700 1 -- 172.21.15.52:0/4009765610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d70199070 con 0x7f8d70107870 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7587b700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d70103150 0x7f8d7019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.599+0000 7f8d7687d700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70107870 0x7f8d70198910 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c3c0 tx=0x7f8d6800c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.600+0000 7f8d677fe700 1 -- 172.21.15.52:0/4009765610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004bb0 con 0x7f8d70107870 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.600+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d70199300 con 0x7f8d70107870 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.600+0000 7f8d677fe700 1 -- 172.21.15.52:0/4009765610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d68004d50 con 0x7f8d70107870 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.600+0000 7f8d677fe700 1 -- 172.21.15.52:0/4009765610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d68004440 con 0x7f8d70107870 2024-06-02T23:25:58.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.600+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d700746c0 con 0x7f8d70107870 2024-06-02T23:25:58.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.604+0000 7f8d677fe700 1 -- 172.21.15.52:0/4009765610 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8d68021020 con 0x7f8d70107870 2024-06-02T23:25:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.605+0000 7f8d677fe700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d5c0a2ca0 0x7f8d5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.605+0000 7f8d677fe700 1 -- 172.21.15.52:0/4009765610 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8d680d26e0 con 0x7f8d70107870 2024-06-02T23:25:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.605+0000 7f8d7607c700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d5c0a2ca0 0x7f8d5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.605+0000 7f8d82b63b80 1 Processor -- start 2024-06-02T23:25:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.605+0000 7f8d82b63b80 1 -- start start 2024-06-02T23:25:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5630a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630f60 0x562de562a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de562af20 0x562de562b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562de557c810 con 0x562de5630f60 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562de5524720 con 0x562de5630640 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562de55787d0 con 0x562de562af20 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7607c700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d5c0a2ca0 0x7f8d5c0a5160 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8d70107320 tx=0x7f8d6c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5630a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5630a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38384/0 (socket says 172.21.15.52:38384) 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7607c700 1 -- 172.21.15.52:0/1817952039 learned_addr learned my addr 172.21.15.52:0/1817952039 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7587b700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630f60 0x562de562a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7687d700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de562af20 0x562de562b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7587b700 1 -- 172.21.15.52:0/1817952039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de562af20 msgr2=0x562de562b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7587b700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de562af20 0x562de562b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7587b700 1 -- 172.21.15.52:0/1817952039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 msgr2=0x562de5630a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7587b700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5630a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7587b700 1 -- 172.21.15.52:0/1817952039 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de551af50 con 0x562de5630f60 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.606+0000 7f8d7687d700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de562af20 0x562de562b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.607+0000 7f8d7607c700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5630a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.607+0000 7f8d7587b700 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630f60 0x562de562a9e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009990 tx=0x7f8d600036e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ffb3380c9f94f57 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:58.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.607+0000 7f8d64d70700 1 -- 172.21.15.52:0/1817952039 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60027070 con 0x562de5630f60 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.607+0000 7f8d64d70700 1 -- 172.21.15.52:0/1817952039 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d6000bcf0 con 0x562de5630f60 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d64d70700 1 -- 172.21.15.52:0/1817952039 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60003c20 con 0x562de5630f60 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1817952039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630f60 msgr2=0x562de562a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630f60 0x562de562a9e0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009990 tx=0x7f8d600036e0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1817952039 shutdown_connections 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de562af20 0x562de562b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5630a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1817952039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630f60 0x562de562a9e0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1817952039 >> 172.21.15.52:0/1817952039 conn(0x562de5555cc0 msgr2=0x562de5625850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1817952039 shutdown_connections 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.608+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1817952039 wait complete. 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 -- start start 2024-06-02T23:25:58.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 Processor -- start 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de562af20 0x562de5636c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5633f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5634480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de562af20 0x562de5636c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562de56379c0 con 0x562de562af20 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562de551cbf0 con 0x562de5630640 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de562af20 0x562de5636c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53346/0 (socket says 172.21.15.52:53346) 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562de5637b40 con 0x562de5630f60 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d7607c700 1 -- 172.21.15.52:0/4073554539 learned_addr learned my addr 172.21.15.52:0/4073554539 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:58.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d7587b700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5633f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.609+0000 7f8d82b63b80 10 client.0 _ll_register_callbacks cb 0x562de5516920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7687d700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5634480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7587b700 1 -- 172.21.15.52:0/4073554539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 msgr2=0x562de5634480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7587b700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5634480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7587b700 1 -- 172.21.15.52:0/4073554539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de562af20 msgr2=0x562de5636c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7587b700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de562af20 0x562de5636c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7587b700 1 -- 172.21.15.52:0/4073554539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de54cb8d0 con 0x562de5630640 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d7587b700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5633f40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009990 tx=0x7f8d6000faf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c433e36ac4629607 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d59ffb700 1 -- 172.21.15.52:0/4073554539 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60027070 con 0x562de5630640 2024-06-02T23:25:58.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d59ffb700 1 -- 172.21.15.52:0/4073554539 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8d60003900 con 0x562de5630640 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.610+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6000bcf0 con 0x562de5630640 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.611+0000 7f8d59ffb700 1 -- 172.21.15.52:0/4073554539 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8d6000bea0 con 0x562de5630640 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.611+0000 7f8d82b63b80 10 client.28736 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.611+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562de564a960 con 0x562de5630640 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.611+0000 7f8d59ffb700 1 -- 172.21.15.52:0/4073554539 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d6000bcf0 con 0x562de5630640 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d59ffb700 1 -- 172.21.15.52:0/4073554539 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f8d6002c400 con 0x562de5630640 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d82b63b80 10 client.28736 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d82b63b80 10 client.28736 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d82b63b80 20 client.28736 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de564ce30 0x562de564f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d82b63b80 4 client.28736 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562de5589d10 con 0x562de564ce30 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d7607c700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de564ce30 0x562de564f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d59ffb700 10 client.28736 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:25:58.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.612+0000 7f8d7607c700 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de564ce30 0x562de564f220 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5efc7306a3d73d0e server_cookie=eae990c0ec9a5de3 in_seq=0 out_seq=0 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.613+0000 7f8d59ffb700 1 -- 172.21.15.52:0/4073554539 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562de5589d10 con 0x562de564ce30 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.613+0000 7f8d59ffb700 10 client.28736 handle_command_reply: tid=0 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.613+0000 7f8d82b63b80 1 client.28736 shutdown 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 msgr2=0x562de5633f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5633f40 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8d60009990 tx=0x7f8d6000faf0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 shutdown_connections 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de564ce30 0x562de564f220 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5634480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d59ffb700 0 client.28736 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de562af20 0x562de5636c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/4073554539 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de5630640 0x562de5633f40 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 >> 172.21.15.52:0/4073554539 conn(0x562de5555cc0 msgr2=0x562de56299d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 shutdown_connections 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 1 -- 172.21.15.52:0/4073554539 wait complete. 2024-06-02T23:25:58.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.614+0000 7f8d82b63b80 20 client.28736 trim_cache size 0 max 16384 2024-06-02T23:25:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.633+0000 7f8d82b63b80 1 Processor -- start 2024-06-02T23:25:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 -- start start 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de562a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de5632830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5632d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701aa820 con 0x562de5630640 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701ac480 con 0x562de562af20 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701aaef0 con 0x562de5630f60 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de562a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d7587b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de5632830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de562a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38410/0 (socket says 172.21.15.52:38410) 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d7687d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5632d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d7607c700 1 -- 172.21.15.52:0/1258543995 learned_addr learned my addr 172.21.15.52:0/1258543995 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.634+0000 7f8d7687d700 1 -- 172.21.15.52:0/1258543995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 msgr2=0x562de562a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d7687d700 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de562a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d7687d700 1 -- 172.21.15.52:0/1258543995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 msgr2=0x562de5632830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d7687d700 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de5632830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d7687d700 1 -- 172.21.15.52:0/1258543995 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562de5637bb0 con 0x562de5630f60 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d7587b700 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de5632830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d7687d700 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5632d70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d680d01b0 tx=0x7f8d6809ee00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62177eb160a0df06 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d5bfff700 1 -- 172.21.15.52:0/1258543995 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6801e040 con 0x562de5630f60 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d5bfff700 1 -- 172.21.15.52:0/1258543995 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d6809f610 con 0x562de5630f60 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.635+0000 7f8d5bfff700 1 -- 172.21.15.52:0/1258543995 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d6809f3d0 con 0x562de5630f60 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1258543995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 msgr2=0x562de5632d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5632d70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8d680d01b0 tx=0x7f8d6809ee00 comp rx=0 tx=0).stop 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1258543995 shutdown_connections 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de5632d70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de562a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/1258543995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de5632830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1258543995 >> 172.21.15.52:0/1258543995 conn(0x562de5555cc0 msgr2=0x562de5625380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1258543995 shutdown_connections 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.636+0000 7f8d82b63b80 1 -- 172.21.15.52:0/1258543995 wait complete. 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.637+0000 7f8d82b63b80 1 -- start start 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.637+0000 7f8d82b63b80 1 Processor -- start 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.637+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de564efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.637+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de564f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d82b63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de56b5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701abcf0 con 0x562de5630640 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701ab5f0 con 0x562de562af20 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d82b63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d701a87e0 con 0x562de5630f60 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de564efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7607c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de564efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38422/0 (socket says 172.21.15.52:38422) 2024-06-02T23:25:58.640 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7607c700 1 -- 172.21.15.52:0/81929486 learned_addr learned my addr 172.21.15.52:0/81929486 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d82b63b80 10 client.0 _ll_register_callbacks cb 0x562de5516920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de564f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7687d700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de56b5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 -- 172.21.15.52:0/81929486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 msgr2=0x562de56b5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de56b5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 -- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 msgr2=0x562de564efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de564efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 -- 172.21.15.52:0/81929486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562de56b5ca0 con 0x562de5630640 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7687d700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de56b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7607c700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de564efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d7587b700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de564f500 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098e0 tx=0x7f8d60009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd741e6a2f252067 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.638+0000 7f8d59ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:25:58.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d59ffb700 1 -- 172.21.15.52:0/81929486 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d60027040 con 0x562de5630640 2024-06-02T23:25:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d59ffb700 1 -- 172.21.15.52:0/81929486 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8d60003ce0 con 0x562de5630640 2024-06-02T23:25:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d6809f610 con 0x562de5630640 2024-06-02T23:25:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d59ffb700 1 -- 172.21.15.52:0/81929486 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f8d6000bcf0 con 0x562de5630640 2024-06-02T23:25:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 10 client.19335 fetch_fsmap learned FSMap version 37 2024-06-02T23:25:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x562de56b6dc0 con 0x562de5630640 2024-06-02T23:25:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d59ffb700 1 -- 172.21.15.52:0/81929486 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8d6809f610 con 0x562de5630640 2024-06-02T23:25:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d59ffb700 1 -- 172.21.15.52:0/81929486 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f8d60022400 con 0x562de5630640 2024-06-02T23:25:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 10 client.19335 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:25:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 10 client.19335 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:25:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 20 client.19335 populate_metadata read hostname 'smithi052' 2024-06-02T23:25:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de56b7400 0x562de56b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:25:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 4 client.19335 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:25:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.639+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562de5584290 con 0x562de56b7400 2024-06-02T23:25:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.640+0000 7f8d7607c700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de56b7400 0x562de56b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:25:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.640+0000 7f8d7607c700 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de56b7400 0x562de56b97f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2d41d5f94586f6b2 server_cookie=aca6d9f8c9c7e99e in_seq=0 out_seq=0 2024-06-02T23:25:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.640+0000 7f8d59ffb700 10 client.19335 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:25:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.640+0000 7f8d59ffb700 1 -- 172.21.15.52:0/81929486 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562de5584290 con 0x562de56b7400 2024-06-02T23:25:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.640+0000 7f8d59ffb700 10 client.19335 handle_command_reply: tid=0 2024-06-02T23:25:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.640+0000 7f8d82b63b80 1 client.19335 shutdown 2024-06-02T23:25:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 msgr2=0x562de564f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de564f500 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f8d600098e0 tx=0x7f8d60009cd0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 shutdown_connections 2024-06-02T23:25:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x562de56b7400 0x562de56b97f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562de5630f60 0x562de56b5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562de562af20 0x562de564efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 --2- 172.21.15.52:0/81929486 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562de5630640 0x562de564f500 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 >> 172.21.15.52:0/81929486 conn(0x562de5555cc0 msgr2=0x562de5631340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:58.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d59ffb700 0 client.19335 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:25:58.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 shutdown_connections 2024-06-02T23:25:58.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 1 -- 172.21.15.52:0/81929486 wait complete. 2024-06-02T23:25:58.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.641+0000 7f8d82b63b80 20 client.19335 trim_cache size 0 max 16384 2024-06-02T23:25:58.650 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:25:58.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d5c0a2ca0 msgr2=0x7f8d5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d5c0a2ca0 0x7f8d5c0a5160 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8d70107320 tx=0x7f8d6c009260 comp rx=0 tx=0).stop 2024-06-02T23:25:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70107870 msgr2=0x7f8d70198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:25:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70107870 0x7f8d70198910 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f8d6800c3c0 tx=0x7f8d6800c6d0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 shutdown_connections 2024-06-02T23:25:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8d5c0a2ca0 0x7f8d5c0a5160 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8d70107870 0x7f8d70198910 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d701027a0 0x7f8d7019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 --2- 172.21.15.52:0/4009765610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d70103150 0x7f8d7019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:25:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 >> 172.21.15.52:0/4009765610 conn(0x7f8d70079570 msgr2=0x7f8d701007f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:25:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 shutdown_connections 2024-06-02T23:25:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:25:58.642+0000 7f8d782e0700 1 -- 172.21.15.52:0/4009765610 wait complete. 2024-06-02T23:25:59.401 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:25:59.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 8841 9.63 MB/sec warmup 67 sec latency 95.316 ms 2024-06-02T23:25:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:25:59 smithi052 ceph-mon[32340]: pgmap v854: 97 pgs: 97 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 668 KiB/s wr, 78 op/s 2024-06-02T23:25:59.771 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:25:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:25:59 smithi174 ceph-mon[35994]: pgmap v854: 97 pgs: 97 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 668 KiB/s wr, 78 op/s 2024-06-02T23:26:00.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:25:59 smithi123 ceph-mon[35977]: pgmap v854: 97 pgs: 97 active+clean; 316 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 668 KiB/s wr, 78 op/s 2024-06-02T23:26:00.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 9021 9.60 MB/sec warmup 68 sec latency 71.679 ms 2024-06-02T23:26:01.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 9175 9.49 MB/sec warmup 69 sec latency 54.322 ms 2024-06-02T23:26:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 -- 172.21.15.52:0/926889190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 msgr2=0x7fae8c06ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 --2- 172.21.15.52:0/926889190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c06ae10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fae7c0098f0 tx=0x7fae7c009c00 comp rx=0 tx=0).stop 2024-06-02T23:26:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 -- 172.21.15.52:0/926889190 shutdown_connections 2024-06-02T23:26:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 --2- 172.21.15.52:0/926889190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 0x7fae8c111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 --2- 172.21.15.52:0/926889190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c06ae10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 --2- 172.21.15.52:0/926889190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 -- 172.21.15.52:0/926889190 >> 172.21.15.52:0/926889190 conn(0x7fae8c078300 msgr2=0x7fae8c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 -- 172.21.15.52:0/926889190 shutdown_connections 2024-06-02T23:26:01.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.667+0000 7fae917c7700 1 -- 172.21.15.52:0/926889190 wait complete. 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.668+0000 7fae917c7700 1 Processor -- start 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.668+0000 7fae917c7700 1 -- start start 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae917c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae917c7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c19b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae917c7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 0x7fae8c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c19b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c19b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53378/0 (socket says 172.21.15.52:53378) 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae8affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c19ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38438/0 (socket says 172.21.15.52:38438) 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 0x7fae8c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae917c7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c114890 con 0x7fae8c06b350 2024-06-02T23:26:01.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c114710 con 0x7fae8c06aa30 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae8a7fc700 1 -- 172.21.15.52:0/485705370 learned_addr learned my addr 172.21.15.52:0/485705370 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.669+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c114a10 con 0x7fae8c10ef30 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8b7fe700 1 -- 172.21.15.52:0/485705370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 msgr2=0x7fae8c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8b7fe700 1 -- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 msgr2=0x7fae8c19b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c19b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8b7fe700 1 -- 172.21.15.52:0/485705370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c19fe30 con 0x7fae8c10ef30 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8a7fc700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 0x7fae8c19f6f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae8c069f30 tx=0x7fae8400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae83fff700 1 -- 172.21.15.52:0/485705370 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8400ca50 con 0x7fae8c10ef30 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae83fff700 1 -- 172.21.15.52:0/485705370 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae8400cbf0 con 0x7fae8c10ef30 2024-06-02T23:26:01.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae83fff700 1 -- 172.21.15.52:0/485705370 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840047f0 con 0x7fae8c10ef30 2024-06-02T23:26:01.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c1a0120 con 0x7fae8c10ef30 2024-06-02T23:26:01.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.670+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c1a79a0 con 0x7fae8c10ef30 2024-06-02T23:26:01.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.673+0000 7fae83fff700 1 -- 172.21.15.52:0/485705370 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fae84012050 con 0x7fae8c10ef30 2024-06-02T23:26:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.674+0000 7fae83fff700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae780a2ca0 0x7fae780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.674+0000 7fae8affd700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae780a2ca0 0x7fae780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.674+0000 7fae83fff700 1 -- 172.21.15.52:0/485705370 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fae840ce870 con 0x7fae8c10ef30 2024-06-02T23:26:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.675+0000 7fae8affd700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae780a2ca0 0x7fae780a5160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009680 tx=0x7fae7c019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:01.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.675+0000 7fae9c04ab80 1 Processor -- start 2024-06-02T23:26:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.675+0000 7fae9c04ab80 1 -- start start 2024-06-02T23:26:01.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e3ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e39340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564595d8a810 con 0x564595e3ef60 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564595d32720 con 0x564595e3e640 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564595d867d0 con 0x564595e38f20 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e39340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e39340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37144/0 (socket says 172.21.15.52:37144) 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2407231637 learned_addr learned my addr 172.21.15.52:0/2407231637 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:01.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.676+0000 7fae8affd700 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e3ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2407231637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 msgr2=0x564595e3ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e3ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2407231637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 msgr2=0x564595e389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2407231637 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564595d28f50 con 0x564595e38f20 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8a7fc700 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:01.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e39340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae84097b60 tx=0x7fae840974b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21b49e892b569479 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae817fa700 1 -- 172.21.15.52:0/2407231637 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8409b890 con 0x564595e38f20 2024-06-02T23:26:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae817fa700 1 -- 172.21.15.52:0/2407231637 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae8409b190 con 0x564595e38f20 2024-06-02T23:26:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.677+0000 7fae817fa700 1 -- 172.21.15.52:0/2407231637 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840bb0d0 con 0x564595e38f20 2024-06-02T23:26:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2407231637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 msgr2=0x564595e39340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e39340 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fae84097b60 tx=0x7fae840974b0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2407231637 shutdown_connections 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e39340 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e3ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2407231637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2407231637 >> 172.21.15.52:0/2407231637 conn(0x564595d63cc0 msgr2=0x564595e33850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2407231637 shutdown_connections 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.678+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2407231637 wait complete. 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 -- start start 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 Processor -- start 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e44cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e41ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e42530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564595e45970 con 0x564595e38f20 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564595d2abf0 con 0x564595e3ef60 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564595e45af0 con 0x564595e3e640 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae9c04ab80 10 client.0 _ll_register_callbacks cb 0x564595d24920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e41ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e42530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e41ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37152/0 (socket says 172.21.15.52:37152) 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae8a7fc700 1 -- 172.21.15.52:0/2495916363 learned_addr learned my addr 172.21.15.52:0/2495916363 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.679+0000 7fae8affd700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e44cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 msgr2=0x564595e41ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e41ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2495916363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 msgr2=0x564595e44cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e44cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8b7fe700 1 -- 172.21.15.52:0/2495916363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564595cd98d0 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8a7fc700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e41ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e42530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fae840cc3f0 tx=0x7fae8400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7abab08e8251bad3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae727fc700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae727fc700 1 -- 172.21.15.52:0/2495916363 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae8409b890 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae727fc700 1 -- 172.21.15.52:0/2495916363 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fae84012dd0 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.680+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8409b190 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae727fc700 1 -- 172.21.15.52:0/2495916363 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fae840bb0d0 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 10 client.28754 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564595e50270 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae727fc700 1 -- 172.21.15.52:0/2495916363 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae840843e0 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae727fc700 1 -- 172.21.15.52:0/2495916363 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fae84004bd0 con 0x564595e3ef60 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 10 client.28754 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:01.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 10 client.28754 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:26:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 20 client.28754 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595e52740 0x564595e54b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 4 client.28754 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fae8409b190 con 0x564595e52740 2024-06-02T23:26:01.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.681+0000 7fae8affd700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595e52740 0x564595e54b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.682+0000 7fae8affd700 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595e52740 0x564595e54b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a2295a12090df2bb server_cookie=a6f6035c26d7c50b in_seq=0 out_seq=0 2024-06-02T23:26:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.682+0000 7fae727fc700 10 client.28754 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:26:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae727fc700 1 -- 172.21.15.52:0/2495916363 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fae8409b190 con 0x564595e52740 2024-06-02T23:26:01.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae727fc700 10 client.28754 handle_command_reply: tid=0 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 client.28754 shutdown 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 msgr2=0x564595e42530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e42530 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fae840cc3f0 tx=0x7fae8400c850 comp rx=0 tx=0).stop 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 shutdown_connections 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595e52740 0x564595e54b30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae727fc700 0 client.28754 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e42530 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e44cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/2495916363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e41ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 >> 172.21.15.52:0/2495916363 conn(0x564595d63cc0 msgr2=0x564595e379d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 shutdown_connections 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 1 -- 172.21.15.52:0/2495916363 wait complete. 2024-06-02T23:26:01.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.683+0000 7fae9c04ab80 20 client.28754 trim_cache size 0 max 16384 2024-06-02T23:26:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.706+0000 7fae9c04ab80 1 Processor -- start 2024-06-02T23:26:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 -- start start 2024-06-02T23:26:01.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e426c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e44cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1aee10 con 0x564595e38f20 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1b0a70 con 0x564595e3ef60 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1af4e0 con 0x564595e3e640 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e44cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e426c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e44cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38466/0 (socket says 172.21.15.52:38466) 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e389e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53422/0 (socket says 172.21.15.52:53422) 2024-06-02T23:26:01.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8b7fe700 1 -- 172.21.15.52:0/220844169 learned_addr learned my addr 172.21.15.52:0/220844169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8b7fe700 1 -- 172.21.15.52:0/220844169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 msgr2=0x564595e426c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.707+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e426c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae8b7fe700 1 -- 172.21.15.52:0/220844169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 msgr2=0x564595e389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae8b7fe700 1 -- 172.21.15.52:0/220844169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564595e39300 con 0x564595e3ef60 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e44cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fae84099440 tx=0x7fae8400efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=767ad46a9a35a80f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae80ff9700 1 -- 172.21.15.52:0/220844169 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840a7070 con 0x564595e3ef60 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae80ff9700 1 -- 172.21.15.52:0/220844169 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae8409b890 con 0x564595e3ef60 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.708+0000 7fae80ff9700 1 -- 172.21.15.52:0/220844169 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae840a3d70 con 0x564595e3ef60 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 -- 172.21.15.52:0/220844169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 msgr2=0x564595e44cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e44cf0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fae84099440 tx=0x7fae8400efb0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 -- 172.21.15.52:0/220844169 shutdown_connections 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3ef60 0x564595e44cf0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e38f20 0x564595e389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/220844169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e3e640 0x564595e426c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 -- 172.21.15.52:0/220844169 >> 172.21.15.52:0/220844169 conn(0x564595d63cc0 msgr2=0x564595e41fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 -- 172.21.15.52:0/220844169 shutdown_connections 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.709+0000 7fae9c04ab80 1 -- 172.21.15.52:0/220844169 wait complete. 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 -- start start 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 Processor -- start 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e4ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e4b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e590f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1b02e0 con 0x564595e3ef60 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1afbe0 con 0x564595e3e640 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c1acdb0 con 0x564595e38f20 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae8affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e4ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae8affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e4ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37168/0 (socket says 172.21.15.52:37168) 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae8a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e4b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae8affd700 1 -- 172.21.15.52:0/1442446448 learned_addr learned my addr 172.21.15.52:0/1442446448 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.710+0000 7fae9c04ab80 10 client.0 _ll_register_callbacks cb 0x564595d24920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8affd700 1 -- 172.21.15.52:0/1442446448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 msgr2=0x564595e4b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8affd700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e4b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8affd700 1 -- 172.21.15.52:0/1442446448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 msgr2=0x564595e590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8affd700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8affd700 1 -- 172.21.15.52:0/1442446448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564595e4b970 con 0x564595e38f20 2024-06-02T23:26:01.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8b7fe700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:01.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8a7fc700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:01.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae8affd700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e4ad80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00feb0 tx=0x7fae7c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d667ca9f16b66f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:01.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae727fc700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae727fc700 1 -- 172.21.15.52:0/1442446448 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c005510 con 0x564595e38f20 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae727fc700 1 -- 172.21.15.52:0/1442446448 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fae7c0046d0 con 0x564595e38f20 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae8409b890 con 0x564595e38f20 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.711+0000 7fae727fc700 1 -- 172.21.15.52:0/1442446448 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fae7c0056b0 con 0x564595e38f20 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 10 client.28435 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x564595ebc640 con 0x564595e38f20 2024-06-02T23:26:01.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae727fc700 1 -- 172.21.15.52:0/1442446448 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fae8409b890 con 0x564595e38f20 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae727fc700 1 -- 172.21.15.52:0/1442446448 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fae7c038070 con 0x564595e38f20 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 10 client.28435 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 10 client.28435 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 20 client.28435 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595ebcdd0 0x564595ebf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 4 client.28435 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.712+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564595d97d10 con 0x564595ebcdd0 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.713+0000 7fae8a7fc700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595ebcdd0 0x564595ebf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.713+0000 7fae8a7fc700 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595ebcdd0 0x564595ebf1f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d498daf8ac3fdfe4 server_cookie=78957413d59b8db7 in_seq=0 out_seq=0 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.713+0000 7fae727fc700 10 client.28435 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae727fc700 1 -- 172.21.15.52:0/1442446448 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564595d97d10 con 0x564595ebcdd0 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae727fc700 10 client.28435 handle_command_reply: tid=0 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 client.28435 shutdown 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 msgr2=0x564595e4ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e4ad80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fae7c00feb0 tx=0x7fae7c009260 comp rx=0 tx=0).stop 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 shutdown_connections 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x564595ebcdd0 0x564595ebf1f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564595e3ef60 0x564595e590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564595e38f20 0x564595e4ad80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 --2- 172.21.15.52:0/1442446448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564595e3e640 0x564595e4b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae727fc700 0 client.28435 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 >> 172.21.15.52:0/1442446448 conn(0x564595d63cc0 msgr2=0x564595e2ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 shutdown_connections 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.714+0000 7fae9c04ab80 1 -- 172.21.15.52:0/1442446448 wait complete. 2024-06-02T23:26:01.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae9c04ab80 20 client.28435 trim_cache size 0 max 16384 2024-06-02T23:26:01.718 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:26:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae780a2ca0 msgr2=0x7fae780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.718 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae780a2ca0 0x7fae780a5160 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009680 tx=0x7fae7c019040 comp rx=0 tx=0).stop 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 msgr2=0x7fae8c19f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 0x7fae8c19f6f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae8c069f30 tx=0x7fae8400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 shutdown_connections 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fae780a2ca0 0x7fae780a5160 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fae8c10ef30 0x7fae8c19f6f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fae8c06aa30 0x7fae8c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 --2- 172.21.15.52:0/485705370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fae8c06b350 0x7fae8c19b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 >> 172.21.15.52:0/485705370 conn(0x7fae8c078300 msgr2=0x7fae8c110480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.715+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 shutdown_connections 2024-06-02T23:26:01.719 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:01.716+0000 7fae917c7700 1 -- 172.21.15.52:0/485705370 wait complete. 2024-06-02T23:26:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:01 smithi174 ceph-mon[35994]: pgmap v855: 97 pgs: 97 active+clean; 313 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-06-02T23:26:01.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:01 smithi052 ceph-mon[32340]: pgmap v855: 97 pgs: 97 active+clean; 313 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-06-02T23:26:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:01 smithi123 ceph-mon[35977]: pgmap v855: 97 pgs: 97 active+clean; 313 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 263 KiB/s rd, 1.3 MiB/s wr, 127 op/s 2024-06-02T23:26:02.411 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:26:02.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 9369 9.53 MB/sec warmup 70 sec latency 74.094 ms 2024-06-02T23:26:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/3044067250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c981024b0 msgr2=0x7f0c98108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/3044067250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c981024b0 0x7f0c98108d40 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009ca0 tx=0x7f0c9002cad0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/3044067250 shutdown_connections 2024-06-02T23:26:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/3044067250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c981024b0 0x7f0c98108d40 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/3044067250 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c98101af0 0x7f0c98101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/3044067250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98100900 0x7f0c98100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/3044067250 >> 172.21.15.52:0/3044067250 conn(0x7f0c980fbe70 msgr2=0x7f0c980fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/3044067250 shutdown_connections 2024-06-02T23:26:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.474+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/3044067250 wait complete. 2024-06-02T23:26:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 Processor -- start 2024-06-02T23:26:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 -- start start 2024-06-02T23:26:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c98100900 0x7f0c98195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98101af0 0x7f0c98195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c981024b0 0x7f0c9819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9810b330 con 0x7f0c981024b0 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9810b1b0 con 0x7f0c98100900 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.475+0000 7f0c9dcc3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c9810b4b0 con 0x7f0c98101af0 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c98100900 0x7f0c98195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98101af0 0x7f0c98195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c98100900 0x7f0c98195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38490/0 (socket says 172.21.15.52:38490) 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c977fe640 1 -- 172.21.15.52:0/2570919056 learned_addr learned my addr 172.21.15.52:0/2570919056 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c97fff640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c981024b0 0x7f0c9819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 -- 172.21.15.52:0/2570919056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c98100900 msgr2=0x7f0c98195920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c98100900 0x7f0c98195920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 -- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c981024b0 msgr2=0x7f0c9819ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c981024b0 0x7f0c9819ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 -- 172.21.15.52:0/2570919056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c9819d540 con 0x7f0c98101af0 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c97fff640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c981024b0 0x7f0c9819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98101af0 0x7f0c98195e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00e9f0 tx=0x7f0c8c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c94ff9640 1 -- 172.21.15.52:0/2570919056 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c00cd80 con 0x7f0c98101af0 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.476+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c9819d830 con 0x7f0c98101af0 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.477+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c9819ddd0 con 0x7f0c98101af0 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.477+0000 7f0c94ff9640 1 -- 172.21.15.52:0/2570919056 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c8c0035f0 con 0x7f0c98101af0 2024-06-02T23:26:02.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.477+0000 7f0c94ff9640 1 -- 172.21.15.52:0/2570919056 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c8c010690 con 0x7f0c98101af0 2024-06-02T23:26:02.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.480+0000 7f0c94ff9640 1 -- 172.21.15.52:0/2570919056 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0c8c0037a0 con 0x7f0c98101af0 2024-06-02T23:26:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 Processor -- start 2024-06-02T23:26:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 -- start start 2024-06-02T23:26:02.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0c94ff9640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0c800a2d30 0x7f0c800a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 0x562311825430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x56231182a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x56231182d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5623117884f0 con 0x562311825010 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562311726870 con 0x56231182ab70 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5623117844b0 con 0x562311825a00 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0c97fff640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0c800a2d30 0x7f0c800a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 0x562311825430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0c94ff9640 1 -- 172.21.15.52:0/2570919056 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0c8c0cf390 con 0x7f0c98101af0 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 0x562311825430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53454/0 (socket says 172.21.15.52:53454) 2024-06-02T23:26:02.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.481+0000 7f0c977fe640 1 -- 172.21.15.52:0/4260193621 learned_addr learned my addr 172.21.15.52:0/4260193621 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x56231182a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c977fe640 1 -- 172.21.15.52:0/4260193621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 msgr2=0x56231182a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c977fe640 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x56231182a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c977fe640 1 -- 172.21.15.52:0/4260193621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 msgr2=0x56231182d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c97fff640 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x56231182d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c977fe640 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x56231182d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c977fe640 1 -- 172.21.15.52:0/4260193621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623117429b0 con 0x562311825010 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c97fff640 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x56231182d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c977fe640 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 0x562311825430 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009d70 tx=0x7f0c88011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e72b1943e0715c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.482+0000 7f0c97fff640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0c800a2d30 0x7f0c800a51f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0c90006010 tx=0x7f0c90009900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.483+0000 7f0c7dffb640 1 -- 172.21.15.52:0/4260193621 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88004220 con 0x562311825010 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.483+0000 7f0c7dffb640 1 -- 172.21.15.52:0/4260193621 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c880043c0 con 0x562311825010 2024-06-02T23:26:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.483+0000 7f0c7dffb640 1 -- 172.21.15.52:0/4260193621 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c880198d0 con 0x562311825010 2024-06-02T23:26:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 -- 172.21.15.52:0/4260193621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 msgr2=0x562311825430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 0x562311825430 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009d70 tx=0x7f0c88011810 comp rx=0 tx=0).stop 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 -- 172.21.15.52:0/4260193621 shutdown_connections 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x56231182a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x56231182d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/4260193621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825010 0x562311825430 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 -- 172.21.15.52:0/4260193621 >> 172.21.15.52:0/4260193621 conn(0x562311820680 msgr2=0x562311820a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 -- 172.21.15.52:0/4260193621 shutdown_connections 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 -- 172.21.15.52:0/4260193621 wait complete. 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.484+0000 7f0ca09cc740 1 -- start start 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 Processor -- start 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825010 0x56231182f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825a00 0x56231182f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x562311836720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0c96ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825a00 0x56231182f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825010 0x56231182f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0c96ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825a00 0x56231182f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53458/0 (socket says 172.21.15.52:53458) 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0c96ffd640 1 -- 172.21.15.52:0/2785869833 learned_addr learned my addr 172.21.15.52:0/2785869833 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56231182db20 con 0x562311825a00 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562311711ae0 con 0x56231182ab70 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56231182dca0 con 0x562311825010 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0c97fff640 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x562311836720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.485+0000 7f0ca09cc740 10 client.0 _ll_register_callbacks cb 0x56231174a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c977fe640 1 -- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 msgr2=0x562311836720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c977fe640 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x562311836720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c977fe640 1 -- 172.21.15.52:0/2785869833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825a00 msgr2=0x56231182f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c977fe640 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825a00 0x56231182f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c977fe640 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56231149c210 con 0x562311825010 2024-06-02T23:26:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c880043c0 con 0x562311825010 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c977fe640 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825010 0x56231182f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c8801b040 tx=0x7f0c88005250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6017ac17e3ec83f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c76ffd640 10 client.28447 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c76ffd640 1 -- 172.21.15.52:0/2785869833 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88032050 con 0x562311825010 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.486+0000 7f0c76ffd640 1 -- 172.21.15.52:0/2785869833 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0c880035f0 con 0x562311825010 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0c76ffd640 1 -- 172.21.15.52:0/2785869833 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0c88028070 con 0x562311825010 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 10 client.28447 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=37}) -- 0x562311568620 con 0x562311825010 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0c76ffd640 1 -- 172.21.15.52:0/2785869833 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f0c88003850 con 0x562311825010 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 10 client.28447 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 10 client.28447 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 20 client.28447 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231183ab10 0x56231183cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 4 client.28447 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56231179bf10 con 0x56231183ab10 2024-06-02T23:26:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.487+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231183ab10 0x56231183cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.488+0000 7f0c76ffd640 10 client.28447 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.488+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231183ab10 0x56231183cfd0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88dee5c5a03bb45 server_cookie=336a6999d0cd0c8e in_seq=0 out_seq=0 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.488+0000 7f0c76ffd640 1 -- 172.21.15.52:0/2785869833 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56231179bf10 con 0x56231183ab10 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.488+0000 7f0c76ffd640 10 client.28447 handle_command_reply: tid=0 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.488+0000 7f0ca09cc740 1 client.28447 shutdown 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825010 msgr2=0x56231182f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825010 0x56231182f1a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0c8801b040 tx=0x7f0c88005250 comp rx=0 tx=0).stop 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 shutdown_connections 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231183ab10 0x56231183cfd0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56231182ab70 0x562311836720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311825a00 0x56231182f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/2785869833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825010 0x56231182f1a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 >> 172.21.15.52:0/2785869833 conn(0x562311820680 msgr2=0x56231181f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0c76ffd640 0 client.28447 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 shutdown_connections 2024-06-02T23:26:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 1 -- 172.21.15.52:0/2785869833 wait complete. 2024-06-02T23:26:02.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.489+0000 7f0ca09cc740 20 client.28447 trim_cache size 0 max 16384 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.514+0000 7f0ca09cc740 1 Processor -- start 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.514+0000 7f0ca09cc740 1 -- start start 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x562311823590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x562311836640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x562311823ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a6a20 con 0x562311820680 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x562311823590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x562311823590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53472/0 (socket says 172.21.15.52:53472) 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 -- 172.21.15.52:0/1287435303 learned_addr learned my addr 172.21.15.52:0/1287435303 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c96ffd640 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x562311836640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a8680 con 0x562311825010 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a70f0 con 0x562311825a00 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c97fff640 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x562311823ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 -- 172.21.15.52:0/1287435303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 msgr2=0x562311823ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x562311823ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 -- 172.21.15.52:0/1287435303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 msgr2=0x562311836640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x562311836640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.515+0000 7f0c977fe640 1 -- 172.21.15.52:0/1287435303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5623118204b0 con 0x562311820680 2024-06-02T23:26:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.516+0000 7f0c977fe640 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x562311823590 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009c90 tx=0x7f0c88025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42665c1192b4dc44 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.516+0000 7f0c7d7fa640 1 -- 172.21.15.52:0/1287435303 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88032070 con 0x562311820680 2024-06-02T23:26:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.516+0000 7f0c7d7fa640 1 -- 172.21.15.52:0/1287435303 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c880043c0 con 0x562311820680 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.516+0000 7f0c7d7fa640 1 -- 172.21.15.52:0/1287435303 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c88019430 con 0x562311820680 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 msgr2=0x562311823590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x562311823590 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0c88009c90 tx=0x7f0c88025000 comp rx=0 tx=0).stop 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 shutdown_connections 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x562311823ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x562311836640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1287435303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x562311823590 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 >> 172.21.15.52:0/1287435303 conn(0x56231182e3b0 msgr2=0x562311826b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 shutdown_connections 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.517+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1287435303 wait complete. 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 -- start start 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 Processor -- start 2024-06-02T23:26:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x5623118332a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x5623118337e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x5623118ac070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a7ef0 con 0x562311820680 2024-06-02T23:26:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a77f0 con 0x562311825010 2024-06-02T23:26:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0c981a4a00 con 0x562311825a00 2024-06-02T23:26:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0ca09cc740 10 client.0 _ll_register_callbacks cb 0x7f0c88004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0c96ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x5623118337e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x5623118332a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0c97fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x5623118ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0c977fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x5623118332a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53488/0 (socket says 172.21.15.52:53488) 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0c97fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x5623118ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37222/0 (socket says 172.21.15.52:37222) 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.518+0000 7f0c977fe640 1 -- 172.21.15.52:0/1372967851 learned_addr learned my addr 172.21.15.52:0/1372967851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:02.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c97fff640 1 -- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 msgr2=0x5623118337e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c97fff640 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x5623118337e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c97fff640 1 -- 172.21.15.52:0/1372967851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 msgr2=0x5623118332a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c97fff640 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x5623118332a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c97fff640 1 -- 172.21.15.52:0/1372967851 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562311836840 con 0x562311825a00 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c977fe640 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x5623118332a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c97fff640 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x5623118ac070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009bf0 tx=0x7f0c900095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea8d5b302289ae9a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.519+0000 7f0c76ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0c76ffd640 1 -- 172.21.15.52:0/1372967851 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c90047070 con 0x562311825a00 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0c76ffd640 1 -- 172.21.15.52:0/1372967851 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0c90004310 con 0x562311825a00 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c880043c0 con 0x562311825a00 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0c76ffd640 1 -- 172.21.15.52:0/1372967851 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0c90004510 con 0x562311825a00 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0ca09cc740 10 client.28453 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5623118ad5a0 con 0x562311825a00 2024-06-02T23:26:02.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0c76ffd640 1 -- 172.21.15.52:0/1372967851 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0c880043c0 con 0x562311825a00 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.520+0000 7f0c76ffd640 1 -- 172.21.15.52:0/1372967851 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f0c9002e040 con 0x562311825a00 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0ca09cc740 10 client.28453 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0ca09cc740 10 client.28453 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0ca09cc740 20 client.28453 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231182ab70 0x5623118b0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0ca09cc740 4 client.28453 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562311826690 con 0x56231182ab70 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0c977fe640 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231182ab70 0x5623118b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0c76ffd640 10 client.28453 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.521+0000 7f0c977fe640 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231182ab70 0x5623118b0040 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=635f7ca796656d18 server_cookie=18ce3f27b5f014a9 in_seq=0 out_seq=0 2024-06-02T23:26:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0c76ffd640 1 -- 172.21.15.52:0/1372967851 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562311826690 con 0x56231182ab70 2024-06-02T23:26:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0c76ffd640 10 client.28453 handle_command_reply: tid=0 2024-06-02T23:26:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 client.28453 shutdown 2024-06-02T23:26:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 msgr2=0x5623118ac070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x5623118ac070 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009bf0 tx=0x7f0c900095c0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 shutdown_connections 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56231182ab70 0x5623118b0040 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562311825a00 0x5623118ac070 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562311825010 0x5623118337e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0c76ffd640 0 client.28453 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 --2- 172.21.15.52:0/1372967851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562311820680 0x5623118332a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 >> 172.21.15.52:0/1372967851 conn(0x56231182e3b0 msgr2=0x562311823590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 shutdown_connections 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 1 -- 172.21.15.52:0/1372967851 wait complete. 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.559+0000 7f0ca09cc740 20 client.28453 trim_cache size 0 max 16384 2024-06-02T23:26:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:26:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:26:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:26:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0c800a2d30 msgr2=0x7f0c800a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0c800a2d30 0x7f0c800a51f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0c90006010 tx=0x7f0c90009900 comp rx=0 tx=0).stop 2024-06-02T23:26:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98101af0 msgr2=0x7f0c98195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98101af0 0x7f0c98195e60 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0c8c00e9f0 tx=0x7f0c8c00eeb0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 shutdown_connections 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0c800a2d30 0x7f0c800a51f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0c98101af0 0x7f0c98195e60 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0c981024b0 0x7f0c9819ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 --2- 172.21.15.52:0/2570919056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c98100900 0x7f0c98195920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 >> 172.21.15.52:0/2570919056 conn(0x7f0c980fbe70 msgr2=0x7f0c98104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.560+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 shutdown_connections 2024-06-02T23:26:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:02.561+0000 7f0c9dcc3640 1 -- 172.21.15.52:0/2570919056 wait complete. 2024-06-02T23:26:02.773 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:03.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 9545 9.52 MB/sec warmup 71 sec latency 76.952 ms 2024-06-02T23:26:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:03 smithi052 ceph-mon[32340]: pgmap v856: 97 pgs: 97 active+clean; 314 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-06-02T23:26:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:03 smithi174 ceph-mon[35994]: pgmap v856: 97 pgs: 97 active+clean; 314 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-06-02T23:26:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:03 smithi123 ceph-mon[35977]: pgmap v856: 97 pgs: 97 active+clean; 314 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 424 KiB/s rd, 1.7 MiB/s wr, 178 op/s 2024-06-02T23:26:04.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 9690 9.45 MB/sec warmup 72 sec latency 108.226 ms 2024-06-02T23:26:05.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 9848 9.48 MB/sec warmup 73 sec latency 110.977 ms 2024-06-02T23:26:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:05 smithi174 ceph-mon[35994]: pgmap v857: 97 pgs: 97 active+clean; 315 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-06-02T23:26:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:05 smithi052 ceph-mon[32340]: pgmap v857: 97 pgs: 97 active+clean; 315 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-06-02T23:26:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:05 smithi123 ceph-mon[35977]: pgmap v857: 97 pgs: 97 active+clean; 315 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 2.1 MiB/s wr, 199 op/s 2024-06-02T23:26:06.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10059 9.63 MB/sec warmup 74 sec latency 125.265 ms 2024-06-02T23:26:07.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10279 9.66 MB/sec warmup 75 sec latency 123.661 ms 2024-06-02T23:26:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:07 smithi174 ceph-mon[35994]: pgmap v858: 97 pgs: 97 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-06-02T23:26:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:07 smithi052 ceph-mon[32340]: pgmap v858: 97 pgs: 97 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-06-02T23:26:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:07 smithi123 ceph-mon[35977]: pgmap v858: 97 pgs: 97 active+clean; 323 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 673 KiB/s rd, 3.3 MiB/s wr, 273 op/s 2024-06-02T23:26:08.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10407 9.66 MB/sec warmup 76 sec latency 114.683 ms 2024-06-02T23:26:09.449 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10508 9.62 MB/sec warmup 77 sec latency 127.008 ms 2024-06-02T23:26:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:09 smithi174 ceph-mon[35994]: pgmap v859: 97 pgs: 97 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.3 MiB/s wr, 249 op/s 2024-06-02T23:26:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:09 smithi052 ceph-mon[32340]: pgmap v859: 97 pgs: 97 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.3 MiB/s wr, 249 op/s 2024-06-02T23:26:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:09 smithi123 ceph-mon[35977]: pgmap v859: 97 pgs: 97 active+clean; 325 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 3.3 MiB/s wr, 249 op/s 2024-06-02T23:26:10.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10696 9.74 MB/sec warmup 78 sec latency 136.748 ms 2024-06-02T23:26:11.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10838 9.74 MB/sec warmup 79 sec latency 152.783 ms 2024-06-02T23:26:11.744 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:11 smithi052 ceph-mon[32340]: pgmap v860: 97 pgs: 97 active+clean; 332 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-06-02T23:26:11.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 -- 172.21.15.52:0/2383092712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101380 msgr2=0x7ff674101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 --2- 172.21.15.52:0/2383092712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101380 0x7ff674101800 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ff668009990 tx=0x7ff668009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 -- 172.21.15.52:0/2383092712 shutdown_connections 2024-06-02T23:26:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 --2- 172.21.15.52:0/2383092712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101d40 0x7ff67410e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 --2- 172.21.15.52:0/2383092712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6741073c0 0x7ff6741077a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 --2- 172.21.15.52:0/2383092712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101380 0x7ff674101800 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.810+0000 7ff67967a700 1 -- 172.21.15.52:0/2383092712 >> 172.21.15.52:0/2383092712 conn(0x7ff674078330 msgr2=0x7ff674078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.811+0000 7ff67967a700 1 -- 172.21.15.52:0/2383092712 shutdown_connections 2024-06-02T23:26:11.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.811+0000 7ff67967a700 1 -- 172.21.15.52:0/2383092712 wait complete. 2024-06-02T23:26:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.811+0000 7ff67967a700 1 Processor -- start 2024-06-02T23:26:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff67967a700 1 -- start start 2024-06-02T23:26:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff67967a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101380 0x7ff6741999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff67967a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 0x7ff674199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff67967a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6741073c0 0x7ff6741a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff672ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101380 0x7ff6741999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff672ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101380 0x7ff6741999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54172/0 (socket says 172.21.15.52:54172) 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.812+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 0x7ff674199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 0x7ff674199f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44054/0 (socket says 172.21.15.52:44054) 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff672ffd700 1 -- 172.21.15.52:0/4209464994 learned_addr learned my addr 172.21.15.52:0/4209464994 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741114e0 con 0x7ff674101380 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff674111360 con 0x7ff674101d40 2024-06-02T23:26:11.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff674111660 con 0x7ff6741073c0 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6737fe700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6741073c0 0x7ff6741a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6727fc700 1 -- 172.21.15.52:0/4209464994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6741073c0 msgr2=0x7ff6741a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6727fc700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6741073c0 0x7ff6741a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6727fc700 1 -- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101380 msgr2=0x7ff6741999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6727fc700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101380 0x7ff6741999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.813+0000 7ff6727fc700 1 -- 172.21.15.52:0/4209464994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6741a4f80 con 0x7ff674101d40 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.814+0000 7ff6727fc700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 0x7ff674199f00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff6680059d0 tx=0x7ff668004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:11.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.814+0000 7ff65bfff700 1 -- 172.21.15.52:0/4209464994 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668027070 con 0x7ff674101d40 2024-06-02T23:26:11.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.814+0000 7ff65bfff700 1 -- 172.21.15.52:0/4209464994 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff668022470 con 0x7ff674101d40 2024-06-02T23:26:11.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.814+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6741a5150 con 0x7ff674101d40 2024-06-02T23:26:11.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.815+0000 7ff65bfff700 1 -- 172.21.15.52:0/4209464994 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668017650 con 0x7ff674101d40 2024-06-02T23:26:11.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.815+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6741a5600 con 0x7ff674101d40 2024-06-02T23:26:11.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.817+0000 7ff65bfff700 1 -- 172.21.15.52:0/4209464994 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff66802a070 con 0x7ff674101d40 2024-06-02T23:26:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.818+0000 7ff65bfff700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff65c0a2d20 0x7ff65c0a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.819+0000 7ff65bfff700 1 -- 172.21.15.52:0/4209464994 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff6680cfd20 con 0x7ff674101d40 2024-06-02T23:26:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.819+0000 7ff672ffd700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff65c0a2d20 0x7ff65c0a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.819+0000 7ff672ffd700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff65c0a2d20 0x7ff65c0a51e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff664009c90 tx=0x7ff664009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:11.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 Processor -- start 2024-06-02T23:26:11.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 -- start start 2024-06-02T23:26:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0be640 0x558efa0bea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0bef60 0x558efa0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558efa00a810 con 0x558efa0b8f20 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff672ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0be640 0x558efa0bea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ef9fb2720 con 0x558efa0be640 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff683efdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558efa0067d0 con 0x558efa0bef60 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0bef60 0x558efa0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.820+0000 7ff672ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0be640 0x558efa0bea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:44062/0 (socket says 172.21.15.52:44062) 2024-06-02T23:26:11.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff672ffd700 1 -- 172.21.15.52:0/122812535 learned_addr learned my addr 172.21.15.52:0/122812535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6727fc700 1 -- 172.21.15.52:0/122812535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0be640 msgr2=0x558efa0bea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6727fc700 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0be640 0x558efa0bea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6727fc700 1 -- 172.21.15.52:0/122812535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 msgr2=0x558efa0b9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6727fc700 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6727fc700 1 -- 172.21.15.52:0/122812535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ef9fa8f50 con 0x558efa0bef60 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6727fc700 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0bef60 0x558efa0b89e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff66809c570 tx=0x7ff66809bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=781d923b5a627b27 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.821+0000 7ff6597fa700 1 -- 172.21.15.52:0/122812535 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66809c5e0 con 0x558efa0bef60 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.822+0000 7ff6597fa700 1 -- 172.21.15.52:0/122812535 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6680a0990 con 0x558efa0bef60 2024-06-02T23:26:11.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.822+0000 7ff6597fa700 1 -- 172.21.15.52:0/122812535 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66809c840 con 0x558efa0bef60 2024-06-02T23:26:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 -- 172.21.15.52:0/122812535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0bef60 msgr2=0x558efa0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0bef60 0x558efa0b89e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff66809c570 tx=0x7ff66809bf40 comp rx=0 tx=0).stop 2024-06-02T23:26:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 -- 172.21.15.52:0/122812535 shutdown_connections 2024-06-02T23:26:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0be640 0x558efa0bea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 --2- 172.21.15.52:0/122812535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0bef60 0x558efa0b89e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 -- 172.21.15.52:0/122812535 >> 172.21.15.52:0/122812535 conn(0x558ef9fe3cc0 msgr2=0x558efa0b3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 -- 172.21.15.52:0/122812535 shutdown_connections 2024-06-02T23:26:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 -- 172.21.15.52:0/122812535 wait complete. 2024-06-02T23:26:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.823+0000 7ff683efdb80 1 -- start start 2024-06-02T23:26:11.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 Processor -- start 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0b8f20 0x558efa0c4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0c2050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0bef60 0x558efa0c2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558efa0c56f0 con 0x558efa0bef60 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0bef60 0x558efa0c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44638/0 (socket says 172.21.15.52:44638) 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff6737fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0bef60 0x558efa0c2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54202/0 (socket says 172.21.15.52:54202) 2024-06-02T23:26:11.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff6727fc700 1 -- 172.21.15.52:0/3291650098 learned_addr learned my addr 172.21.15.52:0/3291650098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558ef9faabf0 con 0x558efa0b8f20 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558efa0c5870 con 0x558efa0be640 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff683efdb80 10 client.0 _ll_register_callbacks cb 0x558ef9fa4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.824+0000 7ff672ffd700 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0b8f20 0x558efa0c4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff6727fc700 1 -- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0b8f20 msgr2=0x558efa0c4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff6727fc700 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0b8f20 0x558efa0c4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff6727fc700 1 -- 172.21.15.52:0/3291650098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0bef60 msgr2=0x558efa0c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff6727fc700 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0bef60 0x558efa0c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff6727fc700 1 -- 172.21.15.52:0/3291650098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ef9f598d0 con 0x558efa0be640 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff6727fc700 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0c2050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff6680982c0 tx=0x7ff668098160 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e7c140aee2125c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff656ff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff656ff5700 1 -- 172.21.15.52:0/3291650098 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66809c5e0 con 0x558efa0be640 2024-06-02T23:26:11.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff656ff5700 1 -- 172.21.15.52:0/3291650098 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff66809c840 con 0x558efa0be640 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.825+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6680a0990 con 0x558efa0be640 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff656ff5700 1 -- 172.21.15.52:0/3291650098 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff66809ca40 con 0x558efa0be640 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 10 client.28465 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x558efa0c56f0 con 0x558efa0be640 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff656ff5700 1 -- 172.21.15.52:0/3291650098 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6680a4d30 con 0x558efa0be640 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff656ff5700 1 -- 172.21.15.52:0/3291650098 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7ff66809bac0 con 0x558efa0be640 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 10 client.28465 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 10 client.28465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 20 client.28465 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa0dabd0 0x558efa0dcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 4 client.28465 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff6680a0990 con 0x558efa0dabd0 2024-06-02T23:26:11.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.826+0000 7ff672ffd700 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa0dabd0 0x558efa0dcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.827+0000 7ff656ff5700 10 client.28465 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.827+0000 7ff672ffd700 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa0dabd0 0x558efa0dcfc0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5571de30779e69c3 server_cookie=c1f4870a31cc7e54 in_seq=0 out_seq=0 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.828+0000 7ff656ff5700 1 -- 172.21.15.52:0/3291650098 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7ff6680a0990 con 0x558efa0dabd0 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.828+0000 7ff656ff5700 10 client.28465 handle_command_reply: tid=0 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.828+0000 7ff683efdb80 1 client.28465 shutdown 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.828+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 msgr2=0x558efa0c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.828+0000 7ff683efdb80 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0c2050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff6680982c0 tx=0x7ff668098160 comp rx=0 tx=0).stop 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 shutdown_connections 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa0dabd0 0x558efa0dcfc0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff656ff5700 0 client.28465 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0bef60 0x558efa0c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0b8f20 0x558efa0c4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 --2- 172.21.15.52:0/3291650098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0c2050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 >> 172.21.15.52:0/3291650098 conn(0x558ef9fe3cc0 msgr2=0x558efa0b35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 shutdown_connections 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 1 -- 172.21.15.52:0/3291650098 wait complete. 2024-06-02T23:26:11.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.829+0000 7ff683efdb80 20 client.28465 trim_cache size 0 max 16384 2024-06-02T23:26:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 Processor -- start 2024-06-02T23:26:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 -- start start 2024-06-02T23:26:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0bfc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741aa800 con 0x558efa0b8f20 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741ac460 con 0x558efa0bef60 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff683efdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741aaed0 con 0x558efa0be640 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0bfc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0bfc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44652/0 (socket says 172.21.15.52:44652) 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff6727fc700 1 -- 172.21.15.52:0/2367256025 learned_addr learned my addr 172.21.15.52:0/2367256025 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff672ffd700 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.863+0000 7ff6737fe700 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff6727fc700 1 -- 172.21.15.52:0/2367256025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 msgr2=0x558efa0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff6727fc700 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff6727fc700 1 -- 172.21.15.52:0/2367256025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 msgr2=0x558efa0b89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff6727fc700 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff6727fc700 1 -- 172.21.15.52:0/2367256025 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558efa0b4470 con 0x558efa0be640 2024-06-02T23:26:11.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff6727fc700 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0bfc10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff66809a890 tx=0x7ff6680a3020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fe4eef89e7743d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff658ff9700 1 -- 172.21.15.52:0/2367256025 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66809bf40 con 0x558efa0be640 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff658ff9700 1 -- 172.21.15.52:0/2367256025 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff6680a0990 con 0x558efa0be640 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.864+0000 7ff658ff9700 1 -- 172.21.15.52:0/2367256025 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66809ba30 con 0x558efa0be640 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 -- 172.21.15.52:0/2367256025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 msgr2=0x558efa0bfc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0bfc10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff66809a890 tx=0x7ff6680a3020 comp rx=0 tx=0).stop 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 -- 172.21.15.52:0/2367256025 shutdown_connections 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0b8f20 0x558efa0b89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 --2- 172.21.15.52:0/2367256025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0be640 0x558efa0bfc10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.865+0000 7ff683efdb80 1 -- 172.21.15.52:0/2367256025 >> 172.21.15.52:0/2367256025 conn(0x558ef9fe3cc0 msgr2=0x558efa0ace00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.866+0000 7ff683efdb80 1 -- 172.21.15.52:0/2367256025 shutdown_connections 2024-06-02T23:26:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.866+0000 7ff683efdb80 1 -- 172.21.15.52:0/2367256025 wait complete. 2024-06-02T23:26:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.866+0000 7ff683efdb80 1 -- start start 2024-06-02T23:26:11.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.866+0000 7ff683efdb80 1 Processor -- start 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0b8f20 0x558efa0ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0be640 0x558efa0ceea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff683efdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa143850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff683efdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741abcd0 con 0x558efa0be640 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff672ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0b8f20 0x558efa0ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0be640 0x558efa0ceea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff6727fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0be640 0x558efa0ceea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54218/0 (socket says 172.21.15.52:54218) 2024-06-02T23:26:11.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.867+0000 7ff6727fc700 1 -- 172.21.15.52:0/379245810 learned_addr learned my addr 172.21.15.52:0/379245810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff6737fe700 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa143850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741ab5d0 con 0x558efa0bef60 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6741a87c0 con 0x558efa0b8f20 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff6737fe700 1 -- 172.21.15.52:0/379245810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0b8f20 msgr2=0x558efa0ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff6737fe700 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0b8f20 0x558efa0ce960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff6737fe700 1 -- 172.21.15.52:0/379245810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0be640 msgr2=0x558efa0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff6737fe700 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0be640 0x558efa0ceea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.868+0000 7ff6737fe700 1 -- 172.21.15.52:0/379245810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558efa143e90 con 0x558efa0bef60 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff683efdb80 10 client.0 _ll_register_callbacks cb 0x558ef9fa4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff6737fe700 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa143850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff66c007be0 tx=0x7ff66c007ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff11f4e5ebb0a44 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff656ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff656ff5700 1 -- 172.21.15.52:0/379245810 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff66c004bb0 con 0x558efa0bef60 2024-06-02T23:26:11.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff656ff5700 1 -- 172.21.15.52:0/379245810 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff66c004d50 con 0x558efa0bef60 2024-06-02T23:26:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff6680a0990 con 0x558efa0bef60 2024-06-02T23:26:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff656ff5700 1 -- 172.21.15.52:0/379245810 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7ff66c015b00 con 0x558efa0bef60 2024-06-02T23:26:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff683efdb80 10 client.28793 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:11.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.869+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=37}) -- 0x558efa144fe0 con 0x558efa0bef60 2024-06-02T23:26:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff656ff5700 1 -- 172.21.15.52:0/379245810 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7ff66c015ca0 con 0x558efa0bef60 2024-06-02T23:26:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff683efdb80 10 client.28793 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff683efdb80 10 client.28793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:11.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff683efdb80 20 client.28793 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff683efdb80 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa1456a0 0x558efa147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff683efdb80 4 client.28793 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff6680a0990 con 0x558efa1456a0 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.870+0000 7ff672ffd700 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa1456a0 0x558efa147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.871+0000 7ff672ffd700 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa1456a0 0x558efa147ae0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb259674d4de29fa server_cookie=f14b467e44d8e5b6 in_seq=0 out_seq=0 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.871+0000 7ff656ff5700 10 client.28793 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.871+0000 7ff656ff5700 1 -- 172.21.15.52:0/379245810 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff6680a0990 con 0x558efa1456a0 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.871+0000 7ff656ff5700 10 client.28793 handle_command_reply: tid=0 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.871+0000 7ff683efdb80 1 client.28793 shutdown 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 msgr2=0x558efa143850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa143850 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff66c007be0 tx=0x7ff66c007ef0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 shutdown_connections 2024-06-02T23:26:11.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558efa1456a0 0x558efa147ae0 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.875 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:26:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558efa0bef60 0x558efa143850 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558efa0b8f20 0x558efa0ce960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 --2- 172.21.15.52:0/379245810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558efa0be640 0x558efa0ceea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 >> 172.21.15.52:0/379245810 conn(0x558ef9fe3cc0 msgr2=0x558efa0bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:11.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff656ff5700 0 client.28793 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 shutdown_connections 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 1 -- 172.21.15.52:0/379245810 wait complete. 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.872+0000 7ff683efdb80 20 client.28793 trim_cache size 0 max 16384 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff65c0a2d20 msgr2=0x7ff65c0a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff65c0a2d20 0x7ff65c0a51e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff664009c90 tx=0x7ff664009260 comp rx=0 tx=0).stop 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 msgr2=0x7ff674199f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 0x7ff674199f00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff6680059d0 tx=0x7ff668004790 comp rx=0 tx=0).stop 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 shutdown_connections 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff65c0a2d20 0x7ff65c0a51e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6741073c0 0x7ff6741a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff674101380 0x7ff6741999c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 --2- 172.21.15.52:0/4209464994 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff674101d40 0x7ff674199f00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:11.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 >> 172.21.15.52:0/4209464994 conn(0x7ff674078330 msgr2=0x7ff674100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.873+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 shutdown_connections 2024-06-02T23:26:11.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:11.874+0000 7ff67967a700 1 -- 172.21.15.52:0/4209464994 wait complete. 2024-06-02T23:26:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:11 smithi174 ceph-mon[35994]: pgmap v860: 97 pgs: 97 active+clean; 332 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-06-02T23:26:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:11 smithi123 ceph-mon[35977]: pgmap v860: 97 pgs: 97 active+clean; 332 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 844 KiB/s rd, 4.2 MiB/s wr, 270 op/s 2024-06-02T23:26:12.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 10969 9.70 MB/sec warmup 80 sec latency 65.340 ms 2024-06-02T23:26:13.303 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:26:13.303 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-02T23:26:13.303 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:26:13.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 11121 9.66 MB/sec warmup 81 sec latency 58.851 ms 2024-06-02T23:26:13.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:13 smithi052 ceph-mon[32340]: pgmap v861: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-06-02T23:26:13.694 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:13.708 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:26:13.708 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:26:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:13 smithi174 ceph-mon[35994]: pgmap v861: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-06-02T23:26:14.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:13 smithi123 ceph-mon[35977]: pgmap v861: 97 pgs: 97 active+clean; 336 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-06-02T23:26:14.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 11237 9.55 MB/sec warmup 82 sec latency 72.450 ms 2024-06-02T23:26:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:15.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 11376 9.48 MB/sec warmup 83 sec latency 135.466 ms 2024-06-02T23:26:15.676 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:15 smithi052 ceph-mon[32340]: pgmap v862: 97 pgs: 97 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 237 op/s 2024-06-02T23:26:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 -- 172.21.15.52:0/1404938637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1002b0 msgr2=0x7f363c1006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 --2- 172.21.15.52:0/1404938637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1002b0 0x7f363c1006d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3634009ca0 tx=0x7f363402a8c0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 -- 172.21.15.52:0/1404938637 shutdown_connections 2024-06-02T23:26:15.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 --2- 172.21.15.52:0/1404938637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c1015b0 0x7f363c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 --2- 172.21.15.52:0/1404938637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c100c10 0x7f363c101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 --2- 172.21.15.52:0/1404938637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1002b0 0x7f363c1006d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.704+0000 7f36441ee640 1 -- 172.21.15.52:0/1404938637 >> 172.21.15.52:0/1404938637 conn(0x7f363c0fbe70 msgr2=0x7f363c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.705+0000 7f36441ee640 1 -- 172.21.15.52:0/1404938637 shutdown_connections 2024-06-02T23:26:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.705+0000 7f36441ee640 1 -- 172.21.15.52:0/1404938637 wait complete. 2024-06-02T23:26:15.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.706+0000 7f36441ee640 1 Processor -- start 2024-06-02T23:26:15.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.706+0000 7f36441ee640 1 -- start start 2024-06-02T23:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.707+0000 7f36441ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 0x7f363c199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.707+0000 7f36441ee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 0x7f363c19a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.707+0000 7f3641f8a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 0x7f363c199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.707+0000 7f3641f8a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 0x7f363c199d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38290/0 (socket says 172.21.15.52:38290) 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.707+0000 7f3641789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 0x7f363c19a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.707+0000 7f3641789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 0x7f363c19a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58046/0 (socket says 172.21.15.52:58046) 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.708+0000 7f3641f8a640 1 -- 172.21.15.52:0/3651367621 learned_addr learned my addr 172.21.15.52:0/3651367621 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.708+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1015b0 0x7f363c1a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.708+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10f690 con 0x7f363c100c10 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.708+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10f510 con 0x7f363c1002b0 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.708+0000 7f364278b640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1015b0 0x7f363c1a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.709 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.708+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f363c10f810 con 0x7f363c1015b0 2024-06-02T23:26:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f3641f8a640 1 -- 172.21.15.52:0/3651367621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1015b0 msgr2=0x7f363c1a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f3641f8a640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1015b0 0x7f363c1a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.710 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f3641f8a640 1 -- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 msgr2=0x7f363c19a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f364278b640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1015b0 0x7f363c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f3641f8a640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 0x7f363c19a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f3641f8a640 1 -- 172.21.15.52:0/3651367621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363c1a1960 con 0x7f363c1002b0 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.709+0000 7f3641789640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 0x7f363c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.710+0000 7f3641f8a640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 0x7f363c199d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f36340319d0 tx=0x7f3634003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.710+0000 7f3632ffd640 1 -- 172.21.15.52:0/3651367621 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36340317c0 con 0x7f363c1002b0 2024-06-02T23:26:15.711 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.710+0000 7f3632ffd640 1 -- 172.21.15.52:0/3651367621 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f363402f380 con 0x7f363c1002b0 2024-06-02T23:26:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.711+0000 7f3632ffd640 1 -- 172.21.15.52:0/3651367621 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3634030b70 con 0x7f363c1002b0 2024-06-02T23:26:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.711+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f363c1a1bf0 con 0x7f363c1002b0 2024-06-02T23:26:15.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.711+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f363c1a2190 con 0x7f363c1002b0 2024-06-02T23:26:15.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.714+0000 7f3632ffd640 1 -- 172.21.15.52:0/3651367621 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3634030d10 con 0x7f363c1002b0 2024-06-02T23:26:15.715 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 Processor -- start 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 -- start start 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d2000e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5a00 0x55d2000ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d2000ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d2000484f0 con 0x55d2000eab70 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d1fffe6870 con 0x55d2000e5010 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3646fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2000444b0 con 0x55d2000e5a00 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f364278b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d2000e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f364278b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d2000e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38294/0 (socket says 172.21.15.52:38294) 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3641f8a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5a00 0x55d2000ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f364278b640 1 -- 172.21.15.52:0/1345906781 learned_addr learned my addr 172.21.15.52:0/1345906781 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.715+0000 7f3641789640 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d2000ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f364278b640 1 -- 172.21.15.52:0/1345906781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5a00 msgr2=0x55d2000ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f364278b640 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5a00 0x55d2000ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f364278b640 1 -- 172.21.15.52:0/1345906781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 msgr2=0x55d2000ed000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f364278b640 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d2000ed000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f364278b640 1 -- 172.21.15.52:0/1345906781 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2000029b0 con 0x55d2000e5010 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f3641789640 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d2000ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f364278b640 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d2000e5430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f363800bd30 tx=0x7f363800f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb94e8620bf8b3a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:15.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f3627fff640 1 -- 172.21.15.52:0/1345906781 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638011640 con 0x55d2000e5010 2024-06-02T23:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f3627fff640 1 -- 172.21.15.52:0/1345906781 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f363800be20 con 0x55d2000e5010 2024-06-02T23:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f3627fff640 1 -- 172.21.15.52:0/1345906781 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380109d0 con 0x55d2000e5010 2024-06-02T23:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.716+0000 7f3632ffd640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36280a2cb0 0x7f36280a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3641f8a640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36280a2cb0 0x7f36280a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 -- 172.21.15.52:0/1345906781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 msgr2=0x55d2000e5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d2000e5430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f363800bd30 tx=0x7f363800f730 comp rx=0 tx=0).stop 2024-06-02T23:26:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 -- 172.21.15.52:0/1345906781 shutdown_connections 2024-06-02T23:26:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5a00 0x55d2000ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d2000ed000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1345906781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d2000e5430 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3646fdc740 1 -- 172.21.15.52:0/1345906781 >> 172.21.15.52:0/1345906781 conn(0x55d2000e0680 msgr2=0x55d2000e0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.717+0000 7f3632ffd640 1 -- 172.21.15.52:0/3651367621 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36340f04f0 con 0x7f363c1002b0 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.718+0000 7f3641f8a640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36280a2cb0 0x7f36280a5170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f36340ee390 tx=0x7f36340f5040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.718+0000 7f3646fdc740 1 -- 172.21.15.52:0/1345906781 shutdown_connections 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.718+0000 7f3646fdc740 1 -- 172.21.15.52:0/1345906781 wait complete. 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.718+0000 7f3646fdc740 1 -- start start 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.718+0000 7f3646fdc740 1 Processor -- start 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5010 0x55d200108bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200109100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d200110140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d2000feb30 con 0x55d2000eab70 2024-06-02T23:26:15.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d1fffd1ae0 con 0x55d2000e5a00 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2000fecb0 con 0x55d2000e5010 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3646fdc740 10 client.0 _ll_register_callbacks cb 0x55d20000a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3641789640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3641789640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38308/0 (socket says 172.21.15.52:38308) 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f3641789640 1 -- 172.21.15.52:0/429503335 learned_addr learned my addr 172.21.15.52:0/429503335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f364278b640 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d200110140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f364278b640 1 -- 172.21.15.52:0/429503335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5010 msgr2=0x55d200108bc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f364278b640 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5010 0x55d200108bc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f364278b640 1 -- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 msgr2=0x55d200109100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f364278b640 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200109100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.719+0000 7f364278b640 1 -- 172.21.15.52:0/429503335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1ffd5c210 con 0x55d2000eab70 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f364278b640 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d200110140 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f363800bfd0 tx=0x7f36380056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824189a03060f967 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:15.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f36257fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f36257fa640 1 -- 172.21.15.52:0/429503335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380107f0 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f363800be20 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f36257fa640 1 -- 172.21.15.52:0/429503335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3638010990 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f36257fa640 1 -- 172.21.15.52:0/429503335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f3638010b90 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f3646fdc740 10 client.19401 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d1ffe28620 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.720+0000 7f36257fa640 1 -- 172.21.15.52:0/429503335 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f363800be20 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f36257fa640 1 -- 172.21.15.52:0/429503335 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f3638023070 con 0x55d2000eab70 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3646fdc740 10 client.19401 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3646fdc740 10 client.19401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3646fdc740 20 client.19401 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3646fdc740 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d200114c50 0x55d200117070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3646fdc740 4 client.19401 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:15.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d20005bf10 con 0x55d200114c50 2024-06-02T23:26:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f3641789640 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d200114c50 0x55d200117070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.721+0000 7f36257fa640 10 client.19401 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:15.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.722+0000 7f3641789640 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d200114c50 0x55d200117070 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3983dabb990beafe server_cookie=3ade2a3bb3d5a907 in_seq=0 out_seq=0 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f36257fa640 1 -- 172.21.15.52:0/429503335 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d20005bf10 con 0x55d200114c50 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f36257fa640 10 client.19401 handle_command_reply: tid=0 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 client.19401 shutdown 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 msgr2=0x55d200110140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d200110140 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f363800bfd0 tx=0x7f36380056a0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 shutdown_connections 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d200114c50 0x55d200117070 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000eab70 0x55d200110140 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 --2- 172.21.15.52:0/429503335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000e5010 0x55d200108bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 >> 172.21.15.52:0/429503335 conn(0x55d2000e0680 msgr2=0x55d2000df7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f36257fa640 0 client.19401 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 shutdown_connections 2024-06-02T23:26:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 1 -- 172.21.15.52:0/429503335 wait complete. 2024-06-02T23:26:15.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.723+0000 7f3646fdc740 20 client.19401 trim_cache size 0 max 16384 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 Processor -- start 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 -- start start 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 0x55d200107910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200110060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d2000e6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f363c06fad0 con 0x55d2000e5010 2024-06-02T23:26:15.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f363c071730 con 0x55d2000e5a00 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3646fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f363c0701a0 con 0x55d2000eab70 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f364278b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 0x55d200107910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f364278b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 0x55d200107910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58082/0 (socket says 172.21.15.52:58082) 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f364278b640 1 -- 172.21.15.52:0/3118915737 learned_addr learned my addr 172.21.15.52:0/3118915737 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f3641789640 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200110060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.748+0000 7f364278b640 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d2000e6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f364278b640 1 -- 172.21.15.52:0/3118915737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 msgr2=0x55d2000e6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f364278b640 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d2000e6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f364278b640 1 -- 172.21.15.52:0/3118915737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 msgr2=0x55d200110060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f364278b640 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200110060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f364278b640 1 -- 172.21.15.52:0/3118915737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2000e5410 con 0x55d2000e5010 2024-06-02T23:26:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f3641789640 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f364278b640 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 0x55d200107910 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f363800fcb0 tx=0x7f3638000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3128ceee1d073 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f36277fe640 1 -- 172.21.15.52:0/3118915737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3638035040 con 0x55d2000e5010 2024-06-02T23:26:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f36277fe640 1 -- 172.21.15.52:0/3118915737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3638010d30 con 0x55d2000e5010 2024-06-02T23:26:15.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.749+0000 7f36277fe640 1 -- 172.21.15.52:0/3118915737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36380109b0 con 0x55d2000e5010 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 -- 172.21.15.52:0/3118915737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 msgr2=0x55d200107910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 0x55d200107910 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f363800fcb0 tx=0x7f3638000bc0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 -- 172.21.15.52:0/3118915737 shutdown_connections 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d2000e6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5a00 0x55d200110060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 --2- 172.21.15.52:0/3118915737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5010 0x55d200107910 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 -- 172.21.15.52:0/3118915737 >> 172.21.15.52:0/3118915737 conn(0x55d2000e0680 msgr2=0x55d200108b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 -- 172.21.15.52:0/3118915737 shutdown_connections 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.750+0000 7f3646fdc740 1 -- 172.21.15.52:0/3118915737 wait complete. 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 -- start start 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 Processor -- start 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d20010f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 0x55d20010fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d20010c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f363c070fa0 con 0x55d2000e5a00 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f363c0708a0 con 0x55d2000e5010 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f363c06dab0 con 0x55d2000eab70 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3646fdc740 10 client.0 _ll_register_callbacks cb 0x7f3638015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3641789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 0x55d20010fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f364278b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d20010f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3641789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 0x55d20010fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58096/0 (socket says 172.21.15.52:58096) 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f3641789640 1 -- 172.21.15.52:0/1715820744 learned_addr learned my addr 172.21.15.52:0/1715820744 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.751+0000 7f364278b640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d20010c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3641789640 1 -- 172.21.15.52:0/1715820744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 msgr2=0x55d20010c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3641789640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d20010c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3641789640 1 -- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 msgr2=0x55d20010f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3641789640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d20010f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3641789640 1 -- 172.21.15.52:0/1715820744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d20010cb80 con 0x55d2000e5a00 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f364278b640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d20010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3641789640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 0x55d20010fe90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x55d2001178b0 tx=0x7f362c012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5daa37572d872c78 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f36257fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f36257fa640 1 -- 172.21.15.52:0/1715820744 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f362c0035f0 con 0x55d2000e5a00 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f36257fa640 1 -- 172.21.15.52:0/1715820744 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f362c003790 con 0x55d2000e5a00 2024-06-02T23:26:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3638010d30 con 0x55d2000e5a00 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f36257fa640 1 -- 172.21.15.52:0/1715820744 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f362c003990 con 0x55d2000e5a00 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.752+0000 7f3646fdc740 10 client.19407 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55d20017e640 con 0x55d2000e5a00 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f36257fa640 1 -- 172.21.15.52:0/1715820744 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3638010d30 con 0x55d2000e5a00 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f36257fa640 1 -- 172.21.15.52:0/1715820744 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f362c027070 con 0x55d2000e5a00 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 10 client.19407 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 10 client.19407 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 20 client.19407 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d20017f0b0 0x55d2001814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 4 client.19407 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d200055c20 con 0x55d20017f0b0 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.753+0000 7f364278b640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d20017f0b0 0x55d2001814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.754+0000 7f36257fa640 10 client.19407 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:15.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.754+0000 7f364278b640 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d20017f0b0 0x55d2001814d0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb1a0b42094d67e6 server_cookie=e8c8e27a95f9e7d3 in_seq=0 out_seq=0 2024-06-02T23:26:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.799+0000 7f36257fa640 1 -- 172.21.15.52:0/1715820744 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d200055c20 con 0x55d20017f0b0 2024-06-02T23:26:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.799+0000 7f36257fa640 10 client.19407 handle_command_reply: tid=0 2024-06-02T23:26:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.799+0000 7f3646fdc740 1 client.19407 shutdown 2024-06-02T23:26:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.799+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 msgr2=0x55d20010fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.799+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 0x55d20010fe90 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x55d2001178b0 tx=0x7f362c012a40 comp rx=0 tx=0).stop 2024-06-02T23:26:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 shutdown_connections 2024-06-02T23:26:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d20017f0b0 0x55d2001814d0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f36257fa640 0 client.19407 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2000eab70 0x55d20010c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d2000e5a00 0x55d20010fe90 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 --2- 172.21.15.52:0/1715820744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d2000e5010 0x55d20010f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 >> 172.21.15.52:0/1715820744 conn(0x55d2000e0680 msgr2=0x55d2000e6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 shutdown_connections 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 1 -- 172.21.15.52:0/1715820744 wait complete. 2024-06-02T23:26:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.800+0000 7f3646fdc740 20 client.19407 trim_cache size 0 max 16384 2024-06-02T23:26:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:26:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:26:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.801+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36280a2cb0 msgr2=0x7f36280a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.801+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36280a2cb0 0x7f36280a5170 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f36340ee390 tx=0x7f36340f5040 comp rx=0 tx=0).stop 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.801+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 msgr2=0x7f363c199d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.801+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 0x7f363c199d40 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f36340319d0 tx=0x7f3634003ba0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 shutdown_connections 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f36280a2cb0 0x7f36280a5170 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f363c1015b0 0x7f363c1a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f363c100c10 0x7f363c19a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 --2- 172.21.15.52:0/3651367621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f363c1002b0 0x7f363c199d40 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 >> 172.21.15.52:0/3651367621 conn(0x7f363c0fbe70 msgr2=0x7f363c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.802+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 shutdown_connections 2024-06-02T23:26:15.805 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:15.803+0000 7f36441ee640 1 -- 172.21.15.52:0/3651367621 wait complete. 2024-06-02T23:26:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:15 smithi174 ceph-mon[35994]: pgmap v862: 97 pgs: 97 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 237 op/s 2024-06-02T23:26:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:15 smithi123 ceph-mon[35977]: pgmap v862: 97 pgs: 97 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 237 op/s 2024-06-02T23:26:16.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 -- 172.21.15.52:0/2196495159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101390 msgr2=0x7f1910101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:16.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 --2- 172.21.15.52:0/2196495159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101390 0x7f1910101810 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f1900009990 tx=0x7f1900009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 -- 172.21.15.52:0/2196495159 shutdown_connections 2024-06-02T23:26:16.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 --2- 172.21.15.52:0/2196495159 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1910101d50 0x7f191010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 --2- 172.21.15.52:0/2196495159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 0x7f19101077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 --2- 172.21.15.52:0/2196495159 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101390 0x7f1910101810 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.267+0000 7f1917464700 1 -- 172.21.15.52:0/2196495159 >> 172.21.15.52:0/2196495159 conn(0x7f1910078340 msgr2=0x7f1910078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:16.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.268+0000 7f1917464700 1 -- 172.21.15.52:0/2196495159 shutdown_connections 2024-06-02T23:26:16.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.268+0000 7f1917464700 1 -- 172.21.15.52:0/2196495159 wait complete. 2024-06-02T23:26:16.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 Processor -- start 2024-06-02T23:26:16.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 -- start start 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1910101390 0x7f191019e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 0x7f191019ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 0x7f1910198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1910112630 con 0x7f1910101d50 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19101124b0 con 0x7f1910107400 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1917464700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f19101127b0 con 0x7f1910101390 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1915200700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1910101390 0x7f191019e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:16.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1915a01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 0x7f1910198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f19149ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 0x7f191019ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1915a01700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 0x7f1910198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38336/0 (socket says 172.21.15.52:38336) 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f19149ff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 0x7f191019ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58118/0 (socket says 172.21.15.52:58118) 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.269+0000 7f1915a01700 1 -- 172.21.15.52:0/546837177 learned_addr learned my addr 172.21.15.52:0/546837177 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19149ff700 1 -- 172.21.15.52:0/546837177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1910101390 msgr2=0x7f191019e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19149ff700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1910101390 0x7f191019e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19149ff700 1 -- 172.21.15.52:0/546837177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 msgr2=0x7f1910198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19149ff700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 0x7f1910198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19149ff700 1 -- 172.21.15.52:0/546837177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19101990d0 con 0x7f1910101d50 2024-06-02T23:26:16.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19149ff700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 0x7f191019ecf0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f19000059d0 tx=0x7f1900004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:16.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1900027070 con 0x7f1910101d50 2024-06-02T23:26:16.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1910199360 con 0x7f1910101d50 2024-06-02T23:26:16.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19101a5680 con 0x7f1910101d50 2024-06-02T23:26:16.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.270+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1900022470 con 0x7f1910101d50 2024-06-02T23:26:16.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.271+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190000b610 con 0x7f1910101d50 2024-06-02T23:26:16.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.272+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1910069180 con 0x7f1910101d50 2024-06-02T23:26:16.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.273+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f190002a070 con 0x7f1910101d50 2024-06-02T23:26:16.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.274+0000 7f19067fc700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18fc0a2ca0 0x7f18fc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:16.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.275+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f19000cfce0 con 0x7f1910101d50 2024-06-02T23:26:16.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.275+0000 7f1915200700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18fc0a2ca0 0x7f18fc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:16.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.275+0000 7f1915200700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18fc0a2ca0 0x7f18fc0a5160 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1910199ab0 tx=0x7f190c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:16.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.280+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f190005e850 con 0x7f1910101d50 2024-06-02T23:26:16.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 11548 9.45 MB/sec warmup 84 sec latency 77.855 ms 2024-06-02T23:26:16.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.707+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f191002dca0 con 0x7f1910101d50 2024-06-02T23:26:16.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.708+0000 7f19067fc700 1 -- 172.21.15.52:0/546837177 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7f1900013070 con 0x7f1910101d50 2024-06-02T23:26:16.709 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:26:16.709 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:26:16.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18fc0a2ca0 msgr2=0x7f18fc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:16.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18fc0a2ca0 0x7f18fc0a5160 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f1910199ab0 tx=0x7f190c009260 comp rx=0 tx=0).stop 2024-06-02T23:26:16.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 msgr2=0x7f191019ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:16.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 0x7f191019ecf0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f19000059d0 tx=0x7f1900004790 comp rx=0 tx=0).stop 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 shutdown_connections 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f18fc0a2ca0 0x7f18fc0a5160 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1910107400 0x7f1910198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1910101390 0x7f191019e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 --2- 172.21.15.52:0/546837177 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1910101d50 0x7f191019ecf0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 >> 172.21.15.52:0/546837177 conn(0x7f1910078340 msgr2=0x7f19100fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 shutdown_connections 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:16.712+0000 7f1917464700 1 -- 172.21.15.52:0/546837177 wait complete. 2024-06-02T23:26:16.713 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:26:17.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 11702 9.43 MB/sec warmup 85 sec latency 77.774 ms 2024-06-02T23:26:17.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:17 smithi052 ceph-mon[32340]: pgmap v863: 97 pgs: 97 active+clean; 337 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 275 op/s 2024-06-02T23:26:17.844 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:17 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/546837177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:17.881 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-02T23:26:17.881 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-02T23:26:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:17 smithi174 ceph-mon[35994]: pgmap v863: 97 pgs: 97 active+clean; 337 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 275 op/s 2024-06-02T23:26:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:17 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/546837177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:17 smithi123 ceph-mon[35977]: pgmap v863: 97 pgs: 97 active+clean; 337 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 275 op/s 2024-06-02T23:26:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:17 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/546837177' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.111+0000 7f7b0a23c700 1 -- 172.21.15.52:0/3725259906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101380 msgr2=0x7f7b04101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.111+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/3725259906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101380 0x7f7b04101800 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009990 tx=0x7f7af8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 -- 172.21.15.52:0/3725259906 shutdown_connections 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/3725259906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101d40 0x7f7b0410e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/3725259906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b041073f0 0x7f7b041077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/3725259906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b04101380 0x7f7b04101800 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 -- 172.21.15.52:0/3725259906 >> 172.21.15.52:0/3725259906 conn(0x7f7b04078300 msgr2=0x7f7b04078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 -- 172.21.15.52:0/3725259906 shutdown_connections 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.112+0000 7f7b0a23c700 1 -- 172.21.15.52:0/3725259906 wait complete. 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.113+0000 7f7b0a23c700 1 Processor -- start 2024-06-02T23:26:18.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.113+0000 7f7b0a23c700 1 -- start start 2024-06-02T23:26:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.113+0000 7f7b0a23c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101380 0x7f7b0419f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b0a23c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 0x7f7b0419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b0a23c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b041073f0 0x7f7b041a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101380 0x7f7b0419f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 0x7f7b0419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 0x7f7b0419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47214/0 (socket says 172.21.15.52:47214) 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b0a23c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110480 con 0x7f7b041073f0 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 -- 172.21.15.52:0/1935951331 learned_addr learned my addr 172.21.15.52:0/1935951331 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110300 con 0x7f7b04101380 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b04110600 con 0x7f7b04101d40 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b03fff700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b041073f0 0x7f7b041a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 -- 172.21.15.52:0/1935951331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101380 msgr2=0x7f7b0419f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101380 0x7f7b0419f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 -- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b041073f0 msgr2=0x7f7b041a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b041073f0 0x7f7b041a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b02ffd700 1 -- 172.21.15.52:0/1935951331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b041a4110 con 0x7f7b04101d40 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b037fe700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101380 0x7f7b0419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.114+0000 7f7b03fff700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b041073f0 0x7f7b041a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.115+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 0x7f7b0419f640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7af8000c00 tx=0x7f7af80047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.115+0000 7f7b00ff9700 1 -- 172.21.15.52:0/1935951331 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af8004230 con 0x7f7b04101d40 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.115+0000 7f7b00ff9700 1 -- 172.21.15.52:0/1935951331 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af8004ce0 con 0x7f7b04101d40 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.115+0000 7f7b00ff9700 1 -- 172.21.15.52:0/1935951331 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af80052c0 con 0x7f7b04101d40 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.115+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b041a43a0 con 0x7f7b04101d40 2024-06-02T23:26:18.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.115+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b041a47d0 con 0x7f7b04101d40 2024-06-02T23:26:18.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.119+0000 7f7b00ff9700 1 -- 172.21.15.52:0/1935951331 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7af8009d10 con 0x7f7b04101d40 2024-06-02T23:26:18.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.120+0000 7f7b00ff9700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7aec0a2df0 0x7f7aec0a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.120+0000 7f7b037fe700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7aec0a2df0 0x7f7aec0a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.120+0000 7f7b00ff9700 1 -- 172.21.15.52:0/1935951331 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7af80cfc60 con 0x7f7b04101d40 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 Processor -- start 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 -- start start 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b037fe700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7aec0a2df0 0x7f7aec0a52b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7af4011320 tx=0x7f7af400d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d193c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d19900 0x55e100d132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d13820 0x55e100d13c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e100c05fe0 con 0x55e100d18fe0 2024-06-02T23:26:18.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e100c648d0 con 0x55e100d13820 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b14abfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e100bcc4e0 con 0x55e100d19900 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d193c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d193c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58156/0 (socket says 172.21.15.52:58156) 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b037fe700 1 -- 172.21.15.52:0/576213242 learned_addr learned my addr 172.21.15.52:0/576213242 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d19900 0x55e100d132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.121+0000 7f7b03fff700 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d13820 0x55e100d13c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7b037fe700 1 -- 172.21.15.52:0/576213242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d19900 msgr2=0x55e100d132e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7b037fe700 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d19900 0x55e100d132e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7b037fe700 1 -- 172.21.15.52:0/576213242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d13820 msgr2=0x55e100d13c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7b037fe700 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d13820 0x55e100d13c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7b037fe700 1 -- 172.21.15.52:0/576213242 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e100bb3950 con 0x55e100d18fe0 2024-06-02T23:26:18.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7b037fe700 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d193c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f7af40174a0 tx=0x7f7af4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c56dde374a7bbbe6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:18.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7ae9ffb700 1 -- 172.21.15.52:0/576213242 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af40215f0 con 0x55e100d18fe0 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7ae9ffb700 1 -- 172.21.15.52:0/576213242 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af401b070 con 0x55e100d18fe0 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.122+0000 7f7ae9ffb700 1 -- 172.21.15.52:0/576213242 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4015960 con 0x55e100d18fe0 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 -- 172.21.15.52:0/576213242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 msgr2=0x55e100d193c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d193c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f7af40174a0 tx=0x7f7af4014000 comp rx=0 tx=0).stop 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 -- 172.21.15.52:0/576213242 shutdown_connections 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d13820 0x55e100d13c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d193c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/576213242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d19900 0x55e100d132e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 -- 172.21.15.52:0/576213242 >> 172.21.15.52:0/576213242 conn(0x55e100c40100 msgr2=0x55e100d0e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.123+0000 7f7b14abfb80 1 -- 172.21.15.52:0/576213242 shutdown_connections 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.124+0000 7f7b14abfb80 1 -- 172.21.15.52:0/576213242 wait complete. 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.124+0000 7f7b14abfb80 1 -- start start 2024-06-02T23:26:18.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.124+0000 7f7b14abfb80 1 Processor -- start 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d1c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d1ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e100c05040 con 0x55e100d18fe0 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e100c60890 con 0x55e100d19900 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e100d202e0 con 0x55e100d13820 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b14abfb80 10 client.0 _ll_register_callbacks cb 0x55e100bfa630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47234/0 (socket says 172.21.15.52:47234) 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b037fe700 1 -- 172.21.15.52:0/86704266 learned_addr learned my addr 172.21.15.52:0/86704266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d1c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.125+0000 7f7b03fff700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d1ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b037fe700 1 -- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 msgr2=0x55e100d1ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b037fe700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d1ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b037fe700 1 -- 172.21.15.52:0/86704266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 msgr2=0x55e100d1c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b037fe700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d1c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b037fe700 1 -- 172.21.15.52:0/86704266 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e100c2a7d0 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b037fe700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005640 tx=0x7f7af4005670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb7520fa5ffbf03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7ae2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/86704266 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7af4004bc0 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7af401b070 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/86704266 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7af4004d60 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.126+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/86704266 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7af403a070 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 10 client.28504 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e100d2aa60 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/86704266 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7af401b070 con 0x55e100d13820 2024-06-02T23:26:18.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/86704266 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f7af4031020 con 0x55e100d13820 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 10 client.28504 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 10 client.28504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 20 client.28504 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d2d310 0x55e100d2f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 4 client.28504 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e100c71dd0 con 0x55e100d2d310 2024-06-02T23:26:18.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.127+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d2d310 0x55e100d2f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.128+0000 7f7ae2ffd700 10 client.28504 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:18.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.128+0000 7f7b02ffd700 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d2d310 0x55e100d2f790 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7de925a8b54ff2 server_cookie=a3dab1e8c6441bf8 in_seq=0 out_seq=0 2024-06-02T23:26:18.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/86704266 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e100c71dd0 con 0x55e100d2d310 2024-06-02T23:26:18.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7ae2ffd700 10 client.28504 handle_command_reply: tid=0 2024-06-02T23:26:18.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 client.28504 shutdown 2024-06-02T23:26:18.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 msgr2=0x55e100d1f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f670 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7af4005640 tx=0x7f7af4005670 comp rx=0 tx=0).stop 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 shutdown_connections 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d2d310 0x55e100d2f790 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d1ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f670 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/86704266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d1c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 >> 172.21.15.52:0/86704266 conn(0x55e100c40100 msgr2=0x55e100d0d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.129+0000 7f7ae2ffd700 0 client.28504 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.130+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 shutdown_connections 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.130+0000 7f7b14abfb80 1 -- 172.21.15.52:0/86704266 wait complete. 2024-06-02T23:26:18.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.130+0000 7f7b14abfb80 20 client.28504 trim_cache size 0 max 16384 2024-06-02T23:26:18.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.164+0000 7f7b14abfb80 1 Processor -- start 2024-06-02T23:26:18.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.164+0000 7f7b14abfb80 1 -- start start 2024-06-02T23:26:18.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d18fe0 0x55e100d0df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 0x55e100d1c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 0x55e100d1c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 0x55e100d1c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58186/0 (socket says 172.21.15.52:58186) 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 -- 172.21.15.52:0/2867756585 learned_addr learned my addr 172.21.15.52:0/2867756585 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b3030 con 0x55e100d19900 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b4c90 con 0x55e100d18fe0 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b3700 con 0x55e100d13820 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 -- 172.21.15.52:0/2867756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 msgr2=0x55e100d1f1b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f1b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 -- 172.21.15.52:0/2867756585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d18fe0 msgr2=0x55e100d0df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d18fe0 0x55e100d0df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.165+0000 7f7b03fff700 1 -- 172.21.15.52:0/2867756585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e100d202e0 con 0x55e100d19900 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.166+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 0x55e100d1c930 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00ba40 tx=0x7f7afc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f834a82031450ba1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.166+0000 7f7ae97fa700 1 -- 172.21.15.52:0/2867756585 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc00d2b0 con 0x55e100d19900 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.166+0000 7f7ae97fa700 1 -- 172.21.15.52:0/2867756585 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc01e070 con 0x55e100d19900 2024-06-02T23:26:18.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.166+0000 7f7ae97fa700 1 -- 172.21.15.52:0/2867756585 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc019d50 con 0x55e100d19900 2024-06-02T23:26:18.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 msgr2=0x55e100d1c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 0x55e100d1c930 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00ba40 tx=0x7f7afc00b040 comp rx=0 tx=0).stop 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 shutdown_connections 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d19900 0x55e100d1c930 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d1f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2867756585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d18fe0 0x55e100d0df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 >> 172.21.15.52:0/2867756585 conn(0x55e100c40100 msgr2=0x55e100d1c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 shutdown_connections 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.167+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2867756585 wait complete. 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.168+0000 7f7b14abfb80 1 -- start start 2024-06-02T23:26:18.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.168+0000 7f7b14abfb80 1 Processor -- start 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d25780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d25cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d25780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b037fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d25780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47246/0 (socket says 172.21.15.52:47246) 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b02ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d25cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d22570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b02ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d25cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58188/0 (socket says 172.21.15.52:58188) 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b02ffd700 1 -- 172.21.15.52:0/2354964797 learned_addr learned my addr 172.21.15.52:0/2354964797 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b4500 con 0x55e100d18fe0 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b3e00 con 0x55e100d19900 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7b041b0ff0 con 0x55e100d13820 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b14abfb80 10 client.0 _ll_register_callbacks cb 0x55e100d1be00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.169+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d22570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b03fff700 1 -- 172.21.15.52:0/2354964797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 msgr2=0x55e100d25780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d25780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b03fff700 1 -- 172.21.15.52:0/2354964797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 msgr2=0x55e100d25cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d25cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b03fff700 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e100d22c20 con 0x55e100d19900 2024-06-02T23:26:18.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b03fff700 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d22570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00dd70 tx=0x7f7afc0196b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a5ae3e8011200e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:18.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7ae2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:26:18.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/2354964797 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc029020 con 0x55e100d19900 2024-06-02T23:26:18.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/2354964797 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7afc003c00 con 0x55e100d19900 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.170+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7afc01e070 con 0x55e100d19900 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.171+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/2354964797 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f7afc019e20 con 0x55e100d19900 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.171+0000 7f7b14abfb80 10 client.28832 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.171+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e100d98a60 con 0x55e100d19900 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.171+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/2354964797 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7afc003e00 con 0x55e100d19900 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.171+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/2354964797 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f7afc004250 con 0x55e100d19900 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b14abfb80 10 client.28832 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b14abfb80 10 client.28832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b14abfb80 20 client.28832 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d99120 0x55e100d9b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b14abfb80 4 client.28832 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:26:18.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7afc01e070 con 0x55e100d99120 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.172+0000 7f7b037fe700 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d99120 0x55e100d9b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.173+0000 7f7b037fe700 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d99120 0x55e100d9b540 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560e7cefca204c3f server_cookie=efca5cd15e01d129 in_seq=0 out_seq=0 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.173+0000 7f7ae2ffd700 10 client.28832 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.173+0000 7f7ae2ffd700 1 -- 172.21.15.52:0/2354964797 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7afc01e070 con 0x55e100d99120 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.173+0000 7f7ae2ffd700 10 client.28832 handle_command_reply: tid=0 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.173+0000 7f7b14abfb80 1 client.28832 shutdown 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.173+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 msgr2=0x55e100d22570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d22570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7afc00dd70 tx=0x7f7afc0196b0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 shutdown_connections 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e100d99120 0x55e100d9b540 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e100d19900 0x55e100d22570 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e100d13820 0x55e100d25780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.175 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 --2- 172.21.15.52:0/2354964797 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e100d18fe0 0x55e100d25cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 >> 172.21.15.52:0/2354964797 conn(0x55e100c40100 msgr2=0x55e100d1ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7ae2ffd700 0 client.28832 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 shutdown_connections 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 1 -- 172.21.15.52:0/2354964797 wait complete. 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.174+0000 7f7b14abfb80 20 client.28832 trim_cache size 0 max 16384 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "c29a69a6-a0f8-4ee2-afe7-9a819f5dbffc", 2024-06-02T23:26:18.176 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:26:18.177 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:26:18.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7aec0a2df0 msgr2=0x7f7aec0a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7aec0a2df0 0x7f7aec0a52b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7af4011320 tx=0x7f7af400d040 comp rx=0 tx=0).stop 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 msgr2=0x7f7b0419f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 0x7f7b0419f640 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f7af8000c00 tx=0x7f7af80047b0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 shutdown_connections 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7aec0a2df0 0x7f7aec0a52b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7b041073f0 0x7f7b041a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b04101380 0x7f7b0419f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 --2- 172.21.15.52:0/1935951331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7b04101d40 0x7f7b0419f640 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 >> 172.21.15.52:0/1935951331 conn(0x7f7b04078300 msgr2=0x7f7b040fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 shutdown_connections 2024-06-02T23:26:18.178 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:18.175+0000 7f7b0a23c700 1 -- 172.21.15.52:0/1935951331 wait complete. 2024-06-02T23:26:18.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 11864 9.40 MB/sec warmup 86 sec latency 84.584 ms 2024-06-02T23:26:18.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:26:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:26:18.240+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004b8 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX/ auth{4=1} v2812 ap=2 f(v70 m2024-06-02T23:26:05.287435+0000 24=24+0) n(v4 rc2024-06-02T23:26:05.287435+0000 b5230239 25=24+1) (inest sync->mix g=4) caps={15216=pAsLsXsFs/-@1956} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9816680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004b8:[<0x1000000033f/PARADOX v108>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004b8:[<0x1000000033f/PARADOX v2812>,<0x10000000335/~dmtmp v592>,<0x10000000007/client3 v2434>,<0x10000000006/clients v1664>,<0x10000000005/tmp v1466>,<0x10000000003/client.0 v1426>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1402>,<0x10000000001/sv_0 v1403>,<0x10000000000/qa v1344>,<0x1/volumes v1361>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:05.247881+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:05.287435+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v70 m2024-06-02T23:26:05.287435+0000 24=24+0)","memory_value.rstat":"n(v4 rc2024-06-02T23:26:05.287435+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:26:18.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:26:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:26:18.246+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD/ auth{3=1} v2225 ap=2 f(v48 m2024-06-02T23:26:01.757143+0000 5=5+0) n(v4 rc2024-06-02T23:26:01.757143+0000 b814326 6=5+1) (inest sync->mix g=3) caps={15216=pAsLsXsFs/-@1546} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ca0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000036d:[<0x10000000336/WORD v8>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000036d:[<0x10000000336/WORD v2225>,<0x10000000331/~dmtmp v984>,<0x10000000007/client1 v2432>,<0x10000000006/clients v1664>,<0x10000000005/tmp v1466>,<0x10000000003/client.0 v1426>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1402>,<0x10000000001/sv_0 v1403>,<0x10000000000/qa v1344>,<0x1/volumes v1361>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:01.717429+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:01.757143+0000 b814326 6=5+1)","memory_value.dirstat":"f(v48 m2024-06-02T23:26:01.757143+0000 5=5+0)","memory_value.rstat":"n(v4 rc2024-06-02T23:26:01.757143+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:26:18.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:26:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:26:18.255+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX/ auth{3=1} v2227 ap=2 f(v70 m2024-06-02T23:26:07.910030+0000 24=24+0) n(v4 rc2024-06-02T23:26:07.910030+0000 b5230239 25=24+1) (inest sync->mix g=3) caps={15216=pAsLsXsFs/-@1927} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb97ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004ac:[<0x10000000336/PARADOX v108>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004ac:[<0x10000000336/PARADOX v2227>,<0x10000000331/~dmtmp v988>,<0x10000000007/client1 v2432>,<0x10000000006/clients v1664>,<0x10000000005/tmp v1466>,<0x10000000003/client.0 v1426>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1402>,<0x10000000001/sv_0 v1403>,<0x10000000000/qa v1344>,<0x1/volumes v1361>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:07.866663+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:07.910030+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v70 m2024-06-02T23:26:07.910030+0000 24=24+0)","memory_value.rstat":"n(v4 rc2024-06-02T23:26:07.910030+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:26:18.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:26:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:26:18.259+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT/ auth{3=1} v2223 ap=2 f(v56 m2024-06-02T23:25:59.772367+0000 7=7+0) n(v4 rc2024-06-02T23:25:59.772367+0000 b1462582 8=7+1) (inest sync->mix g=3) caps={15216=pAsLsXsFs/-@1544} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b64100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003ac:[<0x10000000336/PWRPNT v214>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003ac:[<0x10000000336/PWRPNT v2223>,<0x10000000331/~dmtmp v988>,<0x10000000007/client1 v2432>,<0x10000000006/clients v1664>,<0x10000000005/tmp v1466>,<0x10000000003/client.0 v1426>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1402>,<0x10000000001/sv_0 v1403>,<0x10000000000/qa v1344>,<0x1/volumes v1361>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:25:59.690775+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:25:59.772367+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v56 m2024-06-02T23:25:59.772367+0000 7=7+0)","memory_value.rstat":"n(v4 rc2024-06-02T23:25:59.772367+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:26:18.996 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:26:19.106 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:19.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12060 9.46 MB/sec warmup 87 sec latency 69.503 ms 2024-06-02T23:26:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: pgmap v864: 97 pgs: 97 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 220 op/s 2024-06-02T23:26:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:26:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:19 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: pgmap v864: 97 pgs: 97 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 220 op/s 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:19.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:19 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: pgmap v864: 97 pgs: 97 active+clean; 336 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 220 op/s 2024-06-02T23:26:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:26:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:26:20.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:26:20.009 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.009 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:26:20.009 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:26:20.009 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.009 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:19 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:26:20.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12245 9.49 MB/sec warmup 88 sec latency 81.389 ms 2024-06-02T23:26:21.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 -- 172.21.15.52:0/805800687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c107850 msgr2=0x7fb37c10f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 --2- 172.21.15.52:0/805800687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c107850 0x7fb37c10f3d0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fb37400b0d0 tx=0x7fb37400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 -- 172.21.15.52:0/805800687 shutdown_connections 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 --2- 172.21.15.52:0/805800687 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c107850 0x7fb37c10f3d0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 --2- 172.21.15.52:0/805800687 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06a3d0 0x7fb37c06a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 --2- 172.21.15.52:0/805800687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c06ad80 0x7fb37c107120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 -- 172.21.15.52:0/805800687 >> 172.21.15.52:0/805800687 conn(0x7fb37c078330 msgr2=0x7fb37c078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 -- 172.21.15.52:0/805800687 shutdown_connections 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.176+0000 7fb3818cd700 1 -- 172.21.15.52:0/805800687 wait complete. 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 Processor -- start 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 -- start start 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c06a3d0 0x7fb37c101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 0x7fb37c1019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c107850 0x7fb37c101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c112640 con 0x7fb37c06a3d0 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb3818cd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1124c0 con 0x7fb37c107850 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 0x7fb37c1019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.177+0000 7fb37affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c06a3d0 0x7fb37c101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 0x7fb37c1019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47256/0 (socket says 172.21.15.52:47256) 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37a7fc700 1 -- 172.21.15.52:0/3132745187 learned_addr learned my addr 172.21.15.52:0/3132745187 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1127c0 con 0x7fb37c06ad80 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c107850 0x7fb37c101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 -- 172.21.15.52:0/3132745187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 msgr2=0x7fb37c1019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 0x7fb37c1019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 -- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c06a3d0 msgr2=0x7fb37c101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c06a3d0 0x7fb37c101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 -- 172.21.15.52:0/3132745187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1a6f10 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 0x7fb37c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb37b7fe700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c107850 0x7fb37c101ee0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb37400b0d0 tx=0x7fb374007ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.178+0000 7fb3808cb700 1 -- 172.21.15.52:0/3132745187 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37401f070 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.179+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1a71a0 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.179+0000 7fb3808cb700 1 -- 172.21.15.52:0/3132745187 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb374007d90 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.179+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1a7650 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.179+0000 7fb3808cb700 1 -- 172.21.15.52:0/3132745187 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37401b690 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.182+0000 7fb3808cb700 1 -- 172.21.15.52:0/3132745187 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb374022070 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.183+0000 7fb3808cb700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3640a2d40 0x7fb3640a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.183+0000 7fb37affd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3640a2d40 0x7fb3640a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.183+0000 7fb3808cb700 1 -- 172.21.15.52:0/3132745187 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb37400f020 con 0x7fb37c107850 2024-06-02T23:26:21.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 Processor -- start 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 -- start start 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac7da0 0x55646fac8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37affd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3640a2d40 0x7fb3640a5200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009bf0 tx=0x7fb36c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fac1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fac2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55646f9ac0f0 con 0x55646fac24f0 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55646fa0a9e0 con 0x55646fac7da0 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb38c150b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55646f9725f0 con 0x55646fac86c0 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fac1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fac1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47270/0 (socket says 172.21.15.52:47270) 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 -- 172.21.15.52:0/2918716565 learned_addr learned my addr 172.21.15.52:0/2918716565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:21.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37b7fe700 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fac2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37affd700 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac7da0 0x55646fac8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 -- 172.21.15.52:0/2918716565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac7da0 msgr2=0x55646fac8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac7da0 0x55646fac8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 -- 172.21.15.52:0/2918716565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 msgr2=0x55646fac2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fac2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.184+0000 7fb37a7fc700 1 -- 172.21.15.52:0/2918716565 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646f959a60 con 0x55646fac86c0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.185+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fac1fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb370009df0 tx=0x7fb3700106b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a634d681150cbbf7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.185+0000 7fb361ffb700 1 -- 172.21.15.52:0/2918716565 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370010e10 con 0x55646fac86c0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.185+0000 7fb361ffb700 1 -- 172.21.15.52:0/2918716565 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb37001c930 con 0x55646fac86c0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.185+0000 7fb361ffb700 1 -- 172.21.15.52:0/2918716565 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370011960 con 0x55646fac86c0 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 -- 172.21.15.52:0/2918716565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 msgr2=0x55646fac1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fac1fb0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb370009df0 tx=0x7fb3700106b0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 -- 172.21.15.52:0/2918716565 shutdown_connections 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fac2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac7da0 0x55646fac8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 --2- 172.21.15.52:0/2918716565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fac1fb0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 -- 172.21.15.52:0/2918716565 >> 172.21.15.52:0/2918716565 conn(0x55646f9e6210 msgr2=0x55646fab4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 -- 172.21.15.52:0/2918716565 shutdown_connections 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 -- 172.21.15.52:0/2918716565 wait complete. 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 -- start start 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.186+0000 7fb38c150b80 1 Processor -- start 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646face420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac86c0 0x55646facbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55646f9ab150 con 0x55646fac86c0 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55646fa069a0 con 0x55646fac24f0 2024-06-02T23:26:21.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55646facf0a0 con 0x55646fac7da0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb38c150b80 10 client.0 _ll_register_callbacks cb 0x55646f9a0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac86c0 0x55646facbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facb720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47274/0 (socket says 172.21.15.52:47274) 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 -- 172.21.15.52:0/4159144152 learned_addr learned my addr 172.21.15.52:0/4159144152 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37affd700 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646face420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 -- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 msgr2=0x55646face420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646face420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 -- 172.21.15.52:0/4159144152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac86c0 msgr2=0x55646facbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac86c0 0x55646facbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 -- 172.21.15.52:0/4159144152 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646f9d08e0 con 0x55646fac7da0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.187+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facb720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb37001dda0 tx=0x7fb37001ddd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf774f5e861db21d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:21.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb35affd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb35affd700 1 -- 172.21.15.52:0/4159144152 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370005480 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb37001c930 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb35affd700 1 -- 172.21.15.52:0/4159144152 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb370005620 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb35affd700 1 -- 172.21.15.52:0/4159144152 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb37001c740 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 10 client.28519 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55646fad9640 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb35affd700 1 -- 172.21.15.52:0/4159144152 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb37001c930 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb35affd700 1 -- 172.21.15.52:0/4159144152 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fb370024030 con 0x55646fac7da0 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 10 client.28519 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 10 client.28519 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 20 client.28519 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.188+0000 7fb38c150b80 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646faddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.189+0000 7fb38c150b80 4 client.28519 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:21.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.189+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55646fa17ee0 con 0x55646fadbb70 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.189+0000 7fb37affd700 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646faddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.189+0000 7fb35affd700 10 client.28519 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.189+0000 7fb37affd700 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646faddf60 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3852f9febdea7dc9 server_cookie=2bca02ff87ee4c92 in_seq=0 out_seq=0 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb35affd700 1 -- 172.21.15.52:0/4159144152 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55646fa17ee0 con 0x55646fadbb70 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb35affd700 10 client.28519 handle_command_reply: tid=0 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 client.28519 shutdown 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 msgr2=0x55646facb720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facb720 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb37001dda0 tx=0x7fb37001ddd0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 shutdown_connections 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646faddf60 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac86c0 0x55646facbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb35affd700 0 client.28519 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:21.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646face420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 --2- 172.21.15.52:0/4159144152 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facb720 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 >> 172.21.15.52:0/4159144152 conn(0x55646f9e6210 msgr2=0x55646fab3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.190+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 shutdown_connections 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.191+0000 7fb38c150b80 1 -- 172.21.15.52:0/4159144152 wait complete. 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.191+0000 7fb38c150b80 20 client.28519 trim_cache size 0 max 16384 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.235+0000 7fb38c150b80 1 Processor -- start 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.235+0000 7fb38c150b80 1 -- start start 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fab8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac86c0 0x55646fae3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb38c150b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1aa660 con 0x55646fac24f0 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb38c150b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ac2c0 con 0x55646fac86c0 2024-06-02T23:26:21.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb38c150b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1aad30 con 0x55646fac7da0 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fab8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fab8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58242/0 (socket says 172.21.15.52:58242) 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37affd700 1 -- 172.21.15.52:0/11010076 learned_addr learned my addr 172.21.15.52:0/11010076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37b7fe700 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac86c0 0x55646fae3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37a7fc700 1 -- 172.21.15.52:0/11010076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac86c0 msgr2=0x55646fae3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac86c0 0x55646fae3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37a7fc700 1 -- 172.21.15.52:0/11010076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 msgr2=0x55646fab8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fab8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37a7fc700 1 -- 172.21.15.52:0/11010076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55646facec20 con 0x55646fac7da0 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.236+0000 7fb37affd700 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fab8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:21.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.237+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facdf60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb370011cb0 tx=0x7fb370009640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da92f6466284b492 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.237+0000 7fb3617fa700 1 -- 172.21.15.52:0/11010076 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb370027310 con 0x55646fac7da0 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.237+0000 7fb3617fa700 1 -- 172.21.15.52:0/11010076 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb370005280 con 0x55646fac7da0 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.237+0000 7fb3617fa700 1 -- 172.21.15.52:0/11010076 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37001cd50 con 0x55646fac7da0 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 -- 172.21.15.52:0/11010076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 msgr2=0x55646facdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facdf60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb370011cb0 tx=0x7fb370009640 comp rx=0 tx=0).stop 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 -- 172.21.15.52:0/11010076 shutdown_connections 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac86c0 0x55646fae3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac24f0 0x55646fab8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 --2- 172.21.15.52:0/11010076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac7da0 0x55646facdf60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 -- 172.21.15.52:0/11010076 >> 172.21.15.52:0/11010076 conn(0x55646f9e6210 msgr2=0x55646fad7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 -- 172.21.15.52:0/11010076 shutdown_connections 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 -- 172.21.15.52:0/11010076 wait complete. 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.238+0000 7fb38c150b80 1 -- start start 2024-06-02T23:26:21.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 Processor -- start 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646fad4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac7da0 0x55646fad2c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fad3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1abb30 con 0x55646fac7da0 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ab430 con 0x55646fac24f0 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c19c750 con 0x55646fac86c0 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb38c150b80 10 client.0 _ll_register_callbacks cb 0x55646f9a0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646fad4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fad3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac7da0 0x55646fad2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646fad4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38418/0 (socket says 172.21.15.52:38418) 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fad3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47290/0 (socket says 172.21.15.52:47290) 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37affd700 1 -- 172.21.15.52:0/3151042672 learned_addr learned my addr 172.21.15.52:0/3151042672 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.239+0000 7fb37a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac7da0 0x55646fad2c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58250/0 (socket says 172.21.15.52:58250) 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb37affd700 1 -- 172.21.15.52:0/3151042672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 msgr2=0x55646fad3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb37affd700 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fad3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb37affd700 1 -- 172.21.15.52:0/3151042672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac7da0 msgr2=0x55646fad2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb37affd700 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac7da0 0x55646fad2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb37affd700 1 -- 172.21.15.52:0/3151042672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55646fad3730 con 0x55646fac24f0 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb37affd700 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646fad4880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0124c0 tx=0x7fb36c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6de030477b84441e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:21.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb35affd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb35affd700 1 -- 172.21.15.52:0/3151042672 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c004250 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb35affd700 1 -- 172.21.15.52:0/3151042672 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb36c0043f0 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.240+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb370005280 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb35affd700 1 -- 172.21.15.52:0/3151042672 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7fb36c003680 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 10 client.28850 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55646fb47550 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb35affd700 1 -- 172.21.15.52:0/3151042672 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb36c003cd0 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb35affd700 1 -- 172.21.15.52:0/3151042672 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7fb36c02b980 con 0x55646fac24f0 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 10 client.28850 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 10 client.28850 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 20 client.28850 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646fb49e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:21.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 4 client.28850 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-02T23:26:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fb370005280 con 0x55646fadbb70 2024-06-02T23:26:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.241+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646fb49e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.242+0000 7fb35affd700 10 client.28850 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:21.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.242+0000 7fb37a7fc700 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646fb49e50 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b645fdf28142764 server_cookie=4eb2b99cb4a6140f in_seq=0 out_seq=0 2024-06-02T23:26:21.435 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:21.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12358 9.40 MB/sec warmup 89 sec latency 225.754 ms 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb35affd700 1 -- 172.21.15.52:0/3151042672 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fb370005280 con 0x55646fadbb70 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb35affd700 10 client.28850 handle_command_reply: tid=0 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 client.28850 shutdown 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 msgr2=0x55646fad4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646fad4880 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0124c0 tx=0x7fb36c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 shutdown_connections 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55646fadbb70 0x55646fb49e50 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55646fac86c0 0x55646fad3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55646fac24f0 0x55646fad4880 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 --2- 172.21.15.52:0/3151042672 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55646fac7da0 0x55646fad2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 >> 172.21.15.52:0/3151042672 conn(0x55646f9e6210 msgr2=0x55646faccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb35affd700 0 client.28850 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 shutdown_connections 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 1 -- 172.21.15.52:0/3151042672 wait complete. 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.506+0000 7fb38c150b80 20 client.28850 trim_cache size 0 max 16384 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 37, 2024-06-02T23:26:21.508 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 50, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "376ccfc3": { 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 50, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:26:21.509 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:21.510 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3640a2d40 msgr2=0x7fb3640a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3640a2d40 0x7fb3640a5200 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fb36c009bf0 tx=0x7fb36c009260 comp rx=0 tx=0).stop 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c107850 msgr2=0x7fb37c101ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c107850 0x7fb37c101ee0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb37400b0d0 tx=0x7fb374007ab0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 shutdown_connections 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3640a2d40 0x7fb3640a5200 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c107850 0x7fb37c101ee0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb37c06a3d0 0x7fb37c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 --2- 172.21.15.52:0/3132745187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb37c06ad80 0x7fb37c1019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.507+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 >> 172.21.15.52:0/3132745187 conn(0x7fb37c078330 msgr2=0x7fb37c0ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:21.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.508+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 shutdown_connections 2024-06-02T23:26:21.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:21.508+0000 7fb3818cd700 1 -- 172.21.15.52:0/3132745187 wait complete. 2024-06-02T23:26:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:21 smithi174 ceph-mon[35994]: pgmap v865: 97 pgs: 97 active+clean; 330 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-06-02T23:26:21.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:21 smithi052 ceph-mon[32340]: pgmap v865: 97 pgs: 97 active+clean; 330 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-06-02T23:26:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:21 smithi123 ceph-mon[35977]: pgmap v865: 97 pgs: 97 active+clean; 330 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 295 op/s 2024-06-02T23:26:22.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 9.30 MB/sec warmup 90 sec latency 1046.907 ms 2024-06-02T23:26:23.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 9.20 MB/sec warmup 91 sec latency 2046.939 ms 2024-06-02T23:26:23.766 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '376ccfc3', quiesce took 10.5 seconds. Will release after: 19.5 2024-06-02T23:26:23.854 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:23 smithi052 ceph-mon[32340]: pgmap v866: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 341 op/s 2024-06-02T23:26:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:23 smithi174 ceph-mon[35994]: pgmap v866: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 341 op/s 2024-06-02T23:26:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 -- 172.21.15.52:0/2359663266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c101ed0 msgr2=0x7f534c1022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 --2- 172.21.15.52:0/2359663266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c101ed0 0x7f534c1022b0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f5348009990 tx=0x7f5348009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 -- 172.21.15.52:0/2359663266 shutdown_connections 2024-06-02T23:26:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 --2- 172.21.15.52:0/2359663266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c10ef70 0x7f534c111360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 --2- 172.21.15.52:0/2359663266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c101ed0 0x7f534c1022b0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 --2- 172.21.15.52:0/2359663266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f534c1027f0 0x7f534c10ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 -- 172.21.15.52:0/2359663266 >> 172.21.15.52:0/2359663266 conn(0x7f534c077400 msgr2=0x7f534c077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 -- 172.21.15.52:0/2359663266 shutdown_connections 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.966+0000 7f535396f700 1 -- 172.21.15.52:0/2359663266 wait complete. 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 Processor -- start 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 -- start start 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f534c101ed0 0x7f534c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c1027f0 0x7f534c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 0x7f534c19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f534c114860 con 0x7f534c10ef70 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1146e0 con 0x7f534c1027f0 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f535396f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1149e0 con 0x7f534c101ed0 2024-06-02T23:26:23.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f5351f0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 0x7f534c19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.967+0000 7f5351f0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 0x7f534c19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51048/0 (socket says 172.21.15.52:51048) 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 -- 172.21.15.52:0/2385636213 learned_addr learned my addr 172.21.15.52:0/2385636213 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f535170b700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f534c101ed0 0x7f534c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5350f0a700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c1027f0 0x7f534c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 -- 172.21.15.52:0/2385636213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f534c101ed0 msgr2=0x7f534c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f534c101ed0 0x7f534c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 -- 172.21.15.52:0/2385636213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c1027f0 msgr2=0x7f534c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c1027f0 0x7f534c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 -- 172.21.15.52:0/2385636213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c19fd80 con 0x7f534c10ef70 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5350f0a700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c1027f0 0x7f534c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:23.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f5351f0c700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 0x7f534c19f640 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f53440094f0 tx=0x7f534400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f53427fc700 1 -- 172.21.15.52:0/2385636213 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534400ddd0 con 0x7f534c10ef70 2024-06-02T23:26:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1a0070 con 0x7f534c10ef70 2024-06-02T23:26:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f53427fc700 1 -- 172.21.15.52:0/2385636213 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5344009e20 con 0x7f534c10ef70 2024-06-02T23:26:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f53427fc700 1 -- 172.21.15.52:0/2385636213 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5344015400 con 0x7f534c10ef70 2024-06-02T23:26:23.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.968+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1a78f0 con 0x7f534c10ef70 2024-06-02T23:26:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.971+0000 7f53427fc700 1 -- 172.21.15.52:0/2385636213 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5344004ad0 con 0x7f534c10ef70 2024-06-02T23:26:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.971+0000 7f53427fc700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f53380a2c20 0x7f53380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.971+0000 7f53427fc700 1 -- 172.21.15.52:0/2385636213 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f53440ce9e0 con 0x7f534c10ef70 2024-06-02T23:26:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.971+0000 7f535170b700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f53380a2c20 0x7f53380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.972+0000 7f535170b700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f53380a2c20 0x7f53380a50e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f534800b5c0 tx=0x7f5348005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:23.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.972+0000 7f535e1f2b80 1 Processor -- start 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.972+0000 7f535e1f2b80 1 -- start start 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d29340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d23260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d237a0 0x559391d23bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559391c15f60 con 0x559391d29880 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559391c74850 con 0x559391d237a0 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559391bdc460 con 0x559391d28f60 2024-06-02T23:26:23.979 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d23260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d23260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51056/0 (socket says 172.21.15.52:51056) 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 -- 172.21.15.52:0/3225494427 learned_addr learned my addr 172.21.15.52:0/3225494427 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5351f0c700 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d237a0 0x559391d23bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f535170b700 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d29340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 -- 172.21.15.52:0/3225494427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 msgr2=0x559391d29340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d29340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 -- 172.21.15.52:0/3225494427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d237a0 msgr2=0x559391d23bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d237a0 0x559391d23bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 -- 172.21.15.52:0/3225494427 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559391bc38d0 con 0x559391d29880 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5351f0c700 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d237a0 0x559391d23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f5350f0a700 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d23260 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f533c00bce0 tx=0x7f533c00f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f127717ce490ca5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.973+0000 7f53377fe700 1 -- 172.21.15.52:0/3225494427 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c015070 con 0x559391d29880 2024-06-02T23:26:23.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f53377fe700 1 -- 172.21.15.52:0/3225494427 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f533c00fc90 con 0x559391d29880 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f53377fe700 1 -- 172.21.15.52:0/3225494427 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f533c011730 con 0x559391d29880 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3225494427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 msgr2=0x559391d23260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d23260 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f533c00bce0 tx=0x7f533c00f360 comp rx=0 tx=0).stop 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3225494427 shutdown_connections 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d237a0 0x559391d23bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d29340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3225494427 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d23260 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3225494427 >> 172.21.15.52:0/3225494427 conn(0x559391c50080 msgr2=0x559391d1e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3225494427 shutdown_connections 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.974+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3225494427 wait complete. 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 -- start start 2024-06-02T23:26:23.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 Processor -- start 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d2f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d29880 0x559391d2ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559391c14fc0 con 0x559391d237a0 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559391c70810 con 0x559391d28f60 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559391d30260 con 0x559391d29880 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535e1f2b80 10 client.0 _ll_register_callbacks cb 0x559391c0a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535170b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d2f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535170b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d2f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51066/0 (socket says 172.21.15.52:51066) 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.975+0000 7f535170b700 1 -- 172.21.15.52:0/2793227931 learned_addr learned my addr 172.21.15.52:0/2793227931 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f5350f0a700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f5351f0c700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d29880 0x559391d2ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535170b700 1 -- 172.21.15.52:0/2793227931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d29880 msgr2=0x559391d2ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535170b700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d29880 0x559391d2ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535170b700 1 -- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 msgr2=0x559391d2c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535170b700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535170b700 1 -- 172.21.15.52:0/2793227931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559391c3a750 con 0x559391d237a0 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535170b700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d2f670 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5348015000 tx=0x7f5348009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98668d0f87d61cd9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f5334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f5334ff9700 1 -- 172.21.15.52:0/2793227931 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5348034060 con 0x559391d237a0 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f5334ff9700 1 -- 172.21.15.52:0/2793227931 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f534802f640 con 0x559391d237a0 2024-06-02T23:26:23.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.976+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f533c00fc90 con 0x559391d237a0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f5334ff9700 1 -- 172.21.15.52:0/2793227931 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f534802f840 con 0x559391d237a0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 10 client.19461 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559391d3aa60 con 0x559391d237a0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f5334ff9700 1 -- 172.21.15.52:0/2793227931 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f533c00fc90 con 0x559391d237a0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f5334ff9700 1 -- 172.21.15.52:0/2793227931 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f5348024970 con 0x559391d237a0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 10 client.19461 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 10 client.19461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 20 client.19461 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391d3d0f0 0x559391d3f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 4 client.19461 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559391c81d50 con 0x559391d3d0f0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.977+0000 7f5350f0a700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391d3d0f0 0x559391d3f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:23.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.978+0000 7f5350f0a700 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391d3d0f0 0x559391d3f560 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d671fb0050673c1a server_cookie=e5fa4e5590742340 in_seq=0 out_seq=0 2024-06-02T23:26:23.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.978+0000 7f5334ff9700 10 client.19461 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f5334ff9700 1 -- 172.21.15.52:0/2793227931 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559391c81d50 con 0x559391d3d0f0 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f5334ff9700 10 client.19461 handle_command_reply: tid=0 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 client.19461 shutdown 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 msgr2=0x559391d2f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d2f670 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f5348015000 tx=0x7f5348009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 shutdown_connections 2024-06-02T23:26:23.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391d3d0f0 0x559391d3f560 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d29880 0x559391d2ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d2f670 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/2793227931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 >> 172.21.15.52:0/2793227931 conn(0x559391c50080 msgr2=0x559391d1d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f5334ff9700 0 client.19461 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 shutdown_connections 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 1 -- 172.21.15.52:0/2793227931 wait complete. 2024-06-02T23:26:23.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:23.979+0000 7f535e1f2b80 20 client.19461 trim_cache size 0 max 16384 2024-06-02T23:26:24.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 Processor -- start 2024-06-02T23:26:24.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 -- start start 2024-06-02T23:26:24.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d237a0 0x559391d2baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d1de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1aecc0 con 0x559391d29880 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1b0920 con 0x559391d28f60 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1af390 con 0x559391d237a0 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d1de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d1de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51082/0 (socket says 172.21.15.52:51082) 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 -- 172.21.15.52:0/1026891871 learned_addr learned my addr 172.21.15.52:0/1026891871 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f535170b700 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d237a0 0x559391d2baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5350f0a700 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 -- 172.21.15.52:0/1026891871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d237a0 msgr2=0x559391d2baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d237a0 0x559391d2baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 -- 172.21.15.52:0/1026891871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 msgr2=0x559391d2e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.001+0000 7f5351f0c700 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.002+0000 7f5351f0c700 1 -- 172.21.15.52:0/1026891871 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559391d26ab0 con 0x559391d29880 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.002+0000 7f5350f0a700 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.002+0000 7f5351f0c700 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d1de80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f53440cc880 tx=0x7f534409b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da7d6198d132f1d5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.002+0000 7f5336ffd700 1 -- 172.21.15.52:0/1026891871 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53440b4070 con 0x559391d29880 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.002+0000 7f5336ffd700 1 -- 172.21.15.52:0/1026891871 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f534409f620 con 0x559391d29880 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.002+0000 7f5336ffd700 1 -- 172.21.15.52:0/1026891871 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534409f170 con 0x559391d29880 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 -- 172.21.15.52:0/1026891871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 msgr2=0x559391d1de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d1de80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f53440cc880 tx=0x7f534409b9f0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 -- 172.21.15.52:0/1026891871 shutdown_connections 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d29880 0x559391d1de80 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d237a0 0x559391d2baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/1026891871 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d28f60 0x559391d2e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 -- 172.21.15.52:0/1026891871 >> 172.21.15.52:0/1026891871 conn(0x559391c50080 msgr2=0x559391d2e1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:24.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 -- 172.21.15.52:0/1026891871 shutdown_connections 2024-06-02T23:26:24.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 -- 172.21.15.52:0/1026891871 wait complete. 2024-06-02T23:26:24.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 -- start start 2024-06-02T23:26:24.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.003+0000 7f535e1f2b80 1 Processor -- start 2024-06-02T23:26:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:23 smithi123 ceph-mon[35977]: pgmap v866: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 341 op/s 2024-06-02T23:26:24.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d35a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d30df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d29880 0x559391d31330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1b0190 con 0x559391d237a0 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1afa90 con 0x559391d29880 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f534c1acc80 con 0x559391d28f60 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 10 client.0 _ll_register_callbacks cb 0x559391c0a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d35a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d35a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51086/0 (socket says 172.21.15.52:51086) 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 -- 172.21.15.52:0/3152379275 learned_addr learned my addr 172.21.15.52:0/3152379275 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f5350f0a700 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d30df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 -- 172.21.15.52:0/3152379275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 msgr2=0x559391d30df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d30df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 -- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d29880 msgr2=0x559391d31330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d29880 0x559391d31330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 -- 172.21.15.52:0/3152379275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559391d31970 con 0x559391d237a0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535170b700 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d35a70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5348000c00 tx=0x7f534802fd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294cdab6c42349ea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f5334ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f5334ff9700 1 -- 172.21.15.52:0/3152379275 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f534803f070 con 0x559391d237a0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f5334ff9700 1 -- 172.21.15.52:0/3152379275 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f53480245e0 con 0x559391d237a0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f534409f620 con 0x559391d237a0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.004+0000 7f5334ff9700 1 -- 172.21.15.52:0/3152379275 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5348034040 con 0x559391d237a0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 10 client.19467 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x559391da7c80 con 0x559391d237a0 2024-06-02T23:26:24.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f5334ff9700 1 -- 172.21.15.52:0/3152379275 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f534409f620 con 0x559391d237a0 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f5334ff9700 1 -- 172.21.15.52:0/3152379275 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f5348042020 con 0x559391d237a0 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 10 client.19467 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 10 client.19467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 20 client.19467 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391da8360 0x559391daa780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 4 client.19467 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559391c7c2d0 con 0x559391da8360 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.005+0000 7f5350f0a700 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391da8360 0x559391daa780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f5350f0a700 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391da8360 0x559391daa780 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=949fbfb4b5496d51 server_cookie=2ea44c518e8c7733 in_seq=0 out_seq=0 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f5334ff9700 10 client.19467 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f5334ff9700 1 -- 172.21.15.52:0/3152379275 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559391c7c2d0 con 0x559391da8360 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f5334ff9700 10 client.19467 handle_command_reply: tid=0 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 client.19467 shutdown 2024-06-02T23:26:24.013 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 msgr2=0x559391d35a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d35a70 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f5348000c00 tx=0x7f534802fd40 comp rx=0 tx=0).stop 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 shutdown_connections 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559391da8360 0x559391daa780 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559391d29880 0x559391d31330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559391d237a0 0x559391d35a70 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 --2- 172.21.15.52:0/3152379275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559391d28f60 0x559391d30df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 >> 172.21.15.52:0/3152379275 conn(0x559391c50080 msgr2=0x559391d22250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f5334ff9700 0 client.19467 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 shutdown_connections 2024-06-02T23:26:24.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.006+0000 7f535e1f2b80 1 -- 172.21.15.52:0/3152379275 wait complete. 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535e1f2b80 20 client.19467 trim_cache size 0 max 16384 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: "c29a69a6-a0f8-4ee2-afe7-9a819f5dbffc": { 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "c29a69a6-a0f8-4ee2-afe7-9a819f5dbffc", 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:24.015 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f53380a2c20 msgr2=0x7f53380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f53380a2c20 0x7f53380a50e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f534800b5c0 tx=0x7f5348005960 comp rx=0 tx=0).stop 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 msgr2=0x7f534c19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 0x7f534c19f640 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f53440094f0 tx=0x7f534400d3b0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 shutdown_connections 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f53380a2c20 0x7f53380a50e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f534c10ef70 0x7f534c19f640 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f534c101ed0 0x7f534c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 --2- 172.21.15.52:0/2385636213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f534c1027f0 0x7f534c19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:24.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 >> 172.21.15.52:0/2385636213 conn(0x7f534c077400 msgr2=0x7f534c110420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:24.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 shutdown_connections 2024-06-02T23:26:24.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:24.007+0000 7f535396f700 1 -- 172.21.15.52:0/2385636213 wait complete. 2024-06-02T23:26:24.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 9.10 MB/sec warmup 92 sec latency 3046.981 ms 2024-06-02T23:26:24.826 INFO:tasks.cephfs.filesystem:scrub status for tag:c29a69a6-a0f8-4ee2-afe7-9a819f5dbffc - {'path': '/', 'tag': 'c29a69a6-a0f8-4ee2-afe7-9a819f5dbffc', 'options': 'recursive,force'} 2024-06-02T23:26:25.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 9.00 MB/sec warmup 93 sec latency 4047.032 ms 2024-06-02T23:26:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:25 smithi174 ceph-mon[35994]: pgmap v867: 97 pgs: 97 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-06-02T23:26:25.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:25 smithi052 ceph-mon[32340]: pgmap v867: 97 pgs: 97 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-06-02T23:26:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:25 smithi123 ceph-mon[35977]: pgmap v867: 97 pgs: 97 active+clean; 329 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 332 op/s 2024-06-02T23:26:26.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.90 MB/sec warmup 94 sec latency 5047.125 ms 2024-06-02T23:26:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:26 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:26:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:26 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:26:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:26 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:26:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:26 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:26:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:26 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:26:26.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:26 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:26:26.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:26 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:26:26.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:26 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:26:26.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:26 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:26:26.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:26 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:26:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:26 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:26:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:26 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:26:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:26 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:26:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:26 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:26:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:26 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:26:27.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.81 MB/sec warmup 95 sec latency 6047.246 ms 2024-06-02T23:26:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:27 smithi174 ceph-mon[35994]: pgmap v868: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-06-02T23:26:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:27 smithi052 ceph-mon[32340]: pgmap v868: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-06-02T23:26:28.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:27 smithi123 ceph-mon[35977]: pgmap v868: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.2 MiB/s wr, 329 op/s 2024-06-02T23:26:28.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.72 MB/sec warmup 96 sec latency 7047.323 ms 2024-06-02T23:26:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 -- 172.21.15.52:0/3874863503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0107780 msgr2=0x7f44a0109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 --2- 172.21.15.52:0/3874863503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0107780 0x7f44a0109b70 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f4498009ca0 tx=0x7f449802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 -- 172.21.15.52:0/3874863503 shutdown_connections 2024-06-02T23:26:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 --2- 172.21.15.52:0/3874863503 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0107780 0x7f44a0109b70 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 --2- 172.21.15.52:0/3874863503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a0104e50 0x7f44a0107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.910 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 --2- 172.21.15.52:0/3874863503 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0102520 0x7f44a0104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 -- 172.21.15.52:0/3874863503 >> 172.21.15.52:0/3874863503 conn(0x7f44a00fbe70 msgr2=0x7f44a00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 -- 172.21.15.52:0/3874863503 shutdown_connections 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.908+0000 7f44a72a5640 1 -- 172.21.15.52:0/3874863503 wait complete. 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 Processor -- start 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 -- start start 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 0x7f44a0197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0104e50 0x7f44a01980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a0107780 0x7f44a019f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d550 con 0x7f44a0102520 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d3d0 con 0x7f44a0107780 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a72a5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44a010d6d0 con 0x7f44a0104e50 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a5041640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 0x7f44a0197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a5041640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 0x7f44a0197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51106/0 (socket says 172.21.15.52:51106) 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a5041640 1 -- 172.21.15.52:0/1291169237 learned_addr learned my addr 172.21.15.52:0/1291169237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:28.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a5842640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a0107780 0x7f44a019f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.909+0000 7f44a4840640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0104e50 0x7f44a01980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a5041640 1 -- 172.21.15.52:0/1291169237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0104e50 msgr2=0x7f44a01980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a5041640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0104e50 0x7f44a01980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a5041640 1 -- 172.21.15.52:0/1291169237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a0107780 msgr2=0x7f44a019f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a5041640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a0107780 0x7f44a019f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a5041640 1 -- 172.21.15.52:0/1291169237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44a019f7c0 con 0x7f44a0102520 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a4840640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0104e50 0x7f44a01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a5041640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 0x7f44a0197ba0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f449c00d770 tx=0x7f449c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44967fc640 1 -- 172.21.15.52:0/1291169237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c017070 con 0x7f44a0102520 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44967fc640 1 -- 172.21.15.52:0/1291169237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f449c0054d0 con 0x7f44a0102520 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44a019fab0 con 0x7f44a0102520 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44967fc640 1 -- 172.21.15.52:0/1291169237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c004cf0 con 0x7f44a0102520 2024-06-02T23:26:28.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.910+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44a01a0000 con 0x7f44a0102520 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.912+0000 7f44967fc640 1 -- 172.21.15.52:0/1291169237 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f449c004e90 con 0x7f44a0102520 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 Processor -- start 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 -- start start 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c340430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 0x56092c345500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c345b70 0x56092c348000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56092c2a34f0 con 0x56092c340a00 2024-06-02T23:26:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56092c241870 con 0x56092c340010 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56092c29f4b0 con 0x56092c345b70 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a5041640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 0x56092c345500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44967fc640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f448c0a2cb0 0x7f448c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a5041640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 0x56092c345500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51122/0 (socket says 172.21.15.52:51122) 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.913+0000 7f44a5041640 1 -- 172.21.15.52:0/2431024538 learned_addr learned my addr 172.21.15.52:0/2431024538 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5842640 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c340430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5842640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f448c0a2cb0 0x7f448c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a4840640 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c345b70 0x56092c348000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44967fc640 1 -- 172.21.15.52:0/1291169237 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f449c0cea00 con 0x7f44a0102520 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5041640 1 -- 172.21.15.52:0/2431024538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c345b70 msgr2=0x56092c348000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5041640 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c345b70 0x56092c348000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5041640 1 -- 172.21.15.52:0/2431024538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 msgr2=0x56092c340430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5041640 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c340430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5041640 1 -- 172.21.15.52:0/2431024538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092c25d9b0 con 0x56092c340a00 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5842640 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c340430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5041640 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 0x56092c345500 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f449c0c6a80 tx=0x7f449c005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5322fb09ee940ac4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f44a5842640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f448c0a2cb0 0x7f448c0a5170 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4498009430 tx=0x7f4498005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f448b7fe640 1 -- 172.21.15.52:0/2431024538 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c020070 con 0x56092c340a00 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f448b7fe640 1 -- 172.21.15.52:0/2431024538 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f449c009c30 con 0x56092c340a00 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.914+0000 7f448b7fe640 1 -- 172.21.15.52:0/2431024538 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f449c097900 con 0x56092c340a00 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2431024538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 msgr2=0x56092c345500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 0x56092c345500 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f449c0c6a80 tx=0x7f449c005680 comp rx=0 tx=0).stop 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2431024538 shutdown_connections 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c345b70 0x56092c348000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c340a00 0x56092c345500 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2431024538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c340430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2431024538 >> 172.21.15.52:0/2431024538 conn(0x56092c33b680 msgr2=0x56092c33ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2431024538 shutdown_connections 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2431024538 wait complete. 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.915+0000 7f44a9fbc740 1 -- start start 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 Processor -- start 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56092c348ac0 con 0x56092c345b70 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56092c22cae0 con 0x56092c340010 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56092c348c40 con 0x56092c340a00 2024-06-02T23:26:28.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a9fbc740 10 client.0 _ll_register_callbacks cb 0x56092c2659b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51136/0 (socket says 172.21.15.52:51136) 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 -- 172.21.15.52:0/1548437257 learned_addr learned my addr 172.21.15.52:0/1548437257 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a4840640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5041640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 -- 172.21.15.52:0/1548437257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 msgr2=0x56092c34a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 -- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 msgr2=0x56092c34a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.916+0000 7f44a5842640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f44a5842640 1 -- 172.21.15.52:0/1548437257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092bfb7210 con 0x56092c345b70 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f44a5842640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351750 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4498009ca0 tx=0x7f449802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9bb6119f2f9d37 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f4488ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:28.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f4488ff9640 1 -- 172.21.15.52:0/1548437257 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498004450 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f4488ff9640 1 -- 172.21.15.52:0/1548437257 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f44980045f0 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56092c2b6f10 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f4488ff9640 1 -- 172.21.15.52:0/1548437257 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f44980047f0 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f44a9fbc740 10 client.19479 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56092c083620 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.917+0000 7f4488ff9640 1 -- 172.21.15.52:0/1548437257 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x56092c2b6f10 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f4488ff9640 1 -- 172.21.15.52:0/1548437257 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f449804c040 con 0x56092c345b70 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a9fbc740 10 client.19479 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a9fbc740 10 client.19479 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a9fbc740 20 client.19479 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3562b0 0x56092c358710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a9fbc740 4 client.19479 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56092c2b0c20 con 0x56092c3562b0 2024-06-02T23:26:28.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a5041640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3562b0 0x56092c358710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f44a5041640 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3562b0 0x56092c358710 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6db865b9135ccb44 server_cookie=8ec3c708f3738196 in_seq=0 out_seq=0 2024-06-02T23:26:28.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.918+0000 7f4488ff9640 10 client.19479 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f4488ff9640 1 -- 172.21.15.52:0/1548437257 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56092c2b0c20 con 0x56092c3562b0 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f4488ff9640 10 client.19479 handle_command_reply: tid=0 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 client.19479 shutdown 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 msgr2=0x56092c351750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351750 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4498009ca0 tx=0x7f449802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 shutdown_connections 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3562b0 0x56092c358710 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351750 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f4488ff9640 0 client.19479 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1548437257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 >> 172.21.15.52:0/1548437257 conn(0x56092c33b680 msgr2=0x56092c33a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.919+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 shutdown_connections 2024-06-02T23:26:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.920+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1548437257 wait complete. 2024-06-02T23:26:28.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.920+0000 7f44a9fbc740 20 client.19479 trim_cache size 0 max 16384 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.935+0000 7f44a9fbc740 1 Processor -- start 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.935+0000 7f44a9fbc740 1 -- start start 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.935+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.935+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.935+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01a8b70 con 0x56092c345b70 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01aa7d0 con 0x56092c340a00 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01a9240 con 0x56092c340010 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51150/0 (socket says 172.21.15.52:51150) 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 -- 172.21.15.52:0/1543338837 learned_addr learned my addr 172.21.15.52:0/1543338837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a4840640 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5041640 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 -- 172.21.15.52:0/1543338837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340010 msgr2=0x56092c34a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 -- 172.21.15.52:0/1543338837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340a00 msgr2=0x56092c34a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 -- 172.21.15.52:0/1543338837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56092c33b4b0 con 0x56092c345b70 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f44a5842640 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351710 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4498031970 tx=0x7f4498004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800755b1c2fae8ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f448affd640 1 -- 172.21.15.52:0/1543338837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498032050 con 0x56092c345b70 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f448affd640 1 -- 172.21.15.52:0/1543338837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4498053070 con 0x56092c345b70 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.936+0000 7f448affd640 1 -- 172.21.15.52:0/1543338837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44980305c0 con 0x56092c345b70 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1543338837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 msgr2=0x56092c351710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351710 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4498031970 tx=0x7f4498004990 comp rx=0 tx=0).stop 2024-06-02T23:26:28.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1543338837 shutdown_connections 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c351710 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340a00 0x56092c34a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/1543338837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340010 0x56092c34a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1543338837 >> 172.21.15.52:0/1543338837 conn(0x56092c33b680 msgr2=0x56092c345230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1543338837 shutdown_connections 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 -- 172.21.15.52:0/1543338837 wait complete. 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 -- start start 2024-06-02T23:26:28.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.937+0000 7f44a9fbc740 1 Processor -- start 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c3516b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c34e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01aa040 con 0x56092c345b70 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01a9940 con 0x56092c340010 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f44a01a6b50 con 0x56092c340a00 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a9fbc740 10 client.0 _ll_register_callbacks cb 0x7f4498009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c34e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c34e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51158/0 (socket says 172.21.15.52:51158) 2024-06-02T23:26:28.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 -- 172.21.15.52:0/2969533330 learned_addr learned my addr 172.21.15.52:0/2969533330 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a4840640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5041640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c3516b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 -- 172.21.15.52:0/2969533330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 msgr2=0x56092c34e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 -- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 msgr2=0x56092c3516b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c3516b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5842640 1 -- 172.21.15.52:0/2969533330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56092c33c350 con 0x56092c345b70 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.938+0000 7f44a5041640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c3516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a5842640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c34e730 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4498005e30 tx=0x7f4498004a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=191d6b495067fb29 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f4488ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f4488ff9640 1 -- 172.21.15.52:0/2969533330 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4498068020 con 0x56092c345b70 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f4488ff9640 1 -- 172.21.15.52:0/2969533330 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f44980045a0 con 0x56092c345b70 2024-06-02T23:26:28.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4498053070 con 0x56092c345b70 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f4488ff9640 1 -- 172.21.15.52:0/2969533330 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4498004b40 con 0x56092c345b70 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 10 client.19485 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56092c3c0130 con 0x56092c345b70 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f4488ff9640 1 -- 172.21.15.52:0/2969533330 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4498053070 con 0x56092c345b70 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f4488ff9640 1 -- 172.21.15.52:0/2969533330 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f4498062020 con 0x56092c345b70 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 10 client.19485 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 10 client.19485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 20 client.19485 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3c0b60 0x56092c3c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 4 client.19485 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.939+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56092c2aaf70 con 0x56092c3c0b60 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.940+0000 7f44a5041640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3c0b60 0x56092c3c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.940+0000 7f4488ff9640 10 client.19485 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.940+0000 7f44a5041640 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3c0b60 0x56092c3c2f80 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81987403f971a3fc server_cookie=337f5191098b7f09 in_seq=0 out_seq=0 2024-06-02T23:26:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f4488ff9640 1 -- 172.21.15.52:0/2969533330 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56092c2aaf70 con 0x56092c3c0b60 2024-06-02T23:26:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f4488ff9640 10 client.19485 handle_command_reply: tid=0 2024-06-02T23:26:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 client.19485 shutdown 2024-06-02T23:26:28.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 msgr2=0x56092c34e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c34e730 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4498005e30 tx=0x7f4498004a80 comp rx=0 tx=0).stop 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 shutdown_connections 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56092c3c0b60 0x56092c3c2f80 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56092c340a00 0x56092c34e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56092c345b70 0x56092c34e730 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 --2- 172.21.15.52:0/2969533330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56092c340010 0x56092c3516b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 >> 172.21.15.52:0/2969533330 conn(0x56092c33b680 msgr2=0x56092c34a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f4488ff9640 0 client.19485 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 shutdown_connections 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 1 -- 172.21.15.52:0/2969533330 wait complete. 2024-06-02T23:26:28.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.970+0000 7f44a9fbc740 20 client.19485 trim_cache size 0 max 16384 2024-06-02T23:26:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:26:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:26:28.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f448c0a2cb0 msgr2=0x7f448c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f448c0a2cb0 0x7f448c0a5170 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4498009430 tx=0x7f4498005d80 comp rx=0 tx=0).stop 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 msgr2=0x7f44a0197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 0x7f44a0197ba0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f449c00d770 tx=0x7f449c00dc30 comp rx=0 tx=0).stop 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 shutdown_connections 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f448c0a2cb0 0x7f448c0a5170 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f44a0104e50 0x7f44a01980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f44a0107780 0x7f44a019f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 --2- 172.21.15.52:0/1291169237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f44a0102520 0x7f44a0197ba0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 >> 172.21.15.52:0/1291169237 conn(0x7f44a00fbe70 msgr2=0x7f44a0102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 shutdown_connections 2024-06-02T23:26:28.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:28.971+0000 7f44a72a5640 1 -- 172.21.15.52:0/1291169237 wait complete. 2024-06-02T23:26:29.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.63 MB/sec warmup 97 sec latency 8047.371 ms 2024-06-02T23:26:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:29 smithi174 ceph-mon[35994]: pgmap v869: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-06-02T23:26:29.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:29 smithi052 ceph-mon[32340]: pgmap v869: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-06-02T23:26:30.010 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:29 smithi123 ceph-mon[35977]: pgmap v869: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 698 KiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-06-02T23:26:30.450 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.54 MB/sec warmup 98 sec latency 9047.446 ms 2024-06-02T23:26:31.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.45 MB/sec warmup 99 sec latency 10047.551 ms 2024-06-02T23:26:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:31 smithi174 ceph-mon[35994]: pgmap v870: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-06-02T23:26:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:31 smithi052 ceph-mon[32340]: pgmap v870: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-06-02T23:26:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:31 smithi123 ceph-mon[35977]: pgmap v870: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 1.8 MiB/s wr, 251 op/s 2024-06-02T23:26:32.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.37 MB/sec warmup 100 sec latency 11047.662 ms 2024-06-02T23:26:33.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.29 MB/sec warmup 101 sec latency 12047.716 ms 2024-06-02T23:26:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:33 smithi174 ceph-mon[35994]: pgmap v871: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 972 KiB/s wr, 165 op/s 2024-06-02T23:26:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:33 smithi052 ceph-mon[32340]: pgmap v871: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 972 KiB/s wr, 165 op/s 2024-06-02T23:26:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:33 smithi123 ceph-mon[35977]: pgmap v871: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 342 KiB/s rd, 972 KiB/s wr, 165 op/s 2024-06-02T23:26:34.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.20 MB/sec warmup 102 sec latency 13047.832 ms 2024-06-02T23:26:35.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.12 MB/sec warmup 103 sec latency 14047.894 ms 2024-06-02T23:26:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:35 smithi174 ceph-mon[35994]: pgmap v872: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 495 KiB/s wr, 90 op/s 2024-06-02T23:26:35.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:35 smithi052 ceph-mon[32340]: pgmap v872: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 495 KiB/s wr, 90 op/s 2024-06-02T23:26:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:35 smithi123 ceph-mon[35977]: pgmap v872: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 495 KiB/s wr, 90 op/s 2024-06-02T23:26:36.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 8.05 MB/sec warmup 104 sec latency 15048.027 ms 2024-06-02T23:26:37.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.97 MB/sec warmup 105 sec latency 16048.128 ms 2024-06-02T23:26:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:37 smithi174 ceph-mon[35994]: pgmap v873: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 220 KiB/s wr, 46 op/s 2024-06-02T23:26:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:37 smithi052 ceph-mon[32340]: pgmap v873: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 220 KiB/s wr, 46 op/s 2024-06-02T23:26:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:37 smithi123 ceph-mon[35977]: pgmap v873: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 220 KiB/s wr, 46 op/s 2024-06-02T23:26:38.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.89 MB/sec warmup 106 sec latency 17048.209 ms 2024-06-02T23:26:39.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.82 MB/sec warmup 107 sec latency 18048.309 ms 2024-06-02T23:26:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:39 smithi174 ceph-mon[35994]: pgmap v874: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 169 KiB/s wr, 26 op/s 2024-06-02T23:26:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:39 smithi052 ceph-mon[32340]: pgmap v874: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 169 KiB/s wr, 26 op/s 2024-06-02T23:26:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:39 smithi123 ceph-mon[35977]: pgmap v874: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 169 KiB/s wr, 26 op/s 2024-06-02T23:26:40.451 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.75 MB/sec warmup 108 sec latency 19048.366 ms 2024-06-02T23:26:41.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.68 MB/sec warmup 109 sec latency 20048.462 ms 2024-06-02T23:26:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:41 smithi174 ceph-mon[35994]: pgmap v875: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 115 KiB/s wr, 24 op/s 2024-06-02T23:26:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:41 smithi052 ceph-mon[32340]: pgmap v875: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 115 KiB/s wr, 24 op/s 2024-06-02T23:26:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:41 smithi123 ceph-mon[35977]: pgmap v875: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 115 KiB/s wr, 24 op/s 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 -- 172.21.15.52:0/2604629207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 msgr2=0x7f5bb4109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/2604629207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb4109e40 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009ca0 tx=0x7f5bac02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 -- 172.21.15.52:0/2604629207 shutdown_connections 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/2604629207 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb4109e40 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/2604629207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 0x7f5bb4103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/2604629207 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 0x7f5bb4101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 -- 172.21.15.52:0/2604629207 >> 172.21.15.52:0/2604629207 conn(0x7f5bb40fcf90 msgr2=0x7f5bb40ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 -- 172.21.15.52:0/2604629207 shutdown_connections 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.073+0000 7f5bb99ee640 1 -- 172.21.15.52:0/2604629207 wait complete. 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 Processor -- start 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 -- start start 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 0x7f5bb4197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 0x7f5bb4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb410d510 con 0x7f5bb41035b0 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb410d390 con 0x7f5bb4102bf0 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb99ee640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb410d690 con 0x7f5bb4101a00 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb419efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37136/0 (socket says 172.21.15.52:37136) 2024-06-02T23:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb37fe640 1 -- 172.21.15.52:0/3479346485 learned_addr learned my addr 172.21.15.52:0/3479346485 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5babfff640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 0x7f5bb4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.074+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 0x7f5bb4197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb37fe640 1 -- 172.21.15.52:0/3479346485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 msgr2=0x7f5bb4197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 0x7f5bb4197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb37fe640 1 -- 172.21.15.52:0/3479346485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 msgr2=0x7f5bb4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 0x7f5bb4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb37fe640 1 -- 172.21.15.52:0/3479346485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb419f630 con 0x7f5bb41035b0 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5babfff640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 0x7f5bb4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 0x7f5bb4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb419efc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009dc0 tx=0x7f5bac003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb89ec640 1 -- 172.21.15.52:0/3479346485 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac004040 con 0x7f5bb41035b0 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb89ec640 1 -- 172.21.15.52:0/3479346485 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5bac004690 con 0x7f5bb41035b0 2024-06-02T23:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb419f8c0 con 0x7f5bb41035b0 2024-06-02T23:26:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb89ec640 1 -- 172.21.15.52:0/3479346485 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bac02f4c0 con 0x7f5bb41035b0 2024-06-02T23:26:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.075+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb419fe10 con 0x7f5bb41035b0 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bb89ec640 1 -- 172.21.15.52:0/3479346485 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5bac0305c0 con 0x7f5bb41035b0 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 Processor -- start 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 -- start start 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768bc500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768bf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e37681a4f0 con 0x55e3768b7010 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3767b8870 con 0x55e3768b7a00 2024-06-02T23:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3768164b0 con 0x55e3768bcb70 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37140/0 (socket says 172.21.15.52:37140) 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5babfff640 1 -- 172.21.15.52:0/3390903548 learned_addr learned my addr 172.21.15.52:0/3390903548 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5babfff640 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768bf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.078+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768bc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5bb89ec640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba00a2c30 0x7f5ba00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5babfff640 1 -- 172.21.15.52:0/3390903548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 msgr2=0x55e3768bf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5babfff640 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768bf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5babfff640 1 -- 172.21.15.52:0/3390903548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 msgr2=0x55e3768bc500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5babfff640 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768bc500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5babfff640 1 -- 172.21.15.52:0/3390903548 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3767d49b0 con 0x55e3768b7010 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5babfff640 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b7430 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c014da0 tx=0x7f5b9c00ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54366e90b25185e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba00a2c30 0x7f5ba00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5bb89ec640 1 -- 172.21.15.52:0/3479346485 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5bac0f07d0 con 0x7f5bb41035b0 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5ba9ffb640 1 -- 172.21.15.52:0/3390903548 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0140d0 con 0x55e3768b7010 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5ba9ffb640 1 -- 172.21.15.52:0/3390903548 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b9c014560 con 0x55e3768b7010 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.079+0000 7f5ba9ffb640 1 -- 172.21.15.52:0/3390903548 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c004c30 con 0x55e3768b7010 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/3390903548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 msgr2=0x55e3768b7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bb37fe640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba00a2c30 0x7f5ba00a50f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0bcd60 tx=0x7f5bac087c30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b7430 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c014da0 tx=0x7f5b9c00ab40 comp rx=0 tx=0).stop 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/3390903548 shutdown_connections 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768bf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768bc500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/3390903548 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b7430 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/3390903548 >> 172.21.15.52:0/3390903548 conn(0x55e3768b2680 msgr2=0x55e3768b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/3390903548 shutdown_connections 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/3390903548 wait complete. 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.080+0000 7f5bbc7dc740 1 -- start start 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 Processor -- start 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768c12e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7a00 0x55e3768c1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768bcb70 0x55e3768c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e3768bfb20 con 0x55e3768b7010 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e3767a3ae0 con 0x55e3768bcb70 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e3768bfca0 con 0x55e3768b7a00 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bbc7dc740 10 client.0 _ll_register_callbacks cb 0x55e3767dc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768c12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768c12e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37154/0 (socket says 172.21.15.52:37154) 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 -- 172.21.15.52:0/2878372864 learned_addr learned my addr 172.21.15.52:0/2878372864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7a00 0x55e3768c1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 -- 172.21.15.52:0/2878372864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7a00 msgr2=0x55e3768c1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7a00 0x55e3768c1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 -- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768bcb70 msgr2=0x55e3768c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768bcb70 0x55e3768c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.081+0000 7f5babfff640 1 -- 172.21.15.52:0/2878372864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37652e210 con 0x55e3768b7010 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5babfff640 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768c12e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c004750 tx=0x7f5b9c00a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6291fb82b1e82893 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5b92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5b92ffd640 1 -- 172.21.15.52:0/2878372864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c004e80 con 0x55e3768b7010 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5b92ffd640 1 -- 172.21.15.52:0/2878372864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b9c005020 con 0x55e3768b7010 2024-06-02T23:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b9c014560 con 0x55e3768b7010 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5b92ffd640 1 -- 172.21.15.52:0/2878372864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5b9c005220 con 0x55e3768b7010 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5bbc7dc740 10 client.19497 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e3765fa620 con 0x55e3768b7010 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5b92ffd640 1 -- 172.21.15.52:0/2878372864 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b9c014560 con 0x55e3768b7010 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5b92ffd640 1 -- 172.21.15.52:0/2878372864 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f5b9c01f430 con 0x55e3768b7010 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5bbc7dc740 10 client.19497 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5bbc7dc740 10 client.19497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.082+0000 7f5bbc7dc740 20 client.19497 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.083+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e3768cce00 0x55e3768cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.083+0000 7f5bbc7dc740 4 client.19497 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.083+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e37682df10 con 0x55e3768cce00 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.083+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e3768cce00 0x55e3768cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.083+0000 7f5b92ffd640 10 client.19497 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.083+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e3768cce00 0x55e3768cf1f0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c38e0c833779fff2 server_cookie=e8019f79651705bd in_seq=0 out_seq=0 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5b92ffd640 1 -- 172.21.15.52:0/2878372864 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e37682df10 con 0x55e3768cce00 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5b92ffd640 10 client.19497 handle_command_reply: tid=0 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 client.19497 shutdown 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 msgr2=0x55e3768c12e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768c12e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c004750 tx=0x7f5b9c00a420 comp rx=0 tx=0).stop 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 shutdown_connections 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e3768cce00 0x55e3768cf1f0 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7a00 0x55e3768c1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768bcb70 0x55e3768c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/2878372864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768c12e0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5b92ffd640 0 client.19497 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 >> 172.21.15.52:0/2878372864 conn(0x55e3768b2680 msgr2=0x55e3768b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 shutdown_connections 2024-06-02T23:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.084+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/2878372864 wait complete. 2024-06-02T23:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.085+0000 7f5bbc7dc740 20 client.19497 trim_cache size 0 max 16384 2024-06-02T23:26:42.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 Processor -- start 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 -- start start 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41a8b70 con 0x55e3768b7010 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41aa7d0 con 0x55e3768b7a00 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41a9240 con 0x55e3768bcb70 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37162/0 (socket says 172.21.15.52:37162) 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.100+0000 7f5babfff640 1 -- 172.21.15.52:0/1152537613 learned_addr learned my addr 172.21.15.52:0/1152537613 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5babfff640 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.101 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/1152537613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 msgr2=0x55e3768c0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768c0ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/1152537613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 msgr2=0x55e3768b5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5bb2ffd640 1 -- 172.21.15.52:0/1152537613 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3768b1f60 con 0x55e3768b7a00 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768b5b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55e3768cfa20 tx=0x7f5ba4000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13a725e536b5baa7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5ba97fa640 1 -- 172.21.15.52:0/1152537613 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba4004030 con 0x55e3768b7a00 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5ba97fa640 1 -- 172.21.15.52:0/1152537613 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba40035f0 con 0x55e3768b7a00 2024-06-02T23:26:42.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.101+0000 7f5ba97fa640 1 -- 172.21.15.52:0/1152537613 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba401b6a0 con 0x55e3768b7a00 2024-06-02T23:26:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1152537613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 msgr2=0x55e3768b5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768b5b90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x55e3768cfa20 tx=0x7f5ba4000bc0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1152537613 shutdown_connections 2024-06-02T23:26:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768bcb70 0x55e3768c0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e3768b5b90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1152537613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768b7010 0x55e3768b5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1152537613 >> 172.21.15.52:0/1152537613 conn(0x55e3768b2680 msgr2=0x55e3768b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1152537613 shutdown_connections 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1152537613 wait complete. 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 -- start start 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.102+0000 7f5bbc7dc740 1 Processor -- start 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7010 0x55e376932730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e376932c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 0x55e376939cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41aa040 con 0x55e3768bcb70 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41a9940 con 0x55e3768b7a00 2024-06-02T23:26:42.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb41a6b50 con 0x55e3768b7010 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bbc7dc740 10 client.0 _ll_register_callbacks cb 0x7f5b9c014f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 0x55e376939cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 0x55e376939cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37168/0 (socket says 172.21.15.52:37168) 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 -- 172.21.15.52:0/1839762766 learned_addr learned my addr 172.21.15.52:0/1839762766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e376932c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7010 0x55e376932730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 -- 172.21.15.52:0/1839762766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7010 msgr2=0x55e376932730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7010 0x55e376932730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 -- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 msgr2=0x55e376932c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e376932c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 -- 172.21.15.52:0/1839762766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e37693a2a0 con 0x55e3768bcb70 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5babfff640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 0x55e376939cb0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00fbc0 tx=0x7f5b9c00a2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ebe775e7a249010 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5b92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:42.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.103+0000 7f5b92ffd640 1 -- 172.21.15.52:0/1839762766 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c0055d0 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5b92ffd640 1 -- 172.21.15.52:0/1839762766 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b9c01f430 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ba40035f0 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5b92ffd640 1 -- 172.21.15.52:0/1839762766 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f5b9c01fcc0 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 10 client.19503 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55e37693b5b0 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5b92ffd640 1 -- 172.21.15.52:0/1839762766 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ba40035f0 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5b92ffd640 1 -- 172.21.15.52:0/1839762766 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f5b9c0124e0 con 0x55e3768bcb70 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 10 client.19503 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 10 client.19503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 20 client.19503 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e37693bfb0 0x55e37693e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 4 client.19503 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:26:42.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.104+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e376827c20 con 0x55e37693bfb0 2024-06-02T23:26:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.105+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e37693bfb0 0x55e37693e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.105+0000 7f5bb2ffd640 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e37693bfb0 0x55e37693e3d0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=808d15635abc2705 server_cookie=d3a0a9de63615f62 in_seq=0 out_seq=0 2024-06-02T23:26:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.105+0000 7f5b92ffd640 10 client.19503 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5b92ffd640 1 -- 172.21.15.52:0/1839762766 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e376827c20 con 0x55e37693bfb0 2024-06-02T23:26:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5b92ffd640 10 client.19503 handle_command_reply: tid=0 2024-06-02T23:26:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 client.19503 shutdown 2024-06-02T23:26:42.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 msgr2=0x55e376939cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 0x55e376939cb0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c00fbc0 tx=0x7f5b9c00a2f0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 shutdown_connections 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e37693bfb0 0x55e37693e3d0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e3768bcb70 0x55e376939cb0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5b92ffd640 0 client.19503 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e3768b7a00 0x55e376932c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 --2- 172.21.15.52:0/1839762766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e3768b7010 0x55e376932730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 >> 172.21.15.52:0/1839762766 conn(0x55e3768b2680 msgr2=0x55e3768b5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 shutdown_connections 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 1 -- 172.21.15.52:0/1839762766 wait complete. 2024-06-02T23:26:42.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.127+0000 7f5bbc7dc740 20 client.19503 trim_cache size 0 max 16384 2024-06-02T23:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba00a2c30 msgr2=0x7f5ba00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba00a2c30 0x7f5ba00a50f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5bac0bcd60 tx=0x7f5bac087c30 comp rx=0 tx=0).stop 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 msgr2=0x7f5bb419efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb419efc0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f5bac009dc0 tx=0x7f5bac003ba0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 shutdown_connections 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ba00a2c30 0x7f5ba00a50f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5bb41035b0 0x7f5bb419efc0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5bb4102bf0 0x7f5bb4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 --2- 172.21.15.52:0/3479346485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5bb4101a00 0x7f5bb4197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 >> 172.21.15.52:0/3479346485 conn(0x7f5bb40fcf90 msgr2=0x7f5bb4105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 shutdown_connections 2024-06-02T23:26:42.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:42.128+0000 7f5bb99ee640 1 -- 172.21.15.52:0/3479346485 wait complete. 2024-06-02T23:26:42.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.61 MB/sec warmup 110 sec latency 21048.583 ms 2024-06-02T23:26:43.267 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:26:43.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.54 MB/sec warmup 111 sec latency 22048.635 ms 2024-06-02T23:26:43.534 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:43 smithi174 ceph-mon[35994]: pgmap v876: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-06-02T23:26:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:43 smithi052 ceph-mon[32340]: pgmap v876: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-06-02T23:26:44.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:43 smithi123 ceph-mon[35977]: pgmap v876: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 50 KiB/s wr, 17 op/s 2024-06-02T23:26:44.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.47 MB/sec warmup 112 sec latency 23048.726 ms 2024-06-02T23:26:45.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.41 MB/sec warmup 113 sec latency 24048.803 ms 2024-06-02T23:26:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:45 smithi174 ceph-mon[35994]: pgmap v877: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-06-02T23:26:45.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:45 smithi052 ceph-mon[32340]: pgmap v877: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-06-02T23:26:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:45 smithi123 ceph-mon[35977]: pgmap v877: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-06-02T23:26:46.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.34 MB/sec warmup 114 sec latency 25048.855 ms 2024-06-02T23:26:46.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 -- 172.21.15.52:0/1920148241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 msgr2=0x7fcf7810f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/1920148241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7810f260 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fcf7000b0a0 tx=0x7fcf7000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 -- 172.21.15.52:0/1920148241 shutdown_connections 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/1920148241 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7810f260 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/1920148241 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf78107550 0x7fcf78107930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/1920148241 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf78100e30 0x7fcf781012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 -- 172.21.15.52:0/1920148241 >> 172.21.15.52:0/1920148241 conn(0x7fcf780766b0 msgr2=0x7fcf78076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 -- 172.21.15.52:0/1920148241 shutdown_connections 2024-06-02T23:26:46.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 -- 172.21.15.52:0/1920148241 wait complete. 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.566+0000 7fcf7df9f700 1 Processor -- start 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 -- start start 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf78100e30 0x7fcf7819ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7819b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf78107550 0x7fcf7819f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcf78112660 con 0x7fcf781017f0 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781124e0 con 0x7fcf78107550 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf7df9f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcf781127e0 con 0x7fcf78100e30 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7819b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:46.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7819b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60954/0 (socket says 172.21.15.52:60954) 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 -- 172.21.15.52:0/2177106157 learned_addr learned my addr 172.21.15.52:0/2177106157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf777fe700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf78100e30 0x7fcf7819ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf77fff700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf78107550 0x7fcf7819f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 -- 172.21.15.52:0/2177106157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf78100e30 msgr2=0x7fcf7819ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf78100e30 0x7fcf7819ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 -- 172.21.15.52:0/2177106157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf78107550 msgr2=0x7fcf7819f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf78107550 0x7fcf7819f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.567+0000 7fcf76ffd700 1 -- 172.21.15.52:0/2177106157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7819fd60 con 0x7fcf781017f0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.568+0000 7fcf76ffd700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7819b290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00d950 tx=0x7fcf6c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.568+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c016070 con 0x7fcf781017f0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.568+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcf6c00fd40 con 0x7fcf781017f0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.568+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf781a0050 con 0x7fcf781017f0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.568+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00e3b0 con 0x7fcf781017f0 2024-06-02T23:26:46.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.568+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf781a78d0 con 0x7fcf781017f0 2024-06-02T23:26:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.569+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf78069180 con 0x7fcf781017f0 2024-06-02T23:26:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.570+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcf6c019070 con 0x7fcf781017f0 2024-06-02T23:26:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.571+0000 7fcf74ff9700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf600a2c20 0x7fcf600a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:46.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.571+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcf6c0cea70 con 0x7fcf781017f0 2024-06-02T23:26:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.571+0000 7fcf777fe700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf600a2c20 0x7fcf600a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:46.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.571+0000 7fcf777fe700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf600a2c20 0x7fcf600a50e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcf68005950 tx=0x7fcf6800a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:46.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.575+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fcf6c09ba00 con 0x7fcf781017f0 2024-06-02T23:26:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.829+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf781a7ad0 con 0x7fcf781017f0 2024-06-02T23:26:46.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.830+0000 7fcf74ff9700 1 -- 172.21.15.52:0/2177106157 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7fcf6c09b300 con 0x7fcf781017f0 2024-06-02T23:26:46.829 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:26:46.830 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf600a2c20 msgr2=0x7fcf600a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf600a2c20 0x7fcf600a50e0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fcf68005950 tx=0x7fcf6800a300 comp rx=0 tx=0).stop 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 msgr2=0x7fcf7819b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7819b290 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00d950 tx=0x7fcf6c00dc60 comp rx=0 tx=0).stop 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 shutdown_connections 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcf600a2c20 0x7fcf600a50e0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf78107550 0x7fcf7819f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcf78100e30 0x7fcf7819ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 --2- 172.21.15.52:0/2177106157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcf781017f0 0x7fcf7819b290 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 >> 172.21.15.52:0/2177106157 conn(0x7fcf780766b0 msgr2=0x7fcf78102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 shutdown_connections 2024-06-02T23:26:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:46.832+0000 7fcf7df9f700 1 -- 172.21.15.52:0/2177106157 wait complete. 2024-06-02T23:26:46.833 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:26:47.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.28 MB/sec warmup 115 sec latency 26048.910 ms 2024-06-02T23:26:47.535 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 376ccfc3 --release --await' 2024-06-02T23:26:47.536 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 376ccfc3 --release --await 2024-06-02T23:26:47.803 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:47 smithi052 ceph-mon[32340]: pgmap v878: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 14 op/s 2024-06-02T23:26:47.839 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:47 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2177106157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:47 smithi174 ceph-mon[35994]: pgmap v878: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 14 op/s 2024-06-02T23:26:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:47 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2177106157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:47 smithi123 ceph-mon[35977]: pgmap v878: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.7 KiB/s wr, 14 op/s 2024-06-02T23:26:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:47 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2177106157' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:48.452 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.21 MB/sec warmup 116 sec latency 27048.956 ms 2024-06-02T23:26:49.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12364 7.15 MB/sec warmup 117 sec latency 28048.990 ms 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 -- 172.21.15.52:0/2155788636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 msgr2=0x7f236c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 --2- 172.21.15.52:0/2155788636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10e230 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f236400b0a0 tx=0x7f236400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 -- 172.21.15.52:0/2155788636 shutdown_connections 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 --2- 172.21.15.52:0/2155788636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10e230 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 --2- 172.21.15.52:0/2155788636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236c1073f0 0x7f236c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 --2- 172.21.15.52:0/2155788636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 0x7f236c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 -- 172.21.15.52:0/2155788636 >> 172.21.15.52:0/2155788636 conn(0x7f236c078300 msgr2=0x7f236c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:49.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 -- 172.21.15.52:0/2155788636 shutdown_connections 2024-06-02T23:26:49.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.502+0000 7f2370e4c700 1 -- 172.21.15.52:0/2155788636 wait complete. 2024-06-02T23:26:49.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.503+0000 7f2370e4c700 1 Processor -- start 2024-06-02T23:26:49.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.503+0000 7f2370e4c700 1 -- start start 2024-06-02T23:26:49.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2370e4c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 0x7f236c10eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2370e4c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2370e4c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236c1073f0 0x7f236c10f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2370e4c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f236c112690 con 0x7f236c101d40 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2370e4c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f236c112510 con 0x7f236c101380 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2370e4c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f236c112810 con 0x7f236c1073f0 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60972/0 (socket says 172.21.15.52:60972) 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 -- 172.21.15.52:0/1240199014 learned_addr learned my addr 172.21.15.52:0/1240199014 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f236ad9d700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236c1073f0 0x7f236c10f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f236a59c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 0x7f236c10eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 -- 172.21.15.52:0/1240199014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236c1073f0 msgr2=0x7f236c10f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236c1073f0 0x7f236c10f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 -- 172.21.15.52:0/1240199014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 msgr2=0x7f236c10eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 0x7f236c10eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f2369d9b700 1 -- 172.21.15.52:0/1240199014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f236c1a7020 con 0x7f236c101d40 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.504+0000 7f236a59c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 0x7f236c10eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.505+0000 7f2369d9b700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10f080 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f236000d950 tx=0x7f236000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.505+0000 7f235b7fe700 1 -- 172.21.15.52:0/1240199014 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360016070 con 0x7f236c101d40 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.505+0000 7f235b7fe700 1 -- 172.21.15.52:0/1240199014 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2360009df0 con 0x7f236c101d40 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.505+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f236c1a7310 con 0x7f236c101d40 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.505+0000 7f235b7fe700 1 -- 172.21.15.52:0/1240199014 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f236000f3c0 con 0x7f236c101d40 2024-06-02T23:26:49.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.505+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f236c1a7850 con 0x7f236c101d40 2024-06-02T23:26:49.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.508+0000 7f235b7fe700 1 -- 172.21.15.52:0/1240199014 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2360019070 con 0x7f236c101d40 2024-06-02T23:26:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.508+0000 7f235b7fe700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23540a2c20 0x7f23540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.509+0000 7f235b7fe700 1 -- 172.21.15.52:0/1240199014 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f23600cec50 con 0x7f236c101d40 2024-06-02T23:26:49.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.509+0000 7f236a59c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23540a2c20 0x7f23540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.509+0000 7f236a59c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23540a2c20 0x7f23540a50e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f235c005e10 tx=0x7f235c005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.509+0000 7f237b6cfb80 1 Processor -- start 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.509+0000 7f237b6cfb80 1 -- start start 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x565037159020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037152e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037153810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5650370a4910 con 0x5650371533d0 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56503704c820 con 0x565037159560 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650370a08d0 con 0x565037158c40 2024-06-02T23:26:49.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037153810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60988/0 (socket says 172.21.15.52:60988) 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 -- 172.21.15.52:0/2384815005 learned_addr learned my addr 172.21.15.52:0/2384815005 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f2369d9b700 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236a59c700 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x565037159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 -- 172.21.15.52:0/2384815005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 msgr2=0x565037159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x565037159020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 -- 172.21.15.52:0/2384815005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 msgr2=0x565037152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 -- 172.21.15.52:0/2384815005 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565037043050 con 0x5650371533d0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f236ad9d700 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037153810 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f2364009100 tx=0x7f2364009130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd7560d9d491ccb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.510+0000 7f2358ff9700 1 -- 172.21.15.52:0/2384815005 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364020070 con 0x5650371533d0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f2358ff9700 1 -- 172.21.15.52:0/2384815005 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23640074e0 con 0x5650371533d0 2024-06-02T23:26:49.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f2358ff9700 1 -- 172.21.15.52:0/2384815005 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23640052a0 con 0x5650371533d0 2024-06-02T23:26:49.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2384815005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 msgr2=0x565037153810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037153810 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f2364009100 tx=0x7f2364009130 comp rx=0 tx=0).stop 2024-06-02T23:26:49.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2384815005 shutdown_connections 2024-06-02T23:26:49.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037153810 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x565037159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2384815005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2384815005 >> 172.21.15.52:0/2384815005 conn(0x56503707ddc0 msgr2=0x56503714e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2384815005 shutdown_connections 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.511+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2384815005 wait complete. 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 -- start start 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 Processor -- start 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650371533d0 0x56503715f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037158c40 0x56503715c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 0x56503715cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56503715ff70 con 0x565037159560 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565037044cf0 con 0x565037158c40 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650371600f0 con 0x5650371533d0 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f237b6cfb80 10 client.0 _ll_register_callbacks cb 0x56503703ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 0x56503715cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f236ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 0x56503715cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60992/0 (socket says 172.21.15.52:60992) 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f236ad9d700 1 -- 172.21.15.52:0/4212844975 learned_addr learned my addr 172.21.15.52:0/4212844975 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f2369d9b700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037158c40 0x56503715c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.512+0000 7f236a59c700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650371533d0 0x56503715f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f236ad9d700 1 -- 172.21.15.52:0/4212844975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650371533d0 msgr2=0x56503715f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f236ad9d700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650371533d0 0x56503715f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f236ad9d700 1 -- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037158c40 msgr2=0x56503715c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f236ad9d700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037158c40 0x56503715c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f236ad9d700 1 -- 172.21.15.52:0/4212844975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565036ff39d0 con 0x565037159560 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f236ad9d700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 0x56503715cac0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f236400b3f0 tx=0x7f2364009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e67c99a1e2a5cbbd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f2351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f2351ffb700 1 -- 172.21.15.52:0/4212844975 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364020040 con 0x565037159560 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23640074e0 con 0x565037159560 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f2351ffb700 1 -- 172.21.15.52:0/4212844975 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f23640050c0 con 0x565037159560 2024-06-02T23:26:49.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f2351ffb700 1 -- 172.21.15.52:0/4212844975 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2364036050 con 0x565037159560 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f237b6cfb80 10 client.19521 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x56503716a840 con 0x565037159560 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.513+0000 7f2351ffb700 1 -- 172.21.15.52:0/4212844975 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f23640074e0 con 0x565037159560 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f2351ffb700 1 -- 172.21.15.52:0/4212844975 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f2364011070 con 0x565037159560 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f237b6cfb80 10 client.19521 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f237b6cfb80 10 client.19521 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f237b6cfb80 20 client.19521 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56503716cd10 0x56503716f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f237b6cfb80 4 client.19521 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650370b1e10 con 0x56503716cd10 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.514+0000 7f236a59c700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56503716cd10 0x56503716f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.515+0000 7f2351ffb700 10 client.19521 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:49.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.515+0000 7f236a59c700 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56503716cd10 0x56503716f100 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8652fceb4b6bb241 server_cookie=3452f32f126c118e in_seq=0 out_seq=0 2024-06-02T23:26:49.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f2351ffb700 1 -- 172.21.15.52:0/4212844975 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5650370b1e10 con 0x56503716cd10 2024-06-02T23:26:49.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f2351ffb700 10 client.19521 handle_command_reply: tid=0 2024-06-02T23:26:49.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 client.19521 shutdown 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 msgr2=0x56503715cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 0x56503715cac0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f236400b3f0 tx=0x7f2364009580 comp rx=0 tx=0).stop 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 shutdown_connections 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56503716cd10 0x56503716f100 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037159560 0x56503715cac0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650371533d0 0x56503715f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/4212844975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037158c40 0x56503715c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 >> 172.21.15.52:0/4212844975 conn(0x56503707ddc0 msgr2=0x56503714dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f2351ffb700 0 client.19521 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 shutdown_connections 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 1 -- 172.21.15.52:0/4212844975 wait complete. 2024-06-02T23:26:49.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.516+0000 7f237b6cfb80 20 client.19521 trim_cache size 0 max 16384 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 Processor -- start 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 -- start start 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037152e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x56503715b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037171d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1aead0 con 0x5650371533d0 2024-06-02T23:26:49.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1b0730 con 0x565037159560 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1af1a0 con 0x565037158c40 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32770/0 (socket says 172.21.15.52:32770) 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 -- 172.21.15.52:0/2965552862 learned_addr learned my addr 172.21.15.52:0/2965552862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f2369d9b700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x56503715b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236ad9d700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037171d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 -- 172.21.15.52:0/2965552862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 msgr2=0x56503715b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x56503715b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 -- 172.21.15.52:0/2965552862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 msgr2=0x565037171d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037171d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236a59c700 1 -- 172.21.15.52:0/2965552862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56503714f6e0 con 0x5650371533d0 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f236ad9d700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037171d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:49.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.537+0000 7f2369d9b700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x56503715b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f236a59c700 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037152e90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f235c005ee0 tx=0x7f235c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d390c1fe3b3a9f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f2353fff700 1 -- 172.21.15.52:0/2965552862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c020ab0 con 0x5650371533d0 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f2353fff700 1 -- 172.21.15.52:0/2965552862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f235c02e070 con 0x5650371533d0 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f2353fff700 1 -- 172.21.15.52:0/2965552862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f235c0116a0 con 0x5650371533d0 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2965552862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 msgr2=0x565037152e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037152e90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f235c005ee0 tx=0x7f235c015000 comp rx=0 tx=0).stop 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2965552862 shutdown_connections 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565037159560 0x565037171d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5650371533d0 0x565037152e90 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/2965552862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037158c40 0x56503715b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.538+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2965552862 >> 172.21.15.52:0/2965552862 conn(0x56503707ddc0 msgr2=0x56503715ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2965552862 shutdown_connections 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 -- 172.21.15.52:0/2965552862 wait complete. 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 -- start start 2024-06-02T23:26:49.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 Processor -- start 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650371533d0 0x565037171ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 0x5650371723e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037159560 0x565037170410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1affa0 con 0x565037158c40 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1af8a0 con 0x5650371533d0 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f236c1aca90 con 0x565037159560 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f237b6cfb80 10 client.0 _ll_register_callbacks cb 0x56503703ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 0x5650371723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f2369d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 0x5650371723e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:32774/0 (socket says 172.21.15.52:32774) 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f236a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650371533d0 0x565037171ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.539+0000 7f2369d9b700 1 -- 172.21.15.52:0/3100607927 learned_addr learned my addr 172.21.15.52:0/3100607927 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f236ad9d700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037159560 0x565037170410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2369d9b700 1 -- 172.21.15.52:0/3100607927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037159560 msgr2=0x565037170410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2369d9b700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037159560 0x565037170410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2369d9b700 1 -- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650371533d0 msgr2=0x565037171ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2369d9b700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650371533d0 0x565037171ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2369d9b700 1 -- 172.21.15.52:0/3100607927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565037170b10 con 0x565037158c40 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f236ad9d700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037159560 0x565037170410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f236a59c700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650371533d0 0x565037171ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2369d9b700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 0x5650371723e0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f23600cc780 tx=0x7f2360009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41dd3ded40a577aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2351ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2351ffb700 1 -- 172.21.15.52:0/3100607927 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360020040 con 0x565037158c40 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2351ffb700 1 -- 172.21.15.52:0/3100607927 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f236009b740 con 0x565037158c40 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f235c02e070 con 0x565037158c40 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2351ffb700 1 -- 172.21.15.52:0/3100607927 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f23600b44b0 con 0x565037158c40 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f237b6cfb80 10 client.19527 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5650371d81c0 con 0x565037158c40 2024-06-02T23:26:49.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.540+0000 7f2351ffb700 1 -- 172.21.15.52:0/3100607927 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f235c02e070 con 0x565037158c40 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f2351ffb700 1 -- 172.21.15.52:0/3100607927 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f23600a3cc0 con 0x565037158c40 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f237b6cfb80 10 client.19527 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f237b6cfb80 10 client.19527 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f237b6cfb80 20 client.19527 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650371d8910 0x5650371dad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f237b6cfb80 4 client.19527 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "376ccfc3", "release": true, "await": true}] 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "376ccfc3", "release": true, "await": true}) -- 0x5650370ac390 con 0x5650371d8910 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.541+0000 7f236a59c700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650371d8910 0x5650371dad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:49.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.542+0000 7f2351ffb700 10 client.19527 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:49.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.542+0000 7f236a59c700 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650371d8910 0x5650371dad30 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93210b4c7057cc13 server_cookie=948daf4f71acbf67 in_seq=0 out_seq=0 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f2351ffb700 1 -- 172.21.15.52:0/3100607927 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650370ac390 con 0x5650371d8910 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f2351ffb700 10 client.19527 handle_command_reply: tid=0 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 client.19527 shutdown 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 msgr2=0x5650371723e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 0x5650371723e0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f23600cc780 tx=0x7f2360009650 comp rx=0 tx=0).stop 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 shutdown_connections 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5650371d8910 0x5650371dad30 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565037159560 0x565037170410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f2351ffb700 0 client.19527 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5650371533d0 0x565037171ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 --2- 172.21.15.52:0/3100607927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565037158c40 0x5650371723e0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 >> 172.21.15.52:0/3100607927 conn(0x56503707ddc0 msgr2=0x565037157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 shutdown_connections 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 1 -- 172.21.15.52:0/3100607927 wait complete. 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.580+0000 7f237b6cfb80 20 client.19527 trim_cache size 0 max 16384 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 37, 2024-06-02T23:26:49.582 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 52, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "376ccfc3": { 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 52, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:26:49.583 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:49.584 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23540a2c20 msgr2=0x7f23540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23540a2c20 0x7f23540a50e0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f235c005e10 tx=0x7f235c005d80 comp rx=0 tx=0).stop 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 msgr2=0x7f236c10f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10f080 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f236000d950 tx=0x7f236000dc60 comp rx=0 tx=0).stop 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 shutdown_connections 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23540a2c20 0x7f23540a50e0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f236c1073f0 0x7f236c10f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f236c101380 0x7f236c10eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 --2- 172.21.15.52:0/1240199014 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f236c101d40 0x7f236c10f080 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:49.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 >> 172.21.15.52:0/1240199014 conn(0x7f236c078300 msgr2=0x7f236c100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:49.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 shutdown_connections 2024-06-02T23:26:49.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:49.581+0000 7f2370e4c700 1 -- 172.21.15.52:0/1240199014 wait complete. 2024-06-02T23:26:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:49 smithi174 ceph-mon[35994]: pgmap v879: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-06-02T23:26:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:49 smithi052 ceph-mon[32340]: pgmap v879: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-06-02T23:26:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:49 smithi123 ceph-mon[35977]: pgmap v879: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-06-02T23:26:50.363 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '376ccfc3', total seconds elapsed: 37.1 2024-06-02T23:26:50.363 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-02T23:26:50.364 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:26:50.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12519 7.10 MB/sec warmup 118 sec latency 28149.520 ms 2024-06-02T23:26:50.734 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:51.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 12699 7.12 MB/sec warmup 119 sec latency 60.903 ms 2024-06-02T23:26:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:51 smithi174 ceph-mon[35994]: pgmap v880: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-06-02T23:26:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:51 smithi052 ceph-mon[32340]: pgmap v880: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-06-02T23:26:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:51 smithi123 ceph-mon[35977]: pgmap v880: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.7 KiB/s wr, 15 op/s 2024-06-02T23:26:52.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 -- 172.21.15.52:0/3246454563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 msgr2=0x7f62881077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:52.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 --2- 172.21.15.52:0/3246454563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f62881077e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f6280009f90 tx=0x7f628000b330 comp rx=0 tx=0).stop 2024-06-02T23:26:52.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 -- 172.21.15.52:0/3246454563 shutdown_connections 2024-06-02T23:26:52.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 --2- 172.21.15.52:0/3246454563 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6288101d50 0x7f628810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:52.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 --2- 172.21.15.52:0/3246454563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f62881077e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:52.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 --2- 172.21.15.52:0/3246454563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101390 0x7f6288101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:52.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.609+0000 7f628cfed700 1 -- 172.21.15.52:0/3246454563 >> 172.21.15.52:0/3246454563 conn(0x7f6288078340 msgr2=0x7f6288078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:52.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.610+0000 7f628cfed700 1 -- 172.21.15.52:0/3246454563 shutdown_connections 2024-06-02T23:26:52.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.610+0000 7f628cfed700 1 -- 172.21.15.52:0/3246454563 wait complete. 2024-06-02T23:26:52.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.610+0000 7f628cfed700 1 Processor -- start 2024-06-02T23:26:52.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 -- start start 2024-06-02T23:26:52.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6288101390 0x7f628810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:52.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 0x7f628810f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:52.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f628810f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:52.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6288112640 con 0x7f6288101390 2024-06-02T23:26:52.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62881124c0 con 0x7f6288107400 2024-06-02T23:26:52.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628cfed700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62881127c0 con 0x7f6288101d50 2024-06-02T23:26:52.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f628659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6288101390 0x7f628810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f627dd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 0x7f628810f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f627dd9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 0x7f628810f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33246/0 (socket says 172.21.15.52:33246) 2024-06-02T23:26:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f627dd9b700 1 -- 172.21.15.52:0/1081593263 learned_addr learned my addr 172.21.15.52:0/1081593263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.611+0000 7f6286d9d700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f628810f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:52.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627dd9b700 1 -- 172.21.15.52:0/1081593263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 msgr2=0x7f628810f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627dd9b700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f628810f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627dd9b700 1 -- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6288101390 msgr2=0x7f628810eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627dd9b700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6288101390 0x7f628810eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627dd9b700 1 -- 172.21.15.52:0/1081593263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62881a7020 con 0x7f6288101d50 2024-06-02T23:26:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f6286d9d700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f628810f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:52.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627dd9b700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 0x7f628810f090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9a0 tx=0x7f627000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627000c9c0 con 0x7f6288101d50 2024-06-02T23:26:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f627000cb60 con 0x7f6288101d50 2024-06-02T23:26:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62881a7310 con 0x7f6288101d50 2024-06-02T23:26:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.612+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6270018680 con 0x7f6288101d50 2024-06-02T23:26:52.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.613+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62881a78a0 con 0x7f6288101d50 2024-06-02T23:26:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.614+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6288069180 con 0x7f6288101d50 2024-06-02T23:26:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.616+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f627000cd10 con 0x7f6288101d50 2024-06-02T23:26:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.617+0000 7f627ffff700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62740a2ca0 0x7f62740a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.617+0000 7f628659c700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62740a2ca0 0x7f62740a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.617+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f62700cf110 con 0x7f6288101d50 2024-06-02T23:26:52.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.617+0000 7f628659c700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62740a2ca0 0x7f62740a5160 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f628000b6c0 tx=0x7f6280009ca0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:52.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:52.622+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f627009c0a0 con 0x7f6288101d50 2024-06-02T23:26:53.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.073+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6288110080 con 0x7f6288101d50 2024-06-02T23:26:53.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.074+0000 7f627ffff700 1 -- 172.21.15.52:0/1081593263 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7f627009b9a0 con 0x7f6288101d50 2024-06-02T23:26:53.075 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:26:53.075 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62740a2ca0 msgr2=0x7f62740a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62740a2ca0 0x7f62740a5160 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f628000b6c0 tx=0x7f6280009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 msgr2=0x7f628810f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 0x7f628810f090 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f627000e9a0 tx=0x7f627000ecb0 comp rx=0 tx=0).stop 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 shutdown_connections 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f62740a2ca0 0x7f62740a5160 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6288107400 0x7f628810f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:53.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6288101390 0x7f628810eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:53.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 --2- 172.21.15.52:0/1081593263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6288101d50 0x7f628810f090 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:53.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.077+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 >> 172.21.15.52:0/1081593263 conn(0x7f6288078340 msgr2=0x7f6288100170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:53.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.078+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 shutdown_connections 2024-06-02T23:26:53.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:53.078+0000 7f628cfed700 1 -- 172.21.15.52:0/1081593263 wait complete. 2024-06-02T23:26:53.079 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:26:53.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13038 10.53 MB/sec execute 1 sec latency 110.473 ms 2024-06-02T23:26:53.755 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:53 smithi052 ceph-mon[32340]: pgmap v881: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-06-02T23:26:53.755 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:53 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1081593263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:53.795 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-02T23:26:53.795 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-02T23:26:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:53 smithi174 ceph-mon[35994]: pgmap v881: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-06-02T23:26:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:53 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1081593263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:53 smithi123 ceph-mon[35977]: pgmap v881: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-06-02T23:26:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:53 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1081593263' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:26:54.154 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:54.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13207 11.21 MB/sec execute 2 sec latency 60.563 ms 2024-06-02T23:26:54.827 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 -- 172.21.15.52:0/359913824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c107720 msgr2=0x7f1b2c109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 --2- 172.21.15.52:0/359913824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c107720 0x7f1b2c109b70 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009ca0 tx=0x7f1b2402a8c0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 -- 172.21.15.52:0/359913824 shutdown_connections 2024-06-02T23:26:55.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 --2- 172.21.15.52:0/359913824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b2c104df0 0x7f1b2c1071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 --2- 172.21.15.52:0/359913824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c107720 0x7f1b2c109b70 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 --2- 172.21.15.52:0/359913824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 -- 172.21.15.52:0/359913824 >> 172.21.15.52:0/359913824 conn(0x7f1b2c0fbeb0 msgr2=0x7f1b2c0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 -- 172.21.15.52:0/359913824 shutdown_connections 2024-06-02T23:26:55.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.250+0000 7f1b31af7640 1 -- 172.21.15.52:0/359913824 wait complete. 2024-06-02T23:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.251+0000 7f1b31af7640 1 Processor -- start 2024-06-02T23:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.251+0000 7f1b31af7640 1 -- start start 2024-06-02T23:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.251+0000 7f1b31af7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b31af7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 0x7f1b2c19a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b31af7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b2c107720 0x7f1b2c1a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b31af7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10d590 con 0x7f1b2c104df0 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b31af7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10d410 con 0x7f1b2c1024c0 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b31af7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c10d710 con 0x7f1b2c107720 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 0x7f1b2c19a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c199dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39000/0 (socket says 172.21.15.52:39000) 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 -- 172.21.15.52:0/885481712 learned_addr learned my addr 172.21.15.52:0/885481712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 0x7f1b2c19a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60432/0 (socket says 172.21.15.52:60432) 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b2c107720 0x7f1b2c1a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 -- 172.21.15.52:0/885481712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b2c107720 msgr2=0x7f1b2c1a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b2c107720 0x7f1b2c1a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 -- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 msgr2=0x7f1b2c19a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 0x7f1b2c19a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 -- 172.21.15.52:0/885481712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b2c1a19d0 con 0x7f1b2c1024c0 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2affd640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 0x7f1b2c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.252+0000 7f1b2b7fe640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c199dc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00e9f0 tx=0x7f1b1c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.253+0000 7f1b28ff9640 1 -- 172.21.15.52:0/885481712 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c00cd80 con 0x7f1b2c1024c0 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.253+0000 7f1b28ff9640 1 -- 172.21.15.52:0/885481712 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b1c0108e0 con 0x7f1b2c1024c0 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.253+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b2c1a1cc0 con 0x7f1b2c1024c0 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.253+0000 7f1b28ff9640 1 -- 172.21.15.52:0/885481712 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c018920 con 0x7f1b2c1024c0 2024-06-02T23:26:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.253+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2c1a21c0 con 0x7f1b2c1024c0 2024-06-02T23:26:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 Processor -- start 2024-06-02T23:26:55.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 -- start start 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b28ff9640 1 -- 172.21.15.52:0/885481712 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1b1c010a90 con 0x7f1b2c1024c0 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb0493430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb0498500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb049b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cfb03f64f0 con 0x55cfb0493a00 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cfb0394870 con 0x55cfb0498b70 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.256+0000 7f1b347fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cfb03f24b0 con 0x55cfb0493010 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb0498500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb0493430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb049b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb049b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39008/0 (socket says 172.21.15.52:39008) 2024-06-02T23:26:55.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2affd640 1 -- 172.21.15.52:0/2905580018 learned_addr learned my addr 172.21.15.52:0/2905580018 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb0493430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53796/0 (socket says 172.21.15.52:53796) 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 -- 172.21.15.52:0/2905580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 msgr2=0x55cfb049b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb049b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 -- 172.21.15.52:0/2905580018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 msgr2=0x55cfb0498500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb0498500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 -- 172.21.15.52:0/2905580018 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb03b09b0 con 0x55cfb0493010 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2affd640 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb049b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.257+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb0493430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1b24031040 tx=0x7f1b24009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d0ef9b4101cf1bb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.258+0000 7f1b28ff9640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1b140a2c30 0x7f1b140a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.258+0000 7f1b11ffb640 1 -- 172.21.15.52:0/2905580018 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24031650 con 0x55cfb0493010 2024-06-02T23:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.258+0000 7f1b11ffb640 1 -- 172.21.15.52:0/2905580018 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b24036030 con 0x55cfb0493010 2024-06-02T23:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.258+0000 7f1b11ffb640 1 -- 172.21.15.52:0/2905580018 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24004420 con 0x55cfb0493010 2024-06-02T23:26:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.258+0000 7f1b2b7fe640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1b140a2c30 0x7f1b140a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.258+0000 7f1b28ff9640 1 -- 172.21.15.52:0/885481712 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1b1c0cf370 con 0x7f1b2c1024c0 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 -- 172.21.15.52:0/2905580018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 msgr2=0x55cfb0493430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b2b7fe640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1b140a2c30 0x7f1b140a50f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0cd210 tx=0x7f1b1c0d2e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb0493430 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1b24031040 tx=0x7f1b24009a20 comp rx=0 tx=0).stop 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 -- 172.21.15.52:0/2905580018 shutdown_connections 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb049b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb0498500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2905580018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb0493430 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 -- 172.21.15.52:0/2905580018 >> 172.21.15.52:0/2905580018 conn(0x55cfb048e680 msgr2=0x55cfb048ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 -- 172.21.15.52:0/2905580018 shutdown_connections 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 -- 172.21.15.52:0/2905580018 wait complete. 2024-06-02T23:26:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 -- start start 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.259+0000 7f1b347fd740 1 Processor -- start 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb04ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cfb049b940 con 0x55cfb0493a00 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cfb037fae0 con 0x55cfb0498b70 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cfb049bac0 con 0x55cfb0493010 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b347fd740 10 client.0 _ll_register_callbacks cb 0x55cfb03b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04a5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60454/0 (socket says 172.21.15.52:60454) 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2affd640 1 -- 172.21.15.52:0/757378618 learned_addr learned my addr 172.21.15.52:0/757378618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53808/0 (socket says 172.21.15.52:53808) 2024-06-02T23:26:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb04ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 -- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 msgr2=0x55cfb04ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb04ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 -- 172.21.15.52:0/757378618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 msgr2=0x55cfb04a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 -- 172.21.15.52:0/757378618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb010a210 con 0x55cfb0493010 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a5970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009bf0 tx=0x7f1b240095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=313638d378c15d6a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.260+0000 7f1b0affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b0affd640 1 -- 172.21.15.52:0/757378618 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24031d40 con 0x55cfb0493010 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b0affd640 1 -- 172.21.15.52:0/757378618 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1b24004b40 con 0x55cfb0493010 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b24036030 con 0x55cfb0493010 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b0affd640 1 -- 172.21.15.52:0/757378618 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1b24004d40 con 0x55cfb0493010 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 10 client.28603 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cfb01d6620 con 0x55cfb0493010 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b0affd640 1 -- 172.21.15.52:0/757378618 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b24036030 con 0x55cfb0493010 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b0affd640 1 -- 172.21.15.52:0/757378618 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f1b24043070 con 0x55cfb0493010 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 10 client.28603 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 10 client.28603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 20 client.28603 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb04b3930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 4 client.28603 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.261+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfb0409f10 con 0x55cfb04b1540 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.262+0000 7f1b2affd640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb04b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.262+0000 7f1b0affd640 10 client.28603 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:55.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.262+0000 7f1b2affd640 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb04b3930 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea945cfbbac7c2 server_cookie=279d0048ca8b2fe in_seq=0 out_seq=0 2024-06-02T23:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b0affd640 1 -- 172.21.15.52:0/757378618 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cfb0409f10 con 0x55cfb04b1540 2024-06-02T23:26:55.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b0affd640 10 client.28603 handle_command_reply: tid=0 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 client.28603 shutdown 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 msgr2=0x55cfb04a5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a5970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009bf0 tx=0x7f1b240095c0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 shutdown_connections 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb04b3930 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b0affd640 0 client.28603 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb04ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 --2- 172.21.15.52:0/757378618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a5970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.263+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 >> 172.21.15.52:0/757378618 conn(0x55cfb048e680 msgr2=0x55cfb048d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.264+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 shutdown_connections 2024-06-02T23:26:55.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.264+0000 7f1b347fd740 1 -- 172.21.15.52:0/757378618 wait complete. 2024-06-02T23:26:55.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.264+0000 7f1b347fd740 20 client.28603 trim_cache size 0 max 16384 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 Processor -- start 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 -- start start 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0493010 0x55cfb04a5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 0x55cfb04ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0498b70 0x55cfb04a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c1a6970 con 0x55cfb0498b70 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c1a85d0 con 0x55cfb0493010 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b347fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c1a7040 con 0x55cfb0493a00 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 0x55cfb04ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.292 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 0x55cfb04ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53812/0 (socket says 172.21.15.52:53812) 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b2affd640 1 -- 172.21.15.52:0/2456478763 learned_addr learned my addr 172.21.15.52:0/2456478763 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0498b70 0x55cfb04a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.290+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0493010 0x55cfb04a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2affd640 1 -- 172.21.15.52:0/2456478763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0493010 msgr2=0x55cfb04a5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2affd640 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0493010 0x55cfb04a5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2affd640 1 -- 172.21.15.52:0/2456478763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0498b70 msgr2=0x55cfb04a5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2affd640 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0498b70 0x55cfb04a5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2affd640 1 -- 172.21.15.52:0/2456478763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfb04b3ca0 con 0x55cfb0493a00 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0498b70 0x55cfb04a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b2affd640 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 0x55cfb04ace40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b20012160 tx=0x7f1b20005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ab0ec8e0031ca8c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b117fa640 1 -- 172.21.15.52:0/2456478763 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b200035f0 con 0x55cfb0493a00 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b117fa640 1 -- 172.21.15.52:0/2456478763 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b20003c40 con 0x55cfb0493a00 2024-06-02T23:26:55.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.291+0000 7f1b117fa640 1 -- 172.21.15.52:0/2456478763 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b20027df0 con 0x55cfb0493a00 2024-06-02T23:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 -- 172.21.15.52:0/2456478763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 msgr2=0x55cfb04ace40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 0x55cfb04ace40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1b20012160 tx=0x7f1b20005f90 comp rx=0 tx=0).stop 2024-06-02T23:26:55.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 -- 172.21.15.52:0/2456478763 shutdown_connections 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493a00 0x55cfb04ace40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0498b70 0x55cfb04a5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 --2- 172.21.15.52:0/2456478763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0493010 0x55cfb04a5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 -- 172.21.15.52:0/2456478763 >> 172.21.15.52:0/2456478763 conn(0x55cfb048e680 msgr2=0x55cfb048ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 -- 172.21.15.52:0/2456478763 shutdown_connections 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 -- 172.21.15.52:0/2456478763 wait complete. 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.292+0000 7f1b347fd740 1 -- start start 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 Processor -- start 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb051a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c1a7e40 con 0x55cfb0493a00 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c1a7740 con 0x55cfb0498b70 2024-06-02T23:26:55.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b2c1a4970 con 0x55cfb0493010 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b347fd740 10 client.0 _ll_register_callbacks cb 0x7f1b1c0d2d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b2bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b2affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04aa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b2bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53828/0 (socket says 172.21.15.52:53828) 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b2bfff640 1 -- 172.21.15.52:0/3491418537 learned_addr learned my addr 172.21.15.52:0/3491418537 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.293+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb051a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2bfff640 1 -- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 msgr2=0x55cfb051a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb051a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2bfff640 1 -- 172.21.15.52:0/3491418537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 msgr2=0x55cfb04aa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04aa1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2bfff640 1 -- 172.21.15.52:0/3491418537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfb051a6e0 con 0x55cfb0493010 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2affd640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:55.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b2bfff640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a9cb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b24030c10 tx=0x7f1b24000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bc35065589a8227 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b0affd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b0affd640 1 -- 172.21.15.52:0/3491418537 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b24004ee0 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b0affd640 1 -- 172.21.15.52:0/3491418537 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1b24004b40 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b20003c40 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b0affd640 1 -- 172.21.15.52:0/3491418537 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1b24031b40 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b347fd740 10 client.28609 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.294+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55cfb051b8b0 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b0affd640 1 -- 172.21.15.52:0/3491418537 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1b20003c40 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b0affd640 1 -- 172.21.15.52:0/3491418537 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f1b24043040 con 0x55cfb0493010 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b347fd740 10 client.28609 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b347fd740 10 client.28609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b347fd740 20 client.28609 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b347fd740 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb051e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:55.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b347fd740 4 client.28609 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cfb0403c20 con 0x55cfb04b1540 2024-06-02T23:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.295+0000 7f1b2affd640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb051e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.296+0000 7f1b2affd640 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb051e300 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffde6e0bc89fcc10 server_cookie=867699bb8ef5b82b in_seq=0 out_seq=0 2024-06-02T23:26:55.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.296+0000 7f1b0affd640 10 client.28609 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b0affd640 1 -- 172.21.15.52:0/3491418537 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55cfb0403c20 con 0x55cfb04b1540 2024-06-02T23:26:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b0affd640 10 client.28609 handle_command_reply: tid=0 2024-06-02T23:26:55.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 client.28609 shutdown 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 msgr2=0x55cfb04a9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a9cb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1b24030c10 tx=0x7f1b24000f40 comp rx=0 tx=0).stop 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 shutdown_connections 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cfb04b1540 0x55cfb051e300 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfb0498b70 0x55cfb051a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cfb0493a00 0x55cfb04aa1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b0affd640 0 client.28609 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 --2- 172.21.15.52:0/3491418537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cfb0493010 0x55cfb04a9cb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 >> 172.21.15.52:0/3491418537 conn(0x55cfb048e680 msgr2=0x55cfb048ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 shutdown_connections 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 1 -- 172.21.15.52:0/3491418537 wait complete. 2024-06-02T23:26:55.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.327+0000 7f1b347fd740 20 client.28609 trim_cache size 0 max 16384 2024-06-02T23:26:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:26:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:26:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1b140a2c30 msgr2=0x7f1b140a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1b140a2c30 0x7f1b140a50f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c0cd210 tx=0x7f1b1c0d2e50 comp rx=0 tx=0).stop 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 msgr2=0x7f1b2c199dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c199dc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c00e9f0 tx=0x7f1b1c00eeb0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 shutdown_connections 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1b140a2c30 0x7f1b140a50f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b2c107720 0x7f1b2c1a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1b2c104df0 0x7f1b2c19a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 --2- 172.21.15.52:0/885481712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b2c1024c0 0x7f1b2c199dc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 >> 172.21.15.52:0/885481712 conn(0x7f1b2c0fbeb0 msgr2=0x7f1b2c101b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 shutdown_connections 2024-06-02T23:26:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:26:55.328+0000 7f1b31af7640 1 -- 172.21.15.52:0/885481712 wait complete. 2024-06-02T23:26:55.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13416 12.65 MB/sec execute 3 sec latency 67.848 ms 2024-06-02T23:26:55.675 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:55 smithi052 ceph-mon[32340]: pgmap v882: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 362 KiB/s wr, 65 op/s 2024-06-02T23:26:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:55 smithi174 ceph-mon[35994]: pgmap v882: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 362 KiB/s wr, 65 op/s 2024-06-02T23:26:56.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:55 smithi123 ceph-mon[35977]: pgmap v882: 97 pgs: 97 active+clean; 328 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 201 KiB/s rd, 362 KiB/s wr, 65 op/s 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.141+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/1882183570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 msgr2=0x7f4acc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.141+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/1882183570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc108a00 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0098f0 tx=0x7f4abc009c00 comp rx=0 tx=0).stop 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.141+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/1882183570 shutdown_connections 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.141+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/1882183570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 0x7f4acc10f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.141+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/1882183570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc108a00 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.141+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/1882183570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.142+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/1882183570 >> 172.21.15.52:0/1882183570 conn(0x7f4acc079640 msgr2=0x7f4acc079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.142+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/1882183570 shutdown_connections 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.142+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/1882183570 wait complete. 2024-06-02T23:26:56.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.142+0000 7f4ad1fd8700 1 Processor -- start 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 -- start start 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc19e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 0x7f4acc19ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1126d0 con 0x7f4acc102ed0 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112550 con 0x7f4acc108620 2024-06-02T23:26:56.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4ad1fd8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc112850 con 0x7f4acc102510 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc19e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc19e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53856/0 (socket says 172.21.15.52:53856) 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3282191106 learned_addr learned my addr 172.21.15.52:0/3282191106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.143+0000 7f4acbfff700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acaffd700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 0x7f4acc19ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3282191106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 msgr2=0x7f4acc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 msgr2=0x7f4acc19ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 0x7f4acc19ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3282191106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4acc199070 con 0x7f4acc102510 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acbfff700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acaffd700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 0x7f4acc19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:56.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc19e750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009fd0 tx=0x7f4abc005e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:56.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4ac8ff9700 1 -- 172.21.15.52:0/3282191106 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc027070 con 0x7f4acc102510 2024-06-02T23:26:56.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4ac8ff9700 1 -- 172.21.15.52:0/3282191106 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4abc00b560 con 0x7f4acc102510 2024-06-02T23:26:56.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4acc199300 con 0x7f4acc102510 2024-06-02T23:26:56.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4ac8ff9700 1 -- 172.21.15.52:0/3282191106 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc004a50 con 0x7f4acc102510 2024-06-02T23:26:56.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.144+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4acc1a5620 con 0x7f4acc102510 2024-06-02T23:26:56.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.147+0000 7f4ac8ff9700 1 -- 172.21.15.52:0/3282191106 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4abc02a070 con 0x7f4acc102510 2024-06-02T23:26:56.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.148+0000 7f4ac8ff9700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ab40a2d40 0x7f4ab40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.148+0000 7f4acaffd700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ab40a2d40 0x7f4ab40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.148+0000 7f4ac8ff9700 1 -- 172.21.15.52:0/3282191106 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4abc0cfb20 con 0x7f4acc102510 2024-06-02T23:26:56.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.150+0000 7f4acaffd700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ab40a2d40 0x7f4ab40a5200 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005fd0 tx=0x7f4ac0005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:56.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.150+0000 7f4adc85bb80 1 Processor -- start 2024-06-02T23:26:56.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.150+0000 7f4adc85bb80 1 -- start start 2024-06-02T23:26:56.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8b6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6eb0 0x55908b8b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8b6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8b6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60494/0 (socket says 172.21.15.52:60494) 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4acb7fe700 1 -- 172.21.15.52:0/1022905293 learned_addr learned my addr 172.21.15.52:0/1022905293 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4acaffd700 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6eb0 0x55908b8b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8b1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.151+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55908b7a30f0 con 0x55908b8b6590 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55908b8019e0 con 0x55908b8b0ce0 2024-06-02T23:26:56.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55908b7695f0 con 0x55908b8b6eb0 2024-06-02T23:26:56.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acbfff700 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8b1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acb7fe700 1 -- 172.21.15.52:0/1022905293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6eb0 msgr2=0x55908b8b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6eb0 0x55908b8b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acb7fe700 1 -- 172.21.15.52:0/1022905293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 msgr2=0x55908b8b1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8b1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acb7fe700 1 -- 172.21.15.52:0/1022905293 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908b750a60 con 0x55908b8b6590 2024-06-02T23:26:56.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.152+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8b6970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0a4af0 tx=0x7f4abc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ce517b1958d5d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:56.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.153+0000 7f4ab1ffb700 1 -- 172.21.15.52:0/1022905293 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc09c440 con 0x55908b8b6590 2024-06-02T23:26:56.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.153+0000 7f4ab1ffb700 1 -- 172.21.15.52:0/1022905293 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4abc0a07f0 con 0x55908b8b6590 2024-06-02T23:26:56.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.153+0000 7f4ab1ffb700 1 -- 172.21.15.52:0/1022905293 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc09c900 con 0x55908b8b6590 2024-06-02T23:26:56.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 msgr2=0x55908b8b6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8b6970 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f4abc0a4af0 tx=0x7f4abc009c00 comp rx=0 tx=0).stop 2024-06-02T23:26:56.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 shutdown_connections 2024-06-02T23:26:56.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8b1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8b6970 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/1022905293 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6eb0 0x55908b8b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 >> 172.21.15.52:0/1022905293 conn(0x55908b7dd210 msgr2=0x55908b8aba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 shutdown_connections 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.154+0000 7f4adc85bb80 1 -- 172.21.15.52:0/1022905293 wait complete. 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.155+0000 7f4adc85bb80 1 -- start start 2024-06-02T23:26:56.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.155+0000 7f4adc85bb80 1 Processor -- start 2024-06-02T23:26:56.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.155+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8c4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.155+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6eb0 0x55908b8be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4adc85bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55908b7a2150 con 0x55908b8b6590 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4adc85bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55908b7fd9a0 con 0x55908b8b6eb0 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4adc85bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55908b8bd640 con 0x55908b8b0ce0 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4adc85bb80 10 client.0 _ll_register_callbacks cb 0x55908b797740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8c4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53876/0 (socket says 172.21.15.52:53876) 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 -- 172.21.15.52:0/4254292290 learned_addr learned my addr 172.21.15.52:0/4254292290 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acbfff700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6eb0 0x55908b8be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 -- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6eb0 msgr2=0x55908b8be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6eb0 0x55908b8be6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 -- 172.21.15.52:0/4254292290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 msgr2=0x55908b8c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 -- 172.21.15.52:0/4254292290 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908b7c78e0 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8c4360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4abc098850 tx=0x7f4abc09ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee062ebaf7b6f958 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4aaaffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.156+0000 7f4acbfff700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6eb0 0x55908b8be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4aaaffd700 1 -- 172.21.15.52:0/4254292290 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0ba060 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4aaaffd700 1 -- 172.21.15.52:0/4254292290 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4abc09c6a0 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4abc0a07f0 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4aaaffd700 1 -- 172.21.15.52:0/4254292290 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4abc0b5370 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4adc85bb80 10 client.28621 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55908b8cc0c0 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4aaaffd700 1 -- 172.21.15.52:0/4254292290 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4abc0a5020 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4aaaffd700 1 -- 172.21.15.52:0/4254292290 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f4abc09b890 con 0x55908b8b0ce0 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.157+0000 7f4adc85bb80 10 client.28621 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:56.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4adc85bb80 10 client.28621 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4adc85bb80 20 client.28621 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b8d09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4adc85bb80 4 client.28621 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4abc0a07f0 con 0x55908b8ce5e0 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4acaffd700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b8d09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4aaaffd700 10 client.28621 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:56.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.158+0000 7f4acaffd700 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b8d09d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7551b2deb851e21 server_cookie=4153d7d6c9b0711c in_seq=0 out_seq=0 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4aaaffd700 1 -- 172.21.15.52:0/4254292290 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f4abc0a07f0 con 0x55908b8ce5e0 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4aaaffd700 10 client.28621 handle_command_reply: tid=0 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 client.28621 shutdown 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 msgr2=0x55908b8c4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8c4360 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4abc098850 tx=0x7f4abc09ccb0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 shutdown_connections 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b8d09d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6eb0 0x55908b8be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8c4360 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4aaaffd700 0 client.28621 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:56.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.159+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/4254292290 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6590 0x55908b8c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.160+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 >> 172.21.15.52:0/4254292290 conn(0x55908b7dd210 msgr2=0x55908b8aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:56.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.160+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 shutdown_connections 2024-06-02T23:26:56.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.160+0000 7f4adc85bb80 1 -- 172.21.15.52:0/4254292290 wait complete. 2024-06-02T23:26:56.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.160+0000 7f4adc85bb80 20 client.28621 trim_cache size 0 max 16384 2024-06-02T23:26:56.171 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.200+0000 7f4adc85bb80 1 Processor -- start 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.200+0000 7f4adc85bb80 1 -- start start 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 0x55908b8b2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b8aaac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 0x55908b8b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 0x55908b8b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39064/0 (socket says 172.21.15.52:39064) 2024-06-02T23:26:56.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4acbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b8aaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53880/0 (socket says 172.21.15.52:53880) 2024-06-02T23:26:56.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4acb7fe700 1 -- 172.21.15.52:0/446802188 learned_addr learned my addr 172.21.15.52:0/446802188 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:56.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1aa6d0 con 0x55908b8b6eb0 2024-06-02T23:26:56.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1ac330 con 0x55908b8b6590 2024-06-02T23:26:56.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.201+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1aada0 con 0x55908b8b0ce0 2024-06-02T23:26:56.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acbfff700 1 -- 172.21.15.52:0/446802188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 msgr2=0x55908b8b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acbfff700 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8b07a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acbfff700 1 -- 172.21.15.52:0/446802188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 msgr2=0x55908b8b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acbfff700 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 0x55908b8b2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acbfff700 1 -- 172.21.15.52:0/446802188 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55908b8abfa0 con 0x55908b8b6eb0 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acaffd700 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 0x55908b8b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4acbfff700 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b8aaac0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009cd0 tx=0x7f4ac400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154c12ae958d0970 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4ab17fa700 1 -- 172.21.15.52:0/446802188 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac401a030 con 0x55908b8b6eb0 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4ab17fa700 1 -- 172.21.15.52:0/446802188 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4ac4009450 con 0x55908b8b6eb0 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.202+0000 7f4ab17fa700 1 -- 172.21.15.52:0/446802188 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4005020 con 0x55908b8b6eb0 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 msgr2=0x55908b8aaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b8aaac0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4009cd0 tx=0x7f4ac400b040 comp rx=0 tx=0).stop 2024-06-02T23:26:56.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 shutdown_connections 2024-06-02T23:26:56.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b8aaac0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b0ce0 0x55908b8b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/446802188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b6590 0x55908b8b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 >> 172.21.15.52:0/446802188 conn(0x55908b7dd210 msgr2=0x55908b8aa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 shutdown_connections 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.204+0000 7f4adc85bb80 1 -- 172.21.15.52:0/446802188 wait complete. 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.205+0000 7f4adc85bb80 1 -- start start 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.205+0000 7f4adc85bb80 1 Processor -- start 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8c6d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6590 0x55908b8c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4adc85bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b932220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8c6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4acb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8c6d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39068/0 (socket says 172.21.15.52:39068) 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6590 0x55908b8c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.206+0000 7f4acaffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6590 0x55908b8c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53888/0 (socket says 172.21.15.52:53888) 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4acbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b932220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4acbfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b932220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60514/0 (socket says 172.21.15.52:60514) 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3623994355 learned_addr learned my addr 172.21.15.52:0/3623994355 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1abba0 con 0x55908b8b6eb0 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1ab4a0 con 0x55908b8b0ce0 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4acc1a86b0 con 0x55908b8b6590 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3623994355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6590 msgr2=0x55908b8c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6590 0x55908b8c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.207+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3623994355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 msgr2=0x55908b932220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:56.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b932220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4acb7fe700 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55908b932860 con 0x55908b8b0ce0 2024-06-02T23:26:56.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4adc85bb80 10 client.0 _ll_register_callbacks cb 0x55908b797740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:56.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac4009450 con 0x55908b8b0ce0 2024-06-02T23:26:56.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4acb7fe700 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8c6d10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4abc09bc00 tx=0x7f4abc004e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06f3ead4d3e5d5b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:56.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4aaaffd700 10 client.28946 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4acbfff700 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b932220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4aaaffd700 1 -- 172.21.15.52:0/3623994355 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0ba070 con 0x55908b8b0ce0 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4aaaffd700 1 -- 172.21.15.52:0/3623994355 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4abc0a4da0 con 0x55908b8b0ce0 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4aaaffd700 1 -- 172.21.15.52:0/3623994355 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f4abc0cde30 con 0x55908b8b0ce0 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4adc85bb80 10 client.28946 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.208+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=37}) -- 0x55908b932860 con 0x55908b8b0ce0 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4aaaffd700 1 -- 172.21.15.52:0/3623994355 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f4abc09c6a0 con 0x55908b8b0ce0 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4adc85bb80 10 client.28946 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4adc85bb80 10 client.28946 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4adc85bb80 20 client.28946 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b93bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4adc85bb80 4 client.28946 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-02T23:26:56.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x55908b80eee0 con 0x55908b8ce5e0 2024-06-02T23:26:56.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.209+0000 7f4acaffd700 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b93bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:56.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.210+0000 7f4acaffd700 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b93bd30 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=796563b379557127 server_cookie=d11dc9d6b0e8827e in_seq=0 out_seq=0 2024-06-02T23:26:56.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.210+0000 7f4aaaffd700 10 client.28946 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:56.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13621 13.55 MB/sec execute 4 sec latency 266.897 ms 2024-06-02T23:26:56.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.512+0000 7f4aaaffd700 1 -- 172.21.15.52:0/3623994355 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55908b80eee0 con 0x55908b8ce5e0 2024-06-02T23:26:56.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.512+0000 7f4aaaffd700 10 client.28946 handle_command_reply: tid=0 2024-06-02T23:26:56.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.512+0000 7f4adc85bb80 1 client.28946 shutdown 2024-06-02T23:26:56.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.512+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 msgr2=0x55908b8c6d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.512+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8c6d10 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4abc09bc00 tx=0x7f4abc004e30 comp rx=0 tx=0).stop 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 shutdown_connections 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55908b8ce5e0 0x55908b93bd30 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4aaaffd700 0 client.28946 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55908b8b6eb0 0x55908b932220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55908b8b0ce0 0x55908b8c6d10 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 --2- 172.21.15.52:0/3623994355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55908b8b6590 0x55908b8c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 >> 172.21.15.52:0/3623994355 conn(0x55908b7dd210 msgr2=0x55908b8b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 shutdown_connections 2024-06-02T23:26:56.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 1 -- 172.21.15.52:0/3623994355 wait complete. 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.513+0000 7f4adc85bb80 20 client.28946 trim_cache size 0 max 16384 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 37, 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 54, 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "b6419fd5": { 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 54, 2024-06-02T23:26:56.515 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:56.516 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:56.517 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:56.517 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:26:56.517 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:26:56.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ab40a2d40 msgr2=0x7f4ab40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ab40a2d40 0x7f4ab40a5200 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0005fd0 tx=0x7f4ac0005ee0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 msgr2=0x7f4acc19e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc19e750 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009fd0 tx=0x7f4abc005e40 comp rx=0 tx=0).stop 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 shutdown_connections 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4ab40a2d40 0x7f4ab40a5200 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4acc108620 0x7f4acc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4acc102510 0x7f4acc19e750 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 --2- 172.21.15.52:0/3282191106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4acc102ed0 0x7f4acc19ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 >> 172.21.15.52:0/3282191106 conn(0x7f4acc079640 msgr2=0x7f4acc100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 shutdown_connections 2024-06-02T23:26:56.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:56.514+0000 7f4ad1fd8700 1 -- 172.21.15.52:0/3282191106 wait complete. 2024-06-02T23:26:57.254 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b6419fd5', quiesce took 6.9 seconds. Will release after: 23.1 2024-06-02T23:26:57.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 10.84 MB/sec execute 5 sec latency 1115.633 ms 2024-06-02T23:26:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: pgmap v883: 97 pgs: 97 active+clean; 334 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-06-02T23:26:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:26:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:57 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: pgmap v883: 97 pgs: 97 active+clean; 334 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:57 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: pgmap v883: 97 pgs: 97 active+clean; 334 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 1.5 MiB/s wr, 104 op/s 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:57 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:26:58.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 9.04 MB/sec execute 6 sec latency 2115.694 ms 2024-06-02T23:26:58.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/3373237814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8107400 msgr2=0x7f6cb81077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/3373237814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8107400 0x7f6cb81077e0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4009990 tx=0x7f6cb4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/3373237814 shutdown_connections 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/3373237814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8101d50 0x7f6cb810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/3373237814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8107400 0x7f6cb81077e0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/3373237814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cb8101390 0x7f6cb8101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/3373237814 >> 172.21.15.52:0/3373237814 conn(0x7f6cb8078340 msgr2=0x7f6cb8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/3373237814 shutdown_connections 2024-06-02T23:26:58.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.818+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/3373237814 wait complete. 2024-06-02T23:26:58.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:26:58.819 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:26:58.820 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:26:58.820 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:26:58.820 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:26:58.820 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:26:58.820 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:26:58.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 Processor -- start 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 -- start start 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 0x7f6cb810eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8101d50 0x7f6cb810f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cb8107400 0x7f6cb810f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81126d0 con 0x7f6cb8101390 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8112550 con 0x7f6cb8101d50 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbfa4f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb8112850 con 0x7f6cb8107400 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 0x7f6cb810eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 0x7f6cb810eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60546/0 (socket says 172.21.15.52:60546) 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/2441545594 learned_addr learned my addr 172.21.15.52:0/2441545594 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbdfec700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cb8107400 0x7f6cb810f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8101d50 0x7f6cb810f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/2441545594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cb8107400 msgr2=0x7f6cb810f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cb8107400 0x7f6cb810f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/2441545594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8101d50 msgr2=0x7f6cb810f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8101d50 0x7f6cb810f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.819+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/2441545594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cb81a7050 con 0x7f6cb8101390 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.820+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 0x7f6cb810eb50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005910 tx=0x7f6cb400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.820+0000 7f6cae7fc700 1 -- 172.21.15.52:0/2441545594 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4027070 con 0x7f6cb8101390 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.820+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cb81a72e0 con 0x7f6cb8101390 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.820+0000 7f6cae7fc700 1 -- 172.21.15.52:0/2441545594 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb4022470 con 0x7f6cb8101390 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.820+0000 7f6cae7fc700 1 -- 172.21.15.52:0/2441545594 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4017650 con 0x7f6cb8101390 2024-06-02T23:26:58.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.820+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cb81a7730 con 0x7f6cb8101390 2024-06-02T23:26:58.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.822+0000 7f6cae7fc700 1 -- 172.21.15.52:0/2441545594 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6cb402a070 con 0x7f6cb8101390 2024-06-02T23:26:58.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.823+0000 7f6cae7fc700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca40a2c20 0x7f6ca40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.823+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca40a2c20 0x7f6ca40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.823+0000 7f6cae7fc700 1 -- 172.21.15.52:0/2441545594 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6cb40cfe90 con 0x7f6cb8101390 2024-06-02T23:26:58.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.823+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca40a2c20 0x7f6ca40a50e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8078990 tx=0x7f6ca8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:58.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 Processor -- start 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 -- start start 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84058a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058f60 0x55df840529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84052f20 0x55df84053340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55df83fa4810 con 0x55df84058640 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df83f4c720 con 0x55df84058f60 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df83fa07d0 con 0x55df84052f20 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cbd7eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84058a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cbd7eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84058a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60560/0 (socket says 172.21.15.52:60560) 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/3164284667 learned_addr learned my addr 172.21.15.52:0/3164284667 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.824+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058f60 0x55df840529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbdfec700 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84052f20 0x55df84053340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/3164284667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84052f20 msgr2=0x55df84053340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84052f20 0x55df84053340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/3164284667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058f60 msgr2=0x55df840529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058f60 0x55df840529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/3164284667 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df83f42f50 con 0x55df84058640 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84058a20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40a4e60 tx=0x7f6cb4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44f6b0a02bb23b96 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6ca37fe700 1 -- 172.21.15.52:0/3164284667 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb409c7b0 con 0x55df84058640 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6ca37fe700 1 -- 172.21.15.52:0/3164284667 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb40a0b60 con 0x55df84058640 2024-06-02T23:26:58.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.825+0000 7f6ca37fe700 1 -- 172.21.15.52:0/3164284667 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb4022a70 con 0x55df84058640 2024-06-02T23:26:58.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3164284667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 msgr2=0x55df84058a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84058a20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40a4e60 tx=0x7f6cb4009310 comp rx=0 tx=0).stop 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3164284667 shutdown_connections 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84052f20 0x55df84053340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84058a20 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3164284667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058f60 0x55df840529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3164284667 >> 172.21.15.52:0/3164284667 conn(0x55df83f7dcc0 msgr2=0x55df8404d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3164284667 shutdown_connections 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3164284667 wait complete. 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 -- start start 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.826+0000 7f6cca2d2b80 1 Processor -- start 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df8405ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df8405c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55df8405f6f0 con 0x55df84058640 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55df83f44bf0 con 0x55df84052f20 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df8405f870 con 0x55df84058f60 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cca2d2b80 10 client.0 _ll_register_callbacks cb 0x55df83f3e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df8405c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df8405c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60570/0 (socket says 172.21.15.52:60570) 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 -- 172.21.15.52:0/927213150 learned_addr learned my addr 172.21.15.52:0/927213150 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df8405ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbdfec700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 -- 172.21.15.52:0/927213150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 msgr2=0x55df8405c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 -- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 msgr2=0x55df8405ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df8405ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.827+0000 7f6cbcfea700 1 -- 172.21.15.52:0/927213150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df83ef38d0 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df8405c030 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8012010 tx=0x7f6ca80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb06e508f73592d1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6ca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/927213150 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca8029670 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb40a0b60 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/927213150 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6ca8029810 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/927213150 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6ca8030070 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cca2d2b80 10 client.19578 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55df8406a150 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/927213150 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb40a0b60 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/927213150 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f6ca8020c00 con 0x55df84058640 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cca2d2b80 10 client.19578 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cca2d2b80 10 client.19578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.828+0000 7f6cca2d2b80 20 client.19578 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:58.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.829+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df8406c790 0x55df8406eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.829+0000 7f6cca2d2b80 4 client.19578 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.829+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df83fb1d10 con 0x55df8406c790 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.829+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df8406c790 0x55df8406eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.829+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df8406c790 0x55df8406eba0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e099aa098e87c99 server_cookie=ffca7f668fbe614a in_seq=0 out_seq=0 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.829+0000 7f6ca0ff9700 10 client.19578 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/927213150 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55df83fb1d10 con 0x55df8406c790 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6ca0ff9700 10 client.19578 handle_command_reply: tid=0 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 client.19578 shutdown 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 msgr2=0x55df8405c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df8405c030 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8012010 tx=0x7f6ca80047c0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 shutdown_connections 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df8406c790 0x55df8406eba0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df8405ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6ca0ff9700 0 client.19578 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:58.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/927213150 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df8405c030 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 >> 172.21.15.52:0/927213150 conn(0x55df83f7dcc0 msgr2=0x55df8404d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:58.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 shutdown_connections 2024-06-02T23:26:58.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/927213150 wait complete. 2024-06-02T23:26:58.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.830+0000 7f6cca2d2b80 20 client.19578 trim_cache size 0 max 16384 2024-06-02T23:26:58.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.851+0000 7f6cca2d2b80 1 Processor -- start 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.851+0000 7f6cca2d2b80 1 -- start start 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 0x55df840529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058640 0x55df84047690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81aeaa0 con 0x55df84052f20 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81b0700 con 0x55df84058640 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81af170 con 0x55df84058f60 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 0x55df840529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 0x55df840529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60582/0 (socket says 172.21.15.52:60582) 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/796173482 learned_addr learned my addr 172.21.15.52:0/796173482 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:58.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbdfec700 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058640 0x55df84047690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/796173482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 msgr2=0x55df8405b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/796173482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058640 msgr2=0x55df84047690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058640 0x55df84047690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 -- 172.21.15.52:0/796173482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df84059a20 con 0x55df84052f20 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 0x55df840529e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40cda10 tx=0x7f6cb4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=399ce1246e8f8072 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6ca2ffd700 1 -- 172.21.15.52:0/796173482 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40a5390 con 0x55df84052f20 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6ca2ffd700 1 -- 172.21.15.52:0/796173482 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb409c7b0 con 0x55df84052f20 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.852+0000 7f6ca2ffd700 1 -- 172.21.15.52:0/796173482 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40870b0 con 0x55df84052f20 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/796173482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 msgr2=0x55df840529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 0x55df840529e0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f6cb40cda10 tx=0x7f6cb4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/796173482 shutdown_connections 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df8405b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84052f20 0x55df840529e0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/796173482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84058640 0x55df84047690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/796173482 >> 172.21.15.52:0/796173482 conn(0x55df83f7dcc0 msgr2=0x55df8405a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/796173482 shutdown_connections 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/796173482 wait complete. 2024-06-02T23:26:58.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.853+0000 7f6cca2d2b80 1 -- start start 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 Processor -- start 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df84064f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84061b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df84062040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81aff70 con 0x55df84058640 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81af870 con 0x55df84052f20 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6cb81aca60 con 0x55df84058f60 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cca2d2b80 10 client.0 _ll_register_callbacks cb 0x55df83f3e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84061b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84061b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60592/0 (socket says 172.21.15.52:60592) 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 -- 172.21.15.52:0/3889687695 learned_addr learned my addr 172.21.15.52:0/3889687695 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:26:58.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbdfec700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df84062040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df84064f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 -- 172.21.15.52:0/3889687695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 msgr2=0x55df84062040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df84062040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 -- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 msgr2=0x55df84064f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df84064f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.854+0000 7f6cbcfea700 1 -- 172.21.15.52:0/3889687695 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df84062580 con 0x55df84058640 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6cbcfea700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84061b00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8012010 tx=0x7f6ca801f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31eb105db78b731 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6ca0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/3889687695 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ca80119f0 con 0x55df84058640 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6cb409c7b0 con 0x55df84058640 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/3889687695 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6ca801fc50 con 0x55df84058640 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/3889687695 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f6ca8012c00 con 0x55df84058640 2024-06-02T23:26:58.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6cca2d2b80 10 client.19584 fetch_fsmap learned FSMap version 37 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55df840d81b0 con 0x55df84058640 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/3889687695 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6cb409c7b0 con 0x55df84058640 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.855+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/3889687695 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f6ca8003ac0 con 0x55df84058640 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cca2d2b80 10 client.19584 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cca2d2b80 10 client.19584 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cca2d2b80 20 client.19584 populate_metadata read hostname 'smithi052' 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df840d8940 0x55df840dad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cca2d2b80 4 client.19584 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55df83fac290 con 0x55df840d8940 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df840d8940 0x55df840dad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:26:58.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6cbd7eb700 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df840d8940 0x55df840dad60 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47c83b5db64351c1 server_cookie=a81955720e0ec292 in_seq=0 out_seq=0 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.856+0000 7f6ca0ff9700 10 client.19584 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6ca0ff9700 1 -- 172.21.15.52:0/3889687695 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55df83fac290 con 0x55df840d8940 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6ca0ff9700 10 client.19584 handle_command_reply: tid=0 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 client.19584 shutdown 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 msgr2=0x55df84061b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84061b00 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f6ca8012010 tx=0x7f6ca801f850 comp rx=0 tx=0).stop 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 shutdown_connections 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55df840d8940 0x55df840dad60 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df84058f60 0x55df84062040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55df84052f20 0x55df84064f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 --2- 172.21.15.52:0/3889687695 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55df84058640 0x55df84061b00 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6ca0ff9700 0 client.19584 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:26:58.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 >> 172.21.15.52:0/3889687695 conn(0x55df83f7dcc0 msgr2=0x55df8405b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:58.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 shutdown_connections 2024-06-02T23:26:58.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 1 -- 172.21.15.52:0/3889687695 wait complete. 2024-06-02T23:26:58.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.857+0000 7f6cca2d2b80 20 client.19584 trim_cache size 0 max 16384 2024-06-02T23:26:58.868 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:26:58.868 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:26:58.869 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:26:58.869 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:26:58.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca40a2c20 msgr2=0x7f6ca40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca40a2c20 0x7f6ca40a50e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8078990 tx=0x7f6ca8009260 comp rx=0 tx=0).stop 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 msgr2=0x7f6cb810eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 0x7f6cb810eb50 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f6cb4005910 tx=0x7f6cb400b6c0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 shutdown_connections 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ca40a2c20 0x7f6ca40a50e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6cb8107400 0x7f6cb810f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6cb8101390 0x7f6cb810eb50 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 --2- 172.21.15.52:0/2441545594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6cb8101d50 0x7f6cb810f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 >> 172.21.15.52:0/2441545594 conn(0x7f6cb8078340 msgr2=0x7f6cb80ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 shutdown_connections 2024-06-02T23:26:58.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:26:58.858+0000 7f6cbfa4f700 1 -- 172.21.15.52:0/2441545594 wait complete. 2024-06-02T23:26:58.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:26:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:26:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:26:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:26:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:26:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:26:58.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:26:59.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:26:59.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 7.75 MB/sec execute 7 sec latency 3115.772 ms 2024-06-02T23:26:59.817 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:26:59.817 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:26:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:26:59 smithi174 ceph-mon[35994]: pgmap v884: 97 pgs: 97 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-06-02T23:26:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:26:59 smithi052 ceph-mon[32340]: pgmap v884: 97 pgs: 97 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-06-02T23:27:00.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:26:59 smithi123 ceph-mon[35977]: pgmap v884: 97 pgs: 97 active+clean; 338 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 457 KiB/s rd, 2.1 MiB/s wr, 141 op/s 2024-06-02T23:27:00.077 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:00.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 6.78 MB/sec execute 8 sec latency 4115.816 ms 2024-06-02T23:27:01.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 6.02 MB/sec execute 9 sec latency 5115.847 ms 2024-06-02T23:27:01.818 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:01 smithi052 ceph-mon[32340]: pgmap v885: 97 pgs: 97 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-06-02T23:27:01.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 -- 172.21.15.52:0/2734443491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f341073d0 msgr2=0x7f6f341077b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:01.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/2734443491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f341073d0 0x7f6f341077b0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f6f24009990 tx=0x7f6f24009ca0 comp rx=0 tx=0).stop 2024-06-02T23:27:01.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 -- 172.21.15.52:0/2734443491 shutdown_connections 2024-06-02T23:27:01.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/2734443491 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f34101d30 0x7f6f3410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:01.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/2734443491 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f341073d0 0x7f6f341077b0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:01.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/2734443491 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f34101390 0x7f6f341017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:01.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 -- 172.21.15.52:0/2734443491 >> 172.21.15.52:0/2734443491 conn(0x7f6f340782c0 msgr2=0x7f6f340786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:01.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 -- 172.21.15.52:0/2734443491 shutdown_connections 2024-06-02T23:27:01.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.837+0000 7f6f3ae75700 1 -- 172.21.15.52:0/2734443491 wait complete. 2024-06-02T23:27:01.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 Processor -- start 2024-06-02T23:27:01.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 -- start start 2024-06-02T23:27:01.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f34101390 0x7f6f34103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 0x7f6f34103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f341073d0 0x7f6f34103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111590 con 0x7f6f34101d30 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111410 con 0x7f6f341073d0 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f3ae75700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6f34111710 con 0x7f6f34101390 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f33fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 0x7f6f34103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f33fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 0x7f6f34103580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60600/0 (socket says 172.21.15.52:60600) 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f33fff700 1 -- 172.21.15.52:0/3810388638 learned_addr learned my addr 172.21.15.52:0/3810388638 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f38c11700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f34101390 0x7f6f34103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.838+0000 7f6f39412700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f341073d0 0x7f6f34103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f33fff700 1 -- 172.21.15.52:0/3810388638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f34101390 msgr2=0x7f6f34103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:01.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f33fff700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f34101390 0x7f6f34103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f33fff700 1 -- 172.21.15.52:0/3810388638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f341073d0 msgr2=0x7f6f34103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f33fff700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f341073d0 0x7f6f34103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f33fff700 1 -- 172.21.15.52:0/3810388638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f341a70e0 con 0x7f6f34101d30 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f33fff700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 0x7f6f34103580 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c9b0 tx=0x7f6f1c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c007b00 con 0x7f6f34101d30 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6f1c00ce50 con 0x7f6f34101d30 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f341a73d0 con 0x7f6f34101d30 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f1c018660 con 0x7f6f34101d30 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.839+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f341a7960 con 0x7f6f34101d30 2024-06-02T23:27:01.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.840+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f34069180 con 0x7f6f34101d30 2024-06-02T23:27:01.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.841+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6f1c00f450 con 0x7f6f34101d30 2024-06-02T23:27:01.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.842+0000 7f6f31ffb700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f200a2ca0 0x7f6f200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:01.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.842+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6f1c0ceec0 con 0x7f6f34101d30 2024-06-02T23:27:01.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.842+0000 7f6f38c11700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f200a2ca0 0x7f6f200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:01.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.843+0000 7f6f38c11700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f200a2ca0 0x7f6f200a5160 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400b5c0 tx=0x7f6f24005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:01.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:01.847+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f6f1c014030 con 0x7f6f34101d30 2024-06-02T23:27:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:01 smithi174 ceph-mon[35994]: pgmap v885: 97 pgs: 97 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-06-02T23:27:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:01 smithi123 ceph-mon[35977]: pgmap v885: 97 pgs: 97 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 571 KiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-06-02T23:27:02.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.099+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6f341a7c20 con 0x7f6f34101d30 2024-06-02T23:27:02.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.100+0000 7f6f31ffb700 1 -- 172.21.15.52:0/3810388638 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+11949 (secure 0 0 0) 0x7f6f1c09be50 con 0x7f6f34101d30 2024-06-02T23:27:02.101 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:27:02.101 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":37,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:25:15.536394+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24449,24470,24451,24434,24436]},"id":1}]} 2024-06-02T23:27:02.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f200a2ca0 msgr2=0x7f6f200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f200a2ca0 0x7f6f200a5160 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400b5c0 tx=0x7f6f24005fd0 comp rx=0 tx=0).stop 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 msgr2=0x7f6f34103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 0x7f6f34103580 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f6f1c00c9b0 tx=0x7f6f1c00ccc0 comp rx=0 tx=0).stop 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 shutdown_connections 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6f200a2ca0 0x7f6f200a5160 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6f341073d0 0x7f6f34103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6f34101390 0x7f6f34103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 --2- 172.21.15.52:0/3810388638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6f34101d30 0x7f6f34103580 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 >> 172.21.15.52:0/3810388638 conn(0x7f6f340782c0 msgr2=0x7f6f34100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 shutdown_connections 2024-06-02T23:27:02.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:02.103+0000 7f6f3ae75700 1 -- 172.21.15.52:0/3810388638 wait complete. 2024-06-02T23:27:02.105 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 37 2024-06-02T23:27:02.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 5.42 MB/sec execute 10 sec latency 6115.876 ms 2024-06-02T23:27:02.755 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:02 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3810388638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:27:02.784 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:27:02.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:02 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3810388638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:27:03.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:02 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3810388638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:27:03.035 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:03.453 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 4.93 MB/sec execute 11 sec latency 7116.224 ms 2024-06-02T23:27:03.887 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:03 smithi052 ceph-mon[32340]: pgmap v886: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-06-02T23:27:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:03 smithi174 ceph-mon[35994]: pgmap v886: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-06-02T23:27:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:03 smithi123 ceph-mon[35977]: pgmap v886: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 2.6 MiB/s wr, 191 op/s 2024-06-02T23:27:04.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 4.52 MB/sec execute 12 sec latency 8116.341 ms 2024-06-02T23:27:04.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 -- 172.21.15.52:0/1295378715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406ab10 msgr2=0x7f081410f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 --2- 172.21.15.52:0/1295378715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406ab10 0x7f081410f320 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f080c00b0d0 tx=0x7f080c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 -- 172.21.15.52:0/1295378715 shutdown_connections 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 --2- 172.21.15.52:0/1295378715 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406ab10 0x7f081410f320 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 --2- 172.21.15.52:0/1295378715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f08141084e0 0x7f08141088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 --2- 172.21.15.52:0/1295378715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f081406a170 0x7f081406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 -- 172.21.15.52:0/1295378715 >> 172.21.15.52:0/1295378715 conn(0x7f08140782c0 msgr2=0x7f08140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 -- 172.21.15.52:0/1295378715 shutdown_connections 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.817+0000 7f08195ae700 1 -- 172.21.15.52:0/1295378715 wait complete. 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 Processor -- start 2024-06-02T23:27:04.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 -- start start 2024-06-02T23:27:04.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 0x7f081419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f081406ab10 0x7f081419ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08141084e0 0x7f0814198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08141125e0 con 0x7f081406a170 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0814112460 con 0x7f08141084e0 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.818+0000 7f08195ae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0814112760 con 0x7f081406ab10 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 0x7f081419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 0x7f081419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59502/0 (socket says 172.21.15.52:59502) 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 -- 172.21.15.52:0/1671883009 learned_addr learned my addr 172.21.15.52:0/1671883009 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f080a7fc700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f081406ab10 0x7f081419ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f08137fe700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08141084e0 0x7f0814198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 -- 172.21.15.52:0/1671883009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f081406ab10 msgr2=0x7f081419ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f081406ab10 0x7f081419ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 -- 172.21.15.52:0/1671883009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08141084e0 msgr2=0x7f0814198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08141084e0 0x7f0814198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 -- 172.21.15.52:0/1671883009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0814198fe0 con 0x7f081406a170 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0812ffd700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 0x7f081419e680 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f080400c9b0 tx=0x7f080400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0810ff9700 1 -- 172.21.15.52:0/1671883009 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804007b00 con 0x7f081406a170 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08141992d0 con 0x7f081406a170 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0810ff9700 1 -- 172.21.15.52:0/1671883009 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0804007ca0 con 0x7f081406a170 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.819+0000 7f0810ff9700 1 -- 172.21.15.52:0/1671883009 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0804018660 con 0x7f081406a170 2024-06-02T23:27:04.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.820+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08141a54e0 con 0x7f081406a170 2024-06-02T23:27:04.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.822+0000 7f0810ff9700 1 -- 172.21.15.52:0/1671883009 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0804004830 con 0x7f081406a170 2024-06-02T23:27:04.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.822+0000 7f0810ff9700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f08000a2c20 0x7f08000a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f0810ff9700 1 -- 172.21.15.52:0/1671883009 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f08040cefb0 con 0x7f081406a170 2024-06-02T23:27:04.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f080a7fc700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f08000a2c20 0x7f08000a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f0823e31b80 1 Processor -- start 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f080a7fc700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f08000a2c20 0x7f08000a50e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f08140787e0 tx=0x7f07fc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f0823e31b80 1 -- start start 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 0x5565af2313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af2318f0 0x5565af2356f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.823+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af236090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0823e31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565af17e810 con 0x5565af230fd0 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0823e31b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565af126720 con 0x5565af235c30 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0823e31b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565af17a7d0 con 0x5565af2318f0 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 0x5565af2313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 0x5565af2313b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59508/0 (socket says 172.21.15.52:59508) 2024-06-02T23:27:04.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 -- 172.21.15.52:0/147177299 learned_addr learned my addr 172.21.15.52:0/147177299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f08137fe700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af236090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f080a7fc700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af2318f0 0x5565af2356f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 -- 172.21.15.52:0/147177299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af2318f0 msgr2=0x5565af2356f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af2318f0 0x5565af2356f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 -- 172.21.15.52:0/147177299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 msgr2=0x5565af236090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af236090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 -- 172.21.15.52:0/147177299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565af11cf50 con 0x5565af230fd0 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f08137fe700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af236090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f080a7fc700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af2318f0 0x5565af2356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f0812ffd700 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 0x5565af2313b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f08040a3f80 tx=0x7f080400a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7739762cde17344 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f08097fa700 1 -- 172.21.15.52:0/147177299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08040b3020 con 0x5565af230fd0 2024-06-02T23:27:04.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f08097fa700 1 -- 172.21.15.52:0/147177299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08040a44b0 con 0x5565af230fd0 2024-06-02T23:27:04.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.824+0000 7f08097fa700 1 -- 172.21.15.52:0/147177299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f080409cb70 con 0x5565af230fd0 2024-06-02T23:27:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 -- 172.21.15.52:0/147177299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 msgr2=0x5565af2313b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 0x5565af2313b0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f08040a3f80 tx=0x7f080400a6d0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 -- 172.21.15.52:0/147177299 shutdown_connections 2024-06-02T23:27:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af236090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af230fd0 0x5565af2313b0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 --2- 172.21.15.52:0/147177299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af2318f0 0x5565af2356f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 -- 172.21.15.52:0/147177299 >> 172.21.15.52:0/147177299 conn(0x5565af157cc0 msgr2=0x5565af227850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 -- 172.21.15.52:0/147177299 shutdown_connections 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.825+0000 7f0823e31b80 1 -- 172.21.15.52:0/147177299 wait complete. 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 -- start start 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 Processor -- start 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af230fd0 0x5565af23d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af235c30 0x5565af23aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5565af23ddd0 con 0x5565af2318f0 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565af11ebf0 con 0x5565af230fd0 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5565af23df50 con 0x5565af235c30 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0823e31b80 10 client.0 _ll_register_callbacks cb 0x5565af118920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59512/0 (socket says 172.21.15.52:59512) 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 -- 172.21.15.52:0/4025751278 learned_addr learned my addr 172.21.15.52:0/4025751278 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f0812ffd700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af230fd0 0x5565af23d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f08137fe700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af235c30 0x5565af23aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 -- 172.21.15.52:0/4025751278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af235c30 msgr2=0x5565af23aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af235c30 0x5565af23aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 -- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af230fd0 msgr2=0x5565af23d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af230fd0 0x5565af23d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.826+0000 7f080a7fc700 1 -- 172.21.15.52:0/4025751278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565af0cd8d0 con 0x5565af2318f0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f080a7fc700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23a530 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0046e0 tx=0x7f07fc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd91cb110711c944 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f07fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f07fa7fc700 1 -- 172.21.15.52:0/4025751278 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc016070 con 0x5565af2318f0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f07fa7fc700 1 -- 172.21.15.52:0/4025751278 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f07fc027670 con 0x5565af2318f0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f08040a44b0 con 0x5565af2318f0 2024-06-02T23:27:04.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f07fa7fc700 1 -- 172.21.15.52:0/4025751278 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f07fc02e070 con 0x5565af2318f0 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f0823e31b80 10 client.19602 fetch_fsmap learned FSMap version 37 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5565af248800 con 0x5565af2318f0 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f07fa7fc700 1 -- 172.21.15.52:0/4025751278 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f08040a44b0 con 0x5565af2318f0 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f07fa7fc700 1 -- 172.21.15.52:0/4025751278 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f07fc019070 con 0x5565af2318f0 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f0823e31b80 10 client.19602 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f0823e31b80 10 client.19602 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.827+0000 7f0823e31b80 20 client.19602 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.828+0000 7f0823e31b80 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af24ae40 0x5565af24d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.828+0000 7f0823e31b80 4 client.19602 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.828+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565af18bd10 con 0x5565af24ae40 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.828+0000 7f0812ffd700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af24ae40 0x5565af24d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.828+0000 7f0812ffd700 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af24ae40 0x5565af24d230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e7a0ae4b2da68268 server_cookie=70f77fc5cdeefd5b in_seq=0 out_seq=0 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.828+0000 7f07fa7fc700 10 client.19602 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:27:04.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f07fa7fc700 1 -- 172.21.15.52:0/4025751278 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5565af18bd10 con 0x5565af24ae40 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f07fa7fc700 10 client.19602 handle_command_reply: tid=0 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 client.19602 shutdown 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 msgr2=0x5565af23a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23a530 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f07fc0046e0 tx=0x7f07fc0047c0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 shutdown_connections 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af235c30 0x5565af23aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af24ae40 0x5565af24d230 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af230fd0 0x5565af23d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 --2- 172.21.15.52:0/4025751278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23a530 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 >> 172.21.15.52:0/4025751278 conn(0x5565af157cc0 msgr2=0x5565af22b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f07fa7fc700 0 client.19602 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 shutdown_connections 2024-06-02T23:27:04.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 1 -- 172.21.15.52:0/4025751278 wait complete. 2024-06-02T23:27:04.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.829+0000 7f0823e31b80 20 client.19602 trim_cache size 0 max 16384 2024-06-02T23:27:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 Processor -- start 2024-06-02T23:27:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 -- start start 2024-06-02T23:27:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af247070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af236010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af238e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08141aa7c0 con 0x5565af2318f0 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08141ac420 con 0x5565af235c30 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0823e31b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08141aae90 con 0x5565af230fd0 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f080a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af236010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f080a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af236010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59516/0 (socket says 172.21.15.52:59516) 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f080a7fc700 1 -- 172.21.15.52:0/3104003368 learned_addr learned my addr 172.21.15.52:0/3104003368 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f0812ffd700 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af247070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.845+0000 7f08137fe700 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af238e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f080a7fc700 1 -- 172.21.15.52:0/3104003368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 msgr2=0x5565af247070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f080a7fc700 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af247070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f080a7fc700 1 -- 172.21.15.52:0/3104003368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 msgr2=0x5565af238e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f080a7fc700 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af238e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f080a7fc700 1 -- 172.21.15.52:0/3104003368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565af23ce00 con 0x5565af2318f0 2024-06-02T23:27:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f08137fe700 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af238e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f080a7fc700 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af236010 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=185e4bcf4c0d7321 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f0808ff9700 1 -- 172.21.15.52:0/3104003368 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc010630 con 0x5565af2318f0 2024-06-02T23:27:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f0808ff9700 1 -- 172.21.15.52:0/3104003368 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07fc022070 con 0x5565af2318f0 2024-06-02T23:27:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.846+0000 7f0808ff9700 1 -- 172.21.15.52:0/3104003368 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc0276d0 con 0x5565af2318f0 2024-06-02T23:27:04.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 -- 172.21.15.52:0/3104003368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 msgr2=0x5565af236010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af236010 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc004750 comp rx=0 tx=0).stop 2024-06-02T23:27:04.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 -- 172.21.15.52:0/3104003368 shutdown_connections 2024-06-02T23:27:04.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af238e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af247070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 --2- 172.21.15.52:0/3104003368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af236010 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 -- 172.21.15.52:0/3104003368 >> 172.21.15.52:0/3104003368 conn(0x5565af157cc0 msgr2=0x5565af23a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 -- 172.21.15.52:0/3104003368 shutdown_connections 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 -- 172.21.15.52:0/3104003368 wait complete. 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 -- start start 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.847+0000 7f0823e31b80 1 Processor -- start 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af23e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af2b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f08141abc90 con 0x5565af2318f0 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f08141ab590 con 0x5565af235c30 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f08141a8760 con 0x5565af230fd0 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0823e31b80 10 client.0 _ll_register_callbacks cb 0x5565af118920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59528/0 (socket says 172.21.15.52:59528) 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 -- 172.21.15.52:0/941620808 learned_addr learned my addr 172.21.15.52:0/941620808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f0812ffd700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af23e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f08137fe700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af2b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 -- 172.21.15.52:0/941620808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 msgr2=0x5565af23e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af23e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 -- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 msgr2=0x5565af2b3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af2b3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 -- 172.21.15.52:0/941620808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565af2b4000 con 0x5565af2318f0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f08137fe700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af2b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f080a7fc700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23ecc0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f07fc010c20 tx=0x7f07fc0106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62704924e9e47f44 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f07fa7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.848+0000 7f07fa7fc700 1 -- 172.21.15.52:0/941620808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07fc040070 con 0x5565af2318f0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f07fa7fc700 1 -- 172.21.15.52:0/941620808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f07fc033890 con 0x5565af2318f0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07fc022070 con 0x5565af2318f0 2024-06-02T23:27:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f07fa7fc700 1 -- 172.21.15.52:0/941620808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f07fc016070 con 0x5565af2318f0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 10 client.19608 fetch_fsmap learned FSMap version 37 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x5565af2b5260 con 0x5565af2318f0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f07fa7fc700 1 -- 172.21.15.52:0/941620808 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07fc022070 con 0x5565af2318f0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f07fa7fc700 1 -- 172.21.15.52:0/941620808 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 37) ==== 8382+0+0 (secure 0 0 0) 0x7f07fc03b070 con 0x5565af2318f0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 10 client.19608 fetch_fsmap finished waiting for FSMap version 37 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 10 client.19608 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 20 client.19608 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af2b57d0 0x5565af2b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 4 client.19608 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565af186290 con 0x5565af2b57d0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.849+0000 7f0812ffd700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af2b57d0 0x5565af2b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0812ffd700 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af2b57d0 0x5565af2b7bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ab26b242466a09d3 server_cookie=6955412d1300924c in_seq=0 out_seq=0 2024-06-02T23:27:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f07fa7fc700 10 client.19608 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f07fa7fc700 1 -- 172.21.15.52:0/941620808 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565af186290 con 0x5565af2b57d0 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f07fa7fc700 10 client.19608 handle_command_reply: tid=0 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 client.19608 shutdown 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 msgr2=0x5565af23ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23ecc0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f07fc010c20 tx=0x7f07fc0106c0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 shutdown_connections 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5565af2b57d0 0x5565af2b7bf0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565af235c30 0x5565af2b3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5565af230fd0 0x5565af23e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f07fa7fc700 0 client.19608 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 --2- 172.21.15.52:0/941620808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5565af2318f0 0x5565af23ecc0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 >> 172.21.15.52:0/941620808 conn(0x5565af157cc0 msgr2=0x5565af238de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 shutdown_connections 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 1 -- 172.21.15.52:0/941620808 wait complete. 2024-06-02T23:27:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.850+0000 7f0823e31b80 20 client.19608 trim_cache size 0 max 16384 2024-06-02T23:27:04.856 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:27:04.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f08000a2c20 msgr2=0x7f08000a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f08000a2c20 0x7f08000a50e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f08140787e0 tx=0x7f07fc00a380 comp rx=0 tx=0).stop 2024-06-02T23:27:04.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 msgr2=0x7f081419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 0x7f081419e680 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f080400c9b0 tx=0x7f080400ccc0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 shutdown_connections 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f08000a2c20 0x7f08000a50e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f08141084e0 0x7f0814198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f081406a170 0x7f081419e680 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 --2- 172.21.15.52:0/1671883009 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f081406ab10 0x7f081419ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 >> 172.21.15.52:0/1671883009 conn(0x7f08140782c0 msgr2=0x7f08140ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 shutdown_connections 2024-06-02T23:27:04.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:04.851+0000 7f08195ae700 1 -- 172.21.15.52:0/1671883009 wait complete. 2024-06-02T23:27:05.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 4.17 MB/sec execute 13 sec latency 9116.397 ms 2024-06-02T23:27:05.542 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:27:05.792 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:05 smithi174 ceph-mon[35994]: pgmap v887: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.7 MiB/s wr, 187 op/s 2024-06-02T23:27:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:05 smithi052 ceph-mon[32340]: pgmap v887: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.7 MiB/s wr, 187 op/s 2024-06-02T23:27:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:05 smithi123 ceph-mon[35977]: pgmap v887: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 543 KiB/s rd, 2.7 MiB/s wr, 187 op/s 2024-06-02T23:27:06.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 3.87 MB/sec execute 14 sec latency 10116.456 ms 2024-06-02T23:27:06.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:06 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:06.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:06 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:06 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:07.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 3.61 MB/sec execute 15 sec latency 11116.523 ms 2024-06-02T23:27:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 -- 172.21.15.52:0/1901015815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb50108500 msgr2=0x7ffb501088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 --2- 172.21.15.52:0/1901015815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb50108500 0x7ffb501088e0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7ffb40009990 tx=0x7ffb40009ca0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 -- 172.21.15.52:0/1901015815 shutdown_connections 2024-06-02T23:27:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 --2- 172.21.15.52:0/1901015815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb5006ab30 0x7ffb5010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 --2- 172.21.15.52:0/1901015815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb50108500 0x7ffb501088e0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 --2- 172.21.15.52:0/1901015815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb5006a170 0x7ffb5006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 -- 172.21.15.52:0/1901015815 >> 172.21.15.52:0/1901015815 conn(0x7ffb50078340 msgr2=0x7ffb50078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:07.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 -- 172.21.15.52:0/1901015815 shutdown_connections 2024-06-02T23:27:07.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.517+0000 7ffb561ce700 1 -- 172.21.15.52:0/1901015815 wait complete. 2024-06-02T23:27:07.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.518+0000 7ffb561ce700 1 Processor -- start 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.518+0000 7ffb561ce700 1 -- start start 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb561ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 0x7ffb5019e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb561ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb5006ab30 0x7ffb5019eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb561ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb50108500 0x7ffb50198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb561ce700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb50112630 con 0x7ffb5006a170 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb561ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501124b0 con 0x7ffb50108500 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb561ce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501127b0 con 0x7ffb5006ab30 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 0x7ffb5019e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 0x7ffb5019e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59544/0 (socket says 172.21.15.52:59544) 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2878174252 learned_addr learned my addr 172.21.15.52:0/2878174252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb50108500 0x7ffb50198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4effd700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb5006ab30 0x7ffb5019eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2878174252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb5006ab30 msgr2=0x7ffb5019eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb5006ab30 0x7ffb5019eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2878174252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb50108500 msgr2=0x7ffb50198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb50108500 0x7ffb50198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2878174252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb501990d0 con 0x7ffb5006a170 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.519+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 0x7ffb5019e760 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffb40000c00 tx=0x7ffb40003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.520+0000 7ffb4cff9700 1 -- 172.21.15.52:0/2878174252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40027070 con 0x7ffb5006a170 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.520+0000 7ffb4cff9700 1 -- 172.21.15.52:0/2878174252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4000bdc0 con 0x7ffb5006a170 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.520+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb50199360 con 0x7ffb5006a170 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.520+0000 7ffb4cff9700 1 -- 172.21.15.52:0/2878174252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb40022cd0 con 0x7ffb5006a170 2024-06-02T23:27:07.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.520+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb501a5630 con 0x7ffb5006a170 2024-06-02T23:27:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.522+0000 7ffb4cff9700 1 -- 172.21.15.52:0/2878174252 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ffb4002a070 con 0x7ffb5006a170 2024-06-02T23:27:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.523+0000 7ffb4cff9700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffb380a2c20 0x7ffb380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.523+0000 7ffb4cff9700 1 -- 172.21.15.52:0/2878174252 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffb400cfd20 con 0x7ffb5006a170 2024-06-02T23:27:07.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.523+0000 7ffb4effd700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffb380a2c20 0x7ffb380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.523+0000 7ffb4effd700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffb380a2c20 0x7ffb380a50e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ffb500789b0 tx=0x7ffb4400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:07.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 Processor -- start 2024-06-02T23:27:07.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 -- start start 2024-06-02T23:27:07.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 0x559fe5f1ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1b390 0x559fe5f14df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f15750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559fe5e62810 con 0x559fe5f1aa70 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559fe5e0a720 con 0x559fe5f15330 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559fe5e5e7d0 con 0x559fe5f1b390 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 0x559fe5f1ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 0x559fe5f1ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59556/0 (socket says 172.21.15.52:59556) 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2490960109 learned_addr learned my addr 172.21.15.52:0/2490960109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4effd700 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1b390 0x559fe5f14df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f15750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2490960109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1b390 msgr2=0x559fe5f14df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1b390 0x559fe5f14df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2490960109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 msgr2=0x559fe5f15750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.526 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f15750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.524+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/2490960109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe5e00f50 con 0x559fe5f1aa70 2024-06-02T23:27:07.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.525+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f15750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:07.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.525+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 0x559fe5f1ae50 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffb40098aa0 tx=0x7ffb4009c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e52ed5253a999b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:07.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.525+0000 7ffb35ffb700 1 -- 172.21.15.52:0/2490960109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4009cd40 con 0x559fe5f1aa70 2024-06-02T23:27:07.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.525+0000 7ffb35ffb700 1 -- 172.21.15.52:0/2490960109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4009c640 con 0x559fe5f1aa70 2024-06-02T23:27:07.527 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.525+0000 7ffb35ffb700 1 -- 172.21.15.52:0/2490960109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4009cb00 con 0x559fe5f1aa70 2024-06-02T23:27:07.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2490960109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 msgr2=0x559fe5f1ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 0x559fe5f1ae50 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7ffb40098aa0 tx=0x7ffb4009c4e0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2490960109 shutdown_connections 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f15750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1aa70 0x559fe5f1ae50 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2490960109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1b390 0x559fe5f14df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2490960109 >> 172.21.15.52:0/2490960109 conn(0x559fe5e3bcc0 msgr2=0x559fe5f0b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2490960109 shutdown_connections 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2490960109 wait complete. 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 -- start start 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.526+0000 7ffb60a51b80 1 Processor -- start 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f15330 0x559fe5f211c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1aa70 0x559fe5f1e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f1ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559fe5f21af0 con 0x559fe5f1b390 2024-06-02T23:27:07.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559fe5e02bf0 con 0x559fe5f1aa70 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559fe5f21c70 con 0x559fe5f15330 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 10 client.0 _ll_register_callbacks cb 0x559fe5dfc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f1ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f1ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59566/0 (socket says 172.21.15.52:59566) 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 -- 172.21.15.52:0/959244082 learned_addr learned my addr 172.21.15.52:0/959244082 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4effd700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1aa70 0x559fe5f1e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f15330 0x559fe5f211c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 -- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f15330 msgr2=0x559fe5f211c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f15330 0x559fe5f211c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 -- 172.21.15.52:0/959244082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1aa70 msgr2=0x559fe5f1e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1aa70 0x559fe5f1e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 -- 172.21.15.52:0/959244082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe5db18d0 con 0x559fe5f1b390 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f15330 0x559fe5f211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:07.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f1ea00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7ffb48011d40 tx=0x7ffb4800f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7780d7817a35b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb2effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb2effd700 1 -- 172.21.15.52:0/959244082 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4801b020 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb2effd700 1 -- 172.21.15.52:0/959244082 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffb4800fac0 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.527+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4009c640 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb2effd700 1 -- 172.21.15.52:0/959244082 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ffb4800fcc0 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 10 client.19620 fetch_fsmap learned FSMap version 38 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559fe5f2c4f0 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb2effd700 1 -- 172.21.15.52:0/959244082 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4009c640 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb2effd700 1 -- 172.21.15.52:0/959244082 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7ffb48015430 con 0x559fe5f1b390 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 10 client.19620 fetch_fsmap finished waiting for FSMap version 38 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 10 client.19620 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 20 client.19620 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f30fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 4 client.19620 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.528+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fe5e6fd10 con 0x559fe5f2ebd0 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.529+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f30fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.529+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f30fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5cdb07d56c53ea0f server_cookie=9822a109326cd0d8 in_seq=0 out_seq=0 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.529+0000 7ffb2effd700 10 client.19620 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb2effd700 1 -- 172.21.15.52:0/959244082 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559fe5e6fd10 con 0x559fe5f2ebd0 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb2effd700 10 client.19620 handle_command_reply: tid=0 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 client.19620 shutdown 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 msgr2=0x559fe5f1ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f1ea00 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7ffb48011d40 tx=0x7ffb4800f4b0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 shutdown_connections 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f1ea00 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f30fe0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb2effd700 0 client.19620 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:27:07.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f15330 0x559fe5f211c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/959244082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1aa70 0x559fe5f1e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 >> 172.21.15.52:0/959244082 conn(0x559fe5e3bcc0 msgr2=0x559fe5f0f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:07.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 shutdown_connections 2024-06-02T23:27:07.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 1 -- 172.21.15.52:0/959244082 wait complete. 2024-06-02T23:27:07.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.530+0000 7ffb60a51b80 20 client.19620 trim_cache size 0 max 16384 2024-06-02T23:27:07.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.545+0000 7ffb60a51b80 1 Processor -- start 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.545+0000 7ffb60a51b80 1 -- start start 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f1de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f2ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f100e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501aa730 con 0x559fe5f1b390 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501ac360 con 0x559fe5f15330 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501aae00 con 0x559fe5f1aa70 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f100e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59570/0 (socket says 172.21.15.52:59570) 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 -- 172.21.15.52:0/2382970134 learned_addr learned my addr 172.21.15.52:0/2382970134 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:07.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f1de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4effd700 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f2ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 -- 172.21.15.52:0/2382970134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 msgr2=0x559fe5f2ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f2ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 -- 172.21.15.52:0/2382970134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 msgr2=0x559fe5f1de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f1de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 -- 172.21.15.52:0/2382970134 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fe5f1ed30 con 0x559fe5f1b390 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f100e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000f80 tx=0x7ffb4800f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de48becd7ab7c4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb357fa700 1 -- 172.21.15.52:0/2382970134 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4801b020 con 0x559fe5f1b390 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb357fa700 1 -- 172.21.15.52:0/2382970134 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4800fac0 con 0x559fe5f1b390 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.546+0000 7ffb357fa700 1 -- 172.21.15.52:0/2382970134 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4800fe60 con 0x559fe5f1b390 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2382970134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 msgr2=0x559fe5f100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f100e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000f80 tx=0x7ffb4800f050 comp rx=0 tx=0).stop 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2382970134 shutdown_connections 2024-06-02T23:27:07.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f1b390 0x559fe5f100e0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f15330 0x559fe5f1de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/2382970134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f2ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2382970134 >> 172.21.15.52:0/2382970134 conn(0x559fe5e3bcc0 msgr2=0x559fe5f0f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2382970134 shutdown_connections 2024-06-02T23:27:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 -- 172.21.15.52:0/2382970134 wait complete. 2024-06-02T23:27:07.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.547+0000 7ffb60a51b80 1 -- start start 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 Processor -- start 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 0x559fe5f270a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1b390 0x559fe5f35ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501abbd0 con 0x559fe5f15330 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501ab500 con 0x559fe5f1b390 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffb501a8710 con 0x559fe5f1aa70 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb60a51b80 10 client.0 _ll_register_callbacks cb 0x559fe5dfc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:07.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 0x559fe5f270a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb4f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 0x559fe5f270a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59580/0 (socket says 172.21.15.52:59580) 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/796718283 learned_addr learned my addr 172.21.15.52:0/796718283 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb4effd700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.548+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1b390 0x559fe5f35ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 msgr2=0x559fe5f275e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f275e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/796718283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1b390 msgr2=0x559fe5f35ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1b390 0x559fe5f35ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4f7fe700 1 -- 172.21.15.52:0/796718283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fe5f364b0 con 0x559fe5f15330 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4ffff700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1b390 0x559fe5f35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb4f7fe700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 0x559fe5f270a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ffb4009c5d0 tx=0x7ffb40009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be55d946f302f3f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb2effd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:07.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb2effd700 1 -- 172.21.15.52:0/796718283 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb400b5070 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb2effd700 1 -- 172.21.15.52:0/796718283 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ffb4009c8a0 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb4800fac0 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb2effd700 1 -- 172.21.15.52:0/796718283 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7ffb4009cd40 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb60a51b80 10 client.19626 fetch_fsmap learned FSMap version 38 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x559fe5f99340 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb2effd700 1 -- 172.21.15.52:0/796718283 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ffb4800fac0 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.549+0000 7ffb2effd700 1 -- 172.21.15.52:0/796718283 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7ffb40085bd0 con 0x559fe5f15330 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb60a51b80 10 client.19626 fetch_fsmap finished waiting for FSMap version 38 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb60a51b80 10 client.19626 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb60a51b80 20 client.19626 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f9b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb60a51b80 4 client.19626 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559fe5e6a290 con 0x559fe5f2ebd0 2024-06-02T23:27:07.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb4effd700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f9b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb4effd700 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f9b830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=abf7d93809cf2fc6 server_cookie=6cce87bf232bb210 in_seq=0 out_seq=0 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.550+0000 7ffb2effd700 10 client.19626 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb2effd700 1 -- 172.21.15.52:0/796718283 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559fe5e6a290 con 0x559fe5f2ebd0 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb2effd700 10 client.19626 handle_command_reply: tid=0 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 client.19626 shutdown 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 msgr2=0x559fe5f270a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 0x559fe5f270a0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ffb4009c5d0 tx=0x7ffb40009ca0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 shutdown_connections 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fe5f1b390 0x559fe5f35ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559fe5f15330 0x559fe5f270a0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559fe5f2ebd0 0x559fe5f9b830 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 --2- 172.21.15.52:0/796718283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559fe5f1aa70 0x559fe5f275e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 >> 172.21.15.52:0/796718283 conn(0x559fe5e3bcc0 msgr2=0x559fe5f0f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:07.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb2effd700 0 client.19626 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:27:07.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 shutdown_connections 2024-06-02T23:27:07.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 1 -- 172.21.15.52:0/796718283 wait complete. 2024-06-02T23:27:07.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.551+0000 7ffb60a51b80 20 client.19626 trim_cache size 0 max 16384 2024-06-02T23:27:07.557 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffb380a2c20 msgr2=0x7ffb380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffb380a2c20 0x7ffb380a50e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ffb500789b0 tx=0x7ffb4400b3c0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 msgr2=0x7ffb5019e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 0x7ffb5019e760 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7ffb40000c00 tx=0x7ffb40003a80 comp rx=0 tx=0).stop 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 shutdown_connections 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ffb380a2c20 0x7ffb380a50e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb50108500 0x7ffb50198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ffb5006a170 0x7ffb5019e760 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 --2- 172.21.15.52:0/2878174252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffb5006ab30 0x7ffb5019eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:07.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 >> 172.21.15.52:0/2878174252 conn(0x7ffb50078340 msgr2=0x7ffb50100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:07.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 shutdown_connections 2024-06-02T23:27:07.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:07.552+0000 7ffb561ce700 1 -- 172.21.15.52:0/2878174252 wait complete. 2024-06-02T23:27:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:07 smithi174 ceph-mon[35994]: pgmap v888: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 2.5 MiB/s wr, 145 op/s 2024-06-02T23:27:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:07 smithi052 ceph-mon[32340]: pgmap v888: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 2.5 MiB/s wr, 145 op/s 2024-06-02T23:27:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:07 smithi123 ceph-mon[35977]: pgmap v888: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 402 KiB/s rd, 2.5 MiB/s wr, 145 op/s 2024-06-02T23:27:08.234 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:27:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 -- 172.21.15.52:0/1866997251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 msgr2=0x7f14d4101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 --2- 172.21.15.52:0/1866997251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4101e40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f14c4009900 tx=0x7f14c402a410 comp rx=0 tx=0).stop 2024-06-02T23:27:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 -- 172.21.15.52:0/1866997251 shutdown_connections 2024-06-02T23:27:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 --2- 172.21.15.52:0/1866997251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14d4102c10 0x7f14d4103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.426 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 --2- 172.21.15.52:0/1866997251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14d41035d0 0x7f14d4109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 --2- 172.21.15.52:0/1866997251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4101e40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 -- 172.21.15.52:0/1866997251 >> 172.21.15.52:0/1866997251 conn(0x7f14d40fcf90 msgr2=0x7f14d40ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 -- 172.21.15.52:0/1866997251 shutdown_connections 2024-06-02T23:27:08.427 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.425+0000 7f14d8b90640 1 -- 172.21.15.52:0/1866997251 wait complete. 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 Processor -- start 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 -- start start 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4197c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14d4102c10 0x7f14d4198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14d41035d0 0x7f14d419f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14d410d540 con 0x7f14d4101a20 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14d410d3c0 con 0x7f14d4102c10 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.426+0000 7f14d8b90640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14d410d6c0 con 0x7f14d41035d0 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4197c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4197c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59588/0 (socket says 172.21.15.52:59588) 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 -- 172.21.15.52:0/3167067796 learned_addr learned my addr 172.21.15.52:0/3167067796 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:08.429 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14d41035d0 0x7f14d419f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14d4102c10 0x7f14d4198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 -- 172.21.15.52:0/3167067796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14d41035d0 msgr2=0x7f14d419f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14d41035d0 0x7f14d419f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 -- 172.21.15.52:0/3167067796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14d4102c10 msgr2=0x7f14d4198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14d4102c10 0x7f14d4198160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 -- 172.21.15.52:0/3167067796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14d419f7f0 con 0x7f14d4101a20 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.427+0000 7f14d259c640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4197c20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f14c40314e0 tx=0x7f14c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.428+0000 7f14c37fe640 1 -- 172.21.15.52:0/3167067796 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c403f030 con 0x7f14d4101a20 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.428+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14d419fa80 con 0x7f14d4101a20 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.428+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14d419ffc0 con 0x7f14d4101a20 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.428+0000 7f14c37fe640 1 -- 172.21.15.52:0/3167067796 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c4003aa0 con 0x7f14d4101a20 2024-06-02T23:27:08.430 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.428+0000 7f14c37fe640 1 -- 172.21.15.52:0/3167067796 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4049060 con 0x7f14d4101a20 2024-06-02T23:27:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.430+0000 7f14c37fe640 1 -- 172.21.15.52:0/3167067796 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f14c4042030 con 0x7f14d4101a20 2024-06-02T23:27:08.432 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.430+0000 7f14db97c740 1 Processor -- start 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 -- start start 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bc9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bce910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bd1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c395b284f0 con 0x55c395bcef80 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c395ac6870 con 0x55c395bc9420 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14db97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c395b244b0 con 0x55c395bc9e10 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bd1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bd1410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59592/0 (socket says 172.21.15.52:59592) 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 -- 172.21.15.52:0/443651794 learned_addr learned my addr 172.21.15.52:0/443651794 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bc9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d259c640 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bce910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.433 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14c37fe640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14bc0a2c30 0x7f14bc0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 -- 172.21.15.52:0/443651794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 msgr2=0x55c395bce910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bce910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 -- 172.21.15.52:0/443651794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 msgr2=0x55c395bc9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bc9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14bc0a2c30 0x7f14bc0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14d1d9b640 1 -- 172.21.15.52:0/443651794 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c395ae29b0 con 0x55c395bcef80 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.431+0000 7f14c37fe640 1 -- 172.21.15.52:0/3167067796 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f14c40ef3e0 con 0x7f14d4101a20 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.432+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bd1410 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009ca0 tx=0x7f14c8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71c40fdc36cbe688 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.432+0000 7f14c0ff9640 1 -- 172.21.15.52:0/443651794 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c801e070 con 0x55c395bcef80 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.432+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14bc0a2c30 0x7f14bc0a50f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55c395bca670 tx=0x7f14cc004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.432+0000 7f14c0ff9640 1 -- 172.21.15.52:0/443651794 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c8019ae0 con 0x55c395bcef80 2024-06-02T23:27:08.434 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.432+0000 7f14c0ff9640 1 -- 172.21.15.52:0/443651794 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c800e9b0 con 0x55c395bcef80 2024-06-02T23:27:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- 172.21.15.52:0/443651794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 msgr2=0x55c395bd1410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bd1410 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f14c8009ca0 tx=0x7f14c8011810 comp rx=0 tx=0).stop 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- 172.21.15.52:0/443651794 shutdown_connections 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bce910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bd1410 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- 172.21.15.52:0/443651794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bc9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- 172.21.15.52:0/443651794 >> 172.21.15.52:0/443651794 conn(0x55c395bc0680 msgr2=0x55c395bc0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- 172.21.15.52:0/443651794 shutdown_connections 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- 172.21.15.52:0/443651794 wait complete. 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- start start 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 Processor -- start 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bd36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bd3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.433+0000 7f14db97c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c395bd1f30 con 0x55c395bcef80 2024-06-02T23:27:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14db97c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c395ab1ae0 con 0x55c395bc9420 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14db97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c395bd20b0 con 0x55c395bc9e10 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14db97c740 10 client.0 _ll_register_callbacks cb 0x55c395aea9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59596/0 (socket says 172.21.15.52:59596) 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 -- 172.21.15.52:0/718677789 learned_addr learned my addr 172.21.15.52:0/718677789 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bd3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d259c640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bd36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 -- 172.21.15.52:0/718677789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 msgr2=0x55c395bd3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bd3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 -- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 msgr2=0x55c395bd36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bd36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 -- 172.21.15.52:0/718677789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c39583c210 con 0x55c395bcef80 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f14cc003dc0 tx=0x7f14cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33c49a3491f0ade server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:08.440 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14b9ffb640 1 -- 172.21.15.52:0/718677789 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc025030 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14b9ffb640 1 -- 172.21.15.52:0/718677789 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f14cc004550 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.434+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14c8019ae0 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14b9ffb640 1 -- 172.21.15.52:0/718677789 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f14cc031050 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 10 client.19638 fetch_fsmap learned FSMap version 38 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c395908620 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14b9ffb640 1 -- 172.21.15.52:0/718677789 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14c8019ae0 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14b9ffb640 1 -- 172.21.15.52:0/718677789 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f14cc01f040 con 0x55c395bcef80 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 10 client.19638 fetch_fsmap finished waiting for FSMap version 38 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 10 client.19638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 20 client.19638 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395bdf640 0x55c395be1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 4 client.19638 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c395b3bf10 con 0x55c395bdf640 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.435+0000 7f14d259c640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395bdf640 0x55c395be1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.436+0000 7f14d259c640 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395bdf640 0x55c395be1ae0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c6ebddba32c3d7a server_cookie=9942f0520e55d674 in_seq=0 out_seq=0 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.436+0000 7f14b9ffb640 10 client.19638 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14b9ffb640 1 -- 172.21.15.52:0/718677789 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c395b3bf10 con 0x55c395bdf640 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14b9ffb640 10 client.19638 handle_command_reply: tid=0 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 client.19638 shutdown 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 msgr2=0x55c395bdac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac50 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f14cc003dc0 tx=0x7f14cc004a40 comp rx=0 tx=0).stop 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 shutdown_connections 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395bdf640 0x55c395be1ae0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bd3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac50 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14b9ffb640 0 client.19638 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 --2- 172.21.15.52:0/718677789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bd36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 >> 172.21.15.52:0/718677789 conn(0x55c395bc0680 msgr2=0x55c395bbf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 shutdown_connections 2024-06-02T23:27:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 1 -- 172.21.15.52:0/718677789 wait complete. 2024-06-02T23:27:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.437+0000 7f14db97c740 20 client.19638 trim_cache size 0 max 16384 2024-06-02T23:27:08.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 3.39 MB/sec execute 16 sec latency 12116.553 ms 2024-06-02T23:27:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.453+0000 7f14db97c740 1 Processor -- start 2024-06-02T23:27:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.453+0000 7f14db97c740 1 -- start start 2024-06-02T23:27:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14db97c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bcb030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14db97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bcb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14db97c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14db97c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14b400e3b0 con 0x55c395bcef80 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14db97c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14b400ffe0 con 0x55c395bc9420 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14db97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14b400ea80 con 0x55c395bc9e10 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14d2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14d2d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59604/0 (socket says 172.21.15.52:59604) 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14d2d9d640 1 -- 172.21.15.52:0/3722200333 learned_addr learned my addr 172.21.15.52:0/3722200333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14d259c640 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bcb030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.454+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bcb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14d2d9d640 1 -- 172.21.15.52:0/3722200333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 msgr2=0x55c395bcb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bcb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14d2d9d640 1 -- 172.21.15.52:0/3722200333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 msgr2=0x55c395bcb030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bcb030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14d2d9d640 1 -- 172.21.15.52:0/3722200333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c395bc04b0 con 0x55c395bcef80 2024-06-02T23:27:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006010 tx=0x7f14cc0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2823fdf20584758 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14bbfff640 1 -- 172.21.15.52:0/3722200333 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc02e020 con 0x55c395bcef80 2024-06-02T23:27:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14bbfff640 1 -- 172.21.15.52:0/3722200333 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14cc004620 con 0x55c395bcef80 2024-06-02T23:27:08.457 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.455+0000 7f14bbfff640 1 -- 172.21.15.52:0/3722200333 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14cc025040 con 0x55c395bcef80 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- 172.21.15.52:0/3722200333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 msgr2=0x55c395bdac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac10 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f14cc006010 tx=0x7f14cc0058e0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- 172.21.15.52:0/3722200333 shutdown_connections 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bc9e10 0x55c395bcb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bcef80 0x55c395bdac10 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- 172.21.15.52:0/3722200333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9420 0x55c395bcb030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- 172.21.15.52:0/3722200333 >> 172.21.15.52:0/3722200333 conn(0x55c395bc0680 msgr2=0x55c395bd36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- 172.21.15.52:0/3722200333 shutdown_connections 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- 172.21.15.52:0/3722200333 wait complete. 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- start start 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 Processor -- start 2024-06-02T23:27:08.460 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 0x55c395bd7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9e10 0x55c395bd83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bcef80 0x55c395bd9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14b400f850 con 0x55c395bc9420 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14b400f180 con 0x55c395bc9e10 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.456+0000 7f14db97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14b400c390 con 0x55c395bcef80 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14db97c740 10 client.0 _ll_register_callbacks cb 0x7f14cc003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 0x55c395bd7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 0x55c395bd7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59608/0 (socket says 172.21.15.52:59608) 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 -- 172.21.15.52:0/1816948111 learned_addr learned my addr 172.21.15.52:0/1816948111 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9e10 0x55c395bd83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d2d9d640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bcef80 0x55c395bd9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 -- 172.21.15.52:0/1816948111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bcef80 msgr2=0x55c395bd9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bcef80 0x55c395bd9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.461 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 -- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9e10 msgr2=0x55c395bd83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9e10 0x55c395bd83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 -- 172.21.15.52:0/1816948111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c395bd9930 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14d259c640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 0x55c395bd7eb0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f14c40ed280 tx=0x7f14c402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3391da7a8af68f1b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14b9ffb640 1 -- 172.21.15.52:0/1816948111 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c4035040 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14b9ffb640 1 -- 172.21.15.52:0/1816948111 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f14c40b8390 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14cc004620 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14b9ffb640 1 -- 172.21.15.52:0/1816948111 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f14c4003c50 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14db97c740 10 client.19644 fetch_fsmap learned FSMap version 38 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.457+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55c395bdad10 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14b9ffb640 1 -- 172.21.15.52:0/1816948111 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14cc004620 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14b9ffb640 1 -- 172.21.15.52:0/1816948111 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 38) ==== 8378+0+0 (secure 0 0 0) 0x7f14c40bc010 con 0x55c395bc9420 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14db97c740 10 client.19644 fetch_fsmap finished waiting for FSMap version 38 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14db97c740 10 client.19644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:08.462 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14db97c740 20 client.19644 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14db97c740 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395c4a8d0 0x55c395c4ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14db97c740 4 client.19644 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:27:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c395b35c20 con 0x55c395c4a8d0 2024-06-02T23:27:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395c4a8d0 0x55c395c4ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.459+0000 7f14b9ffb640 10 client.19644 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:08.463 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.458+0000 7f14d1d9b640 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395c4a8d0 0x55c395c4ccf0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41aeee9c00ac1711 server_cookie=3423cb62636ceac1 in_seq=0 out_seq=0 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14b9ffb640 1 -- 172.21.15.52:0/1816948111 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c395b35c20 con 0x55c395c4a8d0 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14b9ffb640 10 client.19644 handle_command_reply: tid=0 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 client.19644 shutdown 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 msgr2=0x55c395bd7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 0x55c395bd7eb0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f14c40ed280 tx=0x7f14c402a410 comp rx=0 tx=0).stop 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 shutdown_connections 2024-06-02T23:27:08.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c395c4a8d0 0x55c395c4ccf0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c395bcef80 0x55c395bd9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c395bc9e10 0x55c395bd83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14b9ffb640 0 client.19644 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 --2- 172.21.15.52:0/1816948111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c395bc9420 0x55c395bd7eb0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.483+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 >> 172.21.15.52:0/1816948111 conn(0x55c395bc0680 msgr2=0x55c395bd3610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 shutdown_connections 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14db97c740 1 -- 172.21.15.52:0/1816948111 wait complete. 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14db97c740 20 client.19644 trim_cache size 0 max 16384 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:27:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:27:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:27:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14bc0a2c30 msgr2=0x7f14bc0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14bc0a2c30 0x7f14bc0a50f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x55c395bca670 tx=0x7f14cc004ab0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 msgr2=0x7f14d4197c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4197c20 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f14c40314e0 tx=0x7f14c4004270 comp rx=0 tx=0).stop 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 shutdown_connections 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14bc0a2c30 0x7f14bc0a50f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14d41035d0 0x7f14d419f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14d4102c10 0x7f14d4198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 --2- 172.21.15.52:0/3167067796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f14d4101a20 0x7f14d4197c20 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.484+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 >> 172.21.15.52:0/3167067796 conn(0x7f14d40fcf90 msgr2=0x7f14d4105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.485+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 shutdown_connections 2024-06-02T23:27:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:08.485+0000 7f14d8b90640 1 -- 172.21.15.52:0/3167067796 wait complete. 2024-06-02T23:27:08.494 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:09.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 3.19 MB/sec execute 17 sec latency 13116.643 ms 2024-06-02T23:27:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:09 smithi174 ceph-mon[35994]: pgmap v889: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-06-02T23:27:09.897 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:09 smithi052 ceph-mon[32340]: pgmap v889: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-06-02T23:27:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:09 smithi123 ceph-mon[35977]: pgmap v889: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 378 KiB/s rd, 1.4 MiB/s wr, 104 op/s 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/278518440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 msgr2=0x7f2eec107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/278518440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec107110 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40099c0 tx=0x7f2ed4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/278518440 shutdown_connections 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/278518440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eec107840 0x7f2eec10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/278518440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec102740 0x7f2eec102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/278518440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec107110 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/278518440 >> 172.21.15.52:0/278518440 conn(0x7f2eec0795f0 msgr2=0x7f2eec079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/278518440 shutdown_connections 2024-06-02T23:27:10.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.267+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/278518440 wait complete. 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 Processor -- start 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 -- start start 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec102740 0x7f2eec074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eec107840 0x7f2eec075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec112620 con 0x7f2eec1030f0 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1124a0 con 0x7f2eec102740 2024-06-02T23:27:10.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ef0fa0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1127a0 con 0x7f2eec107840 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59624/0 (socket says 172.21.15.52:59624) 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/3825983792 learned_addr learned my addr 172.21.15.52:0/3825983792 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2eead9d700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eec107840 0x7f2eec075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2eea59c700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec102740 0x7f2eec074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/3825983792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eec107840 msgr2=0x7f2eec075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eec107840 0x7f2eec075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/3825983792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec102740 msgr2=0x7f2eec074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.268+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec102740 0x7f2eec074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/3825983792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec1a6f40 con 0x7f2eec1030f0 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec074cd0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4006010 tx=0x7f2ed4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ee37fe700 1 -- 172.21.15.52:0/3825983792 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4026070 con 0x7f2eec1030f0 2024-06-02T23:27:10.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ee37fe700 1 -- 172.21.15.52:0/3825983792 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed4004ca0 con 0x7f2eec1030f0 2024-06-02T23:27:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ee37fe700 1 -- 172.21.15.52:0/3825983792 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4003d20 con 0x7f2eec1030f0 2024-06-02T23:27:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec1a71d0 con 0x7f2eec1030f0 2024-06-02T23:27:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.269+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec1a75d0 con 0x7f2eec1030f0 2024-06-02T23:27:10.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.272+0000 7f2ee37fe700 1 -- 172.21.15.52:0/3825983792 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ed4029070 con 0x7f2eec1030f0 2024-06-02T23:27:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.273+0000 7f2ee37fe700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed80a2ca0 0x7f2ed80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.273+0000 7f2eea59c700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed80a2ca0 0x7f2ed80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.273+0000 7f2ee37fe700 1 -- 172.21.15.52:0/3825983792 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ed40cf800 con 0x7f2eec1030f0 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.273+0000 7f2efb823b80 1 Processor -- start 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.273+0000 7f2eea59c700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed80a2ca0 0x7f2ed80a5160 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ac50 tx=0x7f2edc00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.273+0000 7f2efb823b80 1 -- start start 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2acea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2acef60 0x5586a2ac89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2ac9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2efb823b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586a2a1a810 con 0x5586a2ace640 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2efb823b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5586a29c2720 con 0x5586a2ac8f20 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2efb823b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586a2a167d0 con 0x5586a2acef60 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2acea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2acea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59632/0 (socket says 172.21.15.52:59632) 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 -- 172.21.15.52:0/2620149106 learned_addr learned my addr 172.21.15.52:0/2620149106 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2acef60 0x5586a2ac89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eead9d700 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2ac9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 -- 172.21.15.52:0/2620149106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2acef60 msgr2=0x5586a2ac89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2acef60 0x5586a2ac89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 -- 172.21.15.52:0/2620149106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 msgr2=0x5586a2ac9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2ac9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 -- 172.21.15.52:0/2620149106 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a29b8f50 con 0x5586a2ace640 2024-06-02T23:27:10.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.274+0000 7f2eea59c700 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2acea20 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ff90 tx=0x7f2edc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668687a47a4e1476 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:10.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2ee0ff9700 1 -- 172.21.15.52:0/2620149106 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc016070 con 0x5586a2ace640 2024-06-02T23:27:10.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2ee0ff9700 1 -- 172.21.15.52:0/2620149106 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc014070 con 0x5586a2ace640 2024-06-02T23:27:10.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2ee0ff9700 1 -- 172.21.15.52:0/2620149106 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc010e60 con 0x5586a2ace640 2024-06-02T23:27:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 -- 172.21.15.52:0/2620149106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 msgr2=0x5586a2acea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2acea20 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ff90 tx=0x7f2edc012000 comp rx=0 tx=0).stop 2024-06-02T23:27:10.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 -- 172.21.15.52:0/2620149106 shutdown_connections 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2ac9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2acea20 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 --2- 172.21.15.52:0/2620149106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2acef60 0x5586a2ac89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 -- 172.21.15.52:0/2620149106 >> 172.21.15.52:0/2620149106 conn(0x5586a29f3cc0 msgr2=0x5586a2ac3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 -- 172.21.15.52:0/2620149106 shutdown_connections 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.275+0000 7f2efb823b80 1 -- 172.21.15.52:0/2620149106 wait complete. 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 -- start start 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 Processor -- start 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ad4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ace640 0x5586a2ad1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ad2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5586a2ad59d0 con 0x5586a2acef60 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5586a29babf0 con 0x5586a2ace640 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586a2ad5b50 con 0x5586a2ac8f20 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2efb823b80 10 client.0 _ll_register_callbacks cb 0x5586a29b4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2eead9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ad2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2eea59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ad4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2eead9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ad2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59636/0 (socket says 172.21.15.52:59636) 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.276+0000 7f2eead9d700 1 -- 172.21.15.52:0/1894306162 learned_addr learned my addr 172.21.15.52:0/1894306162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ace640 0x5586a2ad1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2eead9d700 1 -- 172.21.15.52:0/1894306162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 msgr2=0x5586a2ad4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2eead9d700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ad4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2eead9d700 1 -- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ace640 msgr2=0x5586a2ad1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2eead9d700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ace640 0x5586a2ad1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2eead9d700 1 -- 172.21.15.52:0/1894306162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a29698d0 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ace640 0x5586a2ad1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2eead9d700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ad2510 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4004070 tx=0x7f2ee4004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c011dc7828fe7831 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ed1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/1894306162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee401a020 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/1894306162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ee40079d0 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2edc014070 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/1894306162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2ee4007bd0 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2efb823b80 10 client.19656 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586a2ae0330 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.277+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/1894306162 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc014070 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/1894306162 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f2ee4021400 con 0x5586a2acef60 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2efb823b80 10 client.19656 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:10.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2efb823b80 10 client.19656 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2efb823b80 20 client.19656 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2efb823b80 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2ae29c0 0x5586a2ae4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2efb823b80 4 client.19656 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586a2a27d10 con 0x5586a2ae29c0 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2eea59c700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2ae29c0 0x5586a2ae4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2eea59c700 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2ae29c0 0x5586a2ae4db0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30fb1baf27c14320 server_cookie=f9d8d02fab38904b in_seq=0 out_seq=0 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.278+0000 7f2ed1ffb700 10 client.19656 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/1894306162 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5586a2a27d10 con 0x5586a2ae29c0 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2ed1ffb700 10 client.19656 handle_command_reply: tid=0 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 client.19656 shutdown 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 msgr2=0x5586a2ad2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ad2510 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4004070 tx=0x7f2ee4004380 comp rx=0 tx=0).stop 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 shutdown_connections 2024-06-02T23:27:10.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2ae29c0 0x5586a2ae4db0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ad2510 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ad4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 --2- 172.21.15.52:0/1894306162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ace640 0x5586a2ad1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 >> 172.21.15.52:0/1894306162 conn(0x5586a29f3cc0 msgr2=0x5586a2ac35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.279+0000 7f2ed1ffb700 0 client.19656 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.280+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 shutdown_connections 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.280+0000 7f2efb823b80 1 -- 172.21.15.52:0/1894306162 wait complete. 2024-06-02T23:27:10.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.280+0000 7f2efb823b80 20 client.19656 trim_cache size 0 max 16384 2024-06-02T23:27:10.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 Processor -- start 2024-06-02T23:27:10.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 -- start start 2024-06-02T23:27:10.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ac89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2ad1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2acef60 0x5586a2ad24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1aeb10 con 0x5586a2ace640 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1b0740 con 0x5586a2acef60 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2efb823b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1af1e0 con 0x5586a2ac8f20 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2ad1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2ee9d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2ad1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59646/0 (socket says 172.21.15.52:59646) 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.295+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/254351164 learned_addr learned my addr 172.21.15.52:0/254351164 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2eead9d700 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2acef60 0x5586a2ad24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2eea59c700 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ac89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/254351164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 msgr2=0x5586a2ac89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ac89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/254351164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2acef60 msgr2=0x5586a2ad24d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2acef60 0x5586a2ad24d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ee9d9b700 1 -- 172.21.15.52:0/254351164 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586a2ad0560 con 0x5586a2ace640 2024-06-02T23:27:10.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2ad1f90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40cd330 tx=0x7f2ed4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8889fb7c170afa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:10.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ed3fff700 1 -- 172.21.15.52:0/254351164 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40a04d0 con 0x5586a2ace640 2024-06-02T23:27:10.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ed3fff700 1 -- 172.21.15.52:0/254351164 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed40a4d00 con 0x5586a2ace640 2024-06-02T23:27:10.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.296+0000 7f2ed3fff700 1 -- 172.21.15.52:0/254351164 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed409c5e0 con 0x5586a2ace640 2024-06-02T23:27:10.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 -- 172.21.15.52:0/254351164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 msgr2=0x5586a2ad1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2ad1f90 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40cd330 tx=0x7f2ed4009310 comp rx=0 tx=0).stop 2024-06-02T23:27:10.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 -- 172.21.15.52:0/254351164 shutdown_connections 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2acef60 0x5586a2ad24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ac8f20 0x5586a2ac89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 --2- 172.21.15.52:0/254351164 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2ace640 0x5586a2ad1f90 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 -- 172.21.15.52:0/254351164 >> 172.21.15.52:0/254351164 conn(0x5586a29f3cc0 msgr2=0x5586a2ad08e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 -- 172.21.15.52:0/254351164 shutdown_connections 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 -- 172.21.15.52:0/254351164 wait complete. 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 -- start start 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.297+0000 7f2efb823b80 1 Processor -- start 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2adaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ace640 0x5586a2adb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ae9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1affb0 con 0x5586a2acef60 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1af8e0 con 0x5586a2ac8f20 2024-06-02T23:27:10.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec1acad0 con 0x5586a2ace640 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 10 client.0 _ll_register_callbacks cb 0x5586a29b4920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ae9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ae9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59654/0 (socket says 172.21.15.52:59654) 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 -- 172.21.15.52:0/3583824909 learned_addr learned my addr 172.21.15.52:0/3583824909 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2ee9d9b700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ace640 0x5586a2adb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eea59c700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2adaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 -- 172.21.15.52:0/3583824909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ace640 msgr2=0x5586a2adb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ace640 0x5586a2adb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 -- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 msgr2=0x5586a2adaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2adaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 -- 172.21.15.52:0/3583824909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586a2aea170 con 0x5586a2acef60 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2eead9d700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ae9a70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009db0 tx=0x7f2ee40048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b2e50066e785a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2ed1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/3583824909 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee401a040 con 0x5586a2acef60 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/3583824909 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ee4002030 con 0x5586a2acef60 2024-06-02T23:27:10.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.298+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ed40a4d00 con 0x5586a2acef60 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/3583824909 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2ee4005c60 con 0x5586a2acef60 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 10 client.19662 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5586a2b4d030 con 0x5586a2acef60 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/3583824909 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ed40a4d00 con 0x5586a2acef60 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/3583824909 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f2ee40164f0 con 0x5586a2acef60 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 10 client.19662 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 10 client.19662 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 20 client.19662 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2b4d730 0x5586a2b4fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 4 client.19662 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586a2a22290 con 0x5586a2b4d730 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.299+0000 7f2eea59c700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2b4d730 0x5586a2b4fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2eea59c700 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2b4d730 0x5586a2b4fb50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eb47561303ae5df7 server_cookie=97b410dafeaf5775 in_seq=0 out_seq=0 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2ed1ffb700 10 client.19662 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:27:10.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2ed1ffb700 1 -- 172.21.15.52:0/3583824909 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586a2a22290 con 0x5586a2b4d730 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2ed1ffb700 10 client.19662 handle_command_reply: tid=0 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 client.19662 shutdown 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 msgr2=0x5586a2ae9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ae9a70 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4009db0 tx=0x7f2ee40048c0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 shutdown_connections 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5586a2b4d730 0x5586a2b4fb50 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586a2acef60 0x5586a2ae9a70 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586a2ac8f20 0x5586a2adaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2ed1ffb700 0 client.19662 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 --2- 172.21.15.52:0/3583824909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586a2ace640 0x5586a2adb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 >> 172.21.15.52:0/3583824909 conn(0x5586a29f3cc0 msgr2=0x5586a2ad0770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 shutdown_connections 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.300+0000 7f2efb823b80 1 -- 172.21.15.52:0/3583824909 wait complete. 2024-06-02T23:27:10.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2efb823b80 20 client.19662 trim_cache size 0 max 16384 2024-06-02T23:27:10.306 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:27:10.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed80a2ca0 msgr2=0x7f2ed80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed80a2ca0 0x7f2ed80a5160 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ac50 tx=0x7f2edc00a380 comp rx=0 tx=0).stop 2024-06-02T23:27:10.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 msgr2=0x7f2eec074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:10.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec074cd0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4006010 tx=0x7f2ed4009f80 comp rx=0 tx=0).stop 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 shutdown_connections 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed80a2ca0 0x7f2ed80a5160 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2eec107840 0x7f2eec075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2eec102740 0x7f2eec074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 --2- 172.21.15.52:0/3825983792 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2eec1030f0 0x7f2eec074cd0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 >> 172.21.15.52:0/3825983792 conn(0x7f2eec0795f0 msgr2=0x7f2eec1006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 shutdown_connections 2024-06-02T23:27:10.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:10.301+0000 7f2ef0fa0700 1 -- 172.21.15.52:0/3825983792 wait complete. 2024-06-02T23:27:10.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 3.01 MB/sec execute 18 sec latency 14116.749 ms 2024-06-02T23:27:10.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:10 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:10 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:10 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:11.178 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:27:11.429 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:11.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.85 MB/sec execute 19 sec latency 15116.792 ms 2024-06-02T23:27:11.875 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:11 smithi052 ceph-mon[32340]: pgmap v890: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 793 KiB/s wr, 70 op/s 2024-06-02T23:27:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:11 smithi123 ceph-mon[35977]: pgmap v890: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 793 KiB/s wr, 70 op/s 2024-06-02T23:27:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:11 smithi174 ceph-mon[35994]: pgmap v890: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 150 KiB/s rd, 793 KiB/s wr, 70 op/s 2024-06-02T23:27:12.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.71 MB/sec execute 20 sec latency 16116.825 ms 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 -- 172.21.15.52:0/522903481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 msgr2=0x7f687c10f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 --2- 172.21.15.52:0/522903481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c10f3c0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f687400b0d0 tx=0x7f687400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 -- 172.21.15.52:0/522903481 shutdown_connections 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 --2- 172.21.15.52:0/522903481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c10f3c0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 --2- 172.21.15.52:0/522903481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f687c102740 0x7f687c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 --2- 172.21.15.52:0/522903481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f687c1030f0 0x7f687c107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.225+0000 7f6882b49700 1 -- 172.21.15.52:0/522903481 >> 172.21.15.52:0/522903481 conn(0x7f687c0795f0 msgr2=0x7f687c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 -- 172.21.15.52:0/522903481 shutdown_connections 2024-06-02T23:27:13.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 -- 172.21.15.52:0/522903481 wait complete. 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 Processor -- start 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 -- start start 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f687c102740 0x7f687c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f687c1030f0 0x7f687c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f687c112610 con 0x7f687c107840 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f687c112490 con 0x7f687c1030f0 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.226+0000 7f6882b49700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f687c112790 con 0x7f687c102740 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59674/0 (socket says 172.21.15.52:59674) 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 -- 172.21.15.52:0/3183223853 learned_addr learned my addr 172.21.15.52:0/3183223853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f687bfff700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f687c1030f0 0x7f687c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68808e5700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f687c102740 0x7f687c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 -- 172.21.15.52:0/3183223853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f687c102740 msgr2=0x7f687c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f687c102740 0x7f687c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 -- 172.21.15.52:0/3183223853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f687c1030f0 msgr2=0x7f687c074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f687c1030f0 0x7f687c074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 -- 172.21.15.52:0/3183223853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f687c1a6f50 con 0x7f687c107840 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f68810e6700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c075210 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f68740095c0 tx=0x7f6874007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f6879ffb700 1 -- 172.21.15.52:0/3183223853 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687401f070 con 0x7f687c107840 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f687c1a71e0 con 0x7f687c107840 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f6879ffb700 1 -- 172.21.15.52:0/3183223853 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6874003c00 con 0x7f687c107840 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.227+0000 7f6879ffb700 1 -- 172.21.15.52:0/3183223853 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687401b6a0 con 0x7f687c107840 2024-06-02T23:27:13.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.228+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f687c1a7690 con 0x7f687c107840 2024-06-02T23:27:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.230+0000 7f6879ffb700 1 -- 172.21.15.52:0/3183223853 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6874022070 con 0x7f687c107840 2024-06-02T23:27:13.231 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.231+0000 7f6879ffb700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68640a2c20 0x7f68640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.231+0000 7f68808e5700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68640a2c20 0x7f68640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.231+0000 7f6879ffb700 1 -- 172.21.15.52:0/3183223853 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f687400f020 con 0x7f687c107840 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.231+0000 7f68808e5700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68640a2c20 0x7f68640a50e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f686c005ea0 tx=0x7f686c005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.231+0000 7f688d3ccb80 1 Processor -- start 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.231+0000 7f688d3ccb80 1 -- start start 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029dad340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029da7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029da77a0 0x561029da7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561029c99f60 con 0x561029dacf60 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561029cf8850 con 0x561029da77a0 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561029c60460 con 0x561029dad880 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029dad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029dad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59684/0 (socket says 172.21.15.52:59684) 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 -- 172.21.15.52:0/3103205901 learned_addr learned my addr 172.21.15.52:0/3103205901 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f687bfff700 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029da7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 -- 172.21.15.52:0/3103205901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 msgr2=0x561029da7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029da7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 -- 172.21.15.52:0/3103205901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029da77a0 msgr2=0x561029da7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029da77a0 0x561029da7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 -- 172.21.15.52:0/3103205901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561029c478d0 con 0x561029dacf60 2024-06-02T23:27:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f687bfff700 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029da7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.232+0000 7f68808e5700 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029dad340 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f686c00a520 tx=0x7f686c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2609265e2d47db7c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f6862ffd700 1 -- 172.21.15.52:0/3103205901 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c010d80 con 0x561029dacf60 2024-06-02T23:27:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f6862ffd700 1 -- 172.21.15.52:0/3103205901 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c022070 con 0x561029dacf60 2024-06-02T23:27:13.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f6862ffd700 1 -- 172.21.15.52:0/3103205901 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c00f6b0 con 0x561029dacf60 2024-06-02T23:27:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3103205901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 msgr2=0x561029dad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029dad340 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f686c00a520 tx=0x7f686c00af90 comp rx=0 tx=0).stop 2024-06-02T23:27:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3103205901 shutdown_connections 2024-06-02T23:27:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029da77a0 0x561029da7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029dad340 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3103205901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029da7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3103205901 >> 172.21.15.52:0/3103205901 conn(0x561029cd4080 msgr2=0x561029da2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3103205901 shutdown_connections 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.233+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3103205901 wait complete. 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 -- start start 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 Processor -- start 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561029c98fc0 con 0x561029dacf60 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561029cf4810 con 0x561029dad880 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.234+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561029db4260 con 0x561029da77a0 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f688d3ccb80 10 client.0 _ll_register_callbacks cb 0x561029c8e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59700/0 (socket says 172.21.15.52:59700) 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 -- 172.21.15.52:0/3474179304 learned_addr learned my addr 172.21.15.52:0/3474179304 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f68808e5700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f68810e6700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 -- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 msgr2=0x561029db36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 -- 172.21.15.52:0/3474179304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 msgr2=0x561029db0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 -- 172.21.15.52:0/3474179304 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561029cbe750 con 0x561029dacf60 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f687bfff700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db09f0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f687000ba60 tx=0x7f687000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef05b8b0052d3a30 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f685bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f685bfff700 1 -- 172.21.15.52:0/3474179304 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870003d70 con 0x561029dacf60 2024-06-02T23:27:13.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f685bfff700 1 -- 172.21.15.52:0/3474179304 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6870004830 con 0x561029dacf60 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c022070 con 0x561029dacf60 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f685bfff700 1 -- 172.21.15.52:0/3474179304 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6870021070 con 0x561029dacf60 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.235+0000 7f688d3ccb80 10 client.19674 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561029dbeb50 con 0x561029dacf60 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f685bfff700 1 -- 172.21.15.52:0/3474179304 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c022070 con 0x561029dacf60 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f685bfff700 1 -- 172.21.15.52:0/3474179304 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f6870019030 con 0x561029dacf60 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 10 client.19674 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 10 client.19674 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 20 client.19674 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029dc11e0 0x561029dc35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 4 client.19674 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561029d05d50 con 0x561029dc11e0 2024-06-02T23:27:13.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.236+0000 7f68808e5700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029dc11e0 0x561029dc35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.237+0000 7f68808e5700 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029dc11e0 0x561029dc35d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1460ea0cfbc8a550 server_cookie=a7e124a03cd52fa4 in_seq=0 out_seq=0 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.237+0000 7f685bfff700 10 client.19674 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.237+0000 7f685bfff700 1 -- 172.21.15.52:0/3474179304 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561029d05d50 con 0x561029dc11e0 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.237+0000 7f685bfff700 10 client.19674 handle_command_reply: tid=0 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.237+0000 7f688d3ccb80 1 client.19674 shutdown 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 msgr2=0x561029db09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db09f0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f687000ba60 tx=0x7f687000bd70 comp rx=0 tx=0).stop 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 shutdown_connections 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029dc11e0 0x561029dc35d0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f685bfff700 0 client.19674 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/3474179304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db09f0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 >> 172.21.15.52:0/3474179304 conn(0x561029cd4080 msgr2=0x561029da1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:13.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 shutdown_connections 2024-06-02T23:27:13.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 1 -- 172.21.15.52:0/3474179304 wait complete. 2024-06-02T23:27:13.243 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.238+0000 7f688d3ccb80 20 client.19674 trim_cache size 0 max 16384 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 Processor -- start 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 -- start start 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 0x561029dbd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dacf60 0x561029dab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029dae370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1aebd0 con 0x561029da77a0 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1b0800 con 0x561029dacf60 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1af2a0 con 0x561029dad880 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.253+0000 7f68808e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 0x561029dbd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 0x561029dbd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59714/0 (socket says 172.21.15.52:59714) 2024-06-02T23:27:13.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 -- 172.21.15.52:0/729747162 learned_addr learned my addr 172.21.15.52:0/729747162 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f687bfff700 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dacf60 0x561029dab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 -- 172.21.15.52:0/729747162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 msgr2=0x561029dae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029dae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 -- 172.21.15.52:0/729747162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dacf60 msgr2=0x561029dab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dacf60 0x561029dab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 -- 172.21.15.52:0/729747162 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561029dc3c00 con 0x561029da77a0 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f687bfff700 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dacf60 0x561029dab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68808e5700 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 0x561029dbd3b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f686c00f010 tx=0x7f686c01de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dcda41b786d3b53 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68627fc700 1 -- 172.21.15.52:0/729747162 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c014040 con 0x561029da77a0 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68627fc700 1 -- 172.21.15.52:0/729747162 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c022070 con 0x561029da77a0 2024-06-02T23:27:13.256 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.254+0000 7f68627fc700 1 -- 172.21.15.52:0/729747162 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c004970 con 0x561029da77a0 2024-06-02T23:27:13.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 -- 172.21.15.52:0/729747162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 msgr2=0x561029dbd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 0x561029dbd3b0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f686c00f010 tx=0x7f686c01de50 comp rx=0 tx=0).stop 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 -- 172.21.15.52:0/729747162 shutdown_connections 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029dad880 0x561029dae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029da77a0 0x561029dbd3b0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/729747162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dacf60 0x561029dab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 -- 172.21.15.52:0/729747162 >> 172.21.15.52:0/729747162 conn(0x561029cd4080 msgr2=0x561029dadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 -- 172.21.15.52:0/729747162 shutdown_connections 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 -- 172.21.15.52:0/729747162 wait complete. 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 -- start start 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.255+0000 7f688d3ccb80 1 Processor -- start 2024-06-02T23:27:13.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1b0070 con 0x561029dacf60 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1af9a0 con 0x561029dad880 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f687c1acb90 con 0x561029da77a0 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 10 client.0 _ll_register_callbacks cb 0x561029c8e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59732/0 (socket says 172.21.15.52:59732) 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 -- 172.21.15.52:0/1472947218 learned_addr learned my addr 172.21.15.52:0/1472947218 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f68810e6700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 -- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 msgr2=0x561029db9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 -- 172.21.15.52:0/1472947218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 msgr2=0x561029db64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:13.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 -- 172.21.15.52:0/1472947218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561029db6a30 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f68810e6700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f687bfff700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db9a60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6870004420 tx=0x7f68700059c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e59ea0208bdaf7b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f685bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f685bfff700 1 -- 172.21.15.52:0/1472947218 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870003f80 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f685bfff700 1 -- 172.21.15.52:0/1472947218 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68700107f0 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.256+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686c022070 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f685bfff700 1 -- 172.21.15.52:0/1472947218 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6870036050 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 10 client.19680 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x561029e2c490 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f685bfff700 1 -- 172.21.15.52:0/1472947218 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f686c022070 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f685bfff700 1 -- 172.21.15.52:0/1472947218 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f6870019050 con 0x561029dacf60 2024-06-02T23:27:13.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 10 client.19680 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 10 client.19680 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 20 client.19680 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029e2cbf0 0x561029e2f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 4 client.19680 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561029d002d0 con 0x561029e2cbf0 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.257+0000 7f68808e5700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029e2cbf0 0x561029e2f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f68808e5700 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029e2cbf0 0x561029e2f010 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=61fe10c9560961dc server_cookie=28c011561065355 in_seq=0 out_seq=0 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f685bfff700 10 client.19680 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f685bfff700 1 -- 172.21.15.52:0/1472947218 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561029d002d0 con 0x561029e2cbf0 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f685bfff700 10 client.19680 handle_command_reply: tid=0 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 client.19680 shutdown 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 msgr2=0x561029db9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db9a60 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f6870004420 tx=0x7f68700059c0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 shutdown_connections 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x561029e2cbf0 0x561029e2f010 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561029dad880 0x561029db64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561029da77a0 0x561029db9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 --2- 172.21.15.52:0/1472947218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561029dacf60 0x561029db9a60 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f685bfff700 0 client.19680 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 >> 172.21.15.52:0/1472947218 conn(0x561029cd4080 msgr2=0x561029db2ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 shutdown_connections 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 1 -- 172.21.15.52:0/1472947218 wait complete. 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.258+0000 7f688d3ccb80 20 client.19680 trim_cache size 0 max 16384 2024-06-02T23:27:13.264 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:27:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68640a2c20 msgr2=0x7f68640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68640a2c20 0x7f68640a50e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f686c005ea0 tx=0x7f686c005e30 comp rx=0 tx=0).stop 2024-06-02T23:27:13.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 msgr2=0x7f687c075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c075210 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f68740095c0 tx=0x7f6874007a20 comp rx=0 tx=0).stop 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 shutdown_connections 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f68640a2c20 0x7f68640a50e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f687c107840 0x7f687c075210 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f687c102740 0x7f687c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 --2- 172.21.15.52:0/3183223853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f687c1030f0 0x7f687c074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 >> 172.21.15.52:0/3183223853 conn(0x7f687c0795f0 msgr2=0x7f687c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 shutdown_connections 2024-06-02T23:27:13.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:13.259+0000 7f6882b49700 1 -- 172.21.15.52:0/3183223853 wait complete. 2024-06-02T23:27:13.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.58 MB/sec execute 21 sec latency 17116.880 ms 2024-06-02T23:27:14.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:13 smithi123 ceph-mon[35977]: pgmap v891: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 318 KiB/s wr, 24 op/s 2024-06-02T23:27:14.030 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:13 smithi052 ceph-mon[32340]: pgmap v891: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 318 KiB/s wr, 24 op/s 2024-06-02T23:27:14.060 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:27:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:13 smithi174 ceph-mon[35994]: pgmap v891: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 318 KiB/s wr, 24 op/s 2024-06-02T23:27:14.309 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:14.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.46 MB/sec execute 22 sec latency 18116.958 ms 2024-06-02T23:27:15.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.36 MB/sec execute 23 sec latency 19117.005 ms 2024-06-02T23:27:15.814 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:15 smithi052 ceph-mon[32340]: pgmap v892: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 242 KiB/s wr, 22 op/s 2024-06-02T23:27:15.945 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:15 smithi174 ceph-mon[35994]: pgmap v892: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 242 KiB/s wr, 22 op/s 2024-06-02T23:27:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:15 smithi123 ceph-mon[35977]: pgmap v892: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 242 KiB/s wr, 22 op/s 2024-06-02T23:27:16.454 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.26 MB/sec execute 24 sec latency 20117.060 ms 2024-06-02T23:27:17.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 -- 172.21.15.52:0/3098135615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 msgr2=0x7f596c10e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 --2- 172.21.15.52:0/3098135615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c10e240 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f596400b0a0 tx=0x7f596400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 -- 172.21.15.52:0/3098135615 shutdown_connections 2024-06-02T23:27:17.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 --2- 172.21.15.52:0/3098135615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c10e240 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 --2- 172.21.15.52:0/3098135615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f596c107400 0x7f596c1077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 --2- 172.21.15.52:0/3098135615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f596c101390 0x7f596c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 -- 172.21.15.52:0/3098135615 >> 172.21.15.52:0/3098135615 conn(0x7f596c078340 msgr2=0x7f596c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 -- 172.21.15.52:0/3098135615 shutdown_connections 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.623+0000 7f597296f700 1 -- 172.21.15.52:0/3098135615 wait complete. 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 Processor -- start 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 -- start start 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f596c101390 0x7f596c199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f596c107400 0x7f596c1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f596c111520 con 0x7f596c101d50 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1113a0 con 0x7f596c101390 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f597296f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1116a0 con 0x7f596c107400 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f596b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f596b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60680/0 (socket says 172.21.15.52:60680) 2024-06-02T23:27:17.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f596b7fe700 1 -- 172.21.15.52:0/663929900 learned_addr learned my addr 172.21.15.52:0/663929900 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f596bfff700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f596c101390 0x7f596c199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.624+0000 7f5970f0c700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f596c107400 0x7f596c1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f596b7fe700 1 -- 172.21.15.52:0/663929900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f596c107400 msgr2=0x7f596c1a48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f596b7fe700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f596c107400 0x7f596c1a48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f596b7fe700 1 -- 172.21.15.52:0/663929900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f596c101390 msgr2=0x7f596c199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f596b7fe700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f596c101390 0x7f596c199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f596b7fe700 1 -- 172.21.15.52:0/663929900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1a4e00 con 0x7f596c101d50 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f596b7fe700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c199d80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f596000d950 tx=0x7f596000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f59697fa700 1 -- 172.21.15.52:0/663929900 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5960016070 con 0x7f596c101d50 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f59697fa700 1 -- 172.21.15.52:0/663929900 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f596000fd40 con 0x7f596c101d50 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1a5090 con 0x7f596c101d50 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f59697fa700 1 -- 172.21.15.52:0/663929900 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f596000e3b0 con 0x7f596c101d50 2024-06-02T23:27:17.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.625+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1a55a0 con 0x7f596c101d50 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.629+0000 7f59697fa700 1 -- 172.21.15.52:0/663929900 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5960019070 con 0x7f596c101d50 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.629+0000 7f597d1f2b80 1 Processor -- start 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.629+0000 7f59697fa700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59540a2c20 0x7f59540a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.629+0000 7f597d1f2b80 1 -- start start 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.629+0000 7f5970f0c700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59540a2c20 0x7f59540a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.629+0000 7f59697fa700 1 -- 172.21.15.52:0/663929900 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f59600cea90 con 0x7f596c101d50 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83243c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83244170 0x55ec83248010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83248550 0x55ec83248990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ec83131f60 con 0x55ec83243850 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec83190850 con 0x55ec83244170 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec830f8460 con 0x55ec83248550 2024-06-02T23:27:17.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83243c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83243c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60692/0 (socket says 172.21.15.52:60692) 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 -- 172.21.15.52:0/3719175498 learned_addr learned my addr 172.21.15.52:0/3719175498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f5970f0c700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59540a2c20 0x7f59540a50e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5964009fd0 tx=0x7f596400b7d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f5970f0c700 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83248550 0x55ec83248990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596b7fe700 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83244170 0x55ec83248010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 -- 172.21.15.52:0/3719175498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83248550 msgr2=0x55ec83248990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83248550 0x55ec83248990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 -- 172.21.15.52:0/3719175498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83244170 msgr2=0x55ec83248010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83244170 0x55ec83248010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.630+0000 7f596bfff700 1 -- 172.21.15.52:0/3719175498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec830df8d0 con 0x55ec83243850 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.631+0000 7f596bfff700 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83243c30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f595c00ad00 tx=0x7f595c00f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c34be9c64e1c92 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.631+0000 7f59527fc700 1 -- 172.21.15.52:0/3719175498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c00ae50 con 0x55ec83243850 2024-06-02T23:27:17.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.631+0000 7f59527fc700 1 -- 172.21.15.52:0/3719175498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595c0044d0 con 0x55ec83243850 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.631+0000 7f59527fc700 1 -- 172.21.15.52:0/3719175498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c0186a0 con 0x55ec83243850 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3719175498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 msgr2=0x55ec83243c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83243c30 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f595c00ad00 tx=0x7f595c00f6a0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3719175498 shutdown_connections 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83248550 0x55ec83248990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83243c30 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3719175498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83244170 0x55ec83248010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3719175498 >> 172.21.15.52:0/3719175498 conn(0x55ec8316c080 msgr2=0x55ec8323a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3719175498 shutdown_connections 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3719175498 wait complete. 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 -- start start 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.632+0000 7f597d1f2b80 1 Processor -- start 2024-06-02T23:27:17.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec8324f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ec83130fc0 con 0x55ec83243850 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ec8318c810 con 0x55ec83248550 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ec83250680 con 0x55ec83244170 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f597d1f2b80 10 client.0 _ll_register_callbacks cb 0x55ec831265b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec8324f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec8324f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60704/0 (socket says 172.21.15.52:60704) 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 -- 172.21.15.52:0/3564928266 learned_addr learned my addr 172.21.15.52:0/3564928266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f5970f0c700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596b7fe700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 -- 172.21.15.52:0/3564928266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 msgr2=0x55ec8324ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 -- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 msgr2=0x55ec8324d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 -- 172.21.15.52:0/3564928266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec83156750 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f596bfff700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec8324f980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f595c00a630 tx=0x7f595c003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9a58a05ffb84ba server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f594b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.633+0000 7f594b7fe700 1 -- 172.21.15.52:0/3564928266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c026020 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f594b7fe700 1 -- 172.21.15.52:0/3564928266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f595c005040 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c0044d0 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f594b7fe700 1 -- 172.21.15.52:0/3564928266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f595c00ae50 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 10 client.19692 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec8325ad40 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f594b7fe700 1 -- 172.21.15.52:0/3564928266 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595c0044d0 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f594b7fe700 1 -- 172.21.15.52:0/3564928266 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f595c004c00 con 0x55ec83243850 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 10 client.19692 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:17.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 10 client.19692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 20 client.19692 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec8325d210 0x55ec8325f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 4 client.19692 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec8319dd50 con 0x55ec8325d210 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.634+0000 7f596b7fe700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec8325d210 0x55ec8325f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.635+0000 7f594b7fe700 10 client.19692 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.635+0000 7f596b7fe700 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec8325d210 0x55ec8325f600 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cae36d9806d5d68a server_cookie=c8642d81b041c9f9 in_seq=0 out_seq=0 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f594b7fe700 1 -- 172.21.15.52:0/3564928266 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ec8319dd50 con 0x55ec8325d210 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f594b7fe700 10 client.19692 handle_command_reply: tid=0 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 client.19692 shutdown 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 msgr2=0x55ec8324f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec8324f980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f595c00a630 tx=0x7f595c003fb0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 shutdown_connections 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec8325d210 0x55ec8325f600 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec8324f980 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f594b7fe700 0 client.19692 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/3564928266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 >> 172.21.15.52:0/3564928266 conn(0x55ec8316c080 msgr2=0x55ec83249a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 shutdown_connections 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 1 -- 172.21.15.52:0/3564928266 wait complete. 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.636+0000 7f597d1f2b80 20 client.19692 trim_cache size 0 max 16384 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 Processor -- start 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 -- start start 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83249940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1aa880 con 0x55ec83243850 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1ac4b0 con 0x55ec83248550 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1aaf50 con 0x55ec83244170 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83249940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83249940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60710/0 (socket says 172.21.15.52:60710) 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 -- 172.21.15.52:0/2074141178 learned_addr learned my addr 172.21.15.52:0/2074141178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f5970f0c700 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596b7fe700 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 -- 172.21.15.52:0/2074141178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 msgr2=0x55ec8324b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 -- 172.21.15.52:0/2074141178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 msgr2=0x55ec8324a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f596bfff700 1 -- 172.21.15.52:0/2074141178 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec832482c0 con 0x55ec83243850 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.652+0000 7f5970f0c700 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.653+0000 7f596bfff700 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83249940 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f595c02f040 tx=0x7f595c004020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37c50cb7f47891b1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.653+0000 7f5951ffb700 1 -- 172.21.15.52:0/2074141178 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c03a070 con 0x55ec83243850 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.653+0000 7f5951ffb700 1 -- 172.21.15.52:0/2074141178 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595c00f9f0 con 0x55ec83243850 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.653+0000 7f5951ffb700 1 -- 172.21.15.52:0/2074141178 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c022b70 con 0x55ec83243850 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 -- 172.21.15.52:0/2074141178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 msgr2=0x55ec83249940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83249940 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f595c02f040 tx=0x7f595c004020 comp rx=0 tx=0).stop 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 -- 172.21.15.52:0/2074141178 shutdown_connections 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec8324a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83249940 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/2074141178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec8324b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 -- 172.21.15.52:0/2074141178 >> 172.21.15.52:0/2074141178 conn(0x55ec8316c080 msgr2=0x55ec83239690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 -- 172.21.15.52:0/2074141178 shutdown_connections 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 -- 172.21.15.52:0/2074141178 wait complete. 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.654+0000 7f597d1f2b80 1 -- start start 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 Processor -- start 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83245230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec83245770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec83254910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1abd20 con 0x55ec83243850 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1ab650 con 0x55ec83248550 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f596c1a8840 con 0x55ec83244170 2024-06-02T23:27:17.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f597d1f2b80 10 client.0 _ll_register_callbacks cb 0x55ec831265b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83245230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83245230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60712/0 (socket says 172.21.15.52:60712) 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 -- 172.21.15.52:0/1747750615 learned_addr learned my addr 172.21.15.52:0/1747750615 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f5970f0c700 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec83254910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 -- 172.21.15.52:0/1747750615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 msgr2=0x55ec83245770 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec83245770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 -- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 msgr2=0x55ec83254910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec83254910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.655+0000 7f596bfff700 1 -- 172.21.15.52:0/1747750615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec83245fd0 con 0x55ec83243850 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.656+0000 7f596bfff700 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83245230 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f595c003ff0 tx=0x7f595c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35391da99cdbdcb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.656+0000 7f594b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f594b7fe700 1 -- 172.21.15.52:0/1747750615 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c03a070 con 0x55ec83243850 2024-06-02T23:27:17.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f594b7fe700 1 -- 172.21.15.52:0/1747750615 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f595c0226f0 con 0x55ec83243850 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f595c00f9f0 con 0x55ec83243850 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f594b7fe700 1 -- 172.21.15.52:0/1747750615 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f595c0228f0 con 0x55ec83243850 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f597d1f2b80 10 client.19698 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ec83255ee0 con 0x55ec83243850 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f594b7fe700 1 -- 172.21.15.52:0/1747750615 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f595c00f9f0 con 0x55ec83243850 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f594b7fe700 1 -- 172.21.15.52:0/1747750615 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f595c034070 con 0x55ec83243850 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f597d1f2b80 10 client.19698 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.658+0000 7f597d1f2b80 10 client.19698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f597d1f2b80 20 client.19698 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec832c7cb0 0x55ec832ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f597d1f2b80 4 client.19698 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec831982d0 con 0x55ec832c7cb0 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f596b7fe700 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec832c7cb0 0x55ec832ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f594b7fe700 10 client.19698 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:17.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f596b7fe700 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec832c7cb0 0x55ec832ca0a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cf61b694f2c4ef3 server_cookie=fe1124f436e1a2ec in_seq=0 out_seq=0 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f594b7fe700 1 -- 172.21.15.52:0/1747750615 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec831982d0 con 0x55ec832c7cb0 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f594b7fe700 10 client.19698 handle_command_reply: tid=0 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.659+0000 7f597d1f2b80 1 client.19698 shutdown 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 msgr2=0x55ec83245230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83245230 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f595c003ff0 tx=0x7f595c0058d0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 shutdown_connections 2024-06-02T23:27:17.311 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ec832c7cb0 0x55ec832ca0a0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ec83248550 0x55ec83254910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ec83243850 0x55ec83245230 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 --2- 172.21.15.52:0/1747750615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ec83244170 0x55ec83245770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f594b7fe700 0 client.19698 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 >> 172.21.15.52:0/1747750615 conn(0x55ec8316c080 msgr2=0x55ec8323e250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 shutdown_connections 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 1 -- 172.21.15.52:0/1747750615 wait complete. 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.660+0000 7f597d1f2b80 20 client.19698 trim_cache size 0 max 16384 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59540a2c20 msgr2=0x7f59540a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59540a2c20 0x7f59540a50e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5964009fd0 tx=0x7f596400b7d0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 msgr2=0x7f596c199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c199d80 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f596000d950 tx=0x7f596000dc60 comp rx=0 tx=0).stop 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 shutdown_connections 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59540a2c20 0x7f59540a50e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f596c107400 0x7f596c1a48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f596c101390 0x7f596c199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 --2- 172.21.15.52:0/663929900 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f596c101d50 0x7f596c199d80 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 >> 172.21.15.52:0/663929900 conn(0x7f596c078340 msgr2=0x7f596c0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 shutdown_connections 2024-06-02T23:27:17.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:16.661+0000 7f597296f700 1 -- 172.21.15.52:0/663929900 wait complete. 2024-06-02T23:27:17.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.17 MB/sec execute 25 sec latency 21117.434 ms 2024-06-02T23:27:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:17 smithi123 ceph-mon[35977]: pgmap v893: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 181 KiB/s wr, 16 op/s 2024-06-02T23:27:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:17 smithi174 ceph-mon[35994]: pgmap v893: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 181 KiB/s wr, 16 op/s 2024-06-02T23:27:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:17 smithi052 ceph-mon[32340]: pgmap v893: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 181 KiB/s wr, 16 op/s 2024-06-02T23:27:18.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.09 MB/sec execute 26 sec latency 22117.532 ms 2024-06-02T23:27:19.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 2.01 MB/sec execute 27 sec latency 23117.645 ms 2024-06-02T23:27:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:19 smithi123 ceph-mon[35977]: pgmap v894: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-06-02T23:27:20.065 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:27:20.065 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:27:20.095 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:19 smithi052 ceph-mon[32340]: pgmap v894: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-06-02T23:27:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:19 smithi174 ceph-mon[35994]: pgmap v894: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-06-02T23:27:20.332 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:20.356 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:27:20.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.94 MB/sec execute 28 sec latency 24117.696 ms 2024-06-02T23:27:21.082 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:21.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.87 MB/sec execute 29 sec latency 25117.746 ms 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 -- 172.21.15.52:0/42010208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81024b0 msgr2=0x7f59c8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 --2- 172.21.15.52:0/42010208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81024b0 0x7f59c8108d40 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f59c0009ca0 tx=0x7f59c002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 -- 172.21.15.52:0/42010208 shutdown_connections 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 --2- 172.21.15.52:0/42010208 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c81024b0 0x7f59c8108d40 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 --2- 172.21.15.52:0/42010208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59c8101af0 0x7f59c8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 --2- 172.21.15.52:0/42010208 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59c8100900 0x7f59c8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 -- 172.21.15.52:0/42010208 >> 172.21.15.52:0/42010208 conn(0x7f59c80fbe70 msgr2=0x7f59c80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 -- 172.21.15.52:0/42010208 shutdown_connections 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 -- 172.21.15.52:0/42010208 wait complete. 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 Processor -- start 2024-06-02T23:27:21.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.579+0000 7f59cdd77640 1 -- start start 2024-06-02T23:27:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59cdd77640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59c8100900 0x7f59c8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59cdd77640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 0x7f59c8198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59cdd77640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59c81024b0 0x7f59c819f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59cdd77640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810c410 con 0x7f59c8101af0 2024-06-02T23:27:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59cdd77640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810c290 con 0x7f59c81024b0 2024-06-02T23:27:21.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59cdd77640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59c810c590 con 0x7f59c8100900 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 0x7f59c8198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 0x7f59c8198060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60732/0 (socket says 172.21.15.52:60732) 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 -- 172.21.15.52:0/4189466726 learned_addr learned my addr 172.21.15.52:0/4189466726 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59c77fe640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59c8100900 0x7f59c8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59c7fff640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59c81024b0 0x7f59c819f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 -- 172.21.15.52:0/4189466726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59c8100900 msgr2=0x7f59c8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59c8100900 0x7f59c8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 -- 172.21.15.52:0/4189466726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59c81024b0 msgr2=0x7f59c819f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59c81024b0 0x7f59c819f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 -- 172.21.15.52:0/4189466726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59c819f740 con 0x7f59c8101af0 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.580+0000 7f59beffd640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 0x7f59c8198060 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f59b000d950 tx=0x7f59b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.581+0000 7f59c57fa640 1 -- 172.21.15.52:0/4189466726 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0017030 con 0x7f59c8101af0 2024-06-02T23:27:21.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.581+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59c819fa30 con 0x7f59c8101af0 2024-06-02T23:27:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.581+0000 7f59c57fa640 1 -- 172.21.15.52:0/4189466726 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b0004020 con 0x7f59c8101af0 2024-06-02T23:27:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.581+0000 7f59c57fa640 1 -- 172.21.15.52:0/4189466726 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00047a0 con 0x7f59c8101af0 2024-06-02T23:27:21.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.581+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59c819ffd0 con 0x7f59c8101af0 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.583+0000 7f59c57fa640 1 -- 172.21.15.52:0/4189466726 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f59b00041d0 con 0x7f59c8101af0 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.584+0000 7f59c57fa640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59b40a2c30 0x7f59b40a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.584+0000 7f59c57fa640 1 -- 172.21.15.52:0/4189466726 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f59b00ceb50 con 0x7f59c8101af0 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.584+0000 7f59c77fe640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59b40a2c30 0x7f59b40a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.584+0000 7f59c77fe640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59b40a2c30 0x7f59b40a50f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005db0 tx=0x7f59b8005d40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.584+0000 7f59d0b63740 1 Processor -- start 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 -- start start 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d412b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d4130500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4133000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f0d408e4f0 con 0x55f0d412ba00 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f0d402c870 con 0x55f0d412b010 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59d0b63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f0d408a4b0 con 0x55f0d4130b70 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d4130500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d4130500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60742/0 (socket says 172.21.15.52:60742) 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 -- 172.21.15.52:0/2583530371 learned_addr learned my addr 172.21.15.52:0/2583530371 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59c7fff640 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4133000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59c77fe640 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d412b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 -- 172.21.15.52:0/2583530371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 msgr2=0x55f0d4133000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4133000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 -- 172.21.15.52:0/2583530371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 msgr2=0x55f0d412b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d412b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 -- 172.21.15.52:0/2583530371 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0d40489b0 con 0x55f0d412ba00 2024-06-02T23:27:21.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59c7fff640 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4133000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.585+0000 7f59beffd640 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d4130500 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f59b0097840 tx=0x7f59b009ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdde161027b8e24 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59bdffb640 1 -- 172.21.15.52:0/2583530371 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b009f790 con 0x55f0d412ba00 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59bdffb640 1 -- 172.21.15.52:0/2583530371 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b00a4050 con 0x55f0d412ba00 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59bdffb640 1 -- 172.21.15.52:0/2583530371 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b009a830 con 0x55f0d412ba00 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 -- 172.21.15.52:0/2583530371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 msgr2=0x55f0d4130500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d4130500 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f59b0097840 tx=0x7f59b009ae00 comp rx=0 tx=0).stop 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 -- 172.21.15.52:0/2583530371 shutdown_connections 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4133000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d4130500 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 --2- 172.21.15.52:0/2583530371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d412b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 -- 172.21.15.52:0/2583530371 >> 172.21.15.52:0/2583530371 conn(0x55f0d4126680 msgr2=0x55f0d4126a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 -- 172.21.15.52:0/2583530371 shutdown_connections 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.586+0000 7f59d0b63740 1 -- 172.21.15.52:0/2583530371 wait complete. 2024-06-02T23:27:21.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.587+0000 7f59d0b63740 1 -- start start 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.587+0000 7f59d0b63740 1 Processor -- start 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.587+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.587+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d4135710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 0x55f0d413c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59d0b63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f0d4133a60 con 0x55f0d4130b70 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59d0b63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f0d4017ae0 con 0x55f0d412ba00 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59d0b63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f0d4133be0 con 0x55f0d412b010 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59d0b63740 10 client.0 _ll_register_callbacks cb 0x55f0d40509b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 0x55f0d413c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 0x55f0d413c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60758/0 (socket says 172.21.15.52:60758) 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 -- 172.21.15.52:0/3868980031 learned_addr learned my addr 172.21.15.52:0/3868980031 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59beffd640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d4135710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c77fe640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 -- 172.21.15.52:0/3868980031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d412b010 msgr2=0x55f0d41351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 -- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 msgr2=0x55f0d4135710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d4135710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 -- 172.21.15.52:0/3868980031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0d3da2210 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59c7fff640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 0x55f0d413c750 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f59c0035000 tx=0x7f59c0031b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b04fb201bf3ff54 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59a6ffd640 1 -- 172.21.15.52:0/3868980031 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0003d80 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59a6ffd640 1 -- 172.21.15.52:0/3868980031 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f59c0004040 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.588+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b00a4050 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59a6ffd640 1 -- 172.21.15.52:0/3868980031 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59c0004240 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 10 client.19710 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f0d3e6e620 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59a6ffd640 1 -- 172.21.15.52:0/3868980031 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b00a4050 con 0x55f0d4130b70 2024-06-02T23:27:21.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59a6ffd640 1 -- 172.21.15.52:0/3868980031 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f59c0030cb0 con 0x55f0d4130b70 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 10 client.19710 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 10 client.19710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 20 client.19710 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d4141190 0x55f0d41435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 4 client.19710 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0d40a1f10 con 0x55f0d4141190 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.589+0000 7f59c77fe640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d4141190 0x55f0d41435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.590+0000 7f59c77fe640 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d4141190 0x55f0d41435f0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08badc0e24d739b server_cookie=1cfb4decf986753 in_seq=0 out_seq=0 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.590+0000 7f59a6ffd640 10 client.19710 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59a6ffd640 1 -- 172.21.15.52:0/3868980031 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f0d40a1f10 con 0x55f0d4141190 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59a6ffd640 10 client.19710 handle_command_reply: tid=0 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 client.19710 shutdown 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 msgr2=0x55f0d413c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 0x55f0d413c750 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f59c0035000 tx=0x7f59c0031b40 comp rx=0 tx=0).stop 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 shutdown_connections 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d4141190 0x55f0d41435f0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d4130b70 0x55f0d413c750 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d4135710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3868980031 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 >> 172.21.15.52:0/3868980031 conn(0x55f0d4126680 msgr2=0x55f0d41257e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59a6ffd640 0 client.19710 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 shutdown_connections 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 1 -- 172.21.15.52:0/3868980031 wait complete. 2024-06-02T23:27:21.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.591+0000 7f59d0b63740 20 client.19710 trim_cache size 0 max 16384 2024-06-02T23:27:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 Processor -- start 2024-06-02T23:27:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 -- start start 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d413c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4135710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a8c70 con 0x55f0d412ba00 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81aa8a0 con 0x55f0d412b010 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59d0b63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a9340 con 0x55f0d4130b70 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d413c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d413c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60768/0 (socket says 172.21.15.52:60768) 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 -- 172.21.15.52:0/3914091279 learned_addr learned my addr 172.21.15.52:0/3914091279 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59c77fe640 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59c7fff640 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4135710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 -- 172.21.15.52:0/3914091279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 msgr2=0x55f0d4135710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.609 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4135710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 -- 172.21.15.52:0/3914091279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 msgr2=0x55f0d41351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59beffd640 1 -- 172.21.15.52:0/3914091279 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0d412b7a0 con 0x55f0d412ba00 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.607+0000 7f59c7fff640 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4135710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.608+0000 7f59beffd640 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d413c710 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f59b00cc9f0 tx=0x7f59b009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b871a4117a9119 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.608+0000 7f59bd7fa640 1 -- 172.21.15.52:0/3914091279 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b0014070 con 0x55f0d412ba00 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.608+0000 7f59bd7fa640 1 -- 172.21.15.52:0/3914091279 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b0099480 con 0x55f0d412ba00 2024-06-02T23:27:21.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.608+0000 7f59bd7fa640 1 -- 172.21.15.52:0/3914091279 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b00a3e20 con 0x55f0d412ba00 2024-06-02T23:27:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.608+0000 7f59d0b63740 1 -- 172.21.15.52:0/3914091279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 msgr2=0x55f0d413c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d413c710 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f59b00cc9f0 tx=0x7f59b009ae70 comp rx=0 tx=0).stop 2024-06-02T23:27:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- 172.21.15.52:0/3914091279 shutdown_connections 2024-06-02T23:27:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d4135710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412ba00 0x55f0d413c710 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- 172.21.15.52:0/3914091279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412b010 0x55f0d41351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- 172.21.15.52:0/3914091279 >> 172.21.15.52:0/3914091279 conn(0x55f0d4126680 msgr2=0x55f0d412cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- 172.21.15.52:0/3914091279 shutdown_connections 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- 172.21.15.52:0/3914091279 wait complete. 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- start start 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 Processor -- start 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 0x55f0d41392b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d41397f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d41a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81aa110 con 0x55f0d412b010 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a9a40 con 0x55f0d412ba00 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.609+0000 7f59d0b63740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59c81a6c50 con 0x55f0d4130b70 2024-06-02T23:27:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59d0b63740 10 client.0 _ll_register_callbacks cb 0x7f59c0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 0x55f0d41392b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 0x55f0d41392b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60774/0 (socket says 172.21.15.52:60774) 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 -- 172.21.15.52:0/4037010567 learned_addr learned my addr 172.21.15.52:0/4037010567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59beffd640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d41397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c7fff640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d41a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 -- 172.21.15.52:0/4037010567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 msgr2=0x55f0d41a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d41a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 -- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 msgr2=0x55f0d41397f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d41397f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 -- 172.21.15.52:0/4037010567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0d413c7d0 con 0x55f0d412b010 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59beffd640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d41397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c7fff640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d41a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59c77fe640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 0x55f0d41392b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x55f0d4143e30 tx=0x7f59b8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38f049a76b26be9f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59a6ffd640 1 -- 172.21.15.52:0/4037010567 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8004440 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59a6ffd640 1 -- 172.21.15.52:0/4037010567 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f59b80035f0 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.610+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b0099480 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59a6ffd640 1 -- 172.21.15.52:0/4037010567 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f59b8038050 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 10 client.19716 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f0d41ab320 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59a6ffd640 1 -- 172.21.15.52:0/4037010567 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f59b0099480 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59a6ffd640 1 -- 172.21.15.52:0/4037010567 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f59b80286d0 con 0x55f0d412b010 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 10 client.19716 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 10 client.19716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 20 client.19716 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d41abdc0 0x55f0d41ae1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 4 client.19716 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f0d409bc20 con 0x55f0d41abdc0 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.611+0000 7f59beffd640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d41abdc0 0x55f0d41ae1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:21.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.612+0000 7f59beffd640 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d41abdc0 0x55f0d41ae1e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=137843c22b7031b3 server_cookie=835d8610d4a5848f in_seq=0 out_seq=0 2024-06-02T23:27:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.612+0000 7f59a6ffd640 10 client.19716 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59a6ffd640 1 -- 172.21.15.52:0/4037010567 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f0d409bc20 con 0x55f0d41abdc0 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59a6ffd640 10 client.19716 handle_command_reply: tid=0 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 client.19716 shutdown 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 msgr2=0x55f0d41392b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 0x55f0d41392b0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x55f0d4143e30 tx=0x7f59b8004730 comp rx=0 tx=0).stop 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 shutdown_connections 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f0d41abdc0 0x55f0d41ae1e0 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0d4130b70 0x55f0d41a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.634 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0d412ba00 0x55f0d41397f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59a6ffd640 0 client.19716 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 --2- 172.21.15.52:0/4037010567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0d412b010 0x55f0d41392b0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 >> 172.21.15.52:0/4037010567 conn(0x55f0d4126680 msgr2=0x55f0d412cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 shutdown_connections 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 1 -- 172.21.15.52:0/4037010567 wait complete. 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.633+0000 7f59d0b63740 20 client.19716 trim_cache size 0 max 16384 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:27:21.635 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59b40a2c30 msgr2=0x7f59b40a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59b40a2c30 0x7f59b40a50f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f59b8005db0 tx=0x7f59b8005d40 comp rx=0 tx=0).stop 2024-06-02T23:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 msgr2=0x7f59c8198060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 0x7f59c8198060 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f59b000d950 tx=0x7f59b000de10 comp rx=0 tx=0).stop 2024-06-02T23:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 shutdown_connections 2024-06-02T23:27:21.636 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f59b40a2c30 0x7f59b40a50f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59c81024b0 0x7f59c819f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f59c8101af0 0x7f59c8198060 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 --2- 172.21.15.52:0/4189466726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f59c8100900 0x7f59c8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 >> 172.21.15.52:0/4189466726 conn(0x7f59c80fbe70 msgr2=0x7f59c80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 shutdown_connections 2024-06-02T23:27:21.637 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:21.634+0000 7f59cdd77640 1 -- 172.21.15.52:0/4189466726 wait complete. 2024-06-02T23:27:21.859 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:21 smithi052 ceph-mon[32340]: pgmap v895: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-02T23:27:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:21 smithi123 ceph-mon[35977]: pgmap v895: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-02T23:27:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:21 smithi174 ceph-mon[35994]: pgmap v895: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-06-02T23:27:22.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.314+0000 7f3f84c73700 1 -- 172.21.15.52:0/4048897719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 msgr2=0x7f3f80108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.314+0000 7f3f84c73700 1 --2- 172.21.15.52:0/4048897719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80108900 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 -- 172.21.15.52:0/4048897719 shutdown_connections 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 --2- 172.21.15.52:0/4048897719 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 0x7f3f8010f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 --2- 172.21.15.52:0/4048897719 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80108900 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 --2- 172.21.15.52:0/4048897719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 0x7f3f80102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 -- 172.21.15.52:0/4048897719 >> 172.21.15.52:0/4048897719 conn(0x7f3f80079640 msgr2=0x7f3f80079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 -- 172.21.15.52:0/4048897719 shutdown_connections 2024-06-02T23:27:22.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.315+0000 7f3f84c73700 1 -- 172.21.15.52:0/4048897719 wait complete. 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 Processor -- start 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 -- start start 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 0x7f3f8019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 0x7f3f8019ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80112790 con 0x7f3f80108520 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80112610 con 0x7f3f80102ed0 2024-06-02T23:27:22.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f84c73700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f80112910 con 0x7f3f80102510 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60790/0 (socket says 172.21.15.52:60790) 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/2599513606 learned_addr learned my addr 172.21.15.52:0/2599513606 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 0x7f3f8019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.316+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 0x7f3f8019ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/2599513606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 msgr2=0x7f3f8019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 0x7f3f8019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/2599513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 msgr2=0x7f3f8019ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 0x7f3f8019ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/2599513606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f80199250 con 0x7f3f80108520 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 0x7f3f8019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 0x7f3f8019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80198af0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b9e0 tx=0x7f3f7800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:22.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2599513606 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78019070 con 0x7f3f80108520 2024-06-02T23:27:22.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f80199540 con 0x7f3f80108520 2024-06-02T23:27:22.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2599513606 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f7800ce00 con 0x7f3f80108520 2024-06-02T23:27:22.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2599513606 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f78007370 con 0x7f3f80108520 2024-06-02T23:27:22.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.317+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f801a5800 con 0x7f3f80108520 2024-06-02T23:27:22.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.319+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2599513606 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3f7801c070 con 0x7f3f80108520 2024-06-02T23:27:22.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.320+0000 7f3f6f7fe700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f680a2c20 0x7f3f680a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.321+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f680a2c20 0x7f3f680a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.321+0000 7f3f6f7fe700 1 -- 172.21.15.52:0/2599513606 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3f780ce420 con 0x7f3f80108520 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.321+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f680a2c20 0x7f3f680a50e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009750 tx=0x7f3f70005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.321+0000 7f3f8f4f6b80 1 Processor -- start 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.321+0000 7f3f8f4f6b80 1 -- start start 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.321+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176050 0x564373176430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x56437317a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5643730c3890 con 0x56437317acb0 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56437306b7a0 con 0x564373176970 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5643730bf850 con 0x564373176050 2024-06-02T23:27:22.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60800/0 (socket says 172.21.15.52:60800) 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/3674726403 learned_addr learned my addr 172.21.15.52:0/3674726403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x56437317a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176050 0x564373176430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/3674726403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176050 msgr2=0x564373176430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176050 0x564373176430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/3674726403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 msgr2=0x56437317a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x56437317a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/3674726403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564373061fd0 con 0x56437317acb0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317b110 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3f780a3390 tx=0x7f3f78004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ae5d0988037ca2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f6cff9700 1 -- 172.21.15.52:0/3674726403 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780b0030 con 0x56437317acb0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f6cff9700 1 -- 172.21.15.52:0/3674726403 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f7809ac50 con 0x56437317acb0 2024-06-02T23:27:22.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.322+0000 7f3f6cff9700 1 -- 172.21.15.52:0/3674726403 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f7809aeb0 con 0x56437317acb0 2024-06-02T23:27:22.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3674726403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 msgr2=0x56437317b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317b110 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f3f780a3390 tx=0x7f3f78004be0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3674726403 shutdown_connections 2024-06-02T23:27:22.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317b110 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176050 0x564373176430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3674726403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x56437317a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3674726403 >> 172.21.15.52:0/3674726403 conn(0x56437309cd40 msgr2=0x56437316c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3674726403 shutdown_connections 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.323+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3674726403 wait complete. 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 -- start start 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 Processor -- start 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x564373182250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176970 0x56437317f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564373182e50 con 0x56437317acb0 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564373063c70 con 0x564373176050 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564373182fd0 con 0x564373176970 2024-06-02T23:27:22.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f8f4f6b80 10 client.0 _ll_register_callbacks cb 0x56437305d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f7ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60802/0 (socket says 172.21.15.52:60802) 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/871345736 learned_addr learned my addr 172.21.15.52:0/871345736 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176970 0x56437317f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.324+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x564373182250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/871345736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176970 msgr2=0x56437317f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176970 0x56437317f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 msgr2=0x564373182250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x564373182250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7ed9d700 1 -- 172.21.15.52:0/871345736 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564373012950 con 0x56437317acb0 2024-06-02T23:27:22.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x564373182250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317fa90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3f78097620 tx=0x7f3f7809f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93abb32c8c53f361 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f65ffb700 1 -- 172.21.15.52:0/871345736 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f780b5e60 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f7809ac50 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f65ffb700 1 -- 172.21.15.52:0/871345736 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3f78098f50 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f65ffb700 1 -- 172.21.15.52:0/871345736 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3f780b0040 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f8f4f6b80 10 client.19728 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56437318d7a0 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f65ffb700 1 -- 172.21.15.52:0/871345736 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f7809ac50 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.325+0000 7f3f65ffb700 1 -- 172.21.15.52:0/871345736 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f3f780969a0 con 0x56437317acb0 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f8f4f6b80 10 client.19728 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f8f4f6b80 10 client.19728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:22.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f8f4f6b80 20 client.19728 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56437318fc70 0x564373192060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f8f4f6b80 4 client.19728 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643730d0d90 con 0x56437318fc70 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56437318fc70 0x564373192060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56437318fc70 0x564373192060 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cca16637be024ea server_cookie=2beabc9c505f4b16 in_seq=0 out_seq=0 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.326+0000 7f3f65ffb700 10 client.19728 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f65ffb700 1 -- 172.21.15.52:0/871345736 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5643730d0d90 con 0x56437318fc70 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f65ffb700 10 client.19728 handle_command_reply: tid=0 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 client.19728 shutdown 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 msgr2=0x56437317fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317fa90 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f3f78097620 tx=0x7f3f7809f000 comp rx=0 tx=0).stop 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 shutdown_connections 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56437318fc70 0x564373192060 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56437317acb0 0x56437317fa90 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x564373182250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/871345736 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564373176970 0x56437317f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 >> 172.21.15.52:0/871345736 conn(0x56437309cd40 msgr2=0x564373170a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f65ffb700 0 client.19728 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 shutdown_connections 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.327+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/871345736 wait complete. 2024-06-02T23:27:22.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.328+0000 7f3f8f4f6b80 20 client.19728 trim_cache size 0 max 16384 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.348+0000 7f3f8f4f6b80 1 Processor -- start 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 -- start start 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x56437317fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 0x56437316ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x564373197630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801aa9b0 con 0x564373176970 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801ac5e0 con 0x564373176050 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801ab080 con 0x56437317acb0 2024-06-02T23:27:22.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 0x56437316ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 0x56437316ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60808/0 (socket says 172.21.15.52:60808) 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 -- 172.21.15.52:0/1655020939 learned_addr learned my addr 172.21.15.52:0/1655020939 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x564373197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x56437317fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 -- 172.21.15.52:0/1655020939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 msgr2=0x564373197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x564373197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 -- 172.21.15.52:0/1655020939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 msgr2=0x56437317fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x56437317fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 -- 172.21.15.52:0/1655020939 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564373170ef0 con 0x564373176970 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x564373197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.349+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 0x56437316ca70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400d8f0 tx=0x7f3f7400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f5506ae9100e3f9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f67fff700 1 -- 172.21.15.52:0/1655020939 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74019030 con 0x564373176970 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f67fff700 1 -- 172.21.15.52:0/1655020939 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f74012ab0 con 0x564373176970 2024-06-02T23:27:22.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f67fff700 1 -- 172.21.15.52:0/1655020939 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f74015a40 con 0x564373176970 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/1655020939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 msgr2=0x56437316ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 0x56437316ca70 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f3f7400d8f0 tx=0x7f3f7400dc00 comp rx=0 tx=0).stop 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/1655020939 shutdown_connections 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x564373197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176050 0x56437317fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/1655020939 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176970 0x56437316ca70 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/1655020939 >> 172.21.15.52:0/1655020939 conn(0x56437309cd40 msgr2=0x56437317f510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/1655020939 shutdown_connections 2024-06-02T23:27:22.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.350+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/1655020939 wait complete. 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 -- start start 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 Processor -- start 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 0x564373184730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x564373184c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x5643731f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801abe50 con 0x564373176050 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801ab780 con 0x564373176970 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f801a8950 con 0x56437317acb0 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f8f4f6b80 10 client.0 _ll_register_callbacks cb 0x56437305d9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f7e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 0x564373184730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f7e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 0x564373184730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60814/0 (socket says 172.21.15.52:60814) 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f7e59c700 1 -- 172.21.15.52:0/3304581502 learned_addr learned my addr 172.21.15.52:0/3304581502 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:22.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f7ed9d700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x5643731f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.351+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x564373184c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f7e59c700 1 -- 172.21.15.52:0/3304581502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 msgr2=0x5643731f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x5643731f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f7e59c700 1 -- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 msgr2=0x564373184c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x564373184c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f7e59c700 1 -- 172.21.15.52:0/3304581502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643731f9b40 con 0x564373176050 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f7e59c700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 0x564373184730 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69d89c3470c5a0e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f65ffb700 1 -- 172.21.15.52:0/3304581502 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f700052c0 con 0x564373176050 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f65ffb700 1 -- 172.21.15.52:0/3304581502 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3f70005460 con 0x564373176050 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f74012ab0 con 0x564373176050 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f65ffb700 1 -- 172.21.15.52:0/3304581502 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3f70005660 con 0x564373176050 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f8f4f6b80 10 client.19734 fetch_fsmap learned FSMap version 39 2024-06-02T23:27:22.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5643731fae80 con 0x564373176050 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f65ffb700 1 -- 172.21.15.52:0/3304581502 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3f74012ab0 con 0x564373176050 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f65ffb700 1 -- 172.21.15.52:0/3304581502 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 39) ==== 8374+0+0 (secure 0 0 0) 0x7f3f7003f480 con 0x564373176050 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f8f4f6b80 10 client.19734 fetch_fsmap finished waiting for FSMap version 39 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.352+0000 7f3f8f4f6b80 10 client.19734 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f8f4f6b80 20 client.19734 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643731fb600 0x5643731fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f8f4f6b80 4 client.19734 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5643730cb310 con 0x5643731fb600 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643731fb600 0x5643731fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f7dd9b700 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643731fb600 0x5643731fda20 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a592a825eab238 server_cookie=767945dbcdcf875d in_seq=0 out_seq=0 2024-06-02T23:27:22.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.353+0000 7f3f65ffb700 10 client.19734 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:22.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.354+0000 7f3f65ffb700 1 -- 172.21.15.52:0/3304581502 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5643730cb310 con 0x5643731fb600 2024-06-02T23:27:22.358 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.354+0000 7f3f65ffb700 10 client.19734 handle_command_reply: tid=0 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.354+0000 7f3f8f4f6b80 1 client.19734 shutdown 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.354+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 msgr2=0x564373184730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.354+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 0x564373184730 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009990 tx=0x7f3f70009ca0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 shutdown_connections 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5643731fb600 0x5643731fda20 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56437317acb0 0x5643731f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564373176050 0x564373184730 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 --2- 172.21.15.52:0/3304581502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564373176970 0x564373184c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 >> 172.21.15.52:0/3304581502 conn(0x56437309cd40 msgr2=0x564373165e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f65ffb700 0 client.19734 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 shutdown_connections 2024-06-02T23:27:22.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 1 -- 172.21.15.52:0/3304581502 wait complete. 2024-06-02T23:27:22.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f8f4f6b80 20 client.19734 trim_cache size 0 max 16384 2024-06-02T23:27:22.360 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:27:22.360 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:27:22.360 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "80c1e53f-4dd0-4b01-8072-aae079a44855", 2024-06-02T23:27:22.360 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:27:22.360 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f680a2c20 msgr2=0x7f3f680a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f84c73700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f680a2c20 0x7f3f680a50e0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f3f70009750 tx=0x7f3f70005fd0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 msgr2=0x7f3f80198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f84c73700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80198af0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f3f7800b9e0 tx=0x7f3f7800bda0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 shutdown_connections 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.355+0000 7f3f84c73700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3f680a2c20 0x7f3f680a50e0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.356+0000 7f3f84c73700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3f80108520 0x7f3f80198af0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.356+0000 7f3f84c73700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f80102510 0x7f3f8019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.356+0000 7f3f84c73700 1 --2- 172.21.15.52:0/2599513606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3f80102ed0 0x7f3f8019ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.356+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 >> 172.21.15.52:0/2599513606 conn(0x7f3f80079640 msgr2=0x7f3f80101380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.356+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 shutdown_connections 2024-06-02T23:27:22.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:22.356+0000 7f3f84c73700 1 -- 172.21.15.52:0/2599513606 wait complete. 2024-06-02T23:27:22.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.81 MB/sec execute 30 sec latency 26117.779 ms 2024-06-02T23:27:22.670 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.386+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f3 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL/ auth{4=1} v3285 ap=3 f(v56 m2024-06-02T23:26:18.066187+0000 11=11+0) n(v5 rc2024-06-02T23:26:18.066187+0000 b2861056 12=11+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:754 nref=3)) caps={15216=pAsLsXsFs/-@1838} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb967a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003f3:[<0x1000000033f/COREL v50>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003f3:[<0x1000000033f/COREL v3285>,<0x10000000335/~dmtmp v696>,<0x10000000007/client3 v2575>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:18.033295+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:18.066187+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v56 m2024-06-02T23:26:18.066187+0000 11=11+0)","memory_value.rstat":"n(v5 rc2024-06-02T23:26:18.066187+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.670 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.389+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO/ auth{4=1} v3280 ap=3 f(v76 m2024-06-02T23:26:18.279544+0000 3=3+0) n(v6 rc2024-06-02T23:26:18.279544+0000 b780288 4=3+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:758 nref=3)) caps={15216=pAsLsXsFs/-@1903} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9645180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000046d:[<0x1000000033f/WORDPRO v58>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000046d:[<0x1000000033f/WORDPRO v3280>,<0x10000000335/~dmtmp v696>,<0x10000000007/client3 v2575>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:18.244085+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:18.279544+0000 b780288 4=3+1)","memory_value.dirstat":"f(v76 m2024-06-02T23:26:18.279544+0000 3=3+0)","memory_value.rstat":"n(v6 rc2024-06-02T23:26:18.279544+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.390+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004b8 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX/ auth{4=1} v3468 ap=3 f(v105 m2024-06-02T23:26:56.166660+0000 24=24+0) n(v6 rc2024-06-02T23:26:56.166660+0000 b5230239 25=24+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:759 nref=3)) caps={15216=pAsLsXsFs/-@2336} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9816680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004b8:[<0x1000000033f/PARADOX v108>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004b8:[<0x1000000033f/PARADOX v3468>,<0x10000000335/~dmtmp v696>,<0x10000000007/client3 v2575>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:56.128855+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:56.166660+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v105 m2024-06-02T23:26:56.166660+0000 24=24+0)","memory_value.rstat":"n(v6 rc2024-06-02T23:26:56.166660+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.394+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003eb [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL/ auth{3=1} v2670 ap=3 f(v56 m2024-06-02T23:26:50.033508+0000 11=11+0) n(v6 rc2024-06-02T23:26:50.033508+0000 b2861056 12=11+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:784 nref=3)) caps={15216=pAsLsXsFs/-@1752} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb964b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003eb:[<0x10000000336/COREL v50>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003eb:[<0x10000000336/COREL v2670>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:50.005757+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:50.033508+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v56 m2024-06-02T23:26:50.033508+0000 11=11+0)","memory_value.rstat":"n(v6 rc2024-06-02T23:26:50.033508+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.395+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038f [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL/ auth{3=1} v2666 ap=3 f(v82 m2024-06-02T23:26:49.839612+0000 2=2+0) n(v7 rc2024-06-02T23:26:49.839612+0000 b520192 3=2+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:785 nref=3)) caps={15216=pAsLsXsFs/-@1837} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000038f:[<0x10000000336/EXCEL v14>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000038f:[<0x10000000336/EXCEL v2666>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:49.813933+0000 3=3+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:49.839612+0000 b520192 3=2+1)","memory_value.dirstat":"f(v82 m2024-06-02T23:26:49.839612+0000 2=2+0)","memory_value.rstat":"n(v7 rc2024-06-02T23:26:49.839612+0000 b520192 3=2+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.396+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD/ auth{3=1} v2664 ap=3 f(v72 m2024-06-02T23:26:54.819773+0000 5=5+0) n(v5 rc2024-06-02T23:26:54.819773+0000 b814326 6=5+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:786 nref=3)) caps={15216=pAsLsXsFs/-@1778} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ca0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000036d:[<0x10000000336/WORD v8>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000036d:[<0x10000000336/WORD v2664>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:54.792111+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:54.819773+0000 b814326 6=5+1)","memory_value.dirstat":"f(v72 m2024-06-02T23:26:54.819773+0000 5=5+0)","memory_value.rstat":"n(v5 rc2024-06-02T23:26:54.819773+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.399+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000462 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORDPRO/ auth{3=1} v2672 ap=3 f(v76 m2024-06-02T23:26:50.196079+0000 3=3+0) n(v6 rc2024-06-02T23:26:50.196079+0000 b780288 4=3+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:788 nref=3)) caps={15216=pAsLsXsFs/-@1818} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ac2100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000462:[<0x10000000336/WORDPRO v58>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x10000000462:[<0x10000000336/WORDPRO v2672>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:50.168819+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:50.196079+0000 b780288 4=3+1)","memory_value.dirstat":"f(v76 m2024-06-02T23:26:50.196079+0000 3=3+0)","memory_value.rstat":"n(v6 rc2024-06-02T23:26:50.196079+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.402+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX/ auth{3=1} v2680 ap=3 f(v83 m2024-06-02T23:26:56.159351+0000 23=23+0) n(v6 rc2024-06-02T23:26:56.168926+0000 b5210655 24=23+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:789 nref=3)) caps={15216=pAsLsXsFs/-@2131} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb97ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004ac:[<0x10000000336/PARADOX v108>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004ac:[<0x10000000336/PARADOX v2680>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:56.099478+0000 24=24+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:56.168926+0000 b5210655 24=23+1)","memory_value.dirstat":"f(v83 m2024-06-02T23:26:56.159351+0000 23=23+0)","memory_value.rstat":"n(v6 rc2024-06-02T23:26:56.168926+0000 b5210655 24=23+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.404+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000568 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS/ auth{3=1} v2676 ap=3 f(v31 m2024-06-02T23:26:50.440806+0000 1=1+0) n(v6 rc2024-06-02T23:26:50.440806+0000 b260096 2=1+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:792 nref=3)) caps={15216=pAsLsXsFs/-@1535} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9850100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000568:[<0x10000000336/ACCESS v112>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x10000000568:[<0x10000000336/ACCESS v2676>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:50.410732+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:50.440806+0000 b260096 2=1+1)","memory_value.dirstat":"f(v31 m2024-06-02T23:26:50.440806+0000 1=1+0)","memory_value.rstat":"n(v6 rc2024-06-02T23:26:50.440806+0000 b260096 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.671 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:27:22.406+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT/ auth{3=1} v2668 ap=3 f(v84 m2024-06-02T23:26:53.026005+0000 7=7+0) n(v5 rc2024-06-02T23:26:53.026005+0000 b1462582 8=7+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:791 nref=3)) caps={15216=pAsLsXsFs/-@1772} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b64100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003ac:[<0x10000000336/PWRPNT v214>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003ac:[<0x10000000336/PWRPNT v2668>,<0x10000000331/~dmtmp v1211>,<0x10000000007/client1 v2591>,<0x10000000006/clients v1771>,<0x10000000005/tmp v1527>,<0x10000000003/client.0 v1490>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1466>,<0x10000000001/sv_0 v1468>,<0x10000000000/qa v1407>,<0x1/volumes v1412>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:26:52.955594+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:26:53.026005+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v84 m2024-06-02T23:26:53.026005+0000 7=7+0)","memory_value.rstat":"n(v5 rc2024-06-02T23:26:53.026005+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.898 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000462 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000568 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:27:22.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:22.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:22 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000462 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000568 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:22 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:27:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:27:23.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:27:23.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000462 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000568 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:27:23.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:22 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.75 MB/sec execute 31 sec latency 27117.833 ms 2024-06-02T23:27:23.614 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:27:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: pgmap v896: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-06-02T23:27:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:27:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:27:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:27:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:27:23.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:23.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:23 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: pgmap v896: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-06-02T23:27:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:27:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:27:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:27:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:27:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:23 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: pgmap v896: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 13 op/s 2024-06-02T23:27:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:27:24.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:23 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:24.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.69 MB/sec execute 32 sec latency 28117.912 ms 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.575+0000 7f8a36bae700 1 -- 172.21.15.52:0/3114120437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30107440 msgr2=0x7f8a30107820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.575+0000 7f8a36bae700 1 --2- 172.21.15.52:0/3114120437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30107440 0x7f8a30107820 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f8a20009990 tx=0x7f8a20009ca0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 -- 172.21.15.52:0/3114120437 shutdown_connections 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 --2- 172.21.15.52:0/3114120437 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a30101d80 0x7f8a3010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 --2- 172.21.15.52:0/3114120437 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30107440 0x7f8a30107820 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 --2- 172.21.15.52:0/3114120437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a301013c0 0x7f8a30101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 -- 172.21.15.52:0/3114120437 >> 172.21.15.52:0/3114120437 conn(0x7f8a30077400 msgr2=0x7f8a30077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 -- 172.21.15.52:0/3114120437 shutdown_connections 2024-06-02T23:27:24.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 -- 172.21.15.52:0/3114120437 wait complete. 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 Processor -- start 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 -- start start 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a301013c0 0x7f8a30199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.576+0000 7f8a36bae700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 0x7f8a30199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a36bae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a30107440 0x7f8a301a4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a36bae700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30111560 con 0x7f8a30101d80 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a36bae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a301113e0 con 0x7f8a301013c0 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a36bae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a301116e0 con 0x7f8a30107440 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 0x7f8a30199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 0x7f8a30199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43224/0 (socket says 172.21.15.52:43224) 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 -- 172.21.15.52:0/1269088646 learned_addr learned my addr 172.21.15.52:0/1269088646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:24.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a3494a700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a301013c0 0x7f8a30199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a3514b700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a30107440 0x7f8a301a4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 -- 172.21.15.52:0/1269088646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a30107440 msgr2=0x7f8a301a4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a30107440 0x7f8a301a4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 -- 172.21.15.52:0/1269088646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a301013c0 msgr2=0x7f8a30199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a301013c0 0x7f8a30199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 -- 172.21.15.52:0/1269088646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a301a4fe0 con 0x7f8a30101d80 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2ffff700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 0x7f8a30199f60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800d950 tx=0x7f8a1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a18016070 con 0x7f8a30101d80 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a301a5210 con 0x7f8a30101d80 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a1800fd40 con 0x7f8a30101d80 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.577+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a1800e3b0 con 0x7f8a30101d80 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.578+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301a5720 con 0x7f8a30101d80 2024-06-02T23:27:24.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.578+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a30109900 con 0x7f8a30101d80 2024-06-02T23:27:24.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.580+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8a18019070 con 0x7f8a30101d80 2024-06-02T23:27:24.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.580+0000 7f8a2dffb700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a1c0a2c20 0x7f8a1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:24.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.580+0000 7f8a3494a700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a1c0a2c20 0x7f8a1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:24.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.580+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8a180cea90 con 0x7f8a30101d80 2024-06-02T23:27:24.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.581+0000 7f8a3494a700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a1c0a2c20 0x7f8a1c0a50e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000b5c0 tx=0x7f8a20005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:24.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.585+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f8a1809ba20 con 0x7f8a30101d80 2024-06-02T23:27:24.623 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:24.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.849+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a30102ab0 con 0x7f8a30101d80 2024-06-02T23:27:24.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.850+0000 7f8a2dffb700 1 -- 172.21.15.52:0/1269088646 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) ==== 94+0+11946 (secure 0 0 0) 0x7f8a1809b320 con 0x7f8a30101d80 2024-06-02T23:27:24.851 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:27:24.851 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":40,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":40,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:27:21.892360+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[1,2,3,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:27:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a1c0a2c20 msgr2=0x7f8a1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a1c0a2c20 0x7f8a1c0a50e0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8a2000b5c0 tx=0x7f8a20005fd0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 msgr2=0x7f8a30199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:24.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 0x7f8a30199f60 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f8a1800d950 tx=0x7f8a1800dc60 comp rx=0 tx=0).stop 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 shutdown_connections 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a1c0a2c20 0x7f8a1c0a50e0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a30107440 0x7f8a301a4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a301013c0 0x7f8a30199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 --2- 172.21.15.52:0/1269088646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a30101d80 0x7f8a30199f60 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 >> 172.21.15.52:0/1269088646 conn(0x7f8a30077400 msgr2=0x7f8a30100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 shutdown_connections 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:24.853+0000 7f8a36bae700 1 -- 172.21.15.52:0/1269088646 wait complete. 2024-06-02T23:27:24.854 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 40 2024-06-02T23:27:25.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.64 MB/sec execute 33 sec latency 29117.997 ms 2024-06-02T23:27:25.981 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:25 smithi052 ceph-mon[32340]: pgmap v897: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 KiB/s wr, 24 op/s 2024-06-02T23:27:25.982 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:25 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1269088646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:27:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:25 smithi123 ceph-mon[35977]: pgmap v897: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 KiB/s wr, 24 op/s 2024-06-02T23:27:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:25 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1269088646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:27:26.014 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id b6419fd5 --release --await' 2024-06-02T23:27:26.014 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id b6419fd5 --release --await 2024-06-02T23:27:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:25 smithi174 ceph-mon[35994]: pgmap v897: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 6.6 KiB/s wr, 24 op/s 2024-06-02T23:27:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:25 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1269088646' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:27:26.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.59 MB/sec execute 34 sec latency 30118.067 ms 2024-06-02T23:27:26.613 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 -- 172.21.15.52:0/4241503541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c102380 msgr2=0x7f811c105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 --2- 172.21.15.52:0/4241503541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c102380 0x7f811c105ff0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f81040099c0 tx=0x7f8104009cd0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 -- 172.21.15.52:0/4241503541 shutdown_connections 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 --2- 172.21.15.52:0/4241503541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c106720 0x7f811c10e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 --2- 172.21.15.52:0/4241503541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c1019d0 0x7f811c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 --2- 172.21.15.52:0/4241503541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c102380 0x7f811c105ff0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 -- 172.21.15.52:0/4241503541 >> 172.21.15.52:0/4241503541 conn(0x7f811c078340 msgr2=0x7f811c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 -- 172.21.15.52:0/4241503541 shutdown_connections 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.306+0000 7f81219aa700 1 -- 172.21.15.52:0/4241503541 wait complete. 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.307+0000 7f81219aa700 1 Processor -- start 2024-06-02T23:27:27.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.307+0000 7f81219aa700 1 -- start start 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.307+0000 7f81219aa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c1019d0 0x7f811c103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.307+0000 7f81219aa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 0x7f811c1035d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.307+0000 7f81219aa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 0x7f811c103b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f81219aa700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1115c0 con 0x7f811c106720 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f81219aa700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f811c111440 con 0x7f811c102380 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f81219aa700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f811c111740 con 0x7f811c1019d0 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 0x7f811c103b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 0x7f811c1035d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 0x7f811c103b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43252/0 (socket says 172.21.15.52:43252) 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811b7fe700 1 -- 172.21.15.52:0/1611397708 learned_addr learned my addr 172.21.15.52:0/1611397708 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 0x7f811c1035d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59290/0 (socket says 172.21.15.52:59290) 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811affd700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c1019d0 0x7f811c103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 -- 172.21.15.52:0/1611397708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c1019d0 msgr2=0x7f811c103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c1019d0 0x7f811c103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 -- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 msgr2=0x7f811c103b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 0x7f811c103b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 -- 172.21.15.52:0/1611397708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811c1a7190 con 0x7f811c102380 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811b7fe700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 0x7f811c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f811a7fc700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 0x7f811c1035d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81040098e0 tx=0x7f81040047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.308+0000 7f81209a8700 1 -- 172.21.15.52:0/1611397708 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104027070 con 0x7f811c102380 2024-06-02T23:27:27.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.309+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811c1a7420 con 0x7f811c102380 2024-06-02T23:27:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.309+0000 7f81209a8700 1 -- 172.21.15.52:0/1611397708 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8104003680 con 0x7f811c102380 2024-06-02T23:27:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.309+0000 7f81209a8700 1 -- 172.21.15.52:0/1611397708 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104022ac0 con 0x7f811c102380 2024-06-02T23:27:27.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.309+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f811c1a78d0 con 0x7f811c102380 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.311+0000 7f81209a8700 1 -- 172.21.15.52:0/1611397708 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f810402a070 con 0x7f811c102380 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f81209a8700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81080a2c20 0x7f81080a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f811affd700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81080a2c20 0x7f81080a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f81209a8700 1 -- 172.21.15.52:0/1611397708 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f81040cff40 con 0x7f811c102380 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 Processor -- start 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 -- start start 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f811affd700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81080a2c20 0x7f81080a50e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f810c009a10 tx=0x7f810c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5ba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5babc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56073c4acf60 con 0x56073c5c0880 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56073c50b850 con 0x56073c5ba7a0 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.312+0000 7f812c22db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56073c473460 con 0x56073c5bff60 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5ba260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43262/0 (socket says 172.21.15.52:43262) 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 -- 172.21.15.52:0/1541538730 learned_addr learned my addr 172.21.15.52:0/1541538730 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811b7fe700 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5babc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811affd700 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 -- 172.21.15.52:0/1541538730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 msgr2=0x56073c5c0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 -- 172.21.15.52:0/1541538730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 msgr2=0x56073c5babc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5babc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 -- 172.21.15.52:0/1541538730 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c45a8d0 con 0x56073c5c0880 2024-06-02T23:27:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f811a7fc700 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5ba260 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8104098c60 tx=0x7f8104009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61bab7eacc50c3c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f8111ffb700 1 -- 172.21.15.52:0/1541538730 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81040a30f0 con 0x56073c5c0880 2024-06-02T23:27:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f8111ffb700 1 -- 172.21.15.52:0/1541538730 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f810409cf00 con 0x56073c5c0880 2024-06-02T23:27:27.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.313+0000 7f8111ffb700 1 -- 172.21.15.52:0/1541538730 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104087120 con 0x56073c5c0880 2024-06-02T23:27:27.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 -- 172.21.15.52:0/1541538730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 msgr2=0x56073c5ba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5ba260 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f8104098c60 tx=0x7f8104009f90 comp rx=0 tx=0).stop 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 -- 172.21.15.52:0/1541538730 shutdown_connections 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5babc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 --2- 172.21.15.52:0/1541538730 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5ba260 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 -- 172.21.15.52:0/1541538730 >> 172.21.15.52:0/1541538730 conn(0x56073c4e7080 msgr2=0x56073c5b5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 -- 172.21.15.52:0/1541538730 shutdown_connections 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.314+0000 7f812c22db80 1 -- 172.21.15.52:0/1541538730 wait complete. 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 -- start start 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 Processor -- start 2024-06-02T23:27:27.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56073c4abfc0 con 0x56073c5c0880 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56073c507810 con 0x56073c5ba7a0 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56073c5c7010 con 0x56073c5bff60 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f812c22db80 10 client.0 _ll_register_callbacks cb 0x56073c4a15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43270/0 (socket says 172.21.15.52:43270) 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f811b7fe700 1 -- 172.21.15.52:0/2007689217 learned_addr learned my addr 172.21.15.52:0/2007689217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.315+0000 7f811affd700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811a7fc700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811b7fe700 1 -- 172.21.15.52:0/2007689217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 msgr2=0x56073c5c3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811b7fe700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811b7fe700 1 -- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 msgr2=0x56073c5c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811b7fe700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811b7fe700 1 -- 172.21.15.52:0/2007689217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073c4d1750 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f811b7fe700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c3eb0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8114007f70 tx=0x7f8114012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d39913387f20513 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f80faffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f80faffd700 1 -- 172.21.15.52:0/2007689217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811401a070 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f80faffd700 1 -- 172.21.15.52:0/2007689217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8114016420 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810409cf00 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f80faffd700 1 -- 172.21.15.52:0/2007689217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f8114026050 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f812c22db80 10 client.19752 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56073c5d17a0 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.316+0000 7f80faffd700 1 -- 172.21.15.52:0/2007689217 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f810409cf00 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f80faffd700 1 -- 172.21.15.52:0/2007689217 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f8114015650 con 0x56073c5c0880 2024-06-02T23:27:27.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f812c22db80 10 client.19752 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f812c22db80 10 client.19752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f812c22db80 20 client.19752 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f812c22db80 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c5d3d80 0x56073c5d61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f812c22db80 4 client.19752 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56073c518d50 con 0x56073c5d3d80 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.317+0000 7f811affd700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c5d3d80 0x56073c5d61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.318+0000 7f811affd700 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c5d3d80 0x56073c5d61e0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd7b45ea5a6c04d6 server_cookie=a3502d5abc65205e in_seq=0 out_seq=0 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.318+0000 7f80faffd700 10 client.19752 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.318+0000 7f80faffd700 1 -- 172.21.15.52:0/2007689217 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56073c518d50 con 0x56073c5d3d80 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.318+0000 7f80faffd700 10 client.19752 handle_command_reply: tid=0 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.318+0000 7f812c22db80 1 client.19752 shutdown 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 msgr2=0x56073c5c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c3eb0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f8114007f70 tx=0x7f8114012520 comp rx=0 tx=0).stop 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 shutdown_connections 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c5d3d80 0x56073c5d61e0 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c3eb0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5ba7a0 0x56073c5c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 --2- 172.21.15.52:0/2007689217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5bff60 0x56073c5c3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f80faffd700 0 client.19752 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 >> 172.21.15.52:0/2007689217 conn(0x56073c4e7080 msgr2=0x56073c5b4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 shutdown_connections 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 1 -- 172.21.15.52:0/2007689217 wait complete. 2024-06-02T23:27:27.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.319+0000 7f812c22db80 20 client.19752 trim_cache size 0 max 16384 2024-06-02T23:27:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 Processor -- start 2024-06-02T23:27:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 -- start start 2024-06-02T23:27:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5ba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c1390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1aebb0 con 0x56073c5c0880 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1b07b0 con 0x56073c5bff60 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f812c22db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1af280 con 0x56073c5ba7a0 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43280/0 (socket says 172.21.15.52:43280) 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 -- 172.21.15.52:0/359850273 learned_addr learned my addr 172.21.15.52:0/359850273 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811affd700 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5ba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811a7fc700 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 -- 172.21.15.52:0/359850273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 msgr2=0x56073c5ba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5ba260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 -- 172.21.15.52:0/359850273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 msgr2=0x56073c5c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.340+0000 7f811b7fe700 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.341+0000 7f811b7fe700 1 -- 172.21.15.52:0/359850273 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56073c5c1ee0 con 0x56073c5c0880 2024-06-02T23:27:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.341+0000 7f811b7fe700 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c1390 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8114007c90 tx=0x7f8114015e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c229c0ad80f296 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.341+0000 7f81117fa700 1 -- 172.21.15.52:0/359850273 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811402f070 con 0x56073c5c0880 2024-06-02T23:27:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.341+0000 7f81117fa700 1 -- 172.21.15.52:0/359850273 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8114023030 con 0x56073c5c0880 2024-06-02T23:27:27.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.341+0000 7f81117fa700 1 -- 172.21.15.52:0/359850273 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811401a070 con 0x56073c5c0880 2024-06-02T23:27:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 -- 172.21.15.52:0/359850273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 msgr2=0x56073c5c1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c1390 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f8114007c90 tx=0x7f8114015e20 comp rx=0 tx=0).stop 2024-06-02T23:27:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 -- 172.21.15.52:0/359850273 shutdown_connections 2024-06-02T23:27:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5c1390 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5ba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 --2- 172.21.15.52:0/359850273 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 -- 172.21.15.52:0/359850273 >> 172.21.15.52:0/359850273 conn(0x56073c4e7080 msgr2=0x56073c5c59e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 -- 172.21.15.52:0/359850273 shutdown_connections 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 -- 172.21.15.52:0/359850273 wait complete. 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 -- start start 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.342+0000 7f812c22db80 1 Processor -- start 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5dbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5dc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1b0020 con 0x56073c5c0880 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1af950 con 0x56073c5bff60 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f811c1acb70 con 0x56073c5ba7a0 2024-06-02T23:27:27.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f812c22db80 10 client.0 _ll_register_callbacks cb 0x56073c4a15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5dc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5dc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43296/0 (socket says 172.21.15.52:43296) 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 -- 172.21.15.52:0/4114755729 learned_addr learned my addr 172.21.15.52:0/4114755729 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811affd700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811a7fc700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5dbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 -- 172.21.15.52:0/4114755729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 msgr2=0x56073c5db680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5db680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 -- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 msgr2=0x56073c5dbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5dbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 -- 172.21.15.52:0/4114755729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56073c63da80 con 0x56073c5c0880 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811a7fc700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5dbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f811b7fe700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5dc100 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f8114015aa0 tx=0x7f81140091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b811185df46bef2b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f80faffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:27.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.343+0000 7f80faffd700 1 -- 172.21.15.52:0/4114755729 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f811402f050 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f80faffd700 1 -- 172.21.15.52:0/4114755729 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f81140153c0 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8114023030 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f80faffd700 1 -- 172.21.15.52:0/4114755729 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f811401a040 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 10 client.19758 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x56073c63ed40 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f80faffd700 1 -- 172.21.15.52:0/4114755729 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8114023030 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f80faffd700 1 -- 172.21.15.52:0/4114755729 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f811403faf0 con 0x56073c5c0880 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 10 client.19758 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 10 client.19758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 20 client.19758 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c63f3c0 0x56073c6417e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 4 client.19758 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:27:27.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56073c5132d0 con 0x56073c63f3c0 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.344+0000 7f811affd700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c63f3c0 0x56073c6417e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f811affd700 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c63f3c0 0x56073c6417e0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4875034e52852f77 server_cookie=6765ee488c67acc3 in_seq=0 out_seq=0 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f80faffd700 10 client.19758 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f80faffd700 1 -- 172.21.15.52:0/4114755729 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56073c5132d0 con 0x56073c63f3c0 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f80faffd700 10 client.19758 handle_command_reply: tid=0 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 client.19758 shutdown 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 msgr2=0x56073c5dc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5dc100 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f8114015aa0 tx=0x7f81140091b0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 shutdown_connections 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56073c63f3c0 0x56073c6417e0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56073c5c0880 0x56073c5dc100 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56073c5ba7a0 0x56073c5db680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.350 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f80faffd700 0 client.19758 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 --2- 172.21.15.52:0/4114755729 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56073c5bff60 0x56073c5dbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 >> 172.21.15.52:0/4114755729 conn(0x56073c4e7080 msgr2=0x56073c5bf310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.345+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 shutdown_connections 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f812c22db80 1 -- 172.21.15.52:0/4114755729 wait complete. 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f812c22db80 20 client.19758 trim_cache size 0 max 16384 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stdout: "80c1e53f-4dd0-4b01-8072-aae079a44855": { 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:27:27.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "80c1e53f-4dd0-4b01-8072-aae079a44855", 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81080a2c20 msgr2=0x7f81080a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81080a2c20 0x7f81080a50e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f810c009a10 tx=0x7f810c009190 comp rx=0 tx=0).stop 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 msgr2=0x7f811c1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:27.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 0x7f811c1035d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f81040098e0 tx=0x7f81040047b0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 shutdown_connections 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f81080a2c20 0x7f81080a50e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f811c106720 0x7f811c103b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f811c1019d0 0x7f811c103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 --2- 172.21.15.52:0/1611397708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f811c102380 0x7f811c1035d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 >> 172.21.15.52:0/1611397708 conn(0x7f811c078340 msgr2=0x7f811c06afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 shutdown_connections 2024-06-02T23:27:27.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:27.346+0000 7f81219aa700 1 -- 172.21.15.52:0/1611397708 wait complete. 2024-06-02T23:27:27.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.55 MB/sec execute 35 sec latency 31118.112 ms 2024-06-02T23:27:27.811 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:27 smithi052 ceph-mon[32340]: pgmap v898: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.1 KiB/s wr, 59 op/s 2024-06-02T23:27:28.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:27 smithi123 ceph-mon[35977]: pgmap v898: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.1 KiB/s wr, 59 op/s 2024-06-02T23:27:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:27 smithi174 ceph-mon[35994]: pgmap v898: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 6.1 KiB/s wr, 59 op/s 2024-06-02T23:27:28.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.51 MB/sec execute 36 sec latency 32118.219 ms 2024-06-02T23:27:28.947 INFO:tasks.cephfs.filesystem:scrub status for tag:80c1e53f-4dd0-4b01-8072-aae079a44855 - {'path': '/', 'tag': '80c1e53f-4dd0-4b01-8072-aae079a44855', 'options': 'recursive,force'} 2024-06-02T23:27:29.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13628 1.47 MB/sec execute 37 sec latency 33118.275 ms 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 -- 172.21.15.52:0/141363788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7658108580 msgr2=0x7f7658108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 --2- 172.21.15.52:0/141363788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7658108580 0x7f7658108960 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f76480099c0 tx=0x7f7648009cd0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 -- 172.21.15.52:0/141363788 shutdown_connections 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 --2- 172.21.15.52:0/141363788 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 0x7f765810f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 --2- 172.21.15.52:0/141363788 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7658108580 0x7f7658108960 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 --2- 172.21.15.52:0/141363788 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f765806ac60 0x7f765806b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 -- 172.21.15.52:0/141363788 >> 172.21.15.52:0/141363788 conn(0x7f7658078430 msgr2=0x7f7658078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:29.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 -- 172.21.15.52:0/141363788 shutdown_connections 2024-06-02T23:27:29.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.481+0000 7f765e994700 1 -- 172.21.15.52:0/141363788 wait complete. 2024-06-02T23:27:29.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 Processor -- start 2024-06-02T23:27:29.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 -- start start 2024-06-02T23:27:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 0x7f765819e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 0x7f765819ee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7658108580 0x7f7658198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7658112740 con 0x7f765806ac60 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76581125c0 con 0x7f765806b620 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765e994700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76581128c0 con 0x7f7658108580 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 0x7f765819e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 0x7f765819e8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43308/0 (socket says 172.21.15.52:43308) 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 -- 172.21.15.52:0/1767046299 learned_addr learned my addr 172.21.15.52:0/1767046299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f765cf31700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7658108580 0x7f7658198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 -- 172.21.15.52:0/1767046299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7658108580 msgr2=0x7f7658198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7658108580 0x7f7658198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f764ffff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 0x7f765819ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 -- 172.21.15.52:0/1767046299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 msgr2=0x7f765819ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 0x7f765819ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f7657fff700 1 -- 172.21.15.52:0/1767046299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7658199200 con 0x7f765806ac60 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.482+0000 7f764ffff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 0x7f765819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.483+0000 7f7657fff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 0x7f765819e8e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7648009990 tx=0x7f7648009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.483+0000 7f7655ffb700 1 -- 172.21.15.52:0/1767046299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648027070 con 0x7f765806ac60 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.483+0000 7f7655ffb700 1 -- 172.21.15.52:0/1767046299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7648003fd0 con 0x7f765806ac60 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.483+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7658199490 con 0x7f765806ac60 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.483+0000 7f7655ffb700 1 -- 172.21.15.52:0/1767046299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7648022c70 con 0x7f765806ac60 2024-06-02T23:27:29.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.483+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76581a57b0 con 0x7f765806ac60 2024-06-02T23:27:29.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.485+0000 7f7655ffb700 1 -- 172.21.15.52:0/1767046299 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f764802a070 con 0x7f765806ac60 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.486+0000 7f7655ffb700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f76440a2c20 0x7f76440a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.486+0000 7f764ffff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f76440a2c20 0x7f76440a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.486+0000 7f7655ffb700 1 -- 172.21.15.52:0/1767046299 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f76480cffe0 con 0x7f765806ac60 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.486+0000 7f7669217b80 1 Processor -- start 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.486+0000 7f7669217b80 1 -- start start 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.486+0000 7f764ffff700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f76440a2c20 0x7f76440a50e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7658078b30 tx=0x7f7640008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7669217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7669217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7669217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0eb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7669217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c55c03c910 con 0x55c55c0f1560 2024-06-02T23:27:29.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7669217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c55bfe4820 con 0x55c55c0f0c40 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7669217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c55c0388d0 con 0x55c55c0eb3d0 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0eae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43312/0 (socket says 172.21.15.52:43312) 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 -- 172.21.15.52:0/3169794780 learned_addr learned my addr 172.21.15.52:0/3169794780 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f7657fff700 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f1020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f765cf31700 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0eb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 -- 172.21.15.52:0/3169794780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 msgr2=0x55c55c0eb810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0eb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 -- 172.21.15.52:0/3169794780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 msgr2=0x55c55c0f1020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f1020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 -- 172.21.15.52:0/3169794780 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55bfdb050 con 0x55c55c0f1560 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764ffff700 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0eae90 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f764000c910 tx=0x7f764000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64bfad3b3541787 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:29.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.487+0000 7f764e7fc700 1 -- 172.21.15.52:0/3169794780 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7640023070 con 0x55c55c0f1560 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f764e7fc700 1 -- 172.21.15.52:0/3169794780 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7640013de0 con 0x55c55c0f1560 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f764e7fc700 1 -- 172.21.15.52:0/3169794780 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764001ed80 con 0x55c55c0f1560 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 -- 172.21.15.52:0/3169794780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 msgr2=0x55c55c0eae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0eae90 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f764000c910 tx=0x7f764000f000 comp rx=0 tx=0).stop 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 -- 172.21.15.52:0/3169794780 shutdown_connections 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0eb810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f1020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 --2- 172.21.15.52:0/3169794780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0eae90 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 -- 172.21.15.52:0/3169794780 >> 172.21.15.52:0/3169794780 conn(0x55c55c015dc0 msgr2=0x55c55c0e6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 -- 172.21.15.52:0/3169794780 shutdown_connections 2024-06-02T23:27:29.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.488+0000 7f7669217b80 1 -- 172.21.15.52:0/3169794780 wait complete. 2024-06-02T23:27:29.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 -- start start 2024-06-02T23:27:29.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 Processor -- start 2024-06-02T23:27:29.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0f72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c55c0f7fd0 con 0x55c55c0f1560 2024-06-02T23:27:29.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c55bfdccf0 con 0x55c55c0f0c40 2024-06-02T23:27:29.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c55c0f8150 con 0x55c55c0eb3d0 2024-06-02T23:27:29.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f7669217b80 10 client.0 _ll_register_callbacks cb 0x55c55bfd6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:29.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f765cf31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f765cf31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43320/0 (socket says 172.21.15.52:43320) 2024-06-02T23:27:29.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.489+0000 7f765cf31700 1 -- 172.21.15.52:0/24939120 learned_addr learned my addr 172.21.15.52:0/24939120 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:29.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f764ffff700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f7657fff700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0f72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f765cf31700 1 -- 172.21.15.52:0/24939120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 msgr2=0x55c55c0f72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f765cf31700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0f72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f765cf31700 1 -- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 msgr2=0x55c55c0f45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.498 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f765cf31700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f765cf31700 1 -- 172.21.15.52:0/24939120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55bf8b9d0 con 0x55c55c0f1560 2024-06-02T23:27:29.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f765cf31700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f4ae0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7650007940 tx=0x7f7650007d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bcd3b0d6ebd8b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:29.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f76377fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:29.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f76377fe700 1 -- 172.21.15.52:0/24939120 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650019020 con 0x55c55c0f1560 2024-06-02T23:27:29.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f76377fe700 1 -- 172.21.15.52:0/24939120 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7650010860 con 0x55c55c0f1560 2024-06-02T23:27:29.499 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7640013de0 con 0x55c55c0f1560 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f76377fe700 1 -- 172.21.15.52:0/24939120 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7650010a60 con 0x55c55c0f1560 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f7669217b80 10 client.19770 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c55c102910 con 0x55c55c0f1560 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.490+0000 7f76377fe700 1 -- 172.21.15.52:0/24939120 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7640013de0 con 0x55c55c0f1560 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f76377fe700 1 -- 172.21.15.52:0/24939120 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f7650020cc0 con 0x55c55c0f1560 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7669217b80 10 client.19770 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:29.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7669217b80 10 client.19770 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:27:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7669217b80 20 client.19770 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7669217b80 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c104ff0 0x55c55c107490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7669217b80 4 client.19770 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c55c049e10 con 0x55c55c104ff0 2024-06-02T23:27:29.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7657fff700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c104ff0 0x55c55c107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f7657fff700 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c104ff0 0x55c55c107490 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc993d18eeb49fd server_cookie=a73f33a1afb349ed in_seq=0 out_seq=0 2024-06-02T23:27:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.491+0000 7f76377fe700 10 client.19770 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.492+0000 7f76377fe700 1 -- 172.21.15.52:0/24939120 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c55c049e10 con 0x55c55c104ff0 2024-06-02T23:27:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.492+0000 7f76377fe700 10 client.19770 handle_command_reply: tid=0 2024-06-02T23:27:29.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.492+0000 7f7669217b80 1 client.19770 shutdown 2024-06-02T23:27:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.492+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 msgr2=0x55c55c0f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.492+0000 7f7669217b80 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f4ae0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f7650007940 tx=0x7f7650007d00 comp rx=0 tx=0).stop 2024-06-02T23:27:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 shutdown_connections 2024-06-02T23:27:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c104ff0 0x55c55c107490 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f4ae0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c0f72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f76377fe700 0 client.19770 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 --2- 172.21.15.52:0/24939120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 >> 172.21.15.52:0/24939120 conn(0x55c55c015dc0 msgr2=0x55c55c0e9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 shutdown_connections 2024-06-02T23:27:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 1 -- 172.21.15.52:0/24939120 wait complete. 2024-06-02T23:27:29.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.493+0000 7f7669217b80 20 client.19770 trim_cache size 0 max 16384 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.513+0000 7f7669217b80 1 Processor -- start 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.513+0000 7f7669217b80 1 -- start start 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.513+0000 7f7669217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c1012a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.513+0000 7f7669217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f7669217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f7669217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76581aca60 con 0x55c55c0f1560 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f7669217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76581ae660 con 0x55c55c0f0c40 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f7669217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76581ad130 con 0x55c55c0eb3d0 2024-06-02T23:27:29.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43328/0 (socket says 172.21.15.52:43328) 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 -- 172.21.15.52:0/2447952038 learned_addr learned my addr 172.21.15.52:0/2447952038 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f764ffff700 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f7657fff700 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c1012a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 -- 172.21.15.52:0/2447952038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 msgr2=0x55c55c1012a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c1012a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 -- 172.21.15.52:0/2447952038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 msgr2=0x55c55c0f2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 -- 172.21.15.52:0/2447952038 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c55c0f4c30 con 0x55c55c0f1560 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f764ffff700 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f765cf31700 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f3d70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7650002230 tx=0x7f7650005b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bfeaeb0ab03d301 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f764dffb700 1 -- 172.21.15.52:0/2447952038 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765002e020 con 0x55c55c0f1560 2024-06-02T23:27:29.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f764dffb700 1 -- 172.21.15.52:0/2447952038 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76500092f0 con 0x55c55c0f1560 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.514+0000 7f764dffb700 1 -- 172.21.15.52:0/2447952038 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650019040 con 0x55c55c0f1560 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 -- 172.21.15.52:0/2447952038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 msgr2=0x55c55c0f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f3d70 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7650002230 tx=0x7f7650005b80 comp rx=0 tx=0).stop 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 -- 172.21.15.52:0/2447952038 shutdown_connections 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f1560 0x55c55c0f3d70 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0eb3d0 0x55c55c1012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 --2- 172.21.15.52:0/2447952038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0f0c40 0x55c55c0f2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 -- 172.21.15.52:0/2447952038 >> 172.21.15.52:0/2447952038 conn(0x55c55c015dc0 msgr2=0x55c55c0f1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 -- 172.21.15.52:0/2447952038 shutdown_connections 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.515+0000 7f7669217b80 1 -- 172.21.15.52:0/2447952038 wait complete. 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 -- start start 2024-06-02T23:27:29.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 Processor -- start 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0eb3d0 0x55c55c0fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 0x55c55c0fa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0f1560 0x55c55c0fab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f76581aded0 con 0x55c55c0f0c40 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76581ad800 con 0x55c55c0eb3d0 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76581aaa00 con 0x55c55c0f1560 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f764ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 0x55c55c0fa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7669217b80 10 client.0 _ll_register_callbacks cb 0x55c55bfd6a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f764ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 0x55c55c0fa3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43336/0 (socket says 172.21.15.52:43336) 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f764ffff700 1 -- 172.21.15.52:0/2456738832 learned_addr learned my addr 172.21.15.52:0/2456738832 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.516+0000 7f7657fff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0eb3d0 0x55c55c0fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f765cf31700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0f1560 0x55c55c0fab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f764ffff700 1 -- 172.21.15.52:0/2456738832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0f1560 msgr2=0x55c55c0fab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f764ffff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0f1560 0x55c55c0fab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f764ffff700 1 -- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0eb3d0 msgr2=0x55c55c0fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f764ffff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0eb3d0 0x55c55c0fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f764ffff700 1 -- 172.21.15.52:0/2456738832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c55c174150 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f7657fff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0eb3d0 0x55c55c0fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f764ffff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 0x55c55c0fa3e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f764000c8e0 tx=0x7f764001e480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee2ac486bb154fa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f76377fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f76377fe700 1 -- 172.21.15.52:0/2456738832 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f764002d070 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f76377fe700 1 -- 172.21.15.52:0/2456738832 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f764001e950 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76500092f0 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f76377fe700 1 -- 172.21.15.52:0/2456738832 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f7640039050 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f7669217b80 10 client.19776 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.517+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55c55c175290 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f76377fe700 1 -- 172.21.15.52:0/2456738832 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f76500092f0 con 0x55c55c0f0c40 2024-06-02T23:27:29.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f76377fe700 1 -- 172.21.15.52:0/2456738832 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f76400293f0 con 0x55c55c0f0c40 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7669217b80 10 client.19776 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7669217b80 10 client.19776 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7669217b80 20 client.19776 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7669217b80 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c1756b0 0x55c55c177ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7669217b80 4 client.19776 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b6419fd5", "release": true, "await": true}] 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b6419fd5", "release": true, "await": true}) -- 0x55c55c044390 con 0x55c55c1756b0 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.518+0000 7f7657fff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c1756b0 0x55c55c177ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.519+0000 7f76377fe700 10 client.19776 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:29.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.519+0000 7f7657fff700 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c1756b0 0x55c55c177ad0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2a1bb2e658cb1f5 server_cookie=b4f3de35993662c4 in_seq=0 out_seq=0 2024-06-02T23:27:29.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.553+0000 7f76377fe700 1 -- 172.21.15.52:0/2456738832 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c55c044390 con 0x55c55c1756b0 2024-06-02T23:27:29.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.553+0000 7f76377fe700 10 client.19776 handle_command_reply: tid=0 2024-06-02T23:27:29.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.553+0000 7f7669217b80 1 client.19776 shutdown 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.553+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 msgr2=0x55c55c0fa3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.553+0000 7f7669217b80 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 0x55c55c0fa3e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f764000c8e0 tx=0x7f764001e480 comp rx=0 tx=0).stop 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 shutdown_connections 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c55c1756b0 0x55c55c177ad0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f76377fe700 0 client.19776 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c55c0f1560 0x55c55c0fab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c55c0eb3d0 0x55c55c0fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 --2- 172.21.15.52:0/2456738832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c55c0f0c40 0x55c55c0fa3e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 >> 172.21.15.52:0/2456738832 conn(0x55c55c015dc0 msgr2=0x55c55c0e5830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 shutdown_connections 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 1 -- 172.21.15.52:0/2456738832 wait complete. 2024-06-02T23:27:29.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.554+0000 7f7669217b80 20 client.19776 trim_cache size 0 max 16384 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 40, 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 56, 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout: "b6419fd5": { 2024-06-02T23:27:29.556 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 56, 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:27:29.557 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:29.558 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:29.558 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:29.558 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:29.558 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:27:29.558 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f76440a2c20 msgr2=0x7f76440a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f76440a2c20 0x7f76440a50e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f7658078b30 tx=0x7f7640008040 comp rx=0 tx=0).stop 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 msgr2=0x7f765819e8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 0x7f765819e8e0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7648009990 tx=0x7f7648009f80 comp rx=0 tx=0).stop 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 shutdown_connections 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f76440a2c20 0x7f76440a50e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7658108580 0x7f7658198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f765806ac60 0x7f765819e8e0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 --2- 172.21.15.52:0/1767046299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f765806b620 0x7f765819ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:29.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 >> 172.21.15.52:0/1767046299 conn(0x7f7658078430 msgr2=0x7f76581000f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:29.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 shutdown_connections 2024-06-02T23:27:29.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:27:29.555+0000 7f765e994700 1 -- 172.21.15.52:0/1767046299 wait complete. 2024-06-02T23:27:29.895 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:29 smithi052 ceph-mon[32340]: pgmap v899: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.0 KiB/s wr, 70 op/s 2024-06-02T23:27:30.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:29 smithi123 ceph-mon[35977]: pgmap v899: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.0 KiB/s wr, 70 op/s 2024-06-02T23:27:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:29 smithi174 ceph-mon[35994]: pgmap v899: 97 pgs: 97 active+clean; 341 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 6.0 KiB/s wr, 70 op/s 2024-06-02T23:27:30.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b6419fd5', total seconds elapsed: 40.0 2024-06-02T23:27:30.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13816 1.63 MB/sec execute 38 sec latency 33189.383 ms 2024-06-02T23:27:31.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:30 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:27:31.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:30 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:27:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:30 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:27:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:30 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:27:31.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:30 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:27:31.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:30 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:27:31.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 13968 1.93 MB/sec execute 39 sec latency 173.473 ms 2024-06-02T23:27:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:31 smithi123 ceph-mon[35977]: pgmap v900: 97 pgs: 97 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 KiB/s wr, 70 op/s 2024-06-02T23:27:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:27:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:27:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:27:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:31 smithi174 ceph-mon[35994]: pgmap v900: 97 pgs: 97 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 KiB/s wr, 70 op/s 2024-06-02T23:27:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:27:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:27:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:27:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:31 smithi052 ceph-mon[32340]: pgmap v900: 97 pgs: 97 active+clean; 342 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 25 KiB/s wr, 70 op/s 2024-06-02T23:27:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:27:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:27:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:27:32.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 14124 2.11 MB/sec execute 40 sec latency 53.046 ms 2024-06-02T23:27:33.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 14341 2.53 MB/sec execute 41 sec latency 57.017 ms 2024-06-02T23:27:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:33 smithi123 ceph-mon[35977]: pgmap v901: 97 pgs: 97 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 170 KiB/s wr, 78 op/s 2024-06-02T23:27:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:33 smithi174 ceph-mon[35994]: pgmap v901: 97 pgs: 97 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 170 KiB/s wr, 78 op/s 2024-06-02T23:27:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:33 smithi052 ceph-mon[32340]: pgmap v901: 97 pgs: 97 active+clean; 343 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 185 KiB/s rd, 170 KiB/s wr, 78 op/s 2024-06-02T23:27:34.455 INFO:tasks.workunit.client.0.smithi052.stdout: 10 14475 2.59 MB/sec execute 42 sec latency 97.153 ms 2024-06-02T23:27:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 -- 172.21.15.52:0/4101543336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c8104e20 msgr2=0x7f07c81052a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 --2- 172.21.15.52:0/4101543336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c8104e20 0x7f07c81052a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f07b4009900 tx=0x7f07b402a410 comp rx=0 tx=0).stop 2024-06-02T23:27:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 -- 172.21.15.52:0/4101543336 shutdown_connections 2024-06-02T23:27:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 --2- 172.21.15.52:0/4101543336 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c81057e0 0x7f07c810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 --2- 172.21.15.52:0/4101543336 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c8104e20 0x7f07c81052a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 --2- 172.21.15.52:0/4101543336 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8104050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.758+0000 7f07cf241640 1 -- 172.21.15.52:0/4101543336 >> 172.21.15.52:0/4101543336 conn(0x7f07c80ff1c0 msgr2=0x7f07c81015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.759+0000 7f07cf241640 1 -- 172.21.15.52:0/4101543336 shutdown_connections 2024-06-02T23:27:34.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.759+0000 7f07cf241640 1 -- 172.21.15.52:0/4101543336 wait complete. 2024-06-02T23:27:34.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.759+0000 7f07cf241640 1 Processor -- start 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.759+0000 7f07cf241640 1 -- start start 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cf241640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cf241640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 0x7f07c819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cf241640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c81057e0 0x7f07c81a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cf241640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810f740 con 0x7f07c8104e20 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cf241640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810f5c0 con 0x7f07c81057e0 2024-06-02T23:27:34.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cf241640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07c810f8c0 con 0x7f07c8103c30 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 0x7f07c819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 0x7f07c819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59534/0 (socket says 172.21.15.52:59534) 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54090/0 (socket says 172.21.15.52:54090) 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cd7de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c81057e0 0x7f07c81a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07c7fff640 1 -- 172.21.15.52:0/1826067302 learned_addr learned my addr 172.21.15.52:0/1826067302 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1826067302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c81057e0 msgr2=0x7f07c81a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c81057e0 0x7f07c81a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 msgr2=0x7f07c819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 0x7f07c819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1826067302 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c81a19c0 con 0x7f07c8103c30 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07cd7de640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c81057e0 0x7f07c81a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:34.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07c7fff640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 0x7f07c819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.760+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8199db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00d6a0 tx=0x7f07bc00db60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.761+0000 7f07c5ffb640 1 -- 172.21.15.52:0/1826067302 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc017070 con 0x7f07c8103c30 2024-06-02T23:27:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.761+0000 7f07c5ffb640 1 -- 172.21.15.52:0/1826067302 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07bc009cd0 con 0x7f07c8103c30 2024-06-02T23:27:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.761+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c81a1cb0 con 0x7f07c8103c30 2024-06-02T23:27:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.761+0000 7f07c5ffb640 1 -- 172.21.15.52:0/1826067302 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc003bf0 con 0x7f07c8103c30 2024-06-02T23:27:34.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.761+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c81a21b0 con 0x7f07c8103c30 2024-06-02T23:27:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.764+0000 7f07c5ffb640 1 -- 172.21.15.52:0/1826067302 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f07bc009820 con 0x7f07c8103c30 2024-06-02T23:27:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.764+0000 7f07d2020740 1 Processor -- start 2024-06-02T23:27:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 -- start start 2024-06-02T23:27:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07c5ffb640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07b80a2c30 0x7f07b80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 0x55880ff84430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff89500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff8c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55880fee74f0 con 0x55880ff84a00 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55880fe85870 con 0x55880ff84010 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07d2020740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55880fee34b0 con 0x55880ff89b70 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07cd7de640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07b80a2c30 0x7f07b80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07c5ffb640 1 -- 172.21.15.52:0/1826067302 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f07bc0cece0 con 0x7f07c8103c30 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 0x55880ff84430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 0x55880ff84430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:52244/0 (socket says 172.21.15.52:52244) 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1989372813 learned_addr learned my addr 172.21.15.52:0/1989372813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07c7fff640 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff89500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07cd7de640 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff8c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1989372813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 msgr2=0x55880ff8c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff8c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1989372813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 msgr2=0x55880ff89500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff89500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07ccfdd640 1 -- 172.21.15.52:0/1989372813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880fea19b0 con 0x55880ff84010 2024-06-02T23:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.765+0000 7f07cd7de640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07b80a2c30 0x7f07b80a50f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f07c00099e0 tx=0x7f07c000ff60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.766+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 0x55880ff84430 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f07bc004610 tx=0x7f07bc09af80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23736617cc9a30f4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.766+0000 7f07b2ffd640 1 -- 172.21.15.52:0/1989372813 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc020070 con 0x55880ff84010 2024-06-02T23:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.766+0000 7f07b2ffd640 1 -- 172.21.15.52:0/1989372813 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07bc009b10 con 0x55880ff84010 2024-06-02T23:27:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.766+0000 7f07b2ffd640 1 -- 172.21.15.52:0/1989372813 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc003e60 con 0x55880ff84010 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 -- 172.21.15.52:0/1989372813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 msgr2=0x55880ff84430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 0x55880ff84430 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f07bc004610 tx=0x7f07bc09af80 comp rx=0 tx=0).stop 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 -- 172.21.15.52:0/1989372813 shutdown_connections 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff8c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff89500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 --2- 172.21.15.52:0/1989372813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84010 0x55880ff84430 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 -- 172.21.15.52:0/1989372813 >> 172.21.15.52:0/1989372813 conn(0x55880ff7f680 msgr2=0x55880ff7fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 -- 172.21.15.52:0/1989372813 shutdown_connections 2024-06-02T23:27:34.772 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 -- 172.21.15.52:0/1989372813 wait complete. 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.767+0000 7f07d2020740 1 -- start start 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 Processor -- start 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 0x55880ff96cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff97210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff9e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55880ff95330 con 0x55880ff84010 2024-06-02T23:27:34.773 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55880fe70ae0 con 0x55880ff84a00 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55880ff954b0 con 0x55880ff89b70 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07d2020740 10 client.0 _ll_register_callbacks cb 0x55880fea99b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 0x55880ff96cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 0x55880ff96cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59542/0 (socket says 172.21.15.52:59542) 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07ccfdd640 1 -- 172.21.15.52:0/3767609903 learned_addr learned my addr 172.21.15.52:0/3767609903 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07cd7de640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff9e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.768+0000 7f07c7fff640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff97210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07cd7de640 1 -- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 msgr2=0x55880ff97210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07cd7de640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff97210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07cd7de640 1 -- 172.21.15.52:0/3767609903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 msgr2=0x55880ff96cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07cd7de640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 0x55880ff96cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07cd7de640 1 -- 172.21.15.52:0/3767609903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880fbfb210 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 0x55880ff96cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07cd7de640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff9e250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07c000ffd0 tx=0x7f07c0004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b5986393792a404 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07a7fff640 1 -- 172.21.15.52:0/3767609903 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0024070 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07a7fff640 1 -- 172.21.15.52:0/3767609903 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f07c0029460 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55880fefaf10 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.769+0000 7f07a7fff640 1 -- 172.21.15.52:0/3767609903 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f07c0030070 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 10 client.28804 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55880fcc7620 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07a7fff640 1 -- 172.21.15.52:0/3767609903 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55880fefaf10 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07a7fff640 1 -- 172.21.15.52:0/3767609903 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f07c0034590 con 0x55880ff89b70 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 10 client.28804 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 10 client.28804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:34.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 20 client.28804 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55880ffa2c60 0x55880ffa50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 4 client.28804 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55880fef4c20 con 0x55880ffa2c60 2024-06-02T23:27:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.770+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55880ffa2c60 0x55880ffa50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.771+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55880ffa2c60 0x55880ffa50b0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=916ec475b88f4b2f server_cookie=6fdb10686eb9f2b0 in_seq=0 out_seq=0 2024-06-02T23:27:34.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.771+0000 7f07a7fff640 10 client.28804 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07a7fff640 1 -- 172.21.15.52:0/3767609903 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55880fef4c20 con 0x55880ffa2c60 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07a7fff640 10 client.28804 handle_command_reply: tid=0 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 client.28804 shutdown 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 msgr2=0x55880ff9e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff9e250 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f07c000ffd0 tx=0x7f07c0004ab0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 shutdown_connections 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55880ffa2c60 0x55880ffa50b0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff89b70 0x55880ff9e250 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff97210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 --2- 172.21.15.52:0/3767609903 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84010 0x55880ff96cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 >> 172.21.15.52:0/3767609903 conn(0x55880ff7f680 msgr2=0x55880ff7e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07a7fff640 0 client.28804 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 shutdown_connections 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.772+0000 7f07d2020740 1 -- 172.21.15.52:0/3767609903 wait complete. 2024-06-02T23:27:34.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.773+0000 7f07d2020740 20 client.28804 trim_cache size 0 max 16384 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 Processor -- start 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 -- start start 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff85e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff9e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff89b70 0x55880ff863a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a69b0 con 0x55880ff89b70 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a85b0 con 0x55880ff84a00 2024-06-02T23:27:34.806 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07d2020740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a7080 con 0x55880ff84010 2024-06-02T23:27:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07cd7de640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff89b70 0x55880ff863a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff85e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff9e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07ccfdd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff85e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54100/0 (socket says 172.21.15.52:54100) 2024-06-02T23:27:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.805+0000 7f07ccfdd640 1 -- 172.21.15.52:0/799455827 learned_addr learned my addr 172.21.15.52:0/799455827 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.806+0000 7f07ccfdd640 1 -- 172.21.15.52:0/799455827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 msgr2=0x55880ff9e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.806+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff9e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.806+0000 7f07ccfdd640 1 -- 172.21.15.52:0/799455827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff89b70 msgr2=0x55880ff863a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.806+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff89b70 0x55880ff863a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.806+0000 7f07ccfdd640 1 -- 172.21.15.52:0/799455827 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55880ff84770 con 0x55880ff84010 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.807+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff85e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f07bc004790 tx=0x7f07bc0971f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68868d44913096c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.807+0000 7f07b27fc640 1 -- 172.21.15.52:0/799455827 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc020040 con 0x55880ff84010 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.807+0000 7f07b27fc640 1 -- 172.21.15.52:0/799455827 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07bc0b9070 con 0x55880ff84010 2024-06-02T23:27:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.807+0000 7f07b27fc640 1 -- 172.21.15.52:0/799455827 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07bc0a3d50 con 0x55880ff84010 2024-06-02T23:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.808+0000 7f07d2020740 1 -- 172.21.15.52:0/799455827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 msgr2=0x55880ff85e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.808+0000 7f07d2020740 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff85e60 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f07bc004790 tx=0x7f07bc0971f0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 -- 172.21.15.52:0/799455827 shutdown_connections 2024-06-02T23:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff89b70 0x55880ff863a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff84a00 0x55880ff9e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 --2- 172.21.15.52:0/799455827 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff85e60 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 -- 172.21.15.52:0/799455827 >> 172.21.15.52:0/799455827 conn(0x55880ff7f680 msgr2=0x55880ff7fa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 -- 172.21.15.52:0/799455827 shutdown_connections 2024-06-02T23:27:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 -- 172.21.15.52:0/799455827 wait complete. 2024-06-02T23:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 -- start start 2024-06-02T23:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.809+0000 7f07d2020740 1 Processor -- start 2024-06-02T23:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff9dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff9e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff89b70 0x55880ff9a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a7e20 con 0x55880ff84a00 2024-06-02T23:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a7750 con 0x55880ff89b70 2024-06-02T23:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81a4990 con 0x55880ff84010 2024-06-02T23:27:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07d2020740 10 client.0 _ll_register_callbacks cb 0x7f07c000f880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff9e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff9e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:59558/0 (socket says 172.21.15.52:59558) 2024-06-02T23:27:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 -- 172.21.15.52:0/493537063 learned_addr learned my addr 172.21.15.52:0/493537063 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07cd7de640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff89b70 0x55880ff9a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff9dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 -- 172.21.15.52:0/493537063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 msgr2=0x55880ff9dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff9dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 -- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff89b70 msgr2=0x55880ff9a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff89b70 0x55880ff9a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07c7fff640 1 -- 172.21.15.52:0/493537063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55880ff9ac20 con 0x55880ff84a00 2024-06-02T23:27:34.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.810+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff9dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:27:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07c7fff640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff9e000 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f07b4032fd0 tx=0x7f07b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e09cbf8723552a8e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:34.817 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07a7fff640 1 -- 172.21.15.52:0/493537063 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4035040 con 0x55880ff84a00 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07a7fff640 1 -- 172.21.15.52:0/493537063 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f07b40037f0 con 0x55880ff84a00 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07bc0b9070 con 0x55880ff84a00 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07a7fff640 1 -- 172.21.15.52:0/493537063 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f07b40039f0 con 0x55880ff84a00 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07d2020740 10 client.19794 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.811+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x55881000c5d0 con 0x55880ff84a00 2024-06-02T23:27:34.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07a7fff640 1 -- 172.21.15.52:0/493537063 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f07bc0b9070 con 0x55880ff84a00 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07a7fff640 1 -- 172.21.15.52:0/493537063 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f07b4047c60 con 0x55880ff84a00 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07d2020740 10 client.19794 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07d2020740 10 client.19794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07d2020740 20 client.19794 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07d2020740 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55881000d0c0 0x55881000f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07d2020740 4 client.19794 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55880feeef70 con 0x55881000d0c0 2024-06-02T23:27:34.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.812+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55881000d0c0 0x55881000f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.813+0000 7f07a7fff640 10 client.19794 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:34.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.813+0000 7f07ccfdd640 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55881000d0c0 0x55881000f4e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1989373c9916cd5 server_cookie=f0ba9d3a1469e9d4 in_seq=0 out_seq=0 2024-06-02T23:27:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07a7fff640 1 -- 172.21.15.52:0/493537063 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55880feeef70 con 0x55881000d0c0 2024-06-02T23:27:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07a7fff640 10 client.19794 handle_command_reply: tid=0 2024-06-02T23:27:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07d2020740 1 client.19794 shutdown 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 msgr2=0x55880ff9e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07d2020740 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff9e000 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f07b4032fd0 tx=0x7f07b4004270 comp rx=0 tx=0).stop 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 shutdown_connections 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.846+0000 7f07d2020740 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55881000d0c0 0x55881000f4e0 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55880ff89b70 0x55880ff9a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55880ff84a00 0x55880ff9e000 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 1 --2- 172.21.15.52:0/493537063 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55880ff84010 0x55880ff9dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 >> 172.21.15.52:0/493537063 conn(0x55880ff7f680 msgr2=0x55880ff96c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07a7fff640 0 client.19794 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 shutdown_connections 2024-06-02T23:27:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 1 -- 172.21.15.52:0/493537063 wait complete. 2024-06-02T23:27:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.847+0000 7f07d2020740 20 client.19794 trim_cache size 0 max 16384 2024-06-02T23:27:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:27:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:27:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07b80a2c30 msgr2=0x7f07b80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07b80a2c30 0x7f07b80a50f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f07c00099e0 tx=0x7f07c000ff60 comp rx=0 tx=0).stop 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 msgr2=0x7f07c8199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8199db0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f07bc00d6a0 tx=0x7f07bc00db60 comp rx=0 tx=0).stop 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 shutdown_connections 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f07b80a2c30 0x7f07b80a50f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.851 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f07c81057e0 0x7f07c81a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f07c8104e20 0x7f07c819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 --2- 172.21.15.52:0/1826067302 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07c8103c30 0x7f07c8199db0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 >> 172.21.15.52:0/1826067302 conn(0x7f07c80ff1c0 msgr2=0x7f07c8108120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 shutdown_connections 2024-06-02T23:27:34.852 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:34.848+0000 7f07cf241640 1 -- 172.21.15.52:0/1826067302 wait complete. 2024-06-02T23:27:35.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 14602 2.59 MB/sec execute 43 sec latency 57.661 ms 2024-06-02T23:27:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:35 smithi123 ceph-mon[35977]: pgmap v902: 97 pgs: 97 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 653 KiB/s wr, 128 op/s 2024-06-02T23:27:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:35 smithi174 ceph-mon[35994]: pgmap v902: 97 pgs: 97 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 653 KiB/s wr, 128 op/s 2024-06-02T23:27:36.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:35 smithi052 ceph-mon[32340]: pgmap v902: 97 pgs: 97 active+clean; 346 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 653 KiB/s wr, 128 op/s 2024-06-02T23:27:36.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 14765 2.70 MB/sec execute 44 sec latency 68.751 ms 2024-06-02T23:27:37.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 14938 2.77 MB/sec execute 45 sec latency 59.506 ms 2024-06-02T23:27:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:37 smithi123 ceph-mon[35977]: pgmap v903: 97 pgs: 97 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-06-02T23:27:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:37 smithi174 ceph-mon[35994]: pgmap v903: 97 pgs: 97 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-06-02T23:27:38.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:37 smithi052 ceph-mon[32340]: pgmap v903: 97 pgs: 97 active+clean; 350 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 618 KiB/s rd, 1.8 MiB/s wr, 147 op/s 2024-06-02T23:27:38.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 15124 2.87 MB/sec execute 46 sec latency 58.188 ms 2024-06-02T23:27:39.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 15282 2.98 MB/sec execute 47 sec latency 61.840 ms 2024-06-02T23:27:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:39 smithi123 ceph-mon[35977]: pgmap v904: 97 pgs: 97 active+clean; 350 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-06-02T23:27:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:39 smithi174 ceph-mon[35994]: pgmap v904: 97 pgs: 97 active+clean; 350 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-06-02T23:27:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:39 smithi052 ceph-mon[32340]: pgmap v904: 97 pgs: 97 active+clean; 350 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-06-02T23:27:40.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 15463 3.19 MB/sec execute 48 sec latency 56.298 ms 2024-06-02T23:27:41.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 15675 3.48 MB/sec execute 49 sec latency 60.548 ms 2024-06-02T23:27:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:41 smithi123 ceph-mon[35977]: pgmap v905: 97 pgs: 97 active+clean; 348 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-06-02T23:27:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:41 smithi174 ceph-mon[35994]: pgmap v905: 97 pgs: 97 active+clean; 348 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-06-02T23:27:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:41 smithi052 ceph-mon[32340]: pgmap v905: 97 pgs: 97 active+clean; 348 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 991 KiB/s rd, 2.9 MiB/s wr, 206 op/s 2024-06-02T23:27:42.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 15821 3.44 MB/sec execute 50 sec latency 138.557 ms 2024-06-02T23:27:43.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 15996 3.45 MB/sec execute 51 sec latency 58.542 ms 2024-06-02T23:27:44.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:43 smithi123 ceph-mon[35977]: pgmap v906: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-06-02T23:27:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:43 smithi174 ceph-mon[35994]: pgmap v906: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-06-02T23:27:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:43 smithi052 ceph-mon[32340]: pgmap v906: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 243 op/s 2024-06-02T23:27:44.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 16184 3.53 MB/sec execute 52 sec latency 61.128 ms 2024-06-02T23:27:45.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 16336 3.57 MB/sec execute 53 sec latency 102.470 ms 2024-06-02T23:27:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:45 smithi123 ceph-mon[35977]: pgmap v907: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-06-02T23:27:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:45 smithi174 ceph-mon[35994]: pgmap v907: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-06-02T23:27:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:45 smithi052 ceph-mon[32340]: pgmap v907: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 283 op/s 2024-06-02T23:27:46.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 16508 3.65 MB/sec execute 54 sec latency 55.861 ms 2024-06-02T23:27:47.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 16690 3.85 MB/sec execute 55 sec latency 65.095 ms 2024-06-02T23:27:47.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 -- 172.21.15.52:0/3317958257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101570 msgr2=0x7f4f30108e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3317958257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101570 0x7f4f30108e10 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009cd0 tx=0x7f4f2802a8e0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 -- 172.21.15.52:0/3317958257 shutdown_connections 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3317958257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101570 0x7f4f30108e10 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3317958257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30100bb0 0x7f4f30101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3317958257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30101f10 0x7f4f30102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 -- 172.21.15.52:0/3317958257 >> 172.21.15.52:0/3317958257 conn(0x7f4f300fb640 msgr2=0x7f4f300fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 -- 172.21.15.52:0/3317958257 shutdown_connections 2024-06-02T23:27:47.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.983+0000 7f4f36c12640 1 -- 172.21.15.52:0/3317958257 wait complete. 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 Processor -- start 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 -- start start 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30100bb0 0x7f4f30199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30101570 0x7f4f3019a280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101f10 0x7f4f301a12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f351af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101f10 0x7f4f301a12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f349ae640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30100bb0 0x7f4f30199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f351af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101f10 0x7f4f301a12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53120/0 (socket says 172.21.15.52:53120) 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f351af640 1 -- 172.21.15.52:0/3372293587 learned_addr learned my addr 172.21.15.52:0/3372293587 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:47.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3010b3d0 con 0x7f4f30101f10 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3010b250 con 0x7f4f30101570 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f3010b550 con 0x7f4f30100bb0 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.984+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30101570 0x7f4f3019a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3372293587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30100bb0 msgr2=0x7f4f30199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30100bb0 0x7f4f30199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101f10 msgr2=0x7f4f301a12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101f10 0x7f4f301a12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3372293587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301a1960 con 0x7f4f30101570 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30101570 0x7f4f3019a280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009740 tx=0x7f4f1c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2dffb640 1 -- 172.21.15.52:0/3372293587 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c009e50 con 0x7f4f30101570 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301a1c50 con 0x7f4f30101570 2024-06-02T23:27:47.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2dffb640 1 -- 172.21.15.52:0/3372293587 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f1c005020 con 0x7f4f30101570 2024-06-02T23:27:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f2dffb640 1 -- 172.21.15.52:0/3372293587 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c004c70 con 0x7f4f30101570 2024-06-02T23:27:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.985+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301a21a0 con 0x7f4f30101570 2024-06-02T23:27:47.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.988+0000 7f4f2dffb640 1 -- 172.21.15.52:0/3372293587 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f1c01d020 con 0x7f4f30101570 2024-06-02T23:27:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.989+0000 7f4f399fb740 1 Processor -- start 2024-06-02T23:27:47.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.989+0000 7f4f399fb740 1 -- start start 2024-06-02T23:27:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.989+0000 7f4f2dffb640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f200a2dd0 0x7f4f200a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x558349885430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 0x55834988a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834988ab70 0x55834988d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f399fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5583497e84f0 con 0x558349885010 2024-06-02T23:27:47.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f399fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558349786870 con 0x558349885a00 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f399fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5583497e44b0 con 0x55834988ab70 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f351af640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f200a2dd0 0x7f4f200a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 0x55834988a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 0x55834988a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53726/0 (socket says 172.21.15.52:53726) 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 -- 172.21.15.52:0/1140057361 learned_addr learned my addr 172.21.15.52:0/1140057361 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f351af640 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834988ab70 0x55834988d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f349ae640 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x558349885430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2dffb640 1 -- 172.21.15.52:0/3372293587 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f1c0ce430 con 0x7f4f30101570 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 -- 172.21.15.52:0/1140057361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834988ab70 msgr2=0x55834988d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834988ab70 0x55834988d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 -- 172.21.15.52:0/1140057361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 msgr2=0x558349885430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x558349885430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.990+0000 7f4f2ffff640 1 -- 172.21.15.52:0/1140057361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583497a29b0 con 0x558349885a00 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.991+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 0x55834988a500 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0a33a0 tx=0x7f4f1c005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eca958221110a4b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.991+0000 7f4f351af640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f200a2dd0 0x7f4f200a5290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009a60 tx=0x7f4f280098b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.991+0000 7f4f1affd640 1 -- 172.21.15.52:0/1140057361 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0b3e60 con 0x558349885a00 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.991+0000 7f4f1affd640 1 -- 172.21.15.52:0/1140057361 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f1c0a38d0 con 0x558349885a00 2024-06-02T23:27:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.991+0000 7f4f1affd640 1 -- 172.21.15.52:0/1140057361 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0bc080 con 0x558349885a00 2024-06-02T23:27:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 -- 172.21.15.52:0/1140057361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 msgr2=0x55834988a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 0x55834988a500 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0a33a0 tx=0x7f4f1c005750 comp rx=0 tx=0).stop 2024-06-02T23:27:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 -- 172.21.15.52:0/1140057361 shutdown_connections 2024-06-02T23:27:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55834988ab70 0x55834988d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885a00 0x55834988a500 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1140057361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x558349885430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 -- 172.21.15.52:0/1140057361 >> 172.21.15.52:0/1140057361 conn(0x558349880680 msgr2=0x558349880a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 -- 172.21.15.52:0/1140057361 shutdown_connections 2024-06-02T23:27:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.992+0000 7f4f399fb740 1 -- 172.21.15.52:0/1140057361 wait complete. 2024-06-02T23:27:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.993+0000 7f4f399fb740 1 -- start start 2024-06-02T23:27:47.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.993+0000 7f4f399fb740 1 Processor -- start 2024-06-02T23:27:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.993+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x55834988f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.993+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x55834988f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.993+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349896790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f399fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55834988dac0 con 0x558349885010 2024-06-02T23:27:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f399fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558349771ae0 con 0x55834988ab70 2024-06-02T23:27:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f399fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55834988dc40 con 0x558349885a00 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x55834988f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f399fb740 10 client.0 _ll_register_callbacks cb 0x5583497aa9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x55834988f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54760/0 (socket says 172.21.15.52:54760) 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 -- 172.21.15.52:0/97908890 learned_addr learned my addr 172.21.15.52:0/97908890 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f351af640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349896790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f349ae640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x55834988f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 -- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 msgr2=0x558349896790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349896790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 -- 172.21.15.52:0/97908890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 msgr2=0x55834988f280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x55834988f280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 -- 172.21.15.52:0/97908890 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583494fc210 con 0x558349885a00 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x55834988f7c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c09a960 tx=0x7f4f1c096860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=701d3993dceb1c07 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:27:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.994+0000 7f4f0ffff640 1 -- 172.21.15.52:0/97908890 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0bf070 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f0ffff640 1 -- 172.21.15.52:0/97908890 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f1c0a3440 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c0a38d0 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f0ffff640 1 -- 172.21.15.52:0/97908890 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4f1c0b7050 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 10 client.28822 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x5583495c8620 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f0ffff640 1 -- 172.21.15.52:0/97908890 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f1c0a38d0 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f0ffff640 1 -- 172.21.15.52:0/97908890 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f4f1c0bfea0 con 0x558349885a00 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 10 client.28822 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 10 client.28822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 20 client.28822 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55834989ac80 0x55834989d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 4 client.28822 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.995+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583497fbf10 con 0x55834989ac80 2024-06-02T23:27:47.997 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.996+0000 7f4f349ae640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55834989ac80 0x55834989d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.996+0000 7f4f0ffff640 10 client.28822 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.996+0000 7f4f349ae640 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55834989ac80 0x55834989d070 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=989264cf83e9ba86 server_cookie=de1fda9d82173e5c in_seq=0 out_seq=0 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f0ffff640 1 -- 172.21.15.52:0/97908890 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5583497fbf10 con 0x55834989ac80 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f0ffff640 10 client.28822 handle_command_reply: tid=0 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 client.28822 shutdown 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 msgr2=0x55834988f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x55834988f7c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c09a960 tx=0x7f4f1c096860 comp rx=0 tx=0).stop 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 shutdown_connections 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55834989ac80 0x55834989d070 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x55834988f7c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349896790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f0ffff640 0 client.28822 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 --2- 172.21.15.52:0/97908890 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885010 0x55834988f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 >> 172.21.15.52:0/97908890 conn(0x558349880680 msgr2=0x55834987f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 shutdown_connections 2024-06-02T23:27:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.997+0000 7f4f399fb740 1 -- 172.21.15.52:0/97908890 wait complete. 2024-06-02T23:27:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:47.998+0000 7f4f399fb740 20 client.28822 trim_cache size 0 max 16384 2024-06-02T23:27:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:47 smithi123 ceph-mon[35977]: pgmap v908: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-06-02T23:27:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 Processor -- start 2024-06-02T23:27:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 -- start start 2024-06-02T23:27:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885010 0x558349883560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 0x558349896750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349883aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301aaeb0 con 0x558349885a00 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301acab0 con 0x55834988ab70 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f399fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301ab580 con 0x558349885010 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 0x558349896750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f2ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 0x558349896750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53154/0 (socket says 172.21.15.52:53154) 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3904457813 learned_addr learned my addr 172.21.15.52:0/3904457813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f351af640 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349883aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.023+0000 7f4f349ae640 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885010 0x558349883560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3904457813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885010 msgr2=0x558349883560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885010 0x558349883560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3904457813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 msgr2=0x558349883aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349883aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f2ffff640 1 -- 172.21.15.52:0/3904457813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55834989d6f0 con 0x558349885a00 2024-06-02T23:27:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f351af640 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349883aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:27:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 0x558349896750 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c09abf0 tx=0x7f4f1c0053e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f28c29edd76d35 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f1a7fc640 1 -- 172.21.15.52:0/3904457813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0bf040 con 0x558349885a00 2024-06-02T23:27:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f1a7fc640 1 -- 172.21.15.52:0/3904457813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f1c0b0070 con 0x558349885a00 2024-06-02T23:27:48.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.024+0000 7f4f1a7fc640 1 -- 172.21.15.52:0/3904457813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0b3e60 con 0x558349885a00 2024-06-02T23:27:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 -- 172.21.15.52:0/3904457813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 msgr2=0x558349896750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 0x558349896750 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c09abf0 tx=0x7f4f1c0053e0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 -- 172.21.15.52:0/3904457813 shutdown_connections 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55834988ab70 0x558349883aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558349885a00 0x558349896750 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 --2- 172.21.15.52:0/3904457813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885010 0x558349883560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 -- 172.21.15.52:0/3904457813 >> 172.21.15.52:0/3904457813 conn(0x558349880680 msgr2=0x558349886c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 -- 172.21.15.52:0/3904457813 shutdown_connections 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 -- 172.21.15.52:0/3904457813 wait complete. 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.025+0000 7f4f399fb740 1 -- start start 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 Processor -- start 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 0x558349893540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x558349893a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55834988ab70 0x558349903940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f349ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 0x558349893540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301ac320 con 0x55834988ab70 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301abc50 con 0x558349885010 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f351af640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55834988ab70 0x558349903940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f349ae640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 0x558349893540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53734/0 (socket says 172.21.15.52:53734) 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301a8e90 con 0x558349885a00 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f349ae640 1 -- 172.21.15.52:0/1501294310 learned_addr learned my addr 172.21.15.52:0/1501294310 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x558349893a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.026+0000 7f4f399fb740 10 client.0 _ll_register_callbacks cb 0x7f4f240097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f349ae640 1 -- 172.21.15.52:0/1501294310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 msgr2=0x558349893a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f349ae640 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x558349893a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f349ae640 1 -- 172.21.15.52:0/1501294310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55834988ab70 msgr2=0x558349903940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f349ae640 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55834988ab70 0x558349903940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f349ae640 1 -- 172.21.15.52:0/1501294310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583498804b0 con 0x558349885010 2024-06-02T23:27:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f349ae640 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 0x558349893540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4f24000c00 tx=0x7f4f2401f9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc05a381dc50aeb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f0ffff640 1 -- 172.21.15.52:0/1501294310 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24004220 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f0ffff640 1 -- 172.21.15.52:0/1501294310 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f240043c0 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f1c0b0070 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f0ffff640 1 -- 172.21.15.52:0/1501294310 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) ==== 24+0+0 (secure 0 0 0) 0x7f4f2402d070 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f399fb740 10 client.29156 fetch_fsmap learned FSMap version 40 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.027+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) -- 0x558349904f30 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f0ffff640 1 -- 172.21.15.52:0/1501294310 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f2401fcd0 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f0ffff640 1 -- 172.21.15.52:0/1501294310 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 40) ==== 8370+0+0 (secure 0 0 0) 0x7f4f24003c30 con 0x558349885010 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f399fb740 10 client.29156 fetch_fsmap finished waiting for FSMap version 40 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f399fb740 10 client.29156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f399fb740 20 client.29156 populate_metadata read hostname 'smithi052' 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558349905950 0x558349907d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:27:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f399fb740 4 client.29156 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:27:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f4f1c0b0070 con 0x558349905950 2024-06-02T23:27:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.028+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558349905950 0x558349907d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:27:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.029+0000 7f4f0ffff640 10 client.29156 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.029+0000 7f4f2ffff640 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558349905950 0x558349907d70 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f5547cef26779b1 server_cookie=91e5a3332ee6a6f0 in_seq=0 out_seq=0 2024-06-02T23:27:48.065 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f0ffff640 1 -- 172.21.15.52:0/1501294310 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f4f1c0b0070 con 0x558349905950 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f0ffff640 10 client.29156 handle_command_reply: tid=0 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 client.29156 shutdown 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 msgr2=0x558349893540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 0x558349893540 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f4f24000c00 tx=0x7f4f2401f9c0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 shutdown_connections 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558349905950 0x558349907d70 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558349885a00 0x558349893a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f0ffff640 0 client.29156 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55834988ab70 0x558349903940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 --2- 172.21.15.52:0/1501294310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558349885010 0x558349893540 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 >> 172.21.15.52:0/1501294310 conn(0x558349880680 msgr2=0x558349883560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:48.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 shutdown_connections 2024-06-02T23:27:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 1 -- 172.21.15.52:0/1501294310 wait complete. 2024-06-02T23:27:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.066+0000 7f4f399fb740 20 client.29156 trim_cache size 0 max 16384 2024-06-02T23:27:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:27:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:27:48.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f200a2dd0 msgr2=0x7f4f200a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f200a2dd0 0x7f4f200a5290 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4f28009a60 tx=0x7f4f280098b0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30101570 msgr2=0x7f4f3019a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30101570 0x7f4f3019a280 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009740 tx=0x7f4f1c009c00 comp rx=0 tx=0).stop 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 shutdown_connections 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f200a2dd0 0x7f4f200a5290 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30101f10 0x7f4f301a12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30101570 0x7f4f3019a280 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 --2- 172.21.15.52:0/3372293587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30100bb0 0x7f4f30199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 >> 172.21.15.52:0/3372293587 conn(0x7f4f300fb640 msgr2=0x7f4f30104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 shutdown_connections 2024-06-02T23:27:48.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:27:48.067+0000 7f4f36c12640 1 -- 172.21.15.52:0/3372293587 wait complete. 2024-06-02T23:27:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:47 smithi174 ceph-mon[35994]: pgmap v908: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-06-02T23:27:48.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:47 smithi052 ceph-mon[32340]: pgmap v908: 97 pgs: 97 active+clean; 345 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 286 op/s 2024-06-02T23:27:48.456 INFO:tasks.workunit.client.0.smithi052.stdout: 10 16909 3.95 MB/sec execute 56 sec latency 76.133 ms 2024-06-02T23:27:49.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 17133 4.25 MB/sec execute 57 sec latency 53.962 ms 2024-06-02T23:27:49.951 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:49 smithi052 ceph-mon[32340]: pgmap v909: 97 pgs: 97 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-06-02T23:27:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:49 smithi123 ceph-mon[35977]: pgmap v909: 97 pgs: 97 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-06-02T23:27:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:49 smithi174 ceph-mon[35994]: pgmap v909: 97 pgs: 97 active+clean; 347 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 292 op/s 2024-06-02T23:27:50.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 17322 4.35 MB/sec execute 58 sec latency 82.894 ms 2024-06-02T23:27:51.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 17468 4.50 MB/sec execute 59 sec latency 57.924 ms 2024-06-02T23:27:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:51 smithi052 ceph-mon[32340]: pgmap v910: 97 pgs: 97 active+clean; 356 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 309 op/s 2024-06-02T23:27:51.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:51 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:51 smithi123 ceph-mon[35977]: pgmap v910: 97 pgs: 97 active+clean; 356 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 309 op/s 2024-06-02T23:27:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:51 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:51 smithi174 ceph-mon[35994]: pgmap v910: 97 pgs: 97 active+clean; 356 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 309 op/s 2024-06-02T23:27:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:51 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:27:52.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 17598 4.54 MB/sec execute 60 sec latency 63.008 ms 2024-06-02T23:27:53.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 17800 4.77 MB/sec execute 61 sec latency 54.293 ms 2024-06-02T23:27:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:53 smithi123 ceph-mon[35977]: pgmap v911: 97 pgs: 97 active+clean; 360 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 278 op/s 2024-06-02T23:27:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:53 smithi174 ceph-mon[35994]: pgmap v911: 97 pgs: 97 active+clean; 360 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 278 op/s 2024-06-02T23:27:54.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:53 smithi052 ceph-mon[32340]: pgmap v911: 97 pgs: 97 active+clean; 360 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.3 MiB/s wr, 278 op/s 2024-06-02T23:27:54.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 17964 4.79 MB/sec execute 62 sec latency 52.301 ms 2024-06-02T23:27:55.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 18100 4.76 MB/sec execute 63 sec latency 50.494 ms 2024-06-02T23:27:56.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:55 smithi123 ceph-mon[35977]: pgmap v912: 97 pgs: 97 active+clean; 363 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-06-02T23:27:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:55 smithi174 ceph-mon[35994]: pgmap v912: 97 pgs: 97 active+clean; 363 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-06-02T23:27:56.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:55 smithi052 ceph-mon[32340]: pgmap v912: 97 pgs: 97 active+clean; 363 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 272 op/s 2024-06-02T23:27:56.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 18245 4.73 MB/sec execute 64 sec latency 110.791 ms 2024-06-02T23:27:57.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 18404 4.75 MB/sec execute 65 sec latency 73.480 ms 2024-06-02T23:27:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:57 smithi123 ceph-mon[35977]: pgmap v913: 97 pgs: 97 active+clean; 366 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 295 op/s 2024-06-02T23:27:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:57 smithi174 ceph-mon[35994]: pgmap v913: 97 pgs: 97 active+clean; 366 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 295 op/s 2024-06-02T23:27:58.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:57 smithi052 ceph-mon[32340]: pgmap v913: 97 pgs: 97 active+clean; 366 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 295 op/s 2024-06-02T23:27:58.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 18581 4.79 MB/sec execute 66 sec latency 64.047 ms 2024-06-02T23:27:58.948 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:27:58.986 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:58 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:27:59.006 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:58 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:27:59.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:58 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:27:59.335 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:27:59.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 18751 4.91 MB/sec execute 67 sec latency 69.579 ms 2024-06-02T23:27:59.721 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:27:59 smithi052 ceph-mon[32340]: pgmap v914: 97 pgs: 97 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-06-02T23:28:00.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:27:59 smithi123 ceph-mon[35977]: pgmap v914: 97 pgs: 97 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-06-02T23:28:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:27:59 smithi174 ceph-mon[35994]: pgmap v914: 97 pgs: 97 active+clean; 368 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 266 op/s 2024-06-02T23:28:00.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 18944 5.04 MB/sec execute 68 sec latency 66.715 ms 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 -- 172.21.15.52:0/1734749610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c1024b0 msgr2=0x7f898c069050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 --2- 172.21.15.52:0/1734749610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c1024b0 0x7f898c069050 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f8984009ca0 tx=0x7f898402a8c0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 -- 172.21.15.52:0/1734749610 shutdown_connections 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 --2- 172.21.15.52:0/1734749610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c101af0 0x7f898c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 --2- 172.21.15.52:0/1734749610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c1024b0 0x7f898c069050 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 --2- 172.21.15.52:0/1734749610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c100900 0x7f898c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 -- 172.21.15.52:0/1734749610 >> 172.21.15.52:0/1734749610 conn(0x7f898c0fbe70 msgr2=0x7f898c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 -- 172.21.15.52:0/1734749610 shutdown_connections 2024-06-02T23:28:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.197+0000 7f8990d90640 1 -- 172.21.15.52:0/1734749610 wait complete. 2024-06-02T23:28:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.198+0000 7f8990d90640 1 Processor -- start 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.198+0000 7f8990d90640 1 -- start start 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8990d90640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 0x7f898c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8990d90640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c101af0 0x7f898c1980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8990d90640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c1024b0 0x7f898c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8990d90640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f898c10d540 con 0x7f898c100900 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8990d90640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f898c10d3c0 con 0x7f898c101af0 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8990d90640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f898c10d6c0 con 0x7f898c1024b0 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 0x7f898c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 0x7f898c197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58378/0 (socket says 172.21.15.52:58378) 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 -- 172.21.15.52:0/1412353619 learned_addr learned my addr 172.21.15.52:0/1412353619 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8989d9b640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c101af0 0x7f898c1980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898ad9d640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c1024b0 0x7f898c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 -- 172.21.15.52:0/1412353619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c1024b0 msgr2=0x7f898c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c1024b0 0x7f898c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 -- 172.21.15.52:0/1412353619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c101af0 msgr2=0x7f898c1980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c101af0 0x7f898c1980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f898a59c640 1 -- 172.21.15.52:0/1412353619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f898c19f7b0 con 0x7f898c100900 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.199+0000 7f8989d9b640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c101af0 0x7f898c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.200+0000 7f898a59c640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 0x7f898c197b90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e9c0 tx=0x7f897c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.200+0000 7f897b7fe640 1 -- 172.21.15.52:0/1412353619 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c00cd80 con 0x7f898c100900 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.200+0000 7f897b7fe640 1 -- 172.21.15.52:0/1412353619 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c0035f0 con 0x7f898c100900 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.200+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f898c19faa0 con 0x7f898c100900 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.200+0000 7f897b7fe640 1 -- 172.21.15.52:0/1412353619 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c010690 con 0x7f898c100900 2024-06-02T23:28:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.200+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f898c1a0040 con 0x7f898c100900 2024-06-02T23:28:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.203+0000 7f897b7fe640 1 -- 172.21.15.52:0/1412353619 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f897c0037a0 con 0x7f898c100900 2024-06-02T23:28:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.204+0000 7f8993b7f740 1 Processor -- start 2024-06-02T23:28:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.204+0000 7f897b7fe640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89740a2d00 0x7f89740a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f897b7fe640 1 -- 172.21.15.52:0/1412353619 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f897c0cf3f0 con 0x7f898c100900 2024-06-02T23:28:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f898ad9d640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89740a2d00 0x7f89740a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f8993b7f740 1 -- start start 2024-06-02T23:28:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x5638690223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f898a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x5638690223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f898a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x5638690223b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51120/0 (socket says 172.21.15.52:51120) 2024-06-02T23:28:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.205+0000 7f898ad9d640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89740a2d00 0x7f89740a51c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8984031aa0 tx=0x7f89840099a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x563869025470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898a59c640 1 -- 172.21.15.52:0/2483631140 learned_addr learned my addr 172.21.15.52:0/2483631140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f8993b7f740 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869025ae0 0x563869032930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563868f854f0 con 0x563869022980 2024-06-02T23:28:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563868f23870 con 0x563869021f90 2024-06-02T23:28:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563868f814b0 con 0x563869025ae0 2024-06-02T23:28:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f8989d9b640 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x563869025470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898ad9d640 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869025ae0 0x563869032930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898a59c640 1 -- 172.21.15.52:0/2483631140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869025ae0 msgr2=0x563869032930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898a59c640 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869025ae0 0x563869032930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898a59c640 1 -- 172.21.15.52:0/2483631140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 msgr2=0x563869025470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898a59c640 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x563869025470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.206+0000 7f898a59c640 1 -- 172.21.15.52:0/2483631140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563868f3f9b0 con 0x563869021f90 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.207+0000 7f8989d9b640 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x563869025470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.207+0000 7f898a59c640 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x5638690223b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f897c0980e0 tx=0x7f897c09b880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=885a7c8837e3eeb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.207+0000 7f8978ff9640 1 -- 172.21.15.52:0/2483631140 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c09bc80 con 0x563869021f90 2024-06-02T23:28:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.207+0000 7f8978ff9640 1 -- 172.21.15.52:0/2483631140 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c0a0030 con 0x563869021f90 2024-06-02T23:28:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.207+0000 7f8978ff9640 1 -- 172.21.15.52:0/2483631140 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c0a46c0 con 0x563869021f90 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 msgr2=0x5638690223b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x5638690223b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f897c0980e0 tx=0x7f897c09b880 comp rx=0 tx=0).stop 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 shutdown_connections 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869025ae0 0x563869032930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x563869025470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 --2- 172.21.15.52:0/2483631140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x5638690223b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 >> 172.21.15.52:0/2483631140 conn(0x56386901d680 msgr2=0x56386901da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 shutdown_connections 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.208+0000 7f8993b7f740 1 -- 172.21.15.52:0/2483631140 wait complete. 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 -- start start 2024-06-02T23:28:01.211 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 Processor -- start 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x563869034b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869022980 0x563869035060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5638690332d0 con 0x563869025ae0 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563868f0eae0 con 0x563869021f90 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563869033450 con 0x563869022980 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8993b7f740 10 client.0 _ll_register_callbacks cb 0x563868f479b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f898a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x563869034b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8989d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869022980 0x563869035060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f898a59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x563869034b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51132/0 (socket says 172.21.15.52:51132) 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f8989d9b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869022980 0x563869035060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:43994/0 (socket says 172.21.15.52:43994) 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.209+0000 7f898a59c640 1 -- 172.21.15.52:0/375472840 learned_addr learned my addr 172.21.15.52:0/375472840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.212 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898ad9d640 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898a59c640 1 -- 172.21.15.52:0/375472840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869022980 msgr2=0x563869035060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898a59c640 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869022980 0x563869035060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898a59c640 1 -- 172.21.15.52:0/375472840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 msgr2=0x56386903c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898a59c640 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898a59c640 1 -- 172.21.15.52:0/375472840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563868c99210 con 0x563869021f90 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f898a59c640 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x563869034b20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f897c000c00 tx=0x7f897c0046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fe03082efa37e67 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8971ffb640 1 -- 172.21.15.52:0/375472840 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c07e020 con 0x563869021f90 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8971ffb640 1 -- 172.21.15.52:0/375472840 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f897c09b2d0 con 0x563869021f90 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897c0a0030 con 0x563869021f90 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8971ffb640 1 -- 172.21.15.52:0/375472840 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f897c0a9050 con 0x563869021f90 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8993b7f740 10 client.29168 fetch_fsmap learned FSMap version 42 2024-06-02T23:28:01.213 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.210+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x563868d65620 con 0x563869021f90 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8971ffb640 1 -- 172.21.15.52:0/375472840 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c0a0030 con 0x563869021f90 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8971ffb640 1 -- 172.21.15.52:0/375472840 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f897c0979d0 con 0x563869021f90 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8993b7f740 10 client.29168 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8993b7f740 10 client.29168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8993b7f740 20 client.29168 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8993b7f740 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563869040590 0x563869042980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8993b7f740 4 client.29168 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563868f98f10 con 0x563869040590 2024-06-02T23:28:01.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.211+0000 7f8989d9b640 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563869040590 0x563869042980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.212+0000 7f8971ffb640 10 client.29168 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.212+0000 7f8989d9b640 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563869040590 0x563869042980 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=565b89f64e1063ef server_cookie=3f830cedeb804b3e in_seq=0 out_seq=0 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8971ffb640 1 -- 172.21.15.52:0/375472840 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563868f98f10 con 0x563869040590 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8971ffb640 10 client.29168 handle_command_reply: tid=0 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 client.29168 shutdown 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 msgr2=0x563869034b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x563869034b20 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f897c000c00 tx=0x7f897c0046c0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 shutdown_connections 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563869040590 0x563869042980 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869022980 0x563869035060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8971ffb640 0 client.29168 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 --2- 172.21.15.52:0/375472840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869021f90 0x563869034b20 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 >> 172.21.15.52:0/375472840 conn(0x56386901d680 msgr2=0x56386901c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 shutdown_connections 2024-06-02T23:28:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 1 -- 172.21.15.52:0/375472840 wait complete. 2024-06-02T23:28:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.213+0000 7f8993b7f740 20 client.29168 trim_cache size 0 max 16384 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.237+0000 7f8993b7f740 1 Processor -- start 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.237+0000 7f8993b7f740 1 -- start start 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.237+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869034730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x56386903c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869025ae0 0x563869034c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8993b7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1a6b30 con 0x563869022980 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8993b7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1a86a0 con 0x563869025ae0 2024-06-02T23:28:01.238 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8993b7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1a7200 con 0x563869021f90 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x56386903c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f898a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869034730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x56386903c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58398/0 (socket says 172.21.15.52:58398) 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 -- 172.21.15.52:0/3195624984 learned_addr learned my addr 172.21.15.52:0/3195624984 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f898ad9d640 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869025ae0 0x563869034c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 -- 172.21.15.52:0/3195624984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 msgr2=0x563869034730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869034730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 -- 172.21.15.52:0/3195624984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869025ae0 msgr2=0x563869034c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869025ae0 0x563869034c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 -- 172.21.15.52:0/3195624984 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56386901d4b0 con 0x563869022980 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.238+0000 7f8989d9b640 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x56386903c060 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x5638690233f0 tx=0x7f8980005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bd0d32ba696158 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.239+0000 7f8973fff640 1 -- 172.21.15.52:0/3195624984 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980012e40 con 0x563869022980 2024-06-02T23:28:01.239 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.239+0000 7f8973fff640 1 -- 172.21.15.52:0/3195624984 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8980005020 con 0x563869022980 2024-06-02T23:28:01.240 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.239+0000 7f8973fff640 1 -- 172.21.15.52:0/3195624984 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980011bf0 con 0x563869022980 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.239+0000 7f8993b7f740 1 -- 172.21.15.52:0/3195624984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 msgr2=0x56386903c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.239+0000 7f8993b7f740 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x56386903c060 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x5638690233f0 tx=0x7f8980005f90 comp rx=0 tx=0).stop 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 -- 172.21.15.52:0/3195624984 shutdown_connections 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869025ae0 0x563869034c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869022980 0x56386903c060 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 --2- 172.21.15.52:0/3195624984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869034730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 -- 172.21.15.52:0/3195624984 >> 172.21.15.52:0/3195624984 conn(0x56386901d680 msgr2=0x563869020670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 -- 172.21.15.52:0/3195624984 shutdown_connections 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 -- 172.21.15.52:0/3195624984 wait complete. 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 -- start start 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 Processor -- start 2024-06-02T23:28:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869039440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869022980 0x563869039980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.240+0000 7f8993b7f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f8993b7f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1a7fa0 con 0x563869025ae0 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f8993b7f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1a78d0 con 0x563869022980 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f8993b7f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f898c1a4b10 con 0x563869021f90 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869039440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869039440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44018/0 (socket says 172.21.15.52:44018) 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 -- 172.21.15.52:0/678042346 learned_addr learned my addr 172.21.15.52:0/678042346 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f8989d9b640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869022980 0x563869039980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f8993b7f740 10 client.0 _ll_register_callbacks cb 0x7f8980009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898ad9d640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 -- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869022980 msgr2=0x563869039980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869022980 0x563869039980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 -- 172.21.15.52:0/678042346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 msgr2=0x56386903a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 -- 172.21.15.52:0/678042346 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56386903af80 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f898a59c640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869039440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f897c097ec0 tx=0x7f897c07ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bdb43f16126e898 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.241+0000 7f8971ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8971ffb640 1 -- 172.21.15.52:0/678042346 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c07e070 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8971ffb640 1 -- 172.21.15.52:0/678042346 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f897c09b2d0 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8980005020 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8971ffb640 1 -- 172.21.15.52:0/678042346 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f897c08c020 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 10 client.28846 fetch_fsmap learned FSMap version 42 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x56386903af80 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8971ffb640 1 -- 172.21.15.52:0/678042346 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8980005020 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8971ffb640 1 -- 172.21.15.52:0/678042346 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f897c0ac070 con 0x563869021f90 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 10 client.28846 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T23:28:01.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 10 client.28846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 20 client.28846 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5638690ab3c0 0x5638690ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 4 client.28846 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.242+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563868f92c20 con 0x5638690ab3c0 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.243+0000 7f8989d9b640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5638690ab3c0 0x5638690ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.243+0000 7f8971ffb640 10 client.28846 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.243+0000 7f8989d9b640 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5638690ab3c0 0x5638690ad7b0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2f412019fc7c447 server_cookie=319cf769a9fb372f in_seq=0 out_seq=0 2024-06-02T23:28:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8971ffb640 1 -- 172.21.15.52:0/678042346 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563868f92c20 con 0x5638690ab3c0 2024-06-02T23:28:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8971ffb640 10 client.28846 handle_command_reply: tid=0 2024-06-02T23:28:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 client.28846 shutdown 2024-06-02T23:28:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 msgr2=0x563869039440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869039440 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f897c097ec0 tx=0x7f897c07ad20 comp rx=0 tx=0).stop 2024-06-02T23:28:01.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 shutdown_connections 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5638690ab3c0 0x5638690ad7b0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563869025ae0 0x56386903a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563869022980 0x563869039980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8971ffb640 0 client.28846 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 --2- 172.21.15.52:0/678042346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563869021f90 0x563869039440 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 >> 172.21.15.52:0/678042346 conn(0x56386901d680 msgr2=0x563869034710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 shutdown_connections 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 1 -- 172.21.15.52:0/678042346 wait complete. 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.298+0000 7f8993b7f740 20 client.28846 trim_cache size 0 max 16384 2024-06-02T23:28:01.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:28:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:28:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:28:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89740a2d00 msgr2=0x7f89740a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.300 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89740a2d00 0x7f89740a51c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8984031aa0 tx=0x7f89840099a0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 msgr2=0x7f898c197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 0x7f898c197b90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f897c00e9c0 tx=0x7f897c00ee80 comp rx=0 tx=0).stop 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 shutdown_connections 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89740a2d00 0x7f89740a51c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.299+0000 7f8990d90640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f898c1024b0 0x7f898c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.300+0000 7f8990d90640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f898c101af0 0x7f898c1980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.300+0000 7f8990d90640 1 --2- 172.21.15.52:0/1412353619 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f898c100900 0x7f898c197b90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.300+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 >> 172.21.15.52:0/1412353619 conn(0x7f898c0fbe70 msgr2=0x7f898c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.300+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 shutdown_connections 2024-06-02T23:28:01.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:01.300+0000 7f8990d90640 1 -- 172.21.15.52:0/1412353619 wait complete. 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 -- 172.21.15.52:0/920422094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc1073f0 msgr2=0x7f86fc1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 --2- 172.21.15.52:0/920422094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc1073f0 0x7f86fc1077d0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00faf0 tx=0x7f86ec00fe00 comp rx=0 tx=0).stop 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 -- 172.21.15.52:0/920422094 shutdown_connections 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 --2- 172.21.15.52:0/920422094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc101d40 0x7f86fc10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 --2- 172.21.15.52:0/920422094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc1073f0 0x7f86fc1077d0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 --2- 172.21.15.52:0/920422094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101380 0x7f86fc101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 -- 172.21.15.52:0/920422094 >> 172.21.15.52:0/920422094 conn(0x7f86fc078300 msgr2=0x7f86fc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 -- 172.21.15.52:0/920422094 shutdown_connections 2024-06-02T23:28:01.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.369+0000 7f8701556700 1 -- 172.21.15.52:0/920422094 wait complete. 2024-06-02T23:28:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.370+0000 7f8701556700 1 Processor -- start 2024-06-02T23:28:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.370+0000 7f8701556700 1 -- start start 2024-06-02T23:28:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f8701556700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc101380 0x7f86fc19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f8701556700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 0x7f86fc19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f8701556700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc1073f0 0x7f86fc196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f8701556700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc110530 con 0x7f86fc101380 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f8701556700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1103b0 con 0x7f86fc101d40 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f8701556700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1106b0 con 0x7f86fc1073f0 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 0x7f86fc19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 0x7f86fc19cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51178/0 (socket says 172.21.15.52:51178) 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 -- 172.21.15.52:0/2821412667 learned_addr learned my addr 172.21.15.52:0/2821412667 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86faffd700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc101380 0x7f86fc19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc1073f0 0x7f86fc196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 -- 172.21.15.52:0/2821412667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc1073f0 msgr2=0x7f86fc196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc1073f0 0x7f86fc196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 -- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc101380 msgr2=0x7f86fc19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc101380 0x7f86fc19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fa7fc700 1 -- 172.21.15.52:0/2821412667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc196f70 con 0x7f86fc101d40 2024-06-02T23:28:01.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86faffd700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc101380 0x7f86fc19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.371+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc1073f0 0x7f86fc196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.372+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 0x7f86fc19cb50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86f000e9a0 tx=0x7f86f000ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.372+0000 7f86e3fff700 1 -- 172.21.15.52:0/2821412667 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f000cbd0 con 0x7f86fc101d40 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.372+0000 7f86e3fff700 1 -- 172.21.15.52:0/2821412667 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f000cd70 con 0x7f86fc101d40 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.372+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc197260 con 0x7f86fc101d40 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.372+0000 7f86e3fff700 1 -- 172.21.15.52:0/2821412667 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f0018750 con 0x7f86fc101d40 2024-06-02T23:28:01.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.372+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a3470 con 0x7f86fc101d40 2024-06-02T23:28:01.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.376+0000 7f86e3fff700 1 -- 172.21.15.52:0/2821412667 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f86f0010430 con 0x7f86fc101d40 2024-06-02T23:28:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.376+0000 7f86e3fff700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f86e40a2c70 0x7f86e40a5130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.377+0000 7f86faffd700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f86e40a2c70 0x7f86e40a5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.377+0000 7f86e3fff700 1 -- 172.21.15.52:0/2821412667 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f86f00cf320 con 0x7f86fc101d40 2024-06-02T23:28:01.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.377+0000 7f86faffd700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f86e40a2c70 0x7f86e40a5130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f86ec006010 tx=0x7f86ec011040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 Processor -- start 2024-06-02T23:28:01.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 -- start start 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ec880 0x55f4e99e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f4e98d8f60 con 0x55f4e99e67a0 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4e9937850 con 0x55f4e99ec880 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4e989f460 con 0x55f4e99ebf60 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f86faffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f86fa7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ec880 0x55f4e99e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.378+0000 7f86faffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ec340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:44048/0 (socket says 172.21.15.52:44048) 2024-06-02T23:28:01.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86faffd700 1 -- 172.21.15.52:0/89745096 learned_addr learned my addr 172.21.15.52:0/89745096 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fa7fc700 1 -- 172.21.15.52:0/89745096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 msgr2=0x55f4e99ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fa7fc700 1 -- 172.21.15.52:0/89745096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 msgr2=0x55f4e99e6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fa7fc700 1 -- 172.21.15.52:0/89745096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e98868d0 con 0x55f4e99ec880 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ec880 0x55f4e99e6260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f86f00cd1c0 tx=0x7f86f0097f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c03959a2e980f05 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86e17fa700 1 -- 172.21.15.52:0/89745096 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00b2020 con 0x55f4e99ec880 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86e17fa700 1 -- 172.21.15.52:0/89745096 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f009bbb0 con 0x55f4e99ec880 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.379+0000 7f86e17fa700 1 -- 172.21.15.52:0/89745096 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f00a4390 con 0x55f4e99ec880 2024-06-02T23:28:01.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.380+0000 7f870bdd9b80 1 -- 172.21.15.52:0/89745096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ec880 msgr2=0x55f4e99e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.380+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ec880 0x55f4e99e6260 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f86f00cd1c0 tx=0x7f86f0097f80 comp rx=0 tx=0).stop 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 -- 172.21.15.52:0/89745096 shutdown_connections 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/89745096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ec880 0x55f4e99e6260 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 -- 172.21.15.52:0/89745096 >> 172.21.15.52:0/89745096 conn(0x55f4e9913080 msgr2=0x55f4e99e1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 -- 172.21.15.52:0/89745096 shutdown_connections 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 -- 172.21.15.52:0/89745096 wait complete. 2024-06-02T23:28:01.381 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 -- start start 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.381+0000 7f870bdd9b80 1 Processor -- start 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ef970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e99efeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f4e98d7fc0 con 0x55f4e99ec880 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f4e9933810 con 0x55f4e99e67a0 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4e99f3260 con 0x55f4e99ebf60 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f870bdd9b80 10 client.0 _ll_register_callbacks cb 0x55f4e98cd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:51194/0 (socket says 172.21.15.52:51194) 2024-06-02T23:28:01.383 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 -- 172.21.15.52:0/718210201 learned_addr learned my addr 172.21.15.52:0/718210201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e99efeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ef970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 -- 172.21.15.52:0/718210201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 msgr2=0x55f4e99ef970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ef970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 -- 172.21.15.52:0/718210201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 msgr2=0x55f4e99efeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.382+0000 7f86faffd700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e99efeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.383+0000 7f86faffd700 1 -- 172.21.15.52:0/718210201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e98fd750 con 0x55f4e99e67a0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.383+0000 7f86faffd700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f2670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00f880 tx=0x7f86ec009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb6bd305da29df2c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.383+0000 7f86deff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.383+0000 7f86deff5700 1 -- 172.21.15.52:0/718210201 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0054d0 con 0x55f4e99e67a0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.383+0000 7f86deff5700 1 -- 172.21.15.52:0/718210201 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f86ec005670 con 0x55f4e99e67a0 2024-06-02T23:28:01.384 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86f009bbb0 con 0x55f4e99e67a0 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f86deff5700 1 -- 172.21.15.52:0/718210201 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f86ec009e60 con 0x55f4e99e67a0 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f870bdd9b80 10 client.29186 fetch_fsmap learned FSMap version 42 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4e99fda30 con 0x55f4e99e67a0 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f86deff5700 1 -- 172.21.15.52:0/718210201 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86f009bbb0 con 0x55f4e99e67a0 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f86deff5700 1 -- 172.21.15.52:0/718210201 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f86ec023790 con 0x55f4e99e67a0 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f870bdd9b80 10 client.29186 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T23:28:01.386 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.384+0000 7f870bdd9b80 10 client.29186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f870bdd9b80 20 client.29186 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a00070 0x55f4e9a02460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f870bdd9b80 4 client.29186 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4e9944d50 con 0x55f4e9a00070 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a00070 0x55f4e9a02460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f86deff5700 10 client.29186 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.385+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a00070 0x55f4e9a02460 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b70f992965e54b76 server_cookie=36e7b23def2ffb8a in_seq=0 out_seq=0 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.386+0000 7f86deff5700 1 -- 172.21.15.52:0/718210201 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f4e9944d50 con 0x55f4e9a00070 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.386+0000 7f86deff5700 10 client.29186 handle_command_reply: tid=0 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.386+0000 7f870bdd9b80 1 client.29186 shutdown 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 msgr2=0x55f4e99f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f2670 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f86ec00f880 tx=0x7f86ec009310 comp rx=0 tx=0).stop 2024-06-02T23:28:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 shutdown_connections 2024-06-02T23:28:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a00070 0x55f4e9a02460 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e99efeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f2670 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/718210201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e99ef970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 >> 172.21.15.52:0/718210201 conn(0x55f4e9913080 msgr2=0x55f4e99e0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f86deff5700 0 client.29186 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 shutdown_connections 2024-06-02T23:28:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 1 -- 172.21.15.52:0/718210201 wait complete. 2024-06-02T23:28:01.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.387+0000 7f870bdd9b80 20 client.29186 trim_cache size 0 max 16384 2024-06-02T23:28:01.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.422+0000 7f870bdd9b80 1 Processor -- start 2024-06-02T23:28:01.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.422+0000 7f870bdd9b80 1 -- start start 2024-06-02T23:28:01.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f86faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f86faffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58440/0 (socket says 172.21.15.52:58440) 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f86faffd700 1 -- 172.21.15.52:0/3555790396 learned_addr learned my addr 172.21.15.52:0/3555790396 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ebf60 0x55f4e99eb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ec880 0x55f4e99f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1aa9d0 con 0x55f4e99e67a0 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ebf60 0x55f4e99eb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ac540 con 0x55f4e99ebf60 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ec880 0x55f4e99f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.423+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ab0a0 con 0x55f4e99ec880 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86faffd700 1 -- 172.21.15.52:0/3555790396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ec880 msgr2=0x55f4e99f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86faffd700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ec880 0x55f4e99f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86faffd700 1 -- 172.21.15.52:0/3555790396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ebf60 msgr2=0x55f4e99eb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86faffd700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ebf60 0x55f4e99eb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86faffd700 1 -- 172.21.15.52:0/3555790396 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4e99f0230 con 0x55f4e99e67a0 2024-06-02T23:28:01.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ebf60 0x55f4e99eb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:01.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ec880 0x55f4e99f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:01.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.424+0000 7f86faffd700 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6260 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f86ec000c00 tx=0x7f86ec005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f139035967358df1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.425+0000 7f86e0ff9700 1 -- 172.21.15.52:0/3555790396 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec0054d0 con 0x55f4e99e67a0 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.425+0000 7f86e0ff9700 1 -- 172.21.15.52:0/3555790396 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86ec035070 con 0x55f4e99e67a0 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.425+0000 7f86e0ff9700 1 -- 172.21.15.52:0/3555790396 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ec03c8c0 con 0x55f4e99e67a0 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 msgr2=0x55f4e99e6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6260 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f86ec000c00 tx=0x7f86ec005050 comp rx=0 tx=0).stop 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 shutdown_connections 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ec880 0x55f4e99f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99e67a0 0x55f4e99e6260 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3555790396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99ebf60 0x55f4e99eb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 >> 172.21.15.52:0/3555790396 conn(0x55f4e9913080 msgr2=0x55f4e99ed310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 shutdown_connections 2024-06-02T23:28:01.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.426+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3555790396 wait complete. 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 -- start start 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 Processor -- start 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e9a07500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e9a07a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1abe40 con 0x55f4e99ec880 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1ab770 con 0x55f4e99e67a0 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc1a8970 con 0x55f4e99ebf60 2024-06-02T23:28:01.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.427+0000 7f870bdd9b80 10 client.0 _ll_register_callbacks cb 0x55f4e98cd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e9a07a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e9a07a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58442/0 (socket says 172.21.15.52:58442) 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 -- 172.21.15.52:0/3255426829 learned_addr learned my addr 172.21.15.52:0/3255426829 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86faffd700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fa7fc700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e9a07500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 -- 172.21.15.52:0/3255426829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 msgr2=0x55f4e9a07500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e9a07500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 -- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 msgr2=0x55f4e99f8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.428+0000 7f86fb7fe700 1 -- 172.21.15.52:0/3255426829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4e9a08210 con 0x55f4e99ec880 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f86fb7fe700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e9a07a40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007de0 tx=0x7f86f4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baa35d2d8bea0449 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:01.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f86deff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:01.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f86deff5700 1 -- 172.21.15.52:0/3255426829 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f401d070 con 0x55f4e99ec880 2024-06-02T23:28:01.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ec035070 con 0x55f4e99ec880 2024-06-02T23:28:01.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f86deff5700 1 -- 172.21.15.52:0/3255426829 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f86f4017bb0 con 0x55f4e99ec880 2024-06-02T23:28:01.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f86deff5700 1 -- 172.21.15.52:0/3255426829 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) ==== 24+0+0 (secure 0 0 0) 0x7f86f4028070 con 0x55f4e99ec880 2024-06-02T23:28:01.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f870bdd9b80 10 client.19848 fetch_fsmap learned FSMap version 42 2024-06-02T23:28:01.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.429+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) -- 0x55f4e9a73510 con 0x55f4e99ec880 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f86deff5700 1 -- 172.21.15.52:0/3255426829 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f86ec035070 con 0x55f4e99ec880 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f86deff5700 1 -- 172.21.15.52:0/3255426829 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 42) ==== 8358+0+0 (secure 0 0 0) 0x7f86f4021400 con 0x55f4e99ec880 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f870bdd9b80 10 client.19848 fetch_fsmap finished waiting for FSMap version 42 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f870bdd9b80 10 client.19848 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f870bdd9b80 20 client.19848 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a73c60 0x55f4e9a76080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f870bdd9b80 4 client.19848 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f4e993f2d0 con 0x55f4e9a73c60 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.430+0000 7f86faffd700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a73c60 0x55f4e9a76080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f86faffd700 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a73c60 0x55f4e9a76080 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae74ebea4287afc7 server_cookie=aaadfa765d77c605 in_seq=0 out_seq=0 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f86deff5700 10 client.19848 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f86deff5700 1 -- 172.21.15.52:0/3255426829 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f4e993f2d0 con 0x55f4e9a73c60 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f86deff5700 10 client.19848 handle_command_reply: tid=0 2024-06-02T23:28:01.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 client.19848 shutdown 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 msgr2=0x55f4e9a07a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e9a07a40 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f86f4007de0 tx=0x7f86f4009920 comp rx=0 tx=0).stop 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 shutdown_connections 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f4e9a73c60 0x55f4e9a76080 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f4e99ec880 0x55f4e9a07a40 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f4e99e67a0 0x55f4e99f8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f86deff5700 0 client.19848 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 --2- 172.21.15.52:0/3255426829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4e99ebf60 0x55f4e9a07500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 >> 172.21.15.52:0/3255426829 conn(0x55f4e9913080 msgr2=0x55f4e99efa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 shutdown_connections 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.431+0000 7f870bdd9b80 1 -- 172.21.15.52:0/3255426829 wait complete. 2024-06-02T23:28:01.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.432+0000 7f870bdd9b80 20 client.19848 trim_cache size 0 max 16384 2024-06-02T23:28:01.434 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:28:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:28:01.434 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:28:01.434 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.432+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f86e40a2c70 msgr2=0x7f86e40a5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.432+0000 7f8701556700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f86e40a2c70 0x7f86e40a5130 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f86ec006010 tx=0x7f86ec011040 comp rx=0 tx=0).stop 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.432+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 msgr2=0x7f86fc19cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.432+0000 7f8701556700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 0x7f86fc19cb50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f86f000e9a0 tx=0x7f86f000ed60 comp rx=0 tx=0).stop 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 shutdown_connections 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f86e40a2c70 0x7f86e40a5130 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86fc1073f0 0x7f86fc196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86fc101380 0x7f86fc19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 --2- 172.21.15.52:0/2821412667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f86fc101d40 0x7f86fc19cb50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:01.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 >> 172.21.15.52:0/2821412667 conn(0x7f86fc078300 msgr2=0x7f86fc1001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:01.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 shutdown_connections 2024-06-02T23:28:01.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:01.433+0000 7f8701556700 1 -- 172.21.15.52:0/2821412667 wait complete. 2024-06-02T23:28:01.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 19105 5.05 MB/sec execute 69 sec latency 130.156 ms 2024-06-02T23:28:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:01 smithi123 ceph-mon[35977]: pgmap v915: 97 pgs: 97 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2024-06-02T23:28:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:01 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:28:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:01 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:28:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:01 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:28:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:01 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:28:02.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:01 smithi052 ceph-mon[32340]: pgmap v915: 97 pgs: 97 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2024-06-02T23:28:02.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:01 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:28:02.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:28:02.092 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:28:02.093 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:28:02.133 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:28:02.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:28:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:01 smithi174 ceph-mon[35994]: pgmap v915: 97 pgs: 97 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 279 op/s 2024-06-02T23:28:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:01 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:28:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:01 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:28:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:01 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:28:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:01 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:28:02.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 19270 5.13 MB/sec execute 70 sec latency 62.367 ms 2024-06-02T23:28:02.540 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:03.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 19427 5.12 MB/sec execute 71 sec latency 64.731 ms 2024-06-02T23:28:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:03 smithi123 ceph-mon[35977]: pgmap v916: 97 pgs: 97 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-06-02T23:28:04.082 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:03 smithi052 ceph-mon[32340]: pgmap v916: 97 pgs: 97 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-06-02T23:28:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:03 smithi174 ceph-mon[35994]: pgmap v916: 97 pgs: 97 active+clean; 365 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-06-02T23:28:04.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 19611 5.11 MB/sec execute 72 sec latency 89.680 ms 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/1865383022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 msgr2=0x7f1d14108960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/1865383022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14108960 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1d040099c0 tx=0x7f1d04009cd0 comp rx=0 tx=0).stop 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/1865383022 shutdown_connections 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/1865383022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d14102e90 0x7f1d1410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/1865383022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14108960 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/1865383022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d141024d0 0x7f1d14102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/1865383022 >> 172.21.15.52:0/1865383022 conn(0x7f1d140fdf60 msgr2=0x7f1d14100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:04.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/1865383022 shutdown_connections 2024-06-02T23:28:04.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.621+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/1865383022 wait complete. 2024-06-02T23:28:04.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.622+0000 7f1d1a8ce700 1 Processor -- start 2024-06-02T23:28:04.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.622+0000 7f1d1a8ce700 1 -- start start 2024-06-02T23:28:04.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d1a8ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d141024d0 0x7f1d14074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d1a8ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d14102e90 0x7f1d14074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d1a8ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d1a8ce700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1d14112740 con 0x7f1d14108580 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d1a8ce700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141125c0 con 0x7f1d14102e90 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d1a8ce700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1d141128c0 con 0x7f1d141024d0 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51528/0 (socket says 172.21.15.52:51528) 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 -- 172.21.15.52:0/3960484082 learned_addr learned my addr 172.21.15.52:0/3960484082 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 -- 172.21.15.52:0/3960484082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d141024d0 msgr2=0x7f1d14074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d141024d0 0x7f1d14074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 -- 172.21.15.52:0/3960484082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d14102e90 msgr2=0x7f1d14074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d14102e90 0x7f1d14074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:04.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.623+0000 7f1d18e6b700 1 -- 172.21.15.52:0/3960484082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d141a7060 con 0x7f1d14108580 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.624+0000 7f1d18e6b700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14075210 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c013990 tx=0x7f1d0c013d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.624+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c021070 con 0x7f1d14108580 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.624+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1d0c011580 con 0x7f1d14108580 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.624+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d0c01cb10 con 0x7f1d14108580 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.625+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d141988d0 con 0x7f1d14108580 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.625+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d14198dc0 con 0x7f1d14108580 2024-06-02T23:28:04.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.626+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d14069180 con 0x7f1d14108580 2024-06-02T23:28:04.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.628+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1d0c024070 con 0x7f1d14108580 2024-06-02T23:28:04.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.629+0000 7f1d117fa700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1cfc0a2ca0 0x7f1cfc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:04.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.629+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1d0c0d7ad0 con 0x7f1d14108580 2024-06-02T23:28:04.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.629+0000 7f1d13fff700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1cfc0a2ca0 0x7f1cfc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:04.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.630+0000 7f1d13fff700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1cfc0a2ca0 0x7f1cfc0a5160 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1d04000c00 tx=0x7f1d04005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:04.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:04.635+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1d0c02b070 con 0x7f1d14108580 2024-06-02T23:28:05.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.058+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d14075cc0 con 0x7f1d14108580 2024-06-02T23:28:05.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.059+0000 7f1d117fa700 1 -- 172.21.15.52:0/3960484082 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) ==== 94+0+11940 (secure 0 0 0) 0x7f1d14075cc0 con 0x7f1d14108580 2024-06-02T23:28:05.059 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:28:05.059 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":42,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:00.537887+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24436,24434,24451,24449,24470]},"id":1}]} 2024-06-02T23:28:05.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1cfc0a2ca0 msgr2=0x7f1cfc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1cfc0a2ca0 0x7f1cfc0a5160 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f1d04000c00 tx=0x7f1d04005fb0 comp rx=0 tx=0).stop 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 msgr2=0x7f1d14075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14075210 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1d0c013990 tx=0x7f1d0c013d50 comp rx=0 tx=0).stop 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 shutdown_connections 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1cfc0a2ca0 0x7f1cfc0a5160 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1d14108580 0x7f1d14075210 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1d141024d0 0x7f1d14074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 --2- 172.21.15.52:0/3960484082 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d14102e90 0x7f1d14074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 >> 172.21.15.52:0/3960484082 conn(0x7f1d140fdf60 msgr2=0x7f1d140ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 shutdown_connections 2024-06-02T23:28:05.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:05.062+0000 7f1d1a8ce700 1 -- 172.21.15.52:0/3960484082 wait complete. 2024-06-02T23:28:05.063 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 42 2024-06-02T23:28:05.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 19781 5.13 MB/sec execute 73 sec latency 66.938 ms 2024-06-02T23:28:05.782 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:28:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:05 smithi123 ceph-mon[35977]: pgmap v917: 97 pgs: 97 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-06-02T23:28:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:05 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3960484082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:28:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:05 smithi174 ceph-mon[35994]: pgmap v917: 97 pgs: 97 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-06-02T23:28:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:05 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3960484082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:28:06.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:05 smithi052 ceph-mon[32340]: pgmap v917: 97 pgs: 97 active+clean; 358 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-06-02T23:28:06.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:05 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3960484082' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:28:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:06.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 19948 5.15 MB/sec execute 74 sec latency 65.473 ms 2024-06-02T23:28:06.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:06 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:28:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:06 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:28:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:06 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:28:07.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 20102 5.26 MB/sec execute 75 sec latency 218.689 ms 2024-06-02T23:28:07.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:07 smithi052 ceph-mon[32340]: pgmap v918: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 329 op/s 2024-06-02T23:28:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:07 smithi123 ceph-mon[35977]: pgmap v918: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 329 op/s 2024-06-02T23:28:08.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 -- 172.21.15.52:0/3008906192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba81073f0 msgr2=0x7f8ba81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/3008906192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba81073f0 0x7f8ba81077d0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f8b98009990 tx=0x7f8b98009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 -- 172.21.15.52:0/3008906192 shutdown_connections 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/3008906192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba8101d40 0x7f8ba810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/3008906192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba81073f0 0x7f8ba81077d0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/3008906192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101380 0x7f8ba8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 -- 172.21.15.52:0/3008906192 >> 172.21.15.52:0/3008906192 conn(0x7f8ba8078300 msgr2=0x7f8ba8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 -- 172.21.15.52:0/3008906192 shutdown_connections 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.115+0000 7f8bae9d9700 1 -- 172.21.15.52:0/3008906192 wait complete. 2024-06-02T23:28:08.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.116+0000 7f8bae9d9700 1 Processor -- start 2024-06-02T23:28:08.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.116+0000 7f8bae9d9700 1 -- start start 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.116+0000 7f8bae9d9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba8101380 0x7f8ba819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8bae9d9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 0x7f8ba819cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8bae9d9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba81073f0 0x7f8ba8196740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8bae9d9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba8110420 con 0x7f8ba8101380 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 0x7f8ba819cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8bae9d9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81102a0 con 0x7f8ba8101d40 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8bae9d9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81105a0 con 0x7f8ba81073f0 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 0x7f8ba819cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39988/0 (socket says 172.21.15.52:39988) 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 -- 172.21.15.52:0/1468385793 learned_addr learned my addr 172.21.15.52:0/1468385793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:08.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba8101380 0x7f8ba819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8bacf76700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba81073f0 0x7f8ba8196740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 -- 172.21.15.52:0/1468385793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba81073f0 msgr2=0x7f8ba8196740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba81073f0 0x7f8ba8196740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 -- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba8101380 msgr2=0x7f8ba819c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba8101380 0x7f8ba819c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.117+0000 7f8ba77fe700 1 -- 172.21.15.52:0/1468385793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba8197020 con 0x7f8ba8101d40 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.118+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 0x7f8ba819cbb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9a0 tx=0x7f8b9c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.118+0000 7f8ba57fa700 1 -- 172.21.15.52:0/1468385793 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c00cbd0 con 0x7f8ba8101d40 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.118+0000 7f8ba57fa700 1 -- 172.21.15.52:0/1468385793 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b9c00cd70 con 0x7f8ba8101d40 2024-06-02T23:28:08.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.118+0000 7f8ba57fa700 1 -- 172.21.15.52:0/1468385793 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c018750 con 0x7f8ba8101d40 2024-06-02T23:28:08.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.118+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba8197310 con 0x7f8ba8101d40 2024-06-02T23:28:08.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.118+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba81a35f0 con 0x7f8ba8101d40 2024-06-02T23:28:08.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.121+0000 7f8ba57fa700 1 -- 172.21.15.52:0/1468385793 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8b9c010430 con 0x7f8ba8101d40 2024-06-02T23:28:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.122+0000 7f8ba57fa700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8b900a2ca0 0x7f8b900a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.123+0000 7f8ba57fa700 1 -- 172.21.15.52:0/1468385793 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8b9c0cf380 con 0x7f8ba8101d40 2024-06-02T23:28:08.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.123+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8b900a2ca0 0x7f8b900a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.123+0000 7f8bb925cb80 1 Processor -- start 2024-06-02T23:28:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.123+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8b900a2ca0 0x7f8b900a5160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006010 tx=0x7f8b9801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:08.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.123+0000 7f8bb925cb80 1 -- start start 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 0x56113589d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589d770 0x5611358971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561135897730 0x561135897b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5611357e0810 con 0x561135897730 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561135788720 con 0x56113589ce50 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611357dc7d0 con 0x56113589d770 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 0x56113589d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 0x56113589d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:40000/0 (socket says 172.21.15.52:40000) 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 -- 172.21.15.52:0/2863911304 learned_addr learned my addr 172.21.15.52:0/2863911304 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bacf76700 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561135897730 0x561135897b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589d770 0x5611358971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 -- 172.21.15.52:0/2863911304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589d770 msgr2=0x5611358971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589d770 0x5611358971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 -- 172.21.15.52:0/2863911304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561135897730 msgr2=0x561135897b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561135897730 0x561135897b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8ba7fff700 1 -- 172.21.15.52:0/2863911304 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113577ef50 con 0x56113589ce50 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.124+0000 7f8bacf76700 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561135897730 0x561135897b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.125+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 0x56113589d230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800bd70 tx=0x7f8b9801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=794582679fd4aeb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.125+0000 7f8b8e7fc700 1 -- 172.21.15.52:0/2863911304 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800bda0 con 0x56113589ce50 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.125+0000 7f8b8e7fc700 1 -- 172.21.15.52:0/2863911304 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b98004b40 con 0x56113589ce50 2024-06-02T23:28:08.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.125+0000 7f8b8e7fc700 1 -- 172.21.15.52:0/2863911304 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9800fa00 con 0x56113589ce50 2024-06-02T23:28:08.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 -- 172.21.15.52:0/2863911304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 msgr2=0x56113589d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 0x56113589d230 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f8b9800bd70 tx=0x7f8b9801e000 comp rx=0 tx=0).stop 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 -- 172.21.15.52:0/2863911304 shutdown_connections 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561135897730 0x561135897b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56113589ce50 0x56113589d230 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/2863911304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589d770 0x5611358971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 -- 172.21.15.52:0/2863911304 >> 172.21.15.52:0/2863911304 conn(0x5611357b9cc0 msgr2=0x561135889850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 -- 172.21.15.52:0/2863911304 shutdown_connections 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 -- 172.21.15.52:0/2863911304 wait complete. 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.126+0000 7f8bb925cb80 1 -- start start 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 Processor -- start 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x5611358a3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x5611358a0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 0x5611358a0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5611358a4120 con 0x56113589d770 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561135780bf0 con 0x561135897730 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611358a42a0 con 0x56113589ce50 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bb925cb80 10 client.0 _ll_register_callbacks cb 0x56113577a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bacf76700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 0x5611358a0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8ba77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x5611358a0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bacf76700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 0x5611358a0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51566/0 (socket says 172.21.15.52:51566) 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8bacf76700 1 -- 172.21.15.52:0/3333056059 learned_addr learned my addr 172.21.15.52:0/3333056059 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x5611358a0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:48942/0 (socket says 172.21.15.52:48942) 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.127+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x5611358a3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bacf76700 1 -- 172.21.15.52:0/3333056059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 msgr2=0x5611358a0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bacf76700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x5611358a0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bacf76700 1 -- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 msgr2=0x5611358a3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bacf76700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x5611358a3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bacf76700 1 -- 172.21.15.52:0/3333056059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56113572f8d0 con 0x56113589d770 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x5611358a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bacf76700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 0x5611358a0da0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0010e90 tx=0x7f8ba0013720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2874d6a7fb9eb15c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8b877fe700 1 -- 172.21.15.52:0/3333056059 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba001e450 con 0x56113589d770 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b98004b40 con 0x56113589d770 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.128+0000 7f8b877fe700 1 -- 172.21.15.52:0/3333056059 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8ba001e5f0 con 0x56113589d770 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8b877fe700 1 -- 172.21.15.52:0/3333056059 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8ba0025070 con 0x56113589d770 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 10 client.19866 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:08.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5611358aeb80 con 0x56113589d770 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8b877fe700 1 -- 172.21.15.52:0/3333056059 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b98004b40 con 0x56113589d770 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8b877fe700 1 -- 172.21.15.52:0/3333056059 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8ba00089c0 con 0x56113589d770 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 10 client.19866 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 10 client.19866 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 20 client.19866 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5611358b1200 0x5611358b3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 4 client.19866 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611357edd10 con 0x5611358b1200 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.129+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5611358b1200 0x5611358b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.130+0000 7f8b877fe700 10 client.19866 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:28:08.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.130+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5611358b1200 0x5611358b3640 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3e694743e9e728f9 server_cookie=7f19c2e2ec2e493c in_seq=0 out_seq=0 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8b877fe700 1 -- 172.21.15.52:0/3333056059 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5611357edd10 con 0x5611358b1200 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8b877fe700 10 client.19866 handle_command_reply: tid=0 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 client.19866 shutdown 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 msgr2=0x5611358a0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 0x5611358a0da0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0010e90 tx=0x7f8ba0013720 comp rx=0 tx=0).stop 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 shutdown_connections 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589d770 0x5611358a0da0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5611358b1200 0x5611358b3640 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x5611358a3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/3333056059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x5611358a0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8b877fe700 0 client.19866 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.132+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 >> 172.21.15.52:0/3333056059 conn(0x5611357b9cc0 msgr2=0x5611358961e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.133+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 shutdown_connections 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.133+0000 7f8bb925cb80 1 -- 172.21.15.52:0/3333056059 wait complete. 2024-06-02T23:28:08.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.133+0000 7f8bb925cb80 20 client.19866 trim_cache size 0 max 16384 2024-06-02T23:28:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:07 smithi174 ceph-mon[35994]: pgmap v918: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 329 op/s 2024-06-02T23:28:08.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 Processor -- start 2024-06-02T23:28:08.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 -- start start 2024-06-02T23:28:08.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 0x5611358971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x561135883690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x56113589add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81aa950 con 0x5611357b9cc0 2024-06-02T23:28:08.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ac490 con 0x561135897730 2024-06-02T23:28:08.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.157+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ab020 con 0x56113589ce50 2024-06-02T23:28:08.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 0x5611358971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 0x5611358971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51568/0 (socket says 172.21.15.52:51568) 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 -- 172.21.15.52:0/188950033 learned_addr learned my addr 172.21.15.52:0/188950033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x561135883690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8bacf76700 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x56113589add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 -- 172.21.15.52:0/188950033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 msgr2=0x56113589add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x56113589add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 -- 172.21.15.52:0/188950033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 msgr2=0x561135883690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x561135883690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 -- 172.21.15.52:0/188950033 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56113588a320 con 0x5611357b9cc0 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 0x5611358971f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005850 tx=0x7f8b98019040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e38cf17ee66de3e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.158+0000 7f8b8dffb700 1 -- 172.21.15.52:0/188950033 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b980049c0 con 0x5611357b9cc0 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8b8dffb700 1 -- 172.21.15.52:0/188950033 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b9800f460 con 0x5611357b9cc0 2024-06-02T23:28:08.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8b8dffb700 1 -- 172.21.15.52:0/188950033 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98023400 con 0x5611357b9cc0 2024-06-02T23:28:08.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 -- 172.21.15.52:0/188950033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 msgr2=0x5611358971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 0x5611358971f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005850 tx=0x7f8b98019040 comp rx=0 tx=0).stop 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 -- 172.21.15.52:0/188950033 shutdown_connections 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56113589ce50 0x56113589add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611357b9cc0 0x5611358971f0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/188950033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561135897730 0x561135883690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.159+0000 7f8bb925cb80 1 -- 172.21.15.52:0/188950033 >> 172.21.15.52:0/188950033 conn(0x561135889b80 msgr2=0x56113589a6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.160+0000 7f8bb925cb80 1 -- 172.21.15.52:0/188950033 shutdown_connections 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.160+0000 7f8bb925cb80 1 -- 172.21.15.52:0/188950033 wait complete. 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.160+0000 7f8bb925cb80 1 -- start start 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.160+0000 7f8bb925cb80 1 Processor -- start 2024-06-02T23:28:08.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 0x5611358a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561135897730 0x5611358a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589ce50 0x5611358b83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81abdc0 con 0x56113589ce50 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81ab6f0 con 0x5611357b9cc0 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba81a8910 con 0x561135897730 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 0x5611358a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bb925cb80 10 client.0 _ll_register_callbacks cb 0x56113577a920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 0x5611358a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:40028/0 (socket says 172.21.15.52:40028) 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 -- 172.21.15.52:0/1637833897 learned_addr learned my addr 172.21.15.52:0/1637833897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561135897730 0x5611358a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8bacf76700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589ce50 0x5611358b83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 -- 172.21.15.52:0/1637833897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561135897730 msgr2=0x5611358a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561135897730 0x5611358a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 -- 172.21.15.52:0/1637833897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589ce50 msgr2=0x5611358b83a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589ce50 0x5611358b83a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.161+0000 7f8ba7fff700 1 -- 172.21.15.52:0/1637833897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611358b8be0 con 0x5611357b9cc0 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8ba7fff700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 0x5611358a9770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005610 tx=0x7f8b9800bda0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d02d5027dd210997 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8b877fe700 1 -- 172.21.15.52:0/1637833897 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b98046070 con 0x5611357b9cc0 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8b877fe700 1 -- 172.21.15.52:0/1637833897 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8b9802b040 con 0x5611357b9cc0 2024-06-02T23:28:08.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b9800f460 con 0x5611357b9cc0 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8b877fe700 1 -- 172.21.15.52:0/1637833897 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8b98051050 con 0x5611357b9cc0 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8bb925cb80 10 client.29213 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.162+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56113591b800 con 0x5611357b9cc0 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8b877fe700 1 -- 172.21.15.52:0/1637833897 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8b9800f460 con 0x5611357b9cc0 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8b877fe700 1 -- 172.21.15.52:0/1637833897 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8b98022070 con 0x5611357b9cc0 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8bb925cb80 10 client.29213 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8bb925cb80 10 client.29213 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8bb925cb80 20 client.29213 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56113589d770 0x56113591da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8bb925cb80 4 client.29213 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:28:08.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.163+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5611357e8290 con 0x56113589d770 2024-06-02T23:28:08.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.164+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56113589d770 0x56113591da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:08.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.164+0000 7f8b877fe700 10 client.29213 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:28:08.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.164+0000 7f8ba77fe700 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56113589d770 0x56113591da30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bec04cff8b18b12 server_cookie=4de09b2a7254f044 in_seq=0 out_seq=0 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.165+0000 7f8b877fe700 1 -- 172.21.15.52:0/1637833897 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5611357e8290 con 0x56113589d770 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.165+0000 7f8b877fe700 10 client.29213 handle_command_reply: tid=0 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.165+0000 7f8bb925cb80 1 client.29213 shutdown 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.165+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 msgr2=0x5611358a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.165+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 0x5611358a9770 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8b98005610 tx=0x7f8b9800bda0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 shutdown_connections 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56113589ce50 0x5611358b83a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x56113589d770 0x56113591da30 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611357b9cc0 0x5611358a9770 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8b877fe700 0 client.29213 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 --2- 172.21.15.52:0/1637833897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561135897730 0x5611358a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 >> 172.21.15.52:0/1637833897 conn(0x561135889b80 msgr2=0x56113589b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:08.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 shutdown_connections 2024-06-02T23:28:08.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 1 -- 172.21.15.52:0/1637833897 wait complete. 2024-06-02T23:28:08.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.166+0000 7f8bb925cb80 20 client.29213 trim_cache size 0 max 16384 2024-06-02T23:28:08.170 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.167+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8b900a2ca0 msgr2=0x7f8b900a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.167+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8b900a2ca0 0x7f8b900a5160 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f8b98006010 tx=0x7f8b9801a040 comp rx=0 tx=0).stop 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.167+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 msgr2=0x7f8ba819cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.167+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 0x7f8ba819cbb0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00e9a0 tx=0x7f8b9c00ed60 comp rx=0 tx=0).stop 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 shutdown_connections 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8b900a2ca0 0x7f8b900a5160 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ba81073f0 0x7f8ba8196740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8ba8101380 0x7f8ba819c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 --2- 172.21.15.52:0/1468385793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ba8101d40 0x7f8ba819cbb0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:08.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 >> 172.21.15.52:0/1468385793 conn(0x7f8ba8078300 msgr2=0x7f8ba8100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:08.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 shutdown_connections 2024-06-02T23:28:08.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:08.168+0000 7f8bae9d9700 1 -- 172.21.15.52:0/1468385793 wait complete. 2024-06-02T23:28:08.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 20280 5.33 MB/sec execute 76 sec latency 104.876 ms 2024-06-02T23:28:08.887 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:28:09.253 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:09.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 20488 5.48 MB/sec execute 77 sec latency 66.267 ms 2024-06-02T23:28:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:09 smithi052 ceph-mon[32340]: pgmap v919: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-06-02T23:28:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:09 smithi123 ceph-mon[35977]: pgmap v919: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-06-02T23:28:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:09 smithi174 ceph-mon[35994]: pgmap v919: 97 pgs: 97 active+clean; 357 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1005 KiB/s rd, 3.3 MiB/s wr, 277 op/s 2024-06-02T23:28:10.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 20695 5.56 MB/sec execute 78 sec latency 66.991 ms 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 -- 172.21.15.52:0/2628638763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed4101390 msgr2=0x7f5ed41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 --2- 172.21.15.52:0/2628638763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed4101390 0x7f5ed41017f0 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4009990 tx=0x7f5ec4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 -- 172.21.15.52:0/2628638763 shutdown_connections 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 --2- 172.21.15.52:0/2628638763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101d30 0x7f5ed410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 --2- 172.21.15.52:0/2628638763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed41073d0 0x7f5ed41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 --2- 172.21.15.52:0/2628638763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed4101390 0x7f5ed41017f0 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 -- 172.21.15.52:0/2628638763 >> 172.21.15.52:0/2628638763 conn(0x7f5ed40782c0 msgr2=0x7f5ed40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:11.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.080+0000 7f5edc00e700 1 -- 172.21.15.52:0/2628638763 shutdown_connections 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.081+0000 7f5edc00e700 1 -- 172.21.15.52:0/2628638763 wait complete. 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 Processor -- start 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 -- start start 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101390 0x7f5ed410fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed4101d30 0x7f5ed4110390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 0x7f5ed41108d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed4112640 con 0x7f5ed41073d0 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41124c0 con 0x7f5ed4101390 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5edc00e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41127c0 con 0x7f5ed4101d30 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5eda5ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 0x7f5ed41108d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5eda5ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 0x7f5ed41108d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51596/0 (socket says 172.21.15.52:51596) 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5eda5ab700 1 -- 172.21.15.52:0/251016689 learned_addr learned my addr 172.21.15.52:0/251016689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:11.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.082+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed4101d30 0x7f5ed4110390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101390 0x7f5ed410fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed95a9700 1 -- 172.21.15.52:0/251016689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101390 msgr2=0x7f5ed410fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101390 0x7f5ed410fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed95a9700 1 -- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 msgr2=0x7f5ed41108d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 0x7f5ed41108d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed95a9700 1 -- 172.21.15.52:0/251016689 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed41a7080 con 0x7f5ed4101d30 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5eda5ab700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 0x7f5ed41108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101390 0x7f5ed410fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed4101d30 0x7f5ed4110390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4000c00 tx=0x7f5ec40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ecaffd700 1 -- 172.21.15.52:0/251016689 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4026070 con 0x7f5ed4101d30 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed41a7310 con 0x7f5ed4101d30 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ecaffd700 1 -- 172.21.15.52:0/251016689 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ec4021410 con 0x7f5ed4101d30 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.083+0000 7f5ecaffd700 1 -- 172.21.15.52:0/251016689 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400b610 con 0x7f5ed4101d30 2024-06-02T23:28:11.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.084+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed41a77c0 con 0x7f5ed4101d30 2024-06-02T23:28:11.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.087+0000 7f5ecaffd700 1 -- 172.21.15.52:0/251016689 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ec4029070 con 0x7f5ed4101d30 2024-06-02T23:28:11.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.087+0000 7f5ecaffd700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ec00a2c20 0x7f5ec00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.088+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ec00a2c20 0x7f5ec00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.088+0000 7f5ecaffd700 1 -- 172.21.15.52:0/251016689 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ec40cfde0 con 0x7f5ed4101d30 2024-06-02T23:28:11.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.088+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ec00a2c20 0x7f5ec00a50e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009c90 tx=0x7f5ed0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:11.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 Processor -- start 2024-06-02T23:28:11.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 -- start start 2024-06-02T23:28:11.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69f640 0x556b4b69fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69ff60 0x556b4b6999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b69a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b4b5eb810 con 0x556b4b69ff60 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b4b593720 con 0x556b4b69f640 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b4b5e77d0 con 0x556b4b699f20 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5eda5ab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b69a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5eda5ab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b69a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:48986/0 (socket says 172.21.15.52:48986) 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5eda5ab700 1 -- 172.21.15.52:0/3294090590 learned_addr learned my addr 172.21.15.52:0/3294090590 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.089+0000 7f5ed9daa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69f640 0x556b4b69fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3294090590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 msgr2=0x556b4b69a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b69a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3294090590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69ff60 msgr2=0x556b4b6999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69ff60 0x556b4b6999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3294090590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4b589f50 con 0x556b4b69f640 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69f640 0x556b4b69fa20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0010a40 tx=0x7f5ed0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=172682a065fa3d4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ebffff700 1 -- 172.21.15.52:0/3294090590 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed0011bb0 con 0x556b4b69f640 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ebffff700 1 -- 172.21.15.52:0/3294090590 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ed0011d50 con 0x556b4b69f640 2024-06-02T23:28:11.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.090+0000 7f5ebffff700 1 -- 172.21.15.52:0/3294090590 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed001c670 con 0x556b4b69f640 2024-06-02T23:28:11.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3294090590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69f640 msgr2=0x556b4b69fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69f640 0x556b4b69fa20 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0010a40 tx=0x7f5ed0013000 comp rx=0 tx=0).stop 2024-06-02T23:28:11.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3294090590 shutdown_connections 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b69a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69f640 0x556b4b69fa20 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3294090590 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69ff60 0x556b4b6999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3294090590 >> 172.21.15.52:0/3294090590 conn(0x556b4b5c4cc0 msgr2=0x556b4b694850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.091+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3294090590 shutdown_connections 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.092+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3294090590 wait complete. 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.092+0000 7f5ee6891b80 1 -- start start 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.092+0000 7f5ee6891b80 1 Processor -- start 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.092+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.092+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed9daa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed9daa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:49002/0 (socket says 172.21.15.52:49002) 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed9daa700 1 -- 172.21.15.52:0/2234508023 learned_addr learned my addr 172.21.15.52:0/2234508023 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556b4b6a69d0 con 0x556b4b69f640 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556b4b58bbf0 con 0x556b4b69ff60 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556b4b6a6b50 con 0x556b4b699f20 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ee6891b80 10 client.0 _ll_register_callbacks cb 0x556b4b585920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5eda5ab700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 -- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 msgr2=0x556b4b6a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 -- 172.21.15.52:0/2234508023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 msgr2=0x556b4b6a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4b53a8d0 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6a3100 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400b2e0 tx=0x7f5ec4098390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602dd5fdc1c23ead server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ebd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.093+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/2234508023 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec409ce00 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/2234508023 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ec4021930 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ed0011d50 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/2234508023 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5ec409c700 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 10 client.19881 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556b4b6b13b0 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/2234508023 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ec40a0ab0 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/2234508023 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5ec40b0b90 con 0x556b4b69f640 2024-06-02T23:28:11.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 10 client.19881 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 10 client.19881 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 20 client.19881 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b6b5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 4 client.19881 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.094+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5ed0011d50 con 0x556b4b6b3880 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.095+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b6b5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.095+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b6b5c70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=922ad1b6a9b363ba server_cookie=abd09d05eaaa332a in_seq=0 out_seq=0 2024-06-02T23:28:11.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.095+0000 7f5ebd7fa700 10 client.19881 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:28:11.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/2234508023 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f5ed0011d50 con 0x556b4b6b3880 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ebd7fa700 10 client.19881 handle_command_reply: tid=0 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 client.19881 shutdown 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 msgr2=0x556b4b6a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6a3100 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400b2e0 tx=0x7f5ec4098390 comp rx=0 tx=0).stop 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 shutdown_connections 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b6b5c70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ebd7fa700 0 client.19881 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/2234508023 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6a3100 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.096+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 >> 172.21.15.52:0/2234508023 conn(0x556b4b5c4cc0 msgr2=0x556b4b6945f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.097+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 shutdown_connections 2024-06-02T23:28:11.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.097+0000 7f5ee6891b80 1 -- 172.21.15.52:0/2234508023 wait complete. 2024-06-02T23:28:11.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.097+0000 7f5ee6891b80 20 client.19881 trim_cache size 0 max 16384 2024-06-02T23:28:11.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 Processor -- start 2024-06-02T23:28:11.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 -- start start 2024-06-02T23:28:11.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6afca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6999e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b69d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41aa710 con 0x556b4b69f640 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41ac250 con 0x556b4b69ff60 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41aade0 con 0x556b4b699f20 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ed95a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ed9daa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ed95a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51632/0 (socket says 172.21.15.52:51632) 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ed95a9700 1 -- 172.21.15.52:0/188607498 learned_addr learned my addr 172.21.15.52:0/188607498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5eda5ab700 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b69d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.118+0000 7f5ed95a9700 1 -- 172.21.15.52:0/188607498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 msgr2=0x556b4b6afca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6afca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ed95a9700 1 -- 172.21.15.52:0/188607498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 msgr2=0x556b4b69d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b69d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ed95a9700 1 -- 172.21.15.52:0/188607498 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b4b6a2ed0 con 0x556b4b69f640 2024-06-02T23:28:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6999e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40cdc80 tx=0x7f5ec409bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a5d59f76450e7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ebf7fe700 1 -- 172.21.15.52:0/188607498 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4066070 con 0x556b4b69f640 2024-06-02T23:28:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ebf7fe700 1 -- 172.21.15.52:0/188607498 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ec409ce00 con 0x556b4b69f640 2024-06-02T23:28:11.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.119+0000 7f5ebf7fe700 1 -- 172.21.15.52:0/188607498 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4021930 con 0x556b4b69f640 2024-06-02T23:28:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 -- 172.21.15.52:0/188607498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 msgr2=0x556b4b6999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6999e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5ec40cdc80 tx=0x7f5ec409bb50 comp rx=0 tx=0).stop 2024-06-02T23:28:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 -- 172.21.15.52:0/188607498 shutdown_connections 2024-06-02T23:28:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b69d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b699f20 0x556b4b6afca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/188607498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b69f640 0x556b4b6999e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 -- 172.21.15.52:0/188607498 >> 172.21.15.52:0/188607498 conn(0x556b4b5c4cc0 msgr2=0x556b4b69ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 -- 172.21.15.52:0/188607498 shutdown_connections 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.120+0000 7f5ee6891b80 1 -- 172.21.15.52:0/188607498 wait complete. 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.121+0000 7f5ee6891b80 1 -- start start 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.121+0000 7f5ee6891b80 1 Processor -- start 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.121+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 0x556b4b6abe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.121+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b69f640 0x556b4b6ac350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.121+0000 7f5ee6891b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6ba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 0x556b4b6abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41abb80 con 0x556b4b699f20 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed41ab4b0 con 0x556b4b69ff60 2024-06-02T23:28:11.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ee6891b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed419c730 con 0x556b4b69f640 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 0x556b4b6abe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51640/0 (socket says 172.21.15.52:51640) 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3208858517 learned_addr learned my addr 172.21.15.52:0/3208858517 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ee6891b80 10 client.0 _ll_register_callbacks cb 0x556b4b585920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5eda5ab700 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6ba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b69f640 msgr2=0x556b4b6ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b69f640 0x556b4b6ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3208858517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 msgr2=0x556b4b6ba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6ba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 -- 172.21.15.52:0/3208858517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b4b6aca50 con 0x556b4b699f20 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ed9daa700 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 0x556b4b6abe10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0021040 tx=0x7f5ed0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108e81d9b5a42801 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ebd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.122+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/3208858517 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ed003e070 con 0x556b4b699f20 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/3208858517 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ed001ebe0 con 0x556b4b699f20 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec409ce00 con 0x556b4b699f20 2024-06-02T23:28:11.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/3208858517 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5ed004a070 con 0x556b4b699f20 2024-06-02T23:28:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ee6891b80 10 client.19887 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:11.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556b4b71db00 con 0x556b4b699f20 2024-06-02T23:28:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/3208858517 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ec409ce00 con 0x556b4b699f20 2024-06-02T23:28:11.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/3208858517 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5ed00049d0 con 0x556b4b699f20 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ee6891b80 10 client.19887 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ee6891b80 10 client.19887 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.123+0000 7f5ee6891b80 20 client.19887 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.124+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b71fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.124+0000 7f5ee6891b80 4 client.19887 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.124+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b4b5f8d10 con 0x556b4b6b3880 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.124+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b71fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.124+0000 7f5ebd7fa700 10 client.19887 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.124+0000 7f5ed95a9700 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b71fee0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5b299672a203ba5a server_cookie=72a574ad4e8a407f in_seq=0 out_seq=0 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.125+0000 7f5ebd7fa700 1 -- 172.21.15.52:0/3208858517 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b4b5f8d10 con 0x556b4b6b3880 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.125+0000 7f5ebd7fa700 10 client.19887 handle_command_reply: tid=0 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.125+0000 7f5ee6891b80 1 client.19887 shutdown 2024-06-02T23:28:11.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.125+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 msgr2=0x556b4b6abe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.125+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 0x556b4b6abe10 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0021040 tx=0x7f5ed0009f80 comp rx=0 tx=0).stop 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 shutdown_connections 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556b4b69ff60 0x556b4b6ba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556b4b699f20 0x556b4b6abe10 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x556b4b6b3880 0x556b4b71fee0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 --2- 172.21.15.52:0/3208858517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556b4b69f640 0x556b4b6ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ebd7fa700 0 client.19887 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 >> 172.21.15.52:0/3208858517 conn(0x556b4b5c4cc0 msgr2=0x556b4b68de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 shutdown_connections 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 1 -- 172.21.15.52:0/3208858517 wait complete. 2024-06-02T23:28:11.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.126+0000 7f5ee6891b80 20 client.19887 trim_cache size 0 max 16384 2024-06-02T23:28:11.130 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:28:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ec00a2c20 msgr2=0x7f5ec00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ec00a2c20 0x7f5ec00a50e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5ed0009c90 tx=0x7f5ed0009260 comp rx=0 tx=0).stop 2024-06-02T23:28:11.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed4101d30 msgr2=0x7f5ed4110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed4101d30 0x7f5ed4110390 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4000c00 tx=0x7f5ec40047b0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 shutdown_connections 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5ec00a2c20 0x7f5ec00a50e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5ed41073d0 0x7f5ed41108d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5ed4101390 0x7f5ed410fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 --2- 172.21.15.52:0/251016689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5ed4101d30 0x7f5ed4110390 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 >> 172.21.15.52:0/251016689 conn(0x7f5ed40782c0 msgr2=0x7f5ed41001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.127+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 shutdown_connections 2024-06-02T23:28:11.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:11.128+0000 7f5edc00e700 1 -- 172.21.15.52:0/251016689 wait complete. 2024-06-02T23:28:11.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 20877 5.66 MB/sec execute 79 sec latency 144.392 ms 2024-06-02T23:28:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:11 smithi052 ceph-mon[32340]: pgmap v920: 97 pgs: 97 active+clean; 362 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-06-02T23:28:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:11 smithi123 ceph-mon[35977]: pgmap v920: 97 pgs: 97 active+clean; 362 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-06-02T23:28:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:11 smithi174 ceph-mon[35994]: pgmap v920: 97 pgs: 97 active+clean; 362 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 296 op/s 2024-06-02T23:28:12.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 20985 5.64 MB/sec execute 80 sec latency 114.888 ms 2024-06-02T23:28:13.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21094 5.66 MB/sec execute 81 sec latency 128.672 ms 2024-06-02T23:28:14.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:13 smithi123 ceph-mon[35977]: pgmap v921: 97 pgs: 97 active+clean; 369 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 292 op/s 2024-06-02T23:28:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:13 smithi174 ceph-mon[35994]: pgmap v921: 97 pgs: 97 active+clean; 369 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 292 op/s 2024-06-02T23:28:14.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:13 smithi052 ceph-mon[32340]: pgmap v921: 97 pgs: 97 active+clean; 369 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 292 op/s 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 -- 172.21.15.52:0/2908291725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4100900 msgr2=0x7efef4100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 --2- 172.21.15.52:0/2908291725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4100900 0x7efef4100d20 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7efef0009900 tx=0x7efef002a3f0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 -- 172.21.15.52:0/2908291725 shutdown_connections 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 --2- 172.21.15.52:0/2908291725 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4101af0 0x7efef4101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 --2- 172.21.15.52:0/2908291725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 0x7efef4108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 --2- 172.21.15.52:0/2908291725 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4100900 0x7efef4100d20 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.407 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 -- 172.21.15.52:0/2908291725 >> 172.21.15.52:0/2908291725 conn(0x7efef40fbe70 msgr2=0x7efef40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 -- 172.21.15.52:0/2908291725 shutdown_connections 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.404+0000 7efefc291640 1 -- 172.21.15.52:0/2908291725 wait complete. 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 Processor -- start 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 -- start start 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 0x7efef410b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 0x7efef41098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 0x7efef4109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efef410d5b0 con 0x7efef4101af0 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efef410d430 con 0x7efef41024b0 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.405+0000 7efefc291640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efef410d730 con 0x7efef4100900 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 0x7efef41098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefa02d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 0x7efef410b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefa82e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 0x7efef4109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefa02d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 0x7efef410b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34938/0 (socket says 172.21.15.52:34938) 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 0x7efef41098c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53772/0 (socket says 172.21.15.52:53772) 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefa02d640 1 -- 172.21.15.52:0/1227703823 learned_addr learned my addr 172.21.15.52:0/1227703823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 -- 172.21.15.52:0/1227703823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 msgr2=0x7efef410b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 0x7efef410b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 -- 172.21.15.52:0/1227703823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 msgr2=0x7efef4109e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 0x7efef4109e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 -- 172.21.15.52:0/1227703823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef410a6c0 con 0x7efef4101af0 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefa02d640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 0x7efef410b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefa82e640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 0x7efef4109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efef982c640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 0x7efef41098c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efee400e9f0 tx=0x7efee400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efeeb7fe640 1 -- 172.21.15.52:0/1227703823 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400ccc0 con 0x7efef4101af0 2024-06-02T23:28:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef41a1d40 con 0x7efef4101af0 2024-06-02T23:28:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efeeb7fe640 1 -- 172.21.15.52:0/1227703823 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee4005020 con 0x7efef4101af0 2024-06-02T23:28:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.406+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef41a2200 con 0x7efef4101af0 2024-06-02T23:28:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.407+0000 7efeeb7fe640 1 -- 172.21.15.52:0/1227703823 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee400ce60 con 0x7efef4101af0 2024-06-02T23:28:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.410+0000 7efefef95740 1 Processor -- start 2024-06-02T23:28:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 -- start start 2024-06-02T23:28:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 0x55654fdebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdf3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55654fd464f0 con 0x55654fdf1380 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55654fce4870 con 0x55654fdeb820 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefef95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55654fd424b0 con 0x55654fdec210 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efeeb7fe640 1 -- 172.21.15.52:0/1227703823 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efee40051d0 con 0x7efef4101af0 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefa82e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 0x55654fdebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefa02d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefa82e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 0x55654fdebc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43748/0 (socket says 172.21.15.52:43748) 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efefa82e640 1 -- 172.21.15.52:0/371565132 learned_addr learned my addr 172.21.15.52:0/371565132 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.411+0000 7efef982c640 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdf3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efefa82e640 1 -- 172.21.15.52:0/371565132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 msgr2=0x55654fdf0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efefa82e640 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efefa82e640 1 -- 172.21.15.52:0/371565132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 msgr2=0x55654fdf3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efefa82e640 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdf3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.413 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efefa82e640 1 -- 172.21.15.52:0/371565132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654fd009b0 con 0x55654fdeb820 2024-06-02T23:28:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efef982c640 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdf3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efefa82e640 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 0x55654fdebc40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efeec011810 tx=0x7efeec011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54f614007b5f6764 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efee8bfe640 1 -- 172.21.15.52:0/371565132 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec011310 con 0x55654fdeb820 2024-06-02T23:28:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.412+0000 7efee8bfe640 1 -- 172.21.15.52:0/371565132 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeec00f9a0 con 0x55654fdeb820 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efee8bfe640 1 -- 172.21.15.52:0/371565132 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec005670 con 0x55654fdeb820 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efeeb7fe640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efee00a2c30 0x7efee00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 -- 172.21.15.52:0/371565132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 msgr2=0x55654fdebc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 0x55654fdebc40 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7efeec011810 tx=0x7efeec011cd0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 -- 172.21.15.52:0/371565132 shutdown_connections 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdf3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 --2- 172.21.15.52:0/371565132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdeb820 0x55654fdebc40 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 -- 172.21.15.52:0/371565132 >> 172.21.15.52:0/371565132 conn(0x55654fde6e90 msgr2=0x55654fde7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefa02d640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efee00a2c30 0x7efee00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efeeb7fe640 1 -- 172.21.15.52:0/1227703823 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efee40cef50 con 0x7efef4101af0 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 -- 172.21.15.52:0/371565132 shutdown_connections 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.413+0000 7efefef95740 1 -- 172.21.15.52:0/371565132 wait complete. 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 -- start start 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefa02d640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efee00a2c30 0x7efee00a50f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efef0000c00 tx=0x7efef0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 Processor -- start 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 0x55654fdf5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf61d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdfd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55654fdf4380 con 0x55654fdeb820 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55654fccfae0 con 0x55654fdf1380 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55654fdf4500 con 0x55654fdec210 2024-06-02T23:28:14.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefa02d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 0x55654fdf5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefef95740 10 client.0 _ll_register_callbacks cb 0x55654fd089b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefa02d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 0x55654fdf5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53790/0 (socket says 172.21.15.52:53790) 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefa02d640 1 -- 172.21.15.52:0/1498352107 learned_addr learned my addr 172.21.15.52:0/1498352107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.414+0000 7efefa82e640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdfd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efef982c640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa02d640 1 -- 172.21.15.52:0/1498352107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 msgr2=0x55654fdf61d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa02d640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf61d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa02d640 1 -- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 msgr2=0x55654fdfd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa02d640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdfd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa02d640 1 -- 172.21.15.52:0/1498352107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654fa5a210 con 0x55654fdeb820 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa82e640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdfd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefa02d640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 0x55654fdf5c90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efef0009230 tx=0x7efef002a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ecc17d89734ba09 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:14.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efeddffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efeddffb640 1 -- 172.21.15.52:0/1498352107 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efef0035050 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efeddffb640 1 -- 172.21.15.52:0/1498352107 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efef0031790 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.415+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efeec00f9a0 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efeddffb640 1 -- 172.21.15.52:0/1498352107 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efef005a070 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 10 client.19896 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55654fb26620 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efeddffb640 1 -- 172.21.15.52:0/1498352107 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efeec00f9a0 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efeddffb640 1 -- 172.21.15.52:0/1498352107 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7efef0049a90 con 0x55654fdeb820 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 10 client.19896 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 10 client.19896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 20 client.19896 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe01ec0 0x55654fe04360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 4 client.19896 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55654fd59f10 con 0x55654fe01ec0 2024-06-02T23:28:14.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.416+0000 7efef982c640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe01ec0 0x55654fe04360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.417+0000 7efef982c640 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe01ec0 0x55654fe04360 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fef4b7b6ecdcfac server_cookie=60c7fc0fcfcc767f in_seq=0 out_seq=0 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.417+0000 7efeddffb640 10 client.19896 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efeddffb640 1 -- 172.21.15.52:0/1498352107 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55654fd59f10 con 0x55654fe01ec0 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efeddffb640 10 client.19896 handle_command_reply: tid=0 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 client.19896 shutdown 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 msgr2=0x55654fdf5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 0x55654fdf5c90 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7efef0009230 tx=0x7efef002a3f0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 shutdown_connections 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe01ec0 0x55654fe04360 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdec210 0x55654fdf61d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdfd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efeddffb640 0 client.19896 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 --2- 172.21.15.52:0/1498352107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdeb820 0x55654fdf5c90 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 >> 172.21.15.52:0/1498352107 conn(0x55654fde6e90 msgr2=0x55654fddd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 shutdown_connections 2024-06-02T23:28:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 1 -- 172.21.15.52:0/1498352107 wait complete. 2024-06-02T23:28:14.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.418+0000 7efefef95740 20 client.19896 trim_cache size 0 max 16384 2024-06-02T23:28:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 Processor -- start 2024-06-02T23:28:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 -- start start 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdd9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdec210 0x55654fdfd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efef41a6990 con 0x55654fdf1380 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efef41a84d0 con 0x55654fdec210 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefef95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efef41a7060 con 0x55654fdeb820 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefa02d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdd9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efef982c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdec210 0x55654fdfd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefa02d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdd9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34972/0 (socket says 172.21.15.52:34972) 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefa02d640 1 -- 172.21.15.52:0/3723505626 learned_addr learned my addr 172.21.15.52:0/3723505626 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.447+0000 7efefa82e640 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efef982c640 1 -- 172.21.15.52:0/3723505626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 msgr2=0x55654fdd9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efef982c640 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdd9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efef982c640 1 -- 172.21.15.52:0/3723505626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 msgr2=0x55654fdecaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efef982c640 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdecaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efef982c640 1 -- 172.21.15.52:0/3723505626 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55654fdebf80 con 0x55654fdec210 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efef982c640 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdec210 0x55654fdfd1d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efee40ccad0 tx=0x7efee4086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ba407d2cd6d5520 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efedffff640 1 -- 172.21.15.52:0/3723505626 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee40b3070 con 0x55654fdec210 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efedffff640 1 -- 172.21.15.52:0/3723505626 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee409b7e0 con 0x55654fdec210 2024-06-02T23:28:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.448+0000 7efedffff640 1 -- 172.21.15.52:0/3723505626 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee401c040 con 0x55654fdec210 2024-06-02T23:28:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 -- 172.21.15.52:0/3723505626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdec210 msgr2=0x55654fdfd1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdec210 0x55654fdfd1d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7efee40ccad0 tx=0x7efee4086040 comp rx=0 tx=0).stop 2024-06-02T23:28:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 -- 172.21.15.52:0/3723505626 shutdown_connections 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdf1380 0x55654fdecaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdec210 0x55654fdfd1d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 --2- 172.21.15.52:0/3723505626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdd9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 -- 172.21.15.52:0/3723505626 >> 172.21.15.52:0/3723505626 conn(0x55654fde6e90 msgr2=0x55654fde7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 -- 172.21.15.52:0/3723505626 shutdown_connections 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.449+0000 7efefef95740 1 -- 172.21.15.52:0/3723505626 wait complete. 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 -- start start 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 Processor -- start 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdfca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 0x55654fdfcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdf9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efef41a7e00 con 0x55654fdec210 2024-06-02T23:28:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efef41a7730 con 0x55654fdf1380 2024-06-02T23:28:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efef41a4970 con 0x55654fdeb820 2024-06-02T23:28:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efefef95740 10 client.0 _ll_register_callbacks cb 0x7efef0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.450+0000 7efef982c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 0x55654fdfcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efef982c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 0x55654fdfcfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53810/0 (socket says 172.21.15.52:53810) 2024-06-02T23:28:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efef982c640 1 -- 172.21.15.52:0/346786454 learned_addr learned my addr 172.21.15.52:0/346786454 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa82e640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdf9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa02d640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdfca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa82e640 1 -- 172.21.15.52:0/346786454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 msgr2=0x55654fdfca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa82e640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdfca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa82e640 1 -- 172.21.15.52:0/346786454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 msgr2=0x55654fdfcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa82e640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 0x55654fdfcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efefa82e640 1 -- 172.21.15.52:0/346786454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55654fdf9ca0 con 0x55654fdf1380 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.451+0000 7efef982c640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 0x55654fdfcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efefa82e640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdf9560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efeec00f790 tx=0x7efeec011340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4259bfe69b25b6c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efeddffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efeddffb640 1 -- 172.21.15.52:0/346786454 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec0114b0 con 0x55654fdf1380 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efeddffb640 1 -- 172.21.15.52:0/346786454 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efeec01e050 con 0x55654fdf1380 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee409b7e0 con 0x55654fdf1380 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efeddffb640 1 -- 172.21.15.52:0/346786454 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efeec003b80 con 0x55654fdf1380 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efefef95740 10 client.29249 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55654fe6b880 con 0x55654fdf1380 2024-06-02T23:28:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efeddffb640 1 -- 172.21.15.52:0/346786454 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efee409b7e0 con 0x55654fdf1380 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efeddffb640 1 -- 172.21.15.52:0/346786454 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7efeec024040 con 0x55654fdf1380 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efefef95740 10 client.29249 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.452+0000 7efefef95740 10 client.29249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efefef95740 20 client.29249 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efefef95740 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe6c0e0 0x55654fe6e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efefef95740 4 client.29249 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55654fd53c20 con 0x55654fe6c0e0 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efefa02d640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe6c0e0 0x55654fe6e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efefa02d640 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe6c0e0 0x55654fe6e500 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8abfdf884b0f1a9 server_cookie=60dfa0e86ffce6ac in_seq=0 out_seq=0 2024-06-02T23:28:14.456 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.453+0000 7efeddffb640 10 client.29249 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:14.457 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21250 5.77 MB/sec execute 82 sec latency 151.202 ms 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efeddffb640 1 -- 172.21.15.52:0/346786454 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55654fd53c20 con 0x55654fe6c0e0 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efeddffb640 10 client.29249 handle_command_reply: tid=0 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 client.29249 shutdown 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 msgr2=0x55654fdf9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdf9560 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7efeec00f790 tx=0x7efeec011340 comp rx=0 tx=0).stop 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 shutdown_connections 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55654fe6c0e0 0x55654fe6e500 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55654fdf1380 0x55654fdf9560 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55654fdec210 0x55654fdfcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 --2- 172.21.15.52:0/346786454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55654fdeb820 0x55654fdfca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efeddffb640 0 client.29249 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:14.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 >> 172.21.15.52:0/346786454 conn(0x55654fde6e90 msgr2=0x55654fde7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 shutdown_connections 2024-06-02T23:28:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 1 -- 172.21.15.52:0/346786454 wait complete. 2024-06-02T23:28:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.511+0000 7efefef95740 20 client.29249 trim_cache size 0 max 16384 2024-06-02T23:28:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:28:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:28:14.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efee00a2c30 msgr2=0x7efee00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efee00a2c30 0x7efee00a50f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7efef0000c00 tx=0x7efef0005a20 comp rx=0 tx=0).stop 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 msgr2=0x7efef41098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 0x7efef41098c0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7efee400e9f0 tx=0x7efee400edf0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 shutdown_connections 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efee00a2c30 0x7efee00a50f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efef41024b0 0x7efef4109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efef4101af0 0x7efef41098c0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 --2- 172.21.15.52:0/1227703823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efef4100900 0x7efef410b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 >> 172.21.15.52:0/1227703823 conn(0x7efef40fbe70 msgr2=0x7efef4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:14.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 shutdown_connections 2024-06-02T23:28:14.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:14.512+0000 7efefc291640 1 -- 172.21.15.52:0/1227703823 wait complete. 2024-06-02T23:28:15.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21374 5.76 MB/sec execute 83 sec latency 141.076 ms 2024-06-02T23:28:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:15 smithi123 ceph-mon[35977]: pgmap v922: 97 pgs: 97 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-06-02T23:28:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:15 smithi174 ceph-mon[35994]: pgmap v922: 97 pgs: 97 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-06-02T23:28:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:15 smithi052 ceph-mon[32340]: pgmap v922: 97 pgs: 97 active+clean; 373 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.0 MiB/s wr, 248 op/s 2024-06-02T23:28:16.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21500 5.79 MB/sec execute 84 sec latency 136.962 ms 2024-06-02T23:28:17.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21597 5.76 MB/sec execute 85 sec latency 251.886 ms 2024-06-02T23:28:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:17 smithi123 ceph-mon[35977]: pgmap v923: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 311 op/s 2024-06-02T23:28:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:17 smithi174 ceph-mon[35994]: pgmap v923: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 311 op/s 2024-06-02T23:28:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:17 smithi052 ceph-mon[32340]: pgmap v923: 97 pgs: 97 active+clean; 374 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 311 op/s 2024-06-02T23:28:18.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21750 5.74 MB/sec execute 86 sec latency 58.936 ms 2024-06-02T23:28:18.618 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:28:18.990 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:19.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 21871 5.70 MB/sec execute 87 sec latency 88.613 ms 2024-06-02T23:28:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:19 smithi123 ceph-mon[35977]: pgmap v924: 97 pgs: 97 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 231 op/s 2024-06-02T23:28:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:19 smithi174 ceph-mon[35994]: pgmap v924: 97 pgs: 97 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 231 op/s 2024-06-02T23:28:20.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:19 smithi052 ceph-mon[32340]: pgmap v924: 97 pgs: 97 active+clean; 376 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 231 op/s 2024-06-02T23:28:20.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 22006 5.68 MB/sec execute 88 sec latency 216.387 ms 2024-06-02T23:28:21.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 22147 5.68 MB/sec execute 89 sec latency 147.125 ms 2024-06-02T23:28:22.011 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:21 smithi052 ceph-mon[32340]: pgmap v925: 97 pgs: 97 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-06-02T23:28:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:21 smithi174 ceph-mon[35994]: pgmap v925: 97 pgs: 97 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-06-02T23:28:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:21 smithi123 ceph-mon[35977]: pgmap v925: 97 pgs: 97 active+clean; 378 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 248 op/s 2024-06-02T23:28:22.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 22327 5.78 MB/sec execute 90 sec latency 145.592 ms 2024-06-02T23:28:23.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 22485 5.79 MB/sec execute 91 sec latency 100.943 ms 2024-06-02T23:28:23.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 -- 172.21.15.52:0/4262506710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c101d40 msgr2=0x7fad3c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 --2- 172.21.15.52:0/4262506710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c101d40 0x7fad3c10e230 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b0a0 tx=0x7fad3400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 -- 172.21.15.52:0/4262506710 shutdown_connections 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 --2- 172.21.15.52:0/4262506710 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c101d40 0x7fad3c10e230 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 --2- 172.21.15.52:0/4262506710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c1073f0 0x7fad3c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 --2- 172.21.15.52:0/4262506710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad3c101380 0x7fad3c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.580+0000 7fad43a69700 1 -- 172.21.15.52:0/4262506710 >> 172.21.15.52:0/4262506710 conn(0x7fad3c078300 msgr2=0x7fad3c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.581+0000 7fad43a69700 1 -- 172.21.15.52:0/4262506710 shutdown_connections 2024-06-02T23:28:23.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.581+0000 7fad43a69700 1 -- 172.21.15.52:0/4262506710 wait complete. 2024-06-02T23:28:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.583+0000 7fad43a69700 1 Processor -- start 2024-06-02T23:28:23.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.583+0000 7fad43a69700 1 -- start start 2024-06-02T23:28:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad43a69700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad3c101380 0x7fad3c19e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad43a69700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c101d40 0x7fad3c19ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad43a69700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 0x7fad3c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad43a69700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c111560 con 0x7fad3c1073f0 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad43a69700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c1113e0 con 0x7fad3c101d40 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad42006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 0x7fad3c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad43a69700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad3c1116e0 con 0x7fad3c101380 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41004700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c101d40 0x7fad3c19ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad42006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 0x7fad3c1988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53824/0 (socket says 172.21.15.52:53824) 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad42006700 1 -- 172.21.15.52:0/1457720070 learned_addr learned my addr 172.21.15.52:0/1457720070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41805700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad3c101380 0x7fad3c19e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41004700 1 -- 172.21.15.52:0/1457720070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad3c101380 msgr2=0x7fad3c19e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41004700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad3c101380 0x7fad3c19e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41004700 1 -- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 msgr2=0x7fad3c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41004700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 0x7fad3c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad41004700 1 -- 172.21.15.52:0/1457720070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad3c199050 con 0x7fad3c101d40 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.584+0000 7fad42006700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 0x7fad3c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.585+0000 7fad41004700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c101d40 0x7fad3c19ec70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00d750 tx=0x7fad2c00da60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.585+0000 7fad32ffd700 1 -- 172.21.15.52:0/1457720070 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c014030 con 0x7fad3c101d40 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.585+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad3c199340 con 0x7fad3c101d40 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.585+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3c102f90 con 0x7fad3c101d40 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.585+0000 7fad32ffd700 1 -- 172.21.15.52:0/1457720070 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad2c0044d0 con 0x7fad3c101d40 2024-06-02T23:28:23.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.585+0000 7fad32ffd700 1 -- 172.21.15.52:0/1457720070 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c005420 con 0x7fad3c101d40 2024-06-02T23:28:23.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.588+0000 7fad32ffd700 1 -- 172.21.15.52:0/1457720070 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fad2c004020 con 0x7fad3c101d40 2024-06-02T23:28:23.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.589+0000 7fad32ffd700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad280a2ca0 0x7fad280a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.589+0000 7fad41805700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad280a2ca0 0x7fad280a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad32ffd700 1 -- 172.21.15.52:0/1457720070 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fad2c0cea50 con 0x7fad3c101d40 2024-06-02T23:28:23.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad4e2ecb80 1 Processor -- start 2024-06-02T23:28:23.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad41805700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad280a2ca0 0x7fad280a5160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fad3c199a30 tx=0x7fad38012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:23.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad4e2ecb80 1 -- start start 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab64658a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab646529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.590+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab64653340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ab645a4810 con 0x55ab64652f20 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ab6454c720 con 0x55ab64658640 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab64653340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab64653340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53834/0 (socket says 172.21.15.52:53834) 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 -- 172.21.15.52:0/3161750032 learned_addr learned my addr 172.21.15.52:0/3161750032 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/3161750032 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab645a07d0 con 0x55ab64658f60 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad41805700 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab64658a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 -- 172.21.15.52:0/3161750032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 msgr2=0x55ab646529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab646529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 -- 172.21.15.52:0/3161750032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 msgr2=0x55ab64658a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab64658a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 -- 172.21.15.52:0/3161750032 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab64542f50 con 0x55ab64652f20 2024-06-02T23:28:23.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.591+0000 7fad42006700 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab64653340 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fad34009e60 tx=0x7fad34009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18230a2ee883b63 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.592+0000 7fad27fff700 1 -- 172.21.15.52:0/3161750032 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34009460 con 0x55ab64652f20 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.592+0000 7fad27fff700 1 -- 172.21.15.52:0/3161750032 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad34004a50 con 0x55ab64652f20 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.592+0000 7fad27fff700 1 -- 172.21.15.52:0/3161750032 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34007b90 con 0x55ab64652f20 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/3161750032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 msgr2=0x55ab64653340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab64653340 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fad34009e60 tx=0x7fad34009780 comp rx=0 tx=0).stop 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/3161750032 shutdown_connections 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab64653340 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab64658a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/3161750032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab646529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/3161750032 >> 172.21.15.52:0/3161750032 conn(0x55ab6457dcc0 msgr2=0x55ab6464d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/3161750032 shutdown_connections 2024-06-02T23:28:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/3161750032 wait complete. 2024-06-02T23:28:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.593+0000 7fad4e2ecb80 1 -- start start 2024-06-02T23:28:23.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 Processor -- start 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab6465ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab6465c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab6465c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad41805700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab6465ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ab6465f970 con 0x55ab64652f20 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ab64544bf0 con 0x55ab64658640 2024-06-02T23:28:23.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ab6465faf0 con 0x55ab64658f60 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad41805700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab6465ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53842/0 (socket says 172.21.15.52:53842) 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad41805700 1 -- 172.21.15.52:0/2401543793 learned_addr learned my addr 172.21.15.52:0/2401543793 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.594+0000 7fad41004700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab6465c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad4e2ecb80 10 client.0 _ll_register_callbacks cb 0x55ab6453e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad42006700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab6465c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41004700 1 -- 172.21.15.52:0/2401543793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 msgr2=0x55ab6465c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41004700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab6465c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41004700 1 -- 172.21.15.52:0/2401543793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 msgr2=0x55ab6465ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41004700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab6465ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41004700 1 -- 172.21.15.52:0/2401543793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab644f38d0 con 0x55ab64658640 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41805700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab6465ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad41004700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab6465c060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00ddb0 tx=0x7fad2c004310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90c90b1dbacfaf15 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:23.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad257fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad257fa700 1 -- 172.21.15.52:0/2401543793 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c01d040 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad34004a50 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.595+0000 7fad257fa700 1 -- 172.21.15.52:0/2401543793 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fad2c0b4160 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad257fa700 1 -- 172.21.15.52:0/2401543793 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fad2c0057c0 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 10 client.29261 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ab6466a1b0 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad257fa700 1 -- 172.21.15.52:0/2401543793 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad34004a50 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad257fa700 1 -- 172.21.15.52:0/2401543793 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fad2c0a3ac0 con 0x55ab64658640 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 10 client.29261 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 10 client.29261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 20 client.29261 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab6466ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 4 client.29261 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:23.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab645b1d10 con 0x55ab6466c840 2024-06-02T23:28:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.596+0000 7fad41805700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab6466ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.597+0000 7fad257fa700 10 client.29261 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:28:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.597+0000 7fad41805700 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab6466ec60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=53373a2650b9de96 server_cookie=c4a607874ddb9c39 in_seq=0 out_seq=0 2024-06-02T23:28:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad257fa700 1 -- 172.21.15.52:0/2401543793 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ab645b1d10 con 0x55ab6466c840 2024-06-02T23:28:23.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad257fa700 10 client.29261 handle_command_reply: tid=0 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 client.29261 shutdown 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 msgr2=0x55ab6465c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab6465c060 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00ddb0 tx=0x7fad2c004310 comp rx=0 tx=0).stop 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 shutdown_connections 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab6466ec60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658f60 0x55ab6465c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab6465ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad257fa700 0 client.29261 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/2401543793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658640 0x55ab6465c060 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 >> 172.21.15.52:0/2401543793 conn(0x55ab6457dcc0 msgr2=0x55ab646519d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 shutdown_connections 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.598+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/2401543793 wait complete. 2024-06-02T23:28:23.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.599+0000 7fad4e2ecb80 20 client.29261 trim_cache size 0 max 16384 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.623+0000 7fad4e2ecb80 1 Processor -- start 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 -- start start 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab646529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab6464d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658f60 0x55ab64668ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000d260 con 0x55ab64652f20 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000eda0 con 0x55ab64658f60 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000d930 con 0x55ab64658640 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad41805700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab646529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad41004700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab6464d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad41805700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab646529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53848/0 (socket says 172.21.15.52:53848) 2024-06-02T23:28:23.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad41805700 1 -- 172.21.15.52:0/4132507908 learned_addr learned my addr 172.21.15.52:0/4132507908 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:23.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.624+0000 7fad42006700 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658f60 0x55ab64668ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad41805700 1 -- 172.21.15.52:0/4132507908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 msgr2=0x55ab6464d9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad41805700 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab6464d9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad41805700 1 -- 172.21.15.52:0/4132507908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658f60 msgr2=0x55ab64668ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad41805700 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658f60 0x55ab64668ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad41805700 1 -- 172.21.15.52:0/4132507908 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6465fb40 con 0x55ab64652f20 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad42006700 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658f60 0x55ab64668ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad41805700 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab646529e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fad38010810 tx=0x7fad38014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d7762ec5c2a11b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.625+0000 7fad277fe700 1 -- 172.21.15.52:0/4132507908 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3801bcb0 con 0x55ab64652f20 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.626+0000 7fad277fe700 1 -- 172.21.15.52:0/4132507908 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad3801be50 con 0x55ab64652f20 2024-06-02T23:28:23.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.626+0000 7fad277fe700 1 -- 172.21.15.52:0/4132507908 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38028500 con 0x55ab64652f20 2024-06-02T23:28:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/4132507908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 msgr2=0x55ab646529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab646529e0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fad38010810 tx=0x7fad38014000 comp rx=0 tx=0).stop 2024-06-02T23:28:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/4132507908 shutdown_connections 2024-06-02T23:28:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64658f60 0x55ab64668ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64652f20 0x55ab646529e0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/4132507908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab6464d9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/4132507908 >> 172.21.15.52:0/4132507908 conn(0x55ab6457dcc0 msgr2=0x55ab646560c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/4132507908 shutdown_connections 2024-06-02T23:28:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/4132507908 wait complete. 2024-06-02T23:28:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 -- start start 2024-06-02T23:28:23.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.627+0000 7fad4e2ecb80 1 Processor -- start 2024-06-02T23:28:23.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64652f20 0x55ab646651f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab64665730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64658f60 0x55ab64673cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000e6d0 con 0x55ab64658f60 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000e000 con 0x55ab64652f20 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fad2000b200 con 0x55ab64658640 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad4e2ecb80 10 client.0 _ll_register_callbacks cb 0x55ab6453e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad41805700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64652f20 0x55ab646651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad41805700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64652f20 0x55ab646651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43842/0 (socket says 172.21.15.52:43842) 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad41805700 1 -- 172.21.15.52:0/18523075 learned_addr learned my addr 172.21.15.52:0/18523075 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad41004700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab64665730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.628+0000 7fad42006700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64658f60 0x55ab64673cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad41004700 1 -- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64652f20 msgr2=0x55ab646651f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad41004700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64652f20 0x55ab646651f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad41004700 1 -- 172.21.15.52:0/18523075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64658f60 msgr2=0x55ab64673cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad41004700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64658f60 0x55ab64673cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad41004700 1 -- 172.21.15.52:0/18523075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab64674510 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad41004700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab64665730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0cc8f0 tx=0x7fad2c09b9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96561f3a06e21a84 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad257fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad257fa700 1 -- 172.21.15.52:0/18523075 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad2c01d070 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad3801be50 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad257fa700 1 -- 172.21.15.52:0/18523075 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fad2c0055c0 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad257fa700 1 -- 172.21.15.52:0/18523075 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fad2c0a3f50 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad4e2ecb80 10 client.28927 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.629+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ab646d7150 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad257fa700 1 -- 172.21.15.52:0/18523075 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fad3801be50 con 0x55ab64658640 2024-06-02T23:28:23.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad257fa700 1 -- 172.21.15.52:0/18523075 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fad2c0a8050 con 0x55ab64658640 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad4e2ecb80 10 client.28927 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad4e2ecb80 10 client.28927 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad4e2ecb80 20 client.28927 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab646d9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad4e2ecb80 4 client.28927 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab645ac290 con 0x55ab6466c840 2024-06-02T23:28:23.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.630+0000 7fad41805700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab646d9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:23.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad41805700 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab646d9660 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5acf0c2633129a60 server_cookie=f7792a3a09a5b2bf in_seq=0 out_seq=0 2024-06-02T23:28:23.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad257fa700 10 client.28927 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:28:23.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad257fa700 1 -- 172.21.15.52:0/18523075 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab645ac290 con 0x55ab6466c840 2024-06-02T23:28:23.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad257fa700 10 client.28927 handle_command_reply: tid=0 2024-06-02T23:28:23.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad4e2ecb80 1 client.28927 shutdown 2024-06-02T23:28:23.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 msgr2=0x55ab64665730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab64665730 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fad2c0cc8f0 tx=0x7fad2c09b9e0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 shutdown_connections 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.631+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55ab6466c840 0x55ab646d9660 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ab64658f60 0x55ab64673cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ab64652f20 0x55ab646651f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 1 --2- 172.21.15.52:0/18523075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ab64658640 0x55ab64665730 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 >> 172.21.15.52:0/18523075 conn(0x55ab6457dcc0 msgr2=0x55ab64646e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad257fa700 0 client.28927 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 shutdown_connections 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 1 -- 172.21.15.52:0/18523075 wait complete. 2024-06-02T23:28:23.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.632+0000 7fad4e2ecb80 20 client.28927 trim_cache size 0 max 16384 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad280a2ca0 msgr2=0x7fad280a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad280a2ca0 0x7fad280a5160 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fad3c199a30 tx=0x7fad38012040 comp rx=0 tx=0).stop 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c101d40 msgr2=0x7fad3c19ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c101d40 0x7fad3c19ec70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fad2c00d750 tx=0x7fad2c00da60 comp rx=0 tx=0).stop 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 shutdown_connections 2024-06-02T23:28:23.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fad280a2ca0 0x7fad280a5160 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fad3c1073f0 0x7fad3c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fad3c101380 0x7fad3c19e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 --2- 172.21.15.52:0/1457720070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad3c101d40 0x7fad3c19ec70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 >> 172.21.15.52:0/1457720070 conn(0x7fad3c078300 msgr2=0x7fad3c100610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 shutdown_connections 2024-06-02T23:28:23.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:23.633+0000 7fad43a69700 1 -- 172.21.15.52:0/1457720070 wait complete. 2024-06-02T23:28:24.028 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:23 smithi052 ceph-mon[32340]: pgmap v926: 97 pgs: 97 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2024-06-02T23:28:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:23 smithi174 ceph-mon[35994]: pgmap v926: 97 pgs: 97 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2024-06-02T23:28:24.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:23 smithi123 ceph-mon[35977]: pgmap v926: 97 pgs: 97 active+clean; 380 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 258 op/s 2024-06-02T23:28:24.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 22660 5.89 MB/sec execute 92 sec latency 68.021 ms 2024-06-02T23:28:24.476 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:28:24.842 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:25.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 22819 5.89 MB/sec execute 93 sec latency 72.539 ms 2024-06-02T23:28:25.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:25 smithi052 ceph-mon[32340]: pgmap v927: 97 pgs: 97 active+clean; 375 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-06-02T23:28:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:25 smithi174 ceph-mon[35994]: pgmap v927: 97 pgs: 97 active+clean; 375 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-06-02T23:28:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:25 smithi123 ceph-mon[35977]: pgmap v927: 97 pgs: 97 active+clean; 375 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-06-02T23:28:26.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 23002 5.88 MB/sec execute 94 sec latency 62.855 ms 2024-06-02T23:28:26.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 -- 172.21.15.52:0/1835517958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 msgr2=0x7f6998108900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:26.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 --2- 172.21.15.52:0/1835517958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 0x7f6998108900 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6994009990 tx=0x7f6994009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:26.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 -- 172.21.15.52:0/1835517958 shutdown_connections 2024-06-02T23:28:26.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 --2- 172.21.15.52:0/1835517958 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 0x7f699810f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:26.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 --2- 172.21.15.52:0/1835517958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 0x7f6998108900 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:26.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 --2- 172.21.15.52:0/1835517958 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:26.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 -- 172.21.15.52:0/1835517958 >> 172.21.15.52:0/1835517958 conn(0x7f6998079560 msgr2=0x7f6998079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:26.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 -- 172.21.15.52:0/1835517958 shutdown_connections 2024-06-02T23:28:26.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.991+0000 7f699f354700 1 -- 172.21.15.52:0/1835517958 wait complete. 2024-06-02T23:28:26.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.992+0000 7f699f354700 1 Processor -- start 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.992+0000 7f699f354700 1 -- start start 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699f354700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699f354700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 0x7f6998074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699f354700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 0x7f6998075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699f354700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69981126d0 con 0x7f6998102510 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699f354700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6998112550 con 0x7f6998108520 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48664/0 (socket says 172.21.15.52:48664) 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699f354700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6998112850 con 0x7f6998102ed0 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 -- 172.21.15.52:0/3255000095 learned_addr learned my addr 172.21.15.52:0/3255000095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 0x7f6998074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d8f1700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 0x7f6998075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:26.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 -- 172.21.15.52:0/3255000095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 msgr2=0x7f6998074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 0x7f6998074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 -- 172.21.15.52:0/3255000095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 msgr2=0x7f6998075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 0x7f6998075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 -- 172.21.15.52:0/3255000095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69981ab450 con 0x7f6998102510 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 0x7f6998074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.993+0000 7f699d0f0700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998074790 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f69940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.994+0000 7f698e7fc700 1 -- 172.21.15.52:0/3255000095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994027070 con 0x7f6998102510 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.994+0000 7f698e7fc700 1 -- 172.21.15.52:0/3255000095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6994022470 con 0x7f6998102510 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.994+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69981ab680 con 0x7f6998102510 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.994+0000 7f698e7fc700 1 -- 172.21.15.52:0/3255000095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994017650 con 0x7f6998102510 2024-06-02T23:28:26.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.994+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69981aba70 con 0x7f6998102510 2024-06-02T23:28:26.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.997+0000 7f698e7fc700 1 -- 172.21.15.52:0/3255000095 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f699402a070 con 0x7f6998102510 2024-06-02T23:28:26.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.998+0000 7f698e7fc700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69840a2c20 0x7f69840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.003 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.998+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69840a2c20 0x7f69840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.998+0000 7f698e7fc700 1 -- 172.21.15.52:0/3255000095 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f69940cff10 con 0x7f6998102510 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 Processor -- start 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 -- start start 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69840a2c20 0x7f69840a50e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6998075f50 tx=0x7f6988009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 0x556c3a786340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a780260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f699d0f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 0x556c3a786340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f699d0f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 0x556c3a786340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48666/0 (socket says 172.21.15.52:48666) 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f699d0f0700 1 -- 172.21.15.52:0/1063865499 learned_addr learned my addr 172.21.15.52:0/1063865499 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556c3a672f60 con 0x556c3a785f60 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c3a6d1850 con 0x556c3a7807a0 2024-06-02T23:28:27.004 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:26.999+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c3a639460 con 0x556c3a786880 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d8f1700 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d0f0700 1 -- 172.21.15.52:0/1063865499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 msgr2=0x556c3a780260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d0f0700 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a780260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d0f0700 1 -- 172.21.15.52:0/1063865499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 msgr2=0x556c3a780bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d0f0700 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d0f0700 1 -- 172.21.15.52:0/1063865499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3a6208d0 con 0x556c3a785f60 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f699d0f0700 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 0x556c3a786340 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f69940cd920 tx=0x7f699409c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aec9efb1dc26d89 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f69837fe700 1 -- 172.21.15.52:0/1063865499 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69940a30c0 con 0x556c3a785f60 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.000+0000 7f69837fe700 1 -- 172.21.15.52:0/1063865499 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69940a0b80 con 0x556c3a785f60 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69837fe700 1 -- 172.21.15.52:0/1063865499 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699409ca30 con 0x556c3a785f60 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 msgr2=0x556c3a786340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 0x556c3a786340 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f69940cd920 tx=0x7f699409c1e0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.005 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 shutdown_connections 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a785f60 0x556c3a786340 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/1063865499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a780260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.001+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 >> 172.21.15.52:0/1063865499 conn(0x556c3a6ad080 msgr2=0x556c3a77b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.002+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 shutdown_connections 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.002+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/1063865499 wait complete. 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.002+0000 7f69a9bd7b80 1 -- start start 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.002+0000 7f69a9bd7b80 1 Processor -- start 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a78c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a789970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a789eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.006 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x556c3a671fc0 con 0x556c3a7807a0 2024-06-02T23:28:27.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c3a6cd810 con 0x556c3a785f60 2024-06-02T23:28:27.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c3a78d010 con 0x556c3a786880 2024-06-02T23:28:27.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f69a9bd7b80 10 client.0 _ll_register_callbacks cb 0x556c3a6675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:27.008 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a789970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a789970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48924/0 (socket says 172.21.15.52:48924) 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 -- 172.21.15.52:0/3052878982 learned_addr learned my addr 172.21.15.52:0/3052878982 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699d0f0700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a78c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699d8f1700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a789eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 -- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 msgr2=0x556c3a789eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a789eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 -- 172.21.15.52:0/3052878982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 msgr2=0x556c3a78c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a78c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 -- 172.21.15.52:0/3052878982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3a697750 con 0x556c3a785f60 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.003+0000 7f699c8ef700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a789970 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6988009b70 tx=0x7f69880047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f94ec33f5eb1c0b8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f6980ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f6980ff9700 1 -- 172.21.15.52:0/3052878982 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988030840 con 0x556c3a785f60 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f6980ff9700 1 -- 172.21.15.52:0/3052878982 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f69880309e0 con 0x556c3a785f60 2024-06-02T23:28:27.009 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69940a0b80 con 0x556c3a785f60 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f6980ff9700 1 -- 172.21.15.52:0/3052878982 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f698803f070 con 0x556c3a785f60 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f69a9bd7b80 10 client.29276 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.004+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556c3a797580 con 0x556c3a785f60 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f6980ff9700 1 -- 172.21.15.52:0/3052878982 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6988030be0 con 0x556c3a785f60 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f6980ff9700 1 -- 172.21.15.52:0/3052878982 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6988038400 con 0x556c3a785f60 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f69a9bd7b80 10 client.29276 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f69a9bd7b80 10 client.29276 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f69a9bd7b80 20 client.29276 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a799bc0 0x556c3a79bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f69a9bd7b80 4 client.29276 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f69940a0b80 con 0x556c3a799bc0 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.005+0000 7f699d0f0700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a799bc0 0x556c3a79bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.010 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.006+0000 7f699d0f0700 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a799bc0 0x556c3a79bfb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=29059dc33b5b10be server_cookie=91b2859ef7fd5df4 in_seq=0 out_seq=0 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.006+0000 7f6980ff9700 10 client.29276 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f6980ff9700 1 -- 172.21.15.52:0/3052878982 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f69940a0b80 con 0x556c3a799bc0 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f6980ff9700 10 client.29276 handle_command_reply: tid=0 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 client.29276 shutdown 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 msgr2=0x556c3a789970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a789970 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6988009b70 tx=0x7f69880047c0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 shutdown_connections 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a799bc0 0x556c3a79bfb0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a789eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a78c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f6980ff9700 0 client.29276 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:28:27.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/3052878982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a789970 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.007+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 >> 172.21.15.52:0/3052878982 conn(0x556c3a6ad080 msgr2=0x556c3a77a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.008+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 shutdown_connections 2024-06-02T23:28:27.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.008+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/3052878982 wait complete. 2024-06-02T23:28:27.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.008+0000 7f69a9bd7b80 20 client.29276 trim_cache size 0 max 16384 2024-06-02T23:28:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 Processor -- start 2024-06-02T23:28:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 -- start start 2024-06-02T23:28:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a785f60 0x556c3a77ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a786880 0x556c3a78bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69980731a0 con 0x556c3a786880 2024-06-02T23:28:27.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f69981ac3b0 con 0x556c3a7807a0 2024-06-02T23:28:27.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6998073870 con 0x556c3a785f60 2024-06-02T23:28:27.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699d8f1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a786880 0x556c3a78bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699d0f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699c8ef700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a785f60 0x556c3a77ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699d0f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48932/0 (socket says 172.21.15.52:48932) 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699d0f0700 1 -- 172.21.15.52:0/142041262 learned_addr learned my addr 172.21.15.52:0/142041262 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699c8ef700 1 -- 172.21.15.52:0/142041262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 msgr2=0x556c3a780260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699c8ef700 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699c8ef700 1 -- 172.21.15.52:0/142041262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a786880 msgr2=0x556c3a78bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.039+0000 7f699c8ef700 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a786880 0x556c3a78bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.040+0000 7f699c8ef700 1 -- 172.21.15.52:0/142041262 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c3a789660 con 0x556c3a785f60 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.040+0000 7f699d8f1700 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a786880 0x556c3a78bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.040+0000 7f699c8ef700 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a785f60 0x556c3a77ae80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f69880387d0 tx=0x7f6988009350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbf87e911e9c2aa0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.040+0000 7f6982ffd700 1 -- 172.21.15.52:0/142041262 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988038400 con 0x556c3a785f60 2024-06-02T23:28:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.040+0000 7f6982ffd700 1 -- 172.21.15.52:0/142041262 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69880385a0 con 0x556c3a785f60 2024-06-02T23:28:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.040+0000 7f6982ffd700 1 -- 172.21.15.52:0/142041262 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988030e40 con 0x556c3a785f60 2024-06-02T23:28:27.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/142041262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a785f60 msgr2=0x556c3a77ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a785f60 0x556c3a77ae80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f69880387d0 tx=0x7f6988009350 comp rx=0 tx=0).stop 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/142041262 shutdown_connections 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a786880 0x556c3a78bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a7807a0 0x556c3a780260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/142041262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a785f60 0x556c3a77ae80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/142041262 >> 172.21.15.52:0/142041262 conn(0x556c3a6ad080 msgr2=0x556c3a788280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/142041262 shutdown_connections 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.041+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/142041262 wait complete. 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.042+0000 7f69a9bd7b80 1 -- start start 2024-06-02T23:28:27.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.042+0000 7f69a9bd7b80 1 Processor -- start 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.042+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a7925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.042+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a792b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f69a9bd7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a7a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f69981abd80 con 0x556c3a7807a0 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6998073f40 con 0x556c3a785f60 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f69a9bd7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6998071160 con 0x556c3a786880 2024-06-02T23:28:27.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f69a9bd7b80 10 client.0 _ll_register_callbacks cb 0x556c3a6675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a792b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a792b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48946/0 (socket says 172.21.15.52:48946) 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 -- 172.21.15.52:0/4209120833 learned_addr learned my addr 172.21.15.52:0/4209120833 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 -- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 msgr2=0x556c3a7a0f30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a7a0f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 -- 172.21.15.52:0/4209120833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 msgr2=0x556c3a7925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a7925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 -- 172.21.15.52:0/4209120833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c3a7a1770 con 0x556c3a785f60 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f699c8ef700 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a792b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f69880388f0 tx=0x7f6988038ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c76ce08e5c77ea2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.043+0000 7f6980ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.044+0000 7f6980ff9700 1 -- 172.21.15.52:0/4209120833 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6988053070 con 0x556c3a785f60 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.044+0000 7f6980ff9700 1 -- 172.21.15.52:0/4209120833 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6988005250 con 0x556c3a785f60 2024-06-02T23:28:27.045 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.044+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69880385a0 con 0x556c3a785f60 2024-06-02T23:28:27.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.044+0000 7f6980ff9700 1 -- 172.21.15.52:0/4209120833 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f698805d050 con 0x556c3a785f60 2024-06-02T23:28:27.046 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.044+0000 7f69a9bd7b80 10 client.29282 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.044+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x556c3a804340 con 0x556c3a785f60 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f6980ff9700 1 -- 172.21.15.52:0/4209120833 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f69880385a0 con 0x556c3a785f60 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f6980ff9700 1 -- 172.21.15.52:0/4209120833 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6988030640 con 0x556c3a785f60 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f69a9bd7b80 10 client.29282 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f69a9bd7b80 10 client.29282 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f69a9bd7b80 20 client.29282 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a8049b0 0x556c3a806dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f69a9bd7b80 4 client.29282 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.045+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556c3a6ded50 con 0x556c3a8049b0 2024-06-02T23:28:27.047 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.046+0000 7f699d0f0700 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a8049b0 0x556c3a806dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.047+0000 7f699d0f0700 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a8049b0 0x556c3a806dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c04f95fdd0c56987 server_cookie=175ceaf24d165b37 in_seq=0 out_seq=0 2024-06-02T23:28:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.047+0000 7f6980ff9700 10 client.29282 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:28:27.048 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.047+0000 7f6980ff9700 1 -- 172.21.15.52:0/4209120833 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556c3a6ded50 con 0x556c3a8049b0 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.047+0000 7f6980ff9700 10 client.29282 handle_command_reply: tid=0 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.047+0000 7f69a9bd7b80 1 client.29282 shutdown 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 msgr2=0x556c3a792b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a792b30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f69880388f0 tx=0x7f6988038ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 shutdown_connections 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x556c3a8049b0 0x556c3a806dd0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f6980ff9700 0 client.29282 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c3a786880 0x556c3a7a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x556c3a7807a0 0x556c3a7925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 --2- 172.21.15.52:0/4209120833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c3a785f60 0x556c3a792b30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 >> 172.21.15.52:0/4209120833 conn(0x556c3a6ad080 msgr2=0x556c3a786c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 shutdown_connections 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 1 -- 172.21.15.52:0/4209120833 wait complete. 2024-06-02T23:28:27.049 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.048+0000 7f69a9bd7b80 20 client.29282 trim_cache size 0 max 16384 2024-06-02T23:28:27.050 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:28:27.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69840a2c20 msgr2=0x7f69840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69840a2c20 0x7f69840a50e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6998075f50 tx=0x7f6988009260 comp rx=0 tx=0).stop 2024-06-02T23:28:27.050 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 msgr2=0x7f6998074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998074790 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f6994000c00 tx=0x7f69940047b0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 shutdown_connections 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f69840a2c20 0x7f69840a50e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6998108520 0x7f6998075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6998102510 0x7f6998074790 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 --2- 172.21.15.52:0/3255000095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6998102ed0 0x7f6998074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 >> 172.21.15.52:0/3255000095 conn(0x7f6998079560 msgr2=0x7f6998101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 shutdown_connections 2024-06-02T23:28:27.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:27.049+0000 7f699f354700 1 -- 172.21.15.52:0/3255000095 wait complete. 2024-06-02T23:28:27.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 23165 5.88 MB/sec execute 95 sec latency 87.882 ms 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.637+0000 7fd988488640 1 -- 172.21.15.52:0/2029143215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100900 msgr2=0x7fd980100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.637+0000 7fd988488640 1 --2- 172.21.15.52:0/2029143215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100900 0x7fd980100d20 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009900 tx=0x7fd97c02a410 comp rx=0 tx=0).stop 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- 172.21.15.52:0/2029143215 shutdown_connections 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 --2- 172.21.15.52:0/2029143215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980101af0 0x7fd980101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 --2- 172.21.15.52:0/2029143215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 0x7fd980108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 --2- 172.21.15.52:0/2029143215 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980100900 0x7fd980100d20 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.641 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- 172.21.15.52:0/2029143215 >> 172.21.15.52:0/2029143215 conn(0x7fd9800fbe70 msgr2=0x7fd9800fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- 172.21.15.52:0/2029143215 shutdown_connections 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- 172.21.15.52:0/2029143215 wait complete. 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 Processor -- start 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- start start 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980100900 0x7fd980197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 0x7fd980198120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 0x7fd98019f190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010c420 con 0x7fd980101af0 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010c2a0 con 0x7fd9801024b0 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.638+0000 7fd988488640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd98010c5a0 con 0x7fd980100900 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 0x7fd980198120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 0x7fd980198120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48704/0 (socket says 172.21.15.52:48704) 2024-06-02T23:28:27.642 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 -- 172.21.15.52:0/368220589 learned_addr learned my addr 172.21.15.52:0/368220589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd986a25640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 0x7fd98019f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 -- 172.21.15.52:0/368220589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980100900 msgr2=0x7fd980197be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd986224640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980100900 0x7fd980197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980100900 0x7fd980197be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 -- 172.21.15.52:0/368220589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 msgr2=0x7fd98019f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 0x7fd98019f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 -- 172.21.15.52:0/368220589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98019f800 con 0x7fd980101af0 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd986a25640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 0x7fd98019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd986224640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980100900 0x7fd980197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd985a23640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 0x7fd980198120 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd97000d770 tx=0x7fd97000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd9777fe640 1 -- 172.21.15.52:0/368220589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970017070 con 0x7fd980101af0 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd9777fe640 1 -- 172.21.15.52:0/368220589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9700054d0 con 0x7fd980101af0 2024-06-02T23:28:27.643 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98019faf0 con 0x7fd980101af0 2024-06-02T23:28:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd9777fe640 1 -- 172.21.15.52:0/368220589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970004cf0 con 0x7fd980101af0 2024-06-02T23:28:27.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.639+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9801a0040 con 0x7fd980101af0 2024-06-02T23:28:27.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd9777fe640 1 -- 172.21.15.52:0/368220589 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd970004e90 con 0x7fd980101af0 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 Processor -- start 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 -- start start 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9ce420 0x55658b9ce840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d3910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658b9d6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55658b92d4f0 con 0x55658b9cee10 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55658b8cb870 con 0x55658b9ce420 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.642+0000 7fd98b191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55658b9294b0 con 0x55658b9d3f80 2024-06-02T23:28:27.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986a25640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9ce420 0x55658b9ce840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986a25640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9ce420 0x55658b9ce840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:48968/0 (socket says 172.21.15.52:48968) 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48718/0 (socket says 172.21.15.52:48718) 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986a25640 1 -- 172.21.15.52:0/3644943257 learned_addr learned my addr 172.21.15.52:0/3644943257 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd9777fe640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd96c0a2cb0 0x7fd96c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd985a23640 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658b9d6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 -- 172.21.15.52:0/3644943257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 msgr2=0x55658b9d6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986a25640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd96c0a2cb0 0x7fd96c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658b9d6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 -- 172.21.15.52:0/3644943257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9ce420 msgr2=0x55658b9ce840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9ce420 0x55658b9ce840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 -- 172.21.15.52:0/3644943257 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658b8e79b0 con 0x55658b9cee10 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd985a23640 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658b9d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:27.651 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd9777fe640 1 -- 172.21.15.52:0/368220589 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd9700ceab0 con 0x7fd980101af0 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986224640 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d3910 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd97c006010 tx=0x7fd97c032b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ffe89fe60337e50 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd986a25640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd96c0a2cb0 0x7fd96c0a5170 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55658b9cf670 tx=0x7fd978004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd974bfe640 1 -- 172.21.15.52:0/3644943257 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c03f070 con 0x55658b9cee10 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd974bfe640 1 -- 172.21.15.52:0/3644943257 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd97c0044d0 con 0x55658b9cee10 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.643+0000 7fd974bfe640 1 -- 172.21.15.52:0/3644943257 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c035070 con 0x55658b9cee10 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 -- 172.21.15.52:0/3644943257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 msgr2=0x55658b9d3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d3910 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fd97c006010 tx=0x7fd97c032b80 comp rx=0 tx=0).stop 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 -- 172.21.15.52:0/3644943257 shutdown_connections 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658b9d6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d3910 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 --2- 172.21.15.52:0/3644943257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9ce420 0x55658b9ce840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 -- 172.21.15.52:0/3644943257 >> 172.21.15.52:0/3644943257 conn(0x55658b9c5680 msgr2=0x55658b9c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 -- 172.21.15.52:0/3644943257 shutdown_connections 2024-06-02T23:28:27.652 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.644+0000 7fd98b191740 1 -- 172.21.15.52:0/3644943257 wait complete. 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 -- start start 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 Processor -- start 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9d8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9d3f80 0x55658b9dfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55658b9d6ed0 con 0x55658b9cee10 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55658b8b6ae0 con 0x55658b9d3f80 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55658b9d7050 con 0x55658b9ce420 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd98b191740 10 client.0 _ll_register_callbacks cb 0x55658b8ef9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48724/0 (socket says 172.21.15.52:48724) 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 -- 172.21.15.52:0/4204134366 learned_addr learned my addr 172.21.15.52:0/4204134366 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.653 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 -- 172.21.15.52:0/4204134366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 msgr2=0x55658b9d8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9d8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 -- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9d3f80 msgr2=0x55658b9dfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9d3f80 0x55658b9dfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 -- 172.21.15.52:0/4204134366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658b641210 con 0x55658b9cee10 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd985a23640 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d8b70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd970009610 tx=0x7fd97006b1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b616794c22b9aff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.645+0000 7fd969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd969ffb640 1 -- 172.21.15.52:0/4204134366 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd970020040 con 0x55658b9cee10 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd969ffb640 1 -- 172.21.15.52:0/4204134366 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd97009a790 con 0x55658b9cee10 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd97c0044d0 con 0x55658b9cee10 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd969ffb640 1 -- 172.21.15.52:0/4204134366 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd97009ae80 con 0x55658b9cee10 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 10 client.19947 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55658b70d620 con 0x55658b9cee10 2024-06-02T23:28:27.654 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd969ffb640 1 -- 172.21.15.52:0/4204134366 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd97c0044d0 con 0x55658b9cee10 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd969ffb640 1 -- 172.21.15.52:0/4204134366 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd9700bb0b0 con 0x55658b9cee10 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 10 client.19947 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 10 client.19947 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 20 client.19947 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658b9e4470 0x55658b9e68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 4 client.19947 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55658b940f10 con 0x55658b9e4470 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.646+0000 7fd986224640 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658b9e4470 0x55658b9e68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd986224640 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658b9e4470 0x55658b9e68c0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27b7db8e90800ebb server_cookie=5e812ffcece7bdab in_seq=0 out_seq=0 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd969ffb640 10 client.19947 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd969ffb640 1 -- 172.21.15.52:0/4204134366 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55658b940f10 con 0x55658b9e4470 2024-06-02T23:28:27.655 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd969ffb640 10 client.19947 handle_command_reply: tid=0 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd98b191740 1 client.19947 shutdown 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 msgr2=0x55658b9d8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd98b191740 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d8b70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7fd970009610 tx=0x7fd97006b1f0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 shutdown_connections 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.647+0000 7fd98b191740 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658b9e4470 0x55658b9e68c0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9d3f80 0x55658b9dfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9cee10 0x55658b9d8b70 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 1 --2- 172.21.15.52:0/4204134366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9d8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd969ffb640 0 client.19947 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 >> 172.21.15.52:0/4204134366 conn(0x55658b9c5680 msgr2=0x55658b9c47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 shutdown_connections 2024-06-02T23:28:27.656 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 1 -- 172.21.15.52:0/4204134366 wait complete. 2024-06-02T23:28:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.648+0000 7fd98b191740 20 client.19947 trim_cache size 0 max 16384 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.663+0000 7fd98b191740 1 Processor -- start 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.663+0000 7fd98b191740 1 -- start start 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.663+0000 7fd98b191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9c8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.663+0000 7fd98b191740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.663+0000 7fd98b191740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 0x55658b9c8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd98b191740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a6a20 con 0x55658b9d3f80 2024-06-02T23:28:27.668 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd98b191740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a8560 con 0x55658b9cee10 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd98b191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a70f0 con 0x55658b9ce420 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 0x55658b9c8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 0x55658b9c8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48728/0 (socket says 172.21.15.52:48728) 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 -- 172.21.15.52:0/274446624 learned_addr learned my addr 172.21.15.52:0/274446624 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd985a23640 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986224640 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 -- 172.21.15.52:0/274446624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 msgr2=0x55658b9c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 -- 172.21.15.52:0/274446624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 msgr2=0x55658b9dfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 -- 172.21.15.52:0/274446624 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55658b9cfae0 con 0x55658b9d3f80 2024-06-02T23:28:27.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd985a23640 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd986a25640 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 0x55658b9c8ad0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd9780123a0 tx=0x7fd978009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32a6e85a8d2630ea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd96bfff640 1 -- 172.21.15.52:0/274446624 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978024070 con 0x55658b9d3f80 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd96bfff640 1 -- 172.21.15.52:0/274446624 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd978010920 con 0x55658b9d3f80 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.664+0000 7fd96bfff640 1 -- 172.21.15.52:0/274446624 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780052f0 con 0x55658b9d3f80 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 -- 172.21.15.52:0/274446624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 msgr2=0x55658b9c8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 0x55658b9c8ad0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fd9780123a0 tx=0x7fd978009500 comp rx=0 tx=0).stop 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 -- 172.21.15.52:0/274446624 shutdown_connections 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9d3f80 0x55658b9c8ad0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 --2- 172.21.15.52:0/274446624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9ce420 0x55658b9c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 -- 172.21.15.52:0/274446624 >> 172.21.15.52:0/274446624 conn(0x55658b9c5680 msgr2=0x55658b9cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 -- 172.21.15.52:0/274446624 shutdown_connections 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 -- 172.21.15.52:0/274446624 wait complete. 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 -- start start 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.665+0000 7fd98b191740 1 Processor -- start 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 0x55658b9dc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658ba4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a7e90 con 0x55658b9ce420 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a77c0 con 0x55658b9cee10 2024-06-02T23:28:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9801a4a20 con 0x55658b9d3f80 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 10 client.0 _ll_register_callbacks cb 0x7fd970009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 0x55658b9dc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 0x55658b9dc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48732/0 (socket says 172.21.15.52:48732) 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 -- 172.21.15.52:0/2666419843 learned_addr learned my addr 172.21.15.52:0/2666419843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd985a23640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986a25640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658ba4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 -- 172.21.15.52:0/2666419843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 msgr2=0x55658ba4d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658ba4d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 -- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 msgr2=0x55658b9dced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 -- 172.21.15.52:0/2666419843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55658ba4d6d0 con 0x55658b9ce420 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986a25640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658ba4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd985a23640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd986224640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 0x55658b9dc990 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009900 tx=0x7fd97c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9f4c4528150546 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd969ffb640 1 -- 172.21.15.52:0/2666419843 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd97c04f070 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd969ffb640 1 -- 172.21.15.52:0/2666419843 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd97c004450 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd978010920 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd969ffb640 1 -- 172.21.15.52:0/2666419843 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd97c05b050 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 10 client.19953 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.666+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55658ba4e8f0 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd969ffb640 1 -- 172.21.15.52:0/2666419843 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd978010920 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd969ffb640 1 -- 172.21.15.52:0/2666419843 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd97c031780 con 0x55658b9ce420 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd98b191740 10 client.19953 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd98b191740 10 client.19953 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd98b191740 20 client.19953 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd98b191740 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658ba4eeb0 0x55658ba512a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd98b191740 4 client.19953 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:28:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55658b93ac20 con 0x55658ba4eeb0 2024-06-02T23:28:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.667+0000 7fd985a23640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658ba4eeb0 0x55658ba512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.668+0000 7fd985a23640 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658ba4eeb0 0x55658ba512a0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3e96cec3439ebb9 server_cookie=5e728374666004a7 in_seq=0 out_seq=0 2024-06-02T23:28:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.668+0000 7fd969ffb640 10 client.19953 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.736+0000 7fd969ffb640 1 -- 172.21.15.52:0/2666419843 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55658b93ac20 con 0x55658ba4eeb0 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.736+0000 7fd969ffb640 10 client.19953 handle_command_reply: tid=0 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.736+0000 7fd98b191740 1 client.19953 shutdown 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.736+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 msgr2=0x55658b9dc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.736+0000 7fd98b191740 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 0x55658b9dc990 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009900 tx=0x7fd97c005b50 comp rx=0 tx=0).stop 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 shutdown_connections 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55658ba4eeb0 0x55658ba512a0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55658b9d3f80 0x55658ba4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.738 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55658b9cee10 0x55658b9dced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 --2- 172.21.15.52:0/2666419843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55658b9ce420 0x55658b9dc990 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 >> 172.21.15.52:0/2666419843 conn(0x55658b9c5680 msgr2=0x55658b9cfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd969ffb640 0 client.19953 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 shutdown_connections 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 1 -- 172.21.15.52:0/2666419843 wait complete. 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.737+0000 7fd98b191740 20 client.19953 trim_cache size 0 max 16384 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:28:27.739 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:28:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd96c0a2cb0 msgr2=0x7fd96c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd96c0a2cb0 0x7fd96c0a5170 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x55658b9cf670 tx=0x7fd978004ab0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 msgr2=0x7fd980198120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 0x7fd980198120 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fd97000d770 tx=0x7fd97000dc30 comp rx=0 tx=0).stop 2024-06-02T23:28:27.740 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 shutdown_connections 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd96c0a2cb0 0x7fd96c0a5170 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9801024b0 0x7fd98019f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd980101af0 0x7fd980198120 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 --2- 172.21.15.52:0/368220589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd980100900 0x7fd980197be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 >> 172.21.15.52:0/368220589 conn(0x7fd9800fbe70 msgr2=0x7fd9800fd990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.738+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 shutdown_connections 2024-06-02T23:28:27.741 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:27.739+0000 7fd988488640 1 -- 172.21.15.52:0/368220589 wait complete. 2024-06-02T23:28:28.076 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:27 smithi052 ceph-mon[32340]: pgmap v928: 97 pgs: 97 active+clean; 372 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-06-02T23:28:28.115 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:28:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:27 smithi174 ceph-mon[35994]: pgmap v928: 97 pgs: 97 active+clean; 372 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-06-02T23:28:28.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:27 smithi123 ceph-mon[35977]: pgmap v928: 97 pgs: 97 active+clean; 372 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.1 MiB/s wr, 312 op/s 2024-06-02T23:28:28.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 23335 5.87 MB/sec execute 96 sec latency 131.666 ms 2024-06-02T23:28:28.490 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:29.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 23505 5.92 MB/sec execute 97 sec latency 64.062 ms 2024-06-02T23:28:29.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:29 smithi052 ceph-mon[32340]: pgmap v929: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-06-02T23:28:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:29 smithi174 ceph-mon[35994]: pgmap v929: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-06-02T23:28:30.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:29 smithi123 ceph-mon[35977]: pgmap v929: 97 pgs: 97 active+clean; 371 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.2 MiB/s wr, 241 op/s 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 -- 172.21.15.52:0/2580608607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 msgr2=0x7fcd3c06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 --2- 172.21.15.52:0/2580608607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c06ae80 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009990 tx=0x7fcd30009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 -- 172.21.15.52:0/2580608607 shutdown_connections 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 --2- 172.21.15.52:0/2580608607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c06b3c0 0x7fcd3c10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 --2- 172.21.15.52:0/2580608607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c1084f0 0x7fcd3c1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 --2- 172.21.15.52:0/2580608607 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c06ae80 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 -- 172.21.15.52:0/2580608607 >> 172.21.15.52:0/2580608607 conn(0x7fcd3c078300 msgr2=0x7fcd3c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 -- 172.21.15.52:0/2580608607 shutdown_connections 2024-06-02T23:28:30.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.359+0000 7fcd4118a700 1 -- 172.21.15.52:0/2580608607 wait complete. 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.360+0000 7fcd4118a700 1 Processor -- start 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.360+0000 7fcd4118a700 1 -- start start 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd4118a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd4118a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c06b3c0 0x7fcd3c101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd4118a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c1084f0 0x7fcd3c101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd4118a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c112670 con 0x7fcd3c06aa00 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd4118a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1124f0 con 0x7fcd3c06b3c0 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd4118a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1127f0 con 0x7fcd3c1084f0 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd3ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c101450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48752/0 (socket says 172.21.15.52:48752) 2024-06-02T23:28:30.363 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/1470371140 learned_addr learned my addr 172.21.15.52:0/1470371140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c06b3c0 0x7fcd3c101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.361+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c1084f0 0x7fcd3c101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3b59e700 1 -- 172.21.15.52:0/1470371140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c06b3c0 msgr2=0x7fcd3c101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c06b3c0 0x7fcd3c101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3b59e700 1 -- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 msgr2=0x7fcd3c101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3b59e700 1 -- 172.21.15.52:0/1470371140 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1ab5b0 con 0x7fcd3c1084f0 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c1084f0 0x7fcd3c101ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400efc0 tx=0x7fcd3400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd23fff700 1 -- 172.21.15.52:0/1470371140 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd340092e0 con 0x7fcd3c1084f0 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd23fff700 1 -- 172.21.15.52:0/1470371140 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd34004bb0 con 0x7fcd3c1084f0 2024-06-02T23:28:30.364 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd23fff700 1 -- 172.21.15.52:0/1470371140 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34004020 con 0x7fcd3c1084f0 2024-06-02T23:28:30.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1ab840 con 0x7fcd3c1084f0 2024-06-02T23:28:30.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.362+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c1abd00 con 0x7fcd3c1084f0 2024-06-02T23:28:30.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.365+0000 7fcd23fff700 1 -- 172.21.15.52:0/1470371140 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcd3401b030 con 0x7fcd3c1084f0 2024-06-02T23:28:30.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.366+0000 7fcd23fff700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd240a2ca0 0x7fcd240a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.366+0000 7fcd23fff700 1 -- 172.21.15.52:0/1470371140 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcd340ceac0 con 0x7fcd3c1084f0 2024-06-02T23:28:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.366+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd240a2ca0 0x7fcd240a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.367+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd240a2ca0 0x7fcd240a5160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c011550 tx=0x7fcd2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.368+0000 7fcd4ba0db80 1 Processor -- start 2024-06-02T23:28:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.368+0000 7fcd4ba0db80 1 -- start start 2024-06-02T23:28:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.368+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22640 0x559917c22a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.368+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c1c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.368+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c1d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559917b6e810 con 0x559917c22f60 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559917b16720 con 0x559917c1cf20 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559917b6a7d0 con 0x559917c22640 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c1c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c1c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48756/0 (socket says 172.21.15.52:48756) 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3ad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22640 0x559917c22a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3a59c700 1 -- 172.21.15.52:0/420486987 learned_addr learned my addr 172.21.15.52:0/420486987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c1d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/420486987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 msgr2=0x559917c1d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c1d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/420486987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 msgr2=0x559917c1c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c1c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/420486987 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559917b0cf50 con 0x559917c22640 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c1d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.369+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c1c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:30.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.370+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22640 0x559917c22a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00e660 tx=0x7fcd2c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f88f43b1a911788d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:30.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.370+0000 7fcd217fa700 1 -- 172.21.15.52:0/420486987 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00ee00 con 0x559917c22640 2024-06-02T23:28:30.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.370+0000 7fcd217fa700 1 -- 172.21.15.52:0/420486987 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd2c004ce0 con 0x559917c22640 2024-06-02T23:28:30.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.370+0000 7fcd217fa700 1 -- 172.21.15.52:0/420486987 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c01e680 con 0x559917c22640 2024-06-02T23:28:30.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/420486987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22640 msgr2=0x559917c22a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22640 0x559917c22a20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00e660 tx=0x7fcd2c014000 comp rx=0 tx=0).stop 2024-06-02T23:28:30.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/420486987 shutdown_connections 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c1d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22640 0x559917c22a20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/420486987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c1c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/420486987 >> 172.21.15.52:0/420486987 conn(0x559917b47cc0 msgr2=0x559917c17850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/420486987 shutdown_connections 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.371+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/420486987 wait complete. 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 -- start start 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 Processor -- start 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c26010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c26550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559917c299d0 con 0x559917c22640 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559917b0ebf0 con 0x559917c1cf20 2024-06-02T23:28:30.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559917c29b50 con 0x559917c22f60 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd4ba0db80 10 client.0 _ll_register_callbacks cb 0x559917b08920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd3b59e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c26550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.372+0000 7fcd3b59e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c26550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36478/0 (socket says 172.21.15.52:36478) 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 -- 172.21.15.52:0/2199319144 learned_addr learned my addr 172.21.15.52:0/2199319144 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 -- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 msgr2=0x559917c28d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c28d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 -- 172.21.15.52:0/2199319144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 msgr2=0x559917c26010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c26010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c26010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 -- 172.21.15.52:0/2199319144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559917abd8d0 con 0x559917c22f60 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c26010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:30.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c26550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c6f0 tx=0x7fcd3409b8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fea43dd1271fea65 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd1eff5700 1 -- 172.21.15.52:0/2199319144 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3409a9d0 con 0x559917c22f60 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2c004ce0 con 0x559917c22f60 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.373+0000 7fcd1eff5700 1 -- 172.21.15.52:0/2199319144 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcd340bb0b0 con 0x559917c22f60 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd1eff5700 1 -- 172.21.15.52:0/2199319144 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcd3409f730 con 0x559917c22f60 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 10 client.28954 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559917c34360 con 0x559917c22f60 2024-06-02T23:28:30.377 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd1eff5700 1 -- 172.21.15.52:0/2199319144 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd2c004ce0 con 0x559917c22f60 2024-06-02T23:28:30.378 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd1eff5700 1 -- 172.21.15.52:0/2199319144 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcd340970d0 con 0x559917c22f60 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 10 client.28954 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 10 client.28954 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 20 client.28954 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917c38c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 4 client.28954 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559917b7bd10 con 0x559917c36830 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.374+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917c38c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.375+0000 7fcd1eff5700 10 client.28954 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.375+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917c38c20 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6ec4fc82d2626d1 server_cookie=68207cbf393b7f43 in_seq=0 out_seq=0 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd1eff5700 1 -- 172.21.15.52:0/2199319144 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559917b7bd10 con 0x559917c36830 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd1eff5700 10 client.28954 handle_command_reply: tid=0 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 client.28954 shutdown 2024-06-02T23:28:30.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 msgr2=0x559917c26550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c26550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400c6f0 tx=0x7fcd3409b8c0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 shutdown_connections 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917c38c20 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd1eff5700 0 client.28954 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c26550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c28d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2199319144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c26010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 >> 172.21.15.52:0/2199319144 conn(0x559917b47cc0 msgr2=0x559917c1b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 shutdown_connections 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.376+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2199319144 wait complete. 2024-06-02T23:28:30.380 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.377+0000 7fcd4ba0db80 20 client.28954 trim_cache size 0 max 16384 2024-06-02T23:28:30.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.399+0000 7fcd4ba0db80 1 Processor -- start 2024-06-02T23:28:30.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.399+0000 7fcd4ba0db80 1 -- start start 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c1cf20 0x559917c1c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 0x559917c20740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c23a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b30e0 con 0x559917c22f60 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b4c20 con 0x559917c22640 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd4ba0db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b37b0 con 0x559917c1cf20 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3ad9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c1cf20 0x559917c1c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 0x559917c20740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 0x559917c20740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49030/0 (socket says 172.21.15.52:49030) 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 -- 172.21.15.52:0/2877570891 learned_addr learned my addr 172.21.15.52:0/2877570891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:30.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c23a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 -- 172.21.15.52:0/2877570891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c1cf20 msgr2=0x559917c1c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c1cf20 0x559917c1c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 -- 172.21.15.52:0/2877570891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 msgr2=0x559917c23a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c23a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3a59c700 1 -- 172.21.15.52:0/2877570891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559917c24580 con 0x559917c22640 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.400+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c1cf20 0x559917c1c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:30.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.401+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 0x559917c20740 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009750 tx=0x7fcd3001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2f30025d354e5db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:30.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.401+0000 7fcd20ff9700 1 -- 172.21.15.52:0/2877570891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30029070 con 0x559917c22640 2024-06-02T23:28:30.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.401+0000 7fcd20ff9700 1 -- 172.21.15.52:0/2877570891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd30005640 con 0x559917c22640 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.401+0000 7fcd20ff9700 1 -- 172.21.15.52:0/2877570891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30003cf0 con 0x559917c22640 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2877570891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 msgr2=0x559917c20740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 0x559917c20740 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009750 tx=0x7fcd3001a040 comp rx=0 tx=0).stop 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2877570891 shutdown_connections 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22f60 0x559917c23a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c1cf20 0x559917c1c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/2877570891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c22640 0x559917c20740 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2877570891 >> 172.21.15.52:0/2877570891 conn(0x559917b47cc0 msgr2=0x559917c23340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2877570891 shutdown_connections 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/2877570891 wait complete. 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 -- start start 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.402+0000 7fcd4ba0db80 1 Processor -- start 2024-06-02T23:28:30.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c2f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c2f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c3dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd3ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c2f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd3ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c2f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:49032/0 (socket says 172.21.15.52:49032) 2024-06-02T23:28:30.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/1978788714 learned_addr learned my addr 172.21.15.52:0/1978788714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:30.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b4550 con 0x559917c22640 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b3e80 con 0x559917c1cf20 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c1b10a0 con 0x559917c22f60 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd4ba0db80 10 client.0 _ll_register_callbacks cb 0x559917b08920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c2f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.403+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c3dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/1978788714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 msgr2=0x559917c3dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.408 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c3dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/1978788714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 msgr2=0x559917c2f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c2f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3ad9d700 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559917c3e2f0 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3b59e700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd3ad9d700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c2f0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00e630 tx=0x7fcd2c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b569a0747da6c673 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd1eff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd1eff5700 1 -- 172.21.15.52:0/1978788714 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c01e630 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd1eff5700 1 -- 172.21.15.52:0/1978788714 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcd2c01e7d0 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.404+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd30005640 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd1eff5700 1 -- 172.21.15.52:0/1978788714 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcd2c01e9d0 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 10 client.29315 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559917ca1050 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd1eff5700 1 -- 172.21.15.52:0/1978788714 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcd30005640 con 0x559917c1cf20 2024-06-02T23:28:30.409 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd1eff5700 1 -- 172.21.15.52:0/1978788714 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcd2c039020 con 0x559917c1cf20 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 10 client.29315 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 10 client.29315 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 20 client.29315 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917ca3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 4 client.29315 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559917b76290 con 0x559917c36830 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.405+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917ca3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd3a59c700 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917ca3410 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc768fa7a5e5339 server_cookie=645c0aa29c0de9a9 in_seq=0 out_seq=0 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd1eff5700 10 client.29315 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd1eff5700 1 -- 172.21.15.52:0/1978788714 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559917b76290 con 0x559917c36830 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd1eff5700 10 client.29315 handle_command_reply: tid=0 2024-06-02T23:28:30.410 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 client.29315 shutdown 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 msgr2=0x559917c2f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c2f0f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00e630 tx=0x7fcd2c005d30 comp rx=0 tx=0).stop 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 shutdown_connections 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559917c36830 0x559917ca3410 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd1eff5700 0 client.29315 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559917c22f60 0x559917c3dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559917c1cf20 0x559917c2f0f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.406+0000 7fcd4ba0db80 1 --2- 172.21.15.52:0/1978788714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559917c22640 0x559917c2f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.407+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 >> 172.21.15.52:0/1978788714 conn(0x559917b47cc0 msgr2=0x559917c23340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.407+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 shutdown_connections 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.407+0000 7fcd4ba0db80 1 -- 172.21.15.52:0/1978788714 wait complete. 2024-06-02T23:28:30.411 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.407+0000 7fcd4ba0db80 20 client.29315 trim_cache size 0 max 16384 2024-06-02T23:28:30.412 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:28:30.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd240a2ca0 msgr2=0x7fcd240a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd240a2ca0 0x7fcd240a5160 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c011550 tx=0x7fcd2c009260 comp rx=0 tx=0).stop 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c1084f0 msgr2=0x7fcd3c101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c1084f0 0x7fcd3c101ed0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400efc0 tx=0x7fcd3400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 shutdown_connections 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcd240a2ca0 0x7fcd240a5160 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcd3c1084f0 0x7fcd3c101ed0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcd3c06aa00 0x7fcd3c101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 --2- 172.21.15.52:0/1470371140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd3c06b3c0 0x7fcd3c101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 >> 172.21.15.52:0/1470371140 conn(0x7fcd3c078300 msgr2=0x7fcd3c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 shutdown_connections 2024-06-02T23:28:30.413 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:30.408+0000 7fcd4118a700 1 -- 172.21.15.52:0/1470371140 wait complete. 2024-06-02T23:28:30.458 INFO:tasks.workunit.client.0.smithi052.stdout: 10 23686 5.95 MB/sec execute 98 sec latency 74.525 ms 2024-06-02T23:28:31.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 23864 6.03 MB/sec execute 99 sec latency 53.909 ms 2024-06-02T23:28:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:31 smithi174 ceph-mon[35994]: pgmap v930: 97 pgs: 97 active+clean; 372 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-06-02T23:28:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:31 smithi052 ceph-mon[32340]: pgmap v930: 97 pgs: 97 active+clean; 372 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-06-02T23:28:32.233 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:28:32.233 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:28:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:31 smithi123 ceph-mon[35977]: pgmap v930: 97 pgs: 97 active+clean; 372 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.5 MiB/s wr, 271 op/s 2024-06-02T23:28:32.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 24088 6.19 MB/sec execute 100 sec latency 60.850 ms 2024-06-02T23:28:32.599 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:33.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 24327 6.28 MB/sec execute 101 sec latency 55.591 ms 2024-06-02T23:28:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:33 smithi174 ceph-mon[35994]: pgmap v931: 97 pgs: 97 active+clean; 377 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.5 MiB/s wr, 294 op/s 2024-06-02T23:28:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:33 smithi052 ceph-mon[32340]: pgmap v931: 97 pgs: 97 active+clean; 377 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.5 MiB/s wr, 294 op/s 2024-06-02T23:28:34.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:33 smithi123 ceph-mon[35977]: pgmap v931: 97 pgs: 97 active+clean; 377 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.5 MiB/s wr, 294 op/s 2024-06-02T23:28:34.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 24461 6.31 MB/sec execute 102 sec latency 94.495 ms 2024-06-02T23:28:34.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 -- 172.21.15.52:0/1537596512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410107400 msgr2=0x7fc4101077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 --2- 172.21.15.52:0/1537596512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410107400 0x7fc4101077e0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fc400009990 tx=0x7fc400009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 -- 172.21.15.52:0/1537596512 shutdown_connections 2024-06-02T23:28:34.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 --2- 172.21.15.52:0/1537596512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101d50 0x7fc41010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 --2- 172.21.15.52:0/1537596512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410107400 0x7fc4101077e0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 --2- 172.21.15.52:0/1537596512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410101390 0x7fc410101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 -- 172.21.15.52:0/1537596512 >> 172.21.15.52:0/1537596512 conn(0x7fc410078340 msgr2=0x7fc410078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:34.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 -- 172.21.15.52:0/1537596512 shutdown_connections 2024-06-02T23:28:34.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.637+0000 7fc416b45700 1 -- 172.21.15.52:0/1537596512 wait complete. 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.638+0000 7fc416b45700 1 Processor -- start 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.638+0000 7fc416b45700 1 -- start start 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc416b45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101390 0x7fc4101999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc416b45700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 0x7fc410199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc416b45700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410107400 0x7fc4101a4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc416b45700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc410111570 con 0x7fc410101d50 2024-06-02T23:28:34.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc416b45700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc4101113f0 con 0x7fc410101390 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc416b45700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4101116f0 con 0x7fc410107400 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 0x7fc410199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc4150e2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410107400 0x7fc4101a4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 0x7fc410199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51814/0 (socket says 172.21.15.52:51814) 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 -- 172.21.15.52:0/643527957 learned_addr learned my addr 172.21.15.52:0/643527957 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 -- 172.21.15.52:0/643527957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410107400 msgr2=0x7fc4101a4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410107400 0x7fc4101a4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc4148e1700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101390 0x7fc4101999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 -- 172.21.15.52:0/643527957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101390 msgr2=0x7fc4101999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101390 0x7fc4101999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc40ffff700 1 -- 172.21.15.52:0/643527957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4101a4fb0 con 0x7fc410101d50 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.639+0000 7fc4150e2700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410107400 0x7fc4101a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:34.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc40ffff700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 0x7fc410199f30 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e9a0 tx=0x7fc40400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:34.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc4148e1700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101390 0x7fc4101999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:34.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc40dffb700 1 -- 172.21.15.52:0/643527957 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40400cbd0 con 0x7fc410101d50 2024-06-02T23:28:34.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4101a51e0 con 0x7fc410101d50 2024-06-02T23:28:34.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4101a5770 con 0x7fc410101d50 2024-06-02T23:28:34.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc40dffb700 1 -- 172.21.15.52:0/643527957 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40400cd70 con 0x7fc410101d50 2024-06-02T23:28:34.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.640+0000 7fc40dffb700 1 -- 172.21.15.52:0/643527957 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404018680 con 0x7fc410101d50 2024-06-02T23:28:34.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.644+0000 7fc40dffb700 1 -- 172.21.15.52:0/643527957 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc404018820 con 0x7fc410101d50 2024-06-02T23:28:34.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.644+0000 7fc40dffb700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc3f80a2ca0 0x7fc3f80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.645+0000 7fc40dffb700 1 -- 172.21.15.52:0/643527957 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc4040cf160 con 0x7fc410101d50 2024-06-02T23:28:34.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.645+0000 7fc4148e1700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc3f80a2ca0 0x7fc3f80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.645+0000 7fc4213c8b80 1 Processor -- start 2024-06-02T23:28:34.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.645+0000 7fc4148e1700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc3f80a2ca0 0x7fc3f80a5160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc40000b2e0 tx=0x7fc400009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:34.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.645+0000 7fc4213c8b80 1 -- start start 2024-06-02T23:28:34.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.646+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.646+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6bd10 0x563999a656f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.646+0000 7fc4148e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.646+0000 7fc4148e1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a6b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51816/0 (socket says 172.21.15.52:51816) 2024-06-02T23:28:34.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4148e1700 1 -- 172.21.15.52:0/29704131 learned_addr learned my addr 172.21.15.52:0/29704131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:34.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a65c30 0x563999a66050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563999953fe0 con 0x563999a6b3f0 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc40ffff700 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6bd10 0x563999a656f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639999b28d0 con 0x563999a65c30 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4150e2700 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a65c30 0x563999a66050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56399991a4e0 con 0x563999a6bd10 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc40ffff700 1 -- 172.21.15.52:0/29704131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a65c30 msgr2=0x563999a66050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc40ffff700 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a65c30 0x563999a66050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc40ffff700 1 -- 172.21.15.52:0/29704131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 msgr2=0x563999a6b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc40ffff700 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a6b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc40ffff700 1 -- 172.21.15.52:0/29704131 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563999901950 con 0x563999a6bd10 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.647+0000 7fc4150e2700 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a65c30 0x563999a66050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.648+0000 7fc40ffff700 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6bd10 0x563999a656f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc404010df0 tx=0x7fc40409b350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eba26b4039d9cca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.648+0000 7fc3f6ffd700 1 -- 172.21.15.52:0/29704131 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40409b9f0 con 0x563999a6bd10 2024-06-02T23:28:34.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.648+0000 7fc3f6ffd700 1 -- 172.21.15.52:0/29704131 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40409fda0 con 0x563999a6bd10 2024-06-02T23:28:34.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.648+0000 7fc3f6ffd700 1 -- 172.21.15.52:0/29704131 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404018d70 con 0x563999a6bd10 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6bd10 msgr2=0x563999a656f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6bd10 0x563999a656f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc404010df0 tx=0x7fc40409b350 comp rx=0 tx=0).stop 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 shutdown_connections 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a65c30 0x563999a66050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a6b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/29704131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6bd10 0x563999a656f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 >> 172.21.15.52:0/29704131 conn(0x56399998e100 msgr2=0x563999a5c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 shutdown_connections 2024-06-02T23:28:34.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.649+0000 7fc4213c8b80 1 -- 172.21.15.52:0/29704131 wait complete. 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.650+0000 7fc4213c8b80 1 -- start start 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.650+0000 7fc4213c8b80 1 Processor -- start 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a65c30 0x563999a71a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6b3f0 0x563999a6ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a6f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563999953040 con 0x563999a65c30 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5639999ae890 con 0x563999a6bd10 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563999a726f0 con 0x563999a6b3f0 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a6f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a6f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60734/0 (socket says 172.21.15.52:60734) 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 -- 172.21.15.52:0/4168580466 learned_addr learned my addr 172.21.15.52:0/4168580466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:34.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc40ffff700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6b3f0 0x563999a6ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4213c8b80 10 client.0 _ll_register_callbacks cb 0x563999948630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4148e1700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a65c30 0x563999a71a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 -- 172.21.15.52:0/4168580466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6b3f0 msgr2=0x563999a6ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6b3f0 0x563999a6ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 -- 172.21.15.52:0/4168580466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a65c30 msgr2=0x563999a71a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a65c30 0x563999a71a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 -- 172.21.15.52:0/4168580466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639999787d0 con 0x563999a6bd10 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc4150e2700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a6f340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc408007fb0 tx=0x7fc4080132b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ddb92c8bd960bda server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.651+0000 7fc3effff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc3effff700 1 -- 172.21.15.52:0/4168580466 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408004bb0 con 0x563999a6bd10 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc3effff700 1 -- 172.21.15.52:0/4168580466 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc408004d50 con 0x563999a6bd10 2024-06-02T23:28:34.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc40409fda0 con 0x563999a6bd10 2024-06-02T23:28:34.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc3effff700 1 -- 172.21.15.52:0/4168580466 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc408003a00 con 0x563999a6bd10 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc4213c8b80 10 client.29324 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563999a7cf30 con 0x563999a6bd10 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc3effff700 1 -- 172.21.15.52:0/4168580466 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc40409fda0 con 0x563999a6bd10 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc3effff700 1 -- 172.21.15.52:0/4168580466 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc4080046e0 con 0x563999a6bd10 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc4213c8b80 10 client.29324 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc4213c8b80 10 client.29324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.652+0000 7fc4213c8b80 20 client.29324 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.653+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999a81a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.653+0000 7fc4213c8b80 4 client.29324 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.653+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639999bfdd0 con 0x563999a7f640 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.653+0000 7fc4148e1700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999a81a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.653+0000 7fc4148e1700 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999a81a80 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a468012cbeb2b82 server_cookie=70f3ecf2f8c49359 in_seq=0 out_seq=0 2024-06-02T23:28:34.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.653+0000 7fc3effff700 10 client.29324 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.654+0000 7fc3effff700 1 -- 172.21.15.52:0/4168580466 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5639999bfdd0 con 0x563999a7f640 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.654+0000 7fc3effff700 10 client.29324 handle_command_reply: tid=0 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.654+0000 7fc4213c8b80 1 client.29324 shutdown 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.654+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 msgr2=0x563999a6f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a6f340 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fc408007fb0 tx=0x7fc4080132b0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 shutdown_connections 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999a81a80 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a6f340 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a65c30 0x563999a71a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc3effff700 0 client.29324 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/4168580466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a6b3f0 0x563999a6ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 >> 172.21.15.52:0/4168580466 conn(0x56399998e100 msgr2=0x563999a5b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:34.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 shutdown_connections 2024-06-02T23:28:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 1 -- 172.21.15.52:0/4168580466 wait complete. 2024-06-02T23:28:34.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.655+0000 7fc4213c8b80 20 client.29324 trim_cache size 0 max 16384 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 Processor -- start 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 -- start start 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a6f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a5bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a5c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00d2a0 con 0x563999a6b3f0 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00ede0 con 0x563999a6bd10 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00d970 con 0x563999a65c30 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.685+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a5bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a5bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51838/0 (socket says 172.21.15.52:51838) 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a5c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc40ffff700 1 -- 172.21.15.52:0/87161825 learned_addr learned my addr 172.21.15.52:0/87161825 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4148e1700 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a6f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 -- 172.21.15.52:0/87161825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 msgr2=0x563999a6f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a6f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 -- 172.21.15.52:0/87161825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 msgr2=0x563999a5bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a5bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 -- 172.21.15.52:0/87161825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563999a5da20 con 0x563999a6bd10 2024-06-02T23:28:34.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc40ffff700 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a5bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:34.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc4150e2700 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a5c440 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc408021680 tx=0x7fc408021780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d209f09fcd9a16 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:34.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc3f67fc700 1 -- 172.21.15.52:0/87161825 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408030070 con 0x563999a6bd10 2024-06-02T23:28:34.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc3f67fc700 1 -- 172.21.15.52:0/87161825 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc408021e20 con 0x563999a6bd10 2024-06-02T23:28:34.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.686+0000 7fc3f67fc700 1 -- 172.21.15.52:0/87161825 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40802b9a0 con 0x563999a6bd10 2024-06-02T23:28:34.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 -- 172.21.15.52:0/87161825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 msgr2=0x563999a5c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a5c440 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc408021680 tx=0x7fc408021780 comp rx=0 tx=0).stop 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 -- 172.21.15.52:0/87161825 shutdown_connections 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6bd10 0x563999a5c440 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a6f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/87161825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6b3f0 0x563999a5bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.687+0000 7fc4213c8b80 1 -- 172.21.15.52:0/87161825 >> 172.21.15.52:0/87161825 conn(0x56399998e100 msgr2=0x563999a69ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.688+0000 7fc4213c8b80 1 -- 172.21.15.52:0/87161825 shutdown_connections 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.688+0000 7fc4213c8b80 1 -- 172.21.15.52:0/87161825 wait complete. 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.688+0000 7fc4213c8b80 1 -- start start 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.688+0000 7fc4213c8b80 1 Processor -- start 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.688+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a74980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6b3f0 0x563999a74ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4213c8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6bd10 0x563999a75400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00e710 con 0x563999a6bd10 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00e040 con 0x563999a6b3f0 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4213c8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc00b240 con 0x563999a65c30 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4213c8b80 10 client.0 _ll_register_callbacks cb 0x563999a665c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6b3f0 0x563999a74ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4150e2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6bd10 0x563999a75400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4150e2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6bd10 0x563999a75400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51850/0 (socket says 172.21.15.52:51850) 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4150e2700 1 -- 172.21.15.52:0/753812845 learned_addr learned my addr 172.21.15.52:0/753812845 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc4148e1700 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a74980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 -- 172.21.15.52:0/753812845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 msgr2=0x563999a74980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a74980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 -- 172.21.15.52:0/753812845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6bd10 msgr2=0x563999a75400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6bd10 0x563999a75400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 -- 172.21.15.52:0/753812845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563999aea540 con 0x563999a6b3f0 2024-06-02T23:28:34.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc40ffff700 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6b3f0 0x563999a74ec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc404009230 tx=0x7fc40400efb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7c910494377bf94 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.689+0000 7fc3effff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc3effff700 1 -- 172.21.15.52:0/753812845 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040b8070 con 0x563999a6b3f0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc3effff700 1 -- 172.21.15.52:0/753812845 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc40409b040 con 0x563999a6b3f0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc408021e20 con 0x563999a6b3f0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc3effff700 1 -- 172.21.15.52:0/753812845 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc40409fda0 con 0x563999a6b3f0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc4213c8b80 10 client.29330 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563999aeb860 con 0x563999a6b3f0 2024-06-02T23:28:34.693 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.690+0000 7fc3effff700 1 -- 172.21.15.52:0/753812845 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc408021e20 con 0x563999a6b3f0 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc3effff700 1 -- 172.21.15.52:0/753812845 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc404097740 con 0x563999a6b3f0 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4213c8b80 10 client.29330 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4213c8b80 10 client.29330 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4213c8b80 20 client.29330 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999aedb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4213c8b80 4 client.29330 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5639999ba350 con 0x563999a7f640 2024-06-02T23:28:34.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc4148e1700 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999aedb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.691+0000 7fc3effff700 10 client.29330 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.692+0000 7fc4148e1700 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999aedb50 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdb59cb0c5da390 server_cookie=6ad05512fbc3064d in_seq=0 out_seq=0 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc3effff700 1 -- 172.21.15.52:0/753812845 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5639999ba350 con 0x563999a7f640 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc3effff700 10 client.29330 handle_command_reply: tid=0 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 client.29330 shutdown 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6b3f0 msgr2=0x563999a74ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6b3f0 0x563999a74ec0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc404009230 tx=0x7fc40400efb0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 shutdown_connections 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563999a7f640 0x563999aedb50 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563999a6bd10 0x563999a75400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563999a65c30 0x563999a74980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc3effff700 0 client.29330 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 --2- 172.21.15.52:0/753812845 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563999a6b3f0 0x563999a74ec0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 >> 172.21.15.52:0/753812845 conn(0x56399998e100 msgr2=0x563999a6a760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 shutdown_connections 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 1 -- 172.21.15.52:0/753812845 wait complete. 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.693+0000 7fc4213c8b80 20 client.29330 trim_cache size 0 max 16384 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "c101980c-a202-471a-b6cc-349bfb27df6b", 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:28:34.697 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc3f80a2ca0 msgr2=0x7fc3f80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc3f80a2ca0 0x7fc3f80a5160 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc40000b2e0 tx=0x7fc400009510 comp rx=0 tx=0).stop 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 msgr2=0x7fc410199f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 0x7fc410199f30 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e9a0 tx=0x7fc40400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 shutdown_connections 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc3f80a2ca0 0x7fc3f80a5160 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc410107400 0x7fc4101a4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc410101390 0x7fc4101999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 --2- 172.21.15.52:0/643527957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc410101d50 0x7fc410199f30 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:34.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 >> 172.21.15.52:0/643527957 conn(0x7fc410078340 msgr2=0x7fc4101000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:34.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 shutdown_connections 2024-06-02T23:28:34.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:34.694+0000 7fc416b45700 1 -- 172.21.15.52:0/643527957 wait complete. 2024-06-02T23:28:35.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 24550 6.26 MB/sec execute 103 sec latency 87.132 ms 2024-06-02T23:28:35.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:28:35.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: pgmap v932: 97 pgs: 97 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-06-02T23:28:35.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:28:35.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:35.902 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.027 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:28:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: pgmap v932: 97 pgs: 97 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-06-02T23:28:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: pgmap v932: 97 pgs: 97 active+clean; 379 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 995 KiB/s rd, 3.4 MiB/s wr, 258 op/s 2024-06-02T23:28:36.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.260 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.260 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:28:36.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 24704 6.32 MB/sec execute 104 sec latency 78.691 ms 2024-06-02T23:28:37.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 24863 6.34 MB/sec execute 105 sec latency 91.663 ms 2024-06-02T23:28:37.952 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:37 smithi052 ceph-mon[32340]: pgmap v933: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-06-02T23:28:38.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 -- 172.21.15.52:0/3926617359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378100e50 msgr2=0x7fe3781012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 --2- 172.21.15.52:0/3926617359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378100e50 0x7fe3781012d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009990 tx=0x7fe36c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 -- 172.21.15.52:0/3926617359 shutdown_connections 2024-06-02T23:28:38.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 --2- 172.21.15.52:0/3926617359 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378101810 0x7fe37810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 --2- 172.21.15.52:0/3926617359 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378107570 0x7fe378107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 --2- 172.21.15.52:0/3926617359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378100e50 0x7fe3781012d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 -- 172.21.15.52:0/3926617359 >> 172.21.15.52:0/3926617359 conn(0x7fe3780766b0 msgr2=0x7fe378076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:38.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 -- 172.21.15.52:0/3926617359 shutdown_connections 2024-06-02T23:28:38.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.116+0000 7fe37da9a700 1 -- 172.21.15.52:0/3926617359 wait complete. 2024-06-02T23:28:38.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.117+0000 7fe37da9a700 1 Processor -- start 2024-06-02T23:28:38.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.117+0000 7fe37da9a700 1 -- start start 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.117+0000 7fe37da9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 0x7fe3781a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.117+0000 7fe37da9a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 0x7fe3781a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.117+0000 7fe37da9a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 0x7fe37810e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 0x7fe3781a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe377fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 0x7fe37810e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 0x7fe3781a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 0x7fe3781a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:46982/0 (socket says 172.21.15.52:46982) 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe377fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 0x7fe37810e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51862/0 (socket says 172.21.15.52:51862) 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 0x7fe3781a4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60772/0 (socket says 172.21.15.52:60772) 2024-06-02T23:28:38.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 -- 172.21.15.52:0/799179403 learned_addr learned my addr 172.21.15.52:0/799179403 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781126d0 con 0x7fe378107570 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe378112550 con 0x7fe378101810 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe378112850 con 0x7fe378100e50 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 -- 172.21.15.52:0/799179403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 msgr2=0x7fe3781a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 0x7fe3781a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 -- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 msgr2=0x7fe37810e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 0x7fe37810e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe3777fe700 1 -- 172.21.15.52:0/799179403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe37810ebc0 con 0x7fe378100e50 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.118+0000 7fe377fff700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 0x7fe37810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe376ffd700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 0x7fe3781a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:38.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe3777fe700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 0x7fe3781a3e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e950 tx=0x7fe36800ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:38.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe374ff9700 1 -- 172.21.15.52:0/799179403 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368004230 con 0x7fe378100e50 2024-06-02T23:28:38.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe374ff9700 1 -- 172.21.15.52:0/799179403 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe3680043d0 con 0x7fe378100e50 2024-06-02T23:28:38.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe374ff9700 1 -- 172.21.15.52:0/799179403 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe368003ca0 con 0x7fe378100e50 2024-06-02T23:28:38.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37810eeb0 con 0x7fe378100e50 2024-06-02T23:28:38.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.119+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3781abd40 con 0x7fe378100e50 2024-06-02T23:28:38.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.122+0000 7fe374ff9700 1 -- 172.21.15.52:0/799179403 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe36800c7c0 con 0x7fe378100e50 2024-06-02T23:28:38.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.123+0000 7fe374ff9700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3600a2ca0 0x7fe3600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.123+0000 7fe374ff9700 1 -- 172.21.15.52:0/799179403 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe3680cf220 con 0x7fe378100e50 2024-06-02T23:28:38.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.124+0000 7fe376ffd700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3600a2ca0 0x7fe3600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.124+0000 7fe38831db80 1 Processor -- start 2024-06-02T23:28:38.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 -- start start 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe376ffd700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3600a2ca0 0x7fe3600a5160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe37810f5f0 tx=0x7fe36c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de653ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de65389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6539340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555de648a810 con 0x555de6538f20 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555de6432720 con 0x555de653ef60 2024-06-02T23:28:38.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe38831db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555de64867d0 con 0x555de653e640 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de65389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe377fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6539340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe377fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6539340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51878/0 (socket says 172.21.15.52:51878) 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe377fff700 1 -- 172.21.15.52:0/3111597646 learned_addr learned my addr 172.21.15.52:0/3111597646 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.125+0000 7fe3777fe700 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de653ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe3777fe700 1 -- 172.21.15.52:0/3111597646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 msgr2=0x555de65389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe3777fe700 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de65389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe3777fe700 1 -- 172.21.15.52:0/3111597646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 msgr2=0x555de6539340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe3777fe700 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6539340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe3777fe700 1 -- 172.21.15.52:0/3111597646 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de6428f50 con 0x555de653e640 2024-06-02T23:28:38.128 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe377fff700 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6539340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:38.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe3777fe700 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de653ea20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3680ccd50 tx=0x7fe36809b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c87d71a625874bb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:38.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe35dffb700 1 -- 172.21.15.52:0/3111597646 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3680b3020 con 0x555de653e640 2024-06-02T23:28:38.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe35dffb700 1 -- 172.21.15.52:0/3111597646 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36809c1b0 con 0x555de653e640 2024-06-02T23:28:38.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.126+0000 7fe35dffb700 1 -- 172.21.15.52:0/3111597646 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36800cab0 con 0x555de653e640 2024-06-02T23:28:38.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 -- 172.21.15.52:0/3111597646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 msgr2=0x555de653ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de653ea20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fe3680ccd50 tx=0x7fe36809b880 comp rx=0 tx=0).stop 2024-06-02T23:28:38.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 -- 172.21.15.52:0/3111597646 shutdown_connections 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6539340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de653ea20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 --2- 172.21.15.52:0/3111597646 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de65389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.127+0000 7fe38831db80 1 -- 172.21.15.52:0/3111597646 >> 172.21.15.52:0/3111597646 conn(0x555de6463cc0 msgr2=0x555de6533850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.128+0000 7fe38831db80 1 -- 172.21.15.52:0/3111597646 shutdown_connections 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.128+0000 7fe38831db80 1 -- 172.21.15.52:0/3111597646 wait complete. 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.128+0000 7fe38831db80 1 -- start start 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.128+0000 7fe38831db80 1 Processor -- start 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6544cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de6541ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.132 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de6542530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555de65459d0 con 0x555de6538f20 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555de642abf0 con 0x555de653ef60 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555de6545b50 con 0x555de653e640 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe38831db80 10 client.0 _ll_register_callbacks cb 0x555de6424920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de6541ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de6541ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47000/0 (socket says 172.21.15.52:47000) 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 -- 172.21.15.52:0/1101781764 learned_addr learned my addr 172.21.15.52:0/1101781764 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe377fff700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de6542530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6544cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 -- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 msgr2=0x555de6542530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de6542530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 -- 172.21.15.52:0/1101781764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 msgr2=0x555de6544cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.133 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.129+0000 7fe376ffd700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6544cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe376ffd700 1 -- 172.21.15.52:0/1101781764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de63d98d0 con 0x555de653e640 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe376ffd700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de6541ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005cd0 tx=0x7fe36c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b870ea14186eb029 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe356ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe356ffd700 1 -- 172.21.15.52:0/1101781764 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c046060 con 0x555de653e640 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36809c1b0 con 0x555de653e640 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe356ffd700 1 -- 172.21.15.52:0/1101781764 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe36c00f460 con 0x555de653e640 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.130+0000 7fe356ffd700 1 -- 172.21.15.52:0/1101781764 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe36c00f660 con 0x555de653e640 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 10 client.28990 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:38.134 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555de6550300 con 0x555de653e640 2024-06-02T23:28:38.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe356ffd700 1 -- 172.21.15.52:0/1101781764 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36809c1b0 con 0x555de653e640 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe356ffd700 1 -- 172.21.15.52:0/1101781764 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe36c040070 con 0x555de653e640 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 10 client.28990 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 10 client.28990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 20 client.28990 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de6552c40 0x555de6555060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 4 client.28990 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.131+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555de6497d10 con 0x555de6552c40 2024-06-02T23:28:38.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.132+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de6552c40 0x555de6555060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.132+0000 7fe356ffd700 10 client.28990 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.132+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de6552c40 0x555de6555060 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45c047930e3637a3 server_cookie=36d6971f295d8928 in_seq=0 out_seq=0 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe356ffd700 1 -- 172.21.15.52:0/1101781764 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555de6497d10 con 0x555de6552c40 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe356ffd700 10 client.28990 handle_command_reply: tid=0 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 client.28990 shutdown 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 msgr2=0x555de6541ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de6541ff0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe36c005cd0 tx=0x7fe36c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 shutdown_connections 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de6552c40 0x555de6555060 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de6542530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de6544cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe356ffd700 0 client.28990 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 --2- 172.21.15.52:0/1101781764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de6541ff0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.137 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.134+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 >> 172.21.15.52:0/1101781764 conn(0x555de6463cc0 msgr2=0x555de65379d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:38.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.135+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 shutdown_connections 2024-06-02T23:28:38.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.135+0000 7fe38831db80 1 -- 172.21.15.52:0/1101781764 wait complete. 2024-06-02T23:28:38.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.135+0000 7fe38831db80 20 client.28990 trim_cache size 0 max 16384 2024-06-02T23:28:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:37 smithi174 ceph-mon[35994]: pgmap v933: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-06-02T23:28:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 Processor -- start 2024-06-02T23:28:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 -- start start 2024-06-02T23:28:38.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de65389e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de65339f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de653fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781acb90 con 0x555de6538f20 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe3780730c0 con 0x555de653ef60 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe38831db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3781ac4c0 con 0x555de653e640 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de65339f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de65339f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47016/0 (socket says 172.21.15.52:47016) 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe376ffd700 1 -- 172.21.15.52:0/1668480965 learned_addr learned my addr 172.21.15.52:0/1668480965 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.162+0000 7fe377fff700 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de653fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de65389e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe376ffd700 1 -- 172.21.15.52:0/1668480965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 msgr2=0x555de653fbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe376ffd700 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de653fbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe376ffd700 1 -- 172.21.15.52:0/1668480965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 msgr2=0x555de65389e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe376ffd700 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de65389e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe376ffd700 1 -- 172.21.15.52:0/1668480965 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555de65556c0 con 0x555de653e640 2024-06-02T23:28:38.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe376ffd700 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de65339f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009350 tx=0x7fe36c00bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ecf1017a516fe217 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe35d7fa700 1 -- 172.21.15.52:0/1668480965 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c046040 con 0x555de653e640 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.163+0000 7fe35d7fa700 1 -- 172.21.15.52:0/1668480965 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c009510 con 0x555de653e640 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.164+0000 7fe35d7fa700 1 -- 172.21.15.52:0/1668480965 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe36c026430 con 0x555de653e640 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 -- 172.21.15.52:0/1668480965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 msgr2=0x555de65339f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de65339f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fe36c009350 tx=0x7fe36c00bfd0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 -- 172.21.15.52:0/1668480965 shutdown_connections 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de653ef60 0x555de653fbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de6538f20 0x555de65389e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 --2- 172.21.15.52:0/1668480965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de65339f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 -- 172.21.15.52:0/1668480965 >> 172.21.15.52:0/1668480965 conn(0x555de6463cc0 msgr2=0x555de652ce00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 -- 172.21.15.52:0/1668480965 shutdown_connections 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 -- 172.21.15.52:0/1668480965 wait complete. 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.165+0000 7fe38831db80 1 -- start start 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 Processor -- start 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de6538f20 0x555de654ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de654b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 0x555de6547db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe378073e60 con 0x555de653ef60 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe378073790 con 0x555de6538f20 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe378071060 con 0x555de653e640 2024-06-02T23:28:38.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe38831db80 10 client.0 _ll_register_callbacks cb 0x555de6424920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe377fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 0x555de6547db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de654b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe377fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 0x555de6547db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51890/0 (socket says 172.21.15.52:51890) 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe376ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de654b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47032/0 (socket says 172.21.15.52:47032) 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe377fff700 1 -- 172.21.15.52:0/1720188361 learned_addr learned my addr 172.21.15.52:0/1720188361 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de6538f20 0x555de654ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe377fff700 1 -- 172.21.15.52:0/1720188361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 msgr2=0x555de654b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe377fff700 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de654b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.166+0000 7fe377fff700 1 -- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de6538f20 msgr2=0x555de654ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe377fff700 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de6538f20 0x555de654ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe377fff700 1 -- 172.21.15.52:0/1720188361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555de655b3f0 con 0x555de653ef60 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe377fff700 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 0x555de6547db0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe370004710 tx=0x7fe370004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725f24443f1146dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe356ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe356ffd700 1 -- 172.21.15.52:0/1720188361 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe370007bc0 con 0x555de653ef60 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe356ffd700 1 -- 172.21.15.52:0/1720188361 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe370007d60 con 0x555de653ef60 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe36c009510 con 0x555de653ef60 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe356ffd700 1 -- 172.21.15.52:0/1720188361 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe370004bb0 con 0x555de653ef60 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe38831db80 10 client.20004 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:38.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.167+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555de65bdda0 con 0x555de653ef60 2024-06-02T23:28:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe356ffd700 1 -- 172.21.15.52:0/1720188361 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe36c009510 con 0x555de653ef60 2024-06-02T23:28:38.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe356ffd700 1 -- 172.21.15.52:0/1720188361 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe37002b480 con 0x555de653ef60 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe38831db80 10 client.20004 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe38831db80 10 client.20004 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe38831db80 20 client.20004 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe38831db80 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de65be4e0 0x555de65c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe38831db80 4 client.20004 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555de6492290 con 0x555de65be4e0 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.168+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de65be4e0 0x555de65c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe3777fe700 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de65be4e0 0x555de65c0900 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b38e40dfb2780cf server_cookie=8b1b12ba106af1ff in_seq=0 out_seq=0 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe356ffd700 10 client.20004 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe356ffd700 1 -- 172.21.15.52:0/1720188361 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555de6492290 con 0x555de65be4e0 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe356ffd700 10 client.20004 handle_command_reply: tid=0 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 client.20004 shutdown 2024-06-02T23:28:38.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 msgr2=0x555de6547db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 0x555de6547db0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe370004710 tx=0x7fe370004a20 comp rx=0 tx=0).stop 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 shutdown_connections 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555de65be4e0 0x555de65c0900 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555de653ef60 0x555de6547db0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:28:38.174 INFO:teuthology.orchestra.run.smithi052.stdout: "c101980c-a202-471a-b6cc-349bfb27df6b": { 2024-06-02T23:28:38.175 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:28:38.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "c101980c-a202-471a-b6cc-349bfb27df6b", 2024-06-02T23:28:38.175 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:28:38.175 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:28:38.175 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:28:38.175 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555de6538f20 0x555de654ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe356ffd700 0 client.20004 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 --2- 172.21.15.52:0/1720188361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555de653e640 0x555de654b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.169+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 >> 172.21.15.52:0/1720188361 conn(0x555de6463cc0 msgr2=0x555de653fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.170+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 shutdown_connections 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.170+0000 7fe38831db80 1 -- 172.21.15.52:0/1720188361 wait complete. 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.170+0000 7fe38831db80 20 client.20004 trim_cache size 0 max 16384 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3600a2ca0 msgr2=0x7fe3600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3600a2ca0 0x7fe3600a5160 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fe37810f5f0 tx=0x7fe36c01a040 comp rx=0 tx=0).stop 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 msgr2=0x7fe3781a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 0x7fe3781a3e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe36800e950 tx=0x7fe36800ec60 comp rx=0 tx=0).stop 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 shutdown_connections 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe3600a2ca0 0x7fe3600a5160 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe378107570 0x7fe37810e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe378100e50 0x7fe3781a3e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 --2- 172.21.15.52:0/799179403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe378101810 0x7fe3781a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 >> 172.21.15.52:0/799179403 conn(0x7fe3780766b0 msgr2=0x7fe3780ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 shutdown_connections 2024-06-02T23:28:38.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:28:38.171+0000 7fe37da9a700 1 -- 172.21.15.52:0/799179403 wait complete. 2024-06-02T23:28:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:37 smithi123 ceph-mon[35977]: pgmap v933: 97 pgs: 97 active+clean; 388 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 349 op/s 2024-06-02T23:28:38.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25005 6.33 MB/sec execute 106 sec latency 108.102 ms 2024-06-02T23:28:38.956 INFO:tasks.cephfs.filesystem:scrub status for tag:c101980c-a202-471a-b6cc-349bfb27df6b - {'path': '/', 'tag': 'c101980c-a202-471a-b6cc-349bfb27df6b', 'options': 'recursive,force'} 2024-06-02T23:28:39.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25160 6.34 MB/sec execute 107 sec latency 54.012 ms 2024-06-02T23:28:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:39 smithi174 ceph-mon[35994]: pgmap v934: 97 pgs: 97 active+clean; 390 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2024-06-02T23:28:40.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:39 smithi052 ceph-mon[32340]: pgmap v934: 97 pgs: 97 active+clean; 390 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2024-06-02T23:28:40.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:39 smithi123 ceph-mon[35977]: pgmap v934: 97 pgs: 97 active+clean; 390 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 275 op/s 2024-06-02T23:28:40.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25309 6.33 MB/sec execute 108 sec latency 127.044 ms 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 -- 172.21.15.52:0/1220126467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4104db0 msgr2=0x7f2dd41071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1220126467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4104db0 0x7f2dd41071a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009900 tx=0x7f2dd002a410 comp rx=0 tx=0).stop 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 -- 172.21.15.52:0/1220126467 shutdown_connections 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1220126467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd41076e0 0x7f2dd4109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1220126467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4104db0 0x7f2dd41071a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1220126467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd4102480 0x7f2dd4104870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 -- 172.21.15.52:0/1220126467 >> 172.21.15.52:0/1220126467 conn(0x7f2dd40fbe70 msgr2=0x7f2dd40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 -- 172.21.15.52:0/1220126467 shutdown_connections 2024-06-02T23:28:40.876 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.875+0000 7f2ddc041640 1 -- 172.21.15.52:0/1220126467 wait complete. 2024-06-02T23:28:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 Processor -- start 2024-06-02T23:28:40.878 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 -- start start 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4102480 0x7f2dd4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd4104db0 0x7f2dd419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd41076e0 0x7f2dd41a1370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410f790 con 0x7f2dd4104db0 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410f610 con 0x7f2dd4102480 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.876+0000 7f2ddc041640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd410f910 con 0x7f2dd41076e0 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd4104db0 0x7f2dd419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd4104db0 0x7f2dd419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51896/0 (socket says 172.21.15.52:51896) 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dd95dc640 1 -- 172.21.15.52:0/1942536477 learned_addr learned my addr 172.21.15.52:0/1942536477 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4102480 0x7f2dd4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd41076e0 0x7f2dd41a1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 -- 172.21.15.52:0/1942536477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4102480 msgr2=0x7f2dd4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4102480 0x7f2dd4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 -- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd4104db0 msgr2=0x7f2dd419a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd4104db0 0x7f2dd419a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 -- 172.21.15.52:0/1942536477 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dd41a19e0 con 0x7f2dd41076e0 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.877+0000 7f2dda5de640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd41076e0 0x7f2dd41a1370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009fd0 tx=0x7f2dcc00ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.878+0000 7f2dcaffd640 1 -- 172.21.15.52:0/1942536477 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc00ce20 con 0x7f2dd41076e0 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.878+0000 7f2dcaffd640 1 -- 172.21.15.52:0/1942536477 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dcc005020 con 0x7f2dd41076e0 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.878+0000 7f2dcaffd640 1 -- 172.21.15.52:0/1942536477 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dcc010690 con 0x7f2dd41076e0 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.878+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dd41a1cd0 con 0x7f2dd41076e0 2024-06-02T23:28:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.878+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dd41a21d0 con 0x7f2dd41076e0 2024-06-02T23:28:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.881+0000 7f2dcaffd640 1 -- 172.21.15.52:0/1942536477 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2dcc0051d0 con 0x7f2dd41076e0 2024-06-02T23:28:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.881+0000 7f2ddee20740 1 Processor -- start 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.881+0000 7f2ddee20740 1 -- start start 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc11c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc16d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc19810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2ddee20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fa7cb6c4f0 con 0x55fa7cc12210 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2ddee20740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa7cb0a870 con 0x55fa7cc17380 2024-06-02T23:28:40.882 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2ddee20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa7cb684b0 con 0x55fa7cc11820 2024-06-02T23:28:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2dcaffd640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2dc00a2d00 0x7f2dc00a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc19810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc19810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:60822/0 (socket says 172.21.15.52:60822) 2024-06-02T23:28:40.883 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2dd95dc640 1 -- 172.21.15.52:0/3408362589 learned_addr learned my addr 172.21.15.52:0/3408362589 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2dcaffd640 1 -- 172.21.15.52:0/1942536477 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2dcc0cf660 con 0x7f2dd41076e0 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.882+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc16d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dda5de640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2dc00a2d00 0x7f2dc00a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc11c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/3408362589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 msgr2=0x55fa7cc11c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc11c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/3408362589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 msgr2=0x55fa7cc19810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc19810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/3408362589 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7cb269b0 con 0x55fa7cc12210 2024-06-02T23:28:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.883+0000 7f2dda5de640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2dc00a2d00 0x7f2dc00a51c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc01a040 tx=0x7f2dcc00c890 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.884+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc16d10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400bc40 tx=0x7f2dc40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee21a9a60fc148 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.884+0000 7f2dbffff640 1 -- 172.21.15.52:0/3408362589 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4004020 con 0x55fa7cc12210 2024-06-02T23:28:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.884+0000 7f2dbffff640 1 -- 172.21.15.52:0/3408362589 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc40041c0 con 0x55fa7cc12210 2024-06-02T23:28:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.884+0000 7f2dbffff640 1 -- 172.21.15.52:0/3408362589 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc40116a0 con 0x55fa7cc12210 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.885+0000 7f2ddee20740 1 -- 172.21.15.52:0/3408362589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 msgr2=0x55fa7cc16d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.885+0000 7f2ddee20740 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc16d10 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400bc40 tx=0x7f2dc40107a0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 -- 172.21.15.52:0/3408362589 shutdown_connections 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc19810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc16d10 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 --2- 172.21.15.52:0/3408362589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc11c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 -- 172.21.15.52:0/3408362589 >> 172.21.15.52:0/3408362589 conn(0x55fa7cc0ce90 msgr2=0x55fa7cc0d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 -- 172.21.15.52:0/3408362589 shutdown_connections 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 -- 172.21.15.52:0/3408362589 wait complete. 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 -- start start 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.886+0000 7f2ddee20740 1 Processor -- start 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc17380 0x55fa7cc23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.888 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fa7cc1a330 con 0x55fa7cc17380 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fa7caf5ae0 con 0x55fa7cc11820 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fa7cc1a4b0 con 0x55fa7cc12210 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2ddee20740 10 client.0 _ll_register_callbacks cb 0x55fa7cb2e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc17380 0x55fa7cc23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2dd9ddd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2dd9ddd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47068/0 (socket says 172.21.15.52:47068) 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/4104159270 learned_addr learned my addr 172.21.15.52:0/4104159270 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.887+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc11820 msgr2=0x55fa7cc1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/4104159270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc17380 msgr2=0x55fa7cc23080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc17380 0x55fa7cc23080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.889 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/4104159270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7c880210 con 0x55fa7cc12210 2024-06-02T23:28:40.890 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400bc10 tx=0x7f2dc4004020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e6d94dfdede16cd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.888+0000 7f2dbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.889+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/4104159270 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc402a060 con 0x55fa7cc12210 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.889+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/4104159270 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2dc4003a00 con 0x55fa7cc12210 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.889+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc40041c0 con 0x55fa7cc12210 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/4104159270 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2dc4005040 con 0x55fa7cc12210 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 10 client.29008 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa7c94c620 con 0x55fa7cc12210 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/4104159270 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dc40041c0 con 0x55fa7cc12210 2024-06-02T23:28:40.891 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/4104159270 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2dc4004c90 con 0x55fa7cc12210 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 10 client.29008 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 10 client.29008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 20 client.29008 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc27c40 0x55fa7cc2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 4 client.29008 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.890+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa7cb7ff10 con 0x55fa7cc27c40 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.891+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc27c40 0x55fa7cc2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.891+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc27c40 0x55fa7cc2a080 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25f3c8b7b82e95c5 server_cookie=f91c9f35a7c27946 in_seq=0 out_seq=0 2024-06-02T23:28:40.892 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.891+0000 7f2dbd7fa640 10 client.29008 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/4104159270 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fa7cb7ff10 con 0x55fa7cc27c40 2024-06-02T23:28:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2dbd7fa640 10 client.29008 handle_command_reply: tid=0 2024-06-02T23:28:40.893 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 client.29008 shutdown 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 msgr2=0x55fa7cc1c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2dc400bc10 tx=0x7f2dc4004020 comp rx=0 tx=0).stop 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 shutdown_connections 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc27c40 0x55fa7cc2a080 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2dbd7fa640 0 client.29008 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:40.894 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc17380 0x55fa7cc23080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 --2- 172.21.15.52:0/4104159270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 >> 172.21.15.52:0/4104159270 conn(0x55fa7cc0ce90 msgr2=0x55fa7cc037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 shutdown_connections 2024-06-02T23:28:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 1 -- 172.21.15.52:0/4104159270 wait complete. 2024-06-02T23:28:40.895 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.893+0000 7f2ddee20740 20 client.29008 trim_cache size 0 max 16384 2024-06-02T23:28:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 Processor -- start 2024-06-02T23:28:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 -- start start 2024-06-02T23:28:40.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc23040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41a6a50 con 0x55fa7cc12210 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41a8590 con 0x55fa7cc17380 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2ddee20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41a7120 con 0x55fa7cc11820 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2dd9ddd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47074/0 (socket says 172.21.15.52:47074) 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2dd9ddd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51934/0 (socket says 172.21.15.52:51934) 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2dd95dc640 1 -- 172.21.15.52:0/2607050596 learned_addr learned my addr 172.21.15.52:0/2607050596 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.919+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc23040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd95dc640 1 -- 172.21.15.52:0/2607050596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 msgr2=0x55fa7cc1bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd95dc640 1 -- 172.21.15.52:0/2607050596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 msgr2=0x55fa7cc1c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd95dc640 1 -- 172.21.15.52:0/2607050596 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa7cc1cc30 con 0x55fa7cc17380 2024-06-02T23:28:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc23040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009340 tx=0x7f2dd0005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c90cd73baffd99d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:40.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dbf7fe640 1 -- 172.21.15.52:0/2607050596 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0035040 con 0x55fa7cc17380 2024-06-02T23:28:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dbf7fe640 1 -- 172.21.15.52:0/2607050596 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd0003c20 con 0x55fa7cc17380 2024-06-02T23:28:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.920+0000 7f2dbf7fe640 1 -- 172.21.15.52:0/2607050596 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dd00317e0 con 0x55fa7cc17380 2024-06-02T23:28:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.921+0000 7f2ddee20740 1 -- 172.21.15.52:0/2607050596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 msgr2=0x55fa7cc23040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.921+0000 7f2ddee20740 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc23040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2dd0009340 tx=0x7f2dd0005b50 comp rx=0 tx=0).stop 2024-06-02T23:28:40.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 -- 172.21.15.52:0/2607050596 shutdown_connections 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc17380 0x55fa7cc23040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc12210 0x55fa7cc1c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 --2- 172.21.15.52:0/2607050596 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc11820 0x55fa7cc1bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 -- 172.21.15.52:0/2607050596 >> 172.21.15.52:0/2607050596 conn(0x55fa7cc0ce90 msgr2=0x55fa7cc16b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 -- 172.21.15.52:0/2607050596 shutdown_connections 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 -- 172.21.15.52:0/2607050596 wait complete. 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 -- start start 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.922+0000 7f2ddee20740 1 Processor -- start 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc11820 0x55fa7cc22f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc12210 0x55fa7cc1fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc17380 0x55fa7cc20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41a7ec0 con 0x55fa7cc11820 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc11820 0x55fa7cc22f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41a77f0 con 0x55fa7cc12210 2024-06-02T23:28:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2dd41a4a30 con 0x55fa7cc17380 2024-06-02T23:28:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd95dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc11820 0x55fa7cc22f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51936/0 (socket says 172.21.15.52:51936) 2024-06-02T23:28:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd95dc640 1 -- 172.21.15.52:0/76668332 learned_addr learned my addr 172.21.15.52:0/76668332 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2ddee20740 10 client.0 _ll_register_callbacks cb 0x7f2dd0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc12210 0x55fa7cc1fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc17380 0x55fa7cc20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/76668332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc17380 msgr2=0x55fa7cc20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc17380 0x55fa7cc20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/76668332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc11820 msgr2=0x55fa7cc22f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc11820 0x55fa7cc22f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.923+0000 7f2dd9ddd640 1 -- 172.21.15.52:0/76668332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa7cc90950 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2dd9ddd640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc12210 0x55fa7cc1fbc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4001eb0 tx=0x7f2dc4010d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5b0dd45665cfc17 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2dbd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/76668332 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4035070 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/76668332 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2dc40051e0 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dd0003c20 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/76668332 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2dc402a040 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2ddee20740 10 client.29363 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fa7cc91be0 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.924+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/76668332 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2dd0003c20 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/76668332 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2dc4038030 con 0x55fa7cc12210 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2ddee20740 10 client.29363 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2ddee20740 10 client.29363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:40.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2ddee20740 20 client.29363 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2ddee20740 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc925e0 0x55fa7cc94a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2ddee20740 4 client.29363 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:28:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fa7cb79c20 con 0x55fa7cc925e0 2024-06-02T23:28:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc925e0 0x55fa7cc94a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2dd95dc640 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc925e0 0x55fa7cc94a00 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cabdacb5aa5190a server_cookie=dd5b03c20f3f1145 in_seq=0 out_seq=0 2024-06-02T23:28:40.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.925+0000 7f2dbd7fa640 10 client.29363 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2dbd7fa640 1 -- 172.21.15.52:0/76668332 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55fa7cb79c20 con 0x55fa7cc925e0 2024-06-02T23:28:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2dbd7fa640 10 client.29363 handle_command_reply: tid=0 2024-06-02T23:28:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 client.29363 shutdown 2024-06-02T23:28:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc12210 msgr2=0x55fa7cc1fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc12210 0x55fa7cc1fbc0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4001eb0 tx=0x7f2dc4010d10 comp rx=0 tx=0).stop 2024-06-02T23:28:40.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 shutdown_connections 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fa7cc925e0 0x55fa7cc94a00 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fa7cc17380 0x55fa7cc20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fa7cc12210 0x55fa7cc1fbc0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 --2- 172.21.15.52:0/76668332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fa7cc11820 0x55fa7cc22f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.968+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 >> 172.21.15.52:0/76668332 conn(0x55fa7cc0ce90 msgr2=0x55fa7cc1b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2dbd7fa640 0 client.29363 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 shutdown_connections 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddee20740 1 -- 172.21.15.52:0/76668332 wait complete. 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddee20740 20 client.29363 trim_cache size 0 max 16384 2024-06-02T23:28:40.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:28:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:28:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:28:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2dc00a2d00 msgr2=0x7f2dc00a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2dc00a2d00 0x7f2dc00a51c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc01a040 tx=0x7f2dcc00c890 comp rx=0 tx=0).stop 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd41076e0 msgr2=0x7f2dd41a1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.969+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd41076e0 0x7f2dd41a1370 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2dcc009fd0 tx=0x7f2dcc00ef10 comp rx=0 tx=0).stop 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 shutdown_connections 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2dc00a2d00 0x7f2dc00a51c0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2dd41076e0 0x7f2dd41a1370 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2dd4104db0 0x7f2dd419a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 --2- 172.21.15.52:0/1942536477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dd4102480 0x7f2dd4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 >> 172.21.15.52:0/1942536477 conn(0x7f2dd40fbe70 msgr2=0x7f2dd4102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 shutdown_connections 2024-06-02T23:28:40.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:40.970+0000 7f2ddc041640 1 -- 172.21.15.52:0/1942536477 wait complete. 2024-06-02T23:28:41.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:40 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:28:41.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:40 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:28:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:40 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:28:41.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:40 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:28:41.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:40 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:28:41.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:40 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:28:41.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25488 6.36 MB/sec execute 109 sec latency 199.431 ms 2024-06-02T23:28:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:41 smithi174 ceph-mon[35994]: pgmap v935: 97 pgs: 97 active+clean; 393 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-06-02T23:28:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:28:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:28:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:28:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:41 smithi052 ceph-mon[32340]: pgmap v935: 97 pgs: 97 active+clean; 393 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-06-02T23:28:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:28:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:28:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:28:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:41 smithi123 ceph-mon[35977]: pgmap v935: 97 pgs: 97 active+clean; 393 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 324 op/s 2024-06-02T23:28:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:28:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:28:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:28:42.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25654 6.39 MB/sec execute 110 sec latency 182.506 ms 2024-06-02T23:28:43.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25822 6.38 MB/sec execute 111 sec latency 221.190 ms 2024-06-02T23:28:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:43 smithi174 ceph-mon[35994]: pgmap v936: 97 pgs: 97 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-06-02T23:28:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:43 smithi052 ceph-mon[32340]: pgmap v936: 97 pgs: 97 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-06-02T23:28:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:43 smithi123 ceph-mon[35977]: pgmap v936: 97 pgs: 97 active+clean; 395 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 334 op/s 2024-06-02T23:28:44.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 25996 6.39 MB/sec execute 112 sec latency 119.288 ms 2024-06-02T23:28:45.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 26199 6.49 MB/sec execute 113 sec latency 53.674 ms 2024-06-02T23:28:46.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:45 smithi174 ceph-mon[35994]: pgmap v937: 97 pgs: 97 active+clean; 394 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 309 op/s 2024-06-02T23:28:46.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:45 smithi052 ceph-mon[32340]: pgmap v937: 97 pgs: 97 active+clean; 394 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 309 op/s 2024-06-02T23:28:46.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:45 smithi123 ceph-mon[35977]: pgmap v937: 97 pgs: 97 active+clean; 394 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 309 op/s 2024-06-02T23:28:46.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 26363 6.47 MB/sec execute 114 sec latency 61.914 ms 2024-06-02T23:28:47.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 26528 6.46 MB/sec execute 115 sec latency 63.564 ms 2024-06-02T23:28:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:47 smithi052 ceph-mon[32340]: pgmap v938: 97 pgs: 97 active+clean; 383 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 387 op/s 2024-06-02T23:28:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:47 smithi174 ceph-mon[35994]: pgmap v938: 97 pgs: 97 active+clean; 383 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 387 op/s 2024-06-02T23:28:48.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:47 smithi123 ceph-mon[35977]: pgmap v938: 97 pgs: 97 active+clean; 383 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.2 MiB/s wr, 387 op/s 2024-06-02T23:28:48.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 26692 6.44 MB/sec execute 116 sec latency 51.485 ms 2024-06-02T23:28:49.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 26879 6.46 MB/sec execute 117 sec latency 89.231 ms 2024-06-02T23:28:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:49 smithi174 ceph-mon[35994]: pgmap v939: 97 pgs: 97 active+clean; 380 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 296 op/s 2024-06-02T23:28:50.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:49 smithi052 ceph-mon[32340]: pgmap v939: 97 pgs: 97 active+clean; 380 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 296 op/s 2024-06-02T23:28:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:49 smithi123 ceph-mon[35977]: pgmap v939: 97 pgs: 97 active+clean; 380 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.9 MiB/s wr, 296 op/s 2024-06-02T23:28:50.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 27114 6.52 MB/sec execute 118 sec latency 55.851 ms 2024-06-02T23:28:51.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:50 smithi174 ceph-mon[35994]: pgmap v940: 97 pgs: 97 active+clean; 385 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 323 op/s 2024-06-02T23:28:51.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:50 smithi052 ceph-mon[32340]: pgmap v940: 97 pgs: 97 active+clean; 385 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 323 op/s 2024-06-02T23:28:51.158 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:50 smithi123 ceph-mon[35977]: pgmap v940: 97 pgs: 97 active+clean; 385 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 323 op/s 2024-06-02T23:28:51.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 27278 6.53 MB/sec execute 119 sec latency 61.460 ms 2024-06-02T23:28:52.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 27432 6.58 MB/sec execute 120 sec latency 62.158 ms 2024-06-02T23:28:53.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:53 smithi052 ceph-mon[32340]: pgmap v941: 97 pgs: 97 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 304 op/s 2024-06-02T23:28:53.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 27647 6.70 MB/sec execute 121 sec latency 172.899 ms 2024-06-02T23:28:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:53 smithi123 ceph-mon[35977]: pgmap v941: 97 pgs: 97 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 304 op/s 2024-06-02T23:28:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:53 smithi174 ceph-mon[35994]: pgmap v941: 97 pgs: 97 active+clean; 389 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.6 MiB/s wr, 304 op/s 2024-06-02T23:28:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.098+0000 7f96d2c95640 1 -- 172.21.15.52:0/471869769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc104e50 msgr2=0x7f96cc107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.098+0000 7f96d2c95640 1 --2- 172.21.15.52:0/471869769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc104e50 0x7f96cc107240 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f96b8009860 tx=0x7f96b802a340 comp rx=0 tx=0).stop 2024-06-02T23:28:54.099 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 -- 172.21.15.52:0/471869769 shutdown_connections 2024-06-02T23:28:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 --2- 172.21.15.52:0/471869769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc107780 0x7f96cc109b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 --2- 172.21.15.52:0/471869769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc104e50 0x7f96cc107240 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 --2- 172.21.15.52:0/471869769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96cc102520 0x7f96cc104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 -- 172.21.15.52:0/471869769 >> 172.21.15.52:0/471869769 conn(0x7f96cc0fbe70 msgr2=0x7f96cc0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 -- 172.21.15.52:0/471869769 shutdown_connections 2024-06-02T23:28:54.100 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 -- 172.21.15.52:0/471869769 wait complete. 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 Processor -- start 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 -- start start 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.099+0000 7f96d2c95640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 0x7f96cc199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d2c95640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc104e50 0x7f96cc19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d2c95640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96cc107780 0x7f96cc1a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d2c95640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc10d500 con 0x7f96cc104e50 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d2c95640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc10d380 con 0x7f96cc102520 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d2c95640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc10d680 con 0x7f96cc107780 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 0x7f96cc199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 0x7f96cc199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56690/0 (socket says 172.21.15.52:56690) 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc104e50 0x7f96cc19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 -- 172.21.15.52:0/35372745 learned_addr learned my addr 172.21.15.52:0/35372745 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d1232640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96cc107780 0x7f96cc1a1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 -- 172.21.15.52:0/35372745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96cc107780 msgr2=0x7f96cc1a1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96cc107780 0x7f96cc1a1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 -- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc104e50 msgr2=0x7f96cc19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc104e50 0x7f96cc19a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 -- 172.21.15.52:0/35372745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96cc1a19f0 con 0x7f96cc102520 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96d0a31640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 0x7f96cc199dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f96c000e9c0 tx=0x7f96c000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.100+0000 7f96c9ffb640 1 -- 172.21.15.52:0/35372745 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c000ccc0 con 0x7f96cc102520 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.101+0000 7f96c9ffb640 1 -- 172.21.15.52:0/35372745 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c00054d0 con 0x7f96cc102520 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.101+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96cc1a1ce0 con 0x7f96cc102520 2024-06-02T23:28:54.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.101+0000 7f96c9ffb640 1 -- 172.21.15.52:0/35372745 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c0010430 con 0x7f96cc102520 2024-06-02T23:28:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.101+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96cc1a21a0 con 0x7f96cc102520 2024-06-02T23:28:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 Processor -- start 2024-06-02T23:28:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96c9ffb640 1 -- 172.21.15.52:0/35372745 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f96c00046c0 con 0x7f96cc102520 2024-06-02T23:28:54.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 -- start start 2024-06-02T23:28:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb6f010 0x55f2fcb6f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb74500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 0x55f2fcb77000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f2fcad24f0 con 0x55f2fcb6fa00 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2fca70870 con 0x55f2fcb74b70 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.104+0000 7f96d599c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2fcace4b0 con 0x55f2fcb6f010 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 0x55f2fcb77000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 0x55f2fcb77000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56694/0 (socket says 172.21.15.52:56694) 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 -- 172.21.15.52:0/1585052802 learned_addr learned my addr 172.21.15.52:0/1585052802 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96d0a31640 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb74500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96d1232640 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb6f010 0x55f2fcb6f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 -- 172.21.15.52:0/1585052802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb6f010 msgr2=0x55f2fcb6f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb6f010 0x55f2fcb6f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 -- 172.21.15.52:0/1585052802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 msgr2=0x55f2fcb74500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb74500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 -- 172.21.15.52:0/1585052802 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fca8c9b0 con 0x55f2fcb74b70 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96d0a31640 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb74500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96c9ffb640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96bc0a2cb0 0x7f96bc0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.105+0000 7f96cbfff640 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 0x55f2fcb77000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f96b802a930 tx=0x7f96b8005b20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc92b68d55d6a025 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.106+0000 7f96c9ffb640 1 -- 172.21.15.52:0/35372745 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96c00cf230 con 0x7f96cc102520 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.106+0000 7f96d1232640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96bc0a2cb0 0x7f96bc0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.106+0000 7f96b6ffd640 1 -- 172.21.15.52:0/1585052802 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8043020 con 0x55f2fcb74b70 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.106+0000 7f96b6ffd640 1 -- 172.21.15.52:0/1585052802 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96b8038040 con 0x55f2fcb74b70 2024-06-02T23:28:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.106+0000 7f96b6ffd640 1 -- 172.21.15.52:0/1585052802 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b8037040 con 0x55f2fcb74b70 2024-06-02T23:28:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 -- 172.21.15.52:0/1585052802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 msgr2=0x55f2fcb77000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 0x55f2fcb77000 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f96b802a930 tx=0x7f96b8005b20 comp rx=0 tx=0).stop 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d1232640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96bc0a2cb0 0x7f96bc0a5170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55f2fcb70260 tx=0x7f96c4009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 -- 172.21.15.52:0/1585052802 shutdown_connections 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb74b70 0x55f2fcb77000 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb74500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 --2- 172.21.15.52:0/1585052802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb6f010 0x55f2fcb6f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 -- 172.21.15.52:0/1585052802 >> 172.21.15.52:0/1585052802 conn(0x55f2fcb6a680 msgr2=0x55f2fcb6aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 -- 172.21.15.52:0/1585052802 shutdown_connections 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.107+0000 7f96d599c740 1 -- 172.21.15.52:0/1585052802 wait complete. 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.108+0000 7f96d599c740 1 -- start start 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.108+0000 7f96d599c740 1 Processor -- start 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.108+0000 7f96d599c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 0x55f2fcb791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.112 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.108+0000 7f96d599c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb79700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.108+0000 7f96d599c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb80740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.108+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 0x55f2fcb791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 0x55f2fcb791c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49068/0 (socket says 172.21.15.52:49068) 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d0a31640 1 -- 172.21.15.52:0/1755074775 learned_addr learned my addr 172.21.15.52:0/1755074775 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96cbfff640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb79700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f2fcb77ac0 con 0x55f2fcb6f010 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f2fca5bae0 con 0x55f2fcb6fa00 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2fcb77c40 con 0x55f2fcb74b70 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d1232640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb80740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d1232640 1 -- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6fa00 msgr2=0x55f2fcb79700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d1232640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb79700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d1232640 1 -- 172.21.15.52:0/1755074775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 msgr2=0x55f2fcb791c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d1232640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 0x55f2fcb791c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.113 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d1232640 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fc7e6210 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d0a31640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 0x55f2fcb791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d599c740 10 client.0 _ll_register_callbacks cb 0x55f2fca949b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.109+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96b8038040 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d1232640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb80740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f96c4005fd0 tx=0x7f96c4004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d028c53888b851d7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96abfff640 10 client.29026 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96abfff640 1 -- 172.21.15.52:0/1755074775 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c400f870 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96abfff640 1 -- 172.21.15.52:0/1755074775 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96c400fa10 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96abfff640 1 -- 172.21.15.52:0/1755074775 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f96c400fc30 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d599c740 10 client.29026 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55f2fc8b2620 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96abfff640 1 -- 172.21.15.52:0/1755074775 <== mon.1 v2:172.21.15.174:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f96c4026070 con 0x55f2fcb74b70 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d599c740 10 client.29026 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:54.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d599c740 10 client.29026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d599c740 20 client.29026 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.110+0000 7f96d599c740 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcb84fa0 0x55f2fcb87410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.111+0000 7f96d599c740 4 client.29026 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.111+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2fcae5f10 con 0x55f2fcb84fa0 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.111+0000 7f96d0a31640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcb84fa0 0x55f2fcb87410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.111+0000 7f96abfff640 10 client.29026 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:54.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.111+0000 7f96d0a31640 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcb84fa0 0x55f2fcb87410 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fd5431dd50f932 server_cookie=a5095bfcfec9c49c in_seq=0 out_seq=0 2024-06-02T23:28:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96abfff640 1 -- 172.21.15.52:0/1755074775 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f2fcae5f10 con 0x55f2fcb84fa0 2024-06-02T23:28:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96abfff640 10 client.29026 handle_command_reply: tid=0 2024-06-02T23:28:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 client.29026 shutdown 2024-06-02T23:28:54.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 msgr2=0x55f2fcb80740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb80740 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f96c4005fd0 tx=0x7f96c4004ab0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 shutdown_connections 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcb84fa0 0x55f2fcb87410 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb80740 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb79700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 --2- 172.21.15.52:0/1755074775 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6f010 0x55f2fcb791c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96abfff640 0 client.29026 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 >> 172.21.15.52:0/1755074775 conn(0x55f2fcb6a680 msgr2=0x55f2fcb697e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 shutdown_connections 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.112+0000 7f96d599c740 1 -- 172.21.15.52:0/1755074775 wait complete. 2024-06-02T23:28:54.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.113+0000 7f96d599c740 20 client.29026 trim_cache size 0 max 16384 2024-06-02T23:28:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 Processor -- start 2024-06-02T23:28:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 -- start start 2024-06-02T23:28:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb78490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb80700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1aadb0 con 0x55f2fcb6fa00 2024-06-02T23:28:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1ac8f0 con 0x55f2fcb6f010 2024-06-02T23:28:54.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d599c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1ab480 con 0x55f2fcb74b70 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb78490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb78490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56708/0 (socket says 172.21.15.52:56708) 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d0a31640 1 -- 172.21.15.52:0/3282626129 learned_addr learned my addr 172.21.15.52:0/3282626129 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96d1232640 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.134+0000 7f96cbfff640 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb80700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96d0a31640 1 -- 172.21.15.52:0/3282626129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 msgr2=0x55f2fcb789d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96d0a31640 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb789d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96d0a31640 1 -- 172.21.15.52:0/3282626129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 msgr2=0x55f2fcb80700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96d0a31640 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb80700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96d0a31640 1 -- 172.21.15.52:0/3282626129 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2fcb6a4b0 con 0x55f2fcb6f010 2024-06-02T23:28:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96d0a31640 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb78490 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f96c0097ec0 tx=0x7f96c0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5225cd2596e8d833 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96b67fc640 1 -- 172.21.15.52:0/3282626129 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c009fe10 con 0x55f2fcb6f010 2024-06-02T23:28:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96b67fc640 1 -- 172.21.15.52:0/3282626129 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c00a46d0 con 0x55f2fcb6f010 2024-06-02T23:28:54.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.135+0000 7f96b67fc640 1 -- 172.21.15.52:0/3282626129 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c00a4240 con 0x55f2fcb6f010 2024-06-02T23:28:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 -- 172.21.15.52:0/3282626129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 msgr2=0x55f2fcb78490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb78490 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f96c0097ec0 tx=0x7f96c0005880 comp rx=0 tx=0).stop 2024-06-02T23:28:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 -- 172.21.15.52:0/3282626129 shutdown_connections 2024-06-02T23:28:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb789d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb80700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 --2- 172.21.15.52:0/3282626129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb78490 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 -- 172.21.15.52:0/3282626129 >> 172.21.15.52:0/3282626129 conn(0x55f2fcb6a680 msgr2=0x55f2fcb791c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 -- 172.21.15.52:0/3282626129 shutdown_connections 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.136+0000 7f96d599c740 1 -- 172.21.15.52:0/3282626129 wait complete. 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 -- start start 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 Processor -- start 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb80240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb7cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb7d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1ac220 con 0x55f2fcb6fa00 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1abb50 con 0x55f2fcb6f010 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f96cc1a8d90 con 0x55f2fcb74b70 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d599c740 10 client.0 _ll_register_callbacks cb 0x7f96c4009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb80240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.137+0000 7f96cbfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb7cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d1232640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb7d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb80240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56710/0 (socket says 172.21.15.52:56710) 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 -- 172.21.15.52:0/2728174387 learned_addr learned my addr 172.21.15.52:0/2728174387 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 -- 172.21.15.52:0/2728174387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 msgr2=0x55f2fcb7d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb7d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 -- 172.21.15.52:0/2728174387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 msgr2=0x55f2fcb7cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb7cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 -- 172.21.15.52:0/2728174387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2fcb80870 con 0x55f2fcb6f010 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d1232640 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96abfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d0a31640 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb80240 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96c000cfa0 tx=0x7f96c0009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d12594e4b0115e92 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:28:54.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96abfff640 1 -- 172.21.15.52:0/2728174387 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96c00b9060 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96abfff640 1 -- 172.21.15.52:0/2728174387 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f96c00a4450 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.138+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96c00a46d0 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96abfff640 1 -- 172.21.15.52:0/2728174387 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f96c009fe10 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 10 client.29381 fetch_fsmap learned FSMap version 43 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f2fcbeeba0 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96abfff640 1 -- 172.21.15.52:0/2728174387 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f96c00a46d0 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96abfff640 1 -- 172.21.15.52:0/2728174387 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f96c009aeb0 con 0x55f2fcb6f010 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 10 client.29381 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 10 client.29381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 20 client.29381 populate_metadata read hostname 'smithi052' 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcbef400 0x55f2fcbf17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 4 client.29381 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:28:54.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2fcadfc20 con 0x55f2fcbef400 2024-06-02T23:28:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.139+0000 7f96cbfff640 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcbef400 0x55f2fcbf17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:28:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.140+0000 7f96abfff640 10 client.29381 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:54.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.140+0000 7f96cbfff640 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcbef400 0x55f2fcbf17f0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9322ad3746abdb03 server_cookie=102a8017ebd5a12c in_seq=0 out_seq=0 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96abfff640 1 -- 172.21.15.52:0/2728174387 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f2fcadfc20 con 0x55f2fcbef400 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96abfff640 10 client.29381 handle_command_reply: tid=0 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 client.29381 shutdown 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 msgr2=0x55f2fcb80240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb80240 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f96c000cfa0 tx=0x7f96c0009f90 comp rx=0 tx=0).stop 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 shutdown_connections 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f2fcbef400 0x55f2fcbf17f0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96abfff640 0 client.29381 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2fcb74b70 0x55f2fcb7d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f2fcb6fa00 0x55f2fcb7cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 --2- 172.21.15.52:0/2728174387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f2fcb6f010 0x55f2fcb80240 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 >> 172.21.15.52:0/2728174387 conn(0x55f2fcb6a680 msgr2=0x55f2fcb78490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 shutdown_connections 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 1 -- 172.21.15.52:0/2728174387 wait complete. 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.184+0000 7f96d599c740 20 client.29381 trim_cache size 0 max 16384 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:28:54.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:28:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.185+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96bc0a2cb0 msgr2=0x7f96bc0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.185+0000 7f96d2c95640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96bc0a2cb0 0x7f96bc0a5170 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x55f2fcb70260 tx=0x7f96c4009500 comp rx=0 tx=0).stop 2024-06-02T23:28:54.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 msgr2=0x7f96cc199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 0x7f96cc199dd0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f96c000e9c0 tx=0x7f96c000ee80 comp rx=0 tx=0).stop 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 shutdown_connections 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f96bc0a2cb0 0x7f96bc0a5170 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f96cc107780 0x7f96cc1a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f96cc104e50 0x7f96cc19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 --2- 172.21.15.52:0/35372745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96cc102520 0x7f96cc199dd0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 >> 172.21.15.52:0/35372745 conn(0x7f96cc0fbe70 msgr2=0x7f96cc101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 shutdown_connections 2024-06-02T23:28:54.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:28:54.186+0000 7f96d2c95640 1 -- 172.21.15.52:0/35372745 wait complete. 2024-06-02T23:28:54.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 27859 6.77 MB/sec execute 122 sec latency 76.978 ms 2024-06-02T23:28:55.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28007 6.77 MB/sec execute 123 sec latency 66.438 ms 2024-06-02T23:28:55.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:55 smithi123 ceph-mon[35977]: pgmap v942: 97 pgs: 97 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-06-02T23:28:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:55 smithi174 ceph-mon[35994]: pgmap v942: 97 pgs: 97 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-06-02T23:28:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:55 smithi052 ceph-mon[32340]: pgmap v942: 97 pgs: 97 active+clean; 391 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1015 KiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-06-02T23:28:56.459 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28139 6.77 MB/sec execute 124 sec latency 54.742 ms 2024-06-02T23:28:57.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28318 6.81 MB/sec execute 125 sec latency 53.748 ms 2024-06-02T23:28:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:57 smithi123 ceph-mon[35977]: pgmap v943: 97 pgs: 97 active+clean; 397 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 325 op/s 2024-06-02T23:28:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:57 smithi174 ceph-mon[35994]: pgmap v943: 97 pgs: 97 active+clean; 397 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 325 op/s 2024-06-02T23:28:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:57 smithi052 ceph-mon[32340]: pgmap v943: 97 pgs: 97 active+clean; 397 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 325 op/s 2024-06-02T23:28:58.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28481 6.83 MB/sec execute 126 sec latency 84.421 ms 2024-06-02T23:28:59.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28608 6.80 MB/sec execute 127 sec latency 130.132 ms 2024-06-02T23:28:59.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:28:59 smithi123 ceph-mon[35977]: pgmap v944: 97 pgs: 97 active+clean; 399 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 4.1 MiB/s wr, 245 op/s 2024-06-02T23:28:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:28:59 smithi174 ceph-mon[35994]: pgmap v944: 97 pgs: 97 active+clean; 399 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 4.1 MiB/s wr, 245 op/s 2024-06-02T23:28:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:28:59 smithi052 ceph-mon[32340]: pgmap v944: 97 pgs: 97 active+clean; 399 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1003 KiB/s rd, 4.1 MiB/s wr, 245 op/s 2024-06-02T23:29:00.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28756 6.81 MB/sec execute 128 sec latency 57.072 ms 2024-06-02T23:29:01.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 28935 6.81 MB/sec execute 129 sec latency 57.534 ms 2024-06-02T23:29:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:01 smithi123 ceph-mon[35977]: pgmap v945: 97 pgs: 97 active+clean; 402 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 293 op/s 2024-06-02T23:29:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:01 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:29:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:01 smithi174 ceph-mon[35994]: pgmap v945: 97 pgs: 97 active+clean; 402 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 293 op/s 2024-06-02T23:29:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:01 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:29:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:01 smithi052 ceph-mon[32340]: pgmap v945: 97 pgs: 97 active+clean; 402 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 293 op/s 2024-06-02T23:29:01.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:01 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:29:02.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 29131 6.84 MB/sec execute 130 sec latency 77.183 ms 2024-06-02T23:29:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:29:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:29:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:29:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:29:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:29:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:29:02.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:29:02.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:29:02.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:29:03.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:03 smithi052 ceph-mon[32340]: pgmap v946: 97 pgs: 97 active+clean; 404 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-06-02T23:29:03.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 29280 6.84 MB/sec execute 131 sec latency 72.018 ms 2024-06-02T23:29:03.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:03 smithi123 ceph-mon[35977]: pgmap v946: 97 pgs: 97 active+clean; 404 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-06-02T23:29:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:03 smithi174 ceph-mon[35994]: pgmap v946: 97 pgs: 97 active+clean; 404 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.8 MiB/s wr, 277 op/s 2024-06-02T23:29:04.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 29440 6.81 MB/sec execute 132 sec latency 56.994 ms 2024-06-02T23:29:05.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 29656 6.93 MB/sec execute 133 sec latency 59.369 ms 2024-06-02T23:29:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:05 smithi123 ceph-mon[35977]: pgmap v947: 97 pgs: 97 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-06-02T23:29:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:05 smithi174 ceph-mon[35994]: pgmap v947: 97 pgs: 97 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-06-02T23:29:05.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:05 smithi052 ceph-mon[32340]: pgmap v947: 97 pgs: 97 active+clean; 405 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.7 MiB/s wr, 272 op/s 2024-06-02T23:29:06.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 29806 6.90 MB/sec execute 134 sec latency 112.407 ms 2024-06-02T23:29:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 -- 172.21.15.52:0/40675745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c00759e0 msgr2=0x7f84c0073e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 --2- 172.21.15.52:0/40675745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c00759e0 0x7f84c0073e30 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009ca0 tx=0x7f84b002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 -- 172.21.15.52:0/40675745 shutdown_connections 2024-06-02T23:29:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 --2- 172.21.15.52:0/40675745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c0074400 0x7f84c0074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.320 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 --2- 172.21.15.52:0/40675745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84c01098e0 0x7f84c010bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 --2- 172.21.15.52:0/40675745 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c00759e0 0x7f84c0073e30 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 -- 172.21.15.52:0/40675745 >> 172.21.15.52:0/40675745 conn(0x7f84c00fcf90 msgr2=0x7f84c00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 -- 172.21.15.52:0/40675745 shutdown_connections 2024-06-02T23:29:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.319+0000 7f84c6e64640 1 -- 172.21.15.52:0/40675745 wait complete. 2024-06-02T23:29:07.321 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 Processor -- start 2024-06-02T23:29:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 -- start start 2024-06-02T23:29:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 0x7f84c0199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c00759e0 0x7f84c019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84c01098e0 0x7f84c01a1360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010f710 con 0x7f84c0074400 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010f590 con 0x7f84c01098e0 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c6e64640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84c010f890 con 0x7f84c00759e0 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 0x7f84c0199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c4c00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 0x7f84c0199e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51554/0 (socket says 172.21.15.52:51554) 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c4c00640 1 -- 172.21.15.52:0/1825779415 learned_addr learned my addr 172.21.15.52:0/1825779415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84bffff640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c00759e0 0x7f84c019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.320+0000 7f84c5401640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84c01098e0 0x7f84c01a1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c4c00640 1 -- 172.21.15.52:0/1825779415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c00759e0 msgr2=0x7f84c019a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c4c00640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c00759e0 0x7f84c019a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.324 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c4c00640 1 -- 172.21.15.52:0/1825779415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84c01098e0 msgr2=0x7f84c01a1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c4c00640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84c01098e0 0x7f84c01a1360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c4c00640 1 -- 172.21.15.52:0/1825779415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84c01a19d0 con 0x7f84c0074400 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84bffff640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c00759e0 0x7f84c019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c4c00640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 0x7f84c0199e00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009dc0 tx=0x7f84b0009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84bdffb640 1 -- 172.21.15.52:0/1825779415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0003db0 con 0x7f84c0074400 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84bdffb640 1 -- 172.21.15.52:0/1825779415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b00044f0 con 0x7f84c0074400 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84bdffb640 1 -- 172.21.15.52:0/1825779415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b0030d90 con 0x7f84c0074400 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.321+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84c01a1c60 con 0x7f84c0074400 2024-06-02T23:29:07.325 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.322+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84c01a21b0 con 0x7f84c0074400 2024-06-02T23:29:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84bdffb640 1 -- 172.21.15.52:0/1825779415 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f84b00305c0 con 0x7f84c0074400 2024-06-02T23:29:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 Processor -- start 2024-06-02T23:29:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 -- start start 2024-06-02T23:29:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970a9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970ae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55849700c4f0 con 0x5584970a9a00 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558496faa870 con 0x5584970a9010 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c9b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584970084b0 con 0x5584970aeb70 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c5401640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c5401640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38956/0 (socket says 172.21.15.52:38956) 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.325+0000 7f84c5401640 1 -- 172.21.15.52:0/3643777804 learned_addr learned my addr 172.21.15.52:0/3643777804 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c4c00640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84bffff640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c5401640 1 -- 172.21.15.52:0/3643777804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 msgr2=0x5584970b1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c5401640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c5401640 1 -- 172.21.15.52:0/3643777804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 msgr2=0x5584970ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c5401640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970ae500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c5401640 1 -- 172.21.15.52:0/3643777804 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558496fc69b0 con 0x5584970a9010 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c4c00640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84bffff640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84c5401640 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970a9430 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b6b0 tx=0x7f84b800bb70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e788386a22424c45 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84b6ffd640 1 -- 172.21.15.52:0/3643777804 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8003bd0 con 0x5584970a9010 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84b6ffd640 1 -- 172.21.15.52:0/3643777804 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b8015070 con 0x5584970a9010 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.326+0000 7f84b6ffd640 1 -- 172.21.15.52:0/3643777804 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8003bd0 con 0x5584970a9010 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.327+0000 7f84bdffb640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f84ac0a2cb0 0x7f84ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.327+0000 7f84c4c00640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f84ac0a2cb0 0x7f84ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.327+0000 7f84bdffb640 1 -- 172.21.15.52:0/1825779415 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f84b00f03b0 con 0x7f84c0074400 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.327+0000 7f84c9b74740 1 -- 172.21.15.52:0/3643777804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 msgr2=0x5584970a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.327+0000 7f84c9b74740 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970a9430 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f84b800b6b0 tx=0x7f84b800bb70 comp rx=0 tx=0).stop 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.327+0000 7f84c4c00640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f84ac0a2cb0 0x7f84ac0a5170 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f84b00ee250 tx=0x7f84b002a8c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:07.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 -- 172.21.15.52:0/3643777804 shutdown_connections 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970ae500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 --2- 172.21.15.52:0/3643777804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970a9430 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 -- 172.21.15.52:0/3643777804 >> 172.21.15.52:0/3643777804 conn(0x5584970a4680 msgr2=0x5584970a4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 -- 172.21.15.52:0/3643777804 shutdown_connections 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 -- 172.21.15.52:0/3643777804 wait complete. 2024-06-02T23:29:07.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 -- start start 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.328+0000 7f84c9b74740 1 Processor -- start 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970a9a00 0x5584970b3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970aeb70 0x5584970ba7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5584970b1b20 con 0x5584970aeb70 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558496f95ae0 con 0x5584970a9010 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5584970b1ca0 con 0x5584970a9a00 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.335 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38966/0 (socket says 172.21.15.52:38966) 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c9b74740 10 client.0 _ll_register_callbacks cb 0x558496fce9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84bffff640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970a9a00 0x5584970b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 -- 172.21.15.52:0/2120970298 learned_addr learned my addr 172.21.15.52:0/2120970298 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970aeb70 0x5584970ba7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 -- 172.21.15.52:0/2120970298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970a9a00 msgr2=0x5584970b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970a9a00 0x5584970b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 -- 172.21.15.52:0/2120970298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970aeb70 msgr2=0x5584970ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970aeb70 0x5584970ba7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.329+0000 7f84c5401640 1 -- 172.21.15.52:0/2120970298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558496d20210 con 0x5584970a9010 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f84bffff640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970a9a00 0x5584970b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f84c5401640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b3220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99ca3c907175dc90 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:07.336 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f849ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f849ffff640 1 -- 172.21.15.52:0/2120970298 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8003bd0 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f849ffff640 1 -- 172.21.15.52:0/2120970298 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f84b80117b0 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b8015070 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f849ffff640 1 -- 172.21.15.52:0/2120970298 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f84b8032050 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f84c9b74740 10 client.29393 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.330+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558496dec620 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.331+0000 7f849ffff640 1 -- 172.21.15.52:0/2120970298 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84b8015070 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.331+0000 7f849ffff640 1 -- 172.21.15.52:0/2120970298 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f84b8004b40 con 0x5584970a9010 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.331+0000 7f84c9b74740 10 client.29393 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.331+0000 7f84c9b74740 10 client.29393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:07.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f84c9b74740 20 client.29393 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f84c9b74740 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5584970bec60 0x5584970c1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f84c9b74740 4 client.29393 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55849701ff10 con 0x5584970bec60 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f84bffff640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5584970bec60 0x5584970c1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f849ffff640 10 client.29393 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.332+0000 7f84bffff640 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5584970bec60 0x5584970c1050 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e7d3a1c24f02523 server_cookie=2673eb6eeefc85d4 in_seq=0 out_seq=0 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f849ffff640 1 -- 172.21.15.52:0/2120970298 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55849701ff10 con 0x5584970bec60 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f849ffff640 10 client.29393 handle_command_reply: tid=0 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 client.29393 shutdown 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 msgr2=0x5584970b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b3220 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f84b8006010 tx=0x7f84b80058d0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 shutdown_connections 2024-06-02T23:29:07.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5584970bec60 0x5584970c1050 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970a9a00 0x5584970b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f849ffff640 0 client.29393 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970aeb70 0x5584970ba7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 --2- 172.21.15.52:0/2120970298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b3220 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.334+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 >> 172.21.15.52:0/2120970298 conn(0x5584970a4680 msgr2=0x5584970a37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:07.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.335+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 shutdown_connections 2024-06-02T23:29:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.335+0000 7f84c9b74740 1 -- 172.21.15.52:0/2120970298 wait complete. 2024-06-02T23:29:07.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.335+0000 7f84c9b74740 20 client.29393 trim_cache size 0 max 16384 2024-06-02T23:29:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 Processor -- start 2024-06-02T23:29:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 -- start start 2024-06-02T23:29:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9010 0x5584970aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.366 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 0x5584970ab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970ba760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ab040 con 0x5584970a9010 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01acb80 con 0x5584970a9a00 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.365+0000 7f84c9b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ab710 con 0x5584970aeb70 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 0x5584970ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84c5401640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9010 0x5584970aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 0x5584970ab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:38968/0 (socket says 172.21.15.52:38968) 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84c5401640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9010 0x5584970aabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51588/0 (socket says 172.21.15.52:51588) 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 -- 172.21.15.52:0/208624576 learned_addr learned my addr 172.21.15.52:0/208624576 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84c5401640 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970ba760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 -- 172.21.15.52:0/208624576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 msgr2=0x5584970ba760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970ba760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 -- 172.21.15.52:0/208624576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9010 msgr2=0x5584970aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9010 0x5584970aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.367 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 -- 172.21.15.52:0/208624576 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584970aa6d0 con 0x5584970a9a00 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.366+0000 7f84bffff640 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 0x5584970ab120 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a8014e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59b2436d2609e8c2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.367+0000 7f84b67fc640 1 -- 172.21.15.52:0/208624576 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a8004250 con 0x5584970a9a00 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.367+0000 7f84b67fc640 1 -- 172.21.15.52:0/208624576 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a8012dd0 con 0x5584970a9a00 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.367+0000 7f84b67fc640 1 -- 172.21.15.52:0/208624576 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84a80047a0 con 0x5584970a9a00 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 -- 172.21.15.52:0/208624576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 msgr2=0x5584970ab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 0x5584970ab120 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f84a8000c00 tx=0x7f84a8014e10 comp rx=0 tx=0).stop 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 -- 172.21.15.52:0/208624576 shutdown_connections 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970ba760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9a00 0x5584970ab120 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 --2- 172.21.15.52:0/208624576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9010 0x5584970aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 -- 172.21.15.52:0/208624576 >> 172.21.15.52:0/208624576 conn(0x5584970a4680 msgr2=0x5584970c0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 -- 172.21.15.52:0/208624576 shutdown_connections 2024-06-02T23:29:07.369 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 -- 172.21.15.52:0/208624576 wait complete. 2024-06-02T23:29:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.368+0000 7f84c9b74740 1 -- start start 2024-06-02T23:29:07.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 Processor -- start 2024-06-02T23:29:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01ac4b0 con 0x5584970a9a00 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01abde0 con 0x5584970a9010 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84c01a9020 con 0x5584970aeb70 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c9b74740 10 client.0 _ll_register_callbacks cb 0x7f84b80044f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c5401640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970b6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51594/0 (socket says 172.21.15.52:51594) 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84bffff640 1 -- 172.21.15.52:0/307977098 learned_addr learned my addr 172.21.15.52:0/307977098 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.369+0000 7f84c5401640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c5401640 1 -- 172.21.15.52:0/307977098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 msgr2=0x5584970b9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c5401640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c5401640 1 -- 172.21.15.52:0/307977098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 msgr2=0x5584970b6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:07.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c5401640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970b6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c5401640 1 -- 172.21.15.52:0/307977098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584970ba590 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84bffff640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c5401640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b6ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005940 tx=0x7f84b8013040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5ea58c2212c4994 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f849ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f849ffff640 1 -- 172.21.15.52:0/307977098 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b8003bd0 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f849ffff640 1 -- 172.21.15.52:0/307977098 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f84b8003d70 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84a8012dd0 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f849ffff640 1 -- 172.21.15.52:0/307977098 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f84b801aa60 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c9b74740 10 client.29399 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.370+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558497128310 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f849ffff640 1 -- 172.21.15.52:0/307977098 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f84a8012dd0 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f849ffff640 1 -- 172.21.15.52:0/307977098 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f84b8044030 con 0x5584970a9010 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84c9b74740 10 client.29399 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84c9b74740 10 client.29399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:07.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84c9b74740 20 client.29399 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84c9b74740 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558497128c00 0x55849712b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84c9b74740 4 client.29399 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:29:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558497019c20 con 0x558497128c00 2024-06-02T23:29:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.371+0000 7f84bffff640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558497128c00 0x55849712b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.372+0000 7f84bffff640 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558497128c00 0x55849712b020 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7df995d321d359c server_cookie=248ff02ab981d496 in_seq=0 out_seq=0 2024-06-02T23:29:07.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.372+0000 7f849ffff640 10 client.29399 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.413+0000 7f849ffff640 1 -- 172.21.15.52:0/307977098 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558497019c20 con 0x558497128c00 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.413+0000 7f849ffff640 10 client.29399 handle_command_reply: tid=0 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.413+0000 7f84c9b74740 1 client.29399 shutdown 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 msgr2=0x5584970b6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b6ab0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f84b8005940 tx=0x7f84b8013040 comp rx=0 tx=0).stop 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 shutdown_connections 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x558497128c00 0x55849712b020 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5584970aeb70 0x5584970b9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5584970a9a00 0x5584970b6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 --2- 172.21.15.52:0/307977098 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584970a9010 0x5584970b6ab0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f849ffff640 0 client.29399 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:07.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 >> 172.21.15.52:0/307977098 conn(0x5584970a4680 msgr2=0x5584970c0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 shutdown_connections 2024-06-02T23:29:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 1 -- 172.21.15.52:0/307977098 wait complete. 2024-06-02T23:29:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.414+0000 7f84c9b74740 20 client.29399 trim_cache size 0 max 16384 2024-06-02T23:29:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:29:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:29:07.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f84ac0a2cb0 msgr2=0x7f84ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f84ac0a2cb0 0x7f84ac0a5170 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f84b00ee250 tx=0x7f84b002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 msgr2=0x7f84c0199e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 0x7f84c0199e00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f84b0009dc0 tx=0x7f84b0009380 comp rx=0 tx=0).stop 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 shutdown_connections 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f84ac0a2cb0 0x7f84ac0a5170 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84c00759e0 0x7f84c019a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84c01098e0 0x7f84c01a1360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 --2- 172.21.15.52:0/1825779415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f84c0074400 0x7f84c0199e00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 >> 172.21.15.52:0/1825779415 conn(0x7f84c00fcf90 msgr2=0x7f84c01057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:07.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 shutdown_connections 2024-06-02T23:29:07.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:07.415+0000 7f84c6e64640 1 -- 172.21.15.52:0/1825779415 wait complete. 2024-06-02T23:29:07.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 29984 6.90 MB/sec execute 135 sec latency 65.588 ms 2024-06-02T23:29:07.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:07 smithi123 ceph-mon[35977]: pgmap v948: 97 pgs: 97 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 326 op/s 2024-06-02T23:29:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:07 smithi174 ceph-mon[35994]: pgmap v948: 97 pgs: 97 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 326 op/s 2024-06-02T23:29:07.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:07 smithi052 ceph-mon[32340]: pgmap v948: 97 pgs: 97 active+clean; 400 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 326 op/s 2024-06-02T23:29:08.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 30139 6.88 MB/sec execute 136 sec latency 59.529 ms 2024-06-02T23:29:08.957 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:29:09.320 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:09.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:09 smithi052 ceph-mon[32340]: pgmap v949: 97 pgs: 97 active+clean; 397 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-06-02T23:29:09.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 30315 6.88 MB/sec execute 137 sec latency 69.986 ms 2024-06-02T23:29:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:09 smithi123 ceph-mon[35977]: pgmap v949: 97 pgs: 97 active+clean; 397 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-06-02T23:29:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:09 smithi174 ceph-mon[35994]: pgmap v949: 97 pgs: 97 active+clean; 397 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 270 op/s 2024-06-02T23:29:10.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 30513 6.90 MB/sec execute 138 sec latency 66.212 ms 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.275+0000 7f8308a73700 1 -- 172.21.15.52:0/4193945583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8304108500 msgr2=0x7f83041088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.275+0000 7f8308a73700 1 --2- 172.21.15.52:0/4193945583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8304108500 0x7f83041088e0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f82f40098f0 tx=0x7f82f4009c00 comp rx=0 tx=0).stop 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 -- 172.21.15.52:0/4193945583 shutdown_connections 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 --2- 172.21.15.52:0/4193945583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f830406ab30 0x7f830410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 --2- 172.21.15.52:0/4193945583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8304108500 0x7f83041088e0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 --2- 172.21.15.52:0/4193945583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f830406a170 0x7f830406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 -- 172.21.15.52:0/4193945583 >> 172.21.15.52:0/4193945583 conn(0x7f8304078340 msgr2=0x7f8304078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:11.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 -- 172.21.15.52:0/4193945583 shutdown_connections 2024-06-02T23:29:11.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.276+0000 7f8308a73700 1 -- 172.21.15.52:0/4193945583 wait complete. 2024-06-02T23:29:11.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 Processor -- start 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 -- start start 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 0x7f830419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f830406ab30 0x7f830419ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8304108500 0x7f8304198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8304112630 con 0x7f8304108500 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83041124b0 con 0x7f830406a170 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8308a73700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83041127b0 con 0x7f830406ab30 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8304108500 0x7f8304198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f830259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 0x7f830419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f830259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 0x7f830419e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:39002/0 (socket says 172.21.15.52:39002) 2024-06-02T23:29:11.278 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f830259c700 1 -- 172.21.15.52:0/1001390726 learned_addr learned my addr 172.21.15.52:0/1001390726 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.277+0000 7f8301d9b700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f830406ab30 0x7f830419ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8302d9d700 1 -- 172.21.15.52:0/1001390726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f830406ab30 msgr2=0x7f830419ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8302d9d700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f830406ab30 0x7f830419ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8302d9d700 1 -- 172.21.15.52:0/1001390726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 msgr2=0x7f830419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8302d9d700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 0x7f830419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8302d9d700 1 -- 172.21.15.52:0/1001390726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83041990f0 con 0x7f8304108500 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f830259c700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 0x7f830419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:11.279 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8302d9d700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8304108500 0x7f8304198990 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00efc0 tx=0x7f82fc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:11.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f82f37fe700 1 -- 172.21.15.52:0/1001390726 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc0092e0 con 0x7f8304108500 2024-06-02T23:29:11.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8304199380 con 0x7f8304108500 2024-06-02T23:29:11.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.278+0000 7f82f37fe700 1 -- 172.21.15.52:0/1001390726 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82fc004bb0 con 0x7f8304108500 2024-06-02T23:29:11.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.279+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8304101390 con 0x7f8304108500 2024-06-02T23:29:11.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.279+0000 7f82f37fe700 1 -- 172.21.15.52:0/1001390726 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc004020 con 0x7f8304108500 2024-06-02T23:29:11.280 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.281+0000 7f82f37fe700 1 -- 172.21.15.52:0/1001390726 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f82fc01b030 con 0x7f8304108500 2024-06-02T23:29:11.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.282+0000 7f82f37fe700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f82ec0a2c20 0x7f82ec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.282+0000 7f830259c700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f82ec0a2c20 0x7f82ec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.283+0000 7f82f37fe700 1 -- 172.21.15.52:0/1001390726 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f82fc0cea60 con 0x7f8304108500 2024-06-02T23:29:11.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.283+0000 7f830259c700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f82ec0a2c20 0x7f82ec0a50e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009680 tx=0x7f82f4021040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:11.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 Processor -- start 2024-06-02T23:29:11.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 -- start start 2024-06-02T23:29:11.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 0x55b23b4e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e7880 0x55b23b4e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f830259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 0x55b23b4e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f830259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 0x55b23b4e7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51622/0 (socket says 172.21.15.52:51622) 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f830259c700 1 -- 172.21.15.52:0/1119935753 learned_addr learned my addr 172.21.15.52:0/1119935753 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4e1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b23b3d3f60 con 0x55b23b4e6f60 2024-06-02T23:29:11.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b23b432850 con 0x55b23b4e17a0 2024-06-02T23:29:11.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.284+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b23b39a460 con 0x55b23b4e7880 2024-06-02T23:29:11.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f8301d9b700 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e7880 0x55b23b4e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f8302d9d700 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4e1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f830259c700 1 -- 172.21.15.52:0/1119935753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e7880 msgr2=0x55b23b4e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f830259c700 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e7880 0x55b23b4e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f830259c700 1 -- 172.21.15.52:0/1119935753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 msgr2=0x55b23b4e1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f830259c700 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4e1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f830259c700 1 -- 172.21.15.52:0/1119935753 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23b3818d0 con 0x55b23b4e6f60 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f8302d9d700 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f830259c700 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 0x55b23b4e7340 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8304199ad0 tx=0x7f82f4027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8187dd6cf796d745 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.285+0000 7f82f0ff9700 1 -- 172.21.15.52:0/1119935753 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4031040 con 0x55b23b4e6f60 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.286+0000 7f82f0ff9700 1 -- 172.21.15.52:0/1119935753 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82f403a070 con 0x55b23b4e6f60 2024-06-02T23:29:11.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.286+0000 7f82f0ff9700 1 -- 172.21.15.52:0/1119935753 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f400b760 con 0x55b23b4e6f60 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 msgr2=0x55b23b4e7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 0x55b23b4e7340 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f8304199ad0 tx=0x7f82f4027000 comp rx=0 tx=0).stop 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 shutdown_connections 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4e1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e6f60 0x55b23b4e7340 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1119935753 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e7880 0x55b23b4e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 >> 172.21.15.52:0/1119935753 conn(0x55b23b40e080 msgr2=0x55b23b4dc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:11.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 shutdown_connections 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 -- 172.21.15.52:0/1119935753 wait complete. 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.287+0000 7f83132f6b80 1 -- start start 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 Processor -- start 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4ea8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e7880 0x55b23b4eae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b23b3d2fc0 con 0x55b23b4e7880 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b23b42e810 con 0x55b23b4e17a0 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b23b4ee200 con 0x55b23b4e6f60 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f8301d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4ea8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f8301d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4ea8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34160/0 (socket says 172.21.15.52:34160) 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f830259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f83132f6b80 10 client.0 _ll_register_callbacks cb 0x55b23b3c85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f8301d9b700 1 -- 172.21.15.52:0/320452489 learned_addr learned my addr 172.21.15.52:0/320452489 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.288+0000 7f8302d9d700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e7880 0x55b23b4eae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8301d9b700 1 -- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 msgr2=0x55b23b4ed5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8301d9b700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8301d9b700 1 -- 172.21.15.52:0/320452489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e7880 msgr2=0x55b23b4eae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8301d9b700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e7880 0x55b23b4eae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8301d9b700 1 -- 172.21.15.52:0/320452489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b23b3f8750 con 0x55b23b4e6f60 2024-06-02T23:29:11.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f830259c700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8302d9d700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e7880 0x55b23b4eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f8301d9b700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4ea8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009fd0 tx=0x7f82f8012af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e457bbde6682a487 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f82e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f82e9ffb700 1 -- 172.21.15.52:0/320452489 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f8004030 con 0x55b23b4e6f60 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f82e9ffb700 1 -- 172.21.15.52:0/320452489 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f82f80041d0 con 0x55b23b4e6f60 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f403a070 con 0x55b23b4e6f60 2024-06-02T23:29:11.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f82e9ffb700 1 -- 172.21.15.52:0/320452489 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f82f8023070 con 0x55b23b4e6f60 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f83132f6b80 10 client.29059 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b23b4f8a10 con 0x55b23b4e6f60 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.289+0000 7f82e9ffb700 1 -- 172.21.15.52:0/320452489 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82f403a070 con 0x55b23b4e6f60 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f82e9ffb700 1 -- 172.21.15.52:0/320452489 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f82f800f4e0 con 0x55b23b4e6f60 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f83132f6b80 10 client.29059 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f83132f6b80 10 client.29059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f83132f6b80 20 client.29059 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f83132f6b80 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b4fb0a0 0x55b23b4fd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f83132f6b80 4 client.29059 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b23b43fd50 con 0x55b23b4fb0a0 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f830259c700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b4fb0a0 0x55b23b4fd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f830259c700 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b4fb0a0 0x55b23b4fd490 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71b307b17ab1caae server_cookie=e18ba28ba68d951 in_seq=0 out_seq=0 2024-06-02T23:29:11.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.290+0000 7f82e9ffb700 10 client.29059 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:11.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.291+0000 7f82e9ffb700 1 -- 172.21.15.52:0/320452489 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b23b43fd50 con 0x55b23b4fb0a0 2024-06-02T23:29:11.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.291+0000 7f82e9ffb700 10 client.29059 handle_command_reply: tid=0 2024-06-02T23:29:11.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.291+0000 7f83132f6b80 1 client.29059 shutdown 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 msgr2=0x55b23b4ea8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4ea8c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f82f8009fd0 tx=0x7f82f8012af0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 shutdown_connections 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b4fb0a0 0x55b23b4fd490 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e7880 0x55b23b4eae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f82e9ffb700 0 client.29059 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:11.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 --2- 172.21.15.52:0/320452489 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4ea8c0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 >> 172.21.15.52:0/320452489 conn(0x55b23b40e080 msgr2=0x55b23b4db700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:11.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 shutdown_connections 2024-06-02T23:29:11.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 1 -- 172.21.15.52:0/320452489 wait complete. 2024-06-02T23:29:11.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.292+0000 7f83132f6b80 20 client.29059 trim_cache size 0 max 16384 2024-06-02T23:29:11.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 Processor -- start 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 -- start start 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4dbe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b4dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83041aa7a0 con 0x55b23b4e17a0 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83041ac2e0 con 0x55b23b4e7880 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f83132f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83041aae70 con 0x55b23b4e6f60 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51650/0 (socket says 172.21.15.52:51650) 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 -- 172.21.15.52:0/2001947340 learned_addr learned my addr 172.21.15.52:0/2001947340 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f8301d9b700 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4dbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f8302d9d700 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b4dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 -- 172.21.15.52:0/2001947340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 msgr2=0x55b23b4dbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4dbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 -- 172.21.15.52:0/2001947340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 msgr2=0x55b23b4dc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b4dc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.321+0000 7f830259c700 1 -- 172.21.15.52:0/2001947340 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b23b4fdac0 con 0x55b23b4e17a0 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.322+0000 7f830259c700 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed100 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f82f4011040 tx=0x7f82f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841ab4ed5a4fe68e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.322+0000 7f82ebfff700 1 -- 172.21.15.52:0/2001947340 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4044070 con 0x55b23b4e17a0 2024-06-02T23:29:11.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.322+0000 7f82ebfff700 1 -- 172.21.15.52:0/2001947340 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82f403a040 con 0x55b23b4e17a0 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.322+0000 7f82ebfff700 1 -- 172.21.15.52:0/2001947340 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f4031040 con 0x55b23b4e17a0 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 -- 172.21.15.52:0/2001947340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 msgr2=0x55b23b4ed100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed100 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f82f4011040 tx=0x7f82f4009c00 comp rx=0 tx=0).stop 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 -- 172.21.15.52:0/2001947340 shutdown_connections 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b4dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b4ed100 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 --2- 172.21.15.52:0/2001947340 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b4dbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 -- 172.21.15.52:0/2001947340 >> 172.21.15.52:0/2001947340 conn(0x55b23b40e080 msgr2=0x55b23b4e62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 -- 172.21.15.52:0/2001947340 shutdown_connections 2024-06-02T23:29:11.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 -- 172.21.15.52:0/2001947340 wait complete. 2024-06-02T23:29:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.323+0000 7f83132f6b80 1 -- start start 2024-06-02T23:29:11.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 Processor -- start 2024-06-02T23:29:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b502a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b502fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b503500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83041abc10 con 0x55b23b4e17a0 2024-06-02T23:29:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83041ab540 con 0x55b23b4e7880 2024-06-02T23:29:11.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83041a8760 con 0x55b23b4e6f60 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f83132f6b80 10 client.0 _ll_register_callbacks cb 0x55b23b3c85b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f830259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b502a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f8301d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b502fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f8302d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b503500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f830259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b502a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51660/0 (socket says 172.21.15.52:51660) 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.324+0000 7f830259c700 1 -- 172.21.15.52:0/1269965490 learned_addr learned my addr 172.21.15.52:0/1269965490 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f8301d9b700 1 -- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 msgr2=0x55b23b503500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f8301d9b700 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b503500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f8301d9b700 1 -- 172.21.15.52:0/1269965490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 msgr2=0x55b23b502a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f8301d9b700 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b502a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f8301d9b700 1 -- 172.21.15.52:0/1269965490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b23b564e00 con 0x55b23b4e6f60 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f830259c700 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b502a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f8301d9b700 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b502fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f82f800faf0 tx=0x7f82f800f710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3445884bb0d902e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:11.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f82e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f82e9ffb700 1 -- 172.21.15.52:0/1269965490 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82f800fd80 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f82e9ffb700 1 -- 172.21.15.52:0/1269965490 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f82f80050c0 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82f403a040 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f82e9ffb700 1 -- 172.21.15.52:0/1269965490 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f82f80052c0 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f83132f6b80 10 client.29065 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.325+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b23b5660d0 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f82e9ffb700 1 -- 172.21.15.52:0/1269965490 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f82f403a040 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f82e9ffb700 1 -- 172.21.15.52:0/1269965490 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f82f8032050 con 0x55b23b4e6f60 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f83132f6b80 10 client.29065 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:11.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f83132f6b80 10 client.29065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f83132f6b80 20 client.29065 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b5666f0 0x55b23b568b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f83132f6b80 4 client.29065 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b23b43a2d0 con 0x55b23b5666f0 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f830259c700 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b5666f0 0x55b23b568b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f82e9ffb700 10 client.29065 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.326+0000 7f830259c700 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b5666f0 0x55b23b568b10 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e84932469c6776 server_cookie=4dff1ec0987cbbbf in_seq=0 out_seq=0 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f82e9ffb700 1 -- 172.21.15.52:0/1269965490 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b23b43a2d0 con 0x55b23b5666f0 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f82e9ffb700 10 client.29065 handle_command_reply: tid=0 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 client.29065 shutdown 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 msgr2=0x55b23b502fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b502fc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f82f800faf0 tx=0x7f82f800f710 comp rx=0 tx=0).stop 2024-06-02T23:29:11.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 shutdown_connections 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b23b5666f0 0x55b23b568b10 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f82e9ffb700 0 client.29065 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b23b4e7880 0x55b23b503500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b23b4e17a0 0x55b23b502a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 --2- 172.21.15.52:0/1269965490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b23b4e6f60 0x55b23b502fc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 >> 172.21.15.52:0/1269965490 conn(0x55b23b40e080 msgr2=0x55b23b4e0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 shutdown_connections 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.327+0000 7f83132f6b80 1 -- 172.21.15.52:0/1269965490 wait complete. 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.328+0000 7f83132f6b80 20 client.29065 trim_cache size 0 max 16384 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:29:11.332 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:29:11.333 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:29:11.333 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:29:11.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.328+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f82ec0a2c20 msgr2=0x7f82ec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.328+0000 7f8308a73700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f82ec0a2c20 0x7f82ec0a50e0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f82f4009680 tx=0x7f82f4021040 comp rx=0 tx=0).stop 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.328+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8304108500 msgr2=0x7f8304198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.328+0000 7f8308a73700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8304108500 0x7f8304198990 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f82fc00efc0 tx=0x7f82fc00c3a0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 shutdown_connections 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f82ec0a2c20 0x7f82ec0a50e0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8304108500 0x7f8304198990 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f830406a170 0x7f830419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 --2- 172.21.15.52:0/1001390726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f830406ab30 0x7f830419ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 >> 172.21.15.52:0/1001390726 conn(0x7f8304078340 msgr2=0x7f8304100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 shutdown_connections 2024-06-02T23:29:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:11.329+0000 7f8308a73700 1 -- 172.21.15.52:0/1001390726 wait complete. 2024-06-02T23:29:11.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:11 smithi052 ceph-mon[32340]: pgmap v950: 97 pgs: 97 active+clean; 398 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-06-02T23:29:11.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 30688 6.91 MB/sec execute 139 sec latency 98.549 ms 2024-06-02T23:29:11.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:11 smithi123 ceph-mon[35977]: pgmap v950: 97 pgs: 97 active+clean; 398 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-06-02T23:29:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:11 smithi174 ceph-mon[35994]: pgmap v950: 97 pgs: 97 active+clean; 398 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 311 op/s 2024-06-02T23:29:12.215 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:29:12.215 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:29:12.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 30831 6.92 MB/sec execute 140 sec latency 61.856 ms 2024-06-02T23:29:12.585 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:13.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:13 smithi052 ceph-mon[32340]: pgmap v951: 97 pgs: 97 active+clean; 401 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-06-02T23:29:13.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 31032 7.01 MB/sec execute 141 sec latency 69.297 ms 2024-06-02T23:29:13.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:13 smithi123 ceph-mon[35977]: pgmap v951: 97 pgs: 97 active+clean; 401 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-06-02T23:29:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:13 smithi174 ceph-mon[35994]: pgmap v951: 97 pgs: 97 active+clean; 401 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 297 op/s 2024-06-02T23:29:14.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 31295 7.09 MB/sec execute 142 sec latency 60.785 ms 2024-06-02T23:29:14.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 -- 172.21.15.52:0/3351802960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac10ef30 msgr2=0x7f37ac111320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:14.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 --2- 172.21.15.52:0/3351802960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac10ef30 0x7f37ac111320 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f37a400b0a0 tx=0x7f37a400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 -- 172.21.15.52:0/3351802960 shutdown_connections 2024-06-02T23:29:14.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 --2- 172.21.15.52:0/3351802960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac10ef30 0x7f37ac111320 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 --2- 172.21.15.52:0/3351802960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac06aa30 0x7f37ac06ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 --2- 172.21.15.52:0/3351802960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06b350 0x7f37ac10e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 -- 172.21.15.52:0/3351802960 >> 172.21.15.52:0/3351802960 conn(0x7f37ac078300 msgr2=0x7f37ac078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:14.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 -- 172.21.15.52:0/3351802960 shutdown_connections 2024-06-02T23:29:14.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.504+0000 7f37b32af700 1 -- 172.21.15.52:0/3351802960 wait complete. 2024-06-02T23:29:14.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.505+0000 7f37b32af700 1 Processor -- start 2024-06-02T23:29:14.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.505+0000 7f37b32af700 1 -- start start 2024-06-02T23:29:14.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.505+0000 7f37b32af700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 0x7f37ac103660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:14.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.505+0000 7f37b32af700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 0x7f37ac103ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b32af700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac10ef30 0x7f37ac1040e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b084a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 0x7f37ac103ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b32af700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac114840 con 0x7f37ac06b350 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b32af700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1146c0 con 0x7f37ac06aa30 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b32af700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37ac1149c0 con 0x7f37ac10ef30 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 0x7f37ac103660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 0x7f37ac103660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56286/0 (socket says 172.21.15.52:56286) 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b084a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 0x7f37ac103ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39394/0 (socket says 172.21.15.52:39394) 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 -- 172.21.15.52:0/983122748 learned_addr learned my addr 172.21.15.52:0/983122748 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 -- 172.21.15.52:0/983122748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac10ef30 msgr2=0x7f37ac1040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac10ef30 0x7f37ac1040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 -- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 msgr2=0x7f37ac103ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 0x7f37ac103ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b104b700 1 -- 172.21.15.52:0/983122748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37ac1ab3e0 con 0x7f37ac06aa30 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.506+0000 7f37b084a700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 0x7f37ac103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.507+0000 7f37b104b700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 0x7f37ac103660 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f37a800e9d0 tx=0x7f37a800ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.507+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a800cbd0 con 0x7f37ac06aa30 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.507+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37a800cd70 con 0x7f37ac06aa30 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.507+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37a8018680 con 0x7f37ac06aa30 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.507+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37ac1ab6d0 con 0x7f37ac06aa30 2024-06-02T23:29:14.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.507+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37ac1abbc0 con 0x7f37ac06aa30 2024-06-02T23:29:14.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.508+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37ac1019a0 con 0x7f37ac06aa30 2024-06-02T23:29:14.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.509+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37a8010430 con 0x7f37ac06aa30 2024-06-02T23:29:14.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.510+0000 7f37a27fc700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37980a2d20 0x7f37980a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:14.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.511+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f37a80cf6a0 con 0x7f37ac06aa30 2024-06-02T23:29:14.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.511+0000 7f37b084a700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37980a2d20 0x7f37980a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:14.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.511+0000 7f37b084a700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37980a2d20 0x7f37980a51e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f37ac104e20 tx=0x7f379c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:14.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.515+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f37a809c5d0 con 0x7f37ac06aa30 2024-06-02T23:29:14.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.942+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f37ac02de60 con 0x7f37ac06aa30 2024-06-02T23:29:14.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.943+0000 7f37a27fc700 1 -- 172.21.15.52:0/983122748 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f37a809bed0 con 0x7f37ac06aa30 2024-06-02T23:29:14.943 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:29:14.943 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:29:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37980a2d20 msgr2=0x7f37980a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37980a2d20 0x7f37980a51e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f37ac104e20 tx=0x7f379c009260 comp rx=0 tx=0).stop 2024-06-02T23:29:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 msgr2=0x7f37ac103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 0x7f37ac103660 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f37a800e9d0 tx=0x7f37a800ece0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 shutdown_connections 2024-06-02T23:29:14.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37980a2d20 0x7f37980a51e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37ac10ef30 0x7f37ac1040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37ac06aa30 0x7f37ac103660 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 --2- 172.21.15.52:0/983122748 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37ac06b350 0x7f37ac103ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.946+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 >> 172.21.15.52:0/983122748 conn(0x7f37ac078300 msgr2=0x7f37ac1103a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.947+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 shutdown_connections 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:14.947+0000 7f37b32af700 1 -- 172.21.15.52:0/983122748 wait complete. 2024-06-02T23:29:14.947 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:29:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:15 smithi052 ceph-mon[32340]: pgmap v952: 97 pgs: 97 active+clean; 404 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-06-02T23:29:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:15 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/983122748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:15.445 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:15 smithi174 ceph-mon[35994]: pgmap v952: 97 pgs: 97 active+clean; 404 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-06-02T23:29:15.445 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:15 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/983122748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:15.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 31446 7.11 MB/sec execute 143 sec latency 72.405 ms 2024-06-02T23:29:15.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:15 smithi123 ceph-mon[35977]: pgmap v952: 97 pgs: 97 active+clean; 404 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 286 op/s 2024-06-02T23:29:15.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:15 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/983122748' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:15.759 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:29:16.110 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:16.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 31577 7.11 MB/sec execute 144 sec latency 83.741 ms 2024-06-02T23:29:17.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 31752 7.15 MB/sec execute 145 sec latency 59.150 ms 2024-06-02T23:29:17.487 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:17 smithi052 ceph-mon[32340]: pgmap v953: 97 pgs: 97 active+clean; 413 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 318 op/s 2024-06-02T23:29:17.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:17 smithi123 ceph-mon[35977]: pgmap v953: 97 pgs: 97 active+clean; 413 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 318 op/s 2024-06-02T23:29:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:17 smithi174 ceph-mon[35994]: pgmap v953: 97 pgs: 97 active+clean; 413 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 318 op/s 2024-06-02T23:29:18.187 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.187+0000 7f9fc4587700 1 -- 172.21.15.52:0/4183628483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06aa00 msgr2=0x7f9fbc06ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.187+0000 7f9fc4587700 1 --2- 172.21.15.52:0/4183628483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06aa00 0x7f9fbc06ae80 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f9fac009990 tx=0x7f9fac009ca0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.187+0000 7f9fc4587700 1 -- 172.21.15.52:0/4183628483 shutdown_connections 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.187+0000 7f9fc4587700 1 --2- 172.21.15.52:0/4183628483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc10f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.187+0000 7f9fc4587700 1 --2- 172.21.15.52:0/4183628483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fbc1084f0 0x7f9fbc1088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.187+0000 7f9fc4587700 1 --2- 172.21.15.52:0/4183628483 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06aa00 0x7f9fbc06ae80 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.188+0000 7f9fc4587700 1 -- 172.21.15.52:0/4183628483 >> 172.21.15.52:0/4183628483 conn(0x7f9fbc078300 msgr2=0x7f9fbc078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.188+0000 7f9fc4587700 1 -- 172.21.15.52:0/4183628483 shutdown_connections 2024-06-02T23:29:18.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.188+0000 7f9fc4587700 1 -- 172.21.15.52:0/4183628483 wait complete. 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 Processor -- start 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 -- start start 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fbc06aa00 0x7f9fbc101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fbc1084f0 0x7f9fbc101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1126f0 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc112570 con 0x7f9fbc1084f0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc4587700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc112870 con 0x7f9fbc06aa00 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39422/0 (socket says 172.21.15.52:39422) 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc1b22700 1 -- 172.21.15.52:0/3285380576 learned_addr learned my addr 172.21.15.52:0/3285380576 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc2323700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fbc06aa00 0x7f9fbc101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.189+0000 7f9fc2b24700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fbc1084f0 0x7f9fbc101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc1b22700 1 -- 172.21.15.52:0/3285380576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fbc06aa00 msgr2=0x7f9fbc101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fbc06aa00 0x7f9fbc101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc1b22700 1 -- 172.21.15.52:0/3285380576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fbc1084f0 msgr2=0x7f9fbc101ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fbc1084f0 0x7f9fbc101ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc1b22700 1 -- 172.21.15.52:0/3285380576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fbc1a71b0 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc101990 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00b2e0 tx=0x7f9fac0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fb37fe700 1 -- 172.21.15.52:0/3285380576 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac026070 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fbc1a7440 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fbc1a78f0 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.190+0000 7f9fb37fe700 1 -- 172.21.15.52:0/3285380576 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac0173f0 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.191+0000 7f9fb37fe700 1 -- 172.21.15.52:0/3285380576 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac02a610 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.193+0000 7f9fb37fe700 1 -- 172.21.15.52:0/3285380576 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9fac029070 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.194+0000 7f9fb37fe700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9fa80a2ca0 0x7f9fa80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.194+0000 7f9fb37fe700 1 -- 172.21.15.52:0/3285380576 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9fac0cfcb0 con 0x7f9fbc06b3c0 2024-06-02T23:29:18.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.194+0000 7f9fc2323700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9fa80a2ca0 0x7f9fa80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 Processor -- start 2024-06-02T23:29:18.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 -- start start 2024-06-02T23:29:18.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fc2323700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9fa80a2ca0 0x7f9fa80a5160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009680 tx=0x7f9fb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:18.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 0x55c413148a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c4131493d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c413032f60 con 0x55c41314e770 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c413091850 con 0x55c41314f090 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c412ff9460 con 0x55c413148fb0 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.195+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 0x55c413148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 0x55c413148a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56306/0 (socket says 172.21.15.52:56306) 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 -- 172.21.15.52:0/1351135947 learned_addr learned my addr 172.21.15.52:0/1351135947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc2323700 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 -- 172.21.15.52:0/1351135947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 msgr2=0x55c4131493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c4131493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 -- 172.21.15.52:0/1351135947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 msgr2=0x55c41314eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 -- 172.21.15.52:0/1351135947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c412fe08d0 con 0x55c41314f090 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 0x55c413148a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0a4c80 tx=0x7f9fac09bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=200b8713a974aa4e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:18.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fb0ff9700 1 -- 172.21.15.52:0/1351135947 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0a51b0 con 0x55c41314f090 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fb0ff9700 1 -- 172.21.15.52:0/1351135947 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac0bc040 con 0x55c41314f090 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.196+0000 7f9fb0ff9700 1 -- 172.21.15.52:0/1351135947 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fac09ca70 con 0x55c41314f090 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/1351135947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 msgr2=0x55c413148a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 0x55c413148a70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f9fac0a4c80 tx=0x7f9fac09bfd0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/1351135947 shutdown_connections 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c4131493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/1351135947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314f090 0x55c413148a70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/1351135947 >> 172.21.15.52:0/1351135947 conn(0x55c41306d080 msgr2=0x55c41313b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/1351135947 shutdown_connections 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.197+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/1351135947 wait complete. 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 -- start start 2024-06-02T23:29:18.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 Processor -- start 2024-06-02T23:29:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413154e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c413152160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c4131526a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c413031fc0 con 0x55c41314e770 2024-06-02T23:29:18.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c41308d810 con 0x55c413148fb0 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c413155820 con 0x55c41314f090 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fcee0ab80 10 client.0 _ll_register_callbacks cb 0x55c4130275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c413152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fc2323700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fc2323700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413154e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56308/0 (socket says 172.21.15.52:56308) 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fc2323700 1 -- 172.21.15.52:0/586239592 learned_addr learned my addr 172.21.15.52:0/586239592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.198+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c413152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39432/0 (socket says 172.21.15.52:39432) 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fc2323700 1 -- 172.21.15.52:0/586239592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 msgr2=0x55c4131526a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fc2323700 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c4131526a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fc2323700 1 -- 172.21.15.52:0/586239592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 msgr2=0x55c413152160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fc2323700 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c413152160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fc2323700 1 -- 172.21.15.52:0/586239592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c413057750 con 0x55c413148fb0 2024-06-02T23:29:18.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fc2323700 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413154e60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9fb801e8d0 tx=0x7f9fb80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1932843aa244c58e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.199+0000 7f9fa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/586239592 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb800e8c0 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/586239592 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9fb800ea60 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fac0bc040 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/586239592 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9fb800ec60 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fcee0ab80 10 client.29435 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c413160000 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.200+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/586239592 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fac0bc040 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/586239592 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9fb80323d0 con 0x55c413148fb0 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fcee0ab80 10 client.29435 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fcee0ab80 10 client.29435 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fcee0ab80 20 client.29435 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c413162640 0x55c413164a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fcee0ab80 4 client.29435 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c41309ed50 con 0x55c413162640 2024-06-02T23:29:18.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.201+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c413162640 0x55c413164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.202+0000 7f9fa5ffb700 10 client.29435 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:29:18.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.202+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c413162640 0x55c413164a30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=414754b6def929cf server_cookie=e21e0579b97e7bb2 in_seq=0 out_seq=0 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.202+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/586239592 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c41309ed50 con 0x55c413162640 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.202+0000 7f9fa5ffb700 10 client.29435 handle_command_reply: tid=0 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 client.29435 shutdown 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 msgr2=0x55c413154e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413154e60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9fb801e8d0 tx=0x7f9fb80047c0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 shutdown_connections 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c4131526a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c413162640 0x55c413164a30 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413154e60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/586239592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c413152160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 >> 172.21.15.52:0/586239592 conn(0x55c41306d080 msgr2=0x55c41313a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fa5ffb700 0 client.29435 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:29:18.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 shutdown_connections 2024-06-02T23:29:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.203+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/586239592 wait complete. 2024-06-02T23:29:18.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.204+0000 7f9fcee0ab80 20 client.29435 trim_cache size 0 max 16384 2024-06-02T23:29:18.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.226+0000 7f9fcee0ab80 1 Processor -- start 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.226+0000 7f9fcee0ab80 1 -- start start 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413151100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c41314fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1aa880 con 0x55c41314e770 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ac3c0 con 0x55c413148fb0 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1aaf50 con 0x55c41314f090 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56324/0 (socket says 172.21.15.52:56324) 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 -- 172.21.15.52:0/88131667 learned_addr learned my addr 172.21.15.52:0/88131667 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2b24700 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c41314fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 -- 172.21.15.52:0/88131667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 msgr2=0x55c41314fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c41314fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 -- 172.21.15.52:0/88131667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 msgr2=0x55c41314c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2323700 1 -- 172.21.15.52:0/88131667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c413150710 con 0x55c413148fb0 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.227+0000 7f9fc2b24700 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c41314fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.228+0000 7f9fc2323700 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413151100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005a80 tx=0x7f9fb800e3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54fb5fd5b2b593a1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.228+0000 7f9fa7fff700 1 -- 172.21.15.52:0/88131667 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb80325e0 con 0x55c413148fb0 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.228+0000 7f9fa7fff700 1 -- 172.21.15.52:0/88131667 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fb8032a10 con 0x55c413148fb0 2024-06-02T23:29:18.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.228+0000 7f9fa7fff700 1 -- 172.21.15.52:0/88131667 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb8003680 con 0x55c413148fb0 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/88131667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 msgr2=0x55c413151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413151100 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005a80 tx=0x7f9fb800e3b0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/88131667 shutdown_connections 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c41314f090 0x55c41314fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c413148fb0 0x55c413151100 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/88131667 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314e770 0x55c41314c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/88131667 >> 172.21.15.52:0/88131667 conn(0x55c41306d080 msgr2=0x55c41314f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:18.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/88131667 shutdown_connections 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/88131667 wait complete. 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.229+0000 7f9fcee0ab80 1 -- start start 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 Processor -- start 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c41315aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314e770 0x55c41315afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314f090 0x55c4131693d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1abcf0 con 0x55c41314f090 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc1ab620 con 0x55c41314e770 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9fbc19c790 con 0x55c413148fb0 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fcee0ab80 10 client.0 _ll_register_callbacks cb 0x55c4130275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fc2323700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c41315aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fc1b22700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314e770 0x55c41315afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fc2323700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c41315aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53652/0 (socket says 172.21.15.52:53652) 2024-06-02T23:29:18.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fc2323700 1 -- 172.21.15.52:0/3948519081 learned_addr learned my addr 172.21.15.52:0/3948519081 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.230+0000 7f9fc2b24700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314f090 0x55c4131693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2323700 1 -- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314e770 msgr2=0x55c41315afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2323700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314e770 0x55c41315afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2323700 1 -- 172.21.15.52:0/3948519081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314f090 msgr2=0x55c4131693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2323700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314f090 0x55c4131693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2323700 1 -- 172.21.15.52:0/3948519081 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c413169960 con 0x55c413148fb0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2b24700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314f090 0x55c4131693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fc2323700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c41315aa70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8016040 tx=0x7f9fb80058c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6db0a50e198798ce server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fa5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/3948519081 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fb803f020 con 0x55c413148fb0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9fb8032a10 con 0x55c413148fb0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/3948519081 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9fb8005300 con 0x55c413148fb0 2024-06-02T23:29:18.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/3948519081 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9fb8005500 con 0x55c413148fb0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fcee0ab80 10 client.29077 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c4131ccaf0 con 0x55c413148fb0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.231+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/3948519081 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9fb8032a10 con 0x55c413148fb0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/3948519081 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9fb8029570 con 0x55c413148fb0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fcee0ab80 10 client.29077 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fcee0ab80 10 client.29077 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fcee0ab80 20 client.29077 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c4131cd0e0 0x55c4131cf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fcee0ab80 4 client.29077 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c4130992d0 con 0x55c4131cd0e0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c4131cd0e0 0x55c4131cf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fa5ffb700 10 client.29077 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.232+0000 7f9fc1b22700 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c4131cd0e0 0x55c4131cf500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e0bae0f09724d65 server_cookie=4fc7f908643eae64 in_seq=0 out_seq=0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fa5ffb700 1 -- 172.21.15.52:0/3948519081 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c4130992d0 con 0x55c4131cd0e0 2024-06-02T23:29:18.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fa5ffb700 10 client.29077 handle_command_reply: tid=0 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 client.29077 shutdown 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 msgr2=0x55c41315aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c41315aa70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8016040 tx=0x7f9fb80058c0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 shutdown_connections 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c41314f090 0x55c4131693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c413148fb0 0x55c41315aa70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c4131cd0e0 0x55c4131cf500 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 --2- 172.21.15.52:0/3948519081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c41314e770 0x55c41315afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fa5ffb700 0 client.29077 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 >> 172.21.15.52:0/3948519081 conn(0x55c41306d080 msgr2=0x55c41314c6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 shutdown_connections 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 1 -- 172.21.15.52:0/3948519081 wait complete. 2024-06-02T23:29:18.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.233+0000 7f9fcee0ab80 20 client.29077 trim_cache size 0 max 16384 2024-06-02T23:29:18.238 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:29:18.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9fa80a2ca0 msgr2=0x7f9fa80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9fa80a2ca0 0x7f9fa80a5160 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8009680 tx=0x7f9fb8009260 comp rx=0 tx=0).stop 2024-06-02T23:29:18.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 msgr2=0x7f9fbc101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:18.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc101990 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00b2e0 tx=0x7f9fac0057c0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 shutdown_connections 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9fa80a2ca0 0x7f9fa80a5160 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9fbc1084f0 0x7f9fbc101ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9fbc06aa00 0x7f9fbc101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 --2- 172.21.15.52:0/3285380576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9fbc06b3c0 0x7f9fbc101990 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 >> 172.21.15.52:0/3285380576 conn(0x7f9fbc078300 msgr2=0x7f9fbc1000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.234+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 shutdown_connections 2024-06-02T23:29:18.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:18.235+0000 7f9fc4587700 1 -- 172.21.15.52:0/3285380576 wait complete. 2024-06-02T23:29:18.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 31904 7.16 MB/sec execute 146 sec latency 133.099 ms 2024-06-02T23:29:18.963 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:29:19.326 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:19.460 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:19 smithi052 ceph-mon[32340]: pgmap v954: 97 pgs: 97 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 3.9 MiB/s wr, 250 op/s 2024-06-02T23:29:19.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 32055 7.15 MB/sec execute 147 sec latency 76.322 ms 2024-06-02T23:29:19.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:19 smithi123 ceph-mon[35977]: pgmap v954: 97 pgs: 97 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 3.9 MiB/s wr, 250 op/s 2024-06-02T23:29:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:19 smithi174 ceph-mon[35994]: pgmap v954: 97 pgs: 97 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 897 KiB/s rd, 3.9 MiB/s wr, 250 op/s 2024-06-02T23:29:20.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 32211 7.16 MB/sec execute 148 sec latency 58.842 ms 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 -- 172.21.15.52:0/3683805906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41000e0 msgr2=0x7f92f4100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 --2- 172.21.15.52:0/3683805906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41000e0 0x7f92f4100500 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f92e4009900 tx=0x7f92e402a3f0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 -- 172.21.15.52:0/3683805906 shutdown_connections 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 --2- 172.21.15.52:0/3683805906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f4100a40 0x7f92f4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 --2- 172.21.15.52:0/3683805906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f41013e0 0x7f92f4101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 --2- 172.21.15.52:0/3683805906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41000e0 0x7f92f4100500 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.557 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 -- 172.21.15.52:0/3683805906 >> 172.21.15.52:0/3683805906 conn(0x7f92f40fbce0 msgr2=0x7f92f40fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 -- 172.21.15.52:0/3683805906 shutdown_connections 2024-06-02T23:29:20.558 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.557+0000 7f92f91a0640 1 -- 172.21.15.52:0/3683805906 wait complete. 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 Processor -- start 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 -- start start 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 0x7f92f4199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f4100a40 0x7f92f419a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41013e0 0x7f92f41a1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92f410f550 con 0x7f92f41013e0 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92f410f3d0 con 0x7f92f4100a40 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f91a0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92f410f6d0 con 0x7f92f41000e0 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f899f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41013e0 0x7f92f41a1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 0x7f92f4199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.560 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.558+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f4100a40 0x7f92f419a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 0x7f92f4199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53674/0 (socket says 172.21.15.52:53674) 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f3fff640 1 -- 172.21.15.52:0/2519033653 learned_addr learned my addr 172.21.15.52:0/2519033653 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92ebfff640 1 -- 172.21.15.52:0/2519033653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 msgr2=0x7f92f4199c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 0x7f92f4199c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92ebfff640 1 -- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41013e0 msgr2=0x7f92f41a1230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41013e0 0x7f92f41a1230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92ebfff640 1 -- 172.21.15.52:0/2519033653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f41a18d0 con 0x7f92f4100a40 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 0x7f92f4199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f899f640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41013e0 0x7f92f41a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f4100a40 0x7f92f419a1d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00e9f0 tx=0x7f92dc00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f1ffb640 1 -- 172.21.15.52:0/2519033653 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00ccc0 con 0x7f92f4100a40 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f4102350 con 0x7f92f4100a40 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.559+0000 7f92f1ffb640 1 -- 172.21.15.52:0/2519033653 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92dc0035f0 con 0x7f92f4100a40 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.560+0000 7f92f1ffb640 1 -- 172.21.15.52:0/2519033653 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92dc00ce60 con 0x7f92f4100a40 2024-06-02T23:29:20.561 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.560+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f41028f0 con 0x7f92f4100a40 2024-06-02T23:29:20.563 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.562+0000 7f92f1ffb640 1 -- 172.21.15.52:0/2519033653 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f92dc0037a0 con 0x7f92f4100a40 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.562+0000 7f92fd1ea740 1 Processor -- start 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 -- start start 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d90f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90fbf0 0x55f78d9146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d914d60 0x55f78d9171f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f78d8724f0 con 0x55f78d914d60 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f78d810870 con 0x55f78d90f200 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f78d86e4b0 con 0x55f78d90fbf0 2024-06-02T23:29:20.564 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d90f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d90f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56350/0 (socket says 172.21.15.52:56350) 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d914d60 0x55f78d9171f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 -- 172.21.15.52:0/3821651143 learned_addr learned my addr 172.21.15.52:0/3821651143 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f3fff640 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90fbf0 0x55f78d9146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 -- 172.21.15.52:0/3821651143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90fbf0 msgr2=0x55f78d9146f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90fbf0 0x55f78d9146f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 -- 172.21.15.52:0/3821651143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d914d60 msgr2=0x55f78d9171f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.565 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d914d60 0x55f78d9171f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.563+0000 7f92f899f640 1 -- 172.21.15.52:0/3821651143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d82c9b0 con 0x55f78d90f200 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92f1ffb640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92e00a2c00 0x7f92e00a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92f899f640 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d90f620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92ec014f90 tx=0x7f92ec009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7157333fe2704d01 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92f1ffb640 1 -- 172.21.15.52:0/2519033653 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f92dc0cf2b0 con 0x7f92f4100a40 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92e00a2c00 0x7f92e00a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92ea7fc640 1 -- 172.21.15.52:0/3821651143 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec022030 con 0x55f78d90f200 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92ea7fc640 1 -- 172.21.15.52:0/3821651143 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec00bdd0 con 0x55f78d90f200 2024-06-02T23:29:20.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.564+0000 7f92ea7fc640 1 -- 172.21.15.52:0/3821651143 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec01e400 con 0x55f78d90f200 2024-06-02T23:29:20.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 -- 172.21.15.52:0/3821651143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 msgr2=0x55f78d90f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d90f620 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f92ec014f90 tx=0x7f92ec009500 comp rx=0 tx=0).stop 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92e00a2c00 0x7f92e00a50c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 -- 172.21.15.52:0/3821651143 shutdown_connections 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90fbf0 0x55f78d9146f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d914d60 0x55f78d9171f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/3821651143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d90f620 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 -- 172.21.15.52:0/3821651143 >> 172.21.15.52:0/3821651143 conn(0x55f78d90a870 msgr2=0x55f78d90ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 -- 172.21.15.52:0/3821651143 shutdown_connections 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 -- 172.21.15.52:0/3821651143 wait complete. 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.565+0000 7f92fd1ea740 1 -- start start 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 Processor -- start 2024-06-02T23:29:20.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d9209b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f78d917d10 con 0x55f78d90f200 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f78d7fbae0 con 0x55f78d90fbf0 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f78d917e90 con 0x55f78d914d60 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92fd1ea740 10 client.0 _ll_register_callbacks cb 0x55f78d8349b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92f899f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d9209b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56358/0 (socket says 172.21.15.52:56358) 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39478/0 (socket says 172.21.15.52:39478) 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92ebfff640 1 -- 172.21.15.52:0/2006382699 learned_addr learned my addr 172.21.15.52:0/2006382699 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92f899f640 1 -- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90fbf0 msgr2=0x55f78d9199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.566+0000 7f92f899f640 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.571 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92f899f640 1 -- 172.21.15.52:0/2006382699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90f200 msgr2=0x55f78d9194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92f899f640 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92f899f640 1 -- 172.21.15.52:0/2006382699 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d586210 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92f899f640 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d9209b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f92ec014f60 tx=0x7f92ec005290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a026d1f5a8ac02ee server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92d37fe640 1 -- 172.21.15.52:0/2006382699 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec003970 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92d37fe640 1 -- 172.21.15.52:0/2006382699 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f92ec003b10 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ec00bdd0 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92d37fe640 1 -- 172.21.15.52:0/2006382699 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f92ec003d10 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92fd1ea740 10 client.29089 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.567+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f78d652620 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92d37fe640 1 -- 172.21.15.52:0/2006382699 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec00bdd0 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92d37fe640 1 -- 172.21.15.52:0/2006382699 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f92ec0334d0 con 0x55f78d914d60 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92fd1ea740 10 client.29089 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:20.572 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92fd1ea740 10 client.29089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92fd1ea740 20 client.29089 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d925010 0x55f78d927400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92fd1ea740 4 client.29089 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f78d885f10 con 0x55f78d925010 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.568+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d925010 0x55f78d927400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.569+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d925010 0x55f78d927400 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec0fc08c4c4fafe server_cookie=ade483fcb39b1c0c in_seq=0 out_seq=0 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.569+0000 7f92d37fe640 10 client.29089 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.569+0000 7f92d37fe640 1 -- 172.21.15.52:0/2006382699 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f78d885f10 con 0x55f78d925010 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.569+0000 7f92d37fe640 10 client.29089 handle_command_reply: tid=0 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 client.29089 shutdown 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 msgr2=0x55f78d9209b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d9209b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f92ec014f60 tx=0x7f92ec005290 comp rx=0 tx=0).stop 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 shutdown_connections 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d925010 0x55f78d927400 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d9209b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.573 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92d37fe640 0 client.29089 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2006382699 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 >> 172.21.15.52:0/2006382699 conn(0x55f78d90a870 msgr2=0x55f78d9099d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 shutdown_connections 2024-06-02T23:29:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2006382699 wait complete. 2024-06-02T23:29:20.574 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.570+0000 7f92fd1ea740 20 client.29089 trim_cache size 0 max 16384 2024-06-02T23:29:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 Processor -- start 2024-06-02T23:29:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 -- start start 2024-06-02T23:29:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 0x55f78d9208d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41a6a20 con 0x55f78d90fbf0 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41a8560 con 0x55f78d914d60 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.592+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41a70f0 con 0x55f78d90f200 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 0x55f78d9208d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92ebfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39488/0 (socket says 172.21.15.52:39488) 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92ebfff640 1 -- 172.21.15.52:0/2509081560 learned_addr learned my addr 172.21.15.52:0/2509081560 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 0x55f78d9208d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56362/0 (socket says 172.21.15.52:56362) 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 -- 172.21.15.52:0/2509081560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90f200 msgr2=0x55f78d9194a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 -- 172.21.15.52:0/2509081560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 msgr2=0x55f78d9199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 -- 172.21.15.52:0/2509081560 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f78d927770 con 0x55f78d914d60 2024-06-02T23:29:20.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92f899f640 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 0x55f78d9208d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f92ec033940 tx=0x7f92ec009900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=860c9364af115d98 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92e9ffb640 1 -- 172.21.15.52:0/2509081560 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec040020 con 0x55f78d914d60 2024-06-02T23:29:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.593+0000 7f92e9ffb640 1 -- 172.21.15.52:0/2509081560 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec0335a0 con 0x55f78d914d60 2024-06-02T23:29:20.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92e9ffb640 1 -- 172.21.15.52:0/2509081560 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92ec022040 con 0x55f78d914d60 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2509081560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 msgr2=0x55f78d9208d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 0x55f78d9208d0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f92ec033940 tx=0x7f92ec009900 comp rx=0 tx=0).stop 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2509081560 shutdown_connections 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d914d60 0x55f78d9208d0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d9199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2509081560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d90f200 0x55f78d9194a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2509081560 >> 172.21.15.52:0/2509081560 conn(0x55f78d90a870 msgr2=0x55f78d90ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2509081560 shutdown_connections 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.594+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2509081560 wait complete. 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.595+0000 7f92fd1ea740 1 -- start start 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.595+0000 7f92fd1ea740 1 Processor -- start 2024-06-02T23:29:20.598 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.595+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d920470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.595+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d91cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.595+0000 7f92fd1ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d91d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41a7e90 con 0x55f78d90fbf0 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41a77c0 con 0x55f78d90f200 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92fd1ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f92f41a4a00 con 0x55f78d914d60 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d920470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d920470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56378/0 (socket says 172.21.15.52:56378) 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 -- 172.21.15.52:0/2246710182 learned_addr learned my addr 172.21.15.52:0/2246710182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f899f640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d91d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d91cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92fd1ea740 10 client.0 _ll_register_callbacks cb 0x7f92dc0d2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 -- 172.21.15.52:0/2246710182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 msgr2=0x55f78d91d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d91d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 -- 172.21.15.52:0/2246710182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 msgr2=0x55f78d91cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:20.599 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d91cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 -- 172.21.15.52:0/2246710182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f78d920aa0 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d91cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f899f640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d91d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92f3fff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d920470 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f92e4031a70 tx=0x7f92e4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb01fe93b3dc71dc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.596+0000 7f92d37fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92d37fe640 1 -- 172.21.15.52:0/2246710182 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92e4035040 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92d37fe640 1 -- 172.21.15.52:0/2246710182 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f92e4003c50 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92ec0335a0 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92d37fe640 1 -- 172.21.15.52:0/2246710182 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f92e4003e50 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92fd1ea740 10 client.29459 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f78d98ec80 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92d37fe640 1 -- 172.21.15.52:0/2246710182 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f92ec0335a0 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92d37fe640 1 -- 172.21.15.52:0/2246710182 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f92e4044070 con 0x55f78d90f200 2024-06-02T23:29:20.600 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92fd1ea740 10 client.29459 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.597+0000 7f92fd1ea740 10 client.29459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92fd1ea740 20 client.29459 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d98f580 0x55f78d9919a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92fd1ea740 4 client.29459 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f78d87fc20 con 0x55f78d98f580 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d98f580 0x55f78d9919a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92ebfff640 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d98f580 0x55f78d9919a0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c82234c95d698e server_cookie=ff24a4ed0100cb5b in_seq=0 out_seq=0 2024-06-02T23:29:20.601 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.598+0000 7f92d37fe640 10 client.29459 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92d37fe640 1 -- 172.21.15.52:0/2246710182 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f78d87fc20 con 0x55f78d98f580 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92d37fe640 10 client.29459 handle_command_reply: tid=0 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 client.29459 shutdown 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 msgr2=0x55f78d920470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d920470 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f92e4031a70 tx=0x7f92e4009260 comp rx=0 tx=0).stop 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 shutdown_connections 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f78d98f580 0x55f78d9919a0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f78d914d60 0x55f78d91d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92d37fe640 0 client.29459 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:20.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f78d90fbf0 0x55f78d91cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 --2- 172.21.15.52:0/2246710182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f78d90f200 0x55f78d920470 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 >> 172.21.15.52:0/2246710182 conn(0x55f78d90a870 msgr2=0x55f78d919440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 shutdown_connections 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.644+0000 7f92fd1ea740 1 -- 172.21.15.52:0/2246710182 wait complete. 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92fd1ea740 20 client.29459 trim_cache size 0 max 16384 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:29:20.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92e00a2c00 msgr2=0x7f92e00a50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92e00a2c00 0x7f92e00a50c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f92e4000c00 tx=0x7f92e40058d0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f4100a40 msgr2=0x7f92f419a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f4100a40 0x7f92f419a1d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00e9f0 tx=0x7f92dc00edf0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 shutdown_connections 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f92e00a2c00 0x7f92e00a50c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f92f41013e0 0x7f92f41a1230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92f4100a40 0x7f92f419a1d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 --2- 172.21.15.52:0/2519033653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f92f41000e0 0x7f92f4199c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 >> 172.21.15.52:0/2519033653 conn(0x7f92f40fbce0 msgr2=0x7f92f40ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 shutdown_connections 2024-06-02T23:29:20.647 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:20.645+0000 7f92f91a0640 1 -- 172.21.15.52:0/2519033653 wait complete. 2024-06-02T23:29:21.418 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:21 smithi052 ceph-mon[32340]: pgmap v955: 97 pgs: 97 active+clean; 418 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 279 op/s 2024-06-02T23:29:21.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 32359 7.14 MB/sec execute 149 sec latency 75.203 ms 2024-06-02T23:29:21.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:21 smithi123 ceph-mon[35977]: pgmap v955: 97 pgs: 97 active+clean; 418 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 279 op/s 2024-06-02T23:29:21.646 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:21 smithi174 ceph-mon[35994]: pgmap v955: 97 pgs: 97 active+clean; 418 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.7 MiB/s wr, 279 op/s 2024-06-02T23:29:21.810 INFO:tasks.quiescer.fs.[cephfs]:Total duration (54.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:29:21.811 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.0 2024-06-02T23:29:21.811 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:29:21.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 -- 172.21.15.52:0/875967857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102ee0 msgr2=0x7ff7c810f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/875967857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810f3c0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7ff7c0009f90 tx=0x7ff7c000b330 comp rx=0 tx=0).stop 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 -- 172.21.15.52:0/875967857 shutdown_connections 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/875967857 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810f3c0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/875967857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8107450 0x7ff7c8107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/875967857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102540 0x7ff7c81029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 -- 172.21.15.52:0/875967857 >> 172.21.15.52:0/875967857 conn(0x7ff7c80795f0 msgr2=0x7ff7c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 -- 172.21.15.52:0/875967857 shutdown_connections 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.867+0000 7ff7cf25d700 1 -- 172.21.15.52:0/875967857 wait complete. 2024-06-02T23:29:21.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.868+0000 7ff7cf25d700 1 Processor -- start 2024-06-02T23:29:21.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.868+0000 7ff7cf25d700 1 -- start start 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cf25d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102540 0x7ff7c810e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cf25d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cf25d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8107450 0x7ff7c81a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cf25d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8112690 con 0x7ff7c8102540 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cf25d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8112510 con 0x7ff7c8102ee0 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cf25d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8112810 con 0x7ff7c8107450 2024-06-02T23:29:21.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102540 0x7ff7c810e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102540 0x7ff7c810e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39534/0 (socket says 172.21.15.52:39534) 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7ccff9700 1 -- 172.21.15.52:0/1228171490 learned_addr learned my addr 172.21.15.52:0/1228171490 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8107450 0x7ff7c81a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7c7fff700 1 -- 172.21.15.52:0/1228171490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8107450 msgr2=0x7ff7c81a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8107450 0x7ff7c81a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7c7fff700 1 -- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102540 msgr2=0x7ff7c810e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102540 0x7ff7c810e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.869+0000 7ff7c7fff700 1 -- 172.21.15.52:0/1228171490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c81a40e0 con 0x7ff7c8102ee0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.870+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810ec40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9f0 tx=0x7ff7b400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.870+0000 7ff7c5ffb700 1 -- 172.21.15.52:0/1228171490 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400c9c0 con 0x7ff7c8102ee0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.870+0000 7ff7c5ffb700 1 -- 172.21.15.52:0/1228171490 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7b400cb60 con 0x7ff7c8102ee0 2024-06-02T23:29:21.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.870+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c81a43d0 con 0x7ff7c8102ee0 2024-06-02T23:29:21.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.870+0000 7ff7c5ffb700 1 -- 172.21.15.52:0/1228171490 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4018680 con 0x7ff7c8102ee0 2024-06-02T23:29:21.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.870+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c81abc80 con 0x7ff7c8102ee0 2024-06-02T23:29:21.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.873+0000 7ff7c5ffb700 1 -- 172.21.15.52:0/1228171490 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff7b4010430 con 0x7ff7c8102ee0 2024-06-02T23:29:21.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7c5ffb700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7b80a2c20 0x7ff7b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7b80a2c20 0x7ff7b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7c5ffb700 1 -- 172.21.15.52:0/1228171490 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff7b40cf150 con 0x7ff7c8102ee0 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7d9ae0b80 1 Processor -- start 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7d9ae0b80 1 -- start start 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6748f60 0x5588d6749340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 0x5588d6743260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.875+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d67437a0 0x5588d6743bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7d9ae0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588d6635f60 con 0x5588d6748f60 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7d9ae0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5588d6694850 con 0x5588d6749880 2024-06-02T23:29:21.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7d9ae0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5588d65fc460 con 0x5588d67437a0 2024-06-02T23:29:21.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 0x5588d6743260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6748f60 0x5588d6749340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 0x5588d6743260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56406/0 (socket says 172.21.15.52:56406) 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7c7fff700 1 -- 172.21.15.52:0/3440292525 learned_addr learned my addr 172.21.15.52:0/3440292525 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d67437a0 0x5588d6743bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.876+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7b80a2c20 0x7ff7b80a50e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009750 tx=0x7ff7bc009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7ccff9700 1 -- 172.21.15.52:0/3440292525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d67437a0 msgr2=0x5588d6743bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d67437a0 0x5588d6743bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7ccff9700 1 -- 172.21.15.52:0/3440292525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 msgr2=0x5588d6743260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 0x5588d6743260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7ccff9700 1 -- 172.21.15.52:0/3440292525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d65e38d0 con 0x5588d6748f60 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 0x5588d6743260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:21.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6748f60 0x5588d6749340 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00e4b0 tx=0x7ff7bc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af90df2ce2174b49 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.877+0000 7ff7b2ffd700 1 -- 172.21.15.52:0/3440292525 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc00ee50 con 0x5588d6748f60 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.878+0000 7ff7b2ffd700 1 -- 172.21.15.52:0/3440292525 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7bc0044e0 con 0x5588d6748f60 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.878+0000 7ff7b2ffd700 1 -- 172.21.15.52:0/3440292525 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0266d0 con 0x5588d6748f60 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/3440292525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6748f60 msgr2=0x5588d6749340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6748f60 0x5588d6749340 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00e4b0 tx=0x7ff7bc014000 comp rx=0 tx=0).stop 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/3440292525 shutdown_connections 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d67437a0 0x5588d6743bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6748f60 0x5588d6749340 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/3440292525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6749880 0x5588d6743260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/3440292525 >> 172.21.15.52:0/3440292525 conn(0x5588d6670080 msgr2=0x5588d673e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/3440292525 shutdown_connections 2024-06-02T23:29:21.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/3440292525 wait complete. 2024-06-02T23:29:21.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 -- start start 2024-06-02T23:29:21.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.879+0000 7ff7d9ae0b80 1 Processor -- start 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d674f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d674ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d674cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5588d6634fc0 con 0x5588d6749880 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5588d6690810 con 0x5588d67437a0 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5588d6750260 con 0x5588d6748f60 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7d9ae0b80 10 client.0 _ll_register_callbacks cb 0x5588d662a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:21.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d674f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d674f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56414/0 (socket says 172.21.15.52:56414) 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7ccff9700 1 -- 172.21.15.52:0/1953588039 learned_addr learned my addr 172.21.15.52:0/1953588039 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d674ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d674cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.880+0000 7ff7c7fff700 1 -- 172.21.15.52:0/1953588039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 msgr2=0x5588d674f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d674f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7c7fff700 1 -- 172.21.15.52:0/1953588039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 msgr2=0x5588d674cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d674cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7c7fff700 1 -- 172.21.15.52:0/1953588039 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d665a750 con 0x5588d6748f60 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d674cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d674f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d674ca10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4098350 tx=0x7ff7b400cd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e33acc50c58ffbfc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:21.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7a7fff700 1 -- 172.21.15.52:0/1953588039 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40b4060 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7a7fff700 1 -- 172.21.15.52:0/1953588039 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff7b40a4360 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7bc0044e0 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7a7fff700 1 -- 172.21.15.52:0/1953588039 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff7b409b980 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7d9ae0b80 10 client.29107 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5588d675abd0 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7a7fff700 1 -- 172.21.15.52:0/1953588039 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7bc0044e0 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.881+0000 7ff7a7fff700 1 -- 172.21.15.52:0/1953588039 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff7b409ae10 con 0x5588d6748f60 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7d9ae0b80 10 client.29107 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7d9ae0b80 10 client.29107 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7d9ae0b80 20 client.29107 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d675d160 0x5588d675f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7d9ae0b80 4 client.29107 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588d66a1d50 con 0x5588d675d160 2024-06-02T23:29:21.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d675d160 0x5588d675f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.882+0000 7ff7a7fff700 10 client.29107 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:29:21.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.883+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d675d160 0x5588d675f550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=44d52ff7fd2d12e4 server_cookie=1916c41166e9fcf2 in_seq=0 out_seq=0 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.883+0000 7ff7a7fff700 1 -- 172.21.15.52:0/1953588039 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5588d66a1d50 con 0x5588d675d160 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.883+0000 7ff7a7fff700 10 client.29107 handle_command_reply: tid=0 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.883+0000 7ff7d9ae0b80 1 client.29107 shutdown 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 msgr2=0x5588d674ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d674ca10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4098350 tx=0x7ff7b400cd90 comp rx=0 tx=0).stop 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 shutdown_connections 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d674cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d674f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d675d160 0x5588d675f550 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/1953588039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d674ca10 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 >> 172.21.15.52:0/1953588039 conn(0x5588d6670080 msgr2=0x5588d673d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:21.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7a7fff700 0 client.29107 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:29:21.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 shutdown_connections 2024-06-02T23:29:21.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/1953588039 wait complete. 2024-06-02T23:29:21.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.884+0000 7ff7d9ae0b80 20 client.29107 trim_cache size 0 max 16384 2024-06-02T23:29:21.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.908+0000 7ff7d9ae0b80 1 Processor -- start 2024-06-02T23:29:21.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.908+0000 7ff7d9ae0b80 1 -- start start 2024-06-02T23:29:21.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.908+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 0x5588d6759400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6748f60 0x5588d674a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 0x5588d6759400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 0x5588d6759400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39562/0 (socket says 172.21.15.52:39562) 2024-06-02T23:29:21.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 -- 172.21.15.52:0/2290256160 learned_addr learned my addr 172.21.15.52:0/2290256160 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6749880 0x5588d674f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b3020 con 0x5588d67437a0 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b4b60 con 0x5588d6748f60 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b36f0 con 0x5588d6749880 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6749880 0x5588d674f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6748f60 0x5588d674a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 -- 172.21.15.52:0/2290256160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6749880 msgr2=0x5588d674f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6749880 0x5588d674f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 -- 172.21.15.52:0/2290256160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6748f60 msgr2=0x5588d674a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6748f60 0x5588d674a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7ccff9700 1 -- 172.21.15.52:0/2290256160 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d674af80 con 0x5588d67437a0 2024-06-02T23:29:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.909+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6749880 0x5588d674f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.910+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 0x5588d6759400 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc005b40 tx=0x7ff7bc0039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5856b170e5dd7ee0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.910+0000 7ff7b27fc700 1 -- 172.21.15.52:0/2290256160 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc005240 con 0x5588d67437a0 2024-06-02T23:29:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.910+0000 7ff7b27fc700 1 -- 172.21.15.52:0/2290256160 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7bc0042b0 con 0x5588d67437a0 2024-06-02T23:29:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.910+0000 7ff7b27fc700 1 -- 172.21.15.52:0/2290256160 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc004950 con 0x5588d67437a0 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 msgr2=0x5588d6759400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 0x5588d6759400 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc005b40 tx=0x7ff7bc0039b0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 shutdown_connections 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6749880 0x5588d674f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d67437a0 0x5588d6759400 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/2290256160 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d6748f60 0x5588d674a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 >> 172.21.15.52:0/2290256160 conn(0x5588d6670080 msgr2=0x5588d6749c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 shutdown_connections 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.911+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/2290256160 wait complete. 2024-06-02T23:29:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.912+0000 7ff7d9ae0b80 1 -- start start 2024-06-02T23:29:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.912+0000 7ff7d9ae0b80 1 Processor -- start 2024-06-02T23:29:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d6755570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d6755ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d6763da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d6755570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7ccff9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d6755570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56442/0 (socket says 172.21.15.52:56442) 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7cd7fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d6763da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7cd7fa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d6763da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39568/0 (socket says 172.21.15.52:39568) 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7c7fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d6755ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7ccff9700 1 -- 172.21.15.52:0/803805333 learned_addr learned my addr 172.21.15.52:0/803805333 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b4490 con 0x5588d6749880 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b3dc0 con 0x5588d67437a0 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c81b0fe0 con 0x5588d6748f60 2024-06-02T23:29:21.915 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.913+0000 7ff7d9ae0b80 10 client.0 _ll_register_callbacks cb 0x5588d6747570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7c7fff700 1 -- 172.21.15.52:0/803805333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 msgr2=0x5588d6755570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d6755570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7c7fff700 1 -- 172.21.15.52:0/803805333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 msgr2=0x5588d6763da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d6763da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7c7fff700 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d6765820 con 0x5588d6748f60 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d6755570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7cd7fa700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d6763da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7c7fff700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d6755ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409b910 tx=0x7ff7b40979d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7bb5e70457cddbf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.914+0000 7ff7a7fff700 1 -- 172.21.15.52:0/803805333 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b40ba370 con 0x5588d6748f60 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.915+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7bc0042b0 con 0x5588d6748f60 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.915+0000 7ff7a7fff700 1 -- 172.21.15.52:0/803805333 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff7b40a4160 con 0x5588d6748f60 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.915+0000 7ff7a7fff700 1 -- 172.21.15.52:0/803805333 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff7b409fd30 con 0x5588d6748f60 2024-06-02T23:29:21.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.915+0000 7ff7d9ae0b80 10 client.29113 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:21.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.915+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5588d67c7120 con 0x5588d6748f60 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7a7fff700 1 -- 172.21.15.52:0/803805333 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff7bc0042b0 con 0x5588d6748f60 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7a7fff700 1 -- 172.21.15.52:0/803805333 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff7b40b4b80 con 0x5588d6748f60 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7d9ae0b80 10 client.29113 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7d9ae0b80 10 client.29113 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7d9ae0b80 20 client.29113 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d67c7740 0x5588d67c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7d9ae0b80 4 client.29113 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.916+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588d669c2d0 con 0x5588d67c7740 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.917+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d67c7740 0x5588d67c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.917+0000 7ff7ccff9700 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d67c7740 0x5588d67c9b60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=26e041b5fe73cbc9 server_cookie=4bdb5ebaaee425d1 in_seq=0 out_seq=0 2024-06-02T23:29:21.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.917+0000 7ff7a7fff700 10 client.29113 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:29:21.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7a7fff700 1 -- 172.21.15.52:0/803805333 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588d669c2d0 con 0x5588d67c7740 2024-06-02T23:29:21.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7a7fff700 10 client.29113 handle_command_reply: tid=0 2024-06-02T23:29:21.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 client.29113 shutdown 2024-06-02T23:29:21.919 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 msgr2=0x5588d6755ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d6755ab0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff7b409b910 tx=0x7ff7b40979d0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 shutdown_connections 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5588d6749880 0x5588d6763da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5588d67437a0 0x5588d6755570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x5588d67c7740 0x5588d67c9b60 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7a7fff700 0 client.29113 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 --2- 172.21.15.52:0/803805333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5588d6748f60 0x5588d6755ab0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.919+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 >> 172.21.15.52:0/803805333 conn(0x5588d6670080 msgr2=0x5588d674a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.920+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 shutdown_connections 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.920+0000 7ff7d9ae0b80 1 -- 172.21.15.52:0/803805333 wait complete. 2024-06-02T23:29:21.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.920+0000 7ff7d9ae0b80 20 client.29113 trim_cache size 0 max 16384 2024-06-02T23:29:21.921 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:29:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.920+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7b80a2c20 msgr2=0x7ff7b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.920+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7b80a2c20 0x7ff7b80a50e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009750 tx=0x7ff7bc009310 comp rx=0 tx=0).stop 2024-06-02T23:29:21.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102ee0 msgr2=0x7ff7c810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810ec40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9f0 tx=0x7ff7b400ed00 comp rx=0 tx=0).stop 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 shutdown_connections 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff7b80a2c20 0x7ff7b80a50e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7c8107450 0x7ff7c81a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff7c8102540 0x7ff7c810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 --2- 172.21.15.52:0/1228171490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7c8102ee0 0x7ff7c810ec40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 >> 172.21.15.52:0/1228171490 conn(0x7ff7c80795f0 msgr2=0x7ff7c81003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:21.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 shutdown_connections 2024-06-02T23:29:21.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:21.921+0000 7ff7cf25d700 1 -- 172.21.15.52:0/1228171490 wait complete. 2024-06-02T23:29:22.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 32518 7.16 MB/sec execute 150 sec latency 92.906 ms 2024-06-02T23:29:22.632 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:29:22.689 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:23.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:23 smithi052 ceph-mon[32340]: pgmap v956: 97 pgs: 97 active+clean; 419 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 274 op/s 2024-06-02T23:29:23.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 32669 7.15 MB/sec execute 151 sec latency 72.562 ms 2024-06-02T23:29:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:23 smithi123 ceph-mon[35977]: pgmap v956: 97 pgs: 97 active+clean; 419 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 274 op/s 2024-06-02T23:29:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:23 smithi174 ceph-mon[35994]: pgmap v956: 97 pgs: 97 active+clean; 419 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.9 MiB/s wr, 274 op/s 2024-06-02T23:29:23.954 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:24.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 32828 7.16 MB/sec execute 152 sec latency 89.912 ms 2024-06-02T23:29:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:25 smithi052 ceph-mon[32340]: pgmap v957: 97 pgs: 97 active+clean; 418 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-06-02T23:29:25.460 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33004 7.15 MB/sec execute 153 sec latency 97.796 ms 2024-06-02T23:29:25.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:25 smithi123 ceph-mon[35977]: pgmap v957: 97 pgs: 97 active+clean; 418 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-06-02T23:29:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:25 smithi174 ceph-mon[35994]: pgmap v957: 97 pgs: 97 active+clean; 418 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 249 op/s 2024-06-02T23:29:26.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33150 7.18 MB/sec execute 154 sec latency 119.801 ms 2024-06-02T23:29:27.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:27 smithi052 ceph-mon[32340]: pgmap v958: 97 pgs: 97 active+clean; 416 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.3 MiB/s wr, 309 op/s 2024-06-02T23:29:27.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2920310657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 -- 172.21.15.52:0/2579186067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 msgr2=0x7fe54c10f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2579186067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c10f3e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fe54400b0d0 tx=0x7fe54400b3e0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 -- 172.21.15.52:0/2579186067 shutdown_connections 2024-06-02T23:29:27.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2579186067 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c10f3e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2579186067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe54c102770 0x7fe54c102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2579186067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe54c103120 0x7fe54c107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 -- 172.21.15.52:0/2579186067 >> 172.21.15.52:0/2579186067 conn(0x7fe54c079600 msgr2=0x7fe54c079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.587+0000 7fe550e5d700 1 -- 172.21.15.52:0/2579186067 shutdown_connections 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.588+0000 7fe550e5d700 1 -- 172.21.15.52:0/2579186067 wait complete. 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.588+0000 7fe550e5d700 1 Processor -- start 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 -- start start 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe54c102770 0x7fe54c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe54c103120 0x7fe54c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:27.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c112650 con 0x7fe54c107860 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1124d0 con 0x7fe54c103120 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe550e5d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe54c1127d0 con 0x7fe54c102770 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c198890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43934/0 (socket says 172.21.15.52:43934) 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 -- 172.21.15.52:0/2920310657 learned_addr learned my addr 172.21.15.52:0/2920310657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe543fff700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe54c103120 0x7fe54c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54a59c700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe54c102770 0x7fe54c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 -- 172.21.15.52:0/2920310657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe54c102770 msgr2=0x7fe54c19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe54c102770 0x7fe54c19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 -- 172.21.15.52:0/2920310657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe54c103120 msgr2=0x7fe54c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:27.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe54c103120 0x7fe54c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 -- 172.21.15.52:0/2920310657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54c198ff0 con 0x7fe54c107860 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.589+0000 7fe54ad9d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c198890 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fe5440095c0 tx=0x7fe544007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.590+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54401f070 con 0x7fe54c107860 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.590+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe544007e20 con 0x7fe54c107860 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.590+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54c199280 con 0x7fe54c107860 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.590+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54401b4a0 con 0x7fe54c107860 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.590+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe54c1a55a0 con 0x7fe54c107860 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:29:27.404 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.591+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe54c103bf0 con 0x7fe54c107860 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.592+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe544022070 con 0x7fe54c107860 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.593+0000 7fe5437fe700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5380a2c20 0x7fe5380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.593+0000 7fe54a59c700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5380a2c20 0x7fe5380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.593+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe54400f020 con 0x7fe54c107860 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.593+0000 7fe54a59c700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5380a2c20 0x7fe5380a50e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe54c1999d0 tx=0x7fe53c00b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:26.596+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fe544029020 con 0x7fe54c107860 2024-06-02T23:29:27.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.023+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe54c02dca0 con 0x7fe54c107860 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.024+0000 7fe5437fe700 1 -- 172.21.15.52:0/2920310657 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7fe54409d580 con 0x7fe54c107860 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5380a2c20 msgr2=0x7fe5380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5380a2c20 0x7fe5380a50e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe54c1999d0 tx=0x7fe53c00b3c0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 msgr2=0x7fe54c198890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c198890 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fe5440095c0 tx=0x7fe544007a20 comp rx=0 tx=0).stop 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 shutdown_connections 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe5380a2c20 0x7fe5380a50e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe54c107860 0x7fe54c198890 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe54c102770 0x7fe54c19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 --2- 172.21.15.52:0/2920310657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe54c103120 0x7fe54c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 >> 172.21.15.52:0/2920310657 conn(0x7fe54c079600 msgr2=0x7fe54c1005f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 shutdown_connections 2024-06-02T23:29:27.406 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:27.028+0000 7fe550e5d700 1 -- 172.21.15.52:0/2920310657 wait complete. 2024-06-02T23:29:27.407 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:29:27.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33287 7.17 MB/sec execute 155 sec latency 105.074 ms 2024-06-02T23:29:27.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:27 smithi123 ceph-mon[35977]: pgmap v958: 97 pgs: 97 active+clean; 416 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.3 MiB/s wr, 309 op/s 2024-06-02T23:29:27.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2920310657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:27 smithi174 ceph-mon[35994]: pgmap v958: 97 pgs: 97 active+clean; 416 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.3 MiB/s wr, 309 op/s 2024-06-02T23:29:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2920310657' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:28.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33416 7.15 MB/sec execute 156 sec latency 126.401 ms 2024-06-02T23:29:29.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33550 7.12 MB/sec execute 157 sec latency 138.943 ms 2024-06-02T23:29:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:29 smithi123 ceph-mon[35977]: pgmap v959: 97 pgs: 97 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2024-06-02T23:29:29.513 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:29 smithi052 ceph-mon[32340]: pgmap v959: 97 pgs: 97 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2024-06-02T23:29:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:29 smithi174 ceph-mon[35994]: pgmap v959: 97 pgs: 97 active+clean; 415 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2024-06-02T23:29:30.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33702 7.17 MB/sec execute 158 sec latency 177.348 ms 2024-06-02T23:29:31.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 33837 7.15 MB/sec execute 159 sec latency 141.780 ms 2024-06-02T23:29:31.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:31 smithi123 ceph-mon[35977]: pgmap v960: 97 pgs: 97 active+clean; 414 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-06-02T23:29:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:31 smithi174 ceph-mon[35994]: pgmap v960: 97 pgs: 97 active+clean; 414 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-06-02T23:29:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:31 smithi052 ceph-mon[32340]: pgmap v960: 97 pgs: 97 active+clean; 414 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.0 MiB/s wr, 303 op/s 2024-06-02T23:29:32.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34026 7.16 MB/sec execute 160 sec latency 139.444 ms 2024-06-02T23:29:33.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34134 7.13 MB/sec execute 161 sec latency 143.568 ms 2024-06-02T23:29:33.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:33 smithi123 ceph-mon[35977]: pgmap v961: 97 pgs: 97 active+clean; 415 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 287 op/s 2024-06-02T23:29:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:33 smithi174 ceph-mon[35994]: pgmap v961: 97 pgs: 97 active+clean; 415 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 287 op/s 2024-06-02T23:29:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:33 smithi052 ceph-mon[32340]: pgmap v961: 97 pgs: 97 active+clean; 415 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.3 MiB/s wr, 287 op/s 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 -- 172.21.15.52:0/2526666158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c101a20 msgr2=0x7fb86c101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 --2- 172.21.15.52:0/2526666158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c101a20 0x7fb86c101e40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009900 tx=0x7fb85c02a410 comp rx=0 tx=0).stop 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 -- 172.21.15.52:0/2526666158 shutdown_connections 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 --2- 172.21.15.52:0/2526666158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c1035d0 0x7fb86c109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 --2- 172.21.15.52:0/2526666158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 --2- 172.21.15.52:0/2526666158 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c101a20 0x7fb86c101e40 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 -- 172.21.15.52:0/2526666158 >> 172.21.15.52:0/2526666158 conn(0x7fb86c0fcf90 msgr2=0x7fb86c0ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 -- 172.21.15.52:0/2526666158 shutdown_connections 2024-06-02T23:29:33.774 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.773+0000 7fb8721c8640 1 -- 172.21.15.52:0/2526666158 wait complete. 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.774+0000 7fb8721c8640 1 Processor -- start 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.774+0000 7fb8721c8640 1 -- start start 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.774+0000 7fb8721c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101a20 0x7fb86c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.774+0000 7fb8721c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c070870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb8721c8640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c1035d0 0x7fb86c1a7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c070870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c070870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41306/0 (socket says 172.21.15.52:41306) 2024-06-02T23:29:33.775 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101a20 0x7fb86c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101a20 0x7fb86c070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35792/0 (socket says 172.21.15.52:35792) 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c1035d0 0x7fb86c1a7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c1035d0 0x7fb86c1a7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43946/0 (socket says 172.21.15.52:43946) 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10d570 con 0x7fb86c1035d0 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 -- 172.21.15.52:0/3948843419 learned_addr learned my addr 172.21.15.52:0/3948843419 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10d3f0 con 0x7fb86c102c10 2024-06-02T23:29:33.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c10d6f0 con 0x7fb86c101a20 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 -- 172.21.15.52:0/3948843419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101a20 msgr2=0x7fb86c070330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101a20 0x7fb86c070330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 -- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c1035d0 msgr2=0x7fb86c1a7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c1035d0 0x7fb86c1a7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.775+0000 7fb86affd640 1 -- 172.21.15.52:0/3948843419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86c073b70 con 0x7fb86c102c10 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.776+0000 7fb86affd640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c070870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e9f0 tx=0x7fb85400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.776+0000 7fb868ff9640 1 -- 172.21.15.52:0/3948843419 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85400cd80 con 0x7fb86c102c10 2024-06-02T23:29:33.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.776+0000 7fb868ff9640 1 -- 172.21.15.52:0/3948843419 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8540108e0 con 0x7fb86c102c10 2024-06-02T23:29:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.776+0000 7fb868ff9640 1 -- 172.21.15.52:0/3948843419 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8540188e0 con 0x7fb86c102c10 2024-06-02T23:29:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.776+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86c073e60 con 0x7fb86c102c10 2024-06-02T23:29:33.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.776+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb86c074400 con 0x7fb86c102c10 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb868ff9640 1 -- 172.21.15.52:0/3948843419 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb854010a90 con 0x7fb86c102c10 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 Processor -- start 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 -- start start 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f390500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f393000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55988f2ee4f0 con 0x55988f38ba00 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55988f28c870 con 0x55988f38b010 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.779+0000 7fb874fb4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55988f2ea4b0 con 0x55988f390b70 2024-06-02T23:29:33.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41310/0 (socket says 172.21.15.52:41310) 2024-06-02T23:29:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 -- 172.21.15.52:0/3109411323 learned_addr learned my addr 172.21.15.52:0/3109411323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f390500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86affd640 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f393000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 -- 172.21.15.52:0/3109411323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 msgr2=0x55988f393000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f393000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 -- 172.21.15.52:0/3109411323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 msgr2=0x55988f390500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f390500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 -- 172.21.15.52:0/3109411323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988f2a89b0 con 0x55988f38b010 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb868ff9640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8580a2cb0 0x7fb8580a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8580a2cb0 0x7fb8580a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38b430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb864009fd0 tx=0x7fb8640118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14cf2407835097aa server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:33.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.780+0000 7fb868ff9640 1 -- 172.21.15.52:0/3948843419 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb8540cf440 con 0x7fb86c102c10 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb861ffb640 1 -- 172.21.15.52:0/3109411323 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864004d20 con 0x55988f38b010 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb861ffb640 1 -- 172.21.15.52:0/3109411323 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb864004050 con 0x55988f38b010 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb861ffb640 1 -- 172.21.15.52:0/3109411323 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864005280 con 0x55988f38b010 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 -- 172.21.15.52:0/3109411323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 msgr2=0x55988f38b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8580a2cb0 0x7fb8580a5170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005ab0 tx=0x7fb85c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38b430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb864009fd0 tx=0x7fb8640118d0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 -- 172.21.15.52:0/3109411323 shutdown_connections 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f393000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f390500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3109411323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38b430 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.781+0000 7fb874fb4740 1 -- 172.21.15.52:0/3109411323 >> 172.21.15.52:0/3109411323 conn(0x55988f386680 msgr2=0x55988f386a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 -- 172.21.15.52:0/3109411323 shutdown_connections 2024-06-02T23:29:33.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 -- 172.21.15.52:0/3109411323 wait complete. 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 -- start start 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 Processor -- start 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f3952c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f38ba00 0x55988f395800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.782+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 0x55988f39c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb874fb4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55988f393b20 con 0x55988f390b70 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb874fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55988f277ae0 con 0x55988f38b010 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb874fb4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55988f393ca0 con 0x55988f38ba00 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 0x55988f39c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 0x55988f39c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43962/0 (socket says 172.21.15.52:43962) 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 -- 172.21.15.52:0/3189089808 learned_addr learned my addr 172.21.15.52:0/3189089808 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb874fb4740 10 client.0 _ll_register_callbacks cb 0x55988f2b09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86affd640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f38ba00 0x55988f395800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f3952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 -- 172.21.15.52:0/3189089808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f38ba00 msgr2=0x55988f395800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f38ba00 0x55988f395800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 -- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 msgr2=0x55988f3952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f3952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 -- 172.21.15.52:0/3189089808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988f002210 con 0x55988f390b70 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f3952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86affd640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f38ba00 0x55988f395800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 0x55988f39c840 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb86401b040 tx=0x7fb864004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7127056bad37511 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.783+0000 7fb84affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb84affd640 1 -- 172.21.15.52:0/3189089808 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb864028020 con 0x55988f390b70 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb84affd640 1 -- 172.21.15.52:0/3189089808 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb864004410 con 0x55988f390b70 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb864004050 con 0x55988f390b70 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb84affd640 1 -- 172.21.15.52:0/3189089808 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb864009c50 con 0x55988f390b70 2024-06-02T23:29:33.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 10 client.20151 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55988f0ce620 con 0x55988f390b70 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb84affd640 1 -- 172.21.15.52:0/3189089808 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb864004050 con 0x55988f390b70 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb84affd640 1 -- 172.21.15.52:0/3189089808 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb86400f4e0 con 0x55988f390b70 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 10 client.20151 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 10 client.20151 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 20 client.20151 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f3a0ee0 0x55988f3a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 4 client.20151 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.784+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55988f301f10 con 0x55988f3a0ee0 2024-06-02T23:29:33.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.785+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f3a0ee0 0x55988f3a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.785+0000 7fb84affd640 10 client.20151 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.785+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f3a0ee0 0x55988f3a32d0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a4cd68e2c02b663 server_cookie=ff05da2bb6596c72 in_seq=0 out_seq=0 2024-06-02T23:29:33.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb84affd640 1 -- 172.21.15.52:0/3189089808 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55988f301f10 con 0x55988f3a0ee0 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb84affd640 10 client.20151 handle_command_reply: tid=0 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 client.20151 shutdown 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 msgr2=0x55988f39c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 0x55988f39c840 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb86401b040 tx=0x7fb864004c70 comp rx=0 tx=0).stop 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 shutdown_connections 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f3a0ee0 0x55988f3a32d0 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f38ba00 0x55988f395800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f390b70 0x55988f39c840 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb84affd640 0 client.20151 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3189089808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f3952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.786+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 >> 172.21.15.52:0/3189089808 conn(0x55988f386680 msgr2=0x55988f3857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.787+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 shutdown_connections 2024-06-02T23:29:33.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.787+0000 7fb874fb4740 1 -- 172.21.15.52:0/3189089808 wait complete. 2024-06-02T23:29:33.792 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.787+0000 7fb874fb4740 20 client.20151 trim_cache size 0 max 16384 2024-06-02T23:29:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 Processor -- start 2024-06-02T23:29:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 -- start start 2024-06-02T23:29:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f39c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.818 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f38d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b3790 con 0x55988f38ba00 2024-06-02T23:29:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b52d0 con 0x55988f38b010 2024-06-02T23:29:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb874fb4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b3e60 con 0x55988f390b70 2024-06-02T23:29:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb86affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f39c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f38d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.819 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb86affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f39c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57292/0 (socket says 172.21.15.52:57292) 2024-06-02T23:29:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb86bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f38d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34482/0 (socket says 172.21.15.52:34482) 2024-06-02T23:29:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb86affd640 1 -- 172.21.15.52:0/1322679608 learned_addr learned my addr 172.21.15.52:0/1322679608 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.810+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86affd640 1 -- 172.21.15.52:0/1322679608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 msgr2=0x55988f38d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86affd640 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f38d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86affd640 1 -- 172.21.15.52:0/1322679608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 msgr2=0x55988f38cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86affd640 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86affd640 1 -- 172.21.15.52:0/1322679608 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55988f38ff70 con 0x55988f38ba00 2024-06-02T23:29:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86bfff640 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f38d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb86affd640 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f39c800 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb854098650 tx=0x7fb8540d5160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b18c2dc20f6ab8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:33.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb8617fa640 1 -- 172.21.15.52:0/1322679608 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85409b810 con 0x55988f38ba00 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb8617fa640 1 -- 172.21.15.52:0/1322679608 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb854010de0 con 0x55988f38ba00 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.811+0000 7fb8617fa640 1 -- 172.21.15.52:0/1322679608 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85409b630 con 0x55988f38ba00 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 -- 172.21.15.52:0/1322679608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 msgr2=0x55988f39c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f39c800 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fb854098650 tx=0x7fb8540d5160 comp rx=0 tx=0).stop 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 -- 172.21.15.52:0/1322679608 shutdown_connections 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f38d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38ba00 0x55988f39c800 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 --2- 172.21.15.52:0/1322679608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38b010 0x55988f38cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 -- 172.21.15.52:0/1322679608 >> 172.21.15.52:0/1322679608 conn(0x55988f386680 msgr2=0x55988f3902f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 -- 172.21.15.52:0/1322679608 shutdown_connections 2024-06-02T23:29:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.812+0000 7fb874fb4740 1 -- 172.21.15.52:0/1322679608 wait complete. 2024-06-02T23:29:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.813+0000 7fb874fb4740 1 -- start start 2024-06-02T23:29:33.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.813+0000 7fb874fb4740 1 Processor -- start 2024-06-02T23:29:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.813+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 0x55988f399590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.813+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38ba00 0x55988f399ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.813+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 0x55988f399590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb874fb4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f409b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 0x55988f399590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57306/0 (socket says 172.21.15.52:57306) 2024-06-02T23:29:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb874fb4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b4c00 con 0x55988f38b010 2024-06-02T23:29:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b4530 con 0x55988f38ba00 2024-06-02T23:29:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 -- 172.21.15.52:0/3929912947 learned_addr learned my addr 172.21.15.52:0/3929912947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb86c1b1770 con 0x55988f390b70 2024-06-02T23:29:33.825 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb874fb4740 10 client.0 _ll_register_callbacks cb 0x7fb8640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86bfff640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f409b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86affd640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38ba00 0x55988f399ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 -- 172.21.15.52:0/3929912947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 msgr2=0x55988f409b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.826 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f409b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 -- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38ba00 msgr2=0x55988f399ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38ba00 0x55988f399ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 -- 172.21.15.52:0/3929912947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55988f40a0c0 con 0x55988f38b010 2024-06-02T23:29:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb86b7fe640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 0x55988f399590 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0314e0 tx=0x7fb85c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1237c223cc76debf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.814+0000 7fb84affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb84affd640 1 -- 172.21.15.52:0/3929912947 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c056060 con 0x55988f38b010 2024-06-02T23:29:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb84affd640 1 -- 172.21.15.52:0/3929912947 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb85c0035f0 con 0x55988f38b010 2024-06-02T23:29:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55988f2fbc20 con 0x55988f38b010 2024-06-02T23:29:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb84affd640 1 -- 172.21.15.52:0/3929912947 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb85c035040 con 0x55988f38b010 2024-06-02T23:29:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb874fb4740 10 client.20157 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55988f40b2f0 con 0x55988f38b010 2024-06-02T23:29:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb84affd640 1 -- 172.21.15.52:0/3929912947 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55988f2fbc20 con 0x55988f38b010 2024-06-02T23:29:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.815+0000 7fb84affd640 1 -- 172.21.15.52:0/3929912947 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb85c044040 con 0x55988f38b010 2024-06-02T23:29:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb874fb4740 10 client.20157 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb874fb4740 10 client.20157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb874fb4740 20 client.20157 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:33.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f40bbe0 0x55988f40e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb874fb4740 4 client.20157 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:29:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55988f2f5f70 con 0x55988f40bbe0 2024-06-02T23:29:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb86affd640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f40bbe0 0x55988f40e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb84affd640 10 client.20157 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:33.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:33.816+0000 7fb86affd640 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f40bbe0 0x55988f40e000 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e751a9cda782fd50 server_cookie=24a40c5909bdc59c in_seq=0 out_seq=0 2024-06-02T23:29:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.009+0000 7fb84affd640 1 -- 172.21.15.52:0/3929912947 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55988f2f5f70 con 0x55988f40bbe0 2024-06-02T23:29:34.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.009+0000 7fb84affd640 10 client.20157 handle_command_reply: tid=0 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.009+0000 7fb874fb4740 1 client.20157 shutdown 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.009+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 msgr2=0x55988f399590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.009+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 0x55988f399590 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0314e0 tx=0x7fb85c009260 comp rx=0 tx=0).stop 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 shutdown_connections 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55988f40bbe0 0x55988f40e000 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55988f390b70 0x55988f409b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55988f38ba00 0x55988f399ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb84affd640 0 client.20157 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 --2- 172.21.15.52:0/3929912947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55988f38b010 0x55988f399590 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 >> 172.21.15.52:0/3929912947 conn(0x55988f386680 msgr2=0x55988f38cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 shutdown_connections 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 1 -- 172.21.15.52:0/3929912947 wait complete. 2024-06-02T23:29:34.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.010+0000 7fb874fb4740 20 client.20157 trim_cache size 0 max 16384 2024-06-02T23:29:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:29:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:29:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:29:34.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8580a2cb0 msgr2=0x7fb8580a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8580a2cb0 0x7fb8580a5170 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005ab0 tx=0x7fb85c005a20 comp rx=0 tx=0).stop 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 msgr2=0x7fb86c070870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c070870 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fb85400e9f0 tx=0x7fb85400eeb0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 shutdown_connections 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8580a2cb0 0x7fb8580a5170 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb86c1035d0 0x7fb86c1a7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb86c102c10 0x7fb86c070870 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 --2- 172.21.15.52:0/3948843419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb86c101a20 0x7fb86c070330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 >> 172.21.15.52:0/3948843419 conn(0x7fb86c0fcf90 msgr2=0x7fb86c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 shutdown_connections 2024-06-02T23:29:34.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:34.011+0000 7fb8721c8640 1 -- 172.21.15.52:0/3948843419 wait complete. 2024-06-02T23:29:34.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34264 7.13 MB/sec execute 162 sec latency 88.428 ms 2024-06-02T23:29:34.510 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 147.0 --await-for 45.0' 2024-06-02T23:29:34.511 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 147.0 --await-for 45.0 2024-06-02T23:29:35.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34471 7.21 MB/sec execute 163 sec latency 100.051 ms 2024-06-02T23:29:35.470 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:35 smithi052 ceph-mon[32340]: pgmap v962: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.2 MiB/s wr, 253 op/s 2024-06-02T23:29:35.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:35 smithi123 ceph-mon[35977]: pgmap v962: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.2 MiB/s wr, 253 op/s 2024-06-02T23:29:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:35 smithi174 ceph-mon[35994]: pgmap v962: 97 pgs: 97 active+clean; 412 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.2 MiB/s wr, 253 op/s 2024-06-02T23:29:36.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34683 7.24 MB/sec execute 164 sec latency 69.774 ms 2024-06-02T23:29:37.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34819 7.24 MB/sec execute 165 sec latency 201.797 ms 2024-06-02T23:29:37.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:37 smithi123 ceph-mon[35977]: pgmap v963: 97 pgs: 97 active+clean; 417 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-06-02T23:29:37.582 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:37 smithi052 ceph-mon[32340]: pgmap v963: 97 pgs: 97 active+clean; 417 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-06-02T23:29:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:37 smithi174 ceph-mon[35994]: pgmap v963: 97 pgs: 97 active+clean; 417 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 302 op/s 2024-06-02T23:29:38.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 34916 7.21 MB/sec execute 166 sec latency 160.658 ms 2024-06-02T23:29:38.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 -- 172.21.15.52:0/2514953796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef2806ab30 msgr2=0x7fef2810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/2514953796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef2806ab30 0x7fef2810f320 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fef20009f90 tx=0x7fef2000b330 comp rx=0 tx=0).stop 2024-06-02T23:29:38.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 -- 172.21.15.52:0/2514953796 shutdown_connections 2024-06-02T23:29:38.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/2514953796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef2806ab30 0x7fef2810f320 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/2514953796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef28108500 0x7fef281088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/2514953796 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef2806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 -- 172.21.15.52:0/2514953796 >> 172.21.15.52:0/2514953796 conn(0x7fef28078340 msgr2=0x7fef28078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 -- 172.21.15.52:0/2514953796 shutdown_connections 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.754+0000 7fef2d1f8700 1 -- 172.21.15.52:0/2514953796 wait complete. 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.755+0000 7fef2d1f8700 1 Processor -- start 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.755+0000 7fef2d1f8700 1 -- start start 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.755+0000 7fef2d1f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef28101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.755+0000 7fef2d1f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef2806ab30 0x7fef281019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2d1f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef28108500 0x7fef28101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2d1f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112610 con 0x7fef2806a170 2024-06-02T23:29:38.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2d1f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112490 con 0x7fef2806ab30 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2d1f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef28112790 con 0x7fef28108500 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef28101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef2806ab30 0x7fef281019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef28101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57326/0 (socket says 172.21.15.52:57326) 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 -- 172.21.15.52:0/3766100834 learned_addr learned my addr 172.21.15.52:0/3766100834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2759e700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef28108500 0x7fef28101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 -- 172.21.15.52:0/3766100834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef28108500 msgr2=0x7fef28101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef28108500 0x7fef28101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 -- 172.21.15.52:0/3766100834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef2806ab30 msgr2=0x7fef281019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef2806ab30 0x7fef281019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 -- 172.21.15.52:0/3766100834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef281a6ff0 con 0x7fef2806a170 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2659c700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef2806ab30 0x7fef281019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:38.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef2759e700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef28108500 0x7fef28101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef28101460 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fef1800e9a0 tx=0x7fef1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.756+0000 7fef0ffff700 1 -- 172.21.15.52:0/3766100834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1800ca10 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.757+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef281a72e0 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.757+0000 7fef0ffff700 1 -- 172.21.15.52:0/3766100834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef1800cbb0 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.757+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef281a7870 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.757+0000 7fef0ffff700 1 -- 172.21.15.52:0/3766100834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef18018680 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.760+0000 7fef0ffff700 1 -- 172.21.15.52:0/3766100834 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fef18018820 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.761+0000 7fef0ffff700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef100a2ca0 0x7fef100a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.761+0000 7fef2659c700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef100a2ca0 0x7fef100a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.761+0000 7fef0ffff700 1 -- 172.21.15.52:0/3766100834 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fef180cf190 con 0x7fef2806a170 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 Processor -- start 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 -- start start 2024-06-02T23:29:38.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c63340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c5d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef2659c700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef100a2ca0 0x7fef100a5160 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009750 tx=0x7fef1c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5638f2b4ff60 con 0x5638f2c62f60 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5638f2bae850 con 0x5638f2c5d7a0 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638f2b16460 con 0x5638f2c63880 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c63340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef2659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c5d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef2659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c5d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34514/0 (socket says 172.21.15.52:34514) 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef2659c700 1 -- 172.21.15.52:0/2823902369 learned_addr learned my addr 172.21.15.52:0/2823902369 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c63340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57340/0 (socket says 172.21.15.52:57340) 2024-06-02T23:29:38.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.762+0000 7fef2759e700 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef2659c700 1 -- 172.21.15.52:0/2823902369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 msgr2=0x5638f2c5dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef2659c700 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef2659c700 1 -- 172.21.15.52:0/2823902369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 msgr2=0x5638f2c63340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef2659c700 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c63340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef2659c700 1 -- 172.21.15.52:0/2823902369 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f2afd8d0 con 0x5638f2c63880 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef2659c700 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c5d260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef1c004370 tx=0x7fef1c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f4b74ee08e3322 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef0d7fa700 1 -- 172.21.15.52:0/2823902369 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c003d10 con 0x5638f2c63880 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef0d7fa700 1 -- 172.21.15.52:0/2823902369 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef1c01e8d0 con 0x5638f2c63880 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.763+0000 7fef0d7fa700 1 -- 172.21.15.52:0/2823902369 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1c026860 con 0x5638f2c63880 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2823902369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 msgr2=0x5638f2c5d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c5d260 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fef1c004370 tx=0x7fef1c014000 comp rx=0 tx=0).stop 2024-06-02T23:29:38.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2823902369 shutdown_connections 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c63340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2823902369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c5d260 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2823902369 >> 172.21.15.52:0/2823902369 conn(0x5638f2b8a080 msgr2=0x5638f2c58220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2823902369 shutdown_connections 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.764+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2823902369 wait complete. 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 -- start start 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 Processor -- start 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 0x5638f2c69600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c66950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c63880 0x5638f2c66e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5638f2b4efc0 con 0x5638f2c62f60 2024-06-02T23:29:38.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5638f2baa810 con 0x5638f2c63880 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5638f2c6a260 con 0x5638f2c5d7a0 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef37a7bb80 10 client.0 _ll_register_callbacks cb 0x5638f2b445b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.765+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 0x5638f2c69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 0x5638f2c69600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34522/0 (socket says 172.21.15.52:34522) 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 -- 172.21.15.52:0/1856181727 learned_addr learned my addr 172.21.15.52:0/1856181727 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef2659c700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c66950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef2759e700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c63880 0x5638f2c66e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 -- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c63880 msgr2=0x5638f2c66e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c63880 0x5638f2c66e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 -- 172.21.15.52:0/1856181727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 msgr2=0x5638f2c66950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c66950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 -- 172.21.15.52:0/1856181727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f2b74750 con 0x5638f2c5d7a0 2024-06-02T23:29:38.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef2659c700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef26d9d700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 0x5638f2c69600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fef18097e80 tx=0x7fef1809c1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2340e71ce7070b32 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef0aff5700 1 -- 172.21.15.52:0/1856181727 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1809fdd0 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef1c01e8d0 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef0aff5700 1 -- 172.21.15.52:0/1856181727 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fef1800cd60 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef0aff5700 1 -- 172.21.15.52:0/1856181727 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fef180a4690 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.766+0000 7fef37a7bb80 10 client.29149 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5638f2c74870 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef0aff5700 1 -- 172.21.15.52:0/1856181727 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef1c01e8d0 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef0aff5700 1 -- 172.21.15.52:0/1856181727 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fef18097770 con 0x5638f2c5d7a0 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 10 client.29149 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 10 client.29149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 20 client.29149 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2c771c0 0x5638f2c79620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 4 client.29149 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638f2bbbd50 con 0x5638f2c771c0 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.767+0000 7fef2659c700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2c771c0 0x5638f2c79620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.768+0000 7fef2659c700 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2c771c0 0x5638f2c79620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de14f959edac964e server_cookie=6c79f6ea449cd527 in_seq=0 out_seq=0 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.768+0000 7fef0aff5700 10 client.29149 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.768+0000 7fef0aff5700 1 -- 172.21.15.52:0/1856181727 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5638f2bbbd50 con 0x5638f2c771c0 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.768+0000 7fef0aff5700 10 client.29149 handle_command_reply: tid=0 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.768+0000 7fef37a7bb80 1 client.29149 shutdown 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 msgr2=0x5638f2c69600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 0x5638f2c69600 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fef18097e80 tx=0x7fef1809c1d0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 shutdown_connections 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2c771c0 0x5638f2c79620 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c63880 0x5638f2c66e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c5d7a0 0x5638f2c69600 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/1856181727 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c66950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 >> 172.21.15.52:0/1856181727 conn(0x5638f2b8a080 msgr2=0x5638f2c57700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef0aff5700 0 client.29149 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 shutdown_connections 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 1 -- 172.21.15.52:0/1856181727 wait complete. 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.769+0000 7fef37a7bb80 20 client.29149 trim_cache size 0 max 16384 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.796+0000 7fef37a7bb80 1 Processor -- start 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 -- start start 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c645b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c658d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fef281aa9d0 con 0x5638f2c62f60 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef281ac510 con 0x5638f2c5d7a0 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef281ab0a0 con 0x5638f2c63880 2024-06-02T23:29:38.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2759e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c658d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c645b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57360/0 (socket says 172.21.15.52:57360) 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2659c700 1 -- 172.21.15.52:0/3324923977 learned_addr learned my addr 172.21.15.52:0/3324923977 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2759e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c658d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34532/0 (socket says 172.21.15.52:34532) 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef26d9d700 1 -- 172.21.15.52:0/3324923977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 msgr2=0x5638f2c658d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c658d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef26d9d700 1 -- 172.21.15.52:0/3324923977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 msgr2=0x5638f2c645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef26d9d700 1 -- 172.21.15.52:0/3324923977 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638f2c64ee0 con 0x5638f2c5d7a0 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2759e700 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.797+0000 7fef2659c700 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.798+0000 7fef26d9d700 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5d260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fef18009230 tx=0x7fef1809bb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5155741a3d5bc4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:38.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.798+0000 7fef0cff9700 1 -- 172.21.15.52:0/3324923977 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180ba170 con 0x5638f2c5d7a0 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.798+0000 7fef0cff9700 1 -- 172.21.15.52:0/3324923977 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef180ce070 con 0x5638f2c5d7a0 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.798+0000 7fef0cff9700 1 -- 172.21.15.52:0/3324923977 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef1809ab90 con 0x5638f2c5d7a0 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 -- 172.21.15.52:0/3324923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 msgr2=0x5638f2c5d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5d260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fef18009230 tx=0x7fef1809bb60 comp rx=0 tx=0).stop 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 -- 172.21.15.52:0/3324923977 shutdown_connections 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c63880 0x5638f2c658d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c5d260 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/3324923977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c62f60 0x5638f2c645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 -- 172.21.15.52:0/3324923977 >> 172.21.15.52:0/3324923977 conn(0x5638f2b8a080 msgr2=0x5638f2c68130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 -- 172.21.15.52:0/3324923977 shutdown_connections 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.799+0000 7fef37a7bb80 1 -- 172.21.15.52:0/3324923977 wait complete. 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.800+0000 7fef37a7bb80 1 -- start start 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.800+0000 7fef37a7bb80 1 Processor -- start 2024-06-02T23:29:38.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.800+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c6f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.800+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c62f60 0x5638f2c6fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.800+0000 7fef37a7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c63880 0x5638f2c7e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fef281abe40 con 0x5638f2c63880 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef281ab770 con 0x5638f2c5d7a0 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef37a7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef2819c7a0 con 0x5638f2c62f60 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef37a7bb80 10 client.0 _ll_register_callbacks cb 0x5638f2b445b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c6f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c6f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41026/0 (socket says 172.21.15.52:41026) 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 -- 172.21.15.52:0/2870330379 learned_addr learned my addr 172.21.15.52:0/2870330379 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef2659c700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c62f60 0x5638f2c6fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef2759e700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c63880 0x5638f2c7e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 -- 172.21.15.52:0/2870330379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c62f60 msgr2=0x5638f2c6fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c62f60 0x5638f2c6fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 -- 172.21.15.52:0/2870330379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c63880 msgr2=0x5638f2c7e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c63880 0x5638f2c7e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 -- 172.21.15.52:0/2870330379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638f2c7ed40 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef2659c700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c62f60 0x5638f2c6fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef26d9d700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c6f7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef1809b720 tx=0x7fef180a4430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf935c0468b24c36 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef0aff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef0aff5700 1 -- 172.21.15.52:0/2870330379 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef180ba3a0 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef0aff5700 1 -- 172.21.15.52:0/2870330379 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fef1809a930 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.801+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef180ce070 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef0aff5700 1 -- 172.21.15.52:0/2870330379 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fef1809ab70 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 10 client.29516 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5638f2ce1a40 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef0aff5700 1 -- 172.21.15.52:0/2870330379 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fef180ce070 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef0aff5700 1 -- 172.21.15.52:0/2870330379 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fef180a1160 con 0x5638f2c5d7a0 2024-06-02T23:29:38.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 10 client.29516 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 10 client.29516 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 20 client.29516 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2ce1e90 0x5638f2ce4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 4 client.29516 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5638f2bb62d0 con 0x5638f2ce1e90 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.802+0000 7fef2659c700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2ce1e90 0x5638f2ce4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef0aff5700 10 client.29516 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef2659c700 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2ce1e90 0x5638f2ce4280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ef4eba33f60e36a3 server_cookie=2a4fd68415da3cdb in_seq=0 out_seq=0 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef0aff5700 1 -- 172.21.15.52:0/2870330379 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5638f2bb62d0 con 0x5638f2ce1e90 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef0aff5700 10 client.29516 handle_command_reply: tid=0 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 client.29516 shutdown 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 msgr2=0x5638f2c6f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c6f7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fef1809b720 tx=0x7fef180a4430 comp rx=0 tx=0).stop 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 shutdown_connections 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5638f2ce1e90 0x5638f2ce4280 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5638f2c63880 0x5638f2c7e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef0aff5700 0 client.29516 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5638f2c5d7a0 0x5638f2c6f7a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 --2- 172.21.15.52:0/2870330379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5638f2c62f60 0x5638f2c6fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.803+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 >> 172.21.15.52:0/2870330379 conn(0x5638f2b8a080 msgr2=0x5638f2c64370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 shutdown_connections 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef37a7bb80 1 -- 172.21.15.52:0/2870330379 wait complete. 2024-06-02T23:29:38.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef37a7bb80 20 client.29516 trim_cache size 0 max 16384 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef100a2ca0 msgr2=0x7fef100a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef100a2ca0 0x7fef100a5160 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fef1c009750 tx=0x7fef1c009310 comp rx=0 tx=0).stop 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 msgr2=0x7fef28101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.804+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef28101460 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fef1800e9a0 tx=0x7fef1800ecb0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 shutdown_connections 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fef100a2ca0 0x7fef100a5160 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef28108500 0x7fef28101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fef2806a170 0x7fef28101460 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 --2- 172.21.15.52:0/3766100834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef2806ab30 0x7fef281019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 >> 172.21.15.52:0/3766100834 conn(0x7fef28078340 msgr2=0x7fef280fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 shutdown_connections 2024-06-02T23:29:38.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:37.805+0000 7fef2d1f8700 1 -- 172.21.15.52:0/3766100834 wait complete. 2024-06-02T23:29:39.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35058 7.24 MB/sec execute 167 sec latency 118.475 ms 2024-06-02T23:29:39.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:39 smithi123 ceph-mon[35977]: pgmap v964: 97 pgs: 97 active+clean; 418 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-06-02T23:29:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:39 smithi174 ceph-mon[35994]: pgmap v964: 97 pgs: 97 active+clean; 418 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-06-02T23:29:39.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:39 smithi052 ceph-mon[32340]: pgmap v964: 97 pgs: 97 active+clean; 418 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 848 KiB/s rd, 3.2 MiB/s wr, 236 op/s 2024-06-02T23:29:40.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35234 7.28 MB/sec execute 168 sec latency 148.441 ms 2024-06-02T23:29:40.941 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:29:41.020 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:41.443 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:41 smithi052 ceph-mon[32340]: pgmap v965: 97 pgs: 97 active+clean; 424 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.1 MiB/s wr, 260 op/s 2024-06-02T23:29:41.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35384 7.28 MB/sec execute 169 sec latency 133.344 ms 2024-06-02T23:29:41.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:41 smithi123 ceph-mon[35977]: pgmap v965: 97 pgs: 97 active+clean; 424 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.1 MiB/s wr, 260 op/s 2024-06-02T23:29:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:41 smithi174 ceph-mon[35994]: pgmap v965: 97 pgs: 97 active+clean; 424 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 945 KiB/s rd, 4.1 MiB/s wr, 260 op/s 2024-06-02T23:29:41.818 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:42.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35540 7.29 MB/sec execute 170 sec latency 61.893 ms 2024-06-02T23:29:43.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35700 7.29 MB/sec execute 171 sec latency 77.776 ms 2024-06-02T23:29:43.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:43 smithi123 ceph-mon[35977]: pgmap v966: 97 pgs: 97 active+clean; 429 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-06-02T23:29:43.510 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:43 smithi052 ceph-mon[32340]: pgmap v966: 97 pgs: 97 active+clean; 429 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-06-02T23:29:43.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 -- 172.21.15.52:0/909058529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 msgr2=0x7fe898107830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 --2- 172.21.15.52:0/909058529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 0x7fe898107830 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fe894009990 tx=0x7fe894009ca0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 -- 172.21.15.52:0/909058529 shutdown_connections 2024-06-02T23:29:43.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 --2- 172.21.15.52:0/909058529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898101440 0x7fe89810f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 --2- 172.21.15.52:0/909058529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 0x7fe898107830 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 --2- 172.21.15.52:0/909058529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898100a80 0x7fe898100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 -- 172.21.15.52:0/909058529 >> 172.21.15.52:0/909058529 conn(0x7fe898077a30 msgr2=0x7fe8980fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:43.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.587+0000 7fe89fc32700 1 -- 172.21.15.52:0/909058529 shutdown_connections 2024-06-02T23:29:43.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.588+0000 7fe89fc32700 1 -- 172.21.15.52:0/909058529 wait complete. 2024-06-02T23:29:43.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.589+0000 7fe89fc32700 1 Processor -- start 2024-06-02T23:29:43.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.589+0000 7fe89fc32700 1 -- start start 2024-06-02T23:29:43.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.589+0000 7fe89fc32700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 0x7fe89819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.589+0000 7fe89fc32700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898101440 0x7fe89819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.589+0000 7fe89fc32700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 0x7fe8981989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89fc32700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981126e0 con 0x7fe898107450 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89fc32700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe898112560 con 0x7fe898101440 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89fc32700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe898112860 con 0x7fe898100a80 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 0x7fe89819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 0x7fe89819e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34548/0 (socket says 172.21.15.52:34548) 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 -- 172.21.15.52:0/928511781 learned_addr learned my addr 172.21.15.52:0/928511781 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d1cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898101440 0x7fe89819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 0x7fe8981989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 -- 172.21.15.52:0/928511781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898101440 msgr2=0x7fe89819ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898101440 0x7fe89819ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 -- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 msgr2=0x7fe8981989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 0x7fe8981989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.590+0000 7fe89d9ce700 1 -- 172.21.15.52:0/928511781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe898199100 con 0x7fe898100a80 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.591+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 0x7fe89819e790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe8940057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.591+0000 7fe88effd700 1 -- 172.21.15.52:0/928511781 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe894027070 con 0x7fe898100a80 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.591+0000 7fe88effd700 1 -- 172.21.15.52:0/928511781 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe894022470 con 0x7fe898100a80 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.591+0000 7fe88effd700 1 -- 172.21.15.52:0/928511781 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89400b610 con 0x7fe898100a80 2024-06-02T23:29:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.591+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe898199390 con 0x7fe898100a80 2024-06-02T23:29:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.591+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8981a5660 con 0x7fe898100a80 2024-06-02T23:29:43.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.594+0000 7fe88effd700 1 -- 172.21.15.52:0/928511781 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe89402a070 con 0x7fe898100a80 2024-06-02T23:29:43.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.595+0000 7fe88effd700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe8840a2d40 0x7fe8840a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.596+0000 7fe88effd700 1 -- 172.21.15.52:0/928511781 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe8940cfd40 con 0x7fe898100a80 2024-06-02T23:29:43.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.596+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe8840a2d40 0x7fe8840a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.596+0000 7fe8aa4b5b80 1 Processor -- start 2024-06-02T23:29:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.596+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe8840a2d40 0x7fe8840a5200 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe898106be0 tx=0x7fe888009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 -- start start 2024-06-02T23:29:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c9847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c985100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e7c8770f0 con 0x561e7c984ce0 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e7c8d59e0 con 0x561e7c98aeb0 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e7c83d5f0 con 0x561e7c98a590 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c9847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c9847a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41052/0 (socket says 172.21.15.52:41052) 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1847449519 learned_addr learned my addr 172.21.15.52:0/1847449519 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c985100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1847449519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 msgr2=0x561e7c98a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1847449519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 msgr2=0x561e7c985100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c985100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.597+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1847449519 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7c824a60 con 0x561e7c98aeb0 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.598+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c985100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.598+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c9847a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe888011d50 tx=0x7fe888014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f7eb80e92d153b5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.598+0000 7fe883fff700 1 -- 172.21.15.52:0/1847449519 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88800ebd0 con 0x561e7c98aeb0 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.598+0000 7fe883fff700 1 -- 172.21.15.52:0/1847449519 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe88800ed70 con 0x561e7c98aeb0 2024-06-02T23:29:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.598+0000 7fe883fff700 1 -- 172.21.15.52:0/1847449519 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88801cbd0 con 0x561e7c98aeb0 2024-06-02T23:29:43.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1847449519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 msgr2=0x561e7c9847a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c9847a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fe888011d50 tx=0x7fe888014000 comp rx=0 tx=0).stop 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1847449519 shutdown_connections 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c985100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1847449519 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c9847a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1847449519 >> 172.21.15.52:0/1847449519 conn(0x561e7c8b1210 msgr2=0x561e7c97fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.599+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1847449519 shutdown_connections 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.600+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1847449519 wait complete. 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.600+0000 7fe8aa4b5b80 1 -- start start 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.600+0000 7fe8aa4b5b80 1 Processor -- start 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 0x561e7c990c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98aeb0 0x561e7c98e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e7c876150 con 0x561e7c98aeb0 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e7c8d19a0 con 0x561e7c984ce0 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e7c991890 con 0x561e7c98a590 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 0x561e7c990c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe8aa4b5b80 10 client.0 _ll_register_callbacks cb 0x561e7c86b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 0x561e7c990c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41066/0 (socket says 172.21.15.52:41066) 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 -- 172.21.15.52:0/3793959734 learned_addr learned my addr 172.21.15.52:0/3793959734 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d1cd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98aeb0 0x561e7c98e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 -- 172.21.15.52:0/3793959734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 msgr2=0x561e7c98dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.604 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 -- 172.21.15.52:0/3793959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98aeb0 msgr2=0x561e7c98e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98aeb0 0x561e7c98e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.601+0000 7fe89d9ce700 1 -- 172.21.15.52:0/3793959734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7c89b8e0 con 0x561e7c984ce0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 0x561e7c990c70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe8940cd870 tx=0x7fe89409c4e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7280e0de0c5457de server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8817fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8817fa700 1 -- 172.21.15.52:0/3793959734 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89409bfc0 con 0x561e7c984ce0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88800ed70 con 0x561e7c984ce0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8817fa700 1 -- 172.21.15.52:0/3793959734 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe8940a4db0 con 0x561e7c984ce0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8817fa700 1 -- 172.21.15.52:0/3793959734 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe89409cd60 con 0x561e7c984ce0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8aa4b5b80 10 client.29528 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561e7c99be00 con 0x561e7c984ce0 2024-06-02T23:29:43.605 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.602+0000 7fe8817fa700 1 -- 172.21.15.52:0/3793959734 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe88800ed70 con 0x561e7c984ce0 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8817fa700 1 -- 172.21.15.52:0/3793959734 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe8940a1ba0 con 0x561e7c984ce0 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8aa4b5b80 10 client.29528 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8aa4b5b80 10 client.29528 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8aa4b5b80 20 client.29528 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7c9a07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8aa4b5b80 4 client.29528 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:43.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e7c8e2ee0 con 0x561e7c99e3c0 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7c9a07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe8817fa700 10 client.29528 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.603+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7c9a07d0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d0c207c3a218959 server_cookie=b7608d8c6a5a9967 in_seq=0 out_seq=0 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.604+0000 7fe8817fa700 1 -- 172.21.15.52:0/3793959734 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e7c8e2ee0 con 0x561e7c99e3c0 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.604+0000 7fe8817fa700 10 client.29528 handle_command_reply: tid=0 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.604+0000 7fe8aa4b5b80 1 client.29528 shutdown 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 msgr2=0x561e7c990c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 0x561e7c990c70 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fe8940cd870 tx=0x7fe89409c4e0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 shutdown_connections 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7c9a07d0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98aeb0 0x561e7c98e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8817fa700 0 client.29528 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c984ce0 0x561e7c990c70 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/3793959734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98a590 0x561e7c98dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 >> 172.21.15.52:0/3793959734 conn(0x561e7c8b1210 msgr2=0x561e7c97ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:43.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 shutdown_connections 2024-06-02T23:29:43.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.605+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/3793959734 wait complete. 2024-06-02T23:29:43.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.606+0000 7fe8aa4b5b80 20 client.29528 trim_cache size 0 max 16384 2024-06-02T23:29:43.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 Processor -- start 2024-06-02T23:29:43.641 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 -- start start 2024-06-02T23:29:43.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c9847a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 0x561e7c98d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98aeb0 0x561e7c99a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981aa5d0 con 0x561e7c984ce0 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ac110 con 0x561e7c98a590 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981aaca0 con 0x561e7c98aeb0 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c9847a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe89d1cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 0x561e7c98d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe89d1cd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 0x561e7c98d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41080/0 (socket says 172.21.15.52:41080) 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.641+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1916401567 learned_addr learned my addr 172.21.15.52:0/1916401567 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98aeb0 0x561e7c99a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1916401567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98aeb0 msgr2=0x561e7c99a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98aeb0 0x561e7c99a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1916401567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 msgr2=0x561e7c9847a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c9847a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89d1cd700 1 -- 172.21.15.52:0/1916401567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e7c980660 con 0x561e7c98a590 2024-06-02T23:29:43.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 0x561e7c98d2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe888011d50 tx=0x7fe8880058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7deee45025e4fa31 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:43.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:43 smithi174 ceph-mon[35994]: pgmap v966: 97 pgs: 97 active+clean; 429 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 973 KiB/s rd, 4.1 MiB/s wr, 236 op/s 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe8837fe700 1 -- 172.21.15.52:0/1916401567 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888039030 con 0x561e7c98a590 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe8837fe700 1 -- 172.21.15.52:0/1916401567 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8880119e0 con 0x561e7c98a590 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.642+0000 7fe8837fe700 1 -- 172.21.15.52:0/1916401567 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888005110 con 0x561e7c98a590 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1916401567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 msgr2=0x561e7c98d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 0x561e7c98d2f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe888011d50 tx=0x7fe8880058e0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1916401567 shutdown_connections 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c98aeb0 0x561e7c99a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c984ce0 0x561e7c9847a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/1916401567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98a590 0x561e7c98d2f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.643+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1916401567 >> 172.21.15.52:0/1916401567 conn(0x561e7c8b1210 msgr2=0x561e7c999a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.644+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1916401567 shutdown_connections 2024-06-02T23:29:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.644+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/1916401567 wait complete. 2024-06-02T23:29:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.644+0000 7fe8aa4b5b80 1 -- start start 2024-06-02T23:29:43.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.645+0000 7fe8aa4b5b80 1 Processor -- start 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.645+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c984ce0 0x561e7c997310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.645+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98a590 0x561e7c993b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe8aa4b5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c994090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981aba40 con 0x561e7c98a590 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981ab370 con 0x561e7c98aeb0 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe8aa4b5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe8981a8570 con 0x561e7c984ce0 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe8aa4b5b80 10 client.0 _ll_register_callbacks cb 0x561e7c86b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c984ce0 0x561e7c997310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89d9ce700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c984ce0 0x561e7c997310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34580/0 (socket says 172.21.15.52:34580) 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89d9ce700 1 -- 172.21.15.52:0/4100892860 learned_addr learned my addr 172.21.15.52:0/4100892860 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c994090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98a590 0x561e7c993b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89e1cf700 1 -- 172.21.15.52:0/4100892860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c984ce0 msgr2=0x561e7c997310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c984ce0 0x561e7c997310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89e1cf700 1 -- 172.21.15.52:0/4100892860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98a590 msgr2=0x561e7c993b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98a590 0x561e7c993b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.646+0000 7fe89e1cf700 1 -- 172.21.15.52:0/4100892860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e7c9948a0 con 0x561e7c98aeb0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe89d1cd700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98a590 0x561e7c993b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe89e1cf700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c994090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x561e7c991450 tx=0x7fe890007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70cf172594b69d4c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8817fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8817fa700 1 -- 172.21.15.52:0/4100892860 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890003e10 con 0x561e7c98aeb0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8880119e0 con 0x561e7c98aeb0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8817fa700 1 -- 172.21.15.52:0/4100892860 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe890004bb0 con 0x561e7c98aeb0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8817fa700 1 -- 172.21.15.52:0/4100892860 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe890009390 con 0x561e7c98aeb0 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8aa4b5b80 10 client.29534 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:43.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.647+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561e7ca0a100 con 0x561e7c98aeb0 2024-06-02T23:29:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8817fa700 1 -- 172.21.15.52:0/4100892860 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe8880119e0 con 0x561e7c98aeb0 2024-06-02T23:29:43.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8817fa700 1 -- 172.21.15.52:0/4100892860 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe8900043b0 con 0x561e7c98aeb0 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8aa4b5b80 10 client.29534 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8aa4b5b80 10 client.29534 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8aa4b5b80 20 client.29534 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7ca0c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8aa4b5b80 4 client.29534 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.0, "await_for": 45.0}] 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.0, "await_for": 45.0}) -- 0x561e7c8dd460 con 0x561e7c99e3c0 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.648+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7ca0c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.649+0000 7fe8817fa700 10 client.29534 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:43.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.649+0000 7fe89d9ce700 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7ca0c8f0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5b44a59a88df8 server_cookie=9533f284b65dd9 in_seq=0 out_seq=0 2024-06-02T23:29:43.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8817fa700 1 -- 172.21.15.52:0/4100892860 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e7c8dd460 con 0x561e7c99e3c0 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8817fa700 10 client.29534 handle_command_reply: tid=0 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 client.29534 shutdown 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 msgr2=0x561e7c994090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c994090 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x561e7c991450 tx=0x7fe890007ed0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 shutdown_connections 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e7c99e3c0 0x561e7ca0c8f0 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e7c98aeb0 0x561e7c994090 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8817fa700 0 client.29534 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e7c984ce0 0x561e7c997310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.947+0000 7fe8aa4b5b80 1 --2- 172.21.15.52:0/4100892860 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e7c98a590 0x561e7c993b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 >> 172.21.15.52:0/4100892860 conn(0x561e7c8b1210 msgr2=0x561e7c984020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:43.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 shutdown_connections 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe8aa4b5b80 1 -- 172.21.15.52:0/4100892860 wait complete. 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe8aa4b5b80 20 client.29534 trim_cache size 0 max 16384 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 58, 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "eed044fb": { 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 58, 2024-06-02T23:29:43.950 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.0, 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:43.951 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:43.952 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:43.952 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:43.952 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe8840a2d40 msgr2=0x7fe8840a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe89fc32700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe8840a2d40 0x7fe8840a5200 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe898106be0 tx=0x7fe888009260 comp rx=0 tx=0).stop 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 msgr2=0x7fe89819e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.948+0000 7fe89fc32700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 0x7fe89819e790 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe894000c00 tx=0x7fe8940057a0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 shutdown_connections 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe8840a2d40 0x7fe8840a5200 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe898107450 0x7fe8981989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe898100a80 0x7fe89819e790 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 --2- 172.21.15.52:0/928511781 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe898101440 0x7fe89819ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 >> 172.21.15.52:0/928511781 conn(0x7fe898077a30 msgr2=0x7fe8981052c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 shutdown_connections 2024-06-02T23:29:43.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:43.949+0000 7fe89fc32700 1 -- 172.21.15.52:0/928511781 wait complete. 2024-06-02T23:29:44.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.25 MB/sec execute 172 sec latency 820.059 ms 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 -- 172.21.15.52:0/2375353999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3941086f0 msgr2=0x7fa394108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2375353999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3941086f0 0x7fa394108ad0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fa390009990 tx=0x7fa390009ca0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 -- 172.21.15.52:0/2375353999 shutdown_connections 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2375353999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 0x7fa39410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2375353999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa3941086f0 0x7fa394108ad0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2375353999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa394100a80 0x7fa394100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 -- 172.21.15.52:0/2375353999 >> 172.21.15.52:0/2375353999 conn(0x7fa394077a00 msgr2=0x7fa3940fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 -- 172.21.15.52:0/2375353999 shutdown_connections 2024-06-02T23:29:45.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.054+0000 7fa39bf07700 1 -- 172.21.15.52:0/2375353999 wait complete. 2024-06-02T23:29:45.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 Processor -- start 2024-06-02T23:29:45.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 -- start start 2024-06-02T23:29:45.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 0x7fa394101c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 0x7fa3941021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941086f0 0x7fa3941026e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa394112860 con 0x7fa394100a80 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941126e0 con 0x7fa3941086f0 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39bf07700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941129e0 con 0x7fa394101440 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 0x7fa3941021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa399ca3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 0x7fa394101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa39a4a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941086f0 0x7fa3941026e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 0x7fa3941021a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38460/0 (socket says 172.21.15.52:38460) 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa399ca3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 0x7fa394101c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55960/0 (socket says 172.21.15.52:55960) 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.055+0000 7fa3994a2700 1 -- 172.21.15.52:0/2546061806 learned_addr learned my addr 172.21.15.52:0/2546061806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39a4a4700 1 -- 172.21.15.52:0/2546061806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 msgr2=0x7fa3941021a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39a4a4700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 0x7fa3941021a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39a4a4700 1 -- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 msgr2=0x7fa394101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39a4a4700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 0x7fa394101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39a4a4700 1 -- 172.21.15.52:0/2546061806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3941a7190 con 0x7fa3941086f0 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa399ca3700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 0x7fa394101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39a4a4700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941086f0 0x7fa3941026e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007f10 tx=0x7fa38c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa38affd700 1 -- 172.21.15.52:0/2546061806 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c004db0 con 0x7fa3941086f0 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3941a7480 con 0x7fa3941086f0 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa38affd700 1 -- 172.21.15.52:0/2546061806 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa38c009e20 con 0x7fa3941086f0 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa38affd700 1 -- 172.21.15.52:0/2546061806 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38c015400 con 0x7fa3941086f0 2024-06-02T23:29:45.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.056+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a7a10 con 0x7fa3941086f0 2024-06-02T23:29:45.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.059+0000 7fa38affd700 1 -- 172.21.15.52:0/2546061806 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa38c00db20 con 0x7fa3941086f0 2024-06-02T23:29:45.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.060+0000 7fa3a678ab80 1 Processor -- start 2024-06-02T23:29:45.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.060+0000 7fa3a678ab80 1 -- start start 2024-06-02T23:29:45.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc498b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc499090 0x55facc492a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc492fb0 0x55facc4933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55facc37cf60 con 0x55facc492fb0 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55facc3db850 con 0x55facc499090 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55facc343460 con 0x55facc498770 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc498b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa399ca3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc499090 0x55facc492a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa38affd700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3800a2c20 0x7fa3800a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc498b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38476/0 (socket says 172.21.15.52:38476) 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3994a2700 1 -- 172.21.15.52:0/939783679 learned_addr learned my addr 172.21.15.52:0/939783679 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa3994a2700 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc492fb0 0x55facc4933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa39a4a4700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3800a2c20 0x7fa3800a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.061+0000 7fa38affd700 1 -- 172.21.15.52:0/2546061806 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa38c0ce8d0 con 0x7fa3941086f0 2024-06-02T23:29:45.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa3994a2700 1 -- 172.21.15.52:0/939783679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc499090 msgr2=0x55facc492a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa3994a2700 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc499090 0x55facc492a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa3994a2700 1 -- 172.21.15.52:0/939783679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc492fb0 msgr2=0x55facc4933f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa3994a2700 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc492fb0 0x55facc4933f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa3994a2700 1 -- 172.21.15.52:0/939783679 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55facc32a8d0 con 0x55facc498770 2024-06-02T23:29:45.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa3994a2700 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc498b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3840118f0 tx=0x7fa38400a860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6abca3ea626a1704 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:45.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.062+0000 7fa37ffff700 1 -- 172.21.15.52:0/939783679 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384004830 con 0x55facc498770 2024-06-02T23:29:45.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.063+0000 7fa37ffff700 1 -- 172.21.15.52:0/939783679 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa384005020 con 0x55facc498770 2024-06-02T23:29:45.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.063+0000 7fa37ffff700 1 -- 172.21.15.52:0/939783679 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38400f8a0 con 0x55facc498770 2024-06-02T23:29:45.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.063+0000 7fa39a4a4700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3800a2c20 0x7fa3800a50e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0cc450 tx=0x7fa38c065d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:45.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 -- 172.21.15.52:0/939783679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 msgr2=0x55facc498b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc498b50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3840118f0 tx=0x7fa38400a860 comp rx=0 tx=0).stop 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 -- 172.21.15.52:0/939783679 shutdown_connections 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc492fb0 0x55facc4933f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc498b50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/939783679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc499090 0x55facc492a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 -- 172.21.15.52:0/939783679 >> 172.21.15.52:0/939783679 conn(0x55facc3b7080 msgr2=0x55facc485220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 -- 172.21.15.52:0/939783679 shutdown_connections 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 -- 172.21.15.52:0/939783679 wait complete. 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.064+0000 7fa3a678ab80 1 -- start start 2024-06-02T23:29:45.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 Processor -- start 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc498770 0x55facc49c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 0x55facc49c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55facc37bfc0 con 0x55facc498770 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55facc3d7810 con 0x55facc492fb0 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55facc49fa80 con 0x55facc499090 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3a678ab80 10 client.0 _ll_register_callbacks cb 0x55facc3715b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.065+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 0x55facc49c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 0x55facc49c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38484/0 (socket says 172.21.15.52:38484) 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 -- 172.21.15.52:0/3961180047 learned_addr learned my addr 172.21.15.52:0/3961180047 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa399ca3700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc498770 0x55facc49c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 -- 172.21.15.52:0/3961180047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 msgr2=0x55facc49eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 -- 172.21.15.52:0/3961180047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc498770 msgr2=0x55facc49c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc498770 0x55facc49c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.066+0000 7fa3994a2700 1 -- 172.21.15.52:0/3961180047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55facc3a1750 con 0x55facc499090 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa399ca3700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc498770 0x55facc49c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa3994a2700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 0x55facc49c6f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa384011ee0 tx=0x7fa384011660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51eff1b0780a892a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa37d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa37d7fa700 1 -- 172.21.15.52:0/3961180047 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384028040 con 0x55facc499090 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa384005020 con 0x55facc499090 2024-06-02T23:29:45.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa37d7fa700 1 -- 172.21.15.52:0/3961180047 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa384004980 con 0x55facc499090 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa37d7fa700 1 -- 172.21.15.52:0/3961180047 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa384034050 con 0x55facc499090 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa3a678ab80 10 client.29185 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.067+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55facc4aa2b0 con 0x55facc499090 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa37d7fa700 1 -- 172.21.15.52:0/3961180047 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa384005020 con 0x55facc499090 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa37d7fa700 1 -- 172.21.15.52:0/3961180047 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa38400fc90 con 0x55facc499090 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa3a678ab80 10 client.29185 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa3a678ab80 10 client.29185 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa3a678ab80 20 client.29185 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc4ac780 0x55facc4aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa3a678ab80 4 client.29185 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:45.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.068+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55facc3e8d50 con 0x55facc4ac780 2024-06-02T23:29:45.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.069+0000 7fa399ca3700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc4ac780 0x55facc4aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.070+0000 7fa399ca3700 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc4ac780 0x55facc4aeb70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f51ed93ea4199522 server_cookie=a0faa290c9a5a195 in_seq=0 out_seq=0 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.070+0000 7fa37d7fa700 10 client.29185 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa37d7fa700 1 -- 172.21.15.52:0/3961180047 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55facc3e8d50 con 0x55facc4ac780 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa37d7fa700 10 client.29185 handle_command_reply: tid=0 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 client.29185 shutdown 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 msgr2=0x55facc49c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 0x55facc49c6f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa384011ee0 tx=0x7fa384011660 comp rx=0 tx=0).stop 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 shutdown_connections 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc4ac780 0x55facc4aeb70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc499090 0x55facc49c6f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/3961180047 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc498770 0x55facc49c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 >> 172.21.15.52:0/3961180047 conn(0x55facc3b7080 msgr2=0x55facc484700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa37d7fa700 0 client.29185 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:29:45.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 shutdown_connections 2024-06-02T23:29:45.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 1 -- 172.21.15.52:0/3961180047 wait complete. 2024-06-02T23:29:45.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.071+0000 7fa3a678ab80 20 client.29185 trim_cache size 0 max 16384 2024-06-02T23:29:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 Processor -- start 2024-06-02T23:29:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 -- start start 2024-06-02T23:29:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc4a8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc49e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc484e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941aa8c0 con 0x55facc499090 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ac400 con 0x55facc492fb0 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.087+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941aaf90 con 0x55facc498770 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc484e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc484e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55984/0 (socket says 172.21.15.52:55984) 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 -- 172.21.15.52:0/2999464734 learned_addr learned my addr 172.21.15.52:0/2999464734 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa399ca3700 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc49e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc4a8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 -- 172.21.15.52:0/2999464734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 msgr2=0x55facc49e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc49e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 -- 172.21.15.52:0/2999464734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 msgr2=0x55facc4a8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc4a8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 -- 172.21.15.52:0/2999464734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55facc492cc0 con 0x55facc499090 2024-06-02T23:29:45.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc484e80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa384005950 tx=0x7fa384030040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9edd69a639132c21 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa37f7fe700 1 -- 172.21.15.52:0/2999464734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38400a830 con 0x55facc499090 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa37f7fe700 1 -- 172.21.15.52:0/2999464734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa384023070 con 0x55facc499090 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.088+0000 7fa37f7fe700 1 -- 172.21.15.52:0/2999464734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384004ce0 con 0x55facc499090 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2999464734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 msgr2=0x55facc484e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc484e80 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fa384005950 tx=0x7fa384030040 comp rx=0 tx=0).stop 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2999464734 shutdown_connections 2024-06-02T23:29:45.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc484e80 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc4a8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2999464734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc49e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2999464734 >> 172.21.15.52:0/2999464734 conn(0x55facc3b7080 msgr2=0x55facc49d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2999464734 shutdown_connections 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.089+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2999464734 wait complete. 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 -- start start 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 Processor -- start 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc4a0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc4a0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941abd30 con 0x55facc499090 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ab660 con 0x55facc492fb0 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa39419c7f0 con 0x55facc498770 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3a678ab80 10 client.0 _ll_register_callbacks cb 0x55facc3715b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:45.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc4a0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3994a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc4a0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55988/0 (socket says 172.21.15.52:55988) 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa399ca3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc4a0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.090+0000 7fa3994a2700 1 -- 172.21.15.52:0/2832608979 learned_addr learned my addr 172.21.15.52:0/2832608979 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 -- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 msgr2=0x55facc4a0210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc4a0210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 -- 172.21.15.52:0/2832608979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 msgr2=0x55facc49fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 -- 172.21.15.52:0/2832608979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55facc481810 con 0x55facc499090 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3994a2700 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc4a0750 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x55facc4a10b0 tx=0x7fa384005ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddba41d0e7886d57 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa37d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa37d7fa700 1 -- 172.21.15.52:0/2832608979 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384005660 con 0x55facc499090 2024-06-02T23:29:45.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa37d7fa700 1 -- 172.21.15.52:0/2832608979 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa38401ec70 con 0x55facc499090 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa384023070 con 0x55facc499090 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa37d7fa700 1 -- 172.21.15.52:0/2832608979 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa38404a050 con 0x55facc499090 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3a678ab80 10 client.20208 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55facc5162f0 con 0x55facc499090 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.091+0000 7fa37d7fa700 1 -- 172.21.15.52:0/2832608979 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa384023070 con 0x55facc499090 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa37d7fa700 1 -- 172.21.15.52:0/2832608979 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa38400f9e0 con 0x55facc499090 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa3a678ab80 10 client.20208 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa3a678ab80 10 client.20208 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa3a678ab80 20 client.20208 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc5168c0 0x55facc518ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa3a678ab80 4 client.20208 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55facc3e32d0 con 0x55facc5168c0 2024-06-02T23:29:45.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa399ca3700 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc5168c0 0x55facc518ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:45.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa37d7fa700 10 client.20208 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.092+0000 7fa399ca3700 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc5168c0 0x55facc518ce0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a92a750ca02df5d3 server_cookie=bb7a0f81273a8d6e in_seq=0 out_seq=0 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa37d7fa700 1 -- 172.21.15.52:0/2832608979 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55facc3e32d0 con 0x55facc5168c0 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa37d7fa700 10 client.20208 handle_command_reply: tid=0 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 client.20208 shutdown 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 msgr2=0x55facc4a0750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc4a0750 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x55facc4a10b0 tx=0x7fa384005ba0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 shutdown_connections 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55facc5168c0 0x55facc518ce0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55facc499090 0x55facc4a0750 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55facc492fb0 0x55facc49fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa37d7fa700 0 client.20208 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 --2- 172.21.15.52:0/2832608979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55facc498770 0x55facc4a0210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 >> 172.21.15.52:0/2832608979 conn(0x55facc3b7080 msgr2=0x55facc489ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:45.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 shutdown_connections 2024-06-02T23:29:45.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 1 -- 172.21.15.52:0/2832608979 wait complete. 2024-06-02T23:29:45.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.093+0000 7fa3a678ab80 20 client.20208 trim_cache size 0 max 16384 2024-06-02T23:29:45.098 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3800a2c20 msgr2=0x7fa3800a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3800a2c20 0x7fa3800a50e0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0cc450 tx=0x7fa38c065d30 comp rx=0 tx=0).stop 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941086f0 msgr2=0x7fa3941026e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941086f0 0x7fa3941026e0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fa38c007f10 tx=0x7fa38c00d3b0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 shutdown_connections 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa3800a2c20 0x7fa3800a50e0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941086f0 0x7fa3941026e0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa394100a80 0x7fa394101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 --2- 172.21.15.52:0/2546061806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa394101440 0x7fa3941021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 >> 172.21.15.52:0/2546061806 conn(0x7fa394077a00 msgr2=0x7fa3940ff020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 shutdown_connections 2024-06-02T23:29:45.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:45.094+0000 7fa39bf07700 1 -- 172.21.15.52:0/2546061806 wait complete. 2024-06-02T23:29:45.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.21 MB/sec execute 173 sec latency 1820.095 ms 2024-06-02T23:29:45.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:45 smithi123 ceph-mon[35977]: pgmap v967: 97 pgs: 97 active+clean; 430 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2024-06-02T23:29:45.596 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:45 smithi052 ceph-mon[32340]: pgmap v967: 97 pgs: 97 active+clean; 430 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2024-06-02T23:29:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:45 smithi174 ceph-mon[35994]: pgmap v967: 97 pgs: 97 active+clean; 430 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 936 KiB/s rd, 4.0 MiB/s wr, 217 op/s 2024-06-02T23:29:45.864 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:29:46.433 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'eed044fb', quiesce took 24.6 seconds. Will release after: 2.3999999999999986 2024-06-02T23:29:46.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.17 MB/sec execute 174 sec latency 2820.190 ms 2024-06-02T23:29:46.552 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 -- 172.21.15.52:0/1873447932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 msgr2=0x7ff2f8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.108 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1873447932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f8108d40 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009ca0 tx=0x7ff2f002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 -- 172.21.15.52:0/1873447932 shutdown_connections 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1873447932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f8108d40 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1873447932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f8101af0 0x7ff2f8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1873447932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f8100900 0x7ff2f8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 -- 172.21.15.52:0/1873447932 >> 172.21.15.52:0/1873447932 conn(0x7ff2f80fbe70 msgr2=0x7ff2f80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 -- 172.21.15.52:0/1873447932 shutdown_connections 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 -- 172.21.15.52:0/1873447932 wait complete. 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.106+0000 7ff2fce38640 1 Processor -- start 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 -- start start 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f8100900 0x7ff2f8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f8101af0 0x7ff2f819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f81a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f810c410 con 0x7ff2f81024b0 2024-06-02T23:29:47.109 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f810c290 con 0x7ff2f8101af0 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2fce38640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f810c590 con 0x7ff2f8100900 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f81a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f81a1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:55998/0 (socket says 172.21.15.52:55998) 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 -- 172.21.15.52:0/1639105679 learned_addr learned my addr 172.21.15.52:0/1639105679 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f659c640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f8100900 0x7ff2f8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2effff640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f8101af0 0x7ff2f819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 -- 172.21.15.52:0/1639105679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f8100900 msgr2=0x7ff2f8199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f8100900 0x7ff2f8199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 -- 172.21.15.52:0/1639105679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f8101af0 msgr2=0x7ff2f819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f8101af0 0x7ff2f819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.107+0000 7ff2f6d9d640 1 -- 172.21.15.52:0/1639105679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f81a19b0 con 0x7ff2f81024b0 2024-06-02T23:29:47.110 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.108+0000 7ff2f6d9d640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f81a1310 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009dc0 tx=0x7ff2f0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.108+0000 7ff2ef7fe640 1 -- 172.21.15.52:0/1639105679 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0032ec0 con 0x7ff2f81024b0 2024-06-02T23:29:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.108+0000 7ff2ef7fe640 1 -- 172.21.15.52:0/1639105679 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2f00044f0 con 0x7ff2f81024b0 2024-06-02T23:29:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.108+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f81a1c40 con 0x7ff2f81024b0 2024-06-02T23:29:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.108+0000 7ff2ef7fe640 1 -- 172.21.15.52:0/1639105679 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0030d40 con 0x7ff2f81024b0 2024-06-02T23:29:47.111 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.108+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f81a2190 con 0x7ff2f81024b0 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ef7fe640 1 -- 172.21.15.52:0/1639105679 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff2f0004040 con 0x7ff2f81024b0 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 Processor -- start 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 -- start start 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4100430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4105500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b4108000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5646b40634f0 con 0x5646b4100a00 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646b4001870 con 0x5646b4105b70 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646b405f4b0 con 0x5646b4100010 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.111+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4105500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4105500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56010/0 (socket says 172.21.15.52:56010) 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 -- 172.21.15.52:0/2761184941 learned_addr learned my addr 172.21.15.52:0/2761184941 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2effff640 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4100430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2effff640 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b4108000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 -- 172.21.15.52:0/2761184941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 msgr2=0x5646b4100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4100430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2ef7fe640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2e40a2dd0 0x7ff2e40a5290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 -- 172.21.15.52:0/2761184941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 msgr2=0x5646b4108000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b4108000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 -- 172.21.15.52:0/2761184941 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b401d9b0 con 0x5646b4100a00 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2effff640 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b4108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f6d9d640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2e40a2dd0 0x7ff2e40a5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4105500 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bc70 tx=0x7ff2e80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86798dd912ef6fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2ef7fe640 1 -- 172.21.15.52:0/1639105679 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff2f00f05d0 con 0x7ff2f81024b0 2024-06-02T23:29:47.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2ecff9640 1 -- 172.21.15.52:0/2761184941 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8004020 con 0x5646b4100a00 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2ecff9640 1 -- 172.21.15.52:0/2761184941 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e80041c0 con 0x5646b4100a00 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.112+0000 7ff2ecff9640 1 -- 172.21.15.52:0/2761184941 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8011680 con 0x5646b4100a00 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2761184941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 msgr2=0x5646b4105500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4105500 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bc70 tx=0x7ff2e80107a0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2f6d9d640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2e40a2dd0 0x7ff2e40a5290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00bcb60 tx=0x7ff2f008bd50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2761184941 shutdown_connections 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b4108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4105500 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2761184941 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4100430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2761184941 >> 172.21.15.52:0/2761184941 conn(0x5646b40fb680 msgr2=0x5646b40fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2761184941 shutdown_connections 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2761184941 wait complete. 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.113+0000 7ff2ffc22740 1 -- start start 2024-06-02T23:29:47.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 Processor -- start 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b4111830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5646b4108b20 con 0x5646b4105b70 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646b3fecae0 con 0x5646b4100a00 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646b4108ca0 con 0x5646b4100010 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ffc22740 10 client.0 _ll_register_callbacks cb 0x5646b40259b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b4111830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b4111830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56022/0 (socket says 172.21.15.52:56022) 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 -- 172.21.15.52:0/3543990085 learned_addr learned my addr 172.21.15.52:0/3543990085 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 -- 172.21.15.52:0/3543990085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 msgr2=0x5646b410a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 -- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 msgr2=0x5646b410a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 -- 172.21.15.52:0/3543990085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b3d77210 con 0x5646b4105b70 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2effff640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b4111830 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000bf70 tx=0x7ff2e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b943f49653d0823b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.114+0000 7ff2ddffb640 1 -- 172.21.15.52:0/3543990085 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0022060 con 0x5646b4105b70 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ddffb640 1 -- 172.21.15.52:0/3543990085 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff2e00109a0 con 0x5646b4105b70 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e80041c0 con 0x5646b4105b70 2024-06-02T23:29:47.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ddffb640 1 -- 172.21.15.52:0/3543990085 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff2e0010ba0 con 0x5646b4105b70 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 10 client.20220 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5646b3e43620 con 0x5646b4105b70 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ddffb640 1 -- 172.21.15.52:0/3543990085 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e80041c0 con 0x5646b4105b70 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ddffb640 1 -- 172.21.15.52:0/3543990085 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e001d480 con 0x5646b4105b70 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 10 client.20220 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 10 client.20220 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 20 client.20220 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b41160e0 0x5646b4118530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 4 client.20220 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646b4076f10 con 0x5646b41160e0 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.115+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b41160e0 0x5646b4118530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.116+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b41160e0 0x5646b4118530 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27f680ff31447300 server_cookie=b1b993ed565a0ea5 in_seq=0 out_seq=0 2024-06-02T23:29:47.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.116+0000 7ff2ddffb640 10 client.20220 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ddffb640 1 -- 172.21.15.52:0/3543990085 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5646b4076f10 con 0x5646b41160e0 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ddffb640 10 client.20220 handle_command_reply: tid=0 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 client.20220 shutdown 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 msgr2=0x5646b4111830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b4111830 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff2e000bf70 tx=0x7ff2e00046c0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 shutdown_connections 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b41160e0 0x5646b4118530 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b4111830 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ddffb640 0 client.20220 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3543990085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 >> 172.21.15.52:0/3543990085 conn(0x5646b40fb680 msgr2=0x5646b40fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 shutdown_connections 2024-06-02T23:29:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3543990085 wait complete. 2024-06-02T23:29:47.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.117+0000 7ff2ffc22740 20 client.20220 trim_cache size 0 max 16384 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 Processor -- start 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 -- start start 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4101be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4102120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b41117c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81aae20 con 0x5646b4100a00 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81ac960 con 0x5646b4105b70 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81ab4f0 con 0x5646b4100010 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4102120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56032/0 (socket says 172.21.15.52:56032) 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 -- 172.21.15.52:0/3647175341 learned_addr learned my addr 172.21.15.52:0/3647175341 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:47.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 -- 172.21.15.52:0/3647175341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 msgr2=0x5646b4101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 -- 172.21.15.52:0/3647175341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 msgr2=0x5646b41117c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2effff640 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b41117c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b41117c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2f659c640 1 -- 172.21.15.52:0/3647175341 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646b41016d0 con 0x5646b4100a00 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.132+0000 7ff2effff640 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b41117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2f659c640 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4102120 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bc70 tx=0x7ff2e800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=312ed813301ce43d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2dffff640 1 -- 172.21.15.52:0/3647175341 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8005300 con 0x5646b4100a00 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2dffff640 1 -- 172.21.15.52:0/3647175341 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e80054a0 con 0x5646b4100a00 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2dffff640 1 -- 172.21.15.52:0/3647175341 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80038e0 con 0x5646b4100a00 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3647175341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 msgr2=0x5646b4102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4102120 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff2e800bc70 tx=0x7ff2e800bb70 comp rx=0 tx=0).stop 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3647175341 shutdown_connections 2024-06-02T23:29:47.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4105b70 0x5646b41117c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4100a00 0x5646b4102120 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/3647175341 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b4101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.133+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3647175341 >> 172.21.15.52:0/3647175341 conn(0x5646b40fb680 msgr2=0x5646b41090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3647175341 shutdown_connections 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 -- 172.21.15.52:0/3647175341 wait complete. 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 -- start start 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 Processor -- start 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b410fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81ac290 con 0x5646b4105b70 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81abbc0 con 0x5646b4100a00 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f81a8e00 con 0x5646b4100010 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2ffc22740 10 client.0 _ll_register_callbacks cb 0x7ff2e000b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:47.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b410fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b410fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56036/0 (socket says 172.21.15.52:56036) 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2effff640 1 -- 172.21.15.52:0/2323645947 learned_addr learned my addr 172.21.15.52:0/2323645947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2effff640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.134+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2effff640 1 -- 172.21.15.52:0/2323645947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 msgr2=0x5646b410e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2effff640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2effff640 1 -- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 msgr2=0x5646b410ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2effff640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2effff640 1 -- 172.21.15.52:0/2323645947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646b41103e0 con 0x5646b4105b70 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2effff640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b410fdf0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x5646b41082a0 tx=0x7ff2e0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3285ac4d05079c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ddffb640 1 -- 172.21.15.52:0/2323645947 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e002f070 con 0x5646b4105b70 2024-06-02T23:29:47.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ddffb640 1 -- 172.21.15.52:0/2323645947 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff2e00109a0 con 0x5646b4105b70 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e80054a0 con 0x5646b4105b70 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ddffb640 1 -- 172.21.15.52:0/2323645947 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff2e001da90 con 0x5646b4105b70 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 10 client.20226 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5646b4111660 con 0x5646b4105b70 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ddffb640 1 -- 172.21.15.52:0/2323645947 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff2e80054a0 con 0x5646b4105b70 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ddffb640 1 -- 172.21.15.52:0/2323645947 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e00035f0 con 0x5646b4105b70 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 10 client.20226 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 10 client.20226 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 20 client.20226 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b4181040 0x5646b4183460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 4 client.20226 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.135+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646b4070c20 con 0x5646b4181040 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.136+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b4181040 0x5646b4183460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.136+0000 7ff2f659c640 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b4181040 0x5646b4183460 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45584e643ba522a2 server_cookie=f9b8b341332227b9 in_seq=0 out_seq=0 2024-06-02T23:29:47.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.136+0000 7ff2ddffb640 10 client.20226 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.167+0000 7ff2ddffb640 1 -- 172.21.15.52:0/2323645947 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5646b4070c20 con 0x5646b4181040 2024-06-02T23:29:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.167+0000 7ff2ddffb640 10 client.20226 handle_command_reply: tid=0 2024-06-02T23:29:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.167+0000 7ff2ffc22740 1 client.20226 shutdown 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.167+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 msgr2=0x5646b410fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.167+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b410fdf0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x5646b41082a0 tx=0x7ff2e0001e70 comp rx=0 tx=0).stop 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 shutdown_connections 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646b4181040 0x5646b4183460 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646b4105b70 0x5646b410fdf0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646b4100a00 0x5646b410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ddffb640 0 client.20226 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 --2- 172.21.15.52:0/2323645947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646b4100010 0x5646b410e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 >> 172.21.15.52:0/2323645947 conn(0x5646b40fb680 msgr2=0x5646b41090a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 shutdown_connections 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 1 -- 172.21.15.52:0/2323645947 wait complete. 2024-06-02T23:29:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2ffc22740 20 client.20226 trim_cache size 0 max 16384 2024-06-02T23:29:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:29:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:29:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:29:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2e40a2dd0 msgr2=0x7ff2e40a5290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2e40a2dd0 0x7ff2e40a5290 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff2f00bcb60 tx=0x7ff2f008bd50 comp rx=0 tx=0).stop 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 msgr2=0x7ff2f81a1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f81a1310 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7ff2f0009dc0 tx=0x7ff2f0003ba0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 shutdown_connections 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff2e40a2dd0 0x7ff2e40a5290 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff2f81024b0 0x7ff2f81a1310 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f8101af0 0x7ff2f819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 --2- 172.21.15.52:0/1639105679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f8100900 0x7ff2f8199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.168+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 >> 172.21.15.52:0/1639105679 conn(0x7ff2f80fbe70 msgr2=0x7ff2f80fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.169+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 shutdown_connections 2024-06-02T23:29:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:29:47.169+0000 7ff2fce38640 1 -- 172.21.15.52:0/1639105679 wait complete. 2024-06-02T23:29:47.370 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:47 smithi052 ceph-mon[32340]: pgmap v968: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 285 op/s 2024-06-02T23:29:47.461 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.13 MB/sec execute 175 sec latency 3820.269 ms 2024-06-02T23:29:47.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:47 smithi123 ceph-mon[35977]: pgmap v968: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 285 op/s 2024-06-02T23:29:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:47 smithi174 ceph-mon[35994]: pgmap v968: 97 pgs: 97 active+clean; 432 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 285 op/s 2024-06-02T23:29:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 -- 172.21.15.52:0/1560353163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 msgr2=0x7f4b2410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 --2- 172.21.15.52:0/1560353163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b2410f370 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00b0a0 tx=0x7f4b1c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 -- 172.21.15.52:0/1560353163 shutdown_connections 2024-06-02T23:29:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 --2- 172.21.15.52:0/1560353163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b2410f370 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 --2- 172.21.15.52:0/1560353163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 0x7f4b24108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 --2- 172.21.15.52:0/1560353163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 0x7f4b2406b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 -- 172.21.15.52:0/1560353163 >> 172.21.15.52:0/1560353163 conn(0x7f4b24077400 msgr2=0x7f4b24077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 -- 172.21.15.52:0/1560353163 shutdown_connections 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.292+0000 7f4b2b230700 1 -- 172.21.15.52:0/1560353163 wait complete. 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 Processor -- start 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 -- start start 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 0x7f4b241a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b241a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 0x7f4b2419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24112610 con 0x7f4b2406b5c0 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24112490 con 0x7f4b2406ac00 2024-06-02T23:29:48.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b2b230700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b24112790 con 0x7f4b24108530 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b241a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b241a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56048/0 (socket says 172.21.15.52:56048) 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 -- 172.21.15.52:0/380999986 learned_addr learned my addr 172.21.15.52:0/380999986 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b297cd700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 0x7f4b2419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 0x7f4b241a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 -- 172.21.15.52:0/380999986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 msgr2=0x7f4b2419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 0x7f4b2419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 -- 172.21.15.52:0/380999986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 msgr2=0x7f4b241a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 0x7f4b241a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b23fff700 1 -- 172.21.15.52:0/380999986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b2419b260 con 0x7f4b2406b5c0 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b297cd700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 0x7f4b2419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:48.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.293+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 0x7f4b241a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.294+0000 7f4b23fff700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b241a0e80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e9a0 tx=0x7f4b1000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.294+0000 7f4b21ffb700 1 -- 172.21.15.52:0/380999986 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1000cbd0 con 0x7f4b2406b5c0 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.294+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b2419b550 con 0x7f4b2406b5c0 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.294+0000 7f4b21ffb700 1 -- 172.21.15.52:0/380999986 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b1000cd70 con 0x7f4b2406b5c0 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.294+0000 7f4b21ffb700 1 -- 172.21.15.52:0/380999986 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10018750 con 0x7f4b2406b5c0 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.294+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b241a7810 con 0x7f4b2406b5c0 2024-06-02T23:29:48.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.296+0000 7f4b21ffb700 1 -- 172.21.15.52:0/380999986 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4b10010430 con 0x7f4b2406b5c0 2024-06-02T23:29:48.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.297+0000 7f4b21ffb700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b140a2c20 0x7f4b140a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.297+0000 7f4b21ffb700 1 -- 172.21.15.52:0/380999986 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4b100cf380 con 0x7f4b2406b5c0 2024-06-02T23:29:48.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.297+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b140a2c20 0x7f4b140a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 Processor -- start 2024-06-02T23:29:48.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b140a2c20 0x7f4b140a50e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005fd0 tx=0x7f4b1800a5e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 -- start start 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56427203a640 0x56427203aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x5642720349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 0x564272035340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564271f86810 con 0x564272034f20 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564271f2e720 con 0x56427203a640 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564271f827d0 con 0x56427203af60 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b297cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 0x564272035340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x5642720349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x5642720349e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38578/0 (socket says 172.21.15.52:38578) 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b297cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 0x564272035340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56054/0 (socket says 172.21.15.52:56054) 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b23fff700 1 -- 172.21.15.52:0/3328909602 learned_addr learned my addr 172.21.15.52:0/3328909602 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:48.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.298+0000 7f4b28fcc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56427203a640 0x56427203aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b297cd700 1 -- 172.21.15.52:0/3328909602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 msgr2=0x5642720349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b297cd700 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x5642720349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b297cd700 1 -- 172.21.15.52:0/3328909602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56427203a640 msgr2=0x56427203aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b297cd700 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56427203a640 0x56427203aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b297cd700 1 -- 172.21.15.52:0/3328909602 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564271f24f50 con 0x564272034f20 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b23fff700 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x5642720349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56427203a640 0x56427203aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b297cd700 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 0x564272035340 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00d010 tx=0x7f4b1c0094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=635406fe35dfad8d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b0effd700 1 -- 172.21.15.52:0/3328909602 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c00bcb0 con 0x564272034f20 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b0effd700 1 -- 172.21.15.52:0/3328909602 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b1c0074e0 con 0x564272034f20 2024-06-02T23:29:48.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.299+0000 7f4b0effd700 1 -- 172.21.15.52:0/3328909602 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c025680 con 0x564272034f20 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3328909602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 msgr2=0x564272035340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 0x564272035340 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c00d010 tx=0x7f4b1c0094f0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3328909602 shutdown_connections 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564272034f20 0x564272035340 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56427203a640 0x56427203aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3328909602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x5642720349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3328909602 >> 172.21.15.52:0/3328909602 conn(0x564271f5fcc0 msgr2=0x56427202f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3328909602 shutdown_connections 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3328909602 wait complete. 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 -- start start 2024-06-02T23:29:48.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.300+0000 7f4b35ab3b80 1 Processor -- start 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272040d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x56427203e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x56427203e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564272041970 con 0x56427203af60 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564271f26bf0 con 0x564272034f20 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564272041af0 con 0x56427203a640 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b35ab3b80 10 client.0 _ll_register_callbacks cb 0x564271f20920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x56427203e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x56427203e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56064/0 (socket says 172.21.15.52:56064) 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 -- 172.21.15.52:0/92921618 learned_addr learned my addr 172.21.15.52:0/92921618 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b23fff700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x56427203e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272040d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 -- 172.21.15.52:0/92921618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 msgr2=0x56427203e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x56427203e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 -- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 msgr2=0x564272040d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272040d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b297cd700 1 -- 172.21.15.52:0/92921618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564271ed58d0 con 0x56427203af60 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.301+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272040d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b297cd700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x56427203e580 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c004c60 tx=0x7f4b1c004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0b5adb74da104 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b03fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b03fff700 1 -- 172.21.15.52:0/92921618 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b1c034060 con 0x56427203af60 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b1c0074e0 con 0x56427203af60 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b03fff700 1 -- 172.21.15.52:0/92921618 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4b1c007aa0 con 0x56427203af60 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b03fff700 1 -- 172.21.15.52:0/92921618 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4b1c02b050 con 0x56427203af60 2024-06-02T23:29:48.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 10 client.20238 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56427204c320 con 0x56427203af60 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b03fff700 1 -- 172.21.15.52:0/92921618 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b1c0074e0 con 0x56427203af60 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b03fff700 1 -- 172.21.15.52:0/92921618 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4b1c011070 con 0x56427203af60 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 10 client.20238 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 10 client.20238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 20 client.20238 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x564272050dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 4 client.20238 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.302+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564271f93d10 con 0x56427204e980 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.303+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x564272050dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.303+0000 7f4b03fff700 10 client.20238 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.303+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x564272050dc0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd51d706dab43088 server_cookie=8a7588e92b0b27eb in_seq=0 out_seq=0 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b03fff700 1 -- 172.21.15.52:0/92921618 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564271f93d10 con 0x56427204e980 2024-06-02T23:29:48.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b03fff700 10 client.20238 handle_command_reply: tid=0 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 client.20238 shutdown 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 msgr2=0x56427203e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x56427203e580 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f4b1c004c60 tx=0x7f4b1c004c90 comp rx=0 tx=0).stop 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 shutdown_connections 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x564272050dc0 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x56427203e580 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272040d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b03fff700 0 client.20238 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/92921618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x56427203e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 >> 172.21.15.52:0/92921618 conn(0x564271f5fcc0 msgr2=0x5642720339d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 shutdown_connections 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/92921618 wait complete. 2024-06-02T23:29:48.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.304+0000 7f4b35ab3b80 20 client.20238 trim_cache size 0 max 16384 2024-06-02T23:29:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 Processor -- start 2024-06-02T23:29:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 -- start start 2024-06-02T23:29:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x5642720349e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 0x56427203e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x56427203bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241aeb10 con 0x56427203a640 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241b0650 con 0x564272034f20 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241af1e0 con 0x56427203af60 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 0x56427203e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 0x56427203e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56066/0 (socket says 172.21.15.52:56066) 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 -- 172.21.15.52:0/3284145224 learned_addr learned my addr 172.21.15.52:0/3284145224 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x5642720349e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b297cd700 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x56427203bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 -- 172.21.15.52:0/3284145224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 msgr2=0x56427203bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x56427203bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 -- 172.21.15.52:0/3284145224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 msgr2=0x5642720349e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x5642720349e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.320+0000 7f4b23fff700 1 -- 172.21.15.52:0/3284145224 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564272040980 con 0x56427203a640 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b23fff700 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 0x56427203e670 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4b10011fd0 tx=0x7f4b1001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c8a44d2e137de0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:48.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b0e7fc700 1 -- 172.21.15.52:0/3284145224 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100b9070 con 0x56427203a640 2024-06-02T23:29:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b0e7fc700 1 -- 172.21.15.52:0/3284145224 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b100a4820 con 0x56427203a640 2024-06-02T23:29:48.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b0e7fc700 1 -- 172.21.15.52:0/3284145224 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100a4390 con 0x56427203a640 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3284145224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 msgr2=0x56427203e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 0x56427203e670 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f4b10011fd0 tx=0x7f4b1001a040 comp rx=0 tx=0).stop 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3284145224 shutdown_connections 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203af60 0x56427203bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x5642720349e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/3284145224 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203a640 0x56427203e670 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3284145224 >> 172.21.15.52:0/3284145224 conn(0x564271f5fcc0 msgr2=0x56427203b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:48.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3284145224 shutdown_connections 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.321+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/3284145224 wait complete. 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 -- start start 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 Processor -- start 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272047430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x564272055f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x564272056450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241aff80 con 0x56427203af60 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241af8b0 con 0x564272034f20 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4b241acad0 con 0x56427203a640 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b35ab3b80 10 client.0 _ll_register_callbacks cb 0x564271f20920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b28fcc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272047430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b28fcc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272047430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55752/0 (socket says 172.21.15.52:55752) 2024-06-02T23:29:48.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b28fcc700 1 -- 172.21.15.52:0/1726879711 learned_addr learned my addr 172.21.15.52:0/1726879711 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.322+0000 7f4b297cd700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x564272056450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b23fff700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x564272055f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b28fcc700 1 -- 172.21.15.52:0/1726879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 msgr2=0x564272055f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x564272055f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b28fcc700 1 -- 172.21.15.52:0/1726879711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 msgr2=0x564272056450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x564272056450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b28fcc700 1 -- 172.21.15.52:0/1726879711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564272056b90 con 0x564272034f20 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b28fcc700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272047430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x564272051630 tx=0x7f4b180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3bdcc0f944cb722 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b03fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b03fff700 1 -- 172.21.15.52:0/1726879711 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b18027700 con 0x564272034f20 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b03fff700 1 -- 172.21.15.52:0/1726879711 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4b180278a0 con 0x564272034f20 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b100a4820 con 0x564272034f20 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b03fff700 1 -- 172.21.15.52:0/1726879711 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4b18038070 con 0x564272034f20 2024-06-02T23:29:48.328 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b35ab3b80 10 client.29576 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.323+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5642720b9660 con 0x564272034f20 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b03fff700 1 -- 172.21.15.52:0/1726879711 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4b100a4820 con 0x564272034f20 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b03fff700 1 -- 172.21.15.52:0/1726879711 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4b18010be0 con 0x564272034f20 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b35ab3b80 10 client.29576 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b35ab3b80 10 client.29576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b35ab3b80 20 client.29576 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x5642720bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b35ab3b80 4 client.29576 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564271f8e290 con 0x56427204e980 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.324+0000 7f4b23fff700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x5642720bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b23fff700 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x5642720bbd20 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0bc9c741ad41eb server_cookie=f4e4c2c7d9d1bb7 in_seq=0 out_seq=0 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b03fff700 10 client.29576 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:48.329 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b03fff700 1 -- 172.21.15.52:0/1726879711 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564271f8e290 con 0x56427204e980 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b03fff700 10 client.29576 handle_command_reply: tid=0 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 client.29576 shutdown 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 msgr2=0x564272047430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272047430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x564272051630 tx=0x7f4b180047c0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 shutdown_connections 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56427204e980 0x5642720bbd20 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56427203af60 0x564272056450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564272034f20 0x564272047430 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b03fff700 0 client.29576 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 --2- 172.21.15.52:0/1726879711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56427203a640 0x564272055f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 >> 172.21.15.52:0/1726879711 conn(0x564271f5fcc0 msgr2=0x56427203b400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 shutdown_connections 2024-06-02T23:29:48.330 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 1 -- 172.21.15.52:0/1726879711 wait complete. 2024-06-02T23:29:48.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.325+0000 7f4b35ab3b80 20 client.29576 trim_cache size 0 max 16384 2024-06-02T23:29:48.331 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:29:48.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b140a2c20 msgr2=0x7f4b140a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b140a2c20 0x7f4b140a50e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b18005fd0 tx=0x7f4b1800a5e0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 msgr2=0x7f4b241a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b241a0e80 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000e9a0 tx=0x7f4b1000ed60 comp rx=0 tx=0).stop 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 shutdown_connections 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4b140a2c20 0x7f4b140a50e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4b24108530 0x7f4b2419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b2406ac00 0x7f4b241a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 --2- 172.21.15.52:0/380999986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4b2406b5c0 0x7f4b241a0e80 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 >> 172.21.15.52:0/380999986 conn(0x7f4b24077400 msgr2=0x7f4b2410d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 shutdown_connections 2024-06-02T23:29:48.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:48.326+0000 7f4b2b230700 1 -- 172.21.15.52:0/380999986 wait complete. 2024-06-02T23:29:48.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.09 MB/sec execute 176 sec latency 4820.342 ms 2024-06-02T23:29:48.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:29:49.108 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:49.367 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:49 smithi052 ceph-mon[32340]: pgmap v969: 97 pgs: 97 active+clean; 430 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.4 MiB/s wr, 215 op/s 2024-06-02T23:29:49.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.05 MB/sec execute 177 sec latency 5820.661 ms 2024-06-02T23:29:49.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:49 smithi123 ceph-mon[35977]: pgmap v969: 97 pgs: 97 active+clean; 430 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.4 MiB/s wr, 215 op/s 2024-06-02T23:29:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:49 smithi174 ceph-mon[35994]: pgmap v969: 97 pgs: 97 active+clean; 430 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 963 KiB/s rd, 3.4 MiB/s wr, 215 op/s 2024-06-02T23:29:50.069 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:29:50.069 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:29:50.440 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:50.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 7.01 MB/sec execute 178 sec latency 6820.704 ms 2024-06-02T23:29:51.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 -- 172.21.15.52:0/139292632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 msgr2=0x7f4f1c108910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 --2- 172.21.15.52:0/139292632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c108910 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f4f18009990 tx=0x7f4f18009ca0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 -- 172.21.15.52:0/139292632 shutdown_connections 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 --2- 172.21.15.52:0/139292632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06b5e0 0x7f4f1c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 --2- 172.21.15.52:0/139292632 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c108910 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 --2- 172.21.15.52:0/139292632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c06ac20 0x7f4f1c06b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 -- 172.21.15.52:0/139292632 >> 172.21.15.52:0/139292632 conn(0x7f4f1c0782c0 msgr2=0x7f4f1c0786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 -- 172.21.15.52:0/139292632 shutdown_connections 2024-06-02T23:29:51.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 -- 172.21.15.52:0/139292632 wait complete. 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 Processor -- start 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.155+0000 7f4f2458b700 1 -- start start 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f2458b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06ac20 0x7f4f1c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f2458b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c06b5e0 0x7f4f1c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f2458b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c19f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f2458b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c112690 con 0x7f4f1c108530 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f2458b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c112510 con 0x7f4f1c06ac20 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f2458b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f1c112810 con 0x7f4f1c06b5e0 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c19f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:51.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22327700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06ac20 0x7f4f1c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c19f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56098/0 (socket says 172.21.15.52:56098) 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 -- 172.21.15.52:0/700399382 learned_addr learned my addr 172.21.15.52:0/700399382 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f21b26700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c06b5e0 0x7f4f1c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 -- 172.21.15.52:0/700399382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c06b5e0 msgr2=0x7f4f1c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c06b5e0 0x7f4f1c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 -- 172.21.15.52:0/700399382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06ac20 msgr2=0x7f4f1c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06ac20 0x7f4f1c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 -- 172.21.15.52:0/700399382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f1c19fdd0 con 0x7f4f1c108530 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22327700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06ac20 0x7f4f1c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.156+0000 7f4f22b28700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c19f690 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007ee0 tx=0x7f4f1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.157+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1400dbd0 con 0x7f4f1c108530 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.157+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f1400dd70 con 0x7f4f1c108530 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.157+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f1c1a00c0 con 0x7f4f1c108530 2024-06-02T23:29:51.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.157+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f14005990 con 0x7f4f1c108530 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.157+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f1c1a7940 con 0x7f4f1c108530 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.158+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f1c103bd0 con 0x7f4f1c108530 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.159+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f1401a020 con 0x7f4f1c108530 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.160+0000 7f4f137fe700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f080a2ca0 0x7f4f080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.160+0000 7f4f22327700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f080a2ca0 0x7f4f080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.160+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f140cedf0 con 0x7f4f1c108530 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.160+0000 7f4f22327700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f080a2ca0 0x7f4f080a5160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800b5c0 tx=0x7f4f18005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:51.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.164+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f4f14010070 con 0x7f4f1c108530 2024-06-02T23:29:51.331 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:51 smithi052 ceph-mon[32340]: pgmap v970: 97 pgs: 97 active+clean; 430 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-06-02T23:29:51.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.420+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f1c069180 con 0x7f4f1c108530 2024-06-02T23:29:51.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.422+0000 7f4f137fe700 1 -- 172.21.15.52:0/700399382 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f4f1c069180 con 0x7f4f1c108530 2024-06-02T23:29:51.422 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:29:51.422 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:29:51.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f080a2ca0 msgr2=0x7f4f080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:51.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f080a2ca0 0x7f4f080a5160 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f4f1800b5c0 tx=0x7f4f18005960 comp rx=0 tx=0).stop 2024-06-02T23:29:51.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 msgr2=0x7f4f1c19f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c19f690 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f4f14007ee0 tx=0x7f4f1400d3b0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 shutdown_connections 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f080a2ca0 0x7f4f080a5160 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f1c108530 0x7f4f1c19f690 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f1c06ac20 0x7f4f1c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 --2- 172.21.15.52:0/700399382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f1c06b5e0 0x7f4f1c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 >> 172.21.15.52:0/700399382 conn(0x7f4f1c0782c0 msgr2=0x7f4f1c10d680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 shutdown_connections 2024-06-02T23:29:51.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:51.424+0000 7f4f2458b700 1 -- 172.21.15.52:0/700399382 wait complete. 2024-06-02T23:29:51.426 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:29:51.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 6.97 MB/sec execute 179 sec latency 7820.808 ms 2024-06-02T23:29:51.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:51 smithi123 ceph-mon[35977]: pgmap v970: 97 pgs: 97 active+clean; 430 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-06-02T23:29:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:51 smithi174 ceph-mon[35994]: pgmap v970: 97 pgs: 97 active+clean; 430 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 925 KiB/s rd, 3.1 MiB/s wr, 204 op/s 2024-06-02T23:29:52.363 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:52 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/700399382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:52.398 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id eed044fb --release --await' 2024-06-02T23:29:52.398 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id eed044fb --release --await 2024-06-02T23:29:52.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 6.93 MB/sec execute 180 sec latency 8820.875 ms 2024-06-02T23:29:52.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:52 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/700399382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:52.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:52 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/700399382' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:29:52.891 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:53.371 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:53 smithi052 ceph-mon[32340]: pgmap v971: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-06-02T23:29:53.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 6.89 MB/sec execute 181 sec latency 9820.908 ms 2024-06-02T23:29:53.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-mon[35977]: pgmap v971: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-06-02T23:29:53.507 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:29:53.443+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f3 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL/ auth{4=1} v8703 ap=3 f(v212 m2024-06-02T23:29:34.148264+0000 11=11+0) n(v14 rc2024-06-02T23:29:34.148264+0000 b2861056 12=11+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:893 nref=3)) caps={15216=pAsLsXs/-@4915} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb967a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003f3:[<0x1000000033f/COREL v50>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003f3:[<0x1000000033f/COREL v8703>,<0x10000000335/~dmtmp v2073>,<0x10000000007/client3 v3370>,<0x10000000006/clients v2254>,<0x10000000005/tmp v1801>,<0x10000000003/client.0 v1764>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1740>,<0x10000000001/sv_0 v1742>,<0x10000000000/qa v1681>,<0x1/volumes v1688>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:29:34.050452+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:29:34.148264+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v212 m2024-06-02T23:29:34.148264+0000 11=11+0)","memory_value.rstat":"n(v14 rc2024-06-02T23:29:34.148264+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:29:53.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:29:53.449+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO/ auth{4=1} v8705 ap=3 f(v286 m2024-06-02T23:29:34.296787+0000 3=3+0) n(v18 rc2024-06-02T23:29:34.296787+0000 b780288 4=3+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:897 nref=3)) caps={15216=pAsLsXs/-@5172} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9645180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000046d:[<0x1000000033f/WORDPRO v58>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000046d:[<0x1000000033f/WORDPRO v8705>,<0x10000000335/~dmtmp v2073>,<0x10000000007/client3 v3370>,<0x10000000006/clients v2254>,<0x10000000005/tmp v1801>,<0x10000000003/client.0 v1764>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1740>,<0x10000000001/sv_0 v1742>,<0x10000000000/qa v1681>,<0x1/volumes v1688>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:29:34.273441+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:29:34.296787+0000 b780288 4=3+1)","memory_value.dirstat":"f(v286 m2024-06-02T23:29:34.296787+0000 3=3+0)","memory_value.rstat":"n(v18 rc2024-06-02T23:29:34.296787+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:29:53.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:29:53.455+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003eb [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL/ auth{3=1} v5781 ap=3 f(v186 m2024-06-02T23:29:23.692913+0000 11=11+0) n(v13 rc2024-06-02T23:29:23.692913+0000 b2861056 12=11+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:909 nref=3)) caps={15216=pAsLsXs/-@4581} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb964b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003eb:[<0x10000000336/COREL v50>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003eb:[<0x10000000336/COREL v5781>,<0x10000000331/~dmtmp v3611>,<0x10000000007/client1 v3374>,<0x10000000006/clients v2258>,<0x10000000005/tmp v1801>,<0x10000000003/client.0 v1764>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1740>,<0x10000000001/sv_0 v1742>,<0x10000000000/qa v1681>,<0x1/volumes v1688>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:29:23.665490+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:29:23.692913+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v186 m2024-06-02T23:29:23.692913+0000 11=11+0)","memory_value.rstat":"n(v13 rc2024-06-02T23:29:23.692913+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:29:53.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:29:53.455+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038f [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL/ auth{3=1} v5937 ap=3 f(v304 m2024-06-02T23:29:39.553106+0000 8=8+0) n(v19 rc2024-06-02T23:29:39.559804+0000 b1283356 9=8+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:910 nref=3)) caps={15216=pAsLsXs/-@4982} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000038f:[<0x10000000336/EXCEL v14>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000038f:[<0x10000000336/EXCEL v5937>,<0x10000000331/~dmtmp v3611>,<0x10000000007/client1 v3374>,<0x10000000006/clients v2258>,<0x10000000005/tmp v1801>,<0x10000000003/client.0 v1764>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1740>,<0x10000000001/sv_0 v1742>,<0x10000000000/qa v1681>,<0x1/volumes v1688>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:29:39.513560+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:29:39.559804+0000 b1283356 9=8+1)","memory_value.dirstat":"f(v304 m2024-06-02T23:29:39.553106+0000 8=8+0)","memory_value.rstat":"n(v19 rc2024-06-02T23:29:39.559804+0000 b1283356 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:29:53.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:29:53.456+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD/ auth{3=1} v5855 ap=3 f(v192 m2024-06-02T23:29:29.556953+0000 5=5+0) n(v15 rc2024-06-02T23:29:29.556953+0000 b814326 6=5+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:911 nref=3)) caps={15216=pAsLsXs/-@4572} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ca0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000036d:[<0x10000000336/WORD v8>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000036d:[<0x10000000336/WORD v5855>,<0x10000000331/~dmtmp v3611>,<0x10000000007/client1 v3374>,<0x10000000006/clients v2258>,<0x10000000005/tmp v1801>,<0x10000000003/client.0 v1764>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1740>,<0x10000000001/sv_0 v1742>,<0x10000000000/qa v1681>,<0x1/volumes v1688>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:29:29.526394+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:29:29.556953+0000 b814326 6=5+1)","memory_value.dirstat":"f(v192 m2024-06-02T23:29:29.556953+0000 5=5+0)","memory_value.rstat":"n(v15 rc2024-06-02T23:29:29.556953+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:29:53.508 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:29:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:29:53.465+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT/ auth{3=1} v5859 ap=3 f(v224 m2024-06-02T23:29:27.141100+0000 7=7+0) n(v14 rc2024-06-02T23:29:27.141100+0000 b1462582 8=7+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:916 nref=3)) caps={15216=pAsLsXs/-@4551} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b64100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003ac:[<0x10000000336/PWRPNT v214>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003ac:[<0x10000000336/PWRPNT v5859>,<0x10000000331/~dmtmp v3611>,<0x10000000007/client1 v3374>,<0x10000000006/clients v2258>,<0x10000000005/tmp v1801>,<0x10000000003/client.0 v1764>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1740>,<0x10000000001/sv_0 v1742>,<0x10000000000/qa v1681>,<0x1/volumes v1688>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:29:27.070477+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:29:27.141100+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v224 m2024-06-02T23:29:27.141100+0000 7=7+0)","memory_value.rstat":"n(v14 rc2024-06-02T23:29:27.141100+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:29:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:53 smithi174 ceph-mon[35994]: pgmap v971: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 784 KiB/s rd, 2.0 MiB/s wr, 163 op/s 2024-06-02T23:29:53.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 -- 172.21.15.52:0/2299890789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c01035f0 msgr2=0x7ff1c01039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/2299890789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c01035f0 0x7ff1c01039d0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7ff1b000b0a0 tx=0x7ff1b000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 -- 172.21.15.52:0/2299890789 shutdown_connections 2024-06-02T23:29:53.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/2299890789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c010ca20 0x7ff1c010eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/2299890789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c01035f0 0x7ff1c01039d0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/2299890789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c0103f10 0x7ff1c010c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 -- 172.21.15.52:0/2299890789 >> 172.21.15.52:0/2299890789 conn(0x7ff1c00fce40 msgr2=0x7ff1c00ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 -- 172.21.15.52:0/2299890789 shutdown_connections 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.364+0000 7ff1c69a1700 1 -- 172.21.15.52:0/2299890789 wait complete. 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.365+0000 7ff1c69a1700 1 Processor -- start 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.365+0000 7ff1c69a1700 1 -- start start 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c01035f0 0x7ff1c019e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c0103f10 0x7ff1c019ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 0x7ff1c01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0112690 con 0x7ff1c010ca20 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0112510 con 0x7ff1c01035f0 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c0112810 con 0x7ff1c0103f10 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 0x7ff1c01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 0x7ff1c01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56108/0 (socket says 172.21.15.52:56108) 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 -- 172.21.15.52:0/171113570 learned_addr learned my addr 172.21.15.52:0/171113570 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c599f700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c01035f0 0x7ff1c019e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c519e700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c0103f10 0x7ff1c019ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 -- 172.21.15.52:0/171113570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c0103f10 msgr2=0x7ff1c019ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c0103f10 0x7ff1c019ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 -- 172.21.15.52:0/171113570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c01035f0 msgr2=0x7ff1c019e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c01035f0 0x7ff1c019e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 -- 172.21.15.52:0/171113570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1c0199110 con 0x7ff1c010ca20 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 0x7ff1c01989b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800d980 tx=0x7ff1b800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1beffd700 1 -- 172.21.15.52:0/171113570 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8017070 con 0x7ff1c010ca20 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1beffd700 1 -- 172.21.15.52:0/171113570 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b8005020 con 0x7ff1c010ca20 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.366+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1c0199400 con 0x7ff1c010ca20 2024-06-02T23:29:53.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.367+0000 7ff1beffd700 1 -- 172.21.15.52:0/171113570 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b800fa90 con 0x7ff1c010ca20 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.367+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1c01a56c0 con 0x7ff1c010ca20 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.369+0000 7ff1beffd700 1 -- 172.21.15.52:0/171113570 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff1b80051d0 con 0x7ff1c010ca20 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.369+0000 7ff1beffd700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1a80a2ca0 0x7ff1a80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.369+0000 7ff1c599f700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1a80a2ca0 0x7ff1a80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.369+0000 7ff1beffd700 1 -- 172.21.15.52:0/171113570 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff1b80cecd0 con 0x7ff1c010ca20 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.370+0000 7ff1c599f700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1a80a2ca0 0x7ff1a80a5160 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0199af0 tx=0x7ff1b0006210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.370+0000 7ff1d2486b80 1 Processor -- start 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.370+0000 7ff1d2486b80 1 -- start start 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab221090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab21afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab21b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dcab168890 con 0x55dcab2215d0 2024-06-02T23:29:53.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dcab1107a0 con 0x55dcab21b510 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcab164850 con 0x55dcab220cb0 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab21afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab21afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56120/0 (socket says 172.21.15.52:56120) 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 -- 172.21.15.52:0/1461298428 learned_addr learned my addr 172.21.15.52:0/1461298428 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab221090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 -- 172.21.15.52:0/1461298428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 msgr2=0x55dcab221090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab221090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 -- 172.21.15.52:0/1461298428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 msgr2=0x55dcab21b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab21b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab21b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 -- 172.21.15.52:0/1461298428 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcab106fd0 con 0x55dcab2215d0 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c519e700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab21afd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40046c0 tx=0x7ff1b400b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776107b92903f81 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:53.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1a7fff700 1 -- 172.21.15.52:0/1461298428 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4004a30 con 0x55dcab2215d0 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab21b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1a7fff700 1 -- 172.21.15.52:0/1461298428 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b4015070 con 0x55dcab2215d0 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.371+0000 7ff1a7fff700 1 -- 172.21.15.52:0/1461298428 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4003b00 con 0x55dcab2215d0 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab221090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1461298428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 msgr2=0x55dcab21afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab21afd0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff1b40046c0 tx=0x7ff1b400b9d0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1461298428 shutdown_connections 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab21b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab221090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1461298428 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab21afd0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1461298428 >> 172.21.15.52:0/1461298428 conn(0x55dcab141d40 msgr2=0x55dcab211b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1461298428 shutdown_connections 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.372+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1461298428 wait complete. 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 -- start start 2024-06-02T23:29:53.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 Processor -- start 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab227350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab2246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab224be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dcab228070 con 0x55dcab2215d0 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dcab108c70 con 0x55dcab21b510 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dcab2281f0 con 0x55dcab220cb0 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1d2486b80 10 client.0 _ll_register_callbacks cb 0x55dcab1029a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1c61a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab224be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1c61a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab224be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56134/0 (socket says 172.21.15.52:56134) 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.373+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1771504 learned_addr learned my addr 172.21.15.52:0/1771504 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab227350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c519e700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab2246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1771504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 msgr2=0x55dcab2246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab2246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 msgr2=0x55dcab227350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab227350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1771504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcab0b7950 con 0x55dcab2215d0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab227350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab224be0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005e10 tx=0x7ff1b8005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f83dcf4362f6e91 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1771504 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b809b9c0 con 0x55dcab2215d0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1771504 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff1b80a3d40 con 0x55dcab2215d0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b4015070 con 0x55dcab2215d0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1771504 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff1b809f910 con 0x55dcab2215d0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1d2486b80 10 client.20262 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dcab232910 con 0x55dcab2215d0 2024-06-02T23:29:53.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.374+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1771504 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b4015070 con 0x55dcab2215d0 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1771504 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff1b8084af0 con 0x55dcab2215d0 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1d2486b80 10 client.20262 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1d2486b80 10 client.20262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1d2486b80 20 client.20262 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab234f50 0x55dcab237340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1d2486b80 4 client.20262 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcab175d90 con 0x55dcab234f50 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab234f50 0x55dcab237340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab234f50 0x55dcab237340 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab804dd03b241c8 server_cookie=b4487ddd355f80a5 in_seq=0 out_seq=0 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.375+0000 7ff1a57fa700 10 client.20262 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1771504 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dcab175d90 con 0x55dcab234f50 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1a57fa700 10 client.20262 handle_command_reply: tid=0 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 client.20262 shutdown 2024-06-02T23:29:53.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 msgr2=0x55dcab224be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab224be0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8005e10 tx=0x7ff1b8005680 comp rx=0 tx=0).stop 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 shutdown_connections 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab234f50 0x55dcab237340 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab224be0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab21b510 0x55dcab227350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1771504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab220cb0 0x55dcab2246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1a57fa700 0 client.20262 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 >> 172.21.15.52:0/1771504 conn(0x55dcab141d40 msgr2=0x55dcab215bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.376+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 shutdown_connections 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.377+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1771504 wait complete. 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.377+0000 7ff1d2486b80 20 client.20262 trim_cache size 0 max 16384 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.397+0000 7ff1d2486b80 1 Processor -- start 2024-06-02T23:29:53.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 -- start start 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab20b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 0x55dcab211bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab2215d0 0x55dcab222180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01aa6a0 con 0x55dcab220cb0 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01ac1e0 con 0x55dcab2215d0 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01aad70 con 0x55dcab21b510 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 0x55dcab211bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 0x55dcab211bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56140/0 (socket says 172.21.15.52:56140) 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 -- 172.21.15.52:0/729025921 learned_addr learned my addr 172.21.15.52:0/729025921 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab2215d0 0x55dcab222180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 -- 172.21.15.52:0/729025921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 msgr2=0x55dcab20b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab20b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 -- 172.21.15.52:0/729025921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab2215d0 msgr2=0x55dcab222180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab2215d0 0x55dcab222180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 -- 172.21.15.52:0/729025921 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcab226f90 con 0x55dcab220cb0 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab2215d0 0x55dcab222180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1c519e700 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 0x55dcab211bd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4018000 tx=0x7ff1b40051d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d0b2c3bbce4d9c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.398+0000 7ff1a77fe700 1 -- 172.21.15.52:0/729025921 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b40053a0 con 0x55dcab220cb0 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1a77fe700 1 -- 172.21.15.52:0/729025921 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b4015040 con 0x55dcab220cb0 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1a77fe700 1 -- 172.21.15.52:0/729025921 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b4003ba0 con 0x55dcab220cb0 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 -- 172.21.15.52:0/729025921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 msgr2=0x55dcab211bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 0x55dcab211bd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7ff1b4018000 tx=0x7ff1b40051d0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 -- 172.21.15.52:0/729025921 shutdown_connections 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab2215d0 0x55dcab222180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab20b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/729025921 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab220cb0 0x55dcab211bd0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 -- 172.21.15.52:0/729025921 >> 172.21.15.52:0/729025921 conn(0x55dcab141d40 msgr2=0x55dcab221a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 -- 172.21.15.52:0/729025921 shutdown_connections 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.399+0000 7ff1d2486b80 1 -- 172.21.15.52:0/729025921 wait complete. 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 -- start start 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 Processor -- start 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab22a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab220cb0 0x55dcab22a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab22af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01abb10 con 0x55dcab2215d0 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01ab440 con 0x55dcab220cb0 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1c01a8660 con 0x55dcab21b510 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1d2486b80 10 client.0 _ll_register_callbacks cb 0x55dcab1029a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1c61a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab22af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1c61a0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab22af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56150/0 (socket says 172.21.15.52:56150) 2024-06-02T23:29:53.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1366870304 learned_addr learned my addr 172.21.15.52:0/1366870304 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab22a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.400+0000 7ff1c519e700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab220cb0 0x55dcab22a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1366870304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 msgr2=0x55dcab22a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab22a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab220cb0 msgr2=0x55dcab22a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab220cb0 0x55dcab22a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1c61a0700 1 -- 172.21.15.52:0/1366870304 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcab29fc40 con 0x55dcab2215d0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1c61a0700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab22af00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8097f30 tx=0x7ff1b8097690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad52a51d64649201 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1a57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1366870304 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1b8014030 con 0x55dcab2215d0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1366870304 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff1b800fc30 con 0x55dcab2215d0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1b4015040 con 0x55dcab2215d0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1366870304 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff1b809b330 con 0x55dcab2215d0 2024-06-02T23:29:53.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1d2486b80 10 client.20268 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dcab29fc40 con 0x55dcab2215d0 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.401+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1366870304 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1b4015040 con 0x55dcab2215d0 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1366870304 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff1b809b7c0 con 0x55dcab2215d0 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1d2486b80 10 client.20268 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1d2486b80 10 client.20268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1d2486b80 20 client.20268 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab2a1280 0x55dcab2a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1d2486b80 4 client.20268 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55dcab170310 con 0x55dcab2a1280 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab2a1280 0x55dcab2a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1a57fa700 10 client.20268 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.402+0000 7ff1c599f700 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab2a1280 0x55dcab2a36a0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa084d2f660139c server_cookie=5a71c7a3de0e04c7 in_seq=0 out_seq=0 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1a57fa700 1 -- 172.21.15.52:0/1366870304 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55dcab170310 con 0x55dcab2a1280 2024-06-02T23:29:53.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1a57fa700 10 client.20268 handle_command_reply: tid=0 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 client.20268 shutdown 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 msgr2=0x55dcab22af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab22af00 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7ff1b8097f30 tx=0x7ff1b8097690 comp rx=0 tx=0).stop 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 shutdown_connections 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dcab2a1280 0x55dcab2a36a0 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dcab2215d0 0x55dcab22af00 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1a57fa700 0 client.20268 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dcab21b510 0x55dcab22a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 --2- 172.21.15.52:0/1366870304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcab220cb0 0x55dcab22a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:53.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 >> 172.21.15.52:0/1366870304 conn(0x55dcab141d40 msgr2=0x55dcab21f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "f4b01447-4dcb-4ecd-acce-05238b6cd4df", 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 shutdown_connections 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 1 -- 172.21.15.52:0/1366870304 wait complete. 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.404+0000 7ff1d2486b80 20 client.20268 trim_cache size 0 max 16384 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1a80a2ca0 msgr2=0x7ff1a80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:54.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1a80a2ca0 0x7ff1a80a5160 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff1c0199af0 tx=0x7ff1b0006210 comp rx=0 tx=0).stop 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 msgr2=0x7ff1c01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 0x7ff1c01989b0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800d980 tx=0x7ff1b800dc90 comp rx=0 tx=0).stop 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 shutdown_connections 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff1a80a2ca0 0x7ff1a80a5160 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff1c010ca20 0x7ff1c01989b0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1c01035f0 0x7ff1c019e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 --2- 172.21.15.52:0/171113570 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1c0103f10 0x7ff1c019ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 >> 172.21.15.52:0/171113570 conn(0x7ff1c00fce40 msgr2=0x7ff1c00fdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 shutdown_connections 2024-06-02T23:29:54.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:53.405+0000 7ff1c69a1700 1 -- 172.21.15.52:0/171113570 wait complete. 2024-06-02T23:29:54.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:29:54.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:29:54.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:29:54.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.401 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.402 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:54 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 6.86 MB/sec execute 182 sec latency 10820.984 ms 2024-06-02T23:29:54.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:29:54.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:29:54.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.509 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:54 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:29:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:54 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:29:54.656 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:29:55.149 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:55.340 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:55 smithi052 ceph-mon[32340]: pgmap v972: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-06-02T23:29:55.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35723 6.82 MB/sec execute 183 sec latency 11821.030 ms 2024-06-02T23:29:55.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:55 smithi123 ceph-mon[35977]: pgmap v972: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-06-02T23:29:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:55 smithi174 ceph-mon[35994]: pgmap v972: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 578 KiB/s rd, 1.4 MiB/s wr, 130 op/s 2024-06-02T23:29:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 -- 172.21.15.52:0/2694336815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394102ed0 msgr2=0x7f239410f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 --2- 172.21.15.52:0/2694336815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394102ed0 0x7f239410f330 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f238c00b0d0 tx=0x7f238c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 -- 172.21.15.52:0/2694336815 shutdown_connections 2024-06-02T23:29:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 --2- 172.21.15.52:0/2694336815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394102ed0 0x7f239410f330 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 --2- 172.21.15.52:0/2694336815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394108560 0x7f2394108940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 --2- 172.21.15.52:0/2694336815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2394102510 0x7f2394102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 -- 172.21.15.52:0/2694336815 >> 172.21.15.52:0/2694336815 conn(0x7f23940795e0 msgr2=0x7f23940799f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 -- 172.21.15.52:0/2694336815 shutdown_connections 2024-06-02T23:29:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.023+0000 7f239a2f8700 1 -- 172.21.15.52:0/2694336815 wait complete. 2024-06-02T23:29:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 Processor -- start 2024-06-02T23:29:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 -- start start 2024-06-02T23:29:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394102510 0x7f2394074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2394102ed0 0x7f2394074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 0x7f2394075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2394112670 con 0x7f2394108560 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23941124f0 con 0x7f2394102510 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f239a2f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23941127f0 con 0x7f2394102ed0 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f2398895700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 0x7f2394075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f2398895700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 0x7f2394075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33422/0 (socket says 172.21.15.52:33422) 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f2398895700 1 -- 172.21.15.52:0/1382896613 learned_addr learned my addr 172.21.15.52:0/1382896613 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f23937fe700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2394102ed0 0x7f2394074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.024+0000 7f2393fff700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394102510 0x7f2394074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2398895700 1 -- 172.21.15.52:0/1382896613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2394102ed0 msgr2=0x7f2394074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2398895700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2394102ed0 0x7f2394074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2398895700 1 -- 172.21.15.52:0/1382896613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394102510 msgr2=0x7f2394074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2398895700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394102510 0x7f2394074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2398895700 1 -- 172.21.15.52:0/1382896613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23941a6fb0 con 0x7f2394108560 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2393fff700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394102510 0x7f2394074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f2398895700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 0x7f2394075210 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f238c007a20 tx=0x7f238c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f23917fa700 1 -- 172.21.15.52:0/1382896613 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c01f070 con 0x7f2394108560 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f23917fa700 1 -- 172.21.15.52:0/1382896613 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f238c003c00 con 0x7f2394108560 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23941a7240 con 0x7f2394108560 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f23917fa700 1 -- 172.21.15.52:0/1382896613 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238c01b6a0 con 0x7f2394108560 2024-06-02T23:29:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.025+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23941a76f0 con 0x7f2394108560 2024-06-02T23:29:56.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.027+0000 7f23917fa700 1 -- 172.21.15.52:0/1382896613 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f238c022070 con 0x7f2394108560 2024-06-02T23:29:56.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.028+0000 7f23917fa700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f237c0a2ca0 0x7f237c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.028+0000 7f23917fa700 1 -- 172.21.15.52:0/1382896613 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f238c00f020 con 0x7f2394108560 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.028+0000 7f2393fff700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f237c0a2ca0 0x7f237c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f237c0a2ca0 0x7f237c0a5160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2384005e10 tx=0x7f2384005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 Processor -- start 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 -- start start 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f26070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5653f3e18060 con 0x5653f3f2b500 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653f3e76950 con 0x5653f3f25c50 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653f3dde560 con 0x5653f3f2be20 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.031 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33434/0 (socket says 172.21.15.52:33434) 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 -- 172.21.15.52:0/3049892269 learned_addr learned my addr 172.21.15.52:0/3049892269 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2398895700 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f26070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f23937fe700 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 -- 172.21.15.52:0/3049892269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 msgr2=0x5653f3f25710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f25710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 -- 172.21.15.52:0/3049892269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 msgr2=0x5653f3f26070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f26070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.029+0000 7f2393fff700 1 -- 172.21.15.52:0/3049892269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f3dc59d0 con 0x5653f3f2b500 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.030+0000 7f2393fff700 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2b8e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f23840044e0 tx=0x7f2384012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542a0ff1da54dc2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.030+0000 7f237a7fc700 1 -- 172.21.15.52:0/3049892269 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2384003880 con 0x5653f3f2b500 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.030+0000 7f237a7fc700 1 -- 172.21.15.52:0/3049892269 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2384024070 con 0x5653f3f2b500 2024-06-02T23:29:56.032 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.030+0000 7f237a7fc700 1 -- 172.21.15.52:0/3049892269 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238401fd60 con 0x5653f3f2b500 2024-06-02T23:29:56.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.030+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/3049892269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 msgr2=0x5653f3f2b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.030+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2b8e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f23840044e0 tx=0x7f2384012000 comp rx=0 tx=0).stop 2024-06-02T23:29:56.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/3049892269 shutdown_connections 2024-06-02T23:29:56.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f26070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.036 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2b8e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/3049892269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f25710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/3049892269 >> 172.21.15.52:0/3049892269 conn(0x5653f3e52180 msgr2=0x5653f3f20a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/3049892269 shutdown_connections 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/3049892269 wait complete. 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- start start 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 Processor -- start 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f2f300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5653f3e170c0 con 0x5653f3f2b500 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5653f3e72910 con 0x5653f3f25c50 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.031+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5653f3f32850 con 0x5653f3f2be20 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23a4b7bb80 10 client.0 _ll_register_callbacks cb 0x5653f3e0c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:56.037 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33438/0 (socket says 172.21.15.52:33438) 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 -- 172.21.15.52:0/143184137 learned_addr learned my addr 172.21.15.52:0/143184137 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f2398895700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f2f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f2393fff700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 -- 172.21.15.52:0/143184137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 msgr2=0x5653f3f2f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f2f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 -- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 msgr2=0x5653f3f31ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 -- 172.21.15.52:0/143184137 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f3e3c850 con 0x5653f3f2b500 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f2398895700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23937fe700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2edc0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f238800ba00 tx=0x7f238800bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6709c8abcaad773 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23737fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23737fe700 1 -- 172.21.15.52:0/143184137 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388004830 con 0x5653f3f2b500 2024-06-02T23:29:56.038 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23737fe700 1 -- 172.21.15.52:0/143184137 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f23880049d0 con 0x5653f3f2b500 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.032+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2384024070 con 0x5653f3f2b500 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23737fe700 1 -- 172.21.15.52:0/143184137 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2388004bd0 con 0x5653f3f2b500 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 10 client.20280 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5653f3f3cf90 con 0x5653f3f2b500 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23737fe700 1 -- 172.21.15.52:0/143184137 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2384024070 con 0x5653f3f2b500 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23737fe700 1 -- 172.21.15.52:0/143184137 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2388019030 con 0x5653f3f2b500 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 10 client.20280 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 10 client.20280 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 20 client.20280 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3f3f620 0x5653f3f41a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 4 client.20280 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f3e83e50 con 0x5653f3f3f620 2024-06-02T23:29:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.033+0000 7f2393fff700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3f3f620 0x5653f3f41a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.034+0000 7f2393fff700 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3f3f620 0x5653f3f41a10 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a44427aadc026c78 server_cookie=82749f91635fa445 in_seq=0 out_seq=0 2024-06-02T23:29:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.034+0000 7f23737fe700 10 client.20280 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23737fe700 1 -- 172.21.15.52:0/143184137 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5653f3e83e50 con 0x5653f3f3f620 2024-06-02T23:29:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23737fe700 10 client.20280 handle_command_reply: tid=0 2024-06-02T23:29:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 client.20280 shutdown 2024-06-02T23:29:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 msgr2=0x5653f3f2edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2edc0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f238800ba00 tx=0x7f238800bdc0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 shutdown_connections 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3f3f620 0x5653f3f41a10 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f2f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23737fe700 0 client.20280 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/143184137 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f2edc0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 >> 172.21.15.52:0/143184137 conn(0x5653f3e52180 msgr2=0x5653f3f1fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 shutdown_connections 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/143184137 wait complete. 2024-06-02T23:29:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.035+0000 7f23a4b7bb80 20 client.20280 trim_cache size 0 max 16384 2024-06-02T23:29:56.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 Processor -- start 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 -- start start 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f20330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f20870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f23941aec10 con 0x5653f3f2b500 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b0750 con 0x5653f3f25c50 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23941af2e0 con 0x5653f3f2be20 2024-06-02T23:29:56.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f20330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f20330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33454/0 (socket says 172.21.15.52:33454) 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 -- 172.21.15.52:0/366478574 learned_addr learned my addr 172.21.15.52:0/366478574 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f2398895700 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f20870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 -- 172.21.15.52:0/366478574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 msgr2=0x5653f3f20870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f2393fff700 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f20870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 -- 172.21.15.52:0/366478574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 msgr2=0x5653f3f31600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f23937fe700 1 -- 172.21.15.52:0/366478574 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f3f2af30 con 0x5653f3f2b500 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.056+0000 7f2393fff700 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23937fe700 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f20330 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f23880118a0 tx=0x7f2388003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d314344fda9f7821 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f2379ffb700 1 -- 172.21.15.52:0/366478574 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388003d80 con 0x5653f3f2b500 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f2379ffb700 1 -- 172.21.15.52:0/366478574 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2388011920 con 0x5653f3f2b500 2024-06-02T23:29:56.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f2379ffb700 1 -- 172.21.15.52:0/366478574 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388005230 con 0x5653f3f2b500 2024-06-02T23:29:56.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/366478574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 msgr2=0x5653f3f20330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f20330 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f23880118a0 tx=0x7f2388003960 comp rx=0 tx=0).stop 2024-06-02T23:29:56.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/366478574 shutdown_connections 2024-06-02T23:29:56.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f20870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/366478574 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f20330 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.057+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/366478574 >> 172.21.15.52:0/366478574 conn(0x5653f3e52180 msgr2=0x5653f3f2ef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:56.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/366478574 shutdown_connections 2024-06-02T23:29:56.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/366478574 wait complete. 2024-06-02T23:29:56.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 -- start start 2024-06-02T23:29:56.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 Processor -- start 2024-06-02T23:29:56.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f34b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f35090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f355d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f23941b0080 con 0x5653f3f2b500 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23941af9b0 con 0x5653f3f25c50 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23941acbd0 con 0x5653f3f2be20 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23a4b7bb80 10 client.0 _ll_register_callbacks cb 0x5653f3e0c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.058+0000 7f23937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f35090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f35090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33468/0 (socket says 172.21.15.52:33468) 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 -- 172.21.15.52:0/2495326918 learned_addr learned my addr 172.21.15.52:0/2495326918 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f2393fff700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f34b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f2398895700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f355d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 -- 172.21.15.52:0/2495326918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 msgr2=0x5653f3f355d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f355d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 -- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 msgr2=0x5653f3f34b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f34b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 -- 172.21.15.52:0/2495326918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f3faa520 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23937fe700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f35090 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f238800b740 tx=0x7f2388003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c8963ff2e14524 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23737fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23737fe700 1 -- 172.21.15.52:0/2495326918 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2388012d70 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23737fe700 1 -- 172.21.15.52:0/2495326918 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2388004eb0 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2388011920 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23737fe700 1 -- 172.21.15.52:0/2495326918 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f238803f050 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.059+0000 7f23a4b7bb80 10 client.20286 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5653f3fab7b0 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23737fe700 1 -- 172.21.15.52:0/2495326918 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2388011920 con 0x5653f3f2b500 2024-06-02T23:29:56.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23737fe700 1 -- 172.21.15.52:0/2495326918 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2388019040 con 0x5653f3f2b500 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 10 client.20286 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 10 client.20286 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 20 client.20286 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3fabe00 0x5653f3fae220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 4 client.20286 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "eed044fb", "release": true, "await": true}] 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "eed044fb", "release": true, "await": true}) -- 0x5653f3e7e3d0 con 0x5653f3fabe00 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.060+0000 7f2393fff700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3fabe00 0x5653f3fae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.061+0000 7f2393fff700 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3fabe00 0x5653f3fae220 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ea0bb0dba2ead87 server_cookie=60395159a533fb51 in_seq=0 out_seq=0 2024-06-02T23:29:56.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.061+0000 7f23737fe700 10 client.20286 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.099+0000 7f23737fe700 1 -- 172.21.15.52:0/2495326918 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5653f3e7e3d0 con 0x5653f3fabe00 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.099+0000 7f23737fe700 10 client.20286 handle_command_reply: tid=0 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 client.20286 shutdown 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 msgr2=0x5653f3f35090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f35090 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f238800b740 tx=0x7f2388003a80 comp rx=0 tx=0).stop 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 shutdown_connections 2024-06-02T23:29:56.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5653f3fabe00 0x5653f3fae220 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5653f3f2be20 0x5653f3f355d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5653f3f25c50 0x5653f3f34b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 --2- 172.21.15.52:0/2495326918 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5653f3f2b500 0x5653f3f35090 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 >> 172.21.15.52:0/2495326918 conn(0x5653f3e52180 msgr2=0x5653f3f2e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23737fe700 0 client.20286 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 shutdown_connections 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.100+0000 7f23a4b7bb80 1 -- 172.21.15.52:0/2495326918 wait complete. 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.101+0000 7f23a4b7bb80 20 client.20286 trim_cache size 0 max 16384 2024-06-02T23:29:56.102 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 60, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "eed044fb": { 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 60, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:29:56.103 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.0, 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:56.104 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:29:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.101+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f237c0a2ca0 msgr2=0x7f237c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.101+0000 7f239a2f8700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f237c0a2ca0 0x7f237c0a5160 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f2384005e10 tx=0x7f2384005d60 comp rx=0 tx=0).stop 2024-06-02T23:29:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.101+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 msgr2=0x7f2394075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 0x7f2394075210 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f238c007a20 tx=0x7f238c007a50 comp rx=0 tx=0).stop 2024-06-02T23:29:56.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 shutdown_connections 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f237c0a2ca0 0x7f237c0a5160 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2394108560 0x7f2394075210 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2394102510 0x7f2394074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 --2- 172.21.15.52:0/1382896613 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2394102ed0 0x7f2394074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 >> 172.21.15.52:0/1382896613 conn(0x7f23940795e0 msgr2=0x7f2394101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 shutdown_connections 2024-06-02T23:29:56.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:56.102+0000 7f239a2f8700 1 -- 172.21.15.52:0/1382896613 wait complete. 2024-06-02T23:29:56.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35777 6.79 MB/sec execute 184 sec latency 12450.919 ms 2024-06-02T23:29:56.952 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'eed044fb', total seconds elapsed: 35.1 2024-06-02T23:29:56.953 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.0 2024-06-02T23:29:56.953 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:29:57.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 35947 6.82 MB/sec execute 185 sec latency 69.882 ms 2024-06-02T23:29:57.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:57 smithi123 ceph-mon[35977]: pgmap v973: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-02T23:29:57.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:57 smithi174 ceph-mon[35994]: pgmap v973: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-02T23:29:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:57 smithi052 ceph-mon[32340]: pgmap v973: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 479 KiB/s rd, 1.1 MiB/s wr, 157 op/s 2024-06-02T23:29:58.460 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:29:58.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 36102 6.80 MB/sec execute 186 sec latency 84.021 ms 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.503+0000 7f5887a4b700 1 -- 172.21.15.52:0/4261339927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101d50 msgr2=0x7f588010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.504+0000 7f5887a4b700 1 --2- 172.21.15.52:0/4261339927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101d50 0x7f588010e240 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f587800d210 tx=0x7f587800d520 comp rx=0 tx=0).stop 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.504+0000 7f5887a4b700 1 -- 172.21.15.52:0/4261339927 shutdown_connections 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.504+0000 7f5887a4b700 1 --2- 172.21.15.52:0/4261339927 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101d50 0x7f588010e240 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.504+0000 7f5887a4b700 1 --2- 172.21.15.52:0/4261339927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 0x7f58801077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.504+0000 7f5887a4b700 1 --2- 172.21.15.52:0/4261339927 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101390 0x7f5880101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.504+0000 7f5887a4b700 1 -- 172.21.15.52:0/4261339927 >> 172.21.15.52:0/4261339927 conn(0x7f5880078340 msgr2=0x7f5880078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:58.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.505+0000 7f5887a4b700 1 -- 172.21.15.52:0/4261339927 shutdown_connections 2024-06-02T23:29:58.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.505+0000 7f5887a4b700 1 -- 172.21.15.52:0/4261339927 wait complete. 2024-06-02T23:29:58.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.506+0000 7f5887a4b700 1 Processor -- start 2024-06-02T23:29:58.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.506+0000 7f5887a4b700 1 -- start start 2024-06-02T23:29:58.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.506+0000 7f5887a4b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 0x7f588019c430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5887a4b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101d50 0x7f588019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5887a4b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 0x7f5880196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5887a4b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58801104e0 con 0x7f5880101d50 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5887a4b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5880110360 con 0x7f5880101390 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5887a4b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5880110660 con 0x7f5880107400 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f58857e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 0x7f588019c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f58857e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 0x7f588019c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:40746/0 (socket says 172.21.15.52:40746) 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f58857e7700 1 -- 172.21.15.52:0/406549879 learned_addr learned my addr 172.21.15.52:0/406549879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5885fe8700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 0x7f5880196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5884fe6700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101d50 0x7f588019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5885fe8700 1 -- 172.21.15.52:0/406549879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 msgr2=0x7f588019c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5885fe8700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 0x7f588019c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5885fe8700 1 -- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101d50 msgr2=0x7f588019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5885fe8700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101d50 0x7f588019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5885fe8700 1 -- 172.21.15.52:0/406549879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5880196da0 con 0x7f5880107400 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.507+0000 7f5884fe6700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101d50 0x7f588019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f58857e7700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 0x7f588019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:58.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f5885fe8700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 0x7f5880196640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f587800d210 tx=0x7f58780099d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:58.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f58767fc700 1 -- 172.21.15.52:0/406549879 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878023070 con 0x7f5880107400 2024-06-02T23:29:58.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f58767fc700 1 -- 172.21.15.52:0/406549879 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f587801fd10 con 0x7f5880107400 2024-06-02T23:29:58.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f58767fc700 1 -- 172.21.15.52:0/406549879 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58780056f0 con 0x7f5880107400 2024-06-02T23:29:58.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5880197030 con 0x7f5880107400 2024-06-02T23:29:58.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.508+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58801a3300 con 0x7f5880107400 2024-06-02T23:29:58.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.511+0000 7f58767fc700 1 -- 172.21.15.52:0/406549879 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5878026070 con 0x7f5880107400 2024-06-02T23:29:58.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.512+0000 7f58767fc700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f586c0a70b0 0x7f586c0a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.512+0000 7f58767fc700 1 -- 172.21.15.52:0/406549879 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f58780d0790 con 0x7f5880107400 2024-06-02T23:29:58.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.512+0000 7f58857e7700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f586c0a70b0 0x7f586c0a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58857e7700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f586c0a70b0 0x7f586c0a9570 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f587c009d50 tx=0x7f587c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:58.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 Processor -- start 2024-06-02T23:29:58.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 -- start start 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 0x557f31bc0340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 0x557f31bba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bba7a0 0x557f31bbabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557f31aacf60 con 0x557f31bbff60 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f31b0b850 con 0x557f31bba7a0 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58922ceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557f31a73460 con 0x557f31bc0880 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f58857e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 0x557f31bc0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f5885fe8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bba7a0 0x557f31bbabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.513+0000 7f5884fe6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 0x557f31bba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 0x557f31bba260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47690/0 (socket says 172.21.15.52:47690) 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f58857e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 0x557f31bc0340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33488/0 (socket says 172.21.15.52:33488) 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 -- 172.21.15.52:0/3632928536 learned_addr learned my addr 172.21.15.52:0/3632928536 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 -- 172.21.15.52:0/3632928536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bba7a0 msgr2=0x557f31bbabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bba7a0 0x557f31bbabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 -- 172.21.15.52:0/3632928536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 msgr2=0x557f31bc0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 0x557f31bc0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 -- 172.21.15.52:0/3632928536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f31a5a8d0 con 0x557f31bc0880 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f58857e7700 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 0x557f31bc0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f5884fe6700 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 0x557f31bba260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5870009d70 tx=0x7f58700116c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12247ecf6b712abe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f586b7fe700 1 -- 172.21.15.52:0/3632928536 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870011e20 con 0x557f31bc0880 2024-06-02T23:29:58.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.514+0000 7f586b7fe700 1 -- 172.21.15.52:0/3632928536 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5870012900 con 0x557f31bc0880 2024-06-02T23:29:58.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.515+0000 7f586b7fe700 1 -- 172.21.15.52:0/3632928536 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587000f840 con 0x557f31bc0880 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.515+0000 7f58922ceb80 1 -- 172.21.15.52:0/3632928536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 msgr2=0x557f31bba260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.515+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 0x557f31bba260 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f5870009d70 tx=0x7f58700116c0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 -- 172.21.15.52:0/3632928536 shutdown_connections 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bba7a0 0x557f31bbabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bbff60 0x557f31bc0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3632928536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bc0880 0x557f31bba260 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 -- 172.21.15.52:0/3632928536 >> 172.21.15.52:0/3632928536 conn(0x557f31ae7080 msgr2=0x557f31bb5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 -- 172.21.15.52:0/3632928536 shutdown_connections 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 -- 172.21.15.52:0/3632928536 wait complete. 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.516+0000 7f58922ceb80 1 -- start start 2024-06-02T23:29:58.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 Processor -- start 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 0x557f31bc6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bbff60 0x557f31bc38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bc0880 0x557f31bc3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58857e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 0x557f31bc6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f5884fe6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bbff60 0x557f31bc38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58857e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 0x557f31bc6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33502/0 (socket says 172.21.15.52:33502) 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58857e7700 1 -- 172.21.15.52:0/3627087862 learned_addr learned my addr 172.21.15.52:0/3627087862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557f31aabfc0 con 0x557f31bba7a0 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557f31b07810 con 0x557f31bc0880 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557f31bc72b0 con 0x557f31bbff60 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.517+0000 7f5885fe8700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bc0880 0x557f31bc3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58922ceb80 10 client.0 _ll_register_callbacks cb 0x557f31aa15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58857e7700 1 -- 172.21.15.52:0/3627087862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bbff60 msgr2=0x557f31bc38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.520 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58857e7700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bbff60 0x557f31bc38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58857e7700 1 -- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bc0880 msgr2=0x557f31bc3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58857e7700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bc0880 0x557f31bc3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58857e7700 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f31ad1750 con 0x557f31bba7a0 2024-06-02T23:29:58.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f5885fe8700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bc0880 0x557f31bc3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58857e7700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 0x557f31bc6590 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f587c011c30 tx=0x7f587c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455e7b296656e29a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f5868ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f5868ff9700 1 -- 172.21.15.52:0/3627087862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587c01f840 con 0x557f31bba7a0 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5870012900 con 0x557f31bba7a0 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.518+0000 7f5868ff9700 1 -- 172.21.15.52:0/3627087862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f587c01f9e0 con 0x557f31bba7a0 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f5868ff9700 1 -- 172.21.15.52:0/3627087862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f587c02f070 con 0x557f31bba7a0 2024-06-02T23:29:58.522 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 10 client.20298 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557f31bda080 con 0x557f31bba7a0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f5868ff9700 1 -- 172.21.15.52:0/3627087862 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5870012900 con 0x557f31bba7a0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f5868ff9700 1 -- 172.21.15.52:0/3627087862 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f587c01d3b0 con 0x557f31bba7a0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 10 client.20298 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 10 client.20298 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 20 client.20298 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31bdc9c0 0x557f31bdee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 4 client.20298 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.519+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557f31b18d50 con 0x557f31bdc9c0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.520+0000 7f5884fe6700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31bdc9c0 0x557f31bdee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.521+0000 7f5884fe6700 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31bdc9c0 0x557f31bdee00 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5926f05ab5226f server_cookie=84b8d7df73e818ca in_seq=0 out_seq=0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.521+0000 7f5868ff9700 10 client.20298 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f5868ff9700 1 -- 172.21.15.52:0/3627087862 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557f31b18d50 con 0x557f31bdc9c0 2024-06-02T23:29:58.523 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f5868ff9700 10 client.20298 handle_command_reply: tid=0 2024-06-02T23:29:58.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 client.20298 shutdown 2024-06-02T23:29:58.524 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 msgr2=0x557f31bc6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 0x557f31bc6590 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f587c011c30 tx=0x7f587c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 shutdown_connections 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31bdc9c0 0x557f31bdee00 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bc0880 0x557f31bc3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bba7a0 0x557f31bc6590 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 --2- 172.21.15.52:0/3627087862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bbff60 0x557f31bc38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 >> 172.21.15.52:0/3627087862 conn(0x557f31ae7080 msgr2=0x557f31bb4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f5868ff9700 0 client.20298 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.522+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 shutdown_connections 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.523+0000 7f58922ceb80 1 -- 172.21.15.52:0/3627087862 wait complete. 2024-06-02T23:29:58.525 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.523+0000 7f58922ceb80 20 client.20298 trim_cache size 0 max 16384 2024-06-02T23:29:58.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.553+0000 7f58922ceb80 1 Processor -- start 2024-06-02T23:29:58.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.553+0000 7f58922ceb80 1 -- start start 2024-06-02T23:29:58.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bba260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bc1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31bb4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58922ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58801aa5b0 con 0x557f31bc0880 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58922ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58801ac0f0 con 0x557f31bbff60 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58922ceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58801aac80 con 0x557f31bba7a0 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bc1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bc1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:40786/0 (socket says 172.21.15.52:40786) 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 -- 172.21.15.52:0/1232189466 learned_addr learned my addr 172.21.15.52:0/1232189466 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f58857e7700 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bba260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5885fe8700 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31bb4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 -- 172.21.15.52:0/1232189466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 msgr2=0x557f31bba260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bba260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 -- 172.21.15.52:0/1232189466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 msgr2=0x557f31bb4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31bb4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5884fe6700 1 -- 172.21.15.52:0/1232189466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557f31bc1e50 con 0x557f31bbff60 2024-06-02T23:29:58.556 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.554+0000 7f5885fe8700 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31bb4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.555+0000 7f5884fe6700 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bc1370 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f587000b010 tx=0x7f58700116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea03485452f9a674 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.555+0000 7f586affd700 1 -- 172.21.15.52:0/1232189466 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870011da0 con 0x557f31bbff60 2024-06-02T23:29:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.555+0000 7f586affd700 1 -- 172.21.15.52:0/1232189466 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5870031070 con 0x557f31bbff60 2024-06-02T23:29:58.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.555+0000 7f586affd700 1 -- 172.21.15.52:0/1232189466 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870003cc0 con 0x557f31bbff60 2024-06-02T23:29:58.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 -- 172.21.15.52:0/1232189466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 msgr2=0x557f31bc1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bc1370 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f587000b010 tx=0x7f58700116c0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 -- 172.21.15.52:0/1232189466 shutdown_connections 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31bb4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bba260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 --2- 172.21.15.52:0/1232189466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bc1370 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 -- 172.21.15.52:0/1232189466 >> 172.21.15.52:0/1232189466 conn(0x557f31ae7080 msgr2=0x557f31bc0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 -- 172.21.15.52:0/1232189466 shutdown_connections 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 -- 172.21.15.52:0/1232189466 wait complete. 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 -- start start 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.556+0000 7f58922ceb80 1 Processor -- start 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bcc570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bccab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31be3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f58801aba20 con 0x557f31bc0880 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58801ab350 con 0x557f31bbff60 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f58801a8570 con 0x557f31bba7a0 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58922ceb80 10 client.0 _ll_register_callbacks cb 0x557f31aa15b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f5885fe8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31be3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f5884fe6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f5885fe8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31be3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33518/0 (socket says 172.21.15.52:33518) 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f5885fe8700 1 -- 172.21.15.52:0/275256107 learned_addr learned my addr 172.21.15.52:0/275256107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f58857e7700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bcc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f5884fe6700 1 -- 172.21.15.52:0/275256107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 msgr2=0x557f31bcc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.557+0000 7f5884fe6700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bcc570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5884fe6700 1 -- 172.21.15.52:0/275256107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 msgr2=0x557f31be3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5884fe6700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31be3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5884fe6700 1 -- 172.21.15.52:0/275256107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557f31bcd110 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5885fe8700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31be3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5884fe6700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bccab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x557f31bc2160 tx=0x7f5870005da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74b95f0b1939da27 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5868ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5868ff9700 1 -- 172.21.15.52:0/275256107 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5870039020 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5868ff9700 1 -- 172.21.15.52:0/275256107 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5870003d10 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5870031070 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5868ff9700 1 -- 172.21.15.52:0/275256107 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f58700050b0 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f58922ceb80 10 client.29627 fetch_fsmap learned FSMap version 43 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557f31c46c60 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.558+0000 7f5868ff9700 1 -- 172.21.15.52:0/275256107 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5870012670 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f5868ff9700 1 -- 172.21.15.52:0/275256107 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5870035410 con 0x557f31bbff60 2024-06-02T23:29:58.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58922ceb80 10 client.29627 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:29:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58922ceb80 10 client.29627 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:29:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58922ceb80 20 client.29627 populate_metadata read hostname 'smithi052' 2024-06-02T23:29:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58922ceb80 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31c470c0 0x557f31c494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:29:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58922ceb80 4 client.29627 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:29:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f5870031070 con 0x557f31c470c0 2024-06-02T23:29:58.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.559+0000 7f58857e7700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31c470c0 0x557f31c494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f58857e7700 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31c470c0 0x557f31c494b0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce5398206b3883d server_cookie=94998d3d87ab2d0 in_seq=0 out_seq=0 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f5868ff9700 10 client.29627 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f5868ff9700 1 -- 172.21.15.52:0/275256107 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f5870031070 con 0x557f31c470c0 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f5868ff9700 10 client.29627 handle_command_reply: tid=0 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f58922ceb80 1 client.29627 shutdown 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 msgr2=0x557f31bccab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.560+0000 7f58922ceb80 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bccab0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x557f31bc2160 tx=0x7f5870005da0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 shutdown_connections 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557f31c470c0 0x557f31c494b0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557f31bc0880 0x557f31be3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557f31bba7a0 0x557f31bcc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 --2- 172.21.15.52:0/275256107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557f31bbff60 0x557f31bccab0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f5868ff9700 0 client.29627 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 >> 172.21.15.52:0/275256107 conn(0x557f31ae7080 msgr2=0x557f31bc50c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 shutdown_connections 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 1 -- 172.21.15.52:0/275256107 wait complete. 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.561+0000 7f58922ceb80 20 client.29627 trim_cache size 0 max 16384 2024-06-02T23:29:58.565 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: "f4b01447-4dcb-4ecd-acce-05238b6cd4df": { 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "f4b01447-4dcb-4ecd-acce-05238b6cd4df", 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:29:58.566 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:29:58.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f586c0a70b0 msgr2=0x7f586c0a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f586c0a70b0 0x7f586c0a9570 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f587c009d50 tx=0x7f587c009260 comp rx=0 tx=0).stop 2024-06-02T23:29:58.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 msgr2=0x7f5880196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:29:58.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 0x7f5880196640 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f587800d210 tx=0x7f58780099d0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 shutdown_connections 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f586c0a70b0 0x7f586c0a9570 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5880107400 0x7f5880196640 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5880101390 0x7f588019c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 --2- 172.21.15.52:0/406549879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5880101d50 0x7f588019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 >> 172.21.15.52:0/406549879 conn(0x7f5880078340 msgr2=0x7f58800ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 shutdown_connections 2024-06-02T23:29:58.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:29:58.562+0000 7f5887a4b700 1 -- 172.21.15.52:0/406549879 wait complete. 2024-06-02T23:29:59.426 INFO:tasks.cephfs.filesystem:scrub status for tag:f4b01447-4dcb-4ecd-acce-05238b6cd4df - {'path': '/', 'tag': 'f4b01447-4dcb-4ecd-acce-05238b6cd4df', 'options': 'recursive,force'} 2024-06-02T23:29:59.462 INFO:tasks.workunit.client.0.smithi052.stdout: 10 36268 6.80 MB/sec execute 187 sec latency 71.002 ms 2024-06-02T23:29:59.490 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:29:59 smithi052 ceph-mon[32340]: pgmap v974: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 170 KiB/s wr, 88 op/s 2024-06-02T23:29:59.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:29:59 smithi123 ceph-mon[35977]: pgmap v974: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 170 KiB/s wr, 88 op/s 2024-06-02T23:29:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:29:59 smithi174 ceph-mon[35994]: pgmap v974: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 170 KiB/s wr, 88 op/s 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 -- 172.21.15.52:0/1501372440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c101af0 msgr2=0x7f5d8c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 --2- 172.21.15.52:0/1501372440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c101af0 0x7f5d8c101f70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009860 tx=0x7f5d8002a340 comp rx=0 tx=0).stop 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 -- 172.21.15.52:0/1501372440 shutdown_connections 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 --2- 172.21.15.52:0/1501372440 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c1024b0 0x7f5d8c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 --2- 172.21.15.52:0/1501372440 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c101af0 0x7f5d8c101f70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 --2- 172.21.15.52:0/1501372440 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 0x7f5d8c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 -- 172.21.15.52:0/1501372440 >> 172.21.15.52:0/1501372440 conn(0x7f5d8c0fbe70 msgr2=0x7f5d8c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 -- 172.21.15.52:0/1501372440 shutdown_connections 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.293+0000 7f5d91db7640 1 -- 172.21.15.52:0/1501372440 wait complete. 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.294+0000 7f5d91db7640 1 Processor -- start 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.294+0000 7f5d91db7640 1 -- start start 2024-06-02T23:30:00.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d91db7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 0x7f5d8c199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d91db7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c101af0 0x7f5d8c19a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d91db7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c1024b0 0x7f5d8c1a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d91db7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c10d550 con 0x7f5d8c101af0 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d91db7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c10d3d0 con 0x7f5d8c1024b0 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d91db7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c10d6d0 con 0x7f5d8c100900 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c101af0 0x7f5d8c19a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 0x7f5d8c199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c101af0 0x7f5d8c19a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33544/0 (socket says 172.21.15.52:33544) 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8affd640 1 -- 172.21.15.52:0/3232688463 learned_addr learned my addr 172.21.15.52:0/3232688463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c1024b0 0x7f5d8c1a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3232688463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c1024b0 msgr2=0x7f5d8c1a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c1024b0 0x7f5d8c1a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c101af0 msgr2=0x7f5d8c19a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c101af0 0x7f5d8c19a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.295+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3232688463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d8c1a19c0 con 0x7f5d8c100900 2024-06-02T23:30:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.296+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c1024b0 0x7f5d8c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.296+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 0x7f5d8c199db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e9c0 tx=0x7f5d7c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.296+0000 7f5d88ff9640 1 -- 172.21.15.52:0/3232688463 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00ccc0 con 0x7f5d8c100900 2024-06-02T23:30:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.296+0000 7f5d88ff9640 1 -- 172.21.15.52:0/3232688463 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7c0035f0 con 0x7f5d8c100900 2024-06-02T23:30:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.296+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d8c1a1cb0 con 0x7f5d8c100900 2024-06-02T23:30:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.297+0000 7f5d88ff9640 1 -- 172.21.15.52:0/3232688463 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c00ce60 con 0x7f5d8c100900 2024-06-02T23:30:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.297+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d8c1a21b0 con 0x7f5d8c100900 2024-06-02T23:30:00.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.300+0000 7f5d88ff9640 1 -- 172.21.15.52:0/3232688463 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5d7c010690 con 0x7f5d8c100900 2024-06-02T23:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.301+0000 7f5d88ff9640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d740a2da0 0x7f5d740a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.301+0000 7f5d88ff9640 1 -- 172.21.15.52:0/3232688463 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5d7c0cf3c0 con 0x7f5d8c100900 2024-06-02T23:30:00.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 Processor -- start 2024-06-02T23:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 -- start start 2024-06-02T23:30:00.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d8affd640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d740a2da0 0x7f5d740a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 0x55febea8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8ba00 0x55febea90500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea90b70 0x55febea93000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55febe9ee4f0 con 0x55febea8b010 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55febe98c870 con 0x55febea90b70 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55febe9ea4b0 con 0x55febea8ba00 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 0x55febea8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 0x55febea8b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33556/0 (socket says 172.21.15.52:33556) 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.302+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3567298246 learned_addr learned my addr 172.21.15.52:0/3567298246 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea90b70 0x55febea93000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8affd640 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8ba00 0x55febea90500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3567298246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8ba00 msgr2=0x55febea90500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8ba00 0x55febea90500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3567298246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea90b70 msgr2=0x55febea93000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea90b70 0x55febea93000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3567298246 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55febe9a89b0 con 0x55febea8b010 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 0x55febea8b430 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c098e30 tx=0x7f5d7c09c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be16b17c8f74fb9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d8affd640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d740a2da0 0x7f5d740a5260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d8002a930 tx=0x7f5d80030da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d71ffb640 1 -- 172.21.15.52:0/3567298246 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b4070 con 0x55febea8b010 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.303+0000 7f5d71ffb640 1 -- 172.21.15.52:0/3567298246 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7c09d160 con 0x55febea8b010 2024-06-02T23:30:00.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.304+0000 7f5d71ffb640 1 -- 172.21.15.52:0/3567298246 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0037a0 con 0x55febea8b010 2024-06-02T23:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.304+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3567298246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 msgr2=0x55febea8b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.304+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 0x55febea8b430 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c098e30 tx=0x7f5d7c09c3c0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3567298246 shutdown_connections 2024-06-02T23:30:00.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8ba00 0x55febea90500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea90b70 0x55febea93000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3567298246 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8b010 0x55febea8b430 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3567298246 >> 172.21.15.52:0/3567298246 conn(0x55febea86680 msgr2=0x55febea86a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3567298246 shutdown_connections 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3567298246 wait complete. 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 -- start start 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.305+0000 7f5d94ba2740 1 Processor -- start 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea95260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea957a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea9c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55febea93b60 con 0x55febea8ba00 2024-06-02T23:30:00.312 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55febe977ae0 con 0x55febea8b010 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55febea93ce0 con 0x55febea90b70 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d94ba2740 10 client.0 _ll_register_callbacks cb 0x55febe9b09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33572/0 (socket says 172.21.15.52:33572) 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 -- 172.21.15.52:0/3247393885 learned_addr learned my addr 172.21.15.52:0/3247393885 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea95260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea9c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 -- 172.21.15.52:0/3247393885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 msgr2=0x55febea9c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea9c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.313 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 -- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 msgr2=0x55febea95260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea95260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8affd640 1 -- 172.21.15.52:0/3247393885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55febe702210 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.306+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d8affd640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea957a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009980 tx=0x7f5d8002a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3821e979144724da server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d6affd640 1 -- 172.21.15.52:0/3247393885 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80037040 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d6affd640 1 -- 172.21.15.52:0/3247393885 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5d80003ac0 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7c09d160 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d6affd640 1 -- 172.21.15.52:0/3247393885 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5d80003cc0 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d94ba2740 10 client.20316 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.307+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55febe7ce620 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d6affd640 1 -- 172.21.15.52:0/3247393885 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7c09d160 con 0x55febea8ba00 2024-06-02T23:30:00.314 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d6affd640 1 -- 172.21.15.52:0/3247393885 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5d8004a070 con 0x55febea8ba00 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d94ba2740 10 client.20316 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d94ba2740 10 client.20316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d94ba2740 20 client.20316 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeaa35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d94ba2740 4 client.20316 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55febea01f10 con 0x55febeaa1110 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.308+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeaa35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.309+0000 7f5d6affd640 10 client.20316 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.309+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeaa35a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a85c2981d40b3a1 server_cookie=d8a1820f4440f5ae in_seq=0 out_seq=0 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d6affd640 1 -- 172.21.15.52:0/3247393885 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55febea01f10 con 0x55febeaa1110 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d6affd640 10 client.20316 handle_command_reply: tid=0 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 client.20316 shutdown 2024-06-02T23:30:00.315 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 msgr2=0x55febea957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea957a0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009980 tx=0x7f5d8002a340 comp rx=0 tx=0).stop 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 shutdown_connections 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeaa35a0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d6affd640 0 client.20316 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea9c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea957a0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3247393885 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 >> 172.21.15.52:0/3247393885 conn(0x55febea86680 msgr2=0x55febea857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 shutdown_connections 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3247393885 wait complete. 2024-06-02T23:30:00.316 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.310+0000 7f5d94ba2740 20 client.20316 trim_cache size 0 max 16384 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 Processor -- start 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 -- start start 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea90250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea9c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea89590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1aae90 con 0x55febea8ba00 2024-06-02T23:30:00.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1ac9d0 con 0x55febea8b010 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.336+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1ab560 con 0x55febea90b70 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea9c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea90250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea90250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:40844/0 (socket says 172.21.15.52:40844) 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3130917641 learned_addr learned my addr 172.21.15.52:0/3130917641 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea89590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3130917641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 msgr2=0x55febea89590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea89590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3130917641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 msgr2=0x55febea9c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea9c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 -- 172.21.15.52:0/3130917641 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55febea93df0 con 0x55febea8b010 2024-06-02T23:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.337+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea90250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0993a0 tx=0x7f5d7c09c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61b66ad19765c188 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.338+0000 7f5d717fa640 1 -- 172.21.15.52:0/3130917641 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c0b4070 con 0x55febea8b010 2024-06-02T23:30:00.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.338+0000 7f5d717fa640 1 -- 172.21.15.52:0/3130917641 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7c09d160 con 0x55febea8b010 2024-06-02T23:30:00.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.338+0000 7f5d717fa640 1 -- 172.21.15.52:0/3130917641 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7c098720 con 0x55febea8b010 2024-06-02T23:30:00.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.338+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3130917641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 msgr2=0x55febea90250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.338+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea90250 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c0993a0 tx=0x7f5d7c09c4f0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3130917641 shutdown_connections 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea90b70 0x55febea89590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea8ba00 0x55febea9c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/3130917641 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8b010 0x55febea90250 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3130917641 >> 172.21.15.52:0/3130917641 conn(0x55febea86680 msgr2=0x55febea8cbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3130917641 shutdown_connections 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 -- 172.21.15.52:0/3130917641 wait complete. 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 -- start start 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.339+0000 7f5d94ba2740 1 Processor -- start 2024-06-02T23:30:00.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8b010 0x55febea99730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8ba00 0x55febea99c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 0x55febeb09e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1ac300 con 0x55febea90b70 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1abc30 con 0x55febea8ba00 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5d8c1a8e70 con 0x55febea8b010 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d94ba2740 10 client.0 _ll_register_callbacks cb 0x7f5d84009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8b7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8b010 0x55febea99730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 0x55febeb09e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8ba00 0x55febea99c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 0x55febeb09e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:33592/0 (socket says 172.21.15.52:33592) 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8bfff640 1 -- 172.21.15.52:0/2873043545 learned_addr learned my addr 172.21.15.52:0/2873043545 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:00.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8affd640 1 -- 172.21.15.52:0/2873043545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8b010 msgr2=0x55febea99730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8affd640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8b010 0x55febea99730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8affd640 1 -- 172.21.15.52:0/2873043545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 msgr2=0x55febeb09e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8affd640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 0x55febeb09e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8affd640 1 -- 172.21.15.52:0/2873043545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55febeb0a3b0 con 0x55febea8ba00 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8b010 0x55febea99730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.340+0000 7f5d8bfff640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 0x55febeb09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d8affd640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8ba00 0x55febea99c70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009830 tx=0x7f5d800096b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6c5f488a15e5838 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d6affd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d6affd640 1 -- 172.21.15.52:0/2873043545 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d80003e60 con 0x55febea8ba00 2024-06-02T23:30:00.347 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d6affd640 1 -- 172.21.15.52:0/2873043545 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5d80005040 con 0x55febea8ba00 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d7c09d160 con 0x55febea8ba00 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d6affd640 1 -- 172.21.15.52:0/2873043545 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5d80005240 con 0x55febea8ba00 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d94ba2740 10 client.29645 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55febeb0b600 con 0x55febea8ba00 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.341+0000 7f5d6affd640 1 -- 172.21.15.52:0/2873043545 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5d7c09d160 con 0x55febea8ba00 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d6affd640 1 -- 172.21.15.52:0/2873043545 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5d8004a070 con 0x55febea8ba00 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d94ba2740 10 client.29645 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d94ba2740 10 client.29645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d94ba2740 20 client.29645 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeb0e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d94ba2740 4 client.29645 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55febe9fbc20 con 0x55febeaa1110 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeb0e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d6affd640 10 client.29645 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:00.348 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.342+0000 7f5d8b7fe640 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeb0e010 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ae3e10a423ccea server_cookie=f4a48659e852b118 in_seq=0 out_seq=0 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d6affd640 1 -- 172.21.15.52:0/2873043545 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55febe9fbc20 con 0x55febeaa1110 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d6affd640 10 client.29645 handle_command_reply: tid=0 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 client.29645 shutdown 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8ba00 msgr2=0x55febea99c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8ba00 0x55febea99c70 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5d80009830 tx=0x7f5d800096b0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 shutdown_connections 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55febeaa1110 0x55febeb0e010 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55febea90b70 0x55febeb09e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d6affd640 0 client.29645 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55febea8ba00 0x55febea99c70 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 --2- 172.21.15.52:0/2873043545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55febea8b010 0x55febea99730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.393+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 >> 172.21.15.52:0/2873043545 conn(0x55febea86680 msgr2=0x55febea95220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.394+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 shutdown_connections 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.394+0000 7f5d94ba2740 1 -- 172.21.15.52:0/2873043545 wait complete. 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.394+0000 7f5d94ba2740 20 client.29645 trim_cache size 0 max 16384 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:30:00.397 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:30:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.394+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d740a2da0 msgr2=0x7f5d740a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.394+0000 7f5d91db7640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d740a2da0 0x7f5d740a5260 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5d8002a930 tx=0x7f5d80030da0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 msgr2=0x7f5d8c199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 0x7f5d8c199db0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5d7c00e9c0 tx=0x7f5d7c00ee80 comp rx=0 tx=0).stop 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 shutdown_connections 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5d740a2da0 0x7f5d740a5260 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d8c1024b0 0x7f5d8c1a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5d8c101af0 0x7f5d8c19a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 --2- 172.21.15.52:0/3232688463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5d8c100900 0x7f5d8c199db0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 >> 172.21.15.52:0/3232688463 conn(0x7f5d8c0fbe70 msgr2=0x7f5d8c109700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 shutdown_connections 2024-06-02T23:30:00.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:00.395+0000 7f5d91db7640 1 -- 172.21.15.52:0/3232688463 wait complete. 2024-06-02T23:30:00.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:00 smithi052 ceph-mon[32340]: overall HEALTH_OK 2024-06-02T23:30:00.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 36441 6.82 MB/sec execute 188 sec latency 72.682 ms 2024-06-02T23:30:00.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:00 smithi123 ceph-mon[35977]: overall HEALTH_OK 2024-06-02T23:30:00.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:00 smithi174 ceph-mon[35994]: overall HEALTH_OK 2024-06-02T23:30:01.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 -- 172.21.15.52:0/2004077623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 msgr2=0x7f549c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/2004077623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c108f40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5488009990 tx=0x7f5488009ca0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 -- 172.21.15.52:0/2004077623 shutdown_connections 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/2004077623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f549c109480 0x7f549c115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/2004077623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f549c10eac0 0x7f549c10eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/2004077623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c108f40 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.101+0000 7f54a3c5c700 1 -- 172.21.15.52:0/2004077623 >> 172.21.15.52:0/2004077623 conn(0x7f549c075c60 msgr2=0x7f549c078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.102+0000 7f54a3c5c700 1 -- 172.21.15.52:0/2004077623 shutdown_connections 2024-06-02T23:30:01.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.102+0000 7f54a3c5c700 1 -- 172.21.15.52:0/2004077623 wait complete. 2024-06-02T23:30:01.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 Processor -- start 2024-06-02T23:30:01.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 -- start start 2024-06-02T23:30:01.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:01.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f549c109480 0x7f549c19f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:01.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f549c10eac0 0x7f549c1a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f549c118cc0 con 0x7f549c109480 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f549c118b40 con 0x7f549c108ac0 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a3c5c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f549c118e40 con 0x7f549c10eac0 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a19f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a19f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:40888/0 (socket says 172.21.15.52:40888) 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.103+0000 7f54a19f8700 1 -- 172.21.15.52:0/1115748086 learned_addr learned my addr 172.21.15.52:0/1115748086 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a11f7700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f549c109480 0x7f549c19f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a21f9700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f549c10eac0 0x7f549c1a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a19f8700 1 -- 172.21.15.52:0/1115748086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f549c10eac0 msgr2=0x7f549c1a3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a19f8700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f549c10eac0 0x7f549c1a3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a19f8700 1 -- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f549c109480 msgr2=0x7f549c19f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a19f8700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f549c109480 0x7f549c19f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a19f8700 1 -- 172.21.15.52:0/1115748086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f549c1a3f40 con 0x7f549c108ac0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f54a19f8700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c19ef30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5490007f10 tx=0x7f549000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.104+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549000ddd0 con 0x7f549c108ac0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.105+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5490004bb0 con 0x7f549c108ac0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.105+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f549c1a4230 con 0x7f549c108ac0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.105+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54900046e0 con 0x7f549c108ac0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.105+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f549c1abab0 con 0x7f549c108ac0 2024-06-02T23:30:01.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.106+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f549c069180 con 0x7f549c108ac0 2024-06-02T23:30:01.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.107+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f549001c020 con 0x7f549c108ac0 2024-06-02T23:30:01.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.108+0000 7f549affd700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f548c0a2ca0 0x7f548c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:01.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.109+0000 7f54a11f7700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f548c0a2ca0 0x7f548c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:01.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.109+0000 7f54a11f7700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f548c0a2ca0 0x7f548c0a5160 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f549c10de50 tx=0x7f5488005c80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:01.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.109+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f54900ce860 con 0x7f549c108ac0 2024-06-02T23:30:01.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.114+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5490010070 con 0x7f549c108ac0 2024-06-02T23:30:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:01 smithi052 ceph-mon[32340]: pgmap v975: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 556 KiB/s wr, 112 op/s 2024-06-02T23:30:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:30:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:30:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:30:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:30:01.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:01 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:30:01.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 36610 6.82 MB/sec execute 189 sec latency 95.188 ms 2024-06-02T23:30:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:01 smithi123 ceph-mon[35977]: pgmap v975: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 556 KiB/s wr, 112 op/s 2024-06-02T23:30:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:30:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:30:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:30:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:30:01.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:01 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:30:01.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.546+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f549c1abc80 con 0x7f549c108ac0 2024-06-02T23:30:01.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.546+0000 7f549affd700 1 -- 172.21.15.52:0/1115748086 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f549009b790 con 0x7f549c108ac0 2024-06-02T23:30:01.547 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:30:01.547 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:30:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.549+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f548c0a2ca0 msgr2=0x7f548c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:01.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.549+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f548c0a2ca0 0x7f548c0a5160 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f549c10de50 tx=0x7f5488005c80 comp rx=0 tx=0).stop 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 msgr2=0x7f549c19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c19ef30 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5490007f10 tx=0x7f549000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 shutdown_connections 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f548c0a2ca0 0x7f548c0a5160 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f549c10eac0 0x7f549c1a3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f549c108ac0 0x7f549c19ef30 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 --2- 172.21.15.52:0/1115748086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f549c109480 0x7f549c19f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 >> 172.21.15.52:0/1115748086 conn(0x7f549c075c60 msgr2=0x7f549c0760a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 shutdown_connections 2024-06-02T23:30:01.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:01.550+0000 7f54a3c5c700 1 -- 172.21.15.52:0/1115748086 wait complete. 2024-06-02T23:30:01.552 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:30:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:01 smithi174 ceph-mon[35994]: pgmap v975: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 556 KiB/s wr, 112 op/s 2024-06-02T23:30:01.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:30:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:30:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:30:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:30:01.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:01 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:30:02.210 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 147.0 --await-for 45.0' 2024-06-02T23:30:02.211 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 147.0 --await-for 45.0 2024-06-02T23:30:02.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 36769 6.84 MB/sec execute 190 sec latency 67.406 ms 2024-06-02T23:30:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:02 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1115748086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:02.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:02 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:30:02.634 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:02 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1115748086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:02.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:02 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:30:02.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:02 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1115748086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:02.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:02 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:30:03.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:03 smithi052 ceph-mon[32340]: pgmap v976: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1010 KiB/s wr, 168 op/s 2024-06-02T23:30:03.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 36902 6.82 MB/sec execute 191 sec latency 84.746 ms 2024-06-02T23:30:03.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:03 smithi123 ceph-mon[35977]: pgmap v976: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1010 KiB/s wr, 168 op/s 2024-06-02T23:30:03.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:03 smithi174 ceph-mon[35994]: pgmap v976: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 347 KiB/s rd, 1010 KiB/s wr, 168 op/s 2024-06-02T23:30:04.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37077 6.82 MB/sec execute 192 sec latency 83.278 ms 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 -- 172.21.15.52:0/712228844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 msgr2=0x7ff790107850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 --2- 172.21.15.52:0/712228844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790107850 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b0a0 tx=0x7ff78000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 -- 172.21.15.52:0/712228844 shutdown_connections 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 --2- 172.21.15.52:0/712228844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff790101d80 0x7ff79006aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 --2- 172.21.15.52:0/712228844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790107850 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 --2- 172.21.15.52:0/712228844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7901013c0 0x7ff790101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 -- 172.21.15.52:0/712228844 >> 172.21.15.52:0/712228844 conn(0x7ff7900783d0 msgr2=0x7ff7900787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 -- 172.21.15.52:0/712228844 shutdown_connections 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.869+0000 7ff79690c700 1 -- 172.21.15.52:0/712228844 wait complete. 2024-06-02T23:30:04.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.870+0000 7ff79690c700 1 Processor -- start 2024-06-02T23:30:04.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.870+0000 7ff79690c700 1 -- start start 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff79690c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7901013c0 0x7ff790114010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff79690c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff790101d80 0x7ff7901110c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff79690c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790111600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff79690c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901149c0 con 0x7ff790107470 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff79690c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff790114840 con 0x7ff790101d80 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff79690c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff790114b40 con 0x7ff7901013c0 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790111600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790111600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36706/0 (socket says 172.21.15.52:36706) 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 -- 172.21.15.52:0/3893680408 learned_addr learned my addr 172.21.15.52:0/3893680408 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff7877fe700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff790101d80 0x7ff7901110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7901013c0 0x7ff790114010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 -- 172.21.15.52:0/3893680408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7901013c0 msgr2=0x7ff790114010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7901013c0 0x7ff790114010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 -- 172.21.15.52:0/3893680408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff790101d80 msgr2=0x7ff7901110c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff790101d80 0x7ff7901110c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff794ea9700 1 -- 172.21.15.52:0/3893680408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff790111dc0 con 0x7ff790107470 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.871+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7901013c0 0x7ff790114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.872+0000 7ff794ea9700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790111600 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff78800ea80 tx=0x7ff78800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.872+0000 7ff78dffb700 1 -- 172.21.15.52:0/3893680408 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788004220 con 0x7ff790107470 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.872+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7901ab670 con 0x7ff790107470 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.872+0000 7ff78dffb700 1 -- 172.21.15.52:0/3893680408 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff78800ce00 con 0x7ff790107470 2024-06-02T23:30:04.874 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.872+0000 7ff78dffb700 1 -- 172.21.15.52:0/3893680408 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff788003dd0 con 0x7ff790107470 2024-06-02T23:30:04.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.872+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7901abbb0 con 0x7ff790107470 2024-06-02T23:30:04.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.876+0000 7ff78dffb700 1 -- 172.21.15.52:0/3893680408 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff78800c950 con 0x7ff790107470 2024-06-02T23:30:04.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.877+0000 7ff78dffb700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff77c0a2f20 0x7ff77c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.877+0000 7ff78dffb700 1 -- 172.21.15.52:0/3893680408 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff788003b70 con 0x7ff790107470 2024-06-02T23:30:04.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.877+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff77c0a2f20 0x7ff77c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.877+0000 7ff7a118fb80 1 Processor -- start 2024-06-02T23:30:04.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.877+0000 7ff7a118fb80 1 -- start start 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff77c0a2f20 0x7ff77c0a53e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff7800060b0 tx=0x7ff780006000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3030b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2fcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac2fd460 0x55e4ac2fd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4ac24e9a0 con 0x55e4ac3035f0 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4ac1f68b0 con 0x55e4ac302cd0 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e4ac24a960 con 0x55e4ac2fd460 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2fcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37934/0 (socket says 172.21.15.52:37934) 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2fcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36718/0 (socket says 172.21.15.52:36718) 2024-06-02T23:30:04.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff78ffff700 1 -- 172.21.15.52:0/4053691837 learned_addr learned my addr 172.21.15.52:0/4053691837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:04.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff794ea9700 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac2fd460 0x55e4ac2fd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.878+0000 7ff78ffff700 1 -- 172.21.15.52:0/4053691837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac2fd460 msgr2=0x55e4ac2fd8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff78ffff700 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac2fd460 0x55e4ac2fd8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff78ffff700 1 -- 172.21.15.52:0/4053691837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 msgr2=0x55e4ac2fcf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff78ffff700 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2fcf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff78ffff700 1 -- 172.21.15.52:0/4053691837 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ac1ed0e0 con 0x55e4ac302cd0 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff7877fe700 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:04.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff78ffff700 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3030b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b0a0 tx=0x7ff78001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37064bb3a08f7820 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff7867fc700 1 -- 172.21.15.52:0/4053691837 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78002b070 con 0x55e4ac302cd0 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.879+0000 7ff7867fc700 1 -- 172.21.15.52:0/4053691837 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff780004d50 con 0x55e4ac302cd0 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7867fc700 1 -- 172.21.15.52:0/4053691837 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff780030860 con 0x55e4ac302cd0 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 -- 172.21.15.52:0/4053691837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 msgr2=0x55e4ac3030b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3030b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b0a0 tx=0x7ff78001c000 comp rx=0 tx=0).stop 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 -- 172.21.15.52:0/4053691837 shutdown_connections 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac2fd460 0x55e4ac2fd8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3030b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/4053691837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2fcf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 -- 172.21.15.52:0/4053691837 >> 172.21.15.52:0/4053691837 conn(0x55e4ac227e50 msgr2=0x55e4ac2f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 -- 172.21.15.52:0/4053691837 shutdown_connections 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.880+0000 7ff7a118fb80 1 -- 172.21.15.52:0/4053691837 wait complete. 2024-06-02T23:30:04.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 -- start start 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 Processor -- start 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 0x55e4ac3092d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3065d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac3035f0 0x55e4ac306b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e4ac30a0a0 con 0x55e4ac2fd460 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4ac1eed80 con 0x55e4ac302cd0 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e4ac30a220 con 0x55e4ac3035f0 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.881+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 0x55e4ac3092d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 0x55e4ac3092d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36732/0 (socket says 172.21.15.52:36732) 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 -- 172.21.15.52:0/3335771889 learned_addr learned my addr 172.21.15.52:0/3335771889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:04.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff7877fe700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3065d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff794ea9700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac3035f0 0x55e4ac306b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff7a118fb80 10 client.0 _ll_register_callbacks cb 0x55e4ac1e8ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 -- 172.21.15.52:0/3335771889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac3035f0 msgr2=0x55e4ac306b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac3035f0 0x55e4ac306b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 -- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 msgr2=0x55e4ac3065d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3065d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 -- 172.21.15.52:0/3335771889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ac19da60 con 0x55e4ac2fd460 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 0x55e4ac3092d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b070 tx=0x7ff780004f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b9a5ff1f9dd9bb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.882+0000 7ff76f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3335771889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78002b050 con 0x55e4ac2fd460 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3335771889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff780030460 con 0x55e4ac2fd460 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff780004d50 con 0x55e4ac2fd460 2024-06-02T23:30:04.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3335771889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff780030660 con 0x55e4ac2fd460 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff7a118fb80 10 client.20340 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e4ac3148c0 con 0x55e4ac2fd460 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.883+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3335771889 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff780004d50 con 0x55e4ac2fd460 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3335771889 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff780053050 con 0x55e4ac2fd460 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7a118fb80 10 client.20340 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7a118fb80 10 client.20340 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7a118fb80 20 client.20340 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac316d90 0x55e4ac319180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7a118fb80 4 client.20340 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4ac25bea0 con 0x55e4ac316d90 2024-06-02T23:30:04.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.884+0000 7ff7877fe700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac316d90 0x55e4ac319180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.885+0000 7ff7877fe700 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac316d90 0x55e4ac319180 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74836347e9ca4a92 server_cookie=dd0350589527a9 in_seq=0 out_seq=0 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.885+0000 7ff76f7fe700 10 client.20340 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3335771889 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e4ac25bea0 con 0x55e4ac316d90 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff76f7fe700 10 client.20340 handle_command_reply: tid=0 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 client.20340 shutdown 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 msgr2=0x55e4ac3092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:04.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 0x55e4ac3092d0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff78000b070 tx=0x7ff780004f00 comp rx=0 tx=0).stop 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 shutdown_connections 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac316d90 0x55e4ac319180 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac3035f0 0x55e4ac306b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac2fd460 0x55e4ac3092d0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3335771889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac302cd0 0x55e4ac3065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff76f7fe700 0 client.20340 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 >> 172.21.15.52:0/3335771889 conn(0x55e4ac227e50 msgr2=0x55e4ac2fbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 shutdown_connections 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.886+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3335771889 wait complete. 2024-06-02T23:30:04.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.887+0000 7ff7a118fb80 20 client.20340 trim_cache size 0 max 16384 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 Processor -- start 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 -- start start 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac3055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac2f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901a0ec0 con 0x55e4ac3035f0 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901ac260 con 0x55e4ac2fd460 2024-06-02T23:30:04.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.919+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901a1590 con 0x55e4ac302cd0 2024-06-02T23:30:04.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac3055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac2f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff794ea9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac3055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37962/0 (socket says 172.21.15.52:37962) 2024-06-02T23:30:04.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac2f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:45518/0 (socket says 172.21.15.52:45518) 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 -- 172.21.15.52:0/2960317178 learned_addr learned my addr 172.21.15.52:0/2960317178 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 -- 172.21.15.52:0/2960317178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 msgr2=0x55e4ac2f78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac2f78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 -- 172.21.15.52:0/2960317178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 msgr2=0x55e4ac2f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 -- 172.21.15.52:0/2960317178 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4ac313540 con 0x55e4ac2fd460 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff794ea9700 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.920+0000 7ff78ffff700 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac3055d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff78000d010 tx=0x7ff78000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1732ca07b210d7c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.921+0000 7ff785ffb700 1 -- 172.21.15.52:0/2960317178 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78002b070 con 0x55e4ac2fd460 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.921+0000 7ff785ffb700 1 -- 172.21.15.52:0/2960317178 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff780005c60 con 0x55e4ac2fd460 2024-06-02T23:30:04.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.921+0000 7ff785ffb700 1 -- 172.21.15.52:0/2960317178 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78001f040 con 0x55e4ac2fd460 2024-06-02T23:30:04.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.921+0000 7ff7a118fb80 1 -- 172.21.15.52:0/2960317178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 msgr2=0x55e4ac3055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:04.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac3055d0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7ff78000d010 tx=0x7ff78000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 -- 172.21.15.52:0/2960317178 shutdown_connections 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac2f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac3055d0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/2960317178 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac2f78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 -- 172.21.15.52:0/2960317178 >> 172.21.15.52:0/2960317178 conn(0x55e4ac227e50 msgr2=0x55e4ac304ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 -- 172.21.15.52:0/2960317178 shutdown_connections 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 -- 172.21.15.52:0/2960317178 wait complete. 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.922+0000 7ff7a118fb80 1 -- start start 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.923+0000 7ff7a118fb80 1 Processor -- start 2024-06-02T23:30:04.924 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.923+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac319e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac31a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7a118fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac31a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.926 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901a2330 con 0x55e4ac3035f0 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7901a1c60 con 0x55e4ac2fd460 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7a118fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff79019ee80 con 0x55e4ac302cd0 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7a118fb80 10 client.0 _ll_register_callbacks cb 0x55e4ac1e8ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff794ea9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac31a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac319e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff7877fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac31a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac319e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37964/0 (socket says 172.21.15.52:37964) 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 -- 172.21.15.52:0/3277335593 learned_addr learned my addr 172.21.15.52:0/3277335593 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 -- 172.21.15.52:0/3277335593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 msgr2=0x55e4ac31a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac31a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 -- 172.21.15.52:0/3277335593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 msgr2=0x55e4ac31a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:04.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac31a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 -- 172.21.15.52:0/3277335593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4ac380980 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff794ea9700 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac31a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.924+0000 7ff78ffff700 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac319e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff7800056f0 tx=0x7ff780009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f006ddb8e586b95 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff76f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3277335593 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff78002b070 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3277335593 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff7800094c0 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff780005c60 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3277335593 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff78001f040 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff7a118fb80 10 client.29669 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e4ac381c40 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3277335593 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff780005c60 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.925+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3277335593 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff780053040 con 0x55e4ac2fd460 2024-06-02T23:30:04.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7a118fb80 10 client.29669 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7a118fb80 10 client.29669 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7a118fb80 20 client.29669 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac382160 0x55e4ac384550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7a118fb80 4 client.29669 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.0, "await_for": 45.0}] 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.0, "await_for": 45.0}) -- 0x55e4ac256420 con 0x55e4ac382160 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7877fe700 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac382160 0x55e4ac384550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff7877fe700 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac382160 0x55e4ac384550 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b37acd24387bbf58 server_cookie=c8c464d78862cb38 in_seq=0 out_seq=0 2024-06-02T23:30:04.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:04.926+0000 7ff76f7fe700 10 client.29669 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:05.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:05 smithi052 ceph-mon[32340]: pgmap v977: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-02T23:30:05.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:05 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.236+0000 7ff76f7fe700 1 -- 172.21.15.52:0/3277335593 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e4ac256420 con 0x55e4ac382160 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.236+0000 7ff76f7fe700 10 client.29669 handle_command_reply: tid=0 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.236+0000 7ff7a118fb80 1 client.29669 shutdown 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.236+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 msgr2=0x55e4ac319e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac319e10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7ff7800056f0 tx=0x7ff780009ca0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 shutdown_connections 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e4ac382160 0x55e4ac384550 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff76f7fe700 0 client.29669 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e4ac3035f0 0x55e4ac31a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4ac2fd460 0x55e4ac319e10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 --2- 172.21.15.52:0/3277335593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e4ac302cd0 0x55e4ac31a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 >> 172.21.15.52:0/3277335593 conn(0x55e4ac227e50 msgr2=0x55e4ac2f8150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 shutdown_connections 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 1 -- 172.21.15.52:0/3277335593 wait complete. 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.237+0000 7ff7a118fb80 20 client.29669 trim_cache size 0 max 16384 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:30:05.239 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 62, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "7a5988f4": { 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 62, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.0, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:30:05.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff77c0a2f20 msgr2=0x7ff77c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:05.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff77c0a2f20 0x7ff77c0a53e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7ff7800060b0 tx=0x7ff780006000 comp rx=0 tx=0).stop 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 msgr2=0x7ff790111600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790111600 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff78800ea80 tx=0x7ff78800ed90 comp rx=0 tx=0).stop 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 shutdown_connections 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff77c0a2f20 0x7ff77c0a53e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff790107470 0x7ff790111600 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff7901013c0 0x7ff790114010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 --2- 172.21.15.52:0/3893680408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff790101d80 0x7ff7901110c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.238+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 >> 172.21.15.52:0/3893680408 conn(0x7ff7900783d0 msgr2=0x7ff790102fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.239+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 shutdown_connections 2024-06-02T23:30:05.242 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:05.239+0000 7ff79690c700 1 -- 172.21.15.52:0/3893680408 wait complete. 2024-06-02T23:30:05.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:05 smithi174 ceph-mon[35994]: pgmap v977: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-02T23:30:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:05 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.82 MB/sec execute 193 sec latency 539.834 ms 2024-06-02T23:30:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:05 smithi123 ceph-mon[35977]: pgmap v977: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 356 KiB/s rd, 1.4 MiB/s wr, 179 op/s 2024-06-02T23:30:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:05.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:05 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:06.331 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7a5988f4', quiesce took 9.4 seconds. Will release after: 17.6 2024-06-02T23:30:06.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.79 MB/sec execute 194 sec latency 1539.901 ms 2024-06-02T23:30:07.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.75 MB/sec execute 195 sec latency 2539.930 ms 2024-06-02T23:30:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:07 smithi123 ceph-mon[35977]: pgmap v978: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.4 MiB/s wr, 285 op/s 2024-06-02T23:30:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:30:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:30:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:07 smithi174 ceph-mon[35994]: pgmap v978: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.4 MiB/s wr, 285 op/s 2024-06-02T23:30:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:30:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:30:08.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:07 smithi052 ceph-mon[32340]: pgmap v978: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 679 KiB/s rd, 2.4 MiB/s wr, 285 op/s 2024-06-02T23:30:08.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:30:08.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:30:08.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:30:08.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.72 MB/sec execute 196 sec latency 3539.980 ms 2024-06-02T23:30:09.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.69 MB/sec execute 197 sec latency 4540.109 ms 2024-06-02T23:30:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:09 smithi123 ceph-mon[35977]: pgmap v979: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-06-02T23:30:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:09 smithi174 ceph-mon[35994]: pgmap v979: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-06-02T23:30:10.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:09 smithi052 ceph-mon[32340]: pgmap v979: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 640 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-06-02T23:30:10.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.65 MB/sec execute 198 sec latency 5540.174 ms 2024-06-02T23:30:11.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.62 MB/sec execute 199 sec latency 6540.270 ms 2024-06-02T23:30:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:11 smithi123 ceph-mon[35977]: pgmap v980: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-06-02T23:30:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:11 smithi174 ceph-mon[35994]: pgmap v980: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-06-02T23:30:12.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:11 smithi052 ceph-mon[32340]: pgmap v980: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 639 KiB/s rd, 2.6 MiB/s wr, 256 op/s 2024-06-02T23:30:12.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.59 MB/sec execute 200 sec latency 7540.372 ms 2024-06-02T23:30:13.463 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.55 MB/sec execute 201 sec latency 8540.422 ms 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/1245063264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18107780 msgr2=0x7f0a18109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/1245063264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18107780 0x7f0a18109b70 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0a10009ca0 tx=0x7f0a1002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/1245063264 shutdown_connections 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/1245063264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18107780 0x7f0a18109b70 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/1245063264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a18104e50 0x7f0a18107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/1245063264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18102520 0x7f0a18104910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/1245063264 >> 172.21.15.52:0/1245063264 conn(0x7f0a180fbe70 msgr2=0x7f0a180fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/1245063264 shutdown_connections 2024-06-02T23:30:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.499+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/1245063264 wait complete. 2024-06-02T23:30:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 Processor -- start 2024-06-02T23:30:13.501 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 -- start start 2024-06-02T23:30:13.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 0x7f0a18197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18104e50 0x7f0a181980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a18107780 0x7f0a1819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810d550 con 0x7f0a18102520 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810d3d0 con 0x7f0a18107780 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1e8f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a1810d6d0 con 0x7f0a18104e50 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 0x7f0a18197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 0x7f0a18197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36768/0 (socket says 172.21.15.52:36768) 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a17fff640 1 -- 172.21.15.52:0/2655812510 learned_addr learned my addr 172.21.15.52:0/2655812510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a18107780 0x7f0a1819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.500+0000 7f0a177fe640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18104e50 0x7f0a181980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a17fff640 1 -- 172.21.15.52:0/2655812510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18104e50 msgr2=0x7f0a181980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a17fff640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18104e50 0x7f0a181980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a17fff640 1 -- 172.21.15.52:0/2655812510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a18107780 msgr2=0x7f0a1819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a17fff640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a18107780 0x7f0a1819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a17fff640 1 -- 172.21.15.52:0/2655812510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a1819f7c0 con 0x7f0a18102520 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a177fe640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18104e50 0x7f0a181980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a17fff640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 0x7f0a18197ba0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d770 tx=0x7f0a0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a157fa640 1 -- 172.21.15.52:0/2655812510 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08017070 con 0x7f0a18102520 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a157fa640 1 -- 172.21.15.52:0/2655812510 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a080054d0 con 0x7f0a18102520 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a157fa640 1 -- 172.21.15.52:0/2655812510 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08004cf0 con 0x7f0a18102520 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a1819fab0 con 0x7f0a18102520 2024-06-02T23:30:13.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.501+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a181a0000 con 0x7f0a18102520 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.503+0000 7f0a157fa640 1 -- 172.21.15.52:0/2655812510 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0a08004e90 con 0x7f0a18102520 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 Processor -- start 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 -- start start 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb014910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb017410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c8faf6e4f0 con 0x55c8fb00fe10 2024-06-02T23:30:13.510 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8faf0c870 con 0x55c8fb00f420 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a215fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8faf6a4b0 con 0x55c8fb014f80 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb014910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a17fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb014910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36782/0 (socket says 172.21.15.52:36782) 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a17fff640 1 -- 172.21.15.52:0/828384425 learned_addr learned my addr 172.21.15.52:0/828384425 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a157fa640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a000a2cb0 0x7f0a000a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a177fe640 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb017410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.504+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a000a2cb0 0x7f0a000a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a157fa640 1 -- 172.21.15.52:0/2655812510 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a080cea00 con 0x7f0a18102520 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a17fff640 1 -- 172.21.15.52:0/828384425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 msgr2=0x55c8fb017410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a17fff640 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb017410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a17fff640 1 -- 172.21.15.52:0/828384425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 msgr2=0x55c8fb00f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.511 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a17fff640 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a17fff640 1 -- 172.21.15.52:0/828384425 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8faf289b0 con 0x55c8fb00fe10 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f0a17fff640 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb014910 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f0a080976f0 tx=0x7f0a08005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462149c3ce5fab8b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f09fe7fc640 1 -- 172.21.15.52:0/828384425 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a08020030 con 0x55c8fb00fe10 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f09fe7fc640 1 -- 172.21.15.52:0/828384425 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a08009c30 con 0x55c8fb00fe10 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.505+0000 7f09fe7fc640 1 -- 172.21.15.52:0/828384425 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a080a3f00 con 0x55c8fb00fe10 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 -- 172.21.15.52:0/828384425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 msgr2=0x55c8fb014910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a000a2cb0 0x7f0a000a5170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a10004af0 tx=0x7f0a10005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb014910 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f0a080976f0 tx=0x7f0a08005680 comp rx=0 tx=0).stop 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 -- 172.21.15.52:0/828384425 shutdown_connections 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb017410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.512 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb014910 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 --2- 172.21.15.52:0/828384425 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 -- 172.21.15.52:0/828384425 >> 172.21.15.52:0/828384425 conn(0x55c8fb00aa90 msgr2=0x55c8fb00ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 -- 172.21.15.52:0/828384425 shutdown_connections 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 -- 172.21.15.52:0/828384425 wait complete. 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 -- start start 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.506+0000 7f0a215fd740 1 Processor -- start 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb019580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb019ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb020b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c8fb017e70 con 0x55c8fb00fe10 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c8faef7ae0 con 0x55c8fb00f420 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8fb017ff0 con 0x55c8fb014f80 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a215fd740 10 client.0 _ll_register_callbacks cb 0x55c8faf309b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:13.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb019ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb019ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36790/0 (socket says 172.21.15.52:36790) 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 -- 172.21.15.52:0/3343597281 learned_addr learned my addr 172.21.15.52:0/3343597281 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb019580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb020b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 -- 172.21.15.52:0/3343597281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 msgr2=0x55c8fb020b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb020b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 -- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 msgr2=0x55c8fb019580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb019580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 -- 172.21.15.52:0/3343597281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8fac82210 con 0x55c8fb00fe10 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb019ac0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00bfb0 tx=0x7f0a0c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7aa4b6bdc0106c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.507+0000 7f09f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:13.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f09f77fe640 1 -- 172.21.15.52:0/3343597281 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c0035f0 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f09f77fe640 1 -- 172.21.15.52:0/3343597281 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a0c003790 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c8faf81f10 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f09f77fe640 1 -- 172.21.15.52:0/3343597281 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c003990 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 10 client.20358 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8fad4e620 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f09f77fe640 1 -- 172.21.15.52:0/3343597281 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x55c8faf81f10 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f09f77fe640 1 -- 172.21.15.52:0/3343597281 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0a0c019070 con 0x55c8fb00fe10 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 10 client.20358 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 10 client.20358 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 20 client.20358 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0254b0 0x55c8fb027930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 4 client.20358 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:13.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.508+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8faf7bc20 con 0x55c8fb0254b0 2024-06-02T23:30:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.509+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0254b0 0x55c8fb027930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.509+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0254b0 0x55c8fb027930 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86cfdad8d51461e8 server_cookie=c390886cfd9a8f99 in_seq=0 out_seq=0 2024-06-02T23:30:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.509+0000 7f09f77fe640 10 client.20358 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f09f77fe640 1 -- 172.21.15.52:0/3343597281 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c8faf7bc20 con 0x55c8fb0254b0 2024-06-02T23:30:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f09f77fe640 10 client.20358 handle_command_reply: tid=0 2024-06-02T23:30:13.516 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 client.20358 shutdown 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 msgr2=0x55c8fb019ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb019ac0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00bfb0 tx=0x7f0a0c0128e0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 shutdown_connections 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0254b0 0x55c8fb027930 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb020b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb019ac0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f09f77fe640 0 client.20358 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3343597281 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb019580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 >> 172.21.15.52:0/3343597281 conn(0x55c8fb00aa90 msgr2=0x55c8fb0057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:13.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 shutdown_connections 2024-06-02T23:30:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 1 -- 172.21.15.52:0/3343597281 wait complete. 2024-06-02T23:30:13.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.510+0000 7f0a215fd740 20 client.20358 trim_cache size 0 max 16384 2024-06-02T23:30:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 Processor -- start 2024-06-02T23:30:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 -- start start 2024-06-02T23:30:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb020ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb019580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a6b70 con 0x55c8fb00fe10 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a86b0 con 0x55c8fb00f420 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a215fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a7240 con 0x55c8fb014f80 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb020ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb020ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36806/0 (socket says 172.21.15.52:36806) 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 -- 172.21.15.52:0/3818270512 learned_addr learned my addr 172.21.15.52:0/3818270512 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 -- 172.21.15.52:0/3818270512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 msgr2=0x55c8fb019580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb019580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 -- 172.21.15.52:0/3818270512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 msgr2=0x55c8fb00a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 -- 172.21.15.52:0/3818270512 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8fb00b760 con 0x55c8fb00fe10 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.526+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb020ac0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00bf80 tx=0x7f0a0c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb4ad5214ef28c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:13.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f09fdffb640 1 -- 172.21.15.52:0/3818270512 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c000e60 con 0x55c8fb00fe10 2024-06-02T23:30:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f09fdffb640 1 -- 172.21.15.52:0/3818270512 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a0c0035f0 con 0x55c8fb00fe10 2024-06-02T23:30:13.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f09fdffb640 1 -- 172.21.15.52:0/3818270512 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c010490 con 0x55c8fb00fe10 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 -- 172.21.15.52:0/3818270512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 msgr2=0x55c8fb020ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb020ac0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c00bf80 tx=0x7f0a0c00b7c0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 -- 172.21.15.52:0/3818270512 shutdown_connections 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb019580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb020ac0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3818270512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb00a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 -- 172.21.15.52:0/3818270512 >> 172.21.15.52:0/3818270512 conn(0x55c8fb00aa90 msgr2=0x55c8fb00ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:13.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 -- 172.21.15.52:0/3818270512 shutdown_connections 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.527+0000 7f0a215fd740 1 -- 172.21.15.52:0/3818270512 wait complete. 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 -- start start 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 Processor -- start 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb01e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb01e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb01f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a7fe0 con 0x55c8fb00fe10 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a7910 con 0x55c8fb00f420 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a181a4b50 con 0x55c8fb014f80 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a215fd740 10 client.0 _ll_register_callbacks cb 0x7f0a0c00b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb01e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb01e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36812/0 (socket says 172.21.15.52:36812) 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a177fe640 1 -- 172.21.15.52:0/3805957811 learned_addr learned my addr 172.21.15.52:0/3805957811 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:13.533 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb01e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.528+0000 7f0a1ce94640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb01f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a177fe640 1 -- 172.21.15.52:0/3805957811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 msgr2=0x55c8fb01f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb01f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a177fe640 1 -- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 msgr2=0x55c8fb01e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb01e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a177fe640 1 -- 172.21.15.52:0/3805957811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8fb010ae0 con 0x55c8fb00fe10 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb01e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a177fe640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb01e7b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005bf0 tx=0x7f0a0c0037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4099867b3c33196a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f09f77fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f09f77fe640 1 -- 172.21.15.52:0/3805957811 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a0c033040 con 0x55c8fb00fe10 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f09f77fe640 1 -- 172.21.15.52:0/3805957811 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a0c003b30 con 0x55c8fb00fe10 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a0c0035f0 con 0x55c8fb00fe10 2024-06-02T23:30:13.534 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f09f77fe640 1 -- 172.21.15.52:0/3805957811 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a0c003d30 con 0x55c8fb00fe10 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a215fd740 10 client.20364 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8fb090210 con 0x55c8fb00fe10 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.529+0000 7f09f77fe640 1 -- 172.21.15.52:0/3805957811 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a0c0035f0 con 0x55c8fb00fe10 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f09f77fe640 1 -- 172.21.15.52:0/3805957811 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0a0c03f030 con 0x55c8fb00fe10 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a215fd740 10 client.20364 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a215fd740 10 client.20364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a215fd740 20 client.20364 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0909b0 0x55c8fb092dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a215fd740 4 client.20364 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c8faf75f70 con 0x55c8fb0909b0 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0909b0 0x55c8fb092dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f09f77fe640 10 client.20364 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:13.535 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.530+0000 7f0a17fff640 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0909b0 0x55c8fb092dd0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e16f03e479188547 server_cookie=be7347ab8b028c44 in_seq=0 out_seq=0 2024-06-02T23:30:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.565+0000 7f09f77fe640 1 -- 172.21.15.52:0/3805957811 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c8faf75f70 con 0x55c8fb0909b0 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.565+0000 7f09f77fe640 10 client.20364 handle_command_reply: tid=0 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.565+0000 7f0a215fd740 1 client.20364 shutdown 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 msgr2=0x55c8fb01e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb01e7b0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c005bf0 tx=0x7f0a0c0037a0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 shutdown_connections 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8fb0909b0 0x55c8fb092dd0 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c8fb014f80 0x55c8fb01f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c8fb00fe10 0x55c8fb01e7b0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 --2- 172.21.15.52:0/3805957811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c8fb00f420 0x55c8fb01e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 >> 172.21.15.52:0/3805957811 conn(0x55c8fb00aa90 msgr2=0x55c8fb019560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f09f77fe640 0 client.20364 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:13.567 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 shutdown_connections 2024-06-02T23:30:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 1 -- 172.21.15.52:0/3805957811 wait complete. 2024-06-02T23:30:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.566+0000 7f0a215fd740 20 client.20364 trim_cache size 0 max 16384 2024-06-02T23:30:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:30:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:30:13.568 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a000a2cb0 msgr2=0x7f0a000a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a000a2cb0 0x7f0a000a5170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0a10004af0 tx=0x7f0a10005d80 comp rx=0 tx=0).stop 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 msgr2=0x7f0a18197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 0x7f0a18197ba0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800d770 tx=0x7f0a0800dc30 comp rx=0 tx=0).stop 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 shutdown_connections 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a000a2cb0 0x7f0a000a5170 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a18104e50 0x7f0a181980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a18107780 0x7f0a1819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 --2- 172.21.15.52:0/2655812510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a18102520 0x7f0a18197ba0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 >> 172.21.15.52:0/2655812510 conn(0x7f0a180fbe70 msgr2=0x7f0a18102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:13.569 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 shutdown_connections 2024-06-02T23:30:13.570 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:13.567+0000 7f0a1e8f7640 1 -- 172.21.15.52:0/2655812510 wait complete. 2024-06-02T23:30:14.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:13 smithi123 ceph-mon[35977]: pgmap v981: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-06-02T23:30:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:13 smithi174 ceph-mon[35994]: pgmap v981: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-06-02T23:30:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:13 smithi052 ceph-mon[32340]: pgmap v981: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 604 KiB/s rd, 2.2 MiB/s wr, 230 op/s 2024-06-02T23:30:14.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.52 MB/sec execute 202 sec latency 9540.521 ms 2024-06-02T23:30:15.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.49 MB/sec execute 203 sec latency 10540.602 ms 2024-06-02T23:30:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:15 smithi123 ceph-mon[35977]: pgmap v982: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-06-02T23:30:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:15 smithi174 ceph-mon[35994]: pgmap v982: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-06-02T23:30:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:15 smithi052 ceph-mon[32340]: pgmap v982: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 1.7 MiB/s wr, 166 op/s 2024-06-02T23:30:16.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.46 MB/sec execute 204 sec latency 11540.657 ms 2024-06-02T23:30:17.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.43 MB/sec execute 205 sec latency 12540.756 ms 2024-06-02T23:30:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:17 smithi123 ceph-mon[35977]: pgmap v983: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-02T23:30:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:17 smithi174 ceph-mon[35994]: pgmap v983: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-02T23:30:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:17 smithi052 ceph-mon[32340]: pgmap v983: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 368 KiB/s rd, 1.3 MiB/s wr, 158 op/s 2024-06-02T23:30:18.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.39 MB/sec execute 206 sec latency 13540.899 ms 2024-06-02T23:30:19.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.36 MB/sec execute 207 sec latency 14540.955 ms 2024-06-02T23:30:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:19 smithi123 ceph-mon[35977]: pgmap v984: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 219 KiB/s wr, 49 op/s 2024-06-02T23:30:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:19 smithi174 ceph-mon[35994]: pgmap v984: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 219 KiB/s wr, 49 op/s 2024-06-02T23:30:20.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:19 smithi052 ceph-mon[32340]: pgmap v984: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 219 KiB/s wr, 49 op/s 2024-06-02T23:30:20.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.33 MB/sec execute 208 sec latency 15541.008 ms 2024-06-02T23:30:21.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.30 MB/sec execute 209 sec latency 16541.116 ms 2024-06-02T23:30:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:21 smithi123 ceph-mon[35977]: pgmap v985: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-06-02T23:30:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:21 smithi174 ceph-mon[35994]: pgmap v985: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-06-02T23:30:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:21 smithi052 ceph-mon[32340]: pgmap v985: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-06-02T23:30:22.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.27 MB/sec execute 210 sec latency 17541.244 ms 2024-06-02T23:30:23.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.24 MB/sec execute 211 sec latency 18541.317 ms 2024-06-02T23:30:23.933 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:30:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:23 smithi123 ceph-mon[35977]: pgmap v986: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-06-02T23:30:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:23 smithi174 ceph-mon[35994]: pgmap v986: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-06-02T23:30:24.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:23 smithi052 ceph-mon[32340]: pgmap v986: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-06-02T23:30:24.199 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:24.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.21 MB/sec execute 212 sec latency 19541.356 ms 2024-06-02T23:30:25.464 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.18 MB/sec execute 213 sec latency 20541.434 ms 2024-06-02T23:30:25.959 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:25 smithi052 ceph-mon[32340]: pgmap v987: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-06-02T23:30:25.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 -- 172.21.15.52:0/2808702642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c108570 msgr2=0x7f669c108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 --2- 172.21.15.52:0/2808702642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c108570 0x7f669c108950 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b0a0 tx=0x7f668c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 -- 172.21.15.52:0/2808702642 shutdown_connections 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 --2- 172.21.15.52:0/2808702642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f669c06b5c0 0x7f669c10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 --2- 172.21.15.52:0/2808702642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c108570 0x7f669c108950 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 --2- 172.21.15.52:0/2808702642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06ac00 0x7f669c06b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 -- 172.21.15.52:0/2808702642 >> 172.21.15.52:0/2808702642 conn(0x7f669c0773e0 msgr2=0x7f669c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 -- 172.21.15.52:0/2808702642 shutdown_connections 2024-06-02T23:30:25.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.967+0000 7f66a2be6700 1 -- 172.21.15.52:0/2808702642 wait complete. 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 Processor -- start 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 -- start start 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 0x7f669c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06b5c0 0x7f669c101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f669c108570 0x7f669c101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f669c112650 con 0x7f669c06ac00 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f669c1124d0 con 0x7f669c108570 2024-06-02T23:30:25.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a2be6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f669c1127d0 con 0x7f669c06b5c0 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 0x7f669c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 0x7f669c1014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53570/0 (socket says 172.21.15.52:53570) 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 -- 172.21.15.52:0/3352716487 learned_addr learned my addr 172.21.15.52:0/3352716487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a1183700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f669c108570 0x7f669c101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f669bfff700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06b5c0 0x7f669c101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 -- 172.21.15.52:0/3352716487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06b5c0 msgr2=0x7f669c101a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06b5c0 0x7f669c101a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 -- 172.21.15.52:0/3352716487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f669c108570 msgr2=0x7f669c101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f669c108570 0x7f669c101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f66a0982700 1 -- 172.21.15.52:0/3352716487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f669c1a6f50 con 0x7f669c06ac00 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.968+0000 7f669bfff700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06b5c0 0x7f669c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:25.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.969+0000 7f66a0982700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 0x7f669c1014f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b070 tx=0x7f668c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:25.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.969+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c021070 con 0x7f669c06ac00 2024-06-02T23:30:25.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.969+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f669c1a71e0 con 0x7f669c06ac00 2024-06-02T23:30:25.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.969+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f668c007e30 con 0x7f669c06ac00 2024-06-02T23:30:25.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.969+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668c004840 con 0x7f669c06ac00 2024-06-02T23:30:25.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.969+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f669c1a7690 con 0x7f669c06ac00 2024-06-02T23:30:25.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.970+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f669c069180 con 0x7f669c06ac00 2024-06-02T23:30:25.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.971+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f668c003c00 con 0x7f669c06ac00 2024-06-02T23:30:25.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.972+0000 7f6699ffb700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66880a2c20 0x7f66880a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:25.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.972+0000 7f669bfff700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66880a2c20 0x7f66880a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:25.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.972+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f668c00f020 con 0x7f669c06ac00 2024-06-02T23:30:25.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.973+0000 7f669bfff700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66880a2c20 0x7f66880a50e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6684005fd0 tx=0x7f6684005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:25.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:25.976+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f668c02a020 con 0x7f669c06ac00 2024-06-02T23:30:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:25 smithi123 ceph-mon[35977]: pgmap v987: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-06-02T23:30:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:25 smithi174 ceph-mon[35994]: pgmap v987: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 21 op/s 2024-06-02T23:30:26.229 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.229+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f669c1a7860 con 0x7f669c06ac00 2024-06-02T23:30:26.230 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.230+0000 7f6699ffb700 1 -- 172.21.15.52:0/3352716487 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f668c09d900 con 0x7f669c06ac00 2024-06-02T23:30:26.230 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:30:26.231 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66880a2c20 msgr2=0x7f66880a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66880a2c20 0x7f66880a50e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f6684005fd0 tx=0x7f6684005e30 comp rx=0 tx=0).stop 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 msgr2=0x7f669c1014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 0x7f669c1014f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f668c00b070 tx=0x7f668c004d00 comp rx=0 tx=0).stop 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 shutdown_connections 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66880a2c20 0x7f66880a50e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f669c108570 0x7f669c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f669c06ac00 0x7f669c1014f0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.234 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 --2- 172.21.15.52:0/3352716487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f669c06b5c0 0x7f669c101a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 >> 172.21.15.52:0/3352716487 conn(0x7f669c0773e0 msgr2=0x7f669c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 shutdown_connections 2024-06-02T23:30:26.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:26.233+0000 7f66a2be6700 1 -- 172.21.15.52:0/3352716487 wait complete. 2024-06-02T23:30:26.235 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:30:26.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.15 MB/sec execute 214 sec latency 21541.520 ms 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 -- 172.21.15.52:0/1750656830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0100bb0 msgr2=0x7f35a0101030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1750656830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0100bb0 0x7f35a0101030 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3590009930 tx=0x7f359002a420 comp rx=0 tx=0).stop 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 -- 172.21.15.52:0/1750656830 shutdown_connections 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1750656830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a0101570 0x7f35a0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1750656830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0100bb0 0x7f35a0101030 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1750656830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0101f10 0x7f35a0102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 -- 172.21.15.52:0/1750656830 >> 172.21.15.52:0/1750656830 conn(0x7f35a00fb600 msgr2=0x7f35a00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.659 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 -- 172.21.15.52:0/1750656830 shutdown_connections 2024-06-02T23:30:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.658+0000 7f35a86c3640 1 -- 172.21.15.52:0/1750656830 wait complete. 2024-06-02T23:30:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 Processor -- start 2024-06-02T23:30:26.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 -- start start 2024-06-02T23:30:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0100bb0 0x7f35a0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a0101570 0x7f35a01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 0x7f35a019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010d560 con 0x7f35a0101f10 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010d3e0 con 0x7f35a0101570 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a86c3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35a010d6e0 con 0x7f35a0100bb0 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 0x7f35a019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 0x7f35a019f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53594/0 (socket says 172.21.15.52:53594) 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 -- 172.21.15.52:0/1184237543 learned_addr learned my addr 172.21.15.52:0/1184237543 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a645f640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0100bb0 0x7f35a0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a0101570 0x7f35a01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 -- 172.21.15.52:0/1184237543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0100bb0 msgr2=0x7f35a0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0100bb0 0x7f35a0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 -- 172.21.15.52:0/1184237543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a0101570 msgr2=0x7f35a01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a0101570 0x7f35a01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.659+0000 7f35a6c60640 1 -- 172.21.15.52:0/1184237543 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a019f7b0 con 0x7f35a0101f10 2024-06-02T23:30:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.660+0000 7f35a6c60640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 0x7f35a019f140 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f359800cad0 tx=0x7f359800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.660+0000 7f35977fe640 1 -- 172.21.15.52:0/1184237543 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359800ae20 con 0x7f35a0101f10 2024-06-02T23:30:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.660+0000 7f35977fe640 1 -- 172.21.15.52:0/1184237543 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3598005020 con 0x7f35a0101f10 2024-06-02T23:30:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.660+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a019faa0 con 0x7f35a0101f10 2024-06-02T23:30:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.660+0000 7f35977fe640 1 -- 172.21.15.52:0/1184237543 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359800f6b0 con 0x7f35a0101f10 2024-06-02T23:30:26.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.660+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a019ff70 con 0x7f35a0101f10 2024-06-02T23:30:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.662+0000 7f35977fe640 1 -- 172.21.15.52:0/1184237543 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f35980051d0 con 0x7f35a0101f10 2024-06-02T23:30:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.662+0000 7f35ab3cc740 1 Processor -- start 2024-06-02T23:30:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 -- start start 2024-06-02T23:30:26.669 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd68372430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd68377500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68377b70 0x55dd6837a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd682d54f0 con 0x55dd68372010 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd68273870 con 0x55dd68372a00 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd682d14b0 con 0x55dd68377b70 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd68372430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd68372430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53600/0 (socket says 172.21.15.52:53600) 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 -- 172.21.15.52:0/3580514638 learned_addr learned my addr 172.21.15.52:0/3580514638 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a645f640 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd68377500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35977fe640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f358c0a2c30 0x7f358c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68377b70 0x55dd6837a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.670 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a6c60640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f358c0a2c30 0x7f358c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 -- 172.21.15.52:0/3580514638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68377b70 msgr2=0x55dd6837a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68377b70 0x55dd6837a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.663+0000 7f35a5c5e640 1 -- 172.21.15.52:0/3580514638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 msgr2=0x55dd68377500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd68377500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f35a5c5e640 1 -- 172.21.15.52:0/3580514638 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd6828f9b0 con 0x55dd68372010 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f35a645f640 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd68377500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f35977fe640 1 -- 172.21.15.52:0/1184237543 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f35980cf200 con 0x7f35a0101f10 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd68372430 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3590005940 tx=0x7f35900058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d514de674843ec server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f35a6c60640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f358c0a2c30 0x7f358c0a50f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f359801e000 tx=0x7f3598066aa0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f3594ff9640 1 -- 172.21.15.52:0/3580514638 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590035070 con 0x55dd68372010 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f3594ff9640 1 -- 172.21.15.52:0/3580514638 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3590045070 con 0x55dd68372010 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.664+0000 7f3594ff9640 1 -- 172.21.15.52:0/3580514638 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590003d40 con 0x55dd68372010 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- 172.21.15.52:0/3580514638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 msgr2=0x55dd68372430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.671 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd68372430 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3590005940 tx=0x7f35900058b0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- 172.21.15.52:0/3580514638 shutdown_connections 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68377b70 0x55dd6837a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd68377500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/3580514638 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd68372430 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- 172.21.15.52:0/3580514638 >> 172.21.15.52:0/3580514638 conn(0x55dd6836d680 msgr2=0x55dd6836da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- 172.21.15.52:0/3580514638 shutdown_connections 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- 172.21.15.52:0/3580514638 wait complete. 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- start start 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 Processor -- start 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd6837c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372a00 0x55dd6837c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683837e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55dd6837aac0 con 0x55dd68372010 2024-06-02T23:30:26.672 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dd6825eae0 con 0x55dd68377b70 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.665+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd6837ac40 con 0x55dd68372a00 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35ab3cc740 10 client.0 _ll_register_callbacks cb 0x55dd682979b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd6837c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd6837c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53612/0 (socket says 172.21.15.52:53612) 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 -- 172.21.15.52:0/817730655 learned_addr learned my addr 172.21.15.52:0/817730655 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683837e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a645f640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372a00 0x55dd6837c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 -- 172.21.15.52:0/817730655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372a00 msgr2=0x55dd6837c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372a00 0x55dd6837c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 -- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 msgr2=0x55dd683837e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683837e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 -- 172.21.15.52:0/817730655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd67fe9210 con 0x55dd68372010 2024-06-02T23:30:26.673 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a645f640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372a00 0x55dd6837c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd6837c260 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f359002a940 tx=0x7f35900314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4322a48425f89c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f3589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f3589ffb640 1 -- 172.21.15.52:0/817730655 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590035040 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f3589ffb640 1 -- 172.21.15.52:0/817730655 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3590004030 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3590045070 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f3589ffb640 1 -- 172.21.15.52:0/817730655 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f35900318a0 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35ab3cc740 10 client.20382 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.666+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dd680b5620 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f3589ffb640 1 -- 172.21.15.52:0/817730655 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3590045070 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f3589ffb640 1 -- 172.21.15.52:0/817730655 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3590058c80 con 0x55dd68372010 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35ab3cc740 10 client.20382 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35ab3cc740 10 client.20382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35ab3cc740 20 client.20382 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:26.674 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd68387cd0 0x55dd6838a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35ab3cc740 4 client.20382 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd682e8f10 con 0x55dd68387cd0 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.667+0000 7f35a645f640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd68387cd0 0x55dd6838a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.668+0000 7f3589ffb640 10 client.20382 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.668+0000 7f35a645f640 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd68387cd0 0x55dd6838a0c0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3790af7f19d2b09 server_cookie=f299c3ebcfb83134 in_seq=0 out_seq=0 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.668+0000 7f3589ffb640 1 -- 172.21.15.52:0/817730655 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55dd682e8f10 con 0x55dd68387cd0 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.668+0000 7f3589ffb640 10 client.20382 handle_command_reply: tid=0 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.668+0000 7f35ab3cc740 1 client.20382 shutdown 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 msgr2=0x55dd6837c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd6837c260 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f359002a940 tx=0x7f35900314a0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 shutdown_connections 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd68387cd0 0x55dd6838a0c0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372a00 0x55dd6837c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.675 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f3589ffb640 0 client.20382 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683837e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/817730655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372010 0x55dd6837c260 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 >> 172.21.15.52:0/817730655 conn(0x55dd6836d680 msgr2=0x55dd6836c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 shutdown_connections 2024-06-02T23:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 1 -- 172.21.15.52:0/817730655 wait complete. 2024-06-02T23:30:26.676 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.669+0000 7f35ab3cc740 20 client.20382 trim_cache size 0 max 16384 2024-06-02T23:30:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 Processor -- start 2024-06-02T23:30:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 -- start start 2024-06-02T23:30:26.685 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd6837be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd6837c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 0x55dd68370590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01a8c00 con 0x55dd68377b70 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01aa740 con 0x55dd68372a00 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01a92d0 con 0x55dd68372010 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35a5c5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 0x55dd68370590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35a5c5e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 0x55dd68370590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53614/0 (socket says 172.21.15.52:53614) 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.684+0000 7f35a5c5e640 1 -- 172.21.15.52:0/514499844 learned_addr learned my addr 172.21.15.52:0/514499844 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a645f640 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd6837c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd6837be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 -- 172.21.15.52:0/514499844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 msgr2=0x55dd6837be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.686 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd6837be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 -- 172.21.15.52:0/514499844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 msgr2=0x55dd6837c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd6837c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 -- 172.21.15.52:0/514499844 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd68376f40 con 0x55dd68377b70 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 0x55dd68370590 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f359004c940 tx=0x7f359002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3457587dd388c769 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f358bfff640 1 -- 172.21.15.52:0/514499844 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3590035040 con 0x55dd68377b70 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f358bfff640 1 -- 172.21.15.52:0/514499844 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3590045070 con 0x55dd68377b70 2024-06-02T23:30:26.687 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f358bfff640 1 -- 172.21.15.52:0/514499844 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359003a3f0 con 0x55dd68377b70 2024-06-02T23:30:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35ab3cc740 1 -- 172.21.15.52:0/514499844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 msgr2=0x55dd68370590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.685+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 0x55dd68370590 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f359004c940 tx=0x7f359002a420 comp rx=0 tx=0).stop 2024-06-02T23:30:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- 172.21.15.52:0/514499844 shutdown_connections 2024-06-02T23:30:26.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68377b70 0x55dd68370590 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68372a00 0x55dd6837c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/514499844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd6837be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- 172.21.15.52:0/514499844 >> 172.21.15.52:0/514499844 conn(0x55dd6836d680 msgr2=0x55dd6836da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- 172.21.15.52:0/514499844 shutdown_connections 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- 172.21.15.52:0/514499844 wait complete. 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- start start 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 Processor -- start 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd683ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 0x55dd683eda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01aa070 con 0x55dd68372a00 2024-06-02T23:30:26.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01a99a0 con 0x55dd68377b70 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35a01a6be0 con 0x55dd68372010 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35ab3cc740 10 client.0 _ll_register_callbacks cb 0x7f359c00c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35a645f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 0x55dd683eda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35a645f640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 0x55dd683eda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53618/0 (socket says 172.21.15.52:53618) 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35a645f640 1 -- 172.21.15.52:0/4045222870 learned_addr learned my addr 172.21.15.52:0/4045222870 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.686+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd683ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a645f640 1 -- 172.21.15.52:0/4045222870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 msgr2=0x55dd683ed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a645f640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd683ed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a645f640 1 -- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 msgr2=0x55dd683f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a645f640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a645f640 1 -- 172.21.15.52:0/4045222870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd683f5050 con 0x55dd68372a00 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:26.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35a645f640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 0x55dd683eda20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x55dd68372e00 tx=0x7f359c00cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a38c6f2825c918 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f3589ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f3589ffb640 1 -- 172.21.15.52:0/4045222870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f359c023470 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f3589ffb640 1 -- 172.21.15.52:0/4045222870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f359c011440 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3590045070 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f3589ffb640 1 -- 172.21.15.52:0/4045222870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f359c011640 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 10 client.20388 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55dd683f6460 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f3589ffb640 1 -- 172.21.15.52:0/4045222870 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3590045070 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f3589ffb640 1 -- 172.21.15.52:0/4045222870 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f359c023b40 con 0x55dd68372a00 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 10 client.20388 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 10 client.20388 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 20 client.20388 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:26.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd683f6e40 0x55dd683f9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 4 client.20388 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:30:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.687+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd682e2c20 con 0x55dd683f6e40 2024-06-02T23:30:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.688+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd683f6e40 0x55dd683f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.688+0000 7f35a5c5e640 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd683f6e40 0x55dd683f9260 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17d4f5945d3a5486 server_cookie=c5a08de14906813e in_seq=0 out_seq=0 2024-06-02T23:30:26.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.688+0000 7f3589ffb640 10 client.20388 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f3589ffb640 1 -- 172.21.15.52:0/4045222870 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dd682e2c20 con 0x55dd683f6e40 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f3589ffb640 10 client.20388 handle_command_reply: tid=0 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 client.20388 shutdown 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 msgr2=0x55dd683eda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 0x55dd683eda20 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x55dd68372e00 tx=0x7f359c00cef0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 shutdown_connections 2024-06-02T23:30:26.714 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55dd683f6e40 0x55dd683f9260 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dd68377b70 0x55dd683f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f3589ffb640 0 client.20388 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55dd68372a00 0x55dd683eda20 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 --2- 172.21.15.52:0/4045222870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd68372010 0x55dd683ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 >> 172.21.15.52:0/4045222870 conn(0x55dd6836d680 msgr2=0x55dd6837be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 shutdown_connections 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 1 -- 172.21.15.52:0/4045222870 wait complete. 2024-06-02T23:30:26.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.713+0000 7f35ab3cc740 20 client.20388 trim_cache size 0 max 16384 2024-06-02T23:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f358c0a2c30 msgr2=0x7f358c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f358c0a2c30 0x7f358c0a50f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f359801e000 tx=0x7f3598066aa0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 msgr2=0x7f35a019f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 0x7f35a019f140 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f359800cad0 tx=0x7f359800cf90 comp rx=0 tx=0).stop 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 shutdown_connections 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f358c0a2c30 0x7f358c0a50f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f35a0101f10 0x7f35a019f140 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35a0101570 0x7f35a01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 --2- 172.21.15.52:0/1184237543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f35a0100bb0 0x7f35a0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 >> 172.21.15.52:0/1184237543 conn(0x7f35a00fb600 msgr2=0x7f35a0104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 shutdown_connections 2024-06-02T23:30:26.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:26.714+0000 7f35a86c3640 1 -- 172.21.15.52:0/1184237543 wait complete. 2024-06-02T23:30:26.855 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:26 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3352716487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:26.882 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 7a5988f4 --release --await' 2024-06-02T23:30:26.882 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 7a5988f4 --release --await 2024-06-02T23:30:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:26 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3352716487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:27.134 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:27.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:26 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3352716487' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:27.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.13 MB/sec execute 215 sec latency 22541.608 ms 2024-06-02T23:30:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:27 smithi052 ceph-mon[32340]: pgmap v988: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 22 op/s 2024-06-02T23:30:28.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:27 smithi123 ceph-mon[35977]: pgmap v988: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 22 op/s 2024-06-02T23:30:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:27 smithi174 ceph-mon[35994]: pgmap v988: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 26 KiB/s wr, 22 op/s 2024-06-02T23:30:28.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37194 6.10 MB/sec execute 216 sec latency 23541.664 ms 2024-06-02T23:30:28.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 -- 172.21.15.52:0/3255003101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 msgr2=0x7f4680107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 --2- 172.21.15.52:0/3255003101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f4680107110 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f46680099c0 tx=0x7f4668009cd0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 -- 172.21.15.52:0/3255003101 shutdown_connections 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 --2- 172.21.15.52:0/3255003101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4680107840 0x7f468010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 --2- 172.21.15.52:0/3255003101 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4680102740 0x7f4680102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 --2- 172.21.15.52:0/3255003101 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f4680107110 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 -- 172.21.15.52:0/3255003101 >> 172.21.15.52:0/3255003101 conn(0x7f46800795f0 msgr2=0x7f4680079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.880+0000 7f4684cef700 1 -- 172.21.15.52:0/3255003101 shutdown_connections 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.881+0000 7f4684cef700 1 -- 172.21.15.52:0/3255003101 wait complete. 2024-06-02T23:30:28.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.881+0000 7f4684cef700 1 Processor -- start 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 -- start start 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4680102740 0x7f468019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f468019ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4680107840 0x7f4680198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4680112600 con 0x7f46801030f0 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4680112480 con 0x7f4680102740 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4684cef700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4680112780 con 0x7f4680107840 2024-06-02T23:30:28.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4675d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f468019ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4675d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f468019ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53634/0 (socket says 172.21.15.52:53634) 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4675d9b700 1 -- 172.21.15.52:0/1738760431 learned_addr learned my addr 172.21.15.52:0/1738760431 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f467e59c700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4680102740 0x7f468019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f467ed9d700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4680107840 0x7f4680198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4675d9b700 1 -- 172.21.15.52:0/1738760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4680107840 msgr2=0x7f4680198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4675d9b700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4680107840 0x7f4680198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.882+0000 7f4675d9b700 1 -- 172.21.15.52:0/1738760431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4680102740 msgr2=0x7f468019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4675d9b700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4680102740 0x7f468019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4675d9b700 1 -- 172.21.15.52:0/1738760431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4680198fe0 con 0x7f46801030f0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4675d9b700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f468019ec00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f4668006010 tx=0x7f46680057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4677fff700 1 -- 172.21.15.52:0/1738760431 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668027070 con 0x7f46801030f0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4677fff700 1 -- 172.21.15.52:0/1738760431 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4668003f50 con 0x7f46801030f0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4677fff700 1 -- 172.21.15.52:0/1738760431 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4668022d10 con 0x7f46801030f0 2024-06-02T23:30:28.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4680199270 con 0x7f46801030f0 2024-06-02T23:30:28.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.883+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46801a5590 con 0x7f46801030f0 2024-06-02T23:30:28.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.885+0000 7f4677fff700 1 -- 172.21.15.52:0/1738760431 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f466802a070 con 0x7f46801030f0 2024-06-02T23:30:28.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.886+0000 7f4677fff700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f466c0a2ca0 0x7f466c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.886+0000 7f467e59c700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f466c0a2ca0 0x7f466c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.886+0000 7f4677fff700 1 -- 172.21.15.52:0/1738760431 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f46680cfee0 con 0x7f46801030f0 2024-06-02T23:30:28.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.886+0000 7f467e59c700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f466c0a2ca0 0x7f466c0a5160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4670005950 tx=0x7f467000a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:28.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 Processor -- start 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 -- start start 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7050 0x560efb2d7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7970 0x560efb2d12a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2d1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560efb21e910 con 0x560efb2d17e0 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560efb1c6820 con 0x560efb2d7050 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f468f572b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560efb21a8d0 con 0x560efb2d7970 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f467ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2d1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f467ed9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2d1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53644/0 (socket says 172.21.15.52:53644) 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f467ed9d700 1 -- 172.21.15.52:0/232113131 learned_addr learned my addr 172.21.15.52:0/232113131 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.887+0000 7f4675d9b700 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7970 0x560efb2d12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467e59c700 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7050 0x560efb2d7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467ed9d700 1 -- 172.21.15.52:0/232113131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7970 msgr2=0x560efb2d12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467ed9d700 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7970 0x560efb2d12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467ed9d700 1 -- 172.21.15.52:0/232113131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7050 msgr2=0x560efb2d7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467ed9d700 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7050 0x560efb2d7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467ed9d700 1 -- 172.21.15.52:0/232113131 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efb1bd050 con 0x560efb2d17e0 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467e59c700 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7050 0x560efb2d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f467ed9d700 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2d1c20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f467800f5a0 tx=0x7f467800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdfd7f86c38c1429 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f4674d99700 1 -- 172.21.15.52:0/232113131 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678018030 con 0x560efb2d17e0 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f4674d99700 1 -- 172.21.15.52:0/232113131 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4678004bb0 con 0x560efb2d17e0 2024-06-02T23:30:28.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.888+0000 7f4674d99700 1 -- 172.21.15.52:0/232113131 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4678005020 con 0x560efb2d17e0 2024-06-02T23:30:28.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 -- 172.21.15.52:0/232113131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 msgr2=0x560efb2d1c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2d1c20 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f467800f5a0 tx=0x7f467800f8b0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 -- 172.21.15.52:0/232113131 shutdown_connections 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2d1c20 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7050 0x560efb2d7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 --2- 172.21.15.52:0/232113131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7970 0x560efb2d12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 -- 172.21.15.52:0/232113131 >> 172.21.15.52:0/232113131 conn(0x560efb1f7dc0 msgr2=0x560efb2c8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 -- 172.21.15.52:0/232113131 shutdown_connections 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 -- 172.21.15.52:0/232113131 wait complete. 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 -- start start 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.889+0000 7f468f572b80 1 Processor -- start 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2dd690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2da990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2daed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560efb2de3e0 con 0x560efb2d17e0 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560efb1becf0 con 0x560efb2d7970 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560efb2de560 con 0x560efb2d7050 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f468f572b80 10 client.0 _ll_register_callbacks cb 0x560efb1b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2dd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2dd690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53660/0 (socket says 172.21.15.52:53660) 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 -- 172.21.15.52:0/1768011299 learned_addr learned my addr 172.21.15.52:0/1768011299 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467ed9d700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2daed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f4675d9b700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2da990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 -- 172.21.15.52:0/1768011299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 msgr2=0x560efb2da990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2da990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 -- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 msgr2=0x560efb2daed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2daed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 -- 172.21.15.52:0/1768011299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efb16d9d0 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f467e59c700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2dd690 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f467001da30 tx=0x7f46700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3917e9f9bdda21c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.890+0000 7f4665ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f4665ffb700 1 -- 172.21.15.52:0/1768011299 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f467002a060 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f4665ffb700 1 -- 172.21.15.52:0/1768011299 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4670010840 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4678004bb0 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f4665ffb700 1 -- 172.21.15.52:0/1768011299 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4670016070 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 10 client.20400 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560efb2e8cf0 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f4665ffb700 1 -- 172.21.15.52:0/1768011299 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4678004bb0 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f4665ffb700 1 -- 172.21.15.52:0/1768011299 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4670024030 con 0x560efb2d17e0 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 10 client.20400 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 10 client.20400 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 20 client.20400 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:28.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb2eb380 0x560efb2ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 4 client.20400 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.891+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560efb22be10 con 0x560efb2eb380 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.892+0000 7f4675d9b700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb2eb380 0x560efb2ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.892+0000 7f4665ffb700 10 client.20400 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.892+0000 7f4675d9b700 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb2eb380 0x560efb2ed770 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b46f69572b1ebe6b server_cookie=292cd0cc55cd6c80 in_seq=0 out_seq=0 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f4665ffb700 1 -- 172.21.15.52:0/1768011299 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x560efb22be10 con 0x560efb2eb380 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f4665ffb700 10 client.20400 handle_command_reply: tid=0 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 client.20400 shutdown 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 msgr2=0x560efb2dd690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2dd690 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7f467001da30 tx=0x7f46700047c0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 shutdown_connections 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb2eb380 0x560efb2ed770 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2daed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2dd690 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 --2- 172.21.15.52:0/1768011299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2da990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 >> 172.21.15.52:0/1768011299 conn(0x560efb1f7dc0 msgr2=0x560efb2c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f4665ffb700 0 client.20400 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 shutdown_connections 2024-06-02T23:30:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 1 -- 172.21.15.52:0/1768011299 wait complete. 2024-06-02T23:30:28.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.893+0000 7f468f572b80 20 client.20400 trim_cache size 0 max 16384 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.913+0000 7f468f572b80 1 Processor -- start 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.913+0000 7f468f572b80 1 -- start start 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f468f572b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f468f572b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f468f572b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2d7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f468f572b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46801aa680 con 0x560efb2d17e0 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f468f572b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46801ac1c0 con 0x560efb2d7970 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f468f572b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46801aad50 con 0x560efb2d7050 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2f2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53672/0 (socket says 172.21.15.52:53672) 2024-06-02T23:30:28.916 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 -- 172.21.15.52:0/1913997190 learned_addr learned my addr 172.21.15.52:0/1913997190 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467ed9d700 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2d7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 -- 172.21.15.52:0/1913997190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 msgr2=0x560efb2f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 -- 172.21.15.52:0/1913997190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 msgr2=0x560efb2d7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2d7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 -- 172.21.15.52:0/1913997190 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560efb2d8f20 con 0x560efb2d17e0 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467ed9d700 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2d7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f467e59c700 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2f2fa0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f467001bf70 tx=0x7f467001bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b12da5db94806e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f4667fff700 1 -- 172.21.15.52:0/1913997190 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670035030 con 0x560efb2d17e0 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f4667fff700 1 -- 172.21.15.52:0/1913997190 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f467001b8f0 con 0x560efb2d17e0 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.914+0000 7f4667fff700 1 -- 172.21.15.52:0/1913997190 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670031430 con 0x560efb2d17e0 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 -- 172.21.15.52:0/1913997190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 msgr2=0x560efb2f2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2f2fa0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f467001bf70 tx=0x7f467001bfa0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.917 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 -- 172.21.15.52:0/1913997190 shutdown_connections 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2d7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2f2fa0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 --2- 172.21.15.52:0/1913997190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 -- 172.21.15.52:0/1913997190 >> 172.21.15.52:0/1913997190 conn(0x560efb1f7dc0 msgr2=0x560efb2ed340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 -- 172.21.15.52:0/1913997190 shutdown_connections 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 -- 172.21.15.52:0/1913997190 wait complete. 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.915+0000 7f468f572b80 1 -- start start 2024-06-02T23:30:28.918 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 Processor -- start 2024-06-02T23:30:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.920 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46801abaf0 con 0x560efb2d17e0 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46801ab420 con 0x560efb2d7970 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46801a8640 con 0x560efb2d7050 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f468f572b80 10 client.0 _ll_register_callbacks cb 0x560efb1b8a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2e3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53676/0 (socket says 172.21.15.52:53676) 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 -- 172.21.15.52:0/2498645037 learned_addr learned my addr 172.21.15.52:0/2498645037 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467ed9d700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f4675d9b700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 -- 172.21.15.52:0/2498645037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 msgr2=0x560efb2e07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2e07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 -- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 msgr2=0x560efb2e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:28.921 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2e0d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 -- 172.21.15.52:0/2498645037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560efb2e1560 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f4675d9b700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2e07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.916+0000 7f467e59c700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2e3f30 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f467003b040 tx=0x7f467001b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78ff8915bd36a1a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f4665ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f4665ffb700 1 -- 172.21.15.52:0/2498645037 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4670035070 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f4665ffb700 1 -- 172.21.15.52:0/2498645037 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4670031430 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f467001b8f0 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f4665ffb700 1 -- 172.21.15.52:0/2498645037 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f467001baa0 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 10 client.20406 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560efb356f90 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f4665ffb700 1 -- 172.21.15.52:0/2498645037 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f467001b8f0 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f4665ffb700 1 -- 172.21.15.52:0/2498645037 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4670024040 con 0x560efb2d17e0 2024-06-02T23:30:28.922 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 10 client.20406 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 10 client.20406 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 20 client.20406 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb3574d0 0x560efb3598f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 4 client.20406 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7a5988f4", "release": true, "await": true}] 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.917+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7a5988f4", "release": true, "await": true}) -- 0x560efb226390 con 0x560efb3574d0 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.918+0000 7f4675d9b700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb3574d0 0x560efb3598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.918+0000 7f4675d9b700 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb3574d0 0x560efb3598f0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b6fd282670c617d server_cookie=4ee6c0aee3522e4 in_seq=0 out_seq=0 2024-06-02T23:30:28.923 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.918+0000 7f4665ffb700 10 client.20406 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.952+0000 7f4665ffb700 1 -- 172.21.15.52:0/2498645037 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x560efb226390 con 0x560efb3574d0 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.952+0000 7f4665ffb700 10 client.20406 handle_command_reply: tid=0 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.952+0000 7f468f572b80 1 client.20406 shutdown 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 msgr2=0x560efb2e3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2e3f30 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f467003b040 tx=0x7f467001b650 comp rx=0 tx=0).stop 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 shutdown_connections 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x560efb3574d0 0x560efb3598f0 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560efb2d7970 0x560efb2e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560efb2d17e0 0x560efb2e3f30 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 --2- 172.21.15.52:0/2498645037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560efb2d7050 0x560efb2e07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 >> 172.21.15.52:0/2498645037 conn(0x560efb1f7dc0 msgr2=0x560efb2d7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f4665ffb700 0 client.20406 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 shutdown_connections 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 1 -- 172.21.15.52:0/2498645037 wait complete. 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.953+0000 7f468f572b80 20 client.20406 trim_cache size 0 max 16384 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:30:28.955 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 64, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "7a5988f4": { 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 64, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.0, 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:30:28.956 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:30:28.957 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f466c0a2ca0 msgr2=0x7f466c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f466c0a2ca0 0x7f466c0a5160 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f4670005950 tx=0x7f467000a300 comp rx=0 tx=0).stop 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 msgr2=0x7f468019ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f468019ec00 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f4668006010 tx=0x7f46680057c0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 shutdown_connections 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f466c0a2ca0 0x7f466c0a5160 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4680107840 0x7f4680198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4680102740 0x7f468019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 --2- 172.21.15.52:0/1738760431 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46801030f0 0x7f468019ec00 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:28.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 >> 172.21.15.52:0/1738760431 conn(0x7f46800795f0 msgr2=0x7f4680100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:28.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 shutdown_connections 2024-06-02T23:30:28.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:28.954+0000 7f4684cef700 1 -- 172.21.15.52:0/1738760431 wait complete. 2024-06-02T23:30:29.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:30:29.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37293 6.10 MB/sec execute 217 sec latency 24007.082 ms 2024-06-02T23:30:29.679 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7a5988f4', total seconds elapsed: 32.7 2024-06-02T23:30:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:29.950 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:29 smithi052 ceph-mon[32340]: pgmap v989: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-06-02T23:30:30.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:29 smithi123 ceph-mon[35977]: pgmap v989: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-06-02T23:30:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:29 smithi174 ceph-mon[35994]: pgmap v989: 97 pgs: 97 active+clean; 425 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.6 KiB/s wr, 14 op/s 2024-06-02T23:30:30.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37486 6.11 MB/sec execute 218 sec latency 59.623 ms 2024-06-02T23:30:31.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37637 6.13 MB/sec execute 219 sec latency 59.433 ms 2024-06-02T23:30:31.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 -- 172.21.15.52:0/2143029906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548108510 msgr2=0x7f65481088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 --2- 172.21.15.52:0/2143029906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548108510 0x7f65481088f0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6538009990 tx=0x7f6538009ca0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 -- 172.21.15.52:0/2143029906 shutdown_connections 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 --2- 172.21.15.52:0/2143029906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548101440 0x7f654810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 --2- 172.21.15.52:0/2143029906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548108510 0x7f65481088f0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 --2- 172.21.15.52:0/2143029906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f6548100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 -- 172.21.15.52:0/2143029906 >> 172.21.15.52:0/2143029906 conn(0x7f6548077a00 msgr2=0x7f65480fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 -- 172.21.15.52:0/2143029906 shutdown_connections 2024-06-02T23:30:31.684 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.684+0000 7f654df34700 1 -- 172.21.15.52:0/2143029906 wait complete. 2024-06-02T23:30:31.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 Processor -- start 2024-06-02T23:30:31.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 -- start start 2024-06-02T23:30:31.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f654819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548101440 0x7f654819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548108510 0x7f654819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6548112670 con 0x7f6548101440 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f65477fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f654819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f6546ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548101440 0x7f654819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f65477fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f654819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36760/0 (socket says 172.21.15.52:36760) 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f65477fe700 1 -- 172.21.15.52:0/2840696592 learned_addr learned my addr 172.21.15.52:0/2840696592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65481124f0 con 0x7f6548108510 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65481127f0 con 0x7f6548100a80 2024-06-02T23:30:31.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.685+0000 7f6547fff700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548108510 0x7f654819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6547fff700 1 -- 172.21.15.52:0/2840696592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 msgr2=0x7f654819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6547fff700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f654819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6547fff700 1 -- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548101440 msgr2=0x7f654819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6547fff700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548101440 0x7f654819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6547fff700 1 -- 172.21.15.52:0/2840696592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654819fe10 con 0x7f6548108510 2024-06-02T23:30:31.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f65477fe700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f654819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6547fff700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548108510 0x7f654819f6d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f654000efc0 tx=0x7f654000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6544ff9700 1 -- 172.21.15.52:0/2840696592 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400092e0 con 0x7f6548108510 2024-06-02T23:30:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6544ff9700 1 -- 172.21.15.52:0/2840696592 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6540004bb0 con 0x7f6548108510 2024-06-02T23:30:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.686+0000 7f6544ff9700 1 -- 172.21.15.52:0/2840696592 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540004020 con 0x7f6548108510 2024-06-02T23:30:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.687+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65481a00a0 con 0x7f6548108510 2024-06-02T23:30:31.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.687+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65481a04d0 con 0x7f6548108510 2024-06-02T23:30:31.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.689+0000 7f6544ff9700 1 -- 172.21.15.52:0/2840696592 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f654001b030 con 0x7f6548108510 2024-06-02T23:30:31.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.690+0000 7f6544ff9700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65300a2ca0 0x7f65300a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.690+0000 7f65477fe700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65300a2ca0 0x7f65300a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.690+0000 7f6544ff9700 1 -- 172.21.15.52:0/2840696592 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f65400cea10 con 0x7f6548108510 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 Processor -- start 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 -- start start 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65477fe700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65300a2ca0 0x7f65300a5160 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6538006010 tx=0x7f653800b280 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808231f60 0x55c808232340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80822c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 0x55c80822cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c80811ef60 con 0x55c808231f60 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c80817d850 con 0x55c80822c7a0 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f65587b7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c8080e5460 con 0x55c808232880 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f6547fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 0x55c80822cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f6547fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 0x55c80822cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57438/0 (socket says 172.21.15.52:57438) 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.691+0000 7f6547fff700 1 -- 172.21.15.52:0/3109438636 learned_addr learned my addr 172.21.15.52:0/3109438636 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f65477fe700 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808231f60 0x55c808232340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6546ffd700 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80822c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6547fff700 1 -- 172.21.15.52:0/3109438636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 msgr2=0x55c80822c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6547fff700 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80822c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6547fff700 1 -- 172.21.15.52:0/3109438636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808231f60 msgr2=0x55c808232340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6547fff700 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808231f60 0x55c808232340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.692 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6547fff700 1 -- 172.21.15.52:0/3109438636 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8080cc8d0 con 0x55c80822c7a0 2024-06-02T23:30:31.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f6547fff700 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 0x55c80822cbc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f654009b2c0 tx=0x7f654009ac90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a4d95d12e199ad2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:31.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f652dffb700 1 -- 172.21.15.52:0/3109438636 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654009ba30 con 0x55c80822c7a0 2024-06-02T23:30:31.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f652dffb700 1 -- 172.21.15.52:0/3109438636 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f654009b330 con 0x55c80822c7a0 2024-06-02T23:30:31.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.692+0000 7f652dffb700 1 -- 172.21.15.52:0/3109438636 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400bb320 con 0x55c80822c7a0 2024-06-02T23:30:31.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 -- 172.21.15.52:0/3109438636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 msgr2=0x55c80822cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.694 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 0x55c80822cbc0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f654009b2c0 tx=0x7f654009ac90 comp rx=0 tx=0).stop 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 -- 172.21.15.52:0/3109438636 shutdown_connections 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c80822c7a0 0x55c80822cbc0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808231f60 0x55c808232340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 --2- 172.21.15.52:0/3109438636 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80822c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.693+0000 7f65587b7b80 1 -- 172.21.15.52:0/3109438636 >> 172.21.15.52:0/3109438636 conn(0x55c808159080 msgr2=0x55c808227220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.694+0000 7f65587b7b80 1 -- 172.21.15.52:0/3109438636 shutdown_connections 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.694+0000 7f65587b7b80 1 -- 172.21.15.52:0/3109438636 wait complete. 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.694+0000 7f65587b7b80 1 -- start start 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.694+0000 7f65587b7b80 1 Processor -- start 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c808238630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808235930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c808235e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6547fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c808235e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6547fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c808235e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53708/0 (socket says 172.21.15.52:53708) 2024-06-02T23:30:31.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6547fff700 1 -- 172.21.15.52:0/763692585 learned_addr learned my addr 172.21.15.52:0/763692585 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c80811dfc0 con 0x55c808232880 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c808179810 con 0x55c808231f60 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c808239260 con 0x55c80822c7a0 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808235930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65477fe700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c808238630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f65587b7b80 10 client.0 _ll_register_callbacks cb 0x55c8081135b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 -- 172.21.15.52:0/763692585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 msgr2=0x55c808238630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c808238630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 -- 172.21.15.52:0/763692585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 msgr2=0x55c808235e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c808235e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c808143750 con 0x55c808231f60 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6546ffd700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808235930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f653c009f20 tx=0x7f653c00b5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d23e7db0c0655050 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:31.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.695+0000 7f6526ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.696+0000 7f6526ffd700 1 -- 172.21.15.52:0/763692585 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653c004250 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.696+0000 7f6526ffd700 1 -- 172.21.15.52:0/763692585 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f653c0043f0 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.696+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f654009b330 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.696+0000 7f6526ffd700 1 -- 172.21.15.52:0/763692585 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f653c00be10 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.696+0000 7f65587b7b80 10 client.29732 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.696+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c808243820 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f6526ffd700 1 -- 172.21.15.52:0/763692585 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f653c024070 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f6526ffd700 1 -- 172.21.15.52:0/763692585 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f653c01f7a0 con 0x55c808231f60 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65587b7b80 10 client.29732 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65587b7b80 10 client.29732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65587b7b80 20 client.29732 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65587b7b80 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c808245eb0 0x55c8082482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65587b7b80 4 client.29732 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:31.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f654009b330 con 0x55c808245eb0 2024-06-02T23:30:31.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.697+0000 7f65477fe700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c808245eb0 0x55c8082482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.698+0000 7f6526ffd700 10 client.29732 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:31.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.698+0000 7f65477fe700 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c808245eb0 0x55c8082482d0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4ff8628738637a5 server_cookie=eea3b4484096bbbb in_seq=0 out_seq=0 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f6526ffd700 1 -- 172.21.15.52:0/763692585 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f654009b330 con 0x55c808245eb0 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f6526ffd700 10 client.29732 handle_command_reply: tid=0 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 client.29732 shutdown 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 msgr2=0x55c808235930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808235930 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f653c009f20 tx=0x7f653c00b5a0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 shutdown_connections 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c808245eb0 0x55c8082482d0 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c808235e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c808238630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 --2- 172.21.15.52:0/763692585 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808235930 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 >> 172.21.15.52:0/763692585 conn(0x55c808159080 msgr2=0x55c808226700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f6526ffd700 0 client.29732 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.699+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 shutdown_connections 2024-06-02T23:30:31.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.700+0000 7f65587b7b80 1 -- 172.21.15.52:0/763692585 wait complete. 2024-06-02T23:30:31.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.700+0000 7f65587b7b80 20 client.29732 trim_cache size 0 max 16384 2024-06-02T23:30:31.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 Processor -- start 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 -- start start 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c80822c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808226e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c8082273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65481aa860 con 0x55c808232880 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65481ac3a0 con 0x55c808231f60 2024-06-02T23:30:31.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65587b7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65481aaf30 con 0x55c80822c7a0 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f6546ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808226e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65477fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c80822c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65477fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c80822c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36796/0 (socket says 172.21.15.52:36796) 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f6546ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808226e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57446/0 (socket says 172.21.15.52:57446) 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f6547fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c8082273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.729+0000 7f65477fe700 1 -- 172.21.15.52:0/1987655197 learned_addr learned my addr 172.21.15.52:0/1987655197 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f65477fe700 1 -- 172.21.15.52:0/1987655197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 msgr2=0x55c808226e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f65477fe700 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808226e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f65477fe700 1 -- 172.21.15.52:0/1987655197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 msgr2=0x55c8082273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f65477fe700 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c8082273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f65477fe700 1 -- 172.21.15.52:0/1987655197 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c808232680 con 0x55c80822c7a0 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f65477fe700 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c80822c260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f653800b780 tx=0x7f6538021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a101cd047696f58 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f652d7fa700 1 -- 172.21.15.52:0/1987655197 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538003da0 con 0x55c80822c7a0 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f652d7fa700 1 -- 172.21.15.52:0/1987655197 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f653800f910 con 0x55c80822c7a0 2024-06-02T23:30:31.731 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.730+0000 7f652d7fa700 1 -- 172.21.15.52:0/1987655197 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538022830 con 0x55c80822c7a0 2024-06-02T23:30:31.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 -- 172.21.15.52:0/1987655197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 msgr2=0x55c80822c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c80822c260 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f653800b780 tx=0x7f6538021000 comp rx=0 tx=0).stop 2024-06-02T23:30:31.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 -- 172.21.15.52:0/1987655197 shutdown_connections 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c808232880 0x55c8082273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c80822c7a0 0x55c80822c260 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 --2- 172.21.15.52:0/1987655197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c808226e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 -- 172.21.15.52:0/1987655197 >> 172.21.15.52:0/1987655197 conn(0x55c808159080 msgr2=0x55c8082379f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.731+0000 7f65587b7b80 1 -- 172.21.15.52:0/1987655197 shutdown_connections 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 -- 172.21.15.52:0/1987655197 wait complete. 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 -- start start 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 Processor -- start 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c80822c7a0 0x55c80823c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c80823c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80823cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f65481abcd0 con 0x55c80822c7a0 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65481ab600 con 0x55c808231f60 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f654810d2f0 con 0x55c808232880 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.732+0000 7f65587b7b80 10 client.0 _ll_register_callbacks cb 0x55c8081135b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f65477fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c80822c7a0 0x55c80823c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f65477fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c80822c7a0 0x55c80823c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:53736/0 (socket says 172.21.15.52:53736) 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f65477fe700 1 -- 172.21.15.52:0/2165061478 learned_addr learned my addr 172.21.15.52:0/2165061478 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6546ffd700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c80823c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80823cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 -- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 msgr2=0x55c80823c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c80823c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 -- 172.21.15.52:0/2165061478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c80822c7a0 msgr2=0x55c80823c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c80822c7a0 0x55c80823c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 -- 172.21.15.52:0/2165061478 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8082afb40 con 0x55c808232880 2024-06-02T23:30:31.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6547fff700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80823cd10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f65400cc590 tx=0x7f654000ca50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28a20dd82bee356a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:31.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.733+0000 7f6526ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:31.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.734+0000 7f6526ffd700 1 -- 172.21.15.52:0/2165061478 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65400b9060 con 0x55c808232880 2024-06-02T23:30:31.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.734+0000 7f6526ffd700 1 -- 172.21.15.52:0/2165061478 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f65400bb050 con 0x55c808232880 2024-06-02T23:30:31.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.734+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f653800f910 con 0x55c808232880 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f6526ffd700 1 -- 172.21.15.52:0/2165061478 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f654009ba30 con 0x55c808232880 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f65587b7b80 10 client.29365 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c8082b0e30 con 0x55c808232880 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f6526ffd700 1 -- 172.21.15.52:0/2165061478 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f653800f910 con 0x55c808232880 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f6526ffd700 1 -- 172.21.15.52:0/2165061478 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f654007e020 con 0x55c808232880 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f65587b7b80 10 client.29365 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f65587b7b80 10 client.29365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:31.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.739+0000 7f65587b7b80 20 client.29365 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:31.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f65587b7b80 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8082b1060 0x55c8082b3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:31.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f65587b7b80 4 client.29365 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:30:31.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c80818ad50 con 0x55c8082b1060 2024-06-02T23:30:31.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f65477fe700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8082b1060 0x55c8082b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:31.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f6526ffd700 10 client.29365 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:31.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f65477fe700 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8082b1060 0x55c8082b3450 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d6d3fdaa97698f server_cookie=ed9f9f31de9e6055 in_seq=0 out_seq=0 2024-06-02T23:30:31.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f6526ffd700 1 -- 172.21.15.52:0/2165061478 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c80818ad50 con 0x55c8082b1060 2024-06-02T23:30:31.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f6526ffd700 10 client.29365 handle_command_reply: tid=0 2024-06-02T23:30:31.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.740+0000 7f65587b7b80 1 client.29365 shutdown 2024-06-02T23:30:31.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 msgr2=0x55c80823cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80823cd10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f65400cc590 tx=0x7f654000ca50 comp rx=0 tx=0).stop 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 shutdown_connections 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c8082b1060 0x55c8082b3450 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c808232880 0x55c80823cd10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c80822c7a0 0x55c80823c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 --2- 172.21.15.52:0/2165061478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c808231f60 0x55c80823c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 >> 172.21.15.52:0/2165061478 conn(0x55c808159080 msgr2=0x55c808237160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f6526ffd700 0 client.29365 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 shutdown_connections 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 1 -- 172.21.15.52:0/2165061478 wait complete. 2024-06-02T23:30:31.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.741+0000 7f65587b7b80 20 client.29365 trim_cache size 0 max 16384 2024-06-02T23:30:31.743 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:30:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:30:31.743 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:30:31.743 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65300a2ca0 msgr2=0x7f65300a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65300a2ca0 0x7f65300a5160 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f6538006010 tx=0x7f653800b280 comp rx=0 tx=0).stop 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548108510 msgr2=0x7f654819f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548108510 0x7f654819f6d0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f654000efc0 tx=0x7f654000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 shutdown_connections 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f65300a2ca0 0x7f65300a5160 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6548108510 0x7f654819f6d0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6548100a80 0x7f654819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 --2- 172.21.15.52:0/2840696592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6548101440 0x7f654819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.742+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 >> 172.21.15.52:0/2840696592 conn(0x7f6548077a00 msgr2=0x7f65480fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.743+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 shutdown_connections 2024-06-02T23:30:31.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:31.743+0000 7f654df34700 1 -- 172.21.15.52:0/2840696592 wait complete. 2024-06-02T23:30:31.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:31 smithi052 ceph-mon[32340]: pgmap v990: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 104 KiB/s wr, 17 op/s 2024-06-02T23:30:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:31 smithi123 ceph-mon[35977]: pgmap v990: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 104 KiB/s wr, 17 op/s 2024-06-02T23:30:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:31 smithi174 ceph-mon[35994]: pgmap v990: 97 pgs: 97 active+clean; 426 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 104 KiB/s wr, 17 op/s 2024-06-02T23:30:32.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 37795 6.14 MB/sec execute 220 sec latency 61.311 ms 2024-06-02T23:30:32.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:30:32.551 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:30:32.917 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:33.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 38006 6.19 MB/sec execute 221 sec latency 61.771 ms 2024-06-02T23:30:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:33 smithi052 ceph-mon[32340]: pgmap v991: 97 pgs: 97 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 408 KiB/s wr, 44 op/s 2024-06-02T23:30:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:33 smithi123 ceph-mon[35977]: pgmap v991: 97 pgs: 97 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 408 KiB/s wr, 44 op/s 2024-06-02T23:30:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:33 smithi174 ceph-mon[35994]: pgmap v991: 97 pgs: 97 active+clean; 429 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 157 KiB/s rd, 408 KiB/s wr, 44 op/s 2024-06-02T23:30:34.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 38243 6.21 MB/sec execute 222 sec latency 59.355 ms 2024-06-02T23:30:34.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 -- 172.21.15.52:0/2847769231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 msgr2=0x7fb03010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:34.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 --2- 172.21.15.52:0/2847769231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 0x7fb03010f3a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7fb02800b0d0 tx=0x7fb02800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:30:34.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 -- 172.21.15.52:0/2847769231 shutdown_connections 2024-06-02T23:30:34.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 --2- 172.21.15.52:0/2847769231 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 0x7fb03010f3a0 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:34.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 --2- 172.21.15.52:0/2847769231 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb0301085d0 0x7fb0301089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:34.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 --2- 172.21.15.52:0/2847769231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb030102530 0x7fb0301029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:34.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 -- 172.21.15.52:0/2847769231 >> 172.21.15.52:0/2847769231 conn(0x7fb030079640 msgr2=0x7fb030079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:34.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 -- 172.21.15.52:0/2847769231 shutdown_connections 2024-06-02T23:30:34.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.950+0000 7fb03846b700 1 -- 172.21.15.52:0/2847769231 wait complete. 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.951+0000 7fb03846b700 1 Processor -- start 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.951+0000 7fb03846b700 1 -- start start 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb03846b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 0x7fb03019e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb03846b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 0x7fb03019ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb03846b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0301085d0 0x7fb0301988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb03846b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb0301126e0 con 0x7fb030102ef0 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb03846b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb030112560 con 0x7fb0301085d0 2024-06-02T23:30:34.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb03846b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb030112860 con 0x7fb030102530 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 0x7fb03019e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 0x7fb03019e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56784/0 (socket says 172.21.15.52:56784) 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 -- 172.21.15.52:0/128937163 learned_addr learned my addr 172.21.15.52:0/128937163 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb035a06700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 0x7fb03019ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036a08700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0301085d0 0x7fb0301988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 -- 172.21.15.52:0/128937163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0301085d0 msgr2=0x7fb0301988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0301085d0 0x7fb0301988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 -- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 msgr2=0x7fb03019ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 0x7fb03019ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 -- 172.21.15.52:0/128937163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb030199040 con 0x7fb030102530 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036a08700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0301085d0 0x7fb0301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.952+0000 7fb036207700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 0x7fb03019e720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e9c0 tx=0x7fb02c00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.953+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c00cbd0 con 0x7fb030102530 2024-06-02T23:30:34.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.953+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb02c00cd70 con 0x7fb030102530 2024-06-02T23:30:34.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.953+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb030199330 con 0x7fb030102530 2024-06-02T23:30:34.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.953+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0056c0 con 0x7fb030102530 2024-06-02T23:30:34.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.953+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0301a55f0 con 0x7fb030102530 2024-06-02T23:30:34.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.955+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb030103c40 con 0x7fb030102530 2024-06-02T23:30:34.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.956+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb02c004750 con 0x7fb030102530 2024-06-02T23:30:34.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.956+0000 7fb0277fe700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb01c0a2c20 0x7fb01c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:34.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.957+0000 7fb035a06700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb01c0a2c20 0x7fb01c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:34.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.957+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb02c0cf2f0 con 0x7fb030102530 2024-06-02T23:30:34.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.957+0000 7fb035a06700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb01c0a2c20 0x7fb01c0a50e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb020009680 tx=0x7fb020009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:34.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:34.961+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb02c09c280 con 0x7fb030102530 2024-06-02T23:30:35.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.429+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb03002dca0 con 0x7fb030102530 2024-06-02T23:30:35.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.430+0000 7fb0277fe700 1 -- 172.21.15.52:0/128937163 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7fb02c09bb80 con 0x7fb030102530 2024-06-02T23:30:35.429 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:30:35.430 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb01c0a2c20 msgr2=0x7fb01c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb01c0a2c20 0x7fb01c0a50e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fb020009680 tx=0x7fb020009260 comp rx=0 tx=0).stop 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 msgr2=0x7fb03019e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 0x7fb03019e720 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00e9c0 tx=0x7fb02c00ecd0 comp rx=0 tx=0).stop 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 shutdown_connections 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb01c0a2c20 0x7fb01c0a50e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb0301085d0 0x7fb0301988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:35.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb030102530 0x7fb03019e720 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:35.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 --2- 172.21.15.52:0/128937163 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb030102ef0 0x7fb03019ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:35.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.433+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 >> 172.21.15.52:0/128937163 conn(0x7fb030079640 msgr2=0x7fb030100f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:35.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.434+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 shutdown_connections 2024-06-02T23:30:35.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:35.434+0000 7fb03846b700 1 -- 172.21.15.52:0/128937163 wait complete. 2024-06-02T23:30:35.434 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:30:35.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 38376 6.21 MB/sec execute 223 sec latency 90.658 ms 2024-06-02T23:30:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:35 smithi123 ceph-mon[35977]: pgmap v992: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 787 KiB/s wr, 50 op/s 2024-06-02T23:30:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:35 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/128937163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:36.122 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:35 smithi052 ceph-mon[32340]: pgmap v992: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 787 KiB/s wr, 50 op/s 2024-06-02T23:30:36.122 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:35 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/128937163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:35 smithi174 ceph-mon[35994]: pgmap v992: 97 pgs: 97 active+clean; 431 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 787 KiB/s wr, 50 op/s 2024-06-02T23:30:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:35 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/128937163' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:30:36.464 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:30:36.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 38498 6.21 MB/sec execute 224 sec latency 72.443 ms 2024-06-02T23:30:36.859 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:37.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 38691 6.26 MB/sec execute 225 sec latency 59.914 ms 2024-06-02T23:30:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:37 smithi123 ceph-mon[35977]: pgmap v993: 97 pgs: 97 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-06-02T23:30:38.128 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:37 smithi052 ceph-mon[32340]: pgmap v993: 97 pgs: 97 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-06-02T23:30:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:37 smithi174 ceph-mon[35994]: pgmap v993: 97 pgs: 97 active+clean; 440 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 2.2 MiB/s wr, 137 op/s 2024-06-02T23:30:38.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 38893 6.28 MB/sec execute 226 sec latency 51.857 ms 2024-06-02T23:30:38.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 -- 172.21.15.52:0/3451251125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0102740 msgr2=0x7fc1d0102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3451251125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0102740 0x7fc1d0102b20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009990 tx=0x7fc1cc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 -- 172.21.15.52:0/3451251125 shutdown_connections 2024-06-02T23:30:38.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3451251125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d0107840 0x7fc1d010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3451251125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0102740 0x7fc1d0102b20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3451251125 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d01030f0 0x7fc1d0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 -- 172.21.15.52:0/3451251125 >> 172.21.15.52:0/3451251125 conn(0x7fc1d00795f0 msgr2=0x7fc1d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 -- 172.21.15.52:0/3451251125 shutdown_connections 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.826+0000 7fc1d7808700 1 -- 172.21.15.52:0/3451251125 wait complete. 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.827+0000 7fc1d7808700 1 Processor -- start 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.827+0000 7fc1d7808700 1 -- start start 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d7808700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 0x7fc1d01a0710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d7808700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01030f0 0x7fc1d01a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d7808700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0107840 0x7fc1d019a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d7808700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0075db0 con 0x7fc1d0102740 2024-06-02T23:30:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d7808700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0075c30 con 0x7fc1d01030f0 2024-06-02T23:30:38.829 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d7808700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0075f30 con 0x7fc1d0107840 2024-06-02T23:30:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 0x7fc1d01a0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d4da3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01030f0 0x7fc1d01a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 0x7fc1d01a0710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60218/0 (socket says 172.21.15.52:60218) 2024-06-02T23:30:38.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3657425332 learned_addr learned my addr 172.21.15.52:0/3657425332 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d5da5700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0107840 0x7fc1d019a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3657425332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0107840 msgr2=0x7fc1d019a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0107840 0x7fc1d019a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3657425332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01030f0 msgr2=0x7fc1d01a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01030f0 0x7fc1d01a0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.828+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3657425332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d019b030 con 0x7fc1d0102740 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.829+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 0x7fc1d01a0710 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc006010 tx=0x7fc1cc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.829+0000 7fc1c67fc700 1 -- 172.21.15.52:0/3657425332 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc027070 con 0x7fc1d0102740 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.829+0000 7fc1c67fc700 1 -- 172.21.15.52:0/3657425332 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1cc005da0 con 0x7fc1d0102740 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.829+0000 7fc1c67fc700 1 -- 172.21.15.52:0/3657425332 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc00f6c0 con 0x7fc1d0102740 2024-06-02T23:30:38.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.829+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d019b2c0 con 0x7fc1d0102740 2024-06-02T23:30:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.830+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d0110f20 con 0x7fc1d0102740 2024-06-02T23:30:38.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.833+0000 7fc1c67fc700 1 -- 172.21.15.52:0/3657425332 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc1cc02a070 con 0x7fc1d0102740 2024-06-02T23:30:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.833+0000 7fc1c67fc700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1bc0a2ca0 0x7fc1bc0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.834+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1bc0a2ca0 0x7fc1bc0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.834+0000 7fc1c67fc700 1 -- 172.21.15.52:0/3657425332 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc1cc0d0240 con 0x7fc1d0102740 2024-06-02T23:30:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.834+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1bc0a2ca0 0x7fc1bc0a5160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0108bb0 tx=0x7fc1c000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 Processor -- start 2024-06-02T23:30:38.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 -- start start 2024-06-02T23:30:38.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f0c3f27810 con 0x55f0c3fd5f20 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f0c3ecf720 con 0x55f0c3fdbf60 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f0c3f237d0 con 0x55f0c3fdb640 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1d4da3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1d4da3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59314/0 (socket says 172.21.15.52:59314) 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1d4da3700 1 -- 172.21.15.52:0/2140797906 learned_addr learned my addr 172.21.15.52:0/2140797906 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.835+0000 7fc1d5da5700 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d4da3700 1 -- 172.21.15.52:0/2140797906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 msgr2=0x55f0c3fdba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d4da3700 1 -- 172.21.15.52:0/2140797906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 msgr2=0x55f0c3fd6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d4da3700 1 -- 172.21.15.52:0/2140797906 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c3ec5f50 con 0x55f0c3fdbf60 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd59e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0010f70 tx=0x7fc1c0013000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=feb42d885365da5b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1bb7fe700 1 -- 172.21.15.52:0/2140797906 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c000be60 con 0x55f0c3fdbf60 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1bb7fe700 1 -- 172.21.15.52:0/2140797906 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c0011db0 con 0x55f0c3fdbf60 2024-06-02T23:30:38.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.836+0000 7fc1bb7fe700 1 -- 172.21.15.52:0/2140797906 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001c720 con 0x55f0c3fdbf60 2024-06-02T23:30:38.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.837+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2140797906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 msgr2=0x55f0c3fd59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.837+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd59e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0010f70 tx=0x7fc1c0013000 comp rx=0 tx=0).stop 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2140797906 shutdown_connections 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2140797906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd59e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2140797906 >> 172.21.15.52:0/2140797906 conn(0x55f0c3f00cc0 msgr2=0x55f0c3fd0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2140797906 shutdown_connections 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2140797906 wait complete. 2024-06-02T23:30:38.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 -- start start 2024-06-02T23:30:38.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.838+0000 7fc1e208bb80 1 Processor -- start 2024-06-02T23:30:38.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fdf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f0c3fe29d0 con 0x55f0c3fdbf60 2024-06-02T23:30:38.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f0c3ec7bf0 con 0x55f0c3fdb640 2024-06-02T23:30:38.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f0c3fe2b50 con 0x55f0c3fd5f20 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1e208bb80 10 client.0 _ll_register_callbacks cb 0x55f0c3ec1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56824/0 (socket says 172.21.15.52:56824) 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1d55a4700 1 -- 172.21.15.52:0/2927137829 learned_addr learned my addr 172.21.15.52:0/2927137829 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1d5da5700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fdf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.839+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1d55a4700 1 -- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdb640 msgr2=0x55f0c3fdf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1d55a4700 1 -- 172.21.15.52:0/2927137829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 msgr2=0x55f0c3fdf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fdf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1d55a4700 1 -- 172.21.15.52:0/2927137829 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c3e768d0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe1d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc005560 tx=0x7fc1cc09d0a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a6b396dc163266 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/2927137829 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0095d0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/2927137829 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc1cc0bb2a0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c0011db0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/2927137829 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc0ac050 con 0x55f0c3fd5f20 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1e208bb80 10 client.29383 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.840+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f0c3fe2a30 con 0x55f0c3fd5f20 2024-06-02T23:30:38.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/2927137829 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c0011db0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/2927137829 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc1cc0a3110 con 0x55f0c3fd5f20 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1e208bb80 10 client.29383 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1e208bb80 10 client.29383 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1e208bb80 20 client.29383 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c3fef8e0 0x55f0c3ff1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1e208bb80 4 client.29383 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f0c3f34d10 con 0x55f0c3fef8e0 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.841+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c3fef8e0 0x55f0c3ff1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.842+0000 7fc1b8ff9700 10 client.29383 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:30:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.842+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c3fef8e0 0x55f0c3ff1cd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bca790acc77b27b6 server_cookie=2e9f1e4f6682249a in_seq=0 out_seq=0 2024-06-02T23:30:38.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/2927137829 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f0c3f34d10 con 0x55f0c3fef8e0 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1b8ff9700 10 client.29383 handle_command_reply: tid=0 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 client.29383 shutdown 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 msgr2=0x55f0c3fe1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe1d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc005560 tx=0x7fc1cc09d0a0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 shutdown_connections 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fdf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe1d50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c3fef8e0 0x55f0c3ff1cd0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/2927137829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1b8ff9700 0 client.29383 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.843+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 >> 172.21.15.52:0/2927137829 conn(0x55f0c3f00cc0 msgr2=0x55f0c3fd05f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:38.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.844+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 shutdown_connections 2024-06-02T23:30:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.844+0000 7fc1e208bb80 1 -- 172.21.15.52:0/2927137829 wait complete. 2024-06-02T23:30:38.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.844+0000 7fc1e208bb80 20 client.29383 trim_cache size 0 max 16384 2024-06-02T23:30:38.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 Processor -- start 2024-06-02T23:30:38.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 -- start start 2024-06-02T23:30:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01aa470 con 0x55f0c3fd5f20 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01abfb0 con 0x55f0c3fdbf60 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.866+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01aab40 con 0x55f0c3fdb640 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60238/0 (socket says 172.21.15.52:60238) 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56832/0 (socket says 172.21.15.52:56832) 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d55a4700 1 -- 172.21.15.52:0/218136712 learned_addr learned my addr 172.21.15.52:0/218136712 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d5da5700 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 -- 172.21.15.52:0/218136712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 msgr2=0x55f0c3fd0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 -- 172.21.15.52:0/218136712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 msgr2=0x55f0c3fd59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 -- 172.21.15.52:0/218136712 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f0c3fdbc70 con 0x55f0c3fdb640 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d5da5700 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdcc10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005950 tx=0x7fc1c0011bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd9486364da08580 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.867+0000 7fc1baffd700 1 -- 172.21.15.52:0/218136712 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0037020 con 0x55f0c3fdb640 2024-06-02T23:30:38.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1baffd700 1 -- 172.21.15.52:0/218136712 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c00056f0 con 0x55f0c3fdb640 2024-06-02T23:30:38.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1baffd700 1 -- 172.21.15.52:0/218136712 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c001e420 con 0x55f0c3fdb640 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 -- 172.21.15.52:0/218136712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 msgr2=0x55f0c3fdcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdcc10 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0005950 tx=0x7fc1c0011bb0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 -- 172.21.15.52:0/218136712 shutdown_connections 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fdbf60 0x55f0c3fd0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fd59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/218136712 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fdcc10 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.868+0000 7fc1e208bb80 1 -- 172.21.15.52:0/218136712 >> 172.21.15.52:0/218136712 conn(0x55f0c3f00cc0 msgr2=0x55f0c3fc9e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:38.876 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.869+0000 7fc1e208bb80 1 -- 172.21.15.52:0/218136712 shutdown_connections 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.869+0000 7fc1e208bb80 1 -- 172.21.15.52:0/218136712 wait complete. 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.869+0000 7fc1e208bb80 1 -- start start 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.869+0000 7fc1e208bb80 1 Processor -- start 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fe5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c40525f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01ab8e0 con 0x55f0c3fdbf60 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d01ab210 con 0x55f0c3fd5f20 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0114f60 con 0x55f0c3fdb640 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1e208bb80 10 client.0 _ll_register_callbacks cb 0x55f0c3ec1920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:38.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59336/0 (socket says 172.21.15.52:59336) 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3812424633 learned_addr learned my addr 172.21.15.52:0/3812424633 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fe5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d5da5700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c40525f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3812424633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 msgr2=0x55f0c3fe5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fe5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3812424633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 msgr2=0x55f0c40525f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c40525f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.870+0000 7fc1d55a4700 1 -- 172.21.15.52:0/3812424633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f0c4052c30 con 0x55f0c3fd5f20 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1d55a4700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe55a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc099530 tx=0x7fc1cc005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8469a9fc58896a35 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:38.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/3812424633 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1cc0095d0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/3812424633 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc1cc0a3110 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc1c00056f0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/3812424633 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc1cc09d260 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1e208bb80 10 client.29762 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f0c4059fc0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/3812424633 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc1c00056f0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/3812424633 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc1cc0bb2a0 con 0x55f0c3fd5f20 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.871+0000 7fc1e208bb80 10 client.29762 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1e208bb80 10 client.29762 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1e208bb80 20 client.29762 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c405a510 0x55f0c405c930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:38.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1e208bb80 4 client.29762 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f0c3f2f290 con 0x55f0c405a510 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c405a510 0x55f0c405c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1b8ff9700 10 client.29762 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1d4da3700 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c405a510 0x55f0c405c930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9e6d535dbd81615 server_cookie=6c04856809947f1b in_seq=0 out_seq=0 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1b8ff9700 1 -- 172.21.15.52:0/3812424633 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f0c3f2f290 con 0x55f0c405a510 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1b8ff9700 10 client.29762 handle_command_reply: tid=0 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.872+0000 7fc1e208bb80 1 client.29762 shutdown 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 msgr2=0x55f0c3fe55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe55a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc099530 tx=0x7fc1cc005450 comp rx=0 tx=0).stop 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 shutdown_connections 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f0c3fdbf60 0x55f0c40525f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55f0c405a510 0x55f0c405c930 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f0c3fd5f20 0x55f0c3fe55a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 --2- 172.21.15.52:0/3812424633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f0c3fdb640 0x55f0c3fe5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 >> 172.21.15.52:0/3812424633 conn(0x55f0c3f00cc0 msgr2=0x55f0c3fdca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1b8ff9700 0 client.29762 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 shutdown_connections 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 1 -- 172.21.15.52:0/3812424633 wait complete. 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.873+0000 7fc1e208bb80 20 client.29762 trim_cache size 0 max 16384 2024-06-02T23:30:38.881 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:30:38.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1bc0a2ca0 msgr2=0x7fc1bc0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1bc0a2ca0 0x7fc1bc0a5160 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fc1d0108bb0 tx=0x7fc1c000b410 comp rx=0 tx=0).stop 2024-06-02T23:30:38.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 msgr2=0x7fc1d01a0710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:38.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 0x7fc1d01a0710 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc006010 tx=0x7fc1cc00bda0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 shutdown_connections 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc1bc0a2ca0 0x7fc1bc0a5160 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1d0107840 0x7fc1d019a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc1d0102740 0x7fc1d01a0710 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 --2- 172.21.15.52:0/3657425332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1d01030f0 0x7fc1d01a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 >> 172.21.15.52:0/3657425332 conn(0x7fc1d00795f0 msgr2=0x7fc1d0100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 shutdown_connections 2024-06-02T23:30:38.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:38.874+0000 7fc1d7808700 1 -- 172.21.15.52:0/3657425332 wait complete. 2024-06-02T23:30:39.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39048 6.29 MB/sec execute 227 sec latency 107.619 ms 2024-06-02T23:30:39.832 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:30:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:39 smithi123 ceph-mon[35977]: pgmap v994: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 -- 172.21.15.52:0/257267197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c101a30 msgr2=0x7f2a5c101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 --2- 172.21.15.52:0/257267197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c101a30 0x7f2a5c101e50 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f2a58009900 tx=0x7f2a5802a410 comp rx=0 tx=0).stop 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 -- 172.21.15.52:0/257267197 shutdown_connections 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 --2- 172.21.15.52:0/257267197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c102c20 0x7f2a5c1030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 --2- 172.21.15.52:0/257267197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c1035e0 0x7f2a5c109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 --2- 172.21.15.52:0/257267197 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c101a30 0x7f2a5c101e50 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 -- 172.21.15.52:0/257267197 >> 172.21.15.52:0/257267197 conn(0x7f2a5c0fcfc0 msgr2=0x7f2a5c0ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 -- 172.21.15.52:0/257267197 shutdown_connections 2024-06-02T23:30:40.071 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.071+0000 7f2a63914640 1 -- 172.21.15.52:0/257267197 wait complete. 2024-06-02T23:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.072+0000 7f2a63914640 1 Processor -- start 2024-06-02T23:30:40.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.072+0000 7f2a63914640 1 -- start start 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.072+0000 7f2a63914640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c101a30 0x7f2a5c075700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.072+0000 7f2a63914640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c102c20 0x7f2a5c073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.072+0000 7f2a63914640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c1035e0 0x7f2a5c0742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a63914640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10d550 con 0x7f2a5c1035e0 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a63914640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10d3d0 con 0x7f2a5c102c20 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a63914640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c10d6d0 con 0x7f2a5c101a30 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a61eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c1035e0 0x7f2a5c0742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a616b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c101a30 0x7f2a5c075700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c102c20 0x7f2a5c073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a616b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c101a30 0x7f2a5c075700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56866/0 (socket says 172.21.15.52:56866) 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a616b0640 1 -- 172.21.15.52:0/622115443 learned_addr learned my addr 172.21.15.52:0/622115443 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 -- 172.21.15.52:0/622115443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c101a30 msgr2=0x7f2a5c075700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c101a30 0x7f2a5c075700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 -- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c1035e0 msgr2=0x7f2a5c0742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c1035e0 0x7f2a5c0742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 -- 172.21.15.52:0/622115443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a5c074810 con 0x7f2a5c102c20 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c1035e0 0x7f2a5c0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c102c20 0x7f2a5c073d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e9f0 tx=0x7f2a4c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a527fc640 1 -- 172.21.15.52:0/622115443 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00ccc0 con 0x7f2a5c102c20 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a5c074b00 con 0x7f2a5c102c20 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a527fc640 1 -- 172.21.15.52:0/622115443 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a4c005020 con 0x7f2a5c102c20 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a5c1a66c0 con 0x7f2a5c102c20 2024-06-02T23:30:40.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.073+0000 7f2a527fc640 1 -- 172.21.15.52:0/622115443 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00ce60 con 0x7f2a5c102c20 2024-06-02T23:30:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.076+0000 7f2a527fc640 1 -- 172.21.15.52:0/622115443 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2a4c0051d0 con 0x7f2a5c102c20 2024-06-02T23:30:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 Processor -- start 2024-06-02T23:30:40.076 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 -- start start 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 0x565306daa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306daf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306dafb70 0x565306db2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x565306d0d4f0 con 0x565306daa010 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565306cab870 con 0x565306dafb70 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a66626740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565306d094b0 con 0x565306daaa00 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a61eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 0x565306daa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a61eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 0x565306daa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60278/0 (socket says 172.21.15.52:60278) 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a61eb1640 1 -- 172.21.15.52:0/1591053052 learned_addr learned my addr 172.21.15.52:0/1591053052 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:40.078 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306daf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.077+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306dafb70 0x565306db2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a616b0640 1 -- 172.21.15.52:0/1591053052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306dafb70 msgr2=0x565306db2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306dafb70 0x565306db2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a616b0640 1 -- 172.21.15.52:0/1591053052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 msgr2=0x565306daa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 0x565306daa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a616b0640 1 -- 172.21.15.52:0/1591053052 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565306cc79b0 con 0x565306daaa00 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 0x565306daa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306dafb70 0x565306db2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a527fc640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a480a2c30 0x7f2a480a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306daf500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a58004770 tx=0x7f2a58031af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fee0d01ab7941fd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a480a2c30 0x7f2a480a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a527fc640 1 -- 172.21.15.52:0/622115443 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2a4c0cefc0 con 0x7f2a5c102c20 2024-06-02T23:30:40.079 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a477fe640 1 -- 172.21.15.52:0/1591053052 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58035030 con 0x565306daaa00 2024-06-02T23:30:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.078+0000 7f2a477fe640 1 -- 172.21.15.52:0/1591053052 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a58040070 con 0x565306daaa00 2024-06-02T23:30:40.080 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a477fe640 1 -- 172.21.15.52:0/1591053052 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58003970 con 0x565306daaa00 2024-06-02T23:30:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a66626740 1 -- 172.21.15.52:0/1591053052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 msgr2=0x565306daf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a66626740 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306daf500 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2a58004770 tx=0x7f2a58031af0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a480a2c30 0x7f2a480a50f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x565306dab260 tx=0x7f2a54009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a66626740 1 -- 172.21.15.52:0/1591053052 shutdown_connections 2024-06-02T23:30:40.083 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a66626740 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306daf500 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a66626740 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306dafb70 0x565306db2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.079+0000 7f2a66626740 1 --2- 172.21.15.52:0/1591053052 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306daa010 0x565306daa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 -- 172.21.15.52:0/1591053052 >> 172.21.15.52:0/1591053052 conn(0x565306da5680 msgr2=0x565306da5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 -- 172.21.15.52:0/1591053052 shutdown_connections 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 -- 172.21.15.52:0/1591053052 wait complete. 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 -- start start 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 Processor -- start 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 0x565306db4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daaa00 0x565306db4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.080+0000 7f2a66626740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x565306db2ac0 con 0x565306dafb70 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a66626740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565306c96ae0 con 0x565306daaa00 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a66626740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565306db2c40 con 0x565306daa010 2024-06-02T23:30:40.084 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 0x565306db4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a66626740 10 client.0 _ll_register_callbacks cb 0x565306ccf9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 0x565306db4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56876/0 (socket says 172.21.15.52:56876) 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 -- 172.21.15.52:0/352759268 learned_addr learned my addr 172.21.15.52:0/352759268 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daaa00 0x565306db4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 -- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daaa00 msgr2=0x565306db4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daaa00 0x565306db4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 -- 172.21.15.52:0/352759268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 msgr2=0x565306dbb7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 -- 172.21.15.52:0/352759268 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565306a21210 con 0x565306daa010 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a616b0640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 0x565306db4220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a58043000 tx=0x7f2a580317b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b536475228325ac5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.081+0000 7f2a44ff9640 1 -- 172.21.15.52:0/352759268 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a58052060 con 0x565306daa010 2024-06-02T23:30:40.085 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a44ff9640 1 -- 172.21.15.52:0/352759268 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2a5803c850 con 0x565306daa010 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a58040070 con 0x565306daa010 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a44ff9640 1 -- 172.21.15.52:0/352759268 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2a58035070 con 0x565306daa010 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 10 client.29401 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x565306aed620 con 0x565306daa010 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a44ff9640 1 -- 172.21.15.52:0/352759268 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a58040070 con 0x565306daa010 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a44ff9640 1 -- 172.21.15.52:0/352759268 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2a5804c030 con 0x565306daa010 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 10 client.29401 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 10 client.29401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 20 client.29401 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306dc2080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.082+0000 7f2a66626740 4 client.29401 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.083+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565306d20f10 con 0x565306dbfc90 2024-06-02T23:30:40.086 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.083+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306dc2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.083+0000 7f2a44ff9640 10 client.29401 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:40.087 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.083+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306dc2080 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c070d6955e78d75 server_cookie=7f80728995bf84ec in_seq=0 out_seq=0 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a44ff9640 1 -- 172.21.15.52:0/352759268 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565306d20f10 con 0x565306dbfc90 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a44ff9640 10 client.29401 handle_command_reply: tid=0 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a66626740 1 client.29401 shutdown 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 msgr2=0x565306db4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a66626740 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 0x565306db4220 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2a58043000 tx=0x7f2a580317b0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 shutdown_connections 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.084+0000 7f2a66626740 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306dc2080 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.088 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a44ff9640 0 client.29401 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daaa00 0x565306db4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 1 --2- 172.21.15.52:0/352759268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daa010 0x565306db4220 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 >> 172.21.15.52:0/352759268 conn(0x565306da5680 msgr2=0x565306da47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 shutdown_connections 2024-06-02T23:30:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 1 -- 172.21.15.52:0/352759268 wait complete. 2024-06-02T23:30:40.089 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.085+0000 7f2a66626740 20 client.29401 trim_cache size 0 max 16384 2024-06-02T23:30:40.114 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.115+0000 7f2a66626740 1 Processor -- start 2024-06-02T23:30:40.115 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a66626740 1 -- start start 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a66626740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306da8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a66626740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306da8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a66626740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a66626740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1af2f0 con 0x565306dafb70 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a61eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a61eb1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60290/0 (socket says 172.21.15.52:60290) 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a61eb1640 1 -- 172.21.15.52:0/1810787055 learned_addr learned my addr 172.21.15.52:0/1810787055 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306da8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.116 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.116+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306da8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b0e30 con 0x565306daa010 2024-06-02T23:30:40.117 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1af9c0 con 0x565306daaa00 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a61eb1640 1 -- 172.21.15.52:0/1810787055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 msgr2=0x565306da8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306da8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a61eb1640 1 -- 172.21.15.52:0/1810787055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 msgr2=0x565306da8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306da8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a61eb1640 1 -- 172.21.15.52:0/1810787055 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565306dab6d0 con 0x565306dafb70 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.117+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306da8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.118+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb760 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2a540120b0 tx=0x7f2a5401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e28cfdcfa2a64700 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.118+0000 7f2a46ffd640 1 -- 172.21.15.52:0/1810787055 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a54011a00 con 0x565306dafb70 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.118+0000 7f2a46ffd640 1 -- 172.21.15.52:0/1810787055 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2a54003e20 con 0x565306dafb70 2024-06-02T23:30:40.118 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.118+0000 7f2a46ffd640 1 -- 172.21.15.52:0/1810787055 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a540046d0 con 0x565306dafb70 2024-06-02T23:30:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.119+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 msgr2=0x565306dbb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.119+0000 7f2a66626740 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb760 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f2a540120b0 tx=0x7f2a5401a000 comp rx=0 tx=0).stop 2024-06-02T23:30:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 shutdown_connections 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306da8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306dbb760 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 --2- 172.21.15.52:0/1810787055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306da8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 >> 172.21.15.52:0/1810787055 conn(0x565306da5680 msgr2=0x565306da8360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 shutdown_connections 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 -- 172.21.15.52:0/1810787055 wait complete. 2024-06-02T23:30:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 -- start start 2024-06-02T23:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.120+0000 7f2a66626740 1 Processor -- start 2024-06-02T23:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.121+0000 7f2a66626740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306db83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.121+0000 7f2a66626740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306db8910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.121+0000 7f2a616b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306db83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.121+0000 7f2a616b0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306db83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59408/0 (socket says 172.21.15.52:59408) 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.121+0000 7f2a66626740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306e28820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.121+0000 7f2a616b0640 1 -- 172.21.15.52:0/1831920351 learned_addr learned my addr 172.21.15.52:0/1831920351 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306db8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b0760 con 0x565306dafb70 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1b0090 con 0x565306daa010 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2a5c1ad2d0 con 0x565306daaa00 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a61eb1640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306e28820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a616b0640 1 -- 172.21.15.52:0/1831920351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 msgr2=0x565306db8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306db8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a616b0640 1 -- 172.21.15.52:0/1831920351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 msgr2=0x565306e28820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306e28820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a616b0640 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565306dbb970 con 0x565306daa010 2024-06-02T23:30:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a66626740 10 client.0 _ll_register_callbacks cb 0x7f2a54009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a616b0640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306db83d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a58044aa0 tx=0x7f2a580314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef782fb4392aadcd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a44ff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.122+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a54003e20 con 0x565306daa010 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.123+0000 7f2a44ff9640 1 -- 172.21.15.52:0/1831920351 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a5805d070 con 0x565306daa010 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.123+0000 7f2a44ff9640 1 -- 172.21.15.52:0/1831920351 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2a5803c650 con 0x565306daa010 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.123+0000 7f2a44ff9640 1 -- 172.21.15.52:0/1831920351 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2a58069050 con 0x565306daa010 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.123+0000 7f2a66626740 10 client.29780 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.123+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=43}) -- 0x565306e29a50 con 0x565306daa010 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.123+0000 7f2a44ff9640 1 -- 172.21.15.52:0/1831920351 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2a5804c040 con 0x565306daa010 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a66626740 10 client.29780 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a66626740 10 client.29780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a66626740 20 client.29780 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a66626740 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306e2c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a66626740 4 client.29780 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:30:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f2a54003e20 con 0x565306dbfc90 2024-06-02T23:30:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.124+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306e2c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.125+0000 7f2a44ff9640 10 client.29780 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.125+0000 7f2a60eaf640 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306e2c2c0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724ef304f12f8bf1 server_cookie=e1a8a43186c8846a in_seq=0 out_seq=0 2024-06-02T23:30:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:39 smithi174 ceph-mon[35994]: pgmap v994: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-06-02T23:30:40.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:39 smithi052 ceph-mon[32340]: pgmap v994: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 2.6 MiB/s wr, 147 op/s 2024-06-02T23:30:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a44ff9640 1 -- 172.21.15.52:0/1831920351 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f2a54003e20 con 0x565306dbfc90 2024-06-02T23:30:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a44ff9640 10 client.29780 handle_command_reply: tid=0 2024-06-02T23:30:40.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 client.29780 shutdown 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 msgr2=0x565306db83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306db83d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f2a58044aa0 tx=0x7f2a580314a0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 shutdown_connections 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x565306dbfc90 0x565306e2c2c0 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a44ff9640 0 client.29780 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565306daaa00 0x565306db8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565306dafb70 0x565306e28820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.171+0000 7f2a66626740 1 --2- 172.21.15.52:0/1831920351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565306daa010 0x565306db83d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.172+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 >> 172.21.15.52:0/1831920351 conn(0x565306da5680 msgr2=0x565306da8bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.172+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 shutdown_connections 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.172+0000 7f2a66626740 1 -- 172.21.15.52:0/1831920351 wait complete. 2024-06-02T23:30:40.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.172+0000 7f2a66626740 20 client.29780 trim_cache size 0 max 16384 2024-06-02T23:30:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:30:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:30:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:30:40.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a480a2c30 msgr2=0x7f2a480a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a480a2c30 0x7f2a480a50f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x565306dab260 tx=0x7f2a54009500 comp rx=0 tx=0).stop 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c102c20 msgr2=0x7f2a5c073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c102c20 0x7f2a5c073d90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e9f0 tx=0x7f2a4c00edf0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 shutdown_connections 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2a480a2c30 0x7f2a480a50f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2a5c1035e0 0x7f2a5c0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2a5c102c20 0x7f2a5c073d90 secure :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e9f0 tx=0x7f2a4c00edf0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 --2- 172.21.15.52:0/622115443 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2a5c101a30 0x7f2a5c075700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 >> 172.21.15.52:0/622115443 conn(0x7f2a5c0fcfc0 msgr2=0x7f2a5c105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 shutdown_connections 2024-06-02T23:30:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:40.173+0000 7f2a63914640 1 -- 172.21.15.52:0/622115443 wait complete. 2024-06-02T23:30:40.329 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:40.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39190 6.29 MB/sec execute 228 sec latency 74.376 ms 2024-06-02T23:30:41.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39352 6.31 MB/sec execute 229 sec latency 84.363 ms 2024-06-02T23:30:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:41 smithi123 ceph-mon[35977]: pgmap v995: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.7 MiB/s wr, 185 op/s 2024-06-02T23:30:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:41 smithi174 ceph-mon[35994]: pgmap v995: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.7 MiB/s wr, 185 op/s 2024-06-02T23:30:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:41 smithi052 ceph-mon[32340]: pgmap v995: 97 pgs: 97 active+clean; 441 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 686 KiB/s rd, 3.7 MiB/s wr, 185 op/s 2024-06-02T23:30:42.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39492 6.31 MB/sec execute 230 sec latency 133.599 ms 2024-06-02T23:30:43.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39634 6.30 MB/sec execute 231 sec latency 166.390 ms 2024-06-02T23:30:44.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:43 smithi123 ceph-mon[35977]: pgmap v996: 97 pgs: 97 active+clean; 446 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 246 op/s 2024-06-02T23:30:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:43 smithi174 ceph-mon[35994]: pgmap v996: 97 pgs: 97 active+clean; 446 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 246 op/s 2024-06-02T23:30:44.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:43 smithi052 ceph-mon[32340]: pgmap v996: 97 pgs: 97 active+clean; 446 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 246 op/s 2024-06-02T23:30:44.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39769 6.29 MB/sec execute 232 sec latency 97.912 ms 2024-06-02T23:30:45.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 39924 6.30 MB/sec execute 233 sec latency 87.352 ms 2024-06-02T23:30:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:45 smithi123 ceph-mon[35977]: pgmap v997: 97 pgs: 97 active+clean; 446 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 237 op/s 2024-06-02T23:30:46.044 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:45 smithi052 ceph-mon[32340]: pgmap v997: 97 pgs: 97 active+clean; 446 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 237 op/s 2024-06-02T23:30:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:45 smithi174 ceph-mon[35994]: pgmap v997: 97 pgs: 97 active+clean; 446 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 237 op/s 2024-06-02T23:30:46.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 40086 6.32 MB/sec execute 234 sec latency 115.530 ms 2024-06-02T23:30:47.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 40233 6.33 MB/sec execute 235 sec latency 149.328 ms 2024-06-02T23:30:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:47 smithi123 ceph-mon[35977]: pgmap v998: 97 pgs: 97 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 318 op/s 2024-06-02T23:30:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:47 smithi174 ceph-mon[35994]: pgmap v998: 97 pgs: 97 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 318 op/s 2024-06-02T23:30:48.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:47 smithi052 ceph-mon[32340]: pgmap v998: 97 pgs: 97 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.9 MiB/s wr, 318 op/s 2024-06-02T23:30:48.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 40411 6.33 MB/sec execute 236 sec latency 67.708 ms 2024-06-02T23:30:48.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 -- 172.21.15.52:0/3374781999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 msgr2=0x7f46cc104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 --2- 172.21.15.52:0/3374781999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc104b20 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009990 tx=0x7f46bc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 -- 172.21.15.52:0/3374781999 shutdown_connections 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 --2- 172.21.15.52:0/3374781999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc105060 0x7f46cc111570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 --2- 172.21.15.52:0/3374781999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 0x7f46cc10aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 --2- 172.21.15.52:0/3374781999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc104b20 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 -- 172.21.15.52:0/3374781999 >> 172.21.15.52:0/3374781999 conn(0x7f46cc100170 msgr2=0x7f46cc102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 -- 172.21.15.52:0/3374781999 shutdown_connections 2024-06-02T23:30:48.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.568+0000 7f46d46e0700 1 -- 172.21.15.52:0/3374781999 wait complete. 2024-06-02T23:30:48.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d46e0700 1 Processor -- start 2024-06-02T23:30:48.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d46e0700 1 -- start start 2024-06-02T23:30:48.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d46e0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d46e0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc105060 0x7f46cc19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d46e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 0x7f46cc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d46e0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1148e0 con 0x7f46cc1046a0 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.569+0000 7f46d247c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d247c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc19ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47272/0 (socket says 172.21.15.52:47272) 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d247c700 1 -- 172.21.15.52:0/1602403697 learned_addr learned my addr 172.21.15.52:0/1602403697 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc105060 0x7f46cc19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 0x7f46cc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc114760 con 0x7f46cc10a6a0 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc114a60 con 0x7f46cc105060 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1602403697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc105060 msgr2=0x7f46cc19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc105060 0x7f46cc19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 msgr2=0x7f46cc19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1602403697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46cc19fd80 con 0x7f46cc10a6a0 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.570+0000 7f46d247c700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.571+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 0x7f46cc19f640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46c400efc0 tx=0x7f46c400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.571+0000 7f46c37fe700 1 -- 172.21.15.52:0/1602403697 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40092e0 con 0x7f46cc10a6a0 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.571+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46cc1a0010 con 0x7f46cc10a6a0 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.571+0000 7f46c37fe700 1 -- 172.21.15.52:0/1602403697 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c4004bb0 con 0x7f46cc10a6a0 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.571+0000 7f46c37fe700 1 -- 172.21.15.52:0/1602403697 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c4004020 con 0x7f46cc10a6a0 2024-06-02T23:30:48.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.571+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46cc1a78f0 con 0x7f46cc10a6a0 2024-06-02T23:30:48.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.574+0000 7f46c37fe700 1 -- 172.21.15.52:0/1602403697 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f46c401b030 con 0x7f46cc10a6a0 2024-06-02T23:30:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.575+0000 7f46c37fe700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f46b80a2ca0 0x7f46b80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.575+0000 7f46d247c700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f46b80a2ca0 0x7f46b80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.575 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.576+0000 7f46c37fe700 1 -- 172.21.15.52:0/1602403697 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f46c40ceb80 con 0x7f46cc10a6a0 2024-06-02T23:30:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.576+0000 7f46def63b80 1 Processor -- start 2024-06-02T23:30:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.576+0000 7f46d247c700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f46b80a2ca0 0x7f46b80a5160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46c8011550 tx=0x7f46c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:48.576 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.576+0000 7f46def63b80 1 -- start start 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46def63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 0x55b26f91c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46def63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91c880 0x55b26f916260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46def63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f916bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46d247c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 0x55b26f91c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46d247c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 0x55b26f91c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47284/0 (socket says 172.21.15.52:47284) 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46d247c700 1 -- 172.21.15.52:0/82698773 learned_addr learned my addr 172.21.15.52:0/82698773 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91c880 0x55b26f916260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46def63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b26f808f60 con 0x55b26f91bf60 2024-06-02T23:30:48.577 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b26f867850 con 0x55b26f9167a0 2024-06-02T23:30:48.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f916bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.577+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b26f7cf460 con 0x55b26f91c880 2024-06-02T23:30:48.578 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d1c7b700 1 -- 172.21.15.52:0/82698773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 msgr2=0x55b26f916bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f916bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d1c7b700 1 -- 172.21.15.52:0/82698773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 msgr2=0x55b26f91c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 0x55b26f91c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d1c7b700 1 -- 172.21.15.52:0/82698773 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26f7b68d0 con 0x55b26f91c880 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d247c700 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 0x55b26f91c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91c880 0x55b26f916260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006010 tx=0x7f46bc005bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5043e7cb733f7c41 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.578+0000 7f46c0ff9700 1 -- 172.21.15.52:0/82698773 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc027070 con 0x55b26f91c880 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.579+0000 7f46c0ff9700 1 -- 172.21.15.52:0/82698773 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46bc00bc90 con 0x55b26f91c880 2024-06-02T23:30:48.579 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.579+0000 7f46c0ff9700 1 -- 172.21.15.52:0/82698773 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc003a40 con 0x55b26f91c880 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91c880 msgr2=0x55b26f916260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91c880 0x55b26f916260 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f46bc006010 tx=0x7f46bc005bc0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 shutdown_connections 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f916bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91bf60 0x55b26f91c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 --2- 172.21.15.52:0/82698773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91c880 0x55b26f916260 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.580 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.580+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 >> 172.21.15.52:0/82698773 conn(0x55b26f843080 msgr2=0x55b26f911220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.581+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 shutdown_connections 2024-06-02T23:30:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.581+0000 7f46def63b80 1 -- 172.21.15.52:0/82698773 wait complete. 2024-06-02T23:30:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.581+0000 7f46def63b80 1 -- start start 2024-06-02T23:30:48.581 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.581+0000 7f46def63b80 1 Processor -- start 2024-06-02T23:30:48.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.582+0000 7f46def63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f922590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.582+0000 7f46def63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.582+0000 7f46d247c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f922590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.582+0000 7f46d247c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f922590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33178/0 (socket says 172.21.15.52:33178) 2024-06-02T23:30:48.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.582+0000 7f46d247c700 1 -- 172.21.15.52:0/1468275553 learned_addr learned my addr 172.21.15.52:0/1468275553 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:48.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.583+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.583+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f91fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.583+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b26f807fc0 con 0x55b26f91c880 2024-06-02T23:30:48.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.583+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f91fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b26f863810 con 0x55b26f9167a0 2024-06-02T23:30:48.583 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b26f923200 con 0x55b26f91bf60 2024-06-02T23:30:48.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46def63b80 10 client.0 _ll_register_callbacks cb 0x55b26f7fd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:48.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46d247c700 1 -- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 msgr2=0x55b26f91f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46d247c700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46d247c700 1 -- 172.21.15.52:0/1468275553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 msgr2=0x55b26f91fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46d247c700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f91fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.584+0000 7f46d247c700 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26f82d750 con 0x55b26f9167a0 2024-06-02T23:30:48.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46d247c700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f922590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46c800eec0 tx=0x7f46c80047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941478a647aa7711 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:48.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:48.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46b5ffb700 1 -- 172.21.15.52:0/1468275553 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c801fdb0 con 0x55b26f9167a0 2024-06-02T23:30:48.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46b5ffb700 1 -- 172.21.15.52:0/1468275553 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f46c80273e0 con 0x55b26f9167a0 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46bc00bc90 con 0x55b26f9167a0 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46b5ffb700 1 -- 172.21.15.52:0/1468275553 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f46c80275e0 con 0x55b26f9167a0 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46def63b80 10 client.29792 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55b26f936140 con 0x55b26f9167a0 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.585+0000 7f46b5ffb700 1 -- 172.21.15.52:0/1468275553 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f46c8026070 con 0x55b26f9167a0 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46def63b80 10 client.29792 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:48.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46def63b80 10 client.29792 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46def63b80 20 client.29792 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f938670 0x55b26f93aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46def63b80 4 client.29792 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b26f874d50 con 0x55b26f938670 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f938670 0x55b26f93aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46b5ffb700 10 client.29792 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.586+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f938670 0x55b26f93aaa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c2d63e9af3fa8080 server_cookie=92aad4e25053d2f6 in_seq=0 out_seq=0 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46b5ffb700 1 -- 172.21.15.52:0/1468275553 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b26f874d50 con 0x55b26f938670 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46b5ffb700 10 client.29792 handle_command_reply: tid=0 2024-06-02T23:30:48.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 client.29792 shutdown 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 msgr2=0x55b26f922590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f922590 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f46c800eec0 tx=0x7f46c80047c0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 shutdown_connections 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f91fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f9167a0 0x55b26f922590 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f938670 0x55b26f93aaa0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 --2- 172.21.15.52:0/1468275553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 >> 172.21.15.52:0/1468275553 conn(0x55b26f843080 msgr2=0x55b26f910700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.587+0000 7f46b5ffb700 0 client.29792 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.588+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 shutdown_connections 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.588+0000 7f46def63b80 1 -- 172.21.15.52:0/1468275553 wait complete. 2024-06-02T23:30:48.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.588+0000 7f46def63b80 20 client.29792 trim_cache size 0 max 16384 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 Processor -- start 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 -- start start 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f9167a0 0x55b26f91f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 0x55b26f910e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1aa800 con 0x55b26f9167a0 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1ac340 con 0x55b26f91c880 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46def63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1aaed0 con 0x55b26f91bf60 2024-06-02T23:30:48.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46d247c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f9167a0 0x55b26f91f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46d1c7b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46d2c7d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 0x55b26f910e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46d2c7d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 0x55b26f910e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33194/0 (socket says 172.21.15.52:33194) 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.609+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1636433343 learned_addr learned my addr 172.21.15.52:0/1636433343 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1636433343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 msgr2=0x55b26f91d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1636433343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f9167a0 msgr2=0x55b26f91f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f9167a0 0x55b26f91f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46d2c7d700 1 -- 172.21.15.52:0/1636433343 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b26f9129a0 con 0x55b26f91c880 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 0x55b26f910e80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46c40978a0 tx=0x7f46c401c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c9a6f0b937739fcf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46b7fff700 1 -- 172.21.15.52:0/1636433343 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c409bb40 con 0x55b26f91c880 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46b7fff700 1 -- 172.21.15.52:0/1636433343 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c409b440 con 0x55b26f91c880 2024-06-02T23:30:48.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.610+0000 7f46b7fff700 1 -- 172.21.15.52:0/1636433343 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c40a3df0 con 0x55b26f91c880 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 -- 172.21.15.52:0/1636433343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 msgr2=0x55b26f910e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 0x55b26f910e80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f46c40978a0 tx=0x7f46c401c000 comp rx=0 tx=0).stop 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 -- 172.21.15.52:0/1636433343 shutdown_connections 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91c880 0x55b26f910e80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f9167a0 0x55b26f91f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 --2- 172.21.15.52:0/1636433343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f91bf60 0x55b26f91d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 -- 172.21.15.52:0/1636433343 >> 172.21.15.52:0/1636433343 conn(0x55b26f843080 msgr2=0x55b26f91cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:48.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.611+0000 7f46def63b80 1 -- 172.21.15.52:0/1636433343 shutdown_connections 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 -- 172.21.15.52:0/1636433343 wait complete. 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 -- start start 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 Processor -- start 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 0x55b26f931300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91bf60 0x55b26f93fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f93ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1abc70 con 0x55b26f91c880 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc1ab5a0 con 0x55b26f91bf60 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46cc07a7a0 con 0x55b26f9167a0 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46def63b80 10 client.0 _ll_register_callbacks cb 0x55b26f7fd5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46d247c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 0x55b26f931300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46d247c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 0x55b26f931300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39164/0 (socket says 172.21.15.52:39164) 2024-06-02T23:30:48.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46d247c700 1 -- 172.21.15.52:0/2311877352 learned_addr learned my addr 172.21.15.52:0/2311877352 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46d2c7d700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f93ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.612+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91bf60 0x55b26f93fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d247c700 1 -- 172.21.15.52:0/2311877352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91bf60 msgr2=0x55b26f93fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d247c700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91bf60 0x55b26f93fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d247c700 1 -- 172.21.15.52:0/2311877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 msgr2=0x55b26f93ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d247c700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f93ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d247c700 1 -- 172.21.15.52:0/2311877352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b26f920870 con 0x55b26f9167a0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91bf60 0x55b26f93fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46d247c700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 0x55b26f931300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46c8027a80 tx=0x7f46c8016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d77e118bcd7f0a4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46b5ffb700 1 -- 172.21.15.52:0/2311877352 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8040060 con 0x55b26f9167a0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46b5ffb700 1 -- 172.21.15.52:0/2311877352 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f46c80273e0 con 0x55b26f9167a0 2024-06-02T23:30:48.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.613+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46c409b440 con 0x55b26f9167a0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46b5ffb700 1 -- 172.21.15.52:0/2311877352 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f46c801fdb0 con 0x55b26f9167a0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 10 client.29425 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b26f9a2f20 con 0x55b26f9167a0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46b5ffb700 1 -- 172.21.15.52:0/2311877352 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f46c409b440 con 0x55b26f9167a0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46b5ffb700 1 -- 172.21.15.52:0/2311877352 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f46c8026040 con 0x55b26f9167a0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 10 client.29425 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 10 client.29425 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 20 client.29425 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f9a35a0 0x55b26f9a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 4 client.29425 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b26f86f2d0 con 0x55b26f9a35a0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.614+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f9a35a0 0x55b26f9a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.615+0000 7f46d1c7b700 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f9a35a0 0x55b26f9a59c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d8514bbf2146523c server_cookie=f91594e68b72cfc3 in_seq=0 out_seq=0 2024-06-02T23:30:48.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.615+0000 7f46b5ffb700 10 client.29425 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:30:48.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.615+0000 7f46b5ffb700 1 -- 172.21.15.52:0/2311877352 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b26f86f2d0 con 0x55b26f9a35a0 2024-06-02T23:30:48.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.615+0000 7f46b5ffb700 10 client.29425 handle_command_reply: tid=0 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.615+0000 7f46def63b80 1 client.29425 shutdown 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 msgr2=0x55b26f931300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 0x55b26f931300 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f46c8027a80 tx=0x7f46c8016000 comp rx=0 tx=0).stop 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 shutdown_connections 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b26f91c880 0x55b26f93ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55b26f9a35a0 0x55b26f9a59c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b26f9167a0 0x55b26f931300 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46b5ffb700 0 client.29425 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 --2- 172.21.15.52:0/2311877352 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b26f91bf60 0x55b26f93fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 >> 172.21.15.52:0/2311877352 conn(0x55b26f843080 msgr2=0x55b26f921130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 shutdown_connections 2024-06-02T23:30:48.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 1 -- 172.21.15.52:0/2311877352 wait complete. 2024-06-02T23:30:48.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.616+0000 7f46def63b80 20 client.29425 trim_cache size 0 max 16384 2024-06-02T23:30:48.620 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f46b80a2ca0 msgr2=0x7f46b80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f46b80a2ca0 0x7f46b80a5160 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f46c8011550 tx=0x7f46c8009260 comp rx=0 tx=0).stop 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 msgr2=0x7f46cc19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 0x7f46cc19f640 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f46c400efc0 tx=0x7f46c400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 shutdown_connections 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f46b80a2ca0 0x7f46b80a5160 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46cc10a6a0 0x7f46cc19f640 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f46cc1046a0 0x7f46cc19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 --2- 172.21.15.52:0/1602403697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46cc105060 0x7f46cc19b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 >> 172.21.15.52:0/1602403697 conn(0x7f46cc100170 msgr2=0x7f46cc100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:48.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 shutdown_connections 2024-06-02T23:30:48.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:48.617+0000 7f46d46e0700 1 -- 172.21.15.52:0/1602403697 wait complete. 2024-06-02T23:30:49.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 40581 6.34 MB/sec execute 237 sec latency 69.653 ms 2024-06-02T23:30:49.649 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:30:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:49 smithi052 ceph-mon[32340]: pgmap v999: 97 pgs: 97 active+clean; 442 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 258 op/s 2024-06-02T23:30:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:49 smithi123 ceph-mon[35977]: pgmap v999: 97 pgs: 97 active+clean; 442 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 258 op/s 2024-06-02T23:30:50.086 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:49 smithi174 ceph-mon[35994]: pgmap v999: 97 pgs: 97 active+clean; 442 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.0 MiB/s wr, 258 op/s 2024-06-02T23:30:50.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 40776 6.34 MB/sec execute 238 sec latency 57.550 ms 2024-06-02T23:30:51.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 40957 6.36 MB/sec execute 239 sec latency 57.406 ms 2024-06-02T23:30:51.978 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:51 smithi052 ceph-mon[32340]: pgmap v1000: 97 pgs: 97 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-06-02T23:30:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:51 smithi123 ceph-mon[35977]: pgmap v1000: 97 pgs: 97 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-06-02T23:30:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:51 smithi174 ceph-mon[35994]: pgmap v1000: 97 pgs: 97 active+clean; 443 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 274 op/s 2024-06-02T23:30:52.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 -- 172.21.15.52:0/3019806528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 msgr2=0x7f33501088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 --2- 172.21.15.52:0/3019806528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f33501088d0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f33400098f0 tx=0x7f3340009c00 comp rx=0 tx=0).stop 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 -- 172.21.15.52:0/3019806528 shutdown_connections 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 --2- 172.21.15.52:0/3019806528 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 0x7f335010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 --2- 172.21.15.52:0/3019806528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f33501088d0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 --2- 172.21.15.52:0/3019806528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f335006aa00 0x7f335006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 -- 172.21.15.52:0/3019806528 >> 172.21.15.52:0/3019806528 conn(0x7f3350078300 msgr2=0x7f3350078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 -- 172.21.15.52:0/3019806528 shutdown_connections 2024-06-02T23:30:52.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.146+0000 7f3355418700 1 -- 172.21.15.52:0/3019806528 wait complete. 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.147+0000 7f3355418700 1 Processor -- start 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.147+0000 7f3355418700 1 -- start start 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f3355418700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f335006aa00 0x7f3350101450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f3355418700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 0x7f3350101990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f3355418700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f3350101ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f3355418700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33501126a0 con 0x7f335006b3c0 2024-06-02T23:30:52.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f33467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 0x7f3350101990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f33467fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 0x7f3350101990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47332/0 (socket says 172.21.15.52:47332) 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f33467fc700 1 -- 172.21.15.52:0/2195743407 learned_addr learned my addr 172.21.15.52:0/2195743407 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334effd700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f335006aa00 0x7f3350101450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f3355418700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3350112520 con 0x7f33501084f0 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3350112820 con 0x7f335006aa00 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f3350101ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334f7fe700 1 -- 172.21.15.52:0/2195743407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f335006aa00 msgr2=0x7f3350101450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f335006aa00 0x7f3350101450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334f7fe700 1 -- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 msgr2=0x7f3350101990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 0x7f3350101990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.148+0000 7f334f7fe700 1 -- 172.21.15.52:0/2195743407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33501a7140 con 0x7f33501084f0 2024-06-02T23:30:52.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.149+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f3350101ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3348009cd0 tx=0x7f334800c620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:52.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.149+0000 7f334cff9700 1 -- 172.21.15.52:0/2195743407 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334800cdc0 con 0x7f33501084f0 2024-06-02T23:30:52.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.149+0000 7f334cff9700 1 -- 172.21.15.52:0/2195743407 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3348003a00 con 0x7f33501084f0 2024-06-02T23:30:52.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.149+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33501a73d0 con 0x7f33501084f0 2024-06-02T23:30:52.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.149+0000 7f334cff9700 1 -- 172.21.15.52:0/2195743407 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33480045d0 con 0x7f33501084f0 2024-06-02T23:30:52.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.149+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33501a7910 con 0x7f33501084f0 2024-06-02T23:30:52.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.152+0000 7f334cff9700 1 -- 172.21.15.52:0/2195743407 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f334801a020 con 0x7f33501084f0 2024-06-02T23:30:52.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.153+0000 7f334cff9700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cf0 0x7f333c0a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.153+0000 7f334cff9700 1 -- 172.21.15.52:0/2195743407 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33480ce860 con 0x7f33501084f0 2024-06-02T23:30:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.153+0000 7f334effd700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cf0 0x7f333c0a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 Processor -- start 2024-06-02T23:30:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f334effd700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cf0 0x7f333c0a51b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3340009680 tx=0x7f3340019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:52.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 -- start start 2024-06-02T23:30:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffd69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55833ff28810 con 0x55833ffd6f20 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55833fed0720 con 0x55833ffdcf60 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.154+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55833ff247d0 con 0x55833ffdc640 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffd69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffd69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33218/0 (socket says 172.21.15.52:33218) 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 -- 172.21.15.52:0/2845176569 learned_addr learned my addr 172.21.15.52:0/2845176569 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f334effd700 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 -- 172.21.15.52:0/2845176569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 msgr2=0x55833ffdca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 -- 172.21.15.52:0/2845176569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 msgr2=0x55833ffd7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 -- 172.21.15.52:0/2845176569 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55833fec6f50 con 0x55833ffdcf60 2024-06-02T23:30:52.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33467fc700 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffd69e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d90 tx=0x7f3338011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83e08562032b4f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:52.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33457fa700 1 -- 172.21.15.52:0/2845176569 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338004220 con 0x55833ffdcf60 2024-06-02T23:30:52.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33457fa700 1 -- 172.21.15.52:0/2845176569 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33380043c0 con 0x55833ffdcf60 2024-06-02T23:30:52.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.155+0000 7f33457fa700 1 -- 172.21.15.52:0/2845176569 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33380055c0 con 0x55833ffdcf60 2024-06-02T23:30:52.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2845176569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 msgr2=0x55833ffd69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffd69e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3338009d90 tx=0x7f3338011720 comp rx=0 tx=0).stop 2024-06-02T23:30:52.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2845176569 shutdown_connections 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2845176569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffd69e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.156+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2845176569 >> 172.21.15.52:0/2845176569 conn(0x55833ff01cc0 msgr2=0x55833ffd1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.157+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2845176569 shutdown_connections 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.157+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2845176569 wait complete. 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.157+0000 7f335fc9bb80 1 -- start start 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.157+0000 7f335fc9bb80 1 Processor -- start 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffe2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffe0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55833ffe39d0 con 0x55833ffd6f20 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55833fec8bf0 con 0x55833ffdcf60 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55833ffe3b50 con 0x55833ffdc640 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f334effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffe2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f334effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffe2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47352/0 (socket says 172.21.15.52:47352) 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f334effd700 1 -- 172.21.15.52:0/2907319552 learned_addr learned my addr 172.21.15.52:0/2907319552 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f335fc9bb80 10 client.0 _ll_register_callbacks cb 0x55833fec2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f334f7fe700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffe0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f33467fc700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f33467fc700 1 -- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 msgr2=0x55833ffe0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f33467fc700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffe0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f33467fc700 1 -- 172.21.15.52:0/2907319552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 msgr2=0x55833ffe2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f33467fc700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffe2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.158+0000 7f33467fc700 1 -- 172.21.15.52:0/2907319552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55833fe778d0 con 0x55833ffdc640 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.159+0000 7f33467fc700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdfff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3338009880 tx=0x7f3338005470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad652f606680d272 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.159+0000 7f33367fc700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.159+0000 7f33367fc700 1 -- 172.21.15.52:0/2907319552 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3338028020 con 0x55833ffdc640 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.159+0000 7f33367fc700 1 -- 172.21.15.52:0/2907319552 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3338004960 con 0x55833ffdc640 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.159+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33380043c0 con 0x55833ffdc640 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.160+0000 7f33367fc700 1 -- 172.21.15.52:0/2907319552 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f333801d030 con 0x55833ffdc640 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.160+0000 7f335fc9bb80 10 client.29437 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:52.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.160+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55833fff6a60 con 0x55833ffdc640 2024-06-02T23:30:52.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.160+0000 7f33367fc700 1 -- 172.21.15.52:0/2907319552 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33380043c0 con 0x55833ffdc640 2024-06-02T23:30:52.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f33367fc700 1 -- 172.21.15.52:0/2907319552 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3338023070 con 0x55833ffdc640 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f335fc9bb80 10 client.29437 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f335fc9bb80 10 client.29437 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f335fc9bb80 20 client.29437 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x55833fffb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f335fc9bb80 4 client.29437 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55833ff35d10 con 0x55833fff93a0 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.161+0000 7f334effd700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x55833fffb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.162+0000 7f33367fc700 10 client.29437 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.162+0000 7f334effd700 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x55833fffb820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3213d4ba9fc923 server_cookie=5ad9cae5b5c9bc46 in_seq=0 out_seq=0 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f33367fc700 1 -- 172.21.15.52:0/2907319552 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55833ff35d10 con 0x55833fff93a0 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f33367fc700 10 client.29437 handle_command_reply: tid=0 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 client.29437 shutdown 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 msgr2=0x55833ffdfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.165 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdfff0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f3338009880 tx=0x7f3338005470 comp rx=0 tx=0).stop 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 shutdown_connections 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x55833fffb820 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffe0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f33367fc700 0 client.29437 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffe2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2907319552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffdfff0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.163+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 >> 172.21.15.52:0/2907319552 conn(0x55833ff01cc0 msgr2=0x55833ffd59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.164+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 shutdown_connections 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.164+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2907319552 wait complete. 2024-06-02T23:30:52.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.164+0000 7f335fc9bb80 20 client.29437 trim_cache size 0 max 16384 2024-06-02T23:30:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.186+0000 7f335fc9bb80 1 Processor -- start 2024-06-02T23:30:52.186 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.186+0000 7f335fc9bb80 1 -- start start 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffde8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffdee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33501aaa00 con 0x55833ffd6f20 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33501ac540 con 0x55833ffdcf60 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33501ab0d0 con 0x55833ffdc640 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47366/0 (socket says 172.21.15.52:47366) 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffdee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 -- 172.21.15.52:0/1124513508 learned_addr learned my addr 172.21.15.52:0/1124513508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f33467fc700 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffde8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 -- 172.21.15.52:0/1124513508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 msgr2=0x55833ffde8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffde8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 -- 172.21.15.52:0/1124513508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 msgr2=0x55833ffdee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffdee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334effd700 1 -- 172.21.15.52:0/1124513508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55833ffe3060 con 0x55833ffd6f20 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.187+0000 7f334f7fe700 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffdee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.188+0000 7f334effd700 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd69e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f334000f9b0 tx=0x7f3340009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21c9754bdb75386 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.188+0000 7f3344ff9700 1 -- 172.21.15.52:0/1124513508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334003e070 con 0x55833ffd6f20 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.188+0000 7f3344ff9700 1 -- 172.21.15.52:0/1124513508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f334000f9e0 con 0x55833ffd6f20 2024-06-02T23:30:52.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.188+0000 7f3344ff9700 1 -- 172.21.15.52:0/1124513508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334000b7c0 con 0x55833ffd6f20 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 -- 172.21.15.52:0/1124513508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 msgr2=0x55833ffd69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd69e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f334000f9b0 tx=0x7f3340009c00 comp rx=0 tx=0).stop 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 -- 172.21.15.52:0/1124513508 shutdown_connections 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x55833ffdee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833ffd69e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/1124513508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833ffde8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 -- 172.21.15.52:0/1124513508 >> 172.21.15.52:0/1124513508 conn(0x55833ff01cc0 msgr2=0x55833ffdffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:52.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 -- 172.21.15.52:0/1124513508 shutdown_connections 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.189+0000 7f335fc9bb80 1 -- 172.21.15.52:0/1124513508 wait complete. 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 -- start start 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 Processor -- start 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833fff16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833fff1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x558340000230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33501abe70 con 0x55833ffd6f20 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33501ab7a0 con 0x55833ffdcf60 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f335019c980 con 0x55833ffdc640 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f335fc9bb80 10 client.0 _ll_register_callbacks cb 0x55833fec2920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.190+0000 7f334effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833fff16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833fff16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47376/0 (socket says 172.21.15.52:47376) 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f33467fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833fff1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x558340000230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 -- 172.21.15.52:0/2080812555 learned_addr learned my addr 172.21.15.52:0/2080812555 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 -- 172.21.15.52:0/2080812555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 msgr2=0x55833fff1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833fff1be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 -- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 msgr2=0x558340000230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x558340000230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 -- 172.21.15.52:0/2080812555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558340000820 con 0x55833ffd6f20 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f33467fc700 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833fff1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f334effd700 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833fff16a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f334003c040 tx=0x7f3340005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd21b1f76a0ce26 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f33367fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.191+0000 7f33367fc700 1 -- 172.21.15.52:0/2080812555 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f334003e070 con 0x55833ffd6f20 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f33367fc700 1 -- 172.21.15.52:0/2080812555 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33400048d0 con 0x55833ffd6f20 2024-06-02T23:30:52.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f334000f9e0 con 0x55833ffd6f20 2024-06-02T23:30:52.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f33367fc700 1 -- 172.21.15.52:0/2080812555 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f33400056e0 con 0x55833ffd6f20 2024-06-02T23:30:52.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 10 client.20502 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:52.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558340063950 con 0x55833ffd6f20 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f33367fc700 1 -- 172.21.15.52:0/2080812555 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3340004ad0 con 0x55833ffd6f20 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f33367fc700 1 -- 172.21.15.52:0/2080812555 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f33400394b0 con 0x55833ffd6f20 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 10 client.20502 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 10 client.20502 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 20 client.20502 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x558340065f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 4 client.20502 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.192+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f334000f9e0 con 0x55833fff93a0 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.193+0000 7f33467fc700 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x558340065f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.193+0000 7f33367fc700 10 client.20502 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.193+0000 7f33467fc700 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x558340065f00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9da27e41b7ce581 server_cookie=276e94e3fc2861a7 in_seq=0 out_seq=0 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f33367fc700 1 -- 172.21.15.52:0/2080812555 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f334000f9e0 con 0x55833fff93a0 2024-06-02T23:30:52.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f33367fc700 10 client.20502 handle_command_reply: tid=0 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 client.20502 shutdown 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 msgr2=0x55833fff16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833fff16a0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f334003c040 tx=0x7f3340005a00 comp rx=0 tx=0).stop 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 shutdown_connections 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55833fff93a0 0x558340065f00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55833ffdcf60 0x558340000230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55833ffd6f20 0x55833fff16a0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 --2- 172.21.15.52:0/2080812555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55833ffdc640 0x55833fff1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f33367fc700 0 client.20502 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 >> 172.21.15.52:0/2080812555 conn(0x55833ff01cc0 msgr2=0x55833ffde880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 shutdown_connections 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 1 -- 172.21.15.52:0/2080812555 wait complete. 2024-06-02T23:30:52.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.194+0000 7f335fc9bb80 20 client.20502 trim_cache size 0 max 16384 2024-06-02T23:30:52.198 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cf0 msgr2=0x7f333c0a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cf0 0x7f333c0a51b0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f3340009680 tx=0x7f3340019040 comp rx=0 tx=0).stop 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 msgr2=0x7f3350101ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f3350101ed0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f3348009cd0 tx=0x7f334800c620 comp rx=0 tx=0).stop 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 shutdown_connections 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f333c0a2cf0 0x7f333c0a51b0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33501084f0 0x7f3350101ed0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f335006aa00 0x7f3350101450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 --2- 172.21.15.52:0/2195743407 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f335006b3c0 0x7f3350101990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:52.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 >> 172.21.15.52:0/2195743407 conn(0x7f3350078300 msgr2=0x7f33501000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:52.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 shutdown_connections 2024-06-02T23:30:52.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:52.195+0000 7f3355418700 1 -- 172.21.15.52:0/2195743407 wait complete. 2024-06-02T23:30:52.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 41094 6.36 MB/sec execute 240 sec latency 103.901 ms 2024-06-02T23:30:52.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:30:53.330 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 -- 172.21.15.52:0/3262825524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 msgr2=0x7f6ec0101e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/3262825524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0101e40 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc009860 tx=0x7f6ebc02a340 comp rx=0 tx=0).stop 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 -- 172.21.15.52:0/3262825524 shutdown_connections 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/3262825524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec01035d0 0x7f6ec0109e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/3262825524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ec0102c10 0x7f6ec0103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/3262825524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0101e40 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 -- 172.21.15.52:0/3262825524 >> 172.21.15.52:0/3262825524 conn(0x7f6ec00fcf90 msgr2=0x7f6ec00ff3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 -- 172.21.15.52:0/3262825524 shutdown_connections 2024-06-02T23:30:53.396 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.396+0000 7f6ec83d5640 1 -- 172.21.15.52:0/3262825524 wait complete. 2024-06-02T23:30:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 Processor -- start 2024-06-02T23:30:53.398 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 -- start start 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec0102c10 0x7f6ec01980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ec01035d0 0x7f6ec019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010d520 con 0x7f6ec01035d0 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010d3a0 con 0x7f6ec0101a20 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec83d5640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec010d6a0 con 0x7f6ec0102c10 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec6171640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec6171640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33262/0 (socket says 172.21.15.52:33262) 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec6171640 1 -- 172.21.15.52:0/1754467245 learned_addr learned my addr 172.21.15.52:0/1754467245 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec6972640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ec01035d0 0x7f6ec019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.399 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec5970640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec0102c10 0x7f6ec01980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec6171640 1 -- 172.21.15.52:0/1754467245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec0102c10 msgr2=0x7f6ec01980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.397+0000 7f6ec6171640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec0102c10 0x7f6ec01980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec6171640 1 -- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ec01035d0 msgr2=0x7f6ec019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec6171640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ec01035d0 0x7f6ec019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec6171640 1 -- 172.21.15.52:0/1754467245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ec019f7b0 con 0x7f6ec0101a20 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec5970640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec0102c10 0x7f6ec01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec6171640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0197b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005940 tx=0x7f6ebc030e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6eb77fe640 1 -- 172.21.15.52:0/1754467245 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc037050 con 0x7f6ec0101a20 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6eb77fe640 1 -- 172.21.15.52:0/1754467245 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6ebc043070 con 0x7f6ec0101a20 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ec019fa40 con 0x7f6ec0101a20 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6eb77fe640 1 -- 172.21.15.52:0/1754467245 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc003c70 con 0x7f6ec0101a20 2024-06-02T23:30:53.400 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.398+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ec019ff80 con 0x7f6ec0101a20 2024-06-02T23:30:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.401+0000 7f6eb77fe640 1 -- 172.21.15.52:0/1754467245 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6ebc004040 con 0x7f6ec0101a20 2024-06-02T23:30:53.401 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.402+0000 7f6eb77fe640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6eac0a2cb0 0x7f6eac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.402+0000 7f6ec5970640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6eac0a2cb0 0x7f6eac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.402+0000 7f6eb77fe640 1 -- 172.21.15.52:0/1754467245 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6ebc0ef3d0 con 0x7f6ec0101a20 2024-06-02T23:30:53.402 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.402+0000 7f6ec5970640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6eac0a2cb0 0x7f6eac0a5170 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6ec01990a0 tx=0x7f6eb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 Processor -- start 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 -- start start 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7cfe010 0x5611a7cfe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d03500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7d03b70 0x5611a7d06000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5611a7c614f0 con 0x5611a7d03b70 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5611a7bff870 con 0x5611a7cfea00 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611a7c5d4b0 con 0x5611a7cfe010 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ec6972640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7d03b70 0x5611a7d06000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.436 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d03500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ec6972640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7d03b70 0x5611a7d06000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47400/0 (socket says 172.21.15.52:47400) 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d03500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33278/0 (socket says 172.21.15.52:33278) 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.435+0000 7f6ec6972640 1 -- 172.21.15.52:0/2624803811 learned_addr learned my addr 172.21.15.52:0/2624803811 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7cfe010 0x5611a7cfe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec5970640 1 -- 172.21.15.52:0/2624803811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7cfe010 msgr2=0x5611a7cfe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7cfe010 0x5611a7cfe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec5970640 1 -- 172.21.15.52:0/2624803811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7d03b70 msgr2=0x5611a7d06000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7d03b70 0x5611a7d06000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec5970640 1 -- 172.21.15.52:0/2624803811 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a7c1b9b0 con 0x5611a7cfea00 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d03500 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000f610 tx=0x7f6eb0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78ced678438b6c7f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6eb4bfe640 1 -- 172.21.15.52:0/2624803811 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0026430 con 0x5611a7cfea00 2024-06-02T23:30:53.437 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6eb4bfe640 1 -- 172.21.15.52:0/2624803811 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb001c8c0 con 0x5611a7cfea00 2024-06-02T23:30:53.438 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.436+0000 7f6eb4bfe640 1 -- 172.21.15.52:0/2624803811 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb001d870 con 0x5611a7cfea00 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2624803811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 msgr2=0x5611a7d03500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d03500 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6eb000f610 tx=0x7f6eb0014000 comp rx=0 tx=0).stop 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2624803811 shutdown_connections 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7d03b70 0x5611a7d06000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d03500 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2624803811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7cfe010 0x5611a7cfe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2624803811 >> 172.21.15.52:0/2624803811 conn(0x5611a7cf9680 msgr2=0x5611a7cf9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2624803811 shutdown_connections 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.437+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2624803811 wait complete. 2024-06-02T23:30:53.441 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 -- start start 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 Processor -- start 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5611a7d0aea0 con 0x5611a7cfea00 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5611a7beaae0 con 0x5611a7cfe010 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5611a7d0b020 con 0x5611a7d03b70 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.438+0000 7f6ecb1c0740 10 client.0 _ll_register_callbacks cb 0x5611a7c239b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47402/0 (socket says 172.21.15.52:47402) 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec6171640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 -- 172.21.15.52:0/2142646562 learned_addr learned my addr 172.21.15.52:0/2142646562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:53.442 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec6972640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 -- 172.21.15.52:0/2142646562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 msgr2=0x5611a7d13c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 -- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfe010 msgr2=0x5611a7d0c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 -- 172.21.15.52:0/2142646562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a7975210 con 0x5611a7cfea00 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005890 tx=0x7f6eb001c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d992052b1cde30bd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2142646562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb002b040 con 0x5611a7cfea00 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb001c8c0 con 0x5611a7cfea00 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.439+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2142646562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6eb001cc70 con 0x5611a7cfea00 2024-06-02T23:30:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.440+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2142646562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6eb0005040 con 0x5611a7cfea00 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.440+0000 7f6ecb1c0740 10 client.20514 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.440+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5611a7a41620 con 0x5611a7cfea00 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.440+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2142646562 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb001c8c0 con 0x5611a7cfea00 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2142646562 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6eb002e070 con 0x5611a7cfea00 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ecb1c0740 10 client.20514 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ecb1c0740 10 client.20514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ecb1c0740 20 client.20514 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d181f0 0x5611a7d1a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ecb1c0740 4 client.20514 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5611a7c74f10 con 0x5611a7d181f0 2024-06-02T23:30:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.441+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d181f0 0x5611a7d1a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.442+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d181f0 0x5611a7d1a5e0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a33c89f52a36233 server_cookie=a7c6f23a4d786c0d in_seq=0 out_seq=0 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.442+0000 7f6ea9ffb640 10 client.20514 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.443+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2142646562 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5611a7c74f10 con 0x5611a7d181f0 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.443+0000 7f6ea9ffb640 10 client.20514 handle_command_reply: tid=0 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.443+0000 7f6ecb1c0740 1 client.20514 shutdown 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 msgr2=0x5611a7d0cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0005890 tx=0x7f6eb001c550 comp rx=0 tx=0).stop 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 shutdown_connections 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d181f0 0x5611a7d1a5e0 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2142646562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 >> 172.21.15.52:0/2142646562 conn(0x5611a7cf9680 msgr2=0x5611a7cf87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ea9ffb640 0 client.20514 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.444+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 shutdown_connections 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.445+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2142646562 wait complete. 2024-06-02T23:30:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.445+0000 7f6ecb1c0740 20 client.20514 trim_cache size 0 max 16384 2024-06-02T23:30:53.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 41267 6.39 MB/sec execute 241 sec latency 70.615 ms 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 Processor -- start 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 -- start start 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a6a60 con 0x5611a7cfe010 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a85a0 con 0x5611a7cfea00 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a7130 con 0x5611a7d03b70 2024-06-02T23:30:53.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec6171640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec6171640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47410/0 (socket says 172.21.15.52:47410) 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec6171640 1 -- 172.21.15.52:0/2832272369 learned_addr learned my addr 172.21.15.52:0/2832272369 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec6972640 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec5970640 1 -- 172.21.15.52:0/2832272369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 msgr2=0x5611a7d13bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec5970640 1 -- 172.21.15.52:0/2832272369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 msgr2=0x5611a7d0c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.476+0000 7f6ec5970640 1 -- 172.21.15.52:0/2832272369 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5611a7cf94b0 con 0x5611a7cfea00 2024-06-02T23:30:53.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.477+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0010010 tx=0x7f6eb0009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef45f2b875705a67 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.477+0000 7f6eabfff640 1 -- 172.21.15.52:0/2832272369 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb001d690 con 0x5611a7cfea00 2024-06-02T23:30:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.477+0000 7f6eabfff640 1 -- 172.21.15.52:0/2832272369 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb00051e0 con 0x5611a7cfea00 2024-06-02T23:30:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.477+0000 7f6eabfff640 1 -- 172.21.15.52:0/2832272369 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eb0004b10 con 0x5611a7cfea00 2024-06-02T23:30:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2832272369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 msgr2=0x5611a7d0cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0010010 tx=0x7f6eb0009350 comp rx=0 tx=0).stop 2024-06-02T23:30:53.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2832272369 shutdown_connections 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d13bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d0cbc0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2832272369 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d0c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2832272369 >> 172.21.15.52:0/2832272369 conn(0x5611a7cf9680 msgr2=0x5611a7cf9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2832272369 shutdown_connections 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2832272369 wait complete. 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.478+0000 7f6ecb1c0740 1 -- start start 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 Processor -- start 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d10980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d10ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d80e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a7ed0 con 0x5611a7cfe010 2024-06-02T23:30:53.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a7800 con 0x5611a7cfea00 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6ec01a4a40 con 0x5611a7d03b70 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ecb1c0740 10 client.0 _ll_register_callbacks cb 0x7f6eb8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d10ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec6171640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d10980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec5970640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d10ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33310/0 (socket says 172.21.15.52:33310) 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec5970640 1 -- 172.21.15.52:0/2798767280 learned_addr learned my addr 172.21.15.52:0/2798767280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec6972640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d80e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec6171640 1 -- 172.21.15.52:0/2798767280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 msgr2=0x5611a7d80e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.479+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d80e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ec6171640 1 -- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 msgr2=0x5611a7d10ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d10ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ec6171640 1 -- 172.21.15.52:0/2798767280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5611a7cf4f70 con 0x5611a7cfe010 2024-06-02T23:30:53.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d10ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ec6171640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d10980 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0b8630 tx=0x7f6ebc004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2c369ff8bc5ce9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ea9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2798767280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ebc0d8020 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2798767280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6ebc004330 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6eb00051e0 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2798767280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6ebc0d4e60 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ecb1c0740 10 client.20520 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.480+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5611a7d82290 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2798767280 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6eb00051e0 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2798767280 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6ebc0c4440 con 0x5611a7cfe010 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ecb1c0740 10 client.20520 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:53.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ecb1c0740 10 client.20520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ecb1c0740 20 client.20520 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d82c60 0x5611a7d85080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ecb1c0740 4 client.20520 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5611a7c6ec20 con 0x5611a7d82c60 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.481+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d82c60 0x5611a7d85080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.482+0000 7f6ea9ffb640 10 client.20520 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:53.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.482+0000 7f6ec5970640 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d82c60 0x5611a7d85080 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec338dc3287587c9 server_cookie=615564a955333fef in_seq=0 out_seq=0 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.528+0000 7f6ea9ffb640 1 -- 172.21.15.52:0/2798767280 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5611a7c6ec20 con 0x5611a7d82c60 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.528+0000 7f6ea9ffb640 10 client.20520 handle_command_reply: tid=0 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 client.20520 shutdown 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 msgr2=0x5611a7d10980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d10980 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc0b8630 tx=0x7f6ebc004610 comp rx=0 tx=0).stop 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 shutdown_connections 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5611a7d82c60 0x5611a7d85080 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5611a7d03b70 0x5611a7d80e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5611a7cfea00 0x5611a7d10ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 --2- 172.21.15.52:0/2798767280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5611a7cfe010 0x5611a7d10980 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 >> 172.21.15.52:0/2798767280 conn(0x5611a7cf9680 msgr2=0x5611a7cffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ea9ffb640 0 client.20520 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 shutdown_connections 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 1 -- 172.21.15.52:0/2798767280 wait complete. 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.529+0000 7f6ecb1c0740 20 client.20520 trim_cache size 0 max 16384 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:30:53.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:30:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.530+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6eac0a2cb0 msgr2=0x7f6eac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.530+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6eac0a2cb0 0x7f6eac0a5170 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6ec01990a0 tx=0x7f6eb0009260 comp rx=0 tx=0).stop 2024-06-02T23:30:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.530+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 msgr2=0x7f6ec0197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.530+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0197b90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f6ebc005940 tx=0x7f6ebc030e00 comp rx=0 tx=0).stop 2024-06-02T23:30:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 shutdown_connections 2024-06-02T23:30:53.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6eac0a2cb0 0x7f6eac0a5170 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6ec0102c10 0x7f6ec01980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6ec01035d0 0x7f6ec019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 --2- 172.21.15.52:0/1754467245 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ec0101a20 0x7f6ec0197b90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 >> 172.21.15.52:0/1754467245 conn(0x7f6ec00fcf90 msgr2=0x7f6ec0105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 shutdown_connections 2024-06-02T23:30:53.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:30:53.531+0000 7f6ec83d5640 1 -- 172.21.15.52:0/1754467245 wait complete. 2024-06-02T23:30:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:53 smithi123 ceph-mon[35977]: pgmap v1001: 97 pgs: 97 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2024-06-02T23:30:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:53 smithi174 ceph-mon[35994]: pgmap v1001: 97 pgs: 97 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2024-06-02T23:30:54.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:53 smithi052 ceph-mon[32340]: pgmap v1001: 97 pgs: 97 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 306 op/s 2024-06-02T23:30:54.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 41455 6.40 MB/sec execute 242 sec latency 62.814 ms 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 -- 172.21.15.52:0/4239272557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 msgr2=0x7f5c5c10e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 --2- 172.21.15.52:0/4239272557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c10e230 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b0a0 tx=0x7f5c5400b3b0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 -- 172.21.15.52:0/4239272557 shutdown_connections 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 --2- 172.21.15.52:0/4239272557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c10e230 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 --2- 172.21.15.52:0/4239272557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 0x7f5c5c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 --2- 172.21.15.52:0/4239272557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c101380 0x7f5c5c101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 -- 172.21.15.52:0/4239272557 >> 172.21.15.52:0/4239272557 conn(0x7f5c5c078300 msgr2=0x7f5c5c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 -- 172.21.15.52:0/4239272557 shutdown_connections 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.255+0000 7f5c63296700 1 -- 172.21.15.52:0/4239272557 wait complete. 2024-06-02T23:30:55.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.256+0000 7f5c63296700 1 Processor -- start 2024-06-02T23:30:55.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.256+0000 7f5c63296700 1 -- start start 2024-06-02T23:30:55.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c63296700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c101380 0x7f5c5c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c63296700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c19cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c63296700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 0x7f5c5c196780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c63296700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c110420 con 0x7f5c5c101d40 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c63296700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1102a0 con 0x7f5c5c1073f0 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c63296700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1105a0 con 0x7f5c5c101380 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c60831700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c19cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c60831700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c19cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49728/0 (socket says 172.21.15.52:49728) 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c60831700 1 -- 172.21.15.52:0/2608896737 learned_addr learned my addr 172.21.15.52:0/2608896737 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61833700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 0x7f5c5c196780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61032700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c101380 0x7f5c5c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61833700 1 -- 172.21.15.52:0/2608896737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c101380 msgr2=0x7f5c5c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61833700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c101380 0x7f5c5c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61833700 1 -- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 msgr2=0x7f5c5c19cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61833700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c19cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c61833700 1 -- 172.21.15.52:0/2608896737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c5c197060 con 0x7f5c5c1073f0 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.257+0000 7f5c60831700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.258+0000 7f5c61833700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 0x7f5c5c196780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b070 tx=0x7f5c54004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.258+0000 7f5c527fc700 1 -- 172.21.15.52:0/2608896737 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54021070 con 0x7f5c5c1073f0 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.258+0000 7f5c527fc700 1 -- 172.21.15.52:0/2608896737 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c54007e30 con 0x7f5c5c1073f0 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.258+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c5c1972f0 con 0x7f5c5c1073f0 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.258+0000 7f5c527fc700 1 -- 172.21.15.52:0/2608896737 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c54004630 con 0x7f5c5c1073f0 2024-06-02T23:30:55.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.258+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c5c1a35e0 con 0x7f5c5c1073f0 2024-06-02T23:30:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.261+0000 7f5c527fc700 1 -- 172.21.15.52:0/2608896737 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5c54003d00 con 0x7f5c5c1073f0 2024-06-02T23:30:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.262+0000 7f5c527fc700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5c480a2ca0 0x7f5c480a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.262+0000 7f5c61032700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5c480a2ca0 0x7f5c480a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.262+0000 7f5c527fc700 1 -- 172.21.15.52:0/2608896737 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5c5400f020 con 0x7f5c5c1073f0 2024-06-02T23:30:55.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.262+0000 7f5c61032700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5c480a2ca0 0x7f5c480a5160 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009680 tx=0x7f5c58009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:55.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.264+0000 7f5c6db19b80 1 Processor -- start 2024-06-02T23:30:55.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.264+0000 7f5c6db19b80 1 -- start start 2024-06-02T23:30:55.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1f60 0x55fafe7eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fafe73d810 con 0x55fafe7f1f60 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fafe6e5720 con 0x55fafe7ebf20 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fafe7397d0 con 0x55fafe7f1640 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c60831700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1f60 0x55fafe7eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c61032700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c60831700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1f60 0x55fafe7eb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49742/0 (socket says 172.21.15.52:49742) 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c60831700 1 -- 172.21.15.52:0/1127001347 learned_addr learned my addr 172.21.15.52:0/1127001347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.265+0000 7f5c61833700 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61032700 1 -- 172.21.15.52:0/1127001347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 msgr2=0x55fafe7ec340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61032700 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7ec340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61032700 1 -- 172.21.15.52:0/1127001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1f60 msgr2=0x55fafe7eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61032700 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1f60 0x55fafe7eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61032700 1 -- 172.21.15.52:0/1127001347 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafe6dbf50 con 0x55fafe7f1640 2024-06-02T23:30:55.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61833700 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c61032700 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f1a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c58010d50 tx=0x7f5c58013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7de67290d13f1a4d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c477fe700 1 -- 172.21.15.52:0/1127001347 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58011bb0 con 0x55fafe7f1640 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c477fe700 1 -- 172.21.15.52:0/1127001347 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c58011d50 con 0x55fafe7f1640 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.266+0000 7f5c477fe700 1 -- 172.21.15.52:0/1127001347 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5801c5f0 con 0x55fafe7f1640 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.267+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1127001347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 msgr2=0x55fafe7f1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.267+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f1a20 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f5c58010d50 tx=0x7f5c58013000 comp rx=0 tx=0).stop 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.267+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1127001347 shutdown_connections 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.267+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7ec340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.267+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f1a20 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.267+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1127001347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1f60 0x55fafe7eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.268+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1127001347 >> 172.21.15.52:0/1127001347 conn(0x55fafe716cc0 msgr2=0x55fafe7e6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.268+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1127001347 shutdown_connections 2024-06-02T23:30:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.268+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1127001347 wait complete. 2024-06-02T23:30:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.268+0000 7f5c6db19b80 1 -- start start 2024-06-02T23:30:55.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.268+0000 7f5c6db19b80 1 Processor -- start 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 0x55fafe7f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe7f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fafe7fcde0 con 0x55fafe7ebf20 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fafe6ddbf0 con 0x55fafe7f1f60 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c61032700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 0x55fafe7f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fafe7fcf60 con 0x55fafe7f1640 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c61032700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 0x55fafe7f7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49756/0 (socket says 172.21.15.52:49756) 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c61032700 1 -- 172.21.15.52:0/2916536674 learned_addr learned my addr 172.21.15.52:0/2916536674 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c60831700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c61833700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe7f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.269+0000 7f5c6db19b80 10 client.0 _ll_register_callbacks cb 0x55fafe6d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c60831700 1 -- 172.21.15.52:0/2916536674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 msgr2=0x55fafe7f5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c60831700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe7f5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c60831700 1 -- 172.21.15.52:0/2916536674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 msgr2=0x55fafe7f7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c60831700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 0x55fafe7f7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c60831700 1 -- 172.21.15.52:0/2916536674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafe68c8d0 con 0x55fafe7f1640 2024-06-02T23:30:55.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c61833700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe7f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:55.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c61032700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 0x55fafe7f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c60831700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f4fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009cd0 tx=0x7f5c4c00de60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62c1fb09cf8a1aca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c44ff9700 1 -- 172.21.15.52:0/2916536674 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0043e0 con 0x55fafe7f1640 2024-06-02T23:30:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c44ff9700 1 -- 172.21.15.52:0/2916536674 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5c4c004830 con 0x55fafe7f1640 2024-06-02T23:30:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c58011d50 con 0x55fafe7f1640 2024-06-02T23:30:55.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.270+0000 7f5c44ff9700 1 -- 172.21.15.52:0/2916536674 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5c4c004a30 con 0x55fafe7f1640 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 10 client.29473 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fafe807710 con 0x55fafe7f1640 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c44ff9700 1 -- 172.21.15.52:0/2916536674 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c58011d50 con 0x55fafe7f1640 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c44ff9700 1 -- 172.21.15.52:0/2916536674 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5c4c003a90 con 0x55fafe7f1640 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 10 client.29473 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 10 client.29473 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 20 client.29473 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe80c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 4 client.29473 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fafe74ad10 con 0x55fafe809d50 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.271+0000 7f5c61032700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe80c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.272+0000 7f5c61032700 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe80c180 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a1454a71136f7d server_cookie=62fbeb778651e4d2 in_seq=0 out_seq=0 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.272+0000 7f5c44ff9700 10 client.29473 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c44ff9700 1 -- 172.21.15.52:0/2916536674 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fafe74ad10 con 0x55fafe809d50 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c44ff9700 10 client.29473 handle_command_reply: tid=0 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 client.29473 shutdown 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 msgr2=0x55fafe7f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f4fe0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c009cd0 tx=0x7f5c4c00de60 comp rx=0 tx=0).stop 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 shutdown_connections 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe80c180 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe7f5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c44ff9700 0 client.29473 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:30:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7ebf20 0x55fafe7f7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/2916536674 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1640 0x55fafe7f4fe0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 >> 172.21.15.52:0/2916536674 conn(0x55fafe716cc0 msgr2=0x55fafe7ea9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 shutdown_connections 2024-06-02T23:30:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.273+0000 7f5c6db19b80 1 -- 172.21.15.52:0/2916536674 wait complete. 2024-06-02T23:30:55.277 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.274+0000 7f5c6db19b80 20 client.29473 trim_cache size 0 max 16384 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.295+0000 7f5c6db19b80 1 Processor -- start 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.295+0000 7f5c6db19b80 1 -- start start 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7eb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe7f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1f60 0x55fafe7e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1aa7a0 con 0x55fafe7f1640 2024-06-02T23:30:55.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1ac2e0 con 0x55fafe7ebf20 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1aae70 con 0x55fafe7f1f60 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe7f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe7f56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49760/0 (socket says 172.21.15.52:49760) 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 -- 172.21.15.52:0/1589545810 learned_addr learned my addr 172.21.15.52:0/1589545810 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c61032700 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7eb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 -- 172.21.15.52:0/1589545810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1f60 msgr2=0x55fafe7e69f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1f60 0x55fafe7e69f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 -- 172.21.15.52:0/1589545810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 msgr2=0x55fafe7eb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7eb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c60831700 1 -- 172.21.15.52:0/1589545810 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fafe7f3550 con 0x55fafe7f1640 2024-06-02T23:30:55.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.296+0000 7f5c61032700 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:55.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.297+0000 7f5c60831700 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe7f56b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c003f00 tx=0x7f5c4c019760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3c122e44ab6ea2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:55.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.297+0000 7f5c46ffd700 1 -- 172.21.15.52:0/1589545810 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0043e0 con 0x55fafe7f1640 2024-06-02T23:30:55.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.297+0000 7f5c46ffd700 1 -- 172.21.15.52:0/1589545810 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c4c0198f0 con 0x55fafe7f1640 2024-06-02T23:30:55.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.297+0000 7f5c46ffd700 1 -- 172.21.15.52:0/1589545810 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c4c0038e0 con 0x55fafe7f1640 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1589545810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 msgr2=0x55fafe7f56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe7f56b0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f5c4c003f00 tx=0x7f5c4c019760 comp rx=0 tx=0).stop 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1589545810 shutdown_connections 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7f1f60 0x55fafe7e69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7ebf20 0x55fafe7eb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/1589545810 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe7f56b0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1589545810 >> 172.21.15.52:0/1589545810 conn(0x55fafe716cc0 msgr2=0x55fafe7f4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1589545810 shutdown_connections 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.298+0000 7f5c6db19b80 1 -- 172.21.15.52:0/1589545810 wait complete. 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 -- start start 2024-06-02T23:30:55.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 Processor -- start 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 0x55fafe810e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe811340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe811880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1abc10 con 0x55fafe7f1640 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1ab540 con 0x55fafe7f1f60 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5c5c1a8760 con 0x55fafe7ebf20 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c61032700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 0x55fafe810e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.299+0000 7f5c6db19b80 10 client.0 _ll_register_callbacks cb 0x55fafe6d7920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 0x55fafe810e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39272/0 (socket says 172.21.15.52:39272) 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 -- 172.21.15.52:0/3429867535 learned_addr learned my addr 172.21.15.52:0/3429867535 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61833700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe811880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c60831700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe811340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 -- 172.21.15.52:0/3429867535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 msgr2=0x55fafe811880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe811880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 -- 172.21.15.52:0/3429867535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 msgr2=0x55fafe811340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe811340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 -- 172.21.15.52:0/3429867535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fafe801420 con 0x55fafe7ebf20 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c61032700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 0x55fafe810e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009a00 tx=0x7f5c580092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d49fdac4c32b7689 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c44ff9700 1 -- 172.21.15.52:0/3429867535 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c58037070 con 0x55fafe7ebf20 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.300+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c4c0198f0 con 0x55fafe7ebf20 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c44ff9700 1 -- 172.21.15.52:0/3429867535 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5c58005450 con 0x55fafe7ebf20 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c44ff9700 1 -- 172.21.15.52:0/3429867535 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5c58011d60 con 0x55fafe7ebf20 2024-06-02T23:30:55.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c6db19b80 10 client.29476 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:55.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fafe802700 con 0x55fafe7ebf20 2024-06-02T23:30:55.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c44ff9700 1 -- 172.21.15.52:0/3429867535 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5c4c0198f0 con 0x55fafe7ebf20 2024-06-02T23:30:55.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c44ff9700 1 -- 172.21.15.52:0/3429867535 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5c58004940 con 0x55fafe7ebf20 2024-06-02T23:30:55.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.301+0000 7f5c6db19b80 10 client.29476 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:55.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c6db19b80 10 client.29476 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:30:55.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c6db19b80 20 client.29476 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:55.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe802b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:55.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c6db19b80 4 client.29476 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fafe745290 con 0x55fafe809d50 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c60831700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe802b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c60831700 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe802b40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cd55dd834b2fb5a0 server_cookie=8c18cda9306d14b in_seq=0 out_seq=0 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.302+0000 7f5c44ff9700 10 client.29476 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c44ff9700 1 -- 172.21.15.52:0/3429867535 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fafe745290 con 0x55fafe809d50 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c44ff9700 10 client.29476 handle_command_reply: tid=0 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 client.29476 shutdown 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 msgr2=0x55fafe810e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 0x55fafe810e00 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009a00 tx=0x7f5c580092d0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 shutdown_connections 2024-06-02T23:30:55.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55fafe809d50 0x55fafe802b40 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fafe7f1f60 0x55fafe811880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fafe7ebf20 0x55fafe810e00 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 --2- 172.21.15.52:0/3429867535 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fafe7f1640 0x55fafe811340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 >> 172.21.15.52:0/3429867535 conn(0x55fafe716cc0 msgr2=0x55fafe7dfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c44ff9700 0 client.29476 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 shutdown_connections 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 1 -- 172.21.15.52:0/3429867535 wait complete. 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.303+0000 7f5c6db19b80 20 client.29476 trim_cache size 0 max 16384 2024-06-02T23:30:55.308 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:30:55.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5c480a2ca0 msgr2=0x7f5c480a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5c480a2ca0 0x7f5c480a5160 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f5c58009680 tx=0x7f5c58009260 comp rx=0 tx=0).stop 2024-06-02T23:30:55.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 msgr2=0x7f5c5c196780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:55.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 0x7f5c5c196780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5c5400b070 tx=0x7f5c54004d00 comp rx=0 tx=0).stop 2024-06-02T23:30:55.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 shutdown_connections 2024-06-02T23:30:55.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5c480a2ca0 0x7f5c480a5160 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c5c1073f0 0x7f5c5c196780 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5c5c101380 0x7f5c5c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 --2- 172.21.15.52:0/2608896737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5c5c101d40 0x7f5c5c19cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:55.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.304+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 >> 172.21.15.52:0/2608896737 conn(0x7f5c5c078300 msgr2=0x7f5c5c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:55.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.305+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 shutdown_connections 2024-06-02T23:30:55.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:55.305+0000 7f5c63296700 1 -- 172.21.15.52:0/2608896737 wait complete. 2024-06-02T23:30:55.465 INFO:tasks.workunit.client.0.smithi052.stdout: 10 41669 6.44 MB/sec execute 243 sec latency 70.011 ms 2024-06-02T23:30:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:55 smithi174 ceph-mon[35994]: pgmap v1002: 97 pgs: 97 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-06-02T23:30:56.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:55 smithi052 ceph-mon[32340]: pgmap v1002: 97 pgs: 97 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-06-02T23:30:56.227 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:30:56.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:55 smithi123 ceph-mon[35977]: pgmap v1002: 97 pgs: 97 active+clean; 444 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.2 MiB/s wr, 257 op/s 2024-06-02T23:30:56.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 41842 6.46 MB/sec execute 244 sec latency 112.301 ms 2024-06-02T23:30:56.593 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:30:57.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 41949 6.44 MB/sec execute 245 sec latency 66.668 ms 2024-06-02T23:30:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:57 smithi174 ceph-mon[35994]: pgmap v1003: 97 pgs: 97 active+clean; 447 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-06-02T23:30:58.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:57 smithi052 ceph-mon[32340]: pgmap v1003: 97 pgs: 97 active+clean; 447 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-06-02T23:30:58.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:57 smithi123 ceph-mon[35977]: pgmap v1003: 97 pgs: 97 active+clean; 447 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.4 MiB/s wr, 322 op/s 2024-06-02T23:30:58.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 42119 6.46 MB/sec execute 246 sec latency 81.932 ms 2024-06-02T23:30:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 -- 172.21.15.52:0/3948266331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102570 msgr2=0x7f27b01029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 --2- 172.21.15.52:0/3948266331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102570 0x7f27b01029f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009990 tx=0x7f27a0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 -- 172.21.15.52:0/3948266331 shutdown_connections 2024-06-02T23:30:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 --2- 172.21.15.52:0/3948266331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b0102f30 0x7f27b010f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 --2- 172.21.15.52:0/3948266331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0108680 0x7f27b0108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 --2- 172.21.15.52:0/3948266331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102570 0x7f27b01029f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 -- 172.21.15.52:0/3948266331 >> 172.21.15.52:0/3948266331 conn(0x7f27b0079560 msgr2=0x7f27b0079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 -- 172.21.15.52:0/3948266331 shutdown_connections 2024-06-02T23:30:58.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.651+0000 7f27b859f700 1 -- 172.21.15.52:0/3948266331 wait complete. 2024-06-02T23:30:58.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.652+0000 7f27b859f700 1 Processor -- start 2024-06-02T23:30:58.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.652+0000 7f27b859f700 1 -- start start 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.652+0000 7f27b859f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 0x7f27b0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.652+0000 7f27b859f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102f30 0x7f27b0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.652+0000 7f27b859f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b0108680 0x7f27b0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.652+0000 7f27b859f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0112750 con 0x7f27b0102f30 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b859f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01125d0 con 0x7f27b0108680 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b859f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01128d0 con 0x7f27b0102570 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 0x7f27b0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 0x7f27b0074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39288/0 (socket says 172.21.15.52:39288) 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 -- 172.21.15.52:0/3161554212 learned_addr learned my addr 172.21.15.52:0/3161554212 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b0108680 0x7f27b0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102f30 0x7f27b0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 -- 172.21.15.52:0/3161554212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b0108680 msgr2=0x7f27b0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b0108680 0x7f27b0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 -- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102f30 msgr2=0x7f27b0074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102f30 0x7f27b0074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 -- 172.21.15.52:0/3161554212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b01a7070 con 0x7f27b0102570 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.653+0000 7f27b633b700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 0x7f27b0074790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00e9a0 tx=0x7f27ac00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.654+0000 7f27a77fe700 1 -- 172.21.15.52:0/3161554212 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00cbd0 con 0x7f27b0102570 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.654+0000 7f27a77fe700 1 -- 172.21.15.52:0/3161554212 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27ac00cd70 con 0x7f27b0102570 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.654+0000 7f27a77fe700 1 -- 172.21.15.52:0/3161554212 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac018680 con 0x7f27b0102570 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.654+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01a7360 con 0x7f27b0102570 2024-06-02T23:30:58.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.654+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01a7790 con 0x7f27b0102570 2024-06-02T23:30:58.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.657+0000 7f27a77fe700 1 -- 172.21.15.52:0/3161554212 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f27ac010430 con 0x7f27b0102570 2024-06-02T23:30:58.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.658+0000 7f27a77fe700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f279c0a2ca0 0x7f279c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.658+0000 7f27a77fe700 1 -- 172.21.15.52:0/3161554212 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f27ac0cf1e0 con 0x7f27b0102570 2024-06-02T23:30:58.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.658+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f279c0a2ca0 0x7f279c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.659+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f279c0a2ca0 0x7f279c0a5160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f27a0006010 tx=0x7f27a001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:58.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.659+0000 7f27c2e22b80 1 Processor -- start 2024-06-02T23:30:58.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.659+0000 7f27c2e22b80 1 -- start start 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.659+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abae640 0x564b0abaea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.659+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0aba89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 0x564b0aba9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564b0aafa810 con 0x564b0abae640 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564b0aaa2720 con 0x564b0aba8f20 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564b0aaf67d0 con 0x564b0abaef60 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 0x564b0aba9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b5b3a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0aba89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b5b3a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0aba89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39290/0 (socket says 172.21.15.52:39290) 2024-06-02T23:30:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 0x564b0aba9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42790/0 (socket says 172.21.15.52:42790) 2024-06-02T23:30:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b5b3a700 1 -- 172.21.15.52:0/3739714234 learned_addr learned my addr 172.21.15.52:0/3739714234 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b633b700 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abae640 0x564b0abaea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 -- 172.21.15.52:0/3739714234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 msgr2=0x564b0aba89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0aba89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 -- 172.21.15.52:0/3739714234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abae640 msgr2=0x564b0abaea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abae640 0x564b0abaea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.660+0000 7f27b6b3c700 1 -- 172.21.15.52:0/3739714234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0aa98f50 con 0x564b0aba8f20 2024-06-02T23:30:58.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.661+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 0x564b0aba9340 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f27a8011580 tx=0x7f27a8009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e93bbbb07c5353e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.661+0000 7f27a4ff9700 1 -- 172.21.15.52:0/3739714234 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8011e80 con 0x564b0aba8f20 2024-06-02T23:30:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.661+0000 7f27a4ff9700 1 -- 172.21.15.52:0/3739714234 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27a8004bb0 con 0x564b0aba8f20 2024-06-02T23:30:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.661+0000 7f27a4ff9700 1 -- 172.21.15.52:0/3739714234 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8004740 con 0x564b0aba8f20 2024-06-02T23:30:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 -- 172.21.15.52:0/3739714234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 msgr2=0x564b0aba9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 0x564b0aba9340 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f27a8011580 tx=0x7f27a8009920 comp rx=0 tx=0).stop 2024-06-02T23:30:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 -- 172.21.15.52:0/3739714234 shutdown_connections 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0aba8f20 0x564b0aba9340 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abae640 0x564b0abaea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/3739714234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0aba89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 -- 172.21.15.52:0/3739714234 >> 172.21.15.52:0/3739714234 conn(0x564b0aad3cc0 msgr2=0x564b0aba3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 -- 172.21.15.52:0/3739714234 shutdown_connections 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.662+0000 7f27c2e22b80 1 -- 172.21.15.52:0/3739714234 wait complete. 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.663+0000 7f27c2e22b80 1 -- start start 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.663+0000 7f27c2e22b80 1 Processor -- start 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0aba8f20 0x564b0abb4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 0x564b0abb2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abaef60 0x564b0abb25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564b0abb5a20 con 0x564b0aba8f20 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564b0aa9abf0 con 0x564b0abaef60 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564b0abb5ba0 con 0x564b0abae640 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27c2e22b80 10 client.0 _ll_register_callbacks cb 0x564b0aa94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27b5b3a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 0x564b0abb2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27b5b3a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 0x564b0abb2070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:39304/0 (socket says 172.21.15.52:39304) 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27b5b3a700 1 -- 172.21.15.52:0/927940401 learned_addr learned my addr 172.21.15.52:0/927940401 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.664+0000 7f27b633b700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0aba8f20 0x564b0abb4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abaef60 0x564b0abb25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b633b700 1 -- 172.21.15.52:0/927940401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 msgr2=0x564b0abb2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b633b700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 0x564b0abb2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b633b700 1 -- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abaef60 msgr2=0x564b0abb25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b633b700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abaef60 0x564b0abb25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b633b700 1 -- 172.21.15.52:0/927940401 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0aa498d0 con 0x564b0aba8f20 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abaef60 0x564b0abb25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 0x564b0abb2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27b633b700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0aba8f20 0x564b0abb4d70 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0cccb0 tx=0x7f27ac09bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df48a626b80971cc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f2799ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f2799ffb700 1 -- 172.21.15.52:0/927940401 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0b4060 con 0x564b0aba8f20 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a8004bb0 con 0x564b0aba8f20 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.665+0000 7f2799ffb700 1 -- 172.21.15.52:0/927940401 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f27ac09ae60 con 0x564b0aba8f20 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f2799ffb700 1 -- 172.21.15.52:0/927940401 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f27ac0ce030 con 0x564b0aba8f20 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 10 client.20550 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564b0abc0390 con 0x564b0aba8f20 2024-06-02T23:30:58.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f2799ffb700 1 -- 172.21.15.52:0/927940401 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27a8004bb0 con 0x564b0aba8f20 2024-06-02T23:30:58.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f2799ffb700 1 -- 172.21.15.52:0/927940401 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f27ac0ba170 con 0x564b0aba8f20 2024-06-02T23:30:58.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 10 client.20550 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:58.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 10 client.20550 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:58.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 20 client.20550 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:58.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0abc2dc0 0x564b0abc5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 4 client.20550 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:30:58.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.666+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b0ab07d10 con 0x564b0abc2dc0 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.667+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0abc2dc0 0x564b0abc5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.667+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0abc2dc0 0x564b0abc5260 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7db9106f8ae9d995 server_cookie=b129c6611699bdf4 in_seq=0 out_seq=0 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.667+0000 7f2799ffb700 10 client.20550 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f2799ffb700 1 -- 172.21.15.52:0/927940401 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564b0ab07d10 con 0x564b0abc2dc0 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f2799ffb700 10 client.20550 handle_command_reply: tid=0 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 client.20550 shutdown 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0aba8f20 msgr2=0x564b0abb4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0aba8f20 0x564b0abb4d70 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0cccb0 tx=0x7f27ac09bf50 comp rx=0 tx=0).stop 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 shutdown_connections 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0abc2dc0 0x564b0abc5260 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abaef60 0x564b0abb25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0aba8f20 0x564b0abb4d70 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/927940401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abae640 0x564b0abb2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 >> 172.21.15.52:0/927940401 conn(0x564b0aad3cc0 msgr2=0x564b0aba79d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:58.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f2799ffb700 0 client.20550 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:58.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 shutdown_connections 2024-06-02T23:30:58.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 1 -- 172.21.15.52:0/927940401 wait complete. 2024-06-02T23:30:58.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.668+0000 7f27c2e22b80 20 client.20550 trim_cache size 0 max 16384 2024-06-02T23:30:58.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.695+0000 7f27c2e22b80 1 Processor -- start 2024-06-02T23:30:58.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 -- start start 2024-06-02T23:30:58.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0aba89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 0x564b0ab9d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abbecf0 0x564b0abcab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01aed30 con 0x564b0abaef60 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01b0870 con 0x564b0abae640 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01af400 con 0x564b0abbecf0 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27b5b3a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 0x564b0ab9d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27b5b3a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 0x564b0ab9d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49822/0 (socket says 172.21.15.52:49822) 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27b633b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0aba89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27b5b3a700 1 -- 172.21.15.52:0/1280411655 learned_addr learned my addr 172.21.15.52:0/1280411655 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abbecf0 0x564b0abcab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.696+0000 7f27b5b3a700 1 -- 172.21.15.52:0/1280411655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abbecf0 msgr2=0x564b0abcab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abbecf0 0x564b0abcab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f27b5b3a700 1 -- 172.21.15.52:0/1280411655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 msgr2=0x564b0aba89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0aba89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f27b5b3a700 1 -- 172.21.15.52:0/1280411655 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b0abacb50 con 0x564b0abaef60 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abbecf0 0x564b0abcab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 0x564b0ab9d690 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009960 tx=0x7f27a001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f826100d35744c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f279bfff700 1 -- 172.21.15.52:0/1280411655 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0004b70 con 0x564b0abaef60 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f279bfff700 1 -- 172.21.15.52:0/1280411655 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27a0004d10 con 0x564b0abaef60 2024-06-02T23:30:58.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.697+0000 7f279bfff700 1 -- 172.21.15.52:0/1280411655 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0023660 con 0x564b0abaef60 2024-06-02T23:30:58.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 -- 172.21.15.52:0/1280411655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 msgr2=0x564b0ab9d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 0x564b0ab9d690 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f27a0009960 tx=0x7f27a001e000 comp rx=0 tx=0).stop 2024-06-02T23:30:58.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 -- 172.21.15.52:0/1280411655 shutdown_connections 2024-06-02T23:30:58.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abbecf0 0x564b0abcab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0aba89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/1280411655 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abaef60 0x564b0ab9d690 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.698+0000 7f27c2e22b80 1 -- 172.21.15.52:0/1280411655 >> 172.21.15.52:0/1280411655 conn(0x564b0aad3cc0 msgr2=0x564b0abaf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.699+0000 7f27c2e22b80 1 -- 172.21.15.52:0/1280411655 shutdown_connections 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.699+0000 7f27c2e22b80 1 -- 172.21.15.52:0/1280411655 wait complete. 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.699+0000 7f27c2e22b80 1 -- start start 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.699+0000 7f27c2e22b80 1 Processor -- start 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.699+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0ac29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.699+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0abbae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27c2e22b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abbecf0 0x564b0abbb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01b01a0 con 0x564b0abbecf0 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01afad0 con 0x564b0abae640 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27c2e22b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01accd0 con 0x564b0abaef60 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0ac29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27c2e22b80 10 client.0 _ll_register_callbacks cb 0x564b0aa94920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0ac29390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42810/0 (socket says 172.21.15.52:42810) 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 -- 172.21.15.52:0/696329956 learned_addr learned my addr 172.21.15.52:0/696329956 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b6b3c700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abbecf0 0x564b0abbb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0abbae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 -- 172.21.15.52:0/696329956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 msgr2=0x564b0abbae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0abbae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 -- 172.21.15.52:0/696329956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abbecf0 msgr2=0x564b0abbb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abbecf0 0x564b0abbb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 -- 172.21.15.52:0/696329956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b0abca620 con 0x564b0abae640 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f27b633b700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0ac29390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f27ac09c070 tx=0x7f27ac00e8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1f6978280d2468c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f2799ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.700+0000 7f2799ffb700 1 -- 172.21.15.52:0/696329956 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac09b120 con 0x564b0abae640 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f2799ffb700 1 -- 172.21.15.52:0/696329956 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f27ac097760 con 0x564b0abae640 2024-06-02T23:30:58.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a0004d10 con 0x564b0abae640 2024-06-02T23:30:58.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f2799ffb700 1 -- 172.21.15.52:0/696329956 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f27ac09c110 con 0x564b0abae640 2024-06-02T23:30:58.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 10 client.29867 fetch_fsmap learned FSMap version 43 2024-06-02T23:30:58.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564b0ac364d0 con 0x564b0abae640 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f2799ffb700 1 -- 172.21.15.52:0/696329956 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f27a0004d10 con 0x564b0abae640 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f2799ffb700 1 -- 172.21.15.52:0/696329956 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f27ac097960 con 0x564b0abae640 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 10 client.29867 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 10 client.29867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 20 client.29867 populate_metadata read hostname 'smithi052' 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.701+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0ac36c10 0x564b0ac39030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f27c2e22b80 4 client.29867 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564b0ab02290 con 0x564b0ac36c10 2024-06-02T23:30:58.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0ac36c10 0x564b0ac39030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f2799ffb700 10 client.29867 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f27b5b3a700 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0ac36c10 0x564b0ac39030 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d192d34ab6a445b server_cookie=6d9a11d885acaac0 in_seq=0 out_seq=0 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f2799ffb700 1 -- 172.21.15.52:0/696329956 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564b0ab02290 con 0x564b0ac36c10 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f2799ffb700 10 client.29867 handle_command_reply: tid=0 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.702+0000 7f27c2e22b80 1 client.29867 shutdown 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 msgr2=0x564b0ac29390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0ac29390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f27ac09c070 tx=0x7f27ac00e8f0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 shutdown_connections 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564b0ac36c10 0x564b0ac39030 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564b0abbecf0 0x564b0abbb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564b0abae640 0x564b0ac29390 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 --2- 172.21.15.52:0/696329956 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564b0abaef60 0x564b0abbae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 >> 172.21.15.52:0/696329956 conn(0x564b0aad3cc0 msgr2=0x564b0abb0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:58.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f2799ffb700 0 client.29867 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:30:58.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 shutdown_connections 2024-06-02T23:30:58.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 1 -- 172.21.15.52:0/696329956 wait complete. 2024-06-02T23:30:58.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.703+0000 7f27c2e22b80 20 client.29867 trim_cache size 0 max 16384 2024-06-02T23:30:58.708 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:30:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f279c0a2ca0 msgr2=0x7f279c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f279c0a2ca0 0x7f279c0a5160 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f27a0006010 tx=0x7f27a001a040 comp rx=0 tx=0).stop 2024-06-02T23:30:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 msgr2=0x7f27b0074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:30:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 0x7f27b0074790 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00e9a0 tx=0x7f27ac00ecb0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 shutdown_connections 2024-06-02T23:30:58.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f279c0a2ca0 0x7f279c0a5160 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27b0108680 0x7f27b0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27b0102570 0x7f27b0074790 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 --2- 172.21.15.52:0/3161554212 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f27b0102f30 0x7f27b0074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:30:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 >> 172.21.15.52:0/3161554212 conn(0x7f27b0079560 msgr2=0x7f27b0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:30:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 shutdown_connections 2024-06-02T23:30:58.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:30:58.704+0000 7f27b859f700 1 -- 172.21.15.52:0/3161554212 wait complete. 2024-06-02T23:30:59.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 42327 6.50 MB/sec execute 247 sec latency 84.266 ms 2024-06-02T23:31:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:30:59 smithi174 ceph-mon[35994]: pgmap v1004: 97 pgs: 97 active+clean; 450 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-06-02T23:31:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:30:59 smithi052 ceph-mon[32340]: pgmap v1004: 97 pgs: 97 active+clean; 450 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-06-02T23:31:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:30:59 smithi123 ceph-mon[35977]: pgmap v1004: 97 pgs: 97 active+clean; 450 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 912 KiB/s rd, 4.0 MiB/s wr, 265 op/s 2024-06-02T23:31:00.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 42526 6.52 MB/sec execute 248 sec latency 52.635 ms 2024-06-02T23:31:00.476 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:31:00.477 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:31:00.870 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:01.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 42645 6.51 MB/sec execute 249 sec latency 59.862 ms 2024-06-02T23:31:02.025 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:01 smithi052 ceph-mon[32340]: pgmap v1005: 97 pgs: 97 active+clean; 454 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-06-02T23:31:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:01 smithi174 ceph-mon[35994]: pgmap v1005: 97 pgs: 97 active+clean; 454 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-06-02T23:31:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:01 smithi123 ceph-mon[35977]: pgmap v1005: 97 pgs: 97 active+clean; 454 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 872 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-06-02T23:31:02.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 42780 6.51 MB/sec execute 250 sec latency 60.426 ms 2024-06-02T23:31:02.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.797+0000 7fbf7be6a700 1 -- 172.21.15.52:0/2189482791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf74100a80 msgr2=0x7fbf74100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.797+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/2189482791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf74100a80 0x7fbf74100f00 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbf70009990 tx=0x7fbf70009ca0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 -- 172.21.15.52:0/2189482791 shutdown_connections 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/2189482791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf74101440 0x7fbf7410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/2189482791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf741086f0 0x7fbf74108ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/2189482791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf74100a80 0x7fbf74100f00 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 -- 172.21.15.52:0/2189482791 >> 172.21.15.52:0/2189482791 conn(0x7fbf74077a00 msgr2=0x7fbf740fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 -- 172.21.15.52:0/2189482791 shutdown_connections 2024-06-02T23:31:02.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.798+0000 7fbf7be6a700 1 -- 172.21.15.52:0/2189482791 wait complete. 2024-06-02T23:31:02.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 Processor -- start 2024-06-02T23:31:02.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 -- start start 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf74100a80 0x7fbf741a3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf74101440 0x7fbf741a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 0x7fbf7410e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112890 con 0x7fbf74100a80 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112710 con 0x7fbf741086f0 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7be6a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf74112a10 con 0x7fbf74101440 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7a407700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 0x7fbf7410e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7a407700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 0x7fbf7410e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42818/0 (socket says 172.21.15.52:42818) 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf7a407700 1 -- 172.21.15.52:0/907047815 learned_addr learned my addr 172.21.15.52:0/907047815 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf79405700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf74101440 0x7fbf741a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.799+0000 7fbf79c06700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf74100a80 0x7fbf741a3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7a407700 1 -- 172.21.15.52:0/907047815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf74101440 msgr2=0x7fbf741a4480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7a407700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf74101440 0x7fbf741a4480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.801 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7a407700 1 -- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf74100a80 msgr2=0x7fbf741a3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7a407700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf74100a80 0x7fbf741a3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7a407700 1 -- 172.21.15.52:0/907047815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf7410ee70 con 0x7fbf741086f0 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7a407700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 0x7fbf7410e6e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00ef30 tx=0x7fbf6c00c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf6affd700 1 -- 172.21.15.52:0/907047815 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c0092e0 con 0x7fbf741086f0 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf6affd700 1 -- 172.21.15.52:0/907047815 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6c004bb0 con 0x7fbf741086f0 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf7410f100 con 0x7fbf741086f0 2024-06-02T23:31:02.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf6affd700 1 -- 172.21.15.52:0/907047815 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c004020 con 0x7fbf741086f0 2024-06-02T23:31:02.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.800+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf741abe70 con 0x7fbf741086f0 2024-06-02T23:31:02.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.804+0000 7fbf6affd700 1 -- 172.21.15.52:0/907047815 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbf6c01b030 con 0x7fbf741086f0 2024-06-02T23:31:02.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.805+0000 7fbf6affd700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf600a2ca0 0x7fbf600a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.805+0000 7fbf6affd700 1 -- 172.21.15.52:0/907047815 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbf6c0ceaa0 con 0x7fbf741086f0 2024-06-02T23:31:02.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.805+0000 7fbf79c06700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf600a2ca0 0x7fbf600a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.805+0000 7fbf866edb80 1 Processor -- start 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.805+0000 7fbf79c06700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf600a2ca0 0x7fbf600a5160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbf74069990 tx=0x7fbf64009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 -- start start 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e59aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e53a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563250da5890 con 0x563250e596c0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563250d4d7a0 con 0x563250e53fa0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf866edb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563250da1850 con 0x563250e59fe0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e59aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e59aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49858/0 (socket says 172.21.15.52:49858) 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 -- 172.21.15.52:0/2787405176 learned_addr learned my addr 172.21.15.52:0/2787405176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf7a407700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 -- 172.21.15.52:0/2787405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 msgr2=0x563250e53a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79405700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e53a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e53a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 -- 172.21.15.52:0/2787405176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 msgr2=0x563250e543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79c06700 1 -- 172.21.15.52:0/2787405176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563250d43fd0 con 0x563250e596c0 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf7a407700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:02.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.806+0000 7fbf79405700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e53a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:31:02.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.807+0000 7fbf79c06700 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e59aa0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fbf64011d50 tx=0x7fbf64014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e28939ddb45dc71 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.807+0000 7fbf5ffff700 1 -- 172.21.15.52:0/2787405176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6400ed20 con 0x563250e596c0 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.807+0000 7fbf5ffff700 1 -- 172.21.15.52:0/2787405176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6401c8c0 con 0x563250e596c0 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.807+0000 7fbf5ffff700 1 -- 172.21.15.52:0/2787405176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6401d800 con 0x563250e596c0 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 -- 172.21.15.52:0/2787405176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 msgr2=0x563250e59aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e59aa0 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fbf64011d50 tx=0x7fbf64014000 comp rx=0 tx=0).stop 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 -- 172.21.15.52:0/2787405176 shutdown_connections 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e59aa0 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2787405176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e53a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 -- 172.21.15.52:0/2787405176 >> 172.21.15.52:0/2787405176 conn(0x563250d7ed40 msgr2=0x563250e4e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 -- 172.21.15.52:0/2787405176 shutdown_connections 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.808+0000 7fbf866edb80 1 -- 172.21.15.52:0/2787405176 wait complete. 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 -- start start 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 Processor -- start 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e5fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e596c0 0x563250e5cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e59fe0 0x563250e5d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563250e60a50 con 0x563250e53fa0 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563250d45c70 con 0x563250e59fe0 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563250e60bd0 con 0x563250e596c0 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf866edb80 10 client.0 _ll_register_callbacks cb 0x563250d3f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e5fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.809+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e5fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49860/0 (socket says 172.21.15.52:49860) 2024-06-02T23:31:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf79c06700 1 -- 172.21.15.52:0/2168739756 learned_addr learned my addr 172.21.15.52:0/2168739756 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf79405700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e596c0 0x563250e5cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e59fe0 0x563250e5d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 -- 172.21.15.52:0/2168739756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e596c0 msgr2=0x563250e5cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e596c0 0x563250e5cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 -- 172.21.15.52:0/2168739756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 msgr2=0x563250e5fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e5fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 -- 172.21.15.52:0/2168739756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563250cf4950 con 0x563250e59fe0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf79c06700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e5fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf79405700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e596c0 0x563250e5cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf7a407700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e59fe0 0x563250e5d520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0cc620 tx=0x7fbf6c01b600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f867051e857d649 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.810+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/2168739756 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6c09f770 con 0x563250e59fe0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/2168739756 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbf6c0a3b10 con 0x563250e59fe0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf6401c8c0 con 0x563250e59fe0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/2168739756 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbf6c0a3fa0 con 0x563250e59fe0 2024-06-02T23:31:02.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 10 client.29879 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563250e6b2b0 con 0x563250e59fe0 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/2168739756 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf6401c8c0 con 0x563250e59fe0 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/2168739756 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbf6c0bb150 con 0x563250e59fe0 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 10 client.29879 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 10 client.29879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 20 client.29879 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250e6dc50 0x563250e70060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 4 client.29879 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.811+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563250db2d90 con 0x563250e6dc50 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.812+0000 7fbf79c06700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250e6dc50 0x563250e70060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.812+0000 7fbf5d7fa700 10 client.29879 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.812+0000 7fbf79c06700 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250e6dc50 0x563250e70060 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e02b4c5df49ba39a server_cookie=21f61a4d4099a764 in_seq=0 out_seq=0 2024-06-02T23:31:02.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/2168739756 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563250db2d90 con 0x563250e6dc50 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf5d7fa700 10 client.29879 handle_command_reply: tid=0 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf866edb80 1 client.29879 shutdown 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e59fe0 msgr2=0x563250e5d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e59fe0 0x563250e5d520 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0cc620 tx=0x7fbf6c01b600 comp rx=0 tx=0).stop 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 shutdown_connections 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250e6dc50 0x563250e70060 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.813+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e59fe0 0x563250e5d520 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e5fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf866edb80 1 --2- 172.21.15.52:0/2168739756 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e596c0 0x563250e5cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 >> 172.21.15.52:0/2168739756 conn(0x563250d7ed40 msgr2=0x563250e52a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf5d7fa700 0 client.29879 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 shutdown_connections 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf866edb80 1 -- 172.21.15.52:0/2168739756 wait complete. 2024-06-02T23:31:02.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.814+0000 7fbf866edb80 20 client.29879 trim_cache size 0 max 16384 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 Processor -- start 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 -- start start 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e53a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e596c0 0x563250e4ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b3050 con 0x563250e53fa0 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b4b90 con 0x563250e596c0 2024-06-02T23:31:02.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf866edb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b3720 con 0x563250e59fe0 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e53a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e53a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49870/0 (socket says 172.21.15.52:49870) 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf79c06700 1 -- 172.21.15.52:0/3866158819 learned_addr learned my addr 172.21.15.52:0/3866158819 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf79405700 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e596c0 0x563250e4ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.849+0000 7fbf7a407700 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf79c06700 1 -- 172.21.15.52:0/3866158819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 msgr2=0x563250e5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf79c06700 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf79c06700 1 -- 172.21.15.52:0/3866158819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e596c0 msgr2=0x563250e4ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf79c06700 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e596c0 0x563250e4ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf79c06700 1 -- 172.21.15.52:0/3866158819 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563250e59170 con 0x563250e53fa0 2024-06-02T23:31:02.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf7a407700 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:02.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf79c06700 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e53a60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbf64005950 tx=0x7fbf6401c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0477536c5a66d08 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:02.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf5f7fe700 1 -- 172.21.15.52:0/3866158819 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf64035030 con 0x563250e53fa0 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf5f7fe700 1 -- 172.21.15.52:0/3866158819 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf64012e50 con 0x563250e53fa0 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.850+0000 7fbf5f7fe700 1 -- 172.21.15.52:0/3866158819 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6402b040 con 0x563250e53fa0 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 -- 172.21.15.52:0/3866158819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 msgr2=0x563250e53a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e53a60 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fbf64005950 tx=0x7fbf6401c5d0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 -- 172.21.15.52:0/3866158819 shutdown_connections 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e53fa0 0x563250e53a60 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 --2- 172.21.15.52:0/3866158819 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e596c0 0x563250e4ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 -- 172.21.15.52:0/3866158819 >> 172.21.15.52:0/3866158819 conn(0x563250d7ed40 msgr2=0x563250e5a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 -- 172.21.15.52:0/3866158819 shutdown_connections 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.851+0000 7fbf866edb80 1 -- 172.21.15.52:0/3866158819 wait complete. 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.852+0000 7fbf866edb80 1 -- start start 2024-06-02T23:31:02.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.852+0000 7fbf866edb80 1 Processor -- start 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e65ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e66430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e62ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b44c0 con 0x563250e596c0 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b3df0 con 0x563250e53fa0 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbf741b0ff0 con 0x563250e59fe0 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf866edb80 10 client.0 _ll_register_callbacks cb 0x563250d3f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79405700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e66430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e65ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79405700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e66430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:49882/0 (socket says 172.21.15.52:49882) 2024-06-02T23:31:02.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79c06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e65ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42856/0 (socket says 172.21.15.52:42856) 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79405700 1 -- 172.21.15.52:0/1714608709 learned_addr learned my addr 172.21.15.52:0/1714608709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf7a407700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e62ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79c06700 1 -- 172.21.15.52:0/1714608709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 msgr2=0x563250e62ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79c06700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e62ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79c06700 1 -- 172.21.15.52:0/1714608709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 msgr2=0x563250e66430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.853+0000 7fbf79c06700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e66430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf79c06700 1 -- 172.21.15.52:0/1714608709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563250e66b00 con 0x563250e53fa0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf79405700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e66430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf79c06700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e65ef0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf64005c20 tx=0x7fbf6400ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceab77d77be74f46 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf5d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/1714608709 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf6403f060 con 0x563250e53fa0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/1714608709 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbf64005040 con 0x563250e53fa0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbf64012e50 con 0x563250e53fa0 2024-06-02T23:31:02.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/1714608709 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbf64035070 con 0x563250e53fa0 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf866edb80 10 client.29885 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.854+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563250ed8f80 con 0x563250e53fa0 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/1714608709 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbf64012e50 con 0x563250e53fa0 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/1714608709 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbf6401c640 con 0x563250e53fa0 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf866edb80 10 client.29885 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf866edb80 10 client.29885 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf866edb80 20 client.29885 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf866edb80 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250ed9310 0x563250edb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf866edb80 4 client.29885 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563250dad310 con 0x563250ed9310 2024-06-02T23:31:02.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.855+0000 7fbf79405700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250ed9310 0x563250edb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.856+0000 7fbf5d7fa700 10 client.29885 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.856+0000 7fbf79405700 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250ed9310 0x563250edb730 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7aaa2872881bf61 server_cookie=8ab4038a35dafdb5 in_seq=0 out_seq=0 2024-06-02T23:31:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf5d7fa700 1 -- 172.21.15.52:0/1714608709 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563250dad310 con 0x563250ed9310 2024-06-02T23:31:02.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf5d7fa700 10 client.29885 handle_command_reply: tid=0 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 client.29885 shutdown 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 msgr2=0x563250e65ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e65ef0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbf64005c20 tx=0x7fbf6400ebe0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 shutdown_connections 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563250ed9310 0x563250edb730 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563250e59fe0 0x563250e62ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563250e53fa0 0x563250e65ef0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 --2- 172.21.15.52:0/1714608709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563250e596c0 0x563250e66430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 >> 172.21.15.52:0/1714608709 conn(0x563250d7ed40 msgr2=0x563250e47e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf5d7fa700 0 client.29885 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 shutdown_connections 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 1 -- 172.21.15.52:0/1714608709 wait complete. 2024-06-02T23:31:02.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.857+0000 7fbf866edb80 20 client.29885 trim_cache size 0 max 16384 2024-06-02T23:31:02.861 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:31:02.861 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:31:02.861 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "b08ae585-dcf7-471e-af78-4c8f37e90e75", 2024-06-02T23:31:02.861 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:31:02.861 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf600a2ca0 msgr2=0x7fbf600a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf600a2ca0 0x7fbf600a5160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fbf74069990 tx=0x7fbf64009260 comp rx=0 tx=0).stop 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 msgr2=0x7fbf7410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 0x7fbf7410e6e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c00ef30 tx=0x7fbf6c00c3a0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 shutdown_connections 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf600a2ca0 0x7fbf600a5160 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf741086f0 0x7fbf7410e6e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbf74100a80 0x7fbf741a3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 --2- 172.21.15.52:0/907047815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbf74101440 0x7fbf741a4480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 >> 172.21.15.52:0/907047815 conn(0x7fbf74077a00 msgr2=0x7fbf740fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:02.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 shutdown_connections 2024-06-02T23:31:02.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:02.858+0000 7fbf7be6a700 1 -- 172.21.15.52:0/907047815 wait complete. 2024-06-02T23:31:03.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 42945 6.51 MB/sec execute 251 sec latency 100.636 ms 2024-06-02T23:31:03.570 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: pgmap v1006: 97 pgs: 97 active+clean; 459 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:03.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:03.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:03.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:03.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:03 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:03.947 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: pgmap v1006: 97 pgs: 97 active+clean; 459 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:03 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: pgmap v1006: 97 pgs: 97 active+clean; 459 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 307 op/s 2024-06-02T23:31:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:31:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:03 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:04.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 43125 6.51 MB/sec execute 252 sec latency 77.563 ms 2024-06-02T23:31:05.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:04 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:05.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:04 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:05.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:04 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:31:05.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:04 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:05.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:04 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:05.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:04 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:04 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:31:05.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:04 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:04 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:04 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:04 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:31:05.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:04 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:31:05.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 43281 6.51 MB/sec execute 253 sec latency 95.036 ms 2024-06-02T23:31:06.058 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:05 smithi052 ceph-mon[32340]: pgmap v1007: 97 pgs: 97 active+clean; 457 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 -- 172.21.15.52:0/3991911060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 msgr2=0x7fa7c81077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 --2- 172.21.15.52:0/3991911060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c81077d0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009990 tx=0x7fa7c4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 -- 172.21.15.52:0/3991911060 shutdown_connections 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 --2- 172.21.15.52:0/3991911060 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101d40 0x7fa7c810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 --2- 172.21.15.52:0/3991911060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c81077d0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 --2- 172.21.15.52:0/3991911060 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101380 0x7fa7c8101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 -- 172.21.15.52:0/3991911060 >> 172.21.15.52:0/3991911060 conn(0x7fa7c8078300 msgr2=0x7fa7c8078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 -- 172.21.15.52:0/3991911060 shutdown_connections 2024-06-02T23:31:06.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.112+0000 7fa7cf606700 1 -- 172.21.15.52:0/3991911060 wait complete. 2024-06-02T23:31:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 Processor -- start 2024-06-02T23:31:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 -- start start 2024-06-02T23:31:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 0x7fa7c819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101d40 0x7fa7c819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cd3a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 0x7fa7c819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cd3a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 0x7fa7c819e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54174/0 (socket says 172.21.15.52:54174) 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1332243109 learned_addr learned my addr 172.21.15.52:0/1332243109 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8111510 con 0x7fa7c8101380 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8111390 con 0x7fa7c81073f0 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.113+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c8111690 con 0x7fa7c8101d40 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101d40 0x7fa7c819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1332243109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101d40 msgr2=0x7fa7c819ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101d40 0x7fa7c819ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1332243109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 msgr2=0x7fa7c8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101d40 0x7fa7c819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1332243109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c81990a0 con 0x7fa7c8101380 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 0x7fa7c819e780 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40057d0 tx=0x7fa7c4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.114+0000 7fa7be7fc700 1 -- 172.21.15.52:0/1332243109 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4026070 con 0x7fa7c8101380 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.115+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c8199330 con 0x7fa7c8101380 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.115+0000 7fa7be7fc700 1 -- 172.21.15.52:0/1332243109 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c4021410 con 0x7fa7c8101380 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.115+0000 7fa7be7fc700 1 -- 172.21.15.52:0/1332243109 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c4017650 con 0x7fa7c8101380 2024-06-02T23:31:06.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.115+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c81a5650 con 0x7fa7c8101380 2024-06-02T23:31:06.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.118+0000 7fa7be7fc700 1 -- 172.21.15.52:0/1332243109 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa7c4029070 con 0x7fa7c8101380 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.119+0000 7fa7be7fc700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7b40a2ca0 0x7fa7b40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.119+0000 7fa7d9e89b80 1 Processor -- start 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.119+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7b40a2ca0 0x7fa7b40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.119+0000 7fa7be7fc700 1 -- 172.21.15.52:0/1332243109 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa7c40cfab0 con 0x7fa7c8101380 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.119+0000 7fa7d9e89b80 1 -- start start 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.119+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0f98820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fa5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fa5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7b40a2ca0 0x7fa7b40a5160 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8078890 tx=0x7fa7b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7ccba1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fa5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cd3a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0f98820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7ccba1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fa5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56104/0 (socket says 172.21.15.52:56104) 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cd3a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0f98820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36732/0 (socket says 172.21.15.52:36732) 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fa5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fa5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54188/0 (socket says 172.21.15.52:54188) 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7ccba1700 1 -- 172.21.15.52:0/1546274862 learned_addr learned my addr 172.21.15.52:0/1546274862 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c2d0e8af60 con 0x55c2d0fa59a0 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2d0ee9850 con 0x55c2d0f98d60 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d0e51460 con 0x55c2d0f98440 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 -- 172.21.15.52:0/1546274862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 msgr2=0x55c2d0f98820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0f98820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 -- 172.21.15.52:0/1546274862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 msgr2=0x55c2d0fa5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fa5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cdba3700 1 -- 172.21.15.52:0/1546274862 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d0e388d0 con 0x55c2d0fa59a0 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.120+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0f98820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.121+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fa5e00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009420 tx=0x7fa7c0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=641212122c7be9b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.121+0000 7fa7b37fe700 1 -- 172.21.15.52:0/1546274862 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c001a070 con 0x55c2d0fa59a0 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.121+0000 7fa7b37fe700 1 -- 172.21.15.52:0/1546274862 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7c00158a0 con 0x55c2d0fa59a0 2024-06-02T23:31:06.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.121+0000 7fa7b37fe700 1 -- 172.21.15.52:0/1546274862 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0014860 con 0x55c2d0fa59a0 2024-06-02T23:31:06.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 msgr2=0x55c2d0fa5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fa5e00 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009420 tx=0x7fa7c0009920 comp rx=0 tx=0).stop 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 shutdown_connections 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fa5e00 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0f98820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1546274862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fa5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 >> 172.21.15.52:0/1546274862 conn(0x55c2d0ec5080 msgr2=0x55c2d0f93470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 shutdown_connections 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.122+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1546274862 wait complete. 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.123+0000 7fa7d9e89b80 1 -- start start 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.123+0000 7fa7d9e89b80 1 Processor -- start 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.123+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98440 0x55c2d0facf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.125 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.123+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 0x55c2d0faa200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.123+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faa740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c2d0e89fc0 con 0x55c2d0fa59a0 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c2d0ee5810 con 0x55c2d0f98440 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c2d0fada90 con 0x55c2d0f98d60 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 0x55c2d0faa200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 0x55c2d0faa200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36748/0 (socket says 172.21.15.52:36748) 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 -- 172.21.15.52:0/1991590769 learned_addr learned my addr 172.21.15.52:0/1991590769 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98440 0x55c2d0facf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7d9e89b80 10 client.0 _ll_register_callbacks cb 0x55c2d0e7f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 -- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98440 msgr2=0x55c2d0facf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98440 0x55c2d0facf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 -- 172.21.15.52:0/1991590769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 msgr2=0x55c2d0faa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 -- 172.21.15.52:0/1991590769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d0eaf750 con 0x55c2d0f98d60 2024-06-02T23:31:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 0x55c2d0faa200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8003930 tx=0x7fa7b80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e16285ce10f6d0df server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.127 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.124+0000 7fa7b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1991590769 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8004570 con 0x55c2d0f98d60 2024-06-02T23:31:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1991590769 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa7b8012450 con 0x55c2d0f98d60 2024-06-02T23:31:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7c00158a0 con 0x55c2d0f98d60 2024-06-02T23:31:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1991590769 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa7b8012690 con 0x55c2d0f98d60 2024-06-02T23:31:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7d9e89b80 10 client.29521 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:06.129 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c2d0fb8050 con 0x55c2d0f98d60 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.125+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1991590769 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b8025030 con 0x55c2d0f98d60 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1991590769 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa7b802aa40 con 0x55c2d0f98d60 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7d9e89b80 10 client.29521 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7d9e89b80 10 client.29521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7d9e89b80 20 client.29521 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d0fbaa90 0x55c2d0fbcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7d9e89b80 4 client.29521 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fa7c00158a0 con 0x55c2d0fbaa90 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d0fbaa90 0x55c2d0fbcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d0fbaa90 0x55c2d0fbcf10 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88cc1ce9fe26eb server_cookie=2beef600faf64fa7 in_seq=0 out_seq=0 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.126+0000 7fa7b0ff9700 10 client.29521 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1991590769 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7fa7c00158a0 con 0x55c2d0fbaa90 2024-06-02T23:31:06.130 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7b0ff9700 10 client.29521 handle_command_reply: tid=0 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7d9e89b80 1 client.29521 shutdown 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 msgr2=0x55c2d0faa200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 0x55c2d0faa200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8003930 tx=0x7fa7b80047c0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 shutdown_connections 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.127+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d0fbaa90 0x55c2d0fbcf10 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faa740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98440 0x55c2d0facf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1991590769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98d60 0x55c2d0faa200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 >> 172.21.15.52:0/1991590769 conn(0x55c2d0ec5080 msgr2=0x55c2d0fa6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7b0ff9700 0 client.29521 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 shutdown_connections 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1991590769 wait complete. 2024-06-02T23:31:06.131 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.128+0000 7fa7d9e89b80 20 client.29521 trim_cache size 0 max 16384 2024-06-02T23:31:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:05 smithi174 ceph-mon[35994]: pgmap v1007: 97 pgs: 97 active+clean; 457 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-06-02T23:31:06.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 Processor -- start 2024-06-02T23:31:06.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 -- start start 2024-06-02T23:31:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0fac140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fc2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fc2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81aa7f0 con 0x55c2d0fa59a0 2024-06-02T23:31:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81ac330 con 0x55c2d0f98d60 2024-06-02T23:31:06.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81aaec0 con 0x55c2d0f98440 2024-06-02T23:31:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7cdba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fc2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7ccba1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fc2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7cdba3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fc2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54206/0 (socket says 172.21.15.52:54206) 2024-06-02T23:31:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7cdba3700 1 -- 172.21.15.52:0/3010045662 learned_addr learned my addr 172.21.15.52:0/3010045662 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.161+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0fac140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7ccba1700 1 -- 172.21.15.52:0/3010045662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 msgr2=0x55c2d0fac140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0fac140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7ccba1700 1 -- 172.21.15.52:0/3010045662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 msgr2=0x55c2d0fc2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fc2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7ccba1700 1 -- 172.21.15.52:0/3010045662 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2d0f92590 con 0x55c2d0f98d60 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0fac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fc2820 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8011a50 tx=0x7fa7b8011b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2a8351837ceada0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7b2ffd700 1 -- 172.21.15.52:0/3010045662 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8004550 con 0x55c2d0f98d60 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7b2ffd700 1 -- 172.21.15.52:0/3010045662 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b8025050 con 0x55c2d0f98d60 2024-06-02T23:31:06.164 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.162+0000 7fa7b2ffd700 1 -- 172.21.15.52:0/3010045662 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b802a8c0 con 0x55c2d0f98d60 2024-06-02T23:31:06.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/3010045662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 msgr2=0x55c2d0fc2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fc2820 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8011a50 tx=0x7fa7b8011b60 comp rx=0 tx=0).stop 2024-06-02T23:31:06.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/3010045662 shutdown_connections 2024-06-02T23:31:06.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0fa59a0 0x55c2d0fc2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0f98440 0x55c2d0fac140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/3010045662 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0fc2820 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/3010045662 >> 172.21.15.52:0/3010045662 conn(0x55c2d0ec5080 msgr2=0x55c2d0faba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/3010045662 shutdown_connections 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.163+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/3010045662 wait complete. 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 -- start start 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 Processor -- start 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 0x55c2d0faea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0faef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81abc60 con 0x55c2d0f98440 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81ab590 con 0x55c2d0f98d60 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c81a8790 con 0x55c2d0fa59a0 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7cd3a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 0x55c2d0faea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7cd3a2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 0x55c2d0faea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54220/0 (socket says 172.21.15.52:54220) 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1602154755 learned_addr learned my addr 172.21.15.52:0/1602154755 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7d9e89b80 10 client.0 _ll_register_callbacks cb 0x55c2d0e7f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.164+0000 7fa7cdba3700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0faef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1602154755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0fa59a0 msgr2=0x55c2d0faf4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faf4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 msgr2=0x55c2d0faef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0faef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7cd3a2700 1 -- 172.21.15.52:0/1602154755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2d10247f0 con 0x55c2d0f98440 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0faef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7cd3a2700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 0x55c2d0faea30 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40057a0 tx=0x7fa7c400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e20538160152124 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7b0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1602154755 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7c40094b0 con 0x55c2d0f98440 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa7b8025050 con 0x55c2d0f98440 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.165+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1602154755 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa7c40215c0 con 0x55c2d0f98440 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1602154755 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa7c4021930 con 0x55c2d0f98440 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 10 client.20592 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c2d1025ac0 con 0x55c2d0f98440 2024-06-02T23:31:06.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1602154755 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa7b8025050 con 0x55c2d0f98440 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1602154755 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa7c409c6f0 con 0x55c2d0f98440 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 10 client.20592 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 10 client.20592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 20 client.20592 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d10261a0 0x55c2d10285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 4 client.20592 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c2d0ef6d50 con 0x55c2d10261a0 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.166+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d10261a0 0x55c2d10285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7ccba1700 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d10261a0 0x55c2d10285c0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1463e12b01186dff server_cookie=778232aa2b93319 in_seq=0 out_seq=0 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7b0ff9700 10 client.20592 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7b0ff9700 1 -- 172.21.15.52:0/1602154755 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55c2d0ef6d50 con 0x55c2d10261a0 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7b0ff9700 10 client.20592 handle_command_reply: tid=0 2024-06-02T23:31:06.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 client.20592 shutdown 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 msgr2=0x55c2d0faea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 0x55c2d0faea30 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40057a0 tx=0x7fa7c400b6c0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 shutdown_connections 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c2d10261a0 0x55c2d10285c0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c2d0fa59a0 0x55c2d0faf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c2d0f98440 0x55c2d0faea30 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7b0ff9700 0 client.20592 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 --2- 172.21.15.52:0/1602154755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c2d0f98d60 0x55c2d0faef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.173 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.167+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 >> 172.21.15.52:0/1602154755 conn(0x55c2d0ec5080 msgr2=0x55c2d0f97ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.168+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 shutdown_connections 2024-06-02T23:31:06.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.168+0000 7fa7d9e89b80 1 -- 172.21.15.52:0/1602154755 wait complete. 2024-06-02T23:31:06.174 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.168+0000 7fa7d9e89b80 20 client.20592 trim_cache size 0 max 16384 2024-06-02T23:31:06.174 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:31:06.174 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:31:06.174 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout: "b08ae585-dcf7-471e-af78-4c8f37e90e75": { 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "b08ae585-dcf7-471e-af78-4c8f37e90e75", 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:31:06.175 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7b40a2ca0 msgr2=0x7fa7b40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7b40a2ca0 0x7fa7b40a5160 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8078890 tx=0x7fa7b8009260 comp rx=0 tx=0).stop 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 msgr2=0x7fa7c819e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 0x7fa7c819e780 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fa7c40057d0 tx=0x7fa7c4005800 comp rx=0 tx=0).stop 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 shutdown_connections 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa7b40a2ca0 0x7fa7b40a5160 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c81073f0 0x7fa7c8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.176 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa7c8101380 0x7fa7c819e780 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 --2- 172.21.15.52:0/1332243109 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7c8101d40 0x7fa7c819ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 >> 172.21.15.52:0/1332243109 conn(0x7fa7c8078300 msgr2=0x7fa7c81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 shutdown_connections 2024-06-02T23:31:06.177 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:06.169+0000 7fa7cf606700 1 -- 172.21.15.52:0/1332243109 wait complete. 2024-06-02T23:31:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:05 smithi123 ceph-mon[35977]: pgmap v1007: 97 pgs: 97 active+clean; 457 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 258 op/s 2024-06-02T23:31:06.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 43447 6.52 MB/sec execute 254 sec latency 63.500 ms 2024-06-02T23:31:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 -- 172.21.15.52:0/312928805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 msgr2=0x7f8608100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 --2- 172.21.15.52:0/312928805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608100d20 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009900 tx=0x7f85f802a410 comp rx=0 tx=0).stop 2024-06-02T23:31:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 -- 172.21.15.52:0/312928805 shutdown_connections 2024-06-02T23:31:06.691 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 --2- 172.21.15.52:0/312928805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8608101af0 0x7f8608101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 --2- 172.21.15.52:0/312928805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f86081024b0 0x7f8608108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 --2- 172.21.15.52:0/312928805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608100d20 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 -- 172.21.15.52:0/312928805 >> 172.21.15.52:0/312928805 conn(0x7f86080fbe70 msgr2=0x7f86080fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 -- 172.21.15.52:0/312928805 shutdown_connections 2024-06-02T23:31:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.690+0000 7f860d864640 1 -- 172.21.15.52:0/312928805 wait complete. 2024-06-02T23:31:06.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 Processor -- start 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 -- start start 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 0x7f860819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081024b0 0x7f86081a1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f860810b380 con 0x7f8608101af0 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f860810b200 con 0x7f8608100900 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.691+0000 7f860d864640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f860810b500 con 0x7f86081024b0 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f86067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 0x7f860819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608199db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56130/0 (socket says 172.21.15.52:56130) 2024-06-02T23:31:06.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f86067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 0x7f860819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54248/0 (socket says 172.21.15.52:54248) 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 -- 172.21.15.52:0/2179526426 learned_addr learned my addr 172.21.15.52:0/2179526426 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f86077fe640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081024b0 0x7f86081a1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 -- 172.21.15.52:0/2179526426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081024b0 msgr2=0x7f86081a1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081024b0 0x7f86081a1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 -- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 msgr2=0x7f860819a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 0x7f860819a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 -- 172.21.15.52:0/2179526426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86081a19c0 con 0x7f8608100900 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f86067fc640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 0x7f860819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.692+0000 7f8606ffd640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608199db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f85f80314e0 tx=0x7f85f8003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.693+0000 7f860c862640 1 -- 172.21.15.52:0/2179526426 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f803f070 con 0x7f8608100900 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.693+0000 7f860c862640 1 -- 172.21.15.52:0/2179526426 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85f8031c40 con 0x7f8608100900 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.693+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86081a1c50 con 0x7f8608100900 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.693+0000 7f860c862640 1 -- 172.21.15.52:0/2179526426 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8035030 con 0x7f8608100900 2024-06-02T23:31:06.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.693+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86081a2190 con 0x7f8608100900 2024-06-02T23:31:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.695+0000 7f860c862640 1 -- 172.21.15.52:0/2179526426 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f85f8042070 con 0x7f8608100900 2024-06-02T23:31:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 Processor -- start 2024-06-02T23:31:06.697 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 -- start start 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca5e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca63500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca66000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557b5c9c14f0 con 0x557b5ca5ea00 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b5c95f870 con 0x557b5ca63b70 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8610575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b5c9bd4b0 con 0x557b5ca5e010 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca63500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca63500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54254/0 (socket says 172.21.15.52:54254) 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f8606ffd640 1 -- 172.21.15.52:0/2691124498 learned_addr learned my addr 172.21.15.52:0/2691124498 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f86077fe640 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca5e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f86067fc640 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.696+0000 7f860c862640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f85f00a2cb0 0x7f85f00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86067fc640 1 -- 172.21.15.52:0/2691124498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 msgr2=0x557b5ca5e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86067fc640 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca5e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86067fc640 1 -- 172.21.15.52:0/2691124498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 msgr2=0x557b5ca63500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86067fc640 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca63500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86067fc640 1 -- 172.21.15.52:0/2691124498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5c97b9b0 con 0x557b5ca63b70 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86077fe640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f85f00a2cb0 0x7f85f00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f86067fc640 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca66000 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fd0 tx=0x7f85fc0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea187b7cb24829bf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f860c862640 1 -- 172.21.15.52:0/2179526426 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f85f80ef730 con 0x7f8608100900 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f85edffb640 1 -- 172.21.15.52:0/2691124498 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc01e070 con 0x557b5ca63b70 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f85edffb640 1 -- 172.21.15.52:0/2691124498 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85fc0035f0 con 0x557b5ca63b70 2024-06-02T23:31:06.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.697+0000 7f85edffb640 1 -- 172.21.15.52:0/2691124498 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00f800 con 0x557b5ca63b70 2024-06-02T23:31:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 -- 172.21.15.52:0/2691124498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 msgr2=0x557b5ca66000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f86077fe640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f85f00a2cb0 0x7f85f00a5170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fd0 tx=0x7f8600004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca66000 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f85fc009fd0 tx=0x7f85fc0118d0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 -- 172.21.15.52:0/2691124498 shutdown_connections 2024-06-02T23:31:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca66000 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.702 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca63500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 --2- 172.21.15.52:0/2691124498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca5e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 -- 172.21.15.52:0/2691124498 >> 172.21.15.52:0/2691124498 conn(0x557b5ca59680 msgr2=0x557b5ca59a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 -- 172.21.15.52:0/2691124498 shutdown_connections 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 -- 172.21.15.52:0/2691124498 wait complete. 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.698+0000 7f8610575740 1 -- start start 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 Processor -- start 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca682e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca5ea00 0x557b5ca68820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca63b70 0x557b5ca6f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557b5ca66b20 con 0x557b5ca63b70 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557b5c94aae0 con 0x557b5ca5ea00 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557b5ca66ca0 con 0x557b5ca5e010 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8610575740 10 client.0 _ll_register_callbacks cb 0x557b5c9839b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:06.703 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca682e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f86067fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca5ea00 0x557b5ca68820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca682e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:36810/0 (socket says 172.21.15.52:36810) 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f86067fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca5ea00 0x557b5ca68820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56156/0 (socket says 172.21.15.52:56156) 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 -- 172.21.15.52:0/2290459909 learned_addr learned my addr 172.21.15.52:0/2290459909 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f86077fe640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca63b70 0x557b5ca6f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 -- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca5ea00 msgr2=0x557b5ca68820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca5ea00 0x557b5ca68820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 -- 172.21.15.52:0/2290459909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca63b70 msgr2=0x557b5ca6f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca63b70 0x557b5ca6f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f8606ffd640 1 -- 172.21.15.52:0/2290459909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5c6d5210 con 0x557b5ca5e010 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.699+0000 7f86077fe640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca63b70 0x557b5ca6f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f8606ffd640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca682e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f85f80b8440 tx=0x7f85f80ba080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69179981f461999f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f85e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f85e6ffd640 1 -- 172.21.15.52:0/2290459909 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8049070 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85fc0035f0 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f85e6ffd640 1 -- 172.21.15.52:0/2290459909 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f85f80efbe0 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f85e6ffd640 1 -- 172.21.15.52:0/2290459909 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85f80bbe40 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f8610575740 10 client.29539 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557b5c7a1620 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.700+0000 7f85e6ffd640 1 -- 172.21.15.52:0/2290459909 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85fc0035f0 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f85e6ffd640 1 -- 172.21.15.52:0/2290459909 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f85f80be020 con 0x557b5ca5e010 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f8610575740 10 client.29539 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f8610575740 10 client.29539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f8610575740 20 client.29539 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f8610575740 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5ca741b0 0x557b5ca76620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f8610575740 4 client.29539 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557b5c9d4f10 con 0x557b5ca741b0 2024-06-02T23:31:06.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f86067fc640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5ca741b0 0x557b5ca76620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f85e6ffd640 10 client.29539 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.701+0000 7f86067fc640 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5ca741b0 0x557b5ca76620 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b15d98102ebf4 server_cookie=55d885a91cfd7c81 in_seq=0 out_seq=0 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.702+0000 7f85e6ffd640 1 -- 172.21.15.52:0/2290459909 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557b5c9d4f10 con 0x557b5ca741b0 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f85e6ffd640 10 client.29539 handle_command_reply: tid=0 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 client.29539 shutdown 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 msgr2=0x557b5ca682e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca682e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f85f80b8440 tx=0x7f85f80ba080 comp rx=0 tx=0).stop 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 shutdown_connections 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5ca741b0 0x557b5ca76620 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca63b70 0x557b5ca6f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca5ea00 0x557b5ca68820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 --2- 172.21.15.52:0/2290459909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca682e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f85e6ffd640 0 client.29539 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 >> 172.21.15.52:0/2290459909 conn(0x557b5ca59680 msgr2=0x557b5ca587e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 shutdown_connections 2024-06-02T23:31:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 1 -- 172.21.15.52:0/2290459909 wait complete. 2024-06-02T23:31:06.708 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.703+0000 7f8610575740 20 client.29539 trim_cache size 0 max 16384 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 Processor -- start 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 -- start start 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca70240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca59a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f86067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86081aaec0 con 0x557b5ca5ea00 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86081aca00 con 0x557b5ca63b70 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f86067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54284/0 (socket says 172.21.15.52:54284) 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8610575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ab590 con 0x557b5ca5e010 2024-06-02T23:31:06.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f86067fc640 1 -- 172.21.15.52:0/4254421849 learned_addr learned my addr 172.21.15.52:0/4254421849 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.727+0000 7f8606ffd640 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca70240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86077fe640 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca59a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86067fc640 1 -- 172.21.15.52:0/4254421849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 msgr2=0x557b5ca70240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86067fc640 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca70240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86067fc640 1 -- 172.21.15.52:0/4254421849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 msgr2=0x557b5ca59a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86067fc640 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca59a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86067fc640 1 -- 172.21.15.52:0/4254421849 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557b5ca594b0 con 0x557b5ca5ea00 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f8606ffd640 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:06.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f86067fc640 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6f780 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f85fc01c040 tx=0x7f85fc003b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a34fe447191a94 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f85ed7fa640 1 -- 172.21.15.52:0/4254421849 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc01e040 con 0x557b5ca5ea00 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f85ed7fa640 1 -- 172.21.15.52:0/4254421849 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85fc01f040 con 0x557b5ca5ea00 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.728+0000 7f85ed7fa640 1 -- 172.21.15.52:0/4254421849 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85fc00f5a0 con 0x557b5ca5ea00 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 -- 172.21.15.52:0/4254421849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 msgr2=0x557b5ca6f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6f780 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f85fc01c040 tx=0x7f85fc003b70 comp rx=0 tx=0).stop 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 -- 172.21.15.52:0/4254421849 shutdown_connections 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca59a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6f780 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 --2- 172.21.15.52:0/4254421849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca70240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 -- 172.21.15.52:0/4254421849 >> 172.21.15.52:0/4254421849 conn(0x557b5ca59680 msgr2=0x557b5ca63310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 -- 172.21.15.52:0/4254421849 shutdown_connections 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.729+0000 7f8610575740 1 -- 172.21.15.52:0/4254421849 wait complete. 2024-06-02T23:31:06.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 -- start start 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 Processor -- start 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca6ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca6dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ac330 con 0x557b5ca5ea00 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f86081abc60 con 0x557b5ca63b70 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86081a8ea0 con 0x557b5ca5e010 2024-06-02T23:31:06.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.730+0000 7f8610575740 10 client.0 _ll_register_callbacks cb 0x7f85f8005e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f86067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f86067fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:54296/0 (socket says 172.21.15.52:54296) 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca6ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f86067fc640 1 -- 172.21.15.52:0/1455986925 learned_addr learned my addr 172.21.15.52:0/1455986925 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f86077fe640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca6dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 -- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 msgr2=0x557b5ca6dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca6dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 -- 172.21.15.52:0/1455986925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 msgr2=0x557b5ca6cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 -- 172.21.15.52:0/1455986925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557b5ca6e5d0 con 0x557b5ca5e010 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f86077fe640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f86067fc640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f8606ffd640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca6ca90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009320 tx=0x7f85f803b460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f5a68c553c0c1ff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:06.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.731+0000 7f85e6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f85e6ffd640 1 -- 172.21.15.52:0/1455986925 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8049070 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f85e6ffd640 1 -- 172.21.15.52:0/1455986925 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f85f80c49a0 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85fc01f040 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f85e6ffd640 1 -- 172.21.15.52:0/1455986925 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f85f80ba1b0 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 10 client.29545 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557b5ca6f950 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f85e6ffd640 1 -- 172.21.15.52:0/1455986925 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f85fc01f040 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f85e6ffd640 1 -- 172.21.15.52:0/1455986925 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f85f80d9030 con 0x557b5ca5e010 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 10 client.29545 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 10 client.29545 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 20 client.29545 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5cadf330 0x557b5cae1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:06.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 4 client.29545 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:31:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.732+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557b5c9cec20 con 0x557b5cadf330 2024-06-02T23:31:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.733+0000 7f86067fc640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5cadf330 0x557b5cae1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.733+0000 7f85e6ffd640 10 client.29545 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.733+0000 7f86067fc640 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5cadf330 0x557b5cae1720 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8c18ccdb3038c60 server_cookie=6c72c5ac1455df34 in_seq=0 out_seq=0 2024-06-02T23:31:06.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.776+0000 7f85e6ffd640 1 -- 172.21.15.52:0/1455986925 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557b5c9cec20 con 0x557b5cadf330 2024-06-02T23:31:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.776+0000 7f85e6ffd640 10 client.29545 handle_command_reply: tid=0 2024-06-02T23:31:06.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.776+0000 7f8610575740 1 client.29545 shutdown 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 msgr2=0x557b5ca6ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca6ca90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009320 tx=0x7f85f803b460 comp rx=0 tx=0).stop 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 shutdown_connections 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557b5cadf330 0x557b5cae1720 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557b5ca63b70 0x557b5ca6dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f85e6ffd640 0 client.29545 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557b5ca5ea00 0x557b5ca6cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 --2- 172.21.15.52:0/1455986925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557b5ca5e010 0x557b5ca6ca90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 >> 172.21.15.52:0/1455986925 conn(0x557b5ca59680 msgr2=0x557b5ca59a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 shutdown_connections 2024-06-02T23:31:06.778 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 1 -- 172.21.15.52:0/1455986925 wait complete. 2024-06-02T23:31:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.777+0000 7f8610575740 20 client.29545 trim_cache size 0 max 16384 2024-06-02T23:31:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:31:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:31:06.779 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f85f00a2cb0 msgr2=0x7f85f00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f85f00a2cb0 0x7f85f00a5170 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f8600009fd0 tx=0x7f8600004a40 comp rx=0 tx=0).stop 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 msgr2=0x7f8608199db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608199db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f85f80314e0 tx=0x7f85f8003c10 comp rx=0 tx=0).stop 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 shutdown_connections 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f85f00a2cb0 0x7f85f00a5170 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081024b0 0x7f86081a1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8608101af0 0x7f860819a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 --2- 172.21.15.52:0/2179526426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8608100900 0x7f8608199db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 >> 172.21.15.52:0/2179526426 conn(0x7f86080fbe70 msgr2=0x7f8608104df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 shutdown_connections 2024-06-02T23:31:06.780 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:06.778+0000 7f860d864640 1 -- 172.21.15.52:0/2179526426 wait complete. 2024-06-02T23:31:06.884 INFO:tasks.cephfs.filesystem:scrub status for tag:b08ae585-dcf7-471e-af78-4c8f37e90e75 - {'path': '/', 'tag': 'b08ae585-dcf7-471e-af78-4c8f37e90e75', 'options': 'recursive,force'} 2024-06-02T23:31:07.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 43600 6.53 MB/sec execute 255 sec latency 72.354 ms 2024-06-02T23:31:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:07 smithi174 ceph-mon[35994]: pgmap v1008: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 352 op/s 2024-06-02T23:31:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:07 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:31:08.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:07 smithi052 ceph-mon[32340]: pgmap v1008: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 352 op/s 2024-06-02T23:31:08.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:07 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:31:08.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:07 smithi123 ceph-mon[35977]: pgmap v1008: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 352 op/s 2024-06-02T23:31:08.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:07 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:31:08.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 43757 6.54 MB/sec execute 256 sec latency 67.122 ms 2024-06-02T23:31:09.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 43921 6.54 MB/sec execute 257 sec latency 78.343 ms 2024-06-02T23:31:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: pgmap v1009: 97 pgs: 97 active+clean; 456 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-06-02T23:31:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.508 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:09 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: pgmap v1009: 97 pgs: 97 active+clean; 456 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-06-02T23:31:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:09 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: pgmap v1009: 97 pgs: 97 active+clean; 456 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 322 op/s 2024-06-02T23:31:09.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:09.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:09 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:10.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 44139 6.57 MB/sec execute 258 sec latency 65.269 ms 2024-06-02T23:31:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:31:11.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:31:11.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:31:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:31:11.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:31:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:31:11.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:31:11.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 44342 6.59 MB/sec execute 259 sec latency 52.358 ms 2024-06-02T23:31:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:11 smithi123 ceph-mon[35977]: pgmap v1010: 97 pgs: 97 active+clean; 454 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 346 op/s 2024-06-02T23:31:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:31:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:31:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:31:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:31:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:31:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:11 smithi174 ceph-mon[35994]: pgmap v1010: 97 pgs: 97 active+clean; 454 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 346 op/s 2024-06-02T23:31:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:31:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:31:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:31:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:31:12.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:31:12.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:11 smithi052 ceph-mon[32340]: pgmap v1010: 97 pgs: 97 active+clean; 454 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 346 op/s 2024-06-02T23:31:12.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:31:12.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:31:12.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:31:12.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:31:12.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:31:12.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 44476 6.57 MB/sec execute 260 sec latency 59.686 ms 2024-06-02T23:31:13.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 44632 6.59 MB/sec execute 261 sec latency 103.855 ms 2024-06-02T23:31:14.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:13 smithi123 ceph-mon[35977]: pgmap v1011: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-06-02T23:31:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:13 smithi174 ceph-mon[35994]: pgmap v1011: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-06-02T23:31:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:13 smithi052 ceph-mon[32340]: pgmap v1011: 97 pgs: 97 active+clean; 455 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.8 MiB/s wr, 359 op/s 2024-06-02T23:31:14.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 44835 6.61 MB/sec execute 262 sec latency 53.858 ms 2024-06-02T23:31:15.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 45043 6.65 MB/sec execute 263 sec latency 82.888 ms 2024-06-02T23:31:15.945 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:15 smithi174 ceph-mon[35994]: pgmap v1012: 97 pgs: 97 active+clean; 456 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 313 op/s 2024-06-02T23:31:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:15 smithi123 ceph-mon[35977]: pgmap v1012: 97 pgs: 97 active+clean; 456 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 313 op/s 2024-06-02T23:31:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:15 smithi052 ceph-mon[32340]: pgmap v1012: 97 pgs: 97 active+clean; 456 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 313 op/s 2024-06-02T23:31:16.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 45236 6.66 MB/sec execute 264 sec latency 54.174 ms 2024-06-02T23:31:17.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 45363 6.65 MB/sec execute 265 sec latency 57.897 ms 2024-06-02T23:31:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:17 smithi123 ceph-mon[35977]: pgmap v1013: 97 pgs: 97 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 370 op/s 2024-06-02T23:31:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:17 smithi174 ceph-mon[35994]: pgmap v1013: 97 pgs: 97 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 370 op/s 2024-06-02T23:31:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:17 smithi052 ceph-mon[32340]: pgmap v1013: 97 pgs: 97 active+clean; 465 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 370 op/s 2024-06-02T23:31:18.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 45494 6.66 MB/sec execute 266 sec latency 162.855 ms 2024-06-02T23:31:19.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 45688 6.70 MB/sec execute 267 sec latency 613.267 ms 2024-06-02T23:31:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.910+0000 7f2e604d4640 1 -- 172.21.15.52:0/3458812804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 msgr2=0x7f2e58100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.910+0000 7f2e604d4640 1 --2- 172.21.15.52:0/3458812804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58100d20 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2e54009900 tx=0x7f2e5402a410 comp rx=0 tx=0).stop 2024-06-02T23:31:19.911 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 -- 172.21.15.52:0/3458812804 shutdown_connections 2024-06-02T23:31:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 --2- 172.21.15.52:0/3458812804 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e58101af0 0x7f2e58101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 --2- 172.21.15.52:0/3458812804 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e581024b0 0x7f2e58108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 --2- 172.21.15.52:0/3458812804 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58100d20 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 -- 172.21.15.52:0/3458812804 >> 172.21.15.52:0/3458812804 conn(0x7f2e580fbe70 msgr2=0x7f2e580fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 -- 172.21.15.52:0/3458812804 shutdown_connections 2024-06-02T23:31:19.912 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 -- 172.21.15.52:0/3458812804 wait complete. 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.911+0000 7f2e604d4640 1 Processor -- start 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 -- start start 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e58101af0 0x7f2e5819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e581024b0 0x7f2e581a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58069bb0 con 0x7f2e58100900 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58069a30 con 0x7f2e58101af0 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e604d4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e58069d30 con 0x7f2e581024b0 2024-06-02T23:31:19.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48006/0 (socket says 172.21.15.52:48006) 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 -- 172.21.15.52:0/1862013854 learned_addr learned my addr 172.21.15.52:0/1862013854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e58101af0 0x7f2e5819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e581024b0 0x7f2e581a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 -- 172.21.15.52:0/1862013854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e581024b0 msgr2=0x7f2e581a1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e581024b0 0x7f2e581a1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 -- 172.21.15.52:0/1862013854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e58101af0 msgr2=0x7f2e5819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e58101af0 0x7f2e5819a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5e270640 1 -- 172.21.15.52:0/1862013854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e581a1aa0 con 0x7f2e58100900 2024-06-02T23:31:19.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.912+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e58101af0 0x7f2e5819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.913+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58199e80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2e54004270 tx=0x7f2e540042a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.913+0000 7f2e4f7fe640 1 -- 172.21.15.52:0/1862013854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5403f030 con 0x7f2e58100900 2024-06-02T23:31:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.913+0000 7f2e4f7fe640 1 -- 172.21.15.52:0/1862013854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e540044c0 con 0x7f2e58100900 2024-06-02T23:31:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.913+0000 7f2e4f7fe640 1 -- 172.21.15.52:0/1862013854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54035070 con 0x7f2e58100900 2024-06-02T23:31:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.913+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e581a1d30 con 0x7f2e58100900 2024-06-02T23:31:19.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.913+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e581a21d0 con 0x7f2e58100900 2024-06-02T23:31:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 Processor -- start 2024-06-02T23:31:19.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 -- start start 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747f8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x5572747fb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5572747564f0 con 0x5572747f3010 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572746f4870 con 0x5572747f3a00 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e631e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5572747524b0 con 0x5572747f8b70 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e4f7fe640 1 -- 172.21.15.52:0/1862013854 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2e54042030 con 0x7f2e58100900 2024-06-02T23:31:19.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e5da6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x5572747fb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e5e270640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747f8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e5ea71640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e5da6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x5572747fb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:59728/0 (socket says 172.21.15.52:59728) 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.917+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1799495668 learned_addr learned my addr 172.21.15.52:0/1799495668 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1799495668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 msgr2=0x5572747f8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747f8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1799495668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 msgr2=0x5572747f3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1799495668 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572747109b0 con 0x5572747f8b70 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x5572747fb000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e480114f0 tx=0x7f2e480119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0f033d56ff8b36f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e4cbfe640 1 -- 172.21.15.52:0/1799495668 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48004020 con 0x5572747f8b70 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e4cbfe640 1 -- 172.21.15.52:0/1799495668 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e480041c0 con 0x5572747f8b70 2024-06-02T23:31:19.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e4cbfe640 1 -- 172.21.15.52:0/1799495668 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e4800f8a0 con 0x5572747f8b70 2024-06-02T23:31:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.918+0000 7f2e4f7fe640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e440a2d30 0x7f2e440a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e4f7fe640 1 -- 172.21.15.52:0/1862013854 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2e540ef4e0 con 0x7f2e58100900 2024-06-02T23:31:19.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e440a2d30 0x7f2e440a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 -- 172.21.15.52:0/1799495668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 msgr2=0x5572747fb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x5572747fb000 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2e480114f0 tx=0x7f2e480119b0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 -- 172.21.15.52:0/1799495668 shutdown_connections 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x5572747fb000 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747f8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1799495668 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e631e4740 1 -- 172.21.15.52:0/1799495668 >> 172.21.15.52:0/1799495668 conn(0x5572747ee680 msgr2=0x5572747eea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.919+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e440a2d30 0x7f2e440a51f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5572747f4260 tx=0x7f2e5000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 -- 172.21.15.52:0/1799495668 shutdown_connections 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 -- 172.21.15.52:0/1799495668 wait complete. 2024-06-02T23:31:19.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 -- start start 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 Processor -- start 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747fd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747fd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x557274804890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5572747fbb20 con 0x5572747f3010 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5572746dfae0 con 0x5572747f3a00 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.920+0000 7f2e631e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5572747fbca0 con 0x5572747f8b70 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e631e4740 10 client.0 _ll_register_callbacks cb 0x5572747189b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5da6f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747fd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5da6f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747fd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45562/0 (socket says 172.21.15.52:45562) 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1310489399 learned_addr learned my addr 172.21.15.52:0/1310489399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x557274804890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1310489399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 msgr2=0x557274804890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x557274804890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.921+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1310489399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 msgr2=0x5572747fd310 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:31:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.922+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747fd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.922+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747fd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.922+0000 7f2e5da6f640 1 -- 172.21.15.52:0/1310489399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55727446a210 con 0x5572747f3a00 2024-06-02T23:31:19.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747fd850 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801b040 tx=0x7f2e48009dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96302ac28fd04952 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1310489399 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e48027020 con 0x5572747f3a00 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e480041c0 con 0x5572747f3a00 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1310489399 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2e48016040 con 0x5572747f3a00 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1310489399 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2e48005050 con 0x5572747f3a00 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e631e4740 10 client.29927 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:19.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.923+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557274536620 con 0x5572747f3a00 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1310489399 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e480041c0 con 0x5572747f3a00 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1310489399 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2e4800f5c0 con 0x5572747f3a00 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e631e4740 10 client.29927 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e631e4740 10 client.29927 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e631e4740 20 client.29927 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274808d40 0x55727480b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e631e4740 4 client.29927 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557274769f10 con 0x557274808d40 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.924+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274808d40 0x55727480b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.925+0000 7f2e41ffb640 10 client.29927 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.925+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274808d40 0x55727480b1a0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8cbf6c12a3dd7b7 server_cookie=5bca01b3e744fec8 in_seq=0 out_seq=0 2024-06-02T23:31:19.928 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1310489399 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x557274769f10 con 0x557274808d40 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e41ffb640 10 client.29927 handle_command_reply: tid=0 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 client.29927 shutdown 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 msgr2=0x5572747fd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747fd850 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f2e4801b040 tx=0x7f2e48009dc0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 shutdown_connections 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274808d40 0x55727480b1a0 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f8b70 0x557274804890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e41ffb640 0 client.29927 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f3a00 0x5572747fd850 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1310489399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747fd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 >> 172.21.15.52:0/1310489399 conn(0x5572747ee680 msgr2=0x5572747ed7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 shutdown_connections 2024-06-02T23:31:19.929 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 1 -- 172.21.15.52:0/1310489399 wait complete. 2024-06-02T23:31:19.930 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.926+0000 7f2e631e4740 20 client.29927 trim_cache size 0 max 16384 2024-06-02T23:31:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 Processor -- start 2024-06-02T23:31:19.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 -- start start 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274804850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x5572747f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581aaf70 con 0x5572747f3010 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581acab0 con 0x5572747f8b70 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e631e4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581ab640 con 0x5572747f3a00 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5ea71640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x5572747f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5e270640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5da6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274804850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5ea71640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x5572747f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45566/0 (socket says 172.21.15.52:45566) 2024-06-02T23:31:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5ea71640 1 -- 172.21.15.52:0/2642256841 learned_addr learned my addr 172.21.15.52:0/2642256841 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5ea71640 1 -- 172.21.15.52:0/2642256841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 msgr2=0x557274804850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274804850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.950+0000 7f2e5ea71640 1 -- 172.21.15.52:0/2642256841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 msgr2=0x5572747f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.951+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.951+0000 7f2e5ea71640 1 -- 172.21.15.52:0/2642256841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572747ef350 con 0x5572747f8b70 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.951+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x5572747f51c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2e500189e0 tx=0x7f2e50018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b6560c9cbd6bcbd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.951+0000 7f2e43fff640 1 -- 172.21.15.52:0/2642256841 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e5001e040 con 0x5572747f8b70 2024-06-02T23:31:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.951+0000 7f2e43fff640 1 -- 172.21.15.52:0/2642256841 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e50028070 con 0x5572747f8b70 2024-06-02T23:31:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.951+0000 7f2e43fff640 1 -- 172.21.15.52:0/2642256841 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e500173c0 con 0x5572747f8b70 2024-06-02T23:31:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 -- 172.21.15.52:0/2642256841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 msgr2=0x5572747f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x5572747f51c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f2e500189e0 tx=0x7f2e50018000 comp rx=0 tx=0).stop 2024-06-02T23:31:19.954 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 -- 172.21.15.52:0/2642256841 shutdown_connections 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274804850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x5572747f51c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 --2- 172.21.15.52:0/2642256841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x5572747f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.952+0000 7f2e631e4740 1 -- 172.21.15.52:0/2642256841 >> 172.21.15.52:0/2642256841 conn(0x5572747ee680 msgr2=0x5572747eea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.953+0000 7f2e631e4740 1 -- 172.21.15.52:0/2642256841 shutdown_connections 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.953+0000 7f2e631e4740 1 -- 172.21.15.52:0/2642256841 wait complete. 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.953+0000 7f2e631e4740 1 -- start start 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.953+0000 7f2e631e4740 1 Processor -- start 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x557274801a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274801f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x557274802f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e5e270640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x557274801a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e5e270640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x557274801a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:48044/0 (socket says 172.21.15.52:48044) 2024-06-02T23:31:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e5da6f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274801f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e5e270640 1 -- 172.21.15.52:0/1108900078 learned_addr learned my addr 172.21.15.52:0/1108900078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581ac3e0 con 0x5572747f3010 2024-06-02T23:31:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581abd10 con 0x5572747f8b70 2024-06-02T23:31:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2e581a8f50 con 0x5572747f3a00 2024-06-02T23:31:19.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e631e4740 10 client.0 _ll_register_callbacks cb 0x7f2e500099d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.954+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x557274802f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5e270640 1 -- 172.21.15.52:0/1108900078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 msgr2=0x557274801f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274801f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5e270640 1 -- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 msgr2=0x557274802f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x557274802f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5e270640 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572747ee4b0 con 0x5572747f3010 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5ea71640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x557274802f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:19.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e5e270640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x557274801a00 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2e540314e0 tx=0x7f2e54031a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0768ee28d66ed5a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e41ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1108900078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e54049040 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1108900078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2e540ba0c0 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e50028070 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1108900078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2e540bbdc0 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e631e4740 10 client.20625 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.955+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557274804730 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1108900078 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2e50028070 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1108900078 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2e540ef110 con 0x5572747f3010 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e631e4740 10 client.20625 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e631e4740 10 client.20625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:19.962 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e631e4740 20 client.20625 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274873db0 0x5572748761d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e631e4740 4 client.20625 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:31:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557274763c20 con 0x557274873db0 2024-06-02T23:31:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.956+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274873db0 0x5572748761d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.957+0000 7f2e41ffb640 10 client.20625 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:19.963 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:19.957+0000 7f2e5da6f640 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274873db0 0x5572748761d0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa6d3f702b0d6261 server_cookie=5d6f3502ea48cc4 in_seq=0 out_seq=0 2024-06-02T23:31:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:19 smithi123 ceph-mon[35977]: pgmap v1014: 97 pgs: 97 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-06-02T23:31:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e41ffb640 1 -- 172.21.15.52:0/1108900078 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557274763c20 con 0x557274873db0 2024-06-02T23:31:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e41ffb640 10 client.20625 handle_command_reply: tid=0 2024-06-02T23:31:20.010 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 client.20625 shutdown 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 msgr2=0x557274801a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x557274801a00 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7f2e540314e0 tx=0x7f2e54031a00 comp rx=0 tx=0).stop 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 shutdown_connections 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557274873db0 0x5572748761d0 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5572747f3a00 0x557274801f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e41ffb640 0 client.20625 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5572747f8b70 0x557274802f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 --2- 172.21.15.52:0/1108900078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5572747f3010 0x557274801a00 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.009+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 >> 172.21.15.52:0/1108900078 conn(0x5572747ee680 msgr2=0x5572747fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 shutdown_connections 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e631e4740 1 -- 172.21.15.52:0/1108900078 wait complete. 2024-06-02T23:31:20.011 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e631e4740 20 client.20625 trim_cache size 0 max 16384 2024-06-02T23:31:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:31:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:31:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:31:20.012 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e440a2d30 msgr2=0x7f2e440a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e440a2d30 0x7f2e440a51f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x5572747f4260 tx=0x7f2e5000d040 comp rx=0 tx=0).stop 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 msgr2=0x7f2e58199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58199e80 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f2e54004270 tx=0x7f2e540042a0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 shutdown_connections 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2e440a2d30 0x7f2e440a51f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2e581024b0 0x7f2e581a1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e58101af0 0x7f2e5819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 --2- 172.21.15.52:0/1862013854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2e58100900 0x7f2e58199e80 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.010+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 >> 172.21.15.52:0/1862013854 conn(0x7f2e580fbe70 msgr2=0x7f2e580fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.011+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 shutdown_connections 2024-06-02T23:31:20.013 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:20.011+0000 7f2e604d4640 1 -- 172.21.15.52:0/1862013854 wait complete. 2024-06-02T23:31:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:19 smithi174 ceph-mon[35994]: pgmap v1014: 97 pgs: 97 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-06-02T23:31:20.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:19 smithi052 ceph-mon[32340]: pgmap v1014: 97 pgs: 97 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 281 op/s 2024-06-02T23:31:20.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 45850 6.69 MB/sec execute 268 sec latency 97.098 ms 2024-06-02T23:31:21.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46006 6.70 MB/sec execute 269 sec latency 61.261 ms 2024-06-02T23:31:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:21 smithi123 ceph-mon[35977]: pgmap v1015: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 286 op/s 2024-06-02T23:31:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:21 smithi174 ceph-mon[35994]: pgmap v1015: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 286 op/s 2024-06-02T23:31:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:21 smithi052 ceph-mon[32340]: pgmap v1015: 97 pgs: 97 active+clean; 471 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.8 MiB/s wr, 286 op/s 2024-06-02T23:31:22.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46169 6.71 MB/sec execute 270 sec latency 81.111 ms 2024-06-02T23:31:23.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46330 6.72 MB/sec execute 271 sec latency 59.058 ms 2024-06-02T23:31:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:23 smithi123 ceph-mon[35977]: pgmap v1016: 97 pgs: 97 active+clean; 470 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-06-02T23:31:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:23 smithi174 ceph-mon[35994]: pgmap v1016: 97 pgs: 97 active+clean; 470 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-06-02T23:31:24.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:23 smithi052 ceph-mon[32340]: pgmap v1016: 97 pgs: 97 active+clean; 470 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 4.5 MiB/s wr, 269 op/s 2024-06-02T23:31:24.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46489 6.70 MB/sec execute 272 sec latency 83.878 ms 2024-06-02T23:31:25.466 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46647 6.70 MB/sec execute 273 sec latency 96.165 ms 2024-06-02T23:31:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:25 smithi123 ceph-mon[35977]: pgmap v1017: 97 pgs: 97 active+clean; 473 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 263 op/s 2024-06-02T23:31:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:25 smithi174 ceph-mon[35994]: pgmap v1017: 97 pgs: 97 active+clean; 473 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 263 op/s 2024-06-02T23:31:26.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:25 smithi052 ceph-mon[32340]: pgmap v1017: 97 pgs: 97 active+clean; 473 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 263 op/s 2024-06-02T23:31:26.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46813 6.70 MB/sec execute 274 sec latency 104.220 ms 2024-06-02T23:31:27.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 46986 6.70 MB/sec execute 275 sec latency 58.413 ms 2024-06-02T23:31:28.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:27 smithi123 ceph-mon[35977]: pgmap v1018: 97 pgs: 97 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 317 op/s 2024-06-02T23:31:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:27 smithi174 ceph-mon[35994]: pgmap v1018: 97 pgs: 97 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 317 op/s 2024-06-02T23:31:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:27 smithi052 ceph-mon[32340]: pgmap v1018: 97 pgs: 97 active+clean; 467 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 317 op/s 2024-06-02T23:31:28.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 47180 6.74 MB/sec execute 276 sec latency 88.087 ms 2024-06-02T23:31:29.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 47349 6.73 MB/sec execute 277 sec latency 71.927 ms 2024-06-02T23:31:30.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:29 smithi123 ceph-mon[35977]: pgmap v1019: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 268 op/s 2024-06-02T23:31:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:29 smithi174 ceph-mon[35994]: pgmap v1019: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 268 op/s 2024-06-02T23:31:30.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:29 smithi052 ceph-mon[32340]: pgmap v1019: 97 pgs: 97 active+clean; 466 MiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 268 op/s 2024-06-02T23:31:30.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 47563 6.75 MB/sec execute 278 sec latency 103.679 ms 2024-06-02T23:31:31.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 47767 6.76 MB/sec execute 279 sec latency 55.812 ms 2024-06-02T23:31:32.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:31 smithi123 ceph-mon[35977]: pgmap v1020: 97 pgs: 97 active+clean; 469 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-06-02T23:31:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:31 smithi174 ceph-mon[35994]: pgmap v1020: 97 pgs: 97 active+clean; 469 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-06-02T23:31:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:31 smithi052 ceph-mon[32340]: pgmap v1020: 97 pgs: 97 active+clean; 469 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-06-02T23:31:32.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 47930 6.76 MB/sec execute 280 sec latency 54.822 ms 2024-06-02T23:31:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 -- 172.21.15.52:0/2035576401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8104e10 msgr2=0x7fc9b8105290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2035576401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8104e10 0x7fc9b8105290 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009900 tx=0x7fc9a002a3f0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 -- 172.21.15.52:0/2035576401 shutdown_connections 2024-06-02T23:31:33.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2035576401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b81057d0 0x7fc9b810c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2035576401 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8104e10 0x7fc9b8105290 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2035576401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b8103c20 0x7fc9b8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 -- 172.21.15.52:0/2035576401 >> 172.21.15.52:0/2035576401 conn(0x7fc9b80ff190 msgr2=0x7fc9b81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 -- 172.21.15.52:0/2035576401 shutdown_connections 2024-06-02T23:31:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.144+0000 7fc9bd368640 1 -- 172.21.15.52:0/2035576401 wait complete. 2024-06-02T23:31:33.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 Processor -- start 2024-06-02T23:31:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 -- start start 2024-06-02T23:31:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8103c20 0x7fc9b8079750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 0x7fc9b8079c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 0x7fc9b80766a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b810f750 con 0x7fc9b8103c20 2024-06-02T23:31:33.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b810f5d0 con 0x7fc9b8076210 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9bd368640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9b810f8d0 con 0x7fc9b81057d0 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9b6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8103c20 0x7fc9b8079750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9b67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 0x7fc9b8079c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9b77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 0x7fc9b80766a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9b67fc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 0x7fc9b8079c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47216/0 (socket says 172.21.15.52:47216) 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9b77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 0x7fc9b80766a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58460/0 (socket says 172.21.15.52:58460) 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.145+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2235474441 learned_addr learned my addr 172.21.15.52:0/2235474441 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2235474441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 msgr2=0x7fc9b80766a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 0x7fc9b80766a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8103c20 msgr2=0x7fc9b8079750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8103c20 0x7fc9b8079750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2235474441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9b81a63b0 con 0x7fc9b81057d0 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8103c20 0x7fc9b8079750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.146+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 0x7fc9b80766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.147+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 0x7fc9b8079c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009230 tx=0x7fc9a0004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.147+0000 7fc9a7fff640 1 -- 172.21.15.52:0/2235474441 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a003f030 con 0x7fc9b81057d0 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.147+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9b81a6640 con 0x7fc9b81057d0 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.147+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9b81a6b00 con 0x7fc9b81057d0 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.148+0000 7fc9a7fff640 1 -- 172.21.15.52:0/2235474441 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a0003cb0 con 0x7fc9b81057d0 2024-06-02T23:31:33.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.148+0000 7fc9a7fff640 1 -- 172.21.15.52:0/2235474441 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0035070 con 0x7fc9b81057d0 2024-06-02T23:31:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.151+0000 7fc9c0072740 1 Processor -- start 2024-06-02T23:31:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 -- start start 2024-06-02T23:31:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42d2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42d7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42da000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0e42354f0 con 0x55c0e42d2010 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0e41d3870 con 0x55c0e42d2a00 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9c0072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0e42314b0 con 0x55c0e42d7b70 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9a7fff640 1 -- 172.21.15.52:0/2235474441 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc9a0042030 con 0x7fc9b81057d0 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9b6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42d7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9b77fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42d2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9b6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42d7500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58462/0 (socket says 172.21.15.52:58462) 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.152+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/254672372 learned_addr learned my addr 172.21.15.52:0/254672372 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42da000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/254672372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 msgr2=0x55c0e42da000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42da000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/254672372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 msgr2=0x55c0e42d2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42d2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/254672372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e41ef9b0 con 0x55c0e42d2a00 2024-06-02T23:31:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.153+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42d7500 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8010450 tx=0x7fc9a8010910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9818fdc8c1199ea9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.154+0000 7fc9a57fa640 1 -- 172.21.15.52:0/254672372 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8011760 con 0x55c0e42d2a00 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.154+0000 7fc9a7fff640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9ac0a2cb0 0x7fc9ac0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.154+0000 7fc9a57fa640 1 -- 172.21.15.52:0/254672372 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a8011db0 con 0x55c0e42d2a00 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.154+0000 7fc9a57fa640 1 -- 172.21.15.52:0/254672372 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a801a380 con 0x55c0e42d2a00 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.154+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9ac0a2cb0 0x7fc9ac0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.154+0000 7fc9a7fff640 1 -- 172.21.15.52:0/2235474441 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc9a00ef2d0 con 0x7fc9b81057d0 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 -- 172.21.15.52:0/254672372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 msgr2=0x55c0e42d7500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9ac0a2cb0 0x7fc9ac0a5170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55c0e42d3260 tx=0x7fc9b0007520 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42d7500 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8010450 tx=0x7fc9a8010910 comp rx=0 tx=0).stop 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 -- 172.21.15.52:0/254672372 shutdown_connections 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42da000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42d7500 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 --2- 172.21.15.52:0/254672372 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42d2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.155+0000 7fc9c0072740 1 -- 172.21.15.52:0/254672372 >> 172.21.15.52:0/254672372 conn(0x55c0e42cd680 msgr2=0x55c0e42cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.156+0000 7fc9c0072740 1 -- 172.21.15.52:0/254672372 shutdown_connections 2024-06-02T23:31:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.156+0000 7fc9c0072740 1 -- 172.21.15.52:0/254672372 wait complete. 2024-06-02T23:31:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.156+0000 7fc9c0072740 1 -- start start 2024-06-02T23:31:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.156+0000 7fc9c0072740 1 Processor -- start 2024-06-02T23:31:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2010 0x55c0e42dc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42dc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d7b70 0x55c0e42e3800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.160 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c0e42dab20 con 0x55c0e42d7b70 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c0e41beae0 con 0x55c0e42d2a00 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c0e42daca0 con 0x55c0e42d2010 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42dc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42dc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58470/0 (socket says 172.21.15.52:58470) 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 -- 172.21.15.52:0/307477016 learned_addr learned my addr 172.21.15.52:0/307477016 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d7b70 0x55c0e42e3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2010 0x55c0e42dc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9c0072740 10 client.0 _ll_register_callbacks cb 0x55c0e41f79b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 -- 172.21.15.52:0/307477016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2010 msgr2=0x55c0e42dc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2010 0x55c0e42dc280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 -- 172.21.15.52:0/307477016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d7b70 msgr2=0x55c0e42e3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d7b70 0x55c0e42e3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 -- 172.21.15.52:0/307477016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e3f49210 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.157+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42dc7c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00b8070 tx=0x7fc9a00b7eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abacfbd0a4df390e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc99e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc99e7fc640 1 -- 172.21.15.52:0/307477016 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0049070 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc99e7fc640 1 -- 172.21.15.52:0/307477016 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc9a00bbe70 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a8011db0 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc99e7fc640 1 -- 172.21.15.52:0/307477016 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc9a00bb570 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc9c0072740 10 client.29942 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c0e4015620 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.158+0000 7fc99e7fc640 1 -- 172.21.15.52:0/307477016 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a8011db0 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc99e7fc640 1 -- 172.21.15.52:0/307477016 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc9a00c4340 con 0x55c0e42d2a00 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9c0072740 10 client.29942 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9c0072740 10 client.29942 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:33.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9c0072740 20 client.29942 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9c0072740 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e42ea650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9c0072740 4 client.29942 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0e4248f10 con 0x55c0e42e81b0 2024-06-02T23:31:33.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.159+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e42ea650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.161+0000 7fc99e7fc640 10 client.29942 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.161+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e42ea650 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dbf096abb0b8ea1 server_cookie=81a36ce719fa004b in_seq=0 out_seq=0 2024-06-02T23:31:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.162+0000 7fc99e7fc640 1 -- 172.21.15.52:0/307477016 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c0e4248f10 con 0x55c0e42e81b0 2024-06-02T23:31:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.162+0000 7fc99e7fc640 10 client.29942 handle_command_reply: tid=0 2024-06-02T23:31:33.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.162+0000 7fc9c0072740 1 client.29942 shutdown 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 msgr2=0x55c0e42dc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42dc7c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00b8070 tx=0x7fc9a00b7eb0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 shutdown_connections 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e42ea650 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc99e7fc640 0 client.29942 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d7b70 0x55c0e42e3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2a00 0x55c0e42dc7c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 --2- 172.21.15.52:0/307477016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2010 0x55c0e42dc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 >> 172.21.15.52:0/307477016 conn(0x55c0e42cd680 msgr2=0x55c0e42cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 shutdown_connections 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.163+0000 7fc9c0072740 1 -- 172.21.15.52:0/307477016 wait complete. 2024-06-02T23:31:33.165 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.164+0000 7fc9c0072740 20 client.29942 trim_cache size 0 max 16384 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 Processor -- start 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 -- start start 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2010 0x55c0e42db550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2a00 0x55c0e42e37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42dba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800e3b0 con 0x55c0e42d2a00 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800fef0 con 0x55c0e42d2010 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9c0072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800ea80 con 0x55c0e42d7b70 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9b67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2a00 0x55c0e42e37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9b67fc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2a00 0x55c0e42e37c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50432/0 (socket says 172.21.15.52:50432) 2024-06-02T23:31:33.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9b6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2010 0x55c0e42db550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.186+0000 7fc9b67fc640 1 -- 172.21.15.52:0/351699575 learned_addr learned my addr 172.21.15.52:0/351699575 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:33.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42dba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/351699575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 msgr2=0x55c0e42dba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42dba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.191 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/351699575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2a00 msgr2=0x55c0e42e37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2a00 0x55c0e42e37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/351699575 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0e42cd4b0 con 0x55c0e42d2010 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2010 0x55c0e42db550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55c0e42ce550 tx=0x7fc9a801cb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d6b5a8510717c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9a4ff9640 1 -- 172.21.15.52:0/351699575 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a801c680 con 0x55c0e42d2010 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9a4ff9640 1 -- 172.21.15.52:0/351699575 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a80054f0 con 0x55c0e42d2010 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.187+0000 7fc9a4ff9640 1 -- 172.21.15.52:0/351699575 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8004d00 con 0x55c0e42d2010 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 -- 172.21.15.52:0/351699575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2010 msgr2=0x55c0e42db550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2010 0x55c0e42db550 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55c0e42ce550 tx=0x7fc9a801cb40 comp rx=0 tx=0).stop 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 -- 172.21.15.52:0/351699575 shutdown_connections 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d7b70 0x55c0e42dba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.192 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2a00 0x55c0e42e37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 --2- 172.21.15.52:0/351699575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d2010 0x55c0e42db550 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 -- 172.21.15.52:0/351699575 >> 172.21.15.52:0/351699575 conn(0x55c0e42cd680 msgr2=0x55c0e42d3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 -- 172.21.15.52:0/351699575 shutdown_connections 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.188+0000 7fc9c0072740 1 -- 172.21.15.52:0/351699575 wait complete. 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 -- start start 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 Processor -- start 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42e0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2a00 0x55c0e42e0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d7b70 0x55c0e4350ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800f820 con 0x55c0e42d2010 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800f150 con 0x55c0e42d7b70 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc99800c390 con 0x55c0e42d2a00 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9c0072740 10 client.0 _ll_register_callbacks cb 0x7fc9a0009260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:33.193 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9b6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42e0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9b6ffd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42e0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50448/0 (socket says 172.21.15.52:50448) 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9b6ffd640 1 -- 172.21.15.52:0/2517968840 learned_addr learned my addr 172.21.15.52:0/2517968840 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2a00 0x55c0e42e0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.189+0000 7fc9b77fe640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d7b70 0x55c0e4350ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d7b70 msgr2=0x55c0e4350ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d7b70 0x55c0e4350ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2517968840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 msgr2=0x55c0e42e0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42e0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b67fc640 1 -- 172.21.15.52:0/2517968840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0e42e3a20 con 0x55c0e42d2a00 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:33.194 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9b67fc640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2a00 0x55c0e42e0950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005ec0 tx=0x7fc9a00c45f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12d5cf19dac8807d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc99e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc99e7fc640 1 -- 172.21.15.52:0/2517968840 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0049040 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc99e7fc640 1 -- 172.21.15.52:0/2517968840 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc9a0003800 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a80054f0 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc99e7fc640 1 -- 172.21.15.52:0/2517968840 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc9a00d3070 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9c0072740 10 client.29581 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c0e4352460 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.190+0000 7fc99e7fc640 1 -- 172.21.15.52:0/2517968840 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc9a80054f0 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc99e7fc640 1 -- 172.21.15.52:0/2517968840 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc9a00baad0 con 0x55c0e42d2a00 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9c0072740 10 client.29581 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9c0072740 10 client.29581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9c0072740 20 client.29581 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:33.195 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9c0072740 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e4354b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9c0072740 4 client.29581 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0e4242c20 con 0x55c0e42e81b0 2024-06-02T23:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e4354b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc9b6ffd640 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e4354b10 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89385987801f208 server_cookie=5e4af9291f2da9d1 in_seq=0 out_seq=0 2024-06-02T23:31:33.196 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.191+0000 7fc99e7fc640 10 client.29581 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc99e7fc640 1 -- 172.21.15.52:0/2517968840 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55c0e4242c20 con 0x55c0e42e81b0 2024-06-02T23:31:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc99e7fc640 10 client.29581 handle_command_reply: tid=0 2024-06-02T23:31:33.250 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 client.29581 shutdown 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2a00 msgr2=0x55c0e42e0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2a00 0x55c0e42e0950 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0005ec0 tx=0x7fc9a00c45f0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 shutdown_connections 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c0e42e81b0 0x55c0e4354b10 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c0e42d2a00 0x55c0e42e0950 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c0e42d7b70 0x55c0e4350ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 --2- 172.21.15.52:0/2517968840 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c0e42d2010 0x55c0e42e0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 >> 172.21.15.52:0/2517968840 conn(0x55c0e42cd680 msgr2=0x55c0e42cbf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc99e7fc640 0 client.29581 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 shutdown_connections 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.249+0000 7fc9c0072740 1 -- 172.21.15.52:0/2517968840 wait complete. 2024-06-02T23:31:33.251 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9c0072740 20 client.29581 trim_cache size 0 max 16384 2024-06-02T23:31:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:31:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:31:33.252 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9ac0a2cb0 msgr2=0x7fc9ac0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9ac0a2cb0 0x7fc9ac0a5170 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x55c0e42d3260 tx=0x7fc9b0007520 comp rx=0 tx=0).stop 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 msgr2=0x7fc9b8079c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 0x7fc9b8079c90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009230 tx=0x7fc9a0004360 comp rx=0 tx=0).stop 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 shutdown_connections 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc9ac0a2cb0 0x7fc9ac0a5170 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9b81057d0 0x7fc9b8079c90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9b8076210 0x7fc9b80766a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 --2- 172.21.15.52:0/2235474441 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc9b8103c20 0x7fc9b8079750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:33.253 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 >> 172.21.15.52:0/2235474441 conn(0x7fc9b80ff190 msgr2=0x7fc9b8108110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 shutdown_connections 2024-06-02T23:31:33.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:33.250+0000 7fc9bd368640 1 -- 172.21.15.52:0/2235474441 wait complete. 2024-06-02T23:31:33.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 48078 6.77 MB/sec execute 281 sec latency 88.478 ms 2024-06-02T23:31:34.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:33 smithi123 ceph-mon[35977]: pgmap v1021: 97 pgs: 97 active+clean; 469 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-06-02T23:31:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:33 smithi174 ceph-mon[35994]: pgmap v1021: 97 pgs: 97 active+clean; 469 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-06-02T23:31:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:33 smithi052 ceph-mon[32340]: pgmap v1021: 97 pgs: 97 active+clean; 469 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 280 op/s 2024-06-02T23:31:34.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 48270 6.80 MB/sec execute 282 sec latency 55.935 ms 2024-06-02T23:31:35.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 48500 6.85 MB/sec execute 283 sec latency 104.246 ms 2024-06-02T23:31:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:35 smithi123 ceph-mon[35977]: pgmap v1022: 97 pgs: 97 active+clean; 472 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-06-02T23:31:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:35 smithi174 ceph-mon[35994]: pgmap v1022: 97 pgs: 97 active+clean; 472 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-06-02T23:31:36.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:35 smithi052 ceph-mon[32340]: pgmap v1022: 97 pgs: 97 active+clean; 472 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 281 op/s 2024-06-02T23:31:36.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 48699 6.86 MB/sec execute 284 sec latency 49.625 ms 2024-06-02T23:31:36.886 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:31:37.278 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:37.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 48815 6.85 MB/sec execute 285 sec latency 56.264 ms 2024-06-02T23:31:38.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:37 smithi123 ceph-mon[35977]: pgmap v1023: 97 pgs: 97 active+clean; 474 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 317 op/s 2024-06-02T23:31:38.018 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:37 smithi052 ceph-mon[32340]: pgmap v1023: 97 pgs: 97 active+clean; 474 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 317 op/s 2024-06-02T23:31:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:37 smithi174 ceph-mon[35994]: pgmap v1023: 97 pgs: 97 active+clean; 474 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 317 op/s 2024-06-02T23:31:38.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 48964 6.85 MB/sec execute 286 sec latency 81.746 ms 2024-06-02T23:31:39.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 49168 6.88 MB/sec execute 287 sec latency 63.427 ms 2024-06-02T23:31:39.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 -- 172.21.15.52:0/1922607665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a8103150 msgr2=0x7f93a8107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 --2- 172.21.15.52:0/1922607665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a8103150 0x7f93a8107140 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f93980099c0 tx=0x7f9398009cd0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 -- 172.21.15.52:0/1922607665 shutdown_connections 2024-06-02T23:31:39.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 --2- 172.21.15.52:0/1922607665 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 0x7f93a810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 --2- 172.21.15.52:0/1922607665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a81027a0 0x7f93a8102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 --2- 172.21.15.52:0/1922607665 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a8103150 0x7f93a8107140 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 -- 172.21.15.52:0/1922607665 >> 172.21.15.52:0/1922607665 conn(0x7f93a8079520 msgr2=0x7f93a8079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:39.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 -- 172.21.15.52:0/1922607665 shutdown_connections 2024-06-02T23:31:39.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.877+0000 7f93b07bb700 1 -- 172.21.15.52:0/1922607665 wait complete. 2024-06-02T23:31:39.879 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:39 smithi052 ceph-mon[32340]: pgmap v1024: 97 pgs: 97 active+clean; 476 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.878+0000 7f93b07bb700 1 Processor -- start 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.878+0000 7f93b07bb700 1 -- start start 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.878+0000 7f93b07bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a81027a0 0x7f93a806db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93b07bb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 0x7f93a806e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93b07bb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 0x7f93a8072480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93b07bb700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93a806c480 con 0x7f93a8107870 2024-06-02T23:31:39.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93b07bb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93a806c300 con 0x7f93a81027a0 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93b07bb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93a806c600 con 0x7f93a8103150 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a81027a0 0x7f93a806db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 0x7f93a806e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93aed58700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 0x7f93a8072480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 0x7f93a806e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:56530/0 (socket says 172.21.15.52:56530) 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 -- 172.21.15.52:0/4255966708 learned_addr learned my addr 172.21.15.52:0/4255966708 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 -- 172.21.15.52:0/4255966708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a81027a0 msgr2=0x7f93a806db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a81027a0 0x7f93a806db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 -- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 msgr2=0x7f93a8072480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 0x7f93a8072480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93add56700 1 -- 172.21.15.52:0/4255966708 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93a8072bb0 con 0x7f93a8103150 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.879+0000 7f93aed58700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 0x7f93a8072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:39.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.880+0000 7f93add56700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 0x7f93a806e080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9398005910 tx=0x7f9398004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:39.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.880+0000 7f939f7fe700 1 -- 172.21.15.52:0/4255966708 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9398026070 con 0x7f93a8103150 2024-06-02T23:31:39.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.880+0000 7f939f7fe700 1 -- 172.21.15.52:0/4255966708 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9398003c70 con 0x7f93a8103150 2024-06-02T23:31:39.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.880+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93a8072e40 con 0x7f93a8103150 2024-06-02T23:31:39.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.880+0000 7f939f7fe700 1 -- 172.21.15.52:0/4255966708 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f939800fb50 con 0x7f93a8103150 2024-06-02T23:31:39.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.880+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93a81abcd0 con 0x7f93a8103150 2024-06-02T23:31:39.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.883+0000 7f939f7fe700 1 -- 172.21.15.52:0/4255966708 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9398029070 con 0x7f93a8103150 2024-06-02T23:31:39.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.884+0000 7f939f7fe700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93940a2ca0 0x7f93940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.884+0000 7f939f7fe700 1 -- 172.21.15.52:0/4255966708 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f93980cf800 con 0x7f93a8103150 2024-06-02T23:31:39.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.884+0000 7f93ae557700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93940a2ca0 0x7f93940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.885+0000 7f93ae557700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93940a2ca0 0x7f93940a5160 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009bf0 tx=0x7f93a4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:39.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.895+0000 7f93bb03eb80 1 Processor -- start 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.895+0000 7f93bb03eb80 1 -- start start 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16acd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16acd7a0 0x557c16acdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93bb03eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557c169bff60 con 0x557c16acd7a0 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93bb03eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c16a1e850 con 0x557c16ad2f60 2024-06-02T23:31:39.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93bb03eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557c16986460 con 0x557c16ad3880 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad3340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46896/0 (socket says 172.21.15.52:46896) 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 -- 172.21.15.52:0/260339717 learned_addr learned my addr 172.21.15.52:0/260339717 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93add56700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16acd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93aed58700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16acd7a0 0x557c16acdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 -- 172.21.15.52:0/260339717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 msgr2=0x557c16acd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16acd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 -- 172.21.15.52:0/260339717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16acd7a0 msgr2=0x557c16acdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16acd7a0 0x557c16acdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 -- 172.21.15.52:0/260339717 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c1696d8d0 con 0x557c16ad2f60 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93add56700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16acd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:39.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93aed58700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16acd7a0 0x557c16acdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:39.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.896+0000 7f93ae557700 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad3340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f93a4011e40 tx=0x7f93a4014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a62ff3e093c47b4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:39.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.897+0000 7f939cff9700 1 -- 172.21.15.52:0/260339717 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4004220 con 0x557c16ad2f60 2024-06-02T23:31:39.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.897+0000 7f939cff9700 1 -- 172.21.15.52:0/260339717 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a4011e70 con 0x557c16ad2f60 2024-06-02T23:31:39.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.897+0000 7f939cff9700 1 -- 172.21.15.52:0/260339717 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a401f820 con 0x557c16ad2f60 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.897+0000 7f93bb03eb80 1 -- 172.21.15.52:0/260339717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 msgr2=0x557c16ad3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.897+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad3340 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f93a4011e40 tx=0x7f93a4014000 comp rx=0 tx=0).stop 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 -- 172.21.15.52:0/260339717 shutdown_connections 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16acd7a0 0x557c16acdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad3340 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/260339717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16acd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 -- 172.21.15.52:0/260339717 >> 172.21.15.52:0/260339717 conn(0x557c169fa080 msgr2=0x557c16ac8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:39.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 -- 172.21.15.52:0/260339717 shutdown_connections 2024-06-02T23:31:39.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 -- 172.21.15.52:0/260339717 wait complete. 2024-06-02T23:31:39.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.898+0000 7f93bb03eb80 1 -- start start 2024-06-02T23:31:39.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.899+0000 7f93bb03eb80 1 Processor -- start 2024-06-02T23:31:39.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.899+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16ad9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.899+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16ad6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.899+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16ad6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.899+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16ad9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.899+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16ad9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46910/0 (socket says 172.21.15.52:46910) 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93ae557700 1 -- 172.21.15.52:0/1978003344 learned_addr learned my addr 172.21.15.52:0/1978003344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93aed58700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16ad6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x557c169befc0 con 0x557c16ad2f60 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93add56700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16ad6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557c16a1a810 con 0x557c16acd7a0 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557c16ada260 con 0x557c16ad3880 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93bb03eb80 10 client.0 _ll_register_callbacks cb 0x557c169b45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:39.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93aed58700 1 -- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 msgr2=0x557c16ad9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93aed58700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16ad9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93aed58700 1 -- 172.21.15.52:0/1978003344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 msgr2=0x557c16ad6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93aed58700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16ad6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.900+0000 7f93aed58700 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c169e4750 con 0x557c16ad3880 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f93aed58700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16ad6ed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f93a000eab0 tx=0x7f93a000edc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=231b043678f7c290 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f9391ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f9391ffb700 1 -- 172.21.15.52:0/1978003344 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a001cbb0 con 0x557c16ad3880 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f9391ffb700 1 -- 172.21.15.52:0/1978003344 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f93a001cd50 con 0x557c16ad3880 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a4011e70 con 0x557c16ad3880 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f9391ffb700 1 -- 172.21.15.52:0/1978003344 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f93a0007970 con 0x557c16ad3880 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f93bb03eb80 10 client.29593 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.901+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557c16ae4b00 con 0x557c16ad3880 2024-06-02T23:31:39.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f9391ffb700 1 -- 172.21.15.52:0/1978003344 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a0025430 con 0x557c16ad3880 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f9391ffb700 1 -- 172.21.15.52:0/1978003344 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f93a0030420 con 0x557c16ad3880 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93bb03eb80 10 client.29593 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93bb03eb80 10 client.29593 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93bb03eb80 20 client.29593 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16ae7190 0x557c16ae9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93bb03eb80 4 client.29593 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f93a4011e70 con 0x557c16ae7190 2024-06-02T23:31:39.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.902+0000 7f93ae557700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16ae7190 0x557c16ae9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.903+0000 7f93ae557700 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16ae7190 0x557c16ae9580 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf038d07a834ff80 server_cookie=424e794352bda0fa in_seq=0 out_seq=0 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.903+0000 7f9391ffb700 10 client.29593 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f9391ffb700 1 -- 172.21.15.52:0/1978003344 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f93a4011e70 con 0x557c16ae7190 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f9391ffb700 10 client.29593 handle_command_reply: tid=0 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f93bb03eb80 1 client.29593 shutdown 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 msgr2=0x557c16ad6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16ad6ed0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f93a000eab0 tx=0x7f93a000edc0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 shutdown_connections 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16ae7190 0x557c16ae9580 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.904+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16ad6ed0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16ad9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1978003344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16ad6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f9391ffb700 0 client.29593 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 >> 172.21.15.52:0/1978003344 conn(0x557c169fa080 msgr2=0x557c16ac7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 shutdown_connections 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1978003344 wait complete. 2024-06-02T23:31:39.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.905+0000 7f93bb03eb80 20 client.29593 trim_cache size 0 max 16384 2024-06-02T23:31:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 Processor -- start 2024-06-02T23:31:39.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 -- start start 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16acd7a0 0x557c16acd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad91d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 0x557c16ac7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81b2f30 con 0x557c16ad3880 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81b4a70 con 0x557c16ad2f60 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93bb03eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81b3600 con 0x557c16acd7a0 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93add56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 0x557c16ac7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93add56700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46922/0 (socket says 172.21.15.52:46922) 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 0x557c16ac7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37122/0 (socket says 172.21.15.52:37122) 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93ae557700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16acd7a0 0x557c16acd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 -- 172.21.15.52:0/1625077959 learned_addr learned my addr 172.21.15.52:0/1625077959 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:39.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 -- 172.21.15.52:0/1625077959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16acd7a0 msgr2=0x557c16acd260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16acd7a0 0x557c16acd260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 -- 172.21.15.52:0/1625077959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 msgr2=0x557c16ad91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad91d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.946+0000 7f93aed58700 1 -- 172.21.15.52:0/1625077959 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c16ad3680 con 0x557c16ad3880 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.947+0000 7f93add56700 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.947+0000 7f93aed58700 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 0x557c16ac7e80 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f93a0002230 tx=0x7f93a0030f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21af588c76ffe251 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.947+0000 7f9393fff700 1 -- 172.21.15.52:0/1625077959 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0034070 con 0x557c16ad3880 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.947+0000 7f9393fff700 1 -- 172.21.15.52:0/1625077959 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a0030b70 con 0x557c16ad3880 2024-06-02T23:31:39.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.947+0000 7f9393fff700 1 -- 172.21.15.52:0/1625077959 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0003c60 con 0x557c16ad3880 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1625077959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 msgr2=0x557c16ac7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 0x557c16ac7e80 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f93a0002230 tx=0x7f93a0030f20 comp rx=0 tx=0).stop 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1625077959 shutdown_connections 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad3880 0x557c16ac7e80 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16acd7a0 0x557c16acd260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/1625077959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16ad2f60 0x557c16ad91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1625077959 >> 172.21.15.52:0/1625077959 conn(0x557c169fa080 msgr2=0x557c16ad6ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1625077959 shutdown_connections 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.948+0000 7f93bb03eb80 1 -- 172.21.15.52:0/1625077959 wait complete. 2024-06-02T23:31:39.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93bb03eb80 1 -- start start 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93bb03eb80 1 Processor -- start 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16adf520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16adfa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93bb03eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16adc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16adf520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93ae557700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16adf520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46936/0 (socket says 172.21.15.52:46936) 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93ae557700 1 -- 172.21.15.52:0/480537027 learned_addr learned my addr 172.21.15.52:0/480537027 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81b43a0 con 0x557c16ad2f60 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.949+0000 7f93add56700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16adfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81b3cd0 con 0x557c16acd7a0 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93a81b0ef0 con 0x557c16ad3880 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93ae557700 1 -- 172.21.15.52:0/480537027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 msgr2=0x557c16adc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93ae557700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16adc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93ae557700 1 -- 172.21.15.52:0/480537027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 msgr2=0x557c16adfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93ae557700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16adfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93ae557700 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c16adca30 con 0x557c16acd7a0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93add56700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16adfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93bb03eb80 10 client.0 _ll_register_callbacks cb 0x557c169b45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a0030b70 con 0x557c16acd7a0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f93ae557700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16adf520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93a40097d0 tx=0x7f93a4011f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3bdab44eb77d4c88 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f9391ffb700 10 client.29966 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.950+0000 7f9391ffb700 1 -- 172.21.15.52:0/480537027 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4044050 con 0x557c16acd7a0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f9391ffb700 1 -- 172.21.15.52:0/480537027 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f93a4030040 con 0x557c16acd7a0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f9391ffb700 1 -- 172.21.15.52:0/480537027 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f93a4005440 con 0x557c16acd7a0 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 10 client.29966 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:39.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=43}) -- 0x557c16b52490 con 0x557c16acd7a0 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f9391ffb700 1 -- 172.21.15.52:0/480537027 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f93a40046f0 con 0x557c16acd7a0 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 10 client.29966 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 10 client.29966 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 20 client.29966 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16b528a0 0x557c16b54d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 4 client.29966 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c16a2bd50 con 0x557c16b528a0 2024-06-02T23:31:39.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.951+0000 7f93add56700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16b528a0 0x557c16b54d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93add56700 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16b528a0 0x557c16b54d10 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8af5b893e8aeef1 server_cookie=f06c2824d7d25272 in_seq=0 out_seq=0 2024-06-02T23:31:39.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f9391ffb700 10 client.29966 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f9391ffb700 1 -- 172.21.15.52:0/480537027 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557c16a2bd50 con 0x557c16b528a0 2024-06-02T23:31:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f9391ffb700 10 client.29966 handle_command_reply: tid=0 2024-06-02T23:31:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 client.29966 shutdown 2024-06-02T23:31:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 msgr2=0x557c16adf520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.957 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16adf520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f93a40097d0 tx=0x7f93a4011f50 comp rx=0 tx=0).stop 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 shutdown_connections 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x557c16b528a0 0x557c16b54d10 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557c16ad3880 0x557c16adc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f9391ffb700 0 client.29966 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557c16acd7a0 0x557c16adf520 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.952+0000 7f93bb03eb80 1 --2- 172.21.15.52:0/480537027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557c16ad2f60 0x557c16adfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.953+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 >> 172.21.15.52:0/480537027 conn(0x557c169fa080 msgr2=0x557c16ad1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.953+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 shutdown_connections 2024-06-02T23:31:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.953+0000 7f93bb03eb80 1 -- 172.21.15.52:0/480537027 wait complete. 2024-06-02T23:31:39.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.953+0000 7f93bb03eb80 20 client.29966 trim_cache size 0 max 16384 2024-06-02T23:31:39.959 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:31:39.960 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:31:39.960 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:31:39.960 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:31:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93940a2ca0 msgr2=0x7f93940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93940a2ca0 0x7f93940a5160 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009bf0 tx=0x7f93a4009260 comp rx=0 tx=0).stop 2024-06-02T23:31:39.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 msgr2=0x7f93a806e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 0x7f93a806e080 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9398005910 tx=0x7f9398004270 comp rx=0 tx=0).stop 2024-06-02T23:31:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 shutdown_connections 2024-06-02T23:31:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93940a2ca0 0x7f93940a5160 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93a8107870 0x7f93a8072480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93a81027a0 0x7f93a806db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 --2- 172.21.15.52:0/4255966708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93a8103150 0x7f93a806e080 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 >> 172.21.15.52:0/4255966708 conn(0x7f93a8079520 msgr2=0x7f93a81063d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 shutdown_connections 2024-06-02T23:31:39.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:39.954+0000 7f93b07bb700 1 -- 172.21.15.52:0/4255966708 wait complete. 2024-06-02T23:31:40.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:39 smithi123 ceph-mon[35977]: pgmap v1024: 97 pgs: 97 active+clean; 476 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-06-02T23:31:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:39 smithi174 ceph-mon[35994]: pgmap v1024: 97 pgs: 97 active+clean; 476 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 271 op/s 2024-06-02T23:31:40.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 49320 6.88 MB/sec execute 288 sec latency 62.940 ms 2024-06-02T23:31:40.633 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:31:40.633 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:31:41.005 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:41.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 49452 6.87 MB/sec execute 289 sec latency 59.985 ms 2024-06-02T23:31:42.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:41 smithi123 ceph-mon[35977]: pgmap v1025: 97 pgs: 97 active+clean; 482 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 302 op/s 2024-06-02T23:31:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:41 smithi174 ceph-mon[35994]: pgmap v1025: 97 pgs: 97 active+clean; 482 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 302 op/s 2024-06-02T23:31:42.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:41 smithi052 ceph-mon[32340]: pgmap v1025: 97 pgs: 97 active+clean; 482 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 302 op/s 2024-06-02T23:31:42.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 49621 6.90 MB/sec execute 290 sec latency 86.004 ms 2024-06-02T23:31:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 -- 172.21.15.52:0/1922399024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101d50 msgr2=0x7f9a7010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 --2- 172.21.15.52:0/1922399024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101d50 0x7f9a7010e240 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f9a6800b0a0 tx=0x7f9a6800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 -- 172.21.15.52:0/1922399024 shutdown_connections 2024-06-02T23:31:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 --2- 172.21.15.52:0/1922399024 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101d50 0x7f9a7010e240 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 --2- 172.21.15.52:0/1922399024 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 0x7f9a701077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 --2- 172.21.15.52:0/1922399024 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a70101390 0x7f9a70101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 -- 172.21.15.52:0/1922399024 >> 172.21.15.52:0/1922399024 conn(0x7f9a70078340 msgr2=0x7f9a70078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 -- 172.21.15.52:0/1922399024 shutdown_connections 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.335+0000 7f9a77e44700 1 -- 172.21.15.52:0/1922399024 wait complete. 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.336+0000 7f9a77e44700 1 Processor -- start 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.336+0000 7f9a77e44700 1 -- start start 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a77e44700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101390 0x7f9a7019c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a77e44700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a70101d50 0x7f9a7019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a77e44700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 0x7f9a70196820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a77e44700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9a701104e0 con 0x7f9a70101390 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a77e44700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70110360 con 0x7f9a70107400 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a77e44700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a70110660 con 0x7f9a70101d50 2024-06-02T23:31:43.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a763e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 0x7f9a70196820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a75be0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101390 0x7f9a7019c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a763e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 0x7f9a70196820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:46956/0 (socket says 172.21.15.52:46956) 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a763e1700 1 -- 172.21.15.52:0/2922476549 learned_addr learned my addr 172.21.15.52:0/2922476549 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a753df700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a70101d50 0x7f9a7019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a75be0700 1 -- 172.21.15.52:0/2922476549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a70101d50 msgr2=0x7f9a7019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a75be0700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a70101d50 0x7f9a7019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a75be0700 1 -- 172.21.15.52:0/2922476549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 msgr2=0x7f9a70196820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a75be0700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 0x7f9a70196820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.337+0000 7f9a75be0700 1 -- 172.21.15.52:0/2922476549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a70197000 con 0x7f9a70101390 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.338+0000 7f9a75be0700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101390 0x7f9a7019c6a0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009ef0 tx=0x7f9a6c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.338+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c020070 con 0x7f9a70101390 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.338+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a701972f0 con 0x7f9a70101390 2024-06-02T23:31:43.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.338+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9a6c00c7c0 con 0x7f9a70101390 2024-06-02T23:31:43.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.338+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a701a3500 con 0x7f9a70101390 2024-06-02T23:31:43.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.338+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01b7c0 con 0x7f9a70101390 2024-06-02T23:31:43.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.340+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a70102ab0 con 0x7f9a70101390 2024-06-02T23:31:43.342 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.341+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9a6c023070 con 0x7f9a70101390 2024-06-02T23:31:43.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.342+0000 7f9a66ffd700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9a5c0a2ca0 0x7f9a5c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:43.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.342+0000 7f9a753df700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9a5c0a2ca0 0x7f9a5c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:43.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.342+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9a6c0d2de0 con 0x7f9a70101390 2024-06-02T23:31:43.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.343+0000 7f9a753df700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9a5c0a2ca0 0x7f9a5c0a5160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b3c0 tx=0x7f9a6000d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:43.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.347+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f9a6c09fd70 con 0x7f9a70101390 2024-06-02T23:31:43.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 49788 6.90 MB/sec execute 291 sec latency 63.580 ms 2024-06-02T23:31:43.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.760+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a70197d50 con 0x7f9a70101390 2024-06-02T23:31:43.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.761+0000 7f9a66ffd700 1 -- 172.21.15.52:0/2922476549 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f9a6c09f670 con 0x7f9a70101390 2024-06-02T23:31:43.761 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:31:43.762 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:31:43.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9a5c0a2ca0 msgr2=0x7f9a5c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:43.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9a5c0a2ca0 0x7f9a5c0a5160 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9a6000b3c0 tx=0x7f9a6000d040 comp rx=0 tx=0).stop 2024-06-02T23:31:43.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101390 msgr2=0x7f9a7019c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:43.764 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101390 0x7f9a7019c6a0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c009ef0 tx=0x7f9a6c00ecd0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 shutdown_connections 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9a5c0a2ca0 0x7f9a5c0a5160 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a70107400 0x7f9a70196820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9a70101390 0x7f9a7019c6a0 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 --2- 172.21.15.52:0/2922476549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a70101d50 0x7f9a7019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 >> 172.21.15.52:0/2922476549 conn(0x7f9a70078340 msgr2=0x7f9a701001f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 shutdown_connections 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:43.764+0000 7f9a77e44700 1 -- 172.21.15.52:0/2922476549 wait complete. 2024-06-02T23:31:43.765 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:31:44.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:43 smithi123 ceph-mon[35977]: pgmap v1026: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.7 MiB/s wr, 262 op/s 2024-06-02T23:31:44.104 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:43 smithi052 ceph-mon[32340]: pgmap v1026: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.7 MiB/s wr, 262 op/s 2024-06-02T23:31:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:43 smithi174 ceph-mon[35994]: pgmap v1026: 97 pgs: 97 active+clean; 483 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 935 KiB/s rd, 4.7 MiB/s wr, 262 op/s 2024-06-02T23:31:44.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 49952 6.89 MB/sec execute 292 sec latency 109.952 ms 2024-06-02T23:31:44.469 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:31:44.835 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:44.968 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:44 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2922476549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:31:45.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:44 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2922476549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:31:45.044 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:44 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2922476549' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:31:45.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 50098 6.88 MB/sec execute 293 sec latency 71.576 ms 2024-06-02T23:31:45.993 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:45 smithi052 ceph-mon[32340]: pgmap v1027: 97 pgs: 97 active+clean; 481 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 265 op/s 2024-06-02T23:31:46.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:45 smithi123 ceph-mon[35977]: pgmap v1027: 97 pgs: 97 active+clean; 481 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 265 op/s 2024-06-02T23:31:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:45 smithi174 ceph-mon[35994]: pgmap v1027: 97 pgs: 97 active+clean; 481 MiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.7 MiB/s wr, 265 op/s 2024-06-02T23:31:46.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 50255 6.87 MB/sec execute 294 sec latency 66.115 ms 2024-06-02T23:31:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 -- 172.21.15.52:0/4109609516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9778101b10 msgr2=0x7f9778101f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 --2- 172.21.15.52:0/4109609516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9778101b10 0x7f9778101f90 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f9768009900 tx=0x7f976802a410 comp rx=0 tx=0).stop 2024-06-02T23:31:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 -- 172.21.15.52:0/4109609516 shutdown_connections 2024-06-02T23:31:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 --2- 172.21.15.52:0/4109609516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97781024d0 0x7f9778108d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 --2- 172.21.15.52:0/4109609516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9778101b10 0x7f9778101f90 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 --2- 172.21.15.52:0/4109609516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 0x7f9778100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.468+0000 7f977f963640 1 -- 172.21.15.52:0/4109609516 >> 172.21.15.52:0/4109609516 conn(0x7f97780fbe90 msgr2=0x7f97780fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.469+0000 7f977f963640 1 -- 172.21.15.52:0/4109609516 shutdown_connections 2024-06-02T23:31:46.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.469+0000 7f977f963640 1 -- 172.21.15.52:0/4109609516 wait complete. 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.469+0000 7f977f963640 1 Processor -- start 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 -- start start 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 0x7f9778196ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9778101b10 0x7f9778196ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97781024d0 0x7f977819e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f977810c4a0 con 0x7f97781024d0 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f977810c320 con 0x7f9778101b10 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977f963640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f977810c620 con 0x7f9778100920 2024-06-02T23:31:46.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 0x7f9778196ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977cefe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9778101b10 0x7f9778196ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 0x7f9778196ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:57652/0 (socket says 172.21.15.52:57652) 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977d6ff640 1 -- 172.21.15.52:0/2486369550 learned_addr learned my addr 172.21.15.52:0/2486369550 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977df00640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97781024d0 0x7f977819e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977cefe640 1 -- 172.21.15.52:0/2486369550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 msgr2=0x7f9778196ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977cefe640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 0x7f9778196ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977cefe640 1 -- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97781024d0 msgr2=0x7f977819e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977cefe640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97781024d0 0x7f977819e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.470+0000 7f977cefe640 1 -- 172.21.15.52:0/2486369550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977819e6d0 con 0x7f9778101b10 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.471+0000 7f977df00640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97781024d0 0x7f977819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.471+0000 7f977cefe640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9778101b10 0x7f9778196ff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f976802a930 tx=0x7f9768003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.471+0000 7f976e7fc640 1 -- 172.21.15.52:0/2486369550 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976803f070 con 0x7f9778101b10 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.471+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977806a170 con 0x7f9778101b10 2024-06-02T23:31:46.473 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.471+0000 7f976e7fc640 1 -- 172.21.15.52:0/2486369550 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9768031c40 con 0x7f9778101b10 2024-06-02T23:31:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.471+0000 7f976e7fc640 1 -- 172.21.15.52:0/2486369550 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768035030 con 0x7f9778101b10 2024-06-02T23:31:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.472+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977806a680 con 0x7f9778101b10 2024-06-02T23:31:46.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.474+0000 7f976e7fc640 1 -- 172.21.15.52:0/2486369550 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9768042070 con 0x7f9778101b10 2024-06-02T23:31:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.475+0000 7f976e7fc640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97640a2cb0 0x7f97640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.475+0000 7f976e7fc640 1 -- 172.21.15.52:0/2486369550 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f97680ef690 con 0x7f9778101b10 2024-06-02T23:31:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.475+0000 7f977d6ff640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97640a2cb0 0x7f97640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.476+0000 7f977d6ff640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97640a2cb0 0x7f97640a5170 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97740058c0 tx=0x7f9774010040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 Processor -- start 2024-06-02T23:31:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 -- start start 2024-06-02T23:31:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132bb420 0x5599132bb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132c0f80 0x5599132c3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55991321a4f0 con 0x5599132c0f80 2024-06-02T23:31:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599131b8870 con 0x5599132bb420 2024-06-02T23:31:46.478 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f9782677740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599132164b0 con 0x5599132bbe10 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977df00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132c0f80 0x5599132c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132bb420 0x5599132bb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977cefe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977df00640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132c0f80 0x5599132c3410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36676/0 (socket says 172.21.15.52:36676) 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977df00640 1 -- 172.21.15.52:0/2582496168 learned_addr learned my addr 172.21.15.52:0/2582496168 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977cefe640 1 -- 172.21.15.52:0/2582496168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132bb420 msgr2=0x5599132bb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977cefe640 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132bb420 0x5599132bb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977cefe640 1 -- 172.21.15.52:0/2582496168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132c0f80 msgr2=0x5599132c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977cefe640 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132c0f80 0x5599132c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.477+0000 7f977cefe640 1 -- 172.21.15.52:0/2582496168 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599131d49b0 con 0x5599132bbe10 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.478+0000 7f977cefe640 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132c0910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97680ed210 tx=0x7f97680045e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a0f00b855b50ba0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:46.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.478+0000 7f97637fe640 1 -- 172.21.15.52:0/2582496168 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9768049050 con 0x5599132bbe10 2024-06-02T23:31:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.478+0000 7f97637fe640 1 -- 172.21.15.52:0/2582496168 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97680bbf20 con 0x5599132bbe10 2024-06-02T23:31:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.478+0000 7f97637fe640 1 -- 172.21.15.52:0/2582496168 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f976803b4f0 con 0x5599132bbe10 2024-06-02T23:31:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 -- 172.21.15.52:0/2582496168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 msgr2=0x5599132c0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132c0910 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f97680ed210 tx=0x7f97680045e0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 -- 172.21.15.52:0/2582496168 shutdown_connections 2024-06-02T23:31:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132c0910 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132c0f80 0x5599132c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 --2- 172.21.15.52:0/2582496168 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132bb420 0x5599132bb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 -- 172.21.15.52:0/2582496168 >> 172.21.15.52:0/2582496168 conn(0x5599132b2680 msgr2=0x5599132b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 -- 172.21.15.52:0/2582496168 shutdown_connections 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 -- 172.21.15.52:0/2582496168 wait complete. 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.479+0000 7f9782677740 1 -- start start 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f9782677740 1 Processor -- start 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f9782677740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bb420 0x5599132c5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f9782677740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 0x5599132c59c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bb420 0x5599132c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f977cefe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 0x5599132c59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f977cefe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 0x5599132c59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36678/0 (socket says 172.21.15.52:36678) 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f977cefe640 1 -- 172.21.15.52:0/1240962122 learned_addr learned my addr 172.21.15.52:0/1240962122 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.480+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5599132c3cf0 con 0x5599132bbe10 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5599131a3ae0 con 0x5599132c0f80 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5599132c3e70 con 0x5599132bb420 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977df00640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9782677740 10 client.0 _ll_register_callbacks cb 0x5599131dc9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977cefe640 1 -- 172.21.15.52:0/1240962122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bb420 msgr2=0x5599132c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977cefe640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bb420 0x5599132c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977cefe640 1 -- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 msgr2=0x5599132cca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977cefe640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977cefe640 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559912f2e210 con 0x5599132bbe10 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977df00640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f977cefe640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 0x5599132c59c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f97680dc190 tx=0x7f97680bc750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebe1a340a5c6f63c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97680bbf20 con 0x5599132bbe10 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9760ff9640 1 -- 172.21.15.52:0/1240962122 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97680a2050 con 0x5599132bbe10 2024-06-02T23:31:46.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.481+0000 7f9760ff9640 1 -- 172.21.15.52:0/1240962122 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f97680bc380 con 0x5599132bbe10 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9760ff9640 1 -- 172.21.15.52:0/1240962122 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f97680c4700 con 0x5599132bbe10 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 10 client.20679 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=43}) -- 0x559912ffa620 con 0x5599132bbe10 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9760ff9640 1 -- 172.21.15.52:0/1240962122 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f97680dab60 con 0x5599132bbe10 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 10 client.20679 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 10 client.20679 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 20 client.20679 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5599132d0bd0 0x5599132d3020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 4 client.20679 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55991322df10 con 0x5599132d0bd0 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.482+0000 7f977d6ff640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5599132d0bd0 0x5599132d3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.483+0000 7f977d6ff640 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5599132d0bd0 0x5599132d3020 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=129508cca2958553 server_cookie=d2b6376ba70b7033 in_seq=0 out_seq=0 2024-06-02T23:31:46.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.483+0000 7f9760ff9640 10 client.20679 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.483+0000 7f9760ff9640 1 -- 172.21.15.52:0/1240962122 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55991322df10 con 0x5599132d0bd0 2024-06-02T23:31:46.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.483+0000 7f9760ff9640 10 client.20679 handle_command_reply: tid=0 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.483+0000 7f9782677740 1 client.20679 shutdown 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.483+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 msgr2=0x5599132c59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 0x5599132c59c0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f97680dc190 tx=0x7f97680bc750 comp rx=0 tx=0).stop 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 shutdown_connections 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5599132d0bd0 0x5599132d3020 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bbe10 0x5599132c59c0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9760ff9640 0 client.20679 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 --2- 172.21.15.52:0/1240962122 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bb420 0x5599132c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 >> 172.21.15.52:0/1240962122 conn(0x5599132b2680 msgr2=0x5599132b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 shutdown_connections 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 1 -- 172.21.15.52:0/1240962122 wait complete. 2024-06-02T23:31:46.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.484+0000 7f9782677740 20 client.20679 trim_cache size 0 max 16384 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.511+0000 7f9782677740 1 Processor -- start 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.511+0000 7f9782677740 1 -- start start 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f9782677740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132bd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f9782677740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132cc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f9782677740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132bd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f9782677740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97781a6ad0 con 0x5599132bb420 2024-06-02T23:31:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f9782677740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97781a8610 con 0x5599132c0f80 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f9782677740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97781a71a0 con 0x5599132bbe10 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132bd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132bd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36682/0 (socket says 172.21.15.52:36682) 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 -- 172.21.15.52:0/435451588 learned_addr learned my addr 172.21.15.52:0/435451588 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977cefe640 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132cc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977df00640 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132bd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 -- 172.21.15.52:0/435451588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 msgr2=0x5599132cc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132cc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 -- 172.21.15.52:0/435451588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 msgr2=0x5599132bd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132bd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 -- 172.21.15.52:0/435451588 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599132d36a0 con 0x5599132bb420 2024-06-02T23:31:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977df00640 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132bd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.512+0000 7f977d6ff640 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132bd0d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x5599132bc670 tx=0x7f9774016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cd0acf5bcfb09b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.513+0000 7f9762ffd640 1 -- 172.21.15.52:0/435451588 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774032070 con 0x5599132bb420 2024-06-02T23:31:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.513+0000 7f9762ffd640 1 -- 172.21.15.52:0/435451588 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97740268d0 con 0x5599132bb420 2024-06-02T23:31:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.513+0000 7f9762ffd640 1 -- 172.21.15.52:0/435451588 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9774020040 con 0x5599132bb420 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 -- 172.21.15.52:0/435451588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 msgr2=0x5599132bd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132bd0d0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x5599132bc670 tx=0x7f9774016000 comp rx=0 tx=0).stop 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 -- 172.21.15.52:0/435451588 shutdown_connections 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132cc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132bd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 --2- 172.21.15.52:0/435451588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132bd0d0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.517 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 -- 172.21.15.52:0/435451588 >> 172.21.15.52:0/435451588 conn(0x5599132b2680 msgr2=0x5599132b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 -- 172.21.15.52:0/435451588 shutdown_connections 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 -- 172.21.15.52:0/435451588 wait complete. 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 -- start start 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.514+0000 7f9782677740 1 Processor -- start 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97781a7f40 con 0x5599132bb420 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97781a7870 con 0x5599132c0f80 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97781a4ab0 con 0x5599132bbe10 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f9782677740 10 client.0 _ll_register_callbacks cb 0x7f97680d3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977cefe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132ca1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36692/0 (socket says 172.21.15.52:36692) 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 -- 172.21.15.52:0/1397219368 learned_addr learned my addr 172.21.15.52:0/1397219368 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977df00640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 -- 172.21.15.52:0/1397219368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 msgr2=0x5599132ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 -- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 msgr2=0x5599132cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.515+0000 7f977d6ff640 1 -- 172.21.15.52:0/1397219368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599132b3350 con 0x5599132bb420 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f977df00640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f977d6ff640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132ca1e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9774009870 tx=0x7f97740042c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=113999eaf46a1240 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9760ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9760ff9640 1 -- 172.21.15.52:0/1397219368 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977403b060 con 0x5599132bb420 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9760ff9640 1 -- 172.21.15.52:0/1397219368 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9774026620 con 0x5599132bb420 2024-06-02T23:31:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97740268d0 con 0x5599132bb420 2024-06-02T23:31:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9760ff9640 1 -- 172.21.15.52:0/1397219368 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9774032050 con 0x5599132bb420 2024-06-02T23:31:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9782677740 10 client.20685 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.516+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5599132b24b0 con 0x5599132bb420 2024-06-02T23:31:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9760ff9640 1 -- 172.21.15.52:0/1397219368 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9774005040 con 0x5599132bb420 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9760ff9640 1 -- 172.21.15.52:0/1397219368 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f97740048a0 con 0x5599132bb420 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9782677740 10 client.20685 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9782677740 10 client.20685 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9782677740 20 client.20685 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9782677740 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55991333bf70 0x55991333e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9782677740 4 client.20685 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f97740268d0 con 0x55991333bf70 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.517+0000 7f977cefe640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55991333bf70 0x55991333e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.518+0000 7f9760ff9640 10 client.20685 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.518+0000 7f977cefe640 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55991333bf70 0x55991333e390 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8b554744ea523e5 server_cookie=ba571af9effe8698 in_seq=0 out_seq=0 2024-06-02T23:31:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9760ff9640 1 -- 172.21.15.52:0/1397219368 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f97740268d0 con 0x55991333bf70 2024-06-02T23:31:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9760ff9640 10 client.20685 handle_command_reply: tid=0 2024-06-02T23:31:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 client.20685 shutdown 2024-06-02T23:31:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 msgr2=0x5599132ca1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132ca1e0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f9774009870 tx=0x7f97740042c0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.586 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 shutdown_connections 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55991333bf70 0x55991333e390 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5599132bbe10 0x5599132ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5599132c0f80 0x5599132cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 --2- 172.21.15.52:0/1397219368 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5599132bb420 0x5599132ca1e0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 >> 172.21.15.52:0/1397219368 conn(0x5599132b2680 msgr2=0x5599132c0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9760ff9640 0 client.20685 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 shutdown_connections 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 1 -- 172.21.15.52:0/1397219368 wait complete. 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.585+0000 7f9782677740 20 client.20685 trim_cache size 0 max 16384 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:31:46.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:31:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:31:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97640a2cb0 msgr2=0x7f97640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97640a2cb0 0x7f97640a5170 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f97740058c0 tx=0x7f9774010040 comp rx=0 tx=0).stop 2024-06-02T23:31:46.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9778101b10 msgr2=0x7f9778196ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9778101b10 0x7f9778196ff0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f976802a930 tx=0x7f9768003c10 comp rx=0 tx=0).stop 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 shutdown_connections 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97640a2cb0 0x7f97640a5170 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97781024d0 0x7f977819e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9778101b10 0x7f9778196ff0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 --2- 172.21.15.52:0/2486369550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9778100920 0x7f9778196ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 >> 172.21.15.52:0/2486369550 conn(0x7f97780fbe90 msgr2=0x7f9778104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 shutdown_connections 2024-06-02T23:31:46.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:46.586+0000 7f977f963640 1 -- 172.21.15.52:0/2486369550 wait complete. 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.088+0000 7fa16ae26700 1 -- 172.21.15.52:0/4045551908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164102ed0 msgr2=0x7fa16410f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.088+0000 7fa16ae26700 1 --2- 172.21.15.52:0/4045551908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164102ed0 0x7fa16410f360 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b0a0 tx=0x7fa15c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 -- 172.21.15.52:0/4045551908 shutdown_connections 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 --2- 172.21.15.52:0/4045551908 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164102ed0 0x7fa16410f360 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 --2- 172.21.15.52:0/4045551908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164108520 0x7fa164108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 --2- 172.21.15.52:0/4045551908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa164102510 0x7fa164102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 -- 172.21.15.52:0/4045551908 >> 172.21.15.52:0/4045551908 conn(0x7fa164079560 msgr2=0x7fa164079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 -- 172.21.15.52:0/4045551908 shutdown_connections 2024-06-02T23:31:47.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.089+0000 7fa16ae26700 1 -- 172.21.15.52:0/4045551908 wait complete. 2024-06-02T23:31:47.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 Processor -- start 2024-06-02T23:31:47.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 -- start start 2024-06-02T23:31:47.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102510 0x7fa164073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa164102ed0 0x7fa164074100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 0x7fa16406de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa16406c7b0 con 0x7fa164108520 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa16406c630 con 0x7fa164102510 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.090+0000 7fa16ae26700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa16406c930 con 0x7fa164102ed0 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 0x7fa16406de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102510 0x7fa164073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 0x7fa16406de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36708/0 (socket says 172.21.15.52:36708) 2024-06-02T23:31:47.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 -- 172.21.15.52:0/3938071749 learned_addr learned my addr 172.21.15.52:0/3938071749 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa163fff700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa164102ed0 0x7fa164074100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 -- 172.21.15.52:0/3938071749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa164102ed0 msgr2=0x7fa164074100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa164102ed0 0x7fa164074100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 -- 172.21.15.52:0/3938071749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102510 msgr2=0x7fa164073bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102510 0x7fa164073bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 -- 172.21.15.52:0/3938071749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa16406e720 con 0x7fa164108520 2024-06-02T23:31:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.091+0000 7fa1693c3700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 0x7fa16406de20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b070 tx=0x7fa15c009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.092+0000 7fa161ffb700 1 -- 172.21.15.52:0/3938071749 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c035020 con 0x7fa164108520 2024-06-02T23:31:47.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.092+0000 7fa161ffb700 1 -- 172.21.15.52:0/3938071749 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15c004e40 con 0x7fa164108520 2024-06-02T23:31:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.092+0000 7fa161ffb700 1 -- 172.21.15.52:0/3938071749 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0203d0 con 0x7fa164108520 2024-06-02T23:31:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.092+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa16406e9b0 con 0x7fa164108520 2024-06-02T23:31:47.095 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.092+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa164074810 con 0x7fa164108520 2024-06-02T23:31:47.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.097+0000 7fa161ffb700 1 -- 172.21.15.52:0/3938071749 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa15c020570 con 0x7fa164108520 2024-06-02T23:31:47.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.098+0000 7fa161ffb700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1540a2df0 0x7fa1540a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.099+0000 7fa168bc2700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1540a2df0 0x7fa1540a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.099+0000 7fa161ffb700 1 -- 172.21.15.52:0/3938071749 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa15c00f020 con 0x7fa164108520 2024-06-02T23:31:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.099+0000 7fa168bc2700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1540a2df0 0x7fa1540a52b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa158009680 tx=0x7fa158009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:47.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.100+0000 7fa1756a9b80 1 Processor -- start 2024-06-02T23:31:47.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.100+0000 7fa1756a9b80 1 -- start start 2024-06-02T23:31:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.100+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 0x55a735556340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.103 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735550260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a735550bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a735442f60 con 0x55a735555f60 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a7354a1850 con 0x55a7355507a0 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a735409460 con 0x55a735556880 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 0x55a735556340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 0x55a735556340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36718/0 (socket says 172.21.15.52:36718) 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 -- 172.21.15.52:0/2837360342 learned_addr learned my addr 172.21.15.52:0/2837360342 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:47.104 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa163fff700 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735550260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa1693c3700 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a735550bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 -- 172.21.15.52:0/2837360342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 msgr2=0x55a735550260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735550260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 -- 172.21.15.52:0/2837360342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 msgr2=0x55a735550bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a735550bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.101+0000 7fa168bc2700 1 -- 172.21.15.52:0/2837360342 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7353f08d0 con 0x55a735555f60 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.102+0000 7fa168bc2700 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 0x55a735556340 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa158011d60 tx=0x7fa158014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaada52a31f05c8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.102+0000 7fa163fff700 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735550260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:47.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.102+0000 7fa14effd700 1 -- 172.21.15.52:0/2837360342 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158011d90 con 0x55a735555f60 2024-06-02T23:31:47.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.102+0000 7fa14effd700 1 -- 172.21.15.52:0/2837360342 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa158004020 con 0x55a735555f60 2024-06-02T23:31:47.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.102+0000 7fa14effd700 1 -- 172.21.15.52:0/2837360342 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158003c30 con 0x55a735555f60 2024-06-02T23:31:47.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.103+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2837360342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 msgr2=0x55a735556340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.103+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 0x55a735556340 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fa158011d60 tx=0x7fa158014000 comp rx=0 tx=0).stop 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.105+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2837360342 shutdown_connections 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.105+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a735550bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.105+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a735555f60 0x55a735556340 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.105+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2837360342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735550260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.106+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2837360342 >> 172.21.15.52:0/2837360342 conn(0x55a73547d080 msgr2=0x55a73554b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.106+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2837360342 shutdown_connections 2024-06-02T23:31:47.107 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.106+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2837360342 wait complete. 2024-06-02T23:31:47.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.107+0000 7fa1756a9b80 1 -- start start 2024-06-02T23:31:47.108 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.107+0000 7fa1756a9b80 1 Processor -- start 2024-06-02T23:31:47.109 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.108+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 0x55a73555c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 0x55a73555c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 0x55a73555c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36724/0 (socket says 172.21.15.52:36724) 2024-06-02T23:31:47.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa168bc2700 1 -- 172.21.15.52:0/286697018 learned_addr learned my addr 172.21.15.52:0/286697018 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a735555f60 0x55a7355599e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735559f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a735441fc0 con 0x55a7355507a0 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a73549d810 con 0x55a735555f60 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a73555d260 con 0x55a735556880 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa163fff700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a735555f60 0x55a7355599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.109+0000 7fa1693c3700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735559f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa168bc2700 1 -- 172.21.15.52:0/286697018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 msgr2=0x55a735559f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa168bc2700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735559f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa168bc2700 1 -- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a735555f60 msgr2=0x55a7355599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa168bc2700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a735555f60 0x55a7355599e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa168bc2700 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a735467750 con 0x55a7355507a0 2024-06-02T23:31:47.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa1756a9b80 10 client.0 _ll_register_callbacks cb 0x55a7354375b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:47.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.110+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa158004020 con 0x55a7355507a0 2024-06-02T23:31:47.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.111+0000 7fa168bc2700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 0x55a73555c6e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa158009ab0 tx=0x7fa15800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a9a464122ad46 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:47.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.111+0000 7fa143fff700 10 client.20697 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:47.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.111+0000 7fa143fff700 1 -- 172.21.15.52:0/286697018 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa158003c80 con 0x55a7355507a0 2024-06-02T23:31:47.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.112+0000 7fa143fff700 1 -- 172.21.15.52:0/286697018 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa158003e20 con 0x55a7355507a0 2024-06-02T23:31:47.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.112+0000 7fa143fff700 1 -- 172.21.15.52:0/286697018 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa158032450 con 0x55a7355507a0 2024-06-02T23:31:47.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.112+0000 7fa1756a9b80 10 client.20697 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.112+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55a7355678f0 con 0x55a7355507a0 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.113+0000 7fa143fff700 1 -- 172.21.15.52:0/286697018 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa15803c420 con 0x55a7355507a0 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa1756a9b80 10 client.20697 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa1756a9b80 10 client.20697 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa1756a9b80 20 client.20697 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735569dc0 0x55a73556c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa1756a9b80 4 client.20697 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7354aed50 con 0x55a735569dc0 2024-06-02T23:31:47.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.114+0000 7fa163fff700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735569dc0 0x55a73556c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.116+0000 7fa163fff700 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735569dc0 0x55a73556c1b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3328192e4e7acab7 server_cookie=6299bd0a2c2e7046 in_seq=0 out_seq=0 2024-06-02T23:31:47.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.116+0000 7fa143fff700 10 client.20697 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:31:47.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.117+0000 7fa143fff700 1 -- 172.21.15.52:0/286697018 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a7354aed50 con 0x55a735569dc0 2024-06-02T23:31:47.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.117+0000 7fa143fff700 10 client.20697 handle_command_reply: tid=0 2024-06-02T23:31:47.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.117+0000 7fa1756a9b80 1 client.20697 shutdown 2024-06-02T23:31:47.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 msgr2=0x55a73555c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 0x55a73555c6e0 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7fa158009ab0 tx=0x7fa15800ee00 comp rx=0 tx=0).stop 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 shutdown_connections 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735556880 0x55a735559f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a7355507a0 0x55a73555c6e0 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735569dc0 0x55a73556c1b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/286697018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a735555f60 0x55a7355599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 >> 172.21.15.52:0/286697018 conn(0x55a73547d080 msgr2=0x55a73554a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:47.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.118+0000 7fa143fff700 0 client.20697 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:31:47.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.119+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 shutdown_connections 2024-06-02T23:31:47.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.119+0000 7fa1756a9b80 1 -- 172.21.15.52:0/286697018 wait complete. 2024-06-02T23:31:47.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.119+0000 7fa1756a9b80 20 client.20697 trim_cache size 0 max 16384 2024-06-02T23:31:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 Processor -- start 2024-06-02T23:31:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 -- start start 2024-06-02T23:31:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a7355589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73554ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73554b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641b2fb0 con 0x55a73547d080 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641b4af0 con 0x55a7355507a0 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.144+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641b3680 con 0x55a735555f60 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a7355589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73554ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a7355589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36728/0 (socket says 172.21.15.52:36728) 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa168bc2700 1 -- 172.21.15.52:0/2463238045 learned_addr learned my addr 172.21.15.52:0/2463238045 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa1693c3700 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73554b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 -- 172.21.15.52:0/2463238045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 msgr2=0x55a73554b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73554b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 -- 172.21.15.52:0/2463238045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 msgr2=0x55a7355589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a7355589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 -- 172.21.15.52:0/2463238045 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7355513e0 con 0x55a7355507a0 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa1693c3700 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73554b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa163fff700 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73554ae80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a73549d8a0 tx=0x7fa150009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f116a2e845ebda24 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa14e7fc700 1 -- 172.21.15.52:0/2463238045 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15001e6e0 con 0x55a7355507a0 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa14e7fc700 1 -- 172.21.15.52:0/2463238045 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15001ed30 con 0x55a7355507a0 2024-06-02T23:31:47.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.145+0000 7fa14e7fc700 1 -- 172.21.15.52:0/2463238045 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150026d90 con 0x55a7355507a0 2024-06-02T23:31:47.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2463238045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 msgr2=0x55a73554ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73554ae80 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a73549d8a0 tx=0x7fa150009260 comp rx=0 tx=0).stop 2024-06-02T23:31:47.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2463238045 shutdown_connections 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73554b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a7355589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/2463238045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73554ae80 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2463238045 >> 172.21.15.52:0/2463238045 conn(0x55a73555c840 msgr2=0x55a7355582d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2463238045 shutdown_connections 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 -- 172.21.15.52:0/2463238045 wait complete. 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 -- start start 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.147+0000 7fa1756a9b80 1 Processor -- start 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.148+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a73556eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.148+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73556f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.148+0000 7fa1756a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73556c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.148+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641b4420 con 0x55a73547d080 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.148+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641b3d50 con 0x55a7355507a0 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.148+0000 7fa1756a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1641b0f70 con 0x55a735555f60 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73556c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a73556eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa168bc2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a73556eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36744/0 (socket says 172.21.15.52:36744) 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa168bc2700 1 -- 172.21.15.52:0/743139951 learned_addr learned my addr 172.21.15.52:0/743139951 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa163fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73556f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1756a9b80 10 client.0 _ll_register_callbacks cb 0x55a7354375b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 -- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 msgr2=0x55a73556f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73556f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 -- 172.21.15.52:0/743139951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 msgr2=0x55a73556eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a73556eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 -- 172.21.15.52:0/743139951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73556cb50 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa168bc2700 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a73556eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa1693c3700 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73556c3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0a98b0 tx=0x7fa15c00b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668bfd2bc7062281 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa143fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.149+0000 7fa143fff700 1 -- 172.21.15.52:0/743139951 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa15c0ad450 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa143fff700 1 -- 172.21.15.52:0/743139951 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa15c0074e0 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15001ed30 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa143fff700 1 -- 172.21.15.52:0/743139951 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa15c0b1800 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 10 client.29638 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a7355d41c0 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa143fff700 1 -- 172.21.15.52:0/743139951 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa15001ed30 con 0x55a735555f60 2024-06-02T23:31:47.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa143fff700 1 -- 172.21.15.52:0/743139951 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa15c011070 con 0x55a735555f60 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 10 client.29638 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 10 client.29638 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 20 client.29638 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735556880 0x55a7355d68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 4 client.29638 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.150+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a7354a92d0 con 0x55a735556880 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.151+0000 7fa168bc2700 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735556880 0x55a7355d68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.151+0000 7fa143fff700 10 client.29638 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.151+0000 7fa168bc2700 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735556880 0x55a7355d68b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=784d5849da8b2709 server_cookie=3dc5f052a532b3a8 in_seq=0 out_seq=0 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.151+0000 7fa143fff700 1 -- 172.21.15.52:0/743139951 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a7354a92d0 con 0x55a735556880 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.151+0000 7fa143fff700 10 client.29638 handle_command_reply: tid=0 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.151+0000 7fa1756a9b80 1 client.29638 shutdown 2024-06-02T23:31:47.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 msgr2=0x55a73556c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73556c3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0a98b0 tx=0x7fa15c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 shutdown_connections 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a735555f60 0x55a73556c3b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a73547d080 0x55a73556eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55a735556880 0x55a7355d68b0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 --2- 172.21.15.52:0/743139951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a7355507a0 0x55a73556f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa143fff700 0 client.29638 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 >> 172.21.15.52:0/743139951 conn(0x55a73555c840 msgr2=0x55a73555b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 shutdown_connections 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 1 -- 172.21.15.52:0/743139951 wait complete. 2024-06-02T23:31:47.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.152+0000 7fa1756a9b80 20 client.29638 trim_cache size 0 max 16384 2024-06-02T23:31:47.159 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:31:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1540a2df0 msgr2=0x7fa1540a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1540a2df0 0x7fa1540a52b0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fa158009680 tx=0x7fa158009260 comp rx=0 tx=0).stop 2024-06-02T23:31:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 msgr2=0x7fa16406de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:47.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 0x7fa16406de20 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00b070 tx=0x7fa15c009c90 comp rx=0 tx=0).stop 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 shutdown_connections 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa1540a2df0 0x7fa1540a52b0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa164108520 0x7fa16406de20 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa164102510 0x7fa164073bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 --2- 172.21.15.52:0/3938071749 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa164102ed0 0x7fa164074100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 >> 172.21.15.52:0/3938071749 conn(0x7fa164079560 msgr2=0x7fa164101000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 shutdown_connections 2024-06-02T23:31:47.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:47.153+0000 7fa16ae26700 1 -- 172.21.15.52:0/3938071749 wait complete. 2024-06-02T23:31:47.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 50426 6.88 MB/sec execute 295 sec latency 96.883 ms 2024-06-02T23:31:48.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:47 smithi123 ceph-mon[35977]: pgmap v1028: 97 pgs: 97 active+clean; 481 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 314 op/s 2024-06-02T23:31:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:47 smithi174 ceph-mon[35994]: pgmap v1028: 97 pgs: 97 active+clean; 481 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 314 op/s 2024-06-02T23:31:48.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:47 smithi052 ceph-mon[32340]: pgmap v1028: 97 pgs: 97 active+clean; 481 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 314 op/s 2024-06-02T23:31:48.288 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:31:48.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 50608 6.90 MB/sec execute 296 sec latency 59.285 ms 2024-06-02T23:31:48.655 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:49.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 50769 6.90 MB/sec execute 297 sec latency 63.318 ms 2024-06-02T23:31:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:49 smithi123 ceph-mon[35977]: pgmap v1029: 97 pgs: 97 active+clean; 480 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-06-02T23:31:50.105 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:49 smithi052 ceph-mon[32340]: pgmap v1029: 97 pgs: 97 active+clean; 480 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-06-02T23:31:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:49 smithi174 ceph-mon[35994]: pgmap v1029: 97 pgs: 97 active+clean; 480 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 270 op/s 2024-06-02T23:31:50.467 INFO:tasks.workunit.client.0.smithi052.stdout: 10 50958 6.91 MB/sec execute 298 sec latency 58.698 ms 2024-06-02T23:31:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 -- 172.21.15.52:0/2198345192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750107400 msgr2=0x7f97501077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 --2- 172.21.15.52:0/2198345192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750107400 0x7f97501077e0 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f9744009990 tx=0x7f9744009ca0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 -- 172.21.15.52:0/2198345192 shutdown_connections 2024-06-02T23:31:50.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 --2- 172.21.15.52:0/2198345192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750101d50 0x7f975010e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 --2- 172.21.15.52:0/2198345192 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750107400 0x7f97501077e0 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 --2- 172.21.15.52:0/2198345192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101390 0x7f9750101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 -- 172.21.15.52:0/2198345192 >> 172.21.15.52:0/2198345192 conn(0x7f9750078340 msgr2=0x7f9750078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.766+0000 7f9756ba2700 1 -- 172.21.15.52:0/2198345192 shutdown_connections 2024-06-02T23:31:50.768 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.767+0000 7f9756ba2700 1 -- 172.21.15.52:0/2198345192 wait complete. 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.767+0000 7f9756ba2700 1 Processor -- start 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.767+0000 7f9756ba2700 1 -- start start 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f9756ba2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750101390 0x7f975019e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f9756ba2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 0x7f975019ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f9756ba2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 0x7f97501989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f9756ba2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9750111550 con 0x7f9750101390 2024-06-02T23:31:50.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f9756ba2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97501113d0 con 0x7f9750107400 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f9756ba2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97501116d0 con 0x7f9750101d50 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f975493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750101390 0x7f975019e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f974ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 0x7f975019ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f975513f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 0x7f97501989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f974ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 0x7f975019ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:57784/0 (socket says 172.21.15.52:57784) 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f975513f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 0x7f97501989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33920/0 (socket says 172.21.15.52:33920) 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f974ffff700 1 -- 172.21.15.52:0/1209957251 learned_addr learned my addr 172.21.15.52:0/1209957251 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f974ffff700 1 -- 172.21.15.52:0/1209957251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 msgr2=0x7f97501989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f974ffff700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 0x7f97501989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.768+0000 7f974ffff700 1 -- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750101390 msgr2=0x7f975019e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f974ffff700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750101390 0x7f975019e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f974ffff700 1 -- 172.21.15.52:0/1209957251 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9750199120 con 0x7f9750101d50 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f975513f700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 0x7f97501989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f975493e700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750101390 0x7f975019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f974ffff700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 0x7f975019ed40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f973c00e9c0 tx=0x7f973c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f974dffb700 1 -- 172.21.15.52:0/1209957251 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c00cbd0 con 0x7f9750101d50 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f974dffb700 1 -- 172.21.15.52:0/1209957251 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f973c00cd70 con 0x7f9750101d50 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9750199410 con 0x7f9750101d50 2024-06-02T23:31:50.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f974dffb700 1 -- 172.21.15.52:0/1209957251 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c005650 con 0x7f9750101d50 2024-06-02T23:31:50.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.769+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9750102fa0 con 0x7f9750101d50 2024-06-02T23:31:50.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.772+0000 7f974dffb700 1 -- 172.21.15.52:0/1209957251 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f973c004750 con 0x7f9750101d50 2024-06-02T23:31:50.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.773+0000 7f974dffb700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97400a2d20 0x7f97400a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.773+0000 7f975493e700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97400a2d20 0x7f97400a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.773+0000 7f974dffb700 1 -- 172.21.15.52:0/1209957251 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f973c0cf280 con 0x7f9750101d50 2024-06-02T23:31:50.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.774+0000 7f975493e700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97400a2d20 0x7f97400a51e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f974400b2e0 tx=0x7f9744005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:50.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.774+0000 7f9761425b80 1 Processor -- start 2024-06-02T23:31:50.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.774+0000 7f9761425b80 1 -- start start 2024-06-02T23:31:50.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.774+0000 7f9761425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa8f60 0x558316fa9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.774+0000 7f9761425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa9880 0x558316fa3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f975493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa8f60 0x558316fa9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f975493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa8f60 0x558316fa9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36770/0 (socket says 172.21.15.52:36770) 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f975493e700 1 -- 172.21.15.52:0/2522123670 learned_addr learned my addr 172.21.15.52:0/2522123670 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f9761425b80 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558316e95f60 con 0x558316fa8f60 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa9880 0x558316fa3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558316ef4850 con 0x558316fa9880 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558316e5c460 con 0x558316fa37a0 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f975513f700 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 -- 172.21.15.52:0/2522123670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 msgr2=0x558316fa3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 -- 172.21.15.52:0/2522123670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa8f60 msgr2=0x558316fa9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa8f60 0x558316fa9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 -- 172.21.15.52:0/2522123670 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558316e438d0 con 0x558316fa9880 2024-06-02T23:31:50.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f974ffff700 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa9880 0x558316fa3260 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f973c009fd0 tx=0x7f973c097e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b78c15bbbda566d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.775+0000 7f973affd700 1 -- 172.21.15.52:0/2522123670 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c0b3020 con 0x558316fa9880 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.776+0000 7f973affd700 1 -- 172.21.15.52:0/2522123670 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f973c09c1b0 con 0x558316fa9880 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.776+0000 7f973affd700 1 -- 172.21.15.52:0/2522123670 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c098120 con 0x558316fa9880 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.776+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa9880 msgr2=0x558316fa3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.776+0000 7f9761425b80 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa9880 0x558316fa3260 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f973c009fd0 tx=0x7f973c097e80 comp rx=0 tx=0).stop 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 shutdown_connections 2024-06-02T23:31:50.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa8f60 0x558316fa9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 --2- 172.21.15.52:0/2522123670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa9880 0x558316fa3260 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 >> 172.21.15.52:0/2522123670 conn(0x558316ed0080 msgr2=0x558316f9e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 shutdown_connections 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.777+0000 7f9761425b80 1 -- 172.21.15.52:0/2522123670 wait complete. 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.778+0000 7f9761425b80 1 -- start start 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.778+0000 7f9761425b80 1 Processor -- start 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.778+0000 7f9761425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 0x558316faf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f9761425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316fac960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f9761425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa9880 0x558316facea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f975493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 0x558316faf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f975493e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 0x558316faf660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36782/0 (socket says 172.21.15.52:36782) 2024-06-02T23:31:50.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f975493e700 1 -- 172.21.15.52:0/1663997073 learned_addr learned my addr 172.21.15.52:0/1663997073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:50.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f974ffff700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316fac960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f975513f700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa9880 0x558316facea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f9761425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558316e94fc0 con 0x558316fa37a0 2024-06-02T23:31:50.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558316ef0810 con 0x558316fa8f60 2024-06-02T23:31:50.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558316fb0260 con 0x558316fa9880 2024-06-02T23:31:50.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.779+0000 7f9761425b80 10 client.0 _ll_register_callbacks cb 0x558316e8a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f975493e700 1 -- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa9880 msgr2=0x558316facea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f975493e700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa9880 0x558316facea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f975493e700 1 -- 172.21.15.52:0/1663997073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 msgr2=0x558316fac960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f975493e700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316fac960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f975493e700 1 -- 172.21.15.52:0/1663997073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558316eba750 con 0x558316fa37a0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f975493e700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 0x558316faf660 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9744009600 tx=0x7f9744009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6707cde3c69ea030 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f972ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f972ffff700 1 -- 172.21.15.52:0/1663997073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744034060 con 0x558316fa37a0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.780+0000 7f972ffff700 1 -- 172.21.15.52:0/1663997073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9744004cb0 con 0x558316fa37a0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.781+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973c09c1b0 con 0x558316fa37a0 2024-06-02T23:31:50.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.781+0000 7f972ffff700 1 -- 172.21.15.52:0/1663997073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9744004eb0 con 0x558316fa37a0 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.781+0000 7f9761425b80 10 client.20712 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.782+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558316fbaa10 con 0x558316fa37a0 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.782+0000 7f972ffff700 1 -- 172.21.15.52:0/1663997073 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974402f910 con 0x558316fa37a0 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.782+0000 7f972ffff700 1 -- 172.21.15.52:0/1663997073 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f974400f6c0 con 0x558316fa37a0 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.782+0000 7f9761425b80 10 client.20712 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.782+0000 7f9761425b80 10 client.20712 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.782+0000 7f9761425b80 20 client.20712 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.783+0000 7f9761425b80 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558316fbcfa0 0x558316fbf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.783+0000 7f9761425b80 4 client.20712 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.783+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f973c09c1b0 con 0x558316fbcfa0 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.783+0000 7f974ffff700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558316fbcfa0 0x558316fbf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.783+0000 7f972ffff700 10 client.20712 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:31:50.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.783+0000 7f974ffff700 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558316fbcfa0 0x558316fbf390 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f72eb8265bc0be50 server_cookie=604a8a507a549623 in_seq=0 out_seq=0 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f972ffff700 1 -- 172.21.15.52:0/1663997073 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f973c09c1b0 con 0x558316fbcfa0 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f972ffff700 10 client.20712 handle_command_reply: tid=0 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f9761425b80 1 client.20712 shutdown 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 msgr2=0x558316faf660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f9761425b80 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 0x558316faf660 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f9744009600 tx=0x7f9744009ca0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 shutdown_connections 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.784+0000 7f9761425b80 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558316fbcfa0 0x558316fbf390 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa9880 0x558316facea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa37a0 0x558316faf660 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.787 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 1 --2- 172.21.15.52:0/1663997073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316fac960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 >> 172.21.15.52:0/1663997073 conn(0x558316ed0080 msgr2=0x558316f9d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:50.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f972ffff700 0 client.20712 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:31:50.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 shutdown_connections 2024-06-02T23:31:50.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 1 -- 172.21.15.52:0/1663997073 wait complete. 2024-06-02T23:31:50.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.785+0000 7f9761425b80 20 client.20712 trim_cache size 0 max 16384 2024-06-02T23:31:50.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.805+0000 7f9761425b80 1 Processor -- start 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.805+0000 7f9761425b80 1 -- start start 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f9761425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f9761425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316faa510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f9761425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316faf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f9761425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97501aecb0 con 0x558316fa9880 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f9761425b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97501b07f0 con 0x558316fa8f60 2024-06-02T23:31:50.807 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f9761425b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97501af380 con 0x558316fa37a0 2024-06-02T23:31:50.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f974ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316faa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:57802/0 (socket says 172.21.15.52:57802) 2024-06-02T23:31:50.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 -- 172.21.15.52:0/2499882888 learned_addr learned my addr 172.21.15.52:0/2499882888 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:50.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975513f700 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316faf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 -- 172.21.15.52:0/2499882888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 msgr2=0x558316faa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316faa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 -- 172.21.15.52:0/2499882888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 msgr2=0x558316faf1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316faf1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.806+0000 7f975493e700 1 -- 172.21.15.52:0/2499882888 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558316f9a810 con 0x558316fa37a0 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.807+0000 7f975493e700 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3260 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9744005d70 tx=0x7f9744005730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b561f12d0cbc8540 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.807+0000 7f973a7fc700 1 -- 172.21.15.52:0/2499882888 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744034070 con 0x558316fa37a0 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.807+0000 7f973a7fc700 1 -- 172.21.15.52:0/2499882888 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974400f660 con 0x558316fa37a0 2024-06-02T23:31:50.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.807+0000 7f973a7fc700 1 -- 172.21.15.52:0/2499882888 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974403e080 con 0x558316fa37a0 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 -- 172.21.15.52:0/2499882888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 msgr2=0x558316fa3260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3260 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9744005d70 tx=0x7f9744005730 comp rx=0 tx=0).stop 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 -- 172.21.15.52:0/2499882888 shutdown_connections 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316faf1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa37a0 0x558316fa3260 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 --2- 172.21.15.52:0/2499882888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa8f60 0x558316faa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 -- 172.21.15.52:0/2499882888 >> 172.21.15.52:0/2499882888 conn(0x558316ed0080 msgr2=0x558316fa7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 -- 172.21.15.52:0/2499882888 shutdown_connections 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 -- 172.21.15.52:0/2499882888 wait complete. 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.808+0000 7f9761425b80 1 -- start start 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 Processor -- start 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa37a0 0x558316fb54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa8f60 0x558316fb5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316fc3c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97501b0120 con 0x558316fa9880 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97501afa50 con 0x558316fa37a0 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97501acc70 con 0x558316fa8f60 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f975513f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316fc3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f975513f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316fc3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36796/0 (socket says 172.21.15.52:36796) 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f975513f700 1 -- 172.21.15.52:0/1606001896 learned_addr learned my addr 172.21.15.52:0/1606001896 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f974ffff700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa8f60 0x558316fb5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f975493e700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa37a0 0x558316fb54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.809+0000 7f9761425b80 10 client.0 _ll_register_callbacks cb 0x558316e8a5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975513f700 1 -- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa8f60 msgr2=0x558316fb5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975513f700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa8f60 0x558316fb5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975513f700 1 -- 172.21.15.52:0/1606001896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa37a0 msgr2=0x558316fb54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975513f700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa37a0 0x558316fb54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975513f700 1 -- 172.21.15.52:0/1606001896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558316fc5740 con 0x558316fa9880 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975493e700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa37a0 0x558316fb54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f975513f700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316fc3c70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f97480166b0 tx=0x7f97480169c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92702ecb3c620b14 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f972ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f972ffff700 1 -- 172.21.15.52:0/1606001896 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f974800ee50 con 0x558316fa9880 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f974400f660 con 0x558316fa9880 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.810+0000 7f972ffff700 1 -- 172.21.15.52:0/1606001896 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9748004bb0 con 0x558316fa9880 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f972ffff700 1 -- 172.21.15.52:0/1606001896 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9748004db0 con 0x558316fa9880 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 10 client.20718 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:50.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x558317027160 con 0x558316fa9880 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f972ffff700 1 -- 172.21.15.52:0/1606001896 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f974400f660 con 0x558316fa9880 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f972ffff700 1 -- 172.21.15.52:0/1606001896 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f974802b710 con 0x558316fa9880 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 10 client.20718 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 10 client.20718 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 20 client.20718 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558317027850 0x558317029c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 4 client.20718 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:31:50.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558316f01d50 con 0x558317027850 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.811+0000 7f975493e700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558317027850 0x558317029c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f972ffff700 10 client.20718 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f975493e700 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558317027850 0x558317029c70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=1a9f650d42c97b5d server_cookie=8dda4241b101087b in_seq=0 out_seq=0 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f972ffff700 1 -- 172.21.15.52:0/1606001896 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558316f01d50 con 0x558317027850 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f972ffff700 10 client.20718 handle_command_reply: tid=0 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f9761425b80 1 client.20718 shutdown 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 msgr2=0x558316fc3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.812+0000 7f9761425b80 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316fc3c70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f97480166b0 tx=0x7f97480169c0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 shutdown_connections 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558316fa9880 0x558316fc3c70 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558316fa37a0 0x558316fb54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x558317027850 0x558317029c70 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 --2- 172.21.15.52:0/1606001896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558316fa8f60 0x558316fb5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f972ffff700 0 client.20718 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 >> 172.21.15.52:0/1606001896 conn(0x558316ed0080 msgr2=0x558316fa82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 shutdown_connections 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 1 -- 172.21.15.52:0/1606001896 wait complete. 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.813+0000 7f9761425b80 20 client.20718 trim_cache size 0 max 16384 2024-06-02T23:31:50.818 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97400a2d20 msgr2=0x7f97400a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97400a2d20 0x7f97400a51e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f974400b2e0 tx=0x7f9744005fb0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 msgr2=0x7f975019ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 0x7f975019ed40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f973c00e9c0 tx=0x7f973c00ed80 comp rx=0 tx=0).stop 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 shutdown_connections 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97400a2d20 0x7f97400a51e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9750107400 0x7f97501989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9750101390 0x7f975019e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 --2- 172.21.15.52:0/1209957251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9750101d50 0x7f975019ed40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:50.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 >> 172.21.15.52:0/1209957251 conn(0x7f9750078340 msgr2=0x7f97501007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:50.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 shutdown_connections 2024-06-02T23:31:50.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:31:50.814+0000 7f9756ba2700 1 -- 172.21.15.52:0/1209957251 wait complete. 2024-06-02T23:31:51.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 51158 6.92 MB/sec execute 299 sec latency 63.427 ms 2024-06-02T23:31:51.648 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:31:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:51 smithi052 ceph-mon[32340]: pgmap v1030: 97 pgs: 97 active+clean; 477 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 306 op/s 2024-06-02T23:31:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:51 smithi123 ceph-mon[35977]: pgmap v1030: 97 pgs: 97 active+clean; 477 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 306 op/s 2024-06-02T23:31:52.019 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:31:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:51 smithi174 ceph-mon[35994]: pgmap v1030: 97 pgs: 97 active+clean; 477 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.3 MiB/s wr, 306 op/s 2024-06-02T23:31:52.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 51311 6.92 MB/sec execute 300 sec latency 147.890 ms 2024-06-02T23:31:53.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 51455 6.92 MB/sec execute 301 sec latency 150.172 ms 2024-06-02T23:31:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:53 smithi123 ceph-mon[35977]: pgmap v1031: 97 pgs: 97 active+clean; 477 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 289 op/s 2024-06-02T23:31:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:53 smithi174 ceph-mon[35994]: pgmap v1031: 97 pgs: 97 active+clean; 477 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 289 op/s 2024-06-02T23:31:54.148 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:53 smithi052 ceph-mon[32340]: pgmap v1031: 97 pgs: 97 active+clean; 477 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 289 op/s 2024-06-02T23:31:54.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 51629 6.94 MB/sec execute 302 sec latency 136.954 ms 2024-06-02T23:31:55.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 51791 6.96 MB/sec execute 303 sec latency 133.673 ms 2024-06-02T23:31:56.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:55 smithi123 ceph-mon[35977]: pgmap v1032: 97 pgs: 97 active+clean; 480 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 278 op/s 2024-06-02T23:31:56.014 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:55 smithi052 ceph-mon[32340]: pgmap v1032: 97 pgs: 97 active+clean; 480 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 278 op/s 2024-06-02T23:31:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:55 smithi174 ceph-mon[35994]: pgmap v1032: 97 pgs: 97 active+clean; 480 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.5 MiB/s wr, 278 op/s 2024-06-02T23:31:56.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 51947 6.97 MB/sec execute 304 sec latency 107.757 ms 2024-06-02T23:31:57.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52108 6.99 MB/sec execute 305 sec latency 136.642 ms 2024-06-02T23:31:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:57 smithi123 ceph-mon[35977]: pgmap v1033: 97 pgs: 97 active+clean; 487 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 323 op/s 2024-06-02T23:31:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:57 smithi174 ceph-mon[35994]: pgmap v1033: 97 pgs: 97 active+clean; 487 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 323 op/s 2024-06-02T23:31:58.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:57 smithi052 ceph-mon[32340]: pgmap v1033: 97 pgs: 97 active+clean; 487 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.4 MiB/s wr, 323 op/s 2024-06-02T23:31:58.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52265 6.99 MB/sec execute 306 sec latency 103.713 ms 2024-06-02T23:31:59.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52396 7.00 MB/sec execute 307 sec latency 133.961 ms 2024-06-02T23:31:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 -- 172.21.15.52:0/21234400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 msgr2=0x7fd6f8108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/21234400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 0x7fd6f8108d40 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009ca0 tx=0x7fd6f002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 -- 172.21.15.52:0/21234400 shutdown_connections 2024-06-02T23:31:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/21234400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 0x7fd6f8108d40 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/21234400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8101af0 0x7fd6f8101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/21234400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8100900 0x7fd6f8100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 -- 172.21.15.52:0/21234400 >> 172.21.15.52:0/21234400 conn(0x7fd6f80fbe70 msgr2=0x7fd6f80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 -- 172.21.15.52:0/21234400 shutdown_connections 2024-06-02T23:31:59.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.716+0000 7fd6fed5a640 1 -- 172.21.15.52:0/21234400 wait complete. 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.717+0000 7fd6fed5a640 1 Processor -- start 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.717+0000 7fd6fed5a640 1 -- start start 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fed5a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 0x7fd6f8199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fed5a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8101af0 0x7fd6f819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fed5a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 0x7fd6f81a12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fed5a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810c410 con 0x7fd6f81024b0 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fed5a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810c290 con 0x7fd6f8100900 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fed5a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f810c590 con 0x7fd6f8101af0 2024-06-02T23:31:59.719 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 0x7fd6f8199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 0x7fd6f8199da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59306/0 (socket says 172.21.15.52:59306) 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/1880318663 learned_addr learned my addr 172.21.15.52:0/1880318663 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6effff640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8101af0 0x7fd6f819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 0x7fd6f81a12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/1880318663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8101af0 msgr2=0x7fd6f819a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8101af0 0x7fd6f819a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 msgr2=0x7fd6f81a12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 0x7fd6f81a12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/1880318663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6f81a1960 con 0x7fd6f8100900 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6effff640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8101af0 0x7fd6f819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.718+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 0x7fd6f8199da0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e9c0 tx=0x7fd6f400ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:59.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.719+0000 7fd6edffb640 1 -- 172.21.15.52:0/1880318663 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f400cde0 con 0x7fd6f8100900 2024-06-02T23:31:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.719+0000 7fd6edffb640 1 -- 172.21.15.52:0/1880318663 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6f40108e0 con 0x7fd6f8100900 2024-06-02T23:31:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.719+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6f81a1c50 con 0x7fd6f8100900 2024-06-02T23:31:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.719+0000 7fd6edffb640 1 -- 172.21.15.52:0/1880318663 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4018990 con 0x7fd6f8100900 2024-06-02T23:31:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.719+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6f81a21a0 con 0x7fd6f8100900 2024-06-02T23:31:59.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.722+0000 7fd6edffb640 1 -- 172.21.15.52:0/1880318663 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd6f401f020 con 0x7fd6f8100900 2024-06-02T23:31:59.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 Processor -- start 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd6edffb640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6e00a2c30 0x7fd6e00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 -- start start 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6e00a2c30 0x7fd6e00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd6edffb640 1 -- 172.21.15.52:0/1880318663 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd6f40cf310 con 0x7fd6f8100900 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f393c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f398d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f399380 0x55d16f39b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d16f2ee4f0 con 0x55d16f394210 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d16f28c870 con 0x55d16f399380 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.723+0000 7fd701a68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d16f2ea4b0 con 0x55d16f393820 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f398d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6effff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f398d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56164/0 (socket says 172.21.15.52:56164) 2024-06-02T23:31:59.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6effff640 1 -- 172.21.15.52:0/3118238099 learned_addr learned my addr 172.21.15.52:0/3118238099 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f393c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f399380 0x55d16f39b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/3118238099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f399380 msgr2=0x55d16f39b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f399380 0x55d16f39b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/3118238099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 msgr2=0x55d16f398d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f398d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 -- 172.21.15.52:0/3118238099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d16f2a89b0 con 0x55d16f393820 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6e00a2c30 0x7fd6e00a50f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0004af0 tx=0x7fd6f0009910 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f393c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40058c0 tx=0x7fd6f4010e20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c2b7e5afe3b48b73 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6deffd640 1 -- 172.21.15.52:0/3118238099 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f409bba0 con 0x55d16f393820 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6deffd640 1 -- 172.21.15.52:0/3118238099 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6f409ff50 con 0x55d16f393820 2024-06-02T23:31:59.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.724+0000 7fd6deffd640 1 -- 172.21.15.52:0/3118238099 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40a4380 con 0x55d16f393820 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 -- 172.21.15.52:0/3118238099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 msgr2=0x55d16f393c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f393c40 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd6f40058c0 tx=0x7fd6f4010e20 comp rx=0 tx=0).stop 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 -- 172.21.15.52:0/3118238099 shutdown_connections 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f399380 0x55d16f39b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f398d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 --2- 172.21.15.52:0/3118238099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f393c40 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.725+0000 7fd701a68740 1 -- 172.21.15.52:0/3118238099 >> 172.21.15.52:0/3118238099 conn(0x55d16f386680 msgr2=0x55d16f386a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 -- 172.21.15.52:0/3118238099 shutdown_connections 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 -- 172.21.15.52:0/3118238099 wait complete. 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 -- start start 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 Processor -- start 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f393820 0x55d16f39db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f394210 0x55d16f39e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.726+0000 7fd701a68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 0x55d16f3a5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd701a68740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d16f39c330 con 0x55d16f393820 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd701a68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d16f277ae0 con 0x55d16f394210 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd701a68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d16f39c4b0 con 0x55d16f399380 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd701a68740 10 client.0 _ll_register_callbacks cb 0x55d16f2b09b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 0x55d16f3a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 0x55d16f3a5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:33434/0 (socket says 172.21.15.52:33434) 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2596107923 learned_addr learned my addr 172.21.15.52:0/2596107923 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6effff640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f394210 0x55d16f39e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f393820 0x55d16f39db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f394210 msgr2=0x55d16f39e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f394210 0x55d16f39e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2596107923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f393820 msgr2=0x55d16f39db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f393820 0x55d16f39db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2596107923 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d16f002210 con 0x55d16f399380 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6effff640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f394210 0x55d16f39e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.727+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f393820 0x55d16f39db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 0x55d16f3a5030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009ca0 tx=0x7fd6f002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510aade1e412430e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd6d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:31:59.731 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd6d7fff640 1 -- 172.21.15.52:0/2596107923 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0004450 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6f409ff50 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd6d7fff640 1 -- 172.21.15.52:0/2596107923 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd6f00045f0 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd6d7fff640 1 -- 172.21.15.52:0/2596107923 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd6f00047f0 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd701a68740 10 client.29668 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.728+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d16f0ce620 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd6d7fff640 1 -- 172.21.15.52:0/2596107923 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6f409ff50 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd6d7fff640 1 -- 172.21.15.52:0/2596107923 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd6f002f5c0 con 0x55d16f399380 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd701a68740 10 client.29668 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd701a68740 10 client.29668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd701a68740 20 client.29668 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd701a68740 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f3a9c70 0x55d16f3ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd701a68740 4 client.29668 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d16f301f10 con 0x55d16f3a9c70 2024-06-02T23:31:59.732 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.729+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f3a9c70 0x55d16f3ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd6d7fff640 10 client.29668 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:59.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f3a9c70 0x55d16f3ac0f0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c424d1ad89f3be56 server_cookie=d25741b073d9b6c4 in_seq=0 out_seq=0 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd6d7fff640 1 -- 172.21.15.52:0/2596107923 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d16f301f10 con 0x55d16f3a9c70 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd6d7fff640 10 client.29668 handle_command_reply: tid=0 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd701a68740 1 client.29668 shutdown 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 msgr2=0x55d16f3a5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.730+0000 7fd701a68740 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 0x55d16f3a5030 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0009ca0 tx=0x7fd6f002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 shutdown_connections 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f3a9c70 0x55d16f3ac0f0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f399380 0x55d16f3a5030 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f394210 0x55d16f39e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd6d7fff640 0 client.29668 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 --2- 172.21.15.52:0/2596107923 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f393820 0x55d16f39db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 >> 172.21.15.52:0/2596107923 conn(0x55d16f386680 msgr2=0x55d16f3857e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 shutdown_connections 2024-06-02T23:31:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 1 -- 172.21.15.52:0/2596107923 wait complete. 2024-06-02T23:31:59.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.731+0000 7fd701a68740 20 client.29668 trim_cache size 0 max 16384 2024-06-02T23:31:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 Processor -- start 2024-06-02T23:31:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 -- start start 2024-06-02T23:31:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.755 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f389af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81a6bb0 con 0x55d16f394210 2024-06-02T23:31:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81a86f0 con 0x55d16f386680 2024-06-02T23:31:59.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd701a68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81a7280 con 0x55d16f393820 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd6fd2f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f389af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd6fcaf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd6fd2f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f389af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56180/0 (socket says 172.21.15.52:56180) 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2743224989 learned_addr learned my addr 172.21.15.52:0/2743224989 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.756+0000 7fd6effff640 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2743224989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 msgr2=0x55d16f3a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2743224989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 msgr2=0x55d16f3895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/2743224989 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d16f3864b0 con 0x55d16f394210 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f389af0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0031aa0 tx=0x7fd6f0009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d82af2b276bae10 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6de7fc640 1 -- 172.21.15.52:0/2743224989 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f002fea0 con 0x55d16f394210 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6de7fc640 1 -- 172.21.15.52:0/2743224989 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6f004e070 con 0x55d16f394210 2024-06-02T23:31:59.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.757+0000 7fd6de7fc640 1 -- 172.21.15.52:0/2743224989 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f0030820 con 0x55d16f394210 2024-06-02T23:31:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 -- 172.21.15.52:0/2743224989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 msgr2=0x55d16f389af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f389af0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0031aa0 tx=0x7fd6f0009540 comp rx=0 tx=0).stop 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 -- 172.21.15.52:0/2743224989 shutdown_connections 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f389af0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 --2- 172.21.15.52:0/2743224989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 -- 172.21.15.52:0/2743224989 >> 172.21.15.52:0/2743224989 conn(0x55d16f398e50 msgr2=0x55d16f395490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 -- 172.21.15.52:0/2743224989 shutdown_connections 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 -- 172.21.15.52:0/2743224989 wait complete. 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.758+0000 7fd701a68740 1 -- start start 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 Processor -- start 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3a1c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f4129c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81a8020 con 0x55d16f394210 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81a7950 con 0x55d16f386680 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd6f81a4bb0 con 0x55d16f393820 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd701a68740 10 client.0 _ll_register_callbacks cb 0x7fd6e8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd6fd2f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f4129c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd6effff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd6fcaf6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3a1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd6fd2f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f4129c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56188/0 (socket says 172.21.15.52:56188) 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.759+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/334150260 learned_addr learned my addr 172.21.15.52:0/334150260 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/334150260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 msgr2=0x55d16f3a21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 msgr2=0x55d16f3a1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3a1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fd2f7640 1 -- 172.21.15.52:0/334150260 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d16f3a5270 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6fd2f7640 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f4129c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00047c0 tx=0x7fd6f0004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d556e27b2470cd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6d7fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6d7fff640 1 -- 172.21.15.52:0/334150260 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6f002f4c0 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6d7fff640 1 -- 172.21.15.52:0/334150260 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd6f002f660 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6f004e070 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd6d7fff640 1 -- 172.21.15.52:0/334150260 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd6f006e050 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd701a68740 10 client.20736 fetch_fsmap learned FSMap version 43 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.760+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d16f413f50 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd6d7fff640 1 -- 172.21.15.52:0/334150260 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd6f0036040 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd6d7fff640 1 -- 172.21.15.52:0/334150260 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd6f005d070 con 0x55d16f394210 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd701a68740 10 client.20736 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd701a68740 10 client.20736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:31:59.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd701a68740 20 client.20736 populate_metadata read hostname 'smithi052' 2024-06-02T23:31:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd701a68740 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f399380 0x55d16f4168a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:31:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd701a68740 4 client.20736 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:31:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fd6f004e070 con 0x55d16f399380 2024-06-02T23:31:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f399380 0x55d16f4168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:31:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd6d7fff640 10 client.20736 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:59.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.761+0000 7fd6fcaf6640 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f399380 0x55d16f4168a0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d6e366947b4cb5f server_cookie=d3ca6ae5cd3a0238 in_seq=0 out_seq=0 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.833+0000 7fd6d7fff640 1 -- 172.21.15.52:0/334150260 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fd6f004e070 con 0x55d16f399380 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.833+0000 7fd6d7fff640 10 client.20736 handle_command_reply: tid=0 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.833+0000 7fd701a68740 1 client.20736 shutdown 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 msgr2=0x55d16f4129c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f4129c0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd6f00047c0 tx=0x7fd6f0004500 comp rx=0 tx=0).stop 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 shutdown_connections 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d16f399380 0x55d16f4168a0 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.834 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d16f393820 0x55d16f3a21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d16f394210 0x55d16f4129c0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd6d7fff640 0 client.20736 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 --2- 172.21.15.52:0/334150260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d16f386680 0x55d16f3a1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 >> 172.21.15.52:0/334150260 conn(0x55d16f398e50 msgr2=0x55d16f39dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 shutdown_connections 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 1 -- 172.21.15.52:0/334150260 wait complete. 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.834+0000 7fd701a68740 20 client.20736 trim_cache size 0 max 16384 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:31:59.835 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:31:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6e00a2c30 msgr2=0x7fd6e00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6e00a2c30 0x7fd6e00a50f0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fd6f0004af0 tx=0x7fd6f0009910 comp rx=0 tx=0).stop 2024-06-02T23:31:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 msgr2=0x7fd6f8199da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:31:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 0x7fd6f8199da0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd6f400e9c0 tx=0x7fd6f400ee80 comp rx=0 tx=0).stop 2024-06-02T23:31:59.836 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 shutdown_connections 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd6e00a2c30 0x7fd6e00a50f0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd6f8101af0 0x7fd6f819a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd6f81024b0 0x7fd6f81a12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 --2- 172.21.15.52:0/1880318663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd6f8100900 0x7fd6f8199da0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 >> 172.21.15.52:0/1880318663 conn(0x7fd6f80fbe70 msgr2=0x7fd6f8069050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 shutdown_connections 2024-06-02T23:31:59.837 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:31:59.835+0000 7fd6fed5a640 1 -- 172.21.15.52:0/1880318663 wait complete. 2024-06-02T23:32:00.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:31:59 smithi123 ceph-mon[35977]: pgmap v1034: 97 pgs: 97 active+clean; 489 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 255 op/s 2024-06-02T23:32:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:31:59 smithi174 ceph-mon[35994]: pgmap v1034: 97 pgs: 97 active+clean; 489 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 255 op/s 2024-06-02T23:32:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:31:59 smithi052 ceph-mon[32340]: pgmap v1034: 97 pgs: 97 active+clean; 489 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 255 op/s 2024-06-02T23:32:00.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52551 7.00 MB/sec execute 308 sec latency 93.061 ms 2024-06-02T23:32:01.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52689 7.01 MB/sec execute 309 sec latency 140.869 ms 2024-06-02T23:32:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:01 smithi123 ceph-mon[35977]: pgmap v1035: 97 pgs: 97 active+clean; 496 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 273 op/s 2024-06-02T23:32:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:01 smithi174 ceph-mon[35994]: pgmap v1035: 97 pgs: 97 active+clean; 496 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 273 op/s 2024-06-02T23:32:02.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:01 smithi052 ceph-mon[32340]: pgmap v1035: 97 pgs: 97 active+clean; 496 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 273 op/s 2024-06-02T23:32:02.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52810 7.00 MB/sec execute 310 sec latency 136.081 ms 2024-06-02T23:32:03.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 52917 7.00 MB/sec execute 311 sec latency 150.507 ms 2024-06-02T23:32:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:03 smithi123 ceph-mon[35977]: pgmap v1036: 97 pgs: 97 active+clean; 500 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-06-02T23:32:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:03 smithi174 ceph-mon[35994]: pgmap v1036: 97 pgs: 97 active+clean; 500 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-06-02T23:32:04.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:03 smithi052 ceph-mon[32340]: pgmap v1036: 97 pgs: 97 active+clean; 500 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 988 KiB/s rd, 4.3 MiB/s wr, 247 op/s 2024-06-02T23:32:04.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53044 7.00 MB/sec execute 312 sec latency 155.947 ms 2024-06-02T23:32:05.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53177 7.00 MB/sec execute 313 sec latency 105.764 ms 2024-06-02T23:32:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:05 smithi123 ceph-mon[35977]: pgmap v1037: 97 pgs: 97 active+clean; 499 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-06-02T23:32:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:05 smithi174 ceph-mon[35994]: pgmap v1037: 97 pgs: 97 active+clean; 499 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-06-02T23:32:06.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:05 smithi052 ceph-mon[32340]: pgmap v1037: 97 pgs: 97 active+clean; 499 MiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 833 KiB/s rd, 3.9 MiB/s wr, 217 op/s 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 -- 172.21.15.52:0/3774534832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0781027a0 msgr2=0x7ff078102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 --2- 172.21.15.52:0/3774534832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0781027a0 0x7ff078102b80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff068009990 tx=0x7ff068009ca0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 -- 172.21.15.52:0/3774534832 shutdown_connections 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 --2- 172.21.15.52:0/3774534832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff078107870 0x7ff07810f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 --2- 172.21.15.52:0/3774534832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0781027a0 0x7ff078102b80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 --2- 172.21.15.52:0/3774534832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff078103150 0x7ff078107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 -- 172.21.15.52:0/3774534832 >> 172.21.15.52:0/3774534832 conn(0x7ff078079520 msgr2=0x7ff078079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 -- 172.21.15.52:0/3774534832 shutdown_connections 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.496+0000 7ff07df62700 1 -- 172.21.15.52:0/3774534832 wait complete. 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.497+0000 7ff07df62700 1 Processor -- start 2024-06-02T23:32:06.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.497+0000 7ff07df62700 1 -- start start 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff07df62700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 0x7ff078074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff07df62700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff078103150 0x7ff078074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff07df62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff078107870 0x7ff078075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff07df62700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff078112690 con 0x7ff0781027a0 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff07df62700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff078112510 con 0x7ff078107870 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff07df62700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff078112810 con 0x7ff078103150 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 0x7ff078074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff076ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff078103150 0x7ff078074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 0x7ff078074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36474/0 (socket says 172.21.15.52:36474) 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 -- 172.21.15.52:0/1294760095 learned_addr learned my addr 172.21.15.52:0/1294760095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff077fff700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff078107870 0x7ff078075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 -- 172.21.15.52:0/1294760095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff078103150 msgr2=0x7ff078074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff078103150 0x7ff078074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 -- 172.21.15.52:0/1294760095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff078107870 msgr2=0x7ff078075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff078107870 0x7ff078075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 -- 172.21.15.52:0/1294760095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0781a7050 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff076ffd700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff078103150 0x7ff078074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.498+0000 7ff0777fe700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 0x7ff078074790 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff0680047b0 tx=0x7ff068009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.499+0000 7ff074ff9700 1 -- 172.21.15.52:0/1294760095 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068027070 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.499+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0781a72e0 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.499+0000 7ff074ff9700 1 -- 172.21.15.52:0/1294760095 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff06800be80 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.499+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0781a7730 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.499+0000 7ff074ff9700 1 -- 172.21.15.52:0/1294760095 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff068031060 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.503+0000 7ff074ff9700 1 -- 172.21.15.52:0/1294760095 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff06802a070 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.504+0000 7ff074ff9700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0640a2c20 0x7ff0640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.504+0000 7ff074ff9700 1 -- 172.21.15.52:0/1294760095 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff0680cfd30 con 0x7ff0781027a0 2024-06-02T23:32:06.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.504+0000 7ff076ffd700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0640a2c20 0x7ff0640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff076ffd700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0640a2c20 0x7ff0640a50e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff0781020e0 tx=0x7ff060009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 Processor -- start 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 -- start start 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb24da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb247f20 0x5647fb248340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5647fb199810 con 0x5647fb247f20 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5647fb141720 con 0x5647fb24d640 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647fb1957d0 con 0x5647fb24df60 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0777fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb24da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff076ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0777fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb24da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:54244/0 (socket says 172.21.15.52:54244) 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.505+0000 7ff0777fe700 1 -- 172.21.15.52:0/696405628 learned_addr learned my addr 172.21.15.52:0/696405628 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff077fff700 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb247f20 0x5647fb248340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff0777fe700 1 -- 172.21.15.52:0/696405628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 msgr2=0x5647fb2479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff0777fe700 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff0777fe700 1 -- 172.21.15.52:0/696405628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb247f20 msgr2=0x5647fb248340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff0777fe700 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb247f20 0x5647fb248340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff0777fe700 1 -- 172.21.15.52:0/696405628 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fb137f50 con 0x5647fb24d640 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff076ffd700 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff077fff700 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb247f20 0x5647fb248340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff0777fe700 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb24da20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff06809c580 tx=0x7ff06809c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a241c5db8f453793 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff06dffb700 1 -- 172.21.15.52:0/696405628 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06809bf50 con 0x5647fb24d640 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff06dffb700 1 -- 172.21.15.52:0/696405628 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff068022470 con 0x5647fb24d640 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.506+0000 7ff06dffb700 1 -- 172.21.15.52:0/696405628 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06809c850 con 0x5647fb24d640 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 -- 172.21.15.52:0/696405628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 msgr2=0x5647fb24da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb24da20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7ff06809c580 tx=0x7ff06809c3c0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 -- 172.21.15.52:0/696405628 shutdown_connections 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb247f20 0x5647fb248340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb24da20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/696405628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 -- 172.21.15.52:0/696405628 >> 172.21.15.52:0/696405628 conn(0x5647fb172cc0 msgr2=0x5647fb242850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 -- 172.21.15.52:0/696405628 shutdown_connections 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.507+0000 7ff0887e5b80 1 -- 172.21.15.52:0/696405628 wait complete. 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 -- start start 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 Processor -- start 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb247f20 0x5647fb253d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb251070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2515b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5647fb2546f0 con 0x5647fb24d640 2024-06-02T23:32:06.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5647fb139bf0 con 0x5647fb247f20 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.508+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5647fb254870 con 0x5647fb24df60 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff0887e5b80 10 client.0 _ll_register_callbacks cb 0x5647fb133920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb251070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb251070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36480/0 (socket says 172.21.15.52:36480) 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 -- 172.21.15.52:0/312132120 learned_addr learned my addr 172.21.15.52:0/312132120 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff077fff700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 -- 172.21.15.52:0/312132120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 msgr2=0x5647fb2515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 -- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb247f20 msgr2=0x5647fb253d70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb247f20 0x5647fb253d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 -- 172.21.15.52:0/312132120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fb0e88d0 con 0x5647fb24d640 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff077fff700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff076ffd700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb251070 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff0600038a0 tx=0x7ff0600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9bafce7bfa3962 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:06.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.509+0000 7ff056ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff056ffd700 1 -- 172.21.15.52:0/312132120 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06001f9d0 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff056ffd700 1 -- 172.21.15.52:0/312132120 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff06001fb70 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff068022470 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff056ffd700 1 -- 172.21.15.52:0/312132120 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff06001fd70 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff0887e5b80 10 client.20748 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5647fb25f100 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.510+0000 7ff056ffd700 1 -- 172.21.15.52:0/312132120 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff068022470 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff056ffd700 1 -- 172.21.15.52:0/312132120 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff060026070 con 0x5647fb24d640 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0887e5b80 10 client.20748 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0887e5b80 10 client.20748 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0887e5b80 20 client.20748 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb263b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0887e5b80 4 client.20748 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:06.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647fb1a6d10 con 0x5647fb261790 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.511+0000 7ff0777fe700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb263b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.512+0000 7ff0777fe700 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb263b80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8118dfab0bd458e server_cookie=bb6e5f3b28a5c726 in_seq=0 out_seq=0 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.512+0000 7ff056ffd700 10 client.20748 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.512+0000 7ff056ffd700 1 -- 172.21.15.52:0/312132120 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5647fb1a6d10 con 0x5647fb261790 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.512+0000 7ff056ffd700 10 client.20748 handle_command_reply: tid=0 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.512+0000 7ff0887e5b80 1 client.20748 shutdown 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 msgr2=0x5647fb251070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb251070 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7ff0600038a0 tx=0x7ff0600047c0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 shutdown_connections 2024-06-02T23:32:06.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb263b80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb24df60 0x5647fb2515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb247f20 0x5647fb253d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff056ffd700 0 client.20748 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/312132120 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb251070 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 >> 172.21.15.52:0/312132120 conn(0x5647fb172cc0 msgr2=0x5647fb2469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 shutdown_connections 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 1 -- 172.21.15.52:0/312132120 wait complete. 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.513+0000 7ff0887e5b80 20 client.20748 trim_cache size 0 max 16384 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 Processor -- start 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 -- start start 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb2479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 0x5647fb24ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781aebe0 con 0x5647fb24df60 2024-06-02T23:32:06.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b0720 con 0x5647fb24d640 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781af2b0 con 0x5647fb247f20 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 0x5647fb24ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 0x5647fb24ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36492/0 (socket says 172.21.15.52:36492) 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 -- 172.21.15.52:0/3363790139 learned_addr learned my addr 172.21.15.52:0/3363790139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff0777fe700 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff076ffd700 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb2479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 -- 172.21.15.52:0/3363790139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 msgr2=0x5647fb25d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 -- 172.21.15.52:0/3363790139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 msgr2=0x5647fb2479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.531+0000 7ff077fff700 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb2479e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.532+0000 7ff077fff700 1 -- 172.21.15.52:0/3363790139 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fb2442c0 con 0x5647fb24df60 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.532+0000 7ff077fff700 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 0x5647fb24ea50 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff070009ec0 tx=0x7ff070009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22dbe38e994756 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.532+0000 7ff06d7fa700 1 -- 172.21.15.52:0/3363790139 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070020670 con 0x5647fb24df60 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.532+0000 7ff06d7fa700 1 -- 172.21.15.52:0/3363790139 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff070020cc0 con 0x5647fb24df60 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.532+0000 7ff06d7fa700 1 -- 172.21.15.52:0/3363790139 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff070017dc0 con 0x5647fb24df60 2024-06-02T23:32:06.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3363790139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 msgr2=0x5647fb24ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 0x5647fb24ea50 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7ff070009ec0 tx=0x7ff070009920 comp rx=0 tx=0).stop 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3363790139 shutdown_connections 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24df60 0x5647fb24ea50 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3363790139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24d640 0x5647fb2479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3363790139 >> 172.21.15.52:0/3363790139 conn(0x5647fb172cc0 msgr2=0x5647fb24e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3363790139 shutdown_connections 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3363790139 wait complete. 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- start start 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 Processor -- start 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb268f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24df60 0x5647fb269690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781b0050 con 0x5647fb24d640 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781af980 con 0x5647fb24df60 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.533+0000 7ff0887e5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0781acba0 con 0x5647fb247f20 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff0887e5b80 10 client.0 _ll_register_callbacks cb 0x5647fb133920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb268f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb268f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36500/0 (socket says 172.21.15.52:36500) 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 -- 172.21.15.52:0/3668560795 learned_addr learned my addr 172.21.15.52:0/3668560795 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff0777fe700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff077fff700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24df60 0x5647fb269690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 -- 172.21.15.52:0/3668560795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 msgr2=0x5647fb25a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 -- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24df60 msgr2=0x5647fb269690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24df60 0x5647fb269690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 -- 172.21.15.52:0/3668560795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fb2cb420 con 0x5647fb24d640 2024-06-02T23:32:06.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff076ffd700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb268f50 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff060027540 tx=0x7ff060027e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7a4cecd4b796768 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff056ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff056ffd700 1 -- 172.21.15.52:0/3668560795 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff06001f7d0 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff056ffd700 1 -- 172.21.15.52:0/3668560795 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff06001f970 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.534+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff070020cc0 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff056ffd700 1 -- 172.21.15.52:0/3668560795 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff06001fb70 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 10 client.20754 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5647fb2cc630 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff056ffd700 1 -- 172.21.15.52:0/3668560795 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff070020cc0 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff056ffd700 1 -- 172.21.15.52:0/3668560795 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff060026050 con 0x5647fb24d640 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 10 client.20754 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 10 client.20754 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 20 client.20754 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb2cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:06.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 4 client.20754 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:32:06.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5647fb1a1290 con 0x5647fb261790 2024-06-02T23:32:06.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.535+0000 7ff0777fe700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb2cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:06.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0777fe700 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb2cedb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b79032d7d9cea23 server_cookie=bc91a3250897e64f in_seq=0 out_seq=0 2024-06-02T23:32:06.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff056ffd700 10 client.20754 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:32:06.206 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:32:06.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff056ffd700 1 -- 172.21.15.52:0/3668560795 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5647fb1a1290 con 0x5647fb261790 2024-06-02T23:32:06.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff056ffd700 10 client.20754 handle_command_reply: tid=0 2024-06-02T23:32:06.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 client.20754 shutdown 2024-06-02T23:32:06.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 msgr2=0x5647fb268f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb268f50 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7ff060027540 tx=0x7ff060027e30 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 shutdown_connections 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5647fb261790 0x5647fb2cedb0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5647fb24df60 0x5647fb269690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff056ffd700 0 client.20754 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5647fb247f20 0x5647fb25a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 --2- 172.21.15.52:0/3668560795 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5647fb24d640 0x5647fb268f50 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 >> 172.21.15.52:0/3668560795 conn(0x5647fb172cc0 msgr2=0x5647fb24e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 shutdown_connections 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 1 -- 172.21.15.52:0/3668560795 wait complete. 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.536+0000 7ff0887e5b80 20 client.20754 trim_cache size 0 max 16384 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0640a2c20 msgr2=0x7ff0640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0640a2c20 0x7ff0640a50e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7ff0781020e0 tx=0x7ff060009260 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 msgr2=0x7ff078074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 0x7ff078074790 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7ff0680047b0 tx=0x7ff068009550 comp rx=0 tx=0).stop 2024-06-02T23:32:06.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 shutdown_connections 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0640a2c20 0x7ff0640a50e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff078107870 0x7ff078075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0781027a0 0x7ff078074790 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 --2- 172.21.15.52:0/1294760095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff078103150 0x7ff078074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 >> 172.21.15.52:0/1294760095 conn(0x7ff078079520 msgr2=0x7ff078100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 shutdown_connections 2024-06-02T23:32:06.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:05.537+0000 7ff07df62700 1 -- 172.21.15.52:0/1294760095 wait complete. 2024-06-02T23:32:06.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53329 6.99 MB/sec execute 314 sec latency 122.386 ms 2024-06-02T23:32:07.389 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:32:07.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53501 6.99 MB/sec execute 315 sec latency 93.596 ms 2024-06-02T23:32:07.793 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:07 smithi123 ceph-mon[35977]: pgmap v1038: 97 pgs: 97 active+clean; 499 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-06-02T23:32:08.057 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:07 smithi052 ceph-mon[32340]: pgmap v1038: 97 pgs: 97 active+clean; 499 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-06-02T23:32:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:07 smithi174 ceph-mon[35994]: pgmap v1038: 97 pgs: 97 active+clean; 499 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.8 MiB/s wr, 281 op/s 2024-06-02T23:32:08.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53660 6.98 MB/sec execute 316 sec latency 65.670 ms 2024-06-02T23:32:09.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53831 7.00 MB/sec execute 317 sec latency 74.867 ms 2024-06-02T23:32:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 -- 172.21.15.52:0/3430880387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d81019d0 msgr2=0x7fe4d8101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/3430880387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d81019d0 0x7fe4d8101db0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40098f0 tx=0x7fe4d4009c00 comp rx=0 tx=0).stop 2024-06-02T23:32:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 -- 172.21.15.52:0/3430880387 shutdown_connections 2024-06-02T23:32:09.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/3430880387 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8106720 0x7fe4d810e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/3430880387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d81019d0 0x7fe4d8101db0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/3430880387 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d8102380 0x7fe4d8105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.726+0000 7fe4df7b4700 1 -- 172.21.15.52:0/3430880387 >> 172.21.15.52:0/3430880387 conn(0x7fe4d8078340 msgr2=0x7fe4d8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:09.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.727+0000 7fe4df7b4700 1 -- 172.21.15.52:0/3430880387 shutdown_connections 2024-06-02T23:32:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.727+0000 7fe4df7b4700 1 -- 172.21.15.52:0/3430880387 wait complete. 2024-06-02T23:32:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 Processor -- start 2024-06-02T23:32:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 -- start start 2024-06-02T23:32:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d81019d0 0x7fe4d8111650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 0x7fe4d8111b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.728 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d8106720 0x7fe4d81120d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810fd80 con 0x7fe4d81019d0 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810fc00 con 0x7fe4d8106720 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4df7b4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d810ff00 con 0x7fe4d8102380 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4dcd4f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 0x7fe4d8111b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4dd550700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d81019d0 0x7fe4d8111650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4dcd4f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 0x7fe4d8111b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52748/0 (socket says 172.21.15.52:52748) 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.728+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/4171525713 learned_addr learned my addr 172.21.15.52:0/4171525713 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4ddd51700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d8106720 0x7fe4d81120d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/4171525713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d8106720 msgr2=0x7fe4d81120d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d8106720 0x7fe4d81120d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d81019d0 msgr2=0x7fe4d8111650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d81019d0 0x7fe4d8111650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/4171525713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d81157a0 con 0x7fe4d8102380 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dd550700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d81019d0 0x7fe4d8111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4ddd51700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d8106720 0x7fe4d81120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 0x7fe4d8111b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800ebb0 tx=0x7fe4c800eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4ce7fc700 1 -- 172.21.15.52:0/4171525713 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c800ccf0 con 0x7fe4d8102380 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4ce7fc700 1 -- 172.21.15.52:0/4171525713 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c80044d0 con 0x7fe4d8102380 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.729+0000 7fe4ce7fc700 1 -- 172.21.15.52:0/4171525713 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8010430 con 0x7fe4d8102380 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.730+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d81b3fe0 con 0x7fe4d8102380 2024-06-02T23:32:09.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.730+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d81b4440 con 0x7fe4d8102380 2024-06-02T23:32:09.732 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.733+0000 7fe4ce7fc700 1 -- 172.21.15.52:0/4171525713 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fe4c8004020 con 0x7fe4d8102380 2024-06-02T23:32:09.733 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.734+0000 7fe4ce7fc700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4c40a2df0 0x7fe4c40a52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.734+0000 7fe4dd550700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4c40a2df0 0x7fe4c40a52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.734+0000 7fe4ce7fc700 1 -- 172.21.15.52:0/4171525713 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe4c80cf340 con 0x7fe4d8102380 2024-06-02T23:32:09.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.735+0000 7fe4dd550700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4c40a2df0 0x7fe4c40a52b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40098c0 tx=0x7fe4d4019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:09.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 Processor -- start 2024-06-02T23:32:09.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 -- start start 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbcf60 0x56090afbd340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 0x56090afb7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56090aea9f60 con 0x56090afbd880 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56090af08850 con 0x56090afbcf60 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56090ae70460 con 0x56090afb77a0 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4dcd4f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 0x56090afb7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4dcd4f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 0x56090afb7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36526/0 (socket says 172.21.15.52:36526) 2024-06-02T23:32:09.736 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/3013667389 learned_addr learned my addr 172.21.15.52:0/3013667389 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:09.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.736+0000 7fe4ddd51700 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dd550700 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbcf60 0x56090afbd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/3013667389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 msgr2=0x56090afb7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/3013667389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbcf60 msgr2=0x56090afbd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbcf60 0x56090afbd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/3013667389 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090ae578d0 con 0x56090afbd880 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 0x56090afb7260 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8098540 tx=0x7fe4c809ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffd42f9187b14ec2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4c37fe700 1 -- 172.21.15.52:0/3013667389 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c8004310 con 0x56090afbd880 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4c37fe700 1 -- 172.21.15.52:0/3013667389 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c809c270 con 0x56090afbd880 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.737+0000 7fe4c37fe700 1 -- 172.21.15.52:0/3013667389 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c801f340 con 0x56090afbd880 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 -- 172.21.15.52:0/3013667389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 msgr2=0x56090afb7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 0x56090afb7260 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8098540 tx=0x7fe4c809ba10 comp rx=0 tx=0).stop 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 -- 172.21.15.52:0/3013667389 shutdown_connections 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbcf60 0x56090afbd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/3013667389 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbd880 0x56090afb7260 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 -- 172.21.15.52:0/3013667389 >> 172.21.15.52:0/3013667389 conn(0x56090aee4080 msgr2=0x56090afb2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 -- 172.21.15.52:0/3013667389 shutdown_connections 2024-06-02T23:32:09.739 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.738+0000 7fe4ea037b80 1 -- 172.21.15.52:0/3013667389 wait complete. 2024-06-02T23:32:09.740 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.739+0000 7fe4ea037b80 1 -- start start 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.739+0000 7fe4ea037b80 1 Processor -- start 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.739+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afc3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.739+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbcf60 0x56090afc0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.739+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dd550700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afc3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56090aea8fc0 con 0x56090afbcf60 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56090af04810 con 0x56090afbd880 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56090afc4260 con 0x56090afb77a0 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbcf60 0x56090afc0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dd550700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afc3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52772/0 (socket says 172.21.15.52:52772) 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dd550700 1 -- 172.21.15.52:0/353845789 learned_addr learned my addr 172.21.15.52:0/353845789 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4ea037b80 10 client.0 _ll_register_callbacks cb 0x56090ae9e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:09.741 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4ddd51700 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 msgr2=0x56090afc3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afc3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/353845789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 msgr2=0x56090afc0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/353845789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090aece750 con 0x56090afbcf60 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbcf60 0x56090afc0930 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x56090afc3e20 tx=0x7fe4c8005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f65615e94233b53 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.740+0000 7fe4c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/353845789 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a9060 con 0x56090afbcf60 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/353845789 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe4c809bdd0 con 0x56090afbcf60 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c809c270 con 0x56090afbcf60 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/353845789 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe4c809bb70 con 0x56090afbcf60 2024-06-02T23:32:09.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4ea037b80 10 client.20766 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56090afcea60 con 0x56090afbcf60 2024-06-02T23:32:09.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.741+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/353845789 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c809c270 con 0x56090afbcf60 2024-06-02T23:32:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/353845789 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe4c8097950 con 0x56090afbcf60 2024-06-02T23:32:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4ea037b80 10 client.20766 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4ea037b80 10 client.20766 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:32:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4ea037b80 20 client.20766 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090afd1160 0x56090afd3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4ea037b80 4 client.20766 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56090af15d50 con 0x56090afd1160 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.742+0000 7fe4dd550700 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090afd1160 0x56090afd3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.743+0000 7fe4dd550700 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090afd1160 0x56090afd3550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93eb90a2cd1d9b64 server_cookie=dde43d13ef3dd4a in_seq=0 out_seq=0 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.743+0000 7fe4c0ff9700 10 client.20766 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/353845789 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56090af15d50 con 0x56090afd1160 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4c0ff9700 10 client.20766 handle_command_reply: tid=0 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 client.20766 shutdown 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbcf60 msgr2=0x56090afc0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbcf60 0x56090afc0930 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x56090afc3e20 tx=0x7fe4c8005c40 comp rx=0 tx=0).stop 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 shutdown_connections 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090afd1160 0x56090afd3550 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afc3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/353845789 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbcf60 0x56090afc0930 secure :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x56090afc3e20 tx=0x7fe4c8005c40 comp rx=0 tx=0).stop 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 >> 172.21.15.52:0/353845789 conn(0x56090aee4080 msgr2=0x56090afb1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.744+0000 7fe4c0ff9700 0 client.20766 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.745+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 shutdown_connections 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.745+0000 7fe4ea037b80 1 -- 172.21.15.52:0/353845789 wait complete. 2024-06-02T23:32:09.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.745+0000 7fe4ea037b80 20 client.20766 trim_cache size 0 max 16384 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 Processor -- start 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 -- start start 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afbe5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbf8b0 0x56090afc3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81bba30 con 0x56090afbf8b0 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81bd570 con 0x56090afbd880 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81bc100 con 0x56090afb77a0 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.768+0000 7fe4ddd51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbf8b0 0x56090afc3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.769 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dd550700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4ddd51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbf8b0 0x56090afc3170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36546/0 (socket says 172.21.15.52:36546) 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afbe5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4ddd51700 1 -- 172.21.15.52:0/1774307331 learned_addr learned my addr 172.21.15.52:0/1774307331 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/1774307331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 msgr2=0x56090afb7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/1774307331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbf8b0 msgr2=0x56090afc3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbf8b0 0x56090afc3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 -- 172.21.15.52:0/1774307331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090afd3b30 con 0x56090afbd880 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afbe5b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56090afc19c0 tx=0x7fe4c8000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8cad235f435d479 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4c2ffd700 1 -- 172.21.15.52:0/1774307331 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a9020 con 0x56090afbd880 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4c2ffd700 1 -- 172.21.15.52:0/1774307331 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c8097ac0 con 0x56090afbd880 2024-06-02T23:32:09.770 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.769+0000 7fe4c2ffd700 1 -- 172.21.15.52:0/1774307331 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c80a45b0 con 0x56090afbd880 2024-06-02T23:32:09.771 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 -- 172.21.15.52:0/1774307331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 msgr2=0x56090afbe5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afbe5b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x56090afc19c0 tx=0x7fe4c8000bc0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 -- 172.21.15.52:0/1774307331 shutdown_connections 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afbf8b0 0x56090afc3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afb77a0 0x56090afb7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/1774307331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afbe5b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.770+0000 7fe4ea037b80 1 -- 172.21.15.52:0/1774307331 >> 172.21.15.52:0/1774307331 conn(0x56090aee4080 msgr2=0x56090afbba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.771+0000 7fe4ea037b80 1 -- 172.21.15.52:0/1774307331 shutdown_connections 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.771+0000 7fe4ea037b80 1 -- 172.21.15.52:0/1774307331 wait complete. 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.771+0000 7fe4ea037b80 1 -- start start 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.771+0000 7fe4ea037b80 1 Processor -- start 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 0x56090afc9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.772 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afbf8b0 0x56090afd7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81bcea0 con 0x56090afb77a0 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81bc7d0 con 0x56090afbd880 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d81b99d0 con 0x56090afbf8b0 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ea037b80 10 client.0 _ll_register_callbacks cb 0x56090ae9e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 0x56090afc9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 0x56090afc9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36558/0 (socket says 172.21.15.52:36558) 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 -- 172.21.15.52:0/4251228508 learned_addr learned my addr 172.21.15.52:0/4251228508 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4ddd51700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afbf8b0 0x56090afd7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.774 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 -- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afbf8b0 msgr2=0x56090afd7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afbf8b0 0x56090afd7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 -- 172.21.15.52:0/4251228508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 msgr2=0x56090afc9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.772+0000 7fe4dd550700 1 -- 172.21.15.52:0/4251228508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090afd97f0 con 0x56090afb77a0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.773+0000 7fe4dd550700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 0x56090afc9520 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad65f2694a86ce0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.773+0000 7fe4c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.773+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/4251228508 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4d403e060 con 0x56090afb77a0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.773+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/4251228508 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fe4d4003880 con 0x56090afb77a0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.773+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4c8097ac0 con 0x56090afb77a0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/4251228508 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fe4d4003a80 con 0x56090afb77a0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 10 client.20772 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56090b03b200 con 0x56090afb77a0 2024-06-02T23:32:09.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/4251228508 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fe4c8097ac0 con 0x56090afb77a0 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/4251228508 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fe4d4031440 con 0x56090afb77a0 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 10 client.20772 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 10 client.20772 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 20 client.20772 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090b03b810 0x56090b03dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 4 client.20772 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.774+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56090af102d0 con 0x56090b03b810 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.775+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090b03b810 0x56090b03dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.775+0000 7fe4c0ff9700 10 client.20772 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:32:09.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.775+0000 7fe4dcd4f700 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090b03b810 0x56090b03dc30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e654761c4cde367d server_cookie=95d629dd6756d7c4 in_seq=0 out_seq=0 2024-06-02T23:32:09.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4c0ff9700 1 -- 172.21.15.52:0/4251228508 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56090af102d0 con 0x56090b03b810 2024-06-02T23:32:09.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4c0ff9700 10 client.20772 handle_command_reply: tid=0 2024-06-02T23:32:09.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 client.20772 shutdown 2024-06-02T23:32:09.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 msgr2=0x56090afc9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 0x56090afc9520 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4000c00 tx=0x7fe4d4009260 comp rx=0 tx=0).stop 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 shutdown_connections 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x56090b03b810 0x56090b03dc30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56090afbf8b0 0x56090afd7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56090afb77a0 0x56090afc9520 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 --2- 172.21.15.52:0/4251228508 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56090afbd880 0x56090afc9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 >> 172.21.15.52:0/4251228508 conn(0x56090aee4080 msgr2=0x56090afbe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4c0ff9700 0 client.20772 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 shutdown_connections 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.776+0000 7fe4ea037b80 1 -- 172.21.15.52:0/4251228508 wait complete. 2024-06-02T23:32:09.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.777+0000 7fe4ea037b80 20 client.20772 trim_cache size 0 max 16384 2024-06-02T23:32:09.779 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:32:09.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.777+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4c40a2df0 msgr2=0x7fe4c40a52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4c40a2df0 0x7fe4c40a52b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fe4d40098c0 tx=0x7fe4d4019040 comp rx=0 tx=0).stop 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 msgr2=0x7fe4d8111b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 0x7fe4d8111b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe4c800ebb0 tx=0x7fe4c800eec0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 shutdown_connections 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fe4c40a2df0 0x7fe4c40a52b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4d8106720 0x7fe4d81120d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fe4d81019d0 0x7fe4d8111650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 --2- 172.21.15.52:0/4171525713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d8102380 0x7fe4d8111b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 >> 172.21.15.52:0/4171525713 conn(0x7fe4d8078340 msgr2=0x7fe4d80fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 shutdown_connections 2024-06-02T23:32:09.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:09.778+0000 7fe4df7b4700 1 -- 172.21.15.52:0/4171525713 wait complete. 2024-06-02T23:32:09.965 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 27.4 2024-06-02T23:32:09.965 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:32:10.003 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:09 smithi052 ceph-mon[32340]: pgmap v1039: 97 pgs: 97 active+clean; 499 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-06-02T23:32:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:09 smithi123 ceph-mon[35977]: pgmap v1039: 97 pgs: 97 active+clean; 499 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-06-02T23:32:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:09 smithi174 ceph-mon[35994]: pgmap v1039: 97 pgs: 97 active+clean; 499 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 240 op/s 2024-06-02T23:32:10.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 53981 7.00 MB/sec execute 318 sec latency 84.334 ms 2024-06-02T23:32:10.557 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:32:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:10.833 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:10 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:32:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:10 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:32:11.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:10 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:32:11.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 54129 7.01 MB/sec execute 319 sec latency 116.116 ms 2024-06-02T23:32:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:11 smithi123 ceph-mon[35977]: pgmap v1040: 97 pgs: 97 active+clean; 492 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-06-02T23:32:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:12.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:11 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:11 smithi052 ceph-mon[32340]: pgmap v1040: 97 pgs: 97 active+clean; 492 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-06-02T23:32:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:12.085 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:11 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:11 smithi174 ceph-mon[35994]: pgmap v1040: 97 pgs: 97 active+clean; 492 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.9 MiB/s wr, 268 op/s 2024-06-02T23:32:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:11 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:12.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 54284 7.01 MB/sec execute 320 sec latency 152.026 ms 2024-06-02T23:32:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 -- 172.21.15.52:0/300676080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10102bf0 msgr2=0x7fee10103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 --2- 172.21.15.52:0/300676080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10102bf0 0x7fee10103070 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fee04009900 tx=0x7fee0402a410 comp rx=0 tx=0).stop 2024-06-02T23:32:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 -- 172.21.15.52:0/300676080 shutdown_connections 2024-06-02T23:32:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 --2- 172.21.15.52:0/300676080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101035b0 0x7fee10109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 --2- 172.21.15.52:0/300676080 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10102bf0 0x7fee10103070 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 --2- 172.21.15.52:0/300676080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10101a00 0x7fee10101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 -- 172.21.15.52:0/300676080 >> 172.21.15.52:0/300676080 conn(0x7fee100fcf90 msgr2=0x7fee100ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.954+0000 7fee18789640 1 -- 172.21.15.52:0/300676080 shutdown_connections 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 -- 172.21.15.52:0/300676080 wait complete. 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 Processor -- start 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 -- start start 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 0x7fee10197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102bf0 0x7fee101980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101035b0 0x7fee1019f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010d580 con 0x7fee10101a00 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010d400 con 0x7fee101035b0 2024-06-02T23:32:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee18789640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee1010d700 con 0x7fee10102bf0 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee16525640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 0x7fee10197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee16525640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 0x7fee10197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36586/0 (socket says 172.21.15.52:36586) 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee16525640 1 -- 172.21.15.52:0/1790323190 learned_addr learned my addr 172.21.15.52:0/1790323190 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.955+0000 7fee16d26640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101035b0 0x7fee1019f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee15d24640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102bf0 0x7fee101980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee16525640 1 -- 172.21.15.52:0/1790323190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102bf0 msgr2=0x7fee101980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee16525640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102bf0 0x7fee101980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee16525640 1 -- 172.21.15.52:0/1790323190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101035b0 msgr2=0x7fee1019f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee16525640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101035b0 0x7fee1019f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee16525640 1 -- 172.21.15.52:0/1790323190 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee1019f7b0 con 0x7fee10101a00 2024-06-02T23:32:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee16525640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 0x7fee10197b90 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fee0000ccb0 tx=0x7fee0000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee0f7fe640 1 -- 172.21.15.52:0/1790323190 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee00004b20 con 0x7fee10101a00 2024-06-02T23:32:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee0f7fe640 1 -- 172.21.15.52:0/1790323190 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee00003e40 con 0x7fee10101a00 2024-06-02T23:32:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee1019faa0 con 0x7fee10101a00 2024-06-02T23:32:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee0f7fe640 1 -- 172.21.15.52:0/1790323190 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000047e0 con 0x7fee10101a00 2024-06-02T23:32:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.956+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee1019ffc0 con 0x7fee10101a00 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee0f7fe640 1 -- 172.21.15.52:0/1790323190 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee00003990 con 0x7fee10101a00 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 Processor -- start 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 -- start start 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e920430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920a00 0x55692e925500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e928000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55692e8834f0 con 0x55692e920010 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55692e821870 con 0x55692e920a00 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee1b571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55692e87f4b0 con 0x55692e925b70 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee16d26640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e920430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee16d26640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e920430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36590/0 (socket says 172.21.15.52:36590) 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee16d26640 1 -- 172.21.15.52:0/3013281000 learned_addr learned my addr 172.21.15.52:0/3013281000 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee16525640 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920a00 0x55692e925500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee15d24640 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e928000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee0f7fe640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedf80a2cb0 0x7fedf80a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.959+0000 7fee16d26640 1 -- 172.21.15.52:0/3013281000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 msgr2=0x55692e928000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee16d26640 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e928000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee16d26640 1 -- 172.21.15.52:0/3013281000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920a00 msgr2=0x55692e925500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee16d26640 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920a00 0x55692e925500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee16d26640 1 -- 172.21.15.52:0/3013281000 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692e83d9b0 con 0x55692e920010 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee15d24640 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e928000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee0f7fe640 1 -- 172.21.15.52:0/1790323190 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee000cf070 con 0x7fee10101a00 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee16d26640 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e920430 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fee0800b6b0 tx=0x7fee0800bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe703841f93de034 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee16d26640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedf80a2cb0 0x7fedf80a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee0cff9640 1 -- 172.21.15.52:0/3013281000 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08003dd0 con 0x55692e920010 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee0cff9640 1 -- 172.21.15.52:0/3013281000 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee08015070 con 0x55692e920010 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.960+0000 7fee0cff9640 1 -- 172.21.15.52:0/3013281000 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee08011c50 con 0x55692e920010 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 -- 172.21.15.52:0/3013281000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 msgr2=0x55692e920430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee16d26640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedf80a2cb0 0x7fedf80a5170 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee0801a000 tx=0x7fee08010810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e920430 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7fee0800b6b0 tx=0x7fee0800bb70 comp rx=0 tx=0).stop 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 -- 172.21.15.52:0/3013281000 shutdown_connections 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e928000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920a00 0x55692e925500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 --2- 172.21.15.52:0/3013281000 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e920430 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 -- 172.21.15.52:0/3013281000 >> 172.21.15.52:0/3013281000 conn(0x55692e91b680 msgr2=0x55692e91ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 -- 172.21.15.52:0/3013281000 shutdown_connections 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 -- 172.21.15.52:0/3013281000 wait complete. 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 -- start start 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.961+0000 7fee1b571740 1 Processor -- start 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920010 0x55692e92a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92a980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e9319c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55692e928b20 con 0x55692e920a00 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55692e80cae0 con 0x55692e925b70 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55692e928ca0 con 0x55692e920010 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee1b571740 10 client.0 _ll_register_callbacks cb 0x55692e8459b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:12.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee15d24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee15d24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36594/0 (socket says 172.21.15.52:36594) 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920010 0x55692e92a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee15d24640 1 -- 172.21.15.52:0/3917717429 learned_addr learned my addr 172.21.15.52:0/3917717429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16d26640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e9319c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 -- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 msgr2=0x55692e9319c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e9319c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 -- 172.21.15.52:0/3917717429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 msgr2=0x55692e92a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 -- 172.21.15.52:0/3917717429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692e597210 con 0x55692e920010 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee15d24640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fee16525640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920010 0x55692e92a440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee000ccbf0 tx=0x7fee00097ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bee509fae253248e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fedf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3917717429 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0001d040 con 0x55692e920010 2024-06-02T23:32:12.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.962+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3917717429 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee00004000 con 0x55692e920010 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee08015070 con 0x55692e920010 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3917717429 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fee0009b260 con 0x55692e920010 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 10 client.29719 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55692e663620 con 0x55692e920010 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3917717429 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee08015070 con 0x55692e920010 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3917717429 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fee000d1110 con 0x55692e920010 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 10 client.29719 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 10 client.29719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 20 client.29719 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9362b0 0x55692e9386e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 4 client.29719 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55692e896f10 con 0x55692e9362b0 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.963+0000 7fee15d24640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9362b0 0x55692e9386e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.964+0000 7fedf5ffb640 10 client.29719 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.964+0000 7fee15d24640 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9362b0 0x55692e9386e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=294234a6ce15193f server_cookie=f46f3ec11548b3a7 in_seq=0 out_seq=0 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.964+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3917717429 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55692e896f10 con 0x55692e9362b0 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.964+0000 7fedf5ffb640 10 client.29719 handle_command_reply: tid=0 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.964+0000 7fee1b571740 1 client.29719 shutdown 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920010 msgr2=0x55692e92a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920010 0x55692e92a440 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fee000ccbf0 tx=0x7fee00097ba0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 shutdown_connections 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9362b0 0x55692e9386e0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e9319c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 --2- 172.21.15.52:0/3917717429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920010 0x55692e92a440 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fedf5ffb640 0 client.29719 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 >> 172.21.15.52:0/3917717429 conn(0x55692e91b680 msgr2=0x55692e91a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:12.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 shutdown_connections 2024-06-02T23:32:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 1 -- 172.21.15.52:0/3917717429 wait complete. 2024-06-02T23:32:12.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.965+0000 7fee1b571740 20 client.29719 trim_cache size 0 max 16384 2024-06-02T23:32:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 Processor -- start 2024-06-02T23:32:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 -- start start 2024-06-02T23:32:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e91ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920a00 0x55692e925470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e931950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee101a8d00 con 0x55692e920010 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee101aa840 con 0x55692e925b70 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee1b571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee101a93d0 con 0x55692e920a00 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee16525640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e91ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee16525640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e91ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36602/0 (socket says 172.21.15.52:36602) 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.980+0000 7fee16525640 1 -- 172.21.15.52:0/1028144870 learned_addr learned my addr 172.21.15.52:0/1028144870 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16d26640 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e931950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee15d24640 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920a00 0x55692e925470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16525640 1 -- 172.21.15.52:0/1028144870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920a00 msgr2=0x55692e925470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16525640 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920a00 0x55692e925470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16525640 1 -- 172.21.15.52:0/1028144870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 msgr2=0x55692e931950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16525640 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e931950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16525640 1 -- 172.21.15.52:0/1028144870 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55692e91b4b0 con 0x55692e920010 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16d26640 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e931950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fee16525640 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e91ef00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fee000a4040 tx=0x7fee00099e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f76ae8833e4e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fedf7fff640 1 -- 172.21.15.52:0/1028144870 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee0001d070 con 0x55692e920010 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fedf7fff640 1 -- 172.21.15.52:0/1028144870 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee000d2840 con 0x55692e920010 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.981+0000 7fedf7fff640 1 -- 172.21.15.52:0/1028144870 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee000860c0 con 0x55692e920010 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- 172.21.15.52:0/1028144870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 msgr2=0x55692e91ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e91ef00 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7fee000a4040 tx=0x7fee00099e10 comp rx=0 tx=0).stop 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- 172.21.15.52:0/1028144870 shutdown_connections 2024-06-02T23:32:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e920a00 0x55692e925470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e925b70 0x55692e931950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- 172.21.15.52:0/1028144870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920010 0x55692e91ef00 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- 172.21.15.52:0/1028144870 >> 172.21.15.52:0/1028144870 conn(0x55692e91b680 msgr2=0x55692e91ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- 172.21.15.52:0/1028144870 shutdown_connections 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- 172.21.15.52:0/1028144870 wait complete. 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- start start 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 Processor -- start 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920010 0x55692e931710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e92e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee101aa170 con 0x55692e920a00 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee101a9aa0 con 0x55692e920010 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee101a6ce0 con 0x55692e925b70 2024-06-02T23:32:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee1b571740 10 client.0 _ll_register_callbacks cb 0x7fee000072a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee15d24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee15d24640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36618/0 (socket says 172.21.15.52:36618) 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee15d24640 1 -- 172.21.15.52:0/3205558694 learned_addr learned my addr 172.21.15.52:0/3205558694 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee16525640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920010 0x55692e931710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.982+0000 7fee16d26640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e92e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee15d24640 1 -- 172.21.15.52:0/3205558694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 msgr2=0x55692e92e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee15d24640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e92e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee15d24640 1 -- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920010 msgr2=0x55692e931710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee15d24640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920010 0x55692e931710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee15d24640 1 -- 172.21.15.52:0/3205558694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55692e99ef50 con 0x55692e920a00 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee15d24640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92e200 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fee04009230 tx=0x7fee04005310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38576f930799ae7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fedf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3205558694 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee04035040 con 0x55692e920a00 2024-06-02T23:32:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3205558694 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee040037f0 con 0x55692e920a00 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee000d2840 con 0x55692e920a00 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3205558694 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fee04054070 con 0x55692e920a00 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee1b571740 10 client.20790 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55692e9a01a0 con 0x55692e920a00 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3205558694 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee000d2840 con 0x55692e920a00 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3205558694 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fee040046c0 con 0x55692e920a00 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee1b571740 10 client.20790 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee1b571740 10 client.20790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.983+0000 7fee1b571740 20 client.20790 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.984+0000 7fee1b571740 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9a0b50 0x55692e9a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.984+0000 7fee1b571740 4 client.20790 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.984+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55692e890c20 con 0x55692e9a0b50 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.984+0000 7fee16525640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9a0b50 0x55692e9a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:12.989 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.984+0000 7fee16525640 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9a0b50 0x55692e9a2f70 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad3b9cdf14843458 server_cookie=79114474d5076d40 in_seq=0 out_seq=0 2024-06-02T23:32:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:12.984+0000 7fedf5ffb640 10 client.20790 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:32:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:32:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fedf5ffb640 1 -- 172.21.15.52:0/3205558694 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55692e890c20 con 0x55692e9a0b50 2024-06-02T23:32:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fedf5ffb640 10 client.20790 handle_command_reply: tid=0 2024-06-02T23:32:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 client.20790 shutdown 2024-06-02T23:32:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 msgr2=0x55692e92e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92e200 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fee04009230 tx=0x7fee04005310 comp rx=0 tx=0).stop 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 shutdown_connections 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55692e9a0b50 0x55692e9a2f70 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55692e925b70 0x55692e92e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55692e920a00 0x55692e92e200 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 --2- 172.21.15.52:0/3205558694 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55692e920010 0x55692e931710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 >> 172.21.15.52:0/3205558694 conn(0x55692e91b680 msgr2=0x55692e91e670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fedf5ffb640 0 client.20790 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 shutdown_connections 2024-06-02T23:32:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 1 -- 172.21.15.52:0/3205558694 wait complete. 2024-06-02T23:32:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.033+0000 7fee1b571740 20 client.20790 trim_cache size 0 max 16384 2024-06-02T23:32:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:32:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:32:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedf80a2cb0 msgr2=0x7fedf80a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedf80a2cb0 0x7fedf80a5170 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fee0801a000 tx=0x7fee08010810 comp rx=0 tx=0).stop 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 msgr2=0x7fee10197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 0x7fee10197b90 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fee0000ccb0 tx=0x7fee0000a620 comp rx=0 tx=0).stop 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 shutdown_connections 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fedf80a2cb0 0x7fedf80a5170 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee10102bf0 0x7fee101980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee101035b0 0x7fee1019f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 --2- 172.21.15.52:0/1790323190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee10101a00 0x7fee10197b90 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 >> 172.21.15.52:0/1790323190 conn(0x7fee100fcf90 msgr2=0x7fee10105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 shutdown_connections 2024-06-02T23:32:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:13.034+0000 7fee18789640 1 -- 172.21.15.52:0/1790323190 wait complete. 2024-06-02T23:32:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:32:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:32:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:32:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:32:13.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:32:13.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 54448 7.01 MB/sec execute 321 sec latency 146.259 ms 2024-06-02T23:32:14.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:13 smithi123 ceph-mon[35977]: pgmap v1041: 97 pgs: 97 active+clean; 491 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-06-02T23:32:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:13 smithi174 ceph-mon[35994]: pgmap v1041: 97 pgs: 97 active+clean; 491 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-06-02T23:32:14.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:13 smithi052 ceph-mon[32340]: pgmap v1041: 97 pgs: 97 active+clean; 491 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 279 op/s 2024-06-02T23:32:14.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 54603 7.00 MB/sec execute 322 sec latency 198.439 ms 2024-06-02T23:32:14.776 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:15.468 INFO:tasks.workunit.client.0.smithi052.stdout: 10 54729 6.99 MB/sec execute 323 sec latency 165.614 ms 2024-06-02T23:32:15.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:15 smithi052 ceph-mon[32340]: pgmap v1042: 97 pgs: 97 active+clean; 493 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-06-02T23:32:16.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:15 smithi123 ceph-mon[35977]: pgmap v1042: 97 pgs: 97 active+clean; 493 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-06-02T23:32:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:15 smithi174 ceph-mon[35994]: pgmap v1042: 97 pgs: 97 active+clean; 493 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 265 op/s 2024-06-02T23:32:16.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 54874 6.98 MB/sec execute 324 sec latency 61.578 ms 2024-06-02T23:32:17.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 -- 172.21.15.52:0/2518785877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354074830 msgr2=0x7ff354074c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/2518785877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354074830 0x7ff354074c10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff3500098f0 tx=0x7ff350009c00 comp rx=0 tx=0).stop 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 -- 172.21.15.52:0/2518785877 shutdown_connections 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/2518785877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 0x7ff3541158b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/2518785877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354074830 0x7ff354074c10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/2518785877 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354075150 0x7ff354112f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.065+0000 7ff35c1b5700 1 -- 172.21.15.52:0/2518785877 >> 172.21.15.52:0/2518785877 conn(0x7ff3540fdf60 msgr2=0x7ff354100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.066+0000 7ff35c1b5700 1 -- 172.21.15.52:0/2518785877 shutdown_connections 2024-06-02T23:32:17.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.066+0000 7ff35c1b5700 1 -- 172.21.15.52:0/2518785877 wait complete. 2024-06-02T23:32:17.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.066+0000 7ff35c1b5700 1 Processor -- start 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.066+0000 7ff35c1b5700 1 -- start start 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35c1b5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354074830 0x7ff35419f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35c1b5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354075150 0x7ff35419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35c1b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 0x7ff3541a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35c1b5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff354118cf0 con 0x7ff354074830 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35c1b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff354118b70 con 0x7ff3541134c0 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35c1b5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff354118e70 con 0x7ff354075150 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359f51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354074830 0x7ff35419f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359f51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354074830 0x7ff35419f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46650/0 (socket says 172.21.15.52:46650) 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359f51700 1 -- 172.21.15.52:0/136623912 learned_addr learned my addr 172.21.15.52:0/136623912 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359750700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354075150 0x7ff35419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35a752700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 0x7ff3541a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359750700 1 -- 172.21.15.52:0/136623912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 msgr2=0x7ff3541a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359750700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 0x7ff3541a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359750700 1 -- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354074830 msgr2=0x7ff35419f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359750700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354074830 0x7ff35419f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff359750700 1 -- 172.21.15.52:0/136623912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3541a4230 con 0x7ff354075150 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.067+0000 7ff35a752700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 0x7ff3541a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.068+0000 7ff359750700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354075150 0x7ff35419f760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff34400eb90 tx=0x7ff34400eea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.068+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff34400cc90 con 0x7ff354075150 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.068+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff34400ce30 con 0x7ff354075150 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.068+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3541a4520 con 0x7ff354075150 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.068+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344010430 con 0x7ff354075150 2024-06-02T23:32:17.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.068+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3541abda0 con 0x7ff354075150 2024-06-02T23:32:17.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.069+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff354069180 con 0x7ff354075150 2024-06-02T23:32:17.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.071+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff344003680 con 0x7ff354075150 2024-06-02T23:32:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.072+0000 7ff34affd700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff3400a2ca0 0x7ff3400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.072+0000 7ff359f51700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff3400a2ca0 0x7ff3400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.072+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff3440cf2f0 con 0x7ff354075150 2024-06-02T23:32:17.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.072+0000 7ff359f51700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff3400a2ca0 0x7ff3400a5160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff350009680 tx=0x7ff350019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.079+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff34409c310 con 0x7ff354075150 2024-06-02T23:32:17.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55062 7.01 MB/sec execute 325 sec latency 108.726 ms 2024-06-02T23:32:17.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.511+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff35402dca0 con 0x7ff354075150 2024-06-02T23:32:17.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.512+0000 7ff34affd700 1 -- 172.21.15.52:0/136623912 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7ff34409bc10 con 0x7ff354075150 2024-06-02T23:32:17.513 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:32:17.513 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:32:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff3400a2ca0 msgr2=0x7ff3400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff3400a2ca0 0x7ff3400a5160 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7ff350009680 tx=0x7ff350019040 comp rx=0 tx=0).stop 2024-06-02T23:32:17.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354075150 msgr2=0x7ff35419f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354075150 0x7ff35419f760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ff34400eb90 tx=0x7ff34400eea0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 shutdown_connections 2024-06-02T23:32:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff3400a2ca0 0x7ff3400a5160 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3541134c0 0x7ff3541a3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff354074830 0x7ff35419f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 --2- 172.21.15.52:0/136623912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff354075150 0x7ff35419f760 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 >> 172.21.15.52:0/136623912 conn(0x7ff3540fdf60 msgr2=0x7ff3540fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 shutdown_connections 2024-06-02T23:32:17.518 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.515+0000 7ff35c1b5700 1 -- 172.21.15.52:0/136623912 wait complete. 2024-06-02T23:32:17.519 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/3931273500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae4101610 msgr2=0x7fcae41019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/3931273500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae4101610 0x7fcae41019f0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fcae0009990 tx=0x7fcae0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/3931273500 shutdown_connections 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/3931273500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae41066f0 0x7fcae410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/3931273500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae4101610 0x7fcae41019f0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/3931273500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101fc0 0x7fcae4106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/3931273500 >> 172.21.15.52:0/3931273500 conn(0x7fcae4078430 msgr2=0x7fcae4078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/3931273500 shutdown_connections 2024-06-02T23:32:17.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.602+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/3931273500 wait complete. 2024-06-02T23:32:17.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 Processor -- start 2024-06-02T23:32:17.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 -- start start 2024-06-02T23:32:17.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 0x7fcae4110690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae4101fc0 0x7fcae410eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae41066f0 0x7fcae410f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.606 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcae4112740 con 0x7fcae4101fc0 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41125c0 con 0x7fcae41066f0 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcaeb9f2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41128c0 con 0x7fcae4101610 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcae978e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 0x7fcae4110690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcae978e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 0x7fcae4110690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53670/0 (socket says 172.21.15.52:53670) 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcae978e700 1 -- 172.21.15.52:0/2872216419 learned_addr learned my addr 172.21.15.52:0/2872216419 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae41066f0 0x7fcae410f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.603+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae4101fc0 0x7fcae410eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae978e700 1 -- 172.21.15.52:0/2872216419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae41066f0 msgr2=0x7fcae410f100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae978e700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae41066f0 0x7fcae410f100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae978e700 1 -- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae4101fc0 msgr2=0x7fcae410eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae978e700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae4101fc0 0x7fcae410eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.607 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae978e700 1 -- 172.21.15.52:0/2872216419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae41ab5a0 con 0x7fcae4101610 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae4101fc0 0x7fcae410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcae978e700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 0x7fcae4110690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005910 tx=0x7fcae000ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.604+0000 7fcada7fc700 1 -- 172.21.15.52:0/2872216419 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0027070 con 0x7fcae4101610 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.605+0000 7fcada7fc700 1 -- 172.21.15.52:0/2872216419 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcae0003d40 con 0x7fcae4101610 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.605+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae41ab7d0 con 0x7fcae4101610 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.605+0000 7fcada7fc700 1 -- 172.21.15.52:0/2872216419 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0022dd0 con 0x7fcae4101610 2024-06-02T23:32:17.608 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.605+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae41abc20 con 0x7fcae4101610 2024-06-02T23:32:17.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.608+0000 7fcada7fc700 1 -- 172.21.15.52:0/2872216419 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcae002a070 con 0x7fcae4101610 2024-06-02T23:32:17.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.609+0000 7fcada7fc700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcad00a2ca0 0x7fcad00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.610+0000 7fcada7fc700 1 -- 172.21.15.52:0/2872216419 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcae00cfe00 con 0x7fcae4101610 2024-06-02T23:32:17.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.610+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcad00a2ca0 0x7fcad00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.610+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcad00a2ca0 0x7fcad00a5160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcae4110210 tx=0x7fcad4009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.611+0000 7fcaf6275b80 1 Processor -- start 2024-06-02T23:32:17.612 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.611+0000 7fcaf6275b80 1 -- start start 2024-06-02T23:32:17.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.611+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bfa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874b9f20 0x55fe874ba340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fe8740b810 con 0x55fe874bff60 2024-06-02T23:32:17.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae978e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bfa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae978e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bfa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35354/0 (socket says 172.21.15.52:35354) 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae978e700 1 -- 172.21.15.52:0/1935188832 learned_addr learned my addr 172.21.15.52:0/1935188832 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fe873b3720 con 0x55fe874bf640 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe874077d0 con 0x55fe874b9f20 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874b9f20 0x55fe874ba340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae978e700 1 -- 172.21.15.52:0/1935188832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874b9f20 msgr2=0x55fe874ba340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae978e700 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874b9f20 0x55fe874ba340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.612+0000 7fcae978e700 1 -- 172.21.15.52:0/1935188832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 msgr2=0x55fe874b99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcae978e700 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874b99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcae978e700 1 -- 172.21.15.52:0/1935188832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe873a9f50 con 0x55fe874bf640 2024-06-02T23:32:17.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:17.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcae978e700 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bfa20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcae0098c10 tx=0x7fcae0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de8979b6ad0948a8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcacf7fe700 1 -- 172.21.15.52:0/1935188832 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae009c2f0 con 0x55fe874bf640 2024-06-02T23:32:17.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcacf7fe700 1 -- 172.21.15.52:0/1935188832 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcae00043d0 con 0x55fe874bf640 2024-06-02T23:32:17.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.613+0000 7fcacf7fe700 1 -- 172.21.15.52:0/1935188832 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00a30c0 con 0x55fe874bf640 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 msgr2=0x55fe874bfa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bfa20 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fcae0098c10 tx=0x7fcae0009ca0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 shutdown_connections 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874b9f20 0x55fe874ba340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bfa20 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1935188832 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874b99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 >> 172.21.15.52:0/1935188832 conn(0x55fe873e4cc0 msgr2=0x55fe874b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 shutdown_connections 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.614+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1935188832 wait complete. 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 -- start start 2024-06-02T23:32:17.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 Processor -- start 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874c30c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874c3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fe874c66f0 con 0x55fe874bff60 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fe873abbf0 con 0x55fe874b9f20 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe874c6870 con 0x55fe874bf640 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcaf6275b80 10 client.0 _ll_register_callbacks cb 0x55fe873a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcae8f8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874c30c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcae978e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcae9f8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcae8f8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874c30c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53698/0 (socket says 172.21.15.52:53698) 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.615+0000 7fcae8f8d700 1 -- 172.21.15.52:0/859808186 learned_addr learned my addr 172.21.15.52:0/859808186 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:17.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcae9f8f700 1 -- 172.21.15.52:0/859808186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 msgr2=0x55fe874c30c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874c30c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcae9f8f700 1 -- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 msgr2=0x55fe874c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcae9f8f700 1 -- 172.21.15.52:0/859808186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe8735a8d0 con 0x55fe874bff60 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874c3600 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fcadc011e90 tx=0x7fcadc014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d4b8cb3e0d890cc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaccff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaccff9700 1 -- 172.21.15.52:0/859808186 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc004db0 con 0x55fe874bff60 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae00043d0 con 0x55fe874bff60 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaccff9700 1 -- 172.21.15.52:0/859808186 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcadc014b30 con 0x55fe874bff60 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaccff9700 1 -- 172.21.15.52:0/859808186 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcadc0093a0 con 0x55fe874bff60 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaf6275b80 10 client.20808 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.616+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fe874d10b0 con 0x55fe874bff60 2024-06-02T23:32:17.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaccff9700 1 -- 172.21.15.52:0/859808186 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcae00043d0 con 0x55fe874bff60 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaccff9700 1 -- 172.21.15.52:0/859808186 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcadc0043b0 con 0x55fe874bff60 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaf6275b80 10 client.20808 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaf6275b80 10 client.20808 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaf6275b80 20 client.20808 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe874d3790 0x55fe874d5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaf6275b80 4 client.20808 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe87418d10 con 0x55fe874d3790 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.617+0000 7fcae978e700 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe874d3790 0x55fe874d5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.618+0000 7fcaccff9700 10 client.20808 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.618+0000 7fcae978e700 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe874d3790 0x55fe874d5b80 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ecf5a3f8333bbb server_cookie=d68a80cb63f4c9e1 in_seq=0 out_seq=0 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaccff9700 1 -- 172.21.15.52:0/859808186 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe87418d10 con 0x55fe874d3790 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaccff9700 10 client.20808 handle_command_reply: tid=0 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 client.20808 shutdown 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 msgr2=0x55fe874c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874c3600 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fcadc011e90 tx=0x7fcadc014600 comp rx=0 tx=0).stop 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 shutdown_connections 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe874d3790 0x55fe874d5b80 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874c3600 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/859808186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874c30c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaccff9700 0 client.20808 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 >> 172.21.15.52:0/859808186 conn(0x55fe873e4cc0 msgr2=0x55fe874b45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 shutdown_connections 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 1 -- 172.21.15.52:0/859808186 wait complete. 2024-06-02T23:32:17.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.619+0000 7fcaf6275b80 20 client.20808 trim_cache size 0 max 16384 2024-06-02T23:32:17.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 Processor -- start 2024-06-02T23:32:17.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 -- start start 2024-06-02T23:32:17.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 0x55fe874b99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bff60 0x55fe874db440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41b2fe0 con 0x55fe874b9f20 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41b4b20 con 0x55fe874bf640 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.644+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41b36b0 con 0x55fe874bff60 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 0x55fe874b99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 0x55fe874b99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46704/0 (socket says 172.21.15.52:46704) 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 -- 172.21.15.52:0/1305689737 learned_addr learned my addr 172.21.15.52:0/1305689737 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bff60 0x55fe874db440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 -- 172.21.15.52:0/1305689737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bff60 msgr2=0x55fe874db440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bff60 0x55fe874db440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 -- 172.21.15.52:0/1305689737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 msgr2=0x55fe874bd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 -- 172.21.15.52:0/1305689737 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe874b5520 con 0x55fe874b9f20 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.645+0000 7fcae978e700 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 0x55fe874b99e0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fcae00cdca0 tx=0x7fcae0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e382da05b9679198 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.646+0000 7fcaceffd700 1 -- 172.21.15.52:0/1305689737 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae009c110 con 0x55fe874b9f20 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.646+0000 7fcaceffd700 1 -- 172.21.15.52:0/1305689737 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcae00ba070 con 0x55fe874b9f20 2024-06-02T23:32:17.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.646+0000 7fcaceffd700 1 -- 172.21.15.52:0/1305689737 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae00a30b0 con 0x55fe874b9f20 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1305689737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 msgr2=0x55fe874b99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 0x55fe874b99e0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7fcae00cdca0 tx=0x7fcae0009310 comp rx=0 tx=0).stop 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1305689737 shutdown_connections 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bff60 0x55fe874db440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874b9f20 0x55fe874b99e0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/1305689737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874bf640 0x55fe874bd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1305689737 >> 172.21.15.52:0/1305689737 conn(0x55fe873e4cc0 msgr2=0x55fe874bceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1305689737 shutdown_connections 2024-06-02T23:32:17.652 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 -- 172.21.15.52:0/1305689737 wait complete. 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 -- start start 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 Processor -- start 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.647+0000 7fcaf6275b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874cc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41b4450 con 0x55fe874bff60 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41b3d80 con 0x55fe874b9f20 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaf6275b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcae41b0fa0 con 0x55fe874bf640 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaf6275b80 10 client.0 _ll_register_callbacks cb 0x55fe873a5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae8f8d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46708/0 (socket says 172.21.15.52:46708) 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 -- 172.21.15.52:0/3522239270 learned_addr learned my addr 172.21.15.52:0/3522239270 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:17.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae8f8d700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53710/0 (socket says 172.21.15.52:53710) 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae978e700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874dbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 -- 172.21.15.52:0/3522239270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 msgr2=0x55fe874cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874cbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 -- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 msgr2=0x55fe874dbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874dbac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 -- 172.21.15.52:0/3522239270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe874af3b0 con 0x55fe874bff60 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcae9f8f700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874cc000 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00b010 tx=0x7fcadc009df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a3f29130865480 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaccff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaccff9700 1 -- 172.21.15.52:0/3522239270 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcadc007970 con 0x55fe874bff60 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcae00ba070 con 0x55fe874bff60 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.648+0000 7fcaccff9700 1 -- 172.21.15.52:0/3522239270 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcadc007b10 con 0x55fe874bff60 2024-06-02T23:32:17.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaccff9700 1 -- 172.21.15.52:0/3522239270 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcadc007d50 con 0x55fe874bff60 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaf6275b80 10 client.20814 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fe8753d410 con 0x55fe874bff60 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaccff9700 1 -- 172.21.15.52:0/3522239270 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcae00ba070 con 0x55fe874bff60 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaccff9700 1 -- 172.21.15.52:0/3522239270 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcadc016400 con 0x55fe874bff60 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaf6275b80 10 client.20814 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaf6275b80 10 client.20814 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaf6275b80 20 client.20814 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.649+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe8753da30 0x55fe8753fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcaf6275b80 4 client.20814 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe87413290 con 0x55fe8753da30 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcae978e700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe8753da30 0x55fe8753fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:17.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcaccff9700 10 client.20814 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcae978e700 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe8753da30 0x55fe8753fe50 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adc8be41f5658487 server_cookie=d11a830d82bd3161 in_seq=0 out_seq=0 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcaccff9700 1 -- 172.21.15.52:0/3522239270 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe87413290 con 0x55fe8753da30 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcaccff9700 10 client.20814 handle_command_reply: tid=0 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.650+0000 7fcaf6275b80 1 client.20814 shutdown 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 msgr2=0x55fe874cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874cc000 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7fcadc00b010 tx=0x7fcadc009df0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 shutdown_connections 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe8753da30 0x55fe8753fe50 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaccff9700 0 client.20814 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe874bff60 0x55fe874cc000 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe874b9f20 0x55fe874dbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 --2- 172.21.15.52:0/3522239270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe874bf640 0x55fe874cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 >> 172.21.15.52:0/3522239270 conn(0x55fe873e4cc0 msgr2=0x55fe874ade00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 shutdown_connections 2024-06-02T23:32:17.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 1 -- 172.21.15.52:0/3522239270 wait complete. 2024-06-02T23:32:17.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.651+0000 7fcaf6275b80 20 client.20814 trim_cache size 0 max 16384 2024-06-02T23:32:17.657 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcad00a2ca0 msgr2=0x7fcad00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcad00a2ca0 0x7fcad00a5160 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fcae4110210 tx=0x7fcad4009260 comp rx=0 tx=0).stop 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 msgr2=0x7fcae4110690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 0x7fcae4110690 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fcae0005910 tx=0x7fcae000ba10 comp rx=0 tx=0).stop 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 shutdown_connections 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcad00a2ca0 0x7fcad00a5160 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcae41066f0 0x7fcae410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcae4101610 0x7fcae4110690 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 --2- 172.21.15.52:0/2872216419 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcae4101fc0 0x7fcae410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 >> 172.21.15.52:0/2872216419 conn(0x7fcae4078430 msgr2=0x7fcae40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:17.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 shutdown_connections 2024-06-02T23:32:17.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:17.652+0000 7fcaeb9f2700 1 -- 172.21.15.52:0/2872216419 wait complete. 2024-06-02T23:32:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:17 smithi123 ceph-mon[35977]: pgmap v1043: 97 pgs: 97 active+clean; 495 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-06-02T23:32:18.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:17 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/136623912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:32:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:17 smithi174 ceph-mon[35994]: pgmap v1043: 97 pgs: 97 active+clean; 495 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-06-02T23:32:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:17 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/136623912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:32:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:17 smithi052 ceph-mon[32340]: pgmap v1043: 97 pgs: 97 active+clean; 495 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.7 MiB/s wr, 308 op/s 2024-06-02T23:32:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:17 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/136623912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:32:18.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55250 7.03 MB/sec execute 326 sec latency 66.973 ms 2024-06-02T23:32:19.081 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 147.4 --await-for 45.0' 2024-06-02T23:32:19.081 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 147.4 --await-for 45.0 2024-06-02T23:32:19.439 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:19.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55448 7.05 MB/sec execute 327 sec latency 62.998 ms 2024-06-02T23:32:19.573 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:32:19.573 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:32:19.825 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:19 smithi052 ceph-mon[32340]: pgmap v1044: 97 pgs: 97 active+clean; 497 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-06-02T23:32:20.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:19 smithi123 ceph-mon[35977]: pgmap v1044: 97 pgs: 97 active+clean; 497 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-06-02T23:32:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:19 smithi174 ceph-mon[35994]: pgmap v1044: 97 pgs: 97 active+clean; 497 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 266 op/s 2024-06-02T23:32:20.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55640 7.07 MB/sec execute 328 sec latency 70.552 ms 2024-06-02T23:32:21.223 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:21.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55810 7.08 MB/sec execute 329 sec latency 109.595 ms 2024-06-02T23:32:21.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 -- 172.21.15.52:0/2602460969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941024d0 msgr2=0x7f8994102950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 --2- 172.21.15.52:0/2602460969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941024d0 0x7f8994102950 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f897c0099c0 tx=0x7f897c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 -- 172.21.15.52:0/2602460969 shutdown_connections 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 --2- 172.21.15.52:0/2602460969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994102e90 0x7f899410f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 --2- 172.21.15.52:0/2602460969 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8994108580 0x7f8994108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 --2- 172.21.15.52:0/2602460969 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89941024d0 0x7f8994102950 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 -- 172.21.15.52:0/2602460969 >> 172.21.15.52:0/2602460969 conn(0x7f89940fdf60 msgr2=0x7f8994100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 -- 172.21.15.52:0/2602460969 shutdown_connections 2024-06-02T23:32:21.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.892+0000 7f8998952700 1 -- 172.21.15.52:0/2602460969 wait complete. 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.894+0000 7f8998952700 1 Processor -- start 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.894+0000 7f8998952700 1 -- start start 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.894+0000 7f8998952700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024d0 0x7f899419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.894+0000 7f8998952700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 0x7f899419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f8998952700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108580 0x7f8994198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f8998952700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8994112730 con 0x7f8994102e90 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f8998952700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89941125b0 con 0x7f8994108580 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f8998952700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89941128b0 con 0x7f89941024d0 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f898bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 0x7f899419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024d0 0x7f899419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f898bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 0x7f899419ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46734/0 (socket says 172.21.15.52:46734) 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f898bfff700 1 -- 172.21.15.52:0/1914606709 learned_addr learned my addr 172.21.15.52:0/1914606709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 -- 172.21.15.52:0/1914606709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108580 msgr2=0x7f8994198890 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108580 0x7f8994198890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 -- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 msgr2=0x7f899419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 0x7f899419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 -- 172.21.15.52:0/1914606709 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8994199070 con 0x7f89941024d0 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f898bfff700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 0x7f899419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:21.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.895+0000 7f899259c700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024d0 0x7f899419e6c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f898400c3a0 tx=0x7f898400c6b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:21.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.896+0000 7f898b7fe700 1 -- 172.21.15.52:0/1914606709 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898400ce20 con 0x7f89941024d0 2024-06-02T23:32:21.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.896+0000 7f898b7fe700 1 -- 172.21.15.52:0/1914606709 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8984004ce0 con 0x7f89941024d0 2024-06-02T23:32:21.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.896+0000 7f898b7fe700 1 -- 172.21.15.52:0/1914606709 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898401e690 con 0x7f89941024d0 2024-06-02T23:32:21.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.896+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8994199360 con 0x7f89941024d0 2024-06-02T23:32:21.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.896+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89940746c0 con 0x7f89941024d0 2024-06-02T23:32:21.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.900+0000 7f898b7fe700 1 -- 172.21.15.52:0/1914606709 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8984025020 con 0x7f89941024d0 2024-06-02T23:32:21.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:21 smithi052 ceph-mon[32340]: pgmap v1045: 97 pgs: 97 active+clean; 501 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-02T23:32:21.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.901+0000 7f89a31d5b80 1 Processor -- start 2024-06-02T23:32:21.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.901+0000 7f89a31d5b80 1 -- start start 2024-06-02T23:32:21.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a35f0 0x563db739cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db739d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f8992d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f8992d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35416/0 (socket says 172.21.15.52:35416) 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f8992d9d700 1 -- 172.21.15.52:0/715679924 learned_addr learned my addr 172.21.15.52:0/715679924 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f898bfff700 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db739d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563db72ee9a0 con 0x563db73a35f0 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563db72968b0 con 0x563db73a2cd0 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563db72ea960 con 0x563db739d460 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.902+0000 7f899259c700 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a35f0 0x563db739cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898bfff700 1 -- 172.21.15.52:0/715679924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 msgr2=0x563db73a30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898bfff700 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898bfff700 1 -- 172.21.15.52:0/715679924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a35f0 msgr2=0x563db739cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898bfff700 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a35f0 0x563db739cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898bfff700 1 -- 172.21.15.52:0/715679924 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db728d0e0 con 0x563db739d460 2024-06-02T23:32:21.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f8992d9d700 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898bfff700 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db739d8a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f897c000c00 tx=0x7f897c003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e77392ab8b79e79f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898b7fe700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89800a2d90 0x7f89800a5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.903+0000 7f898b7fe700 1 -- 172.21.15.52:0/1914606709 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f89840d6f90 con 0x7f89941024d0 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.904+0000 7f8988ff9700 1 -- 172.21.15.52:0/715679924 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c00b790 con 0x563db739d460 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.904+0000 7f8988ff9700 1 -- 172.21.15.52:0/715679924 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c00bcf0 con 0x563db739d460 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.904+0000 7f8988ff9700 1 -- 172.21.15.52:0/715679924 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c005500 con 0x563db739d460 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.904+0000 7f8992d9d700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89800a2d90 0x7f89800a5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 msgr2=0x563db739d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f8992d9d700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89800a2d90 0x7f89800a5250 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f898c009c10 tx=0x7f898c00b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db739d8a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f897c000c00 tx=0x7f897c003680 comp rx=0 tx=0).stop 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 shutdown_connections 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db739d8a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/715679924 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a35f0 0x563db739cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 >> 172.21.15.52:0/715679924 conn(0x563db72c7e50 msgr2=0x563db73981b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 shutdown_connections 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 -- 172.21.15.52:0/715679924 wait complete. 2024-06-02T23:32:21.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.905+0000 7f89a31d5b80 1 -- start start 2024-06-02T23:32:21.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 Processor -- start 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db73a9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a2cd0 0x563db73a6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a35f0 0x563db73a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563db73aa000 con 0x563db73a2cd0 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563db728ed80 con 0x563db73a35f0 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563db73aa180 con 0x563db739d460 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f89a31d5b80 10 client.0 _ll_register_callbacks cb 0x563db7288ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f899259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db73a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f899259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db73a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53756/0 (socket says 172.21.15.52:53756) 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f899259c700 1 -- 172.21.15.52:0/2686240805 learned_addr learned my addr 172.21.15.52:0/2686240805 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.906+0000 7f8992d9d700 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a35f0 0x563db73a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f899259c700 1 -- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a35f0 msgr2=0x563db73a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f899259c700 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a35f0 0x563db73a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f899259c700 1 -- 172.21.15.52:0/2686240805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a2cd0 msgr2=0x563db73a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f899259c700 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a2cd0 0x563db73a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f899259c700 1 -- 172.21.15.52:0/2686240805 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db723da60 con 0x563db739d460 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f899259c700 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db73a9350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89840a34c0 tx=0x7f89840a3d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=770bb85ce8bca43d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f8979ffb700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f8979ffb700 1 -- 172.21.15.52:0/2686240805 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840a3120 con 0x563db739d460 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f8979ffb700 1 -- 172.21.15.52:0/2686240805 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f898408e0b0 con 0x563db739d460 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f897c00bcf0 con 0x563db739d460 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f8979ffb700 1 -- 172.21.15.52:0/2686240805 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f898401c330 con 0x563db739d460 2024-06-02T23:32:21.912 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f89a31d5b80 10 client.29758 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563db73b4960 con 0x563db739d460 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.907+0000 7f8979ffb700 1 -- 172.21.15.52:0/2686240805 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f897c00bcf0 con 0x563db739d460 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f8979ffb700 1 -- 172.21.15.52:0/2686240805 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f89840b7ce0 con 0x563db739d460 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f89a31d5b80 10 client.29758 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f89a31d5b80 10 client.29758 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f89a31d5b80 20 client.29758 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db73b6ef0 0x563db73b9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f89a31d5b80 4 client.29758 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563db72fbea0 con 0x563db73b6ef0 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.908+0000 7f898bfff700 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db73b6ef0 0x563db73b9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.909+0000 7f898bfff700 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db73b6ef0 0x563db73b9320 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f011d23285ee8e server_cookie=cfc2f93c53e96697 in_seq=0 out_seq=0 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.909+0000 7f8979ffb700 10 client.29758 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.909+0000 7f8979ffb700 1 -- 172.21.15.52:0/2686240805 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563db72fbea0 con 0x563db73b6ef0 2024-06-02T23:32:21.913 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.909+0000 7f8979ffb700 10 client.29758 handle_command_reply: tid=0 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.909+0000 7f89a31d5b80 1 client.29758 shutdown 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.909+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 msgr2=0x563db73a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db73a9350 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f89840a34c0 tx=0x7f89840a3d00 comp rx=0 tx=0).stop 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 shutdown_connections 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db73b6ef0 0x563db73b9320 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a35f0 0x563db73a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db739d460 0x563db73a9350 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/2686240805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db73a2cd0 0x563db73a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 >> 172.21.15.52:0/2686240805 conn(0x563db72c7e50 msgr2=0x563db7397da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f8979ffb700 0 client.29758 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 shutdown_connections 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 1 -- 172.21.15.52:0/2686240805 wait complete. 2024-06-02T23:32:21.914 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.910+0000 7f89a31d5b80 20 client.29758 trim_cache size 0 max 16384 2024-06-02T23:32:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f89a31d5b80 1 Processor -- start 2024-06-02T23:32:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f89a31d5b80 1 -- start start 2024-06-02T23:32:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db739cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73bebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f899259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db739cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f899259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db739cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46760/0 (socket says 172.21.15.52:46760) 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.943+0000 7f899259c700 1 -- 172.21.15.52:0/905891268 learned_addr learned my addr 172.21.15.52:0/905891268 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89941aa790 con 0x563db739d460 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f898bfff700 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89941ac2d0 con 0x563db73a2cd0 2024-06-02T23:32:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89941aae60 con 0x563db73a35f0 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f899259c700 1 -- 172.21.15.52:0/905891268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 msgr2=0x563db73bebf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f899259c700 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73bebf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f899259c700 1 -- 172.21.15.52:0/905891268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 msgr2=0x563db73a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f899259c700 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f899259c700 1 -- 172.21.15.52:0/905891268 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563db73a4be0 con 0x563db739d460 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.944+0000 7f899259c700 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db739cf20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f89840a3e20 tx=0x7f89840ac280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5aae620573f19b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.945+0000 7f897bfff700 1 -- 172.21.15.52:0/905891268 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89840a4af0 con 0x563db739d460 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.945+0000 7f897bfff700 1 -- 172.21.15.52:0/905891268 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89840a3ec0 con 0x563db739d460 2024-06-02T23:32:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.945+0000 7f897bfff700 1 -- 172.21.15.52:0/905891268 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f898409f770 con 0x563db739d460 2024-06-02T23:32:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 msgr2=0x563db739cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:21.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db739cf20 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f89840a3e20 tx=0x7f89840ac280 comp rx=0 tx=0).stop 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 shutdown_connections 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73bebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db739cf20 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/905891268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 >> 172.21.15.52:0/905891268 conn(0x563db72c7e50 msgr2=0x563db73a3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 shutdown_connections 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.946+0000 7f89a31d5b80 1 -- 172.21.15.52:0/905891268 wait complete. 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.947+0000 7f89a31d5b80 1 -- start start 2024-06-02T23:32:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.947+0000 7f89a31d5b80 1 Processor -- start 2024-06-02T23:32:21.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.947+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db73af720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.947+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73afc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.947+0000 7f89a31d5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f89a31d5b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89941abc00 con 0x563db739d460 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f89a31d5b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89941ab530 con 0x563db73a2cd0 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f89a31d5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89941a8730 con 0x563db73a35f0 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73afc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35436/0 (socket says 172.21.15.52:35436) 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 -- 172.21.15.52:0/1353686282 learned_addr learned my addr 172.21.15.52:0/1353686282 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f89a31d5b80 10 client.0 _ll_register_callbacks cb 0x563db7288ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f8992d9d700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f899259c700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db73af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 -- 172.21.15.52:0/1353686282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 msgr2=0x563db73ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73ac4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 -- 172.21.15.52:0/1353686282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 msgr2=0x563db73af720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db73af720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 -- 172.21.15.52:0/1353686282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563db73accf0 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f899259c700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db73af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f898bfff700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73afc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f897c009990 tx=0x7f897c00b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cae847e703f35e65 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f8979ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.948+0000 7f8979ffb700 1 -- 172.21.15.52:0/1353686282 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f897c032060 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f8979ffb700 1 -- 172.21.15.52:0/1353686282 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f897c01d040 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89840a3ec0 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f8979ffb700 1 -- 172.21.15.52:0/1353686282 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f897c005320 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 10 client.30125 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563db7422790 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f8979ffb700 1 -- 172.21.15.52:0/1353686282 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89840a3ec0 con 0x563db73a2cd0 2024-06-02T23:32:21.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f8979ffb700 1 -- 172.21.15.52:0/1353686282 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f897c024030 con 0x563db73a2cd0 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 10 client.30125 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 10 client.30125 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 20 client.30125 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db7422a90 0x563db7424e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 4 client.30125 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.4, "await_for": 45.0}] 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 147.4, "await_for": 45.0}) -- 0x563db72f6420 con 0x563db7422a90 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.949+0000 7f899259c700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db7422a90 0x563db7424e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.950+0000 7f8979ffb700 10 client.30125 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:21.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:21.950+0000 7f899259c700 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db7422a90 0x563db7424e80 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1caad309b39abac server_cookie=768c26c2fc5abf36 in_seq=0 out_seq=0 2024-06-02T23:32:22.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:21 smithi123 ceph-mon[35977]: pgmap v1045: 97 pgs: 97 active+clean; 501 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-02T23:32:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:21 smithi174 ceph-mon[35994]: pgmap v1045: 97 pgs: 97 active+clean; 501 MiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.256+0000 7f8979ffb700 1 -- 172.21.15.52:0/1353686282 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563db72f6420 con 0x563db7422a90 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.256+0000 7f8979ffb700 10 client.30125 handle_command_reply: tid=0 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.256+0000 7f89a31d5b80 1 client.30125 shutdown 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 msgr2=0x563db73afc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73afc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f897c009990 tx=0x7f897c00b790 comp rx=0 tx=0).stop 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 shutdown_connections 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563db7422a90 0x563db7424e80 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f8979ffb700 0 client.30125 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563db73a35f0 0x563db73ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563db739d460 0x563db73af720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 --2- 172.21.15.52:0/1353686282 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563db73a2cd0 0x563db73afc60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 >> 172.21.15.52:0/1353686282 conn(0x563db72c7e50 msgr2=0x563db73a1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 shutdown_connections 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 1 -- 172.21.15.52:0/1353686282 wait complete. 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.257+0000 7f89a31d5b80 20 client.30125 trim_cache size 0 max 16384 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:32:22.259 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 66, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "bbb6892": { 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 66, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.4, 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:32:22.260 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:22.261 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89800a2d90 msgr2=0x7f89800a5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89800a2d90 0x7f89800a5250 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f898c009c10 tx=0x7f898c00b040 comp rx=0 tx=0).stop 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024d0 msgr2=0x7f899419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024d0 0x7f899419e6c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f898400c3a0 tx=0x7f898400c6b0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 shutdown_connections 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89800a2d90 0x7f89800a5250 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8994108580 0x7f8994198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89941024d0 0x7f899419e6c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 --2- 172.21.15.52:0/1914606709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8994102e90 0x7f899419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:22.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 >> 172.21.15.52:0/1914606709 conn(0x7f89940fdf60 msgr2=0x7f89940ff400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:22.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 shutdown_connections 2024-06-02T23:32:22.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:22.258+0000 7f8998952700 1 -- 172.21.15.52:0/1914606709 wait complete. 2024-06-02T23:32:22.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 7.08 MB/sec execute 330 sec latency 529.422 ms 2024-06-02T23:32:23.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 7.06 MB/sec execute 331 sec latency 1529.516 ms 2024-06-02T23:32:23.507 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bbb6892', quiesce took 13.5 seconds. Will release after: 13.899999999999999 2024-06-02T23:32:24.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:23 smithi123 ceph-mon[35977]: pgmap v1046: 97 pgs: 97 active+clean; 508 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 276 op/s 2024-06-02T23:32:24.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:23 smithi052 ceph-mon[32340]: pgmap v1046: 97 pgs: 97 active+clean; 508 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 276 op/s 2024-06-02T23:32:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:23 smithi174 ceph-mon[35994]: pgmap v1046: 97 pgs: 97 active+clean; 508 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.1 MiB/s wr, 276 op/s 2024-06-02T23:32:24.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 7.04 MB/sec execute 332 sec latency 2529.562 ms 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 -- 172.21.15.52:0/161561831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad4101390 msgr2=0x7fcad41017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 --2- 172.21.15.52:0/161561831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad4101390 0x7fcad41017f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009990 tx=0x7fcabc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 -- 172.21.15.52:0/161561831 shutdown_connections 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 --2- 172.21.15.52:0/161561831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 0x7fcad410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 --2- 172.21.15.52:0/161561831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcad41073d0 0x7fcad41077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 --2- 172.21.15.52:0/161561831 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad4101390 0x7fcad41017f0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 -- 172.21.15.52:0/161561831 >> 172.21.15.52:0/161561831 conn(0x7fcad40782c0 msgr2=0x7fcad40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 -- 172.21.15.52:0/161561831 shutdown_connections 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 -- 172.21.15.52:0/161561831 wait complete. 2024-06-02T23:32:24.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.776+0000 7fcad8d91700 1 Processor -- start 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 -- start start 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcad4101390 0x7fcad4103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 0x7fcad4103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 0x7fcad4103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4111530 con 0x7fcad41073d0 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41113b0 con 0x7fcad4101d30 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad8d91700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41116b0 con 0x7fcad4101390 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 0x7fcad4103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 0x7fcad4103ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44070/0 (socket says 172.21.15.52:44070) 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcacbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 0x7fcad4103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad2d9d700 1 -- 172.21.15.52:0/3841203999 learned_addr learned my addr 172.21.15.52:0/3841203999 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad259c700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcad4101390 0x7fcad4103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcacbfff700 1 -- 172.21.15.52:0/3841203999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcad4101390 msgr2=0x7fcad4103040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcad4101390 0x7fcad4103040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcacbfff700 1 -- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 msgr2=0x7fcad4103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 0x7fcad4103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcacbfff700 1 -- 172.21.15.52:0/3841203999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad41a70e0 con 0x7fcad4101d30 2024-06-02T23:32:24.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.777+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 0x7fcad4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:24.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.778+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 0x7fcad4103580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcabc000c00 tx=0x7fcabc0047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:24.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.778+0000 7fcacb7fe700 1 -- 172.21.15.52:0/3841203999 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc004830 con 0x7fcad4101d30 2024-06-02T23:32:24.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.778+0000 7fcacb7fe700 1 -- 172.21.15.52:0/3841203999 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcabc005020 con 0x7fcad4101d30 2024-06-02T23:32:24.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.778+0000 7fcacb7fe700 1 -- 172.21.15.52:0/3841203999 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc003d30 con 0x7fcad4101d30 2024-06-02T23:32:24.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.778+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad41a7370 con 0x7fcad4101d30 2024-06-02T23:32:24.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.778+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a7760 con 0x7fcad4101d30 2024-06-02T23:32:24.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.781+0000 7fcacb7fe700 1 -- 172.21.15.52:0/3841203999 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fcabc029060 con 0x7fcad4101d30 2024-06-02T23:32:24.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.782+0000 7fcacb7fe700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcac00a2c20 0x7fcac00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.782+0000 7fcad259c700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcac00a2c20 0x7fcac00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.782+0000 7fcacb7fe700 1 -- 172.21.15.52:0/3841203999 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcabc0cfb60 con 0x7fcad4101d30 2024-06-02T23:32:24.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.782+0000 7fcae3614b80 1 Processor -- start 2024-06-02T23:32:24.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 -- start start 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad259c700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcac00a2c20 0x7fcac00a50e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005950 tx=0x7fcac400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e459aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e453a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e4543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555f3e3a5890 con 0x555f3e453fa0 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f3e34d7a0 con 0x555f3e459fe0 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcae3614b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555f3e3a1850 con 0x555f3e4596c0 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e4543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e4543c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44074/0 (socket says 172.21.15.52:44074) 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 -- 172.21.15.52:0/3827421709 learned_addr learned my addr 172.21.15.52:0/3827421709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad259c700 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e459aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e453a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 -- 172.21.15.52:0/3827421709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 msgr2=0x555f3e459aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e459aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 -- 172.21.15.52:0/3827421709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 msgr2=0x555f3e453a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e453a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.783+0000 7fcad2d9d700 1 -- 172.21.15.52:0/3827421709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3e343fd0 con 0x555f3e453fa0 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.784+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e4543c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0078d0 tx=0x7fcacc0091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9e5fad7ebd9d650 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.784+0000 7fcac8ff9700 1 -- 172.21.15.52:0/3827421709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc019020 con 0x555f3e453fa0 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.784+0000 7fcac8ff9700 1 -- 172.21.15.52:0/3827421709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcacc015d00 con 0x555f3e453fa0 2024-06-02T23:32:24.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.784+0000 7fcac8ff9700 1 -- 172.21.15.52:0/3827421709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc014cf0 con 0x555f3e453fa0 2024-06-02T23:32:24.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 -- 172.21.15.52:0/3827421709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 msgr2=0x555f3e4543c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e4543c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0078d0 tx=0x7fcacc0091b0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 -- 172.21.15.52:0/3827421709 shutdown_connections 2024-06-02T23:32:24.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e4543c0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e459aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3827421709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e453a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 -- 172.21.15.52:0/3827421709 >> 172.21.15.52:0/3827421709 conn(0x555f3e37ed40 msgr2=0x555f3e44e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 -- 172.21.15.52:0/3827421709 shutdown_connections 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 -- 172.21.15.52:0/3827421709 wait complete. 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 -- start start 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.785+0000 7fcae3614b80 1 Processor -- start 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e45fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e45d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e45d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555f3e460a50 con 0x555f3e453fa0 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555f3e345c70 con 0x555f3e459fe0 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555f3e460bd0 con 0x555f3e4596c0 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcae3614b80 10 client.0 _ll_register_callbacks cb 0x555f3e33f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:24.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e45fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e45fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44090/0 (socket says 172.21.15.52:44090) 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 -- 172.21.15.52:0/3399012253 learned_addr learned my addr 172.21.15.52:0/3399012253 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e45d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e45d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 -- 172.21.15.52:0/3399012253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 msgr2=0x555f3e45d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e45d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 -- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 msgr2=0x555f3e45d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e45d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 -- 172.21.15.52:0/3399012253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3e2f4950 con 0x555f3e453fa0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcad259c700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e45fd30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fcac401da90 tx=0x7fcac40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b515f16c9300863 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.786+0000 7fcab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcab9ffb700 1 -- 172.21.15.52:0/3399012253 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4016070 con 0x555f3e453fa0 2024-06-02T23:32:24.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcab9ffb700 1 -- 172.21.15.52:0/3399012253 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcac4004a30 con 0x555f3e453fa0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcacc015d00 con 0x555f3e453fa0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcab9ffb700 1 -- 172.21.15.52:0/3399012253 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcac4004c70 con 0x555f3e453fa0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 10 client.20838 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555f3e46b3e0 con 0x555f3e453fa0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcab9ffb700 1 -- 172.21.15.52:0/3399012253 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcacc015d00 con 0x555f3e453fa0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcab9ffb700 1 -- 172.21.15.52:0/3399012253 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcac4024030 con 0x555f3e453fa0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 10 client.20838 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 10 client.20838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 20 client.20838 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e46fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 4 client.20838 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.787+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f3e3b2d90 con 0x555f3e46d970 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.788+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e46fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.788+0000 7fcacbfff700 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e46fda0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73ea2f84580448f2 server_cookie=c79b48eca9377719 in_seq=0 out_seq=0 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.788+0000 7fcab9ffb700 10 client.20838 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcab9ffb700 1 -- 172.21.15.52:0/3399012253 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555f3e3b2d90 con 0x555f3e46d970 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcab9ffb700 10 client.20838 handle_command_reply: tid=0 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 client.20838 shutdown 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 msgr2=0x555f3e45fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e45fd30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fcac401da90 tx=0x7fcac40047c0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 shutdown_connections 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e46fda0 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e459fe0 0x555f3e45d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcab9ffb700 0 client.20838 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e45fd30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 --2- 172.21.15.52:0/3399012253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e4596c0 0x555f3e45d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 >> 172.21.15.52:0/3399012253 conn(0x555f3e37ed40 msgr2=0x555f3e452a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:24.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 shutdown_connections 2024-06-02T23:32:24.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 1 -- 172.21.15.52:0/3399012253 wait complete. 2024-06-02T23:32:24.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.789+0000 7fcae3614b80 20 client.20838 trim_cache size 0 max 16384 2024-06-02T23:32:24.811 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 Processor -- start 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 -- start start 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 0x555f3e453a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e448710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e459fe0 0x555f3e475670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41aeb50 con 0x555f3e459fe0 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b0690 con 0x555f3e4596c0 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.810+0000 7fcae3614b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41af220 con 0x555f3e453fa0 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcad259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 0x555f3e453a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcad259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 0x555f3e453a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:38072/0 (socket says 172.21.15.52:38072) 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcad259c700 1 -- 172.21.15.52:0/561077888 learned_addr learned my addr 172.21.15.52:0/561077888 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:24.812 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e459fe0 0x555f3e475670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e448710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 -- 172.21.15.52:0/561077888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 msgr2=0x555f3e453a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 0x555f3e453a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 -- 172.21.15.52:0/561077888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e459fe0 msgr2=0x555f3e475670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e459fe0 0x555f3e475670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 -- 172.21.15.52:0/561077888 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f3e45f970 con 0x555f3e4596c0 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcad259c700 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 0x555f3e453a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcacbfff700 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e448710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcabc09c120 tx=0x7fcabc0cd500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7aa8e38be2f1ba server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcabbfff700 1 -- 172.21.15.52:0/561077888 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0a07d0 con 0x555f3e4596c0 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcabbfff700 1 -- 172.21.15.52:0/561077888 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcabc0a5000 con 0x555f3e4596c0 2024-06-02T23:32:24.813 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.811+0000 7fcabbfff700 1 -- 172.21.15.52:0/561077888 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0a4b70 con 0x555f3e4596c0 2024-06-02T23:32:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 -- 172.21.15.52:0/561077888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 msgr2=0x555f3e448710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e448710 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcabc09c120 tx=0x7fcabc0cd500 comp rx=0 tx=0).stop 2024-06-02T23:32:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 -- 172.21.15.52:0/561077888 shutdown_connections 2024-06-02T23:32:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e459fe0 0x555f3e475670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e453fa0 0x555f3e453a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 --2- 172.21.15.52:0/561077888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e448710 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 -- 172.21.15.52:0/561077888 >> 172.21.15.52:0/561077888 conn(0x555f3e37ed40 msgr2=0x555f3e45aa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 -- 172.21.15.52:0/561077888 shutdown_connections 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 -- 172.21.15.52:0/561077888 wait complete. 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.812+0000 7fcae3614b80 1 -- start start 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 Processor -- start 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e462f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e4634c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e459fe0 0x555f3e4d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41affc0 con 0x555f3e453fa0 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41af8f0 con 0x555f3e4596c0 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41acaf0 con 0x555f3e459fe0 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcae3614b80 10 client.0 _ll_register_callbacks cb 0x555f3e33f9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:24.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcad259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e462f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcad259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e462f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44106/0 (socket says 172.21.15.52:44106) 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcacbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e4634c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcad259c700 1 -- 172.21.15.52:0/1777124588 learned_addr learned my addr 172.21.15.52:0/1777124588 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcad2d9d700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e459fe0 0x555f3e4d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcacbfff700 1 -- 172.21.15.52:0/1777124588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e459fe0 msgr2=0x555f3e4d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcacbfff700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e459fe0 0x555f3e4d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcacbfff700 1 -- 172.21.15.52:0/1777124588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 msgr2=0x555f3e462f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcacbfff700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e462f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcacbfff700 1 -- 172.21.15.52:0/1777124588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f3e463d80 con 0x555f3e4596c0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.813+0000 7fcad259c700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e462f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcacbfff700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e4634c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x555f3e44efb0 tx=0x7fcabc0a07d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=985c585ccbeb89b6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcab9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcab9ffb700 1 -- 172.21.15.52:0/1777124588 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0cd740 con 0x555f3e4596c0 2024-06-02T23:32:24.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcab9ffb700 1 -- 172.21.15.52:0/1777124588 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fcabc09a720 con 0x555f3e4596c0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc0a5000 con 0x555f3e4596c0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcab9ffb700 1 -- 172.21.15.52:0/1777124588 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fcabc0b5240 con 0x555f3e4596c0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcae3614b80 10 client.30143 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555f3e4d9d20 con 0x555f3e4596c0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcab9ffb700 1 -- 172.21.15.52:0/1777124588 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fcabc0a5000 con 0x555f3e4596c0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.814+0000 7fcab9ffb700 1 -- 172.21.15.52:0/1777124588 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fcabc067260 con 0x555f3e4596c0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcae3614b80 10 client.30143 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcae3614b80 10 client.30143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcae3614b80 20 client.30143 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcae3614b80 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e4dc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcae3614b80 4 client.30143 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f3e3ad310 con 0x555f3e46d970 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcad259c700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e4dc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:24.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcad259c700 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e4dc4f0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0bb080e2b3336b3 server_cookie=13187af88c010a92 in_seq=0 out_seq=0 2024-06-02T23:32:24.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.815+0000 7fcab9ffb700 10 client.30143 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:24.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.816+0000 7fcab9ffb700 1 -- 172.21.15.52:0/1777124588 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f3e3ad310 con 0x555f3e46d970 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.816+0000 7fcab9ffb700 10 client.30143 handle_command_reply: tid=0 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.816+0000 7fcae3614b80 1 client.30143 shutdown 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 msgr2=0x555f3e4634c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e4634c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x555f3e44efb0 tx=0x7fcabc0a07d0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 shutdown_connections 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555f3e46d970 0x555f3e4dc4f0 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555f3e459fe0 0x555f3e4d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555f3e453fa0 0x555f3e462f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 --2- 172.21.15.52:0/1777124588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555f3e4596c0 0x555f3e4634c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 >> 172.21.15.52:0/1777124588 conn(0x555f3e37ed40 msgr2=0x555f3e45e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcab9ffb700 0 client.30143 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 shutdown_connections 2024-06-02T23:32:24.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 1 -- 172.21.15.52:0/1777124588 wait complete. 2024-06-02T23:32:24.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcae3614b80 20 client.30143 trim_cache size 0 max 16384 2024-06-02T23:32:24.822 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:32:24.822 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:32:24.822 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "6b7feea2-aeea-4546-a28f-4afb28f32ed6", 2024-06-02T23:32:24.822 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:32:24.822 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcac00a2c20 msgr2=0x7fcac00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.817+0000 7fcad8d91700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcac00a2c20 0x7fcac00a50e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcac4005950 tx=0x7fcac400a300 comp rx=0 tx=0).stop 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 msgr2=0x7fcad4103580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 0x7fcad4103580 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fcabc000c00 tx=0x7fcabc0047b0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 shutdown_connections 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fcac00a2c20 0x7fcac00a50e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fcad41073d0 0x7fcad4103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcad4101390 0x7fcad4103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 --2- 172.21.15.52:0/3841203999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcad4101d30 0x7fcad4103580 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:24.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 >> 172.21.15.52:0/3841203999 conn(0x7fcad40782c0 msgr2=0x7fcad406a330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:24.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 shutdown_connections 2024-06-02T23:32:24.824 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:24.818+0000 7fcad8d91700 1 -- 172.21.15.52:0/3841203999 wait complete. 2024-06-02T23:32:25.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 7.02 MB/sec execute 333 sec latency 3529.600 ms 2024-06-02T23:32:25.754 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: pgmap v1047: 97 pgs: 97 active+clean; 509 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:32:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:25.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:25.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:25.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:25 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.005 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: pgmap v1047: 97 pgs: 97 active+clean; 509 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:32:26.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:32:26.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:32:26.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:32:26.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:25 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 -- 172.21.15.52:0/2849232516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe44101a00 msgr2=0x7fbe44101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 --2- 172.21.15.52:0/2849232516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe44101a00 0x7fbe44101e20 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fbe34009900 tx=0x7fbe3402a410 comp rx=0 tx=0).stop 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 -- 172.21.15.52:0/2849232516 shutdown_connections 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 --2- 172.21.15.52:0/2849232516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe441035b0 0x7fbe44109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 --2- 172.21.15.52:0/2849232516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44102bf0 0x7fbe44103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 --2- 172.21.15.52:0/2849232516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe44101a00 0x7fbe44101e20 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 -- 172.21.15.52:0/2849232516 >> 172.21.15.52:0/2849232516 conn(0x7fbe440fcf70 msgr2=0x7fbe440ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 -- 172.21.15.52:0/2849232516 shutdown_connections 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.127+0000 7fbe49231640 1 -- 172.21.15.52:0/2849232516 wait complete. 2024-06-02T23:32:26.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 Processor -- start 2024-06-02T23:32:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 -- start start 2024-06-02T23:32:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44101a00 0x7fbe44197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe44102bf0 0x7fbe44198060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 0x7fbe4419f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410d500 con 0x7fbe441035b0 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410d380 con 0x7fbe44101a00 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe49231640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe4410d680 con 0x7fbe44102bf0 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 0x7fbe4419f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 0x7fbe4419f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44124/0 (socket says 172.21.15.52:44124) 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 -- 172.21.15.52:0/1513314429 learned_addr learned my addr 172.21.15.52:0/1513314429 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4259c640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe44102bf0 0x7fbe44198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44101a00 0x7fbe44197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 -- 172.21.15.52:0/1513314429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe44102bf0 msgr2=0x7fbe44198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe44102bf0 0x7fbe44198060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 -- 172.21.15.52:0/1513314429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44101a00 msgr2=0x7fbe44197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44101a00 0x7fbe44197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe4359e640 1 -- 172.21.15.52:0/1513314429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe4419f740 con 0x7fbe441035b0 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.128+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44101a00 0x7fbe44197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.129+0000 7fbe4359e640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 0x7fbe4419f0d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00eaa0 tx=0x7fbe3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.129+0000 7fbe3bfff640 1 -- 172.21.15.52:0/1513314429 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c00cdb0 con 0x7fbe441035b0 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.129+0000 7fbe3bfff640 1 -- 172.21.15.52:0/1513314429 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe3c005020 con 0x7fbe441035b0 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.129+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe4419fa30 con 0x7fbe441035b0 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.129+0000 7fbe3bfff640 1 -- 172.21.15.52:0/1513314429 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c0116d0 con 0x7fbe441035b0 2024-06-02T23:32:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.129+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe4419ff30 con 0x7fbe441035b0 2024-06-02T23:32:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.131+0000 7fbe3bfff640 1 -- 172.21.15.52:0/1513314429 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbe3c0051d0 con 0x7fbe441035b0 2024-06-02T23:32:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 Processor -- start 2024-06-02T23:32:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 -- start start 2024-06-02T23:32:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe3bfff640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe300a2c30 0x7fbe300a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef20840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef25910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef28410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b43ee7f4f0 con 0x55b43ef20e10 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b43ee1d870 con 0x55b43ef20420 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b43ee7b4b0 con 0x55b43ef25f80 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe3bfff640 1 -- 172.21.15.52:0/1513314429 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe3c0cf2d0 con 0x7fbe441035b0 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4259c640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe300a2c30 0x7fbe300a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef25910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef25910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44132/0 (socket says 172.21.15.52:44132) 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4259c640 1 -- 172.21.15.52:0/3253114858 learned_addr learned my addr 172.21.15.52:0/3253114858 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe4359e640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef28410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.132+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef20840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 -- 172.21.15.52:0/3253114858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 msgr2=0x55b43ef28410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef28410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 -- 172.21.15.52:0/3253114858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 msgr2=0x55b43ef20840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef20840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 -- 172.21.15.52:0/3253114858 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43ee399b0 con 0x55b43ef20e10 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef20840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4359e640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef28410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe300a2c30 0x7fbe300a50f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c013990 tx=0x7fbe2c00bb70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe4259c640 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef25910 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c010c10 tx=0x7fbe2c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6048cb6f19014f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe397fa640 1 -- 172.21.15.52:0/3253114858 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c00ec50 con 0x55b43ef20e10 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe397fa640 1 -- 172.21.15.52:0/3253114858 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe2c00edf0 con 0x55b43ef20e10 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.133+0000 7fbe397fa640 1 -- 172.21.15.52:0/3253114858 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c005280 con 0x55b43ef20e10 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3253114858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 msgr2=0x55b43ef25910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.139 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef25910 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c010c10 tx=0x7fbe2c01a000 comp rx=0 tx=0).stop 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3253114858 shutdown_connections 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef28410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef25910 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3253114858 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef20840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3253114858 >> 172.21.15.52:0/3253114858 conn(0x55b43ef17680 msgr2=0x55b43ef17a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3253114858 shutdown_connections 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3253114858 wait complete. 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- start start 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 Processor -- start 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef2a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef2ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef31c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b43ef28ed0 con 0x55b43ef20e10 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b43ee08ae0 con 0x55b43ef20420 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.134+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b43ef29050 con 0x55b43ef25f80 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4c01b740 10 client.0 _ll_register_callbacks cb 0x55b43ee419b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef2ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef2ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44140/0 (socket says 172.21.15.52:44140) 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 -- 172.21.15.52:0/321529422 learned_addr learned my addr 172.21.15.52:0/321529422 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef2a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 -- 172.21.15.52:0/321529422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 msgr2=0x55b43ef31c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef31c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 -- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 msgr2=0x55b43ef2a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef2a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 -- 172.21.15.52:0/321529422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43eb93210 con 0x55b43ef20e10 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef2a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:32:26.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4259c640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef2ac50 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00b3c0 tx=0x7fbe2c005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb317292ea10bcb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/321529422 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c02a070 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2c00edf0 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/321529422 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe2c003800 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/321529422 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbe2c036050 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4c01b740 10 client.20856 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.135+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b43ec5f620 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/321529422 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe2c00edf0 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/321529422 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbe2c0264d0 con 0x55b43ef20e10 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe4c01b740 10 client.20856 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe4c01b740 10 client.20856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe4c01b740 20 client.20856 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:26.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43ef36230 0x55b43ef38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe4c01b740 4 client.20856 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b43ee92f10 con 0x55b43ef36230 2024-06-02T23:32:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.136+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43ef36230 0x55b43ef38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe2a7fc640 10 client.20856 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:26.143 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43ef36230 0x55b43ef38620 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc988f07554da16f server_cookie=b8cc4c0f75fcb5ba in_seq=0 out_seq=0 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/321529422 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b43ee92f10 con 0x55b43ef36230 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe2a7fc640 10 client.20856 handle_command_reply: tid=0 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 client.20856 shutdown 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 msgr2=0x55b43ef2ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef2ac50 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00b3c0 tx=0x7fbe2c005730 comp rx=0 tx=0).stop 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 shutdown_connections 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43ef36230 0x55b43ef38620 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef31c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef2ac50 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe2a7fc640 0 client.20856 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/321529422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef2a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.137+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 >> 172.21.15.52:0/321529422 conn(0x55b43ef17680 msgr2=0x55b43ef167e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.138+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 shutdown_connections 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.138+0000 7fbe4c01b740 1 -- 172.21.15.52:0/321529422 wait complete. 2024-06-02T23:32:26.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.138+0000 7fbe4c01b740 20 client.20856 trim_cache size 0 max 16384 2024-06-02T23:32:26.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: pgmap v1047: 97 pgs: 97 active+clean; 509 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 978 KiB/s rd, 3.6 MiB/s wr, 237 op/s 2024-06-02T23:32:26.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:32:26.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:25 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.154+0000 7fbe4c01b740 1 Processor -- start 2024-06-02T23:32:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.154+0000 7fbe4c01b740 1 -- start start 2024-06-02T23:32:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef15f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef31be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef16490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.156 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441a6ac0 con 0x55b43ef20e10 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441a8600 con 0x55b43ef20420 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441a7190 con 0x55b43ef25f80 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef31be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef31be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44142/0 (socket says 172.21.15.52:44142) 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 -- 172.21.15.52:0/1465923068 learned_addr learned my addr 172.21.15.52:0/1465923068 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef15f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4359e640 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef16490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 -- 172.21.15.52:0/1465923068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 msgr2=0x55b43ef16490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef16490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 -- 172.21.15.52:0/1465923068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 msgr2=0x55b43ef15f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef15f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4259c640 1 -- 172.21.15.52:0/1465923068 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b43ef38950 con 0x55b43ef20e10 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.155+0000 7fbe4359e640 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef16490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:26.157 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.156+0000 7fbe4259c640 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef31be0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c000c00 tx=0x7fbe2c0108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e27ded7fe8f429c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.156+0000 7fbe38ff9640 1 -- 172.21.15.52:0/1465923068 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c03f070 con 0x55b43ef20e10 2024-06-02T23:32:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.156+0000 7fbe38ff9640 1 -- 172.21.15.52:0/1465923068 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe2c00b880 con 0x55b43ef20e10 2024-06-02T23:32:26.158 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.156+0000 7fbe38ff9640 1 -- 172.21.15.52:0/1465923068 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c02a040 con 0x55b43ef20e10 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- 172.21.15.52:0/1465923068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 msgr2=0x55b43ef31be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef31be0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c000c00 tx=0x7fbe2c0108f0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- 172.21.15.52:0/1465923068 shutdown_connections 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef16490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20e10 0x55b43ef31be0 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/1465923068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20420 0x55b43ef15f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- 172.21.15.52:0/1465923068 >> 172.21.15.52:0/1465923068 conn(0x55b43ef17680 msgr2=0x55b43ef220f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- 172.21.15.52:0/1465923068 shutdown_connections 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- 172.21.15.52:0/1465923068 wait complete. 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- start start 2024-06-02T23:32:26.161 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 Processor -- start 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 0x55b43ef2ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20e10 0x55b43ef2f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef301b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441a7f30 con 0x55b43ef20420 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441a7860 con 0x55b43ef20e10 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbe441a4ac0 con 0x55b43ef25f80 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe4c01b740 10 client.0 _ll_register_callbacks cb 0x7fbe34009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.157+0000 7fbe42d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 0x55b43ef2ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 0x55b43ef2ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44154/0 (socket says 172.21.15.52:44154) 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 -- 172.21.15.52:0/3812195303 learned_addr learned my addr 172.21.15.52:0/3812195303 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe4259c640 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20e10 0x55b43ef2f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 -- 172.21.15.52:0/3812195303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 msgr2=0x55b43ef301b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef301b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.162 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 -- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20e10 msgr2=0x55b43ef2f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20e10 0x55b43ef2f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 -- 172.21.15.52:0/3812195303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b43ef307a0 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe42d9d640 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 0x55b43ef2ecd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fbe340314e0 tx=0x7fbe340042b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6cfd413b1acfdb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe2a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/3812195303 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe3404c060 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/3812195303 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbe34005240 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe2c00b880 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/3812195303 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbe34035040 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe4c01b740 10 client.20862 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b43ef31b20 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.158+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/3812195303 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbe2c00b880 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/3812195303 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbe34003710 con 0x55b43ef20420 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4c01b740 10 client.20862 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:26.163 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4c01b740 10 client.20862 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4c01b740 20 client.20862 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43efa12c0 0x55b43efa36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4c01b740 4 client.20862 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b43ee8cc20 con 0x55b43efa12c0 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.159+0000 7fbe4259c640 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43efa12c0 0x55b43efa36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.160+0000 7fbe4259c640 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43efa12c0 0x55b43efa36e0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad16f0eec5ee6f2d server_cookie=e5dc7d30c5cfbf6f in_seq=0 out_seq=0 2024-06-02T23:32:26.164 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.160+0000 7fbe2a7fc640 10 client.20862 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.196+0000 7fbe2a7fc640 1 -- 172.21.15.52:0/3812195303 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55b43ee8cc20 con 0x55b43efa12c0 2024-06-02T23:32:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe2a7fc640 10 client.20862 handle_command_reply: tid=0 2024-06-02T23:32:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 client.20862 shutdown 2024-06-02T23:32:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 msgr2=0x55b43ef2ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 0x55b43ef2ecd0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7fbe340314e0 tx=0x7fbe340042b0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.198 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 shutdown_connections 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55b43efa12c0 0x55b43efa36e0 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b43ef25f80 0x55b43ef301b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b43ef20e10 0x55b43ef2f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 --2- 172.21.15.52:0/3812195303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b43ef20420 0x55b43ef2ecd0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe2a7fc640 0 client.20862 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 >> 172.21.15.52:0/3812195303 conn(0x55b43ef17680 msgr2=0x55b43ef1a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 shutdown_connections 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 1 -- 172.21.15.52:0/3812195303 wait complete. 2024-06-02T23:32:26.199 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe4c01b740 20 client.20862 trim_cache size 0 max 16384 2024-06-02T23:32:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:32:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:32:26.200 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe300a2c30 msgr2=0x7fbe300a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe49231640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe300a2c30 0x7fbe300a50f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c013990 tx=0x7fbe2c00bb70 comp rx=0 tx=0).stop 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 msgr2=0x7fbe4419f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.197+0000 7fbe49231640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 0x7fbe4419f0d0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00eaa0 tx=0x7fbe3c00ef60 comp rx=0 tx=0).stop 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 shutdown_connections 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbe300a2c30 0x7fbe300a50f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbe44102bf0 0x7fbe44198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbe441035b0 0x7fbe4419f0d0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 --2- 172.21.15.52:0/1513314429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbe44101a00 0x7fbe44197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 >> 172.21.15.52:0/1513314429 conn(0x7fbe440fcf70 msgr2=0x7fbe44105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 shutdown_connections 2024-06-02T23:32:26.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:26.198+0000 7fbe49231640 1 -- 172.21.15.52:0/1513314429 wait complete. 2024-06-02T23:32:26.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 7.00 MB/sec execute 334 sec latency 4529.649 ms 2024-06-02T23:32:27.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:26 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:27.126 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:26 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:27.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:26 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:32:27.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.98 MB/sec execute 335 sec latency 5529.678 ms 2024-06-02T23:32:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 -- 172.21.15.52:0/481032182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 msgr2=0x7fc2301076d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 --2- 172.21.15.52:0/481032182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc2301076d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009990 tx=0x7fc22c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 -- 172.21.15.52:0/481032182 shutdown_connections 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 --2- 172.21.15.52:0/481032182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc2301066d0 0x7fc23010e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 --2- 172.21.15.52:0/481032182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc2301076d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 --2- 172.21.15.52:0/481032182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc230107ca0 0x7fc230105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 -- 172.21.15.52:0/481032182 >> 172.21.15.52:0/481032182 conn(0x7fc230077590 msgr2=0x7fc2300fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 -- 172.21.15.52:0/481032182 shutdown_connections 2024-06-02T23:32:27.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 -- 172.21.15.52:0/481032182 wait complete. 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.631+0000 7fc2384a7700 1 Processor -- start 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 -- start start 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2301066d0 0x7fc2301999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc230199f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc230107ca0 0x7fc2301a4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc230111570 con 0x7fc2301072f0 2024-06-02T23:32:27.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301113f0 con 0x7fc2301066d0 2024-06-02T23:32:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc2384a7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301116f0 con 0x7fc230107ca0 2024-06-02T23:32:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc230199f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc230199f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44166/0 (socket says 172.21.15.52:44166) 2024-06-02T23:32:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc236a44700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc230107ca0 0x7fc2301a4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 -- 172.21.15.52:0/618078769 learned_addr learned my addr 172.21.15.52:0/618078769 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:27.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc236243700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2301066d0 0x7fc2301999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 -- 172.21.15.52:0/618078769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc230107ca0 msgr2=0x7fc2301a4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc230107ca0 0x7fc2301a4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 -- 172.21.15.52:0/618078769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2301066d0 msgr2=0x7fc2301999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2301066d0 0x7fc2301999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.632+0000 7fc235a42700 1 -- 172.21.15.52:0/618078769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2301a4f90 con 0x7fc2301072f0 2024-06-02T23:32:27.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.633+0000 7fc235a42700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc230199f10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc22000e970 tx=0x7fc22000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.633+0000 7fc2277fe700 1 -- 172.21.15.52:0/618078769 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220004a30 con 0x7fc2301072f0 2024-06-02T23:32:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.633+0000 7fc2277fe700 1 -- 172.21.15.52:0/618078769 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc22000cc90 con 0x7fc2301072f0 2024-06-02T23:32:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.633+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2301a51c0 con 0x7fc2301072f0 2024-06-02T23:32:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.633+0000 7fc2277fe700 1 -- 172.21.15.52:0/618078769 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200055e0 con 0x7fc2301072f0 2024-06-02T23:32:27.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.633+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2301a5750 con 0x7fc2301072f0 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.635+0000 7fc2277fe700 1 -- 172.21.15.52:0/618078769 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc22000c7e0 con 0x7fc2301072f0 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.636+0000 7fc2277fe700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc21c0a2ca0 0x7fc21c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.636+0000 7fc236243700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc21c0a2ca0 0x7fc21c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.636+0000 7fc2277fe700 1 -- 172.21.15.52:0/618078769 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc2200cf0f0 con 0x7fc2301072f0 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.636+0000 7fc242d2ab80 1 Processor -- start 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 -- start start 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc236243700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc21c0a2ca0 0x7fc21c0a5160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00b2e0 tx=0x7fc22c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:27.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x55857449a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x55857449e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x55857449f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5585743e7810 con 0x558574499fd0 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55857438f720 con 0x55857449a8f0 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585743e37d0 con 0x55857449ec00 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc236243700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x55857449a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x55857449e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x55857449e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45934/0 (socket says 172.21.15.52:45934) 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 -- 172.21.15.52:0/2718456957 learned_addr learned my addr 172.21.15.52:0/2718456957 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc236243700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x55857449a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44182/0 (socket says 172.21.15.52:44182) 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc236a44700 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x55857449f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 -- 172.21.15.52:0/2718456957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 msgr2=0x55857449f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x55857449f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 -- 172.21.15.52:0/2718456957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 msgr2=0x55857449a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.637+0000 7fc235a42700 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x55857449a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.638+0000 7fc235a42700 1 -- 172.21.15.52:0/2718456957 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558574385f50 con 0x55857449a8f0 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.638+0000 7fc235a42700 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x55857449e6c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc220009230 tx=0x7fc220097d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7347909571ae0a35 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.638+0000 7fc224d70700 1 -- 172.21.15.52:0/2718456957 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22000cad0 con 0x55857449a8f0 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.638+0000 7fc224d70700 1 -- 172.21.15.52:0/2718456957 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2200b4070 con 0x55857449a8f0 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.638+0000 7fc224d70700 1 -- 172.21.15.52:0/2718456957 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200a45f0 con 0x55857449a8f0 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2718456957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 msgr2=0x55857449e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x55857449e6c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fc220009230 tx=0x7fc220097d10 comp rx=0 tx=0).stop 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2718456957 shutdown_connections 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x55857449f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x55857449a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2718456957 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x55857449e6c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2718456957 >> 172.21.15.52:0/2718456957 conn(0x5585743c0cc0 msgr2=0x558574490850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:27.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2718456957 shutdown_connections 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2718456957 wait complete. 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 -- start start 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.639+0000 7fc242d2ab80 1 Processor -- start 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 0x5585744a3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449ec00 0x5585744a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5585744a6d60 con 0x55857449a8f0 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558574387bf0 con 0x55857449ec00 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5585744a6ee0 con 0x558574499fd0 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc242d2ab80 10 client.0 _ll_register_callbacks cb 0x558574381920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 0x5585744a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 0x5585744a3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44188/0 (socket says 172.21.15.52:44188) 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 -- 172.21.15.52:0/4023768345 learned_addr learned my addr 172.21.15.52:0/4023768345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:27.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc236243700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc236a44700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449ec00 0x5585744a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 -- 172.21.15.52:0/4023768345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 msgr2=0x5585744a61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 -- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449ec00 msgr2=0x5585744a3a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449ec00 0x5585744a3a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 -- 172.21.15.52:0/4023768345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5585743368d0 con 0x55857449a8f0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc235a42700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 0x5585744a3510 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc2200a40c0 tx=0x7fc22000c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b512afa6e7706c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.640+0000 7fc219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc219ffb700 1 -- 172.21.15.52:0/4023768345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22007f060 con 0x55857449a8f0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc219ffb700 1 -- 172.21.15.52:0/4023768345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc2200ba210 con 0x55857449a8f0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2200b4070 con 0x55857449a8f0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc219ffb700 1 -- 172.21.15.52:0/4023768345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc2200ba070 con 0x55857449a8f0 2024-06-02T23:32:27.649 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 10 client.20874 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5585744b1770 con 0x55857449a8f0 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc219ffb700 1 -- 172.21.15.52:0/4023768345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc2200b4070 con 0x55857449a8f0 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc219ffb700 1 -- 172.21.15.52:0/4023768345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc2200a4160 con 0x55857449a8f0 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 10 client.20874 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 10 client.20874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 20 client.20874 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x5585744b6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 4 client.20874 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.641+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5585743f4d10 con 0x5585744b3c40 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.642+0000 7fc236243700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x5585744b6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.642+0000 7fc236243700 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x5585744b6030 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c95b00ec45ca673f server_cookie=2b4fa04fc40db534 in_seq=0 out_seq=0 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.642+0000 7fc219ffb700 10 client.20874 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc219ffb700 1 -- 172.21.15.52:0/4023768345 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5585743f4d10 con 0x5585744b3c40 2024-06-02T23:32:27.650 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc219ffb700 10 client.20874 handle_command_reply: tid=0 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 client.20874 shutdown 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 msgr2=0x5585744a3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 0x5585744a3510 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7fc2200a40c0 tx=0x7fc22000c3d0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 shutdown_connections 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x5585744b6030 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449ec00 0x5585744a3a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc219ffb700 0 client.20874 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/4023768345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449a8f0 0x5585744a3510 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 >> 172.21.15.52:0/4023768345 conn(0x5585743c0cc0 msgr2=0x5585744949d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 shutdown_connections 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 1 -- 172.21.15.52:0/4023768345 wait complete. 2024-06-02T23:32:27.651 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.643+0000 7fc242d2ab80 20 client.20874 trim_cache size 0 max 16384 2024-06-02T23:32:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 Processor -- start 2024-06-02T23:32:27.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 -- start start 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x5585744b9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 0x5585744b95a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301aa980 con 0x55857449ec00 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301ac4c0 con 0x55857449a8f0 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301ab050 con 0x558574499fd0 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc236a44700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 0x5585744b95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc236a44700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 0x5585744b95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44190/0 (socket says 172.21.15.52:44190) 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.664+0000 7fc236a44700 1 -- 172.21.15.52:0/3687354073 learned_addr learned my addr 172.21.15.52:0/3687354073 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236243700 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc235a42700 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x5585744b9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236a44700 1 -- 172.21.15.52:0/3687354073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 msgr2=0x5585744a55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236a44700 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236a44700 1 -- 172.21.15.52:0/3687354073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 msgr2=0x5585744b9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236a44700 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x5585744b9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236a44700 1 -- 172.21.15.52:0/3687354073 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5585744a6d60 con 0x55857449ec00 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc235a42700 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x5585744b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc236a44700 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 0x5585744b95a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc228009f70 tx=0x7fc2280073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d049df3402aa8e6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc21bfff700 1 -- 172.21.15.52:0/3687354073 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc228007d90 con 0x55857449ec00 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc21bfff700 1 -- 172.21.15.52:0/3687354073 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc22801b070 con 0x55857449ec00 2024-06-02T23:32:27.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.665+0000 7fc21bfff700 1 -- 172.21.15.52:0/3687354073 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2280178b0 con 0x55857449ec00 2024-06-02T23:32:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 -- 172.21.15.52:0/3687354073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 msgr2=0x5585744b95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 0x5585744b95a0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7fc228009f70 tx=0x7fc2280073e0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 -- 172.21.15.52:0/3687354073 shutdown_connections 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55857449ec00 0x5585744b95a0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558574499fd0 0x5585744a55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/3687354073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x5585744b9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 -- 172.21.15.52:0/3687354073 >> 172.21.15.52:0/3687354073 conn(0x5585743c0cc0 msgr2=0x55857448a470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 -- 172.21.15.52:0/3687354073 shutdown_connections 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 -- 172.21.15.52:0/3687354073 wait complete. 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 -- start start 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 Processor -- start 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x558574517be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.666+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x558574518120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc242d2ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x558574512510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301abdf0 con 0x558574499fd0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301ab720 con 0x55857449a8f0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc242d2ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301a8920 con 0x55857449ec00 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc242d2ab80 10 client.0 _ll_register_callbacks cb 0x558574381920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x558574517be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x558574517be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44200/0 (socket says 172.21.15.52:44200) 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 -- 172.21.15.52:0/2238221798 learned_addr learned my addr 172.21.15.52:0/2238221798 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc235a42700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x558574518120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236a44700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x558574512510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 -- 172.21.15.52:0/2238221798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 msgr2=0x558574512510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x558574512510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 -- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 msgr2=0x558574518120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x558574518120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 -- 172.21.15.52:0/2238221798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558574512d50 con 0x558574499fd0 2024-06-02T23:32:27.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236a44700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x558574512510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc236243700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x558574517be0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc22c005cd0 tx=0x7fc22c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b882fd005d69633 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc219ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc219ffb700 1 -- 172.21.15.52:0/2238221798 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc22c003a90 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc219ffb700 1 -- 172.21.15.52:0/2238221798 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc22c003c30 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.667+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc22801b070 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc219ffb700 1 -- 172.21.15.52:0/2238221798 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc22c042050 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 10 client.20880 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55857451f430 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc219ffb700 1 -- 172.21.15.52:0/2238221798 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc22801b070 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc219ffb700 1 -- 172.21.15.52:0/2238221798 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc22c00f460 con 0x558574499fd0 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 10 client.20880 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 10 client.20880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 20 client.20880 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:27.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x558574521b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 4 client.20880 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5585743ef290 con 0x5585744b3c40 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.668+0000 7fc235a42700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x558574521b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc235a42700 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x558574521b80 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b20a3914b4d8ae server_cookie=a07a58c77ece71f8 in_seq=0 out_seq=0 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc219ffb700 10 client.20880 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc219ffb700 1 -- 172.21.15.52:0/2238221798 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5585743ef290 con 0x5585744b3c40 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc219ffb700 10 client.20880 handle_command_reply: tid=0 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 client.20880 shutdown 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 msgr2=0x558574517be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x558574517be0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7fc22c005cd0 tx=0x7fc22c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 shutdown_connections 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5585744b3c40 0x558574521b80 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc219ffb700 0 client.20880 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55857449ec00 0x558574512510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558574499fd0 0x558574517be0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 --2- 172.21.15.52:0/2238221798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55857449a8f0 0x558574518120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 >> 172.21.15.52:0/2238221798 conn(0x5585743c0cc0 msgr2=0x55857449e3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 shutdown_connections 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 1 -- 172.21.15.52:0/2238221798 wait complete. 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.669+0000 7fc242d2ab80 20 client.20880 trim_cache size 0 max 16384 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:32:27.675 INFO:teuthology.orchestra.run.smithi052.stdout: "6b7feea2-aeea-4546-a28f-4afb28f32ed6": { 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "6b7feea2-aeea-4546-a28f-4afb28f32ed6", 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:32:27.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc21c0a2ca0 msgr2=0x7fc21c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc21c0a2ca0 0x7fc21c0a5160 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fc22c00b2e0 tx=0x7fc22c005960 comp rx=0 tx=0).stop 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 msgr2=0x7fc230199f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc230199f10 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7fc22000e970 tx=0x7fc22000ec80 comp rx=0 tx=0).stop 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 shutdown_connections 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc21c0a2ca0 0x7fc21c0a5160 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc230107ca0 0x7fc2301a4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2301066d0 0x7fc2301999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 --2- 172.21.15.52:0/618078769 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2301072f0 0x7fc230199f10 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 >> 172.21.15.52:0/618078769 conn(0x7fc230077590 msgr2=0x7fc230104300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 shutdown_connections 2024-06-02T23:32:27.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:27.670+0000 7fc2384a7700 1 -- 172.21.15.52:0/618078769 wait complete. 2024-06-02T23:32:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:27 smithi174 ceph-mon[35994]: pgmap v1048: 97 pgs: 97 active+clean; 510 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-06-02T23:32:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:27 smithi052 ceph-mon[32340]: pgmap v1048: 97 pgs: 97 active+clean; 510 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-06-02T23:32:28.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:27 smithi123 ceph-mon[35977]: pgmap v1048: 97 pgs: 97 active+clean; 510 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 934 KiB/s rd, 3.8 MiB/s wr, 268 op/s 2024-06-02T23:32:28.462 INFO:tasks.cephfs.filesystem:scrub status for tag:6b7feea2-aeea-4546-a28f-4afb28f32ed6 - {'path': '/', 'tag': '6b7feea2-aeea-4546-a28f-4afb28f32ed6', 'options': 'recursive,force'} 2024-06-02T23:32:28.469 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.95 MB/sec execute 336 sec latency 6530.028 ms 2024-06-02T23:32:29.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.93 MB/sec execute 337 sec latency 7530.117 ms 2024-06-02T23:32:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:29 smithi174 ceph-mon[35994]: pgmap v1049: 97 pgs: 97 active+clean; 509 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-06-02T23:32:30.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:29 smithi052 ceph-mon[32340]: pgmap v1049: 97 pgs: 97 active+clean; 509 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-06-02T23:32:30.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:29 smithi123 ceph-mon[35977]: pgmap v1049: 97 pgs: 97 active+clean; 509 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 883 KiB/s rd, 3.4 MiB/s wr, 241 op/s 2024-06-02T23:32:30.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.91 MB/sec execute 338 sec latency 8530.261 ms 2024-06-02T23:32:31.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:30 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:32:31.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:30 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:32:31.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:30 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:32:31.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:30 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:32:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:30 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:32:31.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:30 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:32:31.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.89 MB/sec execute 339 sec latency 9530.386 ms 2024-06-02T23:32:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:31 smithi174 ceph-mon[35994]: pgmap v1050: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-06-02T23:32:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:32:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:32:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:31 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:32:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:31 smithi052 ceph-mon[32340]: pgmap v1050: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-06-02T23:32:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:32:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:32:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:31 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:32:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:31 smithi123 ceph-mon[35977]: pgmap v1050: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 747 KiB/s rd, 3.1 MiB/s wr, 225 op/s 2024-06-02T23:32:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:32:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:32:32.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:31 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:32:32.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.87 MB/sec execute 340 sec latency 10530.481 ms 2024-06-02T23:32:33.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.85 MB/sec execute 341 sec latency 11530.622 ms 2024-06-02T23:32:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:33 smithi174 ceph-mon[35994]: pgmap v1051: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-06-02T23:32:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:33 smithi052 ceph-mon[32340]: pgmap v1051: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-06-02T23:32:34.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:33 smithi123 ceph-mon[35977]: pgmap v1051: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 2.2 MiB/s wr, 201 op/s 2024-06-02T23:32:34.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.83 MB/sec execute 342 sec latency 12530.749 ms 2024-06-02T23:32:35.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.81 MB/sec execute 343 sec latency 13530.814 ms 2024-06-02T23:32:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:35 smithi174 ceph-mon[35994]: pgmap v1052: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:32:36.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:35 smithi052 ceph-mon[32340]: pgmap v1052: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:32:36.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:35 smithi123 ceph-mon[35977]: pgmap v1052: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 1.1 MiB/s wr, 141 op/s 2024-06-02T23:32:36.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.79 MB/sec execute 344 sec latency 14530.929 ms 2024-06-02T23:32:37.408 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:32:37.470 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.77 MB/sec execute 345 sec latency 15530.982 ms 2024-06-02T23:32:37.673 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:38.047 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:37 smithi052 ceph-mon[32340]: pgmap v1053: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 823 KiB/s wr, 121 op/s 2024-06-02T23:32:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:37 smithi174 ceph-mon[35994]: pgmap v1053: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 823 KiB/s wr, 121 op/s 2024-06-02T23:32:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:37 smithi123 ceph-mon[35977]: pgmap v1053: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 823 KiB/s wr, 121 op/s 2024-06-02T23:32:38.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.75 MB/sec execute 346 sec latency 16531.066 ms 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 -- 172.21.15.52:0/1997363977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 msgr2=0x7f5cf0103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/1997363977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0103140 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4009930 tx=0x7f5ce402a420 comp rx=0 tx=0).stop 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 -- 172.21.15.52:0/1997363977 shutdown_connections 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/1997363977 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 0x7f5cf0109f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/1997363977 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0103140 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/1997363977 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 0x7f5cf0100110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.293 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 -- 172.21.15.52:0/1997363977 >> 172.21.15.52:0/1997363977 conn(0x7f5cf00fb620 msgr2=0x7f5cf00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 -- 172.21.15.52:0/1997363977 shutdown_connections 2024-06-02T23:32:39.294 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.292+0000 7f5cf4be3640 1 -- 172.21.15.52:0/1997363977 wait complete. 2024-06-02T23:32:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 Processor -- start 2024-06-02T23:32:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 -- start start 2024-06-02T23:32:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 0x7f5cf0197bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 0x7f5cf019f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.296 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010d560 con 0x7f5cf0102cc0 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010d3e0 con 0x7f5cf00ffcf0 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cf4be3640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf010d6e0 con 0x7f5cf0103680 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cedd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.293+0000 7f5cedd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0198110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52690/0 (socket says 172.21.15.52:52690) 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 -- 172.21.15.52:0/247007813 learned_addr learned my addr 172.21.15.52:0/247007813 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cee59c640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 0x7f5cf0197bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 0x7f5cf019f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 -- 172.21.15.52:0/247007813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 msgr2=0x7f5cf019f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 0x7f5cf019f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 -- 172.21.15.52:0/247007813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 msgr2=0x7f5cf0197bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 0x7f5cf0197bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 -- 172.21.15.52:0/247007813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cf019f7f0 con 0x7f5cf0102cc0 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 0x7f5cf019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:32:39.297 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cee59c640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 0x7f5cf0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0198110 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40314e0 tx=0x7f5ce4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cdf7fe640 1 -- 172.21.15.52:0/247007813 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce403f030 con 0x7f5cf0102cc0 2024-06-02T23:32:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cf019fa80 con 0x7f5cf0102cc0 2024-06-02T23:32:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cdf7fe640 1 -- 172.21.15.52:0/247007813 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce4003cb0 con 0x7f5cf0102cc0 2024-06-02T23:32:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cdf7fe640 1 -- 172.21.15.52:0/247007813 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4035070 con 0x7f5cf0102cc0 2024-06-02T23:32:39.298 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.294+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cf019ff70 con 0x7f5cf0102cc0 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.296+0000 7f5cdf7fe640 1 -- 172.21.15.52:0/247007813 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5ce4042030 con 0x7f5cf0102cc0 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 Processor -- start 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 -- start start 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ec5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9eca910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ecd410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5581e9e244f0 con 0x5581e9ecaf80 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581e9dc2870 con 0x5581e9ec5e10 2024-06-02T23:32:39.301 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.297+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581e9e204b0 con 0x5581e9ec5420 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5cedd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ecd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5cedd9b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ecd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52700/0 (socket says 172.21.15.52:52700) 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5cedd9b640 1 -- 172.21.15.52:0/3689237805 learned_addr learned my addr 172.21.15.52:0/3689237805 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5cee59c640 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9eca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ec5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5cdf7fe640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cd80a2c30 0x7f5cd80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.298+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cd80a2c30 0x7f5cd80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cedd9b640 1 -- 172.21.15.52:0/3689237805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 msgr2=0x5581e9ec5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ec5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cedd9b640 1 -- 172.21.15.52:0/3689237805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 msgr2=0x5581e9eca910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9eca910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cdf7fe640 1 -- 172.21.15.52:0/247007813 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ce40ef6f0 con 0x7f5cf0102cc0 2024-06-02T23:32:39.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cedd9b640 1 -- 172.21.15.52:0/3689237805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e9dde9b0 con 0x5581e9ecaf80 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cee59c640 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9eca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ecd410 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40bbc80 tx=0x7f5ce4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f666cd63d736e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cd80a2c30 0x7f5cd80a50f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x5581e9ec6670 tx=0x7f5ce8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cdcff9640 1 -- 172.21.15.52:0/3689237805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4049030 con 0x5581e9ecaf80 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cdcff9640 1 -- 172.21.15.52:0/3689237805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce40bc680 con 0x5581e9ecaf80 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.299+0000 7f5cdcff9640 1 -- 172.21.15.52:0/3689237805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce40c4bf0 con 0x5581e9ecaf80 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3689237805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 msgr2=0x5581e9ecd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ecd410 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40bbc80 tx=0x7f5ce4009260 comp rx=0 tx=0).stop 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3689237805 shutdown_connections 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ecd410 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9eca910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3689237805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ec5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3689237805 >> 172.21.15.52:0/3689237805 conn(0x5581e9ebc680 msgr2=0x5581e9ebca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3689237805 shutdown_connections 2024-06-02T23:32:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3689237805 wait complete. 2024-06-02T23:32:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.300+0000 7f5cf79cd740 1 -- start start 2024-06-02T23:32:39.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 Processor -- start 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ecf330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ecf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ed68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5581e9ecdcf0 con 0x5581e9ecaf80 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5581e9dadae0 con 0x5581e9ec5e10 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581e9ecde70 con 0x5581e9ec5420 2024-06-02T23:32:39.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cf79cd740 10 client.0 _ll_register_callbacks cb 0x5581e9de69b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5ceed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ed68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5ceed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ed68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52704/0 (socket says 172.21.15.52:52704) 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5ceed9d640 1 -- 172.21.15.52:0/426732244 learned_addr learned my addr 172.21.15.52:0/426732244 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cee59c640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ecf330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.301+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ecf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5ceed9d640 1 -- 172.21.15.52:0/426732244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 msgr2=0x5581e9ecf330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ecf330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5ceed9d640 1 -- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 msgr2=0x5581e9ecf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ecf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5ceed9d640 1 -- 172.21.15.52:0/426732244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e9b38210 con 0x5581e9ecaf80 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ecf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ed68b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80047d0 tx=0x7f5ce8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=152a516a45e55fa6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.308 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/426732244 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8023420 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/426732244 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ce8014410 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce40bc680 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/426732244 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5ce8014610 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cf79cd740 10 client.20892 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.302+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5581e9c04620 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/426732244 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce40bc680 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/426732244 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce8010070 con 0x5581e9ecaf80 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cf79cd740 10 client.20892 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cf79cd740 10 client.20892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cf79cd740 20 client.20892 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9edb280 0x5581e9edd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cf79cd740 4 client.20892 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:39.309 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581e9e37f10 con 0x5581e9edb280 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.303+0000 7f5cee59c640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9edb280 0x5581e9edd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cee59c640 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9edb280 0x5581e9edd6b0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121e8875dca82859 server_cookie=7e1888b5010dece8 in_seq=0 out_seq=0 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cd5ffb640 10 client.20892 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/426732244 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5581e9e37f10 con 0x5581e9edb280 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cd5ffb640 10 client.20892 handle_command_reply: tid=0 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cf79cd740 1 client.20892 shutdown 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 msgr2=0x5581e9ed68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.304+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ed68b0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5ce80047d0 tx=0x7f5ce8004a40 comp rx=0 tx=0).stop 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 shutdown_connections 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9edb280 0x5581e9edd6b0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ed68b0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.310 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ecf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/426732244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ecf330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cd5ffb640 0 client.20892 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 >> 172.21.15.52:0/426732244 conn(0x5581e9ebc680 msgr2=0x5581e9ebb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 shutdown_connections 2024-06-02T23:32:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 1 -- 172.21.15.52:0/426732244 wait complete. 2024-06-02T23:32:39.311 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.305+0000 7f5cf79cd740 20 client.20892 trim_cache size 0 max 16384 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 Processor -- start 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 -- start start 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ee2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ee3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a6af0 con 0x5581e9ecaf80 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a8630 con 0x5581e9ec5e10 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.320+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a71c0 con 0x5581e9ec5420 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ee3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.322 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ee3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52706/0 (socket says 172.21.15.52:52706) 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 -- 172.21.15.52:0/2496613635 learned_addr learned my addr 172.21.15.52:0/2496613635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5cee59c640 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ee2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 -- 172.21.15.52:0/2496613635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 msgr2=0x5581e9ee2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ee2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 -- 172.21.15.52:0/2496613635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 msgr2=0x5581e9ed6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 -- 172.21.15.52:0/2496613635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581e9eddd30 con 0x5581e9ecaf80 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ee3500 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009c90 tx=0x7f5ce802ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33891d8f0fc3707f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5cd7fff640 1 -- 172.21.15.52:0/2496613635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8004000 con 0x5581e9ecaf80 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5cd7fff640 1 -- 172.21.15.52:0/2496613635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce8032070 con 0x5581e9ecaf80 2024-06-02T23:32:39.323 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.321+0000 7f5cd7fff640 1 -- 172.21.15.52:0/2496613635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce802e640 con 0x5581e9ecaf80 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 -- 172.21.15.52:0/2496613635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 msgr2=0x5581e9ee3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ee3500 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009c90 tx=0x7f5ce802ef70 comp rx=0 tx=0).stop 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 -- 172.21.15.52:0/2496613635 shutdown_connections 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ecaf80 0x5581e9ee3500 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/2496613635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ec5420 0x5581e9ee2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 -- 172.21.15.52:0/2496613635 >> 172.21.15.52:0/2496613635 conn(0x5581e9ebc680 msgr2=0x5581e9ec6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.327 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 -- 172.21.15.52:0/2496613635 shutdown_connections 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.322+0000 7f5cf79cd740 1 -- 172.21.15.52:0/2496613635 wait complete. 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 -- start start 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 Processor -- start 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 0x5581e9ed38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ecaf80 0x5581e9f43f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a7f60 con 0x5581e9ec5420 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a7890 con 0x5581e9ec5e10 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5cf01a4ad0 con 0x5581e9ecaf80 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cf79cd740 10 client.0 _ll_register_callbacks cb 0x7f5ce8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cee59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 0x5581e9ed38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cee59c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 0x5581e9ed38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52720/0 (socket says 172.21.15.52:52720) 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.323+0000 7f5cee59c640 1 -- 172.21.15.52:0/3729337837 learned_addr learned my addr 172.21.15.52:0/3729337837 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:39.328 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5ceed9d640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ecaf80 0x5581e9f43f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cee59c640 1 -- 172.21.15.52:0/3729337837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ecaf80 msgr2=0x5581e9f43f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cee59c640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ecaf80 0x5581e9f43f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cee59c640 1 -- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 msgr2=0x5581e9ed3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cee59c640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cee59c640 1 -- 172.21.15.52:0/3729337837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581e9ec6ae0 con 0x5581e9ec5420 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cee59c640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 0x5581e9ed38d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000be40 tx=0x7f5ce001d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8165eb9b0f08bc14 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cd5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/3729337837 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ce0011d80 con 0x5581e9ec5420 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ce8032070 con 0x5581e9ec5420 2024-06-02T23:32:39.329 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/3729337837 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5ce0005020 con 0x5581e9ec5420 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/3729337837 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5ce0005220 con 0x5581e9ec5420 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cf79cd740 10 client.20898 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.324+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5581e9ebc4b0 con 0x5581e9ec5420 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/3729337837 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5ce8032070 con 0x5581e9ec5420 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/3729337837 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5ce0025400 con 0x5581e9ec5420 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cf79cd740 10 client.20898 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cf79cd740 10 client.20898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cf79cd740 20 client.20898 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9f45f10 0x5581e9f48330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cf79cd740 4 client.20898 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581e9e31c20 con 0x5581e9f45f10 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.325+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9f45f10 0x5581e9f48330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.330 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.326+0000 7f5cd5ffb640 10 client.20898 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:39.331 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.326+0000 7f5cedd9b640 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9f45f10 0x5581e9f48330 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd1c24bd10f26673 server_cookie=491b056d7597ebbd in_seq=0 out_seq=0 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.348+0000 7f5cd5ffb640 1 -- 172.21.15.52:0/3729337837 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5581e9e31c20 con 0x5581e9f45f10 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.348+0000 7f5cd5ffb640 10 client.20898 handle_command_reply: tid=0 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.348+0000 7f5cf79cd740 1 client.20898 shutdown 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 msgr2=0x5581e9ed38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 0x5581e9ed38d0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f5ce000be40 tx=0x7f5ce001d9e0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 shutdown_connections 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5581e9f45f10 0x5581e9f48330 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.350 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581e9ecaf80 0x5581e9f43f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5581e9ec5e10 0x5581e9ed3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 --2- 172.21.15.52:0/3729337837 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5581e9ec5420 0x5581e9ed38d0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cd5ffb640 0 client.20898 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 >> 172.21.15.52:0/3729337837 conn(0x5581e9ebc680 msgr2=0x5581e9ec6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 shutdown_connections 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 1 -- 172.21.15.52:0/3729337837 wait complete. 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf79cd740 20 client.20898 trim_cache size 0 max 16384 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:32:39.351 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:32:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cd80a2c30 msgr2=0x7f5cd80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cd80a2c30 0x7f5cd80a50f0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x5581e9ec6670 tx=0x7f5ce8004ab0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 msgr2=0x7f5cf0198110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.352 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.349+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0198110 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f5ce40314e0 tx=0x7f5ce4004270 comp rx=0 tx=0).stop 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 shutdown_connections 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5cd80a2c30 0x7f5cd80a50f0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5cf0103680 0x7f5cf019f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5cf0102cc0 0x7f5cf0198110 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 --2- 172.21.15.52:0/247007813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cf00ffcf0 0x7f5cf0197bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 >> 172.21.15.52:0/247007813 conn(0x7f5cf00fb620 msgr2=0x7f5cf0105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 shutdown_connections 2024-06-02T23:32:39.353 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:39.350+0000 7f5cf4be3640 1 -- 172.21.15.52:0/247007813 wait complete. 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 -- 172.21.15.52:0/4258613303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 msgr2=0x7fd8841019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 --2- 172.21.15.52:0/4258613303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841019f0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fd8740099c0 tx=0x7fd874009cd0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 -- 172.21.15.52:0/4258613303 shutdown_connections 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 --2- 172.21.15.52:0/4258613303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8841066f0 0x7fd88410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 --2- 172.21.15.52:0/4258613303 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841019f0 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 --2- 172.21.15.52:0/4258613303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd884101fc0 0x7fd884106010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 -- 172.21.15.52:0/4258613303 >> 172.21.15.52:0/4258613303 conn(0x7fd884078430 msgr2=0x7fd884078840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 -- 172.21.15.52:0/4258613303 shutdown_connections 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 -- 172.21.15.52:0/4258613303 wait complete. 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 Processor -- start 2024-06-02T23:32:39.412 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.411+0000 7fd88a27d700 1 -- start start 2024-06-02T23:32:39.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88a27d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88a27d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd884101fc0 0x7fd884199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88a27d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8841066f0 0x7fd8841a4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88a27d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd884111580 con 0x7fd884101610 2024-06-02T23:32:39.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88a27d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd884111400 con 0x7fd884101fc0 2024-06-02T23:32:39.414 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88a27d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd884111700 con 0x7fd8841066f0 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52728/0 (socket says 172.21.15.52:52728) 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 -- 172.21.15.52:0/3400089020 learned_addr learned my addr 172.21.15.52:0/3400089020 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd8837fe700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd884101fc0 0x7fd884199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd88881a700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8841066f0 0x7fd8841a4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 -- 172.21.15.52:0/3400089020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8841066f0 msgr2=0x7fd8841a4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8841066f0 0x7fd8841a4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 -- 172.21.15.52:0/3400089020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd884101fc0 msgr2=0x7fd884199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd884101fc0 0x7fd884199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 -- 172.21.15.52:0/3400089020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8841a4fa0 con 0x7fd884101610 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.412+0000 7fd883fff700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841999e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd874006010 tx=0x7fd8740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.413+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874026070 con 0x7fd884101610 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.413+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8841a5170 con 0x7fd884101610 2024-06-02T23:32:39.415 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.413+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd87400f460 con 0x7fd884101610 2024-06-02T23:32:39.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.413+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd874009e60 con 0x7fd884101610 2024-06-02T23:32:39.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.413+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8841a5620 con 0x7fd884101610 2024-06-02T23:32:39.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.414+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd884069180 con 0x7fd884101610 2024-06-02T23:32:39.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.415+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd874029070 con 0x7fd884101610 2024-06-02T23:32:39.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.416+0000 7fd8817fa700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd86c0a2c20 0x7fd86c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:39.416 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.416+0000 7fd8837fe700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd86c0a2c20 0x7fd86c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:39.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.416+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd8740cfa90 con 0x7fd884101610 2024-06-02T23:32:39.417 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.416+0000 7fd8837fe700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd86c0a2c20 0x7fd86c0a50e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd884100ff0 tx=0x7fd87800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:39.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.419+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd874030080 con 0x7fd884101610 2024-06-02T23:32:39.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.73 MB/sec execute 347 sec latency 17531.131 ms 2024-06-02T23:32:39.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.682+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd88402dca0 con 0x7fd884101610 2024-06-02T23:32:39.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.682+0000 7fd8817fa700 1 -- 172.21.15.52:0/3400089020 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7fd874013070 con 0x7fd884101610 2024-06-02T23:32:39.683 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:32:39.683 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:32:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd86c0a2c20 msgr2=0x7fd86c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd86c0a2c20 0x7fd86c0a50e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd884100ff0 tx=0x7fd87800b410 comp rx=0 tx=0).stop 2024-06-02T23:32:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 msgr2=0x7fd8841999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841999e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fd874006010 tx=0x7fd8740045e0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 shutdown_connections 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd86c0a2c20 0x7fd86c0a50e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8841066f0 0x7fd8841a4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd884101610 0x7fd8841999e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 --2- 172.21.15.52:0/3400089020 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd884101fc0 0x7fd884199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 >> 172.21.15.52:0/3400089020 conn(0x7fd884078430 msgr2=0x7fd884105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 shutdown_connections 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:39.685+0000 7fd88a27d700 1 -- 172.21.15.52:0/3400089020 wait complete. 2024-06-02T23:32:39.686 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:32:40.006 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:39 smithi052 ceph-mon[32340]: pgmap v1054: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 259 KiB/s wr, 66 op/s 2024-06-02T23:32:40.007 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:39 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3400089020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:32:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:39 smithi174 ceph-mon[35994]: pgmap v1054: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 259 KiB/s wr, 66 op/s 2024-06-02T23:32:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:39 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3400089020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:32:40.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:39 smithi123 ceph-mon[35977]: pgmap v1054: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 259 KiB/s wr, 66 op/s 2024-06-02T23:32:40.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:39 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3400089020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:32:40.319 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id bbb6892 --release --await' 2024-06-02T23:32:40.319 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id bbb6892 --release --await 2024-06-02T23:32:40.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.71 MB/sec execute 348 sec latency 18531.184 ms 2024-06-02T23:32:40.572 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:41.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.70 MB/sec execute 349 sec latency 19531.273 ms 2024-06-02T23:32:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:41 smithi174 ceph-mon[35994]: pgmap v1055: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 192 KiB/s wr, 38 op/s 2024-06-02T23:32:42.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:41 smithi052 ceph-mon[32340]: pgmap v1055: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 192 KiB/s wr, 38 op/s 2024-06-02T23:32:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:41 smithi123 ceph-mon[35977]: pgmap v1055: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 192 KiB/s wr, 38 op/s 2024-06-02T23:32:42.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.68 MB/sec execute 350 sec latency 20531.380 ms 2024-06-02T23:32:43.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.66 MB/sec execute 351 sec latency 21531.523 ms 2024-06-02T23:32:43.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:43 smithi052 ceph-mon[32340]: pgmap v1056: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-06-02T23:32:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:43 smithi174 ceph-mon[35994]: pgmap v1056: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-06-02T23:32:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:43 smithi123 ceph-mon[35977]: pgmap v1056: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-06-02T23:32:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 -- 172.21.15.52:0/591756627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 msgr2=0x7fb90c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 --2- 172.21.15.52:0/591756627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c107110 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7fb9000099c0 tx=0x7fb900009cd0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 -- 172.21.15.52:0/591756627 shutdown_connections 2024-06-02T23:32:44.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 --2- 172.21.15.52:0/591756627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb90c107840 0x7fb90c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 --2- 172.21.15.52:0/591756627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb90c102740 0x7fb90c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 --2- 172.21.15.52:0/591756627 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c107110 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 -- 172.21.15.52:0/591756627 >> 172.21.15.52:0/591756627 conn(0x7fb90c0795f0 msgr2=0x7fb90c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 -- 172.21.15.52:0/591756627 shutdown_connections 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.465+0000 7fb912456700 1 -- 172.21.15.52:0/591756627 wait complete. 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 Processor -- start 2024-06-02T23:32:44.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 -- start start 2024-06-02T23:32:44.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb90c102740 0x7fb90c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb90c107840 0x7fb90c075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1126d0 con 0x7fb90c1030f0 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c112550 con 0x7fb90c107840 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb912456700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c112850 con 0x7fb90c102740 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40220/0 (socket says 172.21.15.52:40220) 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb90b7fe700 1 -- 172.21.15.52:0/583576902 learned_addr learned my addr 172.21.15.52:0/583576902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb9109f3700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb90c107840 0x7fb90c075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb90bfff700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb90c102740 0x7fb90c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb90b7fe700 1 -- 172.21.15.52:0/583576902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb90c102740 msgr2=0x7fb90c074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.466+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb90c102740 0x7fb90c074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb90b7fe700 1 -- 172.21.15.52:0/583576902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb90c107840 msgr2=0x7fb90c075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb90c107840 0x7fb90c075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb90b7fe700 1 -- 172.21.15.52:0/583576902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb90c1a70e0 con 0x7fb90c1030f0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c074cd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb900009990 tx=0x7fb9000047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb9097fa700 1 -- 172.21.15.52:0/583576902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900027070 con 0x7fb90c1030f0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb90c1a7370 con 0x7fb90c1030f0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb9097fa700 1 -- 172.21.15.52:0/583576902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb900004050 con 0x7fb90c1030f0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb9097fa700 1 -- 172.21.15.52:0/583576902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb900022dc0 con 0x7fb90c1030f0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.467+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb90c1a7820 con 0x7fb90c1030f0 2024-06-02T23:32:44.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.470+0000 7fb9097fa700 1 -- 172.21.15.52:0/583576902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb90002a070 con 0x7fb90c1030f0 2024-06-02T23:32:44.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 55900 6.64 MB/sec execute 352 sec latency 22531.574 ms 2024-06-02T23:32:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.471+0000 7fb9097fa700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8f40a2c20 0x7fb8f40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.471+0000 7fb90bfff700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8f40a2c20 0x7fb8f40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.471+0000 7fb9097fa700 1 -- 172.21.15.52:0/583576902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb9000cff70 con 0x7fb90c1030f0 2024-06-02T23:32:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.471+0000 7fb91ccd9b80 1 Processor -- start 2024-06-02T23:32:44.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.471+0000 7fb91ccd9b80 1 -- start start 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.471+0000 7fb90bfff700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8f40a2c20 0x7fb8f40a50e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006fd0 tx=0x7fb8fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a8685020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 0x5612a867ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a867f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5612a85d0910 con 0x5612a8685560 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612a8578820 con 0x5612a8684c40 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612a85cc8d0 con 0x5612a867f3d0 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 0x5612a867ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 0x5612a867ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40236/0 (socket says 172.21.15.52:40236) 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1405064851 learned_addr learned my addr 172.21.15.52:0/1405064851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:44.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a8685020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb9109f3700 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a867f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1405064851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 msgr2=0x5612a867f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a867f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1405064851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 msgr2=0x5612a8685020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a8685020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1405064851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a856f050 con 0x5612a8685560 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 0x5612a867ee90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb9000a4f40 tx=0x7fb900009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841bbb7c60f6f8e5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.472+0000 7fb8f27fc700 1 -- 172.21.15.52:0/1405064851 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90009c890 con 0x5612a8685560 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb8f27fc700 1 -- 172.21.15.52:0/1405064851 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9000a0c40 con 0x5612a8685560 2024-06-02T23:32:44.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb8f27fc700 1 -- 172.21.15.52:0/1405064851 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90009caf0 con 0x5612a8685560 2024-06-02T23:32:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1405064851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 msgr2=0x5612a867ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 0x5612a867ee90 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fb9000a4f40 tx=0x7fb900009310 comp rx=0 tx=0).stop 2024-06-02T23:32:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1405064851 shutdown_connections 2024-06-02T23:32:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a867f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.479 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a8685020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1405064851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8685560 0x5612a867ee90 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1405064851 >> 172.21.15.52:0/1405064851 conn(0x5612a85a9dc0 msgr2=0x5612a867a120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1405064851 shutdown_connections 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.473+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1405064851 wait complete. 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 -- start start 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 Processor -- start 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a868b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a86885c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8685560 0x5612a8688b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5612a868bcf0 con 0x5612a867f3d0 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612a8570cf0 con 0x5612a8684c40 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5612a868be70 con 0x5612a8685560 2024-06-02T23:32:44.480 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb91ccd9b80 10 client.0 _ll_register_callbacks cb 0x5612a856aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a868b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a868b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40242/0 (socket says 172.21.15.52:40242) 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb90bfff700 1 -- 172.21.15.52:0/2118872086 learned_addr learned my addr 172.21.15.52:0/2118872086 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a86885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.474+0000 7fb9109f3700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8685560 0x5612a8688b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb90bfff700 1 -- 172.21.15.52:0/2118872086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8685560 msgr2=0x5612a8688b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb90bfff700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8685560 0x5612a8688b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb90bfff700 1 -- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 msgr2=0x5612a86885c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb90bfff700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a86885c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb90bfff700 1 -- 172.21.15.52:0/2118872086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a851f9d0 con 0x5612a867f3d0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb90bfff700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a868b2c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00deb0 tx=0x7fb8fc004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12458522db736c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb8eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:44.481 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/2118872086 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc024020 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/2118872086 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb8fc00f040 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9000a0c40 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/2118872086 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb8fc014e60 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb91ccd9b80 10 client.20916 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5612a8696650 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.475+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/2118872086 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb9000a0c40 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/2118872086 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb8fc013070 con 0x5612a867f3d0 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb91ccd9b80 10 client.20916 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb91ccd9b80 10 client.20916 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb91ccd9b80 20 client.20916 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a8698c90 0x5612a869b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.482 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb91ccd9b80 4 client.20916 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612a85dde10 con 0x5612a8698c90 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a8698c90 0x5612a869b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb8eb7fe700 10 client.20916 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.476+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a8698c90 0x5612a869b0b0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73f32b3ef3a20e6 server_cookie=276e5ff7e1040fc9 in_seq=0 out_seq=0 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/2118872086 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5612a85dde10 con 0x5612a8698c90 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb8eb7fe700 10 client.20916 handle_command_reply: tid=0 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 client.20916 shutdown 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 msgr2=0x5612a868b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a868b2c0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc00deb0 tx=0x7fb8fc004860 comp rx=0 tx=0).stop 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 shutdown_connections 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a8698c90 0x5612a869b0b0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8685560 0x5612a8688b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a868b2c0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.483 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/2118872086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8684c40 0x5612a86885c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 >> 172.21.15.52:0/2118872086 conn(0x5612a85a9dc0 msgr2=0x5612a867de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.477+0000 7fb8eb7fe700 0 client.20916 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.478+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 shutdown_connections 2024-06-02T23:32:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.478+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/2118872086 wait complete. 2024-06-02T23:32:44.484 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.478+0000 7fb91ccd9b80 20 client.20916 trim_cache size 0 max 16384 2024-06-02T23:32:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.498+0000 7fb91ccd9b80 1 Processor -- start 2024-06-02T23:32:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.498+0000 7fb91ccd9b80 1 -- start start 2024-06-02T23:32:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.498+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a867ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.498+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8684c40 0x5612a86860b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.498+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a86a0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.498+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1aec30 con 0x5612a867f3d0 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1b0770 con 0x5612a8685560 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1af300 con 0x5612a8684c40 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a867ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a867ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40248/0 (socket says 172.21.15.52:40248) 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 -- 172.21.15.52:0/1928466787 learned_addr learned my addr 172.21.15.52:0/1928466787 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8684c40 0x5612a86860b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb9109f3700 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a86a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 -- 172.21.15.52:0/1928466787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8684c40 msgr2=0x5612a86860b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8684c40 0x5612a86860b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 -- 172.21.15.52:0/1928466787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 msgr2=0x5612a86a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a86a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 -- 172.21.15.52:0/1928466787 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612a867b5d0 con 0x5612a867f3d0 2024-06-02T23:32:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a867ee40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006dc0 tx=0x7fb8fc009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf8892e2e0df681 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb8f1ffb700 1 -- 172.21.15.52:0/1928466787 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc039070 con 0x5612a867f3d0 2024-06-02T23:32:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb8f1ffb700 1 -- 172.21.15.52:0/1928466787 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc020b60 con 0x5612a867f3d0 2024-06-02T23:32:44.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.499+0000 7fb8f1ffb700 1 -- 172.21.15.52:0/1928466787 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc024070 con 0x5612a867f3d0 2024-06-02T23:32:44.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1928466787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 msgr2=0x5612a867ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a867ee40 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006dc0 tx=0x7fb8fc009640 comp rx=0 tx=0).stop 2024-06-02T23:32:44.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1928466787 shutdown_connections 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a86a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a867f3d0 0x5612a867ee40 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1928466787 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a8684c40 0x5612a86860b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1928466787 >> 172.21.15.52:0/1928466787 conn(0x5612a85a9dc0 msgr2=0x5612a86859a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1928466787 shutdown_connections 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.500+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1928466787 wait complete. 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 -- start start 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 Processor -- start 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a86913c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 0x5612a8691900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a868e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1b00a0 con 0x5612a8684c40 2024-06-02T23:32:44.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1af9d0 con 0x5612a8685560 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb90c1acbf0 con 0x5612a867f3d0 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb91ccd9b80 10 client.0 _ll_register_callbacks cb 0x5612a856aa20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 0x5612a8691900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb90b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 0x5612a8691900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40250/0 (socket says 172.21.15.52:40250) 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1287364237 learned_addr learned my addr 172.21.15.52:0/1287364237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb9109f3700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a868e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.501+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a86913c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1287364237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 msgr2=0x5612a86913c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a86913c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 msgr2=0x5612a868e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a868e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb90b7fe700 1 -- 172.21.15.52:0/1287364237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612a868e950 con 0x5612a8684c40 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb90b7fe700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 0x5612a8691900 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb900009fd0 tx=0x7fb900009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4f353a2b2250adc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:44.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb8eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/1287364237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9000b4070 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/1287364237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb9000bb1c0 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc020b60 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/1287364237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb90009c1f0 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb91ccd9b80 10 client.20922 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.502+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5612a8704520 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/1287364237 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb8fc020b60 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/1287364237 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb90009caf0 con 0x5612a8684c40 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb91ccd9b80 10 client.20922 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb91ccd9b80 10 client.20922 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb91ccd9b80 20 client.20922 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a87049e0 0x5612a8706e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:44.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb91ccd9b80 4 client.20922 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bbb6892", "release": true, "await": true}] 2024-06-02T23:32:44.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bbb6892", "release": true, "await": true}) -- 0x5612a85d8390 con 0x5612a87049e0 2024-06-02T23:32:44.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a87049e0 0x5612a8706e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:44.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb90bfff700 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a87049e0 0x5612a8706e00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41919e927872266 server_cookie=fd792ca3c6e2a98b in_seq=0 out_seq=0 2024-06-02T23:32:44.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.503+0000 7fb8eb7fe700 10 client.20922 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:44.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb8eb7fe700 1 -- 172.21.15.52:0/1287364237 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5612a85d8390 con 0x5612a87049e0 2024-06-02T23:32:44.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb8eb7fe700 10 client.20922 handle_command_reply: tid=0 2024-06-02T23:32:44.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 client.20922 shutdown 2024-06-02T23:32:44.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 msgr2=0x5612a8691900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.540 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 0x5612a8691900 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fb900009fd0 tx=0x7fb900009710 comp rx=0 tx=0).stop 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 shutdown_connections 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5612a87049e0 0x5612a8706e00 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5612a8685560 0x5612a868e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5612a867f3d0 0x5612a86913c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 --2- 172.21.15.52:0/1287364237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5612a8684c40 0x5612a8691900 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 >> 172.21.15.52:0/1287364237 conn(0x5612a85a9dc0 msgr2=0x5612a8679830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.539+0000 7fb8eb7fe700 0 client.20922 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.540+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 shutdown_connections 2024-06-02T23:32:44.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.540+0000 7fb91ccd9b80 1 -- 172.21.15.52:0/1287364237 wait complete. 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.540+0000 7fb91ccd9b80 20 client.20922 trim_cache size 0 max 16384 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 68, 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "bbb6892": { 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 68, 2024-06-02T23:32:44.542 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 147.4, 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:44.543 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:44.544 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:44.544 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:44.544 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:32:44.544 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:32:44.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8f40a2c20 msgr2=0x7fb8f40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8f40a2c20 0x7fb8f40a50e0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc006fd0 tx=0x7fb8fc008040 comp rx=0 tx=0).stop 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 msgr2=0x7fb90c074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c074cd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fb900009990 tx=0x7fb9000047b0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 shutdown_connections 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb8f40a2c20 0x7fb8f40a50e0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb90c107840 0x7fb90c075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb90c102740 0x7fb90c074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 --2- 172.21.15.52:0/583576902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb90c1030f0 0x7fb90c074cd0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 >> 172.21.15.52:0/583576902 conn(0x7fb90c0795f0 msgr2=0x7fb90c100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 shutdown_connections 2024-06-02T23:32:44.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:32:44.541+0000 7fb912456700 1 -- 172.21.15.52:0/583576902 wait complete. 2024-06-02T23:32:45.238 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bbb6892', total seconds elapsed: 35.3 2024-06-02T23:32:45.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56055 6.65 MB/sec execute 353 sec latency 22587.588 ms 2024-06-02T23:32:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:45 smithi174 ceph-mon[35994]: pgmap v1057: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-06-02T23:32:46.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:45 smithi052 ceph-mon[32340]: pgmap v1057: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-06-02T23:32:46.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:45 smithi123 ceph-mon[35977]: pgmap v1057: 97 pgs: 97 active+clean; 505 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-06-02T23:32:46.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56222 6.66 MB/sec execute 354 sec latency 57.296 ms 2024-06-02T23:32:47.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56395 6.67 MB/sec execute 355 sec latency 59.423 ms 2024-06-02T23:32:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:47 smithi174 ceph-mon[35994]: pgmap v1058: 97 pgs: 97 active+clean; 506 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 192 KiB/s wr, 26 op/s 2024-06-02T23:32:48.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:47 smithi052 ceph-mon[32340]: pgmap v1058: 97 pgs: 97 active+clean; 506 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 192 KiB/s wr, 26 op/s 2024-06-02T23:32:48.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:47 smithi123 ceph-mon[35977]: pgmap v1058: 97 pgs: 97 active+clean; 506 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 192 KiB/s wr, 26 op/s 2024-06-02T23:32:48.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56530 6.67 MB/sec execute 356 sec latency 55.714 ms 2024-06-02T23:32:49.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56666 6.66 MB/sec execute 357 sec latency 56.720 ms 2024-06-02T23:32:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:49 smithi174 ceph-mon[35994]: pgmap v1059: 97 pgs: 97 active+clean; 508 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 499 KiB/s wr, 52 op/s 2024-06-02T23:32:50.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:49 smithi052 ceph-mon[32340]: pgmap v1059: 97 pgs: 97 active+clean; 508 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 499 KiB/s wr, 52 op/s 2024-06-02T23:32:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:49 smithi123 ceph-mon[35977]: pgmap v1059: 97 pgs: 97 active+clean; 508 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 122 KiB/s rd, 499 KiB/s wr, 52 op/s 2024-06-02T23:32:50.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56824 6.65 MB/sec execute 358 sec latency 64.127 ms 2024-06-02T23:32:51.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 56998 6.65 MB/sec execute 359 sec latency 151.818 ms 2024-06-02T23:32:52.059 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:51 smithi123 ceph-mon[35977]: pgmap v1060: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-06-02T23:32:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:51 smithi174 ceph-mon[35994]: pgmap v1060: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-06-02T23:32:52.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:51 smithi052 ceph-mon[32340]: pgmap v1060: 97 pgs: 97 active+clean; 511 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 1.2 MiB/s wr, 70 op/s 2024-06-02T23:32:52.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 57183 6.65 MB/sec execute 360 sec latency 83.483 ms 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 -- 172.21.15.52:0/482122572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180107720 msgr2=0x7f9180109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 --2- 172.21.15.52:0/482122572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180107720 0x7f9180109b70 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9178009ca0 tx=0x7f917802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 -- 172.21.15.52:0/482122572 shutdown_connections 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 --2- 172.21.15.52:0/482122572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180107720 0x7f9180109b70 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 --2- 172.21.15.52:0/482122572 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9180104df0 0x7f91801071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 --2- 172.21.15.52:0/482122572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91801024c0 0x7f91801048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 -- 172.21.15.52:0/482122572 >> 172.21.15.52:0/482122572 conn(0x7f91800fbeb0 msgr2=0x7f91800fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 -- 172.21.15.52:0/482122572 shutdown_connections 2024-06-02T23:32:52.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.486+0000 7f91859f4640 1 -- 172.21.15.52:0/482122572 wait complete. 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 Processor -- start 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 -- start start 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91801024c0 0x7f9180199cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180104df0 0x7f918019a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 0x7f91801a1250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d5c0 con 0x7f91801024c0 2024-06-02T23:32:52.489 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d440 con 0x7f9180107720 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f91859f4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f918010d740 con 0x7f9180104df0 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f917effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91801024c0 0x7f9180199cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f917f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 0x7f91801a1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f917f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 0x7f91801a1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:47140/0 (socket says 172.21.15.52:47140) 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f9177fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180104df0 0x7f918019a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.487+0000 7f917f7fe640 1 -- 172.21.15.52:0/2285611634 learned_addr learned my addr 172.21.15.52:0/2285611634 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f917f7fe640 1 -- 172.21.15.52:0/2285611634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180104df0 msgr2=0x7f918019a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f917f7fe640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180104df0 0x7f918019a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f917f7fe640 1 -- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91801024c0 msgr2=0x7f9180199cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f917f7fe640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91801024c0 0x7f9180199cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f917f7fe640 1 -- 172.21.15.52:0/2285611634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91801a18f0 con 0x7f9180107720 2024-06-02T23:32:52.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f917f7fe640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 0x7f91801a1250 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9178031040 tx=0x7f9178003ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f91849f2640 1 -- 172.21.15.52:0/2285611634 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178032ec0 con 0x7f9180107720 2024-06-02T23:32:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91801a1b80 con 0x7f9180107720 2024-06-02T23:32:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f91849f2640 1 -- 172.21.15.52:0/2285611634 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91780044f0 con 0x7f9180107720 2024-06-02T23:32:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f91849f2640 1 -- 172.21.15.52:0/2285611634 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178030bd0 con 0x7f9180107720 2024-06-02T23:32:52.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.488+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91801a2120 con 0x7f9180107720 2024-06-02T23:32:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.491+0000 7f91849f2640 1 -- 172.21.15.52:0/2285611634 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9178004040 con 0x7f9180107720 2024-06-02T23:32:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 Processor -- start 2024-06-02T23:32:52.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 -- start start 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0e010 0x55e880e0e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e13500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e13b70 0x55e880e16000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e880d714f0 con 0x55e880e0ea00 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e880d0f870 con 0x55e880e0e010 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.492+0000 7f91887d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e880d6d4b0 con 0x55e880e13b70 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e13500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e13500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40272/0 (socket says 172.21.15.52:40272) 2024-06-02T23:32:52.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 -- 172.21.15.52:0/1213540113 learned_addr learned my addr 172.21.15.52:0/1213540113 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f9177fff640 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0e010 0x55e880e0e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f9177fff640 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e13b70 0x55e880e16000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f91849f2640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f916c0a2ed0 0x7f916c0a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 -- 172.21.15.52:0/1213540113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e13b70 msgr2=0x55e880e16000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e13b70 0x55e880e16000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 -- 172.21.15.52:0/1213540113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0e010 msgr2=0x55e880e0e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0e010 0x55e880e0e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 -- 172.21.15.52:0/1213540113 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e880d2b9b0 con 0x55e880e0ea00 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917f7fe640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f916c0a2ed0 0x7f916c0a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f91849f2640 1 -- 172.21.15.52:0/2285611634 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f91780f06f0 con 0x7f9180107720 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.493+0000 7f917effd640 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e13500 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f917000ac80 tx=0x7f917000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90d5f93a5c655a85 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.494+0000 7f9175ffb640 1 -- 172.21.15.52:0/1213540113 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917001d070 con 0x55e880e0ea00 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.494+0000 7f9175ffb640 1 -- 172.21.15.52:0/1213540113 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9170018af0 con 0x55e880e0ea00 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.494+0000 7f9175ffb640 1 -- 172.21.15.52:0/1213540113 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170010b10 con 0x55e880e0ea00 2024-06-02T23:32:52.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.494+0000 7f917f7fe640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f916c0a2ed0 0x7f916c0a5390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f91780bcc20 tx=0x7f9178031a60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 -- 172.21.15.52:0/1213540113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 msgr2=0x55e880e13500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e13500 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f917000ac80 tx=0x7f917000f790 comp rx=0 tx=0).stop 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 -- 172.21.15.52:0/1213540113 shutdown_connections 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e13b70 0x55e880e16000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e13500 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 --2- 172.21.15.52:0/1213540113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0e010 0x55e880e0e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 -- 172.21.15.52:0/1213540113 >> 172.21.15.52:0/1213540113 conn(0x55e880e09680 msgr2=0x55e880e09a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.495+0000 7f91887d6740 1 -- 172.21.15.52:0/1213540113 shutdown_connections 2024-06-02T23:32:52.498 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 -- 172.21.15.52:0/1213540113 wait complete. 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 -- start start 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 Processor -- start 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0e010 0x55e880e182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 0x55e880e187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e880e16b20 con 0x55e880e0e010 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e880cfaae0 con 0x55e880e13b70 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.496+0000 7f91887d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e880e16ca0 con 0x55e880e0ea00 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f9177fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0e010 0x55e880e182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 0x55e880e187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f91887d6740 10 client.0 _ll_register_callbacks cb 0x55e880d339b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 0x55e880e187f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54210/0 (socket says 172.21.15.52:54210) 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 -- 172.21.15.52:0/3224419901 learned_addr learned my addr 172.21.15.52:0/3224419901 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:52.499 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 -- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 msgr2=0x55e880e1f830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1f830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 -- 172.21.15.52:0/3224419901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0e010 msgr2=0x55e880e182b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0e010 0x55e880e182b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 -- 172.21.15.52:0/3224419901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e880a85210 con 0x55e880e0ea00 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f9177fff640 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0e010 0x55e880e182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f917effd640 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 0x55e880e187f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f917001a000 tx=0x7f917000aa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb931a6e5f192e9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.497+0000 7f915effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f915effd640 1 -- 172.21.15.52:0/3224419901 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9170027030 con 0x55e880e0ea00 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9170018af0 con 0x55e880e0ea00 2024-06-02T23:32:52.500 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f915effd640 1 -- 172.21.15.52:0/3224419901 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9170005040 con 0x55e880e0ea00 2024-06-02T23:32:52.502 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f915effd640 1 -- 172.21.15.52:0/3224419901 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f917001d070 con 0x55e880e0ea00 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f91887d6740 10 client.29851 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e880b51620 con 0x55e880e0ea00 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f915effd640 1 -- 172.21.15.52:0/3224419901 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9170018af0 con 0x55e880e0ea00 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.498+0000 7f915effd640 1 -- 172.21.15.52:0/3224419901 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f91700046c0 con 0x55e880e0ea00 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f91887d6740 10 client.29851 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f91887d6740 10 client.29851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f91887d6740 20 client.29851 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f91887d6740 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e23dd0 0x55e880e26200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f91887d6740 4 client.29851 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e880d84f10 con 0x55e880e23dd0 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f9177fff640 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e23dd0 0x55e880e26200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.503 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f9177fff640 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e23dd0 0x55e880e26200 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116e659887ea524 server_cookie=8269d8b05c7d0430 in_seq=0 out_seq=0 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.499+0000 7f915effd640 10 client.29851 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.500+0000 7f915effd640 1 -- 172.21.15.52:0/3224419901 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e880d84f10 con 0x55e880e23dd0 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.500+0000 7f915effd640 10 client.29851 handle_command_reply: tid=0 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.500+0000 7f91887d6740 1 client.29851 shutdown 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.500+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 msgr2=0x55e880e187f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.500+0000 7f91887d6740 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 0x55e880e187f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f917001a000 tx=0x7f917000aa00 comp rx=0 tx=0).stop 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.500+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 shutdown_connections 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e23dd0 0x55e880e26200 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0ea00 0x55e880e187f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 --2- 172.21.15.52:0/3224419901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0e010 0x55e880e182b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 >> 172.21.15.52:0/3224419901 conn(0x55e880e09680 msgr2=0x55e880e087e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f915effd640 0 client.29851 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 shutdown_connections 2024-06-02T23:32:52.504 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 1 -- 172.21.15.52:0/3224419901 wait complete. 2024-06-02T23:32:52.505 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.501+0000 7f91887d6740 20 client.29851 trim_cache size 0 max 16384 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.522+0000 7f91887d6740 1 Processor -- start 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.522+0000 7f91887d6740 1 -- start start 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f91887d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e0fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f91887d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0ea00 0x55e880e10120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f91887d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e13b70 0x55e880e1f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f91887d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91801aae00 con 0x55e880e13b70 2024-06-02T23:32:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f91887d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ac940 con 0x55e880e0ea00 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f91887d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ab4d0 con 0x55e880e0e010 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e0fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e0fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54226/0 (socket says 172.21.15.52:54226) 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 -- 172.21.15.52:0/3550972033 learned_addr learned my addr 172.21.15.52:0/3550972033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e13b70 0x55e880e1f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f917effd640 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0ea00 0x55e880e10120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 -- 172.21.15.52:0/3550972033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0ea00 msgr2=0x55e880e10120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0ea00 0x55e880e10120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 -- 172.21.15.52:0/3550972033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e13b70 msgr2=0x55e880e1f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e13b70 0x55e880e1f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 -- 172.21.15.52:0/3550972033 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e880e192a0 con 0x55e880e0e010 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.523+0000 7f9177fff640 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e0fbe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9168009730 tx=0x7f9168004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e174442baa81b078 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.524+0000 7f91757fa640 1 -- 172.21.15.52:0/3550972033 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916800fdf0 con 0x55e880e0e010 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.524+0000 7f91757fa640 1 -- 172.21.15.52:0/3550972033 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91680054d0 con 0x55e880e0e010 2024-06-02T23:32:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.524+0000 7f91757fa640 1 -- 172.21.15.52:0/3550972033 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168004920 con 0x55e880e0e010 2024-06-02T23:32:52.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- 172.21.15.52:0/3550972033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 msgr2=0x55e880e0fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e0fbe0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9168009730 tx=0x7f9168004730 comp rx=0 tx=0).stop 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- 172.21.15.52:0/3550972033 shutdown_connections 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e13b70 0x55e880e1f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e0ea00 0x55e880e10120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- 172.21.15.52:0/3550972033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e0fbe0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- 172.21.15.52:0/3550972033 >> 172.21.15.52:0/3550972033 conn(0x55e880e09680 msgr2=0x55e880e13240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- 172.21.15.52:0/3550972033 shutdown_connections 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- 172.21.15.52:0/3550972033 wait complete. 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- start start 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 Processor -- start 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e1f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e1bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f91801ac270 con 0x55e880e0ea00 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91801abba0 con 0x55e880e13b70 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91801a8de0 con 0x55e880e0e010 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f91887d6740 10 client.0 _ll_register_callbacks cb 0x7f917001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.525+0000 7f917effd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e1bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e1f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e1f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54238/0 (socket says 172.21.15.52:54238) 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 -- 172.21.15.52:0/3413009761 learned_addr learned my addr 172.21.15.52:0/3413009761 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 -- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 msgr2=0x55e880e1c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 -- 172.21.15.52:0/3413009761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 msgr2=0x55e880e1bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:32:52.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e1bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 -- 172.21.15.52:0/3413009761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e880e1f940 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f917effd640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e1bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f9177fff640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e1f310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9168005790 tx=0x7f9168013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=639c4b9a45df56ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f915effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f915effd640 1 -- 172.21.15.52:0/3413009761 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9168037460 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f915effd640 1 -- 172.21.15.52:0/3413009761 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9168024040 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.526+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91680054d0 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.527+0000 7f915effd640 1 -- 172.21.15.52:0/3413009761 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9168037600 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.527+0000 7f91887d6740 10 client.29857 fetch_fsmap learned FSMap version 43 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.527+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e880e8d960 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.527+0000 7f915effd640 1 -- 172.21.15.52:0/3413009761 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f91680054d0 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.527+0000 7f915effd640 1 -- 172.21.15.52:0/3413009761 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9168035040 con 0x55e880e0e010 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.527+0000 7f91887d6740 10 client.29857 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f91887d6740 10 client.29857 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:32:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f91887d6740 20 client.29857 populate_metadata read hostname 'smithi052' 2024-06-02T23:32:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f91887d6740 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e8e2d0 0x55e880e906f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:32:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f91887d6740 4 client.29857 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:32:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e880d7ec20 con 0x55e880e8e2d0 2024-06-02T23:32:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f917effd640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e8e2d0 0x55e880e906f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:32:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f915effd640 10 client.29857 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.528+0000 7f917effd640 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e8e2d0 0x55e880e906f0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7ef3358126da94c server_cookie=c42c5017e1ce8701 in_seq=0 out_seq=0 2024-06-02T23:32:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f915effd640 1 -- 172.21.15.52:0/3413009761 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e880d7ec20 con 0x55e880e8e2d0 2024-06-02T23:32:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f915effd640 10 client.29857 handle_command_reply: tid=0 2024-06-02T23:32:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f91887d6740 1 client.29857 shutdown 2024-06-02T23:32:52.577 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 msgr2=0x55e880e1f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f91887d6740 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e1f310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9168005790 tx=0x7f9168013040 comp rx=0 tx=0).stop 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 shutdown_connections 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.575+0000 7f91887d6740 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e880e8e2d0 0x55e880e906f0 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e880e13b70 0x55e880e1c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e880e0ea00 0x55e880e1bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 1 --2- 172.21.15.52:0/3413009761 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e880e0e010 0x55e880e1f310 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.578 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f915effd640 0 client.29857 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:32:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 >> 172.21.15.52:0/3413009761 conn(0x55e880e09680 msgr2=0x55e880e0fbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 shutdown_connections 2024-06-02T23:32:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 1 -- 172.21.15.52:0/3413009761 wait complete. 2024-06-02T23:32:52.579 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.576+0000 7f91887d6740 20 client.29857 trim_cache size 0 max 16384 2024-06-02T23:32:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:32:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:32:52.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:32:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f916c0a2ed0 msgr2=0x7f916c0a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f916c0a2ed0 0x7f916c0a5390 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f91780bcc20 tx=0x7f9178031a60 comp rx=0 tx=0).stop 2024-06-02T23:32:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 msgr2=0x7f91801a1250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:32:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 0x7f91801a1250 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f9178031040 tx=0x7f9178003ba0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 shutdown_connections 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f916c0a2ed0 0x7f916c0a5390 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9180104df0 0x7f918019a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9180107720 0x7f91801a1250 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 --2- 172.21.15.52:0/2285611634 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f91801024c0 0x7f9180199cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.577+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 >> 172.21.15.52:0/2285611634 conn(0x7f91800fbeb0 msgr2=0x7f91801019f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.578+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 shutdown_connections 2024-06-02T23:32:52.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:32:52.578+0000 7f91859f4640 1 -- 172.21.15.52:0/2285611634 wait complete. 2024-06-02T23:32:53.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 57329 6.66 MB/sec execute 361 sec latency 95.538 ms 2024-06-02T23:32:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:53 smithi174 ceph-mon[35994]: pgmap v1061: 97 pgs: 97 active+clean; 513 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-06-02T23:32:54.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:53 smithi052 ceph-mon[32340]: pgmap v1061: 97 pgs: 97 active+clean; 513 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-06-02T23:32:54.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:53 smithi123 ceph-mon[35977]: pgmap v1061: 97 pgs: 97 active+clean; 513 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 1.8 MiB/s wr, 132 op/s 2024-06-02T23:32:54.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 57489 6.67 MB/sec execute 362 sec latency 61.212 ms 2024-06-02T23:32:55.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 57703 6.70 MB/sec execute 363 sec latency 61.101 ms 2024-06-02T23:32:56.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:55 smithi174 ceph-mon[35994]: pgmap v1062: 97 pgs: 97 active+clean; 512 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 139 op/s 2024-06-02T23:32:56.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:55 smithi052 ceph-mon[32340]: pgmap v1062: 97 pgs: 97 active+clean; 512 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 139 op/s 2024-06-02T23:32:56.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:55 smithi123 ceph-mon[35977]: pgmap v1062: 97 pgs: 97 active+clean; 512 MiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 574 KiB/s rd, 1.9 MiB/s wr, 139 op/s 2024-06-02T23:32:56.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 57915 6.71 MB/sec execute 364 sec latency 139.096 ms 2024-06-02T23:32:57.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 58066 6.70 MB/sec execute 365 sec latency 68.717 ms 2024-06-02T23:32:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:57 smithi174 ceph-mon[35994]: pgmap v1063: 97 pgs: 97 active+clean; 506 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.2 MiB/s wr, 237 op/s 2024-06-02T23:32:58.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:57 smithi052 ceph-mon[32340]: pgmap v1063: 97 pgs: 97 active+clean; 506 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.2 MiB/s wr, 237 op/s 2024-06-02T23:32:58.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:57 smithi123 ceph-mon[35977]: pgmap v1063: 97 pgs: 97 active+clean; 506 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 917 KiB/s rd, 3.2 MiB/s wr, 237 op/s 2024-06-02T23:32:58.463 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:32:58.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 58223 6.69 MB/sec execute 366 sec latency 58.127 ms 2024-06-02T23:32:58.854 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:32:59.471 INFO:tasks.workunit.client.0.smithi052.stdout: 10 58397 6.69 MB/sec execute 367 sec latency 68.124 ms 2024-06-02T23:33:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:32:59 smithi174 ceph-mon[35994]: pgmap v1064: 97 pgs: 97 active+clean; 504 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2024-06-02T23:33:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:32:59 smithi052 ceph-mon[32340]: pgmap v1064: 97 pgs: 97 active+clean; 504 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2024-06-02T23:33:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:32:59 smithi123 ceph-mon[35977]: pgmap v1064: 97 pgs: 97 active+clean; 504 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.4 MiB/s wr, 263 op/s 2024-06-02T23:33:00.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 58569 6.70 MB/sec execute 368 sec latency 66.418 ms 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 -- 172.21.15.52:0/2388603697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0006a1a0 msgr2=0x7f2f0006a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 --2- 172.21.15.52:0/2388603697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0006a1a0 0x7f2f0006a580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2ef00098f0 tx=0x7f2ef0009c00 comp rx=0 tx=0).stop 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 -- 172.21.15.52:0/2388603697 shutdown_connections 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 --2- 172.21.15.52:0/2388603697 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0010ef40 0x7f2f00111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 --2- 172.21.15.52:0/2388603697 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0006a1a0 0x7f2f0006a580 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 --2- 172.21.15.52:0/2388603697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0006aac0 0x7f2f0010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.135 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.134+0000 7f2f05994700 1 -- 172.21.15.52:0/2388603697 >> 172.21.15.52:0/2388603697 conn(0x7f2f00078340 msgr2=0x7f2f00078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.135+0000 7f2f05994700 1 -- 172.21.15.52:0/2388603697 shutdown_connections 2024-06-02T23:33:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.135+0000 7f2f05994700 1 -- 172.21.15.52:0/2388603697 wait complete. 2024-06-02T23:33:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.135+0000 7f2f05994700 1 Processor -- start 2024-06-02T23:33:01.136 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 -- start start 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0006a1a0 0x7f2f0019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 0x7f2f0019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0010ef40 0x7f2f0019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2f00114860 con 0x7f2f0006aac0 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001146e0 con 0x7f2f0006a1a0 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2f05994700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2f001149e0 con 0x7f2f0010ef40 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 0x7f2f0019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 0x7f2f0019b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40200/0 (socket says 172.21.15.52:40200) 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efeffd700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0006a1a0 0x7f2f0019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 -- 172.21.15.52:0/49585278 learned_addr learned my addr 172.21.15.52:0/49585278 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2eff7fe700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0010ef40 0x7f2f0019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 -- 172.21.15.52:0/49585278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0010ef40 msgr2=0x7f2f0019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0010ef40 0x7f2f0019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.138 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 -- 172.21.15.52:0/49585278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0006a1a0 msgr2=0x7f2f0019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0006a1a0 0x7f2f0019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.136+0000 7f2efe7fc700 1 -- 172.21.15.52:0/49585278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f0019fe80 con 0x7f2f0006aac0 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.137+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 0x7f2f0019b3b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400eb30 tx=0x7f2ef400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.137+0000 7f2f04992700 1 -- 172.21.15.52:0/49585278 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4004220 con 0x7f2f0006aac0 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.137+0000 7f2f04992700 1 -- 172.21.15.52:0/49585278 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef400ce00 con 0x7f2f0006aac0 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.137+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f001a0170 con 0x7f2f0006aac0 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.137+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f001035a0 con 0x7f2f0006aac0 2024-06-02T23:33:01.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.138+0000 7f2f04992700 1 -- 172.21.15.52:0/49585278 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4003d50 con 0x7f2f0006aac0 2024-06-02T23:33:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.140+0000 7f2f04992700 1 -- 172.21.15.52:0/49585278 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2ef40043c0 con 0x7f2f0006aac0 2024-06-02T23:33:01.141 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.141+0000 7f2f04992700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ee80a2c20 0x7f2ee80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.141+0000 7f2f04992700 1 -- 172.21.15.52:0/49585278 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ef40cf170 con 0x7f2f0006aac0 2024-06-02T23:33:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.141+0000 7f2efeffd700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ee80a2c20 0x7f2ee80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.142+0000 7f2f10217b80 1 Processor -- start 2024-06-02T23:33:01.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.142+0000 7f2efeffd700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ee80a2c20 0x7f2ee80a50e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009fd0 tx=0x7f2ef0019040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.142+0000 7f2f10217b80 1 -- start start 2024-06-02T23:33:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.142+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe70630340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.142+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70630880 0x55fe7062a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.142+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe7062abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2f10217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fe7051cf60 con 0x55fe70630880 2024-06-02T23:33:01.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2f10217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fe7057b850 con 0x55fe7062a7a0 2024-06-02T23:33:01.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2f10217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe704e3460 con 0x55fe7062ff60 2024-06-02T23:33:01.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe7062abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe7062abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45188/0 (socket says 172.21.15.52:45188) 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 -- 172.21.15.52:0/3860604232 learned_addr learned my addr 172.21.15.52:0/3860604232 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2efeffd700 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe70630340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70630880 0x55fe7062a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 -- 172.21.15.52:0/3860604232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 msgr2=0x55fe70630340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe70630340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 -- 172.21.15.52:0/3860604232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70630880 msgr2=0x55fe7062a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70630880 0x55fe7062a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.143+0000 7f2eff7fe700 1 -- 172.21.15.52:0/3860604232 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe704ca8d0 con 0x55fe7062a7a0 2024-06-02T23:33:01.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.144+0000 7f2eff7fe700 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe7062abc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8014d20 tx=0x7f2ef800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94000da6cf02caa1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:01.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.144+0000 7f2ee5ffb700 1 -- 172.21.15.52:0/3860604232 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8024070 con 0x55fe7062a7a0 2024-06-02T23:33:01.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.145+0000 7f2ee5ffb700 1 -- 172.21.15.52:0/3860604232 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef8019890 con 0x55fe7062a7a0 2024-06-02T23:33:01.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.145+0000 7f2ee5ffb700 1 -- 172.21.15.52:0/3860604232 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8007930 con 0x55fe7062a7a0 2024-06-02T23:33:01.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 -- 172.21.15.52:0/3860604232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 msgr2=0x55fe7062abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe7062abc0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8014d20 tx=0x7f2ef800b040 comp rx=0 tx=0).stop 2024-06-02T23:33:01.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 -- 172.21.15.52:0/3860604232 shutdown_connections 2024-06-02T23:33:01.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe7062abc0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe70630340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 --2- 172.21.15.52:0/3860604232 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70630880 0x55fe7062a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.147 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 -- 172.21.15.52:0/3860604232 >> 172.21.15.52:0/3860604232 conn(0x55fe70557080 msgr2=0x55fe70625220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:01.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 -- 172.21.15.52:0/3860604232 shutdown_connections 2024-06-02T23:33:01.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.146+0000 7f2f10217b80 1 -- 172.21.15.52:0/3860604232 wait complete. 2024-06-02T23:33:01.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.147+0000 7f2f10217b80 1 -- start start 2024-06-02T23:33:01.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.147+0000 7f2f10217b80 1 Processor -- start 2024-06-02T23:33:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe70636560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe7062ff60 0x55fe70633860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe70633da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2f10217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55fe7051bfc0 con 0x55fe7062ff60 2024-06-02T23:33:01.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2f10217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55fe70577810 con 0x55fe7062a7a0 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2f10217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55fe706372b0 con 0x55fe70630880 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2efeffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe70636560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2efeffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe70636560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45192/0 (socket says 172.21.15.52:45192) 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2efeffd700 1 -- 172.21.15.52:0/545007685 learned_addr learned my addr 172.21.15.52:0/545007685 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.148+0000 7f2eff7fe700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe70633da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2f10217b80 10 client.0 _ll_register_callbacks cb 0x55fe705115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efeffd700 1 -- 172.21.15.52:0/545007685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 msgr2=0x55fe70633da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efeffd700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe70633da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efeffd700 1 -- 172.21.15.52:0/545007685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe7062ff60 msgr2=0x55fe70633860 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe7062ff60 0x55fe70633860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efeffd700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe7062ff60 0x55fe70633860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efeffd700 1 -- 172.21.15.52:0/545007685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe70541750 con 0x55fe7062a7a0 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef8019890 con 0x55fe7062a7a0 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.149+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe7062ff60 0x55fe70633860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2efeffd700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe70636560 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009230 tx=0x7f2ef0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0f1db376954826 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2edeffd700 10 client.30233 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2edeffd700 1 -- 172.21.15.52:0/545007685 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0034060 con 0x55fe7062a7a0 2024-06-02T23:33:01.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2edeffd700 1 -- 172.21.15.52:0/545007685 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ef0003680 con 0x55fe7062a7a0 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2edeffd700 1 -- 172.21.15.52:0/545007685 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2ef00038a0 con 0x55fe7062a7a0 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2f10217b80 10 client.30233 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.150+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55fe70641700 con 0x55fe7062a7a0 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2edeffd700 1 -- 172.21.15.52:0/545007685 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2ef000b560 con 0x55fe7062a7a0 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2f10217b80 10 client.30233 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2f10217b80 10 client.30233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2f10217b80 20 client.30233 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2f10217b80 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe70644080 0x55fe70646540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2f10217b80 4 client.30233 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe70588d50 con 0x55fe70644080 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe70644080 0x55fe70646540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe70644080 0x55fe70646540 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1bfc09880fe7919 server_cookie=57fd4a7f0c57e690 in_seq=0 out_seq=0 2024-06-02T23:33:01.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.151+0000 7f2edeffd700 10 client.30233 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.152+0000 7f2edeffd700 1 -- 172.21.15.52:0/545007685 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55fe70588d50 con 0x55fe70644080 2024-06-02T23:33:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.152+0000 7f2edeffd700 10 client.30233 handle_command_reply: tid=0 2024-06-02T23:33:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.152+0000 7f2f10217b80 1 client.30233 shutdown 2024-06-02T23:33:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 msgr2=0x55fe70636560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe70636560 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009230 tx=0x7f2ef0009c00 comp rx=0 tx=0).stop 2024-06-02T23:33:01.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 shutdown_connections 2024-06-02T23:33:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe70644080 0x55fe70646540 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe70633da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062a7a0 0x55fe70636560 secure :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009230 tx=0x7f2ef0009c00 comp rx=0 tx=0).stop 2024-06-02T23:33:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 --2- 172.21.15.52:0/545007685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe7062ff60 0x55fe70633860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2edeffd700 0 client.30233 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:01.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.153+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 >> 172.21.15.52:0/545007685 conn(0x55fe70557080 msgr2=0x55fe70624700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:01.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.155+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 shutdown_connections 2024-06-02T23:33:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.156+0000 7f2f10217b80 1 -- 172.21.15.52:0/545007685 wait complete. 2024-06-02T23:33:01.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.156+0000 7f2f10217b80 20 client.30233 trim_cache size 0 max 16384 2024-06-02T23:33:01.188 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 Processor -- start 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 -- start start 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe7062a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe70630880 0x55fe70635920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe70625300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00d2a0 con 0x55fe70624e80 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00ede0 con 0x55fe70630880 2024-06-02T23:33:01.189 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2f10217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00d970 con 0x55fe7062ff60 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.188+0000 7f2eff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe70625300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe70630880 0x55fe70635920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2eff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe70625300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40232/0 (socket says 172.21.15.52:40232) 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2eff7fe700 1 -- 172.21.15.52:0/191371612 learned_addr learned my addr 172.21.15.52:0/191371612 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efeffd700 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe7062a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 -- 172.21.15.52:0/191371612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 msgr2=0x55fe7062a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe7062a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 -- 172.21.15.52:0/191371612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 msgr2=0x55fe70625300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe70625300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 -- 172.21.15.52:0/191371612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe70646bc0 con 0x55fe70630880 2024-06-02T23:33:01.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe70630880 0x55fe70635920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40cd010 tx=0x7f2ef4086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b06a362e58703717 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:01.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.189+0000 7f2ee57fa700 1 -- 172.21.15.52:0/191371612 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40b8070 con 0x55fe70630880 2024-06-02T23:33:01.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2ee57fa700 1 -- 172.21.15.52:0/191371612 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef409c100 con 0x55fe70630880 2024-06-02T23:33:01.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2ee57fa700 1 -- 172.21.15.52:0/191371612 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40ba310 con 0x55fe70630880 2024-06-02T23:33:01.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 -- 172.21.15.52:0/191371612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe70630880 msgr2=0x55fe70635920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe70630880 0x55fe70635920 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2ef40cd010 tx=0x7f2ef4086040 comp rx=0 tx=0).stop 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 -- 172.21.15.52:0/191371612 shutdown_connections 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe70625300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe7062ff60 0x55fe7062a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 --2- 172.21.15.52:0/191371612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe70630880 0x55fe70635920 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.190+0000 7f2f10217b80 1 -- 172.21.15.52:0/191371612 >> 172.21.15.52:0/191371612 conn(0x55fe70557080 msgr2=0x55fe706312d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.191+0000 7f2f10217b80 1 -- 172.21.15.52:0/191371612 shutdown_connections 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.191+0000 7f2f10217b80 1 -- 172.21.15.52:0/191371612 wait complete. 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.191+0000 7f2f10217b80 1 -- start start 2024-06-02T23:33:01.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.191+0000 7f2f10217b80 1 Processor -- start 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe7063c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062ff60 0x55fe7063cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe7063c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe706395a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00e710 con 0x55fe70624e80 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00e040 con 0x55fe7062ff60 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec00b240 con 0x55fe70630880 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2f10217b80 10 client.0 _ll_register_callbacks cb 0x55fe705115b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe7063c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40246/0 (socket says 172.21.15.52:40246) 2024-06-02T23:33:01.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 -- 172.21.15.52:0/2073971806 learned_addr learned my addr 172.21.15.52:0/2073971806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2eff7fe700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe706395a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062ff60 0x55fe7063cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 -- 172.21.15.52:0/2073971806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 msgr2=0x55fe706395a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe706395a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 -- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062ff60 msgr2=0x55fe7063cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062ff60 0x55fe7063cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efeffd700 1 -- 172.21.15.52:0/2073971806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe70639d90 con 0x55fe70624e80 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.192+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062ff60 0x55fe7063cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2efeffd700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe7063c820 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0000c00 tx=0x7f2ef0020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11477b4392bd7f63 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2edeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2edeffd700 1 -- 172.21.15.52:0/2073971806 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0046060 con 0x55fe70624e80 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef409c100 con 0x55fe70624e80 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2edeffd700 1 -- 172.21.15.52:0/2073971806 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2ef00056d0 con 0x55fe70624e80 2024-06-02T23:33:01.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2edeffd700 1 -- 172.21.15.52:0/2073971806 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2ef0005900 con 0x55fe70624e80 2024-06-02T23:33:01.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2f10217b80 10 client.20955 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:01.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.193+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55fe706af8e0 con 0x55fe70624e80 2024-06-02T23:33:01.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2edeffd700 1 -- 172.21.15.52:0/2073971806 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2ef409c100 con 0x55fe70624e80 2024-06-02T23:33:01.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2edeffd700 1 -- 172.21.15.52:0/2073971806 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2ef004d6c0 con 0x55fe70624e80 2024-06-02T23:33:01.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2f10217b80 10 client.20955 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:01.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2f10217b80 10 client.20955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2f10217b80 20 client.20955 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2f10217b80 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe706aff40 0x55fe706b2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2f10217b80 4 client.20955 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fe705832d0 con 0x55fe706aff40 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.194+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe706aff40 0x55fe706b2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2efe7fc700 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe706aff40 0x55fe706b2360 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9d82d8217ca1d85 server_cookie=a8760e271b60c643 in_seq=0 out_seq=0 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2edeffd700 10 client.20955 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2edeffd700 1 -- 172.21.15.52:0/2073971806 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fe705832d0 con 0x55fe706aff40 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2edeffd700 10 client.20955 handle_command_reply: tid=0 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 client.20955 shutdown 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 msgr2=0x55fe7063c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe7063c820 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0000c00 tx=0x7f2ef0020000 comp rx=0 tx=0).stop 2024-06-02T23:33:01.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 shutdown_connections 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55fe706aff40 0x55fe706b2360 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55fe70630880 0x55fe706395a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55fe70624e80 0x55fe7063c820 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 --2- 172.21.15.52:0/2073971806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55fe7062ff60 0x55fe7063cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 >> 172.21.15.52:0/2073971806 conn(0x55fe70557080 msgr2=0x55fe70624700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2edeffd700 0 client.20955 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 shutdown_connections 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.195+0000 7f2f10217b80 1 -- 172.21.15.52:0/2073971806 wait complete. 2024-06-02T23:33:01.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.196+0000 7f2f10217b80 20 client.20955 trim_cache size 0 max 16384 2024-06-02T23:33:01.201 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:33:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:33:01.201 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:33:01.201 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.196+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ee80a2c20 msgr2=0x7f2ee80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.196+0000 7f2f05994700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ee80a2c20 0x7f2ee80a50e0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009fd0 tx=0x7f2ef0019040 comp rx=0 tx=0).stop 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.196+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 msgr2=0x7f2f0019b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.196+0000 7f2f05994700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 0x7f2f0019b3b0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400eb30 tx=0x7f2ef400ee40 comp rx=0 tx=0).stop 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 shutdown_connections 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ee80a2c20 0x7f2ee80a50e0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2f0010ef40 0x7f2f0019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f0006a1a0 0x7f2f0019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 --2- 172.21.15.52:0/49585278 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2f0006aac0 0x7f2f0019b3b0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 >> 172.21.15.52:0/49585278 conn(0x7f2f00078340 msgr2=0x7f2f00110570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 shutdown_connections 2024-06-02T23:33:01.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:01.197+0000 7f2f05994700 1 -- 172.21.15.52:0/49585278 wait complete. 2024-06-02T23:33:01.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 58735 6.72 MB/sec execute 369 sec latency 90.934 ms 2024-06-02T23:33:01.993 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:33:01.993 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:33:02.066 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:01 smithi052 ceph-mon[32340]: pgmap v1065: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 289 op/s 2024-06-02T23:33:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:01 smithi174 ceph-mon[35994]: pgmap v1065: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 289 op/s 2024-06-02T23:33:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:01 smithi123 ceph-mon[35977]: pgmap v1065: 97 pgs: 97 active+clean; 508 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.8 MiB/s wr, 289 op/s 2024-06-02T23:33:02.366 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:02.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 58908 6.72 MB/sec execute 370 sec latency 187.027 ms 2024-06-02T23:33:03.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59131 6.75 MB/sec execute 371 sec latency 74.141 ms 2024-06-02T23:33:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:03 smithi052 ceph-mon[32340]: pgmap v1066: 97 pgs: 97 active+clean; 512 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 319 op/s 2024-06-02T23:33:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:03 smithi174 ceph-mon[35994]: pgmap v1066: 97 pgs: 97 active+clean; 512 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 319 op/s 2024-06-02T23:33:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:03 smithi123 ceph-mon[35977]: pgmap v1066: 97 pgs: 97 active+clean; 512 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.9 MiB/s wr, 319 op/s 2024-06-02T23:33:04.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59326 6.77 MB/sec execute 372 sec latency 85.183 ms 2024-06-02T23:33:05.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59444 6.77 MB/sec execute 373 sec latency 132.919 ms 2024-06-02T23:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 -- 172.21.15.52:0/1220555678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 msgr2=0x7fc520101e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 --2- 172.21.15.52:0/1220555678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520101e20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fc514009900 tx=0x7fc51402a410 comp rx=0 tx=0).stop 2024-06-02T23:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 -- 172.21.15.52:0/1220555678 shutdown_connections 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 --2- 172.21.15.52:0/1220555678 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc5201035b0 0x7fc520109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 --2- 172.21.15.52:0/1220555678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc520102bf0 0x7fc520103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 --2- 172.21.15.52:0/1220555678 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520101e20 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 -- 172.21.15.52:0/1220555678 >> 172.21.15.52:0/1220555678 conn(0x7fc5200fcf90 msgr2=0x7fc5200ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 -- 172.21.15.52:0/1220555678 shutdown_connections 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.688+0000 7fc526d7d640 1 -- 172.21.15.52:0/1220555678 wait complete. 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 Processor -- start 2024-06-02T23:33:05.690 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 -- start start 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520075750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 0x7fc520073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5201035b0 0x7fc5200742d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010d530 con 0x7fc520101a00 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010d3b0 con 0x7fc5201035b0 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.689+0000 7fc526d7d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc52010d6b0 con 0x7fc520102bf0 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 0x7fc520073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 0x7fc520073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37054/0 (socket says 172.21.15.52:37054) 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 -- 172.21.15.52:0/44042075 learned_addr learned my addr 172.21.15.52:0/44042075 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc524b19640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520075750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc52531a640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5201035b0 0x7fc5200742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 -- 172.21.15.52:0/44042075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5201035b0 msgr2=0x7fc5200742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5201035b0 0x7fc5200742d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 -- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 msgr2=0x7fc520075750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520075750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc51ffff640 1 -- 172.21.15.52:0/44042075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc520074b60 con 0x7fc520102bf0 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc52531a640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5201035b0 0x7fc5200742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.690+0000 7fc524b19640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.691+0000 7fc51ffff640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 0x7fc520073d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00e9f0 tx=0x7fc50c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.691+0000 7fc51dffb640 1 -- 172.21.15.52:0/44042075 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00cd80 con 0x7fc520102bf0 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.691+0000 7fc51dffb640 1 -- 172.21.15.52:0/44042075 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc50c0108e0 con 0x7fc520102bf0 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.691+0000 7fc51dffb640 1 -- 172.21.15.52:0/44042075 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c018920 con 0x7fc520102bf0 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.691+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc520197850 con 0x7fc520102bf0 2024-06-02T23:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.691+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc520197cf0 con 0x7fc520102bf0 2024-06-02T23:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.694+0000 7fc51dffb640 1 -- 172.21.15.52:0/44042075 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc50c010a90 con 0x7fc520102bf0 2024-06-02T23:33:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 Processor -- start 2024-06-02T23:33:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 -- start start 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae7f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae7fe500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae801000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5646ae75c4f0 con 0x5646ae7f9a00 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646ae6fa870 con 0x5646ae7feb70 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc529a85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646ae7584b0 con 0x5646ae7f9010 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc51dffb640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc5100a2cb0 0x7fc5100a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc524b19640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae7fe500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc524b19640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae7fe500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39002/0 (socket says 172.21.15.52:39002) 2024-06-02T23:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc524b19640 1 -- 172.21.15.52:0/608471027 learned_addr learned my addr 172.21.15.52:0/608471027 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae801000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.695+0000 7fc52531a640 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae7f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc52531a640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc5100a2cb0 0x7fc5100a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc51dffb640 1 -- 172.21.15.52:0/44042075 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc50c0cf4a0 con 0x7fc520102bf0 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc524b19640 1 -- 172.21.15.52:0/608471027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 msgr2=0x5646ae7f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc524b19640 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae7f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc524b19640 1 -- 172.21.15.52:0/608471027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 msgr2=0x5646ae801000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc524b19640 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae801000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc524b19640 1 -- 172.21.15.52:0/608471027 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ae7169b0 con 0x5646ae7f9a00 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc52531a640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc5100a2cb0 0x7fc5100a5170 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x5646ae7fa260 tx=0x7fc51800a370 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc524b19640 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae7fe500 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fc514009900 tx=0x7fc514032da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fc115b5fc13008 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc50affd640 1 -- 172.21.15.52:0/608471027 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51403f070 con 0x5646ae7f9a00 2024-06-02T23:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc50affd640 1 -- 172.21.15.52:0/608471027 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc514031780 con 0x5646ae7f9a00 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.696+0000 7fc50affd640 1 -- 172.21.15.52:0/608471027 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc514035070 con 0x5646ae7f9a00 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 -- 172.21.15.52:0/608471027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 msgr2=0x5646ae7fe500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae7fe500 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fc514009900 tx=0x7fc514032da0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 -- 172.21.15.52:0/608471027 shutdown_connections 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae801000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae7fe500 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 --2- 172.21.15.52:0/608471027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae7f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 -- 172.21.15.52:0/608471027 >> 172.21.15.52:0/608471027 conn(0x5646ae7f4680 msgr2=0x5646ae7f4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 -- 172.21.15.52:0/608471027 shutdown_connections 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 -- 172.21.15.52:0/608471027 wait complete. 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 -- start start 2024-06-02T23:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.697+0000 7fc529a85740 1 Processor -- start 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 0x5646ae803200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9a00 0x5646ae803740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae80a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5646ae801ac0 con 0x5646ae7f9010 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646ae6e5ae0 con 0x5646ae7f9a00 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646ae801c40 con 0x5646ae7feb70 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc529a85740 10 client.0 _ll_register_callbacks cb 0x5646ae71e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 0x5646ae803200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 0x5646ae803200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39008/0 (socket says 172.21.15.52:39008) 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 -- 172.21.15.52:0/1769963006 learned_addr learned my addr 172.21.15.52:0/1769963006 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:05.704 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc51ffff640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9a00 0x5646ae803740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc52531a640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae80a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 -- 172.21.15.52:0/1769963006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 msgr2=0x5646ae80a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae80a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 -- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9a00 msgr2=0x5646ae803740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9a00 0x5646ae803740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 -- 172.21.15.52:0/1769963006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ae470210 con 0x5646ae7f9010 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc524b19640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 0x5646ae803200 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fc51402a930 tx=0x7fc514004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e28ffa9b2b052eb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.698+0000 7fc4fffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc4fffff640 1 -- 172.21.15.52:0/1769963006 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51403f070 con 0x5646ae7f9010 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc4fffff640 1 -- 172.21.15.52:0/1769963006 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc514004f00 con 0x5646ae7f9010 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc514031780 con 0x5646ae7f9010 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc4fffff640 1 -- 172.21.15.52:0/1769963006 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc514035040 con 0x5646ae7f9010 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 10 client.20967 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:05.705 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5646ae53c620 con 0x5646ae7f9010 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc4fffff640 1 -- 172.21.15.52:0/1769963006 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc514031780 con 0x5646ae7f9010 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc4fffff640 1 -- 172.21.15.52:0/1769963006 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc5140035f0 con 0x5646ae7f9010 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 10 client.20967 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 10 client.20967 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 20 client.20967 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae80ec70 0x5646ae811060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 4 client.20967 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646ae76ff10 con 0x5646ae80ec70 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.699+0000 7fc51ffff640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae80ec70 0x5646ae811060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.700+0000 7fc51ffff640 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae80ec70 0x5646ae811060 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74636148ff8439bb server_cookie=703ba78a77fc08e7 in_seq=0 out_seq=0 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.700+0000 7fc4fffff640 10 client.20967 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:05.706 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.700+0000 7fc4fffff640 1 -- 172.21.15.52:0/1769963006 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5646ae76ff10 con 0x5646ae80ec70 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.700+0000 7fc4fffff640 10 client.20967 handle_command_reply: tid=0 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.700+0000 7fc529a85740 1 client.20967 shutdown 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 msgr2=0x5646ae803200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 0x5646ae803200 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fc51402a930 tx=0x7fc514004d10 comp rx=0 tx=0).stop 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 shutdown_connections 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae80ec70 0x5646ae811060 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae80a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9a00 0x5646ae803740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc4fffff640 0 client.20967 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 --2- 172.21.15.52:0/1769963006 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9010 0x5646ae803200 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 >> 172.21.15.52:0/1769963006 conn(0x5646ae7f4680 msgr2=0x5646ae7f37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 shutdown_connections 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 1 -- 172.21.15.52:0/1769963006 wait complete. 2024-06-02T23:33:05.707 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.701+0000 7fc529a85740 20 client.20967 trim_cache size 0 max 16384 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 Processor -- start 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 -- start start 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9010 0x5646ae7f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae80a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae8024f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201af250 con 0x5646ae7f9a00 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201b0d90 con 0x5646ae7f9010 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc529a85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201af920 con 0x5646ae7feb70 2024-06-02T23:33:05.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae80a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae80a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39012/0 (socket says 172.21.15.52:39012) 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.716+0000 7fc51ffff640 1 -- 172.21.15.52:0/2321372313 learned_addr learned my addr 172.21.15.52:0/2321372313 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc524b19640 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9010 0x5646ae7f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc52531a640 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae8024f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc51ffff640 1 -- 172.21.15.52:0/2321372313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 msgr2=0x5646ae8024f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc51ffff640 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae8024f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc51ffff640 1 -- 172.21.15.52:0/2321372313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9010 msgr2=0x5646ae7f7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc51ffff640 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9010 0x5646ae7f7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc51ffff640 1 -- 172.21.15.52:0/2321372313 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646ae811640 con 0x5646ae7f9a00 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc52531a640 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae8024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc51ffff640 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae80a740 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c6a0 tx=0x7fc50c0cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71bf46d8c28d42a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc50a7fc640 1 -- 172.21.15.52:0/2321372313 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c09c3d0 con 0x5646ae7f9a00 2024-06-02T23:33:05.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc50a7fc640 1 -- 172.21.15.52:0/2321372313 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc50c09bcd0 con 0x5646ae7f9a00 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc50a7fc640 1 -- 172.21.15.52:0/2321372313 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0cf0b0 con 0x5646ae7f9a00 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc529a85740 1 -- 172.21.15.52:0/2321372313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 msgr2=0x5646ae80a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc529a85740 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae80a740 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00c6a0 tx=0x7fc50c0cf040 comp rx=0 tx=0).stop 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc529a85740 1 -- 172.21.15.52:0/2321372313 shutdown_connections 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc529a85740 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7feb70 0x5646ae8024f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc529a85740 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae80a740 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.717+0000 7fc529a85740 1 --2- 172.21.15.52:0/2321372313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7f9010 0x5646ae7f7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- 172.21.15.52:0/2321372313 >> 172.21.15.52:0/2321372313 conn(0x5646ae7f4680 msgr2=0x5646ae7f7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- 172.21.15.52:0/2321372313 shutdown_connections 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- 172.21.15.52:0/2321372313 wait complete. 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- start start 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 Processor -- start 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae8077e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae807d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae808d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201b06c0 con 0x5646ae7f9a00 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201afff0 con 0x5646ae7feb70 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc5201ad230 con 0x5646ae7f9010 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc529a85740 10 client.0 _ll_register_callbacks cb 0x7fc50c097f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae807d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae807d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39028/0 (socket says 172.21.15.52:39028) 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.718+0000 7fc51ffff640 1 -- 172.21.15.52:0/498599647 learned_addr learned my addr 172.21.15.52:0/498599647 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc52531a640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae808d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc524b19640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae8077e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc51ffff640 1 -- 172.21.15.52:0/498599647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 msgr2=0x5646ae8077e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc51ffff640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae8077e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc51ffff640 1 -- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 msgr2=0x5646ae808d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc51ffff640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae808d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc51ffff640 1 -- 172.21.15.52:0/498599647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646ae809320 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc524b19640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae8077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc51ffff640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae807d20 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0a4410 tx=0x7fc50c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43d3355caf5d0e68 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc4fffff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc4fffff640 1 -- 172.21.15.52:0/498599647 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0b9060 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc4fffff640 1 -- 172.21.15.52:0/498599647 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc50c010430 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc50c09bcd0 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc4fffff640 1 -- 172.21.15.52:0/498599647 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc50c09c3d0 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc529a85740 10 client.20973 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5646ae80a5a0 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc4fffff640 1 -- 172.21.15.52:0/498599647 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc50c09bcd0 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc4fffff640 1 -- 172.21.15.52:0/498599647 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc50c064070 con 0x5646ae7f9a00 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc529a85740 10 client.20973 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.719+0000 7fc529a85740 10 client.20973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc529a85740 20 client.20973 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc529a85740 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae879a90 0x5646ae87beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc529a85740 4 client.20973 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646ae769c20 con 0x5646ae879a90 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc524b19640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae879a90 0x5646ae87beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc4fffff640 10 client.20973 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.720+0000 7fc524b19640 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae879a90 0x5646ae87beb0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=691180025391ba98 server_cookie=b1bc8d01dcab8c15 in_seq=0 out_seq=0 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc4fffff640 1 -- 172.21.15.52:0/498599647 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5646ae769c20 con 0x5646ae879a90 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc4fffff640 10 client.20973 handle_command_reply: tid=0 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 client.20973 shutdown 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 msgr2=0x5646ae807d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae807d20 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0a4410 tx=0x7fc50c000bc0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 shutdown_connections 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5646ae879a90 0x5646ae87beb0 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646ae7feb70 0x5646ae808d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc4fffff640 0 client.20973 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5646ae7f9a00 0x5646ae807d20 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 --2- 172.21.15.52:0/498599647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646ae7f9010 0x5646ae8077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.764+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 >> 172.21.15.52:0/498599647 conn(0x5646ae7f4680 msgr2=0x5646ae7f37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.765+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 shutdown_connections 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.765+0000 7fc529a85740 1 -- 172.21.15.52:0/498599647 wait complete. 2024-06-02T23:33:05.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.765+0000 7fc529a85740 20 client.20973 trim_cache size 0 max 16384 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.766+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc5100a2cb0 msgr2=0x7fc5100a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.766+0000 7fc526d7d640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc5100a2cb0 0x7fc5100a5170 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x5646ae7fa260 tx=0x7fc51800a370 comp rx=0 tx=0).stop 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.766+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 msgr2=0x7fc520073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:05.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.766+0000 7fc526d7d640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 0x7fc520073d90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00e9f0 tx=0x7fc50c00eeb0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.767+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 shutdown_connections 2024-06-02T23:33:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.767+0000 7fc526d7d640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc5100a2cb0 0x7fc5100a5170 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.767+0000 7fc526d7d640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc520102bf0 0x7fc520073d90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.767+0000 7fc526d7d640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc5201035b0 0x7fc5200742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.767+0000 7fc526d7d640 1 --2- 172.21.15.52:0/44042075 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc520101a00 0x7fc520075750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:05.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.767+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 >> 172.21.15.52:0/44042075 conn(0x7fc5200fcf90 msgr2=0x7fc520105ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.768+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 shutdown_connections 2024-06-02T23:33:05.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:05.768+0000 7fc526d7d640 1 -- 172.21.15.52:0/44042075 wait complete. 2024-06-02T23:33:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:05 smithi174 ceph-mon[35994]: pgmap v1067: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-06-02T23:33:06.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:05 smithi052 ceph-mon[32340]: pgmap v1067: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-06-02T23:33:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:05 smithi123 ceph-mon[35977]: pgmap v1067: 97 pgs: 97 active+clean; 514 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.6 MiB/s wr, 267 op/s 2024-06-02T23:33:06.472 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59567 6.78 MB/sec execute 374 sec latency 174.541 ms 2024-06-02T23:33:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 -- 172.21.15.52:0/2338836875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 msgr2=0x7f1138105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 --2- 172.21.15.52:0/2338836875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f1138105f20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f112c0098f0 tx=0x7f112c009c00 comp rx=0 tx=0).stop 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 -- 172.21.15.52:0/2338836875 shutdown_connections 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 --2- 172.21.15.52:0/2338836875 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138106600 0x7f113810e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 --2- 172.21.15.52:0/2338836875 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138101520 0x7f1138101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 --2- 172.21.15.52:0/2338836875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f1138105f20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 -- 172.21.15.52:0/2338836875 >> 172.21.15.52:0/2338836875 conn(0x7f11380fcdb0 msgr2=0x7f11380ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 -- 172.21.15.52:0/2338836875 shutdown_connections 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.412+0000 7f113cb9f700 1 -- 172.21.15.52:0/2338836875 wait complete. 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.413+0000 7f113cb9f700 1 Processor -- start 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 -- start start 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138101520 0x7f113819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f113819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138106600 0x7f1138198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1138111470 con 0x7f1138101520 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11381112f0 con 0x7f1138101ed0 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f113cb9f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f11381115f0 con 0x7f1138106600 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f1136ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f113819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f11377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138101520 0x7f113819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f1136ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f113819ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:50736/0 (socket says 172.21.15.52:50736) 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f11377fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138101520 0x7f113819e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39050/0 (socket says 172.21.15.52:39050) 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f1136ffd700 1 -- 172.21.15.52:0/4117853895 learned_addr learned my addr 172.21.15.52:0/4117853895 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.414+0000 7f1137fff700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138106600 0x7f1138198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1136ffd700 1 -- 172.21.15.52:0/4117853895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138106600 msgr2=0x7f1138198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1136ffd700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138106600 0x7f1138198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1136ffd700 1 -- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138101520 msgr2=0x7f113819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1136ffd700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138101520 0x7f113819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1136ffd700 1 -- 172.21.15.52:0/4117853895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1138199060 con 0x7f1138101ed0 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1137fff700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138106600 0x7f1138198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1136ffd700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f113819ec80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f112c009fd0 tx=0x7f112c00f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c005040 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f112c005690 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11381992f0 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11381a5610 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.415+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c003970 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.417+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1138069180 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.418+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f112c00b560 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.419+0000 7f1134ff9700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f11200a2ca0 0x7f11200a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.419+0000 7f11377fe700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f11200a2ca0 0x7f11200a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.419+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f112c0cfff0 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.420+0000 7f11377fe700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f11200a2ca0 0x7f11200a5160 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1138199a40 tx=0x7f1128009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.422+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f112c013030 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.783+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11381a57e0 con 0x7f1138101ed0 2024-06-02T23:33:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.784+0000 7f1134ff9700 1 -- 172.21.15.52:0/4117853895 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f112c09d010 con 0x7f1138101ed0 2024-06-02T23:33:07.114 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:33:07.114 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:33:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f11200a2ca0 msgr2=0x7f11200a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f11200a2ca0 0x7f11200a5160 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1138199a40 tx=0x7f1128009260 comp rx=0 tx=0).stop 2024-06-02T23:33:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 msgr2=0x7f113819ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f113819ec80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f112c009fd0 tx=0x7f112c00f9b0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 shutdown_connections 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f11200a2ca0 0x7f11200a5160 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1138106600 0x7f1138198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1138101520 0x7f113819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 --2- 172.21.15.52:0/4117853895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1138101ed0 0x7f113819ec80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 >> 172.21.15.52:0/4117853895 conn(0x7f11380fcdb0 msgr2=0x7f11380ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 shutdown_connections 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:06.787+0000 7f113cb9f700 1 -- 172.21.15.52:0/4117853895 wait complete. 2024-06-02T23:33:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:33:07.115 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:06 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4117853895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:33:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:06 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4117853895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:33:07.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:06 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4117853895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:33:07.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59693 6.79 MB/sec execute 375 sec latency 173.604 ms 2024-06-02T23:33:08.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:07 smithi174 ceph-mon[35994]: pgmap v1068: 97 pgs: 97 active+clean; 522 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 335 op/s 2024-06-02T23:33:08.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:07 smithi052 ceph-mon[32340]: pgmap v1068: 97 pgs: 97 active+clean; 522 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 335 op/s 2024-06-02T23:33:08.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:07 smithi123 ceph-mon[35977]: pgmap v1068: 97 pgs: 97 active+clean; 522 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 335 op/s 2024-06-02T23:33:08.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59857 6.80 MB/sec execute 376 sec latency 58.030 ms 2024-06-02T23:33:09.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 59970 6.79 MB/sec execute 377 sec latency 142.655 ms 2024-06-02T23:33:10.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:09 smithi174 ceph-mon[35994]: pgmap v1069: 97 pgs: 97 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2024-06-02T23:33:10.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:09 smithi052 ceph-mon[32340]: pgmap v1069: 97 pgs: 97 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2024-06-02T23:33:10.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:09 smithi123 ceph-mon[35977]: pgmap v1069: 97 pgs: 97 active+clean; 524 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 259 op/s 2024-06-02T23:33:10.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60079 6.79 MB/sec execute 378 sec latency 149.205 ms 2024-06-02T23:33:11.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60187 6.77 MB/sec execute 379 sec latency 166.691 ms 2024-06-02T23:33:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:11 smithi174 ceph-mon[35994]: pgmap v1070: 97 pgs: 97 active+clean; 523 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 4.6 MiB/s wr, 268 op/s 2024-06-02T23:33:12.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:11 smithi052 ceph-mon[32340]: pgmap v1070: 97 pgs: 97 active+clean; 523 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 4.6 MiB/s wr, 268 op/s 2024-06-02T23:33:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:11 smithi123 ceph-mon[35977]: pgmap v1070: 97 pgs: 97 active+clean; 523 MiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1013 KiB/s rd, 4.6 MiB/s wr, 268 op/s 2024-06-02T23:33:12.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60299 6.76 MB/sec execute 380 sec latency 209.547 ms 2024-06-02T23:33:13.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:12 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:33:13.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:12 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:33:13.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:12 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:33:13.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60422 6.75 MB/sec execute 381 sec latency 117.919 ms 2024-06-02T23:33:14.012 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:33:14.089 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:13 smithi052 ceph-mon[32340]: pgmap v1071: 97 pgs: 97 active+clean; 526 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.6 MiB/s wr, 256 op/s 2024-06-02T23:33:14.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:33:14.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:33:14.090 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:33:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:13 smithi174 ceph-mon[35994]: pgmap v1071: 97 pgs: 97 active+clean; 526 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.6 MiB/s wr, 256 op/s 2024-06-02T23:33:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:33:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:33:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:33:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:13 smithi123 ceph-mon[35977]: pgmap v1071: 97 pgs: 97 active+clean; 526 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 990 KiB/s rd, 4.6 MiB/s wr, 256 op/s 2024-06-02T23:33:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:33:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:33:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:33:14.434 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:14.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60605 6.76 MB/sec execute 382 sec latency 124.676 ms 2024-06-02T23:33:15.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60752 6.76 MB/sec execute 383 sec latency 159.855 ms 2024-06-02T23:33:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:15 smithi174 ceph-mon[35994]: pgmap v1072: 97 pgs: 97 active+clean; 526 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.8 MiB/s wr, 211 op/s 2024-06-02T23:33:16.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:15 smithi052 ceph-mon[32340]: pgmap v1072: 97 pgs: 97 active+clean; 526 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.8 MiB/s wr, 211 op/s 2024-06-02T23:33:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:15 smithi123 ceph-mon[35977]: pgmap v1072: 97 pgs: 97 active+clean; 526 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 3.8 MiB/s wr, 211 op/s 2024-06-02T23:33:16.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 60921 6.77 MB/sec execute 384 sec latency 60.598 ms 2024-06-02T23:33:16.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 -- 172.21.15.52:0/1835007894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101d40 msgr2=0x7f66a410e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 --2- 172.21.15.52:0/1835007894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101d40 0x7f66a410e230 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f669c00b0a0 tx=0x7f669c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 -- 172.21.15.52:0/1835007894 shutdown_connections 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 --2- 172.21.15.52:0/1835007894 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101d40 0x7f66a410e230 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 --2- 172.21.15.52:0/1835007894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a41073f0 0x7f66a41077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 --2- 172.21.15.52:0/1835007894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a4101380 0x7f66a4101800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 -- 172.21.15.52:0/1835007894 >> 172.21.15.52:0/1835007894 conn(0x7f66a4078300 msgr2=0x7f66a4078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 -- 172.21.15.52:0/1835007894 shutdown_connections 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.925+0000 7f66a987d700 1 -- 172.21.15.52:0/1835007894 wait complete. 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.926+0000 7f66a987d700 1 Processor -- start 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.926+0000 7f66a987d700 1 -- start start 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a987d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 0x7f66a419e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a987d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d40 0x7f66a419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a987d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a41073f0 0x7f66a4198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 0x7f66a419e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 0x7f66a419e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60166/0 (socket says 172.21.15.52:60166) 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 -- 172.21.15.52:0/3526639635 learned_addr learned my addr 172.21.15.52:0/3526639635 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112690 con 0x7f66a4101380 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112510 con 0x7f66a4101d40 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66a4112810 con 0x7f66a41073f0 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a37fe700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a41073f0 0x7f66a4198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a27fc700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d40 0x7f66a419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 -- 172.21.15.52:0/3526639635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a41073f0 msgr2=0x7f66a4198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a41073f0 0x7f66a4198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 -- 172.21.15.52:0/3526639635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d40 msgr2=0x7f66a419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d40 0x7f66a419ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.927+0000 7f66a2ffd700 1 -- 172.21.15.52:0/3526639635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66a4199070 con 0x7f66a4101380 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a27fc700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d40 0x7f66a419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 0x7f66a419e6c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6694009cb0 tx=0x7f669400b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:16.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a887b700 1 -- 172.21.15.52:0/3526639635 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669401e070 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a887b700 1 -- 172.21.15.52:0/3526639635 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669400bc00 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a887b700 1 -- 172.21.15.52:0/3526639635 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694018610 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66a4199360 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.928+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66a410ebf0 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.931+0000 7f66a887b700 1 -- 172.21.15.52:0/3526639635 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6694021070 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.932+0000 7f66a887b700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f668c0a2ca0 0x7f668c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.932+0000 7f66b4100b80 1 Processor -- start 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66a27fc700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f668c0a2ca0 0x7f668c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66a887b700 1 -- 172.21.15.52:0/3526639635 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f66940d25d0 con 0x7f66a4101380 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 -- start start 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb0e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b0eafcdf60 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0eb02c850 con 0x55b0eb0ea090 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66b4100b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b0eaf94460 con 0x55b0eb0e9770 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66a37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66a27fc700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f668c0a2ca0 0x7f668c0a5160 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6698005fd0 tx=0x7f6698005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66a37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60168/0 (socket says 172.21.15.52:60168) 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.933+0000 7f66a37fe700 1 -- 172.21.15.52:0/1189165343 learned_addr learned my addr 172.21.15.52:0/1189165343 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb0e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a37fe700 1 -- 172.21.15.52:0/1189165343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 msgr2=0x55b0eb0e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a37fe700 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a37fe700 1 -- 172.21.15.52:0/1189165343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 msgr2=0x55b0eb0e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a37fe700 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb0e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a37fe700 1 -- 172.21.15.52:0/1189165343 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0eaf7b8d0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb0e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:16.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f66a37fe700 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e43d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f669c009410 tx=0x7f669c009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84f81b1b65d01d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f6689ffb700 1 -- 172.21.15.52:0/1189165343 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c00bdb0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f6689ffb700 1 -- 172.21.15.52:0/1189165343 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669c0074e0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.934+0000 7f6689ffb700 1 -- 172.21.15.52:0/1189165343 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669c005260 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 -- 172.21.15.52:0/1189165343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 msgr2=0x55b0eb0e43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e43d0 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f669c009410 tx=0x7f669c009440 comp rx=0 tx=0).stop 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 -- 172.21.15.52:0/1189165343 shutdown_connections 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e43d0 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1189165343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb0e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 -- 172.21.15.52:0/1189165343 >> 172.21.15.52:0/1189165343 conn(0x55b0eb008080 msgr2=0x55b0eb0d6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 -- 172.21.15.52:0/1189165343 shutdown_connections 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.935+0000 7f66b4100b80 1 -- 172.21.15.52:0/1189165343 wait complete. 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 -- start start 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 Processor -- start 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0eff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0ed230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0ed770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55b0eafccfc0 con 0x55b0eb0ea090 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b0eb028810 con 0x55b0eb0e9770 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.936+0000 7f66b4100b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b0eb0f0a10 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0eff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66b4100b80 10 client.0 _ll_register_callbacks cb 0x55b0eafc25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0eff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42076/0 (socket says 172.21.15.52:42076) 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 -- 172.21.15.52:0/1303630536 learned_addr learned my addr 172.21.15.52:0/1303630536 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0ed230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a37fe700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0ed770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 -- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 msgr2=0x55b0eb0ed230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0ed230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 -- 172.21.15.52:0/1303630536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 msgr2=0x55b0eb0ed770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0ed770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 -- 172.21.15.52:0/1303630536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0eaff2750 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0eff30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f669409eb60 tx=0x7f669409f3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b788e9921d42fe4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f6682ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f6682ffd700 1 -- 172.21.15.52:0/1303630536 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6694019950 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f669c0074e0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f6682ffd700 1 -- 172.21.15.52:0/1303630536 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f669409e2b0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f6682ffd700 1 -- 172.21.15.52:0/1303630536 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f669409f560 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66b4100b80 10 client.29899 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.937+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b0eb0fb350 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f6682ffd700 1 -- 172.21.15.52:0/1303630536 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f669c0074e0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f6682ffd700 1 -- 172.21.15.52:0/1303630536 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f66940194a0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66b4100b80 10 client.29899 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66b4100b80 10 client.29899 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66b4100b80 20 client.29899 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb0fd9e0 0x55b0eb0ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66b4100b80 4 client.29899 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0eb039d50 con 0x55b0eb0fd9e0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.938+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb0fd9e0 0x55b0eb0ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.939+0000 7f6682ffd700 10 client.29899 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.939+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb0fd9e0 0x55b0eb0ffdd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b42827239fa540bf server_cookie=d4fa3fd51f03dd85 in_seq=0 out_seq=0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.939+0000 7f6682ffd700 1 -- 172.21.15.52:0/1303630536 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55b0eb039d50 con 0x55b0eb0fd9e0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.939+0000 7f6682ffd700 10 client.29899 handle_command_reply: tid=0 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.939+0000 7f66b4100b80 1 client.29899 shutdown 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 msgr2=0x55b0eb0eff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0eff30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f669409eb60 tx=0x7f669409f3a0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 shutdown_connections 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0ed770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0eff30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb0fd9e0 0x55b0eb0ffdd0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1303630536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0ed230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 >> 172.21.15.52:0/1303630536 conn(0x55b0eb008080 msgr2=0x55b0eb0d5700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f6682ffd700 0 client.29899 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 shutdown_connections 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 1 -- 172.21.15.52:0/1303630536 wait complete. 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.940+0000 7f66b4100b80 20 client.29899 trim_cache size 0 max 16384 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.967+0000 7f66b4100b80 1 Processor -- start 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.967+0000 7f66b4100b80 1 -- start start 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.967+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.967+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0eab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0d5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66b4100b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41aa720 con 0x55b0eb0ea090 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66b4100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ac260 con 0x55b0eb0e9770 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66b4100b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41aadf0 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0eab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42086/0 (socket says 172.21.15.52:42086) 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a2ffd700 1 -- 172.21.15.52:0/1286534593 learned_addr learned my addr 172.21.15.52:0/1286534593 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a37fe700 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0d5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 -- 172.21.15.52:0/1286534593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 msgr2=0x55b0eb0e3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 -- 172.21.15.52:0/1286534593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 msgr2=0x55b0eb0d5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0d5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 -- 172.21.15.52:0/1286534593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0eb0e48b0 con 0x55b0eb0e9770 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.968+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0eab80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55b0eb100620 tx=0x7f6698014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eda14e5a3bc2e32 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.969+0000 7f66897fa700 1 -- 172.21.15.52:0/1286534593 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f669801ecf0 con 0x55b0eb0e9770 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.969+0000 7f66897fa700 1 -- 172.21.15.52:0/1286534593 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66980277e0 con 0x55b0eb0e9770 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.969+0000 7f66897fa700 1 -- 172.21.15.52:0/1286534593 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698010790 con 0x55b0eb0e9770 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 -- 172.21.15.52:0/1286534593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 msgr2=0x55b0eb0eab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0eab80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x55b0eb100620 tx=0x7f6698014000 comp rx=0 tx=0).stop 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 -- 172.21.15.52:0/1286534593 shutdown_connections 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0ea090 0x55b0eb0d5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0e3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1286534593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0e9770 0x55b0eb0eab80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 -- 172.21.15.52:0/1286534593 >> 172.21.15.52:0/1286534593 conn(0x55b0eb008080 msgr2=0x55b0eb0ea470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 -- 172.21.15.52:0/1286534593 shutdown_connections 2024-06-02T23:33:16.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 -- 172.21.15.52:0/1286534593 wait complete. 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.970+0000 7f66b4100b80 1 -- start start 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 Processor -- start 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb104a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41abb90 con 0x55b0eb0e3fb0 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41ab4c0 con 0x55b0eb0ea090 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66a41a86e0 con 0x55b0eb0e9770 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.971+0000 7f66b4100b80 10 client.0 _ll_register_callbacks cb 0x55b0eafc25b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42092/0 (socket says 172.21.15.52:42092) 2024-06-02T23:33:16.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a2ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60190/0 (socket says 172.21.15.52:60190) 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 -- 172.21.15.52:0/1059904007 learned_addr learned my addr 172.21.15.52:0/1059904007 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a37fe700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 -- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 msgr2=0x55b0eb104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 -- 172.21.15.52:0/1059904007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 msgr2=0x55b0eb0f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 -- 172.21.15.52:0/1059904007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0eb105270 con 0x55b0eb0e9770 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f66a27fc700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0f63f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f669800a5c0 tx=0x7f669801ed20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd5d3f36b452ebb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.972+0000 7f6682ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f6682ffd700 1 -- 172.21.15.52:0/1059904007 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698027b90 con 0x55b0eb0e9770 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f6682ffd700 1 -- 172.21.15.52:0/1059904007 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6698027d30 con 0x55b0eb0e9770 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66980277e0 con 0x55b0eb0e9770 2024-06-02T23:33:16.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f6682ffd700 1 -- 172.21.15.52:0/1059904007 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6698038440 con 0x55b0eb0e9770 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f66b4100b80 10 client.29905 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55b0eb1680f0 con 0x55b0eb0e9770 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.973+0000 7f6682ffd700 1 -- 172.21.15.52:0/1059904007 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66980277e0 con 0x55b0eb0e9770 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f6682ffd700 1 -- 172.21.15.52:0/1059904007 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6698036070 con 0x55b0eb0e9770 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66b4100b80 10 client.29905 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66b4100b80 10 client.29905 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66b4100b80 20 client.29905 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb1686f0 0x55b0eb16aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66b4100b80 4 client.29905 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b0eb0342d0 con 0x55b0eb1686f0 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb1686f0 0x55b0eb16aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:16.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f6682ffd700 10 client.29905 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.974+0000 7f66a2ffd700 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb1686f0 0x55b0eb16aae0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6e5f7dddc50473fc server_cookie=70c9e50bea931568 in_seq=0 out_seq=0 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f6682ffd700 1 -- 172.21.15.52:0/1059904007 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b0eb0342d0 con 0x55b0eb1686f0 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f6682ffd700 10 client.29905 handle_command_reply: tid=0 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 client.29905 shutdown 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 msgr2=0x55b0eb0f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:16.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0f63f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f669800a5c0 tx=0x7f669801ed20 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 shutdown_connections 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55b0eb1686f0 0x55b0eb16aae0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b0eb0ea090 0x55b0eb104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55b0eb0e3fb0 0x55b0eb0f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 --2- 172.21.15.52:0/1059904007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b0eb0e9770 0x55b0eb0f63f0 secure :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f669800a5c0 tx=0x7f669801ed20 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 >> 172.21.15.52:0/1059904007 conn(0x55b0eb008080 msgr2=0x55b0eb0eb730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f6682ffd700 0 client.29905 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 shutdown_connections 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.975+0000 7f66b4100b80 1 -- 172.21.15.52:0/1059904007 wait complete. 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.976+0000 7f66b4100b80 20 client.29905 trim_cache size 0 max 16384 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.976+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f668c0a2ca0 msgr2=0x7f668c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.976+0000 7f66a987d700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f668c0a2ca0 0x7f668c0a5160 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6698005fd0 tx=0x7f6698005e30 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.976+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 msgr2=0x7f66a419e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.976+0000 7f66a987d700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 0x7f66a419e6c0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f6694009cb0 tx=0x7f669400b2c0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 shutdown_connections 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f668c0a2ca0 0x7f668c0a5160 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66a41073f0 0x7f66a4198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66a4101380 0x7f66a419e6c0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 --2- 172.21.15.52:0/3526639635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66a4101d40 0x7f66a419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 >> 172.21.15.52:0/3526639635 conn(0x7f66a4078300 msgr2=0x7f66a40fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 shutdown_connections 2024-06-02T23:33:17.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:16.977+0000 7f66a987d700 1 -- 172.21.15.52:0/3526639635 wait complete. 2024-06-02T23:33:17.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 61115 6.79 MB/sec execute 385 sec latency 98.354 ms 2024-06-02T23:33:17.654 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:33:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:17 smithi052 ceph-mon[32340]: pgmap v1073: 97 pgs: 97 active+clean; 523 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2024-06-02T23:33:18.054 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:17 smithi174 ceph-mon[35994]: pgmap v1073: 97 pgs: 97 active+clean; 523 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2024-06-02T23:33:18.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:17 smithi123 ceph-mon[35977]: pgmap v1073: 97 pgs: 97 active+clean; 523 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.5 MiB/s wr, 282 op/s 2024-06-02T23:33:18.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 61306 6.80 MB/sec execute 386 sec latency 69.062 ms 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 -- 172.21.15.52:0/570291457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc103640 msgr2=0x7fa4dc109ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 --2- 172.21.15.52:0/570291457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc103640 0x7fa4dc109ed0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009ca0 tx=0x7fa4d402a8c0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 -- 172.21.15.52:0/570291457 shutdown_connections 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 --2- 172.21.15.52:0/570291457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc103640 0x7fa4dc109ed0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 --2- 172.21.15.52:0/570291457 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc102c80 0x7fa4dc103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 --2- 172.21.15.52:0/570291457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc1000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 -- 172.21.15.52:0/570291457 >> 172.21.15.52:0/570291457 conn(0x7fa4dc0fb600 msgr2=0x7fa4dc0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 -- 172.21.15.52:0/570291457 shutdown_connections 2024-06-02T23:33:18.905 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.903+0000 7fa4e335a640 1 -- 172.21.15.52:0/570291457 wait complete. 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 Processor -- start 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 -- start start 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4dc102c80 0x7fa4dc195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc103640 0x7fa4dc19ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc10b390 con 0x7fa4dc103640 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc10b210 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e335a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc10b510 con 0x7fa4dc102c80 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc195930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59344/0 (socket says 172.21.15.52:59344) 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e08f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4dc102c80 0x7fa4dc195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3340425242 learned_addr learned my addr 172.21.15.52:0/3340425242 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:18.906 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.904+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc103640 0x7fa4dc19ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3340425242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4dc102c80 msgr2=0x7fa4dc195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4dc102c80 0x7fa4dc195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc103640 msgr2=0x7fa4dc19ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc103640 0x7fa4dc19ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3340425242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19d500 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc195930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800e9c0 tx=0x7fa4d800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4d27fc640 1 -- 172.21.15.52:0/3340425242 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d800cd80 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4d27fc640 1 -- 172.21.15.52:0/3340425242 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d80035f0 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19d7f0 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc19dd40 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.907 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.905+0000 7fa4d27fc640 1 -- 172.21.15.52:0/3340425242 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d8010690 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.914 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.908+0000 7fa4d27fc640 1 -- 172.21.15.52:0/3340425242 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa4d80037a0 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.908+0000 7fa4e6067740 1 Processor -- start 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.908+0000 7fa4e6067740 1 -- start start 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949aed430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949aeda00 0x561949af2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949af2b70 0x561949af5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e6067740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561949a504f0 con 0x561949aed010 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e6067740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5619499ee870 con 0x561949aeda00 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e6067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561949a4c4b0 con 0x561949af2b70 2024-06-02T23:33:18.915 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e18f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949aed430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e18f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949aed430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60206/0 (socket says 172.21.15.52:60206) 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e18f7640 1 -- 172.21.15.52:0/3319039532 learned_addr learned my addr 172.21.15.52:0/3319039532 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949aeda00 0x561949af2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4d27fc640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa4c80a7040 0x7fa4c80a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949af2b70 0x561949af5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa4c80a7040 0x7fa4c80a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3319039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949af2b70 msgr2=0x561949af5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949af2b70 0x561949af5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.916 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3319039532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 msgr2=0x561949aed430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949aed430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3319039532 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561949a0a9b0 con 0x561949aeda00 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.909+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949af2b70 0x561949af5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.910+0000 7fa4d27fc640 1 -- 172.21.15.52:0/3340425242 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa4d80cf3b0 con 0x7fa4dc0ffcb0 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.910+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949aeda00 0x561949af2500 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cceb0 tx=0x7fa4d80d7470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9dcdb2785fcbadbe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.910+0000 7fa4c77fe640 1 -- 172.21.15.52:0/3319039532 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d809ba10 con 0x561949aeda00 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.910+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa4c80a7040 0x7fa4c80a9500 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4004af0 tx=0x7fa4d4009980 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.910+0000 7fa4c77fe640 1 -- 172.21.15.52:0/3319039532 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d8003b30 con 0x561949aeda00 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.910+0000 7fa4c77fe640 1 -- 172.21.15.52:0/3319039532 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d809b5a0 con 0x561949aeda00 2024-06-02T23:33:18.917 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 -- 172.21.15.52:0/3319039532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949aeda00 msgr2=0x561949af2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949aeda00 0x561949af2500 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cceb0 tx=0x7fa4d80d7470 comp rx=0 tx=0).stop 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 -- 172.21.15.52:0/3319039532 shutdown_connections 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949af2b70 0x561949af5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949aeda00 0x561949af2500 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3319039532 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949aed430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 -- 172.21.15.52:0/3319039532 >> 172.21.15.52:0/3319039532 conn(0x561949ae8680 msgr2=0x561949ae8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 -- 172.21.15.52:0/3319039532 shutdown_connections 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.911+0000 7fa4e6067740 1 -- 172.21.15.52:0/3319039532 wait complete. 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.912+0000 7fa4e6067740 1 -- start start 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.912+0000 7fa4e6067740 1 Processor -- start 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.912+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949affb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.918 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.912+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b00070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.912+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b07040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.912+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949affb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e6067740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561949af5b20 con 0x561949aed010 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949affb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60214/0 (socket says 172.21.15.52:60214) 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e6067740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5619499d9ae0 con 0x561949af2b70 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3480711222 learned_addr learned my addr 172.21.15.52:0/3480711222 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561949af5ca0 con 0x561949aeda00 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b00070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b07040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e6067740 10 client.0 _ll_register_callbacks cb 0x561949a129b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3480711222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 msgr2=0x561949b00070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.919 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b00070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 msgr2=0x561949b07040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b07040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3480711222 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561949764210 con 0x561949aed010 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b00070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949affb30 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cd250 tx=0x7fa4d8003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94267b440435d12a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3480711222 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d809b780 con 0x561949aed010 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561949a63f10 con 0x561949aed010 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.913+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3480711222 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa4d809bea0 con 0x561949aed010 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3480711222 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa4d809b5a0 con 0x561949aed010 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 10 client.21009 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561949830620 con 0x561949aed010 2024-06-02T23:33:18.920 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3480711222 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x561949a63f10 con 0x561949aed010 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3480711222 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa4d80a4420 con 0x561949aed010 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 10 client.21009 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 10 client.21009 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 20 client.21009 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b0b4d0 0x561949b0d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 4 client.21009 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561949a5dc20 con 0x561949b0b4d0 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.914+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b0b4d0 0x561949b0d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.915+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b0b4d0 0x561949b0d8c0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcad23562324493b server_cookie=ad2a076a8ff46319 in_seq=0 out_seq=0 2024-06-02T23:33:18.921 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.915+0000 7fa4c4ff9640 10 client.21009 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3480711222 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561949a5dc20 con 0x561949b0b4d0 2024-06-02T23:33:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4c4ff9640 10 client.21009 handle_command_reply: tid=0 2024-06-02T23:33:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 client.21009 shutdown 2024-06-02T23:33:18.922 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 msgr2=0x561949affb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949affb30 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fa4d80cd250 tx=0x7fa4d8003a90 comp rx=0 tx=0).stop 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 shutdown_connections 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b0b4d0 0x561949b0d8c0 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b00070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b07040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3480711222 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949affb30 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 >> 172.21.15.52:0/3480711222 conn(0x561949ae8680 msgr2=0x561949ae77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4c4ff9640 0 client.21009 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 shutdown_connections 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 1 -- 172.21.15.52:0/3480711222 wait complete. 2024-06-02T23:33:18.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.916+0000 7fa4e6067740 20 client.21009 trim_cache size 0 max 16384 2024-06-02T23:33:18.942 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.941+0000 7fa4e6067740 1 Processor -- start 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.941+0000 7fa4e6067740 1 -- start start 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949ae39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b07000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949affe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e6067740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a6ae0 con 0x561949aed010 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e6067740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a8620 con 0x561949af2b70 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e6067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a71b0 con 0x561949aeda00 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949ae39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.943 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e08f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b07000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e08f5640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b07000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42128/0 (socket says 172.21.15.52:42128) 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e08f5640 1 -- 172.21.15.52:0/71077402 learned_addr learned my addr 172.21.15.52:0/71077402 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949affe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e18f7640 1 -- 172.21.15.52:0/71077402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 msgr2=0x561949b07000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b07000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e18f7640 1 -- 172.21.15.52:0/71077402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 msgr2=0x561949ae39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949ae39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e18f7640 1 -- 172.21.15.52:0/71077402 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561949b084c0 con 0x561949af2b70 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.942+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b07000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.943+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949ae39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:18.944 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.943+0000 7fa4e18f7640 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949affe50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4006010 tx=0x7fa4d4047000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46d63d399b436e61 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.943+0000 7fa4c6ffd640 1 -- 172.21.15.52:0/71077402 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4053070 con 0x561949af2b70 2024-06-02T23:33:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.943+0000 7fa4c6ffd640 1 -- 172.21.15.52:0/71077402 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d4036050 con 0x561949af2b70 2024-06-02T23:33:18.945 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.943+0000 7fa4c6ffd640 1 -- 172.21.15.52:0/71077402 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d404aab0 con 0x561949af2b70 2024-06-02T23:33:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 -- 172.21.15.52:0/71077402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 msgr2=0x561949affe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949affe50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4006010 tx=0x7fa4d4047000 comp rx=0 tx=0).stop 2024-06-02T23:33:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 -- 172.21.15.52:0/71077402 shutdown_connections 2024-06-02T23:33:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aeda00 0x561949b07000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949affe50 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 --2- 172.21.15.52:0/71077402 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aed010 0x561949ae39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 -- 172.21.15.52:0/71077402 >> 172.21.15.52:0/71077402 conn(0x561949ae8680 msgr2=0x561949aff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 -- 172.21.15.52:0/71077402 shutdown_connections 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.944+0000 7fa4e6067740 1 -- 172.21.15.52:0/71077402 wait complete. 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 -- start start 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 Processor -- start 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 0x561949b06ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aeda00 0x561949b037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b03d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a7f50 con 0x561949aeda00 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a7880 con 0x561949af2b70 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc1a4ac0 con 0x561949aed010 2024-06-02T23:33:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e6067740 10 client.0 _ll_register_callbacks cb 0x7fa4cc009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e08f5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aeda00 0x561949b037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 0x561949b06ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e18f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b03d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e10f6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 0x561949b06ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42140/0 (socket says 172.21.15.52:42140) 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3710390592 learned_addr learned my addr 172.21.15.52:0/3710390592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.945+0000 7fa4e08f5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aeda00 0x561949b037d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60232/0 (socket says 172.21.15.52:60232) 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 msgr2=0x561949b03d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b03d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3710390592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aeda00 msgr2=0x561949b037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aeda00 0x561949b037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e10f6640 1 -- 172.21.15.52:0/3710390592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561949b04350 con 0x561949aed010 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e10f6640 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 0x561949b06ce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8099ef0 tx=0x7fa4d809c260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53636ccbfb6c39c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4c4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3710390592 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d809ba10 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3710390592 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa4d809b2a0 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa4d4036050 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3710390592 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa4d809b780 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e6067740 10 client.29923 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561949b04350 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.946+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3710390592 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa4d4036050 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3710390592 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa4d807aab0 con 0x561949aed010 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e6067740 10 client.29923 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e6067740 10 client.29923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e6067740 20 client.29923 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b75620 0x561949b77a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e6067740 4 client.29923 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:33:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561949a57f70 con 0x561949b75620 2024-06-02T23:33:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b75620 0x561949b77a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4c4ff9640 10 client.29923 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.947+0000 7fa4e08f5640 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b75620 0x561949b77a40 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60e02849b56aed5c server_cookie=2be1f623d0460112 in_seq=0 out_seq=0 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4c4ff9640 1 -- 172.21.15.52:0/3710390592 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561949a57f70 con 0x561949b75620 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4c4ff9640 10 client.29923 handle_command_reply: tid=0 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 client.29923 shutdown 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 msgr2=0x561949b06ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 0x561949b06ce0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8099ef0 tx=0x7fa4d809c260 comp rx=0 tx=0).stop 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 shutdown_connections 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561949b75620 0x561949b77a40 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561949af2b70 0x561949b03d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.991 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4c4ff9640 0 client.29923 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561949aeda00 0x561949b037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 --2- 172.21.15.52:0/3710390592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561949aed010 0x561949b06ce0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.990+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 >> 172.21.15.52:0/3710390592 conn(0x561949ae8680 msgr2=0x561949aff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 shutdown_connections 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e6067740 1 -- 172.21.15.52:0/3710390592 wait complete. 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e6067740 20 client.29923 trim_cache size 0 max 16384 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:33:18.992 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:33:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa4c80a7040 msgr2=0x7fa4c80a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e335a640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa4c80a7040 0x7fa4c80a9500 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4004af0 tx=0x7fa4d4009980 comp rx=0 tx=0).stop 2024-06-02T23:33:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 msgr2=0x7fa4dc195930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.991+0000 7fa4e335a640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc195930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa4d800e9c0 tx=0x7fa4d800ee80 comp rx=0 tx=0).stop 2024-06-02T23:33:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 shutdown_connections 2024-06-02T23:33:18.993 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa4c80a7040 0x7fa4c80a9500 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa4dc102c80 0x7fa4dc195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa4dc103640 0x7fa4dc19ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 --2- 172.21.15.52:0/3340425242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4dc0ffcb0 0x7fa4dc195930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 >> 172.21.15.52:0/3340425242 conn(0x7fa4dc0fb600 msgr2=0x7fa4dc105f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 shutdown_connections 2024-06-02T23:33:18.994 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:18.992+0000 7fa4e335a640 1 -- 172.21.15.52:0/3340425242 wait complete. 2024-06-02T23:33:19.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 61459 6.80 MB/sec execute 387 sec latency 69.550 ms 2024-06-02T23:33:19.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 -- 172.21.15.52:0/748375191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 msgr2=0x7f8a8c10ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 --2- 172.21.15.52:0/748375191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10ea00 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009990 tx=0x7f8a80009ca0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 -- 172.21.15.52:0/748375191 shutdown_connections 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 --2- 172.21.15.52:0/748375191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 0x7f8a8c111330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 --2- 172.21.15.52:0/748375191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 0x7f8a8c06a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 --2- 172.21.15.52:0/748375191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10ea00 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 -- 172.21.15.52:0/748375191 >> 172.21.15.52:0/748375191 conn(0x7f8a8c078340 msgr2=0x7f8a8c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 -- 172.21.15.52:0/748375191 shutdown_connections 2024-06-02T23:33:19.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.887+0000 7f8a90d14700 1 -- 172.21.15.52:0/748375191 wait complete. 2024-06-02T23:33:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 Processor -- start 2024-06-02T23:33:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 -- start start 2024-06-02T23:33:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 0x7f8a8c10e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 0x7f8a8c10bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1148c0 con 0x7f8a8c10ef40 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c114740 con 0x7f8a8c06aac0 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a90d14700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c114a40 con 0x7f8a8c06a1a0 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 0x7f8a8c10e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a8ad9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 0x7f8a8c10bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 0x7f8a8c10e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42160/0 (socket says 172.21.15.52:42160) 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59420/0 (socket says 172.21.15.52:59420) 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.888+0000 7f8a8a59c700 1 -- 172.21.15.52:0/62464450 learned_addr learned my addr 172.21.15.52:0/62464450 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a8ad9d700 1 -- 172.21.15.52:0/62464450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 msgr2=0x7f8a8c10e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 0x7f8a8c10e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a8ad9d700 1 -- 172.21.15.52:0/62464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 msgr2=0x7f8a8c10b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a8ad9d700 1 -- 172.21.15.52:0/62464450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c1ab600 con 0x7f8a8c10ef40 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 0x7f8a8c10bfa0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c069d60 tx=0x7f8a8400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a7b7fe700 1 -- 172.21.15.52:0/62464450 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a84003a00 con 0x7f8a8c10ef40 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a7b7fe700 1 -- 172.21.15.52:0/62464450 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a8400ce50 con 0x7f8a8c10ef40 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c1ab7d0 con 0x7f8a8c10ef40 2024-06-02T23:33:19.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a7b7fe700 1 -- 172.21.15.52:0/62464450 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a840046e0 con 0x7f8a8c10ef40 2024-06-02T23:33:19.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.889+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c1abcc0 con 0x7f8a8c10ef40 2024-06-02T23:33:19.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.893+0000 7f8a7b7fe700 1 -- 172.21.15.52:0/62464450 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8a84012050 con 0x7f8a8c10ef40 2024-06-02T23:33:19.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.894+0000 7f8a7b7fe700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a740a2c20 0x7f8a740a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.894+0000 7f8a7b7fe700 1 -- 172.21.15.52:0/62464450 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8a840ce8b0 con 0x7f8a8c10ef40 2024-06-02T23:33:19.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.894+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a740a2c20 0x7f8a740a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.894+0000 7f8a9b597b80 1 Processor -- start 2024-06-02T23:33:19.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.894+0000 7f8a9b597b80 1 -- start start 2024-06-02T23:33:19.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a740a2c20 0x7f8a740a50e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009db0 tx=0x7f8a7c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:19.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db686340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db680260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db680bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f1db572f60 con 0x55f1db686880 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1db5d1850 con 0x55f1db685f60 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1db539460 con 0x55f1db6807a0 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db680260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db680260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60260/0 (socket says 172.21.15.52:60260) 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 -- 172.21.15.52:0/735714264 learned_addr learned my addr 172.21.15.52:0/735714264 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db686340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db680bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 -- 172.21.15.52:0/735714264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 msgr2=0x55f1db680bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db680bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 -- 172.21.15.52:0/735714264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 msgr2=0x55f1db686340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db686340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 -- 172.21.15.52:0/735714264 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1db5208d0 con 0x55f1db686880 2024-06-02T23:33:19.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db680bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:19.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db686340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:19.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.895+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db680260 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009990 tx=0x7f8a80005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b090d7f52873746 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:19.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.896+0000 7f8a78ff9700 1 -- 172.21.15.52:0/735714264 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80027070 con 0x55f1db686880 2024-06-02T23:33:19.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.896+0000 7f8a78ff9700 1 -- 172.21.15.52:0/735714264 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a80005520 con 0x55f1db686880 2024-06-02T23:33:19.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.896+0000 7f8a78ff9700 1 -- 172.21.15.52:0/735714264 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000f460 con 0x55f1db686880 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 -- 172.21.15.52:0/735714264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 msgr2=0x55f1db680260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db680260 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009990 tx=0x7f8a80005be0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 -- 172.21.15.52:0/735714264 shutdown_connections 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db680bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db686340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/735714264 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db680260 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.897+0000 7f8a9b597b80 1 -- 172.21.15.52:0/735714264 >> 172.21.15.52:0/735714264 conn(0x55f1db5ad080 msgr2=0x55f1db67b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.898+0000 7f8a9b597b80 1 -- 172.21.15.52:0/735714264 shutdown_connections 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.898+0000 7f8a9b597b80 1 -- 172.21.15.52:0/735714264 wait complete. 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.898+0000 7f8a9b597b80 1 -- start start 2024-06-02T23:33:19.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.898+0000 7f8a9b597b80 1 Processor -- start 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db68c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db689910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db689e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f1db571fc0 con 0x55f1db686880 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1db5cd810 con 0x55f1db685f60 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f1db68d200 con 0x55f1db6807a0 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a9b597b80 10 client.0 _ll_register_callbacks cb 0x55f1db5675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db689910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db689910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59436/0 (socket says 172.21.15.52:59436) 2024-06-02T23:33:19.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 -- 172.21.15.52:0/4256293463 learned_addr learned my addr 172.21.15.52:0/4256293463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db689e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db68c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 -- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 msgr2=0x55f1db68c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db68c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 -- 172.21.15.52:0/4256293463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 msgr2=0x55f1db689e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db689e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.899+0000 7f8a89d9b700 1 -- 172.21.15.52:0/4256293463 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1db597750 con 0x55f1db685f60 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db689910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009ce0 tx=0x7f8a80004ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52066666fc41a982 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4256293463 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80027070 con 0x55f1db685f60 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4256293463 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8a800058e0 con 0x55f1db685f60 2024-06-02T23:33:19.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a80005520 con 0x55f1db685f60 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4256293463 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8a800056d0 con 0x55f1db685f60 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a9b597b80 10 client.30308 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f1db697920 con 0x55f1db685f60 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.900+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4256293463 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a80005520 con 0x55f1db685f60 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4256293463 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8a80021030 con 0x55f1db685f60 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a9b597b80 10 client.30308 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a9b597b80 10 client.30308 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a9b597b80 20 client.30308 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db699df0 0x55f1db69c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a9b597b80 4 client.30308 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1db5ded50 con 0x55f1db699df0 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db699df0 0x55f1db69c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db699df0 0x55f1db69c1e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bf3b57ed2528c480 server_cookie=d0ac7c9c95a2bc6 in_seq=0 out_seq=0 2024-06-02T23:33:19.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.901+0000 7f8a71ffb700 10 client.30308 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4256293463 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f1db5ded50 con 0x55f1db699df0 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a71ffb700 10 client.30308 handle_command_reply: tid=0 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 client.30308 shutdown 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 msgr2=0x55f1db689910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db689910 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8a80009ce0 tx=0x7f8a80004ec0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 shutdown_connections 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db689e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db699df0 0x55f1db69c1e0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a71ffb700 0 client.30308 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:33:19.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db6807a0 0x55f1db68c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4256293463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db685f60 0x55f1db689910 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 >> 172.21.15.52:0/4256293463 conn(0x55f1db5ad080 msgr2=0x55f1db67a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:19.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.902+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 shutdown_connections 2024-06-02T23:33:19.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.903+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4256293463 wait complete. 2024-06-02T23:33:19.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.903+0000 7f8a9b597b80 20 client.30308 trim_cache size 0 max 16384 2024-06-02T23:33:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 Processor -- start 2024-06-02T23:33:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 -- start start 2024-06-02T23:33:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db687430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db67ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db68a160 0x55f1db67b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b3080 con 0x55f1db686880 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b4bc0 con 0x55f1db6807a0 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b3750 con 0x55f1db68a160 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db67ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a89d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db67ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60280/0 (socket says 172.21.15.52:60280) 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a89d9b700 1 -- 172.21.15.52:0/2528643703 learned_addr learned my addr 172.21.15.52:0/2528643703 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db68a160 0x55f1db67b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.928+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db687430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a89d9b700 1 -- 172.21.15.52:0/2528643703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db68a160 msgr2=0x55f1db67b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db68a160 0x55f1db67b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a89d9b700 1 -- 172.21.15.52:0/2528643703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 msgr2=0x55f1db687430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db687430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a89d9b700 1 -- 172.21.15.52:0/2528643703 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1db687e80 con 0x55f1db686880 2024-06-02T23:33:19.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db67ae80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c010 tx=0x7f8a800092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca136e985b62aac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a73fff700 1 -- 172.21.15.52:0/2528643703 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80004fe0 con 0x55f1db686880 2024-06-02T23:33:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a73fff700 1 -- 172.21.15.52:0/2528643703 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a80028040 con 0x55f1db686880 2024-06-02T23:33:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.929+0000 7f8a73fff700 1 -- 172.21.15.52:0/2528643703 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8000f9a0 con 0x55f1db686880 2024-06-02T23:33:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 -- 172.21.15.52:0/2528643703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 msgr2=0x55f1db67ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db67ae80 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c010 tx=0x7f8a800092d0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 -- 172.21.15.52:0/2528643703 shutdown_connections 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db68a160 0x55f1db67b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db687430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/2528643703 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db686880 0x55f1db67ae80 secure :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000c010 tx=0x7f8a800092d0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 -- 172.21.15.52:0/2528643703 >> 172.21.15.52:0/2528643703 conn(0x55f1db5ad080 msgr2=0x55f1db686d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.930+0000 7f8a9b597b80 1 -- 172.21.15.52:0/2528643703 shutdown_connections 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.931+0000 7f8a9b597b80 1 -- 172.21.15.52:0/2528643703 wait complete. 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.931+0000 7f8a9b597b80 1 -- start start 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.931+0000 7f8a9b597b80 1 Processor -- start 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db69f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db69ccf0 0x55f1db69d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db69d6b0 0x55f1db703730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b44f0 con 0x55f1db69ccf0 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b3e20 con 0x55f1db6807a0 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c1b1040 con 0x55f1db69d6b0 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a9b597b80 10 client.0 _ll_register_callbacks cb 0x55f1db5675b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db69f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db69f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59448/0 (socket says 172.21.15.52:59448) 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 -- 172.21.15.52:0/4078792190 learned_addr learned my addr 172.21.15.52:0/4078792190 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8ad9d700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db69d6b0 0x55f1db703730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db69ccf0 0x55f1db69d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 -- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db69d6b0 msgr2=0x55f1db703730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db69d6b0 0x55f1db703730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 -- 172.21.15.52:0/4078792190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db69ccf0 msgr2=0x55f1db69d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db69ccf0 0x55f1db69d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 -- 172.21.15.52:0/4078792190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1db703cc0 con 0x55f1db6807a0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a8a59c700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db69f9f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c01d990 tx=0x7f8a7c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed10b0c4116b381 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.932+0000 7f8a71ffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.933+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4078792190 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c036060 con 0x55f1db6807a0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.933+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4078792190 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8a7c02a8d0 con 0x55f1db6807a0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.933+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a80028040 con 0x55f1db6807a0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.933+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4078792190 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8a7c02aad0 con 0x55f1db6807a0 2024-06-02T23:33:19.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.933+0000 7f8a9b597b80 10 client.30314 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.933+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f1db705030 con 0x55f1db6807a0 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4078792190 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f8a80028040 con 0x55f1db6807a0 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4078792190 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8a7c030030 con 0x55f1db6807a0 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a9b597b80 10 client.30314 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a9b597b80 10 client.30314 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a9b597b80 20 client.30314 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db7056d0 0x55f1db707af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a9b597b80 4 client.30314 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f1db5d92d0 con 0x55f1db7056d0 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.934+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db7056d0 0x55f1db707af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a89d9b700 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db7056d0 0x55f1db707af0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=884ed1623bf003b0 server_cookie=a79fdf12c35fabfa in_seq=0 out_seq=0 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a71ffb700 10 client.30314 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:33:19.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a71ffb700 1 -- 172.21.15.52:0/4078792190 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f1db5d92d0 con 0x55f1db7056d0 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a71ffb700 10 client.30314 handle_command_reply: tid=0 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 client.30314 shutdown 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 msgr2=0x55f1db69f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db69f9f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c01d990 tx=0x7f8a7c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 shutdown_connections 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55f1db7056d0 0x55f1db707af0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f1db69d6b0 0x55f1db703730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1db6807a0 0x55f1db69f9f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 --2- 172.21.15.52:0/4078792190 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f1db69ccf0 0x55f1db69d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a71ffb700 0 client.30314 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.935+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 >> 172.21.15.52:0/4078792190 conn(0x55f1db5ad080 msgr2=0x55f1db688200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.936+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 shutdown_connections 2024-06-02T23:33:19.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.936+0000 7f8a9b597b80 1 -- 172.21.15.52:0/4078792190 wait complete. 2024-06-02T23:33:19.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.936+0000 7f8a9b597b80 20 client.30314 trim_cache size 0 max 16384 2024-06-02T23:33:19.939 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:33:19.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a740a2c20 msgr2=0x7f8a740a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a740a2c20 0x7f8a740a50e0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c009db0 tx=0x7f8a7c009260 comp rx=0 tx=0).stop 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 msgr2=0x7f8a8c10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 0x7f8a8c10bfa0 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c069d60 tx=0x7f8a8400c3a0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 shutdown_connections 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f8a740a2c20 0x7f8a740a50e0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8a8c10ef40 0x7f8a8c10bfa0 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8a8c06a1a0 0x7f8a8c10e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 --2- 172.21.15.52:0/62464450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a8c06aac0 0x7f8a8c10b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 >> 172.21.15.52:0/62464450 conn(0x7f8a8c078340 msgr2=0x7f8a8c110450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 shutdown_connections 2024-06-02T23:33:19.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:19.937+0000 7f8a90d14700 1 -- 172.21.15.52:0/62464450 wait complete. 2024-06-02T23:33:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:19 smithi174 ceph-mon[35994]: pgmap v1074: 97 pgs: 97 active+clean; 522 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-06-02T23:33:20.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:19 smithi052 ceph-mon[32340]: pgmap v1074: 97 pgs: 97 active+clean; 522 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-06-02T23:33:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:19 smithi123 ceph-mon[35977]: pgmap v1074: 97 pgs: 97 active+clean; 522 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 3.5 MiB/s wr, 237 op/s 2024-06-02T23:33:20.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 61610 6.79 MB/sec execute 388 sec latency 96.078 ms 2024-06-02T23:33:20.623 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:33:20.986 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:21.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 61788 6.80 MB/sec execute 389 sec latency 60.182 ms 2024-06-02T23:33:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:21 smithi174 ceph-mon[35994]: pgmap v1075: 97 pgs: 97 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-06-02T23:33:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:21 smithi052 ceph-mon[32340]: pgmap v1075: 97 pgs: 97 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-06-02T23:33:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:21 smithi123 ceph-mon[35977]: pgmap v1075: 97 pgs: 97 active+clean; 522 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 3.7 MiB/s wr, 257 op/s 2024-06-02T23:33:22.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 61942 6.80 MB/sec execute 390 sec latency 72.053 ms 2024-06-02T23:33:22.877 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 -- 172.21.15.52:0/2595150117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c1024e0 msgr2=0x7fa52c102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 --2- 172.21.15.52:0/2595150117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c1024e0 0x7fa52c102960 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fa520009990 tx=0x7fa520009ca0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 -- 172.21.15.52:0/2595150117 shutdown_connections 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 --2- 172.21.15.52:0/2595150117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c102ea0 0x7fa52c10f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 --2- 172.21.15.52:0/2595150117 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 0x7fa52c1089d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 --2- 172.21.15.52:0/2595150117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c1024e0 0x7fa52c102960 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 -- 172.21.15.52:0/2595150117 >> 172.21.15.52:0/2595150117 conn(0x7fa52c079640 msgr2=0x7fa52c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 -- 172.21.15.52:0/2595150117 shutdown_connections 2024-06-02T23:33:22.878 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.877+0000 7fa532902700 1 -- 172.21.15.52:0/2595150117 wait complete. 2024-06-02T23:33:22.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.879+0000 7fa532902700 1 Processor -- start 2024-06-02T23:33:22.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.879+0000 7fa532902700 1 -- start start 2024-06-02T23:33:22.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa532902700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c1024e0 0x7fa52c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa532902700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c102ea0 0x7fa52c19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa52bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c1024e0 0x7fa52c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa52bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c1024e0 0x7fa52c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42230/0 (socket says 172.21.15.52:42230) 2024-06-02T23:33:22.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa52bfff700 1 -- 172.21.15.52:0/4092365143 learned_addr learned my addr 172.21.15.52:0/4092365143 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:22.880 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 0x7fa52c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c102ea0 0x7fa52c19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c112640 con 0x7fa52c1085f0 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.880+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1124c0 con 0x7fa52c102ea0 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c1127c0 con 0x7fa52c1024e0 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa530e9f700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 0x7fa52c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa52b7fe700 1 -- 172.21.15.52:0/4092365143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c1024e0 msgr2=0x7fa52c19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c1024e0 0x7fa52c19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa52b7fe700 1 -- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 msgr2=0x7fa52c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 0x7fa52c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa52b7fe700 1 -- 172.21.15.52:0/4092365143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa52c1990c0 con 0x7fa52c102ea0 2024-06-02T23:33:22.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa530e9f700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 0x7fa52c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:22.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.881+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c102ea0 0x7fa52c19ece0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa520006010 tx=0x7fa5200057a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:22.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.882+0000 7fa5297fa700 1 -- 172.21.15.52:0/4092365143 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa520026070 con 0x7fa52c102ea0 2024-06-02T23:33:22.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.882+0000 7fa5297fa700 1 -- 172.21.15.52:0/4092365143 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa520021410 con 0x7fa52c102ea0 2024-06-02T23:33:22.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.882+0000 7fa5297fa700 1 -- 172.21.15.52:0/4092365143 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52000b610 con 0x7fa52c102ea0 2024-06-02T23:33:22.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.882+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa52c199350 con 0x7fa52c102ea0 2024-06-02T23:33:22.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.882+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa52c0746c0 con 0x7fa52c102ea0 2024-06-02T23:33:22.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.885+0000 7fa5297fa700 1 -- 172.21.15.52:0/4092365143 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa520029070 con 0x7fa52c102ea0 2024-06-02T23:33:22.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.887+0000 7fa53d185b80 1 Processor -- start 2024-06-02T23:33:22.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.887+0000 7fa5297fa700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5140a2ca0 0x7fa5140a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.887+0000 7fa53d185b80 1 -- start start 2024-06-02T23:33:22.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.887+0000 7fa5297fa700 1 -- 172.21.15.52:0/4092365143 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa5200cfbd0 con 0x7fa52c102ea0 2024-06-02T23:33:22.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa530e9f700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5140a2ca0 0x7fa5140a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa52bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 0x56326b8269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b827340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa53d185b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56326b778810 con 0x56326b82c640 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa53d185b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56326b720720 con 0x56326b826f20 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa53d185b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56326b7747d0 con 0x56326b82cf60 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.888+0000 7fa530e9f700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5140a2ca0 0x7fa5140a5160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa524010c30 tx=0x7fa524012040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 0x56326b8269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 0x56326b8269e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42240/0 (socket says 172.21.15.52:42240) 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 -- 172.21.15.52:0/617980242 learned_addr learned my addr 172.21.15.52:0/617980242 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa530e9f700 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b827340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 -- 172.21.15.52:0/617980242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 msgr2=0x56326b827340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b827340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 -- 172.21.15.52:0/617980242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 msgr2=0x56326b82ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 -- 172.21.15.52:0/617980242 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326b716f50 con 0x56326b82cf60 2024-06-02T23:33:22.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 0x56326b8269e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa52009c1f0 tx=0x7fa52009c030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78553412e2488a2b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:22.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa5127fc700 1 -- 172.21.15.52:0/617980242 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52009c4f0 con 0x56326b82cf60 2024-06-02T23:33:22.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa5127fc700 1 -- 172.21.15.52:0/617980242 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5200a08a0 con 0x56326b82cf60 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.889+0000 7fa5127fc700 1 -- 172.21.15.52:0/617980242 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52009c750 con 0x56326b82cf60 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 -- 172.21.15.52:0/617980242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 msgr2=0x56326b8269e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 0x56326b8269e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa52009c1f0 tx=0x7fa52009c030 comp rx=0 tx=0).stop 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 -- 172.21.15.52:0/617980242 shutdown_connections 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b827340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 --2- 172.21.15.52:0/617980242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82cf60 0x56326b8269e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.890+0000 7fa53d185b80 1 -- 172.21.15.52:0/617980242 >> 172.21.15.52:0/617980242 conn(0x56326b751cc0 msgr2=0x56326b821850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.891+0000 7fa53d185b80 1 -- 172.21.15.52:0/617980242 shutdown_connections 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.891+0000 7fa53d185b80 1 -- 172.21.15.52:0/617980242 wait complete. 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.891+0000 7fa53d185b80 1 -- start start 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 Processor -- start 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b832d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b830040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82cf60 0x56326b830580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56326b8336f0 con 0x56326b82cf60 2024-06-02T23:33:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56326b718bf0 con 0x56326b826f20 2024-06-02T23:33:22.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56326b833870 con 0x56326b82c640 2024-06-02T23:33:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa530e9f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82cf60 0x56326b830580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa52b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b830040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa52b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b830040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42242/0 (socket says 172.21.15.52:42242) 2024-06-02T23:33:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa530e9f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82cf60 0x56326b830580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60310/0 (socket says 172.21.15.52:60310) 2024-06-02T23:33:22.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa52b7fe700 1 -- 172.21.15.52:0/1659739766 learned_addr learned my addr 172.21.15.52:0/1659739766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.892+0000 7fa53d185b80 10 client.0 _ll_register_callbacks cb 0x56326b712920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa52b7fe700 1 -- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 msgr2=0x56326b832d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b832d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa52b7fe700 1 -- 172.21.15.52:0/1659739766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82cf60 msgr2=0x56326b830580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82cf60 0x56326b830580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa52b7fe700 1 -- 172.21.15.52:0/1659739766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56326b6c78d0 con 0x56326b82c640 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b830040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa5200cd750 tx=0x7fa52009cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdc514d4b4a4db03 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa50b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1659739766 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5200b5210 con 0x56326b82c640 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1659739766 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa5200b53b0 con 0x56326b82c640 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.893+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5200a08a0 con 0x56326b82c640 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.894+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1659739766 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa52009c750 con 0x56326b82c640 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.894+0000 7fa53d185b80 10 client.29950 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:22.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.894+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56326b8336f0 con 0x56326b82c640 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.894+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1659739766 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa5200a08a0 con 0x56326b82c640 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1659739766 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa5200cdbc0 con 0x56326b82c640 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa53d185b80 10 client.29950 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa53d185b80 10 client.29950 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa53d185b80 20 client.29950 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b84af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa53d185b80 4 client.29950 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:22.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56326b785d10 con 0x56326b848b00 2024-06-02T23:33:22.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.895+0000 7fa52bfff700 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b84af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.896+0000 7fa52bfff700 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b84af10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d24352ba3acec194 server_cookie=41ddaaf8f8922123 in_seq=0 out_seq=0 2024-06-02T23:33:22.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.896+0000 7fa50b7fe700 10 client.29950 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.896+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1659739766 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56326b785d10 con 0x56326b848b00 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.896+0000 7fa50b7fe700 10 client.29950 handle_command_reply: tid=0 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.896+0000 7fa53d185b80 1 client.29950 shutdown 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 msgr2=0x56326b830040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b830040 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fa5200cd750 tx=0x7fa52009cd20 comp rx=0 tx=0).stop 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 shutdown_connections 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b84af10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa50b7fe700 0 client.29950 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82cf60 0x56326b830580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b826f20 0x56326b832d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1659739766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b830040 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 >> 172.21.15.52:0/1659739766 conn(0x56326b751cc0 msgr2=0x56326b8259d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 shutdown_connections 2024-06-02T23:33:22.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.897+0000 7fa53d185b80 1 -- 172.21.15.52:0/1659739766 wait complete. 2024-06-02T23:33:22.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.898+0000 7fa53d185b80 20 client.29950 trim_cache size 0 max 16384 2024-06-02T23:33:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.927+0000 7fa53d185b80 1 Processor -- start 2024-06-02T23:33:22.927 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 -- start start 2024-06-02T23:33:22.928 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 0x56326b8269e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b82db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c071030 con 0x56326b82c640 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c072b70 con 0x56326b82cf60 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa53d185b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c071700 con 0x56326b826f20 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa52b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa52bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 0x56326b8269e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa52b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:60318/0 (socket says 172.21.15.52:60318) 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa52bfff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 0x56326b8269e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:42248/0 (socket says 172.21.15.52:42248) 2024-06-02T23:33:22.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.928+0000 7fa52b7fe700 1 -- 172.21.15.52:0/2870499541 learned_addr learned my addr 172.21.15.52:0/2870499541 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52bfff700 1 -- 172.21.15.52:0/2870499541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 msgr2=0x56326b82db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52bfff700 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b82db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52bfff700 1 -- 172.21.15.52:0/2870499541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 msgr2=0x56326b82f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52bfff700 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52bfff700 1 -- 172.21.15.52:0/2870499541 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56326b822520 con 0x56326b826f20 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52b7fe700 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:22.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.929+0000 7fa52bfff700 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 0x56326b8269e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56326b84b760 tx=0x7fa51c005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2996b5402450d275 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.930+0000 7fa511ffb700 1 -- 172.21.15.52:0/2870499541 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c021070 con 0x56326b826f20 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.930+0000 7fa511ffb700 1 -- 172.21.15.52:0/2870499541 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa51c004c70 con 0x56326b826f20 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.930+0000 7fa511ffb700 1 -- 172.21.15.52:0/2870499541 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c003b70 con 0x56326b826f20 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.930+0000 7fa53d185b80 1 -- 172.21.15.52:0/2870499541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 msgr2=0x56326b8269e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.930+0000 7fa53d185b80 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 0x56326b8269e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x56326b84b760 tx=0x7fa51c005320 comp rx=0 tx=0).stop 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 -- 172.21.15.52:0/2870499541 shutdown_connections 2024-06-02T23:33:22.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b82db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b826f20 0x56326b8269e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 --2- 172.21.15.52:0/2870499541 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b82c640 0x56326b82f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 -- 172.21.15.52:0/2870499541 >> 172.21.15.52:0/2870499541 conn(0x56326b751cc0 msgr2=0x56326b82d400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:22.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 -- 172.21.15.52:0/2870499541 shutdown_connections 2024-06-02T23:33:22.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.931+0000 7fa53d185b80 1 -- 172.21.15.52:0/2870499541 wait complete. 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.932+0000 7fa53d185b80 1 -- start start 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.932+0000 7fa53d185b80 1 Processor -- start 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b826f20 0x56326b841310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b841850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b83e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c0724a0 con 0x56326b826f20 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c071dd0 con 0x56326b82cf60 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa52c06eff0 con 0x56326b82c640 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa53d185b80 10 client.0 _ll_register_callbacks cb 0x56326b712920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b83e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa52bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b826f20 0x56326b841310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b83e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:59498/0 (socket says 172.21.15.52:59498) 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 -- 172.21.15.52:0/1832164470 learned_addr learned my addr 172.21.15.52:0/1832164470 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 -- 172.21.15.52:0/1832164470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 msgr2=0x56326b841850 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b841850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 -- 172.21.15.52:0/1832164470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b826f20 msgr2=0x56326b841310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b826f20 0x56326b841310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.933+0000 7fa530e9f700 1 -- 172.21.15.52:0/1832164470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56326b841f70 con 0x56326b82cf60 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa530e9f700 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b83e2e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa524015ff0 tx=0x7fa524009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a36dcab46c7968db server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa50b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1832164470 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa524009e60 con 0x56326b82cf60 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1832164470 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa524027420 con 0x56326b82cf60 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa51c004c70 con 0x56326b82cf60 2024-06-02T23:33:22.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1832164470 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa524008e30 con 0x56326b82cf60 2024-06-02T23:33:22.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa53d185b80 10 client.30326 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:22.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.934+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56326b8b3e30 con 0x56326b82cf60 2024-06-02T23:33:22.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1832164470 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa51c004c70 con 0x56326b82cf60 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1832164470 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa52401c7e0 con 0x56326b82cf60 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa53d185b80 10 client.30326 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa53d185b80 10 client.30326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa53d185b80 20 client.30326 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b8b6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa53d185b80 4 client.30326 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56326b780290 con 0x56326b848b00 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.935+0000 7fa52bfff700 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b8b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa50b7fe700 10 client.30326 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa52bfff700 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b8b6600 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8b8268ea0f91d01 server_cookie=8ef856d6e83b7d58 in_seq=0 out_seq=0 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa50b7fe700 1 -- 172.21.15.52:0/1832164470 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56326b780290 con 0x56326b848b00 2024-06-02T23:33:22.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa50b7fe700 10 client.30326 handle_command_reply: tid=0 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 client.30326 shutdown 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 msgr2=0x56326b83e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b83e2e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fa524015ff0 tx=0x7fa524009920 comp rx=0 tx=0).stop 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 shutdown_connections 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x56326b848b00 0x56326b8b6600 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56326b82cf60 0x56326b83e2e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56326b826f20 0x56326b841310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 --2- 172.21.15.52:0/1832164470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56326b82c640 0x56326b841850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 >> 172.21.15.52:0/1832164470 conn(0x56326b751cc0 msgr2=0x56326b82d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.936+0000 7fa50b7fe700 0 client.30326 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.937+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 shutdown_connections 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.937+0000 7fa53d185b80 1 -- 172.21.15.52:0/1832164470 wait complete. 2024-06-02T23:33:22.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.937+0000 7fa53d185b80 20 client.30326 trim_cache size 0 max 16384 2024-06-02T23:33:22.940 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5140a2ca0 msgr2=0x7fa5140a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5140a2ca0 0x7fa5140a5160 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fa524010c30 tx=0x7fa524012040 comp rx=0 tx=0).stop 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c102ea0 msgr2=0x7fa52c19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c102ea0 0x7fa52c19ece0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa520006010 tx=0x7fa5200057a0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 shutdown_connections 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa5140a2ca0 0x7fa5140a5160 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa52c1085f0 0x7fa52c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa52c1024e0 0x7fa52c19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 --2- 172.21.15.52:0/4092365143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa52c102ea0 0x7fa52c19ece0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:22.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 >> 172.21.15.52:0/4092365143 conn(0x7fa52c079640 msgr2=0x7fa52c1010c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:22.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 shutdown_connections 2024-06-02T23:33:22.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:22.938+0000 7fa532902700 1 -- 172.21.15.52:0/4092365143 wait complete. 2024-06-02T23:33:23.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 62102 6.80 MB/sec execute 391 sec latency 84.342 ms 2024-06-02T23:33:24.115 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:23 smithi052 ceph-mon[32340]: pgmap v1076: 97 pgs: 97 active+clean; 522 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-06-02T23:33:24.117 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:33:24.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:23 smithi174 ceph-mon[35994]: pgmap v1076: 97 pgs: 97 active+clean; 522 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-06-02T23:33:24.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:23 smithi123 ceph-mon[35977]: pgmap v1076: 97 pgs: 97 active+clean; 522 MiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.3 MiB/s wr, 274 op/s 2024-06-02T23:33:24.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 62268 6.81 MB/sec execute 392 sec latency 57.758 ms 2024-06-02T23:33:24.507 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:25.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 62467 6.83 MB/sec execute 393 sec latency 64.176 ms 2024-06-02T23:33:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:25 smithi052 ceph-mon[32340]: pgmap v1077: 97 pgs: 97 active+clean; 524 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 257 op/s 2024-06-02T23:33:26.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:25 smithi174 ceph-mon[35994]: pgmap v1077: 97 pgs: 97 active+clean; 524 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 257 op/s 2024-06-02T23:33:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:25 smithi123 ceph-mon[35977]: pgmap v1077: 97 pgs: 97 active+clean; 524 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 257 op/s 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.457+0000 7f03b9b90700 1 -- 172.21.15.52:0/2015637474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b41084e0 msgr2=0x7f03b41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.457+0000 7f03b9b90700 1 --2- 172.21.15.52:0/2015637474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b41084e0 0x7f03b41088c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f03a40099c0 tx=0x7f03a4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 -- 172.21.15.52:0/2015637474 shutdown_connections 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 --2- 172.21.15.52:0/2015637474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03b406ab10 0x7f03b410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 --2- 172.21.15.52:0/2015637474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b41084e0 0x7f03b41088c0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 --2- 172.21.15.52:0/2015637474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406a170 0x7f03b406a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 -- 172.21.15.52:0/2015637474 >> 172.21.15.52:0/2015637474 conn(0x7f03b40782c0 msgr2=0x7f03b40786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 -- 172.21.15.52:0/2015637474 shutdown_connections 2024-06-02T23:33:26.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.458+0000 7f03b9b90700 1 -- 172.21.15.52:0/2015637474 wait complete. 2024-06-02T23:33:26.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.459+0000 7f03b9b90700 1 Processor -- start 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.459+0000 7f03b9b90700 1 -- start start 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b9b90700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03b406a170 0x7f03b4198e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b9b90700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406ab10 0x7f03b4199390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b9b90700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b419d6d0 0x7f03b419db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b9b90700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f03b4112670 con 0x7f03b419d6d0 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b9b90700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41124f0 con 0x7f03b406ab10 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b9b90700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41127f0 con 0x7f03b406a170 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406ab10 0x7f03b4199390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406ab10 0x7f03b4199390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56950/0 (socket says 172.21.15.52:56950) 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b2ffd700 1 -- 172.21.15.52:0/460737438 learned_addr learned my addr 172.21.15.52:0/460737438 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b37fe700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03b406a170 0x7f03b4198e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b3fff700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b419d6d0 0x7f03b419db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b3fff700 1 -- 172.21.15.52:0/460737438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03b406a170 msgr2=0x7f03b4198e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b3fff700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03b406a170 0x7f03b4198e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.460+0000 7f03b3fff700 1 -- 172.21.15.52:0/460737438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406ab10 msgr2=0x7f03b4199390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b3fff700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406ab10 0x7f03b4199390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b3fff700 1 -- 172.21.15.52:0/460737438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b419e1f0 con 0x7f03b419d6d0 2024-06-02T23:33:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b3fff700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b419d6d0 0x7f03b419db50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00bb00 tx=0x7f03ac00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b0ff9700 1 -- 172.21.15.52:0/460737438 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac019070 con 0x7f03b419d6d0 2024-06-02T23:33:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b419e4e0 con 0x7f03b419d6d0 2024-06-02T23:33:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b0ff9700 1 -- 172.21.15.52:0/460737438 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac014440 con 0x7f03b419d6d0 2024-06-02T23:33:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b0ff9700 1 -- 172.21.15.52:0/460737438 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00ce60 con 0x7f03b419d6d0 2024-06-02T23:33:26.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.461+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b41a5cc0 con 0x7f03b419d6d0 2024-06-02T23:33:26.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.464+0000 7f03b0ff9700 1 -- 172.21.15.52:0/460737438 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f03ac01c070 con 0x7f03b419d6d0 2024-06-02T23:33:26.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.465+0000 7f03b0ff9700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f039c0a2ca0 0x7f039c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.464 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.465+0000 7f03b0ff9700 1 -- 172.21.15.52:0/460737438 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f03ac0ce3e0 con 0x7f03b419d6d0 2024-06-02T23:33:26.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.465+0000 7f03b37fe700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f039c0a2ca0 0x7f039c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.466+0000 7f03b37fe700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f039c0a2ca0 0x7f039c0a5160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f03a4000c00 tx=0x7f03a4005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:26.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.466+0000 7f03c4413b80 1 Processor -- start 2024-06-02T23:33:26.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.466+0000 7f03c4413b80 1 -- start start 2024-06-02T23:33:26.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aaddf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aaddfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03c4413b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597aacd1f60 con 0x5597aade4f60 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03c4413b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597aad30850 con 0x5597aaddf7a0 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03c4413b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597aac98460 con 0x5597aade5880 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58434/0 (socket says 172.21.15.52:58434) 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b37fe700 1 -- 172.21.15.52:0/4011243111 learned_addr learned my addr 172.21.15.52:0/4011243111 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b3fff700 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aaddfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aaddf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b3fff700 1 -- 172.21.15.52:0/4011243111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 msgr2=0x5597aaddf260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b3fff700 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aaddf260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b3fff700 1 -- 172.21.15.52:0/4011243111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 msgr2=0x5597aade5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b3fff700 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b3fff700 1 -- 172.21.15.52:0/4011243111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597aac7f8d0 con 0x5597aaddf7a0 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.467+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aaddf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.468+0000 7f03b3fff700 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aaddfbc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f03ac09a910 tx=0x7f03ac09a5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6a5369996b5c188 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.468+0000 7f0399ffb700 1 -- 172.21.15.52:0/4011243111 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac09ac10 con 0x5597aaddf7a0 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.468+0000 7f0399ffb700 1 -- 172.21.15.52:0/4011243111 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac09efc0 con 0x5597aaddf7a0 2024-06-02T23:33:26.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.468+0000 7f0399ffb700 1 -- 172.21.15.52:0/4011243111 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac09b0d0 con 0x5597aaddf7a0 2024-06-02T23:33:26.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 -- 172.21.15.52:0/4011243111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 msgr2=0x5597aaddfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aaddfbc0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f03ac09a910 tx=0x7f03ac09a5f0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 -- 172.21.15.52:0/4011243111 shutdown_connections 2024-06-02T23:33:26.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aaddfbc0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 --2- 172.21.15.52:0/4011243111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aaddf260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 -- 172.21.15.52:0/4011243111 >> 172.21.15.52:0/4011243111 conn(0x5597aad0c080 msgr2=0x5597aadda220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 -- 172.21.15.52:0/4011243111 shutdown_connections 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 -- 172.21.15.52:0/4011243111 wait complete. 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 -- start start 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.469+0000 7f03c4413b80 1 Processor -- start 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aadeb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aade8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aadeb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5597aacd0fc0 con 0x5597aade4f60 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5597aad2c810 con 0x5597aaddf7a0 2024-06-02T23:33:26.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5597aadec200 con 0x5597aade5880 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aadeb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56966/0 (socket says 172.21.15.52:56966) 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03b37fe700 1 -- 172.21.15.52:0/429662685 learned_addr learned my addr 172.21.15.52:0/429662685 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03c4413b80 10 client.0 _ll_register_callbacks cb 0x5597aacc65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.470+0000 7f03b3fff700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aade8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b3fff700 1 -- 172.21.15.52:0/429662685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 msgr2=0x5597aadeb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b3fff700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aadeb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b3fff700 1 -- 172.21.15.52:0/429662685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 msgr2=0x5597aade8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b3fff700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b3fff700 1 -- 172.21.15.52:0/429662685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597aacf6750 con 0x5597aade5880 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b37fe700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aadeb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:26.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03b3fff700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aade8ed0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5597aadebea0 tx=0x7f03ac0cc1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae4c4760ea960504 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f0392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f0392ffd700 1 -- 172.21.15.52:0/429662685 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac07f070 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03ac09efc0 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f0392ffd700 1 -- 172.21.15.52:0/429662685 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f03ac09b080 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f0392ffd700 1 -- 172.21.15.52:0/429662685 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f03ac0a9020 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03c4413b80 10 client.29959 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.471+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5597aadf6900 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f0392ffd700 1 -- 172.21.15.52:0/429662685 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03ac09efc0 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f0392ffd700 1 -- 172.21.15.52:0/429662685 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f03ac0a5b90 con 0x5597aade5880 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03c4413b80 10 client.29959 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03c4413b80 10 client.29959 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03c4413b80 20 client.29959 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03c4413b80 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aadf8dd0 0x5597aadfb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.475 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03c4413b80 4 client.29959 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597aad3dd50 con 0x5597aadf8dd0 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.472+0000 7f03b37fe700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aadf8dd0 0x5597aadfb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.473+0000 7f0392ffd700 10 client.29959 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.473+0000 7f03b37fe700 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aadf8dd0 0x5597aadfb1c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=68f8315ed90354bb server_cookie=d65faf5bee6830fa in_seq=0 out_seq=0 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f0392ffd700 1 -- 172.21.15.52:0/429662685 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5597aad3dd50 con 0x5597aadf8dd0 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f0392ffd700 10 client.29959 handle_command_reply: tid=0 2024-06-02T23:33:26.476 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 client.29959 shutdown 2024-06-02T23:33:26.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 62662 6.84 MB/sec execute 394 sec latency 93.423 ms 2024-06-02T23:33:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 msgr2=0x5597aade8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aade8ed0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x5597aadebea0 tx=0x7f03ac0cc1d0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 shutdown_connections 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aadf8dd0 0x5597aadfb1c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade5880 0x5597aade8ed0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aaddf7a0 0x5597aadeb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 --2- 172.21.15.52:0/429662685 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade4f60 0x5597aade8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 >> 172.21.15.52:0/429662685 conn(0x5597aad0c080 msgr2=0x5597aadd9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f0392ffd700 0 client.29959 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 shutdown_connections 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.474+0000 7f03c4413b80 1 -- 172.21.15.52:0/429662685 wait complete. 2024-06-02T23:33:26.478 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.475+0000 7f03c4413b80 20 client.29959 trim_cache size 0 max 16384 2024-06-02T23:33:26.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 Processor -- start 2024-06-02T23:33:26.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 -- start start 2024-06-02T23:33:26.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 0x5597aaddf260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.504 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade4f60 0x5597aade30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade5880 0x5597aadd9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41aae80 con 0x5597aaddf7a0 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41ac9c0 con 0x5597aade5880 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03c4413b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41ab550 con 0x5597aade4f60 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 0x5597aaddf260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 0x5597aaddf260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58458/0 (socket says 172.21.15.52:58458) 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03b37fe700 1 -- 172.21.15.52:0/526929814 learned_addr learned my addr 172.21.15.52:0/526929814 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.502+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade4f60 0x5597aade30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b3fff700 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade5880 0x5597aadd9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b37fe700 1 -- 172.21.15.52:0/526929814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade4f60 msgr2=0x5597aade30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b37fe700 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade4f60 0x5597aade30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b37fe700 1 -- 172.21.15.52:0/526929814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade5880 msgr2=0x5597aadd9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b37fe700 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade5880 0x5597aadd9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b37fe700 1 -- 172.21.15.52:0/526929814 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597aade98d0 con 0x5597aaddf7a0 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b3fff700 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade5880 0x5597aadd9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03b37fe700 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 0x5597aaddf260 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f03a40098e0 tx=0x7f03a401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f735179b2070f383 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.503+0000 7f03997fa700 1 -- 172.21.15.52:0/526929814 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4004710 con 0x5597aaddf7a0 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.504+0000 7f03997fa700 1 -- 172.21.15.52:0/526929814 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03a4038070 con 0x5597aaddf7a0 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.504+0000 7f03997fa700 1 -- 172.21.15.52:0/526929814 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a4034a70 con 0x5597aaddf7a0 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.504+0000 7f03c4413b80 1 -- 172.21.15.52:0/526929814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 msgr2=0x5597aaddf260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.504+0000 7f03c4413b80 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 0x5597aaddf260 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f03a40098e0 tx=0x7f03a401a050 comp rx=0 tx=0).stop 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 -- 172.21.15.52:0/526929814 shutdown_connections 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade5880 0x5597aadd9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aaddf7a0 0x5597aaddf260 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 --2- 172.21.15.52:0/526929814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aade4f60 0x5597aade30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 -- 172.21.15.52:0/526929814 >> 172.21.15.52:0/526929814 conn(0x5597aad0c080 msgr2=0x5597aade29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:26.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 -- 172.21.15.52:0/526929814 shutdown_connections 2024-06-02T23:33:26.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 -- 172.21.15.52:0/526929814 wait complete. 2024-06-02T23:33:26.507 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 -- start start 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.505+0000 7f03c4413b80 1 Processor -- start 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 0x5597aadfe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade4f60 0x5597aadfe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade5880 0x5597aadfb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41ac2f0 con 0x5597aade5880 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41abc20 con 0x5597aade4f60 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f03b41a8e60 con 0x5597aaddf7a0 2024-06-02T23:33:26.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 0x5597aadfe190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03c4413b80 10 client.0 _ll_register_callbacks cb 0x5597aacc65b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b37fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 0x5597aadfe190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41466/0 (socket says 172.21.15.52:41466) 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b37fe700 1 -- 172.21.15.52:0/2942972335 learned_addr learned my addr 172.21.15.52:0/2942972335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade4f60 0x5597aadfe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b3fff700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade5880 0x5597aadfb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b37fe700 1 -- 172.21.15.52:0/2942972335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade4f60 msgr2=0x5597aadfe6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.506+0000 7f03b37fe700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade4f60 0x5597aadfe6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03b37fe700 1 -- 172.21.15.52:0/2942972335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade5880 msgr2=0x5597aadfb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03b37fe700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade5880 0x5597aadfb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03b37fe700 1 -- 172.21.15.52:0/2942972335 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597aadfbf70 con 0x5597aaddf7a0 2024-06-02T23:33:26.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade4f60 0x5597aadfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03b3fff700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade5880 0x5597aadfb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03b37fe700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 0x5597aadfe190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009fd0 tx=0x7f03a4034430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e97cfc8df3616caf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f0392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f0392ffd700 1 -- 172.21.15.52:0/2942972335 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40348b0 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f0392ffd700 1 -- 172.21.15.52:0/2942972335 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f03a4034a50 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.507+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03a4038070 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f0392ffd700 1 -- 172.21.15.52:0/2942972335 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f03a4034c90 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 10 client.29965 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5597aae63370 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f0392ffd700 1 -- 172.21.15.52:0/2942972335 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f03a4038070 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f0392ffd700 1 -- 172.21.15.52:0/2942972335 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f03a400f880 con 0x5597aaddf7a0 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 10 client.29965 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 10 client.29965 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:33:26.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 20 client.29965 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aae63900 0x5597aae65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 4 client.29965 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5597aad382d0 con 0x5597aae63900 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.508+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aae63900 0x5597aae65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.509+0000 7f0392ffd700 10 client.29965 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.509+0000 7f03b2ffd700 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aae63900 0x5597aae65cf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=10324cbc6bcef641 server_cookie=2143370a8298edf4 in_seq=0 out_seq=0 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.509+0000 7f0392ffd700 1 -- 172.21.15.52:0/2942972335 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5597aad382d0 con 0x5597aae63900 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.509+0000 7f0392ffd700 10 client.29965 handle_command_reply: tid=0 2024-06-02T23:33:26.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.509+0000 7f03c4413b80 1 client.29965 shutdown 2024-06-02T23:33:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 msgr2=0x5597aadfe190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 0x5597aadfe190 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009fd0 tx=0x7f03a4034430 comp rx=0 tx=0).stop 2024-06-02T23:33:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 shutdown_connections 2024-06-02T23:33:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x5597aae63900 0x5597aae65cf0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5597aade5880 0x5597aadfb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5597aaddf7a0 0x5597aadfe190 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 --2- 172.21.15.52:0/2942972335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5597aade4f60 0x5597aadfe6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f0392ffd700 0 client.29965 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 >> 172.21.15.52:0/2942972335 conn(0x5597aad0c080 msgr2=0x5597aade5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 shutdown_connections 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 1 -- 172.21.15.52:0/2942972335 wait complete. 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.510+0000 7f03c4413b80 20 client.29965 trim_cache size 0 max 16384 2024-06-02T23:33:26.514 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:33:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.511+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f039c0a2ca0 msgr2=0x7f039c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.511+0000 7f03b9b90700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f039c0a2ca0 0x7f039c0a5160 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f03a4000c00 tx=0x7f03a4005fb0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.511+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b419d6d0 msgr2=0x7f03b419db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.511+0000 7f03b9b90700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b419d6d0 0x7f03b419db50 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00bb00 tx=0x7f03ac00be10 comp rx=0 tx=0).stop 2024-06-02T23:33:26.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 shutdown_connections 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f039c0a2ca0 0x7f039c0a5160 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f03b419d6d0 0x7f03b419db50 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f03b406a170 0x7f03b4198e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 --2- 172.21.15.52:0/460737438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b406ab10 0x7f03b4199390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 >> 172.21.15.52:0/460737438 conn(0x7f03b40782c0 msgr2=0x7f03b40ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 shutdown_connections 2024-06-02T23:33:26.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:26.512+0000 7f03b9b90700 1 -- 172.21.15.52:0/460737438 wait complete. 2024-06-02T23:33:27.208 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:33:27.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 62858 6.85 MB/sec execute 395 sec latency 65.804 ms 2024-06-02T23:33:27.577 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:28.059 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:27 smithi052 ceph-mon[32340]: pgmap v1078: 97 pgs: 97 active+clean; 533 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-06-02T23:33:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:27 smithi174 ceph-mon[35994]: pgmap v1078: 97 pgs: 97 active+clean; 533 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-06-02T23:33:28.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:27 smithi123 ceph-mon[35977]: pgmap v1078: 97 pgs: 97 active+clean; 533 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.3 MiB/s wr, 312 op/s 2024-06-02T23:33:28.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 62992 6.86 MB/sec execute 396 sec latency 62.564 ms 2024-06-02T23:33:29.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 63148 6.87 MB/sec execute 397 sec latency 58.146 ms 2024-06-02T23:33:29.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/748935502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8101810 msgr2=0x7fbbc810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/748935502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8101810 0x7fbbc810f280 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b0a0 tx=0x7fbbc000b3b0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/748935502 shutdown_connections 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/748935502 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8101810 0x7fbbc810f280 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/748935502 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8107570 0x7fbbc8107950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/748935502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc8100e50 0x7fbbc81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.785+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/748935502 >> 172.21.15.52:0/748935502 conn(0x7fbbc80766b0 msgr2=0x7fbbc8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.786+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/748935502 shutdown_connections 2024-06-02T23:33:29.786 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.786+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/748935502 wait complete. 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 Processor -- start 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 -- start start 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8100e50 0x7fbbc8102030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc8101810 0x7fbbc8102570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 0x7fbbc8102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc81126d0 con 0x7fbbc8107570 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8112550 con 0x7fbbc8101810 2024-06-02T23:33:29.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbcc9f0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbbc8112850 con 0x7fbbc8100e50 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbc6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 0x7fbbc8102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbc6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 0x7fbbc8102ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58484/0 (socket says 172.21.15.52:58484) 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8100e50 0x7fbbc8102030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbc6d9d700 1 -- 172.21.15.52:0/3251252321 learned_addr learned my addr 172.21.15.52:0/3251252321 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.787+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc8101810 0x7fbbc8102570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc6d9d700 1 -- 172.21.15.52:0/3251252321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8100e50 msgr2=0x7fbbc8102030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8100e50 0x7fbbc8102030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc6d9d700 1 -- 172.21.15.52:0/3251252321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc8101810 msgr2=0x7fbbc8102570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc8101810 0x7fbbc8102570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc6d9d700 1 -- 172.21.15.52:0/3251252321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbc81a70a0 con 0x7fbbc8107570 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc659c700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8100e50 0x7fbbc8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 0x7fbbc8102ab0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b070 tx=0x7fbbc0009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbb77fe700 1 -- 172.21.15.52:0/3251252321 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0021070 con 0x7fbbc8107570 2024-06-02T23:33:29.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbc81a7330 con 0x7fbbc8107570 2024-06-02T23:33:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbc81a7830 con 0x7fbbc8107570 2024-06-02T23:33:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbb77fe700 1 -- 172.21.15.52:0/3251252321 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbc0004e40 con 0x7fbbc8107570 2024-06-02T23:33:29.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.788+0000 7fbbb77fe700 1 -- 172.21.15.52:0/3251252321 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0004630 con 0x7fbbc8107570 2024-06-02T23:33:29.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.791+0000 7fbbb77fe700 1 -- 172.21.15.52:0/3251252321 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbbc0003c00 con 0x7fbbc8107570 2024-06-02T23:33:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.792+0000 7fbbb77fe700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbb00a2ca0 0x7fbbb00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.792+0000 7fbbb77fe700 1 -- 172.21.15.52:0/3251252321 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbbc000f020 con 0x7fbbc8107570 2024-06-02T23:33:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.792+0000 7fbbc659c700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbb00a2ca0 0x7fbbb00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.793+0000 7fbbc659c700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbb00a2ca0 0x7fbbb00a5160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8077c30 tx=0x7fbbb8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:29.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 Processor -- start 2024-06-02T23:33:29.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 -- start start 2024-06-02T23:33:29.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6640 0x56534dcf6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcf09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dcf1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbc5d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcf09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbc5d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcf09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41504/0 (socket says 172.21.15.52:41504) 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/597546696 learned_addr learned my addr 172.21.15.52:0/597546696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dcf1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbc659c700 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6640 0x56534dcf6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56534dc42810 con 0x56534dcf6640 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56534dbea720 con 0x56534dcf0f20 2024-06-02T23:33:29.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.794+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56534dc3e7d0 con 0x56534dcf6f60 2024-06-02T23:33:29.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/597546696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 msgr2=0x56534dcf1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dcf1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/597546696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6640 msgr2=0x56534dcf6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.795 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6640 0x56534dcf6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/597546696 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534dbe0f50 con 0x56534dcf6f60 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc659c700 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6640 0x56534dcf6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcf09e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009cf0 tx=0x7fbbbc011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6c3ba675cf94f05 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbb4ff9700 1 -- 172.21.15.52:0/597546696 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc011d80 con 0x56534dcf6f60 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbb4ff9700 1 -- 172.21.15.52:0/597546696 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc004500 con 0x56534dcf6f60 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.795+0000 7fbbb4ff9700 1 -- 172.21.15.52:0/597546696 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc005310 con 0x56534dcf6f60 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 msgr2=0x56534dcf09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.796 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcf09e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009cf0 tx=0x7fbbbc011440 comp rx=0 tx=0).stop 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 shutdown_connections 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dcf1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6640 0x56534dcf6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/597546696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcf09e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 >> 172.21.15.52:0/597546696 conn(0x56534dc1bcc0 msgr2=0x56534dceb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.796+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 shutdown_connections 2024-06-02T23:33:29.798 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.797+0000 7fbbd7273b80 1 -- 172.21.15.52:0/597546696 wait complete. 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.797+0000 7fbbd7273b80 1 -- start start 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.797+0000 7fbbd7273b80 1 Processor -- start 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.797+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcfcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56534dcfd9d0 con 0x56534dcf0f20 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56534dbe2bf0 con 0x56534dcf6640 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56534dcfdb50 con 0x56534dcf6f60 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbd7273b80 10 client.0 _ll_register_callbacks cb 0x56534dbdc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcfcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcfcc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58492/0 (socket says 172.21.15.52:58492) 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 -- 172.21.15.52:0/1506001329 learned_addr learned my addr 172.21.15.52:0/1506001329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.799 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 -- 172.21.15.52:0/1506001329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 msgr2=0x56534dcfa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 -- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 msgr2=0x56534dcf9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.798+0000 7fbbc659c700 1 -- 172.21.15.52:0/1506001329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534db918d0 con 0x56534dcf0f20 2024-06-02T23:33:29.800 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbc659c700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcfcc90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800eba0 tx=0x7fbbb80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3009d313e9daf45b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbadffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbadffb700 1 -- 172.21.15.52:0/1506001329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8023d50 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbadffb700 1 -- 172.21.15.52:0/1506001329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbbb8009040 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc004500 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbadffb700 1 -- 172.21.15.52:0/1506001329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbbb800f850 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbd7273b80 10 client.21081 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.799+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56534dd08350 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbadffb700 1 -- 172.21.15.52:0/1506001329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc004500 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbadffb700 1 -- 172.21.15.52:0/1506001329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbbb8024700 con 0x56534dcf0f20 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbd7273b80 10 client.21081 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:29.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbd7273b80 10 client.21081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbd7273b80 20 client.21081 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd0d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbd7273b80 4 client.21081 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534dc4fd10 con 0x56534dd0ae40 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.800+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd0d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.801+0000 7fbbadffb700 10 client.21081 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.801+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd0d2c0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cae8093363e1b37 server_cookie=492a1bd9b26dab04 in_seq=0 out_seq=0 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.801+0000 7fbbadffb700 1 -- 172.21.15.52:0/1506001329 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56534dc4fd10 con 0x56534dd0ae40 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.801+0000 7fbbadffb700 10 client.21081 handle_command_reply: tid=0 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.801+0000 7fbbd7273b80 1 client.21081 shutdown 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 msgr2=0x56534dcfcc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcfcc90 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800eba0 tx=0x7fbbb80047c0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 shutdown_connections 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd0d2c0 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcfcc90 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/1506001329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbadffb700 0 client.21081 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 >> 172.21.15.52:0/1506001329 conn(0x56534dc1bcc0 msgr2=0x56534dcef9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 shutdown_connections 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 1 -- 172.21.15.52:0/1506001329 wait complete. 2024-06-02T23:33:29.804 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.802+0000 7fbbd7273b80 20 client.21081 trim_cache size 0 max 16384 2024-06-02T23:33:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.830+0000 7fbbd7273b80 1 Processor -- start 2024-06-02T23:33:29.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.830+0000 7fbbd7273b80 1 -- start start 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.830+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcf09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.830+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800d2a0 con 0x56534dcf0f20 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800ede0 con 0x56534dcf6640 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800d970 con 0x56534dcf6f60 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcf09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57020/0 (socket says 172.21.15.52:57020) 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcf09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58496/0 (socket says 172.21.15.52:58496) 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/291963249 learned_addr learned my addr 172.21.15.52:0/291963249 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41520/0 (socket says 172.21.15.52:41520) 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/291963249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 msgr2=0x56534dcfa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/291963249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 msgr2=0x56534dcf09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcf09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 -- 172.21.15.52:0/291963249 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534dcf60e0 con 0x56534dcf6640 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.831+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf8f40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc011790 tx=0x7fbbbc004f60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2e14a67be3622dd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.832+0000 7fbbaffff700 1 -- 172.21.15.52:0/291963249 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc030070 con 0x56534dcf6640 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.832+0000 7fbbaffff700 1 -- 172.21.15.52:0/291963249 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc005480 con 0x56534dcf6640 2024-06-02T23:33:29.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.832+0000 7fbbaffff700 1 -- 172.21.15.52:0/291963249 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc003c80 con 0x56534dcf6640 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 -- 172.21.15.52:0/291963249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 msgr2=0x56534dcf8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf8f40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc011790 tx=0x7fbbbc004f60 comp rx=0 tx=0).stop 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 -- 172.21.15.52:0/291963249 shutdown_connections 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6f60 0x56534dcfa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf0f20 0x56534dcf09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/291963249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf6640 0x56534dcf8f40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 -- 172.21.15.52:0/291963249 >> 172.21.15.52:0/291963249 conn(0x56534dc1bcc0 msgr2=0x56534dcf7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 -- 172.21.15.52:0/291963249 shutdown_connections 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 -- 172.21.15.52:0/291963249 wait complete. 2024-06-02T23:33:29.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 -- start start 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.833+0000 7fbbd7273b80 1 Processor -- start 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dd02e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6640 0x56534dd033b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6f60 0x56534dd11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800e710 con 0x56534dcf6f60 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800e040 con 0x56534dcf0f20 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbba800b240 con 0x56534dcf6640 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dd02e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dd02e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57036/0 (socket says 172.21.15.52:57036) 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbd7273b80 10 client.0 _ll_register_callbacks cb 0x56534dbdc920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 -- 172.21.15.52:0/159232469 learned_addr learned my addr 172.21.15.52:0/159232469 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6640 0x56534dd033b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6f60 0x56534dd11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 -- 172.21.15.52:0/159232469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6640 msgr2=0x56534dd033b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6640 0x56534dd033b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 -- 172.21.15.52:0/159232469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6f60 msgr2=0x56534dd11b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6f60 0x56534dd11b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 -- 172.21.15.52:0/159232469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534dd13690 con 0x56534dcf0f20 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc6d9d700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6f60 0x56534dd11b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:29.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.834+0000 7fbbc659c700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dd02e70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800fca0 tx=0x7fbbb800fcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16640a3a2a273d08 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbadffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbadffb700 1 -- 172.21.15.52:0/159232469 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800f6a0 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbadffb700 1 -- 172.21.15.52:0/159232469 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbbb802a040 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbbc005480 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbadffb700 1 -- 172.21.15.52:0/159232469 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbbb8048070 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbd7273b80 10 client.30362 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56534dd75030 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbadffb700 1 -- 172.21.15.52:0/159232469 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbbbc005480 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbadffb700 1 -- 172.21.15.52:0/159232469 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbbb8041410 con 0x56534dcf0f20 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbd7273b80 10 client.30362 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.835+0000 7fbbd7273b80 10 client.30362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:29.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.836+0000 7fbbd7273b80 20 client.30362 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.836+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd777e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.836+0000 7fbbd7273b80 4 client.30362 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.836+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56534dc4a290 con 0x56534dd0ae40 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.836+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd777e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.836+0000 7fbbadffb700 10 client.30362 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbc5d9b700 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd777e0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7cf62a6a171778b server_cookie=b4a57c6ce68401c in_seq=0 out_seq=0 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbadffb700 1 -- 172.21.15.52:0/159232469 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56534dc4a290 con 0x56534dd0ae40 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbadffb700 10 client.30362 handle_command_reply: tid=0 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 client.30362 shutdown 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 msgr2=0x56534dd02e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dd02e70 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800fca0 tx=0x7fbbb800fcd0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 shutdown_connections 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56534dd0ae40 0x56534dd777e0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56534dcf6f60 0x56534dd11b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56534dcf0f20 0x56534dd02e70 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbadffb700 0 client.30362 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 --2- 172.21.15.52:0/159232469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534dcf6640 0x56534dd033b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.837+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 >> 172.21.15.52:0/159232469 conn(0x56534dc1bcc0 msgr2=0x56534dcf7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.838+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 shutdown_connections 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.838+0000 7fbbd7273b80 1 -- 172.21.15.52:0/159232469 wait complete. 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.838+0000 7fbbd7273b80 20 client.30362 trim_cache size 0 max 16384 2024-06-02T23:33:29.840 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:33:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.838+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbb00a2ca0 msgr2=0x7fbbb00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.838+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbb00a2ca0 0x7fbbb00a5160 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8077c30 tx=0x7fbbb8009260 comp rx=0 tx=0).stop 2024-06-02T23:33:29.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 msgr2=0x7fbbc8102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 0x7fbbc8102ab0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b070 tx=0x7fbbc0009c90 comp rx=0 tx=0).stop 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 shutdown_connections 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbbb00a2ca0 0x7fbbb00a5160 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbbc8107570 0x7fbbc8102ab0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbbc8100e50 0x7fbbc8102030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 --2- 172.21.15.52:0/3251252321 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbc8101810 0x7fbbc8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 >> 172.21.15.52:0/3251252321 conn(0x7fbbc80766b0 msgr2=0x7fbbc80ffa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 shutdown_connections 2024-06-02T23:33:29.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:29.839+0000 7fbbcc9f0700 1 -- 172.21.15.52:0/3251252321 wait complete. 2024-06-02T23:33:30.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:29 smithi174 ceph-mon[35994]: pgmap v1079: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.6 MiB/s wr, 252 op/s 2024-06-02T23:33:30.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:29 smithi052 ceph-mon[32340]: pgmap v1079: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.6 MiB/s wr, 252 op/s 2024-06-02T23:33:30.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:29 smithi123 ceph-mon[35977]: pgmap v1079: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1017 KiB/s rd, 3.6 MiB/s wr, 252 op/s 2024-06-02T23:33:30.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 63337 6.89 MB/sec execute 398 sec latency 88.761 ms 2024-06-02T23:33:31.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 63490 6.89 MB/sec execute 399 sec latency 56.695 ms 2024-06-02T23:33:31.664 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:33:31.664 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:33:32.029 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 -- 172.21.15.52:0/2824881398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634101af0 msgr2=0x7ff634101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 --2- 172.21.15.52:0/2824881398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634101af0 0x7ff634101f70 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7ff624009860 tx=0x7ff62402a340 comp rx=0 tx=0).stop 2024-06-02T23:33:32.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 -- 172.21.15.52:0/2824881398 shutdown_connections 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 --2- 172.21.15.52:0/2824881398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff6341024b0 0x7ff634108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 --2- 172.21.15.52:0/2824881398 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff634101af0 0x7ff634101f70 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 --2- 172.21.15.52:0/2824881398 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 0x7ff634100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 -- 172.21.15.52:0/2824881398 >> 172.21.15.52:0/2824881398 conn(0x7ff6340fbe70 msgr2=0x7ff6340fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 -- 172.21.15.52:0/2824881398 shutdown_connections 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.133+0000 7ff63b638640 1 -- 172.21.15.52:0/2824881398 wait complete. 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.134+0000 7ff63b638640 1 Processor -- start 2024-06-02T23:33:32.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.134+0000 7ff63b638640 1 -- start start 2024-06-02T23:33:32.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.134+0000 7ff63b638640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 0x7ff634197c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.134+0000 7ff63b638640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff634101af0 0x7ff634198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.134+0000 7ff63b638640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6341024b0 0x7ff63419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff63b638640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410d550 con 0x7ff634100900 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff63b638640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410d3d0 con 0x7ff6341024b0 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff63b638640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff63410d6d0 con 0x7ff634101af0 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff638bd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff634101af0 0x7ff634198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff6393d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 0x7ff634197c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6341024b0 0x7ff63419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff638bd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff634101af0 0x7ff634198150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41546/0 (socket says 172.21.15.52:41546) 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff638bd3640 1 -- 172.21.15.52:0/1356629690 learned_addr learned my addr 172.21.15.52:0/1356629690 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 -- 172.21.15.52:0/1356629690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff634101af0 msgr2=0x7ff634198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.136 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff634101af0 0x7ff634198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 -- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 msgr2=0x7ff634197c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 0x7ff634197c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 -- 172.21.15.52:0/1356629690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff63419f7e0 con 0x7ff6341024b0 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff6393d4640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 0x7ff634197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.135+0000 7ff639bd5640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6341024b0 0x7ff63419f170 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00ca30 tx=0x7ff62c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.136+0000 7ff62a7fc640 1 -- 172.21.15.52:0/1356629690 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004d20 con 0x7ff6341024b0 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.136+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff63410aaf0 con 0x7ff6341024b0 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.136+0000 7ff62a7fc640 1 -- 172.21.15.52:0/1356629690 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff62c005020 con 0x7ff6341024b0 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.136+0000 7ff62a7fc640 1 -- 172.21.15.52:0/1356629690 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff62c004d20 con 0x7ff6341024b0 2024-06-02T23:33:32.137 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.136+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff63410afc0 con 0x7ff6341024b0 2024-06-02T23:33:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.138+0000 7ff62a7fc640 1 -- 172.21.15.52:0/1356629690 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff62c0051d0 con 0x7ff6341024b0 2024-06-02T23:33:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.139+0000 7ff63e426740 1 Processor -- start 2024-06-02T23:33:32.138 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.139+0000 7ff63e426740 1 -- start start 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.139+0000 7ff63e426740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467086c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.139+0000 7ff63e426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x56546708bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff63e426740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x56546708e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff63e426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x565466fe14f0 con 0x565467087210 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff63e426740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565466f7f870 con 0x565467086820 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff63e426740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565466fdd4b0 con 0x56546708c380 2024-06-02T23:33:32.140 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467086c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467086c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:57062/0 (socket says 172.21.15.52:57062) 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 -- 172.21.15.52:0/3583105696 learned_addr learned my addr 172.21.15.52:0/3583105696 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x56546708e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff6393d4640 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x56546708bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff62a7fc640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6200a3030 0x7ff6200a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff639bd5640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6200a3030 0x7ff6200a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 -- 172.21.15.52:0/3583105696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 msgr2=0x56546708e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x56546708e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 -- 172.21.15.52:0/3583105696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 msgr2=0x56546708bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x56546708bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 -- 172.21.15.52:0/3583105696 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565466f9b9b0 con 0x565467086820 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff62a7fc640 1 -- 172.21.15.52:0/1356629690 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff62c004b20 con 0x7ff6341024b0 2024-06-02T23:33:32.141 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.140+0000 7ff638bd3640 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467086c40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff6240095f0 tx=0x7ff624005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca452aecdcf90f7a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.141+0000 7ff639bd5640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6200a3030 0x7ff6200a54f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff62c097f00 tx=0x7ff62c09e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.141+0000 7ff61f7fe640 1 -- 172.21.15.52:0/3583105696 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624048070 con 0x565467086820 2024-06-02T23:33:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.141+0000 7ff61f7fe640 1 -- 172.21.15.52:0/3583105696 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff624038040 con 0x565467086820 2024-06-02T23:33:32.142 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.141+0000 7ff61f7fe640 1 -- 172.21.15.52:0/3583105696 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624037070 con 0x565467086820 2024-06-02T23:33:32.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 -- 172.21.15.52:0/3583105696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 msgr2=0x565467086c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467086c40 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff6240095f0 tx=0x7ff624005d20 comp rx=0 tx=0).stop 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 -- 172.21.15.52:0/3583105696 shutdown_connections 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x56546708e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x56546708bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 --2- 172.21.15.52:0/3583105696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467086c40 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 -- 172.21.15.52:0/3583105696 >> 172.21.15.52:0/3583105696 conn(0x565467079680 msgr2=0x565467079a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 -- 172.21.15.52:0/3583105696 shutdown_connections 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 -- 172.21.15.52:0/3583105696 wait complete. 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 -- start start 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.142+0000 7ff63e426740 1 Processor -- start 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467090aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467090fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x565467098020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56546708f2d0 con 0x565467087210 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467090fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565466f6aae0 con 0x565467086820 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56546708f450 con 0x56546708c380 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467090fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58554/0 (socket says 172.21.15.52:58554) 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 -- 172.21.15.52:0/610652378 learned_addr learned my addr 172.21.15.52:0/610652378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff638bd3640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467090aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff63e426740 10 client.0 _ll_register_callbacks cb 0x565466fa39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff638bd3640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x565467098020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 -- 172.21.15.52:0/610652378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 msgr2=0x565467098020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x565467098020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 -- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 msgr2=0x565467090aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467090aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 -- 172.21.15.52:0/610652378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565466cf5210 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff638bd3640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467090aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff6393d4640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467090fe0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff630009ef0 tx=0x7ff63000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b5eac05db652976 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff61cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.143+0000 7ff61cff9640 1 -- 172.21.15.52:0/610652378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630004250 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff61cff9640 1 -- 172.21.15.52:0/610652378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff6300043f0 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff624038040 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff61cff9640 1 -- 172.21.15.52:0/610652378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff6300047c0 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 10 client.21099 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x565466dc1620 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff61cff9640 1 -- 172.21.15.52:0/610652378 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff624038040 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff61cff9640 1 -- 172.21.15.52:0/610652378 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff6300035f0 con 0x565467087210 2024-06-02T23:33:32.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 10 client.21099 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 10 client.21099 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 20 client.21099 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546709ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 4 client.21099 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.144+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565466ff4f10 con 0x56546709c940 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.145+0000 7ff638bd3640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546709ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.145+0000 7ff638bd3640 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546709ee00 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=546c7e9e6c291283 server_cookie=2cc7110b4f44810e in_seq=0 out_seq=0 2024-06-02T23:33:32.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.145+0000 7ff61cff9640 10 client.21099 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:32.149 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:31 smithi174 ceph-mon[35994]: pgmap v1080: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff61cff9640 1 -- 172.21.15.52:0/610652378 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x565466ff4f10 con 0x56546709c940 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff61cff9640 10 client.21099 handle_command_reply: tid=0 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 client.21099 shutdown 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 msgr2=0x565467090fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467090fe0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7ff630009ef0 tx=0x7ff63000e720 comp rx=0 tx=0).stop 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 shutdown_connections 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546709ee00 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56546708c380 0x565467098020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff61cff9640 0 client.21099 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467090fe0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 --2- 172.21.15.52:0/610652378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565467086820 0x565467090aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 >> 172.21.15.52:0/610652378 conn(0x565467079680 msgr2=0x5654670787e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.146+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 shutdown_connections 2024-06-02T23:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.147+0000 7ff63e426740 1 -- 172.21.15.52:0/610652378 wait complete. 2024-06-02T23:33:32.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.147+0000 7ff63e426740 20 client.21099 trim_cache size 0 max 16384 2024-06-02T23:33:32.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:31 smithi052 ceph-mon[32340]: pgmap v1080: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-06-02T23:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.172+0000 7ff63e426740 1 Processor -- start 2024-06-02T23:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 -- start start 2024-06-02T23:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467088490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x5654670889d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467097fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a6ad0 con 0x565467087210 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a8610 con 0x56546708c380 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff63e426740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a71a0 con 0x565467086820 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff6393d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x5654670889d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467088490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467088490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:41582/0 (socket says 172.21.15.52:41582) 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 -- 172.21.15.52:0/4109133660 learned_addr learned my addr 172.21.15.52:0/4109133660 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467097fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 -- 172.21.15.52:0/4109133660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 msgr2=0x565467097fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467097fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 -- 172.21.15.52:0/4109133660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 msgr2=0x5654670889d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x5654670889d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.173+0000 7ff638bd3640 1 -- 172.21.15.52:0/4109133660 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654670794b0 con 0x565467086820 2024-06-02T23:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.174+0000 7ff638bd3640 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467088490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff62402a900 tx=0x7ff624030c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b244b072af7ca9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.174+0000 7ff61effd640 1 -- 172.21.15.52:0/4109133660 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624048070 con 0x565467086820 2024-06-02T23:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.174+0000 7ff61effd640 1 -- 172.21.15.52:0/4109133660 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff624049040 con 0x565467086820 2024-06-02T23:33:32.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.174+0000 7ff61effd640 1 -- 172.21.15.52:0/4109133660 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff624037040 con 0x565467086820 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 -- 172.21.15.52:0/4109133660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 msgr2=0x565467088490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467088490 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff62402a900 tx=0x7ff624030c90 comp rx=0 tx=0).stop 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 -- 172.21.15.52:0/4109133660 shutdown_connections 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467097fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x5654670889d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 --2- 172.21.15.52:0/4109133660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467088490 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.178 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 -- 172.21.15.52:0/4109133660 >> 172.21.15.52:0/4109133660 conn(0x565467079680 msgr2=0x565467079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 -- 172.21.15.52:0/4109133660 shutdown_connections 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 -- 172.21.15.52:0/4109133660 wait complete. 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 -- start start 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.175+0000 7ff63e426740 1 Processor -- start 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467095630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467095b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467096b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a7f40 con 0x565467087210 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a7870 con 0x56546708c380 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff6341a4ab0 con 0x565467086820 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff63e426740 10 client.0 _ll_register_callbacks cb 0x7ff6300103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467095b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467095b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58570/0 (socket says 172.21.15.52:58570) 2024-06-02T23:33:32.179 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 -- 172.21.15.52:0/2184143276 learned_addr learned my addr 172.21.15.52:0/2184143276 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff638bd3640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467096b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff638bd3640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467095630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 -- 172.21.15.52:0/2184143276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 msgr2=0x565467095630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467095630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 -- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 msgr2=0x565467096b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467096b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff6393d4640 1 -- 172.21.15.52:0/2184143276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565467097170 con 0x565467087210 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.176+0000 7ff638bd3640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467096b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff6393d4640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467095b70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff6300092a0 tx=0x7ff630004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971f5298b65491c9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff61cff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff61cff9640 1 -- 172.21.15.52:0/2184143276 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff630005240 con 0x565467087210 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff624049040 con 0x565467087210 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff61cff9640 1 -- 172.21.15.52:0/2184143276 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff6300053e0 con 0x565467087210 2024-06-02T23:33:32.180 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff61cff9640 1 -- 172.21.15.52:0/2184143276 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff630009a80 con 0x565467087210 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff63e426740 10 client.21105 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5654671076e0 con 0x565467087210 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.177+0000 7ff61cff9640 1 -- 172.21.15.52:0/2184143276 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff624049040 con 0x565467087210 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff61cff9640 1 -- 172.21.15.52:0/2184143276 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff6300043c0 con 0x565467087210 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff63e426740 10 client.21105 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff63e426740 10 client.21105 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff63e426740 20 client.21105 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff63e426740 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546710a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff63e426740 4 client.21105 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x565466feec20 con 0x56546709c940 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff638bd3640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546710a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff638bd3640 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546710a050 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b683d46437d27a5 server_cookie=75f6e77071cf3944 in_seq=0 out_seq=0 2024-06-02T23:33:32.181 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.178+0000 7ff61cff9640 10 client.21105 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.227+0000 7ff61cff9640 1 -- 172.21.15.52:0/2184143276 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x565466feec20 con 0x56546709c940 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.227+0000 7ff61cff9640 10 client.21105 handle_command_reply: tid=0 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.227+0000 7ff63e426740 1 client.21105 shutdown 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 msgr2=0x565467095b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467095b70 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7ff6300092a0 tx=0x7ff630004b70 comp rx=0 tx=0).stop 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 shutdown_connections 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56546709c940 0x56546710a050 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56546708c380 0x565467096b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x565467087210 0x565467095b70 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 --2- 172.21.15.52:0/2184143276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x565467086820 0x565467095630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.228 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 >> 172.21.15.52:0/2184143276 conn(0x565467079680 msgr2=0x565467090a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff61cff9640 0 client.21105 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 shutdown_connections 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 1 -- 172.21.15.52:0/2184143276 wait complete. 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.228+0000 7ff63e426740 20 client.21105 trim_cache size 0 max 16384 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:33:32.229 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:33:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6200a3030 msgr2=0x7ff6200a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6200a3030 0x7ff6200a54f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7ff62c097f00 tx=0x7ff62c09e040 comp rx=0 tx=0).stop 2024-06-02T23:33:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6341024b0 msgr2=0x7ff63419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6341024b0 0x7ff63419f170 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00ca30 tx=0x7ff62c00ce30 comp rx=0 tx=0).stop 2024-06-02T23:33:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 shutdown_connections 2024-06-02T23:33:32.230 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff6200a3030 0x7ff6200a54f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff634101af0 0x7ff634198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff6341024b0 0x7ff63419f170 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 --2- 172.21.15.52:0/1356629690 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff634100900 0x7ff634197c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 >> 172.21.15.52:0/1356629690 conn(0x7ff6340fbe70 msgr2=0x7ff634104ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 shutdown_connections 2024-06-02T23:33:32.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:32.229+0000 7ff63b638640 1 -- 172.21.15.52:0/1356629690 wait complete. 2024-06-02T23:33:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:31 smithi123 ceph-mon[35977]: pgmap v1080: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-06-02T23:33:32.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 63640 6.88 MB/sec execute 400 sec latency 77.994 ms 2024-06-02T23:33:33.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:32 smithi052 ceph-mon[32340]: pgmap v1081: 97 pgs: 97 active+clean; 539 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-06-02T23:33:33.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:32 smithi123 ceph-mon[35977]: pgmap v1081: 97 pgs: 97 active+clean; 539 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-06-02T23:33:33.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:32 smithi174 ceph-mon[35994]: pgmap v1081: 97 pgs: 97 active+clean; 539 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 272 op/s 2024-06-02T23:33:33.473 INFO:tasks.workunit.client.0.smithi052.stdout: 10 63782 6.87 MB/sec execute 401 sec latency 67.860 ms 2024-06-02T23:33:34.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 -- 172.21.15.52:0/1844578019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 msgr2=0x7f63141017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 --2- 172.21.15.52:0/1844578019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141017f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f6304009990 tx=0x7f6304009ca0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 -- 172.21.15.52:0/1844578019 shutdown_connections 2024-06-02T23:33:34.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 --2- 172.21.15.52:0/1844578019 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 0x7f631410e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 --2- 172.21.15.52:0/1844578019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63141073d0 0x7f63141077b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.139 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 --2- 172.21.15.52:0/1844578019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141017f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 -- 172.21.15.52:0/1844578019 >> 172.21.15.52:0/1844578019 conn(0x7f63140782c0 msgr2=0x7f63140786d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:34.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 -- 172.21.15.52:0/1844578019 shutdown_connections 2024-06-02T23:33:34.140 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.139+0000 7f631bff5700 1 -- 172.21.15.52:0/1844578019 wait complete. 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 Processor -- start 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 -- start start 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 0x7f6314197d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63141073d0 0x7f631419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6314111530 con 0x7f6314101d30 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f6319d91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63141113b0 con 0x7f6314101390 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f631bff5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f63141116b0 con 0x7f63141073d0 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f6319d91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141977c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:34968/0 (socket says 172.21.15.52:34968) 2024-06-02T23:33:34.142 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.140+0000 7f6319d91700 1 -- 172.21.15.52:0/3531474861 learned_addr learned my addr 172.21.15.52:0/3531474861 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f631a592700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63141073d0 0x7f631419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319590700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 0x7f6314197d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319d91700 1 -- 172.21.15.52:0/3531474861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63141073d0 msgr2=0x7f631419c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319d91700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63141073d0 0x7f631419c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319d91700 1 -- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 msgr2=0x7f6314197d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319d91700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 0x7f6314197d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319d91700 1 -- 172.21.15.52:0/3531474861 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631419cd50 con 0x7f6314101390 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319590700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 0x7f6314197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f6319d91700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141977c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f631000e9c0 tx=0x7f631000ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f630affd700 1 -- 172.21.15.52:0/3531474861 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f631000cbd0 con 0x7f6314101390 2024-06-02T23:33:34.143 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631419d040 con 0x7f6314101390 2024-06-02T23:33:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.141+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631419d580 con 0x7f6314101390 2024-06-02T23:33:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.142+0000 7f630affd700 1 -- 172.21.15.52:0/3531474861 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f631000cd70 con 0x7f6314101390 2024-06-02T23:33:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.142+0000 7f630affd700 1 -- 172.21.15.52:0/3531474861 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310005650 con 0x7f6314101390 2024-06-02T23:33:34.144 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.144+0000 7f630affd700 1 -- 172.21.15.52:0/3531474861 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6310004750 con 0x7f6314101390 2024-06-02T23:33:34.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.145+0000 7f630affd700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63000a2ca0 0x7f63000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.145+0000 7f6319590700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63000a2ca0 0x7f63000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.145+0000 7f630affd700 1 -- 172.21.15.52:0/3531474861 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f63100cf0b0 con 0x7f6314101390 2024-06-02T23:33:34.145 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.146+0000 7f6319590700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63000a2ca0 0x7f63000a5160 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f630400b2e0 tx=0x7f630401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:34.146 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6326878b80 1 Processor -- start 2024-06-02T23:33:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6326878b80 1 -- start start 2024-06-02T23:33:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6326878b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x56342672caa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6326878b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426726a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6326878b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x5634267273c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6319d91700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x56342672caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.148 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6319d91700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x56342672caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:54870/0 (socket says 172.21.15.52:54870) 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6319d91700 1 -- 172.21.15.52:0/3176722947 learned_addr learned my addr 172.21.15.52:0/3176722947 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6326878b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563426678890 con 0x563426726fa0 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.147+0000 7f6319590700 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426726a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5634266207a0 con 0x56342672cfe0 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563426674850 con 0x56342672c6c0 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f631a592700 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x5634267273c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6319d91700 1 -- 172.21.15.52:0/3176722947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 msgr2=0x563426726a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6319d91700 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426726a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6319d91700 1 -- 172.21.15.52:0/3176722947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 msgr2=0x5634267273c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6319d91700 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x5634267273c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6319d91700 1 -- 172.21.15.52:0/3176722947 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563426616fd0 con 0x56342672c6c0 2024-06-02T23:33:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.148+0000 7f6319d91700 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x56342672caa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6310009fd0 tx=0x7f6310005870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dde6a3ddaa896c47 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:34.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.149+0000 7f62fffff700 1 -- 172.21.15.52:0/3176722947 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63100afe60 con 0x56342672c6c0 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.149+0000 7f62fffff700 1 -- 172.21.15.52:0/3176722947 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f631009c040 con 0x56342672c6c0 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.149+0000 7f62fffff700 1 -- 172.21.15.52:0/3176722947 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6310099cd0 con 0x56342672c6c0 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 msgr2=0x56342672caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x56342672caa0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6310009fd0 tx=0x7f6310005870 comp rx=0 tx=0).stop 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 shutdown_connections 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x5634267273c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x56342672caa0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 --2- 172.21.15.52:0/3176722947 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426726a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 >> 172.21.15.52:0/3176722947 conn(0x563426651d40 msgr2=0x5634267218d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 shutdown_connections 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.150+0000 7f6326878b80 1 -- 172.21.15.52:0/3176722947 wait complete. 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.151+0000 7f6326878b80 1 -- start start 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.151+0000 7f6326878b80 1 Processor -- start 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.151+0000 7f6326878b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563426726fa0 0x563426732d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.151+0000 7f6326878b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 0x563426730020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6326878b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426730560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6326878b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563426733a50 con 0x56342672c6c0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6326878b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563426618c70 con 0x56342672cfe0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6326878b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563426733bd0 con 0x563426726fa0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 0x563426730020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 0x563426730020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52458/0 (socket says 172.21.15.52:52458) 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 -- 172.21.15.52:0/770213335 learned_addr learned my addr 172.21.15.52:0/770213335 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6326878b80 10 client.0 _ll_register_callbacks cb 0x5634266129a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319d91700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563426726fa0 0x563426732d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f631a592700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426730560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 -- 172.21.15.52:0/770213335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563426726fa0 msgr2=0x563426732d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563426726fa0 0x563426732d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 -- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 msgr2=0x563426730560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426730560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319590700 1 -- 172.21.15.52:0/770213335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634265c7950 con 0x56342672c6c0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f631a592700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426730560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.152+0000 7f6319d91700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563426726fa0 0x563426732d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f6319590700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 0x563426730020 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6304009990 tx=0x7f630401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8024719930ac7036 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f62fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f62fd7fa700 1 -- 172.21.15.52:0/770213335 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304005220 con 0x56342672c6c0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f62fd7fa700 1 -- 172.21.15.52:0/770213335 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f63040053c0 con 0x56342672c6c0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f631009c040 con 0x56342672c6c0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f62fd7fa700 1 -- 172.21.15.52:0/770213335 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6304004690 con 0x56342672c6c0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f6326878b80 10 client.21117 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56342673e340 con 0x56342672c6c0 2024-06-02T23:33:34.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.153+0000 7f62fd7fa700 1 -- 172.21.15.52:0/770213335 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6304004ce0 con 0x56342672c6c0 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f62fd7fa700 1 -- 172.21.15.52:0/770213335 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6304022070 con 0x56342672c6c0 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6326878b80 10 client.21117 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6326878b80 10 client.21117 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6326878b80 20 client.21117 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6326878b80 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563426740980 0x563426742d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6326878b80 4 client.21117 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f631009c040 con 0x563426740980 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6319d91700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563426740980 0x563426742d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f62fd7fa700 10 client.21117 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:34.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.154+0000 7f6319d91700 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563426740980 0x563426742d70 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ac39a98f4a8b0fc server_cookie=7eb0b9fb0e433f66 in_seq=0 out_seq=0 2024-06-02T23:33:34.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.155+0000 7f62fd7fa700 1 -- 172.21.15.52:0/770213335 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f631009c040 con 0x563426740980 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.155+0000 7f62fd7fa700 10 client.21117 handle_command_reply: tid=0 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.155+0000 7f6326878b80 1 client.21117 shutdown 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.155+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 msgr2=0x563426730020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.155+0000 7f6326878b80 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 0x563426730020 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f6304009990 tx=0x7f630401e000 comp rx=0 tx=0).stop 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 shutdown_connections 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563426740980 0x563426742d70 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672cfe0 0x563426730560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f62fd7fa700 0 client.21117 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563426726fa0 0x563426732d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 --2- 172.21.15.52:0/770213335 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672c6c0 0x563426730020 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 >> 172.21.15.52:0/770213335 conn(0x563426651d40 msgr2=0x563426721670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:34.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 shutdown_connections 2024-06-02T23:33:34.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 1 -- 172.21.15.52:0/770213335 wait complete. 2024-06-02T23:33:34.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.156+0000 7f6326878b80 20 client.21117 trim_cache size 0 max 16384 2024-06-02T23:33:34.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 Processor -- start 2024-06-02T23:33:34.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 -- start start 2024-06-02T23:33:34.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 0x563426726a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x563426730650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672cfe0 0x56342672dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800d2a0 con 0x56342672cfe0 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800ede0 con 0x563426726fa0 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.192+0000 7f6326878b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800d970 con 0x56342672c6c0 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.193+0000 7f6319d91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 0x563426726a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.193+0000 7f6319d91700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 0x563426726a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35008/0 (socket says 172.21.15.52:35008) 2024-06-02T23:33:34.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.193+0000 7f6319d91700 1 -- 172.21.15.52:0/1956483211 learned_addr learned my addr 172.21.15.52:0/1956483211 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:34.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.193+0000 7f631a592700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672cfe0 0x56342672dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f6319590700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x563426730650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f631a592700 1 -- 172.21.15.52:0/1956483211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 msgr2=0x563426730650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f631a592700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x563426730650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f631a592700 1 -- 172.21.15.52:0/1956483211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 msgr2=0x563426726a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f631a592700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 0x563426726a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f631a592700 1 -- 172.21.15.52:0/1956483211 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342672c190 con 0x56342672cfe0 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f6319d91700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 0x563426726a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f6319590700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x563426730650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f631a592700 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672cfe0 0x56342672dc50 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f630c009fd0 tx=0x7f630c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b95db28e8ad6c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.194+0000 7f62ff7fe700 1 -- 172.21.15.52:0/1956483211 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c017e10 con 0x56342672cfe0 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.195+0000 7f62ff7fe700 1 -- 172.21.15.52:0/1956483211 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f630c004bb0 con 0x56342672cfe0 2024-06-02T23:33:34.195 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.195+0000 7f62ff7fe700 1 -- 172.21.15.52:0/1956483211 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630c0047e0 con 0x56342672cfe0 2024-06-02T23:33:34.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 -- 172.21.15.52:0/1956483211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672cfe0 msgr2=0x56342672dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672cfe0 0x56342672dc50 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f630c009fd0 tx=0x7f630c009920 comp rx=0 tx=0).stop 2024-06-02T23:33:34.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 -- 172.21.15.52:0/1956483211 shutdown_connections 2024-06-02T23:33:34.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56342672cfe0 0x56342672dc50 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563426726fa0 0x563426726a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 --2- 172.21.15.52:0/1956483211 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672c6c0 0x563426730650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 -- 172.21.15.52:0/1956483211 >> 172.21.15.52:0/1956483211 conn(0x563426651d40 msgr2=0x56342671ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:34.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 -- 172.21.15.52:0/1956483211 shutdown_connections 2024-06-02T23:33:34.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.196+0000 7f6326878b80 1 -- 172.21.15.52:0/1956483211 wait complete. 2024-06-02T23:33:34.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.197+0000 7f6326878b80 1 -- start start 2024-06-02T23:33:34.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.197+0000 7f6326878b80 1 Processor -- start 2024-06-02T23:33:34.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.197+0000 7f6326878b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x563426735e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.197+0000 7f6326878b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 0x5634267363a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6326878b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672cfe0 0x5634267368e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6326878b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800e710 con 0x563426726fa0 2024-06-02T23:33:34.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6326878b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800e040 con 0x56342672c6c0 2024-06-02T23:33:34.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6326878b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62f800b240 con 0x56342672cfe0 2024-06-02T23:33:34.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6319590700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 0x5634267363a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6319590700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 0x5634267363a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35014/0 (socket says 172.21.15.52:35014) 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6319590700 1 -- 172.21.15.52:0/1037656033 learned_addr learned my addr 172.21.15.52:0/1037656033 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f631a592700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672cfe0 0x5634267368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.198+0000 7f6326878b80 10 client.0 _ll_register_callbacks cb 0x5634266129a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319d91700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x563426735e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319590700 1 -- 172.21.15.52:0/1037656033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672cfe0 msgr2=0x5634267368e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319590700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672cfe0 0x5634267368e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319590700 1 -- 172.21.15.52:0/1037656033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 msgr2=0x563426735e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319590700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x563426735e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319590700 1 -- 172.21.15.52:0/1037656033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634267ab760 con 0x56342672c6c0 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f631a592700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672cfe0 0x5634267368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f6319590700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 0x5634267363a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6304005a60 tx=0x7f63040236b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba17913a5a23b67c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:34.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.199+0000 7f62fd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.200+0000 7f62fd7fa700 1 -- 172.21.15.52:0/1037656033 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304040020 con 0x56342672c6c0 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.200+0000 7f62fd7fa700 1 -- 172.21.15.52:0/1037656033 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f63040178c0 con 0x56342672c6c0 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.200+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f630c004bb0 con 0x56342672c6c0 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.200+0000 7f62fd7fa700 1 -- 172.21.15.52:0/1037656033 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f63040173f0 con 0x56342672c6c0 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.201+0000 7f6326878b80 10 client.30398 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.201+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5634267aca60 con 0x56342672c6c0 2024-06-02T23:33:34.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.201+0000 7f62fd7fa700 1 -- 172.21.15.52:0/1037656033 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6304031480 con 0x56342672c6c0 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.201+0000 7f62fd7fa700 1 -- 172.21.15.52:0/1037656033 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6304022040 con 0x56342672c6c0 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6326878b80 10 client.30398 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6326878b80 10 client.30398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6326878b80 20 client.30398 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6326878b80 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5634267ad010 0x5634267af430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6326878b80 4 client.30398 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f630c004bb0 con 0x5634267ad010 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.202+0000 7f6319d91700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5634267ad010 0x5634267af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.203+0000 7f62fd7fa700 10 client.30398 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:34.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.203+0000 7f6319d91700 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5634267ad010 0x5634267af430 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37eb5edb9bd2cb3 server_cookie=7fd01d107109a39e in_seq=0 out_seq=0 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.205+0000 7f62fd7fa700 1 -- 172.21.15.52:0/1037656033 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f630c004bb0 con 0x5634267ad010 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f62fd7fa700 10 client.30398 handle_command_reply: tid=0 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 client.30398 shutdown 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 msgr2=0x5634267363a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 0x5634267363a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f6304005a60 tx=0x7f63040236b0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 shutdown_connections 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5634267ad010 0x5634267af430 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56342672cfe0 0x5634267368e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f62fd7fa700 0 client.30398 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563426726fa0 0x563426735e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.206+0000 7f6326878b80 1 --2- 172.21.15.52:0/1037656033 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56342672c6c0 0x5634267363a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.207+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 >> 172.21.15.52:0/1037656033 conn(0x563426651d40 msgr2=0x56342671b590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.207+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 shutdown_connections 2024-06-02T23:33:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.207+0000 7f6326878b80 1 -- 172.21.15.52:0/1037656033 wait complete. 2024-06-02T23:33:34.207 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.207+0000 7f6326878b80 20 client.30398 trim_cache size 0 max 16384 2024-06-02T23:33:34.208 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:33:34.208 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:33:34.208 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "03b50edb-4ff4-42e7-823b-35a0b3ca6d61", 2024-06-02T23:33:34.208 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:33:34.208 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63000a2ca0 msgr2=0x7f63000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63000a2ca0 0x7f63000a5160 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f630400b2e0 tx=0x7f630401a040 comp rx=0 tx=0).stop 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 msgr2=0x7f63141977c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141977c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f631000e9c0 tx=0x7f631000ed80 comp rx=0 tx=0).stop 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 shutdown_connections 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f63000a2ca0 0x7f63000a5160 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f63141073d0 0x7f631419c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6314101390 0x7f63141977c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 --2- 172.21.15.52:0/3531474861 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6314101d30 0x7f6314197d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.208+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 >> 172.21.15.52:0/3531474861 conn(0x7f63140782c0 msgr2=0x7f63141001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.209+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 shutdown_connections 2024-06-02T23:33:34.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:34.209+0000 7f631bff5700 1 -- 172.21.15.52:0/3531474861 wait complete. 2024-06-02T23:33:34.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 63920 6.86 MB/sec execute 402 sec latency 98.396 ms 2024-06-02T23:33:34.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:34 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:34.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:34 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:34.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:34 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:34.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:33:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:33:34.287+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004b8 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX/ auth{4=1} v15119 ap=2 f(v560 m2024-06-02T23:33:22.946805+0000 24=24+0) n(v35 rc2024-06-02T23:33:22.946805+0000 b5230239 25=24+1) (inest sync->mix g=4) caps={15216=pAsLsXsFs/-@10425} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9816680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004b8:[<0x1000000033f/PARADOX v108>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004b8:[<0x1000000033f/PARADOX v15119>,<0x10000000335/~dmtmp v3843>,<0x10000000007/client3 v4472>,<0x10000000006/clients v2931>,<0x10000000005/tmp v2173>,<0x10000000003/client.0 v2134>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2099>,<0x10000000001/sv_0 v2098>,<0x10000000000/qa v2034>,<0x1/volumes v2023>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:33:22.889097+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:33:22.946805+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v560 m2024-06-02T23:33:22.946805+0000 24=24+0)","memory_value.rstat":"n(v35 rc2024-06-02T23:33:22.946805+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:33:34.757 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:33:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:33:34.298+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003eb [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL/ auth{3=1} v10094 ap=2 f(v368 m2024-06-02T23:33:16.090670+0000 11=11+0) n(v24 rc2024-06-02T23:33:16.090670+0000 b2861056 12=11+1) (inest sync->mix g=3) caps={15216=pAsLsXsFs/-@8219} | dirtyscattered=0 request=0 lock=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb964b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003eb:[<0x10000000336/COREL v50>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003eb:[<0x10000000336/COREL v10094>,<0x10000000331/~dmtmp v6730>,<0x10000000007/client1 v4482>,<0x10000000006/clients v2935>,<0x10000000005/tmp v2173>,<0x10000000003/client.0 v2134>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2099>,<0x10000000001/sv_0 v2098>,<0x10000000000/qa v2034>,<0x1/volumes v2023>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:33:16.062827+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:33:16.090670+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v368 m2024-06-02T23:33:16.090670+0000 11=11+0)","memory_value.rstat":"n(v24 rc2024-06-02T23:33:16.090670+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:33:35.255 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:33:35.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 64083 6.87 MB/sec execute 403 sec latency 86.193 ms 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: pgmap v1082: 97 pgs: 97 active+clean; 540 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 4.3 MiB/s wr, 236 op/s 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:33:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:33:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:33:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:35 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: pgmap v1082: 97 pgs: 97 active+clean; 540 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 4.3 MiB/s wr, 236 op/s 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:33:35.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:33:35.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:33:35.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:35 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.655 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: pgmap v1082: 97 pgs: 97 active+clean; 540 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 847 KiB/s rd, 4.3 MiB/s wr, 236 op/s 2024-06-02T23:33:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:33:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:33:35.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:35.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:35 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:33:36.125 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-06-02T23:33:36.125 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-02T23:33:36.125 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:33:36.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 64249 6.88 MB/sec execute 404 sec latency 81.806 ms 2024-06-02T23:33:37.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 64443 6.90 MB/sec execute 405 sec latency 91.414 ms 2024-06-02T23:33:37.493 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:37 smithi052 ceph-mon[32340]: pgmap v1083: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 334 op/s 2024-06-02T23:33:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:37 smithi174 ceph-mon[35994]: pgmap v1083: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 334 op/s 2024-06-02T23:33:37.657 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:37 smithi123 ceph-mon[35977]: pgmap v1083: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 5.0 MiB/s wr, 334 op/s 2024-06-02T23:33:38.055 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.968+0000 7f471b6da700 1 -- 172.21.15.52:0/712201932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714107830 msgr2=0x7f471410f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.968+0000 7f471b6da700 1 --2- 172.21.15.52:0/712201932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714107830 0x7f471410f3b0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f470c00b0d0 tx=0x7f470c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 -- 172.21.15.52:0/712201932 shutdown_connections 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 --2- 172.21.15.52:0/712201932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714107830 0x7f471410f3b0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 --2- 172.21.15.52:0/712201932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714102740 0x7f4714102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 --2- 172.21.15.52:0/712201932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 0x7f4714107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 -- 172.21.15.52:0/712201932 >> 172.21.15.52:0/712201932 conn(0x7f47140795b0 msgr2=0x7f47140799c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 -- 172.21.15.52:0/712201932 shutdown_connections 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 -- 172.21.15.52:0/712201932 wait complete. 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.969+0000 7f471b6da700 1 Processor -- start 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 -- start start 2024-06-02T23:33:38.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 0x7f47141a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 0x7f47141a0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714107830 0x7f471419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4714112620 con 0x7f4714102740 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47141124a0 con 0x7f47141030f0 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f471b6da700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47141127a0 con 0x7f4714107830 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 0x7f47141a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:33:38.057 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: "03b50edb-4ff4-42e7-823b-35a0b3ca6d61": { 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "03b50edb-4ff4-42e7-823b-35a0b3ca6d61", 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:38.058 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 0x7f47141a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52504/0 (socket says 172.21.15.52:52504) 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 -- 172.21.15.52:0/3834492045 learned_addr learned my addr 172.21.15.52:0/3834492045 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4718c75700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 0x7f47141a0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719c77700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714107830 0x7f471419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 -- 172.21.15.52:0/3834492045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714107830 msgr2=0x7f471419aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714107830 0x7f471419aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 -- 172.21.15.52:0/3834492045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 msgr2=0x7f47141a0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 0x7f47141a0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.059 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719476700 1 -- 172.21.15.52:0/3834492045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471419b240 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4719c77700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714107830 0x7f471419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.970+0000 7f4718c75700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 0x7f47141a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.971+0000 7f4719476700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 0x7f47141a0920 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9c0 tx=0x7f471000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.971+0000 7f470a7fc700 1 -- 172.21.15.52:0/3834492045 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471000cbd0 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.971+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471419b530 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.971+0000 7f470a7fc700 1 -- 172.21.15.52:0/3834492045 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f471000cd70 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.971+0000 7f470a7fc700 1 -- 172.21.15.52:0/3834492045 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47100056c0 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.971+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47141a77f0 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.974+0000 7f470a7fc700 1 -- 172.21.15.52:0/3834492045 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4710004750 con 0x7f4714102740 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.975+0000 7f470a7fc700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47000a2ca0 0x7f47000a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.975+0000 7f4718c75700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47000a2ca0 0x7f47000a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.975+0000 7f470a7fc700 1 -- 172.21.15.52:0/3834492045 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f47100cf360 con 0x7f4714102740 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.976+0000 7f4718c75700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47000a2ca0 0x7f47000a5160 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4704009680 tx=0x7f4704009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.976+0000 7f4725f5db80 1 Processor -- start 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.976+0000 7f4725f5db80 1 -- start start 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 0x555ba7c49a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c43f20 0x555ba7c44340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4725f5db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ba7b95810 con 0x555ba7c49640 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4725f5db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ba7b3d720 con 0x555ba7c43f20 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4725f5db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555ba7b917d0 con 0x555ba7c49f60 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 0x555ba7c49a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4718c75700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 0x555ba7c49a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52508/0 (socket says 172.21.15.52:52508) 2024-06-02T23:33:38.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4719476700 1 -- 172.21.15.52:0/1226410820 learned_addr learned my addr 172.21.15.52:0/1226410820 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4719c77700 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c43f20 0x555ba7c44340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4718c75700 1 -- 172.21.15.52:0/1226410820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c43f20 msgr2=0x555ba7c44340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4718c75700 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c43f20 0x555ba7c44340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4718c75700 1 -- 172.21.15.52:0/1226410820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 msgr2=0x555ba7c49a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4718c75700 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 0x555ba7c49a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4718c75700 1 -- 172.21.15.52:0/1226410820 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba7b33f50 con 0x555ba7c49f60 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.977+0000 7f4719476700 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 0x555ba7c49a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.978+0000 7f4718c75700 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c439e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f470400e860 tx=0x7f4704014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdb0ada03def2841 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.978+0000 7f46ff7fe700 1 -- 172.21.15.52:0/1226410820 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470401a660 con 0x555ba7c49f60 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.978+0000 7f46ff7fe700 1 -- 172.21.15.52:0/1226410820 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f470401acb0 con 0x555ba7c49f60 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.978+0000 7f46ff7fe700 1 -- 172.21.15.52:0/1226410820 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470401bde0 con 0x555ba7c49f60 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 -- 172.21.15.52:0/1226410820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 msgr2=0x555ba7c439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c439e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f470400e860 tx=0x7f4704014000 comp rx=0 tx=0).stop 2024-06-02T23:33:38.062 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 -- 172.21.15.52:0/1226410820 shutdown_connections 2024-06-02T23:33:38.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c43f20 0x555ba7c44340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c49640 0x555ba7c49a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 --2- 172.21.15.52:0/1226410820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c439e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.979+0000 7f4725f5db80 1 -- 172.21.15.52:0/1226410820 >> 172.21.15.52:0/1226410820 conn(0x555ba7b6ecc0 msgr2=0x555ba7c3e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:38.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.980+0000 7f4725f5db80 1 -- 172.21.15.52:0/1226410820 shutdown_connections 2024-06-02T23:33:38.063 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.980+0000 7f4725f5db80 1 -- 172.21.15.52:0/1226410820 wait complete. 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.980+0000 7f4725f5db80 1 -- start start 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 Processor -- start 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c4fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c4cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c4d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ba7c509d0 con 0x555ba7c43f20 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ba7b35bf0 con 0x555ba7c49f60 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555ba7c50b50 con 0x555ba7c49640 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4725f5db80 10 client.0 _ll_register_callbacks cb 0x555ba7b2f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4718c75700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c4cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.981+0000 7f4719c77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c4d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c4d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35058/0 (socket says 172.21.15.52:35058) 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 -- 172.21.15.52:0/2220432766 learned_addr learned my addr 172.21.15.52:0/2220432766 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719476700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c4fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 -- 172.21.15.52:0/2220432766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 msgr2=0x555ba7c4cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c4cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 -- 172.21.15.52:0/2220432766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 msgr2=0x555ba7c4fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c4fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 -- 172.21.15.52:0/2220432766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba7ae48d0 con 0x555ba7c49f60 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719476700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4719c77700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c4d510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f470c00bd00 tx=0x7f470c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7e817044ce06c93 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f46fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f46fcff9700 1 -- 172.21.15.52:0/2220432766 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c004c50 con 0x555ba7c49f60 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.982+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470401acb0 con 0x555ba7c49f60 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f46fcff9700 1 -- 172.21.15.52:0/2220432766 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f470c004df0 con 0x555ba7c49f60 2024-06-02T23:33:38.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f46fcff9700 1 -- 172.21.15.52:0/2220432766 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f470c0074e0 con 0x555ba7c49f60 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 10 client.30410 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ba7c5b350 con 0x555ba7c49f60 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f46fcff9700 1 -- 172.21.15.52:0/2220432766 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f470c007b30 con 0x555ba7c49f60 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f46fcff9700 1 -- 172.21.15.52:0/2220432766 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f470c011070 con 0x555ba7c49f60 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 10 client.30410 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 10 client.30410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 20 client.30410 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7c601a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 4 client.30410 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.983+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f470401acb0 con 0x555ba7c5dd60 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.984+0000 7f4719476700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7c601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.984+0000 7f46fcff9700 10 client.30410 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:38.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.984+0000 7f4719476700 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7c601a0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15c92349b9336964 server_cookie=3419e70a50b7e147 in_seq=0 out_seq=0 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f46fcff9700 1 -- 172.21.15.52:0/2220432766 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f470401acb0 con 0x555ba7c5dd60 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f46fcff9700 10 client.30410 handle_command_reply: tid=0 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 client.30410 shutdown 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 msgr2=0x555ba7c4d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c4d510 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f470c00bd00 tx=0x7f470c00bd30 comp rx=0 tx=0).stop 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 shutdown_connections 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7c601a0 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c4d510 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c4fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f46fcff9700 0 client.30410 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 --2- 172.21.15.52:0/2220432766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c4cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 >> 172.21.15.52:0/2220432766 conn(0x555ba7b6ecc0 msgr2=0x555ba7c3e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 shutdown_connections 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.985+0000 7f4725f5db80 1 -- 172.21.15.52:0/2220432766 wait complete. 2024-06-02T23:33:38.067 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:37.986+0000 7f4725f5db80 20 client.30410 trim_cache size 0 max 16384 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 Processor -- start 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 -- start start 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c439e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49640 0x555ba7c4d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c60180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f47141aebe0 con 0x555ba7c43f20 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47141b0720 con 0x555ba7c49640 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.026+0000 7f4725f5db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47141af2b0 con 0x555ba7c49f60 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c439e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52518/0 (socket says 172.21.15.52:52518) 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 -- 172.21.15.52:0/69163772 learned_addr learned my addr 172.21.15.52:0/69163772 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4718c75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49640 0x555ba7c4d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719c77700 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c60180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.068 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 -- 172.21.15.52:0/69163772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 msgr2=0x555ba7c60180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c60180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 -- 172.21.15.52:0/69163772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49640 msgr2=0x555ba7c4d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49640 0x555ba7c4d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 -- 172.21.15.52:0/69163772 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ba7c4b540 con 0x555ba7c43f20 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.027+0000 7f4719476700 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c439e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f471000efe0 tx=0x7f4710009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=973d41a91050dc87 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.028+0000 7f46feffd700 1 -- 172.21.15.52:0/69163772 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47100ba070 con 0x555ba7c43f20 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.028+0000 7f46feffd700 1 -- 172.21.15.52:0/69163772 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4710063030 con 0x555ba7c43f20 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.028+0000 7f46feffd700 1 -- 172.21.15.52:0/69163772 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710004d70 con 0x555ba7c43f20 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.028+0000 7f4725f5db80 1 -- 172.21.15.52:0/69163772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 msgr2=0x555ba7c439e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.028+0000 7f4725f5db80 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c439e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f471000efe0 tx=0x7f4710009830 comp rx=0 tx=0).stop 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 -- 172.21.15.52:0/69163772 shutdown_connections 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49f60 0x555ba7c60180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c439e0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 --2- 172.21.15.52:0/69163772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49640 0x555ba7c4d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 -- 172.21.15.52:0/69163772 >> 172.21.15.52:0/69163772 conn(0x555ba7b6ecc0 msgr2=0x555ba7c4cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 -- 172.21.15.52:0/69163772 shutdown_connections 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 -- 172.21.15.52:0/69163772 wait complete. 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 -- start start 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.029+0000 7f4725f5db80 1 Processor -- start 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c56100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f47141b0050 con 0x555ba7c43f20 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f47141af980 con 0x555ba7c49f60 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47141acb80 con 0x555ba7c49640 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4725f5db80 10 client.0 _ll_register_callbacks cb 0x555ba7b2f920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:38.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c56100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719476700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c56100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52530/0 (socket says 172.21.15.52:52530) 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719476700 1 -- 172.21.15.52:0/914475684 learned_addr learned my addr 172.21.15.52:0/914475684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4718c75700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 -- 172.21.15.52:0/914475684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 msgr2=0x555ba7c56640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c56640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 -- 172.21.15.52:0/914475684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 msgr2=0x555ba7c56100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c56100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 -- 172.21.15.52:0/914475684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ba7c658e0 con 0x555ba7c49f60 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719476700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f4719c77700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c650a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f470c005760 tx=0x7f470c005e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d3a83bb2454cf66 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.030+0000 7f46fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f46fcff9700 1 -- 172.21.15.52:0/914475684 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c00b770 con 0x555ba7c49f60 2024-06-02T23:33:38.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f46fcff9700 1 -- 172.21.15.52:0/914475684 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f470c007890 con 0x555ba7c49f60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4710063030 con 0x555ba7c49f60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f46fcff9700 1 -- 172.21.15.52:0/914475684 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f470c007a90 con 0x555ba7c49f60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 10 client.30416 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ba7cc85f0 con 0x555ba7c49f60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f46fcff9700 1 -- 172.21.15.52:0/914475684 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4710063030 con 0x555ba7c49f60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f46fcff9700 1 -- 172.21.15.52:0/914475684 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f470c011040 con 0x555ba7c49f60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 10 client.30416 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 10 client.30416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 20 client.30416 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.031+0000 7f4725f5db80 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7ccad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f4725f5db80 4 client.30416 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555ba7ba2d10 con 0x555ba7c5dd60 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f4719476700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7ccad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f46fcff9700 10 client.30416 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:38.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f4719476700 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7ccad30 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1af0b380f56f4393 server_cookie=71c743685b19b2a9 in_seq=0 out_seq=0 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f46fcff9700 1 -- 172.21.15.52:0/914475684 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x555ba7ba2d10 con 0x555ba7c5dd60 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f46fcff9700 10 client.30416 handle_command_reply: tid=0 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.032+0000 7f4725f5db80 1 client.30416 shutdown 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 msgr2=0x555ba7c650a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c650a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f470c005760 tx=0x7f470c005e90 comp rx=0 tx=0).stop 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 shutdown_connections 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ba7c5dd60 0x555ba7ccad30 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ba7c49f60 0x555ba7c650a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ba7c43f20 0x555ba7c56100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 --2- 172.21.15.52:0/914475684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ba7c49640 0x555ba7c56640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f46fcff9700 0 client.30416 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 >> 172.21.15.52:0/914475684 conn(0x555ba7b6ecc0 msgr2=0x555ba7c4a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 shutdown_connections 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.033+0000 7f4725f5db80 1 -- 172.21.15.52:0/914475684 wait complete. 2024-06-02T23:33:38.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f4725f5db80 20 client.30416 trim_cache size 0 max 16384 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47000a2ca0 msgr2=0x7f47000a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47000a2ca0 0x7f47000a5160 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4704009680 tx=0x7f4704009260 comp rx=0 tx=0).stop 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 msgr2=0x7f47141a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 0x7f47141a0920 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7f471000e9c0 tx=0x7f471000ecd0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 shutdown_connections 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f47000a2ca0 0x7f47000a5160 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4714107830 0x7f471419aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4714102740 0x7f47141a0920 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 --2- 172.21.15.52:0/3834492045 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f47141030f0 0x7f47141a0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:38.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.034+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 >> 172.21.15.52:0/3834492045 conn(0x7f47140795b0 msgr2=0x7f4714100740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:38.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.035+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 shutdown_connections 2024-06-02T23:33:38.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:38.035+0000 7f471b6da700 1 -- 172.21.15.52:0/3834492045 wait complete. 2024-06-02T23:33:38.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 64641 6.91 MB/sec execute 406 sec latency 102.184 ms 2024-06-02T23:33:39.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 64817 6.91 MB/sec execute 407 sec latency 59.368 ms 2024-06-02T23:33:39.540 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:39 smithi052 ceph-mon[32340]: pgmap v1084: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-06-02T23:33:39.576 INFO:tasks.cephfs.filesystem:scrub status for tag:03b50edb-4ff4-42e7-823b-35a0b3ca6d61 - {'path': '/', 'tag': '03b50edb-4ff4-42e7-823b-35a0b3ca6d61', 'options': 'recursive,force'} 2024-06-02T23:33:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:39 smithi174 ceph-mon[35994]: pgmap v1084: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-06-02T23:33:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:39 smithi123 ceph-mon[35977]: pgmap v1084: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.0 MiB/s wr, 316 op/s 2024-06-02T23:33:40.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 64964 6.91 MB/sec execute 408 sec latency 92.332 ms 2024-06-02T23:33:40.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.674+0000 7f1484ff4700 1 -- 172.21.15.52:0/2682004755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f148010eac0 msgr2=0x7f148010eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.674+0000 7f1484ff4700 1 --2- 172.21.15.52:0/2682004755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f148010eac0 0x7f148010eea0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f1470009990 tx=0x7f1470009ca0 comp rx=0 tx=0).stop 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 -- 172.21.15.52:0/2682004755 shutdown_connections 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 --2- 172.21.15.52:0/2682004755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1480109480 0x7f1480115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 --2- 172.21.15.52:0/2682004755 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f148010eac0 0x7f148010eea0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 --2- 172.21.15.52:0/2682004755 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480108ac0 0x7f1480108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 -- 172.21.15.52:0/2682004755 >> 172.21.15.52:0/2682004755 conn(0x7f1480075c60 msgr2=0x7f1480078080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 -- 172.21.15.52:0/2682004755 shutdown_connections 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.675+0000 7f1484ff4700 1 -- 172.21.15.52:0/2682004755 wait complete. 2024-06-02T23:33:40.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.676+0000 7f1484ff4700 1 Processor -- start 2024-06-02T23:33:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.676+0000 7f1484ff4700 1 -- start start 2024-06-02T23:33:40.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.677+0000 7f1484ff4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1480108ac0 0x7f148019efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.677+0000 7f147e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1480108ac0 0x7f148019efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.677+0000 7f147e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1480108ac0 0x7f148019efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52544/0 (socket says 172.21.15.52:52544) 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.677+0000 7f1484ff4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480109480 0x7f148019f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.677+0000 7f147e59c700 1 -- 172.21.15.52:0/3117956516 learned_addr learned my addr 172.21.15.52:0/3117956516 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148010eac0 0x7f14801a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1480118cf0 con 0x7f1480108ac0 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480109480 0x7f148019f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1480118b70 con 0x7f1480109480 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1480118e70 con 0x7f148010eac0 2024-06-02T23:33:40.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147ed9d700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148010eac0 0x7f14801a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:40.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 -- 172.21.15.52:0/3117956516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148010eac0 msgr2=0x7f14801a3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:40.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148010eac0 0x7f14801a3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:40.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 -- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1480108ac0 msgr2=0x7f148019efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:40.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1480108ac0 0x7f148019efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:40.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14801a3fd0 con 0x7f1480109480 2024-06-02T23:33:40.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.678+0000 7f147dd9b700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480109480 0x7f148019f500 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f147400e9f0 tx=0x7f147400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:40.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.679+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147400cb70 con 0x7f1480109480 2024-06-02T23:33:40.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.679+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14801a42c0 con 0x7f1480109480 2024-06-02T23:33:40.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.679+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14801abb40 con 0x7f1480109480 2024-06-02T23:33:40.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.679+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f147400ee90 con 0x7f1480109480 2024-06-02T23:33:40.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.679+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1474003cd0 con 0x7f1480109480 2024-06-02T23:33:40.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.680+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1460005360 con 0x7f1480109480 2024-06-02T23:33:40.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.682+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1474004020 con 0x7f1480109480 2024-06-02T23:33:40.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.682+0000 7f146f7fe700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14680a2ca0 0x7f14680a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:40.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.683+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f14740cef30 con 0x7f1480109480 2024-06-02T23:33:40.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.683+0000 7f147e59c700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14680a2ca0 0x7f14680a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:40.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.683+0000 7f147e59c700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14680a2ca0 0x7f14680a5160 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1470009350 tx=0x7f1470005a90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:40.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:40.688+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f147409bec0 con 0x7f1480109480 2024-06-02T23:33:41.118 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.118+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1460005650 con 0x7f1480109480 2024-06-02T23:33:41.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.119+0000 7f146f7fe700 1 -- 172.21.15.52:0/3117956516 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f147409b7c0 con 0x7f1480109480 2024-06-02T23:33:41.119 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:33:41.119 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:33:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14680a2ca0 msgr2=0x7f14680a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14680a2ca0 0x7f14680a5160 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f1470009350 tx=0x7f1470005a90 comp rx=0 tx=0).stop 2024-06-02T23:33:41.123 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480109480 msgr2=0x7f148019f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480109480 0x7f148019f500 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f147400e9f0 tx=0x7f147400ed00 comp rx=0 tx=0).stop 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 shutdown_connections 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14680a2ca0 0x7f14680a5160 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f148010eac0 0x7f14801a3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1480108ac0 0x7f148019efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 --2- 172.21.15.52:0/3117956516 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1480109480 0x7f148019f500 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 >> 172.21.15.52:0/3117956516 conn(0x7f1480075c60 msgr2=0x7f1480076040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 shutdown_connections 2024-06-02T23:33:41.124 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:41.122+0000 7f1484ff4700 1 -- 172.21.15.52:0/3117956516 wait complete. 2024-06-02T23:33:41.125 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: pgmap v1085: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:33:41.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:41 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3117956516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:33:41.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 65129 6.91 MB/sec execute 409 sec latency 94.630 ms 2024-06-02T23:33:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: pgmap v1085: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-06-02T23:33:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:33:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:33:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:33:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:33:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:33:41.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:41 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3117956516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: pgmap v1085: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.6 MiB/s wr, 353 op/s 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:33:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:41 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3117956516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:33:42.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 65271 6.91 MB/sec execute 410 sec latency 194.718 ms 2024-06-02T23:33:43.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 65404 6.90 MB/sec execute 411 sec latency 158.349 ms 2024-06-02T23:33:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:43 smithi174 ceph-mon[35994]: pgmap v1086: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 349 op/s 2024-06-02T23:33:43.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:43 smithi052 ceph-mon[32340]: pgmap v1086: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 349 op/s 2024-06-02T23:33:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:43 smithi123 ceph-mon[35977]: pgmap v1086: 97 pgs: 97 active+clean; 535 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.2 MiB/s wr, 349 op/s 2024-06-02T23:33:44.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 65556 6.91 MB/sec execute 412 sec latency 112.085 ms 2024-06-02T23:33:45.011 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-02T23:33:45.011 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-02T23:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 -- 172.21.15.52:0/1500202883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22141024b0 msgr2=0x7f2214069050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 --2- 172.21.15.52:0/1500202883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22141024b0 0x7f2214069050 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f2204009900 tx=0x7f220402a3f0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 -- 172.21.15.52:0/1500202883 shutdown_connections 2024-06-02T23:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 --2- 172.21.15.52:0/1500202883 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2214101af0 0x7f2214101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 --2- 172.21.15.52:0/1500202883 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f22141024b0 0x7f2214069050 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 --2- 172.21.15.52:0/1500202883 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2214100900 0x7f2214100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.373+0000 7f221a3dc640 1 -- 172.21.15.52:0/1500202883 >> 172.21.15.52:0/1500202883 conn(0x7f22140fbe70 msgr2=0x7f22140fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.374+0000 7f221a3dc640 1 -- 172.21.15.52:0/1500202883 shutdown_connections 2024-06-02T23:33:45.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.374+0000 7f221a3dc640 1 -- 172.21.15.52:0/1500202883 wait complete. 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.374+0000 7f221a3dc640 1 Processor -- start 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.374+0000 7f221a3dc640 1 -- start start 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f221a3dc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2214100900 0x7f2214199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f221a3dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 0x7f221419a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f221a3dc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141024b0 0x7f22141a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f221a3dc640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f221410f750 con 0x7f22141024b0 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f221a3dc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f221410f5d0 con 0x7f2214101af0 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f221a3dc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f221410f8d0 con 0x7f2214100900 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f22137fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 0x7f221419a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f22137fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 0x7f221419a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:47658/0 (socket says 172.21.15.52:47658) 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f22137fe640 1 -- 172.21.15.52:0/405140459 learned_addr learned my addr 172.21.15.52:0/405140459 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f2213fff640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2214100900 0x7f2214199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.375+0000 7f2218979640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141024b0 0x7f22141a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22137fe640 1 -- 172.21.15.52:0/405140459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2214100900 msgr2=0x7f2214199d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22137fe640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2214100900 0x7f2214199d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22137fe640 1 -- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141024b0 msgr2=0x7f22141a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22137fe640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141024b0 0x7f22141a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22137fe640 1 -- 172.21.15.52:0/405140459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22141a1940 con 0x7f2214101af0 2024-06-02T23:33:45.377 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22137fe640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 0x7f221419a270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f220800ea60 tx=0x7f220800ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22117fa640 1 -- 172.21.15.52:0/405140459 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220800cde0 con 0x7f2214101af0 2024-06-02T23:33:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22141a1c30 con 0x7f2214101af0 2024-06-02T23:33:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22117fa640 1 -- 172.21.15.52:0/405140459 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f22080035f0 con 0x7f2214101af0 2024-06-02T23:33:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f22117fa640 1 -- 172.21.15.52:0/405140459 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208010690 con 0x7f2214101af0 2024-06-02T23:33:45.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.376+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22141a21d0 con 0x7f2214101af0 2024-06-02T23:33:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.378+0000 7f22117fa640 1 -- 172.21.15.52:0/405140459 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f22080037a0 con 0x7f2214101af0 2024-06-02T23:33:45.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.379+0000 7f22117fa640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f21fc0a2d00 0x7f21fc0a51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.379+0000 7f22117fa640 1 -- 172.21.15.52:0/405140459 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f22080cf360 con 0x7f2214101af0 2024-06-02T23:33:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 Processor -- start 2024-06-02T23:33:45.380 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 -- start start 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f2213fff640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f21fc0a2d00 0x7f21fc0a51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d0c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0d5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5621ee02b4f0 con 0x5621ee0d1210 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5621edfc9870 con 0x5621ee0d6380 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f221d1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621ee0274b0 con 0x5621ee0d0820 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f22137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0d5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f22137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0d5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58918/0 (socket says 172.21.15.52:58918) 2024-06-02T23:33:45.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f22137fe640 1 -- 172.21.15.52:0/3253754378 learned_addr learned my addr 172.21.15.52:0/3253754378 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.380+0000 7f2218979640 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f22137fe640 1 -- 172.21.15.52:0/3253754378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 msgr2=0x5621ee0d0c40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:33:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f22137fe640 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d0c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f22137fe640 1 -- 172.21.15.52:0/3253754378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 msgr2=0x5621ee0d8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f22137fe640 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f22137fe640 1 -- 172.21.15.52:0/3253754378 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621edfe59b0 con 0x5621ee0d1210 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f22137fe640 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0d5d10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f220809bb80 tx=0x7f220809b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7636be4c2f6dd08 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f2213fff640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f21fc0a2d00 0x7f21fc0a51c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f221419b200 tx=0x7f220c009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f21fa7fc640 1 -- 172.21.15.52:0/3253754378 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220809c2f0 con 0x5621ee0d1210 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f21fa7fc640 1 -- 172.21.15.52:0/3253754378 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f220809bbf0 con 0x5621ee0d1210 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.381+0000 7f21fa7fc640 1 -- 172.21.15.52:0/3253754378 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208004260 con 0x5621ee0d1210 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.382+0000 7f221d1c7740 1 -- 172.21.15.52:0/3253754378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 msgr2=0x5621ee0d5d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.383 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.382+0000 7f221d1c7740 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0d5d10 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7f220809bb80 tx=0x7f220809b550 comp rx=0 tx=0).stop 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 -- 172.21.15.52:0/3253754378 shutdown_connections 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0d5d10 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 --2- 172.21.15.52:0/3253754378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 -- 172.21.15.52:0/3253754378 >> 172.21.15.52:0/3253754378 conn(0x5621ee0c3680 msgr2=0x5621ee0c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 -- 172.21.15.52:0/3253754378 shutdown_connections 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 -- 172.21.15.52:0/3253754378 wait complete. 2024-06-02T23:33:45.384 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.383+0000 7f221d1c7740 1 -- start start 2024-06-02T23:33:45.385 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 Processor -- start 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d0820 0x5621ee0da9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0daf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5621ee0d92d0 con 0x5621ee0d0820 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5621edfb4ae0 con 0x5621ee0d6380 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621ee0d9450 con 0x5621ee0d1210 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f221d1c7740 10 client.0 _ll_register_callbacks cb 0x5621edfed9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f22137fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f2213fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d0820 0x5621ee0da9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.384+0000 7f22137fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0daf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35118/0 (socket says 172.21.15.52:35118) 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 -- 172.21.15.52:0/2383210021 learned_addr learned my addr 172.21.15.52:0/2383210021 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f2218979640 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 -- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 msgr2=0x5621ee0e1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0e1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 -- 172.21.15.52:0/2383210021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d0820 msgr2=0x5621ee0da9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d0820 0x5621ee0da9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 -- 172.21.15.52:0/2383210021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621edd3f210 con 0x5621ee0d1210 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f22137fe640 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0daf10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f220800d010 tx=0x7f2208097e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=81b7b68f018842f0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f21f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f21f37fe640 1 -- 172.21.15.52:0/2383210021 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220807f060 con 0x5621ee0d1210 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f21f37fe640 1 -- 172.21.15.52:0/2383210021 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2208004210 con 0x5621ee0d1210 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.385+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220809bbf0 con 0x5621ee0d1210 2024-06-02T23:33:45.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f21f37fe640 1 -- 172.21.15.52:0/2383210021 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f22080a4860 con 0x5621ee0d1210 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 10 client.30040 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5621ede0b620 con 0x5621ee0d1210 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f21f37fe640 1 -- 172.21.15.52:0/2383210021 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f220809bbf0 con 0x5621ee0d1210 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f21f37fe640 1 -- 172.21.15.52:0/2383210021 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f22080978b0 con 0x5621ee0d1210 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 10 client.30040 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 10 client.30040 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 20 client.30040 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee0e6440 0x5621ee0e8830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 4 client.30040 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621ee03ef10 con 0x5621ee0e6440 2024-06-02T23:33:45.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.386+0000 7f2213fff640 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee0e6440 0x5621ee0e8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.387+0000 7f21f37fe640 10 client.30040 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.387+0000 7f2213fff640 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee0e6440 0x5621ee0e8830 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=171724870461fd5c server_cookie=baa9c0c552b99640 in_seq=0 out_seq=0 2024-06-02T23:33:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f21f37fe640 1 -- 172.21.15.52:0/2383210021 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5621ee03ef10 con 0x5621ee0e6440 2024-06-02T23:33:45.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f21f37fe640 10 client.30040 handle_command_reply: tid=0 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 client.30040 shutdown 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 msgr2=0x5621ee0daf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0daf10 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f220800d010 tx=0x7f2208097e00 comp rx=0 tx=0).stop 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 shutdown_connections 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee0e6440 0x5621ee0e8830 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0daf10 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0e1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2383210021 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d0820 0x5621ee0da9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f21f37fe640 0 client.30040 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 >> 172.21.15.52:0/2383210021 conn(0x5621ee0c3680 msgr2=0x5621ee0c27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 shutdown_connections 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 1 -- 172.21.15.52:0/2383210021 wait complete. 2024-06-02T23:33:45.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.388+0000 7f221d1c7740 20 client.30040 trim_cache size 0 max 16384 2024-06-02T23:33:45.392 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:33:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.414+0000 7f221d1c7740 1 Processor -- start 2024-06-02T23:33:45.415 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.414+0000 7f221d1c7740 1 -- start start 2024-06-02T23:33:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0e1f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f221d1c7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a6ad0 con 0x5621ee0d1210 2024-06-02T23:33:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f221d1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a8610 con 0x5621ee0d6380 2024-06-02T23:33:45.416 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f221d1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a71a0 con 0x5621ee0d0820 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f2213fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0e1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f2213fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:35134/0 (socket says 172.21.15.52:35134) 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f2213fff640 1 -- 172.21.15.52:0/2668259108 learned_addr learned my addr 172.21.15.52:0/2668259108 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0e1f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58936/0 (socket says 172.21.15.52:58936) 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f2218979640 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 -- 172.21.15.52:0/2668259108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 msgr2=0x5621ee0d5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 -- 172.21.15.52:0/2668259108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 msgr2=0x5621ee0d23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f22137fe640 1 -- 172.21.15.52:0/2668259108 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621ee0e8ba0 con 0x5621ee0d1210 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.415+0000 7f2213fff640 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.416+0000 7f22137fe640 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0e1f10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f22080cd200 tx=0x7f220809b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d585b4f526035b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:45.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.416+0000 7f21f9ffb640 1 -- 172.21.15.52:0/2668259108 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220807f040 con 0x5621ee0d1210 2024-06-02T23:33:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.416+0000 7f21f9ffb640 1 -- 172.21.15.52:0/2668259108 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f220809c2f0 con 0x5621ee0d1210 2024-06-02T23:33:45.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.416+0000 7f21f9ffb640 1 -- 172.21.15.52:0/2668259108 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080978b0 con 0x5621ee0d1210 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.416+0000 7f221d1c7740 1 -- 172.21.15.52:0/2668259108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 msgr2=0x5621ee0e1f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.416+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0e1f10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f22080cd200 tx=0x7f220809b550 comp rx=0 tx=0).stop 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 -- 172.21.15.52:0/2668259108 shutdown_connections 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d6380 0x5621ee0d23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d1210 0x5621ee0e1f10 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 --2- 172.21.15.52:0/2668259108 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d0820 0x5621ee0d5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 -- 172.21.15.52:0/2668259108 >> 172.21.15.52:0/2668259108 conn(0x5621ee0c3680 msgr2=0x5621ee0d97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 -- 172.21.15.52:0/2668259108 shutdown_connections 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 -- 172.21.15.52:0/2668259108 wait complete. 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 -- start start 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 Processor -- start 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 0x5621ee0dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.417+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0defb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f221d1c7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d6380 0x5621ee14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f221d1c7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a7f40 con 0x5621ee0d6380 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f221d1c7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a7870 con 0x5621ee0d0820 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f221d1c7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f22141a4ab0 con 0x5621ee0d1210 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f221d1c7740 10 client.0 _ll_register_callbacks cb 0x7f220800ef90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 0x5621ee0dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 0x5621ee0dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:47680/0 (socket says 172.21.15.52:47680) 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2218979640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d6380 0x5621ee14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 -- 172.21.15.52:0/1862546028 learned_addr learned my addr 172.21.15.52:0/1862546028 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f22137fe640 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0defb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 -- 172.21.15.52:0/1862546028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 msgr2=0x5621ee0defb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0defb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 -- 172.21.15.52:0/1862546028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d6380 msgr2=0x5621ee14efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d6380 0x5621ee14efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 -- 172.21.15.52:0/1862546028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621ee0e2080 con 0x5621ee0d0820 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f2213fff640 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 0x5621ee0dea70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5621ee0d1a70 tx=0x7f220c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34a2f5821abd36de server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.418+0000 7f21f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f21f37fe640 1 -- 172.21.15.52:0/1862546028 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220c00f680 con 0x5621ee0d0820 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f21f37fe640 1 -- 172.21.15.52:0/1862546028 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f220c00f820 con 0x5621ee0d0820 2024-06-02T23:33:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f220809c2f0 con 0x5621ee0d0820 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f21f37fe640 1 -- 172.21.15.52:0/1862546028 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f220c037050 con 0x5621ee0d0820 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f221d1c7740 10 client.30440 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5621ee150470 con 0x5621ee0d0820 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f21f37fe640 1 -- 172.21.15.52:0/1862546028 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f220c02b070 con 0x5621ee0d0820 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.419+0000 7f21f37fe640 1 -- 172.21.15.52:0/1862546028 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f220c01eae0 con 0x5621ee0d0820 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f221d1c7740 10 client.30440 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f221d1c7740 10 client.30440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f221d1c7740 20 client.30440 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f221d1c7740 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee150df0 0x5621ee153210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f221d1c7740 4 client.30440 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f220809c2f0 con 0x5621ee150df0 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f22137fe640 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee150df0 0x5621ee153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f22137fe640 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee150df0 0x5621ee153210 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182095e4a61fdd99 server_cookie=348178266de927c9 in_seq=0 out_seq=0 2024-06-02T23:33:45.424 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.420+0000 7f21f37fe640 10 client.30440 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f21f37fe640 1 -- 172.21.15.52:0/1862546028 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f220809c2f0 con 0x5621ee150df0 2024-06-02T23:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f21f37fe640 10 client.30440 handle_command_reply: tid=0 2024-06-02T23:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 client.30440 shutdown 2024-06-02T23:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 msgr2=0x5621ee0dea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 0x5621ee0dea70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x5621ee0d1a70 tx=0x7f220c014000 comp rx=0 tx=0).stop 2024-06-02T23:33:45.474 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 shutdown_connections 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5621ee150df0 0x5621ee153210 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621ee0d1210 0x5621ee0defb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f21f37fe640 0 client.30440 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5621ee0d6380 0x5621ee14efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 --2- 172.21.15.52:0/1862546028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5621ee0d0820 0x5621ee0dea70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 >> 172.21.15.52:0/1862546028 conn(0x5621ee0c3680 msgr2=0x5621ee0d23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 shutdown_connections 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 1 -- 172.21.15.52:0/1862546028 wait complete. 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.473+0000 7f221d1c7740 20 client.30440 trim_cache size 0 max 16384 2024-06-02T23:33:45.475 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:33:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:33:45.476 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:33:45.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 65724 6.92 MB/sec execute 413 sec latency 97.595 ms 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f21fc0a2d00 msgr2=0x7f21fc0a51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f21fc0a2d00 0x7f21fc0a51c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f221419b200 tx=0x7f220c009500 comp rx=0 tx=0).stop 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 msgr2=0x7f221419a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 0x7f221419a270 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f220800ea60 tx=0x7f220800ef20 comp rx=0 tx=0).stop 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 shutdown_connections 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f21fc0a2d00 0x7f21fc0a51c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f22141024b0 0x7f22141a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2214101af0 0x7f221419a270 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 --2- 172.21.15.52:0/405140459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2214100900 0x7f2214199d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 >> 172.21.15.52:0/405140459 conn(0x7f22140fbe70 msgr2=0x7f2214102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 shutdown_connections 2024-06-02T23:33:45.477 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:45.474+0000 7f221a3dc640 1 -- 172.21.15.52:0/405140459 wait complete. 2024-06-02T23:33:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:45 smithi174 ceph-mon[35994]: pgmap v1087: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-06-02T23:33:45.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:45 smithi052 ceph-mon[32340]: pgmap v1087: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-06-02T23:33:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:45 smithi123 ceph-mon[35977]: pgmap v1087: 97 pgs: 97 active+clean; 536 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 333 op/s 2024-06-02T23:33:46.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 65857 6.92 MB/sec execute 414 sec latency 147.600 ms 2024-06-02T23:33:47.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66011 6.93 MB/sec execute 415 sec latency 160.846 ms 2024-06-02T23:33:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:47 smithi174 ceph-mon[35994]: pgmap v1088: 97 pgs: 97 active+clean; 543 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 382 op/s 2024-06-02T23:33:47.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:47 smithi052 ceph-mon[32340]: pgmap v1088: 97 pgs: 97 active+clean; 543 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 382 op/s 2024-06-02T23:33:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:47 smithi123 ceph-mon[35977]: pgmap v1088: 97 pgs: 97 active+clean; 543 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.4 MiB/s wr, 382 op/s 2024-06-02T23:33:48.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66159 6.93 MB/sec execute 416 sec latency 124.819 ms 2024-06-02T23:33:49.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:49 smithi052 ceph-mon[32340]: pgmap v1089: 97 pgs: 97 active+clean; 543 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-06-02T23:33:49.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66326 6.94 MB/sec execute 417 sec latency 124.388 ms 2024-06-02T23:33:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:49 smithi174 ceph-mon[35994]: pgmap v1089: 97 pgs: 97 active+clean; 543 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-06-02T23:33:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:49 smithi123 ceph-mon[35977]: pgmap v1089: 97 pgs: 97 active+clean; 543 MiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-06-02T23:33:50.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66465 6.95 MB/sec execute 418 sec latency 159.545 ms 2024-06-02T23:33:51.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66614 6.96 MB/sec execute 419 sec latency 149.065 ms 2024-06-02T23:33:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:51 smithi174 ceph-mon[35994]: pgmap v1090: 97 pgs: 97 active+clean; 544 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 4.0 MiB/s wr, 277 op/s 2024-06-02T23:33:51.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:51 smithi052 ceph-mon[32340]: pgmap v1090: 97 pgs: 97 active+clean; 544 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 4.0 MiB/s wr, 277 op/s 2024-06-02T23:33:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:51 smithi123 ceph-mon[35977]: pgmap v1090: 97 pgs: 97 active+clean; 544 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 997 KiB/s rd, 4.0 MiB/s wr, 277 op/s 2024-06-02T23:33:52.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66724 6.97 MB/sec execute 420 sec latency 151.468 ms 2024-06-02T23:33:53.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66850 6.96 MB/sec execute 421 sec latency 158.873 ms 2024-06-02T23:33:53.559 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:53 smithi123 ceph-mon[35977]: pgmap v1091: 97 pgs: 97 active+clean; 547 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-06-02T23:33:53.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:53 smithi174 ceph-mon[35994]: pgmap v1091: 97 pgs: 97 active+clean; 547 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-06-02T23:33:53.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:53 smithi052 ceph-mon[32340]: pgmap v1091: 97 pgs: 97 active+clean; 547 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 268 op/s 2024-06-02T23:33:54.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 66985 6.95 MB/sec execute 422 sec latency 113.086 ms 2024-06-02T23:33:55.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:55 smithi052 ceph-mon[32340]: pgmap v1092: 97 pgs: 97 active+clean; 547 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.6 MiB/s wr, 226 op/s 2024-06-02T23:33:55.474 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67103 6.94 MB/sec execute 423 sec latency 139.162 ms 2024-06-02T23:33:55.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:55 smithi174 ceph-mon[35994]: pgmap v1092: 97 pgs: 97 active+clean; 547 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.6 MiB/s wr, 226 op/s 2024-06-02T23:33:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:55 smithi123 ceph-mon[35977]: pgmap v1092: 97 pgs: 97 active+clean; 547 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 770 KiB/s rd, 3.6 MiB/s wr, 226 op/s 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 -- 172.21.15.52:0/4224168429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 msgr2=0x7fd9ec1089b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/4224168429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 0x7fd9ec1089b0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0099c0 tx=0x7fd9dc009cd0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 -- 172.21.15.52:0/4224168429 shutdown_connections 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/4224168429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9ec100fd0 0x7fd9ec10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/4224168429 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 0x7fd9ec1089b0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/4224168429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec100a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 -- 172.21.15.52:0/4224168429 >> 172.21.15.52:0/4224168429 conn(0x7fd9ec0775c0 msgr2=0x7fd9ec0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 -- 172.21.15.52:0/4224168429 shutdown_connections 2024-06-02T23:33:56.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.023+0000 7fd9f23c9700 1 -- 172.21.15.52:0/4224168429 wait complete. 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 Processor -- start 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 -- start start 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec1a08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9ec100fd0 0x7fd9ec1a0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 0x7fd9ec19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec114930 con 0x7fd9ec1085d0 2024-06-02T23:33:56.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1147b0 con 0x7fd9ec100610 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9f23c9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec114ab0 con 0x7fd9ec100fd0 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.024+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec1a08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec1a08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53282/0 (socket says 172.21.15.52:53282) 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 -- 172.21.15.52:0/1574896809 learned_addr learned my addr 172.21.15.52:0/1574896809 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9f0966700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 0x7fd9ec19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9ec100fd0 0x7fd9ec1a0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 -- 172.21.15.52:0/1574896809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9ec100fd0 msgr2=0x7fd9ec1a0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9ec100fd0 0x7fd9ec1a0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 -- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 msgr2=0x7fd9ec19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 0x7fd9ec19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 -- 172.21.15.52:0/1574896809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ec19b1d0 con 0x7fd9ec100610 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec1a08b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc006010 tx=0x7fd9dc004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:56.027 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9e97fa700 1 -- 172.21.15.52:0/1574896809 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc026070 con 0x7fd9ec100610 2024-06-02T23:33:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9e97fa700 1 -- 172.21.15.52:0/1574896809 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9dc003c70 con 0x7fd9ec100610 2024-06-02T23:33:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.025+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ec19b460 con 0x7fd9ec100610 2024-06-02T23:33:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.026+0000 7fd9e97fa700 1 -- 172.21.15.52:0/1574896809 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc00fb90 con 0x7fd9ec100610 2024-06-02T23:33:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.026+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ec1a7780 con 0x7fd9ec100610 2024-06-02T23:33:56.028 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.028+0000 7fd9e97fa700 1 -- 172.21.15.52:0/1574896809 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd9dc029070 con 0x7fd9ec100610 2024-06-02T23:33:56.029 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.029+0000 7fd9e97fa700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9d40a2d40 0x7fd9d40a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.029+0000 7fd9e97fa700 1 -- 172.21.15.52:0/1574896809 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd9dc0cf730 con 0x7fd9ec100610 2024-06-02T23:33:56.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.029+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9d40a2d40 0x7fd9d40a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.030 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.030+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9d40a2d40 0x7fd9d40a5200 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec107da0 tx=0x7fd9e0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.031+0000 7fd9fcc4cb80 1 Processor -- start 2024-06-02T23:33:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.031+0000 7fd9fcc4cb80 1 -- start start 2024-06-02T23:33:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.033 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9fcc4cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c868caf9a0 con 0x55c868d63cd0 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50022/0 (socket says 172.21.15.52:50022) 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9ebfff700 1 -- 172.21.15.52:0/3669109853 learned_addr learned my addr 172.21.15.52:0/3669109853 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d5df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9f0966700 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c868c578b0 con 0x55c868d5e460 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.032+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c868cab960 con 0x55c868d645f0 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9f0966700 1 -- 172.21.15.52:0/3669109853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 msgr2=0x55c868d5df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9f0966700 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d5df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9f0966700 1 -- 172.21.15.52:0/3669109853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 msgr2=0x55c868d640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.034 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9f0966700 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9f0966700 1 -- 172.21.15.52:0/3669109853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c868c4e0e0 con 0x55c868d5e460 2024-06-02T23:33:56.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:56.035 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9f0966700 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5e8a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4011580 tx=0x7fd9e4009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deca93170fd70050 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9d27fc700 1 -- 172.21.15.52:0/3669109853 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4011e80 con 0x55c868d5e460 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9d27fc700 1 -- 172.21.15.52:0/3669109853 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9e4004bb0 con 0x55c868d5e460 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.033+0000 7fd9d27fc700 1 -- 172.21.15.52:0/3669109853 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e4004740 con 0x55c868d5e460 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 msgr2=0x55c868d5e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5e8a0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4011580 tx=0x7fd9e4009920 comp rx=0 tx=0).stop 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 shutdown_connections 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5e8a0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/3669109853 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.039 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 >> 172.21.15.52:0/3669109853 conn(0x55c868c88e50 msgr2=0x55c868d591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 shutdown_connections 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.034+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/3669109853 wait complete. 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 -- start start 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 Processor -- start 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 0x55c868d6a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d676c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 0x55c868d6a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d645f0 0x55c868d67c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 0x55c868d6a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50024/0 (socket says 172.21.15.52:50024) 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9ebfff700 1 -- 172.21.15.52:0/764071995 learned_addr learned my addr 172.21.15.52:0/764071995 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c868d6b060 con 0x55c868d5e460 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c868c4fd80 con 0x55c868d645f0 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.035+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c868d6b1e0 con 0x55c868d63cd0 2024-06-02T23:33:56.040 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d676c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9f0966700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d645f0 0x55c868d67c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9fcc4cb80 10 client.0 _ll_register_callbacks cb 0x55c868c49ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9ebfff700 1 -- 172.21.15.52:0/764071995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 msgr2=0x55c868d676c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d676c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9ebfff700 1 -- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d645f0 msgr2=0x55c868d67c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d645f0 0x55c868d67c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9ebfff700 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c868bfea60 con 0x55c868d5e460 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 0x55c868d6a3c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0cd5d0 tx=0x7fd9dc003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5a52e25e8733f1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:56.041 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e4004bb0 con 0x55c868d5e460 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/764071995 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9dc09c050 con 0x55c868d5e460 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/764071995 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd9dc09a350 con 0x55c868d5e460 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/764071995 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd9dc0a0400 con 0x55c868d5e460 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9fcc4cb80 10 client.21177 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.036+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=43}) -- 0x55c868d758f0 con 0x55c868d5e460 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/764071995 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd9dc0a47a0 con 0x55c868d5e460 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9fcc4cb80 10 client.21177 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9fcc4cb80 10 client.21177 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9fcc4cb80 20 client.21177 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868d7a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9fcc4cb80 4 client.21177 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:56.042 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c868cbcea0 con 0x55c868d77dc0 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.037+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868d7a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.038+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868d7a200 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa5f8afb4403e3 server_cookie=da625593723aa7a2 in_seq=0 out_seq=0 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.038+0000 7fd9cb7fe700 10 client.21177 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.038+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/764071995 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c868cbcea0 con 0x55c868d77dc0 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.038+0000 7fd9cb7fe700 10 client.21177 handle_command_reply: tid=0 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 client.21177 shutdown 2024-06-02T23:33:56.043 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 msgr2=0x55c868d6a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 0x55c868d6a3c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc0cd5d0 tx=0x7fd9dc003f00 comp rx=0 tx=0).stop 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 shutdown_connections 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868d7a200 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9cb7fe700 0 client.21177 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d645f0 0x55c868d67c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d5e460 0x55c868d6a3c0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/764071995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d676c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 >> 172.21.15.52:0/764071995 conn(0x55c868c88e50 msgr2=0x55c868d58da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 shutdown_connections 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/764071995 wait complete. 2024-06-02T23:33:56.044 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.039+0000 7fd9fcc4cb80 20 client.21177 trim_cache size 0 max 16384 2024-06-02T23:33:56.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.073+0000 7fd9fcc4cb80 1 Processor -- start 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.073+0000 7fd9fcc4cb80 1 -- start start 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d61dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9fcc4cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1aea40 con 0x55c868d63cd0 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9fcc4cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1b0580 con 0x55c868d5e460 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9fcc4cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1af110 con 0x55c868d645f0 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d61dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d61dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50038/0 (socket says 172.21.15.52:50038) 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.075 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.074+0000 7fd9eb7fe700 1 -- 172.21.15.52:0/2433762649 learned_addr learned my addr 172.21.15.52:0/2433762649 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9eb7fe700 1 -- 172.21.15.52:0/2433762649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 msgr2=0x55c868d650e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9eb7fe700 1 -- 172.21.15.52:0/2433762649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 msgr2=0x55c868d5df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9eb7fe700 1 -- 172.21.15.52:0/2433762649 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c868d58f90 con 0x55c868d63cd0 2024-06-02T23:33:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d61dd0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00097d0 tx=0x7fd9e0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e35db36aad41c0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9d1ffb700 1 -- 172.21.15.52:0/2433762649 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0004220 con 0x55c868d63cd0 2024-06-02T23:33:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9d1ffb700 1 -- 172.21.15.52:0/2433762649 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9e00043c0 con 0x55c868d63cd0 2024-06-02T23:33:56.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.075+0000 7fd9d1ffb700 1 -- 172.21.15.52:0/2433762649 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e002a670 con 0x55c868d63cd0 2024-06-02T23:33:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/2433762649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 msgr2=0x55c868d61dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d61dd0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd9e00097d0 tx=0x7fd9e0014000 comp rx=0 tx=0).stop 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/2433762649 shutdown_connections 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d645f0 0x55c868d650e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d5df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/2433762649 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d63cd0 0x55c868d61dd0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.076+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/2433762649 >> 172.21.15.52:0/2433762649 conn(0x55c868c88e50 msgr2=0x55c868d649d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.077+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/2433762649 shutdown_connections 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.077+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/2433762649 wait complete. 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.077+0000 7fd9fcc4cb80 1 -- start start 2024-06-02T23:33:56.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.077+0000 7fd9fcc4cb80 1 Processor -- start 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.078+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d70a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.078+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d6d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.078+0000 7fd9fcc4cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 0x55c868d6d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.078+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d70a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.078+0000 7fd9ebfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d70a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:53336/0 (socket says 172.21.15.52:53336) 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 0x55c868d6d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d6d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 0x55c868d6d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50044/0 (socket says 172.21.15.52:50044) 2024-06-02T23:33:56.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9eb7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d6d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37832/0 (socket says 172.21.15.52:37832) 2024-06-02T23:33:56.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9eb7fe700 1 -- 172.21.15.52:0/546846633 learned_addr learned my addr 172.21.15.52:0/546846633 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:56.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1afeb0 con 0x55c868d645f0 2024-06-02T23:33:56.082 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1af7e0 con 0x55c868d5e460 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ec1ac9e0 con 0x55c868d63cd0 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9fcc4cb80 10 client.0 _ll_register_callbacks cb 0x55c868d53a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 -- 172.21.15.52:0/546846633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 msgr2=0x55c868d6d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d6d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 -- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 msgr2=0x55c868d70a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d70a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9f0966700 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c868d6dfa0 con 0x55c868d645f0 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9eb7fe700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d6d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:56.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.079+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9f0966700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 0x55c868d6d9e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009690 tx=0x7fd9e4004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6b9d9dbcc68c40 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/546846633 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e402b060 con 0x55c868d645f0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/546846633 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd9e4003a00 con 0x55c868d645f0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e00043c0 con 0x55c868d645f0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/546846633 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fd9e4003c40 con 0x55c868d645f0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9fcc4cb80 10 client.21183 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c868dec090 con 0x55c868d645f0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.080+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/546846633 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd9e4007500 con 0x55c868d645f0 2024-06-02T23:33:56.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/546846633 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fd9e4015430 con 0x55c868d645f0 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9fcc4cb80 10 client.21183 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9fcc4cb80 10 client.21183 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9fcc4cb80 20 client.21183 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868dee490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9fcc4cb80 4 client.21183 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x7fd9e00043c0 con 0x55c868d77dc0 2024-06-02T23:33:56.085 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.081+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868dee490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.082+0000 7fd9cb7fe700 10 client.21183 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:56.086 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.082+0000 7fd9ebfff700 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868dee490 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61aa4bcb6004761a server_cookie=73e3a3f17cb3b76d in_seq=0 out_seq=0 2024-06-02T23:33:56.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9cb7fe700 1 -- 172.21.15.52:0/546846633 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7fd9e00043c0 con 0x55c868d77dc0 2024-06-02T23:33:56.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9cb7fe700 10 client.21183 handle_command_reply: tid=0 2024-06-02T23:33:56.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 client.21183 shutdown 2024-06-02T23:33:56.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 msgr2=0x55c868d6d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 0x55c868d6d9e0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd9e4009690 tx=0x7fd9e4004af0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.361 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 shutdown_connections 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c868d77dc0 0x55c868dee490 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9cb7fe700 0 client.21183 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c868d645f0 0x55c868d6d9e0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c868d5e460 0x55c868d70a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 --2- 172.21.15.52:0/546846633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c868d63cd0 0x55c868d6d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 >> 172.21.15.52:0/546846633 conn(0x55c868c88e50 msgr2=0x55c868d68ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 shutdown_connections 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 1 -- 172.21.15.52:0/546846633 wait complete. 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.360+0000 7fd9fcc4cb80 20 client.21183 trim_cache size 0 max 16384 2024-06-02T23:33:56.362 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 70, 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "bec0c6a0": { 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 70, 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:33:56.363 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:33:56.364 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:33:56.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9d40a2d40 msgr2=0x7fd9d40a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9d40a2d40 0x7fd9d40a5200 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec107da0 tx=0x7fd9e0009260 comp rx=0 tx=0).stop 2024-06-02T23:33:56.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 msgr2=0x7fd9ec1a08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:56.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec1a08b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd9dc006010 tx=0x7fd9dc004270 comp rx=0 tx=0).stop 2024-06-02T23:33:56.365 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 shutdown_connections 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd9d40a2d40 0x7fd9d40a5200 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd9ec1085d0 0x7fd9ec19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9ec100610 0x7fd9ec1a08b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 --2- 172.21.15.52:0/1574896809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9ec100fd0 0x7fd9ec1a0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 >> 172.21.15.52:0/1574896809 conn(0x7fd9ec0775c0 msgr2=0x7fd9ec0feb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.361+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 shutdown_connections 2024-06-02T23:33:56.366 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:33:56.362+0000 7fd9f23c9700 1 -- 172.21.15.52:0/1574896809 wait complete. 2024-06-02T23:33:56.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.93 MB/sec execute 424 sec latency 402.447 ms 2024-06-02T23:33:57.031 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bec0c6a0', quiesce took 20.9 seconds. Will release after: 9.100000000000001 2024-06-02T23:33:57.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.91 MB/sec execute 425 sec latency 1402.497 ms 2024-06-02T23:33:57.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:57 smithi174 ceph-mon[35994]: pgmap v1093: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-06-02T23:33:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:57 smithi052 ceph-mon[32340]: pgmap v1093: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-06-02T23:33:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:57 smithi123 ceph-mon[35977]: pgmap v1093: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 984 KiB/s rd, 4.4 MiB/s wr, 275 op/s 2024-06-02T23:33:58.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.90 MB/sec execute 426 sec latency 2402.626 ms 2024-06-02T23:33:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 -- 172.21.15.52:0/1173330602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c1002d0 msgr2=0x7f027c1006f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 --2- 172.21.15.52:0/1173330602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c1002d0 0x7f027c1006f0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f026c009900 tx=0x7f026c02a410 comp rx=0 tx=0).stop 2024-06-02T23:33:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 -- 172.21.15.52:0/1173330602 shutdown_connections 2024-06-02T23:33:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 --2- 172.21.15.52:0/1173330602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f027c1015d0 0x7f027c101a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 --2- 172.21.15.52:0/1173330602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027c100c30 0x7f027c101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 --2- 172.21.15.52:0/1173330602 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c1002d0 0x7f027c1006f0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 -- 172.21.15.52:0/1173330602 >> 172.21.15.52:0/1173330602 conn(0x7f027c0fbe70 msgr2=0x7f027c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 -- 172.21.15.52:0/1173330602 shutdown_connections 2024-06-02T23:33:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.579+0000 7f02822bd640 1 -- 172.21.15.52:0/1173330602 wait complete. 2024-06-02T23:33:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 Processor -- start 2024-06-02T23:33:58.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 -- start start 2024-06-02T23:33:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f027c1002d0 0x7f027c199de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 0x7f027c19a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027c1015d0 0x7f027c1a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f027c10f770 con 0x7f027c100c30 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f027c10f5f0 con 0x7f027c1015d0 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f02822bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f027c10f8f0 con 0x7f027c1002d0 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 0x7f027c19a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f027bfff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f027c1002d0 0x7f027c199de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 0x7f027c19a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50064/0 (socket says 172.21.15.52:50064) 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f027b7fe640 1 -- 172.21.15.52:0/1594792468 learned_addr learned my addr 172.21.15.52:0/1594792468 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.580+0000 7f028085a640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027c1015d0 0x7f027c1a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f027b7fe640 1 -- 172.21.15.52:0/1594792468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f027c1002d0 msgr2=0x7f027c199de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f027c1002d0 0x7f027c199de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f027b7fe640 1 -- 172.21.15.52:0/1594792468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027c1015d0 msgr2=0x7f027c1a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027c1015d0 0x7f027c1a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f027b7fe640 1 -- 172.21.15.52:0/1594792468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c1a1a00 con 0x7f027c100c30 2024-06-02T23:33:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 0x7f027c19a320 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f027000d950 tx=0x7f027000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f02797fa640 1 -- 172.21.15.52:0/1594792468 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0270017070 con 0x7f027c100c30 2024-06-02T23:33:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f02797fa640 1 -- 172.21.15.52:0/1594792468 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0270009c30 con 0x7f027c100c30 2024-06-02T23:33:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c1a1cf0 con 0x7f027c100c30 2024-06-02T23:33:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f02797fa640 1 -- 172.21.15.52:0/1594792468 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027000f620 con 0x7f027c100c30 2024-06-02T23:33:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.581+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1a22b0 con 0x7f027c100c30 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f02797fa640 1 -- 172.21.15.52:0/1594792468 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f02700109a0 con 0x7f027c100c30 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 Processor -- start 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 -- start start 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe622430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe627500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe62a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55eefe5854f0 con 0x55eefe622a00 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55eefe523870 con 0x55eefe627b70 2024-06-02T23:33:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f0284fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eefe5814b0 con 0x55eefe622010 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.584+0000 7f027bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe627500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe627500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50066/0 (socket says 172.21.15.52:50066) 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 -- 172.21.15.52:0/2083692070 learned_addr learned my addr 172.21.15.52:0/2083692070 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f028085a640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe622430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027b7fe640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe62a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f02797fa640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02640a2cb0 0x7f02640a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 -- 172.21.15.52:0/2083692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 msgr2=0x55eefe622430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe622430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 -- 172.21.15.52:0/2083692070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 msgr2=0x55eefe62a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe62a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 -- 172.21.15.52:0/2083692070 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefe53f9b0 con 0x55eefe622a00 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f028085a640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe622430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027b7fe640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe62a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f028085a640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02640a2cb0 0x7f02640a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f02797fa640 1 -- 172.21.15.52:0/1594792468 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f02700ceb20 con 0x7f027c100c30 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f027bfff640 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe627500 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f026c009900 tx=0x7f026c031dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfd456e33bdaac49 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f02627fc640 1 -- 172.21.15.52:0/2083692070 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c035030 con 0x55eefe622a00 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f02627fc640 1 -- 172.21.15.52:0/2083692070 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c040070 con 0x55eefe622a00 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.585+0000 7f02627fc640 1 -- 172.21.15.52:0/2083692070 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c003c90 con 0x55eefe622a00 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 -- 172.21.15.52:0/2083692070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 msgr2=0x55eefe627500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f028085a640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02640a2cb0 0x7f02640a5170 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55eefe623260 tx=0x7f0274004b50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe627500 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f026c009900 tx=0x7f026c031dc0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 -- 172.21.15.52:0/2083692070 shutdown_connections 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe62a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe627500 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2083692070 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe622430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 -- 172.21.15.52:0/2083692070 >> 172.21.15.52:0/2083692070 conn(0x55eefe61d680 msgr2=0x55eefe61da80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 -- 172.21.15.52:0/2083692070 shutdown_connections 2024-06-02T23:33:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.586+0000 7f0284fd1740 1 -- 172.21.15.52:0/2083692070 wait complete. 2024-06-02T23:33:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 -- start start 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 Processor -- start 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe62c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe62c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe6337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55eefe62aac0 con 0x55eefe622a00 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55eefe50eae0 con 0x55eefe627b70 2024-06-02T23:33:58.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55eefe62ac40 con 0x55eefe622010 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f0284fd1740 10 client.0 _ll_register_callbacks cb 0x55eefe5479b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe62c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe62c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50068/0 (socket says 172.21.15.52:50068) 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f027b7fe640 1 -- 172.21.15.52:0/2902393971 learned_addr learned my addr 172.21.15.52:0/2902393971 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.587+0000 7f027bfff640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe62c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f028085a640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe6337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f027b7fe640 1 -- 172.21.15.52:0/2902393971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 msgr2=0x55eefe62c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f027b7fe640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe62c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f027b7fe640 1 -- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 msgr2=0x55eefe6337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f027b7fe640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe6337b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f027b7fe640 1 -- 172.21.15.52:0/2902393971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eefe299210 con 0x55eefe622a00 2024-06-02T23:33:58.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f028085a640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe6337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f027b7fe640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe62c770 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f02700a3af0 tx=0x7f027009b2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32661f04d38d3ea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f025b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f025b7fe640 1 -- 172.21.15.52:0/2902393971 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700d4020 con 0x55eefe622a00 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f025b7fe640 1 -- 172.21.15.52:0/2902393971 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f027009a8b0 con 0x55eefe622a00 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026c040070 con 0x55eefe622a00 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f025b7fe640 1 -- 172.21.15.52:0/2902393971 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0270020050 con 0x55eefe622a00 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f0284fd1740 10 client.21195 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eefe365620 con 0x55eefe622a00 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.588+0000 7f025b7fe640 1 -- 172.21.15.52:0/2902393971 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c040070 con 0x55eefe622a00 2024-06-02T23:33:58.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f025b7fe640 1 -- 172.21.15.52:0/2902393971 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f02700971b0 con 0x55eefe622a00 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f0284fd1740 10 client.21195 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f0284fd1740 10 client.21195 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f0284fd1740 20 client.21195 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6380b0 0x55eefe63a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f0284fd1740 4 client.21195 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eefe598f10 con 0x55eefe6380b0 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f027bfff640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6380b0 0x55eefe63a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f027bfff640 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6380b0 0x55eefe63a530 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d7097103fdd655 server_cookie=391af0bccafb0b82 in_seq=0 out_seq=0 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.589+0000 7f025b7fe640 10 client.21195 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f025b7fe640 1 -- 172.21.15.52:0/2902393971 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55eefe598f10 con 0x55eefe6380b0 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f025b7fe640 10 client.21195 handle_command_reply: tid=0 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 client.21195 shutdown 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 msgr2=0x55eefe62c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe62c770 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f02700a3af0 tx=0x7f027009b2f0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 shutdown_connections 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6380b0 0x55eefe63a530 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe627b70 0x55eefe6337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe62c770 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.590+0000 7f025b7fe640 0 client.21195 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.591+0000 7f0284fd1740 1 --2- 172.21.15.52:0/2902393971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe622010 0x55eefe62c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.591+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 >> 172.21.15.52:0/2902393971 conn(0x55eefe61d680 msgr2=0x55eefe61c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.591+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 shutdown_connections 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.591+0000 7f0284fd1740 1 -- 172.21.15.52:0/2902393971 wait complete. 2024-06-02T23:33:58.597 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.591+0000 7f0284fd1740 20 client.21195 trim_cache size 0 max 16384 2024-06-02T23:33:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 Processor -- start 2024-06-02T23:33:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 -- start start 2024-06-02T23:33:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 0x55eefe623c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.610 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622a00 0x55eefe633770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe624160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1aaf30 con 0x55eefe622010 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1aca70 con 0x55eefe622a00 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f0284fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1ab600 con 0x55eefe627b70 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 0x55eefe623c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 0x55eefe623c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50080/0 (socket says 172.21.15.52:50080) 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 -- 172.21.15.52:0/757340747 learned_addr learned my addr 172.21.15.52:0/757340747 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027b7fe640 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622a00 0x55eefe633770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f028085a640 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe624160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 -- 172.21.15.52:0/757340747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 msgr2=0x55eefe624160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe624160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 -- 172.21.15.52:0/757340747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622a00 msgr2=0x55eefe633770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622a00 0x55eefe633770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.609+0000 7f027bfff640 1 -- 172.21.15.52:0/757340747 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eefe6227a0 con 0x55eefe622010 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.610+0000 7f027bfff640 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 0x55eefe623c20 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x55eefe623470 tx=0x7f026c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da326f7b61a3dc81 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.610+0000 7f0261ffb640 1 -- 172.21.15.52:0/757340747 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c035040 con 0x55eefe622010 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.610+0000 7f0261ffb640 1 -- 172.21.15.52:0/757340747 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c040040 con 0x55eefe622010 2024-06-02T23:33:58.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.610+0000 7f0261ffb640 1 -- 172.21.15.52:0/757340747 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c044670 con 0x55eefe622010 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- 172.21.15.52:0/757340747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 msgr2=0x55eefe623c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 0x55eefe623c20 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x55eefe623470 tx=0x7f026c005fd0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- 172.21.15.52:0/757340747 shutdown_connections 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe624160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622a00 0x55eefe633770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- 172.21.15.52:0/757340747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622010 0x55eefe623c20 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- 172.21.15.52:0/757340747 >> 172.21.15.52:0/757340747 conn(0x55eefe61d680 msgr2=0x55eefe62c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:58.616 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- 172.21.15.52:0/757340747 shutdown_connections 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- 172.21.15.52:0/757340747 wait complete. 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- start start 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 Processor -- start 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622010 0x55eefe6308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe630e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe631e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1ac3a0 con 0x55eefe622a00 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1abcd0 con 0x55eefe622010 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1a8f10 con 0x55eefe627b70 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.611+0000 7f0284fd1740 10 client.0 _ll_register_callbacks cb 0x7f027000df40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe630e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe630e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50082/0 (socket says 172.21.15.52:50082) 2024-06-02T23:33:58.617 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 -- 172.21.15.52:0/1781311764 learned_addr learned my addr 172.21.15.52:0/1781311764 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f028085a640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe631e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027bfff640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622010 0x55eefe6308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 -- 172.21.15.52:0/1781311764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 msgr2=0x55eefe631e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe631e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 -- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622010 msgr2=0x55eefe6308c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622010 0x55eefe6308c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 -- 172.21.15.52:0/1781311764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eefe632400 con 0x55eefe622a00 2024-06-02T23:33:58.618 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027bfff640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622010 0x55eefe6308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f027b7fe640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe630e00 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f02700975d0 tx=0x7f02700a4020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d770b024f3fd46 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f025b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f025b7fe640 1 -- 172.21.15.52:0/1781311764 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02700d4070 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f025b7fe640 1 -- 172.21.15.52:0/1781311764 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f027009aab0 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026c040040 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.612+0000 7f025b7fe640 1 -- 172.21.15.52:0/1781311764 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f027009f760 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 10 client.21201 fetch_fsmap learned FSMap version 43 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eefe633710 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f025b7fe640 1 -- 172.21.15.52:0/1781311764 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f026c040040 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f025b7fe640 1 -- 172.21.15.52:0/1781311764 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f027009a320 con 0x55eefe622a00 2024-06-02T23:33:58.619 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 10 client.21201 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 10 client.21201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 20 client.21201 populate_metadata read hostname 'smithi052' 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6a3100 0x55eefe6a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 4 client.21201 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eefe592c20 con 0x55eefe6a3100 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.613+0000 7f027bfff640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6a3100 0x55eefe6a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.614+0000 7f027bfff640 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6a3100 0x55eefe6a54f0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1654847fb477626 server_cookie=48ddf3dea3f4699 in_seq=0 out_seq=0 2024-06-02T23:33:58.620 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.614+0000 7f025b7fe640 10 client.21201 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.658+0000 7f025b7fe640 1 -- 172.21.15.52:0/1781311764 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55eefe592c20 con 0x55eefe6a3100 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.658+0000 7f025b7fe640 10 client.21201 handle_command_reply: tid=0 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 client.21201 shutdown 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 msgr2=0x55eefe630e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe630e00 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f02700975d0 tx=0x7f02700a4020 comp rx=0 tx=0).stop 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 shutdown_connections 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55eefe6a3100 0x55eefe6a54f0 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55eefe627b70 0x55eefe631e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55eefe622a00 0x55eefe630e00 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.660 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 --2- 172.21.15.52:0/1781311764 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55eefe622010 0x55eefe6308c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f025b7fe640 0 client.21201 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 >> 172.21.15.52:0/1781311764 conn(0x55eefe61d680 msgr2=0x55eefe623c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 shutdown_connections 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 1 -- 172.21.15.52:0/1781311764 wait complete. 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.659+0000 7f0284fd1740 20 client.21201 trim_cache size 0 max 16384 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:33:58.661 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:33:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02640a2cb0 msgr2=0x7f02640a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02640a2cb0 0x7f02640a5170 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55eefe623260 tx=0x7f0274004b50 comp rx=0 tx=0).stop 2024-06-02T23:33:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 msgr2=0x7f027c19a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:33:58.662 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 0x7f027c19a320 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f027000d950 tx=0x7f027000de10 comp rx=0 tx=0).stop 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 shutdown_connections 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f02640a2cb0 0x7f02640a5170 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f027c1015d0 0x7f027c1a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f027c100c30 0x7f027c19a320 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 --2- 172.21.15.52:0/1594792468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f027c1002d0 0x7f027c199de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 >> 172.21.15.52:0/1594792468 conn(0x7f027c0fbe70 msgr2=0x7f027c0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 shutdown_connections 2024-06-02T23:33:58.663 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:33:58.660+0000 7f02822bd640 1 -- 172.21.15.52:0/1594792468 wait complete. 2024-06-02T23:33:59.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.88 MB/sec execute 427 sec latency 3402.702 ms 2024-06-02T23:33:59.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:33:59 smithi174 ceph-mon[35994]: pgmap v1094: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.3 MiB/s wr, 205 op/s 2024-06-02T23:33:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:33:59 smithi052 ceph-mon[32340]: pgmap v1094: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.3 MiB/s wr, 205 op/s 2024-06-02T23:33:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:33:59 smithi123 ceph-mon[35977]: pgmap v1094: 97 pgs: 97 active+clean; 550 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 742 KiB/s rd, 3.3 MiB/s wr, 205 op/s 2024-06-02T23:34:00.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.86 MB/sec execute 428 sec latency 4402.809 ms 2024-06-02T23:34:01.305 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:01 smithi174 ceph-mon[35994]: pgmap v1095: 97 pgs: 97 active+clean; 548 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2024-06-02T23:34:01.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.85 MB/sec execute 429 sec latency 5402.877 ms 2024-06-02T23:34:01.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:01 smithi052 ceph-mon[32340]: pgmap v1095: 97 pgs: 97 active+clean; 548 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2024-06-02T23:34:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:01 smithi123 ceph-mon[35977]: pgmap v1095: 97 pgs: 97 active+clean; 548 MiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 825 KiB/s rd, 3.2 MiB/s wr, 227 op/s 2024-06-02T23:34:02.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.83 MB/sec execute 430 sec latency 6402.940 ms 2024-06-02T23:34:03.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.82 MB/sec execute 431 sec latency 7402.994 ms 2024-06-02T23:34:03.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:03 smithi174 ceph-mon[35994]: pgmap v1096: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-06-02T23:34:03.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:03 smithi052 ceph-mon[32340]: pgmap v1096: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-06-02T23:34:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:03 smithi123 ceph-mon[35977]: pgmap v1096: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 752 KiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-06-02T23:34:04.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.80 MB/sec execute 432 sec latency 8403.119 ms 2024-06-02T23:34:05.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.79 MB/sec execute 433 sec latency 9403.218 ms 2024-06-02T23:34:05.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:05 smithi174 ceph-mon[35994]: pgmap v1097: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-06-02T23:34:05.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:05 smithi052 ceph-mon[32340]: pgmap v1097: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-06-02T23:34:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:05 smithi123 ceph-mon[35977]: pgmap v1097: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 489 KiB/s rd, 1.5 MiB/s wr, 154 op/s 2024-06-02T23:34:06.133 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:34:06.399 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:06.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.77 MB/sec execute 434 sec latency 10403.280 ms 2024-06-02T23:34:07.475 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.75 MB/sec execute 435 sec latency 11403.332 ms 2024-06-02T23:34:07.477 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:07 smithi052 ceph-mon[32340]: pgmap v1098: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-06-02T23:34:07.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:07 smithi174 ceph-mon[35994]: pgmap v1098: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-06-02T23:34:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:07 smithi123 ceph-mon[35977]: pgmap v1098: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 1.3 MiB/s wr, 141 op/s 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 -- 172.21.15.52:0/1834539588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d0107840 msgr2=0x7f33d010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 --2- 172.21.15.52:0/1834539588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d0107840 0x7f33d010f3c0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f33c800b0d0 tx=0x7f33c800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 -- 172.21.15.52:0/1834539588 shutdown_connections 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 --2- 172.21.15.52:0/1834539588 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d0107840 0x7f33d010f3c0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 --2- 172.21.15.52:0/1834539588 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d0102740 0x7f33d0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 --2- 172.21.15.52:0/1834539588 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d01030f0 0x7f33d0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 -- 172.21.15.52:0/1834539588 >> 172.21.15.52:0/1834539588 conn(0x7f33d00795f0 msgr2=0x7f33d0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 -- 172.21.15.52:0/1834539588 shutdown_connections 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.367+0000 7f33d6910700 1 -- 172.21.15.52:0/1834539588 wait complete. 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 Processor -- start 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 -- start start 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d0102740 0x7f33d0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 0x7f33d0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0107840 0x7f33d0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0112680 con 0x7f33d01030f0 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0112500 con 0x7f33d0102740 2024-06-02T23:34:08.372 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33d6910700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0112800 con 0x7f33d0107840 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.368+0000 7f33c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 0x7f33d0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 0x7f33d0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40456/0 (socket says 172.21.15.52:40456) 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 -- 172.21.15.52:0/2584426889 learned_addr learned my addr 172.21.15.52:0/2584426889 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33cffff700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d0102740 0x7f33d0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33d4ead700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0107840 0x7f33d0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 -- 172.21.15.52:0/2584426889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0107840 msgr2=0x7f33d0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0107840 0x7f33d0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 -- 172.21.15.52:0/2584426889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d0102740 msgr2=0x7f33d0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d0102740 0x7f33d0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 -- 172.21.15.52:0/2584426889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d01a6f60 con 0x7f33d01030f0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33c77fe700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 0x7f33d0074cd0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f33b800b530 tx=0x7f33b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8015020 con 0x7f33d01030f0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d01a7250 con 0x7f33d01030f0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f33b8011ce0 con 0x7f33d01030f0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33b8010e40 con 0x7f33d01030f0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.369+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d01a77e0 con 0x7f33d01030f0 2024-06-02T23:34:08.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.370+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33d0069180 con 0x7f33d01030f0 2024-06-02T23:34:08.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.371+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f33b8018020 con 0x7f33d01030f0 2024-06-02T23:34:08.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.372+0000 7f33cdffb700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33bc0a2c20 0x7f33bc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:08.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.372+0000 7f33cffff700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33bc0a2c20 0x7f33bc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:08.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.372+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f33b80cedf0 con 0x7f33d01030f0 2024-06-02T23:34:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.373+0000 7f33cffff700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33bc0a2c20 0x7f33bc0a50e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f33c000ac50 tx=0x7f33c000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:08.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.376+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f33b801f020 con 0x7f33d01030f0 2024-06-02T23:34:08.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.74 MB/sec execute 436 sec latency 12403.383 ms 2024-06-02T23:34:08.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.631+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f33d002dca0 con 0x7f33d01030f0 2024-06-02T23:34:08.633 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.633+0000 7f33cdffb700 1 -- 172.21.15.52:0/2584426889 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f33b8010380 con 0x7f33d01030f0 2024-06-02T23:34:08.633 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:34:08.634 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:34:08.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33bc0a2c20 msgr2=0x7f33bc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33bc0a2c20 0x7f33bc0a50e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f33c000ac50 tx=0x7f33c000a380 comp rx=0 tx=0).stop 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 msgr2=0x7f33d0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 0x7f33d0074cd0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f33b800b530 tx=0x7f33b800b840 comp rx=0 tx=0).stop 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 shutdown_connections 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f33bc0a2c20 0x7f33bc0a50e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f33d0107840 0x7f33d0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f33d0102740 0x7f33d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 --2- 172.21.15.52:0/2584426889 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f33d01030f0 0x7f33d0074cd0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 >> 172.21.15.52:0/2584426889 conn(0x7f33d00795f0 msgr2=0x7f33d01004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 shutdown_connections 2024-06-02T23:34:08.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:08.635+0000 7f33d6910700 1 -- 172.21.15.52:0/2584426889 wait complete. 2024-06-02T23:34:08.637 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:34:09.339 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id bec0c6a0 --release --await' 2024-06-02T23:34:09.339 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id bec0c6a0 --release --await 2024-06-02T23:34:09.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.72 MB/sec execute 437 sec latency 13403.423 ms 2024-06-02T23:34:09.578 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:34:09.607 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:09 smithi052 ceph-mon[32340]: pgmap v1099: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 369 KiB/s wr, 75 op/s 2024-06-02T23:34:09.608 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:09 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2584426889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:09.608 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:09.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:09 smithi174 ceph-mon[35994]: pgmap v1099: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 369 KiB/s wr, 75 op/s 2024-06-02T23:34:09.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:09 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2584426889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:09 smithi123 ceph-mon[35977]: pgmap v1099: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 369 KiB/s wr, 75 op/s 2024-06-02T23:34:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:09 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2584426889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:10.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.71 MB/sec execute 438 sec latency 14403.478 ms 2024-06-02T23:34:10.824 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:11.320 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:11 smithi052 ceph-mon[32340]: pgmap v1100: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 207 KiB/s wr, 69 op/s 2024-06-02T23:34:11.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67197 6.69 MB/sec execute 439 sec latency 15403.550 ms 2024-06-02T23:34:11.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:11 smithi174 ceph-mon[35994]: pgmap v1100: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 207 KiB/s wr, 69 op/s 2024-06-02T23:34:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 -- 172.21.15.52:0/947227624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 msgr2=0x7f6e50103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 --2- 172.21.15.52:0/947227624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e50103120 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009930 tx=0x7f6e3802a420 comp rx=0 tx=0).stop 2024-06-02T23:34:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 -- 172.21.15.52:0/947227624 shutdown_connections 2024-06-02T23:34:11.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 --2- 172.21.15.52:0/947227624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50103660 0x7f6e50109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 --2- 172.21.15.52:0/947227624 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e50103120 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 --2- 172.21.15.52:0/947227624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e500ffcd0 0x7f6e501000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 -- 172.21.15.52:0/947227624 >> 172.21.15.52:0/947227624 conn(0x7f6e500fb600 msgr2=0x7f6e500fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 -- 172.21.15.52:0/947227624 shutdown_connections 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.752+0000 7f6e5504d640 1 -- 172.21.15.52:0/947227624 wait complete. 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 Processor -- start 2024-06-02T23:34:11.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 -- start start 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e500ffcd0 0x7f6e50199e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e5019a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50103660 0x7f6e501a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010d540 con 0x7f6e50102ca0 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010d3c0 con 0x7f6e50103660 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e5504d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e5010d6c0 con 0x7f6e500ffcd0 2024-06-02T23:34:11.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e5019a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e5019a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40474/0 (socket says 172.21.15.52:40474) 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 -- 172.21.15.52:0/3726568183 learned_addr learned my addr 172.21.15.52:0/3726568183 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e500ffcd0 0x7f6e50199e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50103660 0x7f6e501a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 -- 172.21.15.52:0/3726568183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e500ffcd0 msgr2=0x7f6e50199e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e500ffcd0 0x7f6e50199e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 -- 172.21.15.52:0/3726568183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50103660 msgr2=0x7f6e501a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50103660 0x7f6e501a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.753+0000 7f6e47fff640 1 -- 172.21.15.52:0/3726568183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e501a1a80 con 0x7f6e50102ca0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.754+0000 7f6e47fff640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e5019a3a0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f6e380314e0 tx=0x7f6e380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.754+0000 7f6e4cd99640 1 -- 172.21.15.52:0/3726568183 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e3803f030 con 0x7f6e50102ca0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.754+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e501a1d10 con 0x7f6e50102ca0 2024-06-02T23:34:11.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.754+0000 7f6e4cd99640 1 -- 172.21.15.52:0/3726568183 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e38003cb0 con 0x7f6e50102ca0 2024-06-02T23:34:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.754+0000 7f6e4cd99640 1 -- 172.21.15.52:0/3726568183 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38035070 con 0x7f6e50102ca0 2024-06-02T23:34:11.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.754+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e501a21f0 con 0x7f6e50102ca0 2024-06-02T23:34:11.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:11 smithi123 ceph-mon[35977]: pgmap v1100: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 207 KiB/s wr, 69 op/s 2024-06-02T23:34:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.756+0000 7f6e4cd99640 1 -- 172.21.15.52:0/3726568183 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f6e38042030 con 0x7f6e50102ca0 2024-06-02T23:34:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.756+0000 7f6e57e3a740 1 Processor -- start 2024-06-02T23:34:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.756+0000 7f6e57e3a740 1 -- start start 2024-06-02T23:34:11.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb63a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb63f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb642000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5fb59d4f0 con 0x55d5fb63aa00 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d5fb53b870 con 0x55d5fb63a010 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5fb5994b0 con 0x55d5fb63fb70 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb63f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb63f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40490/0 (socket says 172.21.15.52:40490) 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 -- 172.21.15.52:0/623202659 learned_addr learned my addr 172.21.15.52:0/623202659 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb63a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e47fff640 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb642000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 -- 172.21.15.52:0/623202659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 msgr2=0x55d5fb642000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb642000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.765 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 -- 172.21.15.52:0/623202659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 msgr2=0x55d5fb63a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb63a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 -- 172.21.15.52:0/623202659 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5fb5579b0 con 0x55d5fb63aa00 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e47fff640 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb642000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4cd99640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6e3c0a2c30 0x7f6e3c0a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb63f500 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000bcb0 tx=0x7f6e400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b935d1c193e7719a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.757+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6e3c0a2c30 0x7f6e3c0a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e4cd99640 1 -- 172.21.15.52:0/3726568183 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6e380ef5e0 con 0x7f6e50102ca0 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e457fa640 1 -- 172.21.15.52:0/623202659 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40004020 con 0x55d5fb63aa00 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e457fa640 1 -- 172.21.15.52:0/623202659 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e400054d0 con 0x55d5fb63aa00 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e457fa640 1 -- 172.21.15.52:0/623202659 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e40004cb0 con 0x55d5fb63aa00 2024-06-02T23:34:11.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e57e3a740 1 -- 172.21.15.52:0/623202659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 msgr2=0x55d5fb63f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6e3c0a2c30 0x7f6e3c0a50f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55d5fb63b260 tx=0x7f6e48009740 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.758+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb63f500 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f6e4000bcb0 tx=0x7f6e400107a0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- 172.21.15.52:0/623202659 shutdown_connections 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb642000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb63f500 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/623202659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb63a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- 172.21.15.52:0/623202659 >> 172.21.15.52:0/623202659 conn(0x55d5fb635680 msgr2=0x55d5fb635a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- 172.21.15.52:0/623202659 shutdown_connections 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- 172.21.15.52:0/623202659 wait complete. 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- start start 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 Processor -- start 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb644220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63aa00 0x55d5fb644760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 0x55d5fb64b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d5fb642b20 con 0x55d5fb63fb70 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d5fb526ae0 con 0x55d5fb63a010 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d5fb642ca0 con 0x55d5fb63aa00 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e57e3a740 10 client.0 _ll_register_callbacks cb 0x55d5fb55f9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e4f59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 0x55d5fb64b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e4f59e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 0x55d5fb64b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40500/0 (socket says 172.21.15.52:40500) 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e4f59e640 1 -- 172.21.15.52:0/409935526 learned_addr learned my addr 172.21.15.52:0/409935526 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb644220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.759+0000 7f6e47fff640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63aa00 0x55d5fb644760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e4f59e640 1 -- 172.21.15.52:0/409935526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63aa00 msgr2=0x55d5fb644760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63aa00 0x55d5fb644760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e4f59e640 1 -- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 msgr2=0x55d5fb644220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb644220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e4f59e640 1 -- 172.21.15.52:0/409935526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5fb2b1210 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 0x55d5fb64b7a0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005c40 tx=0x7f6e48004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6ca8cb8fcc5294 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e367fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e367fc640 1 -- 172.21.15.52:0/409935526 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e4801ed40 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e367fc640 1 -- 172.21.15.52:0/409935526 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6e4801f420 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e400054d0 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e367fc640 1 -- 172.21.15.52:0/409935526 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6e4801f660 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e57e3a740 10 client.21219 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d5fb37d620 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e367fc640 1 -- 172.21.15.52:0/409935526 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e400054d0 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.760+0000 7f6e367fc640 1 -- 172.21.15.52:0/409935526 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6e4800f040 con 0x55d5fb63fb70 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e57e3a740 10 client.21219 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e57e3a740 10 client.21219 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e57e3a740 20 client.21219 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:11.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb650060 0x55d5fb652520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e57e3a740 4 client.21219 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5fb5b0f10 con 0x55d5fb650060 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb650060 0x55d5fb652520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb650060 0x55d5fb652520 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33dcc244f53a2f71 server_cookie=c702d3f1794934ef in_seq=0 out_seq=0 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.761+0000 7f6e367fc640 10 client.21219 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.762+0000 7f6e367fc640 1 -- 172.21.15.52:0/409935526 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d5fb5b0f10 con 0x55d5fb650060 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.762+0000 7f6e367fc640 10 client.21219 handle_command_reply: tid=0 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.762+0000 7f6e57e3a740 1 client.21219 shutdown 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 msgr2=0x55d5fb64b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 0x55d5fb64b7a0 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f6e48005c40 tx=0x7f6e48004a40 comp rx=0 tx=0).stop 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 shutdown_connections 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb650060 0x55d5fb652520 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63aa00 0x55d5fb644760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63fb70 0x55d5fb64b7a0 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e367fc640 0 client.21219 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/409935526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb644220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 >> 172.21.15.52:0/409935526 conn(0x55d5fb635680 msgr2=0x55d5fb6347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 shutdown_connections 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 1 -- 172.21.15.52:0/409935526 wait complete. 2024-06-02T23:34:11.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.763+0000 7f6e57e3a740 20 client.21219 trim_cache size 0 max 16384 2024-06-02T23:34:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 Processor -- start 2024-06-02T23:34:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 -- start start 2024-06-02T23:34:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb638590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb64b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb638ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501aafd0 con 0x55d5fb63aa00 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501acb10 con 0x55d5fb63a010 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.778+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501ab6a0 con 0x55d5fb63fb70 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb64b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb64b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40508/0 (socket says 172.21.15.52:40508) 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 -- 172.21.15.52:0/1961045929 learned_addr learned my addr 172.21.15.52:0/1961045929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb638590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb638ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 -- 172.21.15.52:0/1961045929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 msgr2=0x55d5fb638ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb638ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 -- 172.21.15.52:0/1961045929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 msgr2=0x55d5fb638590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb638590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 -- 172.21.15.52:0/1961045929 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5fb636350 con 0x55d5fb63aa00 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb638590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e47fff640 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb64b760 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f6e380b8840 tx=0x7f6e38032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac67785fc8846823 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.785 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e44ff9640 1 -- 172.21.15.52:0/1961045929 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38049040 con 0x55d5fb63aa00 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e44ff9640 1 -- 172.21.15.52:0/1961045929 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e380bc570 con 0x55d5fb63aa00 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.779+0000 7f6e44ff9640 1 -- 172.21.15.52:0/1961045929 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e380b84e0 con 0x55d5fb63aa00 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 -- 172.21.15.52:0/1961045929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 msgr2=0x55d5fb64b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb64b760 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f6e380b8840 tx=0x7f6e38032f70 comp rx=0 tx=0).stop 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 -- 172.21.15.52:0/1961045929 shutdown_connections 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb638ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb64b760 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/1961045929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb638590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.786 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 -- 172.21.15.52:0/1961045929 >> 172.21.15.52:0/1961045929 conn(0x55d5fb635680 msgr2=0x55d5fb63bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 -- 172.21.15.52:0/1961045929 shutdown_connections 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 -- 172.21.15.52:0/1961045929 wait complete. 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 -- start start 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.780+0000 7f6e57e3a740 1 Processor -- start 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb648a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb648f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb649f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.787 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501ac440 con 0x55d5fb63aa00 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501abd70 con 0x55d5fb63a010 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e501a8fb0 con 0x55d5fb63fb70 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 10 client.0 _ll_register_callbacks cb 0x7f6e480103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb648f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb648f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40516/0 (socket says 172.21.15.52:40516) 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 -- 172.21.15.52:0/3623284721 learned_addr learned my addr 172.21.15.52:0/3623284721 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb648a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e4f59e640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb649f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 -- 172.21.15.52:0/3623284721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 msgr2=0x55d5fb649f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb649f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.788 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 -- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 msgr2=0x55d5fb648a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb648a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 -- 172.21.15.52:0/3623284721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5fb64a530 con 0x55d5fb63aa00 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb648a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e47fff640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb648f80 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6e380ed160 tx=0x7f6e3802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=676fa69608869bb4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e367fc640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e367fc640 1 -- 172.21.15.52:0/3623284721 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38049050 con 0x55d5fb63aa00 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e367fc640 1 -- 172.21.15.52:0/3623284721 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f6e380c4650 con 0x55d5fb63aa00 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e380bc570 con 0x55d5fb63aa00 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e367fc640 1 -- 172.21.15.52:0/3623284721 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f6e380c0220 con 0x55d5fb63aa00 2024-06-02T23:34:11.789 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 10 client.21225 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.781+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d5fb64b840 con 0x55d5fb63aa00 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e367fc640 1 -- 172.21.15.52:0/3623284721 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6e380bc570 con 0x55d5fb63aa00 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e367fc640 1 -- 172.21.15.52:0/3623284721 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f6e3809ace0 con 0x55d5fb63aa00 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e57e3a740 10 client.21225 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e57e3a740 10 client.21225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e57e3a740 20 client.21225 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb6bb290 0x55d5fb6bd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e57e3a740 4 client.21225 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d5fb5aac20 con 0x55d5fb6bb290 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb6bb290 0x55d5fb6bd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.790 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e367fc640 10 client.21225 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:11.791 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.782+0000 7f6e4ed9d640 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb6bb290 0x55d5fb6bd6b0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fe1b2ac95ae4a7e server_cookie=e3647bada6594409 in_seq=0 out_seq=0 2024-06-02T23:34:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e367fc640 1 -- 172.21.15.52:0/3623284721 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55d5fb5aac20 con 0x55d5fb6bb290 2024-06-02T23:34:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e367fc640 10 client.21225 handle_command_reply: tid=0 2024-06-02T23:34:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 client.21225 shutdown 2024-06-02T23:34:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 msgr2=0x55d5fb648f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb648f80 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f6e380ed160 tx=0x7f6e3802a420 comp rx=0 tx=0).stop 2024-06-02T23:34:11.820 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 shutdown_connections 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d5fb6bb290 0x55d5fb6bd6b0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d5fb63fb70 0x55d5fb649f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d5fb63aa00 0x55d5fb648f80 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 --2- 172.21.15.52:0/3623284721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d5fb63a010 0x55d5fb648a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e367fc640 0 client.21225 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 >> 172.21.15.52:0/3623284721 conn(0x55d5fb635680 msgr2=0x55d5fb638590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 shutdown_connections 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 1 -- 172.21.15.52:0/3623284721 wait complete. 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.819+0000 7f6e57e3a740 20 client.21225 trim_cache size 0 max 16384 2024-06-02T23:34:11.821 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:34:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:34:11.822 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6e3c0a2c30 msgr2=0x7f6e3c0a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6e3c0a2c30 0x7f6e3c0a50f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x55d5fb63b260 tx=0x7f6e48009740 comp rx=0 tx=0).stop 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 msgr2=0x7f6e5019a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e5019a3a0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f6e380314e0 tx=0x7f6e380042f0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 shutdown_connections 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6e3c0a2c30 0x7f6e3c0a50f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e50103660 0x7f6e501a1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f6e50102ca0 0x7f6e5019a3a0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 --2- 172.21.15.52:0/3726568183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e500ffcd0 0x7f6e50199e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.823 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 >> 172.21.15.52:0/3726568183 conn(0x7f6e500fb600 msgr2=0x7f6e50100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 shutdown_connections 2024-06-02T23:34:11.824 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:11.820+0000 7f6e5504d640 1 -- 172.21.15.52:0/3726568183 wait complete. 2024-06-02T23:34:11.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.976+0000 7f4d4d186700 1 -- 172.21.15.52:0/4033867449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d48101380 msgr2=0x7f4d48101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.976+0000 7f4d4d186700 1 --2- 172.21.15.52:0/4033867449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d48101380 0x7f4d48101800 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f4d30009990 tx=0x7f4d30009ca0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 -- 172.21.15.52:0/4033867449 shutdown_connections 2024-06-02T23:34:11.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 --2- 172.21.15.52:0/4033867449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 0x7f4d4810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 --2- 172.21.15.52:0/4033867449 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d481073f0 0x7f4d481077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 --2- 172.21.15.52:0/4033867449 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d48101380 0x7f4d48101800 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 -- 172.21.15.52:0/4033867449 >> 172.21.15.52:0/4033867449 conn(0x7f4d48078300 msgr2=0x7f4d48078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 -- 172.21.15.52:0/4033867449 shutdown_connections 2024-06-02T23:34:11.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 -- 172.21.15.52:0/4033867449 wait complete. 2024-06-02T23:34:11.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.977+0000 7f4d4d186700 1 Processor -- start 2024-06-02T23:34:11.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 -- start start 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d48101380 0x7f4d48103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 0x7f4d481035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 0x7f4d48103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48111530 con 0x7f4d481073f0 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481113b0 con 0x7f4d48101d40 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4d186700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481116b0 con 0x7f4d48101380 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 0x7f4d48103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 0x7f4d48103ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40526/0 (socket says 172.21.15.52:40526) 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 -- 172.21.15.52:0/2634717191 learned_addr learned my addr 172.21.15.52:0/2634717191 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d48101380 0x7f4d48103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 0x7f4d481035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.981 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 -- 172.21.15.52:0/2634717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d48101380 msgr2=0x7f4d48103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d48101380 0x7f4d48103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 -- 172.21.15.52:0/2634717191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 msgr2=0x7f4d481035a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 0x7f4d481035a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d4759e700 1 -- 172.21.15.52:0/2634717191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d481a7090 con 0x7f4d481073f0 2024-06-02T23:34:11.982 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.978+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 0x7f4d481035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:11.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.979+0000 7f4d4759e700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 0x7f4d48103ae0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f4d40007ee0 tx=0x7f4d4000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.979+0000 7f4d44d99700 1 -- 172.21.15.52:0/2634717191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000ddd0 con 0x7f4d481073f0 2024-06-02T23:34:11.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.979+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d481a7380 con 0x7f4d481073f0 2024-06-02T23:34:11.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.979+0000 7f4d44d99700 1 -- 172.21.15.52:0/2634717191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d40004020 con 0x7f4d481073f0 2024-06-02T23:34:11.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.979+0000 7f4d44d99700 1 -- 172.21.15.52:0/2634717191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40015660 con 0x7f4d481073f0 2024-06-02T23:34:11.983 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.979+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481a78c0 con 0x7f4d481073f0 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.981+0000 7f4d44d99700 1 -- 172.21.15.52:0/2634717191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4d40015800 con 0x7f4d481073f0 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.982+0000 7f4d44d99700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d340a2c20 0x7f4d340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.982+0000 7f4d44d99700 1 -- 172.21.15.52:0/2634717191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4d400cebf0 con 0x7f4d481073f0 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.982+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d340a2c20 0x7f4d340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 Processor -- start 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 -- start start 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d340a2c20 0x7f4d340a50e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800bbb0 tx=0x7f4d3800b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d1e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d17e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d18810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e17c69910 con 0x561e17d1e560 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e17c11820 con 0x561e17d1dc40 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e17c658d0 con 0x561e17d183d0 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d17e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d17e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40542/0 (socket says 172.21.15.52:40542) 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d3e59c700 1 -- 172.21.15.52:0/1456477549 learned_addr learned my addr 172.21.15.52:0/1456477549 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d1e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.983+0000 7f4d4759e700 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d18810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3e59c700 1 -- 172.21.15.52:0/1456477549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 msgr2=0x561e17d18810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d18810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3e59c700 1 -- 172.21.15.52:0/1456477549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 msgr2=0x561e17d1e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.990 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d1e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3e59c700 1 -- 172.21.15.52:0/1456477549 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e17c08050 con 0x561e17d1e560 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d17e90 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006010 tx=0x7f4d30005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b5c6d1b948df059 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3d59a700 1 -- 172.21.15.52:0/1456477549 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30027070 con 0x561e17d1e560 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3d59a700 1 -- 172.21.15.52:0/1456477549 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d30009590 con 0x561e17d1e560 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.984+0000 7f4d3d59a700 1 -- 172.21.15.52:0/1456477549 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3000ba30 con 0x561e17d1e560 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 -- 172.21.15.52:0/1456477549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 msgr2=0x561e17d17e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d17e90 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f4d30006010 tx=0x7f4d30005c80 comp rx=0 tx=0).stop 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 -- 172.21.15.52:0/1456477549 shutdown_connections 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d18810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d1e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/1456477549 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d17e90 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 -- 172.21.15.52:0/1456477549 >> 172.21.15.52:0/1456477549 conn(0x561e17c42dc0 msgr2=0x561e17d13120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 -- 172.21.15.52:0/1456477549 shutdown_connections 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 -- 172.21.15.52:0/1456477549 wait complete. 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 -- start start 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.985+0000 7f4d57a09b80 1 Processor -- start 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d21560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d21aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561e17d24f70 con 0x561e17d1e560 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561e17c09cf0 con 0x561e17d1dc40 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561e17d250f0 con 0x561e17d183d0 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d57a09b80 10 client.0 _ll_register_callbacks cb 0x561e17c03a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d21aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d21aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40550/0 (socket says 172.21.15.52:40550) 2024-06-02T23:34:11.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 -- 172.21.15.52:0/4040553615 learned_addr learned my addr 172.21.15.52:0/4040553615 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d21560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 -- 172.21.15.52:0/4040553615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 msgr2=0x561e17d24260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d24260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 -- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 msgr2=0x561e17d21560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d21560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 -- 172.21.15.52:0/4040553615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e17bb89d0 con 0x561e17d1e560 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d21560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d4759e700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d21aa0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4d40004a20 tx=0x7f4d4009ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b101a9a5044d3b73 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.986+0000 7f4d2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/4040553615 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4009f7d0 con 0x561e17d1e560 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/4040553615 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4d4009b680 con 0x561e17d1e560 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d30009590 con 0x561e17d1e560 2024-06-02T23:34:11.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/4040553615 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4d400a4090 con 0x561e17d1e560 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 10 client.21237 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561e17d2f800 con 0x561e17d1e560 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/4040553615 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d30009590 con 0x561e17d1e560 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/4040553615 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4d40004bd0 con 0x561e17d1e560 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 10 client.21237 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 10 client.21237 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 20 client.21237 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d31d90 0x561e17d341b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 4 client.21237 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.987+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e17c76e10 con 0x561e17d31d90 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.988+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d31d90 0x561e17d341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.988+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d31d90 0x561e17d341b0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617ecf3c92b5893d server_cookie=3c0fc83a70069151 in_seq=0 out_seq=0 2024-06-02T23:34:11.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.988+0000 7f4d2e7fc700 10 client.21237 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:11.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/4040553615 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561e17c76e10 con 0x561e17d31d90 2024-06-02T23:34:11.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d2e7fc700 10 client.21237 handle_command_reply: tid=0 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 client.21237 shutdown 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 msgr2=0x561e17d21aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d21aa0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f4d40004a20 tx=0x7f4d4009ae70 comp rx=0 tx=0).stop 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 shutdown_connections 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d31d90 0x561e17d341b0 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d21aa0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d24260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/4040553615 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d21560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 >> 172.21.15.52:0/4040553615 conn(0x561e17c42dc0 msgr2=0x561e17d16e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d2e7fc700 0 client.21237 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.989+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 shutdown_connections 2024-06-02T23:34:11.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.990+0000 7f4d57a09b80 1 -- 172.21.15.52:0/4040553615 wait complete. 2024-06-02T23:34:11.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:11.990+0000 7f4d57a09b80 20 client.21237 trim_cache size 0 max 16384 2024-06-02T23:34:12.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 Processor -- start 2024-06-02T23:34:12.012 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 -- start start 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d1f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d39a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d39fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481aea60 con 0x561e17d1e560 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481b05a0 con 0x561e17d1dc40 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481af130 con 0x561e17d183d0 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d4759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d39fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.012+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d39a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d39a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35330/0 (socket says 172.21.15.52:35330) 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d46d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d1f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d3e59c700 1 -- 172.21.15.52:0/136448255 learned_addr learned my addr 172.21.15.52:0/136448255 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d39fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40566/0 (socket says 172.21.15.52:40566) 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 -- 172.21.15.52:0/136448255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 msgr2=0x561e17d1f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d1f110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 -- 172.21.15.52:0/136448255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 msgr2=0x561e17d39a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d39a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 -- 172.21.15.52:0/136448255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e17d13dd0 con 0x561e17d1e560 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d1f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d4759e700 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d39fa0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4d400a3b60 tx=0x7f4d40099930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f41f7412ab2fa3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d3cd99700 1 -- 172.21.15.52:0/136448255 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40004560 con 0x561e17d1e560 2024-06-02T23:34:12.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d3cd99700 1 -- 172.21.15.52:0/136448255 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d40004bd0 con 0x561e17d1e560 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.013+0000 7f4d3cd99700 1 -- 172.21.15.52:0/136448255 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d4009b680 con 0x561e17d1e560 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 -- 172.21.15.52:0/136448255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 msgr2=0x561e17d39fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d39fa0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7f4d400a3b60 tx=0x7f4d40099930 comp rx=0 tx=0).stop 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 -- 172.21.15.52:0/136448255 shutdown_connections 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1e560 0x561e17d39fa0 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d1f110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/136448255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1dc40 0x561e17d39a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 -- 172.21.15.52:0/136448255 >> 172.21.15.52:0/136448255 conn(0x561e17c42dc0 msgr2=0x561e17d1ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 -- 172.21.15.52:0/136448255 shutdown_connections 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 -- 172.21.15.52:0/136448255 wait complete. 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 -- start start 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.014+0000 7f4d57a09b80 1 Processor -- start 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d27380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 0x561e17d278c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1e560 0x561e17d27e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481afed0 con 0x561e17d1dc40 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481af800 con 0x561e17d1e560 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481aca20 con 0x561e17d183d0 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d57a09b80 10 client.0 _ll_register_callbacks cb 0x561e17c03a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 0x561e17d278c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 0x561e17d278c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40580/0 (socket says 172.21.15.52:40580) 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 -- 172.21.15.52:0/2370286255 learned_addr learned my addr 172.21.15.52:0/2370286255 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d27380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d4759e700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1e560 0x561e17d27e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 -- 172.21.15.52:0/2370286255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 msgr2=0x561e17d27380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d27380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 -- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1e560 msgr2=0x561e17d27e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1e560 0x561e17d27e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 -- 172.21.15.52:0/2370286255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e17d9cc80 con 0x561e17d1dc40 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d27380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d4759e700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1e560 0x561e17d27e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d3e59c700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 0x561e17d278c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4d300098e0 tx=0x7f4d300057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fd8b62419f14e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.015+0000 7f4d2e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/2370286255 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d30030060 con 0x561e17d1dc40 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/2370286255 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4d3000bd20 con 0x561e17d1dc40 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d40004bd0 con 0x561e17d1dc40 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/2370286255 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4d30027070 con 0x561e17d1dc40 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 10 client.21243 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561e17d9df60 con 0x561e17d1dc40 2024-06-02T23:34:12.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/2370286255 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4d40004bd0 con 0x561e17d1dc40 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/2370286255 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4d30022480 con 0x561e17d1dc40 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 10 client.21243 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 10 client.21243 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 20 client.21243 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d9e1e0 0x561e17da05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 4 client.21243 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bec0c6a0", "release": true, "await": true}] 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bec0c6a0", "release": true, "await": true}) -- 0x561e17c71390 con 0x561e17d9e1e0 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.016+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d9e1e0 0x561e17da05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:12.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.017+0000 7f4d46d9d700 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d9e1e0 0x561e17da05d0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a57f37dff27c0b server_cookie=9896d3c949fd46c in_seq=0 out_seq=0 2024-06-02T23:34:12.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.017+0000 7f4d2e7fc700 10 client.21243 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:12.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.050+0000 7f4d2e7fc700 1 -- 172.21.15.52:0/2370286255 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x561e17c71390 con 0x561e17d9e1e0 2024-06-02T23:34:12.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.050+0000 7f4d2e7fc700 10 client.21243 handle_command_reply: tid=0 2024-06-02T23:34:12.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.050+0000 7f4d57a09b80 1 client.21243 shutdown 2024-06-02T23:34:12.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 msgr2=0x561e17d278c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:12.051 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 0x561e17d278c0 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7f4d300098e0 tx=0x7f4d300057d0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 shutdown_connections 2024-06-02T23:34:12.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561e17d9e1e0 0x561e17da05d0 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.052 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561e17d1e560 0x561e17d27e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561e17d183d0 0x561e17d27380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 --2- 172.21.15.52:0/2370286255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561e17d1dc40 0x561e17d278c0 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d2e7fc700 0 client.21243 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.051+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 >> 172.21.15.52:0/2370286255 conn(0x561e17c42dc0 msgr2=0x561e17d12830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.052+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 shutdown_connections 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.052+0000 7f4d57a09b80 1 -- 172.21.15.52:0/2370286255 wait complete. 2024-06-02T23:34:12.053 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.052+0000 7f4d57a09b80 20 client.21243 trim_cache size 0 max 16384 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 72, 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "bec0c6a0": { 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 72, 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:34:12.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:12.055 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:12.056 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:34:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d340a2c20 msgr2=0x7f4d340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:12.056 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d340a2c20 0x7f4d340a50e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4d3800bbb0 tx=0x7f4d3800b590 comp rx=0 tx=0).stop 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 msgr2=0x7f4d48103ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 0x7f4d48103ae0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f4d40007ee0 tx=0x7f4d4000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 shutdown_connections 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4d340a2c20 0x7f4d340a50e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4d481073f0 0x7f4d48103ae0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4d48101380 0x7f4d48103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 --2- 172.21.15.52:0/2634717191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4d48101d40 0x7f4d481035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 >> 172.21.15.52:0/2634717191 conn(0x7f4d48078300 msgr2=0x7f4d48100660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 shutdown_connections 2024-06-02T23:34:12.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:12.053+0000 7f4d4d186700 1 -- 172.21.15.52:0/2634717191 wait complete. 2024-06-02T23:34:12.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67279 6.69 MB/sec execute 440 sec latency 15974.510 ms 2024-06-02T23:34:13.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67451 6.69 MB/sec execute 441 sec latency 74.373 ms 2024-06-02T23:34:13.546 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:13 smithi052 ceph-mon[32340]: pgmap v1101: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 57 KiB/s wr, 37 op/s 2024-06-02T23:34:13.588 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bec0c6a0', total seconds elapsed: 37.5 2024-06-02T23:34:13.588 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-06-02T23:34:13.588 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:34:13.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:13 smithi174 ceph-mon[35994]: pgmap v1101: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 57 KiB/s wr, 37 op/s 2024-06-02T23:34:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:13 smithi123 ceph-mon[35977]: pgmap v1101: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 57 KiB/s wr, 37 op/s 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 -- 172.21.15.52:0/4074413214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc481084f0 msgr2=0x7efc481088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/4074413214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc481084f0 0x7efc481088d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7efc3c009990 tx=0x7efc3c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 -- 172.21.15.52:0/4074413214 shutdown_connections 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/4074413214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102f10 0x7efc4810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/4074413214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc481084f0 0x7efc481088d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/4074413214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc48102550 0x7efc481029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 -- 172.21.15.52:0/4074413214 >> 172.21.15.52:0/4074413214 conn(0x7efc48079650 msgr2=0x7efc48079a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 -- 172.21.15.52:0/4074413214 shutdown_connections 2024-06-02T23:34:13.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.887+0000 7efc4f1e3700 1 -- 172.21.15.52:0/4074413214 wait complete. 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 Processor -- start 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 -- start start 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102550 0x7efc4819ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48102f10 0x7efc4819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc481084f0 0x7efc4819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efc48112600 con 0x7efc481084f0 2024-06-02T23:34:13.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc48112480 con 0x7efc48102f10 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4f1e3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc48112780 con 0x7efc48102550 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102550 0x7efc4819ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102550 0x7efc4819ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:51180/0 (socket says 172.21.15.52:51180) 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.888+0000 7efc4cf7f700 1 -- 172.21.15.52:0/543113648 learned_addr learned my addr 172.21.15.52:0/543113648 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48102f10 0x7efc4819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc4d780700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc481084f0 0x7efc4819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 -- 172.21.15.52:0/543113648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102550 msgr2=0x7efc4819ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102550 0x7efc4819ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 -- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc481084f0 msgr2=0x7efc4819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc481084f0 0x7efc4819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 -- 172.21.15.52:0/543113648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4819fd80 con 0x7efc48102f10 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc4d780700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc481084f0 0x7efc4819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:13.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.889+0000 7efc47fff700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48102f10 0x7efc4819b2b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efc3400e9f0 tx=0x7efc3400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:13.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.890+0000 7efc45ffb700 1 -- 172.21.15.52:0/543113648 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3400cbd0 con 0x7efc48102f10 2024-06-02T23:34:13.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.890+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc481a0070 con 0x7efc48102f10 2024-06-02T23:34:13.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.890+0000 7efc45ffb700 1 -- 172.21.15.52:0/543113648 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3400cd70 con 0x7efc48102f10 2024-06-02T23:34:13.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.890+0000 7efc45ffb700 1 -- 172.21.15.52:0/543113648 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc34018680 con 0x7efc48102f10 2024-06-02T23:34:13.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.890+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc481a78f0 con 0x7efc48102f10 2024-06-02T23:34:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.894+0000 7efc45ffb700 1 -- 172.21.15.52:0/543113648 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7efc34010430 con 0x7efc48102f10 2024-06-02T23:34:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.895+0000 7efc45ffb700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc380a2ca0 0x7efc380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc45ffb700 1 -- 172.21.15.52:0/543113648 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efc340cf190 con 0x7efc48102f10 2024-06-02T23:34:13.896 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 Processor -- start 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc380a2ca0 0x7efc380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 -- start start 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304be9f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9f6c0 0x56304be9a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304be9ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56304bde3810 con 0x56304be9eda0 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56304bd8b720 con 0x56304be9f6c0 2024-06-02T23:34:13.897 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.896+0000 7efc59a66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56304bddf7d0 con 0x56304be9a730 2024-06-02T23:34:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.897+0000 7efc4d780700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304be9ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.897+0000 7efc4d780700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304be9ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:51182/0 (socket says 172.21.15.52:51182) 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.897+0000 7efc4d780700 1 -- 172.21.15.52:0/2946088949 learned_addr learned my addr 172.21.15.52:0/2946088949 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.897+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304be9f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.897+0000 7efc47fff700 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9f6c0 0x56304be9a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.898+0000 7efc4d780700 1 -- 172.21.15.52:0/2946088949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9f6c0 msgr2=0x56304be9a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.898+0000 7efc4d780700 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9f6c0 0x56304be9a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.898+0000 7efc4d780700 1 -- 172.21.15.52:0/2946088949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 msgr2=0x56304be9f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.898+0000 7efc4d780700 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304be9f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.898+0000 7efc4d780700 1 -- 172.21.15.52:0/2946088949 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304bd81f50 con 0x56304be9a730 2024-06-02T23:34:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.898+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc380a2ca0 0x7efc380a5160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b2e0 tx=0x7efc3c009610 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.899+0000 7efc4d780700 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304be9ab70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc40009dd0 tx=0x7efc400077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69e1285552badc6b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.899+0000 7efc32ffd700 1 -- 172.21.15.52:0/2946088949 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40014610 con 0x56304be9a730 2024-06-02T23:34:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.899+0000 7efc32ffd700 1 -- 172.21.15.52:0/2946088949 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc40019070 con 0x56304be9a730 2024-06-02T23:34:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.899+0000 7efc32ffd700 1 -- 172.21.15.52:0/2946088949 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc40013900 con 0x56304be9a730 2024-06-02T23:34:13.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.900+0000 7efc59a66b80 1 -- 172.21.15.52:0/2946088949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 msgr2=0x56304be9ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.900+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304be9ab70 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7efc40009dd0 tx=0x7efc400077e0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.900+0000 7efc59a66b80 1 -- 172.21.15.52:0/2946088949 shutdown_connections 2024-06-02T23:34:13.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.900+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304be9ab70 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.900+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304be9f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.900+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2946088949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9f6c0 0x56304be9a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.901+0000 7efc59a66b80 1 -- 172.21.15.52:0/2946088949 >> 172.21.15.52:0/2946088949 conn(0x56304bdbccc0 msgr2=0x56304be8c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.901+0000 7efc59a66b80 1 -- 172.21.15.52:0/2946088949 shutdown_connections 2024-06-02T23:34:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.901+0000 7efc59a66b80 1 -- 172.21.15.52:0/2946088949 wait complete. 2024-06-02T23:34:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.902+0000 7efc59a66b80 1 -- start start 2024-06-02T23:34:13.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.902+0000 7efc59a66b80 1 Processor -- start 2024-06-02T23:34:13.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304bea3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.904 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9f6c0 0x56304bea3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43802/0 (socket says 172.21.15.52:43802) 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc47fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304bea3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.903+0000 7efc47fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304bea3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50366/0 (socket says 172.21.15.52:50366) 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc4d780700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9f6c0 0x56304bea3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc4cf7f700 1 -- 172.21.15.52:0/1436261738 learned_addr learned my addr 172.21.15.52:0/1436261738 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x56304bea7200 con 0x56304be9eda0 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56304bd83bf0 con 0x56304be9a730 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56304bea7380 con 0x56304be9f6c0 2024-06-02T23:34:13.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc59a66b80 10 client.0 _ll_register_callbacks cb 0x56304bd7d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:13.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc47fff700 1 -- 172.21.15.52:0/1436261738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9f6c0 msgr2=0x56304bea3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc47fff700 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9f6c0 0x56304bea3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc47fff700 1 -- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 msgr2=0x56304bea6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc47fff700 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.904+0000 7efc47fff700 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304bd328d0 con 0x56304be9eda0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc47fff700 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304bea3780 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7efc3400c5b0 tx=0x7efc3409bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51d42d933ad1448 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc27fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc27fff700 1 -- 172.21.15.52:0/1436261738 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3409c0c0 con 0x56304be9eda0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc27fff700 1 -- 172.21.15.52:0/1436261738 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efc34099cc0 con 0x56304be9eda0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc40019070 con 0x56304be9eda0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc27fff700 1 -- 172.21.15.52:0/1436261738 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efc3409b9c0 con 0x56304be9eda0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc59a66b80 10 client.21255 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.905+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56304beb1960 con 0x56304be9eda0 2024-06-02T23:34:13.907 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc27fff700 1 -- 172.21.15.52:0/1436261738 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3409fd70 con 0x56304be9eda0 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc27fff700 1 -- 172.21.15.52:0/1436261738 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7efc340860b0 con 0x56304be9eda0 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc59a66b80 10 client.21255 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc59a66b80 10 client.21255 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc59a66b80 20 client.21255 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304beb3fa0 0x56304beb6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc59a66b80 4 client.21255 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efc40019070 con 0x56304beb3fa0 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.906+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304beb3fa0 0x56304beb6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.907+0000 7efc27fff700 10 client.21255 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:13.908 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.907+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304beb3fa0 0x56304beb6390 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=886ef110b56f77c2 server_cookie=76a1ca4715488f74 in_seq=0 out_seq=0 2024-06-02T23:34:13.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc27fff700 1 -- 172.21.15.52:0/1436261738 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7efc40019070 con 0x56304beb3fa0 2024-06-02T23:34:13.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc27fff700 10 client.21255 handle_command_reply: tid=0 2024-06-02T23:34:13.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc59a66b80 1 client.21255 shutdown 2024-06-02T23:34:13.909 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 msgr2=0x56304bea3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304bea3780 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7efc3400c5b0 tx=0x7efc3409bf00 comp rx=0 tx=0).stop 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 shutdown_connections 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.908+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304beb3fa0 0x56304beb6390 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9f6c0 0x56304bea3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1436261738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9eda0 0x56304bea3780 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 >> 172.21.15.52:0/1436261738 conn(0x56304bdbccc0 msgr2=0x56304be909d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc27fff700 0 client.21255 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 shutdown_connections 2024-06-02T23:34:13.910 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 1 -- 172.21.15.52:0/1436261738 wait complete. 2024-06-02T23:34:13.911 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.909+0000 7efc59a66b80 20 client.21255 trim_cache size 0 max 16384 2024-06-02T23:34:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 Processor -- start 2024-06-02T23:34:13.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 -- start start 2024-06-02T23:34:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9eda0 0x56304be9c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efc481aa870 con 0x56304be9f6c0 2024-06-02T23:34:13.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc481ac3b0 con 0x56304be9a730 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc59a66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc481aaf40 con 0x56304be9eda0 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc4d780700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea1340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43814/0 (socket says 172.21.15.52:43814) 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc4cf7f700 1 -- 172.21.15.52:0/1641228263 learned_addr learned my addr 172.21.15.52:0/1641228263 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.948+0000 7efc47fff700 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9eda0 0x56304be9c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc4cf7f700 1 -- 172.21.15.52:0/1641228263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9eda0 msgr2=0x56304be9c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9eda0 0x56304be9c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc4cf7f700 1 -- 172.21.15.52:0/1641228263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 msgr2=0x56304be9cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc4cf7f700 1 -- 172.21.15.52:0/1641228263 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56304be8e4a0 con 0x56304be9a730 2024-06-02T23:34:13.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea1340 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efc3c006010 tx=0x7efc3c029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=444ef2bfa50b7685 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc327fc700 1 -- 172.21.15.52:0/1641228263 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003880 con 0x56304be9a730 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc327fc700 1 -- 172.21.15.52:0/1641228263 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3c022040 con 0x56304be9a730 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.949+0000 7efc327fc700 1 -- 172.21.15.52:0/1641228263 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c035ca0 con 0x56304be9a730 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 -- 172.21.15.52:0/1641228263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 msgr2=0x56304bea1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea1340 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7efc3c006010 tx=0x7efc3c029000 comp rx=0 tx=0).stop 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 -- 172.21.15.52:0/1641228263 shutdown_connections 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9a730 0x56304bea1340 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 --2- 172.21.15.52:0/1641228263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9eda0 0x56304be9c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 -- 172.21.15.52:0/1641228263 >> 172.21.15.52:0/1641228263 conn(0x56304bdbccc0 msgr2=0x56304bea0c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 -- 172.21.15.52:0/1641228263 shutdown_connections 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.950+0000 7efc59a66b80 1 -- 172.21.15.52:0/1641228263 wait complete. 2024-06-02T23:34:13.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.951+0000 7efc59a66b80 1 -- start start 2024-06-02T23:34:13.953 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.951+0000 7efc59a66b80 1 Processor -- start 2024-06-02T23:34:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304beac670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 0x56304beacbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.954 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7efc481abce0 con 0x56304be9f6c0 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efc481ab610 con 0x56304be9eda0 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efc4810d160 con 0x56304be9a730 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc59a66b80 10 client.0 _ll_register_callbacks cb 0x56304bd7d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 0x56304beacbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304beac670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 0x56304beacbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43822/0 (socket says 172.21.15.52:43822) 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc4cf7f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304beac670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:51212/0 (socket says 172.21.15.52:51212) 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 -- 172.21.15.52:0/2561719854 learned_addr learned my addr 172.21.15.52:0/2561719854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc4d780700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 -- 172.21.15.52:0/2561719854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 msgr2=0x56304beac670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.955 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304beac670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 -- 172.21.15.52:0/2561719854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 msgr2=0x56304be9c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.952+0000 7efc47fff700 1 -- 172.21.15.52:0/2561719854 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56304bebc5b0 con 0x56304be9eda0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.953+0000 7efc4d780700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.953+0000 7efc47fff700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 0x56304beacbb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efc34099c70 tx=0x7efc3400eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bdf6e44a404f641 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.953+0000 7efc27fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.953+0000 7efc27fff700 1 -- 172.21.15.52:0/2561719854 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc340ba370 con 0x56304be9eda0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.953+0000 7efc27fff700 1 -- 172.21.15.52:0/2561719854 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7efc3409ae10 con 0x56304be9eda0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.953+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc3c022040 con 0x56304be9eda0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc27fff700 1 -- 172.21.15.52:0/2561719854 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7efc3409c0c0 con 0x56304be9eda0 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 10 client.30527 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:13.956 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x56304bf1ef40 con 0x56304be9eda0 2024-06-02T23:34:13.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc27fff700 1 -- 172.21.15.52:0/2561719854 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7efc3c022040 con 0x56304be9eda0 2024-06-02T23:34:13.958 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc27fff700 1 -- 172.21.15.52:0/2561719854 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7efc3409a880 con 0x56304be9eda0 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 10 client.30527 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 10 client.30527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 20 client.30527 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304bf1f590 0x56304bf219b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 4 client.30527 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.954+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56304bdf0d10 con 0x56304bf1f590 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.955+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304bf1f590 0x56304bf219b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.955+0000 7efc27fff700 10 client.30527 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc4cf7f700 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304bf1f590 0x56304bf219b0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3df583320b1b86f4 server_cookie=473bed84c1dec577 in_seq=0 out_seq=0 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc27fff700 1 -- 172.21.15.52:0/2561719854 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56304bdf0d10 con 0x56304bf1f590 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc27fff700 10 client.30527 handle_command_reply: tid=0 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 client.30527 shutdown 2024-06-02T23:34:13.959 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 msgr2=0x56304beacbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 0x56304beacbb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7efc34099c70 tx=0x7efc3400eee0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 shutdown_connections 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x56304bf1f590 0x56304bf219b0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x56304be9f6c0 0x56304be9c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56304be9a730 0x56304beac670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:34:13.960 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 --2- 172.21.15.52:0/2561719854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56304be9eda0 0x56304beacbb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 >> 172.21.15.52:0/2561719854 conn(0x56304bdbccc0 msgr2=0x56304bea2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.956+0000 7efc27fff700 0 client.30527 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.957+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 shutdown_connections 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.957+0000 7efc59a66b80 1 -- 172.21.15.52:0/2561719854 wait complete. 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.957+0000 7efc59a66b80 20 client.30527 trim_cache size 0 max 16384 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc380a2ca0 msgr2=0x7efc380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc380a2ca0 0x7efc380a5160 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00b2e0 tx=0x7efc3c009610 comp rx=0 tx=0).stop 2024-06-02T23:34:13.961 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48102f10 msgr2=0x7efc4819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48102f10 0x7efc4819b2b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7efc3400e9f0 tx=0x7efc3400ed00 comp rx=0 tx=0).stop 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 shutdown_connections 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7efc380a2ca0 0x7efc380a5160 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7efc481084f0 0x7efc4819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efc48102550 0x7efc4819ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 --2- 172.21.15.52:0/543113648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efc48102f10 0x7efc4819b2b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 >> 172.21.15.52:0/543113648 conn(0x7efc48079650 msgr2=0x7efc481003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 shutdown_connections 2024-06-02T23:34:13.962 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:13.958+0000 7efc4f1e3700 1 -- 172.21.15.52:0/543113648 wait complete. 2024-06-02T23:34:14.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67589 6.69 MB/sec execute 442 sec latency 82.749 ms 2024-06-02T23:34:14.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:14 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:34:14.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:14 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:34:14.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:14 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:34:14.916 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:34:14.916 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:34:14.973 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:15.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:15 smithi052 ceph-mon[32340]: pgmap v1102: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-06-02T23:34:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:34:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:34:15.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:34:15.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67742 6.69 MB/sec execute 443 sec latency 61.755 ms 2024-06-02T23:34:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:15 smithi174 ceph-mon[35994]: pgmap v1102: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-06-02T23:34:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:34:15.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:34:15.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:34:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:15 smithi123 ceph-mon[35977]: pgmap v1102: 97 pgs: 97 active+clean; 545 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-06-02T23:34:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:34:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:34:15.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:34:16.291 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:16.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 67949 6.72 MB/sec execute 444 sec latency 75.223 ms 2024-06-02T23:34:17.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:17 smithi052 ceph-mon[32340]: pgmap v1103: 97 pgs: 97 active+clean; 549 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 974 KiB/s wr, 76 op/s 2024-06-02T23:34:17.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68145 6.72 MB/sec execute 445 sec latency 65.506 ms 2024-06-02T23:34:17.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2276832567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 msgr2=0x7ff0c010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:17.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2276832567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c010e240 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8009f90 tx=0x7ff0b800b330 comp rx=0 tx=0).stop 2024-06-02T23:34:17.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2276832567 shutdown_connections 2024-06-02T23:34:17.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2276832567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c010e240 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2276832567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0107400 0x7ff0c01077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2276832567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0101390 0x7ff0c0101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.090+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2276832567 >> 172.21.15.52:0/2276832567 conn(0x7ff0c0078340 msgr2=0x7ff0c0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.091+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2276832567 shutdown_connections 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.091+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2276832567 wait complete. 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 Processor -- start 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 -- start start 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0101390 0x7ff0c019ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c019b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0107400 0x7ff0c019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c01126b0 con 0x7ff0c0101390 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0112530 con 0x7ff0c0101d50 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c6aa2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c0112830 con 0x7ff0c0107400 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c483e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0101390 0x7ff0c019ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:17.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c483e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0101390 0x7ff0c019ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50402/0 (socket says 172.21.15.52:50402) 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c483e700 1 -- 172.21.15.52:0/2373750284 learned_addr learned my addr 172.21.15.52:0/2373750284 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0bffff700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c019b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c503f700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0107400 0x7ff0c019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0bffff700 1 -- 172.21.15.52:0/2373750284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0107400 msgr2=0x7ff0c019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0bffff700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0107400 0x7ff0c019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0bffff700 1 -- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0101390 msgr2=0x7ff0c019ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0bffff700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0101390 0x7ff0c019ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0bffff700 1 -- 172.21.15.52:0/2373750284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c019fd20 con 0x7ff0c0101d50 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.092+0000 7ff0c503f700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0107400 0x7ff0c019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.093+0000 7ff0bffff700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c019b250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00e9f0 tx=0x7ff0ac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.093+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac00cb70 con 0x7ff0c0101d50 2024-06-02T23:34:17.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.093+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff0ac0044d0 con 0x7ff0c0101d50 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.093+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac003e10 con 0x7ff0c0101d50 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.093+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c01a0010 con 0x7ff0c0101d50 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.094+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c01a7890 con 0x7ff0c0101d50 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.095+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c0069180 con 0x7ff0c0101d50 2024-06-02T23:34:17.552 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.096+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff0ac004000 con 0x7ff0c0101d50 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.097+0000 7ff0bdffb700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0b00a2ca0 0x7ff0b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.098+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff0ac0cf260 con 0x7ff0c0101d50 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.098+0000 7ff0c483e700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0b00a2ca0 0x7ff0b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.098+0000 7ff0c483e700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0b00a2ca0 0x7ff0b00a5160 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009750 tx=0x7ff0b4009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.102+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff0ac0042f0 con 0x7ff0c0101d50 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.514+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0c019c070 con 0x7ff0c0101d50 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.515+0000 7ff0bdffb700 1 -- 172.21.15.52:0/2373750284 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7ff0ac09c190 con 0x7ff0c0101d50 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0b00a2ca0 msgr2=0x7ff0b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0b00a2ca0 0x7ff0b00a5160 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4009750 tx=0x7ff0b4009310 comp rx=0 tx=0).stop 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 msgr2=0x7ff0c019b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c019b250 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac00e9f0 tx=0x7ff0ac00ed00 comp rx=0 tx=0).stop 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 shutdown_connections 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0b00a2ca0 0x7ff0b00a5160 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.553 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff0c0107400 0x7ff0c019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0c0101390 0x7ff0c019ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 --2- 172.21.15.52:0/2373750284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0c0101d50 0x7ff0c019b250 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:17.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 >> 172.21.15.52:0/2373750284 conn(0x7ff0c0078340 msgr2=0x7ff0c010ed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:17.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 shutdown_connections 2024-06-02T23:34:17.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:17.518+0000 7ff0c6aa2700 1 -- 172.21.15.52:0/2373750284 wait complete. 2024-06-02T23:34:17.554 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:34:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:17 smithi174 ceph-mon[35994]: pgmap v1103: 97 pgs: 97 active+clean; 549 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 974 KiB/s wr, 76 op/s 2024-06-02T23:34:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:17 smithi123 ceph-mon[35977]: pgmap v1103: 97 pgs: 97 active+clean; 549 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 974 KiB/s wr, 76 op/s 2024-06-02T23:34:18.307 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-06-02T23:34:18.307 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-06-02T23:34:18.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68318 6.73 MB/sec execute 446 sec latency 68.730 ms 2024-06-02T23:34:18.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:18 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2373750284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:18.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:18 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2373750284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:18.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:18 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2373750284' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:19.197 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:19.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68444 6.72 MB/sec execute 447 sec latency 101.485 ms 2024-06-02T23:34:19.481 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:19 smithi052 ceph-mon[32340]: pgmap v1104: 97 pgs: 97 active+clean; 548 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-06-02T23:34:19.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:19 smithi174 ceph-mon[35994]: pgmap v1104: 97 pgs: 97 active+clean; 548 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-06-02T23:34:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:19 smithi123 ceph-mon[35977]: pgmap v1104: 97 pgs: 97 active+clean; 548 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 -- 172.21.15.52:0/3291504292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4102380 msgr2=0x7f7fc4105ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 --2- 172.21.15.52:0/3291504292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4102380 0x7f7fc4105ff0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f7fb40099c0 tx=0x7f7fb4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 -- 172.21.15.52:0/3291504292 shutdown_connections 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 --2- 172.21.15.52:0/3291504292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fc4106720 0x7f7fc410e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 --2- 172.21.15.52:0/3291504292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fc41019d0 0x7f7fc4101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 --2- 172.21.15.52:0/3291504292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4102380 0x7f7fc4105ff0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 -- 172.21.15.52:0/3291504292 >> 172.21.15.52:0/3291504292 conn(0x7f7fc4078340 msgr2=0x7f7fc4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 -- 172.21.15.52:0/3291504292 shutdown_connections 2024-06-02T23:34:19.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.758+0000 7f7fcb282700 1 -- 172.21.15.52:0/3291504292 wait complete. 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.759+0000 7f7fcb282700 1 Processor -- start 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.759+0000 7f7fcb282700 1 -- start start 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fcb282700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fc41019d0 0x7f7fc419c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fcb282700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fc4102380 0x7f7fc419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fcb282700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 0x7f7fc4196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fcb282700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc4110540 con 0x7f7fc4106720 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fcb282700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41103c0 con 0x7f7fc4102380 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fcb282700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc41106c0 con 0x7f7fc41019d0 2024-06-02T23:34:19.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 0x7f7fc4196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 0x7f7fc4196760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50414/0 (socket says 172.21.15.52:50414) 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 -- 172.21.15.52:0/1434482786 learned_addr learned my addr 172.21.15.52:0/1434482786 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc881d700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fc4102380 0x7f7fc419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 -- 172.21.15.52:0/1434482786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fc41019d0 msgr2=0x7f7fc419c690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fc41019d0 0x7f7fc419c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 -- 172.21.15.52:0/1434482786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fc4102380 msgr2=0x7f7fc419cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fc4102380 0x7f7fc419cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:19.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.760+0000 7f7fc981f700 1 -- 172.21.15.52:0/1434482786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc4197040 con 0x7f7fc4106720 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.761+0000 7f7fc981f700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 0x7f7fc4196760 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc007d70 tx=0x7f7fbc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.761+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc016020 con 0x7f7fc4106720 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.761+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc4197330 con 0x7f7fc4106720 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.761+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7fbc00ad40 con 0x7f7fc4106720 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.761+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc0115e0 con 0x7f7fc4106720 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.761+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc41a3610 con 0x7f7fc4106720 2024-06-02T23:34:19.763 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.763+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc406a410 con 0x7f7fc4106720 2024-06-02T23:34:19.765 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.765+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7fbc019020 con 0x7f7fc4106720 2024-06-02T23:34:19.766 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.766+0000 7f7fba7fc700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fb00a70b0 0x7f7fb00a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.766+0000 7f7fc901e700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fb00a70b0 0x7f7fb00a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.766+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7fbc0ceff0 con 0x7f7fc4106720 2024-06-02T23:34:19.767 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.767+0000 7f7fc901e700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fb00a70b0 0x7f7fb00a9570 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000da10 tx=0x7f7fc0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:19.773 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:19.772+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f7fbc020020 con 0x7f7fc4106720 2024-06-02T23:34:20.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.207+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7fc4197c90 con 0x7f7fc4106720 2024-06-02T23:34:20.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.208+0000 7f7fba7fc700 1 -- 172.21.15.52:0/1434482786 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f7fbc09bf20 con 0x7f7fc4106720 2024-06-02T23:34:20.209 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:34:20.209 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fb00a70b0 msgr2=0x7f7fb00a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fb00a70b0 0x7f7fb00a9570 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7fc000da10 tx=0x7f7fc0009260 comp rx=0 tx=0).stop 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 msgr2=0x7f7fc4196760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 0x7f7fc4196760 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc007d70 tx=0x7f7fbc00a340 comp rx=0 tx=0).stop 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 shutdown_connections 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7fb00a70b0 0x7f7fb00a9570 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:20.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7fc4106720 0x7f7fc4196760 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7fc41019d0 0x7f7fc419c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 --2- 172.21.15.52:0/1434482786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fc4102380 0x7f7fc419cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 >> 172.21.15.52:0/1434482786 conn(0x7f7fc4078340 msgr2=0x7f7fc40ff990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.211+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 shutdown_connections 2024-06-02T23:34:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:20.212+0000 7f7fcb282700 1 -- 172.21.15.52:0/1434482786 wait complete. 2024-06-02T23:34:20.213 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:34:20.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68618 6.73 MB/sec execute 448 sec latency 68.790 ms 2024-06-02T23:34:20.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:20 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1434482786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:20.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:20 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1434482786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:20.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:20 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1434482786' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:21.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:34:21.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68788 6.74 MB/sec execute 449 sec latency 88.015 ms 2024-06-02T23:34:21.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:21 smithi174 ceph-mon[35994]: pgmap v1105: 97 pgs: 97 active+clean; 551 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-06-02T23:34:21.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:21 smithi052 ceph-mon[32340]: pgmap v1105: 97 pgs: 97 active+clean; 551 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-06-02T23:34:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:21 smithi123 ceph-mon[35977]: pgmap v1105: 97 pgs: 97 active+clean; 551 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 2.1 MiB/s wr, 139 op/s 2024-06-02T23:34:22.345 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:22.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 -- 172.21.15.52:0/1117278477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2101066c0 msgr2=0x7fb21010e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 --2- 172.21.15.52:0/1117278477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2101066c0 0x7fb21010e290 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7fb20800b0d0 tx=0x7fb20800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.428 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 -- 172.21.15.52:0/1117278477 shutdown_connections 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 --2- 172.21.15.52:0/1117278477 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb2101066c0 0x7fb21010e290 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 --2- 172.21.15.52:0/1117278477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 0x7fb2101019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 --2- 172.21.15.52:0/1117278477 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb210101fa0 0x7fb210105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 -- 172.21.15.52:0/1117278477 >> 172.21.15.52:0/1117278477 conn(0x7fb210078330 msgr2=0x7fb210078740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 -- 172.21.15.52:0/1117278477 shutdown_connections 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.427+0000 7fb21702d700 1 -- 172.21.15.52:0/1117278477 wait complete. 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 Processor -- start 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 -- start start 2024-06-02T23:34:22.429 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 0x7fb21019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb210101fa0 0x7fb21019ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.430 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 0x7fb210196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb210111580 con 0x7fb210101fa0 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb210111400 con 0x7fb2101015f0 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb21702d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb210111700 con 0x7fb2101066c0 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb2155ca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 0x7fb210196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb2155ca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 0x7fb210196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:51278/0 (socket says 172.21.15.52:51278) 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb2155ca700 1 -- 172.21.15.52:0/560667592 learned_addr learned my addr 172.21.15.52:0/560667592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.429+0000 7fb20ffff700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb210101fa0 0x7fb21019ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb214dc9700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 0x7fb21019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.431 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb2155ca700 1 -- 172.21.15.52:0/560667592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 msgr2=0x7fb21019b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb2155ca700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 0x7fb21019b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb2155ca700 1 -- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb210101fa0 msgr2=0x7fb21019ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb2155ca700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb210101fa0 0x7fb21019ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb2155ca700 1 -- 172.21.15.52:0/560667592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb210196f90 con 0x7fb2101066c0 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb20ffff700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb210101fa0 0x7fb21019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb214dc9700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 0x7fb21019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb2155ca700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 0x7fb210196a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb208009850 tx=0x7fb208007ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb20dffb700 1 -- 172.21.15.52:0/560667592 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20801f070 con 0x7fb2101066c0 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2101971c0 con 0x7fb2101066c0 2024-06-02T23:34:22.432 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.430+0000 7fb20dffb700 1 -- 172.21.15.52:0/560667592 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb208005500 con 0x7fb2101066c0 2024-06-02T23:34:22.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.431+0000 7fb20dffb700 1 -- 172.21.15.52:0/560667592 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20801b7b0 con 0x7fb2101066c0 2024-06-02T23:34:22.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.431+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb210197630 con 0x7fb2101066c0 2024-06-02T23:34:22.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.434+0000 7fb20dffb700 1 -- 172.21.15.52:0/560667592 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb208022070 con 0x7fb2101066c0 2024-06-02T23:34:22.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.435+0000 7fb20dffb700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2000a2d40 0x7fb2000a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.435 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.435+0000 7fb20dffb700 1 -- 172.21.15.52:0/560667592 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb20800f020 con 0x7fb2101066c0 2024-06-02T23:34:22.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.435+0000 7fb214dc9700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2000a2d40 0x7fb2000a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.436+0000 7fb214dc9700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2000a2d40 0x7fb2000a5200 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb204005fd0 tx=0x7fb204005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:22.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.436+0000 7fb2218b0b80 1 Processor -- start 2024-06-02T23:34:22.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.436+0000 7fb2218b0b80 1 -- start start 2024-06-02T23:34:22.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 0x55808877e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 0x5580887787a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 0x55808877e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 0x55808877e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:51288/0 (socket says 172.21.15.52:51288) 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x558088779100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2218b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55808866b0f0 con 0x558088778ce0 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb20ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 0x5580887787a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb20ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 0x5580887787a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43886/0 (socket says 172.21.15.52:43886) 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb20ffff700 1 -- 172.21.15.52:0/2047804711 learned_addr learned my addr 172.21.15.52:0/2047804711 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5580886c99e0 con 0x55808877eeb0 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580886315f0 con 0x55808877e590 2024-06-02T23:34:22.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.437+0000 7fb2155ca700 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x558088779100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb20ffff700 1 -- 172.21.15.52:0/2047804711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 msgr2=0x55808877e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 0x55808877e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb20ffff700 1 -- 172.21.15.52:0/2047804711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 msgr2=0x558088779100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x558088779100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb20ffff700 1 -- 172.21.15.52:0/2047804711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558088618a60 con 0x55808877eeb0 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb214dc9700 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 0x55808877e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb2155ca700 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x558088779100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.438+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 0x5580887787a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009ce0 tx=0x7fb1fc0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b36d13fa2ec37980 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.439+0000 7fb1faffd700 1 -- 172.21.15.52:0/2047804711 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc010c50 con 0x55808877eeb0 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.439+0000 7fb1faffd700 1 -- 172.21.15.52:0/2047804711 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1fc010df0 con 0x55808877eeb0 2024-06-02T23:34:22.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.439+0000 7fb1faffd700 1 -- 172.21.15.52:0/2047804711 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc011820 con 0x55808877eeb0 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 msgr2=0x5580887787a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 0x5580887787a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009ce0 tx=0x7fb1fc0103d0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 shutdown_connections 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x558088779100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55808877e590 0x55808877e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2047804711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877eeb0 0x5580887787a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 >> 172.21.15.52:0/2047804711 conn(0x5580886a5210 msgr2=0x558088773a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:22.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 shutdown_connections 2024-06-02T23:34:22.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.440+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2047804711 wait complete. 2024-06-02T23:34:22.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.441+0000 7fb2218b0b80 1 -- start start 2024-06-02T23:34:22.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.441+0000 7fb2218b0b80 1 Processor -- start 2024-06-02T23:34:22.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.441+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088778ce0 0x558088784e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x558088782110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.443 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088782650 0x558088782a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088778ce0 0x558088784e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088778ce0 0x558088784e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:51300/0 (socket says 172.21.15.52:51300) 2024-06-02T23:34:22.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb214dc9700 1 -- 172.21.15.52:0/4110348065 learned_addr learned my addr 172.21.15.52:0/4110348065 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:22.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55808866a150 con 0x558088782650 2024-06-02T23:34:22.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x558088782110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.445 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5580886c59a0 con 0x55808877e590 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558088785890 con 0x558088778ce0 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2155ca700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088782650 0x558088782a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 10 client.0 _ll_register_callbacks cb 0x55808865f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 -- 172.21.15.52:0/4110348065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088778ce0 msgr2=0x558088784e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088778ce0 0x558088784e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 -- 172.21.15.52:0/4110348065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088782650 msgr2=0x558088782a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088782650 0x558088782a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55808868f8e0 con 0x55808877e590 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2155ca700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088782650 0x558088782a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1fc010df0 con 0x55808877e590 2024-06-02T23:34:22.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb20ffff700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x558088782110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc014000 tx=0x7fb1fc0098a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=377b62992d1fc915 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.442+0000 7fb1effff700 10 client.30545 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb1effff700 1 -- 172.21.15.52:0/4110348065 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc004960 con 0x55808877e590 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb1effff700 1 -- 172.21.15.52:0/4110348065 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb1fc004b00 con 0x55808877e590 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb1effff700 1 -- 172.21.15.52:0/4110348065 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb1fc004d40 con 0x55808877e590 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 10 client.30545 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5580887a1560 con 0x55808877e590 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb1effff700 1 -- 172.21.15.52:0/4110348065 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb1fc02c030 con 0x55808877e590 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 10 client.30545 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 10 client.30545 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 20 client.30545 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5580887a3de0 0x5580887a6200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 4 client.30545 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:22.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.443+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580886d6ee0 con 0x5580887a3de0 2024-06-02T23:34:22.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.444+0000 7fb214dc9700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5580887a3de0 0x5580887a6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.444+0000 7fb1effff700 10 client.30545 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:22.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.444+0000 7fb214dc9700 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5580887a3de0 0x5580887a6200 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=421ae3df5e6245dd server_cookie=965b5f01b809e25a in_seq=0 out_seq=0 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.445+0000 7fb1effff700 1 -- 172.21.15.52:0/4110348065 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5580886d6ee0 con 0x5580887a3de0 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.445+0000 7fb1effff700 10 client.30545 handle_command_reply: tid=0 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.445+0000 7fb2218b0b80 1 client.30545 shutdown 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 msgr2=0x558088782110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x558088782110 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc014000 tx=0x7fb1fc0098a0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 shutdown_connections 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5580887a3de0 0x5580887a6200 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088782650 0x558088782a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088778ce0 0x558088784e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/4110348065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x558088782110 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 >> 172.21.15.52:0/4110348065 conn(0x5580886a5210 msgr2=0x558088772c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:22.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.446+0000 7fb1effff700 0 client.30545 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:22.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.447+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 shutdown_connections 2024-06-02T23:34:22.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.447+0000 7fb2218b0b80 1 -- 172.21.15.52:0/4110348065 wait complete. 2024-06-02T23:34:22.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.447+0000 7fb2218b0b80 20 client.30545 trim_cache size 0 max 16384 2024-06-02T23:34:22.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68913 6.73 MB/sec execute 450 sec latency 102.592 ms 2024-06-02T23:34:22.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.484+0000 7fb2218b0b80 1 Processor -- start 2024-06-02T23:34:22.485 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.484+0000 7fb2218b0b80 1 -- start start 2024-06-02T23:34:22.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558088778ce0 0x5580887733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 0x558088773900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.486 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887abb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2218b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb210072ea0 con 0x55808877e590 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb20ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 0x558088773900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2218b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2101b49a0 con 0x558088778ce0 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb20ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 0x558088773900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50476/0 (socket says 172.21.15.52:50476) 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb20ffff700 1 -- 172.21.15.52:0/2073685566 learned_addr learned my addr 172.21.15.52:0/2073685566 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558088778ce0 0x5580887733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2073685566 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb210073570 con 0x558088782650 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.485+0000 7fb2155ca700 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887abb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb20ffff700 1 -- 172.21.15.52:0/2073685566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 msgr2=0x5580887abb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887abb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb20ffff700 1 -- 172.21.15.52:0/2073685566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558088778ce0 msgr2=0x5580887733c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558088778ce0 0x5580887733c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb20ffff700 1 -- 172.21.15.52:0/2073685566 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558088781ee0 con 0x55808877e590 2024-06-02T23:34:22.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb214dc9700 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558088778ce0 0x5580887733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:22.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 0x558088773900 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009c00 tx=0x7fb1fc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec218fdbee54edf1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:22.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.486+0000 7fb1fa7fc700 1 -- 172.21.15.52:0/2073685566 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03c070 con 0x55808877e590 2024-06-02T23:34:22.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb1fa7fc700 1 -- 172.21.15.52:0/2073685566 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1fc010df0 con 0x55808877e590 2024-06-02T23:34:22.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb1fa7fc700 1 -- 172.21.15.52:0/2073685566 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc0376d0 con 0x55808877e590 2024-06-02T23:34:22.490 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2073685566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 msgr2=0x558088773900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 0x558088773900 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009c00 tx=0x7fb1fc013000 comp rx=0 tx=0).stop 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2073685566 shutdown_connections 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887abb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558088778ce0 0x5580887733c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2073685566 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55808877e590 0x558088773900 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.487+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2073685566 >> 172.21.15.52:0/2073685566 conn(0x5580886a5210 msgr2=0x5580887841d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.488+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2073685566 shutdown_connections 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.488+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2073685566 wait complete. 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.488+0000 7fb2218b0b80 1 -- start start 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.488+0000 7fb2218b0b80 1 Processor -- start 2024-06-02T23:34:22.491 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x5580887ab770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x5580887abcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x5580887ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb210074310 con 0x558088778ce0 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb214dc9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x5580887ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:50486/0 (socket says 172.21.15.52:50486) 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb214dc9700 1 -- 172.21.15.52:0/2065458093 learned_addr learned my addr 172.21.15.52:0/2065458093 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb210073c40 con 0x55808877e590 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb210070e60 con 0x558088782650 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2218b0b80 10 client.0 _ll_register_callbacks cb 0x55808865f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x5580887abcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.492 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb2155ca700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb20ffff700 1 -- 172.21.15.52:0/2065458093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 msgr2=0x5580887ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb20ffff700 1 -- 172.21.15.52:0/2065458093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 msgr2=0x5580887ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x5580887ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb20ffff700 1 -- 172.21.15.52:0/2065458093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55808880db30 con 0x55808877e590 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.489+0000 7fb214dc9700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x5580887ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb20ffff700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x5580887abcb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5580887745a0 tx=0x7fb1fc0376a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4bbb6bd04c59ee67 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:22.493 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb1effff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb1effff700 1 -- 172.21.15.52:0/2065458093 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1fc03c040 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb1effff700 1 -- 172.21.15.52:0/2065458093 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb1fc037810 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1fc010df0 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb1effff700 1 -- 172.21.15.52:0/2065458093 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb1fc048070 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb2218b0b80 10 client.30551 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.490+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55808880ee10 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb1effff700 1 -- 172.21.15.52:0/2065458093 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb1fc010df0 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb1effff700 1 -- 172.21.15.52:0/2065458093 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb1fc02c070 con 0x55808877e590 2024-06-02T23:34:22.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb2218b0b80 10 client.30551 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb2218b0b80 10 client.30551 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb2218b0b80 20 client.30551 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55808880f410 0x558088811800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb2218b0b80 4 client.30551 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x5580886d1460 con 0x55808880f410 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.491+0000 7fb214dc9700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55808880f410 0x558088811800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.492+0000 7fb214dc9700 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55808880f410 0x558088811800 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e5a9b6f03bd18a6 server_cookie=85d60695b53ebc8a in_seq=0 out_seq=0 2024-06-02T23:34:22.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.492+0000 7fb1effff700 10 client.30551 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.830+0000 7fb1effff700 1 -- 172.21.15.52:0/2065458093 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5580886d1460 con 0x55808880f410 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.830+0000 7fb1effff700 10 client.30551 handle_command_reply: tid=0 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.830+0000 7fb2218b0b80 1 client.30551 shutdown 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 msgr2=0x5580887abcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x5580887abcb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x5580887745a0 tx=0x7fb1fc0376a0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 shutdown_connections 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55808880f410 0x558088811800 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558088782650 0x5580887ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x558088778ce0 0x5580887ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb1effff700 0 client.30551 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 --2- 172.21.15.52:0/2065458093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55808877e590 0x5580887abcb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 >> 172.21.15.52:0/2065458093 conn(0x5580886a5210 msgr2=0x55808877d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 shutdown_connections 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 1 -- 172.21.15.52:0/2065458093 wait complete. 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.831+0000 7fb2218b0b80 20 client.30551 trim_cache size 0 max 16384 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:34:22.833 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 74, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "5a0c4e17": { 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 74, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:34:22.834 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:22.835 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2000a2d40 msgr2=0x7fb2000a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2000a2d40 0x7fb2000a5200 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb204005fd0 tx=0x7fb204005ee0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 msgr2=0x7fb210196a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 0x7fb210196a50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb208009850 tx=0x7fb208007ad0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 shutdown_connections 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb2000a2d40 0x7fb2000a5200 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb2101066c0 0x7fb210196a50 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2101015f0 0x7fb21019b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 --2- 172.21.15.52:0/560667592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb210101fa0 0x7fb21019ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 >> 172.21.15.52:0/560667592 conn(0x7fb210078330 msgr2=0x7fb2100fe370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 shutdown_connections 2024-06-02T23:34:22.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:22.832+0000 7fb21702d700 1 -- 172.21.15.52:0/560667592 wait complete. 2024-06-02T23:34:23.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.72 MB/sec execute 451 sec latency 993.997 ms 2024-06-02T23:34:23.609 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:23 smithi052 ceph-mon[32340]: pgmap v1106: 97 pgs: 97 active+clean; 551 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-06-02T23:34:23.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:23 smithi174 ceph-mon[35994]: pgmap v1106: 97 pgs: 97 active+clean; 551 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-06-02T23:34:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:23 smithi123 ceph-mon[35977]: pgmap v1106: 97 pgs: 97 active+clean; 551 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 813 KiB/s rd, 2.8 MiB/s wr, 183 op/s 2024-06-02T23:34:23.824 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '5a0c4e17', quiesce took 10.2 seconds. Will release after: 19.8 2024-06-02T23:34:24.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.71 MB/sec execute 452 sec latency 1994.087 ms 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 -- 172.21.15.52:0/137886279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450107760 msgr2=0x7f1450109b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 --2- 172.21.15.52:0/137886279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450107760 0x7f1450109b50 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f1448009ca0 tx=0x7f144802a8c0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 -- 172.21.15.52:0/137886279 shutdown_connections 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 --2- 172.21.15.52:0/137886279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450104e30 0x7f1450107220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 --2- 172.21.15.52:0/137886279 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450107760 0x7f1450109b50 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 --2- 172.21.15.52:0/137886279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1450102500 0x7f14501048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 -- 172.21.15.52:0/137886279 >> 172.21.15.52:0/137886279 conn(0x7f14500fbeb0 msgr2=0x7f14500fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 -- 172.21.15.52:0/137886279 shutdown_connections 2024-06-02T23:34:24.923 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 -- 172.21.15.52:0/137886279 wait complete. 2024-06-02T23:34:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 Processor -- start 2024-06-02T23:34:24.924 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.922+0000 7f1456693640 1 -- start start 2024-06-02T23:34:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1456693640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1450102500 0x7f1450197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.925 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1456693640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 0x7f14501980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1456693640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450107760 0x7f145019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1456693640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f145010d570 con 0x7f1450104e30 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1456693640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f145010d3f0 con 0x7f1450102500 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1456693640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f145010d6f0 con 0x7f1450107760 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 0x7f14501980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 0x7f14501980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40182/0 (socket says 172.21.15.52:40182) 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 -- 172.21.15.52:0/1485188394 learned_addr learned my addr 172.21.15.52:0/1485188394 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144ffff640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1450102500 0x7f1450197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f1454c30640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450107760 0x7f145019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 -- 172.21.15.52:0/1485188394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450107760 msgr2=0x7f145019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450107760 0x7f145019f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 -- 172.21.15.52:0/1485188394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1450102500 msgr2=0x7f1450197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:24.926 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1450102500 0x7f1450197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 -- 172.21.15.52:0/1485188394 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f145019f7a0 con 0x7f1450104e30 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.923+0000 7f144f7fe640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 0x7f14501980c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f144400c970 tx=0x7f144400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.924+0000 7f144d7fa640 1 -- 172.21.15.52:0/1485188394 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444007bb0 con 0x7f1450104e30 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.924+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f145019fa90 con 0x7f1450104e30 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.924+0000 7f144d7fa640 1 -- 172.21.15.52:0/1485188394 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1444007d50 con 0x7f1450104e30 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.924+0000 7f144d7fa640 1 -- 172.21.15.52:0/1485188394 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1444018730 con 0x7f1450104e30 2024-06-02T23:34:24.927 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.924+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f145019ff90 con 0x7f1450104e30 2024-06-02T23:34:24.932 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.926+0000 7f144d7fa640 1 -- 172.21.15.52:0/1485188394 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f144400f450 con 0x7f1450104e30 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.926+0000 7f144d7fa640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14380a2c30 0x7f14380a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144ffff640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14380a2c30 0x7f14380a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 Processor -- start 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144d7fa640 1 -- 172.21.15.52:0/1485188394 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f14440cf180 con 0x7f1450104e30 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 -- start start 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abe010 0x563824abe430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824ac3b70 0x563824ac6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563824a214f0 con 0x563824abea00 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5638249bf870 con 0x563824abe010 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f145939f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563824a1d4b0 con 0x563824ac3b70 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.933 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40196/0 (socket says 172.21.15.52:40196) 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144f7fe640 1 -- 172.21.15.52:0/832370436 learned_addr learned my addr 172.21.15.52:0/832370436 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144ffff640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14380a2c30 0x7f14380a50f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1440009bd0 tx=0x7f1440005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f144ffff640 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abe010 0x563824abe430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.927+0000 7f1454c30640 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824ac3b70 0x563824ac6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f144f7fe640 1 -- 172.21.15.52:0/832370436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824ac3b70 msgr2=0x563824ac6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f144f7fe640 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824ac3b70 0x563824ac6000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f144f7fe640 1 -- 172.21.15.52:0/832370436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abe010 msgr2=0x563824abe430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f144f7fe640 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abe010 0x563824abe430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f144f7fe640 1 -- 172.21.15.52:0/832370436 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5638249db9b0 con 0x563824abea00 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f144f7fe640 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac3500 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1444098380 tx=0x7f144409b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0ba259a37369ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f14367fc640 1 -- 172.21.15.52:0/832370436 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440b3020 con 0x563824abea00 2024-06-02T23:34:24.934 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f14367fc640 1 -- 172.21.15.52:0/832370436 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14440a4620 con 0x563824abea00 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.928+0000 7f14367fc640 1 -- 172.21.15.52:0/832370436 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440a4190 con 0x563824abea00 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 -- 172.21.15.52:0/832370436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 msgr2=0x563824ac3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac3500 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f1444098380 tx=0x7f144409b8b0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 -- 172.21.15.52:0/832370436 shutdown_connections 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824ac3b70 0x563824ac6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac3500 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 --2- 172.21.15.52:0/832370436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abe010 0x563824abe430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 -- 172.21.15.52:0/832370436 >> 172.21.15.52:0/832370436 conn(0x563824ab9680 msgr2=0x563824ab9a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 -- 172.21.15.52:0/832370436 shutdown_connections 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 -- 172.21.15.52:0/832370436 wait complete. 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 -- start start 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.929+0000 7f145939f740 1 Processor -- start 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824ac8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.935 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ac3b70 0x563824acf7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563824ac6b20 con 0x563824abea00 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5638249aaae0 con 0x563824ac3b70 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563824ac6ca0 con 0x563824abe010 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40210/0 (socket says 172.21.15.52:40210) 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 -- 172.21.15.52:0/2661716485 learned_addr learned my addr 172.21.15.52:0/2661716485 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144ffff640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824ac8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f145939f740 10 client.0 _ll_register_callbacks cb 0x5638249e39b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f1454c30640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ac3b70 0x563824acf7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 -- 172.21.15.52:0/2661716485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 msgr2=0x563824ac8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.936 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824ac8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 -- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ac3b70 msgr2=0x563824acf7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ac3b70 0x563824acf7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 -- 172.21.15.52:0/2661716485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563824735210 con 0x563824abea00 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144ffff640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824ac8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f144f7fe640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac8790 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1444097dc0 tx=0x7f1444004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c267a3cb3ad9fb2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.930+0000 7f142f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f142f7fe640 1 -- 172.21.15.52:0/2661716485 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14440b3070 con 0x563824abea00 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f142f7fe640 1 -- 172.21.15.52:0/2661716485 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f14440a4190 con 0x563824abea00 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14440a4620 con 0x563824abea00 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f142f7fe640 1 -- 172.21.15.52:0/2661716485 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f14440a43d0 con 0x563824abea00 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 10 client.21303 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563824801620 con 0x563824abea00 2024-06-02T23:34:24.937 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f142f7fe640 1 -- 172.21.15.52:0/2661716485 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f14440a4620 con 0x563824abea00 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f142f7fe640 1 -- 172.21.15.52:0/2661716485 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f144409cce0 con 0x563824abea00 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 10 client.21303 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 10 client.21303 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 20 client.21303 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ad3ca0 0x563824ad6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 4 client.21303 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563824a34f10 con 0x563824ad3ca0 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.931+0000 7f144ffff640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ad3ca0 0x563824ad6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.932+0000 7f142f7fe640 10 client.21303 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:24.938 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.932+0000 7f144ffff640 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ad3ca0 0x563824ad6090 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51f3da90795679ee server_cookie=bb59f556b08f3d84 in_seq=0 out_seq=0 2024-06-02T23:34:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f142f7fe640 1 -- 172.21.15.52:0/2661716485 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563824a34f10 con 0x563824ad3ca0 2024-06-02T23:34:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f142f7fe640 10 client.21303 handle_command_reply: tid=0 2024-06-02T23:34:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 client.21303 shutdown 2024-06-02T23:34:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 msgr2=0x563824ac8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:24.939 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac8790 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f1444097dc0 tx=0x7f1444004730 comp rx=0 tx=0).stop 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 shutdown_connections 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ad3ca0 0x563824ad6090 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ac3b70 0x563824acf7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824ac8790 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 --2- 172.21.15.52:0/2661716485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824ac8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f142f7fe640 0 client.21303 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 >> 172.21.15.52:0/2661716485 conn(0x563824ab9680 msgr2=0x563824ab87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 shutdown_connections 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 1 -- 172.21.15.52:0/2661716485 wait complete. 2024-06-02T23:34:24.940 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.933+0000 7f145939f740 20 client.21303 trim_cache size 0 max 16384 2024-06-02T23:34:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 Processor -- start 2024-06-02T23:34:24.950 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 -- start start 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 0x563824abfbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824acf790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abea00 0x563824ac0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a8be0 con 0x563824ab9680 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14501aa720 con 0x563824abea00 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f145939f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a92b0 con 0x563824abe010 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f144ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 0x563824abfbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f144ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 0x563824abfbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40222/0 (socket says 172.21.15.52:40222) 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f144ffff640 1 -- 172.21.15.52:0/4057837531 learned_addr learned my addr 172.21.15.52:0/4057837531 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f1454c30640 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abea00 0x563824ac0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.949+0000 7f144f7fe640 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824acf790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f144ffff640 1 -- 172.21.15.52:0/4057837531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 msgr2=0x563824acf790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.951 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f144ffff640 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824acf790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f144ffff640 1 -- 172.21.15.52:0/4057837531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abea00 msgr2=0x563824ac0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f144ffff640 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abea00 0x563824ac0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f144ffff640 1 -- 172.21.15.52:0/4057837531 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563824ab94b0 con 0x563824ab9680 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f144ffff640 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 0x563824abfbe0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x563824ad68d0 tx=0x7f1440014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6043ec0a9595ca0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f1435ffb640 1 -- 172.21.15.52:0/4057837531 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440036070 con 0x563824ab9680 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f1435ffb640 1 -- 172.21.15.52:0/4057837531 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1440003d00 con 0x563824ab9680 2024-06-02T23:34:24.952 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.950+0000 7f1435ffb640 1 -- 172.21.15.52:0/4057837531 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144001d700 con 0x563824ab9680 2024-06-02T23:34:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 -- 172.21.15.52:0/4057837531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 msgr2=0x563824abfbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 0x563824abfbe0 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x563824ad68d0 tx=0x7f1440014000 comp rx=0 tx=0).stop 2024-06-02T23:34:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 -- 172.21.15.52:0/4057837531 shutdown_connections 2024-06-02T23:34:24.955 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824acf790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824abea00 0x563824ac0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 --2- 172.21.15.52:0/4057837531 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824ab9680 0x563824abfbe0 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 -- 172.21.15.52:0/4057837531 >> 172.21.15.52:0/4057837531 conn(0x563824abf4c0 msgr2=0x563824ac7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 -- 172.21.15.52:0/4057837531 shutdown_connections 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 -- 172.21.15.52:0/4057837531 wait complete. 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 -- start start 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.951+0000 7f145939f740 1 Processor -- start 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ab9680 0x563824acca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824accf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824acdf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f14501aa050 con 0x563824abea00 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a9980 con 0x563824ab9680 2024-06-02T23:34:24.956 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14501a6bc0 con 0x563824abe010 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f145939f740 10 client.0 _ll_register_callbacks cb 0x7f14400099e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824acdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824acdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40234/0 (socket says 172.21.15.52:40234) 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 -- 172.21.15.52:0/2011479176 learned_addr learned my addr 172.21.15.52:0/2011479176 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f144f7fe640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824accf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f144ffff640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ab9680 0x563824acca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 -- 172.21.15.52:0/2011479176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 msgr2=0x563824accf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824accf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 -- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ab9680 msgr2=0x563824acca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ab9680 0x563824acca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 -- 172.21.15.52:0/2011479176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563824ace550 con 0x563824abea00 2024-06-02T23:34:24.957 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f1454c30640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824acdf60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1448035000 tx=0x7f1448031d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa71f283940c2089 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.952+0000 7f142f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f142f7fe640 1 -- 172.21.15.52:0/2011479176 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f144804a060 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1440003d00 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f142f7fe640 1 -- 172.21.15.52:0/2011479176 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1448003d80 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f142f7fe640 1 -- 172.21.15.52:0/2011479176 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1448004040 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 10 client.21309 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563824acf8d0 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f142f7fe640 1 -- 172.21.15.52:0/2011479176 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1440003d00 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f142f7fe640 1 -- 172.21.15.52:0/2011479176 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1448030a50 con 0x563824abea00 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 10 client.21309 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 10 client.21309 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 20 client.21309 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:24.958 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ac3b70 0x563824b40e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 4 client.21309 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:34:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.953+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563824a2ec20 con 0x563824ac3b70 2024-06-02T23:34:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.954+0000 7f144ffff640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ac3b70 0x563824b40e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.954+0000 7f144ffff640 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ac3b70 0x563824b40e90 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abb9302f6732f991 server_cookie=91b76dbefc30d739 in_seq=0 out_seq=0 2024-06-02T23:34:24.959 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:24.954+0000 7f142f7fe640 10 client.21309 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.004+0000 7f142f7fe640 1 -- 172.21.15.52:0/2011479176 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x563824a2ec20 con 0x563824ac3b70 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.004+0000 7f142f7fe640 10 client.21309 handle_command_reply: tid=0 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.004+0000 7f145939f740 1 client.21309 shutdown 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 msgr2=0x563824acdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824acdf60 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7f1448035000 tx=0x7f1448031d70 comp rx=0 tx=0).stop 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 shutdown_connections 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x563824ac3b70 0x563824b40e90 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563824abe010 0x563824accf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563824abea00 0x563824acdf60 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f142f7fe640 0 client.21309 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 --2- 172.21.15.52:0/2011479176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563824ab9680 0x563824acca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 >> 172.21.15.52:0/2011479176 conn(0x563824abf4c0 msgr2=0x563824ac8230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 shutdown_connections 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 1 -- 172.21.15.52:0/2011479176 wait complete. 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f145939f740 20 client.21309 trim_cache size 0 max 16384 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:34:25.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:34:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14380a2c30 msgr2=0x7f14380a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f1456693640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14380a2c30 0x7f14380a50f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f1440009bd0 tx=0x7f1440005f90 comp rx=0 tx=0).stop 2024-06-02T23:34:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 msgr2=0x7f14501980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.005+0000 7f1456693640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 0x7f14501980c0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f144400c970 tx=0x7f144400ce30 comp rx=0 tx=0).stop 2024-06-02T23:34:25.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 shutdown_connections 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f14380a2c30 0x7f14380a50f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1450107760 0x7f145019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1450104e30 0x7f14501980c0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 --2- 172.21.15.52:0/1485188394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1450102500 0x7f1450197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 >> 172.21.15.52:0/1485188394 conn(0x7f14500fbeb0 msgr2=0x7f1450102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 shutdown_connections 2024-06-02T23:34:25.009 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:25.006+0000 7f1456693640 1 -- 172.21.15.52:0/1485188394 wait complete. 2024-06-02T23:34:25.400 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:25 smithi052 ceph-mon[32340]: pgmap v1107: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-06-02T23:34:25.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 -- 172.21.15.52:0/3780501276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0107870 msgr2=0x7f1ef010f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.433 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 --2- 172.21.15.52:0/3780501276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0107870 0x7f1ef010f3f0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40099c0 tx=0x7f1ee4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 -- 172.21.15.52:0/3780501276 shutdown_connections 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 --2- 172.21.15.52:0/3780501276 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0107870 0x7f1ef010f3f0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 --2- 172.21.15.52:0/3780501276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef01027a0 0x7f1ef0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 --2- 172.21.15.52:0/3780501276 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef0103150 0x7f1ef0107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 -- 172.21.15.52:0/3780501276 >> 172.21.15.52:0/3780501276 conn(0x7f1ef0079520 msgr2=0x7f1ef0079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 -- 172.21.15.52:0/3780501276 shutdown_connections 2024-06-02T23:34:25.434 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 -- 172.21.15.52:0/3780501276 wait complete. 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.433+0000 7f1ef608b700 1 Processor -- start 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 -- start start 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef01027a0 0x7f1ef019e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 0x7f1ef019ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef0107870 0x7f1ef0198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef0112660 con 0x7f1ef0103150 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01124e0 con 0x7f1ef01027a0 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1ef608b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01127e0 con 0x7f1ef0107870 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 0x7f1ef019ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 0x7f1ef019ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40252/0 (socket says 172.21.15.52:40252) 2024-06-02T23:34:25.436 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 -- 172.21.15.52:0/856965587 learned_addr learned my addr 172.21.15.52:0/856965587 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef01027a0 0x7f1ef019e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeffff700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef0107870 0x7f1ef0198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 -- 172.21.15.52:0/856965587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef0107870 msgr2=0x7f1ef0198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef0107870 0x7f1ef0198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 -- 172.21.15.52:0/856965587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef01027a0 msgr2=0x7f1ef019e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef01027a0 0x7f1ef019e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.434+0000 7f1eeeffd700 1 -- 172.21.15.52:0/856965587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ef0198fc0 con 0x7f1ef0103150 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef01027a0 0x7f1ef019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 0x7f1ef019ebe0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80060b0 tx=0x7f1ee800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1eecff9700 1 -- 172.21.15.52:0/856965587 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee801a070 con 0x7f1ef0103150 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1eecff9700 1 -- 172.21.15.52:0/856965587 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ee800ddd0 con 0x7f1ef0103150 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ef01992b0 con 0x7f1ef0103150 2024-06-02T23:34:25.437 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1eecff9700 1 -- 172.21.15.52:0/856965587 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8014720 con 0x7f1ef0103150 2024-06-02T23:34:25.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.435+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ef01a5570 con 0x7f1ef0103150 2024-06-02T23:34:25.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.437+0000 7f1eecff9700 1 -- 172.21.15.52:0/856965587 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1ee801d070 con 0x7f1ef0103150 2024-06-02T23:34:25.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.438+0000 7f1eecff9700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ed80a2ca0 0x7f1ed80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.438+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ed80a2ca0 0x7f1ed80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.438 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.438+0000 7f1eecff9700 1 -- 172.21.15.52:0/856965587 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1ee80ce860 con 0x7f1ef0103150 2024-06-02T23:34:25.440 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.438+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ed80a2ca0 0x7f1ed80a5160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1ef01999a0 tx=0x7f1ee000b2c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 Processor -- start 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 -- start start 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820bb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558681fa9f60 con 0x5586820c1370 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558682008850 con 0x5586820bbbb0 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558681f70460 con 0x5586820c1c90 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eef7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40260/0 (socket says 172.21.15.52:40260) 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eef7fe700 1 -- 172.21.15.52:0/1310471587 learned_addr learned my addr 172.21.15.52:0/1310471587 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:25.441 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820bb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eeffff700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eef7fe700 1 -- 172.21.15.52:0/1310471587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 msgr2=0x5586820bb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820bb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.439+0000 7f1eef7fe700 1 -- 172.21.15.52:0/1310471587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 msgr2=0x5586820bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1eef7fe700 1 -- 172.21.15.52:0/1310471587 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558681f578d0 con 0x5586820c1370 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1eeffff700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c1750 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000ff30 tx=0x7f1ee0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c31a4e9edf8715 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1ed5ffb700 1 -- 172.21.15.52:0/1310471587 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0022070 con 0x5586820c1370 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1ed5ffb700 1 -- 172.21.15.52:0/1310471587 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ee0011c60 con 0x5586820c1370 2024-06-02T23:34:25.442 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.440+0000 7f1ed5ffb700 1 -- 172.21.15.52:0/1310471587 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee001d5b0 con 0x5586820c1370 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1310471587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 msgr2=0x5586820c1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c1750 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f1ee000ff30 tx=0x7f1ee0012000 comp rx=0 tx=0).stop 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1310471587 shutdown_connections 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c1750 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1310471587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820bb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1310471587 >> 172.21.15.52:0/1310471587 conn(0x558681fe4080 msgr2=0x5586820b2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1310471587 shutdown_connections 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1310471587 wait complete. 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 -- start start 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.441+0000 7f1f0090eb80 1 Processor -- start 2024-06-02T23:34:25.446 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1370 0x5586820c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 0x5586820c5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x558681fa8fc0 con 0x5586820c1c90 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558682004810 con 0x5586820bbbb0 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5586820c8680 con 0x5586820c1370 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1f0090eb80 10 client.0 _ll_register_callbacks cb 0x558681f9e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 0x5586820c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 0x5586820c5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40272/0 (socket says 172.21.15.52:40272) 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 -- 172.21.15.52:0/2673673791 learned_addr learned my addr 172.21.15.52:0/2673673791 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1370 0x5586820c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 -- 172.21.15.52:0/2673673791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1370 msgr2=0x5586820c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1370 0x5586820c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 -- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 msgr2=0x5586820c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.447 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 -- 172.21.15.52:0/2673673791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558681fce750 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eeffff700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 0x5586820c5210 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bee0 tx=0x7f1ee400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3827f762b1f4f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.442+0000 7f1eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1eceffd700 1 -- 172.21.15.52:0/2673673791 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee4028060 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1eceffd700 1 -- 172.21.15.52:0/2673673791 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1ee4003a90 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee0011c60 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1eceffd700 1 -- 172.21.15.52:0/2673673791 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ee4003680 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 10 client.21321 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5586820d2d20 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1eceffd700 1 -- 172.21.15.52:0/2673673791 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ee0011c60 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1eceffd700 1 -- 172.21.15.52:0/2673673791 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1ee400f580 con 0x5586820c1c90 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 10 client.21321 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 10 client.21321 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:34:25.448 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 20 client.21321 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586820d5360 0x5586820d7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 4 client.21321 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558682015d50 con 0x5586820d5360 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.443+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586820d5360 0x5586820d7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.444+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586820d5360 0x5586820d7750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=80fb7904f7a098a5 server_cookie=839fac05c130dca5 in_seq=0 out_seq=0 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.444+0000 7f1eceffd700 10 client.21321 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1eceffd700 1 -- 172.21.15.52:0/2673673791 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x558682015d50 con 0x5586820d5360 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1eceffd700 10 client.21321 handle_command_reply: tid=0 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 client.21321 shutdown 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 msgr2=0x5586820c5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 0x5586820c5210 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7f1ee400bee0 tx=0x7f1ee400bf10 comp rx=0 tx=0).stop 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 shutdown_connections 2024-06-02T23:34:25.449 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1c90 0x5586820c5210 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586820d5360 0x5586820d7750 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2673673791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1370 0x5586820c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 >> 172.21.15.52:0/2673673791 conn(0x558681fe4080 msgr2=0x5586820b1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1eceffd700 0 client.21321 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 shutdown_connections 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2673673791 wait complete. 2024-06-02T23:34:25.450 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.445+0000 7f1f0090eb80 20 client.21321 trim_cache size 0 max 16384 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.460+0000 7f1f0090eb80 1 Processor -- start 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 -- start start 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01aa750 con 0x5586820c1370 2024-06-02T23:34:25.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01ac290 con 0x5586820bbbb0 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01aae20 con 0x5586820c1c90 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c2920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40286/0 (socket says 172.21.15.52:40286) 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 -- 172.21.15.52:0/1678817439 learned_addr learned my addr 172.21.15.52:0/1678817439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeffff700 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 -- 172.21.15.52:0/1678817439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 msgr2=0x5586820c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 -- 172.21.15.52:0/1678817439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 msgr2=0x5586820c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 -- 172.21.15.52:0/1678817439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586820c3240 con 0x5586820c1370 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.461+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c2920 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8015e60 tx=0x7f1ee80bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7c06bc7f4ec195 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1ed57fa700 1 -- 172.21.15.52:0/1678817439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee80b3020 con 0x5586820c1370 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1ed57fa700 1 -- 172.21.15.52:0/1678817439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ee809b090 con 0x5586820c1370 2024-06-02T23:34:25.463 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1ed57fa700 1 -- 172.21.15.52:0/1678817439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee809f110 con 0x5586820c1370 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1678817439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 msgr2=0x5586820c2920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c2920 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8015e60 tx=0x7f1ee80bb040 comp rx=0 tx=0).stop 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1678817439 shutdown_connections 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820c1c90 0x5586820c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820bbbb0 0x5586820c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/1678817439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820c2920 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1678817439 >> 172.21.15.52:0/1678817439 conn(0x558681fe4080 msgr2=0x5586820c64b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1678817439 shutdown_connections 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.462+0000 7f1f0090eb80 1 -- 172.21.15.52:0/1678817439 wait complete. 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 -- start start 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 Processor -- start 2024-06-02T23:34:25.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820bbbb0 0x5586820cdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820c1c90 0x5586820dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01abbc0 con 0x5586820c1370 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01ab4f0 con 0x5586820c1c90 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ef01a8710 con 0x5586820bbbb0 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1f0090eb80 10 client.0 _ll_register_callbacks cb 0x558681f9e5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1eeeffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40294/0 (socket says 172.21.15.52:40294) 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1eeeffd700 1 -- 172.21.15.52:0/2890904527 learned_addr learned my addr 172.21.15.52:0/2890904527 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820bbbb0 0x5586820cdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.463+0000 7f1eeffff700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820c1c90 0x5586820dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeeffd700 1 -- 172.21.15.52:0/2890904527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820bbbb0 msgr2=0x5586820cdaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820bbbb0 0x5586820cdaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeeffd700 1 -- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820c1c90 msgr2=0x5586820dc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:25.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820c1c90 0x5586820dc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeeffd700 1 -- 172.21.15.52:0/2890904527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586820dcc00 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeffff700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820c1c90 0x5586820dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eeeffd700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820ce030 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80974f0 tx=0x7f1ee800df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557751fb4372c377 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eceffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eceffd700 1 -- 172.21.15.52:0/2890904527 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ee807d020 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eceffd700 1 -- 172.21.15.52:0/2890904527 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1ee80a3a70 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ee809b090 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eceffd700 1 -- 172.21.15.52:0/2890904527 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1ee80b3070 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 10 client.21327 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55868213fab0 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eceffd700 1 -- 172.21.15.52:0/2890904527 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1ee809b090 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1eceffd700 1 -- 172.21.15.52:0/2890904527 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1ee8096f70 con 0x5586820c1370 2024-06-02T23:34:25.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 10 client.21327 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 10 client.21327 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 20 client.21327 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.464+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586821401e0 0x558682142600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1f0090eb80 4 client.21327 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5586820102d0 con 0x5586821401e0 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586821401e0 0x558682142600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1eceffd700 10 client.21327 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1eef7fe700 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586821401e0 0x558682142600 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7bf548f5d93ec081 server_cookie=17d280ad9ab254bd in_seq=0 out_seq=0 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1eceffd700 1 -- 172.21.15.52:0/2890904527 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5586820102d0 con 0x5586821401e0 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1eceffd700 10 client.21327 handle_command_reply: tid=0 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1f0090eb80 1 client.21327 shutdown 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 msgr2=0x5586820ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.465+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820ce030 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80974f0 tx=0x7f1ee800df80 comp rx=0 tx=0).stop 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 shutdown_connections 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x5586821401e0 0x558682142600 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5586820c1c90 0x5586820dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5586820bbbb0 0x5586820cdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 --2- 172.21.15.52:0/2890904527 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5586820c1370 0x5586820ce030 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 >> 172.21.15.52:0/2890904527 conn(0x558681fe4080 msgr2=0x5586820b6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1eceffd700 0 client.21327 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 shutdown_connections 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 1 -- 172.21.15.52:0/2890904527 wait complete. 2024-06-02T23:34:25.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1f0090eb80 20 client.21327 trim_cache size 0 max 16384 2024-06-02T23:34:25.472 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:34:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ed80a2ca0 msgr2=0x7f1ed80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ed80a2ca0 0x7f1ed80a5160 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1ef01999a0 tx=0x7f1ee000b2c0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 msgr2=0x7f1ef019ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 0x7f1ef019ebe0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f1ee80060b0 tx=0x7f1ee800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 shutdown_connections 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1ed80a2ca0 0x7f1ed80a5160 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.466+0000 7f1ef608b700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ef0107870 0x7f1ef0198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.467+0000 7f1ef608b700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ef01027a0 0x7f1ef019e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.467+0000 7f1ef608b700 1 --2- 172.21.15.52:0/856965587 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ef0103150 0x7f1ef019ebe0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.467+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 >> 172.21.15.52:0/856965587 conn(0x7f1ef0079520 msgr2=0x7f1ef01089c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.467+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 shutdown_connections 2024-06-02T23:34:25.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:25.467+0000 7f1ef608b700 1 -- 172.21.15.52:0/856965587 wait complete. 2024-06-02T23:34:25.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.69 MB/sec execute 453 sec latency 2994.150 ms 2024-06-02T23:34:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:25 smithi174 ceph-mon[35994]: pgmap v1107: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-06-02T23:34:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:25 smithi123 ceph-mon[35977]: pgmap v1107: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 890 KiB/s rd, 2.9 MiB/s wr, 202 op/s 2024-06-02T23:34:26.259 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:34:26.476 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.68 MB/sec execute 454 sec latency 3994.195 ms 2024-06-02T23:34:26.514 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:27.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.66 MB/sec execute 455 sec latency 4994.300 ms 2024-06-02T23:34:27.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:27 smithi174 ceph-mon[35994]: pgmap v1108: 97 pgs: 97 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-06-02T23:34:27.706 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:27 smithi052 ceph-mon[32340]: pgmap v1108: 97 pgs: 97 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-06-02T23:34:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:27 smithi123 ceph-mon[35977]: pgmap v1108: 97 pgs: 97 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.6 MiB/s wr, 264 op/s 2024-06-02T23:34:28.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.65 MB/sec execute 456 sec latency 5994.422 ms 2024-06-02T23:34:29.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.64 MB/sec execute 457 sec latency 6994.524 ms 2024-06-02T23:34:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:29 smithi174 ceph-mon[35994]: pgmap v1109: 97 pgs: 97 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-06-02T23:34:29.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:29 smithi052 ceph-mon[32340]: pgmap v1109: 97 pgs: 97 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-06-02T23:34:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:29 smithi123 ceph-mon[35977]: pgmap v1109: 97 pgs: 97 active+clean; 556 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 950 KiB/s rd, 2.8 MiB/s wr, 216 op/s 2024-06-02T23:34:30.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.62 MB/sec execute 458 sec latency 7994.647 ms 2024-06-02T23:34:31.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.61 MB/sec execute 459 sec latency 8994.716 ms 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 -- 172.21.15.52:0/549249382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 msgr2=0x7f1aa8107130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 --2- 172.21.15.52:0/549249382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8107130 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7f1a900099c0 tx=0x7f1a90009cd0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 -- 172.21.15.52:0/549249382 shutdown_connections 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 --2- 172.21.15.52:0/549249382 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa8107860 0x7f1aa810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 --2- 172.21.15.52:0/549249382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa8102770 0x7f1aa8102b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 --2- 172.21.15.52:0/549249382 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8107130 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 -- 172.21.15.52:0/549249382 >> 172.21.15.52:0/549249382 conn(0x7f1aa8079600 msgr2=0x7f1aa8079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 -- 172.21.15.52:0/549249382 shutdown_connections 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 -- 172.21.15.52:0/549249382 wait complete. 2024-06-02T23:34:31.582 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.581+0000 7f1aadd63700 1 Processor -- start 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 -- start start 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa8102770 0x7f1aa8074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa8107860 0x7f1aa8075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8112690 con 0x7f1aa8103120 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8112510 con 0x7f1aa8107860 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aadd63700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa8112810 con 0x7f1aa8102770 2024-06-02T23:34:31.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40310/0 (socket says 172.21.15.52:40310) 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/3212479987 learned_addr learned my addr 172.21.15.52:0/3212479987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa8102770 0x7f1aa8074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa8107860 0x7f1aa8075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/3212479987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa8102770 msgr2=0x7f1aa8074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa8102770 0x7f1aa8074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/3212479987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa8107860 msgr2=0x7f1aa8075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa8107860 0x7f1aa8075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.582+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/3212479987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa81a7060 con 0x7f1aa8103120 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.583+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8074cd0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1a90005910 tx=0x7f1a900045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.583+0000 7f1aa4ff9700 1 -- 172.21.15.52:0/3212479987 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90026070 con 0x7f1aa8103120 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.583+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa81a72f0 con 0x7f1aa8103120 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.583+0000 7f1aa4ff9700 1 -- 172.21.15.52:0/3212479987 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a9000f460 con 0x7f1aa8103120 2024-06-02T23:34:31.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.583+0000 7f1aa4ff9700 1 -- 172.21.15.52:0/3212479987 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a90009e60 con 0x7f1aa8103120 2024-06-02T23:34:31.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.583+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa81a77a0 con 0x7f1aa8103120 2024-06-02T23:34:31.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.585+0000 7f1aa4ff9700 1 -- 172.21.15.52:0/3212479987 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1a90029070 con 0x7f1aa8103120 2024-06-02T23:34:31.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.586+0000 7f1aa4ff9700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a940a2ca0 0x7f1a940a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.586+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a940a2ca0 0x7f1a940a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.586+0000 7f1aa4ff9700 1 -- 172.21.15.52:0/3212479987 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1a900cf840 con 0x7f1aa8103120 2024-06-02T23:34:31.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.586+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a940a2ca0 0x7f1a940a5160 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009c60 tx=0x7f1a98009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:31.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 Processor -- start 2024-06-02T23:34:31.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 -- start start 2024-06-02T23:34:31.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6df60 0x561ca1e6e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e68bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561ca1d5af60 con 0x561ca1e687a0 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ca1db9850 con 0x561ca1e6df60 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ca1d21460 con 0x561ca1e6e880 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1aa7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e68bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1aa7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e68bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40326/0 (socket says 172.21.15.52:40326) 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1aa7fff700 1 -- 172.21.15.52:0/4165255362 learned_addr learned my addr 172.21.15.52:0/4165255362 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.587+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6df60 0x561ca1e6e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa7fff700 1 -- 172.21.15.52:0/4165255362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 msgr2=0x561ca1e68260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e68260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa7fff700 1 -- 172.21.15.52:0/4165255362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6df60 msgr2=0x561ca1e6e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6df60 0x561ca1e6e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa7fff700 1 -- 172.21.15.52:0/4165255362 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca1d088d0 con 0x561ca1e687a0 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e68260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e68bc0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009d20 tx=0x7f1aa00077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fe4b41a406f095 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1a9dffb700 1 -- 172.21.15.52:0/4165255362 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0004bb0 con 0x561ca1e687a0 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1a9dffb700 1 -- 172.21.15.52:0/4165255362 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aa0004d50 con 0x561ca1e687a0 2024-06-02T23:34:31.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.588+0000 7f1a9dffb700 1 -- 172.21.15.52:0/4165255362 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0014630 con 0x561ca1e687a0 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/4165255362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 msgr2=0x561ca1e68bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e68bc0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009d20 tx=0x7f1aa00077e0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/4165255362 shutdown_connections 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e68bc0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6df60 0x561ca1e6e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/4165255362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e68260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/4165255362 >> 172.21.15.52:0/4165255362 conn(0x561ca1d95080 msgr2=0x561ca1e63220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/4165255362 shutdown_connections 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/4165255362 wait complete. 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 -- start start 2024-06-02T23:34:31.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.589+0000 7f1ab85e6b80 1 Processor -- start 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e74580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e71880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6e880 0x561ca1e71dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561ca1d59fc0 con 0x561ca1e687a0 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561ca1db5810 con 0x561ca1e6e880 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561ca1e75260 con 0x561ca1e6df60 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1ab85e6b80 10 client.0 _ll_register_callbacks cb 0x561ca1d4f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e74580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e74580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40338/0 (socket says 172.21.15.52:40338) 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 -- 172.21.15.52:0/1061973470 learned_addr learned my addr 172.21.15.52:0/1061973470 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6e880 0x561ca1e71dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e71880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 -- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 msgr2=0x561ca1e71880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e71880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 -- 172.21.15.52:0/1061973470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6e880 msgr2=0x561ca1e71dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6e880 0x561ca1e71dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 -- 172.21.15.52:0/1061973470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca1d7f750 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e74580 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1a98003900 tx=0x7f1a980047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd98503d3f1a808a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1a86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.590+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1061973470 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a98027420 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1061973470 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1a980275c0 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1aa0004d50 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1061973470 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1a980277c0 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 10 client.21339 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561ca1e7f9d0 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1061973470 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1aa0004d50 con 0x561ca1e687a0 2024-06-02T23:34:31.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1061973470 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1a9801db10 con 0x561ca1e687a0 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 10 client.21339 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 10 client.21339 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 20 client.21339 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1e82010 0x561ca1e84460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 4 client.21339 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.591+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ca1dc6d50 con 0x561ca1e82010 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.592+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1e82010 0x561ca1e84460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.592+0000 7f1a86ffd700 10 client.21339 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.592+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1e82010 0x561ca1e84460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ed073569b46e3243 server_cookie=2825284f1e5ec050 in_seq=0 out_seq=0 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1061973470 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561ca1dc6d50 con 0x561ca1e82010 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1a86ffd700 10 client.21339 handle_command_reply: tid=0 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 client.21339 shutdown 2024-06-02T23:34:31.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 msgr2=0x561ca1e74580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e74580 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7f1a98003900 tx=0x7f1a980047c0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 shutdown_connections 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e6e880 0x561ca1e71dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e687a0 0x561ca1e74580 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1e82010 0x561ca1e84460 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1061973470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e71880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 >> 172.21.15.52:0/1061973470 conn(0x561ca1d95080 msgr2=0x561ca1e62700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1a86ffd700 0 client.21339 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 shutdown_connections 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1061973470 wait complete. 2024-06-02T23:34:31.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.593+0000 7f1ab85e6b80 20 client.21339 trim_cache size 0 max 16384 2024-06-02T23:34:31.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.608+0000 7f1ab85e6b80 1 Processor -- start 2024-06-02T23:34:31.609 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:31 smithi052 ceph-mon[32340]: pgmap v1110: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-06-02T23:34:31.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.608+0000 7f1ab85e6b80 1 -- start start 2024-06-02T23:34:31.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e68260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 0x561ca1e707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e6f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81aebf0 con 0x561ca1e6df60 2024-06-02T23:34:31.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81b0730 con 0x561ca1e687a0 2024-06-02T23:34:31.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81af2c0 con 0x561ca1e6e880 2024-06-02T23:34:31.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 0x561ca1e707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 0x561ca1e707e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40348/0 (socket says 172.21.15.52:40348) 2024-06-02T23:34:31.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/1348292499 learned_addr learned my addr 172.21.15.52:0/1348292499 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:31.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e68260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e6f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/1348292499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 msgr2=0x561ca1e6f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e6f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/1348292499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 msgr2=0x561ca1e68260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e68260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 -- 172.21.15.52:0/1348292499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ca1e6fdf0 con 0x561ca1e6df60 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e68260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.609+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 0x561ca1e707e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1a900cd310 tx=0x7f1a900bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53be46969357e9f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1a9d7fa700 1 -- 172.21.15.52:0/1348292499 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a9009c100 con 0x561ca1e6df60 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1a9d7fa700 1 -- 172.21.15.52:0/1348292499 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a900a04b0 con 0x561ca1e6df60 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1a9d7fa700 1 -- 172.21.15.52:0/1348292499 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a900a0170 con 0x561ca1e6df60 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1348292499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 msgr2=0x561ca1e707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 0x561ca1e707e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f1a900cd310 tx=0x7f1a900bc040 comp rx=0 tx=0).stop 2024-06-02T23:34:31.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1348292499 shutdown_connections 2024-06-02T23:34:31.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6e880 0x561ca1e6f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e68260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1348292499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6df60 0x561ca1e707e0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1348292499 >> 172.21.15.52:0/1348292499 conn(0x561ca1d95080 msgr2=0x561ca1e730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:31.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1348292499 shutdown_connections 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.610+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1348292499 wait complete. 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 -- start start 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 Processor -- start 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e7a700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e7ac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 0x561ca1e89120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81b0060 con 0x561ca1e6e880 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81af990 con 0x561ca1e687a0 2024-06-02T23:34:31.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa81acbb0 con 0x561ca1e6df60 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.611+0000 7f1ab85e6b80 10 client.0 _ll_register_callbacks cb 0x561ca1d4f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 0x561ca1e89120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e7a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 0x561ca1e89120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40354/0 (socket says 172.21.15.52:40354) 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e7a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35246/0 (socket says 172.21.15.52:35246) 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 -- 172.21.15.52:0/1307454345 learned_addr learned my addr 172.21.15.52:0/1307454345 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa6ffd700 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e7ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 -- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 msgr2=0x561ca1e7ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e7ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 -- 172.21.15.52:0/1307454345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 msgr2=0x561ca1e7a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:31.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e7a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 -- 172.21.15.52:0/1307454345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ca1e89960 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1aa7fff700 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 0x561ca1e89120 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009d20 tx=0x7f1aa0014640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f361df7b71663c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1a86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1307454345 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aa0006000 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1307454345 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1aa00149c0 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a900a04b0 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1307454345 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1aa0031070 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.612+0000 7f1ab85e6b80 10 client.21345 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561ca1eec740 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1307454345 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1a900a04b0 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1307454345 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f1aa002acd0 con 0x561ca1e6e880 2024-06-02T23:34:31.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 10 client.21345 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 10 client.21345 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 20 client.21345 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1eecb90 0x561ca1eeefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 4 client.21345 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.613+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561ca1dc12d0 con 0x561ca1eecb90 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.614+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1eecb90 0x561ca1eeefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.614+0000 7f1a86ffd700 10 client.21345 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.614+0000 7f1aa77fe700 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1eecb90 0x561ca1eeefb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4070a87c46c9708a server_cookie=ee1a655a55826208 in_seq=0 out_seq=0 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.614+0000 7f1a86ffd700 1 -- 172.21.15.52:0/1307454345 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561ca1dc12d0 con 0x561ca1eecb90 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.614+0000 7f1a86ffd700 10 client.21345 handle_command_reply: tid=0 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.614+0000 7f1ab85e6b80 1 client.21345 shutdown 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 msgr2=0x561ca1e89120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 0x561ca1e89120 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f1aa0009d20 tx=0x7f1aa0014640 comp rx=0 tx=0).stop 2024-06-02T23:34:31.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 shutdown_connections 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561ca1e6e880 0x561ca1e89120 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561ca1e687a0 0x561ca1e7a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x561ca1eecb90 0x561ca1eeefb0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 --2- 172.21.15.52:0/1307454345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561ca1e6df60 0x561ca1e7ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 >> 172.21.15.52:0/1307454345 conn(0x561ca1d95080 msgr2=0x561ca1e6f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1a86ffd700 0 client.21345 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 shutdown_connections 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 1 -- 172.21.15.52:0/1307454345 wait complete. 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1ab85e6b80 20 client.21345 trim_cache size 0 max 16384 2024-06-02T23:34:31.628 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a940a2ca0 msgr2=0x7f1a940a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.615+0000 7f1aadd63700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a940a2ca0 0x7f1a940a5160 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1a98009c60 tx=0x7f1a98009260 comp rx=0 tx=0).stop 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 msgr2=0x7f1aa8074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8074cd0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1a90005910 tx=0x7f1a900045e0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 shutdown_connections 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1a940a2ca0 0x7f1a940a5160 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.629 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1aa8107860 0x7f1aa8075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1aa8102770 0x7f1aa8074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 --2- 172.21.15.52:0/3212479987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1aa8103120 0x7f1aa8074cd0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 >> 172.21.15.52:0/3212479987 conn(0x7f1aa8079600 msgr2=0x7f1aa81007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 shutdown_connections 2024-06-02T23:34:31.630 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:31.616+0000 7f1aadd63700 1 -- 172.21.15.52:0/3212479987 wait complete. 2024-06-02T23:34:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:31 smithi174 ceph-mon[35994]: pgmap v1110: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-06-02T23:34:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:31 smithi123 ceph-mon[35977]: pgmap v1110: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 808 KiB/s rd, 2.5 MiB/s wr, 211 op/s 2024-06-02T23:34:32.303 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:34:32.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.59 MB/sec execute 460 sec latency 9994.754 ms 2024-06-02T23:34:32.557 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:33.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.58 MB/sec execute 461 sec latency 10994.813 ms 2024-06-02T23:34:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:33 smithi174 ceph-mon[35994]: pgmap v1111: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-06-02T23:34:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:33 smithi052 ceph-mon[32340]: pgmap v1111: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-06-02T23:34:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:33 smithi123 ceph-mon[35977]: pgmap v1111: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.6 MiB/s wr, 154 op/s 2024-06-02T23:34:34.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 -- 172.21.15.52:0/4275241917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70101084f0 msgr2=0x7f70101088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 --2- 172.21.15.52:0/4275241917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70101084f0 0x7f70101088d0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f7000009990 tx=0x7f7000009ca0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 -- 172.21.15.52:0/4275241917 shutdown_connections 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 --2- 172.21.15.52:0/4275241917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701006b3c0 0x7f701010f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 --2- 172.21.15.52:0/4275241917 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f70101084f0 0x7f70101088d0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 --2- 172.21.15.52:0/4275241917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f701006aa00 0x7f701006ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 -- 172.21.15.52:0/4275241917 >> 172.21.15.52:0/4275241917 conn(0x7f7010078300 msgr2=0x7f7010078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 -- 172.21.15.52:0/4275241917 shutdown_connections 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.280+0000 7f70171a9700 1 -- 172.21.15.52:0/4275241917 wait complete. 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 Processor -- start 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 -- start start 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701006aa00 0x7f701019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 0x7f701019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70101084f0 0x7f70101988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112600 con 0x7f701006b3c0 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112480 con 0x7f701006aa00 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f70171a9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7010112780 con 0x7f70101084f0 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.281+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 0x7f701019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 0x7f701019ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46296/0 (socket says 172.21.15.52:46296) 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 -- 172.21.15.52:0/191625631 learned_addr learned my addr 172.21.15.52:0/191625631 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f7014f45700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701006aa00 0x7f701019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f7015746700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70101084f0 0x7f70101988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 -- 172.21.15.52:0/191625631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70101084f0 msgr2=0x7f70101988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70101084f0 0x7f70101988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 -- 172.21.15.52:0/191625631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701006aa00 msgr2=0x7f701019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701006aa00 0x7f701019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 -- 172.21.15.52:0/191625631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7010199020 con 0x7f701006b3c0 2024-06-02T23:34:34.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700ffff700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 0x7f701019ec40 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f700400c920 tx=0x7f700400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:34.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700dffb700 1 -- 172.21.15.52:0/191625631 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004004a30 con 0x7f701006b3c0 2024-06-02T23:34:34.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700dffb700 1 -- 172.21.15.52:0/191625631 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f70040054d0 con 0x7f701006b3c0 2024-06-02T23:34:34.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7010199310 con 0x7f701006b3c0 2024-06-02T23:34:34.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.282+0000 7f700dffb700 1 -- 172.21.15.52:0/191625631 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004007c30 con 0x7f701006b3c0 2024-06-02T23:34:34.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.283+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70101a55d0 con 0x7f701006b3c0 2024-06-02T23:34:34.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.285+0000 7f700dffb700 1 -- 172.21.15.52:0/191625631 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7004004bd0 con 0x7f701006b3c0 2024-06-02T23:34:34.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.285+0000 7f700dffb700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ff80a2ca0 0x7f6ff80a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.285+0000 7f7014f45700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ff80a2ca0 0x7f6ff80a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.285+0000 7f700dffb700 1 -- 172.21.15.52:0/191625631 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f70040ceb30 con 0x7f701006b3c0 2024-06-02T23:34:34.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.286+0000 7f7014f45700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ff80a2ca0 0x7f6ff80a5160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f700000b5c0 tx=0x7f7000005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:34.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.286+0000 7f7021a2cb80 1 Processor -- start 2024-06-02T23:34:34.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.286+0000 7f7021a2cb80 1 -- start start 2024-06-02T23:34:34.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac8470 0x55a368ac8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 0x55a368ac2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 0x55a368acf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a3689b0f60 con 0x55a368ac2670 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a368a0f850 con 0x55a368ac8470 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a368977460 con 0x55a368ac3030 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 0x55a368ac2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 0x55a368acf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 0x55a368ac2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46308/0 (socket says 172.21.15.52:46308) 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 0x55a368acf210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:49270/0 (socket says 172.21.15.52:49270) 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f700ffff700 1 -- 172.21.15.52:0/2753559731 learned_addr learned my addr 172.21.15.52:0/2753559731 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:34.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7014f45700 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac8470 0x55a368ac8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 -- 172.21.15.52:0/2753559731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac8470 msgr2=0x55a368ac8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.290 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac8470 0x55a368ac8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 -- 172.21.15.52:0/2753559731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 msgr2=0x55a368ac2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 0x55a368ac2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 -- 172.21.15.52:0/2753559731 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36895e8d0 con 0x55a368ac3030 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f700ffff700 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 0x55a368ac2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.287+0000 7f7015746700 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 0x55a368acf210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f700800f330 tx=0x7f700800f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1aeb563235231f5d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f6ff6ffd700 1 -- 172.21.15.52:0/2753559731 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008018030 con 0x55a368ac3030 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f6ff6ffd700 1 -- 172.21.15.52:0/2753559731 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008004bb0 con 0x55a368ac3030 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f6ff6ffd700 1 -- 172.21.15.52:0/2753559731 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008005020 con 0x55a368ac3030 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2753559731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 msgr2=0x55a368acf210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 0x55a368acf210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f700800f330 tx=0x7f700800f640 comp rx=0 tx=0).stop 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2753559731 shutdown_connections 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac3030 0x55a368acf210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac8470 0x55a368ac8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2753559731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac2670 0x55a368ac2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2753559731 >> 172.21.15.52:0/2753559731 conn(0x55a3689eb080 msgr2=0x55a368ab9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.288+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2753559731 shutdown_connections 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2753559731 wait complete. 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 -- start start 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 Processor -- start 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368aceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368acc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a3689affc0 con 0x55a368ac3030 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a368a0b810 con 0x55a368ac2670 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a368acf750 con 0x55a368ac8470 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.289+0000 7f7021a2cb80 10 client.0 _ll_register_callbacks cb 0x55a3689a55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368acc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368aceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368acc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46310/0 (socket says 172.21.15.52:46310) 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368aceef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56752/0 (socket says 172.21.15.52:56752) 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f700ffff700 1 -- 172.21.15.52:0/3561086583 learned_addr learned my addr 172.21.15.52:0/3561086583 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7015746700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 -- 172.21.15.52:0/3561086583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 msgr2=0x55a368acc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 -- 172.21.15.52:0/3561086583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 msgr2=0x55a368acc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368acc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 -- 172.21.15.52:0/3561086583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a3689d5750 con 0x55a368ac2670 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f700ffff700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368acc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7014f45700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368aceef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f70000059d0 tx=0x7f700001a050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69c21267e1863eca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:34.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f6feffff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:34:34.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f6feffff700 1 -- 172.21.15.52:0/3561086583 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7000024820 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008004bb0 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.290+0000 7f6feffff700 1 -- 172.21.15.52:0/3561086583 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f70000249c0 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f6feffff700 1 -- 172.21.15.52:0/3561086583 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7000024c00 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 10 client.30611 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a368ad9e60 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f6feffff700 1 -- 172.21.15.52:0/3561086583 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008004bb0 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f6feffff700 1 -- 172.21.15.52:0/3561086583 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f7000030400 con 0x55a368ac2670 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 10 client.30611 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 10 client.30611 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 20 client.30611 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368adc4a0 0x55a368ade920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 4 client.30611 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a368a1cd50 con 0x55a368adc4a0 2024-06-02T23:34:34.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.291+0000 7f700ffff700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368adc4a0 0x55a368ade920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.292+0000 7f700ffff700 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368adc4a0 0x55a368ade920 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=34c74b8c3d45758d server_cookie=fd15566f936a221d in_seq=0 out_seq=0 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.292+0000 7f6feffff700 10 client.30611 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f6feffff700 1 -- 172.21.15.52:0/3561086583 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a368a1cd50 con 0x55a368adc4a0 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f6feffff700 10 client.30611 handle_command_reply: tid=0 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 client.30611 shutdown 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 msgr2=0x55a368aceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368aceef0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f70000059d0 tx=0x7f700001a050 comp rx=0 tx=0).stop 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 shutdown_connections 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368adc4a0 0x55a368ade920 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368aceef0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3561086583 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368acc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f6feffff700 0 client.30611 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:34:34.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 >> 172.21.15.52:0/3561086583 conn(0x55a3689eb080 msgr2=0x55a368ab8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 shutdown_connections 2024-06-02T23:34:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3561086583 wait complete. 2024-06-02T23:34:34.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.293+0000 7f7021a2cb80 20 client.30611 trim_cache size 0 max 16384 2024-06-02T23:34:34.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 Processor -- start 2024-06-02T23:34:34.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 -- start start 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac2670 0x55a368ace130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac3030 0x55a368ab8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 0x55a368ab93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70101aa750 con 0x55a368ac8470 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70101ac290 con 0x55a368ac3030 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70101aae20 con 0x55a368ac2670 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7015746700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 0x55a368ab93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7015746700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 0x55a368ab93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46320/0 (socket says 172.21.15.52:46320) 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f7015746700 1 -- 172.21.15.52:0/2188163198 learned_addr learned my addr 172.21.15.52:0/2188163198 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.308+0000 7f700ffff700 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac3030 0x55a368ab8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7014f45700 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac2670 0x55a368ace130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7015746700 1 -- 172.21.15.52:0/2188163198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac2670 msgr2=0x55a368ace130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7015746700 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac2670 0x55a368ace130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7015746700 1 -- 172.21.15.52:0/2188163198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac3030 msgr2=0x55a368ab8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7015746700 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac3030 0x55a368ab8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7015746700 1 -- 172.21.15.52:0/2188163198 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a368aca780 con 0x55a368ac8470 2024-06-02T23:34:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f7015746700 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 0x55a368ab93c0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7008007c80 tx=0x7f7008007b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59b1bd8be1a9a6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f6ff67fc700 1 -- 172.21.15.52:0/2188163198 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008014470 con 0x55a368ac8470 2024-06-02T23:34:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f6ff67fc700 1 -- 172.21.15.52:0/2188163198 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008014ac0 con 0x55a368ac8470 2024-06-02T23:34:34.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.309+0000 7f6ff67fc700 1 -- 172.21.15.52:0/2188163198 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7008025400 con 0x55a368ac8470 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2188163198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 msgr2=0x55a368ab93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 0x55a368ab93c0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f7008007c80 tx=0x7f7008007b20 comp rx=0 tx=0).stop 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2188163198 shutdown_connections 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac8470 0x55a368ab93c0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac2670 0x55a368ace130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/2188163198 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac3030 0x55a368ab8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2188163198 >> 172.21.15.52:0/2188163198 conn(0x55a3689eb080 msgr2=0x55a368acda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2188163198 shutdown_connections 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 -- 172.21.15.52:0/2188163198 wait complete. 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 -- start start 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.310+0000 7f7021a2cb80 1 Processor -- start 2024-06-02T23:34:34.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368ad4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368ad50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f70101abbc0 con 0x55a368ac3030 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f70101ab4f0 con 0x55a368ac2670 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f70101a86f0 con 0x55a368ac8470 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 10 client.0 _ll_register_callbacks cb 0x55a3689a55b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368ad50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7014f45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368ad4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368ad50f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46334/0 (socket says 172.21.15.52:46334) 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 -- 172.21.15.52:0/3712770830 learned_addr learned my addr 172.21.15.52:0/3712770830 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7015746700 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 -- 172.21.15.52:0/3712770830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 msgr2=0x55a368acffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 -- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 msgr2=0x55a368ad4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368ad4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 -- 172.21.15.52:0/3712770830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a368ad07e0 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f700ffff700 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368ad50f0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7004000c00 tx=0x7f7004004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b326d1d8a6bfd5a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f6feffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f6feffff700 1 -- 172.21.15.52:0/3712770830 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70040b3020 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7008014ac0 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.311+0000 7f6feffff700 1 -- 172.21.15.52:0/3712770830 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7004005680 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f6feffff700 1 -- 172.21.15.52:0/3712770830 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f700401d030 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 10 client.21363 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a368b46e20 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f6feffff700 1 -- 172.21.15.52:0/3712770830 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7008014ac0 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f6feffff700 1 -- 172.21.15.52:0/3712770830 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f70040a3ba0 con 0x55a368ac3030 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 10 client.21363 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 10 client.21363 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 20 client.21363 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:34.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368b474e0 0x55a368b49900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 4 client.21363 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a368a172d0 con 0x55a368b474e0 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.312+0000 7f7014f45700 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368b474e0 0x55a368b49900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7014f45700 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368b474e0 0x55a368b49900 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=94426010d25ca72 server_cookie=7a66e6b309ce0e8a in_seq=0 out_seq=0 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f6feffff700 10 client.21363 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f6feffff700 1 -- 172.21.15.52:0/3712770830 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a368a172d0 con 0x55a368b474e0 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f6feffff700 10 client.21363 handle_command_reply: tid=0 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 client.21363 shutdown 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 msgr2=0x55a368ad50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368ad50f0 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f7004000c00 tx=0x7f7004004f10 comp rx=0 tx=0).stop 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 shutdown_connections 2024-06-02T23:34:34.318 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x55a368b474e0 0x55a368b49900 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a368ac8470 0x55a368acffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a368ac2670 0x55a368ad4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f6feffff700 0 client.21363 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 --2- 172.21.15.52:0/3712770830 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a368ac3030 0x55a368ad50f0 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 >> 172.21.15.52:0/3712770830 conn(0x55a3689eb080 msgr2=0x55a368abdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 shutdown_connections 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 1 -- 172.21.15.52:0/3712770830 wait complete. 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.313+0000 7f7021a2cb80 20 client.21363 trim_cache size 0 max 16384 2024-06-02T23:34:34.319 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ff80a2ca0 msgr2=0x7f6ff80a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ff80a2ca0 0x7f6ff80a5160 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f700000b5c0 tx=0x7f7000005960 comp rx=0 tx=0).stop 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 msgr2=0x7f701019ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 0x7f701019ec40 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f700400c920 tx=0x7f700400cc30 comp rx=0 tx=0).stop 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 shutdown_connections 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f6ff80a2ca0 0x7f6ff80a5160 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70101084f0 0x7f70101988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701006aa00 0x7f701019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 --2- 172.21.15.52:0/191625631 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f701006b3c0 0x7f701019ec40 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:34.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 >> 172.21.15.52:0/191625631 conn(0x7f7010078300 msgr2=0x7f70100fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:34.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 shutdown_connections 2024-06-02T23:34:34.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:34.314+0000 7f70171a9700 1 -- 172.21.15.52:0/191625631 wait complete. 2024-06-02T23:34:34.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.56 MB/sec execute 462 sec latency 11994.914 ms 2024-06-02T23:34:35.036 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:34:35.289 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:35.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.55 MB/sec execute 463 sec latency 12994.961 ms 2024-06-02T23:34:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:35 smithi174 ceph-mon[35994]: pgmap v1112: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 940 KiB/s wr, 111 op/s 2024-06-02T23:34:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:35 smithi052 ceph-mon[32340]: pgmap v1112: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 940 KiB/s wr, 111 op/s 2024-06-02T23:34:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:35 smithi123 ceph-mon[35977]: pgmap v1112: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 940 KiB/s wr, 111 op/s 2024-06-02T23:34:36.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.54 MB/sec execute 464 sec latency 13994.998 ms 2024-06-02T23:34:36.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 -- 172.21.15.52:0/1715926249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101380 msgr2=0x7fa94c101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:36.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1715926249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101380 0x7fa94c101800 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7fa940009990 tx=0x7fa940009ca0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 -- 172.21.15.52:0/1715926249 shutdown_connections 2024-06-02T23:34:36.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1715926249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa94c101d40 0x7fa94c10e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1715926249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa94c1073f0 0x7fa94c1077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1715926249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101380 0x7fa94c101800 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 -- 172.21.15.52:0/1715926249 >> 172.21.15.52:0/1715926249 conn(0x7fa94c078300 msgr2=0x7fa94c078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:36.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 -- 172.21.15.52:0/1715926249 shutdown_connections 2024-06-02T23:34:36.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.983+0000 7fa951b3a700 1 -- 172.21.15.52:0/1715926249 wait complete. 2024-06-02T23:34:36.985 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.984+0000 7fa951b3a700 1 Processor -- start 2024-06-02T23:34:36.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.984+0000 7fa951b3a700 1 -- start start 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.984+0000 7fa951b3a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa94c101380 0x7fa94c103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.984+0000 7fa951b3a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 0x7fa94c1035a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa951b3a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa94c1073f0 0x7fa94c103ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa951b3a700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c111510 con 0x7fa94c101d40 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa951b3a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c111390 con 0x7fa94c1073f0 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa951b3a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c111690 con 0x7fa94c101380 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 0x7fa94c1035a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 0x7fa94c1035a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46344/0 (socket says 172.21.15.52:46344) 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 -- 172.21.15.52:0/1769558043 learned_addr learned my addr 172.21.15.52:0/1769558043 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa94c101380 0x7fa94c103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa94c1073f0 0x7fa94c103ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 -- 172.21.15.52:0/1769558043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa94c101380 msgr2=0x7fa94c103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:36.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa94c101380 0x7fa94c103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 -- 172.21.15.52:0/1769558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa94c1073f0 msgr2=0x7fa94c103ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa94c1073f0 0x7fa94c103ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 -- 172.21.15.52:0/1769558043 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa94c1a6fb0 con 0x7fa94c101d40 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa94affd700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 0x7fa94c1035a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa940000c00 tx=0x7fa940003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa948ff9700 1 -- 172.21.15.52:0/1769558043 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa940027070 con 0x7fa94c101d40 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa94c1a7240 con 0x7fa94c101d40 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.985+0000 7fa948ff9700 1 -- 172.21.15.52:0/1769558043 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa94000bdc0 con 0x7fa94c101d40 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.986+0000 7fa948ff9700 1 -- 172.21.15.52:0/1769558043 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa94000f460 con 0x7fa94c101d40 2024-06-02T23:34:36.988 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.986+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa94c1a76f0 con 0x7fa94c101d40 2024-06-02T23:34:36.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.988+0000 7fa948ff9700 1 -- 172.21.15.52:0/1769558043 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa94002a070 con 0x7fa94c101d40 2024-06-02T23:34:36.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.988+0000 7fa948ff9700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9340a2c20 0x7fa9340a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.989+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9340a2c20 0x7fa9340a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.989 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.989+0000 7fa948ff9700 1 -- 172.21.15.52:0/1769558043 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa9400cffb0 con 0x7fa94c101d40 2024-06-02T23:34:36.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.989+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9340a2c20 0x7fa9340a50e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00abd0 tx=0x7fa93c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:36.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.989+0000 7fa95c3bdb80 1 Processor -- start 2024-06-02T23:34:36.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.989+0000 7fa95c3bdb80 1 -- start start 2024-06-02T23:34:36.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x557046987a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.991 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469819e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046981f20 0x557046982340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570468d3810 con 0x557046987640 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55704687b720 con 0x557046981f20 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570468cf7d0 con 0x557046987f60 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x557046987a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x557046987a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46358/0 (socket says 172.21.15.52:46358) 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1541991504 learned_addr learned my addr 172.21.15.52:0/1541991504 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94affd700 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046981f20 0x557046982340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1541991504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 msgr2=0x5570469819e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469819e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1541991504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046981f20 msgr2=0x557046982340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:36.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046981f20 0x557046982340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1541991504 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557046871f50 con 0x557046987640 2024-06-02T23:34:36.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x557046987a20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa93c004530 tx=0x7fa93c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc0cc2ef8387853 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:36.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.990+0000 7fa931ffb700 1 -- 172.21.15.52:0/1541991504 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c016070 con 0x557046987640 2024-06-02T23:34:36.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa931ffb700 1 -- 172.21.15.52:0/1541991504 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa93c014070 con 0x557046987640 2024-06-02T23:34:36.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa931ffb700 1 -- 172.21.15.52:0/1541991504 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c01d400 con 0x557046987640 2024-06-02T23:34:36.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1541991504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 msgr2=0x557046987a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x557046987a20 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fa93c004530 tx=0x7fa93c012000 comp rx=0 tx=0).stop 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1541991504 shutdown_connections 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046981f20 0x557046982340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x557046987a20 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1541991504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469819e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1541991504 >> 172.21.15.52:0/1541991504 conn(0x5570468accc0 msgr2=0x55704697c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1541991504 shutdown_connections 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.991+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1541991504 wait complete. 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 -- start start 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 Processor -- start 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x55704698dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x55704698afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x55704698b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55704698e9d0 con 0x557046987640 2024-06-02T23:34:36.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557046873bf0 con 0x557046987f60 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55704698eb50 con 0x557046981f20 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.992+0000 7fa95c3bdb80 10 client.0 _ll_register_callbacks cb 0x55704686d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x55704698afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x55704698afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46364/0 (socket says 172.21.15.52:46364) 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 -- 172.21.15.52:0/1386996651 learned_addr learned my addr 172.21.15.52:0/1386996651 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x55704698b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 -- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 msgr2=0x55704698dc50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x55704698dc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 -- 172.21.15.52:0/1386996651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 msgr2=0x55704698b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x55704698b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 -- 172.21.15.52:0/1386996651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570468228d0 con 0x557046987640 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x55704698b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:36.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa94affd700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x55704698afa0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa9400098e0 tx=0x7fa9400228c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96f1a1cf47a85adb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa92affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa92affd700 1 -- 172.21.15.52:0/1386996651 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400b5060 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa92affd700 1 -- 172.21.15.52:0/1386996651 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa94009cbd0 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa93c014070 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa92affd700 1 -- 172.21.15.52:0/1386996651 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa94009d070 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa95c3bdb80 10 client.21375 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.993+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557046999290 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa92affd700 1 -- 172.21.15.52:0/1386996651 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa93c014070 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa92affd700 1 -- 172.21.15.52:0/1386996651 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa9400bb170 con 0x557046987640 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa95c3bdb80 10 client.21375 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa95c3bdb80 10 client.21375 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:34:36.998 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa95c3bdb80 20 client.21375 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:36.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x55704699db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:36.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa95c3bdb80 4 client.21375 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:36.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.994+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570468e0d10 con 0x55704699b760 2024-06-02T23:34:36.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.995+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x55704699db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.995+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x55704699db80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45bab39031dbe5bf server_cookie=1d50070aecbd09e6 in_seq=0 out_seq=0 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.995+0000 7fa92affd700 10 client.21375 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.996+0000 7fa92affd700 1 -- 172.21.15.52:0/1386996651 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570468e0d10 con 0x55704699b760 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.996+0000 7fa92affd700 10 client.21375 handle_command_reply: tid=0 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.996+0000 7fa95c3bdb80 1 client.21375 shutdown 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.996+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 msgr2=0x55704698afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:37.000 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.996+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x55704698afa0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fa9400098e0 tx=0x7fa9400228c0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 shutdown_connections 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x55704699db80 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x55704698b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa92affd700 0 client.21375 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x55704698dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1386996651 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x55704698afa0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 >> 172.21.15.52:0/1386996651 conn(0x5570468accc0 msgr2=0x55704697c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 shutdown_connections 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1386996651 wait complete. 2024-06-02T23:34:37.001 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:36.997+0000 7fa95c3bdb80 20 client.21375 trim_cache size 0 max 16384 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 Processor -- start 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 -- start start 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 0x5570469819e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987640 0x557046988a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469a3450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1aeb00 con 0x557046981f20 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1b0640 con 0x557046987640 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1af1d0 con 0x557046987f60 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 0x5570469819e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.014 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa94b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 0x5570469819e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46370/0 (socket says 172.21.15.52:46370) 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.012+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1827672689 learned_addr learned my addr 172.21.15.52:0/1827672689 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94affd700 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987640 0x557046988a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469a3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1827672689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 msgr2=0x5570469a3450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469a3450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1827672689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987640 msgr2=0x557046988a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987640 0x557046988a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94b7fe700 1 -- 172.21.15.52:0/1827672689 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557046988f90 con 0x557046981f20 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 0x5570469819e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa93c010ad0 tx=0x7fa93c005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7e82f67997643a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:37.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa9317fa700 1 -- 172.21.15.52:0/1827672689 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c016040 con 0x557046981f20 2024-06-02T23:34:37.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa9317fa700 1 -- 172.21.15.52:0/1827672689 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa93c014040 con 0x557046981f20 2024-06-02T23:34:37.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.013+0000 7fa9317fa700 1 -- 172.21.15.52:0/1827672689 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c01d400 con 0x557046981f20 2024-06-02T23:34:37.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1827672689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 msgr2=0x5570469819e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:37.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 0x5570469819e0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7fa93c010ad0 tx=0x7fa93c005c70 comp rx=0 tx=0).stop 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1827672689 shutdown_connections 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046987f60 0x5570469a3450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046981f20 0x5570469819e0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1827672689 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987640 0x557046988a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1827672689 >> 172.21.15.52:0/1827672689 conn(0x5570468accc0 msgr2=0x557046988340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1827672689 shutdown_connections 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1827672689 wait complete. 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 -- start start 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.014+0000 7fa95c3bdb80 1 Processor -- start 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x5570469944a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x5570469a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.020 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x5570469a3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1aff70 con 0x557046987640 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1af8a0 con 0x557046987f60 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa94c1acac0 con 0x557046981f20 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa95c3bdb80 10 client.0 _ll_register_callbacks cb 0x55704686d920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x5570469a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x5570469a2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46378/0 (socket says 172.21.15.52:46378) 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 -- 172.21.15.52:0/1513606463 learned_addr learned my addr 172.21.15.52:0/1513606463 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94bfff700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x5570469a3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x5570469944a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 -- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 msgr2=0x5570469944a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x5570469944a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.021 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 -- 172.21.15.52:0/1513606463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 msgr2=0x5570469a3190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x5570469a3190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 -- 172.21.15.52:0/1513606463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570469a3840 con 0x557046987640 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa94affd700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x5570469a2c50 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fa940098dd0 tx=0x7fa9400874b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9a30d2df8fcfb0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa92affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.015+0000 7fa92affd700 1 -- 172.21.15.52:0/1513606463 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9400a5250 con 0x557046987640 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa92affd700 1 -- 172.21.15.52:0/1513606463 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa94009cde0 con 0x557046987640 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa93c014040 con 0x557046987640 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa92affd700 1 -- 172.21.15.52:0/1513606463 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa9400a0d20 con 0x557046987640 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 10 client.21381 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x557046a062c0 con 0x557046987640 2024-06-02T23:34:37.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa92affd700 1 -- 172.21.15.52:0/1513606463 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa93c014040 con 0x557046987640 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa92affd700 1 -- 172.21.15.52:0/1513606463 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa9400986c0 con 0x557046987640 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 10 client.21381 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 10 client.21381 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 20 client.21381 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x557046a087f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 4 client.21381 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570468db290 con 0x55704699b760 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.016+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x557046a087f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa94b7fe700 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x557046a087f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3cc316a3e4fb3908 server_cookie=4cf048053250b5f6 in_seq=0 out_seq=0 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa92affd700 10 client.21381 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa92affd700 1 -- 172.21.15.52:0/1513606463 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570468db290 con 0x55704699b760 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa92affd700 10 client.21381 handle_command_reply: tid=0 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 client.21381 shutdown 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 msgr2=0x5570469a2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:37.023 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x5570469a2c50 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7fa940098dd0 tx=0x7fa9400874b0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 shutdown_connections 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55704699b760 0x557046a087f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557046987f60 0x5570469a3190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557046981f20 0x5570469944a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 --2- 172.21.15.52:0/1513606463 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x557046987640 0x5570469a2c50 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa92affd700 0 client.21381 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 >> 172.21.15.52:0/1513606463 conn(0x5570468accc0 msgr2=0x557046975e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 shutdown_connections 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 1 -- 172.21.15.52:0/1513606463 wait complete. 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.017+0000 7fa95c3bdb80 20 client.21381 trim_cache size 0 max 16384 2024-06-02T23:34:37.024 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9340a2c20 msgr2=0x7fa9340a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9340a2c20 0x7fa9340a50e0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00abd0 tx=0x7fa93c00a380 comp rx=0 tx=0).stop 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 msgr2=0x7fa94c1035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 0x7fa94c1035a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7fa940000c00 tx=0x7fa940003a80 comp rx=0 tx=0).stop 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 shutdown_connections 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9340a2c20 0x7fa9340a50e0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.025 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa94c1073f0 0x7fa94c103ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa94c101380 0x7fa94c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 --2- 172.21.15.52:0/1769558043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa94c101d40 0x7fa94c1035a0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 >> 172.21.15.52:0/1769558043 conn(0x7fa94c078300 msgr2=0x7fa94c1005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 shutdown_connections 2024-06-02T23:34:37.026 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:37.018+0000 7fa951b3a700 1 -- 172.21.15.52:0/1769558043 wait complete. 2024-06-02T23:34:37.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.52 MB/sec execute 465 sec latency 14995.077 ms 2024-06-02T23:34:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:37 smithi174 ceph-mon[35994]: pgmap v1113: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 813 KiB/s wr, 93 op/s 2024-06-02T23:34:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:37 smithi052 ceph-mon[32340]: pgmap v1113: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 813 KiB/s wr, 93 op/s 2024-06-02T23:34:37.749 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:34:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:37 smithi123 ceph-mon[35977]: pgmap v1113: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 813 KiB/s wr, 93 op/s 2024-06-02T23:34:38.000 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 -- 172.21.15.52:0/2703270682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 msgr2=0x7f5b6c103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2703270682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c103070 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c009900 tx=0x7f5b5c02a410 comp rx=0 tx=0).stop 2024-06-02T23:34:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 -- 172.21.15.52:0/2703270682 shutdown_connections 2024-06-02T23:34:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2703270682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 0x7f5b6c109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2703270682 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c103070 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.102 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2703270682 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b6c101a00 0x7f5b6c101e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 -- 172.21.15.52:0/2703270682 >> 172.21.15.52:0/2703270682 conn(0x7f5b6c0fcf90 msgr2=0x7f5b6c0ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 -- 172.21.15.52:0/2703270682 shutdown_connections 2024-06-02T23:34:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.097+0000 7f5b73a31640 1 -- 172.21.15.52:0/2703270682 wait complete. 2024-06-02T23:34:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 Processor -- start 2024-06-02T23:34:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 -- start start 2024-06-02T23:34:38.103 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b6c101a00 0x7f5b6c197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 0x7f5b6c19f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10d530 con 0x7f5b6c102bf0 2024-06-02T23:34:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10d3b0 con 0x7f5b6c1035b0 2024-06-02T23:34:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b73a31640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c10d6b0 con 0x7f5b6c101a00 2024-06-02T23:34:38.104 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46388/0 (socket says 172.21.15.52:46388) 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 -- 172.21.15.52:0/2355669285 learned_addr learned my addr 172.21.15.52:0/2355669285 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b717cd640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b6c101a00 0x7f5b6c197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b71fce640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 0x7f5b6c19f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 -- 172.21.15.52:0/2355669285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b6c101a00 msgr2=0x7f5b6c197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b6c101a00 0x7f5b6c197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.105 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 -- 172.21.15.52:0/2355669285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 msgr2=0x7f5b6c19f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 0x7f5b6c19f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b70fcc640 1 -- 172.21.15.52:0/2355669285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b6c19f700 con 0x7f5b6c102bf0 2024-06-02T23:34:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.098+0000 7f5b71fce640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 0x7f5b6c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.099+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c198020 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c004770 tx=0x7f5b5c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.099+0000 7f5b627fc640 1 -- 172.21.15.52:0/2355669285 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c03f030 con 0x7f5b6c102bf0 2024-06-02T23:34:38.106 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.099+0000 7f5b627fc640 1 -- 172.21.15.52:0/2355669285 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b5c003cb0 con 0x7f5b6c102bf0 2024-06-02T23:34:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.099+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b6c19f990 con 0x7f5b6c102bf0 2024-06-02T23:34:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.099+0000 7f5b627fc640 1 -- 172.21.15.52:0/2355669285 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c035070 con 0x7f5b6c102bf0 2024-06-02T23:34:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.099+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6c19fe80 con 0x7f5b6c102bf0 2024-06-02T23:34:38.107 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.101+0000 7f5b627fc640 1 -- 172.21.15.52:0/2355669285 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b5c042030 con 0x7f5b6c102bf0 2024-06-02T23:34:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 Processor -- start 2024-06-02T23:34:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 -- start start 2024-06-02T23:34:38.120 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055437840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e05543c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e05543f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e0553964f0 con 0x55e055437420 2024-06-02T23:34:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e055334870 con 0x55e05543cf80 2024-06-02T23:34:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b76818740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e0553924b0 con 0x55e055437e10 2024-06-02T23:34:38.121 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b627fc640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b580a2c30 0x7f5b580a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055437840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055437840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46396/0 (socket says 172.21.15.52:46396) 2024-06-02T23:34:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 -- 172.21.15.52:0/1064556826 learned_addr learned my addr 172.21.15.52:0/1064556826 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b580a2c30 0x7f5b580a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.122 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e05543f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b717cd640 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e05543c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b627fc640 1 -- 172.21.15.52:0/2355669285 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b5c0efab0 con 0x7f5b6c102bf0 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 -- 172.21.15.52:0/1064556826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 msgr2=0x55e05543c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e05543c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 -- 172.21.15.52:0/1064556826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 msgr2=0x55e05543f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e05543f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.123 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.102+0000 7f5b71fce640 1 -- 172.21.15.52:0/1064556826 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0553509b0 con 0x55e055437420 2024-06-02T23:34:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.103+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e05543f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.103+0000 7f5b71fce640 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055437840 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f5b64012e20 tx=0x7f5b6400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dab7f45b5b3aa43 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.103+0000 7f5b71fce640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b580a2c30 0x7f5b580a50f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400afd0 tx=0x7f5b6401a000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.103+0000 7f5b577fe640 1 -- 172.21.15.52:0/1064556826 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b64020070 con 0x55e055437420 2024-06-02T23:34:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.103+0000 7f5b577fe640 1 -- 172.21.15.52:0/1064556826 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b6400ac00 con 0x55e055437420 2024-06-02T23:34:38.124 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.103+0000 7f5b577fe640 1 -- 172.21.15.52:0/1064556826 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6401cbd0 con 0x55e055437420 2024-06-02T23:34:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 -- 172.21.15.52:0/1064556826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 msgr2=0x55e055437840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055437840 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f5b64012e20 tx=0x7f5b6400f730 comp rx=0 tx=0).stop 2024-06-02T23:34:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 -- 172.21.15.52:0/1064556826 shutdown_connections 2024-06-02T23:34:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e05543c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.125 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e05543f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 --2- 172.21.15.52:0/1064556826 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055437840 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 -- 172.21.15.52:0/1064556826 >> 172.21.15.52:0/1064556826 conn(0x55e05542e680 msgr2=0x55e05542ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 -- 172.21.15.52:0/1064556826 shutdown_connections 2024-06-02T23:34:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 -- 172.21.15.52:0/1064556826 wait complete. 2024-06-02T23:34:38.126 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 -- start start 2024-06-02T23:34:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.104+0000 7f5b76818740 1 Processor -- start 2024-06-02T23:34:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e0554416c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e055437e10 0x55e055441c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05543cf80 0x55e055448c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.127 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e05543fed0 con 0x55e055437420 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e05531fae0 con 0x55e055437e10 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e055440050 con 0x55e05543cf80 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 10 client.0 _ll_register_callbacks cb 0x55e0553589b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e0554416c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e0554416c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46398/0 (socket says 172.21.15.52:46398) 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 -- 172.21.15.52:0/2037100214 learned_addr learned my addr 172.21.15.52:0/2037100214 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:38.128 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e055437e10 0x55e055441c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b71fce640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05543cf80 0x55e055448c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 -- 172.21.15.52:0/2037100214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05543cf80 msgr2=0x55e055448c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05543cf80 0x55e055448c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 -- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e055437e10 msgr2=0x55e055441c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e055437e10 0x55e055441c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.129 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 -- 172.21.15.52:0/2037100214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0550aa210 con 0x55e055437420 2024-06-02T23:34:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e055437e10 0x55e055441c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b717cd640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e0554416c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800be40 tx=0x7f5b68012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a8021773ae2d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b54ff9640 1 -- 172.21.15.52:0/2037100214 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68004230 con 0x55e055437420 2024-06-02T23:34:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b54ff9640 1 -- 172.21.15.52:0/2037100214 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b680043d0 con 0x55e055437420 2024-06-02T23:34:38.130 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.105+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6400ac00 con 0x55e055437420 2024-06-02T23:34:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b54ff9640 1 -- 172.21.15.52:0/2037100214 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5b680106a0 con 0x55e055437420 2024-06-02T23:34:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 10 client.21393 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e055176620 con 0x55e055437420 2024-06-02T23:34:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b54ff9640 1 -- 172.21.15.52:0/2037100214 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b6400ac00 con 0x55e055437420 2024-06-02T23:34:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b54ff9640 1 -- 172.21.15.52:0/2037100214 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5b68003aa0 con 0x55e055437420 2024-06-02T23:34:38.131 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 10 client.21393 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 10 client.21393 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 20 client.21393 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e05544d5a0 0x55e05544fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 4 client.21393 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0553a9f10 con 0x55e05544d5a0 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.106+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e05544d5a0 0x55e05544fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.132 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.107+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e05544d5a0 0x55e05544fa10 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e49b3d69855079a0 server_cookie=d87851ae2497567a in_seq=0 out_seq=0 2024-06-02T23:34:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.107+0000 7f5b54ff9640 10 client.21393 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.107+0000 7f5b54ff9640 1 -- 172.21.15.52:0/2037100214 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e0553a9f10 con 0x55e05544d5a0 2024-06-02T23:34:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.107+0000 7f5b54ff9640 10 client.21393 handle_command_reply: tid=0 2024-06-02T23:34:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 client.21393 shutdown 2024-06-02T23:34:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 msgr2=0x55e0554416c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.133 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e0554416c0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800be40 tx=0x7f5b68012820 comp rx=0 tx=0).stop 2024-06-02T23:34:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 shutdown_connections 2024-06-02T23:34:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e05544d5a0 0x55e05544fa10 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e05543cf80 0x55e055448c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e055437e10 0x55e055441c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b54ff9640 0 client.21393 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:38.134 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 --2- 172.21.15.52:0/2037100214 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e0554416c0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 >> 172.21.15.52:0/2037100214 conn(0x55e05542e680 msgr2=0x55e05542d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 shutdown_connections 2024-06-02T23:34:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 1 -- 172.21.15.52:0/2037100214 wait complete. 2024-06-02T23:34:38.135 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.108+0000 7f5b76818740 20 client.21393 trim_cache size 0 max 16384 2024-06-02T23:34:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 Processor -- start 2024-06-02T23:34:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 -- start start 2024-06-02T23:34:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055440990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e055448c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.144 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055440ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1a6950 con 0x55e055437420 2024-06-02T23:34:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1a8490 con 0x55e05543cf80 2024-06-02T23:34:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b76818740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1a7020 con 0x55e055437e10 2024-06-02T23:34:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055440990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.145 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055440990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46402/0 (socket says 172.21.15.52:46402) 2024-06-02T23:34:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 -- 172.21.15.52:0/496890344 learned_addr learned my addr 172.21.15.52:0/496890344 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e055448c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 -- 172.21.15.52:0/496890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 msgr2=0x55e055448c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e055448c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 -- 172.21.15.52:0/496890344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 msgr2=0x55e055440ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.146 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b71fce640 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055440ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055440ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b717cd640 1 -- 172.21.15.52:0/496890344 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e05542f350 con 0x55e055437420 2024-06-02T23:34:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.124+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e055448c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b717cd640 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055440990 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800be10 tx=0x7f5b68005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f728db6a5a1abbf2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b56ffd640 1 -- 172.21.15.52:0/496890344 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68004000 con 0x55e055437420 2024-06-02T23:34:38.147 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b56ffd640 1 -- 172.21.15.52:0/496890344 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b6801b970 con 0x55e055437420 2024-06-02T23:34:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b56ffd640 1 -- 172.21.15.52:0/496890344 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b68010c10 con 0x55e055437420 2024-06-02T23:34:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 -- 172.21.15.52:0/496890344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 msgr2=0x55e055440990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055440990 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f5b6800be10 tx=0x7f5b68005710 comp rx=0 tx=0).stop 2024-06-02T23:34:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 -- 172.21.15.52:0/496890344 shutdown_connections 2024-06-02T23:34:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437e10 0x55e055448c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.148 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055440ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 --2- 172.21.15.52:0/496890344 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437420 0x55e055440990 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 -- 172.21.15.52:0/496890344 >> 172.21.15.52:0/496890344 conn(0x55e05542e680 msgr2=0x55e055431560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.125+0000 7f5b76818740 1 -- 172.21.15.52:0/496890344 shutdown_connections 2024-06-02T23:34:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 -- 172.21.15.52:0/496890344 wait complete. 2024-06-02T23:34:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 -- start start 2024-06-02T23:34:38.149 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 Processor -- start 2024-06-02T23:34:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437420 0x55e055445ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 0x55e055446430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055447440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1a7dc0 con 0x55e055437e10 2024-06-02T23:34:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1a76f0 con 0x55e05543cf80 2024-06-02T23:34:38.150 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6c1a4930 con 0x55e055437420 2024-06-02T23:34:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b76818740 10 client.0 _ll_register_callbacks cb 0x7f5b6800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b70fcc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 0x55e055446430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b70fcc640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 0x55e055446430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46418/0 (socket says 172.21.15.52:46418) 2024-06-02T23:34:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.126+0000 7f5b70fcc640 1 -- 172.21.15.52:0/1970676329 learned_addr learned my addr 172.21.15.52:0/1970676329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b71fce640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055447440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.151 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b717cd640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437420 0x55e055445ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b70fcc640 1 -- 172.21.15.52:0/1970676329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437420 msgr2=0x55e055445ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437420 0x55e055445ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b70fcc640 1 -- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 msgr2=0x55e055447440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055447440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b70fcc640 1 -- 172.21.15.52:0/1970676329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e055447a30 con 0x55e055437e10 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b717cd640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437420 0x55e055445ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b70fcc640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 0x55e055446430 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005940 tx=0x7f5b5c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1240b82c64f456bd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:38.152 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b54ff9640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b54ff9640 1 -- 172.21.15.52:0/1970676329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b5c049040 con 0x55e055437e10 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b54ff9640 1 -- 172.21.15.52:0/1970676329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5b5c0ed690 con 0x55e055437e10 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b6801b970 con 0x55e055437e10 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b54ff9640 1 -- 172.21.15.52:0/1970676329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5b5c0c4f50 con 0x55e055437e10 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 10 client.21399 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e055448d80 con 0x55e055437e10 2024-06-02T23:34:38.153 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b54ff9640 1 -- 172.21.15.52:0/1970676329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b6801b970 con 0x55e055437e10 2024-06-02T23:34:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b54ff9640 1 -- 172.21.15.52:0/1970676329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5b5c03b460 con 0x55e055437e10 2024-06-02T23:34:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 10 client.21399 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 10 client.21399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 20 client.21399 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e0554b87f0 0x55e0554bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:38.154 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 4 client.21399 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:34:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.127+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e0553a3c20 con 0x55e0554b87f0 2024-06-02T23:34:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.128+0000 7f5b717cd640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e0554b87f0 0x55e0554bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.128+0000 7f5b54ff9640 10 client.21399 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:38.155 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.128+0000 7f5b717cd640 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e0554b87f0 0x55e0554bac10 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fbba33688c252a0d server_cookie=efc338bdb47b2759 in_seq=0 out_seq=0 2024-06-02T23:34:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b54ff9640 1 -- 172.21.15.52:0/1970676329 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e0553a3c20 con 0x55e0554b87f0 2024-06-02T23:34:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b54ff9640 10 client.21399 handle_command_reply: tid=0 2024-06-02T23:34:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 client.21399 shutdown 2024-06-02T23:34:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 msgr2=0x55e055446430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.169 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 0x55e055446430 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c005940 tx=0x7f5b5c02a410 comp rx=0 tx=0).stop 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 shutdown_connections 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e0554b87f0 0x55e0554bac10 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e05543cf80 0x55e055447440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e055437e10 0x55e055446430 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 --2- 172.21.15.52:0/1970676329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e055437420 0x55e055445ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b54ff9640 0 client.21399 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:38.170 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 >> 172.21.15.52:0/1970676329 conn(0x55e05542e680 msgr2=0x55e05542cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 shutdown_connections 2024-06-02T23:34:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 1 -- 172.21.15.52:0/1970676329 wait complete. 2024-06-02T23:34:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b76818740 20 client.21399 trim_cache size 0 max 16384 2024-06-02T23:34:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:34:38.171 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:34:38.172 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:34:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b580a2c30 msgr2=0x7f5b580a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b580a2c30 0x7f5b580a50f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f5b6400afd0 tx=0x7f5b6401a000 comp rx=0 tx=0).stop 2024-06-02T23:34:38.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 msgr2=0x7f5b6c198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c198020 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c004770 tx=0x7f5b5c0042f0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 shutdown_connections 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b580a2c30 0x7f5b580a50f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b6c1035b0 0x7f5b6c19f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b6c102bf0 0x7f5b6c198020 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 --2- 172.21.15.52:0/2355669285 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b6c101a00 0x7f5b6c197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:38.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 >> 172.21.15.52:0/2355669285 conn(0x7f5b6c0fcf90 msgr2=0x7f5b6c105e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 shutdown_connections 2024-06-02T23:34:38.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:38.168+0000 7f5b73a31640 1 -- 172.21.15.52:0/2355669285 wait complete. 2024-06-02T23:34:38.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.51 MB/sec execute 466 sec latency 15995.131 ms 2024-06-02T23:34:39.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.49 MB/sec execute 467 sec latency 16995.222 ms 2024-06-02T23:34:39.498 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:39 smithi052 ceph-mon[32340]: pgmap v1114: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 73 KiB/s wr, 28 op/s 2024-06-02T23:34:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:39 smithi174 ceph-mon[35994]: pgmap v1114: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 73 KiB/s wr, 28 op/s 2024-06-02T23:34:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 -- 172.21.15.52:0/2155069717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b00102740 msgr2=0x7f3b00102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 --2- 172.21.15.52:0/2155069717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b00102740 0x7f3b00102b20 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f3af00099c0 tx=0x7f3af0009cd0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 -- 172.21.15.52:0/2155069717 shutdown_connections 2024-06-02T23:34:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 --2- 172.21.15.52:0/2155069717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 0x7f3b0010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.742 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 --2- 172.21.15.52:0/2155069717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b00102740 0x7f3b00102b20 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 --2- 172.21.15.52:0/2155069717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b001030f0 0x7f3b00107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 -- 172.21.15.52:0/2155069717 >> 172.21.15.52:0/2155069717 conn(0x7f3b000795f0 msgr2=0x7f3b00079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 -- 172.21.15.52:0/2155069717 shutdown_connections 2024-06-02T23:34:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.742+0000 7f3b05d43700 1 -- 172.21.15.52:0/2155069717 wait complete. 2024-06-02T23:34:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 Processor -- start 2024-06-02T23:34:39.743 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 -- start start 2024-06-02T23:34:39.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b00102740 0x7f3b00074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 0x7f3b00074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 0x7f3b00075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00112610 con 0x7f3b001030f0 2024-06-02T23:34:39.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00112490 con 0x7f3b00107840 2024-06-02T23:34:39.745 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3b05d43700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b00112790 con 0x7f3b00102740 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3af6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 0x7f3b00074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3af6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 0x7f3b00074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46446/0 (socket says 172.21.15.52:46446) 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3af6ffd700 1 -- 172.21.15.52:0/1191607610 learned_addr learned my addr 172.21.15.52:0/1191607610 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3afffff700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 0x7f3b00075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b00102740 0x7f3b00074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3af6ffd700 1 -- 172.21.15.52:0/1191607610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b00102740 msgr2=0x7f3b00074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.743+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b00102740 0x7f3b00074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3af6ffd700 1 -- 172.21.15.52:0/1191607610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 msgr2=0x7f3b00075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 0x7f3b00075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.746 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3af6ffd700 1 -- 172.21.15.52:0/1191607610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001a7010 con 0x7f3b001030f0 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3afffff700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 0x7f3b00075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 0x7f3b00074cd0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c920 tx=0x7f3ae800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3afd7fa700 1 -- 172.21.15.52:0/1191607610 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8007b00 con 0x7f3b001030f0 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b001a7300 con 0x7f3b001030f0 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3afd7fa700 1 -- 172.21.15.52:0/1191607610 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3ae8004ce0 con 0x7f3b001030f0 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3afd7fa700 1 -- 172.21.15.52:0/1191607610 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ae8005560 con 0x7f3b001030f0 2024-06-02T23:34:39.747 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.744+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001a7890 con 0x7f3b001030f0 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.747+0000 7f3afd7fa700 1 -- 172.21.15.52:0/1191607610 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3ae8004750 con 0x7f3b001030f0 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.747+0000 7f3afd7fa700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3aec0a2c20 0x7f3aec0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.747+0000 7f3afd7fa700 1 -- 172.21.15.52:0/1191607610 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3ae80cecc0 con 0x7f3b001030f0 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.747+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3aec0a2c20 0x7f3aec0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 Processor -- start 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3aec0a2c20 0x7f3aec0a50e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009f50 tx=0x7f3af0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 -- start start 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a146340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c54a097810 con 0x55c54a145f20 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c54a03f720 con 0x55c54a14bf60 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c54a0937d0 con 0x55c54a14b640 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3afffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a146340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3afffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a146340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46458/0 (socket says 172.21.15.52:46458) 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3afffff700 1 -- 172.21.15.52:0/4282979843 learned_addr learned my addr 172.21.15.52:0/4282979843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.748+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3afffff700 1 -- 172.21.15.52:0/4282979843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 msgr2=0x55c54a14ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3afffff700 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3afffff700 1 -- 172.21.15.52:0/4282979843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 msgr2=0x55c54a1459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3afffff700 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a1459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3afffff700 1 -- 172.21.15.52:0/4282979843 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54a035f50 con 0x55c54a145f20 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3afffff700 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a146340 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009dd0 tx=0x7f3af80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e9bc4df0edb745 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3af5ffb700 1 -- 172.21.15.52:0/4282979843 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8014610 con 0x55c54a145f20 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3af5ffb700 1 -- 172.21.15.52:0/4282979843 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af8019070 con 0x55c54a145f20 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.749+0000 7f3af5ffb700 1 -- 172.21.15.52:0/4282979843 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8013900 con 0x55c54a145f20 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4282979843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 msgr2=0x55c54a146340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a146340 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f3af8009dd0 tx=0x7f3af80077e0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4282979843 shutdown_connections 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a146340 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4282979843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a1459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4282979843 >> 172.21.15.52:0/4282979843 conn(0x55c54a070cc0 msgr2=0x55c54a140850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4282979843 shutdown_connections 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.750+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4282979843 wait complete. 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 -- start start 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 Processor -- start 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a145f20 0x55c54a151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 0x55c54a14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c54a1529d0 con 0x55c54a14bf60 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c54a037bf0 con 0x55c54a145f20 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c54a152b50 con 0x55c54a14b640 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3b105c6b80 10 client.0 _ll_register_callbacks cb 0x55c54a031920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:39.755 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3afffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 0x55c54a14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3afffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 0x55c54a14f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46470/0 (socket says 172.21.15.52:46470) 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3afffff700 1 -- 172.21.15.52:0/4060393363 learned_addr learned my addr 172.21.15.52:0/4060393363 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.751+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a145f20 0x55c54a151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3afffff700 1 -- 172.21.15.52:0/4060393363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 msgr2=0x55c54a14efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3afffff700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3afffff700 1 -- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a145f20 msgr2=0x55c54a151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3afffff700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a145f20 0x55c54a151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3afffff700 1 -- 172.21.15.52:0/4060393363 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c549fe68d0 con 0x55c54a14bf60 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3afffff700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 0x55c54a14f4f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000f80 tx=0x7f3af8014640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a6f977d3f000be server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:39.756 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3adeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:39.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3adeffd700 1 -- 172.21.15.52:0/4060393363 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af8014cc0 con 0x55c54a14bf60 2024-06-02T23:34:39.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3adeffd700 1 -- 172.21.15.52:0/4060393363 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3af8027470 con 0x55c54a14bf60 2024-06-02T23:34:39.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af8019070 con 0x55c54a14bf60 2024-06-02T23:34:39.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3adeffd700 1 -- 172.21.15.52:0/4060393363 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3af8027690 con 0x55c54a14bf60 2024-06-02T23:34:39.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3b105c6b80 10 client.21411 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:39.757 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.752+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c54a15d2b0 con 0x55c54a14bf60 2024-06-02T23:34:39.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:39 smithi123 ceph-mon[35977]: pgmap v1114: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 73 KiB/s wr, 28 op/s 2024-06-02T23:34:39.759 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3adeffd700 1 -- 172.21.15.52:0/4060393363 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af8019070 con 0x55c54a14bf60 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3adeffd700 1 -- 172.21.15.52:0/4060393363 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3af80053c0 con 0x55c54a14bf60 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3b105c6b80 10 client.21411 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3b105c6b80 10 client.21411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3b105c6b80 20 client.21411 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a15f780 0x55c54a161b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3b105c6b80 4 client.21411 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c54a0a4d10 con 0x55c54a15f780 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.753+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a15f780 0x55c54a161b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.754+0000 7f3adeffd700 10 client.21411 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.754+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a15f780 0x55c54a161b70 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8adaf65caf5ee636 server_cookie=cf4a9e5c1a9e3fb6 in_seq=0 out_seq=0 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3adeffd700 1 -- 172.21.15.52:0/4060393363 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c54a0a4d10 con 0x55c54a15f780 2024-06-02T23:34:39.760 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3adeffd700 10 client.21411 handle_command_reply: tid=0 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 client.21411 shutdown 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 msgr2=0x55c54a14f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 0x55c54a14f4f0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f3af8000f80 tx=0x7f3af8014640 comp rx=0 tx=0).stop 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 shutdown_connections 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a15f780 0x55c54a161b70 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a14bf60 0x55c54a14f4f0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a145f20 0x55c54a151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/4060393363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a14efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 >> 172.21.15.52:0/4060393363 conn(0x55c54a070cc0 msgr2=0x55c54a1449d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3adeffd700 0 client.21411 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 shutdown_connections 2024-06-02T23:34:39.761 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 1 -- 172.21.15.52:0/4060393363 wait complete. 2024-06-02T23:34:39.762 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.755+0000 7f3b105c6b80 20 client.21411 trim_cache size 0 max 16384 2024-06-02T23:34:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 Processor -- start 2024-06-02T23:34:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 -- start start 2024-06-02T23:34:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a1497d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a14c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001aed40 con 0x55c54a145f20 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b0880 con 0x55c54a14bf60 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001af410 con 0x55c54a14b640 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a1459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46476/0 (socket says 172.21.15.52:46476) 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.771+0000 7f3aff7fe700 1 -- 172.21.15.52:0/2573768511 learned_addr learned my addr 172.21.15.52:0/2573768511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3afffff700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a1497d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3aff7fe700 1 -- 172.21.15.52:0/2573768511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 msgr2=0x55c54a1497d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a1497d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3aff7fe700 1 -- 172.21.15.52:0/2573768511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 msgr2=0x55c54a14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a14c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3aff7fe700 1 -- 172.21.15.52:0/2573768511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c54a152490 con 0x55c54a145f20 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3afffff700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a1497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a1459e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f3af0003770 tx=0x7f3af001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ff19234f9ad86a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3af57fa700 1 -- 172.21.15.52:0/2573768511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0024a70 con 0x55c54a145f20 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3af57fa700 1 -- 172.21.15.52:0/2573768511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af0024c10 con 0x55c54a145f20 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.772+0000 7f3af57fa700 1 -- 172.21.15.52:0/2573768511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af003c670 con 0x55c54a145f20 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 -- 172.21.15.52:0/2573768511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 msgr2=0x55c54a1459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a1459e0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f3af0003770 tx=0x7f3af001f000 comp rx=0 tx=0).stop 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 -- 172.21.15.52:0/2573768511 shutdown_connections 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14bf60 0x55c54a14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a1459e0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/2573768511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14b640 0x55c54a1497d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 -- 172.21.15.52:0/2573768511 >> 172.21.15.52:0/2573768511 conn(0x55c54a070cc0 msgr2=0x55c54a1490c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 -- 172.21.15.52:0/2573768511 shutdown_connections 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 -- 172.21.15.52:0/2573768511 wait complete. 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 -- start start 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.773+0000 7f3b105c6b80 1 Processor -- start 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a164b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14b640 0x55c54a165040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14bf60 0x55c54a161f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001b01b0 con 0x55c54a145f20 2024-06-02T23:34:39.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001afae0 con 0x55c54a14b640 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b001acd00 con 0x55c54a14bf60 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3b105c6b80 10 client.0 _ll_register_callbacks cb 0x55c54a031920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a164b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46490/0 (socket says 172.21.15.52:46490) 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 -- 172.21.15.52:0/925804705 learned_addr learned my addr 172.21.15.52:0/925804705 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14b640 0x55c54a165040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3afffff700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14bf60 0x55c54a161f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 -- 172.21.15.52:0/925804705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14bf60 msgr2=0x55c54a161f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14bf60 0x55c54a161f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 -- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14b640 msgr2=0x55c54a165040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14b640 0x55c54a165040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 -- 172.21.15.52:0/925804705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c54a1627d0 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3aff7fe700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a164b00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b880 tx=0x7f3af000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821630387518e92a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.774+0000 7f3adeffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3adeffd700 1 -- 172.21.15.52:0/925804705 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af0040070 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3af0024c10 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3adeffd700 1 -- 172.21.15.52:0/925804705 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3af0045660 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3adeffd700 1 -- 172.21.15.52:0/925804705 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3af0045860 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 10 client.21417 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c54a1c9c50 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3adeffd700 1 -- 172.21.15.52:0/925804705 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3af0024c10 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3adeffd700 1 -- 172.21.15.52:0/925804705 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3af000fae0 con 0x55c54a145f20 2024-06-02T23:34:39.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 10 client.21417 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 10 client.21417 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 20 client.21417 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a1ca280 0x55c54a1cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 4 client.21417 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:34:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.775+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c54a09f290 con 0x55c54a1ca280 2024-06-02T23:34:39.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a1ca280 0x55c54a1cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:39.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3af6ffd700 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a1ca280 0x55c54a1cc6a0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28cfa7530ea68427 server_cookie=8ca1d798781db761 in_seq=0 out_seq=0 2024-06-02T23:34:39.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3adeffd700 10 client.21417 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:39.782 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3adeffd700 1 -- 172.21.15.52:0/925804705 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c54a09f290 con 0x55c54a1ca280 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3adeffd700 10 client.21417 handle_command_reply: tid=0 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 client.21417 shutdown 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 msgr2=0x55c54a164b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a164b00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f3af000b880 tx=0x7f3af000be70 comp rx=0 tx=0).stop 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 shutdown_connections 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55c54a1ca280 0x55c54a1cc6a0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c54a14bf60 0x55c54a161f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c54a145f20 0x55c54a164b00 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 --2- 172.21.15.52:0/925804705 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c54a14b640 0x55c54a165040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.776+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 >> 172.21.15.52:0/925804705 conn(0x55c54a070cc0 msgr2=0x55c54a139e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:39.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3adeffd700 0 client.21417 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:39.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 shutdown_connections 2024-06-02T23:34:39.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b105c6b80 1 -- 172.21.15.52:0/925804705 wait complete. 2024-06-02T23:34:39.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b105c6b80 20 client.21417 trim_cache size 0 max 16384 2024-06-02T23:34:39.784 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3aec0a2c20 msgr2=0x7f3aec0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3aec0a2c20 0x7f3aec0a50e0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f3af0009f50 tx=0x7f3af0005fb0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 msgr2=0x7f3b00074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 0x7f3b00074cd0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f3ae800c920 tx=0x7f3ae800cce0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 shutdown_connections 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3aec0a2c20 0x7f3aec0a50e0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b00107840 0x7f3b00075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b00102740 0x7f3b00074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 --2- 172.21.15.52:0/1191607610 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3b001030f0 0x7f3b00074cd0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.777+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 >> 172.21.15.52:0/1191607610 conn(0x7f3b000795f0 msgr2=0x7f3b00100720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.778+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 shutdown_connections 2024-06-02T23:34:39.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:39.778+0000 7f3b05d43700 1 -- 172.21.15.52:0/1191607610 wait complete. 2024-06-02T23:34:40.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.48 MB/sec execute 468 sec latency 17995.289 ms 2024-06-02T23:34:41.426 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:34:41.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:34:41.477 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.47 MB/sec execute 469 sec latency 18995.339 ms 2024-06-02T23:34:41.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:41 smithi174 ceph-mon[35994]: pgmap v1115: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-06-02T23:34:41.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:41 smithi052 ceph-mon[32340]: pgmap v1115: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-06-02T23:34:41.682 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:41 smithi123 ceph-mon[35977]: pgmap v1115: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 21 op/s 2024-06-02T23:34:42.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.45 MB/sec execute 470 sec latency 19995.437 ms 2024-06-02T23:34:43.447 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:43 smithi052 ceph-mon[32340]: pgmap v1116: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-06-02T23:34:43.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.44 MB/sec execute 471 sec latency 20995.489 ms 2024-06-02T23:34:43.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 -- 172.21.15.52:0/3084956962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f301030f0 msgr2=0x7f4f30107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.554 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 --2- 172.21.15.52:0/3084956962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f301030f0 0x7f4f30107110 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f4f240099c0 tx=0x7f4f24009cd0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 -- 172.21.15.52:0/3084956962 shutdown_connections 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 --2- 172.21.15.52:0/3084956962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f30107840 0x7f4f3010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 --2- 172.21.15.52:0/3084956962 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30102740 0x7f4f30102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 --2- 172.21.15.52:0/3084956962 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f301030f0 0x7f4f30107110 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 -- 172.21.15.52:0/3084956962 >> 172.21.15.52:0/3084956962 conn(0x7f4f300795f0 msgr2=0x7f4f30079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 -- 172.21.15.52:0/3084956962 shutdown_connections 2024-06-02T23:34:43.555 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.555+0000 7f4f37e75700 1 -- 172.21.15.52:0/3084956962 wait complete. 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 Processor -- start 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 -- start start 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 0x7f4f30074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f301030f0 0x7f4f30074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30107840 0x7f4f30075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30112680 con 0x7f4f30102740 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30112500 con 0x7f4f301030f0 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f37e75700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f30112800 con 0x7f4f30107840 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f35c11700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 0x7f4f30074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f35c11700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 0x7f4f30074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46504/0 (socket says 172.21.15.52:46504) 2024-06-02T23:34:43.557 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f35c11700 1 -- 172.21.15.52:0/1971738604 learned_addr learned my addr 172.21.15.52:0/1971738604 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f35410700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f301030f0 0x7f4f30074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.556+0000 7f4f36412700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30107840 0x7f4f30075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f35c11700 1 -- 172.21.15.52:0/1971738604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30107840 msgr2=0x7f4f30075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f35c11700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30107840 0x7f4f30075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f35c11700 1 -- 172.21.15.52:0/1971738604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f301030f0 msgr2=0x7f4f30074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f35c11700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f301030f0 0x7f4f30074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f35c11700 1 -- 172.21.15.52:0/1971738604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f301a70c0 con 0x7f4f30102740 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f36412700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30107840 0x7f4f30075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f35c11700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 0x7f4f30074790 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00b530 tx=0x7f4f2c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f22ffd700 1 -- 172.21.15.52:0/1971738604 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c015020 con 0x7f4f30102740 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f22ffd700 1 -- 172.21.15.52:0/1971738604 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f2c011ce0 con 0x7f4f30102740 2024-06-02T23:34:43.558 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f301a73b0 con 0x7f4f30102740 2024-06-02T23:34:43.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f22ffd700 1 -- 172.21.15.52:0/1971738604 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c010e40 con 0x7f4f30102740 2024-06-02T23:34:43.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.557+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f301a78c0 con 0x7f4f30102740 2024-06-02T23:34:43.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.559+0000 7f4f22ffd700 1 -- 172.21.15.52:0/1971738604 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4f2c018020 con 0x7f4f30102740 2024-06-02T23:34:43.559 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.560+0000 7f4f22ffd700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f1c0a2c20 0x7f4f1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.560+0000 7f4f35410700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f1c0a2c20 0x7f4f1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.560+0000 7f4f22ffd700 1 -- 172.21.15.52:0/1971738604 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f2c0ce8e0 con 0x7f4f30102740 2024-06-02T23:34:43.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.560+0000 7f4f35410700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f1c0a2c20 0x7f4f1c0a50e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4f30076220 tx=0x7f4f24005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:43.560 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.561+0000 7f4f426f8b80 1 Processor -- start 2024-06-02T23:34:43.561 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.561+0000 7f4f426f8b80 1 -- start start 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.561+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.561+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 0x55a5dc834a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.561+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc834fa0 0x55a5dc8353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.561+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc786890 con 0x55a5dc83afe0 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc72e7a0 con 0x55a5dc834fa0 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc782850 con 0x55a5dc83a6c0 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 0x55a5dc834a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 0x55a5dc834a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46516/0 (socket says 172.21.15.52:46516) 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 -- 172.21.15.52:0/2162132562 learned_addr learned my addr 172.21.15.52:0/2162132562 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f36412700 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc834fa0 0x55a5dc8353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35c11700 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 -- 172.21.15.52:0/2162132562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc83a6c0 msgr2=0x55a5dc83aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 -- 172.21.15.52:0/2162132562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc834fa0 msgr2=0x55a5dc8353c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc834fa0 0x55a5dc8353c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 -- 172.21.15.52:0/2162132562 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dc724fd0 con 0x55a5dc83afe0 2024-06-02T23:34:43.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f35410700 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 0x55a5dc834a60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4f240099c0 tx=0x7f4f24020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c11408469009ce6b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.562+0000 7f4f1bfff700 1 -- 172.21.15.52:0/2162132562 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24003e20 con 0x55a5dc83afe0 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f1bfff700 1 -- 172.21.15.52:0/2162132562 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f2400bcc0 con 0x55a5dc83afe0 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f1bfff700 1 -- 172.21.15.52:0/2162132562 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f240239d0 con 0x55a5dc83afe0 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 -- 172.21.15.52:0/2162132562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 msgr2=0x55a5dc834a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 0x55a5dc834a60 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f4f240099c0 tx=0x7f4f24020000 comp rx=0 tx=0).stop 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 -- 172.21.15.52:0/2162132562 shutdown_connections 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc834fa0 0x55a5dc8353c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/2162132562 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83afe0 0x55a5dc834a60 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 -- 172.21.15.52:0/2162132562 >> 172.21.15.52:0/2162132562 conn(0x55a5dc75fd40 msgr2=0x55a5dc82f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 -- 172.21.15.52:0/2162132562 shutdown_connections 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.563+0000 7f4f426f8b80 1 -- 172.21.15.52:0/2162132562 wait complete. 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 -- start start 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 Processor -- start 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc840d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83afe0 0x55a5dc83e610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc841990 con 0x55a5dc83a6c0 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc726c70 con 0x55a5dc83afe0 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a5dc841b10 con 0x55a5dc834fa0 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f426f8b80 10 client.0 _ll_register_callbacks cb 0x55a5dc7209a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.564+0000 7f4f35410700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46530/0 (socket says 172.21.15.52:46530) 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 -- 172.21.15.52:0/3257505854 learned_addr learned my addr 172.21.15.52:0/3257505854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f36412700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83afe0 0x55a5dc83e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35c11700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc840d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 -- 172.21.15.52:0/3257505854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 msgr2=0x55a5dc840d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc840d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 -- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83afe0 msgr2=0x55a5dc83e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83afe0 0x55a5dc83e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 -- 172.21.15.52:0/3257505854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dc6d5950 con 0x55a5dc83a6c0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f35410700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83e0d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4f24000c00 tx=0x7f4f2400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4c27ebae7156bb9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f197fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f197fa700 1 -- 172.21.15.52:0/3257505854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2403b060 con 0x55a5dc83a6c0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f197fa700 1 -- 172.21.15.52:0/3257505854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f24023860 con 0x55a5dc83a6c0 2024-06-02T23:34:43.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f2400bcc0 con 0x55a5dc83a6c0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f197fa700 1 -- 172.21.15.52:0/3257505854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4f24023a60 con 0x55a5dc83a6c0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f426f8b80 10 client.21429 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.565+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a5dc84c2c0 con 0x55a5dc83a6c0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f197fa700 1 -- 172.21.15.52:0/3257505854 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f2400bcc0 con 0x55a5dc83a6c0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f197fa700 1 -- 172.21.15.52:0/3257505854 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4f24042cb0 con 0x55a5dc83a6c0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f426f8b80 10 client.21429 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f426f8b80 10 client.21429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f426f8b80 20 client.21429 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc84e790 0x55a5dc850b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f426f8b80 4 client.21429 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5dc793d90 con 0x55a5dc84e790 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.566+0000 7f4f35c11700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc84e790 0x55a5dc850b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f35c11700 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc84e790 0x55a5dc850b80 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f722a89c301b1 server_cookie=bcd3c236719ad622 in_seq=0 out_seq=0 2024-06-02T23:34:43.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f197fa700 10 client.21429 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:43.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f197fa700 1 -- 172.21.15.52:0/3257505854 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55a5dc793d90 con 0x55a5dc84e790 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f197fa700 10 client.21429 handle_command_reply: tid=0 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f426f8b80 1 client.21429 shutdown 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 msgr2=0x55a5dc83e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.567+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83e0d0 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f4f24000c00 tx=0x7f4f2400ba40 comp rx=0 tx=0).stop 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 shutdown_connections 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc84e790 0x55a5dc850b80 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83afe0 0x55a5dc83e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc840d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3257505854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc83a6c0 0x55a5dc83e0d0 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f197fa700 0 client.21429 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 >> 172.21.15.52:0/3257505854 conn(0x55a5dc75fd40 msgr2=0x55a5dc82f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 shutdown_connections 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3257505854 wait complete. 2024-06-02T23:34:43.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.568+0000 7f4f426f8b80 20 client.21429 trim_cache size 0 max 16384 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 Processor -- start 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 -- start start 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc75fd40 0x55a5dc834a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 0x55a5dc829710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc853ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301aed00 con 0x55a5dc834fa0 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.589+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301b0840 con 0x55a5dc83a6c0 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301af3d0 con 0x55a5dc75fd40 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 0x55a5dc829710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 0x55a5dc829710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46542/0 (socket says 172.21.15.52:46542) 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 -- 172.21.15.52:0/3646514182 learned_addr learned my addr 172.21.15.52:0/3646514182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35c11700 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc75fd40 0x55a5dc834a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f36412700 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc853ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 -- 172.21.15.52:0/3646514182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc75fd40 msgr2=0x55a5dc834a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc75fd40 0x55a5dc834a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 -- 172.21.15.52:0/3646514182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 msgr2=0x55a5dc853ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc853ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 -- 172.21.15.52:0/3646514182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dc83c640 con 0x55a5dc834fa0 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f36412700 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc853ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f35410700 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 0x55a5dc829710 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4f24003b10 tx=0x7f4f240037f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4ebdf70ffbaf6e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f1b7fe700 1 -- 172.21.15.52:0/3646514182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2403b040 con 0x55a5dc834fa0 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f1b7fe700 1 -- 172.21.15.52:0/3646514182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f2400f7a0 con 0x55a5dc834fa0 2024-06-02T23:34:43.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.590+0000 7f4f1b7fe700 1 -- 172.21.15.52:0/3646514182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f24042840 con 0x55a5dc834fa0 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3646514182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 msgr2=0x55a5dc829710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 0x55a5dc829710 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f4f24003b10 tx=0x7f4f240037f0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3646514182 shutdown_connections 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc853ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc75fd40 0x55a5dc834a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/3646514182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc834fa0 0x55a5dc829710 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3646514182 >> 172.21.15.52:0/3646514182 conn(0x55a5dc838980 msgr2=0x55a5dc83baf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3646514182 shutdown_connections 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 -- 172.21.15.52:0/3646514182 wait complete. 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.591+0000 7f4f426f8b80 1 -- start start 2024-06-02T23:34:43.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 Processor -- start 2024-06-02T23:34:43.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 0x55a5dc853ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc854420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc8523d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301b0170 con 0x55a5dc75fd40 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301afaa0 con 0x55a5dc83a6c0 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f301accc0 con 0x55a5dc834fa0 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f426f8b80 10 client.0 _ll_register_callbacks cb 0x55a5dc7209a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f35c11700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 0x55a5dc853ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f35c11700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 0x55a5dc853ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46548/0 (socket says 172.21.15.52:46548) 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f35c11700 1 -- 172.21.15.52:0/192480642 learned_addr learned my addr 172.21.15.52:0/192480642 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f35410700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc854420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f36412700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc8523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.592+0000 7f4f35c11700 1 -- 172.21.15.52:0/192480642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 msgr2=0x55a5dc854420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f35c11700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc854420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f35c11700 1 -- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 msgr2=0x55a5dc8523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:43.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f35c11700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc8523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f35c11700 1 -- 172.21.15.52:0/192480642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dc852be0 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f36412700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc8523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f35c11700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 0x55a5dc853ee0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c097520 tx=0x7f4f2c0ce040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=608d2e2ea772d039 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f197fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f197fa700 1 -- 172.21.15.52:0/192480642 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c0a3d80 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f197fa700 1 -- 172.21.15.52:0/192480642 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4f2c09f120 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f2400f7a0 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f197fa700 1 -- 172.21.15.52:0/192480642 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4f2c0661c0 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f426f8b80 10 client.21435 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55a5dc8b9d80 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.593+0000 7f4f197fa700 1 -- 172.21.15.52:0/192480642 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4f2400f7a0 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f197fa700 1 -- 172.21.15.52:0/192480642 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4f2c0a38f0 con 0x55a5dc75fd40 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f426f8b80 10 client.21435 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:43.599 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f426f8b80 10 client.21435 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f426f8b80 20 client.21435 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc83afe0 0x55a5dc8bc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f426f8b80 4 client.21435 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a5dc78e310 con 0x55a5dc83afe0 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.594+0000 7f4f35410700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc83afe0 0x55a5dc8bc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.595+0000 7f4f35410700 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc83afe0 0x55a5dc8bc420 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b9219f6fb8ac7d server_cookie=ac7e38fa73a76609 in_seq=0 out_seq=0 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.595+0000 7f4f197fa700 10 client.21435 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f197fa700 1 -- 172.21.15.52:0/192480642 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a5dc78e310 con 0x55a5dc83afe0 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f197fa700 10 client.21435 handle_command_reply: tid=0 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 client.21435 shutdown 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 msgr2=0x55a5dc853ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.600 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 0x55a5dc853ee0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c097520 tx=0x7f4f2c0ce040 comp rx=0 tx=0).stop 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 shutdown_connections 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55a5dc83afe0 0x55a5dc8bc420 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a5dc83a6c0 0x55a5dc8523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55a5dc75fd40 0x55a5dc853ee0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 --2- 172.21.15.52:0/192480642 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a5dc834fa0 0x55a5dc854420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 >> 172.21.15.52:0/192480642 conn(0x55a5dc838980 msgr2=0x55a5dc8391e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f197fa700 0 client.21435 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 shutdown_connections 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 1 -- 172.21.15.52:0/192480642 wait complete. 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.596+0000 7f4f426f8b80 20 client.21435 trim_cache size 0 max 16384 2024-06-02T23:34:43.601 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:34:43.602 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:34:43.602 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "176d6d5d-c4b3-4ff4-9217-cf5fc4fde499", 2024-06-02T23:34:43.602 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:34:43.602 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:34:43.602 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f1c0a2c20 msgr2=0x7f4f1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f1c0a2c20 0x7f4f1c0a50e0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4f30076220 tx=0x7f4f24005fb0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 msgr2=0x7f4f30074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 0x7f4f30074790 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c00b530 tx=0x7f4f2c00b840 comp rx=0 tx=0).stop 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 shutdown_connections 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4f1c0a2c20 0x7f4f1c0a50e0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f30107840 0x7f4f30075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4f30102740 0x7f4f30074790 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 --2- 172.21.15.52:0/1971738604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4f301030f0 0x7f4f30074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 >> 172.21.15.52:0/1971738604 conn(0x7f4f300795f0 msgr2=0x7f4f30108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 shutdown_connections 2024-06-02T23:34:43.603 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:43.597+0000 7f4f37e75700 1 -- 172.21.15.52:0/1971738604 wait complete. 2024-06-02T23:34:43.626 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:34:43.629 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-mon[35977]: pgmap v1116: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-06-02T23:34:43.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:43 smithi174 ceph-mon[35994]: pgmap v1116: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 10 KiB/s wr, 17 op/s 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.629+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003f3 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL/ auth{4=1} v16633 ap=3 f(v446 m2024-06-02T23:34:15.741053+0000 11=11+0) n(v29 rc2024-06-02T23:34:15.741053+0000 b2861056 12=11+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1388 nref=3)) caps={15216=pAsLsXs/-@9535} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb967a000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003f3:[<0x1000000033f/COREL v50>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003f3:[<0x1000000033f/COREL v16633>,<0x10000000335/~dmtmp v4155>,<0x10000000007/client3 v4758>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:34:15.711357+0000 12=12+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:34:15.741053+0000 b2861056 12=11+1)","memory_value.dirstat":"f(v446 m2024-06-02T23:34:15.741053+0000 11=11+0)","memory_value.rstat":"n(v29 rc2024-06-02T23:34:15.741053+0000 b2861056 12=11+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.630+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003eb [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL/ auth{3=1} v11110 ap=3 f(v416 m2024-06-02T23:34:19.270944+0000 15=15+0) n(v26 rc2024-06-02T23:34:19.301226+0000 b2959419 16=15+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1399 nref=3)) caps={15216=pAsLsXs/-@8887} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb964b080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003eb:[<0x10000000336/COREL v50>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003eb:[<0x10000000336/COREL v11110>,<0x10000000331/~dmtmp v7274>,<0x10000000007/client1 v4760>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:34:19.242051+0000 16=16+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:34:19.301226+0000 b2959419 16=15+1)","memory_value.dirstat":"f(v416 m2024-06-02T23:34:19.270944+0000 15=15+0)","memory_value.rstat":"n(v26 rc2024-06-02T23:34:19.301226+0000 b2959419 16=15+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.631+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000046d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO/ auth{4=1} v16663 ap=3 f(v601 m2024-06-02T23:34:15.913954+0000 3=3+0) n(v36 rc2024-06-02T23:34:15.913954+0000 b780288 4=3+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1392 nref=3)) caps={15216=pAsLsXs/-@10081} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9645180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000046d:[<0x1000000033f/WORDPRO v58>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000046d:[<0x1000000033f/WORDPRO v16663>,<0x10000000335/~dmtmp v4155>,<0x10000000007/client3 v4758>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:34:15.888732+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:34:15.913954+0000 b780288 4=3+1)","memory_value.dirstat":"f(v601 m2024-06-02T23:34:15.913954+0000 3=3+0)","memory_value.rstat":"n(v36 rc2024-06-02T23:34:15.913954+0000 b780288 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.632+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004b8 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX/ auth{4=1} v16665 ap=3 f(v611 m2024-06-02T23:34:22.372523+0000 24=24+0) n(v38 rc2024-06-02T23:34:22.491706+0000 b5214751 25=24+1) (inest sync->mix g=4) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1393 nref=3)) caps={15216=pAsLsXs/-@11249} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9816680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004b8:[<0x1000000033f/PARADOX v108>,<0x10000000335/~dmtmp v32>,<0x10000000007/client3 v1545>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004b8:[<0x1000000033f/PARADOX v16665>,<0x10000000335/~dmtmp v4155>,<0x10000000007/client3 v4758>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:34:22.070735+0000 23=23+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:34:22.491706+0000 b5214751 25=24+1)","memory_value.dirstat":"f(v611 m2024-06-02T23:34:22.372523+0000 24=24+0)","memory_value.rstat":"n(v38 rc2024-06-02T23:34:22.491706+0000 b5214751 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.632+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000038f [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL/ auth{3=1} v10922 ap=3 f(v608 m2024-06-02T23:34:12.117822+0000 8=8+0) n(v37 rc2024-06-02T23:34:12.125008+0000 b1283356 9=8+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1400 nref=3)) caps={15216=pAsLsXs/-@9513} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b87700]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000038f:[<0x10000000336/EXCEL v14>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000038f:[<0x10000000336/EXCEL v10922>,<0x10000000331/~dmtmp v7274>,<0x10000000007/client1 v4760>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:34:12.080900+0000 7=7+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:34:12.125008+0000 b1283356 9=8+1)","memory_value.dirstat":"f(v608 m2024-06-02T23:34:12.117822+0000 8=8+0)","memory_value.rstat":"n(v37 rc2024-06-02T23:34:12.125008+0000 b1283356 9=8+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.636+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000036d [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD/ auth{3=1} v10798 ap=3 f(v384 m2024-06-02T23:33:46.068509+0000 5=5+0) n(v31 rc2024-06-02T23:33:46.068509+0000 b814326 6=5+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1401 nref=3)) caps={15216=pAsLsXsFs/-@8738} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8ca0000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000036d:[<0x10000000336/WORD v8>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x1000000036d:[<0x10000000336/WORD v10798>,<0x10000000331/~dmtmp v7274>,<0x10000000007/client1 v4760>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:33:46.038172+0000 6=6+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:33:46.068509+0000 b814326 6=5+1)","memory_value.dirstat":"f(v384 m2024-06-02T23:33:46.068509+0000 5=5+0)","memory_value.rstat":"n(v31 rc2024-06-02T23:33:46.068509+0000 b814326 6=5+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.008 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.641+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000004ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX/ auth{3=1} v10794 ap=3 f(v560 m2024-06-02T23:33:52.913672+0000 24=24+0) n(v37 rc2024-06-02T23:33:52.913672+0000 b5230239 25=24+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1404 nref=3)) caps={15216=pAsLsXs/-@10442} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyrstat=0 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb97ee100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000004ac:[<0x10000000336/PARADOX v108>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000004ac:[<0x10000000336/PARADOX v10794>,<0x10000000331/~dmtmp v7274>,<0x10000000007/client1 v4760>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:33:52.870631+0000 25=25+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:33:52.913672+0000 b5230239 25=24+1)","memory_value.dirstat":"f(v560 m2024-06-02T23:33:52.913672+0000 24=24+0)","memory_value.rstat":"n(v37 rc2024-06-02T23:33:52.913672+0000 b5230239 25=24+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.009 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.643+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x100000003ac [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT/ auth{3=1} v10796 ap=3 f(v448 m2024-06-02T23:33:43.900128+0000 7=7+0) n(v31 rc2024-06-02T23:33:43.900128+0000 b1462582 8=7+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1406 nref=3)) caps={15216=pAsLsXs/-@8693} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb8b64100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x100000003ac:[<0x10000000336/PWRPNT v214>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x100000003ac:[<0x10000000336/PWRPNT v10796>,<0x10000000331/~dmtmp v7274>,<0x10000000007/client1 v4760>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:33:43.834296+0000 8=8+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:33:43.900128+0000 b1462582 8=7+1)","memory_value.dirstat":"f(v448 m2024-06-02T23:33:43.900128+0000 7=7+0)","memory_value.rstat":"n(v31 rc2024-06-02T23:33:43.900128+0000 b1462582 8=7+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.009 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:34:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:34:43.645+0000 7f7cb7bd4700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000568 [...2,head] /volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS/ auth{3=1} v11108 ap=3 f(v224 m2024-06-02T23:34:17.407876+0000 4=4+0) n(v31 rc2024-06-02T23:34:17.407876+0000 b363650 5=4+1) (inest sync->mix g=3) (ipolicy sync r=1) (iquiesce lock x=1 by request(mds.0:1407 nref=3)) caps={15216=pAsLsXs/-@7984} | dirtyscattered=0 request=1 lock=2 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x55edb9850100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000568:[<0x10000000336/ACCESS v112>,<0x10000000331/~dmtmp v32>,<0x10000000007/client1 v1539>,<0x10000000006/clients v1379>,<0x10000000005/tmp v1314>,<0x10000000003/client.0 v1277>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v1256>,<0x10000000001/sv_0 v1260>,<0x10000000000/qa v1218>,<0x1/volumes v1225>]//[]","memoryvalue":"(2)0x10000000568:[<0x10000000336/ACCESS v11108>,<0x10000000331/~dmtmp v7274>,<0x10000000007/client1 v4760>,<0x10000000006/clients v3114>,<0x10000000005/tmp v2277>,<0x10000000003/client.0 v2237>,<0x10000000002/8f46d269-a88a-4e0c-a87a-3cc369c34abc v2199>,<0x10000000001/sv_0 v2197>,<0x10000000000/qa v2132>,<0x1/volumes v2118>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-06-02T23:34:17.371920+0000 5=5+0)","ondisk_value.rstat":"n(v0 rc2024-06-02T23:34:17.407876+0000 b363650 5=4+1)","memory_value.dirstat":"f(v224 m2024-06-02T23:34:17.407876+0000 4=4+0)","memory_value.rstat":"n(v31 rc2024-06-02T23:34:17.407876+0000 b363650 5=4+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-06-02T23:34:44.361 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:34:44.421 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:44.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.42 MB/sec execute 472 sec latency 21995.541 ms 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:34:44.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: Scrub error on inode 0x10000000568 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.645 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:44 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: Scrub error on inode 0x10000000568 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:44 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:34:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003f3 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003eb (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/COREL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000046d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/WORDPRO) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004b8 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client3/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000038f (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/EXCEL) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x1000000036d (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000004ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PARADOX) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x100000003ac (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/PWRPNT) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: Scrub error on inode 0x10000000568 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client1/~dmtmp/ACCESS) see mds.b log and `damage ls` output for details 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:44.759 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:44 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:34:45.157 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:45.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.41 MB/sec execute 473 sec latency 22995.646 ms 2024-06-02T23:34:45.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:45 smithi174 ceph-mon[35994]: pgmap v1117: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 25 op/s 2024-06-02T23:34:45.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:45 smithi052 ceph-mon[32340]: pgmap v1117: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 25 op/s 2024-06-02T23:34:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:45 smithi123 ceph-mon[35977]: pgmap v1117: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 16 KiB/s wr, 25 op/s 2024-06-02T23:34:46.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.40 MB/sec execute 474 sec latency 23995.759 ms 2024-06-02T23:34:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 -- 172.21.15.52:0/581676409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0984108500 msgr2=0x7f09841088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 --2- 172.21.15.52:0/581676409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0984108500 0x7f09841088e0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f0974009990 tx=0x7f0974009ca0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 -- 172.21.15.52:0/581676409 shutdown_connections 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 --2- 172.21.15.52:0/581676409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f098406ab30 0x7f098410f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 --2- 172.21.15.52:0/581676409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0984108500 0x7f09841088e0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 --2- 172.21.15.52:0/581676409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f098406a170 0x7f098406a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 -- 172.21.15.52:0/581676409 >> 172.21.15.52:0/581676409 conn(0x7f0984078340 msgr2=0x7f0984078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 -- 172.21.15.52:0/581676409 shutdown_connections 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.468+0000 7f098a2b3700 1 -- 172.21.15.52:0/581676409 wait complete. 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 Processor -- start 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 -- start start 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 0x7f098419e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f098406ab30 0x7f098419ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0984108500 0x7f09841989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0984112630 con 0x7f098406a170 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f09841124b0 con 0x7f098406ab30 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f098a2b3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09841127b0 con 0x7f0984108500 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f0983fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 0x7f098419e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.469+0000 7f0983fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 0x7f098419e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42338/0 (socket says 172.21.15.52:42338) 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 -- 172.21.15.52:0/1364182606 learned_addr learned my addr 172.21.15.52:0/1364182606 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0988850700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0984108500 0x7f09841989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f09837fe700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f098406ab30 0x7f098419ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 -- 172.21.15.52:0/1364182606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0984108500 msgr2=0x7f09841989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0984108500 0x7f09841989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 -- 172.21.15.52:0/1364182606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f098406ab30 msgr2=0x7f098419ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f098406ab30 0x7f098419ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 -- 172.21.15.52:0/1364182606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0984199110 con 0x7f098406a170 2024-06-02T23:34:47.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f0983fff700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 0x7f098419e7a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0974000c00 tx=0x7f09740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974027070 con 0x7f098406a170 2024-06-02T23:34:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09841993a0 con 0x7f098406a170 2024-06-02T23:34:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0974022470 con 0x7f098406a170 2024-06-02T23:34:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0974017650 con 0x7f098406a170 2024-06-02T23:34:47.473 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.470+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09841a5670 con 0x7f098406a170 2024-06-02T23:34:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.471+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f098410a9e0 con 0x7f098406a170 2024-06-02T23:34:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.472+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f097402a070 con 0x7f098406a170 2024-06-02T23:34:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.473+0000 7f09817fa700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f096c0a2c20 0x7f096c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.473+0000 7f09837fe700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f096c0a2c20 0x7f096c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.473+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09740cfdf0 con 0x7f098406a170 2024-06-02T23:34:47.474 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.474+0000 7f09837fe700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f096c0a2c20 0x7f096c0a50e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0978005950 tx=0x7f09780058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.477 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.478+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f0974013070 con 0x7f098406a170 2024-06-02T23:34:47.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.38 MB/sec execute 475 sec latency 24995.794 ms 2024-06-02T23:34:47.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:47 smithi174 ceph-mon[35994]: pgmap v1118: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 KiB/s wr, 60 op/s 2024-06-02T23:34:47.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:47 smithi052 ceph-mon[32340]: pgmap v1118: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 KiB/s wr, 60 op/s 2024-06-02T23:34:47.734 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.734+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0984069180 con 0x7f098406a170 2024-06-02T23:34:47.735 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.735+0000 7f09817fa700 1 -- 172.21.15.52:0/1364182606 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f097409ce10 con 0x7f098406a170 2024-06-02T23:34:47.735 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:34:47.735 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:34:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f096c0a2c20 msgr2=0x7f096c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f096c0a2c20 0x7f096c0a50e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f0978005950 tx=0x7f09780058e0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 msgr2=0x7f098419e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 0x7f098419e7a0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f0974000c00 tx=0x7f09740047b0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.737 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 shutdown_connections 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f096c0a2c20 0x7f096c0a50e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0984108500 0x7f09841989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f098406a170 0x7f098419e7a0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 --2- 172.21.15.52:0/1364182606 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f098406ab30 0x7f098419ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.737+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 >> 172.21.15.52:0/1364182606 conn(0x7f0984078340 msgr2=0x7f09841000e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.738+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 shutdown_connections 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.738+0000 7f098a2b3700 1 -- 172.21.15.52:0/1364182606 wait complete. 2024-06-02T23:34:47.738 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:34:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:47 smithi123 ceph-mon[35977]: pgmap v1118: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 12 KiB/s wr, 60 op/s 2024-06-02T23:34:47.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 -- 172.21.15.52:0/3719467681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102530 msgr2=0x7f439c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 --2- 172.21.15.52:0/3719467681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102530 0x7f439c1029b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f43900099c0 tx=0x7f4390009cd0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 -- 172.21.15.52:0/3719467681 shutdown_connections 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 --2- 172.21.15.52:0/3719467681 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c102ef0 0x7f439c10f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 --2- 172.21.15.52:0/3719467681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f439c1085d0 0x7f439c1089b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 --2- 172.21.15.52:0/3719467681 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102530 0x7f439c1029b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 -- 172.21.15.52:0/3719467681 >> 172.21.15.52:0/3719467681 conn(0x7f439c079640 msgr2=0x7f439c079a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 -- 172.21.15.52:0/3719467681 shutdown_connections 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.934+0000 7f43a30bc700 1 -- 172.21.15.52:0/3719467681 wait complete. 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 Processor -- start 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 -- start start 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f439c102530 0x7f439c19e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 0x7f439c19ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c1085d0 0x7f439c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1126c0 con 0x7f439c102ef0 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f439c112540 con 0x7f439c1085d0 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a30bc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f439c112840 con 0x7f439c102530 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 0x7f439c19ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 0x7f439c19ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42362/0 (socket says 172.21.15.52:42362) 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 -- 172.21.15.52:0/407544902 learned_addr learned my addr 172.21.15.52:0/407544902 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a0e58700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f439c102530 0x7f439c19e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a1659700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c1085d0 0x7f439c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 -- 172.21.15.52:0/407544902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f439c102530 msgr2=0x7f439c19e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f439c102530 0x7f439c19e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 -- 172.21.15.52:0/407544902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c1085d0 msgr2=0x7f439c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c1085d0 0x7f439c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f439bfff700 1 -- 172.21.15.52:0/407544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c199060 con 0x7f439c102ef0 2024-06-02T23:34:47.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.935+0000 7f43a1659700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c1085d0 0x7f439c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.936+0000 7f439bfff700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 0x7f439c19ec80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4390006010 tx=0x7f43900056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.936+0000 7f4399ffb700 1 -- 172.21.15.52:0/407544902 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390026070 con 0x7f439c102ef0 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.936+0000 7f4399ffb700 1 -- 172.21.15.52:0/407544902 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f439000f460 con 0x7f439c102ef0 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.936+0000 7f4399ffb700 1 -- 172.21.15.52:0/407544902 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4390009e60 con 0x7f439c102ef0 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.936+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c1992f0 con 0x7f439c102ef0 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.936+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c1a5610 con 0x7f439c102ef0 2024-06-02T23:34:47.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.938+0000 7f4399ffb700 1 -- 172.21.15.52:0/407544902 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4390029070 con 0x7f439c102ef0 2024-06-02T23:34:47.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.939+0000 7f4399ffb700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43840a2c20 0x7f43840a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.940+0000 7f43a0e58700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43840a2c20 0x7f43840a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.940+0000 7f4399ffb700 1 -- 172.21.15.52:0/407544902 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f43900cf970 con 0x7f439c102ef0 2024-06-02T23:34:47.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.940+0000 7f43a0e58700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43840a2c20 0x7f43840a50e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f438c005950 tx=0x7f438c005d10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 Processor -- start 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 -- start start 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49a370 0x55897b49a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b494670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 0x55897b494ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55897b382f60 con 0x55897b494bb0 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55897b3e1850 con 0x55897b49ac90 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55897b349460 con 0x55897b49a370 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 0x55897b494ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 0x55897b494ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42378/0 (socket says 172.21.15.52:42378) 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 -- 172.21.15.52:0/214247560 learned_addr learned my addr 172.21.15.52:0/214247560 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:47.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f439bfff700 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b494670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a0e58700 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49a370 0x55897b49a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 -- 172.21.15.52:0/214247560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49a370 msgr2=0x55897b49a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49a370 0x55897b49a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 -- 172.21.15.52:0/214247560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 msgr2=0x55897b494670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b494670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.941+0000 7f43a1659700 1 -- 172.21.15.52:0/214247560 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897b3308d0 con 0x55897b494bb0 2024-06-02T23:34:47.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.942+0000 7f439bfff700 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b494670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.942+0000 7f43a1659700 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 0x55897b494ff0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4394009400 tx=0x7f4394009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07579d75f51e51c server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.942+0000 7f4382ffd700 1 -- 172.21.15.52:0/214247560 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394014610 con 0x55897b494bb0 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.942+0000 7f4382ffd700 1 -- 172.21.15.52:0/214247560 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4394019070 con 0x55897b494bb0 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.942+0000 7f4382ffd700 1 -- 172.21.15.52:0/214247560 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394013ab0 con 0x55897b494bb0 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 -- 172.21.15.52:0/214247560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 msgr2=0x55897b494ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 0x55897b494ff0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4394009400 tx=0x7f4394009920 comp rx=0 tx=0).stop 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 -- 172.21.15.52:0/214247560 shutdown_connections 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b494bb0 0x55897b494ff0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49a370 0x55897b49a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/214247560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b494670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 -- 172.21.15.52:0/214247560 >> 172.21.15.52:0/214247560 conn(0x55897b3bd080 msgr2=0x55897b48b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 -- 172.21.15.52:0/214247560 shutdown_connections 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 -- 172.21.15.52:0/214247560 wait complete. 2024-06-02T23:34:47.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 -- start start 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 Processor -- start 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b4a0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.943+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b49dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b49e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55897b381fc0 con 0x55897b49a370 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55897b3dd810 con 0x55897b49ac90 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55897b4a1680 con 0x55897b494bb0 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43ad93fb80 10 client.0 _ll_register_callbacks cb 0x55897b3775b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b49dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b49dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42388/0 (socket says 172.21.15.52:42388) 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 -- 172.21.15.52:0/197759779 learned_addr learned my addr 172.21.15.52:0/197759779 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43a0e58700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b4a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f43a1659700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b49e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 -- 172.21.15.52:0/197759779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 msgr2=0x55897b4a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b4a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 -- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 msgr2=0x55897b49e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b49e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 -- 172.21.15.52:0/197759779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897b3a7750 con 0x55897b49a370 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f439bfff700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b49dcd0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f43900cd4a0 tx=0x7f439000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d99550d24fb0a2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f437bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f437bfff700 1 -- 172.21.15.52:0/197759779 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900a0640 con 0x55897b49a370 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.944+0000 7f437bfff700 1 -- 172.21.15.52:0/197759779 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f439009a590 con 0x55897b49a370 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4394019070 con 0x55897b49a370 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f437bfff700 1 -- 172.21.15.52:0/197759779 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f43900b5200 con 0x55897b49a370 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 10 client.21453 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55897b4abd20 con 0x55897b49a370 2024-06-02T23:34:47.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f437bfff700 1 -- 172.21.15.52:0/197759779 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4394019070 con 0x55897b49a370 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f437bfff700 1 -- 172.21.15.52:0/197759779 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f43900a49e0 con 0x55897b49a370 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 10 client.21453 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 10 client.21453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 20 client.21453 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b4ae240 0x55897b4b0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 4 client.21453 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55897b3eed50 con 0x55897b4ae240 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.945+0000 7f43a0e58700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b4ae240 0x55897b4b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.946+0000 7f43a0e58700 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b4ae240 0x55897b4b0630 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=958608d44f45afd8 server_cookie=b58cfdd8a0a9f17f in_seq=0 out_seq=0 2024-06-02T23:34:47.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.946+0000 7f437bfff700 10 client.21453 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:47.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f437bfff700 1 -- 172.21.15.52:0/197759779 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55897b3eed50 con 0x55897b4ae240 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f437bfff700 10 client.21453 handle_command_reply: tid=0 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 client.21453 shutdown 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 msgr2=0x55897b49dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b49dcd0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f43900cd4a0 tx=0x7f439000fc60 comp rx=0 tx=0).stop 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 shutdown_connections 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b4ae240 0x55897b4b0630 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49ac90 0x55897b49e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b4a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/197759779 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b49dcd0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 >> 172.21.15.52:0/197759779 conn(0x55897b3bd080 msgr2=0x55897b48a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f437bfff700 0 client.21453 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 shutdown_connections 2024-06-02T23:34:47.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 1 -- 172.21.15.52:0/197759779 wait complete. 2024-06-02T23:34:47.952 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.947+0000 7f43ad93fb80 20 client.21453 trim_cache size 0 max 16384 2024-06-02T23:34:47.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 Processor -- start 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 -- start start 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b49fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49a370 0x55897b48ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 0x55897b48b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1aa750 con 0x55897b49ac90 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1ac290 con 0x55897b49a370 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1aae20 con 0x55897b494bb0 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43a1659700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 0x55897b48b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43a1659700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 0x55897b48b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42400/0 (socket says 172.21.15.52:42400) 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.968+0000 7f43a1659700 1 -- 172.21.15.52:0/2088342692 learned_addr learned my addr 172.21.15.52:0/2088342692 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f439bfff700 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49a370 0x55897b48ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a0e58700 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b49fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a1659700 1 -- 172.21.15.52:0/2088342692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 msgr2=0x55897b49fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a1659700 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b49fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a1659700 1 -- 172.21.15.52:0/2088342692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49a370 msgr2=0x55897b48ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a1659700 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49a370 0x55897b48ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a1659700 1 -- 172.21.15.52:0/2088342692 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55897b48c040 con 0x55897b49ac90 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a0e58700 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b49fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43a1659700 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 0x55897b48b3c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4394009400 tx=0x7f4394006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d06da11059373fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43827fc700 1 -- 172.21.15.52:0/2088342692 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394004d70 con 0x55897b49ac90 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43827fc700 1 -- 172.21.15.52:0/2088342692 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4394019040 con 0x55897b49ac90 2024-06-02T23:34:47.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.969+0000 7f43827fc700 1 -- 172.21.15.52:0/2088342692 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394014690 con 0x55897b49ac90 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2088342692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 msgr2=0x55897b48b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 0x55897b48b3c0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4394009400 tx=0x7f4394006040 comp rx=0 tx=0).stop 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2088342692 shutdown_connections 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49ac90 0x55897b48b3c0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b494bb0 0x55897b49fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2088342692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b49a370 0x55897b48ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2088342692 >> 172.21.15.52:0/2088342692 conn(0x55897b3bd080 msgr2=0x55897b49f4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2088342692 shutdown_connections 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2088342692 wait complete. 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 -- start start 2024-06-02T23:34:47.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.970+0000 7f43ad93fb80 1 Processor -- start 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b494bb0 0x55897b4a4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b4a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49ac90 0x55897b511ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1abbc0 con 0x55897b49a370 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1ab4f0 con 0x55897b494bb0 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f439c1a86f0 con 0x55897b49ac90 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43ad93fb80 10 client.0 _ll_register_callbacks cb 0x55897b3775b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b4a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b4a54a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42416/0 (socket says 172.21.15.52:42416) 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 -- 172.21.15.52:0/2380853071 learned_addr learned my addr 172.21.15.52:0/2380853071 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43a0e58700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b494bb0 0x55897b4a4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f43a1659700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49ac90 0x55897b511ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 -- 172.21.15.52:0/2380853071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49ac90 msgr2=0x55897b511ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49ac90 0x55897b511ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 -- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b494bb0 msgr2=0x55897b4a4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b494bb0 0x55897b4a4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 -- 172.21.15.52:0/2380853071 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55897b5124a0 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f439bfff700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b4a54a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f439001e040 tx=0x7f439000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb2e42894790c26 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f437bfff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.971+0000 7f437bfff700 1 -- 172.21.15.52:0/2380853071 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43900a0640 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f437bfff700 1 -- 172.21.15.52:0/2380853071 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f439009a590 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4394019040 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f437bfff700 1 -- 172.21.15.52:0/2380853071 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4390098530 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 10 client.21459 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55897b519840 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f437bfff700 1 -- 172.21.15.52:0/2380853071 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4394019040 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f437bfff700 1 -- 172.21.15.52:0/2380853071 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f43900bc0d0 con 0x55897b49a370 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 10 client.21459 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:47.976 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 10 client.21459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 20 client.21459 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b519dc0 0x55897b51c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 4 client.21459 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55897b3e92d0 con 0x55897b519dc0 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.972+0000 7f43a0e58700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b519dc0 0x55897b51c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f437bfff700 10 client.21459 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43a0e58700 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b519dc0 0x55897b51c1e0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa0c2c84624f676b server_cookie=d0e1190490e1ef8c in_seq=0 out_seq=0 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f437bfff700 1 -- 172.21.15.52:0/2380853071 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55897b3e92d0 con 0x55897b519dc0 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f437bfff700 10 client.21459 handle_command_reply: tid=0 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 client.21459 shutdown 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 msgr2=0x55897b4a54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b4a54a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f439001e040 tx=0x7f439000bee0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 shutdown_connections 2024-06-02T23:34:47.977 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55897b519dc0 0x55897b51c1e0 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55897b49ac90 0x55897b511ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55897b494bb0 0x55897b4a4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 --2- 172.21.15.52:0/2380853071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55897b49a370 0x55897b4a54a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 >> 172.21.15.52:0/2380853071 conn(0x55897b3bd080 msgr2=0x55897b49fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f437bfff700 0 client.21459 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 shutdown_connections 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.973+0000 7f43ad93fb80 1 -- 172.21.15.52:0/2380853071 wait complete. 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43ad93fb80 20 client.21459 trim_cache size 0 max 16384 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:34:47.978 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: "176d6d5d-c4b3-4ff4-9217-cf5fc4fde499": { 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "176d6d5d-c4b3-4ff4-9217-cf5fc4fde499", 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:47.979 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43840a2c20 msgr2=0x7f43840a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43840a2c20 0x7f43840a50e0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f438c005950 tx=0x7f438c005d10 comp rx=0 tx=0).stop 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 msgr2=0x7f439c19ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 0x7f439c19ec80 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4390006010 tx=0x7f43900056a0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 shutdown_connections 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43840a2c20 0x7f43840a50e0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f439c1085d0 0x7f439c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f439c102530 0x7f439c19e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 --2- 172.21.15.52:0/407544902 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f439c102ef0 0x7f439c19ec80 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 >> 172.21.15.52:0/407544902 conn(0x7f439c079640 msgr2=0x7f439c101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 shutdown_connections 2024-06-02T23:34:47.980 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:47.974+0000 7f43a30bc700 1 -- 172.21.15.52:0/407544902 wait complete. 2024-06-02T23:34:48.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.37 MB/sec execute 476 sec latency 25995.896 ms 2024-06-02T23:34:48.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:48 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1364182606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:48.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:48 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1364182606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:48.738 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 5a0c4e17 --release --await' 2024-06-02T23:34:48.738 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 5a0c4e17 --release --await 2024-06-02T23:34:48.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:48 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1364182606' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:34:49.472 INFO:tasks.cephfs.filesystem:scrub status for tag:176d6d5d-c4b3-4ff4-9217-cf5fc4fde499 - {'path': '/', 'tag': '176d6d5d-c4b3-4ff4-9217-cf5fc4fde499', 'options': 'recursive,force'} 2024-06-02T23:34:49.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.36 MB/sec execute 477 sec latency 26995.943 ms 2024-06-02T23:34:49.527 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:34:49.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:49 smithi174 ceph-mon[35994]: pgmap v1119: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 KiB/s wr, 73 op/s 2024-06-02T23:34:49.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:49 smithi052 ceph-mon[32340]: pgmap v1119: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 KiB/s wr, 73 op/s 2024-06-02T23:34:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:49 smithi123 ceph-mon[35977]: pgmap v1119: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 12 KiB/s wr, 73 op/s 2024-06-02T23:34:50.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68934 6.34 MB/sec execute 478 sec latency 27995.978 ms 2024-06-02T23:34:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 -- 172.21.15.52:0/2938396780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 msgr2=0x7f390c109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 --2- 172.21.15.52:0/2938396780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c109b30 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f3904009ca0 tx=0x7f390402a8c0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 -- 172.21.15.52:0/2938396780 shutdown_connections 2024-06-02T23:34:51.263 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 --2- 172.21.15.52:0/2938396780 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c104e10 0x7f390c107200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 --2- 172.21.15.52:0/2938396780 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c109b30 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 --2- 172.21.15.52:0/2938396780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c1024e0 0x7f390c1048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 -- 172.21.15.52:0/2938396780 >> 172.21.15.52:0/2938396780 conn(0x7f390c0fbe70 msgr2=0x7f390c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 -- 172.21.15.52:0/2938396780 shutdown_connections 2024-06-02T23:34:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.263+0000 7f3911e64640 1 -- 172.21.15.52:0/2938396780 wait complete. 2024-06-02T23:34:51.264 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.264+0000 7f3911e64640 1 Processor -- start 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.264+0000 7f3911e64640 1 -- start start 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.264+0000 7f3911e64640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c1024e0 0x7f390c197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.264+0000 7f3911e64640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c104e10 0x7f390c198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.264+0000 7f3911e64640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c19f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f3911e64640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10d530 con 0x7f390c107740 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c19f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390bfff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c19f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42424/0 (socket says 172.21.15.52:42424) 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390bfff640 1 -- 172.21.15.52:0/1286478022 learned_addr learned my addr 172.21.15.52:0/1286478022 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f3911e64640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10d3b0 con 0x7f390c104e10 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f390c10d6b0 con 0x7f390c1024e0 2024-06-02T23:34:51.265 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390b7fe640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c1024e0 0x7f390c197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390affd640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c104e10 0x7f390c198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.266 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390bfff640 1 -- 172.21.15.52:0/1286478022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c1024e0 msgr2=0x7f390c197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.265+0000 7f390bfff640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c1024e0 0x7f390c197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.266+0000 7f390bfff640 1 -- 172.21.15.52:0/1286478022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c104e10 msgr2=0x7f390c198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.266+0000 7f390bfff640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c104e10 0x7f390c198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.266+0000 7f390bfff640 1 -- 172.21.15.52:0/1286478022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c1003d0 con 0x7f390c107740 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.266+0000 7f390affd640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c104e10 0x7f390c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.266+0000 7f390b7fe640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c1024e0 0x7f390c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.266+0000 7f390bfff640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c19f100 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f3904031970 tx=0x7f3904003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.267+0000 7f3908ff9640 1 -- 172.21.15.52:0/1286478022 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904004040 con 0x7f390c107740 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.267+0000 7f3908ff9640 1 -- 172.21.15.52:0/1286478022 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3904004690 con 0x7f390c107740 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.267+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c100660 con 0x7f390c107740 2024-06-02T23:34:51.267 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.267+0000 7f3908ff9640 1 -- 172.21.15.52:0/1286478022 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904030c60 con 0x7f390c107740 2024-06-02T23:34:51.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.267+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c100c00 con 0x7f390c107740 2024-06-02T23:34:51.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.270+0000 7f3908ff9640 1 -- 172.21.15.52:0/1286478022 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f390402f4c0 con 0x7f390c107740 2024-06-02T23:34:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.270+0000 7f3914b75740 1 Processor -- start 2024-06-02T23:34:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.270+0000 7f3914b75740 1 -- start start 2024-06-02T23:34:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3914b75740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed493430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3914b75740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493a00 0x5569ed498500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3914b75740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 0x5569ed49b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3914b75740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5569ed3f64f0 con 0x5569ed498b70 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3914b75740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5569ed394870 con 0x5569ed493a00 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3914b75740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5569ed3f24b0 con 0x5569ed493010 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 0x5569ed49b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 0x5569ed49b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42426/0 (socket says 172.21.15.52:42426) 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 -- 172.21.15.52:0/1370244043 learned_addr learned my addr 172.21.15.52:0/1370244043 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390b7fe640 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493a00 0x5569ed498500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed493430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3908ff9640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38f40a2cb0 0x7f38f40a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 -- 172.21.15.52:0/1370244043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 msgr2=0x5569ed493430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed493430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 -- 172.21.15.52:0/1370244043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493a00 msgr2=0x5569ed498500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493a00 0x5569ed498500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 -- 172.21.15.52:0/1370244043 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ed3b09b0 con 0x5569ed498b70 2024-06-02T23:34:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390bfff640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38f40a2cb0 0x7f38f40a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f3908ff9640 1 -- 172.21.15.52:0/1286478022 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f39040f0410 con 0x7f390c107740 2024-06-02T23:34:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.271+0000 7f390affd640 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 0x5569ed49b000 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3900012880 tx=0x7f390000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341175ebfc661ce server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f38f1ffb640 1 -- 172.21.15.52:0/1370244043 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900004220 con 0x5569ed498b70 2024-06-02T23:34:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f38f1ffb640 1 -- 172.21.15.52:0/1370244043 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39000043c0 con 0x5569ed498b70 2024-06-02T23:34:51.273 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f38f1ffb640 1 -- 172.21.15.52:0/1370244043 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900003df0 con 0x5569ed498b70 2024-06-02T23:34:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 -- 172.21.15.52:0/1370244043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 msgr2=0x5569ed49b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 0x5569ed49b000 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f3900012880 tx=0x7f390000bbe0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f390bfff640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38f40a2cb0 0x7f38f40a5170 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f39040ee2b0 tx=0x7f390408cb40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 -- 172.21.15.52:0/1370244043 shutdown_connections 2024-06-02T23:34:51.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed498b70 0x5569ed49b000 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493a00 0x5569ed498500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 --2- 172.21.15.52:0/1370244043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed493430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 -- 172.21.15.52:0/1370244043 >> 172.21.15.52:0/1370244043 conn(0x5569ed48e680 msgr2=0x5569ed48ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 -- 172.21.15.52:0/1370244043 shutdown_connections 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.272+0000 7f3914b75740 1 -- 172.21.15.52:0/1370244043 wait complete. 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 -- start start 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 Processor -- start 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493010 0x5569ed49d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed49d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed498b70 0x5569ed4a4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5569ed49bb20 con 0x5569ed493a00 2024-06-02T23:34:51.278 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5569ed37fae0 con 0x5569ed493010 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5569ed49bca0 con 0x5569ed498b70 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed49d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f3914b75740 10 client.0 _ll_register_callbacks cb 0x5569ed3b89b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed49d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42438/0 (socket says 172.21.15.52:42438) 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f390b7fe640 1 -- 172.21.15.52:0/183887577 learned_addr learned my addr 172.21.15.52:0/183887577 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.273+0000 7f390affd640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493010 0x5569ed49d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390affd640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed498b70 0x5569ed4a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390b7fe640 1 -- 172.21.15.52:0/183887577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed498b70 msgr2=0x5569ed4a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390b7fe640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed498b70 0x5569ed4a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390b7fe640 1 -- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493010 msgr2=0x5569ed49d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390b7fe640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493010 0x5569ed49d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390b7fe640 1 -- 172.21.15.52:0/183887577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ed10a210 con 0x5569ed493a00 2024-06-02T23:34:51.279 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f390b7fe640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed49d7c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f38fc012d20 tx=0x7f38fc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c86c9a66826a7334 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f38eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f38eaffd640 1 -- 172.21.15.52:0/183887577 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc012110 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f38eaffd640 1 -- 172.21.15.52:0/183887577 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f38fc004050 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39000043c0 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f38eaffd640 1 -- 172.21.15.52:0/183887577 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f38fc023070 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f3914b75740 10 client.21471 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5569ed1d6620 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.274+0000 7f38eaffd640 1 -- 172.21.15.52:0/183887577 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f39000043c0 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f38eaffd640 1 -- 172.21.15.52:0/183887577 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f38fc010430 con 0x5569ed493a00 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f3914b75740 10 client.21471 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f3914b75740 10 client.21471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f3914b75740 20 client.21471 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f3914b75740 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed4a9120 0x5569ed4ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.280 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f3914b75740 4 client.21471 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569ed409f10 con 0x5569ed4a9120 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f390affd640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed4a9120 0x5569ed4ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f390affd640 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed4a9120 0x5569ed4ab590 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b191ec6b02ef1e server_cookie=1f3fa9b71c405740 in_seq=0 out_seq=0 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.275+0000 7f38eaffd640 10 client.21471 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f38eaffd640 1 -- 172.21.15.52:0/183887577 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5569ed409f10 con 0x5569ed4a9120 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f38eaffd640 10 client.21471 handle_command_reply: tid=0 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 client.21471 shutdown 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 msgr2=0x5569ed49d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed49d7c0 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f38fc012d20 tx=0x7f38fc00f790 comp rx=0 tx=0).stop 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 shutdown_connections 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed4a9120 0x5569ed4ab590 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed498b70 0x5569ed4a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed49d7c0 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.281 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 --2- 172.21.15.52:0/183887577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed493010 0x5569ed49d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.276+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 >> 172.21.15.52:0/183887577 conn(0x5569ed48e680 msgr2=0x5569ed48d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.277+0000 7f38eaffd640 0 client.21471 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.277+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 shutdown_connections 2024-06-02T23:34:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.277+0000 7f3914b75740 1 -- 172.21.15.52:0/183887577 wait complete. 2024-06-02T23:34:51.282 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.277+0000 7f3914b75740 20 client.21471 trim_cache size 0 max 16384 2024-06-02T23:34:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 -- 172.21.15.52:0/2878316146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102ee0 msgr2=0x7fbc9010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 --2- 172.21.15.52:0/2878316146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102ee0 0x7fbc9010f3c0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7fbc8800b0a0 tx=0x7fbc8800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 -- 172.21.15.52:0/2878316146 shutdown_connections 2024-06-02T23:34:51.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 --2- 172.21.15.52:0/2878316146 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102ee0 0x7fbc9010f3c0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 --2- 172.21.15.52:0/2878316146 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc90107450 0x7fbc90107830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 --2- 172.21.15.52:0/2878316146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc90102540 0x7fbc901029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 -- 172.21.15.52:0/2878316146 >> 172.21.15.52:0/2878316146 conn(0x7fbc900795f0 msgr2=0x7fbc90079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 -- 172.21.15.52:0/2878316146 shutdown_connections 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.278+0000 7fbc97c36700 1 -- 172.21.15.52:0/2878316146 wait complete. 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 Processor -- start 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 -- start start 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 0x7fbc9019e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc90102ee0 0x7fbc9019ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc90107450 0x7fbc901988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901126b0 con 0x7fbc90102540 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90112530 con 0x7fbc90102ee0 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc97c36700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc90112830 con 0x7fbc90107450 2024-06-02T23:34:51.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 0x7fbc9019e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 0x7fbc9019e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42462/0 (socket says 172.21.15.52:42462) 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc959d2700 1 -- 172.21.15.52:0/799278329 learned_addr learned my addr 172.21.15.52:0/799278329 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.279+0000 7fbc951d1700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc90102ee0 0x7fbc9019ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc961d3700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc90107450 0x7fbc901988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc959d2700 1 -- 172.21.15.52:0/799278329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc90107450 msgr2=0x7fbc901988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc959d2700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc90107450 0x7fbc901988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc959d2700 1 -- 172.21.15.52:0/799278329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc90102ee0 msgr2=0x7fbc9019ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc959d2700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc90102ee0 0x7fbc9019ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc959d2700 1 -- 172.21.15.52:0/799278329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc90199050 con 0x7fbc90102540 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc959d2700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 0x7fbc9019e730 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00d980 tx=0x7fbc8c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc86ffd700 1 -- 172.21.15.52:0/799278329 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c017030 con 0x7fbc90102540 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc86ffd700 1 -- 172.21.15.52:0/799278329 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc8c0044d0 con 0x7fbc90102540 2024-06-02T23:34:51.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc90199340 con 0x7fbc90102540 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc86ffd700 1 -- 172.21.15.52:0/799278329 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0039e0 con 0x7fbc90102540 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.280+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901a5600 con 0x7fbc90102540 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.284+0000 7fbc86ffd700 1 -- 172.21.15.52:0/799278329 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbc8c01a070 con 0x7fbc90102540 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbc86ffd700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc7c0a2ca0 0x7fbc7c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 Processor -- start 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbc86ffd700 1 -- 172.21.15.52:0/799278329 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbc8c0ce2c0 con 0x7fbc90102540 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbc951d1700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc7c0a2ca0 0x7fbc7c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 -- start start 2024-06-02T23:34:51.291 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fcf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fc8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fc93d0 0x559753fc9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559753f1a910 con 0x559753fcec40 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559753ec2820 con 0x559753fcf560 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559753f168d0 con 0x559753fc93d0 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.285+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fcf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fcf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42476/0 (socket says 172.21.15.52:42476) 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 -- 172.21.15.52:0/1891249741 learned_addr learned my addr 172.21.15.52:0/1891249741 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc951d1700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc7c0a2ca0 0x7fbc7c0a5160 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbc90102290 tx=0x7fbc8000b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc961d3700 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fc93d0 0x559753fc9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fc8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 -- 172.21.15.52:0/1891249741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fc93d0 msgr2=0x559753fc9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fc93d0 0x559753fc9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.292 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 -- 172.21.15.52:0/1891249741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 msgr2=0x559753fc8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fc8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 -- 172.21.15.52:0/1891249741 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559753eb9050 con 0x559753fcec40 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fc8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc959d2700 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fcf020 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c096f50 tx=0x7fbc8c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae49e768b2ca1dac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc7bfff700 1 -- 172.21.15.52:0/1891249741 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0b3070 con 0x559753fcec40 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc7bfff700 1 -- 172.21.15.52:0/1891249741 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc8c09eea0 con 0x559753fcec40 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.286+0000 7fbc7bfff700 1 -- 172.21.15.52:0/1891249741 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0a3530 con 0x559753fcec40 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1891249741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 msgr2=0x559753fcf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fcf020 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c096f50 tx=0x7fbc8c009b20 comp rx=0 tx=0).stop 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1891249741 shutdown_connections 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fc93d0 0x559753fc9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fcf020 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1891249741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fc8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1891249741 >> 172.21.15.52:0/1891249741 conn(0x559753ef3dc0 msgr2=0x559753fc4120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.293 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1891249741 shutdown_connections 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.287+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1891249741 wait complete. 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 -- start start 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 Processor -- start 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fd52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fd2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559753fd5f70 con 0x559753fc93d0 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559753ebacf0 con 0x559753fcf560 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559753fd60f0 con 0x559753fcec40 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbca24b9b80 10 client.0 _ll_register_callbacks cb 0x559753eb4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fd52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fd52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42486/0 (socket says 172.21.15.52:42486) 2024-06-02T23:34:51.294 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbc959d2700 1 -- 172.21.15.52:0/834012866 learned_addr learned my addr 172.21.15.52:0/834012866 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbc961d3700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.288+0000 7fbc951d1700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fd2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc959d2700 1 -- 172.21.15.52:0/834012866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 msgr2=0x559753fd2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc959d2700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fd2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc959d2700 1 -- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 msgr2=0x559753fd2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc959d2700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc959d2700 1 -- 172.21.15.52:0/834012866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559753e699d0 con 0x559753fc93d0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc961d3700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc959d2700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fd52e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009520 tx=0x7fbc8c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4983b1bea4072e92 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc797fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc797fa700 1 -- 172.21.15.52:0/834012866 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0b3040 con 0x559753fc93d0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc797fa700 1 -- 172.21.15.52:0/834012866 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbc8c0b9300 con 0x559753fc93d0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8c09eea0 con 0x559753fc93d0 2024-06-02T23:34:51.295 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc797fa700 1 -- 172.21.15.52:0/834012866 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc8c0b9500 con 0x559753fc93d0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbca24b9b80 10 client.21483 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559753fd5f70 con 0x559753fc93d0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.289+0000 7fbc797fa700 1 -- 172.21.15.52:0/834012866 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc8c09eea0 con 0x559753fc93d0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbc797fa700 1 -- 172.21.15.52:0/834012866 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbc8c0ab030 con 0x559753fc93d0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbca24b9b80 10 client.21483 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbca24b9b80 10 client.21483 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbca24b9b80 20 client.21483 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559753fe2bb0 0x559753fe4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbca24b9b80 4 client.21483 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559753f27e10 con 0x559753fe2bb0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.290+0000 7fbc951d1700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559753fe2bb0 0x559753fe4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbc951d1700 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559753fe2bb0 0x559753fe4fa0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d70b54c475aa7b5f server_cookie=8d84acc0ccc30f7f in_seq=0 out_seq=0 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbc797fa700 10 client.21483 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.296 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbc797fa700 1 -- 172.21.15.52:0/834012866 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559753f27e10 con 0x559753fe2bb0 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbc797fa700 10 client.21483 handle_command_reply: tid=0 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbca24b9b80 1 client.21483 shutdown 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 msgr2=0x559753fd52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.291+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fd52e0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c009520 tx=0x7fbc8c004750 comp rx=0 tx=0).stop 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 shutdown_connections 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559753fe2bb0 0x559753fe4fa0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fd52e0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/834012866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fd2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 >> 172.21.15.52:0/834012866 conn(0x559753ef3dc0 msgr2=0x559753fc7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbc797fa700 0 client.21483 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 shutdown_connections 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 1 -- 172.21.15.52:0/834012866 wait complete. 2024-06-02T23:34:51.297 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.292+0000 7fbca24b9b80 20 client.21483 trim_cache size 0 max 16384 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 Processor -- start 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 -- start start 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed49c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed4a47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed49d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1af270 con 0x5569ed493a00 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1b0db0 con 0x5569ed498b70 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f3914b75740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1af940 con 0x5569ed493010 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed4a47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.302 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed4a47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42488/0 (socket says 172.21.15.52:42488) 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 -- 172.21.15.52:0/619064714 learned_addr learned my addr 172.21.15.52:0/619064714 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390affd640 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed49c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390affd640 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed49d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 -- 172.21.15.52:0/619064714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 msgr2=0x5569ed49c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed49c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 -- 172.21.15.52:0/619064714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 msgr2=0x5569ed49d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed49d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390b7fe640 1 -- 172.21.15.52:0/619064714 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569ed48e4b0 con 0x5569ed493a00 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.296+0000 7f390affd640 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed49d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.297+0000 7f390b7fe640 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed4a47c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00a010 tx=0x7f38fc0123d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbfac6664a3bc0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.297+0000 7f38f17fa640 1 -- 172.21.15.52:0/619064714 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc00fe80 con 0x5569ed493a00 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.297+0000 7f38f17fa640 1 -- 172.21.15.52:0/619064714 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38fc02c030 con 0x5569ed493a00 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.297+0000 7f38f17fa640 1 -- 172.21.15.52:0/619064714 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38fc0035f0 con 0x5569ed493a00 2024-06-02T23:34:51.303 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.297+0000 7f3914b75740 1 -- 172.21.15.52:0/619064714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 msgr2=0x5569ed4a47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.297+0000 7f3914b75740 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed4a47c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f38fc00a010 tx=0x7f38fc0123d0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- 172.21.15.52:0/619064714 shutdown_connections 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed49d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493a00 0x5569ed4a47c0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 --2- 172.21.15.52:0/619064714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493010 0x5569ed49c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- 172.21.15.52:0/619064714 >> 172.21.15.52:0/619064714 conn(0x5569ed48e680 msgr2=0x5569ed48ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- 172.21.15.52:0/619064714 shutdown_connections 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- 172.21.15.52:0/619064714 wait complete. 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- start start 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 Processor -- start 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 0x5569ed4a4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493a00 0x5569ed4a10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed4a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.304 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1b06e0 con 0x5569ed493010 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1b0010 con 0x5569ed498b70 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f390c1a3820 con 0x5569ed493a00 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f3914b75740 10 client.0 _ll_register_callbacks cb 0x7f38fc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.298+0000 7f390affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 0x5569ed4a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 0x5569ed4a4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42494/0 (socket says 172.21.15.52:42494) 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 -- 172.21.15.52:0/365792600 learned_addr learned my addr 172.21.15.52:0/365792600 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 -- 172.21.15.52:0/365792600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493a00 msgr2=0x5569ed4a10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493a00 0x5569ed4a10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 -- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 msgr2=0x5569ed4a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed4a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 -- 172.21.15.52:0/365792600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569ed511e60 con 0x5569ed493010 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f390affd640 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 0x5569ed4a4600 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3900005940 tx=0x7f3900012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27cbbf2e43ba796a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.305 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f38eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f38eaffd640 1 -- 172.21.15.52:0/365792600 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3900010b10 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f38eaffd640 1 -- 172.21.15.52:0/365792600 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3900010cb0 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38fc02c030 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f38eaffd640 1 -- 172.21.15.52:0/365792600 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3900005030 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f3914b75740 10 client.21489 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5569ed512fe0 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.299+0000 7f38eaffd640 1 -- 172.21.15.52:0/365792600 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f38fc02c030 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f38eaffd640 1 -- 172.21.15.52:0/365792600 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f390001e440 con 0x5569ed493010 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f3914b75740 10 client.21489 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f3914b75740 10 client.21489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f3914b75740 20 client.21489 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f3914b75740 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed513800 0x5569ed515c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.306 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f3914b75740 4 client.21489 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569ed403c20 con 0x5569ed513800 2024-06-02T23:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.300+0000 7f390b7fe640 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed513800 0x5569ed515c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.301+0000 7f38eaffd640 10 client.21489 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.307 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.301+0000 7f390b7fe640 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed513800 0x5569ed515c20 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1c578864fc41bdc server_cookie=74f08cea65a67b7b in_seq=0 out_seq=0 2024-06-02T23:34:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.316+0000 7fbca24b9b80 1 Processor -- start 2024-06-02T23:34:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.316+0000 7fbca24b9b80 1 -- start start 2024-06-02T23:34:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fc93d0 0x559753fd01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fc3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcf560 0x559753fea6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901acb60 con 0x559753fcec40 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901ae6a0 con 0x559753fc93d0 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901ad230 con 0x559753fcf560 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fc93d0 0x559753fd01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fc93d0 0x559753fd01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:43812/0 (socket says 172.21.15.52:43812) 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc959d2700 1 -- 172.21.15.52:0/1059234300 learned_addr learned my addr 172.21.15.52:0/1059234300 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.317 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fc3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc961d3700 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcf560 0x559753fea6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc951d1700 1 -- 172.21.15.52:0/1059234300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcf560 msgr2=0x559753fea6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcf560 0x559753fea6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc951d1700 1 -- 172.21.15.52:0/1059234300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fc93d0 msgr2=0x559753fd01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fc93d0 0x559753fd01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.317+0000 7fbc951d1700 1 -- 172.21.15.52:0/1059234300 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559753fd0ba0 con 0x559753fcec40 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.318+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fc3f40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x559753fe5650 tx=0x7fbc80013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdcd30ed7d60e98 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.318+0000 7fbc7b7fe700 1 -- 172.21.15.52:0/1059234300 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80021110 con 0x559753fcec40 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.318+0000 7fbc7b7fe700 1 -- 172.21.15.52:0/1059234300 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc80004ce0 con 0x559753fcec40 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.318+0000 7fbc7b7fe700 1 -- 172.21.15.52:0/1059234300 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc80005020 con 0x559753fcec40 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1059234300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 msgr2=0x559753fc3f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fc3f40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x559753fe5650 tx=0x7fbc80013000 comp rx=0 tx=0).stop 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1059234300 shutdown_connections 2024-06-02T23:34:51.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcf560 0x559753fea6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fc93d0 0x559753fd01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1059234300 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fcec40 0x559753fc3f40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1059234300 >> 172.21.15.52:0/1059234300 conn(0x559753ef3dc0 msgr2=0x559753fc3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1059234300 shutdown_connections 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1059234300 wait complete. 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 -- start start 2024-06-02T23:34:51.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.319+0000 7fbca24b9b80 1 Processor -- start 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fdb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fdbb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fdb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbc959d2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fdb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42508/0 (socket says 172.21.15.52:42508) 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbc951d1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fdbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbc959d2700 1 -- 172.21.15.52:0/1000931901 learned_addr learned my addr 172.21.15.52:0/1000931901 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901adfd0 con 0x559753fc93d0 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901ad900 con 0x559753fcf560 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc901aab20 con 0x559753fcec40 2024-06-02T23:34:51.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbca24b9b80 10 client.0 _ll_register_callbacks cb 0x559753eb4a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.320+0000 7fbc961d3700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc959d2700 1 -- 172.21.15.52:0/1000931901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 msgr2=0x559753fdbb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc959d2700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fdbb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc959d2700 1 -- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 msgr2=0x559753fd8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc959d2700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc959d2700 1 -- 172.21.15.52:0/1000931901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559753fd8bc0 con 0x559753fc93d0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fdbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc959d2700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fdb600 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c096cc0 tx=0x7fbc8c096a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f83a9d81bfdf23 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc797fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc797fa700 1 -- 172.21.15.52:0/1000931901 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c0b9510 con 0x559753fc93d0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc797fa700 1 -- 172.21.15.52:0/1000931901 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbc8c0a34d0 con 0x559753fc93d0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc80004ce0 con 0x559753fc93d0 2024-06-02T23:34:51.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbc797fa700 1 -- 172.21.15.52:0/1000931901 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbc8c009aa0 con 0x559753fc93d0 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbca24b9b80 10 client.21495 fetch_fsmap learned FSMap version 43 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.321+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559753fd8bc0 con 0x559753fc93d0 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbc797fa700 1 -- 172.21.15.52:0/1000931901 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbc80004ce0 con 0x559753fc93d0 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbc797fa700 1 -- 172.21.15.52:0/1000931901 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbc8c0ab040 con 0x559753fc93d0 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbca24b9b80 10 client.21495 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbca24b9b80 10 client.21495 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbca24b9b80 20 client.21495 populate_metadata read hostname 'smithi052' 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559754045cb0 0x5597540480a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbca24b9b80 4 client.21495 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "5a0c4e17", "release": true, "await": true}] 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "5a0c4e17", "release": true, "await": true}) -- 0x559753f22390 con 0x559754045cb0 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559754045cb0 0x5597540480a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbc797fa700 10 client.21495 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.322+0000 7fbc951d1700 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559754045cb0 0x5597540480a0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42eef0f41e5fcdc9 server_cookie=9493e8eaa2f3b73b in_seq=0 out_seq=0 2024-06-02T23:34:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f38eaffd640 1 -- 172.21.15.52:0/365792600 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5569ed403c20 con 0x5569ed513800 2024-06-02T23:34:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f38eaffd640 10 client.21489 handle_command_reply: tid=0 2024-06-02T23:34:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 client.21489 shutdown 2024-06-02T23:34:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 msgr2=0x5569ed4a4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 0x5569ed4a4600 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f3900005940 tx=0x7f3900012530 comp rx=0 tx=0).stop 2024-06-02T23:34:51.343 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 shutdown_connections 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5569ed513800 0x5569ed515c20 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5569ed493a00 0x5569ed4a10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5569ed498b70 0x5569ed4a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f38eaffd640 0 client.21489 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 --2- 172.21.15.52:0/365792600 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5569ed493010 0x5569ed4a4600 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 >> 172.21.15.52:0/365792600 conn(0x5569ed48e680 msgr2=0x5569ed49c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 shutdown_connections 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 1 -- 172.21.15.52:0/365792600 wait complete. 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.343+0000 7f3914b75740 20 client.21489 trim_cache size 0 max 16384 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:34:51.344 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38f40a2cb0 msgr2=0x7f38f40a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38f40a2cb0 0x7f38f40a5170 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f39040ee2b0 tx=0x7f390408cb40 comp rx=0 tx=0).stop 2024-06-02T23:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 msgr2=0x7f390c19f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c19f100 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f3904031970 tx=0x7f3904003ba0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.345 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 shutdown_connections 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f38f40a2cb0 0x7f38f40a5170 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f390c107740 0x7f390c19f100 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f390c104e10 0x7f390c198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 --2- 172.21.15.52:0/1286478022 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f390c1024e0 0x7f390c197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 >> 172.21.15.52:0/1286478022 conn(0x7f390c0fbe70 msgr2=0x7f390c102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 shutdown_connections 2024-06-02T23:34:51.346 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:34:51.344+0000 7f3911e64640 1 -- 172.21.15.52:0/1286478022 wait complete. 2024-06-02T23:34:51.354 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.355+0000 7fbc797fa700 1 -- 172.21.15.52:0/1000931901 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559753f22390 con 0x559754045cb0 2024-06-02T23:34:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.355+0000 7fbc797fa700 10 client.21495 handle_command_reply: tid=0 2024-06-02T23:34:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.355+0000 7fbca24b9b80 1 client.21495 shutdown 2024-06-02T23:34:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.355+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 msgr2=0x559753fdb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.355+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fdb600 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c096cc0 tx=0x7fbc8c096a70 comp rx=0 tx=0).stop 2024-06-02T23:34:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 shutdown_connections 2024-06-02T23:34:51.355 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559754045cb0 0x5597540480a0 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbc797fa700 0 client.21495 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559753fcf560 0x559753fd8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559753fc93d0 0x559753fdb600 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 --2- 172.21.15.52:0/1000931901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559753fcec40 0x559753fdbb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 >> 172.21.15.52:0/1000931901 conn(0x559753ef3dc0 msgr2=0x559753fc40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 shutdown_connections 2024-06-02T23:34:51.356 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 1 -- 172.21.15.52:0/1000931901 wait complete. 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.356+0000 7fbca24b9b80 20 client.21495 trim_cache size 0 max 16384 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 76, 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "5a0c4e17": { 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 76, 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:34:51.357 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 150.0, 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:51.358 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc7c0a2ca0 msgr2=0x7fbc7c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc7c0a2ca0 0x7fbc7c0a5160 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbc90102290 tx=0x7fbc8000b3c0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 msgr2=0x7fbc9019e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 0x7fbc9019e730 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00d980 tx=0x7fbc8c00dc90 comp rx=0 tx=0).stop 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 shutdown_connections 2024-06-02T23:34:51.359 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbc7c0a2ca0 0x7fbc7c0a5160 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc90107450 0x7fbc901988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbc90102540 0x7fbc9019e730 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 --2- 172.21.15.52:0/799278329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbc90102ee0 0x7fbc9019ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:34:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 >> 172.21.15.52:0/799278329 conn(0x7fbc900795f0 msgr2=0x7fbc901004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:34:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 shutdown_connections 2024-06-02T23:34:51.360 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:34:51.357+0000 7fbc97c36700 1 -- 172.21.15.52:0/799278329 wait complete. 2024-06-02T23:34:51.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 68955 6.33 MB/sec execute 479 sec latency 28871.643 ms 2024-06-02T23:34:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:51 smithi174 ceph-mon[35994]: pgmap v1120: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 KiB/s wr, 72 op/s 2024-06-02T23:34:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:51 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:34:51.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:51 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:34:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:51 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:34:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:51 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:34:51.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:51 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:34:51.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:51 smithi052 ceph-mon[32340]: pgmap v1120: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 KiB/s wr, 72 op/s 2024-06-02T23:34:51.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:51 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:34:51.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:51 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:34:51.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:51 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:34:51.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:51 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:34:51.651 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:51 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:34:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:51 smithi123 ceph-mon[35977]: pgmap v1120: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 11 KiB/s wr, 72 op/s 2024-06-02T23:34:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:51 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:34:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:51 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:34:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:51 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:34:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:51 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:34:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:51 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:34:52.337 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '5a0c4e17', total seconds elapsed: 38.7 2024-06-02T23:34:52.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 69120 6.34 MB/sec execute 480 sec latency 57.369 ms 2024-06-02T23:34:53.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 69321 6.35 MB/sec execute 481 sec latency 50.465 ms 2024-06-02T23:34:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:53 smithi052 ceph-mon[32340]: pgmap v1121: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 KiB/s wr, 70 op/s 2024-06-02T23:34:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:53 smithi123 ceph-mon[35977]: pgmap v1121: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 KiB/s wr, 70 op/s 2024-06-02T23:34:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:53 smithi174 ceph-mon[35994]: pgmap v1121: 97 pgs: 97 active+clean; 552 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 11 KiB/s wr, 70 op/s 2024-06-02T23:34:54.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 69511 6.35 MB/sec execute 482 sec latency 78.638 ms 2024-06-02T23:34:55.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 69694 6.36 MB/sec execute 483 sec latency 66.183 ms 2024-06-02T23:34:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:55 smithi052 ceph-mon[32340]: pgmap v1122: 97 pgs: 97 active+clean; 554 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 257 KiB/s wr, 79 op/s 2024-06-02T23:34:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:55 smithi123 ceph-mon[35977]: pgmap v1122: 97 pgs: 97 active+clean; 554 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 257 KiB/s wr, 79 op/s 2024-06-02T23:34:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:55 smithi174 ceph-mon[35994]: pgmap v1122: 97 pgs: 97 active+clean; 554 MiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 257 KiB/s wr, 79 op/s 2024-06-02T23:34:56.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 69860 6.37 MB/sec execute 484 sec latency 59.446 ms 2024-06-02T23:34:57.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70025 6.39 MB/sec execute 485 sec latency 53.654 ms 2024-06-02T23:34:57.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:57 smithi052 ceph-mon[32340]: pgmap v1123: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-06-02T23:34:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:57 smithi123 ceph-mon[35977]: pgmap v1123: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-06-02T23:34:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:57 smithi174 ceph-mon[35994]: pgmap v1123: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 1.3 MiB/s wr, 124 op/s 2024-06-02T23:34:58.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70205 6.40 MB/sec execute 486 sec latency 52.119 ms 2024-06-02T23:34:59.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70379 6.41 MB/sec execute 487 sec latency 58.785 ms 2024-06-02T23:34:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:34:59 smithi123 ceph-mon[35977]: pgmap v1124: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-06-02T23:34:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:34:59 smithi174 ceph-mon[35994]: pgmap v1124: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-06-02T23:34:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:34:59 smithi052 ceph-mon[32340]: pgmap v1124: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 196 KiB/s rd, 1.6 MiB/s wr, 108 op/s 2024-06-02T23:35:00.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70509 6.40 MB/sec execute 488 sec latency 60.437 ms 2024-06-02T23:35:01.478 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70671 6.39 MB/sec execute 489 sec latency 54.161 ms 2024-06-02T23:35:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:01 smithi123 ceph-mon[35977]: pgmap v1125: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 2.9 MiB/s wr, 160 op/s 2024-06-02T23:35:01.815 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:01 smithi052 ceph-mon[32340]: pgmap v1125: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 2.9 MiB/s wr, 160 op/s 2024-06-02T23:35:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:01 smithi174 ceph-mon[35994]: pgmap v1125: 97 pgs: 97 active+clean; 560 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 417 KiB/s rd, 2.9 MiB/s wr, 160 op/s 2024-06-02T23:35:02.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70850 6.39 MB/sec execute 490 sec latency 63.967 ms 2024-06-02T23:35:03.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 70999 6.39 MB/sec execute 491 sec latency 61.399 ms 2024-06-02T23:35:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:03 smithi123 ceph-mon[35977]: pgmap v1126: 97 pgs: 97 active+clean; 562 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 3.4 MiB/s wr, 195 op/s 2024-06-02T23:35:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:03 smithi174 ceph-mon[35994]: pgmap v1126: 97 pgs: 97 active+clean; 562 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 3.4 MiB/s wr, 195 op/s 2024-06-02T23:35:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:03 smithi052 ceph-mon[32340]: pgmap v1126: 97 pgs: 97 active+clean; 562 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 741 KiB/s rd, 3.4 MiB/s wr, 195 op/s 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 -- 172.21.15.52:0/2967696560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c102480 msgr2=0x7f9e1c104870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 --2- 172.21.15.52:0/2967696560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c102480 0x7f9e1c104870 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009860 tx=0x7f9e1802a340 comp rx=0 tx=0).stop 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 -- 172.21.15.52:0/2967696560 shutdown_connections 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 --2- 172.21.15.52:0/2967696560 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c1076e0 0x7f9e1c109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 --2- 172.21.15.52:0/2967696560 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 0x7f9e1c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 --2- 172.21.15.52:0/2967696560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c102480 0x7f9e1c104870 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 -- 172.21.15.52:0/2967696560 >> 172.21.15.52:0/2967696560 conn(0x7f9e1c0fbe70 msgr2=0x7f9e1c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:04.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 71170 6.40 MB/sec execute 492 sec latency 64.080 ms 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 -- 172.21.15.52:0/2967696560 shutdown_connections 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.478+0000 7f9e2409e640 1 -- 172.21.15.52:0/2967696560 wait complete. 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.479+0000 7f9e2409e640 1 Processor -- start 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.479+0000 7f9e2409e640 1 -- start start 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.479+0000 7f9e2409e640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 0x7f9e1c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.479+0000 7f9e2409e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 0x7f9e1c1980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.479+0000 7f9e2409e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c1076e0 0x7f9e1c19f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e2409e640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10d530 con 0x7f9e1c102480 2024-06-02T23:35:04.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e2409e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10d3b0 con 0x7f9e1c104db0 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e2409e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c10d6b0 con 0x7f9e1c1076e0 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 0x7f9e1c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 0x7f9e1c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43298/0 (socket says 172.21.15.52:43298) 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 -- 172.21.15.52:0/1851688886 learned_addr learned my addr 172.21.15.52:0/1851688886 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21639640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 0x7f9e1c1980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e2263b640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c1076e0 0x7f9e1c19f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 -- 172.21.15.52:0/1851688886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c1076e0 msgr2=0x7f9e1c19f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c1076e0 0x7f9e1c19f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 -- 172.21.15.52:0/1851688886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 msgr2=0x7f9e1c1980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 0x7f9e1c1980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 -- 172.21.15.52:0/1851688886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e1c19f7c0 con 0x7f9e1c102480 2024-06-02T23:35:04.483 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21639640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 0x7f9e1c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.480+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 0x7f9e1c197ba0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f9e18005940 tx=0x7f9e18030e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.481+0000 7f9e12ffd640 1 -- 172.21.15.52:0/1851688886 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18037050 con 0x7f9e1c102480 2024-06-02T23:35:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.481+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e1c19fa50 con 0x7f9e1c102480 2024-06-02T23:35:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.481+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e1c19ff90 con 0x7f9e1c102480 2024-06-02T23:35:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.481+0000 7f9e12ffd640 1 -- 172.21.15.52:0/1851688886 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e18043070 con 0x7f9e1c102480 2024-06-02T23:35:04.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.481+0000 7f9e12ffd640 1 -- 172.21.15.52:0/1851688886 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e18003c60 con 0x7f9e1c102480 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 Processor -- start 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e12ffd640 1 -- 172.21.15.52:0/1851688886 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f9e18004040 con 0x7f9e1c102480 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 -- start start 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5010 0x5619d32e5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ea500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32ed000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5619d32484f0 con 0x5619d32e5010 2024-06-02T23:35:04.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5619d31e6870 con 0x5619d32eab70 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e26daf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5619d32444b0 con 0x5619d32e5a00 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e2263b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5010 0x5619d32e5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ea500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e2263b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5010 0x5619d32e5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43302/0 (socket says 172.21.15.52:43302) 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e2263b640 1 -- 172.21.15.52:0/741124843 learned_addr learned my addr 172.21.15.52:0/741124843 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.484+0000 7f9e21639640 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32ed000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e12ffd640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9e080a2d30 0x7f9e080a51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e21639640 1 -- 172.21.15.52:0/741124843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 msgr2=0x5619d32ea500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e21639640 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ea500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e21639640 1 -- 172.21.15.52:0/741124843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5010 msgr2=0x5619d32e5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e21639640 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5010 0x5619d32e5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e21639640 1 -- 172.21.15.52:0/741124843 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d32029b0 con 0x5619d32eab70 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e2263b640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9e080a2d30 0x7f9e080a51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e12ffd640 1 -- 172.21.15.52:0/1851688886 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9e180ef650 con 0x7f9e1c102480 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e21639640 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32ed000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c069bb0 tx=0x7f9e0c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3fa8f391dba46de server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e07fff640 1 -- 172.21.15.52:0/741124843 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c004030 con 0x5619d32eab70 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.485+0000 7f9e07fff640 1 -- 172.21.15.52:0/741124843 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e0c0041d0 con 0x5619d32eab70 2024-06-02T23:35:04.488 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e07fff640 1 -- 172.21.15.52:0/741124843 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c00f7e0 con 0x5619d32eab70 2024-06-02T23:35:04.490 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 -- 172.21.15.52:0/741124843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 msgr2=0x5619d32ed000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e2263b640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9e080a2d30 0x7f9e080a51f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009fd0 tx=0x7f9e14004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32ed000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c069bb0 tx=0x7f9e0c0118d0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 -- 172.21.15.52:0/741124843 shutdown_connections 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ea500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32ed000 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 --2- 172.21.15.52:0/741124843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5010 0x5619d32e5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.486+0000 7f9e26daf740 1 -- 172.21.15.52:0/741124843 >> 172.21.15.52:0/741124843 conn(0x5619d32e0680 msgr2=0x5619d32e0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 -- 172.21.15.52:0/741124843 shutdown_connections 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 -- 172.21.15.52:0/741124843 wait complete. 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 -- start start 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 Processor -- start 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.491 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d32f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5619d32edbc0 con 0x5619d32eab70 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5619d31d1ae0 con 0x5619d32e5010 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.487+0000 7f9e26daf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5619d32edd40 con 0x5619d32e5a00 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e26daf740 10 client.0 _ll_register_callbacks cb 0x5619d320a9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37720/0 (socket says 172.21.15.52:37720) 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 -- 172.21.15.52:0/580996929 learned_addr learned my addr 172.21.15.52:0/580996929 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 -- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 msgr2=0x5619d32ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e2263b640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d32f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 -- 172.21.15.52:0/580996929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 msgr2=0x5619d32f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d32f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 -- 172.21.15.52:0/580996929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d2f5c210 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e2263b640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d32f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e21639640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01b040 tx=0x7f9e0c003c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=650f71001d4b59e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.488+0000 7f9e057fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e057fa640 1 -- 172.21.15.52:0/580996929 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c028020 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e0c0041d0 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e057fa640 1 -- 172.21.15.52:0/580996929 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9e0c016040 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e057fa640 1 -- 172.21.15.52:0/580996929 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9e0c004380 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 10 client.30319 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5619d3028620 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e057fa640 1 -- 172.21.15.52:0/580996929 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e0c0041d0 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e057fa640 1 -- 172.21.15.52:0/580996929 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9e0c004b50 con 0x5619d32e5a00 2024-06-02T23:35:04.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 10 client.30319 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 10 client.30319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 20 client.30319 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d32fb5c0 0x5619d32fda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 4 client.30319 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619d325bf10 con 0x5619d32fb5c0 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.489+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d32fb5c0 0x5619d32fda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.490+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d32fb5c0 0x5619d32fda20 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe31a32ee0e0e1a server_cookie=345de868b113a25e in_seq=0 out_seq=0 2024-06-02T23:35:04.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.490+0000 7f9e057fa640 10 client.30319 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.491+0000 7f9e057fa640 1 -- 172.21.15.52:0/580996929 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5619d325bf10 con 0x5619d32fb5c0 2024-06-02T23:35:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.491+0000 7f9e057fa640 10 client.30319 handle_command_reply: tid=0 2024-06-02T23:35:04.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.491+0000 7f9e26daf740 1 client.30319 shutdown 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.491+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 msgr2=0x5619d32ef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c01b040 tx=0x7f9e0c003c50 comp rx=0 tx=0).stop 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 shutdown_connections 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d32fb5c0 0x5619d32fda20 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d32f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 --2- 172.21.15.52:0/580996929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 >> 172.21.15.52:0/580996929 conn(0x5619d32e0680 msgr2=0x5619d32df7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e057fa640 0 client.30319 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 shutdown_connections 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 1 -- 172.21.15.52:0/580996929 wait complete. 2024-06-02T23:35:04.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.492+0000 7f9e26daf740 20 client.30319 trim_cache size 0 max 16384 2024-06-02T23:35:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 Processor -- start 2024-06-02T23:35:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 -- start start 2024-06-02T23:35:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1a8de0 con 0x5619d32e5a00 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1aa920 con 0x5619d32eab70 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e26daf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1a94b0 con 0x5619d32e5010 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e21639640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:37732/0 (socket says 172.21.15.52:37732) 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e21639640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:43320/0 (socket says 172.21.15.52:43320) 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e21e3a640 1 -- 172.21.15.52:0/328340237 learned_addr learned my addr 172.21.15.52:0/328340237 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.520+0000 7f9e2263b640 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e21639640 1 -- 172.21.15.52:0/328340237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5010 msgr2=0x5619d32ef350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e21639640 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e21639640 1 -- 172.21.15.52:0/328340237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 msgr2=0x5619d32f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e21639640 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e21639640 1 -- 172.21.15.52:0/328340237 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619d32e04b0 con 0x5619d32e5a00 2024-06-02T23:35:04.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e21639640 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c004f20 tx=0x7f9e0c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7c351d4b17df16 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e077fe640 1 -- 172.21.15.52:0/328340237 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c028070 con 0x5619d32e5a00 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e077fe640 1 -- 172.21.15.52:0/328340237 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e0c02c3a0 con 0x5619d32e5a00 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.521+0000 7f9e077fe640 1 -- 172.21.15.52:0/328340237 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c02cb30 con 0x5619d32e5a00 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 -- 172.21.15.52:0/328340237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 msgr2=0x5619d32ef890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c004f20 tx=0x7f9e0c005fd0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 -- 172.21.15.52:0/328340237 shutdown_connections 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32eab70 0x5619d32f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.526 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32e5a00 0x5619d32ef890 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 --2- 172.21.15.52:0/328340237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5010 0x5619d32ef350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 -- 172.21.15.52:0/328340237 >> 172.21.15.52:0/328340237 conn(0x5619d32e0680 msgr2=0x5619d32db560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 -- 172.21.15.52:0/328340237 shutdown_connections 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 -- 172.21.15.52:0/328340237 wait complete. 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.522+0000 7f9e26daf740 1 -- start start 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 Processor -- start 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d3360520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d3360a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.527 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d33677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1aa250 con 0x5619d32eab70 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1a9b80 con 0x5619d32e5010 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e1c1a6dc0 con 0x5619d32e5a00 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d3360520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e26daf740 10 client.0 _ll_register_callbacks cb 0x7f9e14009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e21e3a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d3360520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37650/0 (socket says 172.21.15.52:37650) 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e21e3a640 1 -- 172.21.15.52:0/3795743213 learned_addr learned my addr 172.21.15.52:0/3795743213 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e21639640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d3360a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.523+0000 7f9e2263b640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d33677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e21e3a640 1 -- 172.21.15.52:0/3795743213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 msgr2=0x5619d3360a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d3360a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e21e3a640 1 -- 172.21.15.52:0/3795743213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 msgr2=0x5619d33677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:04.528 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d33677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e21e3a640 1 -- 172.21.15.52:0/3795743213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619d3367d90 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e2263b640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d33677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e21e3a640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d3360520 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009830 tx=0x7f9e180dc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cbcafa78c6db71dc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e057fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e057fa640 1 -- 172.21.15.52:0/3795743213 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e180bbc50 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e057fa640 1 -- 172.21.15.52:0/3795743213 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f9e180ba180 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e0c02c3a0 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e057fa640 1 -- 172.21.15.52:0/3795743213 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f9e180d8050 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e26daf740 10 client.30743 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.524+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5619d3369110 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e057fa640 1 -- 172.21.15.52:0/3795743213 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f9e18038040 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e057fa640 1 -- 172.21.15.52:0/3795743213 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f9e180b84f0 con 0x5619d32e5010 2024-06-02T23:35:04.529 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e26daf740 10 client.30743 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e26daf740 10 client.30743 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e26daf740 20 client.30743 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e26daf740 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d3369ae0 0x5619d336bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e26daf740 4 client.30743 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f9e0c02c3a0 con 0x5619d3369ae0 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.525+0000 7f9e21639640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d3369ae0 0x5619d336bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.526+0000 7f9e057fa640 10 client.30743 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:04.530 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.526+0000 7f9e21639640 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d3369ae0 0x5619d336bf00 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35128b17d48dcdd server_cookie=718cd22be243c67 in_seq=0 out_seq=0 2024-06-02T23:35:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e057fa640 1 -- 172.21.15.52:0/3795743213 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7f9e0c02c3a0 con 0x5619d3369ae0 2024-06-02T23:35:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e057fa640 10 client.30743 handle_command_reply: tid=0 2024-06-02T23:35:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 client.30743 shutdown 2024-06-02T23:35:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 msgr2=0x5619d3360520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d3360520 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9e18009830 tx=0x7f9e180dc040 comp rx=0 tx=0).stop 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 shutdown_connections 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5619d3369ae0 0x5619d336bf00 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5619d32e5a00 0x5619d3360a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5619d32eab70 0x5619d33677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e057fa640 0 client.30743 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 --2- 172.21.15.52:0/3795743213 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5619d32e5010 0x5619d3360520 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 >> 172.21.15.52:0/3795743213 conn(0x5619d32e0680 msgr2=0x5619d32e6c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 shutdown_connections 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.590+0000 7f9e26daf740 1 -- 172.21.15.52:0/3795743213 wait complete. 2024-06-02T23:35:04.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e26daf740 20 client.30743 trim_cache size 0 max 16384 2024-06-02T23:35:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:35:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:35:04.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9e080a2d30 msgr2=0x7f9e080a51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9e080a2d30 0x7f9e080a51f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f9e14009fd0 tx=0x7f9e14004a40 comp rx=0 tx=0).stop 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 msgr2=0x7f9e1c197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 0x7f9e1c197ba0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f9e18005940 tx=0x7f9e18030e00 comp rx=0 tx=0).stop 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 shutdown_connections 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f9e080a2d30 0x7f9e080a51f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e1c1076e0 0x7f9e1c19f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9e1c104db0 0x7f9e1c1980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 --2- 172.21.15.52:0/1851688886 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f9e1c102480 0x7f9e1c197ba0 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 >> 172.21.15.52:0/1851688886 conn(0x7f9e1c0fbe70 msgr2=0x7f9e1c102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 shutdown_connections 2024-06-02T23:35:04.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:04.591+0000 7f9e2409e640 1 -- 172.21.15.52:0/1851688886 wait complete. 2024-06-02T23:35:05.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 71349 6.41 MB/sec execute 493 sec latency 92.000 ms 2024-06-02T23:35:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:05 smithi123 ceph-mon[35977]: pgmap v1127: 97 pgs: 97 active+clean; 562 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 3.5 MiB/s wr, 216 op/s 2024-06-02T23:35:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:05 smithi174 ceph-mon[35994]: pgmap v1127: 97 pgs: 97 active+clean; 562 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 3.5 MiB/s wr, 216 op/s 2024-06-02T23:35:05.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:05 smithi052 ceph-mon[32340]: pgmap v1127: 97 pgs: 97 active+clean; 562 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 789 KiB/s rd, 3.5 MiB/s wr, 216 op/s 2024-06-02T23:35:06.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 71557 6.42 MB/sec execute 494 sec latency 56.343 ms 2024-06-02T23:35:07.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 71748 6.44 MB/sec execute 495 sec latency 63.162 ms 2024-06-02T23:35:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:07 smithi123 ceph-mon[35977]: pgmap v1128: 97 pgs: 97 active+clean; 559 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 296 op/s 2024-06-02T23:35:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:07 smithi174 ceph-mon[35994]: pgmap v1128: 97 pgs: 97 active+clean; 559 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 296 op/s 2024-06-02T23:35:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:07 smithi052 ceph-mon[32340]: pgmap v1128: 97 pgs: 97 active+clean; 559 MiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 296 op/s 2024-06-02T23:35:08.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 71908 6.43 MB/sec execute 496 sec latency 60.295 ms 2024-06-02T23:35:09.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 72087 6.44 MB/sec execute 497 sec latency 58.208 ms 2024-06-02T23:35:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:09 smithi123 ceph-mon[35977]: pgmap v1129: 97 pgs: 97 active+clean; 559 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-06-02T23:35:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:09 smithi174 ceph-mon[35994]: pgmap v1129: 97 pgs: 97 active+clean; 559 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-06-02T23:35:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:09 smithi052 ceph-mon[32340]: pgmap v1129: 97 pgs: 97 active+clean; 559 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 275 op/s 2024-06-02T23:35:10.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 72242 6.44 MB/sec execute 498 sec latency 65.638 ms 2024-06-02T23:35:11.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 72419 6.46 MB/sec execute 499 sec latency 61.540 ms 2024-06-02T23:35:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:11 smithi123 ceph-mon[35977]: pgmap v1130: 97 pgs: 97 active+clean; 564 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 326 op/s 2024-06-02T23:35:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:11 smithi174 ceph-mon[35994]: pgmap v1130: 97 pgs: 97 active+clean; 564 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 326 op/s 2024-06-02T23:35:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:11 smithi052 ceph-mon[32340]: pgmap v1130: 97 pgs: 97 active+clean; 564 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.5 MiB/s wr, 326 op/s 2024-06-02T23:35:12.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 72596 6.46 MB/sec execute 500 sec latency 150.863 ms 2024-06-02T23:35:13.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 72782 6.47 MB/sec execute 501 sec latency 58.964 ms 2024-06-02T23:35:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:13 smithi123 ceph-mon[35977]: pgmap v1131: 97 pgs: 97 active+clean; 563 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 285 op/s 2024-06-02T23:35:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:13 smithi174 ceph-mon[35994]: pgmap v1131: 97 pgs: 97 active+clean; 563 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 285 op/s 2024-06-02T23:35:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:13 smithi052 ceph-mon[32340]: pgmap v1131: 97 pgs: 97 active+clean; 563 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.8 MiB/s wr, 285 op/s 2024-06-02T23:35:14.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 72959 6.48 MB/sec execute 502 sec latency 104.148 ms 2024-06-02T23:35:15.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 73127 6.48 MB/sec execute 503 sec latency 74.690 ms 2024-06-02T23:35:15.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:15 smithi174 ceph-mon[35994]: pgmap v1132: 97 pgs: 97 active+clean; 565 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-06-02T23:35:15.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:15 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:35:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:15 smithi123 ceph-mon[35977]: pgmap v1132: 97 pgs: 97 active+clean; 565 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-06-02T23:35:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:15 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:35:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:15 smithi052 ceph-mon[32340]: pgmap v1132: 97 pgs: 97 active+clean; 565 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1021 KiB/s rd, 3.5 MiB/s wr, 267 op/s 2024-06-02T23:35:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:15 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:35:16.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 73297 6.49 MB/sec execute 504 sec latency 66.447 ms 2024-06-02T23:35:17.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 73456 6.50 MB/sec execute 505 sec latency 63.478 ms 2024-06-02T23:35:17.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.716+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1114032329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b741024b0 msgr2=0x7f2b74108d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.716+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1114032329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b741024b0 0x7f2b74108d40 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c009ca0 tx=0x7f2b6c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1114032329 shutdown_connections 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1114032329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b74101af0 0x7f2b74101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1114032329 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b741024b0 0x7f2b74108d40 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1114032329 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 0x7f2b74100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1114032329 >> 172.21.15.52:0/1114032329 conn(0x7f2b740fbe90 msgr2=0x7f2b740fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1114032329 shutdown_connections 2024-06-02T23:35:17.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.717+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1114032329 wait complete. 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 Processor -- start 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 -- start start 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 0x7f2b74199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 0x7f2b7419a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b741024b0 0x7f2b741a1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7410c480 con 0x7f2b74101af0 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7410c300 con 0x7f2b74100900 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7c2ca640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7410c600 con 0x7f2b741024b0 2024-06-02T23:35:17.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b79865640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 0x7f2b7419a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b79865640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 0x7f2b7419a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46832/0 (socket says 172.21.15.52:46832) 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b79865640 1 -- 172.21.15.52:0/1104385728 learned_addr learned my addr 172.21.15.52:0/1104385728 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.718+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 0x7f2b74199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a066640 1 -- 172.21.15.52:0/1104385728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b741024b0 msgr2=0x7f2b741a1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a867640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b741024b0 0x7f2b741a1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b741024b0 0x7f2b741a1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a066640 1 -- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 msgr2=0x7f2b7419a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 0x7f2b7419a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a066640 1 -- 172.21.15.52:0/1104385728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b741a1900 con 0x7f2b74100900 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b79865640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 0x7f2b7419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a867640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b741024b0 0x7f2b741a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:35:17.722 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 0x7f2b74199cc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000e9c0 tx=0x7f2b7000ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b6b7fe640 1 -- 172.21.15.52:0/1104385728 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7000cd80 con 0x7f2b74100900 2024-06-02T23:35:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b741a1bf0 con 0x7f2b74100900 2024-06-02T23:35:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b6b7fe640 1 -- 172.21.15.52:0/1104385728 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b700108e0 con 0x7f2b74100900 2024-06-02T23:35:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b6b7fe640 1 -- 172.21.15.52:0/1104385728 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70018860 con 0x7f2b74100900 2024-06-02T23:35:17.723 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.719+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b741a2190 con 0x7f2b74100900 2024-06-02T23:35:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.722+0000 7f2b6b7fe640 1 -- 172.21.15.52:0/1104385728 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2b7001f020 con 0x7f2b74100900 2024-06-02T23:35:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 Processor -- start 2024-06-02T23:35:17.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b6b7fe640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b600a2c30 0x7f2b600a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 -- start start 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b6b7fe640 1 -- 172.21.15.52:0/1104385728 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b700cefc0 con 0x7f2b74100900 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b79865640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b600a2c30 0x7f2b600a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 0x559e777d0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0a00 0x559e777d5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777d8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e777334f0 con 0x559e777d5b70 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e776d1870 con 0x559e777d0a00 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.723+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559e7772f4b0 con 0x559e777d0010 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 0x559e777d0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 0x559e777d0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:32804/0 (socket says 172.21.15.52:32804) 2024-06-02T23:35:17.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 -- 172.21.15.52:0/1437581099 learned_addr learned my addr 172.21.15.52:0/1437581099 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a867640 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b79865640 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0a00 0x559e777d5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b79865640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b600a2c30 0x7f2b600a50f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2b7419b1c0 tx=0x7f2b64011920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 -- 172.21.15.52:0/1437581099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0a00 msgr2=0x559e777d5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0a00 0x559e777d5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 -- 172.21.15.52:0/1437581099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 msgr2=0x559e777d8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777d8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 -- 172.21.15.52:0/1437581099 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e776ed9b0 con 0x559e777d0010 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a867640 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.724+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 0x559e777d0430 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ccb30 tx=0x7f2b70010f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dbd3e53cf9b5d37 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.725+0000 7f2b68bfe640 1 -- 172.21.15.52:0/1437581099 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b7009fca0 con 0x559e777d0010 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.725+0000 7f2b68bfe640 1 -- 172.21.15.52:0/1437581099 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b700a4560 con 0x559e777d0010 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.725+0000 7f2b68bfe640 1 -- 172.21.15.52:0/1437581099 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b70086320 con 0x559e777d0010 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.725+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1437581099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 msgr2=0x559e777d0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.725+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 0x559e777d0430 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2b700ccb30 tx=0x7f2b70010f70 comp rx=0 tx=0).stop 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1437581099 shutdown_connections 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777d8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0a00 0x559e777d5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1437581099 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0010 0x559e777d0430 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1437581099 >> 172.21.15.52:0/1437581099 conn(0x559e777cb680 msgr2=0x559e777cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1437581099 shutdown_connections 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1437581099 wait complete. 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- start start 2024-06-02T23:35:17.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 Processor -- start 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777e1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559e777d8a60 con 0x559e777d5b70 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.726+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559e776bcae0 con 0x559e777d0010 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559e777d8be0 con 0x559e777d0a00 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.733 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777da2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55130/0 (socket says 172.21.15.52:55130) 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 -- 172.21.15.52:0/2581592542 learned_addr learned my addr 172.21.15.52:0/2581592542 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7efd7740 10 client.0 _ll_register_callbacks cb 0x559e776f59b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a867640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777e1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b79865640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 -- 172.21.15.52:0/2581592542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 msgr2=0x559e777da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 -- 172.21.15.52:0/2581592542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 msgr2=0x559e777e1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777e1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 -- 172.21.15.52:0/2581592542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e77447210 con 0x559e777d0010 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a867640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7a066640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777da2a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2b700cf350 tx=0x7f2b7009fca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=927333d1c8d3d278 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:17.734 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b5dffb640 1 -- 172.21.15.52:0/2581592542 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700b3020 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b700a4560 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.727+0000 7f2b5dffb640 1 -- 172.21.15.52:0/2581592542 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b70086050 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b5dffb640 1 -- 172.21.15.52:0/2581592542 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2b70086250 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 10 client.30755 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559e77513620 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b5dffb640 1 -- 172.21.15.52:0/2581592542 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b700a4560 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b5dffb640 1 -- 172.21.15.52:0/2581592542 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2b70097640 con 0x559e777d0010 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 10 client.30755 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 10 client.30755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 20 client.30755 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777e60b0 0x559e777e8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 4 client.30755 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e77746f10 con 0x559e777e60b0 2024-06-02T23:35:17.735 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.728+0000 7f2b79865640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777e60b0 0x559e777e8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.729+0000 7f2b5dffb640 10 client.30755 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.729+0000 7f2b79865640 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777e60b0 0x559e777e8530 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b1eadd369d2cd4a server_cookie=68b71f687a03415d in_seq=0 out_seq=0 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.729+0000 7f2b5dffb640 1 -- 172.21.15.52:0/2581592542 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559e77746f10 con 0x559e777e60b0 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.729+0000 7f2b5dffb640 10 client.30755 handle_command_reply: tid=0 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 client.30755 shutdown 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 msgr2=0x559e777da2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777da2a0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f2b700cf350 tx=0x7f2b7009fca0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 shutdown_connections 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777e60b0 0x559e777e8530 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d5b70 0x559e777e1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/2581592542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777da2a0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 >> 172.21.15.52:0/2581592542 conn(0x559e777cb680 msgr2=0x559e777ca7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b5dffb640 0 client.30755 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:17.736 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 shutdown_connections 2024-06-02T23:35:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 1 -- 172.21.15.52:0/2581592542 wait complete. 2024-06-02T23:35:17.737 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.730+0000 7f2b7efd7740 20 client.30755 trim_cache size 0 max 16384 2024-06-02T23:35:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:17 smithi123 ceph-mon[35977]: pgmap v1133: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-06-02T23:35:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:35:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:35:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:35:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.760+0000 7f2b7efd7740 1 Processor -- start 2024-06-02T23:35:17.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.760+0000 7f2b7efd7740 1 -- start start 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777cb680 0x559e777da5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 0x559e777e17b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777ce590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741aadc0 con 0x559e777d0010 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741ac900 con 0x559e777cb680 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741ab490 con 0x559e777d0a00 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777cb680 0x559e777da5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 0x559e777e17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7a867640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7a867640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777ce590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:32824/0 (socket says 172.21.15.52:32824) 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 0x559e777e17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46852/0 (socket says 172.21.15.52:46852) 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7a867640 1 -- 172.21.15.52:0/4286009670 learned_addr learned my addr 172.21.15.52:0/4286009670 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:17.763 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777cb680 0x559e777da5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55134/0 (socket says 172.21.15.52:55134) 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 -- 172.21.15.52:0/4286009670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 msgr2=0x559e777ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 -- 172.21.15.52:0/4286009670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777cb680 msgr2=0x559e777da5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777cb680 0x559e777da5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b79865640 1 -- 172.21.15.52:0/4286009670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e776d60d0 con 0x559e777d0010 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.761+0000 7f2b7a867640 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.762+0000 7f2b79865640 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 0x559e777e17b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x559e777e8db0 tx=0x7f2b64018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488782decf2500cf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.762+0000 7f2b5ffff640 1 -- 172.21.15.52:0/4286009670 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64017030 con 0x559e777d0010 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.762+0000 7f2b5ffff640 1 -- 172.21.15.52:0/4286009670 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b64019050 con 0x559e777d0010 2024-06-02T23:35:17.764 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.762+0000 7f2b5ffff640 1 -- 172.21.15.52:0/4286009670 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b64028c80 con 0x559e777d0010 2024-06-02T23:35:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 -- 172.21.15.52:0/4286009670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 msgr2=0x559e777e17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 0x559e777e17b0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x559e777e8db0 tx=0x7f2b64018000 comp rx=0 tx=0).stop 2024-06-02T23:35:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 -- 172.21.15.52:0/4286009670 shutdown_connections 2024-06-02T23:35:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777d0010 0x559e777e17b0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.766 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/4286009670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777cb680 0x559e777da5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 -- 172.21.15.52:0/4286009670 >> 172.21.15.52:0/4286009670 conn(0x559e777d5640 msgr2=0x559e777d9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 -- 172.21.15.52:0/4286009670 shutdown_connections 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.763+0000 7f2b7efd7740 1 -- 172.21.15.52:0/4286009670 wait complete. 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 -- start start 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 Processor -- start 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777cb680 0x559e777e1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777de240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777de820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741ac230 con 0x559e777cb680 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741abb60 con 0x559e777d0010 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2b741a8da0 con 0x559e777d0a00 2024-06-02T23:35:17.767 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7efd7740 10 client.0 _ll_register_callbacks cb 0x7f2b7009b4f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7a066640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777cb680 0x559e777e1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.764+0000 7f2b7a867640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777de820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b79865640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777de240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b79865640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777de240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55136/0 (socket says 172.21.15.52:55136) 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b79865640 1 -- 172.21.15.52:0/1937271708 learned_addr learned my addr 172.21.15.52:0/1937271708 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b7a066640 1 -- 172.21.15.52:0/1937271708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 msgr2=0x559e777de820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777de820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b7a066640 1 -- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 msgr2=0x559e777de240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777de240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b7a066640 1 -- 172.21.15.52:0/1937271708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e7784eeb0 con 0x559e777cb680 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b79865640 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b7a066640 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777cb680 0x559e777e1750 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2b7009b5f0 tx=0x7f2b70005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e9ce39021ce20eb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:17.768 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b5dffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b5dffb640 1 -- 172.21.15.52:0/1937271708 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b700b3070 con 0x559e777cb680 2024-06-02T23:35:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.765+0000 7f2b5dffb640 1 -- 172.21.15.52:0/1937271708 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2b70010430 con 0x559e777cb680 2024-06-02T23:35:17.769 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.766+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2b64019050 con 0x559e777cb680 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.766+0000 7f2b5dffb640 1 -- 172.21.15.52:0/1937271708 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2b700860b0 con 0x559e777cb680 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.766+0000 7f2b7efd7740 10 client.21525 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.766+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559e77850160 con 0x559e777cb680 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.766+0000 7f2b5dffb640 1 -- 172.21.15.52:0/1937271708 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2b64019050 con 0x559e777cb680 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.766+0000 7f2b5dffb640 1 -- 172.21.15.52:0/1937271708 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2b700a0e30 con 0x559e777cb680 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b7efd7740 10 client.21525 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b7efd7740 10 client.21525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b7efd7740 20 client.21525 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:17.770 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777d5b70 0x559e778529b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b7efd7740 4 client.21525 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:35:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559e77740c20 con 0x559e777d5b70 2024-06-02T23:35:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.767+0000 7f2b79865640 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777d5b70 0x559e778529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.768+0000 7f2b79865640 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777d5b70 0x559e778529b0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e91d1340c3455365 server_cookie=70f220770c9c037f in_seq=0 out_seq=0 2024-06-02T23:35:17.771 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.768+0000 7f2b5dffb640 10 client.21525 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:17.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b5dffb640 1 -- 172.21.15.52:0/1937271708 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559e77740c20 con 0x559e777d5b70 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b5dffb640 10 client.21525 handle_command_reply: tid=0 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 client.21525 shutdown 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777cb680 msgr2=0x559e777e1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777cb680 0x559e777e1750 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2b7009b5f0 tx=0x7f2b70005af0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 shutdown_connections 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559e777d5b70 0x559e778529b0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559e777d0a00 0x559e777de820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b5dffb640 0 client.21525 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559e777d0010 0x559e777de240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 --2- 172.21.15.52:0/1937271708 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559e777cb680 0x559e777e1750 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.828+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 >> 172.21.15.52:0/1937271708 conn(0x559e777d5640 msgr2=0x559e777d9e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.829+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 shutdown_connections 2024-06-02T23:35:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.829+0000 7f2b7efd7740 1 -- 172.21.15.52:0/1937271708 wait complete. 2024-06-02T23:35:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.829+0000 7f2b7efd7740 20 client.21525 trim_cache size 0 max 16384 2024-06-02T23:35:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:35:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:35:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b600a2c30 msgr2=0x7f2b600a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b600a2c30 0x7f2b600a50f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f2b7419b1c0 tx=0x7f2b64011920 comp rx=0 tx=0).stop 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 msgr2=0x7f2b74199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 0x7f2b74199cc0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2b7000e9c0 tx=0x7f2b7000ee80 comp rx=0 tx=0).stop 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 shutdown_connections 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2b600a2c30 0x7f2b600a50f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2b741024b0 0x7f2b741a1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2b74101af0 0x7f2b7419a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 --2- 172.21.15.52:0/1104385728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b74100900 0x7f2b74199cc0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 >> 172.21.15.52:0/1104385728 conn(0x7f2b740fbe90 msgr2=0x7f2b7406a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 shutdown_connections 2024-06-02T23:35:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:17.830+0000 7f2b7c2ca640 1 -- 172.21.15.52:0/1104385728 wait complete. 2024-06-02T23:35:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:17 smithi174 ceph-mon[35994]: pgmap v1133: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-06-02T23:35:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:35:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:35:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:35:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:17 smithi052 ceph-mon[32340]: pgmap v1133: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 326 op/s 2024-06-02T23:35:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:35:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:35:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:35:18.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 73612 6.51 MB/sec execute 506 sec latency 79.402 ms 2024-06-02T23:35:19.474 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:35:19.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 73777 6.51 MB/sec execute 507 sec latency 59.386 ms 2024-06-02T23:35:19.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:19 smithi123 ceph-mon[35977]: pgmap v1134: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 257 op/s 2024-06-02T23:35:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:19 smithi174 ceph-mon[35994]: pgmap v1134: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 257 op/s 2024-06-02T23:35:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:19 smithi052 ceph-mon[32340]: pgmap v1134: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.1 MiB/s wr, 257 op/s 2024-06-02T23:35:20.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 73926 6.51 MB/sec execute 508 sec latency 60.367 ms 2024-06-02T23:35:21.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 74093 6.51 MB/sec execute 509 sec latency 65.158 ms 2024-06-02T23:35:21.634 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:21 smithi052 ceph-mon[32340]: pgmap v1135: 97 pgs: 97 active+clean; 574 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 296 op/s 2024-06-02T23:35:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:21 smithi123 ceph-mon[35977]: pgmap v1135: 97 pgs: 97 active+clean; 574 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 296 op/s 2024-06-02T23:35:21.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:21 smithi174 ceph-mon[35994]: pgmap v1135: 97 pgs: 97 active+clean; 574 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.9 MiB/s wr, 296 op/s 2024-06-02T23:35:21.929 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 -- 172.21.15.52:0/3886989824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06ab60 msgr2=0x7eff8c06afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 --2- 172.21.15.52:0/3886989824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06ab60 0x7eff8c06afe0 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7eff7c009990 tx=0x7eff7c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 -- 172.21.15.52:0/3886989824 shutdown_connections 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 --2- 172.21.15.52:0/3886989824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff8c06b520 0x7eff8c10f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 --2- 172.21.15.52:0/3886989824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 0x7eff8c108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 --2- 172.21.15.52:0/3886989824 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06ab60 0x7eff8c06afe0 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 -- 172.21.15.52:0/3886989824 >> 172.21.15.52:0/3886989824 conn(0x7eff8c078340 msgr2=0x7eff8c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 -- 172.21.15.52:0/3886989824 shutdown_connections 2024-06-02T23:35:21.930 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.929+0000 7eff93480700 1 -- 172.21.15.52:0/3886989824 wait complete. 2024-06-02T23:35:21.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 Processor -- start 2024-06-02T23:35:21.931 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 -- start start 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff8c06ab60 0x7eff8c19e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 0x7eff8c19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 0x7eff8c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c112680 con 0x7eff8c06b520 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c112500 con 0x7eff8c108520 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff93480700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c112800 con 0x7eff8c06ab60 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff90a1b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 0x7eff8c19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.930+0000 7eff90a1b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 0x7eff8c19ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46880/0 (socket says 172.21.15.52:46880) 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 -- 172.21.15.52:0/665535709 learned_addr learned my addr 172.21.15.52:0/665535709 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff91a1d700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 0x7eff8c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff9121c700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff8c06ab60 0x7eff8c19e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 -- 172.21.15.52:0/665535709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff8c06ab60 msgr2=0x7eff8c19e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.932 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff8c06ab60 0x7eff8c19e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 -- 172.21.15.52:0/665535709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 msgr2=0x7eff8c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 0x7eff8c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 -- 172.21.15.52:0/665535709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c199120 con 0x7eff8c06b520 2024-06-02T23:35:21.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff91a1d700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 0x7eff8c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:21.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff90a1b700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 0x7eff8c19ed40 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000c00 tx=0x7eff7c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:21.933 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.931+0000 7eff827fc700 1 -- 172.21.15.52:0/665535709 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c027070 con 0x7eff8c06b520 2024-06-02T23:35:21.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.932+0000 7eff827fc700 1 -- 172.21.15.52:0/665535709 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff7c022470 con 0x7eff8c06b520 2024-06-02T23:35:21.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.932+0000 7eff827fc700 1 -- 172.21.15.52:0/665535709 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff7c00b610 con 0x7eff8c06b520 2024-06-02T23:35:21.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.932+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c1993b0 con 0x7eff8c06b520 2024-06-02T23:35:21.934 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.932+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c101770 con 0x7eff8c06b520 2024-06-02T23:35:21.935 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.935+0000 7eff827fc700 1 -- 172.21.15.52:0/665535709 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7eff7c02a070 con 0x7eff8c06b520 2024-06-02T23:35:21.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.936+0000 7eff827fc700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff780a2ca0 0x7eff780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.936+0000 7eff827fc700 1 -- 172.21.15.52:0/665535709 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7eff7c0cfdf0 con 0x7eff8c06b520 2024-06-02T23:35:21.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.937+0000 7eff9121c700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff780a2ca0 0x7eff780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.937+0000 7eff9121c700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff780a2ca0 0x7eff780a5160 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7eff8c199b00 tx=0x7eff88009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:21.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.937+0000 7eff9dd03b80 1 Processor -- start 2024-06-02T23:35:21.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.937+0000 7eff9dd03b80 1 -- start start 2024-06-02T23:35:21.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebda340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebd4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebda340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f48eac6f60 con 0x55f48ebda880 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f48eb25850 con 0x55f48ebd47a0 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f48ea8d460 con 0x55f48ebd9f60 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebda340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:32844/0 (socket says 172.21.15.52:32844) 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 -- 172.21.15.52:0/983789750 learned_addr learned my addr 172.21.15.52:0/983789750 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff90a1b700 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebd4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff91a1d700 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 -- 172.21.15.52:0/983789750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 msgr2=0x55f48ebd4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 -- 172.21.15.52:0/983789750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 msgr2=0x55f48ebd4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.938+0000 7eff9121c700 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebd4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.939+0000 7eff9121c700 1 -- 172.21.15.52:0/983789750 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f48ea748d0 con 0x55f48ebd9f60 2024-06-02T23:35:21.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.939+0000 7eff9121c700 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebda340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7eff8800fbe0 tx=0x7eff88012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e260283324617215 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:21.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.939+0000 7eff777fe700 1 -- 172.21.15.52:0/983789750 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88022070 con 0x55f48ebd9f60 2024-06-02T23:35:21.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.939+0000 7eff777fe700 1 -- 172.21.15.52:0/983789750 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8801e450 con 0x55f48ebd9f60 2024-06-02T23:35:21.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.939+0000 7eff777fe700 1 -- 172.21.15.52:0/983789750 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8801d610 con 0x55f48ebd9f60 2024-06-02T23:35:21.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 -- 172.21.15.52:0/983789750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 msgr2=0x55f48ebda340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebda340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7eff8800fbe0 tx=0x7eff88012000 comp rx=0 tx=0).stop 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 -- 172.21.15.52:0/983789750 shutdown_connections 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebda340 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/983789750 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebd4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 -- 172.21.15.52:0/983789750 >> 172.21.15.52:0/983789750 conn(0x55f48eb01080 msgr2=0x55f48ebcf220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 -- 172.21.15.52:0/983789750 shutdown_connections 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.940+0000 7eff9dd03b80 1 -- 172.21.15.52:0/983789750 wait complete. 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 -- start start 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 Processor -- start 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 0x55f48ebe0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebdd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebddea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55f48eac5fc0 con 0x55f48ebd47a0 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f48eb21810 con 0x55f48ebda880 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f48ebe1260 con 0x55f48ebd9f60 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9dd03b80 10 client.0 _ll_register_callbacks cb 0x55f48eabb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9121c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 0x55f48ebe0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9121c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 0x55f48ebe0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:46888/0 (socket says 172.21.15.52:46888) 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.941+0000 7eff9121c700 1 -- 172.21.15.52:0/1523129940 learned_addr learned my addr 172.21.15.52:0/1523129940 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff90a1b700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebdd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff91a1d700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebddea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9121c700 1 -- 172.21.15.52:0/1523129940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 msgr2=0x55f48ebdd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9121c700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebdd960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9121c700 1 -- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 msgr2=0x55f48ebddea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9121c700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebddea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9121c700 1 -- 172.21.15.52:0/1523129940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f48eaeb750 con 0x55f48ebd47a0 2024-06-02T23:35:21.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff91a1d700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9121c700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 0x55f48ebe0660 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7eff88009fd0 tx=0x7eff8801e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e43b15bc355746b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff74ff9700 1 -- 172.21.15.52:0/1523129940 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88035060 con 0x55f48ebd47a0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff74ff9700 1 -- 172.21.15.52:0/1523129940 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7eff88011ae0 con 0x55f48ebd47a0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.942+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8801e450 con 0x55f48ebd47a0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff74ff9700 1 -- 172.21.15.52:0/1523129940 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7eff88022070 con 0x55f48ebd47a0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 10 client.21537 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f48ebeba60 con 0x55f48ebd47a0 2024-06-02T23:35:21.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff74ff9700 1 -- 172.21.15.52:0/1523129940 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8801e450 con 0x55f48ebd47a0 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff74ff9700 1 -- 172.21.15.52:0/1523129940 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7eff880047f0 con 0x55f48ebd47a0 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 10 client.21537 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 10 client.21537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 20 client.21537 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ebee3b0 0x55f48ebf0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 4 client.21537 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.943+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f48eb32d50 con 0x55f48ebee3b0 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.944+0000 7eff90a1b700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ebee3b0 0x55f48ebf0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.944+0000 7eff74ff9700 10 client.21537 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.944+0000 7eff90a1b700 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ebee3b0 0x55f48ebf0810 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6beab93e4ed1d6c server_cookie=6999580dfe64529c in_seq=0 out_seq=0 2024-06-02T23:35:21.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff74ff9700 1 -- 172.21.15.52:0/1523129940 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55f48eb32d50 con 0x55f48ebee3b0 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff74ff9700 10 client.21537 handle_command_reply: tid=0 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 client.21537 shutdown 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 msgr2=0x55f48ebe0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 0x55f48ebe0660 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7eff88009fd0 tx=0x7eff8801e830 comp rx=0 tx=0).stop 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 shutdown_connections 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ebee3b0 0x55f48ebf0810 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebddea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd47a0 0x55f48ebe0660 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1523129940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebdd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 >> 172.21.15.52:0/1523129940 conn(0x55f48eb01080 msgr2=0x55f48ebce700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.945+0000 7eff74ff9700 0 client.21537 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.946+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 shutdown_connections 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.946+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1523129940 wait complete. 2024-06-02T23:35:21.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.946+0000 7eff9dd03b80 20 client.21537 trim_cache size 0 max 16384 2024-06-02T23:35:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 Processor -- start 2024-06-02T23:35:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 -- start start 2024-06-02T23:35:21.984 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd9f60 0x55f48ebd91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebe01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1aa740 con 0x55f48ebd9f60 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1ac280 con 0x55f48ebda880 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1aae10 con 0x55f48ebd47a0 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9121c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9121c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:32870/0 (socket says 172.21.15.52:32870) 2024-06-02T23:35:21.986 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff9121c700 1 -- 172.21.15.52:0/3799681071 learned_addr learned my addr 172.21.15.52:0/3799681071 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff90a1b700 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd9f60 0x55f48ebd91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.984+0000 7eff91a1d700 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebe01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff9121c700 1 -- 172.21.15.52:0/3799681071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 msgr2=0x55f48ebe01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff9121c700 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebe01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff9121c700 1 -- 172.21.15.52:0/3799681071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd9f60 msgr2=0x55f48ebd91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff9121c700 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd9f60 0x55f48ebd91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff9121c700 1 -- 172.21.15.52:0/3799681071 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f48ebdbf10 con 0x55f48ebd47a0 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff9121c700 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7eff88004be0 tx=0x7eff8801e4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54de55544e1d261 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff76ffd700 1 -- 172.21.15.52:0/3799681071 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88040070 con 0x55f48ebd47a0 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff76ffd700 1 -- 172.21.15.52:0/3799681071 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8801ebb0 con 0x55f48ebd47a0 2024-06-02T23:35:21.987 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.985+0000 7eff76ffd700 1 -- 172.21.15.52:0/3799681071 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff88015030 con 0x55f48ebd47a0 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 -- 172.21.15.52:0/3799681071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 msgr2=0x55f48ebd4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4260 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7eff88004be0 tx=0x7eff8801e4d0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 -- 172.21.15.52:0/3799681071 shutdown_connections 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebda880 0x55f48ebe01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd47a0 0x55f48ebd4260 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/3799681071 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebd9f60 0x55f48ebd91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 -- 172.21.15.52:0/3799681071 >> 172.21.15.52:0/3799681071 conn(0x55f48eb01080 msgr2=0x55f48ebd8aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:21.992 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 -- 172.21.15.52:0/3799681071 shutdown_connections 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.986+0000 7eff9dd03b80 1 -- 172.21.15.52:0/3799681071 wait complete. 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 -- start start 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 Processor -- start 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebf6a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebe6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebe6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1abbb0 con 0x55f48ebda880 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1ab4e0 con 0x55f48ebd47a0 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c1a86e0 con 0x55f48ebd9f60 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9dd03b80 10 client.0 _ll_register_callbacks cb 0x55f48eabb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9121c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebf6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9121c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebf6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:55190/0 (socket says 172.21.15.52:55190) 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.987+0000 7eff9121c700 1 -- 172.21.15.52:0/1721506157 learned_addr learned my addr 172.21.15.52:0/1721506157 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:21.993 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebe6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 -- 172.21.15.52:0/1721506157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 msgr2=0x55f48ebe6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff90a1b700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebe6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebe6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 -- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 msgr2=0x55f48ebf6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebf6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 -- 172.21.15.52:0/1721506157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f48ebf5110 con 0x55f48ebda880 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff9121c700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebf6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff91a1d700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebe6bc0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7eff84012b80 tx=0x7eff84012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3785b17176c7fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.988+0000 7eff74ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff74ff9700 1 -- 172.21.15.52:0/1721506157 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8401d430 con 0x55f48ebda880 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff74ff9700 1 -- 172.21.15.52:0/1721506157 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7eff8401d5d0 con 0x55f48ebda880 2024-06-02T23:35:21.994 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff8801ebb0 con 0x55f48ebda880 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff74ff9700 1 -- 172.21.15.52:0/1721506157 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7eff8401d7d0 con 0x55f48ebda880 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff9dd03b80 10 client.21543 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f48ec58410 con 0x55f48ebda880 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff74ff9700 1 -- 172.21.15.52:0/1721506157 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7eff8801ebb0 con 0x55f48ebda880 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff74ff9700 1 -- 172.21.15.52:0/1721506157 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7eff84007a60 con 0x55f48ebda880 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff9dd03b80 10 client.21543 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff9dd03b80 10 client.21543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.989+0000 7eff9dd03b80 20 client.21543 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ec58ba0 0x55f48ec5afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff9dd03b80 4 client.21543 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f48eb2d2d0 con 0x55f48ec58ba0 2024-06-02T23:35:21.995 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff9121c700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ec58ba0 0x55f48ec5afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff9121c700 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ec58ba0 0x55f48ec5afc0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3abf83e2ff4b29 server_cookie=bdbcc5d8f1d20b8a in_seq=0 out_seq=0 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff74ff9700 10 client.21543 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff74ff9700 1 -- 172.21.15.52:0/1721506157 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f48eb2d2d0 con 0x55f48ec58ba0 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff74ff9700 10 client.21543 handle_command_reply: tid=0 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.990+0000 7eff9dd03b80 1 client.21543 shutdown 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 msgr2=0x55f48ebe6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebe6bc0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7eff84012b80 tx=0x7eff84012f40 comp rx=0 tx=0).stop 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 shutdown_connections 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55f48ec58ba0 0x55f48ec5afc0 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55f48ebda880 0x55f48ebe6bc0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f48ebd47a0 0x55f48ebf6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 --2- 172.21.15.52:0/1721506157 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f48ebd9f60 0x55f48ebe6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 >> 172.21.15.52:0/1721506157 conn(0x55f48eb01080 msgr2=0x55f48ebd9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:21.996 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff74ff9700 0 client.21543 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 shutdown_connections 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 1 -- 172.21.15.52:0/1721506157 wait complete. 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.991+0000 7eff9dd03b80 20 client.21543 trim_cache size 0 max 16384 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:35:21.997 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff780a2ca0 msgr2=0x7eff780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff780a2ca0 0x7eff780a5160 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7eff8c199b00 tx=0x7eff88009260 comp rx=0 tx=0).stop 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 msgr2=0x7eff8c19ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 0x7eff8c19ed40 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7eff7c000c00 tx=0x7eff7c0057a0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 shutdown_connections 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7eff780a2ca0 0x7eff780a5160 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff8c108520 0x7eff8c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7eff8c06ab60 0x7eff8c19e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 --2- 172.21.15.52:0/665535709 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7eff8c06b520 0x7eff8c19ed40 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.992+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 >> 172.21.15.52:0/665535709 conn(0x7eff8c078340 msgr2=0x7eff8c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.993+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 shutdown_connections 2024-06-02T23:35:21.999 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:21.993+0000 7eff93480700 1 -- 172.21.15.52:0/665535709 wait complete. 2024-06-02T23:35:22.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 74221 6.50 MB/sec execute 510 sec latency 99.269 ms 2024-06-02T23:35:23.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 74342 6.50 MB/sec execute 511 sec latency 147.877 ms 2024-06-02T23:35:23.728 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:23 smithi052 ceph-mon[32340]: pgmap v1136: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 259 op/s 2024-06-02T23:35:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:23 smithi123 ceph-mon[35977]: pgmap v1136: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 259 op/s 2024-06-02T23:35:23.778 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:35:23.778 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:35:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:23 smithi174 ceph-mon[35994]: pgmap v1136: 97 pgs: 97 active+clean; 576 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 259 op/s 2024-06-02T23:35:24.214 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:24.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 74499 6.50 MB/sec execute 512 sec latency 104.901 ms 2024-06-02T23:35:25.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 74697 6.52 MB/sec execute 513 sec latency 66.468 ms 2024-06-02T23:35:25.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:25 smithi052 ceph-mon[32340]: pgmap v1137: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-06-02T23:35:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:25 smithi123 ceph-mon[35977]: pgmap v1137: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-06-02T23:35:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:25 smithi174 ceph-mon[35994]: pgmap v1137: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 250 op/s 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.365+0000 7fb588c37700 1 -- 172.21.15.52:0/1952079295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102ed0 msgr2=0x7fb58410f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.365+0000 7fb588c37700 1 --2- 172.21.15.52:0/1952079295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102ed0 0x7fb58410f360 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00b0a0 tx=0x7fb57c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 -- 172.21.15.52:0/1952079295 shutdown_connections 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 --2- 172.21.15.52:0/1952079295 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102ed0 0x7fb58410f360 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 --2- 172.21.15.52:0/1952079295 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584108520 0x7fb584108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 --2- 172.21.15.52:0/1952079295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584102510 0x7fb584102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 -- 172.21.15.52:0/1952079295 >> 172.21.15.52:0/1952079295 conn(0x7fb584079560 msgr2=0x7fb584079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:26.367 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 -- 172.21.15.52:0/1952079295 shutdown_connections 2024-06-02T23:35:26.368 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.366+0000 7fb588c37700 1 -- 172.21.15.52:0/1952079295 wait complete. 2024-06-02T23:35:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.367+0000 7fb588c37700 1 Processor -- start 2024-06-02T23:35:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.367+0000 7fb588c37700 1 -- start start 2024-06-02T23:35:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb588c37700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 0x7fb58410bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb588c37700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584102ed0 0x7fb58410c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb588c37700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584108520 0x7fb5841a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:26.369 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb588c37700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb584075d50 con 0x7fb584102510 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb58259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 0x7fb58410bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb58259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 0x7fb58410bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39998/0 (socket says 172.21.15.52:39998) 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb58259c700 1 -- 172.21.15.52:0/820150524 learned_addr learned my addr 172.21.15.52:0/820150524 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb584075bd0 con 0x7fb584108520 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb584075ed0 con 0x7fb584102ed0 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb581d9b700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584102ed0 0x7fb58410c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb582d9d700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584108520 0x7fb5841a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb581d9b700 1 -- 172.21.15.52:0/820150524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584108520 msgr2=0x7fb5841a3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb581d9b700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584108520 0x7fb5841a3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb581d9b700 1 -- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 msgr2=0x7fb58410bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb581d9b700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 0x7fb58410bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb581d9b700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5841a4040 con 0x7fb584102ed0 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.368+0000 7fb58259c700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 0x7fb58410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:26.370 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb582d9d700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584108520 0x7fb5841a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb581d9b700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584102ed0 0x7fb58410c520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb57800e9f0 tx=0x7fb57800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57800c9c0 con 0x7fb584102ed0 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb57800cb60 con 0x7fb584102ed0 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57800c9c0 con 0x7fb584102ed0 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5841a4330 con 0x7fb584102ed0 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.369+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5841abbb0 con 0x7fb584102ed0 2024-06-02T23:35:26.371 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.370+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb584069180 con 0x7fb584102ed0 2024-06-02T23:35:26.373 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.372+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb578010430 con 0x7fb584102ed0 2024-06-02T23:35:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.374+0000 7fb5737fe700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb56c0a3010 0x7fb56c0a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:26.374 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.374+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb57800c7c0 con 0x7fb584102ed0 2024-06-02T23:35:26.375 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.375+0000 7fb58259c700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb56c0a3010 0x7fb56c0a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:26.376 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.375+0000 7fb58259c700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb56c0a3010 0x7fb56c0a54d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb574011550 tx=0x7fb574009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:26.379 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.379+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fb57809baf0 con 0x7fb584102ed0 2024-06-02T23:35:26.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 74870 6.52 MB/sec execute 514 sec latency 69.619 ms 2024-06-02T23:35:26.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.839+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5841abd80 con 0x7fb584102ed0 2024-06-02T23:35:26.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.840+0000 7fb5737fe700 1 -- 172.21.15.52:0/820150524 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7fb57809baf0 con 0x7fb584102ed0 2024-06-02T23:35:26.841 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:35:26.841 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb56c0a3010 msgr2=0x7fb56c0a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb56c0a3010 0x7fb56c0a54d0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb574011550 tx=0x7fb574009260 comp rx=0 tx=0).stop 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584102ed0 msgr2=0x7fb58410c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584102ed0 0x7fb58410c520 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fb57800e9f0 tx=0x7fb57800ed00 comp rx=0 tx=0).stop 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 shutdown_connections 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb56c0a3010 0x7fb56c0a54d0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb584108520 0x7fb5841a3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb584102510 0x7fb58410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 --2- 172.21.15.52:0/820150524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb584102ed0 0x7fb58410c520 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 >> 172.21.15.52:0/820150524 conn(0x7fb584079560 msgr2=0x7fb584106090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 shutdown_connections 2024-06-02T23:35:26.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:26.843+0000 7fb588c37700 1 -- 172.21.15.52:0/820150524 wait complete. 2024-06-02T23:35:26.845 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:35:27.479 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75049 6.52 MB/sec execute 515 sec latency 68.583 ms 2024-06-02T23:35:27.608 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:35:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:27 smithi123 ceph-mon[35977]: pgmap v1138: 97 pgs: 97 active+clean; 573 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 320 op/s 2024-06-02T23:35:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:27 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/820150524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:35:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:27 smithi174 ceph-mon[35994]: pgmap v1138: 97 pgs: 97 active+clean; 573 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 320 op/s 2024-06-02T23:35:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:27 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/820150524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:35:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:27 smithi052 ceph-mon[32340]: pgmap v1138: 97 pgs: 97 active+clean; 573 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 320 op/s 2024-06-02T23:35:27.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:27 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/820150524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:35:27.973 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:28.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75190 6.53 MB/sec execute 516 sec latency 63.831 ms 2024-06-02T23:35:29.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75346 6.53 MB/sec execute 517 sec latency 154.506 ms 2024-06-02T23:35:29.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:29 smithi052 ceph-mon[32340]: pgmap v1139: 97 pgs: 97 active+clean; 575 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.7 MiB/s wr, 265 op/s 2024-06-02T23:35:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:29 smithi123 ceph-mon[35977]: pgmap v1139: 97 pgs: 97 active+clean; 575 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.7 MiB/s wr, 265 op/s 2024-06-02T23:35:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:29 smithi174 ceph-mon[35994]: pgmap v1139: 97 pgs: 97 active+clean; 575 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 959 KiB/s rd, 3.7 MiB/s wr, 265 op/s 2024-06-02T23:35:30.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75517 6.53 MB/sec execute 518 sec latency 118.651 ms 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 -- 172.21.15.52:0/2065557843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c0101a30 msgr2=0x7f89c0101e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2065557843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c0101a30 0x7f89c0101e50 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f89b0009900 tx=0x7f89b002a410 comp rx=0 tx=0).stop 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 -- 172.21.15.52:0/2065557843 shutdown_connections 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2065557843 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 0x7f89c01030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2065557843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c01035e0 0x7f89c0109e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2065557843 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c0101a30 0x7f89c0101e50 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 -- 172.21.15.52:0/2065557843 >> 172.21.15.52:0/2065557843 conn(0x7f89c00fcfc0 msgr2=0x7f89c00ff3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:30.965 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 -- 172.21.15.52:0/2065557843 shutdown_connections 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.963+0000 7f89c5eba640 1 -- 172.21.15.52:0/2065557843 wait complete. 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 Processor -- start 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 -- start start 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 0x7f89c0197c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 0x7f89c0198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c01035e0 0x7f89c019f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89c010d530 con 0x7f89c01035e0 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89c010d3b0 con 0x7f89c0101a30 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.964+0000 7f89c5eba640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89c010d6b0 con 0x7f89c0102c20 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 0x7f89c0197c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c01035e0 0x7f89c019f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.966 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 0x7f89c0197c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35684/0 (socket says 172.21.15.52:35684) 2024-06-02T23:35:30.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 -- 172.21.15.52:0/2064337848 learned_addr learned my addr 172.21.15.52:0/2064337848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89beffd640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 0x7f89c0198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 -- 172.21.15.52:0/2064337848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 msgr2=0x7f89c0198140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 0x7f89c0198140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 -- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c01035e0 msgr2=0x7f89c019f160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c01035e0 0x7f89c019f160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 -- 172.21.15.52:0/2064337848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89c019f7d0 con 0x7f89c0101a30 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bffff640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c01035e0 0x7f89c019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:30.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89beffd640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 0x7f89c0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bf7fe640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 0x7f89c0197c00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f89b00314e0 tx=0x7f89b0004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bcff9640 1 -- 172.21.15.52:0/2064337848 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b003f030 con 0x7f89c0101a30 2024-06-02T23:35:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89c019fa60 con 0x7f89c0101a30 2024-06-02T23:35:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bcff9640 1 -- 172.21.15.52:0/2064337848 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b00318f0 con 0x7f89c0101a30 2024-06-02T23:35:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89c019ffa0 con 0x7f89c0101a30 2024-06-02T23:35:30.969 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.965+0000 7f89bcff9640 1 -- 172.21.15.52:0/2064337848 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b0035070 con 0x7f89c0101a30 2024-06-02T23:35:30.970 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.968+0000 7f89c8bc6740 1 Processor -- start 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bcff9640 1 -- 172.21.15.52:0/2064337848 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f89b0042030 con 0x7f89c0101a30 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 -- start start 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9acd6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6a00 0x55ba9acdb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9acde000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ba9ac394f0 con 0x55ba9acd6a00 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ba9abd7870 con 0x55ba9acd6010 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba9ac354b0 con 0x55ba9acdbb70 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9acd6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bf7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6a00 0x55ba9acdb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89beffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9acde000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9acd6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:35698/0 (socket says 172.21.15.52:35698) 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 -- 172.21.15.52:0/3810200138 learned_addr learned my addr 172.21.15.52:0/3810200138 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 -- 172.21.15.52:0/3810200138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 msgr2=0x55ba9acde000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9acde000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 -- 172.21.15.52:0/3810200138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6a00 msgr2=0x55ba9acdb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6a00 0x55ba9acdb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.969+0000 7f89bffff640 1 -- 172.21.15.52:0/3810200138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9abf39b0 con 0x55ba9acd6010 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89bffff640 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9acd6430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f89b8011810 tx=0x7f89b8011cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ed911aadecaf3e8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89a5ffb640 1 -- 172.21.15.52:0/3810200138 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8011250 con 0x55ba9acd6010 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89bcff9640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89a80a2c30 0x7f89a80a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89a5ffb640 1 -- 172.21.15.52:0/3810200138 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b801a070 con 0x55ba9acd6010 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89a5ffb640 1 -- 172.21.15.52:0/3810200138 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8005420 con 0x55ba9acd6010 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89bf7fe640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89a80a2c30 0x7f89a80a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.970+0000 7f89bcff9640 1 -- 172.21.15.52:0/2064337848 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f89b00ef6e0 con 0x7f89c0101a30 2024-06-02T23:35:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3810200138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 msgr2=0x55ba9acd6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9acd6430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f89b8011810 tx=0x7f89b8011cd0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89bf7fe640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89a80a2c30 0x7f89a80a50f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f89b00ed580 tx=0x7f89b00f5550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3810200138 shutdown_connections 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9acde000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6a00 0x55ba9acdb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3810200138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9acd6430 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3810200138 >> 172.21.15.52:0/3810200138 conn(0x55ba9acd1680 msgr2=0x55ba9acd1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3810200138 shutdown_connections 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3810200138 wait complete. 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 -- start start 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.971+0000 7f89c8bc6740 1 Processor -- start 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9ace02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acdbb70 0x55ba9ace77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ba9acdeb20 con 0x55ba9acd6010 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ba9abc2ae0 con 0x55ba9acdbb70 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ba9acdeca0 con 0x55ba9acd6a00 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89c8bc6740 10 client.0 _ll_register_callbacks cb 0x55ba9abfb9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89beffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9ace02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89beffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:40220/0 (socket says 172.21.15.52:40220) 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89beffd640 1 -- 172.21.15.52:0/3365042965 learned_addr learned my addr 172.21.15.52:0/3365042965 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89beffd640 1 -- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acdbb70 msgr2=0x55ba9ace77d0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89beffd640 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acdbb70 0x55ba9ace77d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.972+0000 7f89beffd640 1 -- 172.21.15.52:0/3365042965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 msgr2=0x55ba9ace02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89beffd640 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9ace02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.978 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89beffd640 1 -- 172.21.15.52:0/3365042965 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9a94d210 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89bffff640 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9ace02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89beffd640 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace0800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89b4012700 tx=0x7f89b4012bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7af9a2cf4a7b04de server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f899effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f899effd640 1 -- 172.21.15.52:0/3365042965 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011b50 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b801a070 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f899effd640 1 -- 172.21.15.52:0/3365042965 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f89b4011cf0 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f899effd640 1 -- 172.21.15.52:0/3365042965 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f89b4023070 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89c8bc6740 10 client.30370 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.973+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ba9aa19620 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f899effd640 1 -- 172.21.15.52:0/3365042965 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b801a070 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f899effd640 1 -- 172.21.15.52:0/3365042965 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f89b4003bc0 con 0x55ba9acd6a00 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89c8bc6740 10 client.30370 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89c8bc6740 10 client.30370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:30.979 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89c8bc6740 20 client.30370 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9acee600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89c8bc6740 4 client.30370 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba9ac4cf10 con 0x55ba9acec180 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89bffff640 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9acee600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f899effd640 10 client.30370 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.974+0000 7f89bffff640 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9acee600 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40468d05ae8fdc9e server_cookie=3b070a05aa2b3af in_seq=0 out_seq=0 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.975+0000 7f899effd640 1 -- 172.21.15.52:0/3365042965 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ba9ac4cf10 con 0x55ba9acec180 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.975+0000 7f899effd640 10 client.30370 handle_command_reply: tid=0 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.975+0000 7f89c8bc6740 1 client.30370 shutdown 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 msgr2=0x55ba9ace0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace0800 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f89b4012700 tx=0x7f89b4012bc0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 shutdown_connections 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9acee600 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f899effd640 0 client.30370 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace0800 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acdbb70 0x55ba9ace77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/3365042965 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9ace02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 >> 172.21.15.52:0/3365042965 conn(0x55ba9acd1680 msgr2=0x55ba9acd07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 shutdown_connections 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 1 -- 172.21.15.52:0/3365042965 wait complete. 2024-06-02T23:35:30.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.976+0000 7f89c8bc6740 20 client.30370 trim_cache size 0 max 16384 2024-06-02T23:35:30.998 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.998+0000 7f89c8bc6740 1 Processor -- start 2024-06-02T23:35:30.999 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.998+0000 7f89c8bc6740 1 -- start start 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9acd7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6a00 0x55ba9acd81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9ace7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a6aa0 con 0x55ba9acd6010 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a85e0 con 0x55ba9acd6a00 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a7170 con 0x55ba9acdbb70 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9acd7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9acd7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40034/0 (socket says 172.21.15.52:40034) 2024-06-02T23:35:31.000 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 -- 172.21.15.52:0/1458720720 learned_addr learned my addr 172.21.15.52:0/1458720720 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9ace7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89beffd640 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6a00 0x55ba9acd81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 -- 172.21.15.52:0/1458720720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 msgr2=0x55ba9ace7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9ace7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 -- 172.21.15.52:0/1458720720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6a00 msgr2=0x55ba9acd81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6a00 0x55ba9acd81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:30.999+0000 7f89bffff640 1 -- 172.21.15.52:0/1458720720 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba9acd14b0 con 0x55ba9acd6010 2024-06-02T23:35:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.000+0000 7f89bffff640 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9acd7c80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f89b80099a0 tx=0x7f89b8009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55234de02d0fc253 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.000+0000 7f89a57fa640 1 -- 172.21.15.52:0/1458720720 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b8023040 con 0x55ba9acd6010 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.000+0000 7f89a57fa640 1 -- 172.21.15.52:0/1458720720 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b801a030 con 0x55ba9acd6010 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.000+0000 7f89a57fa640 1 -- 172.21.15.52:0/1458720720 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b802d480 con 0x55ba9acd6010 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 -- 172.21.15.52:0/1458720720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 msgr2=0x55ba9acd7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9acd7c80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f89b80099a0 tx=0x7f89b8009500 comp rx=0 tx=0).stop 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 -- 172.21.15.52:0/1458720720 shutdown_connections 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acdbb70 0x55ba9ace7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6a00 0x55ba9acd81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/1458720720 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acd6010 0x55ba9acd7c80 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 -- 172.21.15.52:0/1458720720 >> 172.21.15.52:0/1458720720 conn(0x55ba9acd1680 msgr2=0x55ba9acd1a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 -- 172.21.15.52:0/1458720720 shutdown_connections 2024-06-02T23:35:31.005 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 -- 172.21.15.52:0/1458720720 wait complete. 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 -- start start 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.001+0000 7f89c8bc6740 1 Processor -- start 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9ace7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acdbb70 0x55ba9ace4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a7f10 con 0x55ba9acdbb70 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a7840 con 0x55ba9acd6010 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f89c01a4a80 con 0x55ba9acd6a00 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89c8bc6740 10 client.0 _ll_register_callbacks cb 0x7f89b4009e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acdbb70 0x55ba9ace4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89beffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89bffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acdbb70 0x55ba9ace4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40042/0 (socket says 172.21.15.52:40042) 2024-06-02T23:35:31.006 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89bffff640 1 -- 172.21.15.52:0/2302355079 learned_addr learned my addr 172.21.15.52:0/2302355079 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89bffff640 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9ace7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89beffd640 1 -- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 msgr2=0x55ba9ace7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89beffd640 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9ace7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89beffd640 1 -- 172.21.15.52:0/2302355079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acdbb70 msgr2=0x55ba9ace4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89beffd640 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acdbb70 0x55ba9ace4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.002+0000 7f89beffd640 1 -- 172.21.15.52:0/2302355079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba9ad54f00 con 0x55ba9acd6a00 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f89beffd640 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace4280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f89b401c470 tx=0x7f89b4011d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=711987ab41cfb655 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f899effd640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f899effd640 1 -- 172.21.15.52:0/2302355079 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89b4011e70 con 0x55ba9acd6a00 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f899effd640 1 -- 172.21.15.52:0/2302355079 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f89b4018040 con 0x55ba9acd6a00 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f89b801a030 con 0x55ba9acd6a00 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f899effd640 1 -- 172.21.15.52:0/2302355079 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f89b4004210 con 0x55ba9acd6a00 2024-06-02T23:35:31.007 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f89c8bc6740 10 client.30376 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.003+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ba9ad56260 con 0x55ba9acd6a00 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f899effd640 1 -- 172.21.15.52:0/2302355079 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f89b801a030 con 0x55ba9acd6a00 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f899effd640 1 -- 172.21.15.52:0/2302355079 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f89b401c960 con 0x55ba9acd6a00 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89c8bc6740 10 client.30376 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89c8bc6740 10 client.30376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89c8bc6740 20 client.30376 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9ad58bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89c8bc6740 4 client.30376 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ba9ac46c20 con 0x55ba9acec180 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89bffff640 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9ad58bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f89bffff640 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9ad58bd0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efd245d5ec8879fe server_cookie=a1c06dbc39c99f73 in_seq=0 out_seq=0 2024-06-02T23:35:31.008 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.004+0000 7f899effd640 10 client.30376 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f899effd640 1 -- 172.21.15.52:0/2302355079 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55ba9ac46c20 con 0x55ba9acec180 2024-06-02T23:35:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f899effd640 10 client.30376 handle_command_reply: tid=0 2024-06-02T23:35:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 client.30376 shutdown 2024-06-02T23:35:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 msgr2=0x55ba9ace4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:31.072 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace4280 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f89b401c470 tx=0x7f89b4011d80 comp rx=0 tx=0).stop 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 shutdown_connections 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ba9acec180 0x55ba9ad58bd0 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ba9acd6a00 0x55ba9ace4280 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ba9acdbb70 0x55ba9ace4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 --2- 172.21.15.52:0/2302355079 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ba9acd6010 0x55ba9ace7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f899effd640 0 client.30376 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 >> 172.21.15.52:0/2302355079 conn(0x55ba9acd1680 msgr2=0x55ba9acd7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 shutdown_connections 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 1 -- 172.21.15.52:0/2302355079 wait complete. 2024-06-02T23:35:31.073 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.071+0000 7f89c8bc6740 20 client.30376 trim_cache size 0 max 16384 2024-06-02T23:35:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:35:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:35:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:35:31.074 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89a80a2c30 msgr2=0x7f89a80a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89a80a2c30 0x7f89a80a50f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f89b00ed580 tx=0x7f89b00f5550 comp rx=0 tx=0).stop 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 msgr2=0x7f89c0197c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 0x7f89c0197c00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f89b00314e0 tx=0x7f89b0004270 comp rx=0 tx=0).stop 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 shutdown_connections 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f89a80a2c30 0x7f89a80a50f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89c0102c20 0x7f89c0198140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f89c01035e0 0x7f89c019f160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 --2- 172.21.15.52:0/2064337848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c0101a30 0x7f89c0197c00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 >> 172.21.15.52:0/2064337848 conn(0x7f89c00fcfc0 msgr2=0x7f89c0106040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 shutdown_connections 2024-06-02T23:35:31.075 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:31.072+0000 7f89c5eba640 1 -- 172.21.15.52:0/2064337848 wait complete. 2024-06-02T23:35:31.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75665 6.54 MB/sec execute 519 sec latency 125.247 ms 2024-06-02T23:35:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:31 smithi123 ceph-mon[35977]: pgmap v1140: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-06-02T23:35:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:31 smithi174 ceph-mon[35994]: pgmap v1140: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-06-02T23:35:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:31 smithi052 ceph-mon[32340]: pgmap v1140: 97 pgs: 97 active+clean; 577 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.3 MiB/s wr, 291 op/s 2024-06-02T23:35:32.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75795 6.54 MB/sec execute 520 sec latency 127.089 ms 2024-06-02T23:35:33.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 75946 6.54 MB/sec execute 521 sec latency 165.164 ms 2024-06-02T23:35:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:33 smithi123 ceph-mon[35977]: pgmap v1141: 97 pgs: 97 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-06-02T23:35:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:33 smithi174 ceph-mon[35994]: pgmap v1141: 97 pgs: 97 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-06-02T23:35:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:33 smithi052 ceph-mon[32340]: pgmap v1141: 97 pgs: 97 active+clean; 579 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 267 op/s 2024-06-02T23:35:34.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76106 6.55 MB/sec execute 522 sec latency 126.979 ms 2024-06-02T23:35:35.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76266 6.56 MB/sec execute 523 sec latency 108.328 ms 2024-06-02T23:35:35.705 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:35 smithi052 ceph-mon[32340]: pgmap v1142: 97 pgs: 97 active+clean; 581 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-06-02T23:35:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:35 smithi123 ceph-mon[35977]: pgmap v1142: 97 pgs: 97 active+clean; 581 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-06-02T23:35:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:35 smithi174 ceph-mon[35994]: pgmap v1142: 97 pgs: 97 active+clean; 581 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 886 KiB/s rd, 3.5 MiB/s wr, 251 op/s 2024-06-02T23:35:36.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76430 6.57 MB/sec execute 524 sec latency 98.557 ms 2024-06-02T23:35:37.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76597 6.58 MB/sec execute 525 sec latency 82.774 ms 2024-06-02T23:35:37.709 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:37 smithi052 ceph-mon[32340]: pgmap v1143: 97 pgs: 97 active+clean; 586 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-06-02T23:35:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:37 smithi123 ceph-mon[35977]: pgmap v1143: 97 pgs: 97 active+clean; 586 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-06-02T23:35:37.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:37 smithi174 ceph-mon[35994]: pgmap v1143: 97 pgs: 97 active+clean; 586 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 311 op/s 2024-06-02T23:35:38.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76747 6.58 MB/sec execute 526 sec latency 206.796 ms 2024-06-02T23:35:39.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76872 6.58 MB/sec execute 527 sec latency 210.090 ms 2024-06-02T23:35:39.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 -- 172.21.15.52:0/2064069955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 msgr2=0x7f426010e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 --2- 172.21.15.52:0/2064069955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426010e240 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f425800b0a0 tx=0x7f425800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 -- 172.21.15.52:0/2064069955 shutdown_connections 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 --2- 172.21.15.52:0/2064069955 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426010e240 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 --2- 172.21.15.52:0/2064069955 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 0x7f42601077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 --2- 172.21.15.52:0/2064069955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 0x7f4260101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 -- 172.21.15.52:0/2064069955 >> 172.21.15.52:0/2064069955 conn(0x7f4260078340 msgr2=0x7f4260078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.894+0000 7f426820e700 1 -- 172.21.15.52:0/2064069955 shutdown_connections 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.895+0000 7f426820e700 1 -- 172.21.15.52:0/2064069955 wait complete. 2024-06-02T23:35:39.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.895+0000 7f426820e700 1 Processor -- start 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.895+0000 7f426820e700 1 -- start start 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f426820e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 0x7f426019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f426820e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f426820e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 0x7f4260198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f426820e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4260111520 con 0x7f4260101d50 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f426820e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42601113a0 con 0x7f4260107400 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f426820e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42601116a0 con 0x7f4260101390 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42657a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42657a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426019ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58144/0 (socket says 172.21.15.52:58144) 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42657a9700 1 -- 172.21.15.52:0/196764228 learned_addr learned my addr 172.21.15.52:0/196764228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42667ab700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 0x7f4260198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f4265faa700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 0x7f426019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42667ab700 1 -- 172.21.15.52:0/196764228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 msgr2=0x7f426019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42667ab700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 0x7f426019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42667ab700 1 -- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 msgr2=0x7f426019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42667ab700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42667ab700 1 -- 172.21.15.52:0/196764228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4260199080 con 0x7f4260107400 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f4265faa700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 0x7f426019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.896+0000 7f42657a9700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:39.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.897+0000 7f42667ab700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 0x7f4260198920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f425800b070 tx=0x7f4258004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.897+0000 7f4256ffd700 1 -- 172.21.15.52:0/196764228 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4258021070 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.897+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4260199310 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.897+0000 7f4256ffd700 1 -- 172.21.15.52:0/196764228 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4258007e30 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.897+0000 7f4256ffd700 1 -- 172.21.15.52:0/196764228 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42580045c0 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.897+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4260102fa0 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.900+0000 7f4256ffd700 1 -- 172.21.15.52:0/196764228 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4258004760 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.900+0000 7f4256ffd700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f424c0a2ca0 0x7f424c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.901+0000 7f4256ffd700 1 -- 172.21.15.52:0/196764228 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f425800f020 con 0x7f4260107400 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.901+0000 7f4265faa700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f424c0a2ca0 0x7f424c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.901+0000 7f4265faa700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f424c0a2ca0 0x7f424c0a5160 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42600ffbe0 tx=0x7f425c014040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.902+0000 7f4272a91b80 1 Processor -- start 2024-06-02T23:35:39.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.902+0000 7f4272a91b80 1 -- start start 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.902+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360776a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.902+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c3607709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360771340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4272a91b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c3606c2810 con 0x55c360776f60 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f42657a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c3607709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f42657a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c3607709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:58154/0 (socket says 172.21.15.52:58154) 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4272a91b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c36066a720 con 0x55c360776640 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4272a91b80 1 -- 172.21.15.52:0/574383559 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c3606be7d0 con 0x55c360770f20 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f42657a9700 1 -- 172.21.15.52:0/574383559 learned_addr learned my addr 172.21.15.52:0/574383559 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360776a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f42667ab700 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360771340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 -- 172.21.15.52:0/574383559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 msgr2=0x55c360771340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360771340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 -- 172.21.15.52:0/574383559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 msgr2=0x55c3607709e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c3607709e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 -- 172.21.15.52:0/574383559 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c360660f50 con 0x55c360776640 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f4265faa700 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360776a20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f425c004460 tx=0x7f425c018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4333b1dae0f58430 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.903+0000 7f424bfff700 1 -- 172.21.15.52:0/574383559 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c02d070 con 0x55c360776640 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.904+0000 7f424bfff700 1 -- 172.21.15.52:0/574383559 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f425c003df0 con 0x55c360776640 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.904+0000 7f424bfff700 1 -- 172.21.15.52:0/574383559 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c01d670 con 0x55c360776640 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.904+0000 7f4272a91b80 1 -- 172.21.15.52:0/574383559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 msgr2=0x55c360776a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.904+0000 7f4272a91b80 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360776a20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f425c004460 tx=0x7f425c018000 comp rx=0 tx=0).stop 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 -- 172.21.15.52:0/574383559 shutdown_connections 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360771340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360776a20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 --2- 172.21.15.52:0/574383559 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c3607709e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 -- 172.21.15.52:0/574383559 >> 172.21.15.52:0/574383559 conn(0x55c36069bcc0 msgr2=0x55c36076b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 -- 172.21.15.52:0/574383559 shutdown_connections 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 -- 172.21.15.52:0/574383559 wait complete. 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.905+0000 7f4272a91b80 1 -- start start 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 Processor -- start 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c36077ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c360779fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.702 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55c36077d9d0 con 0x55c360776f60 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c360662bf0 con 0x55c360770f20 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55c36077db50 con 0x55c360776640 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4272a91b80 10 client.0 _ll_register_callbacks cb 0x55c36065c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f42657a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c360779fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4265faa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c36077ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4265faa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c36077ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41246/0 (socket says 172.21.15.52:41246) 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f4265faa700 1 -- 172.21.15.52:0/3751216455 learned_addr learned my addr 172.21.15.52:0/3751216455 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.906+0000 7f42667ab700 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4265faa700 1 -- 172.21.15.52:0/3751216455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 msgr2=0x55c360779fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4265faa700 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c360779fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4265faa700 1 -- 172.21.15.52:0/3751216455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 msgr2=0x55c36077a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4265faa700 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4265faa700 1 -- 172.21.15.52:0/3751216455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3606118d0 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4265faa700 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c36077ccc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f425c028da0 tx=0x7f425c005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1968b481eca30f4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f42497fa700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f42497fa700 1 -- 172.21.15.52:0/3751216455 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c02d070 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f42497fa700 1 -- 172.21.15.52:0/3751216455 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f425c005250 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f425c003df0 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f42497fa700 1 -- 172.21.15.52:0/3751216455 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f425c005450 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4272a91b80 10 client.30809 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.907+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c360788300 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f42497fa700 1 -- 172.21.15.52:0/3751216455 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f425c003df0 con 0x55c360770f20 2024-06-02T23:35:39.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f42497fa700 1 -- 172.21.15.52:0/3751216455 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f425c03e460 con 0x55c360770f20 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f4272a91b80 10 client.30809 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f4272a91b80 10 client.30809 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f4272a91b80 20 client.30809 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c36078ac80 0x55c36078d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f4272a91b80 4 client.30809 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c3606cfd10 con 0x55c36078ac80 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.908+0000 7f42657a9700 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c36078ac80 0x55c36078d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.909+0000 7f42497fa700 10 client.30809 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.909+0000 7f42657a9700 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c36078ac80 0x55c36078d0a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c706e0908cea37e9 server_cookie=6e460b1cb7f08518 in_seq=0 out_seq=0 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.909+0000 7f42497fa700 1 -- 172.21.15.52:0/3751216455 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55c3606cfd10 con 0x55c36078ac80 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.909+0000 7f42497fa700 10 client.30809 handle_command_reply: tid=0 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.909+0000 7f4272a91b80 1 client.30809 shutdown 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 msgr2=0x55c36077ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c36077ccc0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f425c028da0 tx=0x7f425c005c80 comp rx=0 tx=0).stop 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 shutdown_connections 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c36078ac80 0x55c36078d0a0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c36077ccc0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3751216455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c360779fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 >> 172.21.15.52:0/3751216455 conn(0x55c36069bcc0 msgr2=0x55c36076f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f42497fa700 0 client.30809 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 shutdown_connections 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 1 -- 172.21.15.52:0/3751216455 wait complete. 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.910+0000 7f4272a91b80 20 client.30809 trim_cache size 0 max 16384 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 Processor -- start 2024-06-02T23:35:39.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 -- start start 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c3607709e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c36076b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42601aa750 con 0x55c360776f60 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42601ac290 con 0x55c360770f20 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4272a91b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42601aae20 con 0x55c360776640 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f4265faa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c3607709e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.938+0000 7f42657a9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c36076b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f42667ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.707 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c3607709e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:41252/0 (socket says 172.21.15.52:41252) 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 -- 172.21.15.52:0/3294995510 learned_addr learned my addr 172.21.15.52:0/3294995510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 -- 172.21.15.52:0/3294995510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 msgr2=0x55c36076b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c36076b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 -- 172.21.15.52:0/3294995510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 msgr2=0x55c36077a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 -- 172.21.15.52:0/3294995510 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36078d680 con 0x55c360770f20 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f4265faa700 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c3607709e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f425c009f40 tx=0x7f425c005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8538008d2a7aa3f3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f424b7fe700 1 -- 172.21.15.52:0/3294995510 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c02d070 con 0x55c360770f20 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f424b7fe700 1 -- 172.21.15.52:0/3294995510 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f425c037070 con 0x55c360770f20 2024-06-02T23:35:39.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.939+0000 7f424b7fe700 1 -- 172.21.15.52:0/3294995510 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c003de0 con 0x55c360770f20 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 -- 172.21.15.52:0/3294995510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 msgr2=0x55c3607709e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c3607709e0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f425c009f40 tx=0x7f425c005fa0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 -- 172.21.15.52:0/3294995510 shutdown_connections 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c36077a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360770f20 0x55c3607709e0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 --2- 172.21.15.52:0/3294995510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360776640 0x55c36076b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 -- 172.21.15.52:0/3294995510 >> 172.21.15.52:0/3294995510 conn(0x55c36069bcc0 msgr2=0x55c360777340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 -- 172.21.15.52:0/3294995510 shutdown_connections 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.940+0000 7f4272a91b80 1 -- 172.21.15.52:0/3294995510 wait complete. 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 -- start start 2024-06-02T23:35:39.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 Processor -- start 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360792c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360793170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c360782ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f42601abbc0 con 0x55c360776f60 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42601ab4f0 con 0x55c360776640 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f42601a86f0 con 0x55c360770f20 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4272a91b80 10 client.0 _ll_register_callbacks cb 0x55c36065c920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4265faa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360792c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f42667ab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c360782ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4265faa700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360792c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:60920/0 (socket says 172.21.15.52:60920) 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f4265faa700 1 -- 172.21.15.52:0/1975460012 learned_addr learned my addr 172.21.15.52:0/1975460012 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:39.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.941+0000 7f42657a9700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360793170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4265faa700 1 -- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 msgr2=0x55c360793170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4265faa700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360793170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4265faa700 1 -- 172.21.15.52:0/1975460012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 msgr2=0x55c360782ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4265faa700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c360782ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4265faa700 1 -- 172.21.15.52:0/1975460012 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3607836e0 con 0x55c360770f20 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f42667ab700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c360782ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4265faa700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360792c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f425c006010 tx=0x7f425c005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c4824d377b53b13 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f42497fa700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f42497fa700 1 -- 172.21.15.52:0/1975460012 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c02d030 con 0x55c360770f20 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f42497fa700 1 -- 172.21.15.52:0/1975460012 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f425c003e50 con 0x55c360770f20 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f425c037070 con 0x55c360770f20 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f42497fa700 1 -- 172.21.15.52:0/1975460012 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f425c056050 con 0x55c360770f20 2024-06-02T23:35:39.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4272a91b80 10 client.30394 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.942+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c360792690 con 0x55c360770f20 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f42497fa700 1 -- 172.21.15.52:0/1975460012 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f425c037070 con 0x55c360770f20 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f42497fa700 1 -- 172.21.15.52:0/1975460012 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f425c032c20 con 0x55c360770f20 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f4272a91b80 10 client.30394 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f4272a91b80 10 client.30394 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f4272a91b80 20 client.30394 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f4272a91b80 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c3607f4ce0 0x55c3607f7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f4272a91b80 4 client.30394 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c3606ca290 con 0x55c3607f4ce0 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.943+0000 7f42657a9700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c3607f4ce0 0x55c3607f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f42497fa700 10 client.30394 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f42657a9700 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c3607f4ce0 0x55c3607f7100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8823790c8134ba4d server_cookie=1c0fa6789eb6686a in_seq=0 out_seq=0 2024-06-02T23:35:39.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f42497fa700 1 -- 172.21.15.52:0/1975460012 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c3606ca290 con 0x55c3607f4ce0 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f42497fa700 10 client.30394 handle_command_reply: tid=0 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 client.30394 shutdown 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 msgr2=0x55c360792c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360792c30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f425c006010 tx=0x7f425c005a00 comp rx=0 tx=0).stop 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 shutdown_connections 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55c360776f60 0x55c360782ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55c360770f20 0x55c360792c30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.713 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55c3607f4ce0 0x55c3607f7100 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.714 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 --2- 172.21.15.52:0/1975460012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c360776640 0x55c360793170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.714 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 >> 172.21.15.52:0/1975460012 conn(0x55c36069bcc0 msgr2=0x55c360777340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f42497fa700 0 client.30394 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 shutdown_connections 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.944+0000 7f4272a91b80 1 -- 172.21.15.52:0/1975460012 wait complete. 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.945+0000 7f4272a91b80 20 client.30394 trim_cache size 0 max 16384 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.945+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f424c0a2ca0 msgr2=0x7f424c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.715 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.945+0000 7f426820e700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f424c0a2ca0 0x7f424c0a5160 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f42600ffbe0 tx=0x7f425c014040 comp rx=0 tx=0).stop 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.945+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 msgr2=0x7f4260198920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.945+0000 7f426820e700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 0x7f4260198920 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f425800b070 tx=0x7f4258004d00 comp rx=0 tx=0).stop 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 shutdown_connections 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f424c0a2ca0 0x7f424c0a5160 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4260107400 0x7f4260198920 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4260101390 0x7f426019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 --2- 172.21.15.52:0/196764228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4260101d50 0x7f426019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 >> 172.21.15.52:0/196764228 conn(0x7f4260078340 msgr2=0x7f4260100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:39.716 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 shutdown_connections 2024-06-02T23:35:39.717 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:38.946+0000 7f426820e700 1 -- 172.21.15.52:0/196764228 wait complete. 2024-06-02T23:35:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:39 smithi123 ceph-mon[35977]: pgmap v1144: 97 pgs: 97 active+clean; 590 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.8 MiB/s wr, 244 op/s 2024-06-02T23:35:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:39 smithi174 ceph-mon[35994]: pgmap v1144: 97 pgs: 97 active+clean; 590 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.8 MiB/s wr, 244 op/s 2024-06-02T23:35:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:39 smithi052 ceph-mon[32340]: pgmap v1144: 97 pgs: 97 active+clean; 590 MiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.8 MiB/s wr, 244 op/s 2024-06-02T23:35:40.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 76995 6.58 MB/sec execute 528 sec latency 141.438 ms 2024-06-02T23:35:41.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 77143 6.58 MB/sec execute 529 sec latency 89.801 ms 2024-06-02T23:35:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:41 smithi123 ceph-mon[35977]: pgmap v1145: 97 pgs: 97 active+clean; 590 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 4.1 MiB/s wr, 243 op/s 2024-06-02T23:35:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:41 smithi174 ceph-mon[35994]: pgmap v1145: 97 pgs: 97 active+clean; 590 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 4.1 MiB/s wr, 243 op/s 2024-06-02T23:35:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:41 smithi052 ceph-mon[32340]: pgmap v1145: 97 pgs: 97 active+clean; 590 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 900 KiB/s rd, 4.1 MiB/s wr, 243 op/s 2024-06-02T23:35:41.981 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:35:42.358 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:42.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 77292 6.58 MB/sec execute 530 sec latency 57.935 ms 2024-06-02T23:35:43.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 77454 6.57 MB/sec execute 531 sec latency 58.832 ms 2024-06-02T23:35:43.717 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:43 smithi052 ceph-mon[32340]: pgmap v1146: 97 pgs: 97 active+clean; 593 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-06-02T23:35:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:43 smithi123 ceph-mon[35977]: pgmap v1146: 97 pgs: 97 active+clean; 593 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-06-02T23:35:43.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:43 smithi174 ceph-mon[35994]: pgmap v1146: 97 pgs: 97 active+clean; 593 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.0 MiB/s wr, 261 op/s 2024-06-02T23:35:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.213+0000 7f0ab009a640 1 -- 172.21.15.52:0/1754753988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 msgr2=0x7f0aa8101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.214 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.213+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1754753988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 0x7f0aa8101a50 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0009ca0 tx=0x7f0aa002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 -- 172.21.15.52:0/1754753988 shutdown_connections 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1754753988 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 0x7f0aa8101a50 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1754753988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 0x7f0aa8101090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1754753988 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa81006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 -- 172.21.15.52:0/1754753988 >> 172.21.15.52:0/1754753988 conn(0x7f0aa80fbe90 msgr2=0x7f0aa80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 -- 172.21.15.52:0/1754753988 shutdown_connections 2024-06-02T23:35:44.215 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.214+0000 7f0ab009a640 1 -- 172.21.15.52:0/1754753988 wait complete. 2024-06-02T23:35:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 Processor -- start 2024-06-02T23:35:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 -- start start 2024-06-02T23:35:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa8199d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 0x7f0aa819a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 0x7f0aa81a12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.217 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810f750 con 0x7f0aa81002d0 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810f5d0 con 0x7f0aa8100c30 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0ab009a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa810f8d0 con 0x7f0aa81015d0 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0aade36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa8199d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0aade36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa8199d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39384/0 (socket says 172.21.15.52:39384) 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0aade36640 1 -- 172.21.15.52:0/1975054140 learned_addr learned my addr 172.21.15.52:0/1975054140 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.215+0000 7f0aad635640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 0x7f0aa819a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aae637640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 0x7f0aa81a12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aade36640 1 -- 172.21.15.52:0/1975054140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 msgr2=0x7f0aa81a12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aade36640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 0x7f0aa81a12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aade36640 1 -- 172.21.15.52:0/1975054140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 msgr2=0x7f0aa819a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aade36640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 0x7f0aa819a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.218 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aade36640 1 -- 172.21.15.52:0/1975054140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa81a1940 con 0x7f0aa81002d0 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aad635640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 0x7f0aa819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0aade36640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa8199d30 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400e9f0 tx=0x7f0aa400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.216+0000 7f0a9effd640 1 -- 172.21.15.52:0/1975054140 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa400cd80 con 0x7f0aa81002d0 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.217+0000 7f0a9effd640 1 -- 172.21.15.52:0/1975054140 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0aa40108e0 con 0x7f0aa81002d0 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.217+0000 7f0a9effd640 1 -- 172.21.15.52:0/1975054140 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa4018920 con 0x7f0aa81002d0 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.217+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa81a1c30 con 0x7f0aa81002d0 2024-06-02T23:35:44.219 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.217+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81a2130 con 0x7f0aa81002d0 2024-06-02T23:35:44.220 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.220+0000 7f0a9effd640 1 -- 172.21.15.52:0/1975054140 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f0aa4010a90 con 0x7f0aa81002d0 2024-06-02T23:35:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.220+0000 7f0ab2da3740 1 Processor -- start 2024-06-02T23:35:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.220+0000 7f0ab2da3740 1 -- start start 2024-06-02T23:35:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265bc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265c1500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 0x5592265c4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.221 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55922651f4f0 con 0x5592265c1b70 2024-06-02T23:35:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592264bd870 con 0x5592265bc010 2024-06-02T23:35:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55922651b4b0 con 0x5592265bca00 2024-06-02T23:35:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aade36640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aad635640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 0x5592265c4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aad635640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 0x5592265c4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39392/0 (socket says 172.21.15.52:39392) 2024-06-02T23:35:44.222 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aad635640 1 -- 172.21.15.52:0/2121608675 learned_addr learned my addr 172.21.15.52:0/2121608675 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aad635640 1 -- 172.21.15.52:0/2121608675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 msgr2=0x5592265c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aad635640 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0aad635640 1 -- 172.21.15.52:0/2121608675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 msgr2=0x5592265bc430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:35:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.221+0000 7f0a9effd640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a940a2cb0 0x7f0a940a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0a9effd640 1 -- 172.21.15.52:0/1975054140 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0aa40cf310 con 0x7f0aa81002d0 2024-06-02T23:35:44.224 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0aae637640 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265bc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0aad635640 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265bc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0aad635640 1 -- 172.21.15.52:0/2121608675 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592264d99b0 con 0x5592265c1b70 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0aad635640 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 0x5592265c4000 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f0a980115f0 tx=0x7f0a98011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b08ef406b0349fac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0aae637640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a940a2cb0 0x7f0a940a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0a93fff640 1 -- 172.21.15.52:0/2121608675 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801e070 con 0x5592265c1b70 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0a93fff640 1 -- 172.21.15.52:0/2121608675 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9800faf0 con 0x5592265c1b70 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.222+0000 7f0a93fff640 1 -- 172.21.15.52:0/2121608675 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98019b10 con 0x5592265c1b70 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2121608675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 msgr2=0x5592265c4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 0x5592265c4000 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f0a980115f0 tx=0x7f0a98011ab0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0aae637640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a940a2cb0 0x7f0a940a5170 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0aa002f3a0 tx=0x7f0aa00095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2121608675 shutdown_connections 2024-06-02T23:35:44.225 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.226 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265c1b70 0x5592265c4000 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2121608675 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265bc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2121608675 >> 172.21.15.52:0/2121608675 conn(0x5592265b7680 msgr2=0x5592265b7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.223+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2121608675 shutdown_connections 2024-06-02T23:35:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2121608675 wait complete. 2024-06-02T23:35:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 -- start start 2024-06-02T23:35:44.227 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 Processor -- start 2024-06-02T23:35:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.231 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 0x5592265c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5592265c4b70 con 0x5592265bc010 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592264a8ae0 con 0x5592265bca00 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5592265c4cf0 con 0x5592265c1b70 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.224+0000 7f0ab2da3740 10 client.0 _ll_register_callbacks cb 0x5592264e19b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aade36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 0x5592265c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.232 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 0x5592265c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33790/0 (socket says 172.21.15.52:33790) 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 -- 172.21.15.52:0/1861176891 learned_addr learned my addr 172.21.15.52:0/1861176891 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 -- 172.21.15.52:0/1861176891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 msgr2=0x5592265cd820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265cd820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 -- 172.21.15.52:0/1861176891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 msgr2=0x5592265c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 -- 172.21.15.52:0/1861176891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559226233210 con 0x5592265bca00 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0aad635640 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 0x5592265c67e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0a9801c040 tx=0x7f0a9800f870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40aafa69af91a7ed server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0a917fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:44.233 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0a917fa640 1 -- 172.21.15.52:0/1861176891 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801e040 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0a917fa640 1 -- 172.21.15.52:0/1861176891 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a98004040 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9800faf0 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.225+0000 7f0a917fa640 1 -- 172.21.15.52:0/1861176891 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a98004240 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 10 client.30824 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5592262ff620 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0a917fa640 1 -- 172.21.15.52:0/1861176891 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9800faf0 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0a917fa640 1 -- 172.21.15.52:0/1861176891 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0a98021070 con 0x5592265bca00 2024-06-02T23:35:44.234 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 10 client.30824 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 10 client.30824 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 20 client.30824 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592265d2430 0x5592265d4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 4 client.30824 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559226532f10 con 0x5592265d2430 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.226+0000 7f0aade36640 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592265d2430 0x5592265d4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.227+0000 7f0aade36640 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592265d2430 0x5592265d4890 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcab1f7e5a9848e8 server_cookie=d45275941204230f in_seq=0 out_seq=0 2024-06-02T23:35:44.235 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.227+0000 7f0a917fa640 10 client.30824 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.227+0000 7f0a917fa640 1 -- 172.21.15.52:0/1861176891 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559226532f10 con 0x5592265d2430 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.227+0000 7f0a917fa640 10 client.30824 handle_command_reply: tid=0 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.227+0000 7f0ab2da3740 1 client.30824 shutdown 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 msgr2=0x5592265c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 0x5592265c67e0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0a9801c040 tx=0x7f0a9800f870 comp rx=0 tx=0).stop 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 shutdown_connections 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5592265d2430 0x5592265d4890 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265cd820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.236 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bca00 0x5592265c67e0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/1861176891 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 >> 172.21.15.52:0/1861176891 conn(0x5592265b7680 msgr2=0x5592265b67e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0a917fa640 0 client.30824 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 shutdown_connections 2024-06-02T23:35:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 1 -- 172.21.15.52:0/1861176891 wait complete. 2024-06-02T23:35:44.237 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.228+0000 7f0ab2da3740 20 client.30824 trim_cache size 0 max 16384 2024-06-02T23:35:44.254 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 Processor -- start 2024-06-02T23:35:44.254 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.253+0000 7f788f0f6700 1 -- 172.21.15.52:0/3571478568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 msgr2=0x7f7888101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.253+0000 7f788f0f6700 1 --2- 172.21.15.52:0/3571478568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888101800 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f78740098f0 tx=0x7f7874009c00 comp rx=0 tx=0).stop 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 -- 172.21.15.52:0/3571478568 shutdown_connections 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 --2- 172.21.15.52:0/3571478568 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7888101d40 0x7f788810e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 --2- 172.21.15.52:0/3571478568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78881073f0 0x7f78881077d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 --2- 172.21.15.52:0/3571478568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888101800 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 -- 172.21.15.52:0/3571478568 >> 172.21.15.52:0/3571478568 conn(0x7f7888078300 msgr2=0x7f7888078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 -- 172.21.15.52:0/3571478568 shutdown_connections 2024-06-02T23:35:44.255 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f788f0f6700 1 -- 172.21.15.52:0/3571478568 wait complete. 2024-06-02T23:35:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 -- start start 2024-06-02T23:35:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265bdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.256 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265c1b70 0x5592265be120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81aafb0 con 0x5592265bc010 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81acaf0 con 0x5592265c1b70 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.254+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81ab680 con 0x5592265bca00 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aade36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265bdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265cd7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53372/0 (socket says 172.21.15.52:53372) 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aade36640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265bdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39418/0 (socket says 172.21.15.52:39418) 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 -- 172.21.15.52:0/2403440664 learned_addr learned my addr 172.21.15.52:0/2403440664 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.257 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aae637640 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265c1b70 0x5592265be120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 Processor -- start 2024-06-02T23:35:44.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 -- start start 2024-06-02T23:35:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 -- 172.21.15.52:0/2403440664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265c1b70 msgr2=0x5592265be120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265c1b70 0x5592265be120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 -- 172.21.15.52:0/2403440664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 msgr2=0x5592265bdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.258 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265bdbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 -- 172.21.15.52:0/2403440664 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592265b74b0 con 0x5592265bca00 2024-06-02T23:35:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aae637640 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265c1b70 0x5592265be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aade36640 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.259 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f0aad635640 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265cd7e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a98010010 tx=0x7f0a98009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=778255563d59b156 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888197830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.259 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7888101d40 0x7f7888197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78881073f0 0x7f78881982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7888111560 con 0x7f78881073f0 2024-06-02T23:35:44.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78881113e0 con 0x7f7888101d40 2024-06-02T23:35:44.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.255+0000 7f788f0f6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78881116e0 con 0x7f7888101380 2024-06-02T23:35:44.260 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f0a937fe640 1 -- 172.21.15.52:0/2403440664 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801e040 con 0x5592265bca00 2024-06-02T23:35:44.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f788ce92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888197830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f788ce92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888197830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53376/0 (socket says 172.21.15.52:53376) 2024-06-02T23:35:44.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f788ce92700 1 -- 172.21.15.52:0/1185042565 learned_addr learned my addr 172.21.15.52:0/1185042565 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7888101d40 0x7f7888197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f788d693700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78881073f0 0x7f78881982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f0a937fe640 1 -- 172.21.15.52:0/2403440664 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9800f8e0 con 0x5592265bca00 2024-06-02T23:35:44.261 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f0a937fe640 1 -- 172.21.15.52:0/2403440664 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98019730 con 0x5592265bca00 2024-06-02T23:35:44.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 -- 172.21.15.52:0/1185042565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 msgr2=0x7f7888197830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.262 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888197830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 -- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78881073f0 msgr2=0x7f78881982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78881073f0 0x7f78881982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 -- 172.21.15.52:0/1185042565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78881ab6a0 con 0x7f7888101d40 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f788ce92700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.256+0000 7f7887fff700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7888101d40 0x7f7888197d70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f787400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f7885ffb700 1 -- 172.21.15.52:0/1185042565 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7874005100 con 0x7f7888101d40 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78881ab870 con 0x7f7888101d40 2024-06-02T23:35:44.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f7885ffb700 1 -- 172.21.15.52:0/1185042565 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f787400b560 con 0x7f7888101d40 2024-06-02T23:35:44.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f7885ffb700 1 -- 172.21.15.52:0/1185042565 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78740038e0 con 0x7f7888101d40 2024-06-02T23:35:44.264 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78881abc10 con 0x7f7888101d40 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2403440664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 msgr2=0x5592265cd7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265cd7e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0a98010010 tx=0x7f0a98009f60 comp rx=0 tx=0).stop 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2403440664 shutdown_connections 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265bca00 0x5592265cd7e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265c1b70 0x5592265be120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/2403440664 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bc010 0x5592265bdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.268 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2403440664 >> 172.21.15.52:0/2403440664 conn(0x5592265b7680 msgr2=0x5592265c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2403440664 shutdown_connections 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.257+0000 7f0ab2da3740 1 -- 172.21.15.52:0/2403440664 wait complete. 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 -- start start 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 Processor -- start 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bca00 0x5592265cd610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.269 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81ac420 con 0x5592265bca00 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81abd50 con 0x5592265bc010 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81a8f90 con 0x5592265c1b70 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0ab2da3740 10 client.0 _ll_register_callbacks cb 0x7f0a9801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0aae637640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0aad635640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bca00 0x5592265cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0aae637640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265c9b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53388/0 (socket says 172.21.15.52:53388) 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.258+0000 7f0aae637640 1 -- 172.21.15.52:0/244767982 learned_addr learned my addr 172.21.15.52:0/244767982 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.270 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aade36640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aae637640 1 -- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 msgr2=0x5592265cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aae637640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aae637640 1 -- 172.21.15.52:0/244767982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bca00 msgr2=0x5592265cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aae637640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bca00 0x5592265cd610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aae637640 1 -- 172.21.15.52:0/244767982 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592265bd6d0 con 0x5592265c1b70 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aade36640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aad635640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bca00 0x5592265cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0aae637640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265c9b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0031aa0 tx=0x7f0aa002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c897b8889f4c8f34 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.271 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0a917fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0a917fa640 1 -- 172.21.15.52:0/244767982 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0004240 con 0x5592265c1b70 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.259+0000 7f0a917fa640 1 -- 172.21.15.52:0/244767982 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0aa00043e0 con 0x5592265c1b70 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.260+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a9800f8e0 con 0x5592265c1b70 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.260+0000 7f0a917fa640 1 -- 172.21.15.52:0/244767982 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0aa00045e0 con 0x5592265c1b70 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.260+0000 7f0ab2da3740 10 client.30415 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.260+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5592266444a0 con 0x5592265c1b70 2024-06-02T23:35:44.272 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.260+0000 7f0a917fa640 1 -- 172.21.15.52:0/244767982 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a9800f8e0 con 0x5592265c1b70 2024-06-02T23:35:44.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.260+0000 7f7885ffb700 1 -- 172.21.15.52:0/1185042565 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f787400b710 con 0x7f7888101d40 2024-06-02T23:35:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0a917fa640 1 -- 172.21.15.52:0/244767982 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0aa0044040 con 0x5592265c1b70 2024-06-02T23:35:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0ab2da3740 10 client.30415 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:44.274 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0ab2da3740 10 client.30415 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:44.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f7885ffb700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f78780a2ca0 0x7f78780a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0ab2da3740 20 client.30415 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:44.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f7885ffb700 1 -- 172.21.15.52:0/1185042565 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f78740cfe70 con 0x7f7888101d40 2024-06-02T23:35:44.275 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559226644ed0 0x5592266472c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0ab2da3740 4 client.30415 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:35:44.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f788ce92700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f78780a2ca0 0x7f78780a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55922652cc20 con 0x559226644ed0 2024-06-02T23:35:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.261+0000 7f0aade36640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559226644ed0 0x5592266472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.276 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.262+0000 7f0aade36640 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559226644ed0 0x5592266472c0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84b1b6974c05759 server_cookie=31b5ff7e805a4c5b in_seq=0 out_seq=0 2024-06-02T23:35:44.277 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.262+0000 7f0a917fa640 10 client.30415 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:44.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.262+0000 7f788ce92700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f78780a2ca0 0x7f78780a5160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f787c011550 tx=0x7f787c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.262+0000 7f7899979b80 1 Processor -- start 2024-06-02T23:35:44.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.262+0000 7f7899979b80 1 -- start start 2024-06-02T23:35:44.281 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7899979b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 0x559c4e5aca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7899979b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5acf60 0x559c4e5a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7899979b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5a7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7899979b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c4e4f8810 con 0x559c4e5ac640 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7899979b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c4e4a0720 con 0x559c4e5acf60 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7899979b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c4e4f47d0 con 0x559c4e5a6f20 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788ce92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 0x559c4e5aca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788ce92700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 0x559c4e5aca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39458/0 (socket says 172.21.15.52:39458) 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788ce92700 1 -- 172.21.15.52:0/3245407205 learned_addr learned my addr 172.21.15.52:0/3245407205 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5a7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f7887fff700 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5acf60 0x559c4e5a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 -- 172.21.15.52:0/3245407205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5acf60 msgr2=0x559c4e5a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5acf60 0x559c4e5a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.282 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 -- 172.21.15.52:0/3245407205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 msgr2=0x559c4e5aca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 0x559c4e5aca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 -- 172.21.15.52:0/3245407205 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4e496f50 con 0x559c4e5a6f20 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788ce92700 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 0x559c4e5aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.263+0000 7f788d693700 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5a7340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7880009cd0 tx=0x7f78800099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46ac3e9691d2288 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.264+0000 7f7872ffd700 1 -- 172.21.15.52:0/3245407205 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788001b070 con 0x559c4e5a6f20 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.264+0000 7f7872ffd700 1 -- 172.21.15.52:0/3245407205 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7880004db0 con 0x559c4e5a6f20 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.264+0000 7f7872ffd700 1 -- 172.21.15.52:0/3245407205 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880005240 con 0x559c4e5a6f20 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 -- 172.21.15.52:0/3245407205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 msgr2=0x559c4e5a7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5a7340 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f7880009cd0 tx=0x7f78800099e0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 -- 172.21.15.52:0/3245407205 shutdown_connections 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5a7340 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5ac640 0x559c4e5aca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.283 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 --2- 172.21.15.52:0/3245407205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5acf60 0x559c4e5a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 -- 172.21.15.52:0/3245407205 >> 172.21.15.52:0/3245407205 conn(0x559c4e4d1cc0 msgr2=0x559c4e5a1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 -- 172.21.15.52:0/3245407205 shutdown_connections 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 -- 172.21.15.52:0/3245407205 wait complete. 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 -- start start 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.265+0000 7f7899979b80 1 Processor -- start 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x559c4e5b39d0 con 0x559c4e5acf60 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559c4e498bf0 con 0x559c4e5ac640 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39460/0 (socket says 172.21.15.52:39460) 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 -- 172.21.15.52:0/1023165809 learned_addr learned my addr 172.21.15.52:0/1023165809 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.284 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788ce92700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559c4e5b3b50 con 0x559c4e5a6f20 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7887fff700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f7899979b80 10 client.0 _ll_register_callbacks cb 0x559c4e492920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 -- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 msgr2=0x559c4e5b2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 -- 172.21.15.52:0/1023165809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 msgr2=0x559c4e5afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.266+0000 7f788d693700 1 -- 172.21.15.52:0/1023165809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4e4478d0 con 0x559c4e5acf60 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f788d693700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5b0530 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f788000e1d0 tx=0x7f7880017b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf637e48dc654152 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f7867fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f7867fff700 1 -- 172.21.15.52:0/1023165809 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880025070 con 0x559c4e5acf60 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f7867fff700 1 -- 172.21.15.52:0/1023165809 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f788001f610 con 0x559c4e5acf60 2024-06-02T23:35:44.285 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7880004db0 con 0x559c4e5acf60 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f7867fff700 1 -- 172.21.15.52:0/1023165809 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f788002e050 con 0x559c4e5acf60 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.267+0000 7f7899979b80 10 client.21615 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559c4e5be300 con 0x559c4e5acf60 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7867fff700 1 -- 172.21.15.52:0/1023165809 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7880004db0 con 0x559c4e5acf60 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7867fff700 1 -- 172.21.15.52:0/1023165809 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f78800159a0 con 0x559c4e5acf60 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 10 client.21615 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 10 client.21615 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 20 client.21615 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e5c2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 4 client.21615 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:44.286 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.268+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c4e505d10 con 0x559c4e5c0880 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.269+0000 7f788ce92700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e5c2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.269+0000 7f788ce92700 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e5c2c70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=88e49f738fb089d9 server_cookie=ee199c0aee44828c in_seq=0 out_seq=0 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.269+0000 7f7867fff700 10 client.21615 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.270+0000 7f7867fff700 1 -- 172.21.15.52:0/1023165809 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x559c4e505d10 con 0x559c4e5c0880 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.270+0000 7f7867fff700 10 client.21615 handle_command_reply: tid=0 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.270+0000 7f7899979b80 1 client.21615 shutdown 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 msgr2=0x559c4e5b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5b0530 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f788000e1d0 tx=0x7f7880017b30 comp rx=0 tx=0).stop 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 shutdown_connections 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5b0530 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e5c2c70 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.287 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7867fff700 0 client.21615 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:35:44.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 --2- 172.21.15.52:0/1023165809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 >> 172.21.15.52:0/1023165809 conn(0x559c4e4d1cc0 msgr2=0x559c4e5a59d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 shutdown_connections 2024-06-02T23:35:44.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.271+0000 7f7899979b80 1 -- 172.21.15.52:0/1023165809 wait complete. 2024-06-02T23:35:44.288 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.272+0000 7f7899979b80 20 client.21615 trim_cache size 0 max 16384 2024-06-02T23:35:44.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.299+0000 7f7899979b80 1 Processor -- start 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.299+0000 7f7899979b80 1 -- start start 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.299+0000 7f7899979b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5a6f20 0x559c4e5a69e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.299+0000 7f7899979b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5ac640 0x559c4e5a19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.299+0000 7f7899979b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e59b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f7899979b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78881b30d0 con 0x559c4e5acf60 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f7899979b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78881b4c10 con 0x559c4e5a6f20 2024-06-02T23:35:44.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f7899979b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78881b37a0 con 0x559c4e5ac640 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e59b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e59b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39464/0 (socket says 172.21.15.52:39464) 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 -- 172.21.15.52:0/3224598879 learned_addr learned my addr 172.21.15.52:0/3224598879 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f7887fff700 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5ac640 0x559c4e5a19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788ce92700 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5a6f20 0x559c4e5a69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 -- 172.21.15.52:0/3224598879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5ac640 msgr2=0x559c4e5a19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5ac640 0x559c4e5a19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 -- 172.21.15.52:0/3224598879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5a6f20 msgr2=0x559c4e5a69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5a6f20 0x559c4e5a69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.300+0000 7f788d693700 1 -- 172.21.15.52:0/3224598879 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c4e5b0f20 con 0x559c4e5acf60 2024-06-02T23:35:44.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.301+0000 7f788d693700 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e59b510 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7880015d70 tx=0x7f7880007c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a85f7b3587b234e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.301+0000 7f78727fc700 1 -- 172.21.15.52:0/3224598879 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7880025070 con 0x559c4e5acf60 2024-06-02T23:35:44.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.301+0000 7f78727fc700 1 -- 172.21.15.52:0/3224598879 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7880015410 con 0x559c4e5acf60 2024-06-02T23:35:44.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.302+0000 7f78727fc700 1 -- 172.21.15.52:0/3224598879 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788001b040 con 0x559c4e5acf60 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 -- 172.21.15.52:0/3224598879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 msgr2=0x559c4e59b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e59b510 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f7880015d70 tx=0x7f7880007c20 comp rx=0 tx=0).stop 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 -- 172.21.15.52:0/3224598879 shutdown_connections 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e59b510 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5a6f20 0x559c4e5a69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 --2- 172.21.15.52:0/3224598879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5ac640 0x559c4e5a19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 -- 172.21.15.52:0/3224598879 >> 172.21.15.52:0/3224598879 conn(0x559c4e4d1cc0 msgr2=0x559c4e59ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 -- 172.21.15.52:0/3224598879 shutdown_connections 2024-06-02T23:35:44.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.303+0000 7f7899979b80 1 -- 172.21.15.52:0/3224598879 wait complete. 2024-06-02T23:35:44.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.304+0000 7f7899979b80 1 -- start start 2024-06-02T23:35:44.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.304+0000 7f7899979b80 1 Processor -- start 2024-06-02T23:35:44.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f788ce92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f788ce92700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53422/0 (socket says 172.21.15.52:53422) 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f788ce92700 1 -- 172.21.15.52:0/774135094 learned_addr learned my addr 172.21.15.52:0/774135094 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7887fff700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f78881b4540 con 0x559c4e5acf60 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f788d693700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78881b3e70 con 0x559c4e5ac640 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78881b1090 con 0x559c4e5a6f20 2024-06-02T23:35:44.307 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.305+0000 7f7899979b80 10 client.0 _ll_register_callbacks cb 0x559c4e492920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f788d693700 1 -- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 msgr2=0x559c4e5b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f788d693700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f788d693700 1 -- 172.21.15.52:0/774135094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 msgr2=0x559c4e5b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f788d693700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f788d693700 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c4e5c7fd0 con 0x559c4e5acf60 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f7887fff700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:44.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.306+0000 7f788d693700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5c7800 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f7880009e60 tx=0x7f7880009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6323cf267b5c4291 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.307+0000 7f7867fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.307+0000 7f7867fff700 1 -- 172.21.15.52:0/774135094 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788001b050 con 0x559c4e5acf60 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.307+0000 7f7867fff700 1 -- 172.21.15.52:0/774135094 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7880004040 con 0x559c4e5acf60 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.307+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7880015410 con 0x559c4e5acf60 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.307+0000 7f7867fff700 1 -- 172.21.15.52:0/774135094 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f7880025040 con 0x559c4e5acf60 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.307+0000 7f7899979b80 10 client.21621 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x559c4e62ac40 con 0x559c4e5acf60 2024-06-02T23:35:44.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7867fff700 1 -- 172.21.15.52:0/774135094 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7880017c50 con 0x559c4e5acf60 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7867fff700 1 -- 172.21.15.52:0/774135094 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f788003c720 con 0x559c4e5acf60 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7899979b80 10 client.21621 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7899979b80 10 client.21621 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7899979b80 20 client.21621 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.308+0000 7f7899979b80 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e62d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.309+0000 7f7899979b80 4 client.21621 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.309+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f7880015410 con 0x559c4e5c0880 2024-06-02T23:35:44.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.309+0000 7f788ce92700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e62d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:44.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f7867fff700 10 client.21621 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:35:44.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f788ce92700 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e62d2d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3871c8910a86ea1c server_cookie=7c44bf2999b387db in_seq=0 out_seq=0 2024-06-02T23:35:44.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f7867fff700 1 -- 172.21.15.52:0/774135094 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f7880015410 con 0x559c4e5c0880 2024-06-02T23:35:44.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f7867fff700 10 client.21621 handle_command_reply: tid=0 2024-06-02T23:35:44.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f7899979b80 1 client.21621 shutdown 2024-06-02T23:35:44.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 msgr2=0x559c4e5c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.310+0000 7f7899979b80 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5c7800 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f7880009e60 tx=0x7f7880009960 comp rx=0 tx=0).stop 2024-06-02T23:35:44.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 shutdown_connections 2024-06-02T23:35:44.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7899979b80 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x559c4e5acf60 0x559c4e5c7800 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7899979b80 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x559c4e5c0880 0x559c4e62d2d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7899979b80 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559c4e5a6f20 0x559c4e5b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7899979b80 1 --2- 172.21.15.52:0/774135094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559c4e5ac640 0x559c4e5b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 >> 172.21.15.52:0/774135094 conn(0x559c4e4d1cc0 msgr2=0x559c4e5ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.311+0000 7f7867fff700 0 client.21621 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.312+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 shutdown_connections 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.312+0000 7f7899979b80 1 -- 172.21.15.52:0/774135094 wait complete. 2024-06-02T23:35:44.313 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:35:44.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.312+0000 7f7899979b80 20 client.21621 trim_cache size 0 max 16384 2024-06-02T23:35:44.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f78780a2ca0 msgr2=0x7f78780a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f78780a2ca0 0x7f78780a5160 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f787c011550 tx=0x7f787c009260 comp rx=0 tx=0).stop 2024-06-02T23:35:44.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7888101d40 msgr2=0x7f7888197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7888101d40 0x7f7888197d70 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f787400fa00 comp rx=0 tx=0).stop 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 shutdown_connections 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f78780a2ca0 0x7f78780a5160 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f78881073f0 0x7f78881982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7888101380 0x7f7888197830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 --2- 172.21.15.52:0/1185042565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7888101d40 0x7f7888197d70 secure :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7874000c00 tx=0x7f787400fa00 comp rx=0 tx=0).stop 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 >> 172.21.15.52:0/1185042565 conn(0x7f7888078300 msgr2=0x7f78881001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 shutdown_connections 2024-06-02T23:35:44.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:44.313+0000 7f788f0f6700 1 -- 172.21.15.52:0/1185042565 wait complete. 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.336+0000 7f0a917fa640 1 -- 172.21.15.52:0/244767982 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55922652cc20 con 0x559226644ed0 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.336+0000 7f0a917fa640 10 client.30415 handle_command_reply: tid=0 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.336+0000 7f0ab2da3740 1 client.30415 shutdown 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.336+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 msgr2=0x5592265c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.336+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265c9b70 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0031aa0 tx=0x7f0aa002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 shutdown_connections 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x559226644ed0 0x5592266472c0 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0a917fa640 0 client.30415 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:44.337 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5592265c1b70 0x5592265c9b70 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5592265bca00 0x5592265cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 --2- 172.21.15.52:0/244767982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592265bc010 0x5592265cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 >> 172.21.15.52:0/244767982 conn(0x5592265b7680 msgr2=0x5592265c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 shutdown_connections 2024-06-02T23:35:44.338 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 1 -- 172.21.15.52:0/244767982 wait complete. 2024-06-02T23:35:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.337+0000 7f0ab2da3740 20 client.30415 trim_cache size 0 max 16384 2024-06-02T23:35:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:35:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:35:44.339 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a940a2cb0 msgr2=0x7f0a940a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a940a2cb0 0x7f0a940a5170 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0aa002f3a0 tx=0x7f0aa00095c0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 msgr2=0x7f0aa8199d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa8199d30 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f0aa400e9f0 tx=0x7f0aa400eeb0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 shutdown_connections 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f0a940a2cb0 0x7f0a940a5170 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0aa81015d0 0x7f0aa81a12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.340 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8100c30 0x7f0aa819a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 --2- 172.21.15.52:0/1975054140 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0aa81002d0 0x7f0aa8199d30 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 >> 172.21.15.52:0/1975054140 conn(0x7f0aa80fbe90 msgr2=0x7f0aa80fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 shutdown_connections 2024-06-02T23:35:44.341 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:44.338+0000 7f0ab009a640 1 -- 172.21.15.52:0/1975054140 wait complete. 2024-06-02T23:35:44.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 77628 6.58 MB/sec execute 532 sec latency 96.830 ms 2024-06-02T23:35:45.062 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:35:45.430 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:45.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 77789 6.58 MB/sec execute 533 sec latency 61.614 ms 2024-06-02T23:35:45.710 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:45 smithi052 ceph-mon[32340]: pgmap v1147: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-06-02T23:35:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:45 smithi123 ceph-mon[35977]: pgmap v1147: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-06-02T23:35:45.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:45 smithi174 ceph-mon[35994]: pgmap v1147: 97 pgs: 97 active+clean; 592 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 774 KiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-06-02T23:35:46.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 77966 6.59 MB/sec execute 534 sec latency 135.365 ms 2024-06-02T23:35:47.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 78138 6.60 MB/sec execute 535 sec latency 143.000 ms 2024-06-02T23:35:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:47 smithi123 ceph-mon[35977]: pgmap v1148: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-06-02T23:35:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:47 smithi174 ceph-mon[35994]: pgmap v1148: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-06-02T23:35:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:47 smithi052 ceph-mon[32340]: pgmap v1148: 97 pgs: 97 active+clean; 595 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.6 MiB/s wr, 308 op/s 2024-06-02T23:35:48.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 78310 6.60 MB/sec execute 536 sec latency 101.468 ms 2024-06-02T23:35:49.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 78465 6.61 MB/sec execute 537 sec latency 144.061 ms 2024-06-02T23:35:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:49 smithi123 ceph-mon[35977]: pgmap v1149: 97 pgs: 97 active+clean; 589 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-06-02T23:35:49.800 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:49 smithi052 ceph-mon[32340]: pgmap v1149: 97 pgs: 97 active+clean; 589 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-06-02T23:35:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:49 smithi174 ceph-mon[35994]: pgmap v1149: 97 pgs: 97 active+clean; 589 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 962 KiB/s rd, 3.8 MiB/s wr, 249 op/s 2024-06-02T23:35:50.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 -- 172.21.15.52:0/2839388062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 msgr2=0x7f5e54119d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.257 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 --2- 172.21.15.52:0/2839388062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e54119d70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009f90 tx=0x7f5e4c00b330 comp rx=0 tx=0).stop 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 -- 172.21.15.52:0/2839388062 shutdown_connections 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 --2- 172.21.15.52:0/2839388062 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e54119d70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 --2- 172.21.15.52:0/2839388062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410d0c0 0x7f5e5410d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 --2- 172.21.15.52:0/2839388062 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e5410da70 0x7f5e54111ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 -- 172.21.15.52:0/2839388062 >> 172.21.15.52:0/2839388062 conn(0x7f5e5406d680 msgr2=0x7f5e5410b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 -- 172.21.15.52:0/2839388062 shutdown_connections 2024-06-02T23:35:50.258 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.256+0000 7f5e59111700 1 -- 172.21.15.52:0/2839388062 wait complete. 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.257+0000 7f5e59111700 1 Processor -- start 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 -- start start 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e5410d0c0 0x7f5e541a3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410da70 0x7f5e541a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e541a7e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5411d090 con 0x7f5e5410d0c0 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5411cf10 con 0x7f5e541121a0 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e59111700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e5411d210 con 0x7f5e5410da70 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e5359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e541a7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e52d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e5410d0c0 0x7f5e541a3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e5259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410da70 0x7f5e541a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e5359e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e541a7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33874/0 (socket says 172.21.15.52:33874) 2024-06-02T23:35:50.260 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.258+0000 7f5e5359e700 1 -- 172.21.15.52:0/2962235525 learned_addr learned my addr 172.21.15.52:0/2962235525 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5259c700 1 -- 172.21.15.52:0/2962235525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 msgr2=0x7f5e541a7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e541a7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5259c700 1 -- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e5410d0c0 msgr2=0x7f5e541a3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e5410d0c0 0x7f5e541a3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5259c700 1 -- 172.21.15.52:0/2962235525 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e541a8540 con 0x7f5e5410da70 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5359e700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e541a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410da70 0x7f5e541a3ac0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5e48005fd0 tx=0x7f5e4800eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e3bfff700 1 -- 172.21.15.52:0/2962235525 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e48010900 con 0x7f5e5410da70 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e3bfff700 1 -- 172.21.15.52:0/2962235525 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e48018420 con 0x7f5e5410da70 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e3bfff700 1 -- 172.21.15.52:0/2962235525 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4800f680 con 0x7f5e5410da70 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e541a8830 con 0x7f5e5410da70 2024-06-02T23:35:50.261 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.259+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e5406dee0 con 0x7f5e5410da70 2024-06-02T23:35:50.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.262+0000 7f5e3bfff700 1 -- 172.21.15.52:0/2962235525 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5e480185d0 con 0x7f5e5410da70 2024-06-02T23:35:50.263 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.263+0000 7f5e3bfff700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5e3c0a2c20 0x7f5e3c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.263+0000 7f5e3bfff700 1 -- 172.21.15.52:0/2962235525 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5e480cefe0 con 0x7f5e5410da70 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.263+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5e3c0a2c20 0x7f5e3c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 Processor -- start 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 -- start start 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5e3c0a2c20 0x7f5e3c0a50e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009680 tx=0x7f5e44016040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28640 0x5628a2c28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.265 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c22f20 0x5628a2c23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5628a2b74810 con 0x5628a2c28f60 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5628a2b1c720 con 0x5628a2c22f20 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e63994b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628a2b707d0 con 0x5628a2c28640 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e5259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e5259c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c229e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39488/0 (socket says 172.21.15.52:39488) 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e5259c700 1 -- 172.21.15.52:0/2957755552 learned_addr learned my addr 172.21.15.52:0/2957755552 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e5359e700 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c22f20 0x5628a2c23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.266 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.264+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28640 0x5628a2c28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e5259c700 1 -- 172.21.15.52:0/2957755552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28640 msgr2=0x5628a2c28a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28640 0x5628a2c28a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.267 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e5259c700 1 -- 172.21.15.52:0/2957755552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c22f20 msgr2=0x5628a2c23340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c22f20 0x5628a2c23340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e5259c700 1 -- 172.21.15.52:0/2957755552 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a2b12f50 con 0x5628a2c28f60 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c229e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5e4809b7a0 tx=0x7f5e4809b240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9982aa87bd0b4c52 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e397fa700 1 -- 172.21.15.52:0/2957755552 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4809bf10 con 0x5628a2c28f60 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.265+0000 7f5e397fa700 1 -- 172.21.15.52:0/2957755552 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4809b810 con 0x5628a2c28f60 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.266+0000 7f5e397fa700 1 -- 172.21.15.52:0/2957755552 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480a4250 con 0x5628a2c28f60 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 -- 172.21.15.52:0/2957755552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 msgr2=0x5628a2c229e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c229e0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f5e4809b7a0 tx=0x7f5e4809b240 comp rx=0 tx=0).stop 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 -- 172.21.15.52:0/2957755552 shutdown_connections 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c22f20 0x5628a2c23340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28640 0x5628a2c28a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.268 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2957755552 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c229e0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 -- 172.21.15.52:0/2957755552 >> 172.21.15.52:0/2957755552 conn(0x5628a2b4dcc0 msgr2=0x5628a2c1d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 -- 172.21.15.52:0/2957755552 shutdown_connections 2024-06-02T23:35:50.269 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 -- 172.21.15.52:0/2957755552 wait complete. 2024-06-02T23:35:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 -- start start 2024-06-02T23:35:50.270 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.267+0000 7f5e63994b80 1 Processor -- start 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c2ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c2c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c2c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5628a2c2f970 con 0x5628a2c28f60 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5628a2b14bf0 con 0x5628a2c28640 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5628a2c2faf0 con 0x5628a2c22f20 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e5259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c2c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e52d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c2ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e5359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c2c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e5259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c2c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33882/0 (socket says 172.21.15.52:33882) 2024-06-02T23:35:50.271 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e5259c700 1 -- 172.21.15.52:0/3765447650 learned_addr learned my addr 172.21.15.52:0/3765447650 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.268+0000 7f5e63994b80 10 client.0 _ll_register_callbacks cb 0x5628a2b0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e5259c700 1 -- 172.21.15.52:0/3765447650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 msgr2=0x5628a2c2ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e5259c700 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c2ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e5259c700 1 -- 172.21.15.52:0/3765447650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 msgr2=0x5628a2c2c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e5259c700 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c2c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e5259c700 1 -- 172.21.15.52:0/3765447650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a2ac38d0 con 0x5628a2c28640 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e5259c700 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c2c050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5e48097c70 tx=0x7f5e4809fc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bba4183ab5d7ffb8 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e36ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e36ff5700 1 -- 172.21.15.52:0/3765447650 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e480b9060 con 0x5628a2c28640 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4809b810 con 0x5628a2c28640 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e36ff5700 1 -- 172.21.15.52:0/3765447650 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5e4809cb40 con 0x5628a2c28640 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.269+0000 7f5e36ff5700 1 -- 172.21.15.52:0/3765447650 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5e480a4480 con 0x5628a2c28640 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.270+0000 7f5e63994b80 10 client.30860 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:50.272 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.270+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5628a2c3a2a0 con 0x5628a2c28640 2024-06-02T23:35:50.273 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.270+0000 7f5e36ff5700 1 -- 172.21.15.52:0/3765447650 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4809cd40 con 0x5628a2c28640 2024-06-02T23:35:50.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.270+0000 7f5e36ff5700 1 -- 172.21.15.52:0/3765447650 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5e4809a6d0 con 0x5628a2c28640 2024-06-02T23:35:50.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.270+0000 7f5e63994b80 10 client.30860 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:50.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.270+0000 7f5e63994b80 10 client.30860 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:35:50.274 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.271+0000 7f5e63994b80 20 client.30860 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.271+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2c3c9e0 0x5628a2c3ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.271+0000 7f5e63994b80 4 client.30860 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.271+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5e4809b810 con 0x5628a2c3c9e0 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.271+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2c3c9e0 0x5628a2c3ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.272+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2c3c9e0 0x5628a2c3ee00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3de24d2630e5c368 server_cookie=7c76d3464877edcb in_seq=0 out_seq=0 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.272+0000 7f5e36ff5700 10 client.30860 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e36ff5700 1 -- 172.21.15.52:0/3765447650 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x7f5e4809b810 con 0x5628a2c3c9e0 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e36ff5700 10 client.30860 handle_command_reply: tid=0 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 client.30860 shutdown 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 msgr2=0x5628a2c2c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c2c050 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f5e48097c70 tx=0x7f5e4809fc40 comp rx=0 tx=0).stop 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 shutdown_connections 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2c3c9e0 0x5628a2c3ee00 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.275 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c2c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c2ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3765447650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c2c050 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 >> 172.21.15.52:0/3765447650 conn(0x5628a2b4dcc0 msgr2=0x5628a2c1d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e36ff5700 0 client.30860 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.273+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 shutdown_connections 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.274+0000 7f5e63994b80 1 -- 172.21.15.52:0/3765447650 wait complete. 2024-06-02T23:35:50.276 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.274+0000 7f5e63994b80 20 client.30860 trim_cache size 0 max 16384 2024-06-02T23:35:50.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 Processor -- start 2024-06-02T23:35:50.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 -- start start 2024-06-02T23:35:50.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.303 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c29c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c17510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541b30e0 con 0x5628a2c28f60 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541b4c20 con 0x5628a2c28640 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e63994b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541b37b0 con 0x5628a2c22f20 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e5359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c17510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e5359e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c17510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39512/0 (socket says 172.21.15.52:39512) 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e5359e700 1 -- 172.21.15.52:0/3264099439 learned_addr learned my addr 172.21.15.52:0/3264099439 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.300+0000 7f5e5259c700 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c29c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e5359e700 1 -- 172.21.15.52:0/3264099439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 msgr2=0x5628a2c229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e5359e700 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e5359e700 1 -- 172.21.15.52:0/3264099439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 msgr2=0x5628a2c29c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e5359e700 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c29c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e5359e700 1 -- 172.21.15.52:0/3264099439 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628a2c2a5c0 con 0x5628a2c28f60 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e5359e700 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c17510 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009550 tx=0x7f5e4c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa29dbe3bbd3864 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e38ff9700 1 -- 172.21.15.52:0/3264099439 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c00bb20 con 0x5628a2c28f60 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e38ff9700 1 -- 172.21.15.52:0/3264099439 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4c00bcc0 con 0x5628a2c28f60 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.301+0000 7f5e38ff9700 1 -- 172.21.15.52:0/3264099439 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c005280 con 0x5628a2c28f60 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 -- 172.21.15.52:0/3264099439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 msgr2=0x5628a2c17510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c17510 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c009550 tx=0x7f5e4c009d60 comp rx=0 tx=0).stop 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 -- 172.21.15.52:0/3264099439 shutdown_connections 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c28f60 0x5628a2c17510 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c22f20 0x5628a2c229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 --2- 172.21.15.52:0/3264099439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c29c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.302+0000 7f5e63994b80 1 -- 172.21.15.52:0/3264099439 >> 172.21.15.52:0/3264099439 conn(0x5628a2b4dcc0 msgr2=0x5628a2c16e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:50.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.303+0000 7f5e63994b80 1 -- 172.21.15.52:0/3264099439 shutdown_connections 2024-06-02T23:35:50.306 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.303+0000 7f5e63994b80 1 -- 172.21.15.52:0/3264099439 wait complete. 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.303+0000 7f5e63994b80 1 -- start start 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.303+0000 7f5e63994b80 1 Processor -- start 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 0x5628a2c31d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c32290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28f60 0x5628a2c327d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541b4550 con 0x5628a2c22f20 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5e541b3e80 con 0x5628a2c28640 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e540720c0 con 0x5628a2c28f60 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e63994b80 10 client.0 _ll_register_callbacks cb 0x5628a2b0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 0x5628a2c31d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 0x5628a2c31d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39514/0 (socket says 172.21.15.52:39514) 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 -- 172.21.15.52:0/2036120255 learned_addr learned my addr 172.21.15.52:0/2036120255 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c32290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e5359e700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28f60 0x5628a2c327d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 -- 172.21.15.52:0/2036120255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28f60 msgr2=0x5628a2c327d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28f60 0x5628a2c327d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 -- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 msgr2=0x5628a2c32290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c32290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 -- 172.21.15.52:0/2036120255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628a2ca5630 con 0x5628a2c22f20 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e52d9d700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 0x5628a2c31d50 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5e44020690 tx=0x7f5e44004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf9bb1aeaf0af3e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.304+0000 7f5e36ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e36ff5700 1 -- 172.21.15.52:0/2036120255 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4402c630 con 0x5628a2c22f20 2024-06-02T23:35:50.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e36ff5700 1 -- 172.21.15.52:0/2036120255 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5e44009040 con 0x5628a2c22f20 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e4c00bcc0 con 0x5628a2c22f20 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e36ff5700 1 -- 172.21.15.52:0/2036120255 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5e44042070 con 0x5628a2c22f20 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e63994b80 10 client.21639 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5628a2ca68c0 con 0x5628a2c22f20 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.305+0000 7f5e36ff5700 1 -- 172.21.15.52:0/2036120255 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5e4402cc80 con 0x5628a2c22f20 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e36ff5700 1 -- 172.21.15.52:0/2036120255 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f5e4403b990 con 0x5628a2c22f20 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e63994b80 10 client.21639 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e63994b80 10 client.21639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e63994b80 20 client.21639 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2ca6fb0 0x5628a2ca93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e63994b80 4 client.21639 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f5e4c00bcc0 con 0x5628a2ca6fb0 2024-06-02T23:35:50.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2ca6fb0 0x5628a2ca93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:50.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e5259c700 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2ca6fb0 0x5628a2ca93d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=29a9a8955c4fe9c0 server_cookie=5de8dd4a4e8e7df6 in_seq=0 out_seq=0 2024-06-02T23:35:50.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.306+0000 7f5e36ff5700 10 client.21639 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e36ff5700 1 -- 172.21.15.52:0/2036120255 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f5e4c00bcc0 con 0x5628a2ca6fb0 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e36ff5700 10 client.21639 handle_command_reply: tid=0 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 client.21639 shutdown 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 msgr2=0x5628a2c31d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 0x5628a2c31d50 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f5e44020690 tx=0x7f5e44004750 comp rx=0 tx=0).stop 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 shutdown_connections 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5628a2ca6fb0 0x5628a2ca93d0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5628a2c28f60 0x5628a2c327d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5628a2c22f20 0x5628a2c31d50 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 --2- 172.21.15.52:0/2036120255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5628a2c28640 0x5628a2c32290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e36ff5700 0 client.21639 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:35:50.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 >> 172.21.15.52:0/2036120255 conn(0x5628a2b4dcc0 msgr2=0x5628a2c271a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:50.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 shutdown_connections 2024-06-02T23:35:50.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.308+0000 7f5e63994b80 1 -- 172.21.15.52:0/2036120255 wait complete. 2024-06-02T23:35:50.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e63994b80 20 client.21639 trim_cache size 0 max 16384 2024-06-02T23:35:50.313 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5e3c0a2c20 msgr2=0x7f5e3c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5e3c0a2c20 0x7f5e3c0a50e0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009680 tx=0x7f5e44016040 comp rx=0 tx=0).stop 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410da70 msgr2=0x7f5e541a3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410da70 0x7f5e541a3ac0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5e48005fd0 tx=0x7f5e4800eca0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 shutdown_connections 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5e3c0a2c20 0x7f5e3c0a50e0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5e541121a0 0x7f5e541a7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5e5410d0c0 0x7f5e541a3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 --2- 172.21.15.52:0/2962235525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e5410da70 0x7f5e541a3ac0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:50.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.309+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 >> 172.21.15.52:0/2962235525 conn(0x7f5e5406d680 msgr2=0x7f5e5410b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:50.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.310+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 shutdown_connections 2024-06-02T23:35:50.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:50.310+0000 7f5e59111700 1 -- 172.21.15.52:0/2962235525 wait complete. 2024-06-02T23:35:50.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 78598 6.60 MB/sec execute 538 sec latency 139.433 ms 2024-06-02T23:35:51.206 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:35:51.463 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:51 smithi052 ceph-mon[32340]: pgmap v1150: 97 pgs: 97 active+clean; 587 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-06-02T23:35:51.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 78785 6.61 MB/sec execute 539 sec latency 60.848 ms 2024-06-02T23:35:51.559 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:51 smithi123 ceph-mon[35977]: pgmap v1150: 97 pgs: 97 active+clean; 587 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-06-02T23:35:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:51 smithi174 ceph-mon[35994]: pgmap v1150: 97 pgs: 97 active+clean; 587 MiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 810 KiB/s rd, 4.1 MiB/s wr, 262 op/s 2024-06-02T23:35:52.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 78959 6.61 MB/sec execute 540 sec latency 63.217 ms 2024-06-02T23:35:53.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 79123 6.61 MB/sec execute 541 sec latency 94.465 ms 2024-06-02T23:35:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 -- 172.21.15.52:0/2416411567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8020108500 msgr2=0x7f80201088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 --2- 172.21.15.52:0/2416411567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8020108500 0x7f80201088e0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f8010009990 tx=0x7f8010009ca0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 -- 172.21.15.52:0/2416411567 shutdown_connections 2024-06-02T23:35:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 --2- 172.21.15.52:0/2416411567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f802006ab30 0x7f802010f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 --2- 172.21.15.52:0/2416411567 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f8020108500 0x7f80201088e0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.653 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 --2- 172.21.15.52:0/2416411567 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006a170 0x7f802006a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 -- 172.21.15.52:0/2416411567 >> 172.21.15.52:0/2416411567 conn(0x7f8020078340 msgr2=0x7f8020078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:53.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 -- 172.21.15.52:0/2416411567 shutdown_connections 2024-06-02T23:35:53.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.652+0000 7f8025330700 1 -- 172.21.15.52:0/2416411567 wait complete. 2024-06-02T23:35:53.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.653+0000 7f8025330700 1 Processor -- start 2024-06-02T23:35:53.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.653+0000 7f8025330700 1 -- start start 2024-06-02T23:35:53.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f8025330700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 0x7f80201a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f8025330700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 0x7f80201a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f801effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 0x7f80201a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f8025330700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8020108500 0x7f802019ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f801effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 0x7f80201a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39538/0 (socket says 172.21.15.52:39538) 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 0x7f80201a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 0x7f80201a0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53492/0 (socket says 172.21.15.52:53492) 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f801effd700 1 -- 172.21.15.52:0/3013132467 learned_addr learned my addr 172.21.15.52:0/3013132467 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f8020102040 con 0x7f802006a170 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8020101ec0 con 0x7f8020108500 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80201021c0 con 0x7f802006ab30 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.654+0000 7f801f7fe700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8020108500 0x7f802019ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801e7fc700 1 -- 172.21.15.52:0/3013132467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8020108500 msgr2=0x7f802019ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801e7fc700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8020108500 0x7f802019ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801e7fc700 1 -- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 msgr2=0x7f80201a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801e7fc700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 0x7f80201a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801e7fc700 1 -- 172.21.15.52:0/3013132467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802019b410 con 0x7f802006ab30 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801effd700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 0x7f80201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f801e7fc700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 0x7f80201a0fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f801400d950 tx=0x7f801400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f8007fff700 1 -- 172.21.15.52:0/3013132467 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8014016070 con 0x7f802006ab30 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f8007fff700 1 -- 172.21.15.52:0/3013132467 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801400fd40 con 0x7f802006ab30 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f8007fff700 1 -- 172.21.15.52:0/3013132467 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801400e3b0 con 0x7f802006ab30 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f802019b700 con 0x7f802006ab30 2024-06-02T23:35:53.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.655+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80201a79e0 con 0x7f802006ab30 2024-06-02T23:35:53.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.659+0000 7f8007fff700 1 -- 172.21.15.52:0/3013132467 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f8014019070 con 0x7f802006ab30 2024-06-02T23:35:53.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.660+0000 7f8007fff700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80080a2ca0 0x7f80080a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.660+0000 7f8007fff700 1 -- 172.21.15.52:0/3013132467 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f80140ce9f0 con 0x7f802006ab30 2024-06-02T23:35:53.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.660+0000 7f801effd700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80080a2ca0 0x7f80080a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.660+0000 7f802fbb3b80 1 Processor -- start 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.660+0000 7f802fbb3b80 1 -- start start 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed9359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed936340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f802fbb3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563bed887810 con 0x563bed93b640 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f802fbb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563bed82f720 con 0x563bed935f20 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f802fbb3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563bed8837d0 con 0x563bed93bf60 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801effd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39544/0 (socket says 172.21.15.52:39544) 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801effd700 1 -- 172.21.15.52:0/508000757 learned_addr learned my addr 172.21.15.52:0/508000757 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801f7fe700 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed936340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.663 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801e7fc700 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed9359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801f7fe700 1 -- 172.21.15.52:0/508000757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 msgr2=0x563bed9359e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801f7fe700 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed9359e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801f7fe700 1 -- 172.21.15.52:0/508000757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 msgr2=0x563bed93ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801f7fe700 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801f7fe700 1 -- 172.21.15.52:0/508000757 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bed825f50 con 0x563bed935f20 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.661+0000 7f801effd700 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.662+0000 7f801effd700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80080a2ca0 0x7f80080a5160 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f801000b2e0 tx=0x7f801001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.662+0000 7f801f7fe700 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed936340 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8018017d10 tx=0x7f8018009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de727a94e1d522bc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.662+0000 7f80057fa700 1 -- 172.21.15.52:0/508000757 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80180093a0 con 0x563bed935f20 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.662+0000 7f80057fa700 1 -- 172.21.15.52:0/508000757 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801800e870 con 0x563bed935f20 2024-06-02T23:35:53.664 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.662+0000 7f80057fa700 1 -- 172.21.15.52:0/508000757 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8018004250 con 0x563bed935f20 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 -- 172.21.15.52:0/508000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 msgr2=0x563bed936340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed936340 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8018017d10 tx=0x7f8018009920 comp rx=0 tx=0).stop 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 -- 172.21.15.52:0/508000757 shutdown_connections 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed936340 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/508000757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed9359e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 -- 172.21.15.52:0/508000757 >> 172.21.15.52:0/508000757 conn(0x563bed860cc0 msgr2=0x563bed930850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:53.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 -- 172.21.15.52:0/508000757 shutdown_connections 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.663+0000 7f802fbb3b80 1 -- 172.21.15.52:0/508000757 wait complete. 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.664+0000 7f802fbb3b80 1 -- start start 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.664+0000 7f802fbb3b80 1 Processor -- start 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.664+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed941d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.664+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed93f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed941d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed941d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:33920/0 (socket says 172.21.15.52:33920) 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39556/0 (socket says 172.21.15.52:39556) 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 -- 172.21.15.52:0/1911859966 learned_addr learned my addr 172.21.15.52:0/1911859966 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563bed9429d0 con 0x563bed93b640 2024-06-02T23:35:53.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563bed827bf0 con 0x563bed935f20 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563bed942b50 con 0x563bed93bf60 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801f7fe700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed93f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f802fbb3b80 10 client.0 _ll_register_callbacks cb 0x563bed821920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 -- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 msgr2=0x563bed93f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed93f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 -- 172.21.15.52:0/1911859966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 msgr2=0x563bed93f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801effd700 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bed7d68d0 con 0x563bed935f20 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801e7fc700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.665+0000 7f801f7fe700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed93f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f801effd700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed941d60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8010006010 tx=0x7f8010009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e3cc269150df8bb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f8002ff5700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:53.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f8002ff5700 1 -- 172.21.15.52:0/1911859966 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801002b040 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f8002ff5700 1 -- 172.21.15.52:0/1911859966 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8010004830 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801800e870 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f8002ff5700 1 -- 172.21.15.52:0/1911859966 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f8010004a30 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f802fbb3b80 10 client.30878 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563bed94d320 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.666+0000 7f8002ff5700 1 -- 172.21.15.52:0/1911859966 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801800e870 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f8002ff5700 1 -- 172.21.15.52:0/1911859966 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f801002e070 con 0x563bed935f20 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f802fbb3b80 10 client.30878 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f802fbb3b80 10 client.30878 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f802fbb3b80 20 client.30878 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed94f960 0x563bed951d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f802fbb3b80 4 client.30878 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563bed894d10 con 0x563bed94f960 2024-06-02T23:35:53.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f801e7fc700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed94f960 0x563bed951d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f801e7fc700 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed94f960 0x563bed951d90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1bd7910adb88ec64 server_cookie=7f355f5d153c3442 in_seq=0 out_seq=0 2024-06-02T23:35:53.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.667+0000 7f8002ff5700 10 client.30878 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f8002ff5700 1 -- 172.21.15.52:0/1911859966 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563bed894d10 con 0x563bed94f960 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f8002ff5700 10 client.30878 handle_command_reply: tid=0 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 client.30878 shutdown 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 msgr2=0x563bed941d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed941d60 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f8010006010 tx=0x7f8010009310 comp rx=0 tx=0).stop 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 shutdown_connections 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed94f960 0x563bed951d90 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93bf60 0x563bed93f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed941d60 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f8002ff5700 0 client.30878 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:35:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.668+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/1911859966 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed93f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.669+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 >> 172.21.15.52:0/1911859966 conn(0x563bed860cc0 msgr2=0x563bed9349d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.669+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 shutdown_connections 2024-06-02T23:35:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.669+0000 7f802fbb3b80 1 -- 172.21.15.52:0/1911859966 wait complete. 2024-06-02T23:35:53.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.669+0000 7f802fbb3b80 20 client.30878 trim_cache size 0 max 16384 2024-06-02T23:35:53.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.697+0000 7f802fbb3b80 1 Processor -- start 2024-06-02T23:35:53.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.697+0000 7f802fbb3b80 1 -- start start 2024-06-02T23:35:53.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed935f20 0x563bed9359e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.699 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed92a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed93bf60 0x563bed93dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f802fbb3b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80201aa8d0 con 0x563bed93b640 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f802fbb3b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80201ac410 con 0x563bed93bf60 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f802fbb3b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f80201aafa0 con 0x563bed935f20 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed92a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed92a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39570/0 (socket says 172.21.15.52:39570) 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801e7fc700 1 -- 172.21.15.52:0/669967701 learned_addr learned my addr 172.21.15.52:0/669967701 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801effd700 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed935f20 0x563bed9359e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801f7fe700 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed93bf60 0x563bed93dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801effd700 1 -- 172.21.15.52:0/669967701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed93bf60 msgr2=0x563bed93dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801effd700 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed93bf60 0x563bed93dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801effd700 1 -- 172.21.15.52:0/669967701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 msgr2=0x563bed92a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801effd700 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed92a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.700 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.698+0000 7f801effd700 1 -- 172.21.15.52:0/669967701 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bed932080 con 0x563bed935f20 2024-06-02T23:35:53.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.699+0000 7f801e7fc700 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed92a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.699+0000 7f801effd700 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed935f20 0x563bed9359e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f80100096a0 tx=0x7f80100059c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b6d68548a051c5d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:53.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.699+0000 7f8004ff9700 1 -- 172.21.15.52:0/669967701 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801002b070 con 0x563bed935f20 2024-06-02T23:35:53.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.699+0000 7f8004ff9700 1 -- 172.21.15.52:0/669967701 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801002f430 con 0x563bed935f20 2024-06-02T23:35:53.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.699+0000 7f8004ff9700 1 -- 172.21.15.52:0/669967701 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801000fb40 con 0x563bed935f20 2024-06-02T23:35:53.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 -- 172.21.15.52:0/669967701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed935f20 msgr2=0x563bed9359e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.703 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed935f20 0x563bed9359e0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f80100096a0 tx=0x7f80100059c0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 -- 172.21.15.52:0/669967701 shutdown_connections 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed93bf60 0x563bed93dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed935f20 0x563bed9359e0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/669967701 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93b640 0x563bed92a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 -- 172.21.15.52:0/669967701 >> 172.21.15.52:0/669967701 conn(0x563bed860cc0 msgr2=0x563bed93c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 -- 172.21.15.52:0/669967701 shutdown_connections 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.700+0000 7f802fbb3b80 1 -- 172.21.15.52:0/669967701 wait complete. 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 -- start start 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 Processor -- start 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed948680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 0x563bed957110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93bf60 0x563bed957740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.704 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 0x563bed957110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f801effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed948680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f801e7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 0x563bed957110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53516/0 (socket says 172.21.15.52:53516) 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f801e7fc700 1 -- 172.21.15.52:0/4286154399 learned_addr learned my addr 172.21.15.52:0/4286154399 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f80201abd40 con 0x563bed93bf60 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f80201ab670 con 0x563bed935f20 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.701+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8020115810 con 0x563bed93b640 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f802fbb3b80 10 client.0 _ll_register_callbacks cb 0x563bed821920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93bf60 0x563bed957740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.705 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 -- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 msgr2=0x563bed957110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 0x563bed957110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 -- 172.21.15.52:0/4286154399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 msgr2=0x563bed948680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed948680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bed9b9630 con 0x563bed93bf60 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801e7fc700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 0x563bed957110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801effd700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed948680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f801f7fe700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93bf60 0x563bed957740 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f801800b010 tx=0x7f8018009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809a0aec00207d6 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.702+0000 7f8002ff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f8002ff5700 1 -- 172.21.15.52:0/4286154399 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f801802f060 con 0x563bed93bf60 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f8002ff5700 1 -- 172.21.15.52:0/4286154399 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f8018017e00 con 0x563bed93bf60 2024-06-02T23:35:53.706 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f801002f430 con 0x563bed93bf60 2024-06-02T23:35:53.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f8002ff5700 1 -- 172.21.15.52:0/4286154399 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f80180093a0 con 0x563bed93bf60 2024-06-02T23:35:53.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f802fbb3b80 10 client.21657 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:53.708 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563bed9ba7b0 con 0x563bed93bf60 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.703+0000 7f8002ff5700 1 -- 172.21.15.52:0/4286154399 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f801002f430 con 0x563bed93bf60 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f8002ff5700 1 -- 172.21.15.52:0/4286154399 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f8018021660 con 0x563bed93bf60 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f802fbb3b80 10 client.21657 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f802fbb3b80 10 client.21657 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f802fbb3b80 20 client.21657 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed9bab30 0x563bed9bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f802fbb3b80 4 client.21657 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563bed88f290 con 0x563bed9bab30 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f801effd700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed9bab30 0x563bed9bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f801effd700 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed9bab30 0x563bed9bcf20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=65e34987425506c7 server_cookie=be10e9f898a0dd21 in_seq=0 out_seq=0 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.704+0000 7f8002ff5700 10 client.21657 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:35:53.709 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.705+0000 7f8002ff5700 1 -- 172.21.15.52:0/4286154399 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563bed88f290 con 0x563bed9bab30 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.705+0000 7f8002ff5700 10 client.21657 handle_command_reply: tid=0 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.705+0000 7f802fbb3b80 1 client.21657 shutdown 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93bf60 msgr2=0x563bed957740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93bf60 0x563bed957740 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f801800b010 tx=0x7f8018009a30 comp rx=0 tx=0).stop 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 shutdown_connections 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563bed9bab30 0x563bed9bcf20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563bed93bf60 0x563bed957740 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563bed935f20 0x563bed948680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 --2- 172.21.15.52:0/4286154399 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563bed93b640 0x563bed957110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f8002ff5700 0 client.21657 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 >> 172.21.15.52:0/4286154399 conn(0x563bed860cc0 msgr2=0x563bed93cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 shutdown_connections 2024-06-02T23:35:53.710 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 1 -- 172.21.15.52:0/4286154399 wait complete. 2024-06-02T23:35:53.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.706+0000 7f802fbb3b80 20 client.21657 trim_cache size 0 max 16384 2024-06-02T23:35:53.711 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:35:53.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.707+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80080a2ca0 msgr2=0x7f80080a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.711 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.707+0000 7f8025330700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80080a2ca0 0x7f80080a5160 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f801000b2e0 tx=0x7f801001a040 comp rx=0 tx=0).stop 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.707+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 msgr2=0x7f80201a0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.707+0000 7f8025330700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 0x7f80201a0fa0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f801400d950 tx=0x7f801400dc60 comp rx=0 tx=0).stop 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.707+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 shutdown_connections 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.707+0000 7f8025330700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f80080a2ca0 0x7f80080a5160 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.708+0000 7f8025330700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8020108500 0x7f802019ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.708+0000 7f8025330700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f802006a170 0x7f80201a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.708+0000 7f8025330700 1 --2- 172.21.15.52:0/3013132467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f802006ab30 0x7f80201a0fa0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.708+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 >> 172.21.15.52:0/3013132467 conn(0x7f8020078340 msgr2=0x7f8020100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.708+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 shutdown_connections 2024-06-02T23:35:53.712 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:53.708+0000 7f8025330700 1 -- 172.21.15.52:0/3013132467 wait complete. 2024-06-02T23:35:53.724 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:53 smithi052 ceph-mon[32340]: pgmap v1151: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 313 op/s 2024-06-02T23:35:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:53 smithi123 ceph-mon[35977]: pgmap v1151: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 313 op/s 2024-06-02T23:35:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:53 smithi174 ceph-mon[35994]: pgmap v1151: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.2 MiB/s wr, 313 op/s 2024-06-02T23:35:54.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 79290 6.62 MB/sec execute 542 sec latency 69.346 ms 2024-06-02T23:35:54.825 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:35:55.183 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:55.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 79452 6.62 MB/sec execute 543 sec latency 185.575 ms 2024-06-02T23:35:55.579 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:55 smithi052 ceph-mon[32340]: pgmap v1152: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-06-02T23:35:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:55 smithi123 ceph-mon[35977]: pgmap v1152: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-06-02T23:35:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:55 smithi174 ceph-mon[35994]: pgmap v1152: 97 pgs: 97 active+clean; 587 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 3.5 MiB/s wr, 261 op/s 2024-06-02T23:35:56.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 79633 6.63 MB/sec execute 544 sec latency 68.339 ms 2024-06-02T23:35:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 -- 172.21.15.52:0/2503551623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d8108ac0 msgr2=0x7ff4d8108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/2503551623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d8108ac0 0x7ff4d8108f40 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc009990 tx=0x7ff4cc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 -- 172.21.15.52:0/2503551623 shutdown_connections 2024-06-02T23:35:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/2503551623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d8109480 0x7ff4d8115990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/2503551623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4d810eac0 0x7ff4d810eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.149 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/2503551623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d8108ac0 0x7ff4d8108f40 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 -- 172.21.15.52:0/2503551623 >> 172.21.15.52:0/2503551623 conn(0x7ff4d8075c60 msgr2=0x7ff4d81072b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 -- 172.21.15.52:0/2503551623 shutdown_connections 2024-06-02T23:35:57.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.148+0000 7ff4ddec3700 1 -- 172.21.15.52:0/2503551623 wait complete. 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.149+0000 7ff4ddec3700 1 Processor -- start 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.149+0000 7ff4ddec3700 1 -- start start 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4ddec3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4d8108ac0 0x7ff4d819f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4ddec3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d8109480 0x7ff4d819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4ddec3700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 0x7ff4d81a3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4ddec3700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8118d60 con 0x7ff4d810eac0 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4ddec3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8118be0 con 0x7ff4d8108ac0 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4ddec3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d8118ee0 con 0x7ff4d8109480 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 0x7ff4d81a3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 0x7ff4d81a3a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47384/0 (socket says 172.21.15.52:47384) 2024-06-02T23:35:57.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4d7fff700 1 -- 172.21.15.52:0/1420666731 learned_addr learned my addr 172.21.15.52:0/1420666731 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d8109480 0x7ff4d819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.150+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4d8108ac0 0x7ff4d819f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d7fff700 1 -- 172.21.15.52:0/1420666731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d8109480 msgr2=0x7ff4d819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d8109480 0x7ff4d819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d7fff700 1 -- 172.21.15.52:0/1420666731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4d8108ac0 msgr2=0x7ff4d819f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4d8108ac0 0x7ff4d819f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d7fff700 1 -- 172.21.15.52:0/1420666731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d81a4190 con 0x7ff4d810eac0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 0x7ff4d81a3a50 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ef30 tx=0x7ff4d000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d4ff9700 1 -- 172.21.15.52:0/1420666731 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00092e0 con 0x7ff4d810eac0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d81a4420 con 0x7ff4d810eac0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d4ff9700 1 -- 172.21.15.52:0/1420666731 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4d0004bb0 con 0x7ff4d810eac0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4d4ff9700 1 -- 172.21.15.52:0/1420666731 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0004020 con 0x7ff4d810eac0 2024-06-02T23:35:57.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.151+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d81abd00 con 0x7ff4d810eac0 2024-06-02T23:35:57.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.155+0000 7ff4d4ff9700 1 -- 172.21.15.52:0/1420666731 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff4d001b030 con 0x7ff4d810eac0 2024-06-02T23:35:57.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.155+0000 7ff4d4ff9700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff4c00a2cf0 0x7ff4c00a51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.155+0000 7ff4d4ff9700 1 -- 172.21.15.52:0/1420666731 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff4d00ceaf0 con 0x7ff4d810eac0 2024-06-02T23:35:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.155+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff4c00a2cf0 0x7ff4c00a51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.156+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff4c00a2cf0 0x7ff4c00a51b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009be0 tx=0x7ff4c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 Processor -- start 2024-06-02T23:35:57.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 -- start start 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b9e880 0x562132b98260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b987a0 0x562132b98bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562132a8af60 con 0x562132b9df60 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562132ae9850 con 0x562132b987a0 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562132a51460 con 0x562132b9e880 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47400/0 (socket says 172.21.15.52:47400) 2024-06-02T23:35:57.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 -- 172.21.15.52:0/1055182078 learned_addr learned my addr 172.21.15.52:0/1055182078 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b987a0 0x562132b98bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b9e880 0x562132b98260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 -- 172.21.15.52:0/1055182078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b9e880 msgr2=0x562132b98260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b9e880 0x562132b98260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 -- 172.21.15.52:0/1055182078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b987a0 msgr2=0x562132b98bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.157+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b987a0 0x562132b98bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.158+0000 7ff4d77fe700 1 -- 172.21.15.52:0/1055182078 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562132a388d0 con 0x562132b9df60 2024-06-02T23:35:57.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.158+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9e340 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8011d70 tx=0x7ff4c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f6340debd04854 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.158+0000 7ff4bdffb700 1 -- 172.21.15.52:0/1055182078 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8011da0 con 0x562132b9df60 2024-06-02T23:35:57.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.158+0000 7ff4bdffb700 1 -- 172.21.15.52:0/1055182078 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4c8004020 con 0x562132b9df60 2024-06-02T23:35:57.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.158+0000 7ff4bdffb700 1 -- 172.21.15.52:0/1055182078 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c8003d20 con 0x562132b9df60 2024-06-02T23:35:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1055182078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 msgr2=0x562132b9e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9e340 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8011d70 tx=0x7ff4c8014000 comp rx=0 tx=0).stop 2024-06-02T23:35:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1055182078 shutdown_connections 2024-06-02T23:35:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b987a0 0x562132b98bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9e340 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1055182078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b9e880 0x562132b98260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1055182078 >> 172.21.15.52:0/1055182078 conn(0x562132ac5080 msgr2=0x562132b93220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1055182078 shutdown_connections 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.159+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1055182078 wait complete. 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.160+0000 7ff4e8746b80 1 -- start start 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.160+0000 7ff4e8746b80 1 Processor -- start 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.160+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132ba18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132ba1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x562132a89fc0 con 0x562132b9e880 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562132ae5810 con 0x562132b9df60 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562132ba52b0 con 0x562132b987a0 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4e8746b80 10 client.0 _ll_register_callbacks cb 0x562132a7f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:49090/0 (socket says 172.21.15.52:49090) 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d77fe700 1 -- 172.21.15.52:0/3672102851 learned_addr learned my addr 172.21.15.52:0/3672102851 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132ba18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132ba1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d7fff700 1 -- 172.21.15.52:0/3672102851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 msgr2=0x562132ba45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d7fff700 1 -- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 msgr2=0x562132ba18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132ba18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d7fff700 1 -- 172.21.15.52:0/3672102851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562132aaf750 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.161+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132ba1de0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ef00 tx=0x7ff4d009b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2e1151862df6dda server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/3672102851 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00b4060 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/3672102851 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff4d00b4200 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4c8004020 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/3672102851 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff4d009bab0 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4e8746b80 10 client.21669 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562132bafa70 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/3672102851 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4c8004020 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/3672102851 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff4d00bb220 con 0x562132b9e880 2024-06-02T23:35:57.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4e8746b80 10 client.21669 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.162+0000 7ff4e8746b80 10 client.21669 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4e8746b80 20 client.21669 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132bb20b0 0x562132bb44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4e8746b80 4 client.21669 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562132af6d50 con 0x562132bb20b0 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132bb20b0 0x562132bb44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4b6ffd700 10 client.21669 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.163+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132bb20b0 0x562132bb44a0 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a322177f99ed3ae server_cookie=84a392030b021177 in_seq=0 out_seq=0 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.164+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/3672102851 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x562132af6d50 con 0x562132bb20b0 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.164+0000 7ff4b6ffd700 10 client.21669 handle_command_reply: tid=0 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.164+0000 7ff4e8746b80 1 client.21669 shutdown 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.164+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 msgr2=0x562132ba1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.164+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132ba1de0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ef00 tx=0x7ff4d009b8f0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 shutdown_connections 2024-06-02T23:35:57.171 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132bb20b0 0x562132bb44a0 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132ba1de0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3672102851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132ba18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 >> 172.21.15.52:0/3672102851 conn(0x562132ac5080 msgr2=0x562132b92700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4b6ffd700 0 client.21669 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 shutdown_connections 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3672102851 wait complete. 2024-06-02T23:35:57.172 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.165+0000 7ff4e8746b80 20 client.21669 trim_cache size 0 max 16384 2024-06-02T23:35:57.190 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.190+0000 7ff4e8746b80 1 Processor -- start 2024-06-02T23:35:57.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.190+0000 7ff4e8746b80 1 -- start start 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.190+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.190+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.190+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9e880 0x562132b92e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.190+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b3150 con 0x562132b9df60 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b4c90 con 0x562132b9e880 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b3820 con 0x562132b987a0 2024-06-02T23:35:57.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d6ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47420/0 (socket says 172.21.15.52:47420) 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d6ffd700 1 -- 172.21.15.52:0/3904796393 learned_addr learned my addr 172.21.15.52:0/3904796393 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9e880 0x562132b92e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d7fff700 1 -- 172.21.15.52:0/3904796393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 msgr2=0x562132ba11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba11a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d7fff700 1 -- 172.21.15.52:0/3904796393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 msgr2=0x562132b9f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d7fff700 1 -- 172.21.15.52:0/3904796393 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562132bb4a80 con 0x562132b9e880 2024-06-02T23:35:57.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.191+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:57.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.192+0000 7ff4d7fff700 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9e880 0x562132b92e80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0099660 tx=0x7ff4d0009480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27b649bdc26df54f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.192+0000 7ff4bd7fa700 1 -- 172.21.15.52:0/3904796393 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00b4260 con 0x562132b9e880 2024-06-02T23:35:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.192+0000 7ff4bd7fa700 1 -- 172.21.15.52:0/3904796393 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4d00a3f90 con 0x562132b9e880 2024-06-02T23:35:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.192+0000 7ff4bd7fa700 1 -- 172.21.15.52:0/3904796393 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0097100 con 0x562132b9e880 2024-06-02T23:35:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.193+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3904796393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9e880 msgr2=0x562132b92e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.193+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9e880 0x562132b92e80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0099660 tx=0x7ff4d0009480 comp rx=0 tx=0).stop 2024-06-02T23:35:57.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.193+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3904796393 shutdown_connections 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.193+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9e880 0x562132b92e80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.193+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132ba11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.194+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/3904796393 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9df60 0x562132b9f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.194+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3904796393 >> 172.21.15.52:0/3904796393 conn(0x562132ac5080 msgr2=0x562132b9ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.194+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3904796393 shutdown_connections 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.194+0000 7ff4e8746b80 1 -- 172.21.15.52:0/3904796393 wait complete. 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.194+0000 7ff4e8746b80 1 -- start start 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.194+0000 7ff4e8746b80 1 Processor -- start 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132bba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132baa5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132baabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b45c0 con 0x562132b9e880 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b3ef0 con 0x562132b9df60 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d81b10f0 con 0x562132b987a0 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4e8746b80 10 client.0 _ll_register_callbacks cb 0x562132a7f5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132baa5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d7fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132baabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132baa5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37780/0 (socket says 172.21.15.52:37780) 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 -- 172.21.15.52:0/1323595760 learned_addr learned my addr 172.21.15.52:0/1323595760 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132bba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 -- 172.21.15.52:0/1323595760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 msgr2=0x562132bba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132bba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 -- 172.21.15.52:0/1323595760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 msgr2=0x562132baabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.195+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132baabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4d6ffd700 1 -- 172.21.15.52:0/1323595760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562132bb8cc0 con 0x562132b9df60 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132bba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4d6ffd700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132baa5a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000c00 tx=0x7ff4cc0057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cc32d2a1089e812 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/1323595760 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4cc029020 con 0x562132b9df60 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/1323595760 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff4cc00bcc0 con 0x562132b9df60 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4d00a3f90 con 0x562132b9df60 2024-06-02T23:35:57.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/1323595760 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff4cc003680 con 0x562132b9df60 2024-06-02T23:35:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4e8746b80 10 client.30902 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.196+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x562132c1bf70 con 0x562132b9df60 2024-06-02T23:35:57.201 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/1323595760 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff4d00a3f90 con 0x562132b9df60 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/1323595760 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff4cc00f8e0 con 0x562132b9df60 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4e8746b80 10 client.30902 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4e8746b80 10 client.30902 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4e8746b80 20 client.30902 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132c1c700 0x562132c1eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4e8746b80 4 client.30902 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562132af12d0 con 0x562132c1c700 2024-06-02T23:35:57.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132c1c700 0x562132c1eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.197+0000 7ff4b6ffd700 10 client.30902 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4d77fe700 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132c1c700 0x562132c1eb20 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa947be86a95298 server_cookie=bc5bb721e13c243a in_seq=0 out_seq=0 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4b6ffd700 1 -- 172.21.15.52:0/1323595760 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562132af12d0 con 0x562132c1c700 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4b6ffd700 10 client.30902 handle_command_reply: tid=0 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 client.30902 shutdown 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 msgr2=0x562132baa5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132baa5a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff4cc000c00 tx=0x7ff4cc0057b0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 shutdown_connections 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x562132c1c700 0x562132c1eb20 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x562132b9e880 0x562132baabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562132b987a0 0x562132bba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 --2- 172.21.15.52:0/1323595760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562132b9df60 0x562132baa5a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 >> 172.21.15.52:0/1323595760 conn(0x562132ac5080 msgr2=0x562132ba30d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4b6ffd700 0 client.30902 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 shutdown_connections 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 1 -- 172.21.15.52:0/1323595760 wait complete. 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.198+0000 7ff4e8746b80 20 client.30902 trim_cache size 0 max 16384 2024-06-02T23:35:57.204 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff4c00a2cf0 msgr2=0x7ff4c00a51b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff4c00a2cf0 0x7ff4c00a51b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff4c8009be0 tx=0x7ff4c8009260 comp rx=0 tx=0).stop 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 msgr2=0x7ff4d81a3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 0x7ff4d81a3a50 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7ff4d000ef30 tx=0x7ff4d000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 shutdown_connections 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff4c00a2cf0 0x7ff4c00a51b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff4d810eac0 0x7ff4d81a3a50 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff4d8108ac0 0x7ff4d819f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 --2- 172.21.15.52:0/1420666731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d8109480 0x7ff4d819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.199+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 >> 172.21.15.52:0/1420666731 conn(0x7ff4d8075c60 msgr2=0x7ff4d8076220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.200+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 shutdown_connections 2024-06-02T23:35:57.206 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:35:57.200+0000 7ff4ddec3700 1 -- 172.21.15.52:0/1420666731 wait complete. 2024-06-02T23:35:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 -- 172.21.15.52:0/1986394692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220100900 msgr2=0x7fc220100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 --2- 172.21.15.52:0/1986394692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220100900 0x7fc220100d20 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc210009860 tx=0x7fc21002a340 comp rx=0 tx=0).stop 2024-06-02T23:35:57.479 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 -- 172.21.15.52:0/1986394692 shutdown_connections 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 --2- 172.21.15.52:0/1986394692 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2201024b0 0x7fc220108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 --2- 172.21.15.52:0/1986394692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220101af0 0x7fc220101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 --2- 172.21.15.52:0/1986394692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220100900 0x7fc220100d20 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 -- 172.21.15.52:0/1986394692 >> 172.21.15.52:0/1986394692 conn(0x7fc2200fbe70 msgr2=0x7fc2200fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.475+0000 7fc226453640 1 -- 172.21.15.52:0/1986394692 shutdown_connections 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.476+0000 7fc226453640 1 -- 172.21.15.52:0/1986394692 wait complete. 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.476+0000 7fc226453640 1 Processor -- start 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.476+0000 7fc226453640 1 -- start start 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.476+0000 7fc226453640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 0x7fc22019e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc226453640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220101af0 0x7fc22019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc226453640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2201024b0 0x7fc2201a5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.480 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc226453640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010b150 con 0x7fc2201024b0 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc226453640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010afd0 con 0x7fc220100900 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc226453640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc22010b2d0 con 0x7fc220101af0 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 0x7fc22019e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 0x7fc22019e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37796/0 (socket says 172.21.15.52:37796) 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc21ffff640 1 -- 172.21.15.52:0/3731609529 learned_addr learned my addr 172.21.15.52:0/3731609529 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2249f0640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2201024b0 0x7fc2201a5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220101af0 0x7fc22019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 -- 172.21.15.52:0/3731609529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 msgr2=0x7fc22019e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 0x7fc22019e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 -- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2201024b0 msgr2=0x7fc2201a5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2201024b0 0x7fc2201a5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 -- 172.21.15.52:0/3731609529 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2201a5dd0 con 0x7fc220101af0 2024-06-02T23:35:57.481 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc21ffff640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 0x7fc22019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.477+0000 7fc2177fe640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220101af0 0x7fc22019e6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e9f0 tx=0x7fc20800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.478+0000 7fc21dffb640 1 -- 172.21.15.52:0/3731609529 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800ccc0 con 0x7fc220101af0 2024-06-02T23:35:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.478+0000 7fc21dffb640 1 -- 172.21.15.52:0/3731609529 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc208005020 con 0x7fc220101af0 2024-06-02T23:35:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.478+0000 7fc21dffb640 1 -- 172.21.15.52:0/3731609529 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208004a00 con 0x7fc220101af0 2024-06-02T23:35:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.478+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2201a60c0 con 0x7fc220101af0 2024-06-02T23:35:57.482 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.478+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2201a6610 con 0x7fc220101af0 2024-06-02T23:35:57.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 79812 6.64 MB/sec execute 545 sec latency 66.633 ms 2024-06-02T23:35:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.480+0000 7fc21dffb640 1 -- 172.21.15.52:0/3731609529 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc208004ba0 con 0x7fc220101af0 2024-06-02T23:35:57.484 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.481+0000 7fc229234740 1 Processor -- start 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 -- start start 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd49430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd4e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd4eb70 0x5602bdd51000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5602bdcac4f0 con 0x5602bdd49a00 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5602bdc4a870 con 0x5602bdd49010 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc229234740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602bdca84b0 con 0x5602bdd4eb70 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd49430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd4e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd4e500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47458/0 (socket says 172.21.15.52:47458) 2024-06-02T23:35:57.485 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc21ffff640 1 -- 172.21.15.52:0/1747863036 learned_addr learned my addr 172.21.15.52:0/1747863036 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc21dffb640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc20c0a2cb0 0x7fc20c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc2249f0640 1 -- 172.21.15.52:0/1747863036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd4eb70 msgr2=0x5602bdd51000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc2249f0640 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd4eb70 0x5602bdd51000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc2249f0640 1 -- 172.21.15.52:0/1747863036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 msgr2=0x5602bdd4e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc2249f0640 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd4e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.482+0000 7fc2249f0640 1 -- 172.21.15.52:0/1747863036 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602bdc669b0 con 0x5602bdd49010 2024-06-02T23:35:57.486 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.483+0000 7fc21dffb640 1 -- 172.21.15.52:0/3731609529 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc2080cef20 con 0x7fc220101af0 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.483+0000 7fc2249f0640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc20c0a2cb0 0x7fc20c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.483+0000 7fc2249f0640 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd49430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc218005c30 tx=0x7fc2180095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4a6a8dfe099e52f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.483+0000 7fc2167fc640 1 -- 172.21.15.52:0/1747863036 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21800da70 con 0x5602bdd49010 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.483+0000 7fc2167fc640 1 -- 172.21.15.52:0/1747863036 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc218017c70 con 0x5602bdd49010 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.483+0000 7fc2167fc640 1 -- 172.21.15.52:0/1747863036 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2180235d0 con 0x5602bdd49010 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 -- 172.21.15.52:0/1747863036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 msgr2=0x5602bdd49430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc2249f0640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc20c0a2cb0 0x7fc20c0a5170 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc218021000 tx=0x7fc218024000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd49430 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fc218005c30 tx=0x7fc2180095c0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 -- 172.21.15.52:0/1747863036 shutdown_connections 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd4eb70 0x5602bdd51000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd4e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 --2- 172.21.15.52:0/1747863036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd49430 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.487 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.484+0000 7fc229234740 1 -- 172.21.15.52:0/1747863036 >> 172.21.15.52:0/1747863036 conn(0x5602bdd44680 msgr2=0x5602bdd44a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.485+0000 7fc229234740 1 -- 172.21.15.52:0/1747863036 shutdown_connections 2024-06-02T23:35:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.485+0000 7fc229234740 1 -- 172.21.15.52:0/1747863036 wait complete. 2024-06-02T23:35:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.485+0000 7fc229234740 1 -- start start 2024-06-02T23:35:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.485+0000 7fc229234740 1 Processor -- start 2024-06-02T23:35:57.492 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd539b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 0x5602bdd5add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5602bdd51b20 con 0x5602bdd49a00 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5602bdc35ae0 con 0x5602bdd589e0 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5602bdd51ca0 con 0x5602bdd49010 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc229234740 10 client.0 _ll_register_callbacks cb 0x5602bdc6e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 0x5602bdd5add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 0x5602bdd5add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37824/0 (socket says 172.21.15.52:37824) 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 -- 172.21.15.52:0/4109243771 learned_addr learned my addr 172.21.15.52:0/4109243771 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc21ffff640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2177fe640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 -- 172.21.15.52:0/4109243771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 msgr2=0x5602bdd53470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.493 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd53470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 -- 172.21.15.52:0/4109243771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 msgr2=0x5602bdd539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 -- 172.21.15.52:0/4109243771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602bd9c0210 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2177fe640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc2249f0640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 0x5602bdd5add0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc218024aa0 tx=0x7fc218009d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2993a9b61487de8e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.486+0000 7fc1ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/4109243771 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218033070 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/4109243771 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc218023390 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc218017c70 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/4109243771 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc21803f050 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 10 client.30914 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5602bda8c620 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/4109243771 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc218017c70 con 0x5602bdd589e0 2024-06-02T23:35:57.494 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/4109243771 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc21802e040 con 0x5602bdd589e0 2024-06-02T23:35:57.495 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 10 client.30914 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 10 client.30914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 20 client.30914 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bdd5fa50 0x5602bdd61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.487+0000 7fc229234740 4 client.30914 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.488+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5602bdcbff10 con 0x5602bdd5fa50 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.488+0000 7fc21ffff640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bdd5fa50 0x5602bdd61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.488+0000 7fc1ff7fe640 10 client.30914 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.488+0000 7fc21ffff640 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bdd5fa50 0x5602bdd61ed0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa43f3e61d7746fa server_cookie=c348da07baa43578 in_seq=0 out_seq=0 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/4109243771 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5602bdcbff10 con 0x5602bdd5fa50 2024-06-02T23:35:57.496 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc1ff7fe640 10 client.30914 handle_command_reply: tid=0 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 client.30914 shutdown 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 msgr2=0x5602bdd5add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 0x5602bdd5add0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fc218024aa0 tx=0x7fc218009d60 comp rx=0 tx=0).stop 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 shutdown_connections 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bdd5fa50 0x5602bdd61ed0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd589e0 0x5602bdd5add0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd49a00 0x5602bdd539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.489+0000 7fc229234740 1 --2- 172.21.15.52:0/4109243771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd53470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.490+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 >> 172.21.15.52:0/4109243771 conn(0x5602bdd44680 msgr2=0x5602bdd437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.490+0000 7fc1ff7fe640 0 client.30914 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.490+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 shutdown_connections 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.490+0000 7fc229234740 1 -- 172.21.15.52:0/4109243771 wait complete. 2024-06-02T23:35:57.497 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.490+0000 7fc229234740 20 client.30914 trim_cache size 0 max 16384 2024-06-02T23:35:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.513+0000 7fc229234740 1 Processor -- start 2024-06-02T23:35:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.513+0000 7fc229234740 1 -- start start 2024-06-02T23:35:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc229234740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd4e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.514 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc229234740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49a00 0x5602bdd52040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc229234740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd53470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc229234740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201af2d0 con 0x5602bdd589e0 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc229234740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201b0e10 con 0x5602bdd49010 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc229234740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201af9a0 con 0x5602bdd49a00 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd53470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2177fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49a00 0x5602bdd52040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd53470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47478/0 (socket says 172.21.15.52:47478) 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc21ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd4e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.515 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2249f0640 1 -- 172.21.15.52:0/2117233823 learned_addr learned my addr 172.21.15.52:0/2117233823 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.519 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2177fe640 1 -- 172.21.15.52:0/2117233823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 msgr2=0x5602bdd4e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2177fe640 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd4e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2177fe640 1 -- 172.21.15.52:0/2117233823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 msgr2=0x5602bdd53470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2177fe640 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd53470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc2177fe640 1 -- 172.21.15.52:0/2117233823 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602bdd4e120 con 0x5602bdd49a00 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.514+0000 7fc21ffff640 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd4e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.515+0000 7fc2177fe640 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49a00 0x5602bdd52040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5602bdd4a290 tx=0x7fc2080054d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0d0f75f2b0074bb server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.515+0000 7fc215ffb640 1 -- 172.21.15.52:0/2117233823 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20801c040 con 0x5602bdd49a00 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.515+0000 7fc215ffb640 1 -- 172.21.15.52:0/2117233823 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc20809b7b0 con 0x5602bdd49a00 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.515+0000 7fc215ffb640 1 -- 172.21.15.52:0/2117233823 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc208086070 con 0x5602bdd49a00 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 -- 172.21.15.52:0/2117233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49a00 msgr2=0x5602bdd52040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49a00 0x5602bdd52040 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x5602bdd4a290 tx=0x7fc2080054d0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 -- 172.21.15.52:0/2117233823 shutdown_connections 2024-06-02T23:35:57.520 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49a00 0x5602bdd52040 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd53470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 --2- 172.21.15.52:0/2117233823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49010 0x5602bdd4e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 -- 172.21.15.52:0/2117233823 >> 172.21.15.52:0/2117233823 conn(0x5602bdd44680 msgr2=0x5602bdd44a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 -- 172.21.15.52:0/2117233823 shutdown_connections 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 -- 172.21.15.52:0/2117233823 wait complete. 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 -- start start 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.516+0000 7fc229234740 1 Processor -- start 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd73960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49a00 0x5602bdd70a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd70fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201b0740 con 0x5602bdd589e0 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201b0070 con 0x5602bdd49a00 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc2201ad2b0 con 0x5602bdd49010 2024-06-02T23:35:57.521 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd70fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd70fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47482/0 (socket says 172.21.15.52:47482) 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 -- 172.21.15.52:0/3519107746 learned_addr learned my addr 172.21.15.52:0/3519107746 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2177fe640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49a00 0x5602bdd70a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc229234740 10 client.0 _ll_register_callbacks cb 0x7fc218009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc21ffff640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd73960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 -- 172.21.15.52:0/3519107746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 msgr2=0x5602bdd73960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd73960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 -- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49a00 msgr2=0x5602bdd70a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49a00 0x5602bdd70a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2249f0640 1 -- 172.21.15.52:0/3519107746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602bdd73fa0 con 0x5602bdd589e0 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc21ffff640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd73960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.517+0000 7fc2177fe640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49a00 0x5602bdd70a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:35:57.522 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc2249f0640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd70fa0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fc218004eb0 tx=0x7fc21800d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc13796cf4e0e73 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc1ff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/3519107746 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc218033040 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc20809b7b0 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/3519107746 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc218024ca0 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/3519107746 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fc21800da70 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc229234740 10 client.21690 fetch_fsmap learned FSMap version 43 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5602bddc94f0 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.518+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/3519107746 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc20809b7b0 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/3519107746 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fc21802e050 con 0x5602bdd589e0 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc229234740 10 client.21690 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc229234740 10 client.21690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:35:57.523 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc229234740 20 client.21690 populate_metadata read hostname 'smithi052' 2024-06-02T23:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc229234740 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bddc9e90 0x5602bddcc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc229234740 4 client.21690 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5602bdcb9c20 con 0x5602bddc9e90 2024-06-02T23:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc21ffff640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bddc9e90 0x5602bddcc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc21ffff640 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bddc9e90 0x5602bddcc2b0 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74964c0d9c5ea43c server_cookie=1dbf97fea429cce1 in_seq=0 out_seq=0 2024-06-02T23:35:57.524 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.519+0000 7fc1ff7fe640 10 client.21690 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc1ff7fe640 1 -- 172.21.15.52:0/3519107746 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5602bdcb9c20 con 0x5602bddc9e90 2024-06-02T23:35:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc1ff7fe640 10 client.21690 handle_command_reply: tid=0 2024-06-02T23:35:57.587 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 client.21690 shutdown 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 msgr2=0x5602bdd70fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd70fa0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fc218004eb0 tx=0x7fc21800d270 comp rx=0 tx=0).stop 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 shutdown_connections 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5602bddc9e90 0x5602bddcc2b0 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5602bdd589e0 0x5602bdd70fa0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc1ff7fe640 0 client.21690 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5602bdd49a00 0x5602bdd70a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 --2- 172.21.15.52:0/3519107746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5602bdd49010 0x5602bdd73960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 >> 172.21.15.52:0/3519107746 conn(0x5602bdd44680 msgr2=0x5602bdd4ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 shutdown_connections 2024-06-02T23:35:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 1 -- 172.21.15.52:0/3519107746 wait complete. 2024-06-02T23:35:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.586+0000 7fc229234740 20 client.21690 trim_cache size 0 max 16384 2024-06-02T23:35:57.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:35:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:35:57.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:35:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc20c0a2cb0 msgr2=0x7fc20c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc20c0a2cb0 0x7fc20c0a5170 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc218021000 tx=0x7fc218024000 comp rx=0 tx=0).stop 2024-06-02T23:35:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220101af0 msgr2=0x7fc22019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:35:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220101af0 0x7fc22019e6f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc20800e9f0 tx=0x7fc20800eeb0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 shutdown_connections 2024-06-02T23:35:57.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc20c0a2cb0 0x7fc20c0a5170 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc220101af0 0x7fc22019e6f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc2201024b0 0x7fc2201a5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 --2- 172.21.15.52:0/3731609529 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc220100900 0x7fc22019e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:35:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 >> 172.21.15.52:0/3731609529 conn(0x7fc2200fbe70 msgr2=0x7fc22010de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:35:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 shutdown_connections 2024-06-02T23:35:57.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:35:57.587+0000 7fc226453640 1 -- 172.21.15.52:0/3731609529 wait complete. 2024-06-02T23:35:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:57 smithi123 ceph-mon[35977]: pgmap v1153: 97 pgs: 97 active+clean; 595 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 324 op/s 2024-06-02T23:35:57.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:57 smithi174 ceph-mon[35994]: pgmap v1153: 97 pgs: 97 active+clean; 595 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 324 op/s 2024-06-02T23:35:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:57 smithi052 ceph-mon[32340]: pgmap v1153: 97 pgs: 97 active+clean; 595 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 324 op/s 2024-06-02T23:35:58.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80040 6.66 MB/sec execute 546 sec latency 64.712 ms 2024-06-02T23:35:58.986 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:35:58.987 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:35:59.311 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:35:59.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80198 6.67 MB/sec execute 547 sec latency 187.370 ms 2024-06-02T23:35:59.695 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:35:59 smithi052 ceph-mon[32340]: pgmap v1154: 97 pgs: 97 active+clean; 597 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-06-02T23:35:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:35:59 smithi123 ceph-mon[35977]: pgmap v1154: 97 pgs: 97 active+clean; 597 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-06-02T23:35:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:35:59 smithi174 ceph-mon[35994]: pgmap v1154: 97 pgs: 97 active+clean; 597 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 907 KiB/s rd, 3.5 MiB/s wr, 231 op/s 2024-06-02T23:36:00.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80338 6.67 MB/sec execute 548 sec latency 80.565 ms 2024-06-02T23:36:01.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80478 6.67 MB/sec execute 549 sec latency 82.714 ms 2024-06-02T23:36:01.506 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:01 smithi052 ceph-mon[32340]: pgmap v1155: 97 pgs: 97 active+clean; 600 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.561+0000 7f682a029700 1 -- 172.21.15.52:0/3816018326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682406b350 msgr2=0x7f682410e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.561+0000 7f682a029700 1 --2- 172.21.15.52:0/3816018326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682406b350 0x7f682410e9f0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f6818009990 tx=0x7f6818009ca0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 -- 172.21.15.52:0/3816018326 shutdown_connections 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 --2- 172.21.15.52:0/3816018326 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682410ef30 0x7f6824111320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 --2- 172.21.15.52:0/3816018326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f682406aa30 0x7f682406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 --2- 172.21.15.52:0/3816018326 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682406b350 0x7f682410e9f0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 -- 172.21.15.52:0/3816018326 >> 172.21.15.52:0/3816018326 conn(0x7f6824078300 msgr2=0x7f6824078710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 -- 172.21.15.52:0/3816018326 shutdown_connections 2024-06-02T23:36:01.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.562+0000 7f682a029700 1 -- 172.21.15.52:0/3816018326 wait complete. 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 Processor -- start 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 -- start start 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f682406aa30 0x7f682419ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682406b350 0x7f682419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 0x7f682419f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f6824114840 con 0x7f682410ef30 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68241146c0 con 0x7f682406b350 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f682a029700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68241149c0 con 0x7f682406aa30 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f6823fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 0x7f682419f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.565 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f6823fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 0x7f682419f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47502/0 (socket says 172.21.15.52:47502) 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f6822ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682406b350 0x7f682419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f6823fff700 1 -- 172.21.15.52:0/3023061688 learned_addr learned my addr 172.21.15.52:0/3023061688 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.563+0000 7f68237fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f682406aa30 0x7f682419ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6822ffd700 1 -- 172.21.15.52:0/3023061688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f682406aa30 msgr2=0x7f682419ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6822ffd700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f682406aa30 0x7f682419ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6822ffd700 1 -- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 msgr2=0x7f682419f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6822ffd700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 0x7f682419f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6822ffd700 1 -- 172.21.15.52:0/3023061688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682419fd80 con 0x7f682406b350 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6823fff700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 0x7f682419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6822ffd700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682406b350 0x7f682419b2b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f68180059d0 tx=0x7f6818004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6820ff9700 1 -- 172.21.15.52:0/3023061688 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6818027070 con 0x7f682406b350 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f6820ff9700 1 -- 172.21.15.52:0/3023061688 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f6818022470 con 0x7f682406b350 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.564+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68241a0010 con 0x7f682406b350 2024-06-02T23:36:01.566 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.565+0000 7f6820ff9700 1 -- 172.21.15.52:0/3023061688 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681800b610 con 0x7f682406b350 2024-06-02T23:36:01.567 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.565+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68241a78f0 con 0x7f682406b350 2024-06-02T23:36:01.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.568+0000 7f6820ff9700 1 -- 172.21.15.52:0/3023061688 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f681802a070 con 0x7f682406b350 2024-06-02T23:36:01.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.569+0000 7f6820ff9700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f680c0a2ca0 0x7f680c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.569+0000 7f68237fe700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f680c0a2ca0 0x7f680c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f68237fe700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f680c0a2ca0 0x7f680c0a5160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6814009c60 tx=0x7f6814009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:01.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f6820ff9700 1 -- 172.21.15.52:0/3023061688 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f68180cfbb0 con 0x7f682406b350 2024-06-02T23:36:01.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f68348acb80 1 Processor -- start 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f68348acb80 1 -- start start 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f68348acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42a98aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f68348acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42a92a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.570+0000 7f68348acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42a933c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68348acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef429e4890 con 0x55ef42a98fe0 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68348acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef4298c7a0 con 0x55ef42a92fa0 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68348acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef429e0850 con 0x55ef42a986c0 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f6822ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42a92a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42a98aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f6822ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42a92a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47512/0 (socket says 172.21.15.52:47512) 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f6822ffd700 1 -- 172.21.15.52:0/396678872 learned_addr learned my addr 172.21.15.52:0/396678872 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f6823fff700 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42a933c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 -- 172.21.15.52:0/396678872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 msgr2=0x55ef42a933c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42a933c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 -- 172.21.15.52:0/396678872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 msgr2=0x55ef42a92a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42a92a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 -- 172.21.15.52:0/396678872 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef42982fd0 con 0x55ef42a986c0 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f6822ffd700 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42a92a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.571+0000 7f68237fe700 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42a98aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f681400e3b0 tx=0x7f6814014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a51bdaa07af00a72 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f6809ffb700 1 -- 172.21.15.52:0/396678872 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681400ed50 con 0x55ef42a986c0 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f6809ffb700 1 -- 172.21.15.52:0/396678872 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68140044e0 con 0x55ef42a986c0 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f6809ffb700 1 -- 172.21.15.52:0/396678872 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68140266d0 con 0x55ef42a986c0 2024-06-02T23:36:01.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 -- 172.21.15.52:0/396678872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 msgr2=0x55ef42a98aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42a98aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f681400e3b0 tx=0x7f6814014000 comp rx=0 tx=0).stop 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 -- 172.21.15.52:0/396678872 shutdown_connections 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42a933c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42a98aa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 --2- 172.21.15.52:0/396678872 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42a92a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 -- 172.21.15.52:0/396678872 >> 172.21.15.52:0/396678872 conn(0x55ef429bdd40 msgr2=0x55ef42a8d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.572+0000 7f68348acb80 1 -- 172.21.15.52:0/396678872 shutdown_connections 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 -- 172.21.15.52:0/396678872 wait complete. 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 -- start start 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 Processor -- start 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a92fa0 0x55ef42a9ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a986c0 0x55ef42a9c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 0x55ef42a9c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ef42a9fa50 con 0x55ef42a986c0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef42984c70 con 0x55ef42a98fe0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef42a9fbd0 con 0x55ef42a92fa0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68348acb80 10 client.0 _ll_register_callbacks cb 0x55ef4297e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f68237fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a92fa0 0x55ef42a9ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.573+0000 7f6823fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 0x55ef42a9c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 0x55ef42a9c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37874/0 (socket says 172.21.15.52:37874) 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 -- 172.21.15.52:0/2806863865 learned_addr learned my addr 172.21.15.52:0/2806863865 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6822ffd700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a986c0 0x55ef42a9c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 -- 172.21.15.52:0/2806863865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a92fa0 msgr2=0x55ef42a9ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a92fa0 0x55ef42a9ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 -- 172.21.15.52:0/2806863865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a986c0 msgr2=0x55ef42a9c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a986c0 0x55ef42a9c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 -- 172.21.15.52:0/2806863865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef42933950 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6822ffd700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a986c0 0x55ef42a9c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6823fff700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 0x55ef42a9c590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f681c007f70 tx=0x7f681c00e7d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73b326ef1eb74d02 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.574+0000 7f6802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f6802ffd700 1 -- 172.21.15.52:0/2806863865 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c003c00 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f6802ffd700 1 -- 172.21.15.52:0/2806863865 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f681c003da0 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68140044e0 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f6802ffd700 1 -- 172.21.15.52:0/2806863865 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f681c025050 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f68348acb80 10 client.30932 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ef42aaa3e0 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f6802ffd700 1 -- 172.21.15.52:0/2806863865 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68140044e0 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f6802ffd700 1 -- 172.21.15.52:0/2806863865 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f681c020540 con 0x55ef42a98fe0 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f68348acb80 10 client.30932 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.575+0000 7f68348acb80 10 client.30932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f68348acb80 20 client.30932 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:01.589 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f68348acb80 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42aaee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f68348acb80 4 client.30932 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef429f1d90 con 0x55ef42aaca20 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f68237fe700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42aaee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f6802ffd700 10 client.30932 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.576+0000 7f68237fe700 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42aaee40 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7f4d4d675b21363 server_cookie=16b3aafc806efb6d in_seq=0 out_seq=0 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f6802ffd700 1 -- 172.21.15.52:0/2806863865 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ef429f1d90 con 0x55ef42aaca20 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f6802ffd700 10 client.30932 handle_command_reply: tid=0 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 client.30932 shutdown 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 msgr2=0x55ef42a9c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 0x55ef42a9c590 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f681c007f70 tx=0x7f681c00e7d0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 shutdown_connections 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42aaee40 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f6802ffd700 0 client.30932 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a98fe0 0x55ef42a9c590 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a92fa0 0x55ef42a9ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 --2- 172.21.15.52:0/2806863865 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a986c0 0x55ef42a9c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 >> 172.21.15.52:0/2806863865 conn(0x55ef429bdd40 msgr2=0x55ef42a91a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.577+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 shutdown_connections 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.578+0000 7f68348acb80 1 -- 172.21.15.52:0/2806863865 wait complete. 2024-06-02T23:36:01.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.578+0000 7f68348acb80 20 client.30932 trim_cache size 0 max 16384 2024-06-02T23:36:01.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.608+0000 7f68348acb80 1 Processor -- start 2024-06-02T23:36:01.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.608+0000 7f68348acb80 1 -- start start 2024-06-02T23:36:01.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.608+0000 7f68348acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 0x55ef42a9c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.608+0000 7f68348acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a986c0 0x55ef42a9c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.609 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.608+0000 7f68348acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a98fe0 0x55ef42a993c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68348acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68241aa7e0 con 0x55ef42a92fa0 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68348acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68241ac320 con 0x55ef42a986c0 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68348acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68241aaeb0 con 0x55ef42a98fe0 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 0x55ef42a9c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 0x55ef42a9c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47530/0 (socket says 172.21.15.52:47530) 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 -- 172.21.15.52:0/2107188473 learned_addr learned my addr 172.21.15.52:0/2107188473 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:01.610 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f6822ffd700 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a986c0 0x55ef42a9c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f6823fff700 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a98fe0 0x55ef42a993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 -- 172.21.15.52:0/2107188473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a98fe0 msgr2=0x55ef42a993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a98fe0 0x55ef42a993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 -- 172.21.15.52:0/2107188473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a986c0 msgr2=0x55ef42a9c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a986c0 0x55ef42a9c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 -- 172.21.15.52:0/2107188473 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef42a9ca90 con 0x55ef42a92fa0 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f6823fff700 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a98fe0 0x55ef42a993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.609+0000 7f68237fe700 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 0x55ef42a9c010 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f6814011ab0 tx=0x7f681401f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feec426a96673bf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.610+0000 7f68097fa700 1 -- 172.21.15.52:0/2107188473 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6814005240 con 0x55ef42a92fa0 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.610+0000 7f68097fa700 1 -- 172.21.15.52:0/2107188473 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f68140053e0 con 0x55ef42a92fa0 2024-06-02T23:36:01.611 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.610+0000 7f68097fa700 1 -- 172.21.15.52:0/2107188473 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68140266d0 con 0x55ef42a92fa0 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 -- 172.21.15.52:0/2107188473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 msgr2=0x55ef42a9c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 0x55ef42a9c010 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f6814011ab0 tx=0x7f681401f040 comp rx=0 tx=0).stop 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 -- 172.21.15.52:0/2107188473 shutdown_connections 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a98fe0 0x55ef42a993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a92fa0 0x55ef42a9c010 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 --2- 172.21.15.52:0/2107188473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a986c0 0x55ef42a9c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 -- 172.21.15.52:0/2107188473 >> 172.21.15.52:0/2107188473 conn(0x55ef429bdd40 msgr2=0x55ef42a9a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 -- 172.21.15.52:0/2107188473 shutdown_connections 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 -- 172.21.15.52:0/2107188473 wait complete. 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 -- start start 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.611+0000 7f68348acb80 1 Processor -- start 2024-06-02T23:36:01.613 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42aa4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42aa5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42aa1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f68241abc50 con 0x55ef42a98fe0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68241ab580 con 0x55ef42a92fa0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6824107460 con 0x55ef42a986c0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68348acb80 10 client.0 _ll_register_callbacks cb 0x55ef4297e9a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68237fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42aa4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6823fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42aa1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6822ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42aa5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68237fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42aa4ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:37892/0 (socket says 172.21.15.52:37892) 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6822ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42aa5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:49248/0 (socket says 172.21.15.52:49248) 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f68237fe700 1 -- 172.21.15.52:0/3244357854 learned_addr learned my addr 172.21.15.52:0/3244357854 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:01.614 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6823fff700 1 -- 172.21.15.52:0/3244357854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 msgr2=0x55ef42aa5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6823fff700 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42aa5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6823fff700 1 -- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 msgr2=0x55ef42aa4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:01.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6823fff700 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42aa4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6823fff700 1 -- 172.21.15.52:0/3244357854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef42aa23d0 con 0x55ef42a98fe0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.612+0000 7f6822ffd700 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42aa5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.613+0000 7f6823fff700 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42aa1e90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f681c0209b0 tx=0x7f681c0206f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb206ccaef45227 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.613+0000 7f6802ffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.613+0000 7f6802ffd700 1 -- 172.21.15.52:0/3244357854 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00ee60 con 0x55ef42a98fe0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.613+0000 7f6802ffd700 1 -- 172.21.15.52:0/3244357854 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f681c003a00 con 0x55ef42a98fe0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.613+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68140053e0 con 0x55ef42a98fe0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.614+0000 7f6802ffd700 1 -- 172.21.15.52:0/3244357854 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f681c003c00 con 0x55ef42a98fe0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.614+0000 7f68348acb80 10 client.21708 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.614+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ef42b17ef0 con 0x55ef42a98fe0 2024-06-02T23:36:01.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.614+0000 7f6802ffd700 1 -- 172.21.15.52:0/3244357854 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f681c003da0 con 0x55ef42a98fe0 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.614+0000 7f6802ffd700 1 -- 172.21.15.52:0/3244357854 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f681c0046b0 con 0x55ef42a98fe0 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68348acb80 10 client.21708 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68348acb80 10 client.21708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68348acb80 20 client.21708 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68348acb80 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42b1a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68348acb80 4 client.21708 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f68140053e0 con 0x55ef42aaca20 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68237fe700 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42b1a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f68237fe700 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42b1a610 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628b8f0fe68df64e server_cookie=6130403648cfdd8b in_seq=0 out_seq=0 2024-06-02T23:36:01.617 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.615+0000 7f6802ffd700 10 client.21708 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:01.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f6802ffd700 1 -- 172.21.15.52:0/3244357854 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f68140053e0 con 0x55ef42aaca20 2024-06-02T23:36:01.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f6802ffd700 10 client.21708 handle_command_reply: tid=0 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 client.21708 shutdown 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 msgr2=0x55ef42aa1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42aa1e90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f681c0209b0 tx=0x7f681c0206f0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 shutdown_connections 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ef42aaca20 0x55ef42b1a610 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ef42a98fe0 0x55ef42aa1e90 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f6802ffd700 0 client.21708 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef42a92fa0 0x55ef42aa4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 --2- 172.21.15.52:0/3244357854 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef42a986c0 0x55ef42aa5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 >> 172.21.15.52:0/3244357854 conn(0x55ef429bdd40 msgr2=0x55ef42a9a890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 shutdown_connections 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 1 -- 172.21.15.52:0/3244357854 wait complete. 2024-06-02T23:36:01.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.617+0000 7f68348acb80 20 client.21708 trim_cache size 0 max 16384 2024-06-02T23:36:01.620 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:36:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:36:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "6d6f0ace-54e2-444d-8fbf-19cb804f5654", 2024-06-02T23:36:01.620 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:36:01.620 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.618+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f680c0a2ca0 msgr2=0x7f680c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.618+0000 7f682a029700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f680c0a2ca0 0x7f680c0a5160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f6814009c60 tx=0x7f6814009260 comp rx=0 tx=0).stop 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.618+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682406b350 msgr2=0x7f682419b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.618+0000 7f682a029700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682406b350 0x7f682419b2b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f68180059d0 tx=0x7f6818004790 comp rx=0 tx=0).stop 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.618+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 shutdown_connections 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.618+0000 7f682a029700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f680c0a2ca0 0x7f680c0a5160 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.619+0000 7f682a029700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f682410ef30 0x7f682419f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.619+0000 7f682a029700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f682406aa30 0x7f682419ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.619+0000 7f682a029700 1 --2- 172.21.15.52:0/3023061688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f682406b350 0x7f682419b2b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.619+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 >> 172.21.15.52:0/3023061688 conn(0x7f6824078300 msgr2=0x7f6824110380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.619+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 shutdown_connections 2024-06-02T23:36:01.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:01.619+0000 7f682a029700 1 -- 172.21.15.52:0/3023061688 wait complete. 2024-06-02T23:36:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:01 smithi123 ceph-mon[35977]: pgmap v1155: 97 pgs: 97 active+clean; 600 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-06-02T23:36:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:01 smithi174 ceph-mon[35994]: pgmap v1155: 97 pgs: 97 active+clean; 600 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 898 KiB/s rd, 4.3 MiB/s wr, 269 op/s 2024-06-02T23:36:02.473 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:36:02.480 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80649 6.68 MB/sec execute 550 sec latency 70.754 ms 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:02 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.834 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:02.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:36:02.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.895 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:02 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:36:02.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:02.901 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:02 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:36:03.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80805 6.67 MB/sec execute 551 sec latency 69.467 ms 2024-06-02T23:36:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:03 smithi123 ceph-mon[35977]: pgmap v1156: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 304 op/s 2024-06-02T23:36:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:03 smithi174 ceph-mon[35994]: pgmap v1156: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 304 op/s 2024-06-02T23:36:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:03 smithi052 ceph-mon[32340]: pgmap v1156: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.4 MiB/s wr, 304 op/s 2024-06-02T23:36:04.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 80956 6.67 MB/sec execute 552 sec latency 63.750 ms 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.842+0000 7f263eb2e700 1 -- 172.21.15.52:0/3122360118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806ab30 msgr2=0x7f263810f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.842+0000 7f263eb2e700 1 --2- 172.21.15.52:0/3122360118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806ab30 0x7f263810f320 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f2630009f90 tx=0x7f263000b330 comp rx=0 tx=0).stop 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 -- 172.21.15.52:0/3122360118 shutdown_connections 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 --2- 172.21.15.52:0/3122360118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806ab30 0x7f263810f320 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 --2- 172.21.15.52:0/3122360118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2638108500 0x7f26381088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 --2- 172.21.15.52:0/3122360118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f263806a170 0x7f263806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 -- 172.21.15.52:0/3122360118 >> 172.21.15.52:0/3122360118 conn(0x7f2638078340 msgr2=0x7f2638078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 -- 172.21.15.52:0/3122360118 shutdown_connections 2024-06-02T23:36:04.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.843+0000 7f263eb2e700 1 -- 172.21.15.52:0/3122360118 wait complete. 2024-06-02T23:36:04.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 Processor -- start 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 -- start start 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806a170 0x7f263819e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 0x7f263819ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 0x7f2638198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2638112680 con 0x7f263806a170 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2638112500 con 0x7f263806ab30 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.844+0000 7f263eb2e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2638112800 con 0x7f2638108500 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 0x7f263819ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f263d0cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 0x7f2638198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 0x7f263819ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56152/0 (socket says 172.21.15.52:56152) 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f263d0cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 0x7f2638198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34044/0 (socket says 172.21.15.52:34044) 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 -- 172.21.15.52:0/1939281042 learned_addr learned my addr 172.21.15.52:0/1939281042 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:04.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806a170 0x7f263819e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 -- 172.21.15.52:0/1939281042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 msgr2=0x7f2638198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 0x7f2638198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 -- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806a170 msgr2=0x7f263819e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806a170 0x7f263819e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 -- 172.21.15.52:0/1939281042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26381990c0 con 0x7f263806ab30 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 0x7f2638198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806a170 0x7f263819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:04.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.845+0000 7f2637fff700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 0x7f263819ece0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f262400e9c0 tx=0x7f262400ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.846+0000 7f2635ffb700 1 -- 172.21.15.52:0/1939281042 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262400cbd0 con 0x7f263806ab30 2024-06-02T23:36:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.846+0000 7f2635ffb700 1 -- 172.21.15.52:0/1939281042 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f262400cd70 con 0x7f263806ab30 2024-06-02T23:36:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.846+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26381993b0 con 0x7f263806ab30 2024-06-02T23:36:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.846+0000 7f2635ffb700 1 -- 172.21.15.52:0/1939281042 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26240056c0 con 0x7f263806ab30 2024-06-02T23:36:04.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.846+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2638101390 con 0x7f263806ab30 2024-06-02T23:36:04.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.848+0000 7f2635ffb700 1 -- 172.21.15.52:0/1939281042 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2624004750 con 0x7f263806ab30 2024-06-02T23:36:04.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.850+0000 7f2635ffb700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26280a2d70 0x7f26280a5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.850+0000 7f2635ffb700 1 -- 172.21.15.52:0/1939281042 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f26240cf260 con 0x7f263806ab30 2024-06-02T23:36:04.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.850+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26280a2d70 0x7f26280a5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26280a2d70 0x7f26280a5230 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2638199aa0 tx=0x7f262c009310 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:04.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 Processor -- start 2024-06-02T23:36:04.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 -- start start 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 0x55ea1fc23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fb74810 con 0x55ea1fc22f20 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fb1c720 con 0x55ea1fc28640 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.851+0000 7f26493b1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fb707d0 con 0x55ea1fc28f60 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 0x55ea1fc23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f2637fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 0x55ea1fc23340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45610/0 (socket says 172.21.15.52:45610) 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 -- 172.21.15.52:0/1052633373 learned_addr learned my addr 172.21.15.52:0/1052633373 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 -- 172.21.15.52:0/1052633373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 msgr2=0x55ea1fc229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 -- 172.21.15.52:0/1052633373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 msgr2=0x55ea1fc28a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc28a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 -- 172.21.15.52:0/1052633373 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1fb12f50 con 0x55ea1fc22f20 2024-06-02T23:36:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 0x55ea1fc23340 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2630004e10 tx=0x7f2630004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6f8a0a37e14d239 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:04.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.852+0000 7f2622ffd700 1 -- 172.21.15.52:0/1052633373 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630020070 con 0x55ea1fc22f20 2024-06-02T23:36:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.853+0000 7f2622ffd700 1 -- 172.21.15.52:0/1052633373 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2630007420 con 0x55ea1fc22f20 2024-06-02T23:36:04.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.853+0000 7f2622ffd700 1 -- 172.21.15.52:0/1052633373 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26300052a0 con 0x55ea1fc22f20 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.853+0000 7f26493b1b80 1 -- 172.21.15.52:0/1052633373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 msgr2=0x55ea1fc23340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.853+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 0x55ea1fc23340 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f2630004e10 tx=0x7f2630004e40 comp rx=0 tx=0).stop 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 -- 172.21.15.52:0/1052633373 shutdown_connections 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc22f20 0x55ea1fc23340 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc28a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1052633373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 -- 172.21.15.52:0/1052633373 >> 172.21.15.52:0/1052633373 conn(0x55ea1fb4dcc0 msgr2=0x55ea1fc1d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 -- 172.21.15.52:0/1052633373 shutdown_connections 2024-06-02T23:36:04.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.854+0000 7f26493b1b80 1 -- 172.21.15.52:0/1052633373 wait complete. 2024-06-02T23:36:04.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.855+0000 7f26493b1b80 1 -- start start 2024-06-02T23:36:04.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.855+0000 7f26493b1b80 1 Processor -- start 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc2ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc2c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28f60 0x55ea1fc2c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f263c8ca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc2ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fc2f970 con 0x55ea1fc28f60 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fb14bf0 con 0x55ea1fc28640 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ea1fc2faf0 con 0x55ea1fc22f20 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f263c8ca700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc2ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:34070/0 (socket says 172.21.15.52:34070) 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f263c8ca700 1 -- 172.21.15.52:0/865127476 learned_addr learned my addr 172.21.15.52:0/865127476 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f26493b1b80 10 client.0 _ll_register_callbacks cb 0x55ea1fb0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:04.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f2637fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc2c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.856+0000 7f263d0cb700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28f60 0x55ea1fc2c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f263c8ca700 1 -- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 msgr2=0x55ea1fc2c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f263c8ca700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc2c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f263c8ca700 1 -- 172.21.15.52:0/865127476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28f60 msgr2=0x55ea1fc2c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f263c8ca700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28f60 0x55ea1fc2c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f263c8ca700 1 -- 172.21.15.52:0/865127476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1fac38d0 con 0x55ea1fc22f20 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f2637fff700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc2c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f263d0cb700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28f60 0x55ea1fc2c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.857+0000 7f2617fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f263c8ca700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc2ed70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f262c012c40 tx=0x7f262c0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4713fd5e466fa6ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f2617fff700 1 -- 172.21.15.52:0/865127476 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c020640 con 0x55ea1fc22f20 2024-06-02T23:36:04.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2630007420 con 0x55ea1fc22f20 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f2617fff700 1 -- 172.21.15.52:0/865127476 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f262c011e40 con 0x55ea1fc22f20 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f2617fff700 1 -- 172.21.15.52:0/865127476 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f262c030070 con 0x55ea1fc22f20 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f26493b1b80 10 client.30526 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.858+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea1fc3a3c0 con 0x55ea1fc22f20 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f2617fff700 1 -- 172.21.15.52:0/865127476 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2630007420 con 0x55ea1fc22f20 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f2617fff700 1 -- 172.21.15.52:0/865127476 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f262c01f990 con 0x55ea1fc22f20 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f26493b1b80 10 client.30526 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f26493b1b80 10 client.30526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f26493b1b80 20 client.30526 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:04.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f26493b1b80 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fc3ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f26493b1b80 4 client.30526 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:04.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ea1fb81d10 con 0x55ea1fc3c950 2024-06-02T23:36:04.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.859+0000 7f2637fff700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fc3ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.860+0000 7f2617fff700 10 client.30526 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:04.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.860+0000 7f2637fff700 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fc3ed40 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e7d1611d273f821 server_cookie=eedf8c417167d698 in_seq=0 out_seq=0 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f2617fff700 1 -- 172.21.15.52:0/865127476 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ea1fb81d10 con 0x55ea1fc3c950 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f2617fff700 10 client.30526 handle_command_reply: tid=0 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 client.30526 shutdown 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 msgr2=0x55ea1fc2ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc2ed70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f262c012c40 tx=0x7f262c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 shutdown_connections 2024-06-02T23:36:04.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fc3ed40 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f2617fff700 0 client.30526 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28f60 0x55ea1fc2c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc2ed70 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 --2- 172.21.15.52:0/865127476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28640 0x55ea1fc2c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 >> 172.21.15.52:0/865127476 conn(0x55ea1fb4dcc0 msgr2=0x55ea1fc219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 shutdown_connections 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 1 -- 172.21.15.52:0/865127476 wait complete. 2024-06-02T23:36:04.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.863+0000 7f26493b1b80 20 client.30526 trim_cache size 0 max 16384 2024-06-02T23:36:04.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 Processor -- start 2024-06-02T23:36:04.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 -- start start 2024-06-02T23:36:04.891 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc22f20 0x55ea1fc229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc27a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26381aebb0 con 0x55ea1fc28640 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26381b06f0 con 0x55ea1fc22f20 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f26493b1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26381af280 con 0x55ea1fc28f60 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f2637fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc27a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f2637fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc27a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45622/0 (socket says 172.21.15.52:45622) 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f2637fff700 1 -- 172.21.15.52:0/2016790825 learned_addr learned my addr 172.21.15.52:0/2016790825 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f263d0cb700 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.891+0000 7f263c8ca700 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc22f20 0x55ea1fc229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f263d0cb700 1 -- 172.21.15.52:0/2016790825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc22f20 msgr2=0x55ea1fc229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f263d0cb700 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc22f20 0x55ea1fc229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f263d0cb700 1 -- 172.21.15.52:0/2016790825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 msgr2=0x55ea1fc27a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.892 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f263d0cb700 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc27a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f263d0cb700 1 -- 172.21.15.52:0/2016790825 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ea1fc26b50 con 0x55ea1fc28f60 2024-06-02T23:36:04.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f2637fff700 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc27a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:04.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f263d0cb700 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc445f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f263000b6c0 tx=0x7f26300061a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dfb91ed4665cc18 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f26227fc700 1 -- 172.21.15.52:0/2016790825 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630020040 con 0x55ea1fc28f60 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f26227fc700 1 -- 172.21.15.52:0/2016790825 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f263000b9f0 con 0x55ea1fc28f60 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.892+0000 7f26227fc700 1 -- 172.21.15.52:0/2016790825 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263001c9e0 con 0x55ea1fc28f60 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 -- 172.21.15.52:0/2016790825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 msgr2=0x55ea1fc445f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc445f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f263000b6c0 tx=0x7f26300061a0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 -- 172.21.15.52:0/2016790825 shutdown_connections 2024-06-02T23:36:04.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc28f60 0x55ea1fc445f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc22f20 0x55ea1fc229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 --2- 172.21.15.52:0/2016790825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc27a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.893+0000 7f26493b1b80 1 -- 172.21.15.52:0/2016790825 >> 172.21.15.52:0/2016790825 conn(0x55ea1fb4dcc0 msgr2=0x55ea1fc17470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.894+0000 7f26493b1b80 1 -- 172.21.15.52:0/2016790825 shutdown_connections 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.894+0000 7f26493b1b80 1 -- 172.21.15.52:0/2016790825 wait complete. 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.894+0000 7f26493b1b80 1 -- start start 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.894+0000 7f26493b1b80 1 Processor -- start 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc33440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc33980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 0x55ea1fca05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f26381b0020 con 0x55ea1fc28640 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26381af950 con 0x55ea1fc28f60 2024-06-02T23:36:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f26381acb90 con 0x55ea1fc22f20 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f26493b1b80 10 client.0 _ll_register_callbacks cb 0x55ea1fb0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 0x55ea1fca05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f2637fff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc33980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 0x55ea1fca05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56180/0 (socket says 172.21.15.52:56180) 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 -- 172.21.15.52:0/1171747323 learned_addr learned my addr 172.21.15.52:0/1171747323 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc33440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 -- 172.21.15.52:0/1171747323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 msgr2=0x55ea1fc33440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc33440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 -- 172.21.15.52:0/1171747323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 msgr2=0x55ea1fc33980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc33980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f263d0cb700 1 -- 172.21.15.52:0/1171747323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ea1fca0be0 con 0x55ea1fc28f60 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.895+0000 7f2637fff700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc33980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f263d0cb700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 0x55ea1fca05a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f263000b370 tx=0x7f2630007bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77cbdb8ff24cbcf server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:04.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f2617fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f2617fff700 1 -- 172.21.15.52:0/1171747323 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2630020040 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263000b9f0 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f2617fff700 1 -- 172.21.15.52:0/1171747323 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2630039520 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f2617fff700 1 -- 172.21.15.52:0/1171747323 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f2630039740 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f26493b1b80 10 client.30956 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55ea1fca7f80 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.896+0000 7f2617fff700 1 -- 172.21.15.52:0/1171747323 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f263000b9f0 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f2617fff700 1 -- 172.21.15.52:0/1171747323 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f2630011070 con 0x55ea1fc28f60 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f26493b1b80 10 client.30956 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f26493b1b80 10 client.30956 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f26493b1b80 20 client.30956 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fcaa160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:04.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f26493b1b80 4 client.30956 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ea1fb7c290 con 0x55ea1fc3c950 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fcaa160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f263c8ca700 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fcaa160 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38342f7af36474c server_cookie=48fd0b8f1610d5cd in_seq=0 out_seq=0 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.897+0000 7f2617fff700 10 client.30956 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f2617fff700 1 -- 172.21.15.52:0/1171747323 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ea1fb7c290 con 0x55ea1fc3c950 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f2617fff700 10 client.30956 handle_command_reply: tid=0 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 client.30956 shutdown 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 msgr2=0x55ea1fca05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 0x55ea1fca05a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f263000b370 tx=0x7f2630007bc0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 shutdown_connections 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ea1fc3c950 0x55ea1fcaa160 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ea1fc28f60 0x55ea1fca05a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ea1fc22f20 0x55ea1fc33440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.902 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 --2- 172.21.15.52:0/1171747323 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ea1fc28640 0x55ea1fc33980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 >> 172.21.15.52:0/1171747323 conn(0x55ea1fb4dcc0 msgr2=0x55ea1fc27a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f2617fff700 0 client.30956 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 shutdown_connections 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 1 -- 172.21.15.52:0/1171747323 wait complete. 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.898+0000 7f26493b1b80 20 client.30956 trim_cache size 0 max 16384 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stdout: "6d6f0ace-54e2-444d-8fbf-19cb804f5654": { 2024-06-02T23:36:04.903 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:36:04.904 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "6d6f0ace-54e2-444d-8fbf-19cb804f5654", 2024-06-02T23:36:04.904 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:36:04.904 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:04.904 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:04.904 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:36:04.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26280a2d70 msgr2=0x7f26280a5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26280a2d70 0x7f26280a5230 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f2638199aa0 tx=0x7f262c009310 comp rx=0 tx=0).stop 2024-06-02T23:36:04.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 msgr2=0x7f263819ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:04.905 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 0x7f263819ece0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f262400e9c0 tx=0x7f262400ecd0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 shutdown_connections 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f26280a2d70 0x7f26280a5230 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2638108500 0x7f2638198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f263806a170 0x7f263819e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 --2- 172.21.15.52:0/1939281042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f263806ab30 0x7f263819ece0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 >> 172.21.15.52:0/1939281042 conn(0x7f2638078340 msgr2=0x7f26380fff90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 shutdown_connections 2024-06-02T23:36:04.906 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:04.899+0000 7f263eb2e700 1 -- 172.21.15.52:0/1939281042 wait complete. 2024-06-02T23:36:05.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 81102 6.67 MB/sec execute 553 sec latency 72.214 ms 2024-06-02T23:36:05.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:05 smithi052 ceph-mon[32340]: pgmap v1157: 97 pgs: 97 active+clean; 603 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.9 MiB/s wr, 261 op/s 2024-06-02T23:36:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:05 smithi123 ceph-mon[35977]: pgmap v1157: 97 pgs: 97 active+clean; 603 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.9 MiB/s wr, 261 op/s 2024-06-02T23:36:05.796 INFO:tasks.cephfs.filesystem:scrub status for tag:6d6f0ace-54e2-444d-8fbf-19cb804f5654 - {'path': '/', 'tag': '6d6f0ace-54e2-444d-8fbf-19cb804f5654', 'options': 'recursive,force'} 2024-06-02T23:36:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:05 smithi174 ceph-mon[35994]: pgmap v1157: 97 pgs: 97 active+clean; 603 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 3.9 MiB/s wr, 261 op/s 2024-06-02T23:36:06.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 81268 6.68 MB/sec execute 554 sec latency 69.629 ms 2024-06-02T23:36:07.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 81462 6.68 MB/sec execute 555 sec latency 56.463 ms 2024-06-02T23:36:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:07 smithi123 ceph-mon[35977]: pgmap v1158: 97 pgs: 97 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-06-02T23:36:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:07 smithi174 ceph-mon[35994]: pgmap v1158: 97 pgs: 97 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-06-02T23:36:07.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:07 smithi052 ceph-mon[32340]: pgmap v1158: 97 pgs: 97 active+clean; 604 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 360 op/s 2024-06-02T23:36:08.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 81678 6.69 MB/sec execute 556 sec latency 58.714 ms 2024-06-02T23:36:09.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 81871 6.70 MB/sec execute 557 sec latency 77.375 ms 2024-06-02T23:36:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:09 smithi123 ceph-mon[35977]: pgmap v1159: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-06-02T23:36:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:09 smithi174 ceph-mon[35994]: pgmap v1159: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-06-02T23:36:09.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:09 smithi052 ceph-mon[32340]: pgmap v1159: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 979 KiB/s rd, 4.1 MiB/s wr, 304 op/s 2024-06-02T23:36:10.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 81986 6.69 MB/sec execute 558 sec latency 254.146 ms 2024-06-02T23:36:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.712+0000 7f3db7d16640 1 -- 172.21.15.52:0/2281405499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 msgr2=0x7f3db0108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.712 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.712+0000 7f3db7d16640 1 --2- 172.21.15.52:0/2281405499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 0x7f3db0108d60 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009be0 tx=0x7f3da802a6c0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 -- 172.21.15.52:0/2281405499 shutdown_connections 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 --2- 172.21.15.52:0/2281405499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 0x7f3db0108d60 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 --2- 172.21.15.52:0/2281405499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0101b10 0x7f3db0101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 --2- 172.21.15.52:0/2281405499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0100920 0x7f3db0100d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 -- 172.21.15.52:0/2281405499 >> 172.21.15.52:0/2281405499 conn(0x7f3db00fbeb0 msgr2=0x7f3db00fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 -- 172.21.15.52:0/2281405499 shutdown_connections 2024-06-02T23:36:10.713 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 -- 172.21.15.52:0/2281405499 wait complete. 2024-06-02T23:36:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 Processor -- start 2024-06-02T23:36:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.713+0000 7f3db7d16640 1 -- start start 2024-06-02T23:36:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db7d16640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 0x7f3db0197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db7d16640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0101b10 0x7f3db0198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.715 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db7d16640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 0x7f3db019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db7d16640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3db010c430 con 0x7f3db0100920 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db7d16640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db010c2b0 con 0x7f3db01024d0 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db7d16640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3db010c5b0 con 0x7f3db0101b10 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db5ab2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 0x7f3db0197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db5ab2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 0x7f3db0197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45656/0 (socket says 172.21.15.52:45656) 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db5ab2640 1 -- 172.21.15.52:0/1279435592 learned_addr learned my addr 172.21.15.52:0/1279435592 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0101b10 0x7f3db0198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db62b3640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 0x7f3db019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db52b1640 1 -- 172.21.15.52:0/1279435592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 msgr2=0x7f3db019f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 0x7f3db019f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db52b1640 1 -- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 msgr2=0x7f3db0197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 0x7f3db0197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.716 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db52b1640 1 -- 172.21.15.52:0/1279435592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3db019f750 con 0x7f3db0101b10 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.714+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 0x7f3db0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.715+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0101b10 0x7f3db0198070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3da0005fd0 tx=0x7f3da000ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.715+0000 7f3da6ffd640 1 -- 172.21.15.52:0/1279435592 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da000cd00 con 0x7f3db0101b10 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.715+0000 7f3da6ffd640 1 -- 172.21.15.52:0/1279435592 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da00035f0 con 0x7f3db0101b10 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.715+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3db019fa40 con 0x7f3db0101b10 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.715+0000 7f3da6ffd640 1 -- 172.21.15.52:0/1279435592 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0010690 con 0x7f3db0101b10 2024-06-02T23:36:10.717 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.715+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3db019ffe0 con 0x7f3db0101b10 2024-06-02T23:36:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.718+0000 7f3da6ffd640 1 -- 172.21.15.52:0/1279435592 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3da00037a0 con 0x7f3db0101b10 2024-06-02T23:36:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 Processor -- start 2024-06-02T23:36:10.718 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 -- start start 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf27840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf2c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf2f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e27ce864f0 con 0x55e27cf27e10 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e27ce24870 con 0x55e27cf2cf80 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e27ce824b0 con 0x55e27cf27420 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3db52b1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf2f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3db52b1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf2f410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56216/0 (socket says 172.21.15.52:56216) 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3db52b1640 1 -- 172.21.15.52:0/3945643096 learned_addr learned my addr 172.21.15.52:0/3945643096 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:10.720 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3db62b3640 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf27840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.719+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf2c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db52b1640 1 -- 172.21.15.52:0/3945643096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 msgr2=0x55e27cf27840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db52b1640 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf27840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db52b1640 1 -- 172.21.15.52:0/3945643096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 msgr2=0x55e27cf2c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db52b1640 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf2c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db52b1640 1 -- 172.21.15.52:0/3945643096 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27ce409b0 con 0x55e27cf2cf80 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3da6ffd640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d9c0a2cb0 0x7f3d9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db62b3640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d9c0a2cb0 0x7f3d9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db52b1640 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf2f410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3da00d72f0 tx=0x7f3da00d77b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=83856d9c4baad58b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3da6ffd640 1 -- 172.21.15.52:0/1279435592 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3da00cf230 con 0x7f3db0101b10 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3d9bfff640 1 -- 172.21.15.52:0/3945643096 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0097d60 con 0x55e27cf2cf80 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3db62b3640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d9c0a2cb0 0x7f3d9c0a5170 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009370 tx=0x7f3da8030920 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.720+0000 7f3d9bfff640 1 -- 172.21.15.52:0/3945643096 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da009c1c0 con 0x55e27cf2cf80 2024-06-02T23:36:10.721 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3d9bfff640 1 -- 172.21.15.52:0/3945643096 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da009b420 con 0x55e27cf2cf80 2024-06-02T23:36:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/3945643096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 msgr2=0x55e27cf2f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.724 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf2f410 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3da00d72f0 tx=0x7f3da00d77b0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/3945643096 shutdown_connections 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf2f410 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf2c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/3945643096 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf27840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/3945643096 >> 172.21.15.52:0/3945643096 conn(0x55e27cf1e680 msgr2=0x55e27cf1ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/3945643096 shutdown_connections 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.721+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/3945643096 wait complete. 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 -- start start 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 Processor -- start 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf31660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf31ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf38be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.725 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e27cf2ff30 con 0x55e27cf27e10 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e27ce0fae0 con 0x55e27cf2cf80 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e27cf300b0 con 0x55e27cf27420 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3db52b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf31ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3dbaa1a740 10 client.0 _ll_register_callbacks cb 0x55e27ce489b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3db52b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf31ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45672/0 (socket says 172.21.15.52:45672) 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3db52b1640 1 -- 172.21.15.52:0/1139135201 learned_addr learned my addr 172.21.15.52:0/1139135201 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf31660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.722+0000 7f3db62b3640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf38be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db52b1640 1 -- 172.21.15.52:0/1139135201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 msgr2=0x55e27cf31660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf31660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db52b1640 1 -- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 msgr2=0x55e27cf38be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf38be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db52b1640 1 -- 172.21.15.52:0/1139135201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27cb9a210 con 0x55e27cf27e10 2024-06-02T23:36:10.726 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db62b3640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf38be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3db52b1640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf31ba0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3da0098490 tx=0x7f3da000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe750a3cf21603c7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3d997fa640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3d997fa640 1 -- 172.21.15.52:0/1139135201 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da00b4060 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3d997fa640 1 -- 172.21.15.52:0/1139135201 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3da0003b10 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da009c1c0 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3d997fa640 1 -- 172.21.15.52:0/1139135201 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3da009b600 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3dbaa1a740 10 client.21738 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.723+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e27cc66620 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3d997fa640 1 -- 172.21.15.52:0/1139135201 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da009c1c0 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3d997fa640 1 -- 172.21.15.52:0/1139135201 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3da009bdb0 con 0x55e27cf27e10 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3dbaa1a740 10 client.21738 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3dbaa1a740 10 client.21738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:10.727 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3dbaa1a740 20 client.21738 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cf3d0b0 0x55e27cf3f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3dbaa1a740 4 client.21738 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.724+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e27ce99f10 con 0x55e27cf3d0b0 2024-06-02T23:36:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.725+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cf3d0b0 0x55e27cf3f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.725+0000 7f3d997fa640 10 client.21738 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:10.728 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.725+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cf3d0b0 0x55e27cf3f4a0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb8d31101717446 server_cookie=7e11bee905d433e9 in_seq=0 out_seq=0 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3d997fa640 1 -- 172.21.15.52:0/1139135201 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e27ce99f10 con 0x55e27cf3d0b0 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3d997fa640 10 client.21738 handle_command_reply: tid=0 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 client.21738 shutdown 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 msgr2=0x55e27cf31ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf31ba0 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3da0098490 tx=0x7f3da000cf30 comp rx=0 tx=0).stop 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 shutdown_connections 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cf3d0b0 0x55e27cf3f4a0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf38be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.729 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf31ba0 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/1139135201 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf31660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 >> 172.21.15.52:0/1139135201 conn(0x55e27cf1e680 msgr2=0x55e27cf1d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.726+0000 7f3d997fa640 0 client.21738 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.727+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 shutdown_connections 2024-06-02T23:36:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.727+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/1139135201 wait complete. 2024-06-02T23:36:10.730 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.727+0000 7f3dbaa1a740 20 client.21738 trim_cache size 0 max 16384 2024-06-02T23:36:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 Processor -- start 2024-06-02T23:36:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 -- start start 2024-06-02T23:36:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf29030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf38ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf29570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a6b00 con 0x55e27cf27e10 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a8640 con 0x55e27cf2cf80 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a71d0 con 0x55e27cf27420 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db52b1640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf38ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db62b3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf29570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db5ab2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf29030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db62b3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf29570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:56236/0 (socket says 172.21.15.52:56236) 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db62b3640 1 -- 172.21.15.52:0/662483623 learned_addr learned my addr 172.21.15.52:0/662483623 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db52b1640 1 -- 172.21.15.52:0/662483623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 msgr2=0x55e27cf29030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db52b1640 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf29030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db52b1640 1 -- 172.21.15.52:0/662483623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 msgr2=0x55e27cf29570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.753 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.752+0000 7f3db52b1640 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf29570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.753+0000 7f3db52b1640 1 -- 172.21.15.52:0/662483623 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e27cf1e4b0 con 0x55e27cf27e10 2024-06-02T23:36:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.753+0000 7f3db62b3640 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf29570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.753+0000 7f3db52b1640 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf38ba0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3da009ba50 tx=0x7f3da00d77b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3172df9b13a1a9d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.753+0000 7f3d9b7fe640 1 -- 172.21.15.52:0/662483623 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da00b4070 con 0x55e27cf27e10 2024-06-02T23:36:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.753+0000 7f3d9b7fe640 1 -- 172.21.15.52:0/662483623 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da009c1c0 con 0x55e27cf27e10 2024-06-02T23:36:10.754 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.753+0000 7f3d9b7fe640 1 -- 172.21.15.52:0/662483623 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da009b890 con 0x55e27cf27e10 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/662483623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 msgr2=0x55e27cf38ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf38ba0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f3da009ba50 tx=0x7f3da00d77b0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/662483623 shutdown_connections 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf29570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27e10 0x55e27cf38ba0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/662483623 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27420 0x55e27cf29030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/662483623 >> 172.21.15.52:0/662483623 conn(0x55e27cf1e680 msgr2=0x55e27cf31660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/662483623 shutdown_connections 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/662483623 wait complete. 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.754+0000 7f3dbaa1a740 1 -- start start 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 Processor -- start 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 0x55e27cf386c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27e10 0x55e27cf35160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a7f70 con 0x55e27cf27420 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a78a0 con 0x55e27cf2cf80 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3db01a4ae0 con 0x55e27cf27e10 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3dbaa1a740 10 client.0 _ll_register_callbacks cb 0x7f3dac009530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db5ab2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 0x55e27cf386c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db52b1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27e10 0x55e27cf35160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db5ab2640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 0x55e27cf386c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:45684/0 (socket says 172.21.15.52:45684) 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db5ab2640 1 -- 172.21.15.52:0/154120806 learned_addr learned my addr 172.21.15.52:0/154120806 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db62b3640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db52b1640 1 -- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 msgr2=0x55e27cf356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db52b1640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db52b1640 1 -- 172.21.15.52:0/154120806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 msgr2=0x55e27cf386c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db52b1640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 0x55e27cf386c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db52b1640 1 -- 172.21.15.52:0/154120806 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e27cf38cf0 con 0x55e27cf27e10 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.755+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 0x55e27cf386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3d997fa640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3db52b1640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27e10 0x55e27cf35160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3da009ba50 tx=0x7f3da00a4410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca550af401434cd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3d997fa640 1 -- 172.21.15.52:0/154120806 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da00b4070 con 0x55e27cf27e10 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3d997fa640 1 -- 172.21.15.52:0/154120806 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3da0010830 con 0x55e27cf27e10 2024-06-02T23:36:10.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3da009c1c0 con 0x55e27cf27e10 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3d997fa640 1 -- 172.21.15.52:0/154120806 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f3da009b890 con 0x55e27cf27e10 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3dbaa1a740 10 client.30550 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e27cf38cf0 con 0x55e27cf27e10 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.756+0000 7f3d997fa640 1 -- 172.21.15.52:0/154120806 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3da009c1c0 con 0x55e27cf27e10 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3d997fa640 1 -- 172.21.15.52:0/154120806 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f3da00ba190 con 0x55e27cf27e10 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3dbaa1a740 10 client.30550 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3dbaa1a740 10 client.30550 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3dbaa1a740 20 client.30550 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cfa6ed0 0x55e27cfa92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3dbaa1a740 4 client.30550 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e27ce93c20 con 0x55e27cfa6ed0 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cfa6ed0 0x55e27cfa92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3db5ab2640 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cfa6ed0 0x55e27cfa92c0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4128f93ecd9b737 server_cookie=72ebc669deb156c4 in_seq=0 out_seq=0 2024-06-02T23:36:10.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.757+0000 7f3d997fa640 10 client.30550 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.828+0000 7f3d997fa640 1 -- 172.21.15.52:0/154120806 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e27ce93c20 con 0x55e27cfa6ed0 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.828+0000 7f3d997fa640 10 client.30550 handle_command_reply: tid=0 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.828+0000 7f3dbaa1a740 1 client.30550 shutdown 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.828+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27e10 msgr2=0x55e27cf35160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27e10 0x55e27cf35160 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3da009ba50 tx=0x7f3da00a4410 comp rx=0 tx=0).stop 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 shutdown_connections 2024-06-02T23:36:10.829 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e27cfa6ed0 0x55e27cfa92c0 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e27cf27e10 0x55e27cf35160 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e27cf2cf80 0x55e27cf356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3d997fa640 0 client.30550 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 --2- 172.21.15.52:0/154120806 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e27cf27420 0x55e27cf386c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 >> 172.21.15.52:0/154120806 conn(0x55e27cf1e680 msgr2=0x55e27cf31590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 shutdown_connections 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 1 -- 172.21.15.52:0/154120806 wait complete. 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.829+0000 7f3dbaa1a740 20 client.30550 trim_cache size 0 max 16384 2024-06-02T23:36:10.830 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:36:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:36:10.831 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d9c0a2cb0 msgr2=0x7f3d9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d9c0a2cb0 0x7f3d9c0a5170 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009370 tx=0x7f3da8030920 comp rx=0 tx=0).stop 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0101b10 msgr2=0x7f3db0198070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0101b10 0x7f3db0198070 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f3da0005fd0 tx=0x7f3da000ee50 comp rx=0 tx=0).stop 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 shutdown_connections 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d9c0a2cb0 0x7f3d9c0a5170 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3db0101b10 0x7f3db0198070 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3db01024d0 0x7f3db019f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.832 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 --2- 172.21.15.52:0/1279435592 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3db0100920 0x7f3db0197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 >> 172.21.15.52:0/1279435592 conn(0x7f3db00fbeb0 msgr2=0x7f3db00fd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 shutdown_connections 2024-06-02T23:36:10.833 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:10.830+0000 7f3db7d16640 1 -- 172.21.15.52:0/1279435592 wait complete. 2024-06-02T23:36:11.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 82186 6.71 MB/sec execute 559 sec latency 69.004 ms 2024-06-02T23:36:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:11 smithi123 ceph-mon[35977]: pgmap v1160: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-02T23:36:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:36:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:36:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:36:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:36:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:11 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:36:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:11 smithi174 ceph-mon[35994]: pgmap v1160: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-02T23:36:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:36:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:36:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:36:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:36:11.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:11 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:36:11.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:11 smithi052 ceph-mon[32340]: pgmap v1160: 97 pgs: 97 active+clean; 602 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 355 op/s 2024-06-02T23:36:11.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:36:11.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:36:11.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:36:11.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:36:11.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:11 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:36:12.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 82356 6.71 MB/sec execute 560 sec latency 55.984 ms 2024-06-02T23:36:13.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 82519 6.70 MB/sec execute 561 sec latency 55.286 ms 2024-06-02T23:36:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:13 smithi123 ceph-mon[35977]: pgmap v1161: 97 pgs: 97 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 355 op/s 2024-06-02T23:36:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:13 smithi174 ceph-mon[35994]: pgmap v1161: 97 pgs: 97 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 355 op/s 2024-06-02T23:36:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:13 smithi052 ceph-mon[32340]: pgmap v1161: 97 pgs: 97 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.3 MiB/s wr, 355 op/s 2024-06-02T23:36:14.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 82719 6.71 MB/sec execute 562 sec latency 52.704 ms 2024-06-02T23:36:15.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 82914 6.72 MB/sec execute 563 sec latency 53.804 ms 2024-06-02T23:36:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:15 smithi123 ceph-mon[35977]: pgmap v1162: 97 pgs: 97 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-06-02T23:36:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:15 smithi174 ceph-mon[35994]: pgmap v1162: 97 pgs: 97 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-06-02T23:36:15.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:15 smithi052 ceph-mon[32340]: pgmap v1162: 97 pgs: 97 active+clean; 601 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 946 KiB/s rd, 3.6 MiB/s wr, 305 op/s 2024-06-02T23:36:16.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83089 6.72 MB/sec execute 564 sec latency 101.294 ms 2024-06-02T23:36:17.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83238 6.72 MB/sec execute 565 sec latency 119.249 ms 2024-06-02T23:36:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:17 smithi123 ceph-mon[35977]: pgmap v1163: 97 pgs: 97 active+clean; 609 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 360 op/s 2024-06-02T23:36:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:17 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:36:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:17 smithi174 ceph-mon[35994]: pgmap v1163: 97 pgs: 97 active+clean; 609 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 360 op/s 2024-06-02T23:36:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:17 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:36:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:17 smithi052 ceph-mon[32340]: pgmap v1163: 97 pgs: 97 active+clean; 609 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.7 MiB/s wr, 360 op/s 2024-06-02T23:36:17.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:17 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:36:18.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83416 6.74 MB/sec execute 566 sec latency 89.198 ms 2024-06-02T23:36:19.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83571 6.75 MB/sec execute 567 sec latency 149.111 ms 2024-06-02T23:36:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:19 smithi123 ceph-mon[35977]: pgmap v1164: 97 pgs: 97 active+clean; 609 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-06-02T23:36:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:19 smithi174 ceph-mon[35994]: pgmap v1164: 97 pgs: 97 active+clean; 609 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-06-02T23:36:19.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:19 smithi052 ceph-mon[32340]: pgmap v1164: 97 pgs: 97 active+clean; 609 MiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 951 KiB/s rd, 3.8 MiB/s wr, 263 op/s 2024-06-02T23:36:20.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83717 6.75 MB/sec execute 568 sec latency 151.833 ms 2024-06-02T23:36:21.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83838 6.76 MB/sec execute 569 sec latency 157.750 ms 2024-06-02T23:36:21.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:21 smithi052 ceph-mon[32340]: pgmap v1165: 97 pgs: 97 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.6 MiB/s wr, 294 op/s 2024-06-02T23:36:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:21 smithi123 ceph-mon[35977]: pgmap v1165: 97 pgs: 97 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.6 MiB/s wr, 294 op/s 2024-06-02T23:36:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:21 smithi174 ceph-mon[35994]: pgmap v1165: 97 pgs: 97 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 4.6 MiB/s wr, 294 op/s 2024-06-02T23:36:22.481 INFO:tasks.workunit.client.0.smithi052.stdout: 10 83954 6.75 MB/sec execute 570 sec latency 95.575 ms 2024-06-02T23:36:23.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84111 6.76 MB/sec execute 571 sec latency 96.811 ms 2024-06-02T23:36:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:23 smithi123 ceph-mon[35977]: pgmap v1166: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 285 op/s 2024-06-02T23:36:23.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:23 smithi174 ceph-mon[35994]: pgmap v1166: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 285 op/s 2024-06-02T23:36:23.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:23 smithi052 ceph-mon[32340]: pgmap v1166: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.6 MiB/s wr, 285 op/s 2024-06-02T23:36:23.967 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 -- 172.21.15.52:0/2424002945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194103c30 msgr2=0x7ff194104050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 --2- 172.21.15.52:0/2424002945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194103c30 0x7ff194104050 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7ff188009900 tx=0x7ff18802a410 comp rx=0 tx=0).stop 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 -- 172.21.15.52:0/2424002945 shutdown_connections 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 --2- 172.21.15.52:0/2424002945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff194104e20 0x7ff1941052a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 --2- 172.21.15.52:0/2424002945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1941057e0 0x7ff19410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 --2- 172.21.15.52:0/2424002945 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194103c30 0x7ff194104050 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 -- 172.21.15.52:0/2424002945 >> 172.21.15.52:0/2424002945 conn(0x7ff1940ff1c0 msgr2=0x7ff1941015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 -- 172.21.15.52:0/2424002945 shutdown_connections 2024-06-02T23:36:23.968 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.968+0000 7ff19b15a640 1 -- 172.21.15.52:0/2424002945 wait complete. 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 Processor -- start 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 -- start start 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 0x7ff194199df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194104e20 0x7ff19419a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1941057e0 0x7ff1941a1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff19410f780 con 0x7ff194104e20 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff19410f600 con 0x7ff194103c30 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff19b15a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff19410f900 con 0x7ff1941057e0 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff198ef6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 0x7ff194199df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.971 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff198ef6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 0x7ff194199df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42448/0 (socket says 172.21.15.52:42448) 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff193fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194104e20 0x7ff19419a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff198ef6640 1 -- 172.21.15.52:0/1604650822 learned_addr learned my addr 172.21.15.52:0/1604650822 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.969+0000 7ff1996f7640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1941057e0 0x7ff1941a1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff198ef6640 1 -- 172.21.15.52:0/1604650822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1941057e0 msgr2=0x7ff1941a1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff198ef6640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1941057e0 0x7ff1941a1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff198ef6640 1 -- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194104e20 msgr2=0x7ff19419a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff198ef6640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194104e20 0x7ff19419a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff198ef6640 1 -- 172.21.15.52:0/1604650822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1941a19b0 con 0x7ff194103c30 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff1996f7640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1941057e0 0x7ff1941a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff193fff640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194104e20 0x7ff19419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff198ef6640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 0x7ff194199df0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff1880314e0 tx=0x7ff188003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff191ffb640 1 -- 172.21.15.52:0/1604650822 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18803f070 con 0x7ff194103c30 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff191ffb640 1 -- 172.21.15.52:0/1604650822 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff188031c40 con 0x7ff194103c30 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1941a1c40 con 0x7ff194103c30 2024-06-02T23:36:23.972 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff191ffb640 1 -- 172.21.15.52:0/1604650822 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff188035030 con 0x7ff194103c30 2024-06-02T23:36:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.970+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1941a2130 con 0x7ff194103c30 2024-06-02T23:36:23.973 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.973+0000 7ff191ffb640 1 -- 172.21.15.52:0/1604650822 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff188042070 con 0x7ff194103c30 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 Processor -- start 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 -- start start 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89130430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff191ffb640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff17c0a2da0 0x7ff17c0a5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89135500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89138000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561f890934f0 con 0x561f89135b70 2024-06-02T23:36:23.975 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f89031870 con 0x561f89130010 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.974+0000 7ff19de62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f8908f4b0 con 0x561f89130a00 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff1996f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89130430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff191ffb640 1 -- 172.21.15.52:0/1604650822 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff1880ef730 con 0x7ff194103c30 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff1996f7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89130430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42460/0 (socket says 172.21.15.52:42460) 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff1996f7640 1 -- 172.21.15.52:0/1460549557 learned_addr learned my addr 172.21.15.52:0/1460549557 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff193fff640 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89135500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff198ef6640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff17c0a2da0 0x7ff17c0a5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff1996f7640 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff193fff640 1 -- 172.21.15.52:0/1460549557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 msgr2=0x561f89130430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff193fff640 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89130430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff193fff640 1 -- 172.21.15.52:0/1460549557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 msgr2=0x561f89138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff193fff640 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89138000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.976 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff193fff640 1 -- 172.21.15.52:0/1460549557 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f8904d9b0 con 0x561f89130a00 2024-06-02T23:36:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.975+0000 7ff198ef6640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff17c0a2da0 0x7ff17c0a5260 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff1880bbd70 tx=0x7ff188009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.976+0000 7ff193fff640 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89135500 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff184009fd0 tx=0x7ff1840118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e4253bc50076355 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:23.977 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.976+0000 7ff17affd640 1 -- 172.21.15.52:0/1460549557 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184004030 con 0x561f89130a00 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.976+0000 7ff17affd640 1 -- 172.21.15.52:0/1460549557 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1840054d0 con 0x561f89130a00 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.976+0000 7ff17affd640 1 -- 172.21.15.52:0/1460549557 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184004c40 con 0x561f89130a00 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 -- 172.21.15.52:0/1460549557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 msgr2=0x561f89135500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89135500 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff184009fd0 tx=0x7ff1840118d0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 -- 172.21.15.52:0/1460549557 shutdown_connections 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89135500 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.980 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 --2- 172.21.15.52:0/1460549557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89130430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 -- 172.21.15.52:0/1460549557 >> 172.21.15.52:0/1460549557 conn(0x561f8912b680 msgr2=0x561f8912ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 -- 172.21.15.52:0/1460549557 shutdown_connections 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 -- 172.21.15.52:0/1460549557 wait complete. 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.977+0000 7ff19de62740 1 -- start start 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.978+0000 7ff19de62740 1 Processor -- start 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.978+0000 7ff19de62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130010 0x561f8913a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.978+0000 7ff19de62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.981 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.978+0000 7ff19de62740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89141a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff19de62740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x561f89138b20 con 0x561f89135b70 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff1996f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130010 0x561f8913a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:42470/0 (socket says 172.21.15.52:42470) 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 -- 172.21.15.52:0/139943174 learned_addr learned my addr 172.21.15.52:0/139943174 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561f8901cae0 con 0x561f89130a00 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561f89138ca0 con 0x561f89130010 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff19de62740 10 client.0 _ll_register_callbacks cb 0x561f890559b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff1996f7640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89141a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.982 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 -- 172.21.15.52:0/139943174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130010 msgr2=0x561f8913a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130010 0x561f8913a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 -- 172.21.15.52:0/139943174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 msgr2=0x561f89141a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89141a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff193fff640 1 -- 172.21.15.52:0/139943174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f88da7210 con 0x561f89130a00 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.979+0000 7ff1996f7640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130010 0x561f8913a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff193fff640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913aa00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff184009fa0 tx=0x7ff18400fd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c07d16a1a90d0d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff173fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff173fff640 1 -- 172.21.15.52:0/139943174 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184029020 con 0x561f89130a00 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff1840054d0 con 0x561f89130a00 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff173fff640 1 -- 172.21.15.52:0/139943174 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff184025680 con 0x561f89130a00 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff173fff640 1 -- 172.21.15.52:0/139943174 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff184025880 con 0x561f89130a00 2024-06-02T23:36:23.983 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff19de62740 10 client.30986 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.980+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561f88e73620 con 0x561f89130a00 2024-06-02T23:36:23.984 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff173fff640 1 -- 172.21.15.52:0/139943174 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff1840054d0 con 0x561f89130a00 2024-06-02T23:36:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff173fff640 1 -- 172.21.15.52:0/139943174 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff184004c50 con 0x561f89130a00 2024-06-02T23:36:23.985 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff19de62740 10 client.30986 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff19de62740 10 client.30986 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff19de62740 20 client.30986 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff19de62740 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f89146190 0x561f89148580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff19de62740 4 client.30986 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f890a6f10 con 0x561f89146190 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.981+0000 7ff1996f7640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f89146190 0x561f89148580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff1996f7640 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f89146190 0x561f89148580 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddeaa66838ff07d4 server_cookie=7191550cb289b7d9 in_seq=0 out_seq=0 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff173fff640 10 client.30986 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:23.986 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff173fff640 1 -- 172.21.15.52:0/139943174 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x561f890a6f10 con 0x561f89146190 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff173fff640 10 client.30986 handle_command_reply: tid=0 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff19de62740 1 client.30986 shutdown 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 msgr2=0x561f8913aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff19de62740 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913aa00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff184009fa0 tx=0x7ff18400fd30 comp rx=0 tx=0).stop 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 shutdown_connections 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.982+0000 7ff19de62740 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f89146190 0x561f89148580 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f89141a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913aa00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 1 --2- 172.21.15.52:0/139943174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130010 0x561f8913a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:23.987 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 >> 172.21.15.52:0/139943174 conn(0x561f8912b680 msgr2=0x561f8912a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff173fff640 0 client.30986 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 shutdown_connections 2024-06-02T23:36:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 1 -- 172.21.15.52:0/139943174 wait complete. 2024-06-02T23:36:23.988 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:23.983+0000 7ff19de62740 20 client.30986 trim_cache size 0 max 16384 2024-06-02T23:36:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 Processor -- start 2024-06-02T23:36:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 -- start start 2024-06-02T23:36:24.018 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89126840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89141a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f8912e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941aaf10 con 0x561f89135b70 2024-06-02T23:36:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941aca50 con 0x561f89130010 2024-06-02T23:36:24.019 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.010+0000 7ff19de62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941ab5e0 con 0x561f89130a00 2024-06-02T23:36:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:52836/0 (socket says 172.21.15.52:52836) 2024-06-02T23:36:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 -- 172.21.15.52:0/2746240095 learned_addr learned my addr 172.21.15.52:0/2746240095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:24.020 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff1996f7640 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f8912e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff1996f7640 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89126840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 -- 172.21.15.52:0/2746240095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 msgr2=0x561f89126840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89126840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.021 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 -- 172.21.15.52:0/2746240095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 msgr2=0x561f8912e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f8912e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 -- 172.21.15.52:0/2746240095 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f89130de0 con 0x561f89130a00 2024-06-02T23:36:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff193fff640 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89141a00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff184032040 tx=0x7ff18400f9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f6fc9d1ea9780a9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff17a7fc640 1 -- 172.21.15.52:0/2746240095 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff184029070 con 0x561f89130a00 2024-06-02T23:36:24.022 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.011+0000 7ff17a7fc640 1 -- 172.21.15.52:0/2746240095 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff184009b50 con 0x561f89130a00 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.012+0000 7ff17a7fc640 1 -- 172.21.15.52:0/2746240095 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1840049e0 con 0x561f89130a00 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.012+0000 7ff19de62740 1 -- 172.21.15.52:0/2746240095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 msgr2=0x561f89141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.012+0000 7ff19de62740 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89141a00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff184032040 tx=0x7ff18400f9a0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 -- 172.21.15.52:0/2746240095 shutdown_connections 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89130a00 0x561f89141a00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89135b70 0x561f8912e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.023 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 --2- 172.21.15.52:0/2746240095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130010 0x561f89126840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 -- 172.21.15.52:0/2746240095 >> 172.21.15.52:0/2746240095 conn(0x561f8912b680 msgr2=0x561f89131e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 -- 172.21.15.52:0/2746240095 shutdown_connections 2024-06-02T23:36:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 -- 172.21.15.52:0/2746240095 wait complete. 2024-06-02T23:36:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 -- start start 2024-06-02T23:36:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 Processor -- start 2024-06-02T23:36:24.024 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 0x561f89141710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.013+0000 7ff19de62740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89135b70 0x561f8913e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff19de62740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941ac380 con 0x561f89130010 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff19de62740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941abcb0 con 0x561f89130a00 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff19de62740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff1941a8ef0 con 0x561f89135b70 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 0x561f89141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff193fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 0x561f89141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:42068/0 (socket says 172.21.15.52:42068) 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 -- 172.21.15.52:0/1160881763 learned_addr learned my addr 172.21.15.52:0/1160881763 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:24.025 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff19de62740 10 client.0 _ll_register_callbacks cb 0x7ff18c012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 -- 172.21.15.52:0/1160881763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89135b70 msgr2=0x561f8913e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89135b70 0x561f8913e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 -- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 msgr2=0x561f8913e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 -- 172.21.15.52:0/1160881763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f8913edd0 con 0x561f89130010 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff184009b50 con 0x561f89130010 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff1996f7640 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 0x561f89141710 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x561f89148de0 tx=0x7ff18c012f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a76e5d71e2d609 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff173fff640 10 client.21762 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.014+0000 7ff173fff640 1 -- 172.21.15.52:0/1160881763 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff18c031020 con 0x561f89130010 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.015+0000 7ff173fff640 1 -- 172.21.15.52:0/1160881763 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff18c011dd0 con 0x561f89130010 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.015+0000 7ff173fff640 1 -- 172.21.15.52:0/1160881763 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ff18c004b20 con 0x561f89130010 2024-06-02T23:36:24.026 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.015+0000 7ff19de62740 10 client.21762 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.015+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({fsmap=43}) -- 0x561f891afec0 con 0x561f89130010 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.015+0000 7ff173fff640 1 -- 172.21.15.52:0/1160881763 <== mon.0 v2:172.21.15.52:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7ff18c013040 con 0x561f89130010 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.015+0000 7ff19de62740 10 client.21762 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff19de62740 10 client.21762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff19de62740 20 client.21762 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff19de62740 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f891b06e0 0x561f891b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff19de62740 4 client.21762 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f890a0c20 con 0x561f891b06e0 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff193fff640 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f891b06e0 0x561f891b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff193fff640 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f891b06e0 0x561f891b2b00 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a88594f711ddc59 server_cookie=ece0249f2b8bb812 in_seq=0 out_seq=0 2024-06-02T23:36:24.027 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.016+0000 7ff173fff640 10 client.21762 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff173fff640 1 -- 172.21.15.52:0/1160881763 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x561f890a0c20 con 0x561f891b06e0 2024-06-02T23:36:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff173fff640 10 client.21762 handle_command_reply: tid=0 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 client.21762 shutdown 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 msgr2=0x561f89141710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 0x561f89141710 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x561f89148de0 tx=0x7ff18c012f30 comp rx=0 tx=0).stop 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 shutdown_connections 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x561f891b06e0 0x561f891b2b00 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561f89135b70 0x561f8913e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561f89130a00 0x561f8913e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 --2- 172.21.15.52:0/1160881763 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x561f89130010 0x561f89141710 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 >> 172.21.15.52:0/1160881763 conn(0x561f8912b680 msgr2=0x561f89129f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff173fff640 0 client.21762 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 shutdown_connections 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 1 -- 172.21.15.52:0/1160881763 wait complete. 2024-06-02T23:36:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.066+0000 7ff19de62740 20 client.21762 trim_cache size 0 max 16384 2024-06-02T23:36:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:36:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:36:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:36:24.068 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff17c0a2da0 msgr2=0x7ff17c0a5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff17c0a2da0 0x7ff17c0a5260 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7ff1880bbd70 tx=0x7ff188009260 comp rx=0 tx=0).stop 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 msgr2=0x7ff194199df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 0x7ff194199df0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff1880314e0 tx=0x7ff188003c10 comp rx=0 tx=0).stop 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 shutdown_connections 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff17c0a2da0 0x7ff17c0a5260 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1941057e0 0x7ff1941a1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff194104e20 0x7ff19419a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 --2- 172.21.15.52:0/1604650822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff194103c30 0x7ff194199df0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 >> 172.21.15.52:0/1604650822 conn(0x7ff1940ff1c0 msgr2=0x7ff194108240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 shutdown_connections 2024-06-02T23:36:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:24.067+0000 7ff19b15a640 1 -- 172.21.15.52:0/1604650822 wait complete. 2024-06-02T23:36:24.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84244 6.75 MB/sec execute 572 sec latency 104.553 ms 2024-06-02T23:36:25.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84372 6.74 MB/sec execute 573 sec latency 118.010 ms 2024-06-02T23:36:26.048 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:25 smithi052 ceph-mon[32340]: pgmap v1167: 97 pgs: 97 active+clean; 616 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 253 op/s 2024-06-02T23:36:26.048 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:26.048 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:25 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:25 smithi174 ceph-mon[35994]: pgmap v1167: 97 pgs: 97 active+clean; 616 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 253 op/s 2024-06-02T23:36:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:26.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:25 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:25 smithi123 ceph-mon[35977]: pgmap v1167: 97 pgs: 97 active+clean; 616 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.1 MiB/s wr, 253 op/s 2024-06-02T23:36:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:26.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:25 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:26.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84510 6.75 MB/sec execute 574 sec latency 131.853 ms 2024-06-02T23:36:27.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84633 6.74 MB/sec execute 575 sec latency 169.261 ms 2024-06-02T23:36:28.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:27 smithi174 ceph-mon[35994]: pgmap v1168: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 293 op/s 2024-06-02T23:36:28.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:27 smithi052 ceph-mon[32340]: pgmap v1168: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 293 op/s 2024-06-02T23:36:28.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:27 smithi123 ceph-mon[35977]: pgmap v1168: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 293 op/s 2024-06-02T23:36:28.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84778 6.74 MB/sec execute 576 sec latency 181.964 ms 2024-06-02T23:36:29.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 84971 6.76 MB/sec execute 577 sec latency 88.132 ms 2024-06-02T23:36:30.042 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:29 smithi052 ceph-mon[32340]: pgmap v1169: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-06-02T23:36:30.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:29 smithi174 ceph-mon[35994]: pgmap v1169: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-06-02T23:36:30.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:29 smithi123 ceph-mon[35977]: pgmap v1169: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1008 KiB/s rd, 3.6 MiB/s wr, 231 op/s 2024-06-02T23:36:30.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 85154 6.76 MB/sec execute 578 sec latency 73.978 ms 2024-06-02T23:36:31.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 85318 6.77 MB/sec execute 579 sec latency 88.372 ms 2024-06-02T23:36:32.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: pgmap v1170: 97 pgs: 97 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 4.1 MiB/s wr, 246 op/s 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:36:32.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:31 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: pgmap v1170: 97 pgs: 97 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 4.1 MiB/s wr, 246 op/s 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:36:32.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:31 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: pgmap v1170: 97 pgs: 97 active+clean; 612 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 915 KiB/s rd, 4.1 MiB/s wr, 246 op/s 2024-06-02T23:36:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:36:32.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:36:32.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:36:32.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:31 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:36:32.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 85475 6.76 MB/sec execute 580 sec latency 53.507 ms 2024-06-02T23:36:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:32 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:36:33.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:32 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:36:33.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:32 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:36:33.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:32 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:36:33.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:32 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi052 to 177.2M 2024-06-02T23:36:33.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:32 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi052 to 185844736: error parsing value: Value '185844736' is below minimum 939524096 2024-06-02T23:36:33.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 85665 6.77 MB/sec execute 581 sec latency 60.838 ms 2024-06-02T23:36:34.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:33 smithi174 ceph-mon[35994]: pgmap v1171: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2024-06-02T23:36:34.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:33 smithi052 ceph-mon[32340]: pgmap v1171: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2024-06-02T23:36:34.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:33 smithi123 ceph-mon[35977]: pgmap v1171: 97 pgs: 97 active+clean; 615 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.8 MiB/s wr, 261 op/s 2024-06-02T23:36:34.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 85837 6.77 MB/sec execute 582 sec latency 60.632 ms 2024-06-02T23:36:35.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86012 6.78 MB/sec execute 583 sec latency 62.460 ms 2024-06-02T23:36:35.797 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:36:36.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:35 smithi123 ceph-mon[35977]: pgmap v1172: 97 pgs: 97 active+clean; 616 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-06-02T23:36:36.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:35 smithi174 ceph-mon[35994]: pgmap v1172: 97 pgs: 97 active+clean; 616 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-06-02T23:36:36.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:35 smithi052 ceph-mon[32340]: pgmap v1172: 97 pgs: 97 active+clean; 616 MiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 839 KiB/s rd, 3.3 MiB/s wr, 238 op/s 2024-06-02T23:36:36.194 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:36.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86179 6.78 MB/sec execute 584 sec latency 64.687 ms 2024-06-02T23:36:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.170+0000 7f0a0cee7640 1 -- 172.21.15.52:0/2853634997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 msgr2=0x7f0a08101aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.170+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/2853634997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a08101aa0 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009ca0 tx=0x7f0a0002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 -- 172.21.15.52:0/2853634997 shutdown_connections 2024-06-02T23:36:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/2853634997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100c80 0x7f0a081010e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/2853634997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a08101aa0 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.173 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/2853634997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a08100320 0x7f0a08100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 -- 172.21.15.52:0/2853634997 >> 172.21.15.52:0/2853634997 conn(0x7f0a080fbf20 msgr2=0x7f0a080fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 -- 172.21.15.52:0/2853634997 shutdown_connections 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 -- 172.21.15.52:0/2853634997 wait complete. 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 Processor -- start 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.171+0000 7f0a0cee7640 1 -- start start 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100320 0x7f0a08199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a08100c80 0x7f0a0819a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a081a1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0810f760 con 0x7f0a08101620 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0810f5e0 con 0x7f0a08100c80 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0810f8e0 con 0x7f0a08100320 2024-06-02T23:36:37.174 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a081a1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100320 0x7f0a08199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a081a1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47632/0 (socket says 172.21.15.52:47632) 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a06d9d640 1 -- 172.21.15.52:0/146870093 learned_addr learned my addr 172.21.15.52:0/146870093 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a08100c80 0x7f0a0819a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 -- 172.21.15.52:0/146870093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a08100c80 msgr2=0x7f0a0819a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a08100c80 0x7f0a0819a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 -- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 msgr2=0x7f0a081a1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a081a1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 -- 172.21.15.52:0/146870093 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a081a1920 con 0x7f0a08100320 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a081a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0659c640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100320 0x7f0a08199d10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09f800e9f0 tx=0x7f09f800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f09f77fe640 1 -- 172.21.15.52:0/146870093 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f800ccc0 con 0x7f0a08100320 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a081a1c10 con 0x7f0a08100320 2024-06-02T23:36:37.175 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.172+0000 7f09f77fe640 1 -- 172.21.15.52:0/146870093 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f80108e0 con 0x7f0a08100320 2024-06-02T23:36:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.173+0000 7f09f77fe640 1 -- 172.21.15.52:0/146870093 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8018880 con 0x7f0a08100320 2024-06-02T23:36:37.176 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.173+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a081a2160 con 0x7f0a08100320 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f09f77fe640 1 -- 172.21.15.52:0/146870093 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f09f8010a90 con 0x7f0a08100320 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 Processor -- start 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 -- start start 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cd010 0x55e53b6cd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 0x55e53b6d2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6d2b70 0x55e53b6d5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e53b6304f0 con 0x55e53b6cda00 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e53b5ce870 con 0x55e53b6cd010 2024-06-02T23:36:37.183 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e53b62c4b0 con 0x55e53b6d2b70 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 0x55e53b6d2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 0x55e53b6d2500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47634/0 (socket says 172.21.15.52:47634) 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a0659c640 1 -- 172.21.15.52:0/305208625 learned_addr learned my addr 172.21.15.52:0/305208625 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6d2b70 0x55e53b6d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.175+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cd010 0x55e53b6cd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a0659c640 1 -- 172.21.15.52:0/305208625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6d2b70 msgr2=0x55e53b6d5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a0659c640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6d2b70 0x55e53b6d5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a0659c640 1 -- 172.21.15.52:0/305208625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cd010 msgr2=0x55e53b6cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a0659c640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cd010 0x55e53b6cd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f09f77fe640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09f00a2c30 0x7f09f00a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a0659c640 1 -- 172.21.15.52:0/305208625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53b5ea9b0 con 0x55e53b6cda00 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6d2b70 0x55e53b6d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:37.184 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cd010 0x55e53b6cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09f00a2c30 0x7f09f00a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f09f77fe640 1 -- 172.21.15.52:0/146870093 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09f80cf3f0 con 0x7f0a08100320 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f0a0659c640 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 0x55e53b6d2500 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f09f809c7d0 tx=0x7f09f80d7950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64376e99559ff244 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f09f4ff9640 1 -- 172.21.15.52:0/305208625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8026030 con 0x55e53b6cda00 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f09f4ff9640 1 -- 172.21.15.52:0/305208625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f80d7e50 con 0x55e53b6cda00 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.176+0000 7f09f4ff9640 1 -- 172.21.15.52:0/305208625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8099e70 con 0x55e53b6cda00 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/305208625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 msgr2=0x55e53b6d2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09f00a2c30 0x7f09f00a50f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0a000318d0 tx=0x7f0a00031860 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 0x55e53b6d2500 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f09f809c7d0 tx=0x7f09f80d7950 comp rx=0 tx=0).stop 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/305208625 shutdown_connections 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6d2b70 0x55e53b6d5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cda00 0x55e53b6d2500 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.185 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/305208625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cd010 0x55e53b6cd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/305208625 >> 172.21.15.52:0/305208625 conn(0x55e53b6c8680 msgr2=0x55e53b6c8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/305208625 shutdown_connections 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/305208625 wait complete. 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- start start 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 Processor -- start 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6dea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e53b6d5b20 con 0x55e53b6d2b70 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e53b5b9ae0 con 0x55e53b6cda00 2024-06-02T23:36:37.186 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e53b6d5ca0 con 0x55e53b6cd010 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.177+0000 7f0a0fbf1740 10 client.0 _ll_register_callbacks cb 0x55e53b5f29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6dea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6dea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47650/0 (socket says 172.21.15.52:47650) 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 -- 172.21.15.52:0/1828267864 learned_addr learned my addr 172.21.15.52:0/1828267864 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a0659c640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 -- 172.21.15.52:0/1828267864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 msgr2=0x55e53b6d74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6d74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 -- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 msgr2=0x55e53b6d7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6d7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 -- 172.21.15.52:0/1828267864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53b344210 con 0x55e53b6d2b70 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6dea70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0a000096a0 tx=0x7f0a0002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c09e6b0d3e95eed3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:37.187 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f09edffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f09edffb640 1 -- 172.21.15.52:0/1828267864 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a000043d0 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f09edffb640 1 -- 172.21.15.52:0/1828267864 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f0a00004570 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f80d7e50 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f09edffb640 1 -- 172.21.15.52:0/1828267864 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0a00004770 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a0fbf1740 10 client.21774 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.178+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e53b410620 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f09edffb640 1 -- 172.21.15.52:0/1828267864 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f09f80d7e50 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f09edffb640 1 -- 172.21.15.52:0/1828267864 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f0a00051070 con 0x55e53b6d2b70 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0fbf1740 10 client.21774 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0fbf1740 10 client.21774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:37.188 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0fbf1740 20 client.21774 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b6e34d0 0x55e53b6e5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0fbf1740 4 client.21774 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e53b643f10 con 0x55e53b6e34d0 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0659c640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b6e34d0 0x55e53b6e5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f09edffb640 10 client.21774 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.179+0000 7f0a0659c640 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b6e34d0 0x55e53b6e5910 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94054713fce82b28 server_cookie=7731193d141d4523 in_seq=0 out_seq=0 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f09edffb640 1 -- 172.21.15.52:0/1828267864 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e53b643f10 con 0x55e53b6e34d0 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f09edffb640 10 client.21774 handle_command_reply: tid=0 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 client.21774 shutdown 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 msgr2=0x55e53b6dea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6dea70 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f0a000096a0 tx=0x7f0a0002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 shutdown_connections 2024-06-02T23:36:37.189 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b6e34d0 0x55e53b6e5910 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6dea70 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6d7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/1828267864 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f09edffb640 0 client.21774 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 >> 172.21.15.52:0/1828267864 conn(0x55e53b6c8680 msgr2=0x55e53b6c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 shutdown_connections 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/1828267864 wait complete. 2024-06-02T23:36:37.190 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.180+0000 7f0a0fbf1740 20 client.21774 trim_cache size 0 max 16384 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 Processor -- start 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 -- start start 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081aad00 con 0x55e53b6d2b70 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081ac840 con 0x55e53b6cda00 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081ab3d0 con 0x55e53b6cd010 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.195+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6d74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47660/0 (socket says 172.21.15.52:47660) 2024-06-02T23:36:37.201 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 -- 172.21.15.52:0/268192034 learned_addr learned my addr 172.21.15.52:0/268192034 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 -- 172.21.15.52:0/268192034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 msgr2=0x55e53b6c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 -- 172.21.15.52:0/268192034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 msgr2=0x55e53b6dea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 -- 172.21.15.52:0/268192034 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e53b6c9350 con 0x55e53b6d2b70 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6d74f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0a00043040 tx=0x7f0a00003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4cef83bccdd8e10 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f09effff640 1 -- 172.21.15.52:0/268192034 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a00009890 con 0x55e53b6d2b70 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f09effff640 1 -- 172.21.15.52:0/268192034 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a00062020 con 0x55e53b6d2b70 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f09effff640 1 -- 172.21.15.52:0/268192034 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a000308b0 con 0x55e53b6d2b70 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.196+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/268192034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 msgr2=0x55e53b6d74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6d74f0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f0a00043040 tx=0x7f0a00003e00 comp rx=0 tx=0).stop 2024-06-02T23:36:37.202 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/268192034 shutdown_connections 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6d2b70 0x55e53b6d74f0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6cda00 0x55e53b6dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/268192034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cd010 0x55e53b6c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/268192034 >> 172.21.15.52:0/268192034 conn(0x55e53b6c8680 msgr2=0x55e53b6c8a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/268192034 shutdown_connections 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/268192034 wait complete. 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- start start 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 Processor -- start 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 0x55e53b6db5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cda00 0x55e53b6dbae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6d2b70 0x55e53b74c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081ac170 con 0x55e53b6cd010 2024-06-02T23:36:37.203 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081abaa0 con 0x55e53b6d2b70 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0a081a8ce0 con 0x55e53b6cda00 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0fbf1740 10 client.0 _ll_register_callbacks cb 0x7f0a00009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 0x55e53b6db5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0659c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 0x55e53b6db5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47676/0 (socket says 172.21.15.52:47676) 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a0659c640 1 -- 172.21.15.52:0/3184041142 learned_addr learned my addr 172.21.15.52:0/3184041142 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cda00 0x55e53b6dbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.197+0000 7f0a06d9d640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6d2b70 0x55e53b74c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0659c640 1 -- 172.21.15.52:0/3184041142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cda00 msgr2=0x55e53b6dbae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0659c640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cda00 0x55e53b6dbae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0659c640 1 -- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6d2b70 msgr2=0x55e53b74c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0659c640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6d2b70 0x55e53b74c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0659c640 1 -- 172.21.15.52:0/3184041142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e53b6deb60 con 0x55e53b6cd010 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cda00 0x55e53b6dbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:37.204 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0659c640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 0x55e53b6db5a0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f09f809bcc0 tx=0x7f09f8098000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f9fa225141fda7f server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f09edffb640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f09edffb640 1 -- 172.21.15.52:0/3184041142 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8099e70 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f09edffb640 1 -- 172.21.15.52:0/3184041142 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f09f80ccf70 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a00062020 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f09edffb640 1 -- 172.21.15.52:0/3184041142 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f09f809b870 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 10 client.21780 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e53b74d5e0 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f09edffb640 1 -- 172.21.15.52:0/3184041142 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f0a00062020 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f09edffb640 1 -- 172.21.15.52:0/3184041142 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f09f8098310 con 0x55e53b6cd010 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 10 client.21780 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 10 client.21780 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 20 client.21780 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b74db60 0x55e53b74ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:37.205 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 4 client.21780 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:36:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.198+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e53b63dc20 con 0x55e53b74db60 2024-06-02T23:36:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.199+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b74db60 0x55e53b74ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.199+0000 7f0a05d9b640 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b74db60 0x55e53b74ff80 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=653287f29c2e13a5 server_cookie=52a317d358577900 in_seq=0 out_seq=0 2024-06-02T23:36:37.206 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.199+0000 7f09edffb640 10 client.21780 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.243+0000 7f09edffb640 1 -- 172.21.15.52:0/3184041142 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e53b63dc20 con 0x55e53b74db60 2024-06-02T23:36:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.243+0000 7f09edffb640 10 client.21780 handle_command_reply: tid=0 2024-06-02T23:36:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 client.21780 shutdown 2024-06-02T23:36:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 msgr2=0x55e53b6db5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 0x55e53b6db5a0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f09f809bcc0 tx=0x7f09f8098000 comp rx=0 tx=0).stop 2024-06-02T23:36:37.245 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 shutdown_connections 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e53b74db60 0x55e53b74ff80 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e53b6cda00 0x55e53b6dbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f09edffb640 0 client.21780 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e53b6d2b70 0x55e53b74c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 --2- 172.21.15.52:0/3184041142 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e53b6cd010 0x55e53b6db5a0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 >> 172.21.15.52:0/3184041142 conn(0x55e53b6c8680 msgr2=0x55e53b6d74f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 shutdown_connections 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 1 -- 172.21.15.52:0/3184041142 wait complete. 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0fbf1740 20 client.21780 trim_cache size 0 max 16384 2024-06-02T23:36:37.246 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:36:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:36:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:36:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09f00a2c30 msgr2=0x7f09f00a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09f00a2c30 0x7f09f00a50f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f0a000318d0 tx=0x7f0a00031860 comp rx=0 tx=0).stop 2024-06-02T23:36:37.247 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100320 msgr2=0x7f0a08199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100320 0x7f0a08199d10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f09f800e9f0 tx=0x7f09f800eeb0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 shutdown_connections 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f09f00a2c30 0x7f09f00a50f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f0a08101620 0x7f0a081a1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.244+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a08100c80 0x7f0a0819a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.245+0000 7f0a0cee7640 1 --2- 172.21.15.52:0/146870093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0a08100320 0x7f0a08199d10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.245+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 >> 172.21.15.52:0/146870093 conn(0x7f0a080fbf20 msgr2=0x7f0a080fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.245+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 shutdown_connections 2024-06-02T23:36:37.248 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:37.245+0000 7f0a0cee7640 1 -- 172.21.15.52:0/146870093 wait complete. 2024-06-02T23:36:37.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86288 6.78 MB/sec execute 585 sec latency 190.400 ms 2024-06-02T23:36:37.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:37 smithi052 ceph-mon[32340]: pgmap v1173: 97 pgs: 97 active+clean; 617 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 4.0 MiB/s wr, 286 op/s 2024-06-02T23:36:38.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:37 smithi174 ceph-mon[35994]: pgmap v1173: 97 pgs: 97 active+clean; 617 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 4.0 MiB/s wr, 286 op/s 2024-06-02T23:36:38.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:37 smithi123 ceph-mon[35977]: pgmap v1173: 97 pgs: 97 active+clean; 617 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 949 KiB/s rd, 4.0 MiB/s wr, 286 op/s 2024-06-02T23:36:38.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86487 6.79 MB/sec execute 586 sec latency 127.734 ms 2024-06-02T23:36:39.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86656 6.79 MB/sec execute 587 sec latency 89.073 ms 2024-06-02T23:36:39.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.527+0000 7fa922075700 1 -- 172.21.15.52:0/2540609253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109ee0 msgr2=0x7fa91c10d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.528 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.527+0000 7fa922075700 1 --2- 172.21.15.52:0/2540609253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109ee0 0x7fa91c10d770 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7fa9100099c0 tx=0x7fa910009cd0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 -- 172.21.15.52:0/2540609253 shutdown_connections 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 --2- 172.21.15.52:0/2540609253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10de50 0x7fa91c1159e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 --2- 172.21.15.52:0/2540609253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c109530 0x7fa91c109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 --2- 172.21.15.52:0/2540609253 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109ee0 0x7fa91c10d770 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 -- 172.21.15.52:0/2540609253 >> 172.21.15.52:0/2540609253 conn(0x7fa91c078360 msgr2=0x7fa91c07a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 -- 172.21.15.52:0/2540609253 shutdown_connections 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.528+0000 7fa922075700 1 -- 172.21.15.52:0/2540609253 wait complete. 2024-06-02T23:36:39.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 Processor -- start 2024-06-02T23:36:39.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 -- start start 2024-06-02T23:36:39.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 0x7fa91c19f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c109ee0 0x7fa91c19f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10de50 0x7fa91c1a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.530 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118ce0 con 0x7fa91c109530 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118b60 con 0x7fa91c10de50 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.529+0000 7fa922075700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa91c118e60 con 0x7fa91c109ee0 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 0x7fa91c19f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 0x7fa91c19f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47686/0 (socket says 172.21.15.52:47686) 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 -- 172.21.15.52:0/1350111094 learned_addr learned my addr 172.21.15.52:0/1350111094 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91bfff700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10de50 0x7fa91c1a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91affd700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c109ee0 0x7fa91c19f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 -- 172.21.15.52:0/1350111094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c109ee0 msgr2=0x7fa91c19f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c109ee0 0x7fa91c19f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 -- 172.21.15.52:0/1350111094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10de50 msgr2=0x7fa91c1a3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10de50 0x7fa91c1a3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.530+0000 7fa91b7fe700 1 -- 172.21.15.52:0/1350111094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91c1a4180 con 0x7fa91c109530 2024-06-02T23:36:39.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.531+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 0x7fa91c19f170 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00eb30 tx=0x7fa90c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:39.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.531+0000 7fa918ff9700 1 -- 172.21.15.52:0/1350111094 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00cbd0 con 0x7fa91c109530 2024-06-02T23:36:39.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.531+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91c1a4470 con 0x7fa91c109530 2024-06-02T23:36:39.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.531+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa91c1abcf0 con 0x7fa91c109530 2024-06-02T23:36:39.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.531+0000 7fa918ff9700 1 -- 172.21.15.52:0/1350111094 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90c0044d0 con 0x7fa91c109530 2024-06-02T23:36:39.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.531+0000 7fa918ff9700 1 -- 172.21.15.52:0/1350111094 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c003e40 con 0x7fa91c109530 2024-06-02T23:36:39.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.534+0000 7fa918ff9700 1 -- 172.21.15.52:0/1350111094 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fa90c004020 con 0x7fa91c109530 2024-06-02T23:36:39.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.535+0000 7fa918ff9700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9040a2ca0 0x7fa9040a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.535+0000 7fa918ff9700 1 -- 172.21.15.52:0/1350111094 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa90c0cf1e0 con 0x7fa91c109530 2024-06-02T23:36:39.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.535+0000 7fa91affd700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9040a2ca0 0x7fa9040a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.536+0000 7fa91affd700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9040a2ca0 0x7fa9040a5160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa910005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:39.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.537+0000 7fa92c8f8b80 1 Processor -- start 2024-06-02T23:36:39.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.537+0000 7fa92c8f8b80 1 -- start start 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2cffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2cf99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2cfa340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5593a2c4b810 con 0x5593a2cfff60 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593a2bf3720 con 0x5593a2cf9f20 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2cffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593a2c477d0 con 0x5593a2cff640 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2cffa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47772/0 (socket says 172.21.15.52:47772) 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 -- 172.21.15.52:0/459839139 learned_addr learned my addr 172.21.15.52:0/459839139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:39.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91affd700 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2cf99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91bfff700 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2cfa340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 -- 172.21.15.52:0/459839139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 msgr2=0x5593a2cfa340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2cfa340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 -- 172.21.15.52:0/459839139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 msgr2=0x5593a2cf99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2cf99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.538+0000 7fa91b7fe700 1 -- 172.21.15.52:0/459839139 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593a2be9f50 con 0x5593a2cff640 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.539+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2cffa20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa90c097ed0 tx=0x7fa90c09b680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d583eebcbb512523 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:39.541 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.539+0000 7fa901ffb700 1 -- 172.21.15.52:0/459839139 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c09c170 con 0x5593a2cff640 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.539+0000 7fa901ffb700 1 -- 172.21.15.52:0/459839139 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90c09ba70 con 0x5593a2cff640 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.539+0000 7fa901ffb700 1 -- 172.21.15.52:0/459839139 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00cd70 con 0x5593a2cff640 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/459839139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 msgr2=0x5593a2cffa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2cffa20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa90c097ed0 tx=0x7fa90c09b680 comp rx=0 tx=0).stop 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/459839139 shutdown_connections 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2cfa340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2cffa20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/459839139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2cf99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/459839139 >> 172.21.15.52:0/459839139 conn(0x5593a2c24cc0 msgr2=0x5593a2cf4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.540+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/459839139 shutdown_connections 2024-06-02T23:36:39.542 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.541+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/459839139 wait complete. 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.541+0000 7fa92c8f8b80 1 -- start start 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 Processor -- start 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2d05ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d02fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2d03520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5593a2d069d0 con 0x5593a2cf9f20 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5593a2bebbf0 con 0x5593a2cfff60 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5593a2d06b50 con 0x5593a2cff640 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa92c8f8b80 10 client.0 _ll_register_callbacks cb 0x5593a2be5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2d05ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa91affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d02fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2d05ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:47696/0 (socket says 172.21.15.52:47696) 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.542+0000 7fa91b7fe700 1 -- 172.21.15.52:0/2027472269 learned_addr learned my addr 172.21.15.52:0/2027472269 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91bfff700 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2d03520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91b7fe700 1 -- 172.21.15.52:0/2027472269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 msgr2=0x5593a2d02fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d02fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91b7fe700 1 -- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 msgr2=0x5593a2d03520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2d03520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91b7fe700 1 -- 172.21.15.52:0/2027472269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593a2b9a8d0 con 0x5593a2cf9f20 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.543+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2d05ce0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0ccd60 tx=0x7fa90c0098f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc869993ff92695 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:39.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.544+0000 7fa8faffd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:39.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.544+0000 7fa8faffd700 1 -- 172.21.15.52:0/2027472269 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0b8070 con 0x5593a2cf9f20 2024-06-02T23:36:39.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.544+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c09ba70 con 0x5593a2cf9f20 2024-06-02T23:36:39.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.544+0000 7fa8faffd700 1 -- 172.21.15.52:0/2027472269 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa90c003680 con 0x5593a2cf9f20 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.545+0000 7fa8faffd700 1 -- 172.21.15.52:0/2027472269 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa90c09fe20 con 0x5593a2cf9f20 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.545+0000 7fa92c8f8b80 10 client.21792 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.545+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5593a2d11310 con 0x5593a2cf9f20 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.545+0000 7fa8faffd700 1 -- 172.21.15.52:0/2027472269 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa90c09ba70 con 0x5593a2cf9f20 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa8faffd700 1 -- 172.21.15.52:0/2027472269 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa90c0a4250 con 0x5593a2cf9f20 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa92c8f8b80 10 client.21792 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa92c8f8b80 10 client.21792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa92c8f8b80 20 client.21792 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d138d0 0x5593a2d15d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa92c8f8b80 4 client.21792 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.546+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593a2c58d10 con 0x5593a2d138d0 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.547+0000 7fa91affd700 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d138d0 0x5593a2d15d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.548 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.547+0000 7fa91affd700 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d138d0 0x5593a2d15d50 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418176192edc3804 server_cookie=78520c99c42c362e in_seq=0 out_seq=0 2024-06-02T23:36:39.549 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.547+0000 7fa8faffd700 10 client.21792 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.548+0000 7fa8faffd700 1 -- 172.21.15.52:0/2027472269 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5593a2c58d10 con 0x5593a2d138d0 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.548+0000 7fa8faffd700 10 client.21792 handle_command_reply: tid=0 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.548+0000 7fa92c8f8b80 1 client.21792 shutdown 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.548+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 msgr2=0x5593a2d05ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.548+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2d05ce0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fa90c0ccd60 tx=0x7fa90c0098f0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 shutdown_connections 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d138d0 0x5593a2d15d50 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2d03520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2d05ce0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/2027472269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d02fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.550 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 >> 172.21.15.52:0/2027472269 conn(0x5593a2c24cc0 msgr2=0x5593a2cf89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa8faffd700 0 client.21792 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 shutdown_connections 2024-06-02T23:36:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/2027472269 wait complete. 2024-06-02T23:36:39.551 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.549+0000 7fa92c8f8b80 20 client.21792 trim_cache size 0 max 16384 2024-06-02T23:36:39.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.583+0000 7fa92c8f8b80 1 Processor -- start 2024-06-02T23:36:39.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.583+0000 7fa92c8f8b80 1 -- start start 2024-06-02T23:36:39.584 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2cf99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d00c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2cee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800d370 con 0x5593a2cf9f20 2024-06-02T23:36:39.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800eeb0 con 0x5593a2cfff60 2024-06-02T23:36:39.585 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800da40 con 0x5593a2cff640 2024-06-02T23:36:39.586 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d00c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2cf99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d00c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47780/0 (socket says 172.21.15.52:47780) 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 -- 172.21.15.52:0/185995076 learned_addr learned my addr 172.21.15.52:0/185995076 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2cee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 -- 172.21.15.52:0/185995076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 msgr2=0x5593a2cee510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2cee510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 -- 172.21.15.52:0/185995076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 msgr2=0x5593a2cf99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2cf99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.584+0000 7fa91affd700 1 -- 172.21.15.52:0/185995076 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593a2cfd790 con 0x5593a2cff640 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.585+0000 7fa91affd700 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d00c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa91c06af50 tx=0x7fa91001a070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=303953dd467d4b4b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.585+0000 7fa9017fa700 1 -- 172.21.15.52:0/185995076 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9100056d0 con 0x5593a2cff640 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.585+0000 7fa9017fa700 1 -- 172.21.15.52:0/185995076 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa910004e90 con 0x5593a2cff640 2024-06-02T23:36:39.587 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.585+0000 7fa9017fa700 1 -- 172.21.15.52:0/185995076 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91003b990 con 0x5593a2cff640 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/185995076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 msgr2=0x5593a2d00c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d00c90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa91c06af50 tx=0x7fa91001a070 comp rx=0 tx=0).stop 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/185995076 shutdown_connections 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cfff60 0x5593a2cee510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cf9f20 0x5593a2cf99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/185995076 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d00c90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/185995076 >> 172.21.15.52:0/185995076 conn(0x5593a2c24cc0 msgr2=0x5593a2cede00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/185995076 shutdown_connections 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/185995076 wait complete. 2024-06-02T23:36:39.588 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.586+0000 7fa92c8f8b80 1 -- start start 2024-06-02T23:36:39.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 Processor -- start 2024-06-02T23:36:39.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2d093f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d09930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2d7c630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2d093f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.590 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa91affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d09930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa91b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2d093f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36510/0 (socket says 172.21.15.52:36510) 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa91affd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d09930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:47782/0 (socket says 172.21.15.52:47782) 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800e7e0 con 0x5593a2cfff60 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800e110 con 0x5593a2cf9f20 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa92c8f8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa90800b310 con 0x5593a2cff640 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.587+0000 7fa91bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2d7c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa92c8f8b80 10 client.0 _ll_register_callbacks cb 0x5593a2be5920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91b7fe700 1 -- 172.21.15.52:0/3900816114 learned_addr learned my addr 172.21.15.52:0/3900816114 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91affd700 1 -- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 msgr2=0x5593a2d093f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91affd700 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2d093f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91affd700 1 -- 172.21.15.52:0/3900816114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 msgr2=0x5593a2d7c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91affd700 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2d7c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91affd700 1 -- 172.21.15.52:0/3900816114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593a2d7cb70 con 0x5593a2cff640 2024-06-02T23:36:39.591 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2d093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa91affd700 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d09930 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa91000b580 tx=0x7fa91000b6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=429957ee38537fb5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa8faffd700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa8faffd700 1 -- 172.21.15.52:0/3900816114 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91000b990 con 0x5593a2cff640 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.588+0000 7fa8faffd700 1 -- 172.21.15.52:0/3900816114 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fa91000bb30 con 0x5593a2cff640 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.589+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5593a2c53290 con 0x5593a2cff640 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.589+0000 7fa8faffd700 1 -- 172.21.15.52:0/3900816114 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fa91000bd30 con 0x5593a2cff640 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.589+0000 7fa92c8f8b80 10 client.30598 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.589+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5593a2d7de80 con 0x5593a2cff640 2024-06-02T23:36:39.592 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.589+0000 7fa8faffd700 1 -- 172.21.15.52:0/3900816114 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fa91003bac0 con 0x5593a2cff640 2024-06-02T23:36:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa8faffd700 1 -- 172.21.15.52:0/3900816114 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fa9100235d0 con 0x5593a2cff640 2024-06-02T23:36:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa92c8f8b80 10 client.30598 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa92c8f8b80 10 client.30598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa92c8f8b80 20 client.30598 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d7e3d0 0x5593a2d807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:39.593 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa92c8f8b80 4 client.30598 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.590+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5593a2c53290 con 0x5593a2d7e3d0 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.591+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d7e3d0 0x5593a2d807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.591+0000 7fa91b7fe700 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d7e3d0 0x5593a2d807f0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4856387d07fd19d server_cookie=fcbd198d52c0a0c9 in_seq=0 out_seq=0 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.591+0000 7fa8faffd700 10 client.30598 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.592+0000 7fa8faffd700 1 -- 172.21.15.52:0/3900816114 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5593a2c53290 con 0x5593a2d7e3d0 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.592+0000 7fa8faffd700 10 client.30598 handle_command_reply: tid=0 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.592+0000 7fa92c8f8b80 1 client.30598 shutdown 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 msgr2=0x5593a2d09930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.594 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d09930 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa91000b580 tx=0x7fa91000b6c0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 shutdown_connections 2024-06-02T23:36:39.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5593a2d7e3d0 0x5593a2d807f0 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5593a2cfff60 0x5593a2d7c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5593a2cf9f20 0x5593a2d093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 --2- 172.21.15.52:0/3900816114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5593a2cff640 0x5593a2d09930 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.595 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 >> 172.21.15.52:0/3900816114 conn(0x5593a2c24cc0 msgr2=0x5593a2d00bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa8faffd700 0 client.30598 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 shutdown_connections 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 1 -- 172.21.15.52:0/3900816114 wait complete. 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.593+0000 7fa92c8f8b80 20 client.30598 trim_cache size 0 max 16384 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:36:39.596 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9040a2ca0 msgr2=0x7fa9040a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9040a2ca0 0x7fa9040a5160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa910005fb0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 msgr2=0x7fa91c19f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 0x7fa91c19f170 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fa90c00eb30 tx=0x7fa90c00ee40 comp rx=0 tx=0).stop 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 shutdown_connections 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fa9040a2ca0 0x7fa9040a5160 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa91c10de50 0x7fa91c1a3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.597 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fa91c109530 0x7fa91c19f170 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 --2- 172.21.15.52:0/1350111094 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa91c109ee0 0x7fa91c19f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.594+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 >> 172.21.15.52:0/1350111094 conn(0x7fa91c078360 msgr2=0x7fa91c07a750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.595+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 shutdown_connections 2024-06-02T23:36:39.598 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:39.595+0000 7fa922075700 1 -- 172.21.15.52:0/1350111094 wait complete. 2024-06-02T23:36:40.084 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:39 smithi052 ceph-mon[32340]: pgmap v1174: 97 pgs: 97 active+clean; 618 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-02T23:36:40.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:39 smithi174 ceph-mon[35994]: pgmap v1174: 97 pgs: 97 active+clean; 618 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-02T23:36:40.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:39 smithi123 ceph-mon[35977]: pgmap v1174: 97 pgs: 97 active+clean; 618 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 3.6 MiB/s wr, 261 op/s 2024-06-02T23:36:40.412 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:36:40.413 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:36:40.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86817 6.80 MB/sec execute 588 sec latency 128.454 ms 2024-06-02T23:36:40.784 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:41.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 86996 6.81 MB/sec execute 589 sec latency 76.609 ms 2024-06-02T23:36:42.077 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:41 smithi052 ceph-mon[32340]: pgmap v1175: 97 pgs: 97 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-06-02T23:36:42.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:41 smithi174 ceph-mon[35994]: pgmap v1175: 97 pgs: 97 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-06-02T23:36:42.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:41 smithi123 ceph-mon[35977]: pgmap v1175: 97 pgs: 97 active+clean; 623 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 971 KiB/s rd, 4.3 MiB/s wr, 282 op/s 2024-06-02T23:36:42.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 87198 6.82 MB/sec execute 590 sec latency 59.675 ms 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 -- 172.21.15.52:0/410519870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 msgr2=0x7fc29c1088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/410519870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 0x7fc29c1088e0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009990 tx=0x7fc28c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 -- 172.21.15.52:0/410519870 shutdown_connections 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/410519870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06ab30 0x7fc29c10f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/410519870 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 0x7fc29c1088e0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/410519870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06a170 0x7fc29c06a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 -- 172.21.15.52:0/410519870 >> 172.21.15.52:0/410519870 conn(0x7fc29c078340 msgr2=0x7fc29c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 -- 172.21.15.52:0/410519870 shutdown_connections 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.747+0000 7fc2a0c21700 1 -- 172.21.15.52:0/410519870 wait complete. 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.748+0000 7fc2a0c21700 1 Processor -- start 2024-06-02T23:36:42.881 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.748+0000 7fc2a0c21700 1 -- start start 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc2a0c21700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06a170 0x7fc29c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc2a0c21700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06ab30 0x7fc29c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc2a0c21700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 0x7fc29c198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc2a0c21700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c112660 con 0x7fc29c108500 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc2a0c21700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1124e0 con 0x7fc29c06ab30 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc2a0c21700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc29c1127e0 con 0x7fc29c06a170 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc299d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06ab30 0x7fc29c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc299d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06ab30 0x7fc29c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:36534/0 (socket says 172.21.15.52:36534) 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc299d9b700 1 -- 172.21.15.52:0/4044158036 learned_addr learned my addr 172.21.15.52:0/4044158036 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc29ad9d700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 0x7fc29c198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.749+0000 7fc29a59c700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06a170 0x7fc29c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc29a59c700 1 -- 172.21.15.52:0/4044158036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06ab30 msgr2=0x7fc29c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc29a59c700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06ab30 0x7fc29c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:42.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc29a59c700 1 -- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 msgr2=0x7fc29c198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc29a59c700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 0x7fc29c198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc29a59c700 1 -- 172.21.15.52:0/4044158036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29c198fc0 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc29a59c700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06a170 0x7fc29c19e6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc28c015000 tx=0x7fc28c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c026070 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fc28c0218c0 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc28c017650 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.750+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29c199250 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.751+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29c199670 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.752+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29c10a9e0 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.753+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fc28c029070 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.754+0000 7fc28b7fe700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2840a2ca0 0x7fc2840a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.754+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fc28c0cfe60 con 0x7fc29c06a170 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.754+0000 7fc299d9b700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2840a2ca0 0x7fc2840a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:42.883 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.755+0000 7fc299d9b700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2840a2ca0 0x7fc2840a5160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0788d0 tx=0x7fc290007480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:42.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:42.759+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fc28c013070 con 0x7fc29c06a170 2024-06-02T23:36:43.227 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.227+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc29c069180 con 0x7fc29c06a170 2024-06-02T23:36:43.228 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.227+0000 7fc28b7fe700 1 -- 172.21.15.52:0/4044158036 <== mon.1 v2:172.21.15.174:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7fc28c09ce20 con 0x7fc29c06a170 2024-06-02T23:36:43.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:36:43.228 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:36:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2840a2ca0 msgr2=0x7fc2840a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2840a2ca0 0x7fc2840a5160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fc29c0788d0 tx=0x7fc290007480 comp rx=0 tx=0).stop 2024-06-02T23:36:43.232 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06a170 msgr2=0x7fc29c19e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06a170 0x7fc29c19e6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc28c015000 tx=0x7fc28c00f690 comp rx=0 tx=0).stop 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 shutdown_connections 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fc2840a2ca0 0x7fc2840a5160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fc29c108500 0x7fc29c198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc29c06a170 0x7fc29c19e6a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 --2- 172.21.15.52:0/4044158036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc29c06ab30 0x7fc29c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.231+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 >> 172.21.15.52:0/4044158036 conn(0x7fc29c078340 msgr2=0x7fc29c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.232+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 shutdown_connections 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:43.232+0000 7fc2a0c21700 1 -- 172.21.15.52:0/4044158036 wait complete. 2024-06-02T23:36:43.233 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:36:43.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 87331 6.82 MB/sec execute 591 sec latency 86.129 ms 2024-06-02T23:36:43.767 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.9 2024-06-02T23:36:43.767 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:36:43.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:43 smithi052 ceph-mon[32340]: pgmap v1176: 97 pgs: 97 active+clean; 626 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-06-02T23:36:43.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:43 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4044158036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:36:43.985 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:36:44.135 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:43 smithi174 ceph-mon[35994]: pgmap v1176: 97 pgs: 97 active+clean; 626 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-06-02T23:36:44.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:43 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4044158036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:36:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:43 smithi123 ceph-mon[35977]: pgmap v1176: 97 pgs: 97 active+clean; 626 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.5 MiB/s wr, 300 op/s 2024-06-02T23:36:44.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:43 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4044158036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:36:44.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 87467 6.82 MB/sec execute 592 sec latency 69.236 ms 2024-06-02T23:36:45.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 87628 6.82 MB/sec execute 593 sec latency 61.475 ms 2024-06-02T23:36:45.721 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:46.087 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:45 smithi052 ceph-mon[32340]: pgmap v1177: 97 pgs: 97 active+clean; 627 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-06-02T23:36:46.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:45 smithi174 ceph-mon[35994]: pgmap v1177: 97 pgs: 97 active+clean; 627 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-06-02T23:36:46.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:45 smithi123 ceph-mon[35977]: pgmap v1177: 97 pgs: 97 active+clean; 627 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1000 KiB/s rd, 4.0 MiB/s wr, 273 op/s 2024-06-02T23:36:46.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 87782 6.82 MB/sec execute 594 sec latency 78.057 ms 2024-06-02T23:36:47.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 87940 6.82 MB/sec execute 595 sec latency 83.966 ms 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.393+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/1981220593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8101390 msgr2=0x7fd8c8101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.393+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/1981220593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8101390 0x7fd8c8101810 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc009990 tx=0x7fd8bc009ca0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.393+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/1981220593 shutdown_connections 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.394+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/1981220593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.394+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/1981220593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8107400 0x7fd8c81077e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.394+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/1981220593 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8101390 0x7fd8c8101810 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.394+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/1981220593 >> 172.21.15.52:0/1981220593 conn(0x7fd8c8078340 msgr2=0x7fd8c8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.394+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/1981220593 shutdown_connections 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.394+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/1981220593 wait complete. 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 Processor -- start 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 -- start start 2024-06-02T23:36:47.487 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8101390 0x7fd8c819e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8107400 0x7fd8c8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8111520 con 0x7fd8c8107400 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81113a0 con 0x7fd8c8101d50 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.395+0000 7fd8cd4e5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81116a0 con 0x7fd8c8101390 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8107400 0x7fd8c8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58504/0 (socket says 172.21.15.52:58504) 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 -- 172.21.15.52:0/2532140841 learned_addr learned my addr 172.21.15.52:0/2532140841 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c6ffd700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8101390 0x7fd8c819e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 -- 172.21.15.52:0/2532140841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8101390 msgr2=0x7fd8c819e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8101390 0x7fd8c819e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 -- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8107400 msgr2=0x7fd8c8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:47.488 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8107400 0x7fd8c8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 -- 172.21.15.52:0/2532140841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c81990a0 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.396+0000 7fd8c67fc700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c819ecc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005850 tx=0x7fd8bc005ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.397+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc02b070 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.397+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd8bc01ed40 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.397+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c8199330 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.397+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc026650 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.397+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c8102fa0 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.398+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c8102ab0 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.399+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd8bc02e070 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.400+0000 7fd8affff700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd8b00a2ca0 0x7fd8b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.400+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd8bc0d41b0 con 0x7fd8c8101d50 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.400+0000 7fd8c6ffd700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd8b00a2ca0 0x7fd8b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.401+0000 7fd8c6ffd700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd8b00a2ca0 0x7fd8b00a5160 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8199a80 tx=0x7fd8b8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:47.489 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.405+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd8bc062850 con 0x7fd8c8101d50 2024-06-02T23:36:47.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.854+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8c8069180 con 0x7fd8c8101d50 2024-06-02T23:36:47.855 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.855+0000 7fd8affff700 1 -- 172.21.15.52:0/2532140841 <== mon.2 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7fd8c8069180 con 0x7fd8c8101d50 2024-06-02T23:36:47.856 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:36:47.856 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:36:47.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd8b00a2ca0 msgr2=0x7fd8b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd8b00a2ca0 0x7fd8b00a5160 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd8c8199a80 tx=0x7fd8b8009260 comp rx=0 tx=0).stop 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 msgr2=0x7fd8c819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c819ecc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc005850 tx=0x7fd8bc005ef0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 shutdown_connections 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd8b00a2ca0 0x7fd8b00a5160 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd8c8107400 0x7fd8c8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8101390 0x7fd8c819e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 --2- 172.21.15.52:0/2532140841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd8c8101d50 0x7fd8c819ecc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:47.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 >> 172.21.15.52:0/2532140841 conn(0x7fd8c8078340 msgr2=0x7fd8c81006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:47.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.858+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 shutdown_connections 2024-06-02T23:36:47.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:47.859+0000 7fd8cd4e5700 1 -- 172.21.15.52:0/2532140841 wait complete. 2024-06-02T23:36:47.861 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:36:47.992 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:47 smithi052 ceph-mon[32340]: pgmap v1178: 97 pgs: 97 active+clean; 627 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 292 op/s 2024-06-02T23:36:48.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:47 smithi174 ceph-mon[35994]: pgmap v1178: 97 pgs: 97 active+clean; 627 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 292 op/s 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.149+0000 7f431b94e700 1 -- 172.21.15.52:0/3383313731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314100b30 msgr2=0x7f4314100fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.149+0000 7f431b94e700 1 --2- 172.21.15.52:0/3383313731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314100b30 0x7f4314100fb0 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f4304009990 tx=0x7f4304009ca0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 -- 172.21.15.52:0/3383313731 shutdown_connections 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 --2- 172.21.15.52:0/3383313731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43141014f0 0x7f431410d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 --2- 172.21.15.52:0/3383313731 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314106be0 0x7f4314106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 --2- 172.21.15.52:0/3383313731 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314100b30 0x7f4314100fb0 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.150 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 -- 172.21.15.52:0/3383313731 >> 172.21.15.52:0/3383313731 conn(0x7f4314077990 msgr2=0x7f43140fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:48.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 -- 172.21.15.52:0/3383313731 shutdown_connections 2024-06-02T23:36:48.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.150+0000 7f431b94e700 1 -- 172.21.15.52:0/3383313731 wait complete. 2024-06-02T23:36:48.151 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.151+0000 7f431b94e700 1 Processor -- start 2024-06-02T23:36:48.152 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.151+0000 7f431b94e700 1 -- start start 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f431b94e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314100b30 0x7f431410f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f431b94e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43141014f0 0x7f431410f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f431b94e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314106be0 0x7f431410fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f431b94e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43141126e0 con 0x7f4314106be0 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f431b94e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4314112560 con 0x7f43141014f0 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f431b94e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4314112860 con 0x7f4314100b30 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f4319eeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314106be0 0x7f431410fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f4319eeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314106be0 0x7f431410fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37610/0 (socket says 172.21.15.52:37610) 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f4319eeb700 1 -- 172.21.15.52:0/1046847683 learned_addr learned my addr 172.21.15.52:0/1046847683 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:48.153 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f43196ea700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314100b30 0x7f431410f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f4318ee9700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43141014f0 0x7f431410f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f43196ea700 1 -- 172.21.15.52:0/1046847683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43141014f0 msgr2=0x7f431410f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f43196ea700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43141014f0 0x7f431410f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f43196ea700 1 -- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314106be0 msgr2=0x7f431410fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f43196ea700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314106be0 0x7f431410fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.152+0000 7f43196ea700 1 -- 172.21.15.52:0/1046847683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43141a7060 con 0x7f4314100b30 2024-06-02T23:36:48.154 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.153+0000 7f43196ea700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314100b30 0x7f431410f250 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f431000e9f0 tx=0x7f431000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:48.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.153+0000 7f430a7fc700 1 -- 172.21.15.52:0/1046847683 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431000c9c0 con 0x7f4314100b30 2024-06-02T23:36:48.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.153+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43141a7350 con 0x7f4314100b30 2024-06-02T23:36:48.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.153+0000 7f430a7fc700 1 -- 172.21.15.52:0/1046847683 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f431000cb60 con 0x7f4314100b30 2024-06-02T23:36:48.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.153+0000 7f430a7fc700 1 -- 172.21.15.52:0/1046847683 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310018680 con 0x7f4314100b30 2024-06-02T23:36:48.155 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.153+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43141a78e0 con 0x7f4314100b30 2024-06-02T23:36:48.156 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.156+0000 7f430a7fc700 1 -- 172.21.15.52:0/1046847683 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4310010430 con 0x7f4314100b30 2024-06-02T23:36:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.156+0000 7f430a7fc700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43000a2d20 0x7f43000a51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.157+0000 7f430a7fc700 1 -- 172.21.15.52:0/1046847683 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f43100cf030 con 0x7f4314100b30 2024-06-02T23:36:48.157 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.157+0000 7f4318ee9700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43000a2d20 0x7f43000a51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.158 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.157+0000 7f4318ee9700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43000a2d20 0x7f43000a51e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4314106290 tx=0x7f4304005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.158+0000 7f43261d1b80 1 Processor -- start 2024-06-02T23:36:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.158+0000 7f43261d1b80 1 -- start start 2024-06-02T23:36:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744760a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x55574475a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x55574475b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.159 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43196ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744760a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43261d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5557446ac810 con 0x55574475af20 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43261d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555744654720 con 0x555744760640 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43196ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744760a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58534/0 (socket says 172.21.15.52:58534) 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43261d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5557446a87d0 con 0x555744760f60 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4319eeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x55574475b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f43196ea700 1 -- 172.21.15.52:0/246690855 learned_addr learned my addr 172.21.15.52:0/246690855 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:48.160 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4318ee9700 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x55574475a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4319eeb700 1 -- 172.21.15.52:0/246690855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 msgr2=0x55574475a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.161 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4319eeb700 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x55574475a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4319eeb700 1 -- 172.21.15.52:0/246690855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 msgr2=0x555744760a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4319eeb700 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744760a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.159+0000 7f4319eeb700 1 -- 172.21.15.52:0/246690855 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55574464af50 con 0x55574475af20 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.160+0000 7f43196ea700 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744760a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.160+0000 7f4319eeb700 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x55574475b340 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f430c009c10 tx=0x7f430c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2291fe2ab343de8b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.160+0000 7f42ff7fe700 1 -- 172.21.15.52:0/246690855 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c019030 con 0x55574475af20 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.160+0000 7f42ff7fe700 1 -- 172.21.15.52:0/246690855 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f430c0092e0 con 0x55574475af20 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.160+0000 7f42ff7fe700 1 -- 172.21.15.52:0/246690855 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430c007500 con 0x55574475af20 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 -- 172.21.15.52:0/246690855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 msgr2=0x55574475b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x55574475b340 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f430c009c10 tx=0x7f430c009920 comp rx=0 tx=0).stop 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 -- 172.21.15.52:0/246690855 shutdown_connections 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x55574475b340 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.162 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744760a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 --2- 172.21.15.52:0/246690855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x55574475a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 -- 172.21.15.52:0/246690855 >> 172.21.15.52:0/246690855 conn(0x555744685cc0 msgr2=0x555744755850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 -- 172.21.15.52:0/246690855 shutdown_connections 2024-06-02T23:36:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 -- 172.21.15.52:0/246690855 wait complete. 2024-06-02T23:36:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 -- start start 2024-06-02T23:36:48.163 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.161+0000 7f43261d1b80 1 Processor -- start 2024-06-02T23:36:48.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x555744766d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744764020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x555744764560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5557447679d0 con 0x55574475af20 2024-06-02T23:36:48.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55574464cbf0 con 0x555744760640 2024-06-02T23:36:48.166 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555744767b50 con 0x555744760f60 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744764020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43261d1b80 10 client.0 _ll_register_callbacks cb 0x555744646920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744764020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58544/0 (socket says 172.21.15.52:58544) 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 -- 172.21.15.52:0/2998678987 learned_addr learned my addr 172.21.15.52:0/2998678987 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43196ea700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x555744766d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4319eeb700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x555744764560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 -- 172.21.15.52:0/2998678987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 msgr2=0x555744764560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x555744764560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 -- 172.21.15.52:0/2998678987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 msgr2=0x555744766d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x555744766d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f4318ee9700 1 -- 172.21.15.52:0/2998678987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557445fb8d0 con 0x555744760640 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.162+0000 7f43196ea700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x555744766d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:48.167 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f4318ee9700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744764020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4304009990 tx=0x7f4304009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30f9267fa6b914e3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c0092e0 con 0x555744760640 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f42fcff9700 10 client.31043 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f42fcff9700 1 -- 172.21.15.52:0/2998678987 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400be40 con 0x555744760640 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f42fcff9700 1 -- 172.21.15.52:0/2998678987 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4304031460 con 0x555744760640 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f42fcff9700 1 -- 172.21.15.52:0/2998678987 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f43040316a0 con 0x555744760640 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 10 client.31043 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=43}) -- 0x5557447720e0 con 0x555744760640 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f42fcff9700 1 -- 172.21.15.52:0/2998678987 <== mon.2 v2:172.21.15.123:3300/0 4 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4304031840 con 0x555744760640 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 10 client.31043 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 10 client.31043 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 20 client.31043 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.163+0000 7f43261d1b80 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x555744776a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.164+0000 7f43261d1b80 4 client.31043 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:48.168 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.164+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557446b9d10 con 0x555744774610 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.164+0000 7f43196ea700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x555744776a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.164+0000 7f43196ea700 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x555744776a00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e6ae745f73fe40dd server_cookie=a2281ad765567aab in_seq=0 out_seq=0 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.164+0000 7f42fcff9700 10 client.31043 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f42fcff9700 1 -- 172.21.15.52:0/2998678987 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5557446b9d10 con 0x555744774610 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f42fcff9700 10 client.31043 handle_command_reply: tid=0 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 client.31043 shutdown 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 msgr2=0x555744764020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744764020 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f4304009990 tx=0x7f4304009310 comp rx=0 tx=0).stop 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 shutdown_connections 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760f60 0x555744764560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55574475af20 0x555744766d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x555744776a00 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f43261d1b80 1 --2- 172.21.15.52:0/2998678987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x555744764020 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.169 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.165+0000 7f42fcff9700 0 client.31043 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:36:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.166+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 >> 172.21.15.52:0/2998678987 conn(0x555744685cc0 msgr2=0x5557447599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.166+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 shutdown_connections 2024-06-02T23:36:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.166+0000 7f43261d1b80 1 -- 172.21.15.52:0/2998678987 wait complete. 2024-06-02T23:36:48.170 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.166+0000 7f43261d1b80 20 client.31043 trim_cache size 0 max 16384 2024-06-02T23:36:48.191 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 Processor -- start 2024-06-02T23:36:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 -- start start 2024-06-02T23:36:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 0x55574475a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x55574475e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.192 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x555744762f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43141aa8b0 con 0x555744760f60 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ac3f0 con 0x555744760640 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.191+0000 7f43261d1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f43141aaf80 con 0x55574475af20 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f43196ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 0x55574475a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x55574475e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4319eeb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x555744762f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f43196ea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 0x55574475a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53614/0 (socket says 172.21.15.52:53614) 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x55574475e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58554/0 (socket says 172.21.15.52:58554) 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f43196ea700 1 -- 172.21.15.52:0/3908317839 learned_addr learned my addr 172.21.15.52:0/3908317839 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 -- 172.21.15.52:0/3908317839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 msgr2=0x55574475a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 0x55574475a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 -- 172.21.15.52:0/3908317839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 msgr2=0x555744762f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.193 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x555744762f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 -- 172.21.15.52:0/3908317839 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555744756500 con 0x555744760640 2024-06-02T23:36:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f43196ea700 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 0x55574475a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4319eeb700 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x555744762f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:48.194 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f4318ee9700 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x55574475e7d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f43040098e0 tx=0x7f4304029000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d2ba12f62199e75 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.192+0000 7f42feffd700 1 -- 172.21.15.52:0/3908317839 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400bd40 con 0x555744760640 2024-06-02T23:36:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.193+0000 7f42feffd700 1 -- 172.21.15.52:0/3908317839 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f430c0092e0 con 0x555744760640 2024-06-02T23:36:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.193+0000 7f42feffd700 1 -- 172.21.15.52:0/3908317839 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f430400f460 con 0x555744760640 2024-06-02T23:36:48.196 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.193+0000 7f43261d1b80 1 -- 172.21.15.52:0/3908317839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 msgr2=0x55574475e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.193+0000 7f43261d1b80 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x55574475e7d0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f43040098e0 tx=0x7f4304029000 comp rx=0 tx=0).stop 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.193+0000 7f43261d1b80 1 -- 172.21.15.52:0/3908317839 shutdown_connections 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.193+0000 7f43261d1b80 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x555744762f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55574475af20 0x55574475a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 --2- 172.21.15.52:0/3908317839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555744760640 0x55574475e7d0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 -- 172.21.15.52:0/3908317839 >> 172.21.15.52:0/3908317839 conn(0x555744685cc0 msgr2=0x55574475e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 -- 172.21.15.52:0/3908317839 shutdown_connections 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 -- 172.21.15.52:0/3908317839 wait complete. 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 -- start start 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.194+0000 7f43261d1b80 1 Processor -- start 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 0x55574476d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760640 0x55574477bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.197 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x55574477c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 0x55574476d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 0x55574476d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58556/0 (socket says 172.21.15.52:58556) 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 -- 172.21.15.52:0/1383218202 learned_addr learned my addr 172.21.15.52:0/1383218202 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f43141abd20 con 0x555744760f60 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43141ab650 con 0x55574475af20 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f431419c770 con 0x555744760640 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f4318ee9700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760640 0x55574477bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f4319eeb700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x55574477c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43261d1b80 10 client.0 _ll_register_callbacks cb 0x555744646920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 -- 172.21.15.52:0/1383218202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760640 msgr2=0x55574477bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760640 0x55574477bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 -- 172.21.15.52:0/1383218202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 msgr2=0x55574477c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x55574477c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.198 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.195+0000 7f43196ea700 1 -- 172.21.15.52:0/1383218202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55574477c8f0 con 0x55574475af20 2024-06-02T23:36:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f43196ea700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 0x55574476d5b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4310009800 tx=0x7f4310009790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed997bfab8d2cbc3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f42fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:36:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f42fcff9700 1 -- 172.21.15.52:0/1383218202 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43100b9060 con 0x55574475af20 2024-06-02T23:36:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f42fcff9700 1 -- 172.21.15.52:0/1383218202 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4310099b60 con 0x55574475af20 2024-06-02T23:36:48.199 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f430c0092e0 con 0x55574475af20 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f42fcff9700 1 -- 172.21.15.52:0/1383218202 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f43100b3030 con 0x55574475af20 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f43261d1b80 10 client.31049 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.196+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5557447df250 con 0x55574475af20 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.197+0000 7f42fcff9700 1 -- 172.21.15.52:0/1383218202 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f430c0092e0 con 0x55574475af20 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.197+0000 7f42fcff9700 1 -- 172.21.15.52:0/1383218202 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f431009bac0 con 0x55574475af20 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.197+0000 7f43261d1b80 10 client.31049 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.198+0000 7f43261d1b80 10 client.31049 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.198+0000 7f43261d1b80 20 client.31049 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.198+0000 7f43261d1b80 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x5557447e1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.198+0000 7f43261d1b80 4 client.31049 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:36:48.200 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.198+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557446b4290 con 0x555744774610 2024-06-02T23:36:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f4318ee9700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x5557447e1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f4318ee9700 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x5557447e1940 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e1cb5a775416c524 server_cookie=a9df7a49f62ff2d4 in_seq=0 out_seq=0 2024-06-02T23:36:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f42fcff9700 10 client.31049 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:36:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f42fcff9700 1 -- 172.21.15.52:0/1383218202 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557446b4290 con 0x555744774610 2024-06-02T23:36:48.202 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f42fcff9700 10 client.31049 handle_command_reply: tid=0 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f43261d1b80 1 client.31049 shutdown 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 msgr2=0x55574476d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.200+0000 7f43261d1b80 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 0x55574476d5b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f4310009800 tx=0x7f4310009790 comp rx=0 tx=0).stop 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 shutdown_connections 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555744760f60 0x55574477c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x555744774610 0x5557447e1940 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f42fcff9700 0 client.31049 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55574475af20 0x55574476d5b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 --2- 172.21.15.52:0/1383218202 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555744760640 0x55574477bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 >> 172.21.15.52:0/1383218202 conn(0x555744685cc0 msgr2=0x55574474ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 shutdown_connections 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 1 -- 172.21.15.52:0/1383218202 wait complete. 2024-06-02T23:36:48.203 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.201+0000 7f43261d1b80 20 client.31049 trim_cache size 0 max 16384 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43000a2d20 msgr2=0x7f43000a51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43000a2d20 0x7f43000a51e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f4314106290 tx=0x7f4304005fd0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314100b30 msgr2=0x7f431410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314100b30 0x7f431410f250 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f431000e9f0 tx=0x7f431000ed00 comp rx=0 tx=0).stop 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 shutdown_connections 2024-06-02T23:36:48.204 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f43000a2d20 0x7f43000a51e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4314106be0 0x7f431410fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4314100b30 0x7f431410f250 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 --2- 172.21.15.52:0/1046847683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43141014f0 0x7f431410f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 >> 172.21.15.52:0/1046847683 conn(0x7f4314077990 msgr2=0x7f43140ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 shutdown_connections 2024-06-02T23:36:48.205 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:48.202+0000 7f431b94e700 1 -- 172.21.15.52:0/1046847683 wait complete. 2024-06-02T23:36:48.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:47 smithi123 ceph-mon[35977]: pgmap v1178: 97 pgs: 97 active+clean; 627 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 4.5 MiB/s wr, 292 op/s 2024-06-02T23:36:48.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 88095 6.82 MB/sec execute 596 sec latency 75.339 ms 2024-06-02T23:36:48.899 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db / --timeout 90 --expiration 136.9 --await-for 45.0' 2024-06-02T23:36:48.899 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db / --timeout 90 --expiration 136.9 --await-for 45.0 2024-06-02T23:36:49.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:48 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2532140841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:36:49.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:48 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2532140841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:36:49.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:48 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2532140841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:36:49.482 INFO:tasks.workunit.client.0.smithi052.stdout: 10 88244 6.82 MB/sec execute 597 sec latency 58.830 ms 2024-06-02T23:36:49.693 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:36:49.781 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:49.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:49 smithi052 ceph-mon[32340]: pgmap v1179: 97 pgs: 97 active+clean; 629 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-06-02T23:36:50.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:49 smithi174 ceph-mon[35994]: pgmap v1179: 97 pgs: 97 active+clean; 629 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-06-02T23:36:50.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:49 smithi123 ceph-mon[35977]: pgmap v1179: 97 pgs: 97 active+clean; 629 MiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 999 KiB/s rd, 3.8 MiB/s wr, 253 op/s 2024-06-02T23:36:50.370 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 -- 172.21.15.52:0/3047701879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 msgr2=0x7fbfb4109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 --2- 172.21.15.52:0/3047701879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb4109b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009ca0 tx=0x7fbfac02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 -- 172.21.15.52:0/3047701879 shutdown_connections 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 --2- 172.21.15.52:0/3047701879 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb4109b90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 --2- 172.21.15.52:0/3047701879 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb4107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 --2- 172.21.15.52:0/3047701879 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb4102540 0x7fbfb4104930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 -- 172.21.15.52:0/3047701879 >> 172.21.15.52:0/3047701879 conn(0x7fbfb40fbe90 msgr2=0x7fbfb40fe2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 -- 172.21.15.52:0/3047701879 shutdown_connections 2024-06-02T23:36:50.371 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.370+0000 7fbfba95c640 1 -- 172.21.15.52:0/3047701879 wait complete. 2024-06-02T23:36:50.372 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 Processor -- start 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 -- start start 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb4102540 0x7fbfb4197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb41980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410d5c0 con 0x7fbfb4104e70 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410d440 con 0x7fbfb41077a0 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfba95c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb410d740 con 0x7fbfb4102540 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb41980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb41980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37682/0 (socket says 172.21.15.52:37682) 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1556909609 learned_addr learned my addr 172.21.15.52:0/1556909609 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.371+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb4102540 0x7fbfb4197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.373 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1556909609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb4102540 msgr2=0x7fbfb4197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb4102540 0x7fbfb4197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1556909609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 msgr2=0x7fbfb419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1556909609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb419f7a0 con 0x7fbfb4104e70 2024-06-02T23:36:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:50.374 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.372+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb41980c0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e9c0 tx=0x7fbfa800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.373+0000 7fbfb17fa640 1 -- 172.21.15.52:0/1556909609 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa800cd80 con 0x7fbfb4104e70 2024-06-02T23:36:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.373+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb419fa90 con 0x7fbfb4104e70 2024-06-02T23:36:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.373+0000 7fbfb17fa640 1 -- 172.21.15.52:0/1556909609 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa80054d0 con 0x7fbfb4104e70 2024-06-02T23:36:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.373+0000 7fbfb17fa640 1 -- 172.21.15.52:0/1556909609 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8010430 con 0x7fbfb4104e70 2024-06-02T23:36:50.375 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.373+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb41a0030 con 0x7fbfb4104e70 2024-06-02T23:36:50.376 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.376+0000 7fbfb17fa640 1 -- 172.21.15.52:0/1556909609 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fbfa8004fd0 con 0x7fbfb4104e70 2024-06-02T23:36:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.377+0000 7fbfb17fa640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf9c0a2cb0 0x7fbf9c0a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.377+0000 7fbfbd65f740 1 Processor -- start 2024-06-02T23:36:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.377+0000 7fbfbd65f740 1 -- start start 2024-06-02T23:36:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfb17fa640 1 -- 172.21.15.52:0/1556909609 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbfa80cf430 con 0x7fbfb4104e70 2024-06-02T23:36:50.378 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648bd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648c5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e26481c4f0 con 0x55e2648bd420 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2647ba870 con 0x55e2648bde10 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf9c0a2cb0 0x7fbf9c0a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58576/0 (socket says 172.21.15.52:58576) 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1455938544 learned_addr learned my addr 172.21.15.52:0/1455938544 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648c5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.379 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.378+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2648184b0 con 0x55e2648c2f80 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648bd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1455938544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 msgr2=0x55e2648c5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648c5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1455938544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 msgr2=0x55e2648bd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648bd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb37fe640 1 -- 172.21.15.52:0/1455938544 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2647d69b0 con 0x55e2648bde10 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf9c0a2cb0 0x7fbf9c0a5170 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009c20 tx=0x7fbfa4011810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:50.381 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c2910 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8098690 tx=0x7fbfa809b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8cc1464d9716589 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbf9a7fc640 1 -- 172.21.15.52:0/1455938544 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa809bcc0 con 0x55e2648bde10 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.379+0000 7fbf9a7fc640 1 -- 172.21.15.52:0/1455938544 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa80a0070 con 0x55e2648bde10 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbf9a7fc640 1 -- 172.21.15.52:0/1455938544 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80cf0e0 con 0x55e2648bde10 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 -- 172.21.15.52:0/1455938544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 msgr2=0x55e2648c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c2910 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8098690 tx=0x7fbfa809b800 comp rx=0 tx=0).stop 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 -- 172.21.15.52:0/1455938544 shutdown_connections 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648c5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c2910 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/1455938544 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648bd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.380+0000 7fbfbd65f740 1 -- 172.21.15.52:0/1455938544 >> 172.21.15.52:0/1455938544 conn(0x55e2648b4680 msgr2=0x55e2648b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.381+0000 7fbfbd65f740 1 -- 172.21.15.52:0/1455938544 shutdown_connections 2024-06-02T23:36:50.382 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.381+0000 7fbfbd65f740 1 -- 172.21.15.52:0/1455938544 wait complete. 2024-06-02T23:36:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.381+0000 7fbfbd65f740 1 -- start start 2024-06-02T23:36:50.386 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.381+0000 7fbfbd65f740 1 Processor -- start 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 0x55e2648c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2648b9010 con 0x55e2648c2f80 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2647a5ae0 con 0x55e2648bde10 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2648b9190 con 0x55e2648bd420 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 0x55e2648c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 0x55e2648c0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53662/0 (socket says 172.21.15.52:53662) 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58586/0 (socket says 172.21.15.52:58586) 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 -- 172.21.15.52:0/709688095 learned_addr learned my addr 172.21.15.52:0/709688095 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.387 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfbd65f740 10 client.0 _ll_register_callbacks cb 0x55e2647de9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 -- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 msgr2=0x55e2648c0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 -- 172.21.15.52:0/709688095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 msgr2=0x55e2648c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb3fff640 1 -- 172.21.15.52:0/709688095 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e264530210 con 0x55e2648bd420 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.382+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 0x55e2648c0540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400f950 tx=0x7fbfa4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f2470e88675a04 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbf937fe640 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbf937fe640 1 -- 172.21.15.52:0/709688095 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4017040 con 0x55e2648bd420 2024-06-02T23:36:50.388 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbf937fe640 1 -- 172.21.15.52:0/709688095 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbfa401e410 con 0x55e2648bd420 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa80a0070 con 0x55e2648bd420 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbf937fe640 1 -- 172.21.15.52:0/709688095 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbfa4028630 con 0x55e2648bd420 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbfbd65f740 10 client.30640 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e2645fc620 con 0x55e2648bd420 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.383+0000 7fbf937fe640 1 -- 172.21.15.52:0/709688095 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa80a0070 con 0x55e2648bd420 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbf937fe640 1 -- 172.21.15.52:0/709688095 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbfa401eab0 con 0x55e2648bd420 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfbd65f740 10 client.30640 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfbd65f740 10 client.30640 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfbd65f740 20 client.30640 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e2648d7cf0 0x55e2648da190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfbd65f740 4 client.30640 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:50.389 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e26482ff10 con 0x55e2648d7cf0 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e2648d7cf0 0x55e2648da190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e2648d7cf0 0x55e2648da190 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffcb9b75216be8fc server_cookie=fd18b1f04c3556c5 in_seq=0 out_seq=0 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.384+0000 7fbf937fe640 10 client.30640 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbf937fe640 1 -- 172.21.15.52:0/709688095 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e26482ff10 con 0x55e2648d7cf0 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbf937fe640 10 client.30640 handle_command_reply: tid=0 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbfbd65f740 1 client.30640 shutdown 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 msgr2=0x55e2648c0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 0x55e2648c0540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400f950 tx=0x7fbfa4004730 comp rx=0 tx=0).stop 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 shutdown_connections 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e2648d7cf0 0x55e2648da190 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.390 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.385+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648c0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/709688095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bd420 0x55e2648c0540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 >> 172.21.15.52:0/709688095 conn(0x55e2648b4680 msgr2=0x55e2648b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbf937fe640 0 client.30640 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 shutdown_connections 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbfbd65f740 1 -- 172.21.15.52:0/709688095 wait complete. 2024-06-02T23:36:50.391 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.386+0000 7fbfbd65f740 20 client.30640 trim_cache size 0 max 16384 2024-06-02T23:36:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.407+0000 7fbfbd65f740 1 Processor -- start 2024-06-02T23:36:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.407+0000 7fbfbd65f740 1 -- start start 2024-06-02T23:36:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.407+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bd420 0x55e2648beba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.409 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.407+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bde10 0x55e2648bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c0150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41a6af0 con 0x55e2648c2f80 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41a8630 con 0x55e2648bd420 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41a71c0 con 0x55e2648bde10 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bde10 0x55e2648bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bd420 0x55e2648beba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb37fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bde10 0x55e2648bf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53664/0 (socket says 172.21.15.52:53664) 2024-06-02T23:36:50.410 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb37fe640 1 -- 172.21.15.52:0/344978139 learned_addr learned my addr 172.21.15.52:0/344978139 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 -- 172.21.15.52:0/344978139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bde10 msgr2=0x55e2648bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bde10 0x55e2648bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 -- 172.21.15.52:0/344978139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 msgr2=0x55e2648c0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 -- 172.21.15.52:0/344978139 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2648b44b0 con 0x55e2648bd420 2024-06-02T23:36:50.411 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.408+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bd420 0x55e2648beba0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4014040 tx=0x7fbfa400f7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=911972abf70b8a19 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbf99ffb640 1 -- 172.21.15.52:0/344978139 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4017070 con 0x55e2648bd420 2024-06-02T23:36:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbf99ffb640 1 -- 172.21.15.52:0/344978139 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4037070 con 0x55e2648bd420 2024-06-02T23:36:50.412 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbf99ffb640 1 -- 172.21.15.52:0/344978139 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4028a30 con 0x55e2648bd420 2024-06-02T23:36:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 -- 172.21.15.52:0/344978139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bd420 msgr2=0x55e2648beba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bd420 0x55e2648beba0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4014040 tx=0x7fbfa400f7e0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 -- 172.21.15.52:0/344978139 shutdown_connections 2024-06-02T23:36:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648bde10 0x55e2648bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648c2f80 0x55e2648c0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.417 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/344978139 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bd420 0x55e2648beba0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.409+0000 7fbfbd65f740 1 -- 172.21.15.52:0/344978139 >> 172.21.15.52:0/344978139 conn(0x55e2648b4680 msgr2=0x55e2648b9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.410+0000 7fbfbd65f740 1 -- 172.21.15.52:0/344978139 shutdown_connections 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.410+0000 7fbfbd65f740 1 -- 172.21.15.52:0/344978139 wait complete. 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.410+0000 7fbfbd65f740 1 -- start start 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.410+0000 7fbfbd65f740 1 Processor -- start 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.410+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.418 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfbd65f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648d00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41a7f60 con 0x55e2648bd420 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41a7890 con 0x55e2648bde10 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfbd65f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb41a4ad0 con 0x55e2648c2f80 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfbd65f740 10 client.0 _ll_register_callbacks cb 0x7fbfa4009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648d3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37710/0 (socket says 172.21.15.52:37710) 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 -- 172.21.15.52:0/99790932 learned_addr learned my addr 172.21.15.52:0/99790932 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:50.419 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648d00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 -- 172.21.15.52:0/99790932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 msgr2=0x55e2648d00d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648d00d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 -- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 msgr2=0x55e2648d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 -- 172.21.15.52:0/99790932 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2648c1e60 con 0x55e2648bd420 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb8ef9640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbfb3fff640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648d3630 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fbfa401eef0 tx=0x7fbfa40284c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac6318408a43ca4 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:50.420 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.411+0000 7fbf937fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbf937fe640 1 -- 172.21.15.52:0/99790932 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa4040070 con 0x55e2648bd420 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbf937fe640 1 -- 172.21.15.52:0/99790932 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fbfa4033da0 con 0x55e2648bd420 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa4037070 con 0x55e2648bd420 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbf937fe640 1 -- 172.21.15.52:0/99790932 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fbfa404c050 con 0x55e2648bd420 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbfbd65f740 10 client.21843 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e2649413d0 con 0x55e2648bd420 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbf937fe640 1 -- 172.21.15.52:0/99790932 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fbfa4037070 con 0x55e2648bd420 2024-06-02T23:36:50.421 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.412+0000 7fbf937fe640 1 -- 172.21.15.52:0/99790932 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fbfa4028c90 con 0x55e2648bd420 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfbd65f740 10 client.21843 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfbd65f740 10 client.21843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfbd65f740 20 client.21843 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e264941c70 0x55e264944090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfbd65f740 4 client.21843 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e264829c20 con 0x55e264941c70 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e264941c70 0x55e264944090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbfb37fe640 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e264941c70 0x55e264944090 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f28548ff4ee5a342 server_cookie=9797d610047077aa in_seq=0 out_seq=0 2024-06-02T23:36:50.422 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.413+0000 7fbf937fe640 10 client.21843 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.467+0000 7fbf937fe640 1 -- 172.21.15.52:0/99790932 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55e264829c20 con 0x55e264941c70 2024-06-02T23:36:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.467+0000 7fbf937fe640 10 client.21843 handle_command_reply: tid=0 2024-06-02T23:36:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 client.21843 shutdown 2024-06-02T23:36:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 msgr2=0x55e2648d3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.468 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648d3630 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7fbfa401eef0 tx=0x7fbfa40284c0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 shutdown_connections 2024-06-02T23:36:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55e264941c70 0x55e264944090 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2648c2f80 0x55e2648d00d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.469 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2648bde10 0x55e2648d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 --2- 172.21.15.52:0/99790932 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2648bd420 0x55e2648d3630 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 >> 172.21.15.52:0/99790932 conn(0x55e2648b4680 msgr2=0x55e2648beb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.468+0000 7fbf937fe640 0 client.21843 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.469+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 shutdown_connections 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.469+0000 7fbfbd65f740 1 -- 172.21.15.52:0/99790932 wait complete. 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.469+0000 7fbfbd65f740 20 client.21843 trim_cache size 0 max 16384 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:36:50.470 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf9c0a2cb0 msgr2=0x7fbf9c0a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf9c0a2cb0 0x7fbf9c0a5170 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fbfa4009c20 tx=0x7fbfa4011810 comp rx=0 tx=0).stop 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 msgr2=0x7fbfb41980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb41980c0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fbfa800e9c0 tx=0x7fbfa800ee80 comp rx=0 tx=0).stop 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 shutdown_connections 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fbf9c0a2cb0 0x7fbf9c0a5170 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb41077a0 0x7fbfb419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fbfb4104e70 0x7fbfb41980c0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 --2- 172.21.15.52:0/1556909609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbfb4102540 0x7fbfb4197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 >> 172.21.15.52:0/1556909609 conn(0x7fbfb40fbe90 msgr2=0x7fbfb4102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 shutdown_connections 2024-06-02T23:36:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:36:50.470+0000 7fbfba95c640 1 -- 172.21.15.52:0/1556909609 wait complete. 2024-06-02T23:36:50.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 88430 6.83 MB/sec execute 598 sec latency 128.884 ms 2024-06-02T23:36:51.218 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:51.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 88636 6.84 MB/sec execute 599 sec latency 78.817 ms 2024-06-02T23:36:51.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:51 smithi052 ceph-mon[32340]: pgmap v1180: 97 pgs: 97 active+clean; 629 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-06-02T23:36:52.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:51 smithi174 ceph-mon[35994]: pgmap v1180: 97 pgs: 97 active+clean; 629 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-06-02T23:36:52.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:51 smithi123 ceph-mon[35977]: pgmap v1180: 97 pgs: 97 active+clean; 629 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 4.4 MiB/s wr, 273 op/s 2024-06-02T23:36:52.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 600 sec 2024-06-02T23:36:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 -- 172.21.15.52:0/735675931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4103400 msgr2=0x7f37c406ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 --2- 172.21.15.52:0/735675931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4103400 0x7f37c406ab60 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f37bc00b0a0 tx=0x7f37bc00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 -- 172.21.15.52:0/735675931 shutdown_connections 2024-06-02T23:36:52.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 --2- 172.21.15.52:0/735675931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c410ede0 0x7f37c41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 --2- 172.21.15.52:0/735675931 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c4103400 0x7f37c406ab60 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 --2- 172.21.15.52:0/735675931 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c406b0a0 0x7f37c406b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 -- 172.21.15.52:0/735675931 >> 172.21.15.52:0/735675931 conn(0x7f37c4078340 msgr2=0x7f37c4078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 -- 172.21.15.52:0/735675931 shutdown_connections 2024-06-02T23:36:52.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.670+0000 7f37cadb5700 1 -- 172.21.15.52:0/735675931 wait complete. 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.671+0000 7f37cadb5700 1 Processor -- start 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.671+0000 7f37cadb5700 1 -- start start 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37cadb5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c406b0a0 0x7f37c41a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37cadb5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 0x7f37c41a0e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37cadb5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c410ede0 0x7f37c419ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37cadb5700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41148f0 con 0x7f37c410ede0 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37cadb5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4114770 con 0x7f37c4103400 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37cadb5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37c4114a70 con 0x7f37c406b0a0 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 0x7f37c41a0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 0x7f37c41a0e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58620/0 (socket says 172.21.15.52:58620) 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 -- 172.21.15.52:0/2879247511 learned_addr learned my addr 172.21.15.52:0/2879247511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c8b51700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c406b0a0 0x7f37c41a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c9352700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c410ede0 0x7f37c419ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 -- 172.21.15.52:0/2879247511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c406b0a0 msgr2=0x7f37c41a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c406b0a0 0x7f37c41a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 -- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c410ede0 msgr2=0x7f37c419ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c410ede0 0x7f37c419ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.672+0000 7f37c3fff700 1 -- 172.21.15.52:0/2879247511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c419b2b0 con 0x7f37c4103400 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.673+0000 7f37c3fff700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 0x7f37c41a0e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f37b800e9a0 tx=0x7f37b800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.673+0000 7f37c1ffb700 1 -- 172.21.15.52:0/2879247511 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b800ca10 con 0x7f37c4103400 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.673+0000 7f37c1ffb700 1 -- 172.21.15.52:0/2879247511 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b800cbb0 con 0x7f37c4103400 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.673+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c419b5a0 con 0x7f37c4103400 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.673+0000 7f37c1ffb700 1 -- 172.21.15.52:0/2879247511 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8018680 con 0x7f37c4103400 2024-06-02T23:36:52.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.673+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c41a7810 con 0x7f37c4103400 2024-06-02T23:36:52.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.676+0000 7f37c1ffb700 1 -- 172.21.15.52:0/2879247511 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f37b8010430 con 0x7f37c4103400 2024-06-02T23:36:52.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.677+0000 7f37d5638b80 1 Processor -- start 2024-06-02T23:36:52.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.677+0000 7f37d5638b80 1 -- start start 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9b4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37d5638b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cb8c8fd9a0 con 0x55cb8c9bae00 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37d5638b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb8c8a58b0 con 0x55cb8c9b4c70 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37d5638b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb8c8f9960 con 0x55cb8c9ba4e0 2024-06-02T23:36:52.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37c8b51700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9b4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9b50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:58626/0 (socket says 172.21.15.52:58626) 2024-06-02T23:36:52.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37c3fff700 1 -- 172.21.15.52:0/3144028251 learned_addr learned my addr 172.21.15.52:0/3144028251 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:52.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.678+0000 7f37c1ffb700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37ac0a2d40 0x7f37ac0a5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c3fff700 1 -- 172.21.15.52:0/3144028251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 msgr2=0x55cb8c9ba8c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c3fff700 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9ba8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c3fff700 1 -- 172.21.15.52:0/3144028251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 msgr2=0x55cb8c9b4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c3fff700 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9b4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c1ffb700 1 -- 172.21.15.52:0/2879247511 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f37b80cf140 con 0x7f37c4103400 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c3fff700 1 -- 172.21.15.52:0/3144028251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8c89c0e0 con 0x55cb8c9b4c70 2024-06-02T23:36:52.681 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c9352700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37ac0a2d40 0x7f37ac0a5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37c3fff700 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9b50b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f37b809b6d0 tx=0x7f37b8010ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7aa0e237c66f9ef server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37aaffd700 1 -- 172.21.15.52:0/3144028251 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b809b330 con 0x55cb8c9b4c70 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.679+0000 7f37aaffd700 1 -- 172.21.15.52:0/3144028251 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b809c0d0 con 0x55cb8c9b4c70 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.680+0000 7f37aaffd700 1 -- 172.21.15.52:0/3144028251 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b805d050 con 0x55cb8c9b4c70 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.680+0000 7f37c9352700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37ac0a2d40 0x7f37ac0a5200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005950 tx=0x7f37b400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.680+0000 7f37d5638b80 1 -- 172.21.15.52:0/3144028251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 msgr2=0x55cb8c9b50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.680+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9b50b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f37b809b6d0 tx=0x7f37b8010ad0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 -- 172.21.15.52:0/3144028251 shutdown_connections 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9b50b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3144028251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9b4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.682 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 -- 172.21.15.52:0/3144028251 >> 172.21.15.52:0/3144028251 conn(0x55cb8c8d6e50 msgr2=0x55cb8c9a71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:52.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 -- 172.21.15.52:0/3144028251 shutdown_connections 2024-06-02T23:36:52.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 -- 172.21.15.52:0/3144028251 wait complete. 2024-06-02T23:36:52.685 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 -- start start 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.681+0000 7f37d5638b80 1 Processor -- start 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9be380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55cb8c9c17b0 con 0x55cb8c9bae00 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cb8c89dd80 con 0x55cb8c9ba4e0 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb8c9c1930 con 0x55cb8c9b4c70 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37d5638b80 10 client.0 _ll_register_callbacks cb 0x55cb8c897ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:52.686 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:53720/0 (socket says 172.21.15.52:53720) 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 -- 172.21.15.52:0/18612859 learned_addr learned my addr 172.21.15.52:0/18612859 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c9352700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9be380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c3fff700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 -- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 msgr2=0x55cb8c9bde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bde40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 -- 172.21.15.52:0/18612859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 msgr2=0x55cb8c9be380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9be380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 -- 172.21.15.52:0/18612859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8c84ca60 con 0x55cb8c9b4c70 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c3fff700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c9352700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37c8b51700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c0b40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000f80 tx=0x7f37bc00bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40567b3f5a1225cd server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:52.687 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.682+0000 7f37a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37a3fff700 1 -- 172.21.15.52:0/18612859 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc0076e0 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37a3fff700 1 -- 172.21.15.52:0/18612859 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f37bc007880 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b809c0d0 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37a3fff700 1 -- 172.21.15.52:0/18612859 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f37bc007a80 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37d5638b80 10 client.30655 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cb8c9cc000 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37a3fff700 1 -- 172.21.15.52:0/18612859 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b809c0d0 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.683+0000 7f37a3fff700 1 -- 172.21.15.52:0/18612859 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f37bc011070 con 0x55cb8c9b4c70 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37d5638b80 10 client.30655 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37d5638b80 10 client.30655 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37d5638b80 20 client.30655 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37d5638b80 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8c9d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37d5638b80 4 client.30655 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb8c90aea0 con 0x55cb8c9ce640 2024-06-02T23:36:52.688 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37c3fff700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8c9d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37a3fff700 10 client.30655 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.684+0000 7f37c3fff700 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8c9d0a30 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d952ba2c273140e server_cookie=8e0578db5a6f9f2c in_seq=0 out_seq=0 2024-06-02T23:36:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37a3fff700 1 -- 172.21.15.52:0/18612859 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55cb8c90aea0 con 0x55cb8c9ce640 2024-06-02T23:36:52.689 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37a3fff700 10 client.30655 handle_command_reply: tid=0 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 client.30655 shutdown 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 msgr2=0x55cb8c9c0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c0b40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f37bc000f80 tx=0x7f37bc00bf60 comp rx=0 tx=0).stop 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 shutdown_connections 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8c9d0a30 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9be380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37a3fff700 0 client.30655 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c0b40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 --2- 172.21.15.52:0/18612859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 >> 172.21.15.52:0/18612859 conn(0x55cb8c8d6e50 msgr2=0x55cb8c9a6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 shutdown_connections 2024-06-02T23:36:52.690 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.686+0000 7f37d5638b80 1 -- 172.21.15.52:0/18612859 wait complete. 2024-06-02T23:36:52.691 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.687+0000 7f37d5638b80 20 client.30655 trim_cache size 0 max 16384 2024-06-02T23:36:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 Processor -- start 2024-06-02T23:36:52.720 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 -- start start 2024-06-02T23:36:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9d62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41aa870 con 0x55cb8c9bae00 2024-06-02T23:36:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41ac3b0 con 0x55cb8c9ba4e0 2024-06-02T23:36:52.721 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37d5638b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41aaf40 con 0x55cb8c9b4c70 2024-06-02T23:36:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37c9352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9d62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37c9352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9d62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37762/0 (socket says 172.21.15.52:37762) 2024-06-02T23:36:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.720+0000 7f37c9352700 1 -- 172.21.15.52:0/3363089118 learned_addr learned my addr 172.21.15.52:0/3363089118 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c3fff700 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c8b51700 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.722 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c9352700 1 -- 172.21.15.52:0/3363089118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 msgr2=0x55cb8c9bba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c9352700 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9bba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c9352700 1 -- 172.21.15.52:0/3363089118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 msgr2=0x55cb8c9bce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c9352700 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c9352700 1 -- 172.21.15.52:0/3363089118 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8c9bc3d0 con 0x55cb8c9bae00 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37c9352700 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9d62e0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f37b4003a10 tx=0x7f37b4013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc449ea60932f2e server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37aa7fc700 1 -- 172.21.15.52:0/3363089118 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4028670 con 0x55cb8c9bae00 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37aa7fc700 1 -- 172.21.15.52:0/3363089118 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b4023070 con 0x55cb8c9bae00 2024-06-02T23:36:52.723 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.721+0000 7f37aa7fc700 1 -- 172.21.15.52:0/3363089118 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b401f720 con 0x55cb8c9bae00 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.722+0000 7f37d5638b80 1 -- 172.21.15.52:0/3363089118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 msgr2=0x55cb8c9d62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.722+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9d62e0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f37b4003a10 tx=0x7f37b4013000 comp rx=0 tx=0).stop 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 -- 172.21.15.52:0/3363089118 shutdown_connections 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9d62e0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9bba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 --2- 172.21.15.52:0/3363089118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 -- 172.21.15.52:0/3363089118 >> 172.21.15.52:0/3363089118 conn(0x55cb8c8d6e50 msgr2=0x55cb8c9bc750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 -- 172.21.15.52:0/3363089118 shutdown_connections 2024-06-02T23:36:52.724 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.723+0000 7f37d5638b80 1 -- 172.21.15.52:0/3363089118 wait complete. 2024-06-02T23:36:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 -- start start 2024-06-02T23:36:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 Processor -- start 2024-06-02T23:36:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.725 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41abce0 con 0x55cb8c9bae00 2024-06-02T23:36:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41ab610 con 0x55cb8c9b4c70 2024-06-02T23:36:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f37c41079a0 con 0x55cb8c9ba4e0 2024-06-02T23:36:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.724+0000 7f37d5638b80 10 client.0 _ll_register_callbacks cb 0x55cb8c897ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:52.726 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c3fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9c41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:37768/0 (socket says 172.21.15.52:37768) 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 -- 172.21.15.52:0/521667349 learned_addr learned my addr 172.21.15.52:0/521667349 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c8b51700 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 -- 172.21.15.52:0/521667349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 msgr2=0x55cb8c9c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9c3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 -- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 msgr2=0x55cb8c9c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 -- 172.21.15.52:0/521667349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8c9c49d0 con 0x55cb8c9bae00 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37c9352700 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9c41d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005f90 tx=0x7f37b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73e31076311124df server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.725+0000 7f37a3fff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37a3fff700 1 -- 172.21.15.52:0/521667349 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4005250 con 0x55cb8c9bae00 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37a3fff700 1 -- 172.21.15.52:0/521667349 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f37b40053f0 con 0x55cb8c9bae00 2024-06-02T23:36:52.727 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b4023070 con 0x55cb8c9bae00 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37a3fff700 1 -- 172.21.15.52:0/521667349 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f37b40055f0 con 0x55cb8c9bae00 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37d5638b80 10 client.21861 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55cb8ca3a3a0 con 0x55cb8c9bae00 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.726+0000 7f37a3fff700 1 -- 172.21.15.52:0/521667349 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f37b4004c00 con 0x55cb8c9bae00 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37a3fff700 1 -- 172.21.15.52:0/521667349 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f37b40289a0 con 0x55cb8c9bae00 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37d5638b80 10 client.21861 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37d5638b80 10 client.21861 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37d5638b80 20 client.21861 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37d5638b80 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8ca3ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37d5638b80 4 client.21861 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.9, "await_for": 45.0}] 2024-06-02T23:36:52.729 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.9, "await_for": 45.0}) -- 0x7f37b4023070 con 0x55cb8c9ce640 2024-06-02T23:36:52.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.727+0000 7f37c8b51700 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8ca3ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:52.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.728+0000 7f37c8b51700 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8ca3ca70 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f5ce05637dd762 server_cookie=3287dc2820eed2b1 in_seq=0 out_seq=0 2024-06-02T23:36:52.730 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.728+0000 7f37a3fff700 10 client.21861 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37a3fff700 1 -- 172.21.15.52:0/521667349 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7f37b4023070 con 0x55cb8c9ce640 2024-06-02T23:36:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37a3fff700 10 client.21861 handle_command_reply: tid=0 2024-06-02T23:36:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 client.21861 shutdown 2024-06-02T23:36:52.966 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 msgr2=0x55cb8c9c41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9c41d0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005f90 tx=0x7f37b4011aa0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 shutdown_connections 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55cb8c9ce640 0x55cb8ca3ca70 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37a3fff700 0 client.21861 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55cb8c9bae00 0x55cb8c9c41d0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cb8c9b4c70 0x55cb8c9c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 --2- 172.21.15.52:0/521667349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8c9ba4e0 0x55cb8c9c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 >> 172.21.15.52:0/521667349 conn(0x55cb8c8d6e50 msgr2=0x55cb8c9b9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 shutdown_connections 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 1 -- 172.21.15.52:0/521667349 wait complete. 2024-06-02T23:36:52.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.965+0000 7f37d5638b80 20 client.21861 trim_cache size 0 max 16384 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 78, 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "6ef55db8": { 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 78, 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:36:52.968 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 136.9, 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "QUIESCED", 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:36:52.969 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37ac0a2d40 msgr2=0x7f37ac0a5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37ac0a2d40 0x7f37ac0a5200 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005950 tx=0x7f37b400b390 comp rx=0 tx=0).stop 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 msgr2=0x7f37c41a0e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 0x7f37c41a0e80 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f37b800e9a0 tx=0x7f37b800ecb0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 shutdown_connections 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f37ac0a2d40 0x7f37ac0a5200 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f37c410ede0 0x7f37c419ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f37c406b0a0 0x7f37c41a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 --2- 172.21.15.52:0/2879247511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37c4103400 0x7f37c41a0e80 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.966+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 >> 172.21.15.52:0/2879247511 conn(0x7f37c4078340 msgr2=0x7f37c4100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.967+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 shutdown_connections 2024-06-02T23:36:52.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:52.967+0000 7f37cadb5700 1 -- 172.21.15.52:0/2879247511 wait complete. 2024-06-02T23:36:53.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 601 sec 2024-06-02T23:36:54.050 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:53 smithi052 ceph-mon[32340]: pgmap v1181: 97 pgs: 97 active+clean; 629 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-06-02T23:36:54.059 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:53 smithi123 ceph-mon[35977]: pgmap v1181: 97 pgs: 97 active+clean; 629 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-06-02T23:36:54.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:53 smithi174 ceph-mon[35994]: pgmap v1181: 97 pgs: 97 active+clean; 629 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 4.2 MiB/s wr, 274 op/s 2024-06-02T23:36:54.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 602 sec 2024-06-02T23:36:54.549 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6ef55db8', quiesce took 10.8 seconds. Will release after: 6.099999999999998 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 -- 172.21.15.52:0/2645872500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8108510 msgr2=0x7f83f81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 --2- 172.21.15.52:0/2645872500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8108510 0x7f83f81088f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009990 tx=0x7f83e8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 -- 172.21.15.52:0/2645872500 shutdown_connections 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 --2- 172.21.15.52:0/2645872500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8101440 0x7f83f810f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 --2- 172.21.15.52:0/2645872500 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8108510 0x7f83f81088f0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 --2- 172.21.15.52:0/2645872500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f8100a80 0x7f83f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 -- 172.21.15.52:0/2645872500 >> 172.21.15.52:0/2645872500 conn(0x7f83f8077a00 msgr2=0x7f83f80fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:54.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 -- 172.21.15.52:0/2645872500 shutdown_connections 2024-06-02T23:36:54.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.815+0000 7f83fd025700 1 -- 172.21.15.52:0/2645872500 wait complete. 2024-06-02T23:36:54.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.816+0000 7f83fd025700 1 Processor -- start 2024-06-02T23:36:54.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.816+0000 7f83fd025700 1 -- start start 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.816+0000 7f83fd025700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 0x7f83f819e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.816+0000 7f83fd025700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f8101440 0x7f83f819ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.816+0000 7f83fd025700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8108510 0x7f83f8198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.816+0000 7f83fd025700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83f8112640 con 0x7f83f8100a80 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83fd025700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81124c0 con 0x7f83f8108510 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83fd025700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81127c0 con 0x7f83f8101440 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 0x7f83f819e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 0x7f83f819e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40874/0 (socket says 172.21.15.52:40874) 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3302442248 learned_addr learned my addr 172.21.15.52:0/3302442248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f759e700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8108510 0x7f83f8198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f659c700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f8101440 0x7f83f819ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3302442248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f8101440 msgr2=0x7f83f819ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f8101440 0x7f83f819ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3302442248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8108510 msgr2=0x7f83f8198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8108510 0x7f83f8198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3302442248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f81990a0 con 0x7f83f8100a80 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f759e700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8108510 0x7f83f8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 0x7f83f819e730 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e8003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.817+0000 7f83dffff700 1 -- 172.21.15.52:0/3302442248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e8027070 con 0x7f83f8100a80 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.818+0000 7f83dffff700 1 -- 172.21.15.52:0/3302442248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83e800bdc0 con 0x7f83f8100a80 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.818+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f8199330 con 0x7f83f8100a80 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.818+0000 7f83dffff700 1 -- 172.21.15.52:0/3302442248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e800f460 con 0x7f83f8100a80 2024-06-02T23:36:54.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.818+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f81a5600 con 0x7f83f8100a80 2024-06-02T23:36:54.821 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.821+0000 7f83dffff700 1 -- 172.21.15.52:0/3302442248 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f83e802a070 con 0x7f83f8100a80 2024-06-02T23:36:54.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f83dffff700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83e00a2c20 0x7f83e00a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f83f659c700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83e00a2c20 0x7f83e00a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f84078a8b80 1 Processor -- start 2024-06-02T23:36:54.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f83dffff700 1 -- 172.21.15.52:0/3302442248 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83e80cff30 con 0x7f83f8100a80 2024-06-02T23:36:54.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f84078a8b80 1 -- start start 2024-06-02T23:36:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f83f659c700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83e00a2c20 0x7f83e00a50e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005950 tx=0x7f83ec0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.822+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820903340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x5608208fd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x5608208fdbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f84078a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5608207eff60 con 0x560820903880 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f84078a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56082084e850 con 0x560820902f60 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f84078a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5608207b6460 con 0x5608208fd7a0 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x5608208fd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x5608208fd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40888/0 (socket says 172.21.15.52:40888) 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 -- 172.21.15.52:0/1822865325 learned_addr learned my addr 172.21.15.52:0/1822865325 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820903340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f759e700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x5608208fdbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 -- 172.21.15.52:0/1822865325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 msgr2=0x5608208fdbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x5608208fdbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 -- 172.21.15.52:0/1822865325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 msgr2=0x560820903340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820903340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 -- 172.21.15.52:0/1822865325 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56082079d8d0 con 0x560820903880 2024-06-02T23:36:54.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820903340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f759e700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x5608208fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.823+0000 7f83f659c700 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x5608208fd260 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00fd40 tx=0x7f83ec012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c5727c8aaa2b0a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f83dd7fa700 1 -- 172.21.15.52:0/1822865325 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec022070 con 0x560820903880 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f83dd7fa700 1 -- 172.21.15.52:0/1822865325 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec011c60 con 0x560820903880 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f83dd7fa700 1 -- 172.21.15.52:0/1822865325 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec01d610 con 0x560820903880 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 -- 172.21.15.52:0/1822865325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 msgr2=0x5608208fd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x5608208fd260 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f83ec00fd40 tx=0x7f83ec012000 comp rx=0 tx=0).stop 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 -- 172.21.15.52:0/1822865325 shutdown_connections 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x5608208fdbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820903340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 --2- 172.21.15.52:0/1822865325 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x5608208fd260 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.824+0000 7f84078a8b80 1 -- 172.21.15.52:0/1822865325 >> 172.21.15.52:0/1822865325 conn(0x56082082a080 msgr2=0x5608208f8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:54.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 -- 172.21.15.52:0/1822865325 shutdown_connections 2024-06-02T23:36:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 -- 172.21.15.52:0/1822865325 wait complete. 2024-06-02T23:36:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 -- start start 2024-06-02T23:36:54.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 Processor -- start 2024-06-02T23:36:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820909610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820902f60 0x560820906910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820903880 0x560820906e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5608207eefc0 con 0x5608208fd7a0 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56082084a810 con 0x560820903880 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56082090a260 con 0x560820902f60 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f84078a8b80 10 client.0 _ll_register_callbacks cb 0x5608207e45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820909610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820909610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40902/0 (socket says 172.21.15.52:40902) 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.825+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3562086405 learned_addr learned my addr 172.21.15.52:0/3562086405 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f759e700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820903880 0x560820906e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f659c700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820902f60 0x560820906910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820902f60 msgr2=0x560820906910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820902f60 0x560820906910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3562086405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820903880 msgr2=0x560820906e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820903880 0x560820906e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f6d9d700 1 -- 172.21.15.52:0/3562086405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560820814750 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f759e700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820903880 0x560820906e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820909610 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f83e80cda60 tx=0x7f83e809c1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654282064dc4164 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83daff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83daff5700 1 -- 172.21.15.52:0/3562086405 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e809cf50 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83daff5700 1 -- 172.21.15.52:0/3562086405 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83e80a4fa0 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec011c60 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f83daff5700 1 -- 172.21.15.52:0/3562086405 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f83e809c850 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f84078a8b80 10 client.21873 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.826+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560820914980 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f83daff5700 1 -- 172.21.15.52:0/3562086405 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec011c60 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f83daff5700 1 -- 172.21.15.52:0/3562086405 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f83e80b0ce0 con 0x5608208fd7a0 2024-06-02T23:36:54.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f84078a8b80 10 client.21873 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f84078a8b80 10 client.21873 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f84078a8b80 20 client.21873 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f84078a8b80 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820916f40 0x560820919330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f84078a8b80 4 client.21873 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56082085bd50 con 0x560820916f40 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.827+0000 7f83f659c700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820916f40 0x560820919330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.828+0000 7f83f659c700 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820916f40 0x560820919330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ccc645d627101465 server_cookie=6b4b2e1f7c5729e8 in_seq=0 out_seq=0 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.828+0000 7f83daff5700 10 client.21873 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f83daff5700 1 -- 172.21.15.52:0/3562086405 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x56082085bd50 con 0x560820916f40 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f83daff5700 10 client.21873 handle_command_reply: tid=0 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 client.21873 shutdown 2024-06-02T23:36:54.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 msgr2=0x560820909610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820909610 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f83e80cda60 tx=0x7f83e809c1b0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 shutdown_connections 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820903880 0x560820906e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820909610 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820916f40 0x560820919330 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 --2- 172.21.15.52:0/3562086405 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820902f60 0x560820906910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 >> 172.21.15.52:0/3562086405 conn(0x56082082a080 msgr2=0x5608208f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f83daff5700 0 client.21873 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 shutdown_connections 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 1 -- 172.21.15.52:0/3562086405 wait complete. 2024-06-02T23:36:54.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.829+0000 7f84078a8b80 20 client.21873 trim_cache size 0 max 16384 2024-06-02T23:36:54.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.860+0000 7f84078a8b80 1 Processor -- start 2024-06-02T23:36:54.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.860+0000 7f84078a8b80 1 -- start start 2024-06-02T23:36:54.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820904530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820905910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820903880 0x5608208f7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f84078a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81ac800 con 0x5608208fd7a0 2024-06-02T23:36:54.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f84078a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81ae340 con 0x560820902f60 2024-06-02T23:36:54.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f84078a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81aced0 con 0x560820903880 2024-06-02T23:36:54.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f83f659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820905910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820904530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820904530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40912/0 (socket says 172.21.15.52:40912) 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f83f6d9d700 1 -- 172.21.15.52:0/339513107 learned_addr learned my addr 172.21.15.52:0/339513107 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.861+0000 7f83f759e700 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820903880 0x5608208f7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83f659c700 1 -- 172.21.15.52:0/339513107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820903880 msgr2=0x5608208f7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83f659c700 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820903880 0x5608208f7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83f659c700 1 -- 172.21.15.52:0/339513107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 msgr2=0x560820904530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83f659c700 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820904530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83f659c700 1 -- 172.21.15.52:0/339513107 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560820904e80 con 0x560820902f60 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83f659c700 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820905910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5608208f5060 tx=0x7f83ec015000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d65b67a63c7995ff server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.862+0000 7f83dcff9700 1 -- 172.21.15.52:0/339513107 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec022050 con 0x560820902f60 2024-06-02T23:36:54.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f83dcff9700 1 -- 172.21.15.52:0/339513107 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec0054e0 con 0x560820902f60 2024-06-02T23:36:54.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f83dcff9700 1 -- 172.21.15.52:0/339513107 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83ec004a90 con 0x560820902f60 2024-06-02T23:36:54.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 -- 172.21.15.52:0/339513107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 msgr2=0x560820905910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820905910 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5608208f5060 tx=0x7f83ec015000 comp rx=0 tx=0).stop 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 -- 172.21.15.52:0/339513107 shutdown_connections 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560820903880 0x5608208f7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5608208fd7a0 0x560820904530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 --2- 172.21.15.52:0/339513107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x560820905910 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.863+0000 7f84078a8b80 1 -- 172.21.15.52:0/339513107 >> 172.21.15.52:0/339513107 conn(0x56082082a080 msgr2=0x560820905200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.864+0000 7f84078a8b80 1 -- 172.21.15.52:0/339513107 shutdown_connections 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.864+0000 7f84078a8b80 1 -- 172.21.15.52:0/339513107 wait complete. 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.864+0000 7f84078a8b80 1 -- start start 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 Processor -- start 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x56082090f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x56082090fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x56082090ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81adc70 con 0x560820903880 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81ad5a0 con 0x560820902f60 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83f81aa7c0 con 0x5608208fd7a0 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f84078a8b80 10 client.0 _ll_register_callbacks cb 0x5608207e45b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x56082090f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f83f6d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x56082090f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.52:59506/0 (socket says 172.21.15.52:59506) 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f83f759e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x56082090ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.865+0000 7f83f6d9d700 1 -- 172.21.15.52:0/2763182409 learned_addr learned my addr 172.21.15.52:0/2763182409 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f659c700 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x56082090fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f759e700 1 -- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 msgr2=0x56082090f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f759e700 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x56082090f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f759e700 1 -- 172.21.15.52:0/2763182409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 msgr2=0x56082090fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f759e700 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x56082090fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f759e700 1 -- 172.21.15.52:0/2763182409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56082090b330 con 0x560820903880 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83f759e700 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x56082090ab20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f83f000e7e0 tx=0x7f83f000eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a12dd6590ece14 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83daff5700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83daff5700 1 -- 172.21.15.52:0/2763182409 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f0027060 con 0x560820903880 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f83daff5700 1 -- 172.21.15.52:0/2763182409 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f83f0004050 con 0x560820903880 2024-06-02T23:36:54.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.866+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83ec0054e0 con 0x560820903880 2024-06-02T23:36:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f83daff5700 1 -- 172.21.15.52:0/2763182409 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f83f0003e10 con 0x560820903880 2024-06-02T23:36:54.871 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 10 client.21879 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x560820981760 con 0x560820903880 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f83daff5700 1 -- 172.21.15.52:0/2763182409 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f83ec0054e0 con 0x560820903880 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f83daff5700 1 -- 172.21.15.52:0/2763182409 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f83f0021030 con 0x560820903880 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 10 client.21879 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 10 client.21879 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 20 client.21879 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820981e10 0x560820984230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 4 client.21879 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5608208562d0 con 0x560820981e10 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.867+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820981e10 0x560820984230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f83daff5700 10 client.21879 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f83f6d9d700 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820981e10 0x560820984230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=618a890e962fb5fd server_cookie=1194bcac318cad0 in_seq=0 out_seq=0 2024-06-02T23:36:54.872 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f83daff5700 1 -- 172.21.15.52:0/2763182409 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5608208562d0 con 0x560820981e10 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f83daff5700 10 client.21879 handle_command_reply: tid=0 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 client.21879 shutdown 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 msgr2=0x56082090ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x56082090ab20 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f83f000e7e0 tx=0x7f83f000eaf0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 shutdown_connections 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560820903880 0x56082090ab20 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x560820981e10 0x560820984230 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5608208fd7a0 0x56082090f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 --2- 172.21.15.52:0/2763182409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560820902f60 0x56082090fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 >> 172.21.15.52:0/2763182409 conn(0x56082082a080 msgr2=0x5608209089d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f83daff5700 0 client.21879 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.868+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 shutdown_connections 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f84078a8b80 1 -- 172.21.15.52:0/2763182409 wait complete. 2024-06-02T23:36:54.873 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f84078a8b80 20 client.21879 trim_cache size 0 max 16384 2024-06-02T23:36:54.874 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83e00a2c20 msgr2=0x7f83e00a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83e00a2c20 0x7f83e00a50e0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f83ec005950 tx=0x7f83ec0058e0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 msgr2=0x7f83f819e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 0x7f83f819e730 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f83e8000c00 tx=0x7f83e8003b30 comp rx=0 tx=0).stop 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 shutdown_connections 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f83e00a2c20 0x7f83e00a50e0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f83f8108510 0x7f83f8198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f83f8100a80 0x7f83f819e730 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 --2- 172.21.15.52:0/3302442248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83f8101440 0x7f83f819ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.869+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 >> 172.21.15.52:0/3302442248 conn(0x7f83f8077a00 msgr2=0x7f83f80ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.870+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 shutdown_connections 2024-06-02T23:36:54.875 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:54.870+0000 7f83fd025700 1 -- 172.21.15.52:0/3302442248 wait complete. 2024-06-02T23:36:55.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 603 sec 2024-06-02T23:36:55.751 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:36:56.022 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:36:56.138 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:55 smithi052 ceph-mon[32340]: pgmap v1182: 97 pgs: 97 active+clean; 631 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-06-02T23:36:56.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:55 smithi174 ceph-mon[35994]: pgmap v1182: 97 pgs: 97 active+clean; 631 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-06-02T23:36:56.244 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:55 smithi123 ceph-mon[35977]: pgmap v1182: 97 pgs: 97 active+clean; 631 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 941 KiB/s rd, 3.7 MiB/s wr, 272 op/s 2024-06-02T23:36:56.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 604 sec 2024-06-02T23:36:57.483 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 605 sec 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.826+0000 7f158238e700 1 -- 172.21.15.52:0/3741138059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c108570 msgr2=0x7f157c108950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.826+0000 7f158238e700 1 --2- 172.21.15.52:0/3741138059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c108570 0x7f157c108950 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f156c009990 tx=0x7f156c009ca0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.826+0000 7f158238e700 1 -- 172.21.15.52:0/3741138059 shutdown_connections 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 --2- 172.21.15.52:0/3741138059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f157c06b560 0x7f157c10f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 --2- 172.21.15.52:0/3741138059 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c108570 0x7f157c108950 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 --2- 172.21.15.52:0/3741138059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c06aba0 0x7f157c06b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 -- 172.21.15.52:0/3741138059 >> 172.21.15.52:0/3741138059 conn(0x7f157c0773e0 msgr2=0x7f157c0777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 -- 172.21.15.52:0/3741138059 shutdown_connections 2024-06-02T23:36:57.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 -- 172.21.15.52:0/3741138059 wait complete. 2024-06-02T23:36:57.829 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:57 smithi052 ceph-mon[32340]: pgmap v1183: 97 pgs: 97 active+clean; 615 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.7 MiB/s wr, 283 op/s 2024-06-02T23:36:57.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 Processor -- start 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.827+0000 7f158238e700 1 -- start start 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158238e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f157c06aba0 0x7f157c1014f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158238e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 0x7f157c101a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158238e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c108570 0x7f157c101f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158238e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f157c112670 con 0x7f157c06b560 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158238e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1124f0 con 0x7f157c06aba0 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158238e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1127f0 con 0x7f157c108570 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 0x7f157c101a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 0x7f157c101a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40942/0 (socket says 172.21.15.52:40942) 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 -- 172.21.15.52:0/1912856044 learned_addr learned my addr 172.21.15.52:0/1912856044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:57.832 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157bfff700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f157c06aba0 0x7f157c1014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158092b700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c108570 0x7f157c101f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 -- 172.21.15.52:0/1912856044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c108570 msgr2=0x7f157c101f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c108570 0x7f157c101f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 -- 172.21.15.52:0/1912856044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f157c06aba0 msgr2=0x7f157c1014f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f157c06aba0 0x7f157c1014f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 -- 172.21.15.52:0/1912856044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f157c1a7060 con 0x7f157c06b560 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f158092b700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c108570 0x7f157c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.828+0000 7f157b7fe700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 0x7f157c101a30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f157000d980 tx=0x7f157000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.829+0000 7f15797fa700 1 -- 172.21.15.52:0/1912856044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570017070 con 0x7f157c06b560 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.829+0000 7f15797fa700 1 -- 172.21.15.52:0/1912856044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1570005020 con 0x7f157c06b560 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.829+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f157c1a7350 con 0x7f157c06b560 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.829+0000 7f15797fa700 1 -- 172.21.15.52:0/1912856044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1570009b50 con 0x7f157c06b560 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.829+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f157c1a7840 con 0x7f157c06b560 2024-06-02T23:36:57.833 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.831+0000 7f15797fa700 1 -- 172.21.15.52:0/1912856044 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1570009cf0 con 0x7f157c06b560 2024-06-02T23:36:57.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.832+0000 7f15797fa700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15640a2c20 0x7f15640a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.832+0000 7f15797fa700 1 -- 172.21.15.52:0/1912856044 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f15700cef60 con 0x7f157c06b560 2024-06-02T23:36:57.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.832+0000 7f157bfff700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15640a2c20 0x7f15640a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.832+0000 7f157bfff700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15640a2c20 0x7f15640a50e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b5c0 tx=0x7f156c005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 Processor -- start 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 -- start start 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d9218340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d9212260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d92127a0 0x5570d9212bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570d9104f60 con 0x5570d9217f60 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5570d9163850 con 0x5570d9218880 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158cc11b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570d90cb460 con 0x5570d92127a0 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d9218340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d9218340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40944/0 (socket says 172.21.15.52:40944) 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 -- 172.21.15.52:0/3774556834 learned_addr learned my addr 172.21.15.52:0/3774556834 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157b7fe700 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d9212260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f158092b700 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d92127a0 0x5570d9212bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 -- 172.21.15.52:0/3774556834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d92127a0 msgr2=0x5570d9212bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d92127a0 0x5570d9212bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 -- 172.21.15.52:0/3774556834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 msgr2=0x5570d9212260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d9212260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 -- 172.21.15.52:0/3774556834 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d90b28d0 con 0x5570d9217f60 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.833+0000 7f157bfff700 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d9218340 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f156c009990 tx=0x7f156c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0100397967b58ac server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.834+0000 7f15627fc700 1 -- 172.21.15.52:0/3774556834 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c034070 con 0x5570d9217f60 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.834+0000 7f15627fc700 1 -- 172.21.15.52:0/3774556834 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f156c024930 con 0x5570d9217f60 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.834+0000 7f15627fc700 1 -- 172.21.15.52:0/3774556834 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c02f9c0 con 0x5570d9217f60 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.834+0000 7f158cc11b80 1 -- 172.21.15.52:0/3774556834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 msgr2=0x5570d9218340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.834+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d9218340 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f156c009990 tx=0x7f156c01f000 comp rx=0 tx=0).stop 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- 172.21.15.52:0/3774556834 shutdown_connections 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d92127a0 0x5570d9212bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d9218340 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3774556834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d9212260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- 172.21.15.52:0/3774556834 >> 172.21.15.52:0/3774556834 conn(0x5570d913f080 msgr2=0x5570d920d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- 172.21.15.52:0/3774556834 shutdown_connections 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- 172.21.15.52:0/3774556834 wait complete. 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- start start 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 Processor -- start 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d921e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d921be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5570d9103fc0 con 0x5570d92127a0 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5570d915f810 con 0x5570d9218880 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.835+0000 7f158cc11b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5570d921f200 con 0x5570d9217f60 2024-06-02T23:36:57.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f158cc11b80 10 client.0 _ll_register_callbacks cb 0x5570d90f95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d921e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d921e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40960/0 (socket says 172.21.15.52:40960) 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 -- 172.21.15.52:0/3025369156 learned_addr learned my addr 172.21.15.52:0/3025369156 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f158092b700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d921be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157b7fe700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 -- 172.21.15.52:0/3025369156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 msgr2=0x5570d921b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 -- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 msgr2=0x5570d921be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d921be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 -- 172.21.15.52:0/3025369156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d9129750 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f158092b700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d921be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f157bfff700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d921e650 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f156c005f50 tx=0x7f156c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d60a290513966d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f155b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f155b7fe700 1 -- 172.21.15.52:0/3025369156 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c024db0 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f155b7fe700 1 -- 172.21.15.52:0/3025369156 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f156c02f640 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.836+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f156c024930 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f155b7fe700 1 -- 172.21.15.52:0/3025369156 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f156c02f840 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 10 client.21891 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5570d9229a10 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f155b7fe700 1 -- 172.21.15.52:0/3025369156 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f156c024930 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f155b7fe700 1 -- 172.21.15.52:0/3025369156 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f156c0468c0 con 0x5570d92127a0 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 10 client.21891 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:57.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 10 client.21891 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 20 client.21891 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d922bee0 0x5570d922e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 4 client.21891 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570d9170d50 con 0x5570d922bee0 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.837+0000 7f157b7fe700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d922bee0 0x5570d922e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.838+0000 7f157b7fe700 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d922bee0 0x5570d922e2d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9503c2077f625110 server_cookie=61ebfb080fb0273 in_seq=0 out_seq=0 2024-06-02T23:36:57.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.838+0000 7f155b7fe700 10 client.21891 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.838+0000 7f155b7fe700 1 -- 172.21.15.52:0/3025369156 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5570d9170d50 con 0x5570d922bee0 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.838+0000 7f155b7fe700 10 client.21891 handle_command_reply: tid=0 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 client.21891 shutdown 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 msgr2=0x5570d921e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d921e650 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f156c005f50 tx=0x7f156c009f90 comp rx=0 tx=0).stop 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 shutdown_connections 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d922bee0 0x5570d922e2d0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d921be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d921e650 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 --2- 172.21.15.52:0/3025369156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 >> 172.21.15.52:0/3025369156 conn(0x5570d913f080 msgr2=0x5570d920c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:57.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f155b7fe700 0 client.21891 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:36:57.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 shutdown_connections 2024-06-02T23:36:57.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 1 -- 172.21.15.52:0/3025369156 wait complete. 2024-06-02T23:36:57.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.839+0000 7f158cc11b80 20 client.21891 trim_cache size 0 max 16384 2024-06-02T23:36:57.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.854+0000 7f158cc11b80 1 Processor -- start 2024-06-02T23:36:57.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 -- start start 2024-06-02T23:36:57.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d9212260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d920ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1aa7d0 con 0x5570d92127a0 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1ac310 con 0x5570d9218880 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158cc11b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1aaea0 con 0x5570d9217f60 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d9212260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d9212260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40968/0 (socket says 172.21.15.52:40968) 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 -- 172.21.15.52:0/1264993474 learned_addr learned my addr 172.21.15.52:0/1264993474 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f158092b700 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d920ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157b7fe700 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 -- 172.21.15.52:0/1264993474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 msgr2=0x5570d921a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 -- 172.21.15.52:0/1264993474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 msgr2=0x5570d920ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d920ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 -- 172.21.15.52:0/1264993474 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570d921c890 con 0x5570d92127a0 2024-06-02T23:36:57.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.855+0000 7f157bfff700 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d9212260 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f156c046c90 tx=0x7f156c00bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8019febdd4e0ea38 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:57.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f1561ffb700 1 -- 172.21.15.52:0/1264993474 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c02f9e0 con 0x5570d92127a0 2024-06-02T23:36:57.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f1561ffb700 1 -- 172.21.15.52:0/1264993474 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f156c03be30 con 0x5570d92127a0 2024-06-02T23:36:57.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f1561ffb700 1 -- 172.21.15.52:0/1264993474 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f156c046a20 con 0x5570d92127a0 2024-06-02T23:36:57.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 -- 172.21.15.52:0/1264993474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 msgr2=0x5570d9212260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d9212260 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f156c046c90 tx=0x7f156c00bfb0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 -- 172.21.15.52:0/1264993474 shutdown_connections 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d9218880 0x5570d920ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d92127a0 0x5570d9212260 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 --2- 172.21.15.52:0/1264993474 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9217f60 0x5570d921a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 -- 172.21.15.52:0/1264993474 >> 172.21.15.52:0/1264993474 conn(0x5570d913f080 msgr2=0x5570d921a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 -- 172.21.15.52:0/1264993474 shutdown_connections 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.856+0000 7f158cc11b80 1 -- 172.21.15.52:0/1264993474 wait complete. 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 -- start start 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 Processor -- start 2024-06-02T23:36:57.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d92127a0 0x5570d9231730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d922def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9218880 0x5570d922e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1abc40 con 0x5570d9217f60 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f157c1ab570 con 0x5570d92127a0 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f157c19c7b0 con 0x5570d9218880 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158cc11b80 10 client.0 _ll_register_callbacks cb 0x5570d90f95b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f157b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d922def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f157bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d92127a0 0x5570d9231730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f157b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d922def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40978/0 (socket says 172.21.15.52:40978) 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f157b7fe700 1 -- 172.21.15.52:0/2592328732 learned_addr learned my addr 172.21.15.52:0/2592328732 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.857+0000 7f158092b700 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9218880 0x5570d922e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f157b7fe700 1 -- 172.21.15.52:0/2592328732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9218880 msgr2=0x5570d922e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f157b7fe700 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9218880 0x5570d922e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f157b7fe700 1 -- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d92127a0 msgr2=0x5570d9231730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f157b7fe700 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d92127a0 0x5570d9231730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f157b7fe700 1 -- 172.21.15.52:0/2592328732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570d922ecf0 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f157b7fe700 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d922def0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1570097c50 tx=0x7f15700cc960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76897d181ec332c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f155b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f155b7fe700 1 -- 172.21.15.52:0/2592328732 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15700b8020 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f155b7fe700 1 -- 172.21.15.52:0/2592328732 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1570099af0 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f156c03be30 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f155b7fe700 1 -- 172.21.15.52:0/2592328732 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f157009fba0 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f158cc11b80 10 client.21897 fetch_fsmap learned FSMap version 43 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5570d9295910 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.858+0000 7f155b7fe700 1 -- 172.21.15.52:0/2592328732 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f156c03be30 con 0x5570d9217f60 2024-06-02T23:36:57.864 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f155b7fe700 1 -- 172.21.15.52:0/2592328732 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f15700a3fd0 con 0x5570d9217f60 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 10 client.21897 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 10 client.21897 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 20 client.21897 populate_metadata read hostname 'smithi052' 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d9295fd0 0x5570d92983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 4 client.21897 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570d916b2d0 con 0x5570d9295fd0 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f157bfff700 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d9295fd0 0x5570d92983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f155b7fe700 10 client.21897 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f157bfff700 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d9295fd0 0x5570d92983f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e7d1eca76ecbded8 server_cookie=502410719eda7d0 in_seq=0 out_seq=0 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f155b7fe700 1 -- 172.21.15.52:0/2592328732 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570d916b2d0 con 0x5570d9295fd0 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f155b7fe700 10 client.21897 handle_command_reply: tid=0 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.859+0000 7f158cc11b80 1 client.21897 shutdown 2024-06-02T23:36:57.865 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 msgr2=0x5570d922def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d922def0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f1570097c50 tx=0x7f15700cc960 comp rx=0 tx=0).stop 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 shutdown_connections 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5570d9295fd0 0x5570d92983f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f155b7fe700 0 client.21897 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5570d9218880 0x5570d922e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5570d92127a0 0x5570d9231730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 --2- 172.21.15.52:0/2592328732 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5570d9217f60 0x5570d922def0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 >> 172.21.15.52:0/2592328732 conn(0x5570d913f080 msgr2=0x5570d921da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 shutdown_connections 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 1 -- 172.21.15.52:0/2592328732 wait complete. 2024-06-02T23:36:57.866 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158cc11b80 20 client.21897 trim_cache size 0 max 16384 2024-06-02T23:36:57.867 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:36:57.867 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15640a2c20 msgr2=0x7f15640a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158238e700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15640a2c20 0x7f15640a50e0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f156c00b5c0 tx=0x7f156c005960 comp rx=0 tx=0).stop 2024-06-02T23:36:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 msgr2=0x7f157c101a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:36:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.860+0000 7f158238e700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 0x7f157c101a30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f157000d980 tx=0x7f157000dc90 comp rx=0 tx=0).stop 2024-06-02T23:36:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 shutdown_connections 2024-06-02T23:36:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f15640a2c20 0x7f15640a50e0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.868 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f157c108570 0x7f157c101f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f157c06aba0 0x7f157c1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 --2- 172.21.15.52:0/1912856044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f157c06b560 0x7f157c101a30 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:36:57.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 >> 172.21.15.52:0/1912856044 conn(0x7f157c0773e0 msgr2=0x7f157c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:36:57.869 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 shutdown_connections 2024-06-02T23:36:57.870 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:36:57.861+0000 7f158238e700 1 -- 172.21.15.52:0/1912856044 wait complete. 2024-06-02T23:36:58.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:57 smithi174 ceph-mon[35994]: pgmap v1183: 97 pgs: 97 active+clean; 615 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.7 MiB/s wr, 283 op/s 2024-06-02T23:36:58.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:57 smithi123 ceph-mon[35977]: pgmap v1183: 97 pgs: 97 active+clean; 615 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 928 KiB/s rd, 3.7 MiB/s wr, 283 op/s 2024-06-02T23:36:58.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 606 sec 2024-06-02T23:36:59.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 607 sec 2024-06-02T23:37:00.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:36:59 smithi174 ceph-mon[35994]: pgmap v1184: 97 pgs: 97 active+clean; 603 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.9 MiB/s wr, 246 op/s 2024-06-02T23:37:00.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:36:59 smithi052 ceph-mon[32340]: pgmap v1184: 97 pgs: 97 active+clean; 603 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.9 MiB/s wr, 246 op/s 2024-06-02T23:37:00.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:36:59 smithi123 ceph-mon[35977]: pgmap v1184: 97 pgs: 97 active+clean; 603 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 837 KiB/s rd, 2.9 MiB/s wr, 246 op/s 2024-06-02T23:37:00.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 608 sec 2024-06-02T23:37:00.650 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:37:01.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 609 sec 2024-06-02T23:37:02.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:01 smithi174 ceph-mon[35994]: pgmap v1185: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-06-02T23:37:02.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:01 smithi052 ceph-mon[32340]: pgmap v1185: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-06-02T23:37:02.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:01 smithi123 ceph-mon[35977]: pgmap v1185: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 725 KiB/s rd, 2.7 MiB/s wr, 220 op/s 2024-06-02T23:37:02.379 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:37:02.437 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:02.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 610 sec 2024-06-02T23:37:03.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 611 sec 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.577+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3912963167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 msgr2=0x7f4dc4100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.577+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3912963167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4100dd0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f4dac009900 tx=0x7f4dac02a410 comp rx=0 tx=0).stop 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.577+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3912963167 shutdown_connections 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3912963167 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc4101ba0 0x7f4dc4102020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3912963167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc4102560 0x7f4dc4108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3912963167 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4100dd0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3912963167 >> 172.21.15.52:0/3912963167 conn(0x7f4dc40fbf20 msgr2=0x7f4dc40fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3912963167 shutdown_connections 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3912963167 wait complete. 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 Processor -- start 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- start start 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc4101ba0 0x7f4dc4198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc4102560 0x7f4dc419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410c460 con 0x7f4dc41009b0 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410c2e0 con 0x7f4dc4101ba0 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.578+0000 7f4dcaffb640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc410c5e0 con 0x7f4dc4102560 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:40994/0 (socket says 172.21.15.52:40994) 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3207245430 learned_addr learned my addr 172.21.15.52:0/3207245430 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc4101ba0 0x7f4dc4198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc9598640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc4102560 0x7f4dc419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3207245430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc4102560 msgr2=0x7f4dc419f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc4102560 0x7f4dc419f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3207245430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc4101ba0 msgr2=0x7f4dc4198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc4101ba0 0x7f4dc4198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3207245430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc419f7e0 con 0x7f4dc41009b0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4197bc0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0314e0 tx=0x7f4dac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc1ffb640 1 -- 172.21.15.52:0/3207245430 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac03f030 con 0x7f4dc41009b0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc1ffb640 1 -- 172.21.15.52:0/3207245430 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dac003aa0 con 0x7f4dc41009b0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc419fa70 con 0x7f4dc41009b0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.579+0000 7f4dc1ffb640 1 -- 172.21.15.52:0/3207245430 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac035070 con 0x7f4dc41009b0 2024-06-02T23:37:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.580+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc419ffb0 con 0x7f4dc41009b0 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.582+0000 7f4dc1ffb640 1 -- 172.21.15.52:0/3207245430 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f4dac042030 con 0x7f4dc41009b0 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.582+0000 7f4dcdde0740 1 Processor -- start 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 -- start start 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6010 0x555ca52e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52eb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52ee000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ca52494f0 con 0x555ca52e6a00 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ca51e7870 con 0x555ca52e6010 2024-06-02T23:37:03.588 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555ca52454b0 con 0x555ca52ebb70 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc1ffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4db00a2cb0 0x7f4db00a5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52eb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52eb500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41004/0 (socket says 172.21.15.52:41004) 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3241220156 learned_addr learned my addr 172.21.15.52:0/3241220156 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc9598640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4db00a2cb0 0x7f4db00a5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc1ffb640 1 -- 172.21.15.52:0/3207245430 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4dac0ef650 con 0x7f4dc41009b0 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc9598640 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6010 0x555ca52e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52ee000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3241220156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 msgr2=0x555ca52ee000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52ee000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3241220156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6010 msgr2=0x555ca52e6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6010 0x555ca52e6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc8d97640 1 -- 172.21.15.52:0/3241220156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca52039b0 con 0x555ca52e6a00 2024-06-02T23:37:03.589 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.583+0000 7f4dc9598640 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6010 0x555ca52e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.584+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52eb500 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0b8960 tx=0x7f4dac08f050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0189b915ee3a27 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.584+0000 7f4dc9598640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4db00a2cb0 0x7f4db00a5170 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x555ca52e7260 tx=0x7f4dbc009be0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.584+0000 7f4dbaffd640 1 -- 172.21.15.52:0/3241220156 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac049070 con 0x555ca52e6a00 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.584+0000 7f4dbaffd640 1 -- 172.21.15.52:0/3241220156 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dac0bbf90 con 0x555ca52e6a00 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.584+0000 7f4dbaffd640 1 -- 172.21.15.52:0/3241220156 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac03bca0 con 0x555ca52e6a00 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.584+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3241220156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 msgr2=0x555ca52eb500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52eb500 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0b8960 tx=0x7f4dac08f050 comp rx=0 tx=0).stop 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3241220156 shutdown_connections 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52ee000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52eb500 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3241220156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6010 0x555ca52e6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3241220156 >> 172.21.15.52:0/3241220156 conn(0x555ca52e1680 msgr2=0x555ca52e1a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3241220156 shutdown_connections 2024-06-02T23:37:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3241220156 wait complete. 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- start start 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 Processor -- start 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52effd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52f7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x555ca52ee8e0 con 0x555ca52e6a00 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555ca51d2ae0 con 0x555ca52ebb70 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555ca52eea60 con 0x555ca52e6010 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.585+0000 7f4dcdde0740 10 client.0 _ll_register_callbacks cb 0x555ca520b9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41018/0 (socket says 172.21.15.52:41018) 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 -- 172.21.15.52:0/2554927233 learned_addr learned my addr 172.21.15.52:0/2554927233 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc9598640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 -- 172.21.15.52:0/2554927233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 msgr2=0x555ca52effd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52effd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 -- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 msgr2=0x555ca52f7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52f7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 -- 172.21.15.52:0/2554927233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca4f5d210 con 0x555ca52e6a00 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f0510 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f4db4012720 tx=0x7f4db4012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c72e1b9dab78f0d3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4da3fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4da3fff640 1 -- 172.21.15.52:0/2554927233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db400f8c0 con 0x555ca52e6a00 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4da3fff640 1 -- 172.21.15.52:0/2554927233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4db400fa60 con 0x555ca52e6a00 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dac0bbf90 con 0x555ca52e6a00 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4da3fff640 1 -- 172.21.15.52:0/2554927233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4db400fc60 con 0x555ca52e6a00 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dcdde0740 10 client.21909 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.586+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ca5029620 con 0x555ca52e6a00 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4da3fff640 1 -- 172.21.15.52:0/2554927233 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4dac0bbf90 con 0x555ca52e6a00 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4da3fff640 1 -- 172.21.15.52:0/2554927233 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4db4027420 con 0x555ca52e6a00 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dcdde0740 10 client.21909 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dcdde0740 10 client.21909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dcdde0740 20 client.21909 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca52fbf50 0x555ca52fe3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dcdde0740 4 client.21909 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555ca525cf10 con 0x555ca52fbf50 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.587+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca52fbf50 0x555ca52fe3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca52fbf50 0x555ca52fe3b0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8fb3649457f0c4 server_cookie=ea23e9348ebb08fb in_seq=0 out_seq=0 2024-06-02T23:37:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4da3fff640 10 client.21909 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4da3fff640 1 -- 172.21.15.52:0/2554927233 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x555ca525cf10 con 0x555ca52fbf50 2024-06-02T23:37:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4da3fff640 10 client.21909 handle_command_reply: tid=0 2024-06-02T23:37:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 client.21909 shutdown 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 msgr2=0x555ca52f0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f0510 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f4db4012720 tx=0x7f4db4012be0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 shutdown_connections 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca52fbf50 0x555ca52fe3b0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52f7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f0510 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4da3fff640 0 client.21909 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2554927233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52effd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.588+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 >> 172.21.15.52:0/2554927233 conn(0x555ca52e1680 msgr2=0x555ca52e07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.589+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 shutdown_connections 2024-06-02T23:37:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.589+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2554927233 wait complete. 2024-06-02T23:37:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.589+0000 7f4dcdde0740 20 client.21909 trim_cache size 0 max 16384 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 Processor -- start 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 -- start start 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52dc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f7510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52e4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41a8ca0 con 0x555ca52e6a00 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41aa7e0 con 0x555ca52ebb70 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41a9370 con 0x555ca52e6010 2024-06-02T23:37:03.606 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.604+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41026/0 (socket says 172.21.15.52:41026) 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 -- 172.21.15.52:0/3541185759 learned_addr learned my addr 172.21.15.52:0/3541185759 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc9598640 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52e4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52dc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 -- 172.21.15.52:0/3541185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 msgr2=0x555ca52dc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52dc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 -- 172.21.15.52:0/3541185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 msgr2=0x555ca52e4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52e4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 -- 172.21.15.52:0/3541185759 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555ca52f0e70 con 0x555ca52e6a00 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f7510 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009f40 tx=0x7f4db4009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154c04568e31dc00 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dba7fc640 1 -- 172.21.15.52:0/3541185759 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db402b030 con 0x555ca52e6a00 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dba7fc640 1 -- 172.21.15.52:0/3541185759 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4db4027b50 con 0x555ca52e6a00 2024-06-02T23:37:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.605+0000 7f4dba7fc640 1 -- 172.21.15.52:0/3541185759 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db4005290 con 0x555ca52e6a00 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3541185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 msgr2=0x555ca52f7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f7510 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f4db4009f40 tx=0x7f4db4009640 comp rx=0 tx=0).stop 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3541185759 shutdown_connections 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52ebb70 0x555ca52e4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6a00 0x555ca52f7510 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/3541185759 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52e6010 0x555ca52dc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.611 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3541185759 >> 172.21.15.52:0/3541185759 conn(0x555ca52e1680 msgr2=0x555ca52e4560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3541185759 shutdown_connections 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 -- 172.21.15.52:0/3541185759 wait complete. 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 -- start start 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.606+0000 7f4dcdde0740 1 Processor -- start 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 0x555ca52f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6a00 0x555ca52f5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41aa110 con 0x555ca52e6010 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41a9a40 con 0x555ca52e6a00 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc41a6c80 con 0x555ca52ebb70 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dcdde0740 10 client.0 _ll_register_callbacks cb 0x7f4db4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 0x555ca52f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 0x555ca52f4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:41030/0 (socket says 172.21.15.52:41030) 2024-06-02T23:37:03.612 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 -- 172.21.15.52:0/2774491182 learned_addr learned my addr 172.21.15.52:0/2774491182 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 -- 172.21.15.52:0/2774491182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 msgr2=0x555ca52f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc9598640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6a00 0x555ca52f5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 -- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6a00 msgr2=0x555ca52f5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6a00 0x555ca52f5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 -- 172.21.15.52:0/2774491182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555ca52e14b0 con 0x555ca52e6010 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc9598640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4dc8d97640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 0x555ca52f4ed0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005c40 tx=0x7f4dac02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25933411303269c1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4da3fff640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.607+0000 7f4da3fff640 1 -- 172.21.15.52:0/2774491182 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dac09e070 con 0x555ca52e6010 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db4027b50 con 0x555ca52e6010 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4da3fff640 1 -- 172.21.15.52:0/2774491182 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f4dac0ba030 con 0x555ca52e6010 2024-06-02T23:37:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4da3fff640 1 -- 172.21.15.52:0/2774491182 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4dac049040 con 0x555ca52e6010 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 10 client.21915 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555ca52e2350 con 0x555ca52e6010 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4da3fff640 1 -- 172.21.15.52:0/2774491182 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f4db4027b50 con 0x555ca52e6010 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4da3fff640 1 -- 172.21.15.52:0/2774491182 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f4dac003c50 con 0x555ca52e6010 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 10 client.21915 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 10 client.21915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 20 client.21915 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca53673a0 0x555ca5369790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 4 client.21915 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555ca5256c20 con 0x555ca53673a0 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.608+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca53673a0 0x555ca5369790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.609+0000 7f4dc3fff640 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca53673a0 0x555ca5369790 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa2b46d955de5c7 server_cookie=284949b125e8a0a in_seq=0 out_seq=0 2024-06-02T23:37:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.610+0000 7f4da3fff640 10 client.21915 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4da3fff640 1 -- 172.21.15.52:0/2774491182 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x555ca5256c20 con 0x555ca53673a0 2024-06-02T23:37:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4da3fff640 10 client.21915 handle_command_reply: tid=0 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 client.21915 shutdown 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 msgr2=0x555ca52f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 0x555ca52f4ed0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f4dac005c40 tx=0x7f4dac02a410 comp rx=0 tx=0).stop 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 shutdown_connections 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x555ca53673a0 0x555ca5369790 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555ca52ebb70 0x555ca52f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555ca52e6a00 0x555ca52f5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 --2- 172.21.15.52:0/2774491182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x555ca52e6010 0x555ca52f4ed0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4da3fff640 0 client.21915 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.643+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 >> 172.21.15.52:0/2774491182 conn(0x555ca52e1680 msgr2=0x555ca52dff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 shutdown_connections 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcdde0740 1 -- 172.21.15.52:0/2774491182 wait complete. 2024-06-02T23:37:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcdde0740 20 client.21915 trim_cache size 0 max 16384 2024-06-02T23:37:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:37:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:37:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:37:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4db00a2cb0 msgr2=0x7f4db00a5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4db00a2cb0 0x7f4db00a5170 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x555ca52e7260 tx=0x7f4dbc009be0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 msgr2=0x7f4dc4197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4197bc0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f4dac0314e0 tx=0x7f4dac004270 comp rx=0 tx=0).stop 2024-06-02T23:37:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 shutdown_connections 2024-06-02T23:37:03.649 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f4db00a2cb0 0x7f4db00a5170 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dc4102560 0x7f4dc419f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc4101ba0 0x7f4dc4198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 --2- 172.21.15.52:0/3207245430 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f4dc41009b0 0x7f4dc4197bc0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 >> 172.21.15.52:0/3207245430 conn(0x7f4dc40fbf20 msgr2=0x7f4dc40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 shutdown_connections 2024-06-02T23:37:03.650 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:03.644+0000 7f4dcaffb640 1 -- 172.21.15.52:0/3207245430 wait complete. 2024-06-02T23:37:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:03 smithi052 ceph-mon[32340]: pgmap v1186: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-06-02T23:37:04.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:03 smithi174 ceph-mon[35994]: pgmap v1186: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-06-02T23:37:04.193 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:04.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:03 smithi123 ceph-mon[35977]: pgmap v1186: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 506 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2024-06-02T23:37:04.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 612 sec 2024-06-02T23:37:05.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 613 sec 2024-06-02T23:37:05.944 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:05 smithi052 ceph-mon[32340]: pgmap v1187: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-06-02T23:37:06.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:05 smithi174 ceph-mon[35994]: pgmap v1187: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-06-02T23:37:06.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:05 smithi123 ceph-mon[35977]: pgmap v1187: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.1 MiB/s wr, 127 op/s 2024-06-02T23:37:06.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 614 sec 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 -- 172.21.15.52:0/3390261998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8108500 msgr2=0x7f1ec81088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/3390261998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8108500 0x7f1ec81088e0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009990 tx=0x7f1eb8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 -- 172.21.15.52:0/3390261998 shutdown_connections 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/3390261998 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 0x7f1ec810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/3390261998 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec8108500 0x7f1ec81088e0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/3390261998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ec806a170 0x7f1ec806a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 -- 172.21.15.52:0/3390261998 >> 172.21.15.52:0/3390261998 conn(0x7f1ec8078340 msgr2=0x7f1ec8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 -- 172.21.15.52:0/3390261998 shutdown_connections 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 -- 172.21.15.52:0/3390261998 wait complete. 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.530+0000 7f1ecde1f700 1 Processor -- start 2024-06-02T23:37:06.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 -- start start 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 0x7f1ec8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 0x7f1ec81019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ec8108500 0x7f1ec8101ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec8112630 con 0x7f1ec806a170 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81124b0 con 0x7f1ec8108500 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ecde1f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec81127b0 con 0x7f1ec806ab30 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 0x7f1ec8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 0x7f1ec8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38430/0 (socket says 172.21.15.52:38430) 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 -- 172.21.15.52:0/27291413 learned_addr learned my addr 172.21.15.52:0/27291413 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec7fff700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ec8108500 0x7f1ec8101ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec6ffd700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 0x7f1ec81019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 -- 172.21.15.52:0/27291413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 msgr2=0x7f1ec81019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 0x7f1ec81019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 -- 172.21.15.52:0/27291413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ec8108500 msgr2=0x7f1ec8101ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ec8108500 0x7f1ec8101ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec77fe700 1 -- 172.21.15.52:0/27291413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec81a70b0 con 0x7f1ec806a170 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.531+0000 7f1ec6ffd700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 0x7f1ec81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.532+0000 7f1ec77fe700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 0x7f1ec8101460 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8000c00 tx=0x7f1eb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.532+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8027070 con 0x7f1ec806a170 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.532+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1eb8022470 con 0x7f1ec806a170 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.532+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec81a7340 con 0x7f1ec806a170 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.532+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8017650 con 0x7f1ec806a170 2024-06-02T23:37:06.535 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.532+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec81a77f0 con 0x7f1ec806a170 2024-06-02T23:37:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.533+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea8005330 con 0x7f1ec806a170 2024-06-02T23:37:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.534+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1eb802a070 con 0x7f1ec806a170 2024-06-02T23:37:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.535+0000 7f1ec4ff9700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1eb40a2ca0 0x7f1eb40a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.535+0000 7f1ec6ffd700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1eb40a2ca0 0x7f1eb40a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.535+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1eb80cfed0 con 0x7f1ec806a170 2024-06-02T23:37:06.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.536+0000 7f1ec6ffd700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1eb40a2ca0 0x7f1eb40a5160 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8078990 tx=0x7f1eb0009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.538+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1eb8013070 con 0x7f1ec806a170 2024-06-02T23:37:06.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 -- 172.21.15.52:0/2455591318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b0103150 msgr2=0x7f52b0107140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 --2- 172.21.15.52:0/2455591318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b0103150 0x7f52b0107140 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f52a00099c0 tx=0x7f52a0009cd0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 -- 172.21.15.52:0/2455591318 shutdown_connections 2024-06-02T23:37:06.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 --2- 172.21.15.52:0/2455591318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52b0107870 0x7f52b010f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 --2- 172.21.15.52:0/2455591318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b01027a0 0x7f52b0102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 --2- 172.21.15.52:0/2455591318 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b0103150 0x7f52b0107140 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.776 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 -- 172.21.15.52:0/2455591318 >> 172.21.15.52:0/2455591318 conn(0x7f52b0079520 msgr2=0x7f52b0079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 -- 172.21.15.52:0/2455591318 shutdown_connections 2024-06-02T23:37:06.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.775+0000 7f52b837b700 1 -- 172.21.15.52:0/2455591318 wait complete. 2024-06-02T23:37:06.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 Processor -- start 2024-06-02T23:37:06.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 -- start start 2024-06-02T23:37:06.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 0x7f52b019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.777 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103150 0x7f52b019ec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52b0107870 0x7f52b0198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52b0112630 con 0x7f52b01027a0 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01124b0 con 0x7f52b0103150 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b837b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01127b0 con 0x7f52b0107870 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b6117700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 0x7f52b019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.776+0000 7f52b6117700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 0x7f52b019e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38454/0 (socket says 172.21.15.52:38454) 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 -- 172.21.15.52:0/1610398707 learned_addr learned my addr 172.21.15.52:0/1610398707 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b5916700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103150 0x7f52b019ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6918700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52b0107870 0x7f52b0198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 -- 172.21.15.52:0/1610398707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52b0107870 msgr2=0x7f52b0198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52b0107870 0x7f52b0198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 -- 172.21.15.52:0/1610398707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103150 msgr2=0x7f52b019ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103150 0x7f52b019ec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 -- 172.21.15.52:0/1610398707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52b0199070 con 0x7f52b01027a0 2024-06-02T23:37:06.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b5916700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103150 0x7f52b019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b6117700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 0x7f52b019e700 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9a0 tx=0x7f52ac00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52a77fe700 1 -- 172.21.15.52:0/1610398707 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac00cbd0 con 0x7f52b01027a0 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52a77fe700 1 -- 172.21.15.52:0/1610398707 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52ac00cd70 con 0x7f52b01027a0 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52a77fe700 1 -- 172.21.15.52:0/1610398707 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52ac018750 con 0x7f52b01027a0 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52b0199360 con 0x7f52b01027a0 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.777+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52b01a55d0 con 0x7f52b01027a0 2024-06-02T23:37:06.780 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.780+0000 7f52a77fe700 1 -- 172.21.15.52:0/1610398707 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f52ac010430 con 0x7f52b01027a0 2024-06-02T23:37:06.781 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.781+0000 7f52a77fe700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f529c0a2ca0 0x7f529c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.781+0000 7f52b5916700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f529c0a2ca0 0x7f529c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.781+0000 7f52a77fe700 1 -- 172.21.15.52:0/1610398707 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f52ac0cf300 con 0x7f52b01027a0 2024-06-02T23:37:06.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.781+0000 7f52c2bfeb80 1 Processor -- start 2024-06-02T23:37:06.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.781+0000 7f52c2bfeb80 1 -- start start 2024-06-02T23:37:06.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.781+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9af60 0x563911c9b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.783 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9b880 0x563911c95260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 0x563911c95bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563911b87f60 con 0x563911c957a0 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563911be6850 con 0x563911c9af60 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563911b4e460 con 0x563911c9b880 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 0x563911c95bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 0x563911c95bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38468/0 (socket says 172.21.15.52:38468) 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 -- 172.21.15.52:0/3599218848 learned_addr learned my addr 172.21.15.52:0/3599218848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b5916700 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9b880 0x563911c95260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6117700 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9af60 0x563911c9b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b5916700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f529c0a2ca0 0x7f529c0a5160 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f52b01020d0 tx=0x7f52a0009560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 -- 172.21.15.52:0/3599218848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9b880 msgr2=0x563911c95260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9b880 0x563911c95260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 -- 172.21.15.52:0/3599218848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9af60 msgr2=0x563911c9b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.784 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9af60 0x563911c9b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.782+0000 7f52b6918700 1 -- 172.21.15.52:0/3599218848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563911b358d0 con 0x563911c957a0 2024-06-02T23:37:06.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.783+0000 7f52b6918700 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 0x563911c95bc0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009dc0 tx=0x7f52a80077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe47b81e33f6ab48 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.783+0000 7f52a4d70700 1 -- 172.21.15.52:0/3599218848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8014610 con 0x563911c957a0 2024-06-02T23:37:06.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.783+0000 7f52a4d70700 1 -- 172.21.15.52:0/3599218848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52a8019070 con 0x563911c957a0 2024-06-02T23:37:06.785 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.783+0000 7f52a4d70700 1 -- 172.21.15.52:0/3599218848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8013930 con 0x563911c957a0 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/3599218848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 msgr2=0x563911c95bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 0x563911c95bc0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f52a8009dc0 tx=0x7f52a80077e0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/3599218848 shutdown_connections 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c957a0 0x563911c95bc0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9af60 0x563911c9b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/3599218848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9b880 0x563911c95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/3599218848 >> 172.21.15.52:0/3599218848 conn(0x563911bc2080 msgr2=0x563911c90220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.788 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/3599218848 shutdown_connections 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/3599218848 wait complete. 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 -- start start 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.784+0000 7f52c2bfeb80 1 Processor -- start 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911ca15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c9ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x563911b86fc0 con 0x563911c9af60 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563911be2810 con 0x563911c9b880 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563911ca2260 con 0x563911c957a0 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52c2bfeb80 10 client.0 _ll_register_callbacks cb 0x563911b7c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:06.789 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38476/0 (socket says 172.21.15.52:38476) 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 -- 172.21.15.52:0/4255921287 learned_addr learned my addr 172.21.15.52:0/4255921287 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b6918700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c9ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b6117700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911ca15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 -- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 msgr2=0x563911ca15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911ca15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 -- 172.21.15.52:0/4255921287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 msgr2=0x563911c9ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c9ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 -- 172.21.15.52:0/4255921287 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563911bac750 con 0x563911c9af60 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f52b5916700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9e8c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009750 tx=0x7f52a001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3412406f64406771 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.785+0000 7f5299ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f5299ffb700 1 -- 172.21.15.52:0/4255921287 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a000b9a0 con 0x563911c9af60 2024-06-02T23:37:06.790 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f5299ffb700 1 -- 172.21.15.52:0/4255921287 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f52a000bb40 con 0x563911c9af60 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a8019070 con 0x563911c9af60 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f5299ffb700 1 -- 172.21.15.52:0/4255921287 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f52a000bd40 con 0x563911c9af60 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f52c2bfeb80 10 client.21933 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563911cac9e0 con 0x563911c9af60 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f5299ffb700 1 -- 172.21.15.52:0/4255921287 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52a8019070 con 0x563911c9af60 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f5299ffb700 1 -- 172.21.15.52:0/4255921287 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f52a0038500 con 0x563911c9af60 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f52c2bfeb80 10 client.21933 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.786+0000 7f52c2bfeb80 10 client.21933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f52c2bfeb80 20 client.21933 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911caf020 0x563911cb1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f52c2bfeb80 4 client.21933 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563911bf3d50 con 0x563911caf020 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f52b6117700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911caf020 0x563911cb1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.791 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f52b6117700 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911caf020 0x563911cb1410 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f55f164115e37f9d server_cookie=2d04bfc9f1847fde in_seq=0 out_seq=0 2024-06-02T23:37:06.792 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.787+0000 7f5299ffb700 10 client.21933 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.788+0000 7f5299ffb700 1 -- 172.21.15.52:0/4255921287 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x563911bf3d50 con 0x563911caf020 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.788+0000 7f5299ffb700 10 client.21933 handle_command_reply: tid=0 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.788+0000 7f52c2bfeb80 1 client.21933 shutdown 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.788+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 msgr2=0x563911c9e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.788+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9e8c0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009750 tx=0x7f52a001a050 comp rx=0 tx=0).stop 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 shutdown_connections 2024-06-02T23:37:06.793 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911caf020 0x563911cb1410 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c9ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911ca15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/4255921287 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9e8c0 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 >> 172.21.15.52:0/4255921287 conn(0x563911bc2080 msgr2=0x563911c8f700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f5299ffb700 0 client.21933 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 shutdown_connections 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/4255921287 wait complete. 2024-06-02T23:37:06.794 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.789+0000 7f52c2bfeb80 20 client.21933 trim_cache size 0 max 16384 2024-06-02T23:37:06.802 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.801+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea8005620 con 0x7f1ec806a170 2024-06-02T23:37:06.803 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.802+0000 7f1ec4ff9700 1 -- 172.21.15.52:0/27291413 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11939 (secure 0 0 0) 0x7f1eb809ce90 con 0x7f1ec806a170 2024-06-02T23:37:06.803 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:37:06.803 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":43,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:28:05.537914+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24470,24449,24451,24436,24434]},"id":1}]} 2024-06-02T23:37:06.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1eb40a2ca0 msgr2=0x7f1eb40a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1eb40a2ca0 0x7f1eb40a5160 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8078990 tx=0x7f1eb0009260 comp rx=0 tx=0).stop 2024-06-02T23:37:06.805 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 msgr2=0x7f1ec8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 0x7f1ec8101460 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8000c00 tx=0x7f1eb80047b0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 shutdown_connections 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1eb40a2ca0 0x7f1eb40a5160 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ec8108500 0x7f1ec8101ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1ec806a170 0x7f1ec8101460 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 --2- 172.21.15.52:0/27291413 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1ec806ab30 0x7f1ec81019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 >> 172.21.15.52:0/27291413 conn(0x7f1ec8078340 msgr2=0x7f1ec81001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 shutdown_connections 2024-06-02T23:37:06.806 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.805+0000 7f1ecde1f700 1 -- 172.21.15.52:0/27291413 wait complete. 2024-06-02T23:37:06.807 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 43 2024-06-02T23:37:06.808 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 Processor -- start 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 -- start start 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911c95260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9c370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c8fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01aa860 con 0x563911c9af60 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ac3a0 con 0x563911c9b880 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01aaf30 con 0x563911c957a0 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52b5916700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.807+0000 7f52b5916700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38482/0 (socket says 172.21.15.52:38482) 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 -- 172.21.15.52:0/1561489525 learned_addr learned my addr 172.21.15.52:0/1561489525 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b6918700 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c8fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.809 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b6117700 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911c95260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 -- 172.21.15.52:0/1561489525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 msgr2=0x563911c95260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911c95260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 -- 172.21.15.52:0/1561489525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 msgr2=0x563911c8fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c8fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 -- 172.21.15.52:0/1561489525 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563911c9c8b0 con 0x563911c9af60 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b6918700 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c8fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f52b5916700 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9c370 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005850 tx=0x7f52a0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=552becd598e5142d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f529bfff700 1 -- 172.21.15.52:0/1561489525 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0005920 con 0x563911c9af60 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f529bfff700 1 -- 172.21.15.52:0/1561489525 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52a0033070 con 0x563911c9af60 2024-06-02T23:37:06.810 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.808+0000 7f529bfff700 1 -- 172.21.15.52:0/1561489525 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a002fa00 con 0x563911c9af60 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/1561489525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 msgr2=0x563911c9c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9c370 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005850 tx=0x7f52a0005c00 comp rx=0 tx=0).stop 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/1561489525 shutdown_connections 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c9b880 0x563911c8fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c957a0 0x563911c95260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/1561489525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9af60 0x563911c9c370 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/1561489525 >> 172.21.15.52:0/1561489525 conn(0x563911bc2080 msgr2=0x563911c9bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.814 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/1561489525 shutdown_connections 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.809+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/1561489525 wait complete. 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 -- start start 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 Processor -- start 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c957a0 0x563911ca7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9af60 0x563911ca7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 0x563911cb6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01abcd0 con 0x563911c9b880 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01ab600 con 0x563911c957a0 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52b01a8800 con 0x563911c9af60 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52c2bfeb80 10 client.0 _ll_register_callbacks cb 0x563911b7c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52b6918700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 0x563911cb6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52b6918700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 0x563911cb6110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38488/0 (socket says 172.21.15.52:38488) 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52b6918700 1 -- 172.21.15.52:0/708961347 learned_addr learned my addr 172.21.15.52:0/708961347 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:06.815 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52b6117700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c957a0 0x563911ca7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.810+0000 7f52b5916700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9af60 0x563911ca7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6918700 1 -- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9af60 msgr2=0x563911ca7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6918700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9af60 0x563911ca7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6918700 1 -- 172.21.15.52:0/708961347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c957a0 msgr2=0x563911ca7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6918700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c957a0 0x563911ca7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6918700 1 -- 172.21.15.52:0/708961347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563911cb6950 con 0x563911c9b880 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6117700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c957a0 0x563911ca7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52b6918700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 0x563911cb6110 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f52a8000f80 tx=0x7f52a8013860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de97cd5a060feca server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f5299ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f5299ffb700 1 -- 172.21.15.52:0/708961347 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a8013370 con 0x563911c9b880 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a0033070 con 0x563911c9b880 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f5299ffb700 1 -- 172.21.15.52:0/708961347 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f52a8013c10 con 0x563911c9b880 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f5299ffb700 1 -- 172.21.15.52:0/708961347 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f52a8013e10 con 0x563911c9b880 2024-06-02T23:37:06.816 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52c2bfeb80 10 client.21939 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.811+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563911d19730 con 0x563911c9b880 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f5299ffb700 1 -- 172.21.15.52:0/708961347 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f52a0033070 con 0x563911c9b880 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f5299ffb700 1 -- 172.21.15.52:0/708961347 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7f52a8033420 con 0x563911c9b880 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52c2bfeb80 10 client.21939 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52c2bfeb80 10 client.21939 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52c2bfeb80 20 client.21939 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911d19d40 0x563911d1c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52c2bfeb80 4 client.21939 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563911bee2d0 con 0x563911d19d40 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.812+0000 7f52b6117700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911d19d40 0x563911d1c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f5299ffb700 10 client.21939 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52b6117700 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911d19d40 0x563911d1c160 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f65e29bae6968fe7 server_cookie=3b5a845479e23e1b in_seq=0 out_seq=0 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f5299ffb700 1 -- 172.21.15.52:0/708961347 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563911bee2d0 con 0x563911d19d40 2024-06-02T23:37:06.817 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f5299ffb700 10 client.21939 handle_command_reply: tid=0 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 client.21939 shutdown 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 msgr2=0x563911cb6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 0x563911cb6110 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f52a8000f80 tx=0x7f52a8013860 comp rx=0 tx=0).stop 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 shutdown_connections 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x563911d19d40 0x563911d1c160 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f5299ffb700 0 client.21939 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x563911c9b880 0x563911cb6110 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563911c957a0 0x563911ca7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 --2- 172.21.15.52:0/708961347 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563911c9af60 0x563911ca7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 >> 172.21.15.52:0/708961347 conn(0x563911bc2080 msgr2=0x563911ca00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 shutdown_connections 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 1 -- 172.21.15.52:0/708961347 wait complete. 2024-06-02T23:37:06.818 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.813+0000 7f52c2bfeb80 20 client.21939 trim_cache size 0 max 16384 2024-06-02T23:37:06.819 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:37:06.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f529c0a2ca0 msgr2=0x7f529c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.819 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f529c0a2ca0 0x7f529c0a5160 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f52b01020d0 tx=0x7f52a0009560 comp rx=0 tx=0).stop 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 msgr2=0x7f52b019e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 0x7f52b019e700 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f52ac00e9a0 tx=0x7f52ac00ed60 comp rx=0 tx=0).stop 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 shutdown_connections 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f529c0a2ca0 0x7f529c0a5160 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52b0107870 0x7f52b0198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f52b01027a0 0x7f52b019e700 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 --2- 172.21.15.52:0/1610398707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52b0103150 0x7f52b019ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 >> 172.21.15.52:0/1610398707 conn(0x7f52b0079520 msgr2=0x7f52b01007c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 shutdown_connections 2024-06-02T23:37:06.820 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:06.814+0000 7f52b837b700 1 -- 172.21.15.52:0/1610398707 wait complete. 2024-06-02T23:37:07.059 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:06 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/27291413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:07.128 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:06 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/27291413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:07.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:06 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/27291413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:07.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 615 sec 2024-06-02T23:37:07.967 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:37:08.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:07 smithi174 ceph-mon[35994]: pgmap v1188: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 703 KiB/s wr, 83 op/s 2024-06-02T23:37:08.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:07 smithi052 ceph-mon[32340]: pgmap v1188: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 703 KiB/s wr, 83 op/s 2024-06-02T23:37:08.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:07 smithi123 ceph-mon[35977]: pgmap v1188: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 703 KiB/s wr, 83 op/s 2024-06-02T23:37:08.463 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --set-id 6ef55db8 --release --await' 2024-06-02T23:37:08.464 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --set-id 6ef55db8 --release --await 2024-06-02T23:37:08.484 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 616 sec 2024-06-02T23:37:08.556 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:09.485 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 617 sec 2024-06-02T23:37:09.876 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:10.113 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:09 smithi052 ceph-mon[32340]: pgmap v1189: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 346 KiB/s wr, 40 op/s 2024-06-02T23:37:10.146 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:09 smithi174 ceph-mon[35994]: pgmap v1189: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 346 KiB/s wr, 40 op/s 2024-06-02T23:37:10.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:09 smithi123 ceph-mon[35977]: pgmap v1189: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 346 KiB/s wr, 40 op/s 2024-06-02T23:37:10.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 -- 172.21.15.52:0/1347194472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358102770 msgr2=0x7fb358102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/1347194472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358102770 0x7fb358102b50 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb3480099c0 tx=0x7fb348009cd0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.387 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 -- 172.21.15.52:0/1347194472 shutdown_connections 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/1347194472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb358107860 0x7fb35810f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/1347194472 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358102770 0x7fb358102b50 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/1347194472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358103120 0x7fb358107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 -- 172.21.15.52:0/1347194472 >> 172.21.15.52:0/1347194472 conn(0x7fb358079600 msgr2=0x7fb358079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 -- 172.21.15.52:0/1347194472 shutdown_connections 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 -- 172.21.15.52:0/1347194472 wait complete. 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.385+0000 7fb35c9e7700 1 Processor -- start 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 -- start start 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358102770 0x7fb358074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb358103120 0x7fb358074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 0x7fb358075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb358112680 con 0x7fb358107860 2024-06-02T23:37:10.388 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb358112500 con 0x7fb358103120 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35c9e7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb358112800 con 0x7fb358102770 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 0x7fb358075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 0x7fb358075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38500/0 (socket says 172.21.15.52:38500) 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 -- 172.21.15.52:0/3261758044 learned_addr learned my addr 172.21.15.52:0/3261758044 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35659c700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358102770 0x7fb358074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb355d9b700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb358103120 0x7fb358074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 -- 172.21.15.52:0/3261758044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358102770 msgr2=0x7fb358074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358102770 0x7fb358074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 -- 172.21.15.52:0/3261758044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb358103120 msgr2=0x7fb358074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb358103120 0x7fb358074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 -- 172.21.15.52:0/3261758044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3581a6fe0 con 0x7fb358107860 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb35659c700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358102770 0x7fb358074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:10.389 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.386+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 0x7fb358075210 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb350007d70 tx=0x7fb35000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:10.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.387+0000 7fb3477fe700 1 -- 172.21.15.52:0/3261758044 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350016020 con 0x7fb358107860 2024-06-02T23:37:10.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.387+0000 7fb3477fe700 1 -- 172.21.15.52:0/3261758044 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb35000ad40 con 0x7fb358107860 2024-06-02T23:37:10.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.387+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3581a72d0 con 0x7fb358107860 2024-06-02T23:37:10.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.387+0000 7fb3477fe700 1 -- 172.21.15.52:0/3261758044 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3500115e0 con 0x7fb358107860 2024-06-02T23:37:10.390 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.387+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3581a7810 con 0x7fb358107860 2024-06-02T23:37:10.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.389+0000 7fb3477fe700 1 -- 172.21.15.52:0/3261758044 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fb350019020 con 0x7fb358107860 2024-06-02T23:37:10.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.390+0000 7fb3477fe700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3400a2ca0 0x7fb3400a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.390+0000 7fb35659c700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3400a2ca0 0x7fb3400a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.390+0000 7fb3477fe700 1 -- 172.21.15.52:0/3261758044 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb3500cef30 con 0x7fb358107860 2024-06-02T23:37:10.392 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.390+0000 7fb35659c700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3400a2ca0 0x7fb3400a5160 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb348009fd0 tx=0x7fb348005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:10.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.390+0000 7fb36726ab80 1 Processor -- start 2024-06-02T23:37:10.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 -- start start 2024-06-02T23:37:10.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99241340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9923b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9923bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.393 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564e9912df60 con 0x564e99241880 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564e9918c850 con 0x564e99240f60 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb36726ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e990f4460 con 0x564e9923b7a0 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9923b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9923b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38502/0 (socket says 172.21.15.52:38502) 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 -- 172.21.15.52:0/1955729897 learned_addr learned my addr 172.21.15.52:0/1955729897 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb35659c700 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99241340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb356d9d700 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9923bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 -- 172.21.15.52:0/1955729897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 msgr2=0x564e9923bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9923bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 -- 172.21.15.52:0/1955729897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 msgr2=0x564e99241340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99241340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 -- 172.21.15.52:0/1955729897 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e990db8d0 con 0x564e99241880 2024-06-02T23:37:10.394 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb356d9d700 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9923bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:10.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.391+0000 7fb355d9b700 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9923b260 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00ac40 tx=0x7fb34c00af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12826acc1659979b server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:10.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb344ff9700 1 -- 172.21.15.52:0/1955729897 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c01d070 con 0x564e99241880 2024-06-02T23:37:10.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb344ff9700 1 -- 172.21.15.52:0/1955729897 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34c00fe90 con 0x564e99241880 2024-06-02T23:37:10.395 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb344ff9700 1 -- 172.21.15.52:0/1955729897 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c010800 con 0x564e99241880 2024-06-02T23:37:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 -- 172.21.15.52:0/1955729897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 msgr2=0x564e9923b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.398 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9923b260 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fb34c00ac40 tx=0x7fb34c00af50 comp rx=0 tx=0).stop 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 -- 172.21.15.52:0/1955729897 shutdown_connections 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9923bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99241340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 --2- 172.21.15.52:0/1955729897 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9923b260 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 -- 172.21.15.52:0/1955729897 >> 172.21.15.52:0/1955729897 conn(0x564e99168080 msgr2=0x564e99236220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 -- 172.21.15.52:0/1955729897 shutdown_connections 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.392+0000 7fb36726ab80 1 -- 172.21.15.52:0/1955729897 wait complete. 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 -- start start 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 Processor -- start 2024-06-02T23:37:10.399 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 0x564e99247610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99244910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e99241880 0x564e99244e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564e9912cfc0 con 0x564e9923b7a0 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564e99188810 con 0x564e99240f60 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e99248260 con 0x564e99241880 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb35659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 0x564e99247610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb35659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 0x564e99247610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38516/0 (socket says 172.21.15.52:38516) 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb35659c700 1 -- 172.21.15.52:0/3825567127 learned_addr learned my addr 172.21.15.52:0/3825567127 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb36726ab80 10 client.0 _ll_register_callbacks cb 0x564e991225b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e99241880 0x564e99244e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.393+0000 7fb355d9b700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99244910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.400 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb35659c700 1 -- 172.21.15.52:0/3825567127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e99241880 msgr2=0x564e99244e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb35659c700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e99241880 0x564e99244e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb35659c700 1 -- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 msgr2=0x564e99244910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb35659c700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99244910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.401 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb35659c700 1 -- 172.21.15.52:0/3825567127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e99152750 con 0x564e9923b7a0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb35659c700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 0x564e99247610 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fb3480098e0 tx=0x7fb348009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a528ca959d7c226 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb33dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb33dffb700 1 -- 172.21.15.52:0/3825567127 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb348003890 con 0x564e9923b7a0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb33dffb700 1 -- 172.21.15.52:0/3825567127 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb348003a30 con 0x564e9923b7a0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb34c00fe90 con 0x564e9923b7a0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb33dffb700 1 -- 172.21.15.52:0/3825567127 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb348038070 con 0x564e9923b7a0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb36726ab80 10 client.21951 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564e99252980 con 0x564e9923b7a0 2024-06-02T23:37:10.402 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.394+0000 7fb33dffb700 1 -- 172.21.15.52:0/3825567127 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34c00fe90 con 0x564e9923b7a0 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb33dffb700 1 -- 172.21.15.52:0/3825567127 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb34802f020 con 0x564e9923b7a0 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb36726ab80 10 client.21951 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb36726ab80 10 client.21951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb36726ab80 20 client.21951 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992573d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb36726ab80 4 client.21951 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:10.403 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e99199d50 con 0x564e99254fc0 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb355d9b700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb355d9b700 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992573d0 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7ae19d98f918527 server_cookie=8349fe32d079a44f in_seq=0 out_seq=0 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.395+0000 7fb33dffb700 10 client.21951 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb33dffb700 1 -- 172.21.15.52:0/3825567127 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564e99199d50 con 0x564e99254fc0 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb33dffb700 10 client.21951 handle_command_reply: tid=0 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb36726ab80 1 client.21951 shutdown 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 msgr2=0x564e99247610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 0x564e99247610 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fb3480098e0 tx=0x7fb348009cd0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 shutdown_connections 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.396+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992573d0 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e99241880 0x564e99244e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb33dffb700 0 client.21951 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e9923b7a0 0x564e99247610 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.404 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3825567127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e99244910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 >> 172.21.15.52:0/3825567127 conn(0x564e99168080 msgr2=0x564e99235700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:10.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 shutdown_connections 2024-06-02T23:37:10.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 1 -- 172.21.15.52:0/3825567127 wait complete. 2024-06-02T23:37:10.405 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.397+0000 7fb36726ab80 20 client.21951 trim_cache size 0 max 16384 2024-06-02T23:37:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 Processor -- start 2024-06-02T23:37:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 -- start start 2024-06-02T23:37:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e992422d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 0x564e99235e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.419 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99241880 0x564e992363c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581aec90 con 0x564e99240f60 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b07d0 con 0x564e99241880 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb36726ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581af360 con 0x564e9923b7a0 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb355d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 0x564e99235e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb355d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 0x564e99235e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38526/0 (socket says 172.21.15.52:38526) 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.412+0000 7fb355d9b700 1 -- 172.21.15.52:0/614878995 learned_addr learned my addr 172.21.15.52:0/614878995 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb35659c700 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e992422d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb356d9d700 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99241880 0x564e992363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb355d9b700 1 -- 172.21.15.52:0/614878995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 msgr2=0x564e992422d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb355d9b700 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e992422d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb355d9b700 1 -- 172.21.15.52:0/614878995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99241880 msgr2=0x564e992363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb355d9b700 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99241880 0x564e992363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.420 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb355d9b700 1 -- 172.21.15.52:0/614878995 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e99242e80 con 0x564e99240f60 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb356d9d700 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99241880 0x564e992363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb355d9b700 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 0x564e99235e80 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb34c01b040 tx=0x7fb34c019ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69690b32391986d2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb33ffff700 1 -- 172.21.15.52:0/614878995 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c01d040 con 0x564e99240f60 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb33ffff700 1 -- 172.21.15.52:0/614878995 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34c01e040 con 0x564e99240f60 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.413+0000 7fb33ffff700 1 -- 172.21.15.52:0/614878995 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb34c004950 con 0x564e99240f60 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 -- 172.21.15.52:0/614878995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 msgr2=0x564e99235e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 0x564e99235e80 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7fb34c01b040 tx=0x7fb34c019ea0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 -- 172.21.15.52:0/614878995 shutdown_connections 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99241880 0x564e992363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e992422d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 --2- 172.21.15.52:0/614878995 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99240f60 0x564e99235e80 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 -- 172.21.15.52:0/614878995 >> 172.21.15.52:0/614878995 conn(0x564e99168080 msgr2=0x564e992469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:10.421 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 -- 172.21.15.52:0/614878995 shutdown_connections 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 -- 172.21.15.52:0/614878995 wait complete. 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 -- start start 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.414+0000 7fb36726ab80 1 Processor -- start 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9924db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e9925c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9925c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581b0100 con 0x564e99241880 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581afa30 con 0x564e99240f60 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3581acc50 con 0x564e9923b7a0 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb36726ab80 10 client.0 _ll_register_callbacks cb 0x564e991225b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9925c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9925c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38528/0 (socket says 172.21.15.52:38528) 2024-06-02T23:37:10.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 -- 172.21.15.52:0/3948569538 learned_addr learned my addr 172.21.15.52:0/3948569538 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb355d9b700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e9925c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb35659c700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9924db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 -- 172.21.15.52:0/3948569538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 msgr2=0x564e9924db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9924db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 -- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 msgr2=0x564e9925c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e9925c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 -- 172.21.15.52:0/3948569538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9925d090 con 0x564e99241880 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb356d9d700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9925c9e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fb350006130 tx=0x7fb35009b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8553131d0ccdb1af server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb33dffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.415+0000 7fb33dffb700 1 -- 172.21.15.52:0/3948569538 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb350018d80 con 0x564e99241880 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb33dffb700 1 -- 172.21.15.52:0/3948569538 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fb3500860b0 con 0x564e99241880 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb34c01e040 con 0x564e99241880 2024-06-02T23:37:10.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb33dffb700 1 -- 172.21.15.52:0/3948569538 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fb3500ba170 con 0x564e99241880 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 10 client.21957 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564e992bfae0 con 0x564e99241880 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb33dffb700 1 -- 172.21.15.52:0/3948569538 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fb34c01e040 con 0x564e99241880 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb33dffb700 1 -- 172.21.15.52:0/3948569538 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fb3500974b0 con 0x564e99241880 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 10 client.21957 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 10 client.21957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 20 client.21957 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992c21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 4 client.21957 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e991942d0 con 0x564e99254fc0 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.416+0000 7fb35659c700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992c21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb33dffb700 10 client.21957 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb35659c700 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992c21b0 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c3bfe9f9ca43b56 server_cookie=5c52a77ffb9c504c in_seq=0 out_seq=0 2024-06-02T23:37:10.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb33dffb700 1 -- 172.21.15.52:0/3948569538 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e991942d0 con 0x564e99254fc0 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb33dffb700 10 client.21957 handle_command_reply: tid=0 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 client.21957 shutdown 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 msgr2=0x564e9925c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9925c9e0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7fb350006130 tx=0x7fb35009b890 comp rx=0 tx=0).stop 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 shutdown_connections 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564e99254fc0 0x564e992c21b0 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb33dffb700 0 client.21957 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564e99241880 0x564e9925c9e0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e9923b7a0 0x564e9924db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 --2- 172.21.15.52:0/3948569538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564e99240f60 0x564e9925c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 >> 172.21.15.52:0/3948569538 conn(0x564e99168080 msgr2=0x564e992469d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 shutdown_connections 2024-06-02T23:37:10.425 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 1 -- 172.21.15.52:0/3948569538 wait complete. 2024-06-02T23:37:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.417+0000 7fb36726ab80 20 client.21957 trim_cache size 0 max 16384 2024-06-02T23:37:10.426 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:37:10.426 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3400a2ca0 msgr2=0x7fb3400a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3400a2ca0 0x7fb3400a5160 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fb348009fd0 tx=0x7fb348005fb0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 msgr2=0x7fb358075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 0x7fb358075210 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7fb350007d70 tx=0x7fb35000a340 comp rx=0 tx=0).stop 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 shutdown_connections 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fb3400a2ca0 0x7fb3400a5160 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fb358107860 0x7fb358075210 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb358102770 0x7fb358074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 --2- 172.21.15.52:0/3261758044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb358103120 0x7fb358074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 >> 172.21.15.52:0/3261758044 conn(0x7fb358079600 msgr2=0x7fb358100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 shutdown_connections 2024-06-02T23:37:10.427 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:10.418+0000 7fb35c9e7700 1 -- 172.21.15.52:0/3261758044 wait complete. 2024-06-02T23:37:10.485 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 618 sec 2024-06-02T23:37:11.485 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 619 sec 2024-06-02T23:37:12.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:11 smithi174 ceph-mon[35994]: pgmap v1190: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-06-02T23:37:12.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:11 smithi052 ceph-mon[32340]: pgmap v1190: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-06-02T23:37:12.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:11 smithi123 ceph-mon[35977]: pgmap v1190: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 294 KiB/s wr, 28 op/s 2024-06-02T23:37:12.485 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 620 sec 2024-06-02T23:37:12.915 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-06-02T23:37:12.915 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 -- 172.21.15.52:0/3654229960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 msgr2=0x7fafd806b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 --2- 172.21.15.52:0/3654229960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd806b040 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7fafc8009990 tx=0x7fafc8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 -- 172.21.15.52:0/3654229960 shutdown_connections 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 --2- 172.21.15.52:0/3654229960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd806b580 0x7fafd810f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 --2- 172.21.15.52:0/3654229960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd8108500 0x7fafd81088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 --2- 172.21.15.52:0/3654229960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd806b040 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 -- 172.21.15.52:0/3654229960 >> 172.21.15.52:0/3654229960 conn(0x7fafd8078340 msgr2=0x7fafd8078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 -- 172.21.15.52:0/3654229960 shutdown_connections 2024-06-02T23:37:12.936 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.893+0000 7fafdf30c700 1 -- 172.21.15.52:0/3654229960 wait complete. 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 Processor -- start 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 -- start start 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd8101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd806b580 0x7fafd81019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd8108500 0x7fafd8101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafd8112630 con 0x7fafd806abc0 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81124b0 con 0x7fafd806b580 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdf30c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81127b0 con 0x7fafd8108500 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd8101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd8101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38556/0 (socket says 172.21.15.52:38556) 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 -- 172.21.15.52:0/597262657 learned_addr learned my addr 172.21.15.52:0/597262657 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd806b580 0x7fafd81019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.937 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd8108500 0x7fafd8101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 -- 172.21.15.52:0/597262657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd8108500 msgr2=0x7fafd8101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd8108500 0x7fafd8101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 -- 172.21.15.52:0/597262657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd806b580 msgr2=0x7fafd81019d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd806b580 0x7fafd81019d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.894+0000 7fafdd0a8700 1 -- 172.21.15.52:0/597262657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd81a7110 con 0x7fafd806abc0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd8108500 0x7fafd8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd8101490 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fafd400e9a0 tx=0x7fafd400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafce7fc700 1 -- 172.21.15.52:0/597262657 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd400cbd0 con 0x7fafd806abc0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafce7fc700 1 -- 172.21.15.52:0/597262657 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafd400cd70 con 0x7fafd806abc0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd81a7400 con 0x7fafd806abc0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafce7fc700 1 -- 172.21.15.52:0/597262657 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd4018680 con 0x7fafd806abc0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.895+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd81a7910 con 0x7fafd806abc0 2024-06-02T23:37:12.938 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.897+0000 7fafce7fc700 1 -- 172.21.15.52:0/597262657 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fafd4010430 con 0x7fafd806abc0 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.898+0000 7fafce7fc700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafc40a2c20 0x7fafc40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.898+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafc40a2c20 0x7fafc40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.898+0000 7fafce7fc700 1 -- 172.21.15.52:0/597262657 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fafd40cf260 con 0x7fafd806abc0 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafc40a2c20 0x7fafc40a50e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b2e0 tx=0x7fafc8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 Processor -- start 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 -- start start 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140e9020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140e2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140e3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564a14034910 con 0x564a140e9560 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a13fdc820 con 0x564a140e33d0 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a140308d0 con 0x564a140e8c40 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafdc8a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140e2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.939 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.899+0000 7fafdc8a7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140e2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38562/0 (socket says 172.21.15.52:38562) 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 -- 172.21.15.52:0/1248777100 learned_addr learned my addr 172.21.15.52:0/1248777100 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140e3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 -- 172.21.15.52:0/1248777100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 msgr2=0x564a140e9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140e9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 -- 172.21.15.52:0/1248777100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 msgr2=0x564a140e3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140e3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 -- 172.21.15.52:0/1248777100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a13fd3050 con 0x564a140e9560 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140e2e90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fafc8015000 tx=0x7fafc801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620d5f302f94c0b2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafc37fe700 1 -- 172.21.15.52:0/1248777100 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc800be70 con 0x564a140e9560 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafc37fe700 1 -- 172.21.15.52:0/1248777100 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafc800f910 con 0x564a140e9560 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.900+0000 7fafc37fe700 1 -- 172.21.15.52:0/1248777100 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc8026860 con 0x564a140e9560 2024-06-02T23:37:12.940 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/1248777100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 msgr2=0x564a140e2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140e2e90 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fafc8015000 tx=0x7fafc801f000 comp rx=0 tx=0).stop 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/1248777100 shutdown_connections 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140e3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140e9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/1248777100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140e2e90 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/1248777100 >> 172.21.15.52:0/1248777100 conn(0x564a1400ddc0 msgr2=0x564a140de120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/1248777100 shutdown_connections 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/1248777100 wait complete. 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 -- start start 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.901+0000 7fafe9b8fb80 1 Processor -- start 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140ef260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ec560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140ecaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.941 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x564a140eff70 con 0x564a140e9560 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564a13fd4cf0 con 0x564a140e33d0 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564a140f00f0 con 0x564a140e8c40 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafe9b8fb80 10 client.0 _ll_register_callbacks cb 0x564a13fcea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140ecaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140ecaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38576/0 (socket says 172.21.15.52:38576) 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 -- 172.21.15.52:0/381850409 learned_addr learned my addr 172.21.15.52:0/381850409 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ec560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140ef260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 -- 172.21.15.52:0/381850409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 msgr2=0x564a140ec560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ec560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 -- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 msgr2=0x564a140ef260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140ef260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.942 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd8a9700 1 -- 172.21.15.52:0/381850409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a13f839d0 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.902+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140ecaa0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007eb0 tx=0x7fafd0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c25bbc77201c9e1 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafc0ff9700 1 -- 172.21.15.52:0/381850409 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd001a070 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc800f910 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafc0ff9700 1 -- 172.21.15.52:0/381850409 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fafd0016420 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafc0ff9700 1 -- 172.21.15.52:0/381850409 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fafd0026050 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 10 client.21969 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564a140fa910 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafc0ff9700 1 -- 172.21.15.52:0/381850409 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafc800f910 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafc0ff9700 1 -- 172.21.15.52:0/381850409 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fafd0015500 con 0x564a140e9560 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 10 client.21969 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 10 client.21969 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:37:12.943 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 20 client.21969 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a140fcf40 0x564a140ff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 4 client.21969 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.903+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a14041e10 con 0x564a140fcf40 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.904+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a140fcf40 0x564a140ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.904+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a140fcf40 0x564a140ff330 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d827413091c08e2 server_cookie=4e4bcda86c28031e in_seq=0 out_seq=0 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.904+0000 7fafc0ff9700 10 client.21969 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafc0ff9700 1 -- 172.21.15.52:0/381850409 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x564a14041e10 con 0x564a140fcf40 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafc0ff9700 10 client.21969 handle_command_reply: tid=0 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 client.21969 shutdown 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 msgr2=0x564a140ecaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140ecaa0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007eb0 tx=0x7fafd0007f90 comp rx=0 tx=0).stop 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 shutdown_connections 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a140fcf40 0x564a140ff330 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.944 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140ecaa0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140ef260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/381850409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ec560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafc0ff9700 0 client.21969 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 >> 172.21.15.52:0/381850409 conn(0x564a1400ddc0 msgr2=0x564a140e1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 shutdown_connections 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/381850409 wait complete. 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.905+0000 7fafe9b8fb80 20 client.21969 trim_cache size 0 max 16384 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 Processor -- start 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 -- start start 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 0x564a140ecad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ea110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e9560 0x564a14104c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81aeba0 con 0x564a140e33d0 2024-06-02T23:37:12.945 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81b06e0 con 0x564a140e9560 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81af270 con 0x564a140e8c40 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafdd0a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 0x564a140ecad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafdd0a8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 0x564a140ecad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38588/0 (socket says 172.21.15.52:38588) 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafdd0a8700 1 -- 172.21.15.52:0/105002496 learned_addr learned my addr 172.21.15.52:0/105002496 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.927+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e9560 0x564a14104c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ea110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd0a8700 1 -- 172.21.15.52:0/105002496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 msgr2=0x564a140ea110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ea110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd0a8700 1 -- 172.21.15.52:0/105002496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e9560 msgr2=0x564a14104c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e9560 0x564a14104c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd0a8700 1 -- 172.21.15.52:0/105002496 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a140eaad0 con 0x564a140e33d0 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e9560 0x564a14104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:12.946 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 0x564a140ecad0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fafd400cfd0 tx=0x7fafd40ccc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ab38c3f18ad7316 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafc2ffd700 1 -- 172.21.15.52:0/105002496 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd409ba90 con 0x564a140e33d0 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafc2ffd700 1 -- 172.21.15.52:0/105002496 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafd409fe40 con 0x564a140e33d0 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.928+0000 7fafc2ffd700 1 -- 172.21.15.52:0/105002496 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd40a44d0 con 0x564a140e33d0 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/105002496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 msgr2=0x564a140ecad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 0x564a140ecad0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fafd400cfd0 tx=0x7fafd40ccc00 comp rx=0 tx=0).stop 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/105002496 shutdown_connections 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e9560 0x564a14104c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e33d0 0x564a140ecad0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/105002496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140ea110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/105002496 >> 172.21.15.52:0/105002496 conn(0x564a1400ddc0 msgr2=0x564a140e9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:12.947 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/105002496 shutdown_connections 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/105002496 wait complete. 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.929+0000 7fafe9b8fb80 1 -- start start 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 Processor -- start 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140f29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.948 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81b0010 con 0x564a140e9560 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81af940 con 0x564a140e33d0 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafd81acb60 con 0x564a140e8c40 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafdd8a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140f29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafdd8a9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140f29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38596/0 (socket says 172.21.15.52:38596) 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafdd8a9700 1 -- 172.21.15.52:0/617897100 learned_addr learned my addr 172.21.15.52:0/617897100 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafe9b8fb80 10 client.0 _ll_register_callbacks cb 0x564a13fcea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.930+0000 7fafdc8a7700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafdd8a9700 1 -- 172.21.15.52:0/617897100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 msgr2=0x564a140f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafdd8a9700 1 -- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 msgr2=0x564a140f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafdd8a9700 1 -- 172.21.15.52:0/617897100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a14167be0 con 0x564a140e9560 2024-06-02T23:37:12.949 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafdd8a9700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140f29f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007c00 tx=0x7fafd0015f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a23844ebb1fc14 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafc0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafc0ff9700 1 -- 172.21.15.52:0/617897100 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd002f060 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd409fe40 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafc0ff9700 1 -- 172.21.15.52:0/617897100 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fafd00153c0 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafc0ff9700 1 -- 172.21.15.52:0/617897100 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fafd001a070 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafe9b8fb80 10 client.21975 fetch_fsmap learned FSMap version 43 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.931+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x564a14168d40 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafc0ff9700 1 -- 172.21.15.52:0/617897100 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fafd409fe40 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafc0ff9700 1 -- 172.21.15.52:0/617897100 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 43) ==== 8354+0+0 (secure 0 0 0) 0x7fafd001f6b0 con 0x564a140e9560 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafe9b8fb80 10 client.21975 fetch_fsmap finished waiting for FSMap version 43 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafe9b8fb80 10 client.21975 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafe9b8fb80 20 client.21975 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a14169420 0x564a1416b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:12.950 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafe9b8fb80 4 client.21975 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6ef55db8", "release": true, "await": true}] 2024-06-02T23:37:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6ef55db8", "release": true, "await": true}) -- 0x564a1403c390 con 0x564a14169420 2024-06-02T23:37:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.932+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a14169420 0x564a1416b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.933+0000 7fafdd0a8700 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a14169420 0x564a1416b840 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=812831d8e722821b server_cookie=3c7a97dfb7284816 in_seq=0 out_seq=0 2024-06-02T23:37:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.933+0000 7fafc0ff9700 10 client.21975 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:12.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafc0ff9700 1 -- 172.21.15.52:0/617897100 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564a1403c390 con 0x564a14169420 2024-06-02T23:37:12.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafc0ff9700 10 client.21975 handle_command_reply: tid=0 2024-06-02T23:37:12.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 client.21975 shutdown 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 msgr2=0x564a140f29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140f29f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007c00 tx=0x7fafd0015f50 comp rx=0 tx=0).stop 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 shutdown_connections 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x564a14169420 0x564a1416b840 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x564a140e9560 0x564a140f29f0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafc0ff9700 0 client.21975 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564a140e33d0 0x564a140f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 --2- 172.21.15.52:0/617897100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564a140e8c40 0x564a140f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.968+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 >> 172.21.15.52:0/617897100 conn(0x564a1400ddc0 msgr2=0x564a140dd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:12.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.969+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 shutdown_connections 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.969+0000 7fafe9b8fb80 1 -- 172.21.15.52:0/617897100 wait complete. 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.969+0000 7fafe9b8fb80 20 client.21975 trim_cache size 0 max 16384 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 43, 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 80, 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": { 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "6ef55db8": { 2024-06-02T23:37:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "version": 80, 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "age_ref": 0.0, 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "timeout": 90.0, 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "expiration": 136.9, 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "members": { 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "file:/": { 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "excluded": false, 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "state": { 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RELEASED", 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "age": 0.0 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafc40a2c20 msgr2=0x7fafc40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafc40a2c20 0x7fafc40a50e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fafc800b2e0 tx=0x7fafc8005960 comp rx=0 tx=0).stop 2024-06-02T23:37:12.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 msgr2=0x7fafd8101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd8101490 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fafd400e9a0 tx=0x7fafd400ecb0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 shutdown_connections 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fafc40a2c20 0x7fafc40a50e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd8108500 0x7fafd8101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fafd806abc0 0x7fafd8101490 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 --2- 172.21.15.52:0/597262657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafd806b580 0x7fafd81019d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 >> 172.21.15.52:0/597262657 conn(0x7fafd8078340 msgr2=0x7fafd81001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 shutdown_connections 2024-06-02T23:37:12.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:12.970+0000 7fafdf30c700 1 -- 172.21.15.52:0/597262657 wait complete. 2024-06-02T23:37:13.485 INFO:tasks.workunit.client.0.smithi052.stdout: 10 cleanup 621 sec 2024-06-02T23:37:13.654 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6ef55db8', total seconds elapsed: 29.9 2024-06-02T23:37:13.661 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:13 smithi052 ceph-mon[32340]: pgmap v1191: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 93 KiB/s wr, 22 op/s 2024-06-02T23:37:14.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:13 smithi174 ceph-mon[35994]: pgmap v1191: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 93 KiB/s wr, 22 op/s 2024-06-02T23:37:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:13 smithi123 ceph-mon[35977]: pgmap v1191: 97 pgs: 97 active+clean; 595 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 93 KiB/s wr, 22 op/s 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: 0 cleanup 622 sec 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: Operation Count AvgLat MaxLat 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: ---------------------------------------- 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: NTCreateX 131132 15.085 33189.378 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: Close 96255 0.207 141.293 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: Rename 5578 36.220 23701.261 2024-06-02T23:37:14.299 INFO:tasks.workunit.client.0.smithi052.stdout: Unlink 26540 15.540 33090.061 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: Qpathinfo 118761 5.890 613.261 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: Qfileinfo 20754 0.411 91.945 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: Qfsinfo 21883 1.022 111.001 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: Sfileinfo 10713 9.459 28511.355 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: Find 46009 44.434 318.679 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: WriteX 64881 6.680 33079.175 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: ReadX 206640 0.015 15.950 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: LockX 432 2.200 64.231 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: UnlockX 432 2.015 33.449 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: Flush 9209 7.653 346.521 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout: 2024-06-02T23:37:14.300 INFO:tasks.workunit.client.0.smithi052.stdout:Throughput 6.8382 MB/sec 10 clients 10 procs max_latency=33189.383 ms 2024-06-02T23:37:14.301 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-06-02T23:37:14.301 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-06-02T23:37:15.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:14 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:15.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:14 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:15.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:14 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 -- 172.21.15.52:0/1365244100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38107400 msgr2=0x7fdc381077e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/1365244100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38107400 0x7fdc381077e0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fdc28009990 tx=0x7fdc28009ca0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 -- 172.21.15.52:0/1365244100 shutdown_connections 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/1365244100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 0x7fdc3810e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/1365244100 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38107400 0x7fdc381077e0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/1365244100 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38101390 0x7fdc38101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 -- 172.21.15.52:0/1365244100 >> 172.21.15.52:0/1365244100 conn(0x7fdc38078340 msgr2=0x7fdc38078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 -- 172.21.15.52:0/1365244100 shutdown_connections 2024-06-02T23:37:15.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.452+0000 7fdc3cc77700 1 -- 172.21.15.52:0/1365244100 wait complete. 2024-06-02T23:37:15.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 Processor -- start 2024-06-02T23:37:15.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 -- start start 2024-06-02T23:37:15.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 0x7fdc381033e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 0x7fdc38103920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38107400 0x7fdc38103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc38111520 con 0x7fdc38101390 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381113a0 con 0x7fdc38107400 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3cc77700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381116a0 con 0x7fdc38101d50 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 0x7fdc381033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 0x7fdc381033e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57368/0 (socket says 172.21.15.52:57368) 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc3659c700 1 -- 172.21.15.52:0/2791219460 learned_addr learned my addr 172.21.15.52:0/2791219460 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38107400 0x7fdc38103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.453+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 0x7fdc38103920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2791219460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 msgr2=0x7fdc38103920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 0x7fdc38103920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 msgr2=0x7fdc381033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 0x7fdc381033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2791219460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc381a7040 con 0x7fdc38107400 2024-06-02T23:37:15.456 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc3659c700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 0x7fdc381033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 0x7fdc38103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38107400 0x7fdc38103e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000efc0 tx=0x7fdc3000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc277fe700 1 -- 172.21.15.52:0/2791219460 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300092e0 con 0x7fdc38107400 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc381a72d0 con 0x7fdc38107400 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc277fe700 1 -- 172.21.15.52:0/2791219460 <== mon.2 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc30004bb0 con 0x7fdc38107400 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc277fe700 1 -- 172.21.15.52:0/2791219460 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc30004020 con 0x7fdc38107400 2024-06-02T23:37:15.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.454+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc381a7810 con 0x7fdc38107400 2024-06-02T23:37:15.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.458+0000 7fdc277fe700 1 -- 172.21.15.52:0/2791219460 <== mon.2 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdc3001b030 con 0x7fdc38107400 2024-06-02T23:37:15.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.458+0000 7fdc277fe700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc200a2c20 0x7fdc200a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.458+0000 7fdc3659c700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc200a2c20 0x7fdc200a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.460 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.458+0000 7fdc277fe700 1 -- 172.21.15.52:0/2791219460 <== mon.2 v2:172.21.15.123:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdc300cea60 con 0x7fdc38107400 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 Processor -- start 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 -- start start 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc3659c700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc200a2c20 0x7fdc200a50e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b2e0 tx=0x7fdc2801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb16c0 0x5615d2eb1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eaba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eabfa0 0x5615d2eac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5615d2dfd890 con 0x5615d2eb1fe0 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615d2da57a0 con 0x5615d2eb16c0 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc474fab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615d2df9850 con 0x5615d2eabfa0 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc35d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eaba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb16c0 0x5615d2eb1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc35d9b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eaba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57380/0 (socket says 172.21.15.52:57380) 2024-06-02T23:37:15.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc35d9b700 1 -- 172.21.15.52:0/3930782684 learned_addr learned my addr 172.21.15.52:0/3930782684 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.459+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eabfa0 0x5615d2eac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc35d9b700 1 -- 172.21.15.52:0/3930782684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eabfa0 msgr2=0x5615d2eac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eabfa0 0x5615d2eac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc35d9b700 1 -- 172.21.15.52:0/3930782684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb16c0 msgr2=0x5615d2eb1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb16c0 0x5615d2eb1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc35d9b700 1 -- 172.21.15.52:0/3930782684 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d2d9bfd0 con 0x5615d2eb1fe0 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eabfa0 0x5615d2eac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc3659c700 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb16c0 0x5615d2eb1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eaba60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c011540 tx=0x7fdc2c011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78282166c4b32045 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc24ff9700 1 -- 172.21.15.52:0/3930782684 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c004a30 con 0x5615d2eb1fe0 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc24ff9700 1 -- 172.21.15.52:0/3930782684 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc2c0054d0 con 0x5615d2eb1fe0 2024-06-02T23:37:15.462 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.460+0000 7fdc24ff9700 1 -- 172.21.15.52:0/3930782684 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc2c00e8e0 con 0x5615d2eb1fe0 2024-06-02T23:37:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 -- 172.21.15.52:0/3930782684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 msgr2=0x5615d2eaba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eaba60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fdc2c011540 tx=0x7fdc2c011850 comp rx=0 tx=0).stop 2024-06-02T23:37:15.465 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 -- 172.21.15.52:0/3930782684 shutdown_connections 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eabfa0 0x5615d2eac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb16c0 0x5615d2eb1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 --2- 172.21.15.52:0/3930782684 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eaba60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 -- 172.21.15.52:0/3930782684 >> 172.21.15.52:0/3930782684 conn(0x5615d2dd6d40 msgr2=0x5615d2ea68d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 -- 172.21.15.52:0/3930782684 shutdown_connections 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 -- 172.21.15.52:0/3930782684 wait complete. 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.461+0000 7fdc474fab80 1 -- start start 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 Processor -- start 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 0x5615d2eb7d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5615d2eb8a50 con 0x5615d2eb1fe0 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5615d2d9dc70 con 0x5615d2eabfa0 2024-06-02T23:37:15.466 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615d2eb8bd0 con 0x5615d2eb16c0 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 0x5615d2eb7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc474fab80 10 client.0 _ll_register_callbacks cb 0x5615d2d979a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 0x5615d2eb7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45952/0 (socket says 172.21.15.52:45952) 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc3659c700 1 -- 172.21.15.52:0/1295253113 learned_addr learned my addr 172.21.15.52:0/1295253113 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.462+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc3659c700 1 -- 172.21.15.52:0/1295253113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 msgr2=0x5615d2eb5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc3659c700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc3659c700 1 -- 172.21.15.52:0/1295253113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 msgr2=0x5615d2eb5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc3659c700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc3659c700 1 -- 172.21.15.52:0/1295253113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d2d4c950 con 0x5615d2eabfa0 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.467 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc3659c700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 0x5615d2eb7d00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdc28004940 tx=0x7fdc28009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=942be29cc9a83bea server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc1dffb700 1 -- 172.21.15.52:0/1295253113 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28035060 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc1dffb700 1 -- 172.21.15.52:0/1295253113 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdc2800f670 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc2c0054d0 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc1dffb700 1 -- 172.21.15.52:0/1295253113 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc2800f870 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc474fab80 10 client.31196 fetch_fsmap learned FSMap version 45 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.463+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5615d2ec3300 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc1dffb700 1 -- 172.21.15.52:0/1295253113 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc2c0054d0 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc1dffb700 1 -- 172.21.15.52:0/1295253113 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fdc2803ca50 con 0x5615d2eabfa0 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc474fab80 10 client.31196 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc474fab80 10 client.31196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc474fab80 20 client.31196 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2ec5890 0x5615d2ec7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc474fab80 4 client.31196 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615d2e0ad90 con 0x5615d2ec5890 2024-06-02T23:37:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.464+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2ec5890 0x5615d2ec7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.465+0000 7fdc1dffb700 10 client.31196 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:15.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.465+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2ec5890 0x5615d2ec7ca0 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b89627f731ddfe93 server_cookie=17fa492401c8b64 in_seq=0 out_seq=0 2024-06-02T23:37:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc1dffb700 1 -- 172.21.15.52:0/1295253113 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5615d2e0ad90 con 0x5615d2ec5890 2024-06-02T23:37:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc1dffb700 10 client.31196 handle_command_reply: tid=0 2024-06-02T23:37:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 client.31196 shutdown 2024-06-02T23:37:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 msgr2=0x5615d2eb7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 0x5615d2eb7d00 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fdc28004940 tx=0x7fdc28009ca0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.470 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 shutdown_connections 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2ec5890 0x5615d2ec7ca0 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eabfa0 0x5615d2eb7d00 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1295253113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 >> 172.21.15.52:0/1295253113 conn(0x5615d2dd6d40 msgr2=0x5615d2eaaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc1dffb700 0 client.31196 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 shutdown_connections 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.466+0000 7fdc474fab80 1 -- 172.21.15.52:0/1295253113 wait complete. 2024-06-02T23:37:15.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.467+0000 7fdc474fab80 20 client.31196 trim_cache size 0 max 16384 2024-06-02T23:37:15.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.493+0000 7fdc474fab80 1 Processor -- start 2024-06-02T23:37:15.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 -- start start 2024-06-02T23:37:15.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eabfa0 0x5615d2eaba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.495 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381aec20 con 0x5615d2eabfa0 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381b0790 con 0x5615d2eb1fe0 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc474fab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381af320 con 0x5615d2eb16c0 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc36d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc3659c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eabfa0 0x5615d2eaba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc36d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45960/0 (socket says 172.21.15.52:45960) 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc36d9d700 1 -- 172.21.15.52:0/1037766169 learned_addr learned my addr 172.21.15.52:0/1037766169 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc3659c700 1 -- 172.21.15.52:0/1037766169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 msgr2=0x5615d2eb2c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.494+0000 7fdc3659c700 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb2c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc3659c700 1 -- 172.21.15.52:0/1037766169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 msgr2=0x5615d2eb55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc3659c700 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc3659c700 1 -- 172.21.15.52:0/1037766169 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615d2eb1140 con 0x5615d2eabfa0 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.496 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc3659c700 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eabfa0 0x5615d2eaba60 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fdc28005d90 tx=0x7fdc28005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158e562d113e47c5 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:15.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc1ffff700 1 -- 172.21.15.52:0/1037766169 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28041030 con 0x5615d2eabfa0 2024-06-02T23:37:15.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc1ffff700 1 -- 172.21.15.52:0/1037766169 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc28004c50 con 0x5615d2eabfa0 2024-06-02T23:37:15.497 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.495+0000 7fdc1ffff700 1 -- 172.21.15.52:0/1037766169 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc28035070 con 0x5615d2eabfa0 2024-06-02T23:37:15.498 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-06-02T23:37:15.498 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-06-02T23:37:15.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 -- 172.21.15.52:0/1037766169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eabfa0 msgr2=0x5615d2eaba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.500 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eabfa0 0x5615d2eaba60 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fdc28005d90 tx=0x7fdc28005790 comp rx=0 tx=0).stop 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 -- 172.21.15.52:0/1037766169 shutdown_connections 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2eb1fe0 0x5615d2eb55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eabfa0 0x5615d2eaba60 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 --2- 172.21.15.52:0/1037766169 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2eb2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.496+0000 7fdc474fab80 1 -- 172.21.15.52:0/1037766169 >> 172.21.15.52:0/1037766169 conn(0x5615d2dd6d40 msgr2=0x5615d2ea04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.497+0000 7fdc474fab80 1 -- 172.21.15.52:0/1037766169 shutdown_connections 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.497+0000 7fdc474fab80 1 -- 172.21.15.52:0/1037766169 wait complete. 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.497+0000 7fdc474fab80 1 -- start start 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 Processor -- start 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2ebe720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2ebae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 0x5615d2ebb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.501 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381b00c0 con 0x5615d2eb1fe0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381af9f0 con 0x5615d2ebb3f0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc381acbe0 con 0x5615d2eb16c0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc474fab80 10 client.0 _ll_register_callbacks cb 0x5615d2d979a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc36d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 0x5615d2ebb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc36d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 0x5615d2ebb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.52:45962/0 (socket says 172.21.15.52:45962) 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.498+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2531221280 learned_addr learned my addr 172.21.15.52:0/2531221280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2ebae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc3659c700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2ebe720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2531221280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 msgr2=0x5615d2ebe720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2ebe720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2531221280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 msgr2=0x5615d2ebae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:15.502 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2ebae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc36d9d700 1 -- 172.21.15.52:0/2531221280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615d2f38f70 con 0x5615d2ebb3f0 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc35d9b700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2ebae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc36d9d700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 0x5615d2ebb850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c6f0 tx=0x7fdc300a3db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f75e437b3dcb8413 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.499+0000 7fdc1dffb700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.500+0000 7fdc1dffb700 1 -- 172.21.15.52:0/2531221280 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc300bb2b0 con 0x5615d2ebb3f0 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.500+0000 7fdc1dffb700 1 -- 172.21.15.52:0/2531221280 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdc300970d0 con 0x5615d2ebb3f0 2024-06-02T23:37:15.503 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.500+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc28004c50 con 0x5615d2ebb3f0 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.501+0000 7fdc1dffb700 1 -- 172.21.15.52:0/2531221280 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7fdc3009b380 con 0x5615d2ebb3f0 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.501+0000 7fdc474fab80 10 client.31202 fetch_fsmap learned FSMap version 45 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.501+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x5615d2f3a2f0 con 0x5615d2ebb3f0 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.501+0000 7fdc1dffb700 1 -- 172.21.15.52:0/2531221280 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdc28004c50 con 0x5615d2ebb3f0 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.502+0000 7fdc1dffb700 1 -- 172.21.15.52:0/2531221280 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7fdc300972d0 con 0x5615d2ebb3f0 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.502+0000 7fdc474fab80 10 client.31202 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.502+0000 7fdc474fab80 10 client.31202 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.502+0000 7fdc474fab80 20 client.31202 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.502+0000 7fdc474fab80 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2f3a9f0 0x5615d2f3cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.503+0000 7fdc474fab80 4 client.31202 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.503+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5615d2e05310 con 0x5615d2f3a9f0 2024-06-02T23:37:15.505 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.504+0000 7fdc3659c700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2f3a9f0 0x5615d2f3cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:15.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.505+0000 7fdc3659c700 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2f3a9f0 0x5615d2f3cde0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=505f5ce7881caa75 server_cookie=5da0501c99576605 in_seq=0 out_seq=0 2024-06-02T23:37:15.506 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.505+0000 7fdc1dffb700 10 client.31202 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:15.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.508+0000 7fdc1dffb700 1 -- 172.21.15.52:0/2531221280 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5615d2e05310 con 0x5615d2f3a9f0 2024-06-02T23:37:15.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.508+0000 7fdc1dffb700 10 client.31202 handle_command_reply: tid=0 2024-06-02T23:37:15.508 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.508+0000 7fdc474fab80 1 client.31202 shutdown 2024-06-02T23:37:15.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 msgr2=0x5615d2ebb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 0x5615d2ebb850 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000c6f0 tx=0x7fdc300a3db0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.509 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 shutdown_connections 2024-06-02T23:37:15.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5615d2f3a9f0 0x5615d2f3cde0 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5615d2ebb3f0 0x5615d2ebb850 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615d2eb16c0 0x5615d2ebe720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 --2- 172.21.15.52:0/2531221280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5615d2eb1fe0 0x5615d2ebae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc1dffb700 0 client.31202 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:15.510 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 >> 172.21.15.52:0/2531221280 conn(0x5615d2dd6d40 msgr2=0x5615d2eb0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 shutdown_connections 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 1 -- 172.21.15.52:0/2531221280 wait complete. 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.509+0000 7fdc474fab80 20 client.31202 trim_cache size 0 max 16384 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stdout: "return_code": 0, 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_tag": "a88e88e0-2f1c-46eb-8a59-98b39b97f5d1", 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": "asynchronous" 2024-06-02T23:37:15.511 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc200a2c20 msgr2=0x7fdc200a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc200a2c20 0x7fdc200a50e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800b2e0 tx=0x7fdc2801a040 comp rx=0 tx=0).stop 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38107400 msgr2=0x7fdc38103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38107400 0x7fdc38103e60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdc3000efc0 tx=0x7fdc3000c3a0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 shutdown_connections 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdc200a2c20 0x7fdc200a50e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdc38107400 0x7fdc38103e60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdc38101390 0x7fdc381033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 --2- 172.21.15.52:0/2791219460 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc38101d50 0x7fdc38103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:15.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 >> 172.21.15.52:0/2791219460 conn(0x7fdc38078340 msgr2=0x7fdc38105700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:15.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 shutdown_connections 2024-06-02T23:37:15.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:15.511+0000 7fdc3cc77700 1 -- 172.21.15.52:0/2791219460 wait complete. 2024-06-02T23:37:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:15 smithi174 ceph-mon[35994]: pgmap v1192: 97 pgs: 97 active+clean; 592 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 139 KiB/s wr, 41 op/s 2024-06-02T23:37:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:15 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:15 smithi052 ceph-mon[32340]: pgmap v1192: 97 pgs: 97 active+clean; 592 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 139 KiB/s wr, 41 op/s 2024-06-02T23:37:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:15 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:15 smithi123 ceph-mon[35977]: pgmap v1192: 97 pgs: 97 active+clean; 592 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 139 KiB/s wr, 41 op/s 2024-06-02T23:37:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:15 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:16.427 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:37:16.465 DEBUG:teuthology.parallel:result is None 2024-06-02T23:37:16.466 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-06-02T23:37:16.541 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-06-02T23:37:16.541 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-06-02T23:37:16.552 INFO:tasks.quiescer:joining Quiescers 2024-06-02T23:37:16.552 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-06-02T23:37:16.552 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:37:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 -- 172.21.15.52:0/3191788252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 msgr2=0x7f93b0107200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 --2- 172.21.15.52:0/3191788252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0107200 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7f93a4009900 tx=0x7f93a402a410 comp rx=0 tx=0).stop 2024-06-02T23:37:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 -- 172.21.15.52:0/3191788252 shutdown_connections 2024-06-02T23:37:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 --2- 172.21.15.52:0/3191788252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93b0107740 0x7f93b0109b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 --2- 172.21.15.52:0/3191788252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0107200 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 --2- 172.21.15.52:0/3191788252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b01024e0 0x7f93b01048d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 -- 172.21.15.52:0/3191788252 >> 172.21.15.52:0/3191788252 conn(0x7f93b00fbe70 msgr2=0x7f93b00fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 -- 172.21.15.52:0/3191788252 shutdown_connections 2024-06-02T23:37:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.743+0000 7f93b6310640 1 -- 172.21.15.52:0/3191788252 wait complete. 2024-06-02T23:37:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 Processor -- start 2024-06-02T23:37:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 -- start start 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93b01024e0 0x7f93b0101e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0100490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b0107740 0x7f93b01009d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010d520 con 0x7f93b0104e10 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010d3a0 con 0x7f93b01024e0 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.744+0000 7f93b6310640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93b010d6a0 con 0x7f93b0107740 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0100490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0100490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57440/0 (socket says 172.21.15.52:57440) 2024-06-02T23:37:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 -- 172.21.15.52:0/2714346191 learned_addr learned my addr 172.21.15.52:0/2714346191 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93b48ad640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b0107740 0x7f93b01009d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93affff640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93b01024e0 0x7f93b0101e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 -- 172.21.15.52:0/2714346191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b0107740 msgr2=0x7f93b01009d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b0107740 0x7f93b01009d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 -- 172.21.15.52:0/2714346191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93b01024e0 msgr2=0x7f93b0101e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93b01024e0 0x7f93b0101e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 -- 172.21.15.52:0/2714346191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93b0101260 con 0x7f93b0104e10 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93af7fe640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0100490 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f93a40314e0 tx=0x7f93a4031a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.745+0000 7f93ad7fa640 1 -- 172.21.15.52:0/2714346191 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a403f030 con 0x7f93b0104e10 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.746+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93b01a1c30 con 0x7f93b0104e10 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.746+0000 7f93ad7fa640 1 -- 172.21.15.52:0/2714346191 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a40044b0 con 0x7f93b0104e10 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.746+0000 7f93ad7fa640 1 -- 172.21.15.52:0/2714346191 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4035070 con 0x7f93b0104e10 2024-06-02T23:37:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.746+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93b01a21d0 con 0x7f93b0104e10 2024-06-02T23:37:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.748+0000 7f93ad7fa640 1 -- 172.21.15.52:0/2714346191 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f93a4042030 con 0x7f93b0104e10 2024-06-02T23:37:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.748+0000 7f93b901d740 1 Processor -- start 2024-06-02T23:37:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.748+0000 7f93b901d740 1 -- start start 2024-06-02T23:37:16.750 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.748+0000 7f93b901d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b901d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b901d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946c5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b901d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5582946204f0 con 0x5582946c2b70 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b901d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582945be870 con 0x5582946bd010 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b901d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55829461c4b0 con 0x5582946bda00 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946c5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946c5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57454/0 (socket says 172.21.15.52:57454) 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 -- 172.21.15.52:0/3393378734 learned_addr learned my addr 172.21.15.52:0/3393378734 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b48ad640 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93affff640 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93ad7fa640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93980a2c30 0x7f93980a50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b48ad640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93980a2c30 0x7f93980a50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93ad7fa640 1 -- 172.21.15.52:0/2714346191 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f93a40ef4c0 con 0x7f93b0104e10 2024-06-02T23:37:16.751 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 -- 172.21.15.52:0/3393378734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 msgr2=0x5582946c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 -- 172.21.15.52:0/3393378734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 msgr2=0x5582946bd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946bd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93af7fe640 1 -- 172.21.15.52:0/3393378734 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582945da9b0 con 0x5582946c2b70 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.749+0000 7f93b48ad640 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.750+0000 7f93af7fe640 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946c5000 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f93a40c4490 tx=0x7f93a402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eabb09138fdecdc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.750+0000 7f93b48ad640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93980a2c30 0x7f93980a50f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x5582946be260 tx=0x7f93a8004ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:16.752 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.750+0000 7f93967fc640 1 -- 172.21.15.52:0/3393378734 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a4049050 con 0x5582946c2b70 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.750+0000 7f93967fc640 1 -- 172.21.15.52:0/3393378734 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a403b460 con 0x5582946c2b70 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.750+0000 7f93967fc640 1 -- 172.21.15.52:0/3393378734 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a40ba200 con 0x5582946c2b70 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 -- 172.21.15.52:0/3393378734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 msgr2=0x5582946c5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946c5000 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f93a40c4490 tx=0x7f93a402a410 comp rx=0 tx=0).stop 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 -- 172.21.15.52:0/3393378734 shutdown_connections 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946c5000 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 --2- 172.21.15.52:0/3393378734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 -- 172.21.15.52:0/3393378734 >> 172.21.15.52:0/3393378734 conn(0x5582946b8680 msgr2=0x5582946b8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 -- 172.21.15.52:0/3393378734 shutdown_connections 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 -- 172.21.15.52:0/3393378734 wait complete. 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 -- start start 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.751+0000 7f93b901d740 1 Processor -- start 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5582946204f0 con 0x5582946c2b70 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582945a9ae0 con 0x5582946bd010 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5582946c58e0 con 0x5582946bda00 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b901d740 10 client.0 _ll_register_callbacks cb 0x5582945e29b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57462/0 (socket says 172.21.15.52:57462) 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 -- 172.21.15.52:0/2791222209 learned_addr learned my addr 172.21.15.52:0/2791222209 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93affff640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93af7fe640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 -- 172.21.15.52:0/2791222209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 msgr2=0x5582946c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 -- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 msgr2=0x5582946c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93b48ad640 1 -- 172.21.15.52:0/2791222209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558294334210 con 0x5582946c2b70 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.752+0000 7f93af7fe640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f93b48ad640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce4b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f93a800fcb0 tx=0x7f93a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=987155f38fc0dacc server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f938f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f938f7fe640 1 -- 172.21.15.52:0/2791222209 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a8025070 con 0x5582946c2b70 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f938f7fe640 1 -- 172.21.15.52:0/2791222209 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f93a80047d0 con 0x5582946c2b70 2024-06-02T23:37:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558294633f10 con 0x5582946c2b70 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f938f7fe640 1 -- 172.21.15.52:0/2791222209 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f93a8031050 con 0x5582946c2b70 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f93b901d740 10 client.22005 fetch_fsmap learned FSMap version 45 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x558294400620 con 0x5582946c2b70 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.753+0000 7f938f7fe640 1 -- 172.21.15.52:0/2791222209 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x558294633f10 con 0x5582946c2b70 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f938f7fe640 1 -- 172.21.15.52:0/2791222209 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f93a801f040 con 0x5582946c2b70 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93b901d740 10 client.22005 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93b901d740 10 client.22005 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:16.759 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93b901d740 20 client.22005 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93b901d740 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5582946d32a0 0x5582946d5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93b901d740 4 client.22005 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55829462dc20 con 0x5582946d32a0 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.754+0000 7f93affff640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5582946d32a0 0x5582946d5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.755+0000 7f93affff640 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5582946d32a0 0x5582946d5720 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=174b7fd7de3ed5fc server_cookie=ffa053f7ef540608 in_seq=0 out_seq=0 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.755+0000 7f938f7fe640 10 client.22005 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.755+0000 7f938f7fe640 1 -- 172.21.15.52:0/2791222209 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55829462dc20 con 0x5582946d32a0 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.755+0000 7f938f7fe640 10 client.22005 handle_command_reply: tid=0 2024-06-02T23:37:16.760 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.755+0000 7f93b901d740 1 client.22005 shutdown 2024-06-02T23:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 msgr2=0x5582946ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce4b0 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f93a800fcb0 tx=0x7f93a8004a40 comp rx=0 tx=0).stop 2024-06-02T23:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 shutdown_connections 2024-06-02T23:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5582946d32a0 0x5582946d5720 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce4b0 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 --2- 172.21.15.52:0/2791222209 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 >> 172.21.15.52:0/2791222209 conn(0x5582946b8680 msgr2=0x5582946b77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f938f7fe640 0 client.22005 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 shutdown_connections 2024-06-02T23:37:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 1 -- 172.21.15.52:0/2791222209 wait complete. 2024-06-02T23:37:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.756+0000 7f93b901d740 20 client.22005 trim_cache size 0 max 16384 2024-06-02T23:37:16.766 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 Processor -- start 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 -- start start 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01aaee0 con 0x5582946c2b70 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01aca50 con 0x5582946bda00 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b901d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ab5e0 con 0x5582946bd010 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b48ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93affff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b48ad640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57470/0 (socket says 172.21.15.52:57470) 2024-06-02T23:37:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93b48ad640 1 -- 172.21.15.52:0/462347960 learned_addr learned my addr 172.21.15.52:0/462347960 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.774+0000 7f93af7fe640 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93affff640 1 -- 172.21.15.52:0/462347960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 msgr2=0x5582946c7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93affff640 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93affff640 1 -- 172.21.15.52:0/462347960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 msgr2=0x5582946ce470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93affff640 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93affff640 1 -- 172.21.15.52:0/462347960 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582946c28c0 con 0x5582946bd010 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93af7fe640 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93b48ad640 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f93affff640 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bd010 0x5582946c6f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5582946d5f60 tx=0x7f93a000b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70070172f7c5ff66 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f9395ffb640 1 -- 172.21.15.52:0/462347960 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0003d60 con 0x5582946bd010 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f9395ffb640 1 -- 172.21.15.52:0/462347960 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a00054d0 con 0x5582946bd010 2024-06-02T23:37:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.775+0000 7f9395ffb640 1 -- 172.21.15.52:0/462347960 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0026460 con 0x5582946bd010 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 -- 172.21.15.52:0/462347960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bd010 msgr2=0x5582946c6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bd010 0x5582946c6f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5582946d5f60 tx=0x7f93a000b390 comp rx=0 tx=0).stop 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 -- 172.21.15.52:0/462347960 shutdown_connections 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946c2b70 0x5582946ce470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946c7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 --2- 172.21.15.52:0/462347960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946bd010 0x5582946c6f30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 -- 172.21.15.52:0/462347960 >> 172.21.15.52:0/462347960 conn(0x5582946b8680 msgr2=0x5582946c6550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 -- 172.21.15.52:0/462347960 shutdown_connections 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.776+0000 7f93b901d740 1 -- 172.21.15.52:0/462347960 wait complete. 2024-06-02T23:37:16.781 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.777+0000 7f93b901d740 1 -- start start 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.777+0000 7f93b901d740 1 Processor -- start 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.777+0000 7f93b901d740 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 0x5582946ca620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.777+0000 7f93b901d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.777+0000 7f93b901d740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946c2b70 0x5582946cdb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93b901d740 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01ac380 con 0x5582946bd010 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93b901d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01abcb0 con 0x5582946bda00 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93b901d740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93b01a8ec0 con 0x5582946c2b70 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93b901d740 10 client.0 _ll_register_callbacks cb 0x7f93a80136d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 0x5582946ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 0x5582946ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57478/0 (socket says 172.21.15.52:57478) 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 -- 172.21.15.52:0/986011002 learned_addr learned my addr 172.21.15.52:0/986011002 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93af7fe640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.782 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93b48ad640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946c2b70 0x5582946cdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 -- 172.21.15.52:0/986011002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946c2b70 msgr2=0x5582946cdb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946c2b70 0x5582946cdb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 -- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 msgr2=0x5582946cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 -- 172.21.15.52:0/986011002 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582946ce170 con 0x5582946bd010 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f93affff640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 0x5582946ca620 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b700 tx=0x7f93a000b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ffbae78c934758 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.778+0000 7f938f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f938f7fe640 1 -- 172.21.15.52:0/986011002 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93a0003b60 con 0x5582946bd010 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f938f7fe640 1 -- 172.21.15.52:0/986011002 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f93a0003d00 con 0x5582946bd010 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93a00054d0 con 0x5582946bd010 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f938f7fe640 1 -- 172.21.15.52:0/986011002 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f93a0004e20 con 0x5582946bd010 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f93b901d740 10 client.22011 fetch_fsmap learned FSMap version 45 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55829473ca80 con 0x5582946bd010 2024-06-02T23:37:16.783 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.779+0000 7f938f7fe640 1 -- 172.21.15.52:0/986011002 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f93a00054d0 con 0x5582946bd010 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f938f7fe640 1 -- 172.21.15.52:0/986011002 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 45) ==== 8362+0+0 (secure 0 0 0) 0x7f93a003aa20 con 0x5582946bd010 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93b901d740 10 client.22011 fetch_fsmap finished waiting for FSMap version 45 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93b901d740 10 client.22011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93b901d740 20 client.22011 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93b901d740 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55829473d2c0 0x55829473f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93b901d740 4 client.22011 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558294627f70 con 0x55829473d2c0 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.780+0000 7f93af7fe640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55829473d2c0 0x55829473f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.781+0000 7f93af7fe640 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55829473d2c0 0x55829473f6e0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e392612430093bf9 server_cookie=c2b31343eaa27436 in_seq=0 out_seq=0 2024-06-02T23:37:16.784 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.781+0000 7f938f7fe640 10 client.22011 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.812+0000 7f938f7fe640 1 -- 172.21.15.52:0/986011002 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x558294627f70 con 0x55829473d2c0 2024-06-02T23:37:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.812+0000 7f938f7fe640 10 client.22011 handle_command_reply: tid=0 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.812+0000 7f93b901d740 1 client.22011 shutdown 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 msgr2=0x5582946ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 0x5582946ca620 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b700 tx=0x7f93a000b730 comp rx=0 tx=0).stop 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 shutdown_connections 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55829473d2c0 0x55829473f6e0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f938f7fe640 0 client.22011 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5582946c2b70 0x5582946cdb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582946bda00 0x5582946cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 --2- 172.21.15.52:0/986011002 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5582946bd010 0x5582946ca620 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 >> 172.21.15.52:0/986011002 conn(0x5582946b8680 msgr2=0x5582946c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 shutdown_connections 2024-06-02T23:37:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 1 -- 172.21.15.52:0/986011002 wait complete. 2024-06-02T23:37:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.813+0000 7f93b901d740 20 client.22011 trim_cache size 0 max 16384 2024-06-02T23:37:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:{ 2024-06-02T23:37:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout: "return_code": 0 2024-06-02T23:37:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stdout:} 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93980a2c30 msgr2=0x7f93980a50f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93980a2c30 0x7f93980a50f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x5582946be260 tx=0x7f93a8004ab0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 msgr2=0x7f93b0100490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0100490 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f93a40314e0 tx=0x7f93a4031a30 comp rx=0 tx=0).stop 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 shutdown_connections 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f93980a2c30 0x7f93980a50f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93b0107740 0x7f93b01009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f93b0104e10 0x7f93b0100490 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 --2- 172.21.15.52:0/2714346191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f93b01024e0 0x7f93b0101e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 >> 172.21.15.52:0/2714346191 conn(0x7f93b00fbe70 msgr2=0x7f93b0102e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 shutdown_connections 2024-06-02T23:37:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi052.front.sepia.ceph.com.smithi052.stderr:2024-06-02T23:37:16.814+0000 7f93b6310640 1 -- 172.21.15.52:0/2714346191 wait complete. 2024-06-02T23:37:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:16 smithi174 ceph-mon[35994]: scrub queued for path: / 2024-06-02T23:37:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:16 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:16 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:37:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:16 smithi174 ceph-mon[35994]: scrub summary: active paths [/] 2024-06-02T23:37:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:16 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:16 smithi174 ceph-mon[35994]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:16 smithi052 ceph-mon[32340]: scrub queued for path: / 2024-06-02T23:37:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:16 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:16 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:37:17.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:16 smithi052 ceph-mon[32340]: scrub summary: active paths [/] 2024-06-02T23:37:17.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:16 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:16 smithi052 ceph-mon[32340]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:16 smithi123 ceph-mon[35977]: scrub queued for path: / 2024-06-02T23:37:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:16 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:16 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:37:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:16 smithi123 ceph-mon[35977]: scrub summary: active paths [/] 2024-06-02T23:37:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:16 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:16 smithi123 ceph-mon[35977]: scrub summary: idle+waiting paths [/] 2024-06-02T23:37:17.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:17 smithi052 ceph-mon[32340]: pgmap v1193: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 147 KiB/s wr, 58 op/s 2024-06-02T23:37:18.100 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:17 smithi174 ceph-mon[35994]: pgmap v1193: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 147 KiB/s wr, 58 op/s 2024-06-02T23:37:18.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:17 smithi123 ceph-mon[35977]: pgmap v1193: 97 pgs: 97 active+clean; 522 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 141 KiB/s rd, 147 KiB/s wr, 58 op/s 2024-06-02T23:37:19.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:18 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:19.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:18 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:19.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:18 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:19.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.809+0000 7f5dc705b700 1 -- 172.21.15.52:0/2506453286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 msgr2=0x7f5dc010f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- 172.21.15.52:0/2506453286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc010f340 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f5db800b0a0 tx=0x7f5db800b3b0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 -- 172.21.15.52:0/2506453286 shutdown_connections 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- 172.21.15.52:0/2506453286 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc010f340 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- 172.21.15.52:0/2506453286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 0x7f5dc0108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- 172.21.15.52:0/2506453286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 0x7f5dc006afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 -- 172.21.15.52:0/2506453286 >> 172.21.15.52:0/2506453286 conn(0x7f5dc0078340 msgr2=0x7f5dc0078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 -- 172.21.15.52:0/2506453286 shutdown_connections 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 -- 172.21.15.52:0/2506453286 wait complete. 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 Processor -- start 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 -- start start 2024-06-02T23:37:19.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 0x7f5dc0101840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc0101d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.810+0000 7f5dc705b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 0x7f5dc01022c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc705b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc0112630 con 0x7f5dc006b520 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc705b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01124b0 con 0x7f5dc006ab60 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc705b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01127b0 con 0x7f5dc0108520 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc0101d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc0101d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57498/0 (socket says 172.21.15.52:57498) 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 -- 172.21.15.52:0/517077575 learned_addr learned my addr 172.21.15.52:0/517077575 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc55f8700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 0x7f5dc01022c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 0x7f5dc0101840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 -- 172.21.15.52:0/517077575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 msgr2=0x7f5dc01022c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 0x7f5dc01022c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 -- 172.21.15.52:0/517077575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 msgr2=0x7f5dc0101840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 0x7f5dc0101840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 -- 172.21.15.52:0/517077575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc01a6f90 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc55f8700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 0x7f5dc01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 0x7f5dc0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbffff700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc0101d80 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c9b0 tx=0x7f5dac00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbdffb700 1 -- 172.21.15.52:0/517077575 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac007b00 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbdffb700 1 -- 172.21.15.52:0/517077575 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dac007ca0 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc01a7280 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dbdffb700 1 -- 172.21.15.52:0/517077575 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac018730 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.811+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc01a7810 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.814+0000 7f5dbdffb700 1 -- 172.21.15.52:0/517077575 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5dac00f450 con 0x7f5dc006b520 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.815+0000 7f5dbdffb700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5db00a2ca0 0x7f5db00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.815+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5db00a2ca0 0x7f5db00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.815+0000 7f5dbdffb700 1 -- 172.21.15.52:0/517077575 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5dac0cf020 con 0x7f5dc006b520 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.815+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5db00a2ca0 0x7f5db00a5160 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009b50 tx=0x7f5db4009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.815+0000 7f5dd18deb80 1 Processor -- start 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 -- start start 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c6b2f60 con 0x55ae2c7c5f60 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c711850 con 0x55ae2c7c6880 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c679460 con 0x55ae2c7c07a0 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57512/0 (socket says 172.21.15.52:57512) 2024-06-02T23:37:19.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 -- 172.21.15.52:0/382863960 learned_addr learned my addr 172.21.15.52:0/382863960 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dbffff700 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 -- 172.21.15.52:0/382863960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 msgr2=0x55ae2c7c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 -- 172.21.15.52:0/382863960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 msgr2=0x55ae2c7c0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 -- 172.21.15.52:0/382863960 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2c6608d0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dbffff700 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c6340 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5db400ed90 tx=0x7f5db4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2c8ac3a7e2710b3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.816+0000 7f5daaffd700 1 -- 172.21.15.52:0/382863960 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db4011c90 con 0x55ae2c7c5f60 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5daaffd700 1 -- 172.21.15.52:0/382863960 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5db4023070 con 0x55ae2c7c5f60 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5daaffd700 1 -- 172.21.15.52:0/382863960 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db40105d0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 -- 172.21.15.52:0/382863960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 msgr2=0x55ae2c7c6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c6340 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f5db400ed90 tx=0x7f5db4011000 comp rx=0 tx=0).stop 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 -- 172.21.15.52:0/382863960 shutdown_connections 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c6340 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/382863960 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 -- 172.21.15.52:0/382863960 >> 172.21.15.52:0/382863960 conn(0x55ae2c6ed080 msgr2=0x55ae2c7bb220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 -- 172.21.15.52:0/382863960 shutdown_connections 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.817+0000 7f5dd18deb80 1 -- 172.21.15.52:0/382863960 wait complete. 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 -- start start 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 Processor -- start 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c6b1fc0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.841 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c70d810 con 0x55ae2c7c6880 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ae2c7cd200 con 0x55ae2c7c07a0 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dd18deb80 10 client.0 _ll_register_callbacks cb 0x55ae2c6a75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57518/0 (socket says 172.21.15.52:57518) 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 -- 172.21.15.52:0/2307771199 learned_addr learned my addr 172.21.15.52:0/2307771199 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dc55f8700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 -- 172.21.15.52:0/2307771199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 msgr2=0x55ae2c7cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 -- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 msgr2=0x55ae2c7c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.842 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.818+0000 7f5dbffff700 1 -- 172.21.15.52:0/2307771199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2c6d7750 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dbffff700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c98f0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5dac098280 tx=0x7f5dac007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e9bc5220ee80486 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5d9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5d9ffff700 1 -- 172.21.15.52:0/2307771199 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0b3070 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5db4023070 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5d9ffff700 1 -- 172.21.15.52:0/2307771199 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5dac00f740 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5d9ffff700 1 -- 172.21.15.52:0/2307771199 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5dac09fc60 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 10 client.22023 fetch_fsmap learned FSMap version 46 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ae2c7d7a30 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5d9ffff700 1 -- 172.21.15.52:0/2307771199 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5db4023070 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5d9ffff700 1 -- 172.21.15.52:0/2307771199 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f5dac09cbe0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 10 client.22023 fetch_fsmap finished waiting for FSMap version 46 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 10 client.22023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:19.843 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 20 client.22023 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c7d9fc0 0x55ae2c7dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 4 client.22023 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.819+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae2c71ed50 con 0x55ae2c7d9fc0 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.820+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c7d9fc0 0x55ae2c7dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.820+0000 7f5d9ffff700 10 client.22023 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.820+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c7d9fc0 0x55ae2c7dc3b0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4672eecf2d99dbf server_cookie=a6b13fd762ea3152 in_seq=0 out_seq=0 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5d9ffff700 1 -- 172.21.15.52:0/2307771199 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55ae2c71ed50 con 0x55ae2c7d9fc0 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5d9ffff700 10 client.22023 handle_command_reply: tid=0 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 client.22023 shutdown 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 msgr2=0x55ae2c7c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c98f0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f5dac098280 tx=0x7f5dac007f90 comp rx=0 tx=0).stop 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 shutdown_connections 2024-06-02T23:37:19.844 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c7d9fc0 0x55ae2c7dc3b0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/2307771199 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7c98f0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 >> 172.21.15.52:0/2307771199 conn(0x55ae2c6ed080 msgr2=0x55ae2c7ba700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5d9ffff700 0 client.22023 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 shutdown_connections 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 1 -- 172.21.15.52:0/2307771199 wait complete. 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.821+0000 7f5dd18deb80 20 client.22023 trim_cache size 0 max 16384 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 Processor -- start 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 -- start start 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.845 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7bb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01aebc0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01b0760 con 0x55ae2c7c6880 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.842+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01af2c0 con 0x55ae2c7c07a0 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7bae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57522/0 (socket says 172.21.15.52:57522) 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 -- 172.21.15.52:0/131328510 learned_addr learned my addr 172.21.15.52:0/131328510 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dc55f8700 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7bb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 -- 172.21.15.52:0/131328510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 msgr2=0x55ae2c7c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 -- 172.21.15.52:0/131328510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 msgr2=0x55ae2c7bb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7bb3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 -- 172.21.15.52:0/131328510 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae2c7c10a0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5dbffff700 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7bae80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0ccec0 tx=0x7f5dac0a4410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c792413987aaa452 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.843+0000 7f5daa7fc700 1 -- 172.21.15.52:0/131328510 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0b3070 con 0x55ae2c7c5f60 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5daa7fc700 1 -- 172.21.15.52:0/131328510 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dac0a4520 con 0x55ae2c7c5f60 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5daa7fc700 1 -- 172.21.15.52:0/131328510 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0860f0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 -- 172.21.15.52:0/131328510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 msgr2=0x55ae2c7bae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7bae80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0ccec0 tx=0x7f5dac0a4410 comp rx=0 tx=0).stop 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 -- 172.21.15.52:0/131328510 shutdown_connections 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7bb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/131328510 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7bae80 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 -- 172.21.15.52:0/131328510 >> 172.21.15.52:0/131328510 conn(0x55ae2c6ed080 msgr2=0x55ae2c7c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.844+0000 7f5dd18deb80 1 -- 172.21.15.52:0/131328510 shutdown_connections 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 -- 172.21.15.52:0/131328510 wait complete. 2024-06-02T23:37:19.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 -- start start 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 Processor -- start 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7d24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7cf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01b0090 con 0x55ae2c7c5f60 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01af9c0 con 0x55ae2c7c6880 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc01acb60 con 0x55ae2c7c07a0 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.845+0000 7f5dd18deb80 10 client.0 _ll_register_callbacks cb 0x55ae2c6a75b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7d2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dc55f8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7d2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57532/0 (socket says 172.21.15.52:57532) 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 -- 172.21.15.52:0/1193765591 learned_addr learned my addr 172.21.15.52:0/1193765591 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7d24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.848 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 -- 172.21.15.52:0/1193765591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 msgr2=0x55ae2c7d24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7d24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 -- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 msgr2=0x55ae2c7cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7cf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 -- 172.21.15.52:0/1193765591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae2c7cf9d0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dc55f8700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dbffff700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7d2a00 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5dac098230 tx=0x7f5dac00f970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=befbe00c1877fa35 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5d9ffff700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5d9ffff700 1 -- 172.21.15.52:0/1193765591 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac0b3050 con 0x55ae2c7c5f60 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5d9ffff700 1 -- 172.21.15.52:0/1193765591 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5dac09ad00 con 0x55ae2c7c5f60 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dac0a4520 con 0x55ae2c7c5f60 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5d9ffff700 1 -- 172.21.15.52:0/1193765591 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f5dac086300 con 0x55ae2c7c5f60 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dd18deb80 10 client.22029 fetch_fsmap learned FSMap version 46 2024-06-02T23:37:19.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.846+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x55ae2c8452d0 con 0x55ae2c7c5f60 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5d9ffff700 1 -- 172.21.15.52:0/1193765591 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5dac0a4520 con 0x55ae2c7c5f60 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5d9ffff700 1 -- 172.21.15.52:0/1193765591 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 46) ==== 8366+0+0 (secure 0 0 0) 0x7f5dac09a770 con 0x55ae2c7c5f60 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dd18deb80 10 client.22029 fetch_fsmap finished waiting for FSMap version 46 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dd18deb80 10 client.22029 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dd18deb80 20 client.22029 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c845710 0x55ae2c847b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dd18deb80 4 client.22029 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae2c7192d0 con 0x55ae2c845710 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.847+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c845710 0x55ae2c847b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dc4df7700 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c845710 0x55ae2c847b30 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b597f89ac227f544 server_cookie=2017e6b781f93bc3 in_seq=0 out_seq=0 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5d9ffff700 10 client.22029 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5d9ffff700 1 -- 172.21.15.52:0/1193765591 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae2c7192d0 con 0x55ae2c845710 2024-06-02T23:37:19.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5d9ffff700 10 client.22029 handle_command_reply: tid=0 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 client.22029 shutdown 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 msgr2=0x55ae2c7d2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7d2a00 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f5dac098230 tx=0x7f5dac00f970 comp rx=0 tx=0).stop 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 shutdown_connections 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55ae2c845710 0x55ae2c847b30 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "IDLE (waiting for more scrubs)", 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": { 2024-06-02T23:37:19.851 INFO:teuthology.orchestra.run.smithi052.stdout: "a88e88e0-2f1c-46eb-8a59-98b39b97f5d1": { 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stdout: "path": "/", 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stdout: "tag": "a88e88e0-2f1c-46eb-8a59-98b39b97f5d1", 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stdout: "options": "recursive,force" 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ae2c7c6880 0x55ae2c7cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ae2c7c07a0 0x55ae2c7d24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 --2- 172.21.15.52:0/1193765591 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55ae2c7c5f60 0x55ae2c7d2a00 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 >> 172.21.15.52:0/1193765591 conn(0x55ae2c6ed080 msgr2=0x55ae2c7c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5d9ffff700 0 client.22029 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 shutdown_connections 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 1 -- 172.21.15.52:0/1193765591 wait complete. 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.848+0000 7f5dd18deb80 20 client.22029 trim_cache size 0 max 16384 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5db00a2ca0 msgr2=0x7f5db00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5db00a2ca0 0x7f5db00a5160 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009b50 tx=0x7f5db4009550 comp rx=0 tx=0).stop 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 msgr2=0x7f5dc0101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc0101d80 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c9b0 tx=0x7f5dac00cd70 comp rx=0 tx=0).stop 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 shutdown_connections 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5db00a2ca0 0x7f5db00a5160 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5dc0108520 0x7f5dc01022c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5dc006ab60 0x7f5dc0101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 --2- 172.21.15.52:0/517077575 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5dc006b520 0x7f5dc0101d80 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:19.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 >> 172.21.15.52:0/517077575 conn(0x7f5dc0078340 msgr2=0x7f5dc01003f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:19.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 shutdown_connections 2024-06-02T23:37:19.854 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:18.849+0000 7f5dc705b700 1 -- 172.21.15.52:0/517077575 wait complete. 2024-06-02T23:37:20.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:19 smithi174 ceph-mon[35994]: pgmap v1194: 97 pgs: 97 active+clean; 481 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 217 KiB/s wr, 92 op/s 2024-06-02T23:37:20.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:19 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:20.147 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:19 smithi052 ceph-mon[32340]: pgmap v1194: 97 pgs: 97 active+clean; 481 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 217 KiB/s wr, 92 op/s 2024-06-02T23:37:20.147 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:19 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:19 smithi123 ceph-mon[35977]: pgmap v1194: 97 pgs: 97 active+clean; 481 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 217 KiB/s wr, 92 op/s 2024-06-02T23:37:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:19 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:20.481 INFO:tasks.cephfs.filesystem:scrub status for tag:a88e88e0-2f1c-46eb-8a59-98b39b97f5d1 - {'path': '/', 'tag': 'a88e88e0-2f1c-46eb-8a59-98b39b97f5d1', 'options': 'recursive,force'} 2024-06-02T23:37:20.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 -- 172.21.15.52:0/4279714760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c81030f0 msgr2=0x7f97c8107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 --2- 172.21.15.52:0/4279714760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c81030f0 0x7f97c8107110 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f97b40099c0 tx=0x7f97b4009cd0 comp rx=0 tx=0).stop 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 -- 172.21.15.52:0/4279714760 shutdown_connections 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 --2- 172.21.15.52:0/4279714760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c8107840 0x7f97c810f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 --2- 172.21.15.52:0/4279714760 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97c8102740 0x7f97c8102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 --2- 172.21.15.52:0/4279714760 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c81030f0 0x7f97c8107110 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 -- 172.21.15.52:0/4279714760 >> 172.21.15.52:0/4279714760 conn(0x7f97c80795f0 msgr2=0x7f97c8079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 -- 172.21.15.52:0/4279714760 shutdown_connections 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.968+0000 7f97ceae0700 1 -- 172.21.15.52:0/4279714760 wait complete. 2024-06-02T23:37:20.971 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 Processor -- start 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 -- start start 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97c8102740 0x7f97c819e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c81030f0 0x7f97c819ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 0x7f97c8198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f97c8112650 con 0x7f97c8107840 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81124d0 con 0x7f97c81030f0 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97ceae0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97c81127d0 con 0x7f97c8102740 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 0x7f97c8198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:20.972 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 0x7f97c8198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:57542/0 (socket says 172.21.15.52:57542) 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 -- 172.21.15.52:0/3910526400 learned_addr learned my addr 172.21.15.52:0/3910526400 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cc87c700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97c8102740 0x7f97c819e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97c7fff700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c81030f0 0x7f97c819ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 -- 172.21.15.52:0/3910526400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97c8102740 msgr2=0x7f97c819e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97c8102740 0x7f97c819e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 -- 172.21.15.52:0/3910526400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c81030f0 msgr2=0x7f97c819ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c81030f0 0x7f97c819ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.969+0000 7f97cd07d700 1 -- 172.21.15.52:0/3910526400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97c81990b0 con 0x7f97c8107840 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.970+0000 7f97cd07d700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 0x7f97c8198950 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f97c0007ee0 tx=0x7f97c000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:20.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.970+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c000dbd0 con 0x7f97c8107840 2024-06-02T23:37:20.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.970+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f97c000dd70 con 0x7f97c8107840 2024-06-02T23:37:20.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.970+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97c81993a0 con 0x7f97c8107840 2024-06-02T23:37:20.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.970+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c0005a20 con 0x7f97c8107840 2024-06-02T23:37:20.974 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.970+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97c81a5610 con 0x7f97c8107840 2024-06-02T23:37:20.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.971+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97c8069180 con 0x7f97c8107840 2024-06-02T23:37:20.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.972+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f97c001a020 con 0x7f97c8107840 2024-06-02T23:37:20.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.973+0000 7f97c5ffb700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97b80a2c20 0x7f97b80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:20.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.973+0000 7f97cc87c700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97b80a2c20 0x7f97b80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:20.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.973+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f97c00ce830 con 0x7f97c8107840 2024-06-02T23:37:20.975 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.974+0000 7f97cc87c700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97b80a2c20 0x7f97b80a50e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f97bc005ea0 tx=0x7f97bc005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:20.978 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:20.978+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f97c0010070 con 0x7f97c8107840 2024-06-02T23:37:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:20 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:21.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:20 smithi174 ceph-mon[35994]: pgmap v1195: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 268 KiB/s wr, 128 op/s 2024-06-02T23:37:21.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:20 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:37:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:20 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:20 smithi052 ceph-mon[32340]: pgmap v1195: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 268 KiB/s wr, 128 op/s 2024-06-02T23:37:21.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:20 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:37:21.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.237+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97c802dca0 con 0x7f97c8107840 2024-06-02T23:37:21.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.238+0000 7f97c5ffb700 1 -- 172.21.15.52:0/3910526400 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11953 (secure 0 0 0) 0x7f97c009b760 con 0x7f97c8107840 2024-06-02T23:37:21.239 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:37:21.239 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:37:19.717233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97b80a2c20 msgr2=0x7f97b80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97b80a2c20 0x7f97b80a50e0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f97bc005ea0 tx=0x7f97bc005e30 comp rx=0 tx=0).stop 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 msgr2=0x7f97c8198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 0x7f97c8198950 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f97c0007ee0 tx=0x7f97c000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 shutdown_connections 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f97b80a2c20 0x7f97b80a50e0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f97c8107840 0x7f97c8198950 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97c8102740 0x7f97c819e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 --2- 172.21.15.52:0/3910526400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c81030f0 0x7f97c819ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 >> 172.21.15.52:0/3910526400 conn(0x7f97c80795f0 msgr2=0x7f97c8100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 shutdown_connections 2024-06-02T23:37:21.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:21.240+0000 7f97ceae0700 1 -- 172.21.15.52:0/3910526400 wait complete. 2024-06-02T23:37:21.242 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 48 2024-06-02T23:37:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:20 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:37:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:20 smithi123 ceph-mon[35977]: pgmap v1195: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 268 KiB/s wr, 128 op/s 2024-06-02T23:37:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:20 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:37:21.936 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24470 quiesce db --cancel --all' 2024-06-02T23:37:21.936 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- timeout 105 ceph tell mds.24470 quiesce db --cancel --all 2024-06-02T23:37:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:21 smithi174 ceph-mon[35994]: scrub summary: idle 2024-06-02T23:37:22.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:21 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3910526400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:21 smithi052 ceph-mon[32340]: scrub summary: idle 2024-06-02T23:37:22.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:21 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3910526400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:22.190 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:21 smithi123 ceph-mon[35977]: scrub summary: idle 2024-06-02T23:37:22.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:21 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3910526400' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:23.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:22 smithi174 ceph-mon[35994]: pgmap v1196: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 269 KiB/s wr, 132 op/s 2024-06-02T23:37:23.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:22 smithi052 ceph-mon[32340]: pgmap v1196: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 269 KiB/s wr, 132 op/s 2024-06-02T23:37:23.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:22 smithi123 ceph-mon[35977]: pgmap v1196: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 269 KiB/s wr, 132 op/s 2024-06-02T23:37:24.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 -- 172.21.15.52:0/1927679558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f0107840 msgr2=0x7f66f010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/1927679558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f0107840 0x7f66f010f3c0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f66e80099c0 tx=0x7f66e8009cd0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 -- 172.21.15.52:0/1927679558 shutdown_connections 2024-06-02T23:37:24.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/1927679558 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f0107840 0x7f66f010f3c0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/1927679558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 0x7f66f0102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.057 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/1927679558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f01030f0 0x7f66f0107110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 -- 172.21.15.52:0/1927679558 >> 172.21.15.52:0/1927679558 conn(0x7f66f00795f0 msgr2=0x7f66f0079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:24.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 -- 172.21.15.52:0/1927679558 shutdown_connections 2024-06-02T23:37:24.058 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.056+0000 7f66f7d5e700 1 -- 172.21.15.52:0/1927679558 wait complete. 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 Processor -- start 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 -- start start 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 0x7f66f0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 0x7f66f0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f0107840 0x7f66f0075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66f0112630 con 0x7f66f01030f0 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01124b0 con 0x7f66f0107840 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f7d5e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01127b0 con 0x7f66f0102740 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f52f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 0x7f66f0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f52f9700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 0x7f66f0074cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56918/0 (socket says 172.21.15.52:56918) 2024-06-02T23:37:24.060 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f52f9700 1 -- 172.21.15.52:0/2433678195 learned_addr learned my addr 172.21.15.52:0/2433678195 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.057+0000 7f66f5afa700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 0x7f66f0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f62fb700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f0107840 0x7f66f0075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f52f9700 1 -- 172.21.15.52:0/2433678195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 msgr2=0x7f66f0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f52f9700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 0x7f66f0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f52f9700 1 -- 172.21.15.52:0/2433678195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f0107840 msgr2=0x7f66f0075210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f52f9700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f0107840 0x7f66f0075210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f52f9700 1 -- 172.21.15.52:0/2433678195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66f01a6fd0 con 0x7f66f01030f0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f5afa700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 0x7f66f0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f52f9700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 0x7f66f0074cd0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c920 tx=0x7f66e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66eeffd700 1 -- 172.21.15.52:0/2433678195 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4007b00 con 0x7f66f01030f0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66f01a72c0 con 0x7f66f01030f0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66eeffd700 1 -- 172.21.15.52:0/2433678195 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e4004ce0 con 0x7f66f01030f0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66eeffd700 1 -- 172.21.15.52:0/2433678195 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e40054b0 con 0x7f66f01030f0 2024-06-02T23:37:24.061 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.058+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66f01a7850 con 0x7f66f01030f0 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.060+0000 7f66eeffd700 1 -- 172.21.15.52:0/2433678195 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f66e4004750 con 0x7f66f01030f0 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.061+0000 7f66eeffd700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66d80a2c20 0x7f66d80a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.061+0000 7f66eeffd700 1 -- 172.21.15.52:0/2433678195 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f66e40ceb80 con 0x7f66f01030f0 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.061+0000 7f66f5afa700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66d80a2c20 0x7f66d80a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f66f5afa700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66d80a2c20 0x7f66d80a50e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f66e0007fd0 tx=0x7f66e0007810 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 Processor -- start 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 -- start start 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 0x55d8f8ab8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ab2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d8f89a4fd0 con 0x55d8f8ab8470 2024-06-02T23:37:24.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8f8a038c0 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f67025e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8f896b4d0 con 0x55d8f8ab8d90 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f66f5afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 0x55d8f8ab8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f66f5afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 0x55d8f8ab8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56926/0 (socket says 172.21.15.52:56926) 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.062+0000 7f66f5afa700 1 -- 172.21.15.52:0/43588370 learned_addr learned my addr 172.21.15.52:0/43588370 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f52f9700 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ab2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f62fb700 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f5afa700 1 -- 172.21.15.52:0/43588370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 msgr2=0x55d8f8ab2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f5afa700 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ab2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f5afa700 1 -- 172.21.15.52:0/43588370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab2bc0 msgr2=0x55d8f8ab2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f5afa700 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f5afa700 1 -- 172.21.15.52:0/43588370 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f8952940 con 0x55d8f8ab8470 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f52f9700 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ab2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66f5afa700 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 0x55d8f8ab8850 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f66e0018850 tx=0x7f66e0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54e27052e8f04ea9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:24.065 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66d7fff700 1 -- 172.21.15.52:0/43588370 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0026070 con 0x55d8f8ab8470 2024-06-02T23:37:24.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66d7fff700 1 -- 172.21.15.52:0/43588370 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e00180d0 con 0x55d8f8ab8470 2024-06-02T23:37:24.066 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.063+0000 7f66d7fff700 1 -- 172.21.15.52:0/43588370 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0016670 con 0x55d8f8ab8470 2024-06-02T23:37:24.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 -- 172.21.15.52:0/43588370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 msgr2=0x55d8f8ab8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 0x55d8f8ab8850 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f66e0018850 tx=0x7f66e0011000 comp rx=0 tx=0).stop 2024-06-02T23:37:24.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 -- 172.21.15.52:0/43588370 shutdown_connections 2024-06-02T23:37:24.069 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8470 0x55d8f8ab8850 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 --2- 172.21.15.52:0/43588370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ab2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 -- 172.21.15.52:0/43588370 >> 172.21.15.52:0/43588370 conn(0x55d8f89df0f0 msgr2=0x55d8f8aad970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 -- 172.21.15.52:0/43588370 shutdown_connections 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.064+0000 7f67025e1b80 1 -- 172.21.15.52:0/43588370 wait complete. 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 -- start start 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 Processor -- start 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8abea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8470 0x55d8f8abbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8d90 0x55d8f8abc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d8f89a4030 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.070 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8f89ff880 con 0x55d8f8ab8d90 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d8f8abf770 con 0x55d8f8ab8470 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f67025e1b80 10 client.0 _ll_register_callbacks cb 0x55d8f8999620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f66f5afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8abea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f66f5afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8abea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56928/0 (socket says 172.21.15.52:56928) 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f66f5afa700 1 -- 172.21.15.52:0/3917384669 learned_addr learned my addr 172.21.15.52:0/3917384669 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f66f62fb700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8d90 0x55d8f8abc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.065+0000 7f66f52f9700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8470 0x55d8f8abbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66f5afa700 1 -- 172.21.15.52:0/3917384669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8470 msgr2=0x55d8f8abbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66f5afa700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8470 0x55d8f8abbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.071 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66f5afa700 1 -- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8d90 msgr2=0x55d8f8abc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66f5afa700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8d90 0x55d8f8abc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66f5afa700 1 -- 172.21.15.52:0/3917384669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f89c97c0 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66f5afa700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8abea80 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f66e0022f80 tx=0x7f66e0018360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f24778859b58f1be server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66d57fa700 1 -- 172.21.15.52:0/3917384669 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0026070 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66d57fa700 1 -- 172.21.15.52:0/3917384669 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f66e0021620 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e00180d0 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f66d57fa700 1 -- 172.21.15.52:0/3917384669 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f66e0021820 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f67025e1b80 10 client.22047 fetch_fsmap learned FSMap version 48 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.066+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d8f8ac9e20 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f66d57fa700 1 -- 172.21.15.52:0/3917384669 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e00180d0 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f66d57fa700 1 -- 172.21.15.52:0/3917384669 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f66e0032030 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f67025e1b80 10 client.22047 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:37:24.072 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f67025e1b80 10 client.22047 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f67025e1b80 20 client.22047 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8acc400 0x55d8f8ace7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f67025e1b80 4 client.22047 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f8a10dc0 con 0x55d8f8acc400 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f66f52f9700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8acc400 0x55d8f8ace7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f66d57fa700 10 client.22047 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.067+0000 7f66f52f9700 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8acc400 0x55d8f8ace7f0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d25522305d206aa server_cookie=396a02efd8266791 in_seq=0 out_seq=0 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f66d57fa700 1 -- 172.21.15.52:0/3917384669 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d8f8a10dc0 con 0x55d8f8acc400 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f66d57fa700 10 client.22047 handle_command_reply: tid=0 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f67025e1b80 1 client.22047 shutdown 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 msgr2=0x55d8f8abea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8abea80 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f66e0022f80 tx=0x7f66e0018360 comp rx=0 tx=0).stop 2024-06-02T23:37:24.073 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 shutdown_connections 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.068+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8acc400 0x55d8f8ace7f0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8d90 0x55d8f8abc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f66d57fa700 0 client.22047 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8abea80 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3917384669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8470 0x55d8f8abbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 >> 172.21.15.52:0/3917384669 conn(0x55d8f89df0f0 msgr2=0x55d8f8aac290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 shutdown_connections 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 1 -- 172.21.15.52:0/3917384669 wait complete. 2024-06-02T23:37:24.074 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.069+0000 7f67025e1b80 20 client.22047 trim_cache size 0 max 16384 2024-06-02T23:37:24.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 Processor -- start 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 -- start start 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8abaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ad3f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01aed10 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b0970 con 0x55d8f8ab8470 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f67025e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01af4a0 con 0x55d8f8ab8d90 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f66f5afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f66f5afa700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56944/0 (socket says 172.21.15.52:56944) 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f66f5afa700 1 -- 172.21.15.52:0/1535048426 learned_addr learned my addr 172.21.15.52:0/1535048426 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.089+0000 7f66f62fb700 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ad3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f52f9700 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8abaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f5afa700 1 -- 172.21.15.52:0/1535048426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 msgr2=0x55d8f8ad3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f5afa700 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ad3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f5afa700 1 -- 172.21.15.52:0/1535048426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 msgr2=0x55d8f8abaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f5afa700 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8abaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f5afa700 1 -- 172.21.15.52:0/1535048426 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f8aba340 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66f5afa700 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2680 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f66e0022cf0 tx=0x7f66e0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cb2667ef62c28a server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66d77fe700 1 -- 172.21.15.52:0/1535048426 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e0026070 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66d77fe700 1 -- 172.21.15.52:0/1535048426 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e0004d80 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.090+0000 7f66d77fe700 1 -- 172.21.15.52:0/1535048426 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e002a410 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 -- 172.21.15.52:0/1535048426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 msgr2=0x55d8f8ab2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2680 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f66e0022cf0 tx=0x7f66e0007a40 comp rx=0 tx=0).stop 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 -- 172.21.15.52:0/1535048426 shutdown_connections 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ad3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ab2680 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 --2- 172.21.15.52:0/1535048426 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8abaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 -- 172.21.15.52:0/1535048426 >> 172.21.15.52:0/1535048426 conn(0x55d8f89df0f0 msgr2=0x55d8f8aba590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 -- 172.21.15.52:0/1535048426 shutdown_connections 2024-06-02T23:37:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 -- 172.21.15.52:0/1535048426 wait complete. 2024-06-02T23:37:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 -- start start 2024-06-02T23:37:24.094 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.091+0000 7f67025e1b80 1 Processor -- start 2024-06-02T23:37:24.096 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ac4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8ac5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ac1a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01b02a0 con 0x55d8f8ab8d90 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01afba0 con 0x55d8f8ab8470 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f66f01accd0 con 0x55d8f8ab2bc0 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f67025e1b80 10 client.0 _ll_register_callbacks cb 0x55d8f8999620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ac1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ac1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:56960/0 (socket says 172.21.15.52:56960) 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 -- 172.21.15.52:0/3216888 learned_addr learned my addr 172.21.15.52:0/3216888 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f5afa700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ac4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f52f9700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8ac5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 -- 172.21.15.52:0/3216888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab2bc0 msgr2=0x55d8f8ac4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ac4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.097 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 -- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 msgr2=0x55d8f8ac5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8ac5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 -- 172.21.15.52:0/3216888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f8ac2270 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f52f9700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8ac5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66f62fb700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ac1a30 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f66e80099c0 tx=0x7f66e8009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396b736e152165fe server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.092+0000 7f66d57fa700 1 -- 172.21.15.52:0/3216888 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e80054c0 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f66d57fa700 1 -- 172.21.15.52:0/3216888 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f66e8005660 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66e0004d80 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f66d57fa700 1 -- 172.21.15.52:0/3216888 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f66e800b790 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 10 client.22053 fetch_fsmap learned FSMap version 48 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d8f8b40230 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f66d57fa700 1 -- 172.21.15.52:0/3216888 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f66e0004d80 con 0x55d8f8ab8d90 2024-06-02T23:37:24.098 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f66d57fa700 1 -- 172.21.15.52:0/3216888 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f66e800fa60 con 0x55d8f8ab8d90 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 10 client.22053 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 10 client.22053 resolve_mds: validated gid 24470 aka daemon mds.b 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 20 client.22053 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8b40970 0x55d8f8b42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 4 client.22053 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55d8f8a0b340 con 0x55d8f8b40970 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.093+0000 7f66f5afa700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8b40970 0x55d8f8b42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f66d57fa700 10 client.22053 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f66f5afa700 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8b40970 0x55d8f8b42d90 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9279bb9aa25aad2 server_cookie=8a7f4f91ae86ce2a in_seq=0 out_seq=0 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f66d57fa700 1 -- 172.21.15.52:0/3216888 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55d8f8a0b340 con 0x55d8f8b40970 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f66d57fa700 10 client.22053 handle_command_reply: tid=0 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 client.22053 shutdown 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 msgr2=0x55d8f8ac1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.099 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ac1a30 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f66e80099c0 tx=0x7f66e8009510 comp rx=0 tx=0).stop 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 shutdown_connections 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d8f8b40970 0x55d8f8b42d90 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d8f8ab8d90 0x55d8f8ac1a30 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d8f8ab2bc0 0x55d8f8ac4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f66d57fa700 0 client.22053 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 --2- 172.21.15.52:0/3216888 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f8ab8470 0x55d8f8ac5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.094+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 >> 172.21.15.52:0/3216888 conn(0x55d8f89df0f0 msgr2=0x55d8f8abae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 shutdown_connections 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f67025e1b80 1 -- 172.21.15.52:0/3216888 wait complete. 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f67025e1b80 20 client.22053 trim_cache size 0 max 16384 2024-06-02T23:37:24.100 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:37:24.101 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 48, 2024-06-02T23:37:24.101 INFO:teuthology.orchestra.run.smithi052.stdout: "leader": 24470, 2024-06-02T23:37:24.101 INFO:teuthology.orchestra.run.smithi052.stdout: "set_version": 80, 2024-06-02T23:37:24.101 INFO:teuthology.orchestra.run.smithi052.stdout: "sets": {} 2024-06-02T23:37:24.101 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:37:24.101 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66d80a2c20 msgr2=0x7f66d80a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66d80a2c20 0x7f66d80a50e0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f66e0007fd0 tx=0x7f66e0007810 comp rx=0 tx=0).stop 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 msgr2=0x7f66f0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 0x7f66f0074cd0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f66e400c920 tx=0x7f66e400cc30 comp rx=0 tx=0).stop 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 shutdown_connections 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f66d80a2c20 0x7f66d80a50e0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f66f0107840 0x7f66f0075210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f66f0102740 0x7f66f0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 --2- 172.21.15.52:0/2433678195 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f66f01030f0 0x7f66f0074cd0 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 >> 172.21.15.52:0/2433678195 conn(0x7f66f00795f0 msgr2=0x7f66f0100700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 shutdown_connections 2024-06-02T23:37:24.102 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:24.095+0000 7f66f7d5e700 1 -- 172.21.15.52:0/2433678195 wait complete. 2024-06-02T23:37:25.191 INFO:tasks.quiescer:done joining Quiescers 2024-06-02T23:37:25.191 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-06-02T23:37:25.202 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:37:25.211 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-06-02T23:37:25.243 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-06-02T23:37:25.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:25 smithi174 ceph-mon[35994]: pgmap v1197: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 276 KiB/s wr, 136 op/s 2024-06-02T23:37:25.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:25 smithi052 ceph-mon[32340]: pgmap v1197: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 276 KiB/s wr, 136 op/s 2024-06-02T23:37:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:25 smithi123 ceph-mon[35977]: pgmap v1197: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 261 KiB/s rd, 276 KiB/s wr, 136 op/s 2024-06-02T23:37:27.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:27 smithi174 ceph-mon[35994]: pgmap v1198: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 153 KiB/s wr, 116 op/s 2024-06-02T23:37:27.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:27 smithi052 ceph-mon[32340]: pgmap v1198: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 153 KiB/s wr, 116 op/s 2024-06-02T23:37:27.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:27 smithi123 ceph-mon[35977]: pgmap v1198: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 144 KiB/s rd, 153 KiB/s wr, 116 op/s 2024-06-02T23:37:29.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:29 smithi174 ceph-mon[35994]: pgmap v1199: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 136 KiB/s wr, 94 op/s 2024-06-02T23:37:29.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:29 smithi052 ceph-mon[32340]: pgmap v1199: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 136 KiB/s wr, 94 op/s 2024-06-02T23:37:29.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:29 smithi123 ceph-mon[35977]: pgmap v1199: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 130 KiB/s rd, 136 KiB/s wr, 94 op/s 2024-06-02T23:37:31.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:31 smithi174 ceph-mon[35994]: pgmap v1200: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 58 op/s 2024-06-02T23:37:31.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:31 smithi052 ceph-mon[32340]: pgmap v1200: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 58 op/s 2024-06-02T23:37:31.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:31 smithi123 ceph-mon[35977]: pgmap v1200: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 65 KiB/s wr, 58 op/s 2024-06-02T23:37:32.618 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:32 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:37:32.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:32 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:37:32.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:32 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:37:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: pgmap v1201: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 18 op/s 2024-06-02T23:37:33.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:37:33.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:33 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:37:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: pgmap v1201: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 18 op/s 2024-06-02T23:37:33.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:37:33.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:33 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:37:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: pgmap v1201: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.2 KiB/s wr, 18 op/s 2024-06-02T23:37:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:33.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi123 to 176.1M 2024-06-02T23:37:33.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:33 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi123 to 184732262: error parsing value: Value '184732262' is below minimum 939524096 2024-06-02T23:37:35.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:35 smithi174 ceph-mon[35994]: pgmap v1202: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-06-02T23:37:35.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:35 smithi052 ceph-mon[32340]: pgmap v1202: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-06-02T23:37:35.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:35 smithi123 ceph-mon[35977]: pgmap v1202: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 15 op/s 2024-06-02T23:37:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: pgmap v1203: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-02T23:37:37.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:37:37.644 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: pgmap v1203: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-02T23:37:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:37:37.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: pgmap v1203: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 767 B/s wr, 10 op/s 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-06-02T23:37:37.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: Adjusting osd_memory_target on smithi174 to 176.1M 2024-06-02T23:37:37.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: Unable to set osd_memory_target on smithi174 to 184718643: error parsing value: Value '184718643' is below minimum 939524096 2024-06-02T23:37:37.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:37:37.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:37:37.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:37:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:39 smithi123 ceph-mon[35977]: pgmap v1204: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-02T23:37:39.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:39 smithi174 ceph-mon[35994]: pgmap v1204: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-02T23:37:39.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:39 smithi052 ceph-mon[32340]: pgmap v1204: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-02T23:37:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:41 smithi123 ceph-mon[35977]: pgmap v1205: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2024-06-02T23:37:41.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:41 smithi174 ceph-mon[35994]: pgmap v1205: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2024-06-02T23:37:41.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:41 smithi052 ceph-mon[32340]: pgmap v1205: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 11 op/s 2024-06-02T23:37:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:43 smithi123 ceph-mon[35977]: pgmap v1206: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T23:37:43.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:43 smithi174 ceph-mon[35994]: pgmap v1206: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T23:37:43.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:43 smithi052 ceph-mon[32340]: pgmap v1206: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T23:37:45.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:45 smithi174 ceph-mon[35994]: pgmap v1207: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 12 op/s 2024-06-02T23:37:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:45 smithi123 ceph-mon[35977]: pgmap v1207: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 12 op/s 2024-06-02T23:37:45.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:45 smithi052 ceph-mon[32340]: pgmap v1207: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 12 op/s 2024-06-02T23:37:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:47 smithi123 ceph-mon[35977]: pgmap v1208: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T23:37:47.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:47 smithi174 ceph-mon[35994]: pgmap v1208: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T23:37:47.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:47 smithi052 ceph-mon[32340]: pgmap v1208: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-06-02T23:37:49.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:49 smithi123 ceph-mon[35977]: pgmap v1209: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-02T23:37:49.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:49 smithi174 ceph-mon[35994]: pgmap v1209: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-02T23:37:49.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:49 smithi052 ceph-mon[32340]: pgmap v1209: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-06-02T23:37:50.482 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-06-02T23:37:50.747 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:51.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:51 smithi123 ceph-mon[35977]: pgmap v1210: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:37:51.871 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:51 smithi052 ceph-mon[32340]: pgmap v1210: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:37:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:51 smithi174 ceph-mon[35994]: pgmap v1210: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:37:52.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 -- 172.21.15.52:0/445711226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102520 msgr2=0x7f7f4c1029a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 --2- 172.21.15.52:0/445711226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102520 0x7f7f4c1029a0 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7f7f340099c0 tx=0x7f7f34009cd0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 -- 172.21.15.52:0/445711226 shutdown_connections 2024-06-02T23:37:52.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 --2- 172.21.15.52:0/445711226 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c10f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 --2- 172.21.15.52:0/445711226 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4c1085c0 0x7f7f4c1089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 --2- 172.21.15.52:0/445711226 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102520 0x7f7f4c1029a0 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 -- 172.21.15.52:0/445711226 >> 172.21.15.52:0/445711226 conn(0x7f7f4c079680 msgr2=0x7f7f4c079a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 -- 172.21.15.52:0/445711226 shutdown_connections 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.528+0000 7f7f51a86700 1 -- 172.21.15.52:0/445711226 wait complete. 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 Processor -- start 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 -- start start 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c102520 0x7f7f4c19e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c19edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4c1085c0 0x7f7f4c198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1127a0 con 0x7f7f4c102ee0 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112620 con 0x7f7f4c102520 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.529+0000 7f7f51a86700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c112920 con 0x7f7f4c1085c0 2024-06-02T23:37:52.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c19edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c19edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38850/0 (socket says 172.21.15.52:38850) 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 -- 172.21.15.52:0/560186805 learned_addr learned my addr 172.21.15.52:0/560186805 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c102520 0x7f7f4c19e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4c1085c0 0x7f7f4c198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 -- 172.21.15.52:0/560186805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4c1085c0 msgr2=0x7f7f4c198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4c1085c0 0x7f7f4c198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 -- 172.21.15.52:0/560186805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c102520 msgr2=0x7f7f4c19e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c102520 0x7f7f4c19e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 -- 172.21.15.52:0/560186805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4c199200 con 0x7f7f4c102ee0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f4affd700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c19edd0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009fd0 tx=0x7f7f340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f48ff9700 1 -- 172.21.15.52:0/560186805 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34027070 con 0x7f7f4c102ee0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f48ff9700 1 -- 172.21.15.52:0/560186805 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f34004050 con 0x7f7f4c102ee0 2024-06-02T23:37:52.533 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4c199490 con 0x7f7f4c102ee0 2024-06-02T23:37:52.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f48ff9700 1 -- 172.21.15.52:0/560186805 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f34022d80 con 0x7f7f4c102ee0 2024-06-02T23:37:52.534 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.530+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f4c1a5760 con 0x7f7f4c102ee0 2024-06-02T23:37:52.536 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.533+0000 7f7f48ff9700 1 -- 172.21.15.52:0/560186805 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f7f3402a070 con 0x7f7f4c102ee0 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.533+0000 7f7f48ff9700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f48ff9700 1 -- 172.21.15.52:0/560186805 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7f340cff60 con 0x7f7f4c102ee0 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c100bb0 tx=0x7f7f3c00d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 Processor -- start 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 -- start start 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27370 0x5622aeb27750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb21670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb21ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5622aea0ff60 con 0x5622aeb21bb0 2024-06-02T23:37:52.537 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622aea6e850 con 0x5622aeb27c90 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.534+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622ae9d6460 con 0x5622aeb27370 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb21ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb21ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38858/0 (socket says 172.21.15.52:38858) 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 -- 172.21.15.52:0/2167998499 learned_addr learned my addr 172.21.15.52:0/2167998499 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4affd700 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb21670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27370 0x5622aeb27750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 -- 172.21.15.52:0/2167998499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27370 msgr2=0x5622aeb27750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27370 0x5622aeb27750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 -- 172.21.15.52:0/2167998499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 msgr2=0x5622aeb21670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb21670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 -- 172.21.15.52:0/2167998499 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622ae9bd8d0 con 0x5622aeb21bb0 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4affd700 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb21670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:52.538 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb21ff0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400b810 tx=0x7f7f4400bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7739272b3592579d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f41ffb700 1 -- 172.21.15.52:0/2167998499 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44014620 con 0x5622aeb21bb0 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f41ffb700 1 -- 172.21.15.52:0/2167998499 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f44014c70 con 0x5622aeb21bb0 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.535+0000 7f7f41ffb700 1 -- 172.21.15.52:0/2167998499 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f44012d30 con 0x5622aeb21bb0 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 -- 172.21.15.52:0/2167998499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 msgr2=0x5622aeb21ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb21ff0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400b810 tx=0x7f7f4400bb20 comp rx=0 tx=0).stop 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 -- 172.21.15.52:0/2167998499 shutdown_connections 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb21ff0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27370 0x5622aeb27750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/2167998499 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb21670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 -- 172.21.15.52:0/2167998499 >> 172.21.15.52:0/2167998499 conn(0x5622aea4a080 msgr2=0x5622aeb18220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 -- 172.21.15.52:0/2167998499 shutdown_connections 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 -- 172.21.15.52:0/2167998499 wait complete. 2024-06-02T23:37:52.539 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.536+0000 7f7f5c309b80 1 -- start start 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 Processor -- start 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb2d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb2acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb2b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5622aea0efc0 con 0x5622aeb21bb0 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622aea6a810 con 0x5622aeb27370 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5622aeb2e680 con 0x5622aeb27c90 2024-06-02T23:37:52.543 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f5c309b80 10 client.0 _ll_register_callbacks cb 0x5622aea045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb2d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb2d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38872/0 (socket says 172.21.15.52:38872) 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/3960030249 learned_addr learned my addr 172.21.15.52:0/3960030249 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb2b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4affd700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb2acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/3960030249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 msgr2=0x5622aeb2b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb2b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 msgr2=0x5622aeb2acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb2acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/3960030249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622aea34750 con 0x5622aeb21bb0 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb2b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.537+0000 7f7f4affd700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb2acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb2d9b0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c016ed0 tx=0x7f7f3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc9cb5e53677d42 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:52.544 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f2affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f2affd700 1 -- 172.21.15.52:0/3960030249 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c022470 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f2affd700 1 -- 172.21.15.52:0/3960030249 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7f3c022610 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f44014c70 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f2affd700 1 -- 172.21.15.52:0/3960030249 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7f3c022810 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 10 client.22065 fetch_fsmap learned FSMap version 48 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5622aeb38dd0 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f2affd700 1 -- 172.21.15.52:0/3960030249 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f44014c70 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f2affd700 1 -- 172.21.15.52:0/3960030249 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f7f3c02ba50 con 0x5622aeb21bb0 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 10 client.22065 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 10 client.22065 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 20 client.22065 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeb3b410 0x5622aeb3d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 4 client.22065 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.538+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622aea7bd50 con 0x5622aeb3b410 2024-06-02T23:37:52.545 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.539+0000 7f7f4affd700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeb3b410 0x5622aeb3d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.539+0000 7f7f4affd700 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeb3b410 0x5622aeb3d8b0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a853d5658e7159b1 server_cookie=9b8bca027eca1751 in_seq=0 out_seq=0 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.539+0000 7f7f2affd700 10 client.22065 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f2affd700 1 -- 172.21.15.52:0/3960030249 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5622aea7bd50 con 0x5622aeb3b410 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f2affd700 10 client.22065 handle_command_reply: tid=0 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 client.22065 shutdown 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 msgr2=0x5622aeb2d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb2d9b0 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c016ed0 tx=0x7f7f3c004ad0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 shutdown_connections 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeb3b410 0x5622aeb3d8b0 crc :-1 s=READY pgs=543 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb2b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb2d9b0 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/3960030249 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb2acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.546 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 >> 172.21.15.52:0/3960030249 conn(0x5622aea4a080 msgr2=0x5622aeb17700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:52.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f2affd700 0 client.22065 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:52.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 shutdown_connections 2024-06-02T23:37:52.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 1 -- 172.21.15.52:0/3960030249 wait complete. 2024-06-02T23:37:52.547 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.540+0000 7f7f5c309b80 20 client.22065 trim_cache size 0 max 16384 2024-06-02T23:37:52.562 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.561+0000 7f7f5c309b80 1 Processor -- start 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.561+0000 7f7f5c309b80 1 -- start start 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb286e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb17e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb183c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aa860 con 0x5622aeb21bb0 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ac4c0 con 0x5622aeb27370 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1aaff0 con 0x5622aeb27c90 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb286e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.563 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb286e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38884/0 (socket says 172.21.15.52:38884) 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/1854658848 learned_addr learned my addr 172.21.15.52:0/1854658848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/1854658848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 msgr2=0x5622aeb183c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb183c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/1854658848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 msgr2=0x5622aeb17e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb17e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 -- 172.21.15.52:0/1854658848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622aeb29240 con 0x5622aeb21bb0 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb286e0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000f80 tx=0x7f7f3c00adb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10db0823f47b1959 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f417fa700 1 -- 172.21.15.52:0/1854658848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c03c070 con 0x5622aeb21bb0 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f417fa700 1 -- 172.21.15.52:0/1854658848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f3c027070 con 0x5622aeb21bb0 2024-06-02T23:37:52.564 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.562+0000 7f7f417fa700 1 -- 172.21.15.52:0/1854658848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c016080 con 0x5622aeb21bb0 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 -- 172.21.15.52:0/1854658848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 msgr2=0x5622aeb286e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb286e0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c000f80 tx=0x7f7f3c00adb0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 -- 172.21.15.52:0/1854658848 shutdown_connections 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb27c90 0x5622aeb183c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb21bb0 0x5622aeb286e0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/1854658848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27370 0x5622aeb17e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 -- 172.21.15.52:0/1854658848 >> 172.21.15.52:0/1854658848 conn(0x5622aea4a080 msgr2=0x5622aeb2cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:52.568 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 -- 172.21.15.52:0/1854658848 shutdown_connections 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 -- 172.21.15.52:0/1854658848 wait complete. 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.563+0000 7f7f5c309b80 1 -- start start 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 Processor -- start 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb21bb0 0x5622aeb33e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 0x5622aeb2f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb2f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1abdf0 con 0x5622aeb27370 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1ab6f0 con 0x5622aeb27c90 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7f4c1a8820 con 0x5622aeb21bb0 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f5c309b80 10 client.0 _ll_register_callbacks cb 0x5622aea045b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 0x5622aeb2f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 0x5622aeb2f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38890/0 (socket says 172.21.15.52:38890) 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 -- 172.21.15.52:0/4018537251 learned_addr learned my addr 172.21.15.52:0/4018537251 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:52.569 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb2f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb21bb0 0x5622aeb33e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 -- 172.21.15.52:0/4018537251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb21bb0 msgr2=0x5622aeb33e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb21bb0 0x5622aeb33e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 -- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 msgr2=0x5622aeb2f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb2f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 -- 172.21.15.52:0/4018537251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622aeb2fd70 con 0x5622aeb27370 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4bfff700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb2f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f4affd700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 0x5622aeb2f1f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7f340059d0 tx=0x7f7f34005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948eebba93f2f756 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.564+0000 7f7f2affd700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f2affd700 1 -- 172.21.15.52:0/4018537251 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f340a0c30 con 0x5622aeb27370 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f2affd700 1 -- 172.21.15.52:0/4018537251 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f7f340a30b0 con 0x5622aeb27370 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f3c027070 con 0x5622aeb27370 2024-06-02T23:37:52.570 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f2affd700 1 -- 172.21.15.52:0/4018537251 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f7f340a5460 con 0x5622aeb27370 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 10 client.22071 fetch_fsmap learned FSMap version 48 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5622aeba5fe0 con 0x5622aeb27370 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f2affd700 1 -- 172.21.15.52:0/4018537251 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f7f3c027070 con 0x5622aeb27370 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f2affd700 1 -- 172.21.15.52:0/4018537251 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f7f340a4fd0 con 0x5622aeb27370 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 10 client.22071 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 10 client.22071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 20 client.22071 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeba64c0 0x5622aeba88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 4 client.22071 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.565+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5622aea762d0 con 0x5622aeba64c0 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeba64c0 0x5622aeba88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f4b7fe700 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeba64c0 0x5622aeba88e0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f662f225dbba36 server_cookie=e50c527a83c843fe in_seq=0 out_seq=0 2024-06-02T23:37:52.571 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f2affd700 10 client.22071 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f2affd700 1 -- 172.21.15.52:0/4018537251 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5622aea762d0 con 0x5622aeba64c0 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f2affd700 10 client.22071 handle_command_reply: tid=0 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f5c309b80 1 client.22071 shutdown 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 msgr2=0x5622aeb2f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 0x5622aeb2f1f0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f7f340059d0 tx=0x7f7f34005fb0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 shutdown_connections 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.566+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x5622aeba64c0 0x5622aeba88e0 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f2affd700 0 client.22071 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622aeb27c90 0x5622aeb2f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5622aeb21bb0 0x5622aeb33e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 1 --2- 172.21.15.52:0/4018537251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x5622aeb27370 0x5622aeb2f1f0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 >> 172.21.15.52:0/4018537251 conn(0x5622aea4a080 msgr2=0x5622aeb1cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 shutdown_connections 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 1 -- 172.21.15.52:0/4018537251 wait complete. 2024-06-02T23:37:52.572 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f5c309b80 20 client.22071 trim_cache size 0 max 16384 2024-06-02T23:37:52.573 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:37:52.573 INFO:teuthology.orchestra.run.smithi052.stdout: "status": "no active scrubs running", 2024-06-02T23:37:52.573 INFO:teuthology.orchestra.run.smithi052.stdout: "scrubs": {} 2024-06-02T23:37:52.573 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 msgr2=0x7f7f380a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c100bb0 tx=0x7f7f3c00d040 comp rx=0 tx=0).stop 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 msgr2=0x7f7f4c19edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c19edd0 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7f7f34009fd0 tx=0x7f7f340047b0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 shutdown_connections 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f7f380a2ca0 0x7f7f380a5160 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7f4c1085c0 0x7f7f4c198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7f4c102520 0x7f7f4c19e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 --2- 172.21.15.52:0/560186805 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f7f4c102ee0 0x7f7f4c19edd0 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.567+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 >> 172.21.15.52:0/560186805 conn(0x7f7f4c079680 msgr2=0x7f7f4c1011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.568+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 shutdown_connections 2024-06-02T23:37:52.574 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:52.568+0000 7f7f51a86700 1 -- 172.21.15.52:0/560186805 wait complete. 2024-06-02T23:37:53.390 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-06-02T23:37:53.391 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:37:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:53 smithi052 ceph-mon[32340]: pgmap v1211: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:37:53.651 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:53.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:53 smithi123 ceph-mon[35977]: pgmap v1211: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:37:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:53 smithi174 ceph-mon[35994]: pgmap v1211: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.410+0000 7f3d70cea700 1 -- 172.21.15.52:0/3484658388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 msgr2=0x7f3d6c06b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.410+0000 7f3d70cea700 1 --2- 172.21.15.52:0/3484658388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c06b520 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f3d54009990 tx=0x7f3d54009ca0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 -- 172.21.15.52:0/3484658388 shutdown_connections 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 --2- 172.21.15.52:0/3484658388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d6c10ede0 0x7f3d6c1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 --2- 172.21.15.52:0/3484658388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c1033e0 0x7f3d6c06ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 --2- 172.21.15.52:0/3484658388 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c06b520 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 -- 172.21.15.52:0/3484658388 >> 172.21.15.52:0/3484658388 conn(0x7f3d6c078340 msgr2=0x7f3d6c078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 -- 172.21.15.52:0/3484658388 shutdown_connections 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 -- 172.21.15.52:0/3484658388 wait complete. 2024-06-02T23:37:55.422 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 Processor -- start 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.411+0000 7f3d70cea700 1 -- start start 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c1a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c1033e0 0x7f3d6c1a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d6c10ede0 0x7f3d6c19ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c114890 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c114710 con 0x7f3d6c1033e0 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3d6c114a10 con 0x7f3d6c10ede0 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c1a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c1a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36534/0 (socket says 172.21.15.52:36534) 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 -- 172.21.15.52:0/1487576018 learned_addr learned my addr 172.21.15.52:0/1487576018 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d69d9b700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c1033e0 0x7f3d6c1a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 -- 172.21.15.52:0/1487576018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d6c10ede0 msgr2=0x7f3d6c19ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d6c10ede0 0x7f3d6c19ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.423 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 -- 172.21.15.52:0/1487576018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c1033e0 msgr2=0x7f3d6c1a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c1033e0 0x7f3d6c1a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 -- 172.21.15.52:0/1487576018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d6c19b410 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d6a59c700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c1a0a60 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0060b0 tx=0x7f3d5c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c01a070 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d6c19b700 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3d5c00de20 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.412+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c014720 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.413+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d6c1a79e0 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.413+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d6c06a410 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.415+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f3d5c01d070 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.416+0000 7f3d637fe700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d580a2c20 0x7f3d580a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.416+0000 7f3d69d9b700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d580a2c20 0x7f3d580a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.416+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f3d5c0cedb0 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.416+0000 7f3d69d9b700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d580a2c20 0x7f3d580a50e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3d54006010 tx=0x7f3d5401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:55.424 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.419+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3d5c010070 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:55 smithi052 ceph-mon[32340]: pgmap v1212: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:37:55.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.672+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d6c1a7bb0 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.673 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.673+0000 7f3d637fe700 1 -- 172.21.15.52:0/1487576018 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11953 (secure 0 0 0) 0x7f3d5c097070 con 0x7f3d6c06b0a0 2024-06-02T23:37:55.673 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:37:55.674 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:37:19.717233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:37:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d580a2c20 msgr2=0x7f3d580a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d580a2c20 0x7f3d580a50e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3d54006010 tx=0x7f3d5401a040 comp rx=0 tx=0).stop 2024-06-02T23:37:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 msgr2=0x7f3d6c1a0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c1a0a60 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0060b0 tx=0x7f3d5c007f90 comp rx=0 tx=0).stop 2024-06-02T23:37:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 shutdown_connections 2024-06-02T23:37:55.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f3d580a2c20 0x7f3d580a50e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3d6c10ede0 0x7f3d6c19ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3d6c06b0a0 0x7f3d6c1a0a60 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 --2- 172.21.15.52:0/1487576018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c1033e0 0x7f3d6c1a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 >> 172.21.15.52:0/1487576018 conn(0x7f3d6c078340 msgr2=0x7f3d6c100760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 shutdown_connections 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:55.675+0000 7f3d70cea700 1 -- 172.21.15.52:0/1487576018 wait complete. 2024-06-02T23:37:55.677 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 48 2024-06-02T23:37:55.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:55 smithi123 ceph-mon[35977]: pgmap v1212: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:37:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:55 smithi174 ceph-mon[35994]: pgmap v1212: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:37:56.584 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:4 damage ls 2024-06-02T23:37:56.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:56 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1487576018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:56.839 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:56.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:56 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1487576018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:56.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:56 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1487576018' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:37:57.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:57 smithi052 ceph-mon[32340]: pgmap v1213: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:37:57.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:57 smithi123 ceph-mon[35977]: pgmap v1213: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:37:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:57 smithi174 ceph-mon[35994]: pgmap v1213: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:37:58.631 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 -- 172.21.15.52:0/1858399997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2524106720 msgr2=0x7f252410e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 --2- 172.21.15.52:0/1858399997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2524106720 0x7f252410e2c0 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f251c00b0d0 tx=0x7f251c00b3e0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 -- 172.21.15.52:0/1858399997 shutdown_connections 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 --2- 172.21.15.52:0/1858399997 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2524106720 0x7f252410e2c0 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 --2- 172.21.15.52:0/1858399997 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25241019d0 0x7f2524101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 --2- 172.21.15.52:0/1858399997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524102380 0x7f2524105ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 -- 172.21.15.52:0/1858399997 >> 172.21.15.52:0/1858399997 conn(0x7f2524078340 msgr2=0x7f2524078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 -- 172.21.15.52:0/1858399997 shutdown_connections 2024-06-02T23:37:58.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.631+0000 7f25296ba700 1 -- 172.21.15.52:0/1858399997 wait complete. 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 Processor -- start 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 -- start start 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 0x7f252419c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2524102380 0x7f252419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524106720 0x7f2524196790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25241104f0 con 0x7f25241019d0 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2524110370 con 0x7f2524102380 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25296ba700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2524110670 con 0x7f2524106720 2024-06-02T23:37:58.634 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 0x7f252419c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 0x7f252419c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36548/0 (socket says 172.21.15.52:36548) 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f2522ffd700 1 -- 172.21.15.52:0/4223674825 learned_addr learned my addr 172.21.15.52:0/4223674825 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.632+0000 7f25237fe700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524106720 0x7f2524196790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f25227fc700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2524102380 0x7f252419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f2522ffd700 1 -- 172.21.15.52:0/4223674825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524106720 msgr2=0x7f2524196790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f2522ffd700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524106720 0x7f2524196790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f2522ffd700 1 -- 172.21.15.52:0/4223674825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2524102380 msgr2=0x7f252419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f2522ffd700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2524102380 0x7f252419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f2522ffd700 1 -- 172.21.15.52:0/4223674825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2524197070 con 0x7f25241019d0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f25237fe700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524106720 0x7f2524196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f2522ffd700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 0x7f252419c670 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f251400d950 tx=0x7f251400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f251bfff700 1 -- 172.21.15.52:0/4223674825 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514016070 con 0x7f25241019d0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2524197360 con 0x7f25241019d0 2024-06-02T23:37:58.635 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f251bfff700 1 -- 172.21.15.52:0/4223674825 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2514010460 con 0x7f25241019d0 2024-06-02T23:37:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f251bfff700 1 -- 172.21.15.52:0/4223674825 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251400f620 con 0x7f25241019d0 2024-06-02T23:37:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.633+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25241a35f0 con 0x7f25241019d0 2024-06-02T23:37:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.635+0000 7f251bfff700 1 -- 172.21.15.52:0/4223674825 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2514019070 con 0x7f25241019d0 2024-06-02T23:37:58.636 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.636+0000 7f251bfff700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25100a4e30 0x7f25100a72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.636+0000 7f25227fc700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25100a4e30 0x7f25100a72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.637 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.636+0000 7f251bfff700 1 -- 172.21.15.52:0/4223674825 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f25140cf0b0 con 0x7f25241019d0 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f25227fc700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25100a4e30 0x7f25100a72f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2524197f00 tx=0x7f250c00d4b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 Processor -- start 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 -- start start 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c841fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c841a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2c836b810 con 0x55e2c8419f20 2024-06-02T23:37:58.638 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2c8313720 con 0x55e2c841ff60 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2533f3db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2c83677d0 con 0x55e2c841f640 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f25237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c841a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f25237fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c841a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36558/0 (socket says 172.21.15.52:36558) 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f25227fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f25237fe700 1 -- 172.21.15.52:0/2827316981 learned_addr learned my addr 172.21.15.52:0/2827316981 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.637+0000 7f2522ffd700 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c841fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25237fe700 1 -- 172.21.15.52:0/2827316981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 msgr2=0x55e2c841fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25237fe700 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c841fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25237fe700 1 -- 172.21.15.52:0/2827316981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 msgr2=0x55e2c84199e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25237fe700 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84199e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25237fe700 1 -- 172.21.15.52:0/2827316981 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c8309f50 con 0x55e2c8419f20 2024-06-02T23:37:58.639 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25227fc700 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:37:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25237fe700 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c841a340 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f251c009bb0 tx=0x7f251c009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fd2901b329e4ad server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:58.642 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25197fa700 1 -- 172.21.15.52:0/2827316981 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c020030 con 0x55e2c8419f20 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25197fa700 1 -- 172.21.15.52:0/2827316981 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f251c004a50 con 0x55e2c8419f20 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.638+0000 7f25197fa700 1 -- 172.21.15.52:0/2827316981 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f251c007bc0 con 0x55e2c8419f20 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 -- 172.21.15.52:0/2827316981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 msgr2=0x55e2c841a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c841a340 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f251c009bb0 tx=0x7f251c009be0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 -- 172.21.15.52:0/2827316981 shutdown_connections 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c841a340 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c841fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2827316981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84199e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 -- 172.21.15.52:0/2827316981 >> 172.21.15.52:0/2827316981 conn(0x55e2c8344cc0 msgr2=0x55e2c8414850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 -- 172.21.15.52:0/2827316981 shutdown_connections 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 -- 172.21.15.52:0/2827316981 wait complete. 2024-06-02T23:37:58.643 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.639+0000 7f2533f3db80 1 -- start start 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 Processor -- start 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c8425c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8422f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84234b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e2c8426970 con 0x55e2c8419f20 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e2c830bbf0 con 0x55e2c841ff60 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e2c8426af0 con 0x55e2c841f640 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2533f3db80 10 client.0 _ll_register_callbacks cb 0x55e2c8305920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c8425c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c8425c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36560/0 (socket says 172.21.15.52:36560) 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 -- 172.21.15.52:0/849211517 learned_addr learned my addr 172.21.15.52:0/849211517 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f25237fe700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84234b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 -- 172.21.15.52:0/849211517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 msgr2=0x55e2c8422f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.644 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f25227fc700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8422f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8422f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 -- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 msgr2=0x55e2c84234b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84234b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 -- 172.21.15.52:0/849211517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c82ba8d0 con 0x55e2c8419f20 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f25237fe700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f2522ffd700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c8425c70 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f251400b2f0 tx=0x7f2514010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77dcb62a973e126 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.640+0000 7f250a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f250a7fc700 1 -- 172.21.15.52:0/849211517 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25140b4060 con 0x55e2c8419f20 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f250a7fc700 1 -- 172.21.15.52:0/849211517 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2514010610 con 0x55e2c8419f20 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f251c004a50 con 0x55e2c8419f20 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f250a7fc700 1 -- 172.21.15.52:0/849211517 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f2514020050 con 0x55e2c8419f20 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 10 client.22089 fetch_fsmap learned FSMap version 48 2024-06-02T23:37:58.645 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e2c84312e0 con 0x55e2c8419f20 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f250a7fc700 1 -- 172.21.15.52:0/849211517 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f251c004a50 con 0x55e2c8419f20 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f250a7fc700 1 -- 172.21.15.52:0/849211517 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f25140860b0 con 0x55e2c8419f20 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 10 client.22089 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 10 client.22089 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 20 client.22089 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c8433970 0x55e2c8435d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 4 client.22089 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.641+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2c8378d10 con 0x55e2c8433970 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.642+0000 7f25227fc700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c8433970 0x55e2c8435d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.642+0000 7f25227fc700 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c8433970 0x55e2c8435d80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bc076151a0f1e5c4 server_cookie=12b872bdcc08b9c7 in_seq=0 out_seq=0 2024-06-02T23:37:58.646 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.642+0000 7f250a7fc700 10 client.22089 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:37:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f250a7fc700 1 -- 172.21.15.52:0/849211517 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e2c8378d10 con 0x55e2c8433970 2024-06-02T23:37:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f250a7fc700 10 client.22089 handle_command_reply: tid=0 2024-06-02T23:37:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 client.22089 shutdown 2024-06-02T23:37:58.647 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 msgr2=0x55e2c8425c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c8425c70 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f251400b2f0 tx=0x7f2514010ad0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 shutdown_connections 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c8433970 0x55e2c8435d80 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f250a7fc700 0 client.22089 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c84234b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c8425c70 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 --2- 172.21.15.52:0/849211517 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8422f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 >> 172.21.15.52:0/849211517 conn(0x55e2c8344cc0 msgr2=0x55e2c84189d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 shutdown_connections 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 1 -- 172.21.15.52:0/849211517 wait complete. 2024-06-02T23:37:58.648 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.643+0000 7f2533f3db80 20 client.22089 trim_cache size 0 max 16384 2024-06-02T23:37:58.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.658+0000 7f2533f3db80 1 Processor -- start 2024-06-02T23:37:58.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.658+0000 7f2533f3db80 1 -- start start 2024-06-02T23:37:58.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c84199e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8420bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c8421ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.660 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2533f3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25241aa730 con 0x55e2c8419f20 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2533f3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25241ac390 con 0x55e2c841ff60 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2533f3db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25241aaec0 con 0x55e2c841f640 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c84199e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c84199e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36566/0 (socket says 172.21.15.52:36566) 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 -- 172.21.15.52:0/2869131415 learned_addr learned my addr 172.21.15.52:0/2869131415 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f25237fe700 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c8421ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 -- 172.21.15.52:0/2869131415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 msgr2=0x55e2c8420bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8420bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 -- 172.21.15.52:0/2869131415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 msgr2=0x55e2c8421ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c8421ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 -- 172.21.15.52:0/2869131415 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2c84214e0 con 0x55e2c8419f20 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f25237fe700 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c8421ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:37:58.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2522ffd700 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c84199e0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2514012040 tx=0x7f2514099df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1c87f2e3d30188 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.659+0000 7f2518ff9700 1 -- 172.21.15.52:0/2869131415 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514097b80 con 0x55e2c8419f20 2024-06-02T23:37:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2518ff9700 1 -- 172.21.15.52:0/2869131415 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25140a4550 con 0x55e2c8419f20 2024-06-02T23:37:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2518ff9700 1 -- 172.21.15.52:0/2869131415 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2514010610 con 0x55e2c8419f20 2024-06-02T23:37:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 -- 172.21.15.52:0/2869131415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 msgr2=0x55e2c84199e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.662 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c84199e0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f2514012040 tx=0x7f2514099df0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 -- 172.21.15.52:0/2869131415 shutdown_connections 2024-06-02T23:37:58.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c841ff60 0x55e2c8421ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c8419f20 0x55e2c84199e0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 --2- 172.21.15.52:0/2869131415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841f640 0x55e2c8420bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.665 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 -- 172.21.15.52:0/2869131415 >> 172.21.15.52:0/2869131415 conn(0x55e2c8344cc0 msgr2=0x55e2c840de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 -- 172.21.15.52:0/2869131415 shutdown_connections 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.660+0000 7f2533f3db80 1 -- 172.21.15.52:0/2869131415 wait complete. 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 -- start start 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 Processor -- start 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c8419f20 0x55e2c842c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 0x55e2c843b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841ff60 0x55e2c843b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f25241abcc0 con 0x55e2c841f640 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f25241ab5c0 con 0x55e2c8419f20 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f25241a86f0 con 0x55e2c841ff60 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2533f3db80 10 client.0 _ll_register_callbacks cb 0x55e2c8305920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f25227fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 0x55e2c843b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.666 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f25227fc700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 0x55e2c843b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36572/0 (socket says 172.21.15.52:36572) 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f25227fc700 1 -- 172.21.15.52:0/1540464458 learned_addr learned my addr 172.21.15.52:0/1540464458 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f2522ffd700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c8419f20 0x55e2c842c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.661+0000 7f25237fe700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841ff60 0x55e2c843b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f25227fc700 1 -- 172.21.15.52:0/1540464458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841ff60 msgr2=0x55e2c843b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f25227fc700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841ff60 0x55e2c843b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f25227fc700 1 -- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c8419f20 msgr2=0x55e2c842c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f25227fc700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c8419f20 0x55e2c842c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f25227fc700 1 -- 172.21.15.52:0/1540464458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2c84a1a30 con 0x55e2c841f640 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f25227fc700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 0x55e2c843b1a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f252406a9d0 tx=0x7f250c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d6f631c420d5b0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f250a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f250a7fc700 1 -- 172.21.15.52:0/1540464458 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c0293b0 con 0x55e2c841f640 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f250a7fc700 1 -- 172.21.15.52:0/1540464458 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f250c029550 con 0x55e2c841f640 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25140a4550 con 0x55e2c841f640 2024-06-02T23:37:58.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f250a7fc700 1 -- 172.21.15.52:0/1540464458 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f250c039050 con 0x55e2c841f640 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f2533f3db80 10 client.22095 fetch_fsmap learned FSMap version 48 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e2c84a2bb0 con 0x55e2c841f640 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f250a7fc700 1 -- 172.21.15.52:0/1540464458 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f25140a4550 con 0x55e2c841f640 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.662+0000 7f250a7fc700 1 -- 172.21.15.52:0/1540464458 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f250c0118d0 con 0x55e2c841f640 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 10 client.22095 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 10 client.22095 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.e 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 20 client.22095 populate_metadata read hostname 'smithi052' 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c84a3360 0x55e2c84a5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 4 client.22095 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 --> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2c8373290 con 0x55e2c84a3360 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2522ffd700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c84a3360 0x55e2c84a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f250a7fc700 10 client.22095 ms_handle_connect on v2:172.21.15.123:6834/3092837185 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2522ffd700 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c84a3360 0x55e2c84a5780 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4ebeedd6f6e71c85 server_cookie=362034a3a03f9c94 in_seq=0 out_seq=0 2024-06-02T23:37:58.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f250a7fc700 1 -- 172.21.15.52:0/1540464458 <== mds.4 v2:172.21.15.123:6834/3092837185 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2c8373290 con 0x55e2c84a3360 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f250a7fc700 10 client.22095 handle_command_reply: tid=0 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 1 client.22095 shutdown 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 msgr2=0x55e2c843b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.663+0000 7f2533f3db80 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 0x55e2c843b1a0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f252406a9d0 tx=0x7f250c004750 comp rx=0 tx=0).stop 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 shutdown_connections 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e2c841ff60 0x55e2c843b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:6834/3092837185,v1:172.21.15.123:6835/3092837185] conn(0x55e2c84a3360 0x55e2c84a5780 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e2c8419f20 0x55e2c842c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 --2- 172.21.15.52:0/1540464458 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e2c841f640 0x55e2c843b1a0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f250a7fc700 0 client.22095 ms_handle_reset on v2:172.21.15.123:6834/3092837185 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 >> 172.21.15.52:0/1540464458 conn(0x55e2c8344cc0 msgr2=0x55e2c8420a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 shutdown_connections 2024-06-02T23:37:58.669 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 1 -- 172.21.15.52:0/1540464458 wait complete. 2024-06-02T23:37:58.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f2533f3db80 20 client.22095 trim_cache size 0 max 16384 2024-06-02T23:37:58.670 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:37:58.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25100a4e30 msgr2=0x7f25100a72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.670 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f25296ba700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25100a4e30 0x7f25100a72f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f2524197f00 tx=0x7f250c00d4b0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 msgr2=0x7f252419c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.664+0000 7f25296ba700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 0x7f252419c670 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f251400d950 tx=0x7f251400dd10 comp rx=0 tx=0).stop 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 shutdown_connections 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f25100a4e30 0x7f25100a72f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2524106720 0x7f2524196790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f25241019d0 0x7f252419c670 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 --2- 172.21.15.52:0/4223674825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2524102380 0x7f252419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 >> 172.21.15.52:0/4223674825 conn(0x7f2524078340 msgr2=0x7f25240ff980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 shutdown_connections 2024-06-02T23:37:58.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:37:58.665+0000 7f25296ba700 1 -- 172.21.15.52:0/4223674825 wait complete. 2024-06-02T23:37:59.391 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:3 damage ls 2024-06-02T23:37:59.644 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:37:59.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:37:59 smithi052 ceph-mon[32340]: pgmap v1214: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:37:59.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:37:59 smithi123 ceph-mon[35977]: pgmap v1214: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:37:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:37:59 smithi174 ceph-mon[35994]: pgmap v1214: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:01.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 -- 172.21.15.52:0/3134484127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 msgr2=0x7ff9d8107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.298 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/3134484127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d8107950 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009990 tx=0x7ff9d4009ca0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 -- 172.21.15.52:0/3134484127 shutdown_connections 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/3134484127 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d8101810 0x7ff9d810f280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/3134484127 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d8107950 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/3134484127 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9d8100e50 0x7ff9d81012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 -- 172.21.15.52:0/3134484127 >> 172.21.15.52:0/3134484127 conn(0x7ff9d80766b0 msgr2=0x7ff9d8076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 -- 172.21.15.52:0/3134484127 shutdown_connections 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.296+0000 7ff9e05c1700 1 -- 172.21.15.52:0/3134484127 wait complete. 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 Processor -- start 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 -- start start 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d8100e50 0x7ff9d819e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9d8101810 0x7ff9d819ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.299 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8112680 con 0x7ff9d8107570 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8112500 con 0x7ff9d8100e50 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9e05c1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d8112800 con 0x7ff9d8101810 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9deb5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9de35d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d8100e50 0x7ff9d819e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9deb5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d81989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36586/0 (socket says 172.21.15.52:36586) 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2926961411 learned_addr learned my addr 172.21.15.52:0/2926961411 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.297+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9d8101810 0x7ff9d819ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2926961411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9d8101810 msgr2=0x7ff9d819ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9d8101810 0x7ff9d819ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2926961411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d8100e50 msgr2=0x7ff9d819e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d8100e50 0x7ff9d819e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2926961411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d8199100 con 0x7ff9d8107570 2024-06-02T23:38:01.300 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d81989a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0007ee0 tx=0x7ff9d000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9cf7fe700 1 -- 172.21.15.52:0/2926961411 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d000dbd0 con 0x7ff9d8107570 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9cf7fe700 1 -- 172.21.15.52:0/2926961411 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d000dd70 con 0x7ff9d8107570 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d81993f0 con 0x7ff9d8107570 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9cf7fe700 1 -- 172.21.15.52:0/2926961411 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0005990 con 0x7ff9d8107570 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.298+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d81a56b0 con 0x7ff9d8107570 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.300+0000 7ff9cf7fe700 1 -- 172.21.15.52:0/2926961411 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff9d001a020 con 0x7ff9d8107570 2024-06-02T23:38:01.301 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.301+0000 7ff9cf7fe700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff9c40a2c20 0x7ff9c40a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.301+0000 7ff9de35d700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff9c40a2c20 0x7ff9c40a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.302 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.301+0000 7ff9cf7fe700 1 -- 172.21.15.52:0/2926961411 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff9d00ce260 con 0x7ff9d8107570 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.301+0000 7ff9de35d700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff9c40a2c20 0x7ff9c40a50e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006010 tx=0x7ff9d4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 Processor -- start 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 -- start start 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba278a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278f60 0x55e9ba2729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba273340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba1c4810 con 0x55e9ba278640 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba16c720 con 0x55e9ba272f20 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba1c07d0 con 0x55e9ba278f60 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9de35d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba278a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.304 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9de35d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba278a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36592/0 (socket says 172.21.15.52:36592) 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9de35d700 1 -- 172.21.15.52:0/1100624391 learned_addr learned my addr 172.21.15.52:0/1100624391 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.302+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba273340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278f60 0x55e9ba2729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9de35d700 1 -- 172.21.15.52:0/1100624391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278f60 msgr2=0x55e9ba2729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9de35d700 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278f60 0x55e9ba2729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9de35d700 1 -- 172.21.15.52:0/1100624391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 msgr2=0x55e9ba273340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9de35d700 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba273340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9de35d700 1 -- 172.21.15.52:0/1100624391 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ba162f50 con 0x55e9ba278640 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba273340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9de35d700 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba278a20 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8077c30 tx=0x7ff9d4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4009aa6f7506f08 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9ccff9700 1 -- 172.21.15.52:0/1100624391 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d400be00 con 0x55e9ba278640 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9ccff9700 1 -- 172.21.15.52:0/1100624391 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d400fc70 con 0x55e9ba278640 2024-06-02T23:38:01.305 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.303+0000 7ff9ccff9700 1 -- 172.21.15.52:0/1100624391 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d4024cd0 con 0x55e9ba278640 2024-06-02T23:38:01.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 -- 172.21.15.52:0/1100624391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 msgr2=0x55e9ba278a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba278a20 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7ff9d8077c30 tx=0x7ff9d4020000 comp rx=0 tx=0).stop 2024-06-02T23:38:01.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 -- 172.21.15.52:0/1100624391 shutdown_connections 2024-06-02T23:38:01.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba273340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba278a20 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.309 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/1100624391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278f60 0x55e9ba2729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 -- 172.21.15.52:0/1100624391 >> 172.21.15.52:0/1100624391 conn(0x55e9ba19dcc0 msgr2=0x55e9ba26d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 -- 172.21.15.52:0/1100624391 shutdown_connections 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 -- 172.21.15.52:0/1100624391 wait complete. 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.304+0000 7ff9eae44b80 1 -- start start 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 Processor -- start 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba27ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278640 0x55e9ba27bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba27c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba27f910 con 0x55e9ba278f60 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba164bf0 con 0x55e9ba278640 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e9ba27fa90 con 0x55e9ba272f20 2024-06-02T23:38:01.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9eae44b80 10 client.0 _ll_register_callbacks cb 0x55e9ba15e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba27c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba27c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36596/0 (socket says 172.21.15.52:36596) 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 -- 172.21.15.52:0/3626571448 learned_addr learned my addr 172.21.15.52:0/3626571448 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9de35d700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba27ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278640 0x55e9ba27bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 -- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 msgr2=0x55e9ba27ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba27ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 -- 172.21.15.52:0/3626571448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278640 msgr2=0x55e9ba27bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278640 0x55e9ba27bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9deb5e700 1 -- 172.21.15.52:0/3626571448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ba1138d0 con 0x55e9ba278f60 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.305+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278640 0x55e9ba27bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba27c500 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000d7b0 tx=0x7ff9d009abc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b63b28fef32b1f7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:01.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/3626571448 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d0004ad0 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/3626571448 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff9d009ad50 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9d400fc70 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/3626571448 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff9d009af50 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 10 client.22107 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e9ba28a270 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/3626571448 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9d400fc70 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/3626571448 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7ff9d00b91b0 con 0x55e9ba278f60 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 10 client.22107 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 10 client.22107 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 20 client.22107 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba28ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 4 client.22107 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:38:01.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.306+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e9ba1d1d10 con 0x55e9ba28c900 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.307+0000 7ff9de35d700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba28ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.307+0000 7ff9de35d700 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba28ecf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=56f8ce8c759ef9ec server_cookie=7c4f2f8df43b30cd in_seq=0 out_seq=0 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.307+0000 7ff9c1ffb700 10 client.22107 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/3626571448 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e9ba1d1d10 con 0x55e9ba28c900 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9c1ffb700 10 client.22107 handle_command_reply: tid=0 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 client.22107 shutdown 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 msgr2=0x55e9ba27c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba27c500 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7ff9d000d7b0 tx=0x7ff9d009abc0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 shutdown_connections 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba27c500 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba28ecf0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9c1ffb700 0 client.22107 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba27ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/3626571448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278640 0x55e9ba27bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 >> 172.21.15.52:0/3626571448 conn(0x55e9ba19dcc0 msgr2=0x55e9ba2719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 shutdown_connections 2024-06-02T23:38:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 1 -- 172.21.15.52:0/3626571448 wait complete. 2024-06-02T23:38:01.314 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.308+0000 7ff9eae44b80 20 client.22107 trim_cache size 0 max 16384 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.323+0000 7ff9eae44b80 1 Processor -- start 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.323+0000 7ff9eae44b80 1 -- start start 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba2729e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba279bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278f60 0x55e9ba267510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81acaa0 con 0x55e9ba278640 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ae700 con 0x55e9ba278f60 2024-06-02T23:38:01.325 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ad230 con 0x55e9ba272f20 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba279bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba279bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36602/0 (socket says 172.21.15.52:36602) 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 -- 172.21.15.52:0/4071019670 learned_addr learned my addr 172.21.15.52:0/4071019670 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278f60 0x55e9ba267510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9de35d700 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba2729e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 -- 172.21.15.52:0/4071019670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 msgr2=0x55e9ba2729e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba2729e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 -- 172.21.15.52:0/4071019670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278f60 msgr2=0x55e9ba267510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278f60 0x55e9ba267510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 -- 172.21.15.52:0/4071019670 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e9ba26f2c0 con 0x55e9ba278640 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9de35d700 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba2729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba279bd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800bd20 tx=0x7ff9c8012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b06b3a5edde03e9d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:01.326 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9c3fff700 1 -- 172.21.15.52:0/4071019670 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8020070 con 0x55e9ba278640 2024-06-02T23:38:01.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.324+0000 7ff9c3fff700 1 -- 172.21.15.52:0/4071019670 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9c801b8c0 con 0x55e9ba278640 2024-06-02T23:38:01.327 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9c3fff700 1 -- 172.21.15.52:0/4071019670 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c80107b0 con 0x55e9ba278640 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 -- 172.21.15.52:0/4071019670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 msgr2=0x55e9ba279bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba279bd0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7ff9c800bd20 tx=0x7ff9c8012730 comp rx=0 tx=0).stop 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 -- 172.21.15.52:0/4071019670 shutdown_connections 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba278f60 0x55e9ba267510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba272f20 0x55e9ba2729e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/4071019670 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278640 0x55e9ba279bd0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 -- 172.21.15.52:0/4071019670 >> 172.21.15.52:0/4071019670 conn(0x55e9ba19dcc0 msgr2=0x55e9ba266e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 -- 172.21.15.52:0/4071019670 shutdown_connections 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 -- 172.21.15.52:0/4071019670 wait complete. 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.325+0000 7ff9eae44b80 1 -- start start 2024-06-02T23:38:01.331 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 Processor -- start 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba285320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278640 0x55e9ba2806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba280be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ae030 con 0x55e9ba278f60 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81ad930 con 0x55e9ba272f20 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d81aaa60 con 0x55e9ba278640 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9eae44b80 10 client.0 _ll_register_callbacks cb 0x55e9ba15e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba280be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba280be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:36608/0 (socket says 172.21.15.52:36608) 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2364399482 learned_addr learned my addr 172.21.15.52:0/2364399482 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9de35d700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba285320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.332 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278640 0x55e9ba2806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278640 msgr2=0x55e9ba2806a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278640 0x55e9ba2806a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2364399482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 msgr2=0x55e9ba285320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba285320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9deb5e700 1 -- 172.21.15.52:0/2364399482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e9ba281290 con 0x55e9ba278f60 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9ddb5c700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278640 0x55e9ba2806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.326+0000 7ff9de35d700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba285320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:38:01.333 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9deb5e700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba280be0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff9d009f060 tx=0x7ff9d009eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d1031c85052ff0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/2364399482 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d00b91b0 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/2364399482 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff9d00b9350 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c801b8c0 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/2364399482 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7ff9d00a32d0 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 10 client.22113 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e9ba2f7390 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/2364399482 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff9c801b8c0 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/2364399482 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7ff9d0004fb0 con 0x55e9ba278f60 2024-06-02T23:38:01.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 10 client.22113 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 10 client.22113 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.c 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 20 client.22113 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba2f9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 4 client.22113 mds_command: new command op to 24436 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.327+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 --> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e9ba1cc290 con 0x55e9ba28c900 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9de35d700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba2f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9de35d700 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba2f9950 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=52a8cdd1c61f5017 server_cookie=614dac39df96e12c in_seq=0 out_seq=0 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9c1ffb700 10 client.22113 ms_handle_connect on v2:172.21.15.174:6836/457473811 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9c1ffb700 1 -- 172.21.15.52:0/2364399482 <== mds.3 v2:172.21.15.174:6836/457473811 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e9ba1cc290 con 0x55e9ba28c900 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9c1ffb700 10 client.22113 handle_command_reply: tid=0 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 client.22113 shutdown 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 msgr2=0x55e9ba280be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba280be0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7ff9d009f060 tx=0x7ff9d009eea0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 shutdown_connections 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e9ba278f60 0x55e9ba280be0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e9ba272f20 0x55e9ba285320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:6836/457473811,v1:172.21.15.174:6837/457473811] conn(0x55e9ba28c900 0x55e9ba2f9950 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9c1ffb700 0 client.22113 ms_handle_reset on v2:172.21.15.174:6836/457473811 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.328+0000 7ff9eae44b80 1 --2- 172.21.15.52:0/2364399482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e9ba278640 0x55e9ba2806a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 >> 172.21.15.52:0/2364399482 conn(0x55e9ba19dcc0 msgr2=0x55e9ba277110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 shutdown_connections 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9eae44b80 1 -- 172.21.15.52:0/2364399482 wait complete. 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9eae44b80 20 client.22113 trim_cache size 0 max 16384 2024-06-02T23:38:01.336 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff9c40a2c20 msgr2=0x7ff9c40a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff9c40a2c20 0x7ff9c40a50e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4006010 tx=0x7ff9d4005960 comp rx=0 tx=0).stop 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 msgr2=0x7ff9d81989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d81989a0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0007ee0 tx=0x7ff9d000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 shutdown_connections 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff9c40a2c20 0x7ff9c40a50e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff9d8107570 0x7ff9d81989a0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d8100e50 0x7ff9d819e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 --2- 172.21.15.52:0/2926961411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9d8101810 0x7ff9d819ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 >> 172.21.15.52:0/2926961411 conn(0x7ff9d80766b0 msgr2=0x7ff9d80ffa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 shutdown_connections 2024-06-02T23:38:01.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:01.329+0000 7ff9e05c1700 1 -- 172.21.15.52:0/2926961411 wait complete. 2024-06-02T23:38:01.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:01 smithi052 ceph-mon[32340]: pgmap v1215: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:01.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:01 smithi123 ceph-mon[35977]: pgmap v1215: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:01.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:01 smithi174 ceph-mon[35994]: pgmap v1215: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:02.194 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-06-02T23:38:02.450 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:03.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:03 smithi052 ceph-mon[32340]: pgmap v1216: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:03.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:03 smithi123 ceph-mon[35977]: pgmap v1216: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:03 smithi174 ceph-mon[35994]: pgmap v1216: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 -- 172.21.15.52:0/4171671182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101fd0 msgr2=0x7fd390105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 --2- 172.21.15.52:0/4171671182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101fd0 0x7fd390105fd0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fd3800099c0 tx=0x7fd380009cd0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 -- 172.21.15.52:0/4171671182 shutdown_connections 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 --2- 172.21.15.52:0/4171671182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 0x7fd39010e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 --2- 172.21.15.52:0/4171671182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd390101620 0x7fd390101a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 --2- 172.21.15.52:0/4171671182 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101fd0 0x7fd390105fd0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 -- 172.21.15.52:0/4171671182 >> 172.21.15.52:0/4171671182 conn(0x7fd390078340 msgr2=0x7fd390078750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 -- 172.21.15.52:0/4171671182 shutdown_connections 2024-06-02T23:38:04.308 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.307+0000 7fd397d39700 1 -- 172.21.15.52:0/4171671182 wait complete. 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 Processor -- start 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 -- start start 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 0x7fd39019e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd390101fd0 0x7fd39019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 0x7fd3901988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd390111520 con 0x7fd390101620 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901113a0 con 0x7fd3901066b0 2024-06-02T23:38:04.310 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd397d39700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901116a0 con 0x7fd390101fd0 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd395ad5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 0x7fd39019e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd395ad5700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 0x7fd39019e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51972/0 (socket says 172.21.15.52:51972) 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd395ad5700 1 -- 172.21.15.52:0/1077656251 learned_addr learned my addr 172.21.15.52:0/1077656251 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.308+0000 7fd3952d4700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd390101fd0 0x7fd39019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd3962d6700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 0x7fd3901988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd395ad5700 1 -- 172.21.15.52:0/1077656251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd390101fd0 msgr2=0x7fd39019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd395ad5700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd390101fd0 0x7fd39019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd395ad5700 1 -- 172.21.15.52:0/1077656251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 msgr2=0x7fd3901988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd395ad5700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 0x7fd3901988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd395ad5700 1 -- 172.21.15.52:0/1077656251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd390199020 con 0x7fd390101620 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd3962d6700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 0x7fd3901988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd395ad5700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 0x7fd39019e6b0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00c9b0 tx=0x7fd38c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:04.311 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd386ffd700 1 -- 172.21.15.52:0/1077656251 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c007b00 con 0x7fd390101620 2024-06-02T23:38:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd390199310 con 0x7fd390101620 2024-06-02T23:38:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd386ffd700 1 -- 172.21.15.52:0/1077656251 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd38c007ca0 con 0x7fd390101620 2024-06-02T23:38:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd386ffd700 1 -- 172.21.15.52:0/1077656251 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38c018660 con 0x7fd390101620 2024-06-02T23:38:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.309+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd390102fa0 con 0x7fd390101620 2024-06-02T23:38:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.312+0000 7fd386ffd700 1 -- 172.21.15.52:0/1077656251 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd38c00f450 con 0x7fd390101620 2024-06-02T23:38:04.312 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.312+0000 7fd386ffd700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2c20 0x7fd37c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3952d4700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2c20 0x7fd37c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.313 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd386ffd700 1 -- 172.21.15.52:0/1077656251 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd38c0cef80 con 0x7fd390101620 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 Processor -- start 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3952d4700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2c20 0x7fd37c0a50e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd380009990 tx=0x7fd380005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 -- start start 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560087892640 0x560087892a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x56008788c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56008788cf20 0x56008788d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x5600877de810 con 0x560087892f60 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560087786720 con 0x560087892640 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.313+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5600877da7d0 con 0x56008788cf20 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x56008788c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x56008788c9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51988/0 (socket says 172.21.15.52:51988) 2024-06-02T23:38:04.315 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 -- 172.21.15.52:0/906051228 learned_addr learned my addr 172.21.15.52:0/906051228 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd395ad5700 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560087892640 0x560087892a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3962d6700 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56008788cf20 0x56008788d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 -- 172.21.15.52:0/906051228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56008788cf20 msgr2=0x56008788d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56008788cf20 0x56008788d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 -- 172.21.15.52:0/906051228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560087892640 msgr2=0x560087892a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560087892640 0x560087892a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 -- 172.21.15.52:0/906051228 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56008777cf50 con 0x560087892f60 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd3952d4700 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x56008788c9e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd380015000 tx=0x7fd380020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25721f64390de89d server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd37bfff700 1 -- 172.21.15.52:0/906051228 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38000bca0 con 0x560087892f60 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd37bfff700 1 -- 172.21.15.52:0/906051228 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd38000be40 con 0x560087892f60 2024-06-02T23:38:04.316 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.314+0000 7fd37bfff700 1 -- 172.21.15.52:0/906051228 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380023c30 con 0x560087892f60 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/906051228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 msgr2=0x56008788c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x56008788c9e0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fd380015000 tx=0x7fd380020000 comp rx=0 tx=0).stop 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/906051228 shutdown_connections 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56008788cf20 0x56008788d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560087892640 0x560087892a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/906051228 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x56008788c9e0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/906051228 >> 172.21.15.52:0/906051228 conn(0x5600877b7cc0 msgr2=0x560087887850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/906051228 shutdown_connections 2024-06-02T23:38:04.319 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.315+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/906051228 wait complete. 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 -- start start 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 Processor -- start 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x560087898cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x560087895fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x560087896500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x560087899910 con 0x560087892f60 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56008777ebf0 con 0x56008788cf20 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560087899a90 con 0x560087892640 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3a25bcb80 10 client.0 _ll_register_callbacks cb 0x560087778920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3962d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x560087896500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3962d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x560087896500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51990/0 (socket says 172.21.15.52:51990) 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.316+0000 7fd3962d6700 1 -- 172.21.15.52:0/3225718637 learned_addr learned my addr 172.21.15.52:0/3225718637 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:04.320 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3952d4700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x560087895fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd395ad5700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x560087898cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3962d6700 1 -- 172.21.15.52:0/3225718637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 msgr2=0x560087895fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3962d6700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x560087895fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3962d6700 1 -- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 msgr2=0x560087898cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3962d6700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x560087898cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3962d6700 1 -- 172.21.15.52:0/3225718637 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56008772d8d0 con 0x560087892f60 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3962d6700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x560087896500 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fd388010b00 tx=0x7fd388010e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275d07757bb5f424 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3797fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3797fa700 1 -- 172.21.15.52:0/3225718637 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38801a020 con 0x560087892f60 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd38000be40 con 0x560087892f60 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3797fa700 1 -- 172.21.15.52:0/3225718637 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd38800e850 con 0x560087892f60 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3797fa700 1 -- 172.21.15.52:0/3225718637 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd38800ea50 con 0x560087892f60 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3a25bcb80 10 client.22125 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:04.321 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.317+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x5600878a42c0 con 0x560087892f60 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3797fa700 1 -- 172.21.15.52:0/3225718637 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd38000be40 con 0x560087892f60 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3797fa700 1 -- 172.21.15.52:0/3225718637 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7fd388016410 con 0x560087892f60 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3a25bcb80 10 client.22125 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3a25bcb80 10 client.22125 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3a25bcb80 20 client.22125 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600878a68a0 0x5600878a8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3a25bcb80 4 client.22125 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600877ebd10 con 0x5600878a68a0 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.318+0000 7fd395ad5700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600878a68a0 0x5600878a8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.319+0000 7fd395ad5700 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600878a68a0 0x5600878a8cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=582cd096ebb2f1b1 server_cookie=52bf8a1aef0b1848 in_seq=0 out_seq=0 2024-06-02T23:38:04.322 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.319+0000 7fd3797fa700 10 client.22125 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:38:04.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.319+0000 7fd3797fa700 1 -- 172.21.15.52:0/3225718637 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x5600877ebd10 con 0x5600878a68a0 2024-06-02T23:38:04.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.319+0000 7fd3797fa700 10 client.22125 handle_command_reply: tid=0 2024-06-02T23:38:04.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 client.22125 shutdown 2024-06-02T23:38:04.323 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 msgr2=0x560087896500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x560087896500 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fd388010b00 tx=0x7fd388010e10 comp rx=0 tx=0).stop 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 shutdown_connections 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600878a68a0 0x5600878a8cf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x560087896500 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x560087898cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3225718637 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x560087895fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3797fa700 0 client.22125 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 >> 172.21.15.52:0/3225718637 conn(0x5600877b7cc0 msgr2=0x5600878875f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 shutdown_connections 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3225718637 wait complete. 2024-06-02T23:38:04.324 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.320+0000 7fd3a25bcb80 20 client.22125 trim_cache size 0 max 16384 2024-06-02T23:38:04.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.335+0000 7fd3a25bcb80 1 Processor -- start 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.335+0000 7fd3a25bcb80 1 -- start start 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.335+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008788c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.335+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 0x560087895f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.335+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892f60 0x5600878964c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901aea80 con 0x560087892640 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b06e0 con 0x56008788cf20 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901af210 con 0x560087892f60 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 0x560087895f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 0x560087895f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:51998/0 (socket says 172.21.15.52:51998) 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 -- 172.21.15.52:0/3383930722 learned_addr learned my addr 172.21.15.52:0/3383930722 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd395ad5700 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008788c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 -- 172.21.15.52:0/3383930722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892f60 msgr2=0x5600878964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892f60 0x5600878964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 -- 172.21.15.52:0/3383930722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 msgr2=0x56008788c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008788c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 -- 172.21.15.52:0/3383930722 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560087892140 con 0x560087892640 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd395ad5700 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008788c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd3952d4700 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 0x560087895f80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd38000bb70 tx=0x7fd38000bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f09401a947e410 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd37b7fe700 1 -- 172.21.15.52:0/3383930722 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380023460 con 0x560087892640 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd37b7fe700 1 -- 172.21.15.52:0/3383930722 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd380023600 con 0x560087892640 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.336+0000 7fd37b7fe700 1 -- 172.21.15.52:0/3383930722 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38000f460 con 0x560087892640 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3383930722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 msgr2=0x560087895f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 0x560087895f80 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fd38000bb70 tx=0x7fd38000bee0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3383930722 shutdown_connections 2024-06-02T23:38:04.339 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892f60 0x5600878964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008788c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/3383930722 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892640 0x560087895f80 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3383930722 >> 172.21.15.52:0/3383930722 conn(0x5600877b7cc0 msgr2=0x5600878948f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3383930722 shutdown_connections 2024-06-02T23:38:04.340 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/3383930722 wait complete. 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.337+0000 7fd3a25bcb80 1 -- start start 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 Processor -- start 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008789f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x56008789f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x5600878adc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901b0010 con 0x560087892f60 2024-06-02T23:38:04.343 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901af910 con 0x56008788cf20 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3901aca20 con 0x560087892640 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3a25bcb80 10 client.0 _ll_register_callbacks cb 0x560087778920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3962d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x5600878adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3962d6700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x5600878adc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52002/0 (socket says 172.21.15.52:52002) 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3962d6700 1 -- 172.21.15.52:0/891672893 learned_addr learned my addr 172.21.15.52:0/891672893 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd3952d4700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x56008789f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.338+0000 7fd395ad5700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008789f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3962d6700 1 -- 172.21.15.52:0/891672893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 msgr2=0x56008789f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3962d6700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x56008789f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3962d6700 1 -- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 msgr2=0x56008789f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3962d6700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008789f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3962d6700 1 -- 172.21.15.52:0/891672893 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600878ae470 con 0x560087892f60 2024-06-02T23:38:04.344 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3962d6700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x5600878adc30 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd388010e50 tx=0x7fd388010630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4e0614c5ac9be9 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3797fa700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3797fa700 1 -- 172.21.15.52:0/891672893 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd388030060 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd380023600 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3797fa700 1 -- 172.21.15.52:0/891672893 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd3880077f0 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3797fa700 1 -- 172.21.15.52:0/891672893 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fd38801a040 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3a25bcb80 10 client.22131 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x560087911150 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.339+0000 7fd3797fa700 1 -- 172.21.15.52:0/891672893 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd380023600 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3797fa700 1 -- 172.21.15.52:0/891672893 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7fd3880050e0 con 0x560087892f60 2024-06-02T23:38:04.345 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3a25bcb80 10 client.22131 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3a25bcb80 10 client.22131 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3a25bcb80 20 client.22131 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600879118c0 0x560087913ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3a25bcb80 4 client.22131 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 --> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5600877e6290 con 0x5600879118c0 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd395ad5700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600879118c0 0x560087913ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd395ad5700 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600879118c0 0x560087913ce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7f89c7f5df287a2d server_cookie=ae68f3bb1f11e525 in_seq=0 out_seq=0 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.340+0000 7fd3797fa700 10 client.22131 ms_handle_connect on v2:172.21.15.123:6836/785283486 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3797fa700 1 -- 172.21.15.52:0/891672893 <== mds.2 v2:172.21.15.123:6836/785283486 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5600877e6290 con 0x5600879118c0 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3797fa700 10 client.22131 handle_command_reply: tid=0 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 client.22131 shutdown 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 msgr2=0x5600878adc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x5600878adc30 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7fd388010e50 tx=0x7fd388010630 comp rx=0 tx=0).stop 2024-06-02T23:38:04.346 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 shutdown_connections 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:6836/785283486,v1:172.21.15.123:6837/785283486] conn(0x5600879118c0 0x560087913ce0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x560087892f60 0x5600878adc30 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3797fa700 0 client.22131 ms_handle_reset on v2:172.21.15.123:6836/785283486 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56008788cf20 0x56008789f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 --2- 172.21.15.52:0/891672893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560087892640 0x56008789f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 >> 172.21.15.52:0/891672893 conn(0x5600877b7cc0 msgr2=0x560087880e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 shutdown_connections 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 1 -- 172.21.15.52:0/891672893 wait complete. 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.341+0000 7fd3a25bcb80 20 client.22131 trim_cache size 0 max 16384 2024-06-02T23:38:04.347 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2c20 msgr2=0x7fd37c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2c20 0x7fd37c0a50e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fd380009990 tx=0x7fd380005fb0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 msgr2=0x7fd39019e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 0x7fd39019e6b0 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7fd38c00c9b0 tx=0x7fd38c00ccc0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 shutdown_connections 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd37c0a2c20 0x7fd37c0a50e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.348 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3901066b0 0x7fd3901988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd390101620 0x7fd39019e6b0 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 --2- 172.21.15.52:0/1077656251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd390101fd0 0x7fd39019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 >> 172.21.15.52:0/1077656251 conn(0x7fd390078340 msgr2=0x7fd39006ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 shutdown_connections 2024-06-02T23:38:04.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:04.342+0000 7fd397d39700 1 -- 172.21.15.52:0/1077656251 wait complete. 2024-06-02T23:38:05.073 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-06-02T23:38:05.322 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:05.723 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:05 smithi052 ceph-mon[32340]: pgmap v1217: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:05.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:05 smithi123 ceph-mon[35977]: pgmap v1217: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:05 smithi174 ceph-mon[35994]: pgmap v1217: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 -- 172.21.15.52:0/3205676218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101090 msgr2=0x7f5fd0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 --2- 172.21.15.52:0/3205676218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101090 0x7f5fd0101470 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00c210 tx=0x7f5fcc00c520 comp rx=0 tx=0).stop 2024-06-02T23:38:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 -- 172.21.15.52:0/3205676218 shutdown_connections 2024-06-02T23:38:07.076 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 --2- 172.21.15.52:0/3205676218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0105f30 0x7f5fd010da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 --2- 172.21.15.52:0/3205676218 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101090 0x7f5fd0101470 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 --2- 172.21.15.52:0/3205676218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fd0101a40 0x7f5fd0105800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.075+0000 7f5fd7910700 1 -- 172.21.15.52:0/3205676218 >> 172.21.15.52:0/3205676218 conn(0x7f5fd00766b0 msgr2=0x7f5fd0076ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 -- 172.21.15.52:0/3205676218 shutdown_connections 2024-06-02T23:38:07.077 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 -- 172.21.15.52:0/3205676218 wait complete. 2024-06-02T23:38:07.078 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 Processor -- start 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 -- start start 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fd0101090 0x7f5fd019e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 0x7f5fd019edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0105f30 0x7f5fd0198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01104e0 con 0x7f5fd0101a40 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0110360 con 0x7f5fd0105f30 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.076+0000 7f5fd7910700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0110660 con 0x7f5fd0101090 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 0x7f5fd019edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 0x7f5fd019edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52012/0 (socket says 172.21.15.52:52012) 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2621574481 learned_addr learned my addr 172.21.15.52:0/2621574481 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fd0101090 0x7f5fd019e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.079 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0105f30 0x7f5fd0198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2621574481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fd0101090 msgr2=0x7f5fd019e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fd0101090 0x7f5fd019e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2621574481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0105f30 msgr2=0x7f5fd0198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0105f30 0x7f5fd0198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2621574481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd0199190 con 0x7f5fd0101a40 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 0x7f5fd019edb0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000d980 tx=0x7f5fc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fc67fc700 1 -- 172.21.15.52:0/2621574481 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0017070 con 0x7f5fd0101a40 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd0199480 con 0x7f5fd0101a40 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fc67fc700 1 -- 172.21.15.52:0/2621574481 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fc0005020 con 0x7f5fd0101a40 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fc67fc700 1 -- 172.21.15.52:0/2621574481 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc000fa60 con 0x7f5fd0101a40 2024-06-02T23:38:07.080 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.077+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a5740 con 0x7f5fd0101a40 2024-06-02T23:38:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.080+0000 7f5fc67fc700 1 -- 172.21.15.52:0/2621574481 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5fc00051d0 con 0x7f5fd0101a40 2024-06-02T23:38:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.080+0000 7f5fc67fc700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fbc0a2c20 0x7f5fbc0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.080+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fbc0a2c20 0x7f5fbc0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.081 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.081+0000 7f5fc67fc700 1 -- 172.21.15.52:0/2621574481 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5fc00ce7b0 con 0x7f5fd0101a40 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.081+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fbc0a2c20 0x7f5fbc0a50e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00af70 tx=0x7f5fcc00c870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.081+0000 7f5fe2193b80 1 Processor -- start 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.081+0000 7f5fe2193b80 1 -- start start 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.081+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 0x55e5fed72a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72f60 0x55e5fed6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed6cf20 0x55e5fed6d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e5fecbe810 con 0x55e5fed72640 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5fec66720 con 0x55e5fed6cf20 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5fecba7d0 con 0x55e5fed72f60 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 0x55e5fed72a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.083 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 0x55e5fed72a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52026/0 (socket says 172.21.15.52:52026) 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 -- 172.21.15.52:0/1550919217 learned_addr learned my addr 172.21.15.52:0/1550919217 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed6cf20 0x55e5fed6d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72f60 0x55e5fed6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 -- 172.21.15.52:0/1550919217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72f60 msgr2=0x55e5fed6c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72f60 0x55e5fed6c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 -- 172.21.15.52:0/1550919217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed6cf20 msgr2=0x55e5fed6d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed6cf20 0x55e5fed6d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 -- 172.21.15.52:0/1550919217 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5fec5cf50 con 0x55e5fed72640 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 0x55e5fed72a20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc003e50 tx=0x7f5fcc01d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed3a190f4ddbe70 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fbb7fe700 1 -- 172.21.15.52:0/1550919217 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc02f070 con 0x55e5fed72640 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fbb7fe700 1 -- 172.21.15.52:0/1550919217 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fcc023030 con 0x55e5fed72640 2024-06-02T23:38:07.084 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.082+0000 7f5fbb7fe700 1 -- 172.21.15.52:0/1550919217 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc007ac0 con 0x55e5fed72640 2024-06-02T23:38:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 -- 172.21.15.52:0/1550919217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 msgr2=0x55e5fed72a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.087 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 0x55e5fed72a20 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc003e50 tx=0x7f5fcc01d000 comp rx=0 tx=0).stop 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 -- 172.21.15.52:0/1550919217 shutdown_connections 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed6cf20 0x55e5fed6d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72640 0x55e5fed72a20 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/1550919217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72f60 0x55e5fed6c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 -- 172.21.15.52:0/1550919217 >> 172.21.15.52:0/1550919217 conn(0x55e5fec97cc0 msgr2=0x55e5fed67850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 -- 172.21.15.52:0/1550919217 shutdown_connections 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.083+0000 7f5fe2193b80 1 -- 172.21.15.52:0/1550919217 wait complete. 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 -- start start 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 Processor -- start 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed78cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed75fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed76510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.088 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55e5fed79970 con 0x55e5fed72f60 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e5fec5ebf0 con 0x55e5fed72640 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e5fed79af0 con 0x55e5fed6cf20 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.084+0000 7f5fe2193b80 10 client.0 _ll_register_callbacks cb 0x55e5fec58920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd5ead700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed76510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed75fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd5ead700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed76510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52038/0 (socket says 172.21.15.52:52038) 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd5ead700 1 -- 172.21.15.52:0/2418327986 learned_addr learned my addr 172.21.15.52:0/2418327986 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed78cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 msgr2=0x55e5fed78cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed78cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2418327986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 msgr2=0x55e5fed76510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed76510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.089 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 -- 172.21.15.52:0/2418327986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5fec0d8d0 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed76510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed75fd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00cc650 tx=0x7f5fc009b180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1b7a686f9508f38 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/2418327986 <== mon.2 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0020050 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/2418327986 <== mon.2 v2:172.21.15.123:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5fc0005580 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.085+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fcc023030 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/2418327986 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5fc009f0d0 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 10 client.31325 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e5fed842f0 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/2418327986 <== mon.2 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fcc023030 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/2418327986 <== mon.2 v2:172.21.15.123:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f5fc0095bb0 con 0x55e5fed72640 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 10 client.31325 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:07.090 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 10 client.31325 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 20 client.31325 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fed86880 0x55e5fed88c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 4 client.31325 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5feccbd10 con 0x55e5fed86880 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.086+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fed86880 0x55e5fed88c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.087+0000 7f5fb8ff9700 10 client.31325 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:38:07.091 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.087+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fed86880 0x55e5fed88c70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8e4748e49aea5d09 server_cookie=b69b2d43b107aa28 in_seq=0 out_seq=0 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/2418327986 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55e5feccbd10 con 0x55e5fed86880 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fb8ff9700 10 client.31325 handle_command_reply: tid=0 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 client.31325 shutdown 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 msgr2=0x55e5fed75fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed75fd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00cc650 tx=0x7f5fc009b180 comp rx=0 tx=0).stop 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 shutdown_connections 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fed86880 0x55e5fed88c70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed76510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed78cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.092 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fb8ff9700 0 client.31325 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:38:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/2418327986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed75fd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 >> 172.21.15.52:0/2418327986 conn(0x55e5fec97cc0 msgr2=0x55e5fed675f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 shutdown_connections 2024-06-02T23:38:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 1 -- 172.21.15.52:0/2418327986 wait complete. 2024-06-02T23:38:07.093 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.088+0000 7f5fe2193b80 20 client.31325 trim_cache size 0 max 16384 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.103+0000 7f5fe2193b80 1 Processor -- start 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.103+0000 7f5fe2193b80 1 -- start start 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed6c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed73b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01acaa0 con 0x55e5fed72f60 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ae700 con 0x55e5fed72640 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ad230 con 0x55e5fed6cf20 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.105 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed8e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52054/0 (socket says 172.21.15.52:52054) 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 -- 172.21.15.52:0/3223724869 learned_addr learned my addr 172.21.15.52:0/3223724869 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed6c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed73b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 -- 172.21.15.52:0/3223724869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 msgr2=0x55e5fed6c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed6c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 -- 172.21.15.52:0/3223724869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 msgr2=0x55e5fed73b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed73b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 -- 172.21.15.52:0/3223724869 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5fed68470 con 0x55e5fed72f60 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed8e580 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80106c0 tx=0x7f5fc80109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8187c149e961ba2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fbaffd700 1 -- 172.21.15.52:0/3223724869 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8021070 con 0x55e5fed72f60 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fbaffd700 1 -- 172.21.15.52:0/3223724869 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fc801caa0 con 0x55e5fed72f60 2024-06-02T23:38:07.106 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.104+0000 7f5fbaffd700 1 -- 172.21.15.52:0/3223724869 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800fbf0 con 0x55e5fed72f60 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 -- 172.21.15.52:0/3223724869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 msgr2=0x55e5fed8e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed8e580 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f5fc80106c0 tx=0x7f5fc80109d0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 -- 172.21.15.52:0/3223724869 shutdown_connections 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed72f60 0x55e5fed8e580 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed6cf20 0x55e5fed6c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/3223724869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72640 0x55e5fed73b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 -- 172.21.15.52:0/3223724869 >> 172.21.15.52:0/3223724869 conn(0x55e5fec97cc0 msgr2=0x55e5fed73400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 -- 172.21.15.52:0/3223724869 shutdown_connections 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 -- 172.21.15.52:0/3223724869 wait complete. 2024-06-02T23:38:07.110 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.105+0000 7f5fe2193b80 1 -- start start 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 Processor -- start 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 0x55e5fed8e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72640 0x55e5fed8e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72f60 0x55e5fed8ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ae030 con 0x55e5fed6cf20 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01ad930 con 0x55e5fed72f60 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01aaa40 con 0x55e5fed72640 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fe2193b80 10 client.0 _ll_register_callbacks cb 0x55e5fec58920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 0x55e5fed8e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 0x55e5fed8e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52068/0 (socket says 172.21.15.52:52068) 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 -- 172.21.15.52:0/939381717 learned_addr learned my addr 172.21.15.52:0/939381717 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72640 0x55e5fed8e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd5ead700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72f60 0x55e5fed8ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.111 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 -- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72640 msgr2=0x55e5fed8e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72640 0x55e5fed8e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 -- 172.21.15.52:0/939381717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72f60 msgr2=0x55e5fed8ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.106+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72f60 0x55e5fed8ecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fd56ac700 1 -- 172.21.15.52:0/939381717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5fed72140 con 0x55e5fed6cf20 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fd56ac700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 0x55e5fed8e230 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc003df0 tx=0x7f5fcc00ab60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505dbbb7b2288293 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fb8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/939381717 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc02f040 con 0x55e5fed6cf20 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/939381717 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f5fcc004da0 con 0x55e5fed6cf20 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5fc801caa0 con 0x55e5fed6cf20 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/939381717 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7f5fcc044050 con 0x55e5fed6cf20 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fe2193b80 10 client.22149 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55e5fedf5b20 con 0x55e5fed6cf20 2024-06-02T23:38:07.112 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.107+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/939381717 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5fc801caa0 con 0x55e5fed6cf20 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/939381717 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7f5fcc047020 con 0x55e5fed6cf20 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 10 client.22149 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 10 client.22149 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 20 client.22149 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fedf5d50 0x55e5fedf8140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 4 client.22149 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 --> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e5fecc6290 con 0x55e5fedf5d50 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fedf5d50 0x55e5fedf8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fd4eab700 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fedf5d50 0x55e5fedf8140 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9cffe181525ee085 server_cookie=34682c6e74da1c7f in_seq=0 out_seq=0 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fb8ff9700 10 client.22149 ms_handle_connect on v2:172.21.15.174:6838/2115890873 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fb8ff9700 1 -- 172.21.15.52:0/939381717 <== mds.1 v2:172.21.15.174:6838/2115890873 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e5fecc6290 con 0x55e5fedf5d50 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fb8ff9700 10 client.22149 handle_command_reply: tid=0 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.108+0000 7f5fe2193b80 1 client.22149 shutdown 2024-06-02T23:38:07.113 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 msgr2=0x55e5fed8e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 0x55e5fed8e230 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc003df0 tx=0x7f5fcc00ab60 comp rx=0 tx=0).stop 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 shutdown_connections 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:6838/2115890873,v1:172.21.15.174:6839/2115890873] conn(0x55e5fedf5d50 0x55e5fedf8140 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e5fed72f60 0x55e5fed8ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55e5fed6cf20 0x55e5fed8e230 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fb8ff9700 0 client.22149 ms_handle_reset on v2:172.21.15.174:6838/2115890873 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 --2- 172.21.15.52:0/939381717 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5fed72640 0x55e5fed8e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 >> 172.21.15.52:0/939381717 conn(0x55e5fec97cc0 msgr2=0x55e5fed60e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 shutdown_connections 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 1 -- 172.21.15.52:0/939381717 wait complete. 2024-06-02T23:38:07.114 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fe2193b80 20 client.22149 trim_cache size 0 max 16384 2024-06-02T23:38:07.115 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:38:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fbc0a2c20 msgr2=0x7f5fbc0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fd7910700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fbc0a2c20 0x7f5fbc0a50e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00af70 tx=0x7f5fcc00c870 comp rx=0 tx=0).stop 2024-06-02T23:38:07.115 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 msgr2=0x7f5fd019edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.109+0000 7f5fd7910700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 0x7f5fd019edb0 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f5fc000d980 tx=0x7f5fc000dc90 comp rx=0 tx=0).stop 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 shutdown_connections 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5fbc0a2c20 0x7f5fbc0a50e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0105f30 0x7f5fd0198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5fd0101090 0x7f5fd019e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 --2- 172.21.15.52:0/2621574481 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5fd0101a40 0x7f5fd019edb0 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 >> 172.21.15.52:0/2621574481 conn(0x7f5fd00766b0 msgr2=0x7f5fd00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 shutdown_connections 2024-06-02T23:38:07.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:07.110+0000 7f5fd7910700 1 -- 172.21.15.52:0/2621574481 wait complete. 2024-06-02T23:38:07.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:07 smithi123 ceph-mon[35977]: pgmap v1218: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:07.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:07 smithi052 ceph-mon[32340]: pgmap v1218: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:07.834 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-06-02T23:38:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:07 smithi174 ceph-mon[35994]: pgmap v1218: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:08.087 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:09.553 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:09 smithi052 ceph-mon[32340]: pgmap v1219: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:09.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:09 smithi123 ceph-mon[35977]: pgmap v1219: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:09.822 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.821+0000 7fee87415700 1 -- 172.21.15.52:0/1609772747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee801030f0 msgr2=0x7fee80107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.821+0000 7fee87415700 1 --2- 172.21.15.52:0/1609772747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee801030f0 0x7fee80107110 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7fee700099c0 tx=0x7fee70009cd0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 -- 172.21.15.52:0/1609772747 shutdown_connections 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 --2- 172.21.15.52:0/1609772747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee80107840 0x7fee8010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 --2- 172.21.15.52:0/1609772747 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee80102740 0x7fee80102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 --2- 172.21.15.52:0/1609772747 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee801030f0 0x7fee80107110 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 -- 172.21.15.52:0/1609772747 >> 172.21.15.52:0/1609772747 conn(0x7fee800795f0 msgr2=0x7fee80079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 -- 172.21.15.52:0/1609772747 shutdown_connections 2024-06-02T23:38:09.823 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.822+0000 7fee87415700 1 -- 172.21.15.52:0/1609772747 wait complete. 2024-06-02T23:38:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 Processor -- start 2024-06-02T23:38:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 -- start start 2024-06-02T23:38:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee80102740 0x7fee80074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee801030f0 0x7fee80074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.825 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 0x7fee80075210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee80112620 con 0x7fee80107840 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee801124a0 con 0x7fee801030f0 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee87415700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee801127a0 con 0x7fee80102740 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 0x7fee80075210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 0x7fee80075210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52090/0 (socket says 172.21.15.52:52090) 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 -- 172.21.15.52:0/3535849144 learned_addr learned my addr 172.21.15.52:0/3535849144 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee851b1700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee80102740 0x7fee80074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee849b0700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee801030f0 0x7fee80074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.826 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 -- 172.21.15.52:0/3535849144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee80102740 msgr2=0x7fee80074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee80102740 0x7fee80074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 -- 172.21.15.52:0/3535849144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee801030f0 msgr2=0x7fee80074cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee801030f0 0x7fee80074cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.823+0000 7fee859b2700 1 -- 172.21.15.52:0/3535849144 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee801a7000 con 0x7fee80107840 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.824+0000 7fee859b2700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 0x7fee80075210 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fee78007ee0 tx=0x7fee7800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.824+0000 7fee767fc700 1 -- 172.21.15.52:0/3535849144 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78004bb0 con 0x7fee80107840 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.824+0000 7fee767fc700 1 -- 172.21.15.52:0/3535849144 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee78004d50 con 0x7fee80107840 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.824+0000 7fee767fc700 1 -- 172.21.15.52:0/3535849144 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780058e0 con 0x7fee80107840 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.824+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee801a72f0 con 0x7fee80107840 2024-06-02T23:38:09.827 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.824+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a77b0 con 0x7fee80107840 2024-06-02T23:38:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.826+0000 7fee767fc700 1 -- 172.21.15.52:0/3535849144 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee7801a020 con 0x7fee80107840 2024-06-02T23:38:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.827+0000 7fee767fc700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.827+0000 7fee851b1700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.827+0000 7fee767fc700 1 -- 172.21.15.52:0/3535849144 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee780ce660 con 0x7fee80107840 2024-06-02T23:38:09.828 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee851b1700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fee7c005950 tx=0x7fee7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 Processor -- start 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 -- start start 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c28a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d826b74810 con 0x55d826c28f60 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d826b1c720 con 0x55d826c22f20 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee91c98b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d826b707d0 con 0x55d826c28640 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee849b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee849b0700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c229e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52102/0 (socket says 172.21.15.52:52102) 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.828+0000 7fee849b0700 1 -- 172.21.15.52:0/3241474511 learned_addr learned my addr 172.21.15.52:0/3241474511 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:09.830 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee859b2700 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee851b1700 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c28a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee849b0700 1 -- 172.21.15.52:0/3241474511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 msgr2=0x55d826c28a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee849b0700 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c28a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee849b0700 1 -- 172.21.15.52:0/3241474511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 msgr2=0x55d826c23340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee849b0700 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c23340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee849b0700 1 -- 172.21.15.52:0/3241474511 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d826b12f50 con 0x55d826c28f60 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee849b0700 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c229e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fee700099c0 tx=0x7fee70005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a81a9e0c36b03f2 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee6b7fe700 1 -- 172.21.15.52:0/3241474511 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70028070 con 0x55d826c28f60 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee6b7fe700 1 -- 172.21.15.52:0/3241474511 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee70005070 con 0x55d826c28f60 2024-06-02T23:38:09.831 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.829+0000 7fee6b7fe700 1 -- 172.21.15.52:0/3241474511 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7000ba80 con 0x55d826c28f60 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 -- 172.21.15.52:0/3241474511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 msgr2=0x55d826c229e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c229e0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fee700099c0 tx=0x7fee70005c70 comp rx=0 tx=0).stop 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 -- 172.21.15.52:0/3241474511 shutdown_connections 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c23340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c28a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3241474511 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c229e0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 -- 172.21.15.52:0/3241474511 >> 172.21.15.52:0/3241474511 conn(0x55d826b4dcc0 msgr2=0x55d826c1d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 -- 172.21.15.52:0/3241474511 shutdown_connections 2024-06-02T23:38:09.834 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 -- 172.21.15.52:0/3241474511 wait complete. 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 -- start start 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.830+0000 7fee91c98b80 1 Processor -- start 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c2ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c2bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28f60 0x55d826c2c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x55d826c2f970 con 0x55d826c22f20 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d826b14bf0 con 0x55d826c28f60 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d826c2faf0 con 0x55d826c28640 2024-06-02T23:38:09.835 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee91c98b80 10 client.0 _ll_register_callbacks cb 0x55d826b0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c2ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c2ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52118/0 (socket says 172.21.15.52:52118) 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 -- 172.21.15.52:0/2012327354 learned_addr learned my addr 172.21.15.52:0/2012327354 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee849b0700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c2bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 -- 172.21.15.52:0/2012327354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 msgr2=0x55d826c2bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c2bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee859b2700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28f60 0x55d826c2c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 -- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28f60 msgr2=0x55d826c2c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28f60 0x55d826c2c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee851b1700 1 -- 172.21.15.52:0/2012327354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d826ac38d0 con 0x55d826c22f20 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.831+0000 7fee849b0700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:38:09.836 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee851b1700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c2ecc0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0116c0 tx=0x7fee7c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6897c12f63bad9c3 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee68ff9700 1 -- 172.21.15.52:0/2012327354 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004a30 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee70005070 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee68ff9700 1 -- 172.21.15.52:0/2012327354 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee7c004bd0 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee68ff9700 1 -- 172.21.15.52:0/2012327354 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fee7c02e070 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee91c98b80 10 client.22161 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d826c42ae0 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee68ff9700 1 -- 172.21.15.52:0/2012327354 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee70005070 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee68ff9700 1 -- 172.21.15.52:0/2012327354 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7fee7c01f7f0 con 0x55d826c22f20 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee91c98b80 10 client.22161 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:09.837 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee91c98b80 10 client.22161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.832+0000 7fee91c98b80 20 client.22161 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.833+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826c45120 0x55d826c47530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.833+0000 7fee91c98b80 4 client.22161 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.833+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d826b81d10 con 0x55d826c45120 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.833+0000 7fee849b0700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826c45120 0x55d826c47530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.833+0000 7fee68ff9700 10 client.22161 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:38:09.838 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.833+0000 7fee849b0700 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826c45120 0x55d826c47530 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143f6610cb8e8a56 server_cookie=1571d94b751ac9f9 in_seq=0 out_seq=0 2024-06-02T23:38:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee68ff9700 1 -- 172.21.15.52:0/2012327354 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24173 (crc 0 0 0) 0x55d826b81d10 con 0x55d826c45120 2024-06-02T23:38:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee68ff9700 10 client.22161 handle_command_reply: tid=0 2024-06-02T23:38:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 client.22161 shutdown 2024-06-02T23:38:09.839 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 msgr2=0x55d826c2ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c2ecc0 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fee7c0116c0 tx=0x7fee7c0047c0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 shutdown_connections 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826c45120 0x55d826c47530 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28f60 0x55d826c2c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee68ff9700 0 client.22161 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c2ecc0 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2012327354 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 >> 172.21.15.52:0/2012327354 conn(0x55d826b4dcc0 msgr2=0x55d826c219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 shutdown_connections 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.834+0000 7fee91c98b80 1 -- 172.21.15.52:0/2012327354 wait complete. 2024-06-02T23:38:09.840 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.835+0000 7fee91c98b80 20 client.22161 trim_cache size 0 max 16384 2024-06-02T23:38:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 Processor -- start 2024-06-02T23:38:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 -- start start 2024-06-02T23:38:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c229e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c29a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c4ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aa800 con 0x55d826c28f60 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ac460 con 0x55d826c22f20 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee91c98b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee801aaf90 con 0x55d826c28640 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c4ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c4ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52126/0 (socket says 172.21.15.52:52126) 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 -- 172.21.15.52:0/2464058721 learned_addr learned my addr 172.21.15.52:0/2464058721 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee851b1700 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c229e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 -- 172.21.15.52:0/2464058721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 msgr2=0x55d826c29a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c29a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 -- 172.21.15.52:0/2464058721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 msgr2=0x55d826c229e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c229e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.850+0000 7fee859b2700 1 -- 172.21.15.52:0/2464058721 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d826c29f90 con 0x55d826c28f60 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee851b1700 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee859b2700 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c4ce00 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fee78097350 tx=0x7fee78097190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edd946a9effb3140 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:09.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee6affd700 1 -- 172.21.15.52:0/2464058721 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7809a850 con 0x55d826c28f60 2024-06-02T23:38:09.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee6affd700 1 -- 172.21.15.52:0/2464058721 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee7809b5f0 con 0x55d826c28f60 2024-06-02T23:38:09.853 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee6affd700 1 -- 172.21.15.52:0/2464058721 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee780a3b60 con 0x55d826c28f60 2024-06-02T23:38:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee91c98b80 1 -- 172.21.15.52:0/2464058721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 msgr2=0x55d826c4ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c4ce00 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fee78097350 tx=0x7fee78097190 comp rx=0 tx=0).stop 2024-06-02T23:38:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee91c98b80 1 -- 172.21.15.52:0/2464058721 shutdown_connections 2024-06-02T23:38:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c28f60 0x55d826c4ce00 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c22f20 0x55d826c229e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.851+0000 7fee91c98b80 1 --2- 172.21.15.52:0/2464058721 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28640 0x55d826c29a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- 172.21.15.52:0/2464058721 >> 172.21.15.52:0/2464058721 conn(0x55d826b4dcc0 msgr2=0x55d826c29340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- 172.21.15.52:0/2464058721 shutdown_connections 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- 172.21.15.52:0/2464058721 wait complete. 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- start start 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 Processor -- start 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c354c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28640 0x55d826c4c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28f60 0x55d826c4cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee801abd90 con 0x55d826c22f20 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee801ab690 con 0x55d826c28640 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee8019c7b0 con 0x55d826c28f60 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.852+0000 7fee91c98b80 10 client.0 _ll_register_callbacks cb 0x55d826b0e920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-06-02T23:38:09.857 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c354c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c354c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52132/0 (socket says 172.21.15.52:52132) 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 -- 172.21.15.52:0/3744876292 learned_addr learned my addr 172.21.15.52:0/3744876292 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee849b0700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28640 0x55d826c4c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee859b2700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28f60 0x55d826c4cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 -- 172.21.15.52:0/3744876292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28f60 msgr2=0x55d826c4cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28f60 0x55d826c4cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 -- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28640 msgr2=0x55d826c4c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28640 0x55d826c4c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 -- 172.21.15.52:0/3744876292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d826c4d500 con 0x55d826c22f20 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee851b1700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c354c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fee7c006010 tx=0x7fee7c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47264f33ab9b8ef7 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.52:3300/0 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee68ff9700 1 -- 172.21.15.52:0/3744876292 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c011e10 con 0x55d826c22f20 2024-06-02T23:38:09.858 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee68ff9700 1 -- 172.21.15.52:0/3744876292 <== mon.0 v2:172.21.15.52:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee7c01f850 con 0x55d826c22f20 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.853+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7809b5f0 con 0x55d826c22f20 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee68ff9700 1 -- 172.21.15.52:0/3744876292 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) ==== 24+0+0 (secure 0 0 0) 0x7fee7c004830 con 0x55d826c22f20 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 10 client.22167 fetch_fsmap learned FSMap version 48 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) -- 0x55d826cafdc0 con 0x55d826c22f20 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee68ff9700 1 -- 172.21.15.52:0/3744876292 <== mon.0 v2:172.21.15.52:3300/0 4 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee7809b5f0 con 0x55d826c22f20 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee68ff9700 1 -- 172.21.15.52:0/3744876292 <== mon.0 v2:172.21.15.52:3300/0 5 ==== fsmap(e 48) ==== 8390+0+0 (secure 0 0 0) 0x7fee7c0055a0 con 0x55d826c22f20 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 10 client.22167 fetch_fsmap finished waiting for FSMap version 48 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 10 client.22167 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 20 client.22167 populate_metadata read hostname 'smithi052' 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826cb0500 0x55d826cb2920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 4 client.22167 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 --> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d826b7c290 con 0x55d826cb0500 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.854+0000 7fee849b0700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826cb0500 0x55d826cb2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:09.859 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee849b0700 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826cb0500 0x55d826cb2920 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=614a98914ba3f8f4 server_cookie=91086a1ababc9b78 in_seq=0 out_seq=0 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee68ff9700 10 client.22167 ms_handle_connect on v2:172.21.15.123:6838/2043673684 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee68ff9700 1 -- 172.21.15.52:0/3744876292 <== mds.0 v2:172.21.15.123:6838/2043673684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d826b7c290 con 0x55d826cb0500 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee68ff9700 10 client.22167 handle_command_reply: tid=0 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 client.22167 shutdown 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 msgr2=0x55d826c354c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c354c0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fee7c006010 tx=0x7fee7c004700 comp rx=0 tx=0).stop 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 shutdown_connections 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:6838/2043673684,v1:172.21.15.123:6839/2043673684] conn(0x55d826cb0500 0x55d826cb2920 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d826c28f60 0x55d826c4cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x55d826c22f20 0x55d826c354c0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.860 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 --2- 172.21.15.52:0/3744876292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d826c28640 0x55d826c4c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee68ff9700 0 client.22167 ms_handle_reset on v2:172.21.15.123:6838/2043673684 2024-06-02T23:38:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 >> 172.21.15.52:0/3744876292 conn(0x55d826b4dcc0 msgr2=0x55d826c2a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 shutdown_connections 2024-06-02T23:38:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 1 -- 172.21.15.52:0/3744876292 wait complete. 2024-06-02T23:38:09.861 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.855+0000 7fee91c98b80 20 client.22167 trim_cache size 0 max 16384 2024-06-02T23:38:09.861 INFO:teuthology.orchestra.run.smithi052.stdout:[] 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee6c0a2ca0 msgr2=0x7fee6c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fee7c005950 tx=0x7fee7c00b410 comp rx=0 tx=0).stop 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 msgr2=0x7fee80075210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 0x7fee80075210 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fee78007ee0 tx=0x7fee7800d3b0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 shutdown_connections 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee6c0a2ca0 0x7fee6c0a5160 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee80107840 0x7fee80075210 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee80102740 0x7fee80074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 --2- 172.21.15.52:0/3535849144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee801030f0 0x7fee80074cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:09.862 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 >> 172.21.15.52:0/3535849144 conn(0x7fee800795f0 msgr2=0x7fee80108990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 shutdown_connections 2024-06-02T23:38:09.863 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:09.856+0000 7fee87415700 1 -- 172.21.15.52:0/3535849144 wait complete. 2024-06-02T23:38:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:09 smithi174 ceph-mon[35994]: pgmap v1219: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:10.570 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_dfcc4b3c2c55532c4c04fa47551c9df1dffdf746/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-06-02T23:38:10.571 INFO:tasks.fwd_scrub:done joining 2024-06-02T23:38:10.571 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-06-02T23:38:10.582 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 -- 172.21.15.52:0/2017026269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 msgr2=0x7f2324109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 --2- 172.21.15.52:0/2017026269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f2324109e30 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f231c009ca0 tx=0x7f231c02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 -- 172.21.15.52:0/2017026269 shutdown_connections 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 --2- 172.21.15.52:0/2017026269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2324102be0 0x7f2324103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 --2- 172.21.15.52:0/2017026269 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f2324109e30 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 --2- 172.21.15.52:0/2017026269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23241019f0 0x7f2324101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 -- 172.21.15.52:0/2017026269 >> 172.21.15.52:0/2017026269 conn(0x7f23240fcf80 msgr2=0x7f23240ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 -- 172.21.15.52:0/2017026269 shutdown_connections 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.692+0000 7f23296e7640 1 -- 172.21.15.52:0/2017026269 wait complete. 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 Processor -- start 2024-06-02T23:38:10.695 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 -- start start 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23241019f0 0x7f2324197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2324102be0 0x7f2324198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f232419f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f232410d520 con 0x7f23241035a0 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f232410d3a0 con 0x7f2324102be0 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23296e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f232410d6a0 con 0x7f23241019f0 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23237fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f232419f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23237fe640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f232419f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52154/0 (socket says 172.21.15.52:52154) 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f23237fe640 1 -- 172.21.15.52:0/2856064848 learned_addr learned my addr 172.21.15.52:0/2856064848 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f231bfff640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2324102be0 0x7f2324198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.693+0000 7f2322ffd640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23241019f0 0x7f2324197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23237fe640 1 -- 172.21.15.52:0/2856064848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23241019f0 msgr2=0x7f2324197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23237fe640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23241019f0 0x7f2324197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.696 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23237fe640 1 -- 172.21.15.52:0/2856064848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2324102be0 msgr2=0x7f2324198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23237fe640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2324102be0 0x7f2324198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23237fe640 1 -- 172.21.15.52:0/2856064848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232419f7e0 con 0x7f23241035a0 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23237fe640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f232419f170 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f231c031970 tx=0x7f231c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c004040 con 0x7f23241035a0 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f231c004690 con 0x7f23241035a0 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f231c030c60 con 0x7f23241035a0 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232419fa70 con 0x7f23241035a0 2024-06-02T23:38:10.697 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.694+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23241a0010 con 0x7f23241035a0 2024-06-02T23:38:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.695+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2324108d10 con 0x7f23241035a0 2024-06-02T23:38:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.696+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f231c0041e0 con 0x7f23241035a0 2024-06-02T23:38:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.697+0000 7f2320ff9640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23100a2c10 0x7f23100a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.697+0000 7f2322ffd640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23100a2c10 0x7f23100a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.697+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f231c0f0600 con 0x7f23241035a0 2024-06-02T23:38:10.698 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.697+0000 7f2322ffd640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23100a2c10 0x7f23100a50d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2324102a40 tx=0x7f231400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:10.701 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.701+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f231c0bd530 con 0x7f23241035a0 2024-06-02T23:38:10.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.888+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f232402d460 con 0x7f23241035a0 2024-06-02T23:38:10.890 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.890+0000 7f2320ff9640 1 -- 172.21.15.52:0/2856064848 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7f231c0bce30 con 0x7f23241035a0 2024-06-02T23:38:10.891 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:38:10.891 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23100a2c10 msgr2=0x7f23100a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23100a2c10 0x7f23100a50d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f2324102a40 tx=0x7f231400b390 comp rx=0 tx=0).stop 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 msgr2=0x7f232419f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f232419f170 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f231c031970 tx=0x7f231c003ba0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 shutdown_connections 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f23100a2c10 0x7f23100a50d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f23241035a0 0x7f232419f170 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2324102be0 0x7f2324198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 --2- 172.21.15.52:0/2856064848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23241019f0 0x7f2324197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 >> 172.21.15.52:0/2856064848 conn(0x7f23240fcf80 msgr2=0x7f2324105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 shutdown_connections 2024-06-02T23:38:10.893 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:10.892+0000 7f23296e7640 1 -- 172.21.15.52:0/2856064848 wait complete. 2024-06-02T23:38:10.904 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-06-02T23:02:34.195730+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-06-02T23:07:26.794014+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-06-02T23:07:30.138193+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-06-02T23:38:10.905 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-06-02T23:38:11.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 -- 172.21.15.52:0/1302764925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf101008e0 msgr2=0x7fdf10100d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 --2- 172.21.15.52:0/1302764925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf101008e0 0x7fdf10100d00 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009900 tx=0x7fdf0802a410 comp rx=0 tx=0).stop 2024-06-02T23:38:11.015 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 -- 172.21.15.52:0/1302764925 shutdown_connections 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 --2- 172.21.15.52:0/1302764925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf10102490 0x7fdf10108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 --2- 172.21.15.52:0/1302764925 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf10101ad0 0x7fdf10101f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 --2- 172.21.15.52:0/1302764925 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf101008e0 0x7fdf10100d00 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 -- 172.21.15.52:0/1302764925 >> 172.21.15.52:0/1302764925 conn(0x7fdf100fbe50 msgr2=0x7fdf100fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 -- 172.21.15.52:0/1302764925 shutdown_connections 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 -- 172.21.15.52:0/1302764925 wait complete. 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 Processor -- start 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.013+0000 7fdf1866b640 1 -- start start 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf101008e0 0x7fdf10197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 0x7fdf101981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf10102490 0x7fdf1019f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1010c460 con 0x7fdf10101ad0 2024-06-02T23:38:11.016 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1010c2e0 con 0x7fdf101008e0 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdf1010c5e0 con 0x7fdf10102490 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 0x7fdf101981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 0x7fdf101981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52178/0 (socket says 172.21.15.52:52178) 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 -- 172.21.15.52:0/736501156 learned_addr learned my addr 172.21.15.52:0/736501156 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf16407640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf101008e0 0x7fdf10197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 -- 172.21.15.52:0/736501156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf10102490 msgr2=0x7fdf1019f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf10102490 0x7fdf1019f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 -- 172.21.15.52:0/736501156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf101008e0 msgr2=0x7fdf10197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf101008e0 0x7fdf10197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 -- 172.21.15.52:0/736501156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf1019f8c0 con 0x7fdf10101ad0 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf15c06640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 0x7fdf101981c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400d950 tx=0x7fdf0400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf04017070 con 0x7fdf10101ad0 2024-06-02T23:38:11.017 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fdf04009c30 con 0x7fdf10101ad0 2024-06-02T23:38:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf1019fbb0 con 0x7fdf10101ad0 2024-06-02T23:38:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf0400f620 con 0x7fdf10101ad0 2024-06-02T23:38:11.018 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.014+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf101a0150 con 0x7fdf10101ad0 2024-06-02T23:38:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.015+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf101a03e0 con 0x7fdf10101ad0 2024-06-02T23:38:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.017+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fdf040109a0 con 0x7fdf10101ad0 2024-06-02T23:38:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.017+0000 7fdf0f7fe640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdef80a2c10 0x7fdef80a50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.017+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fdf040cec80 con 0x7fdf10101ad0 2024-06-02T23:38:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.018+0000 7fdf16407640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdef80a2c10 0x7fdef80a50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.019 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.018+0000 7fdf16407640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdef80a2c10 0x7fdef80a50d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdf10198b50 tx=0x7fdf080314a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.022 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.022+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fdf04009de0 con 0x7fdf10101ad0 2024-06-02T23:38:11.208 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.207+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf101a05b0 con 0x7fdf10101ad0 2024-06-02T23:38:11.209 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.208+0000 7fdf0f7fe640 1 -- 172.21.15.52:0/736501156 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) ==== 93+0+10 (secure 0 0 0) 0x7fdf0409bc90 con 0x7fdf10101ad0 2024-06-02T23:38:11.209 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 1 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdef80a2c10 msgr2=0x7fdef80a50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdef80a2c10 0x7fdef80a50d0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fdf10198b50 tx=0x7fdf080314a0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 msgr2=0x7fdf101981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 0x7fdf101981c0 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fdf0400d950 tx=0x7fdf0400de10 comp rx=0 tx=0).stop 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 shutdown_connections 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fdef80a2c10 0x7fdef80a50d0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdf10102490 0x7fdf1019f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fdf10101ad0 0x7fdf101981c0 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 --2- 172.21.15.52:0/736501156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf101008e0 0x7fdf10197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.211 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 >> 172.21.15.52:0/736501156 conn(0x7fdf100fbe50 msgr2=0x7fdf100fd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 shutdown_connections 2024-06-02T23:38:11.212 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.210+0000 7fdf1866b640 1 -- 172.21.15.52:0/736501156 wait complete. 2024-06-02T23:38:11.222 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/1897576935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4101570 msgr2=0x7f2ce41019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/1897576935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4101570 0x7f2ce41019f0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009ca0 tx=0x7f2cdc02a8c0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/1897576935 shutdown_connections 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/1897576935 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100bd0 0x7f2ce4101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/1897576935 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4101570 0x7f2ce41019f0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/1897576935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce4100270 0x7f2ce4100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/1897576935 >> 172.21.15.52:0/1897576935 conn(0x7f2ce40fbe70 msgr2=0x7f2ce40fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/1897576935 shutdown_connections 2024-06-02T23:38:11.334 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.331+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/1897576935 wait complete. 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 Processor -- start 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 -- start start 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100270 0x7f2ce4199dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 0x7f2ce419a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce4101570 0x7f2ce41a1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410f700 con 0x7f2ce4100bd0 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410f580 con 0x7f2ce4101570 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ceb3e7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce410f880 con 0x7f2ce4100270 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ce8982640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 0x7f2ce419a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.335 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ce8982640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 0x7f2ce419a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52194/0 (socket says 172.21.15.52:52194) 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ce8982640 1 -- 172.21.15.52:0/3923252252 learned_addr learned my addr 172.21.15.52:0/3923252252 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.332+0000 7f2ce9984640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce4101570 0x7f2ce41a1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce9183640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100270 0x7f2ce4199dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce8982640 1 -- 172.21.15.52:0/3923252252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100270 msgr2=0x7f2ce4199dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce8982640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100270 0x7f2ce4199dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce8982640 1 -- 172.21.15.52:0/3923252252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce4101570 msgr2=0x7f2ce41a1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce8982640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce4101570 0x7f2ce41a1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce8982640 1 -- 172.21.15.52:0/3923252252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce41a19d0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce9183640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100270 0x7f2ce4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ce8982640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 0x7f2ce419a300 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400d950 tx=0x7f2cd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4017070 con 0x7f2ce4100bd0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce41a1cc0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2cd4009c30 con 0x7f2ce4100bd0 2024-06-02T23:38:11.336 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cd400f620 con 0x7f2ce4100bd0 2024-06-02T23:38:11.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.333+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce41a21b0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.334+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce410b1c0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.335+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2cd40109a0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.337 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.336+0000 7f2cda7fc640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd00a2b90 0x7f2cd00a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.336+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f2cd40cecf0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.336+0000 7f2ce9183640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd00a2b90 0x7f2cd00a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.338 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.337+0000 7f2ce9183640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd00a2b90 0x7f2cd00a5050 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2ce419b290 tx=0x7f2ce0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.341 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.341+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2cd40d3070 con 0x7f2ce4100bd0 2024-06-02T23:38:11.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.528+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2ce402d2e0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.529 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.529+0000 7f2cda7fc640 1 -- 172.21.15.52:0/3923252252 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v90) ==== 104+0+11 (secure 0 0 0) 0x7f2cd4009de0 con 0x7f2ce4100bd0 2024-06-02T23:38:11.529 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 32 2024-06-02T23:38:11.531 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd00a2b90 msgr2=0x7f2cd00a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd00a2b90 0x7f2cd00a5050 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2ce419b290 tx=0x7f2ce0005f90 comp rx=0 tx=0).stop 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 msgr2=0x7f2ce419a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 0x7f2ce419a300 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400d950 tx=0x7f2cd400de10 comp rx=0 tx=0).stop 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 shutdown_connections 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2cd00a2b90 0x7f2cd00a5050 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ce4101570 0x7f2ce41a1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ce4100bd0 0x7f2ce419a300 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 --2- 172.21.15.52:0/3923252252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ce4100270 0x7f2ce4199dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 >> 172.21.15.52:0/3923252252 conn(0x7f2ce40fbe70 msgr2=0x7f2ce40fd5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 shutdown_connections 2024-06-02T23:38:11.532 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.531+0000 7f2ceb3e7640 1 -- 172.21.15.52:0/3923252252 wait complete. 2024-06-02T23:38:11.541 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:11 smithi052 ceph-mon[32340]: pgmap v1220: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:11.541 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2856064848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:38:11.542 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:11 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/736501156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:38:11.543 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-06-02T23:38:11.654 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 -- 172.21.15.52:0/2030271943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c101af0 msgr2=0x7f127c101f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 --2- 172.21.15.52:0/2030271943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c101af0 0x7f127c101f70 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f1264009900 tx=0x7f126402a410 comp rx=0 tx=0).stop 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 -- 172.21.15.52:0/2030271943 shutdown_connections 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 --2- 172.21.15.52:0/2030271943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c1024b0 0x7f127c108d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 --2- 172.21.15.52:0/2030271943 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c101af0 0x7f127c101f70 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 --2- 172.21.15.52:0/2030271943 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c100900 0x7f127c100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 -- 172.21.15.52:0/2030271943 >> 172.21.15.52:0/2030271943 conn(0x7f127c0fbe90 msgr2=0x7f127c0fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 -- 172.21.15.52:0/2030271943 shutdown_connections 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.651+0000 7f128288c640 1 -- 172.21.15.52:0/2030271943 wait complete. 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 Processor -- start 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 -- start start 2024-06-02T23:38:11.655 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c100900 0x7f127c104890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c101af0 0x7f127c102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 0x7f127c103410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f127c10c430 con 0x7f127c1024b0 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f127c10c2b0 con 0x7f127c100900 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f128288c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f127c10c5b0 con 0x7f127c101af0 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f1280e29640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 0x7f127c103410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f1280e29640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 0x7f127c103410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52214/0 (socket says 172.21.15.52:52214) 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.652+0000 7f1280e29640 1 -- 172.21.15.52:0/2363132248 learned_addr learned my addr 172.21.15.52:0/2363132248 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f127bfff640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c100900 0x7f127c104890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f127b7fe640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c101af0 0x7f127c102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f1280e29640 1 -- 172.21.15.52:0/2363132248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c101af0 msgr2=0x7f127c102ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.656 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f1280e29640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c101af0 0x7f127c102ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f1280e29640 1 -- 172.21.15.52:0/2363132248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c100900 msgr2=0x7f127c104890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f1280e29640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c100900 0x7f127c104890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f1280e29640 1 -- 172.21.15.52:0/2363132248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f127c103ca0 con 0x7f127c1024b0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f1280e29640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 0x7f127c103410 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f127400eaa0 tx=0x7f127400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f127400cdb0 con 0x7f127c1024b0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1274005020 con 0x7f127c1024b0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f127c1a1da0 con 0x7f127c1024b0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12740116d0 con 0x7f127c1024b0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.653+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f127c1a21b0 con 0x7f127c1024b0 2024-06-02T23:38:11.657 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.654+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f127c1a24a0 con 0x7f127c1024b0 2024-06-02T23:38:11.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.655+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f12740051d0 con 0x7f127c1024b0 2024-06-02T23:38:11.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.656+0000 7f12797fa640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12680a2b90 0x7f12680a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.656+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f12740cf180 con 0x7f127c1024b0 2024-06-02T23:38:11.658 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.656+0000 7f127bfff640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12680a2b90 0x7f12680a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.659 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.657+0000 7f127bfff640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12680a2b90 0x7f12680a5050 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f126c005fd0 tx=0x7f126c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.661 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.661+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1274016030 con 0x7f127c1024b0 2024-06-02T23:38:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:11 smithi123 ceph-mon[35977]: pgmap v1220: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2856064848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:38:11.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:11 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/736501156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:38:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.846+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f127c1a2830 con 0x7f127c1024b0 2024-06-02T23:38:11.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.846+0000 7f12797fa640 1 -- 172.21.15.52:0/2363132248 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v90) ==== 100+0+11 (secure 0 0 0) 0x7f127409bac0 con 0x7f127c1024b0 2024-06-02T23:38:11.847 INFO:teuthology.orchestra.run.smithi052.stdout:pg_num: 64 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12680a2b90 msgr2=0x7f12680a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12680a2b90 0x7f12680a5050 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f126c005fd0 tx=0x7f126c00b390 comp rx=0 tx=0).stop 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 msgr2=0x7f127c103410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 0x7f127c103410 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f127400eaa0 tx=0x7f127400ef60 comp rx=0 tx=0).stop 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 shutdown_connections 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f12680a2b90 0x7f12680a5050 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f127c101af0 0x7f127c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f127c1024b0 0x7f127c103410 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 --2- 172.21.15.52:0/2363132248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f127c100900 0x7f127c104890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 >> 172.21.15.52:0/2363132248 conn(0x7f127c0fbe90 msgr2=0x7f127c0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 shutdown_connections 2024-06-02T23:38:11.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.848+0000 7f128288c640 1 -- 172.21.15.52:0/2363132248 wait complete. 2024-06-02T23:38:11.860 DEBUG:teuthology.orchestra.run.smithi052:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-06-02T23:38:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:11 smithi174 ceph-mon[35994]: pgmap v1220: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2856064848' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:38:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:11 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/736501156' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 -- 172.21.15.52:0/283605395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0381035c0 msgr2=0x7ff038109e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 --2- 172.21.15.52:0/283605395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0381035c0 0x7ff038109e50 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7ff030009ca0 tx=0x7ff03002a8c0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 -- 172.21.15.52:0/283605395 shutdown_connections 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 --2- 172.21.15.52:0/283605395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff038102c00 0x7ff038103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 --2- 172.21.15.52:0/283605395 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff0381035c0 0x7ff038109e50 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 --2- 172.21.15.52:0/283605395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff038101a10 0x7ff038101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 -- 172.21.15.52:0/283605395 >> 172.21.15.52:0/283605395 conn(0x7ff0380fcfa0 msgr2=0x7ff0380ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 -- 172.21.15.52:0/283605395 shutdown_connections 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.964+0000 7ff03f509640 1 -- 172.21.15.52:0/283605395 wait complete. 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 Processor -- start 2024-06-02T23:38:11.967 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 -- start start 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 0x7ff038199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff038102c00 0x7ff03819a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0381035c0 0x7ff0381a13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810d540 con 0x7ff038101a10 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810d3c0 con 0x7ff0381035c0 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03f509640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff03810d6c0 con 0x7ff038102c00 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 0x7ff038199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 0x7ff038199e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:52234/0 (socket says 172.21.15.52:52234) 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 -- 172.21.15.52:0/663940625 learned_addr learned my addr 172.21.15.52:0/663940625 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 -- 172.21.15.52:0/663940625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff038102c00 msgr2=0x7ff03819a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03caa4640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff038102c00 0x7ff03819a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03daa6640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0381035c0 0x7ff0381a13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff038102c00 0x7ff03819a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.968 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 -- 172.21.15.52:0/663940625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0381035c0 msgr2=0x7ff0381a13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0381035c0 0x7ff0381a13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03d2a5640 1 -- 172.21.15.52:0/663940625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0381a1a20 con 0x7ff038101a10 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.965+0000 7ff03daa6640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0381035c0 0x7ff0381a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.966+0000 7ff03d2a5640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 0x7ff038199e50 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff03400d770 tx=0x7ff03400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.966+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034017070 con 0x7ff038101a10 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.966+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7ff034009c30 con 0x7ff038101a10 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.966+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0381a1d10 con 0x7ff038101a10 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.966+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff034003d00 con 0x7ff038101a10 2024-06-02T23:38:11.969 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.966+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0381a21b0 con 0x7ff038101a10 2024-06-02T23:38:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.967+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0381a2410 con 0x7ff038101a10 2024-06-02T23:38:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.968+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7ff034003ea0 con 0x7ff038101a10 2024-06-02T23:38:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.969+0000 7ff02e7fc640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0240a2b90 0x7ff0240a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.969+0000 7ff03caa4640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0240a2b90 0x7ff0240a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.969+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7ff0340cee00 con 0x7ff038101a10 2024-06-02T23:38:11.970 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.970+0000 7ff03caa4640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0240a2b90 0x7ff0240a5050 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff028005fd0 tx=0x7ff028005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:11.973 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:11.973+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7ff03409bd90 con 0x7ff038101a10 2024-06-02T23:38:12.210 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.209+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7ff0381a25e0 con 0x7ff038101a10 2024-06-02T23:38:12.219 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.218+0000 7ff02e7fc640 1 -- 172.21.15.52:0/663940625 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v18) ==== 81+0+740370 (secure 0 0 0) 0x7ff034009de0 con 0x7ff038101a10 2024-06-02T23:38:12.228 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "epoch": 18, 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "active_gid": 14150, 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "active_name": "x", 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addrs": { 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6800", 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2596667410 2024-06-02T23:38:12.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v1", 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:6801", 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 2596667410 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "active_addr": "172.21.15.52:6801/2596667410", 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "active_change": "2024-06-02T23:00:09.671774+0000", 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "active_mgr_features": 4540701547738038271, 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "available": true, 2024-06-02T23:38:12.230 INFO:teuthology.orchestra.run.smithi052.stdout: "standbys": [ 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 24109, 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "z", 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.231 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.236 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.237 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.239 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.241 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.242 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.243 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.245 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.247 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.248 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-06-02T23:38:12.249 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-06-02T23:38:12.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-06-02T23:38:12.251 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.253 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.255 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.257 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.258 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-06-02T23:38:12.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-02T23:38:12.260 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-06-02T23:38:12.261 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-02T23:38:12.262 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-02T23:38:12.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-02T23:38:12.264 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.267 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.269 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-06-02T23:38:12.270 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-06-02T23:38:12.271 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-06-02T23:38:12.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.273 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.274 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.275 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.279 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.281 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.282 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.283 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.284 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.285 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-06-02T23:38:12.286 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.288 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.289 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-06-02T23:38:12.290 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-06-02T23:38:12.291 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-06-02T23:38:12.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.293 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.295 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.297 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.298 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.300 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.301 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.302 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.303 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.304 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.305 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.306 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.308 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.309 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.311 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.312 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.313 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.314 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.315 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-06-02T23:38:12.316 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-06-02T23:38:12.317 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-06-02T23:38:12.318 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.319 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-06-02T23:38:12.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.321 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.322 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.323 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.324 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.325 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.326 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.327 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.328 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.329 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.330 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.331 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.332 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.334 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.336 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.337 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.338 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.340 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.343 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-02T23:38:12.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-02T23:38:12.347 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-02T23:38:12.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-02T23:38:12.349 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-02T23:38:12.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-02T23:38:12.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-02T23:38:12.352 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-02T23:38:12.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-02T23:38:12.354 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-06-02T23:38:12.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-06-02T23:38:12.356 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-02T23:38:12.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.358 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.359 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.364 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.369 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.370 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.371 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.373 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.375 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.377 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.379 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.381 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.382 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.384 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.385 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.387 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.390 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.391 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.392 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.394 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.396 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.398 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.399 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.400 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.401 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.402 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.403 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.405 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.406 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.407 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.409 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.411 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-02T23:38:12.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.413 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.414 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-06-02T23:38:12.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.416 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.417 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.418 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.420 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.422 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.423 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.424 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-06-02T23:38:12.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.426 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.428 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.429 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.431 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.432 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.433 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-06-02T23:38:12.434 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-06-02T23:38:12.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-02T23:38:12.436 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-06-02T23:38:12.437 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-06-02T23:38:12.438 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-06-02T23:38:12.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.440 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.441 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.442 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.443 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.444 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.445 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.446 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.447 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.448 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.450 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.451 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.453 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.454 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.456 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.457 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.459 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-06-02T23:38:12.460 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.461 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.463 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.464 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.465 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.466 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.467 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.469 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-06-02T23:38:12.470 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.471 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.472 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.474 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.475 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.476 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.478 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.479 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.480 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.481 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.482 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.483 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.484 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.485 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.486 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.487 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.489 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.490 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.491 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.492 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.493 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.495 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.497 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.498 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.499 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.500 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.501 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.502 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.503 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.504 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.505 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.506 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.509 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.510 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-06-02T23:38:12.513 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.514 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.515 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.517 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.518 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.519 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.520 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.521 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.524 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.525 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.528 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.530 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.531 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.534 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.535 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-02T23:38:12.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-02T23:38:12.537 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.538 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.539 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-02T23:38:12.540 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.541 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.542 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.543 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.545 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.546 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.547 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.548 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.550 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.551 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.552 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.553 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.558 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.559 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.560 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.561 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.562 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.563 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.565 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.566 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.567 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.570 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.571 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.574 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.575 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.576 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.578 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.580 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.581 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-06-02T23:38:12.582 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-06-02T23:38:12.583 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-06-02T23:38:12.584 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-06-02T23:38:12.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-06-02T23:38:12.586 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-06-02T23:38:12.587 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-06-02T23:38:12.588 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.589 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.590 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.591 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.592 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.593 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.594 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.595 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.596 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.597 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.598 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.599 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.600 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.602 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.603 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.604 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.605 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.606 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.609 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.610 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.611 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.612 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.614 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.615 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.616 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.617 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.618 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.619 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.620 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.621 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.622 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.623 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.624 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.625 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.626 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.627 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.628 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.629 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.630 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.631 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.632 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.633 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.634 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.635 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.636 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-06-02T23:38:12.637 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.638 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.639 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.640 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.641 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.642 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.647 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.647 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.648 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.648 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.648 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.648 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.648 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.648 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.649 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.649 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.649 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-02T23:38:12.649 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.649 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.650 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.650 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.650 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.650 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.650 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.650 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.651 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.651 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.651 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.651 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.651 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.651 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.652 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.652 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.652 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.652 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.652 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.652 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.653 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.653 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.653 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.653 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.653 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.654 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.654 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.654 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.654 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.654 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.654 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.655 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.655 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.655 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.655 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.655 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.655 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.656 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.656 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.656 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.656 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.656 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.657 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.657 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.657 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.657 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.657 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.657 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.658 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.658 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.658 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.658 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.658 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.659 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.660 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.661 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.661 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.661 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.661 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.661 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.661 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.662 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.662 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-06-02T23:38:12.662 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-06-02T23:38:12.662 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.662 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.662 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.663 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-06-02T23:38:12.663 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.663 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.663 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.663 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-02T23:38:12.663 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.664 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.665 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.666 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.667 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.668 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.669 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.670 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.671 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.672 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.673 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.674 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.676 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.677 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.678 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.679 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "gid": 24110, 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "y", 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "mgr_features": 4540701547738038271, 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-06-02T23:38:12.680 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.681 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.682 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.683 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.684 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.685 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.686 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.687 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.688 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.689 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.690 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.691 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.692 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.693 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.694 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.695 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.696 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.697 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.698 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.699 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-06-02T23:38:12.700 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-06-02T23:38:12.701 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.702 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T23:38:12.703 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T23:38:12.704 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.705 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.706 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.707 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.708 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.709 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.710 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.711 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-06-02T23:38:12.712 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.713 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-06-02T23:38:12.714 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-06-02T23:38:12.715 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.717 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.718 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.720 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.721 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-06-02T23:38:12.722 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.723 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-06-02T23:38:12.724 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.725 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-06-02T23:38:12.726 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-06-02T23:38:12.727 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.728 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-06-02T23:38:12.729 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-06-02T23:38:12.730 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-06-02T23:38:12.731 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-06-02T23:38:12.732 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.733 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.734 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.735 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.736 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.737 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.738 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.739 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.740 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.741 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.742 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.743 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.744 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-02T23:38:12.745 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.746 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.747 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.748 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.749 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.750 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.751 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.752 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.753 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-06-02T23:38:12.754 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.755 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.756 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-06-02T23:38:12.757 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.758 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.759 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.760 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.761 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.762 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.763 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.764 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.765 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.766 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.767 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.768 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.769 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.770 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.771 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.772 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.773 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-02T23:38:12.774 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-02T23:38:12.775 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-06-02T23:38:12.776 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-02T23:38:12.777 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.778 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-02T23:38:12.779 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-06-02T23:38:12.780 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-02T23:38:12.781 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.782 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.783 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.784 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.785 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.786 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.787 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.788 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.789 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.790 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.791 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.792 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.793 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.794 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.795 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.796 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.797 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-02T23:38:12.798 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-02T23:38:12.799 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-02T23:38:12.800 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-02T23:38:12.801 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.802 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.803 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.804 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.805 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.806 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.807 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.808 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.809 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.810 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.811 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.812 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.813 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.814 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.815 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.816 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.817 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.818 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.819 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.820 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.821 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-02T23:38:12.822 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.823 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.823 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.823 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.823 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.823 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.823 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.824 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.824 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.824 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.824 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.824 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.824 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.825 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-02T23:38:12.826 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.827 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.828 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.829 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.830 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-06-02T23:38:12.831 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-02T23:38:12.832 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-02T23:38:12.833 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.834 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.835 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.836 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.837 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.838 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.839 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.840 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.841 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.842 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.843 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.844 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.845 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.846 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.847 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.847 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.847 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.847 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.847 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.847 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.848 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.849 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.850 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.851 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.852 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.853 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.853 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.853 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.853 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.853 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.853 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.854 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.855 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.856 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.857 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.858 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.859 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.860 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.861 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-02T23:38:12.862 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.863 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.864 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.865 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.866 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.867 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.868 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.869 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.870 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-02T23:38:12.871 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.872 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-06-02T23:38:12.873 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.874 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.875 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-02T23:38:12.876 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-06-02T23:38:12.877 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.878 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.879 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.880 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.881 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.882 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.883 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.884 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.885 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.886 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:12.887 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-06-02T23:38:12.888 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.889 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.890 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.891 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.892 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.893 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.894 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.895 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.896 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.897 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-06-02T23:38:12.898 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.899 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.900 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.901 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.902 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.903 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.904 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.905 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.906 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.907 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.908 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.909 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.910 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.911 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.912 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.913 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.914 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.915 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.916 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.917 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.918 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.919 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.920 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.921 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.922 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.923 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-06-02T23:38:12.924 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.925 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.926 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.927 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.928 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.929 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.930 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.931 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.932 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.933 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.934 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.935 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.936 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.937 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.938 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.939 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.940 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.941 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.942 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.943 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.944 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.945 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.946 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.947 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.948 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.949 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.950 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.951 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-06-02T23:38:12.952 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.953 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.954 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.955 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.956 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.957 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.958 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-06-02T23:38:12.959 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.960 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.961 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.962 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.963 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.964 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.965 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.966 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.967 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.968 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.969 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.970 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.971 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.972 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.973 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.974 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.975 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.976 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.977 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.978 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.979 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.980 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.981 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.982 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.983 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.984 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.985 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.986 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.987 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:12.988 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.989 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.990 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.991 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.992 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:12.993 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.994 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.995 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.996 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.997 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.998 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:12.999 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.000 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.001 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-06-02T23:38:13.002 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-02T23:38:13.003 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.004 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.005 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.006 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.007 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.008 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.009 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.010 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.011 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.012 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.013 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.014 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.015 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.016 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.017 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.018 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.019 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.020 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.021 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.022 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.023 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.024 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.026 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.027 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.028 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.029 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.030 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.031 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.032 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.033 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.034 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.035 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.036 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.037 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.038 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.039 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.040 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.041 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.042 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.043 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.044 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.045 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.046 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-06-02T23:38:13.047 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-06-02T23:38:13.048 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-06-02T23:38:13.049 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-06-02T23:38:13.050 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-06-02T23:38:13.051 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.052 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.053 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.054 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.055 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.056 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.057 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.058 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.059 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.060 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.061 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.062 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.063 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.064 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.065 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.066 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.067 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.068 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.069 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.070 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.071 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.072 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.073 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.074 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-06-02T23:38:13.075 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.076 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-06-02T23:38:13.077 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.078 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.079 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.080 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.081 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.082 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.083 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.084 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.085 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.086 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.087 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.088 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.089 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-06-02T23:38:13.090 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.091 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.092 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.093 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.094 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.095 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.096 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.097 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.098 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.099 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.100 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.101 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.102 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.103 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.104 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.105 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.106 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.107 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.108 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.109 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.110 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.111 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.112 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.113 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.114 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.115 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.116 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.117 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.118 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.119 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.120 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.121 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.122 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.123 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.124 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.125 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.126 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.127 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.128 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.129 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.130 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.131 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.132 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.133 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.134 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.135 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.136 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.137 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "modules": [ 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard", 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "iostat", 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "nfs", 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "restful" 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: "available_modules": [ 2024-06-02T23:38:13.138 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "alerts", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.139 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How frequently to reexamine health status", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.140 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.141 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.142 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.143 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.144 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_destination": { 2024-06-02T23:38:13.145 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_destination", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email address to send alerts to", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.146 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_from_name": { 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_from_name", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "Ceph", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Email From: name", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.147 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_host": { 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_host", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP server", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.148 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_password": { 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_password", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Password to authenticate with", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.149 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_port": { 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_port", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "465", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP port", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.150 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_sender": { 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_sender", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SMTP envelope sender", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.151 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_ssl": { 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_ssl", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use SSL to connect to SMTP server", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.152 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "smtp_user": { 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "smtp_user", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "User to authenticate as", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.153 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "balancer", 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "active": { 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "active", 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.154 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically balance PGs across cluster", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_time": { 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_time", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0000", 2024-06-02T23:38:13.155 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "beginning time of day to automatically balance", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "begin_weekday": { 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "begin_weekday", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.156 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_max_iterations": { 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_max_iterations", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.157 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "25", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "250", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_metrics": { 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_metrics", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.158 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "pgs,objects,bytes", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "crush_compat_step": { 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crush_compat_step", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.159 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.5", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0.001", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "0.999", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "aggressiveness of optimization", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "end_time": { 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_time", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.160 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2359", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ending time of day to automatically balance", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "end_weekday": { 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "end_weekday", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:13.161 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "0", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "6", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.162 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.163 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.164 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.165 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.166 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "min_score": { 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_score", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.167 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.168 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "upmap", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "crush-compat", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "none", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "read", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap-read" 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Balancer mode", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.169 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_ids": { 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_ids", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.170 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.171 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_deviation": { 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_deviation", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "deviation below which no optimization is attempted", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.172 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "upmap_max_optimizations": { 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "upmap_max_optimizations", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.173 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_down_multiplier": { 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_down_multiplier", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.174 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_refresh_rate": { 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_refresh_rate", 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.175 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "20", 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "agent_starting_port": { 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "agent_starting_port", 2024-06-02T23:38:13.176 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4721", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_ptrace": { 2024-06-02T23:38:13.177 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_ptrace", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.178 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_interval": { 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_interval", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to autotune daemon memory", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.179 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "autotune_memory_target_ratio": { 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "autotune_memory_target_ratio", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0.7", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.180 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm_log_destination": { 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cephadm_log_destination", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "file", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "file,syslog", 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: "syslog" 2024-06-02T23:38:13.181 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "cgroups_split": { 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cgroups_split", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.182 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "config_checks_enabled": { 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_checks_enabled", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.183 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "config_dashboard": { 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_dashboard", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.184 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_alertmanager": { 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_alertmanager", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.185 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_base": { 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_base", 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.186 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/ceph", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Container image name, without the tag", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_elasticsearch": { 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_elasticsearch", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.187 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "elasticsearch container image", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_grafana": { 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_grafana", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.188 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_haproxy": { 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_haproxy", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.189 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "HAproxy container image", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_agent": { 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_agent", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.190 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger agent container image", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_collector": { 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_collector", 2024-06-02T23:38:13.191 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger collector container image", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_jaeger_query": { 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_jaeger_query", 2024-06-02T23:38:13.192 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Jaeger query container image", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_keepalived": { 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_keepalived", 2024-06-02T23:38:13.193 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Keepalived container image", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_loki": { 2024-06-02T23:38:13.194 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_loki", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Loki container image", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_node_exporter": { 2024-06-02T23:38:13.195 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_node_exporter", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_nvmeof": { 2024-06-02T23:38:13.196 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_nvmeof", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Nvme-of container image", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.197 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_prometheus": { 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_prometheus", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Prometheus container image", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.198 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_promtail": { 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_promtail", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Promtail container image", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.199 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_samba": { 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_samba", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Samba/SMB container image", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.200 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "container_image_snmp_gateway": { 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_image_snmp_gateway", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "SNMP Gateway container image", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.201 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "container_init": { 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "container_init", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Run podman/docker with `--init`", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.202 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "daemon_cache_timeout": { 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "daemon_cache_timeout", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.203 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_cephadm_command_timeout": { 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_cephadm_command_timeout", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "900", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-06-02T23:38:13.204 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "default_registry": { 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "default_registry", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "docker.io", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-06-02T23:38:13.205 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "device_cache_timeout": { 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_cache_timeout", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1800", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.206 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache device inventory", 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "device_enhanced_scan": { 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_enhanced_scan", 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.207 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use libstoragemgmt during device scans", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "facts_cache_timeout": { 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "facts_cache_timeout", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.208 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "seconds to cache host facts data", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "grafana_dashboards_path": { 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "grafana_dashboards_path", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.209 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.210 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "host_check_interval": { 2024-06-02T23:38:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "host_check_interval", 2024-06-02T23:38:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.211 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to perform a host check", 2024-06-02T23:38:13.212 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "hw_monitoring": { 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hw_monitoring", 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.213 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-06-02T23:38:13.214 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "inventory_list_all": { 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "inventory_list_all", 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.215 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.216 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.217 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.218 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "log_refresh_metadata": { 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_refresh_metadata", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.219 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.220 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.221 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.222 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.223 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf": { 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.224 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "*", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.225 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "max_count_per_host": { 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_count_per_host", 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.226 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of daemons per service per host", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "max_osd_draining_count": { 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_osd_draining_count", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.227 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "migration_current": { 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "migration_current", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.228 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "internal - do not modify", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "mode": { 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mode", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "root", 2024-06-02T23:38:13.229 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm-package", 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "root" 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "mode for remote execution of cephadm", 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "oob_default_addr": { 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "oob_default_addr", 2024-06-02T23:38:13.230 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "169.254.1.1", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Default address for RedFish API (oob management).", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "prometheus_alerts_path": { 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus_alerts_path", 2024-06-02T23:38:13.231 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_insecure": { 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_insecure", 2024-06-02T23:38:13.232 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_password": { 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_password", 2024-06-02T23:38:13.233 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.234 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_url": { 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_url", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.235 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "registry_username": { 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "registry_username", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.236 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "secure_monitoring_stack": { 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "secure_monitoring_stack", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.237 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "service_discovery_port": { 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "service_discovery_port", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8765", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "cephadm service discovery port", 2024-06-02T23:38:13.238 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "ssh_config_file": { 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssh_config_file", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-06-02T23:38:13.239 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "use_agent": { 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_agent", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-06-02T23:38:13.240 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "use_repo_digest": { 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "use_repo_digest", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.241 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_failed_host_check": { 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_failed_host_check", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.242 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if the host check fails", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_daemons": { 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_daemons", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.243 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_on_stray_hosts": { 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_on_stray_hosts", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.244 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crash", 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.245 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.246 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.247 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.248 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.249 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.250 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "retain_interval": { 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retain_interval", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.251 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "31536000", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain crashes before pruning them", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_recent_interval": { 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_recent_interval", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.252 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1209600", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "time interval in which to warn about recent crashes", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dashboard", 2024-06-02T23:38:13.253 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.254 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_HOST": { 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_HOST", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.255 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.256 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_ENABLED": { 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_ENABLED", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.257 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.258 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "ENABLE_BROWSABLE_API": { 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ENABLE_BROWSABLE_API", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.259 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.260 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.261 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.262 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.263 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_NFS": { 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.264 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RBD": { 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.265 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "FEATURE_TOGGLE_RGW": { 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.266 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-06-02T23:38:13.267 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.268 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_PASSWORD": { 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_PASSWORD", 2024-06-02T23:38:13.269 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.270 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.271 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.272 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.273 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_URL": { 2024-06-02T23:38:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_URL", 2024-06-02T23:38:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.274 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.275 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_API_USERNAME": { 2024-06-02T23:38:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_API_USERNAME", 2024-06-02T23:38:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.276 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.277 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-06-02T23:38:13.278 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-06-02T23:38:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.279 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.280 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-06-02T23:38:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-06-02T23:38:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.281 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.282 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-06-02T23:38:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-06-02T23:38:13.283 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.284 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.285 INFO:teuthology.orchestra.run.smithi052.stdout: "ISSUE_TRACKER_API_KEY": { 2024-06-02T23:38:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-06-02T23:38:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.286 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.287 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_HOST": { 2024-06-02T23:38:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_HOST", 2024-06-02T23:38:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.288 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.289 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-06-02T23:38:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-06-02T23:38:13.290 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.291 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.292 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-06-02T23:38:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-06-02T23:38:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.293 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.294 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-06-02T23:38:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-06-02T23:38:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.295 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.296 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.297 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-06-02T23:38:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-06-02T23:38:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.298 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.299 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-06-02T23:38:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-06-02T23:38:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.300 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.301 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-06-02T23:38:13.302 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-06-02T23:38:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.303 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.304 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-06-02T23:38:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-06-02T23:38:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.305 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.306 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-06-02T23:38:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-06-02T23:38:13.307 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.308 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.309 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_ENABLED": { 2024-06-02T23:38:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_ENABLED", 2024-06-02T23:38:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.310 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.311 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-06-02T23:38:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-06-02T23:38:13.312 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-06-02T23:38:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.313 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.314 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-06-02T23:38:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-06-02T23:38:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.315 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.316 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-06-02T23:38:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-06-02T23:38:13.317 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8", 2024-06-02T23:38:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.318 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.319 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "REST_REQUESTS_TIMEOUT": { 2024-06-02T23:38:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-06-02T23:38:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.320 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "45", 2024-06-02T23:38:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.321 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ACCESS_KEY": { 2024-06-02T23:38:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ACCESS_KEY", 2024-06-02T23:38:13.322 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.323 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.324 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-06-02T23:38:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-06-02T23:38:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.325 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "admin", 2024-06-02T23:38:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.326 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SECRET_KEY": { 2024-06-02T23:38:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SECRET_KEY", 2024-06-02T23:38:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.327 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.328 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "RGW_API_SSL_VERIFY": { 2024-06-02T23:38:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "RGW_API_SSL_VERIFY", 2024-06-02T23:38:13.329 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.330 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-06-02T23:38:13.331 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-06-02T23:38:13.332 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-06-02T23:38:13.333 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "cross_origin_url": { 2024-06-02T23:38:13.334 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cross_origin_url", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "crt_file": { 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "crt_file", 2024-06-02T23:38:13.335 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "debug": { 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "debug", 2024-06-02T23:38:13.336 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Enable/disable debug options", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "jwt_token_ttl": { 2024-06-02T23:38:13.337 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "jwt_token_ttl", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "28800", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-02T23:38:13.338 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.339 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.340 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.341 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.342 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "motd": { 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "motd", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "The message of the day", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect_resolve_ip_addr": { 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "redirect_resolve_ip_addr", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.346 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.347 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8080", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.348 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.349 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl_server_port": { 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl_server_port", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8443", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.350 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "redirect", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.351 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "redirect" 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-02T23:38:13.352 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "url_prefix": { 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url_prefix", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.353 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-02T23:38:13.354 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_monitoring": { 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_monitoring", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "monitor device health metrics", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.355 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.356 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.357 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.358 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.359 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.360 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "mark_out_threshold": { 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mark_out_threshold", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "2419200", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-06-02T23:38:13.361 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "pool_name": { 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pool_name", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "device_health_metrics", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name of pool in which to store device health metrics", 2024-06-02T23:38:13.362 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "retention_period": { 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "retention_period", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15552000", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.363 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long to retain device health metrics", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_frequency": { 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_frequency", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to scrape device health metrics", 2024-06-02T23:38:13.364 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "self_heal": { 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "self_heal", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-06-02T23:38:13.365 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how frequently to wake up and check device health", 2024-06-02T23:38:13.366 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "warn_threshold": { 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "warn_threshold", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7257600", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-06-02T23:38:13.367 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "diskprediction_local", 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.368 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.369 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.370 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.371 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.372 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "predict_interval": { 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predict_interval", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.373 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "86400", 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.374 INFO:teuthology.orchestra.run.smithi052.stdout: "predictor_model": { 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "predictor_model", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "prophetstor", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.375 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "600", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.376 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "influx", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": false, 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "influxdb python module not found", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "batch_size": { 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "batch_size", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5000", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.377 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "database": { 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "database", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "ceph", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.378 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "hostname": { 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "hostname", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.379 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server hostname", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "30", 2024-06-02T23:38:13.380 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "5", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.381 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.382 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.383 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.384 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.385 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "password": { 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "password", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.386 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "password of InfluxDB server user", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "port": { 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "port", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "8086", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.387 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "InfluxDB server port", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "ssl": { 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ssl", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "false", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.388 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "threads": { 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threads", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "32", 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.389 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "username": { 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "username", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.390 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "username of InfluxDB server user", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "verify_ssl": { 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "verify_ssl", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "true", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.391 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "insights", 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.392 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.393 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.394 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.395 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.396 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.397 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "iostat", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.398 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.399 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.400 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.401 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.402 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "k8sevents", 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.403 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_event_retention_days": { 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "ceph_event_retention_days", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "7", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Days to hold ceph event information within local cache", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "config_check_secs": { 2024-06-02T23:38:13.404 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "config_check_secs", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "10", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.405 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.406 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.407 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.408 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.409 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.410 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "localpool", 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "failure_domain": { 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "failure_domain", 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "host", 2024-06-02T23:38:13.411 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "failure domain for any created local pool", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.412 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.413 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.414 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.415 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.416 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "min_size": { 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "min_size", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.417 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default min_size for any created local pool", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "num_rep": { 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "num_rep", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3", 2024-06-02T23:38:13.418 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default replica count for any created local pool", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_num": { 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_num", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "128", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.419 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "default pg_num for any created local pool", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "prefix": { 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prefix", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.420 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "name prefix for any created local pool", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "subtree": { 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "subtree", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "rack", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.421 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "CRUSH level for which to create a local pool", 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mds_autoscaler", 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.422 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.423 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.424 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.425 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.426 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.427 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirroring", 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.428 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.429 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.430 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.431 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.432 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "nfs", 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.433 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.434 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.435 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.436 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.437 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.438 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "fail_fs": { 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "fail_fs", 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.439 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.440 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.441 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.442 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.443 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.444 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator": { 2024-06-02T23:38:13.445 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "orchestrator", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "cephadm", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "rook", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "test_orchestrator" 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Orchestrator backend", 2024-06-02T23:38:13.446 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_perf_query", 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.447 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.448 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.449 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.450 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.451 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.452 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "osd_support", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.453 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.454 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.455 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.456 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.457 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "pg_autoscaler", 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.458 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.459 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.460 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.461 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.462 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:13.463 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "threshold": { 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "threshold", 2024-06-02T23:38:13.464 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "3.0", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1.0", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "scaling threshold", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.465 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "progress", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_pg_recovery_event": { 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_pg_recovery_event", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.466 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow the module to show pg recovery progress", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.467 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.468 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.469 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.470 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.471 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.472 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "max_completed_events": { 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_completed_events", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "50", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.473 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "number of past completed events to remember", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "sleep_interval": { 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "sleep_interval", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "5", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.474 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "how long the module is going to sleep", 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "prometheus", 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "cache": { 2024-06-02T23:38:13.475 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "cache", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "exclude_perf_counters": { 2024-06-02T23:38:13.476 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "exclude_perf_counters", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.477 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.478 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.479 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.480 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.481 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.482 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools": { 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_stats_pools_refresh_interval": { 2024-06-02T23:38:13.483 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "300", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "scrape_interval": { 2024-06-02T23:38:13.484 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "scrape_interval", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15.0", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:13.485 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "::", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:13.486 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "9283", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "stale_cache_strategy": { 2024-06-02T23:38:13.487 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stale_cache_strategy", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "log", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_behaviour": { 2024-06-02T23:38:13.488 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_behaviour", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "default", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "default", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "error" 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.489 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "standby_error_status_code": { 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "standby_error_status_code", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "500", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "400", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "599", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.490 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.491 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.492 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.494 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.495 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_snap_create": { 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_snap_create", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.496 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "mirror_snapshot_schedule": { 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "mirror_snapshot_schedule", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.497 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "trash_purge_schedule": { 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "trash_purge_schedule", 2024-06-02T23:38:13.498 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.499 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "restful", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "enable_auth": { 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enable_auth", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.500 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "key_file": { 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "key_file", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.501 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.502 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.503 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.504 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.505 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.506 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "server_addr": { 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_addr", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.507 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "server_port": { 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "server_port", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.508 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rgw", 2024-06-02T23:38:13.509 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.510 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.511 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.512 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.513 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.514 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rook", 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.515 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.516 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.517 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.518 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.519 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "storage_class": { 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "storage_class", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "local", 2024-06-02T23:38:13.520 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "selftest", 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.521 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.522 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.523 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.524 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.525 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.526 INFO:teuthology.orchestra.run.smithi052.stdout: "roption1": { 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption1", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.527 INFO:teuthology.orchestra.run.smithi052.stdout: "roption2": { 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "roption2", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "xyz", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.528 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption1": { 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption1", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption2": { 2024-06-02T23:38:13.529 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption2", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption3": { 2024-06-02T23:38:13.530 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption3", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "float", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption4": { 2024-06-02T23:38:13.531 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption4", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.532 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption5": { 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption5", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.533 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption6": { 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption6", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.534 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "rwoption7": { 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rwoption7", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "1", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "42", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.535 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "testkey": { 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testkey", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.536 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "testlkey": { 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testlkey", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.537 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "testnewline": { 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "testnewline", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.538 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snap_schedule", 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "allow_m_granularity": { 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "allow_m_granularity", 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.539 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "allow minute scheduled snapshots", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "dump_on_update": { 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "dump_on_update", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.540 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "dump database to debug log on update", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.541 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.542 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.543 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.544 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.545 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.546 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "stats", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.547 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.548 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.549 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.550 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.551 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "status", 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.552 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.553 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.554 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.555 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.556 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.557 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telegraf", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "address": { 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "address", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.558 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "15", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.559 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.560 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.561 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.562 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.563 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.564 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "telemetry", 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.565 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_basic": { 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_basic", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share basic cluster information (size, version)", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.566 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_crash": { 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_crash", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.567 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_device": { 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_device", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-06-02T23:38:13.568 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_ident": { 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_ident", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-06-02T23:38:13.569 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "channel_perf": { 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "channel_perf", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.570 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Share various performance metrics of a cluster", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "contact": { 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "contact", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.571 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "description": { 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "description", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.572 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "device_url": { 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "device_url", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.573 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "enabled": { 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "enabled", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.574 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "24", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "8", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.575 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "last_opt_revision": { 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "last_opt_revision", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "1", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.576 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard": { 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.577 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "leaderboard_description": { 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "leaderboard_description", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.578 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.579 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.580 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.581 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.582 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.583 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "organization": { 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "organization", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.584 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "proxy": { 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "proxy", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.585 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "url": { 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "url", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.586 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "test_orchestrator", 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.587 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.588 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.589 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.590 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.591 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.592 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.593 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.594 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.595 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.596 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.597 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max_concurrent_clones": { 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "max_concurrent_clones", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "4", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Number of asynchronous cloner threads", 2024-06-02T23:38:13.598 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "periodic_async_work": { 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "periodic_async_work", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Periodically check for async work", 2024-06-02T23:38:13.599 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_delay": { 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_delay", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "0", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-06-02T23:38:13.600 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "snapshot_clone_no_wait": { 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "snapshot_clone_no_wait", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "True", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-06-02T23:38:13.601 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix", 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "can_run": true, 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "error_string": "", 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "module_options": { 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "discovery_interval": { 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "discovery_interval", 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "uint", 2024-06-02T23:38:13.602 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "100", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "identifier": { 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "identifier", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.603 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "interval": { 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "interval", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "secs", 2024-06-02T23:38:13.604 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "60", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "log_level": { 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_level", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.605 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.606 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster": { 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.607 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_cluster_level": { 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_cluster_level", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "info", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [ 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "", 2024-06-02T23:38:13.608 INFO:teuthology.orchestra.run.smithi052.stdout: "critical", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "debug", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "error", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "info", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "warning" 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "log_to_file": { 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "log_to_file", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "bool", 2024-06-02T23:38:13.609 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 1, 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "False", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_host": { 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_host", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.610 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_port": { 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_port", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "int", 2024-06-02T23:38:13.611 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "10051", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "zabbix_sender": { 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "zabbix_sender", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "str", 2024-06-02T23:38:13.612 INFO:teuthology.orchestra.run.smithi052.stdout: "level": "advanced", 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "flags": 0, 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "min": "", 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "max": "", 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "enum_allowed": [], 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "desc": "", 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "long_desc": "", 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "tags": [], 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "see_also": [] 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.613 INFO:teuthology.orchestra.run.smithi052.stdout: "services": { 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "dashboard": "https://172.21.15.52:8443/" 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "always_on_modules": { 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "octopus": [ 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-02T23:38:13.614 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "pacific": [ 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "quincy": [ 2024-06-02T23:38:13.615 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "reef": [ 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-02T23:38:13.616 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: ], 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "squid": [ 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "balancer", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "crash", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "devicehealth", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "orchestrator", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "pg_autoscaler", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "progress", 2024-06-02T23:38:13.617 INFO:teuthology.orchestra.run.smithi052.stdout: "rbd_support", 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "status", 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "telemetry", 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "volumes" 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "last_failure_osd_epoch": 3, 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "active_clients": [ 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "devicehealth", 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-02T23:38:13.618 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 3561073402 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "libcephsqlite", 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 1949636686 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.619 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "rbd_support", 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 392979611 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "name": "volumes", 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "addrvec": [ 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: { 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "type": "v2", 2024-06-02T23:38:13.620 INFO:teuthology.orchestra.run.smithi052.stdout: "addr": "172.21.15.52:0", 2024-06-02T23:38:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: "nonce": 4213821482 2024-06-02T23:38:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:13.621 INFO:teuthology.orchestra.run.smithi052.stdout: ] 2024-06-02T23:38:13.621 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:38:13.622 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3923252252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T23:38:13.622 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2363132248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T23:38:13.622 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/663940625' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-02T23:38:13.622 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3923252252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T23:38:13.622 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2363132248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T23:38:13.623 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/663940625' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-02T23:38:13.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0240a2b90 msgr2=0x7ff0240a5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:13.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0240a2b90 0x7ff0240a5050 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ff028005fd0 tx=0x7ff028005850 comp rx=0 tx=0).stop 2024-06-02T23:38:13.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 msgr2=0x7ff038199e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:13.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 0x7ff038199e50 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7ff03400d770 tx=0x7ff03400dc30 comp rx=0 tx=0).stop 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 shutdown_connections 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7ff0240a2b90 0x7ff0240a5050 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff038102c00 0x7ff03819a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0381035c0 0x7ff0381a13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 --2- 172.21.15.52:0/663940625 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7ff038101a10 0x7ff038199e50 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.223+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 >> 172.21.15.52:0/663940625 conn(0x7ff0380fcfa0 msgr2=0x7ff038073bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.224+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 shutdown_connections 2024-06-02T23:38:13.624 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:12.224+0000 7ff03f509640 1 -- 172.21.15.52:0/663940625 wait complete. 2024-06-02T23:38:13.630 DEBUG:tasks.check_counter:Getting stats from a 2024-06-02T23:38:13.630 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.a perf dump 2024-06-02T23:38:13.631 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3923252252' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-06-02T23:38:13.631 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2363132248' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-06-02T23:38:13.631 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/663940625' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-06-02T23:38:13.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:13 smithi123 ceph-mon[35977]: pgmap v1221: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:13.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:13 smithi174 ceph-mon[35994]: pgmap v1221: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:13.894 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:13.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:13 smithi052 ceph-mon[32340]: pgmap v1221: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:15.617 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:15 smithi052 ceph-mon[32340]: pgmap v1222: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:15.666 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:38:15.666 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:15.666 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 3564, 2024-06-02T23:38:15.666 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 3566, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 38952227, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 1133353, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 8, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.330503343, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.093898188, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.110430481, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.100005147, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3564, 2024-06-02T23:38:15.667 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.103389413, 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000029009 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:15.668 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2145, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2118, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 55661314, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 821154, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 6, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 4, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.225516068, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.061086280, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.101821517, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.049507486, 2024-06-02T23:38:15.669 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2118, 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.061451563, 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000029013 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.670 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 725136, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 306880 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 1008, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 1383, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 25418468, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 1237686, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 9, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.122164733, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.040449599, 2024-06-02T23:38:15.671 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.043222809, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.029013962, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 1381, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.040050794, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000029001 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.672 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 918096 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-02T23:38:15.673 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 10462, 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.264098934, 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000025243 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-02T23:38:15.674 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.675 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 10792, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 0, 2024-06-02T23:38:15.676 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 10792, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 10792, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 1, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-06-02T23:38:15.677 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:15.678 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:15.679 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-06-02T23:38:15.682 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-06-02T23:38:15.683 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-06-02T23:38:15.684 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-06-02T23:38:15.685 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 74267, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-06-02T23:38:15.686 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 93, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 8, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 121782320, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 121781182, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.687 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 74267 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 13, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 10832, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 10819, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 11, 2024-06-02T23:38:15.688 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 67, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 56, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 10792, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 11252, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 460, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 63948, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215420 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-06-02T23:38:15.689 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.690 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.691 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.692 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-06-02T23:38:15.693 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.694 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.694 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.694 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.694 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.695 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.696 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.697 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.698 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.699 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.700 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.701 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 1, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 2, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 1, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 1, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 2781, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.702 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:15.703 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:15.704 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 233767, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 81, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-02T23:38:15.705 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 157, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 7640952, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 43296, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:15.706 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 6186, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 6186, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 6186, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 4.138949235, 2024-06-02T23:38:15.707 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000669083 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 6186, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 6187 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 6186, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 6186, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 3516, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 2668, 2024-06-02T23:38:15.708 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:15.709 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-06-02T23:38:15.710 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-06-02T23:38:15.711 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 90, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 2, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 10, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 12, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 2, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:15.712 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-06-02T23:38:15.713 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 6713, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 119397910, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 6713, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:15.714 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 6713, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 119397910, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:15.715 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 6186, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 117588021, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 6186, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 117588021, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:15.716 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 6186, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 6186, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 6186, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 6186, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.717 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-02T23:38:15.718 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x55c6386b66e0": { 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:15.719 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:15.720 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:38:15.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:15 smithi123 ceph-mon[35977]: pgmap v1222: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:15.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:15 smithi174 ceph-mon[35994]: pgmap v1222: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:16.503 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-06-02T23:38:16.503 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-06-02T23:38:16.503 DEBUG:tasks.check_counter:Getting stats from d 2024-06-02T23:38:16.504 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.d perf dump 2024-06-02T23:38:16.756 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:17.572 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:17 smithi052 ceph-mon[32340]: pgmap v1223: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:17.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:17 smithi123 ceph-mon[35977]: pgmap v1223: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:17.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:17 smithi174 ceph-mon[35994]: pgmap v1223: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:18.491 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:38:18.491 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:18.491 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-06-02T23:38:18.491 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-06-02T23:38:18.491 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-06-02T23:38:18.491 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001350175, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000290983, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:18.492 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 7, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 385, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 3424, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 301744, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-06-02T23:38:18.493 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.026462816, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.020811883, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002291180, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 384, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.016176612, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000042126 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:18.494 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 301744 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 502, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 499, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 559360, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 309248, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.062345193, 2024-06-02T23:38:18.495 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.027075845, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.021763407, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.009705420, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 498, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.016777965, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000033690 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.496 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 559360, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 309248 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:18.497 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:18.498 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-02T23:38:18.499 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 147920, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 27, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:18.500 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:18.501 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 505, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 488378, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 505, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:18.502 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 505, 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 488378, 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:18.503 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:38:19.210 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-06-02T23:38:19.211 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-06-02T23:38:19.211 DEBUG:tasks.check_counter:Getting stats from g 2024-06-02T23:38:19.211 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.g perf dump 2024-06-02T23:38:19.463 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:19.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:19 smithi123 ceph-mon[35977]: pgmap v1224: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:19.791 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:19 smithi052 ceph-mon[32340]: pgmap v1224: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:19.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:19 smithi174 ceph-mon[35994]: pgmap v1224: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:21.324 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2799, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2774, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 22419916, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 1030690, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 3, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.293754423, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.110951966, 2024-06-02T23:38:21.325 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.088362681, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.072970671, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 2773, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.095142728, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000034310 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 723344, 2024-06-02T23:38:21.326 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 308512 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 2694, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 3069, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 30698342, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 1687817, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.210191578, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.066400141, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.067723538, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.060834263, 2024-06-02T23:38:21.327 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 3068, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.081019772, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000026408 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 832976 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:21.328 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 904, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 906, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 37977637, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 287484, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 8, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.098911240, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.018683845, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.050852353, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.023658561, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 904, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.026178127, 2024-06-02T23:38:21.329 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000028958 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 1, 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-02T23:38:21.330 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-MDSRank": { 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 9981, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.231463808, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000023190 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "queue_len": 0, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "complete_latency": { 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.331 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "mds": { 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "request": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "reply": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "reply_latency": { 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_reply": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "forward": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:21.332 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_commit": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_split": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_merge": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes": 7278, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_top": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_bottom": 7278, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pin_tail": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_pinned": 7278, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_expired": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "inodes_with_caps": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "caps": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "subtrees": 1, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse": 0, 2024-06-02T23:38:21.333 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_hit": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_forward": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_discover": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "traverse_lock": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "load_cent": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "q": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "exported": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "exported_inodes": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "imported": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "imported_inodes": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_backtrace_fetch": 0, 2024-06-02T23:38:21.334 INFO:teuthology.orchestra.run.smithi052.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rfiles": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rbytes": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "root_rsnaps": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_inodes": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "scrub_file_inodes": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T23:38:21.335 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_grant": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_caps_dirty": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_cap_release": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "process_request_cap_release": 0 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_cache": { 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_update_receipt": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_try_discover": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_send_discover": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "dir_handle_discover": 0, 2024-06-02T23:38:21.336 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_created": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_enqueued": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "strays_migrated": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_started": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "recovery_completed": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_path": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_quiesce_inode": 0, 2024-06-02T23:38:21.337 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_exportdir": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_flush": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ireq_inodestats": 0 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_log": { 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "evlrg": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "evadd": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "evex": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "evtrm": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "ev": 47805, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "evexg": 0, 2024-06-02T23:38:21.338 INFO:teuthology.orchestra.run.smithi052.stdout: "evexd": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "segadd": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "segex": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "segtrm": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "seg": 67, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "segmjr": 5, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "segexg": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "segexd": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "expos": 4194304, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "wrpos": 92923767, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "rdpos": 92922629, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "jlat": { 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.339 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "replayed": 47805 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_mem": { 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "ino": 13, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "ino+": 7297, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "ino-": 7284, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "dir": 11, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "dir+": 48, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "dir-": 37, 2024-06-02T23:38:21.340 INFO:teuthology.orchestra.run.smithi052.stdout: "dn": 7278, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "dn+": 7651, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "dn-": 373, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "cap": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "cap+": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "cap-": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "rss": 59408, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "heap": 215420 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_server": { 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_client_request": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "dispatch_server_request": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_request": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_client_session": 0, 2024-06-02T23:38:21.341 INFO:teuthology.orchestra.run.smithi052.stdout: "handle_peer_request": 0, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "req_create_latency": { 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getattr_latency": { 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.342 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "req_link_latency": { 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookup_latency": { 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.343 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupino_latency": { 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupname_latency": { 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.344 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "req_lssnap_latency": { 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.345 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.346 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.346 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mkdir_latency": { 2024-06-02T23:38:21.346 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.346 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.346 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.347 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.347 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mknod_latency": { 2024-06-02T23:38:21.347 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.347 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "req_mksnap_latency": { 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "req_open_latency": { 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.348 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "req_readdir_latency": { 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rename_latency": { 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.349 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmdir_latency": { 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:21.350 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setattr_latency": { 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.351 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setlayout_latency": { 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.352 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "req_setxattr_latency": { 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "req_symlink_latency": { 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "req_unlink_latency": { 2024-06-02T23:38:21.353 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_sessions": { 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "session_count": 1, 2024-06-02T23:38:21.354 INFO:teuthology.orchestra.run.smithi052.stdout: "session_add": 1, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "session_remove": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_open": 1, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "sessions_stale": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "total_load": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "average_load": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "avg_session_uptime": 2786, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:21.355 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:21.356 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:21.357 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 232903, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 77, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:21.358 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 157, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 5167096, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 29240, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "objecter": { 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "op_active": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "op_laggy": 0, 2024-06-02T23:38:21.359 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send": 5863, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "op_send_bytes": 21, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "op_resend": 3, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "op_reply": 5863, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "op_latency": { 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 5863, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 3.235960324, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000551929 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "op_inflight": 0, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "oplen_avg": { 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 5863, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 5864 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.360 INFO:teuthology.orchestra.run.smithi052.stdout: "op": 5863, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "op_r": 5863, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "op_w": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "op_rmw": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "op_pg": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_stat": 3460, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_create": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_read": 2401, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_write": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writefull": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_writesame": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_append": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_zero": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_truncate": 0, 2024-06-02T23:38:21.361 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_delete": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_mapext": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_getxattr": 1, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_setxattr": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_call": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_watch": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_notify": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls": 0, 2024-06-02T23:38:21.362 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "osdop_other": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_active": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_send": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_resend": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "linger_ping": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_active": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_send": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "poolop_resend": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_active": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_send": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "poolstat_resend": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_active": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_send": 0, 2024-06-02T23:38:21.363 INFO:teuthology.orchestra.run.smithi052.stdout: "statfs_resend": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "command_active": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "command_send": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "command_resend": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "map_epoch": 90, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "map_full": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "map_inc": 2, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_sessions": 10, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_open": 12, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_session_close": 2, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_laggy": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_wr": 0, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_rd": 2, 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_del": 0 2024-06-02T23:38:21.364 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "oft": { 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_objs": 0, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_updates": 0, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "omap_total_removes": 0 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "osd-slow-ops": { 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "slow_ops_count": 0 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "purge_queue": { 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing": 0, 2024-06-02T23:38:21.365 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executing_high_water": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed_ops": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_executed": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 6393, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 90490431, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 6393, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:21.366 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 6393, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 90490431, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-02T23:38:21.367 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:21.368 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:21.368 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:21.368 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:21.368 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:21.368 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 5863, 2024-06-02T23:38:21.368 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 88730539, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 5863, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 88730539, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 1024, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:21.369 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 5863, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 5863, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 5863, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 5863, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.370 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.371 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-write_buf_throttle-0x55ffc79d06e0": { 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 3758096384, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:21.372 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 0, 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 0, 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:21.373 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:38:21.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:21 smithi123 ceph-mon[35977]: pgmap v1225: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:21.787 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:21 smithi052 ceph-mon[32340]: pgmap v1225: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:21.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:21 smithi174 ceph-mon[35994]: pgmap v1225: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:22.165 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-06-02T23:38:22.165 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-06-02T23:38:22.165 DEBUG:tasks.check_counter:Getting stats from j 2024-06-02T23:38:22.166 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.j perf dump 2024-06-02T23:38:22.419 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:23.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:23 smithi123 ceph-mon[35977]: pgmap v1226: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:23.914 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:23 smithi174 ceph-mon[35994]: pgmap v1226: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:23.915 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:23 smithi052 ceph-mon[32340]: pgmap v1226: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:24.260 INFO:teuthology.orchestra.run.smithi052.stdout:{ 2024-06-02T23:38:24.260 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:24.260 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 0, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 2, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 0, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 0, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 4, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 0, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.001488939, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.000000000, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.000336929, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:24.261 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:24.262 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 509, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 504, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 563120, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 311616, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 2, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 1, 2024-06-02T23:38:24.263 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.062954465, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.027124315, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.021979651, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.009822407, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 504, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.018883003, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000037466 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:24.264 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 563120, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 311616 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_messages": 3, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages": 385, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_bytes": 480, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_bytes": 301648, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_created_connections": 5, 2024-06-02T23:38:24.265 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_active_connections": 2, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_total_time": 0.026107317, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_send_time": 0.020651133, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_recv_time": 0.002056152, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 383, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.015746538, 2024-06-02T23:38:24.266 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000041113 2024-06-02T23:38:24.270 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: "msgr_send_encrypted_bytes": 301648 2024-06-02T23:38:24.271 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "cct": { 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "total_workers": 0, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "unhealthy_workers": 0 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "mempool": { 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:24.272 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:24.273 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:24.274 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_items": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_bytes": 147937, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_anon_items": 28, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_bytes": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_items": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:24.275 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_bytes": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_items": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "pgmap_items": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_bytes": 928, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "mds_co_items": 2, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_1_items": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:24.276 INFO:teuthology.orchestra.run.smithi052.stdout: "unittest_2_items": 0 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: }, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "val": 0, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "max": 104857600, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "get_started": 0, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "get": 508, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "get_sum": 488768, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "get_or_fail_success": 508, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "take": 0, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "take_sum": 0, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "put": 508, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "put_sum": 488768, 2024-06-02T23:38:24.277 INFO:teuthology.orchestra.run.smithi052.stdout: "wait": { 2024-06-02T23:38:24.278 INFO:teuthology.orchestra.run.smithi052.stdout: "avgcount": 0, 2024-06-02T23:38:24.278 INFO:teuthology.orchestra.run.smithi052.stdout: "sum": 0.000000000, 2024-06-02T23:38:24.278 INFO:teuthology.orchestra.run.smithi052.stdout: "avgtime": 0.000000000 2024-06-02T23:38:24.278 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:24.278 INFO:teuthology.orchestra.run.smithi052.stdout: } 2024-06-02T23:38:24.278 INFO:teuthology.orchestra.run.smithi052.stdout:} 2024-06-02T23:38:25.074 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-06-02T23:38:25.074 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-06-02T23:38:25.074 DEBUG:tasks.check_counter:Getting stats from b 2024-06-02T23:38:25.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.b perf dump 2024-06-02T23:38:25.386 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:38:25.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:25 smithi123 ceph-mon[35977]: pgmap v1227: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:25.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:25 smithi174 ceph-mon[35994]: pgmap v1227: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:25.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:25 smithi052 ceph-mon[32340]: pgmap v1227: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 105788, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 126991, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 25244230, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 81639496, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 12, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 6.792876104, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 3.092477728, 2024-06-02T23:38:27.248 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 3.801283311, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.490051249, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 126992, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.127690548, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000032503 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 197166, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.030081193, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000152 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:27.249 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 83858, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 97262, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 20356099, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 71876908, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 11, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 5.019292405, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 1.890503243, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 2.908939796, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.673592148, 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:27.250 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 97261, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.969924491, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000051098 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 141129, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.022450814, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000159 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 849040 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 958392, 2024-06-02T23:38:27.251 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 970716, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 280224463, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 574652393, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 529, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 9, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 39.194833772, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 16.116618184, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 28.355534088, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.513834170, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 970712, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 14.733708989, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000015178 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.252 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1891969, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.237116839, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000125 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 823968, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 384816 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:27.253 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 70369, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 39.511450465, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000561489 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7977, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.645543962, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000080925 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.254 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 856140, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 829342, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 829342, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 613.520893231, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000739768 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 26798, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 7, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 2, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 1898, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-02T23:38:27.255 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 139, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 37, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 91, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 11, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 38, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 1354, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 2, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 2, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 7, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 1673244, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 1657141, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 10930, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 40, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:27.256 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 2, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 29, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 1070, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 26, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 271, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 1088, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-02T23:38:27.257 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 1088, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 622, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 25, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 597, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 466, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 27, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 36175, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 12769, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:27.258 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 41632, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 4392, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 398, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 13040 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 40, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 15, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 10, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 45, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 163, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 11, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:27.259 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 3412, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 3401, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 20, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 1179, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 25, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 32, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:27.260 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 50478, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 50449, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 50449, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 29, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 337, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 150, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 149, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 149, 2024-06-02T23:38:27.261 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 1, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 1, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 1, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 112412877, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 112447620, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 34960, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 109.372007582, 2024-06-02T23:38:27.262 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003128489 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 40, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 4415, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 4375, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 36, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 177, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 141, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 139, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 7997, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 7858, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 2, 2024-06-02T23:38:27.263 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 4296, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 4294, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 73832, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 231804 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 890977, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 25969, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 856140, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 361, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 52191, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 4150, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 56.326437105, 2024-06-02T23:38:27.264 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.013572635 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 786382, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 349.542829363, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000444494 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.265 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 27947, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 166.305631438, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005950750 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-02T23:38:27.266 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.267 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:27.268 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.268 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.268 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.268 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.268 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 119, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.413574606, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003475416 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.269 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1557, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 2.931576835, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001882836 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-02T23:38:27.270 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5076, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 27.252576086, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005368907 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 634, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.690217945, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002665958 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.271 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 44, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.278070508, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.006319784 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.272 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.056191312, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.008027330 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.273 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 40, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.063186643, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001579666 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.274 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.021336538, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003048076 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3379, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 8.639264852, 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002556751 2024-06-02T23:38:27.275 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 2, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 1, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-02T23:38:27.276 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 39257, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 39257, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1651, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:27.277 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:27.278 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:27.279 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 302759, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 105, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 157, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:27.280 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 242029, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2709, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 46395, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 113257613, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 31, 2024-06-02T23:38:27.281 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 46395, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 46395, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 227.765841722, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004909275 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 46395, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 54314 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 46395, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 1822, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 44573, 2024-06-02T23:38:27.282 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 1767, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 1529, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 700, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 35635, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 794, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 3963, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1122, 2024-06-02T23:38:27.283 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 3058, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-02T23:38:27.284 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 90, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-02T23:38:27.285 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 4971, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 18, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 757 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-02T23:38:27.286 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 1069 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 6, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 6, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 3401, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 3401, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:27.287 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 1148034, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 222460779, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 1148034, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1148034, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 222460779, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.288 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 46395, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 113600994, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 46395, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 113600994, 2024-06-02T23:38:27.289 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:27.290 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 46395, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 46395, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 46395, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 46395, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:27.291 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 3401, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 343501, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 3401, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 675, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 343501, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55edb87d2500": { 2024-06-02T23:38:27.292 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 50478, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 108253316, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 50478, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 34960, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 108253316, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:27.293 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.294 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:27.294 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-02T23:38:27.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:27 smithi123 ceph-mon[35977]: pgmap v1228: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:27.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:27 smithi174 ceph-mon[35994]: pgmap v1228: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:27.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:27 smithi052 ceph-mon[32340]: pgmap v1228: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:28.060 INFO:tasks.check_counter:Daemon mds.b mds.exported=29 2024-06-02T23:38:28.060 INFO:tasks.check_counter:Daemon mds.b mds.imported=26 2024-06-02T23:38:28.060 DEBUG:tasks.check_counter:Getting stats from e 2024-06-02T23:38:28.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.e perf dump 2024-06-02T23:38:28.303 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:38:29.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:29 smithi123 ceph-mon[35977]: pgmap v1229: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:29.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:29 smithi174 ceph-mon[35994]: pgmap v1229: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:29.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:29 smithi052 ceph-mon[32340]: pgmap v1229: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 40557, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 30890, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 12427745, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 30003467, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 10, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 4, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 2.545093153, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 1.158876824, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 1.450005607, 2024-06-02T23:38:30.213 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.226147166, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 30889, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.903032562, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000029234 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 54295, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.009739558, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000179 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 735840, 2024-06-02T23:38:30.214 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 324672 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 325355, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 366275, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 97484673, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 247229565, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 19, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 16.293630984, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 6.595290673, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 11.875168830, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.212471982, 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:30.215 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 366274, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 6.506317607, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000017763 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 666933, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.110147734, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000165 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 925040 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 15883, 2024-06-02T23:38:30.216 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 14371, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 4366414, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 56445654, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 63, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 8, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 1.475384365, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.362294390, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.333929982, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.747840070, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 14369, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.498201791, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000034671 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.217 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6491, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.001119360, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000172 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:30.218 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25695, 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 25.034460394, 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000974293 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 15058, 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.252847305, 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000083201 2024-06-02T23:38:30.219 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 287970, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 287938, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 287938, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 538.381877967, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001869784 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 32, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:30.220 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 16, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 2, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 2, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 12, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 12, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 105, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 5, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 553988, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 545475, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 32, 2024-06-02T23:38:30.221 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 19, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 2, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 5, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 66, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 5, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 238, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 1891, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-06-02T23:38:30.222 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 1891, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 180, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 180, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 1711, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 17415, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 28611, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:30.223 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 29308, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 9064, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 521, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 26037 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 2, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 51, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 31, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 55, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 4, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-02T23:38:30.224 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 7268, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 7268, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 20, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 2178, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:30.225 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 5, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 47805, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 47805, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-02T23:38:30.226 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 67, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 67, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 5, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 92923767, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 12827, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 47.916316392, 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003735582 2024-06-02T23:38:30.227 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 22, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 7316, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 7294, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 21, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 82, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 61, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 16, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 8631, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 8615, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-06-02T23:38:30.228 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 7277, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 7277, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 69620, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223612 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 303113, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 16, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 287970, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 769, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 2246, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7046, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 247.722990377, 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.035157960 2024-06-02T23:38:30.229 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 254871, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 84.113637931, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000330024 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.230 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2263, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3.856730776, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001704255 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.231 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:30.232 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.233 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-02T23:38:30.234 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 5108, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 3.920988039, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000767617 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 9617, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 35.530030130, 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003694502 2024-06-02T23:38:30.235 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1485, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 19.151803900, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.012896837 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.236 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.092970541, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.004648527 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.237 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:30.238 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 280, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.115454174, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000412336 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.239 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7248, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 143.877272099, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.019850617 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.240 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 32806, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 32806, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1654, 2024-06-02T23:38:30.241 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:30.243 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 305769, 2024-06-02T23:38:30.244 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 115, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 157, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:30.245 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 88680, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 369, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 25017, 2024-06-02T23:38:30.246 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 90058666, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 29, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 25017, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25017, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 76.430279573, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.003055133 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 25017, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 26077 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.247 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 25017, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 2840, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 22177, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 949, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 13776, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 531, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-02T23:38:30.248 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 7301, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1891, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-02T23:38:30.249 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-02T23:38:30.250 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 90, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 16, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 4, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 1077, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 522 2024-06-02T23:38:30.251 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 1960 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-02T23:38:30.252 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 6, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 6, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 7268, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 7268, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 381791, 2024-06-02T23:38:30.253 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 79885191, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 381791, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 381791, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 79885191, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:30.254 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 25017, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 90792734, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 25017, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 90792734, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 25017, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 25017, 2024-06-02T23:38:30.256 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 25017, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 25017, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 7268, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 734068, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:30.257 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 7268, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 949, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 734068, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x55e6077d2500": { 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:30.258 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 47805, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 88729463, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 47805, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 12827, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 88729463, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:30.259 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-02T23:38:30.987 INFO:tasks.check_counter:Daemon mds.e mds.exported=5 2024-06-02T23:38:30.987 INFO:tasks.check_counter:Daemon mds.e mds.imported=5 2024-06-02T23:38:30.987 DEBUG:tasks.check_counter:Getting stats from h 2024-06-02T23:38:30.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.h perf dump 2024-06-02T23:38:31.227 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:38:31.611 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:31 smithi123 ceph-mon[35977]: pgmap v1230: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:31.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:31 smithi174 ceph-mon[35994]: pgmap v1230: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:31.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:31 smithi052 ceph-mon[32340]: pgmap v1230: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:32.982 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-02T23:38:32.982 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 88683, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 77084, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 24612220, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 48298251, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 9, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 4.732658458, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 2.140328705, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 3.094686870, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.198142828, 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:32.983 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 77090, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.492755441, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000019363 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 147882, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.021993286, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000148 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 754496, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 339024 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 45566, 2024-06-02T23:38:32.984 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 34469, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 12154943, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 64910146, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 14, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 2.368131096, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.812520766, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 1.128531322, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.572989531, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 34467, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.823044291, 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000023879 2024-06-02T23:38:32.985 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 41895, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.006328615, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000151 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 480, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 926352 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 383020, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 443403, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 116406919, 2024-06-02T23:38:32.986 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 316810752, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 65, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 12, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 20.009078540, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 8.028893262, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 13.967464985, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.774671102, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 443395, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 7.606307507, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000017154 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 772568, 2024-06-02T23:38:32.987 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.120263061, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000155 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 54333, 2024-06-02T23:38:32.988 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 33.968456066, 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000625190 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 23915, 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 1.469041887, 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000061427 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-02T23:38:32.989 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 318680, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 317384, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 317384, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 528.546943132, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001665323 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 1296, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 11, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-02T23:38:32.990 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 23, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 6, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 7, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 10, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 13, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 216, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 1, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 1, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 6, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 629041, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 594020, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 1291, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 32, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:32.991 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 7, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 7, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 67, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 7, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 281, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 1793, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 4, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 234, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-02T23:38:32.992 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 1793, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 180, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 180, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 1613, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 25157, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 41415, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:32.993 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 43170, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 14178, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 977, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 41553 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 2, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 52, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 32, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 67, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 4, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:32.994 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 10856, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 10856, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 20, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 2326, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 7, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-02T23:38:32.995 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 82002, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 82002, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 98, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-02T23:38:32.996 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 98, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 8, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 130408122, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 0, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 26815, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 71.729479340, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002674975 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 0 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-02T23:38:32.997 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 30, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 10998, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 10968, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 29, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 92, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 63, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 23, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 13402, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 13379, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 1, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 10902, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 10901, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 83796, 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 223612 2024-06-02T23:38:32.998 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 360116, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 54, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 318680, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 1029, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 11046, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10642, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 215.211945605, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.020222885 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 261413, 2024-06-02T23:38:32.999 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 84.968008280, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000325033 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-02T23:38:33.000 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 8288, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 6.109983462, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000737208 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.001 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.002 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-02T23:38:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.003 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-02T23:38:33.004 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 7713, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.948428624, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000641569 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 14926, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 38.784101976, 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002598425 2024-06-02T23:38:33.005 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3064, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 40.956783973, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.013367096 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.006 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 20, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.107349010, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.005367450 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.007 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 19.440106377, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 9.720053188 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.008 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 480, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.164268876, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000342226 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.009 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10836, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 117.855966949, 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.010876335 2024-06-02T23:38:33.010 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-02T23:38:33.011 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 33049, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 33049, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 1658, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:33.012 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:33.013 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:33.014 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 288863, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 105, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 157, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:33.015 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 121805, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 510, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 45238, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 128165644, 2024-06-02T23:38:33.016 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 33, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 45238, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 45238, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 108.369545046, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.002395542 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 45238, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 47191 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 45238, 2024-06-02T23:38:33.017 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 3593, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 41645, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 10, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 1800, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 28615, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 1035, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-02T23:38:33.018 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 10898, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1793, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 20, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-02T23:38:33.019 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-02T23:38:33.020 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 90, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 12, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 2041, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 0, 2024-06-02T23:38:33.021 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 979 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 1, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 2119 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 6, 2024-06-02T23:38:33.022 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 6, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 10856, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 10856, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 517265, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 106586469, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 517265, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:33.023 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 517265, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 106586469, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:33.024 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 45238, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 129262100, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 45238, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 129262100, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:33.025 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 45238, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 45238, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 45238, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 45238, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.026 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 10856, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 1096456, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 10856, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:33.027 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 1800, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 1096456, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x564932484500": { 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 82002, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 126213818, 2024-06-02T23:38:33.028 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 82002, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 26815, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 126213818, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:33.029 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-02T23:38:33.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:33 smithi123 ceph-mon[35977]: pgmap v1231: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:33.891 INFO:tasks.check_counter:Daemon mds.h mds.exported=7 2024-06-02T23:38:33.891 INFO:tasks.check_counter:Daemon mds.h mds.imported=7 2024-06-02T23:38:33.891 DEBUG:tasks.check_counter:Getting stats from k 2024-06-02T23:38:33.891 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.k perf dump 2024-06-02T23:38:33.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:33 smithi174 ceph-mon[35994]: pgmap v1231: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:33.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:33 smithi052 ceph-mon[32340]: pgmap v1231: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:34.129 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.b/config 2024-06-02T23:38:35.621 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:35 smithi123 ceph-mon[35977]: pgmap v1232: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:35.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:35 smithi174 ceph-mon[35994]: pgmap v1232: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2838, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2813, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 18237670, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1046312, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 3, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.270488064, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.099672341, 2024-06-02T23:38:35.894 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.078653505, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.070923007, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2812, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.102041839, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000036287 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 726064, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 314912 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.895 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1126, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 1511, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 33840822, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 1196278, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 8, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 6, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.109304515, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.035765903, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.039042072, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.025591360, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 1509, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.041389645, 2024-06-02T23:38:35.896 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000027428 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 838784 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 2942, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2941, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 51747179, 2024-06-02T23:38:35.897 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 934421, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.204027443, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.046917835, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.079513668, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.065249326, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 2940, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.063707315, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000021669 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:35.898 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 1, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-MDSRank": { 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 10956, 2024-06-02T23:38:35.899 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.224831400, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000020521 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "queue_len": 0, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "complete_latency": { 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": { 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "request": 0, 2024-06-02T23:38:35.900 INFO:teuthology.orchestra.run.smithi123.stdout: "reply": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "reply_latency": { 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_reply": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "forward": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_commit": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_split": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_merge": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes": 8076, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_top": 0, 2024-06-02T23:38:35.901 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_bottom": 8076, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pin_tail": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_pinned": 8075, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_expired": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "inodes_with_caps": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "subtrees": 3, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_hit": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_forward": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_discover": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "traverse_lock": 0, 2024-06-02T23:38:35.902 INFO:teuthology.orchestra.run.smithi123.stdout: "load_cent": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "q": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "exported": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "exported_inodes": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "imported": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "imported_inodes": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_backtrace_fetch": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rfiles": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rbytes": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "root_rsnaps": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:35.903 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_inodes": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_file_inodes": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_grant": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_caps_dirty": 0, 2024-06-02T23:38:35.904 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_cap_release": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "process_request_cap_release": 0 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_cache": { 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_update_receipt": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_try_discover": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_send_discover": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "dir_handle_discover": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_created": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_enqueued": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:35.905 INFO:teuthology.orchestra.run.smithi123.stdout: "strays_migrated": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_started": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "recovery_completed": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_path": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_quiesce_inode": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_exportdir": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_flush": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: "ireq_inodestats": 0 2024-06-02T23:38:35.906 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_log": { 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "evlrg": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "evadd": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "evex": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "evtrm": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "ev": 57429, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "evexg": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "evexd": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "segadd": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "segex": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "segtrm": 0, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "seg": 76, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "segmjr": 6, 2024-06-02T23:38:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: "segexg": 0, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "segexd": 0, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "expos": 4194304, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "wrpos": 105524646, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "rdpos": 105522423, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "jlat": { 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "replayed": 57429 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_mem": { 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "ino": 19, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "ino+": 8079, 2024-06-02T23:38:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: "ino-": 8060, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dir": 17, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dir+": 45, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dir-": 28, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dn": 8076, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dn+": 8369, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dn-": 293, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "cap": 0, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "cap+": 0, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "cap-": 0, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "rss": 60296, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "heap": 215420 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_server": { 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_client_request": 0, 2024-06-02T23:38:35.909 INFO:teuthology.orchestra.run.smithi123.stdout: "dispatch_server_request": 0, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_request": 0, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_client_session": 0, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "handle_peer_request": 0, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "req_create_latency": { 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getattr_latency": { 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.910 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "req_link_latency": { 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookup_latency": { 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.911 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupino_latency": { 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupname_latency": { 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.912 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "req_lssnap_latency": { 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.913 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.914 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.914 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:35 smithi052 ceph-mon[32340]: pgmap v1232: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mkdir_latency": { 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mknod_latency": { 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "req_mksnap_latency": { 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.915 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "req_open_latency": { 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "req_readdir_latency": { 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rename_latency": { 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.916 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmdir_latency": { 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.917 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setattr_latency": { 2024-06-02T23:38:35.918 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.919 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setlayout_latency": { 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "req_setxattr_latency": { 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "req_symlink_latency": { 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.920 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "req_unlink_latency": { 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.921 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_sessions": { 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "session_count": 1, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "session_add": 1, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "session_remove": 0, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_open": 1, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "sessions_stale": 0, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "total_load": 0, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "average_load": 0, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "avg_session_uptime": 2807, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-06-02T23:38:35.922 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:35.923 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:35.924 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:35.925 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:35.926 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 242551, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 85, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-06-02T23:38:35.927 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 157, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 5749751, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 32601, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:35.928 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "objecter": { 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_active": 0, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_laggy": 0, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send": 6361, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_send_bytes": 21, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_resend": 3, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_reply": 6361, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_latency": { 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6361, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 4.047908060, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000636363 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.929 INFO:teuthology.orchestra.run.smithi123.stdout: "op_inflight": 0, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "oplen_avg": { 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 6361, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 6362 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "op": 6361, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "op_r": 6361, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "op_w": 0, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "op_rmw": 0, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "op_pg": 0, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_stat": 3488, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_create": 0, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_read": 2871, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_write": 0, 2024-06-02T23:38:35.930 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writefull": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_writesame": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_append": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_zero": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_truncate": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_delete": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_mapext": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_getxattr": 1, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_setxattr": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_call": 0, 2024-06-02T23:38:35.931 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_watch": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_notify": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "osdop_other": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_active": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_send": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_resend": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "linger_ping": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_active": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_send": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "poolop_resend": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_active": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_send": 0, 2024-06-02T23:38:35.932 INFO:teuthology.orchestra.run.smithi123.stdout: "poolstat_resend": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_active": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_send": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "statfs_resend": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "command_active": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "command_send": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "command_resend": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "map_epoch": 90, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "map_full": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "map_inc": 2, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_sessions": 11, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_open": 12, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_session_close": 1, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_laggy": 0, 2024-06-02T23:38:35.933 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_wr": 0, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_rd": 2, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_del": 0 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "oft": { 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_objs": 0, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_updates": 0, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "omap_total_removes": 0 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "osd-slow-ops": { 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "slow_ops_count": 0 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "purge_queue": { 2024-06-02T23:38:35.934 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executing_high_water": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed_ops": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_executed": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 6902, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 103173877, 2024-06-02T23:38:35.935 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 6902, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 6902, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 103173877, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.936 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 6361, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 101332217, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 6361, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 101332217, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.937 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 1024, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 6361, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 6361, 2024-06-02T23:38:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 6361, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 6361, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:35.939 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-write_buf_throttle-0x557251e7e6e0": { 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 3758096384, 2024-06-02T23:38:35.940 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-06-02T23:38:35.941 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-06-02T23:38:36.713 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-06-02T23:38:36.713 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-06-02T23:38:36.713 DEBUG:tasks.check_counter:Getting stats from c 2024-06-02T23:38:36.714 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.c perf dump 2024-06-02T23:38:37.185 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:38:37.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:37 smithi174 ceph-mon[35994]: pgmap v1233: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:37.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:38:37.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:38:37.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:38:37.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:37 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:38:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:37 smithi123 ceph-mon[35977]: pgmap v1233: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:38:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:38:37.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:38:37.758 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:37 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:38:37.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:37 smithi052 ceph-mon[32340]: pgmap v1233: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:37.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-06-02T23:38:37.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-06-02T23:38:37.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-06-02T23:38:37.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:37 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' 2024-06-02T23:38:39.027 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-06-02T23:38:39.027 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 361977, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 418105, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 110134086, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 296302271, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 16, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 18.335614265, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 7.398231542, 2024-06-02T23:38:39.028 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 13.309805441, 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.376753330, 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 418103, 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.287912277, 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000015039 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:39.029 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 733958, 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.106330834, 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000144 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2096, 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 1744 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:39.030 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 28478, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 22508, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8299892, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 43841748, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 17, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 6, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 1.763769905, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.700936489, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.777682868, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.382890495, 2024-06-02T23:38:39.031 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 22507, 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.519231998, 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000023069 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 26069, 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.003932487, 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000150 2024-06-02T23:38:39.032 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 747168, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 343232 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 91261, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 78623, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 24643925, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 62784148, 2024-06-02T23:38:39.033 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 54, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 7, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 4.285978579, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 1.734774906, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 2.750689279, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.316919741, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 78622, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.517000722, 2024-06-02T23:38:39.034 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000019294 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 140413, 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.018109772, 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000128 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3264, 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 839824 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.035 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-06-02T23:38:39.036 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-06-02T23:38:39.036 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-06-02T23:38:39.036 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.036 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-06-02T23:38:39.037 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:39.037 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:39.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 44269, 2024-06-02T23:38:39.037 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 30.386225949, 2024-06-02T23:38:39.037 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000686399 2024-06-02T23:38:39.038 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.038 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.038 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:39.038 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:39.038 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:39.038 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 22426, 2024-06-02T23:38:39.039 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.260258310, 2024-06-02T23:38:39.039 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000056196 2024-06-02T23:38:39.039 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.039 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.039 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-06-02T23:38:39.039 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 305762, 2024-06-02T23:38:39.040 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 305344, 2024-06-02T23:38:39.040 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-06-02T23:38:39.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 305344, 2024-06-02T23:38:39.040 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 667.584906461, 2024-06-02T23:38:39.040 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002186337 2024-06-02T23:38:39.041 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.041 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-06-02T23:38:39.041 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 418, 2024-06-02T23:38:39.041 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:39.041 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:39.042 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-06-02T23:38:39.042 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-06-02T23:38:39.042 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-06-02T23:38:39.042 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 18, 2024-06-02T23:38:39.042 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 2, 2024-06-02T23:38:39.042 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 2, 2024-06-02T23:38:39.043 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 14, 2024-06-02T23:38:39.043 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 14, 2024-06-02T23:38:39.043 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 247, 2024-06-02T23:38:39.043 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 1, 2024-06-02T23:38:39.043 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 1, 2024-06-02T23:38:39.043 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 6, 2024-06-02T23:38:39.044 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 599981, 2024-06-02T23:38:39.044 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 565509, 2024-06-02T23:38:39.044 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 413, 2024-06-02T23:38:39.044 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 27, 2024-06-02T23:38:39.044 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:39.044 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:39.045 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-06-02T23:38:39.045 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 1000, 2024-06-02T23:38:39.045 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 1, 2024-06-02T23:38:39.045 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 12, 2024-06-02T23:38:39.045 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 136, 2024-06-02T23:38:39.046 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 12, 2024-06-02T23:38:39.046 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 302, 2024-06-02T23:38:39.046 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:39.046 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 2160, 2024-06-02T23:38:39.046 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:39.047 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-06-02T23:38:39.047 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-06-02T23:38:39.047 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-06-02T23:38:39.047 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 2160, 2024-06-02T23:38:39.047 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:39.047 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:39.048 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:39.048 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 180, 2024-06-02T23:38:39.048 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:39.048 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 180, 2024-06-02T23:38:39.048 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1980, 2024-06-02T23:38:39.048 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:39.049 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 22942, 2024-06-02T23:38:39.049 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 38220, 2024-06-02T23:38:39.049 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:39.049 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:39.049 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:39.049 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 38889, 2024-06-02T23:38:39.050 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 11922, 2024-06-02T23:38:39.050 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 758, 2024-06-02T23:38:39.050 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 37564 2024-06-02T23:38:39.050 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.050 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-06-02T23:38:39.051 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 1, 2024-06-02T23:38:39.051 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 52, 2024-06-02T23:38:39.051 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 32, 2024-06-02T23:38:39.051 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 59, 2024-06-02T23:38:39.051 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 3, 2024-06-02T23:38:39.051 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-06-02T23:38:39.052 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:39.052 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:39.052 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 10782, 2024-06-02T23:38:39.052 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 10782, 2024-06-02T23:38:39.052 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:39.052 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-06-02T23:38:39.053 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:39.053 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:39.053 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:39.053 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-06-02T23:38:39.053 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 20, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 2590, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 12, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:39.054 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 74267, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 74267, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 93, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 93, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 8, 2024-06-02T23:38:39.055 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 121782320, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 21718, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 78.196555409, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003600541 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 25, 2024-06-02T23:38:39.056 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 10889, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 10864, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 24, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 93, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 69, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 18, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 14330, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 14312, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 1, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 10808, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 10807, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 79756, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223612 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.057 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 344086, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 20, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 305762, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 728, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 9876, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10530, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 253.075774596, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.024033786 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 256494, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 82.518920843, 2024-06-02T23:38:39.058 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000321718 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.059 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5672, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.822594364, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001202855 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.060 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:39.061 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.062 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.062 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.062 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.062 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-06-02T23:38:39.062 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.062 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-06-02T23:38:39.063 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6166, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.247458346, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000688851 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 13601, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 36.185545788, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002660506 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.064 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1821, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.295305494, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002358761 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.065 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 20, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.081945085, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004097254 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.066 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 39.343110188, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 19.671555094 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:39.067 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 276, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.105926361, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000383791 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.068 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10762, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 240.908325396, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.022385088 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.069 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 33823, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 33823, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1660, 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:39.070 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:39.071 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:39.072 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 305239, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 112, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-06-02T23:38:39.073 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 157, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 101021, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 457, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:39.074 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 39144, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 119440996, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 39144, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 39144, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 117.333299553, 2024-06-02T23:38:39.075 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002997478 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 39144, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 40737 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 39144, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3492, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 35652, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1332, 2024-06-02T23:38:39.076 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 23050, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 812, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 10822, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2160, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:39.077 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-06-02T23:38:39.078 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 90, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-06-02T23:38:39.079 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 1720, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 811 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 2322 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.080 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 6, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 6, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 10782, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 10782, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 481712, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 99689696, 2024-06-02T23:38:39.081 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 481712, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 481712, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 99689696, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:39.082 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 39144, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 120529978, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 39144, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 120529978, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.083 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 39144, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 39144, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 39144, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 39144, 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:39.084 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 10782, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1088982, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 10782, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:39.085 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1332, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1088982, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55c2ddb6c500": { 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 74267, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 117588016, 2024-06-02T23:38:39.086 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 74267, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 21718, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 117588016, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:39.087 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-06-02T23:38:39.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:39 smithi174 ceph-mon[35994]: pgmap v1234: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:39.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:39 smithi123 ceph-mon[35977]: pgmap v1234: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:39.857 INFO:tasks.check_counter:Daemon mds.c mds.exported=12 2024-06-02T23:38:39.857 INFO:tasks.check_counter:Daemon mds.c mds.imported=12 2024-06-02T23:38:39.857 DEBUG:tasks.check_counter:Getting stats from f 2024-06-02T23:38:39.858 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.f perf dump 2024-06-02T23:38:39.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:39 smithi052 ceph-mon[32340]: pgmap v1234: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:40.107 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:38:41.637 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:41 smithi174 ceph-mon[35994]: pgmap v1235: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:41.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:41 smithi123 ceph-mon[35977]: pgmap v1235: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:41.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:41 smithi052 ceph-mon[32340]: pgmap v1235: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:41.923 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-06-02T23:38:41.923 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 4247, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 4222, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 34575846, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1493576, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.390072022, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.122102486, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.121549992, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.114742595, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4221, 2024-06-02T23:38:41.924 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.142360183, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000033726 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 725680, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 313792 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1590, 2024-06-02T23:38:41.925 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1974, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 38149064, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1345274, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 8, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.146686168, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.048418372, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.049576475, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.036346760, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1972, 2024-06-02T23:38:41.926 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.066300689, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000033621 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 840224 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1510, 2024-06-02T23:38:41.927 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1509, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 38130490, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 479045, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.127950482, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.039154612, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.044099922, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.035217695, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1508, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.043363960, 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000028755 2024-06-02T23:38:41.928 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-06-02T23:38:41.929 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 11609, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.315835441, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000027206 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.930 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:41.931 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 3607, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 2498, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 1109, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 38, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 3, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-06-02T23:38:41.932 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-06-02T23:38:41.933 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-06-02T23:38:41.934 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-06-02T23:38:41.935 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-06-02T23:38:41.936 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-06-02T23:38:41.937 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 50478, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 121, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-06-02T23:38:41.938 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 112412877, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 112447620, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 112445969, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 50478 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 32, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 4285, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 4253, 2024-06-02T23:38:41.939 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 28, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 131, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 103, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 3607, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 4997, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 1390, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 58760, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215420 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-06-02T23:38:41.940 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.941 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.942 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-06-02T23:38:41.943 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.944 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-06-02T23:38:41.945 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.945 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.945 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.945 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.945 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-06-02T23:38:41.945 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.946 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-06-02T23:38:41.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.950 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-06-02T23:38:41.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 1, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2814, 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:41.953 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:41.954 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:41.955 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 232881, 2024-06-02T23:38:41.956 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 77, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 157, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:41.957 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 2645917, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 15689, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-06-02T23:38:41.958 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 6803, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 26, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 4, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 6803, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6803, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4.062940573, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000597227 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6803, 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6804 2024-06-02T23:38:41.959 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 6803, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 6803, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 3680, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 3120, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-06-02T23:38:41.960 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-06-02T23:38:41.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-06-02T23:38:41.962 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 90, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 10, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 11, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-06-02T23:38:41.963 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.964 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:41.965 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 7343, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 110163974, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 7343, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 7343, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 110163974, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.966 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 6803, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 108253326, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6803, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 108253326, 2024-06-02T23:38:41.967 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:41.968 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 6803, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 6803, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6803, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 6803, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:41.969 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.970 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55c693fb86e0": { 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:41.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:41.972 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:41.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:41.972 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.972 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:41.972 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-06-02T23:38:42.561 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-06-02T23:38:42.562 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-06-02T23:38:42.562 DEBUG:tasks.check_counter:Getting stats from i 2024-06-02T23:38:42.562 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.i perf dump 2024-06-02T23:38:42.805 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:38:43.641 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:43 smithi174 ceph-mon[35994]: pgmap v1236: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:43.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:43 smithi123 ceph-mon[35977]: pgmap v1236: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:43.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:43 smithi052 ceph-mon[32340]: pgmap v1236: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 63297, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 75332, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 14797439, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 63073397, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 13, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 3.531081395, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 1.513419526, 2024-06-02T23:38:44.454 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 1.974136173, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.286673061, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 75330, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 1.468465130, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000019493 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 111121, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.016221955, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000145 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 0, 2024-06-02T23:38:44.455 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 0 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 364594, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 420875, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 107840807, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 287408131, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 12, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 18.068102354, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 7.315123700, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 13.393998323, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.200635594, 2024-06-02T23:38:44.456 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 420878, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.332784273, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000015046 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 756254, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.105964162, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000140 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 756816, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 347376 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.457 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 13731, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 14115, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 4509673, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 36824383, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 62, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 9, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 1.079384874, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.344985210, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.230960427, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.436583980, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 14110, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.392991001, 2024-06-02T23:38:44.458 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000027851 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 50, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000004262, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000085 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3424, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 930624 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-06-02T23:38:44.459 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 37582, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 25.611211655, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000681475 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 19398, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.937770547, 2024-06-02T23:38:44.460 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000048343 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 301824, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 301807, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 301807, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 454.663265639, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001506470 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 17, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:44.461 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 26, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 7, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 6, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 13, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 15, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 46, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 2, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 2, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 7, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 586406, 2024-06-02T23:38:44.462 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 562389, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 17, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 14, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 170, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 1000, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 1, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 2, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 2, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 5, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 249, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 1812, 2024-06-02T23:38:44.463 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 1812, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 201, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 201, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 1611, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 2265, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 20221, 2024-06-02T23:38:44.464 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 30724, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 35362, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 11754, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 922, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 28154 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 30, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 30, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 20, 2024-06-02T23:38:44.465 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 25, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 77, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 1, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 8052, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 8051, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:44.466 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 20, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 2028, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 4, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-06-02T23:38:44.467 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 57429, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 57429, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 76, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 76, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 6, 2024-06-02T23:38:44.468 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 105524646, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 18122, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 58.711005517, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003239764 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-06-02T23:38:44.469 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 27, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 8101, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 8074, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 26, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 68, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 42, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 26, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 11867, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 11841, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 2, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 8060, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 8058, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 76112, 2024-06-02T23:38:44.470 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223612 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 329235, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 749, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 301824, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 710, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 5081, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7893, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 202.094118860, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.025604221 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-06-02T23:38:44.471 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 260046, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 80.446415376, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000309354 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.472 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7239, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6.697618343, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000925213 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.473 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.474 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.475 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.475 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-06-02T23:38:44.475 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.475 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.475 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.476 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6174, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3.902638965, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000632108 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 10766, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 38.181839538, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.003546520 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-06-02T23:38:44.477 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1360, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 26.502107612, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.019486843 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.478 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 20, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.092913578, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.004645678 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.479 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 77, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 28.889318562, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.375185955 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 200, 2024-06-02T23:38:44.480 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.117487736, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000587438 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.481 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 8032, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 67.738807069, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.008433616 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.482 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 1, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 32496, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 32496, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1666, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.483 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 120, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 120, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:44.484 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:44.485 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 295835, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 103, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-06-02T23:38:44.486 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 157, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 115127, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 701, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:44.487 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 33408, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 102899342, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 33, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 33408, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 33408, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 89.027708054, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002664861 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.488 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 33408, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 35211 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 33408, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3481, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 29927, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1669, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 19791, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 987, 2024-06-02T23:38:44.489 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 8087, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1812, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-06-02T23:38:44.490 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-06-02T23:38:44.491 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 90, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 12, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-06-02T23:38:44.492 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 1929, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 906 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 1840 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-06-02T23:38:44.493 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 6, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 6, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 8051, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 8051, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 441618, 2024-06-02T23:38:44.494 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 87368164, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 441618, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 441618, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 87368164, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:44.495 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 33408, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 103712493, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 33408, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 103712493, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.496 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 33408, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 33408, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 33408, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 33408, 2024-06-02T23:38:44.497 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 8051, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 813151, 2024-06-02T23:38:44.498 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 8051, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 1669, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 813151, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x56357636c500": { 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:44.499 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 57429, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 101330342, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 57429, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 18122, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 101330342, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.500 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:44.501 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-06-02T23:38:45.275 INFO:tasks.check_counter:Daemon mds.i mds.exported=2 2024-06-02T23:38:45.275 INFO:tasks.check_counter:Daemon mds.i mds.imported=5 2024-06-02T23:38:45.276 DEBUG:tasks.check_counter:Getting stats from l 2024-06-02T23:38:45.276 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph daemon mds.l perf dump 2024-06-02T23:38:45.516 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.c/config 2024-06-02T23:38:45.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:45 smithi123 ceph-mon[35977]: pgmap v1237: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:45.781 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:45 smithi174 ceph-mon[35994]: pgmap v1237: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:45.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:45 smithi052 ceph-mon[32340]: pgmap v1237: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1163, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1138, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 21855402, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 514956, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.122676879, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.043580916, 2024-06-02T23:38:47.331 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.045239721, 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.025496727, 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1137, 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.037279380, 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000032787 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.332 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 727152, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 317472 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1880, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2268, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 55001493, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1439753, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:47.333 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.182209936, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.056261248, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.066183261, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.045023699, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2266, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.074082120, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000032692 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.334 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 842544 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 3937, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 3937, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 51872263, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1251200, 2024-06-02T23:38:47.335 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.357167893, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.100579565, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.115639640, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.112030839, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3936, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.132366384, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000033629 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.336 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 11067, 2024-06-02T23:38:47.337 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.247204931, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000022337 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-06-02T23:38:47.338 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 10866, 2024-06-02T23:38:47.339 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 10866, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 10866, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-06-02T23:38:47.340 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-06-02T23:38:47.341 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-06-02T23:38:47.342 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-06-02T23:38:47.343 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-06-02T23:38:47.344 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-06-02T23:38:47.345 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 82002, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-06-02T23:38:47.346 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 98, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 8, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 130408122, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 130406984, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 82002 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.347 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 13, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 10894, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 10881, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 11, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 58, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 47, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 10866, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 11249, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 383, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-06-02T23:38:47.348 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 62768, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215420 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-06-02T23:38:47.349 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.350 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.351 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.352 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.353 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.354 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-06-02T23:38:47.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.354 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.354 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.355 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-06-02T23:38:47.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.355 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.355 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.356 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-06-02T23:38:47.359 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-06-02T23:38:47.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.361 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.362 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.363 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 1, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 0, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-06-02T23:38:47.364 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 2816, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-06-02T23:38:47.365 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-06-02T23:38:47.366 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-06-02T23:38:47.367 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 225607, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 75, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-06-02T23:38:47.368 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 7440, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 157, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 7693048, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 43592, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-06-02T23:38:47.369 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 6432, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 6432, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6432, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3.611947189, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000561558 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.370 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6432, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 6433 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 6432, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 6432, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 3510, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 2920, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-06-02T23:38:47.371 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-06-02T23:38:47.372 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-06-02T23:38:47.373 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 90, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 2, 2024-06-02T23:38:47.374 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 9, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 11, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 2, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-06-02T23:38:47.375 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-06-02T23:38:47.376 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 6976, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 128070496, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 6976, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6976, 2024-06-02T23:38:47.377 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 128070496, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:47.378 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 6432, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 126216074, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6432, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 126216074, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.379 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 6432, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 6432, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6432, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 6432, 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:47.380 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:47.381 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.382 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x559986dba6e0": { 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-06-02T23:38:47.383 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-06-02T23:38:47.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-06-02T23:38:47.384 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-06-02T23:38:47.384 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-06-02T23:38:47.384 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.384 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-06-02T23:38:47.384 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-06-02T23:38:47.757 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:47 smithi123 ceph-mon[35977]: pgmap v1238: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:47.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:47 smithi174 ceph-mon[35994]: pgmap v1238: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:47.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:47 smithi052 ceph-mon[32340]: pgmap v1238: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:48.034 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-06-02T23:38:48.034 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-06-02T23:38:48.035 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-06-02T23:38:48.045 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-06-02T23:38:48.045 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:38:48.045 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T23:38:48.077 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:38:48.077 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T23:38:48.138 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-02T23:38:48.439 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:49.763 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:49 smithi052 ceph-mon[32340]: pgmap v1239: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:49.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:49 smithi174 ceph-mon[35994]: pgmap v1239: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:50.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:49 smithi123 ceph-mon[35977]: pgmap v1239: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 -- 172.21.15.52:0/3798490802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c501030f0 msgr2=0x7f1c50107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 --2- 172.21.15.52:0/3798490802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c501030f0 0x7f1c50107110 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f1c440099c0 tx=0x7f1c44009cd0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 -- 172.21.15.52:0/3798490802 shutdown_connections 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 --2- 172.21.15.52:0/3798490802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 0x7f1c5010f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 --2- 172.21.15.52:0/3798490802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c50102740 0x7f1c50102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 --2- 172.21.15.52:0/3798490802 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c501030f0 0x7f1c50107110 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 -- 172.21.15.52:0/3798490802 >> 172.21.15.52:0/3798490802 conn(0x7f1c500795f0 msgr2=0x7f1c50079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 -- 172.21.15.52:0/3798490802 shutdown_connections 2024-06-02T23:38:50.235 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 -- 172.21.15.52:0/3798490802 wait complete. 2024-06-02T23:38:50.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 Processor -- start 2024-06-02T23:38:50.236 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.235+0000 7f1c564ac700 1 -- start start 2024-06-02T23:38:50.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c564ac700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 0x7f1c5019e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:50.237 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c564ac700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c501030f0 0x7f1c5019ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c564ac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 0x7f1c50198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c564ac700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112680 con 0x7f1c50102740 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c564ac700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112500 con 0x7f1c501030f0 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c564ac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1c50112800 con 0x7f1c50107840 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 0x7f1c5019e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 0x7f1c5019e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44932/0 (socket says 172.21.15.52:44932) 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 -- 172.21.15.52:0/2122619266 learned_addr learned my addr 172.21.15.52:0/2122619266 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4f7fe700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c501030f0 0x7f1c5019ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c54a49700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 0x7f1c50198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 -- 172.21.15.52:0/2122619266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 msgr2=0x7f1c50198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:50.238 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 0x7f1c50198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 -- 172.21.15.52:0/2122619266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c501030f0 msgr2=0x7f1c5019ece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c501030f0 0x7f1c5019ece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c4ffff700 1 -- 172.21.15.52:0/2122619266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c501990c0 con 0x7f1c50102740 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.236+0000 7f1c54a49700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 0x7f1c50198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.237+0000 7f1c4ffff700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 0x7f1c5019e7a0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000d950 tx=0x7f1c4000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.237+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c40016070 con 0x7f1c50102740 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.237+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f1c40009df0 con 0x7f1c50102740 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.237+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c501993b0 con 0x7f1c50102740 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.237+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000f3c0 con 0x7f1c50102740 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.237+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c501a5670 con 0x7f1c50102740 2024-06-02T23:38:50.239 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.238+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c50069180 con 0x7f1c50102740 2024-06-02T23:38:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.240+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f1c40019070 con 0x7f1c50102740 2024-06-02T23:38:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.241+0000 7f1c4d7fa700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1c380a2c20 0x7f1c380a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.241+0000 7f1c4f7fe700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1c380a2c20 0x7f1c380a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:50.240 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.241+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f1c400cebd0 con 0x7f1c50102740 2024-06-02T23:38:50.241 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.241+0000 7f1c4f7fe700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1c380a2c20 0x7f1c380a50e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1c501020d0 tx=0x7f1c44005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:50.244 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.244+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f1c4009bb60 con 0x7f1c50102740 2024-06-02T23:38:50.468 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.469+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1c50199e90 con 0x7f1c50102740 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.469+0000 7f1c4d7fa700 1 -- 172.21.15.52:0/2122619266 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+588 (secure 0 0 0) 0x7f1c4009b460 con 0x7f1c50102740 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/540942425 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/160135232 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2656327338 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2620214668 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/540942425 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1498162594 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3265496994 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/752611388 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:50.469 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2620214668 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:50.470 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1697426194 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:50.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1c380a2c20 msgr2=0x7f1c380a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:50.471 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1c380a2c20 0x7f1c380a50e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1c501020d0 tx=0x7f1c44005fb0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 msgr2=0x7f1c5019e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 0x7f1c5019e7a0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f1c4000d950 tx=0x7f1c4000dc60 comp rx=0 tx=0).stop 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 shutdown_connections 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f1c380a2c20 0x7f1c380a50e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1c50107840 0x7f1c50198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f1c50102740 0x7f1c5019e7a0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 --2- 172.21.15.52:0/2122619266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c501030f0 0x7f1c5019ece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 >> 172.21.15.52:0/2122619266 conn(0x7f1c500795f0 msgr2=0x7f1c50100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 shutdown_connections 2024-06-02T23:38:50.472 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:50.472+0000 7f1c564ac700 1 -- 172.21.15.52:0/2122619266 wait complete. 2024-06-02T23:38:50.473 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-06-02T23:38:50.803 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:50 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2122619266' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T23:38:50.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:50 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2122619266' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T23:38:51.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:50 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2122619266' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T23:38:51.115 DEBUG:teuthology.orchestra.run.smithi052:> set -ex 2024-06-02T23:38:51.116 DEBUG:teuthology.orchestra.run.smithi052:> dd if=/proc/self/mounts of=/dev/stdout 2024-06-02T23:38:51.134 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd blocklist ls 2024-06-02T23:38:51.428 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:38:51.679 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:51 smithi052 ceph-mon[32340]: pgmap v1240: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:51.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:51 smithi174 ceph-mon[35994]: pgmap v1240: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:52.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:51 smithi123 ceph-mon[35977]: pgmap v1240: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 11 op/s 2024-06-02T23:38:53.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 -- 172.21.15.52:0/3125517576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3240107870 msgr2=0x7f324010f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:53.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 --2- 172.21.15.52:0/3125517576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3240107870 0x7f324010f3f0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f323800b0d0 tx=0x7f323800b3e0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 -- 172.21.15.52:0/3125517576 shutdown_connections 2024-06-02T23:38:53.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 --2- 172.21.15.52:0/3125517576 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3240107870 0x7f324010f3f0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 --2- 172.21.15.52:0/3125517576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32401027a0 0x7f3240102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 --2- 172.21.15.52:0/3125517576 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3240103150 0x7f3240107140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 -- 172.21.15.52:0/3125517576 >> 172.21.15.52:0/3125517576 conn(0x7f3240079520 msgr2=0x7f3240079930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 -- 172.21.15.52:0/3125517576 shutdown_connections 2024-06-02T23:38:53.117 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.116+0000 7f3246548700 1 -- 172.21.15.52:0/3125517576 wait complete. 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.117+0000 7f3246548700 1 Processor -- start 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.117+0000 7f3246548700 1 -- start start 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3246548700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 0x7f324019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3246548700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3240103150 0x7f324019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3246548700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3240107870 0x7f3240198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3246548700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3240112640 con 0x7f32401027a0 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3246548700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32401124c0 con 0x7f3240107870 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3246548700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32401127c0 con 0x7f3240103150 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 0x7f324019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 0x7f324019e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:44954/0 (socket says 172.21.15.52:44954) 2024-06-02T23:38:53.119 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 -- 172.21.15.52:0/3410762487 learned_addr learned my addr 172.21.15.52:0/3410762487 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3237fff700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3240103150 0x7f324019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f3244ae5700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3240107870 0x7f3240198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 -- 172.21.15.52:0/3410762487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3240103150 msgr2=0x7f324019eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3240103150 0x7f324019eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 -- 172.21.15.52:0/3410762487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3240107870 msgr2=0x7f3240198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3240107870 0x7f3240198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.118+0000 7f323ffff700 1 -- 172.21.15.52:0/3410762487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3240199000 con 0x7f32401027a0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.119+0000 7f323ffff700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 0x7f324019e650 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f323000c9b0 tx=0x7f323000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.119+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230007b00 con 0x7f32401027a0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.119+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3230007ca0 con 0x7f32401027a0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.119+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32401992f0 con 0x7f32401027a0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.119+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3230018730 con 0x7f32401027a0 2024-06-02T23:38:53.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.119+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32401a54b0 con 0x7f32401027a0 2024-06-02T23:38:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.120+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3240103c00 con 0x7f32401027a0 2024-06-02T23:38:53.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.121+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f323000f450 con 0x7f32401027a0 2024-06-02T23:38:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.122+0000 7f323dffb700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f322c0a2c20 0x7f322c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:38:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.122+0000 7f3237fff700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f322c0a2c20 0x7f322c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:38:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.122+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f32300cf0d0 con 0x7f32401027a0 2024-06-02T23:38:53.122 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.123+0000 7f3237fff700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f322c0a2c20 0x7f322c0a50e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3228005950 tx=0x7f32280058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:38:53.126 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.127+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3230014030 con 0x7f32401027a0 2024-06-02T23:38:53.349 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.350+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f324002dca0 con 0x7f32401027a0 2024-06-02T23:38:53.350 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/540942425 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:53.350 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/160135232 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:53.350 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/2656327338 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:53.350 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/2620214668 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:53.350 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6800/540942425 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:53.350 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1498162594 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/3265496994 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/752611388 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:6801/2620214668 2024-06-03T22:59:43.024153+0000 2024-06-02T23:38:53.351 INFO:teuthology.orchestra.run.smithi052.stdout:172.21.15.52:0/1697426194 2024-06-03T23:00:09.671662+0000 2024-06-02T23:38:53.351 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.350+0000 7f323dffb700 1 -- 172.21.15.52:0/3410762487 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v90) ==== 81+0+588 (secure 0 0 0) 0x7f323009c060 con 0x7f32401027a0 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f322c0a2c20 msgr2=0x7f322c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f322c0a2c20 0x7f322c0a50e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3228005950 tx=0x7f32280058e0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 msgr2=0x7f324019e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 0x7f324019e650 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f323000c9b0 tx=0x7f323000cd70 comp rx=0 tx=0).stop 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 shutdown_connections 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f322c0a2c20 0x7f322c0a50e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.352 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3240107870 0x7f3240198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f32401027a0 0x7f324019e650 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 --2- 172.21.15.52:0/3410762487 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3240103150 0x7f324019eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:38:53.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.352+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 >> 172.21.15.52:0/3410762487 conn(0x7f3240079520 msgr2=0x7f32401004f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:38:53.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.353+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 shutdown_connections 2024-06-02T23:38:53.353 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:38:53.353+0000 7f3246548700 1 -- 172.21.15.52:0/3410762487 wait complete. 2024-06-02T23:38:53.353 INFO:teuthology.orchestra.run.smithi052.stderr:listed 10 entries 2024-06-02T23:38:53.649 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:53 smithi052 ceph-mon[32340]: pgmap v1241: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:53.650 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:53 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/3410762487' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T23:38:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:53 smithi174 ceph-mon[35994]: pgmap v1241: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:53.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:53 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/3410762487' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T23:38:53.979 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi052.front.sepia.ceph.com... 2024-06-02T23:38:53.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:38:53.979 DEBUG:teuthology.orchestra.run.smithi052:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-06-02T23:38:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:53 smithi123 ceph-mon[35977]: pgmap v1241: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-06-02T23:38:54.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:53 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/3410762487' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-06-02T23:38:54.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi052.stderr:ceph-fuse[87498]: fuse finished with error 0 and tester_r 0 2024-06-02T23:38:54.031 INFO:teuthology.orchestra.run:waiting for 300 2024-06-02T23:38:55.809 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:55 smithi123 ceph-mon[35977]: pgmap v1242: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:55.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:55 smithi174 ceph-mon[35994]: pgmap v1242: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:55.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:55 smithi052 ceph-mon[32340]: pgmap v1242: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 op/s 2024-06-02T23:38:57.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:57 smithi174 ceph-mon[35994]: pgmap v1243: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:38:57.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:57 smithi052 ceph-mon[32340]: pgmap v1243: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:38:58.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:57 smithi123 ceph-mon[35977]: pgmap v1243: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 9 op/s 2024-06-02T23:38:59.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:38:59 smithi174 ceph-mon[35994]: pgmap v1244: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-06-02T23:38:59.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:38:59 smithi052 ceph-mon[32340]: pgmap v1244: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-06-02T23:39:00.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:38:59 smithi123 ceph-mon[35977]: pgmap v1244: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 767 B/s wr, 13 op/s 2024-06-02T23:39:00.133 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:39:00.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:39:00.133 DEBUG:teuthology.orchestra.run.smithi052:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-06-02T23:39:00.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:39:00.165 DEBUG:teuthology.orchestra.run.smithi052:> ip netns list 2024-06-02T23:39:00.224 INFO:teuthology.orchestra.run.smithi052.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-06-02T23:39:00.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:39:00.225 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-06-02T23:39:00.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-06-02T23:39:00.299 DEBUG:teuthology.orchestra.run.smithi052:> sudo ip link delete ceph-brx 2024-06-02T23:39:00.405 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-06-02T23:39:00.416 INFO:tasks.cephadm:Teardown begin 2024-06-02T23:39:00.416 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-06-02T23:39:00.416 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:39:00.677 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:01.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:01 smithi174 ceph-mon[35994]: pgmap v1245: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-02T23:39:01.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:01 smithi052 ceph-mon[32340]: pgmap v1245: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-02T23:39:02.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:01 smithi123 ceph-mon[35977]: pgmap v1245: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-02T23:39:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.622+0000 7f8742392700 1 -- 172.21.15.52:0/1896311713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c1030f0 msgr2=0x7f873c107110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:02.622 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 --2- 172.21.15.52:0/1896311713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c1030f0 0x7f873c107110 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f87240099c0 tx=0x7f8724009cd0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 -- 172.21.15.52:0/1896311713 shutdown_connections 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 --2- 172.21.15.52:0/1896311713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 0x7f873c10f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 --2- 172.21.15.52:0/1896311713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c102740 0x7f873c102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 --2- 172.21.15.52:0/1896311713 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c1030f0 0x7f873c107110 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 -- 172.21.15.52:0/1896311713 >> 172.21.15.52:0/1896311713 conn(0x7f873c0795f0 msgr2=0x7f873c079a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 -- 172.21.15.52:0/1896311713 shutdown_connections 2024-06-02T23:39:02.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 -- 172.21.15.52:0/1896311713 wait complete. 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 Processor -- start 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.623+0000 7f8742392700 1 -- start start 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8742392700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 0x7f873c19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8742392700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c1030f0 0x7f873c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8742392700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 0x7f873c1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8742392700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f873c112600 con 0x7f873c102740 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8742392700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f873c112480 con 0x7f873c107840 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8742392700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f873c112780 con 0x7f873c1030f0 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 0x7f873c19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:02.625 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 0x7f873c19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38594/0 (socket says 172.21.15.52:38594) 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 -- 172.21.15.52:0/2898992223 learned_addr learned my addr 172.21.15.52:0/2898992223 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8733fff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c1030f0 0x7f873c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f874092f700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 0x7f873c1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 -- 172.21.15.52:0/2898992223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c1030f0 msgr2=0x7f873c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c1030f0 0x7f873c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 -- 172.21.15.52:0/2898992223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 msgr2=0x7f873c1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 0x7f873c1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f873bfff700 1 -- 172.21.15.52:0/2898992223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f873c199040 con 0x7f873c102740 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f8733fff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c1030f0 0x7f873c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.624+0000 7f874092f700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 0x7f873c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.625+0000 7f873bfff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 0x7f873c19e6d0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f872c00c9b0 tx=0x7f872c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.625+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c007b00 con 0x7f873c102740 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.625+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f872c007ca0 con 0x7f873c102740 2024-06-02T23:39:02.626 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.625+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f873c199330 con 0x7f873c102740 2024-06-02T23:39:02.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.625+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f872c018730 con 0x7f873c102740 2024-06-02T23:39:02.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.625+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f873c1a55a0 con 0x7f873c102740 2024-06-02T23:39:02.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.626+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f873c069180 con 0x7f873c102740 2024-06-02T23:39:02.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.627+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f872c00f450 con 0x7f873c102740 2024-06-02T23:39:02.627 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.628+0000 7f8739ffb700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87280a2c20 0x7f87280a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:02.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.628+0000 7f8733fff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87280a2c20 0x7f87280a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:02.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.628+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f872c0cf0d0 con 0x7f873c102740 2024-06-02T23:39:02.628 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.628+0000 7f8733fff700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87280a2c20 0x7f87280a50e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f873c100f80 tx=0x7f8724005ab0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:02.632 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.632+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f872c014030 con 0x7f873c102740 2024-06-02T23:39:02.884 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.885+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f873c02dca0 con 0x7f873c102740 2024-06-02T23:39:02.885 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.885+0000 7f8739ffb700 1 -- 172.21.15.52:0/2898992223 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11953 (secure 0 0 0) 0x7f872c09c060 con 0x7f873c102740 2024-06-02T23:39:02.885 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:39:02.885 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:37:19.717233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:39:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87280a2c20 msgr2=0x7f87280a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87280a2c20 0x7f87280a50e0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f873c100f80 tx=0x7f8724005ab0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 msgr2=0x7f873c19e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 0x7f873c19e6d0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f872c00c9b0 tx=0x7f872c00cd70 comp rx=0 tx=0).stop 2024-06-02T23:39:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 shutdown_connections 2024-06-02T23:39:02.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f87280a2c20 0x7f87280a50e0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f873c107840 0x7f873c1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f873c102740 0x7f873c19e6d0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 --2- 172.21.15.52:0/2898992223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f873c1030f0 0x7f873c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 >> 172.21.15.52:0/2898992223 conn(0x7f873c0795f0 msgr2=0x7f873c101c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 shutdown_connections 2024-06-02T23:39:02.888 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:02.888+0000 7f8742392700 1 -- 172.21.15.52:0/2898992223 wait complete. 2024-06-02T23:39:02.889 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 48 2024-06-02T23:39:03.519 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs dump --format=json 2024-06-02T23:39:03.778 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:03 smithi174 ceph-mon[35994]: pgmap v1246: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-02T23:39:03.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:03 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2898992223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:39:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:03 smithi052 ceph-mon[32340]: pgmap v1246: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-02T23:39:03.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:03 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2898992223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:39:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:03 smithi123 ceph-mon[35977]: pgmap v1246: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-02T23:39:04.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:03 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2898992223' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:39:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 -- 172.21.15.52:0/4184445078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102ed0 msgr2=0x7f367010f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 --2- 172.21.15.52:0/4184445078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102ed0 0x7f367010f360 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7f3668009990 tx=0x7f3668009ca0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.512 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 -- 172.21.15.52:0/4184445078 shutdown_connections 2024-06-02T23:39:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 --2- 172.21.15.52:0/4184445078 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102ed0 0x7f367010f360 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 --2- 172.21.15.52:0/4184445078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3670108520 0x7f3670108900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 --2- 172.21.15.52:0/4184445078 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3670102510 0x7f3670102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 -- 172.21.15.52:0/4184445078 >> 172.21.15.52:0/4184445078 conn(0x7f3670079560 msgr2=0x7f3670079970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 -- 172.21.15.52:0/4184445078 shutdown_connections 2024-06-02T23:39:05.513 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.512+0000 7f3675a89700 1 -- 172.21.15.52:0/4184445078 wait complete. 2024-06-02T23:39:05.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 Processor -- start 2024-06-02T23:39:05.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 -- start start 2024-06-02T23:39:05.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 0x7f3670198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:05.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3670102ed0 0x7f36701990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:05.515 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3670108520 0x7f367019d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f3670112650 con 0x7f3670102510 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f36701124d0 con 0x7f3670102ed0 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f3675a89700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36701127d0 con 0x7f3670108520 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.513+0000 7f366f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 0x7f3670198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 0x7f3670198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38682/0 (socket says 172.21.15.52:38682) 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 -- 172.21.15.52:0/2223786133 learned_addr learned my addr 172.21.15.52:0/2223786133 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366ffff700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3670108520 0x7f367019d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366effd700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3670102ed0 0x7f36701990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:05.516 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 -- 172.21.15.52:0/2223786133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3670108520 msgr2=0x7f367019d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3670108520 0x7f367019d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 -- 172.21.15.52:0/2223786133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3670102ed0 msgr2=0x7f36701990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3670102ed0 0x7f36701990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 -- 172.21.15.52:0/2223786133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f367019db10 con 0x7f3670102510 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366f7fe700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 0x7f3670198b60 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f3660007f10 tx=0x7f366000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660004db0 con 0x7f3670102510 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f367019de00 con 0x7f3670102510 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f3660003a00 con 0x7f3670102510 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36600045e0 con 0x7f3670102510 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.514+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36701a56c0 con 0x7f3670102510 2024-06-02T23:39:05.517 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.515+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3670069180 con 0x7f3670102510 2024-06-02T23:39:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.516+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f366001c020 con 0x7f3670102510 2024-06-02T23:39:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.517+0000 7f366cff9700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f365c0a2c20 0x7f365c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.517+0000 7f366effd700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f365c0a2c20 0x7f365c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.517+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7f36600ce7a0 con 0x7f3670102510 2024-06-02T23:39:05.519 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.518+0000 7f366effd700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f365c0a2c20 0x7f365c0a50e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3670079a60 tx=0x7f365800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:05.521 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.522+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f3660010070 con 0x7f3670102510 2024-06-02T23:39:05.774 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:05 smithi052 ceph-mon[32340]: pgmap v1247: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-06-02T23:39:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.775+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f367002dca0 con 0x7f3670102510 2024-06-02T23:39:05.775 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.776+0000 7f366cff9700 1 -- 172.21.15.52:0/2223786133 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) ==== 94+0+11953 (secure 0 0 0) 0x7f366009b730 con 0x7f3670102510 2024-06-02T23:39:05.776 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:39:05.776 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":48,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6835/3011548911","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6834","nonce":3011548911},{"type":"v1","addr":"172.21.15.52:6835","nonce":3011548911}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.52:6837/4013141842","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6836","nonce":4013141842},{"type":"v1","addr":"172.21.15.52:6837","nonce":4013141842}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":48,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-06-02T23:07:33.296248+0000","modified":"2024-06-02T23:37:19.717233+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24470,"mds_1":24451,"mds_2":24449,"mds_3":24436,"mds_4":24434},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14604":{"gid":14604,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6839/2175393174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6838","nonce":2175393174},{"type":"v1","addr":"172.21.15.52:6839","nonce":2175393174}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14616":{"gid":14616,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.52:6841/4238066654","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6840","nonce":4238066654},{"type":"v1","addr":"172.21.15.52:6841","nonce":4238066654}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24400":{"gid":24400,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/224943979","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":224943979},{"type":"v1","addr":"172.21.15.174:6833","nonce":224943979}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.123:6833/1267486579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":1267486579},{"type":"v1","addr":"172.21.15.123:6833","nonce":1267486579}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/2486297365","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":2486297365},{"type":"v1","addr":"172.21.15.174:6835","nonce":2486297365}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":16,"addr":"172.21.15.123:6835/3092837185","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3092837185},{"type":"v1","addr":"172.21.15.123:6835","nonce":3092837185}]},"join_fscid":-1,"export_targets":[0,1,2,3],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":14,"addr":"172.21.15.174:6837/457473811","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":457473811},{"type":"v1","addr":"172.21.15.174:6837","nonce":457473811}]},"join_fscid":-1,"export_targets":[0,1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":13,"addr":"172.21.15.123:6837/785283486","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":785283486},{"type":"v1","addr":"172.21.15.123:6837","nonce":785283486}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24451":{"gid":24451,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":12,"addr":"172.21.15.174:6839/2115890873","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":2115890873},{"type":"v1","addr":"172.21.15.174:6839","nonce":2115890873}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.123:6839/2043673684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":2043673684},{"type":"v1","addr":"172.21.15.123:6839","nonce":2043673684}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24470,"qdb_cluster":[24434,24436,24451,24470,24449]},"id":1}]} 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f365c0a2c20 msgr2=0x7f365c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f365c0a2c20 0x7f365c0a50e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f3670079a60 tx=0x7f365800b410 comp rx=0 tx=0).stop 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 msgr2=0x7f3670198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 0x7f3670198b60 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f3660007f10 tx=0x7f366000d3b0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 shutdown_connections 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f365c0a2c20 0x7f365c0a50e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3670108520 0x7f367019d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f3670102510 0x7f3670198b60 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 --2- 172.21.15.52:0/2223786133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3670102ed0 0x7f36701990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.778+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 >> 172.21.15.52:0/2223786133 conn(0x7f3670079560 msgr2=0x7f3670101120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:05.778 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.779+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 shutdown_connections 2024-06-02T23:39:05.779 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:05.779+0000 7f3675a89700 1 -- 172.21.15.52:0/2223786133 wait complete. 2024-06-02T23:39:05.779 INFO:teuthology.orchestra.run.smithi052.stderr:dumped fsmap epoch 48 2024-06-02T23:39:05.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:05 smithi174 ceph-mon[35994]: pgmap v1247: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-06-02T23:39:06.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:05 smithi123 ceph-mon[35977]: pgmap v1247: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-06-02T23:39:06.432 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd dump --format=json 2024-06-02T23:39:06.687 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:06.801 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:06 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2223786133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:39:06.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:06 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2223786133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:39:07.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:06 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2223786133' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-06-02T23:39:07.858 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:07 smithi052 ceph-mon[32340]: pgmap v1248: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-02T23:39:07.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:07 smithi174 ceph-mon[35994]: pgmap v1248: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-02T23:39:08.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:07 smithi123 ceph-mon[35977]: pgmap v1248: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 11 op/s 2024-06-02T23:39:08.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 -- 172.21.15.52:0/3915807525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30102770 msgr2=0x7fee30102b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:08.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 --2- 172.21.15.52:0/3915807525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30102770 0x7fee30102b50 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fee2c0099c0 tx=0x7fee2c009cd0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 -- 172.21.15.52:0/3915807525 shutdown_connections 2024-06-02T23:39:08.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 --2- 172.21.15.52:0/3915807525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee30107860 0x7fee3010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.453 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 --2- 172.21.15.52:0/3915807525 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30102770 0x7fee30102b50 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 --2- 172.21.15.52:0/3915807525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30103120 0x7fee30107130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 -- 172.21.15.52:0/3915807525 >> 172.21.15.52:0/3915807525 conn(0x7fee30079600 msgr2=0x7fee30079a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 -- 172.21.15.52:0/3915807525 shutdown_connections 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.452+0000 7fee387ad700 1 -- 172.21.15.52:0/3915807525 wait complete. 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 Processor -- start 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 -- start start 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30102770 0x7fee3019e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee30103120 0x7fee3019ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 0x7fee301988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fee30112650 con 0x7fee30107860 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee301124d0 con 0x7fee30103120 2024-06-02T23:39:08.454 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee387ad700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fee301127d0 con 0x7fee30102770 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 0x7fee301988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 0x7fee301988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38700/0 (socket says 172.21.15.52:38700) 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 -- 172.21.15.52:0/1055865049 learned_addr learned my addr 172.21.15.52:0/1055865049 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36549700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30102770 0x7fee3019e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee35d48700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee30103120 0x7fee3019ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 -- 172.21.15.52:0/1055865049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30102770 msgr2=0x7fee3019e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30102770 0x7fee3019e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 -- 172.21.15.52:0/1055865049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee30103120 msgr2=0x7fee3019ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:08.455 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee30103120 0x7fee3019ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36d4a700 1 -- 172.21.15.52:0/1055865049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee30199030 con 0x7fee30107860 2024-06-02T23:39:08.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.453+0000 7fee36549700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30102770 0x7fee3019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:39:08.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.454+0000 7fee36d4a700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 0x7fee301988d0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fee2800ef30 tx=0x7fee2800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:08.457 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.454+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee280092e0 con 0x7fee30107860 2024-06-02T23:39:08.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.454+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee301992c0 con 0x7fee30107860 2024-06-02T23:39:08.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.454+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fee28004bb0 con 0x7fee30107860 2024-06-02T23:39:08.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.454+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee28004280 con 0x7fee30107860 2024-06-02T23:39:08.458 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.454+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee301a55e0 con 0x7fee30107860 2024-06-02T23:39:08.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.455+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee3010aaa0 con 0x7fee30107860 2024-06-02T23:39:08.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.456+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fee2801b030 con 0x7fee30107860 2024-06-02T23:39:08.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.457+0000 7fee277fe700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee1c0a2c20 0x7fee1c0a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:08.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.457+0000 7fee36549700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee1c0a2c20 0x7fee1c0a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:08.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.457+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fee280ced00 con 0x7fee30107860 2024-06-02T23:39:08.459 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.457+0000 7fee36549700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee1c0a2c20 0x7fee1c0a50e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000c00 tx=0x7fee2c005940 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:08.461 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.461+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fee28022020 con 0x7fee30107860 2024-06-02T23:39:08.667 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.667+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fee30199e00 con 0x7fee30107860 2024-06-02T23:39:08.668 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.669+0000 7fee277fe700 1 -- 172.21.15.52:0/1055865049 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+22611 (secure 0 0 0) 0x7fee28010040 con 0x7fee30107860 2024-06-02T23:39:08.669 INFO:teuthology.orchestra.run.smithi052.stdout: 2024-06-02T23:39:08.669 INFO:teuthology.orchestra.run.smithi052.stdout:{"epoch":90,"fsid":"9d9fe36c-2133-11ef-bc9b-c7b262605968","created":"2024-06-02T22:59:11.730596+0000","modified":"2024-06-02T23:08:11.318645+0000","last_up_change":"2024-06-02T23:06:23.656839+0000","last_in_change":"2024-06-02T23:06:03.339105+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-06-02T23:02:34.195730+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-06-02T23:07:26.794014+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-06-02T23:07:30.138193+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"01ccc644-7329-48bb-9a2c-f48aafdfad4f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6802","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6803","nonce":1323575430}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6804","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6805","nonce":1323575430}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6808","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6809","nonce":1323575430}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6806","nonce":1323575430},{"type":"v1","addr":"172.21.15.52:6807","nonce":1323575430}]},"public_addr":"172.21.15.52:6803/1323575430","cluster_addr":"172.21.15.52:6805/1323575430","heartbeat_back_addr":"172.21.15.52:6809/1323575430","heartbeat_front_addr":"172.21.15.52:6807/1323575430","state":["exists","up"]},{"osd":1,"uuid":"08319da8-7871-414b-ad99-e708bcd4bed9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6801","nonce":338548510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6803","nonce":338548510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6807","nonce":338548510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":338548510},{"type":"v1","addr":"172.21.15.123:6805","nonce":338548510}]},"public_addr":"172.21.15.123:6801/338548510","cluster_addr":"172.21.15.123:6803/338548510","heartbeat_back_addr":"172.21.15.123:6807/338548510","heartbeat_front_addr":"172.21.15.123:6805/338548510","state":["exists","up"]},{"osd":2,"uuid":"4248f78e-ee22-4cd7-92e9-e449a51cb4b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6801","nonce":2598349360}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6803","nonce":2598349360}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6807","nonce":2598349360}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":2598349360},{"type":"v1","addr":"172.21.15.174:6805","nonce":2598349360}]},"public_addr":"172.21.15.174:6801/2598349360","cluster_addr":"172.21.15.174:6803/2598349360","heartbeat_back_addr":"172.21.15.174:6807/2598349360","heartbeat_front_addr":"172.21.15.174:6805/2598349360","state":["exists","up"]},{"osd":3,"uuid":"e32ed956-2af4-430f-ac3c-e349190ebfa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6810","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6811","nonce":2267151243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6812","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6813","nonce":2267151243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6816","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6817","nonce":2267151243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6814","nonce":2267151243},{"type":"v1","addr":"172.21.15.52:6815","nonce":2267151243}]},"public_addr":"172.21.15.52:6811/2267151243","cluster_addr":"172.21.15.52:6813/2267151243","heartbeat_back_addr":"172.21.15.52:6817/2267151243","heartbeat_front_addr":"172.21.15.52:6815/2267151243","state":["exists","up"]},{"osd":4,"uuid":"a9734ce9-e810-4891-a831-1afaef0f45b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6809","nonce":3937434833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6811","nonce":3937434833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6815","nonce":3937434833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":3937434833},{"type":"v1","addr":"172.21.15.123:6813","nonce":3937434833}]},"public_addr":"172.21.15.123:6809/3937434833","cluster_addr":"172.21.15.123:6811/3937434833","heartbeat_back_addr":"172.21.15.123:6815/3937434833","heartbeat_front_addr":"172.21.15.123:6813/3937434833","state":["exists","up"]},{"osd":5,"uuid":"b7b9fa3f-3eab-4fbb-95ce-8101bf1130fd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6809","nonce":1358350436}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6811","nonce":1358350436}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6815","nonce":1358350436}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":1358350436},{"type":"v1","addr":"172.21.15.174:6813","nonce":1358350436}]},"public_addr":"172.21.15.174:6809/1358350436","cluster_addr":"172.21.15.174:6811/1358350436","heartbeat_back_addr":"172.21.15.174:6815/1358350436","heartbeat_front_addr":"172.21.15.174:6813/1358350436","state":["exists","up"]},{"osd":6,"uuid":"3667d476-c6ce-4999-8954-4ade4fbe51a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6818","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6819","nonce":2952400783}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6820","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6821","nonce":2952400783}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6824","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6825","nonce":2952400783}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6822","nonce":2952400783},{"type":"v1","addr":"172.21.15.52:6823","nonce":2952400783}]},"public_addr":"172.21.15.52:6819/2952400783","cluster_addr":"172.21.15.52:6821/2952400783","heartbeat_back_addr":"172.21.15.52:6825/2952400783","heartbeat_front_addr":"172.21.15.52:6823/2952400783","state":["exists","up"]},{"osd":7,"uuid":"0ef49c6a-2339-4f91-9474-0c747c53a20d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6817","nonce":550996005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6819","nonce":550996005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6823","nonce":550996005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":550996005},{"type":"v1","addr":"172.21.15.123:6821","nonce":550996005}]},"public_addr":"172.21.15.123:6817/550996005","cluster_addr":"172.21.15.123:6819/550996005","heartbeat_back_addr":"172.21.15.123:6823/550996005","heartbeat_front_addr":"172.21.15.123:6821/550996005","state":["exists","up"]},{"osd":8,"uuid":"27a74c0f-68e7-4c68-aaf3-69859a1b47e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6817","nonce":186727045}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6819","nonce":186727045}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6823","nonce":186727045}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":186727045},{"type":"v1","addr":"172.21.15.174:6821","nonce":186727045}]},"public_addr":"172.21.15.174:6817/186727045","cluster_addr":"172.21.15.174:6819/186727045","heartbeat_back_addr":"172.21.15.174:6823/186727045","heartbeat_front_addr":"172.21.15.174:6821/186727045","state":["exists","up"]},{"osd":9,"uuid":"482dc129-e665-4a35-add6-b40d6359d13a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6826","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6827","nonce":3771079645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6828","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6829","nonce":3771079645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6832","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6833","nonce":3771079645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.52:6830","nonce":3771079645},{"type":"v1","addr":"172.21.15.52:6831","nonce":3771079645}]},"public_addr":"172.21.15.52:6827/3771079645","cluster_addr":"172.21.15.52:6829/3771079645","heartbeat_back_addr":"172.21.15.52:6833/3771079645","heartbeat_front_addr":"172.21.15.52:6831/3771079645","state":["exists","up"]},{"osd":10,"uuid":"8bea87b2-167c-4b06-939d-59e6a5c69b07","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6825","nonce":4143797452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6827","nonce":4143797452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6831","nonce":4143797452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4143797452},{"type":"v1","addr":"172.21.15.123:6829","nonce":4143797452}]},"public_addr":"172.21.15.123:6825/4143797452","cluster_addr":"172.21.15.123:6827/4143797452","heartbeat_back_addr":"172.21.15.123:6831/4143797452","heartbeat_front_addr":"172.21.15.123:6829/4143797452","state":["exists","up"]},{"osd":11,"uuid":"d9af8f08-98ca-4c39-8571-2a6fa0c08075","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6825","nonce":190382346}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6827","nonce":190382346}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6831","nonce":190382346}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":190382346},{"type":"v1","addr":"172.21.15.174:6829","nonce":190382346}]},"public_addr":"172.21.15.174:6825/190382346","cluster_addr":"172.21.15.174:6827/190382346","heartbeat_back_addr":"172.21.15.174:6831/190382346","heartbeat_front_addr":"172.21.15.174:6829/190382346","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:01:44.277790+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:08.237820+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:31.578223+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:02:56.974539+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:22.526473+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:03:47.088651+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:12.484921+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:04:38.597271+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:03.679006+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:29.789623+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:05:55.457886+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-06-02T23:06:21.419639+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.10","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11},{"from":2,"to":1}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.25","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.52:6801/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/160135232":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/2656327338":"2024-06-03T23:00:09.671662+0000","172.21.15.52:6800/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6800/540942425":"2024-06-03T23:00:09.671662+0000","172.21.15.52:0/1498162594":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/3265496994":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/752611388":"2024-06-03T22:59:43.024153+0000","172.21.15.52:6801/2620214668":"2024-06-03T22:59:43.024153+0000","172.21.15.52:0/1697426194":"2024-06-03T23:00:09.671662+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-06-02T23:39:08.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.671+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee1c0a2c20 msgr2=0x7fee1c0a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:08.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.671+0000 7fee387ad700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee1c0a2c20 0x7fee1c0a50e0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fee2c000c00 tx=0x7fee2c005940 comp rx=0 tx=0).stop 2024-06-02T23:39:08.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.671+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 msgr2=0x7fee301988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:08.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.671+0000 7fee387ad700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 0x7fee301988d0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fee2800ef30 tx=0x7fee2800c450 comp rx=0 tx=0).stop 2024-06-02T23:39:08.671 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 shutdown_connections 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fee1c0a2c20 0x7fee1c0a50e0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fee30107860 0x7fee301988d0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fee30102770 0x7fee3019e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 --2- 172.21.15.52:0/1055865049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee30103120 0x7fee3019ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 >> 172.21.15.52:0/1055865049 conn(0x7fee30079600 msgr2=0x7fee30100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 shutdown_connections 2024-06-02T23:39:08.672 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:08.672+0000 7fee387ad700 1 -- 172.21.15.52:0/1055865049 wait complete. 2024-06-02T23:39:09.486 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs fail cephfs 2024-06-02T23:39:09.744 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:09 smithi052 ceph-mon[32340]: pgmap v1249: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-02T23:39:09.867 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:09 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/1055865049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:39:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:09 smithi174 ceph-mon[35994]: pgmap v1249: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-02T23:39:09.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:09 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/1055865049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:39:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:09 smithi123 ceph-mon[35977]: pgmap v1249: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.2 KiB/s wr, 13 op/s 2024-06-02T23:39:10.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:09 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/1055865049' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-06-02T23:39:10.889 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:10 smithi052 ceph-mon[32340]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:39:10.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:10 smithi174 ceph-mon[35994]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:39:11.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:10 smithi123 ceph-mon[35977]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-06-02T23:39:11.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 -- 172.21.15.52:0/584956744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 msgr2=0x7fd3c810aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:11.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/584956744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c810aa50 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fd3b8009990 tx=0x7fd3b8009ca0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 -- 172.21.15.52:0/584956744 shutdown_connections 2024-06-02T23:39:11.615 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/584956744 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c8105030 0x7fd3c8111540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/584956744 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c810aa50 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/584956744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3c8104670 0x7fd3c8104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 -- 172.21.15.52:0/584956744 >> 172.21.15.52:0/584956744 conn(0x7fd3c8100140 msgr2=0x7fd3c8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 -- 172.21.15.52:0/584956744 shutdown_connections 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.615+0000 7fd3ce73b700 1 -- 172.21.15.52:0/584956744 wait complete. 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 Processor -- start 2024-06-02T23:39:11.616 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 -- start start 2024-06-02T23:39:11.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c8104670 0x7fd3c81a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:11.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3c8105030 0x7fd3c81a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:11.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:11.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c8114870 con 0x7fd3c810a670 2024-06-02T23:39:11.618 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81146f0 con 0x7fd3c8104670 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3ce73b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c81149f0 con 0x7fd3c8105030 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3cccd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3cccd8700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c819ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38706/0 (socket says 172.21.15.52:38706) 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3c77fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3c8105030 0x7fd3c81a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3cccd8700 1 -- 172.21.15.52:0/2619564280 learned_addr learned my addr 172.21.15.52:0/2619564280 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.616+0000 7fd3c7fff700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c8104670 0x7fd3c81a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3cccd8700 1 -- 172.21.15.52:0/2619564280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3c8105030 msgr2=0x7fd3c81a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3cccd8700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3c8105030 0x7fd3c81a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3cccd8700 1 -- 172.21.15.52:0/2619564280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c8104670 msgr2=0x7fd3c81a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:11.619 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3cccd8700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c8104670 0x7fd3c81a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3cccd8700 1 -- 172.21.15.52:0/2619564280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c819b410 con 0x7fd3c810a670 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3cccd8700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c819ab30 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0007d70 tx=0x7fd3c000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c0016020 con 0x7fd3c810a670 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7fd3c000ac90 con 0x7fd3c810a670 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3c00115e0 con 0x7fd3c810a670 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c819b700 con 0x7fd3c810a670 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.617+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c81a79e0 con 0x7fd3c810a670 2024-06-02T23:39:11.620 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.618+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c810cc00 con 0x7fd3c810a670 2024-06-02T23:39:11.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.619+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7fd3c0019020 con 0x7fd3c810a670 2024-06-02T23:39:11.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.620+0000 7fd3c57fa700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3b00a2ca0 0x7fd3b00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:11.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.620+0000 7fd3c7fff700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3b00a2ca0 0x7fd3b00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:11.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.620+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8190+0+0 (secure 0 0 0) 0x7fd3c00cf3a0 con 0x7fd3c810a670 2024-06-02T23:39:11.621 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.621+0000 7fd3c7fff700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3b00a2ca0 0x7fd3b00a5160 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b5c0 tx=0x7fd3b8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:11.623 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.624+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7fd3c0020020 con 0x7fd3c810a670 2024-06-02T23:39:11.879 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.880+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd3c819bff0 con 0x7fd3c810a670 2024-06-02T23:39:11.880 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:11 smithi052 ceph-mon[32340]: pgmap v1250: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-02T23:39:11.882 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.882+0000 7fd3c57fa700 1 -- 172.21.15.52:0/2619564280 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v50) ==== 161+0+0 (secure 0 0 0) 0x7fd3c009c330 con 0x7fd3c810a670 2024-06-02T23:39:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3b00a2ca0 msgr2=0x7fd3b00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3b00a2ca0 0x7fd3b00a5160 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd3b800b5c0 tx=0x7fd3b8005960 comp rx=0 tx=0).stop 2024-06-02T23:39:11.886 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 msgr2=0x7fd3c819ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c819ab30 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0007d70 tx=0x7fd3c000a340 comp rx=0 tx=0).stop 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 shutdown_connections 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7fd3b00a2ca0 0x7fd3b00a5160 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7fd3c810a670 0x7fd3c819ab30 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3c8104670 0x7fd3c81a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 --2- 172.21.15.52:0/2619564280 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd3c8105030 0x7fd3c81a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.886+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 >> 172.21.15.52:0/2619564280 conn(0x7fd3c8100140 msgr2=0x7fd3c8100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.887+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 shutdown_connections 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:11.887+0000 7fd3ce73b700 1 -- 172.21.15.52:0/2619564280 wait complete. 2024-06-02T23:39:11.887 INFO:teuthology.orchestra.run.smithi052.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-06-02T23:39:11.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-mon[35994]: pgmap v1250: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-02T23:39:11.959 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-mon[35977]: pgmap v1250: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 511 B/s wr, 11 op/s 2024-06-02T23:39:12.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:11 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a[32336]: 2024-06-02T23:39:11.881+0000 7f4ae4f32700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T23:39:12.150 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a[72552]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.150 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a[72552]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.150 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mds[72556]: -- 172.21.15.52:0/2418828871 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e08bc85a40 con 0x55e08bc58800 2024-06-02T23:39:12.151 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a[72552]: starting mds.a at 2024-06-02T23:39:12.151 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g[72100]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.151 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g[72100]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.151 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mds[72116]: -- 172.21.15.52:0/1258383488 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56201c811a40 con 0x56201c7e5000 2024-06-02T23:39:12.151 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g[72100]: starting mds.g at 2024-06-02T23:39:12.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h[65621]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h[65621]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-mds[65625]: -- 172.21.15.123:0/3887877307 <== mon.2 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55ef501d04e0 con 0x55ef501a6800 2024-06-02T23:39:12.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-mds[65625]: -- 172.21.15.123:0/3887877307 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ef501d3a40 con 0x55ef501a6800 2024-06-02T23:39:12.257 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h[65621]: starting mds.h at 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-mds[65926]: -- 172.21.15.123:0/3924385590 <== mon.0 v2:172.21.15.52:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55acdb761a40 con 0x55acdb734800 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: starting mds.b at 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k[65000]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k[65000]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.258 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-mds[65004]: -- 172.21.15.123:0/1449698340 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5583082e9a40 con 0x5583082bc800 2024-06-02T23:39:12.259 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k[65000]: starting mds.k at 2024-06-02T23:39:12.259 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e[65318]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.259 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e[65318]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.259 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-mds[65322]: -- 172.21.15.123:0/2530042649 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d967b3fa40 con 0x55d967b12800 2024-06-02T23:39:12.259 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:39:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e[65318]: starting mds.e at 2024-06-02T23:39:12.393 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f[65778]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f[65778]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-mds[65782]: -- 172.21.15.174:0/3769429257 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5636ec61ba40 con 0x5636ec5ee800 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f[65778]: starting mds.f at 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i[66386]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i[66386]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-mds[66390]: -- 172.21.15.174:0/1875211885 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c76017fa40 con 0x55c760152800 2024-06-02T23:39:12.394 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i[66386]: starting mds.i at 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c[66083]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c[66083]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-mds[66087]: -- 172.21.15.174:0/2041579171 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dd3c6b1a40 con 0x55dd3c684800 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c[66083]: starting mds.c at 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l[65476]: ignoring --setuser ceph since I am not root 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l[65476]: ignoring --setgroup ceph since I am not root 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-mds[65480]: -- 172.21.15.174:0/77233701 <== mon.2 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558faa58da40 con 0x558faa561000 2024-06-02T23:39:12.395 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:39:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l[65476]: starting mds.l at 2024-06-02T23:39:12.696 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-06-02T23:39:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2619564280' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T23:39:12.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:12 smithi174 ceph-mon[35994]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-02T23:39:12.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:12 smithi174 ceph-mon[35994]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T23:39:12.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:12 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2619564280' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-02T23:39:12.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:12 smithi174 ceph-mon[35994]: osdmap e91: 12 total, 12 up, 12 in 2024-06-02T23:39:12.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:12 smithi174 ceph-mon[35994]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-02T23:39:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2619564280' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T23:39:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mon[32340]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-02T23:39:12.899 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mon[32340]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T23:39:12.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2619564280' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-02T23:39:12.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mon[32340]: osdmap e91: 12 total, 12 up, 12 in 2024-06-02T23:39:12.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:12 smithi052 ceph-mon[32340]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-02T23:39:12.951 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2619564280' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-06-02T23:39:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:12 smithi123 ceph-mon[35977]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-06-02T23:39:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:12 smithi123 ceph-mon[35977]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-06-02T23:39:13.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:12 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2619564280' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-06-02T23:39:13.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:12 smithi123 ceph-mon[35977]: osdmap e91: 12 total, 12 up, 12 in 2024-06-02T23:39:13.008 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:12 smithi123 ceph-mon[35977]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: pgmap v1252: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6832/2706137230,v1:172.21.15.174:6833/2706137230] up:boot 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6834/1576404646,v1:172.21.15.174:6835/1576404646] up:boot 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6836/36956510,v1:172.21.15.174:6837/36956510] up:boot 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.174:6838/4161034018,v1:172.21.15.174:6839/4161034018] up:boot 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6833/1961352351,v1:172.21.15.123:6835/1961352351] up:boot 2024-06-02T23:39:14.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6832/1408085712,v1:172.21.15.123:6834/1408085712] up:boot 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6836/3946724963,v1:172.21.15.123:6838/3946724963] up:boot 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.123:6837/1080781746,v1:172.21.15.123:6839/1080781746] up:boot 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.52:6838/4098575389,v1:172.21.15.52:6840/4098575389] up:boot 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: mds.? [v2:172.21.15.52:6839/2828060545,v1:172.21.15.52:6841/2828060545] up:boot 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T23:39:14.145 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:13 smithi174 ceph-mon[35994]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T23:39:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: pgmap v1252: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-02T23:39:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6832/2706137230,v1:172.21.15.174:6833/2706137230] up:boot 2024-06-02T23:39:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6834/1576404646,v1:172.21.15.174:6835/1576404646] up:boot 2024-06-02T23:39:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6836/36956510,v1:172.21.15.174:6837/36956510] up:boot 2024-06-02T23:39:14.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.174:6838/4161034018,v1:172.21.15.174:6839/4161034018] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6833/1961352351,v1:172.21.15.123:6835/1961352351] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6832/1408085712,v1:172.21.15.123:6834/1408085712] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6836/3946724963,v1:172.21.15.123:6838/3946724963] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.123:6837/1080781746,v1:172.21.15.123:6839/1080781746] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.52:6838/4098575389,v1:172.21.15.52:6840/4098575389] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: mds.? [v2:172.21.15.52:6839/2828060545,v1:172.21.15.52:6841/2828060545] up:boot 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T23:39:14.151 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T23:39:14.152 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T23:39:14.152 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T23:39:14.152 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:13 smithi052 ceph-mon[32340]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T23:39:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: pgmap v1252: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-06-02T23:39:14.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6832/2706137230,v1:172.21.15.174:6833/2706137230] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6834/1576404646,v1:172.21.15.174:6835/1576404646] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6836/36956510,v1:172.21.15.174:6837/36956510] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.174:6838/4161034018,v1:172.21.15.174:6839/4161034018] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6833/1961352351,v1:172.21.15.123:6835/1961352351] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6832/1408085712,v1:172.21.15.123:6834/1408085712] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6836/3946724963,v1:172.21.15.123:6838/3946724963] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.123:6837/1080781746,v1:172.21.15.123:6839/1080781746] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.52:6838/4098575389,v1:172.21.15.52:6840/4098575389] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: mds.? [v2:172.21.15.52:6839/2828060545,v1:172.21.15.52:6841/2828060545] up:boot 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-06-02T23:39:14.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-06-02T23:39:14.259 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:13 smithi123 ceph-mon[35977]: from='mgr.14150 172.21.15.52:0/571036288' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-06-02T23:39:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 -- 172.21.15.52:0/745583975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4101520 msgr2=0x7f2ee4101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/745583975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4101520 0x7f2ee4101900 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00099c0 tx=0x7f2ee0009cd0 comp rx=0 tx=0).stop 2024-06-02T23:39:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 -- 172.21.15.52:0/745583975 shutdown_connections 2024-06-02T23:39:14.749 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/745583975 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee4106600 0x7f2ee410e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/745583975 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4101520 0x7f2ee4101900 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/745583975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee4101ed0 0x7f2ee4105f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.747+0000 7f2eeb19e700 1 -- 172.21.15.52:0/745583975 >> 172.21.15.52:0/745583975 conn(0x7f2ee40fcdb0 msgr2=0x7f2ee40ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.748+0000 7f2eeb19e700 1 -- 172.21.15.52:0/745583975 shutdown_connections 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.748+0000 7f2eeb19e700 1 -- 172.21.15.52:0/745583975 wait complete. 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.748+0000 7f2eeb19e700 1 Processor -- start 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 -- start start 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee4101520 0x7f2ee4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee4101ed0 0x7f2ee4199e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 0x7f2ee41a4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111490 con 0x7f2ee4106600 2024-06-02T23:39:14.750 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111310 con 0x7f2ee4101ed0 2024-06-02T23:39:14.751 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eeb19e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee4111610 con 0x7f2ee4101520 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 0x7f2ee41a4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 0x7f2ee41a4990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:38986/0 (socket says 172.21.15.52:38986) 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 -- 172.21.15.52:0/4199828738 learned_addr learned my addr 172.21.15.52:0/4199828738 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2ee999b700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee4101ed0 0x7f2ee4199e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea19c700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee4101520 0x7f2ee4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 -- 172.21.15.52:0/4199828738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee4101520 msgr2=0x7f2ee4199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee4101520 0x7f2ee4199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:14.752 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 -- 172.21.15.52:0/4199828738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee4101ed0 msgr2=0x7f2ee4199e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee4101ed0 0x7f2ee4199e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.749+0000 7f2eea99d700 1 -- 172.21.15.52:0/4199828738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee41a4ed0 con 0x7f2ee4106600 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.750+0000 7f2eea99d700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 0x7f2ee41a4990 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ada0 tx=0x7f2edc00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.750+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc017020 con 0x7f2ee4106600 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.750+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f2edc00cd80 con 0x7f2ee4106600 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.750+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee41a50a0 con 0x7f2ee4106600 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.750+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc005260 con 0x7f2ee4106600 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.750+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee41a5600 con 0x7f2ee4106600 2024-06-02T23:39:14.753 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.751+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ec8005330 con 0x7f2ee4106600 2024-06-02T23:39:14.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.752+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f2edc00c8d0 con 0x7f2ee4106600 2024-06-02T23:39:14.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.753+0000 7f2edb7fe700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed00a2ca0 0x7f2ed00a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:14.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.753+0000 7f2eea19c700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed00a2ca0 0x7f2ed00a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:14.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.753+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8620+0+0 (secure 0 0 0) 0x7f2edc0ce660 con 0x7f2ee4106600 2024-06-02T23:39:14.754 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.753+0000 7f2eea19c700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed00a2ca0 0x7f2ed00a5160 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0000c00 tx=0x7f2ee0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:14.758 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:14.758+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f2edc021020 con 0x7f2ee4106600 2024-06-02T23:39:15.011 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.010+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2ec8005e30 con 0x7f2ee4106600 2024-06-02T23:39:15.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.894+0000 7f2edb7fe700 1 -- 172.21.15.52:0/4199828738 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v52) ==== 104+0+0 (secure 0 0 0) 0x7f2edc0192b0 con 0x7f2ee4106600 2024-06-02T23:39:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed00a2ca0 msgr2=0x7f2ed00a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed00a2ca0 0x7f2ed00a5160 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0000c00 tx=0x7f2ee0005fb0 comp rx=0 tx=0).stop 2024-06-02T23:39:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 msgr2=0x7f2ee41a4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 0x7f2ee41a4990 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00ada0 tx=0x7f2edc00c330 comp rx=0 tx=0).stop 2024-06-02T23:39:15.898 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 shutdown_connections 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f2ed00a2ca0 0x7f2ed00a5160 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f2ee4106600 0x7f2ee41a4990 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ee4101520 0x7f2ee4199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 --2- 172.21.15.52:0/4199828738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ee4101ed0 0x7f2ee4199e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 >> 172.21.15.52:0/4199828738 conn(0x7f2ee40fcdb0 msgr2=0x7f2ee40ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 shutdown_connections 2024-06-02T23:39:15.899 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:15.897+0000 7f2eeb19e700 1 -- 172.21.15.52:0/4199828738 wait complete. 2024-06-02T23:39:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:15 smithi174 ceph-mon[35994]: pgmap v1253: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-06-02T23:39:16.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:15 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4199828738' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:39:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:15 smithi052 ceph-mon[32340]: pgmap v1253: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-06-02T23:39:16.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:15 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4199828738' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:39:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:15 smithi123 ceph-mon[35977]: pgmap v1253: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-06-02T23:39:16.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:15 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4199828738' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-06-02T23:39:16.677 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-06-02T23:39:16.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:16 smithi052 ceph-mon[32340]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-02T23:39:16.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:16 smithi052 ceph-mon[32340]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-02T23:39:16.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:16 smithi052 ceph-mon[32340]: Cluster is now healthy 2024-06-02T23:39:16.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:16 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/4199828738' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-02T23:39:16.900 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:16 smithi052 ceph-mon[32340]: fsmap 12 up:standby 2024-06-02T23:39:16.937 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:16 smithi174 ceph-mon[35994]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-02T23:39:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:16 smithi174 ceph-mon[35994]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-02T23:39:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:16 smithi174 ceph-mon[35994]: Cluster is now healthy 2024-06-02T23:39:17.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:16 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/4199828738' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-02T23:39:17.144 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:16 smithi174 ceph-mon[35994]: fsmap 12 up:standby 2024-06-02T23:39:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:16 smithi123 ceph-mon[35977]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-06-02T23:39:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:16 smithi123 ceph-mon[35977]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-06-02T23:39:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:16 smithi123 ceph-mon[35977]: Cluster is now healthy 2024-06-02T23:39:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:16 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/4199828738' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-06-02T23:39:17.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:16 smithi123 ceph-mon[35977]: fsmap 12 up:standby 2024-06-02T23:39:18.143 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:17 smithi174 ceph-mon[35994]: pgmap v1254: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-02T23:39:18.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:17 smithi052 ceph-mon[32340]: pgmap v1254: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-02T23:39:18.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:17 smithi123 ceph-mon[35977]: pgmap v1254: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.672+0000 7f5599a0f700 1 -- 172.21.15.52:0/2314036537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406b830 msgr2=0x7f5594107120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.672+0000 7f5599a0f700 1 --2- 172.21.15.52:0/2314036537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406b830 0x7f5594107120 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f55880099c0 tx=0x7f5588009cd0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.672+0000 7f5599a0f700 1 -- 172.21.15.52:0/2314036537 shutdown_connections 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.672+0000 7f5599a0f700 1 --2- 172.21.15.52:0/2314036537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 0x7f559410f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.672+0000 7f5599a0f700 1 --2- 172.21.15.52:0/2314036537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559406ae80 0x7f559406b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.672+0000 7f5599a0f700 1 --2- 172.21.15.52:0/2314036537 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406b830 0x7f5594107120 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.673+0000 7f5599a0f700 1 -- 172.21.15.52:0/2314036537 >> 172.21.15.52:0/2314036537 conn(0x7f5594078400 msgr2=0x7f5594078810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:18.674 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.673+0000 7f5599a0f700 1 -- 172.21.15.52:0/2314036537 shutdown_connections 2024-06-02T23:39:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.673+0000 7f5599a0f700 1 -- 172.21.15.52:0/2314036537 wait complete. 2024-06-02T23:39:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.673+0000 7f5599a0f700 1 Processor -- start 2024-06-02T23:39:18.675 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 -- start start 2024-06-02T23:39:18.676 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 0x7f5594101400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559406b830 0x7f5594101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 0x7f5594101e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5594112690 con 0x7f559406ae80 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5594112510 con 0x7f5594107850 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5599a0f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5594112810 con 0x7f559406b830 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 0x7f5594101400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:18.677 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 0x7f5594101400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39000/0 (socket says 172.21.15.52:39000) 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 -- 172.21.15.52:0/48916312 learned_addr learned my addr 172.21.15.52:0/48916312 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f55937fe700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 0x7f5594101e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f55927fc700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559406b830 0x7f5594101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 -- 172.21.15.52:0/48916312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559406b830 msgr2=0x7f5594101940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559406b830 0x7f5594101940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 -- 172.21.15.52:0/48916312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 msgr2=0x7f5594101e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 0x7f5594101e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f5592ffd700 1 -- 172.21.15.52:0/48916312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55941a7040 con 0x7f559406ae80 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.674+0000 7f55937fe700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 0x7f5594101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:39:18.678 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.675+0000 7f5592ffd700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 0x7f5594101400 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f558400b530 tx=0x7f558400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.675+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584015020 con 0x7f559406ae80 2024-06-02T23:39:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.675+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55941a7330 con 0x7f559406ae80 2024-06-02T23:39:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.675+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5584011ce0 con 0x7f559406ae80 2024-06-02T23:39:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.675+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55941a7840 con 0x7f559406ae80 2024-06-02T23:39:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.675+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5584010d60 con 0x7f559406ae80 2024-06-02T23:39:18.679 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.676+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f559410aa90 con 0x7f559406ae80 2024-06-02T23:39:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.677+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5584018020 con 0x7f559406ae80 2024-06-02T23:39:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.678+0000 7f5598a0d700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f557c0a2ca0 0x7f557c0a5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.678+0000 7f55927fc700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f557c0a2ca0 0x7f557c0a5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.678+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8620+0+0 (secure 0 0 0) 0x7f55840cea80 con 0x7f559406ae80 2024-06-02T23:39:18.680 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.678+0000 7f55927fc700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f557c0a2ca0 0x7f557c0a5160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5594102e40 tx=0x7f5588005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:18.683 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.683+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f55840d0e20 con 0x7f559406ae80 2024-06-02T23:39:18.889 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.889+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5594069180 con 0x7f559406ae80 2024-06-02T23:39:18.895 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.895+0000 7f5598a0d700 1 -- 172.21.15.52:0/48916312 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v92) ==== 181+0+0 (secure 0 0 0) 0x7f558401f020 con 0x7f559406ae80 2024-06-02T23:39:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.899+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f557c0a2ca0 msgr2=0x7f557c0a5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.899+0000 7f5599a0f700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f557c0a2ca0 0x7f557c0a5160 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f5594102e40 tx=0x7f5588005fd0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.899+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 msgr2=0x7f5594101400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.899+0000 7f5599a0f700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 0x7f5594101400 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f558400b530 tx=0x7f558400b8f0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 shutdown_connections 2024-06-02T23:39:18.900 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f557c0a2ca0 0x7f557c0a5160 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5594107850 0x7f5594101e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f559406ae80 0x7f5594101400 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 --2- 172.21.15.52:0/48916312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f559406b830 0x7f5594101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 >> 172.21.15.52:0/48916312 conn(0x7f5594078400 msgr2=0x7f55940ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.900+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 shutdown_connections 2024-06-02T23:39:18.901 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:18.901+0000 7f5599a0f700 1 -- 172.21.15.52:0/48916312 wait complete. 2024-06-02T23:39:18.902 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_metadata' removed 2024-06-02T23:39:19.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:18 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/48916312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T23:39:19.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:18 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/48916312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T23:39:19.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:18 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/48916312' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T23:39:19.543 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba shell --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-06-02T23:39:19.791 INFO:teuthology.orchestra.run.smithi052.stderr:Inferring config /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/mon.a/config 2024-06-02T23:39:20.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:19 smithi052 ceph-mon[32340]: pgmap v1255: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-02T23:39:20.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:19 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/48916312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T23:39:20.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:19 smithi052 ceph-mon[32340]: osdmap e92: 12 total, 12 up, 12 in 2024-06-02T23:39:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:19 smithi123 ceph-mon[35977]: pgmap v1255: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-02T23:39:20.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:19 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/48916312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T23:39:20.258 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:19 smithi123 ceph-mon[35977]: osdmap e92: 12 total, 12 up, 12 in 2024-06-02T23:39:20.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:19 smithi174 ceph-mon[35994]: pgmap v1255: 97 pgs: 97 active+clean; 421 MiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-06-02T23:39:20.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:19 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/48916312' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T23:39:20.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:19 smithi174 ceph-mon[35994]: osdmap e92: 12 total, 12 up, 12 in 2024-06-02T23:39:21.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:20 smithi052 ceph-mon[32340]: pgmap v1257: 65 pgs: 65 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:39:21.257 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:20 smithi123 ceph-mon[35977]: pgmap v1257: 65 pgs: 65 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:39:21.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:20 smithi174 ceph-mon[35994]: pgmap v1257: 65 pgs: 65 active+clean; 577 KiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:39:21.846 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 -- 172.21.15.52:0/3689340834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 msgr2=0x7f5b2410f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 --2- 172.21.15.52:0/3689340834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b2410f370 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c00b0a0 tx=0x7f5b1c00b3b0 comp rx=0 tx=0).stop 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 -- 172.21.15.52:0/3689340834 shutdown_connections 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 --2- 172.21.15.52:0/3689340834 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b2410f370 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 --2- 172.21.15.52:0/3689340834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 0x7f5b24108910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 --2- 172.21.15.52:0/3689340834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 0x7f5b2406b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 -- 172.21.15.52:0/3689340834 >> 172.21.15.52:0/3689340834 conn(0x7f5b24077400 msgr2=0x7f5b24077810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 -- 172.21.15.52:0/3689340834 shutdown_connections 2024-06-02T23:39:21.847 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.846+0000 7f5b29775700 1 -- 172.21.15.52:0/3689340834 wait complete. 2024-06-02T23:39:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 Processor -- start 2024-06-02T23:39:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 -- start start 2024-06-02T23:39:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 0x7f5b24101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b241019d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:21.849 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 0x7f5b24101f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 -- --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24112690 con 0x7f5b2406b560 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24112510 con 0x7f5b2406aba0 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b29775700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5b24112810 con 0x7f5b24108530 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b241019d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 --2- >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b241019d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.52:3300/0 says I am v2:172.21.15.52:39014/0 (socket says 172.21.15.52:39014) 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 -- 172.21.15.52:0/2011234233 learned_addr learned my addr 172.21.15.52:0/2011234233 (peer_addr_for_me v2:172.21.15.52:0/0) 2024-06-02T23:39:21.850 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b237fe700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 0x7f5b24101f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b22ffd700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 0x7f5b24101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 -- 172.21.15.52:0/2011234233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 msgr2=0x7f5b24101f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 0x7f5b24101f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 -- 172.21.15.52:0/2011234233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 msgr2=0x7f5b24101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 0x7f5b24101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.847+0000 7f5b1bfff700 1 -- 172.21.15.52:0/2011234233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b241a70a0 con 0x7f5b2406b560 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b237fe700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 0x7f5b24101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b22ffd700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 0x7f5b24101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b1bfff700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b241019d0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c9b0 tx=0x7f5b0c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c007b00 con 0x7f5b2406b560 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 2 ==== config(26 keys) ==== 1102+0+0 (secure 0 0 0) 0x7f5b0c007ca0 con 0x7f5b2406b560 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b0c018730 con 0x7f5b2406b560 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b241a7390 con 0x7f5b2406b560 2024-06-02T23:39:21.851 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.848+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b241a7880 con 0x7f5b2406b560 2024-06-02T23:39:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.849+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b2410aa30 con 0x7f5b2406b560 2024-06-02T23:39:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.850+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 4 ==== mgrmap(e 18) ==== 137833+0+0 (secure 0 0 0) 0x7f5b0c00f450 con 0x7f5b2406b560 2024-06-02T23:39:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.851+0000 7f5b20ff9700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b100a2c20 0x7f5b100a50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-06-02T23:39:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.851+0000 7f5b22ffd700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b100a2c20 0x7f5b100a50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-06-02T23:39:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.851+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8087+0+0 (secure 0 0 0) 0x7f5b0c0cf0c0 con 0x7f5b2406b560 2024-06-02T23:39:21.852 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.851+0000 7f5b22ffd700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b100a2c20 0x7f5b100a50e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099d0 tx=0x7f5b14008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-06-02T23:39:21.856 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:21.855+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+186418 (secure 0 0 0) 0x7f5b0c014030 con 0x7f5b2406b560 2024-06-02T23:39:22.064 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.063+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 --> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5b24103bf0 con 0x7f5b2406b560 2024-06-02T23:39:22.116 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.115+0000 7f5b20ff9700 1 -- 172.21.15.52:0/2011234233 <== mon.0 v2:172.21.15.52:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v93) ==== 169+0+0 (secure 0 0 0) 0x7f5b0c09c060 con 0x7f5b2406b560 2024-06-02T23:39:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b100a2c20 msgr2=0x7f5b100a50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:22.120 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b100a2c20 0x7f5b100a50e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f5b140099d0 tx=0x7f5b14008040 comp rx=0 tx=0).stop 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 msgr2=0x7f5b241019d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b241019d0 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c9b0 tx=0x7f5b0c00cd70 comp rx=0 tx=0).stop 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 shutdown_connections 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:6800/2596667410,v1:172.21.15.52:6801/2596667410] conn(0x7f5b100a2c20 0x7f5b100a50e0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5b24108530 0x7f5b24101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5b2406aba0 0x7f5b24101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 --2- 172.21.15.52:0/2011234233 >> [v2:172.21.15.52:3300/0,v1:172.21.15.52:6789/0] conn(0x7f5b2406b560 0x7f5b241019d0 secure :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f5b0c00c9b0 tx=0x7f5b0c00cd70 comp rx=0 tx=0).stop 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.119+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 >> 172.21.15.52:0/2011234233 conn(0x7f5b24077400 msgr2=0x7f5b240fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.120+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 shutdown_connections 2024-06-02T23:39:22.121 INFO:teuthology.orchestra.run.smithi052.stderr:2024-06-02T23:39:22.120+0000 7f5b29775700 1 -- 172.21.15.52:0/2011234233 wait complete. 2024-06-02T23:39:22.122 INFO:teuthology.orchestra.run.smithi052.stderr:pool 'cephfs_data' removed 2024-06-02T23:39:22.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:22 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2011234233' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T23:39:22.399 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:22 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2011234233' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T23:39:22.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:22 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2011234233' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-06-02T23:39:23.017 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:39:23.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:39:23.096 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:39:23.137 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-06-02T23:39:23.137 DEBUG:teuthology.orchestra.run.smithi052:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-02T23:39:23.153 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-02T23:39:23.169 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-06-02T23:39:23.203 INFO:tasks.cephadm:Stopping all daemons... 2024-06-02T23:39:23.203 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-06-02T23:39:23.203 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a 2024-06-02T23:39:23.243 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 ceph-mon[32340]: from='client.? 172.21.15.52:0/2011234233' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T23:39:23.243 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 ceph-mon[32340]: osdmap e93: 12 total, 12 up, 12 in 2024-06-02T23:39:23.243 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 ceph-mon[32340]: pgmap v1259: 1 pgs: 1 active+clean; 577 KiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:39:23.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:23 smithi174 ceph-mon[35994]: from='client.? 172.21.15.52:0/2011234233' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T23:39:23.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:23 smithi174 ceph-mon[35994]: osdmap e93: 12 total, 12 up, 12 in 2024-06-02T23:39:23.394 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:23 smithi174 ceph-mon[35994]: pgmap v1259: 1 pgs: 1 active+clean; 577 KiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:39:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:23 smithi123 ceph-mon[35977]: from='client.? 172.21.15.52:0/2011234233' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-06-02T23:39:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:23 smithi123 ceph-mon[35977]: osdmap e93: 12 total, 12 up, 12 in 2024-06-02T23:39:23.507 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:23 smithi123 ceph-mon[35977]: pgmap v1259: 1 pgs: 1 active+clean; 577 KiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail 2024-06-02T23:39:23.553 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 systemd[1]: Stopping Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:23.895 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a[32336]: 2024-06-02T23:39:23.552+0000 7f4aea73d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:23.895 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a[32336]: 2024-06-02T23:39:23.552+0000 7f4aea73d700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-06-02T23:39:23.895 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 podman[163410]: 2024-06-02 23:39:23.624132073 +0000 UTC m=+0.208103058 container died a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-02T23:39:24.149 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 podman[163410]: 2024-06-02 23:39:23.977345695 +0000 UTC m=+0.561316681 container cleanup a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS) 2024-06-02T23:39:24.150 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:23 smithi052 bash[163410]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a 2024-06-02T23:39:24.524 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service' 2024-06-02T23:39:24.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:24 smithi052 podman[163424]: 2024-06-02 23:39:24.303866187 +0000 UTC m=+0.676476814 container remove a0857bbe9f85566e07bba301707703bbc22c0070718a0eebb12664b3d898541f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-06-02T23:39:24.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:24 smithi052 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service: Deactivated successfully. 2024-06-02T23:39:24.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:24 smithi052 systemd[1]: Stopped Ceph mon.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:39:24.588 INFO:journalctl@ceph.mon.a.smithi052.stdout:Jun 02 23:39:24 smithi052 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.a.service: Consumed 40.566s CPU time. 2024-06-02T23:39:24.615 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:24.615 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-06-02T23:39:24.615 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-06-02T23:39:24.615 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.b 2024-06-02T23:39:25.007 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:24 smithi123 systemd[1]: Stopping Ceph mon.b for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:25.306 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-b[35972]: 2024-06-02T23:39:25.040+0000 7ff5ceb68700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:25.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-b[35972]: 2024-06-02T23:39:25.040+0000 7ff5ceb68700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-06-02T23:39:25.307 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:25 smithi123 podman[76847]: 2024-06-02 23:39:25.082086275 +0000 UTC m=+0.169843854 container died f4179c996544911c2bf65c52c29ade61a6175e4221bd9d1446c407b79c88aaf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-b, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-06-02T23:39:25.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:25 smithi123 podman[76847]: 2024-06-02 23:39:25.466285558 +0000 UTC m=+0.554043142 container cleanup f4179c996544911c2bf65c52c29ade61a6175e4221bd9d1446c407b79c88aaf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:39:25.561 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:25 smithi123 bash[76847]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-b 2024-06-02T23:39:25.905 INFO:journalctl@ceph.mon.b.smithi123.stdout:Jun 02 23:39:25 smithi123 podman[76860]: 2024-06-02 23:39:25.684643714 +0000 UTC m=+0.606113677 container remove f4179c996544911c2bf65c52c29ade61a6175e4221bd9d1446c407b79c88aaf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:39:25.919 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.b.service' 2024-06-02T23:39:25.999 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:26.000 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-06-02T23:39:26.000 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-06-02T23:39:26.000 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.c 2024-06-02T23:39:26.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:26 smithi174 systemd[1]: Stopping Ceph mon.c for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-c[35968]: 2024-06-02T23:39:26.500+0000 7f96d6ae7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:26.893 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-c[35968]: 2024-06-02T23:39:26.500+0000 7f96d6ae7700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-06-02T23:39:26.894 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:26 smithi174 podman[78427]: 2024-06-02 23:39:26.695337651 +0000 UTC m=+0.380697658 container died 6521b80193e7f12871097e87bd210463c4250fa03947f260c1cf401afa79b389 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-c, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-06-02T23:39:27.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:27 smithi174 podman[78427]: 2024-06-02 23:39:27.067984806 +0000 UTC m=+0.753344810 container cleanup 6521b80193e7f12871097e87bd210463c4250fa03947f260c1cf401afa79b389 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-c, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:39:27.352 INFO:journalctl@ceph.mon.c.smithi174.stdout:Jun 02 23:39:27 smithi174 bash[78427]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mon-c 2024-06-02T23:39:27.493 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mon.c.service' 2024-06-02T23:39:27.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:27.544 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-06-02T23:39:27.544 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-06-02T23:39:27.544 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x 2024-06-02T23:39:27.848 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:39:27 smithi052 systemd[1]: Stopping Ceph mgr.x for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:28.150 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:39:27 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:39:27.847+0000 7f28b97a2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:28.150 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:39:27 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x[32570]: 2024-06-02T23:39:27.847+0000 7f28b97a2700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-06-02T23:39:28.150 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:39:27 smithi052 podman[163531]: 2024-06-02 23:39:27.879359963 +0000 UTC m=+0.134028932 container died b64f2c3c3485e7f08c2e339b2abd54ec17e5edc1c408e73005533f62d5b1100c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-06-02T23:39:28.625 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:39:28 smithi052 podman[163531]: 2024-06-02 23:39:28.35741932 +0000 UTC m=+0.612088288 container cleanup b64f2c3c3485e7f08c2e339b2abd54ec17e5edc1c408e73005533f62d5b1100c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:3d89accfe3201286f000e065a1faea2a88c7a6ba, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:39:28.625 INFO:journalctl@ceph.mgr.x.smithi052.stdout:Jun 02 23:39:28 smithi052 bash[163531]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-x 2024-06-02T23:39:28.804 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.x.service' 2024-06-02T23:39:28.841 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:28.841 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-06-02T23:39:28.841 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-06-02T23:39:28.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.y 2024-06-02T23:39:29.184 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:39:28 smithi123 systemd[1]: Stopping Ceph mgr.y for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:29.480 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:39:29 smithi123 podman[76980]: 2024-06-02 23:39:29.184283369 +0000 UTC m=+0.150257985 container died fd83e0704dfe241fa0f540a4d7352b75c4a8549495aaa66ac8a5e73fa29e60bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:39:29.757 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:39:29 smithi123 podman[76980]: 2024-06-02 23:39:29.572334881 +0000 UTC m=+0.538309486 container cleanup fd83e0704dfe241fa0f540a4d7352b75c4a8549495aaa66ac8a5e73fa29e60bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS) 2024-06-02T23:39:29.757 INFO:journalctl@ceph.mgr.y.smithi123.stdout:Jun 02 23:39:29 smithi123 bash[76980]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-y 2024-06-02T23:39:30.001 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.y.service' 2024-06-02T23:39:30.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:30.037 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-06-02T23:39:30.037 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-06-02T23:39:30.037 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.z 2024-06-02T23:39:30.343 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:39:30 smithi174 systemd[1]: Stopping Ceph mgr.z for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:30.638 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:39:30 smithi174 podman[78550]: 2024-06-02 23:39:30.343644427 +0000 UTC m=+0.113758353 container died 3a242d6759ea6c0145f3d66872fc346eecb431185a75d8a73e113c8706d0fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True) 2024-06-02T23:39:30.893 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:39:30 smithi174 podman[78550]: 2024-06-02 23:39:30.715140463 +0000 UTC m=+0.485254392 container cleanup 3a242d6759ea6c0145f3d66872fc346eecb431185a75d8a73e113c8706d0fa8b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-02T23:39:30.894 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Jun 02 23:39:30 smithi174 bash[78550]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mgr-z 2024-06-02T23:39:31.162 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mgr.z.service' 2024-06-02T23:39:31.197 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:31.197 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-06-02T23:39:31.197 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-06-02T23:39:31.197 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.0 2024-06-02T23:39:31.521 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:31 smithi052 systemd[1]: Stopping Ceph osd.0 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:31.900 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0[44718]: 2024-06-02T23:39:31.519+0000 7f4c428b9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:31.900 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0[44718]: 2024-06-02T23:39:31.519+0000 7f4c428b9700 -1 osd.0 93 *** Got signal Terminated *** 2024-06-02T23:39:31.900 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0[44718]: 2024-06-02T23:39:31.519+0000 7f4c428b9700 -1 osd.0 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:39:37.270 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:36 smithi052 podman[163651]: 2024-06-02 23:39:36.916068072 +0000 UTC m=+5.516558265 container died c09cb70c2a3b2433924a82a976301a29ef5b67a73a6f905b7a443692601554e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:39:37.596 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:37 smithi052 podman[163651]: 2024-06-02 23:39:37.2702641 +0000 UTC m=+5.870754275 container cleanup c09cb70c2a3b2433924a82a976301a29ef5b67a73a6f905b7a443692601554e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-02T23:39:37.596 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:37 smithi052 bash[163651]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0 2024-06-02T23:39:37.899 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:37 smithi052 podman[163666]: 2024-06-02 23:39:37.596707475 +0000 UTC m=+0.674484217 container remove c09cb70c2a3b2433924a82a976301a29ef5b67a73a6f905b7a443692601554e0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:39:38.158 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:37 smithi052 podman[163756]: 2024-06-02 23:39:37.922810519 +0000 UTC m=+0.113299126 container create 578638e53a1a849d7cd7189a702c320a1d10b6eebc42c73631e8b6af771ec4a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0-deactivate, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:39:38.158 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:37 smithi052 podman[163756]: 2024-06-02 23:39:37.827183444 +0000 UTC m=+0.017672015 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:39:38.542 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:38 smithi052 podman[163756]: 2024-06-02 23:39:38.158507599 +0000 UTC m=+0.348996158 container init 578638e53a1a849d7cd7189a702c320a1d10b6eebc42c73631e8b6af771ec4a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-06-02T23:39:38.542 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:38 smithi052 podman[163756]: 2024-06-02 23:39:38.162834467 +0000 UTC m=+0.353323033 container start 578638e53a1a849d7cd7189a702c320a1d10b6eebc42c73631e8b6af771ec4a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-02T23:39:38.542 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:38 smithi052 podman[163756]: 2024-06-02 23:39:38.266918797 +0000 UTC m=+0.457407374 container attach 578638e53a1a849d7cd7189a702c320a1d10b6eebc42c73631e8b6af771ec4a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-06-02T23:39:38.869 INFO:journalctl@ceph.osd.0.smithi052.stdout:Jun 02 23:39:38 smithi052 podman[163756]: 2024-06-02 23:39:38.542657372 +0000 UTC m=+0.733145952 container died 578638e53a1a849d7cd7189a702c320a1d10b6eebc42c73631e8b6af771ec4a0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-02T23:39:39.244 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.0.service' 2024-06-02T23:39:39.289 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:39.289 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-06-02T23:39:39.289 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-06-02T23:39:39.289 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.1 2024-06-02T23:39:39.593 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:39 smithi123 systemd[1]: Stopping Ceph osd.1 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:40.007 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1[40532]: 2024-06-02T23:39:39.593+0000 7ff342417700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:40.007 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1[40532]: 2024-06-02T23:39:39.593+0000 7ff342417700 -1 osd.1 93 *** Got signal Terminated *** 2024-06-02T23:39:40.007 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1[40532]: 2024-06-02T23:39:39.593+0000 7ff342417700 -1 osd.1 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:39:45.436 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:45 smithi123 podman[77104]: 2024-06-02 23:39:45.142781113 +0000 UTC m=+5.653295990 container died 9f50060f829679187c0cf44682b5e1124c29bab8b1a2fbba5c3af66b55618d35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:39:45.757 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:45 smithi123 podman[77104]: 2024-06-02 23:39:45.518516388 +0000 UTC m=+6.029031260 container cleanup 9f50060f829679187c0cf44682b5e1124c29bab8b1a2fbba5c3af66b55618d35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-06-02T23:39:45.757 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:45 smithi123 bash[77104]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1 2024-06-02T23:39:46.150 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:45 smithi123 podman[77118]: 2024-06-02 23:39:45.77807377 +0000 UTC m=+0.635666568 container remove 9f50060f829679187c0cf44682b5e1124c29bab8b1a2fbba5c3af66b55618d35 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True) 2024-06-02T23:39:46.414 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:46 smithi123 podman[77207]: 2024-06-02 23:39:46.053564105 +0000 UTC m=+0.017387921 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:39:46.414 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:46 smithi123 podman[77207]: 2024-06-02 23:39:46.170693539 +0000 UTC m=+0.134517365 container create 3e64d32255382c253bc08269352047bf82409f8b0b3b602751667486f2d63436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-02T23:39:46.415 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:46 smithi123 podman[77207]: 2024-06-02 23:39:46.415036691 +0000 UTC m=+0.378860501 container init 3e64d32255382c253bc08269352047bf82409f8b0b3b602751667486f2d63436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-06-02T23:39:46.751 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:46 smithi123 podman[77207]: 2024-06-02 23:39:46.418792841 +0000 UTC m=+0.382616645 container start 3e64d32255382c253bc08269352047bf82409f8b0b3b602751667486f2d63436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-06-02T23:39:46.751 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:46 smithi123 podman[77207]: 2024-06-02 23:39:46.506689418 +0000 UTC m=+0.470513222 container attach 3e64d32255382c253bc08269352047bf82409f8b0b3b602751667486f2d63436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:39:47.007 INFO:journalctl@ceph.osd.1.smithi123.stdout:Jun 02 23:39:46 smithi123 podman[77207]: 2024-06-02 23:39:46.751375661 +0000 UTC m=+0.715199477 container died 3e64d32255382c253bc08269352047bf82409f8b0b3b602751667486f2d63436 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:39:47.371 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.1.service' 2024-06-02T23:39:47.411 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:47.411 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-06-02T23:39:47.412 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-06-02T23:39:47.412 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.2 2024-06-02T23:39:47.727 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:47 smithi174 systemd[1]: Stopping Ceph osd.2 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:48.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2[41129]: 2024-06-02T23:39:47.727+0000 7f325cc27700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:48.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2[41129]: 2024-06-02T23:39:47.727+0000 7f325cc27700 -1 osd.2 93 *** Got signal Terminated *** 2024-06-02T23:39:48.143 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2[41129]: 2024-06-02T23:39:47.727+0000 7f325cc27700 -1 osd.2 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:39:53.596 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:53 smithi174 podman[78675]: 2024-06-02 23:39:53.318776351 +0000 UTC m=+5.703835824 container died e1a5f09d058b0ac0cd7d6deda07f21a6f1b84b0a89e81ba1b72644a6b50e7864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-06-02T23:39:53.893 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:53 smithi174 podman[78675]: 2024-06-02 23:39:53.685765323 +0000 UTC m=+6.070824786 container cleanup e1a5f09d058b0ac0cd7d6deda07f21a6f1b84b0a89e81ba1b72644a6b50e7864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD) 2024-06-02T23:39:53.893 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:53 smithi174 bash[78675]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2 2024-06-02T23:39:54.318 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:53 smithi174 podman[78702]: 2024-06-02 23:39:53.987227234 +0000 UTC m=+0.669194745 container remove e1a5f09d058b0ac0cd7d6deda07f21a6f1b84b0a89e81ba1b72644a6b50e7864 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:39:54.634 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:54 smithi174 podman[78794]: 2024-06-02 23:39:54.221959183 +0000 UTC m=+0.017015169 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:39:54.634 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:54 smithi174 podman[78794]: 2024-06-02 23:39:54.321354677 +0000 UTC m=+0.116410634 container create e1b7e6f919c9a07060faa0b3481cad7454a2560e01a53e807f784883eb1cc2e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-06-02T23:39:54.634 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:54 smithi174 podman[78794]: 2024-06-02 23:39:54.550728931 +0000 UTC m=+0.345784895 container init e1b7e6f919c9a07060faa0b3481cad7454a2560e01a53e807f784883eb1cc2e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-06-02T23:39:54.634 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:54 smithi174 podman[78794]: 2024-06-02 23:39:54.554315861 +0000 UTC m=+0.349371829 container start e1b7e6f919c9a07060faa0b3481cad7454a2560e01a53e807f784883eb1cc2e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-02T23:39:54.634 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:54 smithi174 podman[78794]: 2024-06-02 23:39:54.63407323 +0000 UTC m=+0.429129210 container attach e1b7e6f919c9a07060faa0b3481cad7454a2560e01a53e807f784883eb1cc2e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-06-02T23:39:54.893 INFO:journalctl@ceph.osd.2.smithi174.stdout:Jun 02 23:39:54 smithi174 podman[78794]: 2024-06-02 23:39:54.858934671 +0000 UTC m=+0.653990639 container died e1b7e6f919c9a07060faa0b3481cad7454a2560e01a53e807f784883eb1cc2e4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-02T23:39:55.472 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.2.service' 2024-06-02T23:39:55.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:39:55.515 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-06-02T23:39:55.516 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-06-02T23:39:55.516 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.3 2024-06-02T23:39:55.849 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:39:55 smithi052 systemd[1]: Stopping Ceph osd.3 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:39:56.129 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:39:55 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:39:55.848+0000 7fdb135c4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:39:56.129 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:39:55 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:39:55.848+0000 7fdb135c4700 -1 osd.3 93 *** Got signal Terminated *** 2024-06-02T23:39:56.129 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:39:55 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:39:55.848+0000 7fdb135c4700 -1 osd.3 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:39:58.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:39:58 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:39:58.210+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:39:58.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:39:58 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:39:58.561+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:39:59.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:39:59 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:39:59.189+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:39:59.649 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:39:59 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:39:59.541+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:39:59.650 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:39:59 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:39:59.397+0000 7fdb0bbb0700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:35.339658+0000 front 2024-06-02T23:39:35.339620+0000 (oldest deadline 2024-06-02T23:39:58.839910+0000) 2024-06-02T23:39:59.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:39:59 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:39:59.491+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:00.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:00 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:00.146+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:00.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:00.331+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:00.900 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:00 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3[51632]: 2024-06-02T23:40:00.433+0000 7fdb0bbb0700 -1 osd.3 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:35.339658+0000 front 2024-06-02T23:39:35.339620+0000 (oldest deadline 2024-06-02T23:39:58.839910+0000) 2024-06-02T23:40:00.900 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:00 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:00.499+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:00.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:00 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:00.549+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:01.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:00.888+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:01.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:01 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:01.186+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:01.608 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:01 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:01.516+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:01.608 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:01 smithi052 podman[163877]: 2024-06-02 23:40:01.313088396 +0000 UTC m=+5.589696003 container died a17a66af35938dad6df1908e3c566ca8f24e9f550ed2a5fd20f6183751beddf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-06-02T23:40:01.609 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:01 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:01.471+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:01.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:01.378+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:01.900 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:01 smithi052 podman[163877]: 2024-06-02 23:40:01.790157858 +0000 UTC m=+6.066765464 container cleanup a17a66af35938dad6df1908e3c566ca8f24e9f550ed2a5fd20f6183751beddf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-02T23:40:01.900 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:01 smithi052 bash[163877]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3 2024-06-02T23:40:02.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:01.893+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:02.399 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:02 smithi052 podman[163906]: 2024-06-02 23:40:02.066745484 +0000 UTC m=+0.744953882 container remove a17a66af35938dad6df1908e3c566ca8f24e9f550ed2a5fd20f6183751beddf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True) 2024-06-02T23:40:02.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:02 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:02.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:02.697 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:02.486+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:02.698 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:02 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:02.479+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:02.698 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:02 smithi052 podman[163997]: 2024-06-02 23:40:02.350234426 +0000 UTC m=+0.017916330 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:02.698 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:02 smithi052 podman[163997]: 2024-06-02 23:40:02.467554246 +0000 UTC m=+0.135236136 container create 86e5ed268741a010f3d11a11e10f1ee89ad6c22c22fe1be94e72b58653767469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:02.698 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:02 smithi052 podman[163997]: 2024-06-02 23:40:02.695121335 +0000 UTC m=+0.362803240 container init 86e5ed268741a010f3d11a11e10f1ee89ad6c22c22fe1be94e72b58653767469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5) 2024-06-02T23:40:02.698 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:02 smithi052 podman[163997]: 2024-06-02 23:40:02.698478503 +0000 UTC m=+0.366160394 container start 86e5ed268741a010f3d11a11e10f1ee89ad6c22c22fe1be94e72b58653767469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc) 2024-06-02T23:40:02.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:02.393+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:03.007 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:02 smithi052 podman[163997]: 2024-06-02 23:40:02.822099725 +0000 UTC m=+0.489781617 container attach 86e5ed268741a010f3d11a11e10f1ee89ad6c22c22fe1be94e72b58653767469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:40:03.299 INFO:journalctl@ceph.osd.3.smithi052.stdout:Jun 02 23:40:03 smithi052 podman[163997]: 2024-06-02 23:40:03.008176962 +0000 UTC m=+0.675858870 container died 86e5ed268741a010f3d11a11e10f1ee89ad6c22c22fe1be94e72b58653767469 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:40:03.351 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:02.928+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:03.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:03 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:03.199+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:03.550 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:03 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:03.499+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:03.550 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:03 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:03.449+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:03.581 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.3.service' 2024-06-02T23:40:03.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:03.623 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-06-02T23:40:03.624 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-06-02T23:40:03.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.4 2024-06-02T23:40:03.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:03.351+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:03.938 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:03 smithi123 systemd[1]: Stopping Ceph osd.4 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:04.208 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:03 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:40:03.938+0000 7fd6b592a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:04.208 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:03 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:40:03.938+0000 7fd6b592a700 -1 osd.4 93 *** Got signal Terminated *** 2024-06-02T23:40:04.209 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:03 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:40:03.938+0000 7fd6b592a700 -1 osd.4 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:04.378 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:03.948+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:04.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:04 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:04.208+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:04.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:04.378+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:04.900 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:04 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:04.468+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:04.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:04 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:04.477+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:05.347 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:04.989+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:05.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:05 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:05.213+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:05.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:05 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:05.346+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:05.899 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:05.440+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:05.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:05 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:05.451+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:06.366 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:05 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:05.996+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:06.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:06 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:06.244+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:06.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:06 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:06.366+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:06.900 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:06 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:06.412+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:06.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:06 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:06.493+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:06.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:06 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:06.493+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:07.393 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:06 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:06.948+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:07.394 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:06 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:06.948+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:07.394 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:06 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:06.995+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:07.466 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:07.431+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:07.466 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:07.431+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:07.507 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:07 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:40:07.431+0000 7fd6b1f1e700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.145347+0000 front 2024-06-02T23:39:44.145370+0000 (oldest deadline 2024-06-02T23:40:07.045512+0000) 2024-06-02T23:40:07.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:07 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:07.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:07.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:07 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:07.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:07.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:07 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:07.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:07.899 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:07.466+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:07.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:07.466+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:08.394 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:07 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:07.916+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:08.394 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:07 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:07.916+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:08.394 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:08 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:08.017+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:08.507 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:08 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4[47097]: 2024-06-02T23:40:08.385+0000 7fd6b1f1e700 -1 osd.4 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.145347+0000 front 2024-06-02T23:39:44.145370+0000 (oldest deadline 2024-06-02T23:40:07.045512+0000) 2024-06-02T23:40:08.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:08 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:08.230+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:08.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:08 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:08.231+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:08.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:08.386+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:08.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:08.386+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:08.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:08.499+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:08.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:08 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:08.499+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:08.878 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:08 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:08.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:08.878 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:08 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:08.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:09.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:08 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:08.878+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:09.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:08 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:08.878+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:09.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:09 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:09.013+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:09.507 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:09 smithi123 podman[77326]: 2024-06-02 23:40:09.346598823 +0000 UTC m=+5.520943193 container died ecdcdb6db14dbbd666cea79a6b72aa7ff2c5efda8fbb6806add32ceb62832706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528) 2024-06-02T23:40:09.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:09 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:09.200+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:09.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:09 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:09.200+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:09.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:09 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:09.342+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:09.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:09 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:09.342+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:09.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:09 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:09.490+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:09.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:09 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:09.490+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:09.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:09 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:09.425+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:09.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:09 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:09.425+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:09.964 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:09 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:09.631+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:09.964 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:09 smithi123 podman[77326]: 2024-06-02 23:40:09.688380735 +0000 UTC m=+5.862725094 container cleanup ecdcdb6db14dbbd666cea79a6b72aa7ff2c5efda8fbb6806add32ceb62832706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2) 2024-06-02T23:40:09.964 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:09 smithi123 bash[77326]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4 2024-06-02T23:40:10.257 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:09 smithi123 podman[77341]: 2024-06-02 23:40:09.964782497 +0000 UTC m=+0.621162237 container remove ecdcdb6db14dbbd666cea79a6b72aa7ff2c5efda8fbb6806add32ceb62832706 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-02T23:40:10.257 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:10 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:10.172+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:10.257 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:10 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:10.172+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:10.393 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:09 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:09.897+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:10.393 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:09 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:09.897+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:10.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:09 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:09.966+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:10.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:10 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:10.393+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:10.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:10 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:10.393+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:10.649 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:10 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:10.321+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:10.649 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:10 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:10.321+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:10.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:10 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:10.483+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:10.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:10 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:10.483+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:10.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:10 smithi123 podman[77430]: 2024-06-02 23:40:10.227741164 +0000 UTC m=+0.017144022 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:10.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:10 smithi123 podman[77430]: 2024-06-02 23:40:10.340630939 +0000 UTC m=+0.130033778 container create 867bdb56a5830bab7c546115f2cd2aac8931741e9dd52e683f008fb63c3b434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-02T23:40:10.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:10 smithi123 podman[77430]: 2024-06-02 23:40:10.567951851 +0000 UTC m=+0.357354690 container init 867bdb56a5830bab7c546115f2cd2aac8931741e9dd52e683f008fb63c3b434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-06-02T23:40:10.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:10 smithi123 podman[77430]: 2024-06-02 23:40:10.571330187 +0000 UTC m=+0.360733035 container start 867bdb56a5830bab7c546115f2cd2aac8931741e9dd52e683f008fb63c3b434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-06-02T23:40:10.659 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:10 smithi123 podman[77430]: 2024-06-02 23:40:10.659695903 +0000 UTC m=+0.449098743 container attach 867bdb56a5830bab7c546115f2cd2aac8931741e9dd52e683f008fb63c3b434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-02T23:40:11.007 INFO:journalctl@ceph.osd.4.smithi123.stdout:Jun 02 23:40:10 smithi123 podman[77430]: 2024-06-02 23:40:10.863899422 +0000 UTC m=+0.653302264 container died 867bdb56a5830bab7c546115f2cd2aac8931741e9dd52e683f008fb63c3b434a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-02T23:40:11.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:10 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:10.676+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:11.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:10 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:10.956+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:11.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:10 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:10.861+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:11.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:10 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:10.861+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:11.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:11.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:11.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:11.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:11.547 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.4.service' 2024-06-02T23:40:11.590 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:11.591 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-06-02T23:40:11.591 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-06-02T23:40:11.591 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.5 2024-06-02T23:40:11.649 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:11 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:11.433+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:11.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:11 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:11.433+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:11.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:11 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:11.361+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:11.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:11 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:11.361+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:11.820 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:11 smithi174 systemd[1]: Stopping Ceph osd.5 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:11.820 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:11.420+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:11.820 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:11.420+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:12.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:11 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:11.672+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:12.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:11.920+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:12.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:11.820+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:12.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:11.820+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:12.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:11.913+0000 7fef039d7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:12.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:11.913+0000 7fef039d7700 -1 osd.5 93 *** Got signal Terminated *** 2024-06-02T23:40:12.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:11 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:11.913+0000 7fef039d7700 -1 osd.5 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:12.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:12 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:12.234+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:12.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:12 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:12.234+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:12.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:12 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:12.438+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:12.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:12 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:12.438+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:12.899 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:12.402+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:12.900 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:12.402+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:12.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:12.467+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:12.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:12 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:12.467+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:13.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:12 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:12.694+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:13.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:12 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:12.943+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:13.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:12 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:12.794+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:13.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:12 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:12.794+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:13.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:13.515+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:13.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:13.515+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:13.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:13.365+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:13.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:13.365+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:13.679 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:13 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:13.280+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:13.679 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:13 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:13.280+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:13.798 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:13 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:13.433+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:13.799 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:13 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:13.433+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:14.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:13 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:13.679+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:14.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:13 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:13.799+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:14.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:13 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:13.799+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:14.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:13 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:13.988+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:14.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:13 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:13.988+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:14.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:14 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:14.381+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:14.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:14 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:14.381+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:14.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:14 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:14.550+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:14.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:14 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:14.550+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:14.708 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:14 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:14.282+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:14.708 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:14 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:14.282+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:14.813 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:14 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:14.401+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:14.813 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:14 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:14.401+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:15.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:14 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:14.708+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:15.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:14 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:14.813+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:15.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:14 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:14.813+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:15.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:14 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:14.994+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:15.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:14 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:14.994+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:15.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:15 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:15.363+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:15.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:15 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:15.363+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:15.649 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:15.568+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:15.649 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:15.568+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:15.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:15.341+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:15.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:15.341+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:15.718 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:15 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:15.300+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:15.719 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:15 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:15.300+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:15.719 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:15 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:15.300+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:16.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:15 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:15.718+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:16.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:16.001+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:16.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:16.001+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:16.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:15 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:15.809+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:16.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:15 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:15.809+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:16.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:15 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:15.809+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.496420+0000 front 2024-06-02T23:39:50.496444+0000 (oldest deadline 2024-06-02T23:40:15.196625+0000) 2024-06-02T23:40:16.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:16.376+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:16.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:16.376+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:16.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:16.376+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:16.649 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:16 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:16.613+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:16.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:16 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:16.613+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:16.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:16 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:16.364+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:16.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:16 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:16.364+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:16.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:16 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:16.364+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:16.708 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:16 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:16.310+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:16.708 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:16 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:16.310+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:16.708 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:16 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:16.310+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:17.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:16 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:16.707+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:17.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:17.017+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:17.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:17.017+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:17.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:16.802+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:36.191976+0000 front 2024-06-02T23:39:36.192009+0000 (oldest deadline 2024-06-02T23:40:00.792964+0000) 2024-06-02T23:40:17.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:16.802+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.494239+0000 front 2024-06-02T23:39:42.494223+0000 (oldest deadline 2024-06-02T23:40:06.495092+0000) 2024-06-02T23:40:17.144 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:16 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5[48018]: 2024-06-02T23:40:16.802+0000 7feefbfc3700 -1 osd.5 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.496420+0000 front 2024-06-02T23:39:50.496444+0000 (oldest deadline 2024-06-02T23:40:15.196625+0000) 2024-06-02T23:40:17.643 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:17 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:17.387+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:17.643 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:17 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:17.387+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:17.644 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:17 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:17.387+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:17.754 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:17 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:17.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:17.754 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:17 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:17.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:17.754 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:17 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:17.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:17.823 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:17 smithi174 podman[78900]: 2024-06-02 23:40:17.518181737 +0000 UTC m=+5.730372081 container died 6ec5aa085ddb631d459b95bd95b7363cc63501d1ff407e96a60ebb4ad98a65af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2) 2024-06-02T23:40:17.823 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:17.397+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:17.823 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:17.397+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:17.823 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:17.397+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:17.899 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:17 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:17.644+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:17.899 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:17 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:17.644+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:17.899 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:17 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:17.644+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:18.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:17 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:17.754+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:18.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:17 smithi174 podman[78900]: 2024-06-02 23:40:17.922148586 +0000 UTC m=+6.134338929 container cleanup 6ec5aa085ddb631d459b95bd95b7363cc63501d1ff407e96a60ebb4ad98a65af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-06-02T23:40:18.143 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:17 smithi174 bash[78900]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5 2024-06-02T23:40:18.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:17.978+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:18.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:17 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:17.978+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:18.404 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:18 smithi174 podman[78919]: 2024-06-02 23:40:18.148538494 +0000 UTC m=+0.626468138 container remove 6ec5aa085ddb631d459b95bd95b7363cc63501d1ff407e96a60ebb4ad98a65af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-02T23:40:18.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:18.242+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:18.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:18.242+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:18.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:18.242+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:18.605 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:18.350+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:18.605 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:18.350+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:18.605 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:18.350+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:18.667 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:18 smithi174 podman[79012]: 2024-06-02 23:40:18.371619402 +0000 UTC m=+0.019485395 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:18.667 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:18 smithi174 podman[79012]: 2024-06-02 23:40:18.512808777 +0000 UTC m=+0.160674816 container create 3eb786741c0c08e6a1dc53f37db0cb1e58e85c7878fc923c77b22af7f3ac0cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:18.667 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:18 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:18.444+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:18.667 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:18 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:18.444+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:18.667 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:18 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:18.444+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:18.899 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:18.605+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:18.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:18.605+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:18.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:18 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:18.605+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:19.074 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:18 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:18.952+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:19.074 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:18 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:18.952+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:19.074 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:18 smithi174 podman[79012]: 2024-06-02 23:40:18.777258146 +0000 UTC m=+0.425124135 container init 3eb786741c0c08e6a1dc53f37db0cb1e58e85c7878fc923c77b22af7f3ac0cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5-deactivate, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:40:19.074 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:18 smithi174 podman[79012]: 2024-06-02 23:40:18.780621239 +0000 UTC m=+0.428487217 container start 3eb786741c0c08e6a1dc53f37db0cb1e58e85c7878fc923c77b22af7f3ac0cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528) 2024-06-02T23:40:19.074 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:18 smithi174 podman[79012]: 2024-06-02 23:40:18.877268915 +0000 UTC m=+0.525134915 container attach 3eb786741c0c08e6a1dc53f37db0cb1e58e85c7878fc923c77b22af7f3ac0cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:40:19.074 INFO:journalctl@ceph.osd.5.smithi174.stdout:Jun 02 23:40:19 smithi174 podman[79012]: 2024-06-02 23:40:19.074388129 +0000 UTC m=+0.722254125 container died 3eb786741c0c08e6a1dc53f37db0cb1e58e85c7878fc923c77b22af7f3ac0cee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-5-deactivate, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0) 2024-06-02T23:40:19.215 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:18 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:18.791+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:19.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:19 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:19.215+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:19.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:19 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:19.215+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:19.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:19 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:19.215+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:19.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:19.575+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:19.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:19.575+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:19.650 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:19.575+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:19.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:19.362+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:19.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:19.362+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:19.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:19 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:19.362+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:19.706 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:19.431+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:19.706 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:19.431+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:19.706 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:19.431+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:19.739 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.5.service' 2024-06-02T23:40:19.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:19.780 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-06-02T23:40:19.780 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-06-02T23:40:19.780 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.6 2024-06-02T23:40:20.111 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:19 smithi052 systemd[1]: Stopping Ceph osd.6 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:20.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:19.940+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:20.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:19 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:19.940+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:20.220 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:19 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:19.785+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:20.220 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:19 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:19.785+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:20.400 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:20.111+0000 7f068e775700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:20.400 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:20.111+0000 7f068e775700 -1 osd.6 93 *** Got signal Terminated *** 2024-06-02T23:40:20.400 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:20.111+0000 7f068e775700 -1 osd.6 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:20.400 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:20.313+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:20.400 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:20.313+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:20.400 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:20.313+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:20.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:20 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:20.220+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:20.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:20 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:20.220+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:20.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:20 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:20.220+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:20.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:20 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:20.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:20.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:20 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:20.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:20.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:20 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:20.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:20.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:20.621+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:20.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:20.621+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:20.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:20.621+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:21.223 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:20 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:20.818+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:21.223 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:20 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:20.818+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:21.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:20 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:20.908+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:21.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:20 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:20.908+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:21.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:21 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:21.222+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:21.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:21 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:21.222+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:21.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:21 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:21.222+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:21.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:21.295+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:21.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:21.295+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:21.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:21.295+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:21.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:21.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:21.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:21.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:21.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:21.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:22.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:21.654+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:22.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:21.654+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:22.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:21 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:21.654+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:22.212 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:21 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:21.808+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:22.212 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:21 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:21.808+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:22.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:21.903+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:22.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:21 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:21.903+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:22.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:22.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:22.574 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:22.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:22.574 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:22.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:22.649 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:22.325+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:22.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:22.325+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:22.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:22.326+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:22.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:22.419+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:22.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:22.419+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:22.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:22.419+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:23.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:22.680+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:23.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:22.680+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:23.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:22 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:22.680+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:23.185 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:22.758+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:23.185 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:22.758+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:23.185 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:22 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:22.758+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:23.383 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:22.920+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:23.383 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:22 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:22.920+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:23.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:23.185+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:23.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:23.185+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:23.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:23.185+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:23.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:23.383+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:23.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:23.383+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:23.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:23.383+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:23.644 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:23.337+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:23.644 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:23.337+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:23.644 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:23.337+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:23.899 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:23.645+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:23.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:23.645+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:23.900 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:23 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:23.645+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:24.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:23.729+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:24.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:23.729+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:24.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:23.729+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:24.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:23.919+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:24.394 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:23 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:23.919+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:24.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:24.175+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:24.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:24.175+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:24.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:24.175+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:24.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:24.175+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:24.649 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:24.301+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.585001+0000 front 2024-06-02T23:39:33.585031+0000 (oldest deadline 2024-06-02T23:39:58.885085+0000) 2024-06-02T23:40:24.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:24.301+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.988037+0000 front 2024-06-02T23:39:43.988018+0000 (oldest deadline 2024-06-02T23:40:06.888065+0000) 2024-06-02T23:40:24.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:24.301+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.690040+0000 front 2024-06-02T23:39:52.689966+0000 (oldest deadline 2024-06-02T23:40:16.189891+0000) 2024-06-02T23:40:24.650 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6[58550]: 2024-06-02T23:40:24.301+0000 7f068ad69700 -1 osd.6 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.490758+0000 front 2024-06-02T23:39:58.490594+0000 (oldest deadline 2024-06-02T23:40:23.790797+0000) 2024-06-02T23:40:24.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:24.409+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:24.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:24.409+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:24.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:24.409+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:25.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:24.669+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:25.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:24.669+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:25.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:24.669+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:25.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:24 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:24.669+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:25.153 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:24.764+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:25.153 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:24.764+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:25.153 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:24 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:24.764+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:25.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:24.917+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:25.393 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:24 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:24.917+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:25.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:25.152+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:25.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:25.152+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:25.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:25.152+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:25.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:25.152+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:25.812 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:25 smithi052 podman[164102]: 2024-06-02 23:40:25.521523919 +0000 UTC m=+5.532502055 container died 3ca98a353a7002de51476e794a98a40ad791e7f9f9a94a68712b991254da626e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:40:25.812 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:25.678+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:25.812 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:25.678+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:25.812 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:25.678+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:25.813 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:25 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:25.678+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:25.876 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:25.406+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:25.876 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:25.406+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:25.876 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:25.406+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:26.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:25.717+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:26.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:25.717+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:26.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:25 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:25.717+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:26.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:25.876+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:26.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:25.876+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:26.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:25 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:25.876+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:26.149 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:25 smithi052 podman[164102]: 2024-06-02 23:40:25.961510523 +0000 UTC m=+5.972488658 container cleanup 3ca98a353a7002de51476e794a98a40ad791e7f9f9a94a68712b991254da626e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-06-02T23:40:26.149 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:25 smithi052 bash[164102]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6 2024-06-02T23:40:26.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:26.167+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:26.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:26.167+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:26.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:26.167+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:26.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:26.167+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:26.649 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:26 smithi052 podman[164117]: 2024-06-02 23:40:26.204670033 +0000 UTC m=+0.682961175 container remove 3ca98a353a7002de51476e794a98a40ad791e7f9f9a94a68712b991254da626e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-02T23:40:26.850 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:26.409+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:26.851 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:26.409+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:26.851 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:26.409+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:26.932 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:26 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:26.674+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:26.932 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:26 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:26.674+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:26.932 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:26 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:26.674+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:26.932 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:26 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:26.674+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:26.933 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:26 smithi052 podman[164219]: 2024-06-02 23:40:26.58411172 +0000 UTC m=+0.020134702 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:26.933 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:26 smithi052 podman[164219]: 2024-06-02 23:40:26.683726126 +0000 UTC m=+0.119749087 container create 1d4dac7e3d7634638ba56254ed25c581d1f6f746edbaef89748f39cc78f0b042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-02T23:40:27.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:26.742+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:27.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:26.742+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:27.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:26 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:26.742+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:27.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:26.850+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:27.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:26.850+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:27.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:26 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:26.850+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:27.242 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:26 smithi052 podman[164219]: 2024-06-02 23:40:26.933015697 +0000 UTC m=+0.369038652 container init 1d4dac7e3d7634638ba56254ed25c581d1f6f746edbaef89748f39cc78f0b042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:40:27.242 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:26 smithi052 podman[164219]: 2024-06-02 23:40:26.936927467 +0000 UTC m=+0.372950423 container start 1d4dac7e3d7634638ba56254ed25c581d1f6f746edbaef89748f39cc78f0b042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-06-02T23:40:27.242 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:27 smithi052 podman[164219]: 2024-06-02 23:40:27.033181526 +0000 UTC m=+0.469204515 container attach 1d4dac7e3d7634638ba56254ed25c581d1f6f746edbaef89748f39cc78f0b042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6-deactivate, RELEASE=squid-3d89acc, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:27.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:27.171+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:27.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:27.171+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:27.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:27.171+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:27.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:27.171+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:27.527 INFO:journalctl@ceph.osd.6.smithi052.stdout:Jun 02 23:40:27 smithi052 podman[164219]: 2024-06-02 23:40:27.243182212 +0000 UTC m=+0.679205184 container died 1d4dac7e3d7634638ba56254ed25c581d1f6f746edbaef89748f39cc78f0b042 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, CEPH_POINT_RELEASE=, ceph=True) 2024-06-02T23:40:27.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:27 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:27.370+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:27.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:27 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:27.370+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:27.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:27 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:27.370+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:27.724 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:27 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:27.723+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:27.724 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:27 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:27.723+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:27.724 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:27 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:27.723+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:27.729 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:27 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:27.723+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:27.946 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.6.service' 2024-06-02T23:40:27.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:27.986 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-06-02T23:40:27.987 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-06-02T23:40:27.987 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.7 2024-06-02T23:40:28.013 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:27.758+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:28.013 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:27.758+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:28.013 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:27 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:27.758+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:28.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:27 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:27.889+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:28.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:27 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:27.889+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:28.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:27 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:27.889+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:28.319 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 systemd[1]: Stopping Ceph osd.7 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:28.319 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:28.203+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:28.319 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:28.203+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:28.319 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:28.203+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:28.319 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:28.203+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:28.757 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:28.323+0000 7ffb01db6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:28.757 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:28.323+0000 7ffb01db6700 -1 osd.7 93 *** Got signal Terminated *** 2024-06-02T23:40:28.757 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:28.323+0000 7ffb01db6700 -1 osd.7 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:28.870 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:28 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:28.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:28.870 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:28 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:28.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:28.870 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:28 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:28.400+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:29.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:28 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:28.869+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:29.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:28 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:28.869+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:29.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:28 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:28.869+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:29.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:28 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:28.748+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:29.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:28 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:28.748+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:29.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:28 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:28.748+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:29.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:28 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:28.748+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:29.189 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:28.782+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:29.189 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:28.782+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:29.189 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:28.782+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:29.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:29.188+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:29.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:29.188+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:29.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:29.188+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:29.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:29.188+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:29.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:29 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:29.414+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:29.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:29 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:29.414+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:29.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:29 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:29.414+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:30.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:29.749+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:30.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:29.749+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:30.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:29 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:29.749+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:30.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:29 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:29.777+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:30.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:29 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:29.777+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:30.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:29 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:29.777+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:30.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:29 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:29.777+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:30.373 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:29 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:29.907+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:30.373 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:29 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:29.907+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:30.373 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:29 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:29.907+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:30.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:30.237+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:30.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:30.237+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:30.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:30.237+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:30.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:30.237+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:30.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:30.237+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:30.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:30.372+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:30.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:30.372+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:30.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:30.373+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:31.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:30.755+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:31.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:30.755+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:31.007 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:30 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:30.755+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:31.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:30.881+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:31.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:30.881+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:31.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:30.881+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:31.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:30 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:30.881+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:31.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:30.766+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:31.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:30.766+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:31.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:30.766+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:31.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:30.766+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:31.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:30 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:30.766+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:31.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:31.362+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:31.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:31.362+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:31.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:31.362+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:31.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:31.257+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:31.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:31.257+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:31.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:31.257+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:31.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:31.257+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:31.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:31.257+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:32.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:31.873+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:32.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:31.873+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:32.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:31.873+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:32.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:31 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:31.873+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:32.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:31.769+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:32.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:31.769+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:32.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:31.769+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:32.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:31.769+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:32.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:31 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:31.769+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:32.217 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:31.788+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:32.217 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:31.788+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:32.217 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:31 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:31.788+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:32.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:32.217+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:32.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:32.217+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:32.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:32.217+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:32.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:32.217+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:32.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:32.217+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:32.853 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:32.405+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:32.853 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:32.405+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:32.853 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:32.405+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:33.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:32.853+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:33.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:32.853+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:33.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:32.853+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:33.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:32 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:32.853+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:33.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:32 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:32.807+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:33.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:32 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:32.807+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:33.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:32 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:32.807+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:33.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:32 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:32.807+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:33.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:32 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:32.807+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:33.257 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:32.815+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.560633+0000 front 2024-06-02T23:39:43.560666+0000 (oldest deadline 2024-06-02T23:40:09.460841+0000) 2024-06-02T23:40:33.257 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:32.815+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.461192+0000 front 2024-06-02T23:39:49.461215+0000 (oldest deadline 2024-06-02T23:40:18.862046+0000) 2024-06-02T23:40:33.257 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:32 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7[53853]: 2024-06-02T23:40:32.815+0000 7ffafa3a2700 -1 osd.7 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:58.862849+0000 front 2024-06-02T23:39:58.862644+0000 (oldest deadline 2024-06-02T23:40:22.263272+0000) 2024-06-02T23:40:33.700 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:33.267+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:33.701 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:33.267+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:33.701 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:33.267+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:33.701 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:33.267+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:33.701 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:33.267+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:33.836 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:33.438+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:33.836 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:33.438+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:33.836 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:33.438+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:33.836 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:33.438+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:33.990 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:33 smithi123 podman[77536]: 2024-06-02 23:40:33.701120499 +0000 UTC m=+5.490559024 container died e283f4e5f1d6b0456abcbc2b8831c022df478d13f3facf20f1db30f56595ac98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-02T23:40:34.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:33.836+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:34.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:33.836+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:34.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:33.836+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:34.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:33 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:33.836+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:34.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:33 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:33.789+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:34.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:33 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:33.789+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:34.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:33 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:33.789+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:34.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:33 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:33.789+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:34.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:33 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:33.789+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:34.257 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 podman[77536]: 2024-06-02 23:40:34.081680374 +0000 UTC m=+5.871118908 container cleanup e283f4e5f1d6b0456abcbc2b8831c022df478d13f3facf20f1db30f56595ac98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-06-02T23:40:34.257 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 bash[77536]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7 2024-06-02T23:40:34.258 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:34.227+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:34.258 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:34.227+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:34.258 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:34.227+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:34.258 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:34.227+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:34.258 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:34 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:34.227+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:34.684 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 podman[77563]: 2024-06-02 23:40:34.291571986 +0000 UTC m=+0.585710657 container remove e283f4e5f1d6b0456abcbc2b8831c022df478d13f3facf20f1db30f56595ac98 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-06-02T23:40:34.827 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:34.435+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:34.827 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:34.435+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:34.827 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:34.435+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:34.827 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:34.435+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:34.952 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 podman[77654]: 2024-06-02 23:40:34.684224884 +0000 UTC m=+0.091584159 container create 068d67e2da7a703b10f6d77d3b7741c312936966bd6ce44e365c0b68e73e76e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-06-02T23:40:34.952 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 podman[77654]: 2024-06-02 23:40:34.609605357 +0000 UTC m=+0.016964574 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:35.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:34.827+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:35.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:34.827+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:35.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:34.827+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:35.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:34 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:34.827+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:35.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:34 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:34.792+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:35.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:34 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:34.792+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:35.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:34 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:34.792+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:35.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:34 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:34.792+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:35.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:34 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:34.792+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:35.243 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 podman[77654]: 2024-06-02 23:40:34.953253961 +0000 UTC m=+0.360613165 container init 068d67e2da7a703b10f6d77d3b7741c312936966bd6ce44e365c0b68e73e76e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7-deactivate, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-02T23:40:35.243 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:34 smithi123 podman[77654]: 2024-06-02 23:40:34.956727553 +0000 UTC m=+0.364086763 container start 068d67e2da7a703b10f6d77d3b7741c312936966bd6ce44e365c0b68e73e76e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-06-02T23:40:35.243 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:35 smithi123 podman[77654]: 2024-06-02 23:40:35.044841913 +0000 UTC m=+0.452201130 container attach 068d67e2da7a703b10f6d77d3b7741c312936966bd6ce44e365c0b68e73e76e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD) 2024-06-02T23:40:35.507 INFO:journalctl@ceph.osd.7.smithi123.stdout:Jun 02 23:40:35 smithi123 podman[77654]: 2024-06-02 23:40:35.244186094 +0000 UTC m=+0.651545314 container died 068d67e2da7a703b10f6d77d3b7741c312936966bd6ce44e365c0b68e73e76e3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240528) 2024-06-02T23:40:35.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:35.265+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:35.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:35.265+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:35.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:35.265+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:35.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:35.265+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:35.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:35 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:35.265+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:35.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:35.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:35.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:35.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:35.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:35.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:35.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:35.388+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:35.818 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.7.service' 2024-06-02T23:40:35.898 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:35.898 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-06-02T23:40:35.898 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-06-02T23:40:35.898 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.8 2024-06-02T23:40:36.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:35.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:36.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:35.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:36.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:35.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:36.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:35 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:35.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:36.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:35 smithi174 systemd[1]: Stopping Ceph osd.8 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:36.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:35.813+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:36.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:35.813+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:36.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:35.813+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:36.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:35.813+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:36.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:35 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:35.813+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:36.643 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.274+0000 7fe13dd14700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:36.643 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.274+0000 7fe13dd14700 -1 osd.8 93 *** Got signal Terminated *** 2024-06-02T23:40:36.644 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.274+0000 7fe13dd14700 -1 osd.8 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:36.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:36.402+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:36.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:36.402+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:36.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:36.402+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:36.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:36.402+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:36.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:36 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:36.289+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:36.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:36 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:36.289+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:36.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:36 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:36.289+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:36.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:36 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:36.289+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:36.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:36 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:36.289+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:37.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:37.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:37.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:37.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:36 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:36.852+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:37.147 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:36.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:37.147 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:36.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:37.147 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:36.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:37.147 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:36.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:37.147 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:36 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:36.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:37.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:37.254+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:37.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:37.254+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:37.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:37.254+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:37.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:37.254+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:37.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:37.254+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:37.812 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:37.418+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:37.812 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:37.418+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:37.812 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:37.418+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:37.812 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:37.418+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:38.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:37.812+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:38.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:37.812+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:38.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:37.812+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:38.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:37 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:37.812+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:38.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:37 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:37.804+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:38.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:37 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:37.804+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:38.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:37 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:37.804+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:38.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:37 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:37.804+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:38.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:37 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:37.804+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:38.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:38 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:38.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:38.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:38 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:38.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:38.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:38 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:38.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:38.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:38 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:38.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:38.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:38 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:38.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:38.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:38 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:38.233+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:38.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:38.390+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:38.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:38.390+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:38.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:38.390+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:38.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:38.390+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:39.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:38.820+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:39.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:38.820+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:39.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:38.820+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:39.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:38 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:38.820+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:39.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:38.837+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:39.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:38.837+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:39.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:38.837+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:39.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:38.837+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:39.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:38.837+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:39.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:38 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:38.837+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:39.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:39.261+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:39.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:39.261+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:39.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:39.261+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:39.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:39.261+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:39.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:39.261+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:39.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:39 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:39.261+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:39.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:39.421+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:39.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:39.421+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:39.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:39.421+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:39.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:39.421+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:40.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:39.840+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:40.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:39.840+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:40.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:39.840+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:40.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:39 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:39.840+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:40.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:39.870+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:40.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:39.870+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:40.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:39.870+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:40.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:39.870+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:40.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:39.870+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:40.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:39 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:39.870+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:40.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:40 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:40.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:40.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:40 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:40.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:40.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:40 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:40.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:40.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:40 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:40.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:40.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:40 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:40.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:40.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:40 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:40.212+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:40.880 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:40.454+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:40.880 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:40.454+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:40.880 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:40.454+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:40.880 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:40.454+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:41.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:40.879+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:38.763331+0000 front 2024-06-02T23:39:38.763409+0000 (oldest deadline 2024-06-02T23:40:06.364320+0000) 2024-06-02T23:40:41.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:40.879+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.265396+0000 front 2024-06-02T23:39:49.265320+0000 (oldest deadline 2024-06-02T23:40:13.365451+0000) 2024-06-02T23:40:41.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:40.879+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.168388+0000 front 2024-06-02T23:40:00.168294+0000 (oldest deadline 2024-06-02T23:40:25.468290+0000) 2024-06-02T23:40:41.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:40.879+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.468901+0000 front 2024-06-02T23:40:05.468799+0000 (oldest deadline 2024-06-02T23:40:30.768819+0000) 2024-06-02T23:40:41.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:40 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8[54534]: 2024-06-02T23:40:40.879+0000 7fe13a308700 -1 osd.8 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:15.270800+0000 front 2024-06-02T23:40:15.270840+0000 (oldest deadline 2024-06-02T23:40:40.570880+0000) 2024-06-02T23:40:41.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:40.855+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:41.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:40.855+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:41.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:40.855+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:41.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:40.855+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:41.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:40.855+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:41.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:40 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:40.855+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:41.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:41.202+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:41.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:41.202+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:41.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:41.202+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:41.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:41.202+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:41.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:41.202+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:41.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:41 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:41.202+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:41.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:41.436+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:41.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:41.436+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:41.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:41.436+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:41.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:41.436+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:41.839 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:41.436+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:42.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:41 smithi174 podman[79119]: 2024-06-02 23:40:41.839134709 +0000 UTC m=+5.735483473 container died dea4eddb67e2e896b244664c548f774e8d30f20bb20fda19d8f09c0dbc6fcc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:42.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:41.816+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:42.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:41.816+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:42.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:41.816+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:42.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:41.816+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:42.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:41.816+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:42.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:41 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:41.816+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:42.442 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:42 smithi174 podman[79119]: 2024-06-02 23:40:42.257978415 +0000 UTC m=+6.154327173 container cleanup dea4eddb67e2e896b244664c548f774e8d30f20bb20fda19d8f09c0dbc6fcc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:40:42.442 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:42 smithi174 bash[79119]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8 2024-06-02T23:40:42.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:42 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:42.194+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:42.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:42 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:42.194+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:42.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:42 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:42.194+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:42.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:42 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:42.194+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:42.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:42 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:42.194+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:42.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:42 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:42.194+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:42.876 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:42 smithi174 podman[79137]: 2024-06-02 23:40:42.442550837 +0000 UTC m=+0.604514483 container remove dea4eddb67e2e896b244664c548f774e8d30f20bb20fda19d8f09c0dbc6fcc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2) 2024-06-02T23:40:42.877 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:42 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:42.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:42.877 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:42 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:42.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:42.877 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:42 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:42.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:42.877 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:42 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:42.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:42.877 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:42 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:42.448+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:43.143 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:42 smithi174 podman[79225]: 2024-06-02 23:40:42.780012694 +0000 UTC m=+0.017096590 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:43.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:42 smithi174 podman[79225]: 2024-06-02 23:40:42.884993089 +0000 UTC m=+0.122076965 container create aaab01ba700ccab62d3de53f08f7dbc603c88b83b39debadd5193ea8a43dd166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:40:43.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:43 smithi174 podman[79225]: 2024-06-02 23:40:43.095898792 +0000 UTC m=+0.332982683 container init aaab01ba700ccab62d3de53f08f7dbc603c88b83b39debadd5193ea8a43dd166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-02T23:40:43.144 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:43 smithi174 podman[79225]: 2024-06-02 23:40:43.099278963 +0000 UTC m=+0.336362851 container start aaab01ba700ccab62d3de53f08f7dbc603c88b83b39debadd5193ea8a43dd166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-02T23:40:43.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:42.844+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:43.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:42.844+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:43.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:42.844+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:43.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:42.844+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:43.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:42.844+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:43.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:42 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:42.844+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:43.484 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:43 smithi174 podman[79225]: 2024-06-02 23:40:43.195746498 +0000 UTC m=+0.432830394 container attach aaab01ba700ccab62d3de53f08f7dbc603c88b83b39debadd5193ea8a43dd166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:43.484 INFO:journalctl@ceph.osd.8.smithi174.stdout:Jun 02 23:40:43 smithi174 podman[79225]: 2024-06-02 23:40:43.392941617 +0000 UTC m=+0.630025502 container died aaab01ba700ccab62d3de53f08f7dbc603c88b83b39debadd5193ea8a43dd166 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-06-02T23:40:43.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:43.176+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:43.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:43.176+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:43.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:43.176+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:43.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:43.176+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:43.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:43.176+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:43.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:43 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:43.176+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:43.757 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:43 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:43.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:43.757 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:43 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:43.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:43.757 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:43 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:43.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:43.757 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:43 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:43.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:43.757 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:43 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:43.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:44.089 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.8.service' 2024-06-02T23:40:44.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:44.128 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-06-02T23:40:44.128 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-06-02T23:40:44.128 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.9 2024-06-02T23:40:44.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:43.824+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:44.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:43.824+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:44.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:43.824+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:44.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:43.824+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:44.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:43.824+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:44.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:43 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:43.824+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:44.431 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 systemd[1]: Stopping Ceph osd.9 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:44.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:44 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:44.162+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:44.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:44 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:44.162+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:44.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:44 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:44.162+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:44.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:44 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:44.162+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:44.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:44 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:44.162+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:44.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:44 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:44.162+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:44.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:44 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:44.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:44.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:44 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:44.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:44.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:44 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:44.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:44.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:44 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:44.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:44.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:44 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:44.483+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.430+0000 7fa04d6c6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.430+0000 7fa04d6c6700 -1 osd.9 93 *** Got signal Terminated *** 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.430+0000 7fa04d6c6700 -1 osd.9 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.810+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.810+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.810+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.810+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.810+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:44.811 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:44 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:44.810+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:45.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:45 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:45.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:45.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:45 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:45.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:45.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:45 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:45.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:45.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:45 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:45.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:45.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:45 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:45.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:45.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:45 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:45.207+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:45.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:45.449+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:45.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:45.449+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:45.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:45.449+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:45.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:45.449+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:45.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:45.449+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:46.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:45.834+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:46.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:45.834+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:46.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:45.834+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:46.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:45.834+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:46.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:45.834+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:46.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:45 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:45.834+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:46.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:46 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:46.224+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:46.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:46 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:46.224+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:46.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:46 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:46.224+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:46.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:46 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:46.224+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:46.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:46 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:46.224+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:46.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:46 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:46.224+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:46.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:46 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:46.459+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:46.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:46 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:46.459+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:46.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:46 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:46.459+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:46.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:46 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:46.459+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:46.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:46 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:46.459+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:47.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:46 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:46.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:47.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:46 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:46.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:47.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:46 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:46.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:47.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:46 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:46.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:47.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:46 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:46.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:47.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:46 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:46.829+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:47.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:47 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:47.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:47.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:47 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:47.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:47.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:47 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:47.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:47.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:47 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:47.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:47.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:47 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:47.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:47.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:47 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:47.191+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:47.794 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:47.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:47.795 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:47.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:47.795 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:47.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:47.795 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:47.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:47.795 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:47 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:47.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:48.149 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:47 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:47.859+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:48.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:47 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:47.859+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:48.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:47 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:47.859+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:48.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:47 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:47.859+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:48.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:47 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:47.859+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:48.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:47 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:47.859+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:48.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:48 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:48.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:48.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:48 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:48.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:48.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:48 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:48.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:48.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:48 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:48.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:48.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:48 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:48.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:48.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:48 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:48.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:48.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:48.506+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:48.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:48.506+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:48.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:48.506+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:48.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:48.506+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:48.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:48.506+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:48.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:48 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:48.506+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:31.825513+0000 front 2024-06-02T23:39:31.825540+0000 (oldest deadline 2024-06-02T23:39:57.725636+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:43.527762+0000 front 2024-06-02T23:39:43.527560+0000 (oldest deadline 2024-06-02T23:40:06.427431+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:49.928412+0000 front 2024-06-02T23:39:49.928338+0000 (oldest deadline 2024-06-02T23:40:16.928962+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:40:00.430129+0000 front 2024-06-02T23:40:00.430077+0000 (oldest deadline 2024-06-02T23:40:23.930270+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:07.431595+0000 front 2024-06-02T23:40:07.431703+0000 (oldest deadline 2024-06-02T23:40:30.331672+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:13.232821+0000 front 2024-06-02T23:40:13.232787+0000 (oldest deadline 2024-06-02T23:40:37.932788+0000) 2024-06-02T23:40:49.400 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:48 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9[65132]: 2024-06-02T23:40:48.908+0000 7fa049cba700 -1 osd.9 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.134327+0000 front 2024-06-02T23:40:23.134303+0000 (oldest deadline 2024-06-02T23:40:48.434443+0000) 2024-06-02T23:40:49.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:49 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:49.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:49.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:49 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:49.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:49.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:49 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:49.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:49.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:49 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:49.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:49.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:49 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:49.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:49.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:49 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:49.180+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:49.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:49.495+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:49.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:49.495+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:49.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:49.495+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:49.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:49.495+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:49.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:49.495+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:49.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:49.495+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:50.150 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:49 smithi052 podman[164327]: 2024-06-02 23:40:49.793636049 +0000 UTC m=+5.460274135 container died de75b9ac8b08b6be83fcc2552e05003b954ea5f1305c3cbd021b45fda8269b45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-06-02T23:40:50.423 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 podman[164327]: 2024-06-02 23:40:50.155713341 +0000 UTC m=+5.822351436 container cleanup de75b9ac8b08b6be83fcc2552e05003b954ea5f1305c3cbd021b45fda8269b45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc) 2024-06-02T23:40:50.423 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 bash[164327]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9 2024-06-02T23:40:50.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:50.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:50.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:50.507 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:50.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:50.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:50.508 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:50 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:50.225+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:50.732 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 podman[164344]: 2024-06-02 23:40:50.423619736 +0000 UTC m=+0.626505688 container remove de75b9ac8b08b6be83fcc2552e05003b954ea5f1305c3cbd021b45fda8269b45 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:40:50.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:50 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:50.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:50.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:50 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:50.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:50.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:50 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:50.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:50.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:50 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:50.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:50.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:50 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:50.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:50.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:50 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:50.479+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:51.009 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 podman[164434]: 2024-06-02 23:40:50.732755601 +0000 UTC m=+0.100095351 container create dca0aefbf57853604aa63cdab28ff5e4d6ce5d1c0634e577a11f61e6e50a7848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528) 2024-06-02T23:40:51.009 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 podman[164434]: 2024-06-02 23:40:50.650858836 +0000 UTC m=+0.018198587 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:51.009 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 podman[164434]: 2024-06-02 23:40:50.935232803 +0000 UTC m=+0.302572544 container init dca0aefbf57853604aa63cdab28ff5e4d6ce5d1c0634e577a11f61e6e50a7848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-02T23:40:51.009 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:50 smithi052 podman[164434]: 2024-06-02 23:40:50.938620997 +0000 UTC m=+0.305960750 container start dca0aefbf57853604aa63cdab28ff5e4d6ce5d1c0634e577a11f61e6e50a7848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, ceph=True, CEPH_POINT_RELEASE=) 2024-06-02T23:40:51.009 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:51 smithi052 podman[164434]: 2024-06-02 23:40:51.010092712 +0000 UTC m=+0.377432468 container attach dca0aefbf57853604aa63cdab28ff5e4d6ce5d1c0634e577a11f61e6e50a7848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:40:51.399 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:51 smithi052 podman[164434]: 2024-06-02 23:40:51.242210112 +0000 UTC m=+0.609549870 container died dca0aefbf57853604aa63cdab28ff5e4d6ce5d1c0634e577a11f61e6e50a7848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc) 2024-06-02T23:40:51.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:51.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:51.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:51.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:51.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:51.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:51.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:51.272+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:51.767 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.9.service' 2024-06-02T23:40:51.832 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:51 smithi052 podman[164461]: 2024-06-02 23:40:51.746468298 +0000 UTC m=+0.497866158 container remove dca0aefbf57853604aa63cdab28ff5e4d6ce5d1c0634e577a11f61e6e50a7848 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5) 2024-06-02T23:40:51.832 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:51 smithi052 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.9.service: Deactivated successfully. 2024-06-02T23:40:51.832 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:51 smithi052 systemd[1]: Stopped Ceph osd.9 for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:40:51.832 INFO:journalctl@ceph.osd.9.smithi052.stdout:Jun 02 23:40:51 smithi052 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.9.service: Consumed 1min 38.412s CPU time. 2024-06-02T23:40:51.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:51.849 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-06-02T23:40:51.850 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-06-02T23:40:51.850 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.10 2024-06-02T23:40:51.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:51 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:51.494+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:51.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:51 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:51.494+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:51.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:51 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:51.494+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:51.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:51 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:51.494+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:51.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:51 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:51.494+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:51.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:51 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:51.494+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:52.257 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:51 smithi123 systemd[1]: Stopping Ceph osd.10 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:40:52.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:52.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:52.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:52.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:52.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:52.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:52.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.276+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:52.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.333+0000 7f9bdf943700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:40:52.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.333+0000 7f9bdf943700 -1 osd.10 93 *** Got signal Terminated *** 2024-06-02T23:40:52.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:52 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:52.333+0000 7f9bdf943700 -1 osd.10 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:40:52.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:52 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:52.446+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:52.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:52 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:52.446+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:52.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:52 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:52.446+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:52.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:52 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:52.446+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:52.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:52 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:52.446+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:52.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:52 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:52.446+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:53.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:53.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:53.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:53.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:53.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:53.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:53.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:53 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:53.288+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:53.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:53.439+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:53.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:53.439+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:53.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:53.439+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:53.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:53.439+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:53.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:53.439+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:53.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:53.439+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:54.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:54.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:54.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:54.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:54.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:54.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:54.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:54 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:54.317+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:54.774 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:54 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:54.444+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:54.774 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:54 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:54.445+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:54.774 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:54 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:54.445+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:54.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:54 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:54.445+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:54.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:54 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:54.445+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:54.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:54 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:54.445+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:55.643 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:55 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:55.484+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:55.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:55 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:55.484+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:55.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:55 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:55.484+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:55.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:55 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:55.484+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:55.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:55 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:55.484+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:55.644 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:55 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:55.484+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:55.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:55.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:55.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:55.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:55.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:55.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:55.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:55 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:55.326+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:56.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:32.874853+0000 front 2024-06-02T23:39:32.874786+0000 (oldest deadline 2024-06-02T23:39:58.174661+0000) 2024-06-02T23:40:56.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:42.275710+0000 front 2024-06-02T23:39:42.275738+0000 (oldest deadline 2024-06-02T23:40:06.375845+0000) 2024-06-02T23:40:56.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:50.476987+0000 front 2024-06-02T23:39:50.477043+0000 (oldest deadline 2024-06-02T23:40:14.577064+0000) 2024-06-02T23:40:56.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6814 osd.3 since back 2024-06-02T23:39:56.778821+0000 front 2024-06-02T23:39:56.778902+0000 (oldest deadline 2024-06-02T23:40:23.779527+0000) 2024-06-02T23:40:56.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.381232+0000 front 2024-06-02T23:40:05.381257+0000 (oldest deadline 2024-06-02T23:40:30.081284+0000) 2024-06-02T23:40:56.757 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:11.782382+0000 front 2024-06-02T23:40:11.782291+0000 (oldest deadline 2024-06-02T23:40:37.682440+0000) 2024-06-02T23:40:56.758 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:56 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10[60756]: 2024-06-02T23:40:56.331+0000 7f9bdbf37700 -1 osd.10 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:23.985028+0000 front 2024-06-02T23:40:23.985090+0000 (oldest deadline 2024-06-02T23:40:49.884792+0000) 2024-06-02T23:40:56.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:56.461+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:56.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:56.461+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:56.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:56.461+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:56.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:56.461+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:56.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:56.461+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:56.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:56 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:56.461+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:57.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:57 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:57.453+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:57.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:57 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:57.453+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:57.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:57 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:57.453+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:57.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:57 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:57.453+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:57.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:57 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:57.453+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:57.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:57 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:57.453+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:58.007 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:57 smithi123 podman[77760]: 2024-06-02 23:40:57.756177728 +0000 UTC m=+5.709692627 container died 694d9661983e201ad8d09c28d5ba9eeb49d4ddca24b30979e5cb7724f1687ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True) 2024-06-02T23:40:58.266 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 podman[77760]: 2024-06-02 23:40:58.091273986 +0000 UTC m=+6.044788877 container cleanup 694d9661983e201ad8d09c28d5ba9eeb49d4ddca24b30979e5cb7724f1687ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:58.267 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 bash[77760]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10 2024-06-02T23:40:58.706 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 podman[77775]: 2024-06-02 23:40:58.267432878 +0000 UTC m=+0.513982337 container remove 694d9661983e201ad8d09c28d5ba9eeb49d4ddca24b30979e5cb7724f1687ee3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528) 2024-06-02T23:40:58.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:58 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:58.467+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:58.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:58 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:58.467+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:58.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:58 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:58.467+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:58.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:58 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:58.467+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:58.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:58 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:58.467+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:58.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:58 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:58.467+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:59.007 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 podman[77866]: 2024-06-02 23:40:58.609569035 +0000 UTC m=+0.017239692 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:40:59.007 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 podman[77866]: 2024-06-02 23:40:58.709798036 +0000 UTC m=+0.117468671 container create 21931f7ba136c7e0a4f91c9a24b23fcc393fb8838c66838298aa5780021797be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-06-02T23:40:59.007 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 podman[77866]: 2024-06-02 23:40:58.936985231 +0000 UTC m=+0.344655878 container init 21931f7ba136c7e0a4f91c9a24b23fcc393fb8838c66838298aa5780021797be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2) 2024-06-02T23:40:59.007 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:58 smithi123 podman[77866]: 2024-06-02 23:40:58.940153967 +0000 UTC m=+0.347824611 container start 21931f7ba136c7e0a4f91c9a24b23fcc393fb8838c66838298aa5780021797be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-06-02T23:40:59.503 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:59 smithi123 podman[77866]: 2024-06-02 23:40:59.029442236 +0000 UTC m=+0.437112918 container attach 21931f7ba136c7e0a4f91c9a24b23fcc393fb8838c66838298aa5780021797be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:40:59.503 INFO:journalctl@ceph.osd.10.smithi123.stdout:Jun 02 23:40:59 smithi123 podman[77866]: 2024-06-02 23:40:59.229416431 +0000 UTC m=+0.637087075 container died 21931f7ba136c7e0a4f91c9a24b23fcc393fb8838c66838298aa5780021797be (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:40:59.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.10.service' 2024-06-02T23:40:59.855 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:40:59.855 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-06-02T23:40:59.855 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-06-02T23:40:59.855 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.11 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:40:59.884 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:40:59.514+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-02T23:40:28.278587+0000 front 2024-06-02T23:40:28.278571+0000 (oldest deadline 2024-06-02T23:40:59.478891+0000) 2024-06-02T23:41:00.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:40:59 smithi174 systemd[1]: Stopping Ceph osd.11 for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:00.545 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.151+0000 7ff2cd923700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:00.545 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.151+0000 7ff2cd923700 -1 osd.11 93 *** Got signal Terminated *** 2024-06-02T23:41:00.545 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.151+0000 7ff2cd923700 -1 osd.11 93 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-06-02T23:41:00.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:41:00.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:41:00.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:41:00.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:41:00.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:41:00.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:41:00.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:00 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:00.544+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-02T23:40:28.278587+0000 front 2024-06-02T23:40:28.278571+0000 (oldest deadline 2024-06-02T23:40:59.478891+0000) 2024-06-02T23:41:01.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:41:01.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:41:01.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:41:01.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:41:01.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:41:01.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:41:01.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:01 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:01.551+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-02T23:40:28.278587+0000 front 2024-06-02T23:40:28.278571+0000 (oldest deadline 2024-06-02T23:40:59.478891+0000) 2024-06-02T23:41:02.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:41:02.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:41:02.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:41:02.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:41:02.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:41:02.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:41:02.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:02 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:02.555+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-02T23:40:28.278587+0000 front 2024-06-02T23:40:28.278571+0000 (oldest deadline 2024-06-02T23:40:59.478891+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:41:03.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:03 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:03.554+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-02T23:40:28.278587+0000 front 2024-06-02T23:40:28.278571+0000 (oldest deadline 2024-06-02T23:40:59.478891+0000) 2024-06-02T23:41:04.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6806 osd.0 since back 2024-06-02T23:39:33.768484+0000 front 2024-06-02T23:39:33.768370+0000 (oldest deadline 2024-06-02T23:39:59.668464+0000) 2024-06-02T23:41:04.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6804 osd.1 since back 2024-06-02T23:39:44.270147+0000 front 2024-06-02T23:39:44.270020+0000 (oldest deadline 2024-06-02T23:40:07.770014+0000) 2024-06-02T23:41:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-06-02T23:39:52.471158+0000 front 2024-06-02T23:39:52.470985+0000 (oldest deadline 2024-06-02T23:40:15.371092+0000) 2024-06-02T23:41:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6812 osd.4 since back 2024-06-02T23:40:05.773969+0000 front 2024-06-02T23:40:05.773784+0000 (oldest deadline 2024-06-02T23:40:33.374467+0000) 2024-06-02T23:41:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-06-02T23:40:16.775940+0000 front 2024-06-02T23:40:16.775880+0000 (oldest deadline 2024-06-02T23:40:40.875960+0000) 2024-06-02T23:41:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.52:6822 osd.6 since back 2024-06-02T23:40:24.377209+0000 front 2024-06-02T23:40:24.377320+0000 (oldest deadline 2024-06-02T23:40:47.777535+0000) 2024-06-02T23:41:04.894 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:04 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11[61080]: 2024-06-02T23:41:04.542+0000 7ff2c5f0f700 -1 osd.11 93 heartbeat_check: no reply from 172.21.15.123:6820 osd.7 since back 2024-06-02T23:40:28.278587+0000 front 2024-06-02T23:40:28.278571+0000 (oldest deadline 2024-06-02T23:40:59.478891+0000) 2024-06-02T23:41:05.893 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:05 smithi174 podman[79345]: 2024-06-02 23:41:05.626570317 +0000 UTC m=+5.571793250 container died d24f9ac7caeaa19848b673aa528c1c55a553e6041d5c2ead0addd8d4a55a8f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:41:06.393 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:05 smithi174 podman[79345]: 2024-06-02 23:41:05.91762879 +0000 UTC m=+5.862851730 container cleanup d24f9ac7caeaa19848b673aa528c1c55a553e6041d5c2ead0addd8d4a55a8f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:41:06.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:05 smithi174 bash[79345]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11 2024-06-02T23:41:06.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79361]: 2024-06-02 23:41:06.127107687 +0000 UTC m=+0.494209013 container remove d24f9ac7caeaa19848b673aa528c1c55a553e6041d5c2ead0addd8d4a55a8f40 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-06-02T23:41:06.755 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79455]: 2024-06-02 23:41:06.350110148 +0000 UTC m=+0.016993909 image pull b045ac545f826a15724cc02d20333d2cc28f7e0eeb5a24d748d09a02280116e1 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9 2024-06-02T23:41:06.755 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79455]: 2024-06-02 23:41:06.461200586 +0000 UTC m=+0.128084371 container create 429efc755b8547509cf0334605bcf7e67d4f87681e291f91c4c4bc8e5d3d22aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5) 2024-06-02T23:41:06.755 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79455]: 2024-06-02 23:41:06.680380597 +0000 UTC m=+0.347264340 container init 429efc755b8547509cf0334605bcf7e67d4f87681e291f91c4c4bc8e5d3d22aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-02T23:41:06.755 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79455]: 2024-06-02 23:41:06.68366112 +0000 UTC m=+0.350544863 container start 429efc755b8547509cf0334605bcf7e67d4f87681e291f91c4c4bc8e5d3d22aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-06-02T23:41:06.755 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79455]: 2024-06-02 23:41:06.755165759 +0000 UTC m=+0.422049507 container attach 429efc755b8547509cf0334605bcf7e67d4f87681e291f91c4c4bc8e5d3d22aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True) 2024-06-02T23:41:07.143 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:06 smithi174 podman[79455]: 2024-06-02 23:41:06.979906217 +0000 UTC m=+0.646789960 container died 429efc755b8547509cf0334605bcf7e67d4f87681e291f91c4c4bc8e5d3d22aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-06-02T23:41:07.559 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.11.service' 2024-06-02T23:41:07.623 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:07 smithi174 podman[79481]: 2024-06-02 23:41:07.54175772 +0000 UTC m=+0.556423057 container remove 429efc755b8547509cf0334605bcf7e67d4f87681e291f91c4c4bc8e5d3d22aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-osd-11-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:41:07.623 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:07 smithi174 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.11.service: Deactivated successfully. 2024-06-02T23:41:07.623 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:07 smithi174 systemd[1]: Stopped Ceph osd.11 for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:41:07.623 INFO:journalctl@ceph.osd.11.smithi174.stdout:Jun 02 23:41:07 smithi174 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@osd.11.service: Consumed 1min 41.542s CPU time. 2024-06-02T23:41:07.637 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:07.638 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-06-02T23:41:07.638 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-06-02T23:41:07.638 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.a 2024-06-02T23:41:07.956 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:41:07 smithi052 systemd[1]: Stopping Ceph mds.a for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:08.236 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:41:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a[72552]: 2024-06-02T23:41:07.955+0000 7f7c10906700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:08.236 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:41:07 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a[72552]: 2024-06-02T23:41:07.955+0000 7f7c10906700 -1 mds.a *** got signal Terminated *** 2024-06-02T23:41:12.249 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:41:11 smithi052 podman[164539]: 2024-06-02 23:41:11.967266333 +0000 UTC m=+4.113900732 container died ff3a6c7f7e1e0adb1e3230636cd627de968ae4920edd408863f78e5f0c77db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240528, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=) 2024-06-02T23:41:12.649 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:41:12 smithi052 podman[164539]: 2024-06-02 23:41:12.423249772 +0000 UTC m=+4.569884194 container cleanup ff3a6c7f7e1e0adb1e3230636cd627de968ae4920edd408863f78e5f0c77db2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:41:12.650 INFO:journalctl@ceph.mds.a.smithi052.stdout:Jun 02 23:41:12 smithi052 bash[164539]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-a 2024-06-02T23:41:12.869 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.a.service' 2024-06-02T23:41:12.907 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:12.907 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-06-02T23:41:12.907 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-06-02T23:41:12.907 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.d 2024-06-02T23:41:13.219 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:41:12 smithi052 systemd[1]: Stopping Ceph mds.d for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:13.220 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:41:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-d[71296]: 2024-06-02T23:41:13.218+0000 7f3092221700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:13.220 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:41:13 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-d[71296]: 2024-06-02T23:41:13.218+0000 7f3092221700 -1 mds.d *** got signal Terminated *** 2024-06-02T23:41:14.929 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:41:14 smithi052 podman[164674]: 2024-06-02 23:41:14.666160802 +0000 UTC m=+1.533664135 container died bbe4fc56ae0610473ab87ac67b90a0ec5186bd426fec23dbe32941832f425016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-d, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:41:15.285 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:41:15 smithi052 podman[164674]: 2024-06-02 23:41:15.027024748 +0000 UTC m=+1.894528079 container cleanup bbe4fc56ae0610473ab87ac67b90a0ec5186bd426fec23dbe32941832f425016 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-d, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-3d89acc, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:41:15.286 INFO:journalctl@ceph.mds.d.smithi052.stdout:Jun 02 23:41:15 smithi052 bash[164674]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-d 2024-06-02T23:41:15.425 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.d.service' 2024-06-02T23:41:15.459 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:15.459 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-06-02T23:41:15.459 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-06-02T23:41:15.459 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.g 2024-06-02T23:41:15.900 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:41:15 smithi052 systemd[1]: Stopping Ceph mds.g for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:15.900 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:41:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g[72100]: 2024-06-02T23:41:15.796+0000 7f26dca88700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:15.900 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:41:15 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g[72100]: 2024-06-02T23:41:15.796+0000 7f26dca88700 -1 mds.g *** got signal Terminated *** 2024-06-02T23:41:20.092 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:41:19 smithi052 podman[164797]: 2024-06-02 23:41:19.811075483 +0000 UTC m=+4.113077264 container died ce1e151382b0566ee8c6903aff2207e68ace95bb591e926d53d61d6f77d99e3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-3d89acc, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528) 2024-06-02T23:41:20.399 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:41:20 smithi052 podman[164797]: 2024-06-02 23:41:20.180821363 +0000 UTC m=+4.482823167 container cleanup ce1e151382b0566ee8c6903aff2207e68ace95bb591e926d53d61d6f77d99e3c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-06-02T23:41:20.400 INFO:journalctl@ceph.mds.g.smithi052.stdout:Jun 02 23:41:20 smithi052 bash[164797]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-g 2024-06-02T23:41:20.625 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.g.service' 2024-06-02T23:41:20.658 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:20.661 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-06-02T23:41:20.661 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-06-02T23:41:20.661 DEBUG:teuthology.orchestra.run.smithi052:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.j 2024-06-02T23:41:20.984 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:41:20 smithi052 systemd[1]: Stopping Ceph mds.j for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:21.399 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:41:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-j[70841]: 2024-06-02T23:41:20.983+0000 7f1529ecb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:21.400 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:41:20 smithi052 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-j[70841]: 2024-06-02T23:41:20.983+0000 7f1529ecb700 -1 mds.j *** got signal Terminated *** 2024-06-02T23:41:22.681 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:41:22 smithi052 podman[164918]: 2024-06-02 23:41:22.401161108 +0000 UTC m=+1.511993010 container died 5bd0191c1dbb7a9d159bd8197170a7a266d7997412b78e5a456c4e1502a87e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-j, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-06-02T23:41:23.001 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:41:22 smithi052 podman[164918]: 2024-06-02 23:41:22.825858024 +0000 UTC m=+1.936689905 container cleanup 5bd0191c1dbb7a9d159bd8197170a7a266d7997412b78e5a456c4e1502a87e3d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-j, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True) 2024-06-02T23:41:23.002 INFO:journalctl@ceph.mds.j.smithi052.stdout:Jun 02 23:41:22 smithi052 bash[164918]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-j 2024-06-02T23:41:23.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.j.service' 2024-06-02T23:41:23.250 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:23.250 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-06-02T23:41:23.250 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-06-02T23:41:23.250 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.b 2024-06-02T23:41:23.559 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:23 smithi123 systemd[1]: Stopping Ceph mds.b for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:24.007 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:41:23.558+0000 7fd6f7b06700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:24.007 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:23 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b[65922]: 2024-06-02T23:41:23.558+0000 7fd6f7b06700 -1 mds.b *** got signal Terminated *** 2024-06-02T23:41:27.826 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:27 smithi123 podman[77982]: 2024-06-02 23:41:27.565537893 +0000 UTC m=+4.109088847 container died e52435dd9cf18044936a6d15ad75b6a4d5170d0d5777757c249c71a56aa4e910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, GIT_CLEAN=True, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-06-02T23:41:28.176 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:27 smithi123 podman[77982]: 2024-06-02 23:41:27.991894218 +0000 UTC m=+4.535445171 container cleanup e52435dd9cf18044936a6d15ad75b6a4d5170d0d5777757c249c71a56aa4e910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-3d89acc, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-02T23:41:28.176 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:27 smithi123 bash[77982]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b 2024-06-02T23:41:28.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.b.service' 2024-06-02T23:41:28.443 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:28 smithi123 podman[77995]: 2024-06-02 23:41:28.176868982 +0000 UTC m=+0.607173990 container remove e52435dd9cf18044936a6d15ad75b6a4d5170d0d5777757c249c71a56aa4e910 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-b, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc) 2024-06-02T23:41:28.443 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:28 smithi123 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.b.service: Deactivated successfully. 2024-06-02T23:41:28.443 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:28 smithi123 systemd[1]: Stopped Ceph mds.b for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:41:28.443 INFO:journalctl@ceph.mds.b.smithi123.stdout:Jun 02 23:41:28 smithi123 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.b.service: Consumed 13min 14.754s CPU time. 2024-06-02T23:41:28.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:28.458 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-06-02T23:41:28.458 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-06-02T23:41:28.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.e 2024-06-02T23:41:28.757 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:28 smithi123 systemd[1]: Stopping Ceph mds.e for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:29.257 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e[65318]: 2024-06-02T23:41:28.762+0000 7fbe0fe80700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:29.257 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:28 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e[65318]: 2024-06-02T23:41:28.762+0000 7fbe0fe80700 -1 mds.e *** got signal Terminated *** 2024-06-02T23:41:32.238 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:31 smithi123 podman[78103]: 2024-06-02 23:41:31.985254412 +0000 UTC m=+3.314340809 container died 558a5a69bcdca2cfc5cd54cc7a65ac6bc21cf3bdab25d8c8319e74cc69686ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:41:32.507 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:32 smithi123 podman[78103]: 2024-06-02 23:41:32.328982202 +0000 UTC m=+3.658068592 container cleanup 558a5a69bcdca2cfc5cd54cc7a65ac6bc21cf3bdab25d8c8319e74cc69686ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-3d89acc) 2024-06-02T23:41:32.507 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:32 smithi123 bash[78103]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e 2024-06-02T23:41:32.765 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.e.service' 2024-06-02T23:41:32.787 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:32 smithi123 podman[78129]: 2024-06-02 23:41:32.530004386 +0000 UTC m=+0.543218040 container remove 558a5a69bcdca2cfc5cd54cc7a65ac6bc21cf3bdab25d8c8319e74cc69686ed0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-06-02T23:41:32.787 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:32 smithi123 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.e.service: Deactivated successfully. 2024-06-02T23:41:32.787 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:32 smithi123 systemd[1]: Stopped Ceph mds.e for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:41:32.787 INFO:journalctl@ceph.mds.e.smithi123.stdout:Jun 02 23:41:32 smithi123 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.e.service: Consumed 5min 20.764s CPU time. 2024-06-02T23:41:32.801 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:32.801 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-06-02T23:41:32.801 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-06-02T23:41:32.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.h 2024-06-02T23:41:33.114 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:32 smithi123 systemd[1]: Stopping Ceph mds.h for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:33.507 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h[65621]: 2024-06-02T23:41:33.113+0000 7f4f8cff7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:33.507 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:33 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h[65621]: 2024-06-02T23:41:33.113+0000 7f4f8cff7700 -1 mds.h *** got signal Terminated *** 2024-06-02T23:41:36.231 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:35 smithi123 podman[78237]: 2024-06-02 23:41:35.981781636 +0000 UTC m=+2.961419126 container died da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h, org.label-schema.build-date=20240528, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-06-02T23:41:36.489 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:36 smithi123 podman[78237]: 2024-06-02 23:41:36.288662067 +0000 UTC m=+3.268299560 container cleanup da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240528, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-06-02T23:41:36.489 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:36 smithi123 bash[78237]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h 2024-06-02T23:41:36.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.h.service' 2024-06-02T23:41:36.739 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:36 smithi123 podman[78250]: 2024-06-02 23:41:36.489743784 +0000 UTC m=+0.503543933 container remove da6cf24d91a3e230d73c3ca35696df40add6119cebaf98b5035478982739919b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, CEPH_POINT_RELEASE=, RELEASE=squid-3d89acc, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:41:36.739 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:36 smithi123 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.h.service: Deactivated successfully. 2024-06-02T23:41:36.739 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:36 smithi123 systemd[1]: Stopped Ceph mds.h for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:41:36.739 INFO:journalctl@ceph.mds.h.smithi123.stdout:Jun 02 23:41:36 smithi123 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.h.service: Consumed 6min 40.413s CPU time. 2024-06-02T23:41:36.751 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:36.751 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-06-02T23:41:36.751 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-06-02T23:41:36.751 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.k 2024-06-02T23:41:37.007 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:41:36 smithi123 systemd[1]: Stopping Ceph mds.k for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:37.507 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:41:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k[65000]: 2024-06-02T23:41:37.091+0000 7fc8977e6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:37.507 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:41:37 smithi123 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k[65000]: 2024-06-02T23:41:37.091+0000 7fc8977e6700 -1 mds.k *** got signal Terminated *** 2024-06-02T23:41:40.257 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:41:39 smithi123 podman[78358]: 2024-06-02 23:41:39.980334428 +0000 UTC m=+3.008425469 container died 43be9a76630d842e3dff11148705e5a878cd27e456d2baf71082435fa89c3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:41:40.757 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:41:40 smithi123 podman[78358]: 2024-06-02 23:41:40.567049135 +0000 UTC m=+3.595140168 container cleanup 43be9a76630d842e3dff11148705e5a878cd27e456d2baf71082435fa89c3741 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-3d89acc, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-06-02T23:41:40.757 INFO:journalctl@ceph.mds.k.smithi123.stdout:Jun 02 23:41:40 smithi123 bash[78358]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-k 2024-06-02T23:41:40.953 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.k.service' 2024-06-02T23:41:40.986 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:40.986 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-06-02T23:41:40.986 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-06-02T23:41:40.986 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.c 2024-06-02T23:41:41.374 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:41:41 smithi174 systemd[1]: Stopping Ceph mds.c for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:41.643 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:41:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c[66083]: 2024-06-02T23:41:41.374+0000 7f6a4f0de700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:41.643 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:41:41 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c[66083]: 2024-06-02T23:41:41.374+0000 7f6a4f0de700 -1 mds.c *** got signal Terminated *** 2024-06-02T23:41:44.394 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:41:43 smithi174 podman[79562]: 2024-06-02 23:41:43.95623736 +0000 UTC m=+2.778320413 container died 2ed17d9c38427dc9c777c83636019e611c3a8936cbecdfb48511bb3f4fd2718b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8) 2024-06-02T23:41:44.891 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:41:44 smithi174 podman[79562]: 2024-06-02 23:41:44.615287976 +0000 UTC m=+3.437371035 container cleanup 2ed17d9c38427dc9c777c83636019e611c3a8936cbecdfb48511bb3f4fd2718b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c, RELEASE=squid-3d89acc, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240528, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-06-02T23:41:44.891 INFO:journalctl@ceph.mds.c.smithi174.stdout:Jun 02 23:41:44 smithi174 bash[79562]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-c 2024-06-02T23:41:45.052 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.c.service' 2024-06-02T23:41:45.090 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:45.090 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-06-02T23:41:45.091 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-06-02T23:41:45.091 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.f 2024-06-02T23:41:45.419 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:45 smithi174 systemd[1]: Stopping Ceph mds.f for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:45.893 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f[65778]: 2024-06-02T23:41:45.418+0000 7f8661a8b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:45.893 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:45 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f[65778]: 2024-06-02T23:41:45.418+0000 7f8661a8b700 -1 mds.f *** got signal Terminated *** 2024-06-02T23:41:48.243 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:47 smithi174 podman[79695]: 2024-06-02 23:41:47.968700139 +0000 UTC m=+2.651119477 container died 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-06-02T23:41:48.519 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:48 smithi174 podman[79695]: 2024-06-02 23:41:48.327311088 +0000 UTC m=+3.009730426 container cleanup 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-3d89acc, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux ) 2024-06-02T23:41:48.519 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:48 smithi174 bash[79695]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f 2024-06-02T23:41:48.770 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:48 smithi174 podman[79707]: 2024-06-02 23:41:48.519988441 +0000 UTC m=+0.551239721 container remove 27cc2db1695f64f7a6f2ffdaea6815cb0ae9e70dd861326b87ab2a2ac0710dc4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-f, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-3d89acc, org.label-schema.build-date=20240528, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-06-02T23:41:48.770 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:48 smithi174 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.f.service: Deactivated successfully. 2024-06-02T23:41:48.771 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:48 smithi174 systemd[1]: Stopped Ceph mds.f for 9d9fe36c-2133-11ef-bc9b-c7b262605968. 2024-06-02T23:41:48.771 INFO:journalctl@ceph.mds.f.smithi174.stdout:Jun 02 23:41:48 smithi174 systemd[1]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.f.service: Consumed 11.692s CPU time. 2024-06-02T23:41:48.772 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.f.service' 2024-06-02T23:41:48.805 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:48.805 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-06-02T23:41:48.805 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-06-02T23:41:48.805 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.i 2024-06-02T23:41:49.122 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:41:48 smithi174 systemd[1]: Stopping Ceph mds.i for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:49.393 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:41:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i[66386]: 2024-06-02T23:41:49.122+0000 7f03912ca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:49.393 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:41:49 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i[66386]: 2024-06-02T23:41:49.122+0000 7f03912ca700 -1 mds.i *** got signal Terminated *** 2024-06-02T23:41:52.264 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:41:51 smithi174 podman[79817]: 2024-06-02 23:41:51.968603049 +0000 UTC m=+2.947002772 container died a7cd1c89f433968408ac27630ac56dc5e44cfbcacac7becf05e00e9e18198417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240528) 2024-06-02T23:41:52.597 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:41:52 smithi174 podman[79817]: 2024-06-02 23:41:52.338722161 +0000 UTC m=+3.317121875 container cleanup a7cd1c89f433968408ac27630ac56dc5e44cfbcacac7becf05e00e9e18198417 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i, RELEASE=squid-3d89acc, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, org.label-schema.build-date=20240528, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-06-02T23:41:52.597 INFO:journalctl@ceph.mds.i.smithi174.stdout:Jun 02 23:41:52 smithi174 bash[79817]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-i 2024-06-02T23:41:52.768 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.i.service' 2024-06-02T23:41:52.800 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:52.800 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-06-02T23:41:52.800 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-06-02T23:41:52.800 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.l 2024-06-02T23:41:53.118 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:52 smithi174 systemd[1]: Stopping Ceph mds.l for 9d9fe36c-2133-11ef-bc9b-c7b262605968... 2024-06-02T23:41:53.393 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l[65476]: 2024-06-02T23:41:53.116+0000 7f0e3ebef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-06-02T23:41:53.393 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:53 smithi174 ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l[65476]: 2024-06-02T23:41:53.116+0000 7f0e3ebef700 -1 mds.l *** got signal Terminated *** 2024-06-02T23:41:56.258 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:55 smithi174 podman[79939]: 2024-06-02 23:41:55.963349492 +0000 UTC m=+2.935394804 container died d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, RELEASE=squid-3d89acc, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240528, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-06-02T23:41:56.563 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:56 smithi174 podman[79939]: 2024-06-02 23:41:56.259290747 +0000 UTC m=+3.231335953 container cleanup d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l, RELEASE=squid-3d89acc, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240528, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-06-02T23:41:56.563 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:56 smithi174 bash[79939]: ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l 2024-06-02T23:41:56.563 INFO:journalctl@ceph.mds.l.smithi174.stdout:Jun 02 23:41:56 smithi174 podman[79953]: 2024-06-02 23:41:56.460038262 +0000 UTC m=+0.494909434 container remove d579169b91598de6dd3a72be047fa13ba769aaf1ac2a96c0119353553aba2636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:fe72ccfadf2b7c7c133c6379dcb4e96c4db2f7f512b19bd8eb3efb31f1d8c5d9, name=ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968-mds-l, GIT_COMMIT=fe3baff630f29ddc3eb9757f6b798ccb55ab7bd8, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=squid-3d89acc, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240528) 2024-06-02T23:41:56.577 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9d9fe36c-2133-11ef-bc9b-c7b262605968@mds.l.service' 2024-06-02T23:41:56.609 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-06-02T23:41:56.610 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-06-02T23:41:56.610 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-02T23:41:56.816 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:41:59.410 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-02T23:41:59.602 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:42:02.156 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --force --keep-logs 2024-06-02T23:42:02.346 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:42:04.766 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:42:04.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:42:04.833 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-06-02T23:42:04.861 INFO:tasks.cephadm:Archiving crash dumps... 2024-06-02T23:42:04.863 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi052/crash 2024-06-02T23:42:04.863 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash -- . 2024-06-02T23:42:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:tar: /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-02T23:42:04.899 INFO:teuthology.orchestra.run.smithi052.stderr:tar: Error is not recoverable: exiting now 2024-06-02T23:42:04.902 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi123/crash 2024-06-02T23:42:04.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash -- . 2024-06-02T23:42:04.957 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-02T23:42:04.957 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-06-02T23:42:04.959 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi174/crash 2024-06-02T23:42:04.960 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash -- . 2024-06-02T23:42:05.000 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/crash: Cannot open: No such file or directory 2024-06-02T23:42:05.000 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-06-02T23:42:05.001 INFO:tasks.cephadm:Checking cluster log for badness... 2024-06-02T23:42:05.002 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-02T23:42:05.100 INFO:teuthology.orchestra.run.smithi052.stdout:2024-06-02T23:15:24.161143+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:42:05.100 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-06-02T23:42:05.101 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[SEC\]' /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-02T23:42:05.139 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[ERR\]' /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-02T23:42:05.217 DEBUG:teuthology.orchestra.run.smithi052:> sudo egrep '\[WRN\]' /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-06-02T23:42:05.292 INFO:teuthology.orchestra.run.smithi052.stdout:2024-06-02T23:15:24.161143+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details 2024-06-02T23:42:05.293 INFO:tasks.cephadm:Compressing logs... 2024-06-02T23:42:05.294 DEBUG:teuthology.orchestra.run.smithi052:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-02T23:42:05.336 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-02T23:42:05.338 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-06-02T23:42:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-02T23:42:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-06-02T23:42:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-02T23:42:05.374 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log 2024-06-02T23:42:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-02T23:42:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.g.log 2024-06-02T23:42:05.375 INFO:teuthology.orchestra.run.smithi052.stderr:gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.j.log 2024-06-02T23:42:05.376 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log: /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.3.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.0.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.x.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.a.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.6.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.a.log 2024-06-02T23:42:05.378 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-02T23:42:05.379 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.d.log 2024-06-02T23:42:05.379 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.9.log 2024-06-02T23:42:05.380 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.d.log: 87.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-02T23:42:05.380 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-02T23:42:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-02T23:42:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log 2024-06-02T23:42:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-02T23:42:05.381 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.7.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.b.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.4.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.1.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.y.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.h.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.10.log 2024-06-02T23:42:05.384 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-02T23:42:05.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.e.log 2024-06-02T23:42:05.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-02T23:42:05.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.b.log 2024-06-02T23:42:05.385 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.k.log 2024-06-02T23:42:05.385 INFO:teuthology.orchestra.run.smithi052.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.j.log.gz 2024-06-02T23:42:05.387 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-06-02T23:42:05.388 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-06-02T23:42:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log 2024-06-02T23:42:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.2.log 2024-06-02T23:42:05.389 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.f.log 2024-06-02T23:42:05.391 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.5.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.c.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.c.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.z.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:gzip/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.c.log: -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.i.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.11.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.8.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log 2024-06-02T23:42:05.392 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.l.log 2024-06-02T23:42:05.451 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.b.log: /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.k.log: 93.6% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.y.log.gz 2024-06-02T23:42:05.481 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log: /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.l.log: 84.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-02T23:42:05.497 INFO:teuthology.orchestra.run.smithi174.stderr: 93.7% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.z.log.gz 2024-06-02T23:42:05.607 INFO:teuthology.orchestra.run.smithi052.stderr: 91.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-02T23:42:05.618 INFO:teuthology.orchestra.run.smithi174.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-02T23:42:05.639 INFO:teuthology.orchestra.run.smithi123.stderr: 91.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-02T23:42:05.644 INFO:teuthology.orchestra.run.smithi052.stderr: 95.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-02T23:42:05.735 INFO:teuthology.orchestra.run.smithi052.stderr: 94.7% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.d.log.gz 2024-06-02T23:42:05.777 INFO:teuthology.orchestra.run.smithi052.stderr: 83.9% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-02T23:42:05.793 INFO:teuthology.orchestra.run.smithi174.stderr: 90.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-02T23:42:05.794 INFO:teuthology.orchestra.run.smithi174.stderr: 87.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-02T23:42:05.839 INFO:teuthology.orchestra.run.smithi052.stderr: 90.4% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-02T23:42:05.884 INFO:teuthology.orchestra.run.smithi174.stderr: 91.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-volume.log.gz 2024-06-02T23:42:06.003 INFO:teuthology.orchestra.run.smithi123.stderr: 84.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.cephadm.log.gz 2024-06-02T23:42:06.123 INFO:teuthology.orchestra.run.smithi123.stderr: 87.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.log.gz 2024-06-02T23:42:06.281 INFO:teuthology.orchestra.run.smithi123.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-06-02T23:42:06.294 INFO:teuthology.orchestra.run.smithi123.stderr: 90.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph.audit.log.gz 2024-06-02T23:42:06.613 INFO:teuthology.orchestra.run.smithi052.stderr: 90.3% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mgr.x.log.gz 2024-06-02T23:42:06.995 INFO:teuthology.orchestra.run.smithi174.stderr: 91.6% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.c.log.gz 2024-06-02T23:42:20.542 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.b.log.gz 2024-06-02T23:42:22.181 INFO:teuthology.orchestra.run.smithi052.stderr: 90.7% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mon.a.log.gz 2024-06-02T23:42:31.532 INFO:teuthology.orchestra.run.smithi174.stderr: 95.3% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.f.log.gz 2024-06-02T23:42:41.512 INFO:teuthology.orchestra.run.smithi174.stderr: 95.3% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.l.log.gz 2024-06-02T23:42:55.780 INFO:teuthology.orchestra.run.smithi052.stderr: 95.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.g.log.gz 2024-06-02T23:42:59.138 INFO:teuthology.orchestra.run.smithi174.stderr: 90.7% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.11.log.gz 2024-06-02T23:43:04.531 INFO:teuthology.orchestra.run.smithi174.stderr: 90.8% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.5.log.gz 2024-06-02T23:43:06.155 INFO:teuthology.orchestra.run.smithi123.stderr: 95.0% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.k.log.gz 2024-06-02T23:43:07.719 INFO:teuthology.orchestra.run.smithi052.stderr: 95.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.a.log.gz 2024-06-02T23:43:08.473 INFO:teuthology.orchestra.run.smithi052.stderr: 90.8% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.9.log.gz 2024-06-02T23:43:10.415 INFO:teuthology.orchestra.run.smithi052.stderr: 91.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.0.log.gz 2024-06-02T23:43:22.404 INFO:teuthology.orchestra.run.smithi174.stderr: 91.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.2.log.gz 2024-06-02T23:43:24.246 INFO:teuthology.orchestra.run.smithi174.stderr: 90.8% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.8.log.gz 2024-06-02T23:44:17.189 INFO:teuthology.orchestra.run.smithi052.stderr: 90.8% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.6.log.gz 2024-06-02T23:44:24.129 INFO:teuthology.orchestra.run.smithi174.stderr: 94.4% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.i.log.gz 2024-06-02T23:44:28.931 INFO:teuthology.orchestra.run.smithi123.stderr: 91.4% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.1.log.gz 2024-06-02T23:44:39.793 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.10.log.gz 2024-06-02T23:44:40.381 INFO:teuthology.orchestra.run.smithi052.stderr: 91.1% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.3.log.gz 2024-06-02T23:44:40.972 INFO:teuthology.orchestra.run.smithi174.stderr: 94.2% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.c.log.gz 2024-06-02T23:44:40.974 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-06-02T23:44:40.974 INFO:teuthology.orchestra.run.smithi174.stderr:real 2m35.622s 2024-06-02T23:44:40.974 INFO:teuthology.orchestra.run.smithi174.stderr:user 4m21.570s 2024-06-02T23:44:40.974 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m9.717s 2024-06-02T23:45:49.089 INFO:teuthology.orchestra.run.smithi123.stderr: 91.2% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.4.log.gz 2024-06-02T23:48:08.279 INFO:teuthology.orchestra.run.smithi123.stderr: 91.0% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-osd.7.log.gz 2024-06-02T23:50:03.222 INFO:teuthology.orchestra.run.smithi052.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-06-02T23:50:03.224 INFO:teuthology.orchestra.run.smithi052.stderr: 2024-06-02T23:50:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:real 7m57.875s 2024-06-02T23:50:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:user 6m22.755s 2024-06-02T23:50:03.224 INFO:teuthology.orchestra.run.smithi052.stderr:sys 0m14.704s 2024-06-02T23:53:36.005 INFO:teuthology.orchestra.run.smithi123.stderr: 94.6% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.e.log.gz 2024-06-02T23:54:43.720 INFO:teuthology.orchestra.run.smithi123.stderr: 94.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.h.log.gz 2024-06-02T23:56:37.926 INFO:teuthology.orchestra.run.smithi123.stderr: 95.5% -- replaced with /var/log/ceph/9d9fe36c-2133-11ef-bc9b-c7b262605968/ceph-mds.b.log.gz 2024-06-02T23:56:37.928 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-06-02T23:56:37.928 INFO:teuthology.orchestra.run.smithi123.stderr:real 14m32.574s 2024-06-02T23:56:37.928 INFO:teuthology.orchestra.run.smithi123.stderr:user 7m9.700s 2024-06-02T23:56:37.928 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m20.311s 2024-06-02T23:56:37.929 INFO:tasks.cephadm:Archiving logs... 2024-06-02T23:56:37.930 DEBUG:teuthology.misc:Transferring archived files from smithi052:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi052/log 2024-06-02T23:56:37.931 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-02T23:57:38.179 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi123/log 2024-06-02T23:57:38.180 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-02T23:58:36.100 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi174/log 2024-06-02T23:58:36.101 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-06-02T23:59:15.704 INFO:tasks.cephadm:Removing cluster... 2024-06-02T23:59:15.705 DEBUG:teuthology.orchestra.run.smithi052:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --force 2024-06-02T23:59:15.926 INFO:teuthology.orchestra.run.smithi052.stdout:Deleting cluster with fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:59:16.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --force 2024-06-02T23:59:16.613 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:59:17.214 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9d9fe36c-2133-11ef-bc9b-c7b262605968 --force 2024-06-02T23:59:17.419 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: 9d9fe36c-2133-11ef-bc9b-c7b262605968 2024-06-02T23:59:17.985 INFO:tasks.cephadm:Removing cephadm ... 2024-06-02T23:59:17.985 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-02T23:59:18.006 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-02T23:59:18.024 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-06-02T23:59:18.039 INFO:tasks.cephadm:Teardown complete 2024-06-02T23:59:18.040 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-06-02T23:59:18.052 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-06-02T23:59:18.052 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-02T23:59:18.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-02T23:59:18.066 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-06-02T23:59:18.185 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-06-02T23:59:18.196 INFO:teuthology.task.clock:Checking final clock skew... 2024-06-02T23:59:18.196 DEBUG:teuthology.orchestra.run.smithi052:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T23:59:18.199 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T23:59:18.201 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-06-02T23:59:18.214 INFO:teuthology.orchestra.run.smithi052.stderr:bash: line 1: ntpq: command not found 2024-06-02T23:59:18.214 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-06-02T23:59:18.215 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-06-02T23:59:18.235 INFO:teuthology.orchestra.run.smithi052.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T23:59:18.236 INFO:teuthology.orchestra.run.smithi052.stdout:=============================================================================== 2024-06-02T23:59:18.236 INFO:teuthology.orchestra.run.smithi052.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +178us[ +246us] +/- 39ms 2024-06-02T23:59:18.236 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -865us[ -865us] +/- 62ms 2024-06-02T23:59:18.236 INFO:teuthology.orchestra.run.smithi052.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +659us[ +659us] +/- 62ms 2024-06-02T23:59:18.236 INFO:teuthology.orchestra.run.smithi052.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T23:59:18.237 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T23:59:18.238 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-06-02T23:59:18.238 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 +188us[ +226us] +/- 39ms 2024-06-02T23:59:18.238 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 61 -859us[ -822us] +/- 61ms 2024-06-02T23:59:18.238 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +660us[ +697us] +/- 61ms 2024-06-02T23:59:18.238 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T23:59:18.241 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-06-02T23:59:18.241 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-06-02T23:59:18.241 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +182us[ +213us] +/- 39ms 2024-06-02T23:59:18.241 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 0 -894us[ -894us] +/- 62ms 2024-06-02T23:59:18.241 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 +632us[ +632us] +/- 62ms 2024-06-02T23:59:18.241 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-06-02T23:59:18.242 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-06-02T23:59:18.253 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-06-02T23:59:18.253 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-06-02T23:59:18.278 DEBUG:teuthology.orchestra.run.smithi052:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-02T23:59:18.438 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-02T23:59:18.547 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-06-02T23:59:18.639 DEBUG:teuthology.orchestra.run.smithi052:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T23:59:18.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:59:18.672 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T23:59:18.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:59:18.713 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-06-02T23:59:18.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:59:18.746 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-06-02T23:59:18.845 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-06-02T23:59:18.956 INFO:teuthology.task.internal:Duration was 4024.139775 seconds 2024-06-02T23:59:18.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-06-02T23:59:19.033 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-06-02T23:59:19.033 DEBUG:teuthology.orchestra.run.smithi052:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-02T23:59:19.039 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-02T23:59:19.047 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-06-02T23:59:19.100 INFO:teuthology.orchestra.run.smithi052.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T23:59:19.113 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T23:59:19.148 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-06-02T23:59:19.562 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-06-02T23:59:19.562 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi052.front.sepia.ceph.com 2024-06-02T23:59:19.562 DEBUG:teuthology.orchestra.run.smithi052:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-02T23:59:19.588 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-06-02T23:59:19.588 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-02T23:59:19.617 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-06-02T23:59:19.618 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-06-02T23:59:19.649 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-06-02T23:59:19.649 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-02T23:59:19.652 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-02T23:59:19.660 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-06-02T23:59:19.717 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-06-02T23:59:19.717 DEBUG:teuthology.orchestra.run.smithi052:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-02T23:59:19.725 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-02T23:59:19.732 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-06-02T23:59:21.846 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-06-02T23:59:21.861 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-06-02T23:59:21.861 DEBUG:teuthology.orchestra.run.smithi052:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-02T23:59:21.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-02T23:59:21.933 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-06-02T23:59:21.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-06-02T23:59:21.977 DEBUG:teuthology.orchestra.run.smithi052:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-02T23:59:21.980 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-02T23:59:21.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-06-02T23:59:22.002 INFO:teuthology.orchestra.run.smithi052.stdout:kernel.core_pattern = core 2024-06-02T23:59:22.018 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-06-02T23:59:22.047 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-06-02T23:59:22.095 DEBUG:teuthology.orchestra.run.smithi052:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-02T23:59:22.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:59:22.112 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-02T23:59:22.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:59:22.130 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-06-02T23:59:22.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-06-02T23:59:22.155 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-06-02T23:59:22.167 INFO:teuthology.task.internal:Transferring archived files... 2024-06-02T23:59:22.168 DEBUG:teuthology.misc:Transferring archived files from smithi052:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi052 2024-06-02T23:59:22.168 DEBUG:teuthology.orchestra.run.smithi052:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-02T23:59:22.258 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi123 2024-06-02T23:59:22.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-02T23:59:22.299 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-06-02_21:24:02-fs-squid-distro-default-smithi/7738637/remote/smithi174 2024-06-02T23:59:22.300 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-06-02T23:59:22.342 INFO:teuthology.task.internal:Removing archive directory... 2024-06-02T23:59:22.343 DEBUG:teuthology.orchestra.run.smithi052:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-02T23:59:22.345 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-02T23:59:22.348 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-06-02T23:59:22.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-06-02T23:59:22.408 INFO:teuthology.task.internal:Not uploading archives. 2024-06-02T23:59:22.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-06-02T23:59:22.436 INFO:teuthology.task.internal:Tidying up after the test... 2024-06-02T23:59:22.436 DEBUG:teuthology.orchestra.run.smithi052:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-02T23:59:22.438 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-02T23:59:22.441 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-06-02T23:59:22.451 INFO:teuthology.orchestra.run.smithi052.stdout: 265954 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 2 23:59 /home/ubuntu/cephtest 2024-06-02T23:59:22.455 INFO:teuthology.orchestra.run.smithi123.stdout: 266043 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 2 23:59 /home/ubuntu/cephtest 2024-06-02T23:59:22.457 INFO:teuthology.orchestra.run.smithi174.stdout: 265940 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jun 2 23:59 /home/ubuntu/cephtest 2024-06-02T23:59:22.458 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-06-02T23:59:22.512 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-06-02T23:59:22.564 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 4024.1397745609283 failure_reason: '"2024-06-02T23:15:24.161143+0000 mds.b (mds.0) 29 : cluster [WRN] Scrub error on inode 0x10000000208 (/volumes/qa/sv_0/8f46d269-a88a-4e0c-a87a-3cc369c34abc/client.0/tmp/clients/client0/~dmtmp/WORD) see mds.b log and `damage ls` output for details" in cluster log' flavor: default owner: scheduled_teuthology@teuthology success: false 2024-06-02T23:59:22.564 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-06-02T23:59:22.654 INFO:teuthology.run:FAIL